The challenges f Linearized Physics (LP) in Data Assimilatin (DA) Philippe Lpez, ECMWF with thanks t Marta aniskvá (ECMWF). Outline Brief reminder f what DA is abut. Physics in (4D-Var) DA: Why, where and hw? Cnstraints specific t LP. Benefits f LP. Summary and prspects. ECMWF Annual Seminar n Physical Prcesses, Reading, UK, 1-4 September 2015
The purpse and ingredients f Data Assimilatin * The purpse f DA is t merge infrmatin cming frm bservatins with a priri infrmatin cming frm a frecast mdel t btain an ptimal 3D representatin f the atmspheric state at a given time (= the analysis ). This 3D analysis then prvides initial cnditins t the numerical frecast mdel. * The main ingredients f DA are: a set f bservatins available ver a perid f typically a few hurs. a previus shrt-range frecast frm the NWP mdel ( backgrund infrmatin). sme statistical descriptin f the errrs f bth bservatins and mdel backgrund. a data assimilatin methd (e.g. variatinal DA: 3D-Var, 4D-Var; EnKF, ). This presentatin will fcus n the use f physical parameterizatins in 4D-Var DA.
mdel state analysis time t a 4D-Var All bservatins y are cnsidered at their actual time initial time t 0 y Mdel trajectry frm crrected initial state x a x b Mdel trajectry frm first guess x b 21 0 3 6 9 Time (UTC) 12-hur assimilatin windw 4D-Var prduces the analysis (x a ) which minimizes the distance t a set f available bservatins (y ) and t sme a priri backgrund infrmatin frm the mdel (x b ), given the respective errrs f bservatins and mdel backgrund.
4D-Var Incremental 4D-Var aims at minimizing the fllwing cst functin: δx 0 = 1 2 δx 0 T B δx 0 + 1 1 2 where: x 0 = x 0 x b 0 (increment; at lwer reslutin). B = backgrund errr cvariance matrix. d i = y i H(M i [x b 0]) (innvatin vectr; at high reslutin). M i = tangent-linear f frecast mdel (t 0 t i ). H = tangent-linear f bservatin peratr. i = time index (4D-Var windw is split in n intervals). = bservatin errr cvariance matrix. R i n i=1 (HM i δx 0 d i ) T (HM i δx 0 d i ) R i 1 δx 0 n 1 T T 1 B δx0 Mi [ ti,t0] H R i HM i δx0 i1 d i Adjint f frecast mdel with simplified linearized physics (simplified: t reduce cmputatinal cst and t avid nnlinear prcesses)
In ther wrds, physical parameterizatins are used in 4D-Var DA: * t describe the time evlutin f the mdel state ver the assimilatin windw as accurately as pssible. * t cnvert the mdel state variables (typ. T, wind, humidity, Psurf) int bserved equivalents s that the differences bsmdel can be cmputed (at the time f each bservatin). Side remark: One advantage f the incremental apprach is that the minimizatin f the cst functin can be run at lwer reslutin than the trajectry cmputatins (at ECMWF: 80 km versus 16 km).
The chice f physical parameterizatins will affect the 4D-Var results. Example: M: input = mdel state (T,q v ) utput = surface cnvective rainfall rate. acbians f cnvective surface rainfall rate w.r.t. input T and q v M T M q v M T M q v frm Marécal and Mahfuf (2002) Betts-Miller (adjustment scheme) Tiedtke (ECMWF s per mass-flux scheme)
Testing the tangent-linear cde The crrectness f the tangent-linear mdel must be assessed by checking that the first-rder Taylr apprximatin is valid: δx M x + λ δx M x lim λ 0 λ Mδx = 1 Example f utput frm a successful tangent-linear test: Tiny perturbatins Machine precisin reached Larger perturbatins Imprvement when perturbatin size decreases
Testing the adjint cde The crrectness f the adjint mdel needs t be assessed by checking that it satisfies the mathematical relatinship: δx, δy Mδx, δy = δx, M T δy where M is the tangent-linear mdel and M T is the adjint mdel. Example f utput frm a successful adjint test: <M x, y> = 0.13765102625164E-01 < x, M T y> = 0.13765102625168E-01 The difference is 11.351 times the zer f the machine The adjint test shuld be crrect at the level f machine precisin (typ. 13 t 15 digits fr the entire mdel). Otherwise there must be a bug in the cde!
Linearity assumptin Variatinal assimilatin is based n the strng assumptin that the analysis is perfrmed in a (quasi-)linear framewrk. Hwever, physical prcesses are ften nnlinear and their parameterizatins invlve discntinuities r nn-differentiable functins (e.g. switches r threshlds). The trickiest parameterizatins are cnvectin, large-scale clud prcesses and vertical diffusin. Regularizatin needs t be applied: smthing f functins, reductin f sme perturbatins. y riginal tangent in x 0 Precipitatin frmatin rate 0 Dy (nnlinear) new tangent in x 0 Dy (tangent-linear) x 0 Dx (finite size perturbatin) x Clud water amunt
An example f spurius TL nise caused by a threshld in the autcnversin frmulatin f the large-scale clud scheme. ~700 hpa znal wind increments [m/s] frm 12h mdel integratin. Nnlinear finite difference: M(x+x) M(x) Thursday 15 March 2001 12UTC ECMWF Frecast t+12 VT: Friday 16 March 2001 00UTC Mdel Level 44 **u-velcity 12 Tangent-linear integratin: Mx Thursday 15 March 2001 12UTC ECMWF Frecast t+12 VT: Friday 16 March 2001 00UTC Mdel Level 44 45 **u-velcity 12 8 4 2 1 0.5-0.5-1 -2-4 -8 8 4 2 1 0.5-0.5-1 -1-2 -2-4 -4-8 -8 frm M. aniskvá -12 with perturbatin reductin in autcnversin -12
Linearized physics package used in ECMWF s peratinal 4D-Var (1) Main simplificatins/regularizatins with respect t full nnlinear mdel are highlighted in red. Large-scale cndensatin scheme: [Tmpkins and aniskvá 2004] - based n a unifrm PDF t describe subgrid-scale fluctuatins f ttal water. - melting f snw included. - precipitatin evapratin included. - reductin f clud fractin perturbatin and in autcnversin f clud int rain. Cnvectin scheme: [Lpez and Mreau 2005] - mass-flux apprach [Tiedtke 1989]. - deep cnvectin (CAPE clsure) and shallw cnvectin (q-cnvergence) are treated. - perturbatins f all cnvective quantities are included. - cupling with clud scheme thrugh detrainment f liquid water frm updraught. - sme perturbatins (buyancy, initial updraught vertical velcity) are reduced. Radiatin: TL and AD f lngwave and shrtwave radiatin available [aniskvá et al. 2002] - shrtwave: based n Mrcrette (1991), nly 2 spectral intervals (instead f 6 in nnlinear versin). - lngwave: based n Mrcrette (1989), called every 2 hurs nly.
Linearized physics package used in ECMWF s peratinal 4D-Var (2) Vertical diffusin: [aniskvá and Lpez 2013] - mixing in the surface and planetary bundary layers. - based n K-thery and Blackadar mixing length. - exchange cefficients based n: Luis et al. [1982] in stable cnditins, Mnin-Obukhv in unstable cnditins. - mixed-layer parametrizatin and PBL tp entrainment. - Perturbatins f exchange cefficients are smthed (esp. near the surface). Surface scheme: [aniskvá 2014, pers. cmm.] - evlutin f sil, snw and sea-ice temperature. - perturbatin reductin in snw phase change frmulatin. Orgraphic gravity wave drag: [Mahfuf 1999] - subgrid-scale rgraphic effects [Ltt and Miller 1997]. - nly lw-level blcking part is used. Nn-rgraphic gravity wave drag: [Or et al. 2010] - istrpic spectrum f nn-rgraphic gravity waves [Scincca 2003]. - Perturbatins f utput wind tendencies belw 200 hpa reset t zer.
Impact f linearized physics n TL apprximatin (1) Znal mean crss-sectin f change in TL errr when TL includes: VDIF + rg. GWD + SURF Relative t adiabatic TL run (50-km resl.; 20 runs; after 12h integr.). Blue = TL errr reductin = Temperature
Impact f linearized physics n TL apprximatin (2) Znal mean crss-sectin f change in TL errr when TL includes: VDIF + rg. GWD + SURF + RAD Relative t adiabatic TL run (50-km resl.; 20 runs; after 12h integr.). Blue = TL errr reductin = Temperature
Impact f linearized physics n TL apprximatin (3) Znal mean crss-sectin f change in TL errr when TL includes: VDIF + rg. GWD + SURF + RAD + nn-rg GWD + mist physics Relative t adiabatic TL run (50-km resl.; 20 runs; after 12h integr.). Blue = TL errr reductin = Temperature
Impact f ECMWF linearized physics n frecast scres Cmparisn f tw T511 L91 4D-Var 3-mnth experiments with & withut full linearized physics: Relative change in frecast anmaly crrelatin. > 0 = aniskvá and Lpez (2013)
Influence f integratin length and reslutin n linearity assumptin Cmparisn f pairs f ppsite twin experiments using ECMWF s nnlinear mdel. Time evlutin f crrelatin between M(x+x)M(x) and M(xx)M(x). Inspired frm Walser et al. (2004). P 100 hpa z 10 m The linearity assumptin becmes less valid with integratin time and reslutin, especially clse t the surface and when physics is activated.
Summary and prspects (1) Linearized physical parameterizatins have becme essential cmpnents f variatinal data assimilatin systems: Better representatin f the evlutin f the atmspheric state during the minimizatin f the cst functin (via the adjint mdel integratin). Extractin f infrmatin frm bservatins that are strngly affected by physical prcesses (e.g. by cluds r precipitatin). Hwever, there are sme limitatins t the LP apprach: 1) Theretical: The dmain f validity f the linear hypthesis shrinks with increasing reslutin and integratin length. 2) Technical: Linearized mdels require sustained & time-cnsuming attentin: Testing tangent-linear apprximatin and adjint cde. Regularizatins / simplificatins t eliminate any surce f instability. Revisins t ensure gd match with reference nn-linear frecast mdel.
Summary and prspects (2) In practice, it all cmes dwn t achieving the best cmprmise between: Realism Cst Linearity Alternative data assimilatin methds exist that d nt require the develpment f linearized cde, but s far nne f them has been able t utperfrm 4D-Var, especially in glbal mdels: Ensemble Kalman Filter (EnKF; still relies n the linearity assumptin), Particle filters (difficult t implement fr high-dimensinal prblems). S what is the future f LP?
Frm a small challenge
t a much bigger challenge
Summary and prspects (3) Eventually, it might becme impractical r even impssible t make LP wrk efficiently at reslutins f a few kilmetres, even if the linearity cnstraint can be relaxed (e.g. by using shrter 4D-Var windw r weak-cnstraint 4D- Var).?
Simple example f tangent-linear and adjint cdes. Simplified nnlinear cde: Z = a / X 2 + b Y lg(w) Tangent-linear cde: Z = (2 a / X 3 ) X + b lg(w) Y + (b Y / W) W Adjint cde: X* = 0 Y* = 0 W* = 0 X* = X* (2 a / X 3 ) Z* Y* = Y* + b lg(w) Z* W = W* + (b Y / W) Z* Z* = 0
Example f nnlinear, tangent-linear and adjint peratrs in the cntext f 4D-Var. Nnlinear peratrs are applied t full fields: mdel initial state x 0 T qv u v P s T q u v P q q s liq ice time t i bservatin equivalent time t 0 = satellite cludy radiances v time t i M H ~ y i Rad Rad Rad ch1 ch2 ch3 M = frecast mdel with physics H = radiative transfer mdel
Tangent-linear peratrs are applied t perturbatins: 3 2 1 0 0 ~ ], [ ch ch ch i ice liq s v i s v Rad Rad Rad q q P v u q T P v u q T t t δy δx H M time t i time t 0 Adjint peratrs are applied t the cst functin gradient: s v i T ice liq s v T ch ch ch P v u q T t t q q P v u q T Rad Rad Rad i / / / / / ], [ / / / / / / / / / / 0 0 3 2 1 ~ x y M H time t i time t 0
Illustratin f discntinuity effect n cst functin shape: Mdel backgrund = {T b, q b }; Observatin = RR bs Simple parametrizatin f rain rate: RR = {q q sat (T)} if q > q sat (T), 0 therwise 1 2 T T T b 2 1 2 q q q b 2 1 2 [ q qsat( T)] RR RR bs bs 2 q Saturated backgrund q Dry backgrund {T b, q b } min OK N cnvergence! Single minimum f cst functin T Several lcal minima f cst functin T
M Frecasts Climate runs N OK? Timing: Yes Pure cding N M M(x+x)M(x) M x M* <M x,y> = <x,m*y> OK? Yes N OK? Yes Debugging and testing (incl. regularizatin) N N OK? OK? Singular Vectrs (EPS) 4D-Var (minim) NL TL AD APPL Stages in the develpment f a new linearized parameterizatin
Tangent-linear apprximatin and assciated errr The tangent-linear apprximatin is assessed by cmparing the difference between tw integratins f the full nn-linear mdel: M(xx) M(x) with an integratin f the tangent-linear mdel frm the same initial perturbatin (x = x ana x bg ): Mx The TL errr is then defined as: = < M(xx) M(x) Mx > where < > dentes spatial averaging (e.g. znally r glbally).
TOA Impact f linearized physics n TL apprximatin (4) Mean vertical prfile f change in TL errr fr T, U, V and Q when full linearized physics is included in TL cmputatins. Relative t adiabatic TL run (50-km resl.; twenty runs, 12h integ.) < 0 = surface Inclusin f linearized physics leads t better TL apprximatin.
Effects f mdel nnlinearities versus effects f reslutin Cmparisn f pairs f ppsite twin experiments using ECMWF s nnlinear mdel. Time evlutin f crrelatin between M(x+x)M(x) and M(xx)M(x). Inspired frm Walser et al. (2004). P 100 hpa z 10 m The effects f nnlinearities are f cmparable magnitude with thse f reslutin, except near the surface when physics is activated (the frmer effects then dminate).