Data Assimilation 1 Alan O Nill National Cntr for Earth Obsrvation UK
Plan Motivation & basic idas Univariat (scalar) data assimilation Multivariat (vctor) data assimilation 3d-Variational Mthod (& optimal intrpolation) Kalman Filtr (+ xtndd KF) Ensmbl mthods ( + particl filtr) 4d-Variational Mthod Applications of data assimilation in arth systm scinc
What is data assimilation? Data assimilation is th tchniqu whrby obsrvational data ar combind with output from a numrical modl to produc an optimal stimat of th volving stat of th systm
What ar th bnfits of data assimilation? Quality control Combination of data Errors in data and in modl Filling in data poor rgions Dsigning obsrving systms Maintaining consistncy Estimating unobsrvd quantitis Paramtr stimation in modls *****
Th Data Assimilation Problm How can w combin noisy masurmnts of a systm with output from an imprfct numrical modl to gt th bst stimat of th (volving) stat of th systm?
Answr: Us Bays Thorm with th following information: Th obsrvations Thir rrors Prdictions by a numrical modl of th systm Th rrors in ths prdictions Th ky ida is to combin obsrvations with prdictions giving mor wight to information with th last rror But rrors may not b wll known! Intrnal consistncy chcks on our stat stimats ar possibl, but also nd indpndnt (unassimilatd data)
Conditional Probability & Bays Thorm p(a,b) = p(a B)p(B) = p(b A)p(A), whr A and B ar two random vnts Bays' Thorm: p(a B) = p(b A) p(a) p(b) \ p(x y) = p(y x)p(x), p(y) whr x is a stat variabl of th systm w wish to stimat, and z is a masurmnt of that variabl So if w hav som prior information about p(x),w can updat that information with an obsrvation y to gt p(x y), th probability that th systm variabl has valu x givn that a masurmnt z of that variabl has bn mad W call it th postrior pdf
Errrors vrywhr All significant sourcs of uncrtainty should b accountd for in data assimilation Random rrors: background (a-priori) rrors obsrvation rrors modl rrors rprsntivity rrors Systmatic rrors: biass in background biass in obsrvations biass in modl Exampl 1 rpatd obsrvations of air tmpratur y (T obsrvations) biasd thrmomtr truth truth unbiasd thrmomtr Exampl 2 rprsntivity rrors du to modl grid
Th stat stimation problm ˆ x k f x k +1 = M( ˆ x k ), nonlinar modl pdf volvd by nonlinar modl (Fokkr-Planck quation) Updat with obsrvations at tim k + 1 using Bays s thorm Too xpnsiv to volv th full pdf Estimat th volution of th low ordr momnts: typically man and varianc This is quivalnt to assuming Gaussian statistics NB Th varianc (masur of modl rror) is oftn assumd to volv according to a linarizd vrsion of th modl This may b a srious limitation for data assimilation
Stat Estimation A formula (or algorithm) to stimat th valu of a stat variabl x is calld an stimator (Not: th stimator is a random variabl, bcaus it is xprssd in trms of random variabls, such as y) W oftn driv our stimator by constructing a COST FUNCTION, J, which masurs th fit of our stat variabl(s) x to th data Thn w minimiz this cost function to obtain th optimal x For typically usd cost functions, our stimator is: x ˆ = E[x y] = th man of x givn y xp(x y)dx For Gaussian statistics, w gt our stimator x ˆ as th x that maximizs p(x y) (th mod) or which minimizs J=-ln p(x y)
A Simpl Exampl Assum w hav an obsrvation x o of an unknown variabl x Assum w hav som prior information that th valu of x is x b Assum w know th rror statistics of ths quantitis (th rror variancs) p(x x o ) ~ p(x o x)p(x) = xp - (x - o x)2 xp - (x - x b )2 2 2 2s o 2s b J(x) = -ln p(x x o ) ~ (x - o x)2 + (x - x b )2 2 2 2s o 2s b Find x such that J(x) is a minimum This x is our stimat of x Call it ˆ x x ˆ ~ x o s + x b 2 o s 2 b Easy to show from form of p(x x 0 ) that s 1ˆ = 1 2 s + 1 2 2 o s b Th biggr th varianc, th lss wight is givn to th information Th prcision of th stimat is bttr than thos of th obsrvation or background (To gt an quals sign in th abov, divid by th sum of th wights )
Th Obsrvation Oprator Th obsrvations (obsrvation vctor) ar in gnral not dirct masurmnts of th stat variabls (stat vctor), g in rmot snsing from spac In data assimilation, w nd to compar th obsrvation vctor with th stat vctor Th obsrvation oprator allows this It is a mapping from stat spac to obsrvation spac y mod = h(x) x x x x x x R i = B i (T( p)) dt dp Data assimilation algorithms oftn us th matrix valuatd gnrally at a stat forcast by th modl (background stat or first-guss stat) H = h x x =x B
Thr typs of stimation problm (stimat dsird at tim t) span of availabl obsrvations filtring (g Kalman filtr) t smoothing (g variational DA) t prdiction t
Squntial Data Assimilation start of th systm x f (t 1 ) x a (t 1 ) x f (t 2 ) x a (t 2 ) = obsrvation x tru (t) (unknown) tim x f (t 3 ) This is an xampl of a filtr Data assimilation has: prdiction stags (x f = forcast, prior, background ) analysis stags (x a ) (xtrapolation) (intrpolation)
DATA ASSIMILATION SYSTEM Data Cach Error Statistics modl obsrvations A O DAS A Numrical Modl F B
3d-Variational Data Assimilation
Multivariat Cas x 1 stat vctor x( t) = x 2 y 1 Ł x n ł obsrvation vctor y( ) = y t 2 Ł y m ł
Errors
Th Error Covarianc Matrix ( ) Ł > ł < > < > < > < > < > < > < > < > < >= =< = ł Ł = n n n n n n n n 2 1 2 2 2 1 2 1 2 1 1 1 T 2 1 T 2 1 P 2 i i i s >= <
Background Errors Thy ar th stimation rrors of th background stat (a modl forcast): b = x b - x avrag (bias) covarianc < b > B =< ( b - < b >)( b - < b >) T >
Background rror in obsrvation spac y mod = Hx b If whr is a matrix, thn th rror covarianc for H y mod is givn by: C y mod = HBH T
Obsrvation Errors Thy contain rrors in th obsrvation procss (instrumntal rror), rrors in th dsign of H, and rprsntativnss rrors, i discrtizaton rrors that prvnt x from bing a prfct rprsntation of th tru stat o = y - H(x ) R =< ( - < > )( - < > ) o o o o T >
Rmarks on 3d-VAR Can add constraints to th cost function, g to hlp maintain balanc Can work with non-linar obsrvation oprator H Can assimilat radiancs dirctly (simplr obsrvational rrors) Can prform global analysis instad of OI approach of radius of influnc
Optimal Intrpolation (th BLUE) BLUE = Bst linar unbiasd stimat Algorithm drivd as a spcial cas of 3D-var
BLUE Estimator (rcursiv) Th BLUE stimator or analysis is givn by: x a = x b + K(y - h(x b )) K = BH T (HBH T + R) -1 B Th matrix plays a ky rol in dtrmining th structur of th analysd filds Matrix invrss xpnsiv to comput so rduc dimnsion by local analysis W can driv an xplicit xprssion for th analysis rror covarianc matrix: A = ( I - KH) B
Assumptions Usd in BLUE Linarizd obsrvation oprator: h(x) - h(x b ) = H(x - x b ) Errors ar unbiasd: < x b - x >=< y - h(x) >= 0 Errors ar uncorrlatd: < (x b - x)(y - h(x)) T >= 0
Innovations and Rsiduals Ky to data assimilation is th us of diffrncs btwn obsrvations and th stat vctor of th systm W call W call y - h(x b ) y - h(x a ) th innovation th analysis rsidual Giv important information
Ozon at 10hPa, 12Z 23rd Spt 2002 Analysis MIPAS obsrvations 6 day modl forcast
3D variational data assimilation - ozon at 10hPa x b
3D variational data assimilation - ozon at 10hPa x y - h x ) b ( b
3D variational data assimilation - ozon at 10hPa x y - h x ) b ( b K( y - h( xb))
Th data assimilation cycl: ozon at 10hPa x y - h x ) b ( b xb + K( y - h( xb)) K( y - h( xb))
Choic of Stat Variabls and Prconditioning Fr to choos which variabls to us to dfin stat vctor, x(t) W d lik to mak B diagonal may not know covariancs vry wll want to mak th minimization of J mor fficint by prconditioning : transforming variabls to mak surfacs of constant J narly sphrical in stat spac
Cost Function for Corrlatd Errors x 2 x 1
Cost Function for x 2 Uncorrlatd Errors x 1
Cost Function for Uncorrlatd Errors x 2 Scald Variabls x 1
END