Aspects of the Ensemble Kalman Filter. David Livings

Transcription

1 Aspecs of he Ensemble Kalman Filer David Livings Ocober 3, 25

2 Absrac The Ensemble Kalman Filer (EnKF) is a daa assimilaion mehod designed o provide esimaes of he sae of a sysem by blending informaion from a model of he sysem wih observaions. I mainains an ensemble of sae esimaes from which a single bes sae esimae and an assessmen of esimaion error may be calculaed. Compared o more esablished mehods i offers advanages of reduced compuaional cos, beer handling of nonlineariy, and greaer ease of implemenaion. This disseraion sars by reviewing differen formulaions of he EnKF, covering sochasic and semi-deerminisic varians. Two formulaions are seleced for implemenaion, and he adapaion of heir algorihms for beer numerical behaviour is described. Nex, as a subjec for experimens, a simple mechanical sysem is described ha is of ineres o meeorologiss as an illusraion of he problem of iniialisaion. Experimenal resuls are presened ha show some unexpeced feaures of he implemened filers, including ensemble saisics ha are inconsisen wih he acual error. Explanaions of hese feaures are provided and poin o a poenial flaw in he general framework for semi-deerminisic formulaions of he EnKF, affecing some bu no all such formulaions. This flaw appears o have been overlooked in he lieraure.

3 Conens Lis of Symbols 7 Lis of Abbreviaions 1 1 Inroducion Background Goals Principal Resuls Ouline Formulaions of he Ensemble Kalman Filer Sequenial Daa Assimilaion The Kalman and Exended Kalman Filers The Kalman Filer The Exended Kalman Filer The Ensemble Kalman Filer Noaion The Forecas Sep The Analysis Sep Nonlinear Observaion Operaors Deerminisic Formulaions of he Analysis Sep The General Framework The Direc Mehod The Serial Mehod The Ensemble Transform Kalman Filer

4 2.4.5 The Ensemble Adjusmen Kalman Filer Summary Implemening an Ensemble Kalman Filer Implemening he ETKF Implemening he EAKF The Swinging Spring and Iniialisaion Iniialisaion The Swinging Spring Numerical Mehod of Inegraion Normal Mode Iniialisaion Experimenal Resuls Experimens wih he ETKF Experimens wih he EAKF Summary Explanaion of Experimenal Resuls Inconsisen Analysis Ensemble Saisics Collapse in Number of Disinc Ensemble Members Conclusions Summary and Discussion Furher Work Numerical Inegraion of he Swinging Spring Equaions Iniialisaion and he Ensemble Kalman Filer Inconsisen Analysis Ensemble Saisics A Addiional Operaion Couns 87 A.1 Analysis Sep of KF A.2 Naive Implemenaion of Analysis Sep of Sochasic EnKF.. 87 B The EAKF and he General Deerminisic Framework 89 C Graphs of EAKF Experimens 91 2

5 D Example of an Invalid X a from he ETKF 1 Bibliography 12 Acknowledgemens 15 3

6 Lis of Figures 4.1 Coordinaes and forces for he swinging spring Coordinaes and heir Fourier ransforms for an uniniialised swinging spring Sabiliy funcion for ode45 and he simple harmonic oscillaor y = iy Coordinaes and heir Fourier ransforms for a swinging spring wih linear normal mode iniialisaion Coordinaes and heir Fourier ransforms for a swinging spring wih nonlinear normal mode iniialisaion ETKF, perfec observaions, N = 1: ensemble members ETKF, perfec observaions, N = 1: ensemble saisics ETKF, perfec observaions, N = 5: ensemble members ETKF, perfec observaions, N = 5: ensemble saisics ETKF, imperfec observaions, N = 1: ensemble members ETKF, imperfec observaions, N = 1: ensemble saisics ETKF, imperfec observaions, N = 5: ensemble members ETKF, imperfec observaions, N = 5: ensemble saisics.. 71 C.1 EAKF, perfec observaions, N = 1: ensemble members C.2 EAKF, perfec observaions, N = 1: ensemble saisics C.3 EAKF, perfec observaions, N = 5: ensemble members C.4 EAKF, perfec observaions, N = 5: ensemble saisics C.5 EAKF, imperfec observaions, N = 1: ensemble members.. 96 C.6 EAKF, imperfec observaions, N = 1: ensemble saisics.. 97 C.7 EAKF, imperfec observaions, N = 5: ensemble members

7 C.8 EAKF, imperfec observaions, N = 5: ensemble saisics

8 Lis of Tables 5.1 Observaion error sandard deviaions passed o he filer in experimens wih perfec observaions Fracion of analyses wih ensemble mean wihin one ensemble sandard deviaion of ruh Sandard deviaions used o generae he iniial ensemble in experimens wih imperfec observaions

9 Lis of Symbols Lain A a (superscrip) e (subscrip) F f f (superscrip) G g H H I i (subscrip) J K k l l M M Ensemble adjusmen marix in EAKF. Analysis value. Ensemble value. Orhogonal marix. Frequency. Forecas value. Diagonal marix of square roos of eigenvalues. Acceleraion due o graviy. Nonlinear observaion operaor; Hamilonian. Linear observaion operaor; angen linear operaor of nonlinear observaion operaor. Ideniy marix. Ensemble member. Cos funcion. Kalman gain marix. Elasiciy of swinging spring. Equilibrium lengh of swinging spring. Unsreched lengh of swinging spring. Nonlinear dynamical model. Linear dynamical model; angen linear operaor of nonlinear dynamical model. 7

10 m Dimension of observaion space; mass of bob of swinging spring. N Ensemble size. n Dimension of sae space. P Sae error covariance marix. p Number of nonzero singular values in SVD; generalised momenum. Q Model noise covariance marix. R Observaion noise covariance marix. r Radial polar coordinae. r S T Radial displacemen of swinging spring relaive o equilibrium lengh. Ensemble version of innovaions covariance marix. Period of oscillaion. T Pos-muliplier in deerminisic formulaions of EnKF. (superscrip) True value; ime. k Discree imes a which observaions are available and forecass and analyses are required. U, V, W Orhogonal marices. X Ensemble marix. x Sae vecor. Y Observaion ensemble marix. y Observaion vecor. Z Ad hoc marix. z Ad hoc vecor. Greek β ε ε η Coefficien in serial mehod formulaion of EnKF. Raio of frequencies of swinging spring. Observaion noise vecor. Model noise vecor. 8

11 θ Angular polar coordinae. Λ Diagonal marix of eigenvalues. Σ Marix of singular values in SVD. ω Angular frequency. Miscellaneous Some of he vecor operaors below may be applied o scalars and marices as well. 1 N N N marix in which every elemen is 1/N. z Populaion mean of z. ẑ Version of z in augmened sae space; z scaled by inverse square roo of covariance marix; Fourier ransform of z. z Ensemble mean of z. Z Perurbaion marix of ensemble marix Z. z Analogue in EAKF of z or ẑ in ETKF. 9

12 Lis of Abbreviaions EAKF EKF EnKF ETKF KF NWP SVD Ensemble Adjusmen Kalman Filer Exended Kalman Filer Ensemble Kalman Filer Ensemble Transform Kalman Filer Kalman Filer Numerical Weaher Predicion Singular Value Decomposiion 1

13 Chaper 1 Inroducion 1.1 Background Daa assimilaion addresses he problem of incorporaing observaions ino a model of some sysem. For example, he sysem could be he amosphere of he Earh, he model could be a weaher forecasing model, and he observaions could be measuremens made by surface saions, radiosondes, weaher radars, and saellies. In his case he problem is o combine he sae of he amosphere as prediced by an earlier forecas wih recen observaional daa o produce an updaed esimae of he sae of he amosphere (known as he analysis) ha can be used as he saring poin for a new forecas. For a deailed overview of daa assimilaion in a meeorological conex see Kalnay [15] or Swinbank e al [22]. The daa assimilaion echniques of 3D-Var and 4D-Var are currenly popular a naional meeorology cenres. These are variaional echniques ha use numerical mehods o minimise a cos funcion ha is a weighed measure of he disances from he analysis o he forecas and he observaions. The weighings in he cos funcion are inended o reflec he relaive uncerainies in differen componens of he forecas and observaions. The resuling analysis hus represens a combinaion of he win informaion sources of forecas and observaion, wih greaer weigh being given o more cerain informaion. The difference beween 3D-Var and 4D-Var is ha 3D-Var reas 11

14 all observaions as having occurred a he forecas ime, whils 4D-Var akes some accoun of he evoluion of he sysem beween he forecas ime and he observaion ime. A deailed comparison of 4D-Var and he Ensemble Kalman Filer mehods ha are he subjec of his disseraion may be found in Lorenc [18]. A major par of he effor in implemening 3D-Var or 4D-Var lies in modelling he forecas uncerainy for use in he cos funcion. This uncerainy is usually modelled he same for all ime, which is less han ideal because he forecas uncerainy will vary depending on boh he qualiy of he observaions conribuing o he analysis on which i is based, and he way in which he sysem evolves beween he analysis and he forecas. An evolving forecas uncerainy is provided by he Exended Kalman Filer (EKF), which is a nonlinear generalisaion of he linear Kalman Filer (KF). As well as an esimae of he sae of he sysem, hese filers mainain an error covariance marix ha acs as a measure of he uncerainy in he esimae. This covariance marix is updaed wih every analysis o reflec he new informaion provided by he observaions, and is evolved along wih he sae esimae from he ime of an analysis o he ime of he nex forecas. The KF and EKF are described in more deail laer in his disseraion; for a comprehensive reamen see Gelb [9] or Jazwinski [14]. The EKF rose o prominence in aerospace applicaions where he dimension of he sae space for he model is relaively small, ypically nine or less. Direcly exending he filer o he sor of sysems encounered in numerical weaher predicion (NWP), where he sae space dimension may be 1 7, is beyond he capabiliies of curren compuer echnology. The EKF also shares wih 4D-Var he need o implemen angen linear operaors (Jacobians) for he nonlinear forecas model and he model of how observaions are relaed o he sae of he sysem. Doing his for a large, complicaed sysem such as an NWP model is labour-inensive. The Ensemble Kalman Filer (EnKF) is an aemp o overcome he drawbacks of he EKF. Two of is key ideas are o use an ensemble (saisical sample) of sae esimaes insead of a single sae esimae and o calculae he error covariance marix from his ensemble insead of mainaining a separae covariance marix. If we ake an ensemble size ha is small bu no 12

15 so small ha i is saisically unrepresenaive, hen he exra work needed o mainain an ensemble of sae esimaes is more ha offse by he work saved hrough no mainaining a separae covariance marix. The EnKF also does no use angen linear operaors, which eases implemenaion and may lead o a beer handling of nonlineariy. The EnKF was originally presened in Evensen [7]. An imporan subsequen developmen was he recogniion by Burgers e al [4] (and independenly by Houekamer and Michell [11]) of he need o use an ensemble of pseudo-random observaion perurbaions o obain he righ saisics from he analysis ensemble. Deerminisic mehods for forming an analysis ensemble wih he righ saisics have also been presened. The former approach o he EnKF is comprehensively reviewed in Evensen [8], whils varians of he laer approach are placed in a uniform framework by Tippe e al [23]. These varians include he Ensemble Transform Kalman Filer (ETKF) of Bishop e al [3] and he Ensemble Adjusmen Kalman Filer (EAKF) of Anderson [2]. 1.2 Goals The goals of his disseraion are: o review he principal formulaions of he EnKF; o selec one or more formulaions for implemenaion and o implemen hem; o perform experimens wih he implemened filers using a simple mechanical sysem (see below) as a es case and searching for ineresing phenomena; and o inerpre he experimenal resuls and draw any imporan conclusions. The mechanical sysem o be used as a es case in he experimens is he wo-dimensional swinging spring. This simple sysem is of ineres o meeorologiss because i possesses ineracing moions wih wo disinc imescales, analogous o he Rossby and graviy waves of he amosphere. I may be used as an illusraion of he problem of iniialisaion (see Chaper 4). 13

16 1.3 Principal Resuls The ETKF and EAKF are seleced for implemenaion in Chaper 3. I is found o be advanageous o reformulae he raw algorihms reviewed in Chaper 2 o give algorihms ha are analyically equivalen bu numerically beer behaved. Experimens wih he ETKF and he swinging spring in Chaper 5 show a collapse in he number of disinc ensemble members afer assimilaing each observaion. This collapse is explained in Secion 6.2 and poins o a limied usefulness of he ETKF for low-dimensional sysems such as he swinging spring, alhough he high-dimensional sysems ypical of NWP are unaffeced. The mos imporan resul of he disseraion is ha here is a poenial flaw in he general framework for semi-deerminisic formulaions of he EnKF as presened in Tippe e al [23]. This flaw may lead some formulaions of he EnKF o produce analysis ensembles wih saisics ha are inconsisen wih he acual error, being boh biased (mean ending o be in he wrong place) and overconfiden (sandard deviaions of coordinaes oo small). This is demonsraed experimenally in Chaper 5 and explained heoreically in Secion 6.1. The ETKF is affeced by his flaw (a leas in some circumsances), bu he EAKF is no. 1.4 Ouline Several alernaive formulaions of he EnKF have been published since he original paper of Evensen [7]. The principal alernaives are reviewed in Chaper 2. Formulaions may be classified as sochasic or semi-deerminisic depending on he degree o which hey rely upon pseudo-random numbers o represen uncerainy in he sysem model and he observaions. The ETKF and EAKF are seleced for implemenaion in Chaper 3. This chaper also describes he reformulaion of he raw algorihms of Chaper 2 o give algorihms ha are analyically equivalen bu numerically beer behaved. 14

17 In Chaper 4, as a subjec for experimens, he wo-dimensional swinging spring is inroduced. This simple mechanical sysem is of ineres o meeorologiss as an illusraion of he problem of iniialisaion. The chaper discusses he concep of iniialisaion and is imporance for NWP. Chaper 5 presens he resuls of experimens using he filer implemenaions described in Chaper 3 and he swinging spring sysem of Chaper 4. The experimens reveal some unexpeced feaures in he ETKF, including ensemble saisics ha are inconsisen wih he acual error. Chaper 6 provides explanaions of he feaures observed in Chaper 5. The explanaion of he inconsisen saisics poins o a poenial flaw in he general framework for semi-deerminisic formulaions of he EnKF, affecing some bu no all such formulaions. The ETKF is affeced (a leas in some circumsances), bu he EAKF is no. This flaw appears o have been overlooked in he lieraure. The disseraion concludes in Chaper 7 wih a review of he preceding chapers and some suggesions for furher work. 15

18 Chaper 2 Formulaions of he Ensemble Kalman Filer This chaper presens a unified exposiion of various formulaions of he Ensemble Kalman Filer. I encompasses he sochasic formulaion reviewed in Evensen [8] and he semi-deerminisic formulaions reviewed in Tippe e al [23]. We sar by looking a he problem in daa assimilaion ha he filer is designed o solve and he sandard daa assimilaion echniques of he Kalman and Exended Kalman Filers. 2.1 Sequenial Daa Assimilaion Daa assimilaion seeks o solve he following problem: given a noisy discree model of he dynamics of a sysem and noisy observaions of he sysem, find esimaes of he sae of he sysem. Sequenial daa assimilaion echniques such as he Kalman Filer and is varians break his problem ino a cycle of alernaing forecas and analysis seps. In he forecas sep he sysem dynamical model is used o evolve an earlier sae esimae forward in ime, giving a forecas sae a he ime of he laes observaions. In he analysis sep he observaions are used o updae he forecas sae, giving an improved sae esimae called he analysis. This analysis is used as he saring poin for he nex forecas. 16

19 We shall denoe he dimension of he sae space of he sysem by n and vecors in his space by x, usually wih various argumens, subscrips, and superscrips. In paricular he rue sae of he sysem a ime k will be denoed by x ( k ) and he forecas and analysis a his ime by x f ( k ) and x a ( k ) respecively. We shall denoe he dimension of observaion space by m and he observaion vecor a ime k by y( k ). The argumen k will be dropped from sae and observaion vecors when all quaniies being discussed occur a he same ime. We shall be using random variables o model errors in he sysem dynamical model and in he observaions. These errors lead o random errors in he forecass and analyses. We seek forecass and analyses ha are unbiased in he sense ha x f x = x a x = where angle brackes denoe he expecaion operaor. Informaion abou he size and correlaion of he error componens is given by he error covariance marices P f = (x f x )(x f x ) T P a = (x a x )(x a x ) T. We seek mehods ha calculae covariance marices P f and P a as well as sae esimaes x f and x a. 2.2 The Kalman and Exended Kalman Filers We now briefly review wo esablished daa assimilaion echniques. For a deailed reamen see Gelb [9] or he mahemaically more sophisicaed Jazwinski [14]. The Kalman and Exended Kalman Filers originally rose 17

20 o prominence in aerospace applicaions and are popular for uses such as racking airborne objecs wih radar. For hese applicaions he sae space dimension is relaively small, ypically nine or less. As we shall see, here are problems exending he echniques o he much larger sae space dimensions frequenly encounered in geoscience applicaions The Kalman Filer The Kalman Filer (KF) is a sequenial daa assimilaion echnique for use wih linear models of he sysem dynamics and observaions. I assumes ha he sysem dynamics can be modelled by x ( k ) = Mx ( k 1 ) + η( k 1 ) (2.1) where M is a known marix and η( k 1 ) is random model noise wih known covariance marix Q. I is assumed ha η( k 1 ) is unbiased and ha is values a differen imes are uncorrelaed. The observaions are assumed o be modelled by y( k ) = Hx ( k ) + ε( k ) (2.2) where H is a known marix and ε( k ) is random observaion noise wih known covariance marix R. I is assumed ha ε( k ) is unbiased and ha is values a differen imes are uncorrelaed. I is also assumed ha here is no correlaion beween model noise and observaion noise a any imes (he same or differen). I is no difficul o exend hese models o he case where M, Q, H, and R vary wih ime, bu for noaional simpliciy his case is no explicily considered here. In addiion o he models of he sysem dynamics and he observaions, an iniial sae esimae x a ( ) is required and so is is error covariance marix P a ( ). I is assumed ha he error in his esimae is unbiased and uncorrelaed wih model noise and observaion noise a any ime. If observaions are available a ime, he iniial esimae may be reaed as a forecas raher han an analysis. The forecas sep of he Kalman Filer evolves he analysis and analysis 18

21 error covariance marix a ime k 1 forward o ime k using he equaions x f ( k ) = Mx a ( k 1 ) (2.3) P f ( k ) = MP a ( k 1 )M T + Q. (2.4) The analysis sep a ime k sars by calculaing he Kalman gain marix K( k ) = P f ( k )H T (HP f ( k )H T + R) 1. (2.5) The observaion y( k ) is hen assimilaed using x a ( k ) = x f ( k ) + K( k )(y( k ) Hx f ( k )) (2.6) P a ( k ) = (I K( k )H)P f ( k ). (2.7) Good poins abou he Kalman Filer include ha he sae updae equaions (2.3) and (2.6) preserve unbiasedness; ha he error covariance updae equaions (2.4) and (2.7) are exac; and ha he filer is opimal in he sense ha he analysis defined by (2.6) minimises he cos funcion J (x) = (x x f ) T (P f ) 1 (x x f ) + (y Hx) T R 1 (y Hx). (2.8) This funcion is a weighed measure of he disances from he sae x o he forecas x f and he observaion y. Thus he analysis represens a combinaion of he win informaion sources of forecas and observaion, wih greaer weigh being given o he more cerain componens. Bad poins abou he Kalman Filer are ha i only works for linear sysems and ha in applicaions such as numerical weaher predicion (NWP) he covariance marices are huge. A naional meeorological service may use a forecasing model wih 1 7 sae variables and currenly have o assimilae observaions per assimilaion period. This gives sae error covariance marices of size and poenially requiring hundreds of erabyes of sorage. Also, he calculaion of he Kalman gain (2.5) involves he inversion of a marix of size or larger. In he near fuure wih more saellie daa being assimilaed he number of observaions is expeced 19

22 o become comparable wih he number of sae variables, which will make maers worse The Exended Kalman Filer The Exended Kalman Filer (EKF) is an aemp o exend he Kalman Filer o nonlinear dynamical sysems and nonlinear observaions. The linear models of dynamics (2.1) and observaions (2.2) are replaced by he nonlinear varians x ( k ) = M(x ( k 1 )) + η( k 1 ) (2.9) y( k ) = H(x ( k )) + ε( k ) (2.1) where he marices M and H have been replaced by poenially nonlinear funcions M and H. Noe he convenion of using bold uprigh ype for linear operaors and sandard ialic ype for corresponding nonlinear operaors. Again, i is no difficul o exend hese models o he case where M and H vary wih ime. The nonlinear funcions are used in he sae updae equaions of he forecas and analysis seps: x f ( k ) = M(x a ( k 1 )) x a ( k ) = x f ( k ) + K( k )(y( k ) H(x f k )). The error covariance updae equaions (2.4) and (2.7) and he calculaion of he Kalman gain (2.5) remain he same wih he proviso ha M and H are now he angen linear operaors (Jacobians) of M and H evaluaed a x a ( k 1 ) and x f ( k ) respecively. The EKF can work well, especially if he sysem is only weakly nonlinear. However, i relies on linear approximaions o nonlinear funcions and does no sricly possess several of he good feaures of he Kalman Filer. In paricular he sae updae equaions can no longer be guaraneed o preserve unbiasedness; he error covariance updae equaions are no longer exac; and he filer is no longer opimal in he sense of minimising he cos funcion 2

23 (2.8) (wih Hx replaced by H(x)). The EKF also does nohing o address he problem of huge covariance marices. In addiion i is labour-inensive o implemen because of he need o derive and implemen angen linear models of he dynamics and observaions. 2.3 The Ensemble Kalman Filer The EKF represens nonlineariy using derivaives ha only ake ino accoun behaviour in an infiniesimal neighbourhood of a poin. The Ensemble Kalman Filer (EnKF) is an aemp o represen nonlineariy by using somehing more spread ou. The deails of his approach will be discussed in he following secions, bu he key ideas are o use an ensemble (saisical sample) of sae esimaes insead of a single sae esimae; o calculae he error covariance marix from his ensemble insead of mainaining a separae covariance marix; and o use his calculaed covariance marix o calculae a common Kalman gain ha is used o updae each ensemble member in he analysis sep. The hope is ha he use of an ensemble will provide a beer represenaion of nonlineariy han is achieved by he EKF. A firs sigh he need o mainain a whole ensemble of sae esimaes makes he EnKF look compuaionally much more expensive ha he EKF. However, he EnKF may be he cheaper of he wo filers. One saving comes from he absence of a separae covariance marix o be evolved and updaed. In he case of high-dimensional sysems anoher saving comes if we use ensemble sizes ha are small compared o he number of observaions (as long as hey are no so small ha hey are saisically unrepresenaive). Such ensembles lead o covariance marices wih reduced rank, and his can be exploied in he analysis sep o lower he compuaional cos. The use of a common Kalman gain for all ensemble members also reduces he overhead of he addiional ensemble members. Anoher benefi of he EnKF is ha i does no require angen linear models o be implemened. This makes i especially aracive o individual researchers or small groups who wish o use daa assimilaion o solve problems in meeorology or oceanography wihou having he manpower resources 21

24 of a large organisaion a heir disposal. The original presenaion of he EnKF was in Evensen [7]. This secion follows in essence he review aricle Evensen [8] which incorporaes laer advances including he imporan recogniion by Burgers e al [4] (and independenly by Houekamer and Michell [11]) of he need o use an ensemble of pseudo-random observaion perurbaions in he analysis sep Noaion We shall use he subscrip i o denoe he individual members of an ensemble and N o denoe he size of an ensemble. Thus he members of an ensemble in sae space will be denoed by x i (i = 1,...,N). A superscrip f or a will frequenly be added o denoe a forecas or analysis ensemble. In siuaions where we mus give a single bes sae esimae from an ensemble, we shall use he ensemble mean x = 1 N N x i. i=1 When we require he error covariance marix we shall use he ensemble covariance marix P e = 1 N 1 N (x i x)(x i x) T. i=1 Noe he division by N 1 raher han N. This ensures han P e is an unbiased esimae of he populaion covariance marix P. I is convenien o inroduce he ensemble marix X = 1 ( N 1 x 1 x 2... x N ) (2.11) and he ensemble perurbaion marix X = 1 ( N 1 x 1 x x 2 x... x N x ). (2.12) 22

25 The ensemble covariance marix may hen be expressed as P e = X X T. (2.13) The Forecas Sep A is simples he EnKF assumes he same underlying nonlinear sochasic sysem model (2.9) as he EKF. The forecas sep evolves each ensemble member forward in ime using his model: x f i ( k ) = M(x a i ( k 1)) + η i ( k 1 ) where η i ( k 1 ) is pseudo-random noise o be drawn from a disribuion wih mean zero and covariance Q. There is no covariance marix o evolve and his offses he exra work needed o evolve an ensemble of saes. Noe ha he evoluion of each ensemble member is independen of all oher ensemble members, hus making he forecas sep well-suied for implemenaion on a parallel compuer. In he EnKF here is no need o make he assumpions abou lack of auocorrelaion of model noise and lack of correlaion beween model noise and he iniial analysis error ha were made in Secion These assumpions are required in he KF and EKF so ha η( k 1 ) is uncorrelaed wih he analysis error x a ( k 1 ) x ( k 1 ), hus jusifying he error covariance updae equaion (2.4). In he EnKF his equaion is no longer used and he assumpions are no necessary. The simulaion of ime-correlaed model noise is discussed in Evensen [8, Secion 4.2.1]. The EnKF is no limied o he simple dynamical model (2.9). Any sochasic difference or differenial equaion capable of numerical inegraion may be used. This issue is discussed in Evensen [8, Secion 3.4.2]. An iniial ensemble is required a ime. Assuming we have an iniial bes-guess esimae and some idea of he error in his esimae expressed hrough a covariance marix, we may generae an iniial ensemble by aking he bes-guess esimae and adding random perurbaions from a disribuion deermined by he covariance marix. Because he iniial esimae and covari- 23

26 ance marix may be crude, Evensen [8, Secion 4.1] recommends inegraing he iniial ensemble over a ime inerval conaining a few characerisic ime scales of he dynamical sysem o ensure ha he sysem is in dynamical balance and ha proper mulivariae correlaions have developed. This is fine as long as we are no required o assimilae any observaions during his inerval. Oher, perhaps more sophisicaed, mehods of ensemble generaion have been devised in he conex of ensemble predicion sysems; see, for example, Kalnay [15, Secion 6.5] The Analysis Sep There are various formulaions of he analysis sep of he EnKF. In his secion we consider he sochasic formulaion of Evensen [8]. Alernaive, deerminisic formulaions are he subjec of Secion 2.4. We sar by considering a linear observaion operaor, posponing he case of nonlinear observaions o Secion In he linear case he KF updae equaions (2.5) o (2.7) are exac and opimal. Therefore we wish o mimic hese in he ensemble version of he analysis sep. We may define an ensemble version of he Kalman gain by K e = P f e HT (HP f e HT + R) 1. We could ry updaing each ensemble member using he KF equaion wih his gain: x a i = x f i + K e (y Hx f i ). This yields an updae of he ensemble mean updae ha is like he KF: x a = x f + K e (y Hx f ) bu he ensemble perurbaions saisfy X a = (I K e H)X f 24

27 which implies ha he ensemble covariance updaes as P a e = (I K eh)p f e (I K eh) T. Compared o he KF covariance updae (2.7) his is oo small by a facor of (I K e H) T. To obain he desired saisics from he analysis ensemble we define an observaion ensemble y i = y + ε i (2.14) where ε i is pseudo-random noise drawn from a populaion wih mean zero and covariance R. We may define an observaion ensemble marix Y, an observaion ensemble perurbaion marix Y, and an observaion ensemble covariance marix R e direcly analogous o he marices X, X, and P e associaed wih a sae ensemble by (2.11) o (2.13). We define he ensemble Kalman gain wih R e in place of R: K e = P f eh T (HP f eh T + R e ) 1 (2.15) and updae each ensemble member using his gain and he members of he observaion ensemble: x a i = xf i + K e (y i Hx f i ). (2.16) The ensemble mean updaes as x a = x f + K e (y Hx f ) which is like he KF bu wih he mean of he synheic observaion ensemble y in place of he acual observaion y. We can eiher accep his resul as i is, noing ha y ends o y as he ensemble size increases, or following Evensen [8] we may impose he consrain ε = on our random vecors o ensure ha y = y uncondiionally. The ensemble perurbaion marix updaes as X a = (I K e H)X f + K e Y 25

28 which implies P a e = (I K eh)p f e + (I K eh)x f Y T K T e + K ey (X f ) T (I K e H) T. The firs erm on he righ is he desired expression for P f e. As long as our random vecors ε i are independen of he forecas ensemble perurbaions x f i x f, he produc X f Y T and hence he second and hird erms on he righ hand side will end o zero as he ensemble size increases. Once again we may eiher accep his resul as i is or follow Evensen [8] and impose he addiional consrain (x f x f )ε T = on our random vecors ε i o ensure he desired covariance updae uncondiionally. As presened so far he algorihm for he analysis sep consiss of generaing an observaion ensemble using (2.14), calculaing he ensemble Kalman gain using (2.15), and updaing he ensemble members using (2.16). This does lile o address he problems wih huge covariance marices ha were menioned a he end of Secion The calculaion of K e involves he calculaion and sorage of he n n covariance marix P f e and he inversion of he m m marix HP f e HT + R e. We now consider how he calculaion may be arranged o avoid hese problems. We sar by rewriing (2.15) in erms of ensemble marices: K e = X f (X f ) T H T (HX f (X f ) T H T + Y Y T ) 1. The srucure of he righ hand side is revealed more clearly if we inroduce an ensemble of forecas observaions y f i = Hx f i. Thus y f i is wha he observaion would be if he rue sae of he sysem was x f i and here was no observaion noise. We define marices Y f and Y f for his ensemble in he same way as Y and Y are defined for he ensemble y i. In erms of hese marices K e = X f (Y f ) T (Y f (Y f ) T + Y Y T ) 1. (2.17) 26

29 We now noe ha if we consrain he random vecors ε i in he way discussed above so ha X f Y T =, hen i follows (on muliplying by H) ha Y f Y T = and hence ha he marix we mus inver can be wrien as Y f (Y f ) T + Y Y T = (Y f + Y )(Y f + Y ) T. (2.18) We can make his subsiuion in he formula for he K e even if we are no consraining he ε i, jusifying i on he grounds ha as long as he ε i are independen of he forecas observaion perurbaions y f i y f, he produc Y f Y T ends o zero as he ensemble size increases and hence he new formula for K e is as good an approximaion o he rue Kalman gain K as he old one. Now we ake he singular value decomposiion (SVD) of he m N marix ha is muliplied by is ranspose on he righ hand side of (2.18): Y f + Y = UΣV T where Σ is he p p diagonal marix of nonzero singular values (p being he rank of he marix) and U and V are column-orhogonal marices of sizes m p and N p respecively. From his decomposiion we can find he eigenvalue decomposiion Y f (Y f ) T + Y Y T = UΛU T were Λ = ΣΣ T is he diagonal marix of nonzero eigenvalues, and he inverse follows as (Y f (Y f ) T + Y Y T ) 1 = UΛ 1 U T. If p < m hen he marix o be invered is singular and his inverse is a pseudo-inverse. If he number of observaions m is large as in he NWP example and he ensemble size N is small compared o m, we have reduced an expensive inversion of an m m marix requiring O(m 3 ) floaing poin operaions o a cheaper SVD of an m N marix requiring O(m 2 N +N 3 ) = O(m 2 N) operaions (see Golub and Van Loan [1, Secion 5.4.5]) 1. 1 The O(mN) used in Evensen [8, Secion 4.3.2] appears o be an error. 27

30 Wih he inverse in he calculaion of K e aken care of, we consider how he analysis sep may be compleed wihou having o compue and sore excessively large marices. We confine ourselves o sysems sized as in he NWP example in which p N m n. We assume ha he ensemble and ensemble perurbaion marices have been compued and sored for he forecas, observaion, and forecas observaion ensembles. None of hese marices is larger han n N and he majoriy are m N. We also assume ha he above SVD has been compued and U and Λ 1 sored. These marices are smaller sill a m p and p p respecively. The ensemble updae equaion (2.16) can be wrien in marix form as X a = X f + K e (Y Y f ) (2.19) = X f + X f (Y f ) T UΛ 1 U T (Y Y f ). The compuaions are bes performed in he order se ou in he following able, which also shows he size of he resuling marix and he number of floaing poin operaions required o compue i. Compuaion Size Operaions Z 1 = Λ 1 U T p m mp Z 2 = Z 1 (Y Y f ) p N mnp Z 3 = UZ 2 m N mnp Z 4 = (Y f ) T Z 3 N N mn 2 X a = X f + X f Z 4 n N nn 2 The larges marix is he final analysis ensemble marix X a which has size n N. The mos expensive compuaion is also he final one wih O(nN 2 ) operaions. Assuming ha muliplicaion by H and generaing he random vecors ε i are relaively cheap, he oal cos of an efficien implemenaion of he analysis sep is herefore O(m 2 N + nn 2 ) and he larges marix ha has o be sored is of size n N. This compares o O(m 2 n + n 2 N) and n n for a naive implemenaion, and O(n 3 ) and n n for he analysis sep of he KF (see Appendix A). However, i should be added ha floaing poin 28

31 operaion couns are no he whole sory, especially wih modern compuer archiecures. Memory access may be he main boleneck, which is why he size of marices sored is imporan. On parallel machines he minimisaion of communicaion beween processors will be he dominan consideraion Nonlinear Observaion Operaors Evensen [8, Secion 4.5] presens he following echnique for exending he preceding formulaion of he analysis sep o nonlinear observaion operaors of he ype in (2.1). We augmen he sae vecor wih a diagnosic variable ha is he prediced observaion vecor: ˆx = x H(x) and define a linear observaion operaor on augmened sae space by Ĥ x y = y. We hen carry ou he analysis sep in augmened sae space using ˆx and Ĥ in place of x and H. Superficially, his echnique appears o reduce he nonlinear problem o he previously-solved linear one. However, he linear problem creaed is no quie he same as he problem solved in Secion 2.3.3, because he valid saes of our sysem only occupy a submanifold of augmened sae space insead of he whole space. Thus, whils his is a reasonable way of formulaing he EnKF for nonlinear observaion operaors, i is no as well-founded as he linear case, which can be jusified as an approximaion o he exac and opimal KF. We now ranslae he augmened sae space formulaion of he analysis sep o a formulaion ha does no explicily refer o ha space. Firs noe ha he observaion ensemble y i is independen of sae space, so i and he marices Y and Y are he same as before. The forecas observaion ensemble 29

32 is y f i = Ĥˆxf i = H(x f i ). The mean of his ensemble is y f = H(x f ) and hus he forecas observaion ensemble and ensemble perurbaion marices are Y f = Y f = 1 ( N 1 1 ( N 1 H(x f 1)... H(x f N) H(x f 1) H(x f )... H(x f N ) H(xf ) ) (2.2) ). (2.21) We can now use (2.17) and (2.19) o wrie he ensemble updae in augmened sae space as K e = X f (Y f ) T (Y f (Y f ) T + Y Y T ) 1 X a = X f + K e (Y Y f ). Taking he firs n rows of hese equaions we obain K e = X f (Y f ) T (Y f (Y f ) T + Y Y T ) 1 (2.22) X a = X f + K e (Y Y f ) (2.23) which is he same ensemble updae as in he linear case excep ha Y f and Y f are given in erms of he nonlinear observaion operaor H by (2.2) and (2.21). 2.4 Deerminisic Formulaions of he Analysis Sep The analysis sep of he EnKF as presened in Secion uses a synheic observaion ensemble creaed by adding random vecors o he real 3

33 observaion. This sochasic elemen exposes he mehod o sampling errors. We now consider he alernaive formulaions reviewed in Tippe e al [23]. These are deerminisic formulaions ha do no require a synheic observaion ensemble. However, i should be poined ou ha here is a poenial flaw in some of hese mehods ha appears o have been overlooked in he lieraure. This is illusraed experimenally in Chaper 5 and demonsraed heoreically in Secion 6.1. We shall ignore his flaw for now and give an exposiion ha follows in essence Tippe e al [23]. Unless oherwise saed we assume a nonlinear observaion operaor as in Secion The General Framework All he mehods o be described in his secion spli he ensemble updae ino wo pars: firs he analysis ensemble mean is calculaed, hen he analysis ensemble perurbaions. We sar by examining wha he mehod of Secion does o he ensemble mean. Taking he mean of boh sides of (2.23) gives x a = x f + K e (y y f ). Recall from Secion ha we have y = y, eiher exacly if we impose he righ consrain on our random vecors or in he limi of large ensembles. Making his subsiuion gives us an updae equaion for he ensemble mean in which he observaion ensemble has been averaged ou 2 : x a = x f + K e (y y f ) (2.24) = x f + K e (y H(x f )). We shall use his equaion in all our deerminisic formulaions of he analysis sep. However, we canno use K e as defined by (2.22) because we no longer have an observaion ensemble perurbaion marix Y. Insead we mus 2 Lorenc [18, Equaion (4)] uses H(x f ) insead of H(x f ). The version given above has he advanage ha i arises naurally ou of averaging he sochasic formulaion as shown. I also akes more accoun of he nonlineariy of H. For linear observaion operaors H(x f ) = H(x f ). 31

34 go back o using he populaion version of he observaion error covariance marix R insead of he ensemble version R e = Y Y T. Thus we ake K e = X f (Y f ) T (Y f (Y f ) T + R) 1. The marix invered in his expression is he ensemble version of he innovaions covariance marix. I appears frequenly in wha follows, so we inroduce a special noaion for i: S = Y f (Y f ) T + R. (2.25) We now consider he updae of he ensemble perurbaions. In he case of a linear observaion operaor we would like he ensemble covariance marix o updae like he KF covariance updae (2.7). Thus in his case we require X a (X a ) T = P a e = (I K e H)P f e = (I X f (Y f ) T S 1 H)X f (X f ) T = X f (I (Y f ) T S 1 Y f )(X f ) T. The firs and las erms in his chain of equaions make no menion of he linear operaor H, so we impose heir equaliy as a condiion in he case of nonlinear observaion operaors as well. The equaliy will be saisfied if X a = X f T (2.26) where T is an N N marix ha is a marix square roo of I (Y f ) T S 1 Y f in he sense ha TT T = I (Y f ) T S 1 Y f. (2.27) Noe ha T saisfying (2.27) is no unique and may be replaced by TU where U is an arbirary N N orhogonal marix. The mehods ha we shall now consider differ in heir choice of T. 32

35 2.4.2 The Direc Mehod Tippe e al [23] inroduce a so-called direc mehod which, as is name suggess, akes a direc approach o finding a T saisfying (2.27). The firs sep is o solve he linear sysem SZ = Y f (2.28) for he m N marix Z. In he case where N m (as in he NWP example) Tippe e al [23] sugges exploiing he ideniy S 1 = R 1 R 1 Y f (I + (Y f ) T R 1 Y f ) 1 (Y f ) T R 1 which may be verified by muliplying by S and using he definiion (2.25) of S. Compuing R 1 for use in his ideniy is usually much easier han compuing S 1 because R has a simple srucure, ypically diagonal. The oher marix ha needs o be invered is he N N marix I+(Y f ) T R 1 Y f. Thus he ideniy reduces he inversion of an m m marix o he inversion of an N N one (alhough his is no he full sory in he reducion of work needed o solve (2.28) because he equaion can be solved wihou finding S 1 explicily). Wih Z found, he nex sep is o form I (Y f ) T S 1 Y f = I (Y f ) T Z and find is marix square roo T. Tippe e al [23] do no enjoin a paricular mehod for finding he marix square roo. Since I (Y f ) T S 1 Y f is posiive definie (as follows from (2.31) below) one possibiliy is o use a Cholesky facorisaion algorihm such as hose given in Golub and Van Loan [1, Secion 4.2] The Serial Mehod If he observaions represened by he individual componens of he observaion vecor y have uncorrelaed errors, hey may be assimilaed one a a 33

36 ime insead of all a once. This is a sandard echnique of Kalman filering and has he advanage ha i reduces he inversion of a large marix o he inversion of a sequence of scalars. The procedure is jusified because i is in effec a sequence of sandard assimilaion cycles wih zero-lengh forecas seps. Wha is no obvious is ha he resul is he same as processing all observaions a once. For a proof in he conex of he sandard KF see Dance [5, Appendix A]. The assumpion of uncorrelaed observaion error componens is he basis of he serial mehod of Tippe e al [23], which repeaedly applies an analysis scheme designed for an observaion space of dimension one. In such an observaion space R and S are scalars and Y f is a row vecor. The mehod uses a closed form expression for T obained by subsiuing T = I β(y f ) T Y f (2.29) ino (2.27) and solving for β, giving β = 1 S ± RS. (2.3) The serial mehod avoids expensive marix inversions, bu his has o be raded off agains he need o apply he mehod muliple imes a each analysis sep. See Tippe e al [23] for a more deailed accoun of he issues. Oher insances of he use of serial processing wih he EnKF include Houekamer and Michell [13], where i is used wih a sochasic formulaion of he analysis sep; Bishop e al [3], which discusses is use wih he Ensemble Transform Kalman Filer o be described shorly; and Whiaker and Hamill [25], where i is used in a deerminisic formulaion of he analysis sep ha Tippe e al [23] show o be equivalen o (2.29) wih he plus sign aken in (2.3) The Ensemble Transform Kalman Filer The Ensemble Transform Kalman Filer (ETKF) was originally inroduced in Bishop e al [3], which describes is use o make rapid assessmen of 34

37 he fuure effec on error covariance of alernaive sraegies for deploying observaional resources. The ETKF explois he ideniy I (Y f ) T S 1 Y f = (I + (Y f ) T R 1 Y f ) 1 (2.31) which may be verified by muliplying I (Y f ) T S 1 Y f by I+(Y f ) T R 1 Y f and using he definiion (2.25) of S. The mehod sars by compuing he N N marix (Y f ) T R 1 Y f. This is usually much easier han compuing (Y f ) T S 1 Y f because R usually has a simple srucure. We hen compue he eigenvalue decomposiion (Y f ) T R 1 Y f = UΛU T where U is orhogonal and Λ is diagonal. I follows from (2.31) ha I (Y f ) T S 1 Y f = U(I + Λ) 1 U T and hence ha a square roo of he ype we seek is T = U(I + Λ) 1 2. Noe ha I + Λ is diagonal, so raising i o a power is easy. Using his T in (2.26) gives he ETKF The Ensemble Adjusmen Kalman Filer The Ensemble Adjusmen Kalman Filer (EAKF) was originally inroduced in Anderson [2]. The EAKF differs from he deerminisic mehods discussed so far in ha i may be wrien in he form X a = AX f (2.32) alhough i may be wrien in he pos-muliplier form (2.26) as well. We here assume ha he observaion operaor is linear. The mehod may be exended o nonlinear observaion operaors by using he augmened sae space of 35

38 Secion The firs sage in finding A is o compue he eigenvalue decomposiion P f e = FG 2 F T (2.33) where G is he p p diagonal marix of posiive square roos of nonzero eigenvalues and F is an n p column-orhogonal marix. We nex perform he eigenvalue decomposiion (HFG) T R 1 HFG = Ũ ΛŨT where Λ is p p diagonal and Ũ is p p orhogonal. We hen define A = FGŨ(I + Λ) 1 2 G 1 F T. (2.34) To see ha he above gives he righ analysis ensemble saisics noe ha we can find F and G in (2.33) from he SVD X f = FGW T (2.35) where W is an N p column-orhogonal marix. Subsiuing (2.34) and (2.35) ino (2.32) gives X a = FGŨ(I + Λ) 1 2 W T (2.36) and hence P a e = X a (X a ) T = FGŨ(I + Λ) 1ŨT GF T = FG(I + (HFG) T R 1 HFG) 1 GF T. Now HFG is a marix wih he propery HFG(HFG) T = HFG 2 F T H T = HP f eh T = S R 36

39 which is precisely he propery of Y f ha was used o esablish (2.31). Therefore we may apply his ideniy o HFG o obain P a e = FG(I (HFG) T S 1 HFG)GF T = P f e P f eh T S 1 HP f e = (I K e H)P f e which is he required ensemble covariance updae. To wrie he EAKF in he pos-muliplier form (2.26) noe ha (2.35) and (2.36) imply X a = X f WŨ(I + Λ) 1 2 W T. Noe, however, ha he EAKF does no quie fi ino he framework of Secion 2.4.1, which consiss of no jus he pos-muliplier equaion (2.26) bu he square roo condiion (2.27) as well; see Appendix B for deails. Anoher deerminisic formulaion of he analysis sep capable of being wrien in he pre-muliplier form (2.32) is given in Whiaker and Hamill [25]. 2.5 Summary Following he inroducory maerial in Secions 2.1 and 2.2, his chaper has presened five formulaions of he EnKF. All formulaions differ from he KF and EKF in using an ensemble of sae esimaes insead of a single sae esimae and no mainaining a separae error covariance marix. They all offer advanages over he sandard filers of reduced compuaional cos, beer handling of nonlineariy, and greaer ease of implemenaion hrough no using angen linear models. All formulaions share he forecas sep described in Secion and he echnique for dealing wih nonlinear observaion operaors described in Secion Where he formulaions differ is in he analysis sep. Secion described a sochasic formulaion ha makes use of an ensemble of pseudorandom observaion perurbaions. The oher formulaions are deerminisic formulaions ha fi ino a general framework described in Secion

40 This framework encompasses he direc mehod (Secion 2.4.2, no as precisely defined as he oher mehods), he serial mehod (Secion 2.4.3, limied o uncorrelaed observaions), he ETKF (Secion 2.4.4), and he EAKF (Secion 2.4.5). I was poined ou ha here is a poenial flaw in some of hese mehods; his is a major opic of Chapers 5 and 6. The nex chaper discusses he selecion of an EnKF algorihm for implemenaion. I also describes he problems encounered in implemening he raw algorihms as presened in his chaper and how hey may be reformulaed o give algorihms ha are analyically equivalen bu numerically beer behaved. 38

41 Chaper 3 Implemening an Ensemble Kalman Filer This chaper is abou he implemenaion of an EnKF. I describes some problems ha were encounered, he soluions ha were adoped, and some furher improvemens o he algorihms of Chaper 2. The EnKF is inended for experimens wih he low-dimensional mechanical sysem described in Chaper 4, alhough i is capable of being used wih oher sysems as well. Of he formulaions of he EnKF in Chaper 2, i was iniially decided o implemen he ETKF. A deerminisic formulaion of he analysis sep has he advanage over a sochasic formulaion of eliminaing one source of sampling error. Compared o he oher deerminisic formulaions in Secion 2.4, he ETKF has he advanage over he direc mehod of a more clearly defined algorihm, he advanage over he serial mehod of no requiring uncorrelaed observaions, and he advanage over he EAKF of avoiding one of he eigenvalue decomposiions. However, here are some unexpeced problems wih he ETKF ha are illusraed experimenally in Chaper 5 and explained heoreically in Chaper 6. Once hese were discovered, he EAKF was implemened as well for comparison. The implemenaion of boh filers is described in his chaper in order o keep similar maerial ogeher. 39

42 3.1 Implemening he ETKF The iniial implemenaion of he ETKF closely followed he algorihm of Secion This creaed a problem wih he eigenvalue decomposiion (Y f ) T R 1 Y f = UΛU T (3.1) which produced eigenvalues and eigenvecors having significan imaginary pars. The reason for his is he lack of associaiviy in machine muliplicaion, leading o he osensibly symmeric marix (Y f ) T R 1 Y f becoming asymmeric when evaluaed as ((Y f ) T R 1 )Y f. This may be avoided by inroducing he scaled forecas observaion ensemble perurbaion marix Ŷ f = R 1 2 Y f (3.2) and wriing (Y f ) T R 1 Y f = (Ŷf ) TŶf. (3.3) As long as machine muliplicaion is commuaive his way of evaluaing (Y f ) T R 1 Y f leads o a symmeric marix wih real eigenvalues and eigenvecors. Noe ha finding R 1 2 is easy in he common case of diagonal R. Indeed, i is ofen R 1 2 (he diagonal marix of observaion error sandard deviaions) ha is he primary given quaniy raher han R, which makes evaluaing R 1 2 easier sill. Regardless of he maer of symmery, i is in any case advanageous o scale observaion space quaniies such as Y f by R 1 2 before processing hem furher. Such scaling has he effec of normalising observaions ha are possibly of disparae physical quaniies wih differen error sandard deviaions so ha hey are dimensionless wih sandard deviaion one. This is useful because i prevens informaion becoming los due (say) o rounding errors. The advisabiliy of such a scaling in he conex of he sochasic formulaion of he EnKF is menioned in Evensen [8, Secion 4.3.2]. A scaled observaion operaor is also par of he original presenaion of he ETKF in Bishop e al [3] (alhough here i is no explicily exploied o ensure 4