arxiv: v2 [math.na] 20 Sep 2016

Transcription

1 ANALYSIS OF THE ENSEMBLE KALMAN FILTER FOR INVERSE PROBLEMS CLAUDIA SCHILLINGS AND ANDREW M. STUART arxiv:162.22v2 [mah.na] 2 Sep 216 Absrac. The ensemble Kalman filer (EnKF) is a widely used mehodology for sae esimaion in parial, noisily observed dynamical sysems, and for parameer esimaion in inverse problems. Despie is widespread use in he geophysical sciences, and is gradual adopion in many oher areas of applicaion, analysis of he mehod is in is infancy. Furhermore, much of he exising analysis deals wih he large ensemble limi, far from he regime in which he mehod is ypically used. The goal of his paper is o analyze he mehod when applied o inverse problems wih fixed ensemble size. A coninuous-ime limi is derived and he long-ime behavior of he resuling dynamical sysem is sudied. Mos of he rigorous analysis is confined o he linear forward problem, where we demonsrae ha he coninuous ime limi of he EnKF corresponds o a se of gradien flows for he daa misfi in each ensemble member, coupled hrough a common pre-condiioner which is he empirical covariance marix of he ensemble. Numerical resuls demonsrae ha he conclusions of he analysis exend beyond he linear inverse problem seing. Numerical experimens are also given which demonsrae he benefis of various exensions of he basic mehodology. Key words. Bayesian Inverse Problems, Ensemble Kalman Filer, Opimizaion AMS subjec classificaions. 65N21, 62F15, 65N75 1. Inroducion. The Ensemble Kalman filer (EnKF) has had enormous impac on he applied sciences since is inroducion in he 199s by Evensen and coworkers; see [11] for an overview. I is used for boh daa assimilaion problems, where he objecive is o esimae a parially observed ime-evolving sysem sequenially in ime [17], and inverse problems, where he objecive is o esimae a (ypically disribued) parameer appearing in a differenial equaion [25]. Much of he analysis of he mehod has focussed on he large ensemble limi [24, 23, 13, 2, 1, 22]. However he primary reason for he adopion of he mehod by praciioners is is robusness and perceived effeciveness when used wih small ensemble sizes, as discussed in [2, 3] for example. I is herefore imporan o sudy he properies of he EnKF for fixed ensemble size, in order o beer undersand curren pracice, and o sugges fuure direcions for developmen of he mehod. Such fixed ensemble size analyses are saring o appear in he lieraure for boh daa assimilaion problems [19, 28] and inverse problems [15, 14, 16]. In his paper we analyze he EnKF for inverse problems, adding greaer deph o our undersanding of he basic mehod, as formulaed in [15], as well as varians on he basic mehod which employ echniques such as variance inflaion and localizaion (see [21] and he references herein), ogeher wih new ideas (inroduced here) which borrow from he use of sequenial Mone Carlo (SMC) mehod for inverse problems inroduced in [18]. Le G : X Y be a coninuous mapping beween separable Hilber spaces X and Y. We are ineresed in he inverse problem of recovering unknown u from observaion y where y = G(u) + η. Here η is an observaional noise. We are ypically ineresed in he case where he in- Mahemaics Insiue, Universiy of Warwick, Covenry CV4 7AL, UK (c.schillings@warwick.ac.uk). Mahemaics Insiue, Universiy of Warwick, Covenry CV4 7AL, UK (a.m.suar@warwick.ac.uk). 1

2 2 C. SCHILLINGS AND A. M. STUART version is ill-posed on Y, i.e. one of he following hree condiions is violaed: exisence of soluions, uniqueness, sabiliy. In he linear seing, we can hink for example of a compac operaor violaing he sabiliy condiion. Indeed hroughou we assume in all our rigorous resuls, wihou commen, ha Y = R K for K N excep in a few paricular places where we explicily sae ha Y is infinie dimensional. A key role in such inverse problems is played by he leas squares funcional Φ(u; y) = 1 2 Γ 1 2 (y G(u)) 2 Y. Here Γ > normalizes he model-daa misfi and ofen knowledge of he covariance srucure of ypical noise η is used o define Γ. When he inverse problem is ill-posed, infimizaion of Φ in X is no a wellbehaved problem and some form of regularizaion is required. Classical mehods include Tikhonov regularizaion, infimizaion over a compac ball in X and runcaed ieraive mehods [9]. An alernaive approach is Bayesian regularizaion. In Bayesian regularizaion (u, y) is viewed as a joinly varying random variable in X Y and, assuming ha η N(, Γ) is independen of u µ, he soluion o he inverse problem is he X valued random variable u y disribued according o measure (1) µ(du) = 1 Z exp( Φ(u; y) ) µ (du), where Z is chosen so ha µ is a probabiliy measure: Z := exp ( Φ(u; y) ) µ (du). X See [7] for deails concerning he Bayesian mehodology. The EnKF is derived wihin he Bayesian framework and, hrough is ensemble properies, is viewed as approximaing he poserior disribuion on he random variable u y. However, excep in he large sample limi for linear problems [23, 13, 2] here is lile o subsaniae his viewpoin; indeed he paper [1] demonsraes his quie clearly by showing ha for nonlinear problems he large ensemble limi does no approximae he poserior disribuion. In [22], a relaed resul is proved for he EnKF in he conex of daa assimilaion; in he large ensemble size limi he EnKF is proved o converge o he mean-field EnKF, which provides he opimal linear esimaor of he condiional mean, bu does no reproduce he filering disribuion, excep in he linear Gaussian case. A differen perspecive on he EnKF is ha i consiues a derivaive-free opimizaion echnique, wih he ensemble used as a proxy for derivaive informaion. This opimizaion viewpoin was adoped in [15, 14] and is he one we ake in his paper: hrough analysis and numerical experimens we sudy he properies of he EnKF as a regularizaion echnique for minimizaion of he leassquares misfi funcional Φ a fixed ensemble size. We do, however, use he Bayesian perspecive o derive he algorihm, and o sugges varians of i. In secion 2 we describe he EnKF in is basic form, deriving he algorihm by means of ideas from SMC applied o he Bayesian inverse problem, ogeher wih invocaion of a Gaussian approximaion. Secion 3 describes coninuous ime limis of he mehod, leading o differenial equaions, and in secion 4 we sudy properies of he differenial equaions derived in he linear case and, in paricular, heir long-ime behavior. Using his analysis we obain clear undersanding of he sense in which he EnKF is a regularized opimizaion mehod for Φ. Indeed we show in secion 3 ha

3 ANALYSIS OF THE ENSEMBLE KALMAN FILTER FOR INVERSE PROBLEMS 3 he coninuous ime limi of he EnKF corresponds o a se of precondiioned gradien flows for he daa misfi in each ensemble member. The common precondiioner is he empirical covariance marix of he ensemble which hereby couples he ensemble members ogeher and renders he algorihm nonlinear, even for linear inverse problems. Secion 5 is devoed o numerical sudies which illusrae he foregoing heory for linear problems, and which also demonsrae ha similar ideas apply o nonlinear problems. In secion 6 we discuss varians of he basic EnKF mehod, in paricular he addiion of variance inflaion, localizaion or he use of random search mehods based on SMC, wihin he ensemble mehod; all of hese mehods break he invarian subspace propery of he basic EnKF proved in [15] and we explore numerically he benefis of doing so. 2. The EnKF for Inverse Problems. Here we describe how o derive he ieraive EnKF as an approximaion of he SMC mehod for inverse problems. Recall he poserior disribuion µ given by (1) and define he probabiliy measures µ n by, for h = N 1, µ n (du) exp ( nhφ(u; y) ) µ (du). The measures µ n are inermediae measures defined via likelihoods scaled by he sep size h = N 1. I follows ha µ N = µ he desired measure on u y. Then (2) µ n+1 (du) = 1 Z n exp ( hφ(u; y) ) µ n (du), for Z n = exp( hφ(u; y))µ n (du). Denoing by L n he nonlinear operaor corresponding o applicaion of Bayes heorem o map from µ n o µ n+1 we have (3) µ n+1 = L n µ n. We have inroduced an aricial discree ime dynamical sysem which maps he prior µ ino he poserior µ N = µ. A heurisic worhy of noe is ha alhough we look a he daa y a each of N seps, he effecive variance is amplified by N = 1/h a each sep, compensaing for he redundan, repeaed use of he daa. The idea of SMC is o approximae µ n by a weighed sum of Dirac masses: given a se of paricles and weighs {u (j) n, w (j) n } J j=1 he approximaion akes he form µ n j=1 w n (j) δ un (j), wih δ un (j) denoing he dela-dirac mass locaed a u (j) n. The mehod is defined by he mapping of he paricles and weighs a ime n o hose a ime n + 1. The mehod is inroduced for Bayesian inverse problems in [18] where i is used o sudy he problem of inferring he iniial condiion of he Navier-Sokes equaions from daa. In [4] he mehod is applied o he inverse problem of deermining he coefficien of a divergence form ellipic PDE from linear funcionals of he soluion; furhermore he mehod is also proved o converge in he large paricle limi J. In pracice he SMC mehod can perform poorly. This happens when he weighs degenerae in ha one of he weighs akes a value close o one and all ohers {w (j) n } J j=1

4 4 C. SCHILLINGS AND A. M. STUART are negligible. The EnKF aims o counerac his by always seeking an approximaion in he form µ n 1 J j=1 δ (j) u n and hus µ n+1 1 J j=1 δ (j) u n+1. The mehod is defined by he mapping of he paricles a ime n ino hose a ime n + 1. Le u n = {u (j) n } J j=1. Then using equaion (25) in [15] wih Γ h 1 Γ shows ha his mapping of paricles has he form (4) u (j) n+1 = u(j) n + C up (u n )(C pp (u n ) + h 1 Γ) 1( y (j) n+1 G(u(j) n ) ), j = 1,, J, where y (j) n+1 = y + ξ(j) n+1 and, for u = {u (j) } J j=1, we define he operaors Cpp and C up by (5) C pp (u) = 1 J C up (u) = 1 J u = 1 J ( G(u (j) ) G ) ( G(u (j) ) G ), j=1 ( u (j) u ) ( G(u (j) ) G ), j=1 u (j), G = 1 J j=1 G(u (j) ). j=1 We will consider boh he cases where ξ (j) n+1 and where, wih respec o boh j and n, he ξ (j) n+1 are i.i.d. random variables disribued according o N(, h 1 Γ). We can unify by considering he i.i.d. family of random variables ξ (j) n+1 N(, h 1 Σ) and focussing exclusively on he cases where Σ = Γ and where Σ =. The heoreical resuls will be solely derived for he seing Σ =, i.e. no arificial noise will be added o he observaional daa. The derivaion of he EnKF as presened here relies on a Gaussian approximaion, which can be inerpreed as a linearizaion of he nonlinear operaor L n in he following way: in he large ensemble size limi, he EnKF esimae corresponds o he bes linear esimaor of he condiional mean. See [12] for a general discussion of Bayes linear mehods and [22] for deails in he conex of daa assimilaion. Thus, besides he approximaion of he measures µ n by a J paricle Dirac measure, here is an addiional unconrolled error resuling from he Gaussian approximaion, and analyzed in [1]. In [27], he EnKF in combinaion wih an annealing process is used o accoun for nonlineariies in he forward problem by weigh-correcing he EnKF. For daa assimilaion problems, similar echniques can be applied o improve he performance of he EnKF in he nonlinear regime, see e.g. [5] and he references herein for more deails.

5 ANALYSIS OF THE ENSEMBLE KALMAN FILTER FOR INVERSE PROBLEMS 5 In summary, excep in he Gaussian case of linear problems, here is no convergence o µ n as J. Our focus, hen, is on undersanding he properies of he algorihm for fixed J, as an opimizaion mehod; we do no sudy he approximaion of he measure µ. In his conex we also recall here he invarian subspace propery of he EnKF mehod, as esablished in [15]: Lemma 1. If S is he linear span of {u (j) }J j=1 Z + {1,..., J}. hen u(j) n S for all (n, j) 3. Coninuous Time Limi. Here we sudy a coninuous ime limi of he EnKF mehodology as applied o inverse problems; his limi arises by aking he parameer h, appearing in he incremenal formulaion (2) of he Bayesian inverse problem (1), o zero. We proceed purely formally, wih no proofs of he limiing process, as our main aim in his paper is o sudy he behavior of he coninuous ime limis, no o jusify aking ha limi. However we noe ha he invarian subspace propery of Lemma 1 means ha he desired limi heorems are essenially finie dimensional and sandard mehods from numerical analysis may be used o esablish he limis. In he nex secion all he heoreical resuls are derived under he assumpion ha G is linear and Σ =. However in his secion we derive he coninuous ime limi in a general seing, before specifying o he linear noise-free case The Nonlinear Problem. Recall he definiion of he operaor C up given by (5). We recall ha u n = {u (j) n } J j=1, and assume ha u n u(nh) in (4) in he limi h. The updae sep of he EnKF (4) can be wrien in he form of a ime-sepping scheme: u (j) n+1 = u(j) n + hc up (u n )(hc pp (u n ) + Γ) 1( y G(u (j) n ) ) +hc up (u n )(hc pp (u n ) + Γ) 1 ξ (j) n+1 = u (j) n + hc up (u n )(hc pp (u n ) + Γ) 1( y G(u (j) n ) ) +h 1 2 C up (u n )(hc pp (u n ) + Γ) 1 Σζ (j) n+1, where ζ (j) n+1 N (, I) i.i.d.. If we ake he limi h hen his is clearly a amed Euler-Maruyama ype discreizaion of he se of coupled Iô SDEs (6) du (j) d = C up (u)γ 1 (y G(u (j) )) + C up (u)γ 1 Σ Using he definiion of he operaor C up we see ha (7) where du (j) d = 1 J G(u (k) ) G, y G(u (j) ) + Σ u = 1 J u (j), j=1 G = 1 J dw (j) d G(u (j) ) j=1 dw (j) d Γ( u (k) u ), and, Γ = Γ 1 2, Γ 1 2 wih, he inner-produc on Y. The W (j) are independen cylindrical Brownian moions on X. The consrucion demonsraes ha, provided a soluion o (7) exiss, i will saisfy a generalizaion of he subspace propery of Lemma 1 o coninuous ime because he vecor field is in he linear span of he ensemble iself..

6 6 C. SCHILLINGS AND A. M. STUART 3.2. The Linear Noise-Free Problem. In his subsecion we sudy he linear inverse problem, for which G( ) = A for some A L(X, Y). We also resric aenion o he case where Σ =. Then he coninuous ime limi equaion (7) becomes (8) du (j) d = 1 J A(u (k) u), y Au (j) ( Γ u (k) u ), j = 1,, J. For u = {u (j) } J j=1 we define he empirical covariance operaor C(u) = 1 J ( u (k) u ) ( u (k) u ). Then equaion (8) may be wrien in he form (9) where, in his linear case, du (j) d = C(u)D u Φ(u (j) ; y) (1) Φ(u; y) = 1 2 Γ 1 2 (y Au) 2. Thus each paricle performs a precondiioned gradien descen for Φ( ; y), and all he individual gradien descens are coupled hrough he precondiioning of he flow by he empirical covariance C(u). This gradien flow is hus nonlinear, even hough he forward map is linear. Using he fac ha C is posiive semi-definie i follows ha d d Φ( u(); y ) = d 1 d 2 Γ 1 2 (y Au) 2. This gives an a priori bound on Au() Γ, bu does no give global exisence of a soluion when Γ 1 2 A is compac. However, global exisence may be proved as we show in in he nex secion, using he invarian subspace propery. 4. Asympoic Behavior in he Linear Seing. In his secion we sudy he differenial equaions (8). Alhough derivaion of he coninuous ime limi suggess sopping he inegraion a ime T = 1 i is noneheless of ineres o sudy he dynamical sysem in he long-ime asympoic T as his sheds ligh on he mechanisms a play wihin he ensemble mehodology, and poins o possible improvemens of he algorihm. In he firs subsecion 4.1 we sudy he case where he daa is noise-free. Theorem 2 shows exisence of a soluion saisfying he subspace propery; Theorem 3 shows collapse of all ensemble members owards heir mean value a an algebraic rae; Theorem 4 decomposes he error, in he image space under A, ino an error which decays o zero algebraically (in a subspace deermined by he iniial ensemble) and an error which is consan in ime (in a complemenary space); Corollary 5 ransfers hese resuls o he sae space of he unknown under addiional assumpions. We assume hroughou his secion ha he forward operaor is a bounded, linear operaor. The convergence analysis presened in Theorem 4 addiionally requires he operaor o be injecive. For compac operaors, he convergence resul in he observaional space does no imply convergence in he sae space. However, assuming he forward operaor is boundedly inverible, he generalizaion is sraighforward. Noe ha his assumpion is ypically no fulfilled in he inverse seing, bu opens up he perspecive o use he EnKF as a linear solver. Subsecion 4.2 briefly discusses he noisy daa case where he resuls are analogous o hose in he preceding subsecion.

7 ANALYSIS OF THE ENSEMBLE KALMAN FILTER FOR INVERSE PROBLEMS The Noise-Free Case. In proving he following heorems we will consider he siuaion where he daa y is he image of a ruh u X under A. I is hen useful o define e (j) = u (j) ū, r (j) = u (j) u E lj = Ae (l), Ae (j) Γ, R lj = Ar (l), Ar (j) Γ, F lj = Ar (l), Ae (j) Γ. We view he las hree iems as enries of marices E, R and F. The resuling marices E, R, F R J J saisfy he following: (i) E, R are symmeric; (ii) we may facorize E() = XΛ()X T where X is an orhogonal marix whose columns are he eigenvecors of E(), and Λ() is a diagonal marix of corresponding eigenvalues; (iii) if l = (1,..., 1) T, hen El = F l =. Noe ha e (j) measures deviaion of he j h ensemble member from he mean of he enire ensemble, and r (j) measures deviaion of he j h ensemble member from he ruh u underlying he daa. The marices E, R and F conain informaion abou hese deviaion quaniies, when mapped forward under he operaor A. The following heorem esablishes exisence and uniqueness of soluions o (8). Theorem 2. Assume ha y is he image of a ruh u X under A. Le u (j) () X for j = 1,..., J and define X o be he linear span of he {u (j) ()} J j=1. Then equaion (8) has a unique soluion u (j) ( ) C([, ); X ) for j = 1,..., J. (11) Proof. I follows from (8) ha du (j) d = 1 J F jk e (k) The righ-hand side of equaion (9) is locally Lipschiz in u as a mapping from X o X. Thus local exisence of a soluion in C([, T ); X ) holds for some T >, since X is finie dimensional. Thus i suffices o show ha he soluion does no blow-up in finie ime. To his end we prove in Lemma 7, which is presened in he Appendix 7, ha he marices E() and F () are bounded by a consan depending on iniial condiions, bu no on ime. Using he global bound on F i follows from (11) ha u is globally bounded by a consan depending on iniial condiions, and growing exponenially wih. Global exisence for u follows and he heorem is complee. The following heorem shows ha all ensemble members collapse owards heir mean value a an algebraic rae; and i demonsraes ha he collapse slows down linearly as ensemble size increases. Theorem 3. Assume ha y is he image of a ruh u X under A. Le u (j) () X for j = 1,..., J. Then he marix valued quaniy E() converges o for and, indeed E() = O(J 1 ). Proof. Lemma 7, which is presened in he Appendix 7, shows ha he quaniy E() saisfies he differenial equaion d d E = 2 J E2 wih iniial condiion E() = XΛ()X, Λ() = {λ (1),..., λ(j) } and orhogonal X. Using he eigensysem X gives he soluion E() = XΛ()X, where he enries of he diagonal marix Λ() are given by ( 2 J + 1 ) 1 if λ (j) λ (j) and oherwise. Then, he claim immediaely follows from he explici form of he soluion E().

8 8 C. SCHILLINGS AND A. M. STUART We are now ineresed in he long-ime behavior of he residuals wih respec o he ruh. Theorem 4 characerizes he relaion beween he approximaion qualiy of he iniial ensemble and he convergence behavior of he residuals. Theorem 4. Assume ha y is he image of a ruh u X under A and he forward operaor A is one-o-one. Le Y denoe he linear span of he {Ae (j) ()} J j=1 and le Y denoe he orhogonal complemen of Y in Y wih respec o he inner produc, Γ and assume ha he iniial ensemble members are chosen so ha Y has he maximal dimension min{j 1, dim(y)}. Then Ar (j) () may be decomposed uniquely as Ar (j) () + Ar(j) all j {1,, J}, Ar (j) Ar (j) () = Ar(j) () = Ar(1) (). () wih Ar(j) Y and Ar (j) Y. Furhermore, for () as and, for all j {1,, J} and, Proof. Lemma 8, which is presened in he Appendix 7, shows ha he marix L, he linear ransformaion which deermines how o wrie {Ae (j) ()} J j=1 in erms of he coordinaes {Ae (j) ()} J j=1, is inverible for all and hence ha he linear span of he {Ae (j) ()} J j=1 is equal o Y for all. Lemma 8 also shows ha Ar (j) () may be decomposed uniquely as Ar (j) () + Ar(j) () wih Ar(j) Y and Ar (j) Y and ha Ar (j) () = Ar(j) () for all. I hus remains o show ha Ar(j) () converges o zero as. From Lemma 8 we know ha we may wrie (12) Ar (j) () = α k Ae (k) () + Ar (1), for some (j-dependen) coefficiens α = (α 1,..., α J ) T R J. Furhermore Lemma 8 shows ha if we iniially choose α o be orhogonal o he eigenvecors x (k), k = 1,..., J J of E wih corresponding eigenvalues λ (k) () =, k = 1,..., J J, hen his propery will be preserved for all ime. This is since we have QL 1 x (k) = Qx (k) =, k = 1,..., J J and he marix Υ wih j h row given by α saisfies Q = ΥL so ha Υx (k) =, k = 1,..., J J. Finally we choose coordinaes in which Ar (j) () is orhogonal o Y. Define he seminorms α 2 1 := E 1/2 α 2 α 2 2 := Eα 2. Noe ha ha hese norms are ime-dependen because E is. On he subspace of R J orhogonal o span{x (1),..., x (J J) }, where λ min () = ( 2 J + 1 λ min α 2 2 λ min () α 2 1, Furhermore, for he Euclidean norm, we have ) 1 is he minimal posiive eigenvalue of E, see (25). α 2 1 λ min () α 2 on he subspace of R J orhogonal o span{x (1),..., x (J J) }.

9 ANALYSIS OF THE ENSEMBLE KALMAN FILTER FOR INVERSE PROBLEMS 9 Now noe ha he following differenial equaion holds for he quaniy Ar (j) 2 Γ : We also have 1 d 2 d Ar(j) 2 Γ = 1 J Ar (j), Ae (r) 2 Γ = r=1 = Ar (j), Ae (r) Γ Ar (j), Ae (r) Γ. r=1 α k Ae (k), Ae (r) Γ α l Ae (l), Ae (r) Γ r=1 ( α k l=1 r=1 = α 2 2. l=1 E kr E lr ) α l Using (12), he norm of he residuals can be expressed in erms of he coefficien vecor of he residuals as follows: Ar (j) 2 Γ = = l=1 α k Ae (k) + Ar (j) (), l=1 α k E kl α l + Ar (j) ()2 Γ = α Ar (j) ()2 Γ. α l Ae (l) + Ar (j) () Γ Thus, he coefficien vecor saisfies he following differenial equaion which gives 1 d 2 d α 2 1 = 1 J α 2 2 λ min() α 2 1, J 1 α 2 1 d d α ( 2 J J + 1 λ min Hence, using ha ln α() 2 1 ln α() 2 1 ln 1 λ min vecor can be bounded by In he Euclidean norm, we have and hus α() λ min λ min () α() 2 1 α() 2 1 λ min ln α() 2 1λ min (). λ min α() 2 1 λ(j) ) 1 α() 2 1λ min () λ min ( ) 2 J + 1, he coefficien λ min α() 2. Since α is bounded in ime, and since he e (k) by Lemma 8, he desired resul follows from (12).

10 1 C. SCHILLINGS AND A. M. STUART Corollary 5 generalizes he convergence resuls of he preceding heorem o he infinie dimensional seing, i.e. o he case dim Y = under he addiional assumpion ha he forward operaor A is boundedly inverible. This implies ha A canno be a compac operaor. However, his resul allows o use he EnKF as a linear solver, since he convergence resuls can be ransferred o he sae space under he sricer assumpions on A. Corollary 5. Assume ha y is he image of a ruh u X under A, he forward operaor A is boundedly inverible and he iniial ensemble is chosen such ha he subspace X = span{e (1),..., e (J) } has maximal dimension J 1. Then, here exiss a unique decomposiion of he residual r (j) () = r (j) () + r(j) () wih r(j) X and r (j) X 1, where X = X X1. The wo subspaces X and X 1 are orhogonal wih respec o he inner produc, X := Γ 1 2 A, Γ 1 2 A. Then, r (j) () as and r (j) () = r(j) () = r(1) (). Proof. The assumpion ha he forward operaor is boundedly inverible ensures ha he range of he operaor is closed. Furhermore, he inveribiliy of he operaor A allows o ransfer resuls from he observaional space direcly o he sae space. Thus, he same argumens as in he proof of Theorem 4 prove he claim Noisy Observaional Daa. Very similar analyses o hose in he previous subsecion may be carried ou in he case where he observaional daa y is pollued by addiive noise η R K : y = Au + η. Global exisence of soluions, and ensemble collapse, follow similarly o he proofs of Theorems 2 and 3. Theorem 4 is more complex o generalize since i is no he mapped residual Ar (j) which is decomposed ino a space where i decays o zero and a space where i remains consan, bu raher he quaniy ϑ (j) = Ar (j) η. Driving his quaniy o zero of course leads o over-fiing. Furhermore, he generalizaion o he infinie dimensional seing as presened in Corollary 5 is no longer valid, since he noise may ake he daa ou of he range of he forward operaor. 5. Numerical Resuls. In his secion we presen numerical experimens boh wih and wihou daa, illusraing he heory of he previous secion for he linear inverse problem. We also sudy a nonlinear groundwaer flow inverse problem, demonsraing ha he heory in he linear problem provides useful insigh for he nonlinear problem Linear Forward Model. We consider he one dimensional ellipic equaion (13) d2 p + p = u in D := (, π), p = in D, dx2 where he uncerainy-o-observaion operaor is given by G = O G = O A 1 wih A = d2 dx + id and D(A) = H 2 (I) H 1 2. The observaion operaor O consiss of K = sysem responses a K equispaced observaion poins a x k = k 2, k = 4 1,..., 2 4 1, o k ( ) := δ( x k ), i.e. (O( )) k = o k ( ). The forward problem (13) is solved numerically by a FEM using coninuous, piecewise linear ansaz funcions on a uniform mesh wih meshwidh h = 2 8 (he spaial discreizaion leads o a discreizaion of u, i.e. u R 28 1 ).

11 1 spread e (k) 2 ANALYSIS OF THE ENSEMBLE KALMAN FILTER FOR INVERSE PROBLEMS 11 The goal of he compuaion is o recover he unknown daa u from noisy observaions (14) y = p + η = OA 1 u + η. The measuremen noise is chosen o be normally disribued, η N (, γi), γ R, γ >, I R K K. The iniial ensemble of paricles is chosen o be based on he eigenvalues and eigenfuncions {λ j, z j } j N of he covariance operaor C. Here C = β(a id) 1, β = 1. Alhough we do no pursue he Bayesian inerpreaion of he EnKF, he reader ineresed in he Bayesian perspecive may hink of he prior as being µ = N(, C ), which is a Brownian bridge. In all our experimens we se u (j) () = λ j ζ j z j wih ζ j N (, 1) for j = 1,..., J. Thus he j h elemen of he iniial ensemble may be viewed as he j h erm in a Karhunen-Loève (KL) expansion of a draw from µ which, in his case, is a Fourier sine series expansion Noise-Free Observaional Daa. To numerically verify he heoreical resuls presened in secion 4.1, we firs resric he discussion o he noise-free case, i.e. we assume ha η = in (14) and se Γ = id. The sudy summarized in Figures 1-4 shows he influence of he number of paricles on he dynamical behavior of he quaniies e and r, he marix-valued quaniies and he resuling EnKF esimae /J Σ e (k) 2 2 J=5 spread e (k) 2 2 J=1 1/J Σ e (k) 2 2 J=1 spread e (k) 2 2 J=5 1/J Σ e (k) 2 2 J=5 1 spread Ae (k) Γ 1/J Σ Ae (k) 2 Γ J=5 spread Ae (k) 2 Γ J=1 1/J Σ Ae (k) 2 Γ J=1 spread Ae (k) 2 Γ J=5 1/J Σ Ae (k) 2 Γ J= Fig. 1. Quaniies e 2 2, Ae 2 Γ w.r. o ime, J = 5 (red), J = 1 (blue) and J = 5 (green), β = 1, K = 2 4 1, iniial ensemble chosen based on KL expansion of C = β(a id) 1. As shown in Theorem 3, he rae of convergence of he ensemble collapse is algebraic (cf. Figure 1) wih a consan growing wih larger ensemble size. Comparing he dynamical behavior of he residuals, we observe ha, for he ensemble of size J=5, he esimae can be improved in he beginning, bu reaches a plaeau afer a shor ime. Increasing he number of paricles o J = 1 improves he accuracy of he esimae. For he ensemble size J = 5, Figure 2 shows he convergence of he projeced residuals, i.e. he observaions can be perfecly recovered. The same behavior can be observed by comparing he EnKF esimae wih he ruh and he observaional daa (cf. Figure 4).

12 1 spread r (k) 2 12 C. SCHILLINGS AND A. M. STUART /J Σ r (k) 2 2 J=5 spread r (k) 2 J=1 2 1/J Σ r (k) 2 J=1 2 spread r (k) 2 1/J Σ r (k) 2 1 spread Ar (k) Γ 1/J Σ Ar (k) 2 Γ J=5 spread Ar (k) Γ 2 J=1 1/J Σ Ar (k) Γ 2 J=1 spread Ar (k) Γ 1/J Σ Ar (k) Γ Fig. 2. Quaniies r 2 2, Ar 2 Γ w.r. o ime, J = 5 (red), J = 1 (blue) and J = 5 (green), β = 1, K = 2 4 1, iniial ensemble chosen based on KL expansion of C = β(a id) 1. The resuls derived in his paper hold rue for each paricle, however, for he sake of presenaion, he empirical mean of he quaniies of ineres is shown and he spread indicaes he minimum and maximum deviaions of he ensemble members from he empirical mean. 1 2 R F J=5 1 1 E F J=5 F F J=5 1 R F J=1 E F J=1 F 1-1 F J=1 R F J=5 E 1-2 F J=5 F F J= uruh EnKF es. J=5 EnKF es. J=1 EnKF es. J= observaions EnKF es. J=5 EnKF es. J=1 EnKF es. J= Fig. 3. Quaniies E F, F F, R F w.r. o ime, J = 5 (red), J = 1 (blue) and J = 5 (green), β = 1, K = 2 4 1, iniial ensemble chosen based on KL expansion of C = β(a id) 1. Comparison of he EnKF Fig. 4. Comparison of he EnKF esimae wih he ruh and he observaions, J = 5 (red), J = 1 (blue) and J = 5 (green), β = 1, K = 2 4 1, iniial ensemble chosen based on KL expansion of C = β(a id) 1. Due o he consrucion of he ensembles in he example, he subspace spanned by he ensemble of size 5 is a sric subse of he subspace spanned by he larger ensembles. Thus, due o Theorem 4, which characerizes he convergence of he residuals wih respec o he approximaion qualiy of he subspace spanned by he iniial ensemble, he EnKF esimae can be subsanially improved by conrolling his subspace. As illusraed in Figure 2, he mapped residual of he ensemble of size 5 decreases

13 ANALYSIS OF THE ENSEMBLE KALMAN FILTER FOR INVERSE PROBLEMS 13 monoonically, bu levels off afer a shor ime. Similar convergence properies can be observed for J = 1. The same behavior is expeced for he larger ensemble, when inegraing over a larger ime horizon. This can be also observed for he marix-valued quaniies depiced in Figure 3. We will invesigae his poin furher by comparing he performance of wo ensembles, boh of size 5: one based on he KL expansion and one chosen such ha he conribuion of Ar (j) () in Theorem 4 is minimized. Since we use arificial daa, we can minimize he conribuion of Ar (j) () by ensuring ha Ar(1) = J α kae (k) for some coefficiens α k R. Given u (2),..., u (J) and coefficiens α 1,..., α J, we define u (1) = (1 α 1 + J α k/j) 1 (u α 1 /J J j=2 u(j) + J k=2 α ku (k) α k /J J j=2 u(j) ), which gives he desired propery of he ensemble. Noe ha his approach is no feasible in pracice and has o be replaced by an adapive sraegy minimizing he conribuion of Ar (j) (). However his experimen serves o illusrae he imporan role of he iniial ensemble in deermining he error and is included for his reason. The convergence rae of he mapped residuals and of he ensemble collapse is algebraic in boh cases, wih rae 1 (in he squared Euclidean norm). Figure 5 shows he convergence of he projeced residuals for he adapively chosen ensemble. The decomposiion of he residuals (cf. Figure 6) numerically verifies he presened heory, which moivaes he adapive consrucion of he ensemble. Mehods o realize his sraegy, in he linear and nonlinear case, will be addressed in a subsequen paper. 1 2 spread r k /J Σ r k 2 spread r k 2 ruh 1/J Σ r k 2 ruh spread Ar k 1-5 II 2 Γ J=5 1/J Σ Ar k II Γ spread Ar k ruh II Γ 1/J Σ Ar k ruh 1-1 II Γ spread Ar k Γ 1/J Σ Ar k Γ spread Ar k Γ ruh 1/J Σ Ar k Γ ruh /J Σ Ar k 2 Γ J=5 1/J Σ Ar k ruh Γ Fig. 5. Quaniies r 2 2, Ar 2 Γ w.r. o ime, J = 5 based on KL expansion of C = β(a id) 1 (red) and J = 5 minimizing he conribuion of Ar (j) () (blue), β = 1, K = Comparison of he EnKF esimae Fig. 6. Quaniies Ar (j) 2 Γ, Ar(j) 2 Γ w.r. o ime, J = 5 based on KL expansion of C = β(a id) 1 (red) and J = 5 minimizing he conribuion of Ar (j) () (blue), β = 1, K = Noisy Observaional Daa. We will now allow for noise in he observaional daa, i.e. we assume ha he daa is given by y = OA 1 u + η, where η is a fixed realizaion of he random vecor η N (,.1 2 id). Noe ha he sandard deviaion is chosen o be roughly 1% of he (maximum of he) observed daa. Besides he quaniies e and r, he misfi ϑ (j) = Au (j) y = Ar (j) η of each ensemble member is of ineres, since, in pracice, he residual is no accessible and he misfi is used o check for convergence and o design an appropriae sopping crierion. Besides he wo ensembles wih 5 paricles inroduced in he previous secion, we define an addiional one minimizing he conribuion of ϑ (j) o he misfi, analogously o wha was done in he adapive iniializaion in he previous subsecion, and mo-

14 14 C. SCHILLINGS AND A. M. STUART ivaed by he analogue of Theorem 4 in he noisy daa case. Noe ha he design of an adapive ensemble based on he decomposiion of he projeced residual is, in general, no feasible wihou explici knowledge of he noise η spread r k 2 1/J Σ r k 2 spread r k ruh 2 1/J Σ r k 2 J=5 ruh spread Ar k Γ 1/J Σ Ar k Γ spread Ar k Γ ruh 1/J Σ Ar k Γ ruh Fig. 7. Quaniies r 2 2, Ar 2 Γ w.r. o, J = 5 based on KL expansion of C = β(a id) 1 (red), J = 5 adapively chosen (blue), β = 1, K = 2 4 1, η N (,.1 2 id) spread ϑ Γ 1/J Σ ϑ Γ spread ϑ Γ ruh 1/J Σ ϑ Γ ruh Fig. 8. Misfi ϑ 2 2 w.r. o ime, J = 5 based on KL expansion of C = β(a id) 1 (red), J = 5 adapively chosen (blue), β = 1, K = 2 4 1, η N (,.1 2 id). Figure 7 illusraes he well-known overfiing effec, which arises wihou using appropriae sopping crieria. The mehod ries o fi he noise in he measuremens, which resuls in an increase in he residuals. This effec is no seen in he misfi funcional, cf. Figure 8 and Figure uruh EnKF es. J=5 EnKF es. J=5 ruh /J Σ ϑ II Γ 1/J Σ ϑ II Γ ruh /J Σ ϑ Γ 1/J Σ ϑ Γ ruh Fig. 9. Quaniies ϑ (j) 2 Γ, ϑ(j) 2 Γ w.r. o ime, J = 5 based on KL expansion of C = β(a id) 1 (red), J = 5 minimizing he conribuion of Ar (j) () (blue), β = 1, K = 2 4 1, η N (,.1 2 id)..6.4 observaions.2 EnKF es. J=5 EnKF es. J=5 ruh Fig. 1. Comparison of he EnKF esimae wih he ruh and he observaions, J = 5 based on KL expansion of C = β(a id) 1 (red), J = 5 minimizing he conribuion of Ar (j) () (blue), β = 1, K = 24 1, η N (,.1 2 id). However, he comparison of he EnKF esimaes o he ruh reveals, in Figure 1, he srong overfiing effec and suggess he need for a sopping crierion. The Bayesian seing iself provides a so-called a priori sopping rule, i.e. he SMC viewpoin moivaes a sopping of he ieraions a ime T = 1. Anoher common choice in he deerminisic opimizaion seing is he discrepancy principle, which accouns for he realizaion of he noise by checking he following condiion G(ū()) y Γ τ, where τ > depends on he dimension of he observaional space.

15 ANALYSIS OF THE ENSEMBLE KALMAN FILTER FOR INVERSE PROBLEMS /J Σ r k 2 1/J Σ r k ruh uruh -.2 EnKF es. J=5 EnKF es. J=5 ruh /J Σ Ar k Γ 1/J Σ Ar k Γ ruh Fig. 11. Quaniies r 2 2, Ar 2 Γ w.r. o ime, J = 5 based on KL expansion of C = β(a id) 1 (red), J = 5 minimizing he conribuion of Ar (j) () (blue), β = 1, K = 2 4 1, η N (,.1 2 id), Bayesian sopping rule.j = 5 minimizing he conribuion of Ar (j) () (.4 observaions.2 EnKF es. J=5 EnKF es. J=5 ruh Fig. 12. Comparison of he EnKF esimae wih he ruh and he observaions, J = 5 based on KL expansion of C = β(a id) 1 (red), J = 5 minimizing he conribuion of Ar (j) () (blue), β = 1, K = 24 1, η N (,.1 2 id), Bayesian sopping rule. Figures show he resuls obained by employing he Bayesian sopping rule, i.e. by inegraing up o ime T = 1. The adapively chosen ensemble leads o much beer resuls in he case of he Bayesian sopping rule. Since we do no expec o have explici knowledge of he noise, he adapive sraegy as presened above is in general no feasible. However an adapive choice of he ensemble according o he misfi may lead o a srong overfiing effec, as shown in Figures spread ϑ Γ 1/J Σ ϑ Γ spread ϑ Γ ruh 1/J Σ ϑ Γ ruh spread ϑ Γ ruh mis. 1/J Σ ϑ Γ ruh mis Fig. 13. Misfi ϑ 2 Γ w.r. o ime, J = 5 based on KL expansion of C = β(a id) 1 (red), J = 5 adapively chosen (blue), J = 5 minimizing he conribuion of ϑ w.r. o misfi (gray), β = 1, K = 2 4 1, η N (,.1 2 id) uruh EnKF es. J=5 EnKF es. J=5 ruh EnKF es. J=5 ruh mis observaions EnKF es. J=5 EnKF es. J=5 ruh EnKF es. J=5 ruh mis Fig. 14. Comparison of he EnKF esimae wih he ruh and he observaions, J = 5 based on KL expansion of C = β(a id) 1 (red), J = 5 adapively chosen (blue), J = 5 minimizing he conribuion of ϑ w.r. o misfi (gray), β = 1, K = 2 4 1, η N (,.1 2 id).

16 16 C. SCHILLINGS AND A. M. STUART The overfiing effec is sill presen in he small noise regime as shown below. The sandard deviaion of he noise is reduced by 1, i.e. η N (,.1 2 id) uruh EnKF es. J=5 EnKF es. J=5 ruh EnKF es. J=5 ruh mis spread ϑ Γ 1/J Σ ϑ Γ spread ϑ Γ ruh 1/J Σ ϑ Γ ruh spread ϑ Γ ruh mis. 1/J Σ ϑ Γ ruh mis Fig. 15. Misfi ϑ 2 Γ w.r. o ime, J = 5 based on KL expansion of C = β(a id) 1 (red), J = 5 adapively chosen (blue), J = 5 minimizing he conribuion of ϑ w.r. o misfi (gray), β = 1, K = 2 4 1, η N (,.1 2 id) observaions EnKF es. J=5 EnKF es. J=5 ruh EnKF es. J=5 ruh mis Fig. 16. Comparison of he EnKF esimae wih he ruh and he observaions, J = 5 based on KL expansion of C = β(a id) 1 (red), J = 5 adapively chosen (blue), J = 5 minimizing he conribuion of ϑ w.r. o misfi (gray), β = 1, K = 2 4 1, η N (,.1 2 id). The ill-posedness of he problem leads o he insabiliies of he idenificaion problem and requires he use of an appropriae sopping rule Nonlinear Forward Model. To invesigae he numerical behavior of he EnKF for nonlinear inverse problems, we consider he following wo-dimensional ellipic PDE: div(e u p) = f in D := ( 1, 1) 2, p = in D. We aim o find he log permeabiliy u from 49 observaions of he soluion p on a uniform grid in D. We choose f(x) = 1 for he experimens. The mapping from u o hese observaions is now nonlinear. Again we work in he noise-free case, and ake Γ = I, Σ = and solve (6) o esimae he unknown parameers. The prior is assumed o be Gaussian wih covariance operaor C = ( ) 2, employing homogeneous Dirichle boundary condiions o define he inverse of. We use a FEM approximaion based on coninuous, piecewise linear ansaz funcions on a uniform mesh wih meshwidh h = 2 4. The iniial ensemble of size 5 and 5 is chosen based on he KL expansion of C in he same way as in he previous subsecion. The resuls given in Figure 17 and Figure 18 show a similar behavior as in he linear case. The approximaion qualiy of he subspace spanned by he iniial ensemble clearly influences, also in he nonlinear example, he accuracy of he esimae. Taking a look a he EnKF esimae in his nonlinear seing, we observe a saisfacory approximaion of he ruh and a perfec mach of he observaional daa in he case of he larger ensemble, cf. Figure 19.

17 ANALYSIS OF THE ENSEMBLE KALMAN FILTER FOR INVERSE PROBLEMS 17 1 spread e k 2 1/JΣ e k 2 spread e k /JΣ e k spread r k 2 1/JΣ r k 2 spread r k 2 1/JΣ r k spread G(u k )-1/J Σ G(u j ) Γ 1-1 1/JΣ G(u k )-1/J Σ G(u j ) Γ spread G(u k )-1/J Σ G(u j ) Γ 1/JΣ G(u k )-1/J Σ G(u j ) Γ Fig. 17. Quaniies e (k) 2 2, G(u(k) ) 1 J J j=1 G(u(j) 2 Γ w.r. o ime, J = 5 (red) and J = 5 (green), iniial ensemble chosen based on KL expansion of C = ( ) spread G(u k )-G(u + ) Γ 1/JΣ G(u k )-G(u + ) Γ spread G(u k )-G(u + ) Γ 1/JΣ G(u k )-G(u + ) Γ Fig. 18. Quaniies r (k) 2 2 G(u(k) ) G(u ) 2 Γ w.r. o ime, J = 5 (red) and J = 5 (green), iniial ensemble chosen based on KL expansion of C = ( ) 2. Fig. 19. Comparison of he ruh (lef above) and he EnKF esimae w.r. o x, J=5 (middle above), J = 5 (righ above) and comparison of he forward soluion G(u ) (lef below) and he esimaed soluions of he forward problem J = 5 (middle below), J = 5 (righ below). 6. Varians on EnKF. In his secion we describe hree varians on he EnKF, all formulaed in coninuous ime in order o faciliae comparison wih he preceding sudies of he sandard EnKF in coninuous ime. The firs wo mehods, variance inflaion and localizaion, are commonly used by praciioners in boh he filering and inverse problem scenarios [8, 1]. The hird mehod, random search, is moivaed by he SMC derivaion of he EnKF for inverse problems, and is new in he conex of he EnKF. For all hree mehods we provide numerical resuls which illusrae he behavior of he EnKF varian, in comparison wih he sandard mehod. In he following, we focus on he linear case wih Σ =. The mehods from he firs wo subsecions have generalizaions in he general nonlinear seing, and indeed are widely used in daa assimilaion and, o some exen, in geophysical inverse problems; bu we presen hem in a form ied o he equaion (9) which is derived in he linear case.

18 18 C. SCHILLINGS AND A. M. STUART The mehod in he final subsecion is implemened hrough an enirely derivaive-free MCMC mehod, and is hence auomaically defined, as is, for nonlinear as well as linear inverse problems Variance Inflaion. The empirical covariances C up and C pp all have rank no greaer han J 1 and hence are rank deficien whenever he number of paricles J is less han he dimension of he space X. Variance inflaion proceeds by correcing such rank deficiencies by he addiion of self-adjoin, sricly posiive operaors. A naural variance inflaion echnique is o add a muliple of he prior covariance C o he empirical covariance which gives rise o he equaions (15) du (j) d = ( αc + C(u) ) D u Φ(u (j) ; y), j = 1,..., J, where Φ is as defined in (1). Taking he inner-produc in X wih D u Φ(u (j) ; y) we deduce ha dφ(u (j) ; y) d αc 1 2 D u Φ(u (j) ; y) 2, This implies ha all ω limi poins of he dynamics are conained in he criical poins of Φ( ; y) Localizaion. Localizaion echniques aim o remove spurious long disance correlaions by modifying he covariance operaors C up and C pp, or direcly he Kalman gain C up n+1 (Cpp n+1 + h 1 Γ) 1. Typical convoluion kernels reducing he influence of disan regions are of he form ρ : D D R ρ(x, y) = exp( x y r ), where D R d, d N denoes he physical domain and is a suiable norm in D, r N, cf. [21]. The coninuous ime limi in he linear seing hen reads as (16) du (j) d = C loc (u)d u Φ(u (j) ; y), j = 1,..., J, where C loc (u)φ(x) = φ(y)k(x, y)ρ(x, y)dy wih k denoing he kernel of C(u) and D φ X Randomized Search. We noice ha he mapping on probabiliy measures given by (3) may be replaced by µ n+1 = L n P n µ n. where P n is any Markov kernel which preserves µ n. For example we may ake P n o be he pcn mehod [6] for measure µ n. One sep of he pcn mehod for given paricle u (j) n in ieraion n is realized by Propose v n (j) = (1 β 2 )u (j) n + βι (j), ι (j) N (, C ). Se ũ (j) n = v n (j) wih probabiliy a(u (j) n, v n (j) ). Se ũ (j) n = u (j) n oherwise assuming he prior is Gaussian, i.e. N (, C ). The accepance probabiliy is given by a(u (j) n, v (j) n ) = min{1, exp(nhφ(u (j) n ) nhφ(v n (j) ))}.

19 ANALYSIS OF THE ENSEMBLE KALMAN FILTER FOR INVERSE PROBLEMS 19 The paricles ũ (j) n are used o approximae he measure µ n = P n µ n, which is hen mapped o µ n+1 by he applicaion of Bayes heorem, i.e. µ n+1 = L n µ n. Using he coninuous-ime diffusion limi argumens from [26, Theorem 4], which apply in he nonlinear case, and combining wih he coninuous ime limis described for he EnKF earlier in his paper, we obain (17) du (j) d = 1 J G(u (k) ) G, y G(u (j) ) Γ( u (k) u ) u (j) C D u Φ(u (j) ; y) + dw (j) 2C. d Alhough he limiing equaion involves gradiens of Φ, and hence adjoins for he forward model, he discree ime implemenaion above avoids he gradien compuaion by using he accep-rejec sep, and remains a derivaive free opimizer Numerical Resuls. In he following, o illusrae behavior of he EnKF varians, we presen numerical experimens for he linear forward problem in he noise-free case: (13) and (14) wih η =. The performance of he EnKF varians is compared o he basic algorihms shown in Figure 2 and Figure Inflaion. We invesigae he numerical behavior of variance inflaion of he form given in (15) wih α =.1. Figures 2 and 21 show ha he variance inflaed mehod becomes a precondiioned gradien flow, which, in he linear case, leads o fas convergence of he projeced ieraes. I is noeworhy ha in his case here is very lile difference in behavior beween ensemble sizes of 5 and spread r k VI /J Σ r k 2 2 J=5 VI 1 spread r k 2 VI 1-1 1/J Σ r k 2 2 J=5 VI uruh EnKF es. J=5 VI EnKF es. J=5 VI EnKF es. J=5 orig. EnKF es. J=5 orig spread Ar k VI Γ 1-2 1/J Σ Ar k 2 Γ J=5 VI spread Ar k 2 Γ J=5 VI 1-4 1/J Σ Ar k Γ observaions EnKF es. J=5 VI EnKF es. J=5 VI Fig. 2. Quaniies r 2 2, Ar 2 Γ w.r. o ime, J = 5 wih variance inflaion (red) and J = 5 wih variance inflaion (green), β = 1, K = 2 4 1, iniial ensemble chosen based on KL expansion of C = β(a id) 1. Comparison of he EnKF Fig. 21. Comparison of he EnKF esimae wih he ruh and he observaions, J = 5 wih variance inflaion (red) and J = 5 wih variance inflaion (green), β = 1, K = 2 4 1, iniial ensemble chosen based on KL expansion of C = β(a id) Localizaion. We consider a localizaion of he form given by equaions (16), (16) wih r = 2 and Euclidean norm inside he cu-off kernel. Figures clearly demonsrae he improvemen by he localizaion echnique, which can overcome he linear span propery and hus, leads o beer esimaes of he ruh.

20 2 C. SCHILLINGS AND A. M. STUART 1 2 spread r k 2 loc /J Σ r k 2 loc spread r k 2 loc 1/J Σ r k 2 loc uruh EnKF es. J=5 loc EnKF es. J=5 loc EnKF es. J=5 orig. EnKF es. J=5 orig spread Ar k Γ loc /J Σ Ar k Γ loc spread Ar k Γ loc 1/J Σ Ar k Γ observaions EnKF es. J=5 loc EnKF es. J=5 loc Fig. 22. Quaniies r 2 2, Ar 2 Γ w.r. o ime, J = 5 wih localizaion (red) and J = 5 wih localizaion (green), β = 1, K = 2 4 1, iniial ensemble chosen based on KL expansion of C = β(a id) 1. Comparison of he EnKF Fig. 23. Comparison of he EnKF esimae wih he ruh and he observaions, J = 5 wih localizaion (red) and J = 5 wih localizaion (green), β = 1, K = 2 4 1, iniial ensemble chosen based on KL expansion of C = β(a id) Randomized Search. We invesigae he behavior of randomized search for he linear problem wih G( ) = A. For he numerical soluion of he coninuous limi (17), we employ a spliing scheme wih a linearly implici Euler sep, namely ũ (j) n+1 = 1 2hu (j) n + 2hC ζ n Ku (j) n+1 = ũ(j) n+1 + h(c(ũ n+1)a Γ 1 y + nhc A Γ 1 y ), where ζ n N(, id) and K := I + h(c(ũ n+1 )A Γ 1 A + nhc A Γ 1 A). In all numerical experimens repored we ake h = 2 8. Figure 24 and Figure 25 show ha he randomized search leads o an improved performance compared o he original EnKF mehod. Due o he fixed sep size and he resuling high compuaional coss, he soluion is compued up o ime T = 1. In order o accelerae he numerical soluion of he limi (17), implici schemes can be considered. Noe ha he limi requires he compuaion of he gradiens, which is in pracice undesirable. However, he limi reveals from a heoreical poin of view imporan srucure, whereas he discree version is more suiable for applicaions. The advanage of he randomized search is apparen Summary. The experimens show a similar performance for all discussed varians. The variance inflaion echnique and he localizaion varian boh lead o gradien flows, which are, in he noise-free case, favorable due o he fas convergence. On he oher hand, hese sraegies also accelerae he convergence of he ensemble o he mean (ensemble collapse) and may be considered less desirable for his reason. The randomized search preserves by consrucion he spread of he ensemble. A similar regularizaion effec is achieved by perurbing he observaional daa, see (6). The varians all break he subspace propery of he original version, which resuls in an improvemen in he esimae.

21 ANALYSIS OF THE ENSEMBLE KALMAN FILTER FOR INVERSE PROBLEMS spread r k MM 1/J Σ r k 2 MM 1 spread r k 2 MM 1-1 1/J Σ r k 2 MM uruh EnKF es. J=5 MM EnKF es. J=5 MM EnKF es. J=5 orig. T=1 EnKF es. J=5 orig. T= spread Ar k Γ MM /J Σ Ar k Γ MM spread Ar k Γ MM 1/J Σ Ar k Γ observaions EnKF es. J=5 MM EnKF es. J=5 MM Fig. 24. Quaniies r 2 2, Ar 2 Γ w.r. o ime, J = 5 wih randomized search (red) and J = 5 wih randomized search (green), β = 1, K = 2 4 1, iniial ensemble chosen based on KL expansion of C = β(a id) 1. Comparison of he EnKF Fig. 25. Comparison of he EnKF esimae wih he ruh and he observaions, J = 5 wih randomized search (red) and J = 5 wih randomized search (green), β = 1, K = 2 4 1, iniial ensemble chosen based on KL expansion of C = β(a id) 1. We sudy he same es case as in secion 5.1.2, wih he same realizaion of he measuremen noise (cf. Figure 11 and Figure 12), o allow for a comparison of he hree mehods inroduced in his secion. Combining he various echniques wih he Bayesian sopping rule for noisy observaions, we observe he following behavior given in Figure 26 and Figure /J Σ r k 2 VI 1/J Σ r k 2 loc 1/J Σ r k 2 MM /J Σ Ar k Γ VI 1/J Σ Ar k Γ loc 1/J Σ Ar k Γ MM Fig. 26. Quaniies r 2 2, Ar 2 Γ w.r. o ime, J = 5 (red) for he discussed varians, β = 1, β = 1, K = 2 4 1, iniial ensemble chosen based on KL expansion of C = β(a id) 1. In boh figures, he abbreviaion VI refers o variance inflaion, loc denoes he localizaion echnique and MM sands for he randomized search (Markov mixing). The randomized search clearly ouperforms he wo oher sraegies and leads o a