Finite-size ensemble Kalman filters (EnKF-N)

Transcription

1 Finite-size ensemble Kalman filters (EnKF-N) Marc Bocquet (1,2), Pavel Sakov (3,4) (1) Université Paris-Est, CEREA, joint lab École des Ponts ParisTech and EdF R&D, France (2) INRIA, Paris-Rocquencourt Research center, France (3) BOM, Melbourne, Australia (4) NERSC, Bergen, Norway M. Bocquet EnKF workshop, Albany NY, USA, May / 23

2 Outline The primal EnKF-N 1 The primal EnKF-N 2 The dual EnKF-N 3 Inflation-free iterative ensemble Kalman filters (IEnKF-N) 4 Conclusions M. Bocquet EnKF workshop, Albany NY, USA, May / 23

3 The primal EnKF-N Principle of the EnKF-N Empirical moments of the ensemble: x= 1 N N x k, P= 1 n=1 N 1 N k=1 (x k x)(x k x) T, (1) The prior of EnKF and the prior of EnKF-N: { p(x x,p) exp 1 } 2 (x x)t P 1 (x x) p(x x 1,x 2,...,x N ) (x x)(x x) T + ε N (N 1)P N 2, (2) with ε N =1 (mean-trusting variant), or ε N =1+ 1 N (original variant). Ensemble space decomposition (ETKF version of the filters): x=x+aw. The variational principle of the analysis: J(w)= 1 2 (y H(x+Aw))T R 1 (y H(x+Aw))+ N 1 w T w 2 J(w)= 1 2 (y H(x+Aw))T R 1 (y H(x+Aw))+ N ) (ε 2 ln N +w T w. (3) M. Bocquet EnKF workshop, Albany NY, USA, May / 23

4 The primal EnKF-N EnKF-N: algorithm 1 Requires: The forecast ensemble {x k } k=1,...,n, the observations y, and error covariance matrix R 2 Compute the mean x and the anomalies A from {x k } k=1,...,n. 3 Compute Y =HA, δ =y Hx 4 Find the minimum: { ( )} w a =min (δ Yw) T R 1 (δ Yw)+Nln ε w N +w T w ( ) 1 5 Compute Ω a = Y T R 1 Y+N (ε N+waw T a)i N 2w a wa T (ε N +waw T a ) 2 6 Compute x a =x+aw a. 7 Compute W a =((N 1)Ω a ) 1/2 U 8 Compute x a k =xa +AW a k M. Bocquet EnKF workshop, Albany NY, USA, May / 23

5 The primal EnKF-N Application to the Lorenz 63 model Lorenz 63 toy-model: analysis rmse versus ensemble size for t = 0.10, 0.25, Average analysis rmse ETKF-N ε N =1+1/N t=0.50 ETKF-N ε N =1+1/N t=0.25 ETKF-N ε N =1+1/N t=0.10 ETKF-N ε N =1 t=0.50 ETKF-N ε N =1 t=0.25 ETKF-N ε N =1 t = 0.10 ETKF opt. inflation t=0.50 ETKF opt. inflation t=0.25 ETKF opt. inflation t= Ensemble size M. Bocquet EnKF workshop, Albany NY, USA, May / 23

6 The primal EnKF-N Application to the Lorenz 95 model EnKF-N: analysis rmse versus ensemble size, for t = Average analysis rmse ETKF (optimal inflation) ETKF-N ε N =1+1/N ETKF-N ε Ν = Ensemble size M. Bocquet EnKF workshop, Albany NY, USA, May / 23

7 The primal EnKF-N Application to the Lorenz 95 model Local version: LETKF-N, with N = 10 (in the rank-deficient regime) LETKF (optimal inflation and localization) LETKF-N (optimal localization) Average analysis rmse Time interval between updates t M. Bocquet EnKF workshop, Albany NY, USA, May / 23

8 The primal EnKF-N Application to Kuramato-Sivashinski model Complex 1D (toy-)model of turbulence [Kuramato et al., 1975; Sivashinski, 1977] u t + 1 u 2 2 x + 2 u 2 x + ν 4 u 4 =0. (4) x u(x) defined on an interval [0, L], with cyclic boundary conditions M. Bocquet EnKF workshop, Albany NY, USA, May / 23

9 The primal EnKF-N Application to Kuramato-Sivashinski model ( t = 3) Setup: L=32π, ν =1, M =128, t =3, T =15000, R=I. Comparison of LETKF, LETKF-N ε N =1+ N 1, and OISPF Analysis rmse OISPF with optimal stochastic noise LETKF with optimal inflation and optimisation LETKF-N with optimal inflation Ensemble size M. Bocquet EnKF workshop, Albany NY, USA, May / 23

10 Outline The dual EnKF-N 1 The primal EnKF-N 2 The dual EnKF-N 3 Inflation-free iterative ensemble Kalman filters (IEnKF-N) 4 Conclusions M. Bocquet EnKF workshop, Albany NY, USA, May / 23

11 The dual EnKF-N Lagrangian duality The primal EnKF-N cost function: J(w)= 1 2 (y H(x+Xw))T R 1 (y H(x+Xw))+ N ) (ε 2 ln N +w T w. (5) Idea: We would like to split the radial degree of freedom of w, that is w T w, from its angular degrees of freedom, that is w/ w T w. Lagrangian: L(w,ρ,ζ)= 1 2 g(w)+ 1 ) (w 2 ζ T w ρ + 1 f(ρ). (6) 2 where δ =y Hx, g(w)=(δ Yw) T R 1 (δ Yw), and f(ρ)=nln(ε N + ρ). Dual cost function defined for ζ >0 by D(ζ) = inf sup L(w,ρ,ζ) w ρ 0 = 1 2 δ T( R+Yζ 1 Y T) 1 δ + ε N ζ 2 + N 2 ln N ζ N 2. (7) M. Bocquet EnKF workshop, Albany NY, USA, May / 23

12 The dual EnKF-N Non-convex strong duality Dual and primal problems: = inf D(ζ) and Π=inf J(w). (8) ζ>0 w Strong duality result (non quadratic, non-convex case!!!): =Π. (9) Saddle point equations: ζ = df dρ (ρ )= N ε N + ρ, (10) ζ w = w g(w )= Y T R 1 (δ Yw ). (11) M. Bocquet EnKF workshop, Albany NY, USA, May / 23

13 The dual EnKF-N Illustration of the strong duality Primal and dual cost functions in the one observation case and a series of innovations M. Bocquet EnKF workshop, Albany NY, USA, May / 23

14 The dual EnKF-N The dual EnKF-N scheme 1 Requires: The forecast ensemble {x k } k=1,...,n, the observations y, and error covariance matrix R 2 Compute the mean x and the anomalies A from {x k } k=1,...,n. 3 Compute Y =HA, δ =y Hx 4 Find the minimum: ζ a = min {δ T( R+Yζ 1 Y T) 1 δ + εn ζ +Nln Nζ } N ζ ]0,N/ε N ] (12) 5 Compute Ω a = ( Y T R 1 Y+ ζ a) 1 6 Compute w a =Ω a Y T R 1 δ. 7 Compute x a =x+aw a. 8 Compute W a =((N 1)Ω a ) 1/2 U 9 Compute x a k =xa +AW a k M. Bocquet EnKF workshop, Albany NY, USA, May / 23

15 The dual EnKF-N Assets of the dual scheme Efficiently finds the global minimum of the EnKF-N cost function. Negligible additional cost as compared to the traditional EnKF. The algorithm parallels the traditional EnKF. ζ can be seen as the effective size of the ensemble. It is related to an optimal inflation of the prior. Standard inflation: x k x+λ(x k x), (13) EnKF-N forecasts the following optimal prior inflation factor λ N 1 = ζ. (14) Better stability of the dual scheme as compared to the primal scheme in demanding conditions. Geometrical explanation of why inflation is so good at counter-acting undersampling (of variances): rotational invariance in ensemble space. M. Bocquet EnKF workshop, Albany NY, USA, May / 23

16 The dual EnKF-N Illustration: statistics of ζ and optimally tuned inflation 40 Efficient rank of the prior Mean Median Tenth percentile Percentile Efficient rank from opt. inflation Lorenz 95 model, N = Time interval between updates M. Bocquet EnKF workshop, Albany NY, USA, May / 23

17 Outline Inflation-free iterative ensemble Kalman filters (IEnKF-N) 1 The primal EnKF-N 2 The dual EnKF-N 3 Inflation-free iterative ensemble Kalman filters (IEnKF-N) 4 Conclusions M. Bocquet EnKF workshop, Albany NY, USA, May / 23

18 Inflation-free iterative ensemble Kalman filters (IEnKF-N) Iterative ensemble Kalman filters With strongly nonlinear models, EnKF (as well as EnKF-N) cannot perform well, because the prior is too far off. Iterative Kalman filters Original idea: Iterative extended Kalman filter [Wishner et al., 1969; Jazwinski, 1970] More recently in the EnKF context [Gu & Oliver, 2007; Kalnay & Yang, 2010; Sakov, Oliver & Bertino, 2011, Bocquet & Sakov, 2012] Essentially a one-lag smoother. Does the job of a one-lag 4D-Var, with dynamical error covariance matrix and without the use of the TLM and adjoint! Very efficient in very nonlinear conditions if one can afford the multiple ensemble propagations (Lorenz 63 and Lorenz 95). IEnKF cost function in ensemble space: J(w) = 1 2 (y 2 H(M 1 2 (x 1 +A 1 w))) T R 1 (y 2 H(M 1 2 (x 1 +A 1 w))) (N 1)wT w, (15) and similarly for IEnKF-N. M. Bocquet EnKF workshop, Albany NY, USA, May / 23

19 Inflation-free iterative ensemble Kalman filters (IEnKF-N) Finite-size iterative ensemble Kalman filters Setup: Lorenz 95, M =40, N =40, t = , R=I. Comparison of EnKF (optimal inflation), IEnKF (bundle and transform, optimal inflation), implementation different from [Sakov et al., 2011] N=40 Analysis rmse EnKF (opt. infl.) IEnKF (bundle, opt. infl.) IEnKF (transform, opt. infl.) Time interval between updates M. Bocquet EnKF workshop, Albany NY, USA, May / 23

20 Inflation-free iterative ensemble Kalman filters (IEnKF-N) Finite-size iterative ensemble Kalman filters Setup: Lorenz 95, M =40, N =40, t = , R=I. Comparison of EnKF-N, EnKF (optimal inflation), IEnKF-N (bundle and transform), IEnKF (bundle and transform, optimal inflation) N=40 Analysis rmse EnKF-N EnKF (opt. infl.) IEnKF (bundle, opt. infl.) IEnKF-N (transform) IEnKF (transform, opt. infl.) IEnKF-N (bundle) Time interval between updates M. Bocquet EnKF workshop, Albany NY, USA, May / 23

21 Outline Conclusions 1 The primal EnKF-N 2 The dual EnKF-N 3 Inflation-free iterative ensemble Kalman filters (IEnKF-N) 4 Conclusions M. Bocquet EnKF workshop, Albany NY, USA, May / 23

22 Conclusions Conclusions A new prior for the ensemble forecast meant to be used in an EnKF analysis has been built. It takes into account sampling errors. It yields a new class of filters EnKF-N, including an ensemble transform version ETKF-N, that does not seem to require inflation supposed to account for sampling errors. Local variants (both LA and CL) available. The overall performance without inflation compares well with optimally tuned ensemble filters. Dual variant EnKF-N is an EnKF with built-in optimal inflation (accounting for sampling errors). Combination with iterative EnKF schemes is possible. Almost linear regime / very small ensemble size more problematic. Tests planned in more complex model (e.g. shallow water). M. Bocquet EnKF workshop, Albany NY, USA, May / 23

23 Conclusions References 1 Ensemble Kalman filtering without the intrinsic need for inflation, M. Bocquet, Nonlin. Processes Geophys., 18, , An iterative EnKF for strongly nonlinear systems, P. Sakov, D. Oliver, and L. Bertino, Mon. Wea. Rev., in press, Combining inflation-free and iterative ensemble Kalman filters for strongly nonlinear systems, M. Bocquet and P. Sakov, Nonlin. Processes Geophys., in press, M. Bocquet EnKF workshop, Albany NY, USA, May / 23