A new Hierarchical Bayes approach to ensemble-variational data assimilation

Transcription

1 A new Hierarchical Bayes approach to ensemble-variational data assimilation Michael Tsyrulnikov and Alexander Rakitko HydroMetCenter of Russia College Park, 20 Oct 2014 Michael Tsyrulnikov and Alexander Rakitko (HMC) A new Hierarchical Bayes approach to ensemble-variational College Park, data 20 assimilation Oct / 48

2 Outline 1 Introduction and motivation 2 Methodological problems in the existing data assimilation approaches we intend to alleviate with the new technique 3 Hierarchical Bayes: principle 4 Hierarchical Bayes EnVar 5 HB-EnVar: analysis algorithms 6 HB-EnVar: first performance results Michael Tsyrulnikov and Alexander Rakitko (HMC) A new Hierarchical Bayes approach to ensemble-variational College Park, data 20 assimilation Oct / 48

3 Introduction and motivation Michael Tsyrulnikov and Alexander Rakitko (HMC) A new Hierarchical Bayes approach to ensemble-variational College Park, data 20 assimilation Oct / 48

4 Sequential assimilation in a nutshell: setup Discrete-time observed dynamical system: To be recovered/estimated: Evolution of hidden truth: x 1, x 2, x 3,... We are given: 1 Forecast model x f k+1 = F k(x a k ) Evolution of truth: x k+1 = F k (x k ) + ε k 2 Observations y 3 Observation operator y = H k (x) + η k The goal in filtering: compute p(x k y :k ) Michael Tsyrulnikov and Alexander Rakitko (HMC) A new Hierarchical Bayes approach to ensemble-variational College Park, data 20 assimilation Oct / 48

5 Sequential assimilation in a nutshell: cycling Two-step cycling 1 Time update (forecast): from p(x k 1 y :k 1 ) to p(x k y :k 1 ) 2 Observation update (analysis): from p(x k y :k 1 ) to p(x k y :k ) In the Gaussian case (and linear forecast model), we need to update the mean m and the covariance matrix Γ. Michael Tsyrulnikov and Alexander Rakitko (HMC) A new Hierarchical Bayes approach to ensemble-variational College Park, data 20 assimilation Oct / 48

6 Sequential assimilation in a nutshell: Kalman filter Linear and Gaussian model: x k+1 = F k x k + ε k y = H k x + η k The Kalman filter equations: Primary filter, forecast: x f k = F k 1x a k 1 Primary filter, analysis: x a k = xf k + K(y H kx f k ) where K = BH (HBH + R) 1 and B Γ f k Secondary filter, forecast: Γ f k B k = F k 1 Γ a k F k 1 + Q k = (I KH)B Secondary filter, analysis: Γ a k Two problems with the KF: (1) Too expensive EnKF (2) No feedback primary filter secondary filter Michael Tsyrulnikov and Alexander Rakitko (HMC) A new Hierarchical Bayes approach to ensemble-variational College Park, data 20 assimilation Oct / 48

7 Sequential assimilation in a nutshell: EnKF In the Kalman gain, K = BH (HBH + R) 1, B is not computed but estimated from the background ensemble X e = {x e 1 xe,..., x e N xe }: B = 1 N 1 Xe {X e } the analysis increment appears to belong to the ensemble space (spanned by columns of X e, i.e. the background ensemble perturbations) only N 1 observations can be fitted, ensemble covariances are noisy. the need for localization (e.g. by dividing the domain into sub-domains, covariance tapering: B L, etc.), which is not optimized. Michael Tsyrulnikov and Alexander Rakitko (HMC) A new Hierarchical Bayes approach to ensemble-variational College Park, data 20 assimilation Oct / 48

8 Sequential assimilation in a nutshell: Var In the gain matrix K = BH (HBH + R) 1 B is not computed on-line and not estimated on-line but estimated off-line from an archive. Advantages: a static model (B 0 ) for B is observable and has normally full rank. Disadvantages: no dependence on the atmospheric flow. Michael Tsyrulnikov and Alexander Rakitko (HMC) A new Hierarchical Bayes approach to ensemble-variational College Park, data 20 assimilation Oct / 48

9 Sequential assimilation in a nutshell: EnVar As both static and flow-dependent B are imperfect, let us combine them: B := wb 0 + (1 w)b EnKF And then, again, apply the same analysis equation: x a = x b + K(y Hx b ), where K = BH (HBH + R) 1 A weakness in the EnVar: Taking a linear combination of static and ensemble covariances to specify B is simplistic and, most likely, not optimal. A problem common to all the above approaches: The analysis is optimal, provided that B is precisely known. But it isn t.. Michael Tsyrulnikov and Alexander Rakitko (HMC) A new Hierarchical Bayes approach to ensemble-variational College Park, data 20 assimilation Oct / 48

10 Summary of Motivation: Methodological problems in the existing data assimilation approaches we intend to alleviate with the new technique 1 All existing Var, EnKF, and EnVar analysis equations assume that the effective background-error covariance matrix B is exact. But this is never the case. 2 EnVar takes a linear combination of static and ensemble covariances to specify B. This is ad hoc. 3 EnKF and EnVar use an ad-hoc localization. This is not theoretically optimal. 4 In the Var, EnKF, and EnVar analysis equations, there is no intrinsic feedback from observations to background-error statistics. This requires external adaptation or manual tuning. Michael Tsyrulnikov and Alexander Rakitko (HMC) A new Hierarchical Bayes approach to ensemble-variational College Park, data 20 assimilation Oct / 48

11 Hierarchical Bayes: principle Oct / 48

12 Statistical basics: Frequentist, Bayes, Hierarchical Bayes The parameter estimation problem. 1 Frequentist (non-bayesian) Setup: p(y; θ), where θ is deterministic. Estimation: e.g. maximum likelihood: ^θ = argmax θ p(y; θ) 2 Bayes Setup: p(y θ), where θ is random. Specify the prior p(θ) and look at the posterior: p(θ y) p(θ) p(y θ). Estimation: e.g. maximum posterior probability: ^θ = argmax θ p(θ y). 3 Hierarchical Bayes Setup: p(θ) = p(θ β), where β is random. Specify the hyper-prior p(β) and look at the posterior: p(θ, β y) p(β) p(θ β) p(y θ). Estimation: e.g. (^θ, ^β) = argmax θ,β p(θ, β y). Oct / 48

13 DA: Frequentist and Bayes The analysis step: =The parameter is θ x =The observational likelihood is p(y x). 1 Bayes Setup: The state (parameter) is random. The background affects its prior: p(x x b ). Other parameters of the prior (m, B) are specified. The posterior: p(x x b, y) p(x x b )p(y x) e 1 2 [(x xb ) B 1 (x x b )+(y H(x)) R 1 (y H(x))] 2 Frequentist (non-bayesian) Setup: The background x b is regarded as part of observations: z := (x b, y), where x is deterministic. Oct / 48

14 DA: Hierarchical Bayes For Gaussian distributions, the hyper-parameter is β m, B. Why hyper-priors in DA now? Because on the one hand, the hyper-parameter B remains largely uncertain, and on the other hand, it has become observable with the advent of ensemble techniques. Prior: p(x) = p(x m, B) Hyper-prior: p(m, B) Joint posterior: p(x, m, B x b, y) Oct / 48

15 Hierarchical Bayes analysis: principle We stop specifying the hyper-parameters m = x b and B of the prior distribution of x. We admit that these hyper-parameters, the (true) mean vector m and the (true) covariance matrix B are uncertain and random. We specify priors for the hyper-parameters m, B. In the analysis, we estimate m, B from all the available information along with the state x. Oct / 48

16 Hierarchical Bayes EnVar Oct / 48

17 Hierarchical Bayes EnVar (HB-EnVar): principle In this talk, we focus on the analysis step. We update m and B along with the state x, given the deterministic background x f, the background ensemble X e, and observations y. The joint posterior is p(x, m, B x f, X e, y) p(m x f )p(b)p(x m, B)p(X e m, B)p(y x) The primary goal is the posterior distribution of x and its mean ^x in particular. Oct / 48

18 The traditional terms The prior state conditional distribution: The observational likelihood: p(x m, B) 1 B 1/2 e 1 2 (x m) B 1 (x m) p(y x) e 1 2 (y Hx) R 1 (y Hx) Oct / 48

19 Ensemble likelihood p(x e m, B) 1 B N 2 e 1 N 2 k=1 (xe k m) B 1 (x e k m) no need and no room for approximations. NB: Ensemble members are treated as observations on B. Oct / 48

20 Prior pdf for B So, the only input to the new optimal extended-space analysis technique is the prior distributions p(m x f ) and p(b). This is, perhaps, the good news because these can be, in principle, retrieved from an archive of (adequate) ensembles. Oct / 48

21 Matrix variate probability distributions Vectorization. General matrix: X = vec X := x 1... x n Sparse matrix: include in X only non-zero entries (i.e. in the matrix support). p(x) is identified with p(vec X). Oct / 48

22 Matrix variate Gaussian distribution X is matrix variate Gaussian distributed if X = vec X is the multivariate Gaussian vector with mean M and covariance matrix U V, where U and V are non-random symmetric non-negative definite matrices and U V := U 11 V... U 1n V U n1 V... U nn V The resulting pdf is p(x) e 1 2 tr[(x M)U 1 (X M) V 1 ] Covariances between matrix entries: Cov (X i,j, X i,j ) = V i,i U j,j Simulation: X = M + ΦYΨ, where Φ and Ψ are such that V = ΦΦ and U = ΨΨ, and Y is the pure noise random matrix. Oct / 48

23 Selecting the prior pdf for B: requirements 1 The distribution family should be suitable for modeling random covariance matrices. 2 The distribution family should be rich enough to give rise to realistically complex case-to-case variable background-error covariances. 3 For efficient Monte-Carlo sampling, the distribution should have analytically tractable and fast computable pdf. Oct / 48

24 The candidate probability distributions for B Discrete distribution Truncated Gaussian Wishart / Inverse Wishart Square-root Gaussian (Parametric covariance model with random parameters) Oct / 48

25 Discrete distribution p(b) = M w m δ(b B m ) m=1 Too restrictive, especially if B m are from a climatic archive. Can be used in the cycling mode. Leads to a kind of particle filter at the covariance level. Oct / 48

26 Wishart and Inverse Wishart distributions Conjugate priors for Gaussian likelihoods. The Wishart distribution for the precision matrix C: p(c) C θ 2 exp{ 1 2 tr(θb 0C)} Not flexible enough: only one matrix-variate parameter (B 0 ) for both location and scale of the distribution. The second parameter for scale (dispersion) is only scalar valued (θ). Oct / 48

27 Square-root Gaussian distribution our current choice In the factorization B = WW postulate that W is a Gaussian random matrix. Its pdf is p(w) e 1 2 tr[(w W 0)U 1 (W W 0 ) U 1 ] Oct / 48

28 A random sample from p(b): a row of the random matrix Oct / 48

29 A random sample from p(b): a row of the random matrix Oct / 48

30 A random sample from p(b): a row of the random matrix Oct / 48

31 A random sample from p(b): a row of the random matrix Oct / 48

32 A random sample from p(b): a row of the random matrix Oct / 48

33 A random sample from p(b): a row of the random matrix Oct / 48

34 A random sample from p(b): a row of the random matrix Oct / 48

35 A random sample from p(b) Oct / 48

36 A random sample from p(b) Oct / 48

37 A random sample from p(b) Oct / 48

38 A random sample from p(b) Oct / 48

39 HB-EnVar: analysis algorithms Oct / 48

40 The joint posterior pdf p post (x, W) p(w) p(x m, W) p(x e m, B) p(y x) J(x, W) := 2 log p post (x, W) = J W + J b + J e + J o Oct / 48

41 (1) Deterministic analysis: posterior mode analysis: analytic solution If we impose the Wishart prior for the precision matrix C, then there is an analytic solution for B: ^B = 1 θ + N + 1 (θb 0 + NS + A), where S is the sample covariance matrix and is the adaptivity matrix. A := (^x m)(^x m) T Oct / 48

42 Deterministic analysis: posterior mode analysis: numerical solution: 1-D argmax[p post (x, W)] = argmin[j(x, W)] Oct / 48

43 Posterior mode analysis: numerical solution Numerical optimization (quasi-newton). 1-D: unique mode. 4 and 8 grid points: 3 maxima. Important: with the sqrt-gaussian prior B distribution, the global mode (left) gives the localized W (without any kind of imposed localization!), in contrast to a local mode (right): Oct / 48

44 Analysis: (2) Importance sampling The posterior pdf in the marginalized form: p post (x, W) = p post (W) p post (x W) p post (x W) N (x a (m, W), B a (W)) Oct / 48

45 Importance sampling ^x = p post (W) x a (m, W) dw ^x = E x a p post (W) (m, W) = E q(w) x a (m, W) q(w) ^x x a := M w m x a (m, W + m ) m=1 =Selection of q: Gaussian pdf centered at the EnVar W EV. =Localization is achieved by introducing sparsity within the proposal distribution q for W. =The ordinary analysis step is included in the importance sampling analysis. =EnVar can be reproduced within HB-EnVar by nullifying the prior uncertainty in B. Oct / 48

46 HB-EnVar: first performance results Oct / 48

47 1-D illustrative example: dependence on the ensemble size RMSE of Analysis RMSE Exact B HB Mode HB Importance EnVar Var EnKf N (Ensemble size) In the toy problem, the deterministic HB-EnVar analyses outperform Var, EnKF, and EnVar. Oct / 48

48 Previous work Wikle C.K. and Berliner L.M. (2007) A Bayesian tutorial for data assimilation. Physica D, 230, 1 16: proposed to use the Hierarchical Bayesian paradigm to account for uncertainties in parameters of error statistics used in data assimilation. Myrseth I. and Omre H.(2010) Hierarchical Ensemble Kalman Filter. SPE Journal, v.15(2), : proposed within the EnKF paradigm to remove the assumption that the background-error covariance matrix B and the background-error mean field m are known deterministic quantities, replacing it by the assumption that these are uncertain and random. But they didn t use the ensemble likelihood. Bocquet M. (2011) Ensemble Kalman filtering without the intrinsic need for inflation. Nonlin. Processes Geophys., v. 18, : went further, separating, in EnKF, the distribution of the random matrix B (and random vector m) into the prior and the ensemble likelihood. But he used non-informative priors, whether we propose informative priors. Oct / 48

49 Conclusions Main aspects HB-EnVar Background-error covariance matrix B is treated as a sparse random matrix and updated in the optimal scheme along with the state. The key issue is the prior distribution of B. Ensemble members are treated as observations on the B matrix and assimilated along with ordinary observations. The technique is computationally expensive. Potential benefits of HB-EnVar Optimized hybridization of static and ensemble covariances. Optimized combination of x f and x e. Optimized covariance localization. Optimized feedback from y to the B matrix. Uncertainty in B is explicitly accounted for in the generation of the analysis ensemble, resulting in increased spread. Thank you! Oct / 48