Estimating model evidence using data assimilation

Size: px

Start display at page:

Download "Estimating model evidence using data assimilation"

Dina Lang
5 years ago
Views:

Estimating model evidence using data assimilation Application to the attribution

Michael Ghil 4,5 and the DADA Team (1 CEREA, joint laboratory École des Ponts

(2 Nansen Environmental and Remote Sensing Center, Bergen,Norway (3 IFAECI,

(4 École Normale Supérieure, Paris, France.

1 Estimating model evidence using data assimilation Application to the attribution of climate-related events Marc Bocquet 1, Alberto Carrassi 2, Alexis Hannart 3 & Michael Ghil 4,5 and the DADA Team (1 CEREA, joint laboratory École des Ponts ParisTech and EDF R&D, Université Paris-Est, Champs-sur-Marne, France. (2 Nansen Environmental and Remote Sensing Center, Bergen,Norway (3 IFAECI, CNRS-CONICET-UBA, Buenos Aires, Argentina. (4 École Normale Supérieure, Paris, France. (5 University of California, Los Angeles, USA Workshop on Stochastic Weather Generators 17-2 May, 216, Vannes, France M. Bocquet (ENPC DA for model evidence May 17, / 3

2 Outline 1 Model evidence: a key quantity Definition and interpretation Contextual model evidence Computing model evidence using data assimilation 2 Model evidence: Numerical results Setup Time series Dependence on the forcing and evidencing window Which is right? Discriminating power 3 Attribution of climate related events Premises DADA approach Attribution of extreme events using data assimilation 4 Conclusion M. Bocquet (ENPC DA for model evidence May 17, / 3

3 Model evidence: a key quantity Definition and interpretation Model evidence Definition & Interpretation Model evidence p(y K: M = p(y K: = p(y K, y K 1,..., y Likelihood of (the infinite from the far past observation sequence p(y K: = dx p(y K: xp(x (1 It is the marginal likelihood of the data. p(x plays the role of the prior. It depends on the underlying dynamics (the model. If the dynamics is ergodic = p(x is the invariant distribution on its attractor. p(y K: reflects this invariant distribution at a level of accuracy dependent on the observation model. We define p(y K: as the climatological model evidence. M. Bocquet (ENPC DA for model evidence May 17, / 3

4 Model evidence: a key quantity Definition and interpretation Model evidence Its applications The marginal likelihood can be used as a general metric for model selection and comparison. Comparing the skill of several candidate models (or model settings, or boundary conditions in representing a given observed phenomenon. Calibrating the state-space model s parameters based on the observed data by maximizing the marginal likelihood. Quantifying the evidence supporting several (potentially conflicting theoretical hypothesis associated to the physics of a phenomena. Evidencing the existence (or non-existence of a causal relationship between an external forcing and an observed response. M. Bocquet (ENPC DA for model evidence May 17, / 3

5 Model evidence: a key quantity Definition and interpretation Statistical/climatological model evidence The climatological model evidence embeds significant global information about the system of interest. The accuracy of its estimate is limited by the computational difficulty implied by the large dimension of typical relevant problems. Its climatological character does not permit to adapt it to the present condition of the system. The model evidence has been used in various contexts, including inverse problems. A typical example is the estimation of the error statistics in the source term inversion, or discriminate sources (i.e. nuclear release - Winiarek et al., 211 Atmos. Env. M. Bocquet (ENPC DA for model evidence May 17, / 3

6 Model evidence: a key quantity Contextual model evidence From statistical/climatological to conditional/contextual evidence We propose using data assimilation (DA to 1 compute model evidence, 2 and/or narrow it to the present-time context. Proposal: (leap of faith use instead the conditional/contextual model evidence p(y K: p(y K:1 y : y K:1 is used for evidence diagnostic. y : is used to specify the context. M. Bocquet (ENPC DA for model evidence May 17, / 3

7 Model evidence: a key quantity Contextual model evidence From statistical/climatological to conditional/contextual evidence It narrows the climate perspective to the context at t : p(y K:1 y : = dx p(y K:1 x p(x y : (2 The conditional pdf p(x y : plays the role of the prior density and substitutes p(x of the climatological model evidence. p(y K:1 y : is much easier to compute than p(y K: (but yet complicated in many relevant situations p(x y : is the natural (albeit approximate outcome of the DA machinery. We will show that p(y K:1 x can also be estimated via DA. M. Bocquet (ENPC DA for model evidence May 17, / 3

8 Model evidence: a key quantity Contextual model evidence Conditional/contextual evidence: A useful iterative formula Contextual model evidence can be computed iteratively: p(y K:1 y : = K p(y k y k 1: k=1 The contextual model evidence is the product of the individual contextual model evidences! The individual context evidence p(y k y k 1: is often a tractable output of DA schemes. We will use this decomposition in the sequel to compute model evidence using DA: p(y k y k 1: = dx k p(y k x k p(x k y k 1: (3 p(y k x k is the observation likelihood and p(x k y k 1: is the forecast state pdf at t k. M. Bocquet (ENPC DA for model evidence May 17, / 3

9 Model evidence: a key quantity Computing model evidence using data assimilation How to compute contextual model evidence 1. Importance sampling/monte Carlo Use a DA method up to t. This provides (an approximation of p(x y :. Compute the model evidence via Importance Sampling/Monte Carlo: 1 sample N members from p(x y : and forecast from t to t T 2 weight each member trajectory x (k T : according to their likelihood p(y T :1 x (k T :, so as to obtain p(y T :1 y : N k=1 p(y K:1 x (k 1 N Importance sampling (IS: use an ensemble-da method up to t = the assumption N M can be used in steps (1 and (2 Evaluation of model evidence using a particle filter proposed by Reich and Cotter, 215. M. Bocquet (ENPC DA for model evidence May 17, / 3

10 Model evidence: a key quantity Computing model evidence using data assimilation 2. Filtering: Linear/Gaussian case Kalman filter Use a DA method up to t. This provides (an approximation of p(x y :. In the Gaussian linear case (Kalman filter, the likelihood and forecast pdfs are Gaussian (It possible to show that the contextual model evidence reads p(y K:1 y : = K k=1 ( exp 1 yk H 2 k x f k (2π d Rk + H k P f k HT k 2 R k +H k P f k HT k (4 R k, P f k: observation and forecast error covariances H k : observation operator. Notation remark: x 2 A = xt A 1 x. M. Bocquet (ENPC DA for model evidence May 17, / 3

11 Model evidence: a key quantity Computing model evidence using data assimilation 2. Filtering: nonlinear/gaussian case ensemble Kalman filter Using an ensemble Kalman filter (EnKF, the Kalman filter formula for model evidence now becomes ( K exp 1 yk H 2 k x f 2 k R k +Y k Y k p(y K:1 y : T (2πd R k + Y k Yk T k=1 Y k = H k X k are the normalized observation anomalies, and X k the forecast anomalies. The inversion of R k + Y k Y T k can be done in ensemble subspace. The estimation is exact when the models are linear, the initial condition and observation errors are Gaussian and when the ensemble perturbation span the full range of uncertainty. M. Bocquet (ENPC DA for model evidence May 17, / 3

12 Model evidence: a key quantity Computing model evidence using data assimilation 3. Smoothing Use a DA method up to t. This provides (an approximation of p(x y :. Use a smoothing DA method to estimate p(y T :1 y : Compute the evidence with a saddle-point approximation (Laplace method: p(y K:1 y : = dx p(y T :1 x p(x y : ( dx exp 1 K y k H k M k (x 2 R 2 k + ln p(x y : k=1 = dx exp ( L(y K:1, x exp ( L(y K:1, x 2 x L(x (5 How to obtain x characterizes the problem: 4DVar, En-smoother... M. Bocquet (ENPC DA for model evidence May 17, / 3

13 Model evidence: a key quantity Computing model evidence using data assimilation 3. Smoothing En4DVar Ensemble-4DVar (En4DVar (see nomenclature by Lorenc, 213 The state is parameterized in the ensemble space: x = x + X w The evidence is marginalized over the coefficient vector w: p(y K:1 y : = dw p(y K:1 wp(w y : (6 Using the Laplace approximation on the analysis (the minimum w the evidence reads: ( exp 1 K 2 k=1 y k H k M k: (x 2 R k 1 2 w 2 p(y K:1 y : (2π Kd K k=1 R k I N + (7 K k=1 (Y k T R 1 k Yk M. Bocquet (ENPC DA for model evidence May 17, / 3

14 Model evidence: a key quantity Computing model evidence using data assimilation 3. Smoothing IEnKS (Quasi-Static Iterative Ensemble Kalman Smoother (IEnKS The initial condition and the ensemble at t are sequentially updated first by assimilating y 1 in the first step, then y 2 in the second step and so on until y K. The analysis at k, x k, and the normalized anomaly matrix, X k, are both defined at t, and used in the subsequent step. Each single contextual evidence p(y k y k 1: corresponds to the quasi-static analysis of the IEnKS (Bocquet and Sakov, 214. The iterative formula for model evidence and Laplace approximation on the analysis of the IEnKS gives K exp ( 12 y k H k M k: (x k 2Rk +Y k (Y k T p(y K:1 y : (8 (2π d Rk + Y k (Y k T k=1 M. Bocquet (ENPC DA for model evidence May 17, / 3

Model evidence: Numerical results Setup Model evidence Numerical experiments Forced L63 Model dx dt = σ(y x + λ i cos θ dy dt = ρx y xz + λ i sin θ dz dt = xy βz (9 Correct λ 1 = ;

15 Model evidence: Numerical results Setup Model evidence Numerical experiments Forced L63 Model dx dt = σ(y x + λ i cos θ dy dt = ρx y xz + λ i sin θ dz dt = xy βz (9 Correct λ 1 = ; Incorrect λ { 8 : 8} ; θ = 14. L95 Model dx m = x m 1 (x m+1 x m 2 x m + F i m = 1,..., M (1 dt Correct F 1 = 8; Incorrect F {5 : 11}. M. Bocquet (ENPC DA for model evidence May 17, / 3

16 Model evidence: Numerical results Time series Model evidence time series log(p 1 log(p (a L63 correct (λ 1 =, K=1 IS EnKF En-4D-Var IEnKS time (c L63 incorrect (λ =8, K= time log(p 1 log(p (b L95 correct (F 1 =8, K= time (d L95 incorrect (F =11, K= time M. Bocquet (ENPC DA for model evidence May 17, / 3

17 Model evidence: Numerical results Time series Pdf of model evidence L63 pdf log(p 1 pdf log(p pdf log(p 1 (a pdf of CME correct L63 (λ 1 =, K=1.1 IS EnKF.5 En-4D-Var IEnKS (b pdf of CME incorrect L63 (λ =8, K= (c Zoom on panel (a log(p pdf log(p.2.1 (d Zoom on panel (b log(p M. Bocquet (ENPC DA for model evidence May 17, / 3

18 Model evidence: Numerical results Time series Pdf of model evidence L95 (a pdf of CME correct L95 (F 1 =8, K=1 pdf log(p 1 IS.2 EnKF En-4D-Var IEnKS (b pdf of CME incorrect L95 (F =11, K=1 pdf log(p pdf log(p (c Zoom on panel (a pdf log(p log(p (d Zoom on panel (b log(p M. Bocquet (ENPC DA for model evidence May 17, / 3

19 Model evidence: Numerical results Dependence on the forcing and evidencing window Model evidence versus forcing and length of window log(p log(p (a L63 (K= λ (c L63 (λ = K log(p log(p (b L95 (K=1 IS EnKF En-4D-Var IEnKS F (d L95 (F = K M. Bocquet (ENPC DA for model evidence May 17, / 3

20 Model evidence: Numerical results Which is right? Computation of the integral: Which is right? Comparison with: 1 Hermite-Gauss quadrature with 32 degrees - for L63 2 IS/Monte Carlo with 1 6 particles - for L63 and L95 (a L63 correct (λ 1 = (b L95 correct (F 1 = log(p log(p IS EnKF En 4D Var IEnKS GHQ MC (c L63 incorrect (λ =8 55 IS EnKF En 4D Var IEnKS GHQ MC (d L95 incorrect (F = log(p IS EnKF En 4D Var IEnKS GHQ MC log(p 75 7 IS EnKF En 4D Var IEnKS GHQ MC M. Bocquet (ENPC DA for model evidence May 17, / 3

21 Model evidence: Numerical results Discriminating power Discriminating power versus forcing and length of window (a L63 (K=1 (b L95 (K=1 log(p 1 /p 1 5 log(p 1 /p log(p 1 /p λ (c L63 (λ 1 =; λ =8 log(p 1 /p F (d L95 (F 1 =8; F =11 IS EnKF En-4D-Var IEnKS K K M. Bocquet (ENPC DA for model evidence May 17, / 3

22 Attribution of climate related events Premises The attribution problem M. Bocquet (ENPC DA for model evidence May 17, / 3

23 Attribution of climate related events Premises Attribution standard approach M. Bocquet (ENPC DA for model evidence May 17, / 3

24 Attribution of climate related events Premises Limitations of the standard approach M. Bocquet (ENPC DA for model evidence May 17, / 3

25 Attribution of climate related events DADA approach Event definition: Standard approach versus DADA 1 Standard Approach - Event occurrence based on ad hoc condition like a scalar index Φ(Y exceeding a threshold u p i = P(Φ(Y u Hard to compute - Mathematically intractable in many practical cases Simulation based 2 DADA Proposal - Use of the tightest possible occurrence definition f i (y = {ω Ω Y(ω y h} h and FAR = 1 f (y f 1 (y In DADA we wish to evaluate f,1(y - The likelihood (model evidence associated to the observation y of the event of interest in each two worlds (factual and counter-factual Easy to estimate! (sometimes... The model evidence can be obtained as a side-product of DA procedures aimed at inferring the state-vector The D&A grand challenge associated to the design of an operational platform for near real time attribution of weather events may be solved by piggybacking on existing Data Assimilation meteorological routines. M. Bocquet (ENPC DA for model evidence May 17, / 3

26 Attribution of climate related events Attribution of extreme events using data assimilation Extreme event attribution DADA approach Forced L63 model with stochastic noise (Palmer, 1999, J. Climate: dx = σ(y x + λi cos θi + vx dt dy = ρx y xz + λi sin θi + vy dt dz = xy βz + vz dt (vx, vy, vz N (, σq I3 Factual λ1 = 2; Counter-factual λ = PDF1 PDF PDF1 PDF < 1 1 > 1 3 Factual Counter Factual 1 = 2 1 M. Bocquet (ENPC DA for model evidence 1 5 May 17, / 3

27 Attribution of climate related events Attribution of extreme events using data assimilation DADA application Forced L63 Model GINI index is a measure of the discriminating skill, ranging from for no discrimination to 1 for perfect discrimination DADA - STANDARD Gin Index.6.4 Gin Index λ forcing σ q Model Error Gin Index.6.4 Gini Index σ q λ forcing σ r Obs Error M. Bocquet (ENPC DA for model evidence May 17, / 3

28 Conclusion Conclusion Data assimilation is used to compute model evidence 1 in the contextualization (ensemble-da method used up to t, 2 in the model evidence computation (ensemble-da method used to assimilate from t 1 to t T. The accuracy of this DA-based estimate scales with the sophistication of the DA method (IS EnKF IEnKS. To be (further assessed upon exact calculation of p(y T : (via quadrature or massive Monte Carlo. The model evidence is used as a new way to address the problem of attribution of climate-related event - DADA. The DADA approach has proven to be effective on low-order models and respond to the need of timely attribution evaluation. M. Bocquet (ENPC DA for model evidence May 17, / 3

29 Conclusion Conclusion Fundamental questions that requires attention: 1 The choice of the DA method is not trivial and (unsurprisingly related to the degree of nonlinearity of the problem. 2 Model error is a central concern. 3 It is so also for standard methods of attribution. 4 Indeed it can mask the factor whose casual attribution is under scrutiny confuse discrimination between factual and counter-factual worlds. 5 A DA-based method for attribution has the potential to deal with issue taking advantage of the knowledge in model error treatment in DA schemes. M. Bocquet (ENPC DA for model evidence May 17, / 3

30 Conclusion Key references On the DADA approach Carrassi A., M. Bocquet, A. Hannart and M. Ghil, 216: Estimating model evidence using Data assimilation. Submitted to Q.J.Roy.Meteorol.Soc. Hannart A., A. Carrassi, M. Bocquet, M. Ghil, P. Naveau, M. Pulido, J. Ruiz and P. Tandeo, 216. DADA: Data Assimilation for the Detection and Attribution of Weather- and Climate-related Events. Climatic Change, IN PRESS On the IEnKS Bocquet M. and P. Sakov. An iterative ensemble Kalman smoother. Q. J. R. Meteorol. Soc., 14: , 214. On the use of model evidence Elsheikh A., I. Hoteit I and M. Wheeler. Efficient bayesian inference of subsurface flow models using nested sampling and sparse polynomial chaos surrogates. Comput. Methods Appl. Mech. Engrg., 269: , 214. Winiarek V., J. Vira, M. Bocquet, M Sofiev and O. Saunier. Towards the operational estimation of a radiological plume using data assimilation after a radiological accidental atmospheric release. Atmos. Env., 45: , 211. Reich, S. and Cotter, C. Probabilistic Forecasting and Bayesian Data Assimilation. Cambridge University Press., chapter 9.1, 215. M. Bocquet (ENPC DA for model evidence May 17, / 3

Attribution of climatic events using data assimilation

Attribution of climatic events using data assimilation Alberto Carrassi Nansen Environmental and Remote Sensing Center - Norway University of Maryland College Park 24 th April 2018 A. Carrassi University