Ensemble forecasting and flow-dependent estimates of initial uncertainty Martin Leutbecher acknowledgements: Roberto Buizza, Lars Isaksen Flow-dependent aspects of data assimilation, ECMWF 11 13 June 2007
Ensemble forecasting aims at evolving a sample of the p.d.f. p 0 of the initial state to obtain a sample of the p.d.f. of the atmospheric state at a future time. In the real atmosphere, p 0 will be flow-dependent, i.e. it varies from day to day...... Monday Tuesday Wednesday... Can data assimilation schemes provide flow-dependent estimates of p 0 that can be used to improve the current operational specification of initial uncertainty? The operational ECMWF EPS specifies initial uncertainty by an isotropic Gaussian distribution in the space spanned by the leading singular vectors computed with a total energy norm. What are the improvements we can expect in ensemble forecasting when using more appropriate flow-dependent estimates in order to specify p 0? Martin Leutbecher Ensemble forecasting 2
Outline 1. a few simple experiments with the Lorenz-95 system 2. the operational EPS 3. some preliminary results from ensemble forecasting experiments that use ensemble data assimilation experiments (RB s and LI s experiments) 4. outlook 5. conclusions Martin Leutbecher Ensemble forecasting 3
Lorenz-95 system 3 2 1 40 39 forecast model and system (in perfect model setting) dx i dt = x i 2x i 1 + x i 1 x i+1 x i + F with i = 1, 2... N, cyclic boundary conditions and a forcing of F = 8. Here, N = 40. nonlinear system of ODEs introduced by E. Lorenz in 1995 allegorical for dynamics of weather at a fixed latitude a unit time t = 1 is associated with 5 days error doubling time of about 2 days. Martin Leutbecher Ensemble forecasting 4
L95: observations and data assimilation system Observations: obs at every site i = 1 40, every 6 h uncorrelated, unbiased, normally distributed errors with standard deviation σ o = 0.15σ clim Extended Kalman filter x a = x b + K(y Hx b ) (1) x b = M(x a ) (2) K = P f H T (R + HP f H T ) 1 (3) P f = MP a M T + Q (4) (P a ) 1 = ( P f ) 1 + H T R 1 H (5) Matrix Q is diagonal with variance tuned to avoid filter divergence and give best forecasts σ q = 0.001 and 0.05 for perfect and imperfect model scenario, respectively (σ clim = 3.5). Martin Leutbecher Ensemble forecasting 5
Flow-dependence of P a : standard deviations P^a and < P^a > 0.25 0.20 stdev 0.15 0.10 0.05 0.00 0 5 10 15 20 25 30 35 40 site Martin Leutbecher Ensemble forecasting 6
Flow-dependence of P a : correlations 1.0 0.8 0.6 correlation 0.4 0.2 0.0 0.2 0.4 0.6 0.8 1.0 0 5 10 15 20 25 30 35 40 site Martin Leutbecher Ensemble forecasting 7
100 member L95: Ensemble forecasting initial conditions x j (t = 0), j = 1,...100 are sampled from a Gaussian distribution x j (t = 0) N(x a,a), where A = P a, analysis err. cov. predicted by KF (i.e. valid for the start time of the forecast) A = P a, time-average of P a A I, with same total variance as P a A a random draw from the sample of P a predicted by the KF, i.e. the cov. from the wrong day. some other choice of A that differs systematically from P a. statistics are based on 180 cases; ensemble forecasts are started every 48 h (to avoid too much correlations). perfect model scenario (imperfect model: qualitatively similar results). Martin Leutbecher Ensemble forecasting 8
Spread and ensemble mean error 4.0 initial conditions sample N(x a (t),p a (t)), forecast range t = 6 corresponds to 30 days doubling time of about t = 0.4 equivalent of 2 d RMS 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0 0 1 2 3 4 5 6 lead time spread around EM RMS error EM clim. stdev. Martin Leutbecher Ensemble forecasting 9
Brier skill score for three events (positive anomalies, anomalies larger 1 stdev, anomalies larger 1 stdev) 1.0 0.9 0.8 0.7 0.6 Brier Skill Score (ensemble) BSS 0.5 0.4 0.3 0.2 0.1 >m s >m >m+s 0.0 0 1 2 3 4 5 6 lead time Martin Leutbecher Ensemble forecasting 10
Spread and ensemble mean error: initial time 0.16 t= 0.000 KFT0 M100 0.14 0.12 RMS error 0.10 0.08 0.06 0.04 averaged Pa 0.02 instantaneous Pa random Pa 0.00 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 RMS spread Pa=0.022*Id sample of sites cases stratified by ensemble stdev; 20 bins with 360 values each. Martin Leutbecher Ensemble forecasting 11
Spread and ensemble mean error: t = 12 h t= 0.100 KFT0 M100 0.20 0.15 RMS error 0.10 0.05 Pa=0.022*Id averaged Pa instantaneous Pa random Pa 0.00 0.00 0.05 0.10 0.15 0.20 RMS spread Martin Leutbecher Ensemble forecasting 12
Spread and ensemble mean error: t = 24 h 0.30 t= 0.200 KFT0 M100 0.25 0.20 RMS error 0.15 0.10 0.05 averaged Pa instantaneous Pa random Pa 0.00 0.00 0.05 0.10 0.15 0.20 0.25 0.30 RMS spread Pa=0.022*Id Martin Leutbecher Ensemble forecasting 13
Spread and ensemble mean error: t = 48 h 0.5 t= 0.400 KFT0 M100 0.4 RMS error 0.3 0.2 Pa=0.022*Id 0.1 averaged Pa instantaneous Pa random Pa 0.0 0.0 0.1 0.2 0.3 0.4 0.5 RMS spread Martin Leutbecher Ensemble forecasting 14
Spread and ensemble mean error: t = 120 h 2.0 t= 1.000 KFT0 M100 1.5 RMS error 1.0 0.5 averaged Pa instantaneous Pa random Pa 0.0 0.0 0.5 1.0 1.5 2.0 RMS spread Pa=0.022*Id Martin Leutbecher Ensemble forecasting 15
Over- and underdispersion D+2 D+5 t= 0.400 KFT0 M100 t= 1.000 KFT0 M100 RMS error 0.7 0.6 0.5 0.4 0.3 0.2 Pa=0.022*Id Pa=4*0.022*Id 0.1 instantaneous Pa Pa=0.25*0.022*Id 0.0 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 RMS spread RMS error 2.5 2.0 1.5 1.0 Pa=0.022*Id 0.5 Pa=4*0.022*Id instantaneous Pa Pa=0.25*0.022*Id 0.0 0.0 0.5 1.0 1.5 2.0 2.5 RMS spread Martin Leutbecher Ensemble forecasting 16
Erroneous distributions of variance D+2 D+5 t= 0.400 KFT0 M100 t= 1.000 KFT0 M100 0.5 2.0 0.4 1.5 RMS error 0.3 0.2 RMS error 1.0 0.1 Pa=0.022*Id Pa=4*0.022*diag(1,..,1,0,..,0) instantaneous Pa 0.5 Pa=0.022*Id Pa=4*0.022*diag(1,..,1,0,..,0) instantaneous Pa Pa=4*0.022*diag(1,0,0,0,1,0,..) 0.0 0.0 0.1 0.2 0.3 0.4 0.5 RMS spread Pa=4*0.022*diag(1,0,0,0,1,0,..) 0.0 0.0 0.5 1.0 1.5 2.0 RMS spread variance 1 10 40 1 5 40 Martin Leutbecher Ensemble forecasting 17
A tentative explanation time / variance space Martin Leutbecher Ensemble forecasting 18
Operational ECMWF Ensemble Prediction System 50 perturbed forecast, 1 (3) unperturbed forecasts initial perturbations based on the leading 50 singular vectors (2 sets of 50 for each hemisphere) perturbations in the tropics in the vicinity of active tropical cyclones based on the leading 5 singular vectors stochastic diabatic tendency perturbations (a.k.a. stochastic physics, uniform distr. between 0.5 and 1.5, random numbers change every 3 h and 10 10 ) up to January 2006: T L 255L40 from Feb 2006: T L 399L62 up to D+10, then T L 255L62 to D+15 (VAREPS) Feb 2006: reduction of amplitude assigned to evolved singular vectors by 33% because higher-resolution model is more active improved match between ens. dispersion and ens. mean error Martin Leutbecher Ensemble forecasting 19
z at 500hPa area n.hem.mid symbols: RMSE of Ens. Mean; no sym: Spread around Ens. Mean DJF Z500 spread and ens. mean error DJF 2006 vs 07, 35N 65N 1000 800 RMS 600 400 200 2007 2007 2006 2006 0 0 2 4 6 8 10 12 14 fc-step (d) Martin Leutbecher Ensemble forecasting 20
Z500 Stdev and ens. mean RMSE, 35N 65N, DJF06/07 t = 24 h z500hpa, t = t=+48h, 48 h N.hem.mid t = 120 h z500hpa, t=+24h, N.hem.mid DJF2007 DJF2007 z500hpa, t=+120h, N.hem.mid DJF2007 28 55 110 24 50 45 100 90 RMS error (m) 20 16 12 8 4 4 8 12 16 20 24 28 RMS spread (m) RMS error (m) 40 35 30 25 20 15 10 10 15 20 25 30 35 40 45 50 55 RMS spread (m) RMS error (m) 80 70 60 50 40 30 20 20 30 40 50 60 70 80 90 100 110 RMS spread (m) Martin Leutbecher Ensemble forecasting 21
Flow-dependent initial uncertainty estimates in the ECMWF EPS preliminary results from Roberto s + Lars experiments (31R2): ensemble forecasts: TL255L62, 50 member ensemble data assimilation (EnDA): TL255L91, 10 member, 12-h 4D-Var perturbed observations model tendencies perturbed with backscatter scheme (Markov chain in spectral space for vorticity, vertical correlations from J b, multi-variate through nonlinear balance + ω-equation) 4 configurations for initial perturbations (added to interpolated operational high-resolution analysis TL799L91): initial singular vectors and evolved singular vectors (SV i+e) initial singular vectors only (SV i) perturbations of EnDA members about ens. mean (EnDA) EnDA perturbations and initial singular vectors (EnDA+SV i) 20 cases in Sep/Oct 2006 (every other day) Martin Leutbecher Ensemble forecasting 22
Z500 Ensemble stdev and ensemble mean RMS error, 35N 65N z at 500hPa sample of 20 cases; 2006092212-103012, area n.hem.mid symbols: RMSE of Ens. Mean; no sym: Spread around Ens. Mean 900 800 700 600 RMS 500 400 300 200 100 SV i+e SV i EnDA EnDA+SV i 0 0 1 2 3 4 5 6 7 8 9 10 fc-step (d) Martin Leutbecher Ensemble forecasting 23
Ensemble stdev and ensemble mean RMS error: D+1 28 24 z500hpa, t=+24h, N.hem.mid N20/2006092212TO103012 UF RMS error (m) 20 16 12 8 4 SV i+e SV i EnDA 4 8 12 16 20 24 28 RMS spread (m) 35N 65N EnDA+SV i Martin Leutbecher Ensemble forecasting 24
Z500 Ensemble stdev and ensemble mean RMS error: D+2 50 45 40 z500hpa, t=+48h, N.hem.mid N20/2006092212TO103012 UF RMS error (m) 35 30 25 20 15 10 5 5 10 15 20 25 30 35 40 45 50 RMS spread (m) 35N 65N SV i+e SV i EnDA EnDA+SV i Martin Leutbecher Ensemble forecasting 25
Z500 Ensemble stdev and ensemble mean RMS error: D+5 RMS error (m) 110 100 90 80 70 60 50 40 30 20 10 z500hpa, t=+120h, N.hem.mid N20/2006092212TO103012 UF 10 20 30 40 50 60 70 80 90 100110 RMS spread (m) 35N 65N SV i+e SV i EnDA EnDA+SV i Martin Leutbecher Ensemble forecasting 26
u850 Ensemble stdev and ensemble mean RMS error, 35N 65N u at 850hPa sample of 20 cases; 2006092212-103012, area n.hem.mid symbols: RMSE of Ens. Mean; no sym: Spread around Ens. Mean 6 5 4 RMS 3 2 SV i+e SV i EnDA 1 EnDA+SV i 0 0 1 2 3 4 5 6 7 8 9 10 fc-step (d) Martin Leutbecher Ensemble forecasting 27
Ensemble stdev and ensemble mean RMS error: D+1 u850hpa, t=+24h, N.hem.mid N20/2006092212TO103012 UF 3 RMS error (m/s) 2 1 SV i+e SV i EnDA 1 2 3 RMS spread (m/s) 35N 65N EnDA+SV i Martin Leutbecher Ensemble forecasting 28
Ensemble stdev and ensemble mean RMS error: D+2 6 u850hpa, t=+48h, N.hem.mid N20/2006092212TO103012 UF 5 RMS error (m/s) 4 3 2 1 SV i+e SV i EnDA 1 2 3 4 5 6 RMS spread (m/s) 35N 65N EnDA+SV i Martin Leutbecher Ensemble forecasting 29
u850 Ranked Probability Skill Score, 35N 65N 0.8 u at 850hPa 10 categories, sample of 20 cases; 2006092212-103012, area n.hem.mid Ranked Probability Skill Score 0.72 0.64 0.56 0.48 0.4 0.32 0.24 0.16 0.08 SV i+e SV i EnDA EnDA+SV i 0 1 2 3 4 5 6 7 8 9 10 fc-step (d) Martin Leutbecher Ensemble forecasting 30
u850 Ranked Probability Skill Score, 20N 90N u at 850hPa 10 categories, sample of 20 cases; 2006092212-103012, area n.hem 0.72 Ranked Probability Skill Score 0.64 0.56 0.48 0.4 0.32 0.24 0.16 0.08 SV i+e SV i EnDA EnDA+SV i 0 1 2 3 4 5 6 7 8 9 10 fc-step (d) Martin Leutbecher Ensemble forecasting 31
u850 Ranked Probability Skill Score, 20S 20N u at 850hPa 10 categories, sample of 20 cases; 2006092212-103012, area tropics 0.66 Ranked Probability Skill Score 0.6 0.54 0.48 0.42 0.36 0.3 0.24 SV i+e SV i EnDA EnDA+SV i 0.18 0 1 2 3 4 5 6 7 8 9 10 fc-step (d) Martin Leutbecher Ensemble forecasting 32
u850 Ensemble stdev and ensemble mean RMS error, 20S 20N 3 u at 850hPa sample of 20 cases; 2006092212-103012, area tropics symbols: RMSE of Ens. Mean; no sym: Spread around Ens. Mean 2 RMS SV i+e 1 SV i EnDA EnDA+SV i 0 0 1 2 3 4 5 6 7 8 9 10 fc-step (d) Martin Leutbecher Ensemble forecasting 33
summary (1) Experiments with the 40-variable Lorenz-95 system indicate that the skill of ensemble forecasts benefits from flow-dependent estimates of analysis error covariances at forecast lead times of up to 5 days. suggest that only gross systematic errors in representing initial uncertainty appear to be capable of deteriorating the ensemble forecasts at all lead times. The operational ECMWF EPS (T L 399L62) shows a close match between ens. stdev.and ens. mean RMSE for 500 hpa geopotential during the last DJF overall, but exhibits over-dispersion for situations with large spread and under-dispersion for low spread in the early forecast ranges ( D+3); the spread-skill relationship improves with lead time in a similar manner as in the L95-system (almost ideal relationship at D+5). Martin Leutbecher Ensemble forecasting 34
summary (2) Experiments with the ECMWF EPS (T L 255L62) show that perturbations from current ensembles of data assimilations yield insufficient ensemble dispersion at all forecast ranges up to D+10; indicate that it may be beneficial (tropics!) to replace evolved singular vectors by perturbations from an ensemble of data assimilations; Martin Leutbecher Ensemble forecasting 35
Outlook It seems worth investigating the following aspects impact of EnDA configuration on EPS forecasts EnDA resolution (inner/outer loop) number of members obs. selection, representation of obs. err. corr.... test of improved versions of backscatter algorithm in EnDA and EPS representation of model error using forcing singular vectors in EnDA and in EPS use of singular vectors computed with analysis error (co-)variance metric based on statistics from EnDA (roughly same cost as total energy SVs and possibility of flow-dependent initial metric) cf. Gelaro, Rosmond & Daley (2002); Buehner & Zadra (2006). impact of replacing evolved SVs by EnDA perturbations at operational EPS resolution (T L 399L62) Martin Leutbecher Ensemble forecasting 36
Imperfect model scenario 3 2 1 40 39
The system is given by dx k dt dy j dt Imperfect model scenario: ODEs = x k 1 (x k 2 x k+1 ) x k + F hc b Jk y j (6) k=j(k 1)+1 = cby j+1 (y j+2 y j 1 ) cy j + c b F y + hc b x 1+ j 1 J (7) with k = 1,...,K and j = 1,...,JK. The forecast model is given by dx k dt = x k 1 (x k 2 x k+1 ) x k + F g U (x k ). (8) Here, K = 40,J = 8 and b = 10 amplitude ratio between slow variables and fast variables c = 10 time-scale ratio between slow and fast variables h = 1 coupling strength between slow and fast variables F = F y = 10 forcing amplitude see also Wilks (2005) Martin Leutbecher Ensemble forecasting 38