Numerical Weather prediction at the European Centre for Medium-Range Weather Forecasts (2)

Transcription

1 Numerical Weather prediction at the European Centre for Medium-Range Weather Forecasts (2) Time series curves 500hPa geopotential Correlation coefficent of forecast anomaly N Hemisphere Lat 20.0 to 90.0 Lon to Reaches (60) oper_an od oper 00UTC,12UTC,beginning Score reaches 60 (MA 12m of ccaf) reaching 60% day Massimo Bonavita Data Assimilation Section Massimo.Bonavita@ecmwf.int Slide 1

2 Improving 4DVar The 4D-Var solution implicitly evolves bacground error covariances over the assimilation window (Thepaut et al.,1996) with the tangent linear dynamics: B(t) M B(t 0 )M T But it does not propagate error information from one assimilation cycle to the next: 4DVar behaves lie a Kalman Filter where B(t 0 ) is reset to a climatological, stationary estimate at the beginning of each assimilation window. This is sub-optimal. What to do? Slide 2

3 Improving 4DVar: The non-sequential approach In the non-sequential approach problem is shifted from the estimation of B to the estimation of Q: this is not any easier! It is difficult in the 4D-Var framewor to produce good estimates of the analysis errors, which are fundamental for ensemble prediction Long window, wea constrain 4DVar has been proven in simplified systems, but not in full operational setups yet This forces us to tae another loo at B: Slide 3

4 Improving 4DVar The role of B is: 1. To spread out the information from the observations. 2. To provide statistically consistent increments at the neighbouring gridpoints and levels of the model. 3. To ensure that observations of one model variable (e.g. temperature) produce dynamically consistent increments in the other model variables (e.g. vorticity and divergence) From a practical point of view, even assuming we have access to a good estimate of B, the problem of its size ( 10 8 x10 8 ) forces us to simplify it To a good extent, the history of data assimilation in NWP can be described as the quest to find compact representations of B which still eep its most important features Slide 4

5 Improving 4DVar: The sequential approach The other side of the spectrum of possible solutions is to try to construct a low ran approximation of B and to evolve this explicitly or implicitly This is what is done in the Ensemble Kalman Filter (EnKF, Evensen, 1994; Burgers et al., 1998) EnKF is a low-ran, Monte Carlo approx. of the KF which crucially does not require the Tangent Linear and Adjoint of M and H. It is optimal (as N ens -> ) for linear, Gaussian systems In the EnKF B ens = 1/(N ens -1) X b (X b ) T, (X b are the ensemble perturbations); the subspace spanned by B ens has still dimension N ens 1 << N Slide 5

6 Improving 4DVar: The sequential approach There are ways to artificially increase the effective ensemble size (Shur product covariance localization, Hamill and Whitaer, 2001; Local analysis, Evensen, 2003; Ott et al., 2004), but they (too!) come at a price: a) Dynamical balance is degraded; b) Asymptotic optimality of the EnKF lost; c) More difficult for non-local observations (i.e., observations that have non-local weighting functions; Campbell et al., 2010) Slide 6

7 Improving 4DVar: The sequential approach Conventional observations only, N.Hem. 500 hpa AC Slide 7

8 Improving 4DVar: The sequential approach All observations N.Hem. 500 hpa AC Slide 8

9 Improving 4DVar: The sequential approach All observations S.Hem. 500 hpa AC Slide 9

10 Improving 4DVar: The hybrid approach Hybrid approach: Use cycled, flow-dependent bacground error estimates (from an EnKF/Ensemble of DA system) in a 3/4D-Var analysis algorithm The hybrid formulation would be able: 1. Integrate flow-dependent error covariance information into a variational analysis 2. Keep the full ran representation of B and its implicit evolution inside the assimilation window 3. More robust than pure EnKF for limited ensemble sizes and large model errors 4. Allow consistent localization of ensemble perturbations to be performed in control space (advantageous for radiances, Surf. Press. observations); 5. Allow for flow-dependent QC of observations Slide 10

11 Improving 4DVar: The hybrid approach In operational use (or under development), there are currently at least three main approaches to doing hybrid DA in a VAR context: 1. Alpha control variable method (UK Met Office, NCEP, CMC) 2. 4D-Ens-Var 3. Ensemble of Data Assimilations method (ECMWF, Meteo France) Slide 11

12 The hybrid approach: α control variable The α control variable method (Barer, 1999, Lorenc, 2003) is conceptually equivalent to adding a flow-dependent term to the static model of B: B B 2 c c B c is the static, climatological covariance P e C loc is the localised ensemble covariance The increment is now a weighted sum of the part coming from the static B component and the flow-dependent, localised ensemble perturbations 2 e P e C loc x c B 1 2 c v e X ' α x clim x ens Slide 12

13 The hybrid approach: α control variable Pure ensemble 3D-Var 50/50 hybrid 3D-Var Slide 13 from: Massimo A.Clayton Bonavita - ECMWF (UK Met Office)

14 The hybrid approach: 4D-Ensemble-Var In the α control variable method one uses the ensemble perturbations to estimate B at the start of the 4DVar assimilation window: inside the 4DVar window B is still evolved by the tangent linear dynamics (B(t) MBM T ) In the 4D-Ensemble-Var method (Liu et al., 2008) B is sampled from ensemble trajectories throughout the assimilation window: Slide 14 from: D. Barer (UK Met Office)

15 The hybrid approach: 4D-Ensemble-Var The 4D-Ensemble-Var analysis is thus a localised linear combination of ensemble trajectories perturbations: conceptually very close to a pure EnKF While traditional 4DVar requires repeated, sequential runs of M, M T, ensemble trajectories from the previous assimilation time can be pre-computed in parallel: this can be a significant advantage in an operational environment Developing and maintaining the Tangent Linear and Adjoint models requires substantial resources and it is technically demanding: 4D-Ensemble-Var does not need them However current TL models are very accurate. Can we achieve the same level of accuracy from an ensemble? Slide 15

16 The hybrid approach: EDA The success of the hybrid approach has two main aspects: 1. Ability of the ensemble system to provide realistic, flowdependent estimates of the bacground errors of the reference system; 2. Ability of the reference 4DVar to incorporate the ensemble information effectively Slide 16

17 Slide 17 The Ensemble of Data Assimilations (EDA) For a linear system the data assimilation update is: Under the assumption of statistically independent bacground (P b ), observation (R) and model errors (Q), the evolution of the system error covariances is given by: a b b b a M x x x H y K x x 1 T a b T T b a Q M M P P K R K H K I P H K I P 1

18 Slide 18 The Ensemble of Data Assimilations (EDA) Consider now the evolution of the same system where we perturb the observations and the forecast model with random noise drawn from the respective error covariances: η ~N(0,R), ζ ~N(0,Q). If we define the differences between the perturbed and unperturbed state and, their evolution is obtained by subtracting the unperturbed state evolution equations from the perturbed ones: the perturbations evolve with the same update equations of the state a b b b a ζ M x x x H η y K x x ~ ~ ~ ~ ~ 1 a a a x x ε ~ b b b x x ε ~ a b b b a ζ M ε ε ε H η K ε ε 1

19 Slide 19 The Ensemble of Data Assimilations (EDA) The covariances of the perturbations also evolve with the same update equations of the state error covariances of the reference system: T T a a T b b T T T b b T a a Q M ε ε M ε ε K R K H K I ε ε H K I ε ε 1 1

20 The Ensemble of Data Assimilations (EDA) What does all this mean in practice? 1. We can use an ensemble of perturbed assimilation cycles to simulate the errors of our reference assimilation cycle; 2. The ensemble of perturbed DAs should be as similar as possible to the reference DA (i.e., same or similar K matrix) 3. The applied perturbations η, ζ must have the required error covariances (R, Q); 4. There is no need to explicitly perturb the bacground x b Slide 20

21 The Ensemble of Data Assimilations (EDA) 10 (25 from November 2013) ensemble members using 4D-Var assimilations T399 outer loop, T95/T159 inner loops. (Reference DA: T1279 outer loop, T159/T255/T255 inner loops) Observations randomly perturbed according to their estimated errors SST perturbed with climatological perturbations Model error represented by stochastic methods (SPPT, Leutbecher, 2009) Slide 21

22 The Ensemble of Data Assimilations (EDA) Slide 22

23 The Ensemble of Data Assimilations (EDA) The EDA simulates the error evolution of the 4DVar analysis cycle. As such it has two main applications: 1. Provide a flow-dependent estimate of analysis errors to initialize the ensemble prediction system (EPS) 2. Provide a flow-dependent estimate of bacground errors for use in 4D-Var assimilation Slide 23

24 The Ensemble of Data Assimilations (EDA) Slide 24

25 The Ensemble of Data Assimilations (EDA) Improving Ensemble Prediction System by including EDA perturbations for initial uncertainty (implemented June 2010) The Ensemble Prediction System (EPS) benefits from using EDA based perturbations. Replacing evolved singular vector perturbations by EDA based perturbations improve EPS spread, especially in the tropics. The Ensemble Mean has slightly lower error when EDA is used. Slide 25 Ensemble spread and Ensemble mean RMSE for 850hPa T

26 The Ensemble of Data Assimilations (EDA) The EDA simulates the error evolution of the 4DVar analysis cycle. As such it has two main applications: 1. Provide a flow-dependent estimate of analysis errors to initialize the ensemble prediction system (EPS) 2. Provide a flow-dependent estimate of bacground errors for use in 4D-Var assimilation Slide 26

27 The Ensemble of Data Assimilations (EDA) A short history of B modelling at ECMWF In variational analysis the B matrix is usually defined implicitly in terms of a transformation from the departure δx in state space to a control variable χ: Where L verifies B=LL T δx = Lχ In the spectral formulation (Derber and Bouttier, 1999), L has the form: L = K B u 1/2 K is a balance operator going from the set of variables [ζ, η u, (T,ps) u,q] (the control vector) to the set of state variables [ζ, η, (T,ps),q] Slide 27

28 Slide 28 The Ensemble of Data Assimilations (EDA) A short history of B modelling at ECMWF K accounts for the correlations between variables: B u 1/2 is the symmetric square root of the bacground error covariances of [ζ, η u, (T,ps) u,q], so that B u = B u T/2 B u 1/2 q T,p D ζ I I P N I M I q p T D ζ s u u s ) ( ), ( K

29 Slide 29 The Ensemble of Data Assimilations (EDA) A short history of B modelling at ECMWF Since we assume that the balance operator accounts for all intervariable correlations, B u is bloc diagonal Each bloc in B u is of the form Σ T CΣ Σ is the gridpoint standard deviation of bacground errors C is bloc diagonal, with one bloc for each wavenumber n, and each bloc is the form C n =h n V n q p T D u C C C C u s u ), ( B

30 The Ensemble of Data Assimilations (EDA) A short history of B modelling at ECMWF C is bloc diagonal, with one bloc for each wavenumber n, and each bloc is the form C n =h n V n V n are full vertical correlation matrices; h n are the coefficients that determine the horizontal correlations (modal variances) The spectral covariance model allows full resolution of the variation of correlations with horizontal scales Slide 30

31 The Ensemble of Data Assimilations (EDA) Vorticity vert. corr. n=2 Vorticity vert. corr. n=64 Vorticity Slide 31 errors stdev. 500hPa Vorticity errors length scale 500hPa

32 The Ensemble of Data Assimilations (EDA) Compact model Spectral model checlist Full spectral resolution Non-separable (vertical and horizontal scales of bacground error covariance are non-separable: large horizontal scales tend to have deeper vertical correlations than small horizontal scales. It is important for a B model to retain this property) Homogeneous Isotropic Static Slide 32

33 The Ensemble of Data Assimilations (EDA) A short history of B modelling at ECMWF The spectral B model is one end of the spectrum: full resolution of the variation of vertical correlation with horizontal scale, but it allows no horizontal variability of the vertical/horizontal correlations The other end of the spectrum is represented by the separable formulation which allows full horizontal variation of the vertical correlations (we may specify a different vertical covariance matrix for each horizontal grid point), but has no variation of vertical correlation with horizontal scale The wavelet B (Fisher, 2003) is a compromise between these two extremes and allows a degree of variation of vertical correlation with both wavenumber and horizontal location. Moreover, it also allows horizontal variation of horizontal correlation. Slide 33

34 The Ensemble of Data Assimilations (EDA) A short history of B modelling at ECMWF The wavelet B is based on a wavelet expansion on the sphere. The basis functions (wavelets) are chosen to be band-limited and, to a good approximation, spatially localized Slide 34

35 The Ensemble of Data Assimilations (EDA) A short history of B modelling at ECMWF The covariance matrices C n [N lev xn lev ] are now of the form C j [N lev xn lev ](λ,φ), where j is the index of the wavelet component The choice of the wavelet bandwidths [N j, N j+1 ] determines the trade-off between spectral and spatial resolution. If the bands are narrow, the corresponding wavelet functions are not spatially localized, and vice versa Vorticity errors Lscale 500hPa: Slide 35 Spectral Vorticity errors Lscale 500hPa: Wavelet

36 The Ensemble of Data Assimilations (EDA) Compact model Wavelet model checlist Good spectral resolution Non-separable (vertical and horizontal scales of bacground error covariance are non-separable: large horizontal scales tend to have deeper vertical correlations than small horizontal scales. It is important for a B model to retain this property) Non-homogeneous Isotropic Static Slide 36

37 The Ensemble of Data Assimilations (EDA) The wavelet B formulation: Flow-dependent Wavelet B 1/ 2 2 x x Lχ KΣ C 1/, b b can be made flow-dependent by obtaining flow-dependent estimates of the bacground error variances(σ b ) and correlations (C j (λ,φ)) from the EDA perturbations j j j j Slide 37

38 Flow-dependent Wavelet B Vorticity, 500 hpa, EDA standard deviations The EDA correctly identifies areas of active weather However the EDA is a stochastic system: error of variance estimator ~ 1/ Nens An effective system to filter out sampling noise is needed Slide 38

39 Flow-dependent Wavelet B We can use a spectral filter to disentangle noise from signal Truncation wavenumber is determined by maximizing signal-to-noise ratio of filtered variances (Raynaud et al., 2009; Bonavita et al., 2011) Slide 39

40 Flow-dependent Wavelet B To get statistically consistent EDA variances we need to perform an online calibration (Kolczynsy et al., 2009, 2011; Bonavita et al., 2011) Calibration factors are also state-dependent, i.e. depend on the size of the expected error Need to perform calibration of variances reflects underlying problem in Q and R models, system non-linearities, ensemble size, conditional model biases, etc. Slide 40

41 Flow-dependent Wavelet B EDA based bacground errors (Vorticity, 500 hpa) after filtering and calibration Slide 41

42 Flow-dependent Wavelet B The use of EDA based bacground errors changes the shape and structure of the analysis increments Tropical Cyclone Aere, Philippines 8-9 May Slide 42

43 Flow-dependent Wavelet B Tropical Cyclone Aere, Philippines 8-9 May DVar analysis with Static B errors 4DVar analysis with EDA errors Slide 43

44 Flow-dependent Wavelet B Tropical Cyclone Aere, Philippines 8-9 May Slide 44

45 Flow-dependent log(p B s ) Static errors log(p s ) EDA errors Tropical Cyclone Aere, Philippines 8-9 May Slide 45

46 Flow-dependent Wavelet B The same procedures can be applied to the computation of flow-dependent correlation structures (C j (λ,φ)) Above is the bacground error correlation length scale for Vorticity, 500 hpa Slide 46

47 Flow-dependent Wavelet B The robust estimation of a correlation matrix requires many more samples than the estimation of the error variances Correlations are estimated based on a sliding window of the past 10 days of EDA perturbations Slide 47

48 Flow-dependent Wavelet B Flow-dependent bacground error estimates from the EDA for the balanced part of the control vector have been introduced in ECMWF 4D-Var in April 2011 (Cycle 37R2) Flow-dependent bacground errors estimates from the EDA for the unbalanced part of the control vector have been introduced in ECMWF 4D-Var in June 2013 (Cycle 38R2) Flow-dependent bacground correlations from the EDA variances will be introduced in ECMWF 4D-Var in November 2013 (Cycle 39R1) The introduction of flow-dependent bacground errors and covariances estimated from the EDA has been the single largest source of improvement in recent years in the analysis and forecast sill of the IFS Slide 48

49 The Ensemble of Data Assimilations (EDA) Flow-dependent Wavelet B model checlist Compact model Good spectral resolution Non-separable Non-homogeneous Partially anisotropic (variances) Flow-dependent Slide 49

50 The Ensemble of Data Assimilations (EDA) 1. Improvements to the EDA: Where do we go from here? a) Increase in ensemble size (25) and resolution (T399) b) Improvements to model error parameterizations 2. Improvements to the B model: a) Reduce the length of the window used to compute the correlation structures (increase in ensemble size and use of regularization techniques) b) Introduce anisotropy in wavelet correlations 3. Continue development of tangent linear and adjoint models (M,M T ) which evolve B over the assimilation window. Slide 50

51 Than you for your attention Thans to M. Fisher, L. Isasen, J.N. Thépaut, E. Källén, A. Clayton, D. Barer Slide 51