Filtering of variances and correlations by local spatial averaging. Loïk Berre Météo-France

Transcription

1 Filtering of variances and correlations by local spatial averaging Loïk Berre Météo-France

2 Outline 1. Contrast between two extreme approaches in Var/EnKF? 2.The spatial structure of sampling noise and signal 3. Spatial averaging of ensemble-based variances 4. Spatial averaging of innovation-based variances 5. Spatial averaging of correlations

3 Two usual extreme approaches in Var/EnKF Covariance modelling : Var: often globally averaged (spatially). robust with a very small ensemble, but lacks heterogeneity completely. EnKF: often «purelylocal». a lot of geographical variations potentially, but requires a rather large ensemble, and it ignores spatial coherences. An attractive compromise is to calculate local spatial averages of covariances.

4 Is the usual ensemble covariance optimal? Usual estimation («raw»): B(N) = 1/(N-1) Σi e b (i) e b (i) T = best estimate, for a given ensemble size N? No. Better to account for spatial structures of sampling noise and signal. Similar to accounting for spatial structures of errors in data assimilation, through B and R.

5 The spatial structure of sampling noise & signal in ensemble variance fields

6 Spatial structure of sampling noise (Fisher and Courtier 1995 Fig 6, Raynaud et al 2008a) True variance field V* ~ large scale Sampling noise V e =V(N)-V* ~ large scale? NO. N = 50 L( ε b ) = 200 km While the signal of interest is large scale, the sampling noise is rather small scale.

7 Spatial structure of sampling noise (Raynaud et al 2008b) Spatial covariance of sampling noise V e =V(N)-V* : V e (V e ) T = 2/(N-1) B* B* where B* B* is the Hadamard auto-product of B* = ε b ( ε b ) T. The spatial structure of sampling noise V e is closely connected to the spatial structure of background errors ε b.

8 Spatial structure of sampling noise (Raynaud et al 2008b) Correlation function of sampling noise: cor(v e [i], V e [j]) = cor(ε b [i], ε b [j])² Length-scale L( V e ) of sampling noise: L( V e ) = L( ε b ) / 2 The sampling noise V e is smaller scale than the background error field ε b which is smaller scale than the variance field V* of background error?

9 Spatial structure of signal (Houtekamer and Mitchell 2003, Isaksen et al 2007) Large scale features (data density contrasts, synoptic situation, ) tend to predominate in ensemble-based sigmab maps, which indicates that the signal of interest is large scale.

10 Spatial structure of sampling noise & signal (Isaksen et al 2007) General expectation : increasing the ensemble size reduces sampling noise, whereas the signal remains. N = 10 N = 50 Experimental result : when increasing the ensemble size, small scale details tend to vanish, whereas the large scale part remains. This indicates/confirms that the sampling noise is small scale, and that the signal of interest is large scale.

11 COMPARISON BETWEEN TWO RAW σ b MAPS (Vor( Vor,, 500 hpa) FROM TWO INDEPENDENT 3-MEMBER ENSEMBLES «RAW» σ b ENS #1 «RAW» σ b ENS #2 Differences correspond to sampling noise, which appears to be small scale. Common features correspond to the signal, which appears to be large scale.

12 Optimized spatial averaging of ensemble-based variances

13 OPTIMAL FILTERING OF THE BACKGROUND ERROR VARIANCE FIELD Accounting for spatial structures of signal and noise leads to the application of a low-pass filter ρ (as K in data assim ): ρ V b * ~ ρ V b with ρ = signal / (signal+noise) (Raynaud et al 2008b)

14 OPTIMAL FILTERING OF THE BACKGROUND ERROR VARIANCE FIELD SIMULATED «TRUTH» FILTERED SIGMAB s (N = 6) RAW SIGMAB s (N = 6) (Raynaud et al 2008a)

15 σb ENS 1 «RAW» RESULTS OF THE FILTERING (REAL ENSEMBLE ASSIMILATION) σb ENS 1 «FILTERED» σb ENS 2 «RAW» σb ENS 2 «FILTERED»

16 LINK BETWEEN LOCAL SPATIAL AVERAGING AND INCREASE OF SAMPLE SIZE Multiplication by a low-pass spectral filter latitude Local spatial averaging (convolution) Ng=9 The ensemble size N is MULTIPLIED(!) by a number Ng of gridpoint samples. If N=6 and Ng=9, then the total sample size is N x Ng = 54. The 6-member filtered estimate is as accurate as a 54-member raw estimate, under a local homogeneity asumption. longitude

17 DOES SPATIAL AVERAGING OF VARIANCES Ensemble Var at Météo-France (N=6; operational since July 2008) HAVE AN IMPACT IN THE (VERY) END? with FILTERED σb (Raynaud 2008) with RAW σb obs-analysis obs-guess

18 Validation with spatially averaged innovation-based variances

19 Spatial filtering of innovation-based sigmab s (Lindskog et al 2006) «Raw» innovation-based sigmab s «Filtered» innovation-based sigmab s Some relevant geographical variations (e.g. data density effects), especially after spatial averaging.

20 Innovation-based sigmab estimate (Desroziers et al 2005) cov( H dx, dy ) ~ H B H T This can be calculated for a specific date, to examine flow-dependent features, but then the local sigmab is calculated from a single error realization ( N = 1 )! Conversely, if we calculate local spatial averages of these sigmab s, the sample size will be increased, and comparison with ensemble can be considered.

21 Innovation-based sigmab s «of the day» HIRS 7 (28/08/ h) before spatial averaging after spatial averaging (with a radius of 500 km) (Desroziers 2006) After spatial averaging, some geographical patterns can be identified. Can this be compared with ensemble estimates?

22 Validation of ensemble sigmab s «of the day» HIRS 7 (28/08/ h) Ensemble sigmab s «Innovation-based» sigmab s Spatial averaging makes the two estimates easier to compare and to validate.

23 Spatial averaging of correlations

24 Spatial structure of raw correlation length-scale field RAW L( ε b )= 1/ (-2 d²cor/ds²) s=0 «TRUTH» N = 10 Sampling noise : artificial small scale variations. (Pannekoucke et al 2007)

25 Raw ensemble correlation length-scale field «of the day» Geographical patterns are difficult to identify, due to sampling noise (N = 6).

26 Spatial structure of raw correlation length-scale field N = 5 N = 15 N = 30 Reduction of small scale sampling noise, when the ensemble size increases. N = 60 Sampling noise ~ relatively small scale. Use spatial filtering. ex : wavelets. (Pannekoucke 2008)

27 Wavelet diagonal modelling of B (Fisher 2003, Pannekoucke et al 2007) It amounts to local spatial averages of correlation functions cor(x,s): cor W (x,s) ~ Σx cor(x,s) Φ(x,s) with scale-dependent weighting functions Φ : small-scale contributions to correlation functions are averaged over smaller regions than large-scale contributions.

28 Wavelet filtering of correlation functions RAW WAVELET Wavelet approach : sampling noise is reduced, leading to a lesser need of Schur localization. N = 10 (Pannekoucke et al 2007)

29 Impact of wavelet filtering on analysis quality N = 10 homogeneous raw wavelet Schur Length Wavelet approach : sampling noise is reduced, and there is a lesser need of Schur localization. (Pannekoucke et al 2007)

30 Wavelet filtering of flow-dependent correlations Synoptic situation (geopotential near 500 hpa) Anisotropic wavelet based correlation functions ( N = 12 ) (Lindskog et al 2007, Deckmyn et al 2005)

31 Wavelet filtering of correlations «of the day» Raw length-scales Wavelet length-scales (Fisher 2003, Pannekoucke et al 2007) N = 6

32 Conclusions Spatial structures of signal and sampling noise tend to be different. This leads to «optimal» spatial averaging/filtering techniques (as in data assimilation). The increase of sample size reduces the estimation error (thus it helps to use smaller (high resolution) ) ensembles). Useful for ensemble-based based covariance estimation, but also for validation with flow-dependent innovation-based estimates.

33 Perspectives Make a bridge with similar/other filtering techniques. ex : spatial BMA technique in probab. forecasting (Berrocal et al 2007). ex : ergodic space/time averages in turbulence. ex : local time averages of ensemble covariances (Xu et al 2008). The 4D nature of atmosphere & covariance estimation may suggest 4D filtering techniques (instead of 2D currently). There may be a natural path towards «a data assimilation/filtering» of ensemble- & innovation-based covariances, in order to achieve an optimal covariance estimation?

34 Thank you for your attention!