Diagnosis of Ensemble Forecasting Systems
|
|
|
- Norah Hart
- 5 years ago
- Views:
Transcription
1 Diagnosis of Ensemble Forecasting Systems M. Leutbecher September 2009 M. Leutbecher Ensemble Forecasting Systems September / 57
2 Diagnosis of Ensemble Forecasting Systems M. Leutbecher September 2009 probability distribution mean variance M. Leutbecher Ensemble Forecasting Systems September / 57
3 Diagnosis of Ensemble Forecasting Systems M. Leutbecher September 2009 probability distribution mean variance Acknowledgements: Renate Hagedorn, Hans Hersbach, Linus Magnusson,..., Thorpex PDP IG5: O. Talagrand, F. Atger,... M. Leutbecher Ensemble Forecasting Systems September / 57
4 1 Introduction 2 Standard diagnostics for ensembles Introduction to standard diagnostics Examples of standard diagnostics Scale-dependent spread-error diagnostic 3 Assessing the spatio-temporal variation of the pdf-shape Spread-reliability TIGGE Changing event type Evaluation of the pdf of a continuous variable Perfect probabilistic forecast? Results with Dressed CF and dressed EM for DJF09 4 Diagnosis & Numerical Experimentation 5 Conclusions M. Leutbecher Ensemble Forecasting Systems September / 57
5 Diagnosis of ensemble forecast systems Why? M. Leutbecher Ensemble Forecasting Systems September / 57
6 Diagnosis of ensemble forecast systems Why? Aid forecast system development: Quantify meteorologically relevant differences between different forecast systems. Identify deficiencies Provide guidance for refining the representation of initial uncertainty and model uncertainty M. Leutbecher Ensemble Forecasting Systems September / 57
7 Diagnosis of ensemble forecast systems Why? Aid forecast system development: Quantify meteorologically relevant differences between different forecast systems. Identify deficiencies Provide guidance for refining the representation of initial uncertainty and model uncertainty Understand dynamics of (initially small) perturbations, i.e. errors, in the global circulation Examine origin of large forecast errors M. Leutbecher Ensemble Forecasting Systems September / 57
8 Diagnosis of ensemble forecast systems Why? Aid forecast system development: Quantify meteorologically relevant differences between different forecast systems. Identify deficiencies Provide guidance for refining the representation of initial uncertainty and model uncertainty Understand dynamics of (initially small) perturbations, i.e. errors, in the global circulation Examine origin of large forecast errors Limitations: not exhaustive, not only new developments, some of the new things are work in progress M. Leutbecher Ensemble Forecasting Systems September / 57
9 Diagnosis of the numerical model used in the ensemble forecast system ensemble forecast model model used for deterministic forecast: resolution, timestep,... look at performance of the control forecast (unperturbed member of ensemble) realism of model climate of perturbed forecast model (including impact of model perturbations) Everything as would be done for the deterministic system (except for the model perturbations). M. Leutbecher Ensemble Forecasting Systems September / 57
10 Getting the climate right: An example freq(pert.) / freq(unpert.) ECMWF EPS uses Stochastically Perturbed Parametrization Tendencies (SPPT) ( stochastic physics ) Operational SPPT ( 35R2) distorts the tail of the climatological distribution of precipitation. A recent major revision of SPPT has improved precipitation distribution n.hem Northern Extra-tropics N40/ to rate (mm/6h) SP BMP SP1 M SP1 L SP2 freq(pert.) / freq(unpert.) tropics Tropics N40/ to rate (mm/6h) Precipitation frequency ratios between forecasts using tendency perturbations and forecasts without tendency perturbations. operational SPPT revised SPPT M. Leutbecher Ensemble Forecasting Systems September / 57 SP BMP SP1 M SP1 L SP2
11 Diagnosis of deterministic forecasts M. Leutbecher Ensemble Forecasting Systems September / 57
12 Diagnosis of probabilistic forecasts M. Leutbecher Ensemble Forecasting Systems September / 57
13 Diagnosis of probabilistic forecasts M. Leutbecher Ensemble Forecasting Systems September / 57
14 Diagnosis of probabilistic forecasts M. Leutbecher Ensemble Forecasting Systems September / 57
15 Diagnosis of probabilistic forecasts M. Leutbecher Ensemble Forecasting Systems September / 57
16 Diagnosis of probabilistic forecasts conclusions for single cases only for exceptional failures forecast and verification are different objects M. Leutbecher Ensemble Forecasting Systems September / 57
17 Diagnosis of probabilistic forecasts: Reliability M. Leutbecher Ensemble Forecasting Systems September / 57
18 Diagnosis of probabilistic forecasts: Reliability M. Leutbecher Ensemble Forecasting Systems September / 57
19 Diagnosis of probabilistic forecasts: Reliability M. Leutbecher Ensemble Forecasting Systems September / 57
20 Diagnosis of probabilistic forecasts: Reliability M. Leutbecher Ensemble Forecasting Systems September / 57
21 Diagnosis of probabilistic forecasts: Sharpness/Resolution M. Leutbecher Ensemble Forecasting Systems September / 57
22 Diagnosis of probabilistic forecasts: Sharpness/Resolution M. Leutbecher Ensemble Forecasting Systems September / 57
23 Diagnosis of probabilistic forecasts: Sharpness/Resolution M. Leutbecher Ensemble Forecasting Systems September / 57
24 Diagnosis of probabilistic forecasts: Sharpness/Resolution M. Leutbecher Ensemble Forecasting Systems September / 57
25 Diagnosis of probabilistic forecasts: Sharpness/Resolution M. Leutbecher Ensemble Forecasting Systems September / 57
26 Diagnosis of probabilistic forecasts: Sharpness/Resolution M. Leutbecher Ensemble Forecasting Systems September / 57
27 Diagnosis of probabilistic forecasts: Sharpness/Resolution M. Leutbecher Ensemble Forecasting Systems September / 57
28 Diagnosis of probabilistic forecasts: Sharpness/Resolution M. Leutbecher Ensemble Forecasting Systems September / 57
29 Diagnosis of probabilistic forecasts: Skill M. Leutbecher Ensemble Forecasting Systems September / 57
30 Diagnosis of probabilistic forecasts: Skill M. Leutbecher Ensemble Forecasting Systems September / 57
31 Diagnosis of probabilistic forecasts: Skill M. Leutbecher Ensemble Forecasting Systems September / 57
32 Standard methods to diagnose the pdf Impractical to assess all aspects of a multivariate probabilistic prediction How can the assessment be simplified? 1 Limit assessment of probability distribution: univariate prediction: e.g. geopotential at 500 hpa binary events: does TC strike at x; prediction of a cold anomaly M. Leutbecher Ensemble Forecasting Systems September / 57
33 Standard methods to diagnose the pdf Impractical to assess all aspects of a multivariate probabilistic prediction How can the assessment be simplified? 1 Limit assessment of probability distribution: univariate prediction: e.g. geopotential at 500 hpa binary events: does TC strike at x; prediction of a cold anomaly 2 Use summary measures of the overall quality of a predicted pdf (or some aspect of it): Skill of the ensemble mean Match between Ens. Mean RMS error and Ensemble Stdev. (reliability) M. Leutbecher Ensemble Forecasting Systems September / 57
34 Standard methods to diagnose the pdf Impractical to assess all aspects of a multivariate probabilistic prediction How can the assessment be simplified? 1 Limit assessment of probability distribution: univariate prediction: e.g. geopotential at 500 hpa binary events: does TC strike at x; prediction of a cold anomaly 2 Use summary measures of the overall quality of a predicted pdf (or some aspect of it): Skill of the ensemble mean Match between Ens. Mean RMS error and Ensemble Stdev. (reliability) Rank Histogram: reliability Brier score, (Continuous) Ranked Probability Score: reliability and resolution M. Leutbecher Ensemble Forecasting Systems September / 57
35 Standard methods to diagnose the pdf Impractical to assess all aspects of a multivariate probabilistic prediction How can the assessment be simplified? 1 Limit assessment of probability distribution: univariate prediction: e.g. geopotential at 500 hpa binary events: does TC strike at x; prediction of a cold anomaly 2 Use summary measures of the overall quality of a predicted pdf (or some aspect of it): Skill of the ensemble mean Match between Ens. Mean RMS error and Ensemble Stdev. (reliability) Rank Histogram: reliability Brier score, (Continuous) Ranked Probability Score: reliability and resolution (decomposition!) Relative Operating Characterisitic (ROC): (discrimination) Logarithmic Score (Ignorance): reliability and resolution M. Leutbecher Ensemble Forecasting Systems September / 57
36 Proper scores strictly proper implies that optimizing the score leads to the correct probability distribution optimization of a score that is not proper is likely to lead to a wrong distribution concise mathematical definitions of proper and strictly proper are available (see Gneiting and Raftery, 2004) examples of proper scores: BS, RPS, CRPS, logarithmic score Fig. 1, Gneiting and Raftery (2004) LINEARLY TRANSFORMED SCORE LogS QS SphS CRPS PS LinS INFLATION FACTOR R Figure 1: Probabilistic sea-level pressure forecasts over the North American Pacific North M. Leutbecher Ensemble Forecasting Systems September / 57
37 Spread-error relationship and ensemble size Assume a perfectly reliable (statistically consistent) M-member ensemble: Ens. members x j, j = 1,..., M and truth y are independent draws from a distribution with mean µ and variance σ 2. M. Leutbecher Ensemble Forecasting Systems September / 57
38 Spread-error relationship and ensemble size Assume a perfectly reliable (statistically consistent) M-member ensemble: Ens. members x j, j = 1,..., M and truth y are independent draws from a distribution with mean µ and variance σ 2. Expected squared error of ensemble mean E 1 M x j y M j=1 }{{} µ+sampling error 2 = ( ) σ 2 M M. Leutbecher Ensemble Forecasting Systems September / 57
39 Spread-error relationship and ensemble size Assume a perfectly reliable (statistically consistent) M-member ensemble: Ens. members x j, j = 1,..., M and truth y are independent draws from a distribution with mean µ and variance σ 2. Expected squared error of ensemble mean E 1 M x j y M j=1 }{{} µ+sampling error 2 = ( ) σ 2 M Expected ensemble variance E 1 M M x j 1 M x k M j=1 k=1 }{{} µ, contains x j 2 = ( 1 1 ) σ 2 M M. Leutbecher Ensemble Forecasting Systems September / 57
40 Spread-error relationship and ensemble size Perfectly reliable M-member ensemble: Ens. members x j, j = 1,..., M and truth y are independent draws from a distribution with mean µ and variance σ 2. For large ensembles, e.g. M = 50, in practice. ensemble variance = squared ensemble mean error M. Leutbecher Ensemble Forecasting Systems September / 57
41 Spread-error relationship and ensemble size Perfectly reliable M-member ensemble: Ens. members x j, j = 1,..., M and truth y are independent draws from a distribution with mean µ and variance σ 2. For large ensembles, e.g. M = 50, in practice. ensemble variance = squared ensemble mean error For smaller ensemble size, e.g. M 20, ( 1 1 ) 1 ( ens. variance = ) 1 squared ens. mean error M M M. Leutbecher Ensemble Forecasting Systems September / 57
42 Spread versus error in hpa geopotential height, N.-Hem. extra-tropics dashed: ens. stdev. solid: EM RMSE 10 member ensembles ECMWF(T L 255L40), MSC, NCEP MAY 2005 B U I Z Z A E T A L. 1 May July 2002 Fig. 5, Buizza et al FIG.5.May Jun Jul 2002 average rms error of the ensemble mean (solid lines) and ensemble standard deviation (dotted lines) of the EC-EPS (gray lines with full circles), the MSC-EPS (black lines with open circles), and the NCEP-EPS (black lines with crosses). Values refer to the 500-hPa geopotential height over the Northern Hemisphere latitudinal band N. lead time again may be due to the use of multiple model initial perturbations are important during the first versions in that ensemble. days, their influence diminishes with increasing l M. Leutbecher Ensemble Forecasting Systems time since the perturbations September 2009 rotate toward 15 / directi 57
43 Spread versus error in 2007 (a) 700 RMSE and spread (m 2 s 2 ) (c) 1200 E and spread (m 2 s 2 ) hpasimulating geopotential, ATMOSPHERIC N.-Hem. VARIABILITY extra-tropics ECMWF, cycle 32r2 cycle 32r3 Northern hemisphere Southern hemisphere (b) 700 RMSE and spread (m 2 s 2 ) Northern hemisphere Spread 100 RMSE Lead time (days) Fig. 12, Bechtold (d) et 1200 al member, T L 399L62Southern hemisphere 69 cases, June-Sept E and spread (m 2 s 2 ) Spread RMSE M. Leutbecher Ensemble Forecasting Systems September / Lead time (days)
44 Spread versus error in 2007 (a) 700 RMSE and spread (m 2 s 2 ) hpasimulating geopotential, ATMOSPHERIC N.-Hem. VARIABILITY extra-tropics ECMWF, cycle 32r2 cycle 32r3 Northern hemisphere (b) 700 (c) 1200 Fig. 12, Bechtold (d) et 1200 al Southern hemisphere 50 member, T L 399L62Southern hemisphere cases, June-Sept Improved 800 match due to revised model physics together 800 with a 30% reduction of the 600 initial perturbation amplitude 600 E and spread (m 2 s 2 ) RMSE and spread (m 2 s 2 ) Northern hemisphere Spread 100 RMSE Lead time (days) E and spread (m 2 s 2 ) Spread RMSE Lead time (days) M. Leutbecher Ensemble Forecasting Systems September / 57
45 Spread versus error in 2007 (a) 700 RMSE and spread (m 2 s 2 ) hpasimulating geopotential, ATMOSPHERIC N.-Hem. VARIABILITY extra-tropics ECMWF, cycle 32r2 cycle 32r3 Northern hemisphere (b) 700 (c) 1200 Fig. 12, Bechtold (d) et 1200 al Southern hemisphere 50 member, T L 399L62Southern hemisphere cases, June-Sept Improved 800 match due to revised model physics together 800 with a 30% reduction of the initial perturbation amplitude Time to retire? E and spread (m 2 s 2 ) RMSE and spread (m 2 s 2 ) Northern hemisphere Spread 100 RMSE Lead time (days) E and spread (m 2 s 2 ) Spread RMSE Lead time (days) M. Leutbecher Ensemble Forecasting Systems September / 57
46 Explore more directions in phase space Agreement for other variables (e.g. T850)? Other regions (e.g. tropics)? Detailed geographical distribution of spread? Different spatial scales? M. Leutbecher Ensemble Forecasting Systems September / 57
47 Example: v850 tropics (20 S 20 N) SPPT revision v850hpa, Tropics spread_em, rmse_em (40) Ens. Mean RMSE + Ens. Stdev RMS fc-step (d) operational SPPT revised SPPT NoTenPert SP BMP SP1 M SP1 L SP2 40 cases Nov/Dec Jul/Aug 2008, T L 399(255 from D+10) M. Leutbecher Ensemble Forecasting Systems September / 57
48 Scale-dependent spread-error diagnostic spread map D+2: unfiltered fields D+2 RMSE Z500 UF DJF region spread : RMSE N N 1.22 Jung and Leutbecher (2008) D+2 Spread Z500 UF DJF 2006 M. Leutbecher Ensemble Forecasting Systems September /
49 Scale-dependent spread-error diagnostic spread map D+2: total wavenumber 8 21 D+2 RMSE Z500 T8-21 DJF region spread : RMSE N N 1.37 Jung and Leutbecher (2008) D+2 Spread Z500 T8-21 DJF 2006 M. Leutbecher Ensemble Forecasting Systems September /
50 Assessing flow-dependent and data-dependent variations in pdf-shape sample size limited: therefore initially focus on 2nd moment of pdf, i.e. variance 1 Stratification by ensemble standard deviation: spread-reliability 2 Modified event definition: EM error > θ, where threshold θ depends on a climatological stdev of the EM error 3 Gaussian centred on CF or EM as reference. Stdev of Gaussian can vary geographically and seasonally. M. Leutbecher Ensemble Forecasting Systems September / 57
51 spread-reliability: methodology M. Leutbecher Ensemble Forecasting Systems September / 57
52 spread-reliability: methodology M. Leutbecher Ensemble Forecasting Systems September / 57
53 spread-reliability: methodology M. Leutbecher Ensemble Forecasting Systems September / 57
54 spread-reliability: methodology M. Leutbecher Ensemble Forecasting Systems September / 57
55 spread-reliability: methodology M. Leutbecher Ensemble Forecasting Systems September / 57
56 spread-reliability: methodology see also Leutbecher and Palmer (2008); Leutbecher et al. (2007) M. Leutbecher Ensemble Forecasting Systems September / 57
57 spread-reliability: Z500 DJF06/07 Z500 Stdev and ens. mean RMSE, 35N 65N, DJF06/07 t = 24 h t = 48 h t = 120 h z500hpa, t=+24h, N.hem.mid DJF2007 z500hpa, t=+48h, N.hem.mid DJF2007 z500hpa, t=+120h, N.hem.mid DJF2007 RMS error (m) RMS spread (m) RMS error (m) RMS spread (m) Fig. 7 from Leutbecher, Buizza & Isaksen (2007) RMS error (m) RMS spread (m) M. Leutbecher Ensemble Forecasting Systems September / 57
58 spread-reliability: Z500 DJF06/07 RMS error (m) Z500 Stdev and ens. mean RMSE, 35N 65N, DJF06/07 t = 24 h t = 48 h t = 120 h z500hpa, t=+24h, N.hem.mid DJF RMS spread (m) RMS error (m) z500hpa, t=+48h, N.hem.mid DJF RMS spread (m) RMS error (m) z500hpa, t=+120h, N.hem.mid DJF RMS spread (m) Fig. 7 from Leutbecher, Buizza & Isaksen (2007) deficiency at the early forecast ranges; reliability improves with lead time M. Leutbecher Ensemble Forecasting Systems September / 57
59 Comparison with other ensembles in TIGGE data provided by Renate Hagedorn direct model output verified with quasi-independent analysis: ERA-Interim period: DJF2008/2009 (0 UTC, 90 start dates) region: N.-Hem midlatitudes (35 65 N) M. Leutbecher Ensemble Forecasting Systems September / 57
60 Spread-reliability: GH 500 hpa TIGGE comparison RMS error (m) gh500hpa, t=+24h, N.hem.mid N90/ TO RMS spread (m) 24 h 48 h Multi-Model CMC ECMWF MetOffice RMS error (m) gh500hpa, t=+48h, N.hem.mid N90/ TO NCEP RMS spread (m) M. Leutbecher Ensemble Forecasting Systems September / 57
61 Ens. stdev and EM RMS error: 500 hpa geopotential TIGGE comparison 120 gh500hpa, Northern Mid-latitudes spread_em, rmse_em (90) RMS Multi-Model CMC ECMWF MetOffice NCEP fc-step (d) + Ens. Mean RMSE Ens. stdev M. Leutbecher Ensemble Forecasting Systems September / 57
62 CRPS: 500 hpa geopotential TIGGE comparison gh500hpa, Northern Mid-latitudes ContinuousRankedProbabilitySkillScore (90) CRPSS Multi-Model CMC ECMWF MetOffice NCEP fc-step (d) M. Leutbecher Ensemble Forecasting Systems September / 57
63 Binary events based on the climate consider a short lead time... M. Leutbecher Ensemble Forecasting Systems September / 57
64 Binary events based on the climate consider a short lead time... M. Leutbecher Ensemble Forecasting Systems September / 57
65 Binary events based on the climate consider a short lead time... M. Leutbecher Ensemble Forecasting Systems September / 57
66 Binary events based on the climate consider a short lead time... M. Leutbecher Ensemble Forecasting Systems September / 57
67 Binary events based on the climate consider a short lead time... M. Leutbecher Ensemble Forecasting Systems September / 57
68 Binary events based on the climate consider an even shorter lead time... M. Leutbecher Ensemble Forecasting Systems September / 57
69 Binary events based on Ens. mean and its error climate M. Leutbecher Ensemble Forecasting Systems September / 57
70 Binary events based on Ens. mean and its error climate M. Leutbecher Ensemble Forecasting Systems September / 57
71 Binary events based on Ens. mean and its error climate M. Leutbecher Ensemble Forecasting Systems September / 57
72 Binary events based on Ens. mean and its error climate scales naturally with lead time, expect to be better suited to diagnose skill of variations in pdf-shape M. Leutbecher Ensemble Forecasting Systems September / 57
73 Binary events based on Ens. mean and its error climate scales naturally with lead time, expect to be better suited to diagnose skill of variations in pdf-shape can also use CF and climate of CF errors... M. Leutbecher Ensemble Forecasting Systems September / 57
74 An error climatology based on reanalyses and reforecasts reforecast started from ERA-40 and operational analyses (reforecasts from ERA-Interim operational since March 2009) 5 members (CF + 4 PF) Ens. mean slightly less accurate 9 weeks centred on week of interest 18 years, once weekly 18 9 = 162 errors errors for climatology computed with ERA-Interim analyses verification for DJF09 with operational analyses M. Leutbecher Ensemble Forecasting Systems September / 57
75 Probability of different kinds of events 48-hour fc of 850 hpa meridional velocity valid at 0 UTC on 31 January hPa **V velocity (Exp: P(x 0001) > - Ensemble µ clim + member σ clim number ) 50 of 51 Thursday 29 January UTC ECMWF EPS Perturbed Forecast t+48 VT: Saturday 31 January UTC M. Leutbecher Ensemble Forecasting Systems September / 57
76 Probability of different kinds of events 48-hour fc of 850 hpa meridional velocity valid at 0 UTC on 31 January hPa **V velocity (Exp: P(x 0001) > - Ensemble µ clim + member σ clim number ) 50 of 51 Thursday 29 January UTC ECMWF EPS Perturbed Forecast t+48 VT: Saturday 31 January UTC Thursday 29 January UTC ECMWF EPS Perturbed Forecast t+48 VT: Saturday 31 January UTC 850hPa P(x **V velocity > EM (Exp: ) σ err - Ensemble ) expect member onnumber average 50 of M. Leutbecher Ensemble Forecasting Systems September / 57
77 Probability of different kinds of events 48-hour fc of 850 hpa meridional velocity valid at 0 UTC on 31 January hPa **V velocity (Exp: P(x 0001) > - Ensemble µ clim + member σ clim number ) 50 of 51 Thursday 29 January UTC ECMWF EPS Perturbed Forecast t+48 VT: Saturday 31 January UTC Thursday 29 January UTC ECMWF EPS Perturbed Forecast t+48 VT: Saturday 31 January UTC 850hPa P(x **V velocity > EM (Exp: ) σ err - Ensemble ) expect member onnumber average 50 of Thursday 29 January UTC ECMWF EPS Perturbed Forecast t+48 VT: Saturday 31 January UTC 850hPa P(x **V velocity > EM (Exp: 0001) σ err - Ensemble ) expect member onnumber average 50 of M. Leutbecher Ensemble Forecasting Systems September / 57
78 Probability of different kinds of events (2) 48-hour fc of 850 hpa meridional velocity valid at 0 UTC on 31 January 2009 P(x > EM + σ err ) and mslp ECMWF Analysis VT:Saturday 31 January UTC Surface: Mean sea level pressure (Exp: 0001 ) Thursday 29 January UTC ECMWF EPS Perturbed Forecast t+48 VT: Saturday 31 January UTC 850hPa **V velocity (Exp: 0001) - Ensemble member number 50 of 51 H H H H H H H H L H L L H L L H H H H L L L L L H H H H L L L L L H H H H H L ECMWF Analysis VT:Saturday 31 January UTC 925hPa **Equivalent H potential temperature Thursday 29 January UTC ECMWF EPS Perturbed Forecast t+48 VT: Saturday 31 January UTC 850hPa P(x **V > velocity EM - + Ensemble σ err ) member andnumber θ e at of 51 hpa H L H L H L H H H H L L H L H L H L H L L H H H L L H L L H L H H L H H L L L H L L H L ECMWF Analysis VT:Saturday 31 January UTC 850hPa U velocity/v velocity Thursday 29 January UTC ECMWF EPS Perturbed Forecast t+48 VT: Saturday 31 January UTC 850hPa P(x > **V EM velocity + - Ensemble σ err ) member and wind number 50 atof hpa m/s M. Leutbecher Ensemble Forecasting Systems September / 57
79 Probabilistic scores for new types of events Brier Score: (p o) 2 M. Leutbecher Ensemble Forecasting Systems September / 57
80 Probabilistic scores for new types of events Brier Score: (p o) 2 Logarithmic Score (Ignorance): (o log(p (W ) ) + (1 o) log(1 p (W ) )), where p (W ) (n) = n + 2/3 [ M + 4/3 2 3M + 4, 3M + 2 ] 3M + 4 with n being the number of members predicting the event and M being the ensemble size. The p (W ) (n) are known as Tukey plotting position; cf. also Cromwell s rule and Wilks (2006). M. Leutbecher Ensemble Forecasting Systems September / 57
81 Probabilistic scores for new types of events Brier Score: (p o) 2 Logarithmic Score (Ignorance): (o log(p (W ) ) + (1 o) log(1 p (W ) )), where p (W ) (n) = n + 2/3 [ M + 4/3 2 3M + 4, 3M + 2 ] 3M + 4 with n being the number of members predicting the event and M being the ensemble size. The p (W ) (n) are known as Tukey plotting position; cf. also Cromwell s rule and Wilks (2006). ROC-area: 1 0 H df [0.5, 1], where H and F denote Hit Rate and False Alarm Rate, respectively. M. Leutbecher Ensemble Forecasting Systems September / 57
82 Logarithmic Score for x > EM + σ err 0.45 v850hpa, anom.>1 stdev (em-err-clim), Northern Mid-latitudes IgnoranceScore, IgnoranceScoreGaussianClimate, IgnoranceScoreQuantileClimate (90) Gaussian error climate Quantile error climate EPS IgnS fc-step (d) M. Leutbecher Ensemble Forecasting Systems September / 57
83 Area under the ROC for x > EM + σ err ROC-Area v850hpa, anom.>1 stdev (em-err-clim), Northern Mid-latitudes ROCarea (90) TMP161ec fc-step (d) M. Leutbecher Ensemble Forecasting Systems September / 57
84 Intermediate Summary: Events relative to EM/CF Overall, scores (BS, IgnS, ROC-area) indicate that EPS has more skill in predicting variations in pdf shape than climatological error pdf However, additional EPS skill tends to be relatively small initially. It increases to max typically around t 4 ± 2 d. Then, additional skill gradually decreases Similar results for T850, Z500, and also for MAM09, and for verification with ERA-Interim analyses (not shown) M. Leutbecher Ensemble Forecasting Systems September / 57
85 Intermediate Summary: Events relative to EM/CF Overall, scores (BS, IgnS, ROC-area) indicate that EPS has more skill in predicting variations in pdf shape than climatological error pdf However, additional EPS skill tends to be relatively small initially. It increases to max typically around t 4 ± 2 d. Then, additional skill gradually decreases Similar results for T850, Z500, and also for MAM09, and for verification with ERA-Interim analyses (not shown) Initial skill increase consistent with fact that spread-error reliability improves with lead time M. Leutbecher Ensemble Forecasting Systems September / 57
86 Intermediate Summary: Events relative to EM/CF Overall, scores (BS, IgnS, ROC-area) indicate that EPS has more skill in predicting variations in pdf shape than climatological error pdf However, additional EPS skill tends to be relatively small initially. It increases to max typically around t 4 ± 2 d. Then, additional skill gradually decreases Similar results for T850, Z500, and also for MAM09, and for verification with ERA-Interim analyses (not shown) Initial skill increase consistent with fact that spread-error reliability improves with lead time Work in progress... What should be expected from a good EPS system? Can we get additional insight by using this technique to compare different ensemble configurations? What can we learn from this for ensemble calibration techniques? M. Leutbecher Ensemble Forecasting Systems September / 57
87 Evaluation of the pdf p(x) of a continuous variable Two proper scores Continuous Ranked Probability Score (CRPS) Continuous Ignorance Score (CIgnS) Two reference forecasts are considered: N(CF, σ 2 err (CF)): score between EPS and N(CF, σerr) 2 evaluates all moments of pdf N(EM, σ 2 err (EM)): score between EPS and N(EM, σerr) 2 assesses 2nd and higher moments of pdf M. Leutbecher Ensemble Forecasting Systems September / 57
88 Evaluation of the pdf p(x) of a continuous variable Two proper scores Continuous Ranked Probability Score (CRPS) Continuous Ignorance Score (CIgnS) Two reference forecasts are considered: N(CF, σ 2 err (CF)): score between EPS and N(CF, σerr) 2 evaluates all moments of pdf N(EM, σ 2 err (EM)): score between EPS and N(EM, σerr) 2 assesses 2nd and higher moments of pdf What difference should be expected? define two kinds of perfect probabilistic forecast an analytical example Results for the operational ECMWF EPS M. Leutbecher Ensemble Forecasting Systems September / 57
89 The Continuous Ranked Probability Score CRPS = Mean squared error of the cumulative distribution P fc cdf of truth P y (x) = P(y x) = H(x y) (1) cdf of forecast P fc (x) = P(x fc x) (2) CRPS = (P fc (x) P y (x)) 2 dx (3) = BS x dx (4) 1.0 (P_fc P_obs)^2 P_fc P_obs 1.0 (P_fc P_obs)^2 P_fc P_obs 1.0 (P_fc P_obs)^2 P_fc P_obs P 0.6 P 0.6 P x x x M. Leutbecher Ensemble Forecasting Systems September / 57
90 The Continuous Ranked Probability Score CRPS = Mean squared error of the cumulative distribution P fc cdf of truth P y (x) = P(y x) = H(x y) (1) cdf of forecast P fc (x) = P(x fc x) (2) CRPS = (P fc (x) P y (x)) 2 dx (3) = BS x dx (4) 1.0 (P_fc P_obs)^2 P_fc P_obs 1.0 (P_fc P_obs)^2 P_fc P_obs 1.0 (P_fc P_obs)^2 P_fc P_obs P 0.6 P 0.6 P x x x equal to Mean Absolute Error for a deterministic forecast M. Leutbecher Ensemble Forecasting Systems September / 57
91 Continuous Ignorance Score or Continuous Logarithmic Score Let y denote truth and p the forecasted probability density CIgnS = log p(y) M. Leutbecher Ensemble Forecasting Systems September / 57
92 Continuous Ignorance Score or Continuous Logarithmic Score Let y denote truth and p the forecasted probability density CIgnS = log p(y) M. Leutbecher Ensemble Forecasting Systems September / 57
93 Continuous Ignorance Score or Continuous Logarithmic Score Let y denote truth and p the forecasted probability density CIgnS = log p(y) For a Gaussian forecast N(µ, σ 2 ), we obtain CIgnS = log(σ 2π) + (y µ)2 2σ 2 M. Leutbecher Ensemble Forecasting Systems September / 57
94 Continuous Ignorance Score or Continuous Logarithmic Score Let y denote truth and p the forecasted probability density CIgnS = log p(y) For a Gaussian forecast N(µ, σ 2 ), we obtain CIgnS = log(σ 2π) + (y µ)2 2σ 2 Mean squared error of reduced centred variable plus logarithmic penalty term for the spread (σ). M. Leutbecher Ensemble Forecasting Systems September / 57
95 Perfect probabilistic forecasts Usually: skill score = 0 as good as climatology skill score = 1 perfect deterministic forecast We may still get closer to 1 but will never reach it! M. Leutbecher Ensemble Forecasting Systems September / 57
96 Perfect probabilistic forecasts Usually: skill score = 0 as good as climatology skill score = 1 perfect deterministic forecast We may still get closer to 1 but will never reach it! Obs. and model uncertainties + perturbation growth characteristics of the atmosphere impose a lower limit on the forecast error variance σ 2 f > 0. M. Leutbecher Ensemble Forecasting Systems September / 57
97 Perfect probabilistic forecasts Usually: skill score = 0 as good as climatology skill score = 1 perfect deterministic forecast We may still get closer to 1 but will never reach it! Obs. and model uncertainties + perturbation growth characteristics of the atmosphere impose a lower limit on the forecast error variance σ 2 f > 0. Define a perfect probabilistic forecast under the constraint v σ 2 f = constant (consider a fixed lead time) M. Leutbecher Ensemble Forecasting Systems September / 57
98 Perfect probabilistic forecasts Usually: skill score = 0 as good as climatology skill score = 1 perfect deterministic forecast We may still get closer to 1 but will never reach it! Obs. and model uncertainties + perturbation growth characteristics of the atmosphere impose a lower limit on the forecast error variance σ 2 f > 0. Define a perfect probabilistic forecast under the constraint v σ 2 f = constant (consider a fixed lead time) M. Leutbecher Ensemble Forecasting Systems September / 57
99 Levels of perfection use label t to refer to different valid times of the forecast (lead time fixed) Perfect dynamic forecast: Perfect flow- and data-dependent variations in pdf-shape p t(x) = p d (x µ t, t) with p d statistically consistent with error of the mean µ t for each t, R given average variance E t x 2 p d (x, t) = v and mean zero for each t. M. Leutbecher Ensemble Forecasting Systems September / 57
100 Levels of perfection use label t to refer to different valid times of the forecast (lead time fixed) Perfect dynamic forecast: Perfect flow- and data-dependent variations in pdf-shape p t(x) = p d (x µ t, t) with p d statistically consistent with error of the mean µ t for each t, R given average variance E t x 2 p d (x, t) = v and mean zero for each t. Perfect static forecast: Constant (or seasonally varying) flow- and data-independent pdf-shape which is perfect: p t(x) = p s(x µ t) with p s statistically consistent with the error of the mean µ t in the time-average sense, and R xp s dx = 0, and R x 2 p s dx = v. M. Leutbecher Ensemble Forecasting Systems September / 57
101 An idealized example with Gaussian distributions Now, focus on variance prediction. Let Ens. Mean error be a random variable distributed according to p (x, t) = 1 σ(t) 2π exp( x 2 2σ 2 (t) ) perfect dynamic forecast: issue p d = N(µ t, σ 2 (t)) perfect static forecast: issue p s = N(µ t, σ 2 ) with σ 2 = E t σ 2 (t) M. Leutbecher Ensemble Forecasting Systems September / 57
102 An idealized example with Gaussian distributions Now, focus on variance prediction. Let Ens. Mean error be a random variable distributed according to p (x, t) = 1 σ(t) 2π exp( x 2 2σ 2 (t) ) perfect dynamic forecast: issue p d = N(µ t, σ 2 (t)) perfect static forecast: issue p s = N(µ t, σ 2 ) with σ 2 = E t σ 2 (t) What is the difference in probabilistic scores (CRPS, CIgnS) between the perfect dynamic forecast and the perfect static forecast? M. Leutbecher Ensemble Forecasting Systems September / 57
103 Expected value of CRPS Let y denote the true value of the EM error. We see that the expected value of the CRPS is E y CRPS(N(0, σf 2 ), y) = σ [ t σ ] f σ 2 π σ f /σ2 t t 1.5 C sqrt(pi) / sigma_t sigma_f / sigma_t The CRPS has a minimum value at σ f = σ t. This is not surprising as the CRPS is a proper score. M. Leutbecher Ensemble Forecasting Systems September / 57
104 Expected value of CIgnS Again let y denote the true value of the EM error. The expected value of the Continuous Ignorance Score is E y CIgnS(N(0, σ 2 f ), y) = 1 2 [ ln(2πσ 2 f ) + (σ t /σ f ) 2] I I_perfect sigma_f / sigma_t The minimum is again at σ f = σ t ; CIgnS is proper! M. Leutbecher Ensemble Forecasting Systems September / 57
105 Two particular distributions of variance Let v = σ 2 denote the variance continuous uniform distribution: v U(v 1, v 2 ) discrete uniform distribution: v 1 2 δ(v v 1) δ(v v 2) Introduce dimensionless parameter δ = v 2 v 1 2v [0, 1] M. Leutbecher Ensemble Forecasting Systems September / 57
106 Expected CRPS for uniform variance distributions CRPS ratio: dynamic forecast / static forecast 1.00 uniform variance distribution C_d / C_s continuous discrete delta M. Leutbecher Ensemble Forecasting Systems September / 57
107 Expected CRPS for uniform variance distributions CRPS ratio: dynamic forecast / static forecast 1.00 uniform variance distribution C_d / C_s continuous discrete delta for 48-hour error doubling: 3% (20%) reduction in CRPS = 2-hour (13-hour) gain in lead time M. Leutbecher Ensemble Forecasting Systems September / 57
108 Expected CIgnS for uniform variance distributions CIgnS( static forecast ) CIgnS( dynamic forecast ) continuous discrete 0.35 I_s I_d delta M. Leutbecher Ensemble Forecasting Systems September / 57
109 Expected CIgnS for uniform variance distributions CIgnS( static forecast ) CIgnS( dynamic forecast ) I_s I_d continuous discrete delta for 48-hour error doubling: reduction of CIgnS by 0.15 (0.7) = 10-hour (48-hour) gain in lead time M. Leutbecher Ensemble Forecasting Systems September / 57
110 Dressed ens. mean forecast: v 850 hpa, N, DJF09 EPS raw prob. for CRPS; Gaussian for CIgnS N(EM, σerr(em)) 2 σ err estimated from reforecasts v850hpa, Northern Mid-latitudes CRPS ContinuousRankedProbabilityScore, ContinuousRankedProbabilityScoreEMWithErrClim (90) 3 CRPS 2 1 TMP161ec fc-step (d) M. Leutbecher Ensemble Forecasting Systems September / 57
111 Dressed ens. mean forecast: v 850 hpa, N, DJF09 EPS raw prob. for CRPS; Gaussian for CIgnS N(EM, σerr(em)) 2 σ err estimated from reforecasts CRPS v850hpa, Northern Mid-latitudes CRPS ContinuousRankedProbabilityScore, ContinuousRankedProbabilityScoreEMWithErrClim (90) fc-step (d) TMP161ec7 0 CIgnS fc-step (d) ContinuousIgnoranceScoreGaussian 1.4 v850hpa, Northern Mid-latitudes ContinuousIgnoranceScoreGaussian, ContinuousIgnoranceScoreEMWithGaussianErrClim (90) M. Leutbecher Ensemble Forecasting Systems September / 57
112 Dressed ens. mean forecast: v 850 hpa, N, DJF09 EPS raw prob. for CRPS; Gaussian for CIgnS N(EM, σerr(em)) 2 σ err estimated from reforecasts v850hpa, Northern Mid-latitudes CRPS ContinuousRankedProbabilityScore, ContinuousRankedProbabilityScoreEMWithErrClim (90) v850hpa, Northern Mid-latitudes CIgnS ContinuousIgnoranceScoreGaussian, ContinuousIgnoranceScoreEMWithGaussianErrClim (90) CRPS fc-step (d) T850, Z500, qualitatively similar but TMP161ec fc-step (d) Deficiencies in the short-range can be addressed via calibration (to a certain extent). ContinuousIgnoranceScoreGaussian 1.4 M. Leutbecher Ensemble Forecasting Systems September / 57
113 Dressed control forecast: v 850 hpa, N, DJF09 EPS raw prob. for CRPS; Gaussian for CIgnS N(CF, σerr(cf)) 2 σ err estimated from reforecasts v850hpa, Northern Mid-latitudes CRPS ContinuousRankedProbabilityScore, ContinuousRankedProbabilityScoreCFWithErrClim (90) v850hpa, Northern Mid-latitudes CIgnS ContinuousIgnoranceScoreGaussian, ContinuousIgnoranceScoreCFWithGaussianErrClim (90) CRPS fc-step (d) ContinuousIgnoranceScoreGaussian 1 TMP161ec fc-step (d) M. Leutbecher Ensemble Forecasting Systems September / 57
114 Diagnosis & Numerical Experimentation Initial uncertainty Deeper understanding from applying diagnostic techniques to clean numerical experimentation designed to answer specific questions M. Leutbecher Ensemble Forecasting Systems September / 57
115 Diagnosis & Numerical Experimentation Initial uncertainty Deeper understanding from applying diagnostic techniques to clean numerical experimentation designed to answer specific questions In the early ranges, say up to day 2, EM dressed with a climatological error distribution as good as or better than EPS. M. Leutbecher Ensemble Forecasting Systems September / 57
116 Diagnosis & Numerical Experimentation Initial uncertainty Deeper understanding from applying diagnostic techniques to clean numerical experimentation designed to answer specific questions In the early ranges, say up to day 2, EM dressed with a climatological error distribution as good as or better than EPS. If CF/EM + past errors provide skilful probabilistic forecasts, then one may ask whether past errors might be a successful EPS perturbation strategy M. Leutbecher Ensemble Forecasting Systems September / 57
117 Flow-independent perturbations Mureau, Molteni & Palmer (1993) initial perturbations based on 6-hour forecast errors from past 30 days & Gram-Schmidt-orthonormalisation assimilation OI model T63 M. Leutbecher Ensemble Forecasting Systems September / 57
118 Flow-independent perturbations Mureau, Molteni & Palmer (1993) initial perturbations based on 6-hour forecast errors from past 30 days & Gram-Schmidt-orthonormalisation assimilation OI model T63 conclusion: SV perturbations are superior Magnusson, Nycander & Källén (2008): flow-independent perts. constructed from scaled differences of randomly picked atmospheric states. M. Leutbecher Ensemble Forecasting Systems September / 57
119 Flow-independent perturbations Mureau, Molteni & Palmer (1993) initial perturbations based on 6-hour forecast errors from past 30 days & Gram-Schmidt-orthonormalisation assimilation OI model T63 conclusion: SV perturbations are superior Magnusson, Nycander & Källén (2008): flow-independent perts. constructed from scaled differences of randomly picked atmospheric states. Initially quite overdispersive in Z500, but skill close to ensemble using operational SV perturbations. M. Leutbecher Ensemble Forecasting Systems September / 57
120 Flow-independent perturbations Mureau, Molteni & Palmer (1993) initial perturbations based on 6-hour forecast errors from past 30 days & Gram-Schmidt-orthonormalisation assimilation OI model T63 conclusion: SV perturbations are superior Magnusson, Nycander & Källén (2008): flow-independent perts. constructed from scaled differences of randomly picked atmospheric states. Initially quite overdispersive in Z500, but skill close to ensemble using operational SV perturbations. Here: use random sample from past 24-hour forecast errors as initial perturbations (advantage: characteristics of short-range fc errors are closer to those of analysis errors than scaled differences of full fields) M. Leutbecher Ensemble Forecasting Systems September / 57
121 Time mean spread vs. RMSE of Ens. mean Meridional wind component (m s 1 ) at 850 hpa, t=48 h singular vector init. perts. Friday 23 November UTC ECMWF EPS Perturbed Forecast t+48 VT: Sunday 25 November UTC 850hPa V velocity (Exp: ezh1) - Ensemble member number 1 of hour fc. error init. perts. Friday 23 November UTC ECMWF EPS Perturbed Forecast t+48 VT: Sunday 25 November UTC 850hPa V velocity (Exp: f0u0) - Ensemble member number 1 of 51 M. Leutbecher Ensemble Forecasting Systems September / 57
122 Time mean spread vs. RMSE of Ens. mean Meridional wind component (m s 1 ) at 850 hpa, t=48 h singular vector init. perts. Friday 23 November UTC ECMWF EPS Perturbed Forecast t+48 VT: Sunday 25 November UTC 850hPa V velocity (Exp: ezh1) - Ensemble member number 1 of hour fc. error init. perts. Friday 23 November UTC ECMWF EPS Perturbed Forecast t+48 VT: Sunday 25 November UTC 850hPa V velocity (Exp: f0u0) - Ensemble member number 1 of 51 Friday 23 November UTC ECMWF EPS Perturbed Forecast t+48 VT: Sunday 25 November UTC 850hPa V velocity (Exp: ezh1) - Ensemble member number 1 of 51 Friday 23 November UTC ECMWF EPS Perturbed Forecast t+48 VT: Sunday 25 November UTC hPa V velocity (Exp: f0u0) - Ensemble member number 1 of 51 8 top: ens. stdev.; bottom: ens. mean RMS error; 50 cases: 23 Nov Feb 08 T L 255, 32r3, unscaled 24-hour FCEs M. Leutbecher Ensemble Forecasting Systems September / 57
123 CRPS difference: FCE SV Meridional wind component (m s 1 ) at 850 hpa, t=48 h Friday 23 November UTC ECMWF EPS Control Forecast t+48 VT: Sunday 25 November UTC 850hPa **V velocity (Exp: f0u0) CRPS (Continuous Ranked Probability Score mean squared error of the cumulative distribution) Blue means EPS based on short-range forecast errors is more skilful. 50 cases: 23 Nov Feb 2008 M. Leutbecher Ensemble Forecasting Systems September /
124 Spread-reliability 850 hpa temperature, N t850hpa, t=+24h, N.hem.mid N50/ TO t850hpa, t=+48h, N.hem.mid N50/ TO RMS error (K) RMS spread (K) RMS error (K) 2 1 SV (oper) FCE24 x RMS spread (K) SV (oper) FCE24 x 1.00 M. Leutbecher Ensemble Forecasting Systems September / 57
125 Conclusions Comparison of Spread and EM-error continues to be an essential tool Have not achieved a well tuned system for all variables and regions. Achieving a reliable distribution of spread in space and time in the early forecast ranges is one of the major challenges for the future. M. Leutbecher Ensemble Forecasting Systems September / 57
126 Conclusions Comparison of Spread and EM-error continues to be an essential tool Have not achieved a well tuned system for all variables and regions. Achieving a reliable distribution of spread in space and time in the early forecast ranges is one of the major challenges for the future. Moment-based decomposition of the ensemble skill has been explored Skill of Ens. mean + skill of pdf of Ens. mean errors Continuous Ignorance Score appears better suited than CRPS to evaluate flow- and data-dependent variations in spread. M. Leutbecher Ensemble Forecasting Systems September / 57
127 Conclusions Comparison of Spread and EM-error continues to be an essential tool Have not achieved a well tuned system for all variables and regions. Achieving a reliable distribution of spread in space and time in the early forecast ranges is one of the major challenges for the future. Moment-based decomposition of the ensemble skill has been explored Skill of Ens. mean + skill of pdf of Ens. mean errors Continuous Ignorance Score appears better suited than CRPS to evaluate flow- and data-dependent variations in spread. Initial perturbations based on past short-range forecast errors challenging benchmark for flow-dependent initial perturbations (e.g. singular vectors). Rare locally large perturbations that are inconsistent with the flow may be an obstacle for operational implementation of the benchmark system. Further diagnostic work in this area is expected to help the development of ensemble prediction system with improved flow-dependent variations of the pdf (in particular in the earlier forecast ranges). M. Leutbecher Ensemble Forecasting Systems September / 57
128 Issues to verify initial uncertainty truth not available short-range fc error correlated with analysis error require obs (or perhaps an independent analysis) obs uncertainty/analysis uncertainty needs to be accounted for if ensemble spread is smaller than or of similar magnitude as the obs/an uncertainty. M. Leutbecher Ensemble Forecasting Systems September / 57
129 Multivariate verification Why? Ensemble of assimilations should provide a varying background error covariance to the assimilation system. One should attempt to verify the flow-dependent covariances. There are user applications that will be dependent on the joint pdf of several variables (e.g. distributed in space or several variables). Implications for suitable ensemble size can be different from univariate case. M. Leutbecher Ensemble Forecasting Systems September / 57
Diagnosis of Ensemble Forecasting Systems
Diagnosis of Ensemble Forecasting Systems artin eutbecher ECWF, Shinfield Park, Reading RG2 9AX, United Kingdom ABSTRACT Ensemble forecasts should strive for quantifying forecast uncertainties as accurately
The ECMWF Ensemble Prediction System
The ECMWF Ensemble Prediction System Design, Diagnosis and Developments Martin eutbecher European Centre for Medium-Range Weather Forecasts MPIPKS Dresden, January 00 Acknowledgements: Renate agedorn,
Ensemble forecasting and flow-dependent estimates of initial uncertainty. Martin Leutbecher
Ensemble forecasting and flow-dependent estimates of initial uncertainty Martin Leutbecher acknowledgements: Roberto Buizza, Lars Isaksen Flow-dependent aspects of data assimilation, ECMWF 11 13 June 2007
Ensemble Verification Metrics
Ensemble Verification Metrics Debbie Hudson (Bureau of Meteorology, Australia) ECMWF Annual Seminar 207 Acknowledgements: Beth Ebert Overview. Introduction 2. Attributes of forecast quality 3. Metrics:
Initial ensemble perturbations - basic concepts
Initial ensemble perturbations - basic concepts Linus Magnusson Acknowledgements: Erland Källén, Martin Leutbecher,, Slide 1 Introduction Perturbed forecast Optimal = analysis + Perturbation Perturbations
4.3.2 Configuration. 4.3 Ensemble Prediction System Introduction
4.3 Ensemble Prediction System 4.3.1 Introduction JMA launched its operational ensemble prediction systems (EPSs) for one-month forecasting, one-week forecasting, and seasonal forecasting in March of 1996,
Probabilistic Weather Forecasting and the EPS at ECMWF
Probabilistic Weather Forecasting and the EPS at ECMWF Renate Hagedorn European Centre for Medium-Range Weather Forecasts 30 January 2009: Ensemble Prediction at ECMWF 1/ 30 Questions What is an Ensemble
Renate Hagedorn 1, Roberto Buizza, Thomas M. Hamill 2, Martin Leutbecher and T.N. Palmer
663 Comparing TIGGE multimodel forecasts with reforecast-calibrated ECMWF ensemble forecasts Renate Hagedorn 1, Roberto Buizza, Thomas M. Hamill 2, Martin Leutbecher and T.N. Palmer Research Department
EMC Probabilistic Forecast Verification for Sub-season Scales
EMC Probabilistic Forecast Verification for Sub-season Scales Yuejian Zhu Environmental Modeling Center NCEP/NWS/NOAA Acknowledgement: Wei Li, Hong Guan and Eric Sinsky Present for the DTC Test Plan and
Standardized Anomaly Model Output Statistics Over Complex Terrain.
Standardized Anomaly Model Output Statistics Over Complex Terrain Reto.Stauffer@uibk.ac.at Outline statistical ensemble postprocessing introduction to SAMOS new snow amount forecasts in Tyrol sub-seasonal
The ECMWF Extended range forecasts
The ECMWF Extended range forecasts Laura.Ferranti@ecmwf.int ECMWF, Reading, U.K. Slide 1 TC January 2014 Slide 1 The operational forecasting system l High resolution forecast: twice per day 16 km 91-level,
NOTES AND CORRESPONDENCE. Improving Week-2 Forecasts with Multimodel Reforecast Ensembles
AUGUST 2006 N O T E S A N D C O R R E S P O N D E N C E 2279 NOTES AND CORRESPONDENCE Improving Week-2 Forecasts with Multimodel Reforecast Ensembles JEFFREY S. WHITAKER AND XUE WEI NOAA CIRES Climate
Current status and future developments of the ECMWF Ensemble Prediction System
Current status and future developments of the ECMWF Ensemble Prediction System Roberto Buizza European Centre for Medium-Range Weather Forecasts EFAS WS, ECMWF, 29-30 Jan 2009 Roberto Buizza: Current status
Verification of Probability Forecasts
Verification of Probability Forecasts Beth Ebert Bureau of Meteorology Research Centre (BMRC) Melbourne, Australia 3rd International Verification Methods Workshop, 29 January 2 February 27 Topics Verification
Upgrade of JMA s Typhoon Ensemble Prediction System
Upgrade of JMA s Typhoon Ensemble Prediction System Masayuki Kyouda Numerical Prediction Division, Japan Meteorological Agency and Masakazu Higaki Office of Marine Prediction, Japan Meteorological Agency
Stochastic methods for representing atmospheric model uncertainties in ECMWF's IFS model
Stochastic methods for representing atmospheric model uncertainties in ECMWF's IFS model Sarah-Jane Lock Model Uncertainty, Research Department, ECMWF With thanks to Martin Leutbecher, Simon Lang, Pirkka
TC/PR/RB Lecture 3 - Simulation of Random Model Errors
TC/PR/RB Lecture 3 - Simulation of Random Model Errors Roberto Buizza (buizza@ecmwf.int) European Centre for Medium-Range Weather Forecasts http://www.ecmwf.int Roberto Buizza (buizza@ecmwf.int) 1 ECMWF
Medium-range Ensemble Forecasts at the Met Office
Medium-range Ensemble Forecasts at the Met Office Christine Johnson, Richard Swinbank, Helen Titley and Simon Thompson ECMWF workshop on Ensembles Crown copyright 2007 Page 1 Medium-range ensembles at
Five years of limited-area ensemble activities at ARPA-SIM: the COSMO-LEPS system
Five years of limited-area ensemble activities at ARPA-SIM: the COSMO-LEPS system Andrea Montani, Chiara Marsigli and Tiziana Paccagnella ARPA-SIM Hydrometeorological service of Emilia-Romagna, Italy 11
Predicting uncertainty in forecasts of weather and climate (Also published as ECMWF Technical Memorandum No. 294)
Predicting uncertainty in forecasts of weather and climate (Also published as ECMWF Technical Memorandum No. 294) By T.N. Palmer Research Department November 999 Abstract The predictability of weather
Exploring and extending the limits of weather predictability? Antje Weisheimer
Exploring and extending the limits of weather predictability? Antje Weisheimer Arnt Eliassen s legacy for NWP ECMWF is an independent intergovernmental organisation supported by 34 states. ECMWF produces
Monthly forecasting system
424 Monthly forecasting system Frédéric Vitart Research Department SAC report October 23 Series: ECMWF Technical Memoranda A full list of ECMWF Publications can be found on our web site under: http://www.ecmwf.int/publications/
TIGGE at ECMWF. David Richardson, Head, Meteorological Operations Section Slide 1. Slide 1
TIGGE at ECMWF David Richardson, Head, Meteorological Operations Section david.richardson@ecmwf.int Slide 1 Slide 1 ECMWF TIGGE archive The TIGGE database now contains five years of global EPS data Holds
Focus on parameter variation results
Accounting for Model Uncertainty in the Navy s Global Ensemble Forecasting System C. Reynolds, M. Flatau, D. Hodyss, J. McLay, J. Moskaitis, J. Ridout, C. Sampson, J. Cummings Naval Research Lab, Monterey,
The Impact of Horizontal Resolution and Ensemble Size on Probabilistic Forecasts of Precipitation by the ECMWF EPS
The Impact of Horizontal Resolution and Ensemble Size on Probabilistic Forecasts of Precipitation by the ECMWF EPS S. L. Mullen Univ. of Arizona R. Buizza ECMWF University of Wisconsin Predictability Workshop,
Application and verification of the ECMWF products Report 2007
Application and verification of the ECMWF products Report 2007 National Meteorological Administration Romania 1. Summary of major highlights The medium range forecast activity within the National Meteorological
The new ECMWF seasonal forecast system (system 4)
The new ECMWF seasonal forecast system (system 4) Franco Molteni, Tim Stockdale, Magdalena Balmaseda, Roberto Buizza, Laura Ferranti, Linus Magnusson, Kristian Mogensen, Tim Palmer, Frederic Vitart Met.
Reanalyses use in operational weather forecasting
Reanalyses use in operational weather forecasting Roberto Buizza ECMWF, Shinfield Park, RG2 9AX, Reading, UK 1 2017: the ECMWF IFS includes many components Model components Initial conditions Forecasts
How far in advance can we forecast cold/heat spells?
Sub-seasonal time scales: a user-oriented verification approach How far in advance can we forecast cold/heat spells? Laura Ferranti, L. Magnusson, F. Vitart, D. Richardson, M. Rodwell Danube, Feb 2012
Evaluation of ensemble forecast uncertainty using a new proper score: Application to medium-range and seasonal forecasts
Quarterly Journal of the Royal Meteorological Society Q. J. R. Meteorol. Soc. (2) DOI:.2/qj.2375 Evaluation of ensemble forecast uncertainty using a new proper score: Application to medium-range and seasonal
Potential Use of an Ensemble of Analyses in the ECMWF Ensemble Prediction System
Potential Use of an Ensemble of Analyses in the ECMWF Ensemble Prediction System Roberto Buizza, Martin Leutbecher and Lars Isaksen European Centre for Medium-Range Weather Forecasts Reading UK www.ecmwf.int
Diagnostics of forecasts for polar regions
Diagnostics of forecasts for polar regions Linus Magnusson ECMWF, Shinfield Park, Reading RG2 9AX, United Kingdom linus.magnusson@ecmwf.int 1 Introduction European Centre for Medium-range Weather Forecasts
Long range predictability of winter circulation
Long range predictability of winter circulation Tim Stockdale, Franco Molteni and Laura Ferranti ECMWF Outline ECMWF System 4 Predicting the Arctic Oscillation and other modes Atmospheric initial conditions
NOTES AND CORRESPONDENCE. On Ensemble Prediction Using Singular Vectors Started from Forecasts
3038 M O N T H L Y W E A T H E R R E V I E W VOLUME 133 NOTES AND CORRESPONDENCE On Ensemble Prediction Using Singular Vectors Started from Forecasts MARTIN LEUTBECHER European Centre for Medium-Range
István Ihász, Máté Mile and Zoltán Üveges Hungarian Meteorological Service, Budapest, Hungary
Comprehensive study of the calibrated EPS products István Ihász, Máté Mile and Zoltán Üveges Hungarian Meteorological Service, Budapest, Hungary 1. Introduction Calibration of ensemble forecasts is a new
Methods of forecast verification
Methods of forecast verification Kiyotoshi Takahashi Climate Prediction Division Japan Meteorological Agency 1 Outline 1. Purposes of verification 2. Verification methods For deterministic forecasts For
The ECMWF Hybrid 4D-Var and Ensemble of Data Assimilations
The Hybrid 4D-Var and Ensemble of Data Assimilations Lars Isaksen, Massimo Bonavita and Elias Holm Data Assimilation Section lars.isaksen@ecmwf.int Acknowledgements to: Mike Fisher and Marta Janiskova
An extended re-forecast set for ECMWF system 4. in the context of EUROSIP
An extended re-forecast set for ECMWF system 4 in the context of EUROSIP Tim Stockdale Acknowledgements: Magdalena Balmaseda, Susanna Corti, Laura Ferranti, Kristian Mogensen, Franco Molteni, Frederic
Verification at JMA on Ensemble Prediction
Verification at JMA on Ensemble Prediction - Part Ⅱ : Seasonal prediction - Yukiko Naruse, Hitoshi Sato Climate Prediction Division Japan Meteorological Agency 05/11/08 05/11/08 Training seminar on Forecasting
AROME-EPS development in Météo-France
AROME-EPS development in Météo-France Int.Conf.Ens.Methods Toulouse, 15 Nov 2012 francois.bouttier@meteo.fr collaborators: Olivier Nuissier, Laure Raynaud, Benoît Vié Ensemble setup Boundary conditions
Exploring ensemble forecast calibration issues using reforecast data sets
NOAA Earth System Research Laboratory Exploring ensemble forecast calibration issues using reforecast data sets Tom Hamill and Jeff Whitaker NOAA Earth System Research Lab, Boulder, CO tom.hamill@noaa.gov
Fernando Prates. Evaluation Section. Slide 1
Fernando Prates Evaluation Section Slide 1 Objectives Ø Have a better understanding of the Tropical Cyclone Products generated at ECMWF Ø Learn the recent developments in the forecast system and its impact
Model error and seasonal forecasting
Model error and seasonal forecasting Antje Weisheimer European Centre for Medium-Range Weather Forecasts ECMWF, Reading, UK with thanks to Paco Doblas-Reyes and Tim Palmer Model error and model uncertainty
The Impacts on Extended-Range Predictability of Midlatitude Weather Patterns due to Recurving Tropical Cyclones
The Impacts on Extended-Range Predictability of Midlatitude Weather Patterns due to Recurving Tropical Cyclones Patrick A. Harr and Heather M. Archambault Naval Postgraduate School, Monterey, CA Hurricane
ECMWF 10 th workshop on Meteorological Operational Systems
ECMWF 10 th workshop on Meteorological Operational Systems 18th November 2005 Crown copyright 2004 Page 1 Monthly range prediction products: Post-processing methods and verification Bernd Becker, Richard
Clustering Techniques and their applications at ECMWF
Clustering Techniques and their applications at ECMWF Laura Ferranti European Centre for Medium-Range Weather Forecasts Training Course NWP-PR: Clustering techniques and their applications at ECMWF 1/32
Diagnostics of the prediction and maintenance of Euro-Atlantic blocking
Diagnostics of the prediction and maintenance of Euro-Atlantic blocking Mark Rodwell, Laura Ferranti, Linus Magnusson Workshop on Atmospheric Blocking 6-8 April 2016, University of Reading European Centre
Developing Operational MME Forecasts for Subseasonal Timescales
Developing Operational MME Forecasts for Subseasonal Timescales Dan C. Collins NOAA Climate Prediction Center (CPC) Acknowledgements: Stephen Baxter and Augustin Vintzileos (CPC and UMD) 1 Outline I. Operational
Forecasting Extreme Events
Forecasting Extreme Events Ivan Tsonevsky, ivan.tsonevsky@ecmwf.int Slide 1 Outline Introduction How can we define what is extreme? - Model climate (M-climate); The Extreme Forecast Index (EFI) Use and
Comparison of a 51-member low-resolution (T L 399L62) ensemble with a 6-member high-resolution (T L 799L91) lagged-forecast ensemble
559 Comparison of a 51-member low-resolution (T L 399L62) ensemble with a 6-member high-resolution (T L 799L91) lagged-forecast ensemble Roberto Buizza Research Department To appear in Mon. Wea.Rev. March
The Canadian approach to ensemble prediction
The Canadian approach to ensemble prediction ECMWF 2017 Annual seminar: Ensemble prediction : past, present and future. Pieter Houtekamer Montreal, Canada Overview. The Canadian approach. What are the
Revisiting predictability of the strongest storms that have hit France over the past 32 years.
Revisiting predictability of the strongest storms that have hit France over the past 32 years. Marie Boisserie L. Descamps, P. Arbogast GMAP/RECYF 20 August 2014 Introduction Improving early detection
Model error and parameter estimation
Model error and parameter estimation Chiara Piccolo and Mike Cullen ECMWF Annual Seminar, 11 September 2018 Summary The application of interest is atmospheric data assimilation focus on EDA; A good ensemble
Recent advances in Tropical Cyclone prediction using ensembles
Recent advances in Tropical Cyclone prediction using ensembles Richard Swinbank, with thanks to Many colleagues in Met Office, GIFS-TIGGE WG & others HC-35 meeting, Curacao, April 2013 Recent advances
Sub-seasonal predictions at ECMWF and links with international programmes
Sub-seasonal predictions at ECMWF and links with international programmes Frederic Vitart and Franco Molteni ECMWF, Reading, U.K. 1 Outline 30 years ago: the start of ensemble, extended-range predictions
Probabilistic verification
Probabilistic verification Chiara Marsigli with the help of the WG and Laurie Wilson in particular Goals of this session Increase understanding of scores used for probability forecast verification Characteristics,
Have a better understanding of the Tropical Cyclone Products generated at ECMWF
Objectives Have a better understanding of the Tropical Cyclone Products generated at ECMWF Learn about the recent developments in the forecast system and its impact on the Tropical Cyclone forecast Learn
Ensemble forecasting
Available online at www.sciencedirect.com Journal of Computational Physics 227 (28) 3515 3539 www.elsevier.com/locate/jcp Ensemble forecasting M. eutbecher *, T.N. Palmer European Centre for Medium-Range
Towards Operational Probabilistic Precipitation Forecast
5 Working Group on Verification and Case Studies 56 Towards Operational Probabilistic Precipitation Forecast Marco Turco, Massimo Milelli ARPA Piemonte, Via Pio VII 9, I-10135 Torino, Italy 1 Aim of the
Effects of observation errors on the statistics for ensemble spread and reliability
393 Effects of observation errors on the statistics for ensemble spread and reliability Øyvind Saetra, Jean-Raymond Bidlot, Hans Hersbach and David Richardson Research Department December 22 For additional
A review on recent progresses of THORPEX activities in JMA
4th THORPEX workshop 31 Oct. 2012, Kunming, China A review on recent progresses of THORPEX activities in JMA Masaomi NAKAMURA Typhoon Research Department Meteorological Research Institute / JMA Contents
Sub-seasonal predictions at ECMWF and links with international programmes
Sub-seasonal predictions at ECMWF and links with international programmes Frederic Vitart and Franco Molteni ECMWF, Reading, U.K. Using ECMWF forecasts, 4-6 June 2014 1 Outline Recent progress and plans
Numerical Weather Prediction. Medium-range multi-model ensemble combination and calibration
Numerical Weather Prediction Medium-range multi-model ensemble combination and calibration Forecasting Research Technical Report No. 517 Christine Johnson and Richard Swinbank c Crown Copyright email:nwp
Verification statistics and evaluations of ECMWF forecasts in
635 Verification statistics and evaluations of ECMWF forecasts in 29-21 D.S. Richardson, J. Bidlot, L. Ferranti, A. Ghelli, T. Hewson, M. Janousek, F. Prates and F. Vitart Operations Department October
Ensemble prediction: A pedagogical perspective
from Newsletter Number 16 Winter 25/6 METEOROLOGY Ensemble prediction: A pedagogical perspective doi:1.21957/ab12956ew This article appeared in the Meteorology section of ECMWF Newsletter No. 16 Winter
ECMWF products to represent, quantify and communicate forecast uncertainty
ECMWF products to represent, quantify and communicate forecast uncertainty Using ECMWF s Forecasts, 2015 David Richardson Head of Evaluation, Forecast Department David.Richardson@ecmwf.int ECMWF June 12,
P3.11 A COMPARISON OF AN ENSEMBLE OF POSITIVE/NEGATIVE PAIRS AND A CENTERED SPHERICAL SIMPLEX ENSEMBLE
P3.11 A COMPARISON OF AN ENSEMBLE OF POSITIVE/NEGATIVE PAIRS AND A CENTERED SPHERICAL SIMPLEX ENSEMBLE 1 INTRODUCTION Xuguang Wang* The Pennsylvania State University, University Park, PA Craig H. Bishop
The WWRP Polar Prediction Project
The Polar Prediction Project Trond Iversen Member of the Polar Prediction Project Steering Group Norwegian Meteorological Institute / ECMWF 11th meeting, THORPEX GIFS-TIGGE, WG; June 2013 1 Background
Recent ECMWF Developments
Recent ECMWF Developments Tim Hewson (with contributions from many ECMWF colleagues!) tim.hewson@ecmwf.int ECMWF November 2, 2017 Outline Last Year IFS upgrade highlights 43r1 and 43r3 Standard web Chart
The benefits and developments in ensemble wind forecasting
The benefits and developments in ensemble wind forecasting Erik Andersson Slide 1 ECMWF European Centre for Medium-Range Weather Forecasts Slide 1 ECMWF s global forecasting system High resolution forecast
Monthly forecast and the Summer 2003 heat wave over Europe: a case study
ATMOSPHERIC SCIENCE LETTERS Atmos. Sci. Let. 6: 112 117 (2005) Published online 21 April 2005 in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/asl.99 Monthly forecast and the Summer 2003
Horizontal resolution impact on short- and long-range forecast error
Quarterly Journal of the Royal Meteorological Society Q. J. R. Meteorol. Soc. :, April Part B Horizontal resolution impact on short- and long-range forecast error Roberto Buizza European Centre for Medium-Range
Feature-specific verification of ensemble forecasts
Feature-specific verification of ensemble forecasts www.cawcr.gov.au Beth Ebert CAWCR Weather & Environmental Prediction Group Uncertainty information in forecasting For high impact events, forecasters
A three-dimensional multivariate modal analysis of atmospheric predictability with application to the ECMWF ensemble
77 A three-dimensional multivariate modal analysis of atmospheric predictability with application to the ECMWF ensemble Nedjeljka Žagar, Roberto Buizza and Joseph Tribbia Research Department University
Stochastic Parametrization and Model Uncertainty
598 Stochastic Parametrization and Model Uncertainty Palmer, T.N., R. Buizza, F. Doblas-Reyes, T. Jung, M. Leutbecher, G.J. Shutts, M. Steinheimer, A. Weisheimer Research Department October 8, 9 Series:
Synoptic systems: Flowdependent. predictability
Synoptic systems: Flowdependent and ensemble predictability Federico Grazzini ARPA-SIMC, Bologna, Italy Thanks to Stefano Tibaldi and Valerio Lucarini for useful discussions and suggestions. Many thanks
Ensemble-variational assimilation with NEMOVAR Part 2: experiments with the ECMWF system
Ensemble-variational assimilation with NEMOVAR Part 2: experiments with the ECMWF system La Spezia, 12/10/2017 Marcin Chrust 1, Anthony Weaver 2 and Hao Zuo 1 1 ECMWF, UK 2 CERFACS, FR Marcin.chrust@ecmwf.int
NCEP Global Ensemble Forecast System (GEFS) Yuejian Zhu Ensemble Team Leader Environmental Modeling Center NCEP/NWS/NOAA February
NCEP Global Ensemble Forecast System (GEFS) Yuejian Zhu Ensemble Team Leader Environmental Modeling Center NCEP/NWS/NOAA February 20 2014 Current Status (since Feb 2012) Model GFS V9.01 (Spectrum, Euler
15 day VarEPS introduced at. 28 November 2006
Comprehensive study of the calibrated EPS products István Ihász Hungarian Meteorological Service Thanks to Máté Mile Zoltán Üveges & Gergő Kiss Mihály Szűcs Topics 15 day VarEPS introduced at the ECMWF
NCEP ENSEMBLE FORECAST SYSTEMS
NCEP ENSEMBLE FORECAST SYSTEMS Zoltan Toth Environmental Modeling Center NOAA/NWS/NCEP Acknowledgements: Y. Zhu, R. Wobus, M. Wei, D. Hou, G. Yuan, L. Holland, J. McQueen, J. Du, B. Zhou, H.-L. Pan, and
3.6 NCEP s Global Icing Ensemble Prediction and Evaluation
1 3.6 NCEP s Global Icing Ensemble Prediction and Evaluation Binbin Zhou 1,2, Yali Mao 1,2, Hui-ya Chuang 2 and Yuejian Zhu 2 1. I.M. System Group, Inc. 2. EMC/NCEP AMS 18th Conference on Aviation, Range,
25 years of ensemble forecasting at ECMWF
from Newsletter Number 153 Autumn 2017 METEOROLOGY 25 years of ensemble forecasting at ECMWF 25 YEARS OF ENSEMBLE PREDICTION doi:10.21957/bv418o This article appeared in the Meteorology section of ECMWF
LAMEPS Limited area ensemble forecasting in Norway, using targeted EPS
LAMEPS Limited area ensemble forecasting in Norway, using targeted EPS Inger-Lise Frogner, Marit H. Jensen, Hilde Haakenstad and Ole Vignes Limited area ensemble forecasting in Norway - outline Ensembles
Evaluating Forecast Quality
Evaluating Forecast Quality Simon J. Mason International Research Institute for Climate Prediction Questions How do we decide whether a forecast was correct? How do we decide whether a set of forecasts
Precipitation verification. Thanks to CMC, CPTEC, DWD, ECMWF, JMA, MF, NCEP, NRL, RHMC, UKMO
Precipitation verification Thanks to CMC, CPTEC, DWD, ECMWF, JMA, MF, NCEP, NRL, RHMC, UKMO Outline 1) Status of WGNE QPF intercomparisons 2) Overview of the use of recommended methods for the verification
Assessment of the status of global ensemble prediction
Assessment of the status of global ensemble prediction Roberto Buizza (1), Peter L. Houtekamer (2), Zoltan Toth (3), Gerald Pellerin (2), Mozheng Wei (4), Yueian Zhu (3) (1) European Centre for Medium-Range
Assessment of Ensemble Forecasts
Assessment of Ensemble Forecasts S. L. Mullen Univ. of Arizona HEPEX Workshop, 7 March 2004 Talk Overview Ensemble Performance for Precipitation Global EPS and Mesoscale 12 km RSM Biases, Event Discrimination
Model verification / validation A distributions-oriented approach
Model verification / validation A distributions-oriented approach Dr. Christian Ohlwein Hans-Ertel-Centre for Weather Research Meteorological Institute, University of Bonn, Germany Ringvorlesung: Quantitative
How ECMWF has addressed requests from the data users
How ECMWF has addressed requests from the data users David Richardson Head of Evaluation Section, Forecast Department, ECMWF David.richardson@ecmwf.int ECMWF June 14, 2017 Overview Review the efforts made
Clustering Forecast System for Southern Africa SWFDP. Stephanie Landman Susanna Hopsch RES-PST-SASAS2014-LAN
Clustering Forecast System for Southern Africa SWFDP Stephanie Landman Susanna Hopsch Introduction The southern Africa SWFDP is reliant on objective forecast data for days 1 to 5 for issuing guidance maps.
Improvements in IFS forecasts of heavy precipitation
from Newsletter Number 144 Suer 215 METEOROLOGY Improvements in IFS forecasts of heavy precipitation cosmin4/istock/thinkstock doi:1.21957/jxtonky This article appeared in the Meteorology section of ECMWF
Representing model error in ensemble data assimilation
Nonlin. Processes Geophys., 21, 971 985, 214 www.nonlin-processes-geophys.net/21/971/214/ doi:1.5194/npg-21-971-214 Author(s) 214. CC Attribution 3. License. Representing model error in ensemble data assimilation
Main characteristics and performance of COSMO LEPS
Main characteristics and performance of COSMO LEPS Andrea Montani, Chiara Marsigli, Tiziana Paccagnella ARPA Emilia Romagna, Idro Meteo Clima Service Montani Marsigli Paccagnella Stochastic forcing, Ensemble
Probabilistic predictions of monsoon rainfall with the ECMWF Monthly and Seasonal Forecast Systems
Probabilistic predictions of monsoon rainfall with the ECMWF Monthly and Seasonal Forecast Systems Franco Molteni, Frederic Vitart, Tim Stockdale, Laura Ferranti, Magdalena Balmaseda European Centre for
A simple method for seamless verification applied to precipitation hindcasts from two global models
A simple method for seamless verification applied to precipitation hindcasts from two global models Matthew Wheeler 1, Hongyan Zhu 1, Adam Sobel 2, Debra Hudson 1 and Frederic Vitart 3 1 Bureau of Meteorology,
Calibration of extreme temperature forecasts of MOS_EPS model over Romania with the Bayesian Model Averaging
Volume 11 Issues 1-2 2014 Calibration of extreme temperature forecasts of MOS_EPS model over Romania with the Bayesian Model Averaging Mihaela-Silvana NEACSU National Meteorological Administration, Bucharest
Variational ensemble DA at Météo-France Cliquez pour modifier le style des sous-titres du masque
Cliquez pour modifier le style du titre Variational ensemble DA at Météo-France Cliquez pour modifier le style des sous-titres du masque L. Berre, G. Desroziers, H. Varella, L. Raynaud, C. Labadie and
Verification of ensemble and probability forecasts
Verification of ensemble and probability forecasts Barbara Brown NCAR, USA bgb@ucar.edu Collaborators: Tara Jensen (NCAR), Eric Gilleland (NCAR), Ed Tollerud (NOAA/ESRL), Beth Ebert (CAWCR), Laurence Wilson
Forecasting precipitation for hydroelectric power management: how to exploit GCM s seasonal ensemble forecasts
INTERNATIONAL JOURNAL OF CLIMATOLOGY Int. J. Climatol. 27: 1691 1705 (2007) Published online in Wiley InterScience (www.interscience.wiley.com).1608 Forecasting precipitation for hydroelectric power management:
The Use of a Self-Evolving Additive Inflation in the CNMCA Ensemble Data Assimilation System
The Use of a Self-Evolving Additive Inflation in the CNMCA Ensemble Data Assimilation System Lucio Torrisi and Francesca Marcucci CNMCA, Italian National Met Center Outline Implementation of the LETKF