Main characteristics and performance of COSMO LEPS Andrea Montani, Chiara Marsigli, Tiziana Paccagnella ARPA Emilia Romagna, Idro Meteo Clima Service Montani Marsigli Paccagnella Stochastic forcing, Ensemble prediction and TIGGE @ OSC Montreal August 2014
Introduction to COSMO-LEPS Verification of precipitation Outline trend during the years of operational activity (synop network) verification against high resolution observational network over Northern Italy; comparison with ECMWF EPS Conclusions and plans TIGGE LAM publicity
COSMO LEPS 00/12 UTC 7 km, 40 levels 16 members 132 h
Ics and BCs from 16 members out of two (51+51) ECMWF EPSs The 16 member selection is made by: performing a Cluster Analysis (16 clusters) of the 102 EPS members selecting 16 members as representative of their own cluster Perturbations of the model physics parameters are applied to the 16 COSMO runs Half the members use the Tiedtke and half the members the IFS Bechtold convection schemes The system is run at ECMWF as a critical application and the required computer time is provided by ECMWF member states in COSMO. 00/12 UTC 7 km, 40 levels 16 members 132 h COSMO LEPS
Relocation of COSMO LEPS for the SOCHI Olympics FROST 2014 By Roshydromet COSMO-S14-EPS (run at ECMWF) Horizontal resolution (km) 7 Vertical resolution (ML) 40 Forecast range (h) 72 Ensemble size 10 Starting times (UTC) 00/12 Initial and boundary conditions Interpolated from selected members of ECMWF EPS COSMO-S14-EPS A.Montani; COSMO-LEPS Model perturbations Type of convection Perturbations to the physical parameterizations Parameterised
COSMO-LEPS main changes COSMO LEPS integration domain
COSMO-LEPS main changes COSMO LEPS integration domain Reforecast is performed by MeteoSwiss
Time-series verification of COSMO-LEPS SYNOP on the GTS Main features: variable: 12h cumulated precip (18-06, 06-18 UTC); period : from Dec 2002 to Feb 2014; region: 43-50N, 2-18E (MAP D-PHASE area); method: nearest grid point; no-weighted fcst; obs: synop reports (about 470 stations/day); fcst ranges: 6-18h, 18-30h,, 102-114h, 114-126h; thresholds: 1, 5, 10, 15, 25, 50 mm/12h; system: COSMO-LEPS; scores: ROC area, BSS, RPSS, Outliers, both monthly and seasonal scores were computed
The higher, the better Time series of ROC area (6-month running mean)
The higher, the better Time series of ROC area (6-month running mean) A.Montani; The COSMO-LEPS system.
Ranked Probability Skill Score: time series + seasonal scores (DJF) Useful forecast systems for RPSS > 0. Performance of the system assessed as time series and for the last 4 winters (DJF).
Ranked Probability Skill Score: time series + seasonal scores (DJF) Useful forecast systems for RPSS > 0. Performance of the system assessed as time series and for the last 4 winters (DJF).
Outliers: time series + seasonal scores (DJF) How many times the analysis is out of the forecast interval spanned by the ensemble members. the lower the better Performance of the system assessed as time series (6-month running mean) and for the last 4 winters.
Outliers: time series + seasonal scores (DJF) How many times the analysis is out of the forecast interval spanned by the ensemble members. the lower the better Performance of the system assessed as time series (6-month running mean) and for the last 4 winters.
Some comments Time series verification of COSMO-LEPS indicates a positive trend in terms of ROC, RPSS and Outliers for the probabilistic prediction of 12-h precipitation. Good performance of COSMO-LEPS during past winter.
Now a look to more details and compare with ECMWF EPS high-resolution network Main features: variable: 24h cumulated precip (06-06 UTC); period: from December 2009 to May 2014; region: Northern Italy; method: BOXES (1.0 x 1.0); obs: non-gts network (~1000 stations x day); fcst ranges: 18-42h, 42-66h, 66-90h, 90-114h; thresholds: 1, 5, 10, 15, 25, 50 mm/24h; systems: - COSMO-LEPS (16m, 7 km, 40ML) cleps16 - full EPS (51m, 30 km, 62ML) eps51
OBSERVATION MASK COSMO-LEPS EPS Verification grid
Verification of the distributions The verification made in terms of: Average value Only Maximum value shown here Maximum value 50th percentile (Median) in a box 75 th, 90 th, 95 th percentiles Station observation Grid point forecast two measures of precipitation: the cumulative volume of water deployed over a specific region; the rainfall peaks occurring within the same region.
Time series of Ranked Probability Skill Score maximum values (boxes 1.0 X 1.0) (1) BSS cumulated over all thresholds. RPSS is written as 1-RPS/RPS ref. Sample climate is the reference system. RPS is the extension of the Brier Score to the multi-event situation; useful forecast systems for RPSS > 0 RPSS depends on the ensemble size N and penalises small ensemble sizes. Consider debiased RPSS: RPSS D = 1 (RPS/(RPS ref +RPS ref /N)); a 3-month running mean is applied. RPSS RPSS D
Time series of Ranked Probability Skill Score maximum values (boxes 1.0 X 1.0) (1) BSS cumulated over all thresholds. RPSS is written as 1-RPS/RPS ref. Sample climate is the reference system. RPS is the extension of the Brier Score to the multi-event situation; useful forecast systems for RPSS > 0 RPSS depends on the ensemble size N and penalises small ensemble sizes. Consider debiased RPSS: RPSS D = 1 (RPS/(RPS ref +RPS ref /N)); a 3-month running mean is applied. RPSS RPSS D Problems with raingauge measurements due to the big amaunt of snow A.Montani; The COSMO-LEPS system.
Time series of Ranked Probability Skill Score maximum values (boxes 1.0 X 1.0) (1) BSS cumulated over all thresholds. RPSS is written as 1-RPS/RPS ref. Sample climate is the reference system. RPS is the extension of the Brier Score to the multi-event situation; useful forecast systems for RPSS > 0 RPSS depends on the ensemble size N and penalises small ensemble sizes. Consider debiased RPSS: RPSS D = 1 (RPS/(RPS ref +RPS ref /N)); a 3-month running mean is applied. RPSS RPSS D ECMWF-EPS had initially higher scores; then, COSMO-LEPS has had higher scores than ECMWF-EPS since 2013 in the short range
Box-plot representation of cumulated precipitation in the region. Observed and forecasted precipotation. Precip. cumulated in the range +42 +66 Montani Marsigli Paccagnella OSC Montreal August 2014
Time series of Ranked Probability Skill Score maximum values (boxes 1.0 X 1.0) (2) BSS cumulated over all thresholds. RPSS is written as 1-RPS/RPS ref. Sample climate is the reference system. RPS is the extension of the Brier Score to the multi-event situation; useful forecast systems for +ve RPSS. RPSS depends on the ensemble size N and penalises small ensemble sizes. Consider debiased RPSS: RPSS D = 1 (RPS/(RPS ref +RPS ref /N)) 66-90h RPSS D 90-114h The same for all the forecast ranges.
Time series of Outliers maximum values (boxes 1.0 X 1.0) How many times the analysis is out of the forecast interval spanned by the ensemble members. the lower the better The performances of the systems are assessed for two different forecast ranges (18-42h and 90-114h) More outliers in winters Better performance (fewer outliers) for COSMO-LEPS both in the short and early-medium range
Time series of ROC area maximum values (boxes 1.0 X 1.0) Area under the curve in the HIT rate vs FAR diagram; the higher, the better Valuable forecast systems have ROC area values > 0.6. The performances of the systems are assessed for the event: 10mm of tp in 24h at two forecast ranges. 18-42h 90-114h In the short range, similar performance of the 2 systems throughout the years. For longer ranges, higher skill of ECMWF-EPS.
COSMO LEPS major future developments Initial and Boundary conditions: revision of the Ensemble Reduction procedure to account of the changes of ECMWF EPS Participation, under the C-SRNWP coordination, to a project to optimize the support of ECMWF to member states as regards the provision of Ics and BCs for convective LAM EPS (resolution, frequency) evaluate the impact of using higher resolution BCs from ECMWF EPS has been carried on. Model errors: test of stochastic physics perturbation tendencies approach (Torrisi and Marcucci) Initial conditions perturbations: LETKF? Montani Marsigli Paccagnella OSC Montreal August 2014
Thanks for your attention!