Spatial Verification. for Ensemble. at DWD

Transcription

1 Spatial Verification for Ensemble at DWD Susanne Theis Deutscher Wetterdienst (DWD)

2 Introduction Spatial Verification of Single Simulations Box Statistics (e.g. averaging) Fraction Skill Score Wavelets (Base Transformation) MODE, CRA, SAL, DAS

3 Introduction Spatial Verification of Single Simulations Spatial Verification of Ensemble Simulations Box Statistics (e.g. averaging) Fraction Skill Score Wavelets (Base Transformation) MODE, CRA, SAL, DAS

4 Introduction Spatial Verification of Single Simulations Spatial Verification of Ensemble Simulations Box Statistics (e.g. averaging) Fraction Skill Score Wavelets (Base Transformation) MODE, CRA, SAL, DAS Verification of Spatial Ensemble Products

5 Introduction Spatial Verification of Single Simulations Spatial Verification of Ensemble Simulations Box Statistics (e.g. averaging) Fraction Skill Score Wavelets (Base Transformation) MODE, CRA, SAL, DAS Verification of Spatial Ensemble Products scale-related multivariate, i.e. several locations base transformation object-oriented

6 Introduction Spatial Verification of Single Simulations Spatial Verification of Ensemble Simulations Box Statistics (e.g. averaging) Fraction Skill Score Wavelets (Base Transformation) MODE, CRA, SAL, DAS Verification of Spatial Ensemble Products Box Statistics e.g. Marsigli et al. (2008)

7 History: Spatial Ensemble Products at DWD 2009: Awareness, Implementation 2011: Forecaster s Feedback and Verification Results since 2012: Operational 2015: Evaluation in European Severe Storms Laboratory 2015: Information on DWD Web Site 2015: Discovering Similar Issues in Renewable Energies 2016: Discussion at WWRP Working Group on Predictability

8 Awareness (2009)

9 EPS Product Example: Probability Maps Event somewhere in 2,8km-Box % Source: EWGLAM / SRNWP meeting, Athens, 2009

10 EPS Product Example: Probability Maps Event somewhere in 2,8km-Box Forecasters: Probabilities are too low! % Source: EWGLAM / SRNWP meeting, Athens, 2009

11 EPS Product Example: Probability Maps Event somewhere in 2,8km-Box Reason: Forecasters are used to larger reference regions Experiment: Forecasters: Probabilities are too low! % present probabilities on coarser grid Source: EWGLAM / SRNWP meeting, Athens, 2009

12 EPS Product Example: Probability Maps Event somewhere in 2,8km-Box Event somewhere in 28km-Box % Source: EWGLAM / SRNWP meeting, Athens, 2009

13 EPS Product Example: Probability Maps Event somewhere in 2,8km-Box Event somewhere in 28km-Box % Relevance of reference region! Source: EWGLAM / SRNWP meeting, Athens, 2009

14 EPS Product Example: Probability Maps Event somewhere in 2,8km-Box Event somewhere in 28km-Box Implementation of a Spatial Ensemble Relevance Productof % reference region!

15 Forecaster s Feedback at DWD (2011)

16 Forecasters Feedback what they prefer to use: 90%-quantile of precipitation precipitation probabilities for an area (10x10 grid points) Source: COSMO General Meeting, Rome, 2011

17 Forecasters Feedback what they prefer to use: 90%-quantile of precipitation precipitation probabilities for an area (10x10 grid points) Key Product of COSMO-DE-EPS

18 Verification Results (2011)

19 Verification Results Ben Bouallègue, Z. (2011): Upscaled and fuzzy probabilistic forecasts: verification results. COSMO Newsletter 11, Ben Bouallègue, Z. and S.E. Theis (2014): Spatial Techniques Applied to Precipitation Ensemble Forecasts: From Verification Results to Probabilistic Products. Meteorological Applications, 21,

20 Verification Results Observations: Radar data, upscaled in the same way Ben Bouallègue and Theis (2014)

21 Verification Results upscaling substantial quality gain (looking at ROC area, high precipitation thresholds) optimal window size: balance between good verification results and access to fine-grid information Ben Bouallègue and Theis (2014)

22 Verification Results upscaling substantial quality gain (looking at ROC area, high precipitation thresholds) optimal window size: balance between good verification results Looked at Quality of Spatial Ensemble Product and access to fine-grid information Ben Bouallègue and Theis (2014)

23 History: Spatial Ensemble Products at DWD 2009: Awareness, Implementation 2011: Forecaster s Feedback and Verification Results since 2012: Operational 2015: Evaluation in European Severe Storms Laboratory 2015: Information on DWD Web Site 2015: Discovering Similar Issues in Renewable Energies 2016: Discussion at WWRP Working Group on Predictability

24 Evaluation at ESSL (2015)

25 Evaluation at ESSL simulated reflectivity, 06 UTC Run on May 12th, + 15 hours forecast fraction of members > 40 dbz fraction of members > 40 dbz anywhere within a square

26 Evaluation at ESSL simulated reflectivity, 06 UTC Run on May 12th, + 15 hours forecast fraction of members > 40 dbz fraction of members > 40 dbz anywhere within a square somewhat useful very favourably received

27 Evaluation at ESSL simulated reflectivity, 06 UTC Run on May 12th, + 15 hours forecast fraction of members > 40 dbz fraction of members > 40 dbz anywhere within a square somewhat useful very favourably received

28 Evaluation at ESSL simulated reflectivity, 06 UTC Run on May 12th, + 15 hours forecast fraction of members > 40 dbz fraction of members > 40 dbz anywhere within a square somewhat useful very favourably received smoothed version somewhat related to - Fraction Skill Score - Neighbourhood Method

29 Evaluation at ESSL simulated reflectivity, 06 UTC Run on May 12th, + 15 hours forecast fraction of members > 40 dbz fraction of members > 40 dbz anywhere within a square somewhat useful very favourably received smoothed version somewhat related to - Fraction Skill Score - Neighbourhood Method smoothing out the signal to below the lowest threshold

30 Evaluation at ESSL simulated reflectivity, 06 UTC Run on May 12th, + 15 hours forecast fraction of members > 40 dbz fraction of members > 40 dbz anywhere within a square somewhat useful very favourably received smoothed version despite critical evaluation, verification result still favourable Precip thres: 10mm/6h, Reference: COSMO-DE-EPS (Ben Bouallègue and Theis, 2014)

31 Evaluation at ESSL simulated reflectivity, 06 UTC Run on May 12th, + 15 hours forecast fraction of members > 40 dbz fraction of members > 40 dbz anywhere within a square somewhat useful very favourably received smoothed version verification still favourable Relevance of Evaluation Precip thres: 10mm/6h, Reference: COSMO-DE-EPS (Ben Bouallègue and Theis, 2014)

32 Information on DWD web site (2015)

33 English (klick on En at the top) RESEARCH WEATHER FORECASTING NUMERICAL MODELLING ENSEMBLE METHODS ENSEMBLE PREDICTION (Figure: EMS Opening Session, 2011)

34 English (klick on En at the top) RESEARCH WEATHER FORECASTING NUMERICAL MODELLING ENSEMBLE METHODS ENSEMBLE PREDICTION

35 Similar Issues in Forecasts for Renewable Energies (2015)

36 Renewable Energies Probabilistic Wind Ramp Forecast Colors: Probability of Windramps based on COSMO-DE-EPS forecasts Blue dots: Installed Electric Power per County Figure: Marcel Schäfer, Master Thesis, University of Mainz

37 Amplitude Renewable Energies Verification of Wind Ramps: Wind Ramps directly Spectrum of Time Series Correlation in Time Multivariate Scores: p-variogram Score, Multivariate Rank Histogram Frequency member 1 obs Energy Score Ben Bouallègue et al. (2016), submitted

38 Discussion at WWRP Working Group on Predictability (2016)

39 km-scale.probabilistic high-impact. Default Product: Probabilities for each grid point and time Source: WWRP PDEF WG meeting, Exeter, 2016

40 km-scale.probabilistic high-impact. Default Product: Probabilities for each grid point and time Questions: Does high-impact involve any spatial or temporal structure? Are we interested in the rough time and location? High-impact Event not necessarily visible in standard probabilities Source: WWRP PDEF WG meeting, Exeter, 2016

41 km-scale.probabilistic high-impact. Default Product: Probabilities for each grid point and time verification Questions: optimization Does high-impact involve any spatial or temporal structure? Are we interested in the rough time and location? High-impact Event not necessarily visible in standard probabilities Source: WWRP PDEF WG meeting, Exeter, 2016

42 km-scale.probabilistic high-impact. Default Product: Probabilities for each grid point and time verification event of interest? Questions: optimization Does high-impact involve any spatial or temporal structure? Are we interested in the rough time and location? High-impact Event not necessarily visible in standard probabilities Source: WWRP PDEF WG meeting, Exeter, 2016

43 km-scale.probabilistic high-impact. Default Product: Probabilities for each grid point and time verification event of interest? Questions: optimization Does high-impact involve any spatial or temporal structure? Are we interested in the rough time and location? thoughtful definition of event + targeted verification High-impact Event not necessarily visible in standard probabilities