Detective Miss Marple in Murder at the Gallop. 3rd verification workshop, ECMWF 2007, Göber

Transcription

1 m Detective Miss Marple in Murder at the Gallop

2 Slide m Now, this is how we verify the forecasters at DWD: we watch them intensily and when they get it wrong, they'll get a bang over their head. OK, I'm slightly exaggerating and I will explain later what Detective Miss Marple has to do with warning verification. mgoeber, 3//27

3 Comparing peaches and apples On the accuracy of gust warnings issued by forecasters and the accuracy of the model guidance Martin Göber Department Weather Forecasting Deutscher Wetterdienst Acknowledgements: R. Kirchner, T. Kratzsch, D. Richardson, G. Schweigert, S. Tremmel 2

4 3

5 Heidke skill score forecaster Local model p,9,8,7,6,5,4,3,2, near gale gale storm violent storm hurrican force 4

6 hit rate forecaster Local model p,9,8,7,6,5,4,3,2, near gale gale storm violent storm hurrican force 5

7 false alarm ratio forecaster Local model p,9,8,7,6,5,4,3,2, near gale gale storm violent storm hurrican force 6

8 Frequency bias forecaster Local model p near gale gale storm violent storm hurrican force 7

9 relative value for C/L=, forecaster Local model,8,6,4 p,2 -,2 near gale gale storm violent storm hurrican -,4 8

10 relative value for C/L=, forecaster Local model,8,6,4 p,2 -,2 near gale gale storm violent storm hurrican -,4 9

11 Signal Detection Theory

12 all all cases frequency indicator (=CAPE, wind speed, finger prints,...) all cases

13 35 3 event no event all cases 25 frequency event all cases indicator 2

14 frequency event no event all cases False alarms POD=7% FAR=5% ETS=5% Bias=8% misses indicator threshold event all cases 3

15 frequency event no event all cases False alarms POD=9% FAR=4% ETS=42% Bias=5% misses indicator threshold event all cases 4

16 m2 Detective Miss Marple in Murder at the Gallop 5

17 Slide 5 m2 Now, this is how we verify the forecasters at DWD: we watch them intensily and when they get it wrong, they'll get a bang over their head. OK, I'm slightly exaggerating and I will explain later what Detective Miss Marple has to do with warning verification. mgoeber, 3//27

18 Violent storm warning: gusts>= 29 m/s n(f O<29) n(f O>=29) frequency m/s 6

19 Violent storm warning: gusts>= 29 m/s correct NO missed false alarm hit frequency m/s model at face value 7

20 Violent storm warning: gusts>= 29 m/s correct NO missed false alarm hit frequency m/s 8

21 Violent storm warning: gusts>= 29 m/s 3 hit rate 25 false alarm ratio Bias 2 Heidke skill score % 5 hit rate>9% Bias free threshold in m/s 9

22 Re-labeling model bias = forecaster bias model in m/s ----> model gust interpretation for warnings > 4 (near gale) > 8 (gale) > 25 (storm) > 29 (violent storm) > 33 (hurricane force) Verification of heavily biased model? Quite similar to forecaster! 2

23 Relative Operating Characteristics (ROC),9,8,7 forecaster smaller model values= overforecasting H,6,5,4,3,2, larger model values= underforecasting Model at face value model: near gale (>4m/s) forecaster: near gale (>4m/s) no skill,,2,4,6,8, F 2

24 forecaster,9,8,7 overforecasting H,6,5,4,3,2, underforecasting Face value,,2,4,6,8, F model: violent storm forecaster: violent storm no skill model: near gale (>4m/s) forecaster: near gale 22

25 H F near gale gale storm violent storm hurrican force forecaster: gale forecaster: storm forecaster: violent storm forecaster: hurrican force no skill 23

26 My conclusions End user forecast verification: face value (incl. space-time point) Guidance verification: measure potential of the guidance using Fuzzy, Object,..., Signal detection Theory (ROC) 24