Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 1 Drought forecasting methods Blaz Kurnik DESERT Action JRC
Motivations for drought forecasting Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 2 Transport and energy Crop production Water resources management Health and diseases Tourism and pleasure
Methods of drought forecasting Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 3 STATISTICAL Based on observed historical correlations between global variables and regional weather patterns NUMERICAL Based on the use of WPMs easy to calculate (no need of the complicated physical models) Physical equations used for describing the connections between variables Probabilistic approach using number of model simulations historical correlation between global variables and regional climate is stationary (Climate changing!!??) Chaotic behaviour of the climate systems produce uncertainty for long range forecasts
Time scale for drought forecasting Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 4 Creeping phenomena long range forecast needed!! Starting 06/2003 Ending 09/2003 Starting 04/2007 Ending 09/2007
Probabilistic or deterministic approach Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 5 Drought indicator probabilistic Time Deterministic
Hydrological prediction system for drought forecasting Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 6 Meteorological forecasts Deterministic forecasts Ensemble forecasts Meteorological observations Hydrological model Land cover, soil properties, topography, Hydrological model Meteorological observations Forecasted soil moisture Probability of exceeding SM threshold Probability of exceeding index threshold up to 10 days 5 km resolution monthly forecasts 25-50 km resolution
Probabilistic or deterministic approach Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 7 Deterministic weather forecasts are rarely used for drought forecasting because of the additional unpredictable uncertainty beyond few days. No added value for better decision making Using ensemble predictions allows (some) quantification of the uncertainty
Model or Index Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 8 Production of soil moisture information: the LISFLOOD model Standardized Precipitation Index - SPI calculations A physically based distributed rainfall-runoff model programmed in a dynamic GIS-language The SPI is an index based on the probability of recording a given amount of precipitation, and the probabilities are standardized so that an index of zero indicates the median precipitation amount McKee et al. 1993 Calculation of the soil moisture important drought indicator Complex model which needs a lot of validation and calibration for different regions Easy to compute (historical and operational precipitation data) Drought is calculated using only precipitations
SPI-3 forecasting using ensemble ECMWF monthly forecasts Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 9 1 2 3 ERA-40 2 months precipitations over Europe 50 ensemble precipitation monthly ECMWF forecasts over Europe SPI calculation using historical (1975-2004) ERA-40 time series.. between 2005 and 2008 48 49 50
Going from ensembles to probability Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 10 50 possible states of the forecast Probabilistic approach (calculating the probability for exceeding thresholds )
Probabilistic approach for SPI-3 forecasts Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 11 Probability that SPI-3 for the next month is severe dry or worse (SPI < -1.5) 1-month probabilistic forecast of SPI-3 (based on ERA-40 and monthly ensemble ECMWF forecasts)
Data availability Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 12 ERA-40 data Directly from ECMWF ecaccess server using retrieve scripts Special account needed Indirectly from JRC-FOODSEC page (free) http://marsfood.jrc.it/meteo/frameview.phtml ECMWF monthly forecasts Directly from ECMWF ecaccess server using retrieve scripts Special account needed GPCC precipitation data Directly from DWD server (free) http://gpcc.dwd.de
Forecasted probability vs. observed SPI-3 Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 13 Forecasts (probability for SPI<-1.5) based on the ERA-40 and monthly ensemble forecasts Observations (SPI<-1.5 - severe&extreme dry) based on the GPCC precipitation gridded measurements
Climatological probability vs. observed SPI-3 Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 14 Forecasts (probability for SPI<-1.5) based on the GPCC precipitation database historical probability Observations of SPI-3 based on the GPCC precipitation database
Validation of the probabilistic forecasts Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 15 Direct validation of probability forecasts Conversion to the categorical forecasts Probability threshold For N Y Obs Y N a c b d Contingency table Brier score (BS) Brier skill score (BSS) Proportion correct (PC) Hit rate (HR) False alarm rate (FAR) Pierce s skill score (PSS) Relative Operation Characteristic - ROC
Indirect validation conversion to the CT Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 16 If (prob of SPI-3 >= 60 % ) then FORECAST = 1 Obs 1 0 If (prob of SPI-3 < 60 % ) then FORECAST = 0 If (SPI-3 <= -1.5 ) then OBSERVATION = 1 For 0 1 a c b d Contingency table If (SPI-3 > -1.5 ) then OBSERVATION = 0 counting over all grid cells Proportion correct (PC) Hit rate (Probability of detection) False alarm rate PC = a + d n HR = a a + c FAR = b b + d
Indirect validation conversion to the CT Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 17 If (prob of SPI-3 >= 60 % ) then FORECAST = 1 Obs 1 0 If (prob of SPI-3 < 60 % ) then FORECAST = 0 If (SPI-3 <= -1.5 ) then OBSERVATION = 1 For 0 1 a c b d Contingency table If (SPI-3 > -1.5 ) then OBSERVATION = 0 counting over all grid cells Relative Operation Characteristic - ROC plotting HR against FAR Pierce s skill score or Kuipers skill score PSS = HR FAR hr decision thresholds far PSS = ad bc ( a + c)( b + d)
Direct validation of probability forecasts Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 18 Brier score (BS) : quadratic scoring measure BS = 1 n n j= 1 ( p j x j ) 2 p j = probability (SPI-3 <= -1.5) if (SPI-3 <= -1.5) ; 1 X j = if (SPI-3 > -1.5) ; 0 BS = 0 : perfect forecast BS = 1 : systematically erroneous forecast
Direct validation of probability forecasts Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 19 Brier skill score (BSS) BSS = 1 BS BS ref if ( BS < BS ref ); BSS > 0 If ( BS >= BS ref ); BSS <=0 BSS = 1 : perfect skilled forecast BSS < 0 : poorer than reference system BSS - inf
What is a reference system? Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 20 Reference system can be ensemble of climatological forecasts, where probability is base rate (climatological) probability of occurrence of the event. Climatological probabilistic forecasts are often assumed as a low skill forecasts.
How climatological SPI-3 probabilistic forecasts are produced? Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 21 1 2 3 2 months precipitations observation over Europe 33 years of precipitations (33 years of data between 1975 2008) SPI calculation using historical (1975-2004) time series.. between 2005 and 2008 31 32 33
Going from historical events to probability Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 22 33 possible states of the forecast Probabilistic approach (calculating the probability for exceeding thresholds, based on historical probability )
Results: Contingency table measures Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 23 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 Proportion correct: EPS Proportion correct: CLI 2005/ 01 2005/ 02 2005/ 03 2005/ 04 2005/ 05 2005/ 06 2005/ 07 2005/ 08 2005/ 09 2005/ 10 2005/ 11 2005/ 12 2006/ 01 2006/ 02 2006/ 03 2006/ 04 2006/ 05 2006/ 06 2006/ 07 2006/ 08 2006/ 09 2006/ 10 2006/ 11 2006/ 12 2007/ 01 2007/ 02 2007/ 03 2007/ 04 2007/ 05 2007/ 06 2007/ 07 2007/ 08 2007/ 09 2007/ 10 2007/ 11 2007/ 12 2008/ 01 2008/ 02 2008/ 03 2008/ 04 2008/ 05 2008/ 06 2008/ 07 2008/ 08 2008/ 09 2008/ 10 2008/ 11 2008/ 12
1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 Results: Contingency table measures Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 24 Hit Rate: EPS Hit Rate: CLI 2005/ 01 2005/ 02 2005/ 03 2005/ 04 2005/ 05 2005/ 06 2005/ 07 2005/ 08 2005/ 09 2005/ 10 2005/ 11 2005/ 12 2006/ 01 2006/ 02 2006/ 03 2006/ 04 2006/ 05 2006/ 06 2006/ 07 2006/ 08 2006/ 09 2006/ 10 2006/ 11 2006/ 12 2007/ 01 2007/ 02 2007/ 03 2007/ 04 2007/ 05 2007/ 06 2007/ 07 2007/ 08 2007/ 09 2007/ 10 2007/ 11 2007/ 12 2008/ 01 2008/ 02 2008/ 03 2008/ 04 2008/ 05 2008/ 06 2008/ 07 2008/ 08 2008/ 09 2008/ 10 2008/ 11 2008/ 12 2005/ 01 2005/ 02 2005/ 03 2005/ 04 2005/ 05 2005/ 06 2005/ 07 2005/ 08 2005/ 09 2005/ 10 2005/ 11 2005/ 12 2006/ 01 2006/ 02 2006/ 03 2006/ 04 2006/ 05 2006/ 06 2006/ 07 2006/ 08 2006/ 09 2006/ 10 2006/ 11 2006/ 12 2007/ 01 2007/ 02 2007/ 03 2007/ 04 2007/ 05 2007/ 06 2007/ 07 2007/ 08 2007/ 09 2007/ 10 2007/ 11 2007/ 12 2008/ 01 2008/ 02 2008/ 03 2008/ 04 2008/ 05 2008/ 06 2008/ 07 2008/ 08 2008/ 09 2008/ 10 2008/ 11 2008/ 12 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 False Alarm Rate: EPS False Alarm Rate: CLI
Results: ROC scatter plot Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 25 1 0.9 0.8 Forecasts yjex Climatology Low system TH 0.7 Medium system TH Hit Rate 0.6 0.5 0.4 0.3 High system TH FAR > HR : no skill 0.2 0.1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 False Alarm Rate
Results : Brier Score Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 26 Seasonality??? Extremely low BS and BS ref
1 0.5 0-0.5-1 -1.5-2 Results : Brier Skill Score Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 27 Brier Skill Score 2005/01 2005/02 2005/03 2005/04 2005/05 2005/06 2005/07 2005/08 2005/09 2005/10 2005/11 2005/12 2006/01 2006/02 2006/03 2006/04 2006/05 2006/06 2006/07 2006/08 2006/09 2006/10 2006/11 2006/12 2007/01 2007/02 2007/03 2007/04 2007/05 2007/06 2007/07 2007/08 2007/09 2007/10 2007/11 2007/12 2008/01 2008/02 2008/03 2008/04 2008/05 2008/06 2008/07 2008/08 2008/09 2008/10 2008/11 2008/12
Results: BS and BS ref maps Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 28 Average BS and BS ref maps in the period between 2005 2008
Results : BSS map Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 29 Average BSS map over the period 2005-2008
Conclusions Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 30 for drought forecasting long range meteorological forecasts are needed (more than 2 weeks); drought forecasting should be done with probabilistic approach; both statistical and numerical forecasting have to be considered; low or no skill detected over the whole European domain. However spatial and temporal variation is high; integrated approach for drought forecasting is necessary - both hydrological indicators (soil moisture, low flows, ) and meteorological indicators (SPI, PDSI, ) should be forecasted; validation of the forecast has to be improved, using longer timeseries and regional specifications.