PRET, the Probability of RETurn: a new probabilistic product based on generalized extreme-value theory

Transcription

1 Quarterly Journal of the Royal Meteorological Society Q. J. R. Meteorol. Soc. 37: 2 37, January 2 B PRET, the Probability of RETurn: a new probabilistic product based on generalized extreme-value theory Fernando Prates* and Roberto Buizza European Centre for Medium-Range Weather Forecasts, Reading, UK *Correspondence to: Mr Fernando Prates, ECMWF Operations Shinfield Park Reading Berkshire RG2 9AX United Kingdom. fernando.prates@ecmwf.int A new probabilistic forecast product, the Probability of RETurn (PRET), is introduced. PRET, the probability of occurrence of an event that corresponds to a specific return period, is computed from forecasts given by the ECMWF Ensemble Prediction System. It has been designed to provide easy-to-interpret and valuable information on the intensity and rarity of the expected severe weather, especially when the ensemble-based forecast distribution falls outside the model climate distribution. PRET definition relies on the Generalized Extreme Value family of distributions, which has been applied to study the statistics of the extremes in the model forecasts and observed datasets, and to estimate the levels corresponding to return periods not included in the datasets. PRET forecasts for the 2-metre maximum and minimum temperatures over Europe have been generated for six summer and six winter seasons (23 to 29). Case-studies have been used to illustrate that the new product is easier to interpret than products that are now commonly used, such as probability forecasts and maps of Extreme Forecast Indices. Average diagnostics of PRET forecasts indicate that the skill in predicting extremely hot temperatures in the warm season is higher than the skill in predicting extremely cold temperatures in the cold season. Copyright c 2 Royal Meteorological Society Key Words: severe weather; probabilistic prediction; extreme value theory; probabilistic products Received 4 October 29; Revised 3 October 2; Accepted 3 November 2; Published online in Wiley Online Library March 2 Citation: Prates F, Buizza R. 2. PRET, the Probability of RETurn: a new probabilistic product based on generalized extreme-value theory. Q. J. R. Meteorol. Soc. 37: DOI:.2/qj.79. Probabilistic prediction applied to severe weather events One of the European Centre for Medium-Range Weather Forecasts (ECMWF) main goals is to support its Member States in issuing early warnings of severe weather events. This is achieved by generating a set of products that include both single forecasts, e.g. the one given by the high-resolution global model, and probabilistic forecasts based on the ECMWF Ensemble Prediction System (EPS: Palmer et al., 27). The value of these latter types of forecasts in providing users with earlier warnings has been increasingly recognized in the past decade. Examples of the EPS forecasts of severe storms that hit Europe have been given by e.g. Buizza and Hollingsworth (22), who showed that EPS probabilistic forecasts gave earlier indications than single high-resolution forecasts, and were especially useful when the ECMWF single high-resolution forecasts issued on successive days and valid for the same verification time were highly inconsistent. More recently, Buizza et al. (27) further documented the capability of the EPS to predict severe weather conditions such as Hurricane Katrina and the extremely severe flood that affected Italy in 966. The value of a probabilistic approach was also recently quantified by Zsoter et al. (29), who showed that EPS-based ensemble-mean forecasts are less inconsistent than corresponding single control forecasts. Copyright c 2 Royal Meteorological Society

2 22 F. Prates and R. Buizza The EPS became part of the ECMWF operational suite of forecast systems in December 992 (Molteni et al., 996). Since then, it has been upgraded several times (see Palmer et al. (27) for a review). From March 28 to 2 January 2 it has been running twice a day with members at T L 399 L62 (triangular spectral truncation 399 with linear grid and 62 vertical levels) resolution up to day, and T L 2 L62 afterwards. Since 26 January 2, the EPS has been running with a T L 639 L62 resolution up to day (corresponding to a grid-point resolution at midlatitudes of about 32 km), and T L 39 L62 thereafter (corresponding to a grid-point resolution of about 6 km). Over the years, the range of ensemble-based products that ECMWF produces routinely has been increasing substantially, and today includes ensemble-mean forecasts, maps of ensemble spread, clusters, probabilistic forecasts, EPS-grams, and Extreme Forecast Index (EFI) maps. This latter product, in particular, was developed precisely to help forecasters to manage weather risk by identifying regions where the EPS probability forecast distribution is substantially different from the model climate (Lalaurette, 23). The EFI can be seen as an alternative to the traditional forecast calibration methods, such as Model Output Statistics, since its value is determined by the relative difference of the EPS forecast and model climate distributions. In the EFI, the detection of extreme events is done independently of the observed local climate, and thus can be generated also for areas where observations are sparse or unavailable (Fundel et al., 2). Possibly, the main weakness of the EFI is that it relies on the assumption that the frequency of rare events in the model is the same as the one in the observation spaces. The key advantage of the EFI, compared to a corresponding probability map, is that it normalizes the probability forecast using the model climate distribution, thus helping the users to identify abnormal conditions. The EFI has proved to be a very successful product, especially in identifying regions of extreme temperature, wind and rainfall conditions. Despite its popularity and proven utility, users have occasionally raised the criticism that sometimes it is not immediately possible to translate an EFI value into potential damages, or losses, even when the EFI reaches its maximum of %. Typical questions that have been raised are: what does an EFI of 9% mean? How often does an event with an EFI of 9% occur in reality? This work aims to address this criticism by introducing a new type of product that provides similar information to the EFI, but is more immediately understood. This new probabilistic product, the Probability of RETurn, is computed as the fraction of the EPS members that exceed a return level. The return levels are estimated with a distribution fitted to the annual extremes from the model climate (the same climate that is used to compute the EFI) at each grid point as described later. PRET forecasts are designed to provide a better understanding of the intensity and rarity of the expected severe weather, especially when the EPS forecast distribution falls outside the model climate distribution domain. To explain the basic idea behind this new product, consider the temperature prediction over Reading for July 29, a summer day during which temperature is, on average, about 8 C. Suppose there is a chance of very hot weather, and there is the need to assess the likelihood of temperature exceeding the 3 C threshold. The EPS gives a 3% probability of temperature exceeding 3 C, and the EFI has a value of.7. While a 3% probability is low, an EFI of.7 indicates that the EPS distribution is very different from the model climate distribution. Although it is easier to translate this EFI forecast into a warning of the possibility of severe weather conditions than the probability forecast, there is still a piece of information that is missing from the EFI forecast: how often does Reading experience a temperature of 3 C? In other words, is this an event that occurs once every July, once in a summer, or once every years? The new PRET product has been designed to provide precisely this information. A PRET forecast for this event could be that there is a 4% probability of experiencing temperatures that are observed once every years. The main difficulty in the generation of PRET forecasts is the estimation of the N -year return level, i.e. the value corresponding to an N -year return period. This is especially true for rare events. Conventionally, the return levels are derived from the extreme quantiles of the cumulative distribution function of the observed or model data. However, they can be computed beyond the length of the data sample only using extrapolation techniques. For example, a daily maximum 2-metre temperature associated with a 4-year return period cannot be calculated from a dataset of 8 annual maxima, but it can be extrapolated from a properly fitted distribution for the annual extremes. The Generalized Extreme Value (GEV) family of distributions gives a simple and general methodology to do the calculation, and has been used to study the statistics of the extremes in meteorology and hydrology. Kharin and Zwiers (2) applied GEV distributions to assess the sensitivity of return levels for temperature, wind and precipitation to different radiative forcing scenarios. They used a threemember ensemble of integrations over three 2-year periods corresponding to 63 simulated years each. They stated that compared to other distributions used to study extremes (e.g. the Gumbel distribution: Gumbel, 98), the GEV family of distributions gives more flexibility, and improves the fit of the model distribution to the data. Van den Brink et al. (2) investigated the extreme-value distribution of sea storm surge levels, waves and river discharges in the Netherlands, by applying GEV distributions to ECMWF seasonal forecast data (representing a total of simulated years). They were able to increase the precision of the return level estimates of rare events (e.g. Rhine discharge levels that occur every 2 years) by a factor of three. In a different study of the past and present climate, Felici et al. (27) also applied GEV to infer the extreme value statistics of a 2-level quasi-geostrophic model of the midlatitude atmosphere. They concluded that a GEV-based statistical method could be used to study climate variations. In this work, the approach of van den Brink et al. (2) has been applied to EPS daily maximum and minimum 2- metre temperature model forecasts and to the observations over Europe to generate PRET products. For the model forecasts, firstly, for each grid point, annual maxima (minima) of the daily maximum (minimum) temperature have been extracted from a dataset of EPS past forecasts (model climate); these data have been used by fitting the GEV distribution function. Secondly, levels corresponding to different return periods, obtained from the fitted GEV function, have been computed also for return periods that are not included in the dataset of the model climate, so that probabilities could have been generated also for these very rare events. Thirdly, PRET forecasts have been generated, Copyright c 2RoyalMeteorologicalSociety Q. J. R. Meteorol. Soc. 37: 2 37 (2)

3 PRET, the Probability of RETurn 23 Table I. Operational configuration of the ECMWF EPS. Day Day Day 32 (Thu only) UTC T L 399 L62 ( km) Persisted SST anom. T L 2 L62 ( 8 km) Persisted SST anom. T L 2 L62 ( 8 km) Coupled Ocean T L 639L62 ( 32 km) T L 39L62 ( 63 km) T L 39L62 ( 63 km) 2 UTC T L 399 L62 ( km) Persisted T L 2 L62 ( 8 km) Coupled SST anom. Ocean T L 639L62 ( 32 km) T L 39L62 ( 63 km) Ensemble Prediction System (EPS) at the time of writing (summer 29), and in the operational configuration since 26 January 2 (in bold italic if different). and, finally, objective verification measures have been applied to assess the average quality of the PRET forecasts. A similar approach has been applied to the observations. After this introduction, the EPS is briefly reviewed in section 2 and the theoretical framework used to define the new PRET product is discussed in section 3. Two cases of heatwaves and one of cold anomalies that affected Europe in recent years are presented in section 4 to illustrate the potential usefulness of PRET to forecasters. Average diagnostics are presented in section. The potential value of PRET in weather-risk management is discussed and some general conclusions drawn in section The ECMWF Ensemble Prediction System The EPS became part of the ECMWF operational suite of systems in December 992, when it started running from the 2 UTC analysis on Fridays, Saturdays and Sundays, with 33 members run for days at T63L9 resolution (Buizza and Palmer, 99; Molteni et al., 996). The EPS has been designed to simulate the effects of initial condition and model uncertainties on the forecast error. Until 2 June 2, the initial uncertainties were simulated using a combination of initial-time and evolved singular vectors (Buizza and Palmer, 99) computed at T L 42 L62 resolution, with 48-hour optimization time interval and a total-energy norm. Since 22 June 2, the EPS initial perturbations have been generated using a combination of Ensembles of Data Assimilation perturbations and initialtime singular vectors (Buizza et al., 28). Singular vectors are computed to maximize the final-time norm over different areas (Barkmeijer et al., 999, 2), combined and scaled to have initial amplitude comparable to an estimate of the analysis error. Model uncertainties due to physical parametrizations are simulated using a revised version of the Buizza et al. (999) stochastic scheme (Palmer et al., 29). On March 28, the EPS was merged with the monthly system (Vitart et al., 28). Table I summarizes the key characteristics of the EPS operational at the time of writing (July 2). At the time of the merge in February 28, an EPS re-forecast suite was also implemented, to provide the data required to estimate the EPS model climatology for the whole 32-day forecast period (Hagedorn et al., 28). Within the EPS re-forecast suite, for each Thursday (i.e. the starting time of the 32-day EPS) of the previous 8 years, a five-member ensemble is run in the operational configuration (i.e. same resolution and model cycle, 32-day forecast length) starting from the Era-Interim analysis. The re-forecasts are run in such a way that every Thursday, re-forecasts for the weeks centred on the current Thursday are available. Thus, each Thursday after the completion of the UTC ensemble, users can access: forecasts, given by the -member EPS, valid for the next 32 days; 4 re-forecasts, given by the -member EPS started at UTC of all Thursdays of the weeks centred on the current Thursday for the previous 8 years (i.e. 8 forecasts). For each grid point, the 4 re-forecasts are used routinely at ECMWF to estimate the model climate, and thus to calibrate EPS products. For example, operational weeklyaverage anomaly forecasts for weekly forecasts from days to are computed by removing from the (most recent) weekly-average forecast the weekly-average model climatology computed using the 4 re-forecasts. Operational EFI forecasts are generated by comparing the cumulative distribution function of the operational - member EPS with the climate distribution defined by the 4 re-forecasts. The current configuration of the re-forecast suite at ECMWF evolves with time (at time of writing the period spans the years from 992 to 29) to better represent any changes in the climate that may occur, particularly regarding extremes events. 3. PRET generation: Computation of the return level of rare events using generalized extreme-value theory The first step for the generation of the new PRET products is the computation of the return levels for any period of interest (e.g. with a return period of,, 4 or even years), in particular the levels of very rare events characterized by very long return periods. Due to the limited sample of the EPS re-forecasts and the observation datasets, the only way to estimate the return levels for very rare events (e.g. with return periods of years or longer) is to compute them by fitting an appropriate set of distribution functions to the available data. As mentioned in the introduction, following the work of other authors the appropriate distribution functions that have been used in this work belong to the Generalized Extreme Value family. In the first part of this section, the EPS re-forecast and observation datasets are described. Then, the GEV family of distributions is introduced, and the methodology used is discussed. Finally, model-based return levels are compared to observation-based return levels. Copyright c 2RoyalMeteorologicalSociety Q. J. R. Meteorol. Soc. 37: 2 37 (2)

4 24 F. Prates and R. Buizza Relative frequency (%) Relative frequency (%) Rel Freq of Normalized Anomaly for 2mTmax Extremes for different forecast lead times over Europe (land points) [-3,-2.] [-2..,-2] [-2,-.] [-.,-] [-,-.] [-.,] [,.] [.,] StDev class [,.] [..,2] [2,2.] [2.,3] D D3 D D7 Rel Freq of Normalized Anomaly for 2m Tmin Extremes for different forecast lead times over Europe (land points) [-3,-2.] [-2..,-2] [-2,-.] [-.,-] [-,-.] [-.,] [,.] [.,] StDev class Figure. Top panel: histogram of the standardized anomaly of annual daily maximum temperature (TMAX) as a function of the forecast lead time ( 7 days) for all grid land-points in Europe (7 N/2. W/3 N/42. E) from the model climate. Bottom panel: as top panel but for TMIN. [,.] [..,2] 3.. EPS re-forecasts and observation datasets The two datasets used in this work are the EPS re-forecasts, which provided the data to estimate the model-gev, anda set of European observations from SYNOP stations, which provided the data to estimate the observed-gev.themodel forecasts are interpolated at the SYNOP locations, and the SYNOP data are used for the verification of the PRET product EPS re-forecasts dataset The EPS re-forecasts are defined by the 8-year, -member EPS re-forecast suite (Hagedorn et al., 28). Data have been extracted on a regular,., latitude longitude grid (this resolution corresponds to the EPS maximumavailable resolution on the linear grid), using the bi-linear interpolation software used routinely at ECMWF. Consider, for example, the daily maximum 2-metre temperature (TMAX). At each grid point, for each of the five re-forecast [2,2.] [2.,3] [3,3.] D D3 D D7 [3,3.] members and for each of the 8 years, the maximum value of TMAX is extracted from the dataset. This gives a total of 9 (8 ) TMAX values. These 9 TMAX values are used to compute GEV distribution of the EPS at each grid point (see next subsection). A similar procedure is applied to the daily minimum 2-metre temperature (TMIN). To guarantee a more complete representation of the annual extremes of daily maximum temperatures that overcomes the fact that the re-forecasts are generated only once a week (with starting time UTC on Thursdays), 7 day forecasts with a 6-hour frequency are used from each forecast member. This ensures that forecasts are available every 6 hours, for every day of the 8 years, and not just for Thursdays. The combination of 7 day forecasts is justifiable only if the model climatology does not show any significant drift (e.g. bias) with the forecast step (routine model error statistics of EPS control forecasts confirmed that there is no model drift in the 2 m temperature EPS forecasts between days and 7 over Europe). To assess whether this is the case, the standardized distributions of TMAX and TMIN at forecast days, 3, and 7 have been compared. The standardization is needed in order to remove any influence of the location and variability from the data, especially when it spans a large region (Wilks, 26). For each forecast day and at each grid point, first the standardized distribution function, i.e. the distribution of the standardized anomalies (x x) / s,wherex is one of the 9 grid-point values of TMAX or TMIN, x is their average and s is the standard deviation, has been computed. Then, the histograms of the relative frequency of the standardized distributions have been computed for all European grid points. Figure shows the relative frequencies at forecast days, 3, and 7, binned in classes with a. C width. Results show that, for each class, the differences between the relative frequencies at the different forecast times are less than 2%, suggesting that the model climate distribution does not show any significant drift with the forecast time Observations dataset The observations dataset includes daily maximum and minimum temperatures from synoptic stations across Europe (stations with coordinates with latitude between 3 Nand7 N and longitude between 2. W and 42. E), available from the European Climate Assessment & Dataset (ECA&D) project (Klein Tank et al., 22). This dataset includes observations from 3 stations across WMO region VI, provided by 4 countries throughout Europe and the Mediterranean region (i.e. these stations cover a larger area than the one used in this study). A combination of different statistical tests is performed to evaluate the homogeneities in the dataset, assigning a quality-control flag; however, high-resolution daily time series can limit the efficiency of these statistical tests. The quality control reduces the dataset to approximately 2 stations; these are the only stations over the European area for which data series that cover the 8-year period were complete, and for which the difference between the real and model heights was less than 2 m. Mimicking what was done for the re-forecasts, the annual observed TMAX (TMIN) values for this 8-year period have been extracted for each of the 2 selected SYNOP locations, and have been used to compute the GEV distributions of TMAX (TMIN) of the observations. The choice of using observations for only Copyright c 2RoyalMeteorologicalSociety Q. J. R. Meteorol. Soc. 37: 2 37 (2)

5 PRET, the Probability of RETurn 2 Figure 2. Return level plots. Top left: annual extreme TMAX for EPS re-forecasts (dots) and model-gev (solid line) for Hannover (Germany). Top right: observed annual extreme TMAX and observed-gev. Bottom panels: as top panels but for TMIN with temperature values multiplied by (i.e. shown with opposite sign). The 9% confidence intervals are indicated by dashed lines. Copyright c 2RoyalMeteorologicalSociety Q. J. R. Meteorol. Soc. 37: 2 37 (2)

6 26 F. Prates and R. Buizza 7 N Return Level for a year Return Period (daily maximum temperature in C) N Differences of Temperature Return Levels between -Year and -Year Return Period N 36 6 N N 27 N N N 2 2 E 4 E 2 E 4 E 7 N Differences of Temperature Return Levels between -Year and -Year Return Period 9 7 N Differences of Temperature Return Levels between 4-Year and -Year Return Period N 7 6 N N 4 N N 2 4 N 2 2 E 4 E 2 E 4 E Figure 3. Top left panel: spatial distribution of the -year return values of TMAX ( C) obtained from the model-gev. Other panels: differences ( C) between the - and the -year return values (top right), the - and the -year return values (bottom left) and the 4- and the -year return values (bottom right). the past 8 years ensures that both model and observed GEV fitted distributions are computed using the same reference period. It should be stressed that the validity of the return level estimates is limited to the period of reference of the climate (van den Brink et al., 2) because the statistical properties of the extremes can be very sensitive to the length of the period in question but also if the sample of extremes is nonstationary. For example, Klein Tank et al. (22) identified a warming of the cold and warm tails of the distributions of TMIN and TMAX for the period A similar effect has been detected during our study; see section 3.2 for a discussion Model- and observed-gev distribution functions As discussed in section 3., for both TMAX and TMIN, 9 re-forecast (five members for 8 years) and 8 observed values are available at each grid point and SYNOP station respectively. Since, as mentioned above, the ranges spanned by these datasets do not include extreme events that can occur out of the analysed period, the statistics of these variables beyond the spanned ranges have been extrapolated applyingtheextremevaluetheory. The sensitivity of the observed climate distribution to the choice of the length of the data period has been calculated (this sensitivity study could not have been done for the model forecasts since these were not available). Estimates of the return levels have been computed using the annual extremes for two periods: 8 (99 27) and 6 (946 27) years. Results (not shown) indicate that for TMAX the average difference for the -year and the 2-year levels is. and.46 C, respectively. For TMIN, the corresponding average differences are 2. and 2. C. This indicates that results are sensitive to the choice of the period used to compute the fitted GEV curves, and the corresponding return levels, especially for TMIN (incidentally, note that the differences are positive, indicating a warming of the climate over Europe, confirming the conclusions of Klein Tank et al. (22)). In other words, using only the most recent 8 years is a way to avoid climate-change influences on the statistics of theextremes.thedebateinthescientificcommunityis still going on, whether to account for the non-stationarity conditions in the extreme-value analysis and how is the best way to do this, see e.g. Klein Tank et al. (29). Extreme-value theory contains an important result with respect to the behaviour of block maxima (or minima). Under suitable assumptions, the distributions of block maxima (or minima) converge to one of three types of extreme-value distribution, which can be combined into a single GEV family: [ exp { + ξ ( x µ )} ] /ξ σ, for block maxima [ G(x)= { )} ] /ξ exp ξ, for block minima ( x µ σ () Copyright c 2RoyalMeteorologicalSociety Q. J. R. Meteorol. Soc. 37: 2 37 (2)

7 PRET, the Probability of RETurn 27 can be computed as: T = G(x T ) (2) Figure 4. Differences ( C) between the TMAX model-gev and the observed-gev values with a -year (top), -year (middle) and 4-year (bottom) return period. where G(x) isdefinedfor{x :+ ξ(x µ)/σ > } or {x : ξ(x µ)/σ > } by three parameters, namely the location parameter <µ, µ <, the scale parameter σ> and the shape parameter <ξ< (Coles, 2). The sign of the shape parameter determines whether the distribution belongs to the Gumbel (ξ = ), Fréchet (ξ >) or Weibull (ξ <) family (these families are also known as the type I, II and III GEV distributions). It is usually more convenient to interpret GEV models in terms of return levels by inverting the above Eq. (). The return period is defined as the upper (or lower) quantile of the GEV distribution, and for the case of block maxima (minima) with x T representing the return level for the T-period. Thus, x T can be interpreted as the value that will be exceeded on average once every T-period. The shape parameter determines the behaviour of the distributions in the uppertail regions for very rare events (this can be seen in return level plots, see e.g. Coles (2), p ). For the Gumbel distribution, the return level curve is linear while the Frechet curve shows a concave shape for long return periods. The Weibull distribution has a horizontal asymptote for an infinite return period event. To compute the unknown parameters of the GEV model (location, scale and shape parameters), the maximumlikelihood estimation method has been applied for its adaptability to model-change (Coles, 2; Klein Tank et al., 29). Confidence intervals for the return level curve were also computed by applying the delta method as in Coles (2). A simple check of goodness-of-fit between GEV and annual extremes was performed. The fitted GEV distribution at each grid point has been considered inadequate whenever the empirical distribution for the annual extreme at predefined T-periods (specifically,, and 4 years, see results in section 3.3) falls outside the 9% confidence interval computed from the return level curve at the same T-periods. Results show that only about 8% of the total fitted GEV distributions are not adequate for the annual extremes. (All the computations were carried out using the statistical package R, freely available software at under the General Public License.) To illustrate the fit of the GEV function to the data, Figure 2 displays the 9 re-forecast values and the model- GEV, and the 8 annual observations and the observed-gev for TMAX and TMIN for a randomly chosen synoptic station, Hannover (Germany). For TMAX, the model-gev fitted distributions show a cold bias with a near-constant bias of 2 C. The largest difference between the two distributions is in terms of the precision of the estimation of the return levels, with smaller confidence intervals detected in the model-gev. The return level plot for TMIN (please note that in Figure 2 the TMIN values have been multiplied by ) exhibits a small warm bias between the model and observed GEVs of about 2 C for short return periods only. As for TMAX, the precision of the return levels estimates is lower for the observed-gev than for the model. Note that the departures between the annual extremes and the GEVs distribution for longer return periods are larger for TMIN than TMAX, indicating that for TMIN the GEV fitforrareeventsisworsethanfortmax.inthetmin case, the estimate for the shape parameter for TMIN is negative (ξ =., bounded distribution), though very close to zero, and the 9% confidence intervals (.2,.9) indicate the possibility of other types of distribution (unbounded) which could be fitted to the data. The comparison of the model and observed return levels at station level, e.g. in the format shown in Figure 2, is a useful diagnostic tool, since it visualizes the difference between the model (i.e. re-forecast) and the observed climates. The quality of the fitted distributions for the model and observed annual extremes can be assessed by plotting the confidence levels of the estimated return levels and the uncertainty Copyright c 2RoyalMeteorologicalSociety Q. J. R. Meteorol. Soc. 37: 2 37 (2)

8 28 F. Prates and R. Buizza 7 N 2 E 4 E -3 7 N 2 E 4 E N -9 6 N N -8 N N N E 4 E -2 2 E 4 E 7 N 2 E 4 E - 7 N 2 E 4 E N -3 6 N N -6 N N N E 4 E -2 2 E 4 E Figure. Top left panel: spatial distribution of the -year return values of TMIN ( C) obtained from the model-gev. Other panels: differences ( C) between the - and the -year return values (top right), the - and the -year return values (bottom left) and the 4- and the -year return values (bottom right). shape parameter. In the case of Hannover, for example, results indicate that the EPS model climate for TMIN is rather different from the observed climate; the discussion in the forthcoming sections will document the generality of this conclusion, and how this bias affects the EPS performance in predicting extreme cold conditions. It is worth mentioning that in this work it has been assumed that the five EPS re-forecasts can be considered as representing an independent sample from the same climate distribution. This assumption is commonly made in ensemble forecasting (Doblas-Reyes et al., 2; Leutbecher and Palmer, 28). The validity of this assumption has been assessed by considering the five re-forecasts for some specific locations. For Hannover, for example, the average correlation between any two of the 8-value TMAX curves is.8, and the average t-test is.3. The first number indicates that the five members are neither completely independent nor dependent, and the second number indicates that there is a 3% probability that they are sampled from the same underlying distribution. The fact that the correlations and the t-testvaluesarenotzerois not surprising in our view, since the five re-forecasts are all supposed to sample the same model climate distribution, and indicate that the five re-forecasts can be considered as different samples of the model climate distribution. If they were dependent, then the correlation between any two curves and any two anomaly curves would have been %, and the actual model data sample would have been 8 and not 9. In the case of TMIN for Hannover, the correlation average between any two curves is.9 and the average t-test is.33, suggesting that also for TMIN, the five EPS re-forecasts are not fully dependent. Indeed in many applications, as in this work, pragmatic considerations often lead to the consideration of a less strict version of the assumption, in the classical extreme-value theory, that the events are independent and come from the same distribution. Yet this approximation does not invalidate the GEV as a candidate distribution to parametrize the statistic of extreme values (Coles, 2; Wilks, 26) Comparison of the return levels of the EPS forecasts and the observations Figure 3 shows the maps of the estimated TMAX return levels with a -, -, - and 4-year return period of the EPS over Europe. The top-left panel shows the -year return levels: TMAX values range from above 36 C for the countries facing the Mediterranean, to values between and 24 C for Scotland and Scandinavia. The other three panels of Figure 3 show the differences between the -, - and 4- year levels and the -year values: differences between the - and the -year levels are less than 4 C, while the difference between the 4- and the -year levels is larger than Cfora few regions (southwest of the United Kingdom, France and southeast Europe). The discrepancies between the model and the observed GEV distributions can be assessed by comparing the return levels for some specific return periods. Figure 2 shows that, for the specific station of Hannover, the difference of the TMAX -year return levels is about 2 C. Figure 4 shows the differences of the -, - and 4-year return levels of the model- and observed-gev distributions for some of Copyright c 2RoyalMeteorologicalSociety Q. J. R. Meteorol. Soc. 37: 2 37 (2)

9 PRET, the Probability of RETurn 29 in many places across Europe (see the annual report on verification of ECMWF products by the member states (Richardson et al., 28)). Maps of TMIN return levels for the model-gev, and differences between the model- and observed-gev levels with -, - and 4-year return levels are shown in Figures and 6, respectively. The continental effect on winter temperatures is evident from the -year return level map. Figure indicates that the TMIN return levels increase more rapidly when the return period is lengthened than do the TMAX return levels (Figure 4). Physically, this can be because European temperatures are more variable in winter than in summer since the presence of snow/ice can have a strong effect on the ground thermal conditions, and since the advection of air from different directions can lead to much larger variations. Since the mathematical framework used to estimate (by extrapolation) the levels with very long return periods does not take into account the physical characteristics of the system, care must be taken when extrapolating far beyond the sample maxima (see also the discussion in Coles (2), p 66). Figure 6 shows the differences, at the European synoptic stations, of the -, - and 4-year TMIN return levels of the model- and the observed-gev distributions. Results indicate that the return levels of TMIN estimated from the model are generally warmer than for the ECA&D station data, indicating in this case a model warm bias during the cold season in predicting cold temperatures. This bias is consistent with the model warm bias detected during the cold season over Europe and reported in the annual report on verification of ECMWF products by the member states (e.g. Richardson et al., 28). Compared to TMAX, the differences between the re-forecast and the observed return periods are, in general, larger for TMIN than TMAX: 3. Cinsteadof.6 C for a -year return period, and 4. Cinsteadof.2 C for a 4-year return period. The large positive bias detected over central and eastern Europe and over Scandinavia affects the EPS average performance in predicting extreme cold conditions (see sections and 6). 4. PRET forecasts for three case-studies Figure 6. Differences ( C) between the TMIN model-gev and the observed-gev values with a -year (top), -year (middle) and 4-year (bottom) return period. European SYNOP stations (the values for only a few stations are plotted to make the plot readable). In general, a small cold model bias can be detected for all three return levels throughout the whole region: the comparison of the average values for the -, - and 4-year return levels indicate cold biases of 3.3, 3. and 4. C, respectively. This model bias could be due to its coarse spatial resolution, with the model not being capable of fully representing the subgrid meteorological scales detected in the observations. This underestimation of the TMAX extremes is in agreement with the model cold bias detected during the warm season Three case-studies are discussed in this section to illustrate the value of the new PRET forecasts: two of them are associated with a heatwave and one is characterized by a cold anomaly (these cases have not been selected because the EPS PRET forecasts showed very good skill, but because they were interesting cases to discuss). For the first two cases, t+4 h forecasts from a UTC run are discussed. This forecast length was chosen for two main reasons: the t+4 h TMAX forecast is defined as the maximum temperature predicted between t+8 h and t+4 h, a period that corresponds to the early afternoon, when maximum temperatures are reached over Europe. Furthermore, this forecast length was chosen because it is the closest forecast length to t+2 h, which is the maximum forecast length for which the EFI forecast was available. For the third case, a shorter forecast length, t+78 h from a UTC run, was chosen for the following two reasons: the t+78 h TMIN forecast is defined as the minimum temperature predicted between t+72 h and t+78 h, a period that corresponds to the night, when minimum temperatures are reached over Europe. A shorter time step than for the two heatwave cases was selected Copyright c 2RoyalMeteorologicalSociety Q. J. R. Meteorol. Soc. 37: 2 37 (2)

10 3 F. Prates and R. Buizza 7 N N Observed -Year Return Period Event of Daily Maximum Temperature VT : 2 August 23 6 N 6 N 4 N 4 N 2 E 4 E 7 N N 7 N 6 N N 4 N 8 August 23 Forecast probability t+4 VT: 2 August 23 8 UTC Daily maximum temperature: probability of exceedance of a year return period event 2 E 4 E N N Observed -Year Return Period Event of Daily Maximum Temperature VT : 2 August 23 6 N 6 N 4 N 4 N 2 E 4 E 7 N N 8 August 23 Forecast probability t+4 VT: 2 August 23 8 UTC Daily maximum temperature: probability of exceedance of a year return period event 7 N 6 N N 4 N 2 E 4 E N N Observed 4-Year Return Period Event of Daily Maximum Temperature VT : 2 August 23 6 N 6 N 4 N 4 N 2 E 4 E 7 N N 8 August 23 Forecast probability t+4 VT: 2 August 23 8 UTC Daily maximum temperature: probability of exceedance of a 4 year return period event 7 N 6 N N 4 N 2 E 4 E Figure 7. August 23. Left panels: synoptic observations reporting TMAX above (labelled ) or below (labelled ) a -year return value (top left), a -year return value (middle left) and a 4-year return value (bottom left). Grey shading areas represent the radius of influence for the synoptic stations reporting an N-year return period to highlight observed events at each station. Right panels: t+4 h probabilistic forecast (%) started at 2 UTC of the 8 th and valid for 8 UTC 2 August 23 of TMAX above a -year return value (top right), a -year return value (middle right) and a 4-year return value (bottom right). because, on average, TMIN forecasts are less skilful (see discussion in section 6). 4.. PRET forecasts of TMAX for a case in August 23 (heatwave) In August 23, Europe experienced the hottest summer on record. Maximum temperature records were broken in many parts of Europe (Schär et al., 24), with very high daily mean temperatures persisting for many consecutive days. Therefore, several countries suffered an exceptional number of casualties and experienced widespread forest fires. The temperature kept rising from the beginning of the month, and reached peak values between 2 and August. During this period, weak gradients in mean-sea-level pressure and anticyclonic conditions dominated the western part of Europe (Black et al., 24). At that time, ECMWF EPS forecasts were issued in terms of the single, most likely scenario (e.g. given by the ensemble mean), probabilities of temperature anomalies exceeding fixed thresholds, or EFI. In general, these forecasts were correctly suggesting the possibility of extremely hot weather conditions. Figure 7 shows how a PRET forecast issued about days before the event would have appeared, and how it compares with the observed conditions. More specifically, Figure 7 shows the t+4 h PRET forecasts for -, - and 4-year Copyright c 2RoyalMeteorologicalSociety Q. J. R. Meteorol. Soc. 37: 2 37 (2)

11 PRET, the Probability of RETurn 3 and the PRET forecast indicated the same area of potential risk of extreme high temperatures, if PRET maps had been available to the forecasters, they could have more easily issued statements such as... there is a high probability of maximum temperature reaching values experienced once every years over a large part of eastern Europe, with TMAX exceeding the 4-year return level over Hungary PRET forecast of TMIN for a case in January 26 (cold wave) Figure 8. Extreme Forecast Index for 2-metre temperature forecast day valid at 2 UTC 2 August 23 (contour intervals every % and shading for values above %). Contour lines in grey indicate model orography, T L 2 L4 (from Grazzini et al., 23). return levels that could have been issued at 2 UTC 8 August for 8 UTC 2 August if this new product had been available. PRET forecasts are contrasted with maps that show the synoptic stations that observed TMAX in excess of the -, - and 4-year return levels. Figure 7 shows that PRET forecasts correctly identify the area where extreme TMAX values were observed. These forecasts are consistent with the t+2 h EFI for 2-metre temperature valid for that day; indeed, Figure 8 shows that the EFI exhibits large values close to % in central and southwestern Europe indicating significant departures of the EPS forecast toward the upper tail of the model climate or even beyond (Grazzini et al., 23). Although the PRET and EFI forecasts identify the same region, the PRET maps are easier to interpret, since they could be used to issue statements such as... there is a high probability of maximum temperature reaching values that are experienced only once every 2 years PRET forecasts of TMAX for a case in July 27 (heatwave) In July 27, southeastern Europe experienced another heatwave, with temperatures rising to well over 4 Cin many areas, promoting forest fires across the region. The excessive heat and dry conditions are thought to be partly responsible for the deaths of hundreds of people. Figure 9 shows the t+4 h PRET forecasts for -, - and 4-year return levels that could have been issued at UTC 6 July for 8 UTC 2 July if this new product had been available. PRET forecasts are compared with maps that show the synoptic stations that observed TMAX in excess of the -, - and 4-year return levels. Figure 9 shows that the PRET forecasts correctly identify the area over southeastern Europe where extreme TMAX values were observed. The PRET forecast for the -year return level shows an extensive area with a probability between 8 and % extending from northeastern Italy to Ukraine, while the PRET forecast for the 4-year return level limits the region to Hungary, with the probability dropping to 4 8%. Once again, the t+8 h 2-metre EFI identifies the same area of risk of extreme temperature that is predicted by the PRET product. Although both the EFI (Figure ) In January 26, abnormally cold winter conditions affected eastern Europe and Russia towards the end of the month. An intense continental anticyclone in northern Russia brought very cold temperatures on 2 January in many parts of the west of the country that farther extended to central Europe later on and claimed hundreds of lives. Figure shows the t+78 h PRET forecast for the - and -year return levels that could have been issued at UTC 2 January for 6 UTC 23 January. This figure shows an extensive area with a probability above 8% from the Kola Peninsula to the Black Sea for a -year return level, with a smaller probability for a -year event. As in the previous examples the EFI of 2-metre temperature at t+84 h (Figure 2) also indicates a risk of extreme temperatures for this day. It can be seen that, generally, the high risk of extreme cold temperatures are related to the PRET for -year return level high probabilities in many places. This result is not a surprise since the very high (low) values of the EFI occurs when the forecast distribution falls into the upper (lower) tail of the model climate distribution. In this case, the PRET forecast overestimated the severity of the cold spell in many places as shown by the maps of the synoptic stations that recorded TMIN below the - and -year return level, except for some places in Ukraine. Compared to the heatwave cases, the forecasts for this cold-wave case are less accurate, especially for cold temperatures with a longer return period (e.g. - or 4-year return period, not shown). The lower accuracy is also a consequence of the lower skill of the EPS in predicting cold temperatures during the analysed period, as will be pointed out in the following section.. Average performance of PREP forecasts The average quality of PRET forecasts has been assessed using established probabilistic measures considering a larger number of cases. More precisely, PRET forecasts for TMAX have been evaluated for a 8-day period covering six summer seasons, JJA (June July August) 23 29, using the Brier skill score and the area under the Relative Operating Characteristic (ROC) curve as accuracy measures. Similarly, PRET forecasts for TMIN have been evaluated for a 8-day period covering six winter seasons, DJF (December year- and January February year) The Brier score (Brier, 96; Wilks, 26) is the equivalent of the root-mean-square error for a probabilistic forecast of a categorical event (negatively oriented), and the Brier skill score (BSS) is the corresponding skill score computed with a climatological forecast as reference. The area under the relative operating curve (ROCA: see e.g. Swets, 986; Wilks, 26) measures the capability of a probabilistic forecast for a categorical event to discriminate between occurrence and non-occurrence. To compute the ROCA, since the events considered in this work are rare and the ROC curve points Copyright c 2RoyalMeteorologicalSociety Q. J. R. Meteorol. Soc. 37: 2 37 (2)

12 32 F. Prates and R. Buizza 7 N N Observed -Year Return Period Event of Daily Maximum Temperature VT : 2 July 27 6 N 6 N 4 N 4 N 7 N N 2 E 4 E Observed -Year Return Period Event of Daily Maximum Temperature VT : 2 July 27 7 N N 7 N 6 N 6 N N 4 N 4 N 2 E 4 E Observed 4-Year Return Period Event of Daily Maximum Temperature VT : 2 July 27 7 N 7 N 6 N 6 N N 4 N 4 N 2 E 4 E N 6 July 27 Forecast probability t+4 VT: 2 July 27 8 UTC Daily maximum temperature: probability of exceedance of a year return period event 7 N 6 N N 4 N 6 N N 4 N E 4 E 6 July 27 Forecast probability t+4 VT: 2 July 27 8 UTC Daily maximum temperature: probability of exceedance of a year return period event 7 N 6 N N 4 N E 4 E 6 July 27 Forecast probability t+4 VT: 2 July 27 8 UTC Daily maximum temperature: probability of exceedance of a 4 year return period event 7 N E 4 E Figure 9. July 27. Left panels: synoptic observations reporting TMAX above (labelled ) or below (labelled ) a -year return value (top left), a -year return value (middle left) and a 4-year return value (bottom left). Grey shading as in Figure 7. Right panels: t+4 h probabilistic forecast (%) started at UTC on the 6 th and valid for 8 UTC 2 July 27 of TMAX above a -year return value (top right), a -year return value (middle right) and a 4-year return value (bottom right). are all very close to the y-axis, two approaches were followed: the trapezoidal method and the method proposed by Wilson (2) based on a bi-normal model. EPS PRET forecasts of TMAX and TMIN valid for the European SYNOP stations have been verified against the observed values. Figure 3 (top panel) shows the average (JJA 23 29) reliability diagrams at forecast days 3 and of the PRET of TMAX higher than the 3-, - and -year return levels. Results indicate that the model is too confident, with forecast probabilities being higher than the observed frequencies. Table II lists the Brier score and skill score for all three PRET forecasts of TMAX at days 3 and. On average, forecasts are skilful for 3-, - and -year return period events at both lead times, with the skill decreasing for rarer (hotter) events. Figure 3 (bottom panel) shows the JJA ROC curves for the same probabilistic forecasts. AlthoughtheROCcurveslieabovethediagonalcurve, indicating that the EPS is capable of discriminating between the occurrence and non-occurrence of the events, the points used to construct them lie very close to the y-axis. This is because the false alarm rates (false alarms by the total of no events) are dominated by the non-occurrence events which are, typically, quite high in the case of rare events. In this case, as pointed out by Lalaurette (23), care must be taken in interpreting positive values of the area under the ROC curve, and the ROCA computed using the trapezoidal method and the normal-deviates give rather different values. The ROCA computed using the bi-normal method agrees with the trapezoidal value when the hit and false-alarm rate data fit the bi-normal model that is used Copyright c 2RoyalMeteorologicalSociety Q. J. R. Meteorol. Soc. 37: 2 37 (2)

13 PRET, the Probability of RETurn 33 Table II. Brier scores and skill scores, and area under the Relative Operating Characteristic curve. Day 3 (Day ) Brier Score Brier Skill Score ROCA (trapezoidal) ROCA (bi-normal) % explained variance of bi-normal method 3-Year Return Period (.9) (.2) (.73) (.8) (.99) -Year Return Period (.6) (.2) (.67) (.77) (.97) -Year Return Period (.3) (.42) (.6) (.8) (.92) Computed for PRET forecasts of TMAX higher than the 3-, - and -year return values at forecast days 3 and (bold values in brackets). The ROCA has been computed using the trapezoidal method (4 th column) and the bi-normal method ( th column); the 6 th column reports the percentage of variance explained by the bi-normal method (a value of. indicatesa perfect fit). Table III. Brier scores and skill scores, and area under the ROC curve. Day 3 (Day ) Brier Score Brier Skill Score ROCA (trapezoidal) ROCA (bi-normal) % explained variance of bi-normal method 3-Year Return Period (.4) (.98) (.8) (.96) (.) -Year Return Period (.3) (.76) (.78) (.97) (.3) -Year Return Period (.) (.34) (.73) (.99) (.3) Computed using the trapezoidal method and the bi-normal method proposed by Wilson (2), for PRET forecasts of TMIN lower than the 3-, -, -year return values at forecast days 3 and (bold values in brackets). The ROCA has been computed using the trapezoidal method (4 th column) and the bi-normal method ( th column); the 6 th column reports the percentage of variance explained by the bi-normal method (a value of. indicates a perfect fit). Figure. Extreme Forecast Index for 2-metre temperature forecast t+8 h valid at 2 UTC 2 July 27. Contour intervals every % (contours and shading as in Figure 8). to compute the area, but they differ substantially otherwise. The comparison of the two ROCA values with the Brier skill scores suggest that in cases when the data fit to the bi-normal model is not good, the ROCA computed using the trapezoidal method better reflects the actual capability of the probabilistic forecasts to discriminate between the occurrence and the non-occurrence of the events (Table II). This result is consistent with Marzban (24), who clearly points out that the knowledge of the underlying distributions of occurrences and non-occurrences should guide the choice of the method to use to compute the area under the ROC curve. Marzban (24) shows that in the case of probabilistic forecasts, which are bounded between and, the bi-normal model should not be applied. It is also worth pointing out that one of the advantages of the trapezoidal method over the bi-normal one is that it does not rely on any assumption about the underlying distributions. Overall, Tables II and III show that for both TMIN and TMAX, the ROCA computed using the trapezoidal method becomes smaller for rarer events and for longer forecast ranges. Figures 4 show the reliability diagrams and the ROC curves for the PRET of TMIN lower than the 3-, - and -year return levels. Results indicate that for TMIN the forecast probabilities lower than % are very reliable, especially for day, in contrast to the results obtained for TMAX. For high probabilities (>%) the reliability curves are affected by the reduced number of forecast observed pairs falling in each of those classes. Table III lists the Brier score and skill score and the area under the ROC curve for all five PRET forecasts of TMIN at days 3 and. Generally speaking, TMIN skill scores (Table III) are slightly lower than TMAX skill scores (Table II). 6. Conclusions: PRET value for weather riskmanagement In this work a new type of product, the Probability of RETurn (PRET, i.e. the probability of occurrence of a value that corresponds to a specific return period) has been introduced. PRET is based on the concept of a return level, i.e. of a level that is reached only once every return period, and it has been designed to provide easier-to-interpret forecasts of severe weather, especially when the EPS forecast distribution falls outside the model climate distribution domain. Since a PRET product is more similar to risk-management products Copyright c 2RoyalMeteorologicalSociety Q. J. R. Meteorol. Soc. 37: 2 37 (2)

14 34 F. Prates and R. Buizza Observed -Year Return Period Event of Daily Minimum Temperature VT : 23 January 26 7 N 7 N 6 N 6 N N N 4 N 4 N 7 N N 2 E 4 E Observed -Year Return Period Event of Daily Minimum Temperature VT : 23 January 26 6 N 6 N 4 N 4 N 2 E 4 E 7 N N 7 N 6 N N 4 N 7 N 6 N N 4 N 2 January 26 Forecast probability t+78 VT: 23 January 26 6 UTC Daily minimum temperature: probability of exceedance of a year return period event E 4 E 2 January 26 Forecast probability t+78 VT: 23 January 26 6 UT Daily minimum temperature: probability of exceedance of a year return period event E 4 E Figure. January 26. Left panels: synoptic observations reporting TMIN above (labelled ) or below (labelled ) a -year return value (top left) and a -year return value (bottom left). Grey shading as in Figure 9. Right panels: t+78 h probabilistic forecast started at UTC on the 2 th and valid for 6 UTC 23 January 26 of TMIN above a -year return value (top right) and a -year return value (bottom right). Figure 2. Extreme Forecast Index for 2-metre temperature t+78 h forecast valid at 2 UTC 23 January 26. Contours and shading as in Figure but multiplied by. available in other fields (e.g. in the insurance sector, or in the engineering world, where extremes are defined in terms of the frequency of occurrence), it should further increase the value of ensemble-based, probabilistic forecasts for weather risk-management. To illustrate why it is believed that PRET is easier to interpret, consider De Vries (29), who discussed the case of a storm surge in November 27 along the Dutch coast where for the first time in more than 3 years the coastal barriers had to be closed. The decision to close them was made because the EPS-based forecasts for water levels indicated a substantial probability that the level corresponding to an extreme event (alarm thresholds) could have been reached. For a wide range of applications in hydrology and climate studies, the return level is a fundamental quantity considered when building dykes, establishing flood planning policies, and studying weather and climate phenomena linked to the behaviour of the upper tail of a distribution (Guillou et al., 28). Thus having an EPS-based forecast expressed in terms of the probability of a return level (PRET) should make its use easier and more immediate. The main difficulty in the generation of PRET forecasts is the estimation of the levels corresponding to different return periods, especially for rare events. This estimation can be achieved using the Generalized Extreme Value (GEV) family of distributions. Return levels, with their confidence intervals, can be computed using the fitted GEV distribution function also for periods not included in the available dataset. In this work, this approach has been applied to the prediction of European extremely hot and cold conditions expressed in terms of the 2-metre temperature. The usefulness of the newly proposed PRET product has been illustrated considering three case-studies, and the average accuracy of the PRET forecasts have been evaluated for six summer (JJA 23 29) and six winter (DJF 23 29) seasons. The quality of EPS PRET forecasts has been assessed comparing forecast and observed values at about 2 European SYNOP stations for a 6-year period (23 29). Copyright c 2RoyalMeteorologicalSociety Q. J. R. Meteorol. Soc. 37: 2 37 (2)