Forecasting precipitation for hydroelectric power management: how to exploit GCM s seasonal ensemble forecasts

Transcription

1 INTERNATIONAL JOURNAL OF CLIMATOLOGY Int. J. Climatol. 27: (2007) Published online in Wiley InterScience ( Forecasting precipitation for hydroelectric power management: how to exploit GCM s seasonal ensemble forecasts Marta Benito García-Morales* and Laurent Dubus EDF Research and Development Division, Electricité de France, France Abstract: The EDF group is the biggest French electric power producer and distributor. Its activities are greatly related to weather and climate. In particular, optimal management of the hydroelectric power production system requires a good forecast of water resources, from several days to several months in advance. Currently, only climatology at the seasonal timescale is used for operational production management. Seasonal probabilistic forecasts would improve watershed management at some months lead-time if they are skilful enough. For this, two main problems have to be addressed: first, direct precipitation forecasts at this timescale have little, but positive, skill over Europe; second, the spatial scales of seasonal forecasting models are not adequate to predict local precipitation at the river basin scale. This study aims to evaluate the quality of seasonal forecasts of precipitation for 48 catchments in southern France. These are obtained by spatially downscaling global scale seasonal forecasts of geopotential height at 850 hpa. The method used is based on singular value decomposition and multiple linear regression. The statistical downscaling model is calculated from 45 years of observed local precipitation in the watersheds and geopotential fields from ERA40 re-analysis data. The statistical model is then applied to the seasonal hindcasts from the DEMETER project. Two main results arise from this work. First, we show that it is possible to obtain useful and valuable information for EDF at the local scale from global seasonal averaged information. Second, we find that only a probabilistic multi-model ensemble forecast approach provides useful information for EDF catchments, even with quite low skill, and that a deterministic approach, using only the ensemble mean of the forecasts, is not better than a forecast based on climatology. It has, nevertheless, to be pointed out that for operational purposes, being able to know that a forecast for a given location or date is not reliable is, in itself, valuable information. Copyright 2007 Royal Meteorological Society KEY WORDS downscaling; hydroelectric power; precipitation; seasonal forecasts; multi-model probabilistic forecasts Received 31 August 2006; Revised 2 July 2007; Accepted 3 July 2007 INTRODUCTION EDF group activities are greatly dependent on weather and climate. Electricity demand is directly linked to outside air temperature, but electricity production is also weather-dependent. In particular, hydroelectric power production is governed by water incomes, and therefore, by precipitation in the region where it is located. Skilful quantitative precipitation forecasts a few weeks or months in advance would hence, allow a better forecast of the availability of water stocks. This would provide better optimisation of hydroelectric power production and, as a consequence, the whole production system. It has also to be kept in mind that the fact that a forecasting system has no skill for a given location or a given season is in itself interesting information. It will lead decision makers to use climatological information rather than forecasts, thus avoiding the bad useage of information with no * Correspondence to: Marta Benito García-Morales, EDF Research and Development Division, Electricité de France, France. marta.benito-garcia-morales@edf.fr skill. Forecasting hydropower production is particularly important in winter, where it is used to meet peak demand at relatively low cost. In summer, it is useful to forecast river flows and temperatures, both to forecast hydropower production and the cooling capacity of thermal units set along the main rivers such as the Rhône or the Loire. The recent progress in monthly and seasonal forecasting presents an interesting field of investigation for EDF, but two main problems must be addressed: first, the spatial resolution of numerical General Circulation Models (GCMs) outputs are too coarse to be integrated into hydropower optimization tools; second, the raw precipitation forecasts from the models are known to have very little skill, especially over Europe. In order to improve the production forecasts, it is important to choose appropriate model fields and to adapt the GCM large-scale predictions to the local scale of interest (Figure 1). This is known as downscaling and has been widely studied and documented (e.g. Murphy, 1999, see Fowler et al., this issue, for a review). Downscaling techniques are divided into two main families: dynamical downscaling, in which the large-scale model outputs Copyright 2007 Royal Meteorological Society

2 1692 M. BENITO GARCIA-MORALES AND L. DUBUS Figure 1. Illustration of the downscaling problem and catchments locations. The main zones which the text refers to are: Pyrenees (diamonds), Alps (triangles), Rhone Valley (big circles), northern and central France (squares) and Corsica (small circles). The numbers (between 1 and 32, and between 51 and 66) correspond to the catchments references in the EDF database. are used as input variables into a regional, finer-scale dynamical model (e.g. Murphy, 1999); and statistical downscaling, where an empirical statistical relationship is found between the large-scale model outputs and the local variable to be forecast (e.g. von Storch et al., 2000). This study uses a statistical method based on singular value decomposition (SVD) and multiple linear regression (Bretherton et al., 1992) in order to forecast seasonal monthly average cumulative precipitation at 48 French catchments, using large-scale fields from the European Centre for Medium-Range Weather Forecasts (ECMWF) ERA40 re-analysis (Uppala et al., 2005), and hindcasts from the DEMETER models (Palmer et al., 2004). As will be discussed later, the results show little positive skill and there is a lack of significance of the scores. This is partly due to the small sample time-span used in the study. However, the benefits of optimizing the hydropower production plan make it worth investigating all positive improvements with respect to climatologic forecasts. DATA DESCRIPTION The statistical downscaling model used in this study is built using observed precipitation (local scale) data and ERA40 re-analysis (Uppala et al., 2005) geopotential fields (global scale). It is then used in the forecasting mode, using as input data the DEMETER system (Palmer et al., 2004) large-scale fields. This approach is known as the perfect prog approach (Wilks, 1995). Observed local precipitation The variable of interest is the monthly cumulative precipitation in 48 French catchments. These are mainly concentrated in mid-southern France (Figure 1). The numbers in Figure 1 correspond to their references in the EDF database: from 1 32 and from Historical observed precipitation data have been supplied by the General Technical Division (DTG) of EDF in Grenoble. The database contains daily precipitation data for the period 1 January December All 48 timeseries are complete and homogeneous over this period. Only the period 1 September August 2002, common to the large-scale data from ERA40, will be considered. For each day, the cumulative precipitation in one catchment is calculated using a weighted spatial average of the precipitation recorded by several rain gauges in the area. The locations indicated in Figure 1 correspond approximately to the barycentre of each group of rain gauges. The catchments are concentrated mainly in the Alps, the Pyrenees and the Rhône Valley. Others are located in flatter regions in the centre of France, Brittany and northeastern France. Two catchments are located on Corsica. Very different geographical conditions are thus considered. In order to study the seasonal precipitation, the initial daily time-series were accumulated month by month. Each season has been treated separately. The statistical downscaling operational system used at EDF (Obled et al., 2002) for forecasts up to 10 days uses a 4-month window. This method, based on an analogue approach, has been tested on seasonal forecasts and it will serve as

3 FORECASTING PRECIPITATION FOR HYDROELECTRIC POWER MANAGEMENT 1693 a reference against which to analyse the results obtained with the SVD method. For symmetry with this method, and for reasons of homogeneity when comparing the results, 4-month seasons have also been used in this study: Spring: FMAM (February March April May) Summer: MJJA (May June July August) Autumn: ASON (August September October November) Winter: NDJF (November December January February) The average precipitation is very different between watersheds. The highest values correspond to the northern Alps, the western Pyrenees, the Rhône Valley and Corsica. The variability of precipitation is very high in all catchments: between 50% (watershed number 2, Doubs watershed in the east) and almost 100% (watershed number 25 in the Rhône Valley) of the annual average precipitation. The monthly means of the accumulated precipitation reproduce this high variability. For most watersheds, the maxima of precipitation occur in autumn (October/November) and spring (April/May), the amount of precipitation in autumn being higher. However, for some catchments (numbers 62, 63, 64 and 65 in Figure 1) in the eastern Pyrenees, the higher amount of precipitation occurs in spring. The global minima are in summer, July being, in general, the driest month. Although all catchments share these general characteristics (strong precipitation in spring and/or autumn, weak precipitation in summer), not all of them show a strong variability between seasons. Some of them, mainly those located in the northern Alps (numbers 6, 7, 8, 11, 12 and 13 in Figure 1), exhibit a quite uniform profile of monthly accumulated precipitation through the year. For each watershed, linear trends were examined for individual seasons and for the whole year. If such a trend exists and can be identified, it is then possible to eliminate it from the statistical model. As it is usually not possible to know whether trends in the predictand and the predictor variables are related, possible trends are removed in order to avoid over-fitting. When a linear trend is detected, its degree of significance is determined using a t-test with a significance level of 5%, i.e. the probability of rejecting incorrectly the hypothesis that there is no trend is 5%. The observed annual and seasonal trends were found to be very weak (in absolute values, less than 5 mm per year, or per season, respectively) and not statistically significant. For these reasons, they were not removed. ERA40 data ERA40 (Uppala et al., 2005) is the latest re-analysis data established by the ECMWF, Reading, UK. This database is used as a substitute for real observations even though it is affected by modelling errors. Here, the ERA40 re-analysis is considered as the historical data at the global scale. The available period extends from September 1957 to August The meteorological field used in this study is monthly average geopotential height at 850 hpa (Z850). This field is representative of the large-scale atmospheric circulation in the lower troposphere. This predictor has been chosen apriori as the one nearest to the optimal predictors used in the analogue method at EDF (Obled et al., 2002), i.e. Z1000 and Z700, among those being available simultaneously in the ERA40 database and in the DEMETER project outputs. Weather and climate in Europe and France are mainly influenced by the north Atlantic, particularly at monthly and seasonal timescales (Hurrell, 1995; Robertson et al., 2000; Cassou and Terray, 2001a,b; Drevillon et al., 2001). For this reason, the region of influence used here covers Europe and the north Atlantic (80 W 30 E, 20 N 80 N); 1125 grid points with a 2.5 resolution (about km over France). The monthly average level of Z850 over the studied period, considering all seasons, is located at low altitudes (below 1400 m) in the northern half of the region. In the southern region, including most of Europe and France, this pressure level is located at higher altitudes (above 1400 m). The same behaviour is observed for each season. The lowest geopotential heights occur in winter and the highest ones in summer. The variability is also higher in winter, with a strong northward positive gradient. In summer, this variability is lower and the gradient is less pronounced. Spring and autumn are intermediate periods, although the variability is slightly higher and the geopotential heights are slightly lower in autumn. No significant trend has been detected in the geopotential heights over the period DEMETER hindcasts Once the statistical model has been determined from the ERA40 database, it is tested in a hindcast mode with the Z850 fields issued from the seasonal forecasting models developed within the Development of a European Multimodel Ensemble system for seasonal to inter-annual prediction project (DEMETER). This project aimed to develop and analyse a multi-model system integrating some of the most recent ocean atmosphere coupled models (Palmer et al., 2004). Here, we use results from the DEMETER I system with data from February 1981 to March 2002 from seven models. Each model provides 4 forecasts per year (initialization dates in February, May, August and November) and, for each of these dates, 9 simulations (ensemble members) have been performed for up to 6 months lead time. The spatial resolution of the model outputs is , as for the ERA40 data. These hindcasts have also been supplied by the ECMWF. The main condition for statistically significant good downscaled seasonal forecasts is to have significant good scores for the large-scale seasonal forecasts used as predictors. It is well known that this is a strong constraint

4 1694 M. BENITO GARCIA-MORALES AND L. DUBUS for forecasting in the Euro-Atlantic region, which is characterized by a weak predictability at medium to long timescales (Pavan et al., 2005). This was shown in the exhaustive validation of the DEMETER models within the DEMETER project ( demeter). Concerning the pressure fields, only Z500 and MSLP hindcasts have been validated in this study and both show similar characteristics. There is a little but overall positive skill over the Euro-Atlantic region. Depending on the season and on the model, the anomaly correlation coefficient is not higher than 0.3, the higher scores corresponding to winter hindcast (initialization date in November). From a deterministic point of view, the mean squared skill error, relative to the climatology s mean squared error, is negative in most cases: the DEMETER hindcasts do not reduce the errors from using climatology alone. From the probabilistic point of view, and according to several different scores however, the hindcasts do better than a climatological forecast; the multi-model average performs better than every single model, particularly in winter and spring where it shows a slight positive skill. Despite the moderate skill values, it is justified in studying the potential improvement that statistical or dynamical downscaling can bring to the raw model hindcasts. Several downscaling studies have been carried out using as input predictor Z500 and/or MSLP seasonal forecasts: Pavan et al. (2005) applied a perfect prog approach to estimate winter precipitation over northern Italy by downscaling Z500 seasonal forecasts; Díez et al. (2005) tested the same predictor to forecast precipitation over the Iberian Peninsula using an analog method; Frías et al. (2005) applied a perfect prog downscaling method to forecast Tmax with MSLP seasonal forecasts as input. The choice of Z850 as the predictor in the present study was determined by some studies already performed at EDF. These showed that the geopotential levels at 1000 and 700 hpa were the more skilful predictors for estimating precipitation in EDF catchments. Z850 has been considered, as it was the only available level in the DEMETER hindcasts within this pressure range. As a comprehensive analysis of the Z850 forecast skill is beyond the scope of this work, the results from the validation of Z500 forecasts have been extrapolated, with the assumption that the skill of the lower Z850 level should be of the same order. METHOD DESCRIPTION The SVD method is a classical technique that has been, and still is, widely used for spatial downscaling (Widmann et al., 2003; Pavan et al., 2005; Widmann, 2005). It is based on the principle that a variable can be decomposed as a linear combination of a certain number of vectors affected by a coefficient. This method has been described in detail in Bretherton et al. (1992) and this paper uses the same notations. The SVD method consists of two steps: 1. The relationship between the variable at the local scale (predictand) and the atmospheric information at the global scale (predictor) is determined. This relationship corresponds to the coupled structures of co-variability between these two variables: the coupled-modes of co-variability. These are obtained by the SVD of the covariance matrix between the two variables. The predictor and the predictand can then be expressed in terms of these coupled modes: s(t) N a k (t) u k ; z(t) k=1 N b k (t) v k (1) k=1 where s(t) and z(t) are the temporal series of the predictor and the predictand, respectively, (in our case, the temporal series of the standardized anomalies of the predictor and the predictand), a k (t) and b k (t) are the expansion coefficients of the co-variability mode k, and u k and v k are the singular vectors of the covariance matrix corresponding to the mode k. The expansion coefficients a k (t) and b k (t) inform on the temporal evolution of these modes. Every couple (u k, v k ) represents one mode of co-variability and explains a fraction of the total covariance between the two variables. This allows the classification of the modes dependent on the fraction of the covariance that they explain and limits the study to a restricted number of modes, N, that restore a fixed percentage of the total covariance. 2. In the second step, a transfer function from the global scale information to the local scale is calculated. This is made by a linear regression on the expansion coefficients of the predictor and the predictand associated with the N modes that have been kept. Both a simple and a multiple regression were tested, the last one presenting the best performance. For a given mode k, the expansion coefficient b k (t) of the predictand is estimated from the expansion coefficients of the predictor associated to n N modes, where n is the number of the expansion coefficients a k (t) that contribute significantly (at 5% level) to the linear fitting: b k (t) = θ 1 a 1 (t) θ n a n (t) (2) MODEL DESIGN AND EVALUATION WITH THE ERA40 RE-ANALYSIS The design of the statistical model and its validation are carried out with independent datasets. The ERA40 and the observed precipitation databases are split into two sets: 75% of the data are used to train the statistical model and the validation is performed on the remaining 25%. A cross-validation process is performed to test the robustness of the model. The validation dataset, which is a continuous time series, moves through all the available time series, and at each step the model s parameters are calculated using the rest of the data. At the end, a validation by pieces is obtained over the whole period

5 FORECASTING PRECIPITATION FOR HYDROELECTRIC POWER MANAGEMENT 1695 ( ). The errors (correlations) presented below were calculated over the whole available period and can be considered as a superior (inferior) bound of the errors (correlations) associated with the final model, which is determined over the whole period. In other words, it is expected that the final model will perform better than this model by pieces. The correlations between the estimated monthly accumulated precipitation for each watershed and each season are shown in Figure 2. For all catchments, the best performances for this score correspond to spring and winter, and the worst to summer. For a given season, the model performs differently depending on the catchment. Correlation values present a regional distribution. In spring (Figure 2(a)), three groups of catchments can be identified, according to their correlation values: the catchments in the northern Alps in the centre, and in mid-northern France, with correlations higher than 0.6; the catchments in the southern Alps and western Pyrenees, with correlations of about ; and the catchments in the Rhône Valley and the eastern Pyrenees, with correlations lower than 0.4. In winter (Figure 2(d)), the general pattern is similar, but the correlations are slightly higher, in particular in the Rhône Valley (correlations about ). In autumn (Figure 2(c)), the structure changes a little: the Alps and Rhône Valley catchments have the highest correlations ( ). In summer (Figure 2(b)), all the correlations are lower than 0.5, and it is more difficult to group catchments with respect to correlation. The root mean square errors (RMSEs), presented in Figure 3, are very variable between catchments for every season, from 30 to 90% of the seasonal average of the monthly accumulated precipitation for each catchment. The values are similar for all seasons. The largest errors correspond to the southern Alps and Rhône Valley catchments. SVD results compared to a climatologic forecast The operational method currently used at EDF at monthly and seasonal lead times consists of considering as a forecast the climatology of each catchment, i.e. the average of the observation data over a season (seasonal climatology) or over a month (monthly climatology). The RMSE error associated with use of these original climatologies as well as the RMSE of the SVD method are shown in Figure 4. To compare the RMSE for the two methods we compute the reduction of error statistic: RE = 1 RMSE SVD /RMSE Clim (3) Figure 2. Correlations between the downscaled monthly precipitation (from the ERA40 Z850 field) and the observed accumulated precipitation for every season and the whole period

6 1696 M. BENITO GARCIA-MORALES AND L. DUBUS Figure 3. RMSE between the downscaled monthly precipitation (from the ERA40 Z850 field) and the observed accumulated precipitation for every season and the whole period The values are as a percentage of the seasonal average of the monthly accumulated precipitation for each catchment. Figure 4. Comparison between the SVD results (with the ERA40 Z850) field and the seasonal/monthly climatology for every season.

7 FORECASTING PRECIPITATION FOR HYDROELECTRIC POWER MANAGEMENT 1697 On average, over all catchments, the reduction of error in forecasts when using the SVD method is 20 21% in winter depending on the climatology (Figure 4(d)), 18% in spring (Figure 4(a)), 14 16% in autumn (Figure 4(c)) and 9% in summer (Figure 4(b)). This reduction of error depends not only on the season but also on the catchment. For example, the reduction of error by using the SVD method in winter for catchment 65 (Figure 1) is about 5 8%, while in catchment 32 (Figure 1) the reduction is 40%. In some catchments the downscaling model produces a large improvement in forecasting compared to a raw climatologic prediction. SVD results compared to the analogues method forecast An analogue-based method is used operationally at EDF to produce weekly precipitation forecasts for the 48 catchments. This method is being tested, at present, at the monthly and seasonal scales, and the first results are compared to those produced by the SVD methods for the Ain catchment (number 4 in Figure 1). This catchment is the second most important in terms of production capacity (450 hm 3 of potentially useful water). Moreover, its pluviometric regime is interesting as this catchment is not only under the influence of maritime flows, but also the southern flows from the Mediterranean Sea. A full description of the analogue method can be found in Obled et al. (2002) but, briefly, this method selects at day J, from the past 50 years of daily observations of geopotential heights at 700 hpa and 1000 hpa, the days in which the pressure situations were similar to the geopotential forecasts for days J and J + 1 (analogues). For a given date, the analogues are selected within a pool of observations starting 2 months prior to the given date and terminating 2 months after. The corresponding observed precipitation is then extracted from the database. For every forecast, 50 analogues are selected, and a probabilistic precipitation forecast is provided. This method has a daily time-step and an 8-day lead time. The results presented here have been obtained for the ERA40 re-analysis at a daily time-step and then accumulated over a month. The results obtained by this method are thus expected to be more skilful than those obtained by the SVD method, but they will serve as a reference to determine the skill of the SVD model. The correlation associated with the analogues method for the Ain catchment is indeed higher than that for the SVD method for every season (Table I), the largest difference in summer (0.28) and the smallest in winter (0.12). The analogues method is, unlike the SVD method, biased (Table II): in winter and spring there is a positive bias (the downscaled mean value is higher than the observed mean value, which is represented by a ratio higher than 1 in Table II) and there is a negative bias for the other seasons. The analogue results need to be unbiased. When this is performed, the RMSE, as a percentage of the catchment s observed mean precipitation, decreases slightly, particularly in summer. Table III shows this score for the SVD and the analogues methods, both, biased and unbiased. As expected, the latter is more skilful in every season, the best results being obtained when unbiased. The SVD method shows a RMSE about 10% higher than the unbiased analogues results. According to the correlation values, the best results are obtained, for both methods, in winter and spring. As mentioned above, the analogues method is, by construction, expected to be more skilful than the SVD method due to the fact that the estimated monthly precipitation is calculated from daily values of both the local precipitation and the large-scale geopotential heights. However, this method, as currently implemented, cannot be directly applied to seasonal forecasts, as their skilful time-step is for a month or, at best, a week. The SVD method integrates directly information at the monthly scale with comparable results to the analogues method. In an operational system, the SVD method can, therefore, provide additional information at the seasonal scale to be used as a complement to the 10-day analogues forecasts. Table I. Correlation values between the observed accumulated precipitation and the downscaled accumulated precipitation for the SVD and the analogues methods (Ain catchment). Correlation (Ain catchment) SVD Analogues Spring Summer Autumn Winter Table II. Ratio between the mean downscaled accumulated precipitation and the observed mean accumulated precipitation, for the SVD and the analogues methods (Ain catchment). Bias (Ain catchment) SVD Analogues Spring Summer Autumn Winter Table III. RMSE associated with the SVD method and the biased and unbiased analogues method (Ain catchment). RMSE (Ain catchment) SVD (%) Analogues (%) Analogues (unbiased) (%) Spring Summer Autumn Winter

8 1698 M. BENITO GARCIA-MORALES AND L. DUBUS DOWNSCALING DEMETER SEASONAL HINDCASTS The final parameters of the downscaling statistical model tested above are calculated over the whole period of the ERA40 database ( ). This downscaling model, calculated at a monthly time-step, is then applied to the outputs of the seven ensemble GCMs of the DEME- TER project for the period The downscaled monthly accumulated precipitation (or the corresponding anomaly) obtained is averaged over the corresponding season, as it is the relevant time-step of seasonal forecasts. The results presented below thus correspond to 4 monthly averages, as described in Section Observed local precipitation. Two approaches have been tested: a deterministic approach and a probabilistic one. In the first one, an individual model hindcast is the average of the 9 members of each ensemble for each of the 7 models; the multimodel hindcast is an average of the 7 individual model hindcasts. The downscaling model is applied to these averaged hindcasts. In the probabilistic approach, every member s scenario is downscaled for each model and for the multi-model. In this section, we present the result in terms of the downscaled standardized anomalies of the monthly accumulated precipitation. Deterministic results The correlation between the observed data and the downscaled deterministic DEMETER forecasts is lower than 0.5 for all seasons and all models, including the multi-model deterministic forecast (Figure 5). Moreover, it takes negative values, up to 0.5, in more than 50% of the forecasts, except in autumn, where this rate is 36%. The higher percentage of negative correlation values is mainly for winter (75%). However, for all catchments there is at least one individual model forecast with a positive correlation. There is not one model that performs systematically better than the others in terms of correlation and the multi-model does not perform better than the individual models. The multi-model correlation is never the highest. In winter, it is positive only for one catchment. In autumn, spring and summer, the multimodel correlation is positive in 85, 48 and 25% of the catchments, respectively. The RMSE for the downscaled standardized anomalies lies in the interval mm/season (30 80%) for all seasons and all models (Figure 6). The error values are catchment-dependent. For one particular catchment, the difference between the highest and the lowest error values (corresponding to different forecasting models) is about 0.1 mm or less (10% of the observed mean values in the case of the reconstructed accumulated precipitation). The scores also depend on the season. On average, over all the catchments, the RMSE in spring and summer for the different models varies between mm in terms of the anomalies (26 30% in terms of the accumulated precipitation); these values are slightly higher for the two other seasons: mm Figure 5. Correlations between observations and the downscaled DEMETER hindcasts (individual models hindcasts and multi-model hindcasts).

9 FORECASTING PRECIPITATION FOR HYDROELECTRIC POWER MANAGEMENT 1699 Figure 6. RMSE between the observations and the downscaled DEMETER hindcasts (individual model hindcasts and multi-model hindcasts). (31 34%) in autumn and mm (33 37%) in winter. With respect to these scores, the multi-model forecast always performs as well as the best individual model, but not better. Indeed, it does not improve upon the climatologic forecast in any season, except in autumn. It does not perform better than the climatology in winter for any catchment, and does so only in 25% of the catchments in spring and 17% in summer. In autumn, this percentage increases to upto 65% of the catchments, but with a very small difference between the climatology and multi-model RMSE (less than 0.01 mm anomaly). In summary, the deterministic approach does not provide any extra information compared to the use of climatology alone. To improve the forecast process, the hindcasts have to be used with a probabilistic approach as will be shown in the next section. Individual model probabilistic results When analysing the results probabilistically, i.e. downscaling every scenario proposed by every ensemble model and then calculating probabilistic scores, the first interesting thing is to calculate how many times, through the DEMETER hindcasts period, the observations fall into the ensembles for every individual model. This is very variable depending on the season. The number of catchments in which the observations are in the individual model s ensemble more than 50% of the time is in spring (from 48 catchments) depending on the GCM, in summer, in autumn and in winter. For all seasons, these catchments are located mainly in the western Pyrenees, the northern Alps and in central and northern France. Figure 7 shows an example of the ability of the ensembles to catch the observations for the Ain catchment (number 4 in Figure 1), located in the northern Alps. The rank histograms (Talagrand s diagrams) in Figure 7 are constructed as follows: for one particular season, all the downscaled ensemble s forecasts are ordered, which gives a number of bins; then the observation for one particular date is placed in the corresponding bin. This is repeated for all the available dates and the total number of cases per bin is represented in a histogram. In the case of the Ain catchment, the spread of the ensemble is too wide in spring, autumn and winter (Figure 7(a), (c) and (d)) as the observation almost never falls outside the range of the forecasts (first and last bins), and almost all the observations fall in the intermediate bins. This over-spread is observed in most catchments. In summer (Figure 7(b)) the histogram is, on the contrary, quite flat, which indicates

10 1700 M. BENITO GARCIA-MORALES AND L. DUBUS Figure 7. Rank histogram of the downscaled standardized anomalies for the Ain catchment and all seasons. that the probability of the distribution has been better sampled. The Brier Score (BS) measures the difference between the forecast probability of an event and its frequency of occurrence, expressed as 0 or 1, dependent on whether the event occurred or not. As with RMSE, the lower the BS the better the skill. To compare the BS of the SVD method with the BS of a reference method, the Brier Skill Score (BSS) is used. It is conventionally defined as the relative probability score compared with the probability score of the reference forecast (Persson and Grazzini, 2005). The reference forecast is, in general, a probabilistic climatologic forecast, which considers a climatologic ensemble that corresponds to the ensemble of the observed anomalies of the monthly accumulated precipitation through the ERA40 period. The BSS for correct forecasts of the sign of the anomaly, with respect to the probabilistic climatologic forecast, is negative in most catchments for all seasons and all GCMs. This means that individual model probabilistic forecasts do not improve upon the score of a climatologic forecast. Most models show a positive BSS in no more than 10 catchments (out of 48) for every season. The maximum positive values vary from one model to another. Depending on the season, some models produce scores of , while for others these values are lower than 0.1. Therefore, individual models do not provide additional valuable information on average. Multi-model probabilistic results In the multi-model probabilistic approach, the ensemble of forecasts is formed by all members of all the individual GCMs, i.e. 63 different scenarios for each hindcast date. The skill in this approach is also very limited and of the same order as the individual forecasts, but it depends on the studied event. In winter, there is no skill for correctly predicting the sign of the anomaly, which means that there is no improvement with respect to the use of a climatologic forecast only. For the other seasons, there is slightly more skill, with more than 9 catchments out of 48 showing a positive BSS and maximum values of 0.1. The BSS has been calculated for 5 other events: the probability of correctly predicting an anomaly in the lowest 15% quantile of the climatology (BSS15), in the lowest tercile (BSS33), in the middle tercile (BSS66), in the upper tercile (BSS99) and in the upper 15% quantile (BSS85). The scores are different for every event and every season. On average, there is no skill for the multi-model approach nor for the individual model s forecasts compared to the climatology. However, for particular catchments, there is positive skill. Figure 8 shows, for every season, the event that shows the strongest positive signal in the BSS in terms of the number of catchments in which this score is positive. In spring, the events corresponding to the three lowest percentiles (BSS15, in Figure 8(a), BSS33 and BSS66) show more positive BSS through the

11 FORECASTING PRECIPITATION FOR HYDROELECTRIC POWER MANAGEMENT 1701 different catchments, their values varying between 0.01 (almost no skill) and 0.3. These catchments are mainly located in the Rhône Valley and the Pyrenees (BSS15 and BSS33) and in the northern Alps. There is no skill for the other two events. In summer, there is no skill for the two lowest percentiles (BSS15, BSS33), and the events with more positive skill score are BSS66, BSS99 and BSS85 (in Figure 8(b)) with globally smaller values than in spring, varying from The catchments with a more positive signal in this season are mainly located in the centre and north of France. In autumn, it is the event associated with the lowest percentile (BSS15, in Figure 8(c)) that presents the highest number of positive scores (values between ). Unlike the other seasons, all the events have a positive BSS in autumn in some catchments, more than 14 and mainly those in the Pyrenees, with values from 0.01 to 0.2. Finally, in winter the BSS is low. Only the event corresponding to the middle tercile, BSS66 (Figure 8d), shows a slightly positive BSS in some catchments, with very small values (between and 0.1). Table IV summarizes the BSS for every season and every event. It shows the number of catchments in which the scores have a positive signal, that is, positive values of BSS and maximum values. In summary, even if there is no clear and significant signal in the BSS, the results show that: in spring, the probabilistic multi-model ensemble seems to provide better forecasts for some catchments of the lowest percentiles (deficiency of precipitation) and the middle tercile (precipitation around normal values); in summer, forecasts of precipitation above and around the normal value present the highest skill (middle tercile and the two upper percentiles); in winter, there is almost no skill, but a slightly positive signal in forecasting precipitation around normal values (middle tercile); finally in autumn, all the events show a positive signal in some catchments, the most significant being for the lowest percentile BSS15. Hence, the downscaling model does not show a systematic improvement for all the catchments in all seasons, but rather, local improvements in one season or another, for different events. This information, even though not homogeneous for all the catchments, could be used operationally, at least in a qualitative way, to help decision makers. Another score used to evaluate the probabilistic forecasts is the area under the ROC curve (ROCA). Probabilistic forecasts can be transformed into categorical yes/no forecasts defined by some probability threshold. For different thresholds the corresponding hit rates, H, and false alarm rates, F, can be computed. This information can be displayed in the two-dimensional Relative Operating Characteristics or (ROC) diagram with H defining the x-axis and F the y-axis. A point in the Figure 8. BSS of the most skilfully forecast event for every season, corresponding to the downscaled multi-model DEMETER hindcasts: lowest 15% percentile in spring (a), upper 15% percentile in summer (b), lowest 15% percentile in autumn (c) and middle tercile in winter (d).

12 1702 M. BENITO GARCIA-MORALES AND L. DUBUS Table IV. Number of catchments where the BSS (ROCA) for a given event is higher than 0 (0.5) and the maximum value attained by this score, corresponding to the downscaled multi-model DEMETER hindcasts. Spring Summer Autumn Winter Event Number of catch. BSS (ROCA) Number of catch. BSS (ROCA) Number of catch. BSS (ROCA) Number of catch. BSS (ROCA) 15% 26 (45) 0.11 (0.80) 3 (18) 0.02 (0.86) 27 (26) 0.19 (0.90) 6 (15) 0.09 (0.84) 33% 19 (25) 0.27 (0.87) 2 (19) 0.05 (0.63) 21 (29) 0.12 (0.77) 3 (9) 0.04 (0.61) 66% 19 (31) 0.13 (0.81) 16 (24) 0.16 (0.84) 17 (25) 0.12 (0.76) 21 (25) 0.12 (0.79) 99% 3 (9) 0.11 (0.78) 10 (15) 0.20 (0.83) 16 (25) 0.13 (0.74) 1 (5) 0.02 (0.55) 85% 2 (10) 0.12 (0.74) 13 (30) 0.22 (0.90) 19 (30) 0.14 (0.81) 5 (11) 0.05 (0.71) Sign 8 (15) 0.11 (0.78) 7 (16) 0.11 (0.70) 14 (21) 0.12 (0.65) 1 (5) 0.01 (0.63) ROC diagram is then defined by the false alarm rate on the x-axis and the hit rate value on the y-axis (Persson and Grazzini, 2005). A perfect forecast system has all its points in H = 100% and F = 0. The area under the ROCA represents the skill of the forecast. An area of 1 represents a perfect forecast; an area of 0.5 represents a random forecast. If the ROC area is lower than 0.5, the probabilistic forecast does not improve upon a random forecast. The values in brackets in Table IV give the number of catchments, for every season and every event, in which the multi-model (and the individual models) show a ROCA value higher than 0.5. Once more, the multi-model forecasts are not systematically better than the individual forecasts, but this depends on the season and the studied event. In winter, the multimodel forecasts show a ROCA higher than 0.5 in more than half the catchments but only for the event associated with the middle tercile, with a maximum value of In spring, the score is higher than 0.5 in more than 50% of the catchments when forecasting the anomalies in the two lowest percentiles (lowest of 15 and 33%) and in the middle tercile, with maxima of 0.80, 0.87 and 0.81, respectively. In autumn, all the events show a ROCA superior to 0.5 in more than 50% of the catchments; in the case of the upper percentile (upper 15%) this percentage reaches 62%, with a maximum ROCA of Finally, in summer the multi-model shows a positive skill (ROCA higher than 0.5) most frequently when forecasting anomalies in the middle tercile and the upper 15% percentile, the maxima being 0.84 and 0.90 respectively. In any case the multi-model does not perform the worst, compared to the individual models. It has to be stressed that most of the score values are not strictly statistically significant as the sample size is restricted, particularly for the events in the extreme percentiles. The number of occurrences of a particular event is necessarily less than 22 (the DEMETER hindcasts are only available over the past 22 years) and for some events, particularly those concerning the extreme percentiles, the number of occurrences may only be 2 or 3. The results obtained must therefore be taken only as an indication of the potential behaviour of these forecasts. An independent validation (not detailed here) has been carried out by downscaling the hindcasts of the DEME- TER II system, composed of only three models over (which means, only 27 members in the multimodel ensemble), but which covers twice the length of the time period than that of the DEMETER I system (7 models over , used as a reference in this paper). The deterministic and probabilistic scores corresponding to the downscaled DEMETER II hindcasts are of the same order as those corresponding to DEME- TER I. There is some positive skill that corroborates the results obtained with the DEMETER I system. In terms of the deterministic approach (ensemble mean), the multimodel performs slightly worse for DEMETER II than for DEMETER I. This is probably partly due to the fact that, in the first case, the multi-model forecast corresponds only to the average of 27 members, while in the second one it corresponds to a 63-member average. In the probabilistic approach, the scores are quantitatively similar, and the problem of significance remains. The events and the catchments that present positive skill are not necessarily the same for both forecasting systems. However, there are overall positive signals for the performance of forecasts. In terms of the BSS, for instance, autumn is the best predicted season, using both systems, presenting the highest scores for every event; in spring and summer, the scores obtained using DEMETER I forecasts in the middle tercile are better than the results obtained with DEMETER II; in winter, better results are obtained with DEMETER II. Two particular examples To illustrate the kind of information that could be valuable for EDF, two examples are presented below. This exercise shows a way in which the probabilistic seasonal forecasts could be used in practise. The event forecasting anomalies in the lowest 15% percentile in autumn is of particular interest because it represents a situation in which precipitation is far below the normal values in a season where precipitation is expected. If there is a strong and reliable probability that the autumn will be dry, this could affect the management of the catchments in the previous months, for example,

13 FORECASTING PRECIPITATION FOR HYDROELECTRIC POWER MANAGEMENT 1703 by limiting the power production of the corresponding hydroelectric plants. An example of a catchment in which the results show little information is the Ain catchment located in the northeastern Alps (number 4 in Figure 1). Figure 9(a) shows a boxplot of all the scenarios provided by the different models (multi-model forecast), as well as a boxplot of the observed values through the ERA40 period. In this case, there is almost no skill when using the BS (3% of improvement with respect to the climatology), although the ROCA indicates that there is some skill in the multi-model forecasts, with a value of The boxplots show only two occurrences of this event in 22 years, which are not predicted by the multi-model hindcast with a higher probability than the climatology. Figure 9(b) shows, for every year, the probability of an anomaly in the lowest 15% percentile. If the model performs well, one would expect this probability to be higher than 15% (corresponding to the climatology) when this event was effectively observed. In the case of the Ain basin, there were five false alarms with a multimodel probability higher than 20% and no hits. When the event occurred, the seasonal forecasts did not add any information to the climatology. On the other hand, catchment 62 in Figure 1, located in the Pyrenees, presents good skill when forecasting this lowest 15% event. The BSS is 0.19, which means an improvement of 19% with respect to the climatology. The boxplots (Figure 10(a)) show a great number of observed negative anomalies over the studied period, and the multi-model also tends to forecast negative anomalies. There are four extreme observed negative anomalies (in the lowest 15% percentile), and at least three of them were forecasted with a probability higher than 20% by the multi-model (Figure 10(b)). The false alarms have associated probabilities of 20% or less. This event is well represented by the multi-model forecasts, which is confirmed by the high ROC area of 0.9. If this seasonal multi-model ensemble forecast indicates a relatively high probability of having a strong negative anomaly in this catchment, this information would be used in operations, which would not be the case for the Ain catchment. DISCUSSION The first part of this work has concerned the evaluation of a downscaling method based on a SVD process. Globally, the error scores obtained using the ERA40 re-analysis are of the same order, though slightly higher, than those corresponding to the analogues method operational at EDF, when applied to monthly/seasonal timescales. This means that the SVD method, simpler to implement at present, gives comparable results to the EDF downscaling reference model. The results showed that the performance of the SVD method depends on the season and on the basin. The fact that the summer monthly accumulated precipitation is globally less well estimated is likely to be related to the lower influence of large-scale pressure structures (the predictor) upon local precipitation in this season Figure 9. Boxplots of the multi-model members (9(a)): every member is represented by x and the observation by o, in grey to the right, the boxplot of all the observations at the Ain catchment through the ERA40 period. In 9(b), the probabilities of predicting an anomaly in the lowest 15% percentile in autumn for every hindcast: OK represents an occurrence of the event, and x a non-occurrence of the event.

14 1704 M. BENITO GARCIA-MORALES AND L. DUBUS Figure 10. Boxplots of the multi-model members (10(a)): every member is represented by x and the observation by o, in grey to the right, the boxplot of the all the observations at a Pyrenees catchment through the ERA40 period. In 10(b), the probabilities of predicting an anomaly in the lowest 15% percentile in autumn for every hindcast: OK represents an occurrence of the event, and x a non-occurrence of the event. compared to local convective processes which tend to have more influence in summer than in winter. The SVD downscaling method produces different scores for different catchments. In particular, the precipitation in catchments of the Rhône Valley, and some catchments of the southern Alps is, in general, estimated more poorly than the others. The main reason for this may be that the histograms of their observed accumulated precipitation are far from being Gaussian: they are very asymmetric and have strong positives skew (figures not shown). The linear processes, such as those involved in the SVD method, do not work well in these cases. Further studies are in progress to find the most appropriate transformation to normality of these histograms; the SVD method then will be tested to determine if there is a significant improvement of the downscaling results for these catchments. In the second part of this study, the statistical SVD downscaling model was applied to the seasonal outputs of the 7 models of the DEMETER project. The main aim was to analyse the information (or the lack of information) that this seasonal forecasting system can provide for precipitation at some catchments of particular interest for EDF. The deterministic approach is of no particular interest compared to the use of the catchments climatology. The error is not reduced by the seasonal forecasts. The probabilistic approach is more informative, although only in a qualitative way. In fact, in most cases, the scores obtained were not statistically significant. The probabilistic scores are calculated using only 22 points per season, and 9 members per model, which means that there are likely to be few occurrences of particular events, especially for more extreme events. Thus, the calculated probabilities cannot be very representative. However, valuable qualitative information was obtained on how to interpret the probabilistic forecasts and the ways in which they may be used. Globally, there is a slight positive signal for whether the probabilistic forecasts improve the climatology. The studied scores, BSS and ROCA, are clearly positive in some catchments and for some events. In general, forecasts in autumn are the most skilful, followed by spring and summer. In winter, there is almost no skill. The more skilful events are the probability of negative anomalies in the lowest 15% percentile in autumn and spring. In the latter, the forecasts for the lowest 33% tercile are also quite skilful. In summer, the best forecasts are obtained for the strongly positive anomalies (upper 15% percentile). There is also a geographical component to the distribution of catchments in which the seasonal probabilistic forecasts works better than the climatology. For some events, the downscaled probabilistic forecasts show positive skills where the SVD method does not work well