Assessment of the ENSEMBLES regional climate models in the representation of precipitation variability and extremes over Portugal

Transcription

1 JOURNAL OF GEOPHYSICAL RESEARCH, VOL. 117,, doi: /2011jd016768, 2012 Assessment of the ENSEMBLES regional climate models in the representation of precipitation variability and extremes over Portugal Pedro M. M. Soares, 1,2 Rita M. Cardoso, 1 Pedro M. A. Miranda, 1 Pedro Viterbo, 1,3 and Margarida Belo-Pereira 3 Received 23 August 2011; revised 1 March 2012; accepted 1 March 2012; published 14 April [1] A new data set of daily gridded observations of precipitation, computed from over 400 stations in Portugal, is used to assess the performance of 12 regional climate models at 25 km resolution, from the ENSEMBLES set, all forced by ERA-40 boundary conditions, for the period. Standard point error statistics, calculated from grid point and basin aggregated data, and precipitation related climate indices are used to analyze the performance of the different models in representing the main spatial and temporal features of the regional climate, and its extreme events. As a whole, the ENSEMBLES models are found to achieve a good representation of those features, with good spatial correlations with observations. There is a small but relevant negative bias in precipitation, especially in the driest months, leading to systematic errors in related climate indices. The underprediction of precipitation occurs in most percentiles, although this deficiency is partially corrected at the basin level. Interestingly, some of the conclusions concerning the performance of the models are different of what has been found for the contiguous territory of Spain; in particular, ENSEMBLES models appear too dry over Portugal and too wet over Spain. Finally, models behave quite differently in the simulation of some important aspects of local climate, from the mean climatology to high precipitation regimes in localized mountain ranges and in the subsequent drier regions. Citation: Soares, P. M. M., R. M. Cardoso, P. M. A. Miranda, P. Viterbo, and M. Belo-Pereira (2012), Assessment of the ENSEMBLES regional climate models in the representation of precipitation variability and extremes over Portugal, J. Geophys. Res., 117,, doi: /2011jd Introduction [2] Regional Climate Models (RCMs) are increasingly used to assess the impact of climate change at regional and local scales [Giorgi and Mearns, 1999; Wang et al., 2004; Christensen and Christensen, 2007]. While Global Climate Models (GCMs) are suitable to study global atmospheric properties and global warming, the understanding of regional or local effects can be directly addressed through regional climate modeling. In regions where local features affecting the atmospheric flow, such as topography and coastal processes, are prevalent, finer resolution simulations with state-of-the-art mesoscale models are required to reproduce observed weather and climate [Mass et al., 2002; Salathé et al., 2008]. [3] The ENSEMBLES project [van der Linden and Mitchell, 2009] produced a set of multimodel RCM simulations to characterize climate change in Europe, including 1 Instituto Dom Luiz, CGUL, University of Lisbon, Lisbon, Portugal. 2 ADEC, Instituto Superior de Engenharia de Lisboa, Lisbon, Portugal. 3 Instituto de Meteorologia, Lisbon, Portugal. Copyright 2012 by the American Geophysical Union /12/2011JD an estimate of model uncertainty. Thirteen different RCMs used ERA-40 reanalysis [Uppala et al., 2005] as boundary conditions for the ENSEMBLES simulations of current climate, constituting an ideal setup for the assessment of model quality. ENSEMBLES RCM results have been evaluated at the European scale in many studies [e.g., Christensen et al., 2010; Kjellström et al., 2010; Rauscher et al., 2010; Lorenz and Jacob, 2010;Boberg et al., 2010;Sanchez-Gomez et al., 2009; Lenderink, 2010], and in more local contexts, e.g., Kostopoulou et al. [2009] in the Balkans, Herrera et al. [2010] in Spain, and Heikkilä et al. [2010] in Norway. The European studies emphasized a number of conclusions: a general good agreement of RCMs with observed precipitation, the need for dense observational networks in regions of complex topography, but a poor reproduction by RCM of the precipitation PDFs over different regions, as the Iberian Peninsula. In particular, Kjellström et al. [2010] found that biases in daily precipitation were more important in the wettest part of the probability distributions, where RCMs predominantly overestimate precipitation. These authors showed that winter daily precipitation skill scores were high in large areas of Europe, albeit with a poorer performance in the Mediterranean area. Sanchez-Gomez et al. [2009] studied the ability of ENSEMBLES RCMs to reproduce the weather regimes over 1of18

2 Table 1. ENSEMBLES Regional Climate Models Institution Reference Model Acronym Abdus Salam International Centre for Theoretical Physics Pal et al. [2007] REGCM3 ICTP Centre National de Recherches Meteorologiques Radu et al. [2008] RM4.5 CNRM Danish Meteorological Institute Christensen et al. [2006] HIRHAM5 DMI Hadley Center - UK Met Office Collins et al. [2006] HadRM3Q3 HadRM3Q3 HadRM3Q16 HadRM3Q16 HadRM3Q0 HadRM3Q0 Koninklijk Nederlands Meteorologisch Instituut van Meijgaard et al. [2008] RACMO2 KNMI Max Planck Institute for Meteorology Jacob et al. [2001] REMO MPI Norwegian Meteorological Institute Haugen and Haakenstad [2005] HIRHAM METNO Swedish Meteorological and Hydrological Institute Samuelsson et al. [2011] RCA3 SMHI Swiss Institute of Technology Jaeger et al. [2008] CLM ETHZ Universidad de Castilla la Mancha Sánchez et al. [2004] PROMES UCLM Europe-Atlantic in the ERA-40 period, showing a large discrepancy in day-to-day weather regimes history, more noticeable in summer than in winter. These authors didn t identify a significant impact of higher resolution on the representation of weather regimes by the RCMs. [4] Precipitation is a crucial variable when assessing climate change impacts, due to its direct influence on hydrology, agriculture and energy. Within the ENSEMBLES framework, Rauscher et al. [2010] focused on the analysis of model resolution dependency of seasonal precipitation, indicating that most of the models overpredict precipitation, while the majority benefits from higher resolution. Recently, Heikkilä et al. [2010] compared Norwegian precipitation results of a Weather Research and Forecast (WRF) model [Skamarock et al., 2008] simulation (two nested grids at 10 and 30 km resolution) with the 25 km ENSEMBLES RCMs, concluding for an improvement of the histograms and quantiles of daily precipitation in WRF, largely attributed to the 10 km higher resolution. [5] Herrera et al. [2010] focused on the analysis of the Spanish mean and extreme precipitation regimes representation by ENSEMBLES RCMs, comparing model grid point data and integrated river basin precipitation with the corresponding values of a recent Spanish observational gridded data set, with 0.2 grid spacing [Herrera et al., 2012]. These authors conclude that models have a good overall description of the mean precipitation and seasonal cycle, but with large spread. However, ENSEMBLES models present an overestimation of rainfall frequency leading to an erroneous representation of wet and dry spells. Moreover, extreme events are poorly reproduced by the RCMs. This last study is particular important to the current paper, since Spain is the only country with borders with Portugal, and share most of the meteorological phenomenology and modeling challenges. [6] The precipitation regime in Iberia is characterized by large intra and interannual variability and high spatial heterogeneity [Soares et al., 2012; Esteban-Parra et al., 1998; Muñoz-Díaz and Rodrigo, 2004] and is known for its sensitivity to climate change in what concerns precipitation [Giorgi, 2002]. Iberian variability is partially due to the Mediterranean climate properties, but it is substantially enhanced by complex topography and coastal processes. Portugal and Spain, side by side on Iberia, share most atmospheric systems. However, being at the western edge, Portugal has significantly higher mean annual precipitation, with a large mean precipitation gradient; over a distance of 500 km, the mean annual precipitation varies from 2500 mm in the NW, to circa 400 mm in the SE. [7] The evaluation of high-resolution RCMs crucially requires a dense climate observation network [Laprise et al., 2008; Rauscher et al., 2010], absent over most of the world. ENSEMBLES results validation has been based mostly on ECAD gridded observations [Haylock et al., 2008],which relied on no more than 20 stations in Portugal, a number that is largely insufficient to represent observed spatial heterogeneity. Recently, Belo-Pereira et al. [2011] developed a new daily precipitation gridded data set over mainland Portugal, that was merged with the above mentioned Spanish data set [Herrera et al., 2012] creating a high resolution ( ) Iberian Peninsula regular grid. This data set covers the period from 1950 to 2003 and is based on a dense network of rain gauges, about 2400 in Spain, and more than 400 in Portugal. [8] In this study, the new Portuguese precipitation data set is used to evaluate the ability of 12 ENSEMBLES RCMs to capture the complex spatial and temporal variability of precipitation in Portugal. The analysis looks at grid point, river basin and mean country precipitation, characterizing its synoptic variability, seasonal cycle, interannual variability and extreme events. The study complements the results of Herrera et al. [2010] in its regional extension and broadens the analysis of extreme precipitation events. [9] This paper is organized as follows. Section 2 describes the observational data set and the models. Section 3 presents an analysis of RCM performance on grid point, river basin and mean country precipitation. A summary and conclusions are presented in section Data and Methods 2.1. Model Data [10] RCM simulations used in the present study were performed in the framework of the ENSEMBLES project ( aiming to develop a multimodel ensemble prediction system based on state-of-the-art RCMs to study climate change in Europe and estimate its uncertainty [van der Linden and Mitchell, 2009]. Here, 12 RCMs from ENSEMBLES with available daily data (Table 1) are evaluated. The RCM simulations cover the period Model domains are slightly different, but share a common minimum domain. The horizontal resolution of the models is similar, of the order of 25 km, with subtle differences associated with the map projection. More details can be found in the ENSEMBLES project website [Christensen et al., 2010]. 2of18

3 Figure 1. Portuguese orography (m) according to Gtopo 30 data set (30 resolution), ENSEMBLES RCMs (25 km resolution) and ERA-40 reanalysis (1.125 resolution). Main Portuguese river basins. [11] Only daily values of precipitation at land grid points are taken into account. Twenty-five and 50 km model resolutions are available, but only the higher resolution output is considered here; the horizontal spacing closer to the observation data set allows a direct grid-box to grid-box comparison, although both observational gridded data sets and model results are not expected to be representative of the fine spatial details. ERA-40 reanalysis data [Uppala et al., 2005] is also used as a reference for RCMs results Observational Data [12] The observations rely on the Portuguese grid precipitation data set developed by Belo-Pereira et al. [2011], at , from observed rain gauges daily precipitation. This data set covers the period , and will be referred as ObsIM_0.2. Two other well established, but lower resolution, observational data sets are also used: the Climate Research Unit [Mitchell and Jones, 2005] data set, containing monthly precipitation at 0.5 resolution (hereafter CRU), and the version 4 of the ENSEMBLES observational gridded data set for Europe (E-OBS) [Klein Tank et al., 2002; Haylock et al., 2008; Klok and Klein Tank, 2009], which includes daily precipitation values at 0.25 (herein ObsENS_0.25). However, errors are not computed for the latter grids since those were already explored in previous studies. [13] The ObsIM_0.2 data set was not corrected for undercatch. Weedon et al. [2011] developed a global monthly grid of undercatch correction factors, following Adam and Lettenmaier [2003]. Applying those factors to Portuguese precipitation leads to a modest increase of annual precipitation below 5%. However, Weedon et al. [2011] did not take into account orographic effects [Adam et al., 2006], so the uncertainty of gridded precipitation in high terrain areas is largely an open question Comparison Methods [14] RCM precipitation output is bilinearly interpolated on to the higher resolution observational grid ( ), following Herrera et al. [2012], allowing for a direct comparison at the grid point level. High resolution studies have shown that the simple interpolation of precipitation fields is questionable, considering the influence of local winds [Esteban and Chen, 2008] and the intrinsically asymmetric nature of topographic precipitation and its dependence on vertical profiles of moisture and buoyancy [Smith and Barstad, 2004]. However, since there are no consensual interpolating methods that can be used generally, no topographic correction is applied here. [15] On the other hand, because Portuguese climatological data is computed from 9 to 9 UTC whereas ENSEMBLES RCMs data is archived for the interval 0 24 UTC,, it is not possible to perform a daily error analysis. Instead, at each grid point, the precipitation is accumulated in different time periods: 5, 8, 15 and 20-days, monthly, seasonally and annually. Subsequently, the following point error statistics are computed: the correlation coefficient, the bias, the normalized bias (ratio of bias to mean observed precipitation), the mean absolute error (MAE), the mean absolute percentage error (MAPE), and the root mean square error (RMSE). Additionally, an overall measure of variability simulated by each model is computed as the ratio of its standard deviation, in both space and time, to the corresponding standard deviation of the ObsIM_0.2 grid. Furthermore, since one of the main applications of RCM precipitation results is to generate input to hydrological models and assist in water resources management, point error statistics are also computed for river basin precipitation (Figure 1). The basins presented in Figure 1 are a simplification of the intricate Portuguese river basin system, including the three major Iberian rivers (Douro, Tejo and Guadiana), and three areas (NW, Estremadura and SW) aggregating smaller basins. The approximate basin map roughly represents the large northsouth and west-east observed precipitation gradients. Basin statistics allow a fairer assessment of the impact of the large scale forcing, coming from ERA-40, which cannot resolve finer scale details, contributing to the robustness of spatial analysis [Caldwell et al., 2009]. 3. Results 3.1. General Evaluation of Model Results [16] Figure 2 shows the geographical distribution of the Portuguese annual precipitation, from observations and ENSEMBLES RCMs. The southeast-northwest precipitation gradient and the orography signal are immediately recognized in the observations and in all RCMs. In some models, e.g., ICTP and KNMI, the annual precipitation map is 3 of 18

4 Figure 2. Annual precipitation climatology ( ) from gridded observations, ERA40, and ENSEMBLES RCMs (interpolated to the ObsIM_0.2 grid). remarkably similar to observations. However, most of the models show a slightly more heterogeneous spatial pattern than the observational grids, with overestimation of the NW and underestimation of SE annual precipitation by ETHZ, HadRM3xx, MPI, SMHI, UCLM, and more notably METNO. DMI shows a severe underestimation throughout most of the territory, and CNRM a smoother gradient with overprediction in the south and underprediction in the north. In the mountainous areas it is likely that the observational grid underestimates real precipitation, as it was not corrected for topographic undercatch [Adam et al., 2006] and may suffer from undersampling [Frei et al., 2003]. [17] An objective analysis of the performance of the RCMs is presented in Figure 3, showing the statistical errors for the different models against the 0.2 resolution gridded data set, for different accumulation periods, from 5 days to yearly, and pooling together all the grid points. All statistics shown in Figure 3 compare an observation grid box against an interpolated model result, therefore by comparing all grid boxes at the same time they incorporate both time and space variability. The models by CNRM, DMI and METNO present the lowest yearly correlation coefficient, in the range , and the highest correlation is shown by the ETHZ (r = 0.88) and KNMI (r = 0.89) models. This correlation ranking is very much in agreement with Herrera et al. [2010] for the Spanish territory. The normalized bias shows that RCMs mostly underestimate the Portuguese annual precipitation, by less than 20%, but DMI underpredicts it by more than 40%, MPI has no bias and ICTP indicates about 5% of overestimation. For Spain, Herrera et al. [2010] also found an overall good agreement between gridded observations and RCMs, but with an overestimation in most areas by most models. Likewise Rauscher et al. [2010] found an overprediction of precipitation in Europe by the ENSEMBLES RCMs, with a domain average bias for the ensemble mean around 20% in winter and 10% in summer. [18] Annual MAPEs of the different RCMs are situated in a wide range from 18% (ICTP) to 46% (DMI). However, most of the models show values around 20 25%, which supports the idea of an overall good description of the Portuguese annual precipitation. RMSE (Figure 3f) gives an extra weight to the individual errors, giving the same kind but more emphatic inter-model spread than MAPE. [19] An indicator of the model ability to reproduce the interannual space and time variability is the yearly normalized standard deviation presented in Figure 3b, computed as the ratio between model and observed yearly standard deviations. This indicator ranks the models skills in a way that is not easily anticipated from the global error statistics. RCMs showing a best interannual variability are ETHZ, ICTP, METNO and SMHI. The KNMI model, one of the best in most annual indicators, underestimates interannual variability. The METNO model, on the other hand, shows a good representation of interannual variability which is not supported by the error statistics, namely MAPE and RMSE. [20] Figure 3 also shows error statistics for a set of different accumulation intervals (seasonal, monthly, 15 days and 5 days) allowing a rather detailed analysis of the representation of intra-annual variability. There is a systematic gain in correlation (Figure 3a) from annual to monthly and then to seasonal, indicating a good description of the annual cycle. The shortest accumulation period (5 days) leads to lower correlation values in most models (with exception of METNO and SMHI), with the largest model spread ( ), but still with correlations that can be as high as 0.85 (KNMI) indicating a good representation of the synoptic variability. MAPEs (Figure 3d) reveal a systematic increase with diminishing accumulation period, and model ranking depends on accumulation period. Seasonal MAPEs are in the range of 24 49% (KNMI-DMI), 5-day MAPEs are between 48% (KNMI) and 79% (CNRM). RMSEs (Figures 3e and 3f), as expected, vary in the same direction as the accumulation period. The model ranking is maintained in the largest accumulation periods (yearly to monthly), but the range of 5-day model RMSE is relatively narrow, of the order of 5 mm (for a multimodel mean around 15 mm). The ICTP model, for instance, loses relative performance at the 5-day scale, while ETHZ and SMHI gain. [21] Models have rather different behavior in the representation of intra-annual model variability (Figure 3b). ETHZ, METNO, MPI behave better in all intra-annual periods, when compared to yearly data, CNRM, ICTP, and SMHI perform better for interannual variability, the other models present roughly similar skills. Many models have a 4of18

5 Figure 3 5of18

6 Figure 4. Model to observations (ObsIM_0.2) interannual correlation of seasonally mean grid-box precipitation. DJF for winter, MAM for spring, JJA for summer and SON for autumn. Correlation computed pooling all seasonal mean data together. normalized standard deviation between 0.90 and 1.10, a very good match with observations. CNRM, DMI and KNMI significantly underestimate both inter and intra-annual variability, by up to 20% in the case of CNRM interannual (Figure 2). [22] In spite of its underestimation of both interannual and intra-annual variability, the KNMI model shows a very good performance in the error statistics of Figure 3 independently of the accumulation time. The KNMI model may benefit from the use of a wider boundary relaxation region for the wind when compared to other RCMs. Lenderink et al. [2003] suggested that this feature allows maintaining the RCM atmospheric circulation closer to the driving model. CNRM, ICTP and UCLM, and, to a lesser extent, DMI and METNO, present the larger dependency of the accumulation period, loosing up to 15% of correlation at 5-days, indicating that these models may have problems in representing the phase of the frontal systems traversing Portugal, when compared to its monthly or seasonal performance. These deficiencies look more drastic in the MAPE and RMSE data for CNRM, DMI and UCLM models. [23] Model interannual variability is highly dependent on season, a feature that is relevant in the Mediterranean climate. Figure 4 presents interannual correlations of model to observations, aggregated by season. All models reveal significantly worst performance in Summer, and most models have Spring as a (distant) second worst. Best results are obtained by KNMI (in the range 0.76 in Summer to 0.93 in Winter). Largest spread is found in ICTP (0.47 to 0.90), UCLM, and CNRM, hinting of possible deficiencies, may be in the representation of mesoscale circulations or in soilatmosphere interactions. [24] In summary, ENSEMBLES models offer a good representation of Portuguese precipitation, although with a tendency to underestimate its mean values, and a significant spread in the different parameters Error Spatial Distribution [25] In order to understand the regions where models have bigger problems in reproducing the observed precipitation patterns, Figures 5, 6 and 7 present maps of grid point correlation coefficients, normalized bias and MAPE, for the 40 years period. The grid point yearly correlation coefficient (not shown) varies from over in the driest areas (near the eastern border) to over 0.9 in the western NW (in some models). The seasonal correlation (Figure 5a) is good in most models, above 0.8 in all areas for ETHZ, HadRM3xx, KNMI, METNO, and SMHI. Lower values are generally found in the driest spots, most notably in the NE, where for example CNRM goes down to 0.5, and the SE where some models go down to 0.7. The 5-days correlation is, of course, lower, but with the best models (e.g., ETHZ, KNMI) still attaining 0.9 in the NW. Two models (KNMI and SMHI) are above 0.8 in most of the territory, with only small areas in the NE and SE with slightly lower values (above 0.7). [26] It was shown in Figure 3 that most models presented a mean negative bias, with the exception of ICTP, slightly wet, and MPI, exactly on spot. However, most models show significant spatial heterogeneity in its grid point normalized bias distribution (Figure 6). Normalized bias is by definition independent of accumulation period. All models have positive bias in a few grid points corresponding to the highest NW mountain peaks, in some cases simulating more than 150% of observed precipitation. However, it is important to keep in mind that this may be partially explained by known difficulties of gridded data sets, such as ObsIM_0.2Grid, to accurately represent the precipitation in complex and high mountains. Other localized wet bias patterns also appear to be associated with other mountain chains, namely in the HadRM3xx, METNO, and SMHI models. The DMI model is very dry throughout the territory, except in the NW wet spot. The CNRM model is very dry along the coast and very wet inland, seeming to produce the precipitation too late when the frontal systems advance through Portugal, a behavior that may be related to the proximity of its western lateral limit. Interestingly, the zero normalized global bias of the MPI model results from cancellation between a very wet littoral area and a very dry inland region. In what concerns bias, KNMI shows a quite homogeneous negative bias pattern, like do the HadRMxx models, the latter with a more emphatic orographic signature. The METNO model presents the strongest topographic wet bias. Finally, the ICTP model, which on average has wettest results, shows precipitation overestimation at the coast and inland. [27] The distributions of the seasonal MAPE (Figure 7a) are quite homogeneous in the best performing models, with grid point values below 30% in most of the territory in KNMI and SMHI, with the local exception of the NW wet spot. Some models show an enlargement of MAPE from Figure 3. Global error measures of the ENSEMBLES RCMs precipitation against the observational grid (ObsIM_0.2) for the Portuguese mainland. The error measures are (a) correlation coefficients (correlation), (b) normalized standard deviation, (c) normalized bias (Bias%), (d) mean absolute percentage error (MAPE), and (e and f) root mean square error (RMSE). The presented errors are computed for different accumulation periods of precipitation (5 days, 15 days, monthly, seasonally and yearly), pooling all data together. 6of18

7 Figure 5. Grid-box correlation coefficients of ENSEMBLES RCMs against observations, for the ERA period and the aggregation intervals: (a) seasonal and (b) 5-days. NW to SE. Not accounting the NW wet spot, the larger grid point seasonal MAPEs are presented by DMI in the southwest (above 70%). At the 5-days scale, MAPEs are much larger, maybe with a stronger west-east signature (Figure 7b). All models show the majority of the territory with MAPEs above 40%, and above 50% inland. Significant interior areas are covered with MAPEs above 90% in CNRM model, in the case of DMI a clear north-south gradient is present, with MAPEs around 80% in the south, and, finally, UCLM shows MAPEs above 70% over more than half of Portugal. The poorest 5-days grid point MAPEs occur for models which showed worst global 5-days correlations and MAPEs (Figure 3). [28] It looks from this analysis of error geographical distribution that the best performing models lose precipitation prediction skill in the transition from west to east, not representing very well the inland evolution of the synoptic scale precipitation patterns or local processes modifying the precipitation field Basin Analysis [29] The ability of RCMs to reproduce basin precipitation is crucial to assess how RCMs may be used to assist, for example, on water management. Monthly basin precipitation averages, observed and simulated, for the main Portuguese basins, can be seen in Figure 8. Basin observational data represents in a simplified way the north-south and east-west gradients, the two precipitation maximums, the absolute one in winter, and a local secondary one in April, consistent with a low rainfall persistently recorded in March in recent decades. Paredes et al. [2006] suggested that this rainfall anomaly is associated to a more northern mean trajectory of the Atlantic fronts in March, related with a persistent positive anomaly of the March North Atlantic Oscillation index in the later decades of the 20th century. Finally, in all basins the minimum precipitation rate is recorded in July and August. These features are in general captured by Figure 6. Spatial distribution of normalized bias (%) between ENSEMBLES RCMs and the observational grid (ObsIM_0.2), for the 40 years period. 7of18

8 Figure 7. Same as Figure 6 but for the mean absolute percentage errors (MAPE, %) for (a) seasonal and (b) 5-days. the RCMs, but with a large spread. The NW basin (Figure 8) shows the largest precipitation rate, with most of the models overestimating precipitation throughout the seasonal cycle; KNMI and SMHI models perform very well in this basin. Overall, for all the basins, the DMI model is too dry, and CNRM has a tendency to largely overvalue the late spring precipitation. As detected in Spanish basins by Herrera et al. [2010], CNRM and DMI are the main contributors to the late spring model spread in most regions, that jointly with winter presents the larger source of inter-model basin spread. In winter most models underestimate precipitation in all basins, except in the NW. If CNRM and DMI were withdrawn, the overall agreement is notable, and the late spring spread is strongly reduced. CNRM and DMI seem to have special difficulties in representing the precipitation associated to the last cold air fronts and the beginning of the convective activity [Herrera et al., 2010]. [30] To add insight to the evaluation of model basin precipitation, correlations and MAPEs of the mean basin precipitation are shown in Figure 9. The remarkable general good agreement previously stated is easily identified in Figure 9a, where the majority of RCMs show correlation coefficients around or above 0.90 for 15 or more days of accumulation time. Noteworthy is the performance of KNMI, SMHI and ETHZ with high correlations even for the 5-days accumulation period, revealing a very good ability to match the frontal activity in each basin. On the other end, CNRM, DMI, and UCLM show, in diverse ways, a poorer performance. On a yearly scale SMHI has a real small basin MAPE, near 10%. KNMI shows the best performance on the shortest time scales. DMI presents little sensitivity to the aggregation interval, pointing that errors in DMI are mostly related with its strong dry bias and not as much with precipitation synchronization or location within a given basin. [31] A more detailed understanding of the skills of RCMs in reproducing the precipitation regimes, is given by the distribution of the daily precipitation quantiles for wet days (>0.1 mm), in a logarithm scale. Figure 10 shows the precipitation quantiles in the two extreme basins, the wettest (NW, Figures 10a and 10c) and driest (Guadiana, Figures 10b and 10d), pooling together all the basin grid points in Figures 10a and 10b, and the basin average precipitation in Figures 10c and 10d. It is clear, at a first glance, that the different RCMs are able to reproduce the two basins quantile distributions, in both grid point and basin average views. In the NW basin, the vast majority of models underestimate the observed quantiles bellow quantile 90, with a tendency to overestimation of the higher ranking quantiles. There is a large spread among RCMs, namely in the wettest quantiles (note the logarithmic scale), in agreement with Kjellström et al. [2010] for Europe, although is also occurs in a relative sense for all quantiles. However, ICTP shows a good representation of all quantiles, except for the highest ranking ones, where it reveals overprediction. In the driest basin (Guadiana) there is an overall underestimation in most RCMs, except in the most extreme quantile (99.9, strongest precipitation events). The Guadiana basin behavior was also found by Herrera et al. [2010] for the majority of the Spanish basins. Here, the ICTP model seems to perform slightly better in the lowest ranking quantiles. [32] The basin averaged precipitation quantiles (Figure 10b) still show a large spread between models, increasing with the ranking of quantiles. As expected, RCMs show an improvement in the representation of the observed quantiles, 8of18

9 Figure 8. Monthly mean precipitation for each basin, from ENSEMBLES RCMs and the observational grid (ObsIM_0.2). The basins are illustrated in Figure 1. when compared to top row values. In the NW, most of models still present overestimation, with underestimation only in the quantiles between 40 and 90. RCMs in Guadiana also reveal an improvement, but still with underestimation in the majority of the quantiles. These results point to RCMs fragilities in the reproduction of precipitation regimes, more obviously at the intrabasin scale. It is important to state that RCMs were run at a slightly coarser resolution (25 km) that the observational grid; assuming that model grid precipitation represents averaged grid box precipitation, this can give rise to less intense simulated daily precipitation [Osborn and Hulme, 1998]. On the other hand, Osborn and Hulme [1997] refer to errors introduced by interpolation methods in the generation of regular observational grids and their possible relevance in orographic and coastal regions. In those places, precipitation at neighbor weather stations can have very low cross-correlation, but are used for averaging for each grid box, leading to less representative gridded observational data sets. To assess the 9of18

10 Figure 9. (a) Correlation of the mean basin precipitation, and (b) MAPE, of the ENSEMBLES RCMs precipitation against the observational grid (ObsIM_0.2). The errors are computed for different accumulation periods (5 days, 15 days, monthly, seasonally and yearly). relative importance of the two contradictory grid effects, coming from model resolution limitations and from spatial averaging in observational grids, one would need to look at specific station data in very high-resolution networks, a generally difficult task, outside the scope of the present study. [33] When aggregated at the basin level, precipitation statistics are considerably improved. Those improvements are relevant in the representation of moderate to intense precipitation regimes, indicating that ENSEMBLES data is more reliable for full basin hydrological purposes Extreme Precipitation [34] Several studies, based on either GCMs or RCMs, have suggested recent increases in the frequency and severity of precipitation extremes [Zwiers and Kharin, 1998; Intergovernmental Panel on Climate Change, 2007] implying the need of a better understanding and characterization of extreme weather statistics, as of key importance for climate change assessments. Here, the ability of RCMs to reproduce extreme precipitation features is evaluated through the analysis of the distribution of wet days, high precipitation percentiles, and standard climate indices [Sillmann and Roeckner, 2008]. Most of these results can be directly compared with the study of Herrera et al. [2010] for Spain. [35] Wet days, may be defined as days with daily precipitation above 0.1 mm (including very light precipitation) or above 1 mm (including light precipitation). The relative difference of wet days, between RCMs and observations, for the 6 river basins and whole Portugal (Figures 11a and 11b) shows that most models overestimate the frequency of very light precipitation and underestimate the frequency of light precipitation, implying that models rain too often and too little, in agreement with findings of Kjellström et al. [2010]. The dryness of the DMI model, already mentioned, is associated with rather low precipitation frequency, while the CNRM and ICTP wettest character is related to large values of wet-days frequency. These views are basin focused, but for the whole Portuguese territory, models dramatically overestimate the wet-days above 0.1 mm and are mostly good in describing the wet-days above 1 mm, a behavior that is largely determined by the relative importance of the NW basins precipitation in the Portuguese context. [36] As before, the precipitation percentiles are computed for wet days, with a minimum precipitation of 0.1 mm. Percentiles corresponding to extreme precipitation events were computed for each basin pooling together all the correspondent grid points. Figures 11c, 11d, and 11e present the percentiles 95, 99 and 99.9, respectively, representing very wet days, extremely wet days and the strongest precipitation events. All three percentiles show a larger spread in the NW and Estremadura basins, which, along with the Douro basin, show the highest percentile values. The two former basins are those first facing the Atlantic fronts that lead to most of the extreme precipitation events. This shows how differently the models simulate inland evolution of the fronts and the orographic enhancement processes, in spite of sharing the same driving forcing (ERA-40) and the same resolution; although with significant differences in important numerical details of Global to RCM model coupling, e.g., domain size and location and boundary relaxation method. Percentile 95 is mostly underestimated by RCMs with the exception of the NW. The Guadiana, SW & Algarve and Tejo basins show the largest underprediction of this percentile. Extremely wet days (percentile 99) are slightly better reproduced by the ensemble, 10 of 18

11 Figure 10. Quantiles of ENSEMBLES RCM daily precipitation in wet days (>0.1 mm) for the (a and c) wettest (NW) and (b and d) driest (Guadiana) basins. The scatters correspond to RCMs quantiles (2.5, 10, 20, 25, 30, 40, 50, 60, 70, 75, 80, 90, 95, 97.5, 99 and 99.9). Grid box daily precipitation is pooled together for each basin (Figures 10a and 10b), and basin mean daily precipitation (Figures 10c and 10d). Axes are different for improved plot legibility. in particular in the Douro basin. ICTP appears to reasonably represent both percentiles, when the basin differences are taken into account. Basin distribution of the strongest precipitation events (percentile 99.9) reveals overestimation by most models, except in the drier southern basins. [37] The geographical distribution of yearly mean climate indices is shown in Figures 12 and 13. These indices are all computed from daily precipitation data, for each year and grid point, observed and simulated, and include the computation of yearly means. Figure 12 represents frequency of wet days above 1 mm (wd), number of moderately wet days (rainfall above the percentile 75, r75p), maximum number of consecutive days without precipitation (dd), and maximum number of consecutive days with precipitation (cwd). Figure 13 represents number of days with a maximum precipitation over 10 mm (r10), maximum daily precipitation (rx1day), maximum precipitation in 5 days (rx5day), and percentage of precipitation above 95th percentile events (%r 95p). Obviously, ERA-40 is very poor in the geographical characterization of the climate indicators referred; nevertheless its results are also shown in Figures 12 and 13, along with those of ObsEns_0.25 grid. The spatial correlation coefficients between climate indices from RCMs and ObsIM_0.2 grid were also computed, and are shown in Figure 14. [38] The observed spatial distribution of wet-days (wd, with precipitation above 1 mm) presents the southeast-northwest gradient, with values spanning from around 15% to above 35%, with little orographic signature. RCMs describe very well the spatial distribution, with spatial correlations in the range , most notably in the cases of ETHZ and KNMI, less well in the cases of CNRM and DMI. Moderate wet days, i.e., the wet days above the 75 percentile, is 11 of 18

12 Figure 11. Percentage of model relative to observation of: days with precipitation (a) above 0.1 mm, and (b) above 1 mm. Wet days grid box daily precipitation percentiles: (c) 95 (d) 99, and (e) All daily percentiles are in mm. Results are presented in the 6 basins for all panels, but Figures 11a and 11b include also all Portugal statistics. Observations are in red. in the range of days per year. RCMs reproduce this index quite well, with a spatial correlations of 0.77 (CNRM and DMI) to 0.89 (ETHZ and KNMI). [39] Consecutive dry days vary greatly throughout Portugal, from 90 days in the SW to less than 30 days in the NW. RCMs show a reasonable depiction of the geographical pattern of consecutive dry days, especially ETHZ. Spatial correlations of consecutive dry days span from 0.81 (HadRM3Q16) to 0.97 (ETHZ). The RCM patterns of consecutive wet days are poorer. In a small area in the south and extreme east, rainfall occurs up to 8 days in a row, whereas in the northwest it lasts up to 20 days. RCMs results vary significantly, however the number of consecutive wet days is generally underpredicted. ICTP and KNMI present the largest spatial correlations (0.84) while DMI and MPI are not better than DMI strongly underestimates the number of consecutive 12 of 18

13 Figure 12. Geographical distribution of yearly mean values of (a) frequency of wet days, with daily rainfall above 1 mm (wd), (b) moderate wet days with precipitation above the percentile 75 (r75p) (computed for wet days), (c) maximum number of consecutive days without precipitation (cdd), and (d) maximum number of consecutive days with precipitation (cwd). 13 of 18

14 Figure 13. Same as Figure 12 but for (a) number of days with a maximum precipitation over 10 mm (r10), (b) maximum daily precipitation (rx1day), (c) maximum of precipitation during 5 days (rx5day), and (d) percentage of precipitation above the 95th percentile events (%r95p). 14 of 18

15 Figure 14. Spatial correlations with ObsIM_0.2Grid (cdd - number of consecutive dry days; freq - frequency of wet days, with daily rainfall above 1 mm; wd - number of wet days (r > 1 mm); cwd - number of consecutive wet days; rx1day - maximum daily precipitation; rx5day - maximum precipitation in five days; r10 - number of days with precipitation above 10 mm; d > r75p - number of days with precipitation above 75th percentile; r95p - 95th percentile; %r95p - percentage of precipitation above 95th percentile; and temporal correlations for annual, seasonal, monthly and five day accumulation periods. wet days. On the contrary, Herrera et al. [2010], found for Spain a tendency for overestimation of rainfall frequency (>1 mm), leading to overprediction of consecutive wet days and underprediction of consecutive dry days. [40] The number of days with precipitation above 10 mm has a very similar pattern to the annual mean precipitation climatology (Figure 13a). In the south and northeast around 15 days per year can be found, and in a small northwest area more than 60 days. RCMs show a reasonable agreement, but with a tendency to underestimate in the northeast and south, and some cases even in the NW. The spatial correlations are in the range 0.77 (METNO) and 0.94 (ICTP and KNMI). The maximum daily precipitation (Figure 13b) presents yearly mean values around 40 mm (range mm). In general, RCMs show more heterogeneous patterns, and overall good spatial correlations; in agreement with the special intense precipitation gradients, already referred, the METNO model has the smallest spatial correlation (0.59), and ETHZ model has the highest one (0.85). RCMs capability to reproduce the maximum of precipitation during 5 days is improved in most models, compared to previous index, and the spatial correlations ranges between 0.77 (METNO) and 0.89 (KNMI). This improvement varies greatly from model to model. Overall, RCMs reveal a better performance in representing these last three climate indices than in Spain (compare with work of Herrera et al. [2010]). [41] The geographical distribution of percentile 95 (not shown) also roughly recalls the annual mean precipitation, and RCMs show a reasonable ability to reproduce this pattern, with spatial correlations in the range of 0.64 (METNO) to 0.83 (DMI and KNMI). Percentage of precipitation above the 95th percentile shows a quite homogeneous observational pattern: in the north small areas have values of 16% (of total annual precipitation) rising to 25% in the SE. Partially due to this homogeneity; RCMs show the poorest description of this climate indicator within the studied indices. In fact, the HadRMxx models are even anti-correlated ( 0.10 to 0.04), and the better correlated models only reach values of 0.48 (ICTP) and 0.49 (MPI). One of the models that in general have shown to best perform, the KNMI model, only shows a spatial correlation of The models seem to be too dependent of topography, a feature that appears less relevant for the current observation index. [42] It is worth noting that the new grid, ObsIM_0.2, displays improvements relative to ObsENS_0.25, with a better spatial and temporal distribution of precipitation which leads to a more accurate representation of the presented indices. To summarize, Figure 14 shows an overview of some RCM results, consisting of spatial correlations of climate indices and the temporal correlations described in section 3.1. [43] In summary, the extreme precipitation in the southern, and driest, basins is generally underestimated by ENSEMBLES models. However, most results climate indices are well represented, with spatial correlations above 0.75, but the spatial distribution of the amount of precipitation associated with the wettest days (%r95p) is very poor. 4. Discussion and Conclusions [44] RCM models forced by reanalysis data offer an excellent way to assess RCM performance, not only in what concerns its ability to represent observed climate variability, in its 15 of 18

16 temporal and spatial domains, but also in the description of real world weather systems, including those responsible for extreme weather events. Whereas in control runs, where RCMs are forced by GCMs, it is in general impossible to discriminate between errors related with the boundary conditions and those produced by RCM model dynamics and physics, here all models were forced by the same driving reanalysis. Furthermore, due to the proximity to the western boundary, from where most weather systems arrive to Portugal, it is also possible to compute direct point error statistics, comparing model predictions with synchronized observations aggregated at different time scales, even without nudging from large scale data. The present study aimed to evaluate the ENSEMBLES RCMs precipitation, mean and variability, in mainland Portugal, using of a new highresolution gridded observational data set developed by Belo- Pereira et al. [2011] interannual. The study complements recent work by Herrera et al. [2010], for the contiguous territory of Spain, although, interestingly, some results are not only quantitatively but also qualitatively different. [45] As also found by Rauscher et al. [2010] and Herrera et al. [2010], ENSEMBLES RCMs do an overall good representation of observed precipitation, in what concerns the main spatial patterns of annual precipitation. However, unlike what was found for both Europe and Spain, 10 of the 12 analyzed models underpredict Portuguese precipitation. From a point a view of water mass conservation, this is consistent with overprediction over Spain, as the same weather systems precipitate, in general, in the two territories. [46] Variability of precipitation is also rather well captured by most models with good correlations with synchronized gridded observations at the yearly scale, with smaller but still good values at the 5-days scale. These results are probably attributable to ERA-40 boundary conditions, since the Portuguese area is at the west limit of the ENSEMBLES domain, but they still indicate the ability of the RCMs to consistently adjust to the imposed forcing, which is not a small achievement. However, most models do a worse simulation of the presumably smaller scale summer interannual variability and perform better in winter. On the other hand, half of the models underrepresent observed variability of daily precipitation, as given by the corresponding normalized standard deviations. [47] The analysis of the performance of RCMs in the representation of extreme precipitation was done both at grid box level and for 6 basins of comparable sizes, roughly describing the main climate sub-regions. When using a threshold of 1 mm/day to define a wet day, models were found to underestimate the number of wet days, a result unlike what was found for Spain by Herrera et al. [2010]. However, if the threshold is set at 0.1 mm/day, the conclusion is the opposite: models generally generate rain too often but not enough of it. This supports the idea that the definition of wet days is critical, and may need a model based statistical adjustment, as suggested by Schmidli et al. [2006] and supported by Herrera et al. [2010]. Grid point percentiles of precipitation are generally underpredicted, with the exception of higher ranking percentiles (above 90%) in the wettest basin, and of the 99.9 percentile in all basins. However, basin aggregated data shows some improvement, especially in the lower to mid ranking percentiles. In all cases, there is a significant model spread. [48] The representation of high ranking percentiles, associated with less frequent and more intense precipitation, was found to be reasonably good in the wettest NW region but with a tendency for underprediction of intense precipitation in the south. The 95th percentile is underpredicted by all models in most of the country, with exception of the NW. Percentile 99 is underpredicted by most models in the 3 driest basins. However, percentile 99.9 is generally overpredicted. When looking at grid point climate indices, spatial correlations against observations were found to be in general good, above 0.7, with the exception of the amount of precipitation due to intense rain (above percentile 95) whose spatial distribution is poorly represented by all models. [49] As expected, there is an important model spread in all analyzed variables, and the performance of most models varies with location. Most models do a very good simulation of the annual cycle of basin precipitation, being able to describe the double peak first in winter and then in April, due to reduced precipitation in March. Three of the models do a much worse job, suggesting the benefits of a reduced ensemble, as proposed by Herrera et al. [2010] from a slightly different set of ENSEMBLES models. However, not surprisingly considering the different sign of the bias, the set of best performing models does not coincide. KNMI s RACMO model stands out as the model with an overall better performance in this region, although at the cost of reduced variability. Some models do an overall good job but partially as a result of cancellation of symmetric errors in different locations, namely in rugged terrain. The need for ensembles of models is clearly justified by the present study. [50] RCMs used in this study constitute a state-of-the-art set of numerical models, especially organized by the EUfunded project ENSEMBLES to assess climate change at the European level. However, the assessment of its results is highly dependent on the quality of the observational grid, pointing to limitations in the preliminary Europeanscale model evaluations done with the low number of weather stations used to produce the ECAD 0.25 grid. Some errors found in higher terrain, namely in the NW wet spot, may be due to limitations in the representation of localized precipitation in mountain environments by gridded data sets, as found in the European Alps [Frei et al., 2003; van Meijgaard et al., 2008], suggesting a need for further studies. Furthermore, the present study used a set of similar resolution RCMs, all at around 25 km. Rauscher et al. [2010] concluded for the superiority of the 25 km ENSEMBLES set against the 50 km set with the same models. However, recent studies [Caldwell et al., 2009; Zhang et al., 2009; Heikkilä et al., 2010] are suggesting the need for even higher horizontal resolutions, to improve model performance in coastal and mountainous terrains. Higher horizontal resolutions are likely to improve both the simulation of extreme weather events and summer circulations. [51] ENSEMBLES models have established a standard for regional climate assessment, emphasizing the need for ensembles of simulations, to cope with uncertainty, and of higher horizontal resolutions. The present results suggest that there is still some way to go in this research. While multimodel ensembles are a requirement for these studies, an objective selection of the participant models, and their individual improvement in time, are crucial, since some models 16 of 18