EVALUATION OF THE EUROPEAN DAILY PRECIPITATION CHARACTERISTICS FROM THE ATMOSPHERIC MODEL INTERCOMPARISON PROJECT

Transcription

1 INTERNATIONAL JOURNAL OF CLIMATOLOGY Int. J. Climatol. 18: (1998) EVALUATION OF THE EUROPEAN DAILY PRECIPITATION CHARACTERISTICS FROM THE ATMOSPHERIC MODEL INTERCOMPARISON PROJECT TIMOTHY J. OSBORN* and MIKE HULME Climatic Research Unit, Uniersity of East Anglia, Norwich NR4 7TJ, UK Receied 20 December 1996 Reised 27 October 1997 Accepted 28 October 1997 ABSTRACT Evaluating the daily precipitation performance of general circulation models at a regional scale is beset with two (related) difficulties: (i) ensuring an adequate network of observed daily station data across a region; and (ii) establishing true areal means of daily precipitation from these station data to ensure a like-with-like comparison. These two difficulties have generally restricted past evaluations of daily model precipitation to individual grid boxes and to comparisons of model grid-box data with observed sub-grid-box scale data. In this paper we exploit a dataset of several dozen daily station time series in western Europe using a recently published method for estimating true areal mean daily precipitation to evaluate the daily precipitation performance in this region of 12 atmospheric general circulation model simulations undertaken as part of the Atmospheric Model Intercomparison Project. We examine four quantities derived from the simulated daily precipitation: seasonal mean precipitation, daily standard deviation, the frequency of raindays and the mean precipitation intensity of those raindays. We show that simulated winter precipitation tends to be too great, falls on too many days, but is generally less intense than that observed. In summer, model results are more variable with some model simulations yielding statistics quite similar to those observed. Although there is some uncertainty in these evaluation statistics owing to the effect of inter-decadal climate variability, there is a weak suggestion of improved performance with increased spatial resolution. Any systematic effect in model performance due to different convective parameterisation schemes is obscured by other confounding factors Royal Meteorological Society. KEY WORDS: Europe; daily precipitation; spatial aggregation; climate models; model evaluation; AMIP 1. INTRODUCTION Some recent climate modelling studies of the enhanced greenhouse effect show more (or similar) mean precipitation in the future, but falling on fewer raindays (Gordon et al., 1992; Whetton et al., 1993; Fowler and Hennessy, 1995; Gregory and Mitchell, 1995; Hennessy et al., 1997). This implies an increase in the frequency of heavy precipitation amounts (see also Noda and Tokioka, 1989), that may turn out to be one of the most important climate changes. Similar results have been obtained using high resolution regional climate models (Mearns et al., 1995). Whilst there is some observational evidence to support such a change (Karl et al., 1995; Suppiah and Hennessy, 1996), some doubt must remain over these model results because previous work has shown such models to have systematic errors in their simulation of rainday frequencies (Reed, 1986; Wilson and Mitchell, 1987; Rind et al., 1989; Gregory and Mitchell, 1995; Mearns et al., 1995). These previous evaluation studies of modelled rainday frequencies have, however, two main deficiencies. First, the paucity of timeseries of observed precipitation data on a daily timescale, available to the * Correspondence to: CRU, University of East Anglia, Norwich, NR4 7TJ, UK. t.osborn@uea.ac.uk Contract grant sponsor: UK Department of Environment, Transport and the Regions; Contract grant number: DETR EPG 1/1/48 Contract grant sponsor: US Department of Energy; Contract grant number: DE-FG02-86ER60397 CCC /98/ $ Royal Meteorological Society

2 506 T.J. OSBORN AND M. HULME scientific community, often prevents the construction of true grid-box mean (i.e. equivalent to an infinitely sampled area) timeseries for comparison with the simulations. Second, such evaluation has been confined to single or only a few grid boxes. One of the main reasons for both of these deficiencies is the lack of continental-scale quality-controlled datasets of observed daily precipitation containing many hundreds if not thousands of stations. Although there are exceptions to this complaint (e.g. the USA), in general it is either impossible (e.g. the tropics) or too expensive (e.g. Europe; see Hulme, 1994) for datasets containing a sufficient density of daily series to be compiled. The situation is eased for high resolution models, where the grid-box mean precipitation corresponds more closely to single-site precipitation (Mearns et al., 1995). We now improve upon these earlier studies by addressing these two deficiencies. Using the method developed by Osborn and Hulme (1997), it is possible to obtain a better estimate of statistical properties of true grid-box-mean precipitation from a small, finite number of stations. Their method removes the effect of varying gauge densities on the estimates obtained the bias towards fewer raindays and greater variability that a single station has compared to an areal mean. It requires, however, that the available stations provide an unbiased estimate of station statistics applicable to the grid box as a whole. This method is then applied to the evaluation of 12 atmosphere general circulation model (AGCM) simulations performed for the Atmospheric Model Intercomparison Project (AMIP, Gates, 1992). The validation is of the models seasonal mean precipitation fields over a greater European region, and of the models daily precipitation temporal standard deviation, rainday frequency and mean precipitation intensity over part of western Europe. Implicit in our approach is that general circulation models simulate areally-averaged precipitation rather than point precipitation. Given that model processes and surface boundary fluxes are parameterised over areas rather than points, this seems to us to be a reasonable assertion. There is an alternative view, however, which allows modelled precipitation data to be seen as representing point precipitation (Skelly and Henderson-Sellers, 1996). How modelled data are regarded will alter the way the model is validated: can modelled precipitation be directly compared with observed station timeseries or do such station timeseries need averaging to produce an areal mean? This paper does not examine these arguments in detail; we merely note that there are alternative views and treat modelled precipitation as an areal quantity. The paper is structured as follows. In Section 2 we describe the observed and modelled daily precipitation data used in our analysis. Sections 3 6 present an overview of the method used (see Osborn and Hulme, 1997, for full details) and the results obtained for the evaluation of seasonal mean precipitation, daily precipitation standard deviation, rainday frequency and mean precipitation intensity, respectively. Section 7 then considers the effect of forced and unforced interdecadal variability on our results. We discuss the implications of our evaluation results in Section OBSERVED AND MODEL DATA Twelve model simulations of daily precipitation are evaluated here. These are simulations by the AGCMs listed in Table I, performed for the AMIP exercise: all are 10-year simulations, forced by a realistic sequence of prescribed monthly sea surface temperatures for the period The first year of each simulation was discarded because some of the models used in the AMIP exercise exhibited initialisation artefacts. Nine years of simulation remained for evaluation, and the statistics of the daily model precipitation were calculated for each grid box that was classified as land in the model, on a seasonal basis. See Phillips (1994) for detailed documentation on all the models used. The European baseline climatology constructed by Hulme et al. (1995) is used to provide estimates of mean precipitation and single-station rainday frequencies. Although this is a gridded climatology, the rainday frequencies are still representative of point statistics (a surface was fitted to the available station rainday frequencies and then sampled on a regular 0.5 by 0.5 grid see Hulme et al., 1995, for further details). The climatology provides more accurate values than have previously been available, due to the

3 EUROPEAN DAILY PRECIPITATION 507 number of stations used and because elevation has been accounted for (the values are estimated at the mean elevation of each grid-cell). In addition, a station network of daily precipitation data covering part of western Europe has been used to compute daily standard deviation and two parameters describing the spatial characteristics of precipitation events (see Sections 4 and 5). The western European dataset is a combination of 185 daily precipitation timeseries from several western European countries Belgium, France, Germany, Ireland, Italy, Netherlands, Switzerland and the UK containing data mostly from 1961 to This network is illustrated in Figure 1, together with the grid box layout of the UKMO AGCM (as an example) and the number of stations per grid box (including non-land grid boxes). 3. EVALUATION OF SEASONAL MEAN PRECIPITATION The validation dataset of seasonal mean precipitation is constructed from the European baseline climatology, covering the region 32 W 66 E and 25 N 81 N. This dataset contains mean precipitation for each month, for every 0.5 by 0.5 grid cell that contains some land. These data were averaged together to produce a seasonal mean for each grid cell [December, January and February (DJF) for winter, and June, July and August (JJA) for summer]. Then, for each AGCM to be evaluated, all the 0.5 cells whose centres fall in each model grid box were averaged (unweighted) to form a mean observed seasonal precipitation for that grid box. The resulting field for the UKMO AGCM grid for JJA is shown in Figure 2(a) (grid boxes that are defined as ocean in the UKMO AGCM are not used). These fields were then compared with the values obtained from the 9-year model simulations (see Figure 2(b) for the UKMO AGCM simulation of JJA). In summer, the UKMO model reproduces most characteristics of the observed mean precipitation field. The main error is that eastern Europe and Russia are too dry in the model. More minor model errors are that northern Spain is too wet, as are Iceland and southern Greenland (although note the reservations of Hulme et al., 1995, about the observed climatology over Greenland). The analysis was repeated for all models, and comparison statistics between simulated and observed fields are given in Figure 3. In this figure, and in all subsequent comparisons, the models are presented in order of increasing horizontal resolution (as given by N in Table I). The mean over the entire region (Figure 3(a)) indicates that half of the models (square symbols) tend to overestimate European precipitation in winter (filled symbols) compared to the observations (triangles), while the other half produce very similar values. The model errors are less systematic in summer (open symbols), with five models too wet, five too dry and two very similar to the observed values. The reason why the observed values change Table I. Details of models validated: acronym, modelling centre, horizontal resolution (spectral or grid-box latitude/longitude size in degrees) and total number of grid boxes/points covering the globe (N). For spectral models, N is calculated from the grid on which physical parameterisations are applied Acronym Modelling centre Resolution N BMRC Bureau of Meteorology Research Centre R CCC Canadian Centre for Climate Research T COLA Centre for Ocean Land Atmosphere Studies R CSIRO Commonwealth Scientific and Industrial Research Organisation R CSU Colorado State University DERF Dynamical Extended Range Forecasting (at GFDL) T ECMWF European Centre for Medium-Range Weather Forecasts T GFDL Geophysical Fluid Dynamics Laboratory R GLA Goddard Laboratory for Atmospheres GSFC Goddard Space Flight Centre MRI Meteorological Research Institute UKMO UK Meteorological Office

4 508 T.J. OSBORN AND M. HULME Figure 1. Location of stations with daily precipitation observations in the western European dataset. The grey shading indicates the number of stations in each UKMO grid-box slightly from model to model is that the land sea masks are also different (with the extremes being the CSIRO model with most wet coastal locations being classified as ocean, while the BMRC model classifies them as land). The pattern of precipitation over the greater European region is more easily simulated in summer than in winter (Figure 3(b)). Few pattern correlations fall below 0.5, with some as high as 0.9 (e.g. GSFC in summer). The root-mean-squared error (RMSE) between models and observations combines elements of the mean and of the pattern. The high RMSE for the summer simulation of the CSU model (Figure 3(c)) is due mainly to the large overestimate of the mean precipitation by that model (Figure 3(a)). Only five models have RMSE values of under 1 mm day 1 for both DJF and JJA simulations of precipitation (GSFC, CSIRO, CCC, UKMO and GFDL).

5 EUROPEAN DAILY PRECIPITATION EVALUATION OF PRECIPITATION DAILY STANDARD DEVIATION Temporal standard deviations of daily precipitation timeseries give an indication of the variability of precipitation at a location or of a region. It is not necessarily the most useful indicator of variability due to the skewness of the distribution of precipitation amounts; nevertheless, AGCMs should produce a standard deviation similar to that observed if they are realistic models. Also, as Gregory and Mitchell (1995) point out, as more stations are averaged together to produce a timeseries of areal-mean precipitation, the distribution of these means becomes less skewed; standard deviations are, therefore, a more meaningful measure of temporal variability when, as in the present paper, grid-box mean timeseries are being considered. (The method used here could also be used to obtain estimates of the coefficient of variation, by dividing the standard deviations by the mean precipitation.) The standard deviations we consider are those of the full timeseries involved dry days are not first Figure 2. Mean summer precipitation (mm day 1 ) for (a) the greater European climatology gridded onto the UKMO model grid; and (b) from the UKMO model AMIP simulation. Only those grid boxes classified as land in the UKMO model are included

6 510 T.J. OSBORN AND M. HULME Figure 3. Validation statistics of seasonal mean precipitation for 12 model simulations for winter (filled symbols) and summer (open symbols) over the greater European region: (a) areal mean observed (triangles) and simulated (squares) values (mm day 1 ); (b) pattern correlations; and (c) root-mean-squared errors (mm day 1 ). Models are shown in order of increasing horizontal resolution removed. Locations with many dry days will have a standard deviation that is biased lower. Again the problem is less severe for the grid-box means considered here, since there are fewer dry days in the grid-box mean timeseries than there are in individual station timeseries.

7 EUROPEAN DAILY PRECIPITATION 511 Extending the earlier work of Kagan (1966), Osborn and Hulme (1997) have shown that a good estimate of the temporal standard deviation (S) of a true (i.e. with infinite observational coverage) grid-box-mean daily precipitation timeseries can be obtained from S 2 =S i2 r (r 0) (1) where S i2 is the mean of the variances of single-station timeseries within the grid box, and r is the mean interstation correlation between all pairs of stations within the box (computed from their timeseries, including dry days). The variance of the grid-box mean is lower than that of individual stations because only the variance that is common to all stations remains after the averaging. The fraction of variance that is common to all stations is given by r the lower this is, the less common variance there is, and the lower the standard deviation of the grid-box mean will be. Equation (1) is valid if S i2 and r are representative of the entire grid box, not just of the available stations. Two causes of error could, therefore, be introduced (if either the variances or the correlations are unrepresentative). The variances of the individual station timeseries in the western European dataset (Figure 1) were averaged together for all stations that fall in each grid box of the model to be evaluated, to obtain a value of S i2 for each box. The JJA values, converted to standard deviations, for the UKMO model grid boxes are given in Figure 4(a) as an example. Summer precipitation timeseries are most variable over the Alps and southern France, with less variability over north western Europe and over the Mediterranean islands (the latter may be due to more dry days). For boxes with few stations, some uncertainty must surround these estimates of single-station daily variances. The values from those boxes with a greater density of stations will be more certain, although bias due to station location (e.g. elevations) may still lead to a bias in the mean single-station variance. The mean interstation correlation between all pairs of stations in each grid box (r ) could be computed directly, by finding the correlations between all possible station pair combinations within a box, and then averaging them. By taking this approach, however, it is not possible to obtain an r value for any grid boxes that have just one station in, and for other boxes the r obtained may be biased by station clustering. Instead, following Osborn and Hulme (1997), we compute a correlation decay length (x 0 ) for each station by fitting the function r=e x/x 0 (2) to the correlation (r) versus separation distance (x) data for each station versus all other stations (including those outside the grid box). Osborn and Hulme (1997) have computed a correlation decay length in this way for each station in the western European dataset and for each season (their Figure 6). For each model to be evaluated, the x 0 values for all stations that fall in each grid box were averaged to produce a mean x 0 for each box. Many (10 000) pairs of points were selected at random from the box, and their separation distances used in Equation (2) together with x 0 (for x 0 ) to obtain the expected correlation for each pair. The correlations were then averaged to obtain a value of r for that box that is unbiased by the original station distribution. The values of r that result from this sequence of calculations are shown in Figure 4(b) for JJA, with calculations done on the UKMO model grid. Highest values (i.e. most coherent or largest-scale precipitation events) occur in western France and northern Scotland, with smallest values over Italy, particularly in the north. These variations are consistent with the dependence of x 0 on the type of precipitation (e.g. frontal or convective) and the degree of orographic variation. Equation (1) was then evaluated for each grid box of each model on a seasonal basis, to obtain an estimate of the observed daily standard deviation of grid-box mean precipitation. These values are shown (Figure 4(c)) for our example (UKMO model grid, for JJA), together with the results of the UKMO model simulation itself (Figure 4(d); land only). The observationally-based estimates of the standard deviation of grid-box mean precipitation (Figure 4(c)) are considerably lower than the observed station precipitation standard deviations (Figure 4(a)), with the difference largest where r is lowest. The Alpine region remains the most variable, but not by as great a margin. The UKMO simulation (Figure 4(d)) also has higher standard deviations over the Alps and southern France, with lower values elsewhere.

8 512 T.J. OSBORN AND M. HULME The outlined region in Figure 4(c) and (d) contains those grid boxes that have observations in and that are classified as land in the UKMO model. It is over this region that the model evaluation is performed, and similarly for all other models for DJF and JJA. The region is not sufficiently large, nor do the observations have a sufficiently coherent pattern in them, to warrant pattern correlation coefficients being used for evaluation. Instead, we simply consider the mean values over this western European region (Figure 5). There is a considerable upward trend in the observed levels of daily standard deviation with increasing resolution; this is expected since in larger boxes less of the variance is common to all locations in the box (i.e. r is lower). Via Equation (1), lower r leads to lower standard deviation for the larger boxes of the low resolution models. This resolution dependence is automatically obtained by the method that is used here (it also applies to the rainday frequency and precipitation intensity evaluations reported in subsequent sections of the present paper). It is important for this resolution dependence to be included for the correct evaluation of models. Here, for example, the GLA and ECMWF AGCMs have very similar JJA standard deviations (Figure 5), yet the finer resolution ECMWF model matches the observations well while the coarser resolution GLA model appears to be considerably too variable. More models tend to overestimate daily precipitation standard deviation than underestimate it (Figure 5). The CSIRO simulation is noticeable in that it is the only one to have lower than observed values in DJF (it does so in JJA too, but so do three other simulations). The CSU model is particularly variable, although part of this bias may be caused by the high mean precipitation in that model for this region (using the coefficient of variation might remove some standard deviation errors that are due to too high or too low mean precipitation). In terms of getting the mean standard deviation right in both seasons, it would appear that the UKMO model does best. This is carried over to the RMSE of the field (not shown); UKMO is one of only four models with an RMSE of 1.5 mm day 1 or less for both seasons (the others are CSIRO, CCC and BMRC). 5. EVALUATION OF RAINDAY FREQUENCIES We define the rainday frequency as the number of days per season that are wet, where a wet day is indicated by a precipitation total of more than 0.1 mm (a rainday might more accurately be called a precipitation day, since it includes days with snowfall). The 0.1 mm threshold was chosen because it is the same as that used in the European baseline climatology (Hulme et al., 1995) which we utilise here. When computing the rainday frequency of a grid-box-mean precipitation timeseries, it is useful to use the probability of a day being dry (P d ), from which the rainday frequency is known [=(1 P d ) number of days in season]. Figure 5. As for Figure 3(a), but for daily precipitation standard deviation

9 EUROPEAN DAILY PRECIPITATION 513 Osborn and Hulme (1997) have shown empirically that the dry-day probability of the mean timeseries of n stations, P d (n), can be estimated from P d (n)=p d (1)[ n + n (p )] (3) where n and n are empirically-defined parameters and p is the mean interstation probability of coincident dry days (cf. r ). The latter parameter is the mean value for each grid box of a statistic, p, between two station timeseries, defined as the probability that both stations are dry on a particular day given that at least one is dry. Thus, if X i is the event that a station x i is dry, then p= P(X 1X 2 ) P(X 1 X 2 ) If all values of p are computed between all combinations of station pairs within a grid box, and then averaged, p is obtained. As for the calculation of r, this method may lead to a biased value if the stations are not uniformly distributed, and cannot be used for boxes with a single station. Instead, the decay length methodology is again utilised (see Section 4 and Osborn and Hulme, 1997) to compute a probability decay length, x p. As before, the decay function (a hyperbolic in this case) is then integrated for each grid box, using the mean x p value for that box and the distribution of all possible separation distances within the box. The mean value of p for each box is thus obtained, independent of the actual location of available stations. For the UKMO model grid, the JJA p values are shown in Figure 4(e). As expected, highest values occur in the driest regions, lowest in the wetter regions (because, even if two stations are so widely separated that they are independent, the probability that their dry days coincide will be higher if they each have many dry days). Superimposed on this is a tendency for regions with smaller-scale precipitation events to have lower p. In the same way that Equation (1) could be biased by a poor estimate of the variance of single-station timeseries, Equation (3) needs an accurate estimate of the dry day probabilities [P d (1)] of single-station timeseries, representative of each entire grid box. In this case we can have confidence in the estimate of P d (1) since it is computed from the European baseline climatology of Hulme et al. (1995), based on a dense network of observations and adjusted for elevation biases. [Inter-daily variance of precipitation is not one of the variables available in this climatology, which is why we had to fall back on the sparser station network that had daily timeseries available (Figure 1) for the analysis of that variable.] The procedure by which grid-box mean values of P d (1) are computed is identical to that used earlier to obtain grid-box mean precipitation from this climatology (see Section 3). For the UKMO model grid, the mean dry-day probability (converted back to rainday frequencies) of single station timeseries within each box is shown in Figure 4(f) for JJA. Values are obtained for all grid boxes with any land in them. Values range from under 20 summer raindays in the Mediterranean to over 56 in the north-west of the region (Figure 4(f)). These are the rainday frequencies to be expected from point observations in each box (although with spatial and elevation variations). Using the values in Figure 4(e) and (f) (or equivalent for different seasons and/or different models), the rainday frequency of the mean timeseries of n stations within a box can be computed by substituting the grid-box mean values of P d (1) and p into Equation (3). Values of n and n are also required, and Osborn and Hulme (1997) estimated them for n=1,..., 15. It is not possible to estimate the rainday frequency of a true grid-box mean precipitation (i.e. as if there were infinite stations within the grid box), since we do not know what the appropriate values of and are for that case. Osborn and Hulme (1997), however, indicate that they are likely to be very similar to those for a 15-station mean ( 15 = 0.22 and 15 =1.21), at least for the size of the grid boxes used by the AMIP models. We therefore compute P d (15) as an estimate of P d (). For our example case (JJA on the UKMO grid) the validation field (Figure 4(g)) thus obtained can be compared with rainday frequencies calculated from the JJA output of the UKMO simulation (Figure 4(h)). The rainday frequencies of grid-box precipitation (Figure 4(g)) are considerably higher than for precipitation at single points (Figure 4(f)). The overall pattern remains the same, but weaker. The UKMO summer simulation has too many raindays in many grid boxes, particularly over mountainous regions. (4)

10 514 T.J. OSBORN AND M. HULME Figure 6. As Figure 3, but for rainday frequency (square brackets indicate ranges of values, see text) Extending the evaluation to all models and two seasons, all 12 AGCMs simulate too many raindays in winter (although the ECMWF simulation is very close to the observed value), while in summer only five models have too many, four are about right and three have too few raindays (Figure 6(a)). This is partly in accord with earlier studies (Reed, 1986; Wilson and Mitchell, 1987; Rind et al., 1989; Gregory and

11 EUROPEAN DAILY PRECIPITATION 515 Mitchell, 1995) that found that AGCMs tend to simulate too many raindays. Our methodology and subsequent results indicate that this bias has been overcome for some models, particularly in summer. Given the small region with available data (see e.g. the region outlined in Figure 4(g) and (h)), pattern correlations between observed and simulated fields might produce rather uncertain results. But the particularly coherent summer pattern (Figure 4(g)) is, in fact, captured to some extent by all models (Figure 6(b)) clearly not by mere chance. The fact that some models fail to capture the weaker winter pattern does not necessarily indicate a poorer model simulation. Evaluation on a grid-box basis is better represented by taking the differences between model and observed rainday frequencies, on a grid-box by grid-box basis, and binning them into error bins of size 5 days (Figure 7; thin line DJF, thick line JJA). Figure 4(g) and (h) are summarised, therefore, by the thick line in Figure 7(g), showing that in only 30% of the grid boxes the UKMO simulation is within 5 days of the observations. Those simulations with particularly wide distributions (CSU in winter; BMRC, DERF, ECMWF and COLA in summer) indicate a failure to replicate the observed pattern of rainday frequencies. Those simulations with a bias to having too many raindays have their error distributions shifted to the right (positive errors; i.e. all winter simulations), while those summer simulations that have too few raindays have error distributions shifted to the left. Those simulations with very narrow error distributions (MRI and GSFC in summer; GSFC, CCC, GFDL and ECMWF in winter) probably replicate the pattern well, although the overall mean is still biased in some simulations. Combining elements of both the mean and the pattern, the RMSE values (Figure 6(c)) give an overall evaluation of the rainday frequency simulations. Ten of the models have winter simulations that are better than, or as good as, their summer simulation of rainday frequencies. If the two weather forecasting models (DERF and ECMWF) are excluded (which could be argued for because they have not, perhaps, been specifically tuned for the type of climatological simulation required for AMIP), then there is the possibility of a relationship between rainday RMSE and horizontal resolution (Figure 6(c)). Given the many other differences between the models and the way in which they have been tuned, it is a rather tentative relationship. 6. EVALUATION OF PRECIPITATION INTENSITY Precipitation intensity is simply defined as the mean amount of precipitation that falls on a rainday, and is computed by dividing the total rainfall for a season by the number of raindays during that season. The former values are obtained from Section 3 (using the European baseline climatology) and the latter values from Section 5 (using the methodology of Osborn and Hulme, 1997, together with western European station data, to estimate p, and the European baseline climatology to estimate the rainday frequency of single-station timeseries in each grid box). As an example, we can compare the observed estimates of intensity for JJA on the UKMO model grid (Figure 8(a)) with the mean intensity that the UKMO model produces (Figure 8(b)). Most intense summer rainfall occurs over the mountainous regions of southern Europe (Figure 8(a)) with less intense rain elsewhere. The UKMO model reproduces part of this pattern, although with lower intensities overall (Figure 8(b)). Error histograms (Figure 9) are again used to evaluate all the model simulations, for both DJF and JJA. The CSU is the only model to have large positive biases (i.e. precipitation that is too intense) in both seasons (Figure 9(a)). Many models are biased towards too weak intensities (MRI in winter; CSIRO, UKMO and GFDL in both seasons; CCC, BMRC and COLA in summer), while others have distributions centred near to zero error. Of the latter, the narrower distributions indicate that the pattern is well captured too (MRI, GLA and ECMWF in summer). The COLA winter simulation is centred close to zero error, but its wide distribution indicates a poor spatial simulation (Figure 9(l), thin line).

12 516 T.J. OSBORN AND M. HULME Figure 7. The distribution of grid-box differences between simulated rainday frequency and our estimate of the rainday frequency of observed grid-box mean precipitation, binned into 5-day bins for 12 different models. Thin lines are DJF, thick lines are JJA, shaded bars indicate very small differences only

13 EUROPEAN DAILY PRECIPITATION INTERDECADAL VARIABILITY The results obtained here, from both the observations and the model simulations, may be affected by interdecadal variability and, hence, the evaluation may be biased. The use of only 9 years of simulated precipitation introduces the largest uncertainties, and this is addressed below. We do not consider variations that are driven by external forcing changes (natural or anthropogenic, including any longtimescale trends due to an enhanced greenhouse effect), although we note that any bias due to these variations may be reduced due to our use of observed data from the period to compare with model simulations that have greenhouse gas concentrations applicable to the period (i.e. some of the external forcings should be the same in the observations as in the models). To assess the influence of random internal variability, we make use of a long timeseries of daily data that is available from the control simulation of the UK Meteorological Office/Hadley Centre coupled ocean atmosphere model (Mitchell et al., 1995). The atmospheric component of that coupled model (HADCM2) is very similar to the model used for the UKMO AMIP simulation. By evaluating, separately, ten non-overlapping 9-year sequences of daily precipitation output from this control simulation, we have obtained an estimate of the range over which the evaluation statistics may vary due to the random internal variability of the HADCM2 coupled model. For the rainday frequency, the ranges of values of the mean, pattern correlation and RMS error obtained from the ten different periods are plotted on Figure 6 for DJF and JJA (they are indicated by the thin square brackets next to the DJF and JJA values for UKMO). In HADCM2, it is clear that mean rainday frequency (Figure 6(a)) has more decadal variability in summer than in winter (up to 9% of the mean in summer, up to 5% of the mean in winter). The pattern changes by only a little, with similar range in each season (Figure 6(b)). The summer RMSE range is more than double the winter range (Figure 6(c)). If the root-mean-squared errors are used to intercompare the models, the UKMO Figure 8. For summer; (a) estimate of the mean intensity (mm day 1 ) of grid-box mean observed precipitation in each UKMO grid box; and (b) mean precipitation intensity (mm day 1 ) simulated by UKMO model for land boxes. The region with observed data that is also classified as land in the UKMO model is outlined

14 518 T.J. OSBORN AND M. HULME Figure 9. As for Figure 7 but for mean precipitation intensity using 0.5 mm day 1 error bins

15 EUROPEAN DAILY PRECIPITATION 519 Table II. Validation statistics between the UKMO model simulation of mean precipitation and the observations from 1961 to 1990 and from 1980 to Areal means and root-mean-squared (RMS) errors have units of mm day Areal means Pattern correlation RMS error 1 2 Difference DJF versus UKMO versus UKMO versus JJA versus UKMO versus UKMO versus AGCM appears to have the best summer simulation; however, the interdecadal variability indicated by the square brackets in Figure 6(c) demonstrates that it is not possible to state with confidence that the UKMO simulation is better than at least six of the other simulations (CSIRO, CCC, BMRC, GFDL, DERF and COLA). Part of the influence of boundary-driven variability has also been assessed. The model simulations were forced by the observed sequence of SST patterns for the period If the precipitation characteristics over Europe are controlled to some extent by the global SST field, then the period may have been different to the period (due to SST differences between the two periods). Yet we have compared the model simulations for (1979 was discarded) with observations averaged over This will have caused no biases if European precipitation is not greatly affected by SST anomalies; there is conflicting evidence regarding this question (e.g. Fraedrich and Muller, 1992; Hurrell, 1995; Zwiers et al., 1995). In case European precipitation is significantly affected by SST anomalies, we should evaluate the models against observations from the period only. This is not a sufficiently long period from which to obtain accurate estimates of p and r, so we cannot use just this period for evaluating the daily standard deviation, the rainday frequency or the mean precipitation intensity. But we can do so for mean precipitation, using values from the European climatology of Hulme et al. (1995) for only. They are only a little different from the mean (DJF was drier over Iberia, northern Morocco and Algeria, Iran and northem Scandinavia, and was wetter over Scotland and Austria; JJA was drier over the Pyrenees, Alps and Dolomites, and was wetter over southern Sweden and Finland and over much of western Russia). The observed means, the observed means and the UKMO model simulated means were intercompared, with results given in Table II. In DJF, the UKMO mean precipitation over this greater European region is very close to both the mean and the mean (which are almost identical). Using the field slightly worsens the comparison, although a minor improvement to the pattern correlation is obtained. Although the two observed mean fields have similar means and similar patterns, their RMS difference is surprisingly large but still more than five times smaller than the best model RMSE. In JJA, there is a larger difference in mean precipitation between the two periods, with being wetter. The UKMO model has a dry bias, though, that is over six times as large. Using observations also worsens the pattern correlation and RMSE statistics. So, the differences between the observations and the observations are not sufficiently large to bring the model simulations much closer to the observations.

16 520 T.J. OSBORN AND M. HULME 8. SUMMARY AND CONCLUSIONS The precipitation simulations of 12 atmospheric general circulation models (AGCMs) have been evaluated over Europe, on seasonal and daily timescales. These AGCM simulations were performed as part of the AMIP exercise (Gates, 1992), and we have evaluated all models that provided daily output (see Table I). Nine-year sequences of output were available from each simulation. This work provides the most comprehensive evaluation yet undertaken of modelled daily precipitation variability, in terms of area covered and number of models involved. As an introductory comparison, the seasonal mean precipitation simulations over a greater European region were evaluated for winter and summer, comparing them with the climatology of Hulme et al. (1995). Winter precipitation simulations are all either too wet or the same as observed, none are too dry. The pattern of precipitation is better simulated in summer than in winter. The CSU summer simulation is almost three times too wet, however, and it is likely that this large error biases the daily statistics too. The daily variability of precipitation is evaluated by the use of three statistics: daily standard deviation, the frequency of raindays and the mean precipitation intensity of those raindays. The difficulty in comparing the statistical characteristics of timeseries from point (station) sources with timeseries from areal-mean (grid box) sources has been highlighted by Osborn and Hulme (1997). This difficulty is increased for variables such as precipitation on a daily timescale whose anomalies occur with characteristic spatial scales that are less than the size of the grid boxes. With a very high density of stations with sufficiently long timeseries of daily data, a good estimate of grid-box mean values could be made, and the difficulty would be resolved. Such a station network, however, is only available to the scientific research community for very limited regions of the world. Evaluation of the statistics of daily precipitation simulated by climate models has been limited because of this problem. For example, Gordon et al. (1992), Whetton et al. (1993), Fowler and Hennessy (1995) and Hennessy et al. (1997) make use of model estimates of daily precipitation, yet make no attempt to evaluate the present-day simulations of daily precipitation made by their climate models. In the present paper, we have used the method developed by Osborn and Hulme (1997) to estimate some of the statistics of grid-box-mean daily precipitation using relatively few stations, by making use of parameters that describe the spatial scale of precipitation events. Note, first, that a different estimate must be constructed for each model, as the grid boxes vary in size and position from one model to another and, second, that some station data are required to derive these estimates and so we have limited the evaluation to part of western Europe where we have such station data (see Figure 1). In winter, all models produce too many raindays (although the bias is quite small for some models), most have too strong day-to-day variability, and (with the exception of the CSU model) all produce precipitation that is either less intense or of the same intensity as the observed estimates. The results are much less systematic in summer: model variability and rainday frequency simulations span either side of the observed values; in terms of intensity, however, all models except the CSU model again produce the same level as, or lower intensity than, observed. The patterns of how these statistics vary over the western European region are better captured in summer than in winter, possibly because the patterns are weaker in winter. Ranking all the model simulations for both summer and winter, and for all four evaluation statistics, suggests that out of the 12 models evaluated the UKMO and GFDL models do best at simulating European precipitation. We have shown, however, that the use of only a 9-year section of model output could significantly affect the evaluation results (due to interdecadal variability). It is not possible, therefore, to attribute firm confidence to such a ranking. If the two models used mainly for weather forecasting (DERF and ECMWF) rather than for climate studies are ignored, then there is a suggestion that simulations improve with increased resolution. This possible relationship is strongest for the evaluation of rainday frequencies, as described by root mean squared errors. The relationship must be considered as very tentative for two reasons: first, the aforementioned interdecadal variability could destroy the resolution relationship (although it could equally enhance it); second, there are many other differences between the models that could explain the differences between simulations.

17 EUROPEAN DAILY PRECIPITATION 521 Table III. Summary of some of the physical parameterisations employed in the models. Types of convection scheme classified as BMF, bulk mass flux; CE, cumulus ensembles; MCA, moist convective adjustment; MCC, moisture convergence closure, although there may be differences within each class Model Gravity-wave drag Convection scheme Shallow convection Evaporation of falling rain BMRC MCC CCC MCA COLA MCC CSIRO MCA CSU CE DERF MCA ECMWF BMF GFDL MCA GLA CE GSFC CE MRI CE UKMO BMF In addition to resolution, we have also categorised the model simulations by the type of convective precipitation parameterisation used in the models (Table III, extracted from Phillips, 1994). The summer simulations of mean precipitation, rainday frequency and mean intensity do not show systematic variations with convective parameterisation. A much larger sample than 12 simulations would be required to effectively compare the performance of such parameterisations. Due to other inter-model differences (and also note that two models with the same parameterisation type may have different implementations or parameter values) it would appear that the only way to test different parameterisations would be to try them in the same model (e.g. Roeckner and Arpe, 1995). Further classifications have been performed, and two significant relationships between model details and performance were identified. First, those models that include evaporation of falling precipitation (see Table III) tend to have more intense (and closer to observed) mean daily intensity in summer than those models that do not include this process. Somewhat confusingly, though, there was no systematic effect on the number of raindays. Second, those models that include a parameterisation of gravity-wave drag (Table III) tend to have less intense summer precipitation than those that do not include it. Again, no relationship is apparent in the number of raindays. These two relationships appear to be significant (with 95% and 99% confidence, respectively), but it is not known what effect the common lineage of some of the models has on the significance estimates. Causality has not been proven in either case. ACKNOWLEDGEMENTS This work was supported by the UK Department of Environment, Transport and the Regions (DETR EPG 1/1/48) and by the US Department of Energy (DE-FG02-86ER60397). The AMIP model data were supplied by PCMDI, LLNL, under AMIP Diagnostic Sub-project No. 21. Their help with data and software is acknowledged. We acknowledge the work of all modelling centres who performed the AMIP simulations used here, and their comments on this manuscript. David Viner (Climate Impacts LINK Project, UK DETR EPG 1/1/16) supplied the Hadley Centre model results and provided computer facilities for this work. Discussions with Phil Jones, Keith Briffa and Jonathan Gregory were useful. REFERENCES Fowler, A.M. and Hennessy, K.J Potential impacts of global warming on the frequency and magnitude of heavy precipitation, Nat. Hazards, 11, Fraedrich, K. and Muller, K Climate anomalies in Europe associated with ENSO extremes, Int. J. Climatol., 12, Gates, W.L AMIP: the Atmospheric Model Intercomparison Project, Bull. Am. Meteorol. Soc., 73, Gordon, H.B., Whetton, P.H., Pittock, A.B., Fowler A.M., and Haylock, M.R Simulated changes in daily rainfall intensity due to the enhanced greenhouse effect: implications for extreme rainfall events, Clim. Dynam., 8,

18 522 T.J. OSBORN AND M. HULME Gregory, J.M. and Mitchell, J.F.B Simulation of daily variability of surface temperature and precipitation over Europe in the current and 2 CO 2 climates using the UKMO climate model, Q. J. R. Meteorol. Soc., 121, Hennessy, K.J., Gregory J.M. and Mitchell, J.F.B Changes in daily precipitation under enhanced greenhouse conditions, Clim. Dynam., 13, Hulme, M The cost of climate data: a European experience, Weather, 49, Hulme, M., Conway, D., Jones, P.D., Jiang, T., Barrow E.M., and Turney, C Construction of a European climatology for climate change modelling and impact applications, Int. J. Climatol., 15, Hurrell, J.W Decadal trends in the North Atlantic Oscillation regional temperatures and precipitation, Science, 269, Kagan, R.L An evaluation of the representativeness of precipitation data, Gidrometeoizdat (in Russian). Karl, T.R., Knight R.W. and Plummer, N Trends in high-frequency climate variability in the twentieth century, Nature, 377, Mearns, L.O., Giorgi, F., McDaniel, L., and Shields, C Analysis of daily variability of precipitation in a nested regional climate model: comparison with observations and doubled CO 2 results, Global Planetary Change, 10, Mitchell, J.F.B., Davis, R.A., Ingram, W.J., and Senior, C.A On surface temperature, greenhouse gases and aerosols: models and observations, J. Climate, 8, Noda, A. and Tokioka, T The effect of doubling the CO 2 concentration on convective and non-convective precipitation in a general circulation model coupled with a simple mixed layer ocean model, J. Meteorol. Soc. Jpn, 67, Osborn, T.J. and Hulme, M Development of a relationship between station and grid-box rainday frequencies for climate model evaluation, J. Climate, 10, Phillips, T.J A Summary Documentation of the AMIP Models, PCMDI Report No. 18, Lawrence Livermore National Laboratory, 343 pp. Reed, D.N Simulation of time series of temperature and precipitation over eastern England by an atmospheric general circulation model, J. Climatol., 6, Rind, D., Goldberg, R. and Ruedy, R Change in climate variability in the 21st century, Climatic Change, 14, Roeckner, E. and Arpe, K AMIP experiments with the new Max Planck Institute model ECHAM4, in Gates, W.L. (ed.), Proceedings of the 1st International AMIP Scientific Conference, WCRP no. 92, Skelly, W.C. and Henderson-Sellers, A Grid-box or grid-point: what type of precipitation data do GCMs deliver?, Int. J. Climatol., 16, Suppiah, R. and Hennessy, K.J Trends in the intensity and frequency of heavy rainfall in tropical Australia and links with the Southern Oscillation, Aust. Meteorol. Mag., 45, Whetton, P.H., Fowler, A.M., Haylock, M.R., and Pittock, A.B Implications of climate change due to the enhanced greenhouse effect on floods and droughts in Australia, Climatic Change, 25, Wilson, C.A. and Mitchell, J.F.B Simulated climate and CO 2 -induced climate change over western Europe, Climatic Change, 10, Zwiers, F.W., Rowell, D.P., Hense, A., Davies, J.R., and Christoph, M AGCM intercomparison diagnostics: SST-forced variability versus internal variability, in Folland, C.K. and Rowell, D.P. (eds), Workshop on Simulations of the Climate of the Twentieth century using GISST, Climate Research Technical Note 56, Hadley Centre,

19