Does the model regional bias affect the projected regional climate change? An analysis of global model projections

Climatic Change (21) 1:787 795 DOI 1.17/s1584-1-9864-z LETTER Does the model regional bias affect the projected regional climate change? An analysis of global model projections A letter Filippo Giorgi Erika Coppola Received: 5 March 21 / Accepted: 12 April 21 / Published online: 8 May 21 Springer Science+Business Media B.V. 21 Abstract An analysis is presented of the dependence of the regional temperature and precipitation change signal on systematic regional biases in global climate change projections. The CMIP3 multi-model ensemble is analyzed over 26 land regions and for the A1B greenhouse gas emission scenario. For temperature, the model regional bias has a negligible effect on the projected regional change. For precipitation, a significant correlation between change and bias is found in about 3% of the seasonal/regional cases analyzed, covering a wide range of different climate regimes. For these cases, a performance-based selection of models in producing climate change scenarios can affect the resulting change estimate, and it is noted that a minimum of four to five models is needed to obtain robust precipitation change estimates. In a number of cases, models with largely different precipitation biases can still produce changes of consistent sign. Overall, it is assessed that in the present generation of models the regional bias does not appear to be a dominant factor in determining the simulated regional change in the majority of cases. 1 Introduction Climate change projections at both the global and regional scale are characterized by multiple sources of uncertainty, as for example associated with model configuration or greenhouse gas (GHG) emission scenario (Giorgi 25; Tebaldi and Knutti 27). In order to characterize such uncertainties, global and regional climate model projections need to be based on probabilistic approaches using multi-model ensembles of experiments (e.g. Murphy et al. 24; Giorgi25; Déqué et al. 25; Tebaldi and Knutti 27). Within this context, an important source of uncertainty is the presence of systematic model errors. It might in fact be argued that systematic model biases can affect the simulated changes, and this argument is for example the basis for F. Giorgi (B) E. Coppola Earth System Physics Section, International Centre for Theoretical Physics, Trieste, Italy e-mail: giorgi@ictp.it

788 Climatic Change (21) 1:787 795 weighting the results from different models based on selected performance metrics (e.g. Giorgi and Mearns 22; Murphy et al. 24; Tebaldi et al. 25; Greene et al. 26; Tebaldi and Knutti 27; Raisanenetal.21). This issue is also important when, for given applications, it is necessary to select only a relatively small number of models, for example when producing driving fields for regional model nesting (Déqué et al. 25). This model selection might indeed be guided by the presence of systematic model errors. Although studies since the eighties have shown that the global climate sensitivity and the regional changes of a model can depend on global and regional biases (e.g. Spelman and Manabe 1984; Kittel et al. 1998; Murphy et al. 24; Shukla et al. 26; Tebaldi and Knutti 27; Whetton et al. 27; Raisanenetal.21) andthat bias assumptions may affect climate projections (Buser et al. 29), a systematic assessment of the dependence of the climate change signal on the underlying model bias at the regional scale throughout the globe is not available in the literature. In this paper we thus present such an assessment. We examine surface air temperature and precipitation results from the CMIP3 ensemble of global climate simulations for the twentieth and twenty-first century (Meehl et al. 27) averaged over the 26 land regions of sub-continental size identified by Giorgi and Bi (25). We first investigate for what regions and seasons a significant correlation is found between model projected changes in a given variable (seasonal temperature and precipitation) and systematic biases in simulating present day values for the same variable. For the regional/seasonal cases in which such a correlation exists, we then examine to what extent the selection of sub-sets of models based on the magnitude of the model bias affects regional projections obtained from this sub-ensemble. The implications of our results for the use of multi-model ensembles to characterize climate change projections and associated uncertainties are discussed in the last section. 2 Models and experiments We use here a sub-set of the CMIP3 ensemble of experiments described by Meehl et al. (27), consisting of twentieth and twenty-first century climate simulations with more than 2 coupled Atmosphere Ocean General Circulation Models (AOGCMs) from laboratories across the world and accessible at the PCMDI web site http://www-pcmdi.llnl.gov. More specifically, we analyze models that completed simulations for both the twentieth century and the twenty-first century under the A1B GHG Emission scenario of IPCC (2). We limit our analysis to the A1B scenario runs noting that results for the other available scenarios (B1, A2) scale well at the sub-continental scale (Giorgi 28) and thus lead to similar conclusions. Eighteen AOGCMs and corresponding simulations are analyzed here as reported in Table 1 of Giorgi and Bi (25). They include models by the CCMA, CNRM, CSIRO, GFDL (two versions), GISS (three versions), IAP, INMCM, IPSL, MIROC (two versions), MIUB, MPI, MRI, NCAR (two versions), UKMO. For some models, multiple realizations were available and in this case we used the ensemble average of all realizations. Since we use 3-year averages for our study (see below), the internal-model variability is low compared to the inter-model variability (Giorgi and Francisco 2), so that our conclusions are not substantially affected by this approach.

Climatic Change (21) 1:787 795 789 All the model data are interpolated onto a common 1 latitude longitude grid as in Giorgi and Bi (25). Model biases are calculated with respect to the observation dataset produced by the Climatic Research Unit (CRU) of the University of East Anglia (New et al. 2). The CRU data is also interpolated on the same common 1 grid, and is used to produce a corresponding land ocean mask grid (see Giorgi and Bi 25). We note that different observational datasets might differ over some regions, especially for precipitation, however this does not affect strongly our basic conclusions on the change bias relationships, since correlations are not strongly dependent on the base observed values used to calculate the biases. We analyze data averaged over the 26 land regions identified by Giorgi and Bi (25), where only land points of the 1 land-mask grid are considered in the averaging. Since different models have different resolutions and thus different land masks, this procedure adds an element of uncertainty in regions characterized by complex coastlines. This problem is however not critical in view of the large, subcontinental size of the regions. Biases for each region are calculated for the reference 3 year period of 1971 2, i.e. the bias is defined as the difference between simulated and observed (CRU) 1971 2 regionally averaged value. With a similar procedure, changes are calculated for the period 271 21, i.e. the change for a variable and a region is given by the difference between the averages of the 271 21 and 1971 2 periods. The choice of the late twenty-first century 271 21 period was made to maximize the change signal. Our main conclusions however apply to earlier periods as well due to the above mentioned scaling properties of the regional change signal (Giorgi 28), although we note that correlations should be lower in the early decades of the century when the changes are more strongly contaminated by natural variability. Regional biases and changes are calculated for December January February (DJF) and June July August (JJA) surface air temperature and precipitation. 3Results We first calculate the inter-model correlation between bias and change for each regional and seasonal case. This is computed by calculating the mean bias and change for each regional/seasonal case and each model and then correlating the bias and changes across the 18 models. A significant correlation implies that for a given regional/seasonal case the model bias is related to, and thus presumably affects, the model simulated change. Table 1 reports all regional/seasonal cases in which a statistically significant inter-model change bias correlation is found at the 9% confidence level (i.e. a correlation coefficient of about.38 and larger). For each case the table shows the correlation coefficient, the ensemble mean change and bias and, for precipitation, the number of models that project a change of the same sign as the ensemble average change (i.e. a measure of model agreement). For precipitation there are 15 regional/seasonal cases of significant inter-model bias change correlation out of 52 total cases (26 each for JJA and DJF), while for temperature we find only four cases with significant correlation. Therefore, the dependency of the mean regional climate change signal on the model regional bias is significant in about 3% of cases for precipitation and it is almost negligible for temperature. The latter result can be understood in view of the dominant dependence

79 Climatic Change (21) 1:787 795 Table 1 Regional cases in which the inter-model correlation between model simulated change and bias is significant at the 9% confidence level Region Season Correlation Mean precipitation Mean precipitation Number of coefficient change bias agreeing models ALA DJF.4 19.7% 68.99% 18 GRL DJF.43 2.78% 5.85% 18 NAS DJF.43 29.5% 4.62% 18 NEU JJA.5 1.7% 9.15% 9 NEE JJA.46 4.39% 19.59% 11 CAS JJA.55 1.87% 5.93% 13 TIB JJA.38 5.71% 63.55% 13 SEA DJF.38 5.16%.21% 14 NAU JJA.47 12.71% 29.72% 13 SAU JJA.55 12.5% 5.97% 15 SAU DJF.4.85% 2.49% 8 SQF JJA.4 5.83% 55.9% 11 CAM JJA.66 12.4% 24.7% 15 AMZ JJA.38 5.28% 41.83% 13 CSA DJF.49 3.57% 4.24% 13 NEU JJA.5 2.82 C.85 C NAU DJF.45 3.3 C.47 C EQF JJA.38 2.97 C.55 C CAM JJA.46 3.23 C.1 C For each case the table shows the correlation coefficient, the ensemble mean change (271 21 vs. 1971 2) and bias (1971 2) and, for precipitation, the number of models agreeing on the sign of the mean change of the regional temperature change signal on the global model sensitivity rather than local processes (Giorgi 28). We note however that this conclusion is limited to temperature for land regions, since a significant and physically expected negative correlation near the sea ice edge has been found in previous studies (Raisanen 27; Raisanen et al. 21). Thus, in the remainder of this paper, the attention will focus on precipitation. Figure 1 shows a map of the precipitation regional/seasonal cases with a significant bias change relationship. An analysis of this figure and of Table 1 reveals that some patterns of change/bias dependence appear to emerge, although not consistently across the globe, indicating that different processes are at play over different regions. A first pattern involves the high latitude northern hemisphere regional cases, for which we always find a negative correlation between the bias and the change (ALA- DJF, GRL-DJF, NAS-DJF, NEU-JJA, NEE-JJA). A negative correlation implies that a lower (higher) or more negative (positive) bias tends to produce a higher (lower) change response. In the winter seasonal cases, ALA-DJF, GRL-DJF and NAS-DJF the changes are all positive and shared by all models, and the biases are positive and large, except for GRL-DJF, which shows a small negative bias. Conversely, in the northern European summer cases, NEU-JJA and NEE-JJA, we find a general underestimate of precipitation (negative bias) and a mixed change signal across models. Therefore, the negative bias change correlation occurs in conjunction with different contexts, large model agreement and positive bias in one case, small model agreement and negative bias in the other. This makes it thus difficult to identify the contribution of underlying processes.

Climatic Change (21) 1:787 795 791 Fig. 1 Sub-continental scale regions for which a statistically significant correlation (9% confidence level) between precipitation bias and change is found across the 18 models analyzed. Continuous (dashed) lines indicate positive (negative) correlation, red (blue) color indicates JJA (DJF). The inter-model correlation coefficients are reported for the regions, which are the same as in Giorgi and Bi (25). Also indicated with thin lines are the remaining regions analyzed for which a significant bias change inter-model correlation was not found All the other cases show a positive correlation between bias and change, implying that a higher (lower) positive or smaller (larger) negative bias will lead to an enhanced (reduced) change response. Again, a wide range of different change patterns show the same bias change correlation sign, so that the identification of the underlying physical processes is difficult. It is worth noting that the inter-model correlation between change and bias appears especially pronounced over the two Australian regions (NAU and SAU) and over Central America (CAM) in JJA. Figure 2 presents illustrative scatter-plots of precipitation change vs. bias across the 18 models for two cases with positive bias change correlation (CAM-JJA, corr =.66; SAU-JJA, corr =.55), a case with negative correlation (ALA-DJF, corr =.4) and a case with large simulated change but negligible bias change correlation (MED-JJA, corr =.8). In the latter case (Fig. 2d), the different models exhibit biases of different sign and magnitude yet they all simulate negative precipitation changes, although with seemingly randomly varying magnitudes. This figure therefore illustrates how the projection of reduced summer precipitation over the Mediterranean region predominant in various generations of model simulations (e.g. Giorgi and Lionello 28) is a general result not related to the regional precipitation bias in the models. This general conclusion is shared by other regions (e.g. South Asia and East Asia and southern Africa and southern South America). The CAM-JJA and SAU-JJA cases are the ones with the largest positive bias change inter-model correlations, one for local summer and one for local winter

792 Climatic Change (21) 1:787 795 5 a CAM-JJA 2 b SAU JJA R 2 =.66 5 1 5 5 bias (%) 2 R 2 =.55 4 6 4 2 2 4 bias (%) 4 c ALA-DJF d MED-JJA 3 2 1 R 2 =.4 2 4 2 4 6 8 1 12 bias (%) 6 1 5 5 bias (%) Fig. 2 Scatter-plots of bias vs. change across the 18 models analyzed for four regional/seasonal cases: a CAM-JJA; b SAU-JJA; c ALA-DJF; d MED-JJA. For the cases in which the bias change inter-model correlation is significant, the correlation coefficient and the corresponding trend line are reported. Asterisks (circles) indicate models with the largest (smallest) magnitude of bias conditions (Fig. 2a, b). It can be seen that in both cases the models with the largest negative biases tend to produce the largest negative changes. A tie between change and bias is also illustrated in Fig. 2c (ALA-DJF) in which all models exhibit a positive bias and a positive change, but smaller changes tend to occur for models with the largest biases. One of the important issues for which the bias change relationship is important is the selection of sub-samples of models for producing climate change projections. It is in fact possible that when only a small sample of models is selected or available, the associated changes may be affected by the underlying biases. Figure 3 presents, for the four illustrative cases of Fig. 2, two sets of change calculations. For each point in the graphs, the change reported in the y-axis is calculated by averaging over the corresponding number of n models in the x-axis. Two curves of points are shown. In the first (denoted as large bias and asterisks) for each value of n the average is taken over the n models with the largest absolute value of the bias, therefore n = 1 indicates the model with the largest bias, n = 2 the average over the two models with the largest bias, etc., until n = 9 is reached, which averages over the worst performing half of the total ensemble. Similar averages are taken for the second curve using models with the lowest absolute values of the bias ( small bias and circles). In other words, the asterisks (circles) denote averages using the n worst (best) performing models. Some interesting features can be observed. In the CAM-JJA case (Fig. 3a), the average change for the best performing models does not depend strongly on n. This

Climatic Change (21) 1:787 795 793 1 a CAM-JJA 2 4 6 b SAU-JJA large bias small bias 2 3 large bias small bias 8 1 12 14 4 1 2 3 4 5 6 7 8 9 number of models 16 1 2 3 4 5 6 7 8 9 number of models 35 3 c ALA-DJF large bias small bias 1 d MED-JJA 25 2 15 2 3 4 large bias small bias 1 1 2 3 4 5 6 7 8 9 number of models 5 1 2 3 4 5 6 7 8 9 number of models Fig. 3 Ensemble average change over sub-sets of models for the four regional/seasonal cases of Fig. 2. For each point in the graphs, the change reported in the y-axis is calculated by averaging over the corresponding number of n models in the x-axis. Two curves of points are shown. In the first ( large bias and asterisks) for each value of n the average is taken over the n models with the largest absolute value of bias, or the n worst performing models; in the second ( small bias and circles)the same calculations are performed but for the best performing models implies that all these models produce similar changes. On the other hand when models with large bias are used, the mean change depends strongly on the number of models and converges towards the best model average for increasing values of n. A different behavior is found in the case of ALA-DJF (Fig. 3c), for which the large and small bias sub-ensembles tend to converge towards different mean change values ( 2% vs. 15%). A separation of the large and small bias sub-ensembles appears also for the SAU-JJA case (Fig. 3b). Finally, as expected from the weak change bias relationship, in the MED-JJA case (Fig. 3d) the large and small bias averages quickly converge after n = 5. Two basic considerations can thus be drawn from the examples of Fig. 3 (which are illustrative also of the behavior over other regions). First, when a significant correlation across models is found between bias and change, and the biases are mostly of the same sign, a performance-based selection of models can substantially affect the change calculated from the sub-ensemble of models. Second, in all cases a sub-set of at least four to six models is needed to obtain a stable estimate of mean change.

794 Climatic Change (21) 1:787 795 4 Concluding remarks This paper presents an analysis of the dependence of the model simulated GHGinduced regional changes in temperature and precipitation on the underlying model regional biases. The CMIP3 ensemble of models is examined for the A1B scenario over 26 land regions of sub-continental size. A first conclusion is that for temperature the bias change relationship is essentially irrelevant, i.e. regional temperature changes simulated by the models essentially do not depend on corresponding regional biases, but on the model global temperature change (Giorgi 28). In these cases a local performance-based ensemble approach, such as that proposed by Giorgi and Mearns (22) or Tebaldi et al. (25) might not provide substantial added value. For precipitation we find a statistically significant bias change correlation across models in about 3% of the seasonal/regional cases. The cases are varied in terms of climatic regimes, ranging from high latitude winter climates to tropical climates, and therefore different mechanisms are likely at play by which the model bias might affect the projected change. Such mechanisms would need to be addressed on a caseby-case basis. For such cases, the performance of models used for the generation of climate change scenarios is relevant and, if the underlying mechanisms are understood, a performance-based ensemble method might provide an important, if not necessary, approach for improving the generation and interpretation of precipitation change scenarios. In addition, the selection of sub-sets of models, e.g. for regional model nesting or use in impact assessment studies, needs to be done carefully, since models characterized by small and large biases can produce substantially different results. Despite this conclusion, overall this analysis indicates that for the present generation of global climate models the model regional bias is not a dominant factor in determining the projected regional change for the majority of cases. For example, the decrease in precipitation projected by models over the Mediterranean region is largely independent of the model bias over this region, and this conclusion applies to other cases as well. It should be stressed that although our analysis is only based on regional precipitation, this variable is an integrator of multiple factors acting on multiple scales. For example, sub-continental scale biases are ultimately determined by a model s ability to simulate large scale circulation features, such as storm tracks or teleconnection patterns. It is therefore a remarkable feature of present day models that over some regions similar change patterns are produced even if the models are characterized by very different biases over the region. It remains to be assessed whether this represents an element of robustness or weakness in the regional climate change patterns produced by present models, and this requires an understanding of related processes. As the models increase in complexity and resolution it will be insightful to investigate whether the bias change relationship investigated here will strengthen or weaken in the next generation of climate change projections. Acknowledgments We acknowledge the modeling groups, the Program for Climate Model Diagnosis and Intercomparison (PCMDI) and the WCRP s Working Group on Coupled Modelling (WGCM) for their roles in making available the WCRP CMIP3 multi-model dataset. Support of this dataset is provided by the Office of Science, U.S. Department of Energy. We also thank two anonymous reviewers for their helpful comments and suggestions.

Climatic Change (21) 1:787 795 795 References Buser CM, Kunsch HR, Luthi D, Wild M, Schär C (29) Bayesian multi-model projection of climate: bias assumptions and interannual variability. Clim Dyn 33:849 868 Déqué M et al (25) Global high resolution vs. regional climate model climate change scenarios over Europe: quantifying confidence level from PRUDENCE results. Clim Dyn 25:653 67 Giorgi F (25) Climate change prediction. Clim Change 73:239 265 Giorgi F (28) A simple equation for regional climate change and associated uncertainties. J Climate 21:1589 164 Giorgi F, Bi X (25) Updated regional precipitation and temperature changes for the 21st century from ensembles of recent AOGCM simulations. Geophys Res Lett 32:L21715 Giorgi F, Francisco R (2) Uncertainties in regional climate change predictions. A regional analysis of ensemble simulations with the HADCM2 GCM. Clim Dyn 16:169 182 Giorgi F, Lionello P (28) Climate change projections for the Mediterranean region. Glob Planet Change 63:9 14 Giorgi F, Mearns LO (22) Calculation of average, uncertainty range and reliability of regional climate changes from AOGCM simulations via the reliability ensemble averaging (REA) method. J Climate 15:1141 1158 Greene A, Goddard L, Lall U (26) Probabilistic multi-model regional temperature change projections. J Climate 19:4326 4343 Intergovernmental Panel on Climate Change (IPCC) (2) Special report on emission scenarios. In: Nakicenovic N et al (eds) Cambridge University Press, New York, p 599 Kittel TGF, Giorgi F, Meehl GA (1998) Intercomparison of regional biases and doubled CO 2 - sensitivity of coupled atmosphere ocean general circulation model experiments. Clim Dyn 14:1 15 Meehl GA et al (27) The WCRP CMIP3 multimodel ensemble a new era in climate change research. Bull Am Meteorol Soc 88:1383 1394 Murphy JM et al (24) Quantification of modeling uncertainties in a large ensemble of climate change simulations. Nature 429:768 772 New M, Hulme M, Jones PD et al (2) Representing twentieth century space time climate variability. Part II: development of 191 1996 monthly grids of terrestrial surface climate. J Climate 13:2217 2238 Raisanen J (27) How reliable are climate models? Tellus A 59:2 29 Raisanen J, Ruolokainen L, Ylhaisi J (21) Weighting model results for improving best estimates of climate change. Clim Dyn. doi:1.17/s382-9-659-8 Shukla J, DelSole T, Fennesy M, Kinter J, Paolino D (26) Climate model fidelity and projections of climate change. Geophys Res Lett 33:L772 Spelman MJ, Manabe S (1984) Influence of oceanic heat transport upon the sensitivity of a model climate. J Geophys Res 89:571 586 Tebaldi C, Knutti R (27) The use of the multi-model ensemble in probabilistic climate projections. Philos Trans R Soc Lond A 365:253 275 Tebaldi C, Smith R, Nychka D, Mearns LO (25) Quantifying uncertainties in projections of regional climate change: a Bayesian approach to the analysis of multi-model ensembles. J Climate 18:1524 154 Whetton P, Macadam I, Bathols J, O Grady J (27) Assessment of the use of current climate patterns to evaluate regional enhanced greenhouse response patterns of climate models. Geophys Res Lett 34:L1471