On the clustering of climate models in ensemble seasonal forecasting

Transcription

1 GEOPHYSICAL RESEARCH LETTERS, VOL. 39,, doi: /2012gl052735, 2012 On the clustering of climate models in ensemble seasonal forecasting Xing Yuan 1 and Eric F. Wood 1 Received 13 June 2012; revised 7 August 2012; accepted 8 August 2012; published 19 September [1] Multi-model ensemble seasonal forecasting system has expanded in recent years, with a dozen coupled climate models around the world being used to produce hindcasts or real-time forecasts. However, many models are sharing similar atmospheric or oceanic components which may result in similar forecasts. This raises questions of whether the ensemble is over-confident if we treat each model equally, or whether we can obtain an effective subset of models that can retain predictability and skill as well. In this study, we use a hierarchical clustering method based on inverse trigonometric cosine function of the anomaly correlation of pairwise model hindcasts to measure the similarities among twelve American and European seasonal forecast models. Though similarities are found between models sharing the same atmospheric component, different versions of models from the same center sometimes produce quite different temperature forecasts, which indicate that detailed physics packages such as radiation and land surface schemes need to be analyzed in interpreting the clustering result. Uncertainties in clustering for different forecast lead times also make reducing redundant models more complicated. Predictability analysis shows that multi-model ensemble is not necessarily better than a single model, while the cluster ensemble shows consistent improvement against individual models. The eight model-based cluster ensemble forecast shows comparable performance to the total twelve model ensemble in terms of probabilistic forecast skill for accuracy and discrimination. This study also manifests that models developed in U.S. and Europe are more independent from each other, suggesting the necessity of international collaboration in enhancing multi-model ensemble seasonal forecasting. Citation: Yuan, X., and E. F. Wood (2012), On the clustering of climate models in ensemble seasonal forecasting, Geophys. Res. Lett., 39,, doi: /2012gl Introduction [2] The concept of seasonal forecast came from Charney and Shukla s [1981] discussion on the contribution of variability over low and mid-latitudes. They concluded that short-term flow instabilities account for most of the variability over mid-latitudes, while boundary anomalies in quantities such as sea surface temperature (SST) contribute to a large part of the low-latitude variability. As surmised by 1 Department of Civil and Environmental Engineering, Princeton University, Princeton, New Jersey, USA. Corresponding author: X. Yuan, Department of Civil and Environmental Engineering, Princeton University, E318 Engineering Quad, Princeton, NJ 08544, USA. (xingy@princeton.edu) American Geophysical Union. All Rights Reserved /12/2012GL Charney and Shukla [1981], the main predictability of climate system in term of seasonal forecast resides in the ocean and land, where SST, snow and soil moisture evolves slower than the atmosphere, which impart additional memory to the climate system [Koster et al., 2010; Slingo and Palmer, 2011]. However, even the ocean is a chaotic and nonlinear system at seasonal to decadal time scales. For instance, the initiation of El Niño in 2003 is difficult to forecast by the coupled atmosphere-ocean-land general circulation models (CGCMs) due to stochastic forcing from the atmosphere [Slingo and Palmer, 2011]. To improve the predictability and quantify the uncertainty, ensemble forecast seems to be an effective approach. The operational climate forecast centers are generating ensembles from realizations with different initial conditions, either by perturbing wind stress and SST [Weisheimer et al., 2009], or running the model with different start dates [Saha et al., 2006]. Nevertheless, single modelbased ensembles tend to have under-dispersion errors due to the model-specific biases [Slingo and Palmer, 2011]. Therefore, multi-model ensembles that allow the forecast space to be sampled more completely, are receiving more attentions over last decade [Krishnamurti et al., 1999; Barnston et al., 2003; Palmer et al., 2004]. The multi-model superiority lies in its enhanced reliability [Hagedorn et al., 2005] and spatially complementary feature [Yuan et al., 2011]. [3] Multi-model ensemble seasonal forecasts are firstly made by utilizing all available models at the time of forecasts, where each model has several realizations from different initial conditions, which is called the ensemble of opportunity. Recently, owing to the increased computing capability and enhanced national/international collaboration, a number of operational forecast centers are producing multi-model ensemble seasonal forecasting, such as European Operational Seasonal to Interannual Prediction (EUROSIP) and National Multi-Model Ensemble (NMME) [Zhang et al., 2011]. More than 30-year hindcasts from a dozen CGCMs are being made available to facilitate skill evaluation [Yuan et al., 2011], model diagnosis [DelSole and Shukla, 2012] and real-time forecast [Wang et al., 2010]. This also provides an opportunity for assessing similarity or dependence in the model space, which is important to avoid over-confident forecasts, especially if some models share the same components or use modules from the same institution (e.g., six of seven NMME models use the ocean models developed at NOAA s Geophysical Fluid Dynamics Laboratory (GFDL)). In fact, a number of studies have assessed model similarity or estimated the effective number of climate models in the context of climate change projection [Knutti, 2010; Masson and Knutti, 2011; Pennell and Reichler, 2011]. To date, more and more scientists are aware of the fact that the ensemble of opportunity (availability) by ignoring inter-model relationship is not justified, and that clustering of climate models 1of7

2 seems to be necessary. However, how does such clustering affect predictability and skill of ensemble prediction? Though verifications for climate change projection are not available, it can be tested for seasonal hindcasts, and its consistency and stability in terms of forecasts with different lead times can also be investigated. [4] In this study, a hierarchical clustering method is applied to the distance matrix of pairwise seasonal precipitation and surface air temperature forecasted by climate models from the ENSEMBLES project [Weisheimer et al., 2009] and the newly established NMME system [Zhang et al., 2011]. The predictability of ensemble mean, and probabilistic forecasting skill in terms of accuracy and discrimination, are evaluated based on the 24-year hindcast dataset. 2. Observation and Hindcast Data [5] The observed precipitation data is from Climate Prediction Center (CPC) Unified Gauge-Based Analysis for the period [Chen et al., 2008], and surface air temperature is from Climate Research Unit (CRU) TS3.1 dataset for [Mitchell and Jones, 2005]. We aggregate them from daily/monthly mean to seasonal mean values, and from half degree to one degree. The aggregations are done by only using observation over land grids. [6] The ENSEMBLES project, as described in Weisheimer et al. [2009], is a seasonal-to-annual multi-model ensemble forecast system containing five models each with nine ensemble members. The five models are from the Euro- Mediterranean Centre for Climate Change (CMCC-INGV), the European Centre for Medium-Range Weather Forecasts (ECMWF), the Leibniz Institute of Marine Sciences at Kiel University (IFM-GEOMAR), Météo France, and UK Met Office (UKMO). The available dataset is at resolution, and the hindcast period covers 46 years ( ), with 7-month forecasts starting on the 1st of February, May, August and November of each year. We bilinearly interpolate precipitation and surface air temperature hindcasts to one degree, and aggregate them to seasonal mean values. [7] The NMME is a collaborative prediction system that is being developed through the National Centers for Environmental Prediction (NCEP) Climate Test Bed (CTB) for experimental monthly and seasonal prediction at CPC [Zhang et al., 2011]. Currently, seven models from University of Miami (COLA-RSMAS-CCSM3 [Kirtman and Min, 2009]), GFDL (GFDL-CM2.1 [Delworth et al., 2006]), International Research Institute for Climate and Society (IRI-ECHAM4.5- AC and IRI-ECHAM4.5-DC [DeWitt, 2005]), National Aeronautics and Space Administration (NASA-GMAO [Molod et al., 2012]) and NCEP (NCEP-CFSv1 and NCEP-CFSv2 [Saha et al., 2006, 2010]) are participating. During the phase I project, monthly mean precipitation, SST and surface air temperature at one degree are being archived by IRI ( iridl.ldeo.columbia.edu/sources/.models/.nmme/). There are 29-year ( ) hindcast data, with ensemble members of each model varying from 6 to 24, and forecasts up to 8 to 12 months. Again, monthly mean values have been aggregated to seasonal means. [8] The overlap period among observation, ENSEMBLES and NMME is To be consistent with ENSEM- BLES models, only the forecasts starting from the 1st of February, May, August and November are being considered, though some ensemble members from NASA-GMAO, NCEP-CFSv1 & v2 are around 0.5 month old at the beginning of those months. There are four seasonal hindcasts that cover all calendar month in each year for each forecast lead. For instance, for the 1 month lead, we will have seasonal mean hindcasts for MAM, JJA, SON and DJF. The data from observation and models used in this study are over land grids. 3. Clustering of Seasonal Forecast Models [9] To quantify the distance between hindcasts of two models, we use inverse trigonometric cosine function of the anomaly correlation (AC), which is non-negative and increases as the similarity increases. AC is a widely used measure of association that operates on pairs of grid point values in the forecast and observed fields [Wilks, 2011]. Here we use a slightly different form of AC as defined by Saha et al. [2006], which considers the correlation both in space and time. And we calculate the AC between ensemble mean of hindcasts from two models instead of between observation and hindcast. We use seasonal anomaly time series throughout years where climatology has been removed for each season separately, so there are 96 forecast cases (4 seasons in 24 years) at each grid point in this study. A hierarchical clustering [Wilks, 2011] is applied to the distance matrix of pairwise model dissimilarities (cosine of AC) to generate a tree diagram. [10] Figure 1 shows hierarchical clustering of twelve seasonal forecast models for precipitation and surface air temperature at different lead times. Consistent with previous studies on Coupled Model Intercomparison Project Phase 3 (CMIP3) models [Masson and Knutti, 2011; Pennell and Reichler, 2011], the models from the same institution (e.g., IRI) are very similar as measured by AC, regardless of different variables or different forecast lead times. For precipitation (Figures 1a, 1c, and 1e), similarities are also strong between models sharing similar atmospheric components (e.g., IRI-ECHAM4.5-AC, IRI-ECHAM4.5-DC, IFM-GEOMAR and CMCC-INGV are using ECHAM as the atmospheric module) or between successive versions of the same model (e.g., CFSv1 and CFSv2). However, it becomes more complicated for surface air temperature. For instance, NASA-GMAO and NCEP-CFSv2 have strong similarity in temperature, while CFSv1 and CFSv2 diverge greatly in the tree plot (Figures 1b and 1d), which might be due to the fact that the forecasted surface air temperature is highly dependent on radiation and land surface modules. Although CFSv2 is a successive version of CFSv1, they use quite different radiation and land surface packages. [11] Besides the difference between precipitation and temperature, there are also some uncertainties in clustering in terms of forecast lead time. For example, COLA-RSMAS- CCSM3 has the strongest dissimilarity with other models for 4 month lead forecast (Figures 1c and 1d), while it has high correlations with others at 0 month lead (Figures 1a and 1b). Given that clustering of climate models for 0 month lead forecast might be affected by initial conditions, we select models for further evaluation based on atmospheric and oceanic components, and the clustering results for 4 month lead hindcasts. For precipitation, IRI-ECHAM4.5-DC, NCEP- CFSv2, ECMWF, Météo France, NASA-GMAO, GFDL- 2of7

3 Figure 1. Hierarchical clustering of twelve seasonal forecast models for seasonal mean precipitation at (a) 0 month and (c) 4 month lead, and surface air temperature at (b) 0 month and (d) 4 month lead. (e) Atmospheric and oceanic components of each model in Figure 1c. The same color in each forecast case represents that the two models are quite similar. CM2.1, UKMO and COLA-RSMAS-CCSM3 are selected to form an eight-model ensemble called cluster ensemble. NCEP-CFSv1 is not selected due to its similarity with other top four models in Figure 1c, and its similarity with NCEP- CFSv2 in the physics (Figure 1e). For surface air temperature, being as consistent with precipitation as possible, we only replace NASA-GMAO with NCEP-CFSv1 to form the cluster ensemble, though CMCC-INGV also shows some independency (Figure 1d). Therefore, the cluster ensemble in this study consists of three ENSEMBLES models that form the EUROSIP system and five NMME models. All models in the multi-model ensembles are weighted equally, so the models with more ensemble members will have less weight for each member in the following sections. 4. Predictability of Ensemble Mean [12] In this study, predictability of ensemble mean is measured by the square of correlation (R 2 ), i.e., the coefficient of determination. Model forecasted seasonal mean precipitation and surface air temperature in each year during are compared to the corresponding observation by plotting a scatter plot and then calculating R 2. The R 2 values represent the fraction of variance explained by the forecasts, and are computed for each grid cell, each season and each forecast lead time. All negative R 2 values are set to zero to avoid noise [Koster et al., 2010], and all arid regions (annual precipitation less than 50 mm) are masked out in the precipitation analysis. [13] Figure 2 shows the spatial distribution of R 2 averaged over four seasons for seasonal mean precipitation and surface air temperature at different lead times. We compare the predictability of CFSv2 with other three multi-model ensembles: NMME (seven models), combining NMME and ENSEMBLES (twelve models, referred to as N + E), and cluster ensemble (eight models selected in the above section). For precipitation forecasts at 0 month lead, NMME increases predictability from CFSv2 over Australia, northern China, northeastern Brazil, and part of North American Monsoon region (Figures 2a and 2b). However, NMME degrades predictability over Russia, western Asia, northern Europe and central Africa, and finally has 2% (though not significant) less grid cells with predictability larger than 0.05 (Figures 2a and 2b). The advantage of multi-model becomes more consistent for long lead precipitation forecasts, where NMME is slightly better than CFSv2 over western U.S., Brazil and Australia (Figures 2e and 2f). Combing NMME and ENSEMBLES (N + E) not only retains the multimodel s advantage at long lead time (Figure 2g), but also increases grid cells with significant predictability for 0 month lead forecast by about 9% as compared with CFSv2 (Figure 2c). Similar to Yuan et al. [2011], the improvement of N + E comes from complementary feature in space for seasonal forecast models developed in U.S. and Europe. For instance, ECMWF and UKMO have higher predictability than all seven American models over southern Africa and western Australia (not shown). Figures 2d and 2h demonstrates that reducing four models from N + E by using the clustering method does not affect (degrade) the predictability of precipitation for both short and long term forecasts. This verifies the usefulness of the clustering method in selecting independent (effective) seasonal forecast models, while retaining the predictability that is consistently beyond single models. [14] The picture becomes clearer with the predictability of surface air temperature (Figures 2i 2p). Due to obvious improvement (upgrade in radiation and land surface modules) of CFSv2 in predicting temperature over high latitudes of the northern hemisphere [Yuan et al., 2011], NMME has lower predictability than CFSv2 even out to 4 6 months (Figures 2i, 2j, 2m, and 2n). In contrast, N + E and cluster ensemble outperforms CFSv2 globally (Figures 2k, 2l, 2o, and 2p). A common feature for multi-model ensemble forecasted temperature at long lead time (Figures 2n 2p) is that they have strong predictability over western U.S. and western China, while negligible predictability over the eastern parts of these two countries. The reasons might be threefold: 1) eastern parts of U.S. and China are wetter than 3of7

4 Figure 2. Predictability (squared correlation R 2 ) of seasonal mean precipitation and surface air temperature averaged over four seasons for CFSv2 and multi-model ensembles at different lead times. The numerals in the brackets are number of models, and the annotated numbers in lower left part of each panel are percentage of grid cells with R 2 larger than their western parts, indicating that there may be more cloud that affect accuracy of radiation modeling in the east regions; 2) snow processes, which are important to seasonal forecasts, are more dominant contributions to the predictability over western parts of these two countries; and 3) the western parts are less densely observed; while the eastern part, though densely observed, might be polluted by urban-island effect, which also results in more uncertainty from observation and hence propagate into predictability analysis. 5. Verification of Probabilistic Forecast Skill [15] Besides assessing the predictability of ensemble means, we use continuous ranked probability skill score (CRPSS) [Wilks, 2011] and Relative Operating Characteristic skill score (ROCSS) [Wilks, 2011] to verify probabilistic forecast in terms of accuracy and discrimination, respectively. CRPSS is defined as 1-CRPS/CRPS Clim, where CRPS and CRPS Clim are continuous ranked probability scores (here calculated by comparing deciles of the forecast distribution with observations) from the model and the climatological forecast, respectively. ROCSS is defined as 2A-1, where A is the area under ROC curve. For both scores, positive values represent that climate model forecasts are more skillful than climatological forecasts, with a maximum of 1 for a perfect forecast. To reduce the impact of bias, climatological means of each season for both model and observation have been removed before calculating skill scores. [16] Figures 3a 3d show the percentages of climate model forecasts that are more skillful than reference forecasts (CRPSS > 0) for seasonal mean precipitation and surface air temperature anomalies over global land area (except Antarctic) during Unsurprisingly, both individual climate models and multi-model ensembles have higher skill in forecasting temperature than precipitation. Though CFSv2 is the most skillful model as compared to other individual models in NMME and ENSEMBLES, multimodel ensembles do obtain additional credit. Specifically, NMME increases percentage of skillful precipitation forecasts from CFSv2 by about 8 9%, and N + E increases it by about 11 12%. This is consistent with previous studies that more models will have greater accuracy and reliability in terms of probabilistic forecasts, although they do not necessarily increase the predictability as we discussed in the above section. For the seasonal surface air temperature forecast, multi-model ensembles have less advantage beyond CFSv2 than for precipitation, where NMME and N + E increases the percentage by 3 4% and 7 9%, respectively. Figures 3a 3d also show that the cluster ensemble has comparable performance to N + E. Therefore, although incorporating more climate models enhances accuracy (e.g., from NMME to N + E), there is an effective subset of models (e.g., the clustered models in this study) that beyond which the skill enhancement diminishes. In addition, the verification rank histograms averaged over the global land area indicate that the characteristics of cluster ensemble and N + E are quite similar (not shown). [17] Another important attribute of forecast performance is discrimination, which reflects the ability of the forecasting system to produce different forecasts for those occasions having different realized outcomes of the predictand [Wilks, 2011]. Here we examine the models discrimination on extreme seasonal temperature anomalies, the precipitation anomaly will not be discussed due to low skill. Figure 3e shows the ROCSS as functions of forecast lead times for surface air temperature anomaly 90th percentile over global land areas. Though there are some chaotic features that forecast discrimination skill of long lead times from some individual models is not necessarily lower than those of short lead times, multi-model ensembles do present obvious decreasing skill over lead times. As compared to CFSv2, NMME has no advantage in discriminating 90th percentile 4of7

5 Figure 3. (a d) Percentages of skillful forecasts (CRPSS > 0) for seasonal mean precipitation and surface air temperature anomalies and (e) Relative Operating Characteristic skill scores (ROCSS) as functions of forecast lead times for seasonal mean surface air temperature anomaly 90th percentile over global land areas. N1 N6 are individual models from NMME, and E1 E5 are from ENSEMBLES. of temperature anomalies at 0 month lead, while its advantage emerges and increases as forecast lead increases. N + E is overall superior to NMME, especially for short lead forecasts. Again, cluster ensemble has similar performance to N + E, emphasizing that clustering of climate models through analyzing covariance structure of ensemble means is an effective method to select independent model as well as to retain ensemble forecast skill. 6. Conclusions and Discussions [18] This study analyzes the similarities among different seasonal forecast models by using a hierarchical clustering method, and evaluates the predictability and skill of the cluster ensemble forecasts. Consistent with previous studies on CMIP3 models [Masson and Knutti, 2011; Pennell and Reichler, 2011], strong similarities are found between models from the same institution or sharing the same atmospheric component; however, the similarity between successive versions of the same model are different for precipitation and temperature forecasts: NCEP-CFSv1 & v2 present some similarity in terms of precipitation forecasts, while they are quite different for temperature forecasts. Such difference indicates that detailed physics packages such as radiation and land surface schemes need to be checked when interpreting the clustering results. [19] Predictability analysis shows that NMME multimodel ensemble is not necessarily better than a single model such as CFSv2, especially for the forecasts at short lead time. However, combining NMME and ENSEMBLE (N + E) results in consistent improvement against CFSv2, which suggests that seasonal forecast models developed in U.S. and Europe complement each other to some extent, and more international collaboration is necessary to enhance predictability of climate models for seasonal forecasting. The multi-model ensemble from clustering, though excluding four models, has comparable performance to N + E in terms of predictability, continuous ranked probability skill score, and a discrimination score for seasonal temperature anomaly 90th percentile. This verifies that clustering 5of7

6 method proposed in this study not only reduces redundancy of climate models, but also retains ensemble forecast skill. [20] This paper also provides some implications for the clustering of climate models in seasonal forecasting. First, some uncertainties exist in clustering in terms of forecast lead time and the fields selected for clustering, which make reducing redundant seasonal forecast models more complicated. Second, all multi-model ensembles including cluster ensemble in this study have negligible predictability for surface air temperature forecasts at long lead time over eastern parts of U.S. and China (Figures 2n 2p), actually such deficiency resides in all twelve individual models (not shown). Whether the common deficiency is related to cloudradiation modeling is an unknown issue that needs further investigation. One possibility is that the state-of-art climate forecast models may have common shortcomings due to similar parameterizations or universal use of the deterministic truncations [Palmer, 2012]. Actually Weisheimer et al. [2011] find that the ensemble based on stochastic parameterization of unresolved atmospheric processes outperforms the multi-model ensemble for precipitation forecasts, and Palmer [2012] suggests that development of such stochastic approach for the land surface and ocean may benefit temperature ensemble forecasts. We believe that a new era for ensemble forecast will begin if the stochastic parameterization could be incorporated into the multi-model ensemble framework. Third, hierarchical clustering is a method to select independent models, thus we get 0/1 weights for different models, which might not be optimal. Other calibration methods such as conditional distribution, Bayesian approach or canonical correlation analysis can be used to implicitly or explicitly weight different models. However, as pointed out by Doblas-Reyes et al. [2005], long hindcast time series are needed to achieve a significant forecast quality improvement. Actually we have used the theory of conditional distribution of multivariate normal distributions to merge different models through cross validation procedure based on 24-year hindcast data. We find that the covariance matrices, which measure the correlation between observation and model, or the cross correlation among different models, are quite different year by year. Consequently, the distribution conditioned on multiple single models, where the weights are dependent on correlation between single model and observation, is less robust than the distribution conditioned on multi-model ensemble mean (e.g., equal weights). However, statistical calibration by using conditional distribution is also less beneficial for the multi-model ensemble mean than for the single models, perhaps due to higher reliability of the former. Furthermore, statistical calibration using hindcasts is challenging since it is based on the assumption that the current climate is stationary, which is not the case because the climate is changing due to the increase in CO2 that affects temperature [Slingo and Palmer, 2011]. Perhaps dynamical downscaling seasonal forecasts would be more physically consistent with climate change. Nevertheless, Yuan et al. [2012] shows that regional climate model (WRF) can improve forecasts of interannual variations only if the driving CGCM has the correct anomaly signal. Therefore, developing advanced global climate forecast models with greater independences in the physical modules will benefit ensemble seasonal forecasting. Also, long period hindcasts are critical to reasonably identifying the contribution of particular models to the multi-model ensemble by addressing issues like natural and anthropogenic climate variability. Finally, clustering is an effective method to reduce model redundancy while retains the predictability and skill. [21] Acknowledgments. We would like to thank IRI to make the NMME forecast information available. We thank two anonymous reviewers for their helpful comments. The research presented in the paper was supported by the NOAA Climate Program Office through grants NA17RJ2612, NA12OAR , and NA10OAR [22] The Editor thanks the two anonymous reviewers for assisting in the evaluation of this paper. References Barnston, A. G., S. J. Mason, L. Goddard, D. G. DeWitt, and S. E. Zebiak (2003), Multimodel ensembling in seasonal climate forecasting at IRI, Bull. Am. Meteorol. Soc., 84, , doi: /bams Charney, J. G., and J. Shukla (1981), Predictability of monsoons, in Monsoon Dynamics, edited by J. Lighthill and R. P. Pearce, pp , Cambridge Univ. Press, Cambridge, U. K., doi: /cbo Chen, M., W. Shi, P. Xie, V. B. S. Silva, V. E. Kousky, R. W. Higgins, and J. E. Janowiak (2008), Assessing objective techniques for gauge-based analyses of global daily precipitation, J. Geophys. Res., 113, D04110, doi: /2007jd DelSole, T., and J. Shukla (2012), Climate models produce skillful predictions of Indian summer monsoon rainfall, Geophys. Res. Lett., 39, L09703, doi: /2012gl Delworth, T. L., et al. (2006), GFDL s CM2 Global Coupled Climate Models. Part I: Formulation and simulation characteristics, J. Clim., 19(5), , doi: /jcli DeWitt, D. G. (2005), Retrospective forecasts of interannual sea surface temperature anomalies from 1982 to present using a directly coupled atmosphere-ocean general circulation model, Mon. Weather Rev., 133, , doi: /mwr Doblas-Reyes, F. J., R. Hagedorn, and T. N. Palmer (2005), The rationale behind the success of multi-model ensembles in seasonal forecasting II. Calibration and combination, Tellus, Ser. A, 57, , doi: / j x. Hagedorn, R., F. J. Doblas-Reyes, and T. N. Palmer (2005), The rationale behind the success of multi-model ensemble in seasonal forecasting I. Basic concept, Tellus, Ser. A, 57, , doi: /j x. Kirtman, B. P., and D. Min (2009), Multimodel ensemble ENSO prediction with CCSM and CFS, Mon. Weather Rev., 137(9), , doi: /2009mwr Knutti, R. (2010), The end of model democracy?, Clim. Change, 102(3 4), , doi: /s Koster, R. D., et al. (2010), Contribution of land surface initialization to subseasonal forecast skill: First results from a multi-model experiment, Geophys. Res. Lett., 37, L02402, doi: /2009gl Krishnamurti, T. N., et al. (1999), Improved weather and seasonal climate forecasts from multimodel superensemble, Science, 285, , doi: /science Masson, D., and R. Knutti (2011), Climate model genealogy, Geophys. Res. Lett., 38, L08703, doi: /2011gl Mitchell, T. D., and P. D. Jones (2005), An improved method of constructing a database of monthly climate observations and associated high-resolution grids, Int. J. Climatol., 25(6), , doi: /joc Molod, A., L. Takacs, M. Suarez, J. Bacmeister, I.-S. Song, and A. Eichmann (2012), The GEOS-5 atmospheric general circulation model: Mean climate and development from MERRA to Fortuna, Tech Rep. Ser. Global Model. Data Assimilation 28, 175 pp., NASA Goddard Space Flight Cent., Greenbelt, Md. Palmer, T. N. (2012), Towards the probabilistic Earth-system simulator: A vision for the future of climate and weather prediction, Q. J. R. Meteorol. Soc., 138, , doi: /qj Palmer, T. N., et al. (2004), Development of a European Multimodel Ensemble System for Seasonal-to-Interannual Prediction (DEMETER), Bull. Am. Meteorol. Soc., 85(6), , doi: /bams Pennell, C., and T. Reichler (2011), On the effective number of climate models, J. Clim., 24, , doi: /2010jcli Saha, S., et al. (2006), The NCEP Climate Forecast System, J. Clim., 19(15), , doi: /jcli Saha, S., et al. (2010), The NCEP Climate Forecast System reanalysis, Bull. Am. Meteorol. Soc., 91(8), , doi: /2010bams Slingo, J., and T. Palmer (2011), Uncertainty in weather and climate prediction, Philos. Trans. R. Soc. A., 369, , doi: / rsta of7

7 Wang, W., M. Chen, and A. Kumar (2010), An assessment of the CFS realtime seasonal forecasts, Weather Forecast., 25(3), , doi: / 2010WAF Weisheimer, A., et al. (2009), ENSEMBLES: A new multi-model ensemble for seasonal-to-annual predictions Skill and progress beyond DEME- TER in forecasting tropical Pacific SSTs, Geophys. Res. Lett., 36, L21711, doi: /2009gl Weisheimer, A., T. N. Palmer, and F. Doblas-Reyes (2011), Assessment of representations of model uncertainty in monthly and seasonal forecast ensemble, Geophys. Res. Lett., 38, L16703, doi: /2011gl Wilks, D. S. (2011), Statistical Methods in the Atmospheric Sciences, Int. Geophys. Ser., vol. 100, 3rd ed., 676 pp., Academic, San Diego, Calif. Yuan, X., E. F. Wood, L. Luo, and M. Pan (2011), A first look at Climate Forecast System version 2 (CFSv2) for hydrological seasonal prediction, Geophys. Res. Lett., 38, L13402, doi: /2011gl Yuan, X., X.-Z. Liang, and E. F. Wood (2012), WRF ensemble downscaling seasonal forecasts of China winter precipitation during , Clim. Dyn., doi: /s , in press. Zhang, Q., et al. (2011), Preliminary evaluation of multi-model ensemble system for monthly and seasonal prediction, in Science and Technology Infusion Climate Bulletin, pp , NOAA, Silver Spring, Md. 7of7