WEATHER AND SEASONAL CLIMATE PREDICTION FOR SOUTH AMERICA USING A MULTI-MODEL SUPERENSEMBLE

Transcription

1 INTERNATIONAL JOURNAL OF CLIMATOLOGY Int. J. Climatol. 5: (5) Published online 7 October 5 in Wiley InterScience ( DOI: 1./joc. WEATHER AND SEASONAL CLIMATE PREDICTION FOR SOUTH AMERICA USING A MULTI-MODEL SUPERENSEMBLE ROSANE R. CHAVES, ROBERT S. ROSS* and T. N. KRISHNAMURTI Department of Meteorology, The Florida State University, Tallahassee, Florida 6-5, USA Received 15 December Revised April 5 Accepted 7 June 5 ABSTRACT This work examines the feasibility of weather and seasonal climate predictions for South America using the multi-model synthetic superensemble approach for climate, and the multi-model conventional superensemble approach for numerical weather prediction, both developed at Florida State University (FSU). The effect on seasonal climate forecasts of the number of models used in the synthetic superensemble is investigated. It is shown that the synthetic superensemble approach for climate and the conventional superensemble approach for numerical weather prediction can reduce the errors over South America in seasonal climate prediction and numerical weather prediction. For climate prediction, a suite of 1 models is used. The forecast lead-time is 1 month for the climate forecasts, which consist of precipitation and surface temperature forecasts. The multi-model ensemble is comprised of four versions of the FSU-Coupled Ocean-Atmosphere Model, seven models from the Development of a European Multi-model Ensemble System for Seasonal to Interannual Prediction (DEMETER), a version of the Community Climate Model (CCM), and a version of the predictive Ocean Atmosphere Model for Australia (POAMA). The results show that conditions over South America are appropriately simulated by the Florida State University Synthetic Superensemble (FSUSSE) in comparison to observations and that the skill of this approach increases with the use of additional models in the ensemble. When compared to observations, the forecasts are generally better than those from both a single climate model and the multi-model ensemble mean, for the variables tested in this study. For numerical weather prediction, the conventional Florida State University Superensemble (FSUSE) is used to predict the mass and motion fields over South America. Predictions of mean sea level pressure, 5 hpa geopotential height, and 5 hpa wind are made with a multi-model superensemble comprised of six global models for the period January, February, and December of. The six global models are from the following forecast centers: FSU, Bureau of Meteorology Research Center (BMRC), Japan Meteorological Agency (JMA), National Centers for Environmental Prediction (NCEP), Naval Research Laboratory (NRL), and Recherche en Prevision Numerique (RPN). Predictions of precipitation are made for the period January, February, and December of 1 with a multi-analysis-multi-model superensemble where, in addition to the six forecast models just mentioned, five additional versions of the FSU model are used in the ensemble, each with a different initialization (analysis) based on different physical initialization procedures. On the basis of observations, the results show that the FSUSE provides the best forecasts of the mass and motion field variables to forecast day 5, when compared to both the models comprising the ensemble and the multi-model ensemble mean during the wet season of December February over South America. Individual case studies show that the FSUSE provides excellent predictions of rainfall for particular synoptic events to forecast day. Copyright 5 Royal Meteorological Society. KEY WORDS: South America; climate; weather; prediction; Superensemble; multi model 1. INTRODUCTION Weather and climate fluctuations over South America significantly impact important economic activities such as agriculture and hydroelectric power generation. One recent example of these impacts was the energy crisis caused by rainfall deficits during austral spring and summer 1 (hereafter the seasons are * Correspondence to: Robert S. Ross, Department of Meteorology, Florida State University, Tallahassee, FL 6-5, USA; e- mail: bross@coven.met.fsu.edu Copyright 5 Royal Meteorological Society

2 R. R. CHAVES, R. S. ROSS AND T. N. KRISHNAMURTI named relative to the Southern Hemisphere). There was a significant reduction in river flow throughout northeastern, west central and southeastern Brazil, thereby reducing the capacity for hydroelectric power generation in these areas. The occurrence of daily extreme precipitation events in South America also has large economic and social impacts. Floods are observed in association with strong South Atlantic Convergence Zone (SACZ) events, especially over southeastern Brazil (Carvalho et al., ). The SACZ is the most important meteorological feature of South America. It is characterized by one elongated convective band typically originating in the Amazon basin, and extending toward southeastern Brazil and the subtropical Atlantic Ocean. Its mean position passes over the most densely populated Brazilian states of São Paulo and Rio de Janeiro. During austral winter, when relatively dry conditions prevail over most of South America, the mid-latitude frontal systems can cause severe weather events over the southeastern and southern regions of the continent through freezing temperatures and frost (Pezza and Ambrizzi, 5). These weather variations can significantly affect the harvests of wheat, coffee, soybeans, and oranges in the agricultural lands of southeastern and southern South America (Marengo et al., 1997). Hence, in addition to being an exciting, tractable scientific problem, forecasting of weather and climate variability in South America has important human impacts. In most of South America, the rainy season is in the austral summer, December to February (DJF), with over 5% of the total annual precipitation occurring during this season (Rao and Hada, 199). Most of this precipitation is associated with the activation of the SACZ. In the northern portion of northeastern Brazil, the main rainy season is during March-April-May (MAM) and is associated with the southward displacement of the Intertropical Convergence Zone (ITCZ). The ITCZ is a zonally elongated atmospheric convergence zone, which lies at the junction of the northeast and southeast trade wind systems. The band of converging surface winds is a few degrees of latitude wide and is characterized by reduced wind speeds. The ITCZ is also closely associated with a band of convection that is an important source of rainfall for the northern portion of northeastern Brazil, as well as an important source of diabatic heating for the atmosphere in this region. The latitudinal position of the ITCZ in the Atlantic varies from a position close to the equator in boreal spring (March May) to a maximum extension of 1 N 15 N in late boreal summer (August). On an inter-annual timescale, the El Nino-Southern Oscillation (ENSO) is the most important coupled ocean atmosphere phenomenon to produce rainfall variability over South America (Ropelewski and Halpert, 197; Aceituno, 19). The influence of this phenomenon in the variability of precipitation has been widely studied with observations (e.g. Zhou and Lau, 1; Barros et al., ) and modeling (e.g. Schneider et al., 1997; Pezzi and Cavalcanti, 1). These studies show that La-Nina is associated with an increase in precipitation over the northern part of South America and a decrease in precipitation over the subtropical regions of South America and the coastal areas of Peru and Ecuador. El-Nino is associated with a decrease in precipitation over the northern part of South America and an increase in precipitation over subtropical regions of South America and coastal areas of Peru and Ecuador. Over southeastern Brazil, in the region of the SACZ, it appears that the ENSO does not significantly influence austral summertime precipitation (Ropelewski and Halpert, 197; Rao and Hada, 199). Predictions for the rainy season in the northern portion of northeastern Brazil have been carried out using various dynamical and statistical models (e.g. Heller and Hastenrath, 1977; Folland et al., 1; Pezzi and Cavalcanti, 1), and all of them have shown high predictability in that region, especially during anomalous years associated with El-Nino (Folland et al., 1). Other regions of South America have received less attention in this matter. Recently, Misra () evaluated the predictability of austral summertime (January to March) precipitation over South America using integrations of the Center for Ocean-Land-Atmosphere Studies (COLA) atmospheric general circulation model and concluded that seasonal simulations are superior to multi-annual simulations over most of South America except for northeastern Brazil. Schemes for objective combination of predictions from different models have been applied to weather and seasonal climate forecasts for several years (e.g. Krishnamurti et al., 1999; Krishnamurti et al., a; Krishnamurti et al., b; Pavan and Doblas-Reyes, ; Krishnamurti et al., ). In these schemes, a systematic improvement in root mean square (RMS) errors is observed for a multi-model forecast over that of the individual model forecasts. The Florida State University Superensemble (FSUSE) shows performance that is superior to the multi-model ensemble mean, since it employs weights (for combining models) that are based Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

3 SOUTH AMERICAN WEATHER AND SEASONAL CLIMATE PREDICTION 1 on past performance. In particular, this superensemble approach has been able to show some improvement in monsoon forecasts. It has been found that a variant of the conventional superensemble formulation, the synthetic superensemble, gives improved skills for seasonal climate forecasts (Yun et al., 5; Krishnamurti et al., 5). This formulation starts with multi-model forecasts and observational data sets, and then additional data sets called synthetic data sets are generated. The synthetic data set is created from a combination of past observations and forecasts. A statistical relationship between the observations and forecasts is determined, as a linear regression problem in empirical orthogonal functions (EOF) space. Sets of such synthetic forecasts are then obtained for the creation of superensemble forecasts. This technique has produced major improvements in the skill of seasonal climate predictions. The method of creation of the synthetic data set and the associated statistical procedures are described in more detail in Section. Palmer et al. () showed, with a comprehensive hindcast using DEMETER models, the enhanced reliability and skill of the multi-model ensemble over a more conventional single-model ensemble approach in the tropics and in the Northern Hemisphere extra-tropics. Hagedorn et al. (5) demonstrated the improvements that can be achieved by using multi-model ensembles instead of single-model ensembles. They showed that the superiority of the multi-model approach is due not only to error compensation but also to greater reliability. The reliability term evaluates the statistical accuracy of the forecast (Stefanova and Krishnamurti, ). A perfectly reliable forecast is one for which the observed conditional frequency is equal to the forecast probability, i.e. over all forecasts for y% chance of ε, ε will occur in y% of the times. The goal of this paper is to examine the feasibility of weather and seasonal climate predictions for South America using the Florida State University (FSU) multi-model synthetic superensemble approach (FSUSSE) for climate prediction, and the conventional FSU multi-model superensemble approach (FSUSE) for numerical weather prediction (NWP). The effect on the seasonal climate forecasts by the number of models used in the FSUSSE is also investigated. It is shown that the FSUSSE approach for climate and the FSUSE approach for NWP can reduce the errors in seasonal climate prediction and NWP over South America. For seasonal climate prediction, 1 different models are used, while for NWP 6 different models are utilized. For the NWP experiments on precipitation, 5 additional versions of the Florida State University (FSU) global spectral model, based on different physical initializations, are used in the ensemble to give 11 models in the so-called multi-analysis-multi-model superensemble. The models used for climate predictions are four versions of the FSU-Coupled Ocean-Atmosphere Model (LaRow and Krishnamurti, 199), seven models from the DEMETER (Palmer et al., ), a version of the Community Climate Model (CCM) (Kiehl et al., 199), and a version of the predictive Ocean Atmosphere Model for Australia (POAMA) (Zhong et al., 1). The forecast lead time is 1 month for the climate forecasts. Variables forecasted are precipitation and surface temperature. For the climate predictions, this study considers South America ( S 1 N and 9 W) and the northern portion of northeastern Brazil (1 S and 5 W) separately, because of differing climatic characteristics (Figure 1 for map of geographical areas). The period considered is from DJF 199/199 to 1/ for all of South America and MAM 199 to for the northern portion of northeastern Brazil. For NWP six global models are used from the following forecast centers: FSU (Krishnamurti et al., 199), Bureau of Meteorology Research Center (BMRC) (Hart et al., 199), Japan Meteorological Agency (JMA) (Surgi et al., 199), National Centers for Environmental Prediction (NCEP) (Kanamitsu, 199), Naval Research Laboratory (NRL) (Hogan and Rosmond, 1991), and Recherche en Prevision Numerique (RPN) (Cote et al., 199). The region considered is S 1 N and 9 W (Figure1formap).ForNWP, the skill of 1 5 day predictions of mean sea level pressure, 5 hpa wind, and 5 hpa geopotential height are evaluated in comparison to the forecasts of the models comprising the ensemble and the multi-model ensemble mean for the period January, February, and December of. The skill of 1 day predictions of precipitation are evaluated for the period January, February, and December of 1. The months of January, February, and December were chosen because this is the primary rainy season over South America. The FSUSE s skill in 1 day predictions of a heavy rain event associated with the SACZ during the period 1 1 December 1 is also presented. Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

4 1 R. R. CHAVES, R. S. ROSS AND T. N. KRISHNAMURTI NEB 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W Figure 1. Areas considered in this study: South America ( S 1 N and 9 W) and the northern portion of northeastern Brazil (1 S and 5 W) The results presented here suggest that the FSUSSE has practical applicability in the development and improvement of climate prediction for South America. The NWP results show that the FSUSE can be expected to provide predictions of precipitation, mean sea level pressure, lower tropospheric winds, and mid-tropospheric heights over South America with a skill that exceeds that of the operational global NWP models of the forecast centers mentioned above (FSU, BMRC, JMA, NCEP, NRL, RPN) that comprise the FSUSE. In Section, an overview of the models used in this research is presented. The methodology of the FSUSSE and the FSUSE approaches is described in Sections and respectively. Section 5 contains the results of the seasonal climate predictions, while Section 6 presents the results of the NWPs. Finally, Section 7 summarizes the main conclusions of this study.. OVERVIEW OF MODELS.1. Florida State University (FSU) coupled model The FSU-coupled model is a combination of the FSU atmospheric global spectral model (Krishnamurti et al., 199) and the Hamburg Ocean Model (HOPE) (Latif, 197). This coupled climate model is described in LaRow and Krishnamurti (199). The FSU multi-model is composed of four versions of the same model. These versions were constructed using two cumulus parameterization schemes, a modified Kuo scheme following Krishnamurti and Bedi (19) and an Arakawa Schubert type scheme following Grell (199), and two radiation parameterization schemes, an older emissivity-absorptivity based radiative transfer algorithm following Chang (1979) and a newer band model for radiative transfer following Lacis and Hansen (197). This allowed for four versions of the FSU-coupled climate model: Kuo scheme with older radiation scheme (KOR), Kuo scheme with newer radiation scheme (KNR), Arakawa Schubert scheme with older radiation scheme (AOR), Arakawa Schubert scheme with newer radiation scheme (ANR). A summary of the four FSU models is presented in Table I. These models were used to carry out 1 experiments ( initial conditions models 1 years) for the period from 1 December 196 to February. Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

5 SOUTH AMERICAN WEATHER AND SEASONAL CLIMATE PREDICTION 15 Table I. Characteristics of models in the FSU synthetic superensemble (FSUSSE) MODELS Atmospheric component Oceanic component Model Res. IC Model Res. IC KOR a,c FSU T6L1 ECMWF with Phy Init e KNR a,d FSU T6L1 ECMWF with Phy Init e AOR b,c FSU T6L1 ECMWF with Phy Init e ANR b,d FSU T6L1 ECMWF with Phy Init e HOPE Global HOPE Global HOPE Global HOPE Global L L L L Coupled Relax Obs SST f Coupled Relax Obs SST f Coupled Relax Obs SST f Coupled Relax Obs SST f CCM T6L6 AVN NCOM Slab SST NCEP/NCAR POAMA g,i BAM T7L17 BAM ACOM h 5 Optimum interp. j analysis L CERFACS ARPEGE T6L1 ERA OPA. ERA 1Levs CNRM ARPEGE T6L1 ERA OPA. 15 GP ERA 1 Levs LODYC IFS T95L ERA OPA. 1 ERA Levs INGV ECHAM- TL19 Coupled AMIP-type OPA ERA MPI ECHAM- 5 TL19 Coupled Run Met Office ARPEGE T6L1 ERA GloSea OGCM Levs MPI-OMI Levs Levs ECMWF IFS T95L ERA HOPE-E Levs a Krishnamurti and Bedi (19). b Grell (199). c Chang (1979). d Lacis and Hansen (197). e Krishnamurti et al. (1991). f LaRow and Krishnamurti (199). g Bureau of Meteorology unified atmospheric model BAM. h Australian Community Ocean Model version (ACOM). i Large et al. (1997). j Schiller et al.,. Coupled Run ERA ERA Validations of the four FSU-coupled models for atmospheric and oceanic variables were carried out for the Asian monsoon region, the equatorial Pacific, and the global tropical region. These experiments, including the physical initialization procedures, are described in detail in Mitra et al. (5) and Krishnamurti et al. (5). Chaves et al. (5) evaluated the performance of the FSU-coupled model for South America. Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

6 R. R. CHAVES, R. S. ROSS AND T. N. KRISHNAMURTI.. DEMETER models Another multi-model data set used in this study is the output of seven global-coupled ocean atmospheric models from DEMETER (Palmer et al., ). This model set is comprised of three models from France: CERFACS (European Centre for Research and Advanced Training in Scientific Computation), CNMR (Centre National de Recherches Meteorologiques), and LODYC (Laboratoire d Oceanographie Dynamique et de Climatologie); one model from Italy: INGV (Instituto Nazionale de Geofisica e Vulcanologie); one model from Germany: MPI (Max-Planck Institut fur Meteorologie); one model from the United Kingdom: UKMET (United Kingdom Meteorological Office); and one model from the European Union: ECMWF (European Centre for Medium-Range Weather Forecasting). Table I describes some features of these coupled models. DEMETER hind casts were started four times a year from February 1, May 1, August 1, and November 1, all at GMT. Each model was integrated for 6 months in ensemble mode on the basis of nine different initial conditions from each start date. The ensemble mean of each model is used to build the multi-model ensemble for the period The predictions considered in this paper correspond to the seasonal averages for the simulation period running from months to, i.e. the predictions are for the following seasons: DJF, March to May (MMA), June to August (JJA) and September to November (SON)... CCM The National Center for Atmospheric Research (NCAR) CCM model solves the dynamical equations in spectral space. The configuration used here is T6L6. Kiehl et al. (199) provide a complete description of the physical and numerical methods used in CCM. Observed SSTs from NCEP/NCAR reanalysis (Reynolds and Smith, 199) are used in the simulations. For the period from 1 December 19 to February, a series of 15-day integrations of the CCM was carried out with initial conditions defined on the 15th of each month from the NCEP Aviation Model (AVN). Several features of the model are presented in Table I... POAMA The POAMA is a state-of-the-art ocean atmosphere-coupled model for seasonal to inter-annual prediction. It was developed in a joint project involving the BMRC and CSIRO Marine Research (CMR). Real-time oceanic and atmospheric initial states are used to initialize the coupled model. These are provided by an ocean data assimilation system that is run in real time as part of the POAMA system, and by operational weather analyses from BMRC. The atmospheric component of the coupled model used in POAMA is the BMRC unified atmospheric model (BAM) with specifications of T7L17. The ocean model component is the Australian Community Ocean Model version (ACOM). It was developed by CMR, and is based on the Geophysical Fluid Dynamics Laboratory Modular Ocean Model (MOM version ). The performance of this model with observed SST forcing is described in Zhong et al. (1). Several features of the model are presented in Table I.. FLORIDA STATE UNIVERSITY SYNTHETIC SUPERENSEMBLE (FSUSSE) METHODOLOGY The superensemble approach is a recent contribution to the general area of weather and climate prediction developed at FSU (Krishnamurti et al., 1999; Krishnamurti et al., a). A variant of the conventional superensemble formulation, the synthetic superensemble (FSUSSE), was created to improve the skill in seasonal climate forecasts (Yun et al., 5; Krishnamurti et al., 5). A synthetic data set is generated from a combination of past observations and forecasts using EOFs. All calculations for the FSUSSE are done using cross-validation, with each year being successively withheld from the training dataset, while the remaining years are used for determination of the model and observed statistics (i.e. the monthly means and regression coefficients). These means and regression coefficients are used to generate the forecasts of precipitation and surface temperature for the verification year. Actual values, rather than anomalies, of precipitation and surface Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

7 SOUTH AMERICAN WEATHER AND SEASONAL CLIMATE PREDICTION 17 temperature are used in the determination of the FSUSSE. Observed rainfall is from Xie and Arkin (1997) and observed surface temperature is from the NCEP/NCAR reanalysis. The synthetic data set is created by finding a statistical relationship between the observed data set (Xie and Arkin, 1997) and the forecast data sets. Once EOFs of the observational data are found, the time series of the observations can be written as a linear combination of EOFs as, O(x,t) = n P n (t)φ n (x) (1) where n is the number of modes selected, and the two variables on the right-hand side of Equation (1) represent the time (principal component PC) and space (EOF) decomposition respectively. The domain used to determine the EOF s was global for each season (DJF, MAM, JJA, SON) considered separately. The PC time series, P(t), represents how EOFs (spatial patterns) evolve in time, and the PCs are independent of each other. Similarly, the forecast data can be projected into the PCs and EOFs for i models in the ensemble as, F i (x, t) = n F i,n (T ) ϕ i,n (x) () where i indicates the number of models (i = 1), n indicates the number of EOF modes, t defines the training period, and T represents the whole forecast time period. ϕ(x) and F(t) are the EOFs and the corresponding PC time series of F(x,t). With the number of EOF modes chosen as 5, 9% of the variance was explained. In order to determine the spatial patterns of the forecast data, which evolve in a consistent way with the EOFs of the observations for the time series considered, a regression relationship between the observation PC time series and a number of PC time series of forecast data is used. The regression relationship is given by, P(t) = n α i,n F i,n (t) + ε(t) () In Equation (), the observation time series P(t) is expressed in terms of a linear combination of forecast time series F(t) in EOF space. The regression coefficients α are estimated by singular vector decomposition (SVD) such that the residual error variance ε(t) is a minimum. Once the regression coefficients are determined, the PC time series of synthetic data can be written as, F reg i (T ) = n α i,n F i,n (T ) () Then the synthetic data set is reconstructed with EOFs and PCs as, F syn i (x, T ) = n F reg i,n (T )φ n(x) (5) A post-processing algorithm based on the synthetic multi-model data, in which all models (1 in this case) are post-processed jointly, has been shown to be one of the best solutions for extended range prediction in comparison to observed fields (Yun et al., 5; Krishnamurti et al., 5). Their research shows that this final product produced by the synthetic superensemble technique (FSUSSE) further reduces the forecast errors below those of the conventional superensemble technique (FSUSE) and increases the predictive skill of extended forecasts.. FLORIDA STATE UNIVERSITY CONVENTIONAL SUPERENSEMBLE (FSUSE) METHODOLOGY Here we present a short description of the conventional superensemble methodology, which is explained more fully in Krishnamurti et al. (1999). An independent forecast equation is derived for each grid point, vertical Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

8 1 R. R. CHAVES, R. S. ROSS AND T. N. KRISHNAMURTI level, forecast period, and forecast variable. The FSUSE forecast equation is written as, S = O + N a i (F i F i ) (6) i=1 where S is the real-time superensemble forecast, O is the mean observed value (from model analysis data) during the training period of days, F i represents the forecasts of one model in the ensemble (the ith model) averaged over the training period, F i is the real-time forecast from that same model (the ith model) i.e. the forecast corresponding in time to the superensemble forecast, and a i is the appropriate coefficient (derived from the training period) for that same model (the ith model). In the application of the FSUSE to NWP in the current study, the training data is independent of each FSUSE forecast. The FSUSE is re-trained for each forecast day using the previous days of data. Regression coefficients, a i, are determined by the Gauss Jordan elimination method. The summation is over all the individual forecast models, which is 6 for all forecast variables except precipitation, where the number of forecast models is 11, as discussed in the introduction. Since a separate forecast equation is derived for each grid point, vertical level, forecast period and forecast variable, over 1 million coefficients are determined by the Gauss Jordan elimination method in the global application of the technique. The benchmark analysis (O) for the mass and motion fields is the ECMWF analysis with FSU physical initialization. For precipitation forecasts, the benchmark analysis (O) is the Tropical Rainfall Measuring Mission (TRMM) and Special Sensor Microwave Instrument (SSM/I) satellite imaging. 5. RESULTS FOR SEASONAL CLIMATE PREDICTION 5.1. Model validation for South America For climate results, we define ensemble mean as the simple average of the forecasts for one model on the basis of different initial conditions. Multi-model ensemble mean refers to the simple average of the ensemble mean of all the models. FSUSSE forecasts refers to the weighted mean of the ensemble mean of all the models. In the validation of any climate model, it is important that the current climates from the model and from observation/analysis be carefully compared. The inter-annual variability of rainfall is better simulated in models that generate a climatology that agrees with observation (Sperber and Palmer, 1996). Models with a better rainfall climatology generally have less systematic errors and have somewhat higher ability to simulate inter-annual variations (Krishnamurti et al., ). Validation of the four FSU-coupled models was carried out by Krishnamurti et al. (5) and Mitra et al. (5) for the Asian monsoon region, the equatorial Pacific, and the global tropics using atmospheric and oceanic variables. The methodology for this forecast evaluation involves: (1) construction of seasonal anomalies of all model forecasts for a number of variables, such as precipitation, 5 hpa winds, -m surface temperature, and sea surface temperature; () exploration of the skill of the multi-model ensemble mean forecasts; () exploration of the skill of the FSU multi-model synthetic superensemble forecasts. The metrics for forecast evaluation include computation of hindcast and verification anomalies from model-observed climatology, time series of specific climate indices, standard deterministic multi-model ensemble mean scores such as anomaly correlation coefficient and RMS error, and evaluation of probabilistic forecast skill using the Brier score. These validations showed that the FSU model simulates the observed climate and its variability well in comparison to the observations, giving confidence in its applicability. Chaves et al. (5) evaluated the performance of the FSU model for South America, in which seasonal means were compared with observations. The model reproduces the rainfall patterns over the continent and neighbouring areas reasonably for the austral summertime and the wintertime when compared with observational rainfall data from Xie and Arkin (1997) and NCEP/NCAR reanalysis. Chaves et al. (5) used the suite of FSU-coupled models in the FSUSSE technique to assess their integrity for climate prediction over South America, specifically their ability to faithfully represent seasonal mean climatological features such as Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

9 SOUTH AMERICAN WEATHER AND SEASONAL CLIMATE PREDICTION 19 the monsoon circulation. The DEMETER multi-model ensemble system was validated in Palmer et al. () using ERA- data and GPCP precipitation. Their results indicate that the DEMETER is a viable pragmatic approach to the problem of representing model uncertainty in seasonal to inter-annual prediction, and it will lead to a more reliable forecasting system than that based on any one single model. In the current study, observed rainfall fields from Xie and Arkin (1997) were compared with simulated rainfall fields by all models in the ensemble for the DJF and MAM periods, the two main rainy seasons over South America. The overall spatial rainfall patterns over South America and adjacent oceanic regions were comparable to the respective seasonal mean analyses. Figure shows the mean rainfall for all models in the ensemble for DJF during the period 199/199 1/ (1 years), while Figure shows the mean rainfall for all models in the ensemble for MAM during the period The DJF months (Figure ) will be examined first. The simulated rainfall fields for the four versions of the FSU model are depicted in Figure (a) (d) The KNR version (Figure (c)) showed the poorest performance; this version underestimates the rainfall over most of South America while overestimating the rainfall over the tropical regions, when compared with the Xie and Arkin precipitation data set (Figure (o)). This version does not also show the SACZ configuration. The ANR version (Figure (a)) reasonably simulates the overall (a) ANR (b) AOR (c) KNR (d) KOR (e) CNFC (f) CNRM (g) LODY (h) SCNR (i) SCWF (j) SMPI (l) UKMO (m) ccm (n) POAMA (o) OBS Figure. Mean rainfall (mm/day) simulated by each model in the multi-model ensemble, along with the observed mean rainfall from Xie and Arkin (1997) over South America for DJF 199/199 1/ Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

10 19 R. R. CHAVES, R. S. ROSS AND T. N. KRISHNAMURTI rainfall patterns over South America when compared with the observed precipitation data (Figure (o)); however, it overestimates the intensity of rainfall over the continent and ITCZ. The AOR (Figure (b)) and KOR (Figure (d)) versions are similar and generally reproduce the observed rainfall patterns, e.g. the SACZ and ITCZ; however, both poorly simulate the ocean portion of the SACZ. The Demeter models (Figure (e) (l)) also appropriately simulate the observed DJF rainfall patterns over South America. The SCNR and UKMO models (Figure (h) and (l)) closely match the observed precipitation over the continent; however, both models overestimate the precipitation in the ITCZ. The LODY and SCWF models (Figure (g) and (i)) underestimate the precipitation over the continent, while showing excessive rain over the ITCZ region. Both models put spurious precipitation over the Andes Cordillera region associated with the Gibbs effect. Navarra et al. (199) have shown that effects of the Gibbs oscillations are evident in long integrations of dynamical forecasting models and may contribute to discrepancies between the simulated and observed climates. Lindberg and Broccoli (1996) showed that Gibbs effects become more pronounced with increasing horizontal resolution, causing deterioration in the fidelity of simulated precipitation in higher resolution models. The LODY and SCWF models are T95L, i.e. models with high resolution. The CNFC and CNRM models (Figure (e) and (f)) simulate the rainfall values well over the continent in comparison to the observed precipitation data (Figure (o)); however, they mislocate the area of maximum precipitation, placing it too far to the southeast over Brazil. The SCNR and SMPI models (Figure (h) and (j)) appear to best represent the rainfall pattern and intensity over South America in DJF. The CCM and POAMA model simulations (Figure (m) and (n)) of the rainfall patterns over South America for DJF both show reasonable agreement with the observed rainfall over the continent; however, the CCM model overestimates the precipitation in the ITCZ, while the POAMA model completely drops any ITCZ precipitation. The POAMA model also places spurious precipitation over the Andes Cordillera region. The deficiency of the models over South America can be related, in part, to their inability to properly represent the effects of latent heating. Latent heat release is a large source of heating in the region, and it is likely responsible for the regional circulation characteristics of the South America summer climate (Silva Dias et al., 19; Figueroa et al., 1995). This deficiency may be associated with the land surface and/or hydrological schemes used in the atmospheric component of the models (Marengo et al., 199). Next we will examine the months of MAM (Figure ). During the main rainy season over the northern portion of northeastern Brazil, associated with the southward displacement of the ITCZ, the FSU models (Figure (a) (d)) show varying success in simulating the rainfall patterns over South America. The ANR and KNR models (Figure (a) and (c)) overestimate the precipitation over the Amazon, northeastern Brazil and the ITCZ. The AOR and KOR models (Figure (b) and (d)) underestimate the precipitation over these regions. In MAM, the DEMETER models (Figure (e) (l)) correctly show a southward shift in the ITCZ, with this system located over the equator or just below it, in agreement with the observed precipitation from Xie and Arkin (1997) (Figure (o)); however, most of these models overestimate the precipitation in the ITCZ and over the continent. This is most apparent in the LODY and SCWF models (Figures (g) and (i)), where mm/day of precipitation in the ITCZ is placed over northeastern Brazil, in comparison to the observed rainfall of only 6 mm/day (Figure (o)). The CCM model (Figure (m)) overestimates the precipitation over northeastern Brazil, while the POAMA model (Figure (n)) underestimates the precipitation over the whole of South America, except over the Andes Cordillera region, because of the Gibbs effect. Figure shows a temporal series of the average seasonal precipitation forecasts for the four FSU models (o), the seven DEMETER models (x), CCM ( ), POAMA ( ), and the observed precipitation (-) from Xie and Arkin (1997). Figure (a) is for South America from DJF 199/199 to 1/, and Figure (b) is for the northern portion of northeastern Brazil from MAM 199 to 1. The averages of the forecasts of the DEMETER models and the averages of the forecasts of the FSU models, as well as the forecasts of the CCM and POAMA models, underestimate the rainfall intensity over South America and the northern portion of northeastern Brazil, except for the CCM model for DJF over South America. The average of the forecasts of the DEMETER models and the forecasts of the CCM model show the best performance for rainfall intensity Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

11 SOUTH AMERICAN WEATHER AND SEASONAL CLIMATE PREDICTION 191 (a) ANR (b) AOR (c) KNR (d) KOR (e) CNFC (f) CNRM (g) LODY (h) SCNR (i) SCWF (j) SMPI (l) UKMO (m) ccm (n) POAMA (o) OBS Figure. Same as Figure, but for MAM over South America in DJF, with values closer to the observed rainfall. The POAMA model shows the worst performance for most years during DJF. The average of the forecasts of the DEMETER models best captures the precipitation inter-annual variability over South America, e.g. the rainy periods during DJF 199/199, 1996/1997, and 1999/ and the drier periods during DJF 199/1995 and 1997/199. The POAMA model shows the worst performance for rainfall intensity over the northern portion of northeastern Brazil for most years during MAM. Although the errors in rainfall intensity are large for the northern portion of northeastern Brazil in the DEMETER ensemble, the high precipitation variability there is well simulated in comparison to the observed values. 5.. Precipitation forecast skill of the FSUSSE In this section, the performance skill of the aforementioned 1 models, the multi-model ensemble mean, and the FSUSSE are compared using RMS error during the period by season: DJF, JJA, and SON for South America ( S 1 N and 9 W) and MAM for the northern portion of northeastern Brazil (1 S and 5 W). (See Figure 1 for areas.) The bar diagrams on the right-hand side of Figure 5 represent 1 years (199/) of averaged RMS errors. As in Yun et al. (5) and Krishnamurti et al. Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

12 19 R. R. CHAVES, R. S. ROSS AND T. N. KRISHNAMURTI (a) rainfall (mm/day) DJF (b) rainfall (mm/day) MAM FSU DEMETER CCM POAMA XIE&A Figure. Time series of mean rainfall from the FSU ensemble of four models, the DEMETER ensemble of seven models, the CCM model, and the POAMA model, along with the mean rainfall observed from Xie and Arkin (1997) for (a) DJF 199/199 1/ over the continent of South America and (b) MAM over the northern portion of northeastern Brazil. (See Figure 1 for areas) (5), the development of the FSUSSE is based on cross-validation, i.e. all the years of forecasts, except for the one being addressed, are made a part of the training database successively. This process was necessary since the data length, i.e. numbers of forecasts, were still quite small for the optimal development of a training phase as discussed in Krishnamurti et al. (a). In Figure 5, the forecasts produced by the FSUSSE algorithm show lower RMS errors (higher skill) than those of the multi-model ensemble mean and the models comprising the ensemble for the overall average of all years. The highest skill is in the 1-year average. For example, in DJF the FSUSSE has a value of about 1.5 mm/day, while the value for the multi-model ensemble mean is about 1.5 mm/day, giving a reduction of error of nearly %. The individual model precipitation RMS error lies between 1.1 and. mm/day for DJF. In the overall average of all years, the RMS errors of the forecasts produced by the FSUSSE algorithm are reduced for all seasons in comparison to the multi-model ensemble mean. Recall that a reduction of RMS error is a built-in feature in the design of the FSUSSE (Equation (6)). The FSUSSE also has smaller systematic errors than the individual models since in the training procedures systematic errors are removed from the individual models in constructing the superensemble (Krishnamurti et al., 5). The SCNR model shows the best performance and the KNR and ANR models show the poorest performance for all seasons. In JJA and SON, when lower rainfall is observed over South America the performance of the FSUSSE forecast is still superior to all models; however, it is only slightly better than the multi-model ensemble mean (Figure 5(c) and (d)). The seasonal precipitation for DJF 1997/199 (Figure 6) and 1/ (Figure 7) represent wet seasons that are relatively dry and wet, respectively, in comparison to the mean observed values of the Xie Arkin data set Figure (a). These figures show the improvement in the geographical distribution of seasonal total rainfall obtained with the use of the FSUSSE. The rainfall was below normal over most of South America in DJF 1997/199 but it was above normal in the subtropical region. During this season the FSUSSE and multi-model ensemble mean performed very well (Figure 6(o) and (p)), particularly with the treatment of the SACZ. Over the northern portion of northeastern Brazil, most of the northern part of South America, and in the ITCZ, the rainfall intensity is better simulated by the FSUSSE than the multi-model ensemble mean Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

13 SOUTH AMERICAN WEATHER AND SEASONAL CLIMATE PREDICTION (a) RMSE (mm/day) DJF (b) MAM 1 1 ANR AOR KNR KOR CNFC CNRM LODY SCNR SCWF SMPI UKMO CCM POAMA ENS FSUSSE (c) JJA (d) SON AVG Figure 5. Time series of RMS errors of precipitation forecasts (mm/day) for all 1 models, the multi-model ensemble mean, and the FSUSSE over the continent of South America for: (a) DJF 199/199 DJF1/; (c) JJA 199 1; (d) SON Same but over the northern portion of northeastern Brazil for (b) MAM The averages over each period are shown on the right-hand side of the figure when compared to the observed rainfall. Such improvements are reflected in the lower RMS errors for the FSUSSE as compared to the multi-model ensemble mean (Figure 5(a)). Figure 7 shows the rainfall simulations for DJF 1/, a period in which the tropical convection and the associated atmospheric circulation anomalies exhibited strong intra-seasonal variability related to the Madden Julian Oscillation (MJO) (Climate Diagnostics Bulletin, /). This variability was a challenge for the models to capture. Over most of South America, the SACZ produced heavy rainfall (Figure 7(q)). The intensity of precipitation in this period was responsible for the increase of the reservoir levels in Brazil, resulting in an end to the energy crisis that forced the government in 1 to impose energy conservation measures in order to avoid total loss of power (blackouts). In this period, the multi-model ensemble mean and FSUSSE reproduce the rainfall patterns (Figure 7(o) and (p)) in comparison to the observed patterns. But the FSUSSE shows better performance than the multi-model ensemble mean in that it reproduces the observed rainfall area greater than 9 mm/day over the central part of South America, although of lesser size than the observed. The SCNR and UKMO models also showed good performance in this period (Figure 7(h) and (l)). Figure shows the rainfall time series from the FSUSSE (x), multi-model ensemble mean (o) and Xie and Arkin (1997) (-) from DJF 199/199 to 1/ for the continent of South America (See Figure 1.) For all seasons, the FSUSSE simulates the rainfall over South America better than the multi-model ensemble mean in comparison to the observed rainfall. The FSUSSE also captures the rainfall inter-annual variability over Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

14 19 R. R. CHAVES, R. S. ROSS AND T. N. KRISHNAMURTI (a) ANR (b) AOR (c) KNR (d) KOR (e) CNFC (f) CNRM (g) LODY (h) SCNR (i) SCWF (j) SMPI (l) UKMO (m) ccm (n) POAMA (o) ENS (p) FSUSSE (q) OBS Figure 6. Mean rainfall (mm/day) over South America for DJF 1997/199 for all models, the multi-model ensemble mean, the FSUSSE, and the mean observed rainfall from Xie and Arkin (1997) South America, from rainy periods such as and 1996/1997 to dry periods such as 1991/199, 199/199 and 1997/199. For the northern portion of northeastern Brazil, a good example of the performance of the FSUSSE in the simulation of rainfall can to be seen for the period MAM. The year was a wet year in this region, as shown in Figure (b). The FSUSSE shows better performance in reproducing the rainfall patterns over South America in this season when compared with the multi-model ensemble mean and all individual models in the ensemble as seen in Figure 9. Even though the FSUSSE underestimates the rainfall intensity over the northern portion of northeastern Brazil, it does show a maximum in precipitation in this region in agreement with the observed rainfall from the Xie and Arkin (1997) data set. The successful prediction of rainfall anomalies for a given region can have very positive socio-economic impacts. Thus, the skill in forecasting seasonal precipitation anomalies is central to any climate forecast. The primary objective of this study was to explore the feasibility of seasonal prediction of the precipitation over South America with the FSUSSE approach on the timescale of 1 months. We now focus on assessing the performance of the model in simulating excessive dryness and wetness as seen in seasonal anomalies. Such seasonal anomalies are verified for DJF 1997/199 and MAM in Figures 1 and 11 respectively. These Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

15 SOUTH AMERICAN WEATHER AND SEASONAL CLIMATE PREDICTION 195 (a) ANR (b) AOR (c) KNR (d) KOR (e) CNFC (f) CNRM (g) LODY (h) SCNR (i) SCWF (j) SMPI (l) UKMO (M) ccm (n) POAMA (o) ENS (p) FSUSSE (q) OBS Figure 7. Same as Figure 6, but for DJF 1/ rainfall (mm/day) DJF ENS FSUSSE XIE&ARKIM Figure. Time series of mean rainfall for the multi-model ensemble mean, the FSUSSE, and the mean observed rainfall from Xie and Arkin (1997) over the continent of South America for DJF 199/199 1/ Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

16 196 R. R. CHAVES, R. S. ROSS AND T. N. KRISHNAMURTI (a) ANR (b) AOR (c) KNR (d) KOR (e) CNFC (f) CNRM (g) LODY (h) SCNR (i) SCWF (j) SMPI (l) UKMO (m) ccm (n) POAMA (o) ENS (p) FSUSSE (q) OBS Figure 9. Mean rainfall (mm/day) over South America for MAM for all models, the multi-model ensemble mean, the FSUSSE, and the mean observed rainfall from Xie and Arkin (1997) anomalies were calculated with respect to the observed seasonal mean values (for the observed anomalies) and with respect to the forecast seasonal mean values (for the FSUSSE and multi-model ensemble mean anomalies). The pattern of precipitation anomalies for DJF 1997/199 are captured very well by the FSUSSE and the multi-model ensemble mean when compared to the observed anomalies (Figure 1). However, the patterns are better reproduced by the FSUSSE approach than by the multi-model ensemble mean. Negative precipitation anomalies are observed over most of South America except over the subtropical region. The FSUSSE underestimates the rainfall anomalies over the subtropical region and overestimates these anomalies over northern South America. The observed precipitation anomalies over subtropical South America are above mm/day, while the anomalies from the FSUSSE are 1 mm/day. The pattern of precipitation anomalies for MAM are also captured reasonably well by the FSUSSE and multi-model ensemble mean (Figure 11). However, these anomalies are underestimated over subtropical South America by both the FSUSSE and the multi-model ensemble mean. Over the northern portion of northeastern Brazil, the FSUSSE overestimates the anomalies, while the multi-model ensemble mean underestimates the anomalies. Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

17 SOUTH AMERICAN WEATHER AND SEASONAL CLIMATE PREDICTION 197 (a) FSUSSE (b) ENS 9W W 7W 6W 5W W W W 9W W 7W 6W 5W W W W (c) XIE&ARKIN 9W W 7W 6W 5W W W W Figure 1. Seasonal rainfall anomalies over South America for DJF 1997/199 (mm/day) for the FSUSSE and the multi-model ensemble mean, along with the observed rainfall anomaly based on Xie and Arkin (1997) 5.. Surface air temperature forecast skill of the FSUSSE The reductions in RMS errors in surface temperature forecasts over South America by the FSUSSE for the seasons of DJF and JJA are depicted in Figure. The average of the RMS errors in the FSUSSE forecasts are. K for DJF (Figure (a), right side panel) and 1. K for JJA (Figure (b), right side panel), while for the models comprising the ensemble, average values range from. 5.5 K for DJF (Figure (a)) and from 1.. K for JJA (Figure (b)). In JJA the temperature variations caused by the mid-latitude frontal systems can affect the harvests of wheat, coffee, soybeans, and oranges in the agricultural lands of southeastern and southern South America. Thus, surface temperature forecasts are very important in this context. Figure 1 shows the temporal series of seasonal surface temperature forecasts over South America during the periods DJF 199/199 1 and JJA for the multi-model ensemble mean and the FSUSSE, with NCEP/NCAR reanalysis values shown for verification. For the DJF seasons, temperature values are well simulated by the FSUSSE forecasts in comparison to the verifying analysis; however, inter-annual variability is not well simulated. The multi-model ensemble mean forecasts overestimate the temperature values for both seasons, but they do capture some of the surface temperature inter-annual variability, e.g. the periods 199/1991, 1997/199, 1999/ in DJF and 1997/199, 199/1999, /1 in JJA. The differences between the multi-model ensemble mean surface temperature forecasts and observations are larger in DJF than in JJA, possibly due to the larger latent heat release in the austral summer season of DJF (Silva Dias et al., 19; Figueroa et al., 1995). This heating amplifies the circulation over the South American Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

18 19 R. R. CHAVES, R. S. ROSS AND T. N. KRISHNAMURTI (a) FSUSSE (b) ENS (c) XIE&ARKIN Figure 11. Same as Figure 1, but for MAM (a) RMSE (K) DJF (b) RMSE (K) JJA ANR AOR KNR KOR CNFC CNRM LODY SCNR SCWF SMPI UKMO CCM PARA ENS FSUSSE Figure. Time series of RMS errors of surface temperature (K) for all 1 models, the multi-model ensemble mean, and the FSUSSE over the continent of South America for: (a) DJF 199/199 1/; (b) JJA The averages over each period are shown on the right-hand side of the figure Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

19 SOUTH AMERICAN WEATHER AND SEASONAL CLIMATE PREDICTION (a) surface temperature (K) DJF (b) surface temperature (K) JJA NCEP/NCAR ENSEMBLE FSUSSE Figure 1. Time series of mean surface temperature over the continent of South America for the multi-model ensemble mean, the FSUSSE, and the observed mean surface temperature from the NCEP/NCAR reanalysis for: (a) DJF 199/199 1/; (b) JJA continent (Rao and Erdogan, 199; Lenters and Cook, 1999) and could be responsible for regional circulation biases in the various models that affect the austral summer surface temperature forecasts. 5.. Effect of multi-model ensemble size on FSUSSE performance Kharin and Zwiers () examined the forecast skill of a multi-model ensemble mean and found that the skill is dependent on the ensemble size. Palmer et al. (), using Brier skill score and the reliability component of the Brier skill score, show that the skill score grows with ensemble size for an ensemble of the ECMWF model alone with less than about members, when used for predicting tropical summer precipitation. However, this threshold was found to change with the region, variable, and event. The idea of the superiority of multiple source prediction systems has been discussed in several works (e.g. Krishnamurti et al., a; Palmer and Shukla, ; Palmer et al., ). Using the DEMETER models, Hagedorn et al. (5) examine the question of whether the improvement in the multi-model ensemble is merely due to increased ensemble size or if the additional information from different models adds to the performance. Figure 1 and Table II show the FSUSSE and multi-model ensemble mean performance over South America from DJF 199/199 to 1/, and over the northern portion of northeastern Brazil from MAM 199 to 1 with FSU models, 7 DEMETER models, 11 models ( FSU models plus 7 DEMETER models), and 1 models ( FSU models plus 7 DEMETER models plus CCM and POAMA models). For DJF using four and seven models in the FSUSSE, the performance is similar, with RMS error means of 1.9 and 1.5 mm/day, respectively. However, when the model number increases to 11 and 1, the performance improves and the RMS error mean is now 1.1 and 1.1 mm/day respectively. Thus, FSUSSE performance generally improves as the number of models in the ensemble is increased, and particularly improves when the diversity of models in the ensemble is increased. The same result is seen for the multi-model ensemble mean for DJF in Figure 1(b) and Table II (RMS errors of 1.71, 1.67, 1., 1.5). A similar result is found for the FSUSSE and the multi-model ensemble mean in MAM, where the greatest improvement in the forecast Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

20 19 R. R. CHAVES, R. S. ROSS AND T. N. KRISHNAMURTI (a) RMSE (mm/day) DJF FSUSEE (b) DJF ENS MEAN (c) MAM FSUSSE (d) MAM ENS MEAN 1 models 7 models 11 models 1 models Figure 1. Time series of the RMS errors of precipitation forecasts (mm/day) with the FSUSSE and multi-model ensemble mean for: (a) FSUSSE for DJF 199/199 1/ over the continent of South America; (b) multi-model ensemble mean for DJF 199/199 1/ over the continent of South America; (c) FSUSSE for MAM over the northern portion of northeastern Brazil; (d) multi-model ensemble mean for MAM over the northern portion of northeastern Brazil. Each set of bar graphs represents the following combination of models from left to right: FSU models, 7 DEMETER models, 11 models comprised of the FSU models plus the DEMETER models, 1 models comprised of the FSU models plus the DEMETER models plus the CCM and POAMA models Table II. Mean RMS Error from different ensembles model a 7 model b 11 model c 1 model d Synthetic super DJF Ensemble mean DJF Synthetic super MAM Ensemble mean MAM a versions of FSU model. b 7 DEMETER models. c versions of FSU model + 7 DEMETER models. d versions of FSU model + 7 DEMETER models + CCM + POAMA Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

21 SOUTH AMERICAN WEATHER AND SEASONAL CLIMATE PREDICTION 191 is found with the 1 model ensembles. The improvement of the FSUSSE and multi-model ensemble mean performance with the increase in model number is associated with the input of additional information from different models and with error cancellation and non-linearity of the diagnostics, as discussed in Hagedorn et al. (5). The improvement in the multi-model ensemble mean and FSUSSE forecasts with increased model number and diversity is not always a straightforward matter. For example, Figure 15 shows seasonal rainfall anomalies for DJF 1997/199 from the FSUSSE with, 7, 11, and 1 models. When we compare with observations in Figure 1(c), we find that the FSUSSE prediction in the tropical areas is better with models than with 7, 11, or 1 models, while in the subtropical areas the prediction is better with 7 models than with, 11, or 1 models. Optimal ensemble size may vary with the region, season, and parameter being forecasted. More research is needed to address this important question Probabilistic forecast evaluation The Brier score (Brier 195) was designed to quantify the performance of a probabilistic forecast of a determined event. The Appendix provides a full treatment of the mathematics of the Brier score, which includes a modern formulation of this score by Wilks (1995), and a decomposition of the Brier score into three terms, reliability, resolution, and uncertainty, as proposed by Murphy (197). The Appendix also presents the formulation of this score as the Brier skill score, which is the formulation used in this paper for all (a) model (b) 7 models (c) 11 models (d) 1 models Figure 15. Seasonal rainfall anomalies for DJF 1997/199 (mm/day) from the FSUSSE with: (a) versions of the FSU model; (b) 7 DEMETER models; (c) 11 models ( FSU + 7 DEMETER models); (d) 1 models ( FSU + 7 DEMETER + CCM + POAMA models) Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

22 19 R. R. CHAVES, R. S. ROSS AND T. N. KRISHNAMURTI calculations, following Stefanova and Krishnamurti, (). For Brier skill scores, a value of 1 indicates a perfect forecast, a value of indicates a forecast that is equal in skill to a forecast of climatology, and a negative value denotes a forecast that has less skill than a forecast of climatology. The reliability term evaluates the statistical accuracy of the forecast, i.e. a perfectly reliable forecast is one for which the observed conditional frequency is equal to the forecast probability. The resolution term considers the distance between the forecast frequency and the unconditional climatological frequency and, thereby, measures the ability of the forecasts to distinguish between different regimes. The uncertainty term is a measure of the variability of the system and is not influenced by the forecasts. In this paper, we will use the Brier skill score to verify the probabilistic skill of the FSUSSE and the multi-model ensemble mean for rainfall over South America (large domain in Figure 1), as well as for the northern portion of northeastern Brazil (small domain in Figure 1). The Brier skill score values are averages over the respective domain and over all seasons, based on forecasts for the period Values are given for three categories of precipitation: 5 mm/day, 5 1 mm/day, and 1 15 mm/day. The Brier skill score results are presented in Tables IV (South America) and V (northern portion of northeastern Brazil). The overall Brier skill score is indicated, as well as Brier skill scores for reliability and resolution. For both South America and the northern portion of northeastern Brazil, there is no clear advantage in skill for either the FSUSSE or the multi-model ensemble mean. Both models have larger values of the overall Brier skill score (greater skill) for the smaller precipitation amounts ( 5 mm/day). Both models have their lowest skill, as reflected in the overall Brier skill score, for the largest precipitation amounts (1 15 mm/day) over the northern portion of northeastern Brazil. For both models and both regions, the reliability scores are higher than the resolution scores, indicating that the statistical accuracy of the forecasts exceeds the ability of the forecasts to distinguish between different precipitation regimes. The reliability values approach a perfect score of 1 for both models, except in the 1 15 mm/day precipitation category over the northern portion of northeastern Brazil, where values are in the range The FSUSSE has a slight advantage over the ensemble mean in the reliability scores for the northern portion of northeastern Brazil for the three precipitation categories. These results are based on only 1 seasons and should be viewed with caution. Future work will have the goal of improving the FSUSSE forecasts for both reliability and resolution. 6. RESULTS FOR NUMERICAL WEATHER PREDICTION (NWP) In this section of the paper, we evaluate the skill of the FSUSE technique as applied to NWP of the mass and motion fields, as well as precipitation, over South America in the region S 1 N and 9 W. This region is delineated in Figure 1. Predictions of mean sea level pressure, 5 hpa geopotential height, and 5 hpa wind are evaluated for the -month period of January, February, and December of. Precipitation forecasts are evaluated for the -month period of January, February, and December of 1. Skill is measured by RMS error for the mass and motion field forecasts. Precipitation forecast skill is measured by RMS error and equitable threatscore (ETS). Mathematicalexpressions for these parametersare presented in the Appendix. The verifying analysis for the mass and motion fields is the European Center for Medium-Range Weather Forecasting (ECMWF) analysis with FSU physical initialization. Precipitation forecasts are verified against rainfall derived from the TRMM and SSM/I. For NWP results, we define multi-model ensemble mean as the simple average of the forecasts of all the models in the ensemble. FSUSE forecasts refers to the weighted mean of the forecasts of all the models in the ensemble. The models used in the FSUSE are listed in Table III. The FSUSE technique used for the mass and motion fields is correctly termed multi-model, meaning that the various global forecast models listed in Table III comprise the ensemble. The FSUSE technique utilized in the precipitation forecasts is termed multi-analysis-multi-model. This is because, in addition to the global forecast models listed in Table III, five additional versions of the FSU global spectral model are used as models in the ensemble, each with a different initialization (analysis) based on different physical initialization procedures (rain-rate algorithms). Thus, the term multi-analysis refers to multiple runs of the same model with different initializations. Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

23 SOUTH AMERICAN WEATHER AND SEASONAL CLIMATE PREDICTION 19 Table III. Models in the FSU superensemble (FSUSE) BMRC Australia Bureau of Meteorology Research Center FSU United States The Florida State University JMA Japan Japan Meteorological Agency NCEP United States National Centers for Environmental Prediction NRL United States Naval Research Laboratory-NOGAPS Model RPN Canada Recherche en Prevision Numerique Table IV. Brier skill scores for the FSUSSE and multi-model ensemble mean South America (large domain in Figure 1) 5 mm/day 5 1 mm/day 1 15 mm/day B B rel B res B B rel B res B B rel B res FSUSSE ENS Table V. Brier skill scores for the FSUSSE and multi-model ensemble mean northern portion of northeastern Brazil (small domain in Figure 1) 5 mm/day 5 1 mm/day 1 15 mm/day B B rel B res B B rel B res B B rel B res FSUSSE ENS Forecasts of mass and motion fields The RMS errors in the FSUSE forecasts of mean sea level pressure, 5 hpa height, and 5 hpa wind speed are shown in Figure. Errors in the various models comprising the ensemble, as well as in the multimodel ensemble mean, are also depicted. The errors shown represent the mean of the daily error values for the -month period (January, February, December ). For the three parameters, the FSUSE forecasts have lower RMS errors than all models in the ensemble and the multi-model ensemble mean for forecast days 1 5. The advantage of the FSUSE over the other models and the multi-model ensemble mean is particularly striking for mean sea level pressure (Figure (a)). 6.. Precipitation forecasts The precipitation forecasts over the continent of South America will be evaluated first by comparing the observed (TRMM and SSM/I) precipitation to the forecast precipitation by the FSUSE, the multi-model ensemble mean, and the best model. The comparison is made for,, and 7-h forecasts in Figure 17. The precipitation shown represents the mean of observed and forecast precipitation for the -month period of January, February, and December of 1. The best model is defined as the model with the lowest RMS error among the models comprising the ensemble. This best model was selected on the basis of the same forecasts as those that were assessed, not from independent, or cross-validation forecasts. The best model was the same model for all lead times (forecast days 1 ). Precipitation amounts are in mm/day. The observed precipitation in Figure 17 shows the near equatorial rain belt associated with the ITCZ over the oceans, generally in the region 5 N. Precipitation is widely distributed over the South American continent with the SACZ indicated by a wide band of precipitation oriented from the northwest to the southeast across the continent and extending into the South Atlantic Ocean near S. The FSUSE forecasts at,, and Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

24 19 R. R. CHAVES, R. S. ROSS AND T. N. KRISHNAMURTI RMSE (MB) (a) South American Region MSLP RMSE JFD (-, 9W-W) 1 5 Model 1 Model Model Model Model 5 ENMEAN SUPENS Forecast Day RMSE (m/s) (b) South American Region 5 MB Wind Speed RMSE JFD (-, 9W-W) 1 5 Model 1 Model Model Model Model 5 ENMEAN SUPENS Forecast Day RMSE (m) (c) South American Region 5 MB Height RMSE JFD (-, 9W-W) 1 5 Forecast Day Model 1 Model Model Model Model 5 ENMEAN SUPENS Figure. RMS errors for models in the ensemble, multi-model ensemble mean, and FSU superensemble for the South American region for January, February, December of : (a) mean sea level pressure; (b) 5 hpa wind speed; (c) 5 hpa height. Forecast lead is 1 5 days. Models are from Table III: RPN, BMRC, FSU, JMA, NRL. NCEP data was not available 7 h capture all of these features very well. The best model tends to over-forecast the rain belts over the ocean and to under-forecast the precipitation over the continent. The near equatorial rain belt in the Atlantic is located progressively too far to the south as one moves from the to the 7 h forecast in the best model. The general pattern of precipitation associated with the SACZ is portrayed by the best model but without the clarity and detail found in the FSUSE. The precipitation forecasts by the multi-model ensemble mean suffer from the same deficiencies as just described for the best model, but these deficiencies are even more pronounced in the multi-model ensemble mean forecasts. Specifically, the near equatorial rain belts over the ocean become very expansive at the expense of rain over the continent by the 7 h forecasts (Figure 17(c)). Again, the SACZ related precipitation can be discerned but less faithfully than in the FSUSE forecasts. The RMS errors in the forecasts of precipitation by the FSUSE, the various models comprising the ensemble, and the multi-model ensemble mean are shown in Figure 1 for the area depicted in Figure 17. The error Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

25 SOUTH AMERICAN WEATHER AND SEASONAL CLIMATE PREDICTION 195 (a) Observed Superensemble Forecast 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W Ensemble Mean Forecast Best Model Forecast 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W Figure 17. Mean daily observed and forecast precipitation by the best model, multi-model ensemble mean, and FSU superensemble for the South American region during the period January, February, December of 1: (a) -h forecast; (b) -h forecast; (c) 7-h forecast values in mm/day represent the mean of the daily error values for the -month period (January, February, December 1). The FSUSE precipitation forecasts are shown to have the lowest RMS values when compared to all models in the ensemble and the multi-model ensemble mean for all forecast days 1. This quantitative measure of performance confirms the superior quality of the FSUSE precipitation forecasts seen in Figure 17. Further quantitative documentation of the quality of the FSUSE precipitation forecasts in relation to the models comprising the ensemble and the multi-model ensemble mean is found in Figure 19, where the ETS is presented for threshold values of. and 5. mm/day. For the. mm/day threshold FSUSE ETS values are.5 or greater, while for the 5. mm/day threshold they are slightly lower, in the range..5. In both cases, these values are substantially greater than any of the models in the ensemble, as well as the multimodel ensemble mean. These higher ETS values indicate that the FSUSE has greater skill in the placement of precipitation areas in comparison to the other models. During the period 17 1 December 1, precipitation was heavy across South America, particularly along the SACZ. Figure shows the observed precipitation (TRMM and SSM/I) for December 1 1, along with forecasts of that precipitation by the FSUSE and the worst and best models as defined by RMS errors. The best and worst models are defined as the models with the smallest and largest RMS errors, respectively, among the models comprising the ensemble. These best and worst models were selected Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

26 196 R. R. CHAVES, R. S. ROSS AND T. N. KRISHNAMURTI (b) Observed Superensemble Forecast 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W Ensemble Mean Forecast Best Model Forecast 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W Figure 17. (Continued) on the basis of the same forecasts as those that were assessed, not from independent, or cross-validation forecasts. The best and worst models were the same models for all lead times (forecast days 1 ). Figure (a) shows h predictions of precipitation verifying on December 1 1, while Figure (b) shows h predictions and Figure (c) shows 7 h predictions verifying on that same date (December 1 1). The FSUSE generally does a very good job, in comparison to observation, with placement and amounts of precipitation for its -, -, and 7-h forecasts along the SACZ. There is some underestimation of rainfall amounts over land. By comparison the best and worst models seriously under-forecast rainfall amounts over land, with the worst model almost eliminating this rainfall. The FSUSE also handles the rainfall much better than the worst and best models along the oceanic ITCZ. The best model tends to widen this precipitation band too much, and the worst model seriously over-forecasts the rainfall amounts in this feature. 7. CONCLUSIONS This paper discusses the feasibility of weather and seasonal climate prediction based on the FSUSE and the FSUSSE approaches respectively. For climate prediction, a multi-model ensemble of 1 models comprised Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

27 SOUTH AMERICAN WEATHER AND SEASONAL CLIMATE PREDICTION 197 (c) Observed Superensemble Forecast 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W Ensemble Mean Forecast Best Model Forecast 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W Figure 17. (Continued) RMSE (mm/day) 1 6 South American Region Precip RMSE JFD 1 ( -1 N, 9 - W) 1 Forecast Day Model 1 Model Model Model Model 5 Model 6 Model 7 Model Model 9 Model 1 Model 11 ENSM SUPENS Figure 1. RMS errors in precipitation forecasts during the period January, February, December of 1 for models in the ensemble, multi-model ensemble mean, and FSU superensemble over the South American region. Forecast lead is 1 days. Models are from Table III: RPN, BMRC, six versions of the FSU model based on different initializations, NCEP, JMA, NRL Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

28 19 R. R. CHAVES, R. S. ROSS AND T. N. KRISHNAMURTI Score (a) South American Region Precip Equitable Threat Score (. MM/DAY Threshold) JFD 1 ( S-1 N, 9 - W) 1 Forecast Day Model 1 Model Model Model Model 5 Model 6 Model 7 Model Model 9 Model 1 Model 11 ENSM SUPENS Score (b) South American Region Precip Equitable Threat Score (5. MM/DAY Threshold) JFD 1 ( S-1 N, 9 - W) 1 Forecast Day Model 1 Model Model Model Model 5 Model 6 Model 7 Model Model 9 Model 1 Model 11 ENSM SUPENS Figure 19. Equitable threat score in precipitation forecasts for January, February, December of 1 by models in the ensemble, multi-model ensemble mean, and FSU superensemble over the South American region: (a). mm/day threshold and (b) 5. mm/day threshold. Forecast lead is 1 days. Models are from Table III: RPN, BMRC, six versions of the FSU model based on different initializations, NCEP, JMA, NRL of versions of the FSU-coupled model, 7 DEMETER coupled models, the POAMA coupled model, and the CCM model is used. The goal of the FSUSSE approach is to improve long-range (monthly and longer) climate forecast skills. The weighted multi-model FSUSSE technique relies on statistical relationships between data sets from individual models in the ensemble, along with past observations. Observed structures, based on principal component time series and EOF, are projected onto the multi-model forecast runs. The multi-model forecast runs are combined in the FSUSSE methodology to produce a forecast that generally outperforms the forecasts of the multi-model ensemble mean and the models comprising the ensemble when compared to observations. The main contribution of this portion of the study was to show the reduction of RMS error and improvement of the seasonal forecasts for South America using the multi-model FSUSSE. The probabilistic skills of the forecasts of precipitation by the FSUSSE and the multi-model ensemble mean over South America and the northern portion of northeastern Brazil were evaluated using the Brier skill scores. These scores were essentially equal for the FSUSSE and the ensemble mean over South America (large domain in Figure 1) in the three categories of precipitation ( 5 mm/day, 5 1 mm/day, 1 15 mm/day). The FSUSSE had a slight advantage over the ensemble mean in the Brier skill score for reliability for the northern portion of northeastern Brazil (small domain in Figure 1) in all three categories of precipitation. There are two possible explanations for the higher skills that are generally seen in the forecasts of the FSUSSE: the PC time series and the EOF selection (roughly 5 modes) filter out some of the higher frequency noise from the full field that can otherwise degrade the forecast and the spatial structures (EOFs) for model forecasts are projected on the basis of the observed part of the training phase (Krishnamurti et al., 5). The FSUSSE was shown to have useful predictive skill for seasonal precipitation anomalies over South America. In comparison to Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

29 SOUTH AMERICAN WEATHER AND SEASONAL CLIMATE PREDICTION 199 (a) Observed Superensemble Forecast 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W Worst Model Forecast Best Model Forecast 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W (b) Observed 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W Superensemble Forecast 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W Worst Model Forecast Best Model Forecast 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W Figure. Observed and forecast precipitation for December 1 1 by the best model, worst model, and FSU superensemble: (a) -h forecast; (b) -h forecast; (c) 7-h forecast Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)

30 191 R. R. CHAVES, R. S. ROSS AND T. N. KRISHNAMURTI (c) Observed Superensemble Forecast 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W Worst Model Forecast Best Model Forecast 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W 9W 5W W 75W 7W 65W 6W 55W 5W 5W W 5W W 5W W Figure. (Continued) observation, this approach reproduced the rainfall anomalies over South America for DJF 1997/199 very well, and reproduced the anomalies for MAM reasonably well. The results are encouraging, suggesting that the vast database and FSUSSE approach are able to provide useful improvement in seasonal climate forecasts in comparison to the use of single models and multi-model ensembles. For NWP, this paper has shown that, in comparison to observation, the FSUSE provides the best forecasts of mean sea level pressure, lower tropospheric wind, mid-tropospheric height, and precipitation for the South American region, in comparison to the global forecast models comprising the ensemble, and the multimodel ensemble mean, during the wet season of December February. The FSUSE forecasts provide the best representation of the near equatorial rain belt associated with the ITCZ and the SACZ, as seen in -month mean maps. Individual case studies also reveal that the FSUSE has the best depiction of the location and amount of precipitation for particular synoptic events. We conclude that the FSUSE is an extremely useful tool for short- to medium-range NWP over the South American region. Overall, this paper has demonstrated the effectiveness of superensemble forecast techniques developed at the FSU for both weather and seasonal climate prediction for the continent of South America. ACKNOWLEDGEMENTS This research was supported by NSF grants ATM-171 and ATM-115, NASA grant NAG5-156, NOAA grant NAG GP165, and FSU Research Foundation grant DEMETER data used in this study was obtained from the ECMWF data server. The authors wish to thank L. Stefanova for her assistance with the Brier skill score calculations. Copyright 5 Royal Meteorological Society Int. J. Climatol. 5: (5)