Evaluation of Sub-monthly Precipitation Forecast Skill. from Global Ensemble Prediction Systems

Transcription

1 1 2 Evaluation of Sub-monthly Precipitation Forecast Skill from Global Ensemble Prediction Systems 3 Shuhua Li and Andrew W. Robertson International Research Institute for Climate and Society, The Earth Institute at Columbia University, Palisades, NY 4 April 25, To be Submitted to Monthly Weather Review. Correspondence address: IRI - Monell 230, 61 Route 9W, Palisades, NY Phone: , Fax: , shuhua@iri.columbia.edu. 1

2 6 Abstract The prediction skill of precipitation at sub-monthly time scales during the boreal summer season is investigated based on hindcasts from three global ensemble prediction systems (EPS). The results valid for up to 4 weeks indicate good skill or predictability over some regions particularly over southeastern Asia and the Maritime Continent. The hindcasts from all the three models correspond to high predictability over the first week compared to the following three weeks. The ECMWF forecast system tends to yield higher prediction skill than the other two systems, in terms of both anomaly correlation and mean squared skill score. The sources of sub-monthly predictability are examined over the Maritime Continent with focus on the intra-seasonal MJO and interannual ENSO phenomena in the EPS hindcasts. Rainfall variations for neutral-enso years are found to correspond well with the dominant MJO phase, whereas for moderate/strong ENSO events, the relationship of rainfall anomaly with MJO appears to become weaker, while the contribution of ENSO to the sub-monthly skill is substantial. However, if a moderate/strong MJO crosses the Indian Ocean (phases 2 3) during typical El Niño years, or MJO phases 5 6 during La Niña years, the MJO impact can become dominant, regardless of whether the ENSO event is strong or moderate. These results support the concept that windows of opportunity of high forecast skill exist as a function of ENSO and the MJO in certain locations and seasons, that may lead to subseasonal to seasonal forecasts of substantial societal value in the future. 2

3 27 1 Introduction There has been a recent surge of interest in prediction on sub-seasonal timescales between the realm of medium-range weather forecasts (up to 7 10 days) and seasonal climate forecasts (3 6 months) (WMO 2013). The sub-seasonal realm lies beyond the predictable impact of atmospheric initial conditions in numerical weather forecasts(lorenz 1975), but falls short of the seasonal averages where anomalies in the atmosphere s surface boundary conditions are felt, principally from sea surface temperatures (SSTs) (Charney and Shukla 1981). This interest has been driven by recent advances in modeling (Vitart 2004) and understanding of climate phenomena on sub-seasonal timescales, and in particular the Madden Julian Oscillation (MJO, see recent review by Zhang (2013)). The concept of seamless prediction, in which there is potential predictive power on all time scales associated with phenomena occurring on that time scale (Hoskins 2012), is particularly attractive in the sub-seasonal time range, where there is evidence that in addition to the inertia of SST anomalies, the MJO (Waliser et al. 2013) and processes involving the stratosphere (Baldwin and Dunkerton 2001, Scaife and Knight 2008), soil moisture (Koster et al. 2010), snow cover (Lin and Wu 2011) surface and sea ice (Holland et al. 2011) can play important roles. Besides the growing scientific basis for sub-seasonal prediction, the weekly-to-seasonal time range is one in which early warning systems for high-impact weather events could be of substantial societal benefit (WMO 2013). This is particularly urgent in the context of increasing societal exposure to extreme weather threats, be they caused by growing populations or due to decadal climate variability or anthropogenic climate change. 3

4 The aim of this paper is to examine sub-monthly precipitation prediction skill for the June September boreal summer season using three ensemble prediction systems (EPS) that are currently used for opertational sub-seasonal prediction. While weather forecasts are usually evaluated on a daily basis, seasonal forecasts are typically issued as 3-month averages, so as to enhance the predictable climate signal by averaging out the daily weather noise (Palmer and Anderson 1994, Mason et al. 1999). In keeping with the intermediate time scale, we focus here on weekly averages at forecast leads of 1 4 weeks. The sub-seasonal range is beyond deterministic weather prediction so that some time aggregation is required, and the weekly scale makes a natural choice. Weekly time averages are chosen for simplicity and convenience of comparison with observations, although predictability is likely to be enhanced (perhaps considerably) using daily rainfall frequency, especially in the tropics (Moron et al. 2006). Due to the limited ensemble size (4 5) for all those models, the skill evaluation will be confined to deterministic skill metrics only In Section 2 a description of models and precipitation data, along with the analysis methods, is provided. The results are presented in Section 3, with a summary and discussion given in Section 4. 4

5 64 2 Models and data Description of models The model forecast data analyzed in this study are hindcasts from the National Centers for Environmental Prediction (NCEP) Coupled Forecast System, version 2 (CFSv2) (Saha et al. 2010), the Japanese Meteorological Agency (JMA) One-month EPS (JMA 2013), and the European Center for Medium-range Weather Forecasts (ECMWF) VarEPS-monthly system used for operational monthly forecasts at ECWMF (Vitart et al. 2008). All the three sets of hindcasts are produced based on global EPS forecast systems (see Table 1 for detail). Since these forecast data have 4 5 ensemble members only, our analysis uses the ensemble mean for each of the three model hindcast sets The NCEP CFSv2 (Saha et al. 2014) is a fully coupled dynamical prediction system. The atmospheric component of CFSv2 is a 2007 version of the NCEP global forecast system (GFS) (Saha et al. 2010), with a spectral truncation of 126 waves (T126) in the horizontal and 64 levels in the vertical. The oceanic component is the Geophysical Fluid Dynamics Laboratory (GFDL) Modular Ocean Model version 4 (MOM4) (Griffies et al. 2004). The CFSv2 system increases the length of skillful MJO forecasts from 6 to 17 days (dramatically improving subseasonal forecasts) over its predecessor (Saha et al. 2014) The ECMWF forecasting system is referred to as VarEPS-monthly, which is a merged 32-day ensemble system updated once a week that has been run since 2002 (Vitart 2004). The EPS integration uses atmospheric model at T399 resolution forced by persisted SST anomalies 5

6 84 85 from day 0 to day 10, and then coupled ocean-atmosphere integrations at T255 resolution from day 9 to day 32 (Vitart et al. 2008, Hirons et al. 2013). The ocean component used for the coupled system is the Hamburg Ocean Primitive Equation (HOPE) model. oceanic initial conditions include one control and four perturbed ocean analysis. The Thus, the VarEPS-monthly reforecasts (or hindcasts) consist of a five-member ensemble of 32-day merged integrations for the period. The model s skill at predicting the MJO has improved significantly since 2002, with an average gain of about 1 day of prediction skill per year (Vitart 2004) JMA s One-month EPS consists of an atmospheric GCM (AGCM) run here at T159 resolution with atmospheric initial conditions obtained from the JMA Global Analysis and initial land surface conditions obtained from the JMA Land Surface Analysis System. Persisted SST anomalies are used as the lower boundary condition, together with a climatological distribution of sea ice ( The MJO forecast in hindcasts is skillful up to a lead time of 13 days, but the model fails to reproduce the realistic eastward and northward propagation of active convection (Matsueda and Takaya 2012) As seen in Table 1, the three forecast systems have different model resolution, different frequency of production (ranging from 5-day to 10-day or so), and different physical configuration. However, they all provide daily (6-hourly for CFSv2) forecasts for up to about one month or beyond. Since our analysis is focused on targets over 4 weeks, all the hindcast data of precipitation were truncated at day-28, but spanning over all the available start dates during late May through mid-september. The common period for the three data sets 6

7 is , and the spatial resolution, varying greatly as shown in Table 1, is re-gridded to the resolution of the observed precipitation data set, i.e., 144 x 72 grid points at longitude/latitude. Note that the original horizontal resolution for ECMWF hindcast is T255, equivalent to approximately 80 km, while the JMA hindcast originally has T159 reso- lution, or about 125 km grid. The data sets for these two hindcast sets provided to us were re-gridded onto 1 and 2.5, respectively, as shown in Table The hindcasts used in this study were obtained in early 2102, and correspond to the versions of the oprational systems used at that time. Both the ECMWF and JMA systems have since been updated, while the CFSv2 is fixed. Table 1: Three global EPS dynamical models. The JMA and ECMWF hindcasts start from May 20, and then updated every 10-day or so (i.e., May 20, 31; June 10, 20, 30; and so on), and every 7 days, respectively. The CFSv2 hindcast starts from May 21, and then updated every 5 days. Model Grid Resolution Ensemble Frequency # of Starts Period JMA 144 x /month CFSv2 384 x day ECMWF 360 x weekly

8 Precipitation data The NCEP Climate Prediction Center (CPC) Merged Analysis of Precipitation (CMAP) provides global precipitation data from 1979 onward at 2.5 resolution, using gauge observations, satellite estimates, and numerical model predictions (Xie and Arkin 1996, 1997). We use the CMAP pentad (5-day average) precipitation data during This dataset is used to obtain daily estimates first using interpolation, from which the weekly averages are then calculated to match the model hindcast data Skill metrics The most basic and commonly-used skill metric for verification of seasonal climate forecasts is the anomaly correlation coefficient (ACC), calculated between ensemble mean of model products against observed quantities (e.g., Barnston et al. 2010). In this study, we use the ensemble means of the model precipitation forecasts, averaged over all available start dates (ranging from 13 to 25 during the boreal summer season, varying among the three models) and valid for weeks 1 to 4. The corresponding 4 weekly observations are computed from CMAP pentad mean rainfall data. Thus, the ACC is computed over all the available start dates during late May through mid-september over the period. Note that in order to reduce the impact of annual cycle, the seasonality over the entire period is removed for the model hindcast and observed rainfall, respectively. 133 A second skill score is the mean square skill score (MSSS), which is based on mean squared 8

9 error and often used in verification of deterministic forecasts (e.g., Goddard et al. 2013). The MSSS is a function of both the prediction to be evaluated, a reference prediction (here climatology), and the corresponding observations (Murphy 1988). Thus the overall bulk MSSS value provides a comparison of forecast performance relative to the reference climatological prediction, together with an evaluation of the contribution from the conditional biases in- 139 herent in the forecasts (see Goddard et al. 2013). Due to the very limited ensemble size 140 (4 5) of the three EPS products, no probabilistic skill metrics are examined in this study Results Anomaly correlation coefficient skill (ACC) Spatial distribution over the globe The ACC maps between JMA precipitation hindcast and CMAP observation over the globe (50 S 60 N) are shown in Figure 1. The correlation skill is generally very high during the first week and drops rapidly in most regions in the subsequent 3 weeks. Nevertheless, there is consistently high ACC for all the leads, or weeks 1 4, over the equatorial Pacific, located to the south of the Intertropical Convergence Zone (ITCZ). Weeks 2 4 also exhibit relatively higher skills ( ) over the tropical Atlantic, and the Maritime Continent, along with some land areas and the Indian Ocean at week 1 only. The 13 hindcasts per year over 17 years yields a very large sample size of 221 hindcasts. Using a two-sided Student t-test with 9

10 degrees of freedom, an ACC value of 0.2 is highly statistically significant, with a p-value of Poor ACC values are evident at all leads in the climatologically dry regions of the southeastern tropical Pacific and much of southern tropical Atlantic The ACC distribution for the CFSv2 and ECMWF precipitation forecasts exhibit most of the above characteristics (Figures 2 and 3). However, the ACC skill of ECMWF is notably higher than the other two models, especially over the tropics for weeks 2 4. Indeed, relatively high ACC over the tropical Pacific resembles slightly broader ITCZ-like characteristic than that from JMA or CFSv2. The high ACC feature from the ECMWF precipitation hindcast is also seen over the equatorial Atlantic and tropical Indian ocean, as well as the Maritime Continent (Figure 3), where higher ACC exists compared to the JMA and CFSv2 hindcasts. These dominant characteristics will be further discussed in next section based on aggregate ACC skill over the tropical land area only and several specific regions of interest Skill levels over the continents are generally disappointing especially in weeks 3 4, although the ECMWF model does exhibit substantial skill at week 2 over South America, Eurasia, 166 and Australia. Skill levels over Africa are poor even in week 1 which suggests that the observational data quality may be poor on the weekly timescale. This may be particularly an issue when constructing the weekly averages from the CMAP pentad data Aggregate ACC over specific regions Figure 4 shows the spatially aggregated ACC values for all the three models over the tropical land only within 20 latitudes, represented by percentage of grid points where the ACC 10

11 exceeds a threshold value between 0.1 and 0.9 (Li et al. 2008). Over weeks 1 4, the aggregate skill from ECMWF hindcast is obviously above the other two models, while JMA and CFSv2 illustrate skill close to each other, especially for the first week Over southeast Asia (Figure 5, ranging from E and 10 S 15 N), relatively higher aggregate skill prevails for all the 4 weeks, even though it gradually drops with lead time 177 as well. Again, the summarized skill from ECMWF hindcast outperforms the other two models, particularly for weeks 2 through 4. A small area within this region over Borneo Island will be further investigated in next subsection, along with exploring possible sources of sub-monthly predictability with focus on this area as a case study Sources of sub-monthly predictability While sub-seasonal predictability may arise from various phenomena as discussed in the introduction, our attention here is devoted to the impact of MJO and ENSO. We first examine the boreal summer teleconnections of each in our CMAP dataset globally, and then focus on a small region of southeast Asia where the skill is particularly high, i.e., Borneo Island ENSO and MJO teleconnections Figure 6 shows the anomaly correlation between precipitation from CMAP and ENSO index (defined as SST anomaly over the NINO3.4 region [5 S 5 N, W]) for June-July- 11

12 August during the period. The ACC map for the three-month season, with a 95% significance mask, indicates a pattern of high positive correlations over the equatorial Pacific, in general correspondence to the relatively high ACC of the model hindcasts in Figures 1 3. Also evident is the negative, but strong correlations over the Maritime Continent, as well as the eastern Indian Ocean and the tropical Atlantic. These statistically significant features imply possible important relationships between ENSO and sub-monthly predictability of precipitation. Within-season changes in ENSO teleconnections are quite pronounced in some regions such as the tropical north Atlantic which suggest that there may be within season changes in model skill as well. This issue is left to future study To further examine the impact of ENSO on the skill of sub-monthly precipitation forecasts, we compute the ACC skill separately for 5 ENSO years vs. 5 neutral years based on the JJA Nino 3.4 index. Figure 7 shows ACC maps for the lead-3 ECMWF hindcasts for the two sets of years. The p-value for an ACC of 0.2 is for a sample size of 90 (5 years x 18 hindcasts per year), so the pink shaded areas in Figure 7 are still highly significant. The ACC skill for the ENSO years is indeed generally much higher than for the neutral years, except over the equatorial Atlantic and to some extent the Indian Ocean as well. Modes of tropical Atlantic variability as well as the Indian Ocean dipole may be important contributors to the sub-monthly skill in these regions Figure 8 shows the correlation between CMAP pentad precipitation and the MJO indices defined by (Wheeler and Hendon 2004), namely the first two Empirical Orthogonal Functions (EOFs) of Real-time Multivariate MJO (RMM) series, RMM1 and RMM2. These represent the two leading principal components of intraseasonal near-equatorially-averaged outgoing 12

13 longwave radiation (OLR) and zonal winds. A correlation value of 0.1 has a p-value of for a sample size of 18 samples per JJA season x 17 years. Strong negative correlations exist for both RMM1 and RMM2 over South and Southeast Asia, consistent with the pattern of the summertime MJO (Yoo et al. 2010), also called the boreal summer intraseasonal oscillation (BSISO, Lee et al. 2013). RMM1 also exhibits significant correlations over Central America, the South Atlantic Convergence Zone, and the Gulf of Guinea. Of these regions of significant MJO teleconnections in CMAP precipitation, only SE Asia is a region of significant ACC skill of the three EPS systems beyond week 2, although this MJO influence may partly account for the skill of the ECMWF system in week 2 over Central and South America The Maritime Continent Turning to the Maritime Continent, Figure 9 shows a scatter plot of the CMAP precipitation anomalies, over a portion of Borneo Island (four grid-box centered ranging 1.25 S 1.25 N, E), versus the ECMWF lead-3 hindcasts, valid for 18 weeks from 6/4 6/10 through 10/1 10/7 during the period There is a clear near-linear relationship with an anomaly correlation of The lead-2 hindcasts (valid for weeks 5/28 6/3 through 9/24 9/30) exhibit reduced scatter (not shown), with an ACC of Figure 10 shows the MJO RMM1-RMM2 phase portrait for June-September 2002 as an example of a typical El Niño year. The MJO was very active with amplitude greater than 1.0 standard deviation during most of June August period. A strong MJO event propagated across the Indian Ocean and Maritime Continent (mostly phases 2 3 4) for mid-june. Strong 13

14 MJO events tend to influence the rainfall pattern over the Maritime Continent and Borneo Island in particular. Figure 8 indicates that the negative polarity of both RMM1 and RMM2, in particular of RMM2, are most strongly associated with enhanced CMAP rainfall over Borneo during the June August season, and vice versa. This corresponds approximately to phases 2 3 and 6 7 respectively, which corresponds to the regional labeling in Figure By compositing the pentad CMAP rainfall anomaly over Borneo (averaged for the 4 grid boxes), in relation to the correponding dominant MJO phase (1 to 8) during boreal summer, , we make the MJO phase composites of Borneo-averaged rainfall anomalies, as 240 shown in Figure 11. It provides overall relationship of MJO phase with Borneo CMAP rainfall anomalies. That is, phases 2 3 of MJO correpond to enhanced Borneo rainfall, while phases 6 7 are associated with greatly-reduced precipitation over Borneo. This appears to be in agreement with the correlation between Borneo rainfall and the leading EOFs of MJO (Figure 8), in particular with the second EOF component, RMM We next select three typical years during to represent El Niño (2002), La Niña (1999), and neutral ENSO (2001), respectively, and examine the evolution of CMAP vs the ECMWF hindcasts averaged over Borneo. Figure 12 shows time-series of weekly precipitation from lead-2 (top panel) and lead-3 (bottom panel) of the ECMWF hindcasts and observed rainfall from CMAP, averaged for the 4 grid boxes over Borneo Island in Note that no bias correction of these anomalies was done beyond subtraction of the respective climatology. The verification period is 5/28 6/3 through 9/24 9/30, i.e. 18 weeks in total, for lead-2, and 6/4 6/10 through 10/1 10/7 for lead-3. The upper-case letters, along with the MJO phase number, at the bottom of each panel denote the dominant MJO phase (amplitude 14

15 ) for the corresponding 7-day periods. That is, A for Western Hemisphere and Africa (phases 1 and 8); I for Indian Ocean (phases 2 and 3); M for Maritime Continent (phases 4 and 5); P for Western Pacific (phases 6 and 7). As shown in Figure 12, there is good correspondence of rainfall anomalies between CMAP and the ECMWF hindcasts. For this El Niño year, the CMAP rainfall anomalies are mostly negative as expected, and this is captured by the model at both leads. However, the observed rainfall becomes far-above average for the third week (6/11 6/17), when the MJO is dominantly over the Indian Ocean, or phases 2 to 3, with strength of around 2.0 (Figure 10). This excursion was captured by the model at both leads, although it was too weak, especially at lead For the La Niña year 1999, the weekly rainfall anomalies are mostly positive (Figure 13), in response to the La Niña impact. However, during the week 7/23 7/29, CMAP rainfall was nearly 5.5 mm/day below the average, while the MJO was in phase 5, with a strength of about 1.0. This negative excursion was captured in the model hindcasts, although again with weaker amplitude than observed For the case of neutral ENSO year 2001 (Figure 14), the CMAP rainfall exhibits large intraseasonal fluctuations, quite symmetrically about zero where less precipitation corresponds to MJO phase over Western Pacific (phase 6 or 7) and above-average rainfall tends to be with MJO phases over the Indian Ocean (e.g., phase 2). These rainfall anomalies are remarkably well captured by the ECMWF model at both lead-2 and lead-3, and the attribution of the model s skill to the MJO is clear. Again, the Borneo rainfall anomalies coresponds well with the evolution of MJO (dominant) phases as manifested in Figure

16 Mean squared skill score The anomaly correlation coefficient (ACC) is a scale-invariant measure of the linear association between the predicted mean and the observations (e.g., Murphy 1988). Therefore, it can only provide a measure of relative association or potential skill. The mean squared skill score (MSSS) is another important skill metric used to assess deterministic forecasts, based on the mean squared error, which takes into account both the potential skill and biases inherent in the forecasts (Murphy 1988). Figure 15 shows global maps of MSSS from the JMA hindcasts over the same period and starts as for ACC. In general, substantial skill exists for the first week over most of tropical ocean/land and even the northern mid-latitudes, but the skill drops considerably from week 2 to 4. Similar patterns of MSSS skill can also be seen for the CFSv2 precipitation hindcast (Figure 16), even though it looks slightly lower overall compared to the JMA hindcast. 287 Like the intercomparison of ACC among these three models, the MSSS from ECMWF hind- 288 cast (Figure 17) still indicates better skill than the other two models. In particular, it demonstrates better skill for the first week, persisting over the tropics for week 2 and over the equatorial Pacific and maritime continent through weeks 3 and As mentioned above, the MSSS skill differs from ACC, mainly in the incorporation of forecast biases relative to the ensemble mean. As a summary metric, MSSS combines the square of the correlation coefficient and the square of the conditional prediction bias (Goddard et al. 2013). In other words, the primary difference between ACC and MSSS lies largely in the representation of forecast biases. If the forecast contained no conditional biases, the MSSS 16

17 skill would be determined by the correlation coefficient alone, i.e., their spatial patterns would be identical. Figure 18 shows the difference between the two skill metrics, ACC and MSSS, valid for week 3 from all the three prediction systems. Overall, the CFSv2 hindcast corresponds to largest ACC-MSSS difference, JMA the second, and ECMWF the least out of the three, suggesting that the ECMWF hindcast contains less prediction bias, in general, than the other two models. This comparison also applies to the other leads, i.e., weeks 1, 2 and 4 (not shown) Summary and discussion This study has compared the performance of sub-monthly precipitation hindcasts based on three ensemble prediction systems against objectively analyzed precipitation data. The model predictions valid for four consecutive weeks (weeks 1 4 of the forecasts), composited from 13, 18, and 25 start dates, respectively (Table 1), are evaluated using processed weekly precipitation datasets during late May through mid-september over the 17-year period The prediction skill is quantified using ACC and MSSS deterministic skill metrics over the globe, as well as the tropical lands and several specific regions In general, all the three model hindcast sets indicate very good skill for the first week, and relatively good skill for the 2nd week over the tropics, but dramatically decreased skill for weeks 3 and 4 except for the equatorial Pacific and the maritime continent. Among the three EPS models, the ECMWF forecast system demonstrates noticeably better skill than the other two, especially for weeks 3 and 4. This comparison applies to the prediction skill as 17

18 manifested by both ACC and MSSS, suggesting that the ECMWF hindcast corresponds to not only the best potential skill, but also the combined measure of relative (linear) association and the biases inherent in the forecasts. 319 The predictability of sub-monthly precipitation has been connected with the low-frequency 320 ENSO variability and intra-seasonal MJO phase/strength. Such associations were exam ined qualitatively by selecting three typical years of neutral, El Niño and La Niña events in correspondence with weekly time-series of precipitation anomalies over a portion of the maritime continent (Borneo Island), along with the corresponding dominant phase of MJO. Our results suggest that the rainfall variation for neutral-enso years corresponds well with the dominant MJO phase (e.g., above-average rainfall in response to the phase over Indian Ocean); whereas for the situation with moderate/strong ENSO events, the relationship of rainfall anomaly with MJO appears to become weaker, while the contribution of ENSO to the sub-monthly skill is substantial. However, if a moderate/strong MJO event crosses over the Indian Ocean near the maritime continent (e.g., phases 2 3 with stength of greater than 2.0), the rainfall tends to be far above-average for typical El Niño years, such as For the La Niña year 1999, below-average rainfall is observed when a moderate MJO is over phases 5 6 during late July. In such cases, the MJO impact or modulation becomes significant, regardless of whether the ENSO event is strong or moderate The advantage of multi-model ensemble (MME) combination forecast has been documented by numerous studies, e.g., Robertson et al. (2004), Li et al. (2008). It has been acknowledged that the overall prediction skill from MME products outperforms the best individual model. Our study here is constrained to model-to-model analysis, simply because the start dates are 18

19 non-uniform among the three EPS systems, which makes it dufficult to verify the performance of MME forecast The association of rainfall predictability with ENSO and MJO is only analyzed here for a portion of the maritime continent and for a few typical ENSO years, and our findings may not extend to other continents and years. Further, it is still challenging to quantify the relative contribution of ENSO and MJO to the predictability of precipitation, since ENSO is confined to a fixed location, while the MJO is a propagating wave with variable amplitude and phase. Nonetheless, these results support the concept that windows of opportunity of high forecast skill exist as a function of ENSO and the MJO in certain locations and seasons, that may lead to subseasonal to seasonal forecasts of substantial societal value in the future Acknowledgments: The authors thank Simon Mason and Anthony Barnston for their insights on skill metrics and ENSO index data usage. This work was funded by grants to the IRI from the US Agency for International Development (AID-OAA-A ) and the National Oceanic and Atmospheric Administration (NA050AR ). 19

20 352 References Baldwin and Dunkerton, 2001: Science, 244, Stratospheric harbingers of anomalous weather regimes, Barnston, A. G., S. Li, S. J. Mason, D. G. DeWitt, L. Goddard, and X. Gong, 2010: Veri- fication of the first 11 years of IRI s seasonal climate forecasts. J. Appl. Meteor. Climat., 49, Charney, J. G., and J. Shukla, 1981: Predictability of monsoons. Pp in Monsoon Dynamics, J. Lighthill abd R. P. Pearce, eds. New York: Cambridge University Press Goddard, L., A. Kumar, A. Solomon, D. Smith, G. Boer, P. Gonzalez, V. Kharin, W. Merry- field, C. Deser and S. J. Mason, et al., 2013: A verification framework for interannnual-to- decadal prediction experiments. Clim. Dyn., 40, , doi: /s Griffies, S. M., M. J. Harrison, R. C. Pacanowski, and A. R. Rosat, 2004: A Technical Guide to MOM4. GFDL Ocean Group Technical Report NO. 5, 342 pp. [Available online at files/smg0301.pdf] Hirons L. C. P. Inness, F. Vitart, and P. Bechtold, 2013: Understanding advances in the simulation of intraseasonal variability in the ECMWF model. Part I: The representation of the MJO. Quart. J. Roy. Meteor. Soc., 139, , doi: /qj Holland, M. M., D. A. Bailey, and S. Vavrus, 2011: Inherent sea ice predictability in the rapidly changinf Arctic environment of the Community Climate System Model, version 3, Clim. Dyn., 36, , doi: /s

21 Hoskins, B., 2012: The potential for skill across the range of the seamless weather-climate prediction problem: a stimulus for our science, Quart. J. Roy. Meteor., 138, , doi: /qj Japan Meteorological Agency, 2013: Outline of the Operational Numerical Weather Prediction at the Japan Meteorological Agency, Appendix to WMO Technical Progress Report on the Global Data-Processing and Forecasting System and Numerical Weather Prediction, 188 pp (available online at Koster, R. D., and Coauthors, 2010: Contribution of land surface initialization to subseasonal forecast skill: First results from a multi-model experiment. Gephys. Res. Let., 37, L02402, doi: /2009GL Lee, J. Y., B. Wang, M. C. Wheeler, X. Fu, D. E. Waliser, and I.-S. Kang, 2013: Real-time multivariate indices for the boreal summer intraseasonal oscillation over the Asian summer monsoon region, Clim. Dyn., 40, , doi: /s Li, S., L. Goddard, and D. G. DeWitt, 2008: Predictive skill of AGCM seasonal climate forecasts subject to different SST prediction methodologies. J. Climate, 21, Lin H., and Z. Wu, 2011: Contribution of the autumn Tibetan Plateau snow cover to seasonal prediction of North American winter temperature. J. Climate, 24, Lorenz, E. N., 1975: Climate Predictability. Pp in The Physical Basis of Climate abd Climate Modelling, GAPR Publiaction Series No. 16. Geneva, World Meteorological Organization. 21

22 Mason, S. J., L. Goddard, N. E. Graham, E. Yulaeva, L. Sun, abd P. A. Arkin, 1999: The IRI seasonal climate prediction system and the 1997/98 El Niño event. Bull. Amer. Meteor. Soc., 80, Matsueda, S., and Y. Takaya, 2012: Forecast skill of MJO with the JMAs One-month Ensemble Prediction System. CAS/JSC WGNE Res. Activ. Atmos. Oceanic Model, 42, Moron, V., A. W. Robertson, and M. N. Ward, 2006: Seasonal predictability and spatial coherence of rainfall characteristics in the tropical setting of Senegal. Mon. Wea. Rev., 134, , doi: Murphy, A. H., 1988: Skill scores based on the mean square error and their relationships to the correlation coefficient. Mon. Wea. Rev., 116, National Academy of Sciences 2010 Assessment of Intraseasonal to Interannual Climate Prediction and Prdictability, National Research Council, Washington DC, ISBN-10: X, 193 pp Palmer, T. N. and D. L. T. Anderson, 1994: The prospects for seasonal forecasting. Quart. J. Roy. Meteor. Soc., 120, Robertson, A. W., U. Lall, S. E. Zebiak, and L. Goddard, 2004: Improved combination of multiple atmospheric GCM ensembles for seasonal prediction. Mon. Wea. Rev., 132, Saha, S., and Coauthors, 2014: The NCEP Climate Forecast System Version 2. J. Climate, 27, , doi: 22

23 Saha, S., S. Moorthi, H. Pan, and coauthors, 2010: The ncep climate forecast system reanal- ysis. Bull. Amer. Meteor. Soc., 91, doi: /2010bams Scaife, A. A, and J. R. Knight, 2008: Ensemble simulations of the cold European winter of 2005/6. Quart. J. Roy. Meteor. Soc., 134, Vitart, F., 2004: Monthly forecasting at ECMWF. Mon. Wea. Rev., 32, Vitart, F., 2014: Evolution of ECMWF sub-seasonal forecast skill scores. Quart. J. Roy. Meteor. Soc., doi: /qj Vitart, F., Buizza, M. A. Balmaseda, G. Balsamo, J-R Bidlot, A. Bonet, M. Fuentes, A. Hofstadler, F. Moleni, and T. N. Palmer, 2008: The new VarEPS-monthly forecasting system: A first step towards seamless prediction. Quart. J. Roy. Meteor. Soc., 134, Waliser, D. E., K. M. Lau, W. Stern, and C. Jones, 2003: Potential predictability of the Madden-Julian oscillation. Bull. Amer. Meteor. Soc., 84, Wheeler, M. C., and H. Hendon, 2004: An all-season real-time multivariate MJO index: Development of an index for monitoring and prediction. Mon. Wea. Rev., 132, World Meteorological Organization, 2013: Sub-seasonal to Seasonal Prediction Research Implementation Plan, December 2013, 71 pp (available online at project main page.html) Xie, P. and P. A. Arkin, 1996: Analyses of global monthly precipitation using gauge obser- vations, satellite estimates, and numerical model predictions. J. Climate, 9,

24 : A 17-year monthly analysis based on gauge observations, satellite estimates and numerical model outputs. Bull. Amer. Meteor. Soc., 78, Yoo, J. H, A. W. Robertson, and I. S. Kang, 2010: Analysis of intraseasonal and interannual variability of the Asian summer monsoon using a hidden Markov model. J. Climate, 23, , doi: Zhang, C., 2013: MaddenJulian Oscillation: Bridging weather and climate. Bull. Amer. Meteor. Soc., 94, , doi: 24

25 441 List of Figures Anomaly correlation coefficient between JMA model precipitation hindcast and CMAP rainfall data for weeks 1 4 during the period Same as Figure 1 but for the CFSv2 hindcast Same as Figure 1 but for the ECMWF hindcast Aggregate anomaly correlation coefficient between model precipitation hindcast and CMAP rainfall data for weeks 1 4, represented by percentage of ACC over the tropical land (20 S 20 N) exceeding discrete thresholds. The ACC values were calculated locally and then averaged spatially Same as Figure 4 but for the Southeast Asia ( E, 10 S 15 N) Anomaly correlation between CMAP precipitation and Nino3.4 index for June-July-August, , with a 95% significance mask Anomaly correlation between ECMWF lead-3 precipitation hindcasts (over 18 start dates) and CAMP observation during (a) 5 ENSO years: , 2002; and (b) 5 Neutral years: 1992, 2001, 2003, 2005, Anomaly correlation between CMAP pentad precipitation and 5-day mean Real-time Multi-variate MJO (RMM) components, RMM1 and RMM2, dur- ing June August, , with a 95% significance mask

26 Scatter plot of CMAP weekly precipitation anomaly versus ECMWF hindcast (lead-3) valid for 18 weeks (6/4 6/10 through 10/1 10/7), , over the Borneo Island. The dotted line is a 45 line Real-time Multi-variate MJO (RMM) phase space (Wheeler and Hendon, 2004) for June 01 through September 30, 2002, a typical year of El Niño Composite Borneo-averaged pentad CMAP precipitation anomaly in relation to pentad dominant MJO phase during June-August, Time series of precipitation anomalies (mm/day) over a portion of Borneo Island, from the CMAP data and ECMWF hindcast, valid for weeks 2 and 3 during June September 2002 (typical El Niño). The upper-case letters along with single digit at the bottom of each panel denote the dominant MJO phase sector and phase numer (A, I, M, P corresponding to MJO phases 8 1, 2 3, 4 5, and 6 7, respectively) with amplitude greater than 1.0 (Wheeler and Hendon 2004) Same as Figure 12 but for 1999 (La Niña) Same as Figure 12 but for 2001 (neutral ENSO) Mean square skill score (MSSS) between the JMA model precipitation hind- cast and CMAP rainfall data over weeks

27 Mean square skill score (MSSS) between CFSv2 precipitation hindcast and CMAP rainfall data over weeks Mean square skill score (MSSS) between ECMWF precipitation hindcast and CMAP rainfall data over weeks Difference between ACC and MSSS of precipitation hindcast versus CMAP rainfall data valid for Week-3: (a) JMA, (b) CFSv2, and (c) ECMWF

28 Figure 1: Anomaly correlation coefficient between JMA model precipitation hindcast and CMAP rainfall data for weeks 1 4 during the period

29 Figure 2: Same as Figure 1 but for the CFSv2 hindcast. 29

30 Figure 3: Same as Figure 1 but for the ECMWF hindcast. 30

31 % Grid pts Exceeding "R" % Grid pts Exceeding "R" Tropics: Week-1 JMA CFSv2 ECMWF Tropics: Week Anomaly Correlation % Grid pts Exceeding "R" % Grid pts Exceeding "R" Tropics: Week Tropics: Week Anomaly Correlation Figure 4: Aggregate anomaly correlation coefficient between model precipitation hindcast and CMAP rainfall data for weeks 1 4, represented by percentage of ACC over the tropical land (20 S 20 N) exceeding discrete thresholds. The ACC values were calculated locally and then averaged spatially. 31

32 % Grid pts Exceeding "R" % Grid pts Exceeding "R" SE Asia: Week-1 JMA CFSv2 ECMWF SE Asia: Week Anomaly Correlation % Grid pts Exceeding "R" % Grid pts Exceeding "R" SE Asia: Week SE Asia: Week Anomaly Correlation Figure 5: Same as Figure 4 but for the Southeast Asia ( E, 10 S 15 N). 32

33 Figure 6: Anomaly correlation between CMAP precipitation and Nino3.4 index for June- July-August, , with a 95% significance mask. 33

34 Figure 7: Anomaly correlation between ECMWF lead-3 precipitation hindcasts (over 18 start dates) and CAMP observation during (a) 5 ENSO years: , 2002; and (b) 5 Neutral years: 1992, 2001, 2003, 2005,

35 Figure 8: Anomaly correlation between CMAP pentad precipitation and 5-day mean Realtime Multi-variate MJO (RMM) components, RMM1 and RMM2, during June August, , with a 95% significance mask. 35

36 PCP Anomaly: ECMWF vs CMAP (Lead-3) 10 ECMWF precip anomaly (mm/day) CMAP precip anomaly (mm/day) Figure 9: Scatter plot of CMAP weekly precipitation anomaly versus ECMWF hindcast (lead-3) valid for 18 weeks (6/4 6/10 through 10/1 10/7), , over the Borneo Island. The dotted line is a 45 line. 36

37 4 (RMM1, RMM2) phase space for 1 Jun to 30 Sep Western Pacific 6 RMM West Hem. and Africa Maritime Continent 4 2 Indian Ocean Labeled dots for each day. RMM1 Blue for Jun, orange for Jul, green for Aug, and red for Sep. Figure 10: Real-time Multi-variate MJO (RMM) phase space (Wheeler and Hendon, 2004) for June 01 through September 30, 2002, a typical year of El Niño. 37

38 3 CMAP precipitation anomaly vs MJO phase 2 Precipitation anomaly (mm/day) MJO phase Figure 11: Composite Borneo-averaged pentad CMAP precipitation anomaly in relation to pentad dominant MJO phase during June-August,

39 ECMWF vs CMAP (Lead-2) CMAP ECMWF 5 PCP anomaly 0-5 A8 I2 I3 M5 P6 P7 P7 A8 M4 P6 A Weeks since May ECMWF vs CMAP (Lead-3) 5 PCP anomaly 0-5 I2 I3 M5 P6 P7 P7 A8 M4 P6 A Weeks since May 28 Figure 12: Time series of precipitation anomalies (mm/day) over a portion of Borneo Island, from the CMAP data and ECMWF hindcast, valid for weeks 2 and 3 during June September 2002 (typical El Niño). The upper-case letters along with single digit at the bottom of each panel denote the dominant MJO phase sector and phase numer (A, I, M, P corresponding to MJO phases 8 1, 2 3, 4 5, and 6 7, respectively) with amplitude greater than 1.0 (Wheeler and Hendon 2004). 39

40 ECMWF vs CMAP (Lead-2) CMAP ECMWF 5 PCP anomaly 0-5 P7 A8 A1 A1 M5 I2 A1 A Weeks since May ECMWF vs CMAP (Lead-3) 5 PCP anomaly 0-5 A8 A1 A1 M5 I2 A1 A Weeks since May 28 Figure 13: Same as Figure 12 but for 1999 (La Niña). 40

41 ECMWF vs CMAP (Lead-2) CMAP ECMWF 5 PCP anomaly 0-5 A1 I2 P6 P7 A8 A1 I2 M4 P6 P7 I2 I2 I Weeks since May ECMWF vs CMAP (Lead-3) 5 PCP anomaly 0-5 I2 P6 P7 A8 A1 I2 M4 P6 P7 I2 I2 I Weeks since May 28 Figure 14: Same as Figure 12 but for 2001 (neutral ENSO). 41

42 Figure 15: Mean square skill score (MSSS) between the JMA model precipitation hindcast and CMAP rainfall data over weeks

43 Figure 16: Mean square skill score (MSSS) between CFSv2 precipitation hindcast and CMAP rainfall data over weeks

44 Figure 17: Mean square skill score (MSSS) between ECMWF precipitation hindcast and CMAP rainfall data over weeks

45 Figure 18: Difference between ACC and MSSS of precipitation hindcast versus CMAP rainfall data valid for Week-3: (a) JMA, (b) CFSv2, and (c) ECMWF. 45