Alternative ways to evaluate a seasonal dynamical downscaling system

Transcription

1 JOURNAL OF GEOPHYSICAL RESEARCH: ATMOSPHERES, VOL. 118, 13,443 13,448, doi:1.12/213jd2519, 213 Alternative ways to evaluate a seasonal dynamical downscaling system D. W. Shin 1 and Steve Cocke 1 Received 5 July 213; revised 25 November 213; accepted 4 December 213; published 23 December 213. [1] The value added by dynamically downscaling using a regional climate model, compared to using only a global climate model, is explicitly unveiled by using nontraditional skill evaluation statistics. The conventional model evaluation methods, such as temporal correlation of seasonal average rainfall, cannot always demonstrate the value of dynamically downscaled data. One of our primary alternative metrics for evaluating downscaling methodologies is comparing crop yields simulated using the downscaled data, assuming nonirrigated conditions, to yields simulated using observations. Rainfed crops in the southeast United States are very sensitive to water stress. In fact, these crops are often more sensitive to periods of wet/dry spells than to seasonal rainfall totals. Thus, using crop models as a performance metric provides an alternative to simply evaluating the prediction/simulation of the seasonal mean of some particular meteorological variables and has more practical relevance. Furthermore, while the discovery of climate signals contributing to total summertime precipitation variability remains elusive, our results suggest that dynamical regional models may better simulate intraseasonal variability than simply the anomalies of the seasonal mean, which further justify their usefulness in application models. Citation: Shin, D. W., and S. Cocke (213), Alternative ways to evaluate a seasonal dynamical downscaling system, J. Geophys. Res. Atmos., 118, 13,443 13,448, doi:1.12/213jd Introduction [2] A number of regional climate models have been developed and extensively used to better resolve regional-scale seasonal climate variability from global model data [e.g., Gao et al., 212; Chan and Misra, 211; Nunes and Roads, 27; Cocke et al., 27; Sun et al., 26; Roads, 24; Fennessy and Shukla, 2; Ji and Vernekar, 1997; Giorgi, 199]. If a regional model captures the anomalies of the seasonal mean well at a higher resolution, it is generally considered as a successful dynamical downscaling system. However, most regional models tend not to resolve those anomalies better if its global model counterpart does not have the correct, or detectable, climate signals. Hence, not everyone agrees on the usefulness of downscaling approaches [e.g., Pielke and Wilby, 212]. Some contend that the only improvement a regional model can provide is more detailed spatial representation of weather systems. Is there any value hidden in using a regional model even if it cannot improve the simulation or prediction of anomalies of the seasonal mean? 1 Center for Ocean-Atmospheric Prediction Studies, Florida State University, Tallahassee, Florida, USA. Corresponding author: D. W. Shin, Center for Ocean-Atmospheric Prediction Studies, Florida State University, Tallahassee, FL , USA. (shin@coaps.fsu.edu) 213. American Geophysical Union. All Rights Reserved X/13/1.12/213JD2519 [3] In order to evaluate weather and/or climate model performance, many evaluation metrics have been used (e.g., anomaly correlations, biases, root-mean-square error, Heidke skill score, Brier skill score) and/or developed [e.g., Moise and Delage, 211; Gleckler et al., 28; Taylor, 21]. While many of them are aimed at large-scale diagnostics (e.g., hemispheric modes of variability, assessment of the annual or diurnal cycle), there are other metrics which could more readily reveal the usefulness of a downscaling system. In this study, the authors attempt to discover the added value in a dynamical downscaling system by applying global and regional model data to an agricultural model. 2. Models [4] Seasonal climate simulations are performed for the period using the Center for Ocean-Atmospheric Prediction Studies (COAPS) global and regional modeling system [Shin et al., 25, 26; Cocke and LaRow, 2]. There are two seasonal ensemble simulations per year: one for the summer crop-growing season (March September) and another for the winter season (October March). Each ensemble has a set of 1 members, developed using slightly different atmospheric initial conditions. All simulations use observed weekly sea surface temperatures. Season-long daily climate data (specifically, rainfall, maximum and minimum temperatures, and incoming solar radiation) are archived for analysis and used in a crop model. Whereas the horizontal resolution of the global model employed is T63 (~1.875 ), that of the nested regional model is ~2 km, roughly resolving the 13,443

2 Figure 1. Temporal correlation of the ensemble mean seasonal forecasts of precipitation for (a) JJA and (b) DJF using a global model and for (c) JJA and (d) DJF using a regional model for the period Colored regions indicate statistical significance at approximately 5% level. Dry Seed Yield (kg/ha) Crop Season Rainfall Total (mm) (a) Maize E E L N N E N N N N N E L L N N E N N (b) Rainfall Total Year Figure 2. Simulated (a) maize yields for and (b) their corresponding rainfall totals at Tifton, Georgia (31.5 N, 83.5 W). The observed weather-driven values are denoted by black dots, the global model-driven values by open circles and whiskers, and the regional model-driven values by closed circles and whiskers. Symbol E denotes El Niño; L, La Niña; and N, Neutral year. 13,444

3 . Water Stress El nino 1989 La Nina Day After Planting Figure 3. Simulated water stresses for maize during 1988 (El Niño) and 1989 (La Niña) crop-growing seasons. county scale. The regional model domain is 23 N 37 N, 92 W 76 W, which covers the southeast United States where weather/climate has major effects on agricultural yields. [5] The Decision Support System for Agrotechnology Transfer (DSSAT) version 4. [Jones et al., 23] is used to perform maize yield simulations. Season-long daily climate data from in situ observations and the COAPS global and regional models are used as inputs for the DSSAT crop model for the period at Tifton, Georgia (31.5 N, 83.5 W). Tifton is chosen as a representative location partly because it is located in the middle of the regional model domain and partly because it is one of a few stations where observed daily surface solar radiation data are available. The crop simulations are performed with site-specific soil profiles, fixed fertilizer applications, and rainfed conditions; the planting date for each year is 1 April. The crop model uses the same initial conditions (e.g., soil moisture) for each run. Therefore, the weather/climate input is the only parameter that can change crop yields in a given year Alternative Evaluation Methods [8] An alternative way to expose the value of a regional model might be through the use of application-oriented models, such as a crop model used in this study [Baigorria et al., 27; Shin et al., 21]. Although a crop model provides a single yield value per year, it uses season-long daily climate data, not seasonal average (or total) climate data. This means that the crop yield values implicitly include the high-frequency variability of seasonal climate information (e.g., dry/wet spell sequences) (a) Observation 3. Evaluation Methods 3.1. A Conventional Model Evaluation Method [6] The skill of seasonal simulation of precipitation of the climate model using one of the most commonly used conventional model evaluation methods is shown in Figure 1. The global model skill scores in terms of temporal correlation for JJA and DJF average precipitation are shown in Figures 1a and 1b, respectively. It is well known that many global models have some degree of predictability over the southeast United States during winter because of the strong teleconnection to tropical Pacific Ocean sea surface temperatures [e.g., Cocke et al., 27; Saha et al., 26; Higgins et al., 2]. But, this is not the case for the summer season. There is no detectable skill over the southeast United States in terms of the anomalies of JJA mean. [7] If the same skill evaluation metric is employed for the regional model simulations, we obtain a similar result for the dynamically downscaled seasonal average rainfall fields (see Figures 1c and 1d), which provide more detailed spatial information of the signal at a county scale, but nothing more than that. Hence, we might be tempted to conclude that there is no value added by using a regional model, except for reducing some of the biases originating from the global model. Accumulated Rainfall (mm) El Nino 1989 La Nina (b) Global Model (c) Regional Model Day After Planting Figure 4. Accumulated rainfall amounts for 1988 (El Niño) and 1989 (La Niña) crop-growing seasons from (a) observation, (b) a global model, and (c) a regional model. 13,445

4 LGMI (mm) El Nino 1989 La Nina (a) Observation (b) Global Model (c) Regional Model Day After Planting Figure 5. Same as Figure 4 but for the Lawn and Garden Moisture Indies (LGMI). [9] Figure 2 shows the simulated maize yields using observed, global, and regional model data and their corresponding rainfall totals for Tifton, Georgia (31.5 N, 83.5 W), for the period Figure 2b is a kind of onedimensional version of Figure 1. Since there is almost no temporal correlation for either the global (r =.32) or regional (r =.37) model rainfall totals with observed total, we could conclude that there is no interannual predictability over this location, especially during the summer season. However, we reach a different conclusion if we evaluate temporal correlations in terms of maize yield amounts (Figure 2a). Whereas the global-model-based maize yields are less correlated with the observed yields (r =.128), the regionalmodel-based maize yields are well correlated with the observed (r =.45, statistically significant at the 5% level). If the year 25 is excluded, the correlation becomes.72. The year 25 was unique in that tropical cyclone activity began very early that year and was a record year for Atlantic hurricane activity overall. There were at least three tropical storms which passed near Tifton, GA and produced significant amount of rain. Currently, while some climate models can predict overall seasonal Atlantic activity [e.g., LaRow et al., 21; Vitart, 26], they cannot accurately forecast landfalling tropical cyclone activity (let alone associated precipitation) months in advance. Thus, we cannot expect these models to predict the variance of crop yields that could be related to tropical cyclone rainfall [1] Climatological average yield amounts are also better produced by the regional model than by the global model data. The underestimation of yield amounts by the global model is most likely because of the greater frequency of rain days and the lesser amount of daily rainfall produced by the model. Further analyses are shown below to determine why the regional model performs better than the global model in the yield simulations. [11] To reveal the hidden value of seasonal dynamical downscaling, we examine an interesting finding shown in the maize yields and rainfall totals for an El Niño year (1988) and a La Niña year (1989). Even though the observed rainfall totals are similar (Figure 2b, ~55 mm) for both years, the maize yields (Figure 2a) are substantially different between year 1988 (4548 kg/ha) and year 1989 (9644 kg/ha). The difference can be explained by examining the water stresses on the maize. Water stress in a crop model is commonly computed by combining many variables such as rainfall, evaporation, soil moisture, runoff, and crop physiology. The crop model provides the daily values of water stress during the crop-growing seasons. The time series of simulated maize water stresses for 1988 and 1989 are shown in Figure 3: the maize experienced long-lasting water stress in 1988, but a brief spell of water stress in This explains why the maize yields in 1989 are almost twice those in A similar argument can be made for the years 1998 and 1999 (not shown). [12] Can we also justify the above conclusion using rainfall data only and not using a crop model? This is partly possible by evaluating the time series of accumulated rainfall. Figure 4a shows the observed accumulated rainfall amounts in Tifton, Georgia, for 1988 and 1989: the higher the slope, the greater the amount of rainfall. We can also infer water availability from the slopes of accumulated rainfall during the crop growing season. Season-long rainfall totals are similar for both years, but the rainfall histories from planting days to harvest days are entirely different. Figure 6. Scatter diagram of correlations of precipitation versus correlations of yields for 1 ensemble members. The correlations are computed by model precipitation against observed precipitation (days 5 85) and by model yields versus observed yields. 13,446

5 Table 1. Variance of Precipitation First 1 Days After Planting STDEV1 STDEV2 <15 >5 Total Cases Observation a 19 Regional model Global model STDEV1: standard deviation across years, averaged over ensemble members. STDEV2: standard deviation across ensemble members, averaged over 19 years. <15: occurrences of less than 15 mm accumulation during the 1 days. >5: occurrences of greater than 5 mm accumulation during the 1 days. a = if 25 is excluded for >5; STDEV1 is if 25 is included. [13] The corresponding time series of accumulated rainfall from the global and the regional models are shown in Figures 4b and 4c, respectively. A simple comparison of these two figures clearly reveals the added value of the regional model relative to the global model. It is evident that the global model rainfall amounts are too small to provide enough water to the crop for the first 1 days after planting in 1989 and for the whole season in This explains why the global-model-based maize yields are much lower than the observed-based yields (Figure 2a). Meanwhile, the slopes (or rainfall histories) of 1988 and 1989 are relatively well reproduced from the regional model, but the seasonal total rainfall amounts are relatively not. These similar slope patterns explain the better maize yield simulations by the regional model in Figure 2a. [14] There are potentially more useful and relevant metrics that one can use for predicted rainfall data to evaluate model performance, especially for agricultural applications. The Lawn-and-Garden Moisture Index (LGMI), one of several drought indices, can be used to express the water availability (soil moisture) during the crop-growing season. The LGMI was developed by the Alabama Office of State Climatology to estimate the capacity of current soil moisture to sustain healthy lawns and gardens. The LGMI is a function of rainfall only. The current index is computed on the basis of how much rainfall occurred in the past 21 days in a specific location. All rainfall during the previous seven-day period is considered to be equally important, but rainfall before that time is discounted according to a linear sliding scale. A detailed computational procedure can be found in Christy [24]. It should be noted that if the LGMI is less than 5 mm, crops begin to experience water stress (dry conditions), but if it is more than 5 mm, crops have sufficient soil moisture to grow. Figure 5 shows the LGMI time series for the 1988 and 1989 crop-growing seasons from (a) observations, (b) the global model, and (c) the regional model, respectively. The problem of the global model rainfall is clearly presented in terms of LGMI (Figure 5b): conditions are too dry for crops to grow. This problem is substantially reduced by the regional model. Although the detailed time evolution of LGMI is quite different from the observation, the regional model captures the overall observed signal reasonably well. Therefore, we can conclude that this derived statistic is a useful tool that can reveal the hidden value of the regional model for downscaling purposes. 4. Discussion [15] As noted above, the total seasonal precipitation was a poor predictor of yield. To examine this issue further, we used a brute force search method and found that the accumulated precipitation for days 5 through 85 after planting (i.e., the reproductive period for maize) had the highest correlation against yield, around.8. Thus, if a model can accurately predict the precipitation during this time period, there is a chance that the model may predict the correct yield. In Figure 6, we show a scatterplot of the model correlation of predicted Day 5 85 precipitation (against observed) versus the correlation of model predicted yield (against observed) for 1 ensemble members each of the global and regional model simulations. These results show that the skill of an ensemble member s prediction of yield is commensurate with the member s prediction of precipitation for the regional model, at least as measured by correlation. For the global model ensembles, the relationship is less distinct. These results also show that the regional ensembles more frequently have higher skill than the global model, with four members having yield correlations higher than.45. However, while the set of ensemble members, as well as the ensemble mean of the regional model, have higher skill, the correlation of the global and regional ensemble mean for Day 5 85 precipitation is very similar,.64 versus.62, when 25 is excluded. Thus, we have what appears to be a paradox, in that while the precipitation totals for Days 5 85 appear to be well-predicted by both the global and regional models, the regional model has good skill at predicting yield (25 notwithstanding), whereas the global model did not. Further analysis indicates this paradox can be explained by exaggerated variability of precipitation of the global model. Table 1 shows the standard deviation across ensemble members (STDEV1) and across years (STDEV2) and counts of very extreme events of <15 mm and >5 mm accumulated precipitation in the first 1 days after planting. It can be seen that the variability of the regional model is much closer to observed, and there are fewer extreme events. By extreme event, we mean events that are outside the range that has been observed over the time period While some extreme events outside the observed range are to be expected in a large number of ensembles, the number produced by the global model appears to be too high, leading to an excessive number of incorrect crop yields. 5. Conclusion [16] This study re-evaluated the performance of a seasonlong regional downscaling model compared to that of a global climate model using nonconventional skill metrics. The conventional model evaluation methods, such as temporal correlation of seasonal average rainfall, often cannot reveal the value of dynamically downscaled data. The added value of using the regional model for downscaling was better uncovered by using a crop model as a forecast evaluation tool because the crop yield data include the high-frequency variability of seasonal climate (e.g., dry/wet spell sequences). Compared to the global model, the regional model better simulated intraseasonal variability than the anomalies of the seasonal mean. In addition, the value of the regional model was also exposed by high-frequency statistics, e.g., the time series of accumulated rainfall and Lawn-and-Garden Moisture Index, which are derived from rainfall data only. [17] This study clearly shows why a dynamical downscaling system could be useful for application models. The resolution 13,447

6 of the state-of-the art global models (e.g., those used in the Coupled Model Intercomparison Project 5 [Taylor et al., 212]) still might be too coarse to capture the signals of regional climate variability. Hence, dynamical downscaling might remain an essential step to use global climate data in application models. It would be interesting to compare statistical downscaling methods with dynamical downscaling methods using the framework presented here. [18] It is worthwhile to mention that evaluation of downscaling systems in the context of applications is an important concept, not only for assessing the value added of downscaling, but also because the applications reflect the impact of climate variability and change on society. In essence, it is not the crop model application that matters, but instead identification of the climatological factors that influence crop development and assessment of the model (or downscaled model) in terms of those specific factors. In this sense, these results demonstrate that existing metrics do not tell the whole story in terms of value added or impact assessment. [19] Acknowledgment. This research was supported via a USDA/ NIFA-funded collaborative EaSM project (FLAW ). References Baigorria, G. A., J. W. Jones, D. W. Shin, A. Mishra, and J. J. O Brien (27), Assessing uncertainties in crop model simulations using daily biascorrected regional circulation model outputs, Clim. Res., 34, , doi:1.3354/cr73. Chan, S., and V. Misra (211), Dynamic downscaling of the North American Monsoon with the ECPC-NCEP regional spectral model from NCEP CFS global model-varied representation, J. Clim., 24, Christy, J. (24), The lawn-and-garden moisture index. Huntsville, Al.: Alabama Office of the State Climatologist, National Space Science and Technology Center, University of Alabama. Available at: uah.edu/aosc/lawn_garden2.htm. Cocke, S., and T. E. LaRow (2), Seasonal predictions using a regional spectral model embedded within a coupled ocean atmosphere model, Mon. Weather Rev., 128, Cocke, S., T. E. LaRow, and D. W. Shin (27), Seasonal rainfall predictions over the southeast United States using the Florida State University nested regional spectral model, J. Geophys. Res., 112, D416, doi:1.129/ 26JD7535. Fennessy, M. J., and J. Shukla (2), Seasonal prediction over North America with a regional model nested in a global model, J. Clim., 13, Gao, X., Y. Shi, D. Zhang, J. Wu, F. Giorgi, Z. Ji, and Y. Wang (212), Uncertainties in monsoon precipitation projections over China: results from two high-resolution RCM simulations, Clim. Res., 52, , doi:1.3354/cr184. Giorgi, F. (199), Simulation of regional climate using a limited area model nested in a general circulation model, J. Clim., 3, Gleckler, P. J., K. E. Taylor, and C. Doutriaux (28), Performance metrics for climate models, J. Geophys. Res., 113, D614, doi:1.129/ 27JD8972. Higgins, R. W., A. Leetmaa, Y. Xue, and A. Barnston (2), Dominant factors influencing the seasonal predictability of U.S. precipitation and surface air temperature, J. Clim., 13, Ji, Y., and A. D. Vernekar (1997), Simulation of the Asian summer monsoons of 1987 and 1988 with a regional model nested in a global GCM, J. Clim., 1, Jones, J. W., et al. (23), The DSSAT cropping system model, Eur. J. Agron., 18, LaRow, T. E., L. Stefanova, D. W. Shin, and S. Cocke (21), Seasonal Atlantic tropical cyclone hindcasting/forecasting using two sea surface temperature datasets, Geophys. Res. Lett., 37, L284, doi:1.129/ 29GL Moise, A. F., and F. P. Delage (211), New climate model metrics based on object-orientated pattern matching of rainfall, J. Geophys. Res., 116, D1218, doi:1.129/21jd Nunes, A. M. B., and J. O. Roads (27), Dynamical influences of precipitation assimilation on regional downscaling, Geophys. Res. Lett., 34, L16817, doi:1.129/27gl3247. Pielke Sr., R. A., and R. L. Wilby (212), Regional climate downscaling what s the point?, Eos Forum, 93(5), 52 53, doi:1.129/212eo58 Roads, J. (24), Experimental weekly to seasonal US forecasts with the regional spectral model, Bull. Am. Meteorol. Soc., 85, Saha, S., et al. (26), The NCEP Climate Forecast System, J. Clim., 19, Shin, D. W., S. Cocke, T. E. LaRow, and J. J. O Brien (25), Seasonal surface air temperature and precipitation in the FSU climate model coupled to the CLM2, J. Clim., 18, Shin,D.W.,J.G.Bellow,T.E.LaRow,S.Cocke,andJ.J.O Brien (26), The role of an advanced land model in seasonal dynamical downscaling for crop model application, J. Appl. Meteorol. Climatol., 45, Shin, D. W., G. A. Baigorria, Y.-K. Lim, S. Cocke, T. E. LaRow, J. J. O Brien, and J. W. Jones (21), Assessing maize and peanut yield simulations with various seasonal climate data in the southeastern United States, J. Appl. Meteorol. Climatol., 49, Sun, L., H. Li, S. E. Zebiak, D. F. Moncunill, F. D. A. D. S. Filho, and A. D. Moura (26), An operational dynamical downscaling prediction system for Nordeste Brazil and the 22 4 real-time forecast evaluation, J. Clim., 19, , doi:1.1175/jcli Taylor, K. E. (21), Summarizing multiple aspects of model performance inasinglediagram, J. Geophys. Res., 16(D7), , doi:1.129/ 2JD9719. Taylor, K. E., R. J. Stouffer, and G. A. Meehl (212), An overview of CMIP5 and the experiment design, Bull. Am. Meteorol. Soc., 93, , doi:1.1175/bams-d Vitart, F. (26), Seasonal forecasting of tropical storm frequency using a multi-model ensemble, Quart. J. Roy. Meteorol. Soc., 132, ,448