Effect of Considering Sub-Grid Scale Uncertainties on the Forecasts of a High-Resolution Limited Area Ensemble Prediction System

Transcription

1 Pure Appl. Geophys. 174 (2017), Ó 2017 Springer International Publishing DOI /s Pure and Applied Geophysics Effect of Considering Sub-Grid Scale Uncertainties on the Forecasts of a High-Resolution Limited Area Ensemble Prediction System SEHYUN KIM 1 and HYUN MEE KIM 1 Abstract The ensemble prediction system (EPS) is widely used in research and at operation center because it can represent the uncertainty of predicted atmospheric state and provide information of probabilities. The high-resolution (so-called convection-permitting ) limited area EPS can represent the convection and turbulence related to precipitation phenomena in more detail, but it is also much sensitive to small-scale or sub-grid scale processes. The convection and turbulence are represented using physical processes in the model and model errors occur due to sub-grid scale processes that were not resolved. This study examined the effect of considering sub-grid scale uncertainties using the high-resolution limited area EPS of the Korea Meteorological Administration (KMA). The developed EPS has horizontal resolution of 3 km and 12 ensemble members. The initial and boundary conditions were provided by the global model. The Random Parameters (RP) scheme was used to represent sub-grid scale uncertainties. The EPSs with and without the RP scheme were developed and the results were compared. During the one month period of July, 2013, a significant difference was shown in the spread of 1.5 m temperature and the Root Mean Square Error and spread of 10 m zonal wind due to application of the RP scheme. For precipitation forecast, the precipitation tended to be overestimated relative to the observation when the RP scheme was applied. Moreover, the forecast became more accurate for heavy precipitations and the longer forecast lead times. For two heavy rainfall cases occurred during the research period, the higher Equitable Threat Score was observed for heavy precipitations in the system with the RP scheme compared to the one without, demonstrating consistency with the statistical results for the research period. Therefore, the predictability for heavy precipitation phenomena that affected the Korean Peninsula increases if the RP scheme is used to consider sub-grid scale uncertainties in forecasting precipitation phenomena using the high-resolution limited area EPS of KMA. 1 Atmospheric Predictability and Data Assimilation Laboratory, Department of Atmospheric Sciences, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul 03722, Republic of Korea. khm@yonsei.ac.kr 1. Introduction The ensemble prediction system (EPS) has been used in various operation centers because it can represent the uncertainty of predicted atmospheric state and provide information of probabilities (Palmer et al. 1997; Toth and Kalnay 1997; Marsigli et al. 2005; Bowler et al. 2008; Montani et al. 2011). Along with recent advancements on computer computability, the resolution for EPS is increasing. Especially, higher resolution holds a great advantage in that it can depict structure of convection related to the precipitation phenomena in more detail compared to the low-resolution models (Kain et al. 2008; Weisman et al. 2008; Schwartz et al. 2009). The high-resolution EPS is mainly applied to regional domain and, thus, the effects of not only initial condition but also boundary condition on forecast uncertainties must be considered. According to Peralta et al. (2012) when comparing between a system that incorporates initial condition uncertainty and one that does not, skill score was more accurate for a system that incorporates uncertainty during the early lead times but as the lead time increases, the two systems showed similar results. In a similar manner, it was reported in Vié et al. (2011) and Kühnlein et al. (2014) that initial conditions are greatly influential in the early lead times for limited area EPS but as the lead time increases, the effects of boundary conditions become larger. Moreover, it has been observed in Hou et al. (2001) and Saito et al. (2012) that incorporating the uncertainty of boundary conditions greatly improves the ensemble spread for the regional model. Caron (2013) has shown through case studies that unrealistic perturbation may occur in forecasting surface pressure when there is a

2 2022 S. Kim and H. M. Kim Pure Appl. Geophys. difference between the ensemble members of boundary and initial conditions provided by the global EPS. Aside from uncertainties associated with initial and boundary conditions, there is model error. The uncertainty due to model error is considered especially significant for high-resolution model (Bouttier et al. 2012). This is because although high-resolution model can better simulate the convection and turbulence than low-resolution model, it is also much sensitive to small scale or sub-grid scale processes (Fritsch and Carbone 2004). The convection and turbulence are simulated using physical processes in the model and during this process nonlinear model error occurs due to sub-grid scale processes that were not resolved. This uncertainty due to model error in high-resolution model may greatly affect the forecast. Various studies have been conducted to assess the forecast uncertainty due to model error using highresolution limited area EPS, typically for precipitation forecast. Baker et al. (2014) used the Random Parameters (RP) scheme which is one of the stochastic physics schemes to account model error due to uncertainty in parameterization of sub-grid scale process, and analyzed its effect. Moreover, while verifying precipitation cases, Baker et al. (2014) showed that although considering model error by RP scheme has positively influenced the ensemble spread of near surface temperature and relative humidity, it has negative influence on performance and spread of precipitation forecast. Gebhardt et al. (2011) also used the stochastic physics scheme, and analyzed the precipitation forecasts of 15-day period that represent the uncertainty generated by the physical processes of the model. It has been observed that during the first 12 h the spread of precipitation forecast increased due to uncertainty of the physical processes; however afterwards, uncertainty of the boundary conditions was found to be more dominant. Using a similar research setup for period from May 1st to August 15th 2011, Kühnlein et al. (2014) examined the effect of model error in physical processes on forecasts in a German region. It has been observed that the model error typically has significant effects during convectively active time and significantly influenced the forecast compared to the uncertainties of initial and boundary conditions when the synoptic forcing is weak. In the Korea Meteorological Administration (KMA), the limited area ensemble forecasts are implemented using initial and boundary conditions downscaled from the global EPS. Kim et al. (2015) explained construction process of the high-resolution limited area EPS of the KMA and assessed the general performance and precipitation forecast of the system according to its resolution and number of ensemble members. However, the system developed by Kim et al. (2015) did not consider model errors. This study applied the RP scheme (Bowler et al. 2008) to the high-resolution limited area EPS of the KMA and examined the effect of model error on forecasts of high-resolution limited area EPS for meteorological phenomena occurring in the Korean Peninsula. The systems with and without RP scheme were developed and the effect of model error on the forecasts of meteorological phenomena in the Korean Peninsula was investigated by examining the differences between various skill scores. Especially to examine the effect of model error on precipitation forecast, quantitative and probabilistic tests about the two rainfall cases that occurred during the research period were conducted. Section 2 presents the experimental framework, which includes developed EPS and the RP scheme. Section 3 describes verification methods, and Sect. 4 presents the results. Finally, Sect. 5 provides a summary and discussion. 2. Methodology 2.1. The High-Resolution Limited Area Ensemble Prediction System For this study, the high-resolution limited area EPS was developed based on the KMA Unified Model (UM) version 8.2. The KMA UM version 8.2 is the nonhydrostatic model using a horizontally staggered Arakawa C grid and the horizontal grid is rotated equatorial latitude longitude. The vertical coordinate system is terrain-following hybrid-height coordinate with Charney Phillips grid staggering (Davies et al. 2005). Parameterizations of physical processes in the developed system include large-scale

3 Vol. 174, (2017) Effect of Considering Sub-Grid Scale Uncertainties on the Forecasts 2023 precipitation (Wilson and Ballard 1999) and boundary-layer mixing (Lock et al. 2000). The large-scale precipitation in the UM corresponds to microphysics scheme. Since the resolution of the system is 3 km, convection (i.e., cumulus) parameterization (Gregory and Rowntree 1990) was not used. But the global EPS which provides initial and boundary conditions used convection parameterization and gravity-wave drag scheme (Webster et al. 2003). The developed high-resolution limited area EPS has grid points horizontally and 70 vertical eta-height hybrid levels with a top at approximately 40 km. The limited area domain covers the Korean Peninsula and this is nested from the global EPS (Fig. 1). The initial and boundary conditions are provided from the global EPS that is operationally used in the KMA, and the detailed processes are shown in Fig. 2. The system shown in Fig. 2 is similar to the system (Fig. 1 in Kim et al. 2015) used in Kim et al. (2015) but it is different in that the initiation time of the developed system is at T? 3 not T? 0, resolution of global ensemble is N320 not N400, and the number of ensemble members also differs. The global EPS of the KMA uses Ensemble Transform Kalman Filter (ETKF, Bowler et al. 2008; Kay et al. 2013; Kay and Kim 2014) to produce 24 ensemble members including the control member. Among these global ensemble members, 12 members are randomly selected and their 3-h forecasts (T? 3) are used as initial conditions for high-resolution limited area EPS. The boundary conditions produced by the selected global ensemble members are used as boundary conditions for the corresponding limited area ensemble members. Moreover, by using the initial and boundary conditions produced from the same ensemble members correspondingly, it is possible to maintain the balance between the initial and boundary conditions (Caron 2013). Since the global ensemble resolution is approximately 40 km at the midlatitude, boundary conditions can lead to noise due to sudden resolution change from global to local ensemble. To relax this noise, variable grid system used in Davies (2014) was used to the high-resolution EPS and boundary conditions were made to apply using gradual resolution change from 40 to 3 km. Since the 3-h forecasts of the global EPS are used as initial conditions for the limited area EPS, actual forecast time is 21 h in limited area EPS. The EPS run started at every 00 and 12 UTC at 12-h intervals The Random Parameters (RP) Scheme In this study, the RP scheme was used to represent the effect of model error. The RP scheme was designed to consider the model error due to uncertainty in parameterization about the sub-grid scale process. Fundamental concepts of the RP scheme are as the following, referenced from mathematical formulas in Bowler et al. (2008) and Baker et al. (2014). The RP scheme considers parameters of various parameterization schemes as stochastic variables and each parameter is calculated using firstorder auto-regression model: P t ¼ l þ rðp t 1 lþþe; ð1þ Figure 1 Model domain and locations of Automatic Weather Station (red dot) used in the verification of ensemble forecasts. The box indicates the region that skills scores, calculated by interpolating AWS observation and model forecasts where P t 1 is the parameter value at t-1 and P t represents the parameter value at t updated by P t 1, l means the standard value of the parameter, and r is the auto-correlation constant which was set as 0.95 in this experiment. e is a shock term and serves to apply the random numbers k 2 ½0; 1Š to Eq. (1) and calculated as:

4 2024 S. Kim and H. M. Kim Pure Appl. Geophys. Figure 2 Schematic of the limited area ensemble prediction system e ¼ð2k 1Þ ðp max P min Þ ; ð2þ 3 where P max and P min denote the maximum and minimum values, respectively, and are set to prevent unrealistic values from the parameters generated using random numbers. Due to P max and P min, P t has a critical value and the finally applied P t is: 8 < P t ¼ : P max P min P t if P t [ P max; if P t \P min; otherwise: ð3þ In this experiment, the RP scheme was applied to large-scale precipitation and boundary-layer mixing processes, and the applied parameters and their maximum and minimum values are listed in Table 1. The maximum and minimum values shown in Table 1 followed general standard setting of the RP scheme and they were set by numerous estimations (Bowler et al. 2008). After implementing various experiments using the standard and modified settings on the precipitation phenomena of the Korean Peninsula, it has been confirmed that the standard setting shows the most consistent and positive results (Kim et al. 2014). In the RP scheme, nine parameters are updated every 3 h and are applied to the forecast. In this experiment, the high-resolution limited area EPS without the RP scheme (CNTL) and the system with the RP scheme (EXP_RP) were constructed, and forecast results of the two systems were compared to examine how model error affects the fundamental performance and precipitation forecast of the EPS. 3. Verification 3.1. Root Mean Square Error (RMSE) and Spread The ensemble mean and ensemble spread are the most standard verification score in the ensemble prediction verification. Probabilistic distribution of the predicted future atmosphere through nonlinear integration can be estimated by examining the mean of ensemble forecast. As for ensemble spread, it represents deviation of the predicted ensemble members from average and can be calculated as: vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u 1 X N spread ¼ t ðf ðiþ f Þ 2 ; ð4þ N 1 where N is the number of ensemble members, f is f ¼ 1 P N N i¼1 f ðiþ, which represents the average of the ensemble forecasts, and f ðiþ is the forecast of ith ensemble member. i¼1

5 Vol. 174, (2017) Effect of Considering Sub-Grid Scale Uncertainties on the Forecasts 2025 Table 1 Parameters used in the RP scheme Scheme Parameter Name Min Default Max LSP Rhc Critical relative humidity m_ci Parameter controlling ice fall speed BL par_mezcla Neutral mixing length g0 Flux profile parameter charnock Charnock parameter lambda_min Minimum mixing length ricrit Critical Richardson number a_ent_1 Entrainment parameter A g1 Cloud-top diffusion control LSP large-scale precipitation, BL boundary-layer mixing Root Mean Square Error (RMSE), which is the difference between the ensemble mean and true state of atmosphere, can be calculated as: vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u 1 X M RMSE ¼ t ðf i o i Þ 2 ; ð5þ M where M is the number of forecast and observation pairs used for verification and o denotes the verification value. In this study, a total of 687 AWS observation values were used for verification values. The accuracy of forecast can be estimated to a certain degree using the ensemble mean and ensemble spread. In general, it can be comprehended that if the ensemble spread is small, the uncertainty is also small, making the ensemble mean closer to the actual atmosphere with smaller RMSE. Whereas if the ensemble spread is large, it can be estimated that the uncertainty relatively increases, making the RMSE large. However, on certain occasions, obtaining the perfect correlation such as this is impossible (Barker 1991). This is because the RMSE can be small even when the ensemble spread is large and RMSE can also be large when the spread is small. Therefore, it can be said that the performance of EPS is reliable when the ensemble spread appropriately represents the error in ensemble prediction and the RMSE is small (Kim et al. 2015). i¼ Equitable Threat Score (ETS) and Frequency Bias Index (FBI) Equitable threat score (ETS) (Schaefer 1990; Wilks 2006) is a well-known deterministic verification score for quantitative precipitation verification Table 2 Contingency table used to calculate ETS and FBI Contingency table Observed Yes No Forecast Yes a (hits) c (false alarms) (Ebert 2001; Jones et al. 2007; Bouallègue et al. 2013; Kühnlein et al. 2014). The ETS was calculated using the method in Wilks (2006) as: a a ref ETS ¼ a a ref þ b þ c ; ð6þ where a ref ¼ðaþbÞða þ cþ=ða þ b þ c þ dþ and explanation about a, b, c, and d is shown in the contingency table in Table 2. The ETS ranges between -1/3 and 1. When the forecast is perfect, a value of ETS is 1 and the forecast performance is none-existent when a value of ETS is negative. Frequency Bias Index (FBI) (Schaefer 1990; Wilks 2006) is also one of the quantitative precipitation verification methods which can determine the degree of bias. The FBI was also calculated using the formula in Wilks (2006) and variables listed in the contingency table of Table 2 and calculated as: FBI ¼ a þ c a þ b : ð7þ The FBI ranges from 0 to infinity. A value of FBI is 1 means perfect forecast, larger than 1 means the No b (misses) d (correct negatives)

6 2026 S. Kim and H. M. Kim Pure Appl. Geophys. forecast has overestimated relative to the actual observation, and smaller than 1 means the forecast has underestimated relative to the actual observation Brier Skill Score (BSS) Advantage of the EPS is that it can provide probabilistic information. Brier Score (BS) is a wellknown method to estimate the accuracy of the probabilistic forecast (Brier 1950; Wilks 2006). The BS is a method that calculates the RMSE average between probabilistic forecast and observation using dichotomous method and calculated as: BS ¼ 1 M X M i¼1 ðh i ðlþ o i ðlþþ 2 ; ð8þ where M represents the number of forecast and observation pairs used for the BS calculation, l represents threshold of the variable, h i ðlþ denotes the forecast probability of the variable for threshold l, and o i ðlþ is 1 when the observation is to exceed threshold l and 0 when it is not. Therefore, the BS ranges between 0 and 1. It is 0 when there was a perfect probabilistic forecast and 1 when the forecast has failed. The BS of the two systems obtained from Eq. (8) can be used to calculate Brier Skill Score (BSS) as: BSS ¼ 1 BS EXP RP : ð9þ BS CNTL The BSS ranges between -1 and 1 and it helps relatively analyze the BS values of the two systems. If the BSS is a positive value, probabilistic information of the EXP_RP system is more accurate. If it is a negative value, probabilistic information of the CNTL system is more accurate Normalized Variance Difference (NVD) To evaluate the relative degree of ensemble variance of the two ensemble systems, Normalized Variance Difference (NVD: Gebhardt et al. 2011) was calculated as: NVD ¼ r2 EXP RP ðk; tþ r2 CNTL ðk; tþ r 2 EXP RP ðk; ; ð10þ tþþr2 CNTL ðk; tþ where r 2 EXP RP ðk; tþ and r2 CNTL ðk; t Þ represent variance of the K variable at t forecast time for EXP_RP and CNTL systems, respectively. If the NVD is a positive value, it indicates the RP scheme having influence in increasing the ensemble variance. In contrast, if it is a negative value, it indicates the RP scheme having influence in decreasing the ensemble variance. In this study, variable K represented the hourly precipitation accumulation. The NVD was calculated depending on the forecast lead time to examine how the RP scheme influences the variance of precipitation. 4. Results 4.1. Effect on RMSE and Ensemble Spread Two kinds of high-resolution limited area ensemble predictions, CNTL and EXP_RP, were conducted for the month of July, 2013 to examine the effect of model error. Both AWS observations and model forecast were interpolated to 15 km resolution for verification. The interpolation method of Shepard (1968), which complements the shortcomings of the pure inverse distance weighting method and changes irregularly distributed data to a continuous field, was used. The reason for interpolating 3 km high-resolution model forecast as 15 km resolution is to minimize the double penalties due to displacement error often occurring in highresolution limited area model results, as mentioned in Ebert (2008). To minimize the double penalties, Caron (2013) and Baker et al. (2014) verified forecast results by interpolating 1.5 km resolution to 15 and 13.5 km, respectively. To examine the effect of the RP scheme on near surface variables that typically were verified in highresolution model forecasts, RMSE and spread were calculated for 24-h forecasts of 1.5 m relative humidity, 1.5 m temperature, and 10 m zonal wind (Fig. 3). For the one month experimental period, both the two systems showed that the spread was around 40% smaller than the RMSE in three variables. Examining closely at each of the variables, the RMSE of EXP_RP was 1.24% larger than the RMSE of CNTL in average for 1.5 m relative humidity and was

7 Vol. 174, (2017) Effect of Considering Sub-Grid Scale Uncertainties on the Forecasts 2027 Figure 3 a, c, e RMSE and b, d, f spread of 1.5 m relative humidity, 10 m zonal wind, and 1.5 m temperature, respectively, for the 24-h forecast of CNTL (black solid) and EXP_RP (red dashed) during July and 0.39% smaller for 10 m zonal wind and 1.5 m temperature, respectively. The spread decreased 0.37 and 4.37% in EXP_RP for 1.5 m relative humidity and 1.5 m temperature, respectively, than in CNTL but it increased 7.21% for 10 m zonal wind. These differences were only significant for RMSE and spread of 10 m zonal wind and spread of 1.5 m temperature at 95% level. This result agrees with the result in Baker et al. (2014). As a result of assessing the sensitivity of RP applied parameters in Baker et al. (2014), it has been found that parameters of boundary-layer scheme, g0, ricrit, par_mezcla, and lambda_min, typically affect the 10 m wind the most and parameter of large-scale precipitation scheme,

8 2028 S. Kim and H. M. Kim Pure Appl. Geophys. particle size distribution for rain, typically affects the 1.5 m temperature and relative humidity. In this study, it is not able to show as a significant difference as that of 10 m zonal wind in 1.5 m temperature and 1.5 m relative humidity because the RP scheme was not applied to particle size distribution for rain of large-scale precipitation scheme Effect on Precipitation Forecast Many verification scores were calculated to investigate the effect of the RP scheme on the forecast of hourly precipitation accumulation (hereafter precipitation). For the verification of precipitation forecast, observation and model forecast interpolated with 15 km resolution were used just as in the case of RMSE and spread calculation for surface variables. The 1-h accumulated precipitation and precipitation average and spread of 24-h ensemble forecast of the two systems have been drawn at all AWS locations at 00 and 12 UTC during the month (Fig. 4). The month-long patterns of precipitation occurrence of the two systems were both similar to the actual observation (compare Fig. 4b, c with a) and EXP_RP predicted approximately % more precipitation than CNTL on average (Fig. 4b), which was a significant increase at the 95% confidence level. In addition, when there was precipitation forecast, the spreads of both systems increased and the spread of EXP_RP was approximately 38.44% larger than the spread of CNTL (Fig. 4c). The ETS and FBI for each ensemble member and ensemble mean have been drawn on Fig. 5, according to the forecast lead time for each threshold. In the case of the ETS, ETS for each ensemble member and ensemble mean of EXP_RP was higher than that for CNTL for relatively heavy precipitation threshold. Moreover for heavy precipitation, the ETS of EXP_RP was higher than that of CNTL as the forecast lead time increases. In the case of the FBI, the effect of the RP scheme was observed more clearly since forecast results for ensemble members of the two systems created clusters that were notably distinguished. Ensemble members of EXP_RP tended to overestimate precipitation compared to observation for all thresholds but this aspect mitigated with relatively heavier precipitation threshold. Moreover, Figure 4 a Sum of observed hourly precipitation accumulation at all AWS stations at every 00, 12 UTC, b mean of 24-h forecast ensemble of hourly precipitation accumulation, and c spread of 24-h forecast ensemble of hourly precipitation accumulation of CNTL (black) and EXP_RP (red) during July 2013 estimation gradually became more similar to the observation as the forecast lead time increases. It was confirmed through the ETS and FBI analysis that EXP_RP overestimates precipitation compared to CNTL and has relatively more accurate forecast with heavier precipitation compared to lighter precipitation. This overestimation of EXP_RP is also reduced when the probability matched mean (PM; Ebert 2001)

9 Vol. 174, (2017) Effect of Considering Sub-Grid Scale Uncertainties on the Forecasts 2029 Figure 5 a, c, e, g ETS and b, d, f, h FBI for threshold 0.1, 0.5, 1.0, and 5.0 mm, respectively, for individual members of CNTL (gray) and EXP_RP (orange) and for ensemble means of CNTL (black) and EXP_RP (red). The scores are presented as a function of the lead time and were statistically calculated during July 2013

10 2030 S. Kim and H. M. Kim Pure Appl. Geophys. is used instead of the ensemble mean (not shown). The PM of EXP_RP shows higher ETS than that of CNTL, and the FBIs of the PM are closer to 1 compared to those of the ensemble mean for both CNTL and EXP_RP because the PM tends to correct the bias of the ensemble mean as mentioned in Chien and Jou (2004). According to BSS that assesses the accuracy of probabilistic information of ensemble system, the probabilistic information of EXP_RP was discovered to be relatively worse than that of CNTL because it showed negative values for most thresholds and forecast lead times (Fig. 6). However, as the absolute value of BSS becomes much smaller for heavy precipitation threshold, the difference in probabilistic information between the two systems was found to be very small (Fig. 6d). The fact that the probabilistic information of EXP_RP is relatively worse than that of CNTL agrees with the result confirmed through the FBI. Since the ensemble members of EXP_RP generally overestimated the actual observation and the forecast of the ensemble members clustered to show similar values, the probabilistic information of EXP_RP was shown to be relatively slightly worse than that of CNTL. The NVD for the month of July was calculated to evaluate the effect of the RP scheme on precipitation distribution (Fig. 7). The NVD showed positive values for all forecast lead times, which implies that the precipitation variances of EXP_RP are larger than those of CNTL for all forecast times. Moreover, NVD can also be interpreted with the difference in the spread amplitude of the two different systems (Peralta et al. 2012). Therefore, through the NVD results, the RP scheme can be interpreted to have effects on increasing the precipitation spread and this corresponds to the results mentioned above about the time series of precipitation spread of the two systems. As a result of examining the precipitation forecast skill using various verification scores, it has been found that EXP_RP predicts the heavy precipitations more accurately than CNTL and the precipitation forecast of EXP_RP becomes more accurate compared to CNTL as the forecast lead time increases. The reason for the better forecast of heavy precipitation in EXP_RP was using the RP scheme with an uncertainty on m_ci, a parameter that controls the ice Figure 6 The BSS during July 2013 as a function of the lead time for a 0.1 mm, b 0.5 mm, c 1.0 mm, and d 5.0 mm thresholds fall speed in large-scale precipitation scheme. The effect of ice fall speed about the structure of precipitation for UM has been confirmed in

11 Vol. 174, (2017) Effect of Considering Sub-Grid Scale Uncertainties on the Forecasts 2031 Figure 7 NVD during July 2013 as a function of lead time Bornemann (2013). It has been examined that a change in ice fall speed noticeably affects the structure of precipitation and increasing ice fall speed creates more intense precipitation and estimates more distinct shape of precipitation cells. Compared to EXP_RP, the experiment using the RP scheme without an uncertainty on m_ci shows less forecast skill in terms of ETS and BSS for relatively heavy precipitation thresholds (not shown), which confirms the effect of considering the uncertainty in ice fall speed in large-scale precipitation scheme for the heavy precipitation forecast Case Study Verifications of 24-h precipitation forecast of the two systems, CNTL and EXP_RP, were carried out on two rainfall cases. The selected two rain fall cases were the two most heavily rainfall cases in the month of July, 2013 of the Korean Peninsula (noticed in Fig. 4a) Case on 14 July 2013 The first case was a case of heavy rainfall that occurred in the midlands of the Korean Peninsula at 00 UTC 14 July 2013 with peak value of 81.5 mm for hourly precipitation accumulation (Fig. 8a). Figure 8b d shows the probabilities of precipitation forecast for the two systems with regard to 0.5 and 5.0 mm thresholds. The forecast area of EXP_RP was wider than that of CNTL not only for 0.5 mm but also for 5.0 mm precipitation thresholds. Moreover, the region of heavy precipitation has been predicted similar to the observation, centered in the mid region of the Korean Peninsula. For quantitative evaluation on the forecasts of the two systems, the ETS and BSS were calculated according to the thresholds (Fig. 9). The ETS for ensemble mean of EXP_RP is lower than that of ensemble mean of CNTL for thresholds below 2.0 mm but it is shown higher for thresholds above 5.0 mm (Fig. 9a). With higher thresholds, not only the ETS of the ensemble mean but that of each ensemble members was found out to be also higher for the ETS of EXP_RP than that of CNTL. In the same way, the BSS showed better probabilistic information for EXP_RP with higher thresholds (Fig. 9b), which corresponds with the month-long statistical verification mentioned above Case on 22 July 2013 The second case was a case of precipitation that occurred in the midlands of the Korean Peninsula at 00 UTC 22 July 2013 with peak value of mm for hourly precipitation accumulation (Fig. 10a). Figure 10b d shows the probabilities of precipitation forecast for the two systems with regard to 0.5 and 5.0 mm thresholds. In this case, CNTL was unable to estimate most of the precipitation in the midlands of the Korean Peninsula where the actual precipitation took place (Fig. 10b, d). Whereas, in EXP_RP, it predicted more precipitation amount than 0.5 and 5.0 mm near the actual areas of precipitation and this difference was confirmed through the ETS and BSS as well. In Fig. 11a, the ETS of ensemble mean and ensemble members for EXP_RP is higher than that of ensemble mean and ensemble members for CNTL for most of the precipitation thresholds. This aspect was shown through the BSS results as well and it was confirmed that EXP_RP has better probabilistic information for all thresholds (Fig. 11b). 5. Summary and Discussion This study analyzed the effect of the RP scheme on ensemble probability forecast in highresolution limited area EPS of the KMA. The effect was able to be confirmed by constructing a system with the RP scheme and one without, then verifying forecasts of the two systems. For general verification

12 2032 S. Kim and H. M. Kim Pure Appl. Geophys.

13 Vol. 174, (2017) Effect of Considering Sub-Grid Scale Uncertainties on the Forecasts 2033 b Figure 8 a Hourly precipitation accumulation of AWS at 00 UTC 14 July 2013; probabilities (%) of hourly precipitation accumulation exceeding 0.5 mm of b CNTL and c EXP_RP for 24-h lead time; Probabilities (%) of hourly precipitation accumulation exceeding 5.0 mm of d CNTL and e EXP_RP for 24-h lead time. The forecast started at 00 UTC 13 July 2013 about the month-long experimental period of July, 2013, the RMSE and spread were calculated for 1.5 m temperature, 1.5 m relative humidity, 10 m zonal wind, and precipitation forecast. Throughout the experimental period, the 24-h forecast RMSE of EXP_RP decreased for 10 m zonal wind and the spread of EXP_RP increased compared to that of CNTL, which revealed that the prediction for 10 m zonal wind improved using RP scheme. In contrast, in the case of 1.5 m relative humidity and 1.5 m temperature, the RMSE of EXP_RP increased and decreased, respectively, in average compared to that of CNTL. The spread of EXP_RP both decreased in average for both the variables. This corresponds to the result confirmed in Baker et al. (2014). Although parameters of boundary-layer scheme with the RP application have positive influence on the wind, there were found not to be a positive influences on the relative humidity and temperature which are more affected by large-scale precipitation scheme than boundary-layer scheme, especially by particle size distribution for rain without the RP application. Therefore, in order to increase the predictability for temperature and relative humidity, it seems that the RP scheme needs to be applied to more diverse parameters of large-scale precipitation scheme. The uncertainties associated with the particle size distribution for rain over the Korean Peninsula have not been fully revealed yet. Therefore, the parameters of large-scale precipitation scheme that are appropriate for the precipitation phenomena over the Korean Peninsula are beyond the scope of this study and would be studied for future work. To examine the effect of considering model error on precipitation forecast in further detail, the precipitation forecasts of the two systems were statistically assessed in various perspectives using ETS, FBI, BSS, and NVD. The results of ETS and FBI revealed that with relatively more intense precipitation and longer the forecast lead time, the precipitation forecast of EXP_RP performed better than that of CNTL. Especially through the FBI, it has been found that EXP_RP tends to overestimate the precipitation compared to observation. Due to this tendency, the BSS had showed the probabilistic information of EXP_RP to be slightly worse than that of CNTL but this was mitigated in the condition of heavy precipitation. The NVD was greater than 0 for all forecast lead times which represents the precipitation variance and precipitation spread of EXP_RP to be larger than those of CNTL for all forecast lead times. The characteristics of the RP scheme found through verification scores were confirmed through application to the two rainfall cases. The chosen Figure 9 a ETS as a function of thresholds for individual members of CNTL (gray) and EXP_RP (orange) and for ensemble means of CNTL (black) and EXP_RP (red); and b BSS as a function of thresholds at 00 UTC 14 July 2013

14 2034 S. Kim and H. M. Kim Pure Appl. Geophys. Figure 10 a Hourly precipitation accumulation of AWS at 00 UTC 22 July 2013; probabilities (%) of hourly precipitation accumulation exceeding 0.5 mm of b CNTL and c EXP_RP for 24-h lead time; Probabilities (%) of hourly precipitation accumulation exceeding 5.0 mm of d CNTL and e EXP_RP for 24-h lead time. The forecast started at 00 UTC 21 July 2013

15 Vol. 174, (2017) Effect of Considering Sub-Grid Scale Uncertainties on the Forecasts 2035 Figure 11 a ETS as a function of thresholds for individual members of CNTL (gray) and EXP_RP (orange) and for ensemble means of CNTL (black) and EXP_RP (red); and b BSS as a function of thresholds at 00 UTC 22 July 2013 rainfall cases were at 00 UTC 14 and 00 UTC 22 July of 2013 and in both the cases, heavy rainfall occurred on the midlands of the Korean Peninsula. For both cases, EXP_RP estimates heavy precipitation more accurately than CNTL in the regions where precipitation occurred. Especially in the second case, CNTL not only could not estimate the precipitation region for 0.5 mm precipitation but also could not estimate most of the region for precipitation greater than 5.0 mm as well. This was identified in the ETS and BSS results for the cases as well. In the second case, the ETS of EXP_RP was higher than that of CNTL not only for heavy precipitation but for weak precipitation as well. Moreover, the probabilistic information was found to be better. In this way, the effect of the RP scheme on precipitation forecast was able to be confirmed not only through the statistics for the month of July, 2013, but through the case applications. The RP scheme was found to cause significant differences with spread of 1.5 m temperature and RMSE and spread of 10 m zonal wind. In addition, the RP scheme has a tendency of overestimating the precipitation compared to when the RP scheme was not used. However, these tendencies were mitigated in the cases of heavy precipitation and, therefore in average, gained better forecast results for heavy precipitation. This was predicted due to consideration of uncertainty for parameter that controls the ice fall speed in large-scale precipitation scheme using the RP scheme. In the research by Bornemann (2013) about the effect of ice fall speed on the structure of precipitation in UM, the increase of ice fall speed was found to be related to heavy precipitation. Therefore, improvement in prediction for heavy precipitation that affects the Korean Peninsula can be anticipated through consideration of model errors using the RP scheme. Further investigations that apply model error on more diverse and segmented physical parameters than those used in this research are in plan to be conducted in order to improve the prediction for a wider range of precipitation intensity for diverse cases. Acknowledgements The authors appreciate two reviewers for their valuable comments. This study was supported by the Korea Meteorological Administration Research and Development Program under Grant KMIPA The authors appreciate the Numerical Weather Prediction Division of the Korea Meteorological Administration for providing computer facility support and resources for this study. REFERENCES Baker, L. H., Rudd, A. C., Migliorini, S., & Bannister, R. N. (2014). Representation of model error in a convective-scale ensemble prediction system. Nonlinear Processes in Geophysics, 21, doi: /npg Barker, T. W. (1991). The relationship between spread and forecast error in extended-range forecasts. Journal of Climate, 4, doi: / (1991)004\0733:trbsaf[2. 0.CO;2.

16 2036 S. Kim and H. M. Kim Pure Appl. Geophys. Bornemann, J., Field, P., Furtado, K., & Weeks, M. (2013). The effect of ice fall speed in the structure of surface precipitation. 10th International SRNWP-Workshop on Non-Hydrostatic Modelling, Offenbach, Germany /34Bornemann_IceFallSpeed_final.pdf. Bouallègue, Z. B., Theis, S. E., & Gebhardt, C. (2013). Enhancing COSMO-DE ensemble forecasts by inexpensive techniques. Meteorologische Zeitschrift, 22, doi: / / 2013/0374. Bouttier, F., Vié, B., Nuissier, O., & Raynaud, L. (2012). Impact of stochastic physics in a convection-permitting ensemble. Monthly Weather Review, 140, doi: /mwr-d Bowler, N. E., Arribas, A., Mylne, K. R., Robertson, K. B., & Beare, S. E. (2008). The MOGREPS short-range ensemble prediction system. Quarterly Journal of the Royal Meteorological Society, 134, doi: /qj.234. Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78, 1 3. doi: / (1950)078\0001:VOFEIT[2.0.CO;2. Caron, J.-F. (2013). Mismatching perturbations at the lateral boundaries in limited-area ensemble forecasting: A case study. Monthly Weather Review, 141, doi: /mwr-d Chien, F.-C., & Jou, B. J.-D. (2004). MM5 ensemble mean precipitation forecasts in the Taiwan area for three early summer convective (Mei-Yu) seasons. Weather and Forecasting, 19, doi: / (2004)019\0735:mempfi[2.0. co;2. Davies, T. (2014). Lateral boundary conditions for limited area models. Quarterly Journal of the Royal Meteorological Society, 140, doi: /qj Davies, T., Cullen, M. J. P., Malcolm, A. J., Mawson, M. H., Staniforth, A., White, A. A., et al. (2005). A new dynamical core for the Met Office s global and regional modelling of the atmosphere. Quarterly Journal of the Royal Meteorological Society, 131, doi: /qj Ebert, E. E. (2001). Ability of a poor man s ensemble to predict the probability and distribution of precipitation. Monthly Weather Review, 129, doi: / (2001)129\2461: AOAPMS[2.0.CO;2. Ebert, E. E. (2008). Fuzzy verification of high-resolution gridded forecasts: a review and proposed framework. Meteorological Applications, 15, doi: /met.25. Fritsch, J. M., & Carbone, R. E. (2004). Improving quantitative precipitation forecasts in the warm season: A USWRP research and development strategy. Bulletin of the American Meteorological Society, 85, doi: /bams Gebhardt, C., Theis, S. E., Paulat, M., & Ben Bouallègue, Z. (2011). Uncertainties in COSMO-DE precipitation forecasts introduced by model perturbations and variation of lateral boundaries. Atmospheric Research, 100, doi: /j. atmosres Gregory, D., & Rowntree, P. R. (1990). A mass flux convection scheme with representation of cloud ensemble characteristics and stability-dependent closure. Monthly Weather Review, 118, doi: / (1990)118\1483:amfcsw[2. 0.CO;2. Hou, D., Kalnay, E., & Droegemeier, K. K. (2001). Objective verification of the SAMEX 98 ensemble forecasts. Monthly Weather Review, 129, doi: / (2001)129\0073:ovotse[2.0.co;2. Jones, M. S., Colle, B. A., & Tongue, J. S. (2007). Evaluation of a mesoscale short-range ensemble forecast system over the northeast United States. Weather and Forecasting, 22, doi: /waf Kain, J. S., et al. (2008). Some practical considerations regarding horizontal resolution in the first generation of operational convection-allowing NWP. Weather and Forecasting, 23, doi: /waf Kay, J. K., & Kim, H. M. (2014). Characteristics of initial perturbations in the ensemble prediction system of the Korea meteorological administration. Weather and Forecasting, 29, doi: /waf-d Kay, J., Kim, H. M., Park, Y.-Y., & Son, J. (2013). Effect of doubling the ensemble size on the performance of ensemble prediction in the warm season using MOGREPS implemented at the KMA. Advances in Atmospheric Sciences, 30, doi: /s y. Kim, H. M., et al. (2014). Improvement of operating technique for numerical weather prediction model III. KMA Tech. Note, p Kim, S., Kim, H. M., Kay, J. K., & Lee, S. W. (2015). Development and Evaluation of the High Resolution Limited Area Ensemble Prediction System in the Korea Meteorological Administration. Atmosphere, 25, Kühnlein, C., Keil, C., Craig, G. C., & Gebhardt, C. (2014). The impact of downscaled initial condition perturbations on convective-scale ensemble forecasts of precipitation. Quarterly Journal of the Royal Meteorological Society, 140, doi: /qj Lock, A. P., Brown, A. R., Bush, M. R., Martin, G. M., & Smith, R. N. B. (2000). A new boundary layer mixing scheme. Part I: Scheme description and single-column model tests. Monthly Weather Review, 128, doi: / (2000)128\3187:anblms[2.0.co;2. Marsigli, C., Boccanera, F., Montani, A., & Paccagnella, T. (2005). The COSMO-LEPS mesoscale ensemble system: validation of the methodology and verification. Nonlinear Processes in Geophysics, 12, Montani, A., Cesari, D., Marsigli, C., & Paccagnella, T. (2011). Seven years of activity in the field of mesoscale ensemble forecasting by the COSMO-LEPS system: main achievements and open challenges. Tellus A, 63, doi: /j x. Palmer, T. N., Barkmeijer, J., Buizza, R., & Petroliagis, T. (1997). The ECMWF ensemble prediction system. Meteorological Applications, 4, doi: /s Peralta, C., Ben Bouallègue, Z., Theis, S. E., Gebhardt, C., & Buchhold, M. (2012). Accounting for initial condition uncertainties in COSMO-DE-EPS. Journal Geophysical Research, 117, D doi: /2011jd Saito, K., Seko, H., Kunii, M., & Miyoshi T. (2012). Effect of lateral boundary perturbations on the breeding method and the local ensemble transform Kalman filter for mesoscale ensemble prediction. Tellus A, 64. doi: /tellusa.v64i Schaefer, J. T. (1990). The critical success index as an indicator of warning skill. Weather and Forecasting, 5, doi: / (1990)005\0570:TCSIAA[2.0.CO;2. Schwartz, C. S., et al. (2009). Next-day convection-allowing WRF model guidance: A second look at 2-km versus 4-km grid

17 Vol. 174, (2017) Effect of Considering Sub-Grid Scale Uncertainties on the Forecasts 2037 spacing. Monthly Weather Review, 137, doi: / 2009MWR Shepard, D. (1968). A two-dimensional interpolation function for irregularly-spaced data. Proc. 23rd ACM National Conference, Association for Computing Machinery, Toth, Z., & Kalnay, E. (1997). Ensemble forecasting at NCEP and the breeding method. Monthly Weather Review, 125, doi: / (1997)125\3297:efanat[2.0.co;2. Vié, B., Nuissier, O., & Ducrocq, V. (2011). Cloud-resolving ensemble simulations of Mediterranean heavy precipitating events: uncertainty on initial conditions and lateral boundary conditions. Monthly Weather Review, 139, doi: /2010mwr Webster, S., Brown, A. R., Cameron, D. R., & Jones, C. P. (2003). Improvements to the representation of orography in the Met Office Unified Model. Quarterly Journal of the Royal Meteorological Society, 129, doi: /qj Weisman, M. L., Davis, C., Wang, W., Manning, K. W., & Klemp, J. B. (2008). Experiences with 0 36-h explicit convective forecasts with the WRF-ARW model. Weather and Forecasting, 23, doi: /2007waf Wilks, D. S. (2006). Statistical methods in the atmospheric sciences (2nd ed.). London: Academic Press. Wilson, D. R., & Ballard, S. P. (1999). A microphysically based precipitation scheme for the UK Meteorological Office Unified Model. Quarterly Journal of the Royal Meteorological Society, 125, doi: /qj (Received August 17, 2016, revised February 24, 2017, accepted March 2, 2017, Published online March 10, 2017)