Meteorol. Appl. 9, 21 31 (2002) Statistical interpretation of NWP products in India Parvinder Maini, Ashok Kumar, S V Singh and L S Rathore, National Center for Medium Range Weather Forecasting, Department of Science and Technology, Mausam Bhavan Complex, Lodi Road, New Delhi 110 003, India Although numerical weather prediction (NWP) models provide an objective forecast, poor representation of local topography and other features in these models, necessitates statistical interpretation (SI) of NWP products in terms of local weather. The Perfect Prognostic Method (PPM) is one of the techniques for accomplishing this. At the National Center for Medium Range Weather Forecasting, PPM models for precipitation (quantitative, probability, yes/no) and maximum/minimum temperatures are developed for monsoon season by using analyses from the European Centre for Medium-Range Weather Forecasts. The SI forecast is then obtained by using these PPM models and output from the operational NWP model at the Center. Direct model output (DMO) obtained from the NWP model and the SI forecast are verified against the actual observations. The present study shows the verification scores obtained during the 1997 monsoon season for 10 locations in India. The results show that the SI forecast has good skill and is an improvement over DMO. 1. Introduction An objective forecast is one which produces only one forecast from a specific set of data. Objective forecasts fall into two categories. One is numerical weather prediction (NWP) while the other consists of statisticaldynamical methods. An objective forecast can also be obtained through Statistical Interpretation (SI) of NWP model output. Weather variables (e.g. temperature, rainfall, wind, cloud cover) when predicted from numerical models have low accuracy. This is because they are dependent on local topography and environmental conditions. In NWP models it is difficult to account for these features at each point considered by the model. The coarser resolution of a general circulation model (GCM) leads to further loss of accuracy. A statistical technique that develops a concurrent relationship between the upper-air circulation and the surface weather parameters can take the local conditions into account. Hence, in order to get a near-accurate local forecast, statistical interpretation of NWP products is essential (Kumar & Maini, 1996). At the National Center for Medium Range Weather Forecasting (NCMRWF) the location-specific medium range weather forecasts are being developed to provide weather-based advice for the farming community. Since 1991 NCMRWF has been engaged in preparing three-day location-specific forecasts. These forecasts are used by a team of agricultural specialists at the Agromet Field Units (AMFU) to provide an agro-advisory service for the farming community under their jurisdiction. Forecasts from the Direct Model Output (DMO) and SI models, graphical output and conventional synoptic techniques are used for the preparation of the forecasts issued to the AMFUs (Kumar et al., 2000). Section 2 of the paper discusses the method for obtaining the location-specific forecast directly from the operational NWP model. In section 3 the different techniques for statistical interpretation of NWP output are described and section 4 describes in detail the SI forecast obtained at NCMRWF based on the Perfect Prognostic Method (PPM). This is followed by the description of the procedure for producing bias-free SI and DMO forecasts in section 5. Finally in section 6 the results of the verification of SI and DMO against the observations are presented and important conclusions are drawn. 2. The DMO forecast The NWP model that has been operational at NCMRWF since 1994 is the T-80 model, which has 18 layers in sigma coordinates. It has a spectral horizontal representation of triangular truncation at wave number 80 which transfers to a resolution of about 150 km 150 km. The model is run everyday based on 0000 UTC initial conditions to give a forecast for the subsequent five days. The forecast is obtained at each timestep of 15 minutes for different surface and upperair weather parameters for the 256 128 Gaussian grid points of the T-80 model. From the Gaussian grid, an Indian window of size 25 25 grid points is extracted 21
Parvinder Maini, Ashok Kumar, S V Singh and L S Rathore covering an area from 4.90 o to 38.52 o N and 66.09 o to 99.84 o E. The model output is hence obtained at the 625 grid points for the following six surface weather elements: surface pressure (hpa) rainfall rate (mm s 1 ) zonal and meridional wind component at 10 feet or 3.048 m (m s 1 ). temperature at 4.5 feet or 1.3716 m ( o C) specific humidity (gm/gm) To get the forecast at a specific location from the DMO, the interpolated value of the forecast from the four grid points surrounding the location is considered. If a location is very near to a grid point, then the forecast at that grid-point can be taken as the forecast for the location. In order to decide which of the two forecasts should be given more weight, the distance of the location from the four grid points is calculated. If the distance of the location from the nearest grid point is less than one-fourth the diagonal distance between any two grid points, then the forecast at the nearest grid point is used. Otherwise the interpolated value from the four surrounding grid points is taken as the forecast for the location. The DMO forecast for 24, 48, 72 and 96 hours are obtained for 10 locations (Figure 1) in monsoon 1997 for maximum temperature, minimum temperature and rainfall. 3. Statistical interpretation of NWP products Basically, statistical interpretation of NWP products can be carried out using two methods. The first method is the Perfect Prognostic Method (PPM) (Klein et al., 1959); the second is Model Output Statistics (MOS) (Glahn & Lowry, 1972). The MOS and PPM techniques have considerably different attributes (Carter et al., 1989), with the source of the dataset used in development distinguishing the two formulations. The MOS approach uses NWP forecasts for both the development and operational application of the model. It requires an archive of dynamical model forecasts (two or three years) for development of equations. A statistical technique, usually multiple linear regression, is then used to determine relationships between the observed weather elements (predictands) and the NWP output variables (predictors). To get an operational forecast, MOS equations are applied to the same dynamical model that provided the developmental sample. The MOS equations partially account for some of the bias and systematic errors of the numerical model from which the equations are derived. Since these systematic errors tend to vary with period into the forecast, a separate equation is developed for each lead-time. If the model undergoes a major change the MOS relations will have to be developed again, and in order to get a stable relation, a sufficient sample of model output will be required for the redevelopment of the MOS equations. This is a drawback because when the model is changed, it is likely that a sufficiently long developmental sample of predictors from the improved NWP model may not be readily available. The MOS type relationships weaken the sharpness of the forecasts with increasing lead time. In the PPM approach, multiple linear regression equations are derived that relate observed surface weather elements (predictands) to concurrent surface and upper-air fields (predictors). In applying the PPM equations for a specific forecast time, model output for that time is substituted for the developmental observations (predictors). A major disadvantage of this approach is that it does not account for any systematic bias and inaccuracies of the model while a major advantage of it is that stable forecasting relations can be derived for individual locations and seasons from a long-period record. It can be applied even if the numerical model undergoes some change (i.e. same relation will still hold good). Moreover, these changes will usually improve the model forecast which in turn will improve the PPM forecast. 4. SI forecast 4.1. Developmental data for PPM models Owing to the limited amount of developmental data from the T-80 model, neither PPM nor MOS equations could be developed using these data. The data from the European Centre for Medium Range Weather Forecasts (ECMWF) were readily available. Therefore, the analysed fields from the ECMWF/TOGA basic level III data sets, which are part of the ECMWF/ WCRP Level III-A Global Atmospheric Data Archive analysis, were used to develop PPM based SI models at NCMRWF. Figure 1. Map of India showing the stations used in this study. 22 As shown by Carter (1986), the development of the model requires data for at least three seasons (of the
same kind) of six months duration. PPM models are developed for rainfall and temperature (maximum/ minimum) for the monsoon season by using six years (1985 1990) of TOGA analyses (2.5 o 2.5 o ) from ECMWF as the predictors and the actual observed values of rainfall/temperature as the predictands. The period of the monsoon season is taken to be June August for the north-west Indian stations and for the rest of the country it is taken to be June September. As the SI forecast is one of the important guiding tools used for preparation of the final forecast given to AMFUs, the development of SI models is carried out for stations for which past observed data (1985 1990) of the predictands are readily available. Keeping in mind the importance of rainfall and daily temperature to a farmer, the predictands chosen initially for the development of the model are quantitative precipitation (QP), probability of precipitation (PoP) and maximum/minimum temperatures. Rainfall is highly variable in the tropics especially in the monsoon season and has a skewed distribution. The distribution is made more normal by taking the cube root of the data values. Hence the model for rainfall is developed by taking the cube root of QP. The observed value of PoP is obtained by taking it as 1 if measurable precipitation is observed and 0 otherwise (Kumar et al.,1999). The threshold value for measurable precipitation is taken as 0.1 mm. After a detailed study, 47 meteorological parameters at 1000, 850, 700 and 500 hpa level are chosen as the possible set of predictors (Table 1). This set includes a few basic variables (e.g. geopotential height, temperatures, winds, vertical velocity) and a few other derived variables (e.g. mean sea level pressure, divergence, vorticity, 1000 500 mb precipitable water, saturation Table 1. Meteorological parameters chosen as predictors. Parameter Level (hpa) Relative humidity 1000, 850, 700, 500 Temperature 1000, 850, 700, 500 Zonal wind component 1000, 850, 700, 500 Meridional wind component 1000, 850, 700, 500 Vertical velocity 1000, 850, 700, 500 Geopotential 1000, 850, 700, 500 Saturation deficit 1000 500 Precipitable water 1000 500 Mean sea level pressure Temperature gradient 850 700, 700 500 Advection of temperature gradient 850 700, 700 500 Advection of temperature 1000, 850, 700, 500 Vorticity 1000, 850, 700, 500 Advection of vorticity 1000, 850, 700, 500 Thickness 850 500 Horizontal water vapour flux div. 1000 500 Mean relative humidity 1000 500 Statistical interpretation of NWP products in India deficit, thickness and the rate of change of moist static energy). Depending upon the reporting time of occurrence of rainfall and temperature values, three reference times, namely 0000 UTC and 1200 UTC of the previous day and 0000 UTC of the same day, are considered. Thus, the reference time at which the values of the predictor are to be considered (Figure 2) for developing the model equations is chosen as follows: 0000 UTC of the same day for minimum temperature 1200 UTC of the previous day for maximum temperature 0000 UTC of the same day, 1200 UTC of the previous day and average of the two, for 24-hour accumulated rainfall (QP) and PoP. The day refers to the calendar date on which the 24-hour rainfall or maximum/minimum temperature is reported. This implies that for the development of temperature models, there are 47 potential predictors and the corresponding figure for rainfall is 141 (47 3) predictors. 4.2. The methodology The value of a predictor at a station is best represented by its value at the nearest grid and the surrounding grids. Canonical correlation (Rousseau, 1982) is used to find the value of any particular predictor, representative of a station, by considering its value at nine grid points surrounding the station of interest (Woodcock, 1984). The first canonical variate is the best linear combination of the values of a predictor at the nine grid points. It also has maximum correlation with the predictand and is taken as the value of a particular predictor at the station. The canonical variates are obtained for each of the predictors to provide a new set of potential predictors. As the data set contains only one predictand and several predictors, the canonical correlation in this case reduces to multiple linear regression. The new set of potential predictors is subjected to a step-wise selection procedure, and equations with only those predictors that explain most of the variance are selected. In order to avoid over-specification of the pre- Rate of change of moist static energy 1000 500 Figure 2. Reference time for rainfall and maximum/minimum temperatures. 23
Parvinder Maini, Ashok Kumar, S V Singh and L S Rathore dictand a procedure is necessary for selecting the most suitable number of predictors. The approach adopted here is that used by Woodcock (1984) in which the selection procedure is terminated when the addition of a further variable to the prediction equation contributes less than a critical value to the percentage of variance explained by the variables already selected. In order to have a significant percentage of variance explained by the predictors selected, this value is taken as: 1.0% for maximum and minimum temperatures and PoP 0.5% for QP. 4.2. Model equations The selected predictors are then used for developing a PPM model equation. The predictors frequently selected for QP, PoP and maximum/minimum temperatures are given in Table 2. A simple linear regression equation is obtained relating the predictand and the set of selected predictors: where X i are the predictors taken from the analysis fields obtained from the ECMWF model and Y is the observed value of the predictand (rainfall/temperature). The multiple regression coefficients, a i, thus obtained and the T-80 model outputs are then used to obtain a statistical forecast. 4.3. SI forecast Y = a0 + a1 X1 + a2 X2 +... + a n X SI forecast (Y) for 24, 48, 72 and 96 hours is obtained for maximum temperature, minimum temperature, QP and PoP by using the a i obtained above and by substituting the X i in equation (1) with the T-80 model forecast for that particular hour. Thus: Y = a + a i where n is the number of selected predictors. Similarly Y 48, Y 72 and Y 96 are obtained. During the monsoon n i= 1 24 0 X i 24 n (1) (2) season of 1997, the SI forecast is obtained for 10 stations by using the daily output from T-80 model and the PPM equations developed for the monsoon season. 5. Bias-free forecasts The forecasts generated on any independent sample are known to have some bias. Owing to this, the predicted value of Y obtained from equation (2) may be either over- or under-predicted and it is advisable to remove this bias. 5.1. Rainfall forecast The bias from the SI forecast obtained in 1997 monsoon season is removed by using the observations and SI forecast of the predictand during two previous seasons viz., 1995 and 1996 monsoon seasons. A linear regression is carried out with the observed weather as the dependent variable, and the forecast weather as the independent variable. Using the coefficients (a I ) thus obtained, an equation of the form: Y = a + a Y new 0 1 is used for producing a bias-free SI forecast for QP during the current season, where Y new is the bias-free forecast and Y old is the original forecast (Glahn et al.,1991). Before fitting the regression line, the optimal threshold value for QP is obtained by maximizing the skill scores. Optimal threshold value implies that if the rainfall amount is less than the threshold then QP is taken as zero, otherwise it is taken as the forecast value. Similarly for PoP a constant factor is added to the forecast probability by maximizing the skill. Table 3 gives the correction factors obtained for QP and PoP in the 1997 monsoon season. Rainfall is obtained as a hybrid of both QP and PoP (Tapp et al., 1986). More weight is given to PoP forecasts as they show superiority in differentiating between a rainy and a non-rainy day (Kumar et al., 1999). If PoP < 0.5 and QP = m then give rain = 0.0 mm If PoP 0.5 and QP = m then give rain = m mm If PoP 0.5 and QP = 0.0 then give rain = 0.1 mm old (3) Table 2. Predictors frequently selected for different parameters. Predictand Number of predictors Predictors Maximum temperature 2 3 1000 500 hpa saturation deficit, 850 hpa temperature. Minimum temperature 3 7 850 hpa temperature, 500 hpa temperature, 850 500 hpa thickness. Probability of Precipitation (PoP) 5 8 Mean relative humidity, 850 hpa meridional wind. Quantitative Precipitation (QF) 6 12 Mean relative humidity, 850 hpa meridional wind, 850 hpa vorticity. 24
Statistical interpretation of NWP products in India where m is the forecast value of rainfall from QP equation and 0.1 mm is the minimum possible rainfall that can be measured. The DMO forecasts obtained in section 2 are also biased. The bias in the rainfall forecast is removed in a similar to that in the SI forecast. The correction factors for QP obtained in the case of DMO forecast are given in Table 4. Tables 3 and 4 also include the observed daily mean rainfall for each of the stations considered. It is seen that the magnitude of the correction factors applied to the DMO forecast is much higher than those applied to the SI forecast in each of the stations. This shows that the DMO forecast is much more biased than the SI forecast. 5.2. Temperature forecast For the temperature forecast a different approach (Glahn et al.,1991) is adopted. A measure of bias is the mean error (ME) and is found by using the observed and forecast (SI/DMO) values of the 1995 and 1996 monsoon seasons. It is given as: ME = f x where f is the average of the forecast (SI/DMO) values and x is the average of the observations. To obtain a bias-free SI and DMO forecast, the value of ME obtained in each case is added to the corresponding SI/DMO forecast obtained in the 1997 monsoon season. This method is followed for both maximum and minimum temperatures. Thus a bias-free SI and DMO forecast is obtained for rainfall and temperatures in 1997. They are each verified against the observations for 1997. 6. Verification of forecasts and conclusion The bias-free SI and DMO forecasts obtained in the 1997 monsoon season at 10 different locations in India are each verified against the observations of the same season. Different measures of accuracy and skill scores (Wilks, 1995) are used for the assessment of the forecast, the details about how to calculate these measures for rainfall are given in the Appendix. For the rainfall verification, Ratio Score (RS), Probability of Detection (POD), False Alarm Rate (FAR), and Hanssen and Kuipers Score (HKS) are calculated; for temperature verification, Correlation Coefficient and Root Mean Square Error (RMSE) are used. The 24-, 48-, 72- and 96-hour forecasts obtained by the DMO and SI methods are verified against the actual (i.e. the observed data of the 1997 monsoon) and are compared. Results of various methods of assessing Table 3. Correction factors to be added to quantitative precipitation and probability of precipitation obtained from SI forecast. Station Observed daily Threshold for QP (mm) (Constant factor for PoP (mm)) mean rainfall mm Akola 4.00 2.00 2.00 2.00 2.00 0.05 0.05 0.00 0.00 Anand 5.90 0.70 0.70 0.70 0.70 0.10 0.10 0.00 0.00 Delhi 6.70 0.15 0.20 0.20 0.20 0.20 0.20 0.15 0.15 Hisar 4.30 1.80 1.50 1.50 1.50 0.10 0.10 0.10 0.15 Jabalpur 9.00 1.00 1.00 1.20 1.20 0.10 0.00 0.00 0.00 Junagadh 6.30 0.50 0.50 0.50 0.50 0.00 0.00 0.00 0.00 Rahuri 3.70 0.15 0.30 0.50 0.60 0.20 0.20 0.20 0.20 Raipur 8.30 0.50 0.60 0.50 0.60 0.05 0.00 0.05 0.10 Ranchi 12.60 5.00 4.00 2.50 2.00 0.20 0.25 0.25 0.25 Udaipur 4.60 1.00 1.00 1.20 1.20 0.05 0.00 0.00 0.00 Table 4. Threshold value to be added to the DMO quantitative precipitation forecast. Station Observed daily Threshold for QP (mm) mean rainfall (mm) 24 h 48 h 72 h 96 h Anand 5.9 4.0 3.5 3.0 2.5 Delhi 6.7 0.2 0.4 0.2 0.3 Hisar 4.3 0.5 0.5 0.5 0.0 Rahuri 3.7 2.0 2.0 1.0 2.0 Udaipur 4.6 3.5 3.0 2.5 1.5 25
Parvinder Maini, Ashok Kumar, S V Singh and L S Rathore Table 5. Verification of rainfall forecast during the 1997 monsoon as given by persistence, SI and DMO forecasts using (a) Ratio Score as a measure of accuracy and (b) Hanssen and Kuipers Score (HKS) as a measure of skill. (a) Ratio Score (RS) Station Akola 79.9 63.0 61.0 58.7 55.4 54.4 55.4 52.2 54.4 Anand 67.7 79.4 71.7 71.7 59.8 65.2 64.3 65.2 57.6 Delhi 58.1 64.1 62.0 60.9 58.7 56.5 53.2 54.3 61.9 Hisar 70.9 70.7 63.0 60.9 65.2 47.8 47.8 55.4 50.0 Jabalpur 77.4 75.0 71.7 68.5 67.4 70.7 62.0 70.0 67.4 Junagadh 65.9 70.5 67.2 63.1 59.8 60.7 61.5 60.6 59.0 Rahuri 67.5 59.8 57.4 53.3 53.3 39.3 41.8 37.7 39.3 Raipur 65.9 68.0 68.0 63.1 60.7 68.9 63.9 63.9 62.3 Ranchi 62.3 78.7 69.7 69.7 68.0 73.0 69.8 68.9 65.6 Udaipur 71.0 78.3 81.5 71.7 73.9 73.9 77.2 70.7 56.5 (b) Hanssen and Kuipers Score (HKS) Station Persistence SI Forecast DMO Forecast Akola 0.37 0.28 0.30 0.25 0.20 0.22 0.23 0.17 0.22 Anand 0.35 0.57 0.44 0.43 0.20 0.30 0.27 0.32 0.18 Delhi 0.13 0.30 0.26 0.21 0.19 0.21 0.12 0.18 0.27 Hisar 0.31 0.39 0.24 0.19 0.23 0.13 0.09 0.15 0.10 Jabalpur 0.54 0.45 0.40 0.34 0.32 0.35 0.18 0.36 0.32 Junagadh 0.31 0.39 0.32 0.26 0.20 0.18 0.21 0.20 0.17 Rahuri 0.20 0.08 0.19 0.17 0.21 0.07 0.10 0.05 0.05 Raipur 0.26 0.39 0.26 0.21 0.17 0.38 0.19 0.16 0.13 Ranchi 0.22 0.54 0.36 0.35 0.29 0.39 0.29 0.29 0.23 Udaipur 0.33 0.59 0.55 0.39 0.40 0.42 0.41 0.32 0.07 accuracy and skill for the rainfall forecasts are given in Table 5. It is observed from Table 5(a) that the RS obtained from the SI models is higher than those obtained from DMO for most of the stations and is equal for one or two stations. This indicates that the capability of SI models to predict correctly the occurrence of rainfall is better than the DMO. The values of POD and FAR are also obtained for both the SI and DMO forecast. The values of POD for both types of forecast are found to be in the range of 0.62 to 0.88 and the values of FAR are found to vary between 0.18 and 0.50. It is observed that for most of the stations, the value of both POD and FAR obtained from DMO is higher than SI for all the lead times. Here the higher value of POD for DMO can be attributed to over-forecasting. This fact is further strengthened by the higher values of FAR for DMO. This shows that the proportion of forecasts that fail to materialize is much higher in DMO than in the case of SI. Table 5(b) gives the skill of rainfall obtained both from the SI and DMO. The HKS obtained by the SI models for all lead times is higher than the DMO for most of the stations. This indicates that the skill of SI is better than DMO. The measures of accuracy and skill of rainfall forecasts obtained with SI are better than the 26 persistence forecast for most of the stations, although certain stations show a decrease in skill with increasing lead-time. Figures 3 and 4 give the RS and the HKS for rainfall versus lead time for four stations (Anand, Rahuri, Raipur and Udaipur) during the monsoon of 1997. It is clearly seen that the RS and HKS of SI forecasts are higher than those of DMO forecasts for all the lead times. It is also seen that HKS decreases with increasing lead time.
Statistical interpretation of NWP products in India Figure 3. Hanssen and Kuipers Score (HKS) for rainfall as a function of lead time during the 1997 monsoon (fs: SI forecast; fd: DMO forecast). Figure 4. Ratio Score (RS) for rainfall as a function of lead time during the 1997 monsoon (fs: SI forecast; fd: DMO forecast). 27
Parvinder Maini, Ashok Kumar, S V Singh and L S Rathore In the case of maximum and minimum temperatures, the correlation coefficient and RMSE are obtained between the observed temperatures of the 1997 monsoon and the forecast temperatures (SI/DMO) for the same period. Tables 6(a) and 7(a) show that the correlation coefficients obtained for SI forecasts are higher than those for the DMO forecasts for most of the stations. The corresponding values of RMSE (Tables 6(b) and 7(b)) are lower for SI forecasts than the DMO forecasts for almost all the stations and for all days. This is clearly seen from Figures 5 and 6 where the RMSE values obtained from SI and DMO forecasts are presented for all lead times for Anand, Rahuri, Raipur and Udaipur during the 1997 monsoon. It is also seen from Tables 6 and 7 that the correlation and RMSE for the SI forecast are better than the persistence for some of the stations and comparable for others. Table 6. Verification of maximum temperature forecast during the 1997 monsoon as given by persistence, SI and DMO forecasts using (a) correlation coefficient and (b) RMSE. (a) Correlation Coefficient Station Persistence SI Forecast DMO Forecast Akola 0.86 0.80 0.78 0.78 0.79 0.75 0.75 0.71 0.73 Anand 0.70 0.75 0.67 0.61 0.54 0.71 0.60 0.54 0.55 Delhi 0.65 0.65 0.65 0.64 0.60 0.68 0.65 0.66 0.64 Hisar 0.78 0.79 0.76 0.73 0.67 0.79 0.75 0.74 0.68 Jablapur 0.88 0.88 0.86 0.81 0.81 0.87 0.82 0.80 0.75 Junagadh 0.79 0.63 0.60 0.65 0.64 0.61 0.51 0.47 0.48 Rahuri 0.76 0.53 0.63 0.47 0.50 0.54 0.48 0.56 0.53 Raipur 0.90 0.87 0.85 0.82 0.80 0.86 0.83 0.81 0.80 Ranchi 0.89 0.81 0.83 0.77 0.77 0.83 0.83 0.81 0.79 Udaipur 0.83 0.80 0.70 0.70 0.69 0.86 0.78 0.74 0.75 (b) RMSE Station Persistence SI Forecast DMO Forecast Akola 2.01 2.22 2.33 2.35 2.35 2.66 2.51 2.79 2.70 Anand 2.11 1.89 2.05 2.23 2.54 2.29 2.61 2.88 3.02 Delhi 2.41 2.33 2.44 2.58 2.84 2.93 3.19 3.37 3.60 Hisar 2.17 2.33 2.55 2.69 2.93 2.41 2.68 2.73 3.07 Jabalpur 2.02 2.37 2.34 2.46 2.38 2.44 2.62 2.65 2.95 Junagadh 1.72 1.93 2.01 2.01 2.06 2.07 2.26 2.35 2.34 Rahuri 1.64 1.94 2.01 2.11 2.12 2.13 2.27 2.40 2.55 Raipur 1.87 2.16 2.18 2.26 2.39 2.40 2.46 2.44 2.52 Ranchi 1.60 1.82 1.88 1.96 2.00 1.92 1.91 1.98 2.03 Udaipur 1.80 1.83 2.13 2.17 2.30 1.99 2.28 2.49 2.59 Table 7. Verification of minimum temperature forecast during the 1997 monsoon as given by persistence, SI and DMO forecasts using (a) correlation coefficient and (b) RMSE. (a) Correlation Coefficient Station Persistence SI Forecast DMO Forecast Akola 0.70 0.65 0.64 0.65 0.65 0.69 0.66 0.63 0.65 Anand 0.60 0.50 0.43 0.40 0.37 0.29 0.28 0.29 0.30 Delhi 0.51 0.38 0.47 0.36 0.40 0.17 0.18 0.23 0.29 Hisar 0.58 0.58 0.58 0.49 0.44 0.27 0.15 0.13 0.22 Jabalpur 0.62 0.75 0.71 0.73 0.71 0.68 0.67 0.68 0.60 Junagadh 0.76 0.69 0.68 0.73 0.74 0.70 0.68 0.67 0.70 Rahuri 0.62 0.69 0.66 0.64 0.61 0.43 0.51 0.50 0.51 Raipur 0.57 0.64 0.54 0.55 0.53 0.52 0.50 0.51 0.57 Ranchi 0.61 0.59 0.60 0.63 0.62 0.46 0.61 0.60 0.54 Udaipur 0.68 0.50 0.43 0.45 0.47 0.47 0.38 0.38 0.41
Statistical interpretation of NWP products in India Table 7. continued. (b) RMSE Station Persistence SI Forecast DMO Forecast Akola 1.35 1.27 1.28 1.23 1.26 1.33 1.37 1.43 1.41 Anand 1.30 1.19 1.26 1.32 1.43 1.69 1.67 1.68 1.69 Delhi 1.97 1.93 1.98 2.10 2.02 2.68 2.59 2.63 2.52 Hisar 2.01 2.81 2.49 2.70 2.67 2.94 3.04 3.17 3.16 Jabalpur 1.47 1.11 1.17 1.14 1.22 1.50 1.47 1.37 1.69 Junagadh 1.25 1.30 1.31 1.24 1.21 1.27 1.34 1.38 1.34 Rahuri 1.30 1.11 1.14 1.18 1.22 1.43 1.41 1.36 1.37 Raipur 1.30 1.08 1.20 1.22 1.28 1.75 1.83 1.77 1.71 Ranchi 1.30 1.20 1.22 1.17 1.20 1.30 1.31 1.39 1.52 Udaipur 1.40 1.44 1.56 1.57 1.58 1.80 1.97 1.88 1.90 One important inference that can be drawn is that the SI forecast is a definite improvement over the DMO forecast, and has considerably better skill as compared to persistence and climatology. Hence, the SI forecast has good potential to be used as an operational local weather forecast. The statistical interpretation of the NWP model output has been performed and applied for the first time in India at NCMRWF. The results obtained are quite encouraging. It is likely that certain variations in approach and technique may provide better forecasts. Hence, it is planned to develop PPM/MOS equations based on the operational T-80 model forecast. Different techniques, such as discriminant analysis and logistic regression, will be attempted to try to improve the rainfall forecast. Similarly, neural network and Kalman filter techniques will be adopted to improve the statistical interpretation forecast of rainfall and temperature. Appendix. Measures of accuracy and skill for rainfall Let A, B, C, D be the contents of the following 2 2 contingency table. Forecast (Rain) Yes Observed (Rain) No Yes A B No C D The total number of cases (M) is given by: (a) Ratio Score M = A + B + C + D Ratio Score (RS), also known as the Hit Rate or Percentage Correct, measures the proportion of correct forecasts. The RS varies from 0 to 100 with 100 indicating perfect forecasts. correct forecasts RS = = total forecasts (b) Probability of Detection Probability of Detection (POD) is simply the fraction of those occasions when rainfall occurred as predicted. The POD for perfect forecasts is 1, and the worst POD is 0. It is possible to score well on the POD by overforecasting the occurrence of rain. (c) False Alarm Rate False Alarm Rate (FAR) is that proportion of forecast rain that fails to materialize. The best possible FAR is equal to 0 and the worst possible FAR is 1. (d) Hanssen and Kuipers Score ( ) 100 A+ D M correct rain forecasts POD = = rain observations false alarms FAR = = rain forecasts B A+ B ( ) A A+ C ( ) Hanssen and Kuipers Score (HKS) (Woodcock, 1976) is the ratio of economic saving over climatology due to the forecast to that of a set of perfect forecasts. In HKS the reference hit rate in the denominator is for random forecasts that are constrained to be unbiased. correct forecast correct forecast HKS = M correct forecast HKS = ( ) ( ) ( AD BC) ( A+ C) ( B+ D) random, unbiased random 29
Parvinder Maini, Ashok Kumar, S V Singh and L S Rathore Figure 5. RMSE for maximum temperature as a function of lead time during the 1997 monsoon (fs: SI forecast; fd: DMO forecast). Figure 6. RMSE for minimum temperature as a function of lead time during the 1997 monsoon (fs: SI forecast; fd: DMO forecast). 30
Statistical interpretation of NWP products in India That is, the imagined random reference forecasts in the denominator have a marginal distribution that is equal to the (sample) climatology (Wilks,1995).The value of HKS varies from 1 to +1. If all forecast are wrong (i.e. A = D = 0) then it is 1, and if all forecast are perfect (i.e. B = C = 0) then it is +1, and random forecasts receive a score of 0. Acknowledgments The authors gratefully acknowledge the help of Dr U. C. Mohanty, Professor Indian Institute of Technology, Delhi in providing the ECMWF analysis data. Thanks are also due to Dr L. H. Prakash, Principal Scientific Officer, NCMRWF, for his graphical help. Finally, the authors wish to thank the National Center for Environmental Prediction (NCEP) for providing the adapted version of the T-80 model. References Carter, G. M. (1986). Moving towards a more responsive statistical guidance system. In Preprints Eleventh Conference on Weather Forecasting and Analysis, Kansas City, MO, Am. Meteorol. Soc., 39 45. Carter, G. M., Dallavalle, J. P. & Glahn H. R. (1989). Statistical forecasts based on the National Meteorological Center s numerical weather prediction system. Wea. and Forecasting, 4: 401 412. Glahn, H. R. & Lowry, D. A. (1972). The use of Model Output Statistics (MOS) in objective weather forecasting. J. Appl. Meteorol., 11: 1203 1211. Glahn, H. R., Murphy, A. H., Wilson, L. J. & Jensenius, J. S. (1991). Programme on Short and Medium-range Weather Prediction Research (PSMP), PSMP No.34, WMO/TD No. 421. Wageningen, The Netherlands, World Meteorological Organization. Klien, W. H., Lewis, B. M. & Enger, I. (1959). Objective prediction of five-day mean temperature during winter. J. Meteorol., 16: 672 682. Kumar, A. & Maini, P. (1996). Statistical interpretation of general circulation model: A prospect for automation of medium range local weather forecast in India. Mausam (formerly Indian Journal of Meteorology, Hydrology & Geophysics), 47: 229 236. Kumar, A., Maini, P. & Singh, S. V. (1999). An operational model for forecasting probability of precipitation and YES/NO forecast. Wea. and Forecasting, 14: 38 48. Kumar, A., Maini, P., Rathore, L. S. & Singh, S. V. (2000). An operational medium range local weather forecasting system developed in India. Int. J. Climatol., 20: 73 87. Rousseau, D. (1982). Work on the statistical adaptation for local forecasts in France. In Proceedings of Statistical Interpretation of Numerical Weather Prediction Products, Seminar/Workshop, Reading, United Kingdom, ECMWF: 395 415. Tapp, R. G., Woodcock, F. & Mills, G. A. (1986). The application of model output statistics to precipitation prediction in Australia. Mon. Wea. Rev., 114: 50 61. Wilks, D. S. (1995). Statistical Methods in the Atmospheric Sciences. An Introduction. Academic Press, San Diego, 467 pp. Woodcock, F. (1976). The evaluation of yes/no forecasts for scientific and administrative purposes. Mon. Wea. Rev., 104: 1209 1214. Woodcock, F. (1984). Australian experimental model output statistics forecasts of daily maximum and minimum temperature. Mon. Wea. Rev., 112: 2112 2121 31