Extending logistic regression to provide full-probability-distribution MOS forecasts

Transcription

1 METEOROLOGICAL APPLICATIONS Meteorol. Appl. 16: (29) Published online 9 March 29 in Wiley InterScience ( Extending logistic regression to provide full-probability-distribution MOS forecasts Daniel S. Wilks* Department of Earth and Atmospheric Sciences, Cornell University, Ithaca NY 14853, USA ABSTRACT: Statistical post-processing of dynamical forecasts, using the Model Output Statistics (MOS) approach, continues to be an essential component of weather forecasting. Even in the current era of ensemble forecasting, ensemble- MOS methods are used to transform raw ensemble forecasts into well-calibrated probability forecasts. Logistic regression has been found to be an especially useful method for this purpose for predictands, such as precipitation amounts, that are distinctly non-gaussian. However, the usual implementation of logistic regression fits separate forecast equations for different predictand thresholds, yielding finite sets of threshold probabilities rather than full forecast probability distributions. Furthermore, these individual threshold probabilities are not constrained to be mutually consistent, so that negative probabilities may be implied for some ranges of the predictand. In this paper, logistic regressions are extended to yield full continuous, and coherent, probability distribution forecasts by including the predictand threshold itself as an additional predictor in the forecast equation. The procedure is illustrated using 6 1 day precipitation forecasts for a sample of locations in the U.S., drawn from the GFS reforecast dataset. Copyright 29 Royal Meteorological Society KEY WORDS MOS; ensemble forecasts; reforecasts; generalized linear models Received 22 October 28; Revised 26 November 28; Accepted 13 January Introduction Dynamical forecast models are the mainstay of weather forecasting for lead times of a few hours through a few weeks. However, even at the lead times for which dynamical models are most accurate, their forecast can be improved through statistical post-processing. Usually the Model Output Statistics (MOS) approach (Glahn and Lowry, 1972) is preferred, in which statistical forecast equations are derived that include one or more dynamical outputs as predictors. The result is amelioration of many of the deficiencies in the raw dynamical forecasts, such as conditional and unconditional biases, yielding more accurate forecasts. Most frequently linear regression is used to compute nonprobabilistic MOS forecasts, although nonlinear statistical models can also be used, and probability forecasts can also be made (e.g. Wilks, 26a). MOS-based probability forecasts have usually been formulated as linear regressions predicting binary predictands, following the seminal work of Glahn and Lowry (1972), which is sometimes known as REEP (Regression Estimation of Event Probabilities) (Glahn, 1985). However, this method presents some theoretical and potential practical difficulties, and a good but computationally more demanding approach is to formulate MOS probability forecasting equations as logistic regressions (e.g. * Correspondence to: Daniel S. Wilks, Department of Earth and Atmospheric Sciences, Cornell University, Ithaca NY 14853, USA. dsw5@cornell.edu Wilks, 26a). Logistic regression appears to have been used relatively infrequently for MOS post-processing of conventional single-integration dynamical-model forecasts (Lemcke and Kruizinga, 1988; Vislocky and Young, 1989; Sokol, 23; Detlefsen, 26), even though performance of logistic-regression MOS compares well with the REEP approach (Applequist et al., 22; Sohn et al., 25). Increasingly, forecasting practice includes representation of forecast uncertainties due to uncertainties in initial conditions through integration of ensembles of dynamical forecasts (Molteni et al., 1996; Toth and Kalnay, 1993). However, initial conditions for individual ensemble members are not chosen as random samples from the probability distribution of initial condition uncertainty (e.g. Hamill et al., 23; Wang and Bishop, 23) and, in addition, ensemble forecasts suffer from the same kinds of structural model deficiencies as do conventional single-integration dynamical forecasts. Consequently a forecast ensemble cannot be regarded as a sample from the true probability distribution of forecast uncertainty (e.g. Hansen, 22; Smith, 26), and probability forecasts derived from forecast ensembles can be improved through MOS post-processing. Ensemble-MOS post-processing actually presents a more difficult problem than ordinary MOS forecasting, because both ordinary biases and biases in ensemble dispersion (usually, ensemble underdispersion) are to be corrected. A variety of methods has been proposed for this purpose (Gneiting et al., 25; Hamill and Colucci, Copyright 29 Royal Meteorological Society

2 362 D. S. WILKS 1998; Hamill et al., 24; Raftery et al., 25; Roulston and Smith, 23; Stephenson et al., 25), and these and other ensemble-mos methods have been compared in an idealized setting in Wilks (26b). Wilks and Hamill (27) examined the performance of the best of these methods using ensembles taken from the GFS reforecast dataset (Hamill et al., 26), concluding that nonhomogeneous Gaussian regression (Gneiting et al., 25) generally performed best for medium-range temperature forecasts, and that logistic regression, a conventional statistical method, was generally best for daily temperature forecasts and for medium-range precipitation forecasts. Although probabilistic MOS forecasts based on logistic regressions have been found to perform well, notable difficulties arise from the conventional approach to deriving these equations. Specifically, separate prediction equations are conventionally derived to predict probabilities corresponding to different predictand thresholds. For example, different logistic regression equations would generally be used to forecast probabilities that future precipitation will be no greater than, 2, 5, 1, 2 mm, etc., even though the same predictor variables (which could be, for example, ensemble mean and ensemble standard deviation) might be used in each of the forecast equations. One problem with this approach is that probabilities for intermediate predictand thresholds (e.g. 15 mm in the above example) must be interpolated from the finite collection MOS equations. In addition, fitting separate equations for different thresholds requires estimation of a relatively large number of regression parameters in total, which may lead to poor estimates unless the available training sample is quite large. However, the most problematic consequence of separate MOS equations for different predictand thresholds is that forecasts derived from the different equations are not constrained to be mutually consistent. For example, because of sampling variations the forecast probability for precipitation at or below 2 mm may be smaller than the forecast probability for precipitation at or below 1 mm. All of these problems can be circumvented by extending the logistic regression structure to allow prediction of probabilities for all thresholds simultaneously, by including the predictand threshold itself as one of the regression predictors. In addition to providing smoothly-varying forecast probabilities for any and all predictand thresholds, the approach requires fitting substantially fewer parameters as compared to many separate logistic regressions, and ensures that nonsense negative probabilities cannot be produced. This kind of extension to ordinary logistic regression is not a new concept, and indeed is an instance of the well-known statistical approach called generalized linear modeling (McCullagh and Nelder, 1989). Section 2 outlines use of logistic regression in the context of MOS forecasts, and the extension proposed here. Section 3 describes the ensemble forecast data used to illustrate the procedure, which are the same GFS reforecasts (Hamill et al., 26) used by Wilks and Hamill (27). Note, however, that the proposed structure is equally applicable to MOS post-processing of conventional single-integration dynamical forecasts. Section 4 presents representative forecast performance results, and Section 5 concludes. 2. Logistic regression Logistic regression is a nonlinear regression method that is well suited to probability forecasting, i.e. situations where the predictand is a probability rather than a measurable physical quantity. Denoting as p the probability being forecast, a logistic regression takes the form: p = exp[f(x)] 1 exp[f(x)] (1) where f(x) is a linear function of the predictor variables, x, f(x) = b b 1 x 1 b 2 x 2 b K x K (2) The mathematical form of the logistic regression equation yields S-shaped prediction functions that are strictly bounded on the unit interval ( <p<1). The name logistic regression follows from the regression equation being linear on the logistic, or log-odds scale: [ ] p ln = f(x) (3) 1 p Even though the form of Equation (3) is linear, standard linear regression methods cannot be applied to estimate the regression parameters because in the training data the predictand values are binary (i.e. or 1), so that the left-hand side of Equation (3) is not defined. Rather, the parameters are generally estimated using an iterative maximum likelihood procedure (e.g. McCullagh and Nelder, 1989; Wilks, 26a). An important recent use of logistic regression has been in the statistical post-processing of ensemble forecasts of continuous predictands such as temperature or precipitation (e.g. Hamill et al., 24; Hamill et al., 28; Wilks and Hamill, 27), for which the forecast probabilities pertain to the occurrence of the verification, V, above or below a prediction threshold corresponding a particular data quantile q: p = Pr {V q} (4) In the ensemble-mos context the primary predictor, x 1, is generally the ensemble mean, and to the extent that ensemble spread provides significant predictive information a second predictor x 2 may involve ensemble standard deviation, either alone (Hamill et al., 28) or multiplied by the ensemble mean (Wilks and Hamill, 27). To date, logistic regressions have been used for MOS post-processing by fitting separate equations for selected predictand quantile thresholds. For example, consider probability forecasts for both the lower tercile (the data value defining the boundary between the lower third and

3 MOS LOGISTIC REGRESSION 363 the remainder of a distribution), q 1/3, and upper tercile, q 2/3, of the climatological distribution of a predictand. The two threshold probabilities, p 1/3 = Pr {V q 1/3 ) and p 2/3 = Pr {V q 2/3 ) would be forecast using the two logistic regression functions ln[p 1/3 /(1 p 1/3 )] = f 1/3 (x) and ln[p 2/3 /(1 p 2/3 )] = f 2/3 (x). Unless the regression functions f 1/3 (x) and f 2/3 (x) are exactly parallel (i.e. they differ only with respect to their intercept parameters, b ) they will cross for some values of the predictor(s) x, leading to the nonsense result of p 1/3 >p 2/3, implying Pr {q 1/3 <V <q 2/3 } <. Other problems with this approacharethatestimating probabilities corresponding to threshold quantiles for which regressions have not been fitted requires some kind of interpolation, yet fitting many prediction equations requires that a large number of parameters be estimated. All of these problems can be alleviated if a wellfitting regression can be estimated simultaneously for all forecast quantiles. A potentially promising approach is to extend Equations (1) and (3) to include a nondecreasing function g(q) of the threshold quantile q, unifying equations for individual quantiles into a single equation that pertains to any quantile: or, p(q) = exp[f(x) g(q)] 1 exp[f(x) g(q)] (5) [ ] p(q) ln = f(x) g(q) (6) 1 p(q) One interpretation of Equation (6) is that it specifies parallel functions of the predictors x, whose intercepts b (q) increase monotonically with the threshold quantile, q: [ ] p(q) ln = b g(q) b 1 x 1 b 2 x 2 b K x K 1 p(q) = b (q) b 1x 1 b 2 x 2 b K x K (7) The question from a practical perspective is whether a functional form for g(q) can be specified, for which Equation (5) provides forecasts of competitive quality to those from the traditional single-quantile Equation (1). 3. Data and unified forecast equations Forecast and observation data sets used here are the same as those used in Wilks and Hamill (27). Ensemble forecasts have been taken from the Hamill et al. (26) reforecast dataset, which contains retrospectively recomputed, 15-member ensemble forecasts beginning in January 1979, using a ca (T62, or roughly km horizontal resolution) version of the U.S. National Centers for Environmental Prediction Global Forecasting Model (GFS) (Caplan et al., 1997). Precipitation forecasts for days 6 1 were aggregated to yield medium-range ensemble forecasts for this lead time, through February 25. These forecasts are available on a grid, and nearest gridpoint values are used to forecast precipitation at 19 U.S. first-order National Weather Service stations: Atlanta, Georgia (ATL); Bismarck, North Dakota (BIS); Boston, Massachusetts (BOS); Buffalo, New York (BUF); Washington, DC (DCA); Denver, Colorado (DTW); Great Falls, Montana (GTF); Los Angeles, California (LAX); Miami, Florida (MIA); Minneapolis, Minnesota (MSP); New Orleans, Louisiana (MSY); Omaha, Nebraska (OMA); Phoenix, Arizona (PHX); Seattle, Washington (SEA); San Francisco, California (SFO); Salt Lake City, Utah (SLC); and St Louis, Missouri (STL). These subjectively chosen stations provide reasonably uniform and representative coverage of the conterminous United States. Probabilistic forecasts for 6 1 day accumulated precipitation were made for the seven climatological quantiles q.5 (5th percentile), q.1 (lower decile), q.33 (lower tercile), q.5 (median), q.67 (upper tercile), q.9 (upper decile) and q.95 (95th percentile); estimated using the full 26 year observation data set. The verification data were constructed from running 5-day totals of the midnight-to-midnight daily precipitation accumulations. The climatological quantiles were tabulated locally, both by forecast date and individually by verifying station, in order to avoid artificial skill deriving from correct forecasting of variations in climatological values (Hamill and Juras, 26). For many locations and times of year, two or more of these seven quantiles of 5-day accumulated precipitation are zero, and in these cases only the single zero quantile corresponding to the largest probability was used in regression fitting and verification of forecasts. For example, if % of the climatological 5-day precipitation values for a particular location and date are zero, then both q.5 and q.1 are equal to mm, but only q.1 and the five larger quantiles are used. Again following Wilks and Hamill (27), forecast equations were fitted using 1, 2, 5, 15, and years of training data, and evaluated using cross validation so that all forecasts are out-of-sample. Separate forecast equations were fitted for each day of the 26 year data period, using a training-data window of ±45 days around the forecast date. To the extent possible, training years were chosen as those immediately preceding the year omitted for cross validation, and to the extent that this was not possible the nearest subsequent years were used. For example, equations used to forecast from 1 March, 198 using 1 year of training data were fitted using data from 15 January through 15 April, These procedures were followed both for individual logistic regressions, Equation (1), and the unified formulation in Equation (5), although as noted above only one quantile corresponding to zero accumulated precipitation was forecast and verified in any one instance. Only a single ensemble predictor, the square-root of the ensemble mean, was used in the function f(x): f(x)= b b 1 xens (8)

4 364 D. S. WILKS This predictor choice yields slightly better, but, overall, very similar forecasts, to equations using the untransformed ensemble mean as the single predictor. Adding the ensemble standard deviation or its square root, alone or in combination with the ensemble mean, did not improve either the separate-equation or the unified forecasts, a result consistent with the medium-range precipitation forecast results reported by Hamill et al. (24) and Wilks and Hamill (27), although ensemble spread has been found to be a significant logistic regression predictor for shorter lead times (Hamill et al., 28; Wilks and Hamill, 27). Unification of the logistic regressions for all forecast quantiles was achieved using the square root of the forecast quantile as the sole predictor in the function g(q): g(q) = b 2 q (9) This choice forg(q) was entirely empirical, but yielded substantially better forecasts than did g(q) = b 2 q,and only marginally less accurate forecasts overall than those made using g(q) = b 2 q b3 q. Thus, a full set of separate-equation forecasts (Equation (1)) for a given location and day required fitting as many as 14 parameters (seven equations with two parameters each), whereas the unified approach (Equation (5)) required fitting only three parameters. 4. Results 4.1. Characteristics of the individual and unified logistic regressions Before presenting the forecast verification statistics, it is worthwhile to illustrate the gains in logical consistency and comprehensiveness that derive from using the unified logistic regression framework. Figure 1(a) shows Equation (6), evaluated at 6 selected climatological quantiles, for the 23 November 21 forecast made for Minneapolis, and fitted using the full year training sample, which pertains to accumulated precipitation the period 28 November-2 December 21. Here f(x)= x ens, so that all of the regression lines are parallel, with slope b 1 = mm 1/2. Here also g(q) =.836 q, and the positive regression parameter b 2 =.836 mm 1/2 ensures that the regression intercepts b (q) (Equation (7)) produce forecast probabilities, given any ensemble mean, that are strictly increasing in q. It is thus impossible for the specified cumulative probability pertaining to a smaller precipitation accumulation threshold to be larger than that for a larger threshold. In contrast, Figure 1(b) shows the six corresponding individual logistic regressions, fitted separately for the same six climatological quantiles, using Equation (3) in each case. Here nothing constrains the six fitted equations to be mutually consistent, and indeed they clearly are not. The equations for q.1 and q.33 happen to exhibit similar slopes, as do the equations for q.5, q.67 and q.95, whereas these two groups of regressions are inconsistent with each other, and the equation for q.9 is clearly inconsistent with all of the others. As a practical matter these equations would not yield jointly nonsensical predictions for relatively small values of x ens, but for x ens larger than about 3 mm (the point at which the regression functions for q.33 and q.5 cross) the resulting forecast probabilities overall would be incoherent. Indeed, unless the separate logistic regression equations are exactly parallel, logically inconsistent sets of forecasts are inevitable for sufficiently extreme values of the predictor. Note that the plotted regressions in Figure 1(a) have been chosen to match the threshold quantiles for those fitted in Figure 1(b), but results in (a) 4 q (b) 4 q q.9 =.6 mm =.6 mm ln [p / (1 p)] 2 2 q.67 q.5 q.33 q.1 = 16.5 mm = 4.83 mm = 2.3 mm =.51 mm =. mm Cumulative Probability ln [p / (1 p)] 2 2 q.67 = 4.83 mm q.5 = 2.3 mm q.9 = 16.5 mm q.33 =.51 mm q.1 =. mm Cumulative Probability Ensemble mean, mm Ensemble mean, mm Figure 1. Logistic regressions plotted on the log-odds scale, for 28 November 2 December 21, fitted using the full year training length, for Minneapolis. Forecasts from Equation (6), evaluated at selected quantiles, are shown by the parallel lines in Figure 1(a), which cannot yield logically inconsistent sets of forecasts. Regressions for the same quantiles, fitted separately using Equation (3), are shown in Figure 1(b). Because these regressions are not constrained to be parallel, logically inconsistent forecasts are inevitable for sufficiently extreme values of the predictor.

5 MOS LOGISTIC REGRESSION 365 (a) 1. X ens = mm (b) 1. X ens = mm Pr (V q) X ens = 1 mm X ens = 2 mm X ens = 5 mm X ens = 1 mm X ens = 15 mm Pr (V q) X ens = 1 mm X ens = 2 mm X ens = 5 mm X ens = 1 mm X ens = 15 mm Threshold quantile q, mm Threshold quantile q, mm Figure 2. Probability distribution functions for accumulated precipitation, evaluated at selected values of the x ens predictor, corresponding to Figure 1. The unified Equation (5), plotted in Figure 2(a), yields smooth, continuous and coherent probabilities. In Figure 2(b) results for precipitation amounts between the six fitted equations in the form of Equation (1) must be interpolated, and results for sufficiently large x ens are necessarily logically inconsistent. Figure 1(a) could have been plotted for any precipitation quantiles. Figure 2 shows an alternative representation of the forecasts portrayed in Figure 1, namely cumulative forecast probabilities as functions of the threshold quantile, for selected values of the predictor x ens. (In this plot the characteristic S-shape of the logistic function is distorted because the threshold quantile rather than its square root is plotted on the horizontal axis.) Figure 2(a), computed from the unified regressions (Equation (5)), shows that this approach allows smooth and continuous probability distributions to be specified, for any value of x ens that may be produced by the dynamical ensemble. In contrast, Figure 2(b) illustrates two shortcomings of the traditional approach of fitting separate logistic regressions for different threshold quantiles. First, Equation (1) can only be evaluated for the six threshold quantiles for which regressions have been fitted, shown as the black dots: each vertical column of black dots has been computed from one of the six fitted logistic regressions in Figure 1(b). The implied probability distribution function for other thresholds must be interpolated between these forecast points, which has been done in Figure 2(b) using straight lines. These implied probability distribution functions are not non-decreasing in q, forx ens larger than about 3 mm, as could have been anticipated from the first crossing of the regression lines in Figure 1(b). For example for x ens = 1 mm (a relatively extreme forecast), these individual regressions jointly specify the nonsense result that Pr { <V 16.5 mm)<. Figure 3, again pertaining to the forecast equations portrayed in Figures 1(a) and 2(a), illustrates that the unified logistic regression equation (Equation (5)) provides probabilities that vary continuously as functions of both the predictor and the threshold quantile. The figure shows cumulative probabilities, for values of both ensemble mean and threshold quantiles between and mm. For any given value of the ensemble mean, the median of the forecast distribution (heavy contour) is larger than x ens, indicating a dry bias in the GFS ensemble for this location and date, which is being corrected by this MOS regression Forecast verification Figure 4 compares Brier scores for the individually fitted regressions (Equation (1), x), and the unified fitting for all threshold quantiles jointly (Equation (5), o), for the threshold quantiles used in fitting the individual (Equation (1)) logistic regressions. (Results for q.5 have been Ensemble mean, mm Threshold quantile q, mm Figure 3. Contour plot of cumulative forecast probabilities, Pr {V q}, according to the unified forecast Equation (5), as a function of both q and the forecast ensemble mean, again for the 28 November 2 December, 21 forecast at Minneapolis shown in Figures 1(a) and 2(a). Forecast medians (bold contour) below the 45 diagonal reflect a dry bias in the ensemble for this location and date

6 366 D. S. WILKS (a).175 (b) BSClim (c) (d) (e) Training length, years (f) Training length, years Figure 4. Brier scores aggregated over the 19 forecast locations, for six climatological precipitation quantiles defined locally with respect to both time of year and geographic location, as a function of training-data length. x indicate individual logistic regressions (Equation (1)) fitted for each threshold quantile, and o indicate unified fitting (Equation (5)) for all threshold quantiles jointly. Dashed horizontal lines show Brier scores for constant climatological relative frequencies. (a) 1th percentile, (b) 33rd percentile, (c) 5th percentile, (d) 67th percentile, (e) 9th percentile, (f) 95th percentile. omitted because q.1 = for a large majority of the locations and dates). As expected, all Brier scores improve (become smaller) as the training-data length is increased. For the larger training lengths, overall accuracy as measured by the Brier score is essentially equivalent for the two forecast methods. That is, no price has been paid in terms of reduced accuracy for the gains in smoothness and enforced logical consistency of the unified approach in Equation (5). Interestingly, Brier scores for the unified approach are somewhat better for the shorter training lengths, which presumably reflects the advantage of fitting fewer parameters when available training data are limited. The number of forecasts averaged for each point in Figure 4 is , so that even allowing for effects of temporal and spatial correlation among the forecasts, the associated sampling variability will be very small. Finally, Figure 5 compares reliability diagrams for forecasts of the lower (Figure 5(a)) and upper (Figure 5(b)) terciles, for individual logistic regression forecasts (Equation (1), x) and unified regressions for all quantiles (Equation (5), o), aggregated over all 19 locations, and for the full year training length. Here, forecast probabilities have been rounded and binned to the nearest.5, and points representing fewer than 5 forecasts have not been plotted. Results for the two logistic regression approaches are quite similar for both threshold quantiles, both with respect to the conditional relative frequencies and the inset histograms indicating frequencies with which the different forecast probabilities have been used. The apparent underforecasting for the lower probabilities in Figure 5(b) is produced mainly by forecasts for the two Pacific-coast stations Los Angeles and San Francisco, and may reflect unrepresentativeness of the nearest GFS gridpoints used to compute the ensemble mean predictors for these locations. 5. Summary and conclusions In this paper, conventional logistic regressions pertaining to single predictand thresholds are extended to yield full continuous, and coherent, probability distribution

7 ( MOS LOGISTIC REGRESSION 367 (a) Observed Relative Frequency ( Eq. 5) ( Eq. 1) (b) Observed Relative Frequency ( Eq. 5) Eq. 1) Forecast Probability Forecast Probability Figure 5. Reliability diagrams for (a) lower-tercile (Pr {V q 1/3 }) and (b) upper-tercile (Pr {V q 2/3 }) forecasts, for all locations pooled and the full year training samples. x indicate individual logistic regressions (Equation (1)) fitted for each threshold quantile, and o indicate unified fitting (Equation (5)) for all threshold quantiles jointly. Cases with fewer than 5 forecasts are not plotted. Insets show frequency of use of the binned forecast probabilities. forecasts for any and all thresholds, by including the predictand threshold itself as an additional predictor in the forecast equation. The procedure is illustrated using 6 1 day precipitation forecasts for 19 locations across the U.S., drawn from the GFS reforecast dataset (Hamill et al., 26). A key advantage to the approach is that the resulting forecast probabilities are necessarily mutually consistent, in the sense that the cumulative probability for a smaller predictand threshold cannot be larger that the probability for a larger threshold, which condition is not guaranteed if regression parameters are estimated separately for different thresholds. Indeed, unless (log-odds) regression slopes for single-threshold logistic regressions are exactly parallel, probabilistically inconsistent forecasts will necessarily occur for sufficiently extreme values of the predictand(s). No accuracy penalty, in terms of Brier score, was paid for these benefits, and indeed accuracy of forecasts from the unified framework was better for smaller training samples, apparently because fewer parameters need to be estimated. The results obtained here could almost certainly be improved upon, because the various choices faced in construction of an MOS scheme have not been exhaustively tuned. For example, Equations (5) and (6) were fitted using the arbitrarily chosen set of threshold quantiles q.5, q.1, q.33, q.5, q.67, q.9,andq.95.how many and which such quantiles will yield best results could be investigated in any given instance. Similarly, the forms of the function f(x) (Equation (8)) involving the GFS ensemble mean that were tried included using xens or x ens only, whereas other authors (Hamill et al., 28; Sloughter et al., 27) have reported that better results were obtained with different transformations of the ensemble-mean precipitation predictor. Similarly, no exploration of the use of other predictors (e.g. quadratic functions of x ens, predictor values from more or other GFS gridpoints, or different GFS predictors such as vorticity or precipitable water) have been attempted. Different logistic regression structures may be more appropriate for different locations and/or different times of year, and different predictor variables may be more effective for predictands other than medium-range precipitation totals. For shorter lead times, use of the ensemble standard deviation may improve the forecasts, as has been demonstrated for ensemble-mos post-processing using conventional single-threshold logistic regression (Hamill et al., 28; Wilks and Hamill, 27), and incorporation of an ensemble spread predictor into f(x) would be straightforward. Different forms for the function g(q), which unifies forecasts for all quantiles in Equation (5), will also likely be required in different forecast settings. Here, results using g(q) = b 2 q (Equation (9)) were substantially better than those achieved using g(q) = b 2 q,andalmost indistinguishably less accurate than those achieved using g(q) = b 2 q b3 q (results not shown). In this last case, it was necessary to constrain the maximum-likelihood parameter-fitting procedure to ensure dp(q)/dq >at each iteration. Finally, use of this extended logistic regression approach is not limited to MOS post-processing of ensemble forecasts. It is equally applicable to MOS postprocessing of output from conventional single-integration dynamical forecasts, for which simple linear regressions (i.e. REEP; Glahn, 1985) are often still used, even though logistic regression can provide more accurate forecasts (Applequist et al., 22; Sohn et al., 25). Indeed, a modification of Equation (6) for conventional linear regression, namely p = f(x) g(q), could also be employed in a REEP setting, although fast modern computing capabilities likely justify preference for the probabilistically consistent structure of logistic regression. Acknowledgements Tom Hamill and two anonymous reviewers provided valuable comments on earlier versions of this manuscript.

8 368 D. S. WILKS References Applequist S, Gahrs GE, Pfeffer FL. 22. Comparison of methodologies for probabilistic quantitative precipitation forecasting. Weather and Forecasting 17: Caplan P, Derber J, Gemmill W, Hong S-Y, Pan H-L, Parrish D Changes to the 1995 NCEP operational medium-range forecast model analysis-forecast system. Weather and Forecasting 12: Detlefsen NK. 26. Probability forecasts for weather-dependent agricultural operations using generalized estimating equations. Telllus A 58: Glahn HR Statistical weather forecasting. In: Probability, Statistics and Decision Making in the Atmospheric Sciences, Murphy AH, Katz RW (eds). Westview Press: Boulder, CO; Glahn HR, Lowry DA The use of Model Output Statistics (MOS) in objective weather forecasting. Journal of Applied Meteorology 11: Gneiting T, Raftery AE, Westveld A, Goldman T. 25. Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation. Monthly Weather Review 133: Hamill TM, Colucci SJ Evaluation of Eta-RSM ensemble probabilistic precipitation forecasts. Monthly Weather Review 126: Hamill TM, Hagedorn R, Whitaker JS. 28. Probabilistic forecast calibration using ECMWF and GFS ensemble reforecasts. Part II: precipitation. Monthly Weather Review 136: Hamill TM, Juras J. 26. Measuring forecast skill: is it real or is it the varying climatology? Quarterly Journal of the Royal Meteorological Society 132: Hamill TM, Snyder C, Whitaker JS. 23. Ensemble forecasts and the properties of flow-dependent analysis-error covariance singular vectors. Monthly Weather Review 131: Hamill TM, Whitaker JS, Mullen SL. 26. Reforecasts: an important dataset for improving weather predictions. Bulletin of the American Meteorological Society 87: Hamill TM, Whitaker JS, Wei X. 24. Ensemble reforecasting: improving medium-range forecast skill using retrospective forecasts. Monthly Weather Review 132: Hansen JA. 22. Accounting for model error in ensemble-based state estimation and forecasting. Monthly Weather Review 13: Lemcke C, Kruizinga S Model output statistics forecasts: three years of operational experience in the Netherlands. Monthly Weather Review 116: McCullagh P, Nelder JA Generalized Linear Models, 2nd Edn. Chapman and Hall: London; 511. Molteni F, Buizza R, Palmer TN, Petroliagis T The new ECMWF ensemble prediction system: methodology and validation. Quarterly Journal of the Royal Meteorological Society 122: Raftery AE, Balabdaoui F, Gneiting T, Polakowski M. 25. Using Bayesian model averaging to calibrate forecast ensembles. Monthly Weather Review 133: Roulston MS, Smith LA. 23. Combining dynamical and statistical ensembles. Tellus Series A 55: Sloughter JM, Raftery AE, Gneiting T, Fraley C. 27. Probabilistic quantitative precipitation forecasting using Bayesian model averaging. Monthly Weather Review 135: Smith LA. 26. Predictability past, predictability present. In Predictability of Weather and Climate, Palmer T, Hagedorn R (eds). Cambridge University Press: New York, 217. Sohn KT, Lee JH, Lee SH, Ryu CS. 25. Statistical prediction of heavy rain in South Korea. Advances in Atmospheric Sciences 22: Sokol Z. 23. MOS-based precipitation forecasts for river basins. Weather and Forecasting 18: Stephenson DB, Coelho CAS, Balmaseda M, Doblas-Reyes FJ. 25. Forecast assimilation: a unified framework for the combination of multi-model weather and climate predictions. Tellus Series A 57: Toth Z, Kalnay E Ensemble forecasting at NMC: the generation of perturbations. Bulletin of the American Meteorological Society 74: Vislocky RL, Young GS The use of perfect prog forecasts to improve model output statistics forecasts of precipitation probability. Weather and Forecasting 4: Wang X, Bishop CH. 23. A comparison of breeding and ensemble transform Kalman filter ensemble forecast schemes. Journal of the Atmospheric Sciences 6: Wilks DS. 26a. Statistical Methods in the Atmospheric Sciences, 2nd Edn. Academic Press; 627. Wilks DS. 26b. Comparison of ensemble-mos methods in the Lorenz 96 setting. Meteorological Applications 13: Wilks DS, Hamill TM. 27. Comparison of ensemble-mos methods using GFS reforecasts. Monthly Weather Review 135: