A review of ensemble forecasting techniques with a focus on tropical cyclone forecasting

Transcription

1 Meteorol. Appl. 8, (2001) A review of ensemble forecasting techniques with a focus on tropical cyclone forecasting Kevin K W Cheung, Department of Physics and Materials Science, City University of Hong Kong, Hong Kong (Current affiliation: Department of Meteorology, Naval Postgraduate School, Monterey, CA 93943, USA) This paper presents a general review of ensemble forecasting techniques, with a focus on short-range and tropical cyclone predictions. The basic ideas and terminology of ensemble forecasting are introduced, and using four measures to evaluate an ensemble (ensemble mean forecast, consistency, spread versus skill, and inclusiveness), various potential utilities (e.g. dynamical probabilistic forecasts) are illustrated. Since the perturbation methodologies designed for medium-range forecasts of midlatitude synoptic-scale systems singular vectors, bred modes, and so on are already quite mature, they are only briefly described here. The general problems encountered in applying ensemble forecasting techniques to short-range and tropical cyclone forecasts are diagnosed, and some recent studies on these topics reviewed. In general, the perturbation methodologies used for short-range ensembles to date can have a skill comparable to or slightly higher than their corresponding highresolution control forecast. However, the complicated problem of the relationship between initial condition errors and model deficiencies persists. A similar situation also applies to ensembles designed for tropical cyclone forecasting. An additional difficulty is the different error characteristics encountered in the tropics, mainly the result of the strong convection in the area and the mutual interaction with the ocean. Studies from several research groups use quite different perturbation methodologies, but the results are encouraging. Most of them performed ensemble forecasting of tropical cyclone motion, but extensions to tropical cyclone intensity forecasts are also being developed. 1. Introduction The first operational numerical weather prediction (NWP) was made in the 1950s (Charney, 1951), which heralded a new era of objective weather forecasting. In the following decade, the atmosphere was demonstrated to be a chaotic system (Lorenz, 1963). Academically, this marks the beginning of dynamical system theory. For practical NWP, however, this earlier work estimated a predictability limit to every numerical forecast of about two weeks (e.g. Lorenz, 1982) for the global atmosphere. Since the model errors of early numerical forecasts were still very large, this predictability limit did not undermine them. With continuous improvements being made to NWP models, a medium-range forecast has become a standard product in operational centres today. In order to increase the time period of a skillful forecast further, the barrier of predictability has to be overcome. Against this background, the ensemble forecasting (EF) technique was introduced early in the 1990s in operational centres such as the European Centre for Medium-range Weather Forecast (ECMWF) (Palmer et al., 1993; Molteni et al., 1996); the US National Center for Environmental Prediction (NCEP) (Toth & Kalnay, 1993; Tracton & Kalnay, 1993); and the Recherche en Prévision Numérique (RPN) in Canada (Houtekamer et al., 1996). An ensemble consists of a collection of two or more forecasts that try to realise the possible uncertainties in a numerical forecast. A properly designed ensemble (discussed in subsequent sections) should be a finite approximation of the probability density function of the atmospheric state in phase space. In other words, each ensemble member is an equally probable state of the real atmosphere. The ensemble mean can then act as a non-linear filter such that its skill is higher than that of individual members in a statistical sense (Toth & Kalnay, 1997). Applications of EF techniques to medium-range weather forecasts have been very successful in recent years. For example, Kalnay et al. (1998) quoted a typical forecast case during the winter of 1997/98 from NCEP. The control deterministic forecast (which starts from the analysis) has a higher anomaly correlation (AC) than individual forecasts of a 10-member ensemble. However, the AC of the ensemble average begins to exceed that of the control forecast after about 5 days. Using a threshold value of 0.6 for the AC, the period of skillful forecast can be extended from 7 days to 8 days for this particular case. In light of this kind of success in the medium range, attempts have been made recently to apply EF ideas to short-range weather elements. In some studies it was found that the methodologies taken from 315

2 K K W Cheung medium-range ensembles work equally well in the short range. In general, however, some modifications have to be made because of the different forecast error characteristics associated with NWP models designed especially for short-range predictions. A very exciting development along this line is the formulation of ensemble systems for forecasting a tropical cyclone (TC), the most severe weather disturbance in low latitudes. In addition to the inherent problems associated with small-scale and short-range predictions, a TC ensemble system also possesses dynamical characteristics of the tropics that are different from those in mid-latitude regions. Studies already performed by a few research groups working on this topic (see, for example, Aberson et al., 1995, 1998a, 1998b; Zhang & Krishnamurti, 1997, 1999; Cheung & Chan, 1999a, 1999b; Ramamurthy & Jewett, 1999) have also shown that each particular forecast aspect of TC (e.g. motion, intensity, etc.) in general requires its own special ensemble design. This results in a wide diversity of possible methodologies and a vast range of results to be compared. Several outstanding introductory articles on EF have been published over the past decade. For example, Palmer et al. (1990) gave a background of the necessity to use EF and described some results from the simplest method to generate an ensemble, the Monte Carlo method. Based on the Lorenz (1963) chaotic model, Palmer (1993) presented the predictability problem for extended-range forecast, and illustrated how to compute the fastest-growing errors using a concept known to date as singular vector (SV). This is currently the methodology used in the ECMWF for generating their global ensembles. In the USA, Toth & Kalnay (1993) gave a thorough discussion on the error characteristics of a medium-range NWP forecast. A simple algorithm called the breeding of growing modes (BGM) was proposed to capture the most important portion of errors in an atmospheric analysis. This BGM method has since become the operational setting in NCEP. Sivillo & Ahlquist (1997) is another recent exposition of the general principles of EF. The present paper intends not only to be an addition to these reviews, but also to include three new features. First, this paper serves to provide an updated reference list for forecasters facing an increasing number of ensemble products from almost all the major NWP centres worldwide. Second, the problems associated with short-range ensemble forecasting (SREF) are discussed in detail, including reviews of some previous studies. Lastly, as suggested by the title, a focus is placed on the development of applying EF techniques to TC predictions. Since this is still a very active area of research, the purpose is also to raise the profile of such applications and stimulate discussion within the TC research community. The structure of this paper is as follows. Section 2 outlines the general philosophy of EF, and explains some 316 of the common terminology used in subsequent discussions. As the EF techniques applied to medium-range forecasts are already quite mature and have been reviewed extensively in the literature, they are described only briefly in section 3 in connection with the work under discussion. In section 4, the general problems confronting short-range ensembles are discussed, together with some of the studies already performed in this area. Section 5 is devoted to the specific topic of TC ensembles, and the paper concludes with a summary in section 6. Some of the abbreviations used in the paper are defined in Table Central ideas of ensemble forecasting In this section, the basic procedure of forming an ensemble and some common terminology are illustrated schematically to provide a background for further discussion. By the nature of the governing equations, NWP models are purely deterministic systems. However, in terms of their response to small perturbations they are qualitatively similar to a stochastic system. This is especially true when the system has already evolved for a certain period from the initial time. Therefore, two different periods are usually distinguished for such non-linear systems. In the early linear period (from the initial time to the intermediate forecast projection in Figure 1), small deviations from the initial conditions remain small in the forecasts, and the trajectories of the members in phase space are still close to one another. In other words, the response of the system to small perturbations is quasi-linear. When the non-linear period (from the intermediate forecast projection to the final projection in Figure 1) begins, qualitative differences among the members can result and this can be seen as a clustering of trajectories in phase space. This phenomenon also illustrates the importance of the method of selection of initial conditions for an ensemble. For example, if the method cannot adequately sample the underlying probability distribution of a forecast parameter and the outcomes concentrate only in one of the groups in Figure 1, the Table 1. Definition of some of the abbreviations used in the paper. Abbreviation AC BGM EF EOF LAF LLV MCF SREF SV TC Meaning Anomaly correlation Breeding of growing modes Ensemble forecasting Empirical orthogonal function Lagged average forecast Local Lyapunov vector Monte Carlo forecast Short-range ensemble forecasting Singular vector Tropical cyclone

3 Ensemble forecasting techniques with a focus on tropical cyclone forecasting possible (by enhanced observational techniques), or by simulating explicitly the analysis error growth by an ensemble and then trying to reduce the fast-growing components. The ensemble approach requires the distribution of the perturbations to be close to that of the analysis errors. This distribution in general depends on the entire observational network and data assimilation system, but is unknown. This explains why perturbation methodology is central to any discussion on EF. Figure 1. Schematic diagram illustrating divergence of forecasts in an ensemble with slightly different initial conditions. The heavy line represents the ordinary deterministic forecast starting from an analysis (the cross). At the intermediate forecast projection, the members are still close to one another. At the final forecast projection, clusters (or regimes) may be formed and one of them probably contains the true state of the atmosphere (adapted from Wilks, 1995). true state may be in the other one. The ensemble will then produce a wrong forecast. Anderson (1997) presented four measures to evaluate an ensemble, each of which covers a different aspect of the utility of an EF: ensemble mean forecast, consistency, spread versus skill, and inclusiveness. The issues related to each of these are discussed in the following subsections Ensemble mean forecast An ensemble usually includes the control forecast which is the one starting from the analysis (the best estimate based on available observations) of the atmospheric state. Other members in the ensemble are then generated by adding perturbations (or errors) to the analysis. The method of producing perturbations depends on the particular system under consideration and its associated spatial scale. The simplest way is to add random noise to the original analysis (termed a Monte Carlo forecast, see next section), but this is not an optimal method because the error characteristics in an analysis is often organised or correlated in some way. Toth & Kalnay (1993) argued that two types of errors exist in an analysis: random, non-growing components and well-organised, fast-growing components. The former consist of low-energy perturbations such as gravitational waves, while the latter consist of fastgrowing ones such as baroclinically unstable modes. Therefore, even if the two types of error have similar magnitudes initially, the growing part will soon dominate the total error. For mid-latitude forecasts, Reynolds et al. (1994) demonstrated using error parametrisation that this kind of inherent error growth is more important than the contribution from model deficiencies. The implication is that the skill of forecasting the mid-latitude circulation can be increased in two ways: by making the initial analysis error as small as How does an ensemble work to improve the skill of a forecast? When model deficiencies are negligible, each ensemble member has an equal probability of representing the real state of the atmosphere. The most direct way of producing a single forecast from an ensemble, like the original control run, is to take the linear average (ensemble mean) of all the members. The action of the ensemble mean is shown in Figure 2. Due to the presence of the fast-growing components in analysis errors, the control forecast can be seen to deviate from the real atmosphere right at the beginning of a forecast. If the perturbations only contain the nongrowing random errors, then the ensemble members will almost resemble the control forecast that is, they are still far from the truth. On the other hand, if the growing portion of the errors can be simulated, at least some of the perturbed forecasts (like the one shown in Figure 2) will get closer to the real atmosphere. The ensemble mean should then have a smaller error than the control. This is also the effect of non-linear filtering mentioned in the introduction (and also explained in Toth & Kalnay, 1997). Note, however, that the result of adding originally non-existing error patterns as perturbations to the analysis can be negative. This is because new errors orthogonal to the true analysis errors (from a phase space point of view) will be introduced even after the ensemble averaging. Moreover, the ensemble mean is by definition a smoother field and it can provide guidance on the large-scale flow. For Figure 2. Schematic illustration of the basic components of ensemble forecasting. The control forecast, based on a regular analysis, diverges from the truth right from the start. If the ensemble perturbation consists of fast-growing errors such that the perturbed forecast also diverges from the control, the true evolution can probably be included in a cloud of ensemble (adapted from Toth & Kalnay, 1993). 317

4 K K W Cheung severe events (e.g. intense cyclones and extreme precipitation), which are characterised by small-scale features, other ensemble products such as probabilistic forecasts constructed using ensemble systems have to be considered Consistency Consistency means how well the ensemble members represent the probability distribution of the true state of the atmosphere. As Anderson (1997) explains, if the verifying truth is indistinguishable from a randomly selected member of an EF over a large set of forecast cases, the EFs are said to be consistent with the truth. This can be seen as a prerequisite for applying the ensemble members to perform certain kinds of probabilistic forecast. The degree of consistency can be obtained statistically: if the real number line (using a one-dimensional example) is partitioned by N values taken from a probability distribution, the chance of another value from the same distribution falling into each of the partitions is the same. This is the basis of the so-called binning method for producing and evaluating probabilistic forecasts using ensembles (Anderson, 1996a). However, consistency is a measure independent of the skill of an ensemble; an ensemble with members selected randomly from the model climate distribution is also consistent, but not necessarily skillful. The application of EF to produce dynamical probabilistic forecasts contrasts with previous practice where uncertainties in a forecast are only estimated empirically or statistically. The advantage of this feature can easily be illustrated when the evolution of the ensemble members undergoes a regime change i.e. when the forecasts of the ensemble consist of groups (or clusters) of qualitatively different flows, similar to the situation shown in Figure 1. Since each member is equally probable to be the true state, the relative number of members within a cluster can be used to estimate its probability of occurrence. Usually, the mean of each cluster is calculated and then further examined to see if the one with the highest probability is a reasonable forecast. In the schematic two-dimensional phase space of Figure 1, the clusters can be identified and visualised easily. When actual meteorological fields are concerned, objective criteria for similarity between two fields are needed to identify the clusters. For example, the phase space is sometimes spanned by some kind of orthogonal function so that the distance between two atmospheric states in the function space determines their similarity Spread versus skill Intuitively, greater confidence can be given to a forecast (or the ensemble mean) when the ensemble members are in close agreement with one another (provided that the analysis errors are adequately represented by the 318 perturbations). This indicates a relatively predictable atmospheric environment. Conversely, if the individual ensemble forecasts are all very different, the uncertainty in any one of them is believed to be large and thus the skill can be low. (Strictly speaking, some of the members including the control forecast can still be very skillful; in this situation, an upper bound of forecast error is imposed.) In other words, there exists a possible relationship between the spread (or dispersion) of an ensemble and the skill of the forecast. If this relationship can be established, forecasting the forecast skill becomes another useful utility of EF. This potential forecast skill of an EF system can be verified by defining a perfect ensemble as an ensemble of integrations of a perfect model, which includes the analysis within the range of the ensemble forecasts (Buizza, 1997). This is fulfilled by considering a randomly chosen perturbed member as the verifying analysis. In particular, the distribution of spread around the control and the control skill distribution should be compared; it is also important to assess whether small spread around the control actually indicates a skillful control forecast. When verified using the observed analysis, the percentage of analysis values lying outside the ensemble forecast range should also be examined in addition to the above two requirements under the perfect model assumption. When this notion was proposed in the early days of applying EF (e.g. Kalnay & Dalcher, 1987; Baker, 1991; Leslie & Holland, 1991; Molteni & Palmer, 1991), the focus was on the correlation between the second moment (variance) of the ensemble and the forecast skill. In general, the correlation was found to be sensitive to the verification domain. When the domain is large (e.g. the entire hemisphere), various areas with high and low skill can be identified, which then gives a poor correlation. Further, Wobus & Kalnay (1995) demonstrated that correlation is the best for regions where the forecast skill varies significantly. The seasonal variation of the ability to forecast the skill was also found to be small. One way of extending this kind of consideration concerning correlation between ensemble spread and skill is to examine the higher moments, because forecast error distributions are not necessarily Gaussian. Anderson (1997) also outlined another method based on evaluating the consistency of the ensemble predictions of forecast errors between the ensemble mean and the truth Inclusiveness Inclusiveness is a measure of whether extreme outliers of the ensemble probability distribution are appropriately sampled. Anderson (1997) suggested two crude measures to examine the inclusiveness. One is the rootmean-square (rms) error of the member with the largest rms spread in an ensemble. The second is the so-called worst forecast bust over a large set of ensemble forecasts, i.e. the ensemble for which the ensemble member

5 Ensemble forecasting techniques with a focus on tropical cyclone forecasting with the smallest rms spread is furthest from the verifying truth. These two measures, however, are unstable statistics and no formal statistical test can be applied to them. Other similar measures can also be devised to examine the tails of the ensemble probability distribution. 3. Perturbations for mid-latitude synoptic systems 3.1. Monte Carlo forecast The Monte Carlo forecast (MCF) (Leith, 1974; Palmer et al., 1990) is a purely statistical method in which random noise is added to the analysis as perturbation (see Figure 3(a)). Leith (1974) was the first to give a theoretical analysis of the Monte Carlo method. In his work, the skill (in a climatic sense) of the method as an approximation to stochastic dynamic prediction (Epstein, 1969) was derived. To take into account both internal and external error sources, a regression analysis can also be made to find the best unbiased estimation of the true state. The results in Leith (1974) also imply that the combination of two or more independent forecasts should have less mean-square error than any of the individual ones. This point was also pointed out in Thompson (1977) where the reduction in the mean-square error was calculated for a weighted linear combination of two forecasts. The weight attached to each forecast is determined by the Gaussian method of least squares and depends on the covariances between independent predictions and between prediction and verification. Since its exposition, the Monte Carlo method has been used quite extensively in medium- and long-range prediction. For example, Seidman (1981) applied the method to a general circulation model so that both the point (defined spatially) and phase (defined by the correlation coefficient) predictability are increased by a few days. His ensemble consisted of four to eight members, while Palmer et al. (1990) illustrated an example of a 24-member ensemble made with a T63 (truncated at wavenumber 63) version of the ECMWF model. Post-processing of the ensembles was performed to produce probabilistic forecasts of precipitation and low-level temperature. These were found to be much more informative than a single deterministic forecast. For example, it can indicate locations where rainfall will probably exceed a certain amount, and where severe low temperatures may occur. In most of the applications of the Monte Carlo method, the perturbations added to the control analysis are purely random. This is not an effective way because the actual errors in an analysis are spatially correlated. As pointed out in Palmer et al. (1990), the spatially uncorrelated perturbations will in general project on to nonmeteorological modes which will be dissipated in the model. On the other hand, the set of meteorological modes belongs to the atmospheric slow manifold which excludes the set of faster gravity modes. While MCF has the advantage of ease of generating a large number of members (though still not comparable to the number of degrees of freedom of the atmosphere), it has little use in modelling highly unstable systems such as mid-latitude baroclinic waves (Toth & Kalnay, 1993). The random noise is difficult to organise into these unstable structures, which are often missed by the ensemble. In the context of the Lorenz three-variable system, Anderson (1996b) demonstrated that ensemble forecasts using unconstrained perturbations (those without any knowledge of the attractor structure) have persistent errors throughout the entire integration period. This is the reason why MCF is not adopted as the operational scheme to generate perturbations by major NWP centres such as NCEP and the ECMWF where medium-range mid-latitude prediction is the main objective Lagged-average forecast The lagged-average forecast (LAF) was introduced as an alternative to the MCF (Hoffman & Kalnay, 1983). Figure 3. Schematic time evolutions of the (a) Monte Carlo forecast (MCF) and (b) lagged-average forecast (LAF) ensembles. The abscissa is forecast time t, and the ordinate is the amplitude of a model variable X. X is observed at intervals of time τ and t f is a particular forecast time (adapted from Hoffman & Kalnay, 1983). 319

6 K K W Cheung The methodology uses forecasts that lag the initial time for different periods to be the ensemble members (Figure 3(b)). Therefore the perturbations are the short-range forecast errors resulted from the growth of the initial errors in the analyses according to the dynamics of the model. Dalcher et al. (1988) applied this technique to the ECMWF forecasts and obtained an improvement in skill over the operational ones after five days although the correlation of the spread of the ensemble with the forecast skill is low. Murphy (1990) further examined the practical utility of the LAF in the extended range by running a seven-member ensemble using the UK Meteorological Office (UKMO) global model out to one month. Positive AC for large-scale forecasts can be obtained at days 5 16, but afterwards the AC is usually low. Compared to the mean score among individual members, a modest increase in skill is achieved by ensemble average. Local forecast skill was also correlated with a measure of the agreement between ensemble members, although the overall relationship is weak beyond the medium range (10 days). Application of the LAF in the extended range to enable the ensemble members to capture some specific features in the extratropics (for example a blocking event), has also not been very successful (Branković et al., 1990). The reason is in part due to the systematic errors inherent in the model, but it also indicates the inadequacy of the LAF methodology in providing some skillful members within the ensemble. The quoted results above may indicate that the LAF technique cannot capture the fast-growing errors in the analyses very successfully. To improve this inadequacy, a slight variation of the LAF can be used. Instead of a simple lagged forecast, the difference between shortrange forecasts started at earlier times but verifying at the initial time of the ensemble are taken as perturbations. It was stated in Toth & Kalnay (1993) that after using such perturbations, there was a clear increase in the growth rate of perturbations, and an accompanied increase in the skill of the ensemble mean Singular vectors The method of optimal perturbations using singular vectors (SVs) originates from an early work of Lorenz (1965) in which the predictability of a 28-variable model was studied by examining the growth of small errors added to the model. It was found that the fastestgrowing perturbations can be obtained as the eigenmodes of the matrix product A * A with the largest eigenvalues. Here A(t 0, t 1 ) is the tangent linear propagator of the model between time t 0 and t 1, and A * its adjoint. These eigenmodes are then termed the SVs optimised with respect to the interval (t 0 t 1 ). The idea of SV was first used as a representation of finite-time instability and developed as perturbations in ensemble forecasting at ECMWF (Molteni & Palmer, 1993; Mureau et al., 1993). The SVs are constructed on the T42L19 version of the ECMWF global operational model, and an optimisation interval of 36 h is used (Buizza & Palmer, 1995; Molteni et al., 1996). The fastest-growing SVs are selected and combinations of them are formed to act as perturbations to the model initial conditions. A higher resolution version of the model (T159L31) is then used to produce the ensemble forecasts. Various forms of dissemination of the products exist including stamp maps and probability maps (Palmer et al., 1997). However, the SV calculation in the ECMWF is projected only onto the mid-latitudes (tropical regions between 30 N and 30 S are excluded). Also, moist physics is not included. Therefore, the value of direct application of these SVs to tropical systems is still unknown. Recently, stochastic representation of model uncertainties was introduced into the ECMWF Ensemble Prediction System (EPS) by randomly perturbing the parametrised tendencies of physical processes (Buizza et al., 1999). It was demonstrated that this can effectively increase the spread of the ensemble, and improves the skill of probabilistic precipitation prediction (Mullen & Buizza, 2000). In recent years, many studies have been carried out to refine the optimal perturbations methodology. These include the examination of the effects of the optimisation interval, norm definition and non-linear normal mode initialisation (Buizza et al., 1993); the relationship between singular vectors, normal modes, adjoint modes, Lyapunov vectors and bred perturbations (see next subsection) (Buizza & Palmer, 1995); the impact of orographic forcing on barotropic singular vectors (Buizza, 1995a); sensitivity to the optimal perturbation amplitude (Buizza, 1995b); and study of linear growth of optimal perturbations with different spatial scales (Hartmann et al., 1995). The theoretical justification of the use of singular vectors in ensemble prediction, however, appeared only recently (Ehrendorfer & Tribbia, 1997). It was shown that SVs constructed using covariance information at the initial time evolve into the eigenvectors of the forecast error covariance matrix at the end of the optimisation time period. Work is in progress at ECMWF to use covariance matrix information in the computation of the SVs used in the EPS (Barkmeijer et al., 1998, 1999). Further, a methodology has also been devised to modify SVs for the nonlinear regime (Barkmeijer, 1997) Breeding of growing modes In order to eliminate the linear approximation, and to improve the situation of limited physics associated with the construction of SVs, a simple method called the breeding of growing modes (BGM) was developed in the Environmental Modeling Center (EMC) of NCEP to generate realistic perturbations that can represent the errors actually present in the analyses. The procedure consists of the following steps: 320

7 Ensemble forecasting techniques with a focus on tropical cyclone forecasting 1. Add a small arbitrary perturbation to the atmospheric analysis. 2. Integrate the model for a short period (e.g. 6 hours) from both the unperturbed (control) and the perturbed initial condition. 3. Subtract the control forecast from the perturbed forecast. 4. Scale down the difference field so that it has the same size as the initial perturbation (Figure 4). This kind of breeding cycle is carried on until the growth rate of the perturbations reaches a saturation value about 3 to 4 days for a global model (Toth & Kalnay, 1993, 1994). Afterwards, the difference field should be independent of the initial arbitrary random error. By construction, this method breeds the non-linear perturbations that grow fastest on the trajectory taken by the evolving atmosphere in phase space. It has been verified that the bred vectors are superpositions of the leading local Lyapunov vectors (LLVs) of the atmosphere (Toth & Kalnay, 1997). (The LLVs represent those directions in which growth has been maximum over a long time period (Toth et al., 1996). In other words, any random perturbations introduced an infinitely long time earlier will develop linearly into the leading LLV. The rate of approach to this LLV is characterised by the leading Lyapunov exponent.) Therefore, the bred modes should be able to represent the growing portion in the analysis errors, and hence become an effective means of forming ensembles. There are several fundamental differences of the BGM method over the SV approach. The optimisation period for generating the SVs starts at the analysis time, while the BGM method tries to capture the fast-growing modes in an interval prior to the initial time. During a data analysis cycle, the impact of observations is to change the structure of the first-guess error (Palmer et al., 1998). This change in structure is probably because of scale-dependence of the constraints provided by the observations (more on the large scales, but less on the small scales). This effect of inducing a rotation in phase space, however, is not included in the construction of the bred modes. On amplification rate, Toth & Kalnay (1993) pointed out that the growth rates from both methods are almost identical. Using a low-resolution (T10L18) general circulation model, Szunyogh et al. (1997) also showed similar results when comparing LLVs and SVs. However, this is inconsistent with the rates calculated by Palmer et al. (1998) on a T21L3 quasi-geostrophic model. Actually, there are still a lot of arguments on the appropriateness of these two approaches of generating initial perturbations used in ensemble predictions (Errico & Langland, 1999a, 1999b; Toth et al., 1999). Nevertheless, as stated at the beginning of this section, the BGM has the advantage of convenience in using a non-linear high resolution full-physics model (but not in the SV calculations due to computational efficiency). Those rapidly growing but low-energy modes such as convection can also be represented in the breeding cycles. Since December 1992, the BGM strategy has been implemented for operational use in the EMC/NCEP (Tracton & Kalnay, 1993). Toth et al. (1997) performed a synoptic evaluation of the NCEP ensemble. In general the results are encouraging. The ensemble mean can improve forecast skill by about a day or more in the medium to extended range. Most of the time, the ensemble cloud encompasses the verification so that alternative possible scenarios of the synoptic environment can be indicated. Sometimes, newly developing systems (e.g. extratropical cyclones) can also be detected by some ensemble members a day earlier than the control forecast. In addition, the spread of the ensemble provides useful information about the predictability of the global atmosphere in different areas and at different times, and even potential model deficiencies (when the spread is small but all forecasts verify poorly) Kalman filtering method Figure 4. Schematic illustration of the 6-h breeding cycle. A small arbitrary perturbation is introduced on the control analysis initially. After a 6-h integration, the difference between the control and perturbed forecasts is scaled back to the size of the initial perturbation and this difference field is then added onto the new analysis. After 3 4 days of cycling, the perturbation is dominated by growing modes due to the natural selection of fast-growing perturbations (adapted from Toth & Kalnay, 1993). The Kalman filter is an estimate of the state of the atmosphere based on observations, a first guess, specified error covariances of these two components and the forecast model (Ghil et al., 1981). In other words, it represents the data assimilation system which produces the regular atmospheric analyses. Several studies have tried to construct an ensemble from the basic error covariance structure of these components. Houtekamer & Derome (1995) compared the optimal perturbations (SVs) and the bred perturbations with an observation 321

8 K K W Cheung system simulation experiment-monte Carlo (OSSE- MC) ensemble using a T21L3 quasi-geostrophic model. The OSSE-MC simulates the response of a data assimilation system to errors in radiosonde and satellite observations. It was found that until day 6 the control performs almost as well as any ensemble mean. Afterwards, for all three methods an ensemble size of eight is sufficient to obtain the main part of possible improvement over the control, and its performance is similar for 32-member ensembles. Houtekamer et al. (1996) extended the OSSE-MC ensemble to a system simulation experiment (SSE) for model uncertainties (Figure 5). In addition to the perturbations obtained from the assimilation cycle, the model was also perturbed by using different options for the parametrisation of horizontal diffusion, convection, radiation, gravity wave drag and orography. It was found that the response to the applied perturbations is strongly non-linear. The spread in the ensemble of first-guess fields, however, is too small when validated against statistics available from the operational data assimilation scheme. Therefore, the simulation of the model error sources still needs to be extended, for example, to vertical diffusion and soil moisture field. Based on this kind of knowledge, Houtekamer & Lefaivre (1997) tried to identify an optimised forecast system by minimising a merit function of all the modifications to the system in a least-square sense. The solution is then the best forecasting system that can be obtained at a given truncation using a given set of parametrisations of physical processes and a given set of possibilities for the data assimilation system. 4. Short-range ensembles 4.1. General considerations As shown in the above discussion, the idea of ensemble forecasting originates from the medium and long range in which error growth in the mid-latitude baroclinic waves has large impact to the skill of a numerical forecast. With the perturbation methodologies like SV and BGM proven to be quite successful in the medium range, there has been increasing interest in attempting to estimate the extent to which ensemble ideas are applicable to short-range predictions, and to determine what modifications (both conceptually and practically) are necessary. Short-range predictions usually focus on sub-synoptic and mesoscale systems. Since predictability in general decreases with length scale, it can be said that it is even more difficult to develop a useful ensemble prediction system in the short range. The first problem is that it is necessary to use a very high-resolution model for the ensembles in order to simulate the effects of mesoscale instabilities, which places a heavy load on available computer resources. Therefore, much discussion has Figure 5. Set-up of the system simulation experiment (adapted from Houtekamer et al., 1996). been carried out on the benefit of generating shortrange ensembles where such costs are acceptable. The study of short-range ensemble forecasting (SREF) began with a workshop in 1994 (Brooks et al., 1995b). SREF was assumed to be able to clarify the potential risk of events that pose a threat to property and public safety (e.g. heavy precipitation). Because of its probabilistic form, an ensemble can provide information about the confidence in a given prediction and the relative probabilities of alternative scenarios. In addition, SREF should be able to signal the possible occurrence of rare but significant events (e.g. severe thunderstorms and flash floods) that might otherwise not be indicated with a single model run. Discussion at the workshop focused on three main questions: (a) How close does an analysis have to be to the true state of the atmosphere? (b) How do we deal with the richness of the forecast ensemble? (c) What are the relative roles of errors in the initial conditions and errors in the numerical model in the evolution of the forecast? A one-year pilot study was proposed to assess SREF and address the above questions Studies using the NCEP bred modes Since the 1994 SREF workshop, a series of tests have been reported, many of them on quantitative precipita- 322

9 Ensemble forecasting techniques with a focus on tropical cyclone forecasting tion and severe weather. Since no a priori knowledge on the appropriate characteristics of perturbations for these systems is available, various methods have been proposed by different research groups. For example, in the two case studies presented by Brooks et al. (1995a) and Brooks et al. (1996), the NCEP Eta mesoscale model at a resolution of 80 km was used to forecast a severe storm. The ensemble was generated in two parts: one using different analyses for initialisation, and the other using the bred modes prepared from the global BGM scheme. Later, several forecasts from the NCEP regional spectral model (RSM) with the same resolution were also included as ensemble members, and Hamill & Colucci (1996, 1997) examined in detail the precipitation forecasts from this ensemble. It was found that the verification rank distribution is non-uniform, indicating problems for the members in sampling different precipitation values. This was probably due to the use of both bred and non-bred perturbations in one ensemble, and the use of two models with different error characteristics. However, the mean and medium forecasts from the ensemble could often have less error than the competing 25-km resolution control run. Hamill & Colucci (1998a) further formulated probabilistic precipitation forecasts using the Eta ensembles, although the ability of using the spread among members to predict the forecast skill was still low. This kind of SREF is being tested semi-operationally in EMC/NCEP (Tracton et al., 1998). Similar results were obtained in Stensrud et al. (1999) based on verification of 81 cases from the Etamodel/RSM ensembles. The ensemble mean forecasts for variables such as temperature, relative humidity, geopotential height, winds and cyclone position were all found to have comparable skill with the high-resolution (25 km) Eta control runs. The ensemble spread, when forecasts from both the Eta model and RSM are included, is larger than when a single model is used. This indicates the importance of changing the model physics in SREF when trying to predict the uncertainty in the forecast. However, in terms of cyclone position the overall correlation between spread and skill of ensemble mean is still low. Therefore, opportunity for improvement of such short-range ensembles should still be large Some other formulations Some other studies have re-examined the skill of MCFtype perturbations despite low utility in the medium range. For example, Du et al. (1996, 1997) performed ensemble quantitative precipitation forecasts by adding random noise to the analyses using the Pennsylvania State University (PSU)/National Center for Atmospheric Research (NCAR) Mesoscale Model version 4 (MM4). Again, skill improvement was obtained over a higher resolution run, nearly 90% of which was obtainable using ensemble sizes as small as 8 to 10. Ramamurthy & Shu (1996) and Manikin & Ramamurthy (1996) performed similar experiments on the NCEP Eta model and found that the general problem of MCF being easily projected onto non-growing gravity modes persisted. Leslie & Speer (1998), however, presented a successful application of Monte Carlo-based SREF for an explosive cyclogenesis event. The estimation of uncertainty in the intensification of the cyclone from the ensemble information correctly triggered the issue of a land gale-force warning. To improve the dispersion problem associated with MCF-type perturbations, and to explore the role of model deficiencies in SREF, some other perturbation designs have been made. Bresch & Bao (1996) combined forecasts using two planetary boundary layer schemes and three convective parametrisation schemes on the PSU/NCAR non-hydrostatic MM5. Similarly, Mullen et al. (1999) also tested several convective schemes on a case of cyclogenesis using the same model configuration as Du et al. (1996, 1997). The impact of differences in analysis-forecast systems (where the initial conditions are also changed in parallel with the parametrisation) was found to be quite significant. In addition, Stensrud et al. (1996a), Stensrud et al. (1996b) and Stensrud et al. (1998) used an adjoint model to modify initial and boundary conditions in order to develop specified responses in the model forecasts that are believed to be important. Hamill & Colucci (1998b) developed ensembles in which land-surface conditions are changed. On the other hand, Colucci & Baumhefner (1996, 1998) and Nutter et al. (1996) concentrated on the prediction of the onset of blocking. Generally speaking, most of the results are still preliminary and have involved case studies. The relative usefulness of these methodologies should be subjected to greater verification. 5. Ensemble forecasting of tropical cyclone 5.1. General considerations Compared with the ensemble forecasting of midlatitude weather and climate, that of the tropical atmosphere is a new area of research. The reason for this lies in the different dynamical characteristics of the tropics. In low-latitude regions, the baroclinity is lower while convective turbulence is strong. Because of this, convection becomes a main contributor of model deficiencies to the error budget (and its growth) in NWP models (Reynolds et al., 1994). This means that the EF techniques developed specifically for the mid-latitude instabilities (which, in particular, consider internal error growth) are not necessarily applicable to the tropics. Furthermore, the general sparseness of observations over the tropical oceans increases the error covariance over the region, and this requires an additional rescaling of the perturbation amplitude. (In fact, regional rescaling is also required in the extra-tropics. For 323

10 K K W Cheung example, the effect of the analysis-error covariance matrix is explicitly taken into account in the Hessian singular vector calculations in the ECMWF EPS (Barkmeijer et al., 1998).) This is currently being done for the NCEP operational BGM scheme (Toth & Kalnay, 1997). The tropical cyclone (TC) is the most severe and significant system in the tropics. A better prediction of all its aspects including movement, intensity and precipitation is obviously important. Some work has already been carried out in applying EF ideas to TCrelated problems. In the climatic range, Vitart et al. (1996) used the Monte Carlo technique in a general circulation model to simulate the interannual variability of cyclogenesis. Statistical tests performed on the simulation results indicate that the potential predictability is particularly strong over the western North Pacific and the eastern North Pacific, but weaker over the western North Atlantic. At NCEP, the ability of the global spectral model (with the bred modes as perturbations) to forecast TC genesis was also studied (Aberson, 1999). It was found that for the 1998 season, 64% of the predicted TC developments in the eastern North Pacific and Atlantic basins did occur subsequently with as much as 8 days lead-times, while eight wrong forecasts were made. This performance was reported to be already better than the high-resolution control forecast. In the short range, several studies have been performed on TC motion prediction. The reason is the simplicity of verification that involves the TC position only. In addition, since the general skill of track-prediction models to date is already quite high, occasionally there are cases where the forecasts from different models seriously disagree with one another (Figure 6). Ensemble forecasting can then act as a non-linear filter to eliminate the less probable forecasts. Most recently, ensemble techniques are also being extended to the forecast of TC intensity, which has long been recognised as a very difficult task in TC prediction. One of the major obstacles comes from the complicated energy exchange with the ocean through the air sea interface. In the following subsections, contributions from several research groups towards the improvement of TC prediction through EF techniques are described. Figure 6. Forecast tracks for hurricane Felix (1995) starting at 0600 UTC 14 August from different numerical models. The start time is the third dot from the bottom from where the predicted tracks diverge. The other dots are at 12-h intervals for a 60-h forecast. CLIP is the climatology and persistence forecast and OBS denotes the observed best track (adapted from Krishnamurti et al., 1997). Aberson et al. (1995) directly applied the NCEP ensemble members (perturbed by the bred modes) as initial conditions for the VICBAR barotropic model to forecast Atlantic hurricanes. Later, the more sophisticated Geophysical Fluid Dynamics Laboratory (GFDL) nested model became the operational hurricane model at NCEP, and the bred modes were then applied to it (Aberson et al., 1998a). Verification statistics from the 1996 and 1997 seasons indicate quite encouraging results. The ensemble mean can outperform the control forecast after 24 hours, though the differences are statistically significant at the 90% level only at 72 hours (Figure 7). It can also be slightly better than the operational high-resolution GFDL forecast after 48 hours. However, no explicit correlation between the spread of ensemble tracks and forecast error can be identified. The only signal available is that the cases in which the forecast error is large also tend to have a large spread, but the reverse implication is not necessarily true. When the same ensemble system is extended to TC intensity forecasts, the results are not so satisfactory (Aberson et al., 1998b). For verifications also carried 5.2. Work performed in the Hurricane Research Division/NOAA Figure 7. Comparison of the skill in predicting tropical cyclone track between operational GFDL model (the reference line), the ensemble control (GFCT) and the ensemble mean (GFMN) using the NCEP bred modes as perturbations. Negative value indicates an improvement over the GFDL (adapted from Aberson et al., 1998a). 324