Tubing: An Alternative to Clustering for the Classification of Ensemble Forecasts

Transcription

1 741 Tubing: An Alternative to Clustering for the Classification of Ensemble Forecasts FRÉDÉRIC ATGER* European Centre for Medium-Range Weather Forecasts, Reading, United Kingdom (Manuscript received 23 March 1998, in final form 1 April 1999) ABSTRACT Tubing is a method of classification of meteorological forecasts. The method has been designed to facilitate a human interpretation of the distribution of forecasts produced by an ensemble prediction system (EPS). This interpretation aims to complement probabilistic forecasts generated from EPS weather parameter probability distributions. Ensemble forecasts are generally classified according to their similarities. On the other hand, ensemble distributions rarely prove multimodal in practice. The tubing method disregards possible modes and gives instead more emphasis to the central part of the distribution where the ensemble mean is located. Ensemble forecasts are then classified according to the way they differ from the ensemble mean. The tubing algorithm first groups the members lying around the ensemble mean into the central cluster. Remaining ensemble members are classified into a number of tubes. A tube is a cylinder ranging from the central cluster to an extreme member of the distribution. The central cluster is represented by its centroid, meant to indicate the meteorological pattern that is the most likely to verify. Tubes are represented by their extreme members, full resolution representatives of possible deviations from the main scenario indicated by the central cluster. 1. Introduction Ensemble prediction systems (EPS) have been developed in an attempt to deal with the intrinsic uncertainty of numerical weather forecasts. The principle of ensemble prediction is to forecast the impact of the uncertainty of initial conditions, and/or of model formulation, on the uncertainty of the forecast. This is done by running a large number of numerical forecasts from slightly different initial states, and/or with slightly different model configurations (Leith 1974). Daily medium-range EPSs are now operational at the National Centers for Environmental Prediction (NCEP) and at the European Centre for Medium-Range Weather Forecasts (ECMWF). These two EPSs take only into account the uncertainty of the initial conditions, by introducing carefully selected perturbations in the analysis. They are described in Tracton and Kalnay (1993) and Palmer et al. (1993), respectively. Other centers run EPSs in an experimental or almost operational setting, for mediumrange forecasting (e.g., Houtekamer et al. 1996) or short-range forecasting (Tracton and Du 1997). Unless otherwise stated, this paper generally refers * Current affiliation: Météo-France, Toulouse, France. Corresponding author address: Frédéric Atger, Météo-France (SCEM/PREVI), 42 Avenue G. Coriolis, Toulouse Cedex, France. frederic.atger@meteo.fr to the ECMWF Ensemble Prediction System, which comprises, in its current, high-resolution version implemented in December 1996, 50 perturbed and an unperturbed, control integration of a TL159L31 version of the ECMWF model (Simmons et al. 1989; Courtier et al. 1991). Initial perturbations are obtained through the computation of the singular vectors of the tangent propagator of the ECMWF model. The methodology is described in detail in Molteni et al. (1996) while Buizza et al. (1998) discuss the advantages of the new, highresolution system. Ensemble prediction generates huge amounts of data. One of the main problems for operational use is the design of manageable products for potential users. The daily output of an EPS is a collection of forecasts, each of them being a possible realization of the meteorological future. The probability distribution of any weather parameter can be derived from this collection: 2-m temperature, quantity of precipitation, wind speed and direction, etc. Statistical methods may be involved to get a weather parameter probability distribution, for instance, parametric adjustment (Wilson 1995) or calibration (Hamill and Colucci 1998). The probability of virtually any meteorological event can be derived from these distributions, for instance the probability of observing frost at least once during a dry period of 48 h, with the wind speed never exceeding 10 kt. Therefore, a complete set of weather parameter probability distributions should fulfill all operational requirements. On the other hand, end users are not necessarily satisfied solely with probabilities. They certainly appre American Meteorological Society

2 742 WEATHER AND FORECASTING VOLUME 14 ciate explanations, designed to help them to make their decisions from the probabilistic information. These explanations are not supposed to add any information to the probabilistic forecast. Rather, they are meant to allow the end user to understand how the probabilities of weather elements occur, by mentioning the various meteorological possibilities. This added value can only be the result of an interpretation of ensemble products by an experienced forecaster, able to understand for instance that a 20% probability of strong precipitation is associated with a given distribution of ensemble forecasts. The problem is that the distribution of ensemble forecasts is rather difficult to interpret on a meteorological basis. It is almost impossible to catch in real time the differences and similarities between a large number of forecasts, even over a limited area. Different products have been designed in order to facilitate the forecasters interpretation (e.g., Tracton and Kalnay 1993; Molteni et al. 1996). Some products allow visualization of specific aspects of the distribution. For example, the ensemble mean field is used as a smooth forecast of the large-scale pattern. The ensemble spread, represented for instance by the standard deviation with respect to the mean, indicates the uncertainty associated to the different meteorological features. Other products are based on an objective classification of ensemble members, indicating the main alternative meteorological patterns. The tubing method belongs to that last category and is proposed as an alternative to clustering technics. The paper is organized as follows. Different methods of classification are discussed in section 2. Some characteristics of ensemble distributions are investigated in section 3. The tubing method is described in section 4. An example of meteorological interpretation based on the tubing classification is discussed in section 5. Some applications of the method to ensemble verification are proposed in section 6. The usefulness of tubing products in an operational forecasting environment is discussed in section 7. The main characteristics of the method are summarized and further discussed in section Classification of ensemble forecasts a. The choice of a classification method Classification of ensemble forecasts is considered in this paper as a way of condensing the voluminous information content of EPSs, in order to facilitate the interpretation performed by forecasters in operational forecasting offices. Classification of ensemble forecasts may be useful for other purposes, for instance, to investigate the structure of growing perturbations or to evaluate the performance of an EPS in detecting bifurcations on the attractor. The qualities of a method obviously depend on the precise aim of the classification. In this section, different methods of classification are compared with respect to their potential usefulness in an operational forecasting environment only. Besides the intrinsic strengths and weaknesses of the different classification methods in that respect, the actual characteristics of ensemble distributions play an important role in the choice of an appropriate method. Some of these characteristics are discussed in section 3. Operational forecasting requires effective ways of displaying relevant information from numerical prediction. An effective display allows a quick and fruitful conceptualization of the future state of the atmosphere, leading a forecaster to achieve a weather forecast. In the case of ensemble prediction, relevant information comes from the distribution of ensemble forecasts. A direct, meteorological interpretation of this distribution is almost impossible when the number of ensemble members is large (e.g., 51 members for the ECMWF EPS). The aim of an objective classification method is to facilitate this interpretation by highlighting the most significant features of an ensemble distribution. Classification methods generally consist in grouping ensemble forecasts in a certain number of classes that indicate alternative meteorological options. The result of the classification should give a proper representation of the meteorological spread, that is, the way ensemble forecasts differ one from another in a meteorological sense. This representation should be effective; that is, it should meet the forecasters requirements and actually facilitate interpretation. On the other hand, the number of classes should be limited in order to allow a quick and easy interpretation. In the present paper, the performance of a classification method is defined as its ability to give an effective representation of the meteorological spread with a limited number of classes. b. Conditional classification methods Conditional methods are based on a classification of meteorological patterns from the climatology. For example, a classification of ECMWF EPS forecasts is performed daily in Switzerland, for operational forecasting purposes. The basis is a 3-yr classification over central Europe, obtained by running a neural network method (Eckert et al. 1996). Ensemble forecasts are classified into 64 classes and the EPS distribution is graphically represented by the distribution of forecasts onto a 8 8 chessboard. Other conditional classifications have been based on the existence of large-scale weather regimes, more likely to be predictable in the late medium range (Santurette et al. 1998, manuscript submitted to Quart. J. Roy. Meteor. Soc.). The conditional classification methods have the advantage of resulting in a limited number of configurations. Therefore, meteorological interpretation may be coupled with objective postprocessing based on the distribution of ensemble forecasts in the different classes (Cattani et al. 1997). On the other hand, the representativeness of the classes for human interpretation is limited by the accuracy of the defined classification. If the ensemble spread is par-

3 743 ticularly small, all forecasts are likely to be found in the same class and smaller-scale uncertainty may be not properly represented. If the ensemble spread is larger than normal, ensemble forecasts are disseminated in a large number of classes and the most predictable largescale pattern may be not easily recognized. c. Unconditional classification methods 1) THE WARD CLUSTERING METHOD Several unconditional algorithms exist (Anderberg 1973), most of them being agglomerative and hierarchical, that is, merging at each stage the two nearest clusters in a new cluster. Algorithms mainly differ according to the chosen distance. The Ward algorithm (Ward 1963) is the most widely used in atmospheric sciences (Brankovic et al. 1990; Palmer et al. 1993). It is used at ECMWF to produce an operational classification of EPS forecasts (Molteni et al. 1996). The distance between two clusters is a function measuring the decrease of the variance between clusters due to the merging. Because it tends to assign a member to a less populated cluster, the Ward algorithm results in clusters of comparable internal variance. For the same reason, members exhibiting important differences tend to group when they are located in a lower density part of the distribution. When applied to an EPS distribution (e.g., the 500- hpa geopotential height over Europe) the Ward classification generally results in 1) a number of big, heavily populated clusters, whose centroids are located in the higher density part of the distribution and are not very different from one another; and 2) a number of small, lightly populated clusters, whose centroids are located in the lower density part of the distribution and exhibit significant differences. The performance of the classification appears limited, in the sense that an acceptable number of clusters often gives a rather poor representation of the meteorological spread. This performance apparently depends on the degree of multimodality of the distribution, only a highly multimodal distribution leading to clearly separated large clusters. 2) OTHER CLUSTERING METHODS An alternative to the Ward classification method is the basic hierarchical algorithm, often called the centroid algorithm, which uses the root-mean-square (rms) difference as a distance. This algorithm is generally not used in climatological studies because it tends to group a large majority of members in one big cluster ( snowballing effect ) while a significant proportion of members are left alone (Kalkstein et al. 1987). This feature is almost systematic when the method is applied to EPS forecasts, unless the distribution is highly multimodal. For interpretation purposes, however, the fact that outliers are left alone can be seen as an advantage over other methods, for example, the Ward clustering, which does not represent properly the edges of the distribution. On the other hand, a large number of forecasts left alone is a definite shortcoming in an operational environment, given the time and technical constraints. Another alternative is the algorithm used at NCEP to produce operational clusters (Tracton and Kalnay 1993). It is an agglomerative algorithm using the anomaly correlation coefficient (ACC) as a distance between ensemble forecasts. An interesting feature is that the algorithm starts from the detection of the two ensemble forecasts that are most dissimilar. Around these outliers, the algorithm builds two clusters by grouping the closest members up to a certain threshold. The process is then iterated until all members are classified. This algorithm has the advantage of highlighting the dissimilarities between ensemble outliers, rather than the similarities between ensemble members likely to be located in the higher density part of the distribution. On the other hand, the choice of the threshold is crucial in order to avoid the formation of two large clusters whose centroids are similar, because of clustering too many forecasts being not similar enough to the corresponding outlier. Despite the differences of conception, subjective comparison has shown that the result of this classification is often similar to that obtained by the Ward method. 3. Characteristics of ensemble distributions a. Multimodality The multimodality of ensemble distributions was implicitly assumed by the developers of EPSs when they anticipated possible operational applications. Ensemble forecasts may cluster into groups of similar trajectories (Tracton and Kalnay 1993) so that a simple way of visualizing the meteorological distribution would be a cluster analysis. This idea was indeed supported by theoretical results on planetary-scale low-frequency atmospheric variability (e.g., Vautard and Legras 1988). On the other hand, meteorological features that are of interest for operational weather forecasting generally have a smaller scale. The lack of multimodality with respect to synoptic-scale features was subjectively noticed by forecasters at the very start of operational ensemble prediction. The first effect of this apparent unimodality is the unsuitability of traditional classification technics. For example, operational clusters produced at ECMWF were criticized as being too similar and not reflecting properly the meteorological spread (Reed and Pickup 1993). This behavior might be a consequence of a failure of the initial perturbations to represent properly a bifurcation on the attractor (Anderson 1997). Nevertheless, it should not be considered as a definite shortcoming of an EPS since it is not clear whether such bifurcations do exist in the atmosphere with respect to

4 744 WEATHER AND FORECASTING VOLUME 14 FIG. 2. Values taken by the multimodality index defined in Fig. 1 for (bottom) a set of 31 ECMWF EPS distributions, (middle) a set of 31 EPS-like monomodal distributions, and (top) a set of 31 EPSlike bimodal distributions. The circle indicates the mean value. The bars indicate, respectively, the 50%, 80%, and 100% intervals, according to the thickness. (500-hPa geopotential height, Europe, 144 h, winter ) FIG. 1. Example of explained variance curve for a Ward clustering classification applied to the ECMWF EPS. The curve represents the ratio of the variance between clusters to the total ensemble variance, as a function of the number of clusters considered at each stage of the agglomerative process (see text). The multimodality index (here 0.79) is defined as the normalized area below the curve (20 Oct 1997, 144 h, 500-hPa geopotential height, Europe). the synoptic-scale usually taken under consideration for operational weather forecasting purposes. A multimodal signal is very difficult to extract with statistical confidence from an ensemble distribution of meteorological fields, given the large number of degrees of freedom and the relatively small number of ensemble members. A multimodality index has been designed to illustrate the lack of multimodality of ensemble distributions when considering meteorological fields over Europe. The index is based on the Ward clustering algorithm described in section 2c(1). At each stage of the agglomerative process, the ratio R of the variance between clusters and the total ensemble variance is computed. Since the Ward algorithm maximizes R at each stage, the area below the curve obtained by plotting R versus the number of clusters taken into account indicates to what extent the forecasts are grouped into natural clusters (Fig. 1). The multimodality index is defined as the integral of R from n clusters to 1 cluster, n being the number of members to classify. Note that the index has no sensitivity to the amplitude of ensemble spread. The multimodality index was computed for a set of 31 EPS 144 h forecasts of 500-hPa geopotential height over Europe, randomly extracted from the winter season 1996/97 (10 December February 1997). The index was then compared to two reference indices. The first reference index was obtained from a set of 31 ensembles comprising 51 individual EPS forecasts randomly extracted from the winter season 1996/97. The considered distribution is assumed to be unimodal, at least as unimodal as the climate distribution (with the advantage of being representative of the model climate during the considered season). The second reference index was obtained from a set of 31 ensembles comprising individual EPS forecasts extracted from two randomly selected dates extracted from the winter season 1996/97. The considered distribution is assumed to be significantly bimodal. The two reference distributions were subjectively checked out, in order to confirm the unimodal and bimodal assumptions. The comparison of the multimodality index for the three distributions shows that only a fraction of EPS cases can be considered as exhibiting a bimodal distribution, while a large majority have indices close to those relative to unimodal distributions (Fig. 2). b. Ensemble mean and ensemble mode The ensemble mean medium-range forecast is known to be on average a closer estimate of the verification than individual members, including the control forecast (e.g., Buizza and Palmer 1998). This might be a consequence of the ensemble mean lying closer to the climate mean than individual members. As demonstrated by Toth (1991), forecasts of meteorological states similar to the climate mean prove more skillful, on average, than forecasts of more anomalous states. This is due to the higher density of meteorological states around the climate mean. The benefit of ensemble averaging seems to derive mainly from smoothing out small-scale, less predictable features, while retaining the more slowly varying large-scale pattern, as predicted by Leith (1974). A similar effect can be achieved at a lower cost by filtering a high-resolution single model integration, for instance, by retaining only the leading components of a spectral decomposition according to the ensemble mean wave spectrum. The performance of the ensemble mean remains, however, slightly better than such a filtered forecast (Bouteloup 1997). Toth and Kalnay (1997) also showed that the ensemble mean performs considerably better than an optimally filtered control forecast.

5 745 FIG. 3. Frequency of gridpoint verification in the interval containing the ensemble mean (solid line) and in the interval containing the largest number of ensemble forecasts (dashed line), when the two intervals are not identical. Frequencies are plotted versus the number of intervals dividing the total range of ensemble forecasts (from the minimum to the maximum forecast value). Also plotted is the frequency of verification in one interval chosen randomly. (ECMWF EPS, 500-hPa geopotential height, 144 h, Europe, Jan Feb 1997.) Although ensemble distributions have generally a low degree of multimodality, one or several modes may be observed in certain situations. The ensemble mean performance should be much reduced in these cases. As pointed out by Palmer (1993) one could expect the ensemble mean to perform better than all ensemble members only when there is no bifurcation in the largescale meteorological pattern evolution. In other words, the mode(s) of the distribution should perform better than the ensemble mean. The performance of the main ensemble mode has been assessed by dividing the ensemble range at every grid point in a certain number of intervals. The frequency of verification in the interval containing the ensemble mean has been compared with the frequency of verification in the most populated interval, when these two intervals are different, that is, when a mode exists (Fig. 3). When the number of intervals increases, the curves indicate that the most likely interval to find the verification is the ensemble mean interval rather than the most populated one. This is true at all time steps, even in the short range (Fig. 4). This result is confirmed in terms of cluster verification, when considering the distribution of ensemble forecast fields over a given area. The central cluster is defined as a spherical cluster grouping the members lying around the ensemble mean, up to a certain distance. The performance of the central cluster is compared with the performance of spherical clusters of the same radius, centered on all ensemble members. The central cluster proves more likely to verify, that is, to contain the verification within its boundaries, than the most populated FIG. 4. Same as Fig. 3 but the total range of ensemble forecasts is divided into 12 intervals for each time step from 24 h to 240 h. spherical cluster (Fig. 5). In other words, when there exists a higher density of forecasts elsewhere than around the ensemble mean, the verification is still more likely to be found around the ensemble mean. 4. The tubing classification method a. The tubing method Tubing is an unconditional method of classification of ensemble forecasts, designed to suit the actual characteristics of ensemble distributions, highlighted in section 3, and to meet operational forecasters requirements. The method has been developed for operational forecasting applications, as discussed in section 2. The aim is thus to get a classification giving a proper representation of the meteorological spread. Two aspects are emphasized by the tubing classification: (i) the expected main characteristics of the future state of the atmosphere; and (ii) the uncertainty of the meteorological evolution. This dual representation is believed to meet operational forecasters requirements in the sense that it is compatible with a conventional, deterministic approach of weather forecasting. Given the lack of multimodality of ensemble distributions, the performance of conventional clustering methods, in terms of representation of the meteorological spread, is necessarily limited. Rather than grouping ensemble forecasts around hypothetical centroids, the first aim of tubing is to classify most forecasts in one big cluster that highlights the main features of the future meteorological state. In this sense, tubing is derived from the basic hierarchical clustering described in section 2c(2). Since the main ensemble mode is less performant than the ensemble mean, the main cluster of the tubing classification is not necessarily built from the

6 746 WEATHER AND FORECASTING VOLUME 14 FIG. 5. (main graph) Frequency of verification in the central cluster (solid line) and in the most populated spherical cluster of the same radius, centered around an ensemble forecast (dashed line), when the latter is more heavily populated than the former. Frequencies are plotted for 10 classes of central cluster population. (inset) Number of cases in each class (the number of cases in the 10th class is too small to appear on the graph). (ECMWF EPS, 500-hPa geopotential height, winter , 12 areas over the Northern Hemisphere.) forecasts lying in the higher density part of the distribution. Instead, tubing starts from the so-called central cluster defined in section 3b, that is, the cluster that groups those forecasts lying around the ensemble mean. Ensemble members that are left aside after the first step of the classification are not grouped into clusters. Given the lower density of ensemble members in the outer part of the distribution, clustering would have the effect of (i) grouping very different forecasts together, and/or (ii) keeping alone a number of outliers. Instead, ensemble forecasts are classified into tubes, along a certain number of axes starting from the ensemble mean. Each axis ends on one of the extreme members of the distribution, that is, one of the members that are most remote from the ensemble mean. The axes of the tubes are meant to represent the directions in which the verification is likely to deviate from the ensemble mean. b. The tubing algorithm The tubing algorithm is illustrated in Fig. 6. A detailed description can be found in appendix A. First, the central cluster is obtained by grouping a certain number of members lying around the ensemble mean. The number of selected members depends on the chosen configuration, as discussed in section 4c. In a second stage, the member that is the most remote from the ensemble mean is located. It becomes the first extreme and defines a tube grouping the members lying in the cylinder whose axis of symmetry goes from the ensemble mean to the FIG. 6. Schematic representation of the tubing algorithm. The central cluster groups those members lying near the ensemble mean. The tubes are defined by their extremes, which are the farthest members from the ensemble mean. extreme of the tube. The radius of the tube is the same as for the central cluster. The process is iterated until all members are classified, with the following role: a member belonging to a tube cannot become an extreme but can still be classified in another tube. When classification is completed ensemble forecasts grouped in the central cluster are averaged. The central cluster mean is similar to the ensemble mean, by construction, but the smoothing effect is limited since the internal spread is reduced compared with the total spread. It indicates the meteorological option to follow in a deterministic approach. The tube members are not averaged. Tubes are instead represented by the extremes, in order to highlight the nature of the common differences between the tube members and the ensemble mean. Therefore the main information is not to be found in the extreme of the tube, but in the difference between this extreme and the central cluster. This difference has to be interpreted as indicating a possible meteorological deviation from the central cluster option. Although any distance may be chosen in the tubing algorithm, the rms difference has been used in all examples shown in this paper, for the sake of simplicity. The choice of a more appropriate distance needs further investigation in order to take into account all the aspects of the problem. Because meteorological variables at different grid points are likely to be correlated, a promising alternative is to work with a Mahalanobis metric in order to get more robust classification results (Stephenson 1997). The rms difference in the Mahalanobis space is equivalent to the square root of the sum of the squares of the leading principal components (PCs), so that it can be interpreted as an rms difference in PC space. Concerning the algorithm, a number of alternative options exist in the choice of the geometry of the tubes.

7 747 For example, tubes have been preferred to cones, starting from (or ending with) the ensemble mean. Also, the radius of the tubes might be different from the radius of the central cluster. Tubes might be represented by averaged fields rather than extremes. The final choice was not arbitrary but made after a subjective evaluation of the different possible configurations. The proposed algorithm is believed to produce an optimal representation of an ensemble distribution for meteorological interpretation. c. Tubing configurations One important parameter of the tubing algorithm is the condition limiting the size of the central cluster when grouping the members lying around the ensemble mean. Two main configurations can be distinguished. 1) Spread-dependent configuration: the limitation depends on the actual ensemble spread. For example, the central cluster variance is limited to 50% of the total ensemble variance. 2) Season-dependent configuration: the limitation depends on the expected, climatological ensemble spread. For example, the central cluster variance is limited to 50% of the expected total ensemble variance. In a spread-dependent configuration, the size and the internal spread of the central cluster vary every day according to the ensemble spread. The smoothness of the averaged field varies consequently, high spread resulting in a smoother field. On the other hand, the population of the central cluster and the number of tubes vary little from day to day. In this configuration, the proportion of information lost in averaging ensemble members in the central cluster is the same everyday. In a season-dependent configuration, the smoothness of the central cluster averaged field varies little from day to day. This is because the internal spread of the central cluster follows the seasonal, slow variations of the meteorological uncertainty. On the other hand, the number of tubes varies from day to day according to the actual ensemble spread. There may be no tube at all in a case of small spread, when all members are grouped in the central cluster, as well as plenty of them when the spread is especially large. Both configurations apply similarly to agglomerative clustering algorithms. A season-dependent limitation of the clusters standard deviation has been applied for several years to the ECMWF EPS clustering classification. A similar configuration is now applied to the ECMWF EPS tubing classification. Indeed, this configuration appears the most suitable for operational purposes. It allows forecasters to directly link the number of meteorological alternatives, given by the tubes or the clusters, to the forecast uncertainty. Also, in the case of the tubing, the limitation of the internal spread of the central cluster allows one to avoid very smooth averaged fields FIG. 7. Tubing classification applied to a two-dimensional ensemble extracted from a Gaussian distribution. The central cluster (dashed line) groups 32 members around the ensemble mean (large square). Other members are classified into five tubes (solid lines) defined by their extremes (circled members). Ensemble members are represented with symbols referring to the Ward clustering classification shown in Fig. 8. when the spread is especially large. This makes the central cluster mean more suitable than the ensemble mean, as a representative of the main forecasting option for interpretation purposes. The operational season-dependent configuration implemented at ECMWF is described in appendix B. The spread-dependent configuration is rather suitable for verification purposes and case studies, since it allows one to visualize a given proportion of information (i.e., of variance) from the ensemble distribution, independently of the normal, expected ensemble spread. d. Tubing versus clustering The differences between the clustering and the tubing methods are important. Clustering algorithms generally focus on ensemble forecast similarities, whereas tubing groups ensemble forecasts according to the fact that they differ from the ensemble mean in a similar way. To illustrate these differences, the tubing and the Ward algorithms have been applied to a two-dimensional ensemble of 51 points randomly extracted from a Gaussian distribution. This idealized ensemble, although based on a unimodal distribution, exhibits local sampling accumulations (Figs. 7 and 8). The same configuration has been used to limit the population of the central cluster of the tubing classification and the clusters of the Ward classification: the standard deviation about the cluster centroid is limited to 50 gpm. The Ward algorithm seems rather deficient in representing such a unimodal distribution. The boundaries between the clusters look ar-

8 748 WEATHER AND FORECASTING VOLUME 14 FIG. 8. Ward algorithm classification applied to the same twodimensional ensemble as in Fig. 7. Six clusters group 8, 9, 12, 10, 6, and 6 members (smaller symbols) around their centroids (larger symbols). bitrary, often separating members that are very close to each other. The cluster centroids are close to each other and to the ensemble mean, and give a poor representation of the spread (Fig. 8). On the contrary, the tubing classification widely covers the whole distribution by localizing the center and the extremes (Fig. 7). Tube boundaries appear as arbitrary as cluster boundaries, but this should not be considered a drawback since members in the tubes are not supposed to be averaged. Rather, the tube extremes indicate the direction of possible departures from the ensemble mean. 5. Forecasting with tubes: An example of interpretation The possible use of the tubing classification is discussed in this section on a forecast case study. The 144-h ECMWF EPS forecast based on 22 January 1997 had a large spread over western Europe. A ridge was forecast by the T213 ECMWF model to build up off Ireland and a consequent northerly anticyclonic flow to establish over Britain and France (Fig. 9). The EPS control forecast (not shown) was almost identical to the T213 model forecast. The ensemble spread mainly related to the development of the ridge and its effects on the circulation over western and central Europe. FIG. 9. ECMWF T213 model forecast based on 22 Jan 1997, 120 and 144 h: 500-hPa geopotential height. a. Tubing The tubing classification has been applied to EPS forecasts with a 60-gpm threshold for the central cluster standard deviation. The threshold was chosen empirically in order to get an appropriate number of tubes (four). The actual threshold in a season-dependent configuration would be around 80 gpm according to statistics based on several winter seasons. This indicates that the spread was slightly smaller than normal on that day. The central cluster clearly supports the control forecast (Fig. 10). With a 60-gpm threshold, the averaged field is hardly smoothed and the main synoptic features are still discernible. Four tubes indicate possible deviations from the central cluster option. R Tube 1: The northerly flow might be more cyclonic and stronger over the British Isles. R Tube 2: On the contrary the ridge might develop farther into the continent. R Tube 3: The ridge might evolve into a dipole with a cutoff low over SW Europe. R Tube 4: A large-scale cutoff low might affect the eastern part of western Europe. The two first tubes indicate the main, larger possible deviations. Moreover, the extreme forecast of the last tube is not very different from the control forecast. The

9 749 FIG. 10. Tubing classification applied to the ECMWF EPS based on 22 Jan 1997, 144 h, 500-hPa geopotential height. The central cluster standard deviation is limited to 60 gpm, corresponding to 30% of the total ensemble variance. The radius is 83 gpm. The domain of classification is western Europe. First row: central cluster (30 members), extreme of tube 1 (11 members), extreme of tube 2 (8 members). Second row: extreme of tube 3 (four members), extreme of tube 4 (two members), verification (28 Jan 1997, 1200 UTC). third tube proves to indicate the right deviation from the ensemble mean. Although the verification pattern (Fig. 10, bottom-right panel) does not match the extreme of the tube (bottom-left panel), it can be subjectively localized somewhere between the ensemble mean and the extreme of the tube. Objective verification confirms this judgment in terms of rms error. The verification is found within the boundaries of the third tube, at 50 gpm from the axis and at 92 gpm from the ensemble mean. Figure 11 shows all ensemble forecasts classified in tube 1 (the cyclonic tendency), together with the control forecast and the closest ensemble forecast to the ensemble mean. The members are sorted according to the distance to the ensemble mean. There is a gradual cyclonic tendency from the closest member to the central cluster boundaries, exhibiting a rather anticyclonic northwesterly flow (top-right panel), to the extreme of the tube (bottom-right panel). It is noticeable that some members, for example, the fifth one, which exhibits a northerly anticyclonic flow over the British Isles (bottom-left panel), do not fully support the tube tendency given by the extreme. As in any classification, the loss of information can be compensated for only by an increase of the number of tubes, obtained by limiting the size of the central cluster. This would be at the expense of the conciseness that is of primary importance for operational use. b. Clustering The Ward algorithm has been applied to the same situation, with a similar configuration (cluster standard deviation limited to 60 gpm). The classification results in five clusters, represented in Fig. 12. In order to investigate the similarities between cluster centroids and tube extremes, an interdistance matrix has been computed, in terms of rms difference and ACC (Table 1). The second, most populated, cluster exhibits almost exactly the same pattern as the tubing central cluster (ACC 99%). From a subjective assessment, cluster 1 supports the tube 2 anticyclonic tendency, while cluster 3 supports the tube 1 cyclonic tendency. Although this is roughly confirmed in terms of rms difference and ACC (e.g., cluster 1 is the closest cluster to tube 2), Table 1 shows that the four first clusters are very similar to the tubing central cluster (ACC 90%). Only cluster 5, containing two members, clearly supports the first tube tendency. Tube 3 and tube 4 meteorological tendencies are not properly represented by the Ward classification, although ACC with respect to the three first clusters are above 75%. Cluster centroids look much more similar than tube extremes. This similarity between cluster mean fields, commonly reported by forecasters, is partly due to smoothing out small-scale features when averaging dif-

10 750 WEATHER AND FORECASTING VOLUME 14 FIG. 11. From left to right and from top to bottom: ensemble control forecast, closest forecast to the ensemble mean, forecasts classified in tube 1 from the closest forecast to the central cluster (top right) to the extreme of the tube (bottom right). (22 Jan 1997, 144 h, 500- hpa geopotential height.) ferent fields. Note that this impact would be reduced if clustering only reflected natural modes, averaged fields being more similar. The similarity between cluster mean fields is also a consequence of the Ward algorithm, the main centroids being located around the ensemble mean rather than along the edges of the distribution (see section 4d). Figure 13 shows the ensemble forecasts that are the closest to the centroids of the five clusters shown Fig. 12. The differences between these representatives are sharper than between the centroids, but the extremes of the tubing classification still give more striking information, for example, tube 2 versus cluster 1. c. Probabilities Gridpoint probabilities have been computed from the EPS distribution, without any parametric adjustment or calibration, to illustrate how the tubing classification can be interpreted by operational forecasters. The aim of this interpretation is to complement probabilistic forecasts by identifying the meteorological variants. Significant rain probability (RR 1 mm/24 h) ranges approximately from 10% to 60% over France (Fig. 14). Except over southwest France, where orographic precipitations may be caused by large-scale orographic forcing over the Pyrenees range, the risk of rain over the western half of the country does not exceed 30% 40%. The synoptic pattern indicated by the central cluster leads indeed to dry anticyclonic conditions over this area. The 30% 40% probability is mostly associated with the tendency represented by tube 1. If the ridge does not develop as much as indicated by the central cluster, a secondary wave is likely to pass through at one stage or another, leading to a rainy interval over the considered area. Frost probabilities (2-m temperature below 5 C) indicate a significant risk of 20% 30% over northern Germany (Fig. 15). A deterministic forecast based on the central cluster synoptic pattern would probably exclude it: a wet, cloudy weather is more likely to occur over this area with the northwesterly cyclonic flow. The risk of frost is obviously associated with the tendency pointed out by tubes 2 and 4, indicating a stronger ridge

11 751 FIG. 12. Ward clustering classification applied to the ECMWF EPS based on 22 Jan 1997, 144 h, 500-hPa geopotential height. The cluster standard deviation is limited to 60 gpm (actual values are indicated in the figure labels as std). First row: mean of cluster 1 (16 members), mean of cluster 2 (18 members), mean of cluster 3 (9 members). Second row: mean of cluster 4 (six members) and mean of cluster 5 (two members). The cluster centroids explain 71% of the total ensemble variance. developing farther over the continent and inducing anticyclonic, dry conditions over this part of Europe. The above interpretation is based on the parallel use of objective probabilistic forecasts and the result of the tubing classification. Such an interpretation might be possible from a number of classification methods, provided the result of the classification allows a forecaster to subjectively assess the meteorological spread of the ensemble, that is, the way ensemble forecasts differ from one another in a meteorological sense. 6. Ensemble verification Although the tubing method has been primarily developed to provide forecasters with suitable products to subjectively investigate ensemble distributions, it can also be used to assess the performance of an EPS. Besides, such verification may be useful in an operational environment, allowing a better knowledge of the limitations of the product interpretation. The following results have been obtained by considering together the 120-h, 144-h, and 168-h ECMWF EPS forecasts of 500-hPa geopotential height over southwest Europe (57.5 N, 15 W; 32.5 N, 17.5 E) during winter (10 December February 1997). The tubing configuration is season dependent (radius 80 gpm). The central cluster verifies if the verification lies within its spherical boundaries, in other words, if the ensemble mean rms error does not exceed the central cluster radius. A tube verifies if the verification lies within the cylinder starting from the central cluster boundaries and ending on the extreme of the tube or within the boundaries of the hemisphere with the tube radius aligned at the end of the tube (Fig. 6). Note that several tubes may verify at the same time. This is consistent with the fact that a member may be classified in several tubes. a. Proportion of outliers Figure 16 shows the tubing ability to catch the verification. The verification may be in the central cluster (52% on average), in one or several tube(s) (32%), TABLE 1. Rms difference and ACC between Ward clustering centroids (columns), tubing central cluster centroid (first row), and tube extremes (next rows). (ECMWF EPS, 500-hPa geopotential height, western Europe, 22 Jan 1997, 144 h.) Rms/ACC Cluster 1 Cluster 2 Cluster 3 Cluster 4 Cluster 5 Central cluster Tube 1 Tube 2 Tube 3 Tube 4 24 m/94% 83 m/54% 64 m/69% 54 m/79% 58 m/75% 8 m/99% 70 m/68% 72 m/62% 54 m/79% 56 m/77% 28 m/93% 56 m/81% 80 m/52% 56 m/77% 57 m/76% 36 m/90% 49 m/85% 84 m/51% 70 m/67% 66 m/69% 54 m/79% 33 m/94% 93 m/44% 81 m/57% 74 m/63%

12 752 WEATHER AND FORECASTING VOLUME 14 FIG. 13. Ensemble forecasts closest to the cluster centroids shown in Fig. 12. or missed (16%). The proportion of missed events matches the values usually reported of the so-called proportion of outliers (Buizza et al. 1997). The performance decreases when the spread increases. The frequency of verification in the central cluster drops, due to its size being smaller and smaller compared with the actual spread. In the same time, the frequency of verification in the tubes increases, due to the number of tubes getting larger. Overall, the number of missed events increases with the meteorological uncertainty. This should not be seen as a drawback of the method, rather as the result of an arbitrary limit of the number of tubes for the sake of human interpretation in real time. b. Central cluster verification Figure 17 shows that the frequency of verification in the central cluster exhibits an almost linear relationship with the population. Forecast probability, as determined by the ratio of the cluster population to the ensemble size (51) is overestimated, that is, greater than the observed frequency of verification. This confirms the results of various studies on the verification of probabilistic forecasts based on EPSs (e.g., Talagrand et al. 1997). For operational purposes, it is interesting to note that a 50% frequency of verification is reached for all categories above 30 members, that is, 80% of the time. FIG. 14. Probability of precipitation above 1 mm (24 h) 1 based on the ECMWF EPS. (22 Jan 1997, 144 h.) c. Tubes verification Figure 18 shows the frequency of verification in the tubes. Although the tube populations vary from one to

13 753 FIG. 16. Frequency of verification in the central cluster and the tubes, for five classes of ensemble standard deviation (five first bars) and for the whole sample (last bar). The verification may be in the central cluster (light shading), in the tubes (medium shading), or outside the tubes and the central cluster (dark shading). (Winter 1996/ 97, western Europe, 144 h, tubing radius 80 gpm.) d. Spread skill relationship One of the advantages of the season-dependent configuration is that the number of tubes, as the number of clusters in a similar configuration, is highly correlated with the ensemble spread. The number of tubes should therefore indicate the forecast uncertainty. Although there will never be a perfect spread skill relationship (Buizza and Palmer 1998) a simple confidence index condensing this information proves successful, as illustrated by a three-level confidence index based on the number of tubes (Table 2). A similar index might be based on the ensemble standard deviation, with no need of classification. The advantage of an index based on the number of tubes is to be consistent with the classification interpreted by the forecaster to elaborate the weather forecast. Considering the tubes as representatives of meteorological variants, a forecaster s confidence index is given by the number of those variants that are significant in terms of weather conditions. FIG. 15. Probability of 2-m temperature below 5 C based on the ECMWF EPS. (22 Jan 1997, 120 to 168 h.) nine members, the frequency of verification varies only slightly around 10%. This is consistent with studies showing an underestimate of small forecast probabilities (e.g., Strauss and Lanzinger 1995). It also indicates that tube membership is not a robust property, more to be used for qualitative than quantitative purposes, since less populated tubes are as likely to represent the verification as more populated ones. 7. Usefulness of tubing products in an operational forecasting environment An operational tubing classification was implemented at ECMWF in May The method is applied daily FIG. 17. Frequency of verification in the central cluster for seven classes of central cluster population (main graph) and number of cases in each class (inset). (Winter 1996/97, western Europe, 144 h, tubing radius 80 gpm.)

14 754 WEATHER AND FORECASTING VOLUME 14 TABLE 2. Confidence index based on the number of tubes in a season-dependent configuration (radius 80 gpm). Frequency of cases for each index category and corresponding rms error of the ensemble mean forecast. (ECMWF EPS, 500-hPa geopotential height, SW Europe, winter 1996/97, 120 h/ 144 h/ 168 h.) No. of tubes Ensemble mean rmse (gpm) Confidence index Frequency (%) High Normal Low FIG. 18. Same as Fig. 17 but for the tubes and for nine classes of tube population. to 500-hPa geopotential height EPS forecasts, from 96 hto 240 h, over several European domains. The configuration is season dependent (described in appendix B). The result of the classification is available for ECMWF member state users. Tubing products are effectively used in a number of operational centers in Europe, especially in France. Before tubing was implemented at ECMWF, Météo- France medium-range forecasters had made an attempt to use the clustering products available from ECMWF. The experimental method of interpretation consisted of a subjective classification of ECMWF clusters in order to identify one or several scenarios, that is, large-scale alternative patterns over Europe and the eastern Atlantic (Atger and Mornet 1995). During the experiment, forecasters had a tendency to highlight a main, most populated cluster, often by grouping together the larger ECMWF clusters. Smaller ECMWF clusters were generally kept separated. The forecast skill was found to be maximum when there was only one scenario, most often supported by the T213 ECMWF forecast. The scenario supported by the T213 forecast was the most performant, even when it was not supported by the most populated subjective cluster. It is important to note that this experiment was led while the resolution of the ECMWF EPS model was rather poor (T63). Météo-France forecasters started to use tubing products on an experimental basis in summer 1997 (the higher resolution EPS scheme was implemented in December 1996). Since May 1998 tubing products have been used routinely, as the main guidance for medium-range forecasting. The method of interpretation is the following. R The large-scale pattern indicated by the central cluster is the basis of a deterministic forecast of the main weather characteristics over France for day 4/day 5 and day 6/day 7. R Alternative weather forecasts are inferred from the differences between tube extremes and the central cluster. These alternative forecasts are not sent out to the public, but are used internally to assess the uncertainty of the main weather forecast. R The forecast uncertainty is mentioned in end forecasts by a confidence index (five levels). R Probabilities are used internally to support technical guidance. Dissemination of weather probabilities to the public is currently under consideration. Météo-France forecasters have found a benefit in using tubing products (Guyon 1998). The reasons for this success might be found in the specific characteristics of the method. R Tubing gives a radial representation of an ensemble distribution. This allows a forecaster to conceptualize the future state of the atmosphere as a continuum of possible states around a central, most plausible option. This can be seen as a more effective approach than a corpuscular interpretation based on a limited number of meteorological states. R Tubing highlights the most important differences between ensemble forecasts. This allows a forecaster to immediately feel the consequences of ensemble spread on the uncertainty of the weather forecast. Weather forecasting is essentially based on the knowledge of typical situations to which a forecaster reflexly refers. The more typical the forecast state, the easier the interpretation. Nevertheless, a certain number of pitfalls have been detected when tubing products are used for human interpretation on a daily basis. 1) Interpretation of the smooth central cluster mean field is sometimes problematic. Beyond a certain level of smoothing, synoptic features are not discernible. The synoptic approach is impracticable in this case, but a larger-scale interpretation is still possible, as described in Atger and Mornet (1995) or Guyon (1998). The main characteristics of the weather are inferred from the large-scale flow pattern, for instance, volatile weather in France is associated with an undulating flow over the eastern Atlantic and