The North Atlantic jet stream: a look at preferred positions, paths and transitions

Transcription

1 Quarterly Journal of the Royal Meteorological Society Q. J. R. Meteorol. Soc. 138: , April 212 B The North Atlantic jet stream: a look at preferred positions, paths and transitions A. Hannachi, a * T. Woollings b and K. Fraedrich c a Department of Meteorology, Stockholm University, Sweden b Department of Meteorology, Reading University, UK c Meteorological Institute, Hamburg University, Germany *Correspondence to: A. Hannachi, Department of Meteorology, Stockholm University, Stockholm, Sweden. a.hannachi@misu.su.se Preferred jet stream positions and their link to regional circulation patterns over the winter North Atlantic/European sector are investigated to corroborate findings of multimodal behaviour of the jet positions and to analyse patterns of preferred paths and transition probabilities between jet regimes using ERA-4 data. Besides the multivariate Gaussian mixture model, hierarchical clustering and data image techniques are used for this purpose. The different approaches all yield circulation patterns that correspond to the preferred jet regimes, namely the southern, central and the northern positions associated respectively with the Greenland anticyclone or blocking, and two opposite phases of an East Atlantic-like flow pattern. Growth and decay patterns as well as preferred paths of the system trajectory are studied using the mixture model within the delay space. The analysis shows that the most preferred paths are associated with central to north and north to south jet stream transitions with a typical time-scale of about 5 days, and with life cycles of 1 2 weeks. The transition paths are found to be consistent with transition probabilities. The analysis also shows that wave breaking seems to be the dominant mechanism behind Greenland blocking. Copyright c 211 Royal Meteorological Society Key Words: jet stream; circulation patterns; Greenland blocking; preferred paths; transition probability Received 17 January 211; Revised 26 May 211; Accepted 28 September 211; Published online in Wiley Online Library 1 November 211 Citation: Hannachi A, Woollings T, Fraedrich K The North Atlantic jet stream: a look at preferred positions, paths and transitions. Q. J. R. Meteorol. Soc. 138: DOI:1.12/qj Introduction The North Atlantic and North Pacific are the two main Northern Hemispheric (NH) storm track regions with active synoptic weather systems. In addition, large-scale preferred flow structures that persist longer than typical midlatitude synoptic weather systems have been identified over the North Pacific and the North Atlantic sectors (e.g. Dole and Gordon, 1983). It is known in fact that the midlatitude North Atlantic/Europe and North Pacific are the main regions of significant departure from normality (Christiansen, 29; Hannachi et al., 29). This non-normality is associated with (i) local high skewness over the Aleutians and Greenland, respectively, observed in the low-frequency part of 5 hpa geopotential height (Z5) and sea-level pressure (SLP) (Nakamura and Wallace, 1991; Rennert and Wallace, 29), and (ii) local large negative excess kurtosis, or platykurtosis (sub-gaussianity) over the northern parts of both basins (Sardeshmukh and Sura, 29). These regions, in particular, are locations of frequent blocking events (Pelly and Hoskins, 23; Woollings and Hoskins, 28; Woollings et al., 28). Hannachi (27, H7 hereafter) investigated preferred structures of planetary wave dynamics using the multivariate Gaussian mixture model applied to daily winter 5 hpa height from the National Center for Environmental Prediction and the National Center for Atmospheric Research (NCEP/NCAR) reanalyses. He identified two preferred planetary flow structures, namely a positive Copyright c 211 Royal Meteorological Society

2 The North Atlantic Jet Stream Preferred Positions 863 Pacific North America (PNA) over the North Pacific sector associated with a positive North Atlantic Oscillation (NAO) over the North Atlantic sector (+PNA/+NAO), and a negative phase of the previous pattern, i.e. PNA/ NAO. In a subsequent study, Hannachi (21) extended the analysis to look at both the North Pacific and the North Atlantic sectors separately, and attempted to find the relationship between the sectoral and hemispheric flow patterns or regimes. The sectoral regimes turn out to be very similar to the hemispheric regimes restricted to the associated sectors. It was then suggested that synchronization between the sectorial circulation regimes could be a key factor explaining the occurrence of planetary circulation regimes (Hannachi, 21; Pinto et al., 211). The message from these studies is simple: it points to the importance of understanding sectorial, e.g. North Atlantic and North Pacific, circulation patterns in order to grasp the planetary scale flow dynamics. The NAO is the dominant mode of atmospheric variability in the extratropical Northern Hemisphere winter. Much of the variability of weather and climate in the North Atlantic region can be explained by changes in the NAO. Recently Woollings et al. (21a) suggested, based on SLP and 5 hpa geopotential height (Z5) derived from the 44 winters DJF 1957/58 2/1 of the 4-year European Centre for Medium-Range Weather Forecasts (ECMWF) Re-Analysis (ERA-4) (Uppala et al., 25), that the NAO can be interpreted as comprising two flow regimes: a high-latitude (or Greenland) blocking regime and a zonal (or no blocking) regime. In addition, they found that changes in both the relative occurrence and the structure of the regimes contribute to the long-term NAO trend over the ERA-4 period. Weather and climate variations in the extratropics are associated to a large extent with meridional shifts of the midlatitude westerly jet stream. For instance, major extratropical teleconnections, including the NAO and the PNA pattern, describe changes in the jet stream (Wittman et al., 25; Monahan and Fyfe, 26). Jet stream shifts are associated with a positive feedback between the mean flow and the transient eddies (e.g. Lorenz and Hartmann, 23). Persistence and meridional shifts of the jet stream could therefore hold the key to any regime-like behaviour. The complex behaviour of jet stream variability means that it requires at least two spatial patterns to describe its dominant variations (Fyfe and Lorenz, 25; Monahan and Fyfe, 26), and for the North Atlantic these are the NAO and the East Atlantic (EA) pattern (Woollings et al., 21b). Woollings et al. (21b, WO1b hereafter) considered the winter (DJF) ERA-4 low-level (925 7 hpa) wind to analyse the latitude and speed of the eddy-driven jet stream. Their analysis suggests that there are three preferred latitudinal positions of the North Atlantic jet stream, and this is in very good agreement with regimes obtained from a Gaussian mixture model applied to the twodimensional (NAO, EA) state space. The three jet stream latitudinal positions/flow regimes identify respectively with a southward shifted ( NAO or Greenland blocking), a northward shifted jet ( EA-like pattern) and an undisturbed state of the jet. Given the importance to climate variability and change of these jet positions and associated regional circulation regimes, which express some sort of quasi-stationarity of regional and large-scale variability, it is desirable to analyse the growth and decay patterns of these structures and quantify the probability of transitions between them. This is particularly useful not only for predictability studies but also for impact studies associated with future climate change. The probability mixture model used in Hannachi (27, 21) and Woollings et al. (21a, 21b) provides a general framework to address questions related to these issues, i.e. patterns of growth and decay, transition paths and transition probabilities. For growth/decay and transition paths, the time information is taken into account using extended empirical orthogonal function (EEOF) analysis, which, when combined with the mixture model, provides a convenient and efficient way to describe not only the growth and decay patterns of the circulation regimes but also the preferred patterns of transition paths. As for transition probabilities, the mixture model again provides an elegant framework to compute the transition probabilities without the need to categorically classify individual observations to a given regime as is usually performed in other methods. The objective of this paper is to analyse the North Atlantic jet stream regimes of WO1b in terms of growth/decay, transition probabilities and preferred paths using the Gaussian mixture model. Section2 outlines the data and the methodology. Section3 reviews the jet latitudinal positions and the associated circulation regimes and provides further supporting analyses. Section4 investigates the growth/decay patterns of the circulation regimes along with the transition probabilities between them. A summary and discussion are provided in the last section. 2. Data and methodology 2.1. Data We have used the 5 hpa geopotential height (Z5) data from the ERA-4 reanalysis project (Uppala et al., 25). The gridded data are defined on a regular grid north of 2 N and span the period December February (DJF) 1957/58 2/1, yielding 44 complete winter (DJF) seasons. Daily and monthly data are used. A smooth seasonal cycle is obtained by averaging daily data over all the years then smoothing with a discrete cosine transform, retaining only the mean and the lowest two Fourier frequencies. Daily anomalies are obtained by subtracting the smooth seasonal cycle from the original daily data. In addition, we have used the jet latitude index (WO1b) computed for the period 1 December 1957 to 28 February 22 by averaging daily mean zonal winds over the levels 925, 85, 775 and 7 hpa and the longitudes 6 W. A 1-day low-pass Lanczos filter is then applied to the data and the maximum wind speed value is used to define the jet latitude and speed. A smooth seasonal cycle is then removed from these to give anomaly values (see WO1b for more details). To analyse preferred transition paths we have used gridded data of potential temperature on the 2 potential vorticity unit (PVU) of the potential vorticity (PV) surface θ PV2, which is a proxy for the dynamical tropopause. Also used here are the wave-breaking episodes of Woollings et al. (28), which have been identified from the θ PV2 data using a two-dimensional version of the blocking index of Pelly and Hoskins (23) Methodology In this paper we have applied the mixture model to identify the circulation regimes, analyse the patterns associated

3 864 A. Hannachi et al. with their growth and decay and compute the transition probabilities between them as well as preferred transition paths. In addition, we have also applied a hierarchical clustering method combined with a data visualization technique to corroborate the obtained circulation regimes. Since the mixture model is heavily used in this manuscript for regime diagnosis (section3) and for transition probabilities (section4) we summarize it below, but for more details refer to H7 or Hannachi and O Neill (21) Mixture model In the mixture model framework, the probability density function (PDF) of the atmospheric state in its state space is expressed as a weighted sum of multivariate Gaussian distributions: f (x) = K α k g k (x, k, µ k ), (1) k=1 where α 1,..., α K are the K mixing proportions or weights of the mixture model and they satisfy <α k < 1, for k = 1,...K, and K α k = 1, (2) k=1 and µ k and k are, respectively, the mean and the covariance matrix of the kth, k = 1,...K, multivariate normal density function g k : g k (x, k, µ k ) = (2π) q/2 k 1/2 exp[ 1 2 (x µ k) T 1 k (x µ k)], (3) where q is the state space dimension. The K(q+1)(q+2) 2 2 unknown parameters µ k, k, k = 1,...K, and α k, k = 1,...K 1 are obtained using the expectation maximization (EM) algorithm (Everitt and Hand, 1981; Hannachi and O Neill, 21) Hierarchical clustering There are essentially three categories of clustering methods: probabilistic (e.g. mixture model), hierarchical and nonhierarchical (e.g. k-means); see, for example, Gordon (1999) or Hastie et al. (21). In the hierarchical clustering algorithm, which is our focus here, a collection of fully nested sets are obtained that link the smallest clusters (i.e. individual observations) to the largest cluster (the whole dataset). Starting, for example, from the smallest clusters the algorithm then proceeds by successively agglomerating the existing sets using various linkages, which simply define the distance between clusters. Once an agglomeration merger is chosen, the result of the hierarchical clustering is normally presented in the form of a tree-like graph or dendrogram representing the nested structure of the partition and showing the links between objects. The dendrogram is identified by its root, nodes and branches. Each node corresponds to a numerical value representing the distance between the two clusters during the merger. Hierarchical clustering is not new to climate analysis, and more details can be found, for example, in Cheng and Wallace (1993), Nakaegawa and Kanamitsu (26), and Hannachi et al. (211). To diagnose the number of clusters we use one particularly robust method, the so-called gap statistic, which is summarized below. The gap statistic was proposed by Tibshirani et al. (21), to which the reader is referred to for more details, to estimate the number of clusters in a dataset and can be applied, in principle, to any clustering technique. The reader is also referred to Hannachi et al. (211) for an application to sudden stratospheric warmings. The gap statistic is based on comparing the within-cluster dispersion with that from a probability reference model or null distribution, normally taken to be a homogeneous Poisson point process. For a given number of clusters, k, the sum of the pairwise distances for all points in a given cluster m, D m, m = 1,...k, is first calculated, then the average of these sums is obtained to yield W k = k m=1 D m 2n m, (4) where n m is the size of the mth cluster. A comparison between the within dispersion index log(w k ) of the data and that expected from the null distribution provides the gap statistic: G(k) = E(log(W k )) log(w k), (5) where the first term on the right-hand side of Eq. (5) is the expectation of the within dispersion log(w) (see Eq. (4)), obtained from the null distribution, and E() is the expectation operator. The null hypothesis of a single cluster can then be rejected or accepted depending on the evidence provided by Eq. (5). Here, a Monte Carlo simulation is used to generate N samples of log(wk )based on the null distribution and to compute the corresponding mean log(wk ) and standard deviation s k. Finally, to account for the simulation error in E(log(Wk )), the inflated standard deviation, s k = s k 1 + 1/N, is used. The appropriate optimum number of clusters, K opt, is then given by the smallest k such that G(k) G(k + 1) s k+1. (6) The gap statistic turns out to be a powerful and robust tool to find the appropriate number of clusters. Tibshirani et al. (21) conducted a comparative study, where the number of clusters was known, and showed that the gap statistic outperforms other methods that have been proposed in the literature Data image One efficient way to visualize high-dimensional data is to use what is known as data image (Ling, 1973; Minnotte and West, 1998; Hannachi et al., 211). It consists of mapping the multivariate data into an image framework where each pixel reflects the magnitude of each observation. Basically, a data image of a times series is a colour-coded image of the series. A data image can be generated from the original space time data matrix or from a dissimilarity (or distance) matrix. Several variants of this image can be incorporated. For instance, rows and columns of the dissimilarity (or

4 The North Atlantic Jet Stream Preferred Positions 865 similarity) matrix can be reordered based on some clustering criterion such as hierarchical clustering as in Minnotte and West (1998). This ordering can also be applied to the original data matrix. This representation allows clusters to emerge when the data image is constructed, particularly when it is used in conjunction with the dissimilarity matrix. In this case the clusters appear discernible along the main diagonal of the image. More discussion will be provided in section3.2, where the method will be applied to our data Extended empirical orthogonal functions EEOF analysis looks primarily for oscillations or propagating signals from single or multi-channel time series and can be used to filter the data. It has been used to study the quasi-biennial oscillation (Fraedrich et al., 1993; Wang et al., 1995), the Madden Julian oscillation (Hannachi et al., 27) and other tropical disturbances (Fraedrich et al., 1997). EEOF analysis has been used extensively in tropospheric/stratospheric studies, for instance by Kuroda and Kodera (1998, 1999) and Kodera (1995). EEOFs have also been used to estimate the dimensions of weather/climate attractors (Fraedrich, 1986) and the correlation dimension of attractors from noisy time series (Fraedrich and Wang, 1993). Below is a brief summary of the method; for details the reader is referred to the review of Hannachi et al. (27). For a single-channel time series w t, t = 1, 2,...n, anmdimensional time series w t, t = 1, 2,...n M + 1, is first obtained using delay coordinates as w t = (w t, w t+1,..., w t+m 1 ) T. (7) The parameter M in Eq. (7), which has to be chosen beforehand, is known as the window length or delay parameter. The covariance matrix C of the multi-channel time series is then computed, i.e. C = n M+1 1 w t wt T n M + 1. (8) To find periodic signals one has to look in general for pairs of (nearly) identical eigenvalues of C that are well separated from the rest of the spectrum; see, for example, the extensive discussion in Allen and Smith (1996) and references therein. In the multivariate case an extension of the original state vector x t = ( x t1,...x tq ), is obtained as t=1 x t = ( x t1,...x t+m 1,1,...x t,q,...x t+m 1,q ), (9) and where the procedure is reapplied. 3. Preferred jet locations and circulation structures 3.1. Preferred jet locations Figure 1 shows an example of the time evolution of the jet stream for the last four consecutive ERA-4 winters (DJF 1998/22), as defined in section2 and in detail in WO1b, for the zonal wind averages along with the latitude of the jet maximum (thick solid line). The figure clearly shows the existence of a coherent jet structure in addition to periods of persistent latitudinal shifts. Preferred latitudinal positions can also be seen in Figure 1 and are discussed below. A segment of the jet latitude time series, corresponding to the first ten winters, is shown in Figure 2(a), which can be compared to the index during the last four winters. The persistent character of the jet is clearly seen, particularly for negative latitude anomalies. This persistence can be checked further by examining the evolution of the time series in delay coordinates (Figure 2(b)). A time lag of 1 days is used in Figure 2(b), but the structure remains similar even for lags up to 3 weeks (not shown) because of the persistence of the jet particularly in the S regime, see, for example, Figure 1. The trajectory evolves nearly parallel to the coordinate axes, reflecting the extended persistence of the latitudinal jet variation in addition to the intermittent abrupt transitions between jet latitudes. In order to check whether the jet location is haphazard or clustered around preferred latitudes, a kernel density estimation (Silverman, 1981) and a Gaussian mixture model, with three components, are used to estimate the PDF of the jet latitude. Figure 2(c) shows the histogram and the obtained PDFs of the jet latitude with three clearly well-separated local maxima. The standard smoothing parameter h = 1.6σ n 1/5,withσ and n representing respectively the standard deviation and the sample size of the time series, has been used in the kernel estimate. The local PDF maxima turn out to be very robust to changes in the smoothing parameter even when the maximal smoothing principle (Terrell, 199) was used. These maxima have also been tested using the Silverman (1981) test based on surrogates and they are found to be significant (WO1b). Note, in particular, the perfect match between both the PDF estimates for the leftmost mode, a point that we will come back to later. Sensitivity with respect to longitudinal averaging as well as the use of only zonal winds at one level has also been performed by WO1b and the results show that the local maxima are quite robust. The flow patterns associated with preferred latitudinal positions of the jet are obtained by compositing over the nearest 3 days to the local peaks in the PDF (Figure 2(c)) using the unfiltered Z5 height anomalies (Figure 3, top). The southern position of the jet is associated with a negative phase of the NAO and shows the familiar Greenland blocking or anticyclone. The central and northern positions are associated more with opposite phases of the EA pattern. In Figure 3 we also show similar composites, based on the mixture model, and these are discussed next Circulation structures Mixture model analysis As suggested in previous studies (Fyfe and Lorenz, 25; Monahan and Fyfe, 26; Sparrow et al., 29) the dominant variations of the eddy-driven jet stream can be described by two patterns, although the description is not perfect. For the North Atlantic sector, in particular, the two obvious candidates are the NAO and the EA patterns, and we follow WO1b in using these patterns to describe the circulation regimes associated with the composites of the preferred latitudinal locations. These NAO and EA patterns are obtained as the leading two EOFs of the winter monthly (DJF) Z5 anomalies over the North Atlantic sector (2 9 N, 75 W 15 E). Daily winter Z5 anomalies are then projected onto the monthly NAO and EA EOFs, and the obtained time series are also referred to as NAO and EA in the text. Figure 4 shows a scatter-plot of the

5 866 A. Hannachi et al. Figure 1. Excerpt of Hovmoller diagrams of the daily zonal mean zonal wind averaged over longitudes 6 W and pressure levels hpa for the last four consecutive ERA-4 winters corresponding to DJF 1998/22 ERA-4 data. The latitudes of the preferred jet positions are also shown (straight dashed). Contour interval 5 m s 1, negative contours dashed and the zero contour omitted. data colour-coded to show the latitude (anomaly) of each day. A particularly striking feature of Figure 4 is the good agreement between the colours and the ellipses, which are discussed next. For our analysis in the following sections we have also used the EOFs/principal components (PCs) of the daily winter Z5 anomalies over the North Atlantic sector (2 9 N, 74 E 14 W). The correlations between the leading two PCs with the NAO and EA time series are respectively.97 and.91. Since the mixture model has been applied to these data (WO1b) we start our discussion by looking at the results from this model. The NAO was shown to depart significantly from normality, which was interpreted by Woollings et al. (21a) in terms of a mixture model with two Gaussian components. These components were in good agreement with the PDFs obtained from the wave-breaking index representing respectively Greenland blocking episodes (GBE) and non-gbe. When the EA pattern is included, a better description of the jet stream can be achieved. In this case the mixture model yields three components (WO1b). The centres of these components, interpreted here as circulation regimes, are shown by small filled circles in Figure 4, along with the covariance structure of the corresponding Gaussians. The crosses in Figure 4 represent the projection onto the NAO and EA (monthly) EOFs of the patterns shown in Figure 3 (top) associated with the peaks of the jet latitude PDF. There is clearly good agreement between the two methods in terms of spread (see the colours and ellipses), and means or averages (see the centres) of the regimes. For instance, the centres of the Gaussian components are located very near to the locations of the three preferred jet latitude composites in this reduced state space (Figure 4). The third component is slightly away from the centre associated with the central jet (see Figure 3 top). For completeness, the flow structures representing the Gaussian jet centres are computed through a composite analysis (Figure 3, bottom) and compared to the previous latitudinal jet regimes. The obtained patterns are very similar

6 The North Atlantic Jet Stream Preferred Positions 867 (a) Latitude Anomaly (b) Jet Latitude (t+1) 2 2 Jan 1958 Jan 196 Jan 1962 Jan 1964 Jan Jet Latitude (t) (c) Frequency Latitude Anomaly (Degrees) Figure 2. (a) Segment of the jet stream latitude time series corresponding to the first ten winters: DJF 1957/1967. (b) Trajectory of the winter jet latitude time series in delay coordinates using a 1-day lag. (c) Histogram and PDFs of the jet stream latitude anomaly using a kernel density estimate (solid) and a three-component Gaussian mixture model (dashed). In (b) and (c) the whole time series (DJF ) was used. to the composites of the preferred latitudinal jet positions. A few slight differences can be noted between both composites explaining the mismatch observed in Figure 4. Compared to the respective latitudinal jet regimes, the low of the central mixture regime is slightly stretched northwestward and the high of the northern mixture regime is slightly shifted westward Hierarchical and data image analysis In order to corroborate the results obtained above we now apply the hierarchical clustering and data image methods. Figure 5(a) shows the data image of the NAO and EA time series, where the vertical axis represents time (every 1 days). The figure shows a simple colour-coded image of the data, i.e. for each variable each colour represents one value of that variable and consequently similar colours represent similar values. No obvious structure is seen. The data image of the dissimilarity matrix is shown in Figure 5(b). Here we have a square (interpoint) distance matrix where again each colour represents one value. In Figure 5(b), dark and light colours represent, respectively, small and large interpoint distances. The diagonal line, which represents the zero value, can also be seen. Again, no evidence of any structure is apparent because the temporal order is not in general a good measure to discriminate between clusters. The dendrogram (Figure 5(c)) shows the hierarchy of clusters in the data. Each node in the tree corresponds to the distance between the merging clusters and can be (a) (b) Figure 3. Z5 hpa anomalies of the three maxima of the jet latitude PDF obtained by compositing over the 3 closest days to each of the three modes (top), and the same composites using the mixture model centres (bottom). Contour interval 2 m with the zero contour omitted.

7 868 A. Hannachi et al. EA Cluster Centroids and Jet Latitude Anomalies (Degrees) NAO Figure 4. Scatter-plot of NAO/EA data colour-coded by the latitude anomaly (Figure 2(a)) of the corresponding day. The crosses mark the locations of the preferred jet stream positions (S, left; C, top right; N, bottom right) and the ellipses and their centres represent respectively the covariances and the means of the Gaussian components of the mixture. The remaining centres refer to the centroids based on weighted (circle), Ward (star) and complete (square) linkages. (a) (c) NAO EA (b) NAO Figure 5. Data image of 1-day means of the NAO and EA time series showing a colour-coded value of each datum as a function of time (a), the dissimilarity matrix (b), the dendrogram (c) and the data image based on the tree order (d). The vertical axis in (a) and both axes in (b) refer to time (every 1 days). The numbers on the axes in (a), (b) and (d) are days/1. In (b) the top left (bottom right) corner corresponds to the pair of points from the beginning (end) of the record. projected and read on the x-axis (Figure 5(c)). Note that for the sake of keeping the figure simple we did not show the values of these distances. The branches and leaves of the tree agglomerate the data into clusters by linking individual observations (right-hand side) to the whole set (left-hand side). Clustering very often focuses on the left-hand side of the dendrogram (Figure 5(c)) associated with a small number of clusters. When the data are ordered following the dendrogram (Figure 5(c), right-hand side) the data image (Figure 5(d)) starts to show some coherency reflected by neighbouring observations; compare Figure 5(a) and (d). It is known that clustering and classification belong to hard class problems in pattern recognition. Aart and Korst (199) (d) EA point out that this class of problems can often be solved by human beings but are very hard to solve by computers. The number of clusters, in particular, is the most difficult part in virtually all clustering methods. Clustering algorithms often respond to geometric features even in the absence of inherent clusters. For example, Christiansen (27) points out that mixture models and k-means can produce artificial clusters from flat or skewed distributions. In order to test the data for clustering we use here the copula method based on transforming the marginal distributions of the data to the uniform distribution (Stephenson et al., 24; Hannachi, 27, 21). The data are then mapped onto the unit square, or probability plane, and a clustering index, L(d), is computed based on the mean number of points within a distance d of a target point, and compared to that obtained from no clustering, i.e. uniform distribution. Deviation from L = 1 is then taken as evidence for clustering. Figure 6 shows a scatter-plot of the NAO-EA data within the probability plane (Figure 6(a)) along with the clustering index (Figure 6(b)) and, for comparison, the same index for the first and second halves of the record are also shown. The same index based on the leading two PCs is also shown. The shading in Figure 6(b) represents the domain between the envelopes of clustering indexes obtained from 1 samples simulated from a homogeneous random Poisson point process, with the same sample size as the data. The figure shows clear indication of clustering particularly for the leading two PCs (and for NAO-EA in the range d =.5.2). Particular clustering during the first half, compared to the second half of the record, is evident. This is due to the decrease in Greenland blocking because of the positive phase shift of NAO in the last few decades. In the next step we attempt to estimate the number of clusters using the gap statistic of the data. The calculation is performed within the NAO-EA probability plane. Figure 6 parts (c) and (d) show, respectively, the within-cluster index log(w) (Eq. (4)) of the data and that expected from a uniform distribution (Figure 6(c)) along with the gap statistic (Figure 6(d)). The case of surrogate data and other null hypotheses is discussed towards the end of the section. Note that the maximum of the gap statistic which satisfies Eq. (6) is at three clusters. Equally important are the gaps for one and two clusters, which are well below that for three clusters. In Figure 7 we examine the images of the data and the dissimilarity matrix based on the three-cluster solution and, for comparison, we also show the cases for two and four clusters. To reduce the size of the images we have applied a 1-day non-overlapping average to the data. This also removes serial correlation effects on the sample size of independent data. We have checked the sensitivity to this smoothing and we found that the gross features are not substantially altered. Note that here the data are ordered (or grouped) according to cluster membership using two, three or four clusters and so no axis label is shown. The case of two clusters (Figure 7(a) and (b)) shows two wellseparated horizontal bands for the NAO, but less so for the EA (Figure 7(a)). This is also reflected in the obtained two clusters shown by two dark blocks along the main diagonal of the dissimilarity matrix data image (Figure 7(b)), where the contrast between the background and the lower right cluster (representing NAO or Greenland blocking) is sharp, but is less so for the other (upper left) cluster (or +NAO). The discrimination between +NAO and NAO is well known (e.g. Barnes and Hartmann, 21; Hannachi,

8 The North Atlantic Jet Stream Preferred Positions 869 (a) 1 (b) EA NAO Clustering L Interpoint distanced (c) 9 (d).4 Log(W) 8 7 Gap Statistic Number of Clusters Number of Clusters Figure 6. (a) Scatter-plot of the NAO-EA times series within the probability plane. (b) Clustering indexes of NAO-EA time series for the whole period (continuous), the first (dotted-dashed) and second (dotted) halves of the record, and the leading two Z5 winter PCs (dashed), and the area between the upper and lower envelopes of clustering indexes obtained from 1 simulated homogeneous Poisson point process (shading). (c) Within-cluster index (log(w)) of NAO and EA (circle) and that expected from a uniform distribution (stars). (d) Associated gap statistic with associated confidence intervals. (a) (c) (e) Figure 7. Data image of low-pass filtered NAO and EA (a, c, e) and that of the dissimilarity matrix (b, d, f) when two (a, b), three (c, d), and four (e, f) clusters are used. In each panel clusters are reflected by homogeneous colours. For the dissimilarity matrices the clusters are shown by dark blocs on the main diagonal. 21; Woollings et al., 21a) but no such discrimination is apparent for the EA in this analysis. Note, however, that the upper half of the data image corresponding to NAO (first column in Figure 7(a)) contains a slight indication of two horizontal bands. Also, it can be noted that the clusters and their sizes can be inferred from the dendrogram (Figure 5(c)). (b) (d) (f) The three-cluster case (Figure 7(c) and (d)) shows three clear horizontal bands in very good agreement between the NAO and EA (Figure 7(c)). The contrast between the clusters and the background in the dissimilarity matrix is also sharp (Figure 7(d)). The two small clusters are obtained from splitting the previous cluster shown in the upper left diagonal (Figure 7(b)), whereas the largest cluster remains unchanged. This is also in very good agreement with WO1b, where the NAO regime (leftmost Gaussian in Figure 4) associated with Greenland blocking or the southern jet position remained unchanged when a two-component mixture was fitted instead, whereas the other two Gaussians (in the three-component mixture model) collapsed into one single regime, representing essentially +NAO, in agreement with Figure 7(a). The largest cluster represents, in fact, an approximation to the southern jet position or Greenland blocking ( NAO). To compare Figures 7(d) and 4 the centroids of the corresponding clusters (Figure 7(d)) are shown by stars in Figure 4. The cluster centroids are not very far from the circulation regimes, except the leftmost cluster, which is on the covariance ellipsoid of the NAO regime. The four-cluster case (Figure 7(e) and (f)) looks quite similar to the three-cluster case with the addition of an insignificant fourth cluster, which can also be inferred from the dendrogram (Figure 5(c)). To check the robustness of the optimal number of clusters we have conducted a Monte Carlo resampling test. One hundred subsamples, each with a sample size that is half the total sample size, are randomly and repeatedly drawn from the data and submitted to the same analysis presented above. To keep the serial correlation of the data the resampling procedure is based on randomized 2-month blocks. Figure 8 shows an example of the obtained gap statistic curves from the surrogate data along with the curve for the whole data (Figure 8(a)) and the frequency of the optimal number of

9 87 A. Hannachi et al. (a) Gap Statistic Number of Clusters (b) Frequency (%) Number of Clusters Figure 8. Gap statistic curves form 1 randomized samples from NAO and EA times series (a) and frequency of the optimal number of clusters K opt (b). The subsamples have a sample size equal to half that of the data and are drawn by randomizing 2-month blocs. (a) Number of Clusters Frequency (%) (b) Number of Clusters Figure 9. Frequency of optimal number of clusters obtained from 1 subsamples based on the leading two (a) and three (b) PCs of winter Z5. clusters (see Eq. (6)) K opt (Figure 8(b)). Although the fourcluster case seems to show relatively large values of the gap statistic (Figure 8(a)), the uncertainty is larger compared to the three-cluster case (Figure 8(a)). More precisely, when the reference (or null) distribution is used the frequency of the optimum number of clusters (Figure 8(b)) shows evidence that, besides the fact that the data are clustered (zero frequency at K opt = 1), the case K opt = 3 is the most likely, although the case K opt = 2 is competitive with the optimal number (Figure 8(b)). The gap statistic of the two leading PCs is very similar to that shown in Figure 6(d) with a slightly stronger peak for three clusters. Figure 9(a) is similar to Figure 8(b) but for the leading two daily PCs. In addition, because the second and third daily EOFs are degenerate (not shown), we also repeated Figure 9(a) using the leading three PCs and the result is shown in Figure 9(b). Lastly, we have looked at the results from other (i.e. Ward and weighted) linkages. The results are summarized in Table I, and the associated cluster centroids are plotted in Figure 4. Although these cluster centroids are separated they do themselves cluster around the centres of the mixture model and the jet position composites. We end this section by discussing the geometric features in the data that could be captured by the clustering algorithm used here. To address these issues we have applied the gap statistic method to various surrogate (independent and autocorrelated) data, namely Gaussian, skewed (based on the gamma distribution), kurtotic and flat distributions. The kurtotic data are simulated as for a first-order autoregressive model, except that the noise is flat. Note that in the kurtotic data we have a negative excess kurtosis (with respect to the value 3 from a Gaussian), i.e. platykurtosis, whereas for the skewed data we get a positive excess kurtosis, i.e. leptokurtosis. A hundred samples of two-dimensional time series, with the same sample size as the NAO-EA data, are simulated and used to compute the optimum number of clusters. The results show that the algorithm is successful for the Gaussian and even skewed data. For the flat distribution, Frequency (%) Table I. Coordinates, in the NAO-EA state space, and percentage population of the cluster centroids based on the complete, weighted and Ward s linkages. The ordering is according to the closeness to the jet latitude positions. The same ordering is obtained when the Gaussian mixture components are used instead. Jet stream position Southern Northern Central Gaussian position Left Bottom Upper right right Complete (.7,.6) (.8,.7) (.1,.9) 33% 24% 43% Weighted ( 1.3,.4) (.1,.8) (.7,.8) 23% 43% 34% Ward ( 1.3,.3) (.5,.5) (.2, 1.2) 24% 49% 27% however, the algorithm yields two clusters. For the kurtotic data the algorithm produces (artificial) clusters for excess kurtosis less than about.45. For Gaussian and skewed data flat and Gaussian nulls lead to the same conclusion. For kurtotic data (with excess kurtosis larger than about.45) only uniform null produces no clustering. It is clear that the algorithm seems to detect the platykurtotic feature of the data. We have also tested the effect of the choice of null hypothesis on the conclusion regarding clustering of NAO- EA data. The results show that for a Gaussian null hypothesis the two- and three-class models are the only competing models. Within the probability plane plus a uniform null, however, the three-class model was obtained. Note also that with the leading two PCs we get three clusters with and without copula transformation and with uniform null. Based on the above simulations we recommend, when applying the gap statistic, using a uniform null complemented for robustness, in the two-dimensional case, by the same null in the probability plane. 4. Preferred transition paths and probabilities of jet stream 4.1. Patterns of growth/decay and preferred transition paths Given the close similarity between the circulation regimes derived from the mixture model and those from the jet stream latitudinal positions we now focus on the former to describe the pattern of their growth and decay, in addition to identifying possible transition paths. The approach is simply to apply the same mixture model to the extended data in the delay coordinates (Eq. (9)). Because the regime centroids obtained will consist of a sequence of consecutive maps they will serve, in principle, three purposes: (i) they will reflect the nature of the quasi-stationarity of the circulation regimes, (ii) they will show growth and decay patterns, and, most importantly, (iii) they can provide information on the pattern of preferred transition paths between the jet stream regimes. In a conventional, e.g. EOF state space, the centroids of the mixture model can be interpreted as preferred states or flow regimes (Hannachi, 27; Woollings Of course, these flat data fail the clustering test based on the copula discussed above.

10 The North Atlantic Jet Stream Preferred Positions 871 et al., 21a, WO1b). In the delay space, because each point is in fact a sequence of maps or more precisely a trajectory, a centroid of the mixture model can therefore be interpreted as a preferred path. The resulting paths are necessarily linked to the above preferred regimes in one way or another. For a given window length M the delay data (9) are constructed using either the NAO/EA time series or the leading three North Atlantic Z5 PCs. The discussion of the results, however, will be limited to the latter case. Because the state space dimension is now quite large (2M or 3M) the mixture model is very expensive to apply, in terms of CPU time, and a dimension reduction using EEOF decomposition becomes necessary. In the results to be discussed we used a window length of M = 4 days, but sensitivity to changes in this parameter has also been examined and the main result is found to be quite insensitive. We note that usual analyses with EEOFs tend to use large window lengths. However, our main objective here is to explore and analyse patterns by examining, for example, the structure of their growth and decay. To analyse patterns of growth and decay of the jet stream regimes we apply the mixture model within the extended state space of the leading 6 to 1 extended PCs (EPCs), considered as representative of the synoptic/regional scales of interest. The number of Gaussian components in the model is also varied and the results will be discussed below. Again, as we mentioned above, the regimes identified previously can be regarded as quasi-stationary states of the large-scale dynamics. In addition, the mixture model components, in delay space, can be regarded as preferred regions with slow evolution (because of the high density) in state space. Therefore, each component thus obtained should reflect, in principle, the broad features of one of the regimes identified above. In addition, after being transformed back to the reduced physical space, each component centroid, composed of a sequence of maps, should describe the growth/decay of and transitions between the corresponding jet regimes. Before discussing the results of the mixture model, let us examine the structure of the obtained EEOFs. Figure 1 shows the spectrum of the covariance matrix of the extended data. Two pairs of nearly equal eigenvalues show up in Figure 1, namely (7,8) and (9,1). These two pairs are approximately associated with propagating patterns in the data. The power spectra of the associated EPCs (not shown) show a clear peak around 25 days. The associated EEOFs show a series of growth/decay of, and transitions between, different patterns, which look like the jet regimes. The example of EEOF7 is shown in Figure 11. Note that because EEOFs are sign-invariant, they and their opposite can be interpreted equally. For example, the first two patterns in the top row (Figure 11) could refer to S or C regimes. We also note here that the leading EEOFs are more stationary and show mostly the same pattern over the whole delay window. We now apply the mixture model to the state space spanned by the leading six EEOFs, but the results are similar even when the leading ten EEOFs are used because of the dominance of the leading stationary modes of variability. When three components are used the centres (or extended regimes) show unambiguously the three jet positions in which stationarity is the dominant feature of the patterns Actually a segment of trajectory in state space. Eigenvalue (%) Eigenvalue spectrum Rank Figure 1. Spectrum of the (delay) covariance matrix using the leading three North Atlantic Z5 PCs with a 4-day delay window. The uncertainties are based on the rule-of-thumb of North et al. (1982) with a heuristic sample size of 5. Eigenvalues associated with propagating features are underlined. over the whole window, though regime C shows a bit less stationarity due arguably to the fact that this regime is in essence a basic state (WO1b) rather than a quasi-stationary state like regime S, see, for example, the discussion in Haines and Hannachi (1995). As we mentioned earlier, the regime centroids from the mixture distribution within the delayed space represent preferred paths and provide information on the evolution structure of the flow between its different states. When the model is applied with three components it picks up the three most preferred paths and, as will be shown in the next section, these paths represent self-transitions, which measure the persistence of the regimes. When more than three components are used we obtain, beside the stationary patterns discussed above, patterns reflecting to some extent the transitions between the jet regimes. An example is shown in Figure 12, which represents the centre of the fourth component in a four-component model showing the growing S regime. The life cycle of the jet regimes and their transition timescales are not, of course, well represented in the (extended) regimes discussed above because these regimes, including Figure 12, are dominated by the stationary structure of the leading EEOFs. In fact, the jet regime transitions and life cycles are localized more in that part of the state space spanned by the propagating patterns, i.e. EEOFs/EPCs 7 1. Note that the extended state space will be populated by a large fraction of states associated with persistence and a small fraction associated with transitions. These transition paths can be obtained by examining the projection onto EEOFs 7 1 of the regimes (based on EEOFs 1 1) associated with transitions. Alternatively, they can be identified by applying the mixture model to EEOFs/EPCs 7 1. Apart from small differences, the results are quite comparable. For example, with a two-component model the largest obtained weight for this model (.87) represents, as anticipated, the fraction of states associated with stationarity (persistence) projected near the origin (low amplitudes) in this 4D space. The remaining proportion (.13) of this two-component model represents the states associated with transitions. The regime centre of these latter states is shown in Figure 13 and reveals the most preferred transitions in a 4-day window. The first row of Figure 13 shows growth/decay of regime C. The largest amplitude is observed at around day 4 and by day 1 the low pressure has weakened and moved eastward,

11 872 A. Hannachi et al Figure 11. The pattern of EEOF7 over the delay window. The patterns shown correspond, from left to right and top to bottom, to day 1, 5 then every 5 days. Contour interval arbitrary Figure 12. Fourth centroid of a four-component mixture model within the delayed space using the leading six EEOFs/EPCs. Time separation as in Figure 11. Contour interval 1 m.

12 The North Atlantic Jet Stream Preferred Positions Figure 13. Second centroid of a two-component mixture model within the delayed space spanned by EPCs 7, 8, 9 and 1, showing the life cycle of jet position C (top panel), S (middle panel) and N (bottom panel) within the delay window. Time separation as in Figure 11. Contour interval 5 m. whereas the high-pressure centre started to grow and move northwestward (Figure 13, upper right panel), and by day 12 (not shown) the pattern looks slightly like a north jet position. By day 15 the Greenland blocking started to build up (Figure 13), and weakened at around day 28, and by around day 32 (not shown) the N regime emerged. Note that the N regime after C is very short lived, with a time-scale of about 5 days, which may be considered as a transition time from C to S. A similar transition time is also observed between S and N (Figure 13, last row). The life cycle of regime C (Figure 13, top row) is about 1 days and is of the order of 14 and 8 days for S (Figure 13, middle row), and N (Figure 13, bottom row) respectively. Thus regime N seems to be the least persistent, in agreement with Barnes and Hartmann (21) and Barnes et al. (21). As is shown later, these transition paths are not inconsistent with the transition probabilities computed within the PC space. To clarify the relationship between the transition paths and wave breaking a similar analysis is applied to the wavebreaking index used by Woollings et al. (28). This index is based on the meridional gradient of potential temperature θ φ on the dynamical tropopause, and identifies large-scale quasi-stationary and persistent blocking events determined by a reversal in θ φ (Pelly and Hoskins, 23). Figure 14 shows the mean state, in delay space, associated with the transition path towards the southern jet position S (Figure 12) over a window length of 4 days. The wave-breaking index identifies the reversal point, so that any anticyclone would be located to the north. Note, in particular, the upstream shift of the North European blocking to produce the Greenland blocking (Woollings et al., 28), but the time-scale is artificially long. In Figure 14 we can see how the blocking is building up over Greenland from day 15 of the window length and reaches maturity around day 3. As the wave breaking starts to increase over the western North Atlantic (Figure 14, day 15) the Greenland blocking acts to divert the jet to the south. This is reflected by the strengthening of the low-pressure centre starting at around day 15 in Figure 12. This wave breaking can be better visualized using maps of the potential temperature on the dynamical tropopause. Figure 15 shows an excerpt of the potential temperature evolution for the period 2 January to 28 February This and other similar periods are chosen based on the closest states of geopotential height to the centroid, in delay space, associated with Figure 12. Note the intrusion of subtropical warm, low-pv air poleward in the North Atlantic at around day 1, and by days 2 3 the subtropical air has settled over Greenland, while a low potential temperature polar air mass lies south of Greenland. Note, however, that Figure 15 corresponds to three (different) blocking events, and so different subtropical air masses, separated by about 5 6 days Transition probabilities between jet stream positions The previous section demonstrated that the mixture model can be used to learn the transitions between the preferred jet locations. In this section we show how the mixture model can further be used, in a quantitative but simplified way, to derive the transition probabilities between the regimes without the need to categorically classify each datum. The