Diagnosis of systematic forecast errors dependent on flow pattern

Transcription

1 Q. J. R. Meteorol. Soc. (02), 128, pp Diagnosis of systematic forecast errors dependent on flow pattern By L. FERRANTI, E. KLINKER, A. HOLLINGSWORTH and B. J. HOSKINS European Centre for Medium-Range Weather Forecasts, UK (Received 2 April 01; revised 25 January 02) SUMMARY Singular-value decomposition (SVD) analysis is employed to study flow-dependent forecast errors. The results presented are based on ECMWF operational forecasts and verifying analysis of 500 hpa heights for the most recent seven winter periods. Beyond forecast day 3 the flow-dependent errors are mainly localized over the Atlantic sector and are associated with the North Atlantic Oscillation- (NAO)-like circulation anomalies. The forecasts systematically underestimate the intensity of pressure anomalies centred over Iceland with the effect of reducing the anomalous westerly/easterly flow over the eastern north Atlantic/western Europe. The flowdependent component of the errors explains about 10% of the total forecast-error variance. However, since it is associated with the NAO mode that dominates the variability of the European weather on longer time-scales, it is essential to identify the model limitations in predicting this flow anomaly in order to guide future work on improving forecasts over Europe. The relationship between forecast-error variability and NAO fluctuations is effectively described by an SVD analysis performed on fields of forecast-error magnitude and analyses. Regions with large (small) forecast-error anomalies are located over the maximum of the anomalous westerly (easterly) jet, while on the flanks of the jet the error anomaly is expected to be minimum/maximum. KEYWORDS: Flow-dependent errors North Atlantic Oscillation Singular-value decomposition 1. INTRODUCTION The success of numerical weather forecasting is limited by the stability properties of the atmospheric flow (Lorenz 1963), by errors in the initial state, and by errors in the model formulation. It has been shown that different forecasts of some synoptic developments can be traced back to errors in initial conditions (Arpe et al. 1985; Hollingsworth et al. 1985). For these cases, a data-assimilation environment is the ideal resource to investigate forecast deficiencies which are related to analysis errors. Rabier et al. (1996) used the adjoint method to calculate the sensitivity of short-range forecast errors to initial conditions and showed that part of the medium-range forecast errors may be explained by defects in the analysis. Forecast errors are commonly separated into a systematic component (defined as the time mean of forecast errors for a certain forecast interval) and a non-systematic or random part. In the past, the systematic errors in the European Centre for Medium- Range Weather Forecasts (ECMWF) model have been documented very extensively (Hollingsworth et al. 1980; Arpe and Klinker 1986; Arpe 1988; Tibaldi et al. 1990). In the 1980s the systematic error growth during the early part of the forecast was approximately linear, and its spatial distribution changed with the forecast range. From about forecast day 5 the error growth was reduced and the large-scale features of the error remained almost unchanged as the forecast range increased. (Wallace et al. 1983; Palmer et al. 1990). For extended forecast ranges, the mean error is almost constant and equal to the difference between the model s climate and the observed climate (climate drift). The fact that the spatial distribution of systematic errors in the medium range has large-scale features similar to the climate drift, indicates that those error components are mainly associated with deficiencies in model formulation. Corresponding author: ECMWF, Shinfield Park, Reading, Berkshire RG2 9AX, UK. mol@ecmwf.int c Royal Meteorological Society,

2 1624 L. FERRANTI et al. When the systematic errors are reminiscent of the pattern associated with localized orographic or thermal forcing in simple models, sensitivity experiments are a successful tool to investigate further the sources and consequences of model deficiency. Wallace et al. (1983) were able to point to an erroneous orographic forcing. Ferranti et al. (1994) showed that imposing a quasi-stationary forcing in a key tropical area, which compensated for systematic model errors, had a significant effect on the simulation of the North Pacific blocking frequency. Based on the fact that the errors at day 1 were smaller but exhibited a similar structure to the medium-range forecast errors, Klinker and Sardeshmukh (1992) suggested a novel method to isolate local errors produced by the various parametrized physical processes. Their techniques was built on a balance requirement between dynamical terms and physical tendencies. A budget residual explained a large fraction of the systematic short-range forecast errors. Such a residual technique has shown considerable success in isolating deficiencies in the momentum budget of the ECMWF model. Figure 1 documents the 500 hpa height systematic errors at the forecast range 1, 3, 5, 7 and 10 days of the ECMWF operational forecasts for seven (1992/93 to 1998/99) winter periods (December January February (DJF)). The error amplitudes at day 1 reach values of about 5 10 m and at day 3 are about 15 m. At forecast range day 5 the largest errors are about 25 m, at day 7 about m, and they reach values of m by day 10. It is noteworthy that the amplitudes of systematic errors are reduced by a factor of 8 when compared with the forecast errors of 15-years earlier (Wallace et al. 1983, their Fig. 2). At present the systematic error, defined as the time-mean error and representing the steady drift more or less independent of flow regime, is only a small part of the total error. On a seasonal basis, the day 10 systematic error in the extratropics is only about 5% of the total forecast error at day 10 (Mureau 1990). Consistent with the results of Wallace et al. (1983), Fig. 1 shows that the mean error doubles from forecast day 1 to forecast day 3. In the medium range (day 7 to day 10) the error growth is reduced. The spatial structure evolves during the first half of the forecast range (day 1 day 5) converging towards a more steady pattern with negative biases over western Europe, Mongolia and the Eastern United States, and positive biases over the Pacific and Greenland. The size and the structure of the systematic errors depend on the averaging time period: for short averaging periods, such as 10 days, the errors become more and more flow dependent. Mureau (1990) showed that the typical error averaged over a 10 to -day period is about three times larger than its seasonal mean. In fact, as several authors have discussed, model errors are dependent on flow regimes, and manifest themselves on various time-scales (Palmer 1988; O Lenic and Livezey 1988; Tibaldi and Molteni 1990). Statistical analysis has proven to be an effective tool to analyse and validate the model performance (Molteni and Buizza 1999). When an empirical orthogonal function (EOF) analysis is computed from a large sample of observed anomalies, and subsequently model fields are projected onto them, the comparison of distributions of observed and modelled principal components (PCs) has provided a first estimate of the flow-dependent model error (Ferranti et al. 1994). In this study a singular-value decomposition (SVD) analysis is used to identify flow regimes that have a characteristic forecast error at short and medium range. Section 2 contains a brief description of the SVD analysis and data used; sections 3 and 4 contain the results. A discussion on the relation between the second-moment statistics of the forecast error and the flow-regime circulation is in section 5. Summary and conclusions are in section 6.

3 DIAGNOSIS OF SYSTEMATIC FORECAST ERRORS O W 180 O 150 O E a) b) 150 O W 180 O 150 O E 1 O W 1 O E 1 O W 1 O E 90 O W 80 O N 90 O E 90 O W 80 O N 90 O E 60 O N 60 O N 60 O W 40 O N 60 O E 60 O W 40 O N 60 O E O N O N 30 O W 0 O 30 O E 30 O W 0 O 30 O E 150 O W 180 O 150 O E c) d) 150 O W 180 O 150 O E 1 O W 1 O E 1 O W 1 O E 90 O W 80 O N 90 O E 90 O W 80 O N 90 O E 60 O N 60 O N 60 O W 40 O N 60 O E 60 O W 40 O N 60 O E O N O N 30 O W 0 O 30 O E 30 O W 0 O 30 O E e) 150 O W 180 O 150 O E 1 O W 1 O E 90 O W 80 O N 90 O E 60 O N 60 O W 40 O N 60 O E O N 30 O W 0 O 30 O E Figure hpa geopotential height mean forecast error for seven winter periods (1992/3 to 1998/9): (a) day 1, (b) day 3, (c) day 5, (d) day 7, and (e) day 10 forecasts (contour interval 5 m). The zero contour is omitted, red and yellow indicate positive values, blue and green negative values.

4 1626 L. FERRANTI et al. Percentage of explained variance (%) PCs Figure 2. Variance distribution among the first 30 principal components (PCs) for the analysis (full line), forecast error at day 1 (dash-dot line), and forecast error at day 10 (dashed line). 2. SINGULAR-VALUE DECOMPOSITION ANALYSIS AND DATA The SVD technique isolates the linear combination of data from the first set of fields and from the second set of fields that have the maximum covariance. That is, SVD identifies new variables that maximize the interrelationship between two datasets. This technique works on the assumption that the relation between analysis and forecast errors is mainly linear. However, it is possible that the flow dependence of error variations might have a nonlinear component. In this study, SVD analysis is used to relate forecast-error anomalies and anomalous flow patterns in the verifying analysis. The computation of SVD is described by Ferranti et al. (1997) and is based on PCs (see Barnett and Preisendorfer (1987)). The data used are ECMWF operational forecast errors and verifying analyses of geopotential height at 500 hpa for the seven winters (DJF) 1992 to The daily fields are interpolated on a regular latitude/longitude grid with 3.75 resolution over the northern hemisphere, and consist of forecasts at day 1, 3, 5, 7 up to 10 and the analyses. They are stratified by verifying date, rather than initial date, in similar fashion to the socalled Lorenz files (Lorenz 1982). Since the ECMWF forecast system is in continuous evolution, the statistical analysis is confined to the most recent winters in order to reduce discontinuities associated with model changes in the forecast data. Forecast-error time series (e.g. the PCs) exhibit fluctuations with higher frequencies than the corresponding time series of the verifying analysis. This effect is mainly related to the fact that the forecast time series consist of a sequence of forecast runs from different initial states. In order to reduce the noise, frequencies with periods shorter than 3 days are filtered out in both the analysis and forecast data by using a low-pass filter. We have checked that the spatial structures of the results are not significantly affected by the time filtering. The seasonal cycle is removed from both the analyses and the forecasts by computing the deviation from a second-order polynomial fitted by least squares. Before the SVD is applied to the low-pass data, EOF analyses for either the whole northern hemisphere or for the North Atlantic sector (90 W 60 E, 80 N N) were performed independently on the forecast error and analysis datasets. Figure 2 shows the variance distribution among the first 30 hemispheric PCs for the analysis, forecast error at day 1, and forecast error at day 10. Bearing in mind that the length-scales in the EOF patterns generally decrease as the EOF index increase, the information included in the

5 DIAGNOSIS OF SYSTEMATIC FORECAST ERRORS 1627 TABLE 1. NUMBERS OF EMPIRICAL ORTHOGONAL FUNCTIONS (EOFS) USED TO RECON- STRUCT THE DIFFERENT DATASETS AS INPUT FOR THE SVD ANALYSIS COMPUTED OVER THE NORTHERN HEMISPHERE AND REGIONAL EOFS OVER THE ATLANTIC SECTOR (90 W 60 E, 80 N N) Forecast error Verifying Area At day 1 At day 3 At day 5 At day 7 At day 10 analysis Northern hemisphere Atlantic first few EOF patterns retains primarily large-scale features. The variance spectrum of forecast error at day 1 is whiter than the variance spectrum of forecast error at day 10. The variance explained by the first 30 EOFs of the forecast error at day 1 is 60%, comparing with the 85% explained by the first 30 EOFs of the forecast error at day 10. It follows that Fig. 2 gives a measure of the differences in spatial scales between the short- and medium-range forecast errors. Both variance spectra of forecast error are whiter than the analysis spectrum. Projections of analysis and forecast error datasets onto a subset of EOFs are the input for the SVD computation. Prefiltering by retaining the leading EOFs decreases the number of degrees of freedom in each field and renders the resulting modes more stable with respect to sampling variability. In order to apply an equal level of filtering, each field set is represented by the number of EOFs needed to explain about 70% of its total variance. Table 1 contains the number of hemispheric EOFs used by the SVD analysis over the northern hemisphere and the number of regional EOFs used by the SVD analysis over the Atlantic sector. The SVD is computed independently between: forecast errors at day 1 and the verifying analysis; forecast errors at day 3 and the verifying analysis; and so on for the other forecast ranges. 3. RELATIONSHIP BETWEEN THE FORECAST ERRORS AND FLOW REGIME (a) Results from an SVD analysis over the northern hemisphere Figure 3 shows the leading SVD pairs of modes between analysis and forecast error at day 1 (Figs. 3(a) and (b)), day 3 (Figs. 3(c) and (d)); and day 7 (Figs. 3(e) and (f)). The flow-anomaly patterns of the verifying analysis in Figs. 3(a), 3(c) and 3(e) are related respectively with the forecast-error patterns at day 1 (Fig. 3(b)) at day 3 (Fig. 3(d)) and at day 7 (Fig. 3(f)). Time correlations between each SVD pair of modes, and their respective explained variance, are in Table 2. Associated with the 1-day forecast error, the flow regime selected by the SVD from analysis fields (Fig. 3(a)) has a signature reminiscent of the Pacific North American (PNA) teleconnection pattern, consisting of a wave pattern which extends from the eastern Atlantic to the eastern Pacific. Over the Atlantic sector a north south oriented dipole structure indicates a positive anomaly in the westerly flow. Associated with this flow regime the forecast-error pattern at day 1 (Fig. 3(b)), exhibits positive/negative values of about 4 m over the western part of the United States and Eurasia and negative/positive values over Alaska and over the eastern Atlantic north of 60 N. The flow-dependent error (hereafter FDE) enhances the analysed anomaly over the western coast of North America and makes the flow slightly more meridional over the northern Atlantic. The FDE pattern has small-scale features with relative minima over the Atlantic Ocean and Europe south of 50 N.

6 1628 L. FERRANTI et al. Analysis Day 1 Day 3 Day 7 a) c) e) 1 W 1 E 1 W 1 E 1 W 1 E 90 W 90 E 90 W 90 E 90 W 90 E - 60 W 60 E 60 W 60 E 60 W 60 E Forecast error b) d) f) W 1 E 1 W 1 E 1 W 1 E W 90 E 90 W 90 E 90 W 90 E W 60 E 60 W 60 E 60 W 60 E Figure 3. Leading pairs of modes from a singular-value decomposition analysis applied to 500 hpa height anomalies of the verifying analysis and 500 hpa height forecast-error anomalies. Patterns of analysis ((a), (c) and (e); contour m) and forecast error at (b) day 1 (contour 1 m), (d) day 3 (contour 6 m) and (f) day 7 (contour m). Colour code as in Fig. 1.

7 DIAGNOSIS OF SYSTEMATIC FORECAST ERRORS 1629 TABLE 2. VALUES ASSOCIATED WITH THE SVD LEADING PAIR OF MODES: ANALYSIS AND FORECAST ERROR AT DAY 1, DAY 3, AND DAY 7 Analysis and forecast error At day 1 At day 3 At day 7 Correlation Sigma (%) 10.4, , , 6.0 First row: time correlation between SVD analysis mode and its corresponding SVD forecast-error mode. Second row: percentage of explained variance associated respectively to the SVD analysis and forecast-error patterns. The FDE pattern for the dominant SVD pair at day 3 (Fig. 3(d)) has values of about 12 m centred over Europe, implying a northward shift of the forecast-flow anomaly. At this forecast range the FDEs have largest values over the Euro-Atlantic region and have a spatial scale comparable with the one of the verifying flow regime (Fig. 3(c)). The analysed flow anomalies (Fig. 3(c)) with a distinct north south dipole structure over the Atlantic have much similarly with the SVD analysis pattern associated with the forecast error at day 1 (Fig. 3(a)). However, the PNA-like anomalies over the Pacific in Fig. 3(a) are substantially reduced. The leading pair for the SVD between the analysis and the forecast error at forecast range day 7 (Figs. 3(e) and 3(f)) show a marked negative correlation between the spatial distribution of error amplitudes and that of the analysis anomalies. At forecast range day 1 and day 3 the FDE peak values are comparable with the mean errors in Fig. 1. However, the maxima of FDE at day 7, about 40 to 60 m, are about twice the mean error maxima at that time (Fig. 1(d)). Over the Atlantic region the SVD analysis patterns associated with forecast errors at day 3 and day 7 (Figs. 3(c) and 3(e)) represent the same anomalous flow characterized by a distinct north south dipole structure with a maximum gradient at about 50 N. This pattern is reminiscent of the well-documented North Atlantic Oscillation (NAO) mode associated with changes in the surface westerlies across the North Atlantic onto Europe (Bjerknes 1964; Van Loon and Roger 1978). To a certain extent also, the flow anomalies related to the forecast error at day 1 (Fig. 3(c)) present the NAO signature. The differences of flow anomalies shown in Figs. 3(a), 3(c) and 3(e) are minor, certainly in the Atlantic, and are mainly due to the fact that the SVD analysis has been performed independently for the individual pairs of forecast error and verifying analysis. It is therefore plausible to consider that Figs. 3(d), 3(f) and to a lesser extent 3(b) represent the forecast-error patterns, at different forecast ranges, all associated with fluctuations of the westerlies across the North Atlantic. The tendency of flow-dependent medium-range forecast errors to grow to larger values in the Atlantic than in the Pacific is a typical feature of northern hemisphere forecast errors. This behaviour emerges very clearly from standard deviation of height errors (not shown) when the forecasts and verifying analyses are spacially filtered to retain only waves longer than total wave number, which is similar to using a definite number of EOFs (as in Table 1). The robustness of the SVD results has been checked by performing the same analysis for different subsets of the data. In addition, following the procedure used by Wallace et al. (1992), an SVD analysis was performed on the full seven winters time-series of 90 days but with the 630 analysis charts of 500 hpa ordered randomly

8 1630 L. FERRANTI et al. TABLE 3. SUMMARY OF RESULTS OF 50 SVD EXPANSIONS, EACH BASED ON THE 630 FIELDS OF 500 hpa ANALYSIS HEIGHT AND 500 hpa FORECAST ERRORS AT DAY 1, DAY 3, AND DAY 7, BUT WITH THE TEMPORAL ORDER OF THE ANALYSIS RANDOMIZED Analysis and forecast error At day 1 At day 3 At day 7 Mean correlation Standard deviation correlation Mean sigma (%) 8.8, , , 4.1 Standard deviation sigma (%) 2.1, , , 0.7 First and second row: mean and standard deviation of correlation values. Third and fourth row: mean and standard deviation of percentage of explained variance associated respectively with the SVD analysis and forecast-error patterns. in the time domain so that most of them were paired with the wrong forecast errors. The results of 50 such Monte Carlo SVD expansions are summarized in Table 3. The correlation values from the scrambled dataset stand out sufficiently from the values of the original data (see Table 2) to state that the first SVD modes are statistically significant. However, the percentage of explained variance associated respectively with the SVD analysis and forecast-error patterns (see Table 2) have values similar to those of the random SVD expansions. In this study we have considered only the leading SVD pair of modes because they present, for any forecast range, a highly significant correlation value. While the spatial patterns of the first SVD modes associated with the analysis are clearly related to the structure of well-documented anomalous circulation regimes, spatial analysis patterns for higher order of SVD modes are more difficult to relate to the known modes of observed variability. (b) Results from an SVD analysis over the North Atlantic sector In the light of the above results, it seems that the North Atlantic sector is the area most affected by flow-dependent errors. The flow circulation associated with large systematic errors is characterized by a north south dipole anomaly with centres near the Icelandic low and Azores high. This Atlantic flow regime, being strongly related to the NAO, represents one of the relevant modes of low-frequency variability for the Euro-Atlantic region. In order to analyse in more detail the Euro-Atlantic sector, the SVD analysis has been repeated using a subset of regional EOFs defined over the Atlantic sector (see Table 1). The regional EOF patterns are computed as covariance between standardized Euro-Atlantic PCs and daily height anomalies. Figure 4 shows the SVD leading pairs of modes for the forecast range day 3 (Figs. 4(a) and (b)), day 5 (Figs. 4(c) and (d)) and day 7 (Figs. 4(e) and (f)). In Table 4 correlations and variances associated with the SVD modes are shown. FDE patterns at forecast ranges 3 and 7 days are remarkably similar to the ones from the hemispheric SVD analysis, indicating that the results are domain independent. Associated with anomalous strong westerlies/easterlies across the West Atlantic the day 3 FDEs with positive/negative amplitudes of about 10 m over Europe between 45 N 60 N tend to reduce the strength of the westerlies/easterlies. At day 5 of the forecast the positive/negative FDEs centred over Great Britain are amplified by a factor of 4 compared with day 3. At day 7, FDE amplitudes are about 100 m and they are in phase with the negative of the anomalous low/high in the analysis SVD pattern.

9 DIAGNOSIS OF SYSTEMATIC FORECAST ERRORS 1631 Analysis Day 3 Day 5 Day 7 a) c) e) 1 W 1 E 1 W 1 E 1 W 1 E - 90 W 90 E 90 W 90 E 90 W 90 E - 60 W 60 E 60 W 60 E 60 W 60 E Forecast error b) d) f) 1 W 1 E 1 W 1 E 1 W 1 E 90 W 90 E 90 W 90 E 90 W 90 E W 60 E 60 W 60 E 60 W 60 E Figure 4. Leading pairs of modes from a regional singular-value decomposition analysis applied to 500 hpa height anomalies of the verifying analysis and 500 hpa height forecast-error anomalies. Patterns of analysis ((a), (c) and (e); contour m) and forecast error at (b) day 3 (contour 6 m), (d) day 5 (contour 10 m) and (f) day 7 (contour m). Colour code as in Fig. 1.

10 1632 L. FERRANTI et al. TABLE 4. TIME CORRELATIONS AND PERCENT- AGES OF EXPLAINED VARIANCE ASSOCIATED WITH SVD LEADING PAIR OF MODES. THE SVD ANALYSIS IS DEFINED OVER THE ATLANTIC SECTOR (90 W 60 E, 80 N N) FOR DAY 3, DAY 5 AND DAY 7. Analysis and forecast error At day 3 At day 5 At day 7 Correlation Sigma (%) 12.1, , , RMS ratio forecast range (days) Figure 5. Ratios of root-mean-squared error (RMSE) values for the northern hemisphere (0 360,80 N 30 N) (filled symbols) and for the Atlantic sector (60 W 60 E, 80 N 30 N) (white symbols) plotted as functions of forecast range. Ratios of RMS values between flow dependent error and total error are represented by circles and ratios of RMS values between mean error and total error by triangles. Therefore forecasts exhibit a consistent weakening of the anomalous low/high over Iceland and systematically underestimate the strength of the westerly/easterly anomalies across western Europe. The significance of the Atlantic SVD modes has been tested by again using the scrambling technique. Similar to the results from the previous section the correlation values obtained from the original data (Table 4) are significantly larger than the mean correlation from the 50 scrambled SVD pairs. Results from an additional SVD analysis based on geopotential height at 1000 hpa (not shown) indicate the barotropic nature of the anomalous flow pattern and of its associated systematic error pattern. In order to relate the FDEs to the total errors and the mean errors, root mean square (RMS) values are computed at forecast range day 3, day 5, and day 7 for the mean error, the total error and the FDE (Figs. 4(b), 4(d) and 4(f)) patterns. In Fig. 5 the ratios between the RMS of FDE and the RMS of total error at different forecast ranges are shown as are the ratios between the RMS of mean errors and the RMS of total errors. Figure 5 shows that the flow-dependent error growth rate is larger than the total-error growth rate. This result suggests that at the extended range (30 days or so) the FDE could have large enough values to imply a poor simulation of the NAO mode. Distribution of occurrences of the identified Atlantic flow regime in the analysis and in the forecast has been studied by using projections of the SVD analysis pattern

11 DIAGNOSIS OF SYSTEMATIC FORECAST ERRORS 1633 P.D.F. (*100) Figure 6. Probability density function (PDF) of analysis (full line) and forecast at day 7 (dashed line) of daily projections onto the Atlantic regime describing anomalous westerlies (positive x-axis) and anomalous easterlies (negative x-axis) across west Europe. (Fig. 4(c)) onto daily values of analysis and forecast at day 7. Since all the SVD analysis patterns (see Figs. 4(a) and 4(e) and Figs. 3(c) and 3(e)) are characterized by the similar spatial structure, the results are not significantly different when projections are computed using any other SVD analysis pattern. Probability density functions (PDFs) (Fig. 6) of forecast at day 7 and analysis projections onto the Atlantic regime show a positive bias in the forecast distribution for anomalous westerlies, with relative small amplitudes, implying an overestimation of minor events of westerly anomalies with respect to the analysis. Figure 6 shows also a reduction of the forecast variance with respect to the analysis variance for events with large westerly or easterly anomalies, implying that the major flow anomalies associated with the Atlantic regime are underestimated by the forecast. From a comparison of the power spectra between the analysis and forecast there is an indication that the forecast generally underestimates the Atlantic regime fluctuations with periods longer than 10 days. The systematic under-forecasting of the NAO-like pattern is partly associated with the underestimation of NAO extremes by the model (as shown by the PDFs) band; it is also associated with a general loss of predictability. It could be argued that the NAO mode, being the dominant mode of variability in the Euro-Atlantic domain, is bound to be picked out in the SVD analysis and therefore that the flow-dependent error is merely connected with loss of predictability. In order to differentiate the effect due to model deficiency from the effect due to loss of predictability, averages of the projections onto the Atlantic regime from the analysis and forecasts have been plotted for four different bins (Fig. 7). The binning is done according to the analysis values and the forecast values for forecast ranges: day 5, day 7 and day 10. If the dominant effect was loss of predictability the binning based on the analysis values and the binning based on the forecast values should not be different. As the forecast range increases, the points move away from the diagonal towards the axes where the predictability is zero (x-axis for the analysis-value points and y-axis for the forecast-value points). The under-forecasting by the model is shown by all the lines representing the analysis values swinging towards the x-axis faster than the lines representing the forecast values swinging towards the y-axis. Consistent with the PDFs results, the model underestimation of variance is not very large. However, it is noticeable so that loss of predictability does not completely dominate.

12 1634 L. FERRANTI et al. 2 1 forecast analysis Figure 7. Values of daily projections onto the Atlantic regime from analysis and forecasts (units:m) averaged between [ 2 1], [ 10][0 1] and [1 2]. Binning according to the analysis values are plotted by filled symbols and binning according to the forecast values by white symbols. Different symbols represents different forecast ranges: day 5 (diamond), day 7 (square) and day 10 (circle). 4. RELATIONSHIP BETWEEN FORECAST ERRORS AND NORTH ATLANTIC OSCILLATION INDEX The synoptic weather pattern over the North Atlantic Ocean, particularly in winter, is often characterized by strong westerly airflow between the Icelandic low and the Azores high with baroclinic waves moving westward along a storm track that extends from North America to western Europe (Bjerknes 1964). The NAO is an important mode of atmospheric variability and is related to the position and strength of the North Atlantic storm tracks (Rogers 1997). The NAO index is traditionally defined as the mean-sealevel pressure difference between the Azores and Iceland. In this study the NAO index has been computed by using the daily analysis of geopotential height at 1000 hpa for seven winters (DJF) 1992 to The Atlantic flow regime that has been identified by the SVD analysis to be associated with FDEs (Fig. 4(c)) in the form of a north south oriented dipolar structure over the Atlantic is reminiscent of the NAO mode. The time correlation between the temporal fluctuations of the Atlantic regime (represented by the SVD time coefficient) and the NAO index is in fact In order to investigate whether the forecast-error structures described by the SVD error patterns are equally related to the NAO index, covariances between the NAO index, normalized by its own standard deviation, and forecast errors at different forecast ranges have been computed. Figure 8 shows the forecast-error pattern associated with the NAO index. The large spatial correlation between those error structures and the SVD error patterns in Fig. 4 give clear evidence that FDEs represented by the SVD patterns are related to the NAO mode. The error amplitudes associated with NAO index ranging from about 8 to 10 m at forecast day 1, from 15 to 25 m at forecast day 5 and from to 50 m at forecast day 7 are comparable with the FDE values. The NAO is strongly related to precipitation over Scandinavia and the southern Mediterranean (Hurrell 1995) and plays a large role in European weather. Errors in

13 DIAGNOSIS OF SYSTEMATIC FORECAST ERRORS O W 180 O 150 O E a) b) 150 O W 180 O 150 O E 1 O W 1 O E 1 O W 1 O E 90 O W 80 O N 90 O E 90 O W 80 O N 90 O E 60 O N 60 O N 60 O W 40 O N 60 O E 60 O W 40 O N 60 O E O N O N 30 O W 0 O 30 O E 30 O W 0 O 30 O E c) 150 O W 180 O 150 O E 1 O W 1 O E 90 O W 80 O N 90 O E 60 O N 60 O W 40 O N 60 O E O N 30 O W 0 O 30 O E Figure 8. Covariance between the North Atlantic Oscillation (NAO) index normalized by its standard deviation and daily 500 hpa geopotential height forecast errors at (a) day 3 (contour 4 m), (b) day 5 (contour 5 m) and (c) day 7 (contour 10 m). Colour code as in Fig. 1. the NAO flow patterns would therefore have a detrimental impact on the medium-range forecasts of those near-surface weather parameters. The FDE associated with the PNA pattern (Wallace and Gutzler 1981) at forecast range day 7 (not shown), has maxima amplitudes of about m over the Pacific Ocean. This error is about five times smaller than the FDE maxima amplitudes associated with the NAO mode. 5. RELATIONSHIP BETWEEN THE MAGNITUDE OF THE FORECAST ERRORS AND FLOW-REGIME CIRCULATION In this section the L1 norm statistics of the forecast error (error magnitude) is briefly analysed by considering the absolute value of the 500 hpa geopotential forecast error. Frequently used measures of forecast accuracy are the mean absolute error (MAE) and the root-mean-squared error (RMSE). The MAE is the mean of the absolute values of the differences between the forecast and verifying analysis. The RMSE is the square root of the average squared differences between the forecast and verifying analysis.

14 1636 L. FERRANTI et al. Since the RMSE is computed by squaring the forecast errors, it is more sensitive to larger error than the MAE. Although based on different norms, MAE and RMSE provide comparable estimates of the forecast error. Forecast errors are characterized by a large variability on different time-scales. Flow-dependent errors are among the several components of such a variability. The relationship between forecast errors and flow type has been studied by several authors (e.g. Branstator 1986; Palmer 1988). To relate forecast errors and the flow pattern an SVD analysis between the verifying analysis flow anomalies and the magnitude of forecast error has been performed. Figure 9 shows the leading SVD pair pattern for the analysis (Fig. 9(a)) and the magnitude of forecast errors at day 7 (Fig. 9(b)). The SVD analysis is computed by using regional EOFs defined over the Euro-Atlantic sector (60 W 60 E, 80 N N). In order to represent about 70% of the variance of each dataset, 7 and 27 EOFs respectively for the analysis and for the magnitudes of the error have been considered. The correlation between the analysis and the magnitude of the error pattern is 0.5. The SVD analysis pattern explained 14% of the total variance. The SVD magnitude of the error mode accounts for 6.6% of the total variance. The SVD analysis pattern (Fig. 9(a)) that has the largest time correlation with the day 7 forecast-error magnitude (Fig. 9(b)) is similar to the SVD analysis patterns (Fig. 4) found for the flow-dependent error, showing a north south dipole structure with a maximum gradient at about 50 N. In situations with such anomalous flow the SVD error magnitude pattern exhibits anomalously large errors in a narrow latitudinal band between 45 N and 55 N over western Europe while south and north of that band the errors are small. The position of maximum and minimum error magnitudes relative to the flow anomaly is of particular interest. The largest height errors are located over the maximum of the westerly wind anomaly. When weaker than normal westerlies are observed in the band between 55 N and 45 N the largest height errors are centred to the north of Iceland and over the Iberian Peninsula. In other words large/small error amplitudes are located in regions of the maximum of the westerly/easterly wind anomaly, while on the flanks of the wind anomaly the error amplitudes have opposite signs. Similar results are found from the SVD leading pair modes computed by using analysis and the squared values of forecast errors at day 5 (not shown). Since the flow-dependent error magnitude pattern (Fig. 9(b)) is an important component of the MAE, it describes the spatial distribution of anomalous forecast errors (measured in terms of MAE) when anomalous westerlies/easterlies across the west Atlantic occur. The SVD analysis applied to the analysis and error magnitude is an effective method to reveal the relationship between MAE variations and the flowanomaly fluctuations. Selecting an area where the relationship is clearly defined we can reproduce the result by using standard MAE values. A covariance between the standard MAE, averaged over 60 W E and 80 N 60 N and normalized by its own standard deviation, and the time series of daily analysis (Fig. 10) gives a clear evidence of that. The covariance describes the Atlantic flow regime identified by the SVD analysis, and it indicates that when anomalous weak westerlies across west Europe are observed in the verifying analysis, increased errors at forecast day 7 between 60 W and E north of 60 N can be expected. Based on the crucial knowledge of the anomalous forecast-error spatial distribution given by the SVD pattern (Fig. 9(b)), the covariance can relate MAE variations in a certain area to the flow anomaly over the northern hemisphere. In fact if the MAE is averaged in an area where the error magnitude does not present coherent values the

15 DIAGNOSIS OF SYSTEMATIC FORECAST ERRORS 1637 a) b) 1 W 1 E 1 W 1 E 90 W 90 E 90 W 90 E 60 W - 60 E 60 W E Figure 9. Leading pairs of modes from a regional singular-value decomposition analysis applied to 500 hpa height anomalies of the verifying analysis and 500 hpa height forecast error magnitudes: (a) analysis pattern (contour m) and (b) forecast-error amplitudes at day 7 (contour 6 m). Colour code as Fig O W 180 O 150 O E 1 O W 1 O E 90 O W 80 O N 90 O E 60 O N 60 O W 40 O N 60 O E O N 30 O W 0 O 30 O E Figure 10. Covariance between the mean absolute error averaged over 60 W E, 80 N 60 N, normalized by its standard deviation, and daily 500 hpa geopotential height analysis (contour 10 m). Colour code as in Fig. 1. covariance is not able to find a strong relation between the MAE fluctuations and the flow anomaly. Is is noteworthy that the covariance between the RMSE, averaged over 60 W E and 80 N 60 N and normalized by its own standard deviation, and the time series of daily analysis (not shown) is very similar to the covariance between the MAE and the analysis (Fig. 10). This implies that the relation between the Atlantic regime (Fig. 9(a)) and the geographical distribution of the forecast errors revealed by the SVD analysis does not depend on the measures (MAE or RMSE) of forecast accuracy. 6. CONCLUSIONS In this study the singular-value decomposition (SVD) analysis has been employed to study the flow dependent errors of the geopotential height at 500 hpa.

16 1638 L. FERRANTI et al. Small synoptic errors Large synoptic errors L H Too weak Too weak Large synoptic errors H Small synoptic errors L Figure 11. Schematic interpretation of the results (see text). At day 1 of the forecast the SVD leading pair of modes associate systematic errors of small spatial scale structures with a PNA-like flow regime over the Pacific and with anomalous westerlies/easterlies across the west Atlantic. Beyond forecast day 3, errors localized over the Atlantic sector and associated with NAO-like circulations are dominant. The forecasts systematically underestimate the depth of the anomaly centred over Iceland with the effect of reducing the westerly/easterly anomalous flow over the eastern North Atlantic/western Europe region. The flow-dependent component of the error explains only a limited percentage of the total error variance. However, since it is associated with the NAO mode that plays such a large role in the variability of the European weather, it is essential to identify the model performance for this flow anomaly in order to guide future work on improving forecasts over Europe. Considering that the RMS values of FDE grow more rapidly than the RMS values of total error, larger effects of the FDE are expected at extended forecast ranges. A study on the characteristics of flow-dependent errors of seasonal simulations and their relations with the systematic errors of seasonal mean flow is planned. As indicated by the PDFs, the NAO variability in the medium-range forecast is reduced with respect to the analysis. Though the reduction is not very large, it is consistent with the results of Pavan et al. (00). This study compared the variability in the Euro-Atlantic region from a set of seasonal, ensemble winter simulations, performed with the ECMWF model, with the analysis statistics. It shows that NAO variability is strongly underestimated by the model to the point that the NAO mode is not dominant as it is in the analysis but is only the third axis of model variability. However, it is possible that some of the forecast failures to capture the flow changes related to the NAO are the consequence of a basic predictability problem. This could be addressed by considering the ensemble forecast. A suggestion for future work would be to investigate whether there is a correlation between flow changes associated with large forecast errors and large uncertainties measured in terms of spread of the ensemble forecast. The relationship between forecast-error variability (measured in terms of MAE) and NAO fluctuations has been found to be well described by an SVD analysis performed on forecast-error magnitude and analysis sets. Regions with large/small error anomalies are located over the jet maximum of the westerlies/easterlies, while in the flanks of the jet the forecast errors are expected to be minimum/maximum. The schematic diagram shown in Fig. 11 proposes an interpretation of the relationship between the flow-dependent errors and the anomalous circulation based on the results from the SVD analysis. Since the results of the SVD analysis in an additional

17 DIAGNOSIS OF SYSTEMATIC FORECAST ERRORS 1639 test are not dramatically different when the dataset of the error magnitude is pre-filtered by retaining only fluctuations with a period shorter than days, we can assume that the errors are mainly synoptic. When strong anomalous westerlies associated with a pressure gradient between Iceland and the Azores (positive NAO phase) are observed, the forecast underestimates the pressure gradient and therefore the strength of the westerlies. In these conditions large synoptic errors mainly related to fast propagating disturbances are located over the maximum of the jet, and relatively small errors are found over the rest of the domain. The fact that larger-than-normal errors are found for a westerly flow in the region of the storm-track exit suggests that baroclinic instability is the main contributor to enhanced error growth. In the case of strong anomalous easterlies (negative NAO phase) the forecast still underestimates the anomalous circulation that is reminiscent of the Atlantic blocking. In this situation large synoptic errors cover most of the domain while small errors are localized over the maximum of the easterly jet. REFERENCES Arpe, K Planetary-scale diabatic forcing errors in the ECMWF model. Proceedings of the ECMWF workshop on diabatic forcing, 30 November to 2 December ECMWF, Reading, UK Arpe, K. and Klinker, E Systematic errors of the ECMWF operational forecasting model in mid-latitudes. Q. J. R. Meteorol. Soc., 112, Arpe, K., Hollingsworth, A., Tracton, M. S., Lorenc, A. C., Uppala, S. and Kallberg, P. Barnett, T. P. and Preisendorfer, R. W The response of numerical weather prediction systems to FGGE level IIb data. II: Forecast verifications and implications for predictability. Q. J. R. Meteorol. Soc., 111, Origins and levels of monthly and seasonal forecast skill for United States surface air temperatures determined by canonical correlation analysis. Mon. Weather Rev., 115, Bjerknes, J Atlantic air sea interaction. Adv. Geophys., 10, 1 82 Branstator, G The variability in skill of 72-hour global-scale NMC forecasts. Mon. Weather Rev., 114, Ferranti, L., Molteni, F. and Palmer, T. N. Ferranti, L., Slingo, J. M., Palmer, T. N. and Hoskins, B. J. Hollingsworth, A., Arpe, K., Tiedtke, M., Capaldo, M. and Savijarvi, H. Hollingsworth, A., Lorenc, A. C., Tracton, M. S., Arpe, K., Cats, G., Uppala, S. and Kallberg, P Impact of localized tropical and extratropical SST anomalies in ensembles of seasonal GCM integrations. Q. J. R. Meteorol. Soc., 1, Relations between interannual and intraseasonal monsoon variability as diagnosed from AMIP integrations. Q. J. R. Meteorol. Soc., 123, The performance of a medium-range forecast model, in winter impact of physical parameterizations. Mon. Weather Rev., 108, The response of numerical weather prediction systems to FGGE level IIb data. I: Analyses. Q. J. R. Meteorol. Soc., 111, 1 66 Hurrell, J. W Decadal trends in the North Atlantic Oscillation: Regional temperature and precipitation. Science, 269, Klinker, E. and Sardeshmukh, P The diagnosis of mechanical dissipation in the atmosphere from large-scale balance requirements. J. Atmos. Sci., 49, Lorenz, E. N Deterministic non periodic flow. J. Atmos. Sci.,, Atmospheric predictability experiments with a large numerical model. Tellus, 34, Molteni, F. and Buizza, R Validation of the ECMWF ensemble prediction system using empirical orthogonal functions. Mon. Weather. Rev., 127, Mureau, R The decrease of the systematic error and the increased predictability of the long waves in the ECMWF model. Tech. Memo. No ECMWF, Reading, UK O Lenic, A. E. and Livezey, R. E Relationship between initial circulation anomalies and forecast errors. Proceedings of the ECMWF workshop on predictability, May ECMWF, Reading, UK Palmer, T. N Medium and extended range predictability and stability of the Pacific North American mode. Q. J. R. Meteorol. Soc., 114,

18 1640 L. FERRANTI et al. Palmer, T. N., Brankovic, C., Molteni, F. and Tibaldi, S. Pavan, V., Molteni, F. and Brankovic, C Extended-range predictions with ECMWF models: Interannual variability in operational model integrations. Q. J. R. Meteorol. Soc., 116, Wintertime variability in the Euro-Atlantic region in observations and in ECMWF seasonal ensemble experiments. Q. J. R. Meteorol. Soc., 126, Sensitivity of forecast errors to initial conditions. Q. J. R. Meteorol. Soc., 122, Rabier, F., Klinker, E., Courtier, P. and Hollingsworth, A. Rogers, J. C North Atlantic storm track variability and its association to the North Atlantic Oscillation and climate variability of northern Europe. J. Climate, 10, Tibaldi, S. and Molteni, F On the operational predictability of blocking. Tellus, 42A, Tibaldi, S., Palmer, T. N., Brankovic, C. and Cubasch, U Extended-range predictions with ECMWF models: Influence of horizontal resolution on systematic error and forecast skill. Q. J. R. Meteorol. Soc., 116, Van Loon, H. and Rogers, J. C The seesaw in winter temperatures between Greenland and northern Europe. Part I: General description. Mon. Weather Rev., 106, Wallace, J. and Gutzler, D. S Teleconnections in the geopotential height field during the northern hemisphere winter. Mon. Weather Rev., 109, Wallace, J., Tibaldi, S. and Simmons, A. J. Wallace, J., Smith, C. and Bretherton, C. S Reduction of systematic forecast errors in the ECMWF model through the introduction of an envelope orography. Q. J. R. Meteorol. Soc., 109, Singular-value decomposition of wintertime sea surface temperature and 500 mb height anomalies. J. Climate, 5,