The Met Office global four-dimensional variational data assimilation scheme

Transcription

1 QUARTERLY JOURNAL OF THE ROYAL METEOROLOGICAL SOCIETY Q. J. R. Meteorol. Soc. 133: (27) Published online in Wiley InterScience ( The Met Office global four-dimensional variational data assimilation scheme F.Rawlins,*S.P.Ballard,K.J.Bovis,A.M.Clayton,D.Li,G.W.Inverarity,A.C.Lorenc andt.j.payne Met Office, Exeter, UK ABSTRACT: The Met Office has developed a 4D-Var data assimilation system, which was implemented in the global forecast system on 5 October 24. This followed a development path based on the previous 3D-Var configuration, with many aspects kept in common. A 4D-Var capability was provided by the introduction of a linear perturbation forecast model based on the Unified Model, the non-hydrostatic grid-point model producing our operational forecasts. There were clear advantages in verification of forecasts compared to the equivalent 3D-Var configuration, with an improvement of 2.6% in a composite skill score verified against observations during pre-operational trials. The largest differences in model evolution occur in storm-track regions in the extratropics. Overall, improvements in verification scores as measured against observations were larger than those measured against analyses, particularly at upper levels. There is an improvement in verification of surface parameters (1 m wind, 1.5 m temperature and relative humidity) against analyses. The strongest impact on fitting observations is seen for satellite radiances with weighting functions peaking in the stratosphere and upper troposphere. The largest changes to assimilation increments occurred in the top model levels, particularly wind increments which became much larger near the model top. Similarities were found in the signal of 4D-Var versus 3D-Var for models at two different resolutions, from which we infer that low-resolution trials remain valid for exploring some aspects of 4D-Var before confirmation in full-scale tests. Crown Copyright 27. Reproduced with the permission of the Controller of HMSO. Published by John Wiley & Sons, Ltd KEY WORDS perturbation forecast; operational NWP Received 19 June 26; Revised 2 November 26; Accepted 26 November Introduction Improvements in operational numerical weather prediction (NWP) systems require the best, affordable use of observations to determine the initial state of the atmosphere. With successive increases in computational power, more accurate methods of data assimilation are now achievable for routine use. In the last few years the most successful such method for global NWP has been four-dimensional variational data assimilation (4D-Var), which between 1997 and 25 was implemented by five centres (Rabier, 25), namely ECMWF (Rabier et al., 2; Mahfouf and Rabier, 2; Klinker et al., 2), France (Janisková et al., 1999; Gauthier and Thépaut, 21), the UK (this paper), Canada (Laroche et al., 25; Gauthier et al., 27) and Japan (Kadowaki, 25). This paper describes the implementation of 4D-Var in the operational global model of the Met Office. In the final section we briefly discuss differences from these other systems. Henceforth, the full software systems for 4D- Var and 3D-Var, as operating at the Met Office, will be labelled 4DVAR and 3DVAR respectively, referring * Correspondence to: F. Rawlins, Met Office, FitzRoy Road, Exeter EX1 3PB, UK. rick.rawlins@metoffice.gov.uk to the systems, not the general technique. On 5 October 24, forecast products from the global model were switched over from 3DVAR to 4DVAR. Data assimilation in the previous global model operational system was achieved with a 3D-Var method (Lorenc et al., 2, henceforth L2), designed from the outset to allow a smooth development path to 4D-Var and in particular to be consistent with a later upgrade of the operational forecast model dynamics scheme. Most aspects of the system not touched directly by the 4D- Var algorithm have been preserved in the transition from 3D-Var and are fully described in L2. The following sections describe the scientific design of the implemented system, concentrating on differences with L2, and include a discussion of elements that differ from those adopted elsewhere. Efficiency of computational aspects is vital for providing timely forecasts and software design is described briefly. The Met Office had operated 3DVAR from 1999, when it improved upon an earlier Analysis Correction nudging method, particularly with the introduction of satellite sounding data from Advanced TIROS Operational Vertical Sounder (ATOVS) instruments, where the direct assimilation of radiances gained significant benefit (English et al., 2). Validation results of 4DVAR trials Crown Copyright 27. Reproduced with the permission of the Controller of HMSO.

2 348 F. RAWLINS ET AL. are given in comparison with 3DVAR controls. These trials provide a measure of which observation types give the largest contributions to improvements. The impact of evaluating trials at a lower horizontal resolution is discussed. Finally, the success of the implementation is demonstrated, with indications of where future improvements in performance are anticipated. The development of 4DVAR provided an opportunity for a study of 4D-Var and 3D-Var within full systems sharing a common environment. An exploration of the improved relative performance of 4D-Var is given in a separate paper (Lorenc and Rawlins, 25). This is the first operational implementation of 4D-Var in a global grid-point model, with novel design allowing a progressive upgrade to include more advanced features. The introduction of 4DVAR, in the initial implementation described here, provided both a significant jump in Met Office forecast skill and a development path for later improvements, including greater use of data. 2. The4D-Varmethod 2.1. Basics 4DVAR is an incremental 4D-Var system (Courtier et al., 1994) with a single outer loop of a nonlinear model providing linearization states and background fields for the assimilation of observations. Increments are obtained with respect to a guess state, chosen to be the background state obtained from the previous 6-hour forecast. The nonlinear model is the operational forecast model at the Met Office, the current version of the Unified Model (UM) (Davieset al., 25), comprising a non-hydrostatic, hybrid height formulation with semi-lagrangian advection and finite differences in a regular latitude longitude grid, labelled as N216L38; i.e points at approximately 6 km horizontal resolution in midlatitudes and 38 vertical levels with a top boundary near 4 km. The derivation of the incremental variational formulation follows that of L2, with a similar method of solution via iteration of an inner loop. The difference here is that the evolution of the model trajectory, and hence error covariance, is modelled directly, replacing the assumption of persistence (Lorenc, 23). Note that, unlike in L2, background field values are extracted at the appropriate observation time in both 3DVAR and 4DVAR using a simple linear interpolation in time, i.e. FGAT (first guess at appropriate time of observation). Notation is based on Ide et al. (1997). H is the generalised interpolation operator that transforms a model state x into an estimate y of the observations y o : y = H(x). (1) Calculations in the inner loop are linearized about the current best guess full-resolution state x g and in our single pass through the outer loop, x g = x b,wherex b is the background model state. In the inner loop we consider lower-resolution increments to this, denoted δw. To perform necessary linearizations, we also need a lowresolution representation of the guess: w g. So our incremental variational problem is to find the perturbation δw that minimizes a penalty with background, observation and weak constraint terms: J(δw) = 1 2 (δw δwb ) T B 1 (δw δw b ) (y yo ) T (E + F) 1 (y y o ) + J c, (2) where B is the background-error covariance, and E and F are the four-dimensional covariances of the observation and representativity errors, respectively. Note that the first term in (2), the background penalty, is expressed in terms of δw and y is regarded as a function of δw, as described next. (Underlines denote four-dimensional variables. This notation makes the 4D-Var equations simpler, and more like those of 3D-Var. In implementation, and in most papers such as Ide et al. (1997), the sequence of states x is represented explicitly using time subscripts, i.e. x i ; this introduces summations into the equations. Omission of the underline is used to refer to the field at the initial time, i.e. x = x.) The weak constraint term J c is included in (2) for completeness, but is disregarded hereafter. A nonlinear simplification operator S, whose tangent linear evaluated at the guess state is denoted by S, transforms to the grid and resolution of δw (to leading order): δw = S(x g + δx) S(x g ) Sδx, δw b = S(x b ) S(x g ). } (3) Solving the former equation for full-resolution increment δx leads to the analysis x a = x g + S I δw a, (4) where δw a minimizes (2) and the incrementing operator S I is the generalized inverse of S. The simplification S is not just a decrease in resolution; multiple moisture and cloud variables are simplified to a single variable. In incremental 4D-Var the calculation of the model prediction of the observed values can be split into stages described by operators: M Forecast model that calculates 4-dimensional state x, composed of the 3-dimensional states x i,alldefined from initial value x through perfect M. L Horizontal and time interpolation of model variables to profiles c x at the observation positions. U Transform between model variables and v, a vector of control variables: δw = Uv = U p U v U h v,where U p is the parameter transform between control and model variables, U v is the vertical transform that uses zonal and seasonal average statistics to produce empirical modes, and U h is the horizontal transform, which filters smaller scales in the horizontal.

3 THE MET OFFICE GLOBAL 4D-VAR SCHEME 349 M Perturbation forecast spanning time-window of observations: δw = M L δw. Horizontal and time interpolation of perturbation profiles at the observation positions: c w = L δw. Column version of the inverse simplification operator used to increment columns: c + x = c x + S I c c w. V Observation operator: y = V(c + x ). S I c The U transforms together provide an implicit representation of the model error covariance, B, and allow us to minimize J v (v) = J(Uv), which is better conditioned and easier to minimize (see L2) than J(δw) in (2). Briefly, B 1 is written in terms of the left inverse T of U (satisfying TU = I, the identity matrix) as B 1 = T T T, leading to the first term in J v (v) being (v v b ) T (v v b )/2, where δw b = Uv b and therefore v b = Tδw b. M is implemented using the UM and L as part of the Observation Processing System (OPS). Both of these steps are only performed in the outer loop; their results are stored and held constant during the variational inner loop. M is implemented using the Perturbation Forecast (PF) model, as a subroutine within the variational analysis. (The tilde reminds us that, because of the simplification, it is not the exact tangent linear of M.) It is applied as a series of time steps M i ; the four-dimensional δw is not stored explicitly. L is implemented as a subroutine in each M i, acting on the observations valid at that time step; no time interpolation is performed. So (1) generalizes to the 4D vector of simulated observations y = V [ L{M(x)}+S I c L Mδw ] (5) and equivalent terms can be defined for the adjoint of each operator, as required for the minimization The Perturbation Forecast model The PF model within the inner loop is run at resolution labelled as N18L38, i.e points, which has half the horizontal resolution of the UM, giving about 12 km resolution in midlatitudes. Unlike applications of 4D-Var elsewhere, the PF model is not tangent-linear to the full model and is based on approximations to the nonlinear equation set that are adequate for small but finite perturbations, with amplitude of order the size of the analysis increments. Lorenc (23) has shown that the role of the PF model is to represent the time dimension in 4D forecast error covariances, which can be approximated by a smooth and simplified integration. In particular, most of the physics from the nonlinear model are omitted, retaining only a simple surface friction term and a boundary-layer scheme in order to retain stability of PF integrations. The philosophy behind the development of the PF model and its future potential applications are described in Lorenc and Payne (27). The adjoint of the PF model is generated using simple rules operating on the PF code; the automatic adjoint technique is straightforward to apply except for the iterative 3D elliptic solver within the dynamics. This adjoint was created by applying an iterative solution to the adjoint of the finite-difference equations, which allowed automatic adjoint methods to simplify coding Minimization of the penalty function Transformations to provide better conditioning for the descent algorithm are achieved through a physically based representation of background errors, using spectral transforms to project onto vertical modes and omitting modes describing non-hydrostatic motion. The statistics of background errors were calculated using the NMC method (Parrish and Derber, 1992) and have not been updated for 4DVAR. Apart from the creation of multiple linearization states, this section is essentially unchanged from L2. The same criterion for meeting convergence criterion was adopted usually requiring 5 6 iterations with little difference between 3DVAR and 4DVAR. In common with many centres, a limited-memory quasi-newton algorithm with Broyden Fletcher Goldfarb Shanno (BFGS) updating (Gilbert and Lemaréchal, 1989; Liu and Nocedal, 1989) is used to minimize J. In its general form, the first-guess step-length is iterated if necessary by cubic interpolation/extrapolation. The cost of such methods is proportional to the number of calls to the simulator, which at v returns J(v) and J(v), and may be reduced (i) by using an improved line search that increases the reduction of J each iteration and avoids the need to have more than one update, and (ii) by cheaper simulations. The observation operator V in (5) is weakly nonlinear due to the presence of ATOVS radiative transfer and Special Sensor Microwave Imager (SSMI) wind speed operators making the observation penalty (second term in (2)) weakly non-quadratic in v. The Met Office exploits the fact that V is only slightly nonlinear to reduce cost in both the above ways with a double inner loop algorithm. Set c + x k = L{M(x)}+S I L MUv c k, with v } =, V k (v) = V(c + x k ) + V(c + x k )S I L MU(v c v k (6) ), where V is the tangent linear of V and v k is the computed minimum of J k 1 for k 1, where J k is just the preconditioned J(Uv) from (2) but with y evaluated using V k (v) instead of (5). As J k is exactly quadratic in v, we can minimize it using an efficient exact line search (cf. Fisher, 1998). Ten iterations of this efficient version of the minimization algorithm are applied to J to yield v 1, and the process is repeated for k = 1, 2,... Within the quadratic loop each iteration is cheaper because fully nonlinear V (which in the case of the ATOVS operator involves the costly radiative transfer equation) is not recalculated. Note however that at the start of each quadratic loop an extra full-cost call to the simulator is required to calculate the coefficients of the linearized observation operator.

4 35 F. RAWLINS ET AL. Table I. Observation types in use in 24. Observation group Details Items assimilated Ground-based profiles TEMP, PILOT, PROFILER Temperature, wind, humidity Satellite-based profiles ATOVS(N-15/16, Aqua), AIRS(Aqua) Direct radiances Aircraft AIREP, ACARS, AMDAR, ASDAR Temperature, wind Atmospheric motion vectors GOES-9/1/12, Meteosat-5/7 Wind Satellite-based surface winds SSMI-13/15, Quikscat Surface wind speed Ground-based surface Land SYNOP, SHIP, BUOY Wind (sea only), pressure It can be shown using the implicit function theorem that, so long as the nonlinearity in V is small enough, J has a unique minimum, which will be found by the above iteration. For its initial operational implementation, the double inner loop is typically iterated 5 6 times with a stopping criterion based on the decrease in J k, which has the advantage over a criterion based on J(v) of being a monotone function of iteration number. When a 3D-Var comparison was made of this method with the standard single-loop method, a slightly stricter stopping criterion wasusedintheformerleadingtoasmallerfinalj and improved skill scores. Even so, it used fewer simulations overall, and each quadratic simulation was 23% cheaper than a full-cost non-quadratic simulation. The cost saving in 4DVAR is relatively less as the cost per simulation is dominated by the call to the PF model Initialization of the analysis increment Gravity wave noise generated by a lack of balance of the uninitialized analysis increment is controlled by a digital filtering technique similar to that originally in operation for 3DVAR. This involves running the PF model adiabatically forwards and backwards for 3 hours to obtain a time series of model states that is digitally filtered with a low-pass Dolph (Lynch 1997) filter (6-hour period, 4-hour stop-band edge period). The PF model used for initialization is the same as described above, except that moisture and physics processes are omitted to allow a valid backward integration. For 4DVAR the only change lay in the shift of the analysis increment to the start of the data window. During preliminary investigations it was found that the degree of balance of uninitialized 3D-Var and 4D-Var increments, as measured by pressure tendencies on the lowest model level, are similar. This contrasts with the findings of Gauthier and Thépaut (21), who reported that 4D-Var led to better-balanced increments The operational system The Met Office global model system comprises an update cycle with a data window of 6 hours, such that observations are ingested from 21 3, 3 9, 9 15, UTC, with a nominal T+ analysis time in the middle of each period, e.g. a T+24 labelled forecast valid at UTC is a 27-hour forecast starting from a 21 UTC state. The data cut-off for each update run is typically 7 hours after T+. Main forecast runs are based on and 12 UTC, with an earlier data cut-off, approximately 2 hours after T+, to provide timely products for dissemination to users. Table I describes the observation types in use during the second half of 24. It should be noted that 4DVAR was not tuned to take account of the better assimilation of asynoptic data. In particular, the thinning of satellite sounding data was based on the full data window, and did not include extra observations that were spatially close but temporally separated. All trials of 4DVAR versus 3DVAR, and the final implementation of 4DVAR, used the same set of observations, allowing a close comparison of the two methods. The sea surface temperature has a separate analysis cycle, not described here. Land surface properties are provided from climatological fields; a surface analysis of soil moisture was introduced at a later date Summary of differences: 4DVAR against 3DVAR Use of PF model to generate model trajectories instead of assumption of persistence; Linearization trajectory provided by 1 linearization state every model timestep from T 3 tot+3 instead of a single state at T+; Background fields for FGAT generated from interpolating model fields at hourly rather than 3-hourly intervals; Analysis increment added at T 3 instead of T+; Six NEC SX6 supercomputer nodes required for assimilation instead of one node. 3. Computational aspects The software system was designed from the outset with 3D-Var as a special case to be implemented first, and 4D-Var development following within the same code. Hence a majority of components are common to 3D-Var and 4D-Var implementations, the main exception being the replacement of an implicit persistence forecast with the PF model. The scientific code is written in Fortran9, with Unix scripting providing the control layer for execution and an interface to C routines for efficient input/output. An important aspect of the design is the

5 THE MET OFFICE GLOBAL 4D-VAR SCHEME 351 Table II. Processing costs of the NWP suite (July 24). OPS VAR UM: T+6 UM: T+72 UM: T+144 Total time 3DVAR No. SX6 nodes Elapsed time DVAR No. SX6 nodes Elapsed time Number of SX6 nodes used and elapsed times (minutes) for the suite to complete component steps and sections of forecasts. separation into distinct modules for observation processing and variational minimization, with communication by files, allowing quality control and observational details to be kept separate from generic algorithms and PF model development. It was found to be vital to retain rigorous tests of self-adjoint and gradient properties in order to minimize errors during the development of adjoint code Computer hardware Initial development was undertaken on a Cray T3E supercomputer with optimization efforts first directed at making code efficient on a scalar architecture. Only limited trials could be achieved with available computing power. The procurement of two 15-node NEC SX6 machines, each node having a nominal peak performance of 64 Gflop, provided sufficient resources for realistic testing and implementation. Optimization work subsequently concentrated on a vector architecture in a multi-node environment Observation pre-processing The Observation Processing System (OPS) extracts observations from a meteorological database, performing quality control and pre-processing of data to be included in 4DVAR (or 3DVAR). All observations are quality controlled against background values and nearby observations, with synoptic-dependent background errors determined empirically. Background fields for FGAT are calculated using simple linear time interpolation of model output at regular intervals. Thinning of observations is performed in both time and space. Satellite sounding data are thinned intelligently to produce no more than one retrieval for every 154 km square box, having first passed a number of standard quality-control checks using a 1D-Var retrieval based on RTTOV-7 radiative transfer calculations. It was important to provide a parallel extraction and processing capability to achieve sufficient speed of operations, with different observation types being distributed over six SX6 nodes such that all finished at similar times, the slowest component being ATOVS, which required three nodes D-Var 4D-Var processing costs within 4DVAR are dominated by the PF and adjoint models. The dynamics formulation for the PF model is the same as the UM and shares much of the software structure. It has therefore adopted the same parallelization and optimization strategy, based on a domain decomposition dividing the globe into regular latitude longitude boxes for parallel execution by each processor. A small halo of data surrounds each domain to provide inter-processor calculations. Observations, and columns of background-equivalent values interpolated from nonlinear model fields to the observation position, are distributed using the same domain decomposition. The observation penalty terms are calculated in parallel for each domain at each minimization iteration. Because of the uneven distribution of observations over the globe, this leads to some inefficiency, but it is small compared to the cost of the PF model. For operational implementation six SX6 nodes with a domain decomposition of 2 21 processors north south : west east provided maximum efficiency Relative costs The average elapsed time taken to run components of a single run of the global model operational suite for 4DVAR and 3DVAR are compared in Table II. Implementation of 4DVAR required an increase in operational computing resource by a factor of three, with the proportion taken by assimilation (OPS + VAR)/(OPS + VAR + UM) rising from 29% at 3DVAR to 74% at 4DVAR, measured over our full operational schedule. 4. Trials of 4DVAR 4.1. Series of 1-month trials A series of trials was undertaken, comparing the latest developments of 4D-Var with the equivalent version of 3D-Var. Since this could be accomplished with the same versions of software, and because there were no changes in observation use between 4DVAR and 3DVAR, the comparisons were direct. Similarly the chosen set of background and observation errors and satellite bias information was kept the same for trial and control. The list of trials is given in Table III. During this later period of 4DVAR development, scientific changes applied to both the assimilation algorithms and to the full model were relatively minor, and key results for 4D-Var were sustained over a range of trials. Each trial and control generated a 6-day forecast starting from 12 UTC,

6 352 F. RAWLINS ET AL. for which the data cut-off was 175 mins, longer than available operationally, but providing a bigger signal of 4D-Var versus 3D-Var impact Parallel suite trial The most thorough testing was achieved through a final pre-operational parallel suite, for which a full range of verification was made available. This was conducted during July October 24, and the parallel suite shadowed changes in the operational suite to provide a fair comparison. This trial and control generated 6-day forecasts from both and 12 UTC, with a data cutoff of 115 mins. EUMETSAT ATOVS Retransmission Service (EARS) data were now available for satellite soundings, which ameliorated the effect of the earlier cut-off. 5. Results 5.1. Verification Objective verification An overall measure of performance is provided by a basket of scores of the main meteorological fields, with a range of forecast times, chosen to reflect use of NWP products. The total index is made up of individual skill scores, measured against persistence, compared with radiosonde and surface observations. An equivalent index versus analyses is calculated where analyses are generated from the same suite as the forecast to be verified. Appendix A provides details of the index. Table III shows the changes in the NWP index for trials covering different time periods using a variety of resolutions on two hardware platforms. A breakdown of the r.m.s. errors of the various index components obtained in the final trial of Table III is shown graphically in Figure 1. There is clearly a significant overall performance improvement, particularly for Southern Hemisphere winter cases. A stronger positive signal is more evident when verifying against observations, with better mean-sea-level pressure being obtained in the winter hemisphere, although improvements are evident in extratropical latitudes in both hemispheres. The biggest improvements lie in the winter storm tracks (Figure 2). There is some indication of a reduction in forecast busts, i.e. significantly poorer individual forecasts within a time series (Figure 3). This would be expected if a Gaussian distribution of error is assumed, such that improvements in the r.m.s. error will have a disproportionate effect on the more extreme events. The sample of cases is insufficient to determine whether there is any extra inherent smoothing of the analysis cycle that makes significant departures less likely Verification of surface parameters versus analyses There is a small but consistent improvement in verification of surface parameters (1 m wind, 1.5 m temperature and relative humidity) against analyses (Table IV). Part of the improvement may be due to 4D-Var T+ fields being generated by a 3-hour forecast following initialization, whereas the 3D-Var T+ fields are calculated on the first timestep in which the initialized increment is applied Subjective verification A systematic examination was carried out for the December 22/June 23/July 24 trial periods examining errors from forecasts compared to analyses, concentrating on significant synoptic differences between 4DVAR and 3DVAR affecting T+72 or later forecasts, using mean sea-level pressure fields. Significant differences were identified for 25 forecasts during the three trials, of which 19 were assessed as 4DVAR better and six worse. These studies confirmed that the biggest differences between 3DVAR and 4DVAR occurred in the extratropical storm-track regions Tropical cyclone verification An objective method (Heming, 1994) of measuring the error in forecasting tropical cyclone events for T+ to T+12 was applied to the parallel suite trial results. Table V shows that trial track forecast errors were lower at all lead times except T+48. The strength of a tropical cyclone, as measured by the 85 mb relative vorticity, is Table III. List of trials and NWP index changes. Trial start date Days Hardware Resolution Change in NWP index versus UM PF Observations Analyses 16 Dec T3E N144 N Dec 22 3 SX6 N216 N Jun 23 3 SX6 N144 N Jun 23 3 SX6 N216 N Sep 23 3 SX6 N216 N Jul SX6 N216 N Verification of components of NWP index (%) as described in Appendix A using observations or analyses as truth. A positive value denotes an improvement in 4DVAR over 3DVAR. Resolution refers to horizontal discretization.

7 THE MET OFFICE GLOBAL 4D-VAR SCHEME versus observations (a) 4DVAR3DVAR RMS Error (%) NH PMSL T+24 NH PMSL T+48 NH PMSL T+72 NH PMSL T+96 NH PMSL T+12 NH H5 T+24 NH H5 T+48 NH H5 T+72 NH W25 T+24 TR W85 T+24 TR W85 T+48 TR W85 T+72 TR W25 T+24 SH PMSL T+24 SH PMSL T+48 SH PMSL T+72 SH PMSL T+96 SH PMSL T+12 SH H5 T+24 SH H5 T+48 SH H5 T+72 SH W25 T+24 versus analyses (b) 4DVAR3DVAR RMS Error (%) NH PMSL T+24 NH PMSL T+48 NH PMSL T+72 NH PMSL T+96 NH PMSL T+12 NH H5 T+24 NH H5 T+48 NH H5 T+72 NH W25 T+24 TR W85 T+24 TR W85 T+48 TR W85 T+72 TR W25 T+24 SH PMSL T+24 SH PMSL T+48 SH PMSL T+72 SH PMSL T+96 SH PMSL T+12 SH H5 T+24 SH H5 T+48 SH H5 T+72 SH W25 T+24 Figure 1. (a) r.m.s. errors verified by observations for individual components of an index of scores (see Appendix A) for the July 24 trial, comparing 4DVAR to 3DVAR (negative scores indicate better performance of 4DVAR). (b) is as (a) but verified by own analyses. Abbreviations are:nh = 9 2 N, TR = 2 N 2 S, SH = 2 9 S, PMSL=mean-sea-level pressure, H5 = geopotential height at 5 mb, and W25, W85 are wind speeds at 25 mb, 85 mb, respectively. larger with 4D-Var, particularly at T+12, countering a model trend to weaken cyclone intensities at longer lead times (J. Heming, personal communication). Overall, however, the skill of forecasting intensities was slightly higher with 3D-Var, possibly as a result of more active features being at a disadvantage in verification by simple r.m.s. scores Use of observations There was no change in thinning or assumed observation and background errors between 3D-Var and 4D-Var but, since OPS quality-control decisions depend partly on the background forecast, differing numbers of observations Table IV. T+24 r.m.s. forecast errors for surface parameters. U V T RH 3DVAR DVAR U, V = zonal and meridional wind (m s 1 )at1m. T, RH = temperature (K) and relative humidity (%) at 1.5 m. Results are for the June 23 (N216) trial, globally averaged. can be rejected. It was found that the number of observations rejected was little changed, by much less than 1%.

8 354 F. RAWLINS ET AL. 9N 6N 3N 3S 6S 9S 3E 6E 9E 12E 15E 18 15W 12W 9W 6W 3W Figure 2. Map of T+24 mean-sea-level pressure r.m.s. forecast error differences 4DVAR minus 3DVAR during the June 23 (N216) trial. The contour interval is 2 Pa; contours are dotted where 4DVAR is better than 3DVAR FC Obs RMS Error DVAR 4DVAR August September Figure 3. Time series of T+24 mean-sea-level pressure r.m.s. forecast errors (Pa) for the Northern Hemisphere (9 2 N), showing a reduction in busts for 4DVAR compared with 3DVAR. Table V. Tropical cyclone verification (28 July to 3 October 24). Track/intensity forecasting T+ T+24 T+48 T+72 T+96 T+12 Number verified D-Var track error (km) D-Var track error (km) D-Var rel. vorticity (x1 6 s 1 ) D-Var rel. vorticity (x1 6 s 1 ) This was an active period for tropical cyclones. The strength of the tropical cyclones is measured here by the relative vorticity at 85 mb Fit of observations The total penalty J at the start of a minimization with zero initial control vector v measures the quality of the background forecast in fitting observations during the subsequent data window. Since observation use and assumed errors are unchanged between 3D-Var and 4D- Var, a reduction in the initial value of J reflects an improvement in quality of the data assimilation system. The penalty at the end of the minimization is dominated by the closeness of fit of observations to the model trajectory on the assimilation grid. Owing to the same

9 THE MET OFFICE GLOBAL 4D-VAR SCHEME (J(4DVAR)/J(3DVAR) 1) * Final iteration mean =.7 % First iteration mean = 2.5 % (a) Days starting from 21 July 24 4 N16 AMSU 1 (RMS(4DVAR)/RMS(3DVAR) 1) * Final iteration mean = 6.7 % First iteration mean = 7.2 % (b) Days starting from 21 July 24 Figure 4. (a) Time series of penalties, J, for initial (full lines) and final (dotted lines) iterations of the minimization of the inner VAR loop during the July 24 trial. Initial and final iterations correspond to the fit to background and analysis values respectively, noting that the final penalty is dominated by the observational component. Each point represents a 6-hour update analysis. (b) is similar to (a), but for r.m.s. differences between NOAA-16 AMSU-1 brightness temperatures (K) and the equivalent model predictions for initial and final fits. This accounts for a large contribution to the reduction in penalty due to 4D-Var. criteria being used to define convergence for 3D-Var and 4D-Var minimizations, differences in the final penalty indicate the relative closeness of fit to observations used in the assimilation. Figure 4(a) shows the percentage difference in initial and final penalties of 4D-Var from a 3D-Var control for August 24. This shows a consistent improvement (i.e. a reduction) in the initial penalty with a smaller improvement in the final penalty. This signal was repeated for the other trials, all showing a systematic improvement in the initial penalty, with a smaller difference in final penalties which could be positive or negative. Given the similarities between 3D-Var and 4D-Var configurations, changes in the final penalty may be considered to arise from two effects: (i) a reduction due to better background fields, and (ii) a change due to the different constraint applied through application of the PF model. The contribution of individual observation types can be seen through initial and final r.m.s. differences of observed and modelled observations, as calculated from

10 356 F. RAWLINS ET AL. Table VI. Percentage difference in r.m.s. observed minus model values (July 24). Observation type o b (%) o a (%) Number o b< o a< (channels or levels) Surface MSLP Surface winds Scatterometer winds SSMI winds Satellite winds Aircraft winds Aircraft T NOAA-15 AMSU NOAA-16 AMSU EOS AMSU AIRS Sonde U Sonde V Sonde T Sonde RH o b is observation minus background and o a is observation minus analysis. A negative o b or o a implies a closer fit of 4DVAR to the background or analysis, respectively. The total number of channels per satellite retrieval is given along with the number that show a closer fit. The radiosonde diagnostics (U = zonal wind; V = meridional wind; T = temperature; RH = relative humidity) here are sampled on 8 model levels (1, 5, 1, 15, 2, 25, 3, 34), though of course all model levels contribute to the minimization. Multi-level observations have been combined for satellite-derived winds and aircraft reports. Table VII. Difference in r.m.s. observed minus model values. AMSU-11 AMSU-1 AMSU-8 AMSU-6 AMSU-5 o b o a o b o a o b o a o b o a o b o a NOAA NOAA EOS As Table VI, but % differences in r.m.s. closeness of fit for individual AMSU channels used from EOS Aqua and NOAA satellites. values on the assimilation grid. Figure 4(b) shows the time series for August 24 of Advanced Microwave Sounding Unit channel 1 (AMSU-1) data from the US National Oceanic and Atmospheric Administration satellite NOAA-16. This demonstrates a large reduction in initial r.m.s. difference, and hence penalty contribution, consistent with Figure 4(a) for the total penalty. Figure 4(b) also shows a reduction in final r.m.s. differences, i.e. that predictions of AMSU-1 brightness temperatures from analyses are fitted more closely to the observations. This was found for all the trial periods, where the largest change from 3D-Var to 4D-Var was seen for satellite retrievals peaking in the upper troposphere. Improvements in the initial fit were also shown for most other observation types, but changes in the final fit were mixed. Table VI gives a breakdown of differences for different observation types for the July 24 trial, which demonstrates that most observations have a better fit to the background, but particularly AMSU radiances, where 2 out of 24 satellite channel combinations have a closer fit. With the exception of AMSU retrievals, our 4D-Var analyses tend to fit observations less closely then their 3D-Var equivalents. Reducing the radiosonde statistics to a single number is misleading because there is significant vertical structure, as shown for example by Figure 5. A minor anomaly can be seen in the statistics from the Earth Observing System (EOS) instrument on the Aqua satellite. Though very similar in orbit to NOAA-16, there appears to be much less impact from 4D-Var than from the NOAA series for the same instrument channels. This is shown in Table VII, where results for different satellites are compared, particularly for channels with the strongest signals. It was also apparent from comparing time series that the 4D-Var signal from EOS was much more variable than from NOAA. This has been attributed to extra noise in the EOS data and to a difference in mapping data to ground locations, such that there is less implicit smoothing.

11 THE MET OFFICE GLOBAL 4D-VAR SCHEME model level 2 model level 2 1 3DVAR 4DVAR 1 (a) potential temperature error (K) (b) difference (%) Figure 5. (a) Vertical profile of r.m.s. fit of radiosonde observations of potential temperature to the background for the July 24 trial. (b) shows the difference between the two plots in (a) (4DVAR minus 3DVAR), with an improved fit for the upper (stratospheric) model levels, and some degradation below about level 19 (height approximately 6 km) model level 2 model level 2 1 3DVAR 4DVAR 1 (a) r.m.s. potential temperature (K) (b) difference (%) Figure 6. (a) Vertical profile of r.m.s. differences in analysis increment of potential temperature, averaged over the July 24 trial, with the two plots nearly overlapping. (b) shows the difference between the two plots in (a) (4DVAR minus 3DVAR) Analysis increments The size of analysis increment passed to the nonlinear model is also an indirect measure of the closeness of fit of the background forecast within the total analysis system. The r.m.s. of each horizontal increment field indicates the relative contribution at different vertical levels. Figure 6 shows the average potential temperature increment for all update cycles during the July 24 trial. This disguises upper-level sonde/satellite bias differences, apparent for and 12 UTC versus 6 and 18 UTC analyses, for which latter times the number of sonde observations is much smaller (not shown here). Changes between 3DVAR and 4DVAR are generally small, with a reduction

12 358 F. RAWLINS ET AL. in size of increment for model levels 2 3 (heights 8 18 km for sea points) and an increase at lower levels, generally consistent with the signal from satellites and radiosondes. A similar pattern is seen for the other increment variables: density, specific humidity and Exner pressure. However, the horizontal wind increments were quite different, showing an r.m.s. increase for 4DVAR at all levels, but being particularly marked in the top few levels of the model, shown for the zonal wind component in Figure 7. It is apparent that the wind analysis has adjusted more than other components and the relatively large model errors for the upper stratosphere allow significant redistribution of increments. This is consistent with a poorer fit of forecasts to analyses for observations above 1 mb, as described earlier. With larger errors and few observations to constrain model evolution, it is expected that the larger differences in analysis increments will occur in the top few model levels. Although r.m.s. differences in increment wind component are relatively large (>1% at the top model level), there was no indication of a systematic feed through to the large-scale flow for subsequent analyses; at the end of each trial period, differences between 4DVAR and 3DVAR wind field analyses remained small N48 versus N216 horizontal resolution Though the full impact of 4D-Var needs to be assessed at operational resolutions, a low-resolution suite with N48L38 resolution, i.e points (both UM and PF models) was constructed for a quick exploration of technical aspects. It was found that similar signals were obtained for the fit to observations for low-resolution comparisons of 4DVAR versus 3DVAR. The meteorological impact was generally of the same sign but weaker in amplitude, viz. the average skill index change was +1.2% at N216L38 and +.8% at N48L38. Figure 8 shows a comparison of the fit of satellite data, contrasting the differences due to 4D-Var for high and low resolutions, showing a close match over the different observations. This indicates (i) that the advantage afforded by 4D-Var is of wide applicability with similar new impact from observations, and (ii) that low-resolution trials remain a useful tool for exploring some aspects of 4D-Var before confirmation in full-scale tests Spin-up following initialization Due to the analysis increment not being fully in dynamical and thermodynamical balance with the nonlinear model at initialization, there is a model adjustment in the first few time steps immediately after addition of the increment. This was found to be very similar between 3D- Var and 4D-Var, apart from the inevitable displacement in time by 3 hours. An example is shown in Figure 9, which shows the time evolution of the global sum of precipitation rate for a single forecast Combined 4D-Var and 3D-Var An alternative strategy was tested in which the four daily operational update runs used 4DVAR, but the main, timecritical, forecast runs were kept at 3DVAR. It was found that this schedule retained about 75% of the improvement in NWP index scores when 4D-Var was used throughout, reflecting the improved assimilation feeding through to the next cycle being the dominant effect. However, sufficient optimizations of run time were achieved such that this option was not necessary to meet scheduling deadlines and the full performance improvement could be realized. 4 3 model level 2 1 3DVAR 4DVAR r.m.s. U (m/s) 4 Figure 7. As Figure 6(a), but for zonal wind speed.

13 THE MET OFFICE GLOBAL 4D-VAR SCHEME (RMS(4DVAR) RMS(3DVAR)) / RMS(3DVAR) (%) 5 5 N15_AMSU2 N15_AMSU18 N15_AMSU1 N15_AMSU9 N15_AMSU8 N15_AMSU7 N15_AMSU6 N15_AMSU5 N15_AMSU4 N16_AMSU2 N16_AMSU19 N16_AMSU18 N16_AMSU11 N16_AMSU1 N16_AMSU8 N16_AMSU7 N48 N216 N16_AMSU6 N16_AMSU5 N16_AMSU4 EOS_AMSU11 EOS_AMSU1 EOS_AMSU8 EOS_AMSU6 EOS_AMSU5 Figure 8. Comparison of fit of AMSU observed radiances to 4D-Var versus 3D-Var background fields at N48 and N216 resolutions for the June 23 trials. The dark and light bars at each channel represent the percentage change in initial fit due to 4D-Var for N48 and N216 UM resolution respectively. The similarity in the signals for the impact of 4D-Var can be seen. 4DVAR Mean Precipitation Rate DVAR 9: 1: 11: 12: 13: 14: 15: 16: 17: 18: 2 July 23 Figure 9. Precipitation rate at each timestep: mean of global field (kg m 2 s 1 ). Spin down is apparent during the initial time steps of a single 4DVAR and 3DVAR forecast Operational implementation; international intercomparison 4DVAR was implemented operationally on 5 October 24. The only change from the set-up for the parallel suite (as used to generate the results given above) was to reduce the data cut-off for the main forecast runs at and 12 UTC by 1 minutes to T+1 hour 45 min. Data cut-off times for update runs were unchanged. Because of the variability of weather types, forecast scores exhibit month-to-month variations typically larger than the improvements discussed here, so measuring the benefit from a change in operational forecast scores would require averaging over long periods before and after it. It is not possible to evaluate 4DVAR alone by this means; in the year following implementation, further important changes were made. Within 4DVAR, latent heat release was included within the PF model, revised background-error covariance statistics were introduced and the external digital filtering initialization was replaced by a weak constraint penalizing gravity wave noise. It was also found to be important that consistency be maintained between approximations made at different stages of the assimilation process, in particular the treatment of conversion to moist variables. There were some changes to the satellite data used, and ATOVS bias correction coefficients were updated. (The change in these was small, indicating that model field biases were not grossly affected despite relatively large reductions in error.) There were also significant improvements to the model parametrizations.

14 36 F. RAWLINS ET AL. 4% RMS errors with mean intra-annual variability removed trend UK ECMWF USA France Germany Japan Canada 3% 2% 1% % 1% 2% 3% Oct-3 Nov-3 Dec-3 Jan-4 Feb-4 Mar-4 Apr-4 May-4 Jun-4 Jul-4 Aug-4 Sep-4 Oct-4 Nov-4 Dec-4 Jan-5 Feb-5 Mar-5 Apr-5 May-5 Jun-5 Jul-5 Aug-5 Sep-5 Figure 1. Percentage difference in r.m.s. errors from seven leading global NWP centres, relative to their mean. The NWP centres shown, and their data assimilation methods, are: ECMWF 4D-Var (Rabier et al., 2; Mahfouf and Rabier, 2; Klinker et al., 2); Météo-France 4D-Var (Janisková et al., 1999; Gauthier et al., 27), the UK Met Office 4D-Var from October 24 (this paper); Environment Canada 4D-Var from March 25 (Laroche et al., 25); Japan Meteorological Agency 4D-Var from February 25 (Kadowaki, 25); USA NCEP 3D-Var (Parrish and Derber, 1992); and Germany DWD 3D Optimal Interpolation. The r.m.s. errors, calculated monthly by each centre against its own analyses, were converted to percentage differences from their average, for each field and length of forecast. The plot shows a weighted average over the selection of fields and times defined in Appendix A. (No forecasts beyond 72 hours were available from France; the averages used what was available.). One way of removing the variability of scores with weather pattern is to compare scores for different systems over the same period. Figure 1 shows the relative performance of seven leading global NWP systems, over the year before and after 4DVAR implementation. Plotted are mean percentage differences in r.m.s. error for each system s forecasts, relative to the average of the seven r.m.s. errors. The values used are those exchanged under the auspices of the World Meteorological Organization s Commission for Basic Systems, with each centre verifying its forecasts against its own analyses. The figure shows a weighted mean over forecast fields and times with weights as in Table A.I. These averaged scores remove most of the variability due to weather patterns; residual changes are often due to NWP system changes. With the caveats that some variability still remains, and that we do not know about all changes to all of the systems, many known changes have a visible signal. The Met Office scores show decreases probably due to 4DVAR in October 24 and the subsequent re-tuning of satellite bias corrections in November 24. In February 25, latent heat release was included within the PF model, and there were technical changes to the humidity assimilation and some observation usage. In June 25, revised background-error covariance statistics were introduced and the external digital filtering initialization was replaced by a weak constraint penalizing gravity wave noise. As is often the case with a major change, the full benefits of 4DVAR are only apparent after later changes like these; there are more changes planned. The Japanese and Canadian scores reflect their implementations of 4D-Var in February 25 and March 25 respectively. The Japanese scores also show the effect of other improvements (Yoshiaki Takeuchi, personal communication): new radiation schemes and improved use of ATOVS data in December 24, better radiative treatment of clouds in July 25 and the increased use of time-distributed ATOVS in August 25. The German improvement in December 23 was due to their use of soundings from ECMWF analyses in their assimilation and hence is partly due to ECMWF s 4D-Var (Werner Wergen, personal communication). 6. Discussion and conclusions The five operational global 4D-Var systems, whose results are shown in Fig. 1, have much in common; they are each based on a 3D-Var system that uses control variable transforms to implicitly represent the effect of background-error covariances. They all use the incremental approach (Courtier et al., 1994) employing a simplified perturbation model and its adjoint in the inner-loop minimization iterations. Lorenc (23) showed that this can be regarded as another control variable transform, in an implicit 4D covariance model. This covariance model (if correct) allows optimal use to be made of observations distributed in space and time, and is therefore key to the success of variational assimilation. Its construction