Observation errors in all-sky data assimilation

Transcription

1 Quarterly Journal of the Royal Meteorological Society Q. J. R. Meteorol. Soc. 137: , October 211 B Observation errors in all-sky data assimilation Alan J. Geer* and Peter Bauer European Centre for Medium-Range Weather Forecasts, Reading, UK. *Correspondence to: A. J. Geer, ECMWF, Shinfield Park, Reading RG2 9AX, UK. alan.geer@ecmwf.int This article examines the first-guess (FG) departures of microwave imager radiances assimilated in all-sky conditions (i.e. clear, cloudy and precipitating). Agreement between FG and observations is good in clear skies, with error standard deviations around 2 K, but in heavy cloud or precipitation errors increase to 2 K. The forecast model is not good at predicting cloud and precipitation with exactly the right intensity or location. This leads to apparently non-gaussian behaviour, both heteroscedasticity, i.e. an increase in error with cloud amount, and boundedness, i.e. the size of errors is close to the geophysical range of the observations, which runs from clear to fully cloudy. However, the dependence of FG departure standard deviations on the mean cloud amount is predictable. Using this dependence to normalise the FG departures gives an error distribution that is close to Gaussian. Thus if errors are treated correctly, all-sky observations can be assimilated successfully under the assumption of Gaussianity on which assimilation systems are based. This symmetric error model can be used to provide a robust threshold quality-control check and to determine the size of observation errors for all-sky assimilation. In practice, however, this observation error is being used to account for the model s difficulty in forecasting cloud, which really comes from errors in the background and in the forecast model. Hence in future it will be necessary to improve the representation of background and model error. Separately, symmetric cloud amount is recommended as a predictor for bias correction schemes, avoiding the sampling problems associated with asymmetric predictors like the FG cloud amount. Copyright c 211 Royal Meteorological Society Key Words: Non-Gaussian behaviour; cloud and rain; microwave imagers Received 26 November 21; Revised 22 March 211; Accepted 25 March 211; Published online in Wiley Online Library 23 May 211 Citation: Geer AJ, Bauer P Observation errors in all-sky data assimilation. Q. J. R. Meteorol. Soc. 137: DOI:1.12/qj Introduction Satellite radiance observations are increasingly being assimilated into operational numerical weather prediction (NWP) systems in cloudy and precipitating areas (e.g. Bauer et al., 26a; Pavelin et al., 28; McNally, 29) and there is much ongoing research (e.g. Auligné et al., 211; Bauer et al., 211). However, the radiances can exhibit substantial non-gaussian behaviour (e.g. Errico et al., 27; Bocquet et al., 21) which, if left uncorrected, could degrade the quality of the analysis. Some aspects to consider are: 1. Boundedness. For channels sensitive to the troposphere, brightness temperatures (TBs) are bounded between hard geophysical limits. At one end is the clear-sky TB; at the other end is that associated with a fully cloudy or rainy scene. This is true whether in the microwave, visible or infrared. For microwave imagers over the ocean, TBs are lowest in clear skies and saturate under conditions of heavy cloud or rain (though scattering in deep convection can complicate this Copyright c 211 Royal Meteorological Society

2 Observation Errors in all-sky Data Assimilation 225 picture). In the infrared, TBs can be no higher than the temperature of the surface and no lower than that of a cloud at the tropopause. In the visible, clear-sky scenes are dark (except over snow or ice) whereas maximum TB is associated with a completely reflective cloud. A similar boundedness problem exists for relative humidity and leads to a non- Gaussian error distribution. However, this can be corrected using a symmetrising transform based on first-guess (FG) and analysed relative humidity (Hólm et al., 22). 2. Heteroscedasticity. In statistics, this describes an error variance that changes as a function of some predictor (e.g. Wilks, 26). Such behaviour can be seen even with clearsky satellite observations (Pires et al., 21) and is a central feature of the Hólm et al. (22) transform. It is also very obvious in the assimilation of all-sky radiances, where errors are much larger in cloudy or precipitating situations than in clear skies. It is typical that moisture, and cloud and precipitation in particular, have larger forecast errors than dynamical variables such as temperature or geopotential. Also, radiative transfer models have larger errors in cloudy and rainy situations than in clear skies. For example, these errors may come from a lack of knowledge of subgridscale cloudiness, or unknown distributions of hydrometeor shapes and sizes. 3. Systematic error. The removal of systematic error is essential even for the assimilation of clear-sky satellite observations. This is achieved using adaptive bias correction schemes where the bias can be modelled as a function of predictors such as layer thickness and skin temperature (e.g. Derber and Wu, 1998; Dee, 24). Traditionally such biases have been thought to come from the simplified radiative transfer modelling used in operational data assimilation systems. In the presence of cloud and rain, systematic errors will be larger, and they may just as likely come from deficiencies in the forecast model as from the observation operator. A four-dimensional variational (4D-Var) assimilation of microwave radiances in all-sky conditions (i.e. clear, cloudy and precipitating) has been operational at the European Centre for Medium-Range Weather Forecasts (ECMWF) since March 29 (Bauer et al., 21; Geer et al., 21). More recently, a new formulation for the observation error has substantially increased the weight of the all-sky observations in the analysis (Geer and Bauer, 21). This formulation deals with the problem of heteroscedasticity by increasing observation error as a function of a cloud amount. The boundedness problem is addressed by using the mean of observed and FG cloud amount as the predictor. However, there is still no correction of systematic errors associated with cloud amount, as these errors are highly situation-dependent and not easy to correct using standard predictor-based schemes. In this article we demonstrate the non-gaussian aspects of all-sky radiances and we attempt to justify and critically examine the new observation error model. We will try to understand the processes that give rise to larger errors under cloudy and rainy conditions. In particular, errors that are currently treated in the assimilation scheme as observation error might actually be coming from the forecast model or from the initial conditions (i.e. the background). 2. Microwave imager assimilation at ECMWF 2.1. Observations Microwave imagers such as the Advanced Microwave Scanning Radiometer for the Earth Observing System (AMSR-E; Kawanishi et al., 23) make use of particular frequencies between 7 GHz and 85 GHz where, in clear skies, the atmosphere is semi-transparent and the radiative transfer is dominated by water vapour absorption. On top of this, cloud and precipitation increase the optical depth of the atmosphere and deep convection can make the atmosphere opaque. At ECMWF, all-sky assimilation is restricted to situations where surface emissivity modelling is considered reliable, i.e. to ice-free oceans, and to frequencies between 19 GHz and 85 GHz. Hence, the all-sky observations are sensitive to ocean surface properties (e.g. surface temperature and wind speed), water vapour, cloud and precipitation. The intention is to use all of this information to improve analyses and forecasts. In this article we will concentrate on AMSR-E observations at 19 GHz and 37 GHz. Vertically and horizontally polarised radiation are measured separately at each frequency to give the channels 19v, 19h, 37v and 37h. Emission from the sea surface is strongly polarised, but atmospheric emission and absorption are not, so the polarised measurements allow a separation of surface and atmospheric effects. This principle is used to form a diagnostic cloud retrieval in section 5. As well as AMSR-E, the latest ECMWF system makes use of observations from the Tropical Rainfall Measuring Mission (TRMM) Microwave Imager (TMI; Kummerow et al., 1998) and Special Sensor Microwave Imager Sounder (SSMIS; Kunkee et al., 28). Previously ECMWF has assimilated observations from Special Sensor Microwave/Imager (SSM/I; Hollinger et al., 199) from a number of satellites, but these sensors and platforms are coming to the end of their lives, so since November 21 the data have been only passively monitored Observation operator The observation operator for all-sky assimilation is RTTOV- SCATT (Bauer et al., 26b), which provides multiplescattering radiative transfer calculations at microwave frequencies as part of the RTTOV-9 package (Eyre, 1991; Saunders, 28) and makes use of Fast Emissivity Modelling v2 (FASTEM-2). Scattering calculations are performed using the delta-eddington approach (Joseph et al., 1976), which gives reasonable results when simulating the radiances measured by microwave radiometers (Smith et al., 22). Scattering properties are pre-calculated using Mie theory and tabulated as a function of frequency, temperature, and hydrometeor type and density. The most important inputs to RTTOV-SCATT are the surface skin temperature and winds, and vertical profiles of pressure, temperature, moisture, cloud liquid water and ice, rain and frozen precipitation fluxes, and cloud cover. We use the revised cloud overlap approach of Geer et al. (29), which results in substantially Radiative Transfer model for Television Infrared Observation Satellite Operational Vertical sounder, version 9.

3 226 A. J. Geer and P. Bauer more accurate cloudy and rainy radiative transfer than with previous versions of RTTOV-SCATT Data assimilation ECMWF produces global forecasts and analyses using a 4D- Var data assimilation system (Rabier et al., 2). Microwave imager radiances in all-sky conditions are assimilated directly in 4D-Var alongside many other conventional and satellite observation types. The direct assimilation means that, in order to fit cloud and precipitation observed in microwave imager radiances, 4D-Var must adjust not only moisture and dynamical variables, but also cloud and precipitation. The former are part of the control vector; the latter are generated by the forecast model in 4D-Var. The initial all-sky implementation in March 29 applied relatively large observation errors (Bauer et al., 21; Geer et al., 21). A new package of changes, including the observation error formulation described in this paper, has substantially increased the weight of the all-sky observations in the analysis (Geer and Bauer, 21). However, the increase in weight came primarily from abandoning the old approach, which inflated observation error excessively as a function of distance between grid point and observation. Use of microwave imager observations is restricted to situations where the forecast model and the observation operator can provide reasonable accuracy. As already mentioned, observations are only assimilated over ice-free oceans; they are further restricted to latitudes equatorward of 6. Some high-latitude areas are affected by a longstanding cold-sector bias that appears to come from a lack of water cloud in the model (e.g. Geer et al., 29); observations in these areas are screened out using tests based on modelled cloud and total-column water vapour (TCWV) amount. The bias could perhaps be explained by known problems in modelling mixed-phase Arctic stratocumulus clouds (Klein et al., 29). There is also an apparent excess scattering by snow in deep convection, resulting in modelled TBs in the 37 GHz and 85 GHz channels that are too low. It is not yet clear whether this comes from excessive snow in the model or the inadequacies of the Mie soft sphere approximation used to represent snow in the radiative transfer model (e.g. Petty and Huang, 21). The higher frequency channels (>3 GHz) are screened out in these excess-scattering regions. The observation and modelling groups at ECMWF are working together to resolve these problems. Further quality control checks are applied, including a threshold check on the size of the departures, and Variational Quality Control (VarQC; Ingleby and Lorenc, 1993; Andersson and Järvinen, 1998) which adaptively reduces the weight of any observations that are significant outliers from the main analysis. Observations are averaged (or superobbed ) in boxes of approximately 8 km 8 km, in order to make the horizontal scales of observed cloud and precipitation more similar to their effective resolution in the model, which is several times the outer-loop grid resolution of 16 km. It is also useful to homogenise the resolution of the different instruments, and hence the response to precipitation, which is scale-dependent. Superobbing results in roughly nine SSMIS, 38 TMI or 5 AMSR-E observations being averaged together before assimilation. A late change that was included in the operational ECMWF system but not documented in Geer and Bauer (21) was to assimilate only one in every two superobs. This was to avoid having to deal with spatially varying observation-error correlations, which have a scale length of about 12 km (Bormann et al., 211). However, in this article, we use all available superobs in our statistics. For simplicity we will refer to the superobs as observations. Variational Bias Correction (VarBC; Dee, 24; Auligné et al., 27) is used to remove systematic biases between the model and observations. Biases are estimated as a global constant plus linear functions of skin temperature, TCWV, and surface wind speed, plus a third-order polynomial in instrument scan position. As mentioned in the introduction, no attempt is made to correct for cloud- or precipitationrelated biases. The examples presented in sections 3 to 5 are taken from the pre-operational testing of the ECMWF cycle 36r4 system, which became operational on 9 November 21. This runs at a spectral resolution of T1279, equivalent to 16 km. The examples in section 6 come from Geer and Bauer (21), which also provides further technical details on the current all-sky approach. 3. Error covariances in data assimilation The atmospheric analyses produced by a data assimilation system are a weighted combination of a background forecast and new observational information (e.g. Kalnay, 23). The weighting is determined solely by the error covariances of background and observations, which are usually estimated or modelled. One of the main sources of information on the structure of these error covariances is the departure (or innovation) between model and observations, here including the VarBC bias correction b i : d i = y o i H i [M i {x b (t )}] b i. (1) Notation follows that of Ide et al. (1997) for 4D-Var assimilation, where x b (t ) is the FG (or background). Observations are distributed within a time window (in our case 12 h) divided into n discrete timeslots i. In each timeslot, observations y o i are compared to the corresponding model estimates from H i which is a nonlinear observation operator that, for example, simulates observed TB given input profiles from the atmospheric model. M i {x(t )} represents running the nonlinear atmospheric model from timestep t to t i. Ignoring cross-correlations between errors in the observations and the background, assuming the bias, b i, is and using the tangent linear operators H and M derived from the nonlinear equivalents H and M, the covariance of FG departures can be shown to be a sum of all the errors in the comparison (e.g. Andersson, 24): } <d, d T > = HPH T + O + F, P = MBM T (2) + Q. Table I lists the meanings of the error covariance matrices O, F, P, B and Q. For notational simplicity we have neglected the timestep index i. In the strong-constraint 4D-Var formulation used at ECMWF, Q is neglected, but we will see later this is not ideal, especially for cloud and rain data assimilation. We will combine the observation-related errors into one matrix R = O + F. Many studies (e.g. Hollingsworth and Lönnberg, 1986; Desroziers et al., 25) have used Eq. (2)

4 Observation Errors in all-sky Data Assimilation 227 Table I. Meanings of the error covariance matrices in Eq. (2). Name Description O True observation error, i.e. the instrumental noise F Representativity error, i.e. the error in the observation operator H (in our case RTTOV-SCATT), including errors from interpolation or mismatched scales between observation and model. R O+ F P Forecast error at the observation time B Background error, i.e. the error in x b (t ) Q Forecast model error, i.e. the error in the forecast model M to estimate the properties of R and B. Here, we will ignore error correlations and concentrate simply on the error variances (i.e. the diagonal part of these matrices), because in cloudy and precipitating conditions these are complicated enough in their own right. Bormann et al. (211) examine the error correlations (i.e. the off-diagonal elements) in all-sky assimilation, finding considerable inter-channel and spatial correlations. Inter-channel correlations become even stronger in cloudy and rainy situations. 4. All-sky observations and their model equivalents Figure 1(a) shows a scatter plot of AMSR-E TB observations y o against the bias-corrected model FG: y b = H[M{x b (t )}] + b, (3) (again ignoring the subscript i for simplicity). The figure is based on a sample of observations between 6 Sand 6 N, using all available oceanic observations between 12 and 19 September 21, except where the model is affected by cold-sector or excess-scattering biases. It also excludes one patch of bad AMSR-E data which is discussed later in this section. For TBs below 22 K, there is a strong correlation between the observations and the model, but for higher TBs the correlation becomes worse. Figure 1(b) shows only the points where both model and observations are free from cloud or precipitation (to be precise, points where the liquid water path retrieved from observed or simulated radiances is smaller than.5 kg m 2 ; Karstens et al., 1994; Geer et al., 28). In this clear-sky sample the agreement between model and observations is good. This shows that global and synoptic-scale patterns of water vapour (and seasurface properties) are well represented in the ECMWF FG, and that the observation operator (including bias correction) works well in clear skies. Hence, the additional scatter in Figure 1(a) comes from situations where there is cloud or precipitation, where the agreement between FG and observations appears to be much worse. Channel 19v observations are primarily sensitive to surface temperature and TCWV variations and, as illustrated by Figure 1(b), this results in roughly a 4 K variation between the Tropics, where TCWV is highest, and the high latitudes, where both surface temperature and TCWV are relatively low. On top of this, the presence of cloud or rain typically acts to increase the brightness temperature, by up to 6 K. Hence the actual TB T could be thought of as the (a) 28 Observed TB [K] r = (b) 28 Observed TB [K] (c) Observed cloud influence [K] First guess TB [K] r = First guess TB [K] 2 r = First guess cloud influence [K] Figure 1. Scatter plots of AMSR-E channel 19v observations versus first guess (FG): (a) globally, (b) restricted to situations with clear observations and clear FG, and (c) cloudy part of TB only (see text). The sample is from 12 to 19 September 21. Density is represented by logarithmic contours at 3, 1, 32, 1 and so on, per 2 K by 2 K bin. Pearson correlation coefficients are given in the corner of each plot. sum of a clear-sky part T clr and a modification coming from cloud or rain, T cld : T = T clr + T cld. (4) The FG TB ignoring the effect of hydrometeors, T b clr,isabyproduct of our observation operator (e.g. Geer et al., 29),

5 228 A. J. Geer and P. Bauer andwecaninferfromfigure1(b)thatitwillbereasonably accurate. In parallel with Eq. (3), the bias-corrected vector of clear-sky FG can be labelled y b clr. Hence the influence of cloud and precipitation can be computed for model and observations (respectively ycld o and yb cld, and using nonbold characters to designate scalar elements taken from the corresponding vectors in bold) as: ycld b = yb yclr b, (5) ycld o yo yclr b. (6) Figure 1(c) shows a scatter plot between ycld o and yb cld. There is little correlation, which confirms that the FG cloud and rain is often very different from what is observed. Essentially, cloud and rain features are often badly located, poorly shaped, or have the wrong intensity in the FG compared to the observations. For simplicity we will refer to this as mislocation error. The aim of data assimilation is to use the discrepancies between modelled and observational information to improve the model state, but Figure 1 encapsulates the challenge of assimilating cloudy observations. We cannot hope to infer very small errors in clear-sky moisture (order 1 2 K in TB; Figure 1(b)) in the presence of very large errors in cloud or precipitation (up to 5 K; Figure 1(c)). However, the logarithmic spacing on these plots means that, in terms of TB, the majority of cases have errors in cloud and precipitation that are comparable to those coming from clear-sky moisture errors (68% of cases show ycld o yb cld < 5 K). Hence, the majority of cloudaffected observations should bring useful direct information on humidity as well as on cloud. Figure 2 examines agreement between model and observations in channel 37v, which is more sensitive to cloud than 19v. Again, in clear skies the FG and observations are in good agreement (Figure 2(b)) and relative to this the cloudy/rainy part of the signal exhibits a much greater scatter (Figure 2(c)). However, there is a slightly better correlation between the model and observations than for channel 19v, which responds more to precipitation. This could be interpreted as showing that cloud is slightly better forecast than precipitation. Figures 1 and 2 also illustrate the problem of boundedness which was mentioned in the introduction. Outside of situations with strong scattering, microwave imager TBs can be no lower than the clear-sky TB (which is mainly dependent on TCWV) and no higher than the point at which TBs saturate in heavy precipitation, which is around K in these channels. However, even clear-sky TBs are bounded by geophysical limits, but in practice they have never caused difficulties in assimilation. The problem is not the bounds themselves so much as the relative size of errors in the model. Model errors are so large that it is perfectly normal to forecast precipitation but not to observe it, or vice-versa. Similarly, it could be argued that it is because the errors in forecast relative humidities are so large compared to the geophysical range that Hólm et al. (22) needed to introduce a symmetrising transform to improve the Gaussianity of the moisture control variable. We have seen from Figures 1 and 2 that observation or background errors must vary widely between clear-sky and cloudy situations, giving rise to the heteroscedasticity described in the introduction. This non-gaussianity can (a) 28 Observed TB [K] (b) Observed TB [K] (c) Observed cloud influence [K] r = First guess TB [K] r = First guess TB [K] 2 r = First guess cloud influence [K] Figure 2. As Figure 1, but for AMSR-E channel 37v, which is more sensitive to cloud. easily be seen in Figure 3. The solid line shows a probability distribution function (PDF) of channel 19v FG departures normalised by the sample standard deviation, which is 4.3 K. For comparison, the dotted line shows a Gaussian. Departures of more than three times standard deviation are far more common than in the Gaussian distribution. These big departures are associated with cloud and rain areas. Departures of less than.5 times standard deviation are also far more common than they should be, and make

6 3 Observation Errors in all-sky Data Assimilation PDF FG departure [K] Normalised FG departure C 37 Figure 3. PDFs of AMSR-E channel 19v FG departures, normalised by the standard deviation of the whole sample (solid line). The dotted line gives the Gaussian distribution. For both curves the integral over the PDF is 1. Figure 4. Scatter plot of AMSR-E channel 19v FG departure against C 37. Density is represented by logarithmic contours. up the majority of the sample. Clearly, a constant standard deviation is not appropriate to describe the variation in all-sky departures. To illustrate this, we could think about assigning a constant observation error of 4.3 K. However, this would be excessive for a clear-sky observation and the information content of the observation would mostly be ignored. In contrast, the error of cloud-affected observations would likely be underestimated and too great a weight would be given to them. In practice, observations which have been assigned insufficiently large observation errors are removed by the threshold check on FG departures (Figure 5 of Bauer et al., 21), which in our system excludes normalised FG departures larger than 2.5 (section 5.3). So in practice, even to get the cloud- and precipitation-affected observations past quality control, observation error (or background error, or both) must vary as a function of cloud amount. 5. FG departure errors as a function of cloud amount 5.1. Measures of cloud amount To examine the errors in FG departure as a function of cloud, it would be tempting to use the model s cloud amount as a predictor, since it is readily accessible. But as we have seen (e.g. Figure 2(c)), that indicates relatively little about the presence of cloud in the observations. Hence, to obtain a comparable measure of cloud from either observations or model, we must work in radiance space, and apply a simple approximate retrieval of the cloud amount. Here, we take the normalised 37 GHz polarisation difference P 37 (Petty and Katsaros, 199; Petty, 1994), which is roughly equal to the square of the slant path transmittance of cloud and precipitation at this frequency, τ 37 : P 37 = Tv T h Tclr v τ Th (7) clr Here, T v and T h are the vertically and horizontally polarised 37 GHz brightness temperatures at the top of the atmosphere, and Tclr v and Th clr are the brightness temperatures for the same profile but without clouds or precipitation. Since, at this frequency, emission at the sea surface is highly polarised, but atmospheric absorption is in general unpolarised, T v T h is a measure of atmospheric opacity, with a completely opaque atmosphere giving no polarisation difference. Hence, P 37 varies between, which represents a profile with opaque cloud, and 1, which represents clear sky. Note that cloud and precipitation have much lower optical depth in the microwave than in the visible or infrared, so only the most intense convection is opaque at 37 GHz. Here, T v and T h come either from the bias-corrected observation or the FG. Tclr v and Th clr are always simulated using the FG profile, making the assumption that errors in modelled moisture and sea-surface state are less important than the cloud signal. In any case, we are still working in observation space and just transforming T v and T h to a quantity that is a rough indicator of the cloud and precipitation amount. For convenience, we will work with a variable that increases with cloud and precipitation amount, which we can define as: C 37 = 1 P 37. (8) For simplicity we will often just refer to cloud amount. We can compute C37 b and Co 37 from the FG and observed TBs. We can also compute the symmetric or mean cloud: C 37 = Cb 37 + Co 37. (9) 2 As we will see, it is the symmetric cloud that will be most useful. Figure 4 shows a scatter plot of FG departures against the symmetric cloud amount. The typical size of FG departures increases with cloud amount. However, noting that the contours are logarithmic, the vast majority of cases show either clear skies or only a small amount of cloud. This broadly log-normal distribution is typical of cloud and rain. In the operational implementation, bias correction is not applied in Eq. (7) because it is not available at the stage when P 37 is evaluated, but any biases have only a small influence on P 37.

7 23 A. J. Geer and P. Bauer 5.2. Symmetric and asymmetric approaches The mean and standard deviation of FG departures can be computed as a function of FG, observed, or symmetric cloud amount (C b 37, Co 37 and C 37), and these are shown in Figure 5. Figure 5(a) shows that, at high cloud amounts, binning by observed cloud gives a positive bias of up to 3 K, while binning by FG cloud gives a negative bias of up to 26 K. These binning strategies are strongly affected by sampling. Where the FG is cloudy, many observations are cloudy too, but it is likely that some observations will show clear skies. Hence, in this sample, the FG will on average contain more cloud than observed, and FG departure biases will be positive. But there is a counterbalancing sample of cases where the observations are cloudy and the FG is clear. Similar sampling biases occur when selecting for clear skies, where binning by FG cloud amount leads to a negative sampling bias and using observed cloud leads to a positive bias. These sampling considerations are one of the main justifications for the all-sky approach (Geer et al., 28; Bauer et al., 21). Binning by the mean cloud amount avoids these asymmetric sampling problems. Similar reasoning was followed when Hólm et al. (22) introduced their symmetrised moisture control variable. Sampling considerations are also important if hoping to apply a bias correction to cloud-affected observations. It is typical to use the FG forecast to provide predictors for bias correction. However, the bias as a function of FG cloud in Figure 5(a) is simply a feature of the sampling, and not a real bias. Using the FG cloud amount as a bias predictor would be incorrect, though it has been done in the past. At ECMWF, FG rain was used as a bias predictor in the 1D+4D-Var system (Geer et al., 27), and FG logarithmic cloud optical depth was used as a bias predictor in experimental assimilation of Moderate Resolution Imaging Spectroradiometer (MODIS) observations (Benedetti and Janisková, 28). In particular, their Figure 3 presents the mean FG departure as a function of FG cloud, and shows exactly the sampling bias we would expect. The effect of treating the sampling bias as a real bias and removing it from the observations is to reduce the average size of the increments, i.e. it reduces the impact of the observations by removing real information. The symmetric cloud amount solves this problem and is the recommended choice of predictor for any bias correction scheme. Despite being unaffected by sampling issues, binning by symmetric cloud still reveals a bias that varies with cloud amount, reaching +2 K at lower cloud amounts and dropping to -5 K at higher amounts (Figure 5(a)). However, it is small compared to the effects of sampling, and small compared to the standard deviations (the ratio is no greater than.4). We did try to correct this bias using VarBC but found very slow spin-up times and little benefit (Geer and Bauer, 21). Hence it was acceptable to leave it uncorrected in the current implementation of all-sky assimilation at ECMWF. However, we will make further attempts to get a symmetric bias correction working. Figure 5(b) shows that, when binning by FG or observed cloud amount, the FG departure standard deviation increases with cloud amount, as we might expect. However, binning by symmetric cloud amount gives standard deviations that peak at around C 37 =.5 before declining towards the cloudy end. This is because the highest values of C 37 indicate agreement between observation and FG, with heavy rain or cloud in both. Similarly when C 37 =., this means (a) Mean [K] (b) Standard deviation [K] C C 37 Figure 5. (a) Mean and (b) standard deviation of AMSR-E channel 19v FG departures, binned as a function of observed cloud amount (C o 37, dashed), FG cloud amount (C b 37, dot-dashed) or symmetric cloud amount (C 37, solid) that both model and observations must be entirely free from cloud, leading to a standard deviation of 1.5 K, as opposed to 2 K when the binning is done by FG or observed C 37 alone. Whichever way the binning is done, it appears that cloud amount is a good predictor of the standard deviation of FG departures. We know from Eq. (2) this is the quadratic sum of the forecast- and observation-error standard deviations, so cloud amount would also be a good predictor of observation error if we could separate this from the forecast error. The negative values of C 37 in Figure 4 indicate one limitation in our retrieval of cloud. These negative values come from negative values of C37 o, which occur when the FG clear-sky polarisation difference Tclr v Th clr is smaller than that the observed all-sky polarisation difference T v T h (Eqs. (7) and (8)). This can happen if TCWV is higher in the model than in reality, so using FG values of Tclr v and Th clr in estimating C37 o does introduce some error in clear-sky areas. When binned separately as in Figure 5, observations with negative C 37 have slightly larger standard deviations of FG departures than do those in the bin with C 37 =. This is to be expected, as we are effectively selecting for situations where FG and observations disagree. In practice, these negative values of C 37 form a relatively small population (1% of the sample), which can be included in the bin with C 37 =. The mean of cloud amount (Eq. (9)) is not the only possible symmetric predictor. We also experimented with the maximum cloud amount, max(c37 b, Co 37 ), and the absolute difference, C37 b Co 37. Using the maximum gives as

8 Observation Errors in all-sky Data Assimilation 231 (a) PDF (b) PDF Normalised FG departure Normalised FG departure Figure 6. PDFs of AMSR-E channel 19v FG departures, normalised by the standard deviation of the whole sample (light solid line) or by the standard deviation binned as a function of FG cloud amount (C37 b, thick solid in (a)) or symmetric cloud amount (C 37, thick solid in (b)). The dotted line gives the Gaussian distribution. For all curves the integral over the PDF is 1. good a fit to the Gaussian as with the mean (see next section), but is not as useful for quality control, because bad data can more easily masquerade as heavy cloud. Using the absolute difference results in a bimodal distribution of errors, which is not helpful Improved quality control Using the binned standard deviations from Figure 5(b), we can normalise the FG departures as a function of cloud amount. Figure 6 shows the PDFs of departures normalised by FG and symmetric cloud amount, with the curves from Figure 3 for comparison. Both normalisations make the departures appear more Gaussian. However, using the FG cloud amount gives a distribution that is too peaked compared to the Gaussian, and there is considerable skewness, showing again that FG cloud amount is a biased predictor. By contrast, the symmetric cloud amount gives a distribution that is similar to the Gaussian out to about 2.5 and Hence, symmetric cloud amount is a good predictor of the error in FG departures. The cloud amount C 37 can be computed from model or observation as part of the FG screening, so we should be able to predict the size of the departures in advance of the variational minimisation. In section 6 we use this knowledge of FG departure errors to model the observation error. However, even before this, we have a useful tool for cloudy data assimilation. As mentioned before, quality control rejects these observations when the normalised departure is greater than 2.5. At ECMWF the normalisation is done using the quadratic sum of observation error and an estimate of background error in observation space. However, we can now predict the error in FG departures directly, without attempting to resolve it into its component parts. Hence, our first use of the symmetric cloud amount is as a predictor for quality control. Figure 7(a) shows a map of FG departures on a day when AMSR-E suffered an instrument anomaly, indicated by a Y. A large cross-swath bias develops towards the south, becoming as large as 3 K on the east side. There seems to be a corresponding positive bias developing on the west side of the swath, but some of this data appears to have been identified as bad and withheld by the data providers. The difficulty in all-sky data assimilation is that a 3 K departure can just as easily represent real geophysical information, such as indicated by X on the figure, where tropical deep convection is slightly misplaced in the FG relative to the observations. Hence, a simple threshold check on FG departures will eliminate useful information along with the bad data. In contrast, by normalising the departures with the symmetric cloud amount (Figure 7(b)), the areas affected by the instrument anomaly stand out as having very large normalised departures. The threshold check is applied in Figure 7(c) with a limit of 2.5, and the majority of the bad data have been removed. There is still some residue of bad data in the centre of the swath, but this would be equally hard to identify using another method. The quality control has also eliminated some negative departures associated with the displaced deep convection at X, but the majority of observations have been retained. Some positive departures near Z have also been removed. This is an area of cold-sector bias, so the data is quite correctly being rejected. A new quality control based on these principles was implemented for all-sky observations in the ECMWF system in the November 21 upgrade. It has been shown to successfully identify bad scan lines in SSM/I data (Geer and Bauer, 21) and radio frequency interference in AMSR- E data in North American coastal areas (B. Krzeminski, personal communication). Some bad data can be allowed into the assimilation if the instrument anomaly is such that C37 o erroneously indicates heavy cloud, but only in combination with a C37 b that indicates cloud in the FG. However, in such situations, the observation errors will be set to relatively large values (section 6) and the observation and FG will be roughly in agreement, so the bad data will have very little impact on the analysis. Overall, the new quality control appears robust. 6. Observation error as a function of cloud amount 6.1. Approximate model for observation error We can now predict the total error of FG departures as a function of cloud amount and we would like to use this to make a model for observation errors. It is quite common to use the FG departures as a guide to the size of observation errors and a number of methods can be used to estimate the partitioning between background and observation error (e.g. Hollingsworth and Lönnberg, 1986; Desroziers et al., 25; Bormann and Bauer, 21). Bormann et al. (211) applied

9 232 A. J. Geer and P. Bauer FG departure (a) X Y Z [K] Normalised FG departure all (b) Normalised FG departure screened (c) Figure 7. AMSR-E channel 19v FG departures at around 1 UTC on 18 September 21 (a) in terms of brightness temperature (K), (b) normalised by the symmetric error distribution (no units), and (c) which is as (b) but with normalised departures removed outside the range 2.5 to See text for explanation of X, Y and Z in (a). the Desroziers et al. (25) method to a clear and a cloudy set of data; applying such methods to each symmetric cloud bin separately is a task for the future. For the November 21 operational implementation, we decided simply to tune an observation-error model to give short-range forecasts with the best fit to the other observations in the analysis. Rather than using binned standard deviations as earlier, the starting point for the error model was to make a piecewise linear fit to the standard deviations of Figure 5(b), or the equivalent curves for other channels and instruments, to give afunctiong(c 37 ), where g is the total error. Figure 8 shows how this is done. We ignore the fall in standard deviations for very high values of C 37. This is done for simplicity, but it is also a cautious approach to avoid giving small errors in very cloudy situations. In any case, the number of observations with very high C 37 is small (e.g. Figure 4). The main advantage of the linear fit over the binned statistics is just that it results in smaller (hence easier to maintain) configuration files in the operational system.

10 Observation Errors in all-sky Data Assimilation C CLR C CLD 2. Error [K] g CLD FG departure std. dev. [K] Ch. 3 (183±1 GHz) Ch. 4 (183±3 GHz) g CLR Ch. 5 (183±7 GHz) C ControlOff Control New 1. New.67 New.33 New. Experiment Figure 8. Error model for AMSR-E channel 19v, showing how the standard deviations of FG departures (thin solid line, from Figure 5(b)) are modelled by a piecewise linear fit defined by g clr, g cld, C clr and C cld (thick dashed line). Also shown is the resulting assigned observation error for α =.5 (thin dashed line). See appendix for more detail. Figure 9. Standard deviation of FG departures (K), for all assimilated AMSU-B and MHS observations globally from 1 to 1 October 29, as a function of experiment, ordered so that the weight of all-sky observations increases towards the right. VarQC is either on (solid line with crosses) or off (dashed line with diamonds) for the all-sky observations. Name Table II. Experiments. Description Control-Off As Control, but with microwave imagers switched off Control 36r1 control New 1. New approach with α =1. New.67 New approach with α =.67 New.33 New approach with α =.33 New. New approach with α =. All microwave imager channels have a similar-shaped dependence on C 37, with a straight line giving an adequate description of the increase in error between clear and cloudy situations. Curves like the one shown in Figure 5(b) are stable with time and can be generated even from a single day s data. The observation error r is modelled as if it can be split into a clear part r clr, which is constant and pre-defined, and a cloudy part r cld, which depends on the total error, and hence on cloud amount: r 2 = rclr 2 + r2 cld, rcld 2 = α { g 2 (C 37 ) g 2 () }. (1) By tuning the parameter α, the cloudy part of the total error can either be assigned completely as observation error (when α = 1) or ignored and assumed to come entirely from background error (when α = ). The appendix gives more detail First-guess fits to observations Geer and Bauer (21) presented a series of experiments which were used to determine the appropriate value of α. These experiments, summarised in Table II, have a resolution of T799 and used the cycle 36r1 version of the ECMWF system. The period of comparison is 1 1 October 29. AMSR-E and SSM/I observations were assimilated in FG departure std. dev. [K] ControlOff Control New 1. New.67 New.33 New. Experiment Figure 1. As Figure 9, but for AMSU-A channel 5. all but the Control-Off experiment. Because the control system assigns an excessively large error to the all-sky observations (Bauer et al., 21; Geer and Bauer, 21), the new approach gives a much greater weight to them than the control, even with α = 1, i.e. using the maximum possible observation errors from Eq. (1). α was varied between 1. and. to give four experiments with the new configuration. All experiments were run twice, once with the normal configuration and once with VarQC (Andersson and Järvinen, 1998) switched off for the all-sky observations. VarQC downweights the influence of observations that disagree with the analysis, which masks any problems coming from excessively small observation errors. Figures 9, 1 and 11 show the standard deviation of Advanced Microwave Sounding Unit B (AMSU-B) and Microwave Humidity Sounder (MHS), AMSU-A, and High Resolution Infrared Radiation Sounder (HIRS) FG departures as a function of experiment, with observation weight increasing towards the right. AMSU-B and MHS channels 3 to 5 are sensitive to humidity in the upper, mid and lower troposphere, respectively. AMSU-A channel 5 is sensitive to temperature over the whole troposphere, and is the lowest-sounding of the AMSU-A channels assimilated at ECMWF. HIRS channels 5, 6 and 7 sense tropospheric temperature with weighting functions peaking at 5 hpa, 8 hpa and the surface. Over the 1-day period of these

11 234 A. J. Geer and P. Bauer FG departure std. dev. [K] Ch. 7 Ch. 6 Ch. 5 ControlOff Control New 1. New.67 New.33 New. Experiment Figure 11. As Figure 9, but for HIRS channels 5 to 7. experiments, radiosondes and other in situ data were not numerous enough to provide reliable statistics. Overall, New 1. improves 12 h forecasts compared to Control or Control-Off, with a better fit to AMSU-B/MHS and HIRS. Though there is a slight degradation of fit to AMSU-A channel 5, this is only of order.1 K and (not shown) limited to the Tropics. Changing α to.67 or.33 makes little difference, but going to. degrades the fit. Here, we are clearly over-constraining the analysis, and this is even more obvious when VarQC is switched off. It is strange that results from AMSU-A channel 5 seem to disagree with the decrease in FG departure standard deviation between Control-Off and New 1. in HIRS channels 5 to 7. HIRS channel 7 is sensitive to lowertropospheric water vapour as well as temperature, while AMSU-A channel 5 is purely sensitive to temperature. Thus the most likely explanation is that the all-sky observations substantially improve the forecast moisture field but very slightly degrade the temperature fit. Further research is needed to confirm this, however, and we will need to investigate the role of the relative humidity control variable in controlling temperature and moisture increments in saturated (i.e. cloudy) areas. From this exercise, the best value for α appeared to be 1, i.e. all cloudy total error should be assigned as observation error. This is what was implemented operationally. Forecast scores for longer time-scales are most easily computed using analysed fields as the reference. These scores are not examined here because any real changes to the forecasts are swamped by changes in the activity of the analyses (Geer et al., 21; Geer and Bauer, 21). Midlatitude geopotential forecast scores can be computed with radiosonde observations as the reference, but changes to the all-sky assimilation have little effect on them (not shown). 7. Discussion The previous section shows that the best fit to other observations comes when the all-sky observations are assigned errors of essentially the same size as the FG departure standard deviation. This seems to suggest that errors in the FG are insignificant compared to the observation error. For AMSR-E channel 19v, the assigned observation error can be as large as 18 K. Yet instrument noise is of order.5 K and the error in simulating cloud and rain radiative transfer at frequencies such as 19 GHz should be small, since it is really an absorption rather than a scattering problem. The errors of the simplified cloud overlap in RTTOV-SCATT can be estimated by comparison to a more detailed overlap (Geer et al., 29) but are no more than 2 K in even the most cloudy situations (unpublished work). Hence, we would have expected the true R to be much smaller than what we are currently assigning. The model of observation error presented so far does not account for the observation-error correlations found by Bormann et al. (211). Recent experiments that attempt to mitigate this by increasing the spacing between observations, or by using fewer channels in cloudy areas, do not substantially alter the fits to other observations (not shown). Therefore spatial or inter-channel observation error correlations do not appear to be the main factors giving rise to such large assumed observation errors in cloudy areas. In fact, the correlation-aware method used by Bormann et al. (211) also suggests that observation errors should be smaller than they are now. Instead, the explanation for the large departures seems obvious: the model often gives a poor forecast of cloud and rain at the observed time- and space-scales (being off in terms of displacement, shape, or intensity) and large errors result. These mislocation errors are clear to see in maps of FG departures, such as near point X in Figure 7(a). The mislocation of cloud in the FG likely results from errors in the initial conditions (i.e. background error B) and errors in the forecast model (Q, Eq. (2)). In the 4D-Var system at ECMWF, the forecast model is effectively part of the observation operator but Q is ignored, giving a strong constraint formulation. Hence in the current approach, the only place these Q errors can be included is in the observation error R, giving a situation where probably the largest component of the observation error really comes from deficiencies in the forecast model s ability to predict cloud and rain correctly. Beyond this, the background error B in the ECMWF system does not yet represent the major differences between precipitating and clear regions found by e.g. Caron and Fillion (21), Montmerle and Berre (21) and Michel et al. (211). For example, precipitating regions should probably have larger background-error variances for divergence and vorticity, very different horizontal and vertical correlations and different balance characteristics. Hence, the inflated observation errors may also compensate for an inadequate specification of B. The model error in observation space (P = MBM T + Q) can be computed from the spread of an ensemble of 4D-Var data assimilations; initial studies (not shown) reveal a very similar functional dependence on symmetric cloud to what is shown in Figure 5(b). This backs up the suggestion that our assigned observation error includes a large component coming from forecast model and/or background errors. Hence, background error modelling must be improved in cloudy and rainy areas, and it would also be useful to start correctly representing the contribution of forecast model error by means of a weak-constraint 4D-Var formulation. On top of this, we have seen that there are small uncorrected systematic biases between model and observations on a global scale (Figure 5(b)) and there may also be unidentified situation-dependent errors in the system, similar to the cold-sector or excess-scattering bias, but less obvious to