Estimating observation impact without adjoint model in an ensemble Kalman filter

Transcription

1 QUARTERLY JOURNAL OF THE ROYAL METEOROLOGICAL SOCIETY Q. J. R. Meteorol. Soc. 134: (28) Published online in Wiley InterScience ( DOI: 1.12/qj.28 Estimating observation impact without adjoint model in an ensemble Kalman filter Junjie Liu 1 * and Eugenia Kalnay 2 1 University of California, Berkeley, California, USA 2 University of Maryland, College Park, Maryland, USA ABSTRACT: We propose an ensemble sensitivity method to calculate observation impacts similar to Langland and Baker (24) but without the need for an adjoint model, which is not always available for numerical weather prediction models. The formulation is tested on the Lorenz 4-variable model, and the results show that the observation impact estimated from the ensemble sensitivity method is similar to that from the adjoint method. Like the adjoint method, the ensemble sensitivity method is able to detect observations that have large random errors or biases. This sensitivity could be routinely calculated in an ensemble Kalman filter, thus providing a powerful tool to monitor the quality of observations and give quantitative estimations of observation impact on the forecasts. Copyright 28 Royal Meteorological Society KEY WORDS observation impact; ensemble sensitivity; adjoint model; monitoring observation quality Received 16 December 27; Revised 14 May 28; Accepted 27 May Introduction In recent years, operational numerical weather prediction centres have assimilated more satellite observations, such as the kilo-channel Advanced InfraRed Sounder (AIRS) and the Infrared Atmospheric Sounding Interferometer (IASI), in addition to in situ observations. Statistically, the assimilation of new observations improves, on average, the accuracy of short-range forecasts (e.g. Joiner et al., 24; Le Marshall et al., 26). However, the value added to the forecast by various observations depends on the instrument type, observation type and observation locations, as well as the presence of other observations. The knowledge of the impact that different observations have on the forecasts is important to better use the observations which have larger impact on the forecasts, and to avoid using observations which have no impact or even negative impact on them. Traditionally, the observation impact has been estimated by carrying out experiments in which part of the observations used in the control experiment were excluded in the data-denial experiments (e.g. Zapotocny et al., 22). However, this requires very large computational resources since a new analysis forecast experiment has to be carried out for any subset of observations that needs to be evaluated. Langland and Baker (24, LB hereafter) proposed an adjoint-based procedure to assess the observation impact on short-range forecasts without carrying out data-denial experiments. This adjoint-based * Correspondence to: Junjie Liu, Earth and Planetary Science Department, University of California, Berkeley, CA, 9472, USA. jjliu@atmos.berkeley.edu procedure can evaluate the impact of any or all observations assimilated in the data assimilation and forecast system on a selected measure of short-range forecast error. In addition, it can be used as a diagnostic tool to monitor the quality of observations, showing which observations make the forecast worse, and can also give an estimate of the relative importance of observations from different sources. Following a similar procedure as LB, Zhu and Gelaro (28) developed the adjoint of the Gridpoint Statistical Interpolation (GSI) assimilation system to estimate observation impact in a quasi-operational data assimilation system. They showed that their procedure provides accurate assessments of the forecast sensitivity with respect to most of the observations assimilated. This adjoint procedure to estimate observation impact requires the adjoint of the forecast model which is complicated to develop and not always available. In this paper, we propose an ensemble-based sensitivity method to assess the observation impact as in LB but without using the adjoint model. It is different from the ensemble sensitivity method proposed by Ancell and Hakim (27) as discussed in Section 2. We compare the observation impact calculated from the proposed ensemble sensitivity method with that from the adjoint method, and further compare the observation impacts calculated from both methods with the actual forecast error reduction due to assimilation of these observations in the Lorenz 4-variable model (Lorenz and Emanuel, 1998). The paper is organized as follows: the derivation of the formula is given in Section 2, and an alternative nonlinear derivation is briefly given in appendix B. The experimental design is presented in Section 3, and the results are in Section 4. Section 5 contains a summary and conclusions. Copyright 28 Royal Meteorological Society

2 1328 J. LIU AND E. KALNAY 2. Derivation of the ensemble sensitivity method to calculate observation impact without using adjoint model 2.1. The sensitivity of forecast errors to the assimilated observations Following LB, in order to study the observation impact on the reduction of forecast errors, we first calculate the sensitivity of a cost function at time t to the observations assimilated at time t = h (Figure 1). LB defined a cost function at time t as the difference of forecast error energy norm between the forecast from initial condition at h (at a time when observations y o were assimilated) and that from initial condition at 6 h, which does not benefit from the use of the observations y o. Without loss of generality, and since we will test our calculation procedure in the Lorenz 4-variable model, we define a cost function as the difference of squared forecast errors (L2 norm) between the forecasts started at h and 6 h and verified at time t: J = 1 2 (et t e t e T t 6 e t 6) = 1 2 (et t + et t 6 )(e t e t 6 ), (1) where e t = x f t xa t, e t 6 = x f t 6 xa t, and xa t is the verification analysis at time t. The forecast length should be short enough that perturbations grow linearly, and long enough that the forecast error is much larger than the error of the verification analysis state. Following LB, we choose t = 24 h. The verification analysis state can be either from the same or from a different analysis system as long as it is more accurate than the forecasts x f t 6 and x f t. In the application to numerical weather forecast models, the cost function (Equation (1)) can be replaced by the forecast error energy norm difference as defined in LB without affecting the following derivations, since the forecast error energy norm is simply the multiplication of the squared forecast errors with the energy scaling coefficients. The use of a forecast error energy norm difference in the cost function allows comparison of the impact of different types of observations such as wind and temperature. The purpose of the following derivation is to rewrite Equation (1) as a function of the observations assimilated at h. We follow Bishop s (27) notation, with the first sub-index indicating the verification time, and the second Obs. e t 6 6hr hr t forecast time Figure 1. Schematic plot of the time relationship of the observation impact on the reduction of forecast errors at time t. (After Langland and Baker, 24, Figure 1). e t sub-index, separated by a vertical bar, indicating the time of the initial conditions of a forecast or forecast error, so that x f t and xf t 6 are the ensemble mean forecasts valid at time t, initialized at h and 6 h respectively. Here, K is the number of ensemble members, x f t = 1/K K M t (x ai ), xf t 6 = 1/K K M t 6(x ai 6 ), and M t and M t 6 represent the integration to time t with the nonlinear model initialized with the analysis at h and 6 h respectively. Substituting the definitions of e t and e t 6 into the above equation, we can write the cost function as: J = 1 2 (2e t 6 + x f t xf t 6 )T (x f t xf t 6 ). (2) In the following derivation, we aim to express the forecast difference (x f t xf t 6 ) valid at time t as a function of the observation increments v = y o yb 6 at h (Figure 1), so that the sensitivity of the cost function to the observation increments J/ v will be a function of the observations assimilated at time h. y b 6 = h(xb 6 ) is the prediction of the observations assimilated at t = h, with h( ) the nonlinear observation operator. As shown in Hunt et al. (27), the i th analysis ensemble member x ai can be written as: x ai = xb 6 + Xb 6 K v + δx ai, (3) where X b 6 = [ δxb1 6 δx bk 6 ] is a matrix whose K columns are background ensemble perturbations with the i th column given by δx bi 6 = xbi 6 xb 6. K = P a YbT R 1 and P a ={(K 1)I + YbT R 1 Yb } 1 are the Kalman gain matrix and analysis error covariance matrix in the ensemble subspace spanned by ensemble forecast perturbations X b 6. Yb is a matrix whose ith column is equal to h(x bi 6 ) h(xb 6 ). R is the observation error covariance. δx ai = xai xa is the ith analysis perturbation vector which is the i th column of the analysis perturbation matrix X a = Xb 6 {(K 1) P a }1/2. An overbar represents an average over K ensemble members, a tilde indicates that a vector or matrix is represented in the subspace of ensemble forecast perturbations X b 6, and δ represents the difference between an ensemble member and the ensemble mean. Based on Equation (3), the forecast x f t = 1/K K ) initialized at t = h is written as: M t (x ai x f t = 1 K M t (x b 6 + Xb K 6 v + δx ai ). (4) Note that although in the following derivation we make a linearization, the actual computation does not require either the tangent linear or adjoint model. We linearize Equation (4) around the background mean state x b 6, and define the tangent linear model as M t, so that: x f t = M t (x b 6 ) + 1 K M t (X b 6 K v + δx ai ). (5) Copyright 28 Royal Meteorological Society Q. J. R. Meteorol. Soc. 134: (28) DOI: 1.12/qj

3 OBSERVATION IMPACT WITHOUT ADJOINT MODEL IN ENSEMBLE KALMAN FILTER 1329 Based on Hunt et al. (27), δx ai = Xb 6 δwai, where δw ai is the ith column of a symmetric K by K matrix of weight perturbations W a ={(K 1) P a }1/2 with elements δw aji. Substituting δxai = Xb 6 δwai into Equation (5), we get the following equation: x f t = M t (x b 6 ) + 1 M t {X b 6 K ( K v + δw ai )}. (6) We define x f t 6 = M t (x b 6 ) = 1/K K M t 6(x ai 6 ). In addition, we assume that t is short enough that perturbations grow linearly, so that X f t 6 = M t X b 6, where X f t 6 = [ δxf1 t 6 δx fk t 6 ] is a matrix whose K columns are forecast ensemble perturbations with the i th column δx fi t 6 = xfi t 6 xf t 6. Note, although the forecast perturbations are formally defined with a linear model, they are actually computed using the nonlinear model. Based on X f t 6 = M t X b 6, Equation (6) becomes: x f t = xb t K = x b t 6 + Xf t 6 K v + 1 K X f t 6 ( K v + δw ai ) X fj t 6 δw aji. (7) The last term in Equation (7) vanishes, because the perturbation weights add up to unity when summed over either the K columns or the K rows: K δwaji = K δwaji = 1 (appendix A), and the average of the forecast ensemble perturbations 1/K K Xfj t 6 is equal to zero. Therefore, we can write Similarly, e t e t 6 = x f t xf t 6 = Xf t 6 K v. (8) e t + e t 6 = x f t xa t + xf t 6 xa t = x f t 6 xa t + xf t 6 xa t + xf t xf t 6 (9) = 2e t 6 + X f t 6 K v. Equation (1) is then written as: J = 1 2 (et t + et t 6 )(e t e t 6 ) = 1 2 (2e t 6 + X f t 6 K v ) T X f t 6 K v. (1) If the model is nonlinear, Equation (1) is a linear approximation of Equation (1). Since the error e t 6 is not correlated with the observations assimilated at t = h, the sensitivity of the forecast error to the observation increments can be written as: J v = ( K T XfT t 6 )(e t 6 + X f t 6 K v ) (11) Note that the sensitivity of the cost function J to the observation increments (Equation (11)) does not require the adjoint model. When an independent analysis state verifying at time t and forecast perturbations initialized at 6 h and verified at time t are available at h, such as in a reanalysis mode, Equation (11) can be calculated along with the data assimilation. In appendix B, we give another derivation of the sensitivity of the cost function J to the observations without linearization, which gives results indistinguishable from those calculated from Equation (11). We note that the definition of ensemble sensitivity used in this paper is different from that of Ancell and Hakim (27) and Torn and Hakim (28), who also proposed a method to calculate the forecast sensitivity to the observations without using adjoint model. In their approach, the sensitivity of the cost function to the observation increments is not defined as a function of the forecast error e t 6, as in LB and in the present study, but defined as a function of linear regression of 6- hour ensemble forecast perturbations to a given forecast metric that results in a different definition of sensitivity from LB s and ours Observation impact on forecasts As discussed in LB, the forecast sensitivity can be used to examine the actual observation impact on the forecast. Using the sensitivity J/ v, the observation impacts on the forecast error reduction can be written as: J = 1 2 (et t e t e T t 6 e t 6) = v, J. (12) v Equation (12) expresses the forecast error reduction as a function of observation increments v. When the assimilated observations improve the forecast at time t, the forecast error is reduced, and the value calculated from Equation (12) will be negative. When the assimilated observations degrade the forecast, the value calculated from Equation (12) will be positive. Furthermore, if the observation errors between subsets of observations are not correlated, the cost function J can be expressed as a sum of J l : J = L J l = l=1 L l=1 v l J v l, (13) where J l is the observational impact caused by the l th subset of the observations y o = [ (yo1 )T,,(y ol )T ] T and v l = yol hl (x b 6 ). Based on Equation (13), we can calculate the observation impact on the forecasts from any subset of observations without conducting data-denial experiments, and can also compare the relative importance of observations from different sources. Note that since J/ v l defined in Equation (11) may contain a covariance between the l th set of observations and other observations because observations are assimilated simultaneously, the reduction of forecast error by the l th set of Copyright 28 Royal Meteorological Society Q. J. R. Meteorol. Soc. 134: (28) DOI: 1.12/qj

4 133 J. LIU AND E. KALNAY observations v l J/ vl is not independent of the other observations. In the following sections, we will compare the observation impact calculated from the ensemble sensitivity method we proposed with that from the adjoint method and compare the results from both methods with the actual observation impact calculated from Equation (1). Further, we will examine whether the ensemble sensitivity method we proposed can actually detect bad observations whose errors have different characteristics than the Gaussian assumption ε o N(, R) made in the assimilation. These experiments are carried out with the Lorenz 4-variable model. In the adjoint method by LB the cost function is written as a function of the observations assimilated at hr as: J = J t ) J t 6 = v, K T M (( J T t 6 )/( x f t 6 ) + ( J t )/( x f t ), J t 6 = 1/2(x f t 6 xa t )T (x f t 6 xa t ), J t = 1/2(x f t xa t )T (x f t xa t ), MT is the adjoint model and K T is the adjoint of Kalman gain matrix which can be directly calculated in the ensemble Kalman filter. A more detailed derivation can be found in LB. 3. Experimental design The Lorenz 4-variable model is governed by the following equation: d dt x j = (x j+1 x j 2 )x j 1 x j + F. (14) The variables (x j,j = 1...J) represent a meteorological variable on a latitude circle with periodic boundary conditions. As in Lorenz and Emanuel (1998), J is chosen to be equal to 4. The time step is.5, which corresponds to a 6-hour integration interval. F is the external forcing, which is 8 for the nature run and 7.6 for the forecast, allowing for some model error in the system. Following LB, we estimate the impact of the observations assimilated at h on the forecast valid at t = 24 h, so that in Equation (1) the cost function is defined as the forecast error difference between a 24-hour forecast (initialized at h) and a 3-hour forecast (initialized at 6 h). The error difference between these two forecasts is solely due to the assimilation of the observations at h in the initial conditions of the 24-hour forecast. Since the forecast perturbations start to grow nonlinearly with time, we also extend the forecasts to 3 and 36 hours in order to compare the impact of nonlinearity on the adjoint and ensemble sensitivities. The observations are made at every grid point, and we present experiments with normal, larger random error and biased observations. In the normal case, the assumed observation-error standard deviation.2 does represent the actual error statistics for every observation, which is the nature run with Gaussian random perturbation. In this scenario, we test the ability of the adjoint method and the ensemble sensitivity method in representing the actual forecast error reduction, and the sensitivity of both methods to the linearity assumption by extending the forecast length by 6 hours. In the larger random error case, the observation at a single grid point (the 11 th grid point) has errors with a standard deviation of.8, four times larger than the other observations. However, in the data assimilation process, we still use the error standard deviation.2 to represent the error statistics for every observation, including the 11 th grid point. Such an experiment simulates real cases when some observations may have larger (or smaller) random errors than assumed in the data assimilation system. In addition, real observations may also have biases, something especially common when we assimilate satellite observations (e.g. Derber and Wu, 1998). Therefore, in the biased case, we include a bias equal to.5 in the observation at the 11 th grid point, but still assume that the observation is unbiased during data assimilation. In these experiments, we use 4 ensemble members in both the data assimilation and ensemble forecast processes, but the results (not shown) are similar even if we use 2 ensemble members only. In both the ensemble sensitivity method and the adjoint method, we use the Local Ensemble Transform Kalman Filter (LETKF: Ott et al., 24; Hunt et al., 27) data assimilation scheme. In the LETKF, the analysis at each grid point is computed independently from the other grid points, using all the observations within a certain distance of that analysis grid point. Due to this localization, each observation is used more than once during data assimilation. In the calculation of observation impact on the reduction of forecast errors based on Equation (12) or (13) and the adjoint method, we use the same localization scheme as in the LETKF and obtain the impact of an observation in different local patches. The final result for each observation is the average of the observation impact of this observation over all the patches where it was used (Liu, 27). The results are not sensitive to the local patch size as long as it is large enough to include several observations impacting the analysis at the central point. In Section 4.1, we show the sensitivity of the observation impact to localization by comparing results from different local patch sizes. We run each experiment for 75 analysis cycles, and present the time average statistics over the last 7 analysis cycles. In these experiments, we check whether our ensemble sensitivity method is comparable with the adjoint method of LB in assessing the observation impact on the forecast error reduction, and compare the ability of both methods to detect poor-quality observations. Unlike the variational assimilation approach, which requires the development of the adjoint of the assimilation system (LB; Zhu and Gelaro, 28), the calculation of observation impact in an Ensemble Kalman Filter (EnKF) does not need to develop the adjoint of the assimilation system because we can calculate the transpose of the Kalman gain matrix directly. Although this is true for both methods, the adjoint method still requires the adjoint of the model. We discuss the results case by case in the next section. Copyright 28 Royal Meteorological Society Q. J. R. Meteorol. Soc. 134: (28) DOI: 1.12/qj

5 OBSERVATION IMPACT WITHOUT ADJOINT MODEL IN ENSEMBLE KALMAN FILTER Results 4.1. Normal case Figure 2 shows the observation impact calculated from the adjoint method (black line with plus signs), the ensemble sensitivity method (grey line with closed circles) and the actual spatially summed forecast error reduction (black line with open circles) between the analysis cycles 57 and 578 for the normal case. It shows that the observation impact calculated from the ensemble sensitivity method is similar to the result from the adjoint sensitivity method, and that both methods succeed in capturing most of the actual forecast improvement due to assimilation of the observations at h. The time average of the absolute difference between the estimated and actual forecast error reduction is very similar:.131 is the difference between the impact estimated from ensemble sensitivity method and the actual forecast error reduction,.134 is the difference between the impact estimated from the adjoint method and the actual error reduction. Since the ensemble sensitivity method and the adjoint method use the same data assimilation scheme, observation increments are the same for both methods. The only difference is how these two methods estimate the sensitivity of the forecast error to the observation increments. In the adjoint method, the sensitivity is based on the adjoint model, which transfers the sensitivity in the forecast time to the analysis time, while the ensemble sensitivity utilizes the nonlinear forecast model to get the forecast perturbations and directly calculates the sensitivity (Equation (11)). In the scenario of a linear model, the adjoint method and the ensemble sensitivity method should give the same conclusion. The difference in the estimation of observation impact from these two methods could originate from the nonlinearity of the forecast evolution even for a 24-hour forecast. Since with increasing forecast length, error growth and perturbations become more nonlinear, we extend the forecast length by 6 hours to examine the sensitivity of both methods on the linearization assumption. Figure 3 shows that, for this extended forecast length, the observation impact based on the ensemble sensitivity method is closer to the actual forecast error reduction than that of the adjoint method. The lower sensitivity of the ensemble method on the forecast length may be due to the fact that the ensemble forecast perturbations are the difference between the nonlinear ensemble forecasts and the mean state, equivalent to evolving perturbations nonlinearly. (See also appendix B, where the derivation of the ensemble sensitivity does not rely on linearization.) Localization is an important component of any ensemble Kalman data assimilation scheme (e.g. Whitaker and Hamill, 22; Ott et al., 24), so we have to test its impact on the ensemble sensitivity method to estimate observation impact. As stated in Section 3, the same localization scheme used in the LETKF is used in the calculation of observation impact. Here, we examine the impact of the different local patch sizes on the observation impact estimation from the ensemble sensitivity method. Figure 4 compares the results from a large local patch (with a size of 39, i.e. with 19 grid points on each side of the central grid point) and a small local patch (size 13, i.e. only 6 grid points on each side). It shows that the observation impact estimated from the smaller local patch gives results almost indistinguishable from those from the larger local patch. We do find that the result gets worse when the local patch size is further reduced to 7, i.e. only 3 grid points on each side (not shown). In this case, however, we found that with a slight inflation of the forecast error, i.e. multiplying ε t 6 by 1.4, the results are similar to those from larger local patches (not shown) Larger random error case When the observation at the 11 th grid point has four times larger random error standard deviation than at the Figure 2. Evolution of the spatially summed forecast error differences and the estimated observation impact for the normal case between analysis cycle 57 and 578. The actual forecast error difference between the 24-hour and 3-hour forecasts: black line with open circles; the observation impact calculated from the adjoint method: black line with plus signs; the observation impact calculated from the ensemble method: grey line with closed circles; zero line: no impact. Figure 3. As Figure 2, except that this is the observation impact on the 3-hour forecast error reduction. The actual forecast error difference between the 3-hour and 36-hour forecasts: open circles; the observation impact calculated from the adjoint method: black line with plus signs; the observation impact calculated from the ensemble sensitivity method: grey line with closed circles; zero line: no impact. Copyright 28 Royal Meteorological Society Q. J. R. Meteorol. Soc. 134: (28) DOI: 1.12/qj

6 1332 J. LIU AND E. KALNAY Figure 4. Comparison of the observation impacts estimated from different local patch sizes between analysis cycle 57 and 578. Observation impact estimated from smaller local patch size: grey line with closed circles; observation impact estimated from larger local patch: black line with open triangle (not always visible because it generally coincides with the smaller patch); the actual forecast error reduction between the 24-hour and 3-hour forecasts: black line with open circles. other locations, both the ensemble sensitivity method and the adjoint method show that the assimilation of this observation increases the forecast error (Figure 5). The signal from ensemble sensitivity method at the 11 th observation location is larger than that of the adjoint method, but elsewhere the observation impacts calculated from both methods have similar values. By denying the 11 th observation, and comparing the 24- hour forecast error difference between full coverage and data-denial experiment, we found that the larger signal from the ensemble sensitivity method is closer to the observation impact calculated from the data denial experiment. However, such a data-denial experiment cannot exactly reflect the actual observation impact obtained in a full coverage because the background error covariances as well as the weights given to the other observations are changed when some observations are deleted. It is interesting to note that the observations at the locations adjacent to bad data points (e.g. 12 th grid point) have larger impact in improving the forecast accuracy than at the other locations due to the larger weights given to these observations through the larger background uncertainty estimated along with the LETKF. Snapshots of the spatially summed impact show that the observation impacts calculated from both methods reflect the actual forecast error difference (Figure 6) even when one of the observations has erroneous error statistics. Because of the poor quality of the observation at the 11 th observation location, the domain-summed observation impact has some large spikes (Figure 6) Biased case When the 11 th observation location has a bias, the ensemble sensitivity method and the adjoint method both indicate that the assimilation of this observation increases Figure 5. Time average (over the last 7 analysis cycles) of the contribution to the reduction of forecast errors from each observation location (four times larger random error at the 11 th grid point). Ensemble sensitivity method: black line with closed circles; the adjoint method: grey line with plus signs; zero line: the black solid line. Figure 6. Evolution of spatially summed forecast error differences and the observation impact from the larger random error case between analysis cycle 57 and 578. The notations are the same as in Figure 2. the forecast error (Figure 7). Again, the negative impact from assimilation of this observation makes the positive impact (reduction of forecast error) of assimilating the adjacent observations larger Computational costs We have shown that the ensemble sensitivity and the adjoint methods give similar observation impacts, and that both methods accurately reflect the actual forecast error reduction due to assimilation of the observations at the analysis time. Like the adjoint method, the ensemble sensitivity method can detect observations that have poor quality because of larger random error or bias. The signal detected by the ensemble sensitivity method is stronger and closer to the observation impact obtained from the Copyright 28 Royal Meteorological Society Q. J. R. Meteorol. Soc. 134: (28) DOI: 1.12/qj

7 OBSERVATION IMPACT WITHOUT ADJOINT MODEL IN ENSEMBLE KALMAN FILTER 1333 Figure 7. The biased case with a bias equal to.5 at 11 th grid point. The notations are the same as in Figure 5. data-denial experiment. Although the experiments were carried out with observations at every point, reducing the observation coverage does not change the conclusions (not shown). With respect to the computational cost, both the ensemble sensitivity method and the adjoint method have advantages and disadvantages. We note that the verifying analysis needed is not necessarily the analysis produced by the current system, and would therefore be available in a reanalysis or a parallel testing mode. If a verifying analysis at time t is available, then the ensemble sensitivity method can be calculated along with the data assimilation. The adjoint method requires the current analysis state, so it has to be calculated after the analysis even if another verifying analysis is available. The ensemble sensitivity method requires an ensemble of forecasts, and the adjoint method only needs one forecast, in addition to the backward integration of the adjoint model and the adjoint of the analysis scheme. Therefore, if the computations are performed serially, the ensemble sensitivity method would require longer wall-clock times than the adjoint method, but with parallel computations and a verifying analysis available, the ensemble method would be faster. In the past, monitoring the quality of observations has been based on observational increments, but in our experiments (not shown) we have found that the observation sensitivity approach is much more effective in detecting poor observations than the monitoring of observational increments. In this paper, following LB, we propose an ensemble sensitivity method to measure observation impact on the reduction of forecast errors due to assimilation of observations. Unlike the adjoint method by LB, the ensemble sensitivity method we propose does not need the adjoint of the numerical weather prediction model or of the data assimilation system, but it uses a nonlinear forecast model to evolve the forecast perturbations and directly calculates the sensitivity. We compare the ensemble sensitivity method to the adjoint method using the Lorenz 4-variable model. The results show that the ensemble sensitivity method and the adjoint method get similar observation impacts, and both reflect most of the actual forecast improvement. When there is some nonlinear growth of error perturbation, the ensemble sensitivity method is more robust, which may be due to the nonlinear evolution of the ensemble forecast perturbations. Both methods can detect the bad observations that are of poor quality, with either larger random errors or with biases, and the ensemble sensitivity method shows a stronger signal in such scenarios. Like the adjoint method by LB, this method can be applied in the observation quality control as well as to compare the importance of different-type observations. With the verifying analysis available, either from the same or a different analysis system, this method could be routinely calculated within an ensemble Kalman filter, thus providing a powerful tool to understand cases of forecast failure and to tune the observation error statistics. Acknowledgements We are very grateful to Kayo Ide, Shu-Chih Yang and Istvan Szunyogh for their suggestions and discussions, and to Ron Gelaro for his early encouragement of this research. Two anonymous reviewers provided us with very constructive and detailed comments that helped clarify several aspects of this study. This work was supported by NASA grants through NNG4GK29G, NNG4GK78A and NNX8AD4G. 5. Summary and conclusions The observations are the central information introduced into the forecast system during data assimilation. However, the quality and impact of observations is always different due to the magnitude of observation error, observation locations and model dynamics. Accurately monitoring the quality and impact of the observations assimilated in the system can help to delete the observations that routinely degrade the forecast, and to better use the observations that have larger impact on the forecast than other observations. Appendix A. Perturbation weights averaged over the ensemble The following derivation is based on Hunt et al. (27). We define a column vector of K ones: v = [ 1, 1 1 ] T. v is an eigenvector of ( P a ) 1 with eigenvalue (K 1) : ( P a ) 1 v ={(K 1)I + (Y b ) T R 1 Y b }v = (K 1)v because the sum of the columns of Y b is zero. Therefore, 1 K 1 ( P a ) 1 v = v. (A1) Copyright 28 Royal Meteorological Society Q. J. R. Meteorol. Soc. 134: (28) DOI: 1.12/qj

8 1334 J. LIU AND E. KALNAY In addition, the matrix of analysis weights is given by W a ={(K 1) P a }1/2, so that W a WaT = (K 1) P a. (A2) Multiplying both sides by the vector v, we get W a WaT v = v, so that v is an eigenvector of W a WaT matrix with eigenvalue equal to 1. Based on the properties of a symmetric matrix, v is also an eigenvector of W a matrix with the eigenvalue equal to 1: W a v = v. (A3) Since v is a column vector of K ones, K δwa(ji) = 1, where δw a(ji) is an element of the W a. Wa is a symmetric matrix; therefore, we have the following equation: δw a(ji) = δw a(ji) = 1. (A4) Appendix B. Derivation of the sensitivity of forecast errors to observations without linearization Unlike the derivation in Section, this derivation does not use linearization. Instead, it assumes the mean forecast at time t can be obtained with the same weights as obtained in the ensemble analyses. More details are in Liu (27). We first express the i th analysis ensemble member at time h as a linear combination of the six-hour ensemble forecasts initialized at 6 hr. Based on Hunt et al. (27), the i th analysis ensemble member is given by: x ai = xb 6 + Xb 6 wai, (B1) where w ai = wa + δwai is a vector of K dimensions, whose elements are w aji = w aj + δwaji,j = 1,,K. w a = K v is the mean weighting vector, and the perturbation weighting vector δw ai is the ith column of the K by K matrix W a ={(K 1) P a }1/2 with the elements. After expanding the terms on the right-hand side of Equation (B1) based on the definitions of each term and combining the same terms together, we get: δw aji x ai = K x bj 6 (waj wa + δw aji ), (B2) where w a = 1/K K waj, and waj is the j th element of the mean weight vector w a. We estimate the i th ensemble forecast at time t initialized at t = h with the ensemble forecasts initialized at t = 6 h using the same weights as at the analysis time: x i t = K x fj t 6 (waj wa + δw aij ) + ei t, (B3) where e i t represent the error from this approximation. We take an ensemble average of these forecasts initialized at t = h, so the mean forecast initialized at h can be written as: x f t = xf t 6 + K x fj t 6 δwaj + e t. (B4) In getting Equation (4), we use the relationship K δ w a(ji) = K δwa(ji) = 1. We note that, although very small, the error e t = x f t xf t 6 K xfj t 6 δwaj cannot be neglected in order to obtain accurate observation sensitivity. After expressing δw a with elements δwaj (j = 1, K ) in terms of the increments v, Equation (B4) can be written as x f t = xf t 6 + Xf t 6 δ K v + e t (B5) where X f t 6 = [ xf1 t 6 x fk t 6 ] is a matrix whose K columns are background ensemble forecasts. Note that this notation is different from that in Section 2. δ K is a K by P matrix whose elements are δ K jp and δ K jp = K jp K p, K p = 1/K K, K jp, K jp is an element of K by P matrix K. Based on Equation (B5), the cost function is written as: J = 1 2 (2eT t 6 + xf t xf t 6 )(xf t xf t 6 ) = 1 2 (2e t 6 + X f t 6 δ K v + e t ) T (X f t 6 δ K v + e t ). (B6) Since the error e t 6 is not correlated with the observations assimilated at t = h, the sensitivity of the forecast error to observations is written as: J v = (δ K T XfT t 6 )(e t 6 + X f t 6 δ K v + e t ) (B7) The sensitivity to the observations in Equation (B7) can be directly calculated based on the weighting function from data assimilation at h, the observation increment at h, and the ensemble forecast initialized at 6 h. This derivation neglects the correlation between e t and the observations assimilated at t = h. If we place e t from (B4) into Equation (B7) and linearize as in Section 2, we get Equation (11). Both (B7) and (11) yield indistinguishable results. References Ancell B, Hakim GJ. 27. Comparing adjoint- and ensemblesensitivity analysis with applications to observation targeting. Mon. Weather Rev. 135: Bishop CH. 27. Introduction to data assimilation research at NRL and flow adaptive error covariance localization. UMD workshop.pdf. Copyright 28 Royal Meteorological Society Q. J. R. Meteorol. Soc. 134: (28) DOI: 1.12/qj

9 OBSERVATION IMPACT WITHOUT ADJOINT MODEL IN ENSEMBLE KALMAN FILTER 1335 Derber JC, Wu W-S The use of TOVS cloud-cleared radiances in the NCEP SSI analysis system. Mon. Weather Rev. 126: Hunt BR, Kostelich EJ, Szunyogh I. 27. Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter. Physica D 23: Joiner J, Poli P, Frank D, Liu HC. 24. Detection of cloud-affected AIRS channels using an adjacent-pixel approach. Q. J. R. Meteorol. Soc. 13: Langland RH, Baker NL. 24. Estimation of observation impact using the NRL atmospheric variational data assimilation adjoint system. Tellus 56A: Le Marshall J, Jung J, Derber J, Chahine M, Treadon R, Lord SJ, Goldberg M, Wolf W, Liu HC, Joiner J, Woollen J, Todling R, van Delst P, Tahara Y. 26. Improving global analysis and forecasting with AIRS. Bull. Am. Meteorol. Soc. 87: Liu J. 27. Applications of the LETKF to adaptive observations, analysis sensitivity, observation impact and the assimilation of moisture. Ph.D. thesis, University of Maryland. Lorenz EN, Emanuel KA Optimal sites for supplementary observations: Simulation with a small model. J. Atmos. Sci. 55: Ott E, Hunt BR, Szunyogh I, Zimin AV, Kostelich EJ, Corazza M, Kalnay E, Patil DJ, Yorke JA. 24. A local ensemble Kalman filter for atmospheric data assimilation. Tellus 56A: Torn RD, Hakim GJ. 28. Ensemble-based sensitivity analysis. Mon. Weather Rev. 136: Whitaker JS, Hamill TM. 22. Ensemble data assimilation without perturbed observations. Mon. Weather Rev. 13: Zapotocny TH, Menzel WP, Nelson III JP, Jung JA. 22. An impact study of five remotely sensed and five in situ data types in the Eta Data Assimilation System. Weather and Forecasting 17: Zhu Y, Gelaro R. 28. Observation sensitivity calculations using the adjoint of the Gridpoint Statistical Interpolation (GSI) analysis system. Mon. Weather Rev. 136: Copyright 28 Royal Meteorological Society Q. J. R. Meteorol. Soc. 134: (28) DOI: 1.12/qj