CONVERGENCE OF THE SQUARE ROOT ENSEMBLE KALMAN FILTER IN THE LARGE ENSEMBLE LIMIT

Transcription

1 COVERGECE OF THE SQUARE ROOT ESEMBLE KALMA FILTER I THE LARGE ESEMBLE LIMIT EVA KWIATKOWSKI AD JA MADEL Abstract. Ensemble filters implement sequential Bayesian estimation by representing the probability distribution by an ensemble mean and covariance. Unbiased square root ensemble filters use deterministic algorithms to produce an analysis (posterior) ensemble with prescribed mean and covariance, consistent with the Kalman update. This includes several filters used in practice, such as the Ensemble Transform Kalman Filter (ETKF), the Ensemble Adjustment Kalman Filter (EAKF), and a filter by Whitaker and Hamill. We show that at every time index, as the number of ensemble members increases to infinity, the mean and covariance of an unbiased ensemble square root filter converge to those of the Kalman filter, in the case a linear model and an initial distribution of which all moments exist. The convergence is in L p and the convergence rate does not depend on the model dimension. The result holds in the infinitely dimensional Hilbert space as well. Key words. Data assimilation, L p laws of large numbers, Hilbert space, ensemble Kalman filter AMS subject classifications. 60F25, 65C05, 65C35 1. Introduction. Data assimilation uses Bayesian estimation to incorporate observations into the state of a model of a physical system, called forecast, to get a better estimate, called analysis. The resulting analysis is used to initialize the next run of the model, producing the next forecast, which is subsequently used in the next analysis, and the process thus continues. Data assimilation is widely used, e.g., in geosciences [15]. The Kalman filter represents probability distributions by the mean and covariance. It is an efficient method for the case when the probability distributions are close to Gaussian. However, in applications, the dimension of the state is large, and it is not feasible to even store the state covariance exactly. Ensemble Kalman filters are variants of Kalman filters in which the state probability distribution is represented by an ensemble of states, and the state mean and covariance are estimated from the ensemble [10]. The dynamics of the model, which could be nonlinear in this case, are applied to each ensemble member to produce the forecast estimate. Simulation studies have shown the ensembe filters are able to efficiently handle nonlinear dynamics and state spaces of high dimension. The major differences between ensemble Kalman filters are in the way that the analysis ensemble is produced from the forecast and the data. The analysis ensemble can be formed in a stochastic manner deterministic manner. The purpose of this paper is to examine ensemble Kalman filters that use a deterministic method to produce an analysis ensemble with exactly the desired statistics. Such filters are called unbiased square root filters, because the ensemble mean equals the prescribed mean, and computation of the analysis to match the prescribed ensemble covariance leads to taking the square root of a matrix. This includes several filters used in practice, such as the Ensemble Transform Kalman Filter (ETKF) [4, 13, 28], the Ensemble Adjustment Kalman Filter (EAKF) [2], and a filter by Whitaker and Hamill [29]. Criteria necessary for a ensemble square root filter to be unbiased are discussed in [20, 27]. The algebra of square root ensemble filters relies on the use of sample covariance, which does not support localization by tapering [12], but a localization Department of Mathematical and Statistical Sciences, University of Colorado Denver 1

2 can be achieved by other means, such as sequential assimilation of observations locally [2, 13]. An important question for ensemble filters is a law of large number as the size of the ensemble grows to infinity. In [19, 21], it was proved independently for the version of the ensemble Kalman filter with radomized data from [6], that the ensemble mean and covariance converge to those of the Kalman filter, as the number of ensemble members grows to infinity. Both analyses obtain convergence in all L p, 1 p <, but the convergence results are not independent of the space dimensions. The paper [21] relies on the fact that ensemble members are exchangeable and it uses uniform integrability, which does not provide a rate of convergence, while [19] uses a mean field approach and stochastic inequalities for the random matrices and vectors entry by entry to obtain convergence with the usual Monte-Carlo rate 1/. Here, we show that at every time index, as the number of ensemble members increases to infinity, the mean and covariance of an unbiased ensemble square root filter converge to those of the Kalman filter, in the case of a linear model and an initial distribution of which all moments exist. The convergence is in all L p, with the usual rate 1/, the constant does not depend on the model state dimension, and the result holds in the infinitely dimensional case as well. The constants in the estimate are constructive and depend only on the model and the data, namely the norms of the model operators and of the inverse of the data covariance, and of the vectors given. The analysis builds on some of the tools developed in [19], and extends them to obtain bounds on the operators involved in the formulation of square root ensemble filters, independently of the state space and data dimension, including in the infinitely dimensional case. The square root esemble filters are deterministic, which avoids technical complications associated with data perturbation in infinite dimension. Convergence of ensemble filters with data perturbation independent of dimension, including infinite, will be studied elsewhere. All of these estimates hold for arbitrary but fixed time; for estimates of the behavior of the ensemble Kalman filter with time, see [16]. The main idea of the analysis is simple: by the law of large numbers, the sample mean and the sample covariance of the initial ensemble converge to the mean and covariance of the background distribution. Every analysis step is a continuous mapping of the mean and the covariance, and the convergence in the large sample ensemble limit follows. The analysis quantifies this argument. The paper is organized as follows: in Section 2 we introduce the notation and review some background concepts. Section 3 contains statement of the Kalman filter and the unbiased square root filter, and shows some basic properties, which are needed later. In section 4.4, we derive the L p laws of large numbers, needed for the convergence of the statistics of the initial ensemble. In Section 5, we show the continuity properties the transformation of the statistics from one time step to the next. Finally, Section 6 presents the main result. 2. otation and preliminaries. We will work with random elements with values in a Hilbert space V. Readers interested in finite dimension only can think of random vectors in V = R n. All notations and proofs are presented in a way that applies in R n, as well as in a Hilbert space Finite dimensional case. Vectors u R n are columns, and the inner product is u, v = u v where u denotes transpose. We will use the notation [V ] for the space of all n by n matrices. For a matrix M, M denotes the transpose, and M stands for the spectral norm. We will also need the Hilbert-Schmidt norm (more 2

3 commonly called Frobenius morm) of matrices, induced by the corresponding inner product of two n by n matrices, A HS = n i,j=1 a 2 ij 1/2 = A, A 1/2 HS, A, B HS = n The Hilbert-Schmidt norm dominates the spectral norm, i,j=1 a ij b ij. A A HS, (2.1) for any matrix A. The notation A 0 means that A is symmetric and positive semidefinite, A > 0 means positive definite. For Q 0, X (u, Q) means that random vector X has the normal distribution on R n, with mean u and covariance matrix Q. For vectors u, v R n, their tensor product is the n by n matrix, It is easy to see that u v = uv [V ]. u v HS = u v, (2.2) because n n u v 2 HS = n (u i v j ) 2 = u 2 i j=1 j=1 n vj 2 = u 2 v Hilbert space case. Readers interested in finite dimension only should skip this section. In the general case, V is a separable Hilbert space equipped with inner product u, v and the norm u = u, u 1/2. The space of all bounded operators from Hilbert space U to Hilbert space V is [U, V ], and [V ] = [V, V ]. For a bounded linear operator A [U, V ], A [V, U] denotes the adjoint operator, and A is the operator norm. The Hilbert-Schmidt norm of a linear operator on V is defined by A HS = A, B 1/2 HS, A, B HS = Ae n, Be n n=1 where {e n } is any complete orthonormal sequence; the values do not depend on the choice of {e n }. An operator on V is called a Hilbert-Schmidt operator if A, B HS <, and HS (V ) is the space of all Hilbert-Schmidt operators on V. The Hilbert- Schmidt norm again dominates the spectral norm, so (2.1) holds, and HS (V ) [V ]. The importance of HS (V ) for us lies in the fact that HS (V ) is a Hilbert space, while [V ] is not. The notation A 0 for operator A [V ] means that A symmetric, A = A, and positive semidefinite, Au, u 0 for all u V. The notation A > 0 means here that A is symmetric and bounded below, that is, Au, u α u 2 for all u V and some α > 0. In particular, if A > 0, then A 1 [V ]. An operator A 0 is trace class if Tr A <, where Tr A is the trace of A, defined by Tr A = Ae n, e n. If A 0 is trace class, then A is Hilbert-Schmidt, because A HS Tr A. n=1 3

4 by For vectors u, v V, their tensor product u v [V ] is now a mapping defined u v : w V u v, w, and the proof of (2.2) becomes x y 2 HS(H) = (x y) e i 2 = x y, e i 2 = x 2 y, e k 2 = x 2 y 2, from Bessel s equality. The mean of a random element E (X) V is defined by E (X), y = E ( X, y ) for all y V. The mean of the tensor product of two random elements is defined by E (X Y ) w, y = E( X, y Y, w ) for all w, y V. Covariance is defined below (2.3). The covariance of a random element X exists when the second moment E( X 2 ) <, and the covariance is a trace class operator. On the other hand, if Q 0 is trace class, the normal distribution (u, Q) can be defined as a probability measure on V, consistently with the finite dimensional case. Just as in the finite dimensional case, if X (u, Q), then X has all moments E ( X p ) <, 1 p <. See [8, 17, 18] for further details Common definitions and properies. The rest is the same regardess if V = R n or V is a general Hilbert space. To unify the nomenclature, matrices are identified with the corresponding operators of matrix-vector multiplication. Covariance of a random vector X in V is defined by Cov(X) = E((X E (X)) (Y E (Y ))) = E(X Y ) E(X) E(Y ), (2.3) if it exists. For 1 p <, the space of all random elements X with values in V and finite moment E ( X p ) < is denoted by L p (V ), and it is equipped with the norm X p = (E( X p )) 1/p. If 1 p q < and X L q (V ), then X L p (V ) and X p X q. If X, Y L 2 (V ) are independent, then Cov(X, Y ) = 0 and E ( X, Y ) = 0. The following lemma will be used repeatedly in obtaining L p estimates. It is characteristic of the present approach that higher order norms are used to bound lower order norms. Lemma 2.1 (L p Cauchy-Schwarz inequality). If U, V L 2p (V ) and 1 p <, then U V p U 2p V 2p. Proof. By the Cauchy-Schwartz inequality in L 2 (V ), applied to the random variables U p and V p, which are in L 2 (R), U V p p = E ( U p V p ) ( E U 2p) 1/2 ( E V 2p ) 1/2 ( (E U 2p = ) 1/2p ( E V 2p ) ) 1/2p p = X p 2p Y p 2p. 4

5 Taking the pth root of both sides yields the desired inequality. An ensemble X consists of random elements X i, i = 1,...,. The ensemble mean is denoted by X or E (X ), and is defined by X = E (X ) = 1 X i. (2.4) The ensemble covariance is denoted by Q or C (X ), and is defined by Q = C (X ) = 1 (X i X ) (X i X ) = E (X X ) E (X ) E (X ), (2.5) where X X = [X 1 X 1,..., X X ]. For A, B [V ], A B means that A and B are symmetric, and B A 0. It is well known from the spectral theory that 0 A B implies A B. 3. Definitions and basic properties of the algorithms. In the Kalman filter, the probability distribution of the state of the system is described by its mean and covariance. We first consider the analysis step, which uses Bayes theorem to incorporate an observation into the forecast state and covariance to produce the analysis state and covariance. The system state X is an R n valued random vector. Denote by X f R n the forecast state mean, Q f R n n the forecast state error covariance, and X a R n and Q a R n n the analysis state mean and covariance, respectively. The observation data vector is d R m, where d HX (0, R), with H R n m the linear observation operator, and R R m m the observation error covariance. Given the forecast mean and covariance, the analysis mean and covariance are chosen as the best fit to the data in the sense of least squares, so that for all X R n, J(X) = (X X f ) (Q f ) 1 (X X f ) + (d H(X)) R 1 (d H(X)) = (X X a ) (Q a ) 1 (X X a ) + const. (3.1) The quantities that create the equality between the two formulations of the least squares functional J(X) are the Kalman Filter analysis mean and covariance, where X a = X f + K(d H(X f )) (3.2) Q a = Q f Q f H (HQ f H + R) 1 HQ f = Q f KHQ f (3.3) K = Q f H (HQ f H + R) 1 (3.4) is the Kalman gain matrix. See, e.g., [13, Sec. 2.1] or [1, 14, 26]. For future reference, we introduce the operators K(Q) = QH (HQH + R) 1 (3.5) B(X, Q) = X + K(Q)(d HX), (3.6) A (Q) = Q QH (HQH + R) 1 HQ (3.7) = Q K(Q)HQ, (3.8) 5

6 which evaluate the Kalman gain, analysis mean, and the analysis covariance, respectively, in the Kalman filter equations (3.2) (3.4). We are now ready to state the complete Kalman filter for reference. The superscript (k) denotes quantities at time step k. However, to simplify notation, we drop the time superscript (k) when we are concerned with one step only. Algorithm 3.1 (Kalman filter). Suppose that the model M (k) at each time k > 0 is linear, M (k) (X) = M (k) X + b (k), and initial mean X (0) and covariance B = Q (0),a of the state are given. At time k, the analysis mean and covariance from the previous time k 1 are advanced by the model, X (k),f = M (k) X (k 1),a + b (k), (3.9) Q (k),f = M (k) Q (k 1),a M (k), (3.10) followed by the analysis step, which incorporates the observation d (k), where H (k) X (k) d (k) has zero mean and covariance R (k), and it gives the analysis mean and covariance X (k),a = B (k) (X (k),f, Q (k),f ), (3.11) Q (k),a = A (k) (Q (k),f ), (3.12) where B (k) and A (k) are defined by (3.6) and (3.8), respectively, with d, H, and R, at time k. In many applications, the state dimension n of the system is large and computing or even storing the exact covariance of the system state is computationally impractial. Ensemble Kalman filters address this concern. Ensemble Kalman filters use a collection of state vectors, called an ensemble, to represent the distribution of the system state. This ensemble will be denoted by X, comprised of random elements X i in V, i = 1,.... The ensemble mean and ensemble covariance, defined by (2.4) and (2.5), are denoted by X and C, respectively, while the Kalman Filter mean and covariance are denoted without subscripts, as X and Q. There are many ways to produce an analysis ensemble corresponding to the Kalman filter algorithm. Given a forecast ensemble X f, ensemble square root filters produce an analysis ensemble X a such that (3.11) (3.12) are satisfied for the sample mean and covariance, in the sense that X a = B(X f, Q f ) = X f + K(Q f )(d H(X f )) (3.13) Q a = A(Q f ) = Q f K(Q f )HQ f. (3.14) The initial ensemble X (k),a is generated around initial state X (0) as a sample from a distribution with mean X (0) and covariance B. The matrix B is called the background covariance. The background covariance does not need to be stored explicitly. Rather, it is used in a factored form [5, 11], and only multiplication of L times a vector is needed, B = LL T, (3.15) X (0) i = X (0) + LY i, Y i (0, I). 6

7 For example, a sample covariance of the form (3.15) can be created from historical data. To represent better the dominant part of the background covariance, L can consist of approximate eigenvectors for the largest eigenvalues, obtained by a variant of the power method [15]. Another popular choice is L = T S, where T is a transformation, such as FFT or wavelet transform, which requires no storage of the matrix entries at all, and S is a sparse matrix [9, 24, 25]. Covariance of Fourier transform of a stationary random field is diagonal, so even an approximation with diagonal matrix S is useful, and computationally very inexpensive. Starting from the initial ensemble, the unbiased square root ensemble filter algorithm proceeds as follows. Algorithm 3.2 (Unbiased square root ensemble filter). Let the initial ensemble X (0),a at time k is given. At time k > 0, the analysis ensemble X (k 1),a i is advanced by the model X (k),f i = M (k) (X (k 1),a i ), i = 1,...,, (3.16) resulting in forecast ensemble X (k),f with the ensemble mean and covariance X (k),f The analysis step creates an ensemble X (k),a d (k) H (k) X (k) = E (X (k),f ), Q(k),f = C (X (k),f ). which incorporates the observation that has zero mean and the covariance R (k). The analysis ensemble is constructed (in a manner to be determined by the specific method) to have the ensemble mean and covariance given by E (X (k),a C (X (k),a ) = B (k) (X (k),f, Q (k),f ), (3.17) ) = A (k) (Q (k),f ), (3.18) where B (k) and A (k) are defined by (3.6) and (3.8) with d, H, and R, at time k. The Ensemble Adjustment Kalman Filter (EAKF, [2]), the filter by Whitaker and Hamill [29], and the Ensemble Transform Filter (ETF, [4]) and its variants, the Local Ensemble Transform Filter (LETKF, [13]) and its revision [28], satisfy properties (3.17) and (3.18). See also [20, 27]. These algorithms satisfy the conditions in the following lemma, which describes their basic properties. Lemma 3.3. Suppose that for each k 1, the model M (k) is linear, M (k) (X) = M (k) X + b (k). Then the ensemble mean and covariance X (k),a = E (X (k),a ), Q(k),a of the ensembles X (k),a, generated by Algorithm 3.2, satisfy = C (X (k),a ), k = 0, 1, 2,... (3.19) X (k),f Q (k),f X (k),a Q (k),a = M (k) X (k 1),a + b (k), (3.20) = M (k) Q (k 1),a M (k) (3.21) = B (k) (X (k),f, Q (k),f ) (3.22) = A (k) (Q (k),f ). (3.23) Proof. For any k, from the definition of the model M (k), X (k),f i = M (k) X (k 1),a i + b (k), i = 1,...,, (3.24) 7

8 and (3.20) and (3.21) follow. ow (3.22) and (3.23) follow from the definitions of the ensemble mean and covariance in (3.19), and from the properties (3.17) and (3.18) of the construction of the analysis ensemble. Using Lemma 3.3, we can now compare the transformations of the ensemble mean and covariance in Algorithms 3.1 and 3.2 in the case of a linear model: Kalman Filter X (k),f = M (k) X (k 1),a + b (k) Q (k),f = M (k) Q (k 1),a M (k) X (k),a = B (k) (X (k),f ) Q (k),a = A (k) (Q (k),f ) X (k),f Unbiased Square Root Ensemble Filter Q (k),f = M (k) X (k 1),a = M (k) Q (k 1),a = B (k) (X (k),f X (k),a Q (k),a + b (k) M (k) ) = A (k) (Q (k),f ) (3.25) It is clear from (3.25) that the Kalman filter mean and covariance and those of the unbiased square root ensemble filter evolve in the same way. Therefore, all that is needed for the convergence of the unbiased square root ensemble filter is the convergence of the initial ensemble mean and covariance, X (0),a X (0) and Q (0),a B, as, and the continuity of the operators A (k) and B (k), in a suitable statistical sense. Similarly as in [19, 21], we will work with convergence in all L p, 1 p <. 4. L p laws of large numbers. To prove convergence in L p of the initial ensemble mean and covariance to the mean and covariance of the background distribution, we need corresponding laws of large numbers L p convergence of sample mean. The L 2 law of large numbers for X 1,..., X L 2 (V ) i.i.d. is classical, E (X ) E (X 1 ) 2 1 X 1 E (X 1 ) 2 2 X 1 2. (4.1) The proof relies on the expansion (assuming E (X 1 ) = 0 without loss of generality), E (X ) 2 2 = 1 X i, 1 X i = 1 2 E ( X i, X k ) (4.2) = 1 2 k=1 E ( X i, X i ) = 1 E ( X 1, X 1 ) = 1 X which yields E (X ) 2 X 1 2 /. To obtain L p laws of large numbers for p 2, the equality (4.2) needs to be replaced by the following. Lemma 4.1. (Marcinkiewicz-Zygmund inequality) If 1 p < and Y i L p (V ), E(Y i ) = 0 and E( Y i p ) <, i = 1,...,, then ( p) ( ) p/2 E Y i B p E Y i 2 (4.3) where B p depends on p only. Proof. For the finite dimensional case, see [23] or [7, p. 367]. In the infinitely dimensional case, note that a separable Hilbert space is Banach space of Rademacher 8

9 type 2 [3, p. 159], which implies (4.3) for any p 1 [30, Proposition 2.1]. All infinitely dimensional separable Hilbert spaces are isometric, so the same B p works for any of them. The Marcinkiewicz-Zygmund inequality allows to prove a variant of the weak law of large numbers in L p norms, similarly as in [7, Corollary 2, page 368]. ote that Marcinkiewicz-Zygmund inequality does not hold in Banach spaces in general, so it is important that V = R n or V sepearable Hilbert space, as assumed throughout. Theorem 4.2. Let X k L p (V ) be i.i.d. and p 2. Then, E (X ) E (X 1 ) p C p X 1 E (X 1 ) p 2C p X 1 p, (4.4) where C p depends on p only. Proof. If p = 2, the statement becomes (4.1). Let p > 2, and consider first the case E (X 1 ) = 0. By Hölder s inequality with the conjugate exponents p 2 p and 2 p, ( ( X i 2 = 1 X i 2) (p 2)/p ( ) 2/p (1 p/p 2)) ( X i 2) p/2 ( 2/p ( p/2) = ( X (p 2)/p i 2) ) 2/p = (p 2)/p X i p. (4.5) Using the Marcinkiewicz-Zygmund inequality (4.3) and (4.5), ( p) ( ) p/2 E X i B p E X i 2 ( ) 2/p p/2 B p E (p 2)/p X i p ( ) B p p/2 1 E X i p = B p p/2 E ( X 1 p ), (4.6) because p p 2 2 p = p 2 1, and the X i are identically distributed. Taking the p-th root of both sides of (4.6) yields X i Bp p 1/p 1/2 X 1 p, and the first inequality in (4.4) follows. The general case when E (X 1 ) 0 follows from the triangle inequality L p convergence of sample covariance. For the convergence ensemble covariance, we use the L p law of large numbers in Hilbert-Schmidt norm, because HS (V ) is a separable Hilbert space, while [V ] is not even a Hilbert space. Convergence in the norm of [V ], the operator norm, then follows from (2.1). See also [19, Lemma 22.3] for a related result using entry-by-entry estimates. 9

10 Theorem 4.3. Let X 1,..., X L 2p (V ) be i.i.d. and p 2. Then, ( ) 2C p C (X ) Cov (X 1 ) HS p + 4C2 2p X 1 2 2p. (4.7) where C p is a constant which depends on p only; in particular, C 2 = 1. Proof. Without loss of generality, let E (X 1 ) = 0. Then C (X X ) Cov (X 1 X 1 ) (4.8) = (E (X X ) E (X 1 X 1 )) + E (X ) E (X ), where X X = [X 1 X 1,..., X X ]. For the first term in (4.8), the L p law of large numbers (4.4) in HS (V ) yields E ( E (X k X k ) E (X 1 X 1 ) p HS )1/p 2C p E ( X 1 X 1 p HS )1/p = 2C ( p E X 1 2p) 1/p 2C = p X 2 2p, using (2.2). For the second term in (4.8), we use again (2.2) to get E (X ) E (X ) HS = E (X k ) 2, and the L p law of large numbers in V yields E ( E (X ) E (X ) p HS )1/p = E ( E (X ) 2p) 1/p = E (X ) 0 2 2p ( 2C2p X 1 2p ) 2. It remains to use the triangle inequality for the general case. 5. Continuity estimates. A fundamental part of our analysis are continuity estimates for the operators A and B, which bring the forecast statistics to the analysis statistics. Our general strategy is to first derive pointwise estimates, which apply to every realization of the random elements separately. In the next section, we will integrate them to get the corresponding L p estimates. We will prove the following estimates for general covariances Q and P, with the state covariance and sample covariance of the filters in mind Pointwise bounds. The first estimate is the continuity the Kalman gain matrix (3.4) as a function of the forecast covariance in the next lemma and its corollary. See also [19, Proposition 22.2]. Lemma 5.1. If R > 0 and P, Q 0, then ( K(Q) K(P ) Q P H R min { P, Q } H 2). Proof. Since R > 0 and P, Q 0, K(Q) and K(P ) are defined in (3.5). For A, B 0, we have the identity, (I + A) 1 (I + B) 1 = (I + A) 1 (B A)(I + B) 1, (5.1) 10

11 which is easily verified by multiplication by I + A on the left and I + B on the right, and (I + A) 1 (I + B) 1 B A, (5.2) which follows from (5.1) using the inequalities (I + A) 1 1, (I + B) 1 1, because A, B 0. ow write (HQH + R) 1 = R 1/2 (R 1/2 HQH R 1/2 + I) 1 R 1/2 using the symmetric square root R 1/2 of R. By (5.2) with A = R 1/2 HQH R 1/2 and B = R 1/2 HP H R 1/2, we have that (HQH + R) 1 (HP H + R) 1 R 1/2 H Q P H R 1/2 = Q P H 2 R 1. (5.3) Using the triangle inequality and the fact that HQH + R R, we have (HQH + R) 1 R 1. (5.4) Using (5.3), (5.4), and the definition of the operator K from (3.5), we have K(Q) K(P ) = QH (HQH + R) 1 P H (HP H + R) 1 = QH (HQH + R) 1 P H (HQH + R) 1 + P H (HQH + R) 1 P H (HP H + R) 1 = (Q P ) H (HQH + R) 1 P H ( (HQH + R) 1 (HP H + R) 1) Q P H R 1 + P H Q P H 2 R 1 ( = Q P H R P H 2). Swapping the roles P and Q yields K(Q) K(P ) Q P H R 1 (1 + Q H 2 ), which completes the proof. A pointwise bounds on the Kalman gain follows. Corollary 5.2. If R > 0 and Q 0, then Proof. Use Lemma 5.1 with P = 0. Corollary 5.3. If R > 0 and Q 0, then K(Q) Q H R 1. (5.5) B(X, Q) X + Q H R 1 ( d + H X ). (5.6) Proof. By the definition of operator B in (3.6), the pointwise bound on the Kalman gain (5.5), and the triangle inequality, B(X, Q) = X + K(Q)(d HX) X + Q H R 1 ( d + H X ). The pointwise continuity of operator A also follows from Lemma 5.1. Lemma 5.4. If R > 0 and P, Q 0, then A(Q) A(P ) Q P ( 1 + H 2 R 1 ( Q + P + H 2 P Q )). 11

12 Proof. Since R > 0 and P, Q 0, it follows that K(Q), K(P ), A(Q), and A(P ) are defined. From the definition of A (3.8), Lemma 5.1, and Corollary 5.2, A(Q) A(P ) = (Q K(Q)HQ) (P K(P )HP ) = Q P + K(P )HP K(Q)HQ = Q P + K(P )HP K(P )HQ + K(P )HQ K(Q)HQ Q P + Q P K(P ) H + K(Q) K(P ) H Q Q P + Q P P H R 1 H + Q P H R 1 (1 + P H 2 ) H Q = Q P ( 1 + H 2 R 1 ( Q + P + H 2 P Q )). Remark 5.5. The choice of P in min{ P, Q } in the proof Lemma 5.4 was made to preserve symmetry. Swapping the roles of P and Q in the proof gives a sharper, but a more complicated bound A(Q) A(P ) Q P ( ( { 1 + H 2 R 1 Q + P + H 2 min P 2, Q P, Q 2})). Instead of setting P = 0 above, we can get a well-known sharper pointwise bound on A(Q) directly from (3.7). The proof is written in a way suitable for our generalization. Lemma 5.6. Let R > 0 and Q 0. Then and Proof. By the definition of operator A in (3.7), 0 A(Q) A, (5.7) A(Q) Q. (5.8) A (Q) = Q QH (HQH + R) 1 HQ Q, because QH (HQH + R) 1 HQ 0. It remains to show that A(Q) 0. ote that for any A, and since R > 0, A A + R A A, (A A + R) 1/2 A A (A A + R) 1/2 I. Since for B 0, B I is the same as the spectral radius ρ (B) 1, and, for any C and D, ρ (CD) = ρ (DC), we have finally Using (5.9) with A = Q 1/2 H gives A (A A + R) 1 A I. (5.9) Q 1/2 H (HQH + R) 1 HQ 1/2 I, 12

13 and, consequently QH (HQH + R) 1 HQ Q, which gives A(Q) 0. Since B = ρ (B) for any symmetric B, (5.8) follows from (5.7). The pointwise continuity of operator B follows from Lemma 5.1 as well. Lemma 5.7. Let R > 0, and Q, P 0. Then, B(X, Q) B(Y, P ) X Y ( 1 + H 2 R 1 Q ) + (5.10) Q P H R 1 (1 + P H 2 )( d HY ). Proof. Estimating the difference and using the pointwise bound on the Kalman gain (5.5) and the pointwise continuity of the Kalman gain from Lemma 5.1, K(Q)(d HX) K(P )(d HY ) Using the triangle inequality, we have which is (5.10). = K(Q)(HY HX) + (K(Q) K(P ))(d HY ) K(Q) H X Y + K(Q) K(P ) d HY Q H R 1 H X Y + Q P H R 1 (1 + P H 2 )( d + HY ). B(X, Q) B(Y, P ) = X + K(Q)(d HX) (Y + K(P )(d HY )) X Y + Q H R 1 H X Y Q P H R 1 (1 + P H 2 )( d + HY, 5.2. L p bounds. Using the pointwise estimate on the continuity of A, we can now estimate continuity in L p. We will need the result only with one of the arguments random and the other one constant (i.e., non random), which simplifies its statement and proof. This is because the application of these estimates will be the ensemble sample covariance, which is random, and the state covariance, which is constant. Lemma 5.8. Let Q be a random operator such that Q 0 almost surely (a.s.), let P 0 be constant, and let R > 0. Then, for all 1 p <, where A(Q) A(P ) p (1 + c P ) Q P p + c(1 + c P ) Q 2p Q P 2p, (5.11) c = H 2 R 1. (5.12) Proof. From Lemma 5.4, the triangle inequality, Lemma 2.1, and recognizing that P is constant, A(Q) A(P ) p Q P (1 + c Q + c P + c 2 Q P ) p Q P p + c Q Q P p + c P Q P p + c 2 Q P Q P p Q P p + c Q 2p Q P 2p + c P Q P p + c 2 P Q 2p Q P 2p = (1 + c P ) Q P p + c(1 + c P ) Q 2p Q P 2p. 13

14 Instead of setting P = 0 above, we get a better bound on A(Q) p directly. Lemma 5.9. Let Q be a random operator such that Q 0 a.s., and R > 0. Then, for all 1 p <, A(Q) p Q p. (5.13) Proof. From (5.8), it follows that E ( A(Q) p ) E ( Q p ). Using the pointwise estimate on the continuity of the operator B, we estimate its continuity in L p. Again, we keep the arguments of one of the terms constant, which is all we will need, resulting in a simplification. Lemma Let Q and X be random operators, and P and Y be constant. Let Q 0 a.s., P 0, and R > 0. Then, B(X, Q) B(Y, P ) p X Y p + H 2 R 1 Q 2p X Y 2p + Q P p H R 1 (1 + P H 2 ) d HY. Proof. Applying the L p norm to the pointwise bound (5.10), using the triangle inequality, recognizing that P and Y are constant, and applying the Cauchy-Schwarz inequality (Lemma 2.1) to the rest, we get B(X, Q) B(Y, P ) p = X Y ( 1 + H 2 R 1 Q ) + Q P H R 1 (1 + P H 2 ) d HY p X Y p + H 2 R 1 Q X Y p + Q P H R 1 (1 + P H 2 ) d HY p X Y p + H 2 R 1 Q 2p X Y 2p + Q P p H R 1 (1 + P H 2 ) d HY. 6. Convergence of the unbiased squared root filter. By the law of large numbers, the sample mean and the sample covariance of the initial ensemble converge to the mean and covariance of the background distribution. Every analysis step is a continuous mapping of the mean and the covariance, and the convergence in the large ensemble limit follows. The theorem below quantifies this argument. Theorem 6.1. Assume that the state space V is finite dimensional or separable Hilbert space, the initial state, denoted X (0) has distribution on V such that all moments exist, E( X (0) p ) < for every 1 p <, the initial ensemble X (0) is an i.i.d. sample from this distribution, and the model is linear, M (k) (X) = M (k) X +b (k) for every time k. Then, for all 1 p < and all k, the ensemble mean X (k),a and covariance Q (k),a from the unbiased square root ensemble filter (Algorithm 3.2) converge to the mean X (k),a and covariance Q (k),a from the Kalman filter (Algorithm 3.1), respectively, in L p as with the convergence rate 1/. Specifically, X (k),a X (k),a p a(k) p, Q (k),a Q (k),a p b(k) p, (6.1) for all = 1, 2,..., where the constants a (k) p and b (k) p depend only on p, k, on the norms M (l), H (l), R (l) 1, d (l), l k, and X (0) and Q (0),a. Proof. Base step. For k = 0 and p 2, (6.1) follows immediately from the L p laws of large numbers in L p, (4.4) and (4.7). For 1 p < 2, it is sufficient to note 14

15 that the L p norm is dominated by the L 2 norm. We will use const (p, k) to denote a generic constant which depends on p, k, on the norms of the various constant (non random) inputs and operators in the problem, and on the background distribution, but not on the dimension of the state space. Recall that X (0) = X (0),a, X (0) = X (0),a, Q (0) = Q(0),a, Q(0) = Q (0),a. A-priori bounds. From (5.8), (3.10), and (3.21), it follows that Q (k),a Q (k),f = M (k) Q (k 1),a M (k) M (k) 2 Q (k 1),a, (6.2) Q (k 1),f p Q (k),f p = M (k) Q (k 1),a M (k) p M (k) 2 Q (k 1),a p. (6.3) Because Q (0),a is a constant and the sequence Q (0),a p, = 1, 2,... is bounded independently due to (6.1) with k = 0, we have for all 1 p <, Q (k),f const (k), Q (k),f p const (p, k). (6.4) To estimate the norm of X (k),a = B(X (k),f, Q (k),f ), we use X (k),f = M (k) X (k 1),a + b (k) M (k) X (k 1),a + b (k), (6.5) and the estimate on the operator B from (5.6), which gives X (k),a = B(X (k),f, Q (k),f X (k),f + Q (k),f H (k) R (k) 1 ( d (k) + H (k) X (k),f ). (6.6) Combining (6.4), (6.6), and (6.4), we get X (k),a const (p, k). (6.7) Induction step. Let k 1 and assume that (6.1) holds with k 1 in place of k, for all > 0, and all 1 p <. Then, comparing (3.10) and (3.21), we have Q (k),f and from Lemma 5.8, Q (k),a Q(k),f p = MQ (k 1),a M MQ (k 1),a M p M 2 Q (k 1),a Q (k 1),a p, (6.8) Q (k),a p = A(Q (k),f ) A(Q(k),a ) p (1 + c Q (k),f ) Q (k),f Q(k),f p + c(1 + c Q (k),f ) Q (k),f 2p Q (k),f Q(k),f 2p, (6.9) where c (k) = H (k) 2 R (k) 1. Combining (6.2) (6.9), we obtain Q (k),a Q (k),a const (p, k) p. For the convergence of the ensemble mean to the Kalman filter mean, we have X (k),f X (k),f p = M (k) (X (k 1),a X (k 1),a ) p M (k) X (k 1),a X (k 1),a p, (6.10) 15

16 from (3.9) and (3.20). By Lemma 5.10, X (k),a X (k),a p = B(X (k),f X (k),f which, together with (6.4) and (6.8), yields, Q (k),f ) B(X(k)f, Q (k),f ) p X (k)f p + H 2 R 1 Q (k),f 2p X (k),f X (k)f 2p + Q (k),f Q(k),f p H R 1 (1 + Q (k),f H 2 ) d HX (k),f, X (k 1),a X (k 1),a const (p, k) p Acknowledgement. This research was partially supported by the U.S. ational Science Foundation under the grant DMS REFERECES [1] Brian D. O. Anderson and John B. Moore, Optimal filtering, Prentice-Hall, Englewood Cliffs,.J., [2] Jeffrey L. Anderson, An ensemble adjustment Kalman filter for data assimilation, Monthly Weather Review, 129 (2001), pp [3] Aloisio Araujo and Evarist Giné, The central limit theorem for real and Banach valued random variables, Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons, ew York-Chichester-Brisbane, [4] Craig H. Bishop, Brian J. Etherton, and Sharanya J. Majumdar, Adaptive sampling with the ensemble transform Kalman filter. Part I: Theoretical aspects, Monthly Weather Review, 129 (2001), pp [5] Mark Buehner, Ensemble-derived stationary and flow-dependent background-error covariances: Evaluation in a quasi-operational WP setting, Quarterly Journal of the Royal Meteorological Society, 131 (2005), pp [6] Gerrit Burgers, Peter Jan van Leeuwen, and Geir Evensen, Analysis scheme in the ensemble Kalman filter, Monthly Weather Review, 126 (1998), pp [7] Yuan Shih Chow and Henry Teicher, Probability theory. Independence, interchangeability, martingales, Springer-Verlag, ew York, second ed., [8] Giuseppe Da Prato, An introduction to infinite-dimensional analysis, Springer-Verlag, Berlin, [9] Alex Deckmyn and Loïk Berre, A wavelet approach to representing background error covariances in a limited-area model, Monthly Weather Review, 133 (2005), pp [10] Geir Evensen, Data Assimilation: The Ensemble Kalman Filter, Springer, 2nd ed., [11] M. Fisher and P. Courtier, Estimating the covariance matrices of analysis and forecast error in variational data assimilation. ECMWF Research Department Tech. Memo 220, [12] Reinhard Furrer and Thomas Bengtsson, Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants, J. Multivariate Anal., 98 (2007), pp [13] B. R. Hunt, E. J. Kostelich, and I. Szunyogh, Efficient data assimilation for spatiotemporal chaos: a local ensemble transform Kalman filter, Physica D: onlinear Phenomena, 230 (2007), pp [14] Andrew H. Jazwinski, Stochastic processes and filtering theory, Academic Press, ew York, [15] Eugenia Kalnay, Atmospheric Modeling, Data Assimilation and Predictability, Cambridge University Press, [16] D. T. B. Kelly, K. J. H. Law, and A. M. Stuart, Well-posedness and accuracy of the ensemble Kalman filter in discrete and continuous time. arxiv: , [17] Hui Hsiung Kuo, Gaussian measures in Banach spaces, Lecture otes in Mathematics, Vol. 463, Springer-Verlag, Berlin, [18] Peter D. Lax, Functional analysis, Pure and Applied Mathematics (ew York), Wiley- Interscience [John Wiley & Sons], ew York, [19] François Le Gland, Valérie Monbet, and Vu-Duc Tran, Large sample asymptotics for the ensemble Kalman filter, in The Oxford Handbook of onlinear Filtering, Dan Crisan and Boris Rozovskii, eds., Oxford University Press,

17 [20] David M. Livings, Sarah L. Dance, and ancy K. ichols, Unbiased ensemble square root filters, Phys. D, 237 (2008), pp [21] Jan Mandel, Loren Cobb, and Jonathan D. Beezley, On the convergence of the ensemble Kalman filter, Applications of Mathematics, 56 (2011), pp [22] Józef Marcinkiewicz, Collected papers, Edited by Antoni Zygmund. With the collaboration of Stanislaw Lojasiewicz, Julian Musielak, Kazimierz Urbanik and Antoni Wiweger. Instytut Matematyczny Polskiej Akademii auk, Państwowe Wydawnictwo aukowe, Warsaw, [23] J. Marcinkiewicz and A. Zygmund, Sur les foncions indépendantes, Fund. Math., 28 (1937), pp Reprinted in [22], pp [24] I. Mirouze and A. T. Weaver, Representation of correlation functions in variational assimilation using an implicit diffusion operator, Quarterly Journal of the Royal Meteorological Society, 136 (2010), pp [25] Olivier Pannekoucke, Loïk Berre, and Gerald Desroziers, Filtering properties of wavelets for local background-error correlations, Quarterly Journal of the Royal Meteorological Society, 133 (2007), pp [26] Dan Simon, Optimal State Estimation: Kalman, H, and onlinear Approaches, John Wiley and Sons, [27] Michael K. Tippett, Jeffrey L. Anderson, Craig H. Bishop, Thomas M. Hamill, and Jeffery S. Whitaker, Ensemble square root filters, Monthly Weather Review, 131 (2003), pp [28] Xuguang Wang, Craig H. Bishop, and Simon J. Julier, Which is better, an ensemble of positive negative pairs or a centered spherical simplex ensemble?, Monthly Weather Review, 132 (2004), pp [29] J. S. Whitaker and T. M. Hamill, Ensemble data assimilation without perturbed observations, Monthly Weather Review, 130 (2002), pp [30] Wojbor A. Woyczyński, On Marcinkiewicz-Zygmund laws of large numbers in Banach spaces and related rates of convergence, Probab. Math. Statist., 1 (1980), pp (1981). 17