Measurement Errors and the Kalman Filter: A Unified Exposition

Transcription

1 Luiss Lab of European Economics LLEE Working Document no. 45 Measurement Errors and the Kalman Filter: A Unified Exposition Salvatore Nisticò February 2007 Outputs from LLEE research in progress, as well contributions from external scholars and draft reports based on LLEE conferences and lectures, are published under this series. Comments are welcome. Unless otherwise indicated, the views expressed are attributable only to the author(s), not to LLEE nor to any institutions of affiliation. Copyright 2007, Salvatore Nisticò Freely available for downloading at the LLEE website ( llee@luiss.it

2 Measurement Errors and the Kalman Filter: A Unified Exposition Salvatore Nisticò* LLEE Working Document No. 45 February 2007 Abstract Exploiting a result on Joint Normal Distributions, these note provides a derivation of the Kalman Filter within a unified framework that nests the three relevant cases regarding the dynamics of the measurement errors: non-correlation, mutual correlation with state disturbances and serial correlation. JEL classification: C10, C32 Key words: Kalman Filter, Linear Dynamic Systems, State-space Models. (*) Address for correspondence: Salvatore Nisticò, Università LUISS Guido Carli, Via Oreste Tommasini 1, Rome, Italy. Url: snistico@luiss.it

3 1 Introduction A widespread textbook about time series analysis (Harvey, 1989) and the fourth volume of the Handbook of Econometrics (Hamilton, 1994b), present the derivation of the formulas for recursively updating the linear projection of an unobservable vector of states of a dynamic system: the Kalman Filter. In both cases, the main assumption on which the methodology and the results depend, is that the measurement errors in the equation of the observables be uncorrelated at all lags and with respect to the residuals in the state equation. Exploiting three theorems about least squares linear projections and orthogonal regressions, Ljungqvist and Sargent (2000) later provide formulas for the case in which measurement errors and state disturbances are possibly mutually correlated. This case nests the one of serially correlated measurement errors explicitly reported in Hansen et al. (1994) insofar as the latter can be reduced to the former. These notes offer a bridge, extending the derivation in Harvey (1989) and Hamilton (1994b) to the more general case of mutually correlated measurement errors and statedisturbances, that include the results stated in Ljungqvist and Sargent (2000) and Hansen et al. (1994). Exploiting a result on Joint Normal Distributions (along Harvey, 1989, and Hamilton, 1994b), therefore, here I provide a unified framework within which the optimal forecast of an unobservable vector of states in t + 1, given observations at t, (rather than simply the update of past linear projections) is given a straightforward derivation. 2 A useful result on JND Assuming that two random vectors have joint multivariate normal distribution [ ] ([ ] [ ]) z1 µ1 Ω11 Ω 12 MV N, z 2 µ 2 Ω 21 Ω 22 (1) the properties of Joint Normal Distributions imply that both the marginal and the conditional distributions are also Gaussian. In particular, the conditional distribution 1

4 of z 1 given z 2 is normal with mean and co-variance matrix, respectively equal to and E[z 1 z 2 ] = µ 1 + Ω 12 Ω 1 22 (z 2 µ 2 ) (2) cov[z 1 z 2 ] = Ω 11 Ω 12 Ω 1 22 Ω 21 (3) Thus, equation (2) can be properly seen as reflecting the optimal forecast of z 1 on the basis of the observation of z 2, while equation (3) provides a measure of the Mean Squared Error (MSE) of such a forecast. 3 The Dynamic System A useful representation of most linear dynamic systems is the so-called state-space form, in which an equation describes the dynamics of the unobservable states of the system (ξ t ) and one accounts for the observable endogenous variables (y t ): ξ t+1 = F ξ t + η t+1 y t = Hξ t + ω t (4) where matrix F is assumed to have all eigenvalues within the unit circle and the vectors η t+1 and ω t collect state disturbances and measurement errors, respectively. The usual assumption about the joint distribution of the two error terms (that drives the results in Harvey, 1989 and Hamilton, 1994b) is that E[η t+s ω t] = E[η t η τ] = E[ω t ω τ] =, for all s and all t τ, with t, τ, s Z, while V η E[η t η t] and V ω E[ω t ω t]. Here I relax this assumption, assuming that the measurement errors and the state disturbances have a JND such that [ ] ([ ] [ ]) ηt+1 Vη V ηω MV N,. (5) ω t The aim is to show that merely applying the result recalled in Section 2 provides a straightforward derivation of the optimal forecast of the unobservable states at t + 1 conditional upon information held through t V ηω V ω ˆξ t+1 t E(ξ t+1 I t ) (6) 2

5 as well as the associated MSE P t+1 t E[(ξ t+1 ˆξ t+1 t )(ξ t+1 ˆξ t+1 t ) I t ] (7) where the information set I t is formally defined as collecting all the realizations observed through date t I t (y t, y t 1, y t 2,..., y 0 ). (8) The Kalman Filter is an algorithm that allows the recursive computation of such a forecast. Starting off with the unconditional mean and covariance matrix of the state vector, such an algorithm sequentially updates the past projections to account for the informational gains that occur along time as realizations of y t become observable. The conditional distribution of the states at t given I t 1 is therefore: ξ t I t 1 MV N(ˆξ t t 1, P t t 1 ). (9) Analogously, the optimal forecast two periods ahead and the corresponding MSE define the conditional distribution of ξ t+1 given I t 1 as ξ t+1 I t 1 MV N(ˆξ t+1 t 1, P t+1 t 1 ). (10) Here we have that ˆξ t+1 t 1 E(ξ t+1 I t 1 ) = E(F ξ t + η t+1 I t 1 ) = F ˆξ t t 1 (11) and P t+1 t 1 E[(ξ t+1 ˆξ t+1 t 1 )(ξ t+1 ˆξ t+1 t 1 ) I t 1 ] = E[(F (ξ t ˆξ t t 1 ) + η t+1 )(F (ξ t ˆξ t t 1 ) + η t+1 ) I t 1 ] = F P t t 1 F + V η, (12) where the cross-products are zero, since the state disturbances in t + 1 are correlated neither to past realizations of the states nor to their past optimal forecast: E(ξ t η t+1) = E(ˆξ t t 1 η t+1) =. 3

6 Turning to the observables y t, the optimal forecast given the information set at t 1 is while its MSE corresponds to ŷ t t 1 E(y t I t 1 ) = E(Hξ t + ω t I t 1 ) = H ˆξ t t 1 (13) Ω t t 1 E[(y t ŷ t t 1 )(y t ŷ t t 1 ) I t 1 ] = E[(H(ξ t ˆξ t t 1 ) + ω t )(H(ξ t ˆξ t t 1 ) + ω t ) I t 1 ] = HP t t 1 H + V ω. (14) Moreover, the cross-correlation between states and observables displays a meaningful term due to the assumption about the joint distribution of the error terms (as expressed by (5)): Σ t t 1 E[(ξ t+1 ˆξ t+1 t 1 )(y t ŷ t t 1 ) I t 1 ] = E[(F (ξ t ˆξ t t 1 ) + η t+1 )(H(ξ t ˆξ t t 1 ) + ω t ) I t 1 ] = F P t t 1 H + V ηω. (15) Given the results above, we are finally able to describe the joint distribution of the state-space representation (4) conditional upon information available at t 1, as [ ] ([ ] [ ]) ξt+1 I t 1 F ˆξt t 1 Pt+1 t 1 Σ t t 1 MV N y t I t 1 H ˆξ,. (16) t t 1 Ω t t 1 4 Applying the result on JND Σ t t 1 From the definition of the information set (8) follows that: [y t, I t 1 ] [y t, (y t 1, y t 2,..., y 0 )] I t. (17) This further implies (ξ t+1 I t 1 ) (y t I t 1 (yt ) ξ t+1, I t 1 I ) ξ t t+1, (18) which is normally distributed with mean ˆξ t+1 t and covariance matrix P t+1 t. These latter statistics are what we are in search of, and applying the result on JND to system (16) provides them both: ˆξ t+1 t = F ˆξ t t 1 + Σ t t 1 Ω 1 t t 1 (y t H ˆξ t t 1 ) = F ˆξ t t 1 + (F P t t 1 H + V ηω )(HP t t 1 H + V ω ) 1 (y t H ˆξ t t 1 ) (19) 4

7 P t+1 t = P t+1 t 1 Σ t t 1 Ω 1 t t 1 Σ t t 1 = F P t t 1 F + V η (F P t t 1 H + V ηω )(HP t t 1 H + V ω ) 1 (F P t t 1 H + V ηω ). (20) In the formulas above, the Kalman Gain matrix, which updates an available forecast through the informational gain provided by the observed forecast error y t H ˆξ t t 1, corresponds to the coefficient of the mentioned forecast error in equation (19): K t Σ t t 1 Ω 1 t t 1 = (F P t t 1H + V ηω )(HP t t 1 H + V ω ) 1. (21) 5 Two alternative assumptions about measurement errors 5.1 No serial nor mutual correlation. The assumption of no correlation whatsoever in the errors dynamics (as in Harvey, 1989, and Hamilton, 1994a and 1994b) is equivalent to assuming that V ηω =. This directly implies that the recursive formulas become: ˆξ t+1 t = F ˆξ t t 1 + F P t t 1 H (HP t t 1 H + V ω ) 1 (y t H ˆξ t t 1 ) (22) P t+1 t = F P t t 1 F + V η F P t t 1 H (HP t t 1 H + V ω ) 1 HP t t 1 F (23) which corresponds to equations (2.16) and (2.17) in Hamilton (1994b), and which imply a Kalman Gain Matrix equal to K t = F P t t 1 H (HP t t 1 H + V ω ) 1. (24) 5.2 Serially correlated measurement errors Let the dynamics of the measurement errors be described by a stationary Vector Auto-Regressive process of order one: ω t = Dω t 1 + ε t (25) where ε t V W N(, V ε ) and all the eigenvalues of matrix D are inside the unit circle. 5

8 Redefining the system in terms of the quasi-difference ȳ t y t+1 Dy t, the case of serially correlated measurement errors can be reduced to the general case addressed in Section 3. The new state-space model, in fact, can be written as: ξ t+1 = F ξ t + η t+1 ȳ t = (HF DH)ξ t + Hη t+1 + ε t+1 = Hξ t + ω t (26) in which the error term ω t is not serially correlated but it is with the state disturbance η t+1. As to the relevant covariance matrices, the transformation above is equivalent to assuming V η ω = V η H and V ω = HV η H + V ε. As a consequence, applying the results on JND directly yields: ˆξ t+1 t = F ˆξ t t 1 + (F P t t 1 H + V η H )( HP t t 1 H + HV η H + V ε ) 1 (ȳ t H ˆξ t t 1 ) (27) and the matrix-riccati-equation P t+1 t = F P t t 1 F + V η (F P t t 1 H + V η H )( HP t t 1 H + HV η H + V ε ) 1 ( HP t t 1 F + HV η ), (28) in which the Kalman Gain matrix takes the form K t = (F P t t 1 H + V η H )( HP t t 1 H + HV η H + V ε ) 1. (29) Equations (28) and (29) are equivalent to equations (101) and (99) in Hansen et al. (1994), up to some notational difference. 6 Conclusions Moving from a basic result about Joint Normal Distributions (along Harvey, 1989, and Hamilton, 1994b) these notes provide a straightforward derivation of the Kalman Filter for a general specification of the dynamics exhibited by a state-space model. It is shown that the proposed formulation nests the three relevant cases about measurement errors (serial-correlation, mutual correlation with state disturbances and noncorrelation), thus extending the derivation in Harvey (1989) and Hamilton (1994b) to include the results stated in Ljungqvist and Sargent (2000) and Hansen et al. (1994). 6

9 References [1] Hamilton, J.D., 1994a, Time Series Analysis. (Princeton University Press, Princeton, NJ). [2] Hamilton, J.D., 1994b, State-Space Models. In: R.F. Engle and D.L. Mc- Fadden, eds., Handbook of Econometrics, Vol.4 (North-Holland, Amsterdam) [3] Harvey, A.C., 1989, Forecasting, Structural Time Series Models and the Kalman Filter. (Cambridge University Press, Cambridge, UK). [4] Hansen, L.P., E.R. McGrattan, and T.J. Sargent, 1994, Mechanics of Forming and Estimating Dynamic Linear Economies. Federal Reserve Bank of Minneapolis, Staff Report 182 [5] Ljungqvist L. and T.J. Sargent, 2000, Recursive Macroeconomic Theory, (The MIT Press, Cambridge, Mass.) 7