Kalman Filter and state-space models: time-varying parameter models; models with unobservable variables; basic tool: Kalman filter; implementation is task-specific. y t = x t β t + e t (1) β t = µ + Fβ t 1 + v t, (2) e t i.i.d. N(0, R); v t i.i.d. N(0, Q); E(e t v t ) = 0. (3) Possible expected values of time-t-beta: β t t 1, β t t, β t T. Notation: P t t 1 =E[(β t β t t 1 )(β t β t t 1 ) ], P t t =E[(β t β t t )(β t β t t ) ], P t T =E[(β t β t T )(β t β t T ) ], y t t 1 =x t β t t 1, η t t 1 =y t y t t 1, f t t 1 =E(η 2 t t 1 ).
and Kalman Filter Prediction Updating β t t 1 = µ + Fβ t 1 t 1, P t t 1 = FP t 1 t 1 F + Q, η t t 1 = y t x t β t t 1, f t t 1 = x t P t t 1 x t + R. β t t = β t t 1 + K t η t t 1, P t t = P t t 1 K t x t P t t 1, where K t = P t t 1 x t f 1 is the Kalman gain. It is a weight, that t t 1 depends on the prediction uncertainty and the variance of the error. This process can be depicted graphically. and Kalman Filter (continued) If we are given the following: values of the parameters µ, F, R, Q, and the starting values, β 0 0 and P 0 0, then we can iterate the Prediciton and Updating equations (in the order listed on the previous slide) over all observations, t [0, T ], so as to compute the estimates of the time-varying parameters, β t t. If we are not given the starting values, but the time-varying parameters are known to be stationary, then we can calculate them: β 0 0 = (I F) 1 µ, and vec(p 0 0 ) = (I F F) 1 vec(q).
and Kalman Filter (continued) If we are not given the statring values, and the time-varying parameters may not be stationary, then: set β 0 0 to any arbitrary values and set diagonal elements of P 0 0 to large values, or set P 0 0 = 0 and estimate β 0 0 using maximum likelihood method, then calculate P 0 0 = cov(β MLE 0 0 ) and then rerun the Kalman filter with these estimates as the starting values. If we are not given the values of the parameters µ, F, R, Q, then estimate them using the maximum likelihood method. and Kalman Filter: ML estimation Given the normality assumptions in equation (3), the log-likelihood function for the model is: L = 1 T ln(2πf 2 t t 1 ) 1 T η t=1 2 t t 1 f 1 t t 1 η t t 1. (4) t=1 If the initial values, β 0 0 and P 0 0 are known or at least the time-varying parameters are stationary, then maximize the function in equation (4) with respect to the unknown parameters.
and Kalman Filter: ML estimation If the initial values, β 0 0 and P 0 0 are unknown and the time-varying parameters are not stationary, then proceed in one of two ways: estimate the initial values: maximize the function in equation (4) with respect to both the unknown parameters and the initial values by setting P 0 0 = 0, calculate P 0 0 = cov(β MLE 0 0 ), then rerun the Kalman filter (i.e., the Prediction and Updating equations) with these estimates as the starting values; or use random initial values: set β 0 0 to any arbitrary values; set the diagonal elements of P 0 0 to very large values; maximize the following log-likelihood funciton: L = 1 T ln(2πf 2 t t 1 ) 1 T η t=τ 2 t t 1 f 1 t t 1 η t t 1, (5) t=τ where τ is large enough as to dampen the effect of the random initial values (i.e., the period [0, τ 1] is the burn-in period). and Kalman Filter: Smoothing The forecast values of time-varying parameters, β t t 1, and their updated values, β t t are used in the basic Kalman filter. Using more information provides more accurate estimates of the timevarying parameters. How about calculating β t T? After running the basic Kalman filter (Prediction + Updating equations), run backwards (i.e., from time T to time 0) the following equations: Smoothing β t T = β t t + P t t F P 1 t+1 t (β t+1 T Fβ t t µ), and P t T = P t t + P t t F P 1 t+1 t (P t+1 T P t+1 t )(P 1 t+1 t ) FP t t. The starting values for the smoothing equations, β T T and P T T, come from where?
Smoothed Kalman Filter Prediction Updating β t t 1 = µ + Fβ t 1 t 1, P t t 1 = FP t 1 t 1 F + Q, η t t 1 = y t x t β t t 1, f t t 1 = x t P t t 1 x t + R. Smoothing β t t = β t t 1 + K t η t t 1, P t t = P t t 1 K t x t P t t 1, β t T = β t t + P t t F P 1 t+1 t (β t+1 T Fβ t t µ), and P t T = P t t + P t t F P 1 t+1 t (P t+1 T P t+1 t )(P 1 t+1 t ) FP t t. Unobservable State Variables Measurement Equation Transition Equation y t = B t x t + Az t + e t ; (6) x t = µ + Fx t 1 + v t, (7) e t i.i.d. N(0, R); v t i.i.d. N(0, Q); E(e t v t) = 0. (8) This model, including the parameters and the unobservable components, can be estimated using the Kalman filter. Notation: P t t 1 =E[(x t x t t 1 )(x t x t t 1 ) ], P t t =E[(x t x t t )(x t x t t ) ], P t T =E[(x t x t T )(x t x t T ) ], y t t 1 =B t x t t 1 + Az t, η t t 1 =y t y t t 1, f t t 1 =E(η t t 1 η t t 1 ).
Prediction Updating and Kalman Filter x t t 1 = µ + Fx t 1 t 1, P t t 1 = FP t 1 t 1 F + Q, η t t 1 = y t B t x t t 1 Az t, f t t 1 = B t P t t 1 B t + R. where K t = P t t 1 B t f 1 t t 1. Smoothing x t t = x t t 1 + K t η t t 1, P t t = P t t 1 K t B t P t t 1, x t T = x t t + P t t F P 1 t+1 t (x t+1 T Fx t t µ), and P t T = P t t + P t t F P 1 t+1 t (P t+1 T P t+1 t )(P 1 t+1 t ) FP t t. Examples Many familiar models can be cast in the form of the State-Space Model. Some examples. AR(2) Model: y t = δ + φ 1 y t 1 + φ 2 y t 2 + w t, w t i.i.d.n(0, σ 2 ); [ ] y t = δ xt + [1 0] ; x t 1 [ ] [ ] [ ] [ ] xt φ1 φ = 2 xt 1 wt +, x t 1 1 0 x t 2 0 where δ = δ/(1 φ 1 φ 2 ).
ARMA(2,1) Model: y t = φ 1 y t 1 + φ 2 y t 2 + w t + θw t 1, w t i.i.d.n(0, σ 2 ); [ ] x1,t y t = [1 θ] ; x 2,t [ ] [ ] [ ] [ ] x1,t φ1 φ = 2 x1,t 1 wt +. x 2,t 1 0 x 2,t 1 0 Time-varying Parameter Model: discussed above; measurement equation is equation (1) above; transition equation is equation (2) above. Kim and Nelson s (1989) model for money supply growth: M t = x t 1 β t + e t, x t = [ 1 i t INF t SURP t M t ]. (9) generates ARCH-like effects via f t t 1 = x t P t t 1 x t + R.
Unobserved-Components Model: y t = y 1t + y 2t, y 1t = δ + y 1,t 1 + e 1t, y 2t = φ 1 y 2,t 1 + φ 2 y 2,t 2 + e 2t, e it i.i.d.n(0, σ 2 i ), i = 1, 2, E(e 1t, e 2s ) = 0 for all t and s. y 1t y t = [1 1 0] y 2t ; y 2,t 1 y 1t δ y 2t = 0 + y 2,t 1 0 1 0 0 0 φ 1 φ 2 0 1 0 y 1,t 1 y 2,t 1 y 2,t 2 + e 1t e 2t. 0 Unobserved-Components Model (continued): A model for log-real GDP: y t = n t + x t, n t = g t 1 + n t 1 + v t, v t i.i.d.n(0, σ 2 v ), g t = g t 1 + w t, w t i.i.d.n(0, σ 2 w), x t = φ 1 x t 1 + φ 2 x t 2 + e t, e t i.i.d.n(0, σ 2 e ), U t = L t + C t, L t = L t 1 + ψ t, ψ t i.i.d.n(0, σ 2 ψ ), C t = α 0 x t + α 1 x t 1 + α 2 x t 2 + ε t, ε t i.i.d.n(0, σ 2 ε ), where all errors are independent.
Unobserved-Components Model (continued): A model for log-real GDP (continued): [ yt U t ] = [ 1 1 0 0 0 0 0 α 0 α 1 α 2 0 1 ] n t x t x t 1 x t 2 g t L t n t 1 0 0 0 1 0 n t 1 x t 0 φ 1 φ 2 0 0 0 x t 1 x t 1 x = 0 1 0 0 0 0 x t 2 t 2 0 0 1 0 0 0 x t 3 g t 0 0 0 0 1 0 g t 1 L t 0 0 0 0 0 1 L t 1 [ ] 0 + ; ε t + v t e t 0 0 w t ψ t. Dynamic Factor Model: y 1t = γ 1 c t + z 1t, y 2t = γ 2 c t + z 2t, c t = φ 1 c t 1 + v t, v t i.i.d.n(0, 1), z it = α i z i,t 1 + e it, e it i.i.d.n(0, σi 2 ), and E(e 1t, e 2t ) = E(e 1t, v t ) = E(e 2t, v t ) = 0. c t z 1t z 2t [ ] y1t = y 2t = φ 1 0 0 0 α 1 0 0 0 α 2 [ ] c γ1 1 0 t z γ 2 0 1 1t ; z 2t c t 1 z 1,t 1 z 2,t 1 + v t e 1t. e 2t
Dynamic Factor Model (continued): A model for coincident economic indicators (see Stock and Watson (1991)): y it = γ i c t + e it, i = 1, 2, 3, 4, c t = φ 1 c t 1 + φ 2 c t 2 + w t, w t i.i.d.n(0, 1), e it = ψ i1 e i,t 1 + ψ i2 e i,t 2 + ɛ it, ɛ it i.i.d.n(0, σi 2 ), i = 1, 2, 3, 4 Dynamic Factor Model (continued): A model for coincident economic indicators (continued): y 1t y 2t y 3t y 4t = γ 1 0 1 0 0 0 0 0 0 0 γ 2 0 0 0 1 0 0 0 0 0 γ 3 0 0 0 0 0 1 0 0 0 γ 4 0 0 0 0 0 0 0 1 0 c t c t 1 e 1t e 1,t 1 e 2t e 2,t 1 e 3t e 3,t 1 e 4t e 4,t 1 ;
Dynamic Factor Model (continued): A model for coincident economic indicators (continued): c t c t 1 e 1t e 1,t 1 e 2t e 2,t 1 e 3t e 3,t 1 e 4t e 4,t 1 = φ 1 φ 2 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 ψ 11 ψ 12 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 ψ 21 ψ 22 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 ψ 31 ψ 32 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 ψ 41 ψ 42 0 0 0 0 0 0 0 0 1 0 + [ w t 0 ɛ it 0 ɛ 2t 0 ɛ 3t 0 ɛ 4t 0 ]. c t 1 c t 2 e 1,t 1 e 1,t 2 e 2,t 1 e 2,t 2 e 3,t 1 e 3,t 2 e 4,t 1 e 4,t 2 Common Stochastic Trend Model: y 1t = z t + x 1t, y 2t = z t + x 2t, z t = z t 1 + ɛ t, ɛ t i.i.d.n(0, σ 2 ɛ ), x 1t = µ 1 + φ 11 x 1,t 1 + φ 12 x 2,t 1 + e 1t + θ 11 e 1,t 1 + θ 12 e 2,t 1, x 2t = µ 2 + φ 21 x 1,t 1 + φ 22 x 2,t 1 + e 2t + θ 21 e 1,t 1 + θ 22 e 2,t 1, [ ] ([ ] [ ]) e1t 0 σ 2 i.i.d.n, 1 σ 12 e 2t 0 σ 12 σ2 2. [ ] y1t = y 2t [ 1 1 0 0 0 1 0 1 0 0 ] z t x 1t x 2t e 1t e 2t ;
Common Stochastic Trend Model (continued): z t 0 1 0 0 0 0 x 1t µ 1 0 φ 11 φ 12 θ 11 θ 12 x 2t = µ 2 + 0 φ21 φ 22 θ 21 θ 22 e 1t 0 0 0 0 0 0 e 2t 0 0 0 0 0 0 1 0 0 0 1 0 + 0 0 1 0 1 0 0 0 1 ɛ t e 1t. e 2t z t 1 x 1,t 1 x 2,t 1 e 1,t 1 e 2,t 1 +