Difference equations. Definitions: A difference equation takes the general form. x t f x t 1,,x t m.

Transcription

1 Difference equations Definitions: A difference equation takes the general form x t fx t 1,x t 2, defining the current value of a variable x as a function of previously generated values. A finite order (mth order) difference equation takes the general form x t fx t 1,,x t m. A linear difference equation takes the general form x t 1 x t 1 2 x t 2 A stochastic difference equation takes the general form x t fx t 1,x t 2,, t where t is a random sequence (often i.i.d. in applications) called the forcing process or driving process. (c) James Davidson /04/2014

2 A linear stochastic difference equation takes the general form x t 1 x t 1 2 x t 2 t The object is to solve these equations (determine the path x 1,x 2,x 3, ) given initial conditions x 0,x 1, and (if present) the sequence t. Consider the non-stochastic case to establish methods and notation. The linear case is the simplest and best understood, and we focus on this. (c) James Davidson /04/2014

3 Some useful formulae A geometric series is A 1 a a 2 a 3 a i. i0 Let the tth partial sum be denoted t 1 At 1 a a 2 a t 2 a t 1 i0 a i. Note that aat a a 2 a t 1 a t At a t 1. If a 1, this has the closed-form solution At aat 1 a t 1 at 1 a. (c) James Davidson /04/2014

4 Cases: 1. If a 1, the geometric series is convergent. a t 0ast, and A limat 1 t 1 a 2. If a 1itdiverges: A. 3. If a 1, no solution. At "flip-flops" between 0 and If a 1, At flip-flops between in limit! 5. Finally, if a 1, At t and A. (c) James Davidson /04/2014

5 Also consider A t a 2a 2 3a 3 ta t By a similar argument, 1 aa t a 2a 2 3a 3 ta t a 2 2a 3 ta t1 a a 2 a 3 a t ta t1 Hence, with a 1, A t a 1 a At tat. When a 1, note that A i1 ia i a 1 a A a 1 a 2 (c) James Davidson /04/2014

6 First order linear difference equation x t 1 x t 1 Given x 0, the solution path is found by iteration, as x 1 1 x 0 x x 0 x t 1 1 t 1 1 t 1 x 0 If 1 1 the series is summable, and as t, j x t 1. j0 1 1 This is called the stable solution, and is independent of x 0. x t approaches this point from any starting point. In the other cases of 1, there is either no stable solution, or an infinite solution. Note: we can guess the solution by putting x t x t 1 x (say) and so solve x 1 1 This must be the stable solution if one exists but otherwise, it is irrelevant. (c) James Davidson /04/2014

7 Second Order Linear Difference Equation x t 1 x t 1 2 x t 2 Our concern is to find conditions for a stable solution to this equation, as in the first-order case. If it exists, this must take the form x However, solution by iteration is obviously difficult: x t 1 x t 1 2 x t x t x t 3? How do we know if the solution converges? (c) James Davidson /04/2014

8 Consider the pair of first-order equations x t 1 x t 1 y t Write these in the form y t 2 y t 1. x t 1 x t 1 2 x t 1 1 x t 2 x t 1 2 x t x t 2 x t 1 x t 1 2 x t 2 It is intuitively clear that the stability conditions have the form 1 1, 2 1 since then y t 1 2 and x t To apply these restrictions, it is necessary to invert the mapping 1, 2 1, 2. (c) James Davidson /04/2014

9 The Lag Operator Let the operator L (alternative notation, B) be defined by Lx t x t 1. Then, for example, L 2 x t LLx t Lx t 1 x t 2. The second-order equation can now be written 1 1 Lx t y t or equivalently, 1 2 Ly t 1 1 L1 2 Lx t L 1 2 L 2 x t 1 1 L 2 L 2 x t. The model therefore defines a quadratic equation in the lag operator. (c) James Davidson /04/2014

10 Reminder: Roots of a Quadratic Consider, for z C, the quadratic equation z 2 1 z 2 z 1 z 2 0 where and and 2 are the roots (zeros) of this equation, and are given by , When /4 these solutions are complex numbers (complex conjugate pair, since 1 and 2 real). Also note: The roots of the equation 1 1 z 2 z z1 2 z 0 are 1/ 1 and 1/ 2, similarly. y t (above) can be complex-valued, although x t is real-valued by construction. (c) James Davidson /04/2014

11 Stability Analysis The stable solution (finite, independent of initial conditions) evidently takes the form x t 1 1 L 2 L provided the inversion of the lag polynomial is a legitimate step. Assume 1 1, 2 1 and consider for z C, z 2 z z1 2 z. Note that, for z 1, so that 1 1 z1 1 z 1 2 z z z j 1 z j. j0 1. Assume 1 2 (both real). Then, for z 1, z1 2 z z z j0 1 j1 2 j1 1 2 z j (c) James Davidson /04/2014

12 2. Assume 1 2 (real). Write in j1 j By L Hôpital s rule*, and hence (Compare page 2.5 above.) 1 j1 1 j z 2 j 1 1 j j0 j 11 j z j as 0, *Iff 0andg 0as 0, the limit of f g is equal to that of f g latter is defined., when the (c) James Davidson /04/2014

13 Complex Roots Let 1 re i 2 re i... a complex conjugate pair in polar coordinates, where r 0, i 1 and 0 2. Their modulus (absolute value) is r. Using the facts we obtain re i rcos irsin (Euler s formula) cos x cosx sin x sinx 1 j1 2 j1 1 2 r j eij1 e ij1 e i e i r j sinj 1 sin, j 1,2,3, (c) James Davidson /04/2014

14 Stability depends on the k lying inside the unit circle, having r μ 1 r θ 0 1 μ 2 1 (c) James Davidson /04/2014

15 Finally..., replace z by the operator L. Since L j for any j, wehavetheresult x t j0 1 j1 2 j1 1 2 real roots, 1, 2 1 j0 j 11 j equal roots, 1 1 j0 r j sinj 1 sin complex roots, r 1 (c) James Davidson /04/2014

16 Third Order Case Exercise: verify that if 1 2, 2 3 and 1 3,then z1 2 z1 3 z etc., etc z z z The General Case: Factorising the polynomial as 1 1 z p z p 1 1 z1 2 z1 p z, the rule is that the difference equation x t 1 x t 1 p x t p has a stable solution if and only if k 1fork 1,,p. Since the roots of this polynomial are 1/ 1,,1/ p, we express the stability condition as: The roots of the lag polynomial lie strictly outside the unit circle. Since the lag coefficients are real, the roots are either real, or in conjugate complex pairs. (c) James Davidson /04/2014

17 Stochastic Linear Difference Equations First Order Case: x t 1 x t 1 t, t 1,2,3, The iterative solution is x t 1 1 t 1 1 t 1 t 1 t t 1 x 0 Assume that t iid0, 2. The solution of this equation is interpreted in terms of the properties of the random variable x t when t is large. Case: 1 1. As t, dependence on starting value becomes negligible. Ex t 1 1 j 1 E t j j1 1 1 Varx t E j0 j 2 1 t j j0 2j 1 E 2 t j j0 2 j0 1 2j k0,k j 1 j 1 k E t j t k (c) James Davidson /04/2014

18 In this calculation, note that 1 j 2 1. j Therefore, if the E t j t k were to take any finite fixed values such that for all t, j and k, we can say that E t j t k B (*) j0 k0,k j 1 j 1 k E t j t k j0 1 j 1 k E t j t k k0 B Hence, since E t j t k 0 whenever j k, the double sum of cross-product terms vanishes unambiguously. (c) James Davidson /04/2014

19 Other Properties Since the forcing process is i.i.d., and x t depends (in effect) on only a finite number of these terms, this is a stationary process when t is large enough. x tm and Ex t tm j 0forj m. Hence, 1 1 j0 1 j tm j m j m t j 1 j 1 tm j j0 j0 1 1 m 1 m m 1 1 x t 1 j 1 tm j j0 Covx t,x tm m 1 Varx t 1 m m. Therefore, x t is a short memory process, since j 2 j Note: we may define a stationary process with starting point t 0, by letting x 0 be a drawing from the stationary distribution of x t. (c) James Davidson /04/2014

20 This stochastic process is called a first-order autoregression (AR(1)). Realization of 100 observations from x t 0.7x t 1 t, t N0,1, x 0 0 Corresponding i.i.d. process, t : (c) James Davidson /04/2014

21 Second Order Case. x t 1 x t 1 1 x t 2 t, t 1,2,3, From previous results, we have the stationary second order solution (MA() representation), j0 1 j1 2 j1 1 2 t j real roots, 1, 2 1 x t t 1 1 L 2 L 2 j j 11 t j equal roots, 1 1 j0 j0 r j sinj 1 sin t j complex roots, r 1 Complex roots imply sinusoidal lag distributions in the MA() representation! (c) James Davidson /04/2014

22 Generalization The AR(p) process is x t 1 x t 1 p x t p t. Using the lag operator, this can be written in the form or where x t 1 Lx t 2 L 2 x t p L p x t t Lx t t L 1 1 L 2 L 2 p L p. The stationary solution of the model, when it exists, can be written in the form x t 1 t L where 1/L is a lag polynomial of infinite order, with summable coefficients. (c) James Davidson /04/2014

23 Autocovariances of AR processes These are conveniently found from the Yule-Walker equations. Multiply the equation by x t j for j 0,1,2,,p and take expected values to yield a system of p 1 equations in the unknowns 0, p. Consider the AR(2): These equations may be solved as (c) James Davidson /04/2014

24 For the higher order cases, solve the difference equation for j 3,4,5, j 1 j 1 2 j 2 Obviously, the conditions for j 0 are identical to the stability conditions for the process. By rearranging the Y-W equations, one can also solve for 2, 1, 2 from 0, 1, 2. (c) James Davidson /04/2014

25 Moving Average Processes Consider a process of the form x t L t where L 1 1 L q L q and t iid0, 2. This is the MA(q) process. The autocovariances are q q q q q 2 q 2 q with j 0forj q. Thus, the process is stationary for all choices of L. If L is invertible, it can be expressed as a difference equation of infinite order, x t L 1 t. All MA(q) processes can be written as AR() except those having a root of unity (over-differenced processes). (c) James Davidson /04/2014

26 Invertibility of MA Processes Consider the MA(1) case first. is a process with the following properties: v t t 1 t 1 Ev t 0, Varv t , Covv t,v t Covv t,v t j 0forj 1. But suppose t iid0, 2 where Then we can write equivalently v t t 1 1 t 1 The processes v t and v t have the same autocovariances, and on this basis are observationally equivalent. They are not distributed identically, and if t is i.i.d then t is not, in general. However, we cannot distinguish them on the basis of first and second moments. With this caveat, there is no loss of generality in choosing the representation with 1 1. (c) James Davidson /04/2014

27 The MA(q) case: v t L t and v t L t are observationally equivalent processes where L 1 1 L1 q L L L1 2 L1 q L and E 2 t 2 1 E 2 t. In total, 2 q equivalent representations! Conventionally, we impose invertibility to identify the model. Further caveat: We can also write, for example with same v t, where by construction t v t 1 1 L t L t 1 1L L t. In this case, both t and t are uncorrelated processes with E t E t 2, but at most one of these processes can be i.i.d, in general. The exception is the Gaussian case, where uncorrelatedness is equivalent to independence. (c) James Davidson /04/2014

28 ARMA Processes Combining AR and MA components yields the ARMA(p,q) process Lx t L t. This represents a flexible class of linear models for stationary processes. Subject to stability/invertibility, the ARMA can be viewed as a difference equation of infinite order (AR( )), L L x t 1 t and as a moving average of infinite order (MA()): x t 1 L L t. (c) James Davidson /04/2014

29 Solving Rational Lag Expansions Method of undetermined coefficients Write z zz and solve for the coefficients of z. For simplicity set q p, sowehave 1 1 z p z p 1 1 z p z p 1 1 z 2 z 2 3 z z p z p 1 z 1 1 z 2 p 1 z p1 p z p 1 p z p1 p p z 2p 1 Equating coefficients of z j for j 1,2,... and rearranging, we obtain and then for j p, p p p p p 1 j p p 1 j p1 1 j 1. These equations can be solved as a recursion for as many steps as required. Stability of the polynomial z ensures that j 0asj. (c) James Davidson /04/2014

30 Vector Autoregressions Let x t be a sequence of m-vectors. The VAR(p) model takes the form x t a 0 A 1 x t 1 A p x t p u t where a 0 is a m-vector of intercepts, A k,k 1,,p are square m m matrices, u t m 1 is a vector of disturbances with Eu t m Eu t u t m1 mm Also write where AL I A 1 L A p L p. ALx t u t (c) James Davidson /04/2014

31 Stability of the VAR Start with the case p 1. Solve the system by repeated substitution as x t a 0 Ax t 1 u t a 0 Aa 0 Ax t 2 u t 1 u t I A A 2...a 0 u t Au t 1 A 2 u t 2 Thus, we need to know the properties of A n as n gets large. (c) James Davidson /04/2014

32 Eigenvalues The eigenvalues of A are the solutions (assumed distinct - possibly complex-valued) to the equation A I 0. A square matrix with distinct eigenvalues 1, 2,, m has diagonalization A CMC 1 where M diag 1,, m and C is the matrix of eigenvectors. Note that A I CMC 1 I CM C 1 CC 1 M I C C 1 M I 1 2 m. Stability condition: A n CMC 1 CMC 1 CMC 1 CM n C 1. The conditions for A n 0 as n are that i 1, for each i. (c) James Davidson /04/2014

33 The General Case Write the VAR(p) model in companion form: x t A 1 A 2 A p 0 x t 1 u t x t 1 I x t 2 0 x t 2 0 I x t p x t p 0 0 I 0 x t p 1 0 or x t A x t 1 u t mp 1 1. Repeat the analysis of p 1 on this model. A (mp 1 mp 1) hasmp 1 eigenvalues of which m are 0. It can be shown that the remaining mp eigenvalues match the inverted roots of A 0. (c) James Davidson /04/2014

34 The Generalized Stability (Invertibility) Condition: Generalizing the AR(p) analysis, we can show that all the roots of the equation A p I p 1 A 1 A p (a polynomial of order mp) must lie outside the unit circle. Note the case p 1. Observe that I A m I A where 1/, so the eigenvalue and polynomial root conditions are equivalent. Note the case m 1. The companion form provides an alternative way to analyse the stability of the AR(p). (c) James Davidson /04/2014

35 The Final Form of a VAR To solve ALx t u t, note that AL 1 1 AL adjal where A is a lag polynomial of order mp and the elements of adjal are lag polynomials of maximum order pm 1. The final form equations are AL x t adjalu t Key facts: The vector on the right-hand side is a sum of m moving average terms in the elements of u t. A sum of MAq terms has zero autocovariances beyond q lags, hence itself has a MA representation. Hence, the final form of a VAR is a vector of univariate ARMA processes! Nominally the AR roots are the same for each equation. In practice there are frequently cancellations of common factors, element by element. (c) James Davidson /04/2014