Chapter 1. Basics. 1.1 Definition. A time series (or stochastic process) is a function Xpt, ωq such that for

Transcription

1 Chapter 1 Basics 1.1 Definition A time series (or stochastic process) is a function Xpt, ωq such that for each fixed t, Xpt, ωq is a random variable [denoted by X t pωq]. For a fixed ω, Xpt, ωq is simply a function of t and is called a realization of the stochastic process. $'& '% Time Domain Approach Frequency Domain Approach $'& '% Continuous Time Discrete Time 11

2 12 CHAPTER 1. BASICS 1.2 Characterization of a time series A time series is characterized by the joint distribution function of any subset X t1,, X tn that is F Xt1,, X tn px t1,, x tn q. Using the joint distribution function, define µ t EpX t q γ t,s CovpX t, X s q, provided that they exist. The first two monents, {µ t }, {γ t,s } completely characterize a Gaussian process. F Xt1,, X tn is in general, very difficult to analyze. In particular, estimation of {µ t }, {γ t,s } appears to be impossible unless a number of different realizations, i.e. repeated observations are available. 1.3 Stationarity Assume F Xt1,,X tn is invariant under transformation of the time indices, that is F Xt1 h,, X tn h px t1,, x tn q F Xt1,,X tn px t1,, x tn q p1q for all sets of indices (t 1,, t n ). This is called the strict stationarity.

3 1.3. STATIONARITY 13 Under this assumption, the joint distribution function depends only on the distance between the elements in the index set. If tx t u is strictly stationary and E X t 8, then EpX t q p2q CovpX t, X s q γ t s p3q tx t u is said to be weakly stationary (or covariance stationary or simply, stationary ) if (2) and (3) holds. REMARKS 1) (Weak) stationarity does not imply strict stationarity. Nor does strict stationarity imply (weak) stationarity. e.g. A strict stationary process may not possess finite moments (e.g. Cauchy). 2) For a Gaussian process, weak and strict stationarity are equivalent.

4 14 CHAPTER 1. BASICS µ and γ t s can be estimated by pµ s X 1 T pγ h c h 1 T Ţ X t t1 Ţ th 1 px t s XqpXt h s Xq, h 0, 1,. If the process is also ergodic (average asymptotic independence), s X and c h are consistent estimators of µ and γ h. 1.4 Autocovariance and Autocorrelation Functions The sequence tγ h u viewed as a function of h is called the autocovariance function. The autocorrelation function is defined by note ρ 0 1 ρ h γ h γ 0, Example 1. (White noise process) tx t u, X t iid p0, σ 2 q, 0 σ 2 8. µ 0, γ h $'& '% σ2 if h 0, 0 otherwise.

5 1.4. AUTOCOVARIANCE AND AUTOCORRELATION FUNCTIONS 15 Example 2. (MA(1) process) Let tε t u be a white noise process with finite variance σ 2. Let X t ε t θε t 1. Then µ EpX t q 0 γ 0 Epε t θε t 1 q 2 Epε 2 t q θ2 Epε 2 t 1 q p1 θ 2 qσ 2 γ h Erpε t θε t 1 qpε t h θε t h 1 qs $'& '% θσ2 if h 1, 0 if h 2. Therefore, ρ h $'& '% 1 if h 0, θ 1 θ 2 if h 1, 0 otherwise.

6 16 CHAPTER 1. BASICS Suppose θ 0.6. Then ρ ρ h h ρ h ρphq : the autocorrelation function. Sample mean of tx t u T t1 s XT Sample variance = c 0 pγ 0 Sample autocovariance = c h, h 1, 2, pγ h Sample autocorrelation = r h por pρ h q c h c 0, h 1, 2, Note r 0 1.

7 1.5. LAG OPERATOR 17 A plot of r h against h 0, 1, is called correlogram. r h h The sample autocorrelations are estimates of the corresponding theoretical autocorrelations and are therefore subject to sampling errors. 1.5 Lag Operator The operator L (sometimes denoted by B) is defined by LX t X t 1.

8 18 CHAPTER 1. BASICS Formally, L operates on the whole sequence... X t 1 X t 2 L : X t ÝÑ X t 1 X t 1. X t. The lead operator L 1 (sometimes denoted by F ) is defined by L 1 X t X t 1. Successive application of L yields L h X t X t h, h 1, 2, We define L 0 X t X t. The lag operator is a linear operator : LpcX t q clx t LpX t Y t q LX t LY t and can be manipulated like a usual algebraic quantity. For example, suppose y t φy t 1 ε t, and φ 1.

9 1.6. GENERAL LINEAR PROCESS 19 Then, p1 φlqy t ε t So, y t 8 ε t 1 φl pφlq j ε t 1 L is called the difference operator. 2 y t p1 Lq 2 y t p1 2L L 2 qy t = y t 2y t 1 y t 2 2 y t p y t q py t y t 1 q = py t y t 1 q py t 1 y t 2 q y t 2y t 1 y t General Linear Process y t ε t ψ 1 ε t 1 ψ 2 ε t 2 ψplqε t ÐÝ linear process where ε t iid p0, σ 2 q and ψplq 1 ψ 1 L ψ 2 L 2 polynomial in lag operators ψplq is sometimes called transfer function

10 20 CHAPTER 1. BASICS A time series ty t u can be viewed as the result of applying a backward (linear) filter to a white noise process. ε t input ÝÑ linear filter ψplq output ÝÑ Y t ψplqε t The sequence tψ j : j 0, 1, u can be finite or infinite. If it is finite of order q, we obtain MApqq process. This is clearly a stationary process. If 8 tψ j u is infinite, we usually assume it is absolutely summable. i.e. ψ j 8. Then the resulting process is stationary. To see this, µ 0 γ 0 σ 2 γ h σ 2 8 ψ j 0 8 ψj 2 σ ψ j 8 8 ψ j ψ j h 8 σ 2 ψ j ψ j h σ ψ j 8

11 1.7. AUTOREGRESSIVE PROCESS 21 The stationary condition is embodied in the condition that ψpzq must converge for z 1, i.e. for z on or within the unit circle. Note that absolute summability of tψ j u is sufficient but not necessary for stationarity. 7 8 ψ j z j 8 ψ j Autoregressive Process The process y t defined by y t φ 1 y t 1 φ p y t p ε t p q is called a p-th order autoregressive process and is denoted by y t ARppq The equation p q is sometimes called a stochastic difference equation.

12 22 CHAPTER 1. BASICS First - Order Autoregressive Process ARp1q is given by y t φy t 1 ε t. By successive substitution, y t ε t φy t 1 ε t φε t 1 φ 2 y t 2 ε t φε t 1 φ 2 ε t 2 φ 3 y t 3 J 1 φ j ε t j φ J y t J, implying Epy t y t J q φ J y t J If φ 1, the value of y t J can affect the prediction of future y t, no matter how far ahead. If φ 1, J 1 y t lim φ j ε t j JÑ8 8 φ j ε t j. lim JÑ8 φ J y t J 8 Note that φ j 8 φ j 1 1 φ 8, if φ 1,

13 1.7. AUTOREGRESSIVE PROCESS 23 so, tφ j u is absolutely summable and ty t u is a linear process. ψplq p1 φlq 1 8 φ j L j, or ψ j φ j Now, Epy 2 t q Epφ2 y 2 t 1 ε2 t 2φy t 1ε t q because Epy t 1 ε t q γ 0 φ 2 γ 0 σ 2 ErEpy t 1 ε t y t 1 qs Ery t 1 Epε t y t 1 qs Epy t 1 0q 0 Therefore, γ 0 σ2 1 φ 2 8, if φ Second - Order Autoregressive Process ARp2q is given by y t φ 1 y t 1 φ 2 y t 2 ε t or φplqy t ε t, where, φplq 1 φ 1 L φ 2 L 2.

14 24 CHAPTER 1. BASICS Now, suppose φpzq can be written as φpzq p1 λ 1 zqp1 λ 2 zq. Then ψpzq φ 1 pzq 1 p1 λ 1 zqp1 λ 2 zq K 1 1 λ 1 z where K 1 λ 1 λ 1 λ 2 and K 2 λ 2 λ 1 λ 2. K 2 1 λ 2 z, Therefore, ψpzq converges for z 1, iff λ 1 1 and λ 2 1. In other words, for ARp2q process to be stationary, the roots of φpzq 1 φ 1 z φ 2 z 2 must lie outside the unit circle. Note that φp0q 1 0. Let m 1 and m 2 be the roots. The necessary and sufficient condition for m 1 1 and m 2 1 are m 1 m 2 1 φ 2 1 ðñ φ 2 1 φp1q 1 φ 1 φ 2 0 φp 1q 1 φ 1 φ 2 0

15 1.7. AUTOREGRESSIVE PROCESS y y 1 φ 1 z φ 2 z 2 Õ z 1 To have real roots, φ 2 1 4φ φ 2 1 real roots Õ φ complex roots

16 26 CHAPTER 1. BASICS p - th Order Autoregressive Process ARppq is given by φplqy t ε t, where φplq 1 φ 1 L φ p L p. This process is stationary if all characteristic roots of φpzq 0 lie outside the unit circle. 1.8 Autocovariance and Autocorrelation Functions AR(1) Process y t φy t 1 ε t Then, Epy t 1 y t q Epφyt 1 2 ε ty t 1 q φepyt 1 2 q. So that γ 1 φγ 0, ρ 1 φρ 0 φ.

17 1.8. AUTOCOVARIANCE AND AUTOCORRELATION FUNCTIONS 27 we have Similarly, and noticing that Epy t h y t q Erφy t h y t 1 y t h ε t s φγ h 1, γ h φγ h 1 φ h γ 0 ρ h φ h ρ τ exponential decay φ 0.6 h ρ τ damped oscillation φ 0.6 h AR(2) Process y t φ 1 y t 1 φ 2 y t 2 ε t Then Epy t y t h q φ 1 Epy t 1 y t h q φ 2 Epy t 2 y t h q Epε t y t h q p1q γ h φ 1 γ h 1 φ 2 γ h 2, h 1, 2, ρ h φ 1 ρ h 1 φ 2 ρ h 2, h 1, 2,. p2q

18 28 CHAPTER 1. BASICS Setting h 1 in (2) yields ρ 1 φ 1 ρ 0 φ 2 ρ 1 ρ 2 φ 1 ρ 1 φ 2 φ 1 1 φ 2 φ2 1 1 φ 2 φ 2 Setting h 0 in (1) yields γ 0 φ 1 γ 1 φ 2 γ 2 σ 2 γ 0 p1 φ 1 ρ 1 φ 2 ρ 2 q σ 2 γ 0 p1 φ2 1 φ2 1φ 2 φ φ 2 1 φ q σ2 2 1 φ2 γ 0 rp1 φ 2 q 2 φ φ s σ2 2 6 γ 0 1 φ2 1 φ 2 σ 2 rp1 φ 2 q 2 φ 2 1s AR(p) Process Epy t y t 1 q Epφ 1 y 2 t 1 φ py t p y t 1 ε t y t 1 q. γ 1 φ 1 γ 0 φ p γ p 1 γ 0 φ 1 γ 1 φ p γ p σ 2.

19 1.9. MOVING AVERAGE PROCESS 29 If h p, γ h φ 1 γ h 1 φ p γ h p Note ρ h φ 1 ρ h 1 φ p ρ h p ρ h ÝÑ 0, as h ÝÑ 8 for AR(p) process. 1.9 Moving Average Process The process y t which is defined by y t ε t θ 1 ε t 1 θ q ε t q is called a q -th order moving average process or MApqq process. Write y t θplqε t, where, θplq 1 θ 1 L θ q L q Finite MA process can be regarded as the output y t from a linear filter with transfer function θplq when the input is WN (White Noise), ε t. Finite M A process is therefore stationary.

20 30 CHAPTER 1. BASICS The autocovariances are γ 0 Ey 2 t Epε t θ 1 ε t 1 θ q ε t q q 2 σ 2 p1 θ 2 1 θ2 q q σ2 q θ 2 j, (where θ 0 1). γ h Epy t y t h q Erpε t θ 1 ε t 1 θ q ε t q qpε t h θ 1 ε t h 1 θ q ε t h q qs σ 2 pθ h θ 1 θ h 1 θ q h θ q q, for h q σ 2 q jh θ j h θ j (convolution). γ h 0, for h q. Note that there is a cut off q. This contrasts with ARppq process, for which γ h 0, for any h. The h th order autocorrelation function ρ h θ h θ 1 θ h 1 θ q h θ q, for h 1 θ1 2 1, 2,, q θq 2 ρ h 0 for h q The autocorrelation function for MA process has a cut - off at h q. Write ΠpLqy t ε t y t θplqε t

21 1.9. MOVING AVERAGE PROCESS 31 If MA process is invertible, ΠpLq θplq 1. The invertiblity condition for M Apqq process is that all roots of characteristic equation θpzq 0 lie outside the unit circle. Write y t q¹ q¹ j1 If it is invertible, j1 p1 ω j Lqε t, p1 ω j L ω 2 j L 2 qy t ε t, that is, the finite order MA process is transformed into the infinite order AR process MA(1) Process y t ε t θε t 1 p1 θlqε t. θ must lie in the range (-1, 1) for the process to be invertible. γ 0 p1 θ 2 qσ 2 γ 1 θσ 2 γ h 0 for h 2, 3,

22 32 CHAPTER 1. BASICS ρ 1 θ 1 θ 2 ρ h 0 for h 2, 3, p q It follows from p q that θ 2 ρ 1 1 θ 1 0 which implies that if θ s θ is a solution, then so is θ s θ 1. So, if θ s θ satisfies the invertibility condition, the other root θ s θ 1 will not satisfy the condition Mixed Autoregressive - Moving Average Model The process y t which is defined by y t φ 1 y t 1 φ p y t p ε t θ 1 ε t 1 θ q ε t q p1q is called a (mixed) autoregressive - moving - average progress of p and q -th order, or ARMApp, qq process. This may be thought of as a p-th order autoregressive process, φplqy t e t, with e t following the q-th order moving average process, e t θplqε t, or alternatively, as an MApqq process y t θplqη t, with η t following ARppq process φplqη t ε t.

23 1.10. MIXED AUTOREGRESSIVE - MOVING AVERAGE MODEL 33 ARMApp, qq process (1) is stationary if φpzq 0 has all its roots lying outside the unit circle, and is invertible if all roots of θpzq 0 lie outside the unit circle. On multiplying throughout in (1) by y t h and taking expectations, we see that the autocovariance function satisfies the deterministic difference equation γ h φ 1 γ h 1 φ p γ h p γ yε phq θ 1 γ yε ph 1q θ q γ yε ph qq p2q where γ yε p q is the cross covariance function between y and ε, and is defined as γ yε phq Epy t h ε t q. It is easy to see γ yε phq 0, for h 0 γ yε phq 0, for h 0. Hence, γ h φ 1 γ h 1 φ p γ h p, for h q 1 and, ρ h φ 1 ρ h 1 φ p ρ h p, for h q 1 or, φplqρ h 0, for h q 1 ÐÝ look like ARppq p3q

24 34 CHAPTER 1. BASICS Setting h 0 in (2), we have γ 0 φ 1 γ 1 φ p γ p σ 2 θ 1 γ yε p 1q θ q γ yε p qq p4q From (3), we see that the autocorrelation function for the mixed process eventually takes the same shape as that of AR process φplqy t ε t ARMA(1, 1) process y t φy t 1 ε t θε t 1 p5q p1 φlqy t p1 θlqε t The process is stationary if φ 1, and invertible if θ 1. Using (2) and (4), we obtain γ 0 φγ 1 σ 2 θγ yε p 1q γ 1 φγ 0 θσ 2 γ h φγ h 1, for h 2, 3, Note that γ yε p 1q Epy t ε t 1 q Epy t 1 ε t q So, on multiplying throughout (5) by ε t 1, and taking expectations, we obtain γ yε p 1q pφ θqσ 2

25 1.10. MIXED AUTOREGRESSIVE - MOVING AVERAGE MODEL 35 Hence, γ 0 φpφγ 0 θσ 2 q σ 2 θpφ θqσ 2 1 θ2 2φθ 1 φ 2 σ 2 γ 1 φp1 θ2 2φθq θp1 φ 2 q 1 φ 2 σ 2 pφ θqp1 φθq 1 φ 2 σ 2 γ h φγ h 1, for h 2, 3, Ð look like ARp1q and so ρ 1 pφ θqp1 φθq 1 θ 2 2φθ ρ h φρ h 1, for h 2, 3, ρ h 1 exponential decay 0.5 φ 0.6 θ h [ρ , ρ , ρ , ρ , ρ ]

26 36 CHAPTER 1. BASICS Theorem (Granger s Lemma) If X t ARMApp, mq and Y t ARMApq, nq, if X t and Y t are independent, and if Z t X t Y t, then Z t ARMApr, lq, where r p q, and l maxpp n, q mq. P roof. Let a 1 plqx t b 1 plqε t and a 2 plqy t b 2 plqη t, where a 1, a 2, b 1, b 2 are polynomials in L of order p, q, m, n respectively, and ε t, η t are independent W N processes. Multiplying Z t X t Y t by a 1 plqa 2 plq, we obtain a 1 plqa 2 plqz t a 1 plqa 2 plqx t a 1 plqa 2 plqy t a 2 plqb 1 plqε t a 1 plqb 2 plqη t. Since a 2 plqb 1 plqε t MApq mq and a 1 plqb 2 plqη t MApp nq, RHS MAplq, where l maxpp n, q mq. The order of a 1 plqa 2 plq is not more than p q, and hence the theorem follows. The inequalities are needed in the theorem since a 1 plq and a 2 plq may contain common roots. REMARKS (1) The theorem implies that if the series analyzed is the sum of two independent ARp1q, then the series will be ARM Ap2, 1q,

27 1.11. AUTOCOVARIANCE GENERATING FUNCTION 37 for example. If the observed series is the sum of a true ARppq process plus a W N measurement error, then an ARMApp, pq process results. (2) Mixed model may achieve as good a fit as the AR model but using fewer parameters. Principle of parsimony (Box and Jenkins) 1.11 Autocovariance Generating Function A compact and convenient way of recording the information contained in a sequence ta j u is by means of the generating function : apzq j a j z j, where z is a possibly complex variable. The individual members of the sequence can easily be recovered from the coefficients associated with the z j s. The quantity z does not necessarily have any interpretation and should be simply considered as the carries of the information in the sequence. Now define γpzq ρpzq 8 8 h 8 h 8 mptq Epe tx q φptq Epe itx q γ h z h ρ h z h autocovariance generating function autocorrelation generating function Moment Generating Function Characteristic Function

28 38 CHAPTER 1. BASICS Useful properties of generating functions : (1) Additivity c j a j b j ÝÑ cpzq apzq bpzq (2) Convolution c j j k0 a k b j k apzq ÝÑ cpzq apzqbpzq 8 a j z j, j 0, 1, 2, Given X 1, X 2,, z - transform Xpzq ţ X t z t Now, let X T pzq Ţ t1 then, ErX T pzqx T pz 1 qs X t z t, Ţ h T pt h qγ h z h. Dividing by T and passing to the limit, we obtain, 1 γpzq lim T Ñ8 T ErX T pzqx T pz 1 qs. Let S t y t y N S N Ş ψ j ε N S j Ş ψ j ε S N j t N ψ j ε t j Ñ 8 ψ j ε t j as N ÝÑ 8 c 0 a 0 b 0 c 1 a 0 b 1 a 1 b 0 c 2 a 0 b 2 a 1 b 1 a 2 b 0 pa 0 a 1 z a 2 z 2 qpb 0 b 1 z b 2 z 2 q a 0 b 0 pa 0 b 1 a 1 b 0 qz pa 0 b 2 a 1 b 1 a 2 b 0 qz 2

29 1.11. AUTOCOVARIANCE GENERATING FUNCTION 39 y 0 ε 0 y 1 ε 1 ψ 1 ε 0 y 2 ε 2 ψ 1 ε 1 ψ 2 ε 0. y T Ţ ψ j ε t j Define, y T pzq ε T pzq ψ T pzq Ţ t0 Ţ t0 Ţ t0 y t z t ε t z t ψ t z t Suppose ty t u is a linear process. Then, Y pzq ψpzqεpzq. 1 Hence, γ Y pzq lim T Ñ8 T Erψ T pzqε T pzqψ T pz 1 qε T pz 1 qs ψpzqψpz 1 qσ MA(q) process y t θplqε t γpzq σ 2 θpzqθpz 1 q

30 40 CHAPTER 1. BASICS Note that θpzqθpz 1 q q q k0 q z h θ j θ k z j k q h q θ j θ j h by setting j k h, and taking θ j 0 for j q. For MA(1), y t ε t θε t 1 θpzqθpz 1 q σ 2 p1 θzqp1 θz 1 q $'& γ 0 p1 θ 2 qσ 2 $'& θ 0 1 σ 2 rp1 θ 2 q θz θz 1 s '% 1 γ 1 θσ 2 γ h 0, for h 2, 3, z h 1 h 1 q 1 '% θ 1 θ θ h 0 for h 0, h 1 θ j θ j h pθ 2 0 θ2 1 q pθ 0θ 1 θ 1 θ 2 qz pθ 0 θ 1 θ 1 θ 0 qz 1 p1 θ 2 q θz θz AR(p) process φplqy t ε t ùñ y t ψplqε t and in generating function form

31 1.11. AUTOCOVARIANCE GENERATING FUNCTION 41 φpzqypzq εpzq ùñ ypzq ψpzqεpzq, where, φpzqψpzq 1. Therefore, γpzq σ 2 ψpzqψpz 1 q σ 2 φpzqφpz 1 q For AR(1), y t φy t 1 ε t γpzq γ 0 σ 2 p1 φzqp1 φz 1 q p1 φz φ 2 z 2 qp1 φz 1 φ 2 z 2 qσ 2 rp1 φ 2 φ 4 q pφ φ 3 φ 5 qz sσ 2 σ2 1 φ 2, γ h φ h γ ARMA(p, q) process γpzq σ 2 θpzqθpz 1 q φpzqφpz 1 q

32 42 CHAPTER 1. BASICS