Lecture 1: Fundamental concepts in Time Series Analysis (part 2)

Transcription

1 Lecture 1: Fundamental concepts in Time Series Analysis (part 2) Florian Pelgrin University of Lausanne, École des HEC Department of mathematics (IMEA-Nice) Sept Jan Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

2 Stationarity, stability, and invertibility 6. Stationarity, stability, and invertibility Consider again a situation where the value of a time series at time t, X t, is a linear function of a constant term, the last p values of X t, the contemporaneous and last q values of a white noise process, denoted by ɛ t : X t = µ + This process rewrites : p φ k X t k + k=1 q θ j ɛ t j. j=0 Φ(L)X }{{} t = µ + Θ(L)ɛ }{{} t Autoregressive part Moving average part with Φ(L) = 1 φ 1 L φ 2 L 2 φ p L p and Θ(L) = 1 + θ 1 L + + θ q L q. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

3 Which conditions? Stationarity, stability, and invertibility Stationarity conditions regard the autoregressive part of the previous (linear) time series model (ARMA(p,q) model). Stability conditions also regard the autoregressive part of the previous (linear) time series model (ARMA(p,q) model). Stability conditions are generally required to avoid explosive solutions of the stochastic difference equation : Φ(L)X t = µ + Θ(L)ɛ t. Invertibility conditions regard the moving average part Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

4 Stationarity, stability, and invertibility Implications of the stability (stationarity) conditions If stability conditions hold, stationarity conditions are satisfied (the converse is not true) and (X t ) is weakly stationary. If the stochastic process (X t ) is stable (and thus weakly stationary), then (X t ) has an infinite moving average representation (MA( ) representation) : X t = Φ 1 (L) (µ + Θ(L)ɛ t ) = Φ 1 (1)µ + C(L)ɛ t µ = 1 p k=1 φ + c i ɛ t i k where the coefficients c i satisfy c i <. i=0 Florian Pelgrin (HEC) Univariate time series Sept Jan / 40 i=0

5 Stationarity, stability, and invertibility This representation shows the impact of past shocks, ɛ t i, on the current value of X t : dx t dɛ t i = c i This is an application of the Wold s decomposition Omitting the constant term, X t and ɛ t are linked by a linear filter in which C(L) = Φ 1 (L)Θ(L) is called the transfer function. The coefficients (c i ) are also referred to as the impulse response function coefficients. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

6 Stationarity, stability, and invertibility If (X t ) t Z is weakly stationary but not stable, then (X t ) has the following representation : X t = Φ 1 (L) (µ + Θ(L)ɛ t ) = Φ 1 (1)µ + C(L)ɛ t = µ 1 p k=1 φ + k c i ɛ t i. where (c j ) is sequence of constants with : i= c i < i= The latter condition insures that (X t ) t Z is weakly stationary. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

7 Stationarity, stability, and invertibility Determination of the stability and stationarity conditions Definition Consider the stochastic process defined by : Φ(L)X t = µ + Θ(L)ɛ t with Φ(L) = 1 φ 1 L φ 2 L 2 φ p L p, Θ(L) = 1 + θ 1 L + + θ q L q, and (ɛ t ) is a white noise process. This process is called stable if the modulus of all the roots, λ i, i = 1,, p, of the (reverse) characteristic equation : Φ(λ) = 0 1 φ 1 λ φ 2 λ 2 φ p λ p = 0 are greater than one, λ i > 1 for all i = 1,, p. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

8 Stationarity, stability, and invertibility Implications of the stability (stationarity) conditions Invertibility is the counterpart to stationarity for the moving average part of the process. If the stochastic process (X t ) is invertible, then (X t ) has an infinite autoregressive representation (AR( ) representation) : Θ 1 (L)Φ(L)X t = Θ 1 (L)µ + ɛ t or X t = µ 1 q k=1 θ k + d i X t i + ɛ t i=1 where the coefficients d i satisfy d i <. i=0 Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

9 Stationarity, stability, and invertibility The AR( ) representation shows the dependence of the current value X t on the past values of X t i. The coefficients are referred as the d-weights of an ARMA model. If the stochastic process (X t ) is not invertible, it may exist a representation of the following form (omitting the constant term) as long as the magnitude of any (characteristic) root of Θ(λ) does not equal unity : X t = i 0 d i X t i + ɛ t Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

10 Stationarity, stability, and invertibility Determination of the invertibility conditions Definition Consider the stochastic process defined by : Φ(L)X t = µ + Θ(L)ɛ t with Φ(L) = 1 φ 1 L φ 2 L 2 φ p L p, Θ(L) = 1 + θ 1 L + + θ q L q, and (ɛ t ) is a white noise process. This process is called invertible if the modulus of all the roots, λ i, i = 1,, q, of the (reverse) characteristic equation : Θ(λ) = θ 1 λ + θ 2 λ θ q λ q = 0 are greater than one, λ i > 1 for all i = 1,, q. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

11 Stationarity, stability, and invertibility Three equivalent representations Definition A stable (and thus weakly stationary), invertible stochastic process, Φ(L)X t = µ + Θ(L)ɛ t has two other equivalent representations forms : 1 An infinite moving average representation : X t = µ 1 p k=1 φ k + c i ɛ t i i=0 2 An infinite autoregressive representation : X t = µ 1 q k=1 θ k + d i X t i + ɛ t i=1 Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

12 Stationarity, stability, and invertibility Each of these representations can shed a light on the model from a different perspective. For instance, the stationarity properties, the estimation of parameters, and the computing of forecasts can use different representations forms (see further). Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

13 Stationarity, stability, and invertibility Examples 1. An autoregressive process of order 1 : where ɛ t WN(0, σ 2 ɛ ) for all t. (1 ρl)x t = ɛ t The stability and (weak) stationarity of (X t ) depends on the (reverse) characteristic root of Φ(λ) = 1 ρλ : The stability condition writes : Φ(λ) = 0 λ = 1 ρ. λ > 1 ρ < 1. The stationarity condition writes : λ 1 ρ 1. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

14 Stationarity, stability, and invertibility Therefore, 1 If ρ < 1, then (X t ) is stable and weakly stationary : (1 ρl) 1 = ρ k L k and X t = ρ k ɛ t k k=0 k=0 2 If ρ > 1, then a non-causal stationary solution (X t ) exists : (1 ρl) 1 = ρ k F k and X t = ρ k ɛ t+k k=1 k=1 3 If ρ = 1, then (X t ) is not stable and stationary : (1 ρl) cannot be inverted! Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

15 Stationarity, stability, and invertibility 2. A moving average process of order 1 : X t = ɛ t θɛ t 1 where ɛ t WN(0, σ 2 ɛ ) for all t. (X t ) is weakly stationary irrespective of the characteristic root of Θ(λ) = 1 θλ (why?). The invertibility of (X t ) depends on the (reverse) characteristic root of Θ(λ) = 1 θλ : Θ(λ) = 0 λ = 1 θ. The invertibility condition writes : λ > 1 θ < 1. and X t = θ k X t k + ɛ t. k=1 Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

16 7. Identification tools Identification tools Need some tools to characterize the main properties of a time series. Among others : Autocovariance function Autocorrelation function (ACF) Sample autocorrelation function (SACF) Partial autocorrelation function (PACF) Sample partial autocorrelation function (SPACF) Spectral density, etc. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

17 Keep in mind that one seeks to identify, estimate and forecast the following ARMA(p,q) model : X t = µ + p φ i X t i + i=1 q θ k ɛ t k k=0 where θ 0 = 0, ɛ t WN(0, σ 2 ɛ ), p and q are unknown orders to identify, (µ, φ 1,, φ p, θ 1,, θ q, σ 2 ɛ ) are unknown parameters to estimate (that depends on p and q). The ACF and PACF functions are used to identify the appropriate time series model : 1 The orders p and q can be identified by using the (sample) autocorrelation and (sample) partial autocorrelation function. 2 The corresponding parameters can be then estimated using several statistical procedures. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

18 Autocovariances The autocovariance function Definition The autocovariance function of a stationary stochastic process (X t ) t Z is defined to be : γ : Z R h γ X (h) = Cov(X t, X t h ). with : γ X (h) = γ X ( h). Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

19 Autocovariances Definition An estimator of γ X (h) (for h < T ) is defined to be : or ˆγ X (h) = 1 T T h (X t X T )(X t+h X T ) t=1 ˆγ X (h) = 1 T h T t=h+1 (X t X T )(X t h X T ). Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

20 Autocovariances Proposition If (X t ) t Z is a second order stationary process with : X t = m + j Z a j ɛ t j where the ɛ t s are i.i.d. with mean zero and variance σɛ, 2 a j <, and E(ɛ 4 t ) <, then (i) ˆγ X (h) is an almost surely convergent and asymptotically unbiased estimator, (ii) the asymptotic disbribution is given by : T ˆγ X (0) γ X (0) ˆγ X (1) γ X (1). ˆγ X (h) γ X (h) where Ω γ = [Ω j,k ] 0 j,k h is such that : Ω j,k = + l N (0, Ω γ ) l= Florian Pelgrin (HEC) Univariate time series Sept Jan / 40 j Z γ X (l) (γ X (l j + k) + γ X (l j k)).

21 Autocorrelations The autocorrelation function Definition The autocorrelation function of a stationary stochastic process (X t ) t Z is defined to be : h Z. ρ X (h) = γ X (h) γ X (0) = Corr(X t, X t h ) The autocorrelation function is obtained after re-scaling the autocovariance function by the variance γ X (0) = V(X t ). Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

22 Autocorrelations Definition The autocorrelation function of a (stationary) stochastic process (X t ) satisfies the following properties : 1 ρ X ( h) = ρ X (h) h 2 ρ X (0) = 1 3 The range of ρ X is [ 1; 1]. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

23 Autocorrelations The autocorrelation (respectively, autocovariance) function of a moving average process of order q, MA(q), is always zero for orders higher than q ( h > q) : MA(q) process has no memory beyond q periods. The autocorrelation (respectively, autocovariance) function of a stationary AR(p) process exhibits exponential decay towards zero (but does not vanish for lags greater than p). The autocorrelation (respectively, autocovariance) function of a stationary ARMA(p, q) process exhibits exponential decay towards zero : it does not cut off but gradually dies out as h increases. The autocorrelation function of a nonstationary process decreases very slowly even at very high lags, long after the autocorrelations from stationary processes have declined to (almost) zero. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

24 Autocorrelations The sample autocorrelation function Definition Given a sample of T observations, x 1,, x T, the sample autocorrelation function, denoted by (ˆρ X (h)), is computed by : ˆρ X (h) = T t=h+1 (x t ˆµ)(x t h ˆµ) T (x t ˆµ) 2 t=1 where ˆµ is the sample mean : ˆµ = 1 T T x t. t=1 Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

25 Autocorrelations Proposition If (X t ) t Z is a second order stationary process with : X t = m + j Z a j ɛ t j where the ɛ t s are i.i.d. with mean zero and variance σɛ 2, a j <, and E(ɛ 4 t ) <, then T ˆρ X (1) ρ X (1). ˆρ X (h) ρ X (h) l N (0, Ω ρ ). j Z Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

26 Partial autocorrelation The partial autocorrelation function The partial autocorrelation function is another tool for identifying the properties of an ARMA process. It is particularly useful to identify pure autoregressive process (AR(p)). A partial correlation coefficient measures the correlation between two random variables at different lags after adjusting for the correlation this pair may have with the intervening lags : the PACF thus represents the sequence of conditional correlations. A correlation coefficient between two random variables at different lags does not adjust for the influence of the intervening lags : the ACF thus represents the sequence of unconditional correlations. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

27 Partial autocorrelation Definition The partial autocorrelation function of a stationary stochastic process (X t ) t Z is defined to be : h > 0. a X (h) = Corr(X t, X t h X t 1,, X t h+1 ) Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

28 Partial autocorrelation Definition The partial autocorrelation function of order h is defined to be : [ ] a X (h) = Corr X t, X t h ( Cov Xt, X ) t h = [ ( ) ( )] 1/2 V X t V X t h where X t = X t EL(X t X t 1,, X t h+1 ) X t h = X t h EL(X t h X t 1,, X t h+1 ). Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

29 Partial autocorrelation Definition The partial autocorrelation function of a second order stationary stochastic process (X t ) t Z satisfies : a X (h) = R (h) R(h) where R(k) is the autocorrelation matrix of order h définie par : ρ X (0) ρ X (1) ρ X (h 1) ρ X (1) ρ X (0) ρ X (h 2) R(h) = ρ X (h 1) ρ X (h 2) ρ X (0) and R (h) is the matrix that is obtained after raplacing the last column of R(h) by ρ = (ρ X (1),, ρ X (k)) t. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

30 Partial autocorrelation Definition The partial correlation function can be viewed as the sequence of the h-th autoregressive coefficients in a h-th order autoregression. Let a hl denote the l-th autoregressive coefficient of an AR(h) process : Then for h = 1, 2, X t = a h1 X t 1 + a h2 X t a hh X t h + ɛ t. a X (h) = a hh Remark : The theoretical partial autocorrelation function of an AR(p) model will be different from zero for the first p terms and exactly zero for higher order terms (why?). Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

31 Partial autocorrelation The sample partial autocorrelation function The sample partial autocorrelation function can be obtained by different methods : 1 Find the ordinary least squares (or maximum likelihood) estimates of a hh 2 Use the recursive equations of the autocorrelation function (Yule-Walker equations) after replacing the autocorrelation coefficients by their estimates (see further). 3 Etc. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

32 Partial autocorrelation Proposition If (X t ) t Z is a second order stationary process with : X t = m + j Z a j ɛ t j where the ɛ t s are i.i.d. with mean zero and variance σɛ 2, a j <, and E(ɛ 4 t ) <, then T â X (1) r X (1). â X (h) r X (h) l N (0, Ω r ). j Z Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

33 Spectral analysis Identification tools Spectral analysis Definition (Fourier transform of the autocovariance function) Let (X t ) be a real-valued stationary process (with absolutely summable autocovariance sequence). The Fourier transform of the autocovariance function (γ X (h)) exists and is given by ; ω [ π; π]. f X (ω) = 1 h=+ γ X (h)exp( iωh) 2π h= = 1 h=+ γ X (h)cos(ωh) 2π h= = 1 2π γ X (0) + 1 h=+ γ X (h)cos(ωh) π h=1 Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

34 Spectral analysis Properties : 1. The spectral density satsifies : f X (ω) is continuous, i.e. f X (ω) = f X (ω), f X (ω) is real-valued ; f X (ω) is a nonnegative function (since the autocovariance function is positive semidefinite). 2. f X (ω) = f X (ω + 2π) : f X is a periodic with period 2π. 3. f X (ω) = f X ( ω) ω : f X is a symmetric even function. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

35 Spectral analysis Properties (cont d) : 4. The variance satisfies : V(X t ) = γ X (0) = π π f X (ω)dω The spectrum f X (ω) may be interpreted as the decomposition of the variance of a process. The term f X (ω)dω is the contribution to the variance attributable to the component of the process with frequencies in the interval (ω, ω + dω). A pick (in the spectrum) indicates an important contribution to the variance. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

36 Spectral analysis Deriving the spectral density function Definition (Autocovariance generating function) For a given sequence of autocovariances γ X (h), h = 0, ±1, ±2, cdots, the autocovariance generating function is defined to be : γ X (L) = where L is the lag operator. + h= γ X (h)l h Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

37 Spectral analysis Proposition Let (X t ) t Z denote a second order stationary process with : X t = m + j Z θ j ɛ t j where the ɛ t s are i.i.d. with mean zero and variance σ 2 ɛ. The autocovariance generating function is defined to be : γ X (L) = σ 2 ɛ Θ(L)Θ(L 1 ). Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

38 Spectral analysis Definition Let γ X (L) and f X (ω) denote respectively the autocovariance generating function and the spectrum : γ X (L) = + f X (ω) = 1 2π h= + γ X (h)l h h= exp( iωh) ω [ π; π]. Then f X (ω) = 1 2π γ X (exp( iω)). Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

39 Spectral analysis Application : Consider a general stationary ARMA(p,q) model Φ p (L)X t = µ + Θ q (L)ɛ t with Φ p (L) = 1 p j=1 φ jl j and Θ q (L) = 1 + q j=1 θ jl j sharing no common factor and having their roots outside the unit circle. Then and γ X (L) = σ 2 ɛ Θ q (L)Θ q (L 1 ) Φ q (L)Φ q (L 1 ) f X (ω) = 1 2π γ X (exp( iω)) = σ2 ɛ Θ q (exp( iω))θ q (exp(iω)) 2π Φ q (exp( iω))φ q (exp(iω)) = σ2 ɛ Θ q (exp( iω)) 2 2π Φ p (exp( iω)). Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

40 Summary Summary Strong and weak stationarity Stationarity/nonstationarity White noise, Trend stationary processes, Difference stationary processes (random walk) Lag operator and writing of time series models AR( ) and MA( ) representation Autocovariance, (sample) autocorrelation, and (sample) partial autocorrelation function Spectral density. Class of ARMA(p,q) processes. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40