1.4 Properties of the autocovariance for stationary time-series

Transcription

1 1.4 Properties of the autocovariance for stationary time-series In general, for a stationary time-series, (i) The variance is given by (0) = E((X t µ) 2 ) 0. (ii) (h) apple (0) for all h 2 Z. ThisfollowsbyCauchy-Schwarzas (iii) ( h) = (h) (followstrivially). (iv) (h) = E((X t µ)(x t+h µ)) apple E((X t µ) 2 )E((X t+h µ) 2 ) 1/2 =[ (0) 2 ] 1/2 = (0). is positive semi-definite, that is for all a 2 R n (and any choice of n 2 N), a i (i j)a j 0. i,j=1 As a proof of (iv), consider the variance of (X 1,...,X n )a = P n i=1 a ix i,wherea 2 R n is a column-vector: 0 apple Var( a i X i )= i=1 which completes the proof. a i a j Cov(X i,x j )= i,j=1 a i (i j)a j, i,j=1 1.5 Estimating the auto-covariance (Chapter 1.6 in the book) For observations x 1,...,x n of a stationary time-series, estimate the mean, auto-covariance and auto-correlation as follows (i) Sample mean ˆµ = x = 1 n P n i=1 x i. (ii) Sample auto-covariance function is, for and set to 0 otherwise. ˆ(h) = 1 n n n apple h apple n h X (x t+ h x)(x t x), 8

2 (iii) Sample auto-correlation is given by ˆ (h) = ˆ(h) ˆ(0). Note that ˆ(h) is identical to the sample covariance of (X 1,X 1+h ),...,(X n h,x n ), except that we normalize by n instead of n h to keep ˆ positive semi-definite (see below). Properties of the sample ACF The four properties of the ACF are also true for the sample ACF: (iii) ˆ( h) =ˆ(h) holdstrivially. (iv) ˆ is positive semi-definite (proof below). (i)+(ii) ˆ(0) 0and ˆ(h) appleˆ(0) follows from property (iv). Proof of (iv): We can write 0 1 ˆ(0) ˆ(1) ˆ(2)... ˆ(n 1) ˆ(1) ˆ(0) ˆ(1)... ˆ(2)... ˆn =... = 1 B... n MMT, A ˆ(n 1)... ˆ(0) where the n (2n 1)-dimensional matrix M is given by X1 X2 X3... Xn 1 Xn X1 X2 X3... Xn 0 M := 0 X1 X2... Xn X 1 X2... Xn C A, where X t := X t ˆµ. Hence, for any a 2 R n, which completes the proof. a T ˆna = 1 n (at M)(M T a)= 1 n km T ak 2 2 0, 1.6 Transforming to stationarity (parts of Chapter 2) Several steps/strategies, not always in the same order 9

3 1Plotthetimeseries:lookfortrends,seasonalcomponents,stepchanges,outliersetc. 2Transformdatasothatresidualsarestationary. (a) Estimate and subtract trend T t and seasonal components S t (b) Di erencing (c) Nonlinear transformations (log, p etc.). 3 Fit a stationary model to residuals. This yields then an overall model for the data. For 2(a), we can use non-parametric estimation (with large bandwidth) to get trend T t and smoothing (with medium bandwidth) to get seasonal component. Seasonal component can also be estimated as empirical average of detrended data in, for example, each given month (if its yearly data). For 2(b), define lag-1 di erence operator via (rx) t =(1 B)X t = X t X t 1, where B is the backshift operator defined via (i) For a linear trend, that is if with N t the noise process, we have (BX) t = X t 1. X t = µ + t + N t, (1 B)X t = +(1 B)N t. If di erenced noise (1 B)N t is stationary, we can estimate slope from data as the mean of the di erenced time-series rx. (ii) For a polynomial trends+noise, that is if X t = kx j=1 jt j + N t, di erence k times to get r k X t =(1 B) k X t = k! k +(1 B) k N t. If k-times di erenced noise (1 B) k N t is stationary, can estimate highest-order term as the mean of the k-times di erenced time-series. 10

4 (iii) For a seasonal variation of length s, definelag-sdi erencingas (1 B s )X t = X t B s X t = X t X t s, where B s is the backshift operator applied s times. If and S t has period s, then X t = T t + S t + N t, (1 B s )X t = T t T t s +(1 B s )N t, and the seasonal component has been removed and we can then proceed as in (i) or (ii), depending on the nature of the trend. 1.7 Cointegration We say (X t ) t2z is integrated of oder d if the d times di erenced time-series (1 B) d X t is stationary but (1 B) d0 X t is not stationary for all d 0 <d. Let (X t ) t2z and (Y t ) t2z be two time-series and integrated of order d =1. Thetwotime-seriesaresaidtobeco-integratedifthereexists alinearcombinationofboththatisintegratedoforder0(thatisthelinearcombinationis stationary), that is there exists a 2 R 2 such that U t = X t 1 + Y t 2 is stationary. The is then clearly not unique and one can for example set 1 =0wlog. Examples are stock prices of Apple and Google (where the di erence is perhaps stationary) or economic data on money supply, income, prices and interest rates. If cointegration hold, we can model the di erence as a stationary process. Cointegration means intuitively that while the processes marginally can drift they cannot drift far apart from each other. 1.8 Regression with correlated errors (related to Chapter 2.1) We can try to estimate a linear trend in two di erent models (i) A trend-stationary model, where X t = µ + t + N t, where N t is stationary noise. 11

5 (ii) A di erence-stationary model with X t = µ + t + N t, where rn t is stationary noise. Our goal is to estimate the trend and give a confidence interval for this parameter. Note: if model (i) is correct, the model (ii) is also correct, but we lose e ciency in the estimation if we proceed with di erencing Trend-stationary models and pre-whitening (this subsection is additional voluntary reading) If model (i) is correct, we would like to use least-squares estimation to estimate the slope and derive a confidence interval as in standard least-squares regression. We can write in vector notation X = Z + N, (2) where X =(X 1,...,X n ) T, N =(N 1,...,N n ) T, Z = B 1 3 A, 1 n µ =. Note that we have chosen the special case (t 1,...,t n )=(1,...,n)inthisexampletosimplify notation. Naive least-squares estimation would then use an estimator ˆ =argmin 0kX Z 0 k 2 2. The least-squares estimator is motivated as maximum-likelihood estimator if the noise contributions are independent and have a Gaussian distribution. In a time-series context, the first assumption will be violated. Assume, for simplicity, though that the Gaussian assumption is correct and N N(0, ) for a n n full-rank matrix of the general form of stationary time-series discussed before. Then, to get the maximum-likelihood estimate argmin 0(X Z 0 ) T 1 (X Z ), 12

6 observe that (2) is equivalent to AX = AZ + AN, (3) for any full-rank matrix A 2 R n n. Now choose A such that AN N(0, 1 n ), where 1 n is the identity matrix in n dimensions. If = C T C is the Cholesky decomposition of (and C invertible since we assumed that has full rank), then such a pre-whitening matrix A is given for example by A = C T, since then Var(AN) =E(ANN T A T )=AE(NN T ) A T = AC T CA T =1 {z } n. = Alternatively, if = T is the eigenvalue decomposition of with orthogonal (specifically, T =1 n )andadiagonal 2 R n n,thenwecanequivalentlychoose A = 1/2 T as a pre-whitening matrix, where 1/2 ij =0ifi6= j and 1/2 ii since then Var(AN) =A A T = A T A T = 1/2 {z} T =1 n T =1/ p ii for i =1,...,n {z} 1/2 =1 n. =1 n Once we have such a pre-whitening matrix A, thenthemaximum-likelihoodestimatorfor is ˆ =argmin 0kAX AZ 0 k 2 2 =((AZ) T (AZ)) 1 (AZ) T (AX). The point-estimate for is thus the second entry in ˆ, ˆ = ((AZ) T (AZ)) 1 (AZ) T (AX) 2. To get a confidence interval for,weneedtoknowthevarianceof ˆ. Aconfidenceinterval is easiest to derive if ˆ if unbiased (that is E( ˆ) = ), which is true for the estimator above) and it has a Gaussian distribution (which it has under the assumption made above that N N(0, )). Otherwise some modifications are necessary. The distribution of ˆ under the Gaussian distribution for the noise is given by ˆ N(, ((AZ) T (AZ)) 1 ). Abriefargumentforthis:notethattheleastsquaresestimatorisgivenby ˆ =((AZ) T (AZ)) 1 (AZ) T (AX). 13

7 The expected value is hence as AX = AZ + AN. Thevarianceofˆ is thus given by Var(ˆ ) =Var ((AZ) T (AZ)) 1 (AZ) T AN, where AN N(0, 1 n ). Thus Var(ˆ ) =((AZ) T (AZ)) 1 (AZ) T E (AN)(AN) T {z } =1 n (AZ)((AZ) T (AZ)) T =((AZ) T (AZ)) 1. The distribution is furthermore joint normal (as its a linear combination of normal random variables) which completes the argument. Remember that ˆ is the second component in ˆ. Wethusknowthat ˆ q Var( ˆ) N(0, 1), where Var( ˆ) =(((AZ) T (AZ)) 1 ) 2,2 in our example. Thus P q apple ˆ q Var( ˆ) apple q 1, where q = 1 (1 /2) the 1 /2 quantileofastandardnormaldistribution(andfor example q for =0.05). A (1 )-confidence interval for is then given by h q ˆ qq Var( ˆ), ˆ + q Var( ˆ) i. If unsure about any of this this, please consult a textbook on regression or introductory statistics. If we estimate from the data, we will have to modify the confidence intervals accordingly (they tend to get wider) but this is beyond the scope here Di erence-stationary models For model (ii), let Z t = rx t be the di erenced time-series. The slope be estimated as the mean of the di erenced time-series, that is ˆ = 1 n Z t. in the model can To get a confidence interval, we need to know again the variance of ˆ. If we do know the variance (and assume a Gaussian distribution of ˆ for simplicity), then a confidence interval is given again by h q ˆ qq Var( ˆ), ˆ + q Var( ˆ) i, 14

8 where the quantile q of the standard normal distribution is defined just as above. Now, if Z t were independent (and Z t stationary), then Var( ˆ) =Var( 1 Z t )=Var(Z 1 )/n. n More generally, if we allow correlations and Z t has autocovariance (a) Var( 1 n Z t )= 1 n (b) If P 1 k= 1 (k) < 1, then,asn!1, nvar( 1 1X Z t )! (k) =Var(Z 1 )( n k= 1 Xn 1 1X k= 1 =Var(Z 1 )(1 + (1 (k)) 1X k= 1,k6=0 k n ) (k),then (k)) = Var(Z 1 )( X (k)) and the asymptotic variance is thus inflated (or deflated) compared to the independence case by the factor (1 + 2 P 1 k=1 (k)), which is typically larger than 1 but can also be less than 1 when negative auto-correlations appear. Proof of (a): and Var( Z t )= = = = Cov(Z t,z s ) s=1 s=1 n 1 X Xn 1 Var( 1 n (t s) (k) (number of pairs (t, s) witht s = k) {z } =n k (k) (n k ) Z t )= 1 n Var( Z 2 t ) = 1 n n 1 X 15 (1 k n ) (k). k=1

9 Proof of (b): Using (a), Xn 1 (1 k n ) (k) = 1 X k= 1 k max{0, 1 n } (k), {z }! (k) as n!1 and the claim follows by dominated convergence. 16