STAT Financial Time Series

Transcription

1 STAT Financial Time Series Chapter 4 - Estimation in the time Domain Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 1 / 46

2 Agenda 1 Introduction 2 Moment Estimates 3 Autoregressive Models (AR model) 4 Moving Average Models (MA model) 5 ARMA Models 6 Maximum Likelihood Estimator 7 partial ACF 8 Order Selection 9 Residual Analysis 10 Model Building Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 2 / 46

3 Introduction ARIMA(p,d,q) Model φ(b)(1 B) d (Y t µ) = θ(b)z t where Z t WN (0, σ 2 ) Unknown order: (p, d, q) Unknown parameters: ( µ, φ 1, φ 2,..., φ p, θ 1, θ 2,..., θ q, σ 2) Question: Given the order (p, d, q), how to estimate the unknown parameters?? Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 3 / 46

4 Agenda 1 Introduction 2 Moment Estimates 3 Autoregressive Models (AR model) 4 Moving Average Models (MA model) 5 ARMA Models 6 Maximum Likelihood Estimator 7 partial ACF 8 Order Selection 9 Residual Analysis 10 Model Building Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 4 / 46

5 Moment Estimators Sample Moments can be computed from the data Theoretical Moments depend on unknown parameters Match the sample and theoretical moments to solve for the unknown parameters Example: i.i.d. Normal: N (µ, σ 2 ) Sample Theoretical 1 st moment X µ 2 nd x 2 moment i n µ 2 + σ 2 Moment estimators: (match 1st moment): µ = X (match 2nd moment): σ 2 x 2 i = n X 2 Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 5 / 46

6 Moment Estimator for Time Series Theoretical Moment Sample Moment µ Y = 1 n nt=1 Y t γ(0) C 0 = 1 n (Yt Y ) 2 γ(k) C k = 1 n n k t=1 (Y t Y )(Y t+k Y ) ρ(0) 1 ρ(k) r k = C k C 0 Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 6 / 46

7 Moment Estimator for Time Series Q: How to check if ρ(k) is significantly different from 0? ( whether a time series is serially correlated at lag k ) A: Theorem If Y t WN (0, σ 2 ) then r k = C k C 0 AN (0, 1 n ) }{{} Asymptotically normal as n If r k has a 95% C.I. ± 2 n, there is a significant correlation at lag k Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 7 / 46

8 Agenda 1 Introduction 2 Moment Estimates 3 Autoregressive Models (AR model) 4 Moving Average Models (MA model) 5 ARMA Models 6 Maximum Likelihood Estimator 7 partial ACF 8 Order Selection 9 Residual Analysis 10 Model Building Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 8 / 46

9 Parameter Estimators in AR Model AR(p): Y t = φ 1 Y t φ p Y t p + Z t Express as a regression problem ) Y t = (φ 1 φ p Y t 1. Y t p + Z t = Y t 1φ + Z t (1) where φ = (φ 1 φ p ) and Y t 1 = (Y t 1 Y t p ) Recall in linear regression model, we have Y = Xβ + ε Least square estimate of β is β) = (X X) 1 X Y Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 9 / 46

10 Parameter Estimators in AR Model Least Square Estimate (LSE) Since (1) can be considered as a regression model form, we have the least squares estimate of φ φ = ( n t=p+1 Y t 1Y t 1 Example - AR(1): Y t = φ 1 Y t 1 + Z t ) 1 ( n t=p+1 Y t 1Y t ) φ 1 = nt=2 Y t 1 Y t nt=2 Y 2 t 1 and σ 2 = 1 n n t=2 (Y t φ 1 Y t 1 ) 2 Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 10 / 46

11 Parameter Estimators in AR Model Performance of the estimator : n( φ φ) N (0, σ 2 Γ 1 p ) where Γ p = γ(0) γ(1) γ(p 1) γ(1) γ(0) γ(p 2) γ(p 1) γ(p 2) γ(0) Confidence Intervals or hypothesis tests can be applied to make inference on the parameter φ Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 11 / 46

12 Parameter Estimators in AR Model Asymptotic distribution of the estimator: γ(0) γ(1) γ(p 1) n( φ φ) N(0, σ 2 Γ 1 γ(1) γ(0) γ(p 2) p ) Γ p = γ(p 1) γ(p 2) γ(0) Confidence Intervals or hypothesis tests to make inference on φ: Example: Estimated AR(2) model: n = 200, Y t = 0.3Y t Y t 2 + Z t ; ˆγ(0) = 1.11, ˆγ(1) = 0.347, ˆσ 2 = 1. 1 ˆσ 2 Γ p = ( ) 1 = ( C.I. for φ 2 : [0.04 ± /n] = [ 0.101, 0.181]. (Not significantly different from 0) ˆφ Testing for φ 1 = 0: z = 1 var( ˆ ˆφ = 0.3/ 0.998/200 = 4.24 > 2. 1 ) (significantly different from 0) Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 12 / 46 )

13 Parameter Estimators in AR Model Yule Walker Equation(Y.W) An alternative method for parameter estimation is via the Yule-Walker Equation (Y.W) Yule-Walker Equation (Y.W): Multiply Y t k to Y t = φ 1 Y t φ p Y t p + Z t where k = 1, 2,..., p Then, take expectation γ(k) = φ 1 γ(k 1) + + φ p γ(k p) ρ(k) = φ 1 ρ(k 1) + + φ p ρ(k p) where k = 1, 2,..., p Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 13 / 46

14 Parameter Estimators in AR Model Matrix Form ρ(1). ρ(p) = Yule-Walker Estimator ρ(0) ρ(1) ρ(p 1) φ ρ(1) ρ(0) ρ(p 2) φ ρ(p 1) ρ(p 2) ρ(0) p 1 r φ 1 r p 1 1 φ =. = r 1 1 r p φ p r p 1 r p r. r p Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 14 / 46

15 Agenda 1 Introduction 2 Moment Estimates 3 Autoregressive Models (AR model) 4 Moving Average Models (MA model) 5 ARMA Models 6 Maximum Likelihood Estimator 7 partial ACF 8 Order Selection 9 Residual Analysis 10 Model Building Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 15 / 46

16 Moving Average Models Moment Estimator MA(q): Y t = Z t θ 1 Z t 1 Z t q Express ρ(k) in terms of unknown parameters θ 1, θ 2,..., θ q Solve r k = ρ(k) for unknown parameters Example - MA(1): Y t = Z t θz t 1 ρ(1) = θ (Yt Y )(Y r 1+θ 2 1 = t+1 Y ) (Yt Y ) 2 The estimate satisfies r 1 = θ r 1 θ 2 + θ + 1 = 0 θ = 1± 1 4r θ2 2r 1 Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 16 / 46

17 Moving Average Models Conditional Least Square(CLS) Method Idea - Find the noise sequence {Z t } from the observation {Y t } Minimize the sum of squares n S (θ) = Zt 2 t=1 Example - MA(1): Y t = Z t θz t 1 Conditional on Z 0 = 0, then Z 1 = Y 1, Z 2 = Y 1 + θz 1,..., Z k = Y k + θy k 1 Minimize S (θ) = n t=1 Z 2 t by numerical optimization Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 17 / 46

18 Agenda 1 Introduction 2 Moment Estimates 3 Autoregressive Models (AR model) 4 Moving Average Models (MA model) 5 ARMA Models 6 Maximum Likelihood Estimator 7 partial ACF 8 Order Selection 9 Residual Analysis 10 Model Building Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 18 / 46

19 ARMA Model Method of moment and Conditional Least Square can be applied similarly as in pure MA model Example (CLS) - ARMA(1,1): Y t φy t 1 = Z t θz t 1 Conditional on Z 0 = 0, Y 0 = 0 Z 1 (φ, θ) = Y 1, Z 2 (φ, θ) = Y 2 φy 1 + θz 1,..., Z k (φ, θ) = Y k φy k 1 + θz k 1 Minimizes S (θ) = n t=1 Z 2 t (φ, θ) Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 19 / 46

20 Agenda 1 Introduction 2 Moment Estimates 3 Autoregressive Models (AR model) 4 Moving Average Models (MA model) 5 ARMA Models 6 Maximum Likelihood Estimator 7 partial ACF 8 Order Selection 9 Residual Analysis 10 Model Building Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 20 / 46

21 Maximum Likelihood Estimator Maximum Likelihood Estimator (MLE) MLE = Most probable parameter value that corresponds closely with the observed data Let X 1, X 2,..., X n be i.i.d. random variables with probability density function f (x, θ) f (x i, θ) is the probability of observing x i The likelihood function L(x, θ) = n i=1 f (x i, θ) is the probability of observing the data set The MLE θ maximizes L(x, θ) The most probable parameter value such that the current data set is obtained Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 21 / 46

22 Maximum Likelihood Estimator MLE for Time Series Idea: Write down the Likelihood function for Y 1, Y 2,..., Y n AR(1): Y t = φy t 1 + Z t Given Y 1, (Y 2,..., Y n ) and (Z 1,..., Z n ) are of one to one correspondence i.e. f (Y 2,..., Y n Y 1 ) = f (Z 2,..., Z n Y 1 ) = f (Z 2,..., Z n ) ( ) n 1 ( = 1 2 exp 1 ) nt=2 Z 2πσ 2 2σ 2 2 t Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 22 / 46

23 Maximum Likelihood Estimator Likelihood L(Y 1, Y 2,..., Y n ) = f (Y 2,..., Y n Y 1 )f (Y 1 ) = = ( ) n exp 1 n 2πσ 2 2σ 2 t=2 Z t 2 ( ) n φ 2πσ 2 exp 2 1 φ 2 2πσ 2 exp 1 2σ 2 (1 φ2 )Y 2 1 { 1 [ nt=2 (Y 2σ 2 t φy t 1 ) 2 + (1 φ 2 )Y1 2 ] } Maximize L(Y 1,..., Y n ) or equivalently log L(Y 1,..., Y n ) by numerical method Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 23 / 46

24 Maximum Likelihood Estimator Likelihood vs Least Square for AR(1) Log likelihood l = log L log(1 φ 2 ) n log σ 2 (1 φ2 )Y 2 1 σ 2 1 σ 2 nt=2 (Y t φy t 1 ) 2 Conditional Least Square S (θ) = n Z 2 t=2 t = n (Y t φy t 1 ) 2 t=2 Unconditional Least Square S(θ) = (1 φ 2 )Y n t=2 (Y t φy t 1 ) 2 Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 24 / 46

25 Maximum Likelihood Estimator If n is large, the term n t=2 (Y t φy t 1 ) 2 dominates Minimizing S(θ), S (θ) and maximizing loglikelihood are similar Three methods will give similar results Except for some simple models such as AR(1), we have to employ numerical optimization for the estimation. Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 25 / 46

26 Agenda 1 Introduction 2 Moment Estimates 3 Autoregressive Models (AR model) 4 Moving Average Models (MA model) 5 ARMA Models 6 Maximum Likelihood Estimator 7 partial ACF 8 Order Selection 9 Residual Analysis 10 Model Building Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 26 / 46

27 Partial ACF (PACF) PACF measures the correlation between Y t and Y t k that is not explained by Y t 1,..., Y t k+1 PACF - ) φ kk = Corr (Y k+1 P sp{y2,...,y k }Y k+1, Y 1 P sp{y2,...,y k }Y 1 where P sp{x,z} Y is the projection of random variable Y onto the linear subspace spanned by the random variables {X, Z} In particular, P sp{x,z} Y = αx + βz where E[(Y αx + βz) 2 ] is minimized over α and β Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 27 / 46

28 Example on PACF Example Assume that {Y 1, Y 2, Y 3 } is a zero mean stationary sequence with ACF of first lag equal to ρ 1 ) φ 22 = Corr (Y 3 P sp{y2 }Y 3, Y 1 P sp{y2 }Y 1 P sp{y2 }Y 3 = βy 2 where E[(Y 3 βy 2 ) 2 ] is minimized E[(Y 3 βy 2 ) 2 ] = γ(0) 2βγ(1) + β 2 γ(0) [ = γ(0) β γ(1) ] 2 γ(0) + γ(0) γ(1) 2 γ(0) β = γ(1) γ(0) = ρ 1 i.e.p sp{y2 }Y 3 = ρ 1 Y 2 Similarly, P sp{y2 }Y 1 = ρ 1 Y 2 Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 28 / 46

29 Example on PACF (continue) Therefore, φ 22 = Corr(Y 3 ρ 1 Y 2, Y 1 ρ 1 Y 2 ) = Cov(Y 3 ρ 1 Y 2,Y 1 ρ 1 Y 2 ) Var(Y3 ρ 1 Y 2 )Var(Y 1 ρ 1 Y 2 ) = ρ 2 ρ ρ 2 1 In particular, if Y t AR(1), then ρ k = φ k, φ 22 = 0 Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 29 / 46

30 Another Interpretation of PACF In the regression Y = β 1 X β k X k + β k+1 X k+1 + ε, the parameter β k+1 can be interpreted as the relationship between Y and X k+1, after accounting for the effect of X 1,..., X k 1, X k It can be shown that, the definition ) φ kk = Corr (Y k+1 P sp{y2,...,y k }Y k+1, Y 1 P sp{y2,...,y k }Y 1 is equivalent to the coefficients φ kk in the representation Y k+1 = φ k1 Y k + φ k2 Y k φ kk Y 1 i.e. φ kk measures the relationship between Y k+1 and Y 1 after accounting for the effect of Y 2,..., Y k Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 30 / 46

31 Computational formula for PACF φ kk is the coefficient in the representation Y k+1 = φ k1 Y k + φ k2 Y k φ kk Y 1 This representation can be found by searching for the minimum of E[(Y k+1 β 1 Y k β k Y 1 ) 2 ] It means that projecting Y k+1 on the space spanned by Y 1,..., Y k Differentiating β 1, β 2,..., β k in terms and set the equations to zero, φ k1,..., φ kk can be obtained by solving ρ(0) ρ(k 1) φ k ρ(k 1) ρ(0) φ kk = ρ(1). ρ(k) Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 31 / 46

32 PACF for AR Models In general, AR(p): Y t = φ 1 Y t φ p Y t p + Z t φ 11, φ 22,..., φ pp are non-zero if Y t is linearly related to Y t k for k p Y p+1 P sp{y1,...,y p}y p+1 = Y p+1 φ 1 Y p φ p Y 1 = Z p+1 which is uncorrelated with all Y k, k p, thus, φ p+1,p+1 ) = Corr (Y p+2, P sp{y2,...,y p+1 }Y p+2, Y 1 P sp{y2,...,y p+1 }Y 1 ) = Corr (Z p+2, Y 1 P sp{y2,...,y p+1 }Y 1 = 0 Similarly, φ kk = 0 for k > p Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 32 / 46

33 Theorem of PACF for AR Models For an AR(p) Model, nφkk AN (0, 1) for k > p To test whether φ kk is significant, Significant if φ kk > 2 n Can conclude that the time series follows AR(p), p k NOT Significant otherwise Can conclude that the time series follows AR(p), p < k Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 33 / 46

34 Using ACF and PACF to determine order of AR and MA model AR(p) Model - ACF plot: No clear pattern except AR(1) shows exponential decay pattern PACF plot: Cut-off at lag p MA(q) Model - ACF plot: Cut-off at lag q PACF plot: No clear pattern except MA(1) shows exponential decay pattern Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 34 / 46

35 Examples on ACF and PACF ACF PACF AR(1) AR(3) Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 35 / 46

36 Examples on ACF and PACF ACF PACF MA(1) MA(4) Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 36 / 46

37 Agenda 1 Introduction 2 Moment Estimates 3 Autoregressive Models (AR model) 4 Moving Average Models (MA model) 5 ARMA Models 6 Maximum Likelihood Estimator 7 partial ACF 8 Order Selection 9 Residual Analysis 10 Model Building Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 37 / 46

38 Order Selection ACF and PACF are graphical methods to determine order of AR and MA model It is more desirable to have a systematic order selection criterion for a general ARMA model Some common model selection criterion: FPE(Final Prediction Error) AIC (Akaike s Information Criterion) BIC (Bayesian Information Criterion) Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 38 / 46

39 Order Selection - Final Prediction Error Final Prediction Error (FPE) FPE = σ ( ) 2 n+p n p where σ 2 is the MLE of σ 2 Idea for an AR(p) Model : 1. Mean Square Error of parameter estimation is ( ) n + p MSE σ 2 n 2. The best unbiased estimator for σ 2 is σ 2 n n p 3. Substituting σ 2 by σ 2 n n p to the MSE gives the formula of FPE 4. Note the tradeoff between goodness of fit and the model complexity Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 39 / 46

40 Order Selection - Akaike s Information Criterion Akaike s Information Criterion (AIC) AIC - 2 log L ( β, S x( β) ) + 2(p + q + 1) n AICC (AIC corrected) - 2 log L ( β, S x( β) ) 2(p + q + 1)n + n n p q 2 where S x ( β) = n t=1 (X t φ 1 X t 1 φ p X t p θ 1 Z t 1 θ q Z t q ) 2 β = ( φ 1,..., φ p, θ 1,..., θ q ) L( β, σ 2 ) = ( ) n 1 2 exp 2π σ 2 [ 1 2π σ 2 S x( β) ] Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 40 / 46

41 Order Selection - Akaike s Information Criterion Idea - Similar to FPE, but AIC and AICC estimates the expected likelihood function E[L Y ( β, σ 2 )] where β, σ 2 are MLE computed From the observation X = {X 1,..., X n } and Y is a new observation independent of X X and Y are different to avoid the problem of overfitting Note the tradeoff between model complexity and goodness of fit Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 41 / 46

42 Order Selection - Bayesian Information Criterion Bayesian Information Criterion (BIC) [ n σ 2 ] BIC = (n p q) log + n(1 + log 2π) n p 1 [ ni=1 Xi 2 n σ 2 ] +(p + q) log p + q Motivated by Bayesian argument: BIC the posterior probability of a particular model given data Consistent order selection procedure As n, the order selected by BIC will equal to the true order that will equal to the true order that the observation follow with probability going to 1 AIC is not consistent (i.e. get a wrong model) Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 42 / 46

43 Agenda 1 Introduction 2 Moment Estimates 3 Autoregressive Models (AR model) 4 Moving Average Models (MA model) 5 ARMA Models 6 Maximum Likelihood Estimator 7 partial ACF 8 Order Selection 9 Residual Analysis 10 Model Building Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 43 / 46

44 Residual Analysis Check the goodness of fit of the model by studying the residual Ẑ t = Y t Ŷt STEPS - 1. Time series plot of Ẑt (should look like white noises) 2. ACF of Ẑt (should not have serial correlation) ( ) 3. Confidence interval for r j is 2 2 n, n 4. Portmanteau Statistics Q = n(n + 2) h j=1 r 2 Z (j) n j A common choice of h is between 15 to 30 Q χ 2 (r p q), thus, if Q is bigger than the 95% percentile of a χ 2 (n p q) distribution, we reject the null hypothesis that {Ẑt} is WN. Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 44 / 46

45 Agenda 1 Introduction 2 Moment Estimates 3 Autoregressive Models (AR model) 4 Moving Average Models (MA model) 5 ARMA Models 6 Maximum Likelihood Estimator 7 partial ACF 8 Order Selection 9 Residual Analysis 10 Model Building Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 45 / 46

46 Model Building Three Stages - 1. Model Specification Trend, seasonal effect, choosing ARIMA model by FPE/AIC/BIC 2. Model Identification (estimating coefficient) CLS,MLE,LSE 3. Model Checking(diagnostic) Residual analysis (plot of {Ẑt}) Portmanteau test Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 46 / 46