Classical Decomposition Model Revisited: I

Transcription

1 Classical Decomposition Model Revisited: I recall classical decomposition model for time series Y t, namely, Y t = m t + s t + W t, where m t is trend; s t is periodic with known period s (i.e., s t s = s t for all t 2 Z) satisfying P s j=1 s j = 0; and W t is a stationary process with zero mean m t & s t often taken to be deterministic as we have seen, SARIMA processes can model stochastic s t, but can also handle deterministic m t & s t through di erencing di erencing to eliminate deterministic m t and/or s t can lead to undesirable overdi erencing of W t alternative approach: regard ( ) to be a regression model with stationary errors ( ) BD 23, CC 27, 30, 32, SS 48 XVII 1

2 Classical Decomposition Model Revisited: II will now consider a linear regression approach in which m t and/or s t depend linearly on a small number of parameters note: nonparametric regression another option (expands upon ideas discussed earlier of using filtering operations to extract trends see overhead III 25 and discussion following it) BD 23, CC 27, 30, 32, SS 48 XVII 2

3 Regression with Stationary Errors: I in context of time series analysis, standard linear regression model would take form where Y = X + Z, Y = [Y 1,..., Y n ] 0 is vector containing series; X is an n k design matrix whose tth row x 0 t has values of explanatory variables for Y t ; = [ 1,..., k] 0 is vector of regression coe cients; and Z = [Z 1,..., Z n ] 0 is vector of WN(0, 2 ) RVs example: Y t = t + Z t, for which tth row of n 2 design matrix X would be x 0 t = [1, t] second example: Y t = cos (2 ft) + 3 sin (2 ft) + Z t BD 211, CC 30, SS 50 XVII 3

4 Regression with Stationary Errors: II ordinary least squares (OLS) estimator of is vector ˆOLS minimizing sum of squared errors: nx S( ) = (Y t x 0 t ) 2 = (Y X ) 0 (Y X ) t=1 ˆOLS is solution to so-called normal equations: X 0 X if X 0 X has full rank = X 0 Y and hence ˆOLS = (X 0 X) 1 X 0 Y ˆOLS is best linear unbiased estimator of (best in that, if ˆ is any other unbiased estimator, then var{c 0 ˆOLS } apple var{c 0 ˆ} for any vector c of constants) covariance matrix for ˆOLS given by 2 (X 0 X) 1 BD 211, SS 50 XVII 4

5 Regression with Stationary Errors: III for time series, uncorrelated errors Z are usually unrealistic often a more realistic model is Y = X + W, where W contains RVs from a stationary process with zero mean, an example being a causal ARMA(p, q) process: (B)W t = (B)Z t, {Z t } WN(0, under this alternative model, ˆOLS is still an unbiased estimator of, but best linear unbiased estimator is generalized least squares (GLS) estimator in the full rank case, this estimator takes the form ˆGLS = (X 0 n 1 X) 1 X 0 n 1 Y, where n is covariance matrix for W 2 ) BD 211, 212 XVII 5

6 Regression with Stationary Errors: IV ˆGLS is minimizer of weighted sum of squares: S( ) = (Y X ) 0 1 n (Y X ) to motivate this estimator, recall innovations representation for stationary process says that W = C n U, where U = W Ŵ (U are the innovations, and Ŵ are 1-step-ahead predictions) recall that, if V has covariance matrix n, then covariance matrix for AV is A n A 0 since U has diagonal covariance matrix D n, get n = C n D n Cn 0 let Dn 1/2 be diagonal matrix such that Dn 1/2 Dn 1/2 = Dn 1 covariance matrix for Dn 1/2 Cn 1 W is identity matrix I n since Dn 1/2 Cn 1 n(cn 1 ) 0 Dn 1/2 = Dn 1/2 Cn 1 C n D n Cn(C 0 n 1 ) 0 Dn 1/2 = I n BD 212, 213 XVII 6

7 Regression with Stationary Errors: V returning now to model Y = X + W, multiplication of both sides by Dn 1/2 Cn 1 yields Dn 1/2 Cn 1 Y = Dn 1/2 Cn 1 X + Dn 1/2 Cn 1 W, which can be reexpressed as Ỹ = X + Z, {Z t } WN(0, 1) for this regression model, best linear unbiased estimator is OLS = ( X 0 X) 1 X0 Ỹ = (X 0 (Cn 1 ) 0 Dn 1/2 Dn 1/2 Cn 1 X) 1 X 0 (Cn 1 ) 0 Dn 1/2 Dn 1/2 Cn 1 Y = (X 0 (Cn 1 ) 0 Dn 1 Cn 1 X) 1 X 0 (Cn 1 ) 0 Dn 1 Cn 1 Y = (X 0 n 1 X) 1 X 0 n 1 Y = ˆGLS since n = C n D n Cn 0 says 1 n = (Cn) 0 1 D n Cn 1 = (Cn 1 ) 0 D n Cn 1 BD 212, 213 XVII 7

8 Regression with Stationary Errors: VI in principle, can use ML under a Gaussian assumption to estimate all parameters in model Y = X + W (i.e., both and parameters associated with, e.g., ARMA model) in practice, following simpler (but sub-optimal) iterative scheme for parameter estimation often works well 1. compute ˆOLS and form residuals Y t x 0 t ˆOLS 2. fit ARMA(p,q) or other stationary model to residuals 3. using fitted model, compute ˆGLS and form residuals Y t x 0 t ˆGLS 4. fit same model to residuals again 5. repeat steps 3 and 4 until parameter estimates have stabilized BD 212, 213 XVII 8

9 1st Di erence of Atomic Clock Data Revisited: I let s reconsider modeling 1st di erence of atomic clock data X t, i.e., rx t 2nd di erence r 2 X t well-modeled by ARMA(1,1) process (overheads XIII 124 to 133), with support for need for 2nd di erencing coming from augmented Dickey Fuller (ADF) unit root test (overhead XIV 37) modeling rx t as a stationary fractionally di erenced (FD) process proved to be questionable (overheads XV 75 to 80) apparent linear increase in rx t suggests exploring model of linear trend + stationary noise more seriously (looked briefly at this approach in context of ADF unit root test, where we found that linear detrending obviated need for 2nd di erencing see overhead XIV 37) XVII 9

10 1st Di erence of Atomic Clock Time Series x t t XVII 10

11 Residuals from OLS Fit of Linear Regression x t ACF PACF h h XIV 34

12 1st Di erence of Atomic Clock Data Revisited: II sample ACF and PACF suggest three models for residuals: AR(p), with p somewhere around 7 or 8 ARMA(p,q), with p + q small ARFIMA(p,,q), with p + q small (= 0 gives FD( )) let s consider various possibilities, using maximum likelihood to fit each model AICC to evaluate di erent models start by considering AR(p) models of orders p = 1,..., 20 XVII 11

13 AICC ML-Based AICC for Residuals Modeled by AR(p) p (AR model order) XVII 12

14 1st Di erence of Atomic Clock Data Revisited: III AR(11) best amongst AR(p) with AICC of here are AICCs for ARMA(p,q) models with p + q apple 4 p q AICC XVII 13

15 1st Di erence of Atomic Clock Data Revisited: IV let s compare AICC for ARMA(2,1) with ones for ARMA(p,q) models with p + q = 5 p q AICC let s now see if we can find an ARFIMA(p,,q) model that has a better AICC than one for ARMA(2,1) model XVII 14

16 1st Di erence of Atomic Clock Data Revisited: V p q AICC conclusion: will go with ARMA(2,1) model XVII 15

17 1st Di erence of Atomic Clock Data Revisited: VI using R function arima to fit model rx t = a + bt + W t, {W t } ARMA(2,1), ML estimates of model parameters a, b, 1, 2 and 1 (along with standard errors (SEs)) are â ˆb ˆ1 ˆ1 ˆ 1 value SE estimate of 2 is ˆ2. = ; next plots show rx t & lines fitted by OLS (red) & GLS (black dashed) comparison of ARMA(2,1) ACF & PACF with sample ACF & PACF for rx t â ˆbt residuals ˆR t returned by arima and associated diagnostics XVII 16

18 1st Di erence of Atomic Clock Time Series x t t XVII 17

19 ACF Sample ACF for rx t â ˆbt & ARMA(2,1) ACF h (lag) XVII 18

20 PACF Sample PACF for rx t â ˆbt & ARMA(2,1) PACF h (lag) XVII 19

21 ARMA(2,1) Residuals ˆR t for Atomic Clock R^ t t XVII 20

22 ACF Sample ACF for ARMA(2,1) Residuals ˆR t h (lag) XVII 21

23 PACF Sample PACF for ARMA(2,1) Residuals ˆR t h (lag) XVII 22

24 Q LB and 0.95% quantile Portmanteau Tests of ARMA(2,1) Residuals ˆR t h XVII 23

25 Diagnostics Tests of ARMA(2,1) Residuals ˆR t expected test test value statistic p-value turning point di erence-sign rank runs AR method AICC order AIC order Yule Walker 0 0 Burg 0 0 OLS 0 4 MLE 0 0 XVII 24

26 Regression with Stationary Errors: VII cautionary note: Y 0 Y 6= Ỹ 0 Ỹ = Y 0 1 n Y, so portion of sum of squares explained by transformed model cannot be related directly to sum of squares for untransformed data assuming W in model Y = X + W has covariance matrix n, covariance matrices for ˆOLS and ˆGLS are, respectively, (X 0 X) 1 X 0 nx(x 0 X) 1 and (X 0 n 1 X) 1 assuming W for rx t clock data is ARMA(2,1) process, can assess standard errors (SEs) for estimated slopes OLS GLS slope ( 3% di erence between estimates) SE ( 3% increase for OLS over GLS) if assume W is WN, SE for OLS slope estimate would be taken as (5 smaller than when correlation is accounted for) XVII 25

27 Harmonic Regression and CO 2 Series: I atomic clock rx t illustrates handling deterministic trend m t in model Y t = m t + W t via a parametric regression approach (m t = a + bt) as a second example, reconsider CO 2 series from Mauna Loa, Hawaii (subject of Problem 11) appropriate model here is full classical decomposition model: Y t = m t + s t + W t where m t is trend; s t is seasonal component with period s = 12 (i.e., s t 12 = s t for all t 2 Z) satisfying P 12 j=1 s j = 0; and W t is a stationary process with zero mean for illustrative purposes, will take m t & s t to be deterministic XVII 26

28 2nd Example: CO 2 Series from Mauna Loa, Hawaii y t III 2 year

29 Preliminary Nonparametric Estimate of Trend y t and m^ t year XVII 27

30 Preliminary Detrended Series u t = y t m^ t year XVII 28

31 Estimated Seasonal Component yearly pattern of s^t Jan Feb Mar Apr May June July Aug Sep Oct Nov Dec month XVII 29

32 Harmonic Regression and CO 2 Series: II factoid: can represent any deterministic s t with period 12 by 6X 6X s t = A j cos (2 f j t) + B j sin (2 f j t) = D j cos (2 f j t + ' j ), j=1 j=1 where f j j/12 is a frequency with associated period 1/f j : treating t momentarily as t 2 R rather than t 2 Z, have cos (2 f j (t + 1 f j )) = cos (2 f j t + 2 ) = cos (2 f j t) with a similar result holding for sin (2 f j t). since P 12 j=1 s j = 0, any eleven s j s determine remaining twelfth six A j s and six B j s seem one too many, but in fact there are only five relevant B j s: B 6 doesn t enter in play because B 6 sin (2 f 6 t) = B 6 sin ( t) = B 6 sin ( t) = 0 for all t XVII 30

33 Harmonic Regression and CO 2 Series: III f 1 is known as the fundamental frequency, whereas f 2,..., f 6 are called first,..., fifth harmonics since f j j/12, have f j = jf 1 for j = 2,..., 6, so harmonics are integer multiples of fundamental frequency if seasonal component s t slowly varying from one month to next, can often get by with using just fundamental frequency and a small number of harmonics: JX s t A j cos (2 f j t) + B j sin (2 f j t), 1 apple J < 6 j=1 following plots show approximations of orders J = 1,... 5 to seasonal component ŝ t estimated in Problem 11 for CO 2 series XVII 31

34 J = 1 Approximation to Seasonal Component yearly pattern of s^t Jan Feb Mar Apr May June July Aug Sep Oct Nov Dec month XVII 32

35 J = 2 Approximation to Seasonal Component yearly pattern of s^t Jan Feb Mar Apr May June July Aug Sep Oct Nov Dec month XVII 33

36 J = 3 Approximation to Seasonal Component yearly pattern of s^t Jan Feb Mar Apr May June July Aug Sep Oct Nov Dec month XVII 34

37 J = 4 Approximation to Seasonal Component yearly pattern of s^t Jan Feb Mar Apr May June July Aug Sep Oct Nov Dec month XVII 35

38 J = 5 Approximation to Seasonal Component yearly pattern of s^t Jan Feb Mar Apr May June July Aug Sep Oct Nov Dec month XVII 36

39 J = 6 Perfect Fit to Seasonal Component yearly pattern of s^t Jan Feb Mar Apr May June July Aug Sep Oct Nov Dec month XVII 37

40 Harmonic Regression and CO 2 Series: IV J = 2 approximation looks reasonable, so will entertain model Y t = m t + s t + W t = a + bt + ct 2 + 2X A j cos (2 f j t) + B j sin (2 f j t) + W t j=1 (A j s & B j s are coe cients for so-called harmonic regressors) start analysis by fitting model using OLS to get estimates ˆm t = â + ˆbt 2X + ĉt 2 and ŝ t = Â j cos (2 f j t) + ˆB j sin (2 f j t) following plots look at j=1 ˆm t + ŝ t as compared to Y t cw t = Y t ˆm t ŝ t (surrogates for W t ) and also r c W t XVII 38

41 Parametric Estimate ˆm t + ŝ t and CO 2 Series y t and m^ t + s^t year XVII 39

42 Residuals c W t from Parametric Fit of m t + s t W^ t ACF PACF h h XVII 40

43 First Di erence of Residuals c W t W^ t ACF PACF h h XVII 41

44 Harmonic Regression and CO 2 Series: V sample ACFs & PACFs for f W t & r f W t do not point to obvious simple model more work needed to find reasonable model starting with f W t, consideration of AR(p), with p = 1,..., 31 (ar.mle bombs for p = 32) ARMA(p,q), with p + q small ARFIMA(p,,q), with p + q small using maximum likelihood to fit each model AICC to evaluate individual models leads to an unsatisfying AR(26) model XVII 42

45 Harmonic Regression and CO 2 Series: VI ADP unit root test on f W t suggests need to di erence, but not strongly so (p-value just a bit above 0.05) need for di erencing also hinted at by close to upper limit of 1/2 estimates, which are sample ACF & PACF for r f W t have large values at lag h = 12, suggesting that deterministic s t might be too simplistic SARIMA model with either d = 1 or D = 1 (or both) worthy of consideration bottom line: finding a suitable model for W t for the atomic clock data was easy, but finding one for the CO 2 series poses more of a problem! XVII 43