Problems from Chapter 3 of Shumway and Stoffer s Book

Transcription

1 UNIVERSITY OF UTAH GUIDED READING TIME SERIES Problems from Chapter 3 of Shumway and Stoffer s Book Author: Curtis MILLER Supervisor: Prof. Lajos HORVATH November 10, 2015

2 UNIVERSITY OF UTAH DEPARTMENT OF MATHEMATICS ARIMA Models Curtis Miller November 10, ESTIMATION 1.1 AR(2) MODEL FOR cmort To estimate the AR(2) process, I first use ordinary least squares (OLS). I then use the Yule- Walker estimate. This is shown in the R code below: # OLS esimate # demean = T results in looking at cmort - mean(cmort) # intercept = F sets the intercept to 0 cmort.ar2.ols <- ar.ols(cmort, order = 2, demean = T) # Yule-Walker estimate cmort.ar2.yw <- ar.yw(cmort, order = 2, demean = T) PARAMETER ESTIMATE COMPARISON # OLS estimate cmort.ar2.ols Call: ar.ols(x = cmort, order.max = 2, demean = T) 2

3 Coefficients: Intercept: (0.2527) Order selected 2 sigma^2 estimated as # Yule-Walker estimate cmort.ar2.yw Call: ar.yw.default(x = cmort, order.max = 2, demean = T) Coefficients: Order selected 2 sigma^2 estimated as Looking at the coefficients of the AR(2) model estimated using the two methods, I see very little difference. OLS and Yule-Walker estimation produce similar results STANDARD ERROR COMPARISON # The standard error of the OLS estimates cmort.ar2.ols$asy.se.coef$ar [1] # The variance matrix of the Yule-Walker estimates cmort.ar2.yw$asy.var.coef [,1] [,2] [1,] [2,] # Corresponding standard error of both parameters sqrt(cmort.ar2.yw$asy.var.coef[1,1]) [1] Looking at the above R output, it appears that both models have the same standard error for the parameters. 3

4 First I generate the data: 1.2 AR(1) SIMULATION AND ESTIMATION ar1.sim <- arima.sim(n = 50, list(ar = c(.99), sd = c(1))) I first estimate the parameter from the simulation using the Yule-Walker estimate. ar1.sim.yw <- ar.yw(ar1.sim, order = 1) # Model estimates ar1.sim.yw Call: ar.yw.default(x = ar1.sim, order.max = 1) Coefficients: Order selected 1 sigma^2 estimated as # Model covariance matrix ar1.sim.yw$asy.var.coef [,1] [1,] Here, I would perform inference on the model by assuming it is Normally distributed. I would use the covariance matrix listed above for estimating the standard error. Bootstrap results in R could be done as follows: tsboot(ar1.sim, function(d) { return(ar.yw(d, order = 1)$ar) }, R = 2000) MODEL BASED BOOTSTRAP FOR TIME SERIES Call: tsboot(tseries = ar1.sim, statistic = function(d) { return(ar.yw(d, order = 1)$ar) }, R = 2000) 4

5 Bootstrap Statistics : original bias std. error t1* The bootstrap standard error is zero, while the theoretical standard error is non-zero. 2 INTEGRATED MODELS FOR NONSTATIONARY DATA 2.1 EWMA MODEL FOR GLACIAL VARVE DATA Here I am interested in the varve dataset. In fact, I am interested in analyzing log(v ar ve), since I believe this may actually be a stationary process. I will be estimating a EWMA model for this data. logvarve <- log(varve) # EWMA for logvarve with lambda =.25 logvarve.ima.25 <- HoltWinters(logvarve[1:100], alpha = , beta = FALSE, gamma = FALSE) logvarve.ima.5 <- HoltWinters(logvarve[1:100], alpha = 1 -.5, beta = FALSE, gamma = FALSE) logvarve.ima.75 <- HoltWinters(logvarve[1:100], alpha = , beta = FALSE, gamma = FALSE) # Plotting results par(mfrow = c(3,1)) plot(logvarve.ima.25, main = "EWMA Fit with Lambda =.25") plot(logvarve.ima.5, main = "EWMA Fit with Lambda =.5") plot(logvarve.ima.75, main = "EWMA Fit with Lambda =.75") The results are shown in 2.1. With a small smoothing parameter (λ), the results are very sensetive to the immediate past, while a high smoothing parameter leads to more stable predictions. 3 BUILDING ARIMA MODELS 5

6 EWMA Fit with Lambda =.25 Observed / Fitted Time EWMA Fit with Lambda =.5 Observed / Fitted Time EWMA Fit with Lambda =.75 Observed / Fitted Time Figure 2.1: EWMA fit for different smoothing parameters 6

7 3.1 AR(1) MODEL FOR GNP DATA Here I am investigating how well an AR(1) (or, more exactly, an ARIMA(1,1,0)) model fits the natural log of U.S. GNP data. I estimate this ARIMA model. gnpgr = diff(log(gnp)) # growth rate of GNP gnp.model <- sarima(gnpgr, 1, 0, 0, details = F) # AR(1) model fit I see disturbing trends in the diagnostic plots shown in Figure 3.1. The residual plot should look like white noise, but I see the variance decreasing as the year increases. The ACF is fine, but the Q-Q plot suggests non-normality. Fortunately, the ACF and p-values for Ljung- Box statistic look as they should be. Still, other models (probably ones that do not assume Gaussian white noise) may be better. 3.2 FITTING CRUDE OIL PRICES WITH AN ARIMA(p,d, q) MODEL My objective is to fit an ARIMA(p,d, q) model for the oil dataset. I start by examining the data: # Prepare layout old.par <- par(mar = c(0, 0, 0, 0), oma = c(4, 4, 1, 1), mfrow = c(4, 1), cex.axis =.75) plot(oil, xaxt = 'n'); mtext(text = "Oil Price", side = 2, line = 2, cex =.75) plot(log(oil), xaxt = 'n'); mtext(text = "Natural Logarithm of Oil Price", side = 2, line = 2, cex =.75) plot(diff(oil), xaxt = 'n'); mtext(text = "First Difference in Oil Price", side = 2, line = 2, cex =.75) plot(diff(log(oil))); mtext(text = "Percent Change in Oil Price", side = 2, line = 2, cex =.75) The first plot in Figure 3.2 shows that oil prices clearly are not a stationary process, and it appears that the variance of the process increases with time. Taking the natural log of oil prices helps control the increasing variability, but not the nonstationary behavior of the series. When looking at the change in oil price from one period to the next, I do see a process that looks more stationary, but the nonconstant variance is not removed. The final attempt is to look at the differences in the natural log of oil prices (which can be interpreted as the percentage change in oil prices). This appears to be stationary and with a mostly constant variance. However, there are large deviations around 2009, and even prior, that would lead one to conclude that the white noise is not Gaussian, which threatens estimation and inference. I now look at the ACF and PACF of log(oil t ): 7

8 Standardized Residuals Time ACF of Residuals Normal Q Q Plot of Std Residuals ACF Sample Quantiles LAG Theoretical Quantiles p values for Ljung Box statistic p value lag Figure 3.1: Diagnostic plots for the AR(1) model 8

9 Percent Change in Oil Price First Difference in Oil Price Natural Logarithm of Oil Price Oil Price Figure 3.2: Basic plots of the oil series 9

10 par(mar = c(0, 0, 0, 0), oma = c(4, 4, 1, 1), mfrow = c(2, 1), cex.axis =.75) acf(diff(log(oil)), xaxt = 'n'); mtext(text = "Sample ACF", side = 2, line = 2) pacf(diff(log(oil))); mtext(text = "Sample PACF", side = 2, line = 2) When looking at the sample PACF in Figure 3.3, I see that the PACF is nonzero as far out as eight lags, which may suggest that p = = 9, or that we should consider lagging the AR term out to as far as nine lags. # noquote(capture.output(), "") used only to make presentation # easier write(capture.output(sarima(log(oil), p = 9, d = 1, q = 0))[32:38],"") Coefficients: ar1 ar2 ar3 ar4 ar5 ar s.e ar7 ar8 ar9 constant The ninth lag does not appear statistically significant, so I drop the number of lags down to eight. I now use the following ARIMA model (with diagnostic plots shown): oil.model <- sarima(log(oil), p = 8, d = 1, q = 0, details = F) write(capture.output(oil.model)[8:14], "") Coefficients: ar1 ar2 ar3 ar4 ar5 ar s.e ar7 ar8 constant s.e The residuals clearly do not appear to be Gaussian; there are large price movements that make this assumption doubtful, and the Q-Q plot does not support the Normality assumption. The ACF of the residuals can get large for some distant lags but otherwise are within the band of reasonable values. The p-values of the Ljung-Box statistics suggest that we do not have dependence in our residuals for large lags. This may be the best fit an ARIMA model can provide. 10

11 Sample PACF Sample ACF Figure 3.3: Sample ACF and PACF for percentage change in oil price 11

12 Standardized Residuals Time ACF of Residuals Normal Q Q Plot of Std Residuals ACF Sample Quantiles LAG Theoretical Quantiles p values for Ljung Box statistic p value lag Figure 3.4: Diagnostic plots for the ARIMA(8,1,0) model for the log(oil) series 12

13 4 REGRESSION WITH AUTOCORRELATED ERRORS 4.1 MONTHLY SALES DATA ARIMA MODEL FITTING The problem first asks for an ARIMA model for the sales data series. I first plot the series. par(mar = c(0, 0, 0, 0), oma = c(4, 4, 1, 1), mfrow = c(3, 1), cex.axis =.75) plot(sales, xaxt = 'n'); mtext(text = "Sales", side = 2, line = 2, cex =.75) plot(diff(sales), xaxt = 'n') mtext(text = "First Order Difference in Sales", side = 2, line = 2, cex =.75) plot(diff(diff(sales))); mtext(text = "Second Order Difference in Sales", side = 2, line = 2, cex =.75) Figure 4.1 shows the plots of the sales series. Clearly, sales t is not stationary. Surprisingly, neither is sales t ; this series shows periodicity. It takes a second-order differencing, ( sales t ), to find a stationary series. I next examine the ACF and PACF functions to try and identify the order of the AR and MA terms. par(mar = c(0, 0, 0, 0), oma = c(4, 4, 1, 1), mfrow = c(2, 1), cex.axis =.75) acf(diff(diff(sales)), xaxt = 'n'); mtext(text = "Sample ACF", side = 2, line = 2) pacf(diff(diff(sales))); mtext(text = "Sample PACF", side = 2, line = 2) As shown in Figure 4.2, the sample ACF cuts off after one lag and the sample PACF appears to be trailing off, so I believe that an ARIMA(0,2,1) should provide a good fit for the data. par(old.par) oil.model <- sarima(sales, p = 0, d = 2, q = 1, details = F) write(capture.output(oil.model)[8:14], "") Coefficients: ma s.e sigma^2 estimated as 1.866: log likelihood = , aic =

14 Second Order Difference in Sales First Order Difference in Sales Sales Figure 4.1: Basic plots of the sales series 14

15 Sample PACF Sample ACF Figure 4.2: Sample ACF and PACF for second order difference in sales 15

16 Standardized Residuals Time ACF of Residuals Normal Q Q Plot of Std Residuals ACF Sample Quantiles LAG Theoretical Quantiles p values for Ljung Box statistic p value lag Figure 4.3: Diagnostic plots for the ARIMA(0,2,1) model for the sales series 16

17 Looking at the diagnostic plots in Figure 4.3, the ARIMA(0,2,1) seems to fit well. The error terms appear Gaussian, there are no strong autocorrelations in the residuals, and the error terms do not appear to be dependent RELATIONSHIP BETWEEN sales AND lead I examine the CCF of sales and lead and a lag plot of sales t and lead t 3 to determine if a regression involving these variables is reasonable. ccf(diff(sales), diff(lead), main = "CCF of sales and lead") As seen in Figure 4.4, while sales and lead are often uncorrelated, around lag 3 they become highly correlated. This fact is emphasized by a lag plot. lag2.plot(lead, sales, max.lag = 3) Figure 4.5 shows a linear relationship with a third lag of lead and contemporary sales. This would justify regressing sales t on lead t REGRESSION WITH ARMA ERRORS Given that the variable lead seems to provide useful information about sales, I try to regress sales on lead. More specifically, I try to regress sales t on lead t 3, while viewing the error term as being some unknown ARMA process. saleslead <- ts.intersect(diff(sales), lag(diff(lead), k = -3)) salesnew <- saleslead[,1] leadnew <- saleslead[,2] fit <- lm(salesnew ~ leadnew) acf2(resid(fit)) ACF PACF [1,] [2,] [3,] [4,] [5,] [6,] [7,] [8,] [9,] [10,] [11,]

18 CCF of sales and lead ACF Lag Figure 4.4: CCF of sales and lead 18

19 lead(t 0) lead(t 1) sales(t) sales(t) lead(t 2) lead(t 3) sales(t) sales(t) Figure 4.5: Lag plot of sales and lead 19

20 Series: resid(fit) ACF LAG PACF LAG Figure 4.6: Sample ACF and PACF for residuals from linear fit 20

21 [12,] [13,] [14,] [15,] [16,] [17,] [18,] [19,] [20,] [21,] [22,] [23,] Figure 4.6 shows the ACF and the PACF of the residuals of the "naïve" fit. The PACF cuts off at 1 and the ACF trails off, so this appears to be an AR(1) process. arima.fit <- sarima(salesnew, 1, 0, 0, xreg=cbind(leadnew), details = F) As shown in Figure 4.7, the diagnostic plots for the process, when interpreting the error terms as an ARMA(1,0) process, look very good. Normality of the white noise residuals, the ACF of the white noise residuals, and the tests of dependence all show desirable properties. stargazer(arima.fit$fit, covariate.labels = c("$\\phi$", "Intercept", "$\\Delta \\text{lead}_{t-3}$"), dep.var.labels = c("$\\delta \\text{sales}_t$"), label = "tab:prob35h", title = "Coefficients of the model for $\\Delta \\text{sales}_t$", table.placement = "ht") Table 4.1 shows the estimates of the coefficients of the model. The AR(1) term (φ) is statistically significant and so is the intercept and the coefficient of the lead t 3 term. 5 MULTIPLICATIVE SEASONAL ARIMA MODELS 5.1 ACF OF AN ARIMA(p,d, q) (P,D,Q) s MODEL The problem asks for a plot of the theoretical ACF of an ARIMA(1,0,0) (0,0,1) 1 2 model, with Φ = 0.8 and θ = 0.5. This model is: The ACF is computed and plotted below: x t =.8x t 12 + w t +.5w t 1 (5.1) 21

22 Standardized Residuals Time ACF of Residuals Normal Q Q Plot of Std Residuals ACF Sample Quantiles LAG Theoretical Quantiles p values for Ljung Box statistic p value lag Figure 4.7: Diagnostic plots for the model for the sales series with ARMA(1,0) error terms 22

23 Table 4.1: Coefficients of the model for sales t Dependent variable: φ Intercept lead t 3 sales t (0.063) (0.177) (0.143) Observations 146 Log Likelihood σ Akaike Inf. Crit Note: p<0.1; p<0.05; p<0.01 ACF <- ARMAacf(ar = c(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,.8), ma = c(.5)) plot(acf, type = "h", xlab = "lag", xlim = c(1, 15), ylim = c(-.5,1)); abline(h=0) Figure 5.1 shows the theoretical ACF of the process. 23

24 ACF lag Figure 5.1: ACF of a seasonal ARIMA process 24