The Problem. Regression With Correlated Errors. Generalized Least Squares. Correlated Errors. Consider the typical regression model.

Transcription

1 The Problem Regression With Correlated Errors Consider the typical regression model y t = β z t + x t where x t is a process with covariance function γ(s, t). The matrix formulation is y = Z β + x where x = (x 1,..., x n ), y = (y 1,..., y n ), and Z = [z 1,..., z n ] When the residual process x t is uncorrelated, then standard theory applies producing the optimal estimator of β to be ˆβ = ( Z Z ) 1 Z y Arthur Berg Regression With Correlated Errors 3/ 21 Correlated Errors Generalized Least Squares However, if the process x has correlation, i.e. when Γ = {γ(s t)} n s,t=1 is not a constant multiple of the identity matrix, then the usual estimator ˆβ is no longer optimal. In this case the optimal estimator is the generalized least squares solution. Given the covariance matrix Γ, it is possible to find a transformation matrix A such that AΓA = I where I denotes the identity matrix (so in particular A A = A 1 ). Multiplying the matrix equation y = Z β + x by A yields the equation Ay = AZ β + Ax = Uβ + w where U = AZ and w is a white noise vector with covariance matrix I. Applying the usual estimator of β in the transformed equation gives ˆβ = (U U) 1 U Ay = (Z A AZ ) 1 Z A Ay = (Z Γ 1 Z ) 1 Z Γ 1 y Arthur Berg Regression With Correlated Errors 4/ 21 Arthur Berg Regression With Correlated Errors 5/ 21

2 Two Approaches Model Transformation with ARMA Errors Without knowing Γ = {γ(s t)} n s,t=1 we are at a loss. Two approaches to estimating Γ are provided and both assume stationarity of the process x t. Nonparametric Approach In this approach Γ is estimated by ˆΓ = {ˆγ(s t)} n s,t=1 = {ˆγ(h)}n s,t=1 However ˆγ(h) is poorly estimated when h is large and modifications are required. Parametric Approach This approach is due to Cochrane and Orcutt (1949) and involves fitting an ARMA model to the residuals then regressing a transformed model. Suppose then the transformation φ(b) θ(b) y t }{{} u t φ(b)x t = θ(b)w t = β φ(b) θ(b) z t }{{} v t +w t produces a least square model with uncorrelated errors, i.e. the model where w t is white noise. u t = β tv t + w t Arthur Berg Regression With Correlated Errors 6/ 21 Arthur Berg Regression With Correlated Errors 7/ 21 Cochrane and Orcutt (1949) Back to Cardiovascular Example 1 Obtain the residuals ˆx t = y t ˆβ z t via the usual least squares routine. 2 Fit an ARMA model to the residuals ˆx t, say, ˆφ(B)ˆx t = ˆθ(B)w t 3 Apply the ARMA transformation to both sides of linear model, i.e. compute u t = ˆφ(B) ˆθ(B) y t and v t = ˆφ(B) ˆθ(B) z t 4 Run ordinary least squares regression on the transformed regression model, i.e. compute where ˆβ = (V V ) 1 V u Arthur Berg Regression With Correlated Errors 8/ 21 Arthur Berg Regression With Correlated Errors 9/ 21

3 Scatterplot Matrix Four Models > pairs(cbind(mort, temp, part)) Data from 1970 to 1979 in LA. M t represents the cardiac mortality. T t represents the temperatures. P t represents the particulate pollution. Four proposed models 1 M t = β 0 + β 1 t + w t 2 M t = β 0 + β 1 t + β 2 (T t T. ) + w t 3 M t = β 0 + β 1 t + β 2 (T t T. ) + β 3 (T t T. ) 2 + w t 4 M t = β 0 + β 1 t + β 2 (T t T. ) + β 3 (T t T. ) 2 + β 4 P t + w t Arthur Berg Regression With Correlated Errors 10/ 21 Arthur Berg Regression With Correlated Errors 11/ 21 Example from Text > temp = temp-mean(temp) > temp2 = temp^2 > trend = time(mort) > fit = lm(mort~ trend + temp + temp2 + part, na.action=null) > summary(fit) > acf(fit$resid) > pacf(fit$resid) lm(formula = mort ~ trend + temp + temp2 + part, na.action = NULL) (Intercept) 2.831e e < 2e-16 *** trend e e < 2e-16 *** temp e e < 2e-16 *** temp e e e-15 *** part 2.554e e < 2e-16 *** Residual standard error: on 503 degrees of freedom Multiple R-Squared: , Adjusted R-squared: F-statistic: 185 on 4 and 503 DF, p-value: < 2.2e-16 Arthur Berg Regression With Correlated Errors 12/ 21 Arthur Berg Regression With Correlated Errors 13/ 21

4 New Fit > (fit2<-ar.ols(fit$resid, aic=f,order=2 )) ar.ols(x = fit$resid, aic = F, order.max = 2) Intercept: (0.2472) Order selected 2 sigma^2 estimated as > Mort<-filter(mort, c(1,-.2205,-.3625),sides=1)[3:508] > Trend<-filter(trend, c(1,-.2205,-.3625),sides=1)[3:508] > Temp<-filter(temp, c(1,-.2205,-.3625),sides=1)[3:508] > Temp2<-filter(temp2, c(1,-.2205,-.3625),sides=1)[3:508] > Part<-filter(part, c(1,-.2205,-.3625),sides=1)[3:508] > > (fit3 = lm(mort~ Trend + Temp + Temp2 + Part)) lm(formula = Mort ~ Trend + Temp + Temp2 + Part) (Intercept) Trend Temp Temp2 Part Arthur Berg Regression With Correlated Errors 14/ 21 Arthur Berg Regression With Correlated Errors 15/ 21 Summary of Old Fit Summary of New Fit > summary(fit) lm(formula = mort ~ trend + temp + temp2 + part) > summary(fit3) lm(formula = Mort ~ Trend + Temp + Temp2 + Part) (Intercept) < 2e-16 *** trend < 2e-16 *** temp < 2e-16 *** temp e-15 *** part < 2e-16 *** Residual standard error: on 503 degrees of freedom Multiple R-Squared: , Adjusted R-squared: F-statistic: Arthur Berg 185 on 4 and Regression 503 With DF, Correlated p-value: Errors < 2.2e-16 16/ 21 (Intercept) < 2e-16 *** Trend e-12 *** Temp e-07 *** Temp e-13 *** Part < 2e-16 *** Residual standard error: on 501 degrees of freedom Multiple R-Squared: , Adjusted R-squared: F-statistic: on 4 and 501 DF, p-value: < 2.2e-16 Arthur Berg Regression With Correlated Errors 17/ 21

5 > acf(fit3$resid,lwd=3) > pacf(fit3$resid,lwd=3) Alternative Method gls > library(nlme) > (fit.gls = gls(mort~trend + temp + temp2 + part, correlation=corarma Generalized least squares fit by maximum likelihood Model: mort ~ trend + temp + temp2 + part Data: NULL Log-likelihood: (Intercept) trend temp temp2 part Arthur Berg Regression With Correlated Errors 18/ 21 Correlation Structure: ARMA(2,0) Formula: ~1 Parameter estimate(s): Phi1 Phi Degrees of freedom: 508 total; 503 residual Residual standard error: ACF plot of residuals showed strong correlation however, so may need some tweaking. Arthur Berg Regression With Correlated Errors 19/ 21 5c Using your assigned number, download your dataset from berg/sta4853/data/ and do the following: Plot the ACF and PACF of x1 Using the Box-Jenkins Method for SARIMA model selection, fit a SARIMA model to x1 Plot the estimated spectrum of x1 Compute the CCF of x1 and x2; which lag is most significant?; would you say x1 leads x2 or visa versa or neither one leads the other? Use the command lm to fit the model y = βx 3 + ε. Analyze the residuals for correlation and use the Cochrane and Orcutt algorithm to improve the estimate. Show plots of the ACF and PACF of the residuals in the original fit and the modified fit. Also provide the estimate of β for each fit. Arthur Berg Regression With Correlated Errors 21/ 21