Time-Series Regression and Generalized Least Squares in R*
|
|
- Cory Cain
- 5 years ago
- Views:
Transcription
1 Time-Series Regression and Generalized Least Squares in R* An Appendix to An R Companion to Applied Regression, third edition John Fox & Sanford Weisberg last revision: Abstract Generalized least-squares (GLS) regression extends ordinary least-squares (OLS) estimation of the normal linear model by providing for possibly unequal error variances and for correlations between different errors. A common application of GLS estimation is to time-series regression, in which it is generally implausible to assume that errors are independent. This appendix to Fox and Weisberg (2019) briefly reviews GLS estimation and demonstrates its application to time-series data using the gls() function in the nlme package, which is part of the standard R distribution. 1 Generalized Least Squares In the standard linear model (for example, in Chapter 4 of the R Companion), or, equivalently E(y X) = Xβ y = Xβ + ε where y is the n 1 response vector; X is an n k +1 model matrix, typically with an initial column of 1s for the regression constant; β is a k vector of regression coefficients to estimate; and ε is an n 1 vector of errors. Assuming that ε N n (0, σ 2 I n ), or at least that the errors are uncorrelated and equally variable, leads to the familiar ordinary-least-squares (OLS) estimator of β, with covariance matrix b OLS = (X X) 1 X y Var(b OLS ) = σ 2 (X X) 1 More generally, we can assume that ε N n (0, Σ), where the error covariance matrix Σ is symmetric and positive-definite. Different diagonal entries in Σ error variances that are not necessarily all equal, while nonzero off-diagonal entries correspond to correlated errors. Suppose, for the time-being, that Σ is known. Then, the log-likelihood for the model is log L(β) = n 2 log 2π 1 2 log(det Σ) 1 2 (y Xβ) Σ 1 (y Xβ) which is maximimized by the generalized-least-squares (GLS) estimator of β, b GLS = (X Σ 1 X) 1 X Σ 1 y with covariance matrix Var(b GLS ) = (X Σ 1 X) 1 1
2 For example, when Σ is a diagonal matrix of (generally) unequal error variances, then b GLS is just the weighted-least-squares (WLS) estimator, which can be fit in R by the lm() function, specifying the weights arguments (see, e.g., Section of the R Companion). In a real application, of course, the error covariance matrix Σ is not known, and must be estimated from the data along with the regression coefficients β. However, Σ has up to n(n+1)/2 free elements, so this general model has more parameters than data points. To make progress we require restrictions on the elements of Σ. 2 Serially Correlated Errors One common context in which the errors from a regression model are unlikely to be independent is in time-series data, where the cases represent different moments or intervals of time, usually equally spaced. We will assume that the process generating the regression errors is stationary: That is, all of the errors have the same expectation (already assumed to be 0) and the same variance (σ 2 ), and the covariance of two errors depends only upon their separation s in time: 1 C(ε t, ε t+s ) = C(ε t, ε t s ) = σ 2 ρ s where ρ s is the error autocorrelation at lag s. In this situation, the error covariance matrix has the following structure: 1 ρ 1 ρ 2 ρ n 1 ρ 1 1 ρ 1 ρ n 2 ρ 2 ρ 1 1 ρ n 3 Σ = σ 2 = σ 2 P ρ n 1 ρ n 2 ρ n 3 1 If we knew the values of σ 2 and the ρs, then we could apply this result to find the GLS estimator of β in a time-series regression, but, of course, these are generally unknown parameters. Moreover, while they are many fewer than the number of elements in the unrestricted error covariance matrix Σ, the large number (n 1) of different ρs makes their estimation (along with σ 2 ) impossible without specifying additional structure for the autocorrelated errors. There are several standard models for stationary time-series; the most common for autocorrelated regression errors is the first-order auto-regressive process, AR(1): ε t = φε t 1 + ν t where the random shocks ν t are assumed to be Gaussian white noise, NID(0, σ 2 ν). Under this model, ρ 1 = φ, ρ s = φ s, and σ 2 = σ 2 ν/(1 φ 2 ). Since we must have φ < 1, the error autocorrelations ρ s decay exponentially towards 0 as s increases. 2 Higher-order autoregressive models are a direct generalization of the first-order model; for example, the second-order autoregressive model, denoted AR(2), is ε t = φ 1 ε t 1 + φ 2 ε t 2 + ν t In contrast, in the first-order moving-average process, MA(1), the current error depends upon the random shock from the current and previous periods (rather than upon the previous regression error), ε t = ν t + ψν t 1 1 Adjacent cases are taken by convention to be separated by 1 unit of time e.g., 1 year in annual time-series data. 2 For the AR(1) process to be stationary, φ cannot be equal to 1. 2
3 and higher-order MA(q) processes are similarly defined. Finally, AR and MA terms are combined in ARMA(p, q) processes; for example, ARMA(1, 1) errors follow the process ε t = φε t 1 + ν t + ψν t 1 Examining the residual autocorrelations from a preliminary OLS regression can suggest a reasonable form for the error-generating process. 3 The lag-s residual autocorrelation is n t=s+1 r s = e te t s n t=1 e2 t If the residuals were independently distributed (which they are not), the standard error of each r s would be approximately 1/ n, a quantity that can be used as a rough guide to the sampling variability of the residual autocorrelations. A more accurate approach to testing hypotheses about autocorrelated errors is to calculate the Dubin-Watson statistics, n t=s+1 D s = (e t e t s ) 2 n t=1 e2 t which have a known, if complex, sampling distribution that depends upon the model matrix X. When the sample size is large, D s 2(1 r s ), and so Durbin-Watson statistics near 2 are indicative of small residual autocorrelation, those below 2 of positive autocorrelation, and those above 2 of negative autocorrelation. 3 Using The gls() Function in R The gls() function in the nlme package (Pinheiro et al., 2018), which is part of the standard R distribution, fits regression models with a variety of correlated-error and non-constant error-variance structures. 4 To illustrate the use of gls(), let us examine time-series data on women s crime rates in Canada, analyzed by Fox and Hartnagel (1979). The data are in the Hartnagel data set in the cardata package, which we load along with the car package: 5 : library("car") Loading required package: cardata brief(hartnagel, c(6, 2)) 38 x 8 data.frame (30 rows omitted) year tfr partic degrees fconvict ftheft mconvict mtheft [i] [i] [i] [n] [n] [n] [n] [n] NA NA NA NA NA NA 3 In identifying an ARMA process, it helps to look as well at the partial autocorrelations of the residuals. For example, an AR(1) process has an exponentially decaying autocorrelation function, and a partial autocorrelation function with a single nonzero spike at lag 1. Conversely, an MA(1) process has an exponentially decaying partial autocorrelation function, and an autocorrelation function with a single nonzero spike at lag 1. Of course, these neat theoretical patterns are subject to sampling error. 4 The nlme package also has functions for fitting linear and nonlinear mixed models, as described in Chapter 7 of the R Companion and the on-line appendix on nonlinear regression. 5 R functions used but not described in this appendix are discussed in?. All the R code used in this appendix can be downloaded from Alternatively, if you are running R and attached to the internet, load the car package and enter the command carweb(script="appendix-timeseries"). 3
4 Convictions per 100,000 Women year Figure 1: Time series of Canadian women s indictable-offense conviction rate, NA NA The variables in the data set are as follows: ˆ year, ˆ tfr, the total fertility rate, births per 1000 women. ˆ partic, women s labor-force participation rate, per ˆ degrees, women s post-secondary degree rate, per 10,000. ˆ fconvict, women s indictable-offense conviction rate, per 100,000. ˆ ftheft, women s theft conviction rate, per 100,000. ˆ mconvict, men s indictable-offense conviction rate, per 100,000. ˆ mtheft, men s theft conviction rate, per 100,000. We will estimate the regression of fconvict on tfr, partic, degrees, and mconvict. The rationale for including the last predictor is to control for omitted variables that affect the crime rate in general. Let us begin by examining the time series for the women s conviction rate (Figure 1): plot(fconvict ~ year, type="n",data=hartnagel, ylab="convictions per 100,000 Women") grid(lty=1) with(hartnagel, points(year, fconvict, type="o", pch=16)) To add grid lines, we used a sequence of commands to draw this graph. 6 First the plot() function is used to set up axes and labels. The argument type="n" suppresses drawing any points. The 6 See Chapter 9 of the R Companion for general information about drawing graphs in R. 4
5 function grid() adds grid lines before adding points and lines. The call to points() adds both points and lines, with arguments type="o" to overplots points and lines, as is traditional for a timeseries graph, and pch=16 to use filled dots as the plotting characters. 7 We can see that the women s conviction rate fluctuated substantially but gradually during this historical period, with no apparent overall trend. A preliminary OLS regression produces the following fit to the data: mod.ols <- lm(fconvict ~ tfr + partic + degrees + mconvict, data=hartnagel) summary(mod.ols) Call: lm(formula = fconvict ~ tfr + partic + degrees + mconvict, data = Hartnagel) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) tfr e-06 partic degrees mconvict Residual standard error: 19.2 on 33 degrees of freedom Multiple R-squared: 0.695, Adjusted R-squared: F-statistic: 18.8 on 4 and 33 DF, p-value: 3.91e-08 The women s crime rate, therefore, appears to decline with fertility and increase with labor-force participation; the coefficients for the other two predictors have large p-values. A graph of the residuals from the OLS regression (Figure 2), however, suggests that they may be substantially autocorrelated: 8 plot(hartnagel$year, residuals(mod.ols), type="o", pch=16, xlab="year", ylab="ols Residuals") abline(h=0, lty=2) The acf() function in the R stats package computes and plots the autocorrelation and partialautocorrelation functions of a time series, here for the OLS residuals (Figure 3), because the residuals vary systematically with time: acf(residuals(mod.ols)) acf(residuals(mod.ols), type="partial") The broken horizontal lines on the plots correspond to approximate 95% confidence limits. The general pattern of the autocorrelation and partial autocorrelation functions sinusoidal decay in the 7 There is a ts.plot() function in the stats package in R for graphing time-series data. Although we will not need them, it is also possible to define special time-series data objects in R. For more information, consult?ts. 8 There also seems to be something unusual going on during World War II that is not accounted for by the predictors, a subject that we will not pursue here. 5
6 OLS Residuals Year Figure 2: Residuals from the OLS regression of women s conviction rate on several predictors. Series residuals(mod.ols) ACF Lag Series residuals(mod.ols) Partial ACF Lag Figure 3: Autocorrelation and partial-autocorrelation functions for the residuals from the OLS regression of women s conviction rate on several predictors. 6
7 former; two spikes, one positive, the other negative, in the latter is suggestive of an AR(2) process with φ 1 > 0 and φ 2 < 0. We follow up by computing Durbin-Watson statistics for the OLS regression, using the durbin- WatsonTest() function in the car package. By default, this function computes bootstrapped p - values for the Durbin-Watson statistics: 9 durbinwatsontest(mod.ols, max.lag=5) lag Autocorrelation D-W Statistic p-value Alternative hypothesis: rho[lag]!= 0 Three of the first five Durbin-Watson statistics have small p=values, including the first. As an alternative, the dwtest() function in the lmtest package (Zeileis and Hothorn, 2002) computes the p-value for the first-order Durbin-Watson statistic analytically: library("lmtest") Loading required package: zoo Attaching package: 'zoo' The following objects are masked from 'package:base': as.date, as.date.numeric dwtest(mod.ols, alternative="two.sided") Durbin-Watson test data: mod.ols DW = 0.617, p-value = 1.4e-08 alternative hypothesis: true autocorrelation is not 0 Many of the arguments for the gls() function are the same as for lm() in particular, gls() takes model, data, subset, and na.action arguments. ˆ In gls(), na.action defaults to na.fail: Missing data in a time-series in any event require special consideration. ˆ The weights argument to gls() can be used to specify a model for the error variance, ˆ As we will illustrate presently, the correlation argument can be used to specify a model for error autocorrelation. ˆ The method argument selects the method of estimation method="ml" for maximum-likelihood estimation See Section of the R Companion and the on-line appendix on bootstrapping. 10 The default method for gls() is "REML" for REstricted Maximum-Likelihood, which may be thought of as correcting for degrees of freedom. In the current illustration, REML estimation produces rather different results from ML. (Try it!) To see the full range of arguments to gls(), consult the on-line help. 7
8 For the Canadian women s crime data: library("nlme") mod.gls <- gls(fconvict ~ tfr + partic + degrees + mconvict, data=hartnagel, correlation=corarma(p=2), method="ml") summary(mod.gls) Generalized least squares fit by maximum likelihood Model: fconvict ~ tfr + partic + degrees + mconvict Data: Hartnagel AIC BIC loglik Correlation Structure: ARMA(2,0) Formula: ~1 Parameter estimate(s): Phi1 Phi Coefficients: Value Std.Error t-value p-value (Intercept) tfr partic degrees mconvict Correlation: (Intr) tfr partic degres tfr partic degrees mconvict Standardized residuals: Min Q1 Med Q3 Max Residual standard error: Degrees of freedom: 38 total; 33 residual Specifying the correlation structure as correlation=corarma(p=2) fits an AR(2) process for the errors; that is, the moving-average component is implicitly of order q=0, and hence is absent. In this instance, the ML estimates of the regression parameters under the AR(2) error-correlation model are not terribly different from the OLS estimates, although the coefficient for mconvict now has a smaller p-value. The ML estimates of the error-autoregressive parameters are sizable, φ 1 = and φ 2 = We can employ likelihood-ratio tests to check whether the parameters of the AR(2) process for the errors are necessary, and whether a second-order autoregressive model is sufficient. We proceed by updating the original "gls" model, respecifying the time-series process for the errors; we then compare nested models using the generic anova() function, which has a method for "gls" objects: 8
9 mod.gls.3 <- update(mod.gls, correlation=corarma(p=3)) mod.gls.1 <- update(mod.gls, correlation=corarma(p=1)) mod.gls.0 <- update(mod.gls, correlation=null) anova(mod.gls, mod.gls.1) # AR(2) vs AR(1) Model df AIC BIC loglik Test L.Ratio p-value mod.gls mod.gls vs anova(mod.gls, mod.gls.0) # AR(2) vs uncorrelated errors Model df AIC BIC loglik Test L.Ratio p-value mod.gls mod.gls vs <.0001 anova(mod.gls.3, mod.gls) # AR(3) vs AR(2) Model df AIC BIC loglik Test L.Ratio p-value mod.gls mod.gls vs An AR(3) specification would be unusually complicated, but in any event the tests support the AR(2) specification. 4 Complementary Reading and References Time-series regression and GLS estimation are covered in Fox (2016, Chap. 16). GLS estimation is a standard topic in econometrics texts. There are substantial treatments in Judge et al. (1985) and in Greene (2018), for example. Likewise, ARMA models are a standard topic in the time-series literature; see, for example, Chatfield (2003). Time-series regression is also a much larger topic than GLS estimation of linear models with autocorrelated errors; in additions to the references cited above, see Pickup (2015) for a broad and accessible overview with social-science examples. References Chatfield, C. (2003). Analysis of Time Series: An Introduction. Chapman and Hall/CRC, London, sixth edition. Fox, J. (2016). Applied Regression Analysis and Generalized Linear Models. Sage, Thousand Oaks CA, third edition. Fox, J. and Hartnagel, T. F. (1979). Changing social roles and female crime in Canada: a time series analysis. Canadian Review of Sociology and Anthropology, 16: Fox, J. and Weisberg, S. (2019). An R Companion to Applied Regression. Sage, Thousand Oaks, CA, third edition. Greene, W. H. (2018). Econometric Analysis. Pearson, Upper Saddle River, NJ, eighth edition. Judge, G. G., Griffiths, W. E., Hill, R. C., Lütkepohl, H., and Lee, T.-C. (1985). The Theory and Practice of Econometrics. Wiley, New York, second edition. Pickup, M. (2015). Introduction to Time Series Analysis. Sage, Thousand Oaks CA. 9
10 Pinheiro, J., Bates, D., DebRoy, S., Sarkar, D., and R Core Team (2018). nlme: Linear and Nonlinear Mixed Effects Models. R package version Zeileis, A. and Hothorn, T. (2002). Diagnostic checking in regression relationships. R News, 2(3):
Non-independence due to Time Correlation (Chapter 14)
Non-independence due to Time Correlation (Chapter 14) When we model the mean structure with ordinary least squares, the mean structure explains the general trends in the data with respect to our dependent
More information22s:152 Applied Linear Regression. Returning to a continuous response variable Y...
22s:152 Applied Linear Regression Generalized Least Squares Returning to a continuous response variable Y... Ordinary Least Squares Estimation The classical models we have fit so far with a continuous
More information22s:152 Applied Linear Regression. In matrix notation, we can write this model: Generalized Least Squares. Y = Xβ + ɛ with ɛ N n (0, Σ)
22s:152 Applied Linear Regression Generalized Least Squares Returning to a continuous response variable Y Ordinary Least Squares Estimation The classical models we have fit so far with a continuous response
More informationITSx: Policy Analysis Using Interrupted Time Series
ITSx: Policy Analysis Using Interrupted Time Series Week 3 Slides Michael Law, Ph.D. The University of British Columbia Layout of the weeks 1. Introduction, setup, data sources 2. Single series interrupted
More informationInference with Heteroskedasticity
Inference with Heteroskedasticity Note on required packages: The following code requires the packages sandwich and lmtest to estimate regression error variance that may change with the explanatory variables.
More informationCircle the single best answer for each multiple choice question. Your choice should be made clearly.
TEST #1 STA 4853 March 6, 2017 Name: Please read the following directions. DO NOT TURN THE PAGE UNTIL INSTRUCTED TO DO SO Directions This exam is closed book and closed notes. There are 32 multiple choice
More informationBIOL 458 BIOMETRY Lab 9 - Correlation and Bivariate Regression
BIOL 458 BIOMETRY Lab 9 - Correlation and Bivariate Regression Introduction to Correlation and Regression The procedures discussed in the previous ANOVA labs are most useful in cases where we are interested
More informationR Output for Linear Models using functions lm(), gls() & glm()
LM 04 lm(), gls() &glm() 1 R Output for Linear Models using functions lm(), gls() & glm() Different kinds of output related to linear models can be obtained in R using function lm() {stats} in the base
More informationThese slides illustrate a few example R commands that can be useful for the analysis of repeated measures data.
These slides illustrate a few example R commands that can be useful for the analysis of repeated measures data. We focus on the experiment designed to compare the effectiveness of three strength training
More informationRegression of Time Series
Mahlerʼs Guide to Regression of Time Series CAS Exam S prepared by Howard C. Mahler, FCAS Copyright 2016 by Howard C. Mahler. Study Aid 2016F-S-9Supplement Howard Mahler hmahler@mac.com www.howardmahler.com/teaching
More informationWorkshop 9.3a: Randomized block designs
-1- Workshop 93a: Randomized block designs Murray Logan November 23, 16 Table of contents 1 Randomized Block (RCB) designs 1 2 Worked Examples 12 1 Randomized Block (RCB) designs 11 RCB design Simple Randomized
More informationSolution to Series 6
Dr. M. Dettling Applied Series Analysis SS 2014 Solution to Series 6 1. a) > r.bel.lm summary(r.bel.lm) Call: lm(formula = NURSING ~., data = d.beluga) Residuals: Min 1Q
More informationLinear, Generalized Linear, and Mixed-Effects Models in R. Linear and Generalized Linear Models in R Topics
Linear, Generalized Linear, and Mixed-Effects Models in R John Fox McMaster University ICPSR 2018 John Fox (McMaster University) Statistical Models in R ICPSR 2018 1 / 19 Linear and Generalized Linear
More informationLecture 9 STK3100/4100
Lecture 9 STK3100/4100 27. October 2014 Plan for lecture: 1. Linear mixed models cont. Models accounting for time dependencies (Ch. 6.1) 2. Generalized linear mixed models (GLMM, Ch. 13.1-13.3) Examples
More informationat least 50 and preferably 100 observations should be available to build a proper model
III Box-Jenkins Methods 1. Pros and Cons of ARIMA Forecasting a) need for data at least 50 and preferably 100 observations should be available to build a proper model used most frequently for hourly or
More informationRef.: Spring SOS3003 Applied data analysis for social science Lecture note
SOS3003 Applied data analysis for social science Lecture note 05-2010 Erling Berge Department of sociology and political science NTNU Spring 2010 Erling Berge 2010 1 Literature Regression criticism I Hamilton
More informationCircle a single answer for each multiple choice question. Your choice should be made clearly.
TEST #1 STA 4853 March 4, 215 Name: Please read the following directions. DO NOT TURN THE PAGE UNTIL INSTRUCTED TO DO SO Directions This exam is closed book and closed notes. There are 31 questions. Circle
More informationFORECASTING SUGARCANE PRODUCTION IN INDIA WITH ARIMA MODEL
FORECASTING SUGARCANE PRODUCTION IN INDIA WITH ARIMA MODEL B. N. MANDAL Abstract: Yearly sugarcane production data for the period of - to - of India were analyzed by time-series methods. Autocorrelation
More informationTIME SERIES ANALYSIS AND FORECASTING USING THE STATISTICAL MODEL ARIMA
CHAPTER 6 TIME SERIES ANALYSIS AND FORECASTING USING THE STATISTICAL MODEL ARIMA 6.1. Introduction A time series is a sequence of observations ordered in time. A basic assumption in the time series analysis
More informationAPPLIED ECONOMETRIC TIME SERIES 4TH EDITION
APPLIED ECONOMETRIC TIME SERIES 4TH EDITION Chapter 2: STATIONARY TIME-SERIES MODELS WALTER ENDERS, UNIVERSITY OF ALABAMA Copyright 2015 John Wiley & Sons, Inc. Section 1 STOCHASTIC DIFFERENCE EQUATION
More informationReading Assignment. Serial Correlation and Heteroskedasticity. Chapters 12 and 11. Kennedy: Chapter 8. AREC-ECON 535 Lec F1 1
Reading Assignment Serial Correlation and Heteroskedasticity Chapters 1 and 11. Kennedy: Chapter 8. AREC-ECON 535 Lec F1 1 Serial Correlation or Autocorrelation y t = β 0 + β 1 x 1t + β x t +... + β k
More informationEconometrics 1. Lecture 8: Linear Regression (2) 黄嘉平
Econometrics 1 Lecture 8: Linear Regression (2) 黄嘉平 中国经济特区研究中 心讲师 办公室 : 文科楼 1726 E-mail: huangjp@szu.edu.cn Tel: (0755) 2695 0548 Office hour: Mon./Tue. 13:00-14:00 The linear regression model The linear
More informationA Handbook of Statistical Analyses Using R 2nd Edition. Brian S. Everitt and Torsten Hothorn
A Handbook of Statistical Analyses Using R 2nd Edition Brian S. Everitt and Torsten Hothorn CHAPTER 12 Analysing Longitudinal Data I: Computerised Delivery of Cognitive Behavioural Therapy Beat the Blues
More informationA Handbook of Statistical Analyses Using R 2nd Edition. Brian S. Everitt and Torsten Hothorn
A Handbook of Statistical Analyses Using R 2nd Edition Brian S. Everitt and Torsten Hothorn CHAPTER 12 Analysing Longitudinal Data I: Computerised Delivery of Cognitive Behavioural Therapy Beat the Blues
More informationForecasting using R. Rob J Hyndman. 2.4 Non-seasonal ARIMA models. Forecasting using R 1
Forecasting using R Rob J Hyndman 2.4 Non-seasonal ARIMA models Forecasting using R 1 Outline 1 Autoregressive models 2 Moving average models 3 Non-seasonal ARIMA models 4 Partial autocorrelations 5 Estimation
More informationCollinearity: Impact and Possible Remedies
Collinearity: Impact and Possible Remedies Deepayan Sarkar What is collinearity? Exact dependence between columns of X make coefficients non-estimable Collinearity refers to the situation where some columns
More informationProblem Set 2: Box-Jenkins methodology
Problem Set : Box-Jenkins methodology 1) For an AR1) process we have: γ0) = σ ε 1 φ σ ε γ0) = 1 φ Hence, For a MA1) process, p lim R = φ γ0) = 1 + θ )σ ε σ ε 1 = γ0) 1 + θ Therefore, p lim R = 1 1 1 +
More informationCh 8. MODEL DIAGNOSTICS. Time Series Analysis
Model diagnostics is concerned with testing the goodness of fit of a model and, if the fit is poor, suggesting appropriate modifications. We shall present two complementary approaches: analysis of residuals
More informationStat 209 Lab: Linear Mixed Models in R This lab covers the Linear Mixed Models tutorial by John Fox. Lab prepared by Karen Kapur. ɛ i Normal(0, σ 2 )
Lab 2 STAT209 1/31/13 A complication in doing all this is that the package nlme (lme) is supplanted by the new and improved lme4 (lmer); both are widely used so I try to do both tracks in separate Rogosa
More informationG. S. Maddala Kajal Lahiri. WILEY A John Wiley and Sons, Ltd., Publication
G. S. Maddala Kajal Lahiri WILEY A John Wiley and Sons, Ltd., Publication TEMT Foreword Preface to the Fourth Edition xvii xix Part I Introduction and the Linear Regression Model 1 CHAPTER 1 What is Econometrics?
More informationEcon 300/QAC 201: Quantitative Methods in Economics/Applied Data Analysis. 17th Class 7/1/10
Econ 300/QAC 201: Quantitative Methods in Economics/Applied Data Analysis 17th Class 7/1/10 The only function of economic forecasting is to make astrology look respectable. --John Kenneth Galbraith show
More informationChapter 8: Model Diagnostics
Chapter 8: Model Diagnostics Model diagnostics involve checking how well the model fits. If the model fits poorly, we consider changing the specification of the model. A major tool of model diagnostics
More informationA Handbook of Statistical Analyses Using R. Brian S. Everitt and Torsten Hothorn
A Handbook of Statistical Analyses Using R Brian S. Everitt and Torsten Hothorn CHAPTER 6 Logistic Regression and Generalised Linear Models: Blood Screening, Women s Role in Society, and Colonic Polyps
More informationCHAPTER 8 MODEL DIAGNOSTICS. 8.1 Residual Analysis
CHAPTER 8 MODEL DIAGNOSTICS We have now discussed methods for specifying models and for efficiently estimating the parameters in those models. Model diagnostics, or model criticism, is concerned with testing
More informationChapter 6: Model Specification for Time Series
Chapter 6: Model Specification for Time Series The ARIMA(p, d, q) class of models as a broad class can describe many real time series. Model specification for ARIMA(p, d, q) models involves 1. Choosing
More information5 Autoregressive-Moving-Average Modeling
5 Autoregressive-Moving-Average Modeling 5. Purpose. Autoregressive-moving-average (ARMA models are mathematical models of the persistence, or autocorrelation, in a time series. ARMA models are widely
More informationCase of single exogenous (iv) variable (with single or multiple mediators) iv à med à dv. = β 0. iv i. med i + α 1
Mediation Analysis: OLS vs. SUR vs. ISUR vs. 3SLS vs. SEM Note by Hubert Gatignon July 7, 2013, updated November 15, 2013, April 11, 2014, May 21, 2016 and August 10, 2016 In Chap. 11 of Statistical Analysis
More informationMISCELLANEOUS REGRESSION TOPICS
DEPARTMENT OF POLITICAL SCIENCE AND INTERNATIONAL RELATIONS Posc/Uapp 816 MISCELLANEOUS REGRESSION TOPICS I. AGENDA: A. Example of correcting for autocorrelation. B. Regression with ordinary independent
More informationPOLSCI 702 Non-Normality and Heteroskedasticity
Goals of this Lecture POLSCI 702 Non-Normality and Heteroskedasticity Dave Armstrong University of Wisconsin Milwaukee Department of Political Science e: armstrod@uwm.edu w: www.quantoid.net/uwm702.html
More informationLinear Probability Model
Linear Probability Model Note on required packages: The following code requires the packages sandwich and lmtest to estimate regression error variance that may change with the explanatory variables. If
More informationThe Problem. Regression With Correlated Errors. Generalized Least Squares. Correlated Errors. Consider the typical regression model.
The Problem Regression With Correlated Errors Consider the typical regression model y t = β z t + x t where x t is a process with covariance function γ(s, t). The matrix formulation is y = Z β + x where
More informationIntroduction and Background to Multilevel Analysis
Introduction and Background to Multilevel Analysis Dr. J. Kyle Roberts Southern Methodist University Simmons School of Education and Human Development Department of Teaching and Learning Background and
More information1. How can you tell if there is serial correlation? 2. AR to model serial correlation. 3. Ignoring serial correlation. 4. GLS. 5. Projects.
1. How can you tell if there is serial correlation? 2. AR to model serial correlation. 3. Ignoring serial correlation. 4. GLS. 5. Projects. 1) Identifying serial correlation. Plot Y t versus Y t 1. See
More informationLecture 2: Univariate Time Series
Lecture 2: Univariate Time Series Analysis: Conditional and Unconditional Densities, Stationarity, ARMA Processes Prof. Massimo Guidolin 20192 Financial Econometrics Spring/Winter 2017 Overview Motivation:
More informationModel Estimation Example
Ronald H. Heck 1 EDEP 606: Multivariate Methods (S2013) April 7, 2013 Model Estimation Example As we have moved through the course this semester, we have encountered the concept of model estimation. Discussions
More informationMixed effects models
Mixed effects models The basic theory and application in R Mitchel van Loon Research Paper Business Analytics Mixed effects models The basic theory and application in R Author: Mitchel van Loon Research
More informationSTAT Financial Time Series
STAT 6104 - Financial Time Series Chapter 4 - Estimation in the time Domain Chun Yip Yau (CUHK) STAT 6104:Financial Time Series 1 / 46 Agenda 1 Introduction 2 Moment Estimates 3 Autoregressive Models (AR
More informationChapter 5 Exercises 1
Chapter 5 Exercises 1 Data Analysis & Graphics Using R, 2 nd edn Solutions to Exercises (December 13, 2006) Preliminaries > library(daag) Exercise 2 For each of the data sets elastic1 and elastic2, determine
More informationUnivariate ARIMA Models
Univariate ARIMA Models ARIMA Model Building Steps: Identification: Using graphs, statistics, ACFs and PACFs, transformations, etc. to achieve stationary and tentatively identify patterns and model components.
More informationTime Series Analysis. James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY
Time Series Analysis James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY & Contents PREFACE xiii 1 1.1. 1.2. Difference Equations First-Order Difference Equations 1 /?th-order Difference
More informationSTATISTICS 174: APPLIED STATISTICS FINAL EXAM DECEMBER 10, 2002
Time allowed: 3 HOURS. STATISTICS 174: APPLIED STATISTICS FINAL EXAM DECEMBER 10, 2002 This is an open book exam: all course notes and the text are allowed, and you are expected to use your own calculator.
More informationCh 6. Model Specification. Time Series Analysis
We start to build ARIMA(p,d,q) models. The subjects include: 1 how to determine p, d, q for a given series (Chapter 6); 2 how to estimate the parameters (φ s and θ s) of a specific ARIMA(p,d,q) model (Chapter
More informationECON 4230 Intermediate Econometric Theory Exam
ECON 4230 Intermediate Econometric Theory Exam Multiple Choice (20 pts). Circle the best answer. 1. The Classical assumption of mean zero errors is satisfied if the regression model a) is linear in the
More informationWeek 11 Heteroskedasticity and Autocorrelation
Week 11 Heteroskedasticity and Autocorrelation İnsan TUNALI Econ 511 Econometrics I Koç University 27 November 2018 Lecture outline 1. OLS and assumptions on V(ε) 2. Violations of V(ε) σ 2 I: 1. Heteroskedasticity
More informationAutocorrelation. Jamie Monogan. Intermediate Political Methodology. University of Georgia. Jamie Monogan (UGA) Autocorrelation POLS / 20
Autocorrelation Jamie Monogan University of Georgia Intermediate Political Methodology Jamie Monogan (UGA) Autocorrelation POLS 7014 1 / 20 Objectives By the end of this meeting, participants should be
More information1 Forecasting House Starts
1396, Time Series, Week 5, Fall 2007 1 In this handout, we will see the application example on chapter 5. We use the same example as illustrated in the textbook and fit the data with several models of
More informationDealing with Heteroskedasticity
Dealing with Heteroskedasticity James H. Steiger Department of Psychology and Human Development Vanderbilt University James H. Steiger (Vanderbilt University) Dealing with Heteroskedasticity 1 / 27 Dealing
More informationEASTERN MEDITERRANEAN UNIVERSITY ECON 604, FALL 2007 DEPARTMENT OF ECONOMICS MEHMET BALCILAR ARIMA MODELS: IDENTIFICATION
ARIMA MODELS: IDENTIFICATION A. Autocorrelations and Partial Autocorrelations 1. Summary of What We Know So Far: a) Series y t is to be modeled by Box-Jenkins methods. The first step was to convert y t
More informationx 21 x 22 x 23 f X 1 X 2 X 3 ε
Chapter 2 Estimation 2.1 Example Let s start with an example. Suppose that Y is the fuel consumption of a particular model of car in m.p.g. Suppose that the predictors are 1. X 1 the weight of the car
More informationTIME SERIES DATA ANALYSIS USING EVIEWS
TIME SERIES DATA ANALYSIS USING EVIEWS I Gusti Ngurah Agung Graduate School Of Management Faculty Of Economics University Of Indonesia Ph.D. in Biostatistics and MSc. in Mathematical Statistics from University
More informationLast updated: Oct 18, 2012 LINEAR REGRESSION PSYC 3031 INTERMEDIATE STATISTICS LABORATORY. J. Elder
Last updated: Oct 18, 2012 LINEAR REGRESSION Acknowledgements 2 Some of these slides have been sourced or modified from slides created by A. Field for Discovering Statistics using R. Simple Linear Objectives
More informationTime Series Analysis. James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY
Time Series Analysis James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY PREFACE xiii 1 Difference Equations 1.1. First-Order Difference Equations 1 1.2. pth-order Difference Equations 7
More informationLab: Box-Jenkins Methodology - US Wholesale Price Indicator
Lab: Box-Jenkins Methodology - US Wholesale Price Indicator In this lab we explore the Box-Jenkins methodology by applying it to a time-series data set comprising quarterly observations of the US Wholesale
More informationUnivariate Time Series Analysis; ARIMA Models
Econometrics 2 Fall 24 Univariate Time Series Analysis; ARIMA Models Heino Bohn Nielsen of4 Outline of the Lecture () Introduction to univariate time series analysis. (2) Stationarity. (3) Characterizing
More informationLecture 6: Hypothesis Testing
Lecture 6: Hypothesis Testing Mauricio Sarrias Universidad Católica del Norte November 6, 2017 1 Moran s I Statistic Mandatory Reading Moran s I based on Cliff and Ord (1972) Kelijan and Prucha (2001)
More informationOutline. Nature of the Problem. Nature of the Problem. Basic Econometrics in Transportation. Autocorrelation
1/30 Outline Basic Econometrics in Transportation Autocorrelation Amir Samimi What is the nature of autocorrelation? What are the theoretical and practical consequences of autocorrelation? Since the assumption
More informationHomework 2. For the homework, be sure to give full explanations where required and to turn in any relevant plots.
Homework 2 1 Data analysis problems For the homework, be sure to give full explanations where required and to turn in any relevant plots. 1. The file berkeley.dat contains average yearly temperatures for
More informationLecture 2. The Simple Linear Regression Model: Matrix Approach
Lecture 2 The Simple Linear Regression Model: Matrix Approach Matrix algebra Matrix representation of simple linear regression model 1 Vectors and Matrices Where it is necessary to consider a distribution
More informationRoss Bettinger, Analytical Consultant, Seattle, WA
ABSTRACT DYNAMIC REGRESSION IN ARIMA MODELING Ross Bettinger, Analytical Consultant, Seattle, WA Box-Jenkins time series models that contain exogenous predictor variables are called dynamic regression
More informationUsing EViews Vox Principles of Econometrics, Third Edition
Using EViews Vox Principles of Econometrics, Third Edition WILLIAM E. GRIFFITHS University of Melbourne R. CARTER HILL Louisiana State University GUAY С LIM University of Melbourne JOHN WILEY & SONS, INC
More informationLecture 2: Linear Models. Bruce Walsh lecture notes Seattle SISG -Mixed Model Course version 23 June 2011
Lecture 2: Linear Models Bruce Walsh lecture notes Seattle SISG -Mixed Model Course version 23 June 2011 1 Quick Review of the Major Points The general linear model can be written as y = X! + e y = vector
More informationModule 3. Descriptive Time Series Statistics and Introduction to Time Series Models
Module 3 Descriptive Time Series Statistics and Introduction to Time Series Models Class notes for Statistics 451: Applied Time Series Iowa State University Copyright 2015 W Q Meeker November 11, 2015
More informationLECTURE 10: MORE ON RANDOM PROCESSES
LECTURE 10: MORE ON RANDOM PROCESSES AND SERIAL CORRELATION 2 Classification of random processes (cont d) stationary vs. non-stationary processes stationary = distribution does not change over time more
More informationIntroduction to Eco n o m et rics
2008 AGI-Information Management Consultants May be used for personal purporses only or by libraries associated to dandelon.com network. Introduction to Eco n o m et rics Third Edition G.S. Maddala Formerly
More informationResiduals and regression diagnostics: focusing on logistic regression
Big-data Clinical Trial Column Page of 8 Residuals and regression diagnostics: focusing on logistic regression Zhongheng Zhang Department of Critical Care Medicine, Jinhua Municipal Central Hospital, Jinhua
More informationCoping with Additional Sources of Variation: ANCOVA and Random Effects
Coping with Additional Sources of Variation: ANCOVA and Random Effects 1/49 More Noise in Experiments & Observations Your fixed coefficients are not always so fixed Continuous variation between samples
More informationProf. Dr. Roland Füss Lecture Series in Applied Econometrics Summer Term Introduction to Time Series Analysis
Introduction to Time Series Analysis 1 Contents: I. Basics of Time Series Analysis... 4 I.1 Stationarity... 5 I.2 Autocorrelation Function... 9 I.3 Partial Autocorrelation Function (PACF)... 14 I.4 Transformation
More informationCOMPUTER SESSION: ARMA PROCESSES
UPPSALA UNIVERSITY Department of Mathematics Jesper Rydén Stationary Stochastic Processes 1MS025 Autumn 2010 COMPUTER SESSION: ARMA PROCESSES 1 Introduction In this computer session, we work within the
More informationA Handbook of Statistical Analyses Using R 2nd Edition. Brian S. Everitt and Torsten Hothorn
A Handbook of Statistical Analyses Using R 2nd Edition Brian S. Everitt and Torsten Hothorn CHAPTER 7 Logistic Regression and Generalised Linear Models: Blood Screening, Women s Role in Society, Colonic
More informationIntroduction to the Analysis of Hierarchical and Longitudinal Data
Introduction to the Analysis of Hierarchical and Longitudinal Data Georges Monette, York University with Ye Sun SPIDA June 7, 2004 1 Graphical overview of selected concepts Nature of hierarchical models
More informationTesting for Unit Roots with Cointegrated Data
Discussion Paper No. 2015-57 August 19, 2015 http://www.economics-ejournal.org/economics/discussionpapers/2015-57 Testing for Unit Roots with Cointegrated Data W. Robert Reed Abstract This paper demonstrates
More informationIntroduction to Econometrics
Introduction to Econometrics T H I R D E D I T I O N Global Edition James H. Stock Harvard University Mark W. Watson Princeton University Boston Columbus Indianapolis New York San Francisco Upper Saddle
More informationUniversity of Oxford. Statistical Methods Autocorrelation. Identification and Estimation
University of Oxford Statistical Methods Autocorrelation Identification and Estimation Dr. Órlaith Burke Michaelmas Term, 2011 Department of Statistics, 1 South Parks Road, Oxford OX1 3TG Contents 1 Model
More informationOverview. 1. Independence. 2. Modeling Autocorrelation. 3. Temporal Autocorrelation Example. 4. Spatial Autocorrelation Example
6. Autocorrelation Overview 1. Independence 2. Modeling Autocorrelation 3. Temporal Autocorrelation Example 4. Spatial Autocorrelation Example 6.1 Independence 6.1 Independence. Model assumptions All linear
More informationSTK4900/ Lecture 10. Program
STK4900/9900 - Lecture 10 Program 1. Repeated measures and longitudinal data 2. Simple analysis approaches 3. Random effects models 4. Generalized estimating equations (GEE) 5. GEE for binary data (and
More informationOutline. Mixed models in R using the lme4 package Part 3: Longitudinal data. Sleep deprivation data. Simple longitudinal data
Outline Mixed models in R using the lme4 package Part 3: Longitudinal data Douglas Bates Longitudinal data: sleepstudy A model with random effects for intercept and slope University of Wisconsin - Madison
More informationLECTURE 11. Introduction to Econometrics. Autocorrelation
LECTURE 11 Introduction to Econometrics Autocorrelation November 29, 2016 1 / 24 ON PREVIOUS LECTURES We discussed the specification of a regression equation Specification consists of choosing: 1. correct
More informationDEPARTMENT OF ECONOMICS AND FINANCE COLLEGE OF BUSINESS AND ECONOMICS UNIVERSITY OF CANTERBURY CHRISTCHURCH, NEW ZEALAND
DEPARTMENT OF ECONOMICS AND FINANCE COLLEGE OF BUSINESS AND ECONOMICS UNIVERSITY OF CANTERBURY CHRISTCHURCH, NEW ZEALAND Testing For Unit Roots With Cointegrated Data NOTE: This paper is a revision of
More informationCh.10 Autocorrelated Disturbances (June 15, 2016)
Ch10 Autocorrelated Disturbances (June 15, 2016) In a time-series linear regression model setting, Y t = x tβ + u t, t = 1, 2,, T, (10-1) a common problem is autocorrelation, or serial correlation of the
More informationLecture 4a: ARMA Model
Lecture 4a: ARMA Model 1 2 Big Picture Most often our goal is to find a statistical model to describe real time series (estimation), and then predict the future (forecasting) One particularly popular model
More informationLAB 5 INSTRUCTIONS LINEAR REGRESSION AND CORRELATION
LAB 5 INSTRUCTIONS LINEAR REGRESSION AND CORRELATION In this lab you will learn how to use Excel to display the relationship between two quantitative variables, measure the strength and direction of the
More informationCorrelated Data: Linear Mixed Models with Random Intercepts
1 Correlated Data: Linear Mixed Models with Random Intercepts Mixed Effects Models This lecture introduces linear mixed effects models. Linear mixed models are a type of regression model, which generalise
More informationSection 2 NABE ASTEF 65
Section 2 NABE ASTEF 65 Econometric (Structural) Models 66 67 The Multiple Regression Model 68 69 Assumptions 70 Components of Model Endogenous variables -- Dependent variables, values of which are determined
More information1 Graphical method of detecting autocorrelation. 2 Run test to detect autocorrelation
1 Graphical method of detecting autocorrelation Residual plot : A graph of the estimated residuals ˆɛ i against time t is plotted. If successive residuals tend to cluster on one side of the zero line of
More informationCovariance Models (*) X i : (n i p) design matrix for fixed effects β : (p 1) regression coefficient for fixed effects
Covariance Models (*) Mixed Models Laird & Ware (1982) Y i = X i β + Z i b i + e i Y i : (n i 1) response vector X i : (n i p) design matrix for fixed effects β : (p 1) regression coefficient for fixed
More informationRoss Bettinger, Analytical Consultant, Seattle, WA
ABSTRACT USING PROC ARIMA TO MODEL TRENDS IN US HOME PRICES Ross Bettinger, Analytical Consultant, Seattle, WA We demonstrate the use of the Box-Jenkins time series modeling methodology to analyze US home
More informationTopic 4 Unit Roots. Gerald P. Dwyer. February Clemson University
Topic 4 Unit Roots Gerald P. Dwyer Clemson University February 2016 Outline 1 Unit Roots Introduction Trend and Difference Stationary Autocorrelations of Series That Have Deterministic or Stochastic Trends
More informationFitting mixed models in R
Fitting mixed models in R Contents 1 Packages 2 2 Specifying the variance-covariance matrix (nlme package) 3 2.1 Illustration:.................................... 3 2.2 Technical point: why to use as.numeric(time)
More informationWU Weiterbildung. Linear Mixed Models
Linear Mixed Effects Models WU Weiterbildung SLIDE 1 Outline 1 Estimation: ML vs. REML 2 Special Models On Two Levels Mixed ANOVA Or Random ANOVA Random Intercept Model Random Coefficients Model Intercept-and-Slopes-as-Outcomes
More informationHeteroskedasticity. y i = β 0 + β 1 x 1i + β 2 x 2i β k x ki + e i. where E(e i. ) σ 2, non-constant variance.
Heteroskedasticity y i = β + β x i + β x i +... + β k x ki + e i where E(e i ) σ, non-constant variance. Common problem with samples over individuals. ê i e ˆi x k x k AREC-ECON 535 Lec F Suppose y i =
More information