Serial Correlation. Edps/Psych/Stat 587. Carolyn J. Anderson. Fall Department of Educational Psychology

Size: px
Start display at page:

Download "Serial Correlation. Edps/Psych/Stat 587. Carolyn J. Anderson. Fall Department of Educational Psychology"

Transcription

1 Serial Correlation Edps/Psych/Stat 587 Carolyn J. Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Fall 017

2 Model for Level 1 Residuals There are three sources of possible variance Between individuals, modeled by Z i U i where U i N(0,T). Within individuals, R it = e (1)it +e ()it Random (measurement error, variables not included, etc.), e (1)it N(0,σ ). Autocorrelated errors, e()it...need model for this. C.J. Anderson (Illinois) Serial Correlation Fall 017.1/ 97

3 Autocorrelated Errors: Mini-Outline Possible models for R it Back to Riesby data. Models for U i and R it. Time varying explanatory variables: Riesby see text by Hedeker & Gibbons (006). Detecting serial correlation. C.J. Anderson (Illinois) Serial Correlation Fall / 97

4 Possible models for R it In an HLM/linear mixed model, Y i (r i 1) = X i Γ + (r i p) (p 1) Z i U i + R i (r i q) (q 1) (r i 1) where i = 1,...,N and r i = number of time points for individual i. U i N(0,T). R i N(0,σ Ω i ) var(y i ) = V i = Z i TZ i +σ Ω i. C.J. Anderson (Illinois) Serial Correlation Fall / 97

5 Possible Models for Serial Correlation Autoregressive (AR). Moving average (MA). Autoregressive with a moving average (ARMA). Toeplitz. Others. C.J. Anderson (Illinois) Serial Correlation Fall / 97

6 Autoregressive Errors First order autoregressive process, AR1: Time 1: Time : R i1 = ǫ i1 R i = ρr i1 +ǫ i = ρǫ i1 +ǫ i Time 3: R i3 = ρr i +ǫ i3 = ρ(ρǫ i1 +ǫ i )+ǫ i3 Time t:.. R it = ρr i(t 1) +ǫ it C.J. Anderson (Illinois) Serial Correlation Fall / 97

7 Autoregressive Errors First order autoregressive process, AR1: where ǫ it N(0,σ ǫ) i.i.d. R it = ρr i(t 1) +ǫ it ρ is autocorrelation coefficient, 0 ρ < 1. Stationarity: variance of R it and covariance between R it and R it are independent of t. Resulting error variance structure... C.J. Anderson (Illinois) Serial Correlation Fall / 97

8 Autoregressive Errors σ Ω = σ ǫ (1 ρ ) 1 ρ ρ... ρ ri 1 ρ 1 ρ... ρ ri ρ ρ 1... ρ ri ρ ri 1 ρ ri ρ ri In SAS, AR1 is parameterized as 1 ρ ρ... ρ ri 1 ρ 1 ρ... ρ ri σ Ω = σǫ ρ ρ 1... ρ ri 3, ρ ri 1 ρ ri ρ ri where σ ǫ = σ ǫ/(1 ρ ).. C.J. Anderson (Illinois) Serial Correlation Fall / 97

9 Notes Regarding AR1 AR1 process is a regression equation in which R it depends on it s past values. Since R it only depends on it s past values, this is a Markov process. Ω is defined by ρ, the autocorrelation coefficient. Non-stationarity: If the variance of R it and the covariance between R it and R it increases over time, then you have non-stationarity. Not available in SAS. Available in Hedekker s MIXREG program. Not sure what s in R. C.J. Anderson (Illinois) Serial Correlation Fall / 97

10 Simulated Data: No Serial Correlation Random Intercept and Random Slope for Time: where Y it = 10+(time) it +U 0j +U 1j (time) it +R it (time)= t = 1...,0, and N = 50 individuals. U i N(0,T) with R it N(0,4). T = ( 4 4 ) C.J. Anderson (Illinois) Serial Correlation Fall / 97

11 No Serial Correlation: Data R it N(0,σ ) iid. C.J. Anderson (Illinois) Serial Correlation Fall / 97

12 No Serial Correlation: R it R it N(0,σ ) iid. C.J. Anderson (Illinois) Serial Correlation Fall / 97

13 No Serial Correlation: R it R it N(0,σ ) iid. C.J. Anderson (Illinois) Serial Correlation Fall / 97

14 Simulated AR1 Data Random intercept and random slope: where Y it = 10+(time) it +U 0j +U 1j (time) it +.75R i(t 1) +ǫ it (time)= t = 1...,0, and N = 50 individuals. U i N(0,T) with ǫ it N(0,4). R it =.75R i(t 1) +ǫ it. T = ( 4 4 ) C.J. Anderson (Illinois) Serial Correlation Fall / 97

15 Eg of AR1: The Data C.J. Anderson (Illinois) Serial Correlation Fall / 97

16 Eg of AR1: The R it C.J. Anderson (Illinois) Serial Correlation Fall / 97

17 Eg of AR1: Some R it C.J. Anderson (Illinois) Serial Correlation Fall / 97

18 Eg of AR1: OLS ˆR it C.J. Anderson (Illinois) Serial Correlation Fall / 97

19 Eg of AR1: OLS ˆR it C.J. Anderson (Illinois) Serial Correlation Fall / 97

20 Eg of AR1: OLS mean ˆR it C.J. Anderson (Illinois) Serial Correlation Fall / 97

21 Example of AR1 (continued) C.J. Anderson (Illinois) Serial Correlation Fall / 97

22 Example of AR1 (continued) C.J. Anderson (Illinois) Serial Correlation Fall 017.1/ 97

23 Example of AR1 (continued) C.J. Anderson (Illinois) Serial Correlation Fall / 97

24 SAS/MIXED and AR1 (continued) PROC MIXED data=new1 method=ml; CLASS i occasion; MODEL y= time / solution ; RANDOM intercept time /type=un subject=i solution G; REPEATED occasion / subject=i type=ar(1) R; REPEATED works much the same way the RANDOM does. Need two time variables (one continuous/numerical & one classification). C.J. Anderson (Illinois) Serial Correlation Fall / 97

25 SAS/MIXED and AR1 (continued) Covariance Parameter Estimates Cov Parm Subject Actual Estimate τ 00 UN(1,1) i τ 10 UN(,1) i τ 11 UN(,) i ρ AR(1) i σǫ Residual Note: ˆσ = ˆσ (1 ˆρ ) = 8.15(1.75 ) = The value used to simulate data was 4. C.J. Anderson (Illinois) Serial Correlation Fall / 97

26 SAS/MIXED and AR1 (continued) Estimated ˆσ ˆΩ r Row Col1 Col Col3 Col4 Col5 Col6 Col7 Co r C.J. Anderson (Illinois) Serial Correlation Fall / 97

27 First Order Moving Average Time 1: Time : Time 3: Time t: R i1 = ǫ i1 R i = ǫ i θǫ i1 R i3 = ǫ i3 θǫ i.. R it = ǫ it θǫ i,(t 1) where ǫ it N(0,σ ǫ) i.i.d. θ is the autocorrelation coefficient. C.J. Anderson (Illinois) Serial Correlation Fall / 97

28 First Order Moving Average The covariance matrix for R it is 1+θ θ θ 1+θ θ... 0 σǫω = σǫ 0 θ 1+θ θ. C.J. Anderson (Illinois) Serial Correlation Fall / 97

29 MA1 and SAS/MIXED SAS/MIXED doesn t estimate 1+θ θ θ 1+θ θ... 0 σǫω = σǫ 0 θ 1+θ θ The closest you can come to this is in SAS is TYPE=TOEP(), σ σ σ 1 σ σ cov(r i ) = 0 σ 1 σ σ. C.J. Anderson (Illinois) Serial Correlation Fall / 97

30 Simulated MA1 Data where Y it = 10+(time) it +U 0j +U 1j (time) it +ǫ it.75ǫ i,(t 1) (time)= t = 1...,0, and N = 50 individuals. U i N(0,T) with ǫ it N(0,4). R it = ǫ it.75ǫ i,(t 1). T = ( 4 4 ) C.J. Anderson (Illinois) Serial Correlation Fall / 97

31 Eg of MA1 Data C.J. Anderson (Illinois) Serial Correlation Fall / 97

32 Eg of MA1 R it C.J. Anderson (Illinois) Serial Correlation Fall / 97

33 Eg of MA1 R it C.J. Anderson (Illinois) Serial Correlation Fall / 97

34 Eg of MA1 OLS ˆR it C.J. Anderson (Illinois) Serial Correlation Fall / 97

35 Eg of MA1 OLS ˆR it C.J. Anderson (Illinois) Serial Correlation Fall / 97

36 Eg of MA1 Mean Sq Residuals C.J. Anderson (Illinois) Serial Correlation Fall / 97

37 Example of MA1 (continued) C.J. Anderson (Illinois) Serial Correlation Fall / 97

38 Example of MA1 (continued) C.J. Anderson (Illinois) Serial Correlation Fall / 97

39 SAS/MIXED and TOEP() (continued) Covariance Parameter Estimates Cov Parm Subject Estimate UN(1,1) i UN(,1) i UN(,) i TOEP() i Residual 6.66 C.J. Anderson (Illinois) Serial Correlation Fall / 97

40 SAS/MIXED and TOEP(): ˆσ ˆΩ Row Col1 Col Col3 Col4 Col5 Col6 Col C.J. Anderson (Illinois) Serial Correlation Fall / 97

41 Autoregressive-Moving Average ARMA(1,1) where ǫ it N(0,σ ǫ) iid. R it = ρr i,t 1 +ǫ it θǫ i,(t 1) ρ is autocorrelation coefficient, ρ < 1. Stationarity: variance of R it and covariance between R it and R it are independent of t. θ is the autocorrelation coefficient. C.J. Anderson (Illinois) Serial Correlation Fall / 97

42 ARMA(1,1) Error Covariance Matrix cov(r i ) = σ ǫ (1 ρ ) ξ 0 ξ 1 ρξ 1... ρ r ξ 1 ξ 1 ξ 0 ξ 1... ρ r 3 ξ ρξ 1 ξ 1 ξ 0... ρ r 4 ξ 1 ρ ξ 1 ρξ 1 ξ 1... ρ (r 5) ξ ρ (r ) ξ 1 ρ (r 3) ξ 1 ρ (r 4)ξ 1... ξ 0 where ξ 0 = 1+θ ρθ and ξ 1 = (1 ρθ)(ρ θ). The MA1 term, θ, changes the lag-1 autocorrelation and then autocorrelations deceases as in AR1. C.J. Anderson (Illinois) Serial Correlation Fall / 97

43 ARMA(1,1) SAS Parameterization cov(r i ) = σ 1 ξ ρξ... ρ r ξ ξ 1 ξ... ρ r 3 ξ ρξ ξ 1... ρ r 4 ξ ρ ξ ρξ ξ... ρ (r 5) ξ ρ (r ) ξ ρ (r 3) ξ ρ (r 4)ξ... 1 where σ = (σ ǫ/(1 ρ ))ξ 0 = (σ ǫ/(1 ρ ))(1+θ ρθ) ξ = (σ ǫ/(1 ρ ))(1 ρθ)(ρ θ). C.J. Anderson (Illinois) Serial Correlation Fall / 97

44 Simulated ARMA(1,1) Data Y it = 10+(time) it +U 0j +U 1j (time) it +R it where R it = ǫ it +.75R i,t 1.5ǫ it (time)= t = 1...,0, and N = 500 individuals. U i N(0,T) with T = ( 4 4 ) ǫ it N(0,4). C.J. Anderson (Illinois) Serial Correlation Fall / 97

45 Eg of ARMA(1,1): The data C.J. Anderson (Illinois) Serial Correlation Fall / 97

46 Eg of ARMA(1,1): The R it C.J. Anderson (Illinois) Serial Correlation Fall / 97

47 Eg of ARMA(1,1): The R it C.J. Anderson (Illinois) Serial Correlation Fall / 97

48 Eg of ARMA(1,1): The OLS ˆR it C.J. Anderson (Illinois) Serial Correlation Fall / 97

49 Eg of ARMA(1,1): The OLS ˆR it C.J. Anderson (Illinois) Serial Correlation Fall / 97

50 Eg of ARMA(1,1): The OLS ˆR it C.J. Anderson (Illinois) Serial Correlation Fall / 97

51 Example of ARMA(1,1) (continued) C.J. Anderson (Illinois) Serial Correlation Fall / 97

52 Example of ARMA(1,1) (continued) C.J. Anderson (Illinois) Serial Correlation Fall / 97

53 SAS/MIXED and ARMA(1,1) Covariance Parameter Estimates Cov Parm Subject Estimate τ 00 UN(1,1) i.5978 τ 10 UN(,1) i τ 11 UN(,) i ρ Rho i.690 ξ Gamma i.5358 Residual C.J. Anderson (Illinois) Serial Correlation Fall / 97

54 SAS/MIXED and ARMA(1,1) (continued) Covariance Matrix for R it : C.J. Anderson (Illinois) Serial Correlation Fall / 97

55 Other Error Structures Autocorrelations of each lag are functionally related: AR(1), MA(1), ARMA(1,1) Gaussian Fractional Polynomials Autocorrelations of each lag are not functionally related: Toeplitz Errors C.J. Anderson (Illinois) Serial Correlation Fall / 97

56 Toeplitz Errors General Toeplitz matrix: 1 ρ 1 ρ ρ 3... ρ r 1 ρ 1 1 ρ 1 ρ... ρ r Σ R = σ Ω = σ ρ ρ 1 1 ρ 1... ρ r ρ r 1 ρ r ρ r 3 ρ r Higher-order lags possible, but usually assumed to be 0. MA(1), which has lag of 1, is Toeplitz(). In SAS TYPE=TOEP(# lags +1) C.J. Anderson (Illinois) Serial Correlation Fall / 97

57 Back to Riesby Data Recall that we decided on the following model: Level 1 model: HamD it = β 0i +β 1i (time) it +β i (time) it +R it where R it N(0,σ ) i.i.d & cov(r i ) = σ I. Level model: β 0i = γ 00 +γ 01 (Endog) i +U 0i β 1i = γ 10 +U 1i β i = γ 0 +U i where U i N(0,T). C.J. Anderson (Illinois) Serial Correlation Fall / 97

58 New Models for Riesby Data Linear Mixed Model: HamD it = γ 00 +γ 10 (time) it +γ 0 (time) it +γ 01(Endog) i Model for cov(r i ) AR(1) MA(1) ARMA(1,1) +U 0i +U 1i (time) it +U i (time) it +R it C.J. Anderson (Illinois) Serial Correlation Fall / 97

59 Global Fit Statistics Model LnLike AIC BIC Empty/Null Preliminary HLM No endog*week No endog*week and no endog AR(1) TOEP() ARMA(1,1) Estimated G matrix is not positive definite C.J. Anderson (Illinois) Serial Correlation Fall / 97

60 Cov. Parameter Estimates: AR(1) Standard Z Cov Parm Estimate Error Value Pr Z UN(1,1) UN(,1) UN(,) UN(3,1) UN(3,) UN(3,3) AR(1) Residual <.00 C.J. Anderson (Illinois) Serial Correlation Fall / 97

61 Estimated cov(r i ): AR(1) Row Col1 Col Col3 Col4 Col5 Col Note: (, 1) covariance equals (.15)(1.417) = (3,1) covariance equals (.15) (1.417) =.84. C.J. Anderson (Illinois) Serial Correlation Fall / 97

62 Cov. Parameter Estimates: TOEP() Standard Z Cov Parm Estimate Error Value Pr Z UN(1,1) UN(,1) UN(,) UN(3,1) UN(3,) UN(3,3) TOEP() Residual <.00 C.J. Anderson (Illinois) Serial Correlation Fall / 97

63 Estimated cov(r i ): TOEP(1) week week 1 week week 3 week 4 week 5 week C.J. Anderson (Illinois) Serial Correlation Fall / 97

64 Testing Random Quadratic Term, U i Model LnLike AIC BIC Empty/Null Preliminary HLM No endog*week AR(1) TOEP() Only U 0i and U 1i (no U i ) AR(1) TOEP() ARMA(1,1) Estimated G matrix is not positive definite C.J. Anderson (Illinois) Serial Correlation Fall / 97

65 Dropping Random Quadratic, U i Hypothesis Test: H o : τ = τ 1 = τ 0 = 0 versus H a : Not H o. Test statistic = ( ) = 3.4. p-value equals a mixture of χ 3 and χ : p-value = 1 ( ) =.58 Retain H o ; drop quadratic random effect. C.J. Anderson (Illinois) Serial Correlation Fall / 97

66 Next Steps? Remove fixed effect for the quadratic term: H o : γ 0 = 0 vs H a : γ 0 0. t = 0.55, Satter. df = 87. p = Global fits Model LnLike AIC BIC Empty/Null Preliminary HLM No endog*week AR(1) AR(1) w/o U i AR(1) w/o quadratic C.J. Anderson (Illinois) Serial Correlation Fall / 97

67 Model Refinement (continued) Random linear trend with AR(1): Covariance Parameter Estimates Standard Z Cov Parm Estimate Error Value Pr Z UN(1,1) UN(,1) UN(,) AR(1) Residual <.01 Do we need a random intercept? C.J. Anderson (Illinois) Serial Correlation Fall / 97

68 Dropping Random Intercept Global fits # Model LnLike AIC BIC Param Empty/Null Preliminary HLM No endog*week AR(1) AR(1) w/o U i AR(1) w/o quadratic U 1i & AR(1) w/o U 0i C.J. Anderson (Illinois) Serial Correlation Fall / 97

69 Dropping Random Intercept Hypothesis: H o : τ0 = τ 01 = 0 versus H a : Not H o Difference between lnlike =.3 p-value is mixture of χ and χ 1, p value = 1 ( ) =.34 Retain H o ; don t need a random intercept. But we keep fixed effects model for the intercept β 0i = γ 00 +γ 01 (Endog) i C.J. Anderson (Illinois) Serial Correlation Fall / 97

70 Parameter Estimates Covariance Parameter Estimates Standard Z Cov Parm Estimate Error Value Pr Z UN(1,1) AR(1) <.0001 Residual <.0001 Solution for Fixed Effects Std. t Effect Estimate Error DF Value Pr t Intercept <.0001 time <.0001 Endog= Endog= C.J. Anderson (Illinois) Serial Correlation Fall / 97

71 Final Model Subjective decision. I like simple ones, so let s look at HamD it = γ 00 +γ 10 (week) it +γ 01 (endog) i +U 1i (week) it +ρr i,(t 1) +ǫ it for (week) it = 0,...,5. Fixed effect structure (overall regression): ĤamD it = (week) it 1.81(endog) i Covariance for Y i? C.J. Anderson (Illinois) Serial Correlation Fall / 97

72 Estimated Covariance Matrix for Y i : Σy i = Z i TZ i +σ ǫω where T = τ 1 and Z i = (0,1,,3,4,5), so Z i TZ i = τ , and C.J. Anderson (Illinois) Serial Correlation Fall / 97

73 Covariance Matrix for Y i Part due to autoregressive model, 1 ρ ρ ρ 3 ρ 4 ρ 5 ρ 1 ρ ρ ρ 3 ρ 4 σǫ Ω = ρ σ ǫ ρ 1 ρ ρ ρ 3 ρ 3 ρ ρ 1 ρ ρ ρ 4 ρ 3 ρ ρ 1 ρ ρ 5 ρ 4 ρ 3 ρ ρ 1 C.J. Anderson (Illinois) Serial Correlation Fall / 97

74 Covariance Matrix for Y i Σy i = Σy i = σ ρσǫ ρ σǫ ρσǫ 3 ρ 4 σǫ ρσǫ 5 ρσǫ τ1 + σy i τ1 + ρσ ǫ 3τ1 + ρ σǫ 4τ1 + ρ3 σǫ 5τ1 + ρ6 σǫ ρ σǫ τ1 + ρσ ǫ 4τ1 + σ ǫ 6τ1 + ρσ ǫ 8τ1 + ρ σǫ 10τ1 + ρ3 σǫ ρ 3 σǫ 3τ1 + ρ σǫ 6τ1 + ρσ ǫ 9τ1 + σ ǫ 1τ1 + ρσ ǫ 15τ1 + ρ σǫ ρ 4 σǫ 4τ1 + ρ3 σǫ 8τ1 + ρ σǫ 1τ1 + ρσ ǫ 16τ1 + σ ǫ 0τ1 + ρσ ǫ ρ 5 σǫ 5τ1 + ρ4 σǫ 10τ1 + ρ3 σǫ 15τ1 + ρ σǫ 0τ1 + ρσ ǫ 5τ1 + σ ǫ C.J. Anderson (Illinois) Serial Correlation Fall / 97

75 Correlation Matrix for Y i ĉorry = C.J. Anderson (Illinois) Serial Correlation Fall / 97

76 Correlation Matrix for Y i Observed = dots, Estimated = lines C.J. Anderson (Illinois) Serial Correlation Fall / 97

77 Quadratic Trend or AR(1)? Observed = dots, Estimated = lines C.J. Anderson (Illinois) Serial Correlation Fall / 97

78 TOEP(6) Observed = dots, Estimated = lines C.J. Anderson (Illinois) Serial Correlation Fall / 97

79 Quadratic Trend or AR(1)? Observed = dots, Estimated = lines C.J. Anderson (Illinois) Serial Correlation Fall / 97

80 Quadratic Trend or TOEP(6) Observed = dots, Estimated = lines C.J. Anderson (Illinois) Serial Correlation Fall / 97

81 Let the data decide! C.J. Anderson (Illinois) Serial Correlation Fall / 97

82 Random Effects vs Serial Correlation Snijders & Bosker: Riesby data: found a reasonable model with complex random effects structure and one with simple random effects with more complex serial correlation structure. Does theory suggest one or the other? There is a wide range of possible combinations of random effects and serial covariance structures... C.J. Anderson (Illinois) Serial Correlation Fall / 97

83 Possible Covariance Structures for Y i Some examples that we ll look at Only random effects (i.e., U i ) We ve looked at these most of the semester. Only serial correlation (i.e., R i ). Both random and serial. Most of this is/was from Hedeker web-site but translated into our notation (any mistakes are mine). C.J. Anderson (Illinois) Serial Correlation Fall / 97

84 Only Random Effects Random-intercepts model: Z i = (1,1,...,1) T = τ 0 R i = σ I i Gives τ0 +σ τ0... τ0 Σy i = Z i TZ τ i +σ 0 I i = τ0 +σ... τ τ0 τ0... τ0 +σ C.J. Anderson (Illinois) Serial Correlation Fall / 97

85 Random Intercept & Random Slope Random linear trend: Z i = ( ) ( τ T = 0 τ 01 τ 01 τ1 ) R i = σ I i Gives Σy i = Z i TZ i +σ I i ( τ 0 +σ τ0 +τ 01 τ0 +τ 01 τ0 +τ 01 τ0 +τ 01 +τ1 +σ τ0 +3τ 01 +τ1 τ0 +τ 01 τ0 +3τ 01 +τ1 τ0 +4τ 01 +4τ1 +σ ) C.J. Anderson (Illinois) Serial Correlation Fall / 97

86 Random Intercept & Random Slopes Random quadratic trend: R i = σ I i Z i = 0 1 T = τ0 τ 01 τ 0 τ 01 τ1 τ 1 τ 0 τ 1 τ Gives Σy i ( τ0 + σ τ0 + τ 01 + τ 0 τ0 + τ τ 0 τ0 + τ 01 + τ 0 τ0 + τ 01 + τ1 + +τ 1τ + σ τ0 + 3τ 01 + τ1 + 5τ 0 + 6τ 1 + 4τ τ0 + τ τ 0 τ0 + 3τ 01 + τ1 + 5τ 0 + 6τ 1 + 4τ τ0 + 4τ τ1 + 8τ τ 1 + σ ) C.J. Anderson (Illinois) Serial Correlation Fall / 97

87 Only Model for R i Compound symmetry: CS σ +σ1 σ1 σ 1 Σy i = σ1 σ +σ1 σ1. σ1 σ1 σ +σ1 How else can you get this structure? Random intercept model, i.e., σ 1 = τ 0. C.J. Anderson (Illinois) Serial Correlation Fall / 97

88 Only Model for R i Frist-Order autoregressive: AR(1) 1 ρ ρ Σy i = σ ρ 1 ρ ρ ρ 1. C.J. Anderson (Illinois) Serial Correlation Fall / 97

89 Only Model for R i (continued) Second-Order Toeplitz: TOEP() σ1 σ 0 Σy i = σ σ1 σ. 0 σ σ1 General Toeplitz: Σy i = σ 1 σ σ 3 σ σ 1 σ σ 3 σ σ 1. C.J. Anderson (Illinois) Serial Correlation Fall / 97

90 Only Model for R i (continued) Unstructured: (UN) Σy i = σ 1 σ 1 σ 13 σ 1 σ σ 3 σ 13 σ 3 σ 3 When do we assume this for the covariance matrix of a response variable? Standard multivariate methods (e.g.,manova) Implication: A way to deal with missing data.. C.J. Anderson (Illinois) Serial Correlation Fall / 97

91 Random Intercept with Compound Symmetry Z = (1,1,1) T = τ 0 cov(r) = CS Σy i = Z i TZ i + σ I i +σ11 i 1 i ( ) ( τ0 τ0 τ0 = τ0 τ0 τ0 + τ0 τ0 τ0 ( ) =...Just compound symmetry? σ + σ1 σ1 σ1 σ1 σ + σ1 σ1 σ1 σ1 σ + σ1 σ + σ1 + τ 0 σ1 + τ 0 σ1 + τ 0 σ1 + τ 0 σ + σ1 + τ 0 σ1 + τ 0 σ1 + τ 0 σ1 + τ 0 σ + σ1 + τ 0 ) C.J. Anderson (Illinois) Serial Correlation Fall / 97

92 Random Intercept with... First Order Autoregressive Z = (1,1,1) T = τ 0 cov(r) = AR(1) Σy i = = ( τ0 τ0 τ0 τ0 τ0 τ0 τ0 τ0 τ0 )+σ ( 1 ρ ρ ρ 1 ρ ρ ρ 1 ( σ +τ 0 ρσ +τ 0 ρ σ +τ 0 ρσ +τ0 σ +τ0 ρσ +τ0 ρ σ +τ0 ρσ +τ0 σ +τ0 ) ) Constant variance, constant (but differing) bands, decreasing covariances. C.J. Anderson (Illinois) Serial Correlation Fall / 97

93 Random Intercept with... Toeplitz Errors Z = (1,1,1) T = τ 0 cov(r) = TOEP() Σy i = = ( τ0 τ0 τ0 τ0 τ0 τ0 τ0 τ0 τ0 ) ( + σ1 σ σ3 σ σ1 σ σ3 σ σ1 ( σ 1 +τ 0 σ +τ 0 σ 3 +τ 0 σ +τ 0 σ1 +τ 0 σ +τ 0 σ3 +τ 0 σ +τ 0 σ1 +τ 0 ) ) Constant variance, constant (by differing) bands, decreasing covariances. C.J. Anderson (Illinois) Serial Correlation Fall / 97

94 Random Linear Trend with... Compound symmetry: cov(r i ) = CS ( ) ( Z τ i = T = 0 τ τ 01 τ1 ) Σy i = ( ( + τ0 τ0 + τ 01 τ0 + τ 01 τ0 + τ 01 τ0 + τ 01 + τ1 τ0 + 3τ 01 + τ1 τ0 + τ 01 τ0 + 3τ 01 + τ0 + 3τ τ1 σ + σ1 σ1 σ1 σ1 σ + σ1 σ1 σ1 σ1 σ + σ1 ) ) C.J. Anderson (Illinois) Serial Correlation Fall / 97

95 Random Linear Trend with... Compound symmetry: cov(r i ) = CS ( ) ( Z τ i = T = 0 τ τ 01 τ1 ) ( Σy i = σ + σ1 + τ 0 σ1 + τ 0 + τ 01 σ1 + τ 0 + τ 01 σ1 + τ 0 + τ 01 σ + σ1 + τ 0 + τ 01 + τ1 σ1 + τ 0 + 3τ 01 + τ1 σ1 + τ 0 + τ 01 σ1 + τ 0 + 3τ 01 + σ + σ1 + τ 0 + 3τ τ1 ) Increasing variances, non-constant covariances. C.J. Anderson (Illinois) Serial Correlation Fall / 97

96 Random Linear Trend with... First Order Autoregressive: R i = AR(1) ( ) ( Z τ i = T = 0 τ τ 01 τ1 ) Σ y i = ( τ0 τ0 + τ 01 τ0 + τ 01 τ0 + τ 01 τ0 + τ 01 + τ1 τ0 + 3τ 01 + τ1 τ0 + τ 01 τ0 + 3τ 01 + τ0 + 3τ τ1 +σ ( 1 ρ ρ ρ 1 ρ ρ ρ 1 ) ) C.J. Anderson (Illinois) Serial Correlation Fall / 97

97 Random Linear Trend with... First Order Autoregressive: R i = AR(1) ( ) ( ) Z τ i = T = 0 τ τ 01 τ1 ( Σy i = σ τ0 ρσ (τ0 + τ 01) ρ σ (τ0 + τ 01) ρσ (τ0 + τ 01) σ (τ0 + τ 01 + τ1 ) ρσ (τ0 + 3τ 01 + τ1 ) ρ σ (τ0 + τ 01) ρσ (τ0 + 3τ 01 + ) σ (τ0 + 3τ τ1 ) ) C.J. Anderson (Illinois) Serial Correlation Fall / 97

98 Random Linear Trend with... Toeplitz Errors: R i = TOEP ( ) Z i = 0 1 ( τ T = 0 τ 01 τ 01 τ1 ) Σy i = = ( + τ0 τ0 + τ 01 τ0 + τ 01 τ0 + τ 01 τ0 + τ 01 + τ1 τ0 + 3τ 01 + τ1 τ0 + τ 01 τ0 + 3τ 01 + τ0 + 3τ τ1 ( σ 1 σ σ 3 σ σ1 σ σ3 σ σ1 ) σ1 +τ 0 σ +τ 0 +τ 01 σ3 +τ 0 +τ 01 σ +τ 0 +τ 01 σ1 +τ 0 +τ 01 +τ1 σ +τ 0 +3τ 01 +τ1 σ3 +τ 0 +τ 01 σ +τ 0 +3τ 01 + σ1 +τ 0 +3τ 01 +4τ1 ) C.J. Anderson (Illinois) Serial Correlation Fall / 97

99 How to Decide? Lots of possibilities. Tools for selection: Look at variances and covariances: Constant or non-constant variance? Constant Bands? Decreasing covariances? LR tests for nested models. Information criteria. What you know about the data & processes. Look for a set of good ones. C.J. Anderson (Illinois) Serial Correlation Fall / 97

Longitudinal Data Analysis via Linear Mixed Model

Longitudinal Data Analysis via Linear Mixed Model Longitudinal Data Analysis via Linear Mixed Model Edps/Psych/Stat 587 Carolyn J. Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Fall 2017 Outline Introduction

More information

Random Intercept Models

Random Intercept Models Random Intercept Models Edps/Psych/Soc 589 Carolyn J. Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Spring 2019 Outline A very simple case of a random intercept

More information

Models for Clustered Data

Models for Clustered Data Models for Clustered Data Edps/Psych/Stat 587 Carolyn J Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Fall 2017 Outline Notation NELS88 data Fixed Effects ANOVA

More information

Models for Clustered Data

Models for Clustered Data Models for Clustered Data Edps/Psych/Soc 589 Carolyn J Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Spring 2019 Outline Notation NELS88 data Fixed Effects ANOVA

More information

over Time line for the means). Specifically, & covariances) just a fixed variance instead. PROC MIXED: to 1000 is default) list models with TYPE=VC */

over Time line for the means). Specifically, & covariances) just a fixed variance instead. PROC MIXED: to 1000 is default) list models with TYPE=VC */ CLP 944 Example 4 page 1 Within-Personn Fluctuation in Symptom Severity over Time These data come from a study of weekly fluctuation in psoriasis severity. There was no intervention and no real reason

More information

SAS Syntax and Output for Data Manipulation:

SAS Syntax and Output for Data Manipulation: CLP 944 Example 5 page 1 Practice with Fixed and Random Effects of Time in Modeling Within-Person Change The models for this example come from Hoffman (2015) chapter 5. We will be examining the extent

More information

Describing Within-Person Fluctuation over Time using Alternative Covariance Structures

Describing Within-Person Fluctuation over Time using Alternative Covariance Structures Describing Within-Person Fluctuation over Time using Alternative Covariance Structures Today s Class: The Big Picture ACS models using the R matrix only Introducing the G, Z, and V matrices ACS models

More information

Estimation: Problems & Solutions

Estimation: Problems & Solutions Estimation: Problems & Solutions Edps/Psych/Stat 587 Carolyn J. Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Fall 2017 Outline 1. Introduction: Estimation of

More information

Statistical Inference: The Marginal Model

Statistical Inference: The Marginal Model Statistical Inference: The Marginal Model Edps/Psych/Stat 587 Carolyn J. Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Fall 2017 Outline Inference for fixed

More information

Random Effects. Edps/Psych/Stat 587. Carolyn J. Anderson. Fall Department of Educational Psychology. university of illinois at urbana-champaign

Random Effects. Edps/Psych/Stat 587. Carolyn J. Anderson. Fall Department of Educational Psychology. university of illinois at urbana-champaign Random Effects Edps/Psych/Stat 587 Carolyn J. Anderson Department of Educational Psychology I L L I N O I S university of illinois at urbana-champaign Fall 2012 Outline Introduction Empirical Bayes inference

More information

Estimation: Problems & Solutions

Estimation: Problems & Solutions Estimation: Problems & Solutions Edps/Psych/Soc 587 Carolyn J. Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Spring 2019 Outline 1. Introduction: Estimation

More information

Random Effects. Edps/Psych/Stat 587. Carolyn J. Anderson. Fall Department of Educational Psychology. c Board of Trustees, University of Illinois

Random Effects. Edps/Psych/Stat 587. Carolyn J. Anderson. Fall Department of Educational Psychology. c Board of Trustees, University of Illinois Random Effects Edps/Psych/Stat 587 Carolyn J. Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Fall 2017 Outline Introduction Empirical Bayes inference Henderson

More information

Models for longitudinal data

Models for longitudinal data Faculty of Health Sciences Contents Models for longitudinal data Analysis of repeated measurements, NFA 016 Julie Lyng Forman & Lene Theil Skovgaard Department of Biostatistics, University of Copenhagen

More information

Advantages of Mixed-effects Regression Models (MRM; aka multilevel, hierarchical linear, linear mixed models) 1. MRM explicitly models individual

Advantages of Mixed-effects Regression Models (MRM; aka multilevel, hierarchical linear, linear mixed models) 1. MRM explicitly models individual Advantages of Mixed-effects Regression Models (MRM; aka multilevel, hierarchical linear, linear mixed models) 1. MRM explicitly models individual change across time 2. MRM more flexible in terms of repeated

More information

Introduction to Within-Person Analysis and RM ANOVA

Introduction to Within-Person Analysis and RM ANOVA Introduction to Within-Person Analysis and RM ANOVA Today s Class: From between-person to within-person ANOVAs for longitudinal data Variance model comparisons using 2 LL CLP 944: Lecture 3 1 The Two Sides

More information

Analysis of variance and regression. May 13, 2008

Analysis of variance and regression. May 13, 2008 Analysis of variance and regression May 13, 2008 Repeated measurements over time Presentation of data Traditional ways of analysis Variance component model (the dogs revisited) Random regression Baseline

More information

Repeated Measures Modeling With PROC MIXED E. Barry Moser, Louisiana State University, Baton Rouge, LA

Repeated Measures Modeling With PROC MIXED E. Barry Moser, Louisiana State University, Baton Rouge, LA Paper 188-29 Repeated Measures Modeling With PROC MIXED E. Barry Moser, Louisiana State University, Baton Rouge, LA ABSTRACT PROC MIXED provides a very flexible environment in which to model many types

More information

Subject-specific observed profiles of log(fev1) vs age First 50 subjects in Six Cities Study

Subject-specific observed profiles of log(fev1) vs age First 50 subjects in Six Cities Study Subject-specific observed profiles of log(fev1) vs age First 50 subjects in Six Cities Study 1.4 0.0-6 7 8 9 10 11 12 13 14 15 16 17 18 19 age Model 1: A simple broken stick model with knot at 14 fit with

More information

STAT 5200 Handout #23. Repeated Measures Example (Ch. 16)

STAT 5200 Handout #23. Repeated Measures Example (Ch. 16) Motivating Example: Glucose STAT 500 Handout #3 Repeated Measures Example (Ch. 16) An experiment is conducted to evaluate the effects of three diets on the serum glucose levels of human subjects. Twelve

More information

Repeated Measures Design. Advertising Sales Example

Repeated Measures Design. Advertising Sales Example STAT:5201 Anaylsis/Applied Statistic II Repeated Measures Design Advertising Sales Example A company is interested in comparing the success of two different advertising campaigns. It has 10 test markets,

More information

Review of Unconditional Multilevel Models for Longitudinal Data

Review of Unconditional Multilevel Models for Longitudinal Data Review of Unconditional Multilevel Models for Longitudinal Data Topics: Course (and MLM) overview Concepts in longitudinal multilevel modeling Model comparisons and significance testing Describing within-person

More information

Stat 579: Generalized Linear Models and Extensions

Stat 579: Generalized Linear Models and Extensions Stat 579: Generalized Linear Models and Extensions Linear Mixed Models for Longitudinal Data Yan Lu April, 2018, week 12 1 / 34 Correlated data multivariate observations clustered data repeated measurement

More information

Correlated data. Repeated measurements over time. Typical set-up for repeated measurements. Traditional presentation of data

Correlated data. Repeated measurements over time. Typical set-up for repeated measurements. Traditional presentation of data Faculty of Health Sciences Repeated measurements over time Correlated data NFA, May 22, 2014 Longitudinal measurements Julie Lyng Forman & Lene Theil Skovgaard Department of Biostatistics University of

More information

Inferences about a Mean Vector

Inferences about a Mean Vector Inferences about a Mean Vector Edps/Soc 584, Psych 594 Carolyn J. Anderson Department of Educational Psychology I L L I N O I S university of illinois at urbana-champaign c Board of Trustees, University

More information

Circle the single best answer for each multiple choice question. Your choice should be made clearly.

Circle the single best answer for each multiple choice question. Your choice should be made clearly. TEST #1 STA 4853 March 6, 2017 Name: Please read the following directions. DO NOT TURN THE PAGE UNTIL INSTRUCTED TO DO SO Directions This exam is closed book and closed notes. There are 32 multiple choice

More information

Review of Multilevel Models for Longitudinal Data

Review of Multilevel Models for Longitudinal Data Review of Multilevel Models for Longitudinal Data Topics: Concepts in longitudinal multilevel modeling Describing within-person fluctuation using ACS models Describing within-person change using random

More information

Non-Stationary Time Series and Unit Root Testing

Non-Stationary Time Series and Unit Root Testing Econometrics II Non-Stationary Time Series and Unit Root Testing Morten Nyboe Tabor Course Outline: Non-Stationary Time Series and Unit Root Testing 1 Stationarity and Deviation from Stationarity Trend-Stationarity

More information

Correlated data. Longitudinal data. Typical set-up for repeated measurements. Examples from literature, I. Faculty of Health Sciences

Correlated data. Longitudinal data. Typical set-up for repeated measurements. Examples from literature, I. Faculty of Health Sciences Faculty of Health Sciences Longitudinal data Correlated data Longitudinal measurements Outline Designs Models for the mean Covariance patterns Lene Theil Skovgaard November 27, 2015 Random regression Baseline

More information

Repeated Measures Data

Repeated Measures Data Repeated Measures Data Mixed Models Lecture Notes By Dr. Hanford page 1 Data where subjects are measured repeatedly over time - predetermined intervals (weekly) - uncontrolled variable intervals between

More information

AR, MA and ARMA models

AR, MA and ARMA models AR, MA and AR by Hedibert Lopes P Based on Tsay s Analysis of Financial Time Series (3rd edition) P 1 Stationarity 2 3 4 5 6 7 P 8 9 10 11 Outline P Linear Time Series Analysis and Its Applications For

More information

MIXED MODELS FOR REPEATED (LONGITUDINAL) DATA PART 2 DAVID C. HOWELL 4/1/2010

MIXED MODELS FOR REPEATED (LONGITUDINAL) DATA PART 2 DAVID C. HOWELL 4/1/2010 MIXED MODELS FOR REPEATED (LONGITUDINAL) DATA PART 2 DAVID C. HOWELL 4/1/2010 Part 1 of this document can be found at http://www.uvm.edu/~dhowell/methods/supplements/mixed Models for Repeated Measures1.pdf

More information

Non-Stationary Time Series and Unit Root Testing

Non-Stationary Time Series and Unit Root Testing Econometrics II Non-Stationary Time Series and Unit Root Testing Morten Nyboe Tabor Course Outline: Non-Stationary Time Series and Unit Root Testing 1 Stationarity and Deviation from Stationarity Trend-Stationarity

More information

Week 5 Quantitative Analysis of Financial Markets Characterizing Cycles

Week 5 Quantitative Analysis of Financial Markets Characterizing Cycles Week 5 Quantitative Analysis of Financial Markets Characterizing Cycles Christopher Ting http://www.mysmu.edu/faculty/christophert/ Christopher Ting : christopherting@smu.edu.sg : 6828 0364 : LKCSB 5036

More information

Topic 17 - Single Factor Analysis of Variance. Outline. One-way ANOVA. The Data / Notation. One way ANOVA Cell means model Factor effects model

Topic 17 - Single Factor Analysis of Variance. Outline. One-way ANOVA. The Data / Notation. One way ANOVA Cell means model Factor effects model Topic 17 - Single Factor Analysis of Variance - Fall 2013 One way ANOVA Cell means model Factor effects model Outline Topic 17 2 One-way ANOVA Response variable Y is continuous Explanatory variable is

More information

Unit Root and Cointegration

Unit Root and Cointegration Unit Root and Cointegration Carlos Hurtado Department of Economics University of Illinois at Urbana-Champaign hrtdmrt@illinois.edu Oct 7th, 016 C. Hurtado (UIUC - Economics) Applied Econometrics On the

More information

Analysis of Longitudinal Data: Comparison Between PROC GLM and PROC MIXED. Maribeth Johnson Medical College of Georgia Augusta, GA

Analysis of Longitudinal Data: Comparison Between PROC GLM and PROC MIXED. Maribeth Johnson Medical College of Georgia Augusta, GA Analysis of Longitudinal Data: Comparison Between PROC GLM and PROC MIXED Maribeth Johnson Medical College of Georgia Augusta, GA Overview Introduction to longitudinal data Describe the data for examples

More information

Sample Geometry. Edps/Soc 584, Psych 594. Carolyn J. Anderson

Sample Geometry. Edps/Soc 584, Psych 594. Carolyn J. Anderson Sample Geometry Edps/Soc 584, Psych 594 Carolyn J. Anderson Department of Educational Psychology I L L I N O I S university of illinois at urbana-champaign c Board of Trustees, University of Illinois Spring

More information

Random Coefficient Model (a.k.a. multilevel model) (Adapted from UCLA Statistical Computing Seminars)

Random Coefficient Model (a.k.a. multilevel model) (Adapted from UCLA Statistical Computing Seminars) STAT:5201 Applied Statistic II Random Coefficient Model (a.k.a. multilevel model) (Adapted from UCLA Statistical Computing Seminars) School math achievement scores The data file consists of 7185 students

More information

Describing Change over Time: Adding Linear Trends

Describing Change over Time: Adding Linear Trends Describing Change over Time: Adding Linear Trends Longitudinal Data Analysis Workshop Section 7 University of Georgia: Institute for Interdisciplinary Research in Education and Human Development Section

More information

Modelling the Covariance

Modelling the Covariance Modelling the Covariance Jamie Monogan Washington University in St Louis February 9, 2010 Jamie Monogan (WUStL) Modelling the Covariance February 9, 2010 1 / 13 Objectives By the end of this meeting, participants

More information

Lecture 4. Random Effects in Completely Randomized Design

Lecture 4. Random Effects in Completely Randomized Design Lecture 4. Random Effects in Completely Randomized Design Montgomery: 3.9, 13.1 and 13.7 1 Lecture 4 Page 1 Random Effects vs Fixed Effects Consider factor with numerous possible levels Want to draw inference

More information

Introduction to Random Effects of Time and Model Estimation

Introduction to Random Effects of Time and Model Estimation Introduction to Random Effects of Time and Model Estimation Today s Class: The Big Picture Multilevel model notation Fixed vs. random effects of time Random intercept vs. random slope models How MLM =

More information

Review of CLDP 944: Multilevel Models for Longitudinal Data

Review of CLDP 944: Multilevel Models for Longitudinal Data Review of CLDP 944: Multilevel Models for Longitudinal Data Topics: Review of general MLM concepts and terminology Model comparisons and significance testing Fixed and random effects of time Significance

More information

SAS Code for Data Manipulation: SPSS Code for Data Manipulation: STATA Code for Data Manipulation: Psyc 945 Example 1 page 1

SAS Code for Data Manipulation: SPSS Code for Data Manipulation: STATA Code for Data Manipulation: Psyc 945 Example 1 page 1 Psyc 945 Example page Example : Unconditional Models for Change in Number Match 3 Response Time (complete data, syntax, and output available for SAS, SPSS, and STATA electronically) These data come from

More information

SAS Syntax and Output for Data Manipulation: CLDP 944 Example 3a page 1

SAS Syntax and Output for Data Manipulation: CLDP 944 Example 3a page 1 CLDP 944 Example 3a page 1 From Between-Person to Within-Person Models for Longitudinal Data The models for this example come from Hoffman (2015) chapter 3 example 3a. We will be examining the extent to

More information

22s:152 Applied Linear Regression. Returning to a continuous response variable Y...

22s:152 Applied Linear Regression. Returning to a continuous response variable Y... 22s:152 Applied Linear Regression Generalized Least Squares Returning to a continuous response variable Y... Ordinary Least Squares Estimation The classical models we have fit so far with a continuous

More information

36-402/608 Homework #10 Solutions 4/1

36-402/608 Homework #10 Solutions 4/1 36-402/608 Homework #10 Solutions 4/1 1. Fixing Breakout 17 (60 points) You must use SAS for this problem! Modify the code in wallaby.sas to load the wallaby data and to create a new outcome in the form

More information

22s:152 Applied Linear Regression. In matrix notation, we can write this model: Generalized Least Squares. Y = Xβ + ɛ with ɛ N n (0, Σ)

22s:152 Applied Linear Regression. In matrix notation, we can write this model: Generalized Least Squares. Y = Xβ + ɛ with ɛ N n (0, Σ) 22s:152 Applied Linear Regression Generalized Least Squares Returning to a continuous response variable Y Ordinary Least Squares Estimation The classical models we have fit so far with a continuous response

More information

F9 F10: Autocorrelation

F9 F10: Autocorrelation F9 F10: Autocorrelation Feng Li Department of Statistics, Stockholm University Introduction In the classic regression model we assume cov(u i, u j x i, x k ) = E(u i, u j ) = 0 What if we break the assumption?

More information

Non-Stationary Time Series and Unit Root Testing

Non-Stationary Time Series and Unit Root Testing Econometrics II Non-Stationary Time Series and Unit Root Testing Morten Nyboe Tabor Course Outline: Non-Stationary Time Series and Unit Root Testing 1 Stationarity and Deviation from Stationarity Trend-Stationarity

More information

1. How can you tell if there is serial correlation? 2. AR to model serial correlation. 3. Ignoring serial correlation. 4. GLS. 5. Projects.

1. How can you tell if there is serial correlation? 2. AR to model serial correlation. 3. Ignoring serial correlation. 4. GLS. 5. Projects. 1. How can you tell if there is serial correlation? 2. AR to model serial correlation. 3. Ignoring serial correlation. 4. GLS. 5. Projects. 1) Identifying serial correlation. Plot Y t versus Y t 1. See

More information

Covariance Structure Approach to Within-Cases

Covariance Structure Approach to Within-Cases Covariance Structure Approach to Within-Cases Remember how the data file grapefruit1.data looks: Store sales1 sales2 sales3 1 62.1 61.3 60.8 2 58.2 57.9 55.1 3 51.6 49.2 46.2 4 53.7 51.5 48.3 5 61.4 58.7

More information

MS&E 226: Small Data

MS&E 226: Small Data MS&E 226: Small Data Lecture 15: Examples of hypothesis tests (v5) Ramesh Johari ramesh.johari@stanford.edu 1 / 32 The recipe 2 / 32 The hypothesis testing recipe In this lecture we repeatedly apply the

More information

Econometrics of Panel Data

Econometrics of Panel Data Econometrics of Panel Data Jakub Mućk Meeting # 4 Jakub Mućk Econometrics of Panel Data Meeting # 4 1 / 30 Outline 1 Two-way Error Component Model Fixed effects model Random effects model 2 Non-spherical

More information

POWER ANALYSIS TO DETERMINE THE IMPORTANCE OF COVARIANCE STRUCTURE CHOICE IN MIXED MODEL REPEATED MEASURES ANOVA

POWER ANALYSIS TO DETERMINE THE IMPORTANCE OF COVARIANCE STRUCTURE CHOICE IN MIXED MODEL REPEATED MEASURES ANOVA POWER ANALYSIS TO DETERMINE THE IMPORTANCE OF COVARIANCE STRUCTURE CHOICE IN MIXED MODEL REPEATED MEASURES ANOVA A Thesis Submitted to the Graduate Faculty of the North Dakota State University of Agriculture

More information

MA Advanced Econometrics: Applying Least Squares to Time Series

MA Advanced Econometrics: Applying Least Squares to Time Series MA Advanced Econometrics: Applying Least Squares to Time Series Karl Whelan School of Economics, UCD February 15, 2011 Karl Whelan (UCD) Time Series February 15, 2011 1 / 24 Part I Time Series: Standard

More information

An Introduction to Multilevel Models. PSYC 943 (930): Fundamentals of Multivariate Modeling Lecture 25: December 7, 2012

An Introduction to Multilevel Models. PSYC 943 (930): Fundamentals of Multivariate Modeling Lecture 25: December 7, 2012 An Introduction to Multilevel Models PSYC 943 (930): Fundamentals of Multivariate Modeling Lecture 25: December 7, 2012 Today s Class Concepts in Longitudinal Modeling Between-Person vs. +Within-Person

More information

I L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN

I L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN Introduction Edps/Psych/Stat/ 584 Applied Multivariate Statistics Carolyn J Anderson Department of Educational Psychology I L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN c Board of Trustees,

More information

10. Time series regression and forecasting

10. Time series regression and forecasting 10. Time series regression and forecasting Key feature of this section: Analysis of data on a single entity observed at multiple points in time (time series data) Typical research questions: What is the

More information

Contrasting Marginal and Mixed Effects Models Recall: two approaches to handling dependence in Generalized Linear Models:

Contrasting Marginal and Mixed Effects Models Recall: two approaches to handling dependence in Generalized Linear Models: Contrasting Marginal and Mixed Effects Models Recall: two approaches to handling dependence in Generalized Linear Models: Marginal models: based on the consequences of dependence on estimating model parameters.

More information

Empirical Market Microstructure Analysis (EMMA)

Empirical Market Microstructure Analysis (EMMA) Empirical Market Microstructure Analysis (EMMA) Lecture 3: Statistical Building Blocks and Econometric Basics Prof. Dr. Michael Stein michael.stein@vwl.uni-freiburg.de Albert-Ludwigs-University of Freiburg

More information

Economics 536 Lecture 7. Introduction to Specification Testing in Dynamic Econometric Models

Economics 536 Lecture 7. Introduction to Specification Testing in Dynamic Econometric Models University of Illinois Fall 2016 Department of Economics Roger Koenker Economics 536 Lecture 7 Introduction to Specification Testing in Dynamic Econometric Models In this lecture I want to briefly describe

More information

Modelling using ARMA processes

Modelling using ARMA processes Modelling using ARMA processes Step 1. ARMA model identification; Step 2. ARMA parameter estimation Step 3. ARMA model selection ; Step 4. ARMA model checking; Step 5. forecasting from ARMA models. 33

More information

Mixed-effects Polynomial Regression Models chapter 5

Mixed-effects Polynomial Regression Models chapter 5 Mixed-effects Polynomial Regression Models chapter 5 1 Figure 5.1 Various curvilinear models: (a) decelerating positive slope; (b) accelerating positive slope; (c) decelerating negative slope; (d) accelerating

More information

Univariate Time Series Analysis; ARIMA Models

Univariate Time Series Analysis; ARIMA Models Econometrics 2 Fall 24 Univariate Time Series Analysis; ARIMA Models Heino Bohn Nielsen of4 Outline of the Lecture () Introduction to univariate time series analysis. (2) Stationarity. (3) Characterizing

More information

Topic 25 - One-Way Random Effects Models. Outline. Random Effects vs Fixed Effects. Data for One-way Random Effects Model. One-way Random effects

Topic 25 - One-Way Random Effects Models. Outline. Random Effects vs Fixed Effects. Data for One-way Random Effects Model. One-way Random effects Topic 5 - One-Way Random Effects Models One-way Random effects Outline Model Variance component estimation - Fall 013 Confidence intervals Topic 5 Random Effects vs Fixed Effects Consider factor with numerous

More information

5 Transfer function modelling

5 Transfer function modelling MSc Further Time Series Analysis 5 Transfer function modelling 5.1 The model Consider the construction of a model for a time series (Y t ) whose values are influenced by the earlier values of a series

More information

Example 7b: Generalized Models for Ordinal Longitudinal Data using SAS GLIMMIX, STATA MEOLOGIT, and MPLUS (last proportional odds model only)

Example 7b: Generalized Models for Ordinal Longitudinal Data using SAS GLIMMIX, STATA MEOLOGIT, and MPLUS (last proportional odds model only) CLDP945 Example 7b page 1 Example 7b: Generalized Models for Ordinal Longitudinal Data using SAS GLIMMIX, STATA MEOLOGIT, and MPLUS (last proportional odds model only) This example comes from real data

More information

Topic 4 Unit Roots. Gerald P. Dwyer. February Clemson University

Topic 4 Unit Roots. Gerald P. Dwyer. February Clemson University Topic 4 Unit Roots Gerald P. Dwyer Clemson University February 2016 Outline 1 Unit Roots Introduction Trend and Difference Stationary Autocorrelations of Series That Have Deterministic or Stochastic Trends

More information

Chapter 4: Models for Stationary Time Series

Chapter 4: Models for Stationary Time Series Chapter 4: Models for Stationary Time Series Now we will introduce some useful parametric models for time series that are stationary processes. We begin by defining the General Linear Process. Let {Y t

More information

Product Held at Accelerated Stability Conditions. José G. Ramírez, PhD Amgen Global Quality Engineering 6/6/2013

Product Held at Accelerated Stability Conditions. José G. Ramírez, PhD Amgen Global Quality Engineering 6/6/2013 Modeling Sub-Visible Particle Data Product Held at Accelerated Stability Conditions José G. Ramírez, PhD Amgen Global Quality Engineering 6/6/2013 Outline Sub-Visible Particle (SbVP) Poisson Negative Binomial

More information

Circle a single answer for each multiple choice question. Your choice should be made clearly.

Circle a single answer for each multiple choice question. Your choice should be made clearly. TEST #1 STA 4853 March 4, 215 Name: Please read the following directions. DO NOT TURN THE PAGE UNTIL INSTRUCTED TO DO SO Directions This exam is closed book and closed notes. There are 31 questions. Circle

More information

Heteroskedasticity in Panel Data

Heteroskedasticity in Panel Data Essex Summer School in Social Science Data Analysis Panel Data Analysis for Comparative Research Heteroskedasticity in Panel Data Christopher Adolph Department of Political Science and Center for Statistics

More information

Hierarchical Generalized Linear Models. ERSH 8990 REMS Seminar on HLM Last Lecture!

Hierarchical Generalized Linear Models. ERSH 8990 REMS Seminar on HLM Last Lecture! Hierarchical Generalized Linear Models ERSH 8990 REMS Seminar on HLM Last Lecture! Hierarchical Generalized Linear Models Introduction to generalized models Models for binary outcomes Interpreting parameter

More information

Heteroskedasticity in Panel Data

Heteroskedasticity in Panel Data Essex Summer School in Social Science Data Analysis Panel Data Analysis for Comparative Research Heteroskedasticity in Panel Data Christopher Adolph Department of Political Science and Center for Statistics

More information

Longitudinal Data Analysis

Longitudinal Data Analysis Longitudinal Data Analysis Mike Allerhand This document has been produced for the CCACE short course: Longitudinal Data Analysis. No part of this document may be reproduced, in any form or by any means,

More information

UNIVERSITY OF TORONTO. Faculty of Arts and Science APRIL 2010 EXAMINATIONS STA 303 H1S / STA 1002 HS. Duration - 3 hours. Aids Allowed: Calculator

UNIVERSITY OF TORONTO. Faculty of Arts and Science APRIL 2010 EXAMINATIONS STA 303 H1S / STA 1002 HS. Duration - 3 hours. Aids Allowed: Calculator UNIVERSITY OF TORONTO Faculty of Arts and Science APRIL 2010 EXAMINATIONS STA 303 H1S / STA 1002 HS Duration - 3 hours Aids Allowed: Calculator LAST NAME: FIRST NAME: STUDENT NUMBER: There are 27 pages

More information

Supplemental Materials. In the main text, we recommend graphing physiological values for individual dyad

Supplemental Materials. In the main text, we recommend graphing physiological values for individual dyad 1 Supplemental Materials Graphing Values for Individual Dyad Members over Time In the main text, we recommend graphing physiological values for individual dyad members over time to aid in the decision

More information

Some general observations.

Some general observations. Modeling and analyzing data from computer experiments. Some general observations. 1. For simplicity, I assume that all factors (inputs) x1, x2,, xd are quantitative. 2. Because the code always produces

More information

13.2 Example: W, LM and LR Tests

13.2 Example: W, LM and LR Tests 13.2 Example: W, LM and LR Tests Date file = cons99.txt (same data as before) Each column denotes year, nominal household expenditures ( 10 billion yen), household disposable income ( 10 billion yen) and

More information

Outline. Linear OLS Models vs: Linear Marginal Models Linear Conditional Models. Random Intercepts Random Intercepts & Slopes

Outline. Linear OLS Models vs: Linear Marginal Models Linear Conditional Models. Random Intercepts Random Intercepts & Slopes Lecture 2.1 Basic Linear LDA 1 Outline Linear OLS Models vs: Linear Marginal Models Linear Conditional Models Random Intercepts Random Intercepts & Slopes Cond l & Marginal Connections Empirical Bayes

More information

Econ 427, Spring Problem Set 3 suggested answers (with minor corrections) Ch 6. Problems and Complements:

Econ 427, Spring Problem Set 3 suggested answers (with minor corrections) Ch 6. Problems and Complements: Econ 427, Spring 2010 Problem Set 3 suggested answers (with minor corrections) Ch 6. Problems and Complements: 1. (page 132) In each case, the idea is to write these out in general form (without the lag

More information

Ch 2: Simple Linear Regression

Ch 2: Simple Linear Regression Ch 2: Simple Linear Regression 1. Simple Linear Regression Model A simple regression model with a single regressor x is y = β 0 + β 1 x + ɛ, where we assume that the error ɛ is independent random component

More information

Model Selection for Geostatistical Models

Model Selection for Geostatistical Models Model Selection for Geostatistical Models Richard A. Davis Colorado State University http://www.stat.colostate.edu/~rdavis/lectures Joint work with: Jennifer A. Hoeting, Colorado State University Andrew

More information

More Linear Algebra. Edps/Soc 584, Psych 594. Carolyn J. Anderson

More Linear Algebra. Edps/Soc 584, Psych 594. Carolyn J. Anderson More Linear Algebra Edps/Soc 584, Psych 594 Carolyn J. Anderson Department of Educational Psychology I L L I N O I S university of illinois at urbana-champaign c Board of Trustees, University of Illinois

More information

Application of Item Response Theory Models for Intensive Longitudinal Data

Application of Item Response Theory Models for Intensive Longitudinal Data Application of Item Response Theory Models for Intensive Longitudinal Data Don Hedeker, Robin Mermelstein, & Brian Flay University of Illinois at Chicago hedeker@uic.edu Models for Intensive Longitudinal

More information

Independence (Null) Baseline Model: Item means and variances, but NO covariances

Independence (Null) Baseline Model: Item means and variances, but NO covariances CFA Example Using Forgiveness of Situations (N = 1103) using SAS MIXED (See Example 4 for corresponding Mplus syntax and output) SAS Code to Read in Mplus Data: * Import data from Mplus, becomes var1-var23

More information

Analyzing the Behavior of Rats by Repeated Measurements

Analyzing the Behavior of Rats by Repeated Measurements Georgia State University ScholarWorks @ Georgia State University Mathematics Theses Department of Mathematics and Statistics 5-3-007 Analyzing the Behavior of Rats by Repeated Measurements Kenita A. Hall

More information

Comment about AR spectral estimation Usually an estimate is produced by computing the AR theoretical spectrum at (ˆφ, ˆσ 2 ). With our Monte Carlo

Comment about AR spectral estimation Usually an estimate is produced by computing the AR theoretical spectrum at (ˆφ, ˆσ 2 ). With our Monte Carlo Comment aout AR spectral estimation Usually an estimate is produced y computing the AR theoretical spectrum at (ˆφ, ˆσ 2 ). With our Monte Carlo simulation approach, for every draw (φ,σ 2 ), we can compute

More information

Lab 11. Multilevel Models. Description of Data

Lab 11. Multilevel Models. Description of Data Lab 11 Multilevel Models Henian Chen, M.D., Ph.D. Description of Data MULTILEVEL.TXT is clustered data for 386 women distributed across 40 groups. ID: 386 women, id from 1 to 386, individual level (level

More information

Homework 5: Answer Key. Plausible Model: E(y) = µt. The expected number of arrests arrests equals a constant times the number who attend the game.

Homework 5: Answer Key. Plausible Model: E(y) = µt. The expected number of arrests arrests equals a constant times the number who attend the game. EdPsych/Psych/Soc 589 C.J. Anderson Homework 5: Answer Key 1. Probelm 3.18 (page 96 of Agresti). (a) Y assume Poisson random variable. Plausible Model: E(y) = µt. The expected number of arrests arrests

More information

I L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN

I L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN Comparisons of Two Means Edps/Soc 584 and Psych 594 Applied Multivariate Statistics Carolyn J. Anderson Department of Educational Psychology I L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN c

More information

CHAPTER 2 SIMPLE LINEAR REGRESSION

CHAPTER 2 SIMPLE LINEAR REGRESSION CHAPTER 2 SIMPLE LINEAR REGRESSION 1 Examples: 1. Amherst, MA, annual mean temperatures, 1836 1997 2. Summer mean temperatures in Mount Airy (NC) and Charleston (SC), 1948 1996 Scatterplots outliers? influential

More information

Non-independence due to Time Correlation (Chapter 14)

Non-independence due to Time Correlation (Chapter 14) Non-independence due to Time Correlation (Chapter 14) When we model the mean structure with ordinary least squares, the mean structure explains the general trends in the data with respect to our dependent

More information

36-309/749 Experimental Design for Behavioral and Social Sciences. Dec 1, 2015 Lecture 11: Mixed Models (HLMs)

36-309/749 Experimental Design for Behavioral and Social Sciences. Dec 1, 2015 Lecture 11: Mixed Models (HLMs) 36-309/749 Experimental Design for Behavioral and Social Sciences Dec 1, 2015 Lecture 11: Mixed Models (HLMs) Independent Errors Assumption An error is the deviation of an individual observed outcome (DV)

More information

Describing Within-Person Change over Time

Describing Within-Person Change over Time Describing Within-Person Change over Time Topics: Multilevel modeling notation and terminology Fixed and random effects of linear time Predicted variances and covariances from random slopes Dependency

More information

Likely causes: The Problem. E u t 0. E u s u p 0

Likely causes: The Problem. E u t 0. E u s u p 0 Autocorrelation This implies that taking the time series regression Y t X t u t but in this case there is some relation between the error terms across observations. E u t 0 E u t E u s u p 0 Thus the error

More information

Econometrics I: Univariate Time Series Econometrics (1)

Econometrics I: Univariate Time Series Econometrics (1) Econometrics I: Dipartimento di Economia Politica e Metodi Quantitativi University of Pavia Overview of the Lecture 1 st EViews Session VI: Some Theoretical Premises 2 Overview of the Lecture 1 st EViews

More information

Regression: Main Ideas Setting: Quantitative outcome with a quantitative explanatory variable. Example, cont.

Regression: Main Ideas Setting: Quantitative outcome with a quantitative explanatory variable. Example, cont. TCELL 9/4/205 36-309/749 Experimental Design for Behavioral and Social Sciences Simple Regression Example Male black wheatear birds carry stones to the nest as a form of sexual display. Soler et al. wanted

More information

Econometrics Honor s Exam Review Session. Spring 2012 Eunice Han

Econometrics Honor s Exam Review Session. Spring 2012 Eunice Han Econometrics Honor s Exam Review Session Spring 2012 Eunice Han Topics 1. OLS The Assumptions Omitted Variable Bias Conditional Mean Independence Hypothesis Testing and Confidence Intervals Homoskedasticity

More information