Modeling the Covariance

Transcription

1 Modeling the Covariance Jamie Monogan University of Georgia February 3, 2016 Jamie Monogan (UGA) Modeling the Covariance February 3, / 16

2 Objectives By the end of this meeting, participants should be able to: Define the structure of common covariance pattern models Make a choice of covariance pattern model for real data Evaluate model specification by examining residuals graphically Test for outlier observations using Mahalanobis distance Jamie Monogan (UGA) Modeling the Covariance February 3, / 16

3 Longitudinal Variance and Covariance We expect between-individual variance to be greater than within-individual variance This is because repeated observations ought to be similar The more similar observations are, the higher their covariance The more similar observations are, the lower their variance Consider: Var(Y i2 Y i1 ) = σ σ2 2 2ρ 12σ 1 σ 2 If ρ 12 is positive (as it usually is), a bigger correlation means a smaller variance If we do not account for correlation among repeated observations: Our estimates will be inefficient Our standard errors will be biased: With positive correlation, our standard errors will be too large With negative correlation (rare in panels), our standard errors will be too small Jamie Monogan (UGA) Modeling the Covariance February 3, / 16

4 Longitudinal Variance and Covariance In regression terms, we account for this by defining Ω within β GLS = [X Ω 1 X] 1 X Ω 1 y (Remember FGLS uses Ω) Typically: Σ O O O Σ O Ω = O O Σ Where Y i = (Y i1, Y i2,, Y in ), Cov(Y i ) = Σ, and: O = Note: for n waves, both Σ and O are n n matrices So what does Cov(Y i ) = Σ look like? How do we estimate Σ? Today we discuss those models Jamie Monogan (UGA) Modeling the Covariance February 3, / 16

5 Unstructured Covariance The broadest model imposes no constraints on Σ other than symmetry However, the number of parameters increases rapidly with the number of waves So this will not miss anything in the covariance structure, but it may not estimate in some circumstances σ 2 1 σ 12 σ 1n σ 21 σ2 2 σ 2n Cov(Y i ) = σ n1 σ n2 σn 2 Jamie Monogan (UGA) Modeling the Covariance February 3, / 16

6 Compound Symmetry This structure is equivalent to an econometric random effects model Suppose the error term as two components, such as b i and ɛ ij That induces this kind of covariance structure In other words, this is how to deal with unit effects using GLS Parsimonious with two parameters, and sometimes the random effect structure sounds right Often, though, we would rather assume that correlation decays with time σ 2 σ 2 ρ σ 2 ρ σ 2 ρ σ 2 ρ σ 2 σ 2 ρ σ 2 ρ Cov(Y i ) = σ 2 ρ σ 2 ρ σ 2 ρ σ 2 Jamie Monogan (UGA) Modeling the Covariance February 3, / 16

7 Toeplitz Fairly flexible, yet cuts parameters considerably relative to unstructured matrix Assumption: Any two observations separated by the same number of time waves will correlate at the same rate Cannot be directly estimated in R For n waves, AR(n 1) is equivalent Structure: σ 2 σ 1 σ 2 σ n 1 σ 1 σ 2 σ 1 σ n 2 Cov(Y i ) = σ n 1 σ n 2 σ n 3 σ 2 Jamie Monogan (UGA) Modeling the Covariance February 3, / 16

8 First-Order Autoregressive Shorthand: AR(1) A special case of Toeplitz that reduces to two parameters Assumption: ɛ ij = ρɛ ij 1 + w ij, where w ij is iid normal This means that the correlation will decay in a specific pattern σ 2 σ 2 ρ σ 2 ρ 2 σ 2 ρ n 1 σ 2 ρ σ 2 σ 2 ρ σ 2 ρ n 2 Cov(Y i ) = σ 2 ρ n 1 σ 2 ρ n 2 σ 2 ρ n 3 σ 2 Jamie Monogan (UGA) Modeling the Covariance February 3, / 16

9 Banded Simplification that imposes zero covariance beyond a certain order Strong assumption about the decay of correlation if it drops to 0 abruptly Example, band size of 2: σ 2 σ 2 ρ σ 2 ρ 1 σ 2 σ 2 ρ 1 0 Cov(Y i ) = σ 2 Jamie Monogan (UGA) Modeling the Covariance February 3, / 16

10 Exponential Also called continuous autoregressive A better choice for irregularly-spaced measurement intervals The logic is similar to AR(1), but sets correlation based on ρ and the temporal separation of two observations, which may be distinct from how many waves separate them σ 2 σ 2 ρ t 1 t 2 σ 2 ρ t 1 t 3 σ 2 ρ t 1 t n σ 2 ρ t 2 t 1 σ 2 σ 2 ρ t 2 t 3 σ 2 ρ t 2 t n Cov(Y i ) = σ 2 ρ tn t 1 σ 2 ρ tn t 2 σ 2 ρ tn t 3 σ 2 Jamie Monogan (UGA) Modeling the Covariance February 3, / 16

11 Recall the Relationship Between Correlation and Covariance If we are willing to assume homoskedasticity, we can think of the covariance matrix Σ as the product of the error variance of regression and a correlation matrix This is handy because software often returns the two components separately, so we have to build Σ out of them ourselves In other words, Cov(Y i ) = σ 2 Cor(Y i ) Consider the first-order autoregressive structure Try multiplying the following correlation matrix by σ 2 : 1 ρ ρ 2 ρ n 1 ρ 1 ρ ρ n 2 Cor(Y i ) = ρ n 1 ρ n 2 ρ n 3 1 Jamie Monogan (UGA) Modeling the Covariance February 3, / 16

12 Estimation in R: corclasses corsymm general correlation matrix, with no additional structure corcompsymm compound symmetry structure (econ random effects) corar1 autoregressive process of order 1 corarma autoregressive moving average process (kluge Toeplitz) corcar1 continuous autoregressive process (exponential) Jamie Monogan (UGA) Modeling the Covariance February 3, / 16

13 Choosing a Covariance Pattern Model Covariance of repeated observations can depend on the specification of the mean model (Case for response profiles) Likelihood Ratio Test: H 0 : r constraints are true G 2 = 2(ˆl f ˆl s ) χ 2 (r) for r restrictions For covariance testing use REML Not ideal if imposing many zeros AIC 2(ˆl) + 2(c) For ˆl maximized REML log-likelihood and c the number of covariance parameters The lowest value is the best fit, contingent on a parsimony penalty BIC (Schwartz s Criterion) 2(ˆl) + log(n )(c) For ˆl maximized REML log-likelihood, N = N p (for p length of β), and c the number of covariance parameters The lowest value is the best fit, contingent on a big parsimony penalty FLW argue this is too slanted towards parsimony Jamie Monogan (UGA) Modeling the Covariance February 3, / 16

14 Computing Transformed Residuals As with other models, we want to conduct post-estimation diagnostics We do not eliminate correlation in the data, but hopefully we account for it If we conduct diagnostics with raw residuals, the results will generally look bad because those features are still in the data Instead, we created transformed residuals that incorporate how we model covariance If these look bad, then we did not sufficiently model the issues Computing each individual s vector of transformed residuals: Cholesky decomposition: Σ i = L i L i Uncorrelated, unit variance residuals: r i = L i 1 (Y i X i ˆβ) Jamie Monogan (UGA) Modeling the Covariance February 3, / 16

15 Residual Diagnostics Is the functional form right? Is there unmodeled heteroscedasticity? Plot transformed residuals against time, transformed predicted values, or input variables like you otherwise would Transformed predicted values: µ = L i 1 X i β Did you model the autocorrelation sufficiently? Plot a semivariogram of the transformed residuals Remember: high variance means low covariance Insufficiently modeled autocorrelation generally implies low semivariance for close times, and high semivariance at distant times Sufficiently-filtered autocorrelation will show no particular pattern in the semivariogram Do you believe the errors are normally distributed? Histograms, density plots, and quantile-quantile plots of transformed residuals Who are outlying individuals? Mahalanobis distance: d i = r i T r i χ 2 (s) where s is the length of r i Jamie Monogan (UGA) Modeling the Covariance February 3, / 16

16 For Next Time Read FLW Chapters 8-9 Read Section 81 from Political Analysis Using R Download the five-wave American National Election Study panel from: Model Bush s thermometer ratings (bush) using parametric curves Your two input variables are partisanship (pid3) and time in months (months) For pid3: 1=Democrat, 2=Independent, and 3=Republican So you are analyzing three groups It this is too much, you may throw away Independents and then use democrat as your two-party predictor variable Try at least three parametric curves and report the AIC for each Use FIML at this stage Switch to REML for the rest of the exercise Try an unstructured, AR(1), and Toeplitz error structure Report the AIC for each Which fits best? Test the hypothesis that trajectories differ by partisanship for your best-fitting model Recommended: Draw a picture of the trajectory for each group With your final selected model, plot the transformed residuals against the transformed fitted values What do you conclude from this? Plot the variogram from this model What do you conclude? Compute Mahalanobis distance for each individual How many individuals are outliers? Jamie Monogan (UGA) Modeling the Covariance February 3, / 16