Outline. Linear OLS Models vs: Linear Marginal Models Linear Conditional Models. Random Intercepts Random Intercepts & Slopes

Transcription

1 Lecture 2.1 Basic Linear LDA 1 Outline Linear OLS Models vs: Linear Marginal Models Linear Conditional Models Random Intercepts Random Intercepts & Slopes Cond l & Marginal Connections Empirical Bayes 2 1

2 Example: Weights of Guinea Pigs Body weights of 48 pigs in 9 successive weeks of follow-up (Table 3.1 DHLZ 2002) The response is measured at n i different times, or under n i different conditions. In the pigs example the time of measurement is referred to as a "withinunits" factor. For the pigs n i (weeks) = 9 Although the pigs example considers a single treatment factor, it is straightforward to extend the situation to one where the groups are formed as the results of a factorial design (for example, if the Pigs were separated into males and female and then allocated to the diet groups) 3 Pigs Data: Spaghetti (Profile) Plot Weight (Kg) sort id time. twoway line weight time, connect(ascending) Questions: Average weight increase per week? Variability in weights between Pigs? Variation in growth patterns? 48 pigs Week 4 2

3 Some Modelling Options 1. Ignore associations (OLS) 1. Y ij = week + ij 2. ij ~ N(0, 2 ) 2. Use a Marginal Model to estimate effects (PA - GEE) 1. Y ij = week + ij 2. ~ N(0, ) 3. = R 2 ; R is a working corr structure (ind,exch,ar,...) 3. Use a Conditional Model to estimate effects (RE) 1. Y ij u i = week + u i + ij 2. ij ~ N(0, 2 ) 3. u i ~ N(0, 2 ) 5. regress weight time Pigs model 1: OLS fit Source SS df MS Number of obs = F( 1, 430) = Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = weight Coef. Std. Err. t P> t [95% Conf. Interval] time _cons OLS results Assumptions? L - Linear I - Independent N - Normal errors E - Equal Variance 6 3

4 Some Modelling Options 1. Ignore associations (OLS) 1. Y ij = week + ij 2. ij ~ N(0, 2 ) 2. Use a Marginal Model to estimate effects (PA - GEE) 1. Y ij = week + ij 2. ~ N(0, ) 3. = R 2 ; R is a working corr structure (ind,exch,ar,...) 3. Use a Conditional Model to estimate effects (RE) 1. Y ij u i = week + u i + ij 2. ij ~ N(0, 2 ) 3. u i ~ N(0, 2 ) 7 Pigs Data: Spaghetti (Profile) Plot Weight (Kg) Recall Corr EDA yesterday: Suppose: 1. Y ij = week + ij 2. ~ N(0, ) 3. = R 2 48 pigs Week 8 4

5 1 Standardized Pigs Data: Scatterplot Matrix 2. graph matrix sweight1-sweight9, half 3 4 A Scatterplot Matrix helps to visualize multiple 2-way relationships 5 6 Note the decreasing corr as distance between weeks (lag) increases acf ACF Pigs Weights: ACF acf Lag lag 5

6 Some Modelling Options 1. Ignore associations (OLS) 1. Y ij = week + ij 2. ij ~ N(0, 2 ) 2. Use a Marginal Model to estimate effects (PA - GEE) 1. Y ij = week + ij 2. ~ N(0, ) 3. = R 2 ; R is a working corr structure (ind,exch,ar,...) 3. Use a Conditional Model to estimate effects (RE) 1. Y ij u i = week + u i + ij 2. ij ~ N(0, 2 ) 3. u i ~ N(0, 2 ) 11 Pigs model 1: PA-IND fit. xtreg weight time, pa i(id) corr(ind) GEE population-averaged model Number of obs = 432 Group variable: Id Number of groups = 48 Link: identity Obs per group: min = 9 Family: Gaussian avg = 9.0 Correlation: independent max = 9 Wald chi2(1) = Scale parameter: Prob > chi2 = Pearson chi2(432): Deviance = Dispersion (Pearson): Dispersion = time _cons Independence correlation model results 12 6

7 . regress weight time Pigs model 1: OLS fit Source SS df MS Number of obs = F( 1, 430) = Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = weight Coef. Std. Err. t P> t [95% Conf. Interval] time _cons OLS results E[ Y i ] = β 0 + β 1 time Corr = ind Marginal Model twoway (line weight time, connect(ascending)) /// (lfit weight time, clwidth(thick)) time weight Fitted values 14 7

8 Marginal Model with a Uniform correlation structure E[Y ij ] 0 1 t j Model for the mean cov(y ij ) ( 2 2 )[ 11' (1 )I] Same as Before where: R = Model for the covariance matrix 15 Pigs Marginal Model Exch. xtreg weight time, pa i(id) corr(exch) Iteration 1: tolerance = 5.585e-15 GEE population-averaged model Number of obs = 432 Group variable: Id Number of groups = 48 Link: identity Obs per group: min = 9 Family: Gaussian avg = 9.0 Correlation: exchangeable max = 9 Wald chi2(1) = Scale parameter: Prob > chi2 = time _cons Population Average, Marginal Model with Exchangeable Correlation structure results 16 8

9 Pigs Marginal Model Exch, GEE. xtgee weight time, i(id) corr(exch) Iteration 1: tolerance = 5.585e-15 GEE population-averaged model Number of obs = 432 Group variable: Id Number of groups = 48 Link: identity Obs per group: min = 9 Family: Gaussian avg = 9.0 Correlation: exchangeable max = 9 Wald chi2(1) = Scale parameter: Prob > chi2 = time _cons GEE fit Marginal Model with Exchangeable Correlation structure results 17 Pigs data model 1 GEE fit. xtgee weight time, i(id) corr(exch). xtcorr Estimated within-id correlation matrix R: c1 c2 c3 c4 c5 c6 c7 c8 c9 r r r r r r r r r GEE fit Marginal Model with Exchangeable Correlation structure results 18 9

10 Pigs Marginal Model AR1, GEE. xtgee weight time, i(id) corr(ar1) t(time) Iteration 1: tolerance = Iteration 2: tolerance = Iteration 3: tolerance = 4.366e-07 GEE population-averaged model Number of obs = 432 Group and time vars: Id time Number of groups = 48 Link: identity Obs per group: min = 9 Family: Gaussian avg = 9.0 Correlation: AR(1) max = 9 Wald chi2(1) = Scale parameter: Prob > chi2 = time _cons GEE fit Marginal Model with Auto Regressive(1) Correlation structure results 19 Pigs Marginal Model AR1, GEE. xtgee weight time, i(id) corr(ar1) t(time). xtcorr Estimated within-id correlation matrix R: c1 c2 c3 c4 c5 c6 c7 c8 c9 r r r r r r r r r GEE fit Marginal Model with AR1 Correlation structure results 20 10

11 Pigs Weights: ACF acf ACF Estimates from: Exch & AR Lag Common types of Corr models. help xtgee correlation description exchangeable exchangeable independent independent unstructured unstructured fixed matname user-specified ar # autoregressive of order # stationary # stationary of order # nonstationary # nonstationary of order # 22 11

12 Pigs Marginal Model Toeplitz, GEE. xtgee weight time, i(id) corr(stationary8) t(time) Iteration 4: tolerance = Iteration 5: tolerance = 3.968e-06 Iteration 6: tolerance = 7.862e-07 GEE population-averaged model Number of obs = 432 Group and time vars: Id time Number of groups = 48 Link: identity Obs per group: min = 9 Family: Gaussian avg = 9.0 Correlation: stationary(8) max = 9 Wald chi2(1) = Scale parameter: Prob > chi2 = time _cons GEE fit Marginal Model with Toeplitz Correlation structure results 23 Pigs Marginal Model Toeplitz, GEE. xtgee weight time, i(id) corr(stationary8) t(time). xtcorr Estimated within-id correlation matrix R: c1 c2 c3 c4 c5 c6 c7 c8 c9 r r r r r r r r r GEE fit Marginal Model with Toeplitz (stationary) Correlation structure results 24 12

13 Robust Standard Errors. xtgee weight time, i(id) corr(exch) robust Semi-robust time _cons xtgee weight time, i(id) corr(ar1) t(time) robust Semi-robust time _cons xtgee weight time, i(id) corr(stationary8) t(time) robust Semi-robust time _cons QIC in model selection. ssc install qic. help qic Description qic calculates the QIC and QIC_u criteria for model selection in GEE, which is an extension of the widely used AIC criterion in ordinary regression (Pan 2001). It allows for specification of all 7 distributions - gaussian, inverse Gaussian, Bernoulli/binomial, Poisson, negative binomial and gamma, all link functions and working correlation structures and all se/robust options, except for the vce option, avaiable in Stata 9.0. It also calculates the trace of the matrix O^{-1}V, where O is the variance estimate under the independent correlation structure and V is the variance estimate under the specified working correlation structure in GEE. When trace is close to the number of parametr p, the QIC_u is a good approximation to QIC

14 QIC in model selection 27 QIC in model selection QIC(R) a function of the assumed working correlation matrix R, so QIC(R) can be potentially useful in examining which correlation structures are better than others QIC u a measure that can be potentially useful in examining which subsets of covariates are better than others when Trace p. Common recommendation is to use QIC u (Independence) These are based on measures of deviance (distance from the data to the model based on the likelihood) so smaller is better Sometimes useful but rely on your EDA skills also 28 14

15 QIC in model selection qic weight time, i(id) corr(exch) robust qic weight time, i(id) corr(ar1) t(time) robust qic weight time, i(id) corr(stationary8) t(time) robust QIC and QIC_u Corr = exch Family = gau Link = iden p = 2 Trace = QIC = QIC_u = QIC and QIC_u Corr = ar1 Family = gau Link = iden p = 2 Trace = QIC = QIC_u = QIC and QIC_u Corr = stationary8 Family = gau Link = iden p = 2 Trace = QIC = QIC_u = Exch corr favored by QIC criterion? Some Modelling Options 1. Ignore associations (OLS) 1. Y ij = week + ij 2. ij ~ N(0, 2 ) 2. Use a Marginal Model to estimate effects (PA - GEE) 1. Y ij = week + ij 2. ~ N(0, ) 3. = R 2 ; R is a working corr structure (ind,exch,ar,...) 3. Use a Conditional Model to estimate effects (RE) 1. Y ij u i = week + u i + ij 2. ij ~ N(0, 2 ) 3. u i ~ N(0, 2 ) 30 15

16 Linear model with random intercept Note Shorthand, Y ij = Y ij u i Y ij U i 0 1 t j ij U i ~ N(0, 2 ) ij ~ N(0, 2 ) Cluster specific intercept U i Variance between Variance within Intraclass correlation coefficient 31 Pigs Conditional Model Ran Int.. xtreg weight time, re i(id) mle var Random-effects ML regression Number of obs = 432 Group variable (i): Id Number of groups = 48 Random effects u_i ~ Gaussian Obs per group: min = 9 avg = 9.0 max = 9 LR chi2(1) = Log likelihood = Prob > chi2 = time _cons /sigma_u /sigma_e rho Linear model with a random intercept - conditional model 32 16

17 Pigs Conditional Model Ran Int.. xtmixed weight time id:, mle Mixed-effects ML regression Number of obs = 432 Group variable: id Number of groups = 48 Obs per group: min = 9 avg = 9.0 max = 9 Wald chi2(1) = Log likelihood = Prob > chi2 = time _cons Random-effects Parameters Estimate Std. Err. [95% Conf. Interval] id: Identity sd(_cons) sd(residual) LR test vs. linear regression: chibar2(01) = Prob >= chibar2 = Same random intercept model - xtmixed 33 Pigs Conditional Model Ran Int.. xtmixed weight time id:, mle var Mixed-effects ML regression Number of obs = 432 Group variable: id Number of groups = 48 Obs per group: min = 9 avg = 9.0 max = 9 Wald chi2(1) = Log likelihood = Prob > chi2 = time _cons Random-effects Parameters Estimate Std. Err. [95% Conf. Interval] id: Identity var(_cons) var(residual) LR test vs. linear regression: chibar2(01) = Prob >= chibar2 = Same random intercept model - xtmixed, var option 34 17

18 Pigs Conditional Model Ran Int.. gllamm weight time, i(id) nip(12) adapt gllamm model log likelihood = time _cons Variance at level ( ) Variances and covariances of random effects ***level 2 (id) var(1): ( ) Linear model with a random intercept - conditional model Random Effects Model E[ Y i U i ] = β 0 + β 1 time + U i E[ Y i ] = β 0 + β 1 time 48 Pigs time weight Linear prediction Xb + u[id] 36 18

19 Random Effects Model E[ Y i U i ] = β 0 + β 1 time + U i E[ Y i ] = β 0 + β 1 time Goat #48: Data, Pred48 Goat #3: Data, Pred3 Marginal Predictor 2 Pigs time weight Linear prediction Xb + u[id] 37 Equivalence of RE & Marginal in the Linear, Exch Corr case (with no missing data) 38 19

20 Pigs Conditional Model Ran Int.. xtreg weight time, re i(id) mle var Random-effects ML regression Number of obs = 432 Group variable (i): Id Number of groups = 48 Random effects u_i ~ Gaussian Obs per group: min = 9 avg = 9.0 max = 9 LR chi2(1) = Log likelihood = Prob > chi2 = time _cons /sigma_u /sigma_e rho Linear model with a random intercept - conditional model 39 Pigs Marginal Model Exch, GEE. xtgee weight time, i(id) corr(exch) Iteration 1: tolerance = 5.585e-15 GEE population-averaged model Number of obs = 432 Group variable: Id Number of groups = 48 Link: identity Obs per group: min = 9 Family: Gaussian avg = 9.0 Correlation: exchangeable max = 9 Wald chi2(1) = Scale parameter: Prob > chi2 = time _cons GEE fit Marginal Model with Exchangeable Correlation structure results 40 20

21 Pigs Marginal Model Exch, GEE. xtgee weight time, i(id) corr(exch). xtcorr Estimated within-id correlation matrix R: c1 c2 c3 c4 c5 c6 c7 c8 c9 r r r r r r r r r GEE fit Marginal Model with Exchangeable Correlation structure results Linear Regression : Random Effects Model Mean Structure = Marginal Model Mean Structure? E[ Y i ] =? E[ Y i ] = E{ E[ Y i U i ] } = How About Correlation Structure? (For Exchangeable) 21

22 Other mean, E(Y ij ), models 43 Pigs Ran Int, linear trend. xtreg weight time, re i(id) mle var Random-effects ML regression Number of obs = 432 Group variable (i): Id Number of groups = 48 Random effects u_i ~ Gaussian Obs per group: min = 9 avg = 9.0 max = 9 LR chi2(1) = Log likelihood = Prob > chi2 = time _cons /sigma_u /sigma_e rho Linear model with a random intercept - conditional model 44 22

23 Pigs Ran Int, quadratic trend. gen timesq = time*time. xtreg weight time timesq, re i(id) mle Random-effects ML regression Number of obs = 432 Group variable (i): Id Number of groups = 48 Random effects u_i ~ Gaussian Obs per group: min = 9 avg = 9.0 max = 9 LR chi2(2) = Log likelihood = Prob > chi2 = time timesq _cons /sigma_u /sigma_e rho Random Intercept results Pigs Marg, quadratic trend. xtgee weight time timesq, i(id) corr(exch) GEE population-averaged model Number of obs = 432 Group variable: Id Number of groups = 48 Link: identity Obs per group: min = 9 Family: Gaussian avg = 9.0 Correlation: exchangeable max = 9 Wald chi2(2) = Scale parameter: Prob > chi2 = time timesq _cons Exchangeable Correlation structure results. xtcorr yields Corr estimate =

24 Equivalent Models E[ Y i U i ] = β 0 + β 1 time + β 2 time 2 + U i E[ Y i ] = Note, these are still called linear models: know why? time weight Xb + u[id] Linear prediction 47 Other Variance models 48 24

25 Pigs Data: Spaghetti (Profile) Plot Weight (Kg) sort id time. twoway line weight time, connect(ascending) Questions: Average weight increase per week? Variability in weights between Pigs? Variation in growth patterns? 48 Pigs Week 49 Pigs Conditional Model Ran Slopes.. xtmixed weight time id: time, mle cov(unstruct) Log likelihood = time _cons Random-effects Parameters Estimate Std. Err. [95% Conf. Interval] Id: Unstructured sd(time) sd(_cons) corr(time,_cons) sd(residual) LR test vs. linear regression: chi2(3) = Prob > chi2 = Note: LR test is conservative and provided only for reference

26 Random Slopes E[ Y i U i ] = β 0 + β 1 time + U 0i + U 1i time = (β 0 + U 0i ) + (β 1 +U 1i )time = β 0i + β 1i time E[ Y i ] = time weight Linear predictor, fixed portion Fitted values: xb + Zu 51 Written Results : RE Linear Regression - Pigs Use a Conditional Model to estimate effects 1. Y ij u i = week + u 0i + u 1i week + ij 2. ij ~ N(0, 2 ) 3. u i =) ij & u i independent 1. Y ij u i = (week) + u 0i + u 1i week + ij 2. ij ~ N(0, ) 3. u 0i ~ N(0, ), u 1i ~ N(0, ), corr(u 0i,u 1i )= ij & u i independent 52 26

27 Extensions : Linear Regression - Pigs Usual Linear Regression (OLS) 1. Y ij = week + 2 week 2 + ij 2. ij ~ N(0, 2 ) Use a Marginal Model to estimate effects 1. Y ij = week + 2 week 2 + ij 2. i ~ N(0, ) 3. = R 2 ; R is a working corr structure (ind,exch,ar,...) Use a Conditional Model to estimate effects 1. Y ij u i = week + 2 week 2 + u i + ij 2. ij ~ N(0, 2 ) 3. u i ~ N(0, 2 ) ij & u i independent 53 Extensions: Linear Regression - general Usual Linear Regression (OLS) 1. Y i = X + i 2. i ~ N(0,I 2 ) Use a Marginal Model to estimate effects 1. Y i = X + i 2. i ~ N(0, ) 3. = R 2 ; R is a working corr structure (ind,exch,ar,...) Use a Conditional Model to estimate effects 1. Y i u i = X + Zu i + i 2. i ~ N(0, ) 3. u i ~ N(0,G) i & u i independent 54 27