Introduction. Dottorato XX ciclo febbraio Outline. A hierarchical structure. Hierarchical structures: type 2. Hierarchical structures: type 1

Transcription

1 Outline Introduction to multilevel analysis. Introduction >. > Leonardo Grilli 3. Estimation > 4. Software & Books > Web: Department of Statistics G. Parenti - University of Florence L. Grilli - Introduction to multilevel analysis - April 6 Introduction A hierarchical structure Multilevel structures Vocabulary Aggregated vs. disaggregated analysis Random effects ANOVA Design effect Relationships within and between clusters district level 4 school level 3 school class class level class 3 class 4 s s s3 s4 s5 s6 s7 s8 s9 s0 s s level - students L. Grilli - Introduction to multilevel analysis - April 6 4 Hierarchical structures: type Hierarchical structures: type Units within clusters pupil, class, school patient, doctor, hospital worker, firm individual, family, region interviewed, interviewer Often the sampling design reflects the hierarchical structure (multi-stage sampling), but this is not necessary!! Multiple responses (level : responses, level : individuals) Multivariate data Longitudinal data (panel, repeated measurements) Remark: a hierarchical structure may combine multiple responses and true clusters: e.g. questionnaire on the students -> item, student, school L. Grilli - Introduction to multilevel analysis - April 6 5 L. Grilli - Introduction to multilevel analysis - April 6 6 Dottorato XX ciclo febbraio 6

2 Beyond standard hierarchical structures Which is the relevant structure? cross-classified structures, e.g. pupil classified by quarter and school Usually it is assumed that each level k unit belongs to one and only one level k+ unit. Otherwise -> multiple membership models e.g. pupils changing their school during the observation interval L. Grilli - Introduction to multilevel analysis - April 6 7 Usually the phenomenon under study can be modelled through several alternative structures: e.g. Pupil, class Pupil, class, school Pupil, school Pupil, teacher Pupil, (school by quarter) The structure to be used in the analysis depends on the aims of the research Even if a complex structure may appear more realistic, for most research purposes a simple structure with or 3 levels is enough L. Grilli - Introduction to multilevel analysis - April 6 8 Vocabulary Vocabulary Populations a) Hierarchical b) Nested c) Cross-classified d) Multilevel Ian Plewis (997) on Multilevel Modelling Newsletter, Vol. 9 No. a) and b) are interchangeable d) incorporates each of a), b) and c) L. Grilli - Introduction to multilevel analysis - April 6 9 Models Multilevel (MLM) Hierarchical linear (HLM) Mixed linear Random coefficients Random effects Subect/unit specific Random intercept Variance components L. Grilli - Introduction to multilevel analysis - April 6 0 Two types of macro (cluster) variables Relationships between levels GLOBAL: feature of the macro unit (cluster) with no corresponding micro level measure Example: public/private school, number of teachers CONTEXTUAL: macro indicator obtained through aggregation of micro level measures (summary of the features of the micro units) Example: average class size, proportion of females, average grade Z Z X Z X macro: level, e.g. school macro-micro relationship micro: level, e.g. pupil adusted macro-micro relationship cross-level interaction X Z micro-macro relationship L. Grilli - Introduction to multilevel analysis - April 6 L. Grilli - Introduction to multilevel analysis - April 6 Dottorato XX ciclo febbraio 6

3 The unit of analysis dilemma Problems with aggregated analysis One can choose to analyse the data at Individual level (e.g. pupil) -> disaggregated analysis Cluster level (e.g. school) -> aggregated analysis (the aggregated analysis entails reducing the dataset by taking the cluster means) Both choices lead to several problems Shift of meaning: the macro variables obtained through aggregation refer to the cluster (not the individual) -> they cannot be used to investigate micro level relationships Ecological fallacy (aggregation bias): macro level relationships micro level relationships Interactions between levels: an aggregated analysis precludes the study of the relationships between levels L. Grilli - Introduction to multilevel analysis - April 6 3 L. Grilli - Introduction to multilevel analysis - April 6 4 Problems with disaggregated analysis Which type of model? Inference on the clusters: impossible to make inference on the clusters (i.e. treat the clusters as a random sample from a population of clusters) Wrong sample size for the macro variables (their sample size should be the number of clusters) Dependence: the units of the same cluster are alike with a positive within cluster correlation the independence assumption of many standard models (e.g. GLM) is violated biased standard errors (often underestimated, leading to type I error rates higher than the nominal level α) Basic question: the hierarchical structure (along with the relationships within and between levels and the associated correlation structure) is of primary interest for the research? es, it is of primary interest multilevel models No, it is merely a nuisance (e.g. the sampling design is multistage but interest is limited to individual level relationships) methods able to correct the standard errors: GEE, robust (sandwich) estimation of the covariance matrix of the estimators L. Grilli - Introduction to multilevel analysis - April 6 5 L. Grilli - Introduction to multilevel analysis - April 6 6 One-Way random effects ANOVA i.e. the simplest multilevel model One-Way random effects ANOVA i.e. the simplest multilevel model ε, E( ε ) = 0, Var( ε ) = σ i i i ε u i i, = µ + u + ε i i =,, clusters (in ANOVA terminology "levels of the factor") i =,, n units in cluster N = n total sample size u, E( u ) = 0, Var( u ) = = Be aware: is the variance, not the standard deviation L. Grilli - Introduction to multilevel analysis - April 6 7 Useful to think in terms of a two-stage data generation process: = µ + u + ε i i ) sampling cluster -> take a realization of u from its distribution (so cluster has a mean of µ+u ) ) sampling unit i within cluster -> take a realization of ε i from its distribution (so unit i of cluster has value µ+u +ε i ) L. Grilli - Introduction to multilevel analysis - April 6 8 Dottorato XX ciclo febbraio 6 3

4 One-Way random effects ANOVA: variances and covariances One-Way random effects ANOVA: covariance matrix Var( ) = Var( u + ε ) = + σ i i = µ + u + ε i i 0 if Cov( i, i ) = if = and i i' Variance of i decomposed in two components: cluster (between) level + individual (within) level Observations belonging to the same cluster are positively correlated Remark: the correlation is necessarily positive since it is generated by a shared latent variable u (it is the same basic idea of factor models, where u is called factor indeed the GLLAMM class of Rabe-Hesketh and Skrondal includes both multilevel and factor models as special cases) L. Grilli - Introduction to multilevel analysis - April 6 9 Example =, n =, n = 3 + σ + σ Var( ) = + σ + σ + σ Block diagonal structure (empty space stands for zero) L. Grilli - Introduction to multilevel analysis - April 6 0 One-Way ANOVA: fixed or random effects? One-Way random effects ANOVA: intraclass correlation coefficient i = µ + α + εi parameters Fixed effects ANOVA when the clusters (levels of the factor) are few and represent an exhaustive classification distributional assumptions are avoided, but impossible to generalize the results to a popoulation of clusters = µ + u + ε i i random variables Random effects ANOVA when the clusters (levels of the factor) are a sample from a population of clusters or anyway when the clusters are many parisimonious description of the observed variability among clusters + generalizability L. Grilli - Introduction to multilevel analysis - April 6 ρ denotes the ICC (intraclass correlation coefficient) cluster variance ρ = Corr( i, i ) = = ρ 0, + σ total variance ρ is a measure of the degree of homogeneity of units belonging to the same cluster when ρ is high it is crucial to adopt an estimation method able to account for the correlation [ ] L. Grilli - Introduction to multilevel analysis - April 6 The consequences of correlation The design effect N µ σ,, (, ) N identically distributed N = i is an estimator of µ N i= ( ) E = µ unbiased regardless of correlation BUT Var σ N ( ) = + Cov( i, i' ) precision of depends on correlation σ Remark : for positive correlations Var ( ) > N i i' L. Grilli - Introduction to multilevel analysis - April 6 3 For simple random sampling (SRS), the variance of the sample mean is σ SRS / N For a design other than SRS, the theory of survey sampling defines deff = σ design / σ SRS (usually deff >) effective sample size = N / deff So the effective sample size is the sample size that under SRS would lead to the same variance of the sample mean L. Grilli - Introduction to multilevel analysis - April 6 4 Dottorato XX ciclo febbraio 6 4

5 The design effect Relationships within and between clusters 8 In a balanced two-stage design ( clusters of size n, so N= n): 7 Example from Sniders & Bosker, p. 7 deff = + (n-) ρ where ρ is the ICC The loss of precision of a balanced two-stage design (w.r.t. SRS) depends on both ρ and n (but not on ) Within Total i X_i X_. _i _ Example: =0, n=0, ρ=0. deff=.9, so the nominal sample size is N= but the effective sample size is /.9 69! 0 Between Difference Between-Within: Ecological fallacy L. Grilli - Introduction to multilevel analysis - April 6 5 L. Grilli - Introduction to multilevel analysis - April 6 6 Relationships within and between clusters Relationships within and between clusters = x i = 8..x.. i i =. +.( xi x. ) i = 8..x. +.( xi x. ) Total Between cluster means Within clusters Multilevel ( ) X β = η β + η β total X between cluster means within clusters SSBX η X = SSTX Remark : β total correlation ratio is a linear combination The multilevel regression model allows to study between and within relationships at the same time L. Grilli - Introduction to multilevel analysis - April 6 7 L. Grilli - Introduction to multilevel analysis - April 6 8 Motivating example: school effectiveness (one covariate at level ) (one covariate at level + one covariate at level ) Centering the covariates Issues in the random slope model Example: school effectiveness Levels: pupils (level ); schools (level ) Response variable : score on the final test Explanatory variable (at level ) X: score on the initial test First consider a single school: = β + β x + ε i 0 i i ε i ~ N(0, σ ) L. Grilli - Introduction to multilevel analysis - April 6 30 Dottorato XX ciclo febbraio 6 5

6 Example: school effectiveness (one covariate at level ) Sample of schools (from a population of schools) Comparison between schools A and B School A more effective (higher predicted for all the range of X) School A more equitable (lower slope) A Level model Equation for the -th school: i = β0 + βx i + εi ε ~ N(0, σ ) i effectiveness : β > β equity : β < β 0A 0B A B B X Remark: each school has its own slope and intercept L. Grilli - Introduction to multilevel analysis - April 6 3 L. Grilli - Introduction to multilevel analysis - April 6 3 (one covariate at level ) (one covariate at level ) Each school has a couple of parameters ( β, β ) 0 Assumption: the parameters ( β0, β) are random variables with a bivariate Normal distribution in the population of schools β N γ 0 0, β γ 0 ( 0, ) β β independent from ε i Other distributions are possible, but the Normal has nice properties and works well in many cases Model parameters γ mean intercept γ 0 mean slope Intercept variance Slope variance 0 Slope-intercept covariance Residual variance (level ) σ Fixed parameters Random parameters L. Grilli - Introduction to multilevel analysis - April 6 33 L. Grilli - Introduction to multilevel analysis - April 6 34 (one covariate at level ) (one covariate at level ) Correlation between slopes and intercepts: 0 corr( β0, β ) = β 0 Example of negative correlation Level model: Level model: Combined model: = β + β x + ε i 0 i i β = γ + u β = γ + u = γ + γ x + u x + u + ε i 0 i i 0 i β Fixed part Random part L. Grilli - Introduction to multilevel analysis - April 6 35 L. Grilli - Introduction to multilevel analysis - April 6 36 Dottorato XX ciclo febbraio 6 6

7 (one covariate at level ) (one covariate at level ) Level errors (random effects): u u = β γ Var( u ) = = β γ Var( u ) = Random effect = unexplained deviation of the value of the parameter in the -th cluster from the mean value in the population The covariates may contribute to explain the deviations (so reducing the corresponding variances) Usually the distributional assumptions are made on the random effects (and not on the random intercepts/slopes): u 0 0, 0 u0 N u 0 εi u indep. from L. Grilli - Introduction to multilevel analysis - April 6 37 The total error is 0 i i implying heteroschedasticity: u + u x + ε Var( x ) = + x + x + σ i i 0 i i non-homogeneous correlation among the responses of the units of the same cluster : Cov(, x, x ) = + ( x + x ) + x x i i' i i' 0 i i' i i' L. Grilli - Introduction to multilevel analysis - April 6 38 (one covariate at level ) (one covariate at level ) Var( x ) = + x + x + σ i i 0 i i The variance function is a parabola with minimum in 0 / Depending on the range of x, in a given application the variance function can be descending, ascending or U-shaped (but never shaped!) Var( x ) i 0 / L. Grilli - Introduction to multilevel analysis - April 6 39 i x i What about estimating the parameters γ and γ 0 using OLS (Ordinary Least Squares) or ML under the standard assumptions of the linear model? = γ + γ x + ε * i 0 i i ε = u + u x + ε * The model is linear but the error term i 0 i i violates the homoschedasticity and uncorrelatedness assumptions Inefficient estimators Biased standard errors L. Grilli - Introduction to multilevel analysis - April 6 40 (one covariate at level + one covariate at level ) (one covariate at level + one covariate at level ) Introduction of level covariates: level covariates represent features of the clusters useful to define a model for the level parameters ( β0, β ) and so reduce the level variances (, ) Example: W is a binary variable coded =public school; 0=private school Level model: i 0 i i Level model: Combined model: = β + β x + ε β = γ + γ w + u β = γ + γ w + u = γ + γ w + γ x + γ w x i 0 0 i i 0 i i Here it becomes clear why the γ have a double index Fixed part + u + u x + ε Random part L. Grilli - Introduction to multilevel analysis - April 6 4 L. Grilli - Introduction to multilevel analysis - April 6 4 Dottorato XX ciclo febbraio 6 7

8 (one covariate at level + one covariate at level ) (one covariate at level + one covariate at level ) Level model: β0 = γ + γ0w + u0 β = γ0 + γw + u γ 0 γ u 0 u Var( u ) = Var( u ) = 0 mean difference in intercept between private and public school mean difference in slope between private and public school deviation of school from the corresponding mean intercept deviation of school from the corresponding mean slope Remark: the distributional assumptions on the random effects are the same as before, but now the variances have a different meaning (remind: the variances are residual w.r.t. to the model covariates) L. Grilli - Introduction to multilevel analysis - April 6 43 In the combined model there is a cross-level interaction wx It arises because the level coefficient depends on the level covariate w A multilevel model can be written down in two ways: a) a single combined equation or, equivalently, b) a system of hierarchical equations. Who uses approach b usually ends up with a more complex model (notably, with more cross-level interactions) i β L. Grilli - Introduction to multilevel analysis - April 6 44 Model assumptions Centering a covariate Level Level εi ~ N(0, σ ) u 0 0, 0 N u 0 u u 0 indep. from εi The critical assumption is that E(u 0 )=0 and E(u )=0 for every value of the covariate, implying that E( x) is correctly specified The normality assumption is usually not so critical (especially with regard to level errors) A covariate can be centered w.r.t. a given constant, such as the grand mean: this affects the intercept γ (in a random slope model also the intercept variance and the intercept-slope covariance, see later on) the cluster mean (CM centering), so if the cluster means are different the centering varies from cluster to cluster: this affects the slope (total effect vs. within effect) L. Grilli - Introduction to multilevel analysis - April 6 45 L. Grilli - Introduction to multilevel analysis - April 6 46 Centering a covariate Centering a covariate Cronbach model: CM centering & cluster mean = γ + γ ( x x ) + γ x + u + ε i 0 i i = γ + γ x + u + ε Between slope = γ ( x x ) + ( ε ε ) i. 0 i. i. Within slope L. Grilli - Introduction to multilevel analysis - April 6 47 contextual model: no CM centering, but cluster mean = γ + γ x + γ x + u + ε i 0 i 0. 0 i replacing xi with ( xi x. ) + x. yields = γ + γ ( x x ) + ( γ + γ ) x + u + ε i 0 i i ust a reparametrization of the Cronbach model!! γ0 = γ0 = within slope γ = γ γ = between slope within slope L. Grilli - Introduction to multilevel analysis - April 6 48 Dottorato XX ciclo febbraio 6 8

9 Centering a covariate Centering a covariate raw covariate model: no CM centering, no cluster mean = γ + γ x + u + ε i 0 i 0 i = γ + γ x + u + ε = γ ( x x ) + ( ε ε ) i. 0 i. i. This model implicitly assumes that the between and within slopes are identical!! L. Grilli - Introduction to multilevel analysis - April 6 49 Summary: ) = + γ x + i total i ) = + γ x + ( γ γ ) x + i within i between within. 3) = + γ ( x x ) i within i. 4) = + γ ( x x ) + γ x + i within i. between. More details in Raudenbush & Bryk and Sniders & Bosker L. Grilli - Introduction to multilevel analysis - April 6 50 The fixed effects model (also called fixed effects ANCOVA) = γ + γ x + u + ε i 0 i 0 i replaced by parameters α so no distributional assumptions on u 0!!! In this way the level variability is completely explained -> impossible to add level covariates (multicollinearity) If many clusters -> too many parameters (inefficiency, overfitting) For a small cluster the α parameter is estimated with low precision Impossible to generalize to a population of clusters Remark: in the fixed effecs model the slope γ 0 is not the total effect, but the within effect (in panel data the corresponding estimator is known as the fixed effects estimator) L. Grilli - Introduction to multilevel analysis - April 6 5 Endogeneity Assume the true model is i = γ + γ 0xi + γ 0x. + u0 + εi where γ, i.e. the between and within slope are different 0 = 0 Assume the model is specified without the cluster mean = α + βx + u + ε * * i i 0 i ( ) u x u E u x * * 0 = γ = γ0. 0 Econometriacians say there is endogeneity, since the random effect is correlated with the covariate estimate of slope is biased L. Grilli - Introduction to multilevel analysis - April 6 5 Endogeneity Versions of the two-level model Solutions to endogeneity:. Replace the random effects with fixed effects. Keep the random effects but add the cluster mean in the equation The famous Hausman specification test is ust a test for the equality of between and within slopes, i.e. H0 : γ 0 = 0 Some econometriacians believe that when the Hausman test reects the null hypotesis one is forced to use solution. But also solution is feasible! L. Grilli - Introduction to multilevel analysis - April 6 53 i = β0 + βxi + εi β0 = γ + u0 β = γ0 + u T 0 = ε i ~ N(0, σ ) u0 0 0 ~ N, u 0 Variance matrix of the random effects Let us consider some special cases of T Remark: when one of the variances is null, also the covariance 0 is null L. Grilli - Introduction to multilevel analysis - April 6 54 Dottorato XX ciclo febbraio 6 9

10 Versions of the two-level model Versions of the two-level model Special case T=0 No variability among clusters Fixed coefficients, same as a standard regression model y x L. Grilli - Introduction to multilevel analysis - April 6 55 Special case 0 T = 0 The variance of the slope is null (and so is the slope-intercept covariance) The variance of the intercept does not depend on X (i.e. centering X is irrelevant) The cluster regression lines are parallel The clusters can be ranked Equi-correlation within clusters (so residual ICC is meaningful) y L. Grilli - Introduction to multilevel analysis - April 6 56 x Versions of the two-level model Versions of the two-level model Special case T singular (rank=) 0 = All the regression lines cross at a point x * If x * is outside the range of X megaphone configuration, so the clusters can be ranked If x * is inside the range of X radial configuration, so there are two specular rankings of the clusters depending on x>x * or x<x * y y X General case T full rank There are many crossing points The clusters cannot be ranked Heterogeneous correlation within clusters (no unique residual ICC) y x L. Grilli - Introduction to multilevel analysis - April 6 57 x L. Grilli - Introduction to multilevel analysis - April 6 58 Versions of the two-level model Unit of measure General case T full rank What it the unit of measure of the variances/covariances? The intercept variance ( ) and the slope-intercept covariance ( 0 ) depend on X and 0 refer to X=0 so it is useful to center X Since the origin of X is often arbitrary the covariance should not be constrained to be zero y x m = unit of measure of (e.g. Kilogrammes) m X = unit of measure of X (e.g. Metres) Then is expressed in (m ) is expressed in (m / m X ) 0 is expressed in (m ) / m X y x Be careful in the interpretation! L. Grilli - Introduction to multilevel analysis - April 6 59 L. Grilli - Introduction to multilevel analysis - April 6 60 Dottorato XX ciclo febbraio 6 0

11 Extensions of the model The three level variance component model Many covariates at both levels: X, X,,W, W, Complex error structure: At level : e.g. heteroschedasticity Var( εi ) = σ x At level : e.g. many random slopes More than two hierarchical levels But be aware that the imagination of the researchers can easily outrun the capacity of the data, the computer, and current optimization techniques to provide robust estimates (Di Prete & Forristal) L. Grilli - Introduction to multilevel analysis - April 6 6 i k =,,, K level 3 units (e.g. schools) =,, i =,, I k k k level units (e.g. classes) level units (e.g. pupils) [ ] = fixed part + v + u + ε ik k k ik v ~ N(0, σ ) u (3) ~ N(0, σ ) k ik () ε ~ N(0, σ ) () independence among levels L. Grilli - Introduction to multilevel analysis - April 6 6 Estimation Estimation in random effects ANOVA Shrinkage estimators and reliability Model specification and likelihood Type of inference: maximum likelihood vs. Bayesian Maximum likelihood: full information vs. restricted Hypotesis testing Diagnostics based on the residuals Comparing the residuals Parameter estimation i= γ+ γ0w+ γ0xi+ γwxi Fixed part + u + u x + ε Random part 0 i i Maximum likelihood step : estimation of fixed parameters (γ, γ 0, γ 0, γ ) and random parameters (σ,, 0, ) step : prediction ofrandom effects (u 0,u : =,,) Bayesian inference the parameters are random variables with a prior distribution no distinction between parameters and random effects L. Grilli - Introduction to multilevel analysis - April 6 64 One-way random effects ANOVA: estimation of γ One-way random effects ANOVA: estimation of γ = γ + u + ε i 0 i Cluster mean. = γ + u0 + ε. E( ) = γ Var( u ) = Var( εi ) = σ 0. σ (. ) = Var( u0 + ε. ) = + = n Var Each cluster mean is un unbiased estimator of γ, but it is inefficient Idea : estimate γ,, combining.. L. Grilli - Introduction to multilevel analysis - April 6 65 The minimum variance unbiased estimator of γ depends on the design (all the sample means Balanced design: n = n = have the same variance) γ =. = Unbalanced design: ˆ γ =. = = is the precision of. Remark: in the unbalanced case it is necessary to have an estimate of the best estimator depends on the variances L. Grilli - Introduction to multilevel analysis - April 6 66 Dottorato XX ciclo febbraio 6

12 One-way random effects ANOVA: estimation of the variance components One-way random effects ANOVA: estimation of the variance components SST = SSW + SSB n n ( i.. ) = ( i. ) + n (... ) = i= = i= = For simplicity consider only the balanced case n =n Two possible estimators of the variance components: n SSW SSB W = = ( i. ) B = = (...) N N = i= n( ) = S S L. Grilli - Introduction to multilevel analysis - April 6 67 S σ E( SW ) = σ E( SB) = + > n σ Var(. ) = + n Not all the variability of the sample cluster means. is due to the variability of the population cluster means µ + u B overestimates since it actually estimates Remark : this fact underlies the classical ANOVA F test ( H : = 0) ˆ ˆ B Un unbiased estimator of is = S ˆ σ ˆ < ˆ = 0 n Remark : can be negative! If S B then 0 ˆ σ n 0 L. Grilli - Introduction to multilevel analysis - April 6 68 oint estimation of fixed and random parameters To estimate the fixed parameters it is necessary to have an estimate of the random parameters However the converse is also true: to estimate the random parameters it is necessary to have an estimate of the fixed parameters In general it is necessary to rely upon iterative procedures Exception: balanced design (all clusters with the same size and the same matrix of covariates) In a balanced design there are closed-form estimators of the variance components (see the textbook by Searle, McCulloch and Casella) L. Grilli - Introduction to multilevel analysis - April 6 69 Effect of the covariates on the variances Consider starting with a random effects ANOVA model = µ + u + ε i 0 i Var( u ) = Var( ε ) = σ 0 both level and level covariates can be added: A level covariate reduces (or leaves unchanged) the level variance but (being constant within each cluster) cannot affect the level variance σ A level covariate reduces (or leaves unchanged) the level variance but its effect on the level variance is unpredictable σ L. Grilli - Introduction to multilevel analysis - April 6 70 i Effect of the covariates on the variances a level covariate by definition varies only between clusters purely between a level covariate can be written as the sum of: A purely between component Reduces and does not affect σ A purely within component Reduces σ and thus increases x i x x If a level covariate is purely within it increases Usually a level covariate varies both within and between the effect on is unpredictable (often reduces it) L. Grilli - Introduction to multilevel analysis - April 6 7 ˆ σ ˆ = SB n This effect is strong only if n is small Effect of the covariates on the variances General rule: a covariate at an arbitrary level does not affect the variances at lower levels reduces (or leaves unchanged) the variances at the same level has an unpredictable effect on the variances at higher levels L. Grilli - Introduction to multilevel analysis - April 6 7 Dottorato XX ciclo febbraio 6

13 One-way random effects ANOVA: prediction of β 0 or u 0 One-way random effects ANOVA: prediction of β 0 or u 0 From. = β0 + ε., with ε. ~ N(0, σ / n), it follows that. is an (unbiased) estimator of β0 with variance σ / n From β0 = γ + u0, with u0 ~ N(0, ), it follows that also ˆ γ is an estimator (biased) of β =. = = Remark: here unbiasedness is evaluated conditional on β 0 (i.e. hold u 0 and calculate E( ) under repeated sampling of ε i ) Good news: A linear combination of estimators and yields a new estimator better then both and in terms of MSE!!! 0 L. Grilli - Introduction to multilevel analysis - April 6 73 The best estimator of of estimators and λ β 0 is given by a linear combination EB β 0. ( ) ˆ = λ + λ γ the weight is a measure of the reliability of. as an estimator of β 0 The superscrip EB stands for Empirical Bayes since 0 is also the posterior mean of calculated by pluggin-in the ML estimates β 0 βˆ EB L. Grilli - Introduction to multilevel analysis - April 6 74 One-way random effects ANOVA: prediction of β 0 or u 0 The reliability coefficient EB β 0. ( ) ˆ = λ + λ γ EB EB u 0 β ˆ 0 (. ˆ γ λ γ ) = = shrinkage estimator OLS residual shrinkage factor Borrowing strength L. Grilli - Introduction to multilevel analysis - April 6 75 The coefficient in the linear combination in β is λ = In fact from the psychometric theory of tests (i=item, =individual) γ + u 0. + σ / n λ true score of individual observed score of individual λ = = + σ / n ˆ EB 0 known as the reliability coefficient variance of true scores λ varies from cluster variance of observed scores to cluster (as it depends also on the cluster size) L. Grilli - Introduction to multilevel analysis - April 6 76 Likelihood Likelihood The equation defining a random effects model includes the random effects, but they are not observable (so cannot appear in the likelihood) the random effects must be integrated out! f ( y u, θ) p( u θ) Distribution of responses, conditional on random effects and parameters Distribution of random effects, conditional on parameters L( θ) = f ( y u, θ) p( u θ) du Likelihood Problem: the integral has analytical solution only for conugate distributions (e.g. Normal-Normal, Binomial-Beta, ) L. Grilli - Introduction to multilevel analysis - April 6 77 Multiple random effects -> multivariate distribution -> the Normal distribution is preferable (and in fact it is the standard choice in applications) Linear model -> Normal-Normal (conugate) -> the integral has analytical solution Non linear model -> f ( y u, θ) -Normal (non conugate) -> the likelihood must be evaluated through approximate integration methods L. Grilli - Introduction to multilevel analysis - April 6 78 Dottorato XX ciclo febbraio 6 3

14 Maximum Likelihood Bayesian inference θ ML = arg max L( θ) = f ( y u, θ) p( u θ) du Once the ML estimates of fixed and random parameters have been obtained -> Empirical Bayes estimation of the random effects p( u y, θ ML) = θ f ( y u, θ ) ( ML p u θ ML ) f ( y u, θ ) ( ML p u θ ML ) du For Bayesian inference a prior distribution p( θ) and the oint posterior distribution is used p( u, θ y ) = Inference on the parameters Inference on the random effects f ( y u, θ) p( u θ) p( θ) f ( y u, θ) p( u θ) p( θ) dudθ is defined p( θ y ) = p( θ, u y ) du p( u y) = p( θ, u y ) dθ L. Grilli - Introduction to multilevel analysis - April 6 79 L. Grilli - Introduction to multilevel analysis - April 6 80 Bayesian inference Hierarchical specification of the model (useful to write down the likelihood) Even for the linear model it is necessary to use approximate integration algorithms (e.g. Gibbs sampling) The Bayesian approach has some well-known pros and cons An important advantage in complex multilevel models is that the estimates properly account for all the uncertainty -> Bayesian methods yield good estimates of the variance components and confidence intervals with appropriate coverage even in highly complex models, where ML methods show a poor performance L. Grilli - Introduction to multilevel analysis - April 6 8 The simple random intercept model = γ + γ x + u + ε i 0 i 0 i can also be written hierarchically (which is the standard way to specify multilevel GLMs) Independence stems from. i xi, u0 N ( γ + γ0xi + u0, σ ) conditioning on the random effects. u N 0, ( ) 0 Remark: in the hierarchical specification the level errors ε i are not written (only their variance) L. Grilli - Introduction to multilevel analysis - April 6 8 Hierarchical construction of the likelihood for the random intercept model Maximum Likelihood The likelihood can be written in steps by exploiting the conditional independencies shown by the hierarchical formulation n L ( ψ u ) = L ( ψ u ) by conditional indep. given u 0 i 0 0 i= L ( θ) = L ( ψ u ) p( u ) du integrating out the random effect u L ( ) ( γ γ0 σ ) N( γ + γ0xi + u0 σ ) = param. of random effects θ= ( ψ, ) where ψ =,, all other parameters density of, Li ( ψ u0 ) = for fixed u evaluated at the observed y u ( θ) = L ( θ) by independence among clusters = L. Grilli - Introduction to multilevel analysis - April i Full Information Maximum Likelihood (FIML) Full likelihood, oint estimation of fixed and random parameters Underestimates the random parameters since it treats the fixed parameters as known quantities (ignoring degrees of freedom) Restricted Maximum Likelihood (REML) The random parameters are estimated by maximizing the restricted likelihood, i.e. the density of the residuals The random parameters are well estimated even in small samples L. Grilli - Introduction to multilevel analysis - April 6 84 Dottorato XX ciclo febbraio 6 4

15 Maximum Likelihood Maximum Likelihood To understand why FIML underestimates the random parameters remind what happens in standard linear regression i = β ' Xi + εi Var( εi)= σ εi = ' ( i β X ) i σ FIML = εi σ OLS = εi E σ OLS = σ n n k i If k is large w.r.t. n, then σ FIML is severely downward biased (it does not correct for the degrees of freedom lost in estimating β) In a multilevel model the sample size which is relevant for the estimation of is, i.e. the number of clusters i L. Grilli - Introduction to multilevel analysis - April 6 85 In a two-level model, REML and FIML lead to: Similar estimates for σ Discordant estimates for the parameters of the random effects if is small (in such a case FIML estimates of variances are lower) Unless the main aim is the estimation of the random parameters, FIML is preferred because: FIML estimators have a lower sampling variance (the comparison in terms of MSE is often in favour of FIML estimators) With FIML the LRT (Likelihood Ratio Test) can be used not only to test the random parameters, but also to test the fixed parameters L. Grilli - Introduction to multilevel analysis - April 6 86 FIML algorithms Properties of FIML estimators i) Iterative Generalized Least Squares (Goldstein 986) ii) Fisher Scoring (Longford 987) Under mild regularity conditions FIML estimators have good asymptotic properties: Consistency Normality Efficiency iii) EM (developed by Dempster, Laird and Rubin for models with missing data, later applied to multilevel models since the random effects are, in a sense, missing data) Remark: here asymptotic requires increasing the number of clusters (increasing the cluster sizes is not enough), so is the key quantity for asymptotics L. Grilli - Introduction to multilevel analysis - April 6 87 L. Grilli - Introduction to multilevel analysis - April 6 88 Hypotesis testing on a single fixed parameter Hypotesis testing on a set of fixed parameters Null hypotesis: H 0 : γ h =0 Wald test statistic: γ h T ( γ h) = se..( γ ) approx units)-#(covariates)- if γ h coeff. covariate at level ( γ h) t d.f. {#(level #(clusters)-#(covariates at level )- if γ h coeff. covariate at level A caveat: with few clusters the standard errors are usually underestimated the test reects the null hypothesis too often (i.e. the type I error rate is higher the nominal level) T h L. Grilli - Introduction to multilevel analysis - April 6 89 Null hypotesis: H 0 : C γ = 0 (C = matrix of contrasts with k rows) ˆ T Q( C' γ) = ( C' γˆ) ΣC' γ ( C' γγ ˆ) ˆ Wald test statistic: Q( C' γ) approx χk Remark: with few clusters the F distribution is preferable Alternative: LRT (asymptotically equivalent) L. Grilli - Introduction to multilevel analysis - April 6 90 Dottorato XX ciclo febbraio 6 5

16 Hypotesis testing on the random parameters LRT on a variance component Hypotesis H : T= T vs. H : T= T 0 0 where T 0 is a restricted version of T (e.g. some elements are constrained to 0) Unless the number of clusters is huge, the Wald test should not be used since the sampling distributions of the estimators of the random parameters are highly asymmetric L. Grilli - Introduction to multilevel analysis - April 6 9 D = deviance of the unrestriced model D 0 = deviance of the model with qq =0 #(restrictions) = + #(corresponding covariances) approx 0 #( restrictions) D D χ 0 = χ #( restrictions) with prob. / with prob. / Practical rule: the p-value must be halved! Remark: with REML the LRT can be used only if the fixed part of the model is unchanged! L. Grilli - Introduction to multilevel analysis - April 6 9 Diagnostics based on the residuals Diagnostics based on the residuals Many residuals Level ˆi ε Level (Empirical Bayes) uˆ, u ˆ, 0 Extension of the techniques used in standard regression Purposes: Check the functional form and the distributional assumptions (normality, homoschedasticity, ) Look for influential units To check the normality assumption use an histogram or a Q-Q plot L. Grilli - Introduction to multilevel analysis - April 6 93 L. Grilli - Introduction to multilevel analysis - April 6 94 Diagnostics based on the residuals Diagnostics based on the residuals To locate anomalous units look at the standardized residuals To look for misspecifications in the fixed part and/or heteroschedasticity, plot the residuals one by one against the predicted values of the response standardized u ˆ standardized u ˆ0 L. Grilli - Introduction to multilevel analysis - April 6 95 L. Grilli - Introduction to multilevel analysis - April 6 96 Dottorato XX ciclo febbraio 6 6

17 Comparing the residuals Comparing the residuals The cluster (level ) residuals are predictions of the corresponding random effects u Often the value of u for a single cluster is not of primary interest; rather, it more interesting to compare the value of u for two clusters (e.g. to see if institution A is more effective than institution B) A statistical problem: the random effects of two given clusters are different? L. Grilli - Introduction to multilevel analysis - April 6 97 A common misconception: thinking that two quantities whose 95% intervals are disoint are significantly different at 5% X N( µ X, σ ) X ±.96 σ N( µ, σ ) ±.96 σ Assuming for simplicity that X and are independent X N ( µ µ, σ ) ( X ) ±.96 σ X µ X is significantly different from µ at level 95% if and only if: the distance (in σ units) between X and exceeds.96 =.77 or the univariate intervals of length.77/ =.39 are disoint L. Grilli - Introduction to multilevel analysis - April 6 98 Comparing the residuals The value.39 stems from assumptions (Normal distribution, independence, same variance) which usually are not satisfied -> the value.39 is an approximation and the significance level 5% is an average level Software & Books Graph for pairwise comparisons More details in Goldstein & Healy (995) RSS A Half-interval=.39* s.e. of the residual L. Grilli - Introduction to multilevel analysis - April 6 99 Software for multilevel modelling Specialized software Specialized software Procedures in general purpose software Review: Data and exercises: Multilevel Modeling Resources at UCLA aml (Panis) HLM (Raudenbush) the MIXOR suite (Hedeker) MLwiN (Goldstein) L. Grilli - Introduction to multilevel analysis - April 6 0 L. Grilli - Introduction to multilevel analysis - April 6 0 Dottorato XX ciclo febbraio 6 7

18 Procedures in general purpose software Books on multilevel modelling BMDP EGRET GENSTAT SPSS SAS PROC MIXED and NLMIXED R and S-PLUS STATA xt suite STATA gllamm (Rabe-Hesketh & Skrondal) LISREL (oreskog) M-plus (Muthén) WINBUGS (for Bayesian analysis) Also structural equations L. Grilli - Introduction to multilevel analysis - April 6 03 Sniders & Bosker Raudenbush & Bryk Hox L. Grilli - Introduction to multilevel analysis - April 6 04 Books on multilevel modelling XT commands in Stata Goldstein Skrondal & Rabe-Hesketh L. Grilli - Introduction to multilevel analysis - April 6 05 For fitting random intercept models = β + β x + u + ε i 0 i 0 i Conceived for panel data, where =individual, i=time point (wave) xtreg (Continuous response) be between-effects estimator fe fixed-effects (conditional) estimator From version 9 xtmixed re GLS random-effects estimator pa GEE population-averaged estimator mle maximum-likelihood random-effects estimator xtlogit, xtprobit (Binary response) xtpois (Counts) L. Grilli - Introduction to multilevel analysis - April 6 06 GLLAMM Generalised Linear Latent And Mixed Models GLLAMM: pros and cons Wide class of latent variable models for a (multivariate mixed) response (continuous, binary, ordinal, polytomous, counts, durations). The ingredients of GLLAMM are: The conditional mean of the response given the covariates and the latent variables E( X,u); The conditional distribution of the response given the covariates and the latent variables f( X,u) The structural equations (if any) for the latent variables The distribution of the latent variables Rabe-Hesketh, Skrondal e Pickles developed the corresponding STATA command gllamm See L. Grilli - Introduction to multilevel analysis - April 6 07 Pros Quite flexible (the GLLAMM class is very wide) Good approximations, accurate estimates Cons Computationally intensive because of Gaussian quadrature. Computational time is proportional on Number of observations Square of the number of parameters Product of the number of quadrature points for all the latent variables For models with continuous response is inefficient L. Grilli - Introduction to multilevel analysis - April 6 08 Dottorato XX ciclo febbraio 6 8

19 To know more about multilevel modelling in Stata For xt commands: Stata manuals For gllamm: manual: web site: For both: the recent book by Rabe-Hesketh & Skrondal L. Grilli - Introduction to multilevel analysis - April 6 09 Dottorato XX ciclo febbraio 6 9