3.1 The Basic Two-Level Model - The Formulas

Transcription

1 CHAPTER 3 3 THE BASIC MULTILEVEL MODEL AND EXTENSIONS In the previos Chapter we introdced a nmber of models and we cleared ot the advantages of Mltilevel Models in the analysis of hierarchically nested data. First of all, these models respect the hierarchy of the data and analyze data simltaneosly in all levels. They allow for variables entry in all levels as well as cross-level interactions (interactions of variables measred in different levels. In Random Coefficient Models the lowest level regression coefficient are treated as random variables at the higher level, which eplains frther the variability of the model. In this Chapter we first elaborate more on the development of a basic -level model. We reconsider alternative ways and notations of setting p and motivating the model and introdce procedres for estimating parameters, forming and testing fnctions of the parameters and constrcting confidence intervals. Then we etend to the natral etensions of the basic -level model by introdcing higher-level strctre, as well as special cases. These are the cross-classified models, the generalized mltilevel models for proportion as otcome and the mltivariate mltilevel model. The scope of this Chapter is, therefore, to present in etent all theoretical aspects and advantages of a Mltilevel Model and to show how this kind of analysis can be effective both in simple hierarchical data problems, as well as in even more comple theoretical statistical data strctres. 3. The Basic Two-Level Model - The Formlas 3.. The -level model and basic notation We first consider a simple model for one grop, relating the response variable to one simple eplanatory variable. We write: y = α β e (3. i i i where standard interpretations can be given to the intercept ( α, slope ( β and residal ( e i. We follow the normal convention of sing Greek letters for the 33

2 regression coefficients and place a circmfle over any coefficient (parameter which is a sample estimate. This is the formal model and describes a single-level relationship. To describe simltaneosly the relationships for several grops we write, for grop y = β e α (3. This is now the formal model where refers to the level nit and i to the level nit. As it stands, (3. is still essentially a single level model, albeit describing a separate relationship for each grop. In some sitations, for eample where there are few grops and interest centres on st those grops in the sample, we may analyze (3. by fitting all the n parameters, namely ( α, β,..., n and = assming a common 'within-grop' residal variance and separate lines for each grop. If we wish to focs not st on these grops, bt on a wider 'poplation' of grops then we need to regard the chosen grops as giving s information abot the characteristics of all the grops in the poplation. Jst as we choose random samples of individals to provide estimates of poplation means etc., so a randomly chosen sample of grops can provide information abot the characteristics of the poplation of grops. In particlar, sch a sample can provide estimates of the variation and covariation between grops in the slope and intercept parameters and will allow s to compare grops with different characteristics. An important class of sitations arises when we wish primarily to have information abot each individal grop in a sample, bt where we have a large nmber of grops so that (3. wold involve estimating a very large nmber of parameters. Frthermore, some grops may have rather small nmbers of observations and application of (3. wold reslt in imprecise estimates. In sch cases, if we regard the grops as members of a poplation and then se or poplation estimates of the mean and between-grop variation, we can tilize this information to obtain more precise estimates for each individal grop. This will be discssed later in the section dealing with 'residals'. e 34

3 To make (3. into a genine -level model we let α, β become random variables. For consistency of notation replace α by β and β by β and assme that where β (3.3a = β β (3.3b = β, are random variables with parameters ( = = E E( (3.4a var( =, var( =, cov(, = (3.4b We can now write (3. in the form where y = β β e (3.5 ( var( e = (3.6 e We shall reqire the etra sffi in the level residal term for models with more comple residal term. We have epressed the response variable y as the sm of a fied part and a random part within the brackets. We shall also generally write the fied part of (3.5 in the matri form E ( Y = Xβ with Y = y } (3.7 { E ( y = X β = ( Xβ, X = X } (3.8 where {} denotes a matri, X is the design matri for the eplanatory variables and { X is the -th row of X. For model (3.5 we have X = { }. Note the alternative representation for the i-th row of the fied part of the model. The random variables are referred to as 'residals' and in the case of a single level model the level residal e becomes the sal linear model residal term. To make the model symmetrical so that each coefficient has an associated eplanatory variable, we can define a frther eplanatory variable for the intercept 35

4 β and its associated residal,, namely, which takes the vale.. For simplicity this variable may often be omitted. The featre of (3.5 which distingishes it from standard linear models of the regression or analysis of variance type is the presence of more than one residal term and this implies that special procedres are reqired to obtain satisfactory parameter estimates. Note that it is the strctre of the random part of the model, which is the key factor. In the fied part the variables can be measred at any level. We can also inclde so called 'compositional' variables sch as the average vale of an eplanatory variable for all individals in each grop. The presence of sch variables does not alter the estimation procedre, althogh reslts will reqire carefl interpretation. We will elaborate more on estimation procedres in the following section. 3.. Parameter estimation for the variance components model Eqation (.5 reqires the estimation of two fied coefficients, β, β, and for other parameters,,, and. We refer to sch variances and e covariances as random parameters. We start, however, by considering the simplest - level model, which incldes only the random parameters,. It is termed a variance components model becase the variance of the response, abot the fied component, the fied predictor, is var( y β, β, = var( e = (3.9 e that is, the sm of a level and a level variance. This model implies that the total variance for each individal is constant and that the covariance between two individals (denoted by i, i in the same grop is given by cov( e, e = cov(, = (3. i i since the level residals are assmed to be independent. The correlation between two sch individals is therefore ρ = ( e (3. which is referred to as the intra-level--nit correlation or the intra-class correlation. This correlation measres the proportion of the total variance which is e 36

5 37 between-grops. In a model with 3 levels, we will have two sch correlations; the intra-level-3-nit correlation and the intra-level--nit correlation, and so on. The eistence of a non-zero intra-nit correlation, reslting from the presence of more than one residal term in the model, means that traditional estimation procedres sch as 'ordinary least sqares' (OLS which are sed for eample in mltiple regression, are inapplicable. A later section illstrates how the application of OLS techniqes leads to incorrect inferences. We now look in more detail at the strctre of a -level data set, focsing on the covariance strctre typified by Figre 3.. Figre 3.: Covariance matri of three first-level nits in a single -level contet for a variance components model e e e The matri in figre 3. is the (3 3 covariance matri for the scores of three individals in a single grop, derived from the above epressions. For two grops, one with three individals and one with two, the overall covariance matri is shown in Figre 3.. This 'block-diagonal' strctre reflects the fact that the covariance between individals in different grops is zero, and clearly etends to any nmber of level nits. Figre 3.: The block-diagonal covariance matri for the response vector Y for a -level variance components model with two level nits = e e e e e B A

6 A more compact way of presenting this matri, which we shall se, again is given in figre 3.3. Figre 3.3: Block-diagonal covariance matri sing general notation V = J I ( 3 e ( 3 J I ( e ( where I ( n is the (n n identity matri and J ( n is the (n n matri of ones. The sbscript for V indicates a -level model. In single-level OLS models is zero and this covariance matri then redces to the standard form I where is the (single residal variance The general -level model inclding random coefficients We now etend (3.5 in the standard way to inclde frther fied eplanatory variables y = β β β ( e (3. h h h= p and more compactly as y = X z e z β h h h= (3.3 where we se new eplanatory variables for the random part of the model and write these more generally as Z = { Z Z } (3.4 where Z } = { i.e a vector of s and Z = { }. 38

7 39 The eplanatory variables for the random part of the model are often a sbset of those in the fied part, as here, bt this is not necessary. Also, any of the eplanatory variables may be measred at any of the levels; for eample we may have individal characteristics at level or grop characteristics at level. This model (3.3, with the coefficient of X random at level, gives rise to the following typical block strctre, for a level-two block with two level-one nits. The matri Ω is the covariance matri of the random intercept and slope at level. Note that we need to distingish careflly between the covariance matri of the responses given in the following strctre and the covariance matri of the random coefficients. We also refer to the intercept as a random coefficient. The matri Ω is the covariance matri for the set of level-one random coefficients; in this case there is st a single variance term at level one. We also write Ω Ω = { } i for the set of these covariance matrices. More eplicitly: Figre 3.4: Response covariance matri for a level nit with two level nits for a -level model with a random intercept and random regression coefficient at level- = ( ( e e C B B A e A = B ( = e C = giving Ω Ω Ω = T X X C B B A where

8 X =, Ω =, Ω = e We also see here the general pattern for constrcting the response covariance matri which generalizes both to higher order models and to comple variation at level Parameter Estimates - Possible Approaches Algorithms It is obvios even from the previos discssion that the parameters estimation for both the fied and the random part of the model is a crcial isse in Mltilevel analysis, especially de to the large nmber of parameters that have to be estimated. For a model of P predictors for the lowest level and Q predictors for the highest level the nmber of estimates is shown in the following Table (taken by Ho (995: Table 3.: Nmber of parameters to be estimated in a fll Mltilevel model Parameters Nmber of Estimates Intercept Lowest level error variance Slopes for the lowest level predictors P Highest level error variances for these slopes P Highest level covariances of the intercept with all slopes P Highest level covariances between all slopes P(P-/ Slopes for the highest level predictors Q Slopes for cross level interactions P Q Several techniqes and principles and their corresponding algorithms have been proposed in order to reach reliable estimates for both fied and random part. As far as Maimm Likelihood techniqe is concerned, two different varieties of Maimm Likelihood estimation are sed for mltilevel regression analysis. One is 4

9 called Fll Maimm Likelihood (FML; in this method both the regression coefficients and the variance components are inclded in the likelihood fnction. The other method is called Restricted Maimm Likelihood (REML, here only the variance components are inclded in the likelihood fnction. The difference is that FML treats the estimates for the regression coefficients as known qantities when the variance components are estimated, while REML treats them as estimates that carry some amont of ncertainty (Bryk & Radenbsh, 99. Since REML is more realistic, it shold, in theory, lead to better estimates, especially when the nmber of grops is small (Bryk & Radenbsh, 99. However, in practice, the differences between the two methods are not very important. Compting the Maimm Likelihood estimates reqires an iterative procedre. The most commons of the algorithms (The Iterative Generalized Least Sqare Method and the EM algorithm are discssed in this chapter, as well as other techniqes and procedres. The Iterative Generalized Least Sqare (IGLS Method We now give an overview of the Iterative Generalized Least Sqares (IGLS method which also forms the basis for many of the developments in more comple analysis. We consider the simple -level variance components model y = β β e (3.5 Sppose that we knew the vales of the variances, and so cold constrct immediately the block-diagonal matri V, which we will refer to simply as V. We can then apply immediately the sal Generalized Least Sqares (GLS estimation procedre to obtain the estimator for the fied coefficients $ T β = ( X V X T X V Y (3.6 where in this case X y y = M M Y = M n m y m m n m (3.7 4

10 with m level nits and n level nits in the -th level nit. When the residals have Normal distribtions (3.6 also yields maimm likelihood estimates. Or estimation procedre is iterative. We wold sally start from 'reasonable' estimates of the fied parameters. Typically these will be those from an initial OLS fit (that is assming From these we form the 'raw' residals =, to give the OLS estimates of the fied coefficients $ β (. y% = y $ β $ β The vector of raw residals is written Y% { % } (3.8 = (3.9 y If we form the cross-prodct matri Y%% Y T we see that the epected vale of this is simply V. We can rearrange this cross prodct matri as a vector by stacking the colmns one on top of the other which is written as vec( YY ~~ T and similarly we can constrct the vector vec( V. For the strctre given in figre 3., these both have 3 = 3 elements. The relationship between these vectors can be epressed as the following linear model ~ y e ~ y ~ y M = M R = M e M R ~ y e (3. where R is a residal vector. The left hand side of (3. is the response vector in the linear model and the right hand side contains two eplanatory variables, with coefficients, which are to be estimated. The estimation involves an e application of GLS sing the estimated covariance matri of vec( Y%% Y T, assming Normality, namely ( V V where is the Kronecker prodct. The Normality assmption allows s to epress this covariance matri as a fnction of the random parameters. Even if the Normality assmption fails to hold, the reslting estimates are still consistent, althogh not flly efficient, bt standard errors, estimated sing the Normality assmption and, for eample confidence intervals will generally not be consistent. For certain variance component models alternative distribtional assmptions have been stdied, especially for discrete response models of the kind 4

11 discssed later in the thesis and maimm likelihood estimates obtained. For more general models, however, with several random coefficients, the assmption of mltivariate Normality is a fleible one, which allows a convenient parameterization for comple covariance strctres at several levels. With the estimates obtained from applying GLS to (3. we retrn to (3.6 to obtain new estimates of the fied effects and so alternate between the random and fied parameter estimation ntil the procedre converges, that is the estimates for all the parameters do not change from one cycle to the net. At convergence, assming mltivariate Normality, the estimates are maimm likelihood. Essentially the same procedre can be sed for the more complicated models discssed later on in the thesis. The maimm likelihood procedre prodces biased estimates of the random parameters becase it takes no accont of the sampling variation of the fied parameters. This may be important in small samples. Goldstein (989a shows how a simple modification leads to restricted iterative generalized least sqares (RIGLS or restricted maimm likelihood (REML estimates which are nbiased. The IGLS algorithm is readily modified to prodce these restricted estimates (RIGLS Fll details of efficient comptational procedres for carrying ot all these calclations are given by Goldstein & Rasbash (99. The EM algorithm To illstrate the procedre, consider the -level variance components model y = ( Xβ e, var( e =, var( = e (3. The vector of level residals is treated as missing data and the 'complete' data therefore consists of the observed vector Y and the treated as observations. The oint distribtion of these, assming Normality, and sing or standard notation is Y = N X V J T β J I, (3. This generalizes readily to the case where there are several random coefficients. If we denote these by β we note that some of them may have zero variances. We can now derive the distribtion of β Y, and we can also write down the Normal log likelihood fnction for (3. with a general set of random coefficients, namely 43

12 T log( L N log( e J log Ω e β Ω β (3.3 e Maimizing the latter for the random parameters we obtain = N (3.4 Ω e e T = m β β (3.5 where m is the nmber of level nits. We do not know the vales of the individal random variables. We reqire the epected vales, conditional on the Y and the crrent parameters, of the terms nder the smmation signs, these being the sfficient statistics. We then sbstitte these epected vales in (3.4 and (3.5 for the pdated random parameters. These conditional vales are based pon the 'shrnken' predicted vales and their (conditional covariance matri. With these pdated vales of the random parameters we can form V and hence obtain the pdated estimates for the fied parameters sing generalized least sqares. We note that the epected vales of the sfficient statistics can be obtained sing the general reslt for a random parameter vector θ. T T E( θθ = cov( θ [ E( θ][ E( θ] (3.6 The prediction is known as the E (epectation step of the algorithm and the comptations in (3.5 and (3.6 the M (maimization step. Given starting vales, based pon OLS, these comptations are iterated ntil convergence is obtained. Convenient comptational formlae for compting these qantities at each iteration can be fond in Bryk & Radenbsh (99. Markov Chain Monte Carlo estimation The Gibbs Sampling Markov Chain Monte Carlo algorithms eploit the properties of Markov chains where the probability of an event is conditionally dependent on a previos state. The procedre is iterative and at each stage from the fll mltivariate distribtion the distribtion of each component conditional on the remaining components is compted and sed to generate a random variable. The components may be variates, regression coefficients, covariance matrices etc. After a sitable nmber of iterations, we obtain a sample of vales from the distribtion of any component, which we can then se to derive any desired characteristic sch as the 44

13 mean, covariance matri, etc. The most common procedre is that of Gibbs Sampling and Gilks et al. (993 provide a comprehensive discssion with applications and an application to a -level logit model is given by Zeger & Karim (99. It allows the fitting of Bayesian models where prior distribtions for the parameters are specified. We otline a Gibbs Sampling procedre for a -level model. We write: ( ( Y = Xβ Z Z e (3.7 ( k We first consider the distribtion β, Y where k refers to the k-th iteration. Given ( k (, Z is st an offset so that we can regress y on to estimate $ ( k var( $ ( k β and β. We can then select a random vector from this distribtion, assmed to be mltivariate normal ( $ ( k ( $ ( k β,var β. ( We now consider the distribtion of Ω k. We have (with a non-informative prior that the (posterior distribtion of Ω covariance matri is a Wishart distribtion with parameter (i.e. J ( k ( k ( k S = with d = J q d.f. (3.8 = T where J is the nmber of level nits and q is the nmber of random coefficients. A simple way of generating sch a Wishart distribtion is to generate d mltivariate normal vectors from N(, S ( k and form their SSP matri. This provides Ω $ ( k. Finally we consider the distribtion β, Ω, Y. These are the sal level residals, for which we have standard epressions for their epected vales and covariance matri. We note that for a -level model (bt not within a three level model these are ( k block-independent. Assming Normality we can now generate a set of and this completes an iterative cycle. There are some particlar comptational details to be noted. For eample 'reection sampling' at each cycle can be sed and we can do several cycles for Ω, for each β since the former tend to have higher atocorrelations across cycles. The procedre can be applied to any eisting models, e.g. logit models, where the conditional distribtional assmptions are eplicit. Gibbs Sampling tends to be comptationally demanding, with hndreds if not thosands of iterations reqired and this can be particlarly brdensome when several different models are being eplored 45

14 for their fit to the data. On the other hand, this approach has the advantage, in small samples, that it takes accont of the ncertainty associated with the estimates of the random parameters and can provide eact measres of ncertainty. The maimm likelihood methods tend to overestimate precision becase they ignore this ncertainty. In small samples this will be important especially when obtaining 'posterior' estimates for residals, which will be discssed in the following section. Gibbs sampling approach is therefore sefl for small and moderate sized samples and when sed in connction with likelihood based EM or IGLS algorithms. Other estimation procedres A variation on IGLS is Epected Generalized Least Sqares (EGLS or the Gass-Newton method as it is mentioned by other athors (Kreft & Leew, 998. This focses interest on the fied part parameters and ses the estimate of V obtained after the first iteration merely to obtain a consistent estimator of the fied part coefficients withot frther iterations. A variant of this separates the level variance from V as a parameter to be estimated iteratively along with the fied part coefficients. Longford (987 developed a procedre based pon a 'Fisher scoring' algorithm which can be seen that it is formally eqivalent to IGLS. This algorithm can also incorporate certain etensions, for eample to handle discrete response data. We have already mentioned the fll Bayesian approach, which has become comptationally feasible with the development of 'Markov Chain Monte Carlo' (MCMC methods, especially Gibbs Sampling (Zeger & Karim, 99. An alternative to the fll Bayes estimation, known as 'Empirical Bayes, ignores the prior distribtions of the random parameters, treating them as known for prposes of inference. When Normality is assmed, these estimates are the same as IGLS or RIGLS. Another approach, which parallels all that was mentioned so far, is that of Generalized Estimating Eqations (GEE introdced by Liang & Zeger (986. The principal difference is that GEE obtains the estimate of V sing simple regression or 'moment' procedres based pon fnctions of the actal calclated raw residals. It is concerned principally with modeling the fied coefficients rather than eploring the strctre of the random component of the model. While the reslting coefficient estimates are consistent they are not flly efficient. In some circmstances, however, 46

15 GEE coefficient estimates may be preferable, since they will sally be qicker to obtain and they make weaker assmptions abot the strctre of V. The GEE procedre can be etended to handle most of the models dealt with more comple cases Estimating the residals In a single level model sch as (3. the sal estimate of the single residal term e i is st %y i the raw residal. In a mltilevel model, however, we shall generally have several residals at different levels. In this chapter we consider estimating the individal residals in all levels. Given the parameter estimates, consider predicting a specific residal, say in a - level variance components model. Specifically we reqire for each level nit $ = E( Y, $ β, Ω $ (3.9 We shall refer to these as estimated or predicted residals or, sing Bayesian terminology, as posterior residal estimates. If we ignore the sampling variation attached to the parameter estimates in (3.9 we have cov( ~ y, var( cov( ~ y, e var( ~ y = = (3.3a e = (3.3b = (3.3c We regard (3.9 as a linear regression of on the set of { y% } for the -th level nit and (3.3 defines the qantities reqired to estimate the regression coefficients and hence $. For the variance components model we obtain n e ~ = y ( n e ~ ~ e y = (3.3b ~ y ( ~ y / n i (3.3a = (3.3c 47

16 where n is the nmber of level nits in the -th level nit. The residal estimates are not, nconditionally, nbiased bt they are consistent. The factor mltiplying the mean ( %y of the raw residals for the -th nit is often referred to as a 'shrinkage factor' since it is always less than or eqal to one. As n increases this factor tends to one, and as the nmber of level nits in a level nit decreases the 'shrinkage estimator' of becomes closer to zero. In many applications the higher level residals are of interest in their own right and the increased shrinkage for a small level nit can be regarded as epressing the relative lack of information in the nit so that the best estimate places the predicted residal close to the overall poplation vale as given by the fied part. These residals therefore can have two roles. Their basic interpretation is as random variables with a distribtion whose parameter vales tell s abot the variation among the level nits, and which provide efficient estimates for the fied coefficients. A second interpretation is as individal estimates for each level nit where we se the assmption that they belong to a poplation of nits to predict their vales. In particlar, for nits which have only a few level nits, we can obtain more precise estimates than if we were to ignore the poplation membership assmption and se only the information from those nits. This becomes especially important for estimates of residals for random coefficients, where in the etreme case of only one level-one nit in a level-two nit we lack information to form an independent estimate. As in single level models we can se the estimated residals to help check on the assmptions of the model. The two particlar assmptions that can be stdied readily are the assmption of Normality and that the variances in the model are constant. Becase the variances of the residal estimates depends in general on the vales of the fied coefficients it is common to standardize the residals by dividing by the appropriate standard errors, which are referred as 'diagnostic' or 'nconditional' standard errors (Goldstein, 995. When the residals at higher levels are of interest in their own right, we need to be able to provide interval estimates and significance tests as well as point estimates for them or fnctions of them. For these prposes we reqire estimates of the standard errors (the so-called 'conditional' or 'comparative' standard errors of the estimated residals, where the sample estimate is viewed as a random realization from 48

17 repeated sampling of the same higher-level nits whose nknown tre vales are of interest. The level residals are generally not of interest in their own right bt are sed rather for model checking, having first been standardized sing the diagnostic standard errors. Checking the model assmptions in a mltilevel model are sed in an eactly analogos way as in simple regression models. In other words, we se plot of the standardized level residals against the fied part predicted vale to check the assmption of a constant level variance ( homoscedasticity and Normal score plots for level-one (and level-two residals to check the assmption of Normality Hypothesis testing and confidence intervals In this section we deal with large sample procedres for constrcting interval estimates for parameters or linear fnctions of parameters and for hypothesis testing. Hypothesis tests are sed sparingly in mltilevel analysis since the sal form of a nll hypothesis, that a parameter vale or a fnction of parameter vales is zero, is sally implasible and also relatively ninteresting. Moreover, with large enogh samples a nll hypothesis will almost certainly be reected. The eception to this is where we are interested in whether a difference is positive or negative, and this is discssed in the section on residals below. Confidence intervals emphasize the ncertainty srronding the parameter estimates and the importance of their sbstantive significance. Fied parameters We have already presented parameter estimates techniqes for the fied part parameters together with their standard errors. These are adeqate for hypothesis testing or confidence interval constrction separately for each parameter. In many cases, however, we are interested in combinations of parameters. For hypothesis testing, this most often arises for groped or categorized eplanatory variables where n grop effects are defined in terms of n dmmy variable contrasts and we wish simltaneosly to test whether these contrasts are zero. We may also be interested in providing a pair of confidence intervals for the parameter estimates. We proceed as follows: 49

18 Define a (r p contrast matri C. This is sed to form linearly independent fnctions of the p fied parameters in the model of the form f = Cβ, so that each row of C defines a particlar linear fnction. Parameters that are not involved have the corresponding elements set to zero. Sppose we wish to test the hypothesis that the coefficients of two variables each having two categories are ointly zero. We define C = f = β, β 3 and the general nll hypothesis is {} H : f = k, k = here We form R = ( f k T [ C( X V X C ] ( f T T k f = Cβ (3.3b (3.3a If the nll hypothesis is tre this is distribted as approimately χ with r degrees of T freedom. Note that the term ( XV$ X is the estimated covariance matri of the fied coefficients. If we find a statistically significant reslt we may wish to eplore which particlar linear combinations of the coefficients involved are significantly different from zero. The common instance of this is where we find that n grops differ and we wish to carry ot all possible pairwise comparisons. A simltaneos comparisons procedre which maintains the overall type I error at the specified level involves carrying ot the above procedre with either a sbset of the rows of C or a set of (less than r linearly independent contrasts. The vale of R obtained is then dged against the critical vales of the chi-sqared distribtion with r degrees of freedom. We can also obtain an α% confidence region for the parameters by setting R $ eqal to the α% tail region of the χ distribtion with r degrees of freedom in the epression 5

19 R $ ( f f $ T [ C( X T V $ X C T = ] ( f f $ (3.33 This yields a qadratic fnction of the estimated coefficients, giving an r-dimensional ellipsoidal region. In some sitations we may be interested in separate confidence intervals for all possible linear fnctions involving a sbset of q parameters or q linearly independent fnctions of the parameters, while maintaining a fied probability that all the intervals inclde the poplation vale of these fnctions of the parameters. As before, this may arise when we have an eplanatory variable with several categories and we are interested in intervals for sets of contrasts. For a ( α % interval write C i for the i-th row of C, then a simltaneos ( α % interval for C i β, for all C i is given by where ( C $ β d, C $ β d (3.34 i i i i T T d = [ C ( X V$ X C ] i i 5. i q,( χ α (3.35 where χ q,( α is the α% point of the χ q distribtion. We can also se the likelihood ratio test criterion for testing hypotheses abot the fied parameters, althogh generally the reslts will be similar. The difference arises becase the random parameter estimates sed in (3.3a and (3.3b are those obtained for the fll model rather than those nder the nll hypothesis assmption, althogh this modification can easily be made. We shall discss the likelihood ratio test in the net section dealing with the random parameters. Random parameters In very large samples it is possible to se the same procedres for hypothesis testing and confidence intervals as for the fied parameters. Generally, however, procedres based pon the likelihood statistic are preferable. To test a nll hypothesis H against an alternative H involving the fitting of additional parameters we form the log likelihood ratio or deviance statistic D = log ( λ / λ (3.36 e 5

20 where λ, λ are the likelihoods for the nll and alternative hypotheses and this is referred to tables of the chi-sqared distribtion with degrees of freedom eqal to the difference (q in the nmber of parameters fitted nder the two models. We can also se (3.36 as the basis for constrcting a ( α % confidence region for the additional parameters. If D is set to the vale of theα% point of the chi sqared distribtion with q degrees of freedom, then a region is constrcted to satisfy (3.36, sing a sitable search procedre. This is a comptationally intensive task, however, since all the parameter estimates are recompted for each search point. An alternative is to se the profile likelihood (McCllagh & Nelder, 989. In this case the likelihood is compted for a sitable region containing vales of the random parameters of interest, for fied vales of the remaining random parameters. Interval estimates can be provided also by bootstrap simlations. Residals In stdies of instittions (e.g. schools, hospitals etc effectiveness (Goldstein & Spiegelhalter, 996, one reqirement is sometimes to try to identify instittions with residals which are sbstantially different. From a significance testing standpoint, we will often be interested in the nll hypothesis that instittion (grop A has a smaller residal than instittion (grop B against the alternative that the residal for instittion (grop A is larger than that for instittion (grop B (ignoring the vanishingly small probability that they are eqal. In the case when a standard significance test accepts the alternative hypothesis (at a chosen level of some difference against the nll hypothesis of no difference, this is eqivalent to accepting one of the alternatives (A > B, A < B at the same level of significance and we shall se this interpretation. Where we can identify two particlar instittions (grops then it is straightforward to constrct a confidence interval for their difference or carry ot a significance test. Often, however, the reslts are made available to a nmber of individals, each of whom are interested in comparing their own instittions (e.g. schools of interest. This may occr, for eample where policy makers wish to select a few schools within a small geographical area for comparison, ot of a mch larger stdy. In the following discssion, we sppose that individals wish to compare only 5

21 pairs of instittions, althogh the procedre can be etended to mltiple comparisons of three or more residals. Frther details are given by Goldstein & Healy (994. When the sample size of a stdy is fairly large, we can assme that the estimated residals together with their comparative standard errors estimates are ncorrelated. First, we order the residals from smallest to largest. We constrct an interval abot each residal so that the criterion for dging statistical significance at the ( α% level for any pair of residals is whether their confidence intervals overlap. For eample, if we consider a pair of residals with a common standard error (se and assming Normality, the confidence interval width for dging a difference significant at the 5% level are given by ±39. ( se. The general procedre defines a set of confidence intervals for each residal i as $ ± c( se (3.37 i For each possible pair of intervals, (3.37 there is a significance level associated with the overlap criterion, and the vale c is determined so that the average, over all possible pairs is ( α %. A search procedre can be devised to determine c. When the ratios of the standard errors do not vary appreciably, say by not more than :, the vale.4 can be sed for c. As this ratio increases so does the vale of c. These kinds of residal analyses are sefl for conveying the inherent ncertainty associated with estimates for individal level (or higher nits, where the nmber of level nits per higher-level nit is not large. This ncertainty in trn places inherent limitations pon sch comparisons. i 3. Etensions of the -Level Linear Model What we have discssed so far refers to notations, techniqes and estimations for the two-level linear model, which is the most common case in the mltilevel analysis theory. However, in order to eamine more demanded applications presented in the net chapter, we need to present the logical etensions of the two-level linear model. In all cases discssed here, the etensions are straightforward and stem either from the hierarchy of the sbects or from the natre of the data that are being measred. The etensions discssed here are: The 3-Level linear Model Cross-Classification Models Models for Discrete response data The Proportions as responses case 53

22 Mltivariate Mltilevel Models The basic -level Mltivariate model Mltilevel Strctral Eqation Models Mltilevel Factor Analysis case 3.. The Three-Level Linear Model The most profond, maybe, etension of a -Level linear model comes when we add more levels of hierarchy in the model. We focs on the 3-level model since higher-level cases are rarely of importance in practice. Some eamples of 3-level hierarchical strctres are stdents (Level- nested within schools (Level- nested within prefectres (Level-3. Or in another point of view repeated visits (Level- of patients (Level- in heath provider nits (Level-3. In the simplest case the basic linear 3-level model can be written as follows: y = β v e (3.38 k k ( k k k where k is a vector of covariates and β a corresponding vector of parameter estimates. The vector of covariates incldes a constant together with eplanatory variables measred at any of the three levels. The error terms v k, considered are considered as random variables with mean zero and variances var( v k var( var( e = (3.39a k k v = (3.39b = (3.39c e k and e k are If we now introdce Z eplanatory variables in the random part of the model, in any of the three levels, we obtain the more general form of the 3-level model, as follows: y k = X k β q 3 h= v hk z (3 hk q h= hk z ( hk q h= e hk z ( hk (3.4 where k is again the vector of covariates, β the corresponding vector of parameter estimates, and z, z (3 hk ( hk and ( z hk the eplanatory variables of the random part of the 3 rd, nd and st level of hierarchy, respectively. Althogh sch models seem more complicated and demanding than the twolevel models, the comptations, estimation techniqes and algorithms are totally analogos to the methods described before for the two-levels case. 54

23 3.. Cross-Classified Models So far we have considered only data where the nits have a prely hierarchical or nested strctre. In many cases, however, a nit may be classified along more than one dimension. An eample is stdents classified both by the school they attend and by the neighborhood where they live. This is diagrammatically represented as follows for three schools and for neighborhoods with between one and si stdents per school/neighborhood cell. The cross classification is at level with stdents at level. Table 3.: A random cross-classification at level School School School 3 Neighborhood Neighborhood Neighborhood 3 Neighborhood 4 Another eample is in a repeated measres stdy where children are measred by different raters at different occasions. If each child has its own set of raters not shared with other children then the cross classification is at level, occasions by raters, nested within children at level. We note that, by definition, a level cross classification has only one nit per cell. These basic cross-classifications occr commonly when a simple hierarchical strctre breaks down in practice. Consider, for eample, a repeated measres design, which follows a sample of stdents over time, say once a year, within a set of classes for a single school. If stdents change classes dring the corse, that is a cross classification at level for classes by stdents. If we now inclde schools these will be classified as level 3 nits, bt if stdents also change schools dring the corse of the stdy then we obtain a level 3 cross classification of stdents by schools with classes nested at level within schools and occasions as the level nits. The stdents have moved from being crossed with classes to being crossed with schools. Note that since stdents are crossed at level 3 with schools they are also atomatically crossed with any nits nested within schools and we do not need separately to specify the crossing of classes with stdents. 55

24 Sch designs will occr also in panel or longitdinal stdies of individals who move from one locality to another, or workers who change their place of employment. Other eamples of sch designs occr in panel stdies of hoseholds where, over time, some hoseholds split p and form new hoseholds. The total set of all hoseholds is crossed with individal at level with occasion at level. The hoseholds, which remain intact for more than one occasion, provide the information for estimating level variation. In health stdies cross-classification occrs natrally in many cases. Consider for eample the case where patients may be classified both by the hospitals they visit and by the clinicians the freqent, so that individals within one hospital clster are not groped in the same way nder clinicians. This type of cross-classification does not occr when clinicians operate within a single medical care, bt this is not always the case. This kind of cross-classification is illstrated diagramatically in the following figre (Rice & Jones, 997: Figre 3.5: Patients within Cross-classified clinicians and Provider Units In another eample, patients may receive care from more that one medical centre dring the year. This arrangement forms a mltiple or cross-nit membership model, a special case of cross-classification (Carey,. 56

25 We now set ot the strctre of the basic models described above and then go on to consider etensions and special cases of interest. A basic cross-classified model We consider first the simple model of Table 3. with variance components at level and a single variance term at level. We shall refer to the two classifications at level sing the sbscripts, and in general parentheses will grop classifications at the same level. We write the model as y = X β e (3.4 i( i( i( The covariance strctre at level can be written in the following form var( y cov( y cov( y y i( / i ( / y i( / / i ( cov( y y = (3.4a = (3.4b = = (3.4c i( i( / i ( Note that if there is no more than one nit per cell, then model (3.4 is still valid and can be sed to specify a level cross classification. Ths the level- variance is the sm of the separate classification variances, the covariance for two level nits in the same classification is eqal to the variance for that classification and the covariance for two level nits, which do not share either classification, is zero. If we have a model where random coefficients are inclded for either or both classifications, then analogos strctres are obtained. We can also add frther ways of classification with obvios etensions to the covariance strctre. We can now show how cross-classified models can be specified and estimated efficiently sing a prely hierarchical formlation, inclding random cross-classified strctres. We illstrate the procedre sing a -level model with crossing at level. The -level cross-classified model, sing the same notations as in previos chapters for the basic model, can be written q y = X z z e i( i( h h h h i( h= h= q β (

26 Parentheses grop the ways of classification at each level. We have two sets of eplanatory variables, type and type, for the random components defined by the colmns of Z ( n p q, Z ( n p q where p, p are respectively the nmber of categories of each classification, i.e. where Z = { z } (3.44a h z h = z him if = m, for m th typelevel nit, otherwise and where Z = z } (3.44b { h zh = zhim if = m, for m th type level nit, otherwise These variables are dmmy variables where for each level nit of type we have q random coefficients with covariance matri Ω ( and likewise for the type nits. To simplify the eposition we restrict orselves to the variance component case where we have ~~ Ω =, Ω = (3.45a ( ( ( ( T T T E YY V Z( ( I ( p Z Z ( (I ( p Z ( = (3.45b It is clear that the second term in (3.45b can be written as T ( ( p ( Z ( I Z J J T = (3.46 where J is a (n vector of ones. The third term is of the general form Z Ω Z T, namely a level 3 contribtion where in this case there is only a single level 3 nit and with no covariances between the random coefficients of the Z h and with the variance terms constrained to be eqal to a single vale, (. More generally we can specify a level cross classified variance components model by modeling one of the classifications as a standard hierarchical component and the second as a set of dmmy eplanatory variables, one for each category, with the random coefficients ncorrelated and with variances constrained to be eqal. We can smmarize the procedre sing the simple model of (3.4. We specify one of the classifications, most efficiently the one with the larger nmber of nits, as a standard 58

27 hierarchical level classification. For the other classification we define a dmmy (, variable for each nit, which is one if the observation belongs to that nit and zero if not. Then we specify that each of these dmmy variables has a coefficient random at level 3 and in addition constrain the reslting set of level 3 variances to be eqal. The variance estimate obtained is that reqired for this classification and the level- variance for the other classification is the one we reqire for that. To etend this to frther ways of classification we add levels. Ths, for a three way cross classification at level we choose one classification, typically that with the largest nmber of categories, to model in standard hierarchical fashion at level, the second to model with coefficients random at level 3 as above and the third to model in a similar fashion with coefficients random at level 4. So we can obtain the third variance by defining a similar set of dmmy variables with coefficients varying at level 4 and variances constrained to be eqal. This procedre generalizes straightforwardly to sets of several random coefficients for each classification, with dmmy variables defined as the prodcts of the basic (, dmmy variables sed in the variance components case and with corresponding variances and covariances constrained to be eqal within classifications. In general a p-way cross classification at any level can be modeled by inserting sets of random variables at the net p- higher levels. Ths in a -level model with two crossed classifications at level we wold obtain a three level model with the original level at level 3 and the level cross classifications occpying levels and. Interactions in cross-classifications If the second (type- classification has frther eplanatory variables with random coefficients as in (3.43 then we form etended dmmy variable interactions as the prodct of the basic dmmy variables and the frther eplanatory variables with random coefficients, so that these coefficients have variances and covariances within the same type- level- nit bt not across nits. In addition the corresponding variances and covariances are constrained to be eqal. We illstrate this case sing the simple model with variance components at level- and a single variance term at level- (3.4. Consider the following etension of eqation (3.4 y = X e β (3.47 i( i( ( i( 59

28 We have now added an interaction term to the model which was previosly an additive one for the two variances. The sal specification for sch a random interaction term is that it has simple variance ( across all the level cells (Searle et al, 99. To fit sch a model we wold define each cell of the cross classification as a level nit with a between cell variance (, a single level 3 nit with a variance and a single level 4 nit with a variance. The adeqacy of sch a model can be tested against an additive model sing a likelihood ratio test criterion. Etensions to this model are possible by adding random coefficients for the interaction component, st as random coefficients can be added to the additive components. Level cross classifications Some interesting models occr when nits are basically cross-classified at level. By definition we have a design with only one nit per cell, and we can also have a level cross classification which is formally eqivalent to a level crossclassification where there is st one nit per cell. This case shold be distingished from the case where a level cross classification happens to prodce no more than level nit in a cell as a reslt of sampling, so that the confonding occrs by chance rather than by design. A -level variance components model with a cross classification at level can be written as y = X e e e β (3.48 ( ii ( ii i i ( ii where for level we se a straightforward etension of the notation for a level cross-classification. The term e ( i, i is analogos to the interaction term in (3.47. To specify this model we wold define the as random at level 4, the e, e as random at levels 3 and, each with a single nit and the interaction term random across the cells of the cross classification at level, within the original level nits. Sppose now that we were able to etend the design by replicating measrements for each cell of the level cross-classification. Then (3.48 wold refer to a 3-level model with replications as level nits, and which cold be written as follows where the sbscript h denotes replications y = X e e e hii ( hii ( hii ( β (3.49 6