Hierarchical Linear Models. Hierarchical Linear Models. Much of this material already seen in Chapters 5 and 14. Hyperprior on K parameters α:

Transcription

1 Hierarchical Linear Models Hierarchical Linear Models Much of this material already seen in Chapters 5 and 14 Hierarchical linear models combine regression framework with hierarchical framework Unified approach to: random effects models mixed models Set-up: Hyperprior on K parameters α: α α 0, Σ α N(α 0, Σ α ) with α 0, Σ α knwown, often p(α) 1. Also need priors for Σ y, Σ β. Finally, might need to set priors for the hyperparameters in the priors for Σ y, Σ β. Typically assume known and fixed. Sampling model: y β,σ y N(Xβ,Σ y ) Often Σ y = σ I Prior dist. for J regression coefficients β α, Σ β N(X β α, Σ β ) Typically, X β =1, Σ β = σβi 1

2 Example: growth curves in rats Rats (cont d) From Gelfand et al., 1990, JASA. CIBA-GEIGY measured the growth of 30 rats weekly, for five weeks. Interest is in growth curve. Assume linear growth (rats are young) and let: y ij : weight of ith rat in jth week, ii =1,..., 30, j = 1,..., 5 x i =(8, 15,, 9, 36) days y ij = α i + β i (x ij x i )+e ij Each rat gets her own curve if α i and β i are random effects parameters. Likelihood: y ij N(µ i,σ ) with µ i = α i + β i (x ij x i ). Population distributions: α i N(α 0,σα) β i N(β 0,σβ) Priors: σ Inv χ (ν, σ0) α 0,β 0 N(0.01, 10000) σα,σ βsiminv χ Priors for σα,σ β can be as non-informative as possible by having very small degrees of freedom parameter. Same for prior for σ if desired. A more reasonable formulation is to model α i,β i as dependent in the population distribution. 3 4

3 Milk production of cows - Mixed model example Data on milk production from n cows. n j cows are daughters of bull j. There are J sires in dataset. Sires group the cows into J different genetic groups. Other covariates are herd and age of cow. Exchangeability: given sire, age and herd, cows are exchangeable. In classical statistics, herd and age are fixed effect, and sire is random effect. For us, all random, but we allocate flat priors to fixed parameters. Cows that are sired by same bull are more similar than those sired by different bulls: intraclass correlation induced by models with random effects. Cows (cont d) Mixed model: y ij = x iβ + s j + e ij with s j N(0,σs), e ij N(0,σ ) and (s, e) independent. x i =(herd, age) are herd and age effects, and (β,σs,σ ) are unknown. Likelihood: In matrix form: y ij N(x iβ,σ s + σ ) yn(xβ,σ I + σ szz ) with X : n p, Z : n q. Intra-class correlation: correlation between milk production of cows sired by same bull: ρ = σ s σ s + σ 5 6

4 View as J regression experiments Intra-class correlation Model for jth experiment is y j β j,σ j N(X j β j,σ j) with y j =(y 1j,y j,..., y nj j). Putting all regression models together into a single model: y 1 X β 1 y y = 0 X. = X =...0 β y J 0...X J β J Priors and hyperpriors, for example: Implied model is β j α, Σ β N(1α, Σ β ) p(α, Σ β ) 1 σj a, b Inv χ (a, b) y j α, σ j, Σ β N(X j α, σ ji + X j Σ β X j) Note: the hierarchy induces a correlation. Random effects introduce correlations Suppose that observations come from J groups or clusters so that y =(y 1,y,..., y J ), and y j =(y 1j,y j,..., y nj j as above. Model: y N(α, Σ y ) Let var(y ij )=η, and let cov(y ij,y kj )=ρη, for same group cov(y ij,y kl )=0, fordifferentgroup For ρ 0, now consider model y N(Xβ,σ I), with X an n J matrix of group indicators. If β N(α, σβi) and if we let η = σ + σβ,then ρ = σβ/(σ +σβ) and the two model formulations are equivalent. To see that models are equivalent, do p(y) = p(y, β)dβ. 7 8

5 Intra-class correlation Mixed effects models Positive intra-class correlations can be accommodated with a random effects model where class membership is reflected by indicators whose regression coefficients have the population distribution β N(1α, σβi) This is general formulation for several more general models Mixed effects models: p(β 1,..., β J1 ) 1 fixed effects p(β J1+1,..., β J ) N(1α, σβi) random effects A more general version of the mixed model has different random effects that generate different sets of intra-class correlations: p(β i ) 1, i =1,..., I b j1 α 1,σ1 N(1α 1,σ1I), j 1 =1,..., J 1. β jk α k,σk N(1α k,σki), ;;j k =1,..., J k The J components of β are divided into K clusters Exchangeability at the level of the observations is achieved by conditioning on the indicators that define the clusters or groups 9 10

6 Computation Hierarchical linear models have nice structure for computation. With conjugate prior, recall that: Observations are N Regression parameters are N Variance components (or variance matrices) are Inv χ (or Wishart). All conditional distributions are of standard form: For location parameters (regression coefficients, means of priors and hyperpriors), conditionals are normal For scale parameters, conditionals are also Inv χ, even if prior is improper. For one example, go back to earlier lecture on Gibbs sampling and example therein. 11