Generalized linear models III Log-linear and related models

Size: px

Start display at page:

Download "Generalized linear models III Log-linear and related models"

Brent Singleton
5 years ago
Views:

1 Generalized linear models III Log-linear and related models Peter McCullagh Department of Statistics University of Chicago Polokwane, South Africa November 2013

2 Outline Log-linear models Binomial models Hypergeometric models Case-control studies Over-dispersion and beta-binomial models

3 Counted data Counted data: Y is the number of events observed First possibility: i is an individual/patient/plot/time interval Y i is the number of events observed for i Number of children of i; emergency-room visits by i,... Number of distinct plant species in plot i... i particular bus on Route 217 Y i is the number of passengers at a specific point i is a particular male fly Y i is number of occasions courting F rather than U Y t = #bankruptcies in year t In a Markov chain i = (r, s) is a pair of states Y i = #transitions r s i = (r, s) is a pair of journals Y i = #citations by r to s

4 Counted data contd Each event has a type or mark or label Y x is the number of events of type x Each citation has a mark x = (r, s) Y x = #marks of type x Each toothpaste purchase has a brand Y x = #units of brand x purchased Every traffic accident has a classification fatal/not; alcohol/not; insurance/not;... Y x is number of accidents of type x Same for crimes, web clicks,... x = (year, state): Y x = #hurricanes PET: Y x photon count at angle x

5 Poisson models Y x,t : number of events of type x observed in (0, t) values recorded for a range of types and intervals Poisson model: Y x,t Po(ρ x t) Independent for types x x Independent for non-overlapping intervals (0, s), (s, t) stationarity: ρ x is the rate for type x Log-linear models: several classification factors x = (r, s, u): log ρ x = α r + β s + γ u (w/o) interaction E(Y x,t ) = e αr e βs e γu t log ρ x = α rs + β st with interaction

6 Maximum likelihood for Poisson log-linear models Model: Y i Po(ρ i t i ) with independent components log µ = Xβ + log(t) X (Y ˆµ) = 0 (local max condition) ˆβ ˆβ 0 = (X Ŵ 0 X) 1 X (Y ˆµ 0 ) ˆµ = exp(x ˆβ 1 + log(t)) (one iteration) W = diag{µ 1,..., µ n } cov( ˆβ) = (X WX) 1 Implications of the condition X Y = X ˆµ Technical point: If n = 1, and y = 0: zero events observed in (0, t] then ˆµ = 0, ˆβ = ; technically no mle same thing can occur more generally

7 Over-dispersed counts Practical point: Often there are reasons to think that var(y i ) > µ i often but not always can check by computing σ 2 = X 2 /(n p) and referring to null distribution χ 2 n p/(n p) Causes of over-dispersion: unrecorded effects, (randomly varying intensity) superposition (double counting); individuals versus families individuals versus twins Remedies: accommodate using over-dispersion parameter negative binomial model Under-dispersion: Much less common, but can occur same event counted in several cells (types not disjoint)

8 Multinomial models Poisson to multinomial: Y 1,..., Y k indep Poisson, means µ 1,..., µ k Conditional distribution given Y. = m: π r = µ r /µ. e µ. µ y 1 1 µy k k y 1! y k! p(y) = p(y Y. ) p(y. ) = y.! y 1! y k! πy 1 1 πy k k e µ. µ y.. y.! Poisson log lik = multinomial log lik + Poisson log lik for sum Can fit a multinomial model artificially using a Poisson formulation Not necessary, but an option...

9 Binomial models ( m ) p(y; m) = π y (1 π) m y 0 y m y ( π ) ( ) m log p = y log + m log(1 π) + log 1 π y θ = log(π/(1 π)); π = e θ /(1 + e θ ); K (θ) = m log(1 π) = m log(1 + e θ ) Linear logistic models: θ η = Xβ Probit models: η Φ 1 (π) = Xβ C-log log model: η log( log(1 π)) = Xβ

10 Parameter interpretation: logistic models Linear logistic model: pr(y = 1 x) = π(x) = eβ 0+β 1 x 1 + e β 0+β 1 x odds(y = 1 x) = π(x) 1 π(x) = eβ 0+β 1 x η(x) = ( π(x) ) log odds(y = 1 x) = log 1 π(x) η(x) = β 0 + β 1 x β is the change in log odds per unit change in x e β is the multiplicative change in odds per unit change in x odds(y = 1 x) = e β odds(y = 1 x 1) Often easier to talk in terms of odds or risks

11 2 2 hypergeometric models Design in matched blocks: m b0 controls, m b1 treated individuals in block b success probabilities in block b: π b0 and π b1 all responses independent! Observed counts: Y 0 B(m 0, π b0 ), Y 1 B(m 1, π b1 ) Model: log(odds(success b, t) = α b + β t Model: odds(success treat) = e β odds(success cntrl) F S Tot control m 0 Y 0 Y 0 m 0 treated m 1 Y 1 Y 1 m 1 Conditional distn of Y 1 given Y 0 + Y 1 = s is hypergeometric ( )( ) m0 m1 pr(y 1 = y m 0, m 1, s) e βy s y y independent of block parameter α b Exponential family with canonical parameter β

12 Case-control studies: retrospective sampling x: exposure level 0 or 1 Y : healthy or diseased, 0 or 1; (rare diseases) pr(y = 1 x) = e α+βx /(1 + e α+βx ) Response-dependent sampling fractions: pr(z i = 1 x i, Y i = 0) = π 0 (for controls) pr(z i = 1 x i, Y i = 1) = π 1 (for cases) By Bayes s theorem pr(y = 1 Z = 1, x) = pr(z = 1 Y = 1, x) pr(y = 1 x) + pr(z = 1 Y = 0, x) pr(y = 0 x) = π 1 e α+βx /(π 0 + π 1 e α+βx ) logit π(y = 1 Z = 1, x) = α + log(π 1 /π 0 ) + βx same β, different intercept

13 beta-binomial models Example: antibody counts for 18 patients pid c1 c2 c3 c4 c5 c6 c7 c8 s1 s2 s3 s4 s5 s6 s7 s8 s9 s10 RHA Tot Dispersion: X 2 = 23.6 for controls and 14.3 for SLE patients. Lexis dispersion index: 37.9/16 = 2.4 beta-binomial distribution ( ) m α y β m y y (α + β) m (0 y m) where α n = α(α + 1) (α + n 1) is the ascending factorial. E(Y ) = mα/(α + β) = mπ var(y ) = mπ(1 π)(α + β + m)/(α + β + 1) beta-binomial = binomial = bernoulli if m = 1

14 Beta-binomial applied to antibody counts Parameter estimates (max likelihood) H 0 H A α c β c α s β s llik Binomial log likelihoods: for H 0 and for H a Conclusion: Signficant over-dispersion (LR = 7.1 on 1 df) but no difference between patients types S and C ˆπ = ˆα/(ˆα + ˆβ) = 0.71; se =??

Regression models, selection effects and sampling bias

Regression models, selection effects and sampling bias Peter McCullagh Department of Statistics University of Chicago Hotelling Lecture I UNC Chapel Hill Dec 1 2008 www.stat.uchicago.edu/~pmcc/reports/bias.pdf