Generalized Linear Models. stat 557 Heike Hofmann

Transcription

1 Generalized Linear Models stat 557 Heike Hofmann

2 Outline Intro to GLM Exponential Family Likelihood Equations GLM for Binomial Response

3 Generalized Linear Models Three components: random, systematic, link Random part: distribution of observations y Systematic component: linear combination of explanatory variables link function: link between random and systematic component

4 Exponential Family Two parameter family, φ > 0 density f(yi; θi, φ) = exp ((yiθi b(θi))/a(φ) + c(yi, φ)) θi is the natural parameter, φ is called dispersion parameter. E[Y] = b (θ) Var[Y] = b (θ) a(φ)

5 Normal Distribution f(x; µ, σ) = 1 exp 2πσ 2 (x µ)2 2σ 2 θx (θ/2) 2 f(x; θ, φ) = exp x 2 /φ 1/2 log(φπ) φ define θ = 2µ, φ = 2σ 2 ( )

6 Binomial Distribution = exp P (X = x) = n π x (1 π) n x x π P (X = x; θ = log xθ n log(1 + e θ ) + log 1 π, φ = 1) = n x

7 Poisson Distribution λ λx P (X = x; λ) =e x! P (X = x; θ = log λ, φ = 1) = exp xθ e θ log(x!)

8 Systematic Component Let X1, X2,..., Xp be explanatory variables systematic component: η i = p β j x ij j=1 with parameters ß1, ß2,..., ßp

9 Link Function g() is called the link function of the GLM, if g differentiable and monotonic, i.e. g -1 exists g(µ) = Xjßj with µ = E[Y] g is called the natural link, if g(µi) = θi

10 Likelihood Equations Want to find estimates for β: L(β) =... = i f (y i ; θ i, φ) Relationship between β and θ i : β is in η(β) η is in g(µ) µ is in b (θ) θ is in L(β) Chain rule: L i β j = L i θ i θ i µ i µ i η i η i β j

11 Likelihood Equations L β = i L i β j = i y i b (θ i ) Var(y i ) x ij µ i η i! =0

12 Binomial GLM Some probability link functions n i Y i B ni,π i E[Y i ]=π i, Var[Y i ]=π i (1 π i )/n i transformation logit probit cloglog cauchit link function g g(π i ) = log g(π i )=π i π i 1 π i logit link logit(p) identity link g(π i )=Φ 1 (π i ) probit link g(π i ) = log( log(1 π i )) log-log link

13 Binomial GLM likelihood equations for logit link: L β i = i n i (y i π i )x ij! =0

14 Binomial GLM For 2 x 2 table: πx = P(Y=1 X=x) Y=0 GLM: g(πx) = α + β x Y=1 β is effect in X, β = g(π1) - g(π0) X=0 X=1 π00 π01 π10 π11 link g is identity => β is difference of proportions log link => β is log relative risk logit link => β is log odds ratio

15 Moth Data Frozen dead moths of two colors are placed on trees at locations of increasing distance from Liverpool, England. This species of moth rests during the day on tree trunks and is active at night. Trees near Liverpool are darkened by smoke to a greater extent than those farther away in the Welsh countryside. At each location, the number of moths of each color that are placed and removed 24 hours later are recorded. One might expect that lighter moths are more likely to be removed near Liverpool and that darker moths are more likely to be removed farther away, as the color of the trees provides more camouflage when the color of the moth is closer.

16 Moth Data > head(moth) Xmorph distance placed removed location 1 light Sefton Park 2 dark Sefton Park 3 light Eastham Ferry 4 dark Eastham Ferry 5 light Hawarden 6 dark Hawarden qplot(distance, removed/placed, data=moth, size=i(3), colour=xmorph) + geom_smooth (method="lm") removed/placed Xmorph dark light distance

17 glm(formula = cbind(removed, placed) ~ Xmorph * distance, family = binomial(link = "identity"), data = moth) Deviance Residuals: Min 1Q Median 3Q Max Moth Data Coefficients: Estimate Std. Error z value Pr(> z ) (Intercept) e-10 *** Xmorphlight distance * Xmorphlight:distance * --- Signif. codes: 0 *** ** 0.01 * (Dispersion parameter for binomial family taken to be 1) Null deviance: on 13 degrees of freedom Residual deviance: on 10 degrees of freedom AIC: Number of Fisher Scoring iterations: 4

18 glm(formula = cbind(removed, placed) ~ Xmorph * distance, family = binomial(link = "logit"), data = moth) Deviance Residuals: Min 1Q Median 3Q Max Moth Data Coefficients: Estimate Std. Error z value Pr(> z ) (Intercept) e-13 *** Xmorphlight distance * Xmorphlight:distance * --- Signif. codes: 0 *** ** 0.01 * (Dispersion parameter for binomial family taken to be 1) Null deviance: on 13 degrees of freedom Residual deviance: on 10 degrees of freedom AIC: Number of Fisher Scoring iterations: 4

19 Next: Model-fitting Deviance Residuals / other diagnostics?