Stat 8053, Fall 2013: Multinomial Logistic Models

Transcription

1 Stat 8053, Fall 2013: Multinomial Logistic Models Here is the example on page 269 of Agresti on food preference of alligators: s is size class, g is sex of the alligator, l is name of the lake, and f is the response category, with 5 levels. The variable y is a count. loc <- url(" str(aa269 <- read.table(loc, header=true)) 'data.frame': 80 obs. of 5 variables: $ y: int $ s: Factor w/ 2 levels "<=2.3",">2.3": $ g: Factor w/ 2 levels "F","M": $ l: Factor w/ 4 levels "George","Hancock",..: $ f: Factor w/ 5 levels "Bird","Fish",..: ftable(xtabs(y ~ l + g + s + f, aa269)) f Bird Fish Invertebrate Other Reptile l g s George F <= > M <= > Hancock F <= >

2 M <= > Oklawaha F <= > M <= > Trafford F <= > M <= > In this table, the counts are generally very small, often zero. This means: (1) goodness-of-fit tests based on comparing G 2 to a χ 2 distribution can be wildly inaccurate; (2) differences in G 2 for comparing models are likely to be useful, but also somewhat inaccurate. In the log-linear model approach, we view the count y as the response variable, which has five unordered categories. Let s fit as follows: p0 <- glm(y ~ f + l*g*s, data=aa269, family=poisson) p1 <- update(p0, ~. + f:(l + g + s)) anova(p0, p1, test="chisq") Analysis of Deviance Table Model 1: y ~ f + l * g * s Model 2: y ~ f + l + g + s + l:g + l:s + g:s + f:l + f:g + f:s + l:g:s Resid. Df Resid. Dev Df Deviance Pr(>Chi) e-07 2

3 The model p0 is the baseline logistic model that y is completely independent of the predictors, while p1 is the first-order logistic model. We do a little model selection. add1(p1,update(formula(p1),~.+f:(l+g+s)^2)) Single term additions Model: y ~ f + l + g + s + l:g + l:s + g:s + f:l + f:g + f:s + l:g:s Df Deviance AIC <none> f:l:g f:l:s f:g:s drop1(p1) Single term deletions Model: y ~ f + l + g + s + l:g + l:s + g:s + f:l + f:g + f:s + l:g:s Df Deviance AIC <none> f:l f:g

4 f:s l:g:s Unlike step, drop1 and add1 do not order its rows according to the value of AIC. From the add1 output, the minimum AIC model adds no terms. Using drop1, we see that removing f:g gives a smaller AIC, and following Agresti we will use this model, and we ignore gender. This means we need to refit the model with all terms that include g removed. print(summary(p2 <- update(p1, ~(f+l+s)^2))$coef,digits=3) Estimate Std. Error z value Pr(> z ) (Intercept) ffish finvertebrate fother freptile lhancock loklawaha ltrafford s> ffish:lhancock finvertebrate:lhancock fother:lhancock freptile:lhancock ffish:loklawaha finvertebrate:loklawaha

5 fother:loklawaha freptile:loklawaha ffish:ltrafford finvertebrate:ltrafford fother:ltrafford freptile:ltrafford ffish:s> finvertebrate:s> fother:s> freptile:s> lhancock:s> loklawaha:s> ltrafford:s> The only relevant coefficients in the above table are those that are interactions with f, namely f:l and f:s. Multinom Many program include special software for the multinomial logistic. The key difference is that parameters are labeled using logsitic regression names, and special purpose graphics may be available. The function multinom in the nnet package will do this. The syntax for this command is a little funky: The response is f, the name of the response categorization, and the counts are specified using the weights argument. # using the multinom function library(nnet) 5

6 a1 <- multinom(f~l+g+s, weight=y, data=aa269) # weights: 35 (24 variable) initial value iter 10 value iter 20 value final value converged a2 <- update(a1,~.-g) # weights: 30 (20 variable) initial value iter 10 value iter 20 value final value converged anova(a1,a2) Model Resid. df Resid. Dev Test Df LR stat. Pr(Chi) 1 l + s NA NA NA 2 l + g + s vs The difference in deviance is the same as using the log-linear approach. The deviances themselves, and the df, are quite different: the log-linear approach uses grouped data, while multinom uses ungrouped data. 6

7 summary(a2,correlation=false) Call: multinom(formula = f ~ l + s, data = aa269, weights = y) Coefficients: (Intercept) lhancock loklawaha ltrafford s>2.3 Fish Invertebrate Other Reptile Std. Errors: (Intercept) lhancock loklawaha ltrafford s>2.3 Fish Invertebrate Other Reptile Residual Deviance: AIC: The coefficients and standard errors for the f:l and f:s interactions on p2 are the same as the coefficients for the main effects in a2. The summary of a problem like this one is so complex that looking simply at the coefficients does not give a coherent solution. Another helpful approach is to look at the fitted cell probabilities. These do not depend on using the baseline-logit model, but would be the same 7

8 for any parameterization that fits the equivalent model. The code below shows how to do this using multinom. sizes <- rep(levels(aa269$s),4) lakes <- rep(levels(aa269$l),c(2,2,2,2)) out <- predict(a2,data.frame(l=lakes,s=sizes),type="p") row.names(out) <- paste(lakes,sizes) print(out,digits=2) Bird Fish Invertebrate Other Reptile George <= George > Hancock <= Hancock > Oklawaha <= Oklawaha > Trafford <= Trafford > matplot(t(out),type="b",xlab="food category",ylab="probability") 8

9 2 Probability

10 For completeness, here is how to get the fitted cell probabilities using glm; it is much too hard. First, sum over g to get a new data frame, and update to fit the model to this data frame. Then, use predict to get the fitted counts for this marginal table. Finally, divide by marginal totals to get probabilities. subtable <- as.data.frame.table(xtabs(y~s+l+f,data=aa269),responsename="y") p3 <- update(p2,data=subtable) out1 <- predict(p3,subtable,type="response") out2 <- xtabs(out1 ~ with(subtable,s:l)+f,data=subtable) print(t(apply(out2,1,function(x) x/sum(x))), digits=2) with(subtable, s:l) Bird Fish Invertebrate Other Reptile <=2.3:George <=2.3:Hancock <=2.3:Oklawaha <=2.3:Trafford >2.3:George >2.3:Hancock >2.3:Oklawaha >2.3:Trafford