4.3 Poisson Regression

Transcription

1 of teratvely reweghted least squares regressons (the IRLS algorthm). We do wthout gvng further detals, but nstead focus on the practcal applcaton. > glm(survval~log(weght)+age, famly="bnomal", data=baby) (Intercept) log(weght) age Ths s only a part of the output, but for the moment the most nterestng one, namely the estmated coeffcents ˆ ˆ 0, 1 and 2 ˆ. 4.3 Posson Regresson The method of Posson Regresson s the formally correct way of dealng wth response varables that are counts. Ths s a strong statement, gven the fact that throughout the dscusson on smple lnear regresson, ths scrptum uses the count varable number of passengers as a response. Whle beng contradctory to some extent, t s tolerable: the counts are all large (n the mllons) and ther range s relatvely small. In such a stuaton, we can proft from the fact that the approxmaton of the Posson to the Gaussan dstrbuton works well,.e. there s lttle dfference between the two. However, there are examples where the use of Posson Regresson s a must: f the sze of the populaton s unknown and the counts are small. f the sze of the populaton s large and hard to come by, and the probablty of an event, and thus the expected counts are small. A typcal example for the latter case s modelng the ncdence of rare forms of cancer n a gven geographcal area. For llustratng the former case, we wll consder the followng example Example: Tortose Speces on Galapagos For 30 of Galapagos Islands we have the response varable Speces,.e. the number of speces of tortose that are present, plus fve geographc predctors. These are the area of the sland, the hghest elevaton, the dstance to the nearest sland, the dstance to Santa Cruz and the area of the adjacent sland. > lbrary(faraway); data(gala); head(gala[,-2]) Speces Area Elevaton Nearest Scruz Adjacent Bartolome Caldwell Champon Coamano Daphne.Major Page 48

2 Fttng a multple lnear regresson wthout dong any transformatons yelds very poor results: there s a bas, the error dstrbuton s long-taled and has nonconstant varance. Addtonally, there are some strong leverage ponts. We leave generatng these dagnostc plots as an exercse. The frst-ad transformatons from secton suggest that all predctors, whch can only take postve values and are strongly skewed to the rght, need to be log-transformed. Snce Scruz has a zero entry, we add ts smallest postve value. The response, whch s a count, s due for takng the square root. The output s: > ft02 <- lm(sqrt(speces) ~ log(area) + log(elevaton) + log(nearest) + I(log(Scruz+0.4)) + log(adjacent), data=gala[,-2]) > summary(ft02) Estmate Std. Error t value Pr(> t ) (Intercept) log(area) e-05 *** log(elevaton) log(nearest) I(log(Scruz + 0.4)) log(adjacent) Resdual standard error: on 24 degrees of freedom Multple R-squared: , Adjusted R-squared: F-statstc: on 5 and 24 DF, p-value: 8.176e-09 Resduals vs Ftted Normal Q-Q Resduals Gardner1 SanCrstobal Standardzed resduals Gardner1 SanCrstobal Ftted values Theoretcal Quantles Standardzed resduals Gardner1 Scale-Locaton SanCrstobal Standardzed resduals Resduals vs Leverage Gardner1 SantaCruz Cook's dstance Ftted values Leverage Page 49

3 Whle error dstrbuton/varance are acceptable (not perfect though!) and there are no leverage ponts anymore, there s a bas. The square root transformaton on the response s somewhat too strong. A better fttng relaton can be obtaned by takng a log-transformaton on Speces nstead of the square root. Better yet s to deal correctly wth the stuaton: due to low counts n the response, the Gaussan approxmaton s questonable. Thus, t s better to use Posson Regresson Model and Estmaton We have count responses Y for whch we, gven the predctors, assume a Posson dstrbuton wth parameter,.e. Y X ~ Pos( ). Our goal s to relate the parameter to the predctors, and because can take postve values only, we wll employ the log as a lnk functon: log( ) x... x p p Because the condtonal expectaton s EY [ X], we agan have the prevously recognzed stuaton that a lnk functons opens the door to modelng the expected value of the condtonal dstrbuton of Y by the lnear predctor. For estmatng the coeffcents,..., 0 p, we agan employ the MLE prncple. By assumng ndependence of the cases, the lkelhood functon can be wrtten as the product of the margnal dstrbutons: 1 1 n n y n n 1 1 y! PY ( y,..., Y y X) PY ( y X) e The parameters and predctors enter the above equaton by replacng wth exp( 0 1x1... pxp). The goal s to maxmze the lkelhood. In the multplcatve form, ths s nconvenent, thus we agan employ the trck of changng to the log-lkelhood: n 1 l( ) y log( ) log( y!) For fndng the optmum, we take partal dervatves wth respect to 0,..., p. As usual for GLMs, ths results n a non-lnear equaton system. There s no closed form soluton and we have to resort to the teratvely reweghted least squares (IRLS) approach for an approxmaton. Ths s already mplemented n R, and we can convenently ft the Posson Regresson model to the Galapagos data: > ft <- glm(speces ~ log(area) + log(elevaton) + log(nearest) + I(log(Scruz+0.4)) + log(adjacent), data=gala, famly=posson) Agan, be aware of the fact that estmatng the coeffcents s based on numercal optmzaton. Should a warnng be gven that the algorthm dd not converge, t s for a reason and to be taken serously. The summary output s as follows: Page 50

4 > summary(ft) Estmate Std. Error z value Pr(> z ) (Intercept) < 2e-16 *** log(area) < 2e-16 *** log(elevaton) log(nearest) ** I(log(Scruz + 0.4)) ** log(adjacent) < 2e-16 *** --- Null devance: on 29 degrees of freedom Resdual devance: on 24 degrees of freedom AIC: Dagnostcs and Inference As for Bnomal Regresson, there s a quck check for model adequacy: f the resdual devance s far n excess of the degrees of freedom n the model, there s too much varaton n the response. More precsely, we have that: n y D2 y log ( y ˆ ) 1 2 n ( p 1) ˆ In our example, the resdual devance s on just 24 degrees of freedom. Ths s far n excess of the degrees of freedom. We can compute the p-value for the null hypothess that the ftted model s correct: > pchsq(devance(ft), df.resdual(ft), lower=false) [1] e-61 The p-value s very small, ndcatng that we have an ll-fttng model f a Posson dstrbuton for the response s correct. Our next job s to recognze why and where the ft s poor. Ths s done usng some dagnostc vsualzaton. We start wth the Tukey-Anscombe plot, where ether the devance resduals or the Pearson resduals are plotted vs. the lnear predctor. We go n lne wth R and use the latter, whch are defned as: ( y ˆ ) P ˆ The Pearson resduals P approxmately follow a standard normal dstrbuton. Ths suggests that cases wth P 2 are extraordnary,.e. show bgger resduals than the Posson model suggests. > xx <- predct(ft, type="lnk") > yy <- resd(ft, type="pearson") > plot(xx, yy, man= Tukey-Anscombe Plot... ) Page 51

5 > smoo <- loess.smooth(xx, yy) > ablne(h=0, col="grey") > lnes(smoo, col="red") Tukey-Anscombe Plot for Galapagos Tortose Pearson Resduals Lnear Predctor We observe that there s hardly a bas, thus the functonal form mght be correct. However, there most of the resduals are larger than the Posson dstrbuton suggests. Ths s most lkely due to the fact that our predctors are too smple: we just take dstance and area of the nearest sland nto account, whch does not reflect clusters of slands n close vcnty well. In the present stuaton, where the predctor-response relaton s correct, but the varance assumpton of the Posson dstrbuton s broken, the pont estmates for ˆ ˆ 0,..., p and thus ˆ are unbased, but the standard errors wll be wrong. Ths makes the model stll sutable for predcton, though the nference results are n queston,.e. we cannot say whch varables are statstcally sgnfcant, because the p-values from the summary output are flawed. Because the Posson dstrbuton has only one sngle parameter, t s not very flexble for emprcal fttng purposes. Ths can be cured by estmatng the dsperson parameter, whch s n turn used for generatng better standard errors. The estmate s defned as: ˆ ˆ ˆ n( p1) 2 ( y ) / Ths s the sum of squared Pearson resduals, dvded by the degrees of freedom n ths model. In R, the command s: > sum(resd(ft, type="pearson")^2)/df.resdual(ft) [1] Page 52

6 An alternatve estmate for the dsperson parameter would be to take the quotent of the resdual devance and the degrees of freedom. Indeed, the result s smlar wth ˆ 15. > summary(ft, dsperson= ) Estmate Std. Error z value Pr(> z ) (Intercept) ** log(area) e-06 *** log(elevaton) log(nearest) I(log(Scruz + 0.4)) log(adjacent) ** --- Dsperson parameter for posson famly: Null devance: on 29 degrees of freedom Resdual devance: on 24 degrees of freedom AIC: Note that the estmaton of dsperson and regresson parameters are ndependent, thus modfyng the dsperson parameter does not change the coeffcents. In our example, some of the predctors now turn out to be non-sgnfcant. Also, the p- values are now not too dfferent to the ones from the multple lnear regresson model. For the mathematcally nterested, please note that snce the Gaussan dstrbuton has two parameters, the dsperson parameter s naturally estmated n multple lnear regresson wth the resdual standard error. Page 53