Regression models. Generalized linear models in R. Normal regression models are not always appropriate. Generalized linear models. Examples.

Transcription

1 Regression models Generalized linear models in R Dr Peter K Dunn Department of Mathematics and Computing University of Southern Queensland ASC, July 00 The usual linear regression models assume data come from a Normal distribution with the mean related to predictors Generalized linear models (GLMs) assume data come from some distribution with a function of the mean related to predictors Model Randomness Structure Regression model Y N(µ, φ) µ =Xβ GLM Y P(µ, φ) g(µ) = Xβ Generalized linear models Generalized linear models have two main components 1 The model for the randomness: Y P(µ, φ) The model for the structure: g(µ) = Xβ We can choose from many distributions P We can choose from many link functions g(µ) in a separate decision (Using a transformation in regression approximately makes both decisions at once) Normal regression models are not always appropriate There are obvious occasions when a Normal distribution is inappropriate: Counts cannot have normal distributions: they are non-negative integers Proportions cannot have normal distributions: they are constrained between 0 and 1 Lots of continuous data are non-negative and have non-constant variance In all cases, the variance cannot be constant since a boundary on the responses exists s Counts may be modelled using a Poisson distribution Usually, use a log link Define µ = E[Y ] as the expected count The model is { Yi Poisson(µ i ) (random) log µ i =Xβ (systematic) The log link ensures µ = exp(x β) is always positive The log link means the effect of the covariates x j on µ is multiplicative not additive s Proportions may be modelled using a binomial distribution Often, use a logit link (to get a logistic regression model) Define µ = E[Y ] as the expected proportion The model is { Yi Binomial(µ i ) (random) logit(µ i ) = Xβ (systematic) log Y i Binomial(µ i ) ( µi 1 µ i ) =Xβ (random) (systematic) Basic fitting of glms in R Fit a regression model in R using lm( y ~ x1 + log( x ) + x3 ) To fit a glm, R must know the distribution and link function Fit a regression model in R using (for example) glm( y ~ x1 + log( x ) + x3, family=poisson( link="log" ) ) What distributions can I choose? gaussian: a Gaussian (Normal) distribution binomial: a binomial distribution for proportions poisson: a Poisson distribution for counts Gamma: a gamma distribution for positive continuous data inverse.gaussian: an inverse Gaussian distribution for positive continuous data

2 What link function can I choose? What link function can I choose? Link function gaussian binomial poisson indentity µ = η log log µ = η inverse 1/µ = η sqrt µ = η logit logit(µ) =η probit probit(µ) =η cauchit cauchit(µ) =η cloglog cloglog(µ) =η Link function gamma inverse.gaussian indentity µ = η log log µ = η inverse 1/µ = η 1/mu^ 1/µ = η In R... To fit a glm in R, we need to specify: The linear predictor: x1+x+log(x3) The distribution: family=poisson The link function: link="log" They work together like this: glm( y ~ x1 + x + log(x3), family=poisson(link = "log") ) Glms in R? Fitting glms is locally like fitting a standard regression model So most regression concepts have (approximate) analogies for glms For example, R allows the user to: fit glms (use glm) find important predictors (F -tests using anova; t-tests using summary) compute residuals (using resid; quantile residuals in package statmod strongly recommended: qresid) perform diagnostics (using plot, hatvalues cooks.distance, etc.) : Poisson 3 children Others < 1 (C = 1) (C = 0) SLE No SLE SLE No SLE (S = 1) (S = 0) (S = 1) (S = 0) Depres. (D = 1) 9 0 OK (D = 0) To fit the minimal model in R: dep.glm <- glm( Counts ~ C + S + D, To fit the full model R: dep.full <- glm( Counts ~ C * S * D, We assume all qualitative variables are declared as factors The data are counts, so use a poisson family (and default log link) Initially, use the linear predictor C + S + D What predictors are significant? Sequential test: > anova(dep.full, test = "Chisq") Analysis of Deviance Table Model: poisson, link: log Response: Counts Terms added sequentially (first to last) Df Deviance Resid. Df Resid. Dev P(> Chi ) NULL D e-7 S e-0 C e-70 D:S e-11 D:C S:C D:S:C e- 0.1 What predictors are significant? Post-fit test: > summary(dep.full) Call: glm(formula = Counts ~ D * S * C, family = poisson(link = log), data = dep) Deviance Residuals: [1] Coefficients: Estimate Std. Error z value Pr(> z ) (Intercept) < e-1 *** D e-1 *** S e-09 *** C < e-1 *** D1:S e-0 *** D1:C S1:C D1:S1:C Signif. codes: 0 *** ** 0.01 * (Dispersion parameter for poisson family taken to be 1) Null deviance: 7.173e+0 on 7 degrees of freedom Residual deviance:.13e- on 0 degrees of freedom AIC: 51. Number of Fisher Scoring iterations: 0

3 To fit one suggested model in R: dep.opt <- glm( Counts ~ C + S * D, Note that S * Dmeans S + Dand the interaction S : D Plots: Hat diagonals > plot(hatvalues(dep.opt), type = "h", lwd =, hatvalues(dep.opt) Plots: Cook s distance Plots: Q Q plots > plot(cooks.distance(dep.opt), type = "h", lwd =, > library(statmod) > qqnorm(qresid(dep.opt)) Normal QQ Plot cooks.distance(dep.opt) Sample Quantiles Typing plot( glm.object ) produces six plots, four by default: 1 Residuals r i vs fitted values ˆµ (default) ri vs ˆµ (default) 3 a Q Q plot (default) A plot of Cook s distance D i 5 A plot of r i vs h i with contours of equal D i (default) A plot of D i vs h i /(1 h i ), with contours of equal D i > par(mfrow = c(, )) > plot(dep.opt) > par(mfrow = c(1, 1)) Residuals Residuals vs Fitted Predicted values 3 0 Normal QQ ScaleLocation Residuals vs Leverage Cook's distance Predicted values Leverage > plot(dep.opt, which = 5) Residuals vs Leverage 0 Cook's distance Leverage glm(counts ~ D * S + C) Hours No. turbines No. fissures Prop. fissures Hours No. turbines No. fissures Prop. fissures The data are proportions: use binomial family

4 Three ways to fit binomial glms in R; here are two: Proportion of turbines with fissures td.glm <- glm( prop ~ Hours, weights=turbines, family=binomial(link=logit) ) td.glm <- glm( cbind(fissures, Turbines) ~ Hours, family=binomial(link=logit) ) Can use alternative links: td.glm <- glm( prop ~ Hours, weights=turbines, family=binomial(link=probit) ) Hours of use td.glm <- glm( prop ~ Hours, weights=turbines, family=binomial(link=cloglog) ) We use the default logit link The fitted model is: > summary(td.glm) Call: glm(formula = prop ~ Hours, family = binomial(link = logit), weights = Turbines) Deviance Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error z value Pr(> z ) (Intercept) <e-1 *** Hours <e-1 *** --- Signif. codes: 0 *** ** 0.01 * (Dispersion parameter for binomial family taken to be 1) > td.cf <- signif(coef(td.glm), 3) > td.cf (Intercept) Hours From R output, the fitted model is ( ) µi log = Hours 1 µ i where µ is the expected proportion of turbines with fissures Null deviance: on degrees of freedom Residual deviance:.331 on 9 degrees of freedom AIC: 9.0 Number of Fisher Scoring iterations: Plots: Hat diagonals Plots: Cook s distance > plot(hatvalues(td.glm), type = "h", lwd =, col = "blue") > plot(cooks.distance(td.glm), type = "h", lwd =, hatvalues(td.glm) cooks.distance(td.glm) Plots: Q Q plots > qqnorm(qresid(td.glm)) Sample Quantiles Normal QQ Plot F H K V Age C P C P C P C P

5 Plots: Number of cancers Rates Number of lung cancer patients is a count, so use a Poisson glm: glm( Cases ~ City + Age, But lung cancer rate probably more useful Expected cancer rate is E[Y i /T i ] = E[Y i ]/T i = µ/t i, where µ i is the expected number of cancers, Note T i is known and not random. Using a logarithmic link, model the cancer rate as log(µ i /T i ) = Xβ or Age group >7 Fredericia Horsens City Kolding Vejle log µ i = log T i +Xβ log T i is an offset: a component of the linear predictor with a known parameter value, here one. Plots: Number of cancers Plots: Rates of cancer Lung cancer rate Lung cancer rate Age group >7 Fredericia Horsens City Kolding Vejle Age group >7 Fredericia Horsens City Kolding Vejle Rates To model lung cancer rate, use a Poisson glm with an offset: lc.glm <- glm( Cases ~ offset( log(population)) + City + Age, Plots: Hat diagonals > plot(hatvalues(lc.glm), type = "h", lwd =, col = "blue") hatvalues(lc.glm) Plots: Cook s distance Plots: Q Q plots > plot(cooks.distance(lc.glm), type = "h", lwd =, > library(statmod) > qqnorm(qresid(lc.glm)) Normal QQ Plot cooks.distance(lc.glm) Sample Quantiles

6 Other models We haved looked at fitting glms to Proportions Counts Rates Can also fit glms to Positive continuous data (family=gamma or family=inverse.gaussian) Overdispersed counts (family=quasipoisson) Overdispersed proportions (family=quasibinomial) Positive continuous data with exact zeros (family=tweedie using package statmod)