STAT5044: Regression and Anova

Transcription

1 STAT5044: Regression and Anova Inyoung Kim 1 / 18

2 Outline 1 Logistic regression for Binary data 2 Poisson regression for Count data 2 / 18

3 GLM Let Y denote a binary response variable. Each observation has one of two outcomes, denoted by 0 or 1, binomial for a single trial. The mean E(Y ) = P(Y = 1) We denote P(Y = 1) by π(x), reflecting its dependence on values x = (x 1,...,x p ) of predictors. The variance of Y is Var(Y ) = π(x)(1 π(x) the binomial variance for one trial. In introducing GLMs for binary data, for simplicity we use a single explanatory variable. 3 / 18

4 Linear Probability model For a binary response, the regression model π(x) = α + β x is called a linear probability model With independent observations it is a GLM with binomial random component and identity link function. 4 / 18

5 Logistic regression model Usually, binary data result from a nonlinear relationship between π(x) and x. In practice, nonlinear relationships between π(x) and x are often monotonic, with π(x) increasing continuously or π(x) decreasing continuously as x increase. The S-shaped curves are typical. The most importance curve with this shape has the model formula π(x) = exp(α + β x) 1 + exp(α + β x) This is the logistic regression model. As x, π(x) 0 when β < 0 and π(x) when β > 0 5 / 18

6 Logistic regression model The link function for which logistic regression is a GLM. For the odds are π(x) = exp(α + β x) 1 π(x) The log odds has the linear relationship π(x) log( 1 π(x) ) = α + β x Thus, the appropriate link is the log odds transformation, the logit Logistic regression models are GLMs with binomial random component and logit link function. Logistic regression models ar called logit models 6 / 18

7 Logistic regression model The logit is the natural parameter of the binomial distribution, so the logit link is its canonical link. Whereas π(x) must fall in the (0,1) ranges, the logit can be any real number. The real numbers are also the range for linear predictors (such as α + β x) that form the systematic component of a GLM. So this model does not have the structural problem that is true of the linear ML fit. 7 / 18

8 Binomial GLM for 2 2 Contingency Tables Among the simplest GLMSs for a binary response is the one having a single explanatory variable X that is also binary. Label is its values by 0 and 1. For a given link function, the GLM has the effect of X described by link[π(x)] = α + β x β = link[π(1)] link[π(0)] For the identity link, β = π(1) π(0) is the difference between proportions. For the log link, β = log[π(1)] log[π(0)] = log[π(1)/π(0)] is the log relative risk For the logit link is the log odds ratio. β = logit[π(1)] logit[pi(0)] = log π(1) 1 π(1) π(1)/(1 π(1)) = log π(0)/(1 π(0)) log π(0) 1 π(0) Measures of association for 2 2 tables are effect parameters in GLMs for binary data 8 / 18

9 Probit and Inverse CDF link functions A monotone regression curve such as the first one has the shape of a cumulative distribution function (cdf) for a continuous random variable This suggests a model for a binary response having form π(x) = F(x) for some cdf F. Using an entire class of location-scale cdf s, such as normal cdf s with their variety of means and variances, permits the curve π(x) = F(x) to have flexibility in the rate of increase and in the location where most of that increase occurs. Φ( ) denote the standard cdf of the class, such as the N(0,1) cdf. Using Φ but writing the model as π(x) = Φ(α + β x) ( ) provides the same flexibility. 9 / 18

10 Probit and Inverse CDF link functions Shape of different cdf s in the class occur as α and β vary. Replacing x by β x permits the curve to increase at a different rate than the standard cdf (or even to decrease if β < 0); varying α moves the curve to the left or right. When Φ is strictly increasing over the entire real line, its inverse function Φ 1 ( ) exists and (*) is equivalently Φ 1 [π(x)] = α + β x. For this class of cdf shapes, the link function for the GLM is Φ 1. The link function maps the (0,1) range of probabilities onto (, ), the range of linear predictors. The curve has the shape of a normal cdf when Φ is the standard normal cdf. This model is the called the probit model. This curve has similar appearance to the logistic regression curve. 10 / 18

11 GLM for counts The best known GLMs for count data assume a Poisson distribution for Y. Poisson GLMs can be applicable for modeling count or rate data for a single discrete response variable. 11 / 18

12 Poisson Loglinear Models The Poisson distribution has a positive mean µ. Although a GLM can model a positive mean using the identity link, it is more common to model the log of the mean. Like the linear predictor α + β x, the log mean can take any real value. The log mean is the natural parameter for the Poisson distribution and the log link is the canonical link for a Poisson GLM. A Poisson loglinear GLM assumes a Poisson distribution for Y and use the log link. 12 / 18

13 Poisson Loglinear Models The Poisson loglinear model with explanatory variable X is log µ = α + β x. For this model, the mean satisfies the exponential relationship µ = exp(α + β x) = e α (e β ) x 13 / 18

14 Overdispersion for Poisson GLMS We note that the count data often show greater variability than the Poisson allows. The variance s are much larger than the means, whereas Poisson distributions have identical mean and variance. The greater variability than predicted by the GLM random component reflects overdispersion A common cause of overdispersion is subject heterogeneity Overdispersion is not an issue in ordinary regression with normally distributed Y, because that distribution has a separate parameter (the variance) to describe variability. For binomial and Poisson distributions, however, the variance is a function of the mean. 14 / 18

15 Overdispersion for Poisson GLMS Overdispersion is common in the modeling count. When the model for the mean is correct but the true distribution is not Poisson, the ML estimates of model parameters are still consistent but standard errors are incorrect. We next introduce an extension of the Poisson GLM that has an extra parameter and accounts better for overdispersion. 15 / 18

16 Negative Binomial GLMs The negative binomial distribution has probability mass function Γ(y + k) f(y;k, µ) = Γ(k)Γ(y + 1) ( k µ + k )k (1 k µ + k )y,( ) y = 0,1,2,.. where k and µ are parameters. This distribution has E(Y ) = µ, var(y ) = µ + µ 2 /k The index k 1 is called a dispersion parameter. As k 1 0, var(y ) µ and the negative binomial distribution converges to the Poisson (Cameron and Trivedi, 1998). Usually, k 1 is unknown. Estimating it helps summarize the extent of overdispersion. 16 / 18

17 Negative Binomial GLMs For k fixed, one can express (**) in natural exponential family form f(y i ;θ i ) = a(θ i )b(y i )exp[y i Q(θ i )]. Then a model with negative binomial random component is a GLM. For simplicity, such models let k be the same constant for all observations but treat it as unknown. As in GLMs for binary data, a variety of link functions are possible. Most common is the log link, as in Poisson loglinear models, but sometimes the identity link is adequate. 17 / 18

18 Poisson Regression for Rates When events of a certain type occur over time, space, or some other index of size, it is usually more relevant to model the rate at which they occur than the number of them. For instance, a study of homicides in a given year for a sample of cities might model the homicide rate, defined for a city as its number of homicides that year divided by its population size. The model might describe how the rate depends on the city s unemployment rate, its residents median income, and the percentage of residents having completed high school. 18 / 18