Logistic Regression Analysis

Transcription

1 Logistic Regression Analysis Predicting whether an event will or will not occur, as well as identifying the variables useful in making the prediction, is important in most academic disciplines as well as in the "real" world. Why do some citizens go on holidays and others not? Why do some people use public transport and others not? Why do some businesses succeed, while others fail? When the dependent variable can have only two values, the assumptions necessary for hypothesis testing in regression analysis are necessarily violated. For example, it is unreasonable to assume the distribution of errors to be normal. Another difficulty with multiple regression analysis is that predicted values cannot be interpreted as probabilities. They are not constrained to fall in the interval between 0 and 1. 1

2 The Logistic Regression Model Estimation In logistic regression you directly estimate the probability of an event occuring. For the case of a single independent variable, the logistic regression model can be written as: Pr ob( event) = 1+ Where B 0 and B 1 are coefficients estimated from the data, X is the independent variable, and e is the base of the natural logarithm, approximately For more than one independent variable the model can be written as: 1 Pr ob( event) = 1 + e Z Where Z is the linear combination e ( 1 B 0 + B1x ) Z = B 0 + B 1 X 1 + B 2 X B p X p The probability of the event not occurring is estimated as: Prob (no event) = 1 - Prob (event) In linear regression, we estimate the parameters of the model using the least-squares method. That is, we select regression coefficients that result in the smallest 2

3 sums of squared distances between the observed and the predicted values of the dependent variable In logistic regression, the parameters of the model are estimated using the maximum-likelihood method. That is, the coefficients that make our observed results most "likely" are selected. Since the logistic regression model is linear, an iterative algorithm is necessary for parameter estimation. Testing Hypotheses about the Coefficients For large sample sizes, the test that a coefficient is 0 can be based on the Wald statistic, which has a chi-square distribution. When a variable has a single degree of freedom, the Wald statistic is just the square of the ratio of the coefficient to its standard error. Unfortunately, the Wald statistic has a very undesirable property. When the absolute value of the regression coefficient becomes large, the estimated standard error is too large. This produces a Wald statistic that is too small, leading you to fail to reject the null hypothesis that the coefficient is 0, when in fact you should. Therefore, whenever you have a large coefficient, you should not rely on the wald statistic for hypothesis testing. Instead, you should build a model with and without that variable and base your hypothesis test on the change in the log liklihood. 3

4 As in the case with multiple regression, the contribution of individual variables in logistic regression is difficult to determine. The contribution of each variable depends on the other variables in the model. This is a problem, particularly when independent variables are highly correlated. A statistic that is used to look at the partial correlation between the dependent variable and each of the independent variables is the R statistic, shown in the figure above. R can range from -1 to +1. A positive value indicates, that as the variable increases in value, so does the liklihood of the event occuring. If R is negative, the opposite is true. Small values for R indicate that the variable has a small partial contribution to the model. 4

5 SPSS-Procedures for Logistic Regression If your independent variables contain not only quantitative and/or dummy variables, but also categorical variables you have to define these variables with the help of the "Categorical" button: 5

6 If you want to display a histogram of the estimated probabilities and some additional statistics, click on the Options dialog box: 6