Regression Methods for Survey Data

Transcription

1 Regression Methods for Survey Data Professor Ron Fricker! Naval Postgraduate School! Monterey, California! 3/26/13 Reading:! Lohr chapter 11! 1

2 Goals for this Lecture! Linear regression! Review of linear regression, including assumptions! Coding nominal independent variables! Running multiple regression in JMP and R! For complex sampling designs, must use R or other specialized software! Logistic regression! Useful for binary and ordinal dependent variables! Running logistic regression in JMP and R! For complex sampling designs, must use R or other specialized software! 3/26/13 2

3 Regression in Surveys! Useful for modeling responses to survey question(s) as function of external data and/or other survey data! Sometimes easier/more efficient then highdimensional multi-way tables! Useful for summarizing how changes in the independent variables (the Xs) affect the dependent variable (the Y ) 3/26/13 3

4 Simple Linear Regression! General expression for a simple linear regression model:! β 0 and are model parameters! ε Y i = β 0 + β 1 x i + ε i β 1 is the error or noise term! Can think of it as modeling the expected value of Y for a given or particular value of x:! ( ) = β 0 + β 1 x i E Y i x i So the result is a model of the mean of Y as a linear function of some independent variable! 3/26/13 4

5 Linear Regression Assumptions! Error terms often assumed independent 2 observations from a distribution! N σ Y N x i (0, ) 2 ~ ( β0 + β1 i, σ ) This encapsulates all the assumptions inherent in linear regression:! The error terms are independent and identically distributed! Dependent variable is normally distributed! Hence, it is not appropriate to apply this methodology when the Y is discrete! So, Y can t be based on closed-ended survey questions such as a 5-point Likert scale! 3/26/13 5

6 Applying Linear Models to Survey Data! As a minimum the Ys must be continuous and must be able to transform it to be symmetric! Combinations of discrete survey data may be approximately normally distributed! Factor analysis and principle components may be useful in this regard! Given some data, we will estimate the parameters with coefficients! E( Y x ) ŷ = ˆβ + ˆβ x i 0 1 i where ŷ is the predicted value of y 3/26/13 6

7 Linear Regression with Categorical Independent Variables! Because survey data often discrete, often have discrete independent variable in model! For example, how to put male and female categories in a regression equation?! Code them as indicator (dummy) variables! Two ways of making dummy variables:! Male = 1, female = 0!» Default in many programs!» Easy to code and interpret! Male = 1, female = -1!» Default in JMP for nominal variables!» Some consider results easier to interpret! Consider a model with only gender as the independent variable! 3/26/13 7

8 Example: Calculus Grade as a Function of Gender! Regression equation: females: y = x 0 males: y = x 1 0/1 coding Compares calc_grade to a baseline group (here, females) Regression equation: females: y = x1 males: y = x (-1) -1/1 coding Compares each group to overall average 3/26/13 8

9 How to Code k Levels! Two coding schemes: 0/1 and 1/0/-1! Use k-1 indicator variables! E.g., three level variable: a, b,, & c! 0/1: use one of the levels as a baseline! Ø Var_a = 1 if level=a, 0 otherwise! Ø Var_b = 1 if level=b, 0 otherwise! Ø Var_c exclude as redundant (baseline)! Example:! 3/26/13 9

10 How to Code k Levels (cont d)! 1/0/-1: use the mean as a baseline! Ø Variable[a] = 1 if variable=a, 0 if variable=b, -1 if variable=c! Ø Variable[b] = 1 if variable=b, 0 if variable=a, -1 if variable=c! Ø Variable[c] exclude as redundant! Example! 3/26/13 10

11 Fitting Simple Linear Regression Models! In JMP, can use either Analyze > Fit Y by X or Analyze > Fit Model! Fill in Y with (continuous) dependent variable! Put X in model by highlighting and then clicking X or Add! Click Run Model when done! In R, use the lm() function in the base package! Syntax lm(dep_var ~ indep_var, data=data.frame) Best to assign results to an object and then look at results using the summary() function! 3/26/13 11

12 Example: New Student Survey! Define a Y variable that represents the complete in-processing experience:! In-processing Total = sum(q2a-q2i)! 3/26/13 12

13 New Student Survey (continued)! Dependent variable looks roughly continuous and approximately normally distributed:! Sample Quantiles Normal Q-Q Plot Summary Statistics from JMP! Theoretical Quantiles Summary Statistics from R! > summary(new_student$ip_total) Min. 1st Qu. Median Mean 3rd Qu. Max. NA's /26/13 13

14 Example: Simple Linear Regression of Total Satisfaction on School in R! > model_results <- lm(ip_total ~ CurricNumber, data=new_student) > summary(model_results) Call: lm(formula = IP_total ~ CurricNumber, data = new_student) Residuals: Min 1Q Median 3Q Max Three 0/1 indicators for 4 schools! Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) <2e-16 *** CurricNumberGSEAS * CurricNumberGSOIS CurricNumberSIGS Signif. codes: 0 *** ** 0.01 * Residual standard error: on 151 degrees of freedom (16 observations deleted due to missingness) Multiple R-squared: , Adjusted R-squared: F-statistic: on 3 and 151 DF, p-value: Result: GSBPP students most satisfied with mean score of ! GSEAS least satisfied with mean score of =28.978! 3/26/13 14

15 From Simple to Multiple Regression! Simple linear regression: One Y variable and one x variable ( Y i = β 0 + β 1 x i + ε i )! Multiple regression: One Y variable and multiple x variables! Like simple regression, we re trying to model how Y depends on x! Only now we are building models where Y may depend on many xs! Y i = β 0 + β 1 x 1i + β 2 x 1i + + β k x ki + ε i 3/26/13 15

16 Multiple Regression in JMP (Assuming Simple Random Sampling)! In JMP, use Analyze > Fit Model to do multiple regression! Fill in Y with (continuous) dependent variable! Put Xs in model by highlighting and then clicking Add! Use Remove to take out Xs! Click Run Model when done! In R, same function and syntax as simple linear regression, just add more terms:! lm(dep_var ~ indep_var1 +indep_var2 + indep_var3, data=data.frame) 3/26/13 16

17 Example: Revisiting Satisfaction with In-processing (1)! GSEAS worst at in-processing? Or are CIVs and USAF least happy? Note this differs from R output because of different coding! But solution is the same: GSEAS = = ! 3/26/13 17

18 Satisfaction with In-processing (2)! Or are Singaporians unhappy? Making a new variable 3/26/13 18

19 Satisfaction with In-processing (3)! Final model?! Normal Quantile Plot 3/26/13 19

20 Equivalent Model in R (Assuming Simple Random Sampling)! > summary(model_results) Call: lm(formula = IP_total ~ Type_Student, data = new_student) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) *** Type_StudentOther FORNAT ** Type_StudentSingapore * Type_StudentUS Air Force Type_StudentUS Army ** Type_StudentUS Marine Corps * Type_StudentUS Navy ** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 148 degrees of freedom (16 observations deleted due to missingness) Multiple R-squared: , Adjusted R-squared: F-statistic: on 6 and 148 DF, p-value: /26/13 20

21 Now, Regression with Complex Sampling! Use the survey package and the svyglm() function! Don t specify family option for linear regression! As with other svy functions, must first specify the sampling design with the svydesign() function and then call it from the svyglm() function! Open question: When fitting regression models, is it necessary to account for sampling design?! Not always clear: It may be possible with some data to estimate relationships in the population without the design and/or weights! 3/26/13 21

22 Logistic Regression! Logistic regression! Response (Y) is binary representing event or not! Model estimates probability of event occurrence! p ln i 1 p i = β + β x + β x + + β x + ε 0 1 1i 2 2i k ki i In surveys, useful for modeling:! Probability respondent says yes (or no )! Can also dichotomize other questions! Probability respondent in a (binary) class! 3/26/13 22

23 Three Numbers for the Same Idea! Probability (p)! Example: Pr(Congress passes FY14 budget) = 1/3 = 0.333! Easily interpretable number between 0 and 1! Odds: p /(1-p)! Example: Odds of an FY14 budget are 1/2 = 0.50! Still interpretable, but not as easy as probability any number > 0! Log odds: ln(p/1-p)! + Very difficult to interpret any number from to! Log odds is often called logit! In spite of interpretation difficulty, the model is very useful (and the logit can be transformed to get the probabilities)! 3/26/13 23

24 Where Logistic Regression Fits! Independent or Predictor Variable! Continuous Categorical Dependent or Response! Continuous Categorical Linear regression! Logistic regression! Linear reg. w/ dummy variables! Logistic reg. w/ dummy variables! 3/26/13 24

25 Why Logistic Regression?! Some reasons:! Estimates of p bounded between 0 and 1! Resulting S curve fits many observed phenomenon! Model follows the same general principles as linear regression! In terms of surveys, discrete data very common! If a variable is not already binary, often relatively easy to convert into binary result (collapse appropriately across some categories)! 3/26/13 25

26 Estimating the Parameters! βs estimated via maximum likelihood! Given estimated βs, probability is calculated as! ˆp i = exp ( ˆβ 0 + ˆβ 1 x 1i + ˆβ 2 x 2i + + ˆβ k x ) ki 1+ exp ˆβ 0 + ˆβ 1 x 1i + ˆβ 2 x 2i + + ˆβ k x ki ( ) In JMP, after Fit Model, red triangle > Save Probability Formula! Creates a new column containing the estimated probabilities, one for each observation! In R, use the predict function on an object that contains the output of svyglm() 3/26/13 26

27 Fitting Logistic Regression Models (Assuming Simple Random Sampling)! In JMP, fit much like multiple regression: Analyze > Fit Model! Fill in Y with nominal binary dependent variable! Put Xs in model by highlighting and then clicking Add! Use Remove to take out Xs! Click Run Model when done! In R, use the svyglm function with the option family=quasibinomial() 3/26/13 27

28 Example: Logistic Regression in JMP for the New Student Survey Q1! Dichotomize Q1 into satisfied (4 or 5) and not satisfied (1, 2, or 3)! Model satisfied on Gender and Type Student! 3/26/13 28

29 Same Example in R! > new_student$satisfied_ind <- as.numeric(new_student$x1>3) > model_results <- glm(satisfied_ind ~ Sex + Type_Student, data=new_student, family=binomial) > summary(model_results) Call: glm(formula = satisfied_ind ~ Sex + Type_Student, family = binomial, data = new_student) Deviance Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error z value Pr(> z ) (Intercept) SexM * Type_StudentOther FORNAT Type_StudentSingapore Type_StudentUS Air Force Type_StudentUS Army Type_StudentUS Marine Corps Type_StudentUS Navy Signif. codes: 0 *** ** 0.01 * (Dispersion parameter for binomial family taken to be 1) Null deviance: on 161 degrees of freedom Residual deviance: on 154 degrees of freedom (9 observations deleted due to missingness) AIC: Number of Fisher Scoring iterations: 4 3/26/13 29

30 Interpreting the Output (1)! Exponentiating both sides of! p ln i 1 p i = β + β X + β X + + β X 0 1 1i 2 2i k ki gives! p i = exp( β ) 1 p 0 exp( β 1 X ) 1i exp( β k X ) ki i Note that the left side is the odds for the i th observation! A useful quantity, and one that is easy to associate the betas to when using a 0/1 coding scheme! 3/26/13 30

31 Interpreting the Output (2)! For example, in the baseline group all the indicators are 0, so ( ) exp β 0 Thus, is the baseline group odds here, civilian female: odds = exp( ) = 0.26 So, the odds a female civilian is satisfied are roughly 1 to 4 In terms of probability, p i = exp( β ) 1 p 0 i ˆp i = exp ˆβ 0 1+ exp ˆβ 0 ( ) ( ) = = /26/13 31

32 Interpreting the Output (3)! Other groups are excursions from the baseline using appropriate values of independent variables For example, consider Navy males: p i 1 p i = exp ( ) exp(1.3364) exp(1.9556) = 7.07 Odds male US Navy officer is satisfied are 7 to 1 In terms of probability, ˆp i = ( ) ( ) exp ˆβ 0 + ˆβ Male + ˆβ Navy 1+ exp ˆβ 0 + ˆβ Male + ˆβ Navy ( ) ( ) = exp = 1+ exp /26/13 32

33 Compare Model Output to Raw Data! Coefficients: Estimate Std. Error z value Pr(> z ) (Intercept) SexM * Type_StudentOther FORNAT Type_StudentSingapore Type_StudentUS Air Force Type_StudentUS Army Type_StudentUS Marine Corps Type_StudentUS Navy /26/13 33

34 Ordinal Logistic Regression! An extension of binary logistic regression when there are k > 2 dependent variable categories! For the ordered responses 1 j k 1, the model is! Pr(Y ln i j)! 1 Pr(Y i j) = α β X β X β X j 1 1i 2 2i k ki Note that the betas are constant across all the js; the only difference are the alpha intercept terms! This is a proportional odds model! In R, use polr() in the MASS library with SRS and svyolr() with complex designs! 3/26/13 34

35 An Example: New Student Survey Modeling All Q1 Likert Scale Levels! > model_results <- polr(as.factor(x1) ~ as.factor(type_student), data=new_student) > summary(model_results) Call: polr(formula = as.factor(x1) ~ as.factor(type_student), data = new_student) Coefficients: Value Std. Error t value as.factor(type_student)other FORNAT as.factor(type_student)singapore as.factor(type_student)us Air Force as.factor(type_student)us Army as.factor(type_student)us Marine Corps as.factor(type_student)us Navy Intercepts: Value Std. Error t value Residual Deviance: AIC: (9 observations deleted due to missingness) 3/26/13 35

36 Interpreting the Output (1)! More complicated to understand this type of model! Most relevant are the coefficients! Each is the estimate of how much the log-odds increases with a one unit change! Can calculate the probabilities as:! Pr Y i j ( ) ( ) ( ) = exp ˆα j ˆβ 1 x 1i ˆβ 2 x 2i ˆβ k x ki 1+ exp ˆα j ˆβ 1 x 1i ˆβ 2 x 2i ˆβ k x ki The minus signs are correct: this is the way the polr() function parameterizes the model! 3/26/13 36

37 Interpreting the Output (2)! So, for Navy students,! exp Similarly Pr Y i 2,, and! Pr( Y i 4) = Thus, the model estimates that!! Pr( Y i 1) = ( ) ( ) = ( ) = Pr( Y i 3) = exp Pr( Y i = j Naval Officer ) = 0.006, 0.041, 0.116, 0.650, 0.187, 3/26/13 37 j = 1 j = 2 j = 3 j = 4 j = 5

38 Interpreting the Output (3)! So, for Singaporean students,! exp Similarly Pr Y i 2,, and! Pr( Y i 4) = Thus, the model estimates that!! Pr( Y i 1) = ( ) ( ) = ( ) = Pr( Y i 3) = exp Pr( Y i = j Singapore Officer ) = 0.044, 0.232, 0.326, 0.369, 0.029, 3/26/13 38 j = 1 j = 2 j = 3 j = 4 j = 5

39 Using R with Complex Sampling! As we ve seen, key is the survey package by Thomas Lumley! See Other useful svy functions include svytable(), svyboxplot(), svyby(), svycdf(), svycoplot(), svycoxph() Good reference text:! Lumley, T. (2010). Complex Surveys: A Guide to Analysis using R. Wiley Series in Survey Methodology, John Wiley and Sons.! 3/26/13 39

40 Other Software for Analyzing Surveys With Complex Sampling Designs! Stata! SAS! SPSS! SUDAAN!! ü Not Excel or JMP! 3/26/13 40

41 What We Have Just Learned! Linear regression! Review of linear regression, including assumptions! Coding nominal independent variables! Running multiple regression in JMP and R! For complex sampling designs, must use R or other specialized software! Logistic regression! Useful for binary and ordinal dependent variables! Running logistic regression in JMP and R! For complex sampling designs, must use R or other specialized software! 3/26/13 41