Correlation and regression. Correlation and regression analysis. Measures of association. Why bother? Positive linear relationship

Transcription

1 1 Correlation and regression analsis 12 Januar 2009 Monda, (C1058) Frank Haege Department of Politics and Public Administration Universit of Limerick Correlation and regression Toda Bivariate relationships Measures of association Tuesda Bivariate relationships Model fit and statistical inference Introduction to multivariate relationships Wednesda Multivariate analsis Thursda Model selection and regression diagnostics 3 4 Measures of association Measures of linear association between two quantitative variables Quantitative variables Interval scale of measurement Linear association Straight line Correlation Measures the strength of an association Bivariate regression Predicts the value of for a given value of Wh bother? Predicting outcomes Guess the value of the response variable from information about the values of the eplanator variable E.g. outcome of US Presidential elections Identifing causal relationships First step in establishing the eistence of a causal relationship consists of demonstrating covariation between response and eplanator variable E.g. relationship between education and income Positive linear relationship 5 Negative linear relationship 6 Strong relationship Weak relationship Strong relationship Weak relationship

2 7 8 No relationship Pearson correlation coefficient r Measures strength of association Onl valid for linear relationships Formula: Numerator determines direction and strength Denominator is onl a scaling factor 9 10 Calculation of r Calculation of r A C B D Observation in quadrant B or C: Contribution to the numerator is positive Observation in quadrant A or D: Contribution to the numerator is negative The more etreme an observation, the larger its contribution to the numerator Beware of outliers! Properties of r The values of r range from -1 to 1 r = -1 or r = 1 if all the observations fall eactl on a line with non-zero slope The larger the absolute value of r, the stronger the association r treats and smmetricall r is independent of the units of measurement of and Effect of variabilit on r Strong relationship r = 0.87 Weak relationship r = 0.48

3 13 14 Perfect relationships: r = 1 Effect of nonlinearit on r Small r does not impl the absence of other forms of relationship between variables! Clear curvilinear relationship, but r = Effect of nonlinearit on r High r does not impl the eistence of a linear relationship! 120 Clear curvilinear relationship, but r = 0.54 Eample: Mental health stud Response variable: Mental impairment (mental) Inde incorporating various dimensions of pschiatric smptoms (e.g., aniet or depression) Eplanator variables: Life events (life) Number and severit of major life events eperienced within the past 3 ears (e.g., death in famil, moving, getting a new job, jail sentence) Socioeconomic status (ses) Inde based on occupation, income, and education Summar of variables. summarize Scatterplot matri. graph matri mental life ses Variable Obs Mean Std. Dev. Min Ma mental life ses Mental impairment Life events Socioeconomic status

4 19 20 Calculation of correlations. pwcorr mental life ses mental life ses mental life ses Medium-sized correlations in the sample Can we predict mental health from life events and/or socioeconomic status? Can we conclude that the associations eist in the population? Bivariate linear regression Given a certain value on the eplanator variable, what value would we epect on the response variable? Assumes a linear relationship between the response and the eplanator variable Formula for a straight line: = α + β = response variable = eplanator variable α = intercept β = slope Linear function: = α + β α = 2 1 β = 0.5 = Eamples of linear functions = = = 3 = = Prediction equation Linear equation: = α + β α and β are unknown parameters that are estimated using sample data Prediction equation: ŷ = a + b The least squares estimates ield the straight line falling closest to the points in the scatterplot The parameter estimates are then inserted into the linear equation to predict -values at fied values of Eample: Mental health stud Question: What is the effect of life events on mental health? Mental impairment Scatterplot indicates that linearit assumption is appropriate Life events

5 25 26 Eample: Mental health stud. regress mental life Source SS df MS Number of obs = F( 1, 38) = 6.11 Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = mental Coef. Std. Err. t P> t [95% Conf. Interval] life _cons Prediction equation: ŷ = Predicted values Mental impairment Life events ŷ = Predicted value at = 10: (10) = 24.2 Predicted value at = 70: (70) = Prediction errors A comparison of actual to predicted -values checks the goodness of the prediction equation Prediction error is called residual The residual is the difference between the observed value and the predicted value of the response variable e = ( - ŷ) e is positive if > ŷ e is negative if < ŷ The smaller the absolute value of e, the better the prediction Prediction errors Mental impairment Life events ŷ = Prediction error at = 10: = -4.2 Prediction error at = 70: = Least squares estimates Prediction errors can be summarized b the sum of their squared values Sum of squared errors (SSE): ( ŷ) 2 If the prediction line falls close to the points in the scatterplot, the prediction errors are small Least squares estimates a and b specif the linear prediction equation that ields the smallest SSE A probabilistic model = α + β describes a deterministic model Each value of corresponds to a single value of We do not epect all subjects who have the same -value to have the same -value A probabilistic model is more realistic Allows for variabilit in at each value of α + β represents the mean of as a function of The regression function describes how the mean of the response variable changes according to the value of an eplanator variable

6 31 32 Linear regression model Linear model with error terms Linear regression model: = α + β + ε α and β are the regression coefficients The error term ε represents the deviation of from the conditional mean α + β Each observation has its own value for ε Prediction equation: In practice, we do not know the n values of ε ŷ = a + b and = a + b + e, then = ŷ + e, so that e = - ŷ estimates ε α + β + ε, (ε > 0) α + β ε ε α + β + ε, (ε < 0) At each value of, ε is assumed to be normall distributed with mean zero and standard deviation σ Estimating the standard deviation Standard deviation is estimated b measuring average variabilit about least squares line n 2 is the number of degrees of freedom (df) In general, df = n - p where p is the number of unknown parameters Software often lists the mean square error (MSE, MS in Stata), which is the ratio of SSE to df Eample: Mental health stud. regress mental life Source SS df MS Number of obs = F( 1, 38) = 6.11 Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = mental Coef. Std. Err. t P> t [95% Conf. Interval] life _cons SSE = , df = 38, MSE = s = ( /38) -2 = (26.35) -2 = Correlation vs. regression b is useful for prediction, but tells us onl the direction of the association, not its strength Regression parameter estimates can onl be compared when predictors have the same measurement units Size of correlation coefficient does not depend on its units of measurement Standardized regression coefficient Correlation is a standardized version of the slope Standard deviations s of and s of depend on their units of measurement Correlation is the value the slope would take for units such that the variables have equal standard deviations Formula: When sample spreads are equal (s X = s Y ), then r = b r is also called standardized regression coefficient

7 37 Eample: Mental health stud. regress mental life, beta noheader mental Coef. Std. Err. t P> t Beta life _cons sum mental life Variable Obs Mean Std. Dev. Min Ma mental life pwcorr mental life mental life mental life r = (s /s )b = (22.62/5.46)0.09 = 0.37