Example: Forced Expiratory Volume (FEV) Program L13. Example: Forced Expiratory Volume (FEV) Example: Forced Expiratory Volume (FEV)

Transcription

1 Program L13 Relationships between two variables Correlation, cont d Regression Relationships between more than two variables Multiple linear regression Two numerical variables Linear or curved relationship? 1 2 Assumptions for Pearson s correlation coefficient: Random samples Both variables are numerical No extreme values Assumptions for Spearman s correlation coefficient: Random samples The variables can at least be ranked (ordered) 3 The two variables Correlations Height (cm) FEV Height (cm) Pearson Correlation 1,869 ** Sig. (2-tailed), N FEV Pearson Correlation,869 ** 1 Sig. (2-tailed), N **. Correlation is significant at the.1 level (2-tailed). Pearson s correlation coefficient ( r ) The correlation between the two variables height and lung capacity (FEV) is quite strong (r =.869). The correlation is significantly different from zero (low P-value). P - value (H a : ρ ) 4 Quantitative Methods Fall 215 1

2 Testing the significance of the correlation A hypothesis test can be performed to test the significance of the correlation coefficient. If there is no correlation, the correlation coefficient is zero. H : ρ = : ρ H a The correlation in the population is denoted by the greek letter ρ (rho) If the P-value is low enough (usually compared to a significance level of 5%), the null hypothesis is rejected and we state that the correlation is significant (or significantly different from zero). 5 The two variables Correlations Height (cm) Spearman's rho Height (cm) Correlation Coefficient 1,,888 ** FEV Sig. (2-tailed)., N FEV Correlation Coefficient,888 ** 1, **. Correlation is significant at the.1 level (2-tailed). Sig. (2-tailed),. N Spearman s correlation coefficient ( r s ) P - value (H a : ρ s ) The correlation between the two variables height and lung capacity (FEV) is quite strong also when measured by the Spearman coefficient (r s =.888). The correlation is significantly different from zero (low P-value). 6 Nonsense correlation Limitations of the correlation coefficient (Pearson) A numerical relationship does not necessarily mean that there is a causal relationship X = Age of school children Y = Height of the same children X = Shoe size of shool girls Y = Drawing skills Z = Age r > X Y Cause-effect r > X Y Z Nonsense correlation 7 The correlation coefficient measures how well the relationship is captured by a straight line, but does not give any indication about which line. Two relationships with the same correlation coefficient: We can only see whether the direction is positive or negative, i.e. if the line is sloping upwards or downwards. 8 Quantitative Methods Fall 215 2

3 Relationship between two quantitative variables If a scatterplot shows a linear relationship, we can summarize this overall pattern by drawing a line on the scatterplot. A regression line summarizes the relationship between two variables in a specific setting; when one of the variables helps explain or predict the other. Linear regression Weight Vikt (kg) Age Ålder (years) 9 We have two variables that we want to investigate the relationship between. Eg: X=Age (years), Y=Weight (kg) 1 Linear regression Regression: Cause and effect Weight Vikt (kg) Age Ålder (years) A straight line can be used to summarize this relationhip. Is there a direction in the relationship? Determine which variable causes effects in the other. What is cause and effect when we study... Height and Weight of 19 year old boys? Price and Milage of used cars? First determine the direction of the relationship. Next find the straight line which best fits the scatter plot, i.e. determine location and direction of the line Quantitative Methods Fall 215 3

4 Regression: Terminology Linear regression The X-variable is assumed to effect the Y-variable. The X-variable is called the explanatory variable (the one that explains the other). The Y-variable is called the response variable (the one that responds to or depends on the other). Response variable (Y) Weight Vikt (kg) Age Ålder (years) Explanatory variable (X) The regression line The regression line For a quantitative variable, we often use a sample to get information about the population mean µ. If there is a relationship between X and Y, then regression analysis can use the information in the X-variable to improve our estimate of the mean of Y. The regression line can be viewed as point estimates of the mean of Y for different values of the X-variable. The line can be seen as point estimates of mean weight for different values of age Quantitative Methods Fall 215 4

5 What is a straight line? The mathematical description of a straight line is y = b + b 1 x What is a straight line? b is called the intercept and determines where the line crosses the Y-axis, i.e. the value of Y when X=. Intercept, b (the value of Y=weight where the line crosses the Y-axis, i.e.when Age=) What is a straight line? b 1 determines the slope of the line, the amount by which Y changes when X increases by one unit. b 1 =the amount by which Y (weight) changes when X increases by one unit What is the best line? So, we want to find a line that fits the data in the best possible way. We want the line to be in the middle of the scatter, and we want it to be as close to the individuals ( data points ) as possible. X (age) increases by one unit (1 year) 19 2 Quantitative Methods Fall 215 5

6 What is the best line? Least Squares Regression As when calculating the variance, the distances are squared, and the line that makes the sum of all squared distances as small as possible is considered to be the best line. This is called the least-squares regression line Linear regression Intercept Explanatory variable (X) Model Coefficients a Unstandardized Coefficients Standardized Coefficients B Std. Error Beta 1 (Constant) -6,88,514-11,846, Height (cm),56,3,869 17,264, a. Dependent Variable: FEV Dependent variable (Y) The effect of Height, b 1 The value of FEV (Y) when Height= (b ) t Sig. The P-value of the coefficients b and b 1 23 In SPSS: Analyze >> Regression >> Linear 24 Quantitative Methods Fall 215 6

7 Testing the significance of the coefficients -Linear regression A hypothesis test can be performed to test the significance of the regression coefficients. If X does not help explain Y, the value of b 1 is. H H a β : 1 = : β 1 The coefficient in the population is denoted by the greek letter β (beta) If the P-value is low enough (usually compared to a significance level of 5%), the null hypothesis is rejected and we state that the coefficient is significant (or significantly different from zero). 25 SPSS gives us the least-squares regression line Estimated from our sample y = b + b 1 x FEV = Height This means that for every cm taller the children are, the lung capacity increases by on average.56 liters (significantly different from zero, P-value <.5). Or, for every dm taller the children are, the lung capacity increases by.56 liters on average. 26 -Linear regression -Linear regression SPSS gives us the least-squares regression line Estimated from our sample y = b + b 1 x FEV = Height The intercept means that when the height is zero, the lung capacity is liters. Negative lung capacity??? 27 The regression line is fitted to the available data, it does not fit to any imaginary data outside the range of existing values. The intercept is the value of Y when X is zero. Height= is not a value that can exist, and it is also far outside our range of values (minimum = 12 cm). 28 Quantitative Methods Fall 215 7

8 Extrapolation Interpretation of the intercept (b ) If we include the value we can see how far it is from the range of existing values. The intercept only denotes the point where the regression line crosses the Y-axis and is not affected by whether this point has a relevant interpretation or not. Drawing conclusions about values outside the range of the sample is called extrapolating. Such predictions are often not accurate and should be avoided Interpretation of the regression coefficients (b and b 1 ) The intercept b is the estimated value of the Y-variable when X is. If X= is far outside the range of values in the sample, or the X-variable cannot take the value, the intercept (b ) does not have an interpretation. The slope b 1 is an estimate of how the population mean of the Y-variable changes when the X-variable increases by one unit. Predicting average values of Y We have the least-squares regression line for the relationship between Height and FEV FEV = Height Say that we are interested in the average lung capacity (FEV) of 14 cm tall children. Plug in the value Height=14 in our regression equation FEV = x 14 We estimate the average lung capacity for 14 cm tall children to x 14 =1.752 litres 32 Quantitative Methods Fall 215 8

9 How strong is the relationship? How much better does the estimation of the mean of the Y-variable (µ y ) get when we use information from the X-variable? The coefficient of determination (R 2 ) We can calculate the share of the total variation in the Y-variable which is explained by the regression. We define the coefficient of determination, R 2 For simple regression (using only one X-variable) the coefficient of determination equals the square of the correlation coefficient, r 2 = R 2. This relationship does not hold for multiple regression (when you have two or more X-variables) The coefficient of determination (R 2 ) The coefficient of determination shows how much of the total variation in the Y-variable that is being explained by the X-variable(s). R 2 1 ( 1%) -coefficient of determination Model R R Square Model Summary Adjusted R Square Std. Error of the Estimate 1,869 a,754,752,47933 a. Predictors: (Constant), Height (cm) Coefficient of determination (R 2 ) 35 In SPSS: Analyze >> Regression >> Linear. Y=dependent variable, X=independent. 36 Quantitative Methods Fall 215 9

10 -coefficient of determination R 2 =.754, i.e. the coefficient of determination is 75.4%. Interpretation: The individuals in the sample have different lung capacity measured by FEV, i.e. there exists a variation in lung capacity. They are also of different height, i.e. there exists a variation in height. The standard deviation (from the perspective of regression) The standard deviation is a measure of the average distance from the mean. In regression analysis, we are interested in the average distance from the regression line, i.e. the standard error of the regression prediction. The coefficient of determination tells us that 75.4% of the variation in lung capacity can be explained by the individuals different heights standard error of the prediction - standard error of the prediction Model R R Square Model Summary Adjusted R Square Std. Error of the Estimate 1,869 a,754,752,47933 a. Predictors: (Constant), Height (cm) The standard error of the regression prediction, i.e. the average distance from the regression line The standard error has to be interpreted in relation to the values of the variable in the sample. Lung capacity takes values between 1 and 6 l in the sample Average distance from the line is.479 liters In SPSS: Analyze >> Regression >> Linear. Y=dependent variable, X=independent Quantitative Methods Fall 215 1

11 - standard error of the prediction The standard error of the prediction is.479 liters. Interpretation: The children have different lung capacity, and different heights. A regression line shows the average lung capacity for different heights of the children. In our sample, the children s lung capacity is on average.479 liters more or less than the regression line. Half a liter is not very much in relation to the values that the variable (FEV) takes in the sample, 1-6 liters. The standard deviation of FEV (without using the information about height) is approx. 1 liter. Multiple linear regression In the previous example we managed to explain about 75% (R 2 =.754) of the variation in lung capacity (measured by FEV) of a sample of children, using their differences in height as the explanatory variable. While this is a very large share, it also means that 25% of the variation in lung capacity is unexplained Multiple linear regression Example: House prices There is often more than one factor behind the variation in the response variable Y. We can try to explain more by adding more explanatory variables (X-variables) in the model, thus making it into a multiple linear regression model. A real estate agent wants to make a prediction of the price of a house in a particular area. She knows that the price is affected by e.g. the size of the house. A sample of 19 recently sold houses is available. Y = house price (kr) X = house size (m 2 ) 43 Quantitative Methods Fall

12 Example: House prices Example: House prices (Constant) House size Price = House size The average price of houses in this area increases with 11 5 kr for every extra m 2 (interpretation of the slope b 1 ). The intercept (b ) cannot be interpreted here since there are no houses with a living area of m % of the variation in price between different houses can be explained by their differences in living area (interpretation of the coefficient of determination R 2 ). 31.3% of the variation in prices is still unexplained. 46 Multiple linear regression Two (or more) explanatory variables A multiple linear regression model with two explanatory variables is written Two X-variables y ˆ = b + + b1x1 b2x2 The coefficients b 1 and b 2 are interpreted as marginal changes, i.e. b 1 is the change in y when x 1 is changed one unit, and x 2 is being kept constant. b is the value of Y when all X-variables are zero. Multiple linear regression equation A model with two explanatory variables y y ˆ = b + + b1x1 b2x2 x 2 47 x 1 Quantitative Methods Fall

13 Example: House prices Dummy variables Can we find any other explanatory variables that might help explain the variation in house prices? The existence of a garage is likely to have an effect on the price of the house, and the agent wants to take this information into account. But whether a house has a garage or not is not a quantitative variable, it is a categorical variable (there either is a garage or not). We can quantify a categorical variable with only two categories as a variable that can take the values or 1, where 1 denotes the property of interest (eg. 1= the house has a garage, and = the house has no garage ). Such variables are called dummy variables and can be used as explanatory variables in regression analysis Example: House prices Example: House prices We expand the model to include the dummy variable Garage Y = house price (kr) X 1 = house size (m 2 ) X 2 = Garage 1 If the house has a garage If the house doesn t have a garage Price = House size Garage A house with one extra m 2 is sold for on average 1 4 kr more, keeping the other variable constant (i.e. not making any changes to the garage variable). A house with a garage cost on average 222 kr more than one without, keeping the other variable constant (i.e. not making any changes to the house size variable). Quantitative Methods Fall

14 What characterizes good explanatory variables? Strong correlation with the response variable. If the pattern of variation in the explanatory variable and the response variable corresponds, i.e. they covary, then one can be used to predict the other. Weak correlation with other explanatory variables. If the explanatory variables are strongly correlated, it is likely that they explain the same part of the variation in the response variable. Which explanatory variables to include in the model? The goal is to predict the value of the response variable. Use only those explanatory variables (X-variables) which have a substance-relation to the response variable (Y). Try not to use several variables which explain the same variation in the response variable Home assignment Deadline for supplements: Dec 17 th 17.. For supplements: READ THE INSTRUCTIONS again! Clearly mark your changes compared to the original report. 55 Quantitative Methods Fall