Chapter 5 Friday, May 21st

Transcription

1 Chapter 5 Friday, May 21 st

2 Overview In this Chapter we will see three different methods we can use to describe a relationship between two quantitative variables. These methods are: Scatterplot Correlation Regression Equation

3 Scatterplot t A scatterplot is a two dimensional graph of the measurements for two numerical variables On the x-axis (horizontal line) we plot the explanatory variable On the y-axis (vertical line) we plot the response variable.

4 Scatterplot t In a scatterplot we try to identify: If we have a straight line or a curved pattern What is the direction of the pattern How much do individual points vary from average pattern If there are any unusual observations

5 Example We take 20 students and we measure the hours of study and their GPA Explanatory variable: hours of study Response variable: GPA The scatterplot is shown on the next page

6 Example GPA Study Hours 15 What can we say about this relationship? Linear and positive.

7 Linear relationship GPA Study Hours 15 Two variables have a linear relationship when the pattern of their relationship resembles a straight line

8 Nonlinear or curvilinear pattern Average Study When the relationship is better described by a curved line then we have a curvilinear relationship

9 Positive Association GPA Study Hours 15 Two variables have positive association when the values of one variable increase as the values of the other variables increase

10 Negative Association Distance Age Two variables have a negative association when the values of one variable tend to decrease as the values of the other variable increase

11 In a scatterplot we can divide the groups. Assume the 20 students t are divided into males and females 4.0 F M 3.5 GPA Study Hours 15

12 Identify Outliers Outliers can have a big impact in identifying the relationship When we are comparing two variables an outlier is a point that have an unusual combination of data values Imagine someone studying 15 hours and having GPA 2.5 Then the previous graph will be like this:

13 Identify Outliers GPA Study Hours 15

14 Regression Analysis Regression analysis is the area of statistics that is used to examine the relationship between a quantitative response variable and one or more explanatory variables In order to describe how, on average, the response variable is related to the explanatory variable we use the regression equation We use the regression equation to predict values of a response variables using known values of the explanatory variable.

15 Regression line A regression line is a straight line that describes how values of a quantitative response variable (y) are related to values of a quantitative explanatory variable (x). This is used for two purposes: p To estimate the average value of y at any specified value of x To predict the value of y for an individual given the To predict the value of y for an individual, given the individual s x value Simple linear regression refers to methods used to analyze straight line relationships

16 Example Regression Plot GPA = Study Hours S = R-Sq = 79.2 % R-Sq(adj) = 78.1 % 4.2 GPA Study Hours 14

17 Equation As you can see in the previous slide above the plot there is an equation. When we have a straight line the equation is of the form: y b bx 0 1

18 Equation b 0 b is the y-intercept b 1 is the slope The y-intercept tells us what is going to be the value of y when x=0 The slope tells us how much the y variable changes for each increase of one unit in the x variable

19 Relationship In a deterministic relationship, if we know the value of one variable we can exactly determine the value of the other. So in this case all the points lie on the straight line In a statistical relationship there is a variation from the average pattern. That s why most of the points are around the straight line

20 Regression equation Since in a statistical relationship the line actually ypredicts the average trend of the y variable given the x variable, we say that y is predicted or estimated So, the regression equation is denoted with ŷy b bx 0 1 where the estimated y variable is denoted with y-hat

21 Prediction error The prediction error of an observation is the difference between the observed value and the predicted value, that is: error y yˆ y is the observed value and y-hat the estimated value The prediction error is also called residual

22 Example In the example with the hours of study and the GPA the Regression equation was of the form GPA * Hours If a student study for 8 hours and he has a GPA of 3.7 what is the error for this observation? Answer: First we calculate the fitted value: *Hours= *8=3.42 Then the error is =

23 Least squares Least squares is the technique that we use in Statistics to find the equation of the regression line A least square line has the property that the sum of the squared errors is smaller than any other line The notation SSE is used to denote the sum of squared errors.

24 Formulas for the Least Square Line Let: x i denotes the x measurement of the ith observation y i i denotes the y measurement of the ith observation x y denotes the mean of x measurements denotes the mean of y measurements

25 Formulas for the Least Square Line Formula for slope: b Formula for intercept: x x y y i i i 1 2 b y bx 0 1 i x i x

26 Correlation The correlation between two quantitative variables is a number that indicates the strength and direction of a straight line relationship The strength is determined by the closeness of the points to a straight line The direction is determined by whether one variable generally increases or generally decreases when other variables increases

27 Correlation Correlation describes only linear relationship When the pattern in not linear then correlation is not an appropriate way to measure the strength We represent correlation with letter r and is also called Pearson correlation coefficient

28 Formula for correlation Let n s s x s y be the sample size the standard deviation of x measurements the standard deviation of y measurements and the rest notation as described earlier. Then the formula for the correlation coefficient is 1 xi x yi y r n 1 i s x s y

29 Interpretation of correlation coefficient It can take values between -1 and 1 The sign of the correlation indicates the direction of fthe relationship The magnitude of the correlation indicates the strength of the relationship A correlation that is either 1 or -1 indicates that there is perfect linear relationship A correlation of 0 indicates that the best straight line is exactly horizontal, so knowing the value of x does not give any information i for y.

30 Example Regression Plot C14 = C13 S = 0 R-Sq = % R-Sq(adj) = % C r= C13 4 5

31 Example Regression Plot C14 = 6-1C13 S = 0 R-Sq = % R-Sq(adj) = % 5 4 C r= C13 4 5

32 Example Regression Plot GPA = Study Hours S = R-Sq = 79.2 % R-Sq(adj) = 78.1 % 4.2 GPA r= Study Hours 14

33 Example Regression Plot Distance = Age S = R-Sq = 20.7 % R-Sq(adj) = 17.8 % Distance r= Age 70 80

34 Squared correlation The squared value of the correlation denoted with r 2 describes the strength of a linear relationship and takes values between 0 and 1 (or between 0% and 100%) Problem: By squaring the values we lose information about the direction of the relationship

35 Squared correlation Squared correlation also sometimes is used to see how much of the proportion p of variation of y is explained by x. So in the example with hours of study and GPA (graph in the following slide) the hours of study explain 79.2% of the variation among the GPA of the students.

36 Example Regression Plot GPA = Study Hours S = R-Sq = 79.2 % R-Sq(adj) = 78.1 % 4.2 A GP Study Hours 14

37 Formula for squared correlation First we have to define sum of squares total (or total variation), denoted by SSTO, to be the sum of squared differences between the observed values y and the sample mean, that is: 2 SSTO y y i i

38 Formula for squared correlation Second we have to remind ourselves that: SSE y y i i ˆ 2 So a formula for squared correlation is: SSTO SSE r 2 SSTO

39 Regression and Correlation Problems There are some problems that can cause misleading results in regression and correlation Extrapolating too far beyond the observed range of x values Allowing g outliers to overly influence the results Combining groups inappropriately Using correlation and a straight-line equation to describe curvilinear data

40 Extrapolation ti There is no guarantee that the relationship will continue beyond the range for which we have observed data. Example: Let s say I want to see the relationship between the weight and height of men. I have a sample of 20 men and the measure of height between 60 inches to 84 inches. The equation I get is Weight=-180+5*Height Now, if I want to predict the weight of the man that has height h 30 inches, what will happen?

41 Influence of outliers Outliers have great impact on correlation and regression results Outliers with extreme x values have the most influence on correlation and regression and are called influential observations Example of GPA and hours of study

42 Example without outliers Regression Plot GPA = Study Hours S = R-Sq = 79.2 % R-Sq(adj) = 78.1 % 4.2 GPA Study Hours 14

43 Example with one outlier Regression Plot GPA = Study Hours S = R-Sq = 17.0 % R-Sq(adj) = 126% GP PA Study Hours 20

44 Inappropriately combining i groups Let s say that we have the example with the Hours of study and the GPA. The results are shown in the following slide

45 Example No problem Regression Plot GPA = Study Hours S = R-Sq = 79.2 % R-Sq(adj) = 78.1 % 4.2 GPA Study Hours 14

46 Example What happens though if we divide the data into two different groups, let s say male and female and there are 12 males and 8 females in the study Next two slides can show that we can get two completely different results.

47 Example only women Regression Plot FGPA = FHours S = R-Sq = 98.5 % R-Sq(adj) = 98.3 % FGP PA FHours

48 Example only men 4.2 Regression Plot MGPA = MHours S = R-Sq = 86.7 % R-Sq(adj) = 85.4 % MGP PA MHours 14

49 Curvilinear data Sometimes there is a curvilinear trend in our data and we mistakenly use a linear regression line to describe the data. This can lead to wrong results. One good example is the results with GPA and hours of study. Using the equation in the following page, what will be the GPA of someone who studies 15 hours? What is the problem with this?

50 Example Regression Plot GPA = Study Hours S = R-Sq = 79.2 % R-Sq(adj) = 78.1 % 4.2 GPA Study Hours 14

51 Correlation does not prove Causation One of the biggest mistake someone can do when dealing with the association of two variables is to say that the explanatory variable cause the response variable to change. This is not always true. Actually in the next slide we describe the four possible outcomes that can come from an experiment

52 Interpretation of an observed association There is causation, that is, the explanatory variable cause the change in a response variable There may be causation, but confounding factors make this causation difficult to prove There is no causation. The observed variable can be explained by how one or more variables affect both the response and the explanatory variable The response variable is the one causing the change to the explanatory variable.

53 Causation The best way to prove a causal connection is by doing a well designed randomized experiment If the data is collected from an observational study then there will probably be confounding variables and we cannot prove causation