Two-Variable Analysis: Simple Linear Regression/ Correlation

Transcription

1 Two-Variable Analysis: Simple Linear Regression/ Correlation 1

2 Topics I. Scatter Plot (X-Y Graph) II. III. Simple Linear Regression Correlation, R IV. Assessing Model Accuracy, R 2 V. Regression Abuses / Misinterpreting Correlation 2

3 I. Scatter Plot Used to visualize relationship between two variables. Common Findings Shown: Linear relationships Non-linear relationships No Relationships (robustness) 3

4 Scatter Plot Shows the relationship between X (predictor) and Y (response) given a range of X. Y Dependent Variable Response Variable X Independent Variable Predictor Variable 4

5 Example 1: Sales Analysis Suppose a sales manager wants to assess the performance of some of their sales reps based on the time they have worked with the company. What is the response, what is the predictor? Sample Time with Company (months) Average Monthly Sales, $ Sample Time with Company (months) Average Monthly Sales, $ 1 11 $25, $68, $32, $53, $24, $52, $36, $72, $31, $72, $35, $55, $34, $65, $42, $70, $40, $73, $44, $78, $55, $52, $37, $66, $54,880 5

6 Scatter Plot: Sales Territory Performance Is there a relationship here? Average Monthly Sales, $ $90,000 $80,000 $70,000 $60,000 $50,000 $40,000 $30,000 $20,000 $10,000 $ Time with Company (months) 6

7 Lecture Exercise: Changing Range of X Suppose you reduce your sample to only include salespersons with time between months. [Excel file: regression.xls] Compute the range of Y (sales) for all the data, and the subset. Is the range of Y (sales) smaller, larger, or the same as over the full range of X? Construct a scatter plot of this new data set? Do you still think a strong relationship exists? 7

8 Lecture Exercise Solution Effect on Y by reducing Variation in X Is there still a strong linear relationship? Time Range,X Sales Range, Y ALL $54, $19,750 Is the range different for say than ? What is the lesson here? Average Monthly Sales, $ $100,000 $80,000 $60,000 $40,000 $20,000 $ Time with Company (months) 8

9 Lessons from Sales Territory Example Relationships between Y and X variables may change depending on the range of X. Scatter plots provide good visualization of relationships between variables, but we need a metric to assess Strength of Relationship. For Two variables we may use regression and correlation to assess this relationship. 9

10 II. Simple Linear Regression Simple Linear Regression examines the relationship between two variables: one response (y), and one predictor (x). If two variables are related, a regression equation may be used to predict a response value given a predictor value with better than random chance. 10

11 Simple Regression Equation Y = β o + β 1 X 1 Y = dependent variable (response) X 1 = independent variable (predictor) β 0 = intercept; the value of Y when X = 0. β 1 = slope; the predicted change in output Y per unit change of input X. Alternatively, Y = mx + b (m is slope, and b is y-intercept) 11

12 Computing Slope and Intercept We typically use software to compute the slope and y-intercept. In Excel, we may use: Or, compute automatically =slope(y-array,x-array); Using QETools and Regression =intercept(y-array,x-array) Tool (Or, Scatter plot tool). A B C D Sample Time with Company Average Monthly 1 (months) Sales, $ $25, $32, $24, $36, $31, $78,400 Slope $192 =slope(c3:c27,b3:b27) Intercept $34,623 =intercept(c3:c27,b3:b27) 12

13 Regression QETools To create a scatter plot with the slope and intercept, we may use Graphical Tools >> Scatter Plot, or Regression >> Linear Regression. Y (Response) = Sales X (Predictor) = Time with company (months) 13

14 Sales Example: Trend Line Example with best fit (trend) line and equation on scatter plot AverageSales$ Time(months) Regression y = x Intercept AverageSales$ (Y) Slope Time(months) (X) 14

15 Slope Values and Trend Lines Positive slope values Increasing trend lines on scatter plot. Negative slope values Decreasing trend lines on scatter plot. No slope (~0) Horizontal trend lines. Comment: be careful with using absolute magnitudes. Depending on units, a very small slope could be significant. 15

16 III. Correlation Correlation (R ) provides a measure of effectiveness model prediction. Perfect correlation suggests that we may pass a line through every observation. Y Y R = 1.0 X X 16

17 Correlation In assessing relationships between variables, we often want to know strength of relationship. The Pearson correlation coefficient, R, measures the extent to which two variables are related. R = ( x x )( y y ) i ( n 1) s x s y i where i = 1..n pairs -1 < R < 1 Microsoft excel function: = correl(array1,array2) 17

18 Correlation Sales Territory Example From Excel: Correl (R ) =correl(b2:b26,c2:c26) R = 0.86 Linear Model R R Time(months) AverageSales$ Scatter Plot Note: Scatter Plot with Correlation using QETools >> Graphical Summary >> Scatter Plot AverageSales$ y = x R 2 = Time(months) 18

19 Correlation Patterns Perfect Positive R = 1.0 Strong Positive R = 0.7 Perfect Negative R = -1.0 Strong Negative R = -0.7 Rule of Thumb: Correlation > 0.7 strong relationship 19

20 No Correlation If no correlation exists, R = 0. Response, Y Predictor, X 20

21 IV. Assessing Model Fit, R 2 Another tool to assess model fit (or predictability) is R 2. R 2 - multiple correlation coefficient R 2 is computed by squaring the correlation, R 0 (no correlation) < R 2 < 1 (perfect correlation) 21

22 What does R 2 Measure? R 2 - measures the % of the variance in Y explained by X over the observed range of X. Suppose R = 1 R 2 = 1, thus all of the variance in Y may be explained by X for the observed range of X. R =0.7 R 2 = 0.49, thus, 49% of the variance in Y may be explained by X for the observed range of X. R =0.1 R 2 = 0.01, thus, only 1% of the variance in Y may be explained by X for the observed range of X. 22

23 Sales Example Revisited Recall our different equations based on the range of X for the Sales example. Over the full range, we have high correlation where Time explains ~74% of Variance of Sales,$. Over the tighter range, Time explains much less of the Variance in Sales (~12%) Time Range,X Sales Range, Y Slope Intercept Correlation, R R 2 ALL $54,020 $192 $34, $19,750 $99 $48,

24 Lecture Exercise: Model Prediction and Correlation Suppose you are in charge of a Six Sigma project in a construction company to penetrate a new market Currently the construction company typically builds homes around 1,500 sq-ft. One of your response variables is home sales price. Note: Recent market data has been collected for the analysis Excel Data File: regression.xls 24

25 Lecture Exercise: Home Sales Analysis Data Analysis: Response: Home Sales Price Predictor: Home Size, Target: Home Sales Price > $150,000 Perform the following: Home Size (Hundreds of Sq-ft) Sales Price (Thousands of Dollars) Scatter plot (Home Size Vs. Sales Price), fitted regression line, Correlation (R), and Assess fit, R 2 25

26 Home Sales Example: Scatter Plot / R 2 Home Sales Example: R = 0.97; R 2 = 0.94 Home Sales Price Vs. Home Size Sales Price (000's $) y = 6.23x R 2 = Home Size (00's Sq-ft) 26

27 Lecture Exercise: Interpreting Results Clearly, home size has a strong relationship with sales price. Suppose the company wants to build homes that sell for more than $150,000, how might we determine the appropriate home size from our model? 1. What home size would the company need to build to sell for $150,000 (Hint: Find X given Y = 150)? 2. What home size would the company need to build to sell for $400,000 (Hint: Find X given Y = 400)? 27

28 QETools >> Response Solver Given an equation, QETools can solve for either X or Y given the other. Response Optimization Use scroll bars or enter values below HomeSize(Sq-ft) Y Response Optimization Use scroll bars or enter values below HomeSize(Sq-ft) Y SalesPrice($) (Y SalesPrice($) HomeSize(Sq-ft) Regression y = 6.231x R 2 = HomeSize(Sq-ft) (X) 28

29 Solve the Equation for X Equation and Estimates: Y = 6.23X If Y = 150, X = or 1,272 Sq-ft If Y = 400, X = or 5,285 Sq-ft Do these values make sense? Home Size (Hundreds of Sq-ft) Sales Price (Thousands of Dollars) Note: From QETools >> Regression >> Linear Regression. 29

30 Re-Examine Scatter Plot Are these data linear? Home Sales Price Vs. Home Size Sales Price (000's $) Home Size (00's Sq-ft) 30

31 Scatter Plot with Quadratic Line Using Graphical Tools >> Scatter Plot, we may select quadratic for best fit line. 240 HomeSize(Sq-ft) SalesPrice($) Scatter Plot 220 y = x x R 2 = SalesPrice($ HomeSize(Sq-ft) 31

32 V. Regression Abuses / Misinterpreting Correlation Between the Sales Territory and Home Sales examples, we have noted several potential abuses of regression: Be careful that you have a linear model when applying linear regression. Do not make inferences outside the region of study (example: home size = 0). Relationships between X and Y may change depending on the range of observed X values. Two additional issues are: Be careful of extreme values and their impact on correlation estimates. Correlation does not necessarily imply causation. 32

33 Extreme Values Consider an experiment between temperature and gasoline consumption. Based on these data, are they strongly related? Temperature (F) Gasoline Consumption Correlation

34 Gas Consumption Example With temperature = 32 reading R = -0.81; without this reading -0.2 Lesson Graph before interpreting correlation! Temperature Vs Gas Consumption Gas Consumption Temperature 34

35 Interpreting Correlation When drawing conclusions based on correlation, several issues must be considered: Correlation coefficient (R) measures the linear relationship (non-linear may exist). Correlation coefficient may be sensitive to extreme values ALWAYS GRAPH. Correlation does not necessarily indicate cause and effect! 35

36 Correlation Vs. Causation Correlation does not necessarily imply causation!! Does your income tend to increase mostly because you are older or because you have more experience/ seniority at your company? Salary, $ Response, Y Response, Y Predictor, X Predictor, X Age 36

37 Verifying Causation To verify that correlation relates to causation, we should conduct controlled experiments. Approach: hold other process variables fixed and then test if Y changes in relation to X. Or, experiment with multiple variables in a systematic approach (Note: Design of Experiments) provides more advanced verification approaches for multiple X variables (covered in Black Belt Course). 37

38 Summary: Regression / Correlation and Six Sigma Regression and correlation provide tools to measure strength of relationships between variables (usually between continuous variables). Explore relationships using scatter plots to avoid abuses. Regression/correlation is often used during the Analysis Phase to assess relationships between outputs (Ys) and inputs (Xs). Or, during the improve and/or control phase to VERIFY relationships. Remember, finding no correlation may be just as important as finding a strong causal correlation. COST AVOIDANCE find setting or range of settings for X in which Y is robust (insensitive)! 38