Regression Analysis Primer DEO PowerPoint, Bureau of Labor Market Statistics

Transcription

1 Regression Analysis Primer DEO PowerPoint, Bureau of Labor Market Statistics September 27-30, 2017

2 Regression Analysis Stephen Birch, Economic Consultant LTIP Technical Lead, Projections Managing Partnership September 27, 2017

3 y-axis PRIMER Illustration of Regression y = x R² = Regression Analysis A way to use what you know, to estimate what you don t x-axis We can use predictor variables about which we have more knowledge, to estimate what is not known about the variable in question. 3

4 General equation of a line Intercept, a, is 2. Graph of the Line y = x/2 + 2 When x increases by 1,... 4, 4.0 5, y increases by 0.5; so the slope, b, is The intercept is the value of y when x is 0. The slope, b is the amount y increases when x is increased by 1. y = bx + a where a is the intercept, and b is the slope. 4

5 y-axis 180 APPLYING TO DATA Illustration of Regression - Linear A graph of data that can be estimated by a line Notice that the data have a generally linear arrangement x-axis 5

6 y-axis DON T TRY THIS AT HOME! 4000 Illustration of Regression - Parabolic Data must be linear If data are curved, your line will not be a good fit x-axis Notice that, data below the line are grouped towards the middle, while data above the line cluster towards the ends. 5

7 y-axis 180 THESE DATA FIT A LINE Illustration of Regression A graph of linear data and a line, for comparison A line is added to our linear data. We want the line that minimizes the distances from the actual data x-axis Notice that data are scattered above and below the line, randomly. 5

8 ASSUMPTIONS BLUE Best Linear Unbiased Estimate - Assumes: Errors (distances from the line) are random. Errors have a mean of zero and constant variance. Errors are not correlated between observations. Random variation within predictor variables is negligible. Measurement error within predictor variables is negligible. 10

9 y-axis BEST FIT Illustration of Regression y = x R² = Regression finds the line that fits best x-axis To define the line that best matches our data, formulae calculate the slope and intercept that minimize the sums of the squares of the errors. Shows the equation of the best fit line, with the R 2 statistic. 5

10 THE NUMBERS The Regression Line Estimate of slope Estimate of intercept We use these estimates to calculate our projections. Coefficient of Determination Commonly known as R 2, pronounced R Squared Gives the share of the variation explained by the model R 2 = 1 is the ideal, all variation is explained. R 2 = 0 is the worst possible, no variation is explained. We usually require R 2 >

11 THE NUMBERS t Statistic Tests confidence of slope and intercept Student t distribution Require t > 2 or t < -2 in the direction of correlation Residuals (errors) The difference between actual and estimated (the line) Random with mean of zero and constant variance Not correlated between observations (serial correlation) Serial Correlation Indicates something is missing from the model Durbin Watson test Can inflate standard error and t statistic 10

12 y-axis RESIDUALS Illustration of Regression - Residuals The distance of each observation from the regression line Random variable with mean of zero and constant variance x-axis There should not be any pattern among the residuals. 5

13 ASSUMPTIONS Sometimes your have a good predictor variable, BUT IT JUST ISN T ENOUGH! 10

14 y-axis REMEMBER THIS ONE? 4000 Illustration of Regression - Parabolic y = x R² = Data must be linear If data are curved, your line will not be a good fit x-axis Clearly, there is correlation. We need more help to get a proper regression. 5

15 y-axis 800 No, that will not do! Illustration of Regression - Residuals 600 A look at the residuals The clear pattern in the residuals shows that this is not a good fit x-axis Do you recognize the pattern in the residuals? It is a parabola; so, we need to add x 2 as a second predictor variable. 5

16 y-axis 4000 Now, that s better. Illustration of Regression - Parabolic y = x x R² = 1 Although a curve, y is a linear function of both x and x Bringing in a second variable, in this case, a transformation of x, we can now find a good fit x-axis Our second variable is the help we needed to get a proper regression. 5

17 Residuals A LOOK AT THE NEW RESIDUALS Illustration of Regression - Residuals How do the residuals turn out? 2 The pattern in the residuals is gone x-axis Now, residuals are random, with mean of zero and a constant variance. 5

18 LET S TRY A REAL-WORLD EXAMPLE x, x 2 and y don t really mean much. You work in a men s clinic, for a doctor who is doing routine physical examinations. It is your job to take some basic measurements before each man sees the doctor. age, height, waist, pulse, arm circumference, and elbow breadth Your scale broke near the end of the day, so you need to estimate the last man s weight based on the other measurements. Before it broke, you had taken measurements for several other men. Let s try a regression of weight against waist on the existing data from the other men. What kind of results will we get? 10

19 Weight in pounds 260 REAL-WORLD EXAMPLE Weight vs Waist y = x R² = This graph shows a correlation between waist and weight. There is a clearly linear pattern in the data; but notice the low R Waist in inches We need another variable that can explain some of the variation not explained by waist alone. 5

20 3-D GRAPH? We bring in another measurement, but we don t have a 3-D graph. A regression of weight against both waist and height has R 2 = , which is still less than 0.9 Waist s t = 13.47, and Height s t = 5.10 are both greater than 2. So, both variables are good predictors, but R 2 is still too low. We need to bring in yet another variable. Let s try arm circumference. 10

21 4-D GRAPH? We bring in yet another measurement. A regression of weight against waist, height and arm circumference has R 2 = ; adjusted R 2 = Waist s t = 8.76, Height s t = 7.39, and arm s t = 6.29 are each greater than 2. So, all variables are good predictors, and R 2 is improved. Even if adjusted because of the added variable, R 2 >

22 A REAL-WORLD EXAMPLE Other checks We should examine the residuals, to check for serial correlation, and we should also check for multicollinearity; but we will not bother, in this example. Results Using our estimates of the intercept, and the coefficients of waist, height and arm circumference, tells us that this man with 36 waist, 5 10 height and arm circumference of 32.9 cm, weighs about 179 pounds. 10

23 LTIP Pre-defined Models Consider the pre-defined models in the LTIP They always include two predictor variables. The first variable involves U.S. employment in the specified industry. The second variable is a broad measure of the local economy. 10

24 QUESTIONS 12

25 CONTACT Thank You. If you have any further questions, you may approach me during office hours, contact me as below, or post a question on the LTIP Forum at: Stephen Birch LTIP technical lead Phone: Stephen.birch@deo.myflorida.com 13