Practice 2 due today. Assignment from Berndt due Monday. If you double the number of programmers the amount of time it takes doubles. Huh?

Transcription

1 Admistrivia Practice 2 due today. Assignment from Berndt due Monday. 1 Story: Pair programming Mythical man month If you double the number of programmers the amount of time it takes doubles. Huh? Invention of Extreme Programming. Always have working code Always modify working code to make it simplier Small changes and lots of testing One Version is called pair programming Person who knows the code doesn t have the typewriter Person who knows the code can only make suggestions By the end of a day working together the other person knows the code Lots of experimental data on how effective pair programming is. Not as many controlled experiments as there should be. Oh well. 2 Status so far The model Y i = α + βx i + ɛ i 1

2 where ɛ i are iid and ɛ i N(0, σ 2 ). First we discussed fitting (α + βx i ) Then we discussed the residuals Now we want to discuss how to estimate the error in ˆβ 3 Why we care If the normal linear model holds, we know that ˆβ has close to a normal ˆβ distribution. In particular, is a t-distribution. We often want to SE( ˆβ) know how accurately we know β purely for its own sake. For example, if our model is Y = sales, and X = advertisments, then β =sales/ad. So if we know that each sale generates $10 profit, and each add costs $1 (think web based advertisements) then we need β >.1 before we make more money in sales than we spent in advertisements. We can also use ˆβ to make predictions: Ŷ = ˆα + ˆβx i So before we can now how accurate a forecast is, we need to know how accurate ˆβ is. Either of these require knowing that ˆβ is a good estimate of β and exactly how good an estimate of it it actually is. So we need the standard error for ˆβ. 2

3 4 Standard errors rely on the linear model assumption The standard errors generated by R/JMP/statistics all require that the standar linear model holds. I.e. the residuals are IID normal. If they aren t: predictions are wrong CI are wrong hypothesis tests are wrong accuracy of slopes are unknown We are left with just guessing. In previous classes, all we did is notice problems. Now we want to fix them (if possible). 5 Hetroskadasticity One problem that we can fix is that of hetroskadasticity. Suppose that we are regression salary (Y) on runs (X). Then we might expect that Y = α + βx will display hetroskadastic errors. In particular, we might expect that the errors grow with x. log-log model If we use logs, we can consider the model log(y ) = α + β log(x) + ɛ 3

4 In this model, we now expect the errors to be homoskadastic. log(y ) = α + β log(x) + ɛ e log(y ) α+β log(x)+ɛ = e Y = e α e β log(x) e ɛ Y = e α (e log(x) ) β e ɛ Y = kx β e ɛ where we inserted a k for e α so the equation looked prettier. The problem with this analysis is that we can no longer address the question, How much is a hit worth? Instead, we can say, How many log(dollars) is a log(hit) worth? If this sounds resonable to you, then you are definitely an economists: A slope between log(y) and log(x) is called the elasticity. Intrinsically linear models Cobb / Douglass production functions are a cool example of a linear model that doesn t look like a linear model: log(output) = log(labor) + log(capital) etc... Weighted least squares: Alternatively, we can do weighted least squares. If we believe that the errors scale with x, then we could write our model precisely as: Y = α + βx + xɛ So when x is large, the errors are large and when x is small the errors are small. This would show up nicely in a plot of ɛ 2 vs x. We can modify our equation by dividing both sides by x: Y/x = α/x + β + ɛ 4

5 Now if we do a regression of Y/x on 1/x we have a homoskadastic regression. This methodology is called weighted least squares. The weights in this case are 1/x. There are two ways of doing a weighted least squares. First, you can ask R to simply use a column of 1/x 2 as weights. Sample hetroskadastic data with evil LS line: > x <- 1:20 > y < x * (3 + rnorm(20)) > plot(x, y) > abline(lm(y ~ x)$coef) y x 5

6 Now with the weighted line: > plot(x, y) > abline(lm(y ~ x, weights = 1/x^2)$coef) y x This is nice since the equation it generates will still use α and β as we have been using them in the equations already. Second, you can create the two new variables Y/x and 1/x and do the regression yourself. This allows you control and will work in any regression package (i.e. excel). But you then have to interpret the slope and intercept carefully. So it looks like: 6

7 > newx = 1/x > newy = y/x > plot(newx, newy) > abline(lm(newy ~ newx)$coef) newy newx Compare the coeffients: > summary(lm(y ~ x, weights = 1/x^2)) Call: lm(formula = y ~ x, weights = 1/x^2) 7

8 Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) e-09 *** x e-09 *** --- Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Residual standard error: on 18 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: on 1 and 18 DF, p-value: 2.148e-09 > summary(lm(newy ~ newx)) Call: lm(formula = newy ~ newx) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) e-09 *** newx e-09 *** --- Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Residual standard error: on 18 degrees of freedom 8

9 Multiple R-squared: 0.856, Adjusted R-squared: F-statistic: 107 on 1 and 18 DF, p-value: 5.286e-09 To check that 1/x is the wrong weights: > summary(lm(y ~ x, weights = 1/x)) Call: lm(formula = y ~ x, weights = 1/x) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) ** x e-08 *** --- Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Residual standard error: on 18 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: on 1 and 18 DF, p-value: 1.948e-08 9