Lecture 13. Simple Linear Regression

Transcription

1 1 / 27 Lecture 13 Simple Linear Regression October 28, 2010

2 2 / 27 Lesson Plan 1. Ordinary Least Squares 2. Interpretation

3 3 / 27 Motivation Suppose we want to approximate the value of Y with a linear function of X. y i α + βx i i = 1, 2,..., n However the data will not fit perfectly. Assume y i = α + βx i + ε i i = 1, 2,..., n What values of α and β are the best? The main idea is to minimize the following quantity ε 2 i = (y i α βx i ) 2

4 4 / 27 OLS Derivation First take the partial derivatives with respect to α and β. n (y i α βx i ) 2 α = na = (2α 2Y i + 2βx i ) = 0 y i β α = ȳ β x x i n (y i α βx i ) 2 β β = x 2 i = ( 2βx 2 i 2x i y i + 2ax i ) = 0 x i y i α x i

5 5 / 27 OLS Derivation If you substitute the value for α in the expression for β, it turns out that, after some algebra manipulations, [ ] β x x i = x i y i ȳ β [ x 2 i x 2 i n x 2 ] = x i x i y i n mȳ x [ nβ x 2 x 2] = n [yx ȳ x] β = [yx ȳ x] [x 2 x 2 ]

6 6 / 27 OLS Derivation If you look at the numerator of β, yx ȳ x looks like E(YX) E(Y )E(X) Therefore you should not be surprised that yx ȳ x [y i ȳ] [x i x] n 1 yx ȳ x is the sample covariance of Y and X.

7 7 / 27 OLS Derivation If you look at the denominator of β, x 2 x 2 looks like E(X 2 ) E(X) 2 Therefore you should not be surprised that x 2 x 2 [x i x] 2 n 1 x 2 x 2 is the sample variance of X.

8 8 / 27 OLS Estimates or α = ȳ β x β = Cov(x, y) Var(x) = (y i ȳ) (x i x) (x i x) 2 β = x i y i x 2 i x i y i n ( ) 2 x i n

9 9 / 27 OLS Estimates Notice that sometimes, for example in the book, a different formula is considered. If you define (centering) x = x x ỹ = y ȳ Then, because x and ỹ have mean zero, α = ȳ β x ỹ i x i β = x 2 i

10 10 / 27 Example: Leaning Tower of Pisa Lean (m) = distance between where a point on the tower would be if the tower were straight and where it actually was. Suppose we want to fit the following line Lean = α + βyear + ε Year Lean

11 Example: Leaning Tower of Pisa Y = Lean; X = Year y i = x i y i = x i = x 2 i = β = α = 13 x i y i 13 xi 2 13 y i 13 β x i y i 13 ( ) 2 13 x i x i = = / 27

12 Example: Leaning Tower of Pisa Lean Year 12 / 27

13 13 / 27 Example: Leaning Tower of Pisa The intercept: α = The value of y when x = 0. Is this number useful? The slope: β = The amount by which the variable y changes when x increases by one unit. Does it matter how we record year(e.g. 87 versus 1987)? The fitted line represents the predicted values of y.

14 14 / 27 Interpolation and Extrapolation Interpolation is the process to determine the response y for a specific value of the explanatory variable x inside the range of x. What is the lean value for the Pisa tower at time ? y = = Extrapolation is the process to determine the response y for a specific value of the explanatory variable x outside the range of x. What is the lean value for the Pisa tower at time 2000? y = =

15 15 / 27 OLS The difference between the true values and the theoretical values represent the error: ε i = y i (α + βx i ) An important property of OLS is that the sum of squared errors is minimum (ordinary least squares) To emphasize that the unknown intercept and slope are estimated by the data, we often write ŷ i = ˆα + ˆβx i ŷ i represents the predicted (fitted) value. It is also an estimate!

16 Example: Leaning Tower of Pisa 16 / 27

17 17 / 27 Y = X + ε If we substitute the data for X, Y into the OLS estimate: X Y ε (10000 ε) e

18 Y = X + ε If we substitute the data for X, Y into an arbitrary estimate of the regression line we can see that X Y ε (10000 ε) / 27

19 19 / 27 Example: α and β to Google Amazon S&P 500 Return Return Return Week Week Week

20 20 / 27 Example: α and β The beta coefficients represents how the expected return of a stock is related to the performance of a financial market. The alpha is the extra return awarded to the investor for taking a risk, instead of accepting the market return. It turns out that: R (stock) t = α + βr (market) t Google versus S&P: α = , β = Amazon versus S&P: α = , β =

21 Example: α and β S&P 500 Google S&P 500 Amazon 21 / 27

22 22 / 27 Example: α and β If you were to invest in stocks, you would like to have high returns and little risk (for example variance). If you were evaluating the linear regression between a single stock and a general portfolio, β would measure the amount of variance (risk) associated with a single stock which cannot be diversified by investing in the portfolio. Because β represents the relationship between the stock and the portfolio return.

23 23 / 27 Demand Elasticity Demand Elasticity is an important concept in economics. Suppose we want to study the relationship between the price and the demand of a specific good. The demand for a drug which cures a severe disease is typically inelastic. No matter how expensive, people prefer to get cured (inelastic demand). If the price for pasta increases people may start buying rice instead (elastic demand).

24 24 / 27 Demand Elasticity Elasticity is defined as Elasticity = q p p q From a practical point of view, the easiest way to evaluate elasticity is to define a model and then derive the corresponding value. For example, given the following data describing how much are you willing to buy (q) at the level price (p) p q

25 25 / 27 Demand Elasticity A simple approach consists in evaluating a linear regression and deriving the corresponding elasticity. 8 (p i p)(q i q) = 8 (p i p) 2 = 8 p i q i = q i 2 = 42 β = = α = q β p = Therefore ˆq = p

26 26 / 27 Demand Elasticity The next step is to evaluate the theoretical elasticity Elasticity = q p p q = β pˆq Therefore p q ˆq Ê If p < 43 the demand is inelastic If p > 43 the demand is elastic

27 27 / 27 Demand Elasticity Elasticity is defined as Elasticity = q p p q = q/q p/p The beta coefficient represents the change in y associated with a unit change in x. The elasticity is measured in percentage. The elasticity is unit-less