Simple Regression Model

Transcription

1 Simple Regressio Model 1. The Model y i 0 1 x i u i where y i depedet variable x i idepedet variable u i disturbace/error term i 1,..., Eg: y wage (measured i 1976 dollars per hr) x educatio (measured i yrs of schoolig)

2 Alterative ames for y explaied variable respose variable predicted variable regressad x explaatory variable cotrol/treatmet predictor regressor

3 :The joit properties of y, x u, x are key to determiig the statistical properties of ay estimator. So havig a good idea of what geerates the disturbace is importat for good empirical work. : What geerates u? left out variables approximatio error for fuctioal form measuremet error i y iheret radomess i relatioship betwee y ad x

4 : 0, 1 are parameters called the "regressio coefficiets: 0 itercept 1 slope coefficiet each choice of 0, 1 defies a u a particular u will defie 0, 1

5 :ASIDE Sometimes it is useful to imagie that the slope coefficiet varies across observatios, i.e. 1i 1 x i, u i, v i where v i is idepedet of x i, u i. Such a uderlyig model explais why regressio coefficiets vary from sample to sample. So for example, respose to the math-stat review might deped o how much math you kow x i, persistece ad itelligece u i, ad luck v i The (classical) radom coefficiets model has 1 v i ot very iterestig. If we have importat oliearities, the we ll see 1 x i. Tough case is if 1 varies with the uobservables u i.

6 : Liearity is restrictive, but ot as bad as it first appears. For example, suppose we have V c 0 W c 1 defie y lv x lw u l. The y 0 1 x u 0 lc 0 1 c 1 Other trasformatios (of either the LHS or RHS) may help to iduce liearity i parameters, which is what we eed.

7 : If we thik of y, x as radom variables, the fy x tells us everythig about how probability assessmet of y vary with x. Regressio models focus o some measure of the "cetral tedecy" of y give iformatio about x. (A) Eu 0 ad covu, x 0 Eu Eux 0 0, 1 are coefficets of BLP (B) Eu x 0 Eugx 0 gx L 2 Ey x 0 1 x coditioal mea or "Populatio Regressio Fuctio" (C) u idepedet of x (D) y 0 1 x is a causal relatioship (ot statistical)

8 2. Derivig the OLS estimates Method 1 By defiitio, OLS estimates satisfy 0, 1 arg mi 0, 1 i1 Rk: Give ay cadidate 0, 1, y i 0 1 x i is called the fitted value, u i y i y i is called the residual. I ll reserve y i, u i for the OLS values. y i 0 1 x i 2

9 From the F.O.C., we obtai the "ormal equatios" N1 N2 i1 y i 0 1 x i 0 i1 x i y i 0 1 x i 0 i1 i1 u i 0 x i u i 0 Rk: Lots of authors use otatio b 0, b 1 for OLS estimatios

10 Method 2 The BLP geerates a disturbace u that satisfies Eu Exu 0. So 0, 1 of iterest satisfies Ey i 0 1 x i 0 Exy i 0 1 x i 0 The method of momets approach picks estimators to satisfy the sample couterparts: N1 1 y i 0 1 x i 0 N2 1 i1 i1 x i y i 0 1 x i 0 1 i1 1 i1 u i 0 x i u i 0

11 Computig 0, 1 From N1/N1, we get y 0 1 x 0 y 1 x Substitute ito N2/N2. If x i x 2 0, the 1 x i x y i y x i x 2 Rks: Sample Regressio fuctio is the estimated PRF (aka fitted regressio lie) y 0 1 x :Prove 1 x iy i y x i x 2 x i x y i x i x 2

12 :We ve show that 0, 1 is uique provided x i x 2 0. Ca you show that the fitted values y i have the same values for ANY solutio of the ormal equatios?

13 Eg: y wage (measured i 1976 dollars per hr) x educatio (measured i yrs of schoolig) Suppose the fitted regressio lie is wage educ (Notice that I do t report may digits use lots for accurate calculatios but do t preset them!) educ 0 the wage. 90 (-90 cets per hr) educ 8 the wage ($3.42 per hr) Q: What s the premium to completig high school vs. grade 8? Q: What s the premium to completig uiversity vs HS?

14 Algebra of OLS Everythig follows from the ormal equatios 1. u i 0 2. x iu i 0 Some implicatios of 1. & y 0 1 x (reg lie goes through sample mea) 4. u i y i 0 Proof: u i y i u i 0 1 x i 0 u i 1 x i u i 0 0 by 1. ad 2. (resp)

15 Rk: The same proof shows u i c 0 c 1 x i 0 c 0,c 1 4. u i y i y 0 5. (Aalysis of variace) y i y i u i y i y y i y u i Therefore y i y 2 y i y 2 u i 2 2 u i y i y y i y 2 u i 2 by 4 SST SSE SSR (Textbook s otatio) Total Sum of Squares Explaied SSResidual SS

16 6. Coefficiet of Determiatio R 2 SSE SST 1 SSR SST :0 R 2 1 R 2 1 says exact li. rel. betwee y ad x R 2 0 says o li. rel. betwee y ad x :R 2 gives the fractio of the variace of y that s "explaied" by the model Exercise: Show R r yx r yy where y y x x r yx y y 2 x x 2 1/2

17 7. Computig the explaied sum of squares y i y x x 2 1 y y x x Rk: This meas we ca compute 0, 1, u 2 i,ad R 2 from y i 2 y i y i x i x i x i 2 (Before proceedig, you should review the Matrix Algebra 1 otes)

18 Regressio Model i Matrix Notatio y x 1 u 1 y x 2 u 2 y 1 2 x u Defie y 1 1 x 1 u 1 y X 1 X 2 u y 1 x u x1 x1

19 X X 1 X 2 x x1 :I matrix otatio, the model is y X u LHS is a vector i RHS is a vector i Equality holds compoet by compoet y i X i u i X i is the i th row of X 1 1 x i 2 u i

20 :Derivatio of the OLS estimator arg mi u u 2 Rk: arg mi 2 y X y X u u u 1 u u 1 i1 u i 2 u

21 :Normal equatios Defie u y X where u, y, X x2, 2 I matrix otatio, the ormal equatios are X u 0 But X y X 0 X 1 1 x 1 x y X y x 1 y 1 2 x

22 X y X y i 1 2 x i x i y i 1 2 x i 0 :Matrix otatio for OLS estimator X y X 0 X y X X 0 Rk: X X X y X y y i x i y i X X x i x i x i 2

23 Q:CawesolveX X X y for? A: Yes (always) ad the solutio is uique iff detx X 0. Usig detx X x i 2 x i x i x i x 2 We see that iff x i x 2 0, the X X 1 s.t. X X 1 X X I 2 X X 1 X X X X 1 X y I 2 X X 1 X y X X 1 X y

24 Usig We get X X 1 1 x i x x i x i x i y 2 x x i x y i y / x i x 2 Rk: I ll show how to derive other results usig matrix otatio whe we cover ch3.

25 :A Importat decompositio Note that we ca write y X XX X 1 X y Py where P P (symmetric), ad P P 2 (idempotet) Also u y y I Py My where M M (symmetric), ad M M 2 (idempotet). Therefore y y u Py My