Economics 130. Lecture 4 Simple Linear Regression Continued

Transcription

1 Economcs 130 Lecture 4 Contnued

2 Readngs for Week 4 Text, Chapter and 3.

3 We contnue wth addressng our second ssue + add n how we evaluate these relatonshps: Where do we get data to do ths analyss? How do we create the model relatng the data? How do we relate data to on another? How do we evaluate these relatonshps?

4 Tonght we wll reprse last week s lecture. We wll then derve the estmates b 1 and b. We wll dscuss what t means that OLS wth certan assumptons s BLUE. We wll dscuss how evaluate our estmates. We wll do a problem.

5 Last week we: Developed a smple lnear regresson model Dscussed the error term Explaned the dfferences between parameters and estmates Presented OLS for obtanng estmates Introduced mnmzng the resduals Introduced R

6 Remember OLS chooses b 1 and b to mnmze the SSE, sum of squared resduals. The solutons: b ( Y Y)( X X ) = ( X X ) b 1= Y b X

7 Frst Dervaton (b 1 ) ê = Y - Ŷ (the resdual) ê = Y b 1 b *X ê ^ = (Y - b 1 - b *X)^ ê ^/ b 1 = *(Y - b 1 - b *X)*-1 0 = *(Y - b 1 - b *X)*-1 0 = (Y - b 1 - b *X) 0 = Y - b 1 - b *X b 1 = Y - b *X b 1 = Y - b *X n b 1 = Y - b *X b 1 = Y - b * X

8 Now let s do b : ê ^ = (Y - b 1 - b *X)^ ê ^/ b = *(Y - b 1 - b *X)*-X 0 = *(Y - b 1 - b *X)*-X 0 = (Y - b 1 - b *X)*X 0 = (YX - b 1 *X - b *X^) 0 = YX - b 1 *X - b *X^

9 YX =b 1 *X + b *X^ YX =b 1 X + b X^ YX =( Y - b X )*X + b X^ YX =[1/nY-b *(1/n)X)]*X + b X^ YX =1/nYX -b *(1/n)X^ + b *X^

10 b = YX - n Y * X X^ - n* X ^ = XY - n Y * X +n Y * X - n Y * X X^ - n* X ^+ n* X ^ - n* X ^

11 b = YX - Y *X - Y* X + n Y * X X^ - X* X + n* X ^ b = (YX - Y *X - Y* X + Y * X ) (X^ - X* X + X ^) b = (Y - Y )*(X - X ) (X - X )^

12 Now we turn to BLUE: Best Lnear Unbased Estmators

13 Why mght OLS be a good estmator? Desrable propertes of an estmator: 1) Unbased,.e., expected value of the estmator equals the true parameter value that we want to estmate. ) Precse,.e., the varance of the estmator, s small. Turns out that least squares estmator s: unbased lnear n the y s among lnear unbased estmators, the best has smallest varance. I.e., OLS (ordnary least squares) estmator s BLUE. Ths s the Gauss Markov theorem.

14 Gauss-Markov Theorem states that OLS estmates of the regresson coeffcents are (1) unbased, () consstent and (3) most effcent. Assumng our 6 assumptons from last week are true.

15 Sx Crtcal Assumptons Lnearty Some observed Xs are dfferent Condtonal mean of e, gven X, = 0 Xs are gven, and can be treated as nonrandom All e s are equally dstrbuted wth the same condtonal varance (σ ) [Homoskedastcty (equal scatter] e s are ndependently dstrbuted; cov (e, e j ) = 0

16 To prove unbased-ness we need to remember our assumptons: Four Assumptons Lnearty Some observed Xs are dfferent Condtonal mean of e, gven X, = 0 Xs are gven, and can be treated as nonrandom

17 Defnton of unbased: E(b) = β Begn wth Please note: b ( x x)( y y) ( x x) = 1) The sum of a varable around ts average s always zero That s: ( x) = 0 x ) For convenence, we wll defne w as w x x ( x x) =

18 Usng these notes, we can rewrte b as follows: ( )( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) = = = = = w y y x x x x x x y x x x x x x y y x x x x y y x x b

19 Snce y = β 1 + β x +e and we can smplfy our equaton: b = = w β + β x + e ( ) 1 = wβ + β wx + we = β + w y 1 we

20 We can fnd the expected value of b usng the fact that the expected value of a sum s the sum of the expected values: Eb ( ) = Eb ( + we) = E(β + we+ we we ) 1 1 N N = β Snce E(e ) = 0. = E(β ) + E( we ) + E( w e ) E( w e ) 1 1 = E(β ) + E( we ) = β + we( e) N N

21 Usng our assumpton that the condtonal mean (expected value) of the error terms = 0: E( b ) = β Therefore, OLS estmates are unbased.

22 What about most effcent? Effcency s defned the by sze of the varance. If there are two unbased estmators, the one wth the smallest varance s the most effcent.

23 What about the varance of our estmates? All e s are equally dstrbuted wth the same condtonal varance (σ ) [Homoskedastcty (equal scatter] e s are ndependently dstrbuted; cov (e, e j ) = 0

24 Remember: b = β +we Therefore: var( b ) = E β + we β = E we = E w e + ww ee = w E e + = = σ ( x x) j j j ( ) ( ) ww E ee j j j σ w

25 If we postulate another dfferent estmate, b * whch dffers from b by a constant, w +c ( ) * var( b) = var β + w + c e ( w c ) var ( e ) = + ( w c ) = σ + = σ w + σ ( b ) ( b ) = var + σ var c c

26 What about the varance of our estmates? All e s are equally dstrbuted wth the same condtonal varance (σ ) [Homoskedastcty (equal scatter] e s are ndependently dstrbuted; cov (e, e j ) = 0 Then, among all unbased, lnear combnatons of Ys, our estmates b 1 and b have the lowest varance = most effcent.

27 As for consstency, t s the property that estmates converge to true values as the sample sze s ncreased ndefntely. Smlar to unbased-ness, f our frst four assumptons hold, (especally #4, whch mples X s and e s are uncorrelated), then OLS estmators are consstent.

28 Therefore, OLS estmators are BLUE. What f the assumptons are not true? To be contnued at a later date...

29 Now we combne statstcal analyss wth OLS estmates.

30 s Statstcal Aspects of Regresson b 1 and b are only estmates of β 1 and β Key queston: How accurate are these estmates? Statstcal procedures allow us to formally address ths queston.

31 s The normal dstrbuton of b, the least squares estmator of β, s b N β ~, ( x ) x A standardzed normal random varable s obtaned from b by subtractng ts mean and dvdng by ts standard devaton: Z = σ ( x x) σ b β ~ N ( 0,1)

32 s We know that: ( ) = P Z Substtutng: P 1.96 σ b β 1.96 = ( x x) 0.95 Rearrangng: P 1.96 b σ = ( ) x x β b σ ( x x) 0. 95

33 s The two end-ponts b ±. provde an nterval estmator. ( x ) 1 σ x 96 In repeated samplng 95% of the ntervals constructed ths way wll contan the true value of the parameter β. Ths easy dervaton of an nterval estmator s based on the assumpton SR6 and that we know the varance of the error term σ.

34 s Replacng σ wth t: σˆ creates a random varable t = σˆ σ b β β β = = ~ ( N ) ( x x) vâr( b ) se( b ) b b t ( ) The rato t = b β se b has a t-dstrbuton wth (N ) degrees of freedom, whch we denote as: t t ~ ( N )

35 s In general we can say, f assumptons SR1-SR6 hold n the smple lnear regresson model, then bk β t = k k se ( ) for 1, ( ) ~ t = N b The t-dstrbuton s a bell shaped curve centered at zero It looks lke the standard normal dstrbuton, except t s more spread out, wth a larger varance and thcker tals The shape of the t-dstrbuton s controlled by a sngle parameter called the degrees of freedom, often abbrevated as df k

36 A Confdence Interval for β k Uncertanty about accuracy of the estmate b can be summarsed n a confdence nterval 95% confdence nterval for β s gven by: b β k k P t = α ( ) c tc 1 se bk P b t se b β t + t se b = 1 [ ( ) ( )] α k c k k c c k t c s a crtcal value from the Student t-dstrbuton se b = standard error of s a measure of the accuracy o s b = SSE ( N ) ( X X )

37 A Confdence Interval for β (cont.) t c controls the confdence level (e.g. t b s bgger for 95% confdence than 90%). se vares drectly wth SSE (.e. how varable the resduals are) se vares nversely wth N, the number of data ponts se vares nversely wth (X X ) whch s related to the varance/varablty of X.

38 Intuton of Confdence Interval: Useful (but formally ncorrect) ntuton: There s a 95% probablty that the true value of β les n the confdence nterval. Correct ntuton: If you repeatedly use the above formula for calculatng a confdence nterval, 95% of the ntervals you construct wll contan the true value for β. Can choose any level of confdence you want (e.g. 90%, 99%).

39 EXAMPLE FROM TEXT: b = 10.1, N = 40 df = 38 var(b ) = 4.38 Create a 95% confdence nterval (α =.05) Crtcal value of t =.04 se = (4.38) ½ =.09 A 95% confdence nterval estmate for β : b ( ) = 10.1±.04(.09) [ 5.97,14.45] ± tcse b = When the procedure we used s appled to many random samples of data from the same populaton, then 95% of all the nterval estmates constructed usng ths procedure wll contan the true parameter!!

40 The most common method of evaluatng estmates s through hypothess testng usng a test statstc usng the t dstrbuton.

41 Hypothess Testng Test whether β=0 (.e. whether X has any explanatory power) One way of dong t: look at confdence nterval, check whether t contans zero. If no, then you are confdent β 0. Alternatve (equvalent) way s to use t-statstc (often called t-rato ) bk c t = ~ t( N ) se( b ) f c = 0, then k Bg values for t ndcate β 0. Small values for t ndcate β=0. t = b se

42 Hypothess Testng (cont.) Q: What do we mean by bg and small? A: Look at p-value. If P-value.05 then t s bg and conclude β 0. If P-value >.05 then t s small and conclude β=0. Useful (but formally ncorrect) ntuton: P-value measures the probablty that β = = 5% = level of sgnfcance Other levels of sgnfcance (e.g. 1% or 10%) occasonally used

43 The Test statstc for H 0 : β =c, s: t 0 = (β -c)/se ~ t(df=n-) Rejecton regons exactly the same as before (dependng whether you are dong a one-sded or two-sded test). p-values exactly the same as before (dependng upon t 0 and whether you are dong a one-sded or two-sded test).

44 Components of Hypothess Tests 1. A null hypothess, H 0. An alternatve hypothess, H 1 3. A test statstc 4. A rejecton regon 5. A concluson

45 1. For our purposes, the null hypothess s H 0 = 0. The alternatve hypothess s H Let α =.05. The crtcal values for ths two-tal test are the.5-percentle t (.05,38 ) =.04 and the 97.5-percentle t (.975,38 ) =.04. We REJECT the null hypothess f the calculated value of t.04 or f t.04. If.04 < t <.04, we DO NOT REJECT the Null.

46 A coeffcent s sad to be STATISTICALLY SIGNIFICANT or SIGNIFICANTLY DIFFERENT FROM ZERO at level α (usually, 1%, 5% or 10%) f you reject the null hypothess that the coeffcent s ZERO (generally wth a two-sded test). Ths s what reported t-ratos test.

47 p-value rule: Reject the null hypothess when the p-value s less than, or equal to, the level of sgnfcance α. That s, f p α then reject H 0. If p > α then do not reject H 0

48 The null hypothess s H 0 : β = 0. The alternatve hypothess s H 1 : β 0 Recall t statstc for b : t = 4.88 p-value for H 0 = p P t( 38) P t( 38) = =

49 Returnng to the housng problem: PRICE = SQFT (1.404) (7.41)

50 Next Week More evaluatng results A few words on Lnear Algebra Begn multple regresson models Multple R and F test