ECON 482 / WH Hong The Simple Regression Model 1. Definition of the Simple Regression Model

Size: px

Start display at page:

Download "ECON 482 / WH Hong The Simple Regression Model 1. Definition of the Simple Regression Model"

Mervyn Arnold
5 years ago
Views:

1 ECON 48 / WH Hog The Smple Regresso Model. Defto of the Smple Regresso Model Smple Regresso Model Expla varable y terms of varable x y = β + β x+ u y : depedet varable, explaed varable, respose varable, regressad,... x : depedet varable, explaatory varable, cotrol varable, regressor,... β : tercept β : slope parameter u : error term or dsturbace; represet factors other tha x that affect y (uobserved)

2 ECON 48 / WH Hog The Smple Regresso Model Iterpretato Holdg other factors costat ( u fxed), the effect of x o y y Δ y = βδ x f Δ u = <=> = β x u as log as = x - Our terest: By how much does the depedet varable chage f the depedet varable s creased by oe ut. => β - Ths terpretato oly correct f all other thgs rema equal whe the depedet varable s u creased by oe ut. => = x (Example) Soybea yeld ad fertlzer: yeld = β + βfertlzer + u β : measures the effect of fertlzer o yeld, holdg all other factors fxed u : cludes rafall, lad qualty, ad so o.

3 ECON 48 / WH Hog The Smple Regresso Model Codto for a causal terpretato <Def> Zero codtoal mea depedece assumpto E( u x ) = - The average value of the uobservables s the same across all slces of the value of x ad the commo average s ecessarly equal to zero. - The explaatory varable must ot cota formato about the mea of the uobserved factors. Note: ) Ths guaratees ) E( u ) = ad mples ) ( ) cov ux, = ) Idepedece betwee u ad ot true x guaratees the mea depedece, but the coverse s (Example) Wage equato: wage = β + βeduc + u where u : ablty, ad so o. The codtoal mea depedece assumpto s ulkely to hold because dvduals wth more educato wll also be more tellget o average 3

4 ECON 48 / WH Hog The Smple Regresso Model Populato Regresso Fucto (PRF) The codtoal mea depedece mples that ( ) = ( β + β + ) = β + β + ( ) E yx E x ux x E ux = β + β x Ths meas that the average value of the depedet varable ca be expressed as a lear fucto of the explaatory varable 4

5 ECON 48 / WH Hog The Smple Regresso Model. Dervg the Ordary Least Squares Estmates Sample data: ( ) { x, y :,..., } = from the populato equato y = β + βx + u 5

6 ECON 48 / WH Hog The Smple Regresso Model Data Plot Populato equato: y = β + βx + u Sample equato: y ˆ ˆ ˆ ˆ ˆ = β + βx + u = y + u where y ˆ ˆ ˆ = β x + β ; ftted values We eed to estmate ˆβ ad ˆβ whch represet the populato relatoshp. The how? => Ft as good as possble a regresso le through the data pots 6

7 ECON 48 / WH Hog The Smple Regresso Model Ordary Least Square estmates y = yˆ u ˆ + ; y ˆ ˆ ˆ = β + βx - explaed by x, uˆ y ˆ = y - uexplaed. Set up the loss fucto whch represets the uexplaed parts ad fd ˆβ ad ˆβ whch mmzes the loss fucto A commo loss fucto s ˆ β, ˆ β Q( ˆ β ˆ, β) = uˆ ; Sum of Squared Resduals = ( ˆ β ˆ β ) = ˆ ( ˆ ˆ ) = β β m Q, u y x = = Frst Order Codtos (FOCs): Q Q ( ˆ β, ˆ β) ˆ β = ( ˆ β, ˆ β) ˆ ( y ˆ ˆ β βx) ( ) = = => y ˆ ˆ ˆ β βx = u β = = () = = ( ˆ ˆ y β βx) ( β β ) = -----() = x = => y ˆ ˆ ˆ x x = ux = = 7

8 ECON 48 / WH Hog The Smple Regresso Model Ordary Least Square estmates(cot'd) ( ) y ˆ β ˆ = From (), y ˆ β ˆ β x = mples that From (), ( β β ) = y y + y ˆ ˆ x x = = + β x or ˆ β ˆ = y β x y ˆ ˆ ˆ ˆ ˆ y + + x x x = y y x x x = => = = => ( β β β β ) β ( ) ˆ = = β ( y ) = ( x ) y x x x Here, ote that ( x x) x = ( x x) = = Therefore, The OLS estmates are ad ( y y) x = ( x x)( y y) ˆ = = β ( x x)( y y) = ( x x) = = ad ˆ β ˆ = y β x 8

9 ECON 48 / WH Hog The Smple Regresso Model Summary of the OLS estmates () ( )( ) ( ) ( ) x x y y x x y ˆ = = β = = = = ( ) x x x x ad () ˆ β ˆ = y β x Dgresso: Why ot mmze some other fucto of the resduals such as the absolute values of the resduals? => Actually, there s. However, NO closed-form soluto. Computatoally dffcult. (Example) Wage ad educato wage = β + β educ + u ; wage - hourly wage dollars, educ - year of educato Ftted regresso: wage = educ Iterpretato: I the sample, oe more year of educato s assocated wth a crease hourly wage by $.54 9

10 ECON 48 / WH Hog The Smple Regresso Model 3. Algebrac Propertes of OLS () The sum ad the sample average of the OLS resdual s zero. = uˆ = () The sample covarace betwee the regressors ad the OLS resduals s zero = xuˆ = (3) The pot ( x, y ) s always o the OLS regresso le. y = ˆ β + ˆ β x Ftted values - y ˆ ˆ ˆ = β + βx ; Resduals - uˆ = y yˆ (4) ŷ = y (5) The sample covarace betwee y ad u s zero. ˆ ˆ = yu ˆˆ=

11 ECON 48 / WH Hog The Smple Regresso Model Goodess-of-Ft How well does the explaatory varable expla the depedet varable? Measures of Varato () total sum of squares (SST): SST = ( y y ) = - represets total varato depedet varable () explaed sum of squares (SSE): SSE = ( yˆ y ) - represets varato explaed by regresso () resdual sum of squares (SSR): SSR = uˆ = = - represets varato ot explaed by regresso

12 ECON 48 / WH Hog The Smple Regresso Model Goodess-of-Ft (Cot'd) Decomposto of total varato: SST = SSE + SSR Proof. ( y y ) = ( y yˆ ) + ( yˆ y ) = uˆ + ( yˆ y ) = = = uˆ ˆ ˆ ˆ u y y y y = = = = SSR + SSE Note that u ( y y ) = ( ) ( ) = + + ˆ ˆ = Goodess-of-Ft measure (R-squared) R SSE SSR = = SST SST - R-squared measures the fracto of the total varato that s explaed by the regresso Cauto: A hgh R-squared does ot ecessarly mea that the regresso has a causal terpretato.

13 ECON 48 / WH Hog The Smple Regresso Model 4. Uts of Measuremets ad Fuctoal Form () Chages uts of measuremets salary = ˆ β + ˆ β roe salary - measured thousads of dollars; roe - retur of equty, measured percetage () What happes o ˆβ ad ˆβ f salary s measured dollars? => Both parameters should be multpled by. () What happes o ˆβ ad ˆβ f roe s dvded by? => Oly ˆβ should be multpled by. 3

14 ECON 48 / WH Hog The Smple Regresso Model () Icorporatg oleartes: Sem-Logarthmc form Regresso of log wages o years of educato ( ) β log wage = + β educ + u Ths chages the terpretato of the regresso coeffcet: Ftted regresso 4

ECON 48 / WH Hog The Smple Regresso Model (3) Icorporatg oleartes: Log-Logarthmc form CEO salary ad frm sales log ( ) β β log( ) salary = + sales + u Ths chages the terpretato of the regresso

15 ECON 48 / WH Hog The Smple Regresso Model (3) Icorporatg oleartes: Log-Logarthmc form CEO salary ad frm sales log ( ) β β log( ) salary = + sales + u Ths chages the terpretato of the regresso coeffcet: Ftted regresso: log ( salary) = log ( sales ) For example: salary +,57$ salary,,$ +.57% salary = = =.57 sales,,$ + % sales sales,,,$ Note: the log-log postulates a costat elastcty model, whereas the sem-log form assumes a sem-elastcty model. 5

16 ECON 48 / WH Hog The Smple Regresso Model 5. Statstcal Propertes of OLS The Gauss-Markov Assumptos for Smple Regresso SLR.. Lear parameters: y = β + βx+ u SLR.. Radom samplg: A radom sample of sze, { x, y :,..., } ( ) model y = β + βx + u. SLR. 3. Sample varato explaatory varable =, radomly draw from the populato The sample outcomes o x are ot all the same value. Or ( x x) SLR. 4. Zero codtoal mea: E( u x ) = = > The value of the explaatory varable must cota o formato about the mea of the uobserved factors. SLR. 5. Homeskedastcty: ( u x ) = σ var 6

17 ECON 48 / WH Hog The Smple Regresso Model Ubasedess of OLS Populato model: y = β + βx + u where s a error Estmated model: y ˆ ˆ ˆ = β + βx + u where u s a resdual OLS estmator: ( )( ) ( ) u ˆ ( ) x x y y x x y ˆ = = β = = = = ( ) x x x x ad ˆ β ˆ = y β x Need to show that E ( ˆ β ) = β ad E ( ˆ β ) = β 7

18 ECON 48 / WH Hog The Smple Regresso Model () Ubasedess of Proof. ˆβ. E ( ˆ β ) = β Rewrte ˆ = β = ( x ) SST x y x where SST = ( x x ) x =. Focusg o the umerator, we ca wrte: ( x x) y = ( x x)( β + βx + u ) --- SLR = = ( x x) β ( x x) x ( x x ) = β + + = = = Note that ( x x) = ad ( ) ( ) = Therefore, ˆ β = β + ( / ) ( ) x x x = x x = SST x = = SSTx x x u --- SLR 3 = ˆ ( ) ( ) β = β+ / ( ) = β+ ( / ) ( ) E x E SSTx x x u x E SSTx x x u x = = ( / SSTx) ( x x) E( u x) ( / SSTx) ( x x) --- SLR ad 4 = β + = β + = β = = u 8

19 ECON 48 / WH Hog The Smple Regresso Model () Ubasedess of Proof. ˆβ. E ( ˆ β ) = β ( ) ˆ β = y ˆ β x = β + β x + u ˆ β x = β + β ˆ β x + u --- SLR ( ) ( ) E ˆ β x = E β β ˆ β x ux β E β ˆ β xx + + = + + E ux ˆ ( ) ( ) = β + xe β β x + E u x =β --- SLR ad 4 Iterpretato of ubasedess The estmated coeffcets may be smaller or larger, depedg o the sample that s the result of a radom draw However, o average, they wll be equal to the values that characterze the true relatoshp betwee y ad x the populato I a gve sample, estmates may dffer cosderably from true values. 9

20 ECON 48 / WH Hog The Smple Regresso Model Varace of the OLS Estmators Depedg o the sample, the estmates wll be earer or farther away from the true populato values. How far ca we expect our estmates to be away from the true populato value o average? More o SLR 5, homoskedastcty: ( u x ) = σ var Note: Homoskedastcy assumpto s ot ecessary to derve the estmated varace but reduces the computatoal burde.

21 ECON 48 / WH Hog The Smple Regresso Model A example for heteroskedastcty: wage ad educato var( u x ) depeds o the varablty of x More educato would cause more varablty of wages, mplyg that error varace o wages creases wth educato.

22 ECON 48 / WH Hog The Smple Regresso Model Samplg varace of the OLS estmators Frst, assume that the populato error varace, OLS estmators are: Proof. var ( ˆ β ) σ = = = ( x x) σ SST x ad var( ˆ ) σ, s kow. The the samplg varace of the σ x = β = = ( x x ) From the prevous result, we kow that ˆ β = β + ( / ) ( ) Uder SLR. through SLR. 5, SST x x u. x = ˆ ( β ) = β+ ( ) ( ) = ( ) ( ) var x var / SSTx x x u x var / SSTx x x u x = = = = ( / SSTx) var ( x x ) u x ( / SSTx) ( x x ) var( u x) = = σ σ = SSTx ( / SST ) ( x x ) = = x

23 ECON 48 / WH Hog The Smple Regresso Model ˆ ( x) var β =? => Homework! var σ x = β ˆ ( x) = SST x Implcato of the samplg varaces The samplg varablty of the estmated regresso coeffcets wll be the hgher the larger the varablty of the uobserved factors, ad the lower, the hgher the varato the explaatory varable. 3

24 ECON 48 / WH Hog The Smple Regresso Model Estmatg the error varace I practce, σ s ot kow. Thus, we eed to estmate the error varace from the resduals. Error varace: var ( u x ) = σ = var ( u ) = uˆ ˆ ˆ u = u = = Varace of resduals σ ( ) Possble try, but ufortuately ths estmator s based Sample varace of resdual: ˆ σ = uˆ = A ubased estmator by adjustg the bas of varace of resduals degree of freedom - Note: It ca be show that uder SLR. - SLR. 5, the sample varace of resdual s ubased,.e., ( ˆ ) E σ = σ. (Theorem.3 the Wooldrdge textbook.) 4

Linear Regression with One Regressor

Linear Regression with One Regressor Lear Regresso wth Oe Regressor AIM QA.7. Expla how regresso aalyss ecoometrcs measures the relatoshp betwee depedet ad depedet varables. A regresso aalyss has the goal of measurg how chages oe varable,