Linear Regression with One Regressor

Size: px

Start display at page:

Download "Linear Regression with One Regressor"

Alban Cross
5 years ago
Views:

1 Lear Regresso wth Oe Regressor AIM QA.7. Expla how regresso aalyss ecoometrcs measures the relatoshp betwee depedet ad depedet varables. A regresso aalyss has the goal of measurg how chages oe varable, called a depedet or explaed varable ca be explaed by chages oe or more other varables called the depedet or explaatory varables. The regresso aalyss measures the relatoshp by estmatg a equato (e.g., lear regresso model). The parameters of the equato dcate the relatoshp. AIM QA.7.2 Iterpret a populato regresso fucto, regresso coeffcets, parameters, slope, tercept, ad the error term. The geeral form of the lear regresso model s: Y β 0 +β X +ε Where The subscrpt rus over observatos,,, ; Y s the depedet varable, the regressad, or smply the left-had varable; X s the depedet varable, the regressor, or smply the rght-had varable; β 0 +β X s the populato regresso le or populato regresso fucto; β 0 s the tercept of the populato regresso le (represets the value of Y f X s zero); β s the slope of the populato regresso le (measures the chage Y for a oe ut chage X); ad ε s the error term or ose compoet; ts expected value s zero. Y (Depedet varable) Y β 0 +β X ε ε β 0 (Itercept) β (Slope) X (Idepedet varable) 57

2 AIM QA.7.3 Iterpret a sample regresso fucto, regresso coeffcets, parameters, slope, tercept, ad the error term. The geeral form of the sample regresso fucto s: Where Yˆ b 0 +b X +e The sample regresso coeffcets are b 0 ad b, whch are the tercept ad slope. ˆ The e s called the resdual ( ): e Y - b 0 +b X Sample statstcs (b 0 +b ) are computed as estmates of the parameters β 0 ad β AIM QA.7.4 Descrbe the key propertes of a lear regresso. The Assumptos of the classcal ormal lear regresso model are:. A lear relato exsts betwee the depedet varable ad the depedet varable 2. The depedet varable s ucorrelated wth the error terms 3. The expected value of the error term s zero. E(ε )0 4. Homoskedastcty ( 同 ):The varace of the error term s the same for all observatos. V(ε ) V(ε ) σ 2 ε 5. No seral correlato ( 關 ) of the error terms; The error term s depedet across observatos. Corr(ε, ε j ) 0 6. The error term s ormally dstrbuted 58

3 AIM QA.7.5 Descrbe a ordary least squares (OLS) regresso ad calculate the tercept ad slope of the regresso. AIM QA.7.6 Descrbe the method ad the three key assumptos of OLS for estmato of parameters. AIM QA.7.7 Summarze the beefts of usg OLS estmators. AIM QA.7.8 Descrbe the propertes of OLS estmators ad ther samplg dstrbutos, ad expla the propertes of cosstet estmators geeral. AIM QA.7.9 Iterpret the explaed sum of squares (ESS), the total sum of squares (TSS), ad the resdual sum of squares (RSS), the stadard error of the regresso (SER), ad the regresso R 2 AIM QA.7.0 Iterpret the results of a OLS regresso The mstake made predctg the th observato s Y Yˆ Y (b + b X ) Y b b X 0 0 The sum of these squared predcto mstakes over all observatos s 0 (Y b b X ) 2 The estmators of the tercept ad slope that mmze the sum of squared mstakes are called the ordary least squares (OLS) estmators of β 0 ad β. The OLS estmators of the slope β ad the tercept β 0 are ˆ β (X X )(Y Y ) (X 2 S S XY ˆ X ) XX X βˆ 0 Y βˆ X β X Y XY X 2 2 The OLS predcted values Yˆ ad resduals εˆ are ˆ βˆ + βˆ X,,, Y 0 εˆ Y Yˆ,,, The estmated tercept ( βˆ 0 ), slope ( βˆ ), ad resdual ( εˆ ) are computed from a sample of observatos of X ad Y,,,. These are estmates of the ukow true populato tercept (β 0 ), slpoe (β ), ad error term (ε ). 59

4 The Least Squares Assumptos: () The error term ε has codtoal mea zero gve X:E(ε X ) 0; (2) (X, Y ), I,, are depedet ad detcally dstrbuted (..d.) draws from ther jot dstrbuto; (3) Large outlers are ulkely: X ad Y have ozero fte fourth momets. The explaed sum of squares (ESS) s also kow as sum of squares regresso ˆ (SSR), s the sum of squared devatos of Y from ther average ESS 60 (Yˆ Y ) 2

5 The total sum of squares (TSS) s also kow as sum of squares total (SST), s the sum of squared devatos of Y from ts average. TSS (Y Y ) The resdual sum of squares (RSS) s also kow as sum of squared errors (SSE), s the sum of the squared OLS resduals. RSS (Y Yˆ ) 2 Total sum of squares Explaed sum of squares + Resdual sum of squares (Y Y ) 2 (Yˆ Y ) (Y Yˆ ) 2 TSS ESS + RSS The R 2, coeffcet of determato ( ), s the fracto of the sample varace of Y explaed by (or predcted by) X. R 2 ESS RSS TSS TSS The stadard error of the regresso (SER) s a estmator of the stadard devato of the regresso error ε. SER RSS 2 6

residual. (Note that usually in descriptions of regression analysis, upper-case

residual. (Note that usually in descriptions of regression analysis, upper-case Regresso Aalyss Regresso aalyss fts or derves a model that descres the varato of a respose (or depedet ) varale as a fucto of oe or more predctor (or depedet ) varales. The geeral regresso model s oe of