Applied Econometrics (QEM)

Transcription

1 Applied Econometrics (QEM) The Simple Linear Regression Model based on Prinicples of Econometrics Jakub Mućk Department of Quantitative Economics Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 1 / 25

2 Intuition Outline 1 Intuition 2 Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 2 / 25

3 Intuition The starting point conditional distribution of Y given X. f(y x=a) f(y x) µ y a y Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 3 / 25

4 Intuition The starting point conditional distribution of Y given X. f(y x=a) f(y x=b) f(y x) µ y a µ y b y Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 3 / 25

5 Intuition E(y x) x E(y x) β 1 = E(y x) x Simply Regression: E (y x) = β 0 +β 1 x x Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 4 / 25

6 Intuition E(y x) β 1 x E(y x) β 1 = E(y x) x Simply Regression: E (y x) = β 0 +β 1 x x Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 4 / 25

7 Intuition : where y is the (outcome) dependent variable; x is independent variable; ε is the error term. y = β 0 + β 1 x + ε (1) The dependent variable is explained with the components that vary with the the dependent variable and the error term. β 0 is the intercept. β 1 is the coefficient (slope) on x. β 1 measures the effect of change in x upon the expected value of y (ceteris paribus). Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 5 / 25

8 Intuition : where y is the (outcome) dependent variable; x is independent variable; ε is the error term. y = β 0 + β 1 x + ε (1) The dependent variable is explained with the components that vary with the the dependent variable and the error term. β 0 is the intercept. β 1 is the coefficient (slope) on x. β 1 measures the effect of change in x upon the expected value of y (ceteris paribus). Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 5 / 25

9 Outline 1 Intuition 2 Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 6 / 25

10 How to estimate the slope and intercept? y (dependent variable) x (independent variable) Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 7 / 25

11 How to estimate the slope and intercept? y (dependent variable) x (independent variable) Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 7 / 25

12 I Assumption #1: true DGP (data generating process): y = β 0 + β 1 x + ε. (2) Assumption #2: the expected value of the error term is zero: E (ε) = 0, (3) and this implies that E (y) = β 0 + β 1 x. Assumption #3 the variance of the error term equals σ: var (ε) = σ 2 = var (y). (4) Assumption #4: the covariance between any pair of ε i and ε j is zero cov (ε i, ε j ) = 0, (5) and this implies that cov (y i, y j ). Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 8 / 25

13 II Assumption #5: Exogeneity. The independent variable is not random and it takes at least two values. Assumption #6 (optional): the normally distributed error term: ε N ( 0, σ 2). (6) Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 9 / 25

14 Fitted values and residuals The fitted values of dependent variable (ŷ i ): ŷ i = ˆβ 0 + ˆβ 1 x i (7) where ˆβ 0 and ˆβ 1 are estimates of intercept and slope, respectively. The residuals (ê i ) : ê i = y i ŷ i = y i ˆβ 0 ˆβ 1 x i, (8) are residuals between observed (empirical) and fitted values of dependent variable. Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 10 / 25

15 The sum of squared residuals (SSE): SSE = N êi 2 = i N (y i ŷ i ) 2. (9) i The SSE can be expressed as function of the parameters β 0 and β 1 : SSE (β 0, β 1 ) = N êi 2 = i N i ( y i ˆβ 0 ˆβ 1 x i ) 2. (10) The least squares principle is a method of the parameter selection that provides the lowest SSE: min β 0,β 1 N i ( y i ˆβ 0 ˆβ 1 x i ) 2. (11) In other words, the least squares principle minimizes the SSE. Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 11 / 25

16 The least squares estimator y (dependent variable) x (independent variable) Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 12 / 25

17 The least squares estimator y (dependent variable) x (independent variable) The LS estimators minimizes the sum of squared residuals (SSE). Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 12 / 25

18 The least squares estimator for the simple regression model: ˆβ LS ˆβ LS 0 = ȳ 1 x, (12) N ˆβ 1 LS i (x i x) (y i ȳ) = N i (x i x) 2. (13) where ȳ and x are the sample averages of dependent and independent variables, respectively. Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 13 / 25

19 Under the assumptions A#1-A#5 of the simple linear regression LS LS model, the least squares estimators ˆβ 0 and ˆβ 1 have the smallest variance of all linear and unbiased estimators of β 0 and β 1. ˆβ LS 0 and and β 1. ˆβ LS 1 are the Best Linear Unbiased Estimators (BLUE) of β 0 Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 14 / 25

20 Remarks on the I ˆβ LS 1 LS The estimators 0 and ˆβ 1 are best when compared to linear and unbiased estimators. Based on the Gauss-Markov theorem we cannot claim that the estimators ˆβ 0 and ˆβ 1 are the best of all possible estimators. LS LS 2 LS LS Why the estimators ˆβ 0 and ˆβ 1 are best? Because they have the minimum variance. 3 The Gauss-Markov theorem holds if assumptions A#1-A#5 are satisfied. LS LS If not, then ˆβ 0 and ˆβ 1 are not BLUE. 4 The Gauss-Markov theorem does not require the assumption of normality (A#6) Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 15 / 25

21 Linearity of estimator The least squares estimator of β 1 : N ˆβ 2 LS i (x i x) (y i ȳ) = N i (x i x) 2 (14) can be rewritten as: ˆβ LS 2 = N w i y i, (15) i=1 where w i = (x i x) / (x i x) 2. After manipulation we get: ˆβ LS 2 = β 2 + N w i ε i. (16) Since the w i are known this is linear function of random variable (ε). i=1 Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 16 / 25

22 Unbiasedness The estimator is unbiased if its expected value equals the true value, i.e., ( ) E ˆβ = β. (17) For the least squares estimator: ( ) ( ( ) N N ) E ˆβLS 2 = E β 2 + w i ε i = E (β 2 ) + E w i ε i = β 2 + i=1 N w i E (ε i ) = β 2. i=1 In the above manipulation, we take the advantage of two assumption: (i) E(ε i ) = 0, and (ii) E(w i ε i ) = w i E(ε i ). The latter assumption is equivalent the exogeneity of the independent variable. The unbiasedness is mostly about the average of our estimates from many samples (drawn form the same population). Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 17 / 25 i=1

23 Example: unbiased estimator f(θ) θ = θ^ θ Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 18 / 25

24 Example: biased estimator f(θ) θ^ θ θ Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 19 / 25

25 The variance and covariance of the LS estimators I In general, variance measures efficiency. If the assumption A#1-A#5 are satisfied then: var var ( ) ˆβLS 0 ( ) ˆβLS 1 ( ) cov ˆβLS 0, β1 LS [ N = σ 2 i=1 x2 i N N i=1 (x i x) 2 σ 2 = N i=1 (x i x) 2 [ ] = σ 2 x (x i x) 2 The greater the variance of the error term (σ 2 ), i.e., the larger role of the error term, the larger variance and covariance of estimates. The larger variability of the dependent variable N i=1 (x i x) 2, the smaller variance of the least squares estimators. ] Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 20 / 25

26 The variance and covariance of the LS estimators II The larger sample size (N ) the smaller variance of the least squares estimators. The larger N i=1 the greater variance of the intercept estimator The covariance of estimator has a sign opposite to that of x and if x is larger then the covriance is greater. Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 21 / 25

27 Example: efficiency f(θ) θ^a = θ θ^b = θ θ Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 22 / 25

28 The probability distribution of the least squares estimators If the assumption of normality is satisfied then: ( ) ˆβ 0 LS N ˆβLS LS 0, var( ˆβ 0 ) ( ) ˆβ 1 LS N ˆβLS LS 1, var( ˆβ 1 ) (18) (19) What if the assumption of normality does not hold? If assumptions A#1-A#5 are satisfied and if the sample (N ) is sufficiently large, the least squares estimators, i.e., β0 LS and β1 LS, have distribution that approximates the normal distributions described above. Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 23 / 25

29 Estimating the variance of the error term The variance of the error term: var(ε i ) = σ 2 = E [ε i E(ε i )] 2 = E(ε i ) 2 (20) since we have assumed that E(ε i ) = 0. The estimates of the error term variance based on the residuals: where ê i = y ŷ i. ˆσ 2 = 1 N 2 N êi 2. (21) The ˆσ 2 can be directly used to estimates the variance/covariance of the least squares estimator. i=1 Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 24 / 25

30 Estimating the variance of the least squares estimators ˆβ LS 1 ) the estimated vari- LS To obtain estimates of the var( ˆβ ance of the error term is used (ˆσ 2 ): var ˆ var ˆ ( ) ˆβLS 0 ( ) ˆβLS 1 ( ) cov ˆ ˆβLS 0, β1 LS 0 ) and var( [ N = ˆσ 2 i=1 x2 i N N i=1 (x i x) 2 ˆσ 2 = N i=1 (x i x) 2 [ ] = ˆσ 2 x (x i x) 2 Based on the variance we can calculate the standard errors are simply the standard deviation of the estimators: ( ) ( ) ( ) ( ) ŝe ˆβLS 0 = var ˆ ˆβLS and ŝe ˆβLS 1 = var ˆ ˆβLS. (22) 0 ] 1 Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple Linear Regression Model 25 / 25