Econometrics A. Simple linear model (2) Keio University, Faculty of Economics. Simon Clinet (Keio University) Econometrics A October 16, / 11

Size: px

Start display at page:

Download "Econometrics A. Simple linear model (2) Keio University, Faculty of Economics. Simon Clinet (Keio University) Econometrics A October 16, / 11"

Brittney Kelley
5 years ago
Views:

1 Econometrics A Keio University, Faculty of Economics Simple linear model (2) Simon Clinet (Keio University) Econometrics A October 16, / 11

2 Estimation of the noise variance σ 2 In practice σ 2 too is unknown. Definition (Estimator of σ 2 ) We define the estimator of σ 2. ˆσ 2 = 1 n 2 n i=1 (Y i ˆβ 0 ˆβ 1 X i ) 2 = RSS n 2 Simon Clinet (Keio University) Econometrics A October 16, / 11

3 Goodness-of-fit Let Then and we define TSS = ESS = n (Y i Y ) 2, i=1 n (Ŷi Y ) 2, i=1 RSS = n Ûi 2. i=1 TSS = ESS + RSS, Definition (coefficient of determination) R 2 = 1 RSS TSS = ESS TSS. Simon Clinet (Keio University) Econometrics A October 16, / 11

4 Properties of R 2 Here are a few properties for R 2. 0 R 2 1. R 2 = ρ 2 XY. R 2 = 1 iff X perfectly explains Y, i.e. we have a perfectly linear relationship Y i = β 0 + β 1 X i. R 2 = 0 iff X and Y are completely uncorrelated. Simon Clinet (Keio University) Econometrics A October 16, / 11

5 Properties of estimators In statistics, an estimator is a quantity which depends on the data, and which is supposed to be close to the value of an unknown parameter (e.g ˆβ 0 for β 0,...). Estimators are random variables because they depend on the data which is random. For θ a parameter, and ˆθ an estimator, we say that ˆθ is unbiased if E[ˆθ] = θ. ˆθ is consistent if when the sample size n +, P( ˆθ θ > a) 0 for any a > 0. We write ˆθ P θ. ( ˆθ converges in probability to θ ) Therefore, an estimator is consistent if its distribution is more and more located around the target value θ. Simon Clinet (Keio University) Econometrics A October 16, / 11

6 Small sample properties of the LSE ˆβ 0, ˆβ 1, and ˆσ 2 are conditionally unbiased (e.g E[ ˆβ 0 X ] = β 0 ). Var[ ˆβ 0 X ] = Var[ ˆβ 1 X ] = σ 2 σ2 n i=1 X 2 i n n i=1 (X i X ) 2. n i=1 (X i X ) 2. In practice we also estimate the standard deviations of the LSE: Standard deviation of the LSE SE[ ˆβ ˆσ 0 ] = 2 n i=1 X i 2 n n i=1 (X i X ), SE[ ˆβ ˆσ 2 1 ] = 2 n i=1 (X i X ). 2 Simon Clinet (Keio University) Econometrics A October 16, / 11

7 Gauss-Markov Theorem Gauss-Markov Theorem The LSE ( ˆβ 0, ˆβ 1 ) is BLUE: Best Linear Unbiased Estimator. This means that the variance of the LSE is minimal among all linear unbiased estimators. Simon Clinet (Keio University) Econometrics A October 16, / 11

8 Small sample properties of the LSE Under [A4] (normal assumption), we have ˆβ 0 β 0 SE[ ˆβ 0 ] X t(n 2), ˆβ 1 β 1 X t(n 2), SE[ ˆβ 1 ] where t(n 2) is a student distribution with n 2 degrees of freedom. In other words, we know the exact distribution of the LSE. Simon Clinet (Keio University) Econometrics A October 16, / 11

9 Large sample properties of the LSE We look at the properties of the LSE when n +. Then: ˆβ 0, ˆβ 1, and ˆσ 2 are consistent (e.g ˆβ 0 P β 0 ). ˆβ 0, ˆβ 1 are asymptotically normal: ˆβ 0 β 0 SE[ ˆβ 0 ] d N (0, 1), ˆβ 1 β 1 SE[ ˆβ 1 ] d N (0, 1). Note: d N (0, 1) means that the the distribution of the left-hand side converges to the standard normal distribution. Simon Clinet (Keio University) Econometrics A October 16, / 11

10 Application: Confidence intervals for small sample Under [A4], as a consequence of the distribution of ( ˆβ 0, ˆβ 1 ) derived in slide 8, we deduce that [ ˆβ i t (1+α)/2 SE[ ˆβ i ], ˆβ i + t (1+α)/2 SE[ ˆβ i ]] is a confidence interval of level α [0, 1] for β i. In the above expression, t 1+α/2 is the (1 + α)/2-quantile of the t(n 2) distribution, i.e. if T t(n 2), P(T t β ) = β. Remark: In practice, we often take α = 95% Simon Clinet (Keio University) Econometrics A October 16, / 11

11 Application: Confidence intervals for large sample As a consequence of the distribution of ( ˆβ 0, ˆβ 1 ) derived in slide 8, we deduce that, when n +, [ ˆβ i z (1+α)/2 SE[ ˆβ i ], ˆβ i + z (1+α)/2 SE[ ˆβ i ]] is a confidence interval of level α [0, 1] for β i. In the above expression, z 1+α/2 is the (1 + α)/2-quantile of the N (0, 1) distribution, i.e. if Z N (0, 1), P(Z z β ) = β. Remark: In practice, we often take α = 95%, and then z (1+α)/2 = Simon Clinet (Keio University) Econometrics A October 16, / 11

Simple Linear Regression: The Model

Simple Linear Regression: The Model task: quantifying the effect of change X in X on Y, with some constant β 1 : Y = β 1 X, linear relationship between X and Y, however, relationship subject to a random