INTRODUCTORY ECONOMETRICS Lesson 2b Dr Javier Fernández etpfemaj@ehu.es Dpt. of Econometrics & Statistics UPV EHU c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 1/192
GLRM: the PRF Recall: model with K explanatory variables: Y t = β 0 + β 1 X 1t + + β K X Kt + u t, Y = Xβ + u (2) 2.3b OLS in the GLRM. is called GLRM. Population Regression Function (PRF): E(u)=0 systematic part or PRF: E(Y t )=β 0 + β 1 X 1t + + β K X Kt E(Y )=Xβ Interpretation of the coefficients: β 0 = E(Y t X 1t = X 2t = = X Kt = 0): Expected value of Y t when all explanatory variables are equal to zero. β k = E(Y t) ΔE(Y t), k = 1...K: Increase in (expected) value Y t when X kt ΔX kt X k one unit (c.p.). c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 61/192 c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 62/192 The Sample Regression Function (SRF) Objective of GLRM: To obtain estimator β =( β 0, β 1..., β K ) of unknown parameter vector in (2). β estimated model, fit or SRF: Notes: Disturbances in PRF: Residuals in SRF: Ŷ t = β 0 + β 1 X 1t + + β K X Kt Ŷ = X β u t = Y t E(Y t )=Y t β 0 β 1 X 1t β K X Kt u = Y E(Y )=Y Xβ û t = Y t Ŷ t = Y t β 0 β 1 X 1t β K X Kt û = Y Ŷ = Y X β Estimation: OLS apply Least-Squares fit to GLRM: Y = Xβ + u, either in observation form: or [ in matrix form: recall: min T β 0...β K t=1 u 2 t where u t = Y t β 0 β 1 X 1t β K X Kt u = ( u 1,u 2,...,u T ) so u u = u 2 1 + u2 2 + + u2 T = T t=1 u2 t that is min β ] u 1 u 2 u =... u T u u where u = Y Xβ Residuals are to the SRF what disturbances are to the PRF. c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 63/192 c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 64/192
Note: vector derivatives 1st.o.c. in matrix form Let u = u(β): derivs of cu and cu 2 with respect to β : u (cu)=c c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 65/192 and With vectors and matrices this is quite similar: The derivative of the linear combination with respect to u c (1 n) (n 1) β is: (u c) = u c (k 1) The derivative of the sum of squares u u (1 n) (n 1) u c u2 = 2u u ( = n i=1 c iu i, i.e. scalar!!) u u with respect to β is: (u u) (k 1) ( = n i=1 u2 i, i.e. scalar!!) = 2 u u min u) where u = Y Xβ β First derivatives of SS u u with respect to β : in the minimum: u u = 2 u u = 2 (Y β X ) u = 2X u 1st.o.c.: X (K+1 T) (T 1) c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 66/192 û = 0 K+1 Estimation: Normal equations & LSE of β Estimation: LSE of β (cont) Solving the 1st.o.c. we obtain the normal equations: X (Y X β)=0 X Y = X X β (K+1 1) (K+1 K+1) (K+1 1) (3) where X X is a [K+1 K+1] matrix: X X (K+1 K+1) [recall X & Y? ] T X 1t X 2t... X Kt X 1t X1t 2 X 1t X 2t... X 1t X Kt =... X Kt X Kt X 1t X Kt X 2t... X 2 Kt Whence premultiplying by (X X) 1 we obtain the OLS estimator: β OLS =(X X) 1 X Y and X Y and β are [K+1 1] vectors: X Y (K+1 1) = Y t X 1t Y t... X Kt Y t β (K+1 1) β 0 β 1 =... β K c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 67/192 c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 68/192
OLS estimator with centred (demeaned) data An alternative way to obtain the OLS estimator is β OLS =(x x) 1 x y for the model coefficients. 2.4b Properties of the SRF....together with the estimated intercept obtained from the first normal equation β 0 = Y β 1 X 1 β K X K Note: special case with K = 1 identical formulae as in SLRM!! (Prove it!!) c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 69/192 c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 70/192 Properties of residuals and SRF (1) β β β 0 1. residuals add up to zero: û t = 0 Demo: directly from 1st.o.c.: 2. Ŷ = Y } X û = 0 Ŷ = X β û = Y Ŷ T 1 ût T 1 X 1tû t T 1 X 2tû t... T 1 X Ktû t = 3. the SRF passes thru vector (X 1,...X K,Y ): Y = β 0 + β 1 X 1 + + β K X K 0 0 0... 0 Properties of residuals and SRF (2) 4. residuals orthogonal to explanatory v.: X û = 0 Demo: directly from 1st.o.c. (see 1) or, alternatively: X û = X (Y X β)=x Y X X β = X Y X X(X X) 1 X Y = 0 }{{} =I K+1 5. residuals orthogonal to explained part of Y : Ŷ û = 0 Demo: Ŷ û =(X β) û = β }{{} X û = 0 =0 Note: These properties 1 thru 3 are fulfilled if the regression has an intercept; that is, if X has a column of ones. c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 71/192 c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 72/192
Goodness of fit: R 2 Revisited Recall (same as before but now we ll do it with vectors): Y Y =(Ŷ + û )(Ŷ + û) 2.5b Goodness of Fit: Coefficient of Determination (R 2 ) & Estimation of the Error Variance. = Ŷ Ŷ + û û + 2Ŷ û = Ŷ Ŷ + û û (from prop 5) Y Y TY 2 = Ŷ Ŷ TŶ 2 + û û (from prop 2) y y ( TSS) = ŷ ŷ ( ESS) + u u ( RSS) R 2 = ESS TSS = 1 RSS TSS 0 R 2 1 c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 73/192 c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 74/192 Goodness of fit: R 2 Revisited (cont) Note 1: R 2 measures the proportion of the dependent variable variation explained by the variation of (a linear combination of) the explanatory variables. Note 2: { û t 0, 1st row of 1st.o.c. no intercept Ŷ Y, not validr 2 (Remember!) c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 75/192 Estimation of Var(u t ) σ 2 = Var ( u t ) = E ( u 2 t ) 1 T T t=1 but with residuals, they must satisfy K+1 linear relationships in X û = 0 so we loose K+1 degrees of freedom: σ 2 1 = T K 1 Therefore we propose the following estimator: σ 2 = T ût 2 t=1 RSS T K 1 which clearly is an unbiased estimator: Demo: E ( σ 2) = E( RSS ) ( ) = T K 1 T K 1 T K 1 = σ 2 ( see textbook) c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 76/192 u 2 t
Properties of the OLS Estimator (1) The estimator β OLS =(X X) 1 X Y has the following properties: Linear: β OLS is a linear combination of disturbances: β =(X X) 1 X (Xβ + u) 2.6 Finite-sample Properties of the Least-Squares Estimator. The Gauss-Markov Theorem. = (X X) 1 X Xβ +(X X) 1 X u = β +(X X) 1 X u = β + Γ u Unbiased: Since E ( u ) = 0, β OLS is unbiased: E ( β) ( = E β + Γ u ) = β + Γ E ( u ) = β c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 77/192 c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 78/192 Properties of the OLS Estimator (2) Properties of the OLS Estimator (2cont) Variance: Recall: Var ( u ) = σ 2 I T, β = β +(X X) 1 X u, Var ( β) ( = E ( β β)( β β) ) = E ( (X X) 1 X uu X(X X) 1) =(X X) 1 X E ( uu ) X(X X) 1 =(X X) 1 X σ 2 I T X(X X) 1 = σ 2 (X X) 1 X X(X X) 1 Var ( β0 ) Cov ( β0, β ) 1... Cov ( β0, β ) K Var ( β) Cov ( β1, β ) 0 Var ( β1 )... Cov ( β1, β ) K =... Cov ( βk, β ) 0 Cov ( βk, β ) 1... Var ( βk ) a 00 a 00 a 01... a 0K a 10 a 11 a 12... a 1K σ 2 (X X) 1 = σ 2 a 20 a 21 a 22... a 2K... a K0 a K1 a K2... a KK i.e. a kk is the (k + 1,k + 1)-element of matrix (X X) 1 : Var ( β) = σ 2 (X X) 1 Var ( βk ) = σ 2 a kk Cov ( βk, β i ) = σ 2 a ki c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 79/192 c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 80/192
The Gauss-Markov Theorem Given the basic assumptions of GLRM, the OLS estimator is that of minimum variance (best) among all the linear and unbiased estimators Demo: β OLS =BLUE = B est L inear U nbiased E stimator Let β be some other linear and unbiased estimator: β =D Y = D (Xβ + u)=d Xβ + D u E ( β) =D Xβ + D E ( u ) = D Xβ = β D X = I K then β = β + D u β β = D u and its variance: Var ( β) [ = E ( β β)( β β) ] = E ( D uu D ) The Gauss-Markov Theorem (cont)...the difference between both covariance matrices is a positive definite matrix: Var ( β) Var ( β) = σ 2 D D σ 2 (X X) 1 = σ 2 [ D D (X X) 1] = σ 2 [ D D D X (X X) 1 X D ] = σ 2 D [ I T X (X X) 1 X ] D }{{} M = σ 2 D (MM)D = σ 2 (D M)(M D)=D D > 0 I.e. in particular all individual variances will be bigger than their OLS counterpart. = D E ( uu ) D = D σ 2 I T D = σ 2 D D c J Fernández (EA3-UPV/EHU), February... 21, 2009 c J Introductory Econometrics - p. 81/192 Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 82/192 Useful expressions for SS 2.3c OLS: Useful expressions & Timeline. TSS= (Y t Y ) 2 = Y 2 t TY 2 = Y Y TY 2 ESS = (Ŷ t Ŷ ) 2 = Ŷ t 2 TŶ 2 = Ŷ t 2 TY 2 = Ŷ Ŷ TY 2 =(X β) (X β) TY 2 = β X X }{{} β TY 2 = β X Y TY 2 X Y RSS = ût 2 = û û = Yt 2 Ŷ t 2 = Y Y β X Y c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 83/192 c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 84/192
c J Fernández (EA3-UPV/EHU), February 21, 2009 Introductory Econometrics - p. 85/192 Main expressions & Timeline Y = Xβ + u (X X) 1 X Y β =(X X) 1 X Y ESS = β X Y TY 2 (needs Y!) TSS= Y Y TY 2 RSS = Y Y β X Y (no Y!) R 2 = ESS TSS = 1 RSS TSS σ 2 = RSS T K 1 Var ( β) = σ 2 (X X) 1