Econometric Analysis Fifth Edition

Transcription

1 Solutios Maual Ecoometric Aalysis Fifth Editio William H. Greee New York Uiversity Pretice Hall, Upper Saddle River, New Jersey 07458

2 Cotets ad Notatio Chapter Itroductio Chapter The Classical Multiple Liear Regressio Model Chapter 3 Least Squares 3 Chapter 4 Fiite-Sample Properties of the Least Squares Estimator 7 Chapter 5 Large-Sample Properties of the Least Squares ad Istrumetal Variables Estimators 4 Chapter 6 Iferece ad Predictio 9 Chapter 7 Fuctioal Form ad Structural Chage 3 Chapter 8 Specificatio Aalysis ad Model Selectio 30 Chapter 9 Noliear Regressio Models 3 Chapter 0 Nospherical Disturbaces - The Geeralized Regressio Model 37 Chapter Heteroscedasticity 4 Chapter Serial Correlatio 49 Chapter 3 Models for Pael Data 53 Chapter 4 Systems of Regressio Equatios 63 Chapter 5 Simultaeous Equatios Models 7 Chapter 6 Estimatio Frameworks i Ecoometrics 78 Chapter 7 Maximum Likelihood Estimatio 84 Chapter 8 The Geeralized Method of Momets 93 Chapter 9 Models with Lagged Variables 97 Chapter 0 Time Series Models 0 Chapter Models for Discrete Choice 06 Chapter Limited Depedet Variable ad Duratio Models Appedix A Matrix Algebra 5 Appedix B Probability ad Distributio Theory 3 Appedix C Estimatio ad Iferece 34 Appedix D Large Sample Distributio Theory 45 Appedix E Computatio ad Optimizatio 46 I the solutios, we deote: scalar values with italic, lower case letters, as i a or α colum vectors with boldface lower case letters, as i b, row vectors as trasposed colum vectors, as i b, sigle populatio parameters with greek letters, as i β, sample estimates of parameters with Eglish letters, as i b as a estimate of β, sample estimates of populatio parameters with a caret, as i α ˆ matrices with boldface upper case letters, as i M or Σ, cross sectio observatios with subscript i, time series observatios with subscript t. These are cosistet with the otatio used i the text.

3 Chapter Itroductio There are o exercises i Chapter.

4 Chapter The Classical Multiple Liear Regressio Model There are o exercises i Chapter.

5 Chapter 3 Least Squares x. (a) Let X =. The ormal equatios are give by (3-),, hece for each of the.. Xe = 0 x colums of X, xk, we kow that x x k e=0. This implies that e = 0 ad = 0. (b) Use i i i i i i e i e = 0 to coclude from the first ormal equatio that a = y bx. (c) Kow that e = 0 ad = 0. It follows the that x e ( ) 0 i i i i i x i i x ei x x y = or ( x x) ( y y b( x x ) = 0 i i i a bx i follows. implies ( )( ) 0 =. Further, the latter i i i i from which the result. Suppose b is the least squares coefficiet vector i the regressio of y o X ad c is ay other Kx vector. Prove that the differece i the two sums of squared residuals is (y-xc) (y-xc) - (y-xb) (y-xb) = (c - b) X X(c - b). Prove that this differece is positive. Write c as b + (c - b). The, the sum of squared residuals based o c is (y - Xc) (y - Xc) = [y - X(b + (c - b))] [y - X(b + (c - b))] = [(y - Xb) + X(c - b)] [(y - Xb) + X(c - b)] = (y - Xb) (y - Xb) + (c - b) X X(c - b) + (c - b) X (y - Xb). But, the third term is zero, as (c - b) X (y - Xb) = (c - b)x e = 0. Therefore, (y - Xc) (y - Xc) = e e + (c - b) X X(c - b) or (y - Xc) (y - Xc) - e e = (c - b) X X(c - b). The right had side ca be writte as d d where d = X(c - b), so it is ecessarily positive. This cofirms what we kew at the outset, least squares is least squares. 3. Cosider the least squares regressio of y o K variables (with a costat), X. Cosider a alterative set of regressors, Z = XP, where P is a osigular matrix. Thus, each colum of Z is a mixture of some of the colums of X. Prove that the residual vectors i the regressios of y o X ad y o Z are idetical. What relevace does this have to the questio of chagig the fit of a regressio by chagig the uits of measuremet of the idepedet variables? The residual vector i the regressio of y o X is M X y = [I - X(X X) - X ]y. The residual vector i the regressio of y o Z is M Z y = [I - Z(Z Z) - Z ]y = [I - XP((XP) (XP)) - (XP) )y = [I - XPP - (X X) - (P ) - P X )y = M X y Sice the residual vectors are idetical, the fits must be as well. Chagig the uits of measuremet of the regressors is equivalet to postmultiplyig by a diagoal P matrix whose kth diagoal elemet is the scale factor to be applied to the kth variable ( if it is to be uchaged). It follows from the result above that this will ot chage the fit of the regressio. 4. I the least squares regressio of y o a costat ad X, i order to compute the regressio coefficiets o X, we ca first trasform y to deviatios from the mea, y, ad, likewise, trasform each colum of X to deviatios from the respective colum meas; secod, regress the trasformed y o the trasformed X without a costat. Do we get the same result if we oly trasform y? What if we oly trasform X? 3

6 I the regressio of y o i ad X, the coefficiets o X are b = (X M 0 X) - X M 0 y. M 0 = I - i(i i) - i is the matrix which trasforms observatios ito deviatios from their colum meas. Sice M 0 is idempotet ad symmetric we may also write the precedig as [(X M 0 )(M 0 X)] - (X M 0 M 0 y) which implies that the regressio of M 0 y o M 0 X produces the least squares slopes. If oly X is trasformed to deviatios, we would compute [(X M 0 )(M 0 X)] - (X M 0 )y but, of course, this is idetical. However, if oly y is trasformed, the result is (X X) - X M 0 y which is likely to be quite differet. We ca exted the result i (6-4) to derive what is produced by this computatio. I the formulatio, we let X be X ad X is the colum of oes, so that b is the least squares itercept. Thus, the coefficiet vector b defied above would be b = (X X) - X (y - ai). But, a = y - b x so b = (X X) - X (y - i( y - b x )). We ca partitio this result to produce (X X) - X (y - i y )= b - (X X) - X i(b x )= (I - (X X) - x x )b. (The last result follows from X i = x.) This does ot provide much guidace, of course, beyod the observatio that if the meas of the regressors are ot zero, the resultig slope vector will differ from the correct least squares coefficiet vector. 5. What is the result of the matrix product M M where M is defied i (3-9) ad M is defied i (3-4)? M M = (I - X (X X ) - X )(I - X(X X) - X ) = M - X (X X ) - X M There is o eed to multiply out the secod term. Each colum of MX is the vector of residuals i the regressio of the correspodig colum of X o all of the colums i X. Sice that x is oe of the colums i X, this regressio provides a perfect fit, so the residuals are zero. Thus, MX is a matrix of zeroes which implies that M M = M. 6. Addig a observatio. A data set cosists of observatios o X ad y. The least squares estimator based o these observatios is b = ( X X ) X y. Aother observatio, x s ad y s, becomes available. Prove that the least squares estimator computed usig this additioal observatio is b s, = b + ( ) ( s y s s + x ( X X ) x X X x x b ). s s Note that the last term is e s, the residual from the predictio of y s usig the coefficiets based o X ad b. Coclude that the ew data chage the results of least squares oly if the ew observatio o y caot be perfectly predicted usig the iformatio already i had. 7. A commo strategy for hadlig a case i which a observatio is missig data for oe or more variables is to fill those missig variables with 0s or add a variable to the model that takes the value for that oe observatio ad 0 for all other observatios. Show that this strategy is equivalet to discardig the observatio as regards the computatio of b but it does have a effect o R. Cosider the special case i which X cotais oly a costat ad oe variable. Show that replacig the missig values of X with the mea of the complete observatios has the same effect as addig the ew variable. 8. Let Y deote total expediture o cosumer durables, odurables, ad services, ad E d, E, ad E s are the expeditures o the three categories. As defied, Y = E d + E + E s. Now, cosider the expediture system E d = α d + β d Y + γ dd P d + γ d P + γ ds P s + εγ d E = α + β Y + γ d P d + γ P + γ s P s + ε E s = α s + β s Y + γ sd P d + γ s P + γ ss P s + ε s. Prove that if all equatios are estimated by ordiary least squares, the the sum of the icome coefficiets will be ad the four other colum sums i the precedig model will be zero. For coveiece, reorder the variables so that X = [i, P d, P, P s, Y]. The three depedet variables are E d, E, ad E s, ad Y = E d + E + E s. The coefficiet vectors are b d = (X X) - X E d, b = (X X) - X E, ad b s = (X X) - X E s. The sum of the three vectors is b = (X X) - X [E d + E + E s ] = (X X) - X Y. Now, Y is the last colum of X, so the precedig sum is the vector of least squares coefficiets i the regressio of the last colum of X o all of the colums of X, icludig the last. Of course, we get a perfect 4

7 fit. I additio, X [E d + E + E s ] is the last colum of X X, so the matrix product is equal to the last colum of a idetity matrix. Thus, the sum of the coefficiets o all variables except icome is 0, while that o icome is. 9. Prove that the adjusted R i (3-30) rises (falls) whe variable x k is deleted from the regressio if the square of the t ratio o x k i the multiple regressio is less (greater) tha oe. The proof draws o the results of the previous problem. Let RK deote the adjusted R i the full regressio o K variables icludig x k, ad let R deote the adjusted R i the short regressio o K- variables whe x k is omitted. Let R K ad R deote their uadjusted couterparts. The, R K = - e e/y M 0 y R = - e e /y M 0 y where e e is the sum of squared residuals i the full regressio, e e is the (larger) sum of squared residuals i the regressio which omits x k, ad y M 0 y = Σ i (y i - y ) The, RK = - [(-)/(-K)]( - RK ) ad R = - [(-)/(-(K-))]( - R ). The differece is the chage i the adjusted R whe x k is added to the regressio, RK - R = [(-)/(-K+)][e e /y M 0 y] - [(-)/(-K)][e e/y M 0 y]. The differece is positive if ad oly if the ratio is greater tha. After cacellig terms, we require for the adjusted R to icrease that e e /(-K+)]/[(-K)/e e] >. From the previous problem, we have that e e = e e + b K (x k M x k ), where M is defied above ad b k is the least squares coefficiet i the full regressio of y o X ad x k. Makig the substitutio, we require [(e e + b K (x k M x k ))(-K)]/[(-K)e e + e e] >. Sice e e = (-K)s, this simplifies to [e e + b K (x k M x k )]/[e e + s ] >. Sice all terms are positive, the fractio is greater tha oe if ad oly b K (x k M x k ) > s or b K (x k M x k /s ) >. The deomiator is the estimated variace of b k, so the result is proved. 0. Suppose you estimate a multiple regressio first with the without a costat. Whether the R is higher i the secod case tha the first will deped i part o how it is computed. Usig the (relatively) stadard method, R = - e e / y M 0 y, which regressio will have a higher R? This R must be lower. The sum of squares associated with the coefficiet vector which omits the costat term must be higher tha the oe which icludes it. We ca write the coefficiet vector i the regressio without a costat as c = (0,b * ) where b * = (W W) - W y, with W beig the other K- colums of X. The, the result of the previous exercise applies directly.. Three variables, N, D, ad Y all have zero meas ad uit variaces. A fourth variable is C = N + D. I the regressio of C o Y, the slope is.8. I the regressio of C o N, the slope is.5. I the regressio of D o Y, the slope is.4. What is the sum of squared residuals i the regressio of C o D? There are observatios ad all momets are computed usig /(-) as the divisor. We use the otatio Var[.] ad Cov[.] to idicate the sample variaces ad covariaces. Our iformatio is Var[N] =, Var[D] =, Var[Y] =. Sice C = N + D, Var[C] = Var[N] + Var[D] + Cov[N,D] = ( + Cov[N,D]). From the regressios, we have Cov[C,Y]/Var[Y] = Cov[C,Y] =.8. But, Cov[C,Y] = Cov[N,Y] + Cov[D,Y]. Also, Cov[C,N]/Var[N] = Cov[C,N] =.5, but, Cov[C,N] = Var[N] + Cov[N,D] = + Cov[N,D], so Cov[N,D] = -.5, so that Var[C] = ( + -.5) =. Ad, Cov[D,Y]/Var[Y] = Cov[D,Y] =.4. Sice Cov[C,Y] =.8 = Cov[N,Y] + Cov[D,Y], Cov[N,Y] =.4. Fially, Cov[C,D] = Cov[N,D] + Var[D] = =.5. Now, i the regressio of C o D, the sum of squared residuals is (-){Var[C] - (Cov[C,D]/Var[D]) Var[D]} 5

8 based o the geeral regressio result Σe = Σ(y i -y ) - b Σ(x i -x ). All of the ecessary figures were obtaied above. Isertig these ad - = 0 produces a sum of squared residuals of 5.. Usig the matrices of sums of squares ad cross products immediately precedig Sectio 3..3, compute the coefficiets i the multiple regressio of real ivestmet o a costat, real GNP ad the iterest rate. Compute R. The relevat submatrices to be used i the calculatios are Ivestmet Costat GNP Iterest Ivestmet * Costat GNP Iterest The iverse of the lower right 3 3 block is (X X) -, (X X) - = The coefficiet vector is b = (X X) - X y = ( ,.356, ). The total sum of squares is y y =.6365, so we ca obtai e e = y y - b X y. X y is give i the top row of the matrix. Makig the substitutio, we obtai e e = = To compute R, we require Σ i (x i - y ) = (3.05/5) = , so R = / = I the December, 969, America Ecoomic Review (pp ), Nathaial Leff reports the followig least squares regressio results for a cross sectio study of the effect of age compositio o savigs i 74 coutries i 964: log S/Y = log Y/N log G log D log D (R = 0.57) log S/N = log Y/N log G log D log D (R = 0.96) where S/Y = domestic savigs ratio, S/N = per capita savigs, Y/N = per capita icome, D = percetage of the populatio uder 5, D = percetage of the populatio over 64, ad G = growth rate of per capita icome. Are these results correct? Explai. The results caot be correct. Sice log S/N = log S/Y + log Y/N by simple, exact algebra, the same result must apply to the least squares regressio results. That meas that the secod equatio estimated must equal the first oe plus log Y/N. Lookig at the equatios, that meas that all of the coefficiets would have to be idetical save for the secod, which would have to equal its couterpart i the first equatio, plus. Therefore, the results caot be correct. I a exchage betwee Leff ad Arthur Goldberger that appeared later i the same joural, Leff argued that the differece was simple roudig error. You ca see that the results i the secod equatio resemble those i the first, but ot eough so that the explaatio is credible. 6

9 Chapter 4 Fiite-Sample Properties of the Least Squares Estimator. Suppose you have two idepedet ubiased estimators of the same parameter, θ, say θ ad θ, with differet variaces, v ad v. What liear combiatio, = c θ + c θ is the miimum variace ubiased estimator of θ? Cosider the optimizatio problem of miimizig the variace of the weighted estimator. If the estimate is to be ubiased, it must be of the form c θ + c θ where c ad c sum to. Thus, c = - c. The fuctio to miimize is Mi c L * = c v + ( - c ) v. The ecessary coditio is L * / c = c v - ( - c )v = 0 which implies c = v / (v + v ). A more ituitively appealig form is obtaied by dividig umerator ad deomiator by v v to obtai c = (/v ) / [/v + /v ]. Thus, the weight is proportioal to the iverse of the variace. The estimator with the smaller variace gets the larger weight.. Cosider the simple regressio y i = βx i + ε i. (a) What is the miimum mea squared error liear estimator of β? [Hit: Let the estimator be β = c y]. Choose c to miimize Var[ β ] + [E( β - β)]. (The aswer is a fuctio of the ukow parameters.) (b) For the estimator i (a), show that ratio of the mea squared error of θ β to that of the ordiary least squares estimator, b, is MSE[ β ] / MSE[b] = τ / ( + τ ) where τ = β / [σ /x x]. Note that τ is the square of the populatio aalog to the `t ratio' for testig the hypothesis that β = 0, which is give after (4-4). How do you iterpret the behavior of this ratio as τ? First, β = c y = c x + c ε. So E[ β ] = βc x ad Var[ β ] = σ c c. Therefore, MSE[ β ] = β [c x - ] + σ c c. To miimize this, we set MSE[ β ]/ c = β [c x - ]x + σ c = 0. Collectig terms, β (c x - )x = -σ c Premultiply by x to obtai β (c x - )x x = -σ x c or c x = β x x / (σ + β x x). The, c = [(-β /σ )(c x - )]x, so c = [/(σ /β + x x)]x. The, β = c y = x y / (σ /β + x x). The expected value of this estimator is so E[ β ] = βx x / (σ /β + x x) E[ β ] - β = β(-σ /β ) / (σ /β + x x) = -(σ /β) / (σ /β + x x) while its variace is Var[x (xβ + ε) / (σ /β + x x)] = σ x x / (σ /β + x x) The mea squared error is the variace plus the squared bias, MSE[β ] = [σ 4 /β + σ x x]/[σ /β + x x]. The ordiary least squares estimator is, as always, ubiased, ad has variace ad mea squared error MSE(b) = σ /x x. 7

10 The ratio is take by dividig each term i the umerator MSEβ = ΜΣΕ(β) 4 ( σ / β )/( σ / xx ' ) + σ xx/ ' ( σ / xx ' ) ( σ / β + xx ' ) = [σ x x/β + (x x) ]/(σ /β + x x) = x x[σ /β + x x]/(σ /β + x x) = x x/(σ /β + x x) Now, multiply umerator ad deomiator by β /σ to obtai MSE[β ]/MSE[b] = β x x/σ /[ + β x x/σ ] = τ /[ + τ ] As τ, the ratio goes to oe. This would follow from the result that the biased estimator ad the ubiased estimator are covergig to the same thig, either as σ goes to zero, i which case the MMSE estimator is the same as OLS, or as x x grows, i which case both estimators are cosistet. 3. Suppose that the classical regressio model applies, but the true value of the costat is zero. Compare the variace of the least squares slope estimator computed without a costat term to that of the estimator computed with a uecessary costat term. The OLS estimator fit without a costat term is b = x y / x x. Assumig that the costat term is, i fact, zero, the variace of this estimator is Var[b] = σ /x x. If a costat term is icluded i the regressio, the, b = ( xi x)( yi y) / ( xi x) i= = The appropriate variace is σ / (xi x) as always. The ratio of these two is i i= i = Var[b]/Var[b ] = [σ /x x] / [σ / ( x x) = But, (xi x) = x x + x i so the ratio is Var[b]/Var[b ] = [x x + x ]/x x = - x /x x = - { x /[S xx + x ]} < It follows that fittig the costat term whe it is uecessary iflates the variace of the least squares estimator if the mea of the regressor is ot zero. 4. Suppose the regressio model is y i = α + βx i + ε i f(ε i ) = (/λ)exp(-ε i /λ) > 0. This is rather a peculiar model i that all of the disturbaces are assumed to be positive. Note that the disturbaces have E[ε i ] = λ. Show that the least squares costat term is ubiased but the itercept is biased. We could write the regressio as y i = (α + λ) + βx i + (ε i - λ) = α * + βx i + ε i *. The, we kow that E[ε i * ] = 0, ad that it is idepedet of x i. Therefore, the secod form of the model satisfies all of our assumptios for the classical regressio. Ordiary least squares will give ubiased estimators of α * ad β. As log as λ is ot zero, the costat term will differ from α. 5. Prove that the least squares itercept estimator i the classical regressio model is the miimum variace liear ubiased estimator. Let the costat term be writte as a = Σ i d i y i = Σ i d i (α + βx i + ε i ) = ασ i d i + βσ i d i x i + Σ i d i ε i. I order for a to be ubiased for all samples of x i, we must have Σ i d i = ad Σ i d i x i = 0. Cosider, the, miimizig the variace of a subject to these two costraits. The Lagragea is L * = Var[a] + λ (Σ i d i - ) + λ Σ i d i x i where Var[a] = Σ i σ d i. Now, we miimize this with respect to d i, λ, ad λ. The (+) ecessary coditios are L * / d i = σ d i + λ + λ x i, L * / λ = Σ i d i -, L * / λ = Σ i d i x i The first equatio implies that d i = [-/(σ )](λ + λ x i ). Therefore, Σ i d i = = [-/(σ )][λ + (Σ i x i )λ ] ad Σ i d i x i = 0 = [-/(σ )][(Σ i x i )λ + (Σ i x i )λ ]. i ] 8

11 We ca solve these two equatios for λ ad λ by first multiplyig both equatios by -σ the writig the Σ x i i λ resultig equatios as The solutio is Σixi Σix i = λ - σ σ λ. 0 λ = Σ ixi. Σixi Σixi 0 Note, first, that Σ i x i = x. Thus, the determiat of the matrix is Σ i x i - ( x ) = (Σ i x i - x ) = S xx where S xx (x i i x = ) λ Σ σ ixi x. The solutio is, therefore, = λ Sxx x 0 0 or λ = (-σ )(Σ i x i /)/S xx λ = (σ x )/S xx The, d i = [Σ i x i / - x x i ]/S xx This simplifies if we writeσx i = S xx + x, so Σ i x i / = S xx / + x. The, d i = / + x ( x - x i )/S xx, or, i a more familiar form, d i = / - x (x i - x )/S xx. i = This makes the itercept term Σ i d i y i = (/)Σ i y i - x ( xi ) 6. As a profit maximizig moopolist, you face the demad curve Q = α + βp + ε. I the past, you have set the followig prices ad sold the accompayig quatities: x yi /S xx = y - b x which was to be show. Q P Suppose your margial cost is 0. Based o the least squares regressio, compute a 95% cofidece iterval for the expected value of the profit maximizig output. Let q = E[Q]. The, q = α + βp, or P = (-α/β) + (/β)q. Usig a well kow result, for a liear demad curve, margial reveue is MR = (-α/$) + (/β)q. The profit maximizig output is that at which margial reveue equals margial cost, or 0. Equatig MR to 0 ad solvig for q produces q = α/ + 5β, so we require a cofidece iterval for this combiatio of the parameters. The least squares regressio results are of the coefficiets is q Q = The estimated covariace matrix The estimate of q is The estimate of the variace of is (/4) ( ) + 5( ) or , so the estimated stadard error is The 95% cutoff value for a t distributio with 3 degrees of freedom is.6, so the cofidece iterval is (.576) to (.576) or 5.04 to The followig sample momets were computed from 00 observatios produced usig a radom umber geerator: X X = , X y = 80, (X X) - =, y y=

12 The true model uderlyig these data is y = x + x + x 3 + ε. (a) Compute the simple correlatios amog the regressors. (b) Compute the ordiary least squares coefficiets i the regressio of y o a costat, x, x, ad x 3. (c) Compute the ordiary least squares coefficiets i the regressio of y o a costat, x, ad x, o a costat, x, ad x 3, ad o a costat, x, ad x 3. (d) Compute the variace iflatio factor associated with each variable). (e) The regressors are obviously colliear. Which is the problem variable? The sample meas are (/00) times the elemets i the first colum of X'X. The sample covariace matrix for the three regressors is obtaied as (/99)[(X X) ij -00 x x i j ] Sample Var[x] = The simple correlatio matrix is The vector of slopes is (X X) - X y = [-.40, 6.3, 5.90, -7.55]. For the three short regressios, the coefficiet vectors are () oe, x, ad x : [-.3,.8,.] () oe, x, ad x 3 [-.0696,.9, 4.05] (3) oe, x, ad x 3 : [-.067, -.098, 4.358] The magificatio factors are for x : [(/(99(.077)) /.9] =.094 for x : [(/99(.75596)) /.] =.09 for x 3 : [(/99( ))/ 4.9] =.068. The problem variable appears to be x 3 sice it has the lowest magificatio factor. I fact, all three are highly itercorrelated. Although the simple correlatios are ot excessively high, the three multiple correlatios are.99 for x o x ad x 3,.988 for x o x ad x 3, ad.99 for x 3 o x ad x. 8. Cosider the multiple regressio of y o K variables, X ad a additioal variable, z. Prove that uder the assumptios A through A6 of the classical regressio model, the true variace of the least squares estimator of the slopes o X is larger whe z is icluded i the regressio tha whe it is ot. Does the same hold for the sample estimate of this covariace matrix? Why or why ot? Assume that X ad z are ostochastic ad that the coefficiet o z is ozero. We cosider two regressios. I the first, y is regressed o K variables, X. The variace of the least squares estimator, b = (X X) - X y, Var[b] = σ (X X) -. I the secod, y is regressed o X ad a additioal variable, z. Usig result (6-8) for the partitioed regressio, the coefficiets o X whe y is regressed o X ad z are b.z = (X M z X) - X M z y where M z = I - z(z z) - z. The true variace of b.z is the upper left K K XX ' Xz ' matrix i Var[b,c] = s zx ' zx '. But, we have already foud this above. The submatrix is Var[b.z ] = s (X M z X) -. We ca show that the secod matrix is larger tha the first by showig that its iverse is smaller. (See Sectio.8.3). Thus, as regards the true variace matrices (Var[b]) - - (Var[b.z ]) - = (/σ )z(z z) - z which is a oegative defiite matrix. Therefore Var[b] - is larger tha Var[b.z ] -, which implies that Var[b] is smaller. Although the true variace of b is smaller tha the true variace of b.z, it does ot follow that the estimated variace will be. The estimated variaces are based o s, ot the true σ. The residual variace estimator based o the short regressio is s = e e/( - K) while that based o the regressio which icludes z is s z = e.z e.z /( - K - ). The umerator of the secod is defiitely smaller tha the umerator of the first, but so is the deomiator. It is ucertai which way the compariso will go. The result is derived i the previous problem. We ca coclude, therefore, that if t ratio o c i the regressio which icludes z is larger tha oe i absolute value, the s z will be smaller tha s. Thus, i the compariso, Est.Var[b] = s (X X) - is based o a smaller matrix, but a larger scale factor tha Est.Var[b.z ] = s z (X M z X) -. Cosequetly, it is ucertai whether the estimated stadard errors i the short regressio will be smaller tha those i the log oe. Note 0

13 that it is ot sufficiet merely for the result of the previous problem to hold, sice the relative sizes of the matrices also play a role. But, to take a polar case, suppose z ad X were ucorrelated. The, XNM z X equals XNX. The, the estimated variace of b.z would be less tha that of b without z eve though the true variace is the same (assumig the premise of the previous problem holds). Now, relax this assumptio while holdig the t ratio o c costat. The matrix i Var[b.z ] is ow larger, but the leadig scalar is ow smaller. Which way the product will go is ucertai. 9. For the classical regressio model y = Xβ + ε with o costat term ad K regressors, assumig that the true value of β is zero, what is the exact expected value of F[K, -K] = (R /K)/[(-R )/(-K)]? The F ratio is computed as [b X Xb/K]/[e e/( - K)]. We substitute e = M, ad b = β + (X X) - X ε = (X X) - X ε. The, F = [ε X(X X) - X X(X X) - X ε/k]/[ε Mε/( - K)] = [ε (I - M)ε/K]/[ε Mε/( - K)]. The exact expectatio of F ca be foud as follows: F = [(-K)/K][ε (I - M)ε]/[ε Mε]. So, its exact expected value is (-K)/K times the expected value of the ratio. To fid that, we ote, first, that M, ad (I - M), are idepedet because M(I - M) = 0. Thus, E{[ε(I - M)ε]/[ε Mε]} = E[ε (I- M)ε] E{/[ε Mε]}. The first of these was obtaied above, E[ε (I - M)ε] = Kσ. The secod is the expected value of the reciprocal of a chi-squared variable. The exact result for the reciprocal of a chi-squared variable is E[/χ (-K)] = /( - K - ). Combiig terms, the exact expectatio is E[F] = ( - K) / ( - K - ). Notice that the mea does ot ivolve the umerator degrees of freedom. ~ 0. Prove that E[b b] = β β + σ Σ k (/λ k ) where b is the ordiary least squares estimator ad λ k is a characteristic root of X X. We write b = β + (X X) - X ε, so b b = β β + ε X(X X) - (X X) - X ε + β (X X) - X ε. The expected value of the last term is zero, ad the first is ostochastic. To fid the expectatio of the secod term, use the trace, ad permute ε X iside the trace operator. Thus, E[β β] = β β + E[ε X(X X) - (X X) - X ε] = β β + E[tr{ε X(X X) - (X X) - X ε}] = β β + E[tr{(X X) - X εε X(X X) - }] = β β + tr[e{(x X) - X εε X(X X) - }] = β β + tr[(x X) - X E[εε ]X(X X) - ] = β β + tr[(x X) - X (σ I)X(X X) - ] = β β + σ tr[(x X) - X X(X X) - ] = β β + σ tr[(x X) - ] = β β + σ Σ k (/λ k ) The trace of the iverse equals the sum of the characteristic roots of the iverse, which are the reciprocals of the characteristic roots of X X.. Data o U.S. gasolie cosumptio i the Uited States i the years 960 to 995 are give i Table F.. (a) Compute the multiple regressio of per capita cosumptio of gasolie, G/Pop, o all of the other explaatory variables, icludig the time tred, ad report all results. Do the sigs of the estimates agree with your expectatios? (b) Test the hypothesis that at least i regard to demad for gasolie, cosumers do ot differetiate betwee chages i the prices of ew ad used cars. (c) Estimate the ow price elasticity of demad, the icome elasticity, ad the cross price elasticity with respect to chages i the price of public trasportatio. (d) Reestimate the regressio i logarithms, so that the coefficiets are direct estimates of the elasticities. (Do ot use the log of the time tred.) How do your estimates compare to the results i the previous questio? Which specificatio do you prefer? (e) Notice that the price idices for the automobile market are ormalized to 967 while the aggregate price idices are achored at 98. Does this discrepacy affect the results? How? If you were to reormalize the idices so that they were all.000 i 98, how would your results chage? Part (a) The regressio results for the regressio of G/Pop o all other variables are:

14 Ordiary least squares regressio Weightig variable = oe Dep. var. = G Mea= , S.D.= Model size: Observatios = 36, Parameters = 0, Deg.Fr.= 6 Residuals: Sum of squares= , Std.Dev.=.66 Fit: R-squared= , Adjusted R-squared =.977 Model test: F[ 9, 6] = 67.85, Prob value = Diagostic: Log-L = , Restricted(b=0) Log-L = LogAmemiyaPrCrt.=.754, Akaike Ifo. Crt.= Autocorrel: Durbi-Watso Statistic =.9448, Rho = Variable Coefficiet Stadard Error t-ratio P[ T >t] Mea of X Costat YEAR PG Y E E PNC PUC PPT PN PD PS The price ad icome coefficiets are what oe would expect of a demad equatio (if that is what this is -- see Chapter 6 for extesive aalysis). The positive coefficiet o the price of ew cars would seem couterituitive. But, ewer cars ted to be more fuel efficiet tha older oes, so a risig price of ew cars reduces demad to the extet that people buy fewer cars, but icreases demad if the effect is to cause people to retai old (used) cars istead of ew oes ad, thereby, icrease the demad for gasolie. The egative coefficiet o the price of used cars is cosistet with this view. Sice public trasportatio is a clear substitute for private cars, the positive coefficiet is to be expected. Sice automobiles are a large compoet of the durables compoet, the positive coefficiet o PD might be idicatig the same effect discussed above. Of course, if the liear regressio is properly specified, the the effect of PD observed above must be explaied by some other meas. This author had o strog prior expectatio for the sigs of the coefficiets o PD ad PN. Fially, sice a large compoet of the services sector of the ecoomy is busiesses which service cars, if the price of these services rises, the effect will be to make it more expesive to use a car, i.e., more expesive to use the gasolie oe purchases. Thus, the egative sig o PS was to be expected. Part (b) The computer results iclude the followig covariace matrix for the coefficiets o PNC ad PUC The test statistic for testig the hypothesis that the slopes o these two variables are equal ca be computed exactly as i the first Exercise. Thus, t[6] = [ ( )]/[( (.673)] / = This is quite small, so the hypothesis is ot rejected. Part (c) The elasticities for the liear model ca be computed usig η = b( x / G/ P op ) for the various xs. The mea of G is The calculatios for ow price, icome, ad the price of public trasportatio are Variable Coefficiet Mea Elasticity PG Y PPT

15 Part (d) The estimates of the coefficiets of the logliear ad liear equatios are Costat YEAR LPG (Elasticity = -0.8) LY (Elasticity = +.09) LPNC LPUC LPPT (Elasticity = +0.64) LPN LPD LPS The estimates are roughly similar, but ot as close as oe might hope. There is little prior iformatio which would suggest which is the better model. Part (e) We would divide P d by.483, P by.375, ad P s by.353. This would have o effect o the fit of the regressio or o the coefficiets o the other regressors. The resultig least squares regressio coefficiets would be multiplied by these values. 3

16 Chapter 5 Large-Sample Properties of the Least Squares ad Istrumetal Variables Estimators. For the classical regressio model y = Xβ + ε with o costat term ad K regressors, what is plim F[K,-K] = plim (R /K)/[(-R )/(-K)] assumig that the true value of β is zero? What is the exact expected value? The F ratio is computed as [b X Xb/K]/[e e/( - K)]. We substitute e = M, ad b = β + (X X) - X ε = (X X) - X ε. The, F = [ε X(X X) - X X(X X) - X ε/k]/[ε Mε/( - K)] = [ε (I - M)ε/K]/[ε Mε/( - K)]. The deomiator coverges to σ as we have see before. The umerator is a idempotet quadratic form i a ormal vector. The trace of (I - M) is K regardless of the sample size, so the umerator is always distributed as σ times a chi-squared variable with K degrees of freedom. Therefore, the umerator of F does ot coverge to a costat, it coverges to σ /K times a chi-squared variable with K degrees of freedom. Sice the deomiator of F coverges to a costat, σ, the statistic coverges to a radom variable, (/K) times a chi-squared variable with K degrees of freedom.. Let e i be the ith residual i the ordiary least squares regressio of y o X i the classical regressio model ad let ε i be the correspodig true disturbace. Prove that plim(e i - ε i ) = 0. We ca write e i as e i = y i - b x i = (β x i + ε i ) - b x i = ε i + (b - β) x i We kow that plim b = β, ad x i is uchaged as icreases, so as, e i is arbitrarily close to ε i. 3. For the simple regressio model, y i = µ + ε i, ε i ~ N(0,σ ), prove that the sample mea is cosistet ad asymptotically ormally distributed. Now, cosider the alterative estimator µ = Σi w i y i, where w i = i/((+)/) = i/σ i i. Note that Σ i w i =. Prove that this is a cosistet estimator of µ ad obtai its asymptotic variace. [Hit: Σ i i = (+)(+)/6.] The estimator is y = (/)Σ i y i = (/)Σ i (µ + ε i ) = µ + (/)Σ i ε i. The, E[ y ] µ+ (/)Σ i E[ε i ] = µ ad Var[ y ]= (/ )Σ i Σ j Cov[ε i,ε j ] = σ /. Sice the mea equals µ ad the variace vaishes as, y is cosistet. I additio, sice y is a liear combiatio of ormally distributed variables, y has a ormal distributio with the mea ad variace give above i every sample. Suppose that ε i were ot ormally distributed. The, ( y -µ) = (/ )(Σ i ε i ) satisfies the requiremets for the cetral limit theorem. Thus, the asymptotic ormal distributio applies whether or ot the disturbaces have a ormal distributio. For the alterative estimator, µ = Σi w i y i, so E[ µ ] = Σi w i E[y i ] = Σ i w i µ = µσ i w i = µ ad Var[ µ ]= Σi w i σ = σ Σ i w i. The sum of squares of the weights is Σ i w i = Σ i i /[Σ i i] = [(+)(+)/6]/[(+)/] = [( + 3/ + /)]/[.5( + + )]. As, the fractio will be domiated by the term (/) ad will ted to zero. This establishes the cosistecy of this estimator. The last expressio also provides the asymptotic variace. The large sample variace ca be foud as Asy.Var[ ] = (/)lim Var[ ( µ - µ)]. For the estimator above, we ca use Asy.Var[ µ ] = (/)lim Var[ - µ] = (/)lim σ [( + 3/ + /)]/[.5( + + )] =.3333σ. Notice that this is uambiguously larger tha the variace of the sample mea, which is the ordiary least squares estimator. µ µ 4

17 4. I the discussio of the istrumetal variables estimator, we showed that the least squares estimator, b, is biased ad icosistet. Noetheless, b does estimate somethig; plim b = θ = β + Q - γ. Derive the asymptotic covariace matrix of b ad show that b is asymptotically ormally distributed. To obtai the asymptotic distributio, write the result already i had as b = (β + Q - γ) + (X X) - X ε - Q - ε. We have established that plim b = β + Q - γ. For coveiece, let θ β deote β + Q - γ = plim b. Write the precedig i the form b - θ = (X X/) - (X ε/) - Q - γ. Sice plim(x X/) = Q, the large sample behavior of the right had side is the same as that of plim (b - θ) = Q - plim(x ε/) - Q - γ. That is, we may replace (X X/) with Q i our derivatio. The, we seek the asymptotic distributio of (b - θ) which is the same as that of [Q - plim(x ε/) - Q - γ] = Q - x i iε i - γ =. From this poit, the derivatio is exactly the same as that whe γ = 0, so there is o eed to redevelop the result. We may proceed directly to the same asymptotic distributio we obtaied before. The oly differece is that the least squares estimator estimates θ, ot β. 5. For the model i (5-5) ad (5-6), prove that whe oly x * is measured with error, the squared correlatio betwee y ad x is less tha that betwee y * ad x *. (Note the assumptio that y * = y.) Does the same hold true if y * is also measured with error? Usig the otatio i the text, Var[x * ] = Q * so, if y = βx * + ε, Corr [y,x * ] = (βq * ) / [(β Q * + σ ε )Q * ] = β Q * /[$ Q * + σ ε )] I terms of the erroeously measured variables, Cov[y,x] = Cov[βx * + ε,x * + u] = βq *, so Corr [y,x] = (βq * ) /[(β Q * + ε ε )(Q * + σ u )] = [Q * /(Q * + σ u )]Corr [y,x * ] If y * is also measured with error, the atteuatio i the correlatio is made eve worse. The umerator of the squared correlatio is uchaged, but the term (β Q * + σ ε ) i the deomiator is replaced with (β Q * + σ ε + σ v ) which reduces the squared correlatio yet further. 6. Christese ad Greee (976) estimate a geeralized Cobb-Douglas fuctio of the form log(c/p f ) = α + βlogq + γlog Y + δ k log(p k /P f ) + δ l log(p l /P f ) + ε. P k, P l, ad P f idicate uit prices of capital, labor, ad fuel, respectively, Q is output ad C is total cost. The purpose of the geeralizatio was to produce a -shaped average total cost curve. (See Example 7.3 for discussio of Nerlove s (963) predecessor to this study.) We are iterested i the output at which the cost curve reaches its miimum. That is the poit at which [ logc/ logq] Q = Q* =, or Q * = 0 ( - β)/(γ). (You ca simplify the aalysis a bit by usig the fact that 0 x = exp(.306x). Thus, Q * = exp(.306[(- β)/(γ)]). The estimated regressio model usig the Christese ad Greee (970) data are as follows, where estimated stadard errors are give i paretheses: l ( C / P ) = lQ ( l Q) / l( P / P ) l( P / P ). ( ) ( ) ( ) ( ) ( ) f k f The estimated asymptotic covariace of the estimators of β ad γ is R = , e e = Usig the estimates give i the example, compute the estimate of this efficiet scale. Estimate the asymptotic distributio of this estimator assumig that the estimate of the asymptotic covariace of ad is The estimate is Q * = exp[.306( -.5)/((.7))] = 448. The asymptotic variace of Q * = exp[.306( - β )/( γ ) is [ Q * / β Q*/ γ] Asy.Var[ β, ][ Q * / β Q*/ γ]. The derivatives are γ β l γ f 5

18 Q * / β = Q*(-.306 β )/( γ ) = -63. Q * / γ = Q*[-.306(- β )]/( γ ) = The estimated asymptotic covariace matrix is. The estimated asymptotic variace of the estimate of Q* is thus 3,095,65. The estimate of the asymptotic stadard deviatio is 369. Notice that this is quite large compared to the estimate. A cofidece iterval formed i the usual fashio icludes egative values. This is commo with highly oliear fuctios such as the oe above. 7. The cosumptio fuctio used i Example 5.3 is a very simple specificatio. Oe might woder if the meager specificatio of the model could help explai the fidig i the Hausma test. The data set used for the example are give i Table F5.. Use these data to carry out the test i a more elaborate specificatio c t = β + β y t + β 3 i t + β 4 c t- + ε t where c t is the log of real cosumptio, y t is the log of real disposable icome ad i t is the iterest rate (90 day T bill rate). Results of the computatios are show below. The Hausma statistic is 5. ad the t statistic for the Wu test is Both are larger tha the table critical values by far, so the hypothesis that least squares is cosistet is rejected i both cases. --> samp;-04$ --> crea;ct=log(realcos);yt=log(realdpi);it=tbilrate$ --> crea;ct=ct[-];yt=yt[-]$ --> samp;-04$ --> ame;x=oe,yt,it,ct;z=oe,it,ct,yt$ --> regr;lhs=ct;rhs=x$ --> calc;s=ssqrd$ --> matr;bls=b;xx=<x'x>$ --> sls;lhs=ct;rhs=x;ist=z$ --> matr;biv=b;xhxh=/ssqrd*varb$ --> matr;d=biv-bls;vb=xhxh-xx$ --> matr;list;h=/s*d'*mpv(vb)*d$ --> regr;lhs=yt;rhs=z;keep=ytf$ --> regr;lhs=ct;rhs=x,ytf$ Ordiary least squares regressio Weightig variable = oe Dep. var. = CT Mea= , S.D.= Model size: Observatios = 03, Parameters = 4, Deg.Fr.= 99 Residuals: Sum of squares= e-0, Std.Dev.=.0084 Fit: R-squared=.99975, Adjusted R-squared = Model test: F[ 3, 99] =********, Prob value = Diagostic: Log-L = , Restricted(b=0) Log-L = LogAmemiyaPrCrt.= , Akaike Ifo. Crt.= Autocorrel: Durbi-Watso Statistic =.90738, Rho = Variable Coefficiet Stadard Error t-ratio P[ T >t] Mea of X Costat E E YT E IT E E CT E Two stage least squares regressio Weightig variable = oe Dep. var. = CT Mea= , S.D.= Model size: Observatios = 03, Parameters = 4, Deg.Fr.= 99 Residuals: Sum of squares= e-0, Std.Dev.=.008 Fit: R-squared=.99974, Adjusted R-squared = (Note: Not usig OLS. R-squared is ot bouded i [0,] Model test: F[ 3, 99] =********, Prob value =

19 Diagostic: Log-L = , Restricted(b=0) Log-L = LogAmemiyaPrCrt.= , Akaike Ifo. Crt.= Autocorrel: Durbi-Watso Statistic =.076, Rho = Variable Coefficiet Stadard Error b/st.er. P[ Z >z] Mea of X Costat E E YT E E IT E E CT E (Note: E+ or E- meas multiply by 0 to + or - power.) Matrix H has rows ad colums Ordiary least squares regressio Weightig variable = oe Dep. var. = YT Mea= , S.D.= Model size: Observatios = 03, Parameters = 4, Deg.Fr.= 99 Residuals: Sum of squares= e-0, Std.Dev.=.0086 Fit: R-squared=.99970, Adjusted R-squared =.9997 Model test: F[ 3, 99] =********, Prob value = Diagostic: Log-L = , Restricted(b=0) Log-L = -5. LogAmemiyaPrCrt.= , Akaike Ifo. Crt.= Autocorrel: Durbi-Watso Statistic =.7759, Rho = Variable Coefficiet Stadard Error t-ratio P[ T >t] Mea of X Costat E E IT E E CT E E YT E Ordiary least squares regressio Weightig variable = oe Dep. var. = CT Mea= , S.D.= Model size: Observatios = 03, Parameters = 5, Deg.Fr.= 98 Residuals: Sum of squares= e-0, Std.Dev.= Fit: R-squared= , Adjusted R-squared = Model test: F[ 4, 98] =********, Prob value = Diagostic: Log-L = , Restricted(b=0) Log-L = LogAmemiyaPrCrt.= -9.78, Akaike Ifo. Crt.= Autocorrel: Durbi-Watso Statistic =.35530, Rho = Variable Coefficiet Stadard Error t-ratio P[ T >t] Mea of X Costat E E YT E IT E E CT E YTF E

20 8. Suppose we chage the assumptios of the model i Sectio 5.3 to AS5: (x i,ε ) are a idepedet ad idetically distributed sequece of radom vectors such that x i has a fiite mea vector, µ x, fiite positive defiite covariace matrix Σ xx ad fiite fourth momets E[x j x k x l x m ] = φ jklm for all variables. How does the proof of cosistecy ad asymptotic ormality of b chage? Are these assumptios weaker or stroger tha the oes made i Sectio 5.? The assumptio above is cosiderably stroger tha the assumptio AD5. Uder these assumptios, the Slutsky theorem ad the Lidberg Levy versios of the cetral limit theorem ca be ivoked. 9. Now, assume oly fiite secod momets of x; E[x i ] is fiite. Is this sufficiet to establish cosistecy of b? (Hit: the Cauchy-Schwartz iequality (Theorem D.3), E[ xy ] {E[x ]} / {E[y ]} / will be helpful.) Is The assumptio will provide that (/)X X coverges to a fiite matrix by virtue of the Cauchy- Schwartz iequality give above. If the assumptios made to esure that plim (/)X ε = 0 cotiue to hold, the cosistecy ca be established by the Slutsky Theorem. 8

21 Chapter 6 Iferece ad Predictio. A multiple regressio of y o a costat, x, ad x produces the results below: y = 4 +.4x +.9x, R = 8/60, e e = 50, = 9, X X = Test the hypothesis that the two slopes sum to. The estimated covariace matrix for the least squares estimates is 3900 / s (X X) - 0 = = where s = 50/(9-3) = 0. The, the test may be based o t = ( )/[ (.05)] / =.399. This is smaller tha the critical value of.056, so we would ot reject the hypothesis... Usig the results i Exercise, test the hypothesis that the slope o x is zero by ruig the restricted regressio ad comparig the two sums of squared deviatios. I order to compute the regressio, we must recover the origial sums of squares ad cross products for y. These arex y = X Xb = [6, 9, 76]. The total sum of squares is foud usig R = - e e/y M 0 y, so y M 0 y = 50 / (5/60) = 600. The meas are x = 0, x = 0, y = 4, so, y y = (4 ) = 064. The slope i the regressio of y o x aloe is b = 76/80, so the regressio sum of squares is b (80) = 7., ad the residual sum of squares is = The test based o the residual sum of squares is F = [( )/]/[50/6] =.390. I the regressio of the previous problem, the t-ratio for testig the same hypothesis would be t =.4/(.40) / =.64 which is the square root of The regressio model to be aalyzed is y = X β + X β +, where X ad X have K ad K colums, respectively. The restrictio is β = 0. (a) Usig (6-4), prove that the restricted estimator is simply [b,0 ] where b is the least squares coefficiet vector i the regressio of y o X. (b) Prove that if the restrictio is β = β 0 for a ozeroβ 0, the restricted estimator of β is b * = (X X ) - X (y - X β). For the curret problem, R = [0,I] where I is the last K colums. Therefore, R(X X) - RN is the lower right K K block of (X X) -. As we have see before, this is (X M X) -. Also, (X X) - R is the last K colums of (X X) -. These are (X X) - -( X' X) X' X( X' MX) R = [See (-74).] Fially, ( X' MX) sice q = 0, Rb - q = (0b + Ib ) - 0 = b. Therefore, the costraied estimator is b b * = (X b - - X X X X X M X ( ' ) ' ( ' ) M X )b, where b ad b are the multiple regressio ( X' MX) coefficiets i the regressio of y o both X ad X. (See Sectio o partitioed regressio.) b b - Collectig terms, this produces b * = -( X' X) X' X b. But, we have from Sectio b that b = (X X ) - X y - (X X ) - ( X ' X ) X ' y X X b so the precedig reduces to b * = which was to 0 be show. 9

22 If, istead, the restrictio is β = β 0 the the precedig is chaged by replacig Rβ - q = 0 with Rβ - β 0 = 0. Thus, Rb - q = b - β 0. The, the costraied estimator is b b * = (X b - - X X X X X M X ( ' ) ' ( ' ) M X )(b - β 0 ) ( X' MX) or b b * = b + 0 ( X' X) X' X( b β ) 0 (β - b) Usig the result of the previous paragraph, we ca rewrite the first part as b * = (X X ) - X y - (X X ) - X X β 0 = (X X ) - X (y - X β 0 ) which was to be show. 4. The expressio for the restricted coefficiet vector i (6-4) may be writte i the form b * = [I - CR]b + w, where w does ot ivolve b. What is C? Show that covariace matrix of the restricted least squares estimator is σ (X X) - - σ (X X) - R [R(X X) - R ] - R(X X) - ad that this matrix may be writte as Var[b]{[Var(b)] - - R [Var(Rb)] - R}Var[b] By factorig the result i (6-4), we obtai b * = [I - CR]b + w where C = (X X) - R [R(X X) - R ] - ad w = Cq. The covariace matrix of the least squares estimator is Var[b * ] = [I - CR]σ (X X) - [I - CR] = σ (X X) - + σ CR(X X) - R C - σ CR(X X) - - σ (X X) - R C. By multiplyig it out, we fid that CR(X X) - = (X X) - R (R(X X) - R ) - R(X X) - = CR(X X) - R C so Var[b * ] = σ (X X) - - σ CR(X X) - R C = σ (X X) - - σ (X X) - R [R(X X) - R ] - R(X X) - This may also be writte as Var[b * ] = σ (X X) - {I - R (R(X X) - R ) - R(X X) - } = σ (X X) - {[σ (X X) - ] - - R [Rσ (X X) - R ] - R}σ (X X) - Sice Var[Rb] = Rσ (X X) - R this is the aswer we seek. 5. Prove the result that the restricted least squares estimator ever has a larger variace matrix tha the urestricted least squares estimator. The variace of the restricted least squares estimator is give i the secod equatio i the previous exercise. We kow that this matrix is positive defiite, sice it is derived i the form B σ (X X) - B, ad σ (X X) - is positive defiite. Therefore, it remais to show oly that the matrix subtracted from Var[b] to obtai Var[b * ] is positive defiite. Cosider, the, a quadratic form i Var[b * ] z Var[b * ]z = z Var[b]z - σ z (X X) - (R [R(X X) - R ] - R)(X X) - z = z Var[b]z - w [R(X X) - R ] - w where w = σr(x X) - z. It remais to show, therefore, that the iverse matrix i brackets is positive defiite. This is obvious sice its iverse is positive defiite. This shows that every quadratic form i Var[b * ] is less tha a quadratic form i Var[b] i the same vector. 6. Prove the result that the R associated with a restricted least squares estimator is ever larger tha that associated with the urestricted least squares estimator. Coclude that imposig restrictios ever improves the fit of the regressio. The result follows immediately from the result which precedes (6-9). Sice the sum of squared residuals must be at least as large, the coefficiet of determiatio, COD = - sum of squares / Σ i (y i - y ), must be o larger. 7. The Lagrage multiplier test of the hypothesis Rβ-q=0 is equivalet to a Wald test of the hypothesis that λ = 0, where λ is defied i (6-4). Prove that χ = λ {Est.Var[λ]} - λ = (-K)[e * e * /e e - ]. Note that the fractio i brackets is the ratio of two estimators of σ. By virtue of (6-5) ad the precedig sectio, we kow that this is greater tha. Fially, prove that the Lagrage multiplier statistic is simply JF, where J is the umber of restrictios beig tested ad F is the covetioal F statistic give i (6-0). 0