University of Lausanne - École des HEC LECTURE NOTES ADVANCED ECONOMETRICS. Preliminary version, do not quote, cite and reproduce without permission

Transcription

1 Uiversity of Lausae - École des HEC LECTURE NOTES ADVANCED ECONOMETRICS Prelimiary versio, do ot quote, cite ad reproduce without permissio Professor : Floria Pelgri CHAPTER 4: THE CONSTRAINED ORDINARY LEAST SQUARES ESTIMATOR

2 Cotets 1 Itroductio Represetatio of costraits Kerel represetatio of a set of liear costraits Image represetatio of a set of liear costraits Examples The costraied multiple liear model ad idetificatio The costraied ordiary least squares estimator Estimatio of β with the kerel represetatio Estimatio of β with the image represetatio Geometrical iterpretatio Projectig oto a costraied subspace Decompositio of the adjusted depedet variable Decompositio of the fitted error terms Statistical properties Ubiasedess properties Efficiecy Parametric model Semi-parametric model Costraied versus ucostraied models Asymptotic theory Testig the ull hypothesis H 0 : Rβ = q: the Studet ad Fisher test The Studet test Fisher test Geometrical iterpretatio Heuristic derivatio A more formal derivatio Other derivatios of the Fisher test statistic Implemetatio of the Fisher test Applicatio: Joit sigificace of the explaatory variables, with the exceptio of the costat term Asymptotic tests Ituitio Framework Liear costraits No liear costraits

3 CONTENTS The Wald test Liear costraits Noliear costraits Discussio about the Wald test The likelihood ratio test Liear costraits Noliear costraits Discussio The Lagrage multiplier or score test Liear costraits Noliar costraits Discussio Applicatios Asymptotic tests i the simple liear regressio model k = The liear model with kow variace The liear model with ukow variace Compariso of tests

4 1 Itroductio 3 1 Itroductio The purpose of this chapter is twofold. First, we explai how estimatio ca be coducted i the presece of liear ad/or o liear costraits, I this respect, the costraied ordiary least squares estimator ad the costraied maximum likelihood estimator are preseted i the case of the multiple liear regressio model ucostraied model. Secod, we explai how testig procedures ca be coducted. We cosider the Fisher test ad the trilogy of asymptotic tests Wald test, Lagrage multiplier or score test, likelihood ratio test. For istace, suppose that a applied ecoomist yes, it exists! would like to estimate a productio fuctio usig a Cobb-Douglas specificatio: Y i = AK α i L β i expu i where Y i, K i, L i ad u i are respectively the added value productio of firm sector i, the stock of capital, the labor factor, ad a multiplicative error term. Usig a log-trasformatio, this model ca be writte as: y i = β 0 + β 1 k i + β 2 l i + u i where β 0 = loga, β 1 = α, β 2 = β k t = logk i, y t = logy i, ad l i = logl t I doig so, oe seeks to impose the assumptio of costat returs to scale: β 1 + β 2 = 1. Two methods ca be cosidered: 1. Estimate the ucostraied model without takig ito accout the costrait ad the test the ull hypothesis H 0 : β 1 + β 2 = 1. If the ull hypothesis is ot rejected at a certai cofidece level, say 5%, the the costrait must be imposed ad the costraied model must be estimated. O the other had, if the ull hypothesis is rejected, the the ucostraied model is estimated with least squares method or the maximum likelihood method Chapters 2 ad Estimate the costraied model ad test whether the costraied model is preferable to the ucostraied model for istace, usig a Fisher test. It is worth oticig that the costraied model is give by: i.e. y i = β β 2 l i + β 2 k i + u i y i l i = β 0 + β 2 k i l i + u i where y i l i ad k i l i are the product added value per workers ad the capital per workers, respectively. Therefore, the costraied ordiary least

5 1 Itroductio 4 squares estimator of β 2 is defied to be the solutio of the followig miimizatio program: ˆβ 2,COLS = argmi y i l i β 0 β 2 k i l i 2. β 2 i=1 O the other had, the costraied ordiary least squares estimator of β 0, β 1, β 2 ca also be defied as the solutio of the followig costraied miimizatio problem: ˆβ0,COLS, ˆβ 1,COLS, ˆβ 2,COLS = argmi β 0,β 1,β 2 y i β 0 β 1 k i β 2 l i 2 i=1 s.t. β 1 + β 2 = 1. These two strategies have importat cosequeces. If oe wrogly imposes the costrait, the the ordiary least squares estimator of the costraied model the costraied ordiary least squares estimator is biased ad is ot cosistet. O the other had, if oe truly imposes the costrait, the ordiary least squares estimator of the ucostraied model should be expected to be close to the costraied ordiary least squares estimator. O the other had, costraits ca be used i order to compare ested models. For istace, istead of estimatig a Cobb-Douglas specificatio, oe may assume a Traslog specificatio as follows: 1 1 y i = β 0 + β 1 k i + β 2 l i + β 3 2 log2 K i + β 4 2 log2 L i + β 5 log K i log L i + v i. I particular, the Cobb-Douglas specificatio is ested i the previous specificatio. More specifically, if the ull hypothesis is ot rejected: β 3 = 0, β 4 = 0 ad β 5 = 0 the oe eds up with the Cobb-Douglas specificatio. I cotrast, suppose that the two models are ot ested at least, they have differet explaatory variables. The first model is give by: whereas the secod model writes: y = X 1 β 1 + u 1 1 y = X 2 β 2 + u 2 2 where X 1 M,k1, β 1 M k1,1, X 2 M,k2, ad β 2 M k2,1. If oe seeks to coduct the followig test o-ested assumptios: H 0 : M 0 is true H a : M 1 is true

6 1 Itroductio 5 the the supra-model might be cosidered: y = X 1 b 1 + X 2 b 2 + ε ad a Fisher test might be coducted. To sum up, we cosider estimatig a liear model: = Xβ + u where Y R, X is a matrix of costat ad radom variables, Eu X = 0 1, ad V u X = σ 2 I, With a set of liear costraits: with R M p k ad q M p 1 ; Rβ = q With a set of oliear costraits: h β = 0 p 1 with h β = h 1 β. h p β where h 1,h 2,,h p take their values i R, are differetiable. But we also cosider testig: A set of p liear costraits stated i the ull hypothesis: H 0 : Rβ = q agaist the alterative hypothesis: H a : Rβ q A set of p oliear costraits stated i the ull hypothesis: H 0 : hβ = 0 p 1 agaist the alterative hypothesis: H a : hβ 0 p 1.

7 2 Represetatio of costraits 6 2 Represetatio of costraits The costraits ca be writte i two differet ways: the kerel represetatio ad the image represetatio. Both forms are equivalet. 2.1 Kerel represetatio of a set of liear costraits Defiitio 1. The represetatio of the liear costraits: is called the kerel represetatio. Rβ = q 2.2 Image represetatio of a set of liear costraits Defiitio 2. The kerel represetatio Rβ = q is fully equivalet to:! β R k p / β = β 0 + S β. with β 0 R k, β R k p ad S M k k p. This represetatio is called the image represetatio. 2.3 Examples 1. Cosider the costraied liear multiple regressio model: s.t. Y = X 1 β 1 + X 2 β 2 + X 3 β 3 + X 4 β 4 + X 5 β 5 + ɛ β 2 = 0 β 4 = 0. a The kerel represetatio is give by: β β β 3 0 =. }{{} β 4 0 }{{} R β q 5 }{{} β

8 2 Represetatio of costraits 7 b The image represetatio writes: β 1 0 β 2 β = β 3 β 4 = β 5 0 }{{} β } {{ } S β 1 β 3 β 5 } {{ } β The costraied parameters are β 2 ad β 4 whereas the ucostraied free parameters are β 1, β 3, ad β Cosider the followig costraits: β 2 = β 3 β 3 = β 4 a The kerel represetatio is give by: }{{} R b The image represetatio writes: β 1 β 2 β = β 3 β 4 = β 5 3. Cosider the costraits: β 1 β 2 β 3 β 4 β 5 } {{ } β 0 =. 0 }{{} q β 1 β 3 β 5 β 1 = β 2 = = β K a The kerel represetatio is give by: β 1 β 2 β 3. β k 1 β k = 0 k 1 1

9 3 The costraied multiple liear model ad idetificatio 8 4. Cosider: Therefore, Rβ = q with R M k 1 k, β R k ad q M k 1 1. k-1 idepedet costraits β 3 + β 4 + β 5 = 1 a The kerel represetatio is give by: β = 1. b The image represetatio writes: β 1 β 1 β 2 β = β 3 β 4 = β 2 1 β 4 β 5 β 4 β 5 β = β 0 + S β The costraied multiple liear model ad idetificatio β 1 β 2 β 4 β 5 Defiitio 3. Let Y R, X M,k be a matrix of costat ad radom regressors, R M p,k ad q M p,1. The costraied liear multiple regressio model is defied to be: Y = Xβ + u s.t. Rβ = q with Eu X = 0 1, V u X = σ 2 I, rkx = k ad rkr = p or PrkX = k = 1 with p k.

10 4 The costraied ordiary least squares estimator 9 Remarks : 1. Throughout the chapter, we assume that rkx = k or PrkX = k = 1. I other words, the ucostraied model is idetifiable. 2. Suppose that the ucostraied mutiple liear regressio model is ot idetifiable rkx = k 1 < k. I particular, there is o uique solutio from the ormal equatios. Let β deote a particular solutio of: X X β = X Y where X X is sigular. The set of solutios is give by: β + KerX = { ˆβ R k / ˆβ = β + x, x KerX} where dim KerX = k k 1 ad KerX is the kerel of X matrix. Oe way to circumvet this issue is to impose some idetifyig costraits: Rβ = q where rkr = k k 1. Such idetifyig costraits are ofte used i practise for istace, the ANOVA, CANOVA models, the two-way fixed effects models, etc. Defiitio 4. The multiple liear costraied model is idetifiable if ad oly if β 1, β 2 R k : Xβ 1 = Xβ 2 et Rβ 1 = Rβ 2 β 1 = β 2. 4 The costraied ordiary least squares estimator 4.1 Estimatio of β with the kerel represetatio Defiitio 5. Uder suitable regularity coditios, the costraied ordiary last squares estimator of β, deoted ˆβ COLS, is the solutio of the followig miimizatio program: mi Y Xβ 2 I β s.t. Rβ = q.

11 4 The costraied ordiary least squares estimator 10 Propositio 1. Uder suitable regularity coditios, the costraied ordiary least squares estimator of β is give by: ˆβ COLS = ˆβ OLS X X 1 R [RX X 1 R ] 1 [R ˆβ OLS q] Proof: Uder suitable regularity coditios, the Lagragia is defied to be: LY, X; β = Y Xβ 2 + λ Rβ q = Y Y + β X Xβ Y Xβ β X Y + λ Rβ q where λ M p,1 is the vector of Lagrage multipliers. The first-order coditios with respect to β ad λ are give as follows: L β. = 0 k 1 2X Y X ˆβ COLS + R ˆλ = 0k 1 3 L λ. = 0 p 1 R ˆβ COLS q = 0 p 1 4 Takig Eq. 3, we get: ad thus: ˆβ COLS = X X 1 X Y 1 2 X X 1 R ˆλ ˆβ COLS = ˆβ OLS 1 2 X X 1 R ˆλ. Replacig this expressio i Eq. 4 yields: ad thus: R ˆβ COLS = q R ˆβ OLS 1 2 RX X 1 R ˆλ = q ˆλ = 2[RX X 1 R ] 1 [R ˆβ OLS q] Therefore, takig the expressio of λ ad replacig it i the expressio of ˆβ COLS, we obtai: Remarks: ˆβ COLS = ˆβ OLS X X 1 R [RX X 1 R ] 1 [R ˆβ OLS q] 1. The derivatio of ˆβ COLS makes use of the derivatio of ˆβ OLS the rak coditio of X is thus ecessary. 2. The matrix RX X 1 R is ivertible. Ideed, sice X X 1 is positive defiite ad R is of full row rak, it follows that RX X 1 R is o sigular ad thus ivertible. 3. Secod-order coditios ad qualificatio of costraits hold.

12 4 The costraied ordiary least squares estimator Estimatio of β with the image represetatio Defiitio 6. Uder suitable regularity coditios, the costraied ordiary least squares estimator of β, deoted ˆβ COLS, is defied to be: ˆβ COLS = β 0 + S ˆ β It is strictly equivalet to: = β 0 + S S X XS 1 S X Y Xβ 0. ˆβ COLS = ˆβ OLS X X 1 R [RX X 1 R ] 1 [R ˆβ OLS q]. Proof: The set of liear costraits ca be writte as: Rβ = q! β R k p / β = β 0 + S β. The costraied liear multiple regressio model is the defied by: or Y = Xβ 0 + S β + u Y Xβ 0 = XS β + u The estimator of β is the obtaied as the ordiary least squares estimator of the trasformed model i.e., the projectio of Y Xβ 0 oto XS. Uder suitable regularity coditios, ˆ β is the solutio of the followig miimizatio program: ˆ β = argmi Ỹ X β s 2 I β The ordiary least squares estimator of β is the: ˆ β = X sx s 1 X sỹ = XS XS 1 XS Ỹ = XS XS 1 XS Y Xβ 0 ˆ β = S X XS 1 S X Y Xβ 0. ad ˆβ COLS = β 0 + S ˆ β = β 0 + S S X XS 1 S X Y Xβ 0.

13 4 The costraied ordiary least squares estimator 12 Example : Cosider the multiple liear regressio model the exogeeity assumptio holds ad the error terms are spherical: Y i = β 1 + β 2 x i,2 + β 3 x i, β k x i,k + u i i.e. Y = Xβ + u where X = e, X 2,..., X k ad β R k. Cosider the simple costrait: β 2 = 1. The costraied ordiary least squares estimator of β ca be writte as: ˆβ COLS = β 0 + ˆ βols with β 0 = ad ˆ βols is the ordiary least squares estimator of the followig model: i.e. Y i x i2 = β 1 + β 3 x i3 + β 4 x i β k x ik + u i Y X 2 = XS β + ε where: ad M k,k x 11 x x 1k x XS = 21 x x 1 x 3... x k

14 5 Geometrical iterpretatio 13 5 Geometrical iterpretatio 5.1 Projectig oto a costraied subspace... Let L deoted the subspace spaed by the colum vectors X 1, X 2,..., X k. Let L 0 ad E 0 deote the followig subspaces: L 0 = {Xβ/β R k ad Rβ = q} = {Xβ 0 + XS β/β R k p } give that β = β 0 + S β. E 0 = {XS β/ β R k p } [Isert Figure 1 aroud here] 5.2 Decompositio of the adjusted depedet variable The fitted estimated depedet variable ŶCOLS is defied by: sice ad Ŷ COLS = X ˆβ COLS = X β 0 + Sˆ β = Xβ 0 + P XS Y Xβ 0 with P XS = XS S X XS 1 S X = P XS Y + I P XS Xβ 0 = P XS P X P X Y + I P XS Y 0 with Y 0 = Xβ 0 = P XS Ŷ OLS + I P XS Y 0 P XS P X = XSS X XS 1 S X X X X 1 X ŶCOLS is the result of both: = XSS X XS 1 S X = P XS P X Y = X ˆβ OLS = ŶOLS. The orthogoal projectio of Y usig the projectio matrix P XS P XS Y

15 5 Geometrical iterpretatio 14 The traslatio give by We get: I P XS Xβ 0. û COLS 2 I > û OLS 2 I. I other words, the fit of the restricted least squares solutio caot be better tha that of the urestricted solutio. 5.3 Decompositio of the fitted error terms Propositio 2. The fitted vector of residuals obtaied from the costraied ordiary least squares estimator ca be decomposed as the sum of two orthogoal vectors: û COLS = û OLS + ˆv where ˆv = X ˆβOLS ˆβ COLS = XX t X 1 R t [ RX t X 1 R t] 1 RX t X 1 X t u = P XX t X 1 R tu Proof: Usig the geometrical represetatio, we get: û COLS = û OLS + ˆv where ˆv = X ˆβOLS ˆβ COLS. Usig the defiitio of the costraied ordiary least squares estimator, oe obtais: { ˆv = X X t X R [ t RX t X 1 R t] } 1 R ˆβOLS q with: R ˆβ OLS q = RX t X 1 X t Y q = RX t X 1 X t Xβ + u q = Rβ q + RX t X 1 X t u = RX t X 1 X t u.

16 6 Statistical properties 15 The result follows. Remark: We get û COLS = Y ŶCOLS = Y X ˆβ COLS = Y X ˆβCOLS ˆβ OLS + ˆβ OLS = [Y X ˆβ ] [ OLS + X ˆβOLS ˆβ ] COLS [ = [u OLS ] + X ˆβOLS ˆβ ] COLS with û OLS ad X ˆβOLS ˆβ COLS are idepedet i probability show this result!. 6 Statistical properties 6.1 Ubiasedess properties Propositio 3. If the model is correctly specified β satisfies the set of liear costraits Rβ = q ad the exogeeity assumptio holds, the the costraied ordiary least squares estimator of β, ˆβ COLS, is ubiased: [ ] E ˆβCOLS = β for all β Θ. Proof: Takig the coditioal expectatio with respect to X of the costraied ordiary least squares estimator of β, we get: [ ] [ ] E ˆβCOLS X = E ˆβOLS X [ X X 1 R [RX X 1 R ] 1 E R ˆβ ] OLS q X. We have: E [ R ˆβ ] OLS q X = RX t X 1 X t E [u X] + Rβ q = 0. The first right-had side term equals zero due to the exogeeity assumptio whereas the secod term equals zero[ sice the] model is correctly specified ad the ull assumptio holds. Sice E ˆβOLS X = β, we get [ ] E ˆβCOLS X = β.

17 6 Statistical properties 16 Propositio 4. If the model is icorrectly specified β does ot satisfy the set of liear costraits Rβ = q, the costraied ordiary least squares estimator of β is biased. Proof: I this case, [ E R ˆβ ] OLS q X 0 p 1 ad the result follows. Propositio 5. The variace-covariace matrix of the costraied ordiary least squares estimator of β is defied to be: V ˆβCOLS X = σ 2 X X [ 1 I k R [RX X 1 R ] 1 RX X 1] ad V ˆβCOLS X V ˆβOLS X Proof: The variace-covariace matrix of the costraied ordiary least squares estimator is defied to be: V ˆβCOLS = σ 2 X X [ 1 I R [RX X 1 R ] 1 RX X 1] ad Give that: V ˆβCOLS < V ˆβOLS. V ˆβCOLS = E ˆβ COLS β ˆβ COLS β where: ˆβ COLS β = ˆβ OLS β X X 1 R [RX X 1 R ] 1 [R ˆβ OLS β + Rβ q] = I X X 1 R [RX X 1 R ] 1 R ˆβOLS β X X 1 R [RX X 1 R ] 1 [Rβ q]

18 6 Statistical properties 17 the secod right-had side term is ostochastic ad this does ot eter i the determiatio of the variace-covariace matrix of the costraied ordiary least squares estimator. Therefore, V ˆβCOLS = σ 2 [ I X X 1 R [RX X 1 R ] 1 R ] X X 1 [ I X X 1 R [RX X 1 R ] 1 R ] = σ 2 { X X 1 X X 1 R [RX X 1 R ] 1 RX X 1 X X 1 R [RX X 1 R ] 1 RX X 1 +X X 1 R [RX X 1 R ] 1 RX X 1 R[RX X 1 R ] 1 RX X 1} = σ { 2 X X 1 X X 1 R [RX X 1 R ] 1 RX X 1} The matrix X X 1 R [RX X 1 R ] 1 RX X 1 is positive semi-defiite ad thus: Remarks: V ˆβ COLS V ˆβ OLS 1. The costraied ordiary least squares estimator might be biased, especially if the model is icorrectly specified. 2. The costrait ordiary least squares estimator is more precise tha the ordiary least squares estimator: V ˆβCOLS X V ˆβOLS X 3. There is a trade-off betwee the bias ad the efficiecy: itroducig more costraits i the model improves the precisio of the estimates but the correspodig estimates are more likely biased! The coverse is also true. Propositio 6. A ubiased estimator of σ 2 is defied to be: ˆσ 2 COLS = û COLS 2 I k + p Proof: Takig the orthogoal decompositio of û COLS, we get: û COLS 2 I = û OLS 2 I + ˆv 2 I.

19 6 Statistical properties 18 Therefore, with: Fially, E [ û COLS 2 I X ] = E [ û OLS 2 I X ] + E [ ˆv 2 I X ] = σ 2 k + T rp XX t X 1 R t T r P XX t X 1 R t = p. E [ û COLS 2 I X ] = σ 2 k + p. The result follows. Remark: I cotrast to Chapter 2, the umber of degrees of freedom is k p sice the umber of free parameters is k p ad ot k. 6.2 Efficiecy Parametric model Propositio 7. The costraied ordiary least squares estimator of β, ˆβ COLS, is the maximum likelihood estimator of β i the costraied liear multiple regressio model. Exercise: Show the previous propositio. If the model is correctly specified uder the ull assumptio, the the costraied ordiary least squares estimator is efficiet its variace-covariace matrix equals the FDCR lower boud. Semi-parametric model Propositio 8. Cosider the coditioal static multiple liear regressio model: Y = Xβ 0 + u s.t.rβ = q where Eu i X = 0 ad V u i X = σ 2 for all i, Y is a -dimesioal vector, X is a k matrix of rak k or PrkX = k = 1, R M p k is of full row rak. The costraied ordiary least squares estimator of β 0 defied by: ˆβ COLS = ˆβ OLS X X 1 R [RX X 1 R ] 1 [R ˆβ OLS q] is the best estimator i the class of liear i Y ubiased estimators of β 0 uder the ull hypothesis H 0 : Rβ = q.

20 6 Statistical properties 19 Proof: This is a applicatio of Chapter 2 part II. 6.3 Costraied versus ucostraied models There are trade-offs betwee costraied ad ucostraied models. specifically, More If the costraied model is preferred, the costraied estimator will be more precise tha the ucostraied estimator as for istace the ordiary least squares estimator. At the same time, if oe wrogly imposes the costraits, the costraied estimator will be biased. If the ucostraied model is preferred, the correspodig estimator will be ubiased. At the same time, the estimator will be less precise tha the costraied estimator if the costraits are true i.e. oe wrogly assume that there are o costraits. Note also that multicolliearity is more likely to occur i the ucostraied model tha i the costraied model. All i all, there is "mea-variace" trade-off. It is thus difficult to coclude whether the first or secod strategy is preferable. To further illustrate this trade-off, cosider the followig two examples. Example 1: Omittig oe explaatory variable Omittig a explaatory variable,which is correlated to other regressors, geerally yields biased estimates. I particular, the exogeeity assumptio does ot hold aymore. To see this, suppose that the true data geeratig process is defied to be: Y = X 1 β 1 + X 2 β 2 + u. Suppose that oe wrogly imposes the costrait β 2 = 0, i.e. that oe estimates the followig model: Y = X 1 β 1 + u. Cosequetly, the costraied estimator of β = β 1, β 2 is give by: β 1,COLS = X 1X 1 1 X 1Y β 2,COLS = 0. Sice the true data geeratig process is the ucostraied model, oe gets that: E β1,cols E β COLS = 0

21 6 Statistical properties 20 where: E β 1,COLS = E [ X 1X 1 1 X 1X 1 β 1 + X 2 β 2 + u ] = X 1X 1 1 X 1X 1 β 1 + X 1X 1 1 X 1X 2 β 2 = β 1 + X 1X 1 1 X 1X 2 β 2. I other words, omittig X 2 yields the followig expressio for the bias of the costraied ordiary least squares estimator: X Bias β OCOLS = 1 X 1 1 X 1X 2 β 2 β 2. Therefore, if the costrait does ot hold i the true data geeratig process, the: 1. The bias of the costraied ordiary least squares estimator of β 1 is X 1X 1 1 X 1X 2 β 2. This bias is proportioal to β The bias of the costraied ordiary least squares estimator of β 2 is exactly mius the true ukow value of β The bias of β 1,COLS is higher whe the vector X 2 is close to the hyperplae spaed by the colum vectors of the X matrix for istace, whe the two explaatory variables are highly correlated. 4. If X 1 ad X 2 are orthogoal or perpedicular, the the bias is zero sice this bias depeds o the ier product betwee these two vectors. This result ca be geeralized if X 1 ad X 2 are some distict sets of explaatory variables. Remark: I the geeral case i which the X 2 vector is ot orthogoal to the liear space spaed by the colum vectors of X 1 or the vector X 1, deoted LX 1, the bias of β 1,COLS is obtaied from the regressio of X 2 oto the colum vectors of X 1. Ideed, sice X 1 is of full colum rak, there exists a uique decompositio of X 2 such that: where γ M k 1,1 ad ε LX 1 X 2 = X 1 γ + ε I this respect, γ is the ordiary least squares estimator of γ i the auxiliary model X 2 = X 1 γ + ε, ad γ = X 1X 1 1 X 1X 2. Therefore, the bias of β 1,COLS ca be expressed as: Bias β 1,COLS = γβ 2. Cosequetly, two sources of bias ca be idetified:

22 6 Statistical properties The degree of multicoliearity betwee X 2 ad LX 1 2. The misspecificatio of the costraits. Example 2: Addig a explaatory variable that is ot preset i the true data geeratig process. Suppose that the true data geeratig process is defied to be: Y = X 1 β 1 + X 2 β 2 + u s.t. β 2 = 0. Suppose that oe estimates the ucostraied model: Y = X 1 β 1 + X 2 β 2 + u. The ucostraied estimator of β = β 1, β 2 is ubiased. I doig so, oe applies the Frisch-Waugh theorem: β 1 = X 1M X2 X 1 1 X 1M X2 Y β 2 = X 2M X1 X 2 1 X 2M X1 Y where M X2 = I N X 2X 2 X 2 2 ad M X1 = I N X 1 X 1 X 1 1 X 1. Sice M X1 X 1 = 0, it follows that: ] β1 E [ β = E E β2 = [ β1 0 ] = β The ucostraied estimator of β, β, is ubiased. The same result applied for the ucostraied estimator of σ 2. However, addig the explaatory variable X 2 icreases the variace of the estimator of β 1. Oe ca show that: V β 1 = σ 2 X 1M X2 X 1 1 = σ 2 X 1X X 2 2 X 1X 2X 2 X 1 The variace-covariace matrix of the costraied estimator is give by V β 1,COLS = σ 2 X 1X d où V β 1,COLS < V β 1 sice X 2 X 1X 2X 2 2 X 1 is positive defiite.

23 7 Testig the ull hypothesis H 0 : Rβ = q: the Studet ad Fisher test Asymptotic theory The large sample properties of the costraied ordiary least squares estimator of β are applicatios of Chapter II part III. I particular, we ca show that the costraied ordiary least squares estimator of β is strogly weakly cosistet if the model is correctly specified i.e, Rβ 0 = q for the true ukow parameter vector. This result ca be eouced i the case of idepedet ad idetically distributed observatios, idepedet ad heterogeeously distributed observatios, depedet ad idetically distributed observatios, ad depedet ad heterogeeously distributed observatios. Moreover, the asymptotic distributio of the re-ormalized costraied ordiary least squares estimator ca be derived i the same way. For istace, i the case of idepedet ad idetically distributed observatios, we get the followig propositio. Propositio 9. Uder suitable regularity coditios, the samplig distributio of the costraied ordiary least squares estimator of β i a semiparametric costraied liear multiple regressio model with radom regressors is: { a ˆβ COLS N β, σ2 [ EX xi x t ] [ 1 i I k R R [ E X xi x t 1 i] ] } 1 [ R R EX xi x t ] 1 i with y i = x t iβ + u i s.t. Rβ = q. where E u i x i = 0, V u i x i = σ 2, ad E X x i x t i is osigular for all i = 1,,. Proof: See series 7. 7 Testig the ull hypothesis H 0 : Rβ = q: the Studet ad Fisher test I this sectio, we cosider the Gaussia parametric multiple liear regressio model ad cosider simple tests oe coefficiet or multiple tests. All i all, we seek to test the ull hypothesis that H 0 : Rβ = q agaist the alterative assumptio that H a : Rβ q. It is worth oticig that other ull or alterative hypothesis could be cosidered. 7.1 The Studet test Cosider the Gaussia liear multiple regressio model: Y = Xβ + u

24 7 Testig the ull hypothesis H 0 : Rβ = q: the Studet ad Fisher test 23 where Y R, X M k is oradom for coveiece, ad u N 0, σ 2 I parametric model. Oe seeks to test the sigificace of oe coefficiet give a certai level, say α: H 0 : β j = 0 H a : β j 0. Usig the results of Chapter II part II, the distributio of ˆβ j is give by: ˆβ j N β j, σ 2 m jj where m jj is the j th diagoal elemet of X X 1. Uder the ull hypothesis, oe gets: ˆβ j β 0 j σ m jj = ˆβ j σ m jj N 0, 1. with βj 0 = 0. If σ 2 is kow, the a cofidece set at the α level is give by: P u 1 α/2 ˆβ j σ u α/2 = 1 α m jj where u 1 α/2 = u α/2 is the α/2-percetile of a stadardized ormal distributio. [Isert Figure 3] If σ 2 is ukow, oe makes use of the followig result: k ˆσ2 σ 2 χ2 k ad the fact that ˆβ OLS ad k ˆσ2 are idepedet i probability. The σ 2 test-statistic is the Studet-distributed: T j = ˆβ j ˆσ m jj Γ k. The procedure of the test goes as follows: 1. Defie the critical regio: T j t α 2 2. Determie the critical value t α/2 usig a Studet s table. 3. Determie the realized value of the test-statistic usig the sample: T j,obs 4. Reject H 0 : β j = 0 at the α level if T j,obs > t α/2.

25 7 Testig the ull hypothesis H 0 : Rβ = q: the Studet ad Fisher test Fisher test Cosider the multiple liear Gaussia model with suitable regularity coditios ad suppose that oe seeks to test: H 0 : Rβ = q costraied model H a : Rβ q ucostraied model I cotrast to the Studet test, the Fisher test is a joit test. Geometrical iterpretatio Heuristic derivatio... If the ull assumptio H 0 caot be rejected, the ucostraied estimator of β i.e., the ordiary least squares estimator should satisfy approximatively the costraits. I other words, ˆβ OLS ad ˆβ COLS must be close give a metric meaig that ŶOLS 2 I ad ŶCOLS 2 I are approximatively the same, ad thus û OLS 2 I ad û COLS 2 I are approximatively equal. I particular, [Isert Figure 2] 1. û OLS is perpedicular to ŶOLS ŶCOLS i.e., the correspodig radom vectors are idepedet i probability 2. Their ormalized squared orm are Chi-square distributed 3. Their degrees of freedom are respectively k ad k+p k = p. Therefore, the Fisher test statistic ca be defied as follows: i.e., F = ŶOLS ŶCOLS 2 /p I û OLS 2 I / k F = ûcols 2 I û OLS 2 I /p û OLS 2 I / k sice û COLS 2 I = û OLS 2 I + ŶOLS ŶCOLS 2. I

26 7 Testig the ull hypothesis H 0 : Rβ = q: the Studet ad Fisher test 25 Defiitio 7. The Fisher test statistic is give by: where F = SSR 0 SSR a SSR a dl a dl 0 dl a F dl 0 dl a, dl a. dl 0 is the umber of degrees of freedom uder the ull hypothesis i.e., dl 0 = k p; dl a is the umber of degrees of freedom uder the alterative hypothesis i.e., dl a = k; SSR 0 is the sum of squared estimated residuals uder the ull hypothesis; SSR a is the sum of squared estimated residuals uder the alterative hypothesis. A more formal derivatio 5, oe gets that: Usig the geometrical iterpretatio of Sectio Ŷ COLS Xβ 0 = P E0 Y Xβ 0. where P E0 = P XS. Therefore, Ŷ ŶCOLS = P X Y Xβ 0 P E0 Y Xβ 0 = P X P E0 Y Xβ 0 = P Y Xβ 0 where P = P X P E0 is the orthogoal space of E 0 i LX. By defiitio sice u N 0, σ 2 I N : Y Xβ 0 EY Xβ0 N, I N σ σ Usig Fisher theorem, oe gets: P Y Xβ 0 σ 2 χ 2 rk P, {}}{ P Y Xβ 2 0 σ degrees of freedom {}}{ eccetricity parameter δ

27 7 Testig the ull hypothesis H 0 : Rβ = q: the Studet ad Fisher test 26 where Therefore, I additio, rk P = dimlx dime 0 X = p. Ŷ ŶCOLS σ 2 2 χ 2 p, δ. k σ 2 χ 2 k σ 2 ad, σ 2 ad Ŷ ŶCOLS 2 are idepedet i probability. Therefore, b Y b Y COLS 2 pσ 2 bσ 2 σ 2 F p, k, δ where F p, k, δ is a geeralized Fisher distributio with eccetricity parameter δ ad degrees of freedom p ad k. The eccetricity parameter is zero sice uder the ull hypothesis EY L 0 ad Xβ COLS L 0 imply that EY Xβ MCC L 0 ad P [EY Xβ COLS ] = 0, i.e. δ = 0. The test statistic follows the a Fisher distributio with p ad k degrees of freedom: Ŷ ŶCOLS 2 H F = 0 F p, k p σ 2 Other derivatios of the Fisher test statistic Defiitio 8. Uder suitable regularity coditios, the Fisher test statistic ca also be writte as usig the kerel represetatio of the costraits: F = 1 [ R β p σ 2 OLS q RX X 1 R ] 1 R β OLS q. Remark: A equivalet test statistic is also: F = 1 βols p σ β 2 COLS X X βols β COLS. Exercise: Show these two results.

28 7 Testig the ull hypothesis H 0 : Rβ = q: the Studet ad Fisher test 27 Implemetatio of the Fisher test 1. Determie SSR 0 costraied model ad SSR a ucostraied model; 2. Determie the realizatio of the Fisher test statistic: F obs = SSR 0 SSR a /p SSR a / k where p is the umber of liearly idepedet costraits. 3. Determie the critical regio: W = {F > F α } ad especially the critical value which correspods to the 1 α percetile of a Fisher distributed radom variable F p, k. 4. Reject the ull assumptio H 0 at the level α if F obs > F α. Applicatio: Joit sigificace of the explaatory variables, with the exceptio of the costat term Cosider the followig multiple liear regressio model: Y = e β 1 + Xβ 2 + u where X M,k 1, β 2 M k 1,1, β 1 R ad u N 0, σ 2 I N, ad the followig test: H 0 : β 2 = 0 k 1 1 H a : β 2 0 k 1 1 Uder the ull assumptio, the model is give by: Y = e β 1 + u c I particular, ŶCOLS = Y e. The umber of degrees of freedom is 1. Uder the alterative assumptio, oe has: Ŷ = e β1 + X β 2 The umber of degrees of freedom is k. The Fisher test statistic is the give by: Ŷ ŶCOLS 2 /k 1 F = û 2 / k

29 8 Asymptotic tests 28 i.e. i.e. F = F = Ŷ Y e Y Ŷ 2 /k 1 2 / k Ŷ Y e 2 / Y Y e 2 Y Ŷ 2 / Y Y e 2 k k 1 i.e. F = R2 k 1 R 2 k 1 where R 2 is the coefficiet of determiatio. Sice the Fisher test statistic is a fuctio of the coefficiet of determiatio i this joit sigificace test, it follows that the ull hypothesis is rejected at stadard levels 1%, 5% or 10% if: 1. The umber of observatios is large; 2. The coefficiet of determiatio is large i.e. close to 1. 8 Asymptotic tests The Studet ad Fisher tests are a importat tool i applied ecoometrics, especially i liear models. Moreover, both tests are exact meaig that we ca characterize the distributio of the test statistic uder the assumptio of ormal error terms. However, their priciple caot be geerally exteded i the case of oliear models. I this respect, oe ca use asymptotic tests, especially the trilogy of asymptotic tests: the Wald test, the Lagrage multiplier or score test, ad the likelihood ratio test. These three tests are quite geeral ad ca be applied with liear, oliear equality iequality costraits. Moreover, they are asymptotically equivalet ad cosistet. 1 At the same time, their fiite sample properties may be differet ad their implemetatio is differet: it depeds o the privileged assumptio H 0 or H a or both. 1 A asymptotic test with critical regio W is said to be cosistet if the probability of a type-ii error is asymptotically equal to zero or the power fuctio is asymptotically equal to oe.

30 8 Asymptotic tests Ituitio Cosider the followig simple liear regressio model: y i = b + u i with u i N 0, σ 2 for all i = 1,, ad σ 2 is kow. The log-likelihood fuctio is approximatively give by: ly; b y i b 2 i=1 ad the maximum likelihood estimator of b is: ˆb Ȳ = 1 Y i. The log-likelihood fuctio evaluated at ˆb y is approximatively give by: ly; ˆb y y i ȳ 2. We cosider the followig test: i=1 i=1 [Isert Figure 1] H 0 : b = b H a : b b Uder the ull hypothesis, the costraied maximum likelihood estimator of b is there is o loger a parameter to estimate uder the ull hypothesis!: ˆb0, = b. Uder the alterative hypothesis, the ucostraied maximum likelihood estimator of b is: ˆba, = Ȳ. The three asymptotic tests ca be explaied as follows. - Wald test: Compare ˆb a, ad ˆb 0,. If the differece is small i a sese to precise the we caot reject the ull hypothesis. Note that the distributio of the ucostraied estimator is required. - Likelihood ratio test: Compare ly, ˆb a, y ad ly, ˆb 0, y. If the differece is small the we caot reject the ull hypothesis. Note that the distributio of ly, ˆb 0, y ly, ˆb 0, y is required. - Lagrage multiplier test: Compare the slope of the taget evaluated at ˆb 0, y, ly, ˆb 0, y with zero.

31 8 Asymptotic tests Framework Liear costraits I the case of liear costraits, we will cosider the followig α-size test test: H 0 : Rβ = q H a : Rβ q where R M p k has rak p the liear costraits must be liearly idepedet or o redudat ad q M p 1. No liear costraits I the case of oliear costraits, we will cosider the followig α-size test test: The ull hypothesis is equivalet to: H 0 : hβ = 0 H a : hβ 0. h 1 β = 0 h 2 β = 0. h p β = 0 The fuctios h 1,h 2,,h p take their values i R, are differetiable, ad: [ ] h t rk β β = p for all β Θ o redudat costrait idetificatio assumptio!!! 8.3 The Wald test Liear costraits Propositio 10. Suppose that rkr = p k. The, uder H 0 : Rβ 0 = q, the Wald test is defied by the critical regio: W = { y/t W y χ 2 1 αp }

32 8 Asymptotic tests 31 where the test statistic is give by: T W = [ R ˆβ t [,a q] ˆVasy R ˆβ ] 1 [,a q R ˆβ ],a q where ˆβ,a is the maximum likelihood estimator of β uder the alterative hypothesis. The Wald test has asymptotic level α ad is cosistet. A large value of the test statistic leads to the rejectio of the ull hypothesis. The asymptotic variace-covariace matrix of R ˆβ,a q is give by: ˆV asy R ˆβ,a q = ˆV asy R ˆβ,a = RV asy ˆβ,a R t. The previous propositio ca be restated as follows usig the results of Chapter II part III i the case of the multiple liear regressio model with stochastic regressors ad i.i.d observatios see also Sectio I Chapter II part III, the followig theorem has bee derived i the case of i.i.d. observatios: Theorem 1 White s theorem. Cosider the multiple liear regressio model: for i = 1,, ad β 0 R k. Suppose Y i = x iβ 0 + u i i {x i, u i} is a idepedet ad idetically distributed sequece; ii E x i u i = 0 k 1, E x i,j u i 2 < for j = 1,, k, ad V V 1/2 X u = V is positive defiite; iii E x ij 2 < for j = 1,, k, ad Q E x i x i is positive defiite. The: where D = Q 1 V Q 1 ˆβ,OLS β 0 a.d. N 0 k 1, D I additio, if there exists ˆV symmetric ad positive semi-defiite such that ˆV V p 0 k k, the: where ˆD = X X/ 1 ˆV X X/ 1. ˆD D p 0 Sice the Wald test is based o the evaluatio of the ucostraied estimator uder the ull hypthesis, we eed this result. Note that it ca be geeralized i other cases.

33 8 Asymptotic tests 32 Propositio 11. Suppose that rkr = p k. The, uder H 0 : Rβ 0 = q: i Give that T = RD R t = RQ 1 V Q 1 R t, T 1/2 R ˆβ a.d.,a q N 0, I p where X X Q p 0 k k X X Q = E ; ii The Wald statistic is defied to be: where [ T W = R ˆβ,a q ] t ˆT 1 ˆT = R ˆD X R t X = R X X [ R ˆβ,a q] a χ 2 p 1 X 1 X ˆV R t ˆV = ˆσ 2 "Pseudo-proof": Uder H 0, oe gets: R ˆβ,a = R ˆβ,a β 0 + β 0 q = R ˆβ,a β 0 + Rβ 0 q = R ˆβ,a β 0. Therefore, the limitig distributio of R ˆβ,a q is give by: R ˆβ,a β 0. Sice Chapter II, part III see before ˆβ,a β 0 a.d. N 0, D where D = Q 1 V Q 1, it follows that: R ˆβ,a β 0 a.d. N 0, RD R t.

34 8 Asymptotic tests 33 I this respect, we get: R ˆβ a.d.,a q N 0, RD R t ad T 1/2 R ˆβ a.d.,a q N 0, I p. It is worth otig that a key issue especially i fiite samples is to compute a estimate of the asymptotic variace-covariace matrix. As we explai before, there are at least three differet methods that are asymptotically equivalet which ca lead to differet fiite sample coclusios. Noliear costraits Propositio 12. Suppose that: [ ] h t rk β β = p for all β Θ. The, uder H 0 : gβ = 0 p 1, the Wald test is defied by the critical regio: W = { y/t W y χ 2 1 αp } where the test statistic is give by: T W = [ ] t h ˆβ,a [V asy h ˆβ ] 1 [ ],a h ˆβ,a. where ˆβ,a is the maximum likelihood estimator of β uder the alterative hypothesis ad V asy h ˆβ,a = h ˆβ,a V β t asy ˆβ,a ht ˆβ,a β The Wald test has asymptotic level α ad is cosistet. Remark: The test statistic ca be rewritte as follows: [ ] [ t h 1 h t ] 1 [ ] T W = h ˆβ,a ˆβ,a I β t 1 ˆβ,a ˆβ,a h ˆβ,a. β where I is the average Fisher iformatio matrix for oe observatio.

35 8 Asymptotic tests 34 Proof: Ituitio of the proof : 1 The Wald test requires the maximizatio of the ucostraied problem i.e., the maximizatio problem uder the alterative hypothesis. 3. Therefore, its asymptotic distributio ca be characterized usig the results of Chapter III. Give the asymptotic distributio of ˆβ,a, oe obtais the asymptotic distributio of h ˆβ,a. This is step 1. 2 The key idea of the Wald test is to cosider the distace betwee ˆβ,a ad ˆβ,0 see figure. I other words, the Wald test assesses to what extet there are some gais to switch from the alterative to the ull hypothesis meaig that the ucostraied estimator is evaluated uder the costraits of H 0. This meas that the asymptotic distributio of h ˆβ,a eeds to be reevaluated uder the ull hypothesis. This is step 2. 3 After evaluatig the asymptotic distributio of ˆβ,a uder the ull hypothesis, we ca deduce the quadratic form associated to the Wald test. I doig so, we eed a cosistet estimator of Fisher iformatio matrix evaluated at β 0 the true ukow parameter vector. This is step 3. Step 1 : Determiatio of the asymptotic distributio of h ˆβ,a. Usig the asymptotic distributio of the maximum likelihood estimator, we have: ˆβ,a β 0 d N 0, Iβ 0 1 ad usig a Taylor series expasio aroud β 0 or the delta method h ˆβ,a hβ 0 d N 0, hβ0 Iβ 0 1 ht β 0. β t β Step 2 : Evaluatio of the asymptotic distributio of h ˆβ,a uder H 0. Uder the ull hypothesis, we have hβ 0 = 0 p 1. 3 The Wald test does ot require the maximizatio of the costraied problem i.e. the maximizatio problem uder the ull hypothesis.

36 8 Asymptotic tests 35 I other words, the asymptotic distributio of h ˆβ,a uder H 0 ca be rewritte as: 4 or h ˆβ,a d N 0, hβ0 Iβ 0 1 ht β 0. β t β hβ 0 Iβ 0 1 ht β 0 1/2 h β t β ˆβ d N 0, I p. Step 3 : Determiatio of the quadratic form. Sice ˆβ,a is a cosistet estimator of β 0, we get: hβ 0 Iβ 0 1 ht β 0 1/2 h β t β ˆβ,a d N 0, I p. β= ˆβ,a Takig the quadratic form, we have: [ ] [ t h 1 h t ] 1 [ ] T W = h ˆβ,a ˆβ I ˆβ,a ˆβ,a h ˆβ,a. β t β where ˆβ,a is the maximum likelihood estimator of β uder the alterative hypothesis. Remark: The asymptotic behavior of the Wald test statistic ca also be derived by usig the multivariate mea value theorem. Discussio about the Wald test Oe mai advatage of the Wald test is that the derivatio of the costraied estimator the estimator uder the ull hypothesis is ot required. I that respect, the Wald test measures the extet to which the urestricted estimate of β fails to satisfy the hypothesized restrictios uder the ull hypothesis. Moreover, the Wald test statistic has the advatage with respect to the two 4 The covergece i distributio must be uderstood as follows: h ˆβ,a d,h0 N 0, hβ0 β t Iβ 0 1 ht β 0. β

37 8 Asymptotic tests 36 other asymptotic tests that it does ot rely o a strog distributioal assumptio step 1 of the proof. There are at least two shortcomigs of this testig procedure. O the oe had, it is a test agaist the ull hypothesis : it is ot really a test for a specific alterative hypothesis...i particular, this may leads to problems regardig the power of the test. O the other had, the Wald test statistic is ot ivariat to the formulatio of the restrictios especially i the oliear case!. This lack of ivariace is geerally ot shared by the two other asymptotic tests. 8.4 The likelihood ratio test Liear costraits I the case of the multiple liear regressio model, the likelihood ratio test ca be eouced as follows. Propositio 13. Suppose that rkr = p k. The, uder H 0 : Rβ 0 = q, the likelihood ratio test is defied by the critical regio: ] T LR = 2 [l ˆβ,a l ˆβ,0. with: T LR = ˆσ 2,0 R ˆβ t,a q R X X 1 1 R t R ˆβ,a q. Proof. See Sectio 9. Remarks: 1. I fact, we ca oly prove that: ] 2 [l ˆβ,a l ˆβ,0 ˆσ 2,0 R ˆβ t,a q R X X 1 R t 1 R ˆβ,a q p The secod term is the Wald statistic with ˆV = ˆσ 2,0X X/. The test statistic is Chi-squared asymptotically distributed provided ˆσ 2,0X X/ is a cosistet estimator of V.

38 8 Asymptotic tests 37 Noliear costraits Propositio 14. The likelihood ratio test is defied by the critical regio: W = { y/t LR y χ 2 1 αr } where the test statistic is give by: ] T LR = 2 [l ˆβ,a l ˆβ,0. where ˆβ,a ad ˆβ,0 are respectively the maximum likelihood estimator of β uder the alterative ad the ull hypothesis. The likelihood ratio test has asymptotic level α ad is cosistet. Proof: Ituitio of the proof : The likelihood ratio test treats symmetrically both hypothesis the ull ad the alterative hypothesis. I particular, the likelihood ratio test requires to fid the maximum likelihood estimator uder the ull ad the alterative hypothesis. I other words, oe has to solve the costraied ull hypothesis maximizatio problem ad the ucostraied alterative hypothesis maximizatio problem. At the same time, it is worth otig that the test statistic does ot require to determie ad evaluate the Fisher iformatio matrix i cotrast to the Wald test ad the Lagrage multiplier test fidig a cosistet estimator of the Fisher iformatio matrix ca be quite difficult! 1 The quatities of iterest by defiitio of the likelihood ratio test are the log-likelihood fuctios evaluated uder the ull ad the alterative hypothesis. If we wat to determie the asymptotic distributio of the test statistic, we eed to characterize their behaviors as approaches ifiity. I other words, we eed to start from a Taylor expasio this is exactly the same idea as the asymptotic distributio of the maximum likelihood estimator of the log-likelihood fuctio uder the ull hypothesis respectively, uder the alterative hypothesis. This is step 1. 2 Therefore, a asymptotic approximatio of the likelihood ratio test statistic is give by the differece betwee the two Taylor expasios i the previous step. This is step 2. 3 After simplifyig the result of step 2, a asymptotic quadratic form of the test ca be obtaied this quadratic form is Chi-squared distributed. This is step 3.

39 8 Asymptotic tests 38 Step 1 : Taylor expasio of the log-likelihood fuctio uder H 0 ad H a. The secod-order Taylor expasio of the log-likelihood fuctio aroud β = β 0 uder the ull hypothesis is give by: a l ˆβ,0 l β 0 + l β 0 ˆβ,0 β t ˆβ,0 β 0 2 l β 0 β t 2 a l β 0 + l β 0 ˆβ,0 β 0 β t 2 a l β 0 + l β 0 ˆβ,0 β 0 β t 2 ˆβ,0 β 0 β β t ] 2 l β 0 ˆβ,0 β 0 β β t [ t ˆβ,0 β 0 1 t ˆβ,0 β 0 Iβ 0 ˆβ,0 β 0. The secod-order Taylor expasio of the log-likelihood fuctio aroud β = β 0 uder the alterative hypothesis is give by: a l ˆβ,a l β 0 + l β 0 ˆβ,a β 0 t ˆβ,a β 0 Iβ 0 ˆβ,a β 0. β t 2 Step 2 : Differece of the two Taylor expasios. Takig the differece of the two previous expressios, we get l ˆβ,a l ˆβ,0 a l β 0 ˆβ,a β ˆβ t,0 t ˆβ,a β 0 Iβ 0 ˆβ,a β ˆβ,0 β 0 t I1 β 0 ˆβ,0 β 0. Therefore, a asymptotic approximatio of the likelihood ratio test statistic is give by: T LR 2 l ˆβ,a l ˆβ,0 a 2 l β 0 ˆβ,a β ˆβ t,0 t ˆβ,a β 0 Iβ 0 ˆβ,a β 0 Step 3 : Simplificatio. t + ˆβ,0 β 0 Iβ 0 ˆβ,0 β 0. Usig propositio 1 or the first order Taylor expasio, we get

40 8 Asymptotic tests 39 1 l β 0 β a I 1 β 0 ˆβ,a β 0 ad thus substitutig i the previous expressio T LR 2 l ˆβ,a l ˆβ,0 ˆβ,a ˆβ,0 a t 2 ˆβ,a β 0 Iβ 0 t ˆβ,a β 0 Iβ 0 ˆβ,a β 0 This expressio ca be writte as: a t T LR 2 ˆβ,a β 0 Iβ 0 ˆβ,a ˆβ,0 ˆβ,a β 0 ˆβ,a β 0 t Iβ 0 + ˆβ,0 ˆβ,a + ˆβ,a β 0 t Iβ 0 t + ˆβ,0 β 0 Iβ 0 ˆβ,0 β 0. ˆβ,0 ˆβ,a + ˆβ,a β 0. i.e. a T LR ˆβ,a ˆβ t,0 Iβ 0 ˆβ,a ˆβ,0. with Iβ 0 1/2 ˆβ,a ˆβ,0 2 χ 2 p. Discussio The likelihood ratio test statistic is defied as the ratio betwee the restricted likelihood fuctio the costraied estimator uder the ull hypothesis ad the urestricted likelihood fuctio or is defied by 2 times the differece betwee the restricted log-likelihood fuctio ad the urestricted log-likelihood fuctio. This fuctio the ratio of the two likelihood fuctio must be betwee zero ad oe sice both likelihood fuctios are positive by assumptio ad the restricted likelihood fuctio caot be larger tha the urestricted likelihood fuctio: a restricted optimum is ever superior to a urestricted optimum. Oe mai difficulty of the likelihood ratio test is that the costraied ad ucostraied estimators are required. I complex models, oe of these estimators might be more difficult to derive.

41 8 Asymptotic tests The Lagrage multiplier or score test Liear costraits Usig the expressio of the Lagrage multipliers i Sectio 4 estimatio with the kerel represetatio, oe ca derive the Lagrage multiplier test i the case of the multiple liear regressio model uder geeral assumptios see also Sectio 9. Propositio 15. Suppose that rkr = p k. The, uder H 0 : Rβ 0 = q: i Give that Λ = 4 RQ 1 R t 1 T RQ 1 R t 1 with T = RD R t = RQ 1 V Q 1 R t, Λ 1/2 ˆλ a.d. N 0, I p ii The Lagrage multiplier statistics is defied to be: T LM = ˆλ 1 ˆΛ ˆλ a χ 2 p where X ˆΛ X X = 4 R R t 1 1 X ˆT R R t with ˆT = R ˆD X R t X = R 1 X ˆV 1 X R t ad ˆV is computed from the costraied regressio ad is a cosistet estimator of V uder H 0. "Pseudo-proof": The estimator of λ is defied to be see Sectio 4: ˆλ = 2 RX X 1 R t R ˆβ,a q. Therefore, the limitig distributio of ˆλ is give by: 2 RX X 1 R t R ˆβ,a q or 2 RX X 1 R t R ˆβ,a β 0

42 8 Asymptotic tests 41 sice R ˆβ,a q = R ˆβ,a β 0 uder the ull hypothesis see Sectio Therefore, 2 RX X 1 R t R ˆβ,a β 0 a.d. N 0, 4 RQ 1 R t 1 T RQ 1 R t 1. Fially, where Noliar costraits Λ 1/2 ˆλ a.d. N 0, I p Λ = 4 RQ 1 R t 1 T RQ 1 R t 1. Propositio 16. The score test is defied by the critical regio: W = { y/t S y χ 2 1 αp } where the test statistic is give by: T S = 1 l ˆβ,0 β t 1 l I ˆβ,0 β l l ˆβ,0 V β t asy ˆβ,0 β ˆβ,0 ˆβ,0 where ˆβ,0 is the maximum likelihood estimator of β uder the ull hypothesis. The score test has asymptotic level α ad is cosistet. Propositio 17. The Lagrage multiplier test is defied by the critical regio: W = { y/t LM y χ 2 1 αp } where the test statistic is give by: T LM = 1 ˆλ t h ˆβ,0 1 h t I ˆβ,0 ˆβ,0 ˆλ β t. β where ˆβ,0 is the maximum likelihood estimator of β uder the ull hypothesis. The Lagrage multiplier test has asymptotic level α ad is cosistet.

43 8 Asymptotic tests 42 Remark: The score ad Lagrage multiplier test statistics are equal: Proof: T LM = T S. Ituitio of the proof : The Lagrage multiplier test oly requires the maximizatio of the costraied problem i.e., the maximizatio problem uder the ull hypothesis. 5. The key idea of the Lagrage multiplier test is to cosider the distace betwee the taget at ˆβ,a ad ˆβ,0 see figure or the distace betwee the score vector evaluated at ˆβ,a ad ˆβ,0. I other words, this test assesses to what extet there are some gais to switch from the ull to the alterative hypothesis meaig that we expect the score or the taget evaluated at the ucostraied estimator to be close to the ull vector or zero. Therefore, the test statistic requires to determie the distributio of the Lagrage multiplier vector of the costraied maximizatio problem or to fid a quadratic form with respect to the score fuctio. Before goig through all the details of the proof, we eed the followig Lemma that characterizes the first-order coditios of the maximum likelihood estimator assumig that suitable regularity coditios hold uder the ull hypothesis. Lemma 1. Uder suitable regularity coditios, the maximum likelihood estimator is the solutio of the followig first-order coditios usig the Lagragia fuctio: l ˆβ,0 β + ht ˆβ,0 ˆλ = 0 k 1 β Step 1 : Taylor expasio aroud β 0 of h ˆβ uder the ull ad the alterative hypotheses. We have h ˆβ,a a hβ0 β t ˆβ,a β 0 ad h ˆβ,0 a hβ0 β t ˆβ,0 β 0. 5 The Wald test does ot require the maximizatio of the ucostraied problem i.e. the maximizatio problem uder the alterative hypothesis.

44 8 Asymptotic tests 43 Step 2 : Differece of the first order Taylor expasios. Takig the differece of two previous expressios, we get: h ˆβ,a h ˆβ a hβ 0,0 ˆβ,a β ˆβ t,0. Sice h ˆβ,0 = 0 uder the ull hypothesis costraied estimator, we obtai: h ˆβ,a a hβ0 β t ˆβ,a ˆβ,0. 5 Step 3 : Rewritig of ˆβ,a ˆβ,0 ˆβ,a ˆβ,0 ca be rewritte by usig first-order Taylor expasios of the score fuctio aroud β 0 uder the ull ad alterative hypothesis. Ideed, we have: l β ˆβ,0 which implies: a a a l β β0 + 2 l β β t β0 ˆβ,0 β 0 l [ β β0 1 ] 2 l β β t β0 ˆβ,0 β 0 l β β0 Iβ 0 ˆβ,0 β 0 1 l β ˆβ,0 a 1 l β β0 Iβ 0 ˆβ,0 β 0. 6 I the same respect, we get: 1 l β ˆβ,a a 1 l β β0 Iβ 0 ˆβ,a β 0 7 with ucostraied estimator: 1 l β ˆβ,a = 0 k 1.

45 8 Asymptotic tests 44 Takig the differece Eq. 6 - Eq. 7, we get: which implies: 1 l ˆβ,0 β a Iβ 0 ˆβ,a ˆβ,0 Step 4 : Usig Step 3 i Eq. 5. ˆβ,a ˆβ a,0 Iβ l ˆβ,0. β Replacig the previous result i Eq. 5, we get: Step 5 : Usig Lemma 1 i Eq. 8. Give that Lemma 1: h ˆβ,a a hβ0 Iβ l ˆβ,0. β t β it follows that: l ˆβ,0 β + ht ˆβ,0 ˆλ = 0 k 1 β which is equivalet to: h ˆβ,a a hβ0 Iβ 0 1 ht ˆβ,0 ˆλ β t β h ˆβ,a a hβ0 Iβ 0 1 ht β 0 ˆλ 8 β t β sice ˆβ,0 is a cosistet estimator of β 0 uder the ull hypothesis.