Chapter 4: Constrained estimators and tests in the multiple linear regression model (Part III) Florian Pelgrin HEC September-December 2010 Florian Pelgrin (HEC) Constrained estimators September-December 2010 1 / 16
1 Introduction 2 Intuition 3 Framework 4 Wald test 5 Likelihood ratio test 6 Lagrange multiplier or Score test 7 Equivalence 8 Summary Florian Pelgrin (HEC) Constrained estimators September-December 2010 2 / 16
1. Introduction Introduction The Student test statistic and the Fisher test statistic are derived under the normality assumption (parametric model). The exact distribution of the ordinary least squares is used and thus the exact distributions of the test statistics are obtained. What happens now in a semi-parametric model or in a model in which one does not have the exact distribution (e.g., maximum likelihood estimator)? One has the large sample distribution or asymptotic distribution. One can derive test statistics using it: the so-called asymptotic tests. Remark: We restrict our attention to asymptotic tests in the case of both the least squares method (for linear models) and the maximum likelihood principle. Florian Pelgrin (HEC) Constrained estimators September-December 2010 3 / 16
Introduction Three asymptotic tests: (a) Wald test (b) Likelihood ratio test (c) Score or Lagrange multiplier test These three asymptotic tests are general and can be applied with linear, nonlinear equality (inequality) constraints. These three tests are asymptotically equivalent. Their finite sample properties are different. Their implementation is different: it depends on the privileged assumption (H 0 or H a or both). Florian Pelgrin (HEC) Constrained estimators September-December 2010 4 / 16
2. Intuition Intuition n (θ ) Wald test ' ˆ n θ n (, ML ) = 0 θˆ n,ml 0 θ Likelihood ratio test ˆ n θ n n (,ML ( θ 0 ) ) θ ' 0 d 0 ( θ ) = n ( θ ) n dθ Lagrange multiplier test Florian Pelgrin (HEC) Constrained estimators September-December 2010 5 / 16
3. Framework Framework Let θ R k and h denote: g : R k R p θ g(θ) where g is linear or nonlinear. Examples: 1 Linear constraints (θ = β): g : R k R p β g(β) Rβ q. 2 Two nonlinear constraints (θ = (θ 1, θ 2 ) ): g : R 2 R 2 ( ) h1 (θ) θ g(θ) = = h 2 (θ) ( θ1 θ 2 θ 2 1 1 ). Florian Pelgrin (HEC) Constrained estimators September-December 2010 6 / 16
Framework Consider the following (α-size test) test: H 0 : h(θ) = 0 H a : h(θ) 0. The null hypothesis is equivalent to: h 1 (θ) = 0 h 2 (θ) = 0. h p (θ) = 0 The functions h 1,h 2,,h p take their values in R, are differentiable, and: [ ] h t rk θ (θ) = p for all θ Θ no redundant constraint (identification assumption)!!! Florian Pelgrin (HEC) Constrained estimators September-December 2010 7 / 16
Framework Suppose that the asymptotic distribution is given by: ) d n (ˆθ n θ N (0, Σ) n with ˆΣ n is (weakly) consistent estimator of Σ: p ˆΣ n Σ. The approximate asymptotic distribution is thus given by: ˆθ n a N ( θ, Vasy (θ) ) where a consistent estimate of the asymptotic variance-covariance matrix is: ˆV asy (ˆθ n ) = n 1 ˆΣ 1 n. Florian Pelgrin (HEC) Constrained estimators September-December 2010 8 / 16
4. The Wald test Wald test Proposition The Wald test is defined by the critical region: } W n = {y/t W (y) χ 2 1 α (p) where the (observed) test statistic is given by (under H 0 ): T W = [ )] [ t h ) ) h t ) h (ˆθn,a (ˆθn,a ˆV θ t asy (ˆθn,a (ˆθn,a ] 1 [ )] h (ˆθn,a θ where ˆθ n,a is the maximum likelihood estimator of θ under the alternative hypothesis and: ) ) ˆV asy (ˆθ n,a = n 1 ˆΣ n (ˆθ n,a. The Wald test has asymptotic level α and is consistent. Florian Pelgrin (HEC) Constrained estimators September-December 2010 9 / 16
Wald test Application: Consider the multiple linear regression model Y = Xβ + u with a set of linear constraints (θ = β): h(β) = Rβ q and h β t = R. Using the least squares method (in a semi-parametric model): [ ] t T W = R ˆβ n,ols q [R ˆV asy ( ˆβ n,ols )R t] 1 [ ] R ˆβ n,ols q with ˆV asy ( ˆβ n,ols ) = n 1 ˆΣn,OLS. Using the maximum likelihood principle: [ T W = R ˆβ ] t n,ml q [R ˆV asy ( ˆβ n,ml )R t] 1 [ R ˆβ ] n,ml q with ˆV asy ( ˆβ n,ml ) = n 1 ˆΣ n,ml. Florian Pelgrin (HEC) Constrained estimators September-December 2010 10 / 16
Likelihood ratio test 5. The likelihood ratio test Proposition The likelihood ratio test is defined by the critical region: } W n = {y/t LR (y) χ 2 1 α (p) where the test statistic is given by (under H 0 ): ) )] T LR = 2 [l n (ˆθ n,a l n (ˆθ n,0. where ˆθ n,a and ˆθ n,0 are respectively the maximum likelihood estimator of θ under the alternative and the null hypothesis. The likelihood ratio test has asymptotic level α and is consistent. Florian Pelgrin (HEC) Constrained estimators September-December 2010 11 / 16
Lagrange multiplier or Score test 6. Lagrange multiplier or Score test Proposition The score test is defined by the critical region: } W n = {y/t S (y) χ 2 1 α (p) where the test statistic is given by: ) t { )}] 1 ) T S = s (ˆθn,0 [ˆV asy s (ˆθn,0 s (ˆθn,0 where ˆθ n,0 is the maximum likelihood estimator of θ under the null hypothesis. The score test has asymptotic level α and is consistent. Florian Pelgrin (HEC) Constrained estimators September-December 2010 12 / 16
Lagrange multiplier or Score test Proposition The Lagrange multiplier test is defined by the critical region: } W n = {y/t LM (y) χ 2 1 α (p) where the test statistic is given by: T LM = ˆλ t n h (ˆθn,0 ) θ t [ˆV asy { s (ˆθ n,0 )}] 1 h t (ˆθn,0 ) where ˆθ n,0 is the maximum likelihood estimator of θ under the null hypothesis. The Lagrange multiplier test has asymptotic level α and is consistent. Remark: The score and Lagrange multiplier test statistics are equal: T LM = T S. θ ˆλ n. Florian Pelgrin (HEC) Constrained estimators September-December 2010 13 / 16
Lagrange multiplier or Score test Remark: For ML estimation (with a correctly specified density), an (asymptotically) equivalent version of the LM test can be obtained from the following auxiliary procedure: Step 1: Estimate the constrained model and obtain ˆθ n,0. Step 2: Form the scores for each observation of the unrestricted model s i (θ) = l i(y i x i ; θ) θ and evaluate them at the constrained ML estimation, s i (ˆθ n,0 ). Step 3: Compute n times the uncentered R 2 (or the model sum of squares) from the auxiliary regression of 1 on s i (ˆθ n,0 ). Florian Pelgrin (HEC) Constrained estimators September-December 2010 14 / 16
7. Equivalence Equivalence Proposition The Wald, likelihood ratio, and Score tests are asymptotically equivalent. Under H 0, the differences T W,n T S,n T LR,n T W,n T LR,n T S,n converge in probability to zero as n. Remark: Their finite sample properties are however different!!! Florian Pelgrin (HEC) Constrained estimators September-December 2010 15 / 16
8. Summary Summary Why do we need asymptotic tests? What are the main asymptotic tests? What is the intuition? Definitions of these asymptotic tests. Interpretation! Florian Pelgrin (HEC) Constrained estimators September-December 2010 16 / 16