AGEC 661 Note Eleven Ximing Wu. Exponential regression model: m (x, θ) = exp (xθ) for y 0

Transcription

1 AGEC 661 ote Eleven Ximing Wu M-estimator So far we ve focused on linear models, where the estimators have a closed form solution. If the population model is nonlinear, the estimators often do not have a closed form solution and are defined as the solution of some minimization problem. Hence the name M-estimator. Example of nonlinear models. Let E y x = m x, θ. Exponential regression model: m x, θ = exp xθ for y 0 Logistic regression model: m x, θ = exp xθ / [1 + exp xθ] for 0 y 1. Formally, let y = m x, θ 0 + u, E u x = 0 where θ 0 denotes the vector of true parameters. onlinear least squares LS assumptions: LS.1: For some θ 0 Θ, E y x = m x, θ 0 LS.2: E { [m x, θ 0 m x, θ] 2} > 0, all θ Θ, θ θ 0. onlinear least squares: [y m x, θ] 2 = [y m x, θ 0 + m x, θ 0 m x, θ] 2 = [y m x, θ 0 ] 2 + [m x, θ 0 m x, θ] [m x, θ m x, θ 0 ] u. Taking expectation on both sides yields E [y m x, θ] 2 = E [y m x, θ 0 ] 2 + E [m x, θ 0 m x, θ] 2. 1

2 It follows that E [y m x, θ] 2 E [y m x, θ 0 ] 2 all θ Θ. For θ 0 to be identified, we need assumption LS.2 to rule out the case that E { [m x, θ 0 m x, θ] 2} = 0, for some θ Θ, θ θ 0. LS Estimator: min 1 θ Θ [y i m x i, θ] 2. More generally, let w = x, y and q w, θ be a function of the random vector w and the parameter vector θ. An M-estimator of θ 0 solves the problem min 1 θ Θ q w i, θ. 1 The parameter vector θ 0 is assumed to uniquely solve the population problem min E [q w, θ]. θ Θ Consistency: Under assumption LS.1 and LS.2, and the conditions of Theorem 12.1 p.347, then a random vector ˆθ, solves problem 1, and ˆθ p θ 0. [The conditions in Theorem 12.1 ensure that the objective function 1 q w i, θ converges to E [q w, θ] in probability for all θ Θ. The primary conditions include that Θ is compact and q w, is continuous in Θ and bounded.] Asymptotic ormality. If q w, is continuously differentiable on the interior of 2

3 Θ, then with probability approaching one ˆθ solves the first-order condition s w i, ˆθ = 0 where s w i, θ = [ q w i, θ / θ 1,..., q w i, θ / θ p ] = θ q w i, θ is the transpose of the gradient of q w i, θ. We call s w, θ the score of the objective function. Denote H i, the Hessian of the objective function q w i, θ, with respect to θ, where H w i, θ = 2 q w i, θ / θ θ = 2 θ q w i, θ. Define A 0 = E [H w, θ 0 ], B 0 = E [ s w, θ 0 s w, θ 0 ] = Var [s w, θ 0 ]. Under the regularity conditions given in Theorem 12.3 p.351, we have d ˆθ θ 0 ormal 0, A 1 0 B 0 A 1 0, Avar ˆθ = A 1 0 B 0 A 1 0 /. Estimation of variance. For convenience, let q w, θ = [y m x, θ] 2 /2. It follows that s w, θ = θ m x, θ [y m x, θ] E [s w, θ 0 x] = θ m x, θ 0 [E y x m x, θ 0 ] = 0. Thus the variance of s w, θ 0 is B 0 = E [ s w, θ 0 s w, θ 0 ] = E [ u 2 θ m x, θ 0 θ m x, θ 0 ]. 3

4 The Hessian of q w, θ is H w, θ = θ m x, θ θ m x, θ 2 θm x, θ [y m x, θ], where 2 θ m x, θ is the Hessian of m x, θ with respect to θ. Since E [y m x, θ 0 x] = 0, we have E [H w, θ 0 x] = θ m x, θ θ m x, θ. Taking expectations on both sides yields A 0 E [H w, θ 0 ] = E {E [H w, θ 0 x]} = E [ θ m x, θ 0 θ m x, θ 0 ]. Generally there is no simple relationship between A 0 and B 0. Under a homoskedasticity assumption, we can show that B 0 is proportional to A 0. Two commonly used estimators for H. 1 H w i, ˆθ 1 p Ĥ i H 2 1 A x i, ˆθ 1 p  i A0 3 B 0 is estimated by 1 s w i, ˆθ s w i, ˆθ 1 ŝ i ŝ p i B 0. We then have Avâr ˆθ θ0 =  1 ˆB 1, where  is estimated by either 2 or 3. 4

5 For nonlinear least squares, we always use  i = where θ ˆm i = θ m w i, ˆθ, and θ ˆm i θ ˆm i where û i = y m x i, ˆθ. We then have ŝ i = θ ˆm i [y i m x i, ˆθ ] = θ ˆm i û i, 1 1 Avâr ˆθ = θ ˆm i θ ˆm i û 2 i θ ˆm i θ ˆm i θ ˆm i θ ˆm i, which is call the heteroskedasticity-robust variance matrix estimator for LS. If we assume that E [ s w, θ 0 s w, θ 0 ] = σ 2 0E [H w, θ 0 ], we then have 1 Avâr ˆθ = ˆσ 2 Ĥ i 1 or Avâr ˆθ = ˆσ 2  i. Assumption LS.3 Vary x =Varu x = σ 2 0. The variance of LS estimator under this assumption takes the form 1 Avâr ˆθ = ˆσ 2 θ ˆm i θ ˆm i, ˆσ 2 = 1 P û 2 i. 5

6 Example: Exponential Regression Suppose y 0 and y = exp xθ 0 + u. We then have m x, θ = exp xθ θ m x, θ = x exp xθ A 0 = E [exp 2xθ 0 x x]. For the estimation for variance, we have θ m i θ m i = exp 2x i ˆθ x ix i. Under Assumption LS.3, the asymptotic variance takes the form ˆσ 2 1 exp 2x i ˆθ x ix i. 6