Graduate Econometrics I: Maximum Likelihood II

Transcription

1 Graduate Econometrics I: Maximum Likelihood II Yves Dominicy Université libre de Bruxelles Solvay Brussels School of Economics and Management ECARES Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 1/30

2 Outline Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 2/30

3 Outline Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 3/30

4 Principle Assume pairs (y i, x i ) with i = 1,..., n of i.i.d. observations and have a pdf f (y i, x i ; θ) that may be decomposed as : f (y i, x i ; θ) = f (y i x i ; θ)f (x i ; θ), where f (y i x i ; θ) and f (x i ; θ) are the conditional and marginal pdfs, respectively. Definition i) A marginal MLE for θ is a solution ˆθ m,n to the problem : max θ Θ n log f (x i ; θ). ii) A conditional MLE of θ is a solution ˆθ c,n to the problem : max θ Θ i=1 n log f (y i x i ; θ). i=1 Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 4/30

5 Principle We partition θ = (α, β, γ ) as : (α, β ) conditional (β, γ ) marginal. Then : f (y i, x i ; θ) = f (y i x i ; α, β)f (x i ; β, γ). In conditional and marginal models, the notion of cut is important. There is a cut when there are no common parameters. Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 5/30

6 Principle Property Suppose that there is a cut so that : f (y i, x i ; θ) = f (y i x i ; α)f (x i ; γ) (α, γ A C). The MLE of θ is ˆθ n = (ˆα c,n, ˆγ m,n). If the parameter of interest is α, we can neglect the marginal distribution. Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 6/30

7 Asymptotic Properties We start with some regularity conditions : A1 The ones given previously but adapted to conditional and marginal models. A2 (Y i, X i ) are i.i.d. pairs f (y i x 1,..., x n; θ) = f (y i x i ; θ) f (y i x i ; θ) = f (y j x j ; θ). A3 (X i ) i = 1,..., n are mutually independent. A4 Identification : f (y x; α, β) = f (y x; α 0, β 0 ) x, y { α = α0 β = β 0, f (x; β, γ) = f (x; β 0, γ 0 ) x { γ = γ0 β = β 0. Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 7/30

8 Asymptotic Properties Notation : E x( ) = X f (x; β, γ)dx E( x) = θ m = (β, γ ) θ c = (α, β ) Y f (y x; α, β)dy Property Marginal MLE. Under the regularity conditions i) ˆβ m,n and ˆγ m,n exist asymptotically. ii) ˆβ m,n and ˆγ m,n are consistent for β 0 and γ 0. iii) ˆβ m,n and ˆγ m,n are asymptotically normal with : (( ) ( )) ˆβm,n β0 n d N(0, Ĩm 1 (θ ˆγ m,n γ 0 )), 0 ( ) where Ĩ m(θ 0 ) = E x 2 log f (x;β 0,γ 0 ). θ m θ m Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 8/30

9 Asymptotic Properties Property Conditional MLE. Under the regularity conditions, ˆα c,n and ˆβ c,n i) ˆα c,n and ˆβ c,n exist asymptotically. ii) ˆα c,n and ˆβ c,n are consistent for α 0 and β 0. iii) ˆα c,n and ˆβ c,n are asymptotically normal with : ) ( )) ((ˆαc,n α0 n ˆβ d N(0, Ĩc 1 (θ 0 )), c,n ( where Ĩ c(θ 0 ) = E xe β 0 2 log f (y x;α 0,β 0 ) θ c θ c ). Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 9/30

10 Asymptotic Properties Remarks : 1 Ĩ c(θ 0 ) depends on γ 0 since it takes expectation with respect to the true marginal distribution of X. 2 Ĩ m(θ 0 ) and Ĩ c(θ 0 ) are the Fisher information matrices of the marginal and conditional model respectively. They are not I(θ 0 ), the Fisher information matrix of the joint density. Property The estimators ˆα c,n, ˆβ c,n, ˆβ m,n and ˆγ m,n are asymptotically at most as efficient as the MLE ˆα n, ˆβ n and ˆγ n. Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 10/30

11 Asymptotic Properties Corollary i) The conditional MLE ˆα c,n is asymptotically as efficient as the ML estimator ˆα n in the following two cases : 1 where there are no β, 2 where ˆα n and ˆβ n are uncorrelated. ii) The marginal MLE ˆγ m,n is asymptotically as efficient as the ML estimator ˆγ n in the following two cases : 1 where there are no β, 2 where ˆβ n and ˆγ n are uncorrelated. Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 11/30

12 Asymptotic Properties Finally, Ĩ m(θ 0 ) and Ĩ c(θ 0 ) may be estimated consistently. For example, for Ĩ c(θ 0 ) : or 1 n n i=1 1 n n i=1 2 log l(y i x i ; ˆα c,n, ˆβ c,n) θ c θ c log f (y i x i ; ˆα c,n, ˆβ c,n) log f (y i x i ; ˆα c,n, ˆβ c,n). θ c θ c Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 12/30

13 Outline Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 13/30

14 Principle We now consider the case where the true model is misspecified. The true distribution is : l 0 (y 1,..., y n x 1,..., x n) = n f 0 (y i x i ), where f 0 (y i x i ) does not belong to the specified family, i.e. i=1 f 0 (y i x i ) {f (y x; θ), θ Θ}. θ is going to converge to some value which is not the true but the pseudo-true θ 0. It corresponds to the distribution in the model that is closest to f 0 : and we assume that θ 0 is unique. θ 0 = arg max θ Θ ExE 0 log f (Y X; θ) Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 14/30

15 Principle Definition A Pseudo-MLE (PMLE) ˆθ n of θ is the solution to max θ Θ n log f (Y i X i ; θ). i=1 Regularity conditions : A1 The pairs (Y i, X i ) are i.i.d. A2 Θ is compact. A3 n i=1 log f (y i x i ; θ) is continuous in θ and integrable with respect to the true distribution of (Y i, X i ) for every θ. A4 1 n n i=1 log f (y i x i ; θ) converges uniformly on Θ to E xe 0 log f (Y X; θ). A5 The limit problem max θ Θ E xe 0 log f (Y X; θ) has an unique solution which is θ 0. Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 15/30

16 Principle Property Under the assumptions A1-A5, the PMLE ˆθ n converges in probability to θ 0. We now establish the asymptotic normality and the asymptotic variance-covariance function. Regularity conditions : A6 n i=1 log f (y i x i ; θ) is twice continuously differentiable in θ. A7 The matrix ( ) J = E xe 0 2 log f (Y X; θ) θ θ exists and is non-singular. Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 16/30

17 Principle Property Under A1-A7, the PMLE is asymptotically normally distributed with : n(ˆθ n θ 0 ) d N(0, J 1 IJ 1 ), where : I = E xe 0 ( log f (Y X; θ 0 ) θ Proof : It relies heavily on the proof of MLE. ) log f (Y X; θ0 ). θ The difference comes from the fact that J and I are NOT equal because the Fisher information matrix cannot be expressed as J. Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 17/30

18 Principle In the ML we had : then : l(y ; θ) θ ( ) log l(y ;θ) l(y ; θ) θ θ 2 log l(y ;θ) l(y ; θ) = θ θ Y I = J. = log l(y ; θ) l(y ; θ) θ = 2 log l(y ;θ) l(y ; θ) + θ θ 2 log l(y ;θ) l(y ; θ)dy = θ θ Y log l(y ;θ) log l(y ;θ) l(y ; θ) θ θ log l(y ;θ) θ log l(y ;θ) l(y ;θ) = 0 θ θ log l(y ;θ) l(y ; θ)dy θ Only possible when the assumed density is the true density (MLE). Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 18/30

19 Principle In other words, in MLE : l(y ;θ) dy = log l(y ;θ) l(y ; θ)dy = 0 Y θ Y θ = log l(y ;θ) l Y θ 0 (Y )dy=0 because l(y ; θ) and l 0 (Y ) are asymptotically the same. In PMLE : Y l(y ;θ) dy = log l(y ;θ) l(y ; θ)dy = 0 θ Y θ log l(y ;θ) l Y θ 0 (Y )dy 0 because l(y ; θ) and l 0 (Y ) are not the same. And since the score has not mean equal to zero, the estimators ˆθ n do not converge to θ 0 but to θ 0. Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 19/30

20 Outline Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 20/30

21 The model now focuses on the relationship between X and Y via the conditional mean of Y given X : where E x(u i X 1,..., X n) = 0. Y i = m(x i, b 0 ) + u i, Some form of the density has to be proposed. The aim is to find those densities that lead to consistent estimators of b 0 even if they are misspecified. More precisely, we consider a family of densities parametrized by a (conditional) mean m : f (y, m) m M, where M contains all possible values for m(x, b). Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 21/30

22 The conditional distribution of Y i given X i is given by f (y i ; m(x i, b)) and the PMLE for ˆb n is the solution to : max b n log f (y i ; m(x i, b)). i=1 We know that if the model is misspecified, ˆb n converges to a pseudo true value b o, which is the solution of the limit problem : max E xe 0 log f (Y ; m(x, b)), b but in some cases, b 0 = b 0 and hence ˆb n is consistent. The model is assumed to be first order (i.e. mean) identified and differentiable with respect to b. Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 22/30

23 Property Under A1-A5, ˆb n is consistent for b 0 for any possible ˆb n, any functional form of m( ) and any conditional distribution, satisfying : Y i = m(x i, b 0 ) + u i, E x(u i X 1,..., X n) = 0, if the pseudo true densities are of the form f (y; m) = exp{a(m) + B(y) + C(m)y}, where m is the mean of f (y; m). Such density is called linear exponential density (LEF). Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 23/30

24 Proof : The PMLE corresponds to the LEF And the limit problem is i.e. or f (y; m) = exp{a(m) + B(y) + C(m)y}. max E xe 0 log f (Y ; m(x, b)), b max E xe 0 [A(m(X, b)) + B(y) + C(m(X, b)y )] b is the solution to this problem is b 0. max E xe 0 [A(m(X, b)) + C(m(X, b)y )] b Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 24/30

25 The Kullback inequality implies : log f (y, m(x, b 0 ))f (y, m(x, b 0 ))dy Y log f (y, m(x, b))f (y, m(x, b 0 ))dy. Y If we denote : D 1 = log f (y, m(x, b))f (y, m(x, b 0 ))dy Y D 2 = log f (y, m(x, b 0 ))f (y, m(x, b 0 ))dy. Y Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 25/30

26 Then D 1 {A(m(X, b)) + B(y) + C(m(X, b))y}f (y, m(x, b 0 ))dy Y = A(m(X, b)) f (y; m(x, b 0 ))dy + Y B(y)f (y; m(x, b 0 ))dy + C(m(X, b)) f (y; m(x, b 0 ))ydy. Y Denoting W = Y B(y)f (y; m(x, b 0))dy and by noting that Y f (y; m(x, b 0))dy = 1 and that Y f (y; m(x, b 0))ydy = m(x, b 0 ), we have : =A(m(X, b)) + W + C(m(X, b))m(x, b 0 ). Y Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 26/30

27 D 2 D 2 = A(m(X, b 0 )) + W + C(m(X, b 0 ))m(x, b 0 ). Therefore the Kullback inequality reduces to : A(m(X, b 0 )) + C(m(X, b 0 ))m(x, b 0 ) A(m(X, b)) + C(m(X, b))m(x, b 0 ), and since b is first order identifiable and b 0 is the PMLE, it follows that the equality is only possible if : m(x, b) = m(x, b 0 ) b = b 0 b 0 = b 0 ˆb n = b 0. And hence b 0 is the value that minimizes the Kullback-Leiber distance. Or it is the value that maximizes the psuedo-likelihood. Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 27/30

28 Remarks : 1 2 C(m(X,b)) m = Σ 1 where Σ is the variance-covariance matrix associated with the pseudo true distribution. A(m(X,b)) m + C(m(X,b)) m(x, b) = 0 is the FOC. m 3 Observations need not have the same support as that of the pseudo true distribution. For instance, a pseudo true family of Poisson can be used when the variables y are not positive integer-valued. 4 However, it is crucial to take into account the constraint m(x, b) M. For instance, the Poisson distribution can be used only if the conditional mean m(x, b) is positive valued. Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 28/30

29 Property Under A1-A7 a consistent PMLE associated with a LEF is asymptotically normally distributed with n(ˆbn b 0 ) d N(0, J 1 IJ 1 ), where : ( ) m I = E x b Σ 1 ΩΣ 1 m b ( ) m m J = E x b Σ 1. b Ω is the true variance covariance matrix of Y conditional on X, Σ is the conditional variance covariance matrix associated with the chosen pseudo-likelihood. Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 29/30

30 Property The variance covariance matrix of a PMLE is at least as large as : K 1 = ( )) m 1 m (E x b Ω 1. b The existence of this lower bound raises the question of whether it is possible to find a consistent estimator of which the asymptotic variance covariance attains the lower bound. This estimator would be asymptotically better than any other consistent PMLE. Yves Dominicy Graduate Econometrics I: Maximum Likelihood II 30/30