High Dimensional Empirical Likelihood for Generalized Estimating Equations with Dependent Data

Transcription

1 High Dimensional Empirical Likelihood for Generalized Estimating Equations with Dependent Data Song Xi CHEN Guanghua School of Management and Center for Statistical Science, Peking University Department of Statistics, Iowa State University A joint work with Jinyuan Chang (Melbourne and SWUFE) and Xiaohong Chen (Yale). June 14, / 43

2 To the Memory of Professor Peter Hall 2 / 43

3 A Letter from Peter in May / 43

4 4 / 43 Professor Hall s Contribution to Empirical Likelihood Hall and La Scala, 1990, Methodology and Algorithms of empirical likelihood. Intl. Statist. Review. (327) DiCiccio, Hall and Romano, 1991, Empirical Likelihood is Bartlett-Correctable. Annals Statist. (191) Chen and Hall, 1993, Smoothed Empirical Likelihood Confidence Intervals for Quantiles. Annals Statist. (115) DiCiccio, Hall and Romano. 1989, Comparison of Parametric and Empirical Likelihood Functions. Biometrika. Hall 1990, Pseudo-Likelihood Theory for Empirical Likelihood. Annals Statist. Hall and Owen, 1993, Empirical Likelihood Confidence Bands in Density Estimation. J. Comput. Graph. Statist.

5 5 / 43 Generalized Method of Moment Consider r-dimensional estimating function g n : R d Θ p R r, g n (X t, θ) = (g n1 (X t, θ),, g nr (X t, θ)) T on d-dimensional data X t R d and p-dim parameter θ Θ p R p. Satisfy the moment restriction Eg n (X t, θ 0 ) = 0 for a θ 0 Θ p.

6 6 / 43 GMM (Hansen, 1982) is the widest framework for statistical and econometrical analysis. Includes as special cases MLE, Regression... A Special case of the M-estimation In Statistics, it is closely related to Generalized Estimating Equations (GEE). Godambe (1991).

7 7 / 43 High Dimensionality Conventional GMM has FIXED r the number of moment restrictions p the dimension of θ, d the dimension of X t. Possible over-identification r p. Modern high dimensional data have much larger r, p and d: r, p, d as n.

8 8 / 43 Empirical Likelihood Owen (1988, Biometrika; 1990, AoS), motivated by the profile likelihood as well as the bootstrap. It produces a nonparametric likelihood and allows likelihood type inference in nonparametric or semiparametric settings. EL admits Wilks Theorem and Bartlett Correction (BC) in wide range of inferential situations.

9 9 / 43 Selected Literature on EL Qin and Lawless (1994): GEE Kitamura (1997): for dependent data EL, Block EL Newey and Smith (2004): Generalized EL Reviews on EL: Owen (2001), Kitamura (2007), Chen and van Keilegom (2009) for overviews.

10 10 / 43 Main properties of EL Have two key properties of the parametric likelihood. (1) Wilks theorem: -2 log EL ratio d χ 2 p. Established for many situations. (2) Bartlett correction: a mean adjustment to the log LR leads to the second order improvement in the convergence of distribution. DiCiccio, Hall and Romano (1991, AoS), Chen and Cui (2006, Biometrika; 2007, JoE).

11 11 / 43 EL for Estimating Equations Qin and Lawless (1994, AoS). X 1,, X n iid F and Eg(Xi ; θ 0 ) = 0. The empirical likelihood for θ { n L n (θ) = sup π i π i > 0, i=1 n π i = 1, i=1 n i=1 } π i g(x i, θ) = 0.

12 12 / 43 Computation of EL For each θ, by Lagrangian method, L n (θ) = n i=1 where λ n (θ) is the solution of n i=1 Computationally Intensive. { } 1 n λ, n(θ)g(x i, θ) g(x i, θ) 1 + λ ng(x i, θ) = 0.

13 13 / 43 Maximum EL Estimator ˆθ n = arg max θ Θ log L n (θ). Has the same asymptotic variance as the GMM estimator. n(ˆθn θ 0 ) d N(0, V ). EL is of one step and GMM is of two step (an extra step due to the weighting matrix selection).

14 14 / 43 EL for Dependent Data Kitamura (1997) used the blocking method to capture the dependence. Obtained the Wilks theorem for the EL ratio.

15 15 / 43 Existing Works for High Dimensional EL Donald, Imbens and Newey (2003) Hjort, McKeague and van Keilegom (2009, AoS), Chen, Peng and Qin (2009, Bioka). EL for HD means, { n L n (µ) = sup π i π i > 0, i=1 n π i = 1, i=1 Log-EL ratio w n (µ) = 2 log{n n L n (µ)} n i=1 Want to investigate the growth rate of p such that (2p) 1/2 {w n (µ) p} d N(0, 1). A HD reflection of the Wilks theorem. } π i (X i µ) = 0.

16 16 / 43 Hjort, Mckeague and van Keilegom found p = o(n 1/6 ) under finite fourth moment. Chen, Peng and Qin (2009) improved to p = o(n 1/4 ) and p = o(n 1/2 1/(8k) ) if the 4k-th moment exists. Leng and Tang (2012) considered Penalized EL for variable selection.

17 17 / 43 Objectives of This Paper (1) Consistency and asymptotic normality of ˆθ n. (2) the asymptotic normality of the EL ratio 2 log{n n L n (θ)} under GMM when (i) {X t } n t=1 are weakly dependent; (ii) the three dimensions r, p and d are all diverging as n.

18 18 / 43 α mixing, a Measure of Dependence For stationary process {X t } and two integers u v, define F v u = σ(x i : u i v) the α mixing coefficient { } α(k) = sup P(A B) P(A) P(B) : A F, l B Fl+k. The process is said to be α-mixing if lim α(k) = 0. k

19 19 / 43 Blocking Method Data blocks of length M with separation L B q = (X (q 1)L+1,, X (q 1)L+M ) for q = 1,, Q = [(n M)/L] + 1. The average of g(x, θ) over the data blocks φ M (B q, θ) = 1 M M g(x (q 1)L+m, θ). m=1

20 20 / 43 Blockwise EL, Kitamura (1997) { Q L n (θ) = sup q=1 Q Q π q π q > 0, π q = 1, q=1 q=1 } π q φ M (B q, θ) = 0. (1) ˆθ n = arg max θ Θ p log L n (θ),

21 21 / 43 Technical Condition (A.1) (i) k=1 kα X (k) 1 2/γ < for a γ > 2. (ii) the block size M and separation L satisfy M L and M/L c 1. (iii) Eg(X t, θ 0 ) = 0; 2 (ε) such that for any ε > 0, there are positive functions 1 (r, p) and inf Eg(X t, θ) 2 1 (r, p) 2 (ε) > 0, {θ Θ p: θ θ 0 2 ε} where lim r,p 1 (r, p) > 0; (iv) sup θ Θp ḡ(θ) Eg(X t, θ) 2 = o p { 1 (r, p)}.

22 22 / 43 Condition (A.2) (i) In a neighborhood of θ 0, g(x, θ) is cont. differentiable wrt θ. (ii) For a scalar ζ 2 (r) and a nonnegative B n ( ) where EB γ n (X t ) C. r 1/2 sup θ Θ p g(x, θ) 2 B n (x), (iii) E g j (X t, θ 0 ) 2γ C for all j = 1,, r.

23 23 / 43 Let V n = Var{n 1/2 n t=1 g(x t, θ 0 )}. (iv) C < λ min (V n ) λ max (V n ) < 1/C for all n, sup θ Θp λ max {n 1 n t=1 g t(θ)g t (θ)} C with w.p.a.1 and λ min {[ θ ḡ(θ)] [ θ ḡ(θ)]} C w.p.a.1 in a neighborhood of θ 0.

24 24 / 43 (A.3) In a neighborhood of θ 0, g(x, θ) is twice continuously differentiable wrt θ, and there exist nonnegative T n,ij (x) and K n,ijk (x) such that for all i = 1,, r and j, k = 1,, p g i (x, θ) θ T n,ij(x) and j 2 g i (x, θ) θ j θ k K n,ijk(x), where ET 4 n,ij (X t) C and EK 2 n,ijk (X t) C for all n.

25 25 / 43 Consistency of Maximum EL Estimator ˆθ n r 2 M 2 2/γ n 2/γ 1 = o(1) and r 2 M 3 n 1 = o(1). (2) Theorem 1: Assume (A.1), (A.2) and the eigenvalues of V M are uniformly bounded. Then, if (2) holds, θ n θ 0 2 p 0. If in addition, r 2 pm 2 n 1 = o(1), then θ n θ 0 2 = O p (r 1/2 n 1/2 ) and λ( θ n ) 2 = O p (r 1/2 Mn 1/2 ).

26 26 / 43 Remark on Consistency: Independent Case For independent data α X (k) 0 and block size M = 1. Then, the restriction in (2) becomes r 2 n 2/γ 1 = o(1). Hence, r = o(n 1/2 1/γ ) ensures the consistency of ˆθ n. If γ is large enough, r will be close to o(n 1/2 ), the best rate we can established.

27 27 / 43 Remark on Consistency: Dependent Case For dependent data, M should diverge faster than r. Explicitly, M = O(n {(γ 2)/(4γ 2)} 1/5 ) and r = o(n {(γ 2)/(4γ 2)} 1/5 ). If γ 8, M = O(n 1/5 ) and r = o(n 1/5 ). A much slowed down rate for r.

28 28 / 43 As Extensions to Existing Results on Consistency If r is fixed and the data are independent, ˆθ n θ 0 2 = O p (n 1/2 ), Qin and Lawless (1994) and Newey and Smith (2004). If r is fixed but the data are dependent, ˆθ n θ 0 2 = O p (n 1/2 ) Kitamura (1997). If r is diverging and the data are independent, both θ n θ 0 2 and λ( θ n ) 2 are O p (r 1/2 n 1/2 ), which retain the results in Donald, Imbens and Newey (2003).

29 29 / 43 Refinement of Consistency Under (A.1)-(A.3), then ˆθ n θ 0 2 = O p (p 1/2 n 1/2 ) provided r 3 M 2 2/γ n 2/γ 1 p 1 = o(1), r 3 M 3 n 1 p 1 = o(1) and p 1/2 r 3/2 M 1 = o(1).

30 30 / 43 Asymptotic Normality Under (A.1)-(A.3), then for any α n R p with unit L 2 -norm, nα n {[E θ g t (θ 0 )] V 1 n [E θ g t (θ 0 )]} 1/2 (ˆθ n θ 0 ) = nα n{[e θ g t (θ 0 )] Vn 1 [E θ g t (θ 0 )]} 1/2 [E θ g t (θ 0 )] Vn 1 ḡ(θ 0 ) + O p (r 3/2 M 3/2 n 1/2 ) + O p (r 3/2 M 1 1/γ n 1/γ 1/2 ) + O p (r 3/2 M 1 ) provided that r 2 M 2 2/γ n 2/γ 1 = o(1), r 2 M 3 n 1 = o(1).

31 31 / 43 Asymptotic Normality for Dependent Data Let β n = Vn 1 [E θ g t (θ 0 )]{[E θ g t (θ 0 )] Vn 1 [E θ g t (θ 0 )]} 1/2 α n,p. The leading term in above expression is 1 n n β ng t (θ 0 ). t=1 For dependent data, we need sup n E β ng t (θ 0 ) 2+v < for some v > 0, which ensures n Var[n 1/2 β ng t (θ 0 )] t=1 exists.

32 32 / 43 For high dimensional g(x, θ) with diverging r, we need A sufficient condition for (3) is to restrict β n D(K ) := where K is a finite constant. sup E β ng t (θ 0 ) γ <. (3) n { (v 1, v 2, ) R : k=1 } v k K,

33 33 / 43 Asy. Normality for Dependent Data, r, p Under conditions (A.1)-(A.3), if r 3 M 2 2/γ n 2/γ 1 = o(1), r 3 M 3 n 1 = o(1) and r 3/2 = o(m), then for any α n R p with unit L 2 -norm such that (3) holds, nα n {[E θ g t (θ 0 )] V 1 n [E θ g t (θ 0 )]} 1/2 (ˆθ n θ 0 ) d N(0, 1).

34 34 / 43 Remark If M = O(n {(γ 2)/(4γ 2)} 1/5 ) and r = o(n {(γ 2)/(6γ 3)} 2/15 ), then nα n {[E θ g t (θ 0 )] V 1 n [E θ g t (θ 0 )]} 1/2 (ˆθ n θ 0 ) d N(0, 1). regardless p being fixed or diverging. The best growth rate for r is r = o(n 2/15 ) when γ 8.

35 35 / 43 Asymptotic Normality for Dependent Data for Fixed p but r Under conditions (A.1)-(A.3), then for any unit α n,p R p, nα n {[E θ g t (θ 0 )] V 1 n [E θ g t (θ 0 )]} 1/2 (ˆθ n θ 0 ) d N(0, 1), provided r 3 M 2 2/γ n 2/γ 1 = o(1), r 3 M 3 n 1 = o(1) and r 3/2 = o(m).

36 36 / 43 Remark Then M = O(n {(γ 2)/(4γ 2)} 1/5 ) and r = o(n {(γ 2)/(6γ 3)} 2/15 ), ensure the asymptotic normality. The best growth rate for r is still r = o(n 2/15 ) when γ 8. No improvement in the rate for r despite p being fixed.

37 37 / 43 Asymptotic Normality for Independent Data, r, p Under conditions (A.1)-(A.3), then for any unit α n R p, nα n {[E θ g t (θ 0 )] V 1 n [E θ g t (θ 0 )]} 1/2 (ˆθ n θ 0 ) d N(0, 1), provided r 3 p 2 n 1 = o(1) and r 3 n 2/γ 1 = o(1).

38 38 / 43 If p is fixed, r = o(n 1/3 2/(3γ) ) as Donald, Imbens and Newey (2003). If p grows with r and p/r y (0, 1], r = o(n {1/3 2/(3γ)} 1/5 ). In particular, if γ 5, r = o(n 1/5 ) as in Leng and Tang (2011). From dependent to independent, a substantial increase in the rate from o(n 1/5 ) to o(n 2/15 ) under the most favorable moment conditions.

39 39 / 43 EL Ratios Let w n (θ) = 2 log{q Q L n (θ)} be the log EL ratio. For fixed dimensional EL, w n (θ 0 ) d χ 2 r for a wide range of situations. Under the high dimensional setting, a natural form of the Wilks theorem is (2r) 1/2 {w n (θ 0 ) r} d N(0, 1).

40 40 / 43 By Taylor expansion, The key is to show w n (θ 0 ) = nḡ (θ 0 )Vn 1 ḡ(θ 0 ) QR n ˆΩ 1 (θ 0 )R n + 2 Q [λ (θ 0 )φ q (θ 0 )] 3 3 [1 + c q1 λ (θ 0 )φ q (θ 0 )] 3 q=1 + O p (r 2 M 1 ) + O p (r 2 M 3/2 n 1/2 ). (2r) 1/2 {nḡ (θ 0 )V 1 n ḡ(θ 0 ) r} d N(0, 1).

41 41 / 43 Modify A.1(i) to Define α X (k) 1 2/γ k η for some η > 8. ξ = η 8 4η {8<η<32} {32 η< } {η=,dependent data} + 1 {η=,independent data}. Suppose (A.1)-(A.2), if r = o(n ξ ), then provided (2r) 1/2 {w n (θ 0 ) r} d N(0, 1) r 3 M 2 2/γ n 2/γ 1 = o(1), r 3 M 3 n 1 = o(1) and r 3/2 M 1 = o(1).

42 42 / 43 Remark For independent data, this theorem implies that the asymptotic normality of the EL ratio is valid if r = o(n 1/3 2/(3γ) ) which is the same as that attained in Hjort et al. (2009). For dependent data, the normality of the EL ratio holds if r = o(n δ ) for { η 8 δ = 4η {8<η<32} + 2 } 11 1 {32 η } { γ 2 6γ 3 1 {2 γ 8} + 2 } 15 1 {γ>8}. The best rate is r = o(n 2/15 ).

43 43 / 43 Chang, J-Y, Chen, S.X. and X. Chen (2015). J. Econometrics, 185, THANK YOU