Estimation with Overidentifying Inequality Moment Conditions Technical Appendix

Transcription

1 Estimatio with Overidetifyig Iequality Momet Coditios Techical Aedix Hyugsik Roger Moo Uiversity of Souther Califoria Frak Schorfheide Uiversity of Pesylvaia, CEPR ad NBER August 27, 2008 Not iteded to aear i rit Corresodece: H.R. Moo: Deartmet of Ecoomics, KAP 300, Uiversity Park Camus Uiversity of Souther Califoria, Los Ageles, CA moor@usc.edu. F. Schorfheide: Deartmet of Ecoomics, 378 Locust Walk, Uiversity of Pesylvaia, Philadelhia, PA schorf@ssc.ue.edu.

2 Cosistecy For a recise defiitio of the otatio see the mai text.. Assumtios Assumtio a X i, i =,..., are strictly statioary o a robability sace Ω, F, P ; b Θ is a m-dimesioal comact subset of R m, where m h, θ,0 θ 0, θ,0 Θ, ad θ 0 Θ; c gx, θ is cotiuous at each θ Θ with robability oe; d IE g X i, θ,0 ] = 0, ad if IE g X i, θ] > 0 for θ θ,0 ; e ν,0 ν 0 ad ν,0 u 0 0, ] h2 ; f IE gx i, θ,0 gx i, θ,0 ] J is o-sigular; g Z = O ; h V = ν R h2 : ν 0 ad v K, ν,0 V, ad ν 0 lies i the iterior of V; i IE su g X i, θ α ] K < for some α > 2; j for ay θ ad θ, g X i, θ g X i, θ L X i l θ θ, for some measurable fuctio L of X i such that su IE L X i <, ad l y 0 as y 0..2 Mai Results Theorem Suose that Assumtio is satisfied. The ˆθ θ,0 0 ad ˆν ν,0 0. Moreover, ˆλˆθ, ˆν 0. Throughout this aedix we are frequetly usig the followig results. Notice that Assumtio imlies that su max g X i, θ = O /α, i su g X i, θ α = O, 2 i= g Xi, θ g X i, θ IE g X i, θ g X i, θ ] = o. 3 i= Accordig to Assumtios ad 2, ] IE g X i, θ ] ad IE g 2 j X i, θ are equicotiuous uiformly i θ, ad su g X i, θ IE g X i, θ] = o 5 i= θ] su g 2 j X i, θ IE g 2 j X i, = o for all j =,..., h. i= 4

3 2 See, for istace, Adrews 992. Proof of Theorem : We have to show that for ay δ > 0 lim ˆθ P Bθ,0, δ, ˆν Bν,0, δ =, where Bθ, δ = θ Θ θ θ < δ, Bν, δ = ν V ν ν < δ. Defie Θ c 0 = Θ B θ,0, δ c ad N c 0 = V B ν,0, δ c. To simlify the otatio we omit the subscrit from the sets Θ c 0 ad N c 0. Recall that accordig to Assumtio i, the costat α > 2 is such that IEsu gx i, θ α ] < K. We show the followig two statemets are true: i For a give ε, δ > 0 ad ζ such that α < ζ < 2, there exist ositive costats η ad κ ad such that for where ad ii P Ḡ θ,0, ν,0 ζ κ η < ε 2, 6 Ḡ θ,0, ν,0 = max G θ,0, ν,0, λ, λ ˆΛ θ,0 P mi Ḡ c 0, ν N 0 c θ, ν ζ η < ε 2. 7 The, from 6 ad 7 we deduce that there exists a η > 0 such that for : P ˆθ B θ,0, δ, ˆν Bν,0, δ P Ḡ θ,0, ν,0 < ζ κ η, mi Ḡ c 0, ν N 0 c θ, ν > ζ η ɛ. Proof of i: By Lemma 2 Ḡ θ,0, ν,0 O /. Choose κ > 0 such that ζ + κ <. The as required. ζ+κ Ḡ θ,0, ν,0 O ζ+κ = o Proof of ii: To obtai a lower boud for Ḡ θ, ν we will evaluate the fuctio G θ, ν, λ at λ = ζ uθ, ν, where the fuctio uθ, ν is defied as such that uθ, ν. uθ, ν = 0 if θ = θ,0, ν = ν,0 IEgX i,θ] M ν IEgX i,θ] M ν otherwise Moreover, we trucate the fuctio gx, θ as follows. Sice α > 2, we ca choose a ositive costat ξ such that α 2 < ξ < 2α.

4 3 Let X = x : su g x, θ ξ We the relace the terms ad g x, θ = I x X g x, θ. l + λ gx, θ λ M ν i the defiitio of the objective fuctio G θ, ν, λ by q x, θ, ν = l + ζ u θ, νg x, θ ζ u θ, νm ν. I what follows, we deduce the required result for ii by showig that ad ii-a: mi c 0,ν N 0 c ii-b: P q X i, θ, ν mi Ḡ θ, v + o ζ c 0,ν N 0 c i= mi c 0,ν N c 0 q X i, θ, ν < ζ η ε 2. i= Proof of ii-a: Notice that ζ u θ, ν Λ ζ ˆΛ θ w..a. by Lemma. The, by Lemma 4 ad by the defiitio of ˆλ θ, v, mi c 0,ν N 0 c q X i, θ, ν i= = mi c 0,ν N c 0 mi c 0,ν N c 0 l + ζ u θ, ν g X i, θ ] ζ u θ, ν M ν + o ζ i= i= = mi Ḡ θ, v + o ζ, c 0,ν N 0 c as required. ] l + ˆλ θ, v g X i, θ ˆλ θ, v M ν + o ζ Proof of ii-b: A secod-order Taylor exasio of q aroud u θ, ν = 0 yields ζ q x, θ, ν = uθ, ν g x, θ M ν ζ u θ, νg x, θg x, θ uθ, ν 2 + ζ u θ, νg x, θ 2, 8 where u θ, ν lies betwee zero ad uθ, ν. The secod-order term of the Taylor aroximatio 8 ca be bouded as follows. For give x, θ, ad ν su ζ u θ, ν g x, θ ζ su g x, θ ζ+ξ ζ/2, ν sice ξ < 2α < ζ 2. Therefore, su, ν ζ uθ, ν g x, θg x, θ uθ, ν + ζ u θ, ν g x, θ 2 su, ν ζ g x, θ 2 uθ, ν 2 ζ/2 2 ζ+2ξ = o. 9

5 4 Now cosider the exected value of ζ q x, θ, ν. From 8, 9, ad by the domiated covergece theorem, we have ζ IE q X i, θ, ν] = u θ, νieg X i, θ] M ν + o 0 = o if θ = θ0, ν = ν,0 IEgX i, θ] M ν + o > 0 otherwise. The o terms absorb the secod-order term of the Taylor aroximatio ad the discreacy betwee IEg X, θ] ad IEgX, θ], which vaishes as X exads. From 0 ad the mootoe covergece theorem we ca deduce that ] lim ζ lim IE if q X i, θ, ν = 0 if θ = θ,0, ν = ν 0 δ 0 θ Bθ,δ, ν Bν,δ > 0 otherwise Sice Θ ad V are comact by assumtio, the sets Θ Bθ,0, δ c ad V Bν,0, δ c are comact. We ca cover both Θ Bθ,0, δ c ad V Bν,0, δ c with Θ j = Bθ j, δ j ad N j = Bν j, δ j s, j =,..., J takig each δ j small eough such there exist η j s such that ] ζ IE if j, ν N j q X i, θ, ν 2η j, j for some ositive umbers η j = η j δ, j =,..., J. By the WLLN ad, for a give ε > 0, we ca fid j s such that j imlies that ε ] P ζ if q X i, θ, ν IE ζ if q X i, θ, ν 2J j, ν N j j, ν N j > η j i= ] P if q X i, θ, ν < IE if q X i, θ, ν ζ η j j, ν N j j, ν N j i= P if q X i, θ, ν < ζ η j j, ν N j i= P if q X i, θ, ν < ζ η j j, ν N j i= for j =,..., J. Now let lettig η = mi η,..., η J ad = max j=,...,j Notice that 2 IE P mi c 0, ν N c 0 P mi j=,...,j J P j= q X i, θ, ν < ζ η i= if j, ν N j if j, ν N j 4 ζ if j, ν N j q X i, θ, ν q X i, θ, ν < ζ η i= q X i, θ, ν < ζ η j ε 2, i=! IE j, we have for su 2 g X i, θ 2 + 2K + 2ζ+4ξ <. 2.

6 5 as required art ii-b. Combiig ii-a ad ii-b we have P mi Ḡ c 0,ν N 0 c θ, ν < ζ η ε 2, as required for ii. Sice ˆθ θ,0 ˆλˆθ, ˆν 0. ad ˆν ν,0 0 we ca deduce from Lemmas 2 ad 3 that.3 Techical Lemmas Lemma Suose that Assumtio is satisfied. The, i su λ g X i, θ 0,,λ Λ ζ, i ii Λ ζ ˆΛ θ w..a.. Proof of Lemma : See roof of Lemma A i Newey ad Smith Lemma 2 Suose that Assumtio is satisfied. Let θ Θ ad ν 0 be sequeces such that θ θ,0 0, ad ν ν,0 0. Moreover, i= g X i, θ = O ad i= g2 X i, θ ν = O. The, i ˆλ θ, ν exists w..a., ii ˆλ θ, ν = O /2, iii G θ, ν, ˆλ θ, ν O. Proof of Lemma 2: The roof is similar to that of Lemma A2 i Newey ad Smith Proof of i: Defie λ θ, ν = arg max G θ, ν, λ λ Λ ζ Sice Λ ζ is comact ad l + λ gx i, θ ν Mλ is cotiuous ad strictly cocave i λ the otimal solutio λ θ, ν exists ad is uique. Statemet i the follows from Lemma.

7 6 Proof of ii ad iii: Write ḡ i = gx i, θ. For some costat C 0 = G θ, ν, 0 G θ, ν, λ θ, ν = l i= = λ θ, ν λ θ, ν + λ θ, νḡ i ν M λ θ, ν i= ḡ i M ν 2 λ θ, ν i= ḡ i M ν C 4 λ θ, ν λ θ, ν, i= ḡ i ḡ i λ θ, + λ ḡ i 2 ν where λ lies o the lie joiig λ θ, ν ad 0. The last iequality holds because max i λ ḡ i = o accordig to Lemma ad i= ḡiḡ i coverges i robability to J, a ositive defiite matrix, by 3 ad Assumtio f. The remaider of the roof follows the roof of Lemma A2 i Newey ad Smith Lemma 3 Suose Assumtio is satisfied. The, i= g X i, ˆθ M ˆv ] = O. Proof of Lemma 3: The roof is similar to that of Lemma A.3 i Newey ad Smith Let ĝ i = g X i, ˆθ M ˆν ad ĝ = i= g X i, ˆθ M ˆν ]. Defie û ˆθ, ˆν = ζ ĝ ĝ. Recall the defiitio of u θ, ν i the roof of cosistecy. Aroximatio G θ, ν, λ with resect to λ aroud λ = 0 at θ, ν, λ = ˆθ, ˆν, û ˆθ, ˆν. The, G ˆθ, ˆν, û ˆθ, ˆν = G G ˆθ, ˆθ, ˆν, 0 ˆν, 0 + λ = ĝ û ˆθ, ˆν 2û ˆθ, ˆν û ˆθ, ˆν + ˆθ, 2 G 2û ˆθ, ˆν, λ ˆν λ λ ĝ i ĝ i + λ 2 û ĝ i i= ˆθ, ˆν, û ˆθ, ˆν where λ is located betwee 0 ad ûˆθ, ˆν. û Notice that max i ˆθ, ˆνĝ i 0 ad û ˆθ, ˆν Also, uder Assumtio i= ĝiĝ i 2 ˆΛ ˆθ by Lemma A. w..a.. = O. i= su g X i, θ 2 + K

8 7 The, w..a., for some costat C, ĝ û ˆθ, ˆν ˆθ, 2û ˆν The, ĝ i ĝ i + λ 2 û ˆθ, ˆν ĝ i i= = ζ ĝ ˆθ, 2û ˆν ζ ĝ C 2ζ G ĝ i ĝ i + λ 2 û ˆθ, ˆν ĝ i i= ζ ĝ 2 max ˆθ, i + λ 2 û ˆν ĝ i ĝ i ĝ i û ˆθ, ˆν ζ ĝ C 2ζ. 3 ˆθ, ˆν, û ˆθ, ˆν G ˆθ, ˆν, ˆλ i= su G θ,0, ν,0, λ O λ ˆΛ θ,0 4 where the first iequality is from 3, the secod ad third iequalities hold because ˆθ, ˆν, ˆλ is a saddle oit, ad the last iequality is from Lemma A.2 with i= g X i, θ,0 M ν,0 ] = O by Assumtio g. Also, by ζ < 2, ζ < 2 < ζ. Solvig 4 for ĝ gives ĝ O ζ. 5, For a give sequece ε 0, let λ = ε ĝ. Accordig to 5 λ = o ζ. Hece, λ Λ ζ w..a.. The, as i 4, we have λ ĝ C λ 2 = ε ĝ 2 Cε 2 ĝ 2 ε ĝ 2 Cε O. For large eough the term Cε is bouded away from zero ad it follows that ε ĝ 2 = O. Sice ε is a arbitrary sequece that teds to zero, we deduce that as required. ĝ = O, Lemma 4 Suose that Assumtio is satisfied. Let g x, θ = Ix X gx, θ where X = x : su g x, θ ξ,

9 8 where α 2 < ξ < 2α ad α > 2 as i Assumtio i. Defie q X i, θ, ν = l + ζ u θ, νg X i, θ ] ζ u θ, νm ν q X i, θ, ν = l + ζ u θ, νgx i, θ ] ζ u θ, νm ν ad assume that uθ, ν. The, su q X i, θ, ν q X i, θ, ν = o ζ., ν 0 i= Proof of Lemma 4: By the mea value theorem, su q, ν 0 X i, θ, ν q X i, θ, ν i= ζ u θ, νg X i, θ = su, ν 0 + ζ u I X i / X 6 i= θ, ν g X i, θ max su ζ u θ, ν g X i, θ i, ν 0 + ζ u θ, ν g X i, θ I su g X i, θ > ξ i= αξ max su ζ u θ, ν g X i, θ i + ζ u θ, ν g X i, θ su g X i, θ α, ν 0 i= where u θ, ν is located betwee 0 ad uθ, ν. The secod term o the right-had side of 6 ca be bouded as follows. Accordig to ζ max su i Moreover, uθ, ν. Therefore, max su ζ u θ, ν g X i, θ i + ζ u θ, ν g X i, θ, ν 0 g X i, θ = ζ+/α O. = 2 ζ max i su g X i, θ 2 ζ max i su g X i, θ ζ+/α O ζ+/α O = ζ+/α O. By Assumtio i ad the Markov iequality, the third term o the right-had side of 6 is O. Sice α < ξ < 2 2α, we are able to deduce that ζ su q X i, θ, ν q X i, θ, ν = αξ+ α O = o, as required., ν 0 i= 2 Limit Distributios Let β = θ, ν, λ ], β,0 = θ,0, ν,0, 0 h ], ad abbreviate G θ, ν, λ as G β. The objective fuctio is exaded aroud β,0 as follows: G β = G qβ + R β, 7

10 9 where G qβ = G β,0 + G β,0 β β,0 + 2 β β,0 G 2 β,0 β β,0. We begi by derivig the coefficiet matrices for the quadratic aroximatio of the objective fuctio G q β = G β,0 + G β,0 β β,0 + 2 β β,0 G 2 β,0 β β,0. 8 A direct calculatio shows that G β = G β θ, G β ν, G β λ], 9 where G β θ = G i= β ν = Mλ, G β λ = i= g X i, θ λ + λ, gx i, θ gxi, θ + λ gx i, θ M v. At β,0 the first derivatives simlify to G 2 β = G β,0 = 0, 0, /2 Z ]. 20 We roceed by artitioig the matrix of secod derivative as follows G 2 β θθ G 2 β θν G 2 β θλ G 2 β νθ G 2 β νν G 2 β νλ G 2 β λθ G 2 β λν G 2 β λλ, 2 where G 2 β θθ = i= h j= λ jg 2 j X i, θ + λ g X i, θ λλ g X i, θ gx i, θ + λ gx i, θ 2, G 2 β θν = 0, G 2 β νν = 0, G 2 β λν = M, G 2 β λθ = g X i, θ G 2 β λλ = i= i= + λ gx i, θ g X i, θ λ g X i, θ + λ gx i, θ 2 gx i, θgx i, θ + λ gx i, θ 2., At β,0 the secod derivatives simlify to G 2 β,0 = 0 0 Q 0 0 M Q M J. 22

11 0 The objective fuctio G qβ i terms of the trasformed arameters is: G qφ, l = 2 l J Z R φ φ,0 ] J l J Z R φ φ,0 ] Z R φ φ,0 J Z R φ φ,0 The fuctio G qφ, l is maximized with resect to l R h by ad the cocetrated objective fuctio is: lq φ = J Z R φ φ,0 24 Ḡ qφ = G qφ, l q φ = 2 Z R φ φ,0 J Z R φ φ, Assumtios Assumtio 2 a The true arameter θ 0 exists i a iterior of Θ; b gx i, θ is twice cotiuously differetiable with resect to θ; c the miimum eigevalue of IE g X i, θ]ie g X i, θ] is bouded below by a costat K > 0; d IE su g X i, θ 2] K <, IE su g 2 j X i, θ ] K < for j =,..., h; e for ay θ ad θ, g 2 j X i, θ g 2 j X i, θ L j X i l j θ θ, for some measurable fuctio L j of X i such that su IE L j X i <, ad l j y 0 as y 0. Assumtio 3 a For each θ, Q θ Qθ ad J θ Jθ. b For each θ, i= g2 j X i, θ lim IE g 2 j X i, θ]. c Z = Z, where Z N 0, J M ν 0 ν 0M. 2.2 Negligible Remaider Lemma 5 Suose Assumtios to 2 are satisfied, the for all γ 0 su β B : β β,0 γ where R β is the remaider term i 7. R β + β β,0 2 = o, 26 Proof of Lemma 5: By Lemma a of Adrews 999, it is sufficiet to rove su G 2 β G 2 β,0 = o, β B : β β,0 γ for every sequece γ 0. G 2 will subsequetly show that is defied i 2. To verify this sufficiet coditio we

12 G 2 i su β B: β β,0 γ β θθ G 2 β,0 θθ = o, G 2 ii su β B: β β,0 γ β λθ G 2 β,0 λθ = o, G 2 iii su β B: β β,0 γ β λλ G 2 β,0 λλ = o. We begi by showig that su + λ g X i, θ = O. 27 β B For ay give 0 < δ < 2, set K = δ. The, sice su i, β B λ g X i, θ δ imlies su i, β B K, P su i, β B +λ gx i,θ + λ g X i, θ > K P su λ g X i, θ > δ 0, i, β B which roves 27. Lemma. The covergece result for the uer boud ca be deduced from i Notice that su β B : β β,0 γ su λ j λ Λ ζ i= su β B, i λj g 2 j X i, θ + λ g X i, θ = O ζ O O = o, λ Λ ζ + λ g X i, θ su g 2 j X i, θ where the last iequality holds by the defiitio of Λ ζ, 27 ad 5. Moreover, g X i, θ λλ g X i, θ su β B : β β,0 γ i= + λ g X i, θ 2 su λ 2 su β B, i + λ g X i, θ 2 su g X i, θ = O 2ζ O O = o. The last iequality holds by the defiitio of Λ ζ, 27 ad 5. i=

13 2 ii Aly the triagle iequality to g X i, θ su β B : β β,0 γ + λ g X i= i, θ g X i, θ 0 g X i, θ su β B : β β,0 γ + λ g X i= i, θ g X i, θ ] + su g X i, θ IE g X i, θ i= ] + su IE g X i, θ IE g X i, θ 0 ] : θ θ 0 γ + i= ] g X i, θ 0 IE g X i, θ 0 = I d + o + o + o, where the last equality holds by 5 ad 4. Next, I d su λ g X i, θ su β B + λ g X i, θ β B = o O O O = o su g X i, θ by Lemma, 27, ad 5. Moreover, g X i, θ λ g X i, θ su β B : β β,0 γ + λ g X i= i, θ + λ g X i, θ /2 su λ su λ Λ ζ β B, i + λ g X i, θ 2 su g X i, θ 2 i= /2 su g X i, θ 2 i= = O ζ O O = o. i=

14 3 iii Similar as before, we have g X i, θ g X i, θ su β B : β β,0 γ i= + λ g X i, θ 2 g X i, θ 0 g X i, θ 0 g X i, θ g X i, θ su β B : β β,0 γ i= + λ g X i, θ 2 g X i, θ g X i, θ + su g Xi, θ g X i, θ IE g Xi, θ g X i, θ ] Θ i= + su IE g Xi, θ g X i, θ ] IE g Xi, θ 0 g X i, θ 0 ] Θ + su g Xi, θ Θ 0 g X i, θ 0 IE g Xi, θ 0 g X i, θ 0 ] i= g X i, θ g X i, θ = su + λ g X i, θ 2 g X i, θ g X i, θ + o. Next, β B : β β,0 γ su β B : β β,0 γ su β B, i su β B, i i= g X i, θ g X i, θ i= + λ g X i, θ 2 g X i, θ g X i, θ λ g X i, θ su β B, i + λ g X i, θ + λ g X i, θ + g X i, θ 2 = o O O O = o. su i= 2.3 Cosistecy Theorem 2 Suose Assumtios 3 are satisfied. The, i β q β,0 = O, ii ˆβ β,0 = O, iii G ˆβ = G q ˆβ +o, iv G q ˆβ = G q β q + o, ad v G ˆβ = G q β q + o. Theorem 2 establishes that ˆβ ad β q are -cosistet. Moreover, the theorem states that the discreacy betwee G β evaluated at ˆβ ad G qβ evaluated at β q vaishes. Thus, the large-samle behavior of likelihood ratios ca be aroximated by the behavior of G q β q. Let ˆb = ˆφ, ˆl ˆφ] ad b q = φ q, l q φ q ] be re-scaled versios of ˆβ ad β q. To rove the theorem, we will itroduce a third estimator ˆbq = ˆφ q, ˆl q ˆφ q ] where ˆlq φ = argmax l Lφ G qφ, l, ˆφq = argmi φ Φ G qφ, ˆl q φ.

15 4 ˆbq is based o the quadratic aroximatio of the objective fuctio, but the domais of φ ad l are restricted. I slight abuse of otatio we let B = Φ u 0 L. Proof of Theorem 2: i Follows from Lemma 7. ii Accordig to Lemma 2, ˆλˆθ, ˆν = O /2. It remais to show that ˆφ = ˆθ θ,0, u,0 + ] ˆν ν,0 is stochastically bouded. The saddleoit roerty imlies that 0 = G ˆφ, 0 G ˆφ, ˆl ˆφ G 0, ˆl0. 28 The usig the quadratic aroximatio 7, the boud for the remaider term give i Lemma 5 ad the defiitio of ˆl ad ˆφ we obtai G ˆφ, ˆl ˆφ = G q ˆφ, ˆl ˆφ + + ˆφ φ,0 2 + ˆl ˆφ 2 o 29 = 2 Z R ˆφ φ,0 J Z R ˆφ φ,0 2 ˆl ˆφ J Z R ˆφ φ,0 ] J ˆl ˆφ J Z R ˆφ φ,0 ] + + ˆφ φ,0 2 + ˆl ˆφ 2 o = 2 Z R ˆφ φ,0 J Z R ˆφ φ,0 + + ˆφ φ,0 2 + ˆl ˆφ 2 o, where φ,0 = 0, u,0]. The last equality is a cosequece of Lemma 8. Similarly, we ca deduce from Lemmas 2, 5, ad Assumtios 2 ad 3 that G 0, ˆl0 = 2ˆl 0 J ˆl 0 + Z ˆl ˆl0 2 o = O. 30 Hece, from 28, 29, ad 30 we obtai the iequality 0 2 Z + o R ˆφ φ,0 J Z + o R ˆφ φ,0 O. 3 Notice that Z + o = O. Accordig to Assumtio, R is full rak ad J is ositive defiite w..a.. Therefore, 3 imlies that ˆφ φ,0 is stochastically bouded. iii We deduce from Lemma 5 ad Part ii that G ˆβ = G q ˆβ β,0 + + ˆβ β,0 2 o = G q ˆβ + O o. iv We roceed by establishig o bouds for G q ˆβ G q β q. We begi with the uer boud. Usig iii we ca rewrite the differetial as G q ˆβ G q β q = G ˆφ, ˆl ˆφ + o Gq φ q, l q φ q 32 G ˆφ q, ˆl ˆφ q Gq φ q, ˆl φ q + o.

16 5 Relacig ˆφ by ˆφ q raises G, whereas substitutig l q with ˆl lowers G q. Usig Lemma 5 the first term o the right-had side of 32 ca be rewritte as G ˆφ q, ˆl ˆφ q = Gq ˆφ q, ˆl ˆφ q + o + ˆφ q φ,0 2 + ˆl ˆφ q 2 33 = G q ˆφ q, ˆl ˆφ q + o. The secod equality i 33 is a cosequece of Lemmas 2 ad 7. Accordig to Lemma 8 ˆl φ = J + o Z R + o φ φ,0 ] for φ = O. Hece, ˆl φq ˆl ˆφ q = J + o R + o ] φ q ˆφ q = o by Lemma 7. Sice Gqφ, l is cotiuous i its argumets we ca ow exress the secod term o the right-had side of 32 as Gq φ q, ˆl φ q = Gq ˆφ q, ˆl ˆφ q + o 34 Pluggig 33 ad 34 ito 32 we obtai the uer boud G q ˆβ G q β q o. Usig similar argumets, we ca establish a lower boud as follows: G q ˆβ G q β q = G ˆφ, ˆl ˆφ Gq φ q, l q φ q + o G ˆφ, ˆl q ˆφ Gq ˆφ, l q ˆφ + o = G ˆφ, ˆl q ˆφ Gq ˆφ, ˆl q ˆφ + o = o which roves iv. v Follows from arts iii ad iv Techical Lemmas Lemma 6 Suose Assumtios to 3 are satisfied. The, b q exists uiquely w..a.. Proof of Lemma 6: The subsequet statemets are true w..a.. Notice that Ḡ qφ, defied i 25, is strictly covex fuctio of φ because R = Q, M ] is a full rak matrix uder Assumtio 2c ad J is ositive defiite. Hece, R J R is a ositive defiite matrix. Moreover, the domai Φ is covex. Therefore, φ q is uique. Fially, from 24 we deduce that l q exists uiquely. Lemma 7 Suose Assumtios to 3 are satisfied. The

17 6 i b q = O, ii ˆb q = b q + o. Proof of Lemma 7: Proof of i: We will show that φ q = O. For otatioal simlicity, deote A = R J R, A 2 = A R J ad write the cocetrated quadratic objective fuctio 25 as Z, ad A 3 = Z J Z A 2A A 2, Ḡ qφ = 2 φ φ,0 + A 2 A φ φ,0 + A A 3. Observe that J, R, ad Z coverge weakly accordig to Assumtios 2 ad 3. Moreover based o Assumtio, A is ositive defiite w..a.. Let φ q = argmi φ R m+h 2 Ḡ qφ = φ,0 A 2 = O. Notice that φ q is the rojectio of φ q oto the set Φu 0 with resect to the ier roduct x, y = x A y. The, φ q λ mi A φ q, φ q /2 λ mi A φ q, φ q /2 = O where λ mi A deotes the smallest eigevalue of A ad is strictly ositive w..a.. Fially, from 24 we ca deduce that l q φ q = O. Proof of ii: Accordig to Lemma 6 the saddleoit roblem mi φ Φu0 max l R h G qφ, l has a uique solutio b q o the domai B = Φu 0 R h. Sice B B for ay ɛ > 0 P ˆb q b q > ɛ P bq B\B P b q B\Φ u 0 Λ ζ + o, where the o term i the last lie holds by Lemma ii. The set Λ ζ cosists of the elemets i Λ ζ multilied by ad exads to R h because ζ < /2. Sice θ 0 is i the iterior of Θ, the first m ordiates of Φ u 0 exad to R m. Ordiate m + j exads to R if u 0,j = ad to R + otherwise. Sice b q = O, we deduce P b q B\Φ u 0 Λ ζ = o. Therefore ˆb q = b q + o, as required. Lemma 8 Suose that Assumtios to 3 are satisfied. Let θ Θ ad ν 0 be sequeces such that θ θ,0 0 ad ν ν,0 0. Let ˆl φ = ˆλ θ, ν, ad φ = s, ū ], where s = θ θ,0 ad ū = u,0 + ν ν,0. The 0 = Z R + o φ φ,0 J + o ˆl φ.

18 7 Proof of Lemma 8: I view of Lemmas ii ad 2, we deduce that ˆλ θ, v is i the iterior of ˆΛ θ w..a.. max λ ˆΛ θ G θ, ν, λ : Hece, ˆλ satisfies the first-order coditios associated with 0 = gx i, θ + ˆλ M ν. gx i, θ We ow aly the mea-value theorem ad multily by : 0 = G i= β,0 λ + G 2 β λθ s M ū u,0 + G 2 β λλ ˆl, where β lies o the lie joiig β,0 ad β = θ, ν, ˆλ θ, ν ]. The matrices G β ad G 2 β ad their artitios are defied i 9 ad 2. Usig the same argumets as i the roof of Lemma 5 ad the defiitios of J, Q, R, ad Z we obtai the desired result.