SPLINE ESTIMATION OF SINGLE-INDEX MODELS

Transcription

1 SPLINE ESIMAION OF SINGLE-INDEX MODELS Li Wang and Lijian Yang University of Georgia and Mihigan State University Supplementary Material his note ontains proofs for the main results he following two propositions play an important role in the proof Proposition A1 establishes the uniform onvergene rate of the derivatives of ˆγ up to order 2 to those of γ in Proposition A2 shows that the derivatives of the risk funtion up to order 2 are uniformly almost surely approximated by their empirial versions Proposition A1 Under Assumptions A2-A6, with probability 1 ˆγ u) γ u) = O nh) 1/2 log n + h 4, 1 p,q d u [0,1] A1) ) max 1 i n ˆγ U,i ) γ U,i ) log n p = O + nh 3 h3, A2) max 2 ) ˆγ U,i ) γ U,i ) log n p q = O + nh 5 h2 A3) 1 i n Proposition A2 Under Assumptions A2-A6, one has for k = 0, 1, 2 k ˆR k d ) R d ) = o1), as d d 1 2 Proofs of heorem 2, Propositions A1 and A2 are given in the following Wherever proofs are inomplete, it is referred to Wang and Yang 2007b) A1 Preliminaries In this setion, we introdue some properties of the B-spline Lemma A1 here exist onstants > 0 suh that for N j= k+1 α j,kb j,k up to order k = 4 h 1/r α r 4 N k=2 j= k+1 α r j,kb j,k 3 r 1 h ) 1/r α r, 1 r h 1/r α r 4 N j= k+1 α r j,kb j,k 3h) 1/r α r, 0 < r < 1, k=2

2 2 LI WANG AND LIJIAN YANG where α := α 1,2, α 0,2,, α N,2,, α N,4 ) In partiular, under Assumption A2, for any fixed h 1/2 α 2 k=2 j= k+1 α j,k B j,k Ch 1/2 α 2 2, Proof It follows from the B-spline property on page 96 of de Boor 2001), 4 N k=2 j= k+1 B j,k 3 on [0, 1] So the right inequality follows immediate for r = When 1 r <, Hölder s inequality implies that k=2 j= k+1 α j,k B j,k 31 1/r k=2 j= k+1 α j,k r B j,k Sine all the knots are equally spaed, B j,k u) du h, the right inequality follows from 1 r α j,k B j,k u) du 3 r 1 h α r r 0 When r < 1, we have k=2 j= k+1 k=2 j= k+1 α j,k B j,k Sine Br j,k u) du t j+k t j = kh and 1 0 k=2 j= k+1 r r α j,k B j,k u) du α r r k=2 j= k+1 α j,k r B r j,k 1/r B r j,k u) du 3h α r r, the right inequality follows in this ase as well For the left inequalities, we derive from heorem 542, DeVore and Lorentz 1993), for any 0 < r α j,k r tj+1 C1h r 1 t j j= k+1 r α j,k B j,k u) du

3 SINGLE-INDEX MODEL 3 Sine eah u [0, 1] appears in at most k intervals t j, t j+k ), adding up these inequalities, we obtain that α r tj+k r C 1h 1 r α j,k B j,k u) du 3Ch 1 r α j,k B j,k t j k=1 j= k+1 j= k+1 he left inequality follows For any funtions φ and ϕ, define the empirial inner produt and the empirial norm as φ, ϕ = 1 0 φ u) ϕ u) f u) du, φ 2 2,n, = n 1 φ 2 U,i ) In addition, if funtions φ, ϕ are L 2 [0, 1]-integrable, define the theoretial inner produt and its orresponding theoretial L 2 norm as 1 φ 2 2, = φ 2 u) f u) du, φ, ϕ n, = n 1 0 φ U,i ) ϕ U,i ) Denote by Γ = Γ 0) Γ 1) Γ 2) the spae of all linear, quadrati and ubi spline funtions on [0, 1] We establish the uniform rate at whih the empirial inner produt approximates the theoretial inner produt for all B-splines B j,k with k = 2, 3, 4 Lemma A2 Under Assumptions A2, A5 and A6, with probability 1 γ 1, γ 2 n, γ 1, γ 2 A n = γ 1,γ 2 Γ γ 1 2, γ 2 2, = O nh) 1/2 log n A4) Proof Denote γ 1 = 4 N k=2 j= k+1 α jkb j,k, γ 2 = 4 N k=2 j= k+1 β jkb j,k without loss of generality hen for fixed r γ 1, γ 2 n, = γ 1 2 2, = γ 2 2 2, = α j,k β j,k B j,k, B j,k n,, k=2 j= k+1 k =2 j = k+1 k=2 j= k+1 k =2 j = k+1 k=2 j= k+1 k =2 j = k+1 α j,k α j,k Bj,k, B j,k, β j,k β j,k Bj,k, B j,k

4 4 LI WANG AND LIJIAN YANG Let α = α 1,2, α 0,2,, α N,2,, α N,4 ) and β = β 1,2, β 0,2,, β N,2,, β N,4 ) Aording to Lemma A1, one has for any S d 1, Hene 1 h α 2 2 γ 1 2 2, 2h α 2 2, 1h β 2 2 γ 2 2 2, 2h β 2 2, A n = A n 0 h 1 1 h α 2 β 2 γ 1 2, γ 2 2, 2 h α 2 β 2 γ 1, γ 2 n, γ 1, γ 2 γ 1 2, γ 2 2, α β 1 h α 2 β 2 1 Bj,k max, B n j,k n, B j,k, B j,k, γ 1 γ,γ 2 Γ k,k =2,3,4 1 j,j N max k,k =2,3,4 1 j,j N 1 n Bj,k, B j,k n, B j,k, B j,k, whih, together with Lemma A2 in Wang and Yang 2007b), imply A4) A2 Proof of Proposition A1 For any fixed, we write the response Y = Y 1,, Y n ) as the sum of a signal vetor γ, a parametri noise vetor E and a systemati noise vetor E, ie, Y = γ + E + E, in whih the vetors γ = γ U,1 ),, γ U,n ), E = m X 1 ) γ U,1 ),, m X n ) γ U,n ), E = σ X 1 ) ε 1,, σ X n ) ε n Remark A1 If m is a genuine single-index funtion, then E 0 0, thus the proposed model given by 11) and 12) is exatly the single-index model We break the ubi spline estimation error ˆγ u ) γ u ) into a bias term γ u ) γ u ) and two noise terms ε u ) and ˆε u ) ˆγ u ) γ u ) = γ u ) γ u ) + ε u ) + ˆε u ), A5) where N γ u) = B j,4 u) 3 j N V 1 n, γ, B j,4, A6) n, j= 3 N ε u) = B j,4 u) 3 j N V 1 n, E, B j,4, A7) n, j= 3 N ˆε u) = B j,4 u) 3 j N V 1 n, E, B j,4 n, j= 3 A8)

5 SINGLE-INDEX MODEL 5 In the above, we denote by V n, the empirial inner produt matrix of the ubi B-spline basis and similarly, the theoretial inner produt matrix as V V n, = 1 N n B B = B j,4, B j,4, V n, = N B j,4, B j,4 j,j = 3 j,j = 3 he next lemma provides the uniform upper bound of V 1 n, A9) and V 1 Lemma A3 Under Assumptions A2, A5 and A6, there exist onstants 0 < V < C V suh that V N 1 w 2 2 w V w C V N 1 w 2 2 and V N 1 w 2 2 w V n, w C V N 1 w 2 2, as, with matries V and V n, in A9) Consequently, there exists a onstant C > 0 suh that V 1 n, CN, as, V 1 S d 1 CN A10) S d 1 In the following, we denote by Q m) the 4-th order quasi-interpolant of m orresponding to the knots, see equation 412), page 146 of DeVore and Lorentz 1993) Lemma A4 Under Assumptions A2, A3, A5 and A6, there exists an absolute onstant C > 0, suh that for funtion γ u) in A6) d k du k γ γ ) C m 4) h 4 k, as, 0 k 2, A11) Proof Aording to heorem A1 of Huang 2003), there exists a onstant C > 0, suh that γ γ C inf γ γ C m 4) h 4, as, A12) γ Γ 2) whih proves A11) for the ase k = 0 Applying heorem 774 in DeVore and Lorentz 1993), one has for 0 k 2 d k du k Q γ ) γ C γ 4) h 4 k C m 4) h 4 k, As a onsequene of A12) and A13) for the ase k = 0, one has Q γ ) γ C m 4) h 4, as, A13)

6 6 LI WANG AND LIJIAN YANG whih, aording to the differentiation of B-spline given in de Boor 2001), entails that d k du k Q γ ) γ C m 4) h 4 k, as, 0 k 2 A14) Combining A13) and A14) proves A11) for k = 1, 2 Lemma A5 Under Assumptions A2, A4 and A5, there exists a onstant C > 0 suh that with probability 1 γ U,i ) γ U,i ) n p C m 4) h 3, A15) 1 p,q d 2 γ U,i ) γ U,i ) n p q C m 4) h 2 A16) Proof Aording to the definition of γ in A6), and the fat that Q γ ) is a ubi spline on the knots, one has Q γ ) γ U,i ) n = P Q γ ) γ U,i ) n p p = Ṗ p Q γ ) γ U,i ) n + P Q γ ) γ U,i ) n p Applying A14) to the following deomposition Q γ ) γ U,i ) n = Q γ p p d n + du Q γ ) γ U,i ) X ip yields A15) he proof of A16) is similar ) n γ U,i ) p Lemma A6 Under Assumptions A2, A5 and A6, there exists a onstant C > 0 suh that with probability 1 n 1 B Ch, n 1 Ḃ p C, A17) S d 1 S d 1 P C, Ṗp Ch 1 A18)

7 SINGLE-INDEX MODEL 7 Proof o prove A17), note that for any vetor a R n, with probability 1 n 1 B a a max 3 j N n 1 B j,4 U,i ) Ch a, n 1 Ḃ p a a max C a 3 j N 1 nh o prove A18), one only needs to use A10), A17) B j,3 B j+1,3 ) U,i ) F d X,i ) X i,p Lemma A7 Under Assumptions A2 and A4-A6, with probability 1 B E ) log n S n = O, B ) ) E log n d 1 nn = O, S d 1 p n nh A19) B E ) log n S n = O, B ) ) E log n d 1 nn = O S d 1 p n nh A20) Proof We deompose the noise variable ε i into a trunated part and a tail part ε i = ε Dn i,1 + εdn i,2 + mdn i, where D n = n η 1/3 < η < 2/5), ε Dn i,1 = ε ii ε i > D n, i,2 = ε ii ε i D n m Dn i, m Dn i = E [ε i I ε i D n X i ] ε Dn Note that the B-spline basis and the onditional variane funtion σ 2 are bounded, so it is straightforward to verify that 1 B j,4 U,i ) σ X i ) m Dn i n = O Dn 2 ) = o n 2/3) he tail part vanishes almost surely P ε n > D n n=1 n=1 D 3 n < So Borel-Cantelli Lemma implies that 1 n n B j,4 U,i ) σ X i ) ε Dn i,1 = O n k) for any k > 0 For the trunated part, using Bernstein s inequality and disretization method B j,4 U,i ) σ X i ) ε Dn i,2 = O log n/ ) nn, as 1 j N n 1

8 8 LI WANG AND LIJIAN YANG herefore with probability 1 1 n B E = o n 2/3) + O n k) + O log n/ ) ) log n nn = O nn he proofs of the seond part of A19) and the first part of A20) are similar sine the onditional expetation of m X i ) γ U,i ) given U,i is 0, but no trunation is needed for the first part of A20) as max m X i) γ U,i ) C < 1 i n Meanwhile, to prove the seond part of A20), note that for any p = 1,, d ) N B E = [B j,4 U,i ) m X i ) γ U,i )] p p j= 3 By 26), γ U ) E m X) U, hene E [B j,4 U ) m X) γ U )] 0, for any S d 1, 3 j N Applying Assumptions A2 and A3, one an differentiate through the expetation, thus E [B j,4 U ) m X) γ U )] 0, p for any S d 1, 1 p d, 3 j N Applying the Bernstein s inequality, with probability 1 N 1 [B j,4 U,i ) m X i ) γ U,i )] n p thus the desired result follows j= 3 = O nh) 1/2 log n, Lemma A8 Under Assumptions A2 and A4-A6, for ˆε u) and ε u) in A8) and A7) ˆε u) = O nh) 1/2 log n, as, A21) u [0,1] u [0,1] ε u) = O nh) 1/2 log n, as A22)

9 SINGLE-INDEX MODEL 9 Proof Denote â â 3,, â N ) = B B ) 1 B E = V 1 n, n 1 B E) By heorem 542 in DeVore and Lorentz 1993) u [0,1] ˆε u) â CN n 1 B E, as, where the last inequality follows from A10) of Lemma A3 Applying A19) of Lemma A7, we have established A21) Similarly, A22) an be proved he next result evaluates the uniform size of the noise derivatives Lemma A9 Under Assumptions A2-A6, one has with probability 1 max S d 1 1 i n ˆε U,i ) p = O max S d 1 1 i n ε U,i ) p = O max 2 1 p,q d S d 1 1 i n ˆε U,i ) p q = O max 2 1 i n ε U,i ) p q = O 1 p,q d n Proof Note that p ˆε U,i ) is equal to nh 3 ) 1/2 log n, A23) nh 3 ) 1/2 log n, A24) nh 5 ) 1/2 log n, A25) nh 5 ) 1/2 log n A26) I P ) Ḃ p B B ) 1 B E + B B B ) 1 Ḃ p I P ) E Applying Lemmas A3, A6 and A7, one derives A23) o prove A24), note that n ε U,i ) = P E = Ṗ p E + P E = 1 + 2, p p p in whih 1 = I P ) Ḃ p B B B ) 1 Ḃ p B B B ) 1 B E, B ) 1 2 = B B n p B E n )

10 10 LI WANG AND LIJIAN YANG By A10), A17), A18) and A19), one derives ) 1 = O n 1/2 N 3/2 log n, as, while A20) of Lemma A7, A10) of Lemma A3 ) ) 2 = N O n 1/2 h 1/2 log n = O n 1/2 h 3/2 log n, as hus A24) has been established he proof for A25) and A26) are similar Proof of Proposition A1 Aording to the deomposition A5), one has p ˆγ γ ) U,i ) = p γ γ ) U,i ) + p γ U,i ) + It is lear from A15), A23) and A24) that with probability 1 max γ γ ) U,i ) p = O h 3), 1 i n max ε U,i ) p + ˆε U,i ) p 1 i n p ˆε U,i ) nh 3 = O ) 1/2 log n Putting together all the above yields A2) he proofs of A3) are similar A3 Proof of Proposition A2 Lemma A10 Under Assumptions A2-A6, Proof Let I 2 = I 3 = 2n 1 I 1 = n 1 ˆγ U,i ) γ U,i ) 2, ˆR ) R ) = o1), as ˆγ U,i ) γ U,i ) γ U,i ) m X i ) σ X i ) ε i, n 1 γ U,i ) m X i ) 2 E γ U ) m X) 2,

11 then I 4 = 2 + n SINGLE-INDEX MODEL 11 1 σ 2 X i ) ε 2 i E [ σ 2 X) ] n γ U,i ) m X i ) σ X i ) ε i, ˆR ) R ) I 1 + I 2 + I 3 + I 4 Bernstein inequality and strong law of large number for α-mixing sequene imply that with probability 1, I 3 + I 4 = o1) Now A1) of Proposition A1 provides that ˆγ u) γ u) = O n 1/2 h 1/2 log n + h 4), as, u [0,1] whih entail that I 1 = O n 1/2 h 1/2 log n ) 2 + h 4 ) 2 almost surely On the other hand I 2 O nh) 1/2 log n + h 4 2n 1 γ U,i ) m X i ) σ X i ) ε i Hene I 2 O n 1/2 h 1/2 log n + h 4) almost surely he lemma follows from Assumption A6 Lemma A11 Under Assumptions A2 - A6, for k = 1, 2, with probability 1 k ˆR ) R ) k = O n 1/2 h 1/2 k log n + h 4 k) A27) Proof Note that for any p = 1, 2,, d 1 ˆR ) = n 1 2 p 1 R ) = E 2 p ˆγ U,i ) Y i p ˆγ U,i ), [ γ U ) m X) σ X) ε p γ U ) ] Denote ξ,i,p = 2 γ U,i ) Y i γ U,i ) R ), p p

12 12 LI WANG AND LIJIAN YANG [ ] then E ξ,i,p ) = 2E γ U,i ) Y i p γ U,i ) p R ) = 0 and with 1 ˆR ) R ) = 2n) 1 ξ,i,p + J 1,,p + J 2,,p + J 3,,p, 2 p J 1,,p = n 1 J 2,,p = n 1 J 3,,p = n 1 Bernstein inequality implies that ˆγ U,i ) γ U,i ) p ˆγ γ ) U,i ), γ U,i ) m X i ) σ X i ) ε i p ˆγ γ ) U,i ), ˆγ U,i ) γ U,i ) p γ U,i ) n 1 ) ξ,i,p = O n 1/2 log n, as Meanwhile, applying A1) and A2) of Proposition A1, one obtains that Note that J 2,,p = n 1 A28) J 1,,p = O n 1 h 2 log 2 n + h 7), as A29) γ U,i ) m X i ) σ X i ) ε i p γ γ ) U,i ) n 1 E + E ) p P E + E ) Applying A1), one gets J 2,,p + n 1 E + E ) P E + E ) p = O h 3), as, while Lemmas A3 and A7 entail that with probability 1 n 1 E + E ) P E + E ) p = O n 1 N log 2 n,

13 SINGLE-INDEX MODEL 13 thus Lastly J 3,,p 1 n J 2,,p = O h 3 + n 1 N log 2 n ), as A30) γ γ ) p γ U,i ) = 1 n E + E ) B B B n ) 1 B n p γ Applying Lemmas A3 and A7 again, one has with probability 1 n S d 1 1 B E+n 1 B E ) B ) 1 B B γ n n p = O n 1 log 2 n + nn) 1/2 log n, while by applying A11) of Lemma A4, one has n 1 γ γ ) γ U,i ) p = O h 4), as, together, the above entail that J 3,,p = O h 4 + n 1 log 2 n + nn) 1/2 log n, as A31) herefore, A28), A29), A30), A31) and Assumption A6 lead to ˆR ) R ) n 1 ξ,i,p = o n 1/2), as, p whih establishes A27) for k = 1 Note that the seond order derivative of ˆR ) and R ) with respet to p, q are [ 2n 1 ˆγ U,i ) Y i [ 2 E γ U ) m X) 2 p q ˆγ U,i ) + ] ˆγ U,i ) ˆγ U,i ), q p 2 γ U ) + E γ U ) ] γ U ) p q q p he proof of A27) for k = 2 follows from A1), A2) and A3)

14 14 LI WANG AND LIJIAN YANG Proof of Proposition A2 A11 he result follows from Lemma A10, Lemma A4 Proof of heorem 2 For any p = 1, 2,, d 1, let Ŝ p d ) be the p-th element of Ŝ d ), and for ) any t [0, 1], let f p t) = Ŝ p tˆ d + 1 t) 0, d, then d dt f d 1 ) ) p t) = Ŝp tˆ d + 1 t) 0, d ˆq 0,q q=1 q Note that Ŝ d ) attains its minimum at ˆ d, ie, Ŝp any p = 1, 2,, d 1, t p [0, 1], one has Ŝ p 0, d ) = f p 1) f p 0) = 2 ˆR t p ˆ d + 1 t p ) 0, d q p q=1,,d 1 then 2 Ŝ 0, d ) = ˆR t p ˆ d + 1 t p ) 0, d q p )p,q=1,,d 1 Now heorem 1 and Proposition A2 with k = 2 imply that ) ˆ d 0 hus, for 2 ) ˆR t p ˆ d + 1 t p ) 0, d l q,p, as, p, q = 1, 2,, d 1 q p ˆ d 0, d ), ˆ d 0, d ) A32) where l p,q is given in heorem 2 Noting that ) n ˆ d 0, d is represented as [ ] 1 2 n Ŝ 0, d), q p ˆR t p ˆ d + 1 t p ) 0, d )p,q=1,,d 1 d 1 where Ŝ 0, d ) = Ŝ p 0, d ) For γ in 26), denote p=1 η i,p := 2 γ p 0,p 1 0,d γ d U 0,i) γ 0 U 0,i) Y i,

15 SINGLE-INDEX MODEL 15 where γ p is value of p γ taking at = 0, for any p, q = 1, 2,, d 1 Aording to Lemma A16 in Wang and Yang 2007b), with probability 1, one has Ŝ p 0, d ) = n 1 η p,i + o n 1/2), E η p,i ) = 0 d 1 Let Ψ 0 ) = ψ pq ) d 1 p,q=1 be the ovariane matrix of n Ŝ p 0, d ) with p=1 ψ pq given in heorem 2 Cramér-Wold devie and entral limit theorem for α mixing sequenes entail that nŝ d 0, d ) N 0, Ψ 0 ) Let [ ] Σ 0 ) = H 0, d ) 1 Ψ 0 ) H 0, d ) 1, with H 0, d ) being the Hessian matrix defined in 23) he above limiting distribution of nŝ 0, d ), A32) and Slutsky s theorem imply the desired result