Journa of Mutvarate Anayss 3 (04) 74 96 Contents sts avaabe at ScenceDrect Journa of Mutvarate Anayss journa homepage: www.esever.com/ocate/jmva Hgh-dmensona sparse MANOVA T. Tony Ca a, Yn Xa b, a Department of Statstcs, The Wharton Schoo, Unversty of Pennsyvana, Phadepha, PA 904, Unted States b Department of Statstcs and Operatons Research, Unversty of North Carona at Chape H, NC 7599, Unted States a r t c e n f o a b s t r a c t Artce hstory: Receved November 03 Avaabe onne 5 Juy 04 AMS subject cassfcatons: 6H5 6G3 Keywords: Extreme vaue dstrbuton Hgh dmensona test Lmtng nu dstrbuton MANOVA Precson matrx Testng equaty of mean vectors Ths paper consders testng the equaty of mutpe hgh-dmensona mean vectors under dependency. We propose a test that s based on a near transformaton of the data by the precson matrx whch ncorporates the dependence structure of the varabes. The mtng nu dstrbuton of the test statstc s derved and s shown to be the extreme vaue dstrbuton of type I. The convergence to the mtng dstrbuton s, however, sow when the number of groups s reatvey arge. An ntermedate correcton factor s ntroduced whch sgnfcanty mproves the accuracy of the test. It s shown that the test s partcuary powerfu aganst sparse aternatves and enjoys certan optmaty. A smuaton study s carred out to examne the numerca performance of the test and compare wth other tests gven n the terature. The numerca resuts show that the proposed test sgnfcanty outperforms those tests aganst sparse aternatves. 04 Esever Inc. A rghts reserved.. Introducton An nterestng testng probem n mutvarate anayss s that of testng the equaty of K popuaton means µ,..., µ K, based on K ndependent random sampes, each from a dstrbuton wth mean µ and a common covarance matrx, where K and K s a fxed constant. Ths testng probem arses n many scentfc appcatons, ncudng genetcs, medca magng and boogy. See, for exampe, [,4,7]. In the Gaussan settng where one observes {X,..., X n } d N(µ, ) for K, the probem can be formuated as testng the hypotheses H 0 : µ = µ = = µ K versus H : µ µ j for some j. A cassca procedure s the kehood rato test wth the test statstc gven by λ = K ( X X) T w ( X X), = where X = n n j= X j, X = n K = n j= X j wth n = n + + n K and w = K n = j= (X j X )(X j X ) T s the wthn-cass sampe covarance matrx up to a constant. The kehood rato test has been we studed. See, for exampe, []. In many contemporary appcatons, hgh dmensona data, whose dmenson s often much arger than the sampe sze, are commony avaabe. In such a settng, the cassca methods whch are desgned for the ow-dmensona case ether () Correspondng author. E-ma addresses: tca@wharton.upenn.edu (T.T. Ca), xayn@ema.unc.edu (Y. Xa). http://dx.do.org/0.06/j.jmva.04.07.00 0047-59X/ 04 Esever Inc. A rghts reserved.
T.T. Ca, Y. Xa / Journa of Mutvarate Anayss 3 (04) 74 96 75 perform poory or are no onger appcabe. For exampe, the kehood rato test s unsatsfactory when the dmenson s hgh reatve to the sampe szes. The two-sampe case,.e. K =, has been reatvey we studed recenty n the hgh-dmensona settng and severa aternatves to the kehood rato test have been proposed. For exampe, Ba and Saranadasa [], Srvastava and Du [0], Srvastava [8], and Chen and Qn [9] proposed tests, whch are based on the sum of squares type statstcs, that perform we under the dense aternatves where the dfference of the two means spreads out. But these tests are known to suffer from ow power under the sparse aternatves where the two mean vectors dffer ony n a sma number of coordnates. Ca, Lu and Xa [7] ntroduced a test, whch s based on the maxmum type statstc, that s shown to be partcuary powerfu aganst sparse aternatves and enjoys certan optmaty. In comparson, the mutpe-sampe case s much ess studed n the hgh-dmensona settng, athough severa proposas for correctng the kehood rato test have aso been ntroduced. Fujkosh, Hmeno and Wakak [] consdered the Dempster trace test, whch s based on the rato of the trace of between-cass sampe covarance matrx b and the trace of the wthn-cass sampe covarance matrx w, where b = K = n ( X X)( X X) T. Instead of the rato, Schott [6] proposed a test statstc based on the dfference of two traces. Srvastava [9] constructed a test statstc by repacng the nverse of the wthn-cass sampe covarance matrx by ts Moore Penrose nverse. A of these test statstcs are based on an estmator of (µ K µ) T A () (µ µ) for some postve defnte matrces A (). We ca these sum of squares type statstcs as they a am to estmate the squared Eucdean norm K (A() ) (µ µ). In genomcs and many other appcatons, the means of the popuatons are typcay ether dentca or are qute smar n the sense that they ony possby dffer n a sma number of coordnates. As n the two-sampe case, the above mentoned sum of squares type tests n the mutpe-sampe case suffer from ow power under sparse aternatves. The goa of the present paper s to deveop a test that s powerfu aganst sparse aternatves for mutpe sampes n the hgh dmensona settng under dependency. To ore the sparsty n the mean dfferences and the dependence between the varabes, the test s based on the near transformaton of the observatons by the precson matrx : {X,..., X n }, for K. The new test statstc s then defned to be the maxmum of the sum of squares of a possbe two sampe t-statstcs of the transformed observatons {X,..., X n } and {X j,..., X jnj } for < j K. The mtng nu dstrbuton of the test statstc s derved and s shown to be the extreme vaue dstrbuton of type I. The convergence of the dstrbuton of the test statstc under the nu to the mtng dstrbuton s, however, sow when the number of groups s reatvey arge. We further ntroduced an ntermedate correcton factor whch sgnfcanty mproves the accuracy of the test. Athough the basc dea underyng the constructon of the test statstc s smar to the one for the two-sampe case n [7], the technques and the ntermedate correcton procedure are new and are much more nvoved than the two-sampe case. Both theoretca and numerca propertes of the test are studed. It s shown that the test s partcuary powerfu aganst sparse aternatves and enjoys certan optmaty. A smuaton study s carred out to examne the numerca performance of the test and compare wth other tests gven n the terature. The numerca resuts show that the proposed test sgnfcanty outperforms those tests aganst sparse aternatves. We aso ustrate the mprovement after usng the correcton factor by comparng ts cumuatve dstrbuton wth the type I extreme vaue dstrbuton as we as the emprca mtng dstrbuton. The mtng dstrbuton after usng the correcton s a much better approxmaton to the emprca dstrbuton, as ustrated n Fg. n Secton 3.. As a drect consequence, numerca resuts show that the sze of the resutng test s cose to the nomna eve. The rest of the paper s organzed as foows. After revewng basc notaton and defntons, Secton ntroduces the new test statstcs. Theoretca propertes of the proposed tests are nvestgated n Secton 3. Lmtng nu dstrbutons of the test statstcs and the power of the tests, both for the case the precson matrx s known and the case s unknown, are anayzed. A smuaton study s carred out n Secton 4 to nvestgate the numerca performance of the tests. Dscussons of the resuts and other reated work are gven n Secton 5. The proofs of man resuts are presented n Secton 6.. Methodoogy We frst construct a testng procedure n the orace settng n Secton. where the covarance matrx s assumed to be known. In addton, another natura testng procedure s ntroduced n ths settng. A data-drven procedure s gven n Secton. for the genera case of unknown covarance matrx. We begn wth basc notaton and defntons. For a vector β = (β,..., β p ) T T R p, defne the q norm by β q = ( p β = q ) /q for q wth the usua modfcaton for q =. A vector β s caed k-sparse f t has at most k nonzero entres. For a matrx A = (a j ) p p, the matrx -norm s the maxmum absoute coumn sum, A L = p max j p = a j, the matrx eementwse nfnty norm s defned to be A = max,j p a j and the eementwse norm s A = p = p j= a j. For a matrx A, we say A s k-sparse f each row/coumn has at most k nonzero entres. We sha denote ( n n n +n (µ µ ), n n 3 n +n 3 (µ µ 3 ),..., nk n K n K +n K (µ K µ K )) T =: δ = (δ () can be equvaenty wrtten as H 0 : δ = 0 for =,..., p. Let δ (j) := (δ (j),..., δ(j) p ) T =,..., δ (K K) ) T so the nu hypothess nj n n j +n (µ j µ ), then the aternatve s caed k-sparse f δ (j) s k-sparse for a j < K. For two sequences of rea numbers {a n } and {b n },
76 T.T. Ca, Y. Xa / Journa of Mutvarate Anayss 3 (04) 74 96 wrte a n = O(b n ) f there exsts a constant C such that a n C b n hods for a suffcenty arge n, wrte a n = o(b n ) f m n a n /b n = 0, and wrte a n b n f there are postve constants c and C such that c a n /b n C for a n... Orace procedure Suppose we observe ndependent p-dmensona random sampes X,..., X n N(µ, ), X,..., X n N(µ, ),..., X K,... X KnK N(µ K, ), where the covarance matrx := (σ j ) s known. In ths case, the nu hypothess H 0 : δ = 0, for =,..., p, s equvaent to H 0 : η = 0, for =,..., p, where η = ((Aδ () ),..., (Aδ (K K) ) ) T for any p p postve defnte matrx A := (a j ). An unbased estmator of η s the sampe mean vector ( n n nk n n +n (A( X X )),..., K n K +n K (A( X K X K )) ) T, where ( Xj,..., Xjp ) =: X j = n j nj t= X jt, j K. For testng the nu hypothess H 0 : δ = 0, for =,..., p, a natura cass of test statstcs s M A = max p j< K n j n n j + n (A( X j X )) b, where (b j ) =: B = AA. In the present paper, we are partcuary nterested n the choce of A = =: := (ω j ), M = max p j< K n j n n j + n (( X j X )) ω. (3) In the two-sampe case, [7] showed that the choce of precson matrx works we and the resutng test enjoys certan optmaty aganst sparse aternatves. The motvaton on the near transformaton of the data by the precson matrx n the mutpe-sampe case s smar as n [7]. Under a sparse aternatve, the power of a test many depends on the magntudes of the sgnas (nonzero coordnates of ( δ,..., δ p ) T ) and the number of the sgnas. It w be shown n Secton 6 that η s approxmatey equa to ω δ for a such that δ 0. The magntudes of the nonzero sgnas δ are then transformed to ω δ after normazed by the standard devaton of the transformed varabe (X). In comparson, the magntudes of the sgnas n the orgna data are δ /σ. It can be seen from the nequaty ω σ for =,..., p that ω δ δ /σ. That s, such a near transformaton magnfes the sgnas and the number of the sgnas due to the dependence n the data. The transformaton thus heps to dstngush the nu and aternatve hypothess. The advantage of ths near transformaton w be dscussed n Secton 5. Smar transformatons are aso studed n, for exampe, the detecton probem through the nnovated hgher crtcsm n [3]. A smar nnovated threshodng method s aso consdered n [0] for an optma cassfcaton procedure. A natura choce of A s A = I. That s, the test s drecty based on the sampe means X j X for j < K. Defne the test statstc M I = max p j< K n j n n j + n ( X j X ) σ, where σ are the dagona eements of. It w be shown n Secton 5 that the test based on M I s unformy outperformed by the test based on M for testng aganst sparse aternatves... Data-drven procedure We have so far focused on the orace case n whch the covarance matrx s known. For testng the hypothess H 0 : µ = µ = = µ K n the case of unknown covarance matrx, motvated by the orace procedure M A gven n Secton., the genera test statstc s M A, where  s an estmator for A, defned by M A = max p j< K n j n n j + n ( A( X j X )) ˆb, where (ˆbj ) =: B = K = n K { K = n t= ( A(X t X ))( A(X t X )) T }. For the specfc choce of A =, we use the constraned mnmzaton method gven n [6] to estmate. Other good estmators of the precson matrx can aso be used. See more dscussons n Remark n Secton 3.3.. Then our fna test statstc s M = max p j< K n j n n j + n ( ( X j X )) ˆb, (6) () (4) (5)
T.T. Ca, Y. Xa / Journa of Mutvarate Anayss 3 (04) 74 96 77 wth (ˆbj ) =: B = { K n ( K = n = t= (X t X ))( (X t X )) T }. The smuaton resuts n Secton 4 show that the K numerca performance of the test based on M s smar to that of the test based on M. 3. Theoretca anayss We now turn to the anayss of the propertes of M and M ncudng the mtng nu dstrbuton and the power of the correspondng tests. An ntermedate correcton for the mtng dstrbuton s ntroduced. We w show that the test based on M enjoys certan optmaty when testng aganst sparse aternatves. Moreover, under sutabe condtons the test based on M performs as we as that based on M and thus shares the same optmaty. The asymptotc nu dstrbuton of M I s aso derved. 3.. Asymptotc dstrbutons of the orace test statstcs We frst estabsh the asymptotc nu dstrbutons for the orace test statstcs M and M I. Let D = dag(σ,..., σ pp ) and D = dag(ω,..., ω pp ), where σ kk and ω kk are the dagona entres of and respectvey. The correaton matrx of X s then Γ = (γ j ) = D / D / and the correaton matrx of X s R = (r j ) = D / D /. To obtan the mtng nu dstrbutons, we assume that the egenvaues of the covarance matrx are bounded from above and beow, and the correatons n Γ and R are bounded away from and. More specfcay we assume the foowng: (C) : C 0 λ mn () λ max () C 0 for some constant C 0 > 0; (C) : max <j p γ j r < for some constant 0 < r < ; (C3) : max <j p r j r < for some constant 0 < r <. Condton (C) on the egenvaues s a common assumpton n the hgh-dmensona settng. Condtons (C) and (C3) are aso md. For exampe, f max <j p r j =, then s snguar. Let Y = σ ( n n n +n ( X X ), n n 3 nk n n +n 3 ( X X 3 ),..., K n K +n K ( X K X K ) ) T, Let 0 be the b b covarance matrx of Y := (Y,..., Y b ) for =,..., p, where b = K(K ). Let σ be the argest egenvaue of 0 and d be the dmenson of the correspondng egenspace. Let σ, < d, be the postve egenvaues of 0 arranged n a nonncreasng order and = 0, d. Let H() := =d+ ( σ /σ ) /. Then the takng nto account the mutpctes. Further, f d <, put σ foowng theorem states the asymptotc nu dstrbutons for the orace statstcs M and M I. Theorem. Let the test statstcs M and M I be defned as n (3) and (4), respectvey. () Suppose (C) and (C3) hod. Then for any x R, as p, P H0 M σ og p (d )σ og og p d x Γ H() x σ where Γ ( ) s the gamma functon. () Suppose (C) and (C) hod. Then for any x R, as p, P M I σ og p (d )σ og og p x Γ d H() x. σ When the sampe szes are equa, that s, n = n = = n K, t s easy to check that σ = K, d = K and H() =. Thus, we have the foowng smpe resson for the asymptotc mtng dstrbuton. Coroary. Let the test statstcs M and M I be defned as n () and (4), respectvey. () Suppose (C) and (C3) hod and n = n = = n K. Then for any x R, as p, K(K 3) K P H0 M K og p og og p x Γ x. K () Suppose (C) and (C) hod and n = n = = n K. Then for any x R, as p, K(K 3) K P M I K og p og og p x Γ x. K Theorem hods for any fxed sampe szes n j for j K and t shows that M and M I have the same asymptotc nu dstrbuton. Based on the mtng nu dstrbuton, we propose the asymptotcay α-eve test Φ α () = I{M σ og p + (d )σ og og p + q α } (7)
78 T.T. Ca, Y. Xa / Journa of Mutvarate Anayss 3 (04) 74 96 Fg.. Comparson of the emprca cumuatve dstrbuton and the mtng cumuatve dstrbutons wth p = 00, n = = n 5 = 00 and K = 5. where q α s the α quante of the type I extreme vaue dstrbuton wth cumuatve dstrbuton functon H() x σ,.e., q α = σ og Γ d + σ og(h()) σ og og( α). The nu hypothess H 0 s rejected f and ony f Φ α ( ) =. Smary, we defne Φ α (I) = I{M I σ og p + (d )σ og og p + q α }. Γ d Athough the asymptotc nu dstrbuton of the test statstcs M and M I are the same, the power of the tests Φ α () and Φ α (I) are qute dfferent. It s shown n Secton 5 that the power of Φ α () unformy domnates the power of Φ α (I) when testng aganst sparse aternatves. 3.. Intermedate correcton factor for arge K When the number of groups s arger than 3, the test Φ α () gven n (7) based on the asymptotc dstrbuton under the nu hypothess summarzed n Theorem has serous sze dstorton because the convergence rate n dstrbuton of the extreme vaue type statstcs s sow. See, for exampe, [,5,4]. Fg. ustrates the sze dstorton of the mtng dstrbuton n Theorem by comparng ts cumuatve dstrbuton wth the emprca dstrbuton when the data are generated from N(0, I), wth p = 00, n = = n K = 00 and K = 5. It can be seen from Fg. that there s a notceabe dfference between the two cumuatve dstrbutons, and drecty appyng the mtng dstrbuton n Theorem woud ead to a test whose true sze s sgnfcanty dfferent from the nomna eve. Ths dstorton many comes from the accumuaton of the norma approxmaton error when K s reatvey arge. Thus, nstead of drecty cacuatng the approxmated norma tas, we derve the foowng ntermedate correcton for the asymptotc mtng nu dstrbuton. Proposton. Defne the test statstcs M and M I as n (3) and (4), respectvey. () Suppose (C) and (C3) hod. Then for any x R, P H0 M x p / p P( Y x p) as p, where x p = σ og p + (d )σ og og p + x and Y s a Gaussan random varabe wth mean zero and covarance matrx 0, where 0 s the b b covarance matrx as defned n Secton 3.. () Suppose (C) and (C) hod. Then for any x R P H0 MI x p / p P( Y x p) as p, where x p = σ og p + (d )σ og og p + x and Y s a Gaussan random varabe wth mean zero and covarance matrx 0. In ght of the resuts gven n Proposton, for any p p postve defnte matrx A, based on the test statstc M A gven n (), a corrected α-eve test can be defned by Ψ α (A) = I{M A t α,p }, where t α,p satsfes P( Y t α,p ) = /p og( α)
T.T. Ca, Y. Xa / Journa of Mutvarate Anayss 3 (04) 74 96 79 Fg.. Comparson of three cumuatve dstrbutons wth p = 00, n = = n 5 = 00 and K = 5. and Y s a Gaussan random varabe wth mean zero and covarance matrx 0. In partcuar, we propose the corrected α-eve test Ψ α () = I{M t α,p }. Smary, we defne Ψ α (I) = I{M I t α,p }. As an ustraton of the accuracy of the corrected dstrbuton n Proposton, we compare ts cumuatve dstrbuton wth the emprca dstrbuton under the same settng as n Fg., as we as the mtng dstrbuton derved n Theorem. We can see from Fg. that the corrected asymptotc dstrbuton s much coser to the emprca dstrbuton and as a resut w provde a much more precse cutoff vaue for a gven nomna eve. Smuaton resuts n Secton 4 show that the actua sze of Ψ α () s cose to the pre-specfed nomna eve. We recommend to use the test Φ α () gven n (7) for K 3 and use the test Ψ α () gven n (8) for K 4. 3.3. The asymptotc propertes of Φ α () and Φ α ( ) In ths secton, we anayze the asymptotc power of the test Φ α () and show that t s mnmax rate optma aganst sparse aternatves. For a gven postve defnte matrx A, the corrected test Ψ α (A) shares the same asymptotc propertes as Φ α (A) snce t s derved from the ntermedate correcton term of the mtng dstrbuton n Theorem nstead of drecty cacuatng the ta probabty. Thus n ths secton we focus the dscusson on the asymptotc propertes of Φ α (A). In practce, s unknown and the test statstc M shoud be used nstead of M. Defne the set of k p -sparse vectors by S(k p ) = where δ (j) = δ (j), j < K : max j< K p = I{δ (j) 0} k p nj n n j +n (µ j µ ). Throughout the secton, we anayze the power of M and M under the aternatve H : {δ (j), j < K} S(k p ) wth k p = p r and the nonzero ocatons of δ (j), for every j < K, are randomy unformy drawn from {,..., p}. As dscussed n [7], the condton on the nonzero coordnates n H s md. The same condton has been mposed n [3]. We show that, under some sutabe assumptons, Φ α ( ) performs as we as Φα () asymptotcay., (8) 3.3.. The asymptotc power of Φ α () and ts optmaty The asymptotc power of Φ α () s anayzed under certan condtons on the separaton among µ j and µ for j < K. Furthermore, a ower bound s derved to show that ths condton s mnmax rate optma n order to dstngush H and H 0 wth probabty tendng to. Theorem. Suppose that (C) hods. If r < /4 and max δ /σ constant ε > 0, then we have, as p, P H (Φ α () = ). σ β og p wth β /(mn σ ω ) + ε for some
80 T.T. Ca, Y. Xa / Journa of Mutvarate Anayss 3 (04) 74 96 We sha show that the condton max δ /σ σ β og p s mnmax rate optma for testng aganst sparse aternatves, whch s a drect resut of Theorem 3 n [7]. Frst we ntroduce some condtons as n [7]. (C4) k p = p r for some r < / and = s s p -sparse wth s p = O((p/k p )γ ) for some 0 < γ <. (C4 ) k p = p r for some r < /4. (C5) L M for some constant M > 0. Defne the cass of α-eve tests by T α = {Φ α : P H0 (Φ α = ) α}. Let A δ,c = S(k p ) {max p δ c og p} be a set of k p -sparse vectors {δ (j), j < K} wth the norm of ( δ,..., δ p ) havng the magntude greater than or equa to c og p for some constant c > 0. The foowng theorem shows that the condton max δ /σ σ β og p s mnmax rate optma. Theorem 3. Assume that (C4) (or (C4 )) and (C5) hod. Let α, ν > 0 and α + ν <. Then there exsts a constant c > 0 such that for a suffcenty arge n and p, =,..., K, nf sup P(Φ α = ) ν. {δ (j), j< K} A δ,c Φ α T α Remark. The ower bound resut foows drecty from Theorem 3 n [7]. We construct µ and µ exacty the same as the worst case n the proof of ower bound resut n [7] and et µ j = 0 for j = 3,..., K. Then the resut of above theorem foows. 3.3.. The asymptotc propertes of Φ α ( ) and ts optmaty We now anayze the propertes of M and the correspondng test ncudng the mtng nu dstrbuton and the asymptotc power. We sha show that M has the same mtng nu dstrbuton as M and defne the correspondng test Φ α ( ) by Φ α ( ) = I{M σ og p + (d )σ og og p + q α }. Under some sutabe assumptons, the asymptotc propertes of Φ α ( ) are smar to those of Φα (). Defne the foowng cass of matrces that beong to an q ba wth 0 q < : p U q (s p, M p ) = 0 : L M p, max ω j q s p. j p = We assume that U q (s p, M p ) so can be we estmated by the CLIME estmator under some condtons on sp and M p ; see [6]. Theorem 4. Suppose that (C) and (C3) hod and U q (s p, M p ) wth s p = o n ( q)/ M q p (og p) (3 q)/ ()Then under the nu hypothess H 0, for any x R, P H0 M σ og p (d )σ og og p x. d Γ H( 0 ) x, K (9) as n j, p for j =,..., K. Furthermore, for any x R, P H0 M x p / p P( Y x p) as n j, p, where x p = σ og p + (d )σ og og p + x and Y s a Gaussan mean zero r.v. wth covarance matrx 0. ()Under the aternatve hypothess H wth r < /6, we have P H Φα ( ) = P H (Φ α () = ), as n j, p for j =,..., K. Furthermore, f max δ /σ ε > 0, then we have, for j =,..., K, P H Φα ( ) =, as nj, p. σ β og p wth β /(mn σ ω ) + ε for some constant
T.T. Ca, Y. Xa / Journa of Mutvarate Anayss 3 (04) 74 96 8 By Theorem 4, we see that M and M have the same asymptotc dstrbuton and power, and so the test Φ α ( ) s aso mnmax rate optma. Remark. The CLIME estmator n [6] s consdered n ths secton. As n the two-sampe case, other good estmators of the precson matrx can aso be used. In genera, Theorem 4 st hods f og p = o(n) and the estmator satsfes the foowng condtons: L = o P and max ˆb b = o P, (0) og p p og p where (b j ) =: B = and (ˆbj ) =: B =. 3.3.3. Comparson wth Φ α (I) It s nterestng to compare the power of the new test wth the maxmum test based on the orgna observatons. More specfcay, we compare the power of the test Φ α () wth that of Φ α (I) under the same aternatve H as n Secton 3.3.. We show n the foowng Proposton that the power of Φ α () domnates the power of Φ α (I) under sutabe condtons. Proposton. Suppose (C) (C3) hod. Then under H wth r < /6, we have m nf p P H (Φ α () = ) P H (Φ α (I) = ). () Proposton shows that, under some sparsty condtons on {δ (j), j < K}, Φ α () s unformy at east as powerfu as Φ α (I). The test Φ α () can be strcty more powerfu than Φ α (I). Assume that p H : max j< K = I{δ (j) 0} = k p = p r, r <, wth nonzero eements σ β 0 og p δ σ β og p. () σ The nonzero ocatons of δ (j), for every j < K, are randomy and unformy drawn from {,..., p}. Proposton 3. Suppose that (C) (C3) hod and mn p σ ω + ε for some ε > 0. Then, under H wth ( r) + ϵ β 0 < β < ( r) mn σ ω p for some ϵ > 0, we have and m P H p (Φ α() = ) = m sup P H (Φ α(i) = ) α. p When the varabes are correated, ω can be strcty arger than /σ. For exampe, et = (φ j ) wth φ <. Then mn p σ ω ( φ ) >. That s, Φ α () s strcty more powerfu than Φ α (I) under H. For reasons of space, we omt the proofs of these two propostons. 4. Smuaton study In ths secton, we consder the numerca performance of the tests Φ α () and Φ α ( ) and compare these tests wth a number of other tests, ncudng the orace test Φ α (I), the tests based on the sum of squares type statstcs n [,6,9], and the commony used kehood rato test. These ast four tests are denoted respectvey by FHW, Sc, Sr and LRT respectvey n the tabes beow. In the smuatons, we consder two settngs on the number of the groups: K = 3 and K = 5. We foow the recommendatons made n Secton 3. by usng the test Φ α () gven n (7) for K = 3 and usng the test Ψ α () gven n (8) for K = 5. We sha aways take µ = 0. Under the nu hypothess, µ = = µ K = 0, whe under the aternatve hypothess, we take µ = (µ,..., µ p ) T, for =,..., K, to have m nonzero entres wth the support S = {,..., m : < < < m }
8 T.T. Ca, Y. Xa / Journa of Mutvarate Anayss 3 (04) 74 96 unformy and randomy drawn from {,..., p}. For any j S, µ j has magntude randomy unformy drawn from the nterva [ og p/n, og p/n]. We take µ k = 0 for k S c. For both K = 3 and K = 5, we consder three dfferent vaues of m: the extreme sparse aternatve, moderate sparse aternatve and non-sparse aternatve. In ths smuaton study, the dmenson p takes vaues 50, 00, 00 and 400, and the correspondng vaues of m for each dmenson are as foows. Under the extreme sparse aternatve, et m = for p = 50 and 00 and m = 5 for p = 00 and 400. We seect m = 5 for p = 50, m = 0 for p = 00, m = 5 for p = 00 and m = 0 for p = 400 when the aternatve s moderate sparse. In the scenaro when the aternatve s non-sparse, we choose m = 0 for p = 50, m = 30 for p = 00, m = 40 for p = 00 and m = 50 for p = 400. We consder m = 50 as a non-sparse aternatve when p = 400, because n ths case the number of nonzero entres of the dfference of any par of mean vectors can be as arge as 00, and the vaue k p as defned n Secton 3.3 s equa to 50 when K = 3 and s equa to 50 when K = 5. Three dfferent settngs of the precson matrx are consdered n the smuaton: s known, s unknown but sparse and the case where the covarance matrx s unknown but sparse. In the case when s known, we compare the orace performance of the three tests based on the maxmum-type statstcs wth the tests based on the sum of squares type statstcs. When s unknown, we use the CLIME estmator n [6] to estmate t when s sparse, whe the nverse of the adaptve threshodng estmator n [5] s used to estmate when s sparse. Let D = (d j ) be a dagona matrx wth dagona eements d = Unf(, 3) for =,..., p. Denote by λ mn (A) the mnmum egenvaue of a symmetrc matrx A. In the case when the precson matrx s known, the foowng two modes for are consdered: Mode : = (σ ) j where σ Mode : = (σ ) j where σ δ = λ mn ( ) + 0.05. =, σ j = 0.5 for j. = D / D /. =, σ j = Unf(0, ) for < j and σ j In the case when the precson matrx s sparse, we consder the foowng two modes: = σ j. = D/ ( + δi)/( + δ)d / wth Mode 3: = (σ j ) where σ =, σ j = 0.8 for (k )+ j k, where k =,..., [p/] and σ j = 0 otherwse. Mode 4: = (σ j ) where σ j = 0.6 j for, j p. The foowng two modes are consdered when the covarance matrx s sparse: Mode 5: = (σ ) j where σ =, σ j otherwse. = D / D /. Mode 6: = (ω j ) where ω j = 0.6 j for, j p. = D / D /. = 0.8 for (k ) + j k, where k =,..., [p/] and σ j = 0 Under each mode, two ndependent random sampes {X k } and {Y } are generated wth the same sampe sze n = 00 and n = 60 for K = 3 and K = 5 respectvey from two mutvarate norma dstrbutons wth the means µ and µ respectvey and a common covarance matrx. The sze and power are cacuated from 000 repcatons. The numerca resuts are summarzed n Tabes 4. It can be seen from Tabe that the estmated szes are cose to the nomna eve 0.05 for a the tests. Tabes 4 summarze the power resuts under varous aternatves. Under the extreme sparsty aternatve, Tabe shows that the tests based on the sum of squares test statstcs have trva power, whe the orace test Φ α () has the hghest power n a sx modes over a dmensons rangng from 50 to 400, and the performance of the test Φ α ( ) based on ether the CLIME estmator or the nverse of the adaptve threshodng estmator s cose to that of the orace test Φ α () n Modes 3 6. Under the moderate sparsty aternatve, smar phenomena are observed, the tests Φ α () and Φ α ( ) are sgnfcanty more powerfu n comparson to the other tests. When the number of nonzero entres ncreases, the powers of a tests ncrease as we. Under the non-sparse aternatve, as can be seen from Tabe 4, the sum of squares type tests aso enjoy hgh power n Modes 3 and 4. In other modes, the tests Φ α () and Φ α ( ) st sgnfcanty outperform the other tests though the aternatve s non-sparse. In summary, The tests Ψ α () and Ψ α ( ) perform smary and sgnfcanty outperform the other tests aganst a fu range of aternatves n the smuaton study. Smar phenomena are observed for the corrected tests as shown n Tabes 4. As a graphca ustraton, we aso summarze the power comparson resuts n Fg. 3 for K = 3 and p = 400. The horzonta axs represents each mode and the vertca axs represents the powers of the four tests. We do not ncude LRT, Sr and Φ α ( ) because, when p = 400 LRT s not we defned and Sr has trva power, and Φα ( ) has smar power as Φ α (). It can be easy seen from Fg. 3 that the test Φ α () sgnfcanty outperforms the other tests. 5. Dscusson We ntroduced n ths paper the data-drven testng procedure Φ α ( ˆ) and showed that t performs partcuary we aganst sparse aternatves. Ths procedure requres a good estmate of the precson matrx. We have many focused n ths paper on the sparse precson matrces for whch the CLIME estmator s known to perform we. The test Φ α ( ˆ) can be used wth a much wder range of covarance/precson matrces. As mentoned n Secton 3.3, one ony needs an estmate ˆ satsfyng the condton (0) and then the resut gven n Theorem 4 extends drecty. For exampe, when the covarance matrx s ether sparse or bandabe, Condton (0) can be acheved by nvertng threshodng or taperng estmators of
T.T. Ca, Y. Xa / Journa of Mutvarate Anayss 3 (04) 74 96 83 Tabe Emprca szes of tests wth α = 0.05. n = 00 when K = 3 and n = 60 when K = 5. Based on 000 repcatons. p 50 00 00 400 50 00 00 400 50 00 00 400 50 00 00 400 50 00 00 400 50 00 00 400 K = 3 Mode Mode Mode 3 Mode 4 Mode 5 Mode 6 LRT 0.06 0.05 0.05 0.05 0.05 0.06 0.06 0.06 0.06 0.04 0.06 0.06 0.05 0.06 0.07 0.06 0.06 0.06 FHW 0.09 0.06 0.07 0.06 0.06 0.05 0.05 0.0 0.04 0.04 0.03 0.0 0.06 0.06 0.03 0.0 0.05 0.05 0.03 0.0 0.05 0.05 0.03 0.0 Sc 0.09 0.06 0.08 0.06 0.06 0.06 0.07 0.05 0.05 0.05 0.05 0.05 0.06 0.07 0.06 0.06 0.06 0.06 0.05 0.06 0.06 0.06 0.06 0.06 Sr 0.06 0.05 0.05 0.00 0.05 0.05 0.06 0.00 0.06 0.06 0.06 0.00 0.04 0.06 0.06 0.00 0.05 0.06 0.07 0.00 0.06 0.06 0.06 0.00 Φ α (I) 0.03 0.03 0.03 0.0 0.04 0.05 0.04 0.06 0.04 0.04 0.04 0.03 0.05 0.04 0.05 0.05 0.04 0.06 0.04 0.03 0.04 0.05 0.05 0.05 Φ α () 0.05 0.06 0.04 0.05 0.04 0.03 0.05 0.03 0.04 0.04 0.06 0.05 0.05 0.05 0.05 0.06 0.04 0.05 0.04 0.05 0.05 0.04 0.06 0.05 Φ α ( ) 0.06 0.06 0.07 0.07 0.06 0.07 0.07 0.08 0.05 0.05 0.05 0.06 0.06 0.04 0.06 0.06 K = 5 LRT 0.06 0.06 0.07 0.05 0.06 0.08 0.06 0.06 0.05 0.06 0.06 0.06 0.05 0.05 0.08 0.07 0.06 0.05 FHW 0.09 0.08 0.07 0.06 0.06 0.06 0.05 0.03 0.05 0.06 0.03 0.0 0.06 0.06 0.04 0.0 0.05 0.04 0.03 0.03 0.05 0.04 0.0 0.0 Sc 0.09 0.08 0.07 0.06 0.07 0.06 0.06 0.06 0.06 0.07 0.06 0.05 0.06 0.07 0.06 0.04 0.06 0.05 0.05 0.06 0.06 0.06 0.05 0.05 Sr 0.06 0.06 0.07 0.00 0.05 0.06 0.08 0.00 0.06 0.06 0.05 0.00 0.06 0.06 0.06 0.00 0.05 0.05 0.08 0.00 0.07 0.06 0.05 0.00 Ψ α (I) 0.04 0.03 0.03 0.03 0.05 0.04 0.05 0.05 0.04 0.06 0.05 0.05 0.04 0.05 0.05 0.05 0.05 0.04 0.05 0.04 0.05 0.05 0.05 0.05 Ψ α () 0.04 0.06 0.05 0.04 0.04 0.05 0.06 0.04 0.03 0.05 0.05 0.06 0.05 0.06 0.06 0.04 0.06 0.05 0.05 0.05 0.05 0.05 0.04 0.05 Ψ α ( ) 0.06 0.07 0.07 0.08 0.06 0.06 0.06 0.07 0.05 0.07 0.07 0.07 0.06 0.06 0.06 0.06 Tabe Powers of tests under extreme sparse aternatve wth α = 0.05. Based on 000 repcatons. p 50 00 00 400 50 00 00 400 50 00 00 400 50 00 00 400 50 00 00 400 50 00 00 400 K = 3 wth extreme sparse aternatve Mode Mode Mode 3 Mode 4 Mode 5 Mode 6 LRT 0. 0.06 0.8 0.3 0.4 0.5 0.6 0.34 0.34 0.6 0. 0.33 0.9 0.4 0. 0.04 0.08 0. FHW 0.08 0. 0.05 0.04 0.4 0.06 0.09 0.09 0.8 0.0 0. 0.9 0.08 0. 0.9 0. 0.3 0.05 0.08 0.08 0.00 0.05 0. 0.09 Sc 0. 0. 0.05 0.04 0.4 0.06 0.05 0.04 0.6 0.0 0. 0.6 0.08 0. 0.9 0.07 0.3 0.03 0.06 0.03 0.00 0.05 0.07 0.05 Sr 0. 0.06 0.8 0.00 0.3 0.4 0.5 0.00 0.6 0.34 0.34 0.00 0.6 0. 0.33 0.00 0.9 0.4 0. 0.00 0.04 0.08 0. 0.00 Φ α (I) 0.07 0.05 0. 0.06 0.6 0.07 0.09 0.09 0.4 0.3 0.6 0.7 0.7 0.6 0.5 0.4 0.08 0.07 0. 0.09 0.05 0.09 0. 0.04 Φ α () 0.07 0. 0.48 0.7 0.9 0.5 0.45 0.73 0.48 0.78 0.8.00 0.8 0.43 0.69 0.59 0.9 0.7 0.45 0.49 0. 0. 0.8 0. Φ α ( ) 0.47 0.80 0.84 0.99 0.3 0.44 0.7 0.60 0.9 0.9 0.47 0.48 0.4 0. 0. 0.3 K = 5 wth extreme sparse aternatve LRT 0.07 0.3 0.33 0.54 0.5 0.5 0.79 0.7 0.7 0.44 0.36 0.9 0. 0.35 0.8 0.9 0.3 0.3 FHW 0.09 0.3 0.07 0.07 0.0 0.09 0.08 0.0 0.8 0.4 0.7 0.5 0.6 0.0 0. 0. 0. 0.4 0. 0.3 0.8 0. 0.09 0. Sc 0.09 0.3 0.07 0.08 0.08 0.07 0.07 0.05 0.8 0. 0. 0. 0.6 0.0 0.0 0.3 0. 0.3 0.07 0.08 0.8 0.09 0.07 0.0 Sr 0.07 0.3 0.33 0.00 0.54 0.5 0.5 0.00 0.79 0.7 0.7 0.00 0.44 0.36 0.9 0.00 0. 0.35 0.8 0.00 0.9 0.3 0.3 0.00 Φ α (I) 0.06 0.6 0. 0.8 0.7 0.4 0.9 0.8 0.34 0.4 0.43 0. 0.6 0.6 0.7 0.3 0.03 0. 0.0 0.5 0.3 0.06 0.07 0.05 Φ α () 0. 0.57 0.59 0.74 0.89 0.34.00.00 0.86 0.94 0.98.00 0.56 0.75 0.53 0.9 0.8 0.43 0.65 0.79 0.5 0.09 0.9 0.3 Φ α ( ) 0.85 0.96 0.98.00 0.58 0.75 0.54 0.93 0.30 0.4 0.66 0.8 0.5 0. 0.7 0.7 Tabe 3 Powers of tests under moderate sparse aternatve wth α = 0.05. Based on 000 repcatons. p 50 00 00 400 50 00 00 400 50 00 00 400 50 00 00 400 50 00 00 400 50 00 00 400 K = 3 wth moderate sparse aternatve Mode Mode Mode 3 Mode 4 Mode 5 Mode 6 LRT 0.36 0.37 0.8 0.54 0.79 0.69 0.8.00.00 0.70 0.83 0.68 0.65 0.78 0.76 0.35 0. 0. FHW 0.5 0.08 0. 0.0 0.05 0.0 0.3 0.5 0.40 0.6 0.68 0.48 0.8 0.4 0.4 0.40 0.5 0.36 0.3 0.8 0. 0.09 0.09 0.4 Sc 0.5 0.08 0. 0.0 0.05 0.9 0.0 0. 0.40 0.58 0.59 0.34 0.8 0.38 0.3 0. 0.5 0.3 0. 0.5 0.0 0.08 0.05 0.05 Sr 0.36 0.37 0.8 0.00 0.54 0.79 0.69 0.00 0.8.00.00 0.00 0.70 0.83 0.68 0.00 0.65 0.78 0.76 0.00 0.35 0. 0. 0.00 Ψ α (I) 0.9 0.04 0.7 0.8 0. 0.5 0.7 0.3 0.8 0.5 0.54 0.40 0.34 0.44 0.38 0.58 0.3 0.6 0.7 0.7 0.06 0. 0.07 0. Ψ α () 0.5 0.3 0.53 0.69 0.9.00.00 0.96 0.85.00.00.00 0.76 0.88 0.87 0.96 0.57 0.8 0.95 0.98 0.38 0.3 0. 0.47 Ψ α ( ) 0.84.00.00.00 0.78 0.87 0.9 0.96 0.57 0.8 0.95 0.99 0.37 0.4 0.5 0.49 K = 5 wth moderate sparse aternatve LRT 0.9 0.6 0.76 0.97 0.96 0.93.00.00.00 0.86 0.79 0.5 0.93 0.99 0.96 0.3 0.83 0.9 FHW 0. 0.0 0.09 0.08 0.7 0. 0.37 0.53 0.56 0.96 0.86 0.95 0.3 0.6 0.0 0.85 0. 0.35 0.53 0.5 0.6 0.34 0.9 0.7 Sc 0. 0.0 0.09 0.08 0.6 0.6 0.3 0.3 0.56 0.94 0.8 0.9 0.30 0.6 0.0 0.8 0.9 0.3 0.45 0.33 0.5 0.9 0. 0.07 Sr 0.9 0.6 0.76 0.00 0.97 0.96 0.93 0.00.00.00.00 0.00 0.86 0.79 0.5 0.00 0.93 0.99 0.96 0.00 0.3 0.83 0.9 0.00 Ψ α (I) 0.5 0.0 0.6 0.43 0.8 0. 0.45 0.8 0.54 0.78 0.68 0.74 0.36 0.37 0.40 0.76 0.0 0.37 0.33 0.9 0. 0.4 0.3 0.3 Ψ α () 0.65 0.63 0.8 0.99.00.00.00.00.00.00.00.00 0.96 0.9 0.94.00 0.80.00.00.00 0.3 0.66 0.8 0.36 Ψ α ( ).00.00.00.00 0.96 0.93 0.9.00 0.8.00.00.00 0.36 0.68 0.7 0.37
84 T.T. Ca, Y. Xa / Journa of Mutvarate Anayss 3 (04) 74 96 Tabe 4 Powers of tests under non-sparse aternatve wth α = 0.05. Based on 000 repcatons. p 50 00 00 400 50 00 00 400 50 00 00 400 50 00 00 400 50 00 00 400 50 00 00 400 K = 3 wth non-sparse aternatve Mode Mode Mode 3 Mode 4 Mode 5 Mode 6 LRT 0.99 0.99.00.00.00 0.99.00.00.00.00.00.00.00.00 0.93 0.69 0.7 0.7 FHW 0.7 0. 0.5 0.06 0.4 0.63 0.6 0.69 0.94 0.98.00.00 0.8 0.96.00 0.98 0.4 0.63 0.65 0.84 0.30 0.43 0.38 0.4 Sc 0.6 0. 0.4 0.06 0.38 0.57 0.5 0.54 0.94 0.98 0.98.00 0.8 0.96.00 0.9 0.4 0.55 0.54 0.7 0.30 0.37 0.8 0. Sr 0.99 0.99.00 0.00.00.00 0.99 0.00.00.00.00 0.00.00.00.00 0.00.00.00 0.93 0.00 0.69 0.7 0.7 0.00 Ψ α (I) 0.34 0.35 0.47 0.54 0.38 0.39 0.44 0.5 0.74 0.64 0.86 0.86 0.68 0.66 0.84 0.84 0.3 0.34 0.4 0.48 0. 0.7 0.4 0. Ψ α () 0.88 0.89.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00 0.73 0.73 0.88 0.69 Ψ α ( ).00.00.00.00.00.00.00.00.00.00.00.00 0.77 0.7 0.90 0.75 K = 5 wth non-sparse aternatve LRT.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00 0.95 0.96 0.9 FHW 0.59 0. 0.7 0.0 0.93 0.96.00.00.00.00.00.00.00.00 0.96.00 0.94.00 0.99.00 0.46 0.53 0.66 0.69 Sc 0.59 0. 0.7 0.0 0.9 0.96 0.98 0.99.00.00.00.00.00.00 0.96.00 0.93.00 0.97 0.97 0.4 0.50 0.54 0.47 Sr.00.00.00 0.00.00.00.00 0.00.00.00.00 0.00.00.00.00 0.00.00.00.00 0.00 0.95 0.96 0.9 0.00 Ψ α (I) 0.83 0.45 0.63 0.6 0.59 0.66 0.87 0.75.00 0.98 0.98.00 0.84 0.8 0.86 0.94 0.59 0.78 0.73 0.85 0.3 0.30 0.6 0. Ψ α ().00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00 0.88 0.87 0.97 0.87 Ψ α ( ).00.00.00.00.00.00.00.00.00.00.00.00 0.90 0.88 0.97 0.86 (a) Extreme sparse. (b) Moderate sparse. (c) Non-sparse. Fg. 3. Pots of the comparsons of powers for a modes. the covarance matrx. The smuaton resuts showed that the data-drven test Φ α ( ˆ) performs we when s sparse. See [8] for further detas on estmatng covarance matrces and ther nverse under the matrx norm. In the present paper, t s shown that the test Φ α () outperforms Φ α (I) when testng aganst sparse aternatves. Smar comparson can be made between Φ α () and Φ α ( / ) as n [7]. The power of Φ α () can be proved to domnate the power of Φ α ( / ) as n Proposton, but under stronger condtons. For reasons of space, we omt the dscusson n ths paper. We have focused on the Gaussan case n ths paper. The resuts can be extended to non-gaussan dstrbutons. Let X j, j =,..., K, be p-dmensona random vectors satsfyng X j = µ j + U j, where U,..., U K are ndependent and dentca dstrbuted random vectors wth mean zero and covarance matrx = (σ j ) p p. Let V j = U j =: (V j,..., V pj ) T for j =,..., K. The resuts n Theorem, Proposton and Theorem 4 st hod wth the Gaussan assumpton repaced by ether of the foowng moment condtons. (C6). (Sub-Gaussan-type tas) Suppose that og p = o(n /4 ). There exst some constants η > 0 and C > 0 such that E (ηv j /ω ) C for p, j K. (C7). (Poynoma-type tas) Suppose that for some constants γ 0, c > 0, p c n γ 0, and for some constants ϵ > 0 and C > 0 E V j /ω γ 0++ϵ C for p, j K. 6. Proof of man resuts We prove the man resuts n ths secton. We begn by coectng and provng n Secton 6. a few technca emmas that w be used n the proofs of the man theorems.
T.T. Ca, Y. Xa / Journa of Mutvarate Anayss 3 (04) 74 96 85 6.. Technca emmas Lemma (Bonferron Inequaty). Let A = p t= A t. For any k < [p/], we have k k ( ) t E t P(A) ( ) t E t, t= t= where E t = < < t p P(A A t ). Lemma (Berman [3]). If X and Y have a bvarate norma dstrbuton wth ectaton zero, unt varance and correaton coeffcent ρ, then m c P (X > c, Y > c) =, [π( ρ) / c ] c ( + ρ) +ρ 3/ unformy for a ρ such that ρ δ, for any δ, 0 < δ <. Lemma 3 (Zootarev []). Let Y be a nondegenerate Gaussan mean zero r.v. wth covarance operator. Let σ be the argest egenvaue of and d be the dmenson of the correspondng egenspace. Let σ, < d, be the postve egenvaues of arranged n a nonncreasng order and takng nto account the mutpctes. Further, f d <, put σ = 0, d. Let H() := =d+ ( σ /σ ) /. Then for y > 0, P{ Y > y} Aσ y d ( y /(σ )), as y, where A := (σ ) d/ Γ (d/)h() wth Γ ( ) the gamma functon. Lemma 4. For genera postve defnte matrx A and (b,j ) =: B = AA, suppose C λ mn (A) λ max (A) C and C λ mn (B) λ max (B) C for some constant C > 0 and has a dagona eements equa to. Then for p r -sparse {δ (j), j < K}, wth r < /4 and nonzero ocatons of δ (j) randomy and unformy drawn from {,..., p} for every j < K, we have P η a δ = O(p r a/ ) max δ, (3) b b max and P max η (j) b a δ (j) b = O(pr a/ ) max δ (j), (4) for j < K and for any r < a < r, as p, where δ = (δ (), δ (3),..., δ (K K) ) T and η = ((Aδ () ),..., (Aδ (K K) ) ) T for H := { p : δ (j) 0 for some j < K} = {,..., m }. Proof of Lemma 4. We ony need to prove (3) because the proof of (4) s smar. We re-order a,..., a p as a () a (p). Let a satsfy r < a < r wth r < /4. Defne I = { < < m p} and I 0 = { < < m p : there exst some k m and some j k and j m, such that a k j a k (p a ) }. We can show that p p I 0 / I = O p p a p r So we have I 0 I j< K p r. = O(p a+r ) = o(). For t m, wrte (Aδ (j) ) t = So we have η t = a t t δ t + j< K p k= a t kδ (j) k = j< K a t t δ (j) t + m m a t q δ q a t t δ t a t q δ q, q=,q t q=,q t m q=,q t a t q δ (j) q.
86 T.T. Ca, Y. Xa / Journa of Mutvarate Anayss 3 (04) 74 96 and η t a t t δ t + m q=,q t a t q δ q. Note that for any (,..., m ) I c 0, m a t q p r C p. a q=,q t It foows that for H I 0 and H, η = a δ + O(p r a/ ) max δ. b b So the emma s proved. 6.. Proof of Theorem Wthout oss of generaty, we assume σ = for =,..., p throughout the proof. Let Y = ( n n n +n ( X X ), n n 3 nk n n +n 3 ( X X 3 ),..., K n K +n K ( X K X K ) ) T. Let 0 be the b b covarance matrx of Y := (Y,..., Y b ) T for =,..., p, where b = K(K ). Let M n = max p Y. Then t s enough to prove the foowng emma. Lemma 5. Suppose that max j p σ j r < and C 0 λ mn () λ max () C 0. We have P M n σ og p (d )σ og og p d x Γ H() ( x/σ ). (5) Proof. Set x p = σ og p + (d )σ og og p + x. By Lemma, we have for any fxed m [p/], where m t= E t = ( ) t E t P < < t p Then t suffces to show that < < t p max Y x p p m ( ) t E t, (6) t= P Y x p,..., Y t x p =: P,..., t = ( + o()) t! Γ t d When t =, by Lemma 3, we have p P = ( + o())γ d H t () H() x. σ < < t p P,..., t. tx. (7) σ Ths mpes (7). It remans to prove the emma when t. Let γ > 0 be a suffcenty sma number whch w be specfed ater. Defne I = < < t p : max σ k p γ. k< t For d =, defne I = < < t p : σ k p γ for every k < t. So when t =, we have I = I. For d t and t 3, defne I d = { < < t p : the cardnaty of S s d, where S s the argest subsetof {,..., t } such that k S, σ k < p γ. So we have I = t d= I d for t. Let Card(I d ) denote the tota number of the vectors (,..., t ) n I d. We can show that Card(I d ) Cp d+γ t. In fact, the tota number of the subsets of {,..., t } wth cardnaty d s p d. For a fxed subset
T.T. Ca, Y. Xa / Journa of Mutvarate Anayss 3 (04) 74 96 87 S wth cardnaty d, the number of such that σ k p γ for some j S s no more than Cdp γ. Ths mpes that Card(I d ) Cp d+γ t. Defne I c = { < < t p} \ I. Then the number of eements n the sum (,..., t ) I c P,..., t p s t O( t d= pd+γ t ) = p t O(p t +γ t ) = ( + o()) p t. To prove Lemma 5, t suffces to show that P,..., t = ( + o())γ t d H t ()p t tx σ (8) unformy n (,..., t ) I c, and for d t, (,..., t ) I d P,..., t 0. (9) By submttng (8) and (9) nto (6), we obtan that ( + o())s m P max Y x p ( + o())s m, (0) p where S m = m t= ( )t t! Γ t ( d )Ht () ( tx ). Note that σ m S m = m d Γ H() ( x/σ ). By ettng p frst and then m n (0), we prove Lemma 5. Frst we prove (8). Let Ỹ = (Y T,..., Y T t ) T and (Z T,..., Z T t ) T =: Z N(0, I bt bt ), where b = K(K ), Z j = (Z j,..., Z bj ) T for j =,..., t and Ỹ and Z are ndependent. Let Ỹ t = mn j t Y j and et λ p = Cp γ /4 for some constant C > 0. Then we have P,..., t = P( Ỹ t x p ) P Ỹ + λ p Z t x p λ p max Z j j t (π) bt/ det( + λ p I) / + P λ p max Z j Cp γ /8 j t (π) bt/ det( + λ p I) / z t x p Cp γ /8 z t x p Cp γ /8 zt ( + λ p I) z dz zt ( + λ p I) z dz + O(p t ), () where z R bt and s the covarance matrx of Ỹ and C s a constant. Let be a bt bt matrx wth jb+:(j+)b,jb+:(j+)b = 0 for j = 0,..., t and j = 0 otherwse. For (,..., t ) I c, we have jb+:(j+)b,jb+:(j+)b = 0 for j = 0,..., t and j < p γ otherwse. Wrte z t x p Cp γ /8 zt ( + λ p I) z dz = z t x p Cp γ /8, z (og p) + z t x p Cp γ /8, z (og p) Because λ max ( + λ p I) λ max ( 0 ) + O(p γ /4 ) M by some constant M > 0, we can get z t x p Cp γ /8, z (og p) zt ( + λ p I) z dz zt ( + λ p I) z dz. () zt ( + λ p I) z dz C ( (og p) /bt) Cp bt, (3) unformy n (,..., t ) I c. For the second part of the sum n (), note that ( + λ p I) ( + λp I) Cλ p p γ Cp γ /, (4)
88 T.T. Ca, Y. Xa / Journa of Mutvarate Anayss 3 (04) 74 96 we can obtan that z t x p Cp γ /8, z (og p) zt ( + λ p I) z dz = z t x p Cp γ /8, z (og p) = ( + O(p γ / (og p) )) z t x p Cp γ /8, z (og p) = ( + O(p γ / (og p) )) = ( + O(p γ / (og p) )) zt (( + λ p I) ( + λp I) )z zt ( + λp I) z dz z t x p Cp γ /8 z x p Cp γ /8 where z R b. So for (,..., t ) I c, we have zt ( + λp I) z dz zt ( + λp I) z zt ( 0 + λ p I) z dz + O(p bt ) dz t + O(p bt ), (5) P,..., t ( + O(p γ / (og p) )) P( Y + λ p Z x p Cp γ /8 ) t + Cp t = ( + o()) P( Y x p ) t + Cp t d = ( + o())γ t H t ()p t tx, (6) σ where the ast equaton comes from Lemma 3. Smary, because P( Ỹ t x p ) P Ỹ + λ p Z t x p + λ p max Z j, (7) j t we can get P,..., t ( o())γ t d H t ()p t tx. (8) σ So (8) s proved. It remans to prove (9). For S I d wth d, wthout oss of generaty, we can assume S = { t d+,..., t }. By the defnton of S and I d, for any k {,..., t d }, there exsts at east one S such that σ k p γ. We dvde I d nto two parts: I d, = { < < t p : there exsts an k {,..., t d } such thatfor some, S wth, σ k p γ and σ k p γ, I d, = I d \ I d,. Ceary, I, = and I, = I. Moreover, we can show that Card(I d, ) Cp d +γ t. Smary as proved n () and (6) (8), for any (,..., t ) I d,, P Y x p,..., Y t x p P Y t d+ x p,..., Y t x p = O(p d ). Hence by ettng γ be suffcenty sma, P,..., t Cp +γ t = o(). I d, (9) For any (,..., t ) I d,, wthout oss of generaty, we assume that σ, t d+ p γ. Note that P Y x p,..., Y t x p P Y x p, Y t d+ x p,..., Y t x p. Let W be the covarance matrx of (Y T, Y T t d+,..., Y T t ) T. We can show that W W = O(p γ ), where W = dag(d, (t d)b+:tb,(t d)b+:tb ) and D s the covarance matrx of (Y T, Y T t d+ ) T. Usng the smar arguments as n () (5), we can get P Y x p,..., Y t x p ( + o())p( Y x p, Y t d+ x p ) O(p d+ ) + O(p t ).
T.T. Ca, Y. Xa / Journa of Mutvarate Anayss 3 (04) 74 96 89 Defne a set A = {, + p α, + p α,..., + [p α ]p α, }, where α s a constant that w be specfed ater and p α s the argest nteger no arger than p α. Because Y = sup z = Y T z, we have P( Y x p, Y t d+ x p ) = P P max z A, z = P sup Y T z x p, sup Y T t d+ z x p z = z = Y T z x p C max Y j p α, max j b z A, z = Y T t d+ z x p C max Y jt d+ p α j b max Y T z A, z = z x p Cp α/, max Y T z A, z = t d+ z x p Cp α/ + O(p t ) ( + o())cp bα max P( Y T z (j) A, z (j) z () x p, Y T t d+ z () x p ) + O(p t ) = ( + o())cp bα max P x x p / V ar(y T z (j) A, z (j) z () ), x x p / V ar(y T t d+ z () ) + O(p t ) = for =,..., b and j =,, where x = Y T z () / V ar(y T z () ) N(0, ) and x = Y T t d+ z () / V ar(y T t d+ z () ) N(0, ) and Because Cov(Y T Cov(x, x ) = z (), Y T t d+ z() ). V ar(y T z () )V ar(y T t d+ z () ) V ar(y T z () ) = j, b Cov(Y j z () j, Y z () ) = j, b ξ j z () j z (), and V ar(y T t d+ z () ) = j, b Cov(Y jt d+ z () j, Y z () ) = j, b ξ j z () j z (), where ξ j = Cov(Y j, Y ), then we have V ar(y T z () )V ar(y T t d+ z () ) = Aso we have Cov(Y T z (), Y T t d+ z() ) = j, b j, b = = j, b j, b ξ j z () j z () ξ j z () j z () ξ j z () j z (). j, b j, b ξ j z () j z () ξ j z () z () j Cov(Y j z () j, Y t d+ z () ) = j, b r t d+ ξ j z () j z (), so we get Cov(x, x ) = r t d+. In addton, V ar(y T z () ) λ max ( 0 ) = σ and V ar(y T t d+ z () ) λ max ( 0 ) = σ, we have P( Y x p, Y t d+ x p ) ( + o())cp bα P( x x p /σ, x x p /σ ) + O(p t ). Thus, by Lemma and the assumpton max j p r j r <, for any (,..., t ) I d,, we have P Y x p,..., Y t x p ( + o())4cp bα p +r O(p d+ ). Thus by ettng γ and α be suffcenty sma, ( + o())4cp d+γ t+bα d+ +r = o(). (30) I d, P,..., t Combnng (9) and (30), we prove (9). The proof of Lemma 5 s compete.
90 T.T. Ca, Y. Xa / Journa of Mutvarate Anayss 3 (04) 74 96 6.3. Proof of Theorem It suffces to prove P max p η / b (σ + ε/) og p. By Lemma 4 and the condton max δ /σ σ β og p wth β /(mn σ a )+ε for some constant ε > 0, we can get max p η / ω (σ + ε/) og p wth probabty tendng to one. So Theorem foows. 6.4. Proof of Theorem 4 We ony prove part () of Theorem 4 n ths secton, part () foows from the proof of part () drecty. Wthout oss of generaty, we assume that σ = for p. Let Y = ( n n n +n ( X X ), n n 3 nk n n +n 3 ( X X 3 ),..., K n K +n K ( X K T X K ) ) T, and et Z = b n n nk n n, +n (A( X X )),..., K n K +n K (A( X K X K )). Let H = { p : δ (j) 0 for some j < K} = {,..., m }. Defne the event G = {max p δ 8 σ og p}. We frst prove the foowng two emmas. Lemma 6. () Suppose (C) and (C) hod. Then under H wth r < /6, we have P(Φ α (I) =, G) = αp(g) + ( α)p(e c, G) + o(), (3) where E = {max Y < x p }, and P(E c, G) = I{G} I{G} P{δ },G Y x p + o(). () Suppose (C) and (C3) hod. Then under H wth r < /6, we have P(Φ α () =, G) = αp(g) + ( α)p(ẽ c, G) + o(), (3) where Ẽ = {max Z < x p }, and P(Ẽ c, G) = I{G} I{G} P{δ },G Z x p + o(). Lemma 7. Let a p = o((og p) / ). We have max max P Y x p + a n P max Y x p = o() (33) k p r k k unformy n the means δ, p, where x p = σ og p + (d )σ og og p + q α, r < /6 and Y, H are ndependent norma random vectors wth covarance matrx 0. Proof of Lemma 6. To prove (3) and (3), we ony need to prove P(Φ α (I) =, G) αp(g) + ( α)p(e c, G) + o(), under (C) and (C) and P(Φ α () =, G) αp(g) + ( α)p(ẽ c, G) + o(), under (C) and (C3). In the case when A =, by Lemma 4, we have η P max ( o()) max δ. p b p Thus we have P Φ α (A) =, G c P 8 σ og p max p j< K = ( o())p(g c ) o(), n j n n j + n (A(Ū j Ū )) b ( + δ) σ og p, G c o()
T.T. Ca, Y. Xa / Journa of Mutvarate Anayss 3 (04) 74 96 9 where U j,..., U jnj N(0, ), j =,..., K, for suffcenty sma δ > 0. We next consder P (Φ α (I) =, G) and P (Φ α (A) =, G). For notaton brefness, we denote P(LG δ ) and P(L δ ) by P {δ },G(L) and P {δ }(L) respectvey for any event L and =,..., p. Let H c = {,..., p} \ H. We have P {δ },G(Φ α (I) = ) = P {δ },G max Y x p + P {δ },G max Y < x p, max Y j x p, (34) j H c where x p = σ og p + (d )σ og og p + q α. Defne H c = {j Hc : σ j p ξ for any H} for r < ξ < ( r)/. It s easy to see that Card (H ) Kp r+ξ. It foows that P max Y j x p Kp r+ξ P Y x p = O(p r+ξ ) = o(). (35) j H We cam that P {δ },G max Y < x p, max Y j x p ( + o())p {δ j H c },G To prove (36), we set E = max Y < x p, F j = { Y j x p }, j H c. max Y < x p P {δ },G max Y j x p + o().(36) j H c Then by Bonferron nequaty, we have for any fxed nteger k > 0, P {δ k },G {E F j } ( ) t P {δ },G E F F t. (37) j H c t= < < t H c Let Y = (Y T, H)T, Y = (Y T,..., Y T t ) T, and et Y m = max Y and Y t = mn j t Y j. Let (Z T,..., Z T m )T =: Z N(0, I bm bm ), ndependent wth Y, and (Z T,..., Z T t ) T =: Z N(0, I bt bt ), ndependent wth Y. Smary as proved n Theorem, et λ p = Cp ξ for some constant C > 0, we have P {δ },G(E) = P {δ },G( Y m < x p ) P {δ },G Y + λ p Z < x p + λ p max Z P {δ },G( Y + λ p Z < x p + Cp ξ/8 ) + O(p M ), (38) for suffcenty arge constant M > 0. We aso have P {δ },G F j = P {δ },G( Y t x p ) j t P {δ },G Y + λ p Z t x p λ p max Z j t j Thus, we have P {δ },G( Y + λ p Z t x p Cp ξ/8 ) + O(p t ). (39) P {δ },G E F F t P{δ },G Y + λ p Z < x p + Cp ξ/8, Y + λ p Z t x p Cp ξ/8 + O(p t ). (40) Let W = (w j ) be the covarance matrx of the vector ((Y + λ p Z ) T, (Y + λ p Z ) T ) T. Let ( w j ) =: W = dag(w, W ), where W and W are the covarance matrces of Y + λ p Z and Y + λ p Z respectvey. So for (,..., t ) H c, we have W W = O(p r ξ ). Set z = (δ T, H, zt,..., z T t ) T and R = { u + δ x p + Cp ξ/8, H, z x p..., z t x p Cp ξ/8 }, R = R max z j 8b t og p, j t R = R max z j > 8b t og p. j t