WALD S EQUATION AND ASYMPTOTIC BIAS OF RANDOMLY STOPPED U-STATISTICS

Transcription

1 PROCEEDINGS OF HE AMERICAN MAHEMAICAL SOCIEY Volume 125, Number 3, March 1997, Pages S (97) WALD S EQUAION AND ASYMPOIC BIAS OF RANDOMLY SOPPED U-SAISICS VICOR H. DE LA PEÑA AND ZE LEUNG LAI (Communicated by Wei Y. Loh) Abstract. In this aer we make use of decouling arguments and martingale inequalities to extend Wald s equation for samle sums to randomly stoed de-normalized U-statistics. We also aly this result in conjunction with nonlinear renewal theory to obtain asymtotic exansions for the means of normalized U-statistics from sequential samles. 1. Introduction Let X 1,,X n be i.i.d. random variables taking values in a measurable sace (S, S) and having a common distribution F. Let g : S k IR be a Borel measurable function that is symmetric, i.e., g(x 1,,x k )=g(x π(1),,x π(k) ) for any ermutation π. Forn k, the normalized U-statistic with kernel g is ( ) n U n = { g(x i1,,x ik )}/, k 1 i 1< <i k n and k is called the degree of the U-statistic. U-statistics were introduced by Halmos [Ha] to rovide unbiased estimates of functionals of the form θ(= θ(f )) = g(x 1,,x k )df (x 1 ) df (x k ) of the oulation distribution F. Hoeffding [H] subsequently develoed a comrehensive theory of U-statistics. In articular, he introduced the basic decomosition (1) U n θ = kn 1 n f 1 (X i )+k(k 1){n(n 1)} 1 +k!{n (n k+1)} 1 1 i 1< <i k n f k (X i1,,x ik ), f 2 (X i,x j )+ Received by the editors October 15, 1994 and, in revised form, July 28, Mathematics Subject Classification. Primary 60G40, 62L12; Secondary 62L10. Key words and hrases. Hoeffding decomosition, decouling, martingales, Wald s equation, stoing times. he first author s research was suorted by the National Science Foundation under DMS he second author s research was suorted by the National Science Foundation under DMS c 1997 American Mathematical Society

2 918 VICOR H. DE LA PEÑA AND ZE LEUNG LAI in which Ef 1 (X 1 )=0andf j is symmetric with E{f j (X 1,,X j ) X 1,,X j 1 }= 0for2 j k.infact, f 1 (x)=eg(x, X 2,,X k ) θ, f 2 (x, y) =Eg(x, y, X 3,,X k ) f 1 (x) f 1 (y)+θ, etc. Using this decomosition, he roved the asymtotic normality of U n. he theory of U-statistics has layed a fundamental role in the develoment of nonarametric statistical methodology. he unbiased roerty EU n = θ no longer holds if we relace the fixed samle size n by a stoing time. o begin with, consider the case k =1,forwhichU n reduces to the samle mean W n = n 1 n W i,wherew i =g(x i ). Following the ioneering work of Cox [Co], much rogress has been made in the develoment of asymtotic aroximations for the bias of W, and in bias-corrected modifications of W for stoing times that are commonly used in sequential hyothesis testing and sequential estimation roblems, cf. [S1], [S2], [W2], [AW]. In Section 3 we derive similar asymtotic aroximations for the bias when W is relaced by U, where U n is a normalized U-statistic of general degree k. De-normalizing the samle mean W exected value is given by the following theorem, often referred to as Wald s equation or Wald s lemma, which has rovided a basic tool in the asymtotic analysis of the bias of W (cf. [AW], [S1]). yields the samle sum W i,whose heorem 1. Let {W i } be a sequence of i.i.d. random variables with EW i = µ, and let S n = n (W i µ). Let be a stoing time adated to {W i } such that E <. hen ES =0. In Section 2 we generalize heorem 1 to the (de-normalized) U-statistics S n = 1 i 1< <i j n f j(x i1,,x ij ). he generalization involves much deeer robabilistic ideas, including recent results from decouling theory. By using a different method, Chow, de la Peña and eicher [CdlP] recently generalized heorem 1 to the multilinear U-statistics S n = 1 i 1< <i k n X i 1 X ik. heir method, however, deends heavily on the multilinear form of S n and cannot be directly extended to the general U-statistics treated in Section Wald s equation for de-normalized U-statistics o highlight the main ideas and to simlify the arguments, we focus here on U-statistics of degree 2, for which we shall rove the following extension of Wald s equation. heorem 2. Let {X i } be a sequence of i.i.d. random variables with values in a measurable sace (S, S). Letbe a stoing time adated to {X i } such that E( 1/( 1) ) < for some 1 < 2. Let f : S S IR be a Borel measurable function such that E f(x 1,X 2 ) < and E(f(X 1,X 2 ) X 1 )=E(f(X 1,X 2 ) X 2 )=0a.s. hen E( f(x i,x j )) = 0. Before starting the roof of this result, let us recall an easy consequence of Levy s inequality which will be used reeatedly.

3 RANDOMLY SOPPED U-SAISICS 919 Lemma 1. Let {X i } be a sequence of indeendent mean zero random variables. hen, for all 1, E max j n j X i C E n X i, where the constant C > 0 deends on only. Proof. he roof of this result follows easily by using a symmetrization argument along with Levy s inequality for the tail robabilities of sums of symmetric random variables. See [K] for a roof. Proof of heorem 2. Since { f(x i,x j )} is a martingale, the desired conclusion holds (using the otional samling theorem) if we relace by the bounded stoing time min(,n). Hence by the dominated convergence theorem, the desired conclusion follows, for general stoing times, if it can be shown that (2) E max n f(x i,x j ) <. We will rove (2) by using a simle decouling argument to obtain a artial decouling between the X s and. A centering argument is used next to neutralize the remaining effect the stoing time has on the original variables. his neutralizing effect is realized by means of an averaging device couled with Hölder s inequality. hroughout the roof we will also use the Burkholder-Gundy (B-G) and Davis inequalities for martingales, cf. [C], Let { X i } and { Xi } be two indeendent coies of {X i }.Set j 1 j 1 d j =( f(x i,x j )) 2 1( j), e j =( f(x i, X j )) 2 1( j). hen {e j } and {d j } are tangent with resect to {F n }, i.e., L(d j F j 1 )=L(e j F j 1 ) for all j 2, where F n = σ(x 1,..., X n ; X 1,..., X n ). Recall from [Z] that if {d i }, {e i } are two tangent sequences of nonnegative random variables, then for all 0 <r<1, (3) n n E( d i ) r CE( e i ) r. his observation along with a double use of Davis inequality yields E max n j 1 f(x i,x j ) CE{ ( f(x i,x j )) 2 1( j)} 1 2 j 1 CE{ ( f(x i, X j )) 2 1( j)} 1 2 CE su n n j 1 ( f(x i, X j ))1( j). Hereandinthesequel,Cdenotes a generic constant which may change from one bound to another. Observe that n ( j 1 f(x i, X j ))1( j) isasumof indeendent mean-zero random variables (with the sum converging a.s. to a limit as n ) once we condition on ({X i },). herefore, using Lemma 1 conditionally,

4 920 VICOR H. DE LA PEÑA AND ZE LEUNG LAI we can bound the last exectation above by CE j 1 ( f(x i, X j ))1( j) = CE CE j=1 =(I)+(II)+(III). f(x i, X j ) + CE f(x i, X j ) f(x j, X i )) + CE f(x i, X i ) Of the three quantities above, (II) and(iii) are easier to bound, and one can decoule (II) further by using (3). As for (I), note that both {X i } and { X i } aear in j=1 f(x i, X j )ina symmetric fashion. We now exloit this fact by means of an averaging device and use Hölder s inequality to obtain (I) = CE j=1 C(E 1 1 C(E 1 1 C(E 1 1 = C(E 1 C(E 1 C(E 1 { f(x i, X j ) = CE (E j=1 1 f(x i, X j ) j=1 f(x i, X } j ) 1 ) 1/ {E E[( j=1 ( f(x i, X } j )) 2 ) 1/ 2 σ({x r },)], by B-G, { j=1 E E[ f(x i, X } 1/ j ) σ({x r },)] (E f(x i, X 1 ) ) 1 {E( f 2 (X i, X 1 )) /2 } 1/, by B-G, (E f(x i, X 1 ) ) 1 = C(E 1 1 (EE f(x1, X 1 ) ) 1, by concavity of the function f(x) =x /2, and by heorem 1 alied conditionally. o bound (II), recall that { X i } and { Xi } are indeendent coies of {X i },and set j 1 d j =( f(x j, X j 1 i )) 2 1( j), ẽ j =( f( Xj, X i )) 2 1( j). hen {ẽ j } and { d j } are tangent with resect to { F n },where F n =σ(x 1,..., X n ; X 1,..., X n ; X1,..., X n ).

5 RANDOMLY SOPPED U-SAISICS 921 Next, Davis inequality and (3) yield (II) =CE f(x j, X i ) = CE j 1 f(x j, X i ) CE j 1 ( f(x j, X i )) 2 1( j) CE j 1 ( f( Xj, X j 1 i )) 2 1( j) CE f( Xj, X i ), where the last inequality follows by alying first Davis inequality and then Lemma 1. Observe that we have now comletely decouled the stoing time. Moreover, by the B-G inequality and the concavity of f(x) =x /2, n j 1 E f( Xj, X n j 1 i ) {E f( Xj, X i ) } 1 C{E( C{ n j 1 ( f( Xj, X i )) 2 ) 1 2 } C{ n f( Xj, X i ) } 1 j 1 E n (j 1)E f( Xj, X 1 ) } 1 2 Cn {E f(x1,x 2 ) } 1. Putting these two observations together, we get (II) =CE n=1 f(x j, X i ) CE j 1 f( Xj, X i ) n j 1 = C E f( Xj, X i ) P ( = n) C(E f(x 1,X 2 ) ) 1 n 2 P ( = n) n=1 = C(E f(x 1,X 2 ) ) 1 E 2 C(E f(x1,x 2 ) ) 1 E 1 1. (III) is the simlest quantity to bound. By Hölder s inequality and the B-G inequality, E f(x i, X i ) (E f(x i, X i ) ) 1 C{E( f 2 (X i, X i )) 1 2 )} C(E f(x i, X i ) ) 1 = C(EE f(x1, X 1 ) ) 1, where we have alied heorem 1 in the last equality. his comletes the roof of heorem 2.

6 922 VICOR H. DE LA PEÑA AND ZE LEUNG LAI he moment conditions on and on f(x 1,X 2 ) for the case =2inheorem2 are (4) E < and Ef 2 (X 1,X 2 )<. Comarison of (4) with the moment conditions, E < and E f(x 1 ) <, of heorem 1 dealing with U-statistics of degree 1 suggests that extension of Wald s equation to U-statistics of degree k may involve higher moment conditions on f(x 1,,X k ) and/or. For the secial case of a multilinear kernel f(x 1,,x k )=x 1 x k, Chow, de la Peña and eicher [CdlP] established that E( 1 i 1< <i k X i 1 X ik ) = 0 under the moment conditions (5) E k 1 < and EX 2 1 <, EX 1 =0, or more generally, under E (k 1)/( 1) < and E X 1 < for some 1 < 2,EX 1 = 0. Concerning the sharness of these results, observe that the case k =2,= 2 along with Wald s first equation give Wald s second equation (involving the second moment of the randomly stoed sum). We are able to extend heorem 2 to U-statistics of degree k>2, with symmetric kernels and under the moment conditions E f(x 1,,X k ) < for some 1 < 2, E{f(X 1,,X k ) X 1,,X h 1,X h+1,,x k }=0a.s. for all 1 h k, ande ρ(k,) <, where ρ(k, ) =( 1/(k 1) 1) 1 < (k 1)/ln. he roof, which involves deeer decouling ideas and is considerably more comlicated than that of heorem 2, will be resented elsewhere together with the decouling results needed. 3. Means of normalized U-statistics in sequential exeriments For notational simlicity we shall again focus on U-statistics of degree 2 in this section, since the same arguments can be alied to the general case. Let g : S S IR be a symmetric Borel measurable function. As ointed out in Section 1, although the normalized U-statistic U n = ( ) n (6) g(x i,x j )/ 2 based on i.i.d. observations X 1,,X n is an unbiased estimate of θ = Eg(X 1,X 2 ), the corresonding U-statistic U based on a samle {X 1,,X }from a sequential exeriment, in which the samle size is not fixed in advance but is sequentially determined from the current and ast data, is biased. We shall analyze the bias EU θ in sequential exeriments whose stoing rules are of the form n (7) =inf{n 2: Y i + ξ n a}, where (X 1,Y 1 ),(X 2,Y 2 ), are i.i.d. random vectors and for some c>0andd>0, (8) EY 1 > 0, EY1 2 <, np {ξ n < (d EY 1 )n} <, 1

7 (9) {[( RANDOMLY SOPPED U-SAISICS 923 n Y i + ξ n n/c) + ],n 1}is uniformly integrable for some 3, 1 (10) lim su δ 0 n 1 P {max k nδ ξ n+k ξ n >ɛ}=0 forallɛ>0. In addition, it is assumed that there are events A n such that for some α 3/2, (11) np ( n=1 k=n A k ) < and {max ξ n+k1(a n+k ) α,n 1}is uniformly integrable, k n where A k denotes the comlement of A k. Stoing rules of the form (7) are commonly used in sequential testing roblems. Here the rimary objective of the sequential exeriment is to test a certain statistical hyothesis, and the exeriment terminates as soon as the test statistic Z n exceeds some threshold a. yically Z n can be reresented in the form of a random walk n Y i lus a remainder ξ n. Regularity conditions of the tye (10) and (11) on ξ n were first introduced by Lai and Siegmund [LS] to develo a renewal theory for the erturbed random walk n 1 Y i + ξ n and have been shown to hold for many commonly used test statistics in sequential analysis, cf. [S2], [W2]. In addition to the rimary objective of testing the hyothesis, the exeriment also has secondary objectives of estimating certain arameters of interest, and therefore the roblem of bias in estimation with U-statistics or other commonly used estimators following a sequential test is of fundamental interest in sequential analysis. Stoing rules of the form (7) also arise naturally in sequential estimation roblems. Here the rimary objective of the sequential exeriment is to estimate certain arameters with fixed accuracy or with an otimal balance between samling cost and mean squared error, and stoing rules of the form (7) rovide asymtotically efficient solutions to these roblems, cf. [AW], [W1], [W2], [S2]. Aras and Woodroofe [AW, heorem 2] recently roved the following asymtotic formula for the bias of the stoed samle mean W = 1 n 1 W i as a : (12) E( W )=E(W 1 )+a 1 {Cov(W 1,Y 1 )+o(1)}, where (= a ) is the stoing time defined in (7) with the Y i and ξ i satisfying (8) (11), and (W 1,Y 1 ),(W 2,Y 2 ), are i.i.d. bivariate random vectors such that E W 1 q < for some q max{4, 2α/(α 1), 2/( 2)}. he following theorem extends (12) to the case where the samle mean W n is relaced by the normalized U-statistic (6). heorem 3. Assume (8) (11) and define (= a ) by a symmetric Borel measurable function such that (7). Letg:S S IRbe (13) E g(x 1,X 2 ) q < for some q max{6, 2α/(α 1)}, and define U n by (6). Letψ(x)=Eg(X 1,x),θ =Eg(X 1,X 2 ). hen as a, (14) E(U )=θ+2a 1 {Cov(ψ(X 1 ),Y 1 )+o(1)}.

8 924 VICOR H. DE LA PEÑA AND ZE LEUNG LAI Proof. Let µ = EY 1 (> 0) and assume without loss of generality that c<1/µ in (9). By Proosition 2 of [AW], /a 1/µ a.s. By (1), (15) U θ =2 1 ψ(x i )+2{( 1)} 1 f 2 (X i,x j ), noting that 2 by (7). he function f 2 in (15) is symmetric, and E{f 2 (X 1,X 2 ) X 1 }=0 By (13) and (12), as a, (16) E{ 1 a.s. ψ(x i )} = a 1 {Cov(ψ(X 1 ),Y 1 )+o(1)}. In view of (15) and (16), (14) follows if it can be shown that (17) lim a aeũ =0, where Ũn = {n(n 1)} 1 f 2 (X i,x j ). By Lemma 1 of [LW], E 1 i<j m f 2(X i,x j ) q =O(m q ). herefore E Ũm q = O(m q ). Since {Ũn,n 1} is a reverse martingale (cf. [GS]), it then follows from Doob s inequality that (18) E(su Ũn q )=O(E Ũm q )=O(m q ). n m By Hölder s inequality and (18), since P { 2a/µ} 0, (19) E( Ũ 1{ 2a/µ}) [E( su Ũn q )] 1/q [P { 2a/µ}] 1 q 1 = o(a 1 ). n 2a/µ Let τ =min{,[2a/µ]} and note that Ũ Ũτ =(Ũ Ũ[2a/µ])1{ >2a/µ}. In view of (18) and (19), (17) would follow if it can be shown that lim a aeũτ =0. Since E{τ(τ 1)Ũτ} = 0 by heorem 2, it suffices to show that (20) lim τ(τ 1) )Ũτ } =0. a a Using (18) and an argument similar to the roof of Proosition 6 of [AW], it can be shown that as a, (21) E( a µ 2 τ(τ 1)/a Ũτ 1{τ ca/4}) 0. Moreover, noting that τ 2a/µ, (22) E( (a µ2 τ(τ 1) )Ũτ q 1{ca/4 τ 2a/µ}) a (3a) q E( su Ũn q )=O(1), n ca/4 by (18). Since (τ a/µ)/ a = O P (1) (cf. [AW]) and since su n ca/4 Ũn = O P (a 1 ) by (18), it follows that {a µ 2 τ(τ 1)/a}Ũτ1{τ ca/4} 0. P Combining this with the uniform integrability result imlied by (22) then yields (23) lim a µ2 τ(τ 1)/a Ũτ 1{τ ca/4})=0. From (21) and (23), (20) follows.

9 RANDOMLY SOPPED U-SAISICS 925 References [AW] Aras,G.andWoodroofe,M.,Asymtotic exansions for the moments of a randomly stoed average, Ann. Statist. 21 (1993), MR 94a:62120 [CdlP] Chow, Y. S., de la Peña, V. H. and eicher, H., Wald s equation for a class of denormalized U-statistics, Ann. Probab. 21 (1993), MR 94c:60064 [C] Chow, Y. S., and eicher, H., Probability heory: Indeendence, Interchangeability, Martingales, Sringer-Verlag, New York, MR 80a:60004 [Co] Cox, D. R., A note on sequential estimation of means, Proc. Cambridge Philos. Soc. 48 (1952), MR 14:190e [GS] Grams, W. F. and Serfling, R. J., Convergence rates for U-statistics and related statistics, Ann. Statist. 1 (1973), MR 49:1561 [Ha] Halmos, P. R., he theory of unbiased estimation, Ann. Math. Statist. 17 (1946), MR 7:463g [H] Hoeffding, W., A class of statistics with asymtotically normal distribution, Ann. Math. Statist. 19 (1948), MR 10:134g [K] Klass, M. J., A method of aroximating exectations of functions of sums of indeendent random variables, Ann. Probab. 9 (1981), MR 82f:60119 [LS] Lai,. L. and Siegmund, D., A nonlinear renewal theory with alications to sequential analysis I, Ann. Statist. 5 (1977), ; II, Ann. Statist. 7 (1979), MR 56:3935; MR 80c:62100 [LW] Lai,. L. and Wang, J. Q., Edgeworth exansions for symmetric statistics with alications to bootstra methods, Statistica Sinica 3 (1993), MR 95c:62019 [S1] Siegmund, D., Estimation following sequential tests, Biometrika 65 (1978), MR 80f:62079 [S2] Siegmund, D., Sequential Analysis: ests and Confidence Intervals, Sringer-Verlag, New York, MR 87h:62145 [W1] Woodroofe, M., Second order aroximation for sequential oint and interval estimation, Ann. Statist. 5 (1977), MR 58:13548 [W2] Woodroofe, M., Nonlinear Renewal heory in Sequential Analysis, SIAM, Philadelhia, MR 83j:62118 [Z] Zinn, J., Comarison of martingale difference sequences, Probability in Banach saces V, Lecture Notes in Math., vol. 1153, Sringer-Verlag, Berlin, 1985, MR 87f:60070 Deartment of Statistics, Columbia University, 617 Mathematics Bldg., New York, New York address: v@wald.stat.columbia.edu Deartment of Statistics, Stanford University, Sequoia Hall, Stanford, California address: karola@layfair.stanford.edu