A contamination model for approximate stochastic order

Transcription

1 A contamination model for approximate stochastic order Eustasio del Barrio Universidad de Valladolid. IMUVA. 3rd Workshop on Analysis, Geometry and Probability - Universität Ulm 28th September - 2dn October 2015, Ulm Eustasio del Barrio Testing approximate stochastic order 1 / 37

2 Outline Outline 1 Stochastic order, testability and relaxed versions of s. o. 2 Inference for approximate stochastic order 3 Implementation, simulation & data example Eustasio del Barrio Testing approximate stochastic order 2 / 37

3 Stochastic order, testability and relaxed versions of s. o. The stochastic order model π(fn,gm) = π(gm,fn) = EDF's Gm Age 10 Boys Fn Age 10 Girls heights Data: National Health and Nutrition Examination Survey Empirical d.f. s for boys and girls at age 10. Are girls taller than boys? Stochastic order (Lehmann, 1955): P, Q probs. on R with d.f. s F, G For NHANES data, P 10 st Q 10? P st Q if F (x) G(x), x R Eustasio del Barrio Testing approximate stochastic order 3 / 37

4 Stochastic order, testability and relaxed versions of s. o. Testing stochastic order Common testing problems in literature (P st Q F st G) a) H 0 : F = G vs H a : F < st G b) H 0 : F st G vs H a : F st G c) H 0 : F st G vs H a : F st G Problem a) focus on statistical evidence for strict relation assumes stochastic order holds both H 0 and H a can be false (here focus on b), c)) Eustasio del Barrio Testing approximate stochastic order 4 / 37

5 Stochastic order, testability and relaxed versions of s. o. Testing stochastic order Common testing problems in literature (P st Q F st G) Problem a) a) H 0 : F = G vs H a : F < st G b) H 0 : F st G vs H a : F st G c) H 0 : F st G vs H a : F st G focus on statistical evidence for strict relation assumes stochastic order holds both H 0 and H a can be false (here focus on b), c)) Problem b) testing for stochastic dominance (McFadden, 1989; Mosler, 1995; Anderson, 1996, Davidson & Duclos, 2000; Linton et al., 2005, 2010,... ) goodness-of-fit problem absence of evidence against s. o. as minimal requirement for a) lack of evidence against H 0 not evidence for F st G Eustasio del Barrio Testing approximate stochastic order 4 / 37

6 Stochastic order, testability and relaxed versions of s. o. Testing stochastic order Problem c) H 0 : F st G vs H a : F < st G: assessing stochastic order rejection provides convincing evidence of F < st G Eustasio del Barrio Testing approximate stochastic order 5 / 37

7 Stochastic order, testability and relaxed versions of s. o. Testing stochastic order Problem c) H 0 : F st G vs H a : F < st G: assessing stochastic order rejection provides convincing evidence of F < st G Unfortunately, no good test for b) exists: Assume X 1,..., X n i.i.d. F < st G Φ an α-level test (E H Φ(X 1,..., X n ) α, H H 0 ) Take x m s.t. G(x m ) > 1 1 m, H m s.t. H m (x m ) = 0 Set F m = (1 1 m )F + 1 m H m; F m st G α E Fm Φ(X 1,..., X n ) (1 1 m )n E F Φ(X 1,..., X n ) Take m no data test (reject H 0 with prob α regardless data) is UMP! Berger, 1988 (one-sample); Davidson & Duclos, 2013 (two-sample) Eustasio del Barrio Testing approximate stochastic order 5 / 37

8 Stochastic order, testability and relaxed versions of s. o. Uniformly consistent tests Uniformly consistent tests X 1, X 2,... i.i.d. P with values in X A 0,n (A 1,n ) acceptance (critical) set for H n against K n based on X 1,..., X n Test uniformly (exponentially) consistent if for some r, r > 0 sup P n (A 1,n ) e rn, P H n sup P n (A 0,n ) e r n P K n Consider the testing problem H : P = P 0 vs K : d(p, P 0 ) > δ If d dominates d T V (Barron, 1989) and P 0 not discrete, no uniformly consistent test of H vs K Eustasio del Barrio Testing approximate stochastic order 6 / 37

9 Stochastic order, testability and relaxed versions of s. o. Uniformly consistent tests Here we propose A relaxed version of stochastic order for which we can expect to get statistical evidence A consistent procedure for gathering that evidence Which is exponentially uniformly consistent (with due corrections) Some of our relaxations does hold Deviation from stochastic order measured through required level of relaxation Easy interpretation Eustasio del Barrio Testing approximate stochastic order 7 / 37

10 Stochastic order, testability and relaxed versions of s. o. Relaxations of stochastic order A relaxation of stochastic order (Arcones et al., 2002) θ(p, Q) := P[X Y ] X, Y independent r.v. s with laws P, Q, resp. P sp Q (stochastically precedes) if θ(p, Q) 1 2 Stochastic ordering implies stochastic precedence: if P st Q P(X Y ) = (1 G(x ))df (x) (1 F (x ))df (x) = P(X X ) 1 2, X independent copy of X Eustasio del Barrio Testing approximate stochastic order 8 / 37

11 Stochastic order, testability and relaxed versions of s. o. Relaxations of stochastic order A relaxation of stochastic order (Arcones et al., 2002) θ(p, Q) := P[X Y ] X, Y independent r.v. s with laws P, Q, resp. P sp Q (stochastically precedes) if θ(p, Q) 1 2 Stochastic ordering implies stochastic precedence: if P st Q P(X Y ) = (1 G(x ))df (x) (1 F (x ))df (x) = P(X X ) 1 2, X independent copy of X Stochastic precedence a less restrictive assumption Eustasio del Barrio Testing approximate stochastic order 8 / 37

12 Stochastic order, testability and relaxed versions of s. o. Relaxations of stochastic order A relaxation of stochastic order (Arcones et al., 2002) θ(p, Q) := P[X Y ] X, Y independent r.v. s with laws P, Q, resp. P sp Q (stochastically precedes) if θ(p, Q) 1 2 Stochastic ordering implies stochastic precedence: if P st Q P(X Y ) = (1 G(x ))df (x) (1 F (x ))df (x) = P(X X ) 1 2, X independent copy of X Stochastic precedence a less restrictive assumption But P sp Q equivalent to median(y X) 0, roughly change in location Eustasio del Barrio Testing approximate stochastic order 8 / 37

13 Stochastic order, testability and relaxed versions of s. o. Relaxations of stochastic order A relaxation of stochastic order (Arcones et al., 2002) θ(p, Q) := P[X Y ] X, Y independent r.v. s with laws P, Q, resp. P sp Q (stochastically precedes) if θ(p, Q) 1 2 Stochastic ordering implies stochastic precedence: if P st Q P(X Y ) = (1 G(x ))df (x) (1 F (x ))df (x) = P(X X ) 1 2, X independent copy of X Stochastic precedence a less restrictive assumption But P sp Q equivalent to median(y X) 0, roughly change in location (different, but similar nature as E(Y X) 0) Eustasio del Barrio Testing approximate stochastic order 8 / 37

14 Stochastic order, testability and relaxed versions of s. o. Tolerance zones around false models False model assessment Assume model F is false (X P, P / F) Is model F an adequate approximation for the data? for P? P θ F is an adequate approximation for the data, X 1,..., X n, if a typical sample of size n from P θ looks like the data Data features (Davies, 1995) Credibility indices (Lindsay & Liu, 2009) P θ F gives an adequate description of P if d(p, P θ ) τ d = χ 2 distance (Hodges & Lehmann, 1954) d = Euclidean distance (Dette & Munk, 2003) d = smallest π such that P = (1 π)p θ + πr (Rudas et al. (1994); Ae-dB-C-M, 2008, 2010, 2011, 2012; Liu & Lindsay, 2009; Cerioli et al., 2012) Choice of τ a hard issue Interpretation of τ simpler for the π-index Eustasio del Barrio Testing approximate stochastic order 9 / 37

15 Essential model validation Stochastic order, testability and relaxed versions of s. o. Essential model validation Observe data X P, test H 1 P = (1 α)r + α P for some R F Observe ind. samples X P, Y Q, test H 2 P = (1 α)r + α P Q = (1 α)s + α Q for some (R, S) F Related problem of interest: Find α 0 = minimal α s.t. null model holds Eustasio del Barrio Testing approximate stochastic order 10 / 37

16 Essential model validation Stochastic order, testability and relaxed versions of s. o. Essential model validation Observe data X P, test H 1 P = (1 α)r + α P for some R F Observe ind. samples X P, Y Q, test H 2 P = (1 α)r + α P Q = (1 α)s + α Q for some (R, S) F Related problem of interest: Find α 0 = minimal α s.t. null model holds Example: the similarity model (AE-dB-C-M, 2012) P and Q α-similar, α [0, 1) if prob, R, s.t. { P = (1 α)r + α P Q = (1 α)r + α Q Eustasio del Barrio Testing approximate stochastic order 10 / 37

17 Essential model validation Stochastic order, testability and relaxed versions of s. o. Essential model validation Observe data X P, test H 1 P = (1 α)r + α P for some R F Observe ind. samples X P, Y Q, test H 2 P = (1 α)r + α P Q = (1 α)s + α Q for some (R, S) F Related problem of interest: Find α 0 = minimal α s.t. null model holds Example: the similarity model (AE-dB-C-M, 2012) P and Q α-similar, α [0, 1) if prob, R, s.t. { P = (1 α)r + α P Q = (1 α)r + α Q P, Q α-similar, d T V (P, Q) α (d T V (P, Q) = sup A P (A) Q(A) ) Eustasio del Barrio Testing approximate stochastic order 10 / 37

18 Essential model validation Stochastic order, testability and relaxed versions of s. o. Essential model validation Observe data X P, test H 1 P = (1 α)r + α P for some R F Observe ind. samples X P, Y Q, test H 2 P = (1 α)r + α P Q = (1 α)s + α Q for some (R, S) F Related problem of interest: Find α 0 = minimal α s.t. null model holds Example: the similarity model (AE-dB-C-M, 2012) P and Q α-similar, α [0, 1) if prob, R, s.t. { P = (1 α)r + α P Q = (1 α)r + α Q P, Q α-similar, d T V (P, Q) α (d T V (P, Q) = sup A P (A) Q(A) ) Here F = {(R, R)} and α 0,sim (P, Q) = d T V (P, Q) Eustasio del Barrio Testing approximate stochastic order 10 / 37

19 Stochastic order, testability and relaxed versions of s. o. Essential model validation Approximate stochastic order: P st,α Q if P = (1 α)r + α P Q = (1 α)s + α Q for some R st S Eustasio del Barrio Testing approximate stochastic order 11 / 37

20 Stochastic order, testability and relaxed versions of s. o. Essential model validation Approximate stochastic order: P st,α Q if P = (1 α)r + α P Q = (1 α)s + α Q for some R st S (F = {(R, S) : R st S}) Eustasio del Barrio Testing approximate stochastic order 11 / 37

21 Stochastic order, testability and relaxed versions of s. o. Essential model validation Approximate stochastic order: P st,α Q if P = (1 α)r + α P Q = (1 α)s + α Q for some R st S (F = {(R, S) : R st S}) (maybe s. o. too restrictive, but core of distribution fits model) Eustasio del Barrio Testing approximate stochastic order 11 / 37

22 Stochastic order, testability and relaxed versions of s. o. Essential model validation Approximate stochastic order: P st,α Q if P = (1 α)r + α P Q = (1 α)s + α Q for some R st S (F = {(R, S) : R st S}) (maybe s. o. too restrictive, but core of distribution fits model) Interest on minimal contamination level s.t. stochastic order model holds α 0 (P, Q) := inf{α : P st,α Q} Eustasio del Barrio Testing approximate stochastic order 11 / 37

23 Stochastic order, testability and relaxed versions of s. o. Trimming methods in essential model validation Trimmed Distributions (X, β) measurable space; P(X, β) prob. measures on (X, β), P P(X, β) { dr R α (P ) = R P(X, β) : R P, dp 1 } 1 α P -a.s. Proposition (a) R α (P ) is a convex set; α 1 α 2 R α1 (P ) R α2 (P ) (b) If α < 1 and (X, β) complete separable metric space then R α (P ) compact for weak convergence. (c) R R α (P ) iff P = (1 α)r + α P Eustasio del Barrio Testing approximate stochastic order 12 / 37

24 Stochastic order, testability and relaxed versions of s. o. Essential model validation & trimming Null models in essential model validation expressable in terms of trimmings Observe X P, test H 1 P = (1 α)r + α P for some R F H 1 holds iff R α (P ) F Observe indep. X P, Y Q test H 2 P = (1 α)r + α P Q = (1 α)s + α Q for some (R, S) F H 2 holds iff (R α (P ) R α (Q)) F If R(P ), F closed for metric d H 1 holds iff d(r α (P ), F) = 0 H 2 holds iff d(r α (P ) R α (Q), F) = 0 Eustasio del Barrio Testing approximate stochastic order 13 / 37

25 Stochastic order, testability and relaxed versions of s. o. Approximate stochastic order & trimming Trimmings mix well with stochastic order: For any P P α, P α in R π (P ) s. t. P α st R st P α for every R R α (P ) Eustasio del Barrio Testing approximate stochastic order 14 / 37

26 Stochastic order, testability and relaxed versions of s. o. Approximate stochastic order & trimming Trimmings mix well with stochastic order: For any P P α, P α in R π (P ) s. t. P α st R st P α for every R R α (P ) P α, P α easily computable Eustasio del Barrio Testing approximate stochastic order 14 / 37

27 Stochastic order, testability and relaxed versions of s. o. Approximate stochastic order & trimming Trimmings mix well with stochastic order: For any P P α, P α in R π (P ) s. t. P α st R st P α for every R R α (P ) P α, P α easily computable Recall P st,α Q iff P R α (P ), Q Rα (Q), s.t. P st Q Eustasio del Barrio Testing approximate stochastic order 14 / 37

28 Stochastic order, testability and relaxed versions of s. o. Approximate stochastic order & trimming Trimmings mix well with stochastic order: For any P P α, P α in R π (P ) s. t. P α st R st P α for every R R α (P ) P α, P α easily computable Recall P st,α Q iff P R α (P ), Q Rα (Q), s.t. P st Hence P st,α Q iff P α st Q α Q Eustasio del Barrio Testing approximate stochastic order 14 / 37

29 Stochastic order, testability and relaxed versions of s. o. Approximate stochastic order & trimming Trimmings mix well with stochastic order: For any P P α, P α in R π (P ) s. t. P α st R st P α for every R R α (P ) P α, P α easily computable Recall P st,α Q iff P R α (P ), Q Rα (Q), s.t. P st Hence P st,α Q iff P α st Q α Conclude from this α 0 (P, Q) = sup(g(x) F (x)) x Q Eustasio del Barrio Testing approximate stochastic order 14 / 37

30 Stochastic order, testability and relaxed versions of s. o. Approximate stochastic order & trimming Trimmings mix well with stochastic order: For any P P α, P α in R π (P ) s. t. P α st R st P α for every R R α (P ) P α, P α easily computable Recall P st,α Q iff P R α (P ), Q Rα (Q), s.t. P st Hence P st,α Q iff P α st Q α Conclude from this α 0 (P, Q) = sup(g(x) F (x)) x Q Equivalently, P st,α Q α 0 (P, Q) α Eustasio del Barrio Testing approximate stochastic order 14 / 37

31 Stochastic order, testability and relaxed versions of s. o. Approximate stochastic order & trimming Example 1. P = N(µ, σ), Q = N(ν, σ), µ > ν Now Q st P and, α 0 (P, Q) = (sup(g(x) F (x)) = 2Φ ( ) µ ν 2σ 1. x µ ν = 0.1σ P st,0.04 Q µ ν = 0.25σ P st, Q µ ν = 0.5σ P st, Q µ ν = σ P st, Q Eustasio del Barrio Testing approximate stochastic order 15 / 37

32 Stochastic order, testability and relaxed versions of s. o. Approximate stochastic order & trimming Example 1. P = N(µ, σ), Q = N(ν, σ), µ > ν Now Q st P and, α 0 (P, Q) = (sup(g(x) F (x)) = 2Φ ( ) µ ν 2σ 1. x µ ν = 0.1σ P st,0.04 Q µ ν = 0.25σ P st, Q µ ν = 0.5σ P st, Q µ ν = σ P st, Q Example 2. P = N(µ, σ), Q = N(ν, τ) Here α 0 (P, Q) depends on µ, ν, σ, τ Eustasio del Barrio Testing approximate stochastic order 15 / 37

33 Stochastic order, testability and relaxed versions of s. o. Approximate stochastic order & trimming Example 1. P = N(µ, σ), Q = N(ν, σ), µ > ν Now Q st P and, α 0 (P, Q) = (sup(g(x) F (x)) = 2Φ ( ) µ ν 2σ 1. x µ ν = 0.1σ P st,0.04 Q µ ν = 0.25σ P st, Q µ ν = 0.5σ P st, Q µ ν = σ P st, Q Example 2. P = N(µ, σ), Q = N(ν, τ) Here α 0 (P, Q) depends on µ, ν, σ, τ θ(p, Q) = µ ν (Arcones et al.,2002) For µ = ν, P sp Q regardless σ, τ Eustasio del Barrio Testing approximate stochastic order 15 / 37

34 Stochastic order, testability and relaxed versions of s. o. Approximate stochastic order & trimming Example 1. P = N(µ, σ), Q = N(ν, σ), µ > ν Now Q st P and, α 0 (P, Q) = (sup(g(x) F (x)) = 2Φ ( ) µ ν 2σ 1. x µ ν = 0.1σ P st,0.04 Q µ ν = 0.25σ P st, Q µ ν = 0.5σ P st, Q µ ν = σ P st, Q Example 2. P = N(µ, σ), Q = N(ν, τ) Here α 0 (P, Q) depends on µ, ν, σ, τ θ(p, Q) = µ ν (Arcones et al.,2002) For µ = ν, P sp Q regardless σ, τ But N(0, σ) takes values greater than N(0, 0) 50% of times! Eustasio del Barrio Testing approximate stochastic order 15 / 37

35 Inference for approximate stochastic order Estimation & testing Inference in approximate stochastic order models Assume X 1,..., X n i.i.d. P ; Y 1,..., Y m i.i.d. Q, independent samples Eustasio del Barrio Testing approximate stochastic order 16 / 37

36 Inference for approximate stochastic order Estimation & testing Inference in approximate stochastic order models Assume X 1,..., X n i.i.d. P ; Y 1,..., Y m i.i.d. Q, independent samples Goals (a) For a fixed α, test H 0 : P st,α Q vs. H 0 : P st,α Q (b) For a fixed α, test H 0 : P st,α Q vs. H 0 : P st,α Q (c) Estimation/confidence intervals/confidence bounds for α 0 (P, Q) Eustasio del Barrio Testing approximate stochastic order 16 / 37

37 Inference for approximate stochastic order Estimation & testing Inference in approximate stochastic order models Assume X 1,..., X n i.i.d. P ; Y 1,..., Y m i.i.d. Q, independent samples Goals (a) For a fixed α, test H 0 : P st,α Q vs. H 0 : P st,α Q (b) For a fixed α, test H 0 : P st,α Q vs. H 0 : P st,α Q (c) Estimation/confidence intervals/confidence bounds for α 0 (P, Q) Recall P st,α Q α 0 (P, Q) α; reformulate (a), (b) as (a) H 0 : α 0 (P, Q) α vs. H a : α 0 (P, Q) > α (testing against approximate s.o.) (b ) H 0 : α 0 (P, Q) α vs. H a : α 0 (P, Q) < α (testing for approximate s.o.) Eustasio del Barrio Testing approximate stochastic order 16 / 37

38 Inference for approximate stochastic order Asymptotic theory Assume F and G continuous; n = m F n, G n empirical d.f. s Theorem α 0 (F n, G n ) a.s. α 0 (F, G), n(α0 (F n, G n ) α 0 (F, G)) w B(F, G) with B(F, G) = B 1, B 2 independent Brownian Bridges; sup (B 1 (F (x)) B 2 (G(x))), x Γ(F,G) Γ(F, G) := {x R : F (x) G(x) = α 0 (F, G)} A bootstrap version also available, but slow approximation (support estimation) Eustasio del Barrio Testing approximate stochastic order 17 / 37

39 Inference for approximate stochastic order Asymptotic theory Quantiles of B(F, G) depend on F, G in a complex way Define B α = B(U(α, 1 + α), U(0, 1)), 0 α 1 Eustasio del Barrio Testing approximate stochastic order 18 / 37

40 Inference for approximate stochastic order Asymptotic theory Quantiles of B(F, G) depend on F, G in a complex way Define B α = B(U(α, 1 + α), U(0, 1)), 0 α 1 B α = sup (B 1 (t) B 2 (t α)) α t 1 P ( B 0 > 2t) = e t2 /2, α = 0, 0.1,..., 0.5 Eustasio del Barrio Testing approximate stochastic order 18 / 37

41 Inference for approximate stochastic order Asymptotic theory Bounds for asymptotic quantiles K β (F, G) (resp. K β (α)) β-quantile of B(F, G) (resp. Bα ) K β (F, G) K β (α(f, G)), β (0, 1) If β (0, 1 2 ] σ(f, G, α(f, G))Φ 1 (β) K β (F, G), where σ(f, G, α(f, G)) = min t T (F,G,α(F,G)) σ t, T (F, G, α(f, G)) = {t : x s.t. F (x) = t, G(x) = t α(f, G)} and σ 2 t = t(1 t) + (t α(f, G))(1 t + α(f, G)) Eustasio del Barrio Testing approximate stochastic order 19 / 37

42 Inference for approximate stochastic order Testing against essential stochastic order Testing against essential stochastic order Consider H 0 : α 0 (F, G) α, vs. H a : α 0 (F, G) > α Eustasio del Barrio Testing approximate stochastic order 20 / 37

43 Inference for approximate stochastic order Testing against essential stochastic order Testing against essential stochastic order Consider H 0 : α 0 (F, G) α, vs. H a : α 0 (F, G) > α (equivalently, H 0 : F st,α G vs. H a : F st,α G) Eustasio del Barrio Testing approximate stochastic order 20 / 37

44 Inference for approximate stochastic order Testing against essential stochastic order Testing against essential stochastic order Consider H 0 : α 0 (F, G) α, vs. H a : α 0 (F, G) > α (equivalently, H 0 : F st,α G vs. H a : F st,α G) Reject H 0 if n(α0 (F n, G n ) α) > K 1 β (α), K 1 β (α) = 1 β quantile of B(α) Eustasio del Barrio Testing approximate stochastic order 20 / 37

45 Inference for approximate stochastic order Testing against essential stochastic order Testing against essential stochastic order Consider H 0 : α 0 (F, G) α, vs. H a : α 0 (F, G) > α (equivalently, H 0 : F st,α G vs. H a : F st,α G) Reject H 0 if n(α0 (F n, G n ) α) > K 1 β (α), K 1 β (α) = 1 β quantile of B(α) Theorem lim n sup P F,G ( n(α 0 (F n, G n ) α) > K 1 β (α)) (F,G) H 0 F 0 U(α, 1 + α), G 0 U(0, 1) = lim n P F 0,G 0 ( n(α 0 (F n, G n ) α) > K 1 β (α)) = β, Eustasio del Barrio Testing approximate stochastic order 20 / 37

46 Inference for approximate stochastic order Testing against essential stochastic order Nonasymptotic bounds Theorem If α 0 (F, G) < α and K 1 β (α) 0 then P F,G ( n(α 0 (F n, G n ) α) > K 1 β (α)) 2e n(α α0(f,g))2. If α 0 (F, G) > α then P F,G ( n(α 0 (F n, G n ) α) K 1 β (α)) e 2( n(α α 0(F,G)) K 1 β (α)) 2. Eustasio del Barrio Testing approximate stochastic order 21 / 37

47 Inference for approximate stochastic order Testing against essential stochastic order Nonasymptotic bounds Theorem If α 0 (F, G) < α and K 1 β (α) 0 then P F,G ( n(α 0 (F n, G n ) α) > K 1 β (α)) 2e n(α α0(f,g))2. If α 0 (F, G) > α then P F,G ( n(α 0 (F n, G n ) α) K 1 β (α)) e 2( n(α α 0(F,G)) K 1 β (α)) 2. Test is u.e.c. for H 0 : α 0 (F, G) α vs. H a : α 0 (F, G) > α if α < α < α Compute sample sizes to guarantee given power against fixed alternatives Eustasio del Barrio Testing approximate stochastic order 21 / 37

48 Inference for approximate stochastic order Testing for essential stochastic order Testing for essential stochastic order Consider H 0 : α 0 (F, G) α, vs. H a : α 0 (F, G) < α Eustasio del Barrio Testing approximate stochastic order 22 / 37

49 Inference for approximate stochastic order Testing for essential stochastic order Testing for essential stochastic order Consider H 0 : α 0 (F, G) α, vs. H a : α 0 (F, G) < α (equivalently, H 0 : F st,α G if α < α vs. H a : F st,α G for some α < α) Eustasio del Barrio Testing approximate stochastic order 22 / 37

50 Inference for approximate stochastic order Testing for essential stochastic order Testing for essential stochastic order Consider H 0 : α 0 (F, G) α, vs. H a : α 0 (F, G) < α (equivalently, H 0 : F st,α G if α < α vs. H a : F st,α G for some α < α) Reject H 0 if n(α0 (F n, G n ) α) < σ α Φ 1 (β), where σ 2 α = 1 α2 2, (assume β < 1 2 ) Eustasio del Barrio Testing approximate stochastic order 22 / 37

51 Inference for approximate stochastic order Testing for essential stochastic order Testing for essential stochastic order Consider H 0 : α 0 (F, G) α, vs. H a : α 0 (F, G) < α (equivalently, H 0 : F st,α G if α < α vs. H a : F st,α G for some α < α) Reject H 0 if n(α0 (F n, G n ) α) < σ α Φ 1 (β), where σ 2 α = 1 α2 2, (assume β < 1 2 ) Theorem lim n F 0 1 α 1+α 2 U(0, sup P F,G ( n(α 0 (F n, G n ) α) < σ α Φ 1 (β)) (F,G) H 0 = lim n P F 0,G 0 ( n(α 0 (F n, G n ) α) < σ α Φ 1 (β)) = β, 2 ) + 1+α 2 U( 1+α 2, 1 + α(1 α) 2 ), G 0 U(0, 1) Eustasio del Barrio Testing approximate stochastic order 22 / 37

52 Inference for approximate stochastic order Testing for essential stochastic order Nonasymptotic bounds Theorem If α 0 (F, G) > α then P F,G ( n(α 0 (F n, G n ) α) < σ α Φ 1 (β)) e 2n(α α0(f,g))2 If α 0 (F, G) < α and n(α α 0 (F, G)) 2 log 2 σ α Φ 1 (β), P F,G ( n(α 0 (F, G) α) σ α Φ 1 (β)) 2e ( σαφ 1 (β)+ n(α α 0(F,G))) 2 Eustasio del Barrio Testing approximate stochastic order 23 / 37

53 Inference for approximate stochastic order Testing for essential stochastic order Nonasymptotic bounds Theorem If α 0 (F, G) > α then P F,G ( n(α 0 (F n, G n ) α) < σ α Φ 1 (β)) e 2n(α α0(f,g))2 If α 0 (F, G) < α and n(α α 0 (F, G)) 2 log 2 σ α Φ 1 (β), P F,G ( n(α 0 (F, G) α) σ α Φ 1 (β)) 2e ( σαφ 1 (β)+ n(α α 0(F,G))) 2 Test is u.e.c. for H 0 : α 0 (F, G) α vs. H a : α 0 (F, G) < α if α < α < α Eustasio del Barrio Testing approximate stochastic order 23 / 37

54 Inference for approximate stochastic order Testing for essential stochastic order Nonasymptotic bounds Theorem If α 0 (F, G) > α then P F,G ( n(α 0 (F n, G n ) α) < σ α Φ 1 (β)) e 2n(α α0(f,g))2 If α 0 (F, G) < α and n(α α 0 (F, G)) 2 log 2 σ α Φ 1 (β), P F,G ( n(α 0 (F, G) α) σ α Φ 1 (β)) 2e ( σαφ 1 (β)+ n(α α 0(F,G))) 2 Test is u.e.c. for H 0 : α 0 (F, G) α vs. H a : α 0 (F, G) < α if α < α < α Compare to case H 0 : F st G vs. H a : F st G Eustasio del Barrio Testing approximate stochastic order 23 / 37

55 Inference for approximate stochastic order Testing for essential stochastic order Nonasymptotic bounds Theorem If α 0 (F, G) > α then P F,G ( n(α 0 (F n, G n ) α) < σ α Φ 1 (β)) e 2n(α α0(f,g))2 If α 0 (F, G) < α and n(α α 0 (F, G)) 2 log 2 σ α Φ 1 (β), P F,G ( n(α 0 (F, G) α) σ α Φ 1 (β)) 2e ( σαφ 1 (β)+ n(α α 0(F,G))) 2 Test is u.e.c. for H 0 : α 0 (F, G) α vs. H a : α 0 (F, G) < α if α < α < α Compare to case H 0 : F st G vs. H a : F st G Try to assess F st G up to α = 0.05 contamination; β = 0.05 Eustasio del Barrio Testing approximate stochastic order 23 / 37

56 Inference for approximate stochastic order Testing for essential stochastic order Nonasymptotic bounds Theorem If α 0 (F, G) > α then P F,G ( n(α 0 (F n, G n ) α) < σ α Φ 1 (β)) e 2n(α α0(f,g))2 If α 0 (F, G) < α and n(α α 0 (F, G)) 2 log 2 σ α Φ 1 (β), P F,G ( n(α 0 (F, G) α) σ α Φ 1 (β)) 2e ( σαφ 1 (β)+ n(α α 0(F,G))) 2 Test is u.e.c. for H 0 : α 0 (F, G) α vs. H a : α 0 (F, G) < α if α < α < α Compare to case H 0 : F st G vs. H a : F st G Try to assess F st G up to α = 0.05 contamination; β = 0.05 Want to detect alternatives with α 0 (F, G) 0.01 with power 0.9 Eustasio del Barrio Testing approximate stochastic order 23 / 37

57 Inference for approximate stochastic order Testing for essential stochastic order Nonasymptotic bounds Theorem If α 0 (F, G) > α then P F,G ( n(α 0 (F n, G n ) α) < σ α Φ 1 (β)) e 2n(α α0(f,g))2 If α 0 (F, G) < α and n(α α 0 (F, G)) 2 log 2 σ α Φ 1 (β), P F,G ( n(α 0 (F, G) α) σ α Φ 1 (β)) 2e ( σαφ 1 (β)+ n(α α 0(F,G))) 2 Test is u.e.c. for H 0 : α 0 (F, G) α vs. H a : α 0 (F, G) < α if α < α < α Compare to case H 0 : F st G vs. H a : F st G Try to assess F st G up to α = 0.05 contamination; β = 0.05 Want to detect alternatives with α 0 (F, G) 0.01 with power 0.9 Take n = 8143 Eustasio del Barrio Testing approximate stochastic order 23 / 37

58 Inference for approximate stochastic order Confidence bounds Confidence bounds Instead of testing for/against contaminated stochastic order, report upper/lower bounds for true contamination level, α 0 (F, G) For β < 1 2, α 0 (F n, G n ) nˆσ n Φ 1 (β) ˆσ 2 n = min t:fn(t) G n(t)=α 0(F n,g n)) σ 2 t, σ 2 t = t(1 t) + (t α 0 (F n, G n ))(1 t + α 0 (F n, G n )) is an upper bound with asymptotic confidence level at least 1 β Better use bias corrected α 0 (F n, G n ) BOOT α 0 (F n, G n ) nk 1 β (α 0 (F n, G n )) is a lower confidence bound for α 0 (F, G) with asymptotic confidence level 1 β Quantiles K 1 β (α 0 (F n, G n )) numerically approximated Eustasio del Barrio Testing approximate stochastic order 24 / 37

59 Inference for approximate stochastic order Paired sampling Dependent data Often X = pre-treatment, Y = post-treatment measurement (X, Y ) H with marginals F and G Has patient improved with treatment? F st G? As before, H 0 : F st G vs. H a : F st G not testable Consider instead H 0 : α 0 (F, G) α, vs. H a : α 0 (F, G) < α (X 1, Y 1 ),..., (X n, Y n ) i.i.d. random vectors with common joint d.f. H H(x, y) = C(F (x), G(y)), C copula α 0 (F, G) depends only on marginals; α 0 (F n, G n ) consistent estimator Distribution of α 0 (F n, G n ) does depend on C: n (α0 (F n, G n ) α 0 (F, G)) w sup B C (G(x), F (x)), x:g(x) F (x)=α 0(F,G) B C centered Gaussian on [0, 1] 2, covariance K C ((s, t), (s, t )) = s s + t t (s t)(s t ) C(t, s) C(t, s ) Eustasio del Barrio Testing approximate stochastic order 25 / 37

60 Inference for approximate stochastic order Paired sampling Testing for approximate stochastic order, dependent data Now α(1 α) Var(B C (t, t α)) 1 α 2 2 t 1+α 2, t [α, 1] equality for antimonotone coupling C(s, t) = (s + t 1) + K C,β (F, G) β-quantile of rhs in limit distribution; for β (0, 1 2 ) K C,β (F, G) (1 α 0 (F, G) 2 ) 1/2 Φ 1 (β) Similar to independent case, set σ 2 α = 1 α 2 ; reject α 0 (F, G) α if n(α0 (F n, G n ) α) < σ α Φ 1 (β) Eustasio del Barrio Testing approximate stochastic order 26 / 37

61 Inference for approximate stochastic order Paired sampling Testing for approximate stochastic order, dependent data As before, test uniformly asymptotically consistent: lim n sup [ P H n(α0 (F n, G n ) α) < σ α Φ 1 (β) ] H H 0 = lim P [ n H n(α0 (F n, G n ) α) < σ π0 Φ 1 (β) ] = β, H joint d.f. with marginals F 1 α 1+α 2 U(0, 2 ) + 1+α 1+α 2 U( 2, 1 + α(1 α) 2 ), G U(0, 1) and copula C(s, t) = (s + t 1) +. Eustasio del Barrio Testing approximate stochastic order 27 / 37

62 Inference for approximate stochastic order Paired sampling Nonasymptotic bounds: if α 0 (F, G) > α then P H [ n(α0 (F n, G n ) α < σ α Φ 1 (β) ] e n 2 (α α0(fn,gn)2, If α 0 (F, G) < α and n(α α 0 (F, G)) 2 2 log 2 σ α Φ 1 (β), P H [ n(α0 (F n, G n ) α) σ α Φ 1 (β) ] 2e n 2 [(α α0(f,g))+ 2 σα n Φ 1 (β)] 2. Eustasio del Barrio Testing approximate stochastic order 28 / 37

63 Inference for approximate stochastic order Paired sampling Nonasymptotic bounds: if α 0 (F, G) > α then P H [ n(α0 (F n, G n ) α < σ α Φ 1 (β) ] e n 2 (α α0(fn,gn)2, If α 0 (F, G) < α and n(α α 0 (F, G)) 2 2 log 2 σ α Φ 1 (β), P H [ n(α0 (F n, G n ) α) σ α Φ 1 (β) ] 2e n 2 [(α α0(f,g))+ 2 σα n Φ 1 (β)] 2. Independent vs. dependent setup In independent setup rejection of H 0 : α 0 (F, G) α when n(α0 (F n, G n ) α) < σα 2 Φ 1 (β). Under dependence, the extra 2 factor allows to control uniformly type I error probability Eustasio del Barrio Testing approximate stochastic order 28 / 37

64 Implementation, simulation & data example Implementation issues Testing for essential stochastic order: finite sample performance Consider again H 0 : α 0 (F, G) α, vs. H a : α 0 (F, G) < α Eustasio del Barrio Testing approximate stochastic order 29 / 37

65 Implementation, simulation & data example Implementation issues Testing for essential stochastic order: finite sample performance Consider again H 0 : α 0 (F, G) α, vs. H a : α 0 (F, G) < α Rejection of H 0 if n(α0 (F n, G n ) α) < σ α Φ 1 (β), σ 2 α = 1 α2 2, asympt. of level β; type I-type II error probs. exponentially 0 Eustasio del Barrio Testing approximate stochastic order 29 / 37

66 Implementation, simulation & data example Implementation issues Testing for essential stochastic order: finite sample performance Consider again H 0 : α 0 (F, G) α, vs. H a : α 0 (F, G) < α Rejection of H 0 if n(α0 (F n, G n ) α) < σ α Φ 1 (β), σ 2 α = 1 α2 2, asympt. of level β; type I-type II error probs. exponentially 0 σ α from worst case choice; possible improvement from estimated ˆσ n A more important improvement: E(α 0 (F n, G n )) α 0 (F, G); estimate bias by bias BOOT (α 0 (F n, G n )) := E (α 0 (F n, G n)) α 0 (F n, G n ) Eustasio del Barrio Testing approximate stochastic order 29 / 37

67 Implementation, simulation & data example Implementation issues Testing for essential stochastic order: finite sample performance Consider again H 0 : α 0 (F, G) α, vs. H a : α 0 (F, G) < α Rejection of H 0 if n(α0 (F n, G n ) α) < σ α Φ 1 (β), σ 2 α = 1 α2 2, asympt. of level β; type I-type II error probs. exponentially 0 σ α from worst case choice; possible improvement from estimated ˆσ n A more important improvement: E(α 0 (F n, G n )) α 0 (F, G); estimate bias by bias BOOT (α 0 (F n, G n )) := E (α 0 (F n, G n)) α 0 (F n, G n ) Define ˆα n,boot := α 0 (F n, G n ) bias BOOT (α 0 (F n, G n )). Eustasio del Barrio Testing approximate stochastic order 29 / 37

68 Implementation, simulation & data example Implementation issues Testing for essential stochastic order: finite sample performance Consider again H 0 : α 0 (F, G) α, vs. H a : α 0 (F, G) < α Rejection of H 0 if n(α0 (F n, G n ) α) < σ α Φ 1 (β), σ 2 α = 1 α2 2, asympt. of level β; type I-type II error probs. exponentially 0 σ α from worst case choice; possible improvement from estimated ˆσ n A more important improvement: E(α 0 (F n, G n )) α 0 (F, G); estimate bias by bias BOOT (α 0 (F n, G n )) := E (α 0 (F n, G n)) α 0 (F n, G n ) Define ˆα n,boot := α 0 (F n, G n ) bias BOOT (α 0 (F n, G n )). Reject H 0 if n(ˆαn,boot α) < ˆσ n Φ 1 (β), Eustasio del Barrio Testing approximate stochastic order 29 / 37

69 Implementation, simulation & data example Implementation issues Testing for essential stochastic order: finite sample performance Consider again H 0 : α 0 (F, G) α, vs. H a : α 0 (F, G) < α Rejection of H 0 if n(α0 (F n, G n ) α) < σ α Φ 1 (β), σ 2 α = 1 α2 2, asympt. of level β; type I-type II error probs. exponentially 0 σ α from worst case choice; possible improvement from estimated ˆσ n A more important improvement: E(α 0 (F n, G n )) α 0 (F, G); estimate bias by bias BOOT (α 0 (F n, G n )) := E (α 0 (F n, G n)) α 0 (F n, G n ) Define ˆα n,boot := α 0 (F n, G n ) bias BOOT (α 0 (F n, G n )). Reject H 0 if n(ˆαn,boot α) < ˆσ n Φ 1 (β), Test asympt. of level β Eustasio del Barrio Testing approximate stochastic order 29 / 37

70 Implementation, simulation & data example Simulation setup F α,a U(a, 1 + a); F α,b 1 α 1+α 2 U(0, G U(0, 1) 2 ) + 1+α 2 U( 1+α 2, 1 + α(1 α) 2 ) F α,a F α,b F α,a, G worst choice in test against essential s.o. F α,b, G worst choice in test for essential s.o. Eustasio del Barrio Testing approximate stochastic order 30 / 37

71 Implementation, simulation & data example Simulation results Testing for essential stochastic order Table : Observed rejection frequencies. H 0 : α 0 (F, G) α vs. H a : α 0 (F, G) < α G = U(0, 1), m = n; reject if n(α 0 (F n, G n ) α) < σ α Φ 1 (0.05) α n F 0.1,a F 0.1,b F 0.05,a F 0.05,b F 0.01,a F 0.01,b F Nonasymptotic estimate n = 8143 Eustasio del Barrio Testing approximate stochastic order 31 / 37

72 Implementation, simulation & data example Simulation results Testing for essential stochastic order Table : Observed rejection frequencies. H 0 : α 0 (F, G) α vs. H a : α 0 (F, G) < α G = U(0, 1), m = n; reject if n(ˆα n,boot α) < ˆσ n Φ 1 (0.05) α n F 0.1,a F 0.1,b F 0.05,a F 0.05,b F 0.01,a F 0.01,b F Eustasio del Barrio Testing approximate stochastic order 32 / 37

73 Implementation, simulation & data example Simulation results Testing against essential stochastic order Table : Observed rejection frequencies. H 0 : α 0 (F, G) α vs. H a : α 0 (F, G) > α G = U(0, 1), m = n; reject if n(α 0 (F n, G n ) α) > K 0.95 (α) α n F 0 F 0.01,b F 0.01,a F 0.05,b F 0.05,a F 0.1,b F 0.1,a Eustasio del Barrio Testing approximate stochastic order 33 / 37

74 Implementation, simulation & data example Simulation results Testing for essential stochastic order, dependent case Table : Observed rejection frequencies. H 0 : α 0 (F, G) α vs. H a : α 0 (F, G) < α G = U(0, 1), m = n; reject if n(ˆα 0 (F n, G n α) < ˆσ n Φ 1 (0.05) α n H 0.1,a H 0.1,b H 0.05,a H 0.05,b H 0.01,a H 0.01,b H H π,a independent marginals U(π, 1 + π), U(0, 1); H 0 = H 0,a H π,b marginals F 1 π 1+π 2 U(0, 2 ) + 1+π 1+π 2 U( 2, 1 + π(1 π) 2 ), G U(0, 1), copula C(s, t) = (s + t 1) + Eustasio del Barrio Testing approximate stochastic order 34 / 37

75 Implementation, simulation & data example Case study Data: National Health and Nutrition Examination Survey Evolution with age of the heights of boys and girls Sample sizes by age (boys, top) π(fn,gm) = π(gm,fn) = EDF's Gm Age 10 Boys Fn Age 10 Girls heights Eustasio del Barrio Testing approximate stochastic order 35 / 37

76 Implementation, simulation & data example Case study 95%-Upper bounds by age for α 0 (F a, G a ) (top row) and α 0 (G a, F a ) (bottom) Upper 95% confidence bounds for the stochastic dominance levels age Statistical evidence that girls are taller than boys at Eustasio del Barrio Testing approximate stochastic order 36 / 37

77 Conclusions Conclusions Trimmed stochastic order models capture adequately deviations from exact stochastic order Provided valid inference models/methods Valid testing procedures with controlled error probabilities Nonasymptotic bounds; uniformly exponentially consistent tests Good finite sample performance through bootstrap correction Eustasio del Barrio Testing approximate stochastic order 37 / 37

78 Conclusions Conclusions Trimmed stochastic order models capture adequately deviations from exact stochastic order Provided valid inference models/methods Valid testing procedures with controlled error probabilities Nonasymptotic bounds; uniformly exponentially consistent tests Good finite sample performance through bootstrap correction Thanks for your attention! Eustasio del Barrio Testing approximate stochastic order 37 / 37