Inference for Empirical Wasserstein Distances on Finite Spaces: Supplementary Material

Transcription

1 Inference for Emirical Wasserstein Distances on Finite Saces: Sulementary Material Max Sommerfeld Axel Munk Keywords: otimal transort, Wasserstein distance, central limit theorem, directional Hadamard derivative, bootstra, hyothesis testing AMS 2010 Subject Classification: Primary: 62G20, 62G10, 65C60 Secondary: 90C08, 90C31 A An alternative reresentation of the limiting distribution We give a second reresentation of the limiting distribution under the alternative r s. The random art of the limiting distribution (8) is the linear rogram max (u,v) Φ (r,s) λ G, u + 1 λ H, v. With the reresentation (3) of Φ (r, s) we obtain the dual linear rogram min zw (r, s) + s.t. x,x X w 0, z R w x,x + zr x = G x x X w x,x + zs x = H x x X w x,x d (x, x ) Note that the constraints can only be satisfied if both λg zr and 1 λh zs have only non-negative entries and z 0. In this case the Felix Bernstein Institute for Mathematical Statistics in the Biosciences and Institute for Mathematical Stochastics, University of Göttingen, Goldschmidtstraße 7, Göttingen Max Planck Institute for Biohysical Chemistry, Am Faßberg 11, Göttingen 1

2 second term in the objective function is clearly minimized by zw, with w an otimal transort lan between these two measures r λg/z and s 1 λh/z and the second term of the objective function is equal to zw (r λg/z, s 1 λh/z). To write this more comactly let us slightly extend our notation. For r, s R X with x r x = x s x = 1 let W W (r, s) if r, s 0; (r, s) = else. With this we can thus write the random variable in the limiting distribution (8) as the one-dimensional non-linear otimization roblem (1) 1 W 1 (r, s) min z W (r + λg/z, s + } 1 λh/z) W (r, s). z 0 B Bootstra In this section we discuss the bootstra for the Wasserstein distance. In addressing the usual measurability issues that arise in the formulation of consistency for the bootstra, we follow Van der Vaart and Wellner (1996). We denote by ˆr n and ŝ m some bootstraed versions of ˆr n and ŝ m. More recisely, let ˆr n a measurable function of X 1,..., X n and random weights W 1,..., W n, indeendent of the data and analogously for ŝ m. This setting is general enough to include many common bootstraing schemes. We say that, with the assumtions and notation of Theorem 1, the bootstra is consistent if the limiting distribution of ρ n,m (ˆr n, ŝ m ) (r, s)} ( λg, 1 λh) is consistently estimated by the law of ρ n,m (ˆr n, ŝ m) (ˆr n, ŝ m )}. To make this recise, we define for A R d, with d N, the set of bounded Lischitz-1 functions } BL 1 (A) = f : A R : su f(x) 1, f(x 1 ) f(x 2 ) x 1 x 2, x A 2

3 where is the Euclidean norm. (ˆr n, ŝ m) are consistent if We say that the bootstra versions (2) su f BL 1 (R X R X ) converges to zero in robability. E [f(ρ n,m (ˆr n, ŝ m) (ˆr n, ŝ m )}) X 1,..., X n, Y 1,..., Y m ] [ E f(( λg, ] 1 λh)) Bootstra for directionally differentiable functions The most straightforward way to bootstra W (ˆr n, ŝ m ) is to simly lug-in ˆr n and ŝ m. That is, trying to aroximate the limiting distribution of ρ n,m W (ˆr n, ŝ m ) W (r, s)} by the law of (3) ρ n,m W (ˆr n, ŝ m) W (ˆr n, ŝ m ) } conditional on the data. While for functions that are Hadamard differentiable this aroach yields a consistent bootstra (e.g. Gill et al. (1989); Van der Vaart and Wellner (1996)), it has been ointed out by Dümbgen (1993) and more recently by Fang and Santos (2014) that this is in general not true for functions that are only directionally Hadamard differentiable. In articular the lug-in aroach fails for the Wasserstein distance. For the Wasserstein distance there are two alternatives. First, Dümbgen (1993) already ointed out that re-samling fewer than n (or m, resectively) observations yield a consistent bootstra. Second, Fang and Santos (2014) roose to lug-in ρ n,m (ˆr n, ŝ m) (ˆr n, ŝ m )} into the derivative of the function. Recall from Section 2 that (4) φ : R N R N R, φ (h 1, h 2 ) = max u, h 2 h 1 u Φ is the directional Hadamard derivative of (r, s) W (r, s) at r = s. With this notation, the following Theorem summarizes the imlications of the results of Dümbgen (1993) and Fang and Santos (2014) for the Wasserstein distance. Theorem 1 (Pro. 2 of Dümbgen (1993) and Thms. 3.2 and 3.3 of Fang and Santos (2014)). Under the assumtions of Theorem 1 let ˆr n and ŝ m be consistent bootstra versions of ˆr n and ŝ m, that is, (2) converges to zero in robability. Then, 3

4 1. the lug-in bootstra (3) is not consistent, that is, su E [ f(ρ n,m W (ˆr n, ŝ m) W (ˆr n, ŝ m ) } ] ) X 1,... X n, Y 1,..., Y m f BL 1 (R) does not converge to zero in robability. E[f(ρ n,m W (ˆr n, ŝ m ) W (r, s) } )] 2. Under the null hyothesis r = s, the derivative lug-in (5) φ (ρ n,m (ˆr n, ŝ m) (ˆr n, ŝ m )}) is consistent, that is su E [f(φ (ρ n,m (ˆr n, ŝ m) (ˆr n, ŝ m )})) X 1,..., X n, Y 1,..., Y m ] f BL 1 (R) E [ f ( ρ n,m W (ˆr n, ŝ m ) W (r, s) })] converges to zero in robability. C Proofs C.1 Proof of Theorem 4 By (Gal et al., 1997, Ch. 3, Thm. 3.1) the function (r, s) W (r, s) is directionally differentiable with derivative (11) in the sense of Gâteaux, that is, the limit (9) exists for a fixed h and not a sequence h n h (see e.g. Shairo (1990)). To see that this is also a directional derivative in the Hadamard sense (9) it suffices (Shairo, 1990, Pro. 3.5) to show that (r, s) W (r, s) is locally Lischitz. That is, we need to show that for r, r, s, s P X W (r, s) W (r, s ) C (r, s) (r, s ), for some constant C > 0 and some (and hence all) norm on R N R N. Exloiting symmetry, it suffices to show that W (r, s) W (r, s ) C s s for some constant C > 0 and some norm. To this end, we emloy an argument similar to that used to rove the triangle inequality for the Wasserstein distance (see e.g. (Villani, 2008,. 94)). Indeed, by the gluing Lemma (Villani, 2008, Ch. 1) there exist random variables X 1, X 2, X 3 with 4

5 marginal distributions r, s and s, resectively, such that E[d (X 1, X 3 )] = W (r, s ) and E[d(X 2, X 3 )] = W 1 (s, s ). Then, since (X 1, X 2 ) has marginals r and s, we have W (r, s) W (r, s ) E [d (X 1, X 2 ) d (X 1, X 3 )] diam(x ) 1 E [ d(x 1, X 2 ) d(x 1, X 3 ) ] diam(x ) 1 E [d(x 2, X 3 )] = diam(x ) 1 W 1 (s, s ) diam(x ) s s 1, where the last inequality follows from (Villani, 2008, Thm. 6.15). This comletes the roof. C.2 Proof of Theorem 5 Simlify the set of dual solutions Φ As a first ste, we rewrite the set of dual solutions Φ given in (3) in our tree notation as (6) Φ = u R X : u x u x d T (x, x ), x, x X }. The key observation is that in the condition u x u x d T (x, x ) we do not need to consider all airs of vertices x, x X, but only those which are joined by an edge. To see this, assume that only the latter condition holds. Let x, x X arbitrary and x = x 1,..., x l = x the sequence of vertices defining the unique ath joining x and x, such that (x j, x j+1 ) E for j = 1,..., n 1. Then n 1 n 1 u x u x = (u xj u xj+1 ) d T (x j, x j+1 ) d T (x, x ), j=1 such that the condition is satisfied for all x, x X. Noting that if two vertices are joined by an edge than one has to be the arent of the other, we can write the set of dual solutions as (7) Φ = u R X : u x u arent(x) d T (x, arent(x)), x X }. Rewrite the target function We define linear oerators S T, D T : R X R X by v x v (D T v) x = arent(x) x root(t ) v root(t ) x = root(t )., (S T u) x = u x. 5 j=1 x children(x)

6 Lemma 1. For u, v R X we have u, v = S T u, D T v. Proof. We comute S T u, D T v = x X(S T u) x (D T v) x = (v x v arent(x) )u x x X \root(t )} x children(x) + = x X x children(root(t )) x children(x) v x u x v root(t ) u x x X \root(t )} x children(x) = x X u x v x, v arent(x) u x which roves the Lemma. To see how the last line follows let children 1 (x) be the set of immediate redecessors of x, that is children of x that are connected to x by an edge. Then we can write the second term in the second to last line above as v y u x x X \root(t )} x children(x) v arent(x) u x = y X = y X and the claim follows. x children 1 (y) x children(x) x children(y)\y} v y u x If u Φ, as given in (7), we have for x root(t ) that (D T u) x = u x u arent(x) d T (x, arent(x)). With these two observations and Lemma 1, we get for G N (0, Σ(r)) and u Φ that (8) G, u = S T G, D T u (S T G) x d T (x, arent(x)). root(t ) x X Therefore, max u Φ G, u is bounded by root(t ) x X (S T G) x d T (x, arent(x)). Since D T is an isomorhism, we can define a vector v R X by (D T v) x = sgn ((S T G) x )d T (x, arent(x)). 6

7 From (7) we see that v Φ and Lemma 1 shows that G, v attains the uer bound in (8). This concludes the roof. C.3 Proof of Corollary 1 In order to use Theorem 5 we define the tree T with vertices x 1,..., x N }, edges E = (x j, x j+1 ), j = 1,..., N 1} and root(t ) = x N. Then, if G N (0, Σ(r)), we have that (S T G) j } j=1,...,n is a Gaussian vector such that for i j cov((s T G) i, (S T G) j ) = k i l j = r i k i l i r k r l k i i<l j E [G k G l ] = k i r k (1 r k ) k i l j k l r k r l r k r l = r i r 2 i r i ( r j r i )) = r i r i r j. Therefore, we have that for a standard Brownian bridge B S T G (B( r 1 ),..., B( r N )). Together with d(x j, arent(x j )) = (x j+1 x j ) 2, and (15) this roves the Corollary. References Dümbgen, L. (1993). On nondifferentiable functions and the bootstra. Probability Theory and Related Fields, 95(1): Fang, Z. and Santos, A. (2014). Inference on directionally differentiable functions. arxiv: Gal, T., Greenberg, H. J., and Hillier, F. S., editors (1997). Advances in Sensitivity Analysis and Parametric Programming, volume 6 of International Series in Oerations Research & Management Science. Sringer. Gill, R. D., Wellner, J. A., and Præstgaard, J. (1989). Non- and semi-arametric maximum likelihood estimators and the von Mises method (Part 1) [with discussion and rely]. Scandinavian Journal of Statistics, 16(2): Shairo, A. (1990). On concets of directional differentiability. Journal of otimization theory and alications, 66(3): Van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence. Sringer. Villani, C. (2008). Otimal Transort: Old and New. Sringer. 7