Talagrand's Inequality

Transcription

1 Talagrand's Inequality Aukosh Jagannath October 2, 2013 Abstract Notes on Talagrand's inequality. This material is a mixture from Talagrand's two famous papers and Ledoux. 1 Introduction The goal of this note is to discuss Talagrand's inequality which is, in a sense we will understand, a generalization of the Isoperimetric inequality to the setting of products of probability spaces. Theorem. (Talagrand's Inequality) Let D A (x) be Talagrand's convex distance functional from x Ω N to A Ω N. Let A t = {D A (x) t} be the t neighborhood if x in this distance.then for all product measures, P(A c t) e r2 The power of this inequality is that, rstly, it lets us go both ways, controlling either A or A t, and, most importantly, it is dimension free. As is usual with concentration, this type of inequality leads to a version of mean deviation for lip functions. The diculty is understanding what lip means. In this document we assume that the underlying space is nice enough that the hamming distance is measurable. 2 Approximation by one point Before we get to Talagrand's inequality, we rst study the approximation by one point inequality. The idea of this inequality is that we use a single point in our set A to study distance to A. First recall the hamming distance Denition 1. the (unnormalized) hamming distance between two points in a product space is d H (x, y) = N i=1 1 xi y i Our goal is to show that w.r.t the hamming distance, product measures have gaussian concentration. More precisely, Theorem. For any product measure on Ω N, with respect to the hamming distance P(A t ) e r2 N One might note that this inequality appears to lend itself toward classical techniques. In fact you get something close in this setting, but it is not quite the same strength. 1

2 Proof. (The Martingale Method) Let the metric space be nite. The trick in this proof is to note that if we let F i = σ(x k i ) and, as usual let d i = Ed H (x, A) F i Ed H (x, A) F i 1, then so by the bounded dierences argument, we know d i 2 P( d(x, A) Ed(x, A) r) e 2r2 N setting r = E(d(x, A)), we get that Ed(x, A) 2 log 2N for 1/2 so that P(d(x, A) t + 2N log 2) e r2 2N To get the full strength we use talagrand's novel idea. In the martingale method, we control probabilities using a bottom up approach: we make the set bigger and bigger and control at each stage. Instead we will use a top down approach. To control the probabilities we step down. Theorem 2. For any product measure on Ω N, Ee λd(x,a) 1 e 1 λ 2 N Lemma 3. Let g be any integrable measurable function. then ) e t 1 g dµ gdµ e t2 / ( Proof. note that e t 1 g 1 + et (1 g). this means that, if we let a = gdµ, LHS = a(1 + e t (1 a)) = f(a) thinking of this as a function of a and maximizing gives f(a) cosh( t 2 )2. now cosh(x) e x2 /2 by a power series expansion comparison. This is an unusual lemma, but why you'd at least try to guess it will be come clear. Proof. (Induct on N) For N = 1, use the above lemma g = 1 A.Now assume the claim for N. Fix A Ω N+1 and let A(ω) = {x Ω N : (x, ω) A} be its ω-slice. Similarly let B be its projection on to the rst N coordinates. The key step is in controlling d((x, ω), A) by d(x, A(ω)) and d(x, B). Indeed note that d((x, ω), A) d(x, B) + 1 since the point can just hop into B and then ip the last coordinate as necessary. Similarly, we have that d((x, ω), A) d(x, A(ω)). Thus simultaneously and Ee λd((x,ω),a) Ee λd(x,a(ω)) 1 P(A(ω)) e λ eλ Ee λd((x,ω),a) Ee λd(x,b) e λ P(B) e λ 2 N 2 N 2

3 so that LHS e λ 2 N ( e t P(B) ) P(B) P(A(ω)) Integrating in ω, using the above lemma, and fubini gives as desired. Ee λd((x,ω),a) e λ2 (N+1) 1 To notice the power of this inequality switch to the normalized hamming metric, then this inequality says that if, say ɛ, then P(d H (x, A) r) 1 Nr2 e, ɛ or in other words, the chance that point doesnt share most of its coordinates with a point in y is essentially nothing in high dimensions. Example. consider coinips as the haar measure on Σ N = {0, 1} N. then if we prescribe, say, the rst k terms in the string, call this rst term l, P(the fraction of coordinates x diers from strings that start with l r) e N r k N log 2 so that even if we prescribe k = o(n)points there is essentially no chance that a sequence of coinips is dierent from one that starts in k 0's in more than 99% of its coordinates is essentially nothing. 3 Convex distance and the Convex Hull approximation Talagrand's next idea is even more revolutionary. Ideally, we would have a dimension free way to control probabilities. To do this we encode the problem in to an identied space, namely ( [0, 1] N, 2 ) and use the geometric properties of this space to control this. In particular, consider weighted hamming metrics, which can be thought of as d a (x, y) = N a i 1 xi y i i=1 it is clear that we can use the approximation by 1 point technique to control these guys as well. Talagrand's idea is then to control all of them simultaneously. To do this he introduces a new kind of distance Denition 5. (Talagrands Convex distance) let A Ω N Talagrand convex distance functional is and let x be a point in this space. Then the D A (x) := sup d a (x, A) a 2 1 In order to understand the geometric content of this functional, we view it as related to a dierent space. Firstly, let h(x, y) = (1 xi y i ) i N For any set A and point x as above, we let U A(x) = { (s i ) i N [0, 1] N : y(s) : s i = 0 = x i = y i } = { s [0, 1] N : y(s) : d(x, y(s)) = (a, s) } = { s [0, 1] N : y(s) : h(x, y) = s } 3

4 where a is the vector of all 1's. Thus we've encoded the distance from x to a by the distance from these vectors to the origin. In particular d H (x, A) = d a (x, A) = s U A (x) s 2 = h(x, y) 2 y A Talagrand's idea is to consider distance to A in a more l 2 way. In particular, if we let V A (x) be the convex hull of U A (x), then we'd like to measure the distance to A as s 2. s V A (x) To do this, we need to add a slightly dierent structure, which will allow for making computations easier. Denition 6. We say that a point y vouches or witnesses s if i, Let and let V A (x) be the convex hull of U A (x) Proposition 7. We have that and h(x, y) i s i U A (x) = {s [0, 1] N : y : h(x, y) i s i } = {s : y that vouches for s} U A(x) U A (x) d(0, V A(x)) = d(0, V A (x)) Proof. To see the rst containment is clear by denition. To see the second, note that for a point s V A (x), we have that {t k }, {s k }, {y k } : t k = 1, t k 0, s k U A (x), and y k vouches for s k such that s = t k s k where s k U A. This means that s i = t k s k i t k h(x, y k ) i = v i for some v V A (x), so d(s, 0) d(v, 0), but by the above, clearly V A V A With this, we then see that we can express Talagrand's convex distance in terms of a convex set. That is Proposition 8. With everything as above, Proof. First, note that D A (x) = y V A y 2 = d l2 (0, V A ) D A (x) = sup a 1 = sup a = sup a = sup a d a(x, y) y A interchainging the sup and the gives us D A (x) d(0, V A ). We can go both ways by the minimax theorem. Instead we nish up as follows. let z achieve d(0, V A ). then y V A, the convex combo satises expanding shows that z (y z) 0 letting a = thus inng over y and suping over z gives the result. s U A V A (x) V A (x) (a, s) z + t(y z) 2 z 2 z z, we get that (a, y) (a, z) = d(0, V A (x))

5 Talagrand's inequality Now were are ready to set up Talagrand's inequality. As before, we'll need an extra Lemma. Theorem 9. On any product space, Talagrand's convex distance function satises the exponential estimate and thus concentration estimate Ee 1 D2 A (x) 1 Proof. Again by induction. For N = 1, note that P(D A (x) t) 1 e t 2 Ee 1 D A(x) 2 = Ee 1 1 x / A = a + (1 a)e 1 where a =. Multiplying by a it suces to show taht this function f(a) 1 for a [0, 1], but f(a) is increasing in that region and f(a) = 1 so we get the inequality. Assume for N. now as before, we let A(ω) and B be projections. and let U and V be as above. The key point is that s U A(ω) (x) = (s, 0) U A ((x, ω)) that is if y such that y vouches for s and (y, ω) A then (y, ω) vouches for (s, 0). Furthermore t U B (x) = (t, 1) U A ((x, ω)) if there is some (y, ω ) A such thaty vouches for t, then (y, ω) vouches for (t, 1). s U A(ω) (x), t U B (x), θ(s, 0) + (1 θ)(t, 1) V A ((x, ω)). so that This means that D A ((x, ω)) 2 = d(0, V A (x, ω)) 2 θ(s, 0) + (1 θ)(t, 1) 2 2 θ s (1 θ) t (1 θ) 2 where the last inequality follows by innerproducts and then convexity of squaring the norm. This means that D A ((x, ω)) 2 θd A(ω) (x) 2 + (1 θ)d B ((x, ω)) 2 + (1 θ) 2 so that now Ee 1 D A((x,ω)) 2 e 1 (1 θ)2 Ee θd A(ω)(x) 2 +(1 θ)d B (x) 2 e 1 (1 θ)2 ( Ee 1 D A(ω)(x) 2) θ ( Ee 1 D(x)2) (1 θ) e (1 θ)2 / 1 P(A(ω)) θ 1 P(B) (1 θ) ) θ = 1 ( 2 P(B) e(1 θ) / P(B) P(A(ω)) / θ e(1 θ)2 r θ 2 r 5

6 so that if we optimize the above in θ,we get integrating in ω, we get Ee 1 D A((x,ω)) 2 1 P(A(ω)) (2 ) P(B) P(B) Ee 1 D A((x,ω)) 2 1 (2 P(B) P(B) ) = 1 (a(2 a)) 1 5 Tools for using talagrands inequality As before we get lipschitz inequalites, once we give an appropriate denition of lipschitz. In particular Denition 10. a function F is D A (x)-lipschitz if x, a(x) such that F (x) F (y) + d a (x, y) Proposition 11. Let F be D(x)-lip. Then if m is a median of F, P( F m r) 2e r 2 Proof. For A = {F m}, we see that F (x) m + D A (x) so that if m = m F and then m = m F r, we get the result. P(F m)p(f m + r) e r2 / Another useful tool is to notice that Talagrand's inequality lends itself toward studying convexity properties. In particular we have Proposition 12. Let A [0, 1] N and P be a product measure on [0, 1], and let B = hull(a). Then d B D A and so Ee d2 B / 1 and P(d B (x) t) 1 e r 2 Proof. Fix an x. then for y B, (θ k ) k [N] P([N]), y k A such that so x y 2 2 = y = θ k y k ( N ( θ k xi yi k ) ) 2 n=1 k ( ) 2 n = n k θ k 1 xi y k i ( ) 2 θ k h(x, y k ) i k 6

7 now, just as numbers, k θ k 1 ak 0 k θ k a k k θ k 1 ak 0 when a k [ 1, 1]. Note that s = k θ kh(x, y k ) V A (x) by denition, and in fact this exhausts the latter set. This means that inmizing over y on the left means inmizing over s V A (x) on the right, so that as desired. d(x, B) s = d(0, V A(x)) = D A (x) s V A (x) Proposition 13. In particular, for all convex Lipschitz functions on [0, 1], where M is a median of f. P( f M f r) 2e r 2 f lip Proof. Two proofs. rstly, noting that f is convex and one lipschitz implies that x, y [0, 1] f(x) f(y) + (x i y i ) i f f(y) + d a (x, y) for a = ( i f ), so its D-lipschitz. and we can apply the above result. The second proof is that since f is convex, if we let A = {f m} where m is a median, we get that the latter set is convex. furthermore if d(x, A) r, then so f(x) m + r P(A c r) e r2 / by the above proposition (where A r is the euclidean dilate). 6 Applications 6.1 Bounded random matrices Suppose that you have a Wigner ensemble with bounded entries. Then up to translation and a dilation, we can think of the probability measure as a product measure on the cube [0, 1] n(n+1) 2. Now recall that the map to the eigenvalues is lipschitz by the Homan-Wielandt lemma, furthermore the map to the trace preserves lipschitz-ness and convexity, so if f is convex and lipschitz, trf(m) is convex ad lipschitz in (hermitian) M. Thus using the arguments above we have gaussian concentration about the median (and thus a mean and gaussian concentration about the mean) 6.2 Ulam's Problem Let X 1,..., X N be i.i.d U[0, 1] random variables, and let L N (x 1,..., x N ) be the length of the longest increasing subsequence of that string. Note that L N (X 1,..., X N ) = d L N (σ) where σ is uniformly distributed on the symmetric group S N, which is how this problem is sometimes posed. The question is to study the properties of L N. In this section we will show that L N concentrates about its medians. In particular, 7

8 Theorem 1. Let L N be as above. Then if we let M be the median of L N and P(L N (x) M r) e r2 M P(L N (x) M + r) 2e r2 (M+r) M is known to be of order NThe proof comes from a fairly clever application of Talagrand's inequality. One might hope to use simpler concentration arguments, noting, for example, that the symmetric group with the hamming metric admits a concenration property. The problm is taht L N is not lipschitz on that space (with any reasonable constant, it is for example clearly N 2 -lipschitz, but that gives the the bound which produces the wrong uctuations. P(L N (x) M + r) e 1 N r2 Lemma 15. Let A(a) = {L n a}, then we have that Furthermore on the set a L N (x) D A (x) L N (x) L N a + v = D A (x) v a + v Proof. Let L N (x) = b.then we know that there is an I [n] with I = b such that (x i ) i I is an increasing subsequence. Let J(x, y) = {i I : x i y i }. Note then that (x i ) i I\J is an increasing subsequence of y. Thus a I\J I J, and thus a I y J(x, y) If we let ã be such that ã i = 1 i I, we have that dã(x, y) = J(x, y) = (a, h(x, y)), renormalizing so that a = ã/ b, we have bdã(x, y) = d a (x, y). this means that J(x, y) = d a(x, A) bd A (x) y applying in this inequality then we get that a I I D A (x) which gives us the rst inequality. To get the second note that the function f(u) = u a u is increasing for u 0. This means that if we re arrange the above, if L N (x) a + v, as desired With this we can then establish the theorem D A (x) L N(x) a LN (x) v a + v Proof. Let M be the median of L N, and let a = M in the above, we get that by direct application of Talagrand's inequality, r P(D A (x) ) e r2 (M+v) M + r as desired. To get the other bound let a = M v and v = v, then A(a) = {L N (x) M v}, and as desired. P(D A (x) v M ) e r2 M Remark 16. This shows that the uctuations of L N are at most M 8