Large deviations for random projections of l p balls

Transcription

1 1/32 Large deviations for random projections of l p balls Nina Gantert CRM, september 5, 2016

2 Goal: Understanding random projections of high-dimensional convex sets. 2/32

3 2/32 Outline Goal: Understanding random projections of high-dimensional convex sets. 1 Motivation and statement of the question 2 A toy example: Cramér s Theorem for randomly weighted sums 3 Large deviations for the dependent case 4 A key ingredient of the proofs

4 Motivation and statement of the question 3/32 The talk is based on joint work with Steven Soojin Kim and Kavita Ramanan, both at Brown University. Cramér s Theorem is atypical, Springer s AWM Series: Advances in the Mathematical Sciences. Large deviations for random projections of l p balls, on arxiv.

5 Motivation and statement of the question 4/32 Let X (n) = (X 1, X 2,..., X n ) be a sample of n i.i.d. real-valued random variables with finite expectation m and finite variance. Then the law of large numbers and the central limit theorem hold, and Cramér s theorem characterizes the exponential rate of decay of the probability of a deviation : for x > m, ( ) P 1 n n X i x i=1 e ni (x). where [ I (x) = sup λx log E(e λx 1 ) ]. λ R

6 Motivation and statement of the question 5/32 Example If the X i have a standard normal law, m = 0 and I (x) = x2 2, i.e. for x > 0 ( ) 1 n n log P 1 n X i x x 2 for n. 2 i=1 We say that the X i satisfy a large deviation principle with speed n and rate I (x) = x2 2.

7 Motivation and statement of the question 6/32 Example Consider X1 4,..., X n 4, where the X i have a standard normal law. Then ( ) 1 n n log P 1 n Xi 4 x 0 for n, i=1 hence the probability of a deviation decays slower than exponentially in n. We have ( ) 1 n 1 x log P n n Xi 4 x for n. 2 i=1 We say that the X 4 rate I (x) = x 2. i satisfy a large deviation principle with speed n and

8 Motivation and statement of the question 7/32 Remark Note that the different decay rates in the previous examples do stand for two different regimes. With µ = N(0, 1), we have I (x) = x 2 2 = min ν:m(ν)=x H(ν µ) = H(ν µ) (1) where ν = N(x, 1). Then ( P X 1 A 1 n ) n X i x i=1 ν(a). (2) On the other hand ( P X1 4 A 1 n n i=1 X 4 i x ) P ( X 4 1 A ). (3)

9 Motivation and statement of the question 8/32 Here, H( ) denotes the relative entropy between two measures: for ν, µ P(R), { ( ) H(ν µ) = R log dν dµ dν, if ν µ, +, else.

10 Motivation and statement of the question 9/32 We take a geometric approach by viewing X (n) as a high-dimensional random vector in R n. Example If X 1 Uniform[ 1, 1], then X (n) is a uniform random point in the cube [ 1, 1] n.

11 Motivation and statement of the question 10/32 In this setting, the empirical mean is the projection of X (n) onto the unit vector with equal weights on all coordinates: i.e., 1 n n i=1 X i = 1 n X (n), ϕ (n) n, where, n is the Euclidean inner product on R n, and ϕ (n) = ( 1 n, 1 n,..., 1 n ) S n 1. Here, S n 1 denotes the unit sphere in R n : S n 1 = { x R n : x x x 2 n = 1 }.

12 Motivation and statement of the question 11/32 Given this geometric view of empirical means, it is natural to investigate directions of projection θ (n) S n 1 other than ϕ (n). Such projections correspond to weighted means, where θ (n) S n 1. 1 n n i=1 X i nθ (n) i = 1 n X (n), θ (n) n.

13 Motivation and statement of the question 12/32 We also want to analyze distributions of X (n) other than product measures. In the geometric setting, one way to encode dependence is to look at uniform measures on convex bodies, which are, in general, not product measures. We will analyze a fundamental class of convex bodies, the l p balls; this offers a natural generalization from the product measure case of p = as in the example, to general p [1, ).

14 Motivation and statement of the question 12/32 We also want to analyze distributions of X (n) other than product measures. In the geometric setting, one way to encode dependence is to look at uniform measures on convex bodies, which are, in general, not product measures. We will analyze a fundamental class of convex bodies, the l p balls; this offers a natural generalization from the product measure case of p = as in the example, to general p [1, ). Motivation: a lot is known about uniform measures on high-dimensional convex bodies (Sudakov, Diaconis-Freedman, Klartag,... ), in particular, there is a central limit theorem under mild conditions, which are satisfied for l p balls.

15 Motivation and statement of the question 13/32 More precisely, if (X 1, X 2,..., X n ) has a log-concave joint density, (X 1, X 2,..., X n ) has the same law as (±X 1, ±X 2,..., ±X n ) for all choices of signs and E[Xi 2 ] = 1 for i = 1, 2,..., n, then ( sup a b P a 1 n ) n X i b i=1 for some universal constant C. 1 2π b a e t2 /2 dt C n

16 Motivation and statement of the question 14/32 We choose a uniform random point from the unit l p ball of R n and project it onto a uniform random direction θ (n) S n 1. Let W (n,p) θ = n1/p n 1/2 X (n,p), θ (n) n = 1 n n i=1 (n 1/p X (n,p) i )(n 1/2 θ (n) i ), n N. Why this scaling? The variance of W (n,p) θ should be of order 1/n in order to have the appropriate scale for a large deviation analysis. Note that both n 1/p X (n,p) i and n 1/2 Θ (n) i are typically of order 1, since they are coordinates of points on n 1/p B n,p and n 1/2 S n 1, respectively. Thus, the sum over all i = 1,..., n is of order n.

17 Motivation and statement of the question 15/32 Question Do we have large deviation principles for random projections of l p balls, i.e. for W (n,p) θ and if so, at what speed and rate? = n1/p n 1/2 X (n,p), θ (n) n

18 Motivation and statement of the question 16/32 Our main results on this question are the following: an annealed LDP, for random directions of projection; a quenched LDP, for fixed directions of projection, with a rate function that coincides for almost every sequence of directions; a variational formula which relates the annealed and quenched rate functions via the entropy of an underlying measure, and gives insight into how a geometric rare event occurs;

19 Motivation and statement of the question 17/32 Remark The annelead large deviation principles have been generalized for projections to k n -dimensional uniform random subspaces, provided k n /n converges to a limit in [0, 1], see recent work of David Alonso-Gutiérrez, Joscha Prochno and Christoph Thäle.

20 A toy example: Cramér s Theorem for randomly weighted sums 18/32 In the p = case, X 1, X 2,..., X n are i.i.d. Assume for simplicity that (n) nθ 1, nθ (n) 2,..., nθ n (n) are i.i.d. as well. Let Y i = nθ (n) i, 1 i n, and ask about large deviations of 1 n n X i Y i i=1 where X 1, X 2,... and Y 1, Y 2..., are independent sequences of i.i.d. random variables, X 1 with law µ and Y 1 with law ν. We are looking both at the quenched law, i.e. for (almost any) fixed realization of Y 1, Y 2,..., and at the annealed law, averaged over the Y 1, Y 2,...

21 A toy example: Cramér s Theorem for randomly weighted sums 19/32 Assume that X 1 Y 1 has (all) exponential moments. It is clear that the annealed or averaged laws of 1 n n i=1 X iy i satisfy a large deviation principle with rate function I an - this is Cramér s Theorem. We have [ ] I an (x) = sup λ R λx log e λxy µ(dx)ν(dy) It is easy to see (and follows from a result of Rüschendorf and Kiesel) that for almost any realization of Y 1, Y 2,... the laws of 1 n n i=1 X iy i satisfy an LDP with rate function I qu ν given by [ ( ) ] I qu ν (x) = sup λ R λx log e λxy µ(dx) ν(dy)

22 A toy example: Cramér s Theorem for randomly weighted sums 20/32 The two rate functions are related by the following variational formula: I an [ (x) = inf I qu ν (x) + H( ν ν)]. ν P(R) Remark Variational formulas of this form are typical for quenched/annealed large deviations in random medium. Another example: large deviations for the speed of a random walk in random environment.

23 Large deviations for the dependent case 21/32 Let B n,p denote the unit l p ball in R n. Recall that S n 1 denotes the unit sphere in R n. Write σ n for the unique rotationally invariant probability measure on S n 1. Let S = n N Sn 1 and let π n : S S n 1 be the coordinate map such that for θ = (θ (1), θ (2), ) S, we have π n (θ) = θ (n). Let σ be a probability measure on (the Borel sets of) S such that for all n N, σ π 1 n = σ n. For example, the product measure σ = n N σ n satisfies the above equation. For n N, let Θ (n) denote a random vector that is distributed according to the uniform measure σ n on S n 1, independent of X (n,p).

24 Large deviations for the dependent case 22/32 Recall W (n,p) θ = n1/p n 1/2 X (n,p), θ (n) n. Theorem (Annealed LDP, p > 2) Let p [2, ). The sequence (W (n,p) ) n N satisfies an LDP with a quasiconvex, symmetric, good rate function I an p.

25 Large deviations for the dependent case 23/32 For p < 2, random projections display significantly different large deviation behavior. For p [1, 2), define I an p (w) = 1 r p w rp, where r p = 2p 2+p. Note that r p < 1 for p < 2, so the following large deviations principle holds with a speed n rp slower than the speed n associated with the case p 2. Theorem (Annealed LDP, p < 2) Let p [1, 2). The sequence (W (n,p) ) n N satisfies an LDP with speed n rp and the quasiconvex, symmetric, good rate function I an p.

26 Large deviations for the dependent case 24/32 Remark We see that the two cases p > 2 and p [1, 2) correspond to two different regimes: p > 2 is the regime where a large deviation comes from a collaboration of all coordinates while p [1, 2) is the regime of extremal events, i.e. the large deviations comes from the large deviation of a single coordinate. (See the examples in the beginning). The geometrical reason is that for p > 2, the vectors in n 1/p B n,p that attain maximal Euclidean norm are the corners (±1, ±1,..., ±1). Meanwhile, for p < 2, the vectors in n 1/p B n,p that attain maximal Euclidean norm are again corners, but this time the corners are in canonical basis directions (±1, 0,..., 0), (0, ±1, 0,..., 0), etc.

27 Large deviations for the dependent case 25/32 Theorem (Quenched LDP, p (1, )) Let p (1, ). For σ-a.e. θ S, the sequence (W (n,p) θ ) n N satisfies an LDP with the quasiconvex, symmetric, good rate function I qu p,µ 2.

28 26/32 More precisely, let Large deviations for the dependent case f p (y) = 1 2p 1/p Γ(1 + 1 p )e y p /p, y R. and let µ p be the corresponding probability measure on R. For p (1, ), ν P(R), define ( ) Λ p (t 1, t 2 ) = log e t 1y+t 2 y p µ p (dy) R Ψ p,ν (t 1, t 2 ) = Λ p (t 1 u, t 2 )ν(du), t 1, t 2 R. R Next, define the associated rate functions through the Legendre transforms of Ψ p,ν : for w R, I qu p,ν(w) = inf τ 1 R,τ 2 >0 : τ 1 τ 1/p 2 =w Ψ p,ν(τ 1, τ 2 )

29 Large deviations for the dependent case 27/32 Let m q denote the q-th absolute moment of a measure, m q (ν) = x q ν(dx), ν P(R). R The annealed and quenched rate functions are related through a variational formula involving the relative entropy. Theorem (Relationship between annealed and quenched LDPs) Let p [2, ). Then, for all w R, I an [ p (w) = inf I qu p,ν(w) + H(ν µ 2 ) (1 m 2(ν)) ]. ν P(R):m 2 (ν) 1

30 Large deviations for the dependent case 28/32 Remark The term H(ν µ 2 ) (1 m 2(ν)) comes from the large deviations of the environment. More precisely, the empirical measures n L n := 1 n i=1 δ nθ (n) i satisfy a large deviation principle with rate function { H(ν µ2 ) + 1 I (ν) = 2 (1 m 2(ν)) m 2 (ν) 1 +, else. (The LDP for the weak topology goes back to Amir Dembo, Gérard Ben Arous and Alice Guionnet, and it was strenghtened to the Wasserstein topology by Steven Soojin Kim and Kavita Ramanan).

31 Large deviations for the dependent case 29/32 Remark The case p = 2 is easy: in this case, the LDP holds for ANY sequence θ S and I an 2 (w) = I qu 2,µ 2 (w) = { 1 2 log(1 w 2 ), w ( 1, 1); +, else. This quantifies the well-known statement that two independently chosen points on S n 1 are almost orthogonal with high probability as n goes to infinity. More precise results are known in this case, see Distributions of angles in random packing on spheres by Tony Cai, Jianqing Fan and Tiefeng Jiang.

32 A key ingredient of the proofs 30/32 Let n N and p [1, ). We have probabilistic representations for random vectors in B n,p and S n 1. Recall the density f p (y) = 1 2p 1/p Γ(1 + 1 p )e y p /p, y R. and let µ p be the corresponding probability measure on R. Assume that U is uniformly distributed on [0, 1]; Y (p) = (Y (n,p) ) n N = ( (Y (n,p) 1,, Y n (n,p) ) ) n N is a triangular array of i.i.d. real-valued random variables, with common distribution µ p ; Z = (Z (n) ) n N = ( (Z (n) 1,, Z n (n) ) ) n N is a triangular array of independent N(0, 1) random variables. Moreover, assume U, Y (p), and Z are independent.

33 A key ingredient of the proofs 31/32 Then, the following properties are well known. Lemma For p [1, ), ( X (n,p), Θ (n)) (d) = ( U 1/n Y (n,p) Z (n) Y (n,p), n,p Z (n) n,2 Moreover, Y (n,p) / Y (n,p) n,p is independent of Y (n,p) n,p, and Z (n) / Z (n) n,2 is independent of Z (n) n,2. ).

34 A key ingredient of the proofs 32/32 Thanks for your attention!