Gaussian Phase Transitions and Conic Intrinsic Volumes: Steining the Steiner formula

Transcription

1 Gaussian Phase Transitions and Conic Intrinsic Volumes: Steining the Steiner formula Larry Goldstein, University of Southern California Nourdin GIoVAnNi Peccati Luxembourg University University British Columbia, Jan 18 th, 2018 Ann. Appl. Prob., 2017

2 Ingredients Convex Geometry Compressed sensing Gaussian inequalities

3 Ingredients Convex Geometry: Angular version of the classical Steiner formula for expansion of compact convex sets. Compressed sensing Gaussian inequalities

4 Ingredients Convex Geometry: Angular version of the classical Steiner formula for expansion of compact convex sets. Compressed sensing: Recovery of unknowns under structural assumptions such as sparsity for vectors, or a low rank condition for matrices. Gaussian inequalities

5 Ingredients Convex Geometry: Angular version of the classical Steiner formula for expansion of compact convex sets reveal the intrinsic volume distributions associated with closed convex cones. Compressed sensing: Recovery of high dimensional unknowns under structural assumptions such as sparsity for vectors or a low rank condition, for matrices. Gaussian inequalities and Stein s Method: For providing finite sample bounds on Gaussian fluctuations.

6 Concentration Connection Subject of this work was motivated by the papers by Amelunxen, Lotz, McCoy and Tropp [AMLT13] and McCoy and Tropp [MT14], where the conic intrinsic volume distributions were studied using Poincaré and log Sobolev inequalities. In particular it was shown that these distributions exhibit concentration phenomenon. Raises the possibility that one might make use of other Gaussian inequalities that may be lurking about in the background.

7 Concentration Connection Subject of this work was motivated by the papers by Amelunxen, Lotz, McCoy and Tropp [AMLT13] and McCoy and Tropp [MT14], where the conic intrinsic volume distributions were studied using Poincaré and log Sobolev inequalities. In particular it was shown that these distributions exhibit concentration phenomenon. Raises the possibility that one might make use of other Gaussian inequalities that may be lurking about in the background.

8 Concentration of Conic Intrisic Volumes From: Living on the Edge [ALMT13] 0.06 PHASE TRANSITIONS IN RANDOM CONVEX PROGRAMS FIGURE 2.2: Concentration of conic intrinsic volumes. This plot displays the conic intrinsic volumes v k (C) of a circular cone C R 128 with angle π/6. The distribution concentrates sharply around the statistical dimension δ(c) See Section 3.4 for further discussion of this example.

9 Convex Geometry For any closed convex set K, let d(x, K) = inf x y. y K Classical Steiner formula (1840) for the expansion of a compact convex set K R d, with B j the unit ball in R j, Vol d {x : d(x, K) λ} = d λ d j Vol(B d j )V j. j=0 Intrinsic volumes, V d is volume, 2V d 1 is surface area... and V 0 is Euler characteristic.

10 Convex Geometry The set C R d is a cone if τ(x + y) C for all {x, y} C and τ > 0. With S j 1 unit sphere in R j, one has the angular analog, Hergoltz (1943), } Vol d 1 {x S d 1 : d 2 (x, C) λ = d β j,d (λ)v j The numbers v 0,..., v d are the conic intrinsic volumes, and are non negative and sum to 1. Can associate a distribution L(V ) given by P(V = j) = v j to the cone C, also write V C. j=0

11 Recovery of Structured Unknowns Observe (a small number) m of random linear combinations of an unknown x 0 R d (in high dimension), z = Ax 0 for A R m d known, with i.i.d. N (0, 1) entries. Unknown x 0 lies in the feasible region F = {x : Ax = z} = Null(A) + x 0. With m < d not possible to recover x 0. Say x 0 is sparse, has some small number s of non-zero entries, x 0 0 = s. Minimizing x 0 over x F is computationally infeasible.

12 Recovery of Structured Unknowns Observe (a small number) m of random linear combinations of an unknown x 0 R d (in high dimension), z = Ax 0 for A R m d known, with i.i.d. N (0, 1) entries. Unknown x 0 lies in the feasible region F = {x : Ax = z} = Null(A) + x 0. With m < d not possible to recover x 0. Say x 0 is sparse, has some small number s of non-zero entries, x 0 0 = s. Minimizing x 0 over x F is computationally infeasible.

13 Recovery of Structured Unknowns Observe (a small number) m of random linear combinations of an unknown x 0 R d (in high dimension), z = Ax 0 for A R m d known, with i.i.d. N (0, 1) entries. Unknown x 0 lies in the feasible region F = {x : Ax = z} = Null(A) + x 0. With m < d not possible to recover x 0. Say x 0 is sparse, has some small number s of non-zero entries, x 0 0 = s. Minimizing x 0 over x F is computationally infeasible.

14 Convex Program Determine a convex function f (x) that promotes the known structure of x 0, and minimize f (x) over x F. Chandrasekaran, Recht, Parrilo, and Willsky (2012) show how to construct f for some given structure. To promote sparsity let f (x) = x 1, the L 1 norm, i.e. d i=1 x i. Candès, Romberg, Tao (2006) and Donoho (2006), Donoho and Tanner (2009). Why does it work?

15 Descent Cone For f (x) = x 1 and x 0 R d, let D(f, x 0 ) = {y R d : τ > 0, x 0 + τy 1 x 0 1 }. The set of all directions from which, starting at x 0, do not increase the L 1 norm.

16 Convex recovery of unknowns From: Living on the Edge [ALMT13] PHASE TRANSITIONS IN RANDOM CONVEX PROGRAMS 7 x 0 + null(a) x 0 + null(a) x 0 x 0 {x : f (x) f (x0)} {x : f (x) f (x0)} x 0 + D(f, x 0) x 0 +D(f, x 0) FIGURE 2.3: The optimality condition for a regularized inverse problem. The condition for the regularized linear inverse problem (2.4) to succeed requires that the descent cone D(f, x 0) and the null space null(a) do not share a ray. [left] The regularized linear inverse problem succeeds. [right] The regularized linear inverse problem fails.

17 Convex recovery of sparse unknowns So, translating by x 0, success if and only if Null(A) C = {0} where C is the descent cone of the L 1 norm at x 0. Pause for an acknowledgment of priority.

18 Convex recovery of sparse unknowns So, translating by x 0, success if and only if Null(A) C = {0} where C is the descent cone of the L 1 norm at x 0. Pause for an acknowledgment of priority.

19 Acknowledgment of priority: Descent cone of the L 1 norm at a sparse vector

20 Convex recovery of sparse unknowns Translating by x 0, success if and only if Null(A) C = {0} where C is the descent cone of the L 1 norm at x 0. We know dim(null(a)) = d m. So if the dimension of C were δ(c), we would want d m + δ(c) d so m δ(c). Statistical Dimension of C is δ(c) = E[V C ]. Recovery success probability p sandwiched P(V m 1) p P(V m). Concentration would imply sharp transition, threshold phenomenon.

21 Metric Projection For K any closed convex set, the infimum d(x, K) = inf x y, y K is attained at a unique vector, called the metric projection of x onto K, denoted by Π K (x).

22 Gaussian Connection The cone C is polyhedral if there exists an integer N and vectors u 1,..., u N in R d such that C = N {x R d : u i, x 0}. i=1 For g N (0, I d ), the conic intrinsic volumes satisfy v j = P (Π C (g) lies in the relative interior of a j-dimensional face of C) Consider the orthant R d +. Cone C is like a subspace of random dimension V, hence with statistical dimension δ(c) = E[V C ].

23 Master Steiner Formula [MT14] Implies that with X i independent χ 2 1 random variables, Π C (g) 2 = d V C X i. j=0 Letting G C = Π C (g) 2, G C = d V C V C X i = (X i 1) + V C, j=0 j=0 so recalling δ C = E[V C ] and letting τ 2 C = Var(V C ), we see E[G C ] = δ C and Var(G C ) = 2δ C + τ 2 C

24 Master Steiner Formula [MT14] Relation G C = d V C X i for G C = Π C (g) 2 j=0 yields (strange) moment generating function identity Ee tv = Ee ξtg with ξ t = 1 2 ( 1 e 2t ).

25 Concentration [MT14] From Ee tv = Ee ξtg with ξ t = 1 2 ( 1 e 2t ), and the fact that G = Π C (g) 2, use log Sobolev inequality to obtain bound on its moment generating function. Translate into bound on moment generating fucntion for V C to show concentration around δ C, implying phase transition for exact recovery.

26 Concentration [MT14] From Ee tv = Ee ξtg with ξ t = 1 2 ( 1 e 2t ), and the fact that G = Π C (g) 2, use log Sobolev inequality to obtain bound on its moment generating function. Translate into bound on moment generating fucntion for V C to show concentration around δ C, implying phase transition for exact recovery.

27 Normal Fluctuations Standardizing G C = d V C (X i 1) + V C = W C + V C j=0 yields G C δ C σ C = d ( 2δC σ C ) ( ) WC τc VC δ C +. 2δC σ C τ C Standardized G C is asymptotically normal and expressions on the right hand side are asymptotically independent. Apply Cramér s theorem. Can derive lower bounds for τ C in terms of E[Π C (g), the statistical center.

28 Normal Fluctuations Standardizing G C = d V C (X i 1) + V C = W C + V C j=0 yields G C δ C σ C = d ( 2δC σ C ) ( ) WC τc VC δ C +. 2δC σ C τ C Standardized G C is asymptotically normal and expressions on the right hand side are asymptotically independent. Apply Cramér s theorem. Can derive lower bounds for τ C in terms of E[Π C (g), the statistical center.

29 Toward the 2 nd order Poincaré Inequality, apply to G C 1 st : Poincaré Inequality. If W (g) is a smooth real valued function of g N (0, I d ), then e.g., Var(W (g)) E W (g) 2. σ 2 C = Var ( Π C (g) 2) E 2Π C (g) 2 = 4δ C.

30 Poincaré Inequality, proof sketch Take E[W ] = 0, ϕ a smooth function, ĝ an independent copy of g, and consider E[W ϕ(w )] = E[(W (g) W (ĝ)) ϕ(w (g))]

31 Poincaré Inequality, proof sketch Take E[W ] = 0, ϕ a smooth function, ĝ an independent copy of g, and let ĝ t = e t g + 1 e 2t ĝ E[W ϕ(w )] = E[(W (g) W (ĝ)) ϕ(w (g))]

32 Poincaré Inequality, proof sketch Take E[W ] = 0, ϕ a smooth function, ĝ an independent copy of g, and let ĝ t = e t g + 1 e 2t ĝ E[W ϕ(w )] = E[(W (ĝ 0 ) W (ĝ )) ϕ(w (ĝ 0 ))]

33 Poincaré Inequality, proof sketch Take E[W ] = 0, ϕ a smooth function, ĝ an independent copy of g, and let ĝ t = e t g + 1 e 2t ĝ E[W ϕ(w )] = E[(W (ĝ 0 ) W (ĝ )) ϕ(w (ĝ 0 ))] d = dt E[W (ĝ t)ϕ(w (ĝ 0 ))]dt 0

34 Poincaré Inequality, proof sketch Take E[W ] = 0, ϕ a smooth function, ĝ an independent copy of g, and let ĝ t = e t g + 1 e 2t ĝ E[W ϕ(w )] = E[(W (ĝ 0 ) W (ĝ )) ϕ(w (ĝ 0 ))] d = dt E[W (ĝ t)ϕ(w (ĝ 0 ))]dt 0 = = E[T ϕ (W (g))], T = 0 e t W (g), Ê( W (ĝ t)) dt. Take ϕ(x) = x, obtain Var(W ) = E[T ], use Cauchy Schwarz.

35 Stein s Method Stein s method is based on an equation characterizing the target distribution. Stein s lemma: Z N (0, 1) if and only if E[Zϕ(Z)] = E[ϕ (Z)] for all absolutely continuous functions for which these expectations exist. For test function h, solve the Stein equation ϕ (w) wϕ(w) = h(w) Eh(Z) for ϕ, evaluate at W, variable whose distribution is to be approximated, and bound expectation Eh(W ) Eh(Z) = E[ϕ (W ) W ϕ(w )]

36 Stein s Method Stein s method is based on an equation characterizing the target distribution. Stein s lemma: Z N (0, 1) if and only if E[Zϕ(Z)] = E[ϕ (Z)] for all absolutely continuous functions for which these expectations exist. For test function h, solve the Stein equation ϕ (w) wϕ(w) = h(w) Eh(Z) for ϕ, evaluate at W, variable whose distribution is to be approximated, and bound expectation Eh(W ) Eh(Z) = E[ϕ (W ) W ϕ(w )]

37 Stein s Method Stein s lemma: Z N (0, 1) if and only if E[Zϕ(Z)] = E[ϕ (Z)]. Suppose for mean zero, variance one variable W one can find T such that E[W ϕ(w )] = E[T ϕ (W )], and note E[T ] = 1. For a continuous test function h : R [0, 1], solution of the Stein equation ϕ, Eh(W ) Eh(Z) = E[ϕ (W ) W ϕ(w )] = E[ϕ (W )(1 T )] ϕ E T 1 2 Var(T ). Taking sup over all such h, obtain bound in total variation.

38 Stein s Method Stein s lemma: Z N (0, 1) if and only if E[Zϕ(Z)] = E[ϕ (Z)]. Suppose for mean zero, variance one variable W one can find T such that E[W ϕ(w )] = E[T ϕ (W )], and note E[T ] = 1. For a continuous test function h : R [0, 1], solution of the Stein equation ϕ, Eh(W ) Eh(Z) = E[ϕ (W ) W ϕ(w )] = E[ϕ (W )(1 T )] ϕ E T 1 2 Var(T ). Taking sup over all such h, obtain bound in total variation.

39 2 nd order Poincaré inequality Recall from proof of Poincaré inequality, where E[W ] = 0 and W = W (g), E[W ϕ(w )] = E[T ϕ (W )] with T = Chatterjee (2009), for this T and Var(W ) = 1 we have d TV (W, Z) 2 Var(T ). 0 e t W (g), Ê( W (ĝ t)) dt. Variance might be difficult to evaluate directly, so use the Poincaré inequality.

40 2 nd order Poincaré inequality Recall from proof of Poincaré inequality, where E[W ] = 0 and W = W (g), E[W ϕ(w )] = E[T ϕ (W )] with T = Chatterjee (2009), for this T and Var(W ) = 1 we have d TV (W, Z) 2 Var(T ). 0 e t W (g), Ê( W (ĝ t)) dt. Variance might be difficult to evaluate directly, so use the Poincaré inequality.

41 Gaussian Projections on Convex Cones Want to obtain limiting normal law, with bounds, for G C = Π C (g) 2 in order to infer same for V C. Π C (x) 2 = 2Π C (x) hence T = 4 Use the Poincaré inequality to bound Var(T ), and that σc 2 = 2δ C + τc 2, to obtain 0 e t Π C (g), ÊΠ C (ĝ t ) dt d TV (G C, Z) 16 δ(c) σ 2 C 8 δ(c).

42 Berry-Esseen bound for conic intrinsic volumes If C is a closed convex cone with δ C = E[V C ] and τ 2 C = Var(V C ) > 0, then with α = τ 2 C /δ C, for δ C 8 [ sup P VC δ C u R τ C ] u P[Z u] h(δ C ) + 48 α log + ( α 2δ C ) where h(δ C ) = 1 72 ( ) 5/2 log δ C δ 3/16 C For m about δ C + tτ C, obtain bound on sup P {x 0 is recovered} 1 t 2π t R e u2 /2 du

43 Berry-Esseen bound for conic intrinsic volumes If C is a closed convex cone with δ C = E[V C ] and τ 2 C = Var(V C ) > 0, then with α = τ 2 C /δ C, for δ C 8 [ sup P VC δ C u R τ C ] u P[Z u] h(δ C ) + 48 α log + ( α 2δ C ) where h(δ C ) = 1 72 ( ) 5/2 log δ C δ 3/16 C For m about δ C + tτ C, obtain bound on sup P {x 0 is recovered} 1 t 2π t R e u2 /2 du

44 Ingredients Convex Geometry Compressed sensing Gaussian inequalities and Stein s Method Poincaré inequality bound a variance by an expectation. Log Sobolev inequality 2 nd order Poincaré inequality bound a total variation by a variance, which is bounded by an expectation.