Intrinsic Volumes of Convex Cones Theory and Applications

Size: px

Start display at page:

Download "Intrinsic Volumes of Convex Cones Theory and Applications"

Erick Ellis
6 years ago
Views:

$Intrinsic Volumes of Convex Cones Theory and Applications M\cr$ Mathematics The University of Manchester with the collaboration

1 Intrinsic Volumes of Convex Cones Theory and Applications M\cr NA Manchester Numerical Analysis Martin Lotz School of Mathematics The University of Manchester with the collaboration of Dennis Amelunxen, Michael B. McCoy, Joel A. Tropp Chicago, July 11, 2014

2 Outline Problems involving cones Some conic integral geometry Concentration

3 Outline Problems involving cones Some conic integral geometry Concentration

4 Conic problems Problem: find a structured solution x 0 of m d system (m < d) Ax = b by minimizing a convex regularizer minimize f(x) subject to Ax = b. ( ) 1 / 27

5 Conic problems Problem: find a structured solution x 0 of m d system (m < d) by minimizing a convex regularizer Examples include: x 0 sparse: f(x) = x 1 ; Ax = b minimize f(x) subject to Ax = b. ( ) X 0 low-rank matrix: f(x) nuclear norm; Models with simultaneous structures ( talk by M. Fazel), atomic norms ( talk by B. Recht) 1 / 27

6 Example: l 1 minimization Let x 0 be s-sparse, b = Ax 0 for random A R m d (s < m < d). minimize x 1 subject to Ax = b. Probability of success Number of equations m s = 50, m = 25, d = / 27

7 Example: l 1 minimization Let x 0 be s-sparse, b = Ax 0 for random A R m d (s < m < d). minimize x 1 subject to Ax = b. Probability of success Number of equations m s = 50, m = 50, d = / 27

8 Example: l 1 minimization Let x 0 be s-sparse, b = Ax 0 for random A R m d (s < m < d). minimize x 1 subject to Ax = b. Probability of success Number of equations m s = 50, m = 75, d = / 27

9 Example: l 1 minimization Let x 0 be s-sparse, b = Ax 0 for random A R m d (s < m < d). minimize x 1 subject to Ax = b. Probability of success Number of equations m s = 50, m = 100, d = / 27

10 Example: l 1 minimization Let x 0 be s-sparse, b = Ax 0 for random A R m d (s < m < d). minimize x 1 subject to Ax = b. Probability of success Number of equations m s = 50, m = 125, d = / 27

11 Example: l 1 minimization Let x 0 be s-sparse, b = Ax 0 for random A R m d (s < m < d). minimize x 1 subject to Ax = b. Probability of success Number of equations m s = 50, m = 150, d = / 27

12 Example: l 1 minimization Let x 0 be s-sparse, b = Ax 0 for random A R m d (s < m < d). minimize x 1 subject to Ax = b. Probability of success Number of equations m s = 50, m = 175, d = / 27

13 Example: l 1 minimization Let x 0 be s-sparse, b = Ax 0 for random A R m d (s < m < d). minimize x 1 subject to Ax = b. Probability of success Number of equations m s = 50, m = 200, d = / 27

Phase transitions 100 900 75 600 50 300 25 0 0 25 50 75 100 0 0 10

14 Phase transitions Phase transitions for l 1 and nuclear norm minimization. 3 / 27

15 Conic problems Problem: find a structured solution x 0 of m d system (m < d) Ax = b by minimizing a convex regularizer minimize f(x) subject to Ax = b. ( ) ( ) has x 0 as unique solution if and only if the optimality condition ker A D(f, x 0 ) = {0} is satisfied, where D(f, x 0 ) is the convex descent cone of f at x 0 : D(f, x 0 ) := { y R d : f(x 0 + τy) f(x 0 ) } τ>0 4 / 27

16 Success of l 1 -minimization minimize x 1 subject to Ax = b x 0 {Ax = b} success { x 1 x 0 1 } 5 / 27

17 Success of l 1 -minimization minimize x 1 subject to Ax = b x 0 failure 5 / 27

18 Success of l 1 -minimization minimize x 1 subject to Ax = b x 0 success descent cone 5 / 27

19 Success of l 1 -minimization minimize x 1 subject to Ax = b x 0 success descent cone l 1 minimization succeeds at finding a sparse vector x 0 if and only if the kernel of A misses the cone of descent directions of 1 at x 0. 5 / 27

20 Conic problems Problem: reconstruct two signals x 0, y 0 from the observation z 0 = x 0 + Qy 0, where Q O(d), by solving minimize f(x) subject to g(y) g(y 0) and z 0 = x + Qy. ( ) for suitable convex functions f and g. 6 / 27

21 Conic problems Problem: reconstruct two signals x 0, y 0 from the observation z 0 = x 0 + Qy 0, where Q O(d), by solving minimize f(x) subject to g(y) g(y 0) and z 0 = x + Qy. ( ) for suitable convex functions f and g. both are sparse x 0 sparse (corruption), y 0 {±1} d (message) x 0 low-rank matrix, y 0 sparse (corruption) 6 / 27

22 Conic problems Problem: reconstruct two signals x 0, y 0 from the observation z 0 = x 0 + Qy 0, where Q O(d), by solving minimize f(x) subject to g(y) g(y 0) and z 0 = x + Qy. ( ) for suitable convex functions f and g. ( ) uniquely recovers x 0, y 0, if and only if D(f, x 0 ) QD(g, y 0 ) = {0}, (McCoy-Tropp (2012,2013) Mike s upcoming talk) 6 / 27

23 Conic problems Projections of polytopes: Let A: R d R m, m < d, P R d polytope, F P a face. Then AF is a face of AP if and only if ker A T F (P ) = {0}, where T F (P ) is tangent cone to F at P (Donoho-Tanner (2006-)). Compressive separation: Given disjoint convex sets S 1, S 2 R d, then AS 1 AS 2 = if and only if (Bandeira-Mixon-Recht (2014)) ker A cone(s 1 S 2 ) = {0}. 7 / 27

24 The mathematical problem The problems mentioned motivate the following question: Given closed convex cones C, D R d and a random orthogonal transformation Q, what is the probability that they intersect: C QD {0}, ( ) Bounds on the intersection probability of cone with linear subspace follow from Gordon s escape through the mesh argument; Exact formulas for probability of intersection are based on the kinematic formula from (spherical) integral geometry. topic of this talk. 8 / 27

25 Outline Problems involving cones Some conic integral geometry Concentration

26 The kinematic formula The probability that a randomly rotated cone intersects another (not both linear spaces) is given by a discrete probability distribution, the spherical intrinsic volumes v 0 (C),..., v d (C): P{C QD {0}} = 2 v i (C)v j (D). k odd i+j=d+k For the case where D = L is a linear subspace of codimension m, v i (L) = 1 if i = d m and v i (L) = 0 else, P{C QL {0}} = 2 v m+k (C). k odd ( ) ( ) is essentially the tail of a discrete probability distribution. 9 / 27

27 Spherical intrinsic volumes C v 1 (C) v 2 (C) v 1 (C) 0 v 0 (C) Let C R d be a polyhedral cone, F k (C) set of k-dimensional faces. The k-th (spherical) intrinsic volume of C is defined as v k (C) = P{Π C (g) relint(f )}. F F k (C) For non-polyhedral cones by approximation or via Steiner formula. A long history in geometry, have also appeared in statistics (as weights of χ 2 distributions). 10 / 27

28 Spherical intrinsic volumes: examples Linear subspace L: v k (L) = 1 if dim L = k, v k (L) = 0 else. 11 / 27

29 Spherical intrinsic volumes: examples Linear subspace L: v k (L) = 1 if dim L = k, v k (L) = 0 else. Orthant R d 0: v k (R d 0) = ( ) d 2 d k 11 / 27

30 Spherical intrinsic volumes: examples Linear subspace L: v k (L) = 1 if dim L = k, v k (L) = 0 else. Orthant R d 0: v k (R d 0) = ( ) d 2 d k Second order cones: ( ( ) 1 d 2 ) v k Circ(d, α) = 2 sin k 1 (α) cos d k 1 (α). 2 k / 27

31 Spherical intrinsic volumes: examples Linear subspace L: v k (L) = 1 if dim L = k, v k (L) = 0 else. Orthant R d 0: v k (R d 0) = ( ) d 2 d k Second order cones: ( ( ) 1 d 2 ) v k Circ(d, α) = 2 sin k 1 (α) cos d k 1 (α). 2 k 1 2 Asymptotics for tangent cones at faces of simplex and l 1 -ball (Vershik-Sporyshev (1992), Donoho (2006)). 11 / 27

32 Spherical intrinsic volumes: examples Linear subspace L: v k (L) = 1 if dim L = k, v k (L) = 0 else. Orthant R d 0: v k (R d 0) = ( ) d 2 d k Second order cones: ( ( ) 1 d 2 ) v k Circ(d, α) = 2 sin k 1 (α) cos d k 1 (α). 2 k 1 2 Asymptotics for tangent cones at faces of simplex and l 1 -ball (Vershik-Sporyshev (1992), Donoho (2006)). Integral representations for the semidefinite cone (Amelunxen-Bürgisser (2012)). 11 / 27

33 Spherical intrinsic volumes: examples Linear subspace L: v k (L) = 1 if dim L = k, v k (L) = 0 else. Orthant R d 0: v k (R d 0) = ( ) d 2 d k Second order cones: ( ( ) 1 d 2 ) v k Circ(d, α) = 2 sin k 1 (α) cos d k 1 (α). 2 k 1 2 Asymptotics for tangent cones at faces of simplex and l 1 -ball (Vershik-Sporyshev (1992), Donoho (2006)). Integral representations for the semidefinite cone (Amelunxen-Bürgisser (2012)). Combinatorial expressions for regions of hyperplane arrangements (Klivans-Swartz (2011)). 11 / 27

34 Spherical intrinsic volumes: examples v k (L), dim L = k v k (R d 0) Circ(d, π/4) v k ({x: x 1 x d }) 12 / 27

35 Concentration Circ(d, π/4) Associate to a cone C the discrete random variable X C with P{X C = k} = v k (C), and define the statistical dimension as the average δ(c) = E[X C ] = d kv k (C). k=0 In the examples it appears that X C concentrates around δ(c). 13 / 27

36 Concentration Theorem [ALMT14] Let C be a convex cone, and X C a discrete random variable with distribution P{X C = k} = v k (C). Let δ(c) = E[X C ]. Then ( λ 2 ) /8 P{ X C δ(c) > λ} 4 exp for λ 0, ω(c) + λ where ω(c) := min{δ(c), d δ(c)}. Improved bounds by McCoy-Tropp (Discr. Comp. Geom. 2014) 14 / 27

37 Approximate kinematic formula Applying the concentration result to the kinematic formula P{C QD {0}} = 2 v i (C)v j (D). k odd i+j=d+k gives rise to Theorem [ALMT14] Fix a tolerance η (0, 1). Assume one of C, D is not a subspace. Then { } δ(c) + δ(d) d a η d = P C QK = {0} 1 η; { } δ(c) + δ(d) d + a η d = P C QK = {0} η, where a η := 4 log(4/η) (a 0.01 < 10 and a < 12). 15 / 27

38 Approximate kinematic formula Applying the concentration result to the kinematic formula P{C QD {0}} = 2 v i (C)v j (D). k odd i+j=d+k gives rise to Theorem [ALMT14] Fix a tolerance η (0, 1). Assume one of C, D is not a subspace. Then { } δ(c) + δ(d) d a η d = P C QK = {0} 1 η; { } δ(c) + δ(d) d + a η d = P C QK = {0} η, where a η := 4 log(4/η) (a 0.01 < 10 and a < 12). Interpretation: convex cones behave like linear subspaces of dimension δ(c), δ(d) in high dimensions. 15 / 27

39 Statistical dimension: basic properties Orthogonal invariance. δ(qc) = δ(c) for each Q O(d). 16 / 27

40 Statistical dimension: basic properties Orthogonal invariance. δ(qc) = δ(c) for each Q O(d). Subspaces. For a subspace L R d, δ(l) = dim(l). 16 / 27

41 Statistical dimension: basic properties Orthogonal invariance. δ(qc) = δ(c) for each Q O(d). Subspaces. Totality. For a subspace L R d, δ(l) = dim(l). δ(c) + δ(c ) = d. This generalises dim(l) + dim(l ) = d for linear L. 16 / 27

42 Statistical dimension: basic properties Orthogonal invariance. δ(qc) = δ(c) for each Q O(d). Subspaces. Totality. For a subspace L R d, δ(l) = dim(l). δ(c) + δ(c ) = d. This generalises dim(l) + dim(l ) = d for linear L. C C 16 / 27

43 Statistical dimension: basic properties Orthogonal invariance. δ(qc) = δ(c) for each Q O(d). Subspaces. Totality. For a subspace L R d, δ(l) = dim(l). δ(c) + δ(c ) = d. This generalises dim(l) + dim(l ) = d for linear L. Direct products. For each cone closed convex cone K, δ(c K) = δ(c) + δ(k). In particular, invariance under embedding. 16 / 27

44 Statistical dimension: basic properties Orthogonal invariance. δ(qc) = δ(c) for each Q O(d). Subspaces. Totality. For a subspace L R d, δ(l) = dim(l). δ(c) + δ(c ) = d. This generalises dim(l) + dim(l ) = d for linear L. Direct products. For each cone closed convex cone K, δ(c K) = δ(c) + δ(k). In particular, invariance under embedding. Monotonicity. C K implies that δ(c) δ(k). 16 / 27

45 Statistical dimension: basic properties Expected squared Gaussian projection: δ(c) = E [ Π C (g) 2], previously appeared in various contexts, among others as proxy to Gaussian width. Spherical formulation: δ(c) := d E [ Π C (θ) 2 ] where θ Uniform(S d 1 ). Relation to Gaussian width w(c) = E [ sup x, g ] : x C S d 1 w(c) 2 δ(c) w(c) Gaussian width has played a role in the analysis of recovery via Gordon s comparison inequality (Rudelson-Vershynin (2008), Stojnic (2009-), Oymak-Hassibi (2010-), Chandrasekaran et al. (2012)). 17 / 27

46 Examples Linear subspaces. δ(l) = dim L Non-negative orthant. δ(r d 0) = d/2. Self-dual cones. We have δ(c) + δ(c ) = d, so that δ(c) = d/2 for any self-dual cone (for example, positive semidefinite matrices). Second-order (ice cream) cones of angle α. Circ(d, α) := { x R n : x 1 / x cos(α) }. Then δ ( Circ(d, α) ) d sin 2 (α) The cone C A = {x : x 1 x d }. δ(c A ) = d k=1 1 k log(d). 18 / 27

47 Computing the statistical dimension In some cases the statistical dimension of a convex cone can be compute exactly from the intrinsic volumes: Spherical cones; Descent cone of f = l ; Regions of hyperplane arrangements with high symmetry. 19 / 27

48 Computing the statistical dimension In some cases the statistical dimension of a convex cone can be compute exactly from the intrinsic volumes: Spherical cones; Descent cone of f = l ; Regions of hyperplane arrangements with high symmetry. For descent cones of convex regularizers, asymptotically expressions follow from a blueprint developed by Stojnic (2008) and refined since: x 0 s-sparse, f = 1. Asymptotic formula for δ( 1, x 0 ) follows from Stojnic X 0 rank r matrix, f = S1. Asymptotic formula based on the Marčenko-Pastur characterisation of the empirical eigenvalue distribution of Wishart matrices. 19 / 27

49 Computing the statistical dimension: an example C A = {x: x 1 x 2 x d }. Normal cone to vertex at permutahedron, suggested as convex regularizer for vectors from lists problem (Chandrasekaran et al (2012)). Using combinatorics (Klivans-Swartz (2011), Stanley): v(t) = d k=0 v k (C A )t k = 1 t (t + 1) (t + d 1). d! Statistical dimension: δ(c A ) = d dt v(t) t=1 = d k=1 1 k log(d). 20 / 27

50 Outline Problems involving cones Some conic integral geometry Concentration

51 The spherical Steiner formula Recall the characterization δ(c) = E [ Π C (g) 2] The measure of points within angle arccos( ε) of cone C on the sphere is given by Spherical Steiner Formula (Herglotz, Allendoerfer, Santaló) P { Π C (θ) 2 ε } = d L k : k-dimensional subspace θ: uniform on S d 1. k=1 P{ Π Lk (θ) 2 ε } v k (C) 21 / 27

52 The spherical Steiner formula Recall the characterization δ(c) = E [ Π C (g) 2] The measure of points within angle arccos( ε) of cone C on the sphere is given by Spherical Steiner Formula (Herglotz, Allendoerfer, Santaló) P { Π C (θ) 2 ε } = d L k : k-dimensional subspace θ: uniform on S d 1. k=1 P{ Π Lk (θ) 2 ε } v k (C) substantially generalized by M. McCoy (McCoy-Tropp (2014)) 21 / 27

53 The spherical Steiner formula Volume of neighbourhood of subspheres P { Π C (θ) 2 ε } = d k=1 P{ Π Lk (θ) 2 ε } }{{} Beta distributed v k (C) Volume of arccos( ε)-neighbourhood of k-dimensional subsphere satisfies P { Π Lk (θ) 2 ε } { 0 if ε > k/d 1 if ε < k/d 22 / 27

54 The spherical Steiner formula Volume of neighbourhood of subspheres P { Π C (θ) 2 ε } d k= εd v k (C). Volume of arccos( ε)-neighbourhood of k-dimensional subsphere satisfies P { Π Lk (θ) 2 ε } { 0 if ε > k/d 1 if ε < k/d 23 / 27

55 The spherical Steiner formula Measure concentration P { Π C (θ) 2 ε } d v k (C). k= εd { 0 if ε > δ(c)/d 1 if ε < δ(c)/d Follows from concentration of measure, since the squared projection is Lipschitz and concentrates near expected value δ(c). 24 / 27

56 The spherical Steiner formula Let X C be a random variable with distribution given by the spherical intrinsic volumes P{X C = k} = v k (C). By the spherical Steiner formula we have P{X C εd} d k= εd v k (C) { 0 if ε > δ(c)/d 1 if ε < δ(c)/d Rigorous implementation uses more advanced concentration of measure technology. 25 / 27

57 Some problems Spherical Hadwiger conjecture: Each continuous, rotation-invariant valuation on closed convex cones is a linear combination of spherical intrinsic volumes. Are the spherical intrinsic volumes log concave? v k (C) 2 v k 1 (C) v k+1 (C) Is the variance of X C maximised by the Lorentz cone Circ π/4? Further develop the combinatorial approach to computing intrinsic volumes with a view towards cones of interest in statistics (isotonic regression). 26 / 27

58 For more details: D. Amelunxen, M. Lotz, M. B. McCoy, and J. A. Tropp. Living on the edge: phase transitions in convex probrams with random data. Information and Inference, 2014 arxiv: M. B. McCoy, J. A. Tropp. From Steiner formulas for cones to concentration of intrinsic volumes. Discrete Comput. Geometry, 2014 D. Amelunxen, M. Lotz. Gordon s inequality and condition numbers in convex optimization. To appear on arxiv these days. Thank You! 27 / 27

Living on the edge: phase transitions in convex programs with random data

Information and Inference: A Journal of the IMA (2014) 3, 224 294 doi:10.1093/imaiai/iau005 Advance Access publication on 30 June 2014 Living on the edge: phase transitions in convex programs with random