Rank-one Generated Spectral Cones Defined by Two Homogeneous Linear Matrix Inequalities

Size: px

Start display at page:

Download "Rank-one Generated Spectral Cones Defined by Two Homogeneous Linear Matrix Inequalities"

Juniper Fowler
5 years ago
Views:

1 Rank-one Generated Spectral Cones Defined by Two Homogeneous Linear Matrix Inequalities C.J. Argue Joint work with Fatma Kılınç-Karzan October 22, 2017 INFORMS Annual Meeting Houston, Texas C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 1 / 21 1/21

2 Introduction Motivation Nonconvex quadratic program max x x Qx s.t. x M i x 0, i = 1,..., k C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 2 / 21 2/21

3 Introduction Motivation Nonconvex quadratic program max x x Qx s.t. x M i x 0, i = 1,..., k We have x Ax = Tr(x Ax) = Tr(Axx ) = A, xx. max x Q, xx s.t. M i, xx 0, i = 1,..., k C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 2 / 21 2/21

4 Introduction Motivation Nonconvex quadratic program max x x Qx s.t. x M i x 0, i = 1,..., k We have x Ax = Tr(x Ax) = Tr(Axx ) = A, xx. max x Q, xx s.t. M i, xx 0, i = 1,..., k Can write X = xx if and only if X 0 and rank(x) = 1. max Q, X X s.t. M i, X 0, i = 1,..., k X 0 rank(x) = 1 C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 2 / 21 2/21

5 Introduction Motivation max Q, X X s.t. M i, X 0, i = 1,..., k X 0 rank(x) = 1 The condition rank(x) = 1 is nonconvex. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 3 / 21 3/21

6 Introduction Motivation max Q, X X s.t. M i, X 0, i = 1,..., k X 0 rank(x) = 1 The condition rank(x) = 1 is nonconvex. Convex (semidefinite program) relaxation: max Q, X X s.t. M i, X 0, i = 1,..., k X 0 C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 3 / 21 3/21

7 Introduction Motivation When is this relaxation tight? C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 4 / 21 4/21

8 Introduction Motivation When is this relaxation tight? Feasible set perspective. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 4 / 21 4/21

9 Introduction Motivation When is this relaxation tight? Feasible set perspective. Tight for every objective function if and only if every extreme ray is rank one (Rank-One Generated/ROG). C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 4 / 21 4/21

10 Introduction Motivation When is this relaxation tight? Feasible set perspective. Tight for every objective function if and only if every extreme ray is rank one (Rank-One Generated/ROG). Analogous to integral polyhedra/total unimodularity. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 4 / 21 4/21

11 Introduction Motivation When is this relaxation tight? Feasible set perspective. Tight for every objective function if and only if every extreme ray is rank one (Rank-One Generated/ROG). Analogous to integral polyhedra/total unimodularity. Burer 15, Hildebrand 16, Blekherman et al. 16. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 4 / 21 4/21

12 Introduction Our Question Question Let M 1, M 2 be n n symmetric matrices. When is S := {Y 0 : Y, M 1 0, Y, M 2 0} an ROG cone? C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 5 / 21 5/21

13 Introduction Outline Two geometric perspectives. Each perspective gives a sufficient condition for S to be ROG. Together these conditions are also necessary. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 6 / 21 6/21

14 Introduction Outline Two geometric perspectives. Each perspective gives a sufficient condition for S to be ROG. Together these conditions are also necessary. M, Y = 0 as a hyperplane in S n := {n n symmetric matrices}. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 6 / 21 6/21

15 Introduction Outline Two geometric perspectives. Each perspective gives a sufficient condition for S to be ROG. Together these conditions are also necessary. M, Y = 0 as a hyperplane in S n := {n n symmetric matrices}. M, xx = x M 1 x as a quadratic form in R n. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 6 / 21 6/21

16 Introduction Recap A set/cone is ROG if all its extreme points/rays have rank 1. SDP relaxations of quadratic programs are tight for every objective function if and only if the feasible set is ROG. Consider S := {Y 0 : Y, M 1 0, Y, M 2 0} (two LMIs). Two geometric perspectives S n and R n. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 7 / 21 7/21

17 Geometry of S n + Rank Consider S n + := {positive semidefinite n n matrices} S n. [ ] 1 0 Red ray: 0 0 [ ] 1 0 Green ray: 0 1 Rank 1 extreme. Rank 2 not extreme. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 8 / 21 8/21

18 Geometry of S n + One LMI Fact [Ye, Zhang 03] S := {Y 0 : M, Y 0} is ROG for any M S 3. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 9 / 21 9/21

19 Geometry of S n + Two LMIs Interacting inside S n +. Non-interacting inside S n +. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 10 / 21 10/21

20 Geometry of S n + Non-interacting LMIs If M 1 and M 2 are non-interacting, then every extreme ray of S = {Y 0 : Y, M 1 0, Y, M 2 0} is an extreme ray of either {Y 0 : Y, M 1 0} or {Y 0 : Y, M 2 0}. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 11 / 21 11/21

21 Geometry of S n + Non-interacting LMIs If M 1 and M 2 are non-interacting, then every extreme ray of S = {Y 0 : Y, M 1 0, Y, M 2 0} is an extreme ray of either {Y 0 : Y, M 1 0} or {Y 0 : Y, M 2 0}. S is ROG. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 11 / 21 11/21

22 Geometry of S n + Non-interacting LMIs Non-interacting inside S n + when: One LMI does not intersect S n +, i.e. when ±M i 0. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 12 / 21 12/21

23 Geometry of S n + Non-interacting LMIs Non-interacting inside S n + when: One LMI does not intersect S n +, i.e. when ±M i 0. ±M 1, X 0 is a consequence of ±M 2, X 0 for X 0. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 12 / 21 12/21

24 Geometry of S n + Non-interacting LMIs Non-interacting inside S n + when: One LMI does not intersect S n +, i.e. when ±M i 0. ±M 1, X 0 is a consequence of ±M 2, X 0 for X 0. Using S-lemma, this is true when λ(±m 1 ) (±M 2 ) 0 for some λ 0. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 12 / 21 12/21

25 Geometry of S n + Non-interacting LMIs Non-interacting inside S n + when: One LMI does not intersect S n +, i.e. when ±M i 0. ±M 1, X 0 is a consequence of ±M 2, X 0 for X 0. Using S-lemma, this is true when λ(±m 1 ) (±M 2 ) 0 for some λ 0. In sum, non-interacting when αm 1 + βm 2 0 for some (α, β) (0, 0). C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 12 / 21 12/21

26 Geometry of S n + Recap Non-interacting LMIs yield ROG cones. M 1, M 2 are non-interacting if ±M 2, Y 0 along with Y 0 implies ±M 1, Y 0. Proposition 1 If αm 1 + βm 2 0 has a nontrivial solution, i.e. (α, β) (0, 0) then S is ROG. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 13 / 21 13/21

27 Showing Y / Ext(S) Question In general, how do we show that a cone is ROG? C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 14 / 21 14/21

28 Showing Y / Ext(S) Question In general, how do we show that a cone is ROG? Show that Y / Ext(S) when: rank(y ) 2. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 14 / 21 14/21

29 Showing Y / Ext(S) Question In general, how do we show that a cone is ROG? Show that Y / Ext(S) when: rank(y ) 2. Y, M 1 = Y, M 2 = 0. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 14 / 21 14/21

30 Showing Y / Ext(S) Question In general, how do we show that a cone is ROG? Show that Y / Ext(S) when: rank(y ) 2. Y, M 1 = Y, M 2 = 0. Find x R n such that Y ± xx S. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 14 / 21 14/21

31 Showing Y / Ext(S) Question In general, how do we show that a cone is ROG? Show that Y / Ext(S) when: rank(y ) 2. Y, M 1 = Y, M 2 = 0. Find x R n such that Y ± xx S. For Y xx 0, need x Range(Y ). C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 14 / 21 14/21

32 Showing Y / Ext(S) Question In general, how do we show that a cone is ROG? Show that Y / Ext(S) when: rank(y ) 2. Y, M 1 = Y, M 2 = 0. Find x R n such that Y ± xx S. For Y xx 0, need x Range(Y ). Since Y, M i = 0, need 0 = xx, M i = x M i x. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 14 / 21 14/21

33 Quadratic Forms Zero Sets Fix a candidate extreme ray Y. Define N 1 := {x R n : x M 1 x = 0}. N 2 := {x R n : x M 2 x = 0}. When is Range(Y ) N 1 N 2 {0}? C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 15 / 21 15/21

34 Quadratic Forms Start with n = 3. Consider Y S 3 +, rank(y ) = 2. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 16 / 21 16/21

35 Quadratic Forms Start with n = 3. Consider Y S 3 +, rank(y ) = 2. Range(Y ) is a plane. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 16 / 21 16/21

36 Quadratic Forms Start with n = 3. Consider Y S 3 +, rank(y ) = 2. Range(Y ) is a plane. If N 1 N 2 R 3 contains a plane, then it intersects every plane nontrivially. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 16 / 21 16/21

37 Quadratic Forms Start with n = 3. Consider Y S 3 +, rank(y ) = 2. Range(Y ) is a plane. If N 1 N 2 R 3 contains a plane, then it intersects every plane nontrivially. Observation S is ROG when N 1 N 2 contains a plane. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 16 / 21 16/21

38 Quadratic Forms N 1 N 2 contains a plane Question When does N 1 N 2 contain a plane? C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 17 / 21 17/21

39 Quadratic Forms N 1 N 2 contains a plane Question When does N 1 N 2 contain a plane? {x R 3 : x Mx = 0} contains a plane when rank(m) 2 and M is indefinite. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 17 / 21 17/21

40 Quadratic Forms N 1 N 2 contains a plane Question When does N 1 N 2 contain a plane? {x R 3 : x Mx = 0} contains a plane when rank(m) 2 and M is indefinite. For any (α, β), N α,β := {x R 3 : x (αm 1 + βm 2 )x = 0} N 1 N 2. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 17 / 21 17/21

41 Quadratic Forms N 1 N 2 contains a plane Question When does N 1 N 2 contain a plane? {x R 3 : x Mx = 0} contains a plane when rank(m) 2 and M is indefinite. For any (α, β), N α,β := {x R 3 : x (αm 1 + βm 2 )x = 0} N 1 N 2. In particular, N α,β contains a plane for all (α, β). C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 17 / 21 17/21

42 Quadratic Forms N 1 N 2 contains a plane Question When does N 1 N 2 contain a plane? Answer {x R 3 : x Mx = 0} contains a plane when rank(m) 2 and M is indefinite. For any (α, β), N α,β := {x R 3 : x (αm 1 + βm 2 )x = 0} N 1 N 2. In particular, N α,β contains a plane for all (α, β). When rank(αm 1 + βm 2 ) 2 for all (α, β). C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 17 / 21 17/21

43 Geometry of Quadratic Forms Recap Y is not an extreme ray when Range(Y ) N 1 N 2 has a nonzero element. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 18 / 21 18/21

44 Geometry of Quadratic Forms Recap Y is not an extreme ray when Range(Y ) N 1 N 2 has a nonzero element. In R 3, if N 1 N 2 contains a plane, no rank 2 extreme rays. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 18 / 21 18/21

45 Geometry of Quadratic Forms Recap Y is not an extreme ray when Range(Y ) N 1 N 2 has a nonzero element. In R 3, if N 1 N 2 contains a plane, no rank 2 extreme rays. N 1 N 2 contains a plane when rank(αm 1 + βm 2 ) 2 for all α, β. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 18 / 21 18/21

46 Geometry of Quadratic Forms Recap Y is not an extreme ray when Range(Y ) N 1 N 2 has a nonzero element. In R 3, if N 1 N 2 contains a plane, no rank 2 extreme rays. N 1 N 2 contains a plane when rank(αm 1 + βm 2 ) 2 for all α, β. Proposition 2 S is ROG when rank(αm 1 + βm 2 ) 2 for all (α, β), Span{Range(M 1 ) Range(M 2 )} has dimension 3, and αm 1 + βm 2 0 has only the trivial solution (α, β) = (0, 0). C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 18 / 21 18/21

47 Main Result Theorem 3 (A, Kılınç-Karzan, 17) {Y 0 : M 1, Y 0, M 2, Y 0} is ROG iff one of the following holds (i) αm 1 + βm 2 0 for some (α, β) (0, 0). (ii) rank(αm 1 + βm 2 ) 2 for all (α, β) and Span{Range(M 1 ) Range(M 2 )} has dimension 3. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 19 / 21 19/21

48 Proving Necessity (Sketch) First consider the case of S 3. Suppose that: (i) αm 1 + βm 2 0 for any (α, β) (0, 0). (ii) rank(am 1 + bm 2 ) 3 for some (a, b). C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 20 / 21 20/21

49 Proving Necessity (Sketch) First consider the case of S 3. Suppose that: (i) αm 1 + βm 2 0 for any (α, β) (0, 0). (ii) rank(am 1 + bm 2 ) 3 for some (a, b). Need to construct an extreme ray Y of rank 2. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 20 / 21 20/21

50 Proving Necessity (Sketch) First consider the case of S 3. Suppose that: (i) αm 1 + βm 2 0 for any (α, β) (0, 0). (ii) rank(am 1 + bm 2 ) 3 for some (a, b). Need to construct an extreme ray Y of rank 2. rank(am 1 + bm 2 ) = 3 implies that N 1 N 2 is sparse. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 20 / 21 20/21

51 Proving Necessity (Sketch) First consider the case of S 3. Suppose that: (i) αm 1 + βm 2 0 for any (α, β) (0, 0). (ii) rank(am 1 + bm 2 ) 3 for some (a, b). Need to construct an extreme ray Y of rank 2. rank(am 1 + bm 2 ) = 3 implies that N 1 N 2 is sparse. Get a vector z that is not spanned by any two vectors of N 1 N 2. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 20 / 21 20/21

52 Proving Necessity (Sketch) First consider the case of S 3. Suppose that: (i) αm 1 + βm 2 0 for any (α, β) (0, 0). (ii) rank(am 1 + bm 2 ) 3 for some (a, b). Need to construct an extreme ray Y of rank 2. rank(am 1 + bm 2 ) = 3 implies that N 1 N 2 is sparse. Get a vector z that is not spanned by any two vectors of N 1 N 2. Use infeasibility of αm 1 + βm 2 0 for (α, β) (0, 0) to get w such that Y = zz T + ww T is tight for both LMIs (w λz for λ R). C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 20 / 21 20/21

53 Proving Necessity (Sketch) First consider the case of S 3. Suppose that: (i) αm 1 + βm 2 0 for any (α, β) (0, 0). (ii) rank(am 1 + bm 2 ) 3 for some (a, b). Need to construct an extreme ray Y of rank 2. rank(am 1 + bm 2 ) = 3 implies that N 1 N 2 is sparse. Get a vector z that is not spanned by any two vectors of N 1 N 2. Use infeasibility of αm 1 + βm 2 0 for (α, β) (0, 0) to get w such that Y = zz T + ww T is tight for both LMIs (w λz for λ R). We reduce the general case of S n to the case of S 3. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 20 / 21 20/21

54 Extensions/Questions Necessary and sufficient conditions for more than 2 LMIs. Use results to analyze conic constraints. Alternate analysis of Burer s work on extensions of the Trust Region Subproblem. Necessary and sufficient conditions for more general conic constraints. C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 21 / 21 21/21

55 Thank you! C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 21 / 21 21/21

56 Further Reading Samuel Burer (2015). A Gentle Geometric Introduction to Copositive Optimization. Mathematical Programming, June 2015, Volume 151, Issue 1, pp Roland Hildebrand (2016). Spectrahedral Cones Generated by Rank-1 Matrices Journal of Global Optimization, Feb. 2016, Volume 64, Issue 2, pp Grigoriy Blekherman et al. (2016). Do Sums of Squares Dream of Free Resolutions? SIAM J. Appl. Algebra Geometry, 1(1), C. Argue, F. Kılınç-Karzan (CMU) ROG spectral cones defined by two LMIs 21 / 21 21/21

The Trust Region Subproblem with Non-Intersecting Linear Constraints

The Trust Region Subproblem with Non-Intersecting Linear Constraints Samuel Burer Boshi Yang February 21, 2013 Abstract This paper studies an extended trust region subproblem (etrs in which the trust region