On the local stability of semidefinite relaxations

Transcription

1 On the local stability of semidefinite relaxations Diego Cifuentes Department of Mathematics Massachusetts Institute of Technology Joint work with Sameer Agarwal (Google), Pablo Parrilo (MIT), Rekha Thomas (U. Washington). arxiv: Real Algebraic Geometry and Optimization - ICERM Cifuentes (MIT) Stability of semidefinite relaxations ICERM 18 1 / 21

2 Nearest point problems Given a variety X R n, and a point θ R n, x x θ 2 s.t. x X A variety is the zero set of some polynomials X := {x R n : f 1 (x) = = f m (x) = 0} Cifuentes (MIT) Stability of semidefinite relaxations ICERM 18 2 / 21

3 Nearest point problems Given a variety X R n, and a point θ R n, x x θ 2 s.t. x X A variety is the zero set of some polynomials X := {x R n : f 1 (x) = = f m (x) = 0} This problem is nonconvex, and computationally challenging. SDP relaxations have been successful in several applications. Cifuentes (MIT) Stability of semidefinite relaxations ICERM 18 2 / 21

4 Nearest point problems Given a variety X R n, and a point θ R n, x x θ 2 s.t. x X A variety is the zero set of some polynomials X := {x R n : f 1 (x) = = f m (x) = 0} This problem is nonconvex, and computationally challenging. SDP relaxations have been successful in several applications. Goal Study the behavior of SDP relaxations in the low noise regime: when x is sufficiently close to X. Cifuentes (MIT) Stability of semidefinite relaxations ICERM 18 2 / 21

5 Nearest point problems Many different applications Cifuentes (MIT) Stability of semidefinite relaxations ICERM 18 3 / 21

6 Nearest point to the twisted cubic x X x θ 2, where X := {(x 1, x 2, x 3 ) : x 2 = x 2 1, x 3 = x 1 x 2 } The twisted cubic X can be parametrized as t (t, t 2, t 3 ). Its Lagrangian dual is the following SDP: max γ,λ 1,λ 2 R γ, s.t. γ+ θ 2 θ 1 λ 1 θ 2 λ 2 θ 3 θ 1 1 2λ 1 λ λ 1 θ 2 λ λ 2 θ Cifuentes (MIT) Stability of semidefinite relaxations ICERM 18 4 / 21

7 Nearest point to the twisted cubic x X x θ 2, where X := {(x 1, x 2, x 3 ) : x 2 = x 2 1, x 3 = x 1 x 2 } 3 Nearest point to the twisted cubic zero duality gap = duality gap Cifuentes (MIT) Stability of semidefinite relaxations ICERM 18 4 / 21

8 Nearest point problem to a quadratic variety Theorem If θ X is a regular point then there is zero-duality-gap for any θ R n that is sufficiently close to θ. Applications: Triangulation problem [Aholt-Agarwal-Thomas] Nearest (symmetric) rank one tensor Cifuentes (MIT) Stability of semidefinite relaxations ICERM 18 5 / 21

9 Parametrized QCQPs Consider a family of quadratically constrained programs (QCQPs): x R N g θ (x) h i θ(x) = 0 for i = 1,..., m (P θ ) where g θ, hθ i are quadratic, and the dependence on θ is continuous. The Lagrangian dual is an SDP. Cifuentes (MIT) Stability of semidefinite relaxations ICERM 18 6 / 21

10 Parametrized QCQPs Consider a family of quadratically constrained programs (QCQPs): x R N g θ (x) h i θ(x) = 0 for i = 1,..., m (P θ ) where g θ, hθ i are quadratic, and the dependence on θ is continuous. The Lagrangian dual is an SDP. Goal: Given θ for which the SDP relaxation is tight, analyze the behavior as θ θ. Cifuentes (MIT) Stability of semidefinite relaxations ICERM 18 6 / 21

11 Parametrized QCQPs Consider a family of quadratically constrained programs (QCQPs): x R N g θ (x) h i θ(x) = 0 for i = 1,..., m (P θ ) where g θ, hθ i are quadratic, and the dependence on θ is continuous. The Lagrangian dual is an SDP. Goal: Given θ for which the SDP relaxation is tight, analyze the behavior as θ θ. Example: For a nearest point problem g θ (x) := x θ 2, h i (x) independent of θ The problem is trivial for any θ X. Cifuentes (MIT) Stability of semidefinite relaxations ICERM 18 6 / 21

12 SDP relaxation of a (homogeneous) QCQP Primal problem Dual problem x R N max λ R m x T G θ x x T H i θx = b i i = 1,..., m d(λ) := i λ ib i Q θ (λ) 0 where Q θ (λ) is the Hessian of the Lagrangian (P θ ) (D θ ) Q θ (λ) := G θ + i λ i H i θ S N. Cifuentes (MIT) Stability of semidefinite relaxations ICERM 18 7 / 21

13 SDP relaxation of a (homogeneous) QCQP Primal problem Dual problem x R N max λ R m x T G θ x x T H i θx = b i i = 1,..., m d(λ) := i λ ib i Q θ (λ) 0 (P θ ) (D θ ) Problem statement Assume that val(p θ ) = val(d θ ), i.e., θ is a zero-duality-gap parameter. Find conditions under which val(p θ ) = val(d θ ) when θ is close to θ. Cifuentes (MIT) Stability of semidefinite relaxations ICERM 18 7 / 21

14 Characterization of zero-duality-gap Given x θ primal feasible, its Lagrange multipliers are: Lemma λ Λ θ (x θ ) λ T h θ (x θ ) = g θ (x θ ) Q θ (λ)x θ = 0. Let x θ R N, λ R m. Then x θ is optimal to (P θ ) and λ is optimal to (D θ ) with val(p θ ) = val(d θ ) iff: 1 h θ (x θ ) = 0 (primal feasibility). 2 Q θ (λ) 0 (dual feasibility). 3 λ Λ θ (x θ ) (complementarity). Cifuentes (MIT) Stability of semidefinite relaxations ICERM 18 8 / 21

15 Characterization of zero-duality-gap Given x θ primal feasible, its Lagrange multipliers are: Lemma λ Λ θ (x θ ) λ T h θ (x θ ) = g θ (x θ ) Q θ (λ)x θ = 0. Let x θ R N, λ R m. Then x θ is optimal to (P θ ) and λ is optimal to (D θ ) with val(p θ ) = val(d θ ) iff: 1 h θ (x θ ) = 0 (primal feasibility). 2 Q θ (λ) 0 (dual feasibility). 3 λ Λ θ (x θ ) (complementarity). Proof. If Q θ (λ)x θ = 0 and h θ (x θ ) = 0, then 0 = x T θ Q θ (λ)x θ = x T θ G θ x θ + i λ i x T θ H i x θ = g θ (x θ ) d(λ). Cifuentes (MIT) Stability of semidefinite relaxations ICERM 18 8 / 21

16 Characterization of zero-duality-gap Lemma Let θ be a zero-duality-gap parameter with ( x, λ) primal/dual optimal. Assume that 1 Q θ( λ) has corank-one (strict-complementarity) 2 x θ feasible for (P θ ), λ θ Λ θ (x θ ) s.t. (x θ, λ θ ) θ θ ( x, λ). Then there is zero-duality-gap when θ is close to θ. Proof. Cifuentes (MIT) Stability of semidefinite relaxations ICERM 18 9 / 21

17 Characterization of zero-duality-gap Lemma Let θ be a zero-duality-gap parameter with ( x, λ) primal/dual optimal. Assume that 1 Q θ( λ) has corank-one (strict-complementarity) 2 x θ feasible for (P θ ), λ θ Λ θ (x θ ) s.t. (x θ, λ θ ) θ θ ( x, λ). Then there is zero-duality-gap when θ is close to θ. Proof. Q θ (λ θ ) has a zero eigenvalue (Q θ (λ θ )x θ = 0). Cifuentes (MIT) Stability of semidefinite relaxations ICERM 18 9 / 21

18 Characterization of zero-duality-gap Lemma Let θ be a zero-duality-gap parameter with ( x, λ) primal/dual optimal. Assume that 1 Q θ( λ) has corank-one (strict-complementarity) 2 x θ feasible for (P θ ), λ θ Λ θ (x θ ) s.t. (x θ, λ θ ) θ θ ( x, λ). Then there is zero-duality-gap when θ is close to θ. Proof. Q θ (λ θ ) has a zero eigenvalue (Q θ (λ θ )x θ = 0). Q θ (λ θ ) Q θ( λ) (the dependence on θ is continuous). Cifuentes (MIT) Stability of semidefinite relaxations ICERM 18 9 / 21

19 Characterization of zero-duality-gap Lemma Let θ be a zero-duality-gap parameter with ( x, λ) primal/dual optimal. Assume that 1 Q θ( λ) has corank-one (strict-complementarity) 2 x θ feasible for (P θ ), λ θ Λ θ (x θ ) s.t. (x θ, λ θ ) θ θ ( x, λ). Then there is zero-duality-gap when θ is close to θ. Proof. Q θ (λ θ ) has a zero eigenvalue (Q θ (λ θ )x θ = 0). Q θ (λ θ ) Q θ( λ) (the dependence on θ is continuous). Q θ( λ) has N 1 positive eigenvalues. Cifuentes (MIT) Stability of semidefinite relaxations ICERM 18 9 / 21

20 Characterization of zero-duality-gap Lemma Let θ be a zero-duality-gap parameter with ( x, λ) primal/dual optimal. Assume that 1 Q θ( λ) has corank-one (strict-complementarity) 2 x θ feasible for (P θ ), λ θ Λ θ (x θ ) s.t. (x θ, λ θ ) θ θ ( x, λ). Then there is zero-duality-gap when θ is close to θ. Proof. Q θ (λ θ ) has a zero eigenvalue (Q θ (λ θ )x θ = 0). Q θ (λ θ ) Q θ( λ) (the dependence on θ is continuous). Q θ( λ) has N 1 positive eigenvalues. Q θ (λ θ ) also has N 1 positive eigenvalues (continuity of eigenvalues). Cifuentes (MIT) Stability of semidefinite relaxations ICERM 18 9 / 21

21 Characterization of zero-duality-gap Lemma Let θ be a zero-duality-gap parameter with ( x, λ) primal/dual optimal. Assume that 1 Q θ( λ) has corank-one (strict-complementarity) 2 x θ feasible for (P θ ), λ θ Λ θ (x θ ) s.t. (x θ, λ θ ) θ θ ( x, λ). Then there is zero-duality-gap when θ is close to θ. Proof. Q θ (λ θ ) has a zero eigenvalue (Q θ (λ θ )x θ = 0). Q θ (λ θ ) Q θ( λ) (the dependence on θ is continuous). Q θ( λ) has N 1 positive eigenvalues. Q θ (λ θ ) also has N 1 positive eigenvalues (continuity of eigenvalues). Q θ (λ θ ) 0, so there is zero-duality-gap. Cifuentes (MIT) Stability of semidefinite relaxations ICERM 18 9 / 21

22 Nearest point to a quadratic variety x X x θ 2, where X := {x R n : f 1 (x) = = f m (x) = 0} Theorem Let θ be a regular point of X, i.e. rank f ( θ) = codim X. Then there is zero-duality-gap for θ close to θ. Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

23 Nearest point to a quadratic variety x X x θ 2, where X := {x R n : f 1 (x) = = f m (x) = 0} Theorem Let θ be a regular point of X, i.e. rank f ( θ) = codim X. Then there is zero-duality-gap for θ close to θ. Proof. Since θ X, then x = θ, and λ = 0. Need to find λ θ Λ θ (x θ ) s.t. λ θ θ θ 0. Regularity implies λ θ 2 σ( f ) x θ θ θ θ 0. Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

24 Nearest point to a quadratic variety x X x θ 2, where X := {x R n : f 1 (x) = = f m (x) = 0} Theorem Let θ be a regular point of X, i.e. rank f ( θ) = codim X. Then there is zero-duality-gap for θ close to θ. Proof. Since θ X, then x = θ, and λ = 0. Need to find λ θ Λ θ (x θ ) s.t. λ θ θ θ 0. Regularity implies λ θ 2 σ( f ) x θ θ θ θ 0. Remark: The theorem generalizes to the case of strictly convex objective. Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

25 Guaranteed region of zero-duality-gap x X x θ 2, where X := {x R 3 : x 2 = x 2 1, x 3 = x 1 x 2 } 6 4 zero duality gap Y Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

26 Application: Triangulation [Aholt-Agarwal-Thomas] Problem Given noisy images û j R 2 of an unknown point, u U u j û j 2 where U is the multiview variety of the cameras. j Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

27 Application: Triangulation [Aholt-Agarwal-Thomas] Problem Given noisy images û j R 2 of an unknown point, u U u j û j 2 where U is the multiview variety of the cameras. j If either n = 2, or n 4 and the camera centers are not coplanar, then U is defined by the (quadratic) epipolar constraints. Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

28 Application: Triangulation [Aholt-Agarwal-Thomas] Problem Given noisy images û j R 2 of an unknown point, u U u j û j 2 where U is the multiview variety of the cameras. j If either n = 2, or n 4 and the camera centers are not coplanar, then U is defined by the (quadratic) epipolar constraints. The regularity condition is easy to check. Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

29 Application: Triangulation [Aholt-Agarwal-Thomas] Problem Given noisy images û j R 2 of an unknown point, u U u j û j 2 where U is the multiview variety of the cameras. j If either n = 2, or n 4 and the camera centers are not coplanar, then U is defined by the (quadratic) epipolar constraints. The regularity condition is easy to check. Under low noise the SDP relaxation is tight. Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

30 Application: Rank one approximation Problem Given a tensor ˆx R n 1 n l, consider x X x ˆx 2 where X is the variety of rank one tensors (Segre). Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

31 Application: Rank one approximation Problem Given a tensor ˆx R n 1 n l, consider x X x ˆx 2 where X is the variety of rank one tensors (Segre). The Segre variety is defined by quadratics (2 2 ors). Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

32 Application: Rank one approximation Problem Given a tensor ˆx R n 1 n l, consider x X x ˆx 2 where X is the variety of rank one tensors (Segre). The Segre variety is defined by quadratics (2 2 ors). Thus, the SDP relaxation is tight under low noise. Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

33 Application: Rotation synchronization Problem Given a graph G = (V, E) and matrices ˆR ij R d d for ij E, R 1,...,R n SO(d) R j ˆR ij R i 2 F ij E Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

34 Application: Rotation synchronization Problem Given a graph G = (V, E) and matrices ˆR ij R d d for ij E, R 1,...,R n SO(d) R j ˆR ij R i 2 F ij E The objective function is strictly convex. Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

35 Application: Rotation synchronization Problem Given a graph G = (V, E) and matrices ˆR ij R d d for ij E, R 1,...,R n SO(d) R j ˆR ij R i 2 F ij E The objective function is strictly convex. Thus, the SDP relaxation is tight under low noise. Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

36 Application: Rotation synchronization Problem Given a graph G = (V, E) and matrices ˆR ij R d d for ij E, R 1,...,R n SO(d) R j ˆR ij R i 2 F ij E The objective function is strictly convex. Thus, the SDP relaxation is tight under low noise. Similar tightness results have been shown [Fredriksson-Olsson], [Rosen-Carlone-Bandeira-Leonard], [Wang-Singer]. Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

37 Application: Orthogonal Procrustes Problem Given matrices A R m 1 n, B R m 1 m 2, C R k m 2, X St(n,k) AXC B 2 F where St(n, k) is the Stiefel manifold. The objective function is strictly convex. Thus, the SDP relaxation is tight under low noise. Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

38 Nearest point to non-quadratic varieties Any variety can be described by quadratics by using auxiliary variables. Example: Nearest point problem to the curve y 2 2 = y 3 1 can be written as y θ 2, s.t. y 2 = y 1 z, y 1 = z 2, y 2 z = y1 2. y R 2,z R The objective is not strict convex z y y Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

39 Stability of SDP relaxations of (arbitrary) QCQPs Consider a general family of QCQPs: x R N g θ (x) h i θ(x) = 0 for i = 1,..., m (P θ ) Let θ be a zero-duality-gap parameter: val(p θ ) = val(d θ ). Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

40 Stability of SDP relaxations of (arbitrary) QCQPs Consider a general family of QCQPs: x R N g θ (x) h i θ(x) = 0 for i = 1,..., m (P θ ) Let θ be a zero-duality-gap parameter: val(p θ ) = val(d θ ). There are bad cases, where the SDP relaxation is non-informative. Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

41 Stability of SDP relaxations of (arbitrary) QCQPs Consider a general family of QCQPs: x R N g θ (x) h i θ(x) = 0 for i = 1,..., m (P θ ) Let θ be a zero-duality-gap parameter: val(p θ ) = val(d θ ). There are bad cases, where the SDP relaxation is non-informative. We introduce a Slater-type condition that guarantees zero-duality-gap nearby θ. Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

42 Stability of SDP relaxations of QCQPs Let θ be a zero-duality-gap parameter with ( x, λ) primal/dual optimal. Assumption (restricted Slater) There is µ R m s.t. the quadratic function Ψ µ (x) := i µ ih ī (x) satisfies: θ Ψ µ ( x) = 0, and Ψ µ is strictly convex on ker Q θ( λ). Theorem Under the restricted Slater assumption and some regularity conditions, there is zero-duality-gap when θ is close to θ. Moreover, the SDP recovers the imizer. Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

43 Stability of SDP relaxations of QCQPs Let θ be a zero-duality-gap parameter with ( x, λ) primal/dual optimal. Assumption (restricted Slater) There is µ R m s.t. the quadratic function Ψ µ (x) := i µ ih ī (x) satisfies: θ Ψ µ ( x) = 0, and Ψ µ is strictly convex on ker Q θ( λ). Theorem Under the restricted Slater assumption and some regularity conditions, there is zero-duality-gap when θ is close to θ. Moreover, the SDP recovers the imizer. Applications (ongoing): Higher levels of SOS/Lasserre hierarchy. For instance: system identification, noisy deconvolution, camera resectioning, homography estimation, approximate GCD. Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

44 Using geometry to derive smaller SDP relaxations Primal problem x X x T Gx X = {x : x T H i x = b i for i = 1,..., m} Dual problem max λ R m,q S N i λ ib i Q = G + i λ ih i Q 0 Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

45 Using geometry to derive smaller SDP relaxations Primal problem x X x T Gx X = {x : x T H i x = b i for i = 1,..., m} Dual problem Let ˆx 1,, ˆx S X max λ R m,q S N i λ ib i ˆx T j Qˆx j = ˆx j T Gˆx j + i λ i ˆx j T H i ˆx j for j = 1,..., S Q 0 Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

46 Using geometry to derive smaller SDP relaxations Primal problem x X x T Gx X = {x : x T H i x = b i for i = 1,..., m} Dual problem Let ˆx 1,, ˆx S X max λ R m,q S N i λ ib i ˆx j T Qˆx j = ˆx j T Gˆx j + i λ ib i for j = 1,..., S Q 0 Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

47 Using geometry to derive smaller SDP relaxations Primal problem x X x T Gx X = {x : x T H i x = b i for i = 1,..., m} Dual problem Let ˆx 1,, ˆx S X max γ R,Q S N γ ˆx j T Qˆx j = ˆx j T Gˆx j + γ for j = 1,..., S Q 0 SDP is smaller, e.g., the multipliers λ R m disappear. relaxation is stronger. Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

48 Example: Orthogonal Procrustes Problem Given matrices A R m 1 n, B R m 1 m 2, C R k m 2, X St(n,k) AXC B 2 F where St(n, k) is the Stiefel manifold. n r Equations SDP Gröbner Sampling SDP variables constraints time(s) basis (s) variables constraints time(s) Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

49 Summary We analyzed the local stability of SDP relaxations. Found sufficient conditions for zero-duality-gap nearby θ. Many applications (triangulation, rank one approximation, rotation synchronization, orthogonal Procrustes). Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21

50 Summary We analyzed the local stability of SDP relaxations. Found sufficient conditions for zero-duality-gap nearby θ. Many applications (triangulation, rank one approximation, rotation synchronization, orthogonal Procrustes). If you want to know more: D. Cifuentes, S. Agarwal, P. Parrilo, R. Thomas, On the local stability of semidefinite relaxations, arxiv: D. Cifuentes, C. Harris, B. Sturmfels, The geometry of SDP-exactness in quadratic optimization, arxiv: D. Cifuentes, P. Parrilo, Sampling algebraic varieties for sum of squares programs, arxiv: Thanks for your attention! Cifuentes (MIT) Stability of semidefinite relaxations ICERM / 21