The Q Method for Symmetric Cone Programming The Q Method for Symmetric Cone Programmin Farid Alizadeh and Yu Xia alizadeh@rutcor.rutgers.edu, xiay@optlab.mcma Large Scale Nonlinear and Semidefinite Progra University of Waterloo, Canada May 2004
The Q Method for Symmetric Cone Programming Basics of Jordan Algebras....... The Symmetric Cone Programs.... The Q Method for SDP........ Polar Decomposition for V....... Optimal Conditions for Symmetric Cone grams.................. An Interior Point Algorithm...... A Newton-type Method........
1 Basics of Jordan Algebras Basics of Jordan Algebras Euclidean Jordan Algebra. 1) A finite dimensional vector space V over R with mapping (product) from V V into V : xy = yx x(x 2 y) = x 2 (xy). 2) There exists an associative, positive definite symm bilinear form on V. Jordan Frame. f i is non-zero and cannot be writte sum of two (necessarily orthogonal) non-zero idempo f 2 i = f i f i f j = 0 (i j) f 1 + + f k = e.
1 Basics of Jordan Algebras Spectral theorem. For x V, there exist a Jorda f 1,..., f r and real numbers λ 1,..., λ r such that x = r λ j f j. (For symmetric matrix X = j=1 r j=1 λ j Symmetric cone. Open, convex, homogeneous, sel cone. There are one-to-one correspondences between cones and Euclidean Jordan algebras. Sq def = {x 2 : x V }, int Sq is a symmetric c x Sq λ i 0.
2 The Symmetric Cone Programs The Symmetric Cone Programs Primal min x c, x s.t. Ax = b x Sq 0 Dual max z, y b, y s.t. A y + z = c z Sq 0 Notations x V, y Y. A: linear mapping from V to Y., def = tr(xy) = r λ i(xy).
3 The Q Method for SDP The Q Method for SDP We extend the Q method for SDP {Alizadeh, Haeberly Overton: SIAM J. Optim. 8 (1998) } to symmetric cone Semidefinite Programming (SDP) Primal min X C X s.t. A i X = b i (i = 1,..., m) X 0 Interior Point Method Dual max y,z s.t. b T y P m i= Z A i X = b i (i = 1,..., m) mx y i A i + Z = C XZ = µi.
3 The Q Method for SDP Facts XZ = µi iff: Q real orthogonal, Λ and Ω diagonal: X = QΛQ T, Z = QΩQ T, ΛΩ = µi. Surjections from skew-symmetric to real ortho matrices: 1. Exponential Function: Q = exp(s) = I + S + 1 2! S2 2. Cayley Transform: Q = (I + 1 S)(I 1 2 2 S) 1 Linear approximation of both: I + S. Basic Ideas of the Q Method Replace (X, Z) by (Q, λ, ω). Replace Q by either of the above mappings and appl
3 The Q Method for SDP The Newton System mx Ω + SΩ ΩS + y i B i = r d B i ( Λ + SΛ ΛS) = (r p ) i (i = 1,... Λ Ω + Ω Λ = µi ΛΩ. Notation: B i = A i Q. Properties 1. No eigenvalue factorization; 2. the Schur complement is symmetric positive definite calculated by Cholesky factorization; 3. the Jacobian is nonsingular stable and highly accu 4. Storage space is small.
4 Polar Decomposition for V Polar Decomposition for V Notation G(Sq) def = {g GL(V ): g(sq) = Sq}. G: connected component of the identity in G(Sq). Polar Decomposition K def = G O(V ), K = Aut 0 (V ). For any Jordan frames f 1,..., f r and d 1,..., d r, A K Af i = d i (1 i r). A Aut(V ) Af i is a Jordan frame. Fix a Jordan frame f 1,..., f r, x V can be decompose x = Q λ i f i (Q K).
5 Optimal Conditions for Symmetric Cone Programs Optimal Conditions for Symmetric C Programs Assume Slater conditions for Primal and Dual (1) sol x, z = 0. Assume x, z in Sq. Then x, z = 0 xz = 0, x and z share a same Jordan fra x = P r λ id i, z = P r ω id i, λ i ω i = 0. Fix a Jordan frame f 1,..., f r. At optimum, replace x an decomposition (2): AQ A y + Q! λ i f i = b! ω i f i = c λ i ω i = 0 λ i 0 ω i 0.
6 An Interior Point Algorithm An Interior Point Algorithm Perturbed system on the central path: Ax = b A y + z = c x, z = µ. x int Sq, µ R xz = µe x and z share a same Jordan frame. Hence Q K : x = Q P r f i, z = Q P r f i, λ i ω i = µ Replace x and z by their decomposition (2).
6 An Interior Point Algorithm Stepsize Update x = Q λ i f i Sq, λ i + λ i 0, Q K Q Q (λ i + λ Linearly Update Q Notations Lie algebra of K: k = m l. W def = a = P r λ if i : λ i R, m def = {S k: a W Exponential Map exp : k K. V is power associative exp ξ = P i=0 Q = P i=0 ξ i i!. S i i! (for S k) linearization of Q is
6 An Interior Point Algorithm The Newton System Notations B k = AQ k, r k p = b Ax k, B k S λ k i fi k + B k λ i f i = r k p (B k ) y + S ωi k f i + r k d = (Q k ) `c z k A y k, (r c ) k i = µ k λ k i ω k i. Properties ω i f i = r k d Λ k ω + Ω k λ = r k c. λ i λ j unique solution for each iteration. Further assume strict complementarity and primal-d nondegeneracy unique solution at optimum.
6 An Interior Point Algorithm The Algorithm Based on an infeasible interior point method {Kojima, Mizuno: Math. Programming 61 (1993) } Stop when ˆ γ p < ɛ p, γ d < ɛ d, λ T ω < ɛ c or ˆ (λ, ω Restrict each iterate in the set N (γ c, γ p, γ d ) def = (λ, ω, y, Q): λ > 0, ω > 0, λ i ω i γ Update each iterate: λ T ω γ p Ax b or Ax b ɛ p, λ T ω γ d A y + z c or A y + z λ + α λ λ y + θ y y ω + β ω ω Q exp(γs) Q Reduce µ at each iteration.
6 An Interior Point Algorithm Convergence Results. If g > 0 such that N > 0, n N so that w = 1, F n w g, where F n is the linear mapping the Newton system at the nth iteration. Then the algor after finite steps. Boundedness of Iterates. Based on {Freund, Jarre, Mizuno: Math. Oper. R (1999)}. Assume an interior feasible solution exists such that ω i δ d. Restrict each iterate in the set Ñ = N { Ax b ζ p, A y + z c where ζ p = 1 δ p, ζ 2 A + d = 1 δ 2 d. Then the iterates are bounded.
7 A Newton-type Method A Newton-type Method Replace the complementarity conditions by some comple function ϕ. Choice of ϕ: min(λ i, ω i ), Optimum Conditions AQ q λ 2 i + ω2 i λ i ω i. λ i f i = b A y + Q ω i f i = c ϕ(λ i, ω i ) = 0 (i = 1,..., r).
7 A Newton-type Method Newton Direction BS B y + S nx λ i f i + B nx ω i f i + Split the index set at optimum into nx λ i f i = r p nx ω i f i = r d p i λ i + q i ω i = r c. L λ = {i: λ i = 0, ω i 0}, L ω = {i: λ i 0, ω i = Assume ϕ satisfy 8 < p i 0 : q i = 0 (i L λ ), 8 < p i = 0 : q i 0 (i L ω ) Then the mapping is one-to-one Pure Newton s meth Q-quadratic convergence rate.
7 A Newton-type Method Numerical Examples (SOCP) The Steiner Minimal Tree Problem {Xue & Ye: SIAM J. Optim., 7 (1997) } Xue & Ye: 23 iterations, feasible initial point, gap redu Interior Point Method: Starting point: x i = (2; 1; 0); z i = (2; 1; 0), y = 0. 34 iterations, total network cost at 28th is better. Accur r p 2, r d 2, r c 1 5.0e 12. Modified Q Method: 29 iterations, total network cost at better. Pure Newton s Method: Perturb each coordinate by a scalar in ( 0.1, 0.1): 4 iter Set point 9 from (2.328223, 9.139549) to (2.5, 9.0): 2 iter