and Second-Order Cone Program () and Algorithms for Optimization Software Jared Erickson JaredErickson2012@u.northwestern.edu Robert 4er@northwestern.edu Northwestern University INFORMS Annual Meeting, Austin, TX November 8, 2010
and 1 2 3 4 5
Second-Order Cone Programs (s) and Can be written as a quadratic program Not positive semi-definite Convex Efficiently solvable with interior-point methods general form: minimize f T x subject to A i x b i 2 (c T i x d i ) 2 i c T i x d i 0 i
and Previous situation: s can be written in numerous equivalent forms The form a modeler wants to use may not be the form a solver accepts Converting the problem for a particular interior-point solver is tedious and error-prone Ideal situation: Write in modeler s form in a general modeling language Automatically transform to a standard quadratic formulation Transform as necessary for each solver
Motivating Example and minimize Ampl model: (x 2) 2 (y 1) 2 (x y) 2 var x; var y; minimize objective: sqrt((x2)(y1))sqrt((xy)); CPLEX 12.2.0.0: at2372.nl contains a nonlinear objective. KNITRO 6.0.0: Current feasible solution estimate cannot be improved. objective 2.12251253; 30 iterations; 209 function evaluations
Motivating Example and Original: minimize Transformed: (x 2) 2 (y 1) 2 (x y) 2 minimize u v (x 2) 2 (y 1) 2 u 2 (x y) 2 v 2 u, v 0
Motivating Example and Ampl model: var x; var y; var u >= 0; var v >= 0; minimize obj: uv; s.t. C1: (x2)(y1) <= u; s.t. C2: (xy) <= v; CPLEX 12.2.0.0: QP Hessian is not positive semi-definite. KNITRO 6.0.0: Locally optimal solution. objective 2.122027399; 3161 iterations; 3276 function evaluations
Motivating Example and Original: minimize (x 2) 2 (y 1) 2 (x y) 2 Transformed: minimize u v r 2 s 2 u 2 t 2 v 2 x 2 = r y 1 = s x y = t u, v 0
Motivating Example and Ampl model: var x; var y; var u >= 0; var v >= 0; var r; var s; var t; minimize obj: uv; s.t. C1: rs <= u; s.t. C2: t <= v; s.t. C3: x2 = r; s.t. C4: y1 = s; s.t. C5: xy = t; CPLEX 12.2.0.0: primal optimal; objective 2.121320344 5 barrier iterations KNITRO 6.0.0: Locally optimal solution. objective 2.122027305; 3087 iterations; 3088 function evaluations
Generally Accepted Form and general form: minimize f T x subject to A i x b i ci T where x R n is the variable f R n A i R m i,n b i R m i c i R n d i R x d i i
Standard Quadratic Form and Objective: a 1 x 1 a n x n Constraints: Quadratic Cone: b 1 x 2 1 b nx 2 n b 0 x 2 0 0 where b i 0 i, x 0 0 Rotated Quadratic Cone: c 2 x 2 2 c nx 2 n c 1 x 0 x 1 0 where c i 0 i, x 0 0, x 1 0 Linear Inequality: d 0 d 1 x 1 d n x n 0 Linear Equality: e 0 e 1 x 1 e n x n = 0 Variable: k L x
Example and Ampl model: minimize (x 2) 2 (y 1) 2 (x y) 2 var x; var y; minimize objective: sqrt((x2)(y1))sqrt((xy));
First Case: Sum and Max of Norms and Any combination of sum, max, and constant multiple of norms can be represented as a.
Sum of Norms and = minimize subject to minimize p i=1 q i j=1 y i u 2 ij y 2 i p F i x g i i=1 0, i = 1..p (F i x g i ) j u ij = 0, i = 1..p, j = 1..q i y i 0, i = 1..p
Max of Norms and = minimize y subject to minimize max i=1..p F ix g i q i uij 2 y 2 0, i = 1..p j=1 (F i x g i ) j u ij = 0, i = 1..p, j = 1..q i y i 0, i = 1..p
Combination and = minimize 4 max{3 F 1 x g 1 2 F 2 x g 2, 7 F 3 x g 3 } minimize 4y subject to 3u 1 2u 2 y 0 7u 3 y 0 q i j=1 v 2 ij u 2 i 0, i = 1, 2, 3 (F i x g i ) j v ij = 0, i = 1, 2, 3, j = 1..q i u i 0, i = 1, 2, 3
Expression Tree Example and (x 2) 2 (y 1) 2 (x y) 2 sqrt sqrt x y x 2 y 1
Sum and Max of Norms (SMN) Function and Rules for SMN: Constant: f (x) = c is SMN. Variable: f (x) = x i is SMN. Sum: f (x) = n i=1 f i(x) is SMN if all the children f i are SMN. Product: f (x) = cg(x) is SMN if c is a positive constant and g is SMN. Maximum: f (x) = max{f 1 (x),..., f n (x)} is SMN if all the children f i are SMN. Square Root: f (x) = g(x) is SMN if g is NS.
Norm Squared (NS) Function and Rules for NS: Constant: f (x) = c is NS if c 0. Sum: f (x) = n i=1 f i(x) is NS if all the children f i are NS. Product: f (x) = cg(x) is NS if c is a positive constant and g is NS. Squared: f (x) = g(x) 2 is NS if g is linear. Maximum: f (x) = max{f 1 (x),..., f n (x)} is NS if all the children f i are NS.
Example and Sum: f (x) = n i=1 f i(x) is SMN if all the children f i are SMN. sqrt sqrt x y x 2 y 1
Example and Square Root: f (x) = g(x) is SMN if g(x) is NS. sqrt sqrt x y x 2 y 1
Example and Sum: f (x) = n i=1 f i(x) is NS if all the children f i are NS. sqrt sqrt x y x 2 y 1
Example and Squared: f (x) = g(x) 2 is NS if g is linear. sqrt sqrt x y x 2 y 1
Process and 1) Determine objective or constraint type with detection rules 2) Apply corresponding transformation algorithm Separate algorithm, starts at root Uses no information from detection Creates new variables and constraints New constraints are formed by adding terms to functions
Conventions and x: vector of variables in the original formulation v: vector of variables in the original formulation and variables created during transformation f (x), g(x), h(x): functions from the original formulation Functions created during transformation: o(v): objectives (linear) l(v): linear inequalities e(v): linear equalities q(v): quadratic cones r(v): rotated quadratic cones c(v): expressions that could fit in multiple categories
Constraint Building Example and Step 1: l 1 (v) := 3x 1 2 Step 2: l 1 (v) := l 1 (v) v 3 Step 3: l 1 (v) 0 Result: 3x 1 2 v 3 0
Functions and newvar(b) n Introduce new variable v n to variable vector v if b is specified Set lower bound of v n to b else Set lower bound of v n to newfunc(c) m c Introduce new objective or constraint function of type c c mc (v) := 0
transformsmn and transformsmn(f (x), c(v), k) switch case f (x) = g(x) h(x) transformsmn(g(x), c(v), k) transformsmn(h(x), c(v), k) case f (x) = i f i(x) transformsmn(f i (x), c(v), k) i case f (x) = αg(x) transformsmn(g(x), c(v), kα)
transformsmn and case f (x) = max i f i (x) newvar() c(v) := c(v) kv n newfunc(l): l ml (v) := v n for i I transformsmn(f i (x), l ml (v), 1) case f (x) = g(x) newvar(0) c(v) := c(v) kv n newfunc(q): q mq (v) := v 2 n transformns(g(x), q mq (v), 1)
transformns and transformns(f (x), q(v), k) switch case f (x) = g(x) h(x) transformns(g(x), c(v), k) transformns(h(x), c(v), k) case f (x) = i f i(x) transformns(f i (x), c(v), k) i case f (x) = αg(x) transformns(g(x), c(v), kα)
transformns and case f (x) = g(x) 2 newvar() q(v) := q(v) kv 2 n newfunc(e): e me (v) := g(x) v n case f (x) = max i f i (x) newvar() q(v) := q(v) kv 2 n for i I newfunc(q): q mq (v) := v 2 n transformns(f i (x), q mq (v), 1)
Example and f (x) = (x 2) 2 (y 1) 2 (x y) 2 f (x) is SMN apply corresponding transformation algorithm Set all index variables to 0 o(v) := 0 transformsmn(f (x), o(v), 1)
Example and f (x) = (x 2) 2 (y 1) 2 (x y) 2 case f (x) = g(x) h(x) transformsmn(g(x), o(v), 1) transformsmn(h(x), o(v), 1) sqrt sqrt x y x 2 y 1
Example and f (x) = case f (x) = g(x) newvar(0) o(v) := o(v) v 1 newfunc(q): q 1 (v) := v 2 1 transformns(g(x), q 1 (v), 1) (x 2) 2 (y 1) 2 sqrt sqrt x y x 2 y 1
Example and f (x) = (x 2) 2 (y 1) 2 case f (x) = g(x) h(x) transformns(g(x), q 1 (v), k) transformns(h(x), q 1 (v), k) sqrt sqrt x y x 2 y 1
Example and f (x) = (x 2) 2 case f (x) = g(x) 2 newvar() q 1 (v) := q 1 (v) v 2 2 newfunc(e): e 1 (v) := g(x) v 2 sqrt sqrt x y x 2 y 1
Current Functions and o(v) = v 1 q 1 (v) = v2 2 v1 2 e 1 (v) = x 2 v 2 v 1 0
Final Functions and o(v) = v 1 v 4 q 1 (v) = v2 2 v3 2 v1 2 0 e 1 (v) = x 2 v 2 = 0 e 2 (v) = y 1 v 3 = 0 q 2 (v) = v5 2 v4 2 0 e 3 (v) = x y v 5 = 0 v 1 0 v 4 0
Motivating Example and Original: minimize (x 2) 2 (y 1) 2 (x y) 2 Transformed: minimize u v r 2 s 2 u 2 t 2 v 2 x 2 = r y 1 = s x y = t u, v 0
Other Objective Forms and Norm Squared: Fractional: minimize minimize where a i x b i > 0 i Logarithmic Chebyshev: p c i (a i x b i ) 2 i=1 p i=1 c i F i x g i 2 a i x b i where a i x > 0 minimize max i=1..p log(a ix) log(b i )
Other Objective Forms and Product of Positive Powers: p maximize (a i x b i ) α i i=1 where α i > 0, α i Q, a i x b i 0 Product of Negative Powers: p minimize (a i x b i ) π i i=1 where π i > 0, π i Q, a i x b i 0 Combinations of these forms made by sum, max, and positive constant multiple, except Log Chebyshev and some cases of Product of Positive Powers. Example: minimize max{ p (a i xb i ) 2, i=1 q j=1 F j x g j 2 y j } r (c k x) π k k=1
Constraint Forms and Sum and Max of Norms: p c i F i x g i ax b i=1 Norm Squared: p c i (a i x b i ) 2 c 0 (a 0 x b 0 ) 2 i=1 where c 0 0, a 0 x b 0 0 Fractional: p i=1 where a i x b i > 0 i k i F i x g i 2 a i x b i cx d
Constraint Forms and Product of Positive Powers: (a ji x b ji ) π ji cx d j i where a ji x b ji 0, π ji > 0, i π ji 1 j Product of Negative Powers: (a ji x b ji ) π ji cx d j i where a ji x b ji 0, π ji > 0 Combinations of these forms made by sum, max, and positive constant multiple
and Implementation for AMPL and several solvers Paper documenting algorithms Extend to functions not included in AMPL
References and Alizadeh, F. and D. Goldfarb. Second-order cone programming. Math. Program., Ser B 95:3 51, 2003. Lobo, M.S., Vandenberghe, S. Boyd, and H. Lebret. Applications of second order cone programming. Linear Algebra Appl. 284:193 228, 1998. Nesterov, Y. and A. Nemirovski. Interior Point Polynomial Methods in Convex Programming: Theory and Applications. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, 1994.
Thank You and JaredErickson2012@u.northwestern.edu This work was supported in part by grant CMMI-0800662 from the National Science Foundation.