Second-Order Cone Program (SOCP) Detection and Transformation Algorithms for Optimization Software
|
|
- Reginald Barker
- 6 years ago
- Views:
Transcription
1 and Second-Order Cone Program () and Algorithms for Optimization Software Jared Erickson Robert Northwestern University INFORMS Annual Meeting, Austin, TX November 8, 2010
2 and
3 Second-Order Cone Programs (s) and Can be written as a quadratic program Not positive semi-definite Convex Efficiently solvable with interior-point methods general form: minimize f T x subject to A i x b i 2 (c T i x d i ) 2 i c T i x d i 0 i
4 and Previous situation: s can be written in numerous equivalent forms The form a modeler wants to use may not be the form a solver accepts Converting the problem for a particular interior-point solver is tedious and error-prone Ideal situation: Write in modeler s form in a general modeling language Automatically transform to a standard quadratic formulation Transform as necessary for each solver
5 Motivating Example and minimize Ampl model: (x 2) 2 (y 1) 2 (x y) 2 var x; var y; minimize objective: sqrt((x2)(y1))sqrt((xy)); CPLEX : at2372.nl contains a nonlinear objective. KNITRO 6.0.0: Current feasible solution estimate cannot be improved. objective ; 30 iterations; 209 function evaluations
6 Motivating Example and Original: minimize Transformed: (x 2) 2 (y 1) 2 (x y) 2 minimize u v (x 2) 2 (y 1) 2 u 2 (x y) 2 v 2 u, v 0
7 Motivating Example and Ampl model: var x; var y; var u >= 0; var v >= 0; minimize obj: uv; s.t. C1: (x2)(y1) <= u; s.t. C2: (xy) <= v; CPLEX : QP Hessian is not positive semi-definite. KNITRO 6.0.0: Locally optimal solution. objective ; 3161 iterations; 3276 function evaluations
8 Motivating Example and Original: minimize (x 2) 2 (y 1) 2 (x y) 2 Transformed: minimize u v r 2 s 2 u 2 t 2 v 2 x 2 = r y 1 = s x y = t u, v 0
9 Motivating Example and Ampl model: var x; var y; var u >= 0; var v >= 0; var r; var s; var t; minimize obj: uv; s.t. C1: rs <= u; s.t. C2: t <= v; s.t. C3: x2 = r; s.t. C4: y1 = s; s.t. C5: xy = t; CPLEX : primal optimal; objective barrier iterations KNITRO 6.0.0: Locally optimal solution. objective ; 3087 iterations; 3088 function evaluations
10 Generally Accepted Form and general form: minimize f T x subject to A i x b i ci T where x R n is the variable f R n A i R m i,n b i R m i c i R n d i R x d i i
11 Standard Quadratic Form and Objective: a 1 x 1 a n x n Constraints: Quadratic Cone: b 1 x 2 1 b nx 2 n b 0 x where b i 0 i, x 0 0 Rotated Quadratic Cone: c 2 x 2 2 c nx 2 n c 1 x 0 x 1 0 where c i 0 i, x 0 0, x 1 0 Linear Inequality: d 0 d 1 x 1 d n x n 0 Linear Equality: e 0 e 1 x 1 e n x n = 0 Variable: k L x
12 Example and Ampl model: minimize (x 2) 2 (y 1) 2 (x y) 2 var x; var y; minimize objective: sqrt((x2)(y1))sqrt((xy));
13 First Case: Sum and Max of Norms and Any combination of sum, max, and constant multiple of norms can be represented as a.
14 Sum of Norms and = minimize subject to minimize p i=1 q i j=1 y i u 2 ij y 2 i p F i x g i i=1 0, i = 1..p (F i x g i ) j u ij = 0, i = 1..p, j = 1..q i y i 0, i = 1..p
15 Max of Norms and = minimize y subject to minimize max i=1..p F ix g i q i uij 2 y 2 0, i = 1..p j=1 (F i x g i ) j u ij = 0, i = 1..p, j = 1..q i y i 0, i = 1..p
16 Combination and = minimize 4 max{3 F 1 x g 1 2 F 2 x g 2, 7 F 3 x g 3 } minimize 4y subject to 3u 1 2u 2 y 0 7u 3 y 0 q i j=1 v 2 ij u 2 i 0, i = 1, 2, 3 (F i x g i ) j v ij = 0, i = 1, 2, 3, j = 1..q i u i 0, i = 1, 2, 3
17 Expression Tree Example and (x 2) 2 (y 1) 2 (x y) 2 sqrt sqrt x y x 2 y 1
18 Sum and Max of Norms (SMN) Function and Rules for SMN: Constant: f (x) = c is SMN. Variable: f (x) = x i is SMN. Sum: f (x) = n i=1 f i(x) is SMN if all the children f i are SMN. Product: f (x) = cg(x) is SMN if c is a positive constant and g is SMN. Maximum: f (x) = max{f 1 (x),..., f n (x)} is SMN if all the children f i are SMN. Square Root: f (x) = g(x) is SMN if g is NS.
19 Norm Squared (NS) Function and Rules for NS: Constant: f (x) = c is NS if c 0. Sum: f (x) = n i=1 f i(x) is NS if all the children f i are NS. Product: f (x) = cg(x) is NS if c is a positive constant and g is NS. Squared: f (x) = g(x) 2 is NS if g is linear. Maximum: f (x) = max{f 1 (x),..., f n (x)} is NS if all the children f i are NS.
20 Example and Sum: f (x) = n i=1 f i(x) is SMN if all the children f i are SMN. sqrt sqrt x y x 2 y 1
21 Example and Square Root: f (x) = g(x) is SMN if g(x) is NS. sqrt sqrt x y x 2 y 1
22 Example and Sum: f (x) = n i=1 f i(x) is NS if all the children f i are NS. sqrt sqrt x y x 2 y 1
23 Example and Squared: f (x) = g(x) 2 is NS if g is linear. sqrt sqrt x y x 2 y 1
24 Process and 1) Determine objective or constraint type with detection rules 2) Apply corresponding transformation algorithm Separate algorithm, starts at root Uses no information from detection Creates new variables and constraints New constraints are formed by adding terms to functions
25 Conventions and x: vector of variables in the original formulation v: vector of variables in the original formulation and variables created during transformation f (x), g(x), h(x): functions from the original formulation Functions created during transformation: o(v): objectives (linear) l(v): linear inequalities e(v): linear equalities q(v): quadratic cones r(v): rotated quadratic cones c(v): expressions that could fit in multiple categories
26 Constraint Building Example and Step 1: l 1 (v) := 3x 1 2 Step 2: l 1 (v) := l 1 (v) v 3 Step 3: l 1 (v) 0 Result: 3x 1 2 v 3 0
27 Functions and newvar(b) n Introduce new variable v n to variable vector v if b is specified Set lower bound of v n to b else Set lower bound of v n to newfunc(c) m c Introduce new objective or constraint function of type c c mc (v) := 0
28 transformsmn and transformsmn(f (x), c(v), k) switch case f (x) = g(x) h(x) transformsmn(g(x), c(v), k) transformsmn(h(x), c(v), k) case f (x) = i f i(x) transformsmn(f i (x), c(v), k) i case f (x) = αg(x) transformsmn(g(x), c(v), kα)
29 transformsmn and case f (x) = max i f i (x) newvar() c(v) := c(v) kv n newfunc(l): l ml (v) := v n for i I transformsmn(f i (x), l ml (v), 1) case f (x) = g(x) newvar(0) c(v) := c(v) kv n newfunc(q): q mq (v) := v 2 n transformns(g(x), q mq (v), 1)
30 transformns and transformns(f (x), q(v), k) switch case f (x) = g(x) h(x) transformns(g(x), c(v), k) transformns(h(x), c(v), k) case f (x) = i f i(x) transformns(f i (x), c(v), k) i case f (x) = αg(x) transformns(g(x), c(v), kα)
31 transformns and case f (x) = g(x) 2 newvar() q(v) := q(v) kv 2 n newfunc(e): e me (v) := g(x) v n case f (x) = max i f i (x) newvar() q(v) := q(v) kv 2 n for i I newfunc(q): q mq (v) := v 2 n transformns(f i (x), q mq (v), 1)
32 Example and f (x) = (x 2) 2 (y 1) 2 (x y) 2 f (x) is SMN apply corresponding transformation algorithm Set all index variables to 0 o(v) := 0 transformsmn(f (x), o(v), 1)
33 Example and f (x) = (x 2) 2 (y 1) 2 (x y) 2 case f (x) = g(x) h(x) transformsmn(g(x), o(v), 1) transformsmn(h(x), o(v), 1) sqrt sqrt x y x 2 y 1
34 Example and f (x) = case f (x) = g(x) newvar(0) o(v) := o(v) v 1 newfunc(q): q 1 (v) := v 2 1 transformns(g(x), q 1 (v), 1) (x 2) 2 (y 1) 2 sqrt sqrt x y x 2 y 1
35 Example and f (x) = (x 2) 2 (y 1) 2 case f (x) = g(x) h(x) transformns(g(x), q 1 (v), k) transformns(h(x), q 1 (v), k) sqrt sqrt x y x 2 y 1
36 Example and f (x) = (x 2) 2 case f (x) = g(x) 2 newvar() q 1 (v) := q 1 (v) v 2 2 newfunc(e): e 1 (v) := g(x) v 2 sqrt sqrt x y x 2 y 1
37 Current Functions and o(v) = v 1 q 1 (v) = v2 2 v1 2 e 1 (v) = x 2 v 2 v 1 0
38 Final Functions and o(v) = v 1 v 4 q 1 (v) = v2 2 v3 2 v1 2 0 e 1 (v) = x 2 v 2 = 0 e 2 (v) = y 1 v 3 = 0 q 2 (v) = v5 2 v4 2 0 e 3 (v) = x y v 5 = 0 v 1 0 v 4 0
39 Motivating Example and Original: minimize (x 2) 2 (y 1) 2 (x y) 2 Transformed: minimize u v r 2 s 2 u 2 t 2 v 2 x 2 = r y 1 = s x y = t u, v 0
40 Other Objective Forms and Norm Squared: Fractional: minimize minimize where a i x b i > 0 i Logarithmic Chebyshev: p c i (a i x b i ) 2 i=1 p i=1 c i F i x g i 2 a i x b i where a i x > 0 minimize max i=1..p log(a ix) log(b i )
41 Other Objective Forms and Product of Positive Powers: p maximize (a i x b i ) α i i=1 where α i > 0, α i Q, a i x b i 0 Product of Negative Powers: p minimize (a i x b i ) π i i=1 where π i > 0, π i Q, a i x b i 0 Combinations of these forms made by sum, max, and positive constant multiple, except Log Chebyshev and some cases of Product of Positive Powers. Example: minimize max{ p (a i xb i ) 2, i=1 q j=1 F j x g j 2 y j } r (c k x) π k k=1
42 Constraint Forms and Sum and Max of Norms: p c i F i x g i ax b i=1 Norm Squared: p c i (a i x b i ) 2 c 0 (a 0 x b 0 ) 2 i=1 where c 0 0, a 0 x b 0 0 Fractional: p i=1 where a i x b i > 0 i k i F i x g i 2 a i x b i cx d
43 Constraint Forms and Product of Positive Powers: (a ji x b ji ) π ji cx d j i where a ji x b ji 0, π ji > 0, i π ji 1 j Product of Negative Powers: (a ji x b ji ) π ji cx d j i where a ji x b ji 0, π ji > 0 Combinations of these forms made by sum, max, and positive constant multiple
44 and Implementation for AMPL and several solvers Paper documenting algorithms Extend to functions not included in AMPL
45 References and Alizadeh, F. and D. Goldfarb. Second-order cone programming. Math. Program., Ser B 95:3 51, Lobo, M.S., Vandenberghe, S. Boyd, and H. Lebret. Applications of second order cone programming. Linear Algebra Appl. 284: , Nesterov, Y. and A. Nemirovski. Interior Point Polynomial Methods in Convex Programming: Theory and Applications. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, 1994.
46 Thank You and This work was supported in part by grant CMMI from the National Science Foundation.
On the interior of the simplex, we have the Hessian of d(x), Hd(x) is diagonal with ith. µd(w) + w T c. minimize. subject to w T 1 = 1,
Math 30 Winter 05 Solution to Homework 3. Recognizing the convexity of g(x) := x log x, from Jensen s inequality we get d(x) n x + + x n n log x + + x n n where the equality is attained only at x = (/n,...,
More informationInterior Point Methods: Second-Order Cone Programming and Semidefinite Programming
School of Mathematics T H E U N I V E R S I T Y O H F E D I N B U R G Interior Point Methods: Second-Order Cone Programming and Semidefinite Programming Jacek Gondzio Email: J.Gondzio@ed.ac.uk URL: http://www.maths.ed.ac.uk/~gondzio
More informationLargest dual ellipsoids inscribed in dual cones
Largest dual ellipsoids inscribed in dual cones M. J. Todd June 23, 2005 Abstract Suppose x and s lie in the interiors of a cone K and its dual K respectively. We seek dual ellipsoidal norms such that
More informationAdvances in Convex Optimization: Theory, Algorithms, and Applications
Advances in Convex Optimization: Theory, Algorithms, and Applications Stephen Boyd Electrical Engineering Department Stanford University (joint work with Lieven Vandenberghe, UCLA) ISIT 02 ISIT 02 Lausanne
More informationInterior Point Methods. We ll discuss linear programming first, followed by three nonlinear problems. Algorithms for Linear Programming Problems
AMSC 607 / CMSC 764 Advanced Numerical Optimization Fall 2008 UNIT 3: Constrained Optimization PART 4: Introduction to Interior Point Methods Dianne P. O Leary c 2008 Interior Point Methods We ll discuss
More informationInput: System of inequalities or equalities over the reals R. Output: Value for variables that minimizes cost function
Linear programming Input: System of inequalities or equalities over the reals R A linear cost function Output: Value for variables that minimizes cost function Example: Minimize 6x+4y Subject to 3x + 2y
More informationResearch Article Constraint Consensus Methods for Finding Interior Feasible Points in Second-Order Cones
Hindawi Publishing Corporation Journal of Applied Mathematics Volume 21, Article ID 3729, 19 pages doi:1.1155/21/3729 Research Article Constraint Consensus Methods for Finding Interior Feasible Points
More informationLecture 18: Optimization Programming
Fall, 2016 Outline Unconstrained Optimization 1 Unconstrained Optimization 2 Equality-constrained Optimization Inequality-constrained Optimization Mixture-constrained Optimization 3 Quadratic Programming
More informationInderjit Dhillon The University of Texas at Austin
Inderjit Dhillon The University of Texas at Austin ( Universidad Carlos III de Madrid; 15 th June, 2012) (Based on joint work with J. Brickell, S. Sra, J. Tropp) Introduction 2 / 29 Notion of distance
More information15. Conic optimization
L. Vandenberghe EE236C (Spring 216) 15. Conic optimization conic linear program examples modeling duality 15-1 Generalized (conic) inequalities Conic inequality: a constraint x K where K is a convex cone
More informationThe Q-parametrization (Youla) Lecture 13: Synthesis by Convex Optimization. Lecture 13: Synthesis by Convex Optimization. Example: Spring-mass System
The Q-parametrization (Youla) Lecture 3: Synthesis by Convex Optimization controlled variables z Plant distubances w Example: Spring-mass system measurements y Controller control inputs u Idea for lecture
More informationThe Surprisingly Complicated Case of [Convex] Quadratic Optimization
The Surprisingly Complicated Case of [Convex] Quadratic Optimization Robert Fourer 4er@ampl.com AMPL Optimization Inc. www.ampl.com +1 773-336-AMPL U.S.-Mexico Workshop on Optimization and Its Applications
More informationModule 04 Optimization Problems KKT Conditions & Solvers
Module 04 Optimization Problems KKT Conditions & Solvers Ahmad F. Taha EE 5243: Introduction to Cyber-Physical Systems Email: ahmad.taha@utsa.edu Webpage: http://engineering.utsa.edu/ taha/index.html September
More information8. Geometric problems
8. Geometric problems Convex Optimization Boyd & Vandenberghe extremal volume ellipsoids centering classification placement and facility location 8 Minimum volume ellipsoid around a set Löwner-John ellipsoid
More informationApplications of Linear Programming
Applications of Linear Programming lecturer: András London University of Szeged Institute of Informatics Department of Computational Optimization Lecture 9 Non-linear programming In case of LP, the goal
More information4. Convex optimization problems
Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization
More informationLifting for conic mixed-integer programming
Math. Program., Ser. A DOI 1.17/s117-9-282-9 FULL LENGTH PAPER Lifting for conic mixed-integer programming Alper Atamtürk Vishnu Narayanan Received: 13 March 28 / Accepted: 28 January 29 The Author(s)
More informationFRTN10 Multivariable Control, Lecture 13. Course outline. The Q-parametrization (Youla) Example: Spring-mass System
FRTN Multivariable Control, Lecture 3 Anders Robertsson Automatic Control LTH, Lund University Course outline The Q-parametrization (Youla) L-L5 Purpose, models and loop-shaping by hand L6-L8 Limitations
More informationThe Simplest Semidefinite Programs are Trivial
The Simplest Semidefinite Programs are Trivial Robert J. Vanderbei Bing Yang Program in Statistics & Operations Research Princeton University Princeton, NJ 08544 January 10, 1994 Technical Report SOR-93-12
More informationAnalytic Center Cutting-Plane Method
Analytic Center Cutting-Plane Method S. Boyd, L. Vandenberghe, and J. Skaf April 14, 2011 Contents 1 Analytic center cutting-plane method 2 2 Computing the analytic center 3 3 Pruning constraints 5 4 Lower
More informationLecture 7: Convex Optimizations
Lecture 7: Convex Optimizations Radu Balan, David Levermore March 29, 2018 Convex Sets. Convex Functions A set S R n is called a convex set if for any points x, y S the line segment [x, y] := {tx + (1
More information3. Convex functions. basic properties and examples. operations that preserve convexity. the conjugate function. quasiconvex functions
3. Convex functions Convex Optimization Boyd & Vandenberghe basic properties and examples operations that preserve convexity the conjugate function quasiconvex functions log-concave and log-convex functions
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More informationA new primal-dual path-following method for convex quadratic programming
Volume 5, N., pp. 97 0, 006 Copyright 006 SBMAC ISSN 00-805 www.scielo.br/cam A new primal-dual path-following method for convex quadratic programming MOHAMED ACHACHE Département de Mathématiques, Faculté
More informationL. Vandenberghe EE236C (Spring 2016) 18. Symmetric cones. definition. spectral decomposition. quadratic representation. log-det barrier 18-1
L. Vandenberghe EE236C (Spring 2016) 18. Symmetric cones definition spectral decomposition quadratic representation log-det barrier 18-1 Introduction This lecture: theoretical properties of the following
More informationCourse Outline. FRTN10 Multivariable Control, Lecture 13. General idea for Lectures Lecture 13 Outline. Example 1 (Doyle Stein, 1979)
Course Outline FRTN Multivariable Control, Lecture Automatic Control LTH, 6 L-L Specifications, models and loop-shaping by hand L6-L8 Limitations on achievable performance L9-L Controller optimization:
More informationAn interior-point trust-region polynomial algorithm for convex programming
An interior-point trust-region polynomial algorithm for convex programming Ye LU and Ya-xiang YUAN Abstract. An interior-point trust-region algorithm is proposed for minimization of a convex quadratic
More information8. Geometric problems
8. Geometric problems Convex Optimization Boyd & Vandenberghe extremal volume ellipsoids centering classification placement and facility location 8 1 Minimum volume ellipsoid around a set Löwner-John ellipsoid
More informationBCOL RESEARCH REPORT 07.04
BCOL RESEARCH REPORT 07.04 Industrial Engineering & Operations Research University of California, Berkeley, CA 94720-1777 LIFTING FOR CONIC MIXED-INTEGER PROGRAMMING ALPER ATAMTÜRK AND VISHNU NARAYANAN
More informationSequential Quadratic Programming Method for Nonlinear Second-Order Cone Programming Problems. Hirokazu KATO
Sequential Quadratic Programming Method for Nonlinear Second-Order Cone Programming Problems Guidance Professor Masao FUKUSHIMA Hirokazu KATO 2004 Graduate Course in Department of Applied Mathematics and
More informationSemidefinite Programming
Semidefinite Programming Basics and SOS Fernando Mário de Oliveira Filho Campos do Jordão, 2 November 23 Available at: www.ime.usp.br/~fmario under talks Conic programming V is a real vector space h, i
More informationPrimal-Dual Interior-Point Methods. Ryan Tibshirani Convex Optimization
Primal-Dual Interior-Point Methods Ryan Tibshirani Convex Optimization 10-725 Given the problem Last time: barrier method min x subject to f(x) h i (x) 0, i = 1,... m Ax = b where f, h i, i = 1,... m are
More informationBarrier Method. Javier Peña Convex Optimization /36-725
Barrier Method Javier Peña Convex Optimization 10-725/36-725 1 Last time: Newton s method For root-finding F (x) = 0 x + = x F (x) 1 F (x) For optimization x f(x) x + = x 2 f(x) 1 f(x) Assume f strongly
More informationSupplement: Universal Self-Concordant Barrier Functions
IE 8534 1 Supplement: Universal Self-Concordant Barrier Functions IE 8534 2 Recall that a self-concordant barrier function for K is a barrier function satisfying 3 F (x)[h, h, h] 2( 2 F (x)[h, h]) 3/2,
More informationDeterminant maximization with linear. S. Boyd, L. Vandenberghe, S.-P. Wu. Information Systems Laboratory. Stanford University
Determinant maximization with linear matrix inequality constraints S. Boyd, L. Vandenberghe, S.-P. Wu Information Systems Laboratory Stanford University SCCM Seminar 5 February 1996 1 MAXDET problem denition
More informationLecture 6: Conic Optimization September 8
IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions
More informationLMI Methods in Optimal and Robust Control
LMI Methods in Optimal and Robust Control Matthew M. Peet Arizona State University Lecture 02: Optimization (Convex and Otherwise) What is Optimization? An Optimization Problem has 3 parts. x F f(x) :
More informationPrimal-Dual Interior-Point Methods. Ryan Tibshirani Convex Optimization /36-725
Primal-Dual Interior-Point Methods Ryan Tibshirani Convex Optimization 10-725/36-725 Given the problem Last time: barrier method min x subject to f(x) h i (x) 0, i = 1,... m Ax = b where f, h i, i = 1,...
More informationInterior Point Methods for Mathematical Programming
Interior Point Methods for Mathematical Programming Clóvis C. Gonzaga Federal University of Santa Catarina, Florianópolis, Brazil EURO - 2013 Roma Our heroes Cauchy Newton Lagrange Early results Unconstrained
More informationPrimal-Dual Interior-Point Methods. Javier Peña Convex Optimization /36-725
Primal-Dual Interior-Point Methods Javier Peña Convex Optimization 10-725/36-725 Last time: duality revisited Consider the problem min x subject to f(x) Ax = b h(x) 0 Lagrangian L(x, u, v) = f(x) + u T
More informationRobust linear optimization under general norms
Operations Research Letters 3 (004) 50 56 Operations Research Letters www.elsevier.com/locate/dsw Robust linear optimization under general norms Dimitris Bertsimas a; ;, Dessislava Pachamanova b, Melvyn
More informationA CONIC DANTZIG-WOLFE DECOMPOSITION APPROACH FOR LARGE SCALE SEMIDEFINITE PROGRAMMING
A CONIC DANTZIG-WOLFE DECOMPOSITION APPROACH FOR LARGE SCALE SEMIDEFINITE PROGRAMMING Kartik Krishnan Advanced Optimization Laboratory McMaster University Joint work with Gema Plaza Martinez and Tamás
More informationA class of Smoothing Method for Linear Second-Order Cone Programming
Columbia International Publishing Journal of Advanced Computing (13) 1: 9-4 doi:1776/jac1313 Research Article A class of Smoothing Method for Linear Second-Order Cone Programming Zhuqing Gui *, Zhibin
More informationPrimal-dual path-following algorithms for circular programming
Primal-dual path-following algorithms for circular programming Baha Alzalg Department of Mathematics, The University of Jordan, Amman 1194, Jordan July, 015 Abstract Circular programming problems are a
More informationCHAPTER 2: QUADRATIC PROGRAMMING
CHAPTER 2: QUADRATIC PROGRAMMING Overview Quadratic programming (QP) problems are characterized by objective functions that are quadratic in the design variables, and linear constraints. In this sense,
More informationCSCI 1951-G Optimization Methods in Finance Part 09: Interior Point Methods
CSCI 1951-G Optimization Methods in Finance Part 09: Interior Point Methods March 23, 2018 1 / 35 This material is covered in S. Boyd, L. Vandenberge s book Convex Optimization https://web.stanford.edu/~boyd/cvxbook/.
More informationOn self-concordant barriers for generalized power cones
On self-concordant barriers for generalized power cones Scott Roy Lin Xiao January 30, 2018 Abstract In the study of interior-point methods for nonsymmetric conic optimization and their applications, Nesterov
More informationLinear and non-linear programming
Linear and non-linear programming Benjamin Recht March 11, 2005 The Gameplan Constrained Optimization Convexity Duality Applications/Taxonomy 1 Constrained Optimization minimize f(x) subject to g j (x)
More informationLecture: Duality of LP, SOCP and SDP
1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:
More information1 Introduction Semidenite programming (SDP) has been an active research area following the seminal work of Nesterov and Nemirovski [9] see also Alizad
Quadratic Maximization and Semidenite Relaxation Shuzhong Zhang Econometric Institute Erasmus University P.O. Box 1738 3000 DR Rotterdam The Netherlands email: zhang@few.eur.nl fax: +31-10-408916 August,
More informationNumerical Optimization. Review: Unconstrained Optimization
Numerical Optimization Finding the best feasible solution Edward P. Gatzke Department of Chemical Engineering University of South Carolina Ed Gatzke (USC CHE ) Numerical Optimization ECHE 589, Spring 2011
More informationConvex Optimization and l 1 -minimization
Convex Optimization and l 1 -minimization Sangwoon Yun Computational Sciences Korea Institute for Advanced Study December 11, 2009 2009 NIMS Thematic Winter School Outline I. Convex Optimization II. l
More informationA Second-Order Path-Following Algorithm for Unconstrained Convex Optimization
A Second-Order Path-Following Algorithm for Unconstrained Convex Optimization Yinyu Ye Department is Management Science & Engineering and Institute of Computational & Mathematical Engineering Stanford
More informationAgenda. Interior Point Methods. 1 Barrier functions. 2 Analytic center. 3 Central path. 4 Barrier method. 5 Primal-dual path following algorithms
Agenda Interior Point Methods 1 Barrier functions 2 Analytic center 3 Central path 4 Barrier method 5 Primal-dual path following algorithms 6 Nesterov Todd scaling 7 Complexity analysis Interior point
More information4TE3/6TE3. Algorithms for. Continuous Optimization
4TE3/6TE3 Algorithms for Continuous Optimization (Algorithms for Constrained Nonlinear Optimization Problems) Tamás TERLAKY Computing and Software McMaster University Hamilton, November 2005 terlaky@mcmaster.ca
More informationAn E cient A ne-scaling Algorithm for Hyperbolic Programming
An E cient A ne-scaling Algorithm for Hyperbolic Programming Jim Renegar joint work with Mutiara Sondjaja 1 Euclidean space A homogeneous polynomial p : E!R is hyperbolic if there is a vector e 2E such
More informationPOLYNOMIAL OPTIMIZATION WITH SUMS-OF-SQUARES INTERPOLANTS
POLYNOMIAL OPTIMIZATION WITH SUMS-OF-SQUARES INTERPOLANTS Sercan Yıldız syildiz@samsi.info in collaboration with Dávid Papp (NCSU) OPT Transition Workshop May 02, 2017 OUTLINE Polynomial optimization and
More information4. Convex optimization problems
Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization
More informationAgenda. 1 Cone programming. 2 Convex cones. 3 Generalized inequalities. 4 Linear programming (LP) 5 Second-order cone programming (SOCP)
Agenda 1 Cone programming 2 Convex cones 3 Generalized inequalities 4 Linear programming (LP) 5 Second-order cone programming (SOCP) 6 Semidefinite programming (SDP) 7 Examples Optimization problem in
More informationLecture: Convex Optimization Problems
1/36 Lecture: Convex Optimization Problems http://bicmr.pku.edu.cn/~wenzw/opt-2015-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/36 optimization
More informationl p -Norm Constrained Quadratic Programming: Conic Approximation Methods
OUTLINE l p -Norm Constrained Quadratic Programming: Conic Approximation Methods Wenxun Xing Department of Mathematical Sciences Tsinghua University, Beijing Email: wxing@math.tsinghua.edu.cn OUTLINE OUTLINE
More informationAdvanced Mathematical Programming IE417. Lecture 24. Dr. Ted Ralphs
Advanced Mathematical Programming IE417 Lecture 24 Dr. Ted Ralphs IE417 Lecture 24 1 Reading for This Lecture Sections 11.2-11.2 IE417 Lecture 24 2 The Linear Complementarity Problem Given M R p p and
More informationCS295: Convex Optimization. Xiaohui Xie Department of Computer Science University of California, Irvine
CS295: Convex Optimization Xiaohui Xie Department of Computer Science University of California, Irvine Course information Prerequisites: multivariate calculus and linear algebra Textbook: Convex Optimization
More informationInterior Point Algorithms for Constrained Convex Optimization
Interior Point Algorithms for Constrained Convex Optimization Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Inequality constrained minimization problems
More informationLecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.
MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.
More informationCuts for Conic Mixed-Integer Programming
Cuts for Conic Mixed-Integer Programming Alper Atamtürk and Vishnu Narayanan Department of Industrial Engineering and Operations Research, University of California, Berkeley, CA 94720-1777 USA atamturk@berkeley.edu,
More informationInfeasibility Detection in Nonlinear Optimization
Infeasibility Detection in Nonlinear Optimization Frank E. Curtis, Lehigh University Hao Wang, Lehigh University SIAM Conference on Optimization 2011 May 16, 2011 (See also Infeasibility Detection in Nonlinear
More informationMathematics of Data: From Theory to Computation
Mathematics of Data: From Theory to Computation Prof. Volkan Cevher volkan.cevher@epfl.ch Lecture 13: Disciplined convex optimization Laboratory for Information and Inference Systems (LIONS) École Polytechnique
More informationInterior Point Methods in Mathematical Programming
Interior Point Methods in Mathematical Programming Clóvis C. Gonzaga Federal University of Santa Catarina, Brazil Journées en l honneur de Pierre Huard Paris, novembre 2008 01 00 11 00 000 000 000 000
More informationConvex Optimization. (EE227A: UC Berkeley) Lecture 6. Suvrit Sra. (Conic optimization) 07 Feb, 2013
Convex Optimization (EE227A: UC Berkeley) Lecture 6 (Conic optimization) 07 Feb, 2013 Suvrit Sra Organizational Info Quiz coming up on 19th Feb. Project teams by 19th Feb Good if you can mix your research
More information10. Unconstrained minimization
Convex Optimization Boyd & Vandenberghe 10. Unconstrained minimization terminology and assumptions gradient descent method steepest descent method Newton s method self-concordant functions implementation
More informationA semidefinite relaxation scheme for quadratically constrained quadratic problems with an additional linear constraint
Iranian Journal of Operations Research Vol. 2, No. 2, 20, pp. 29-34 A semidefinite relaxation scheme for quadratically constrained quadratic problems with an additional linear constraint M. Salahi Semidefinite
More informationCS-E4830 Kernel Methods in Machine Learning
CS-E4830 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 27. September, 2017 Juho Rousu 27. September, 2017 1 / 45 Convex optimization Convex optimisation This
More informationExact SDP Relaxations for Classes of Nonlinear Semidefinite Programming Problems
Exact SDP Relaxations for Classes of Nonlinear Semidefinite Programming Problems V. Jeyakumar and G. Li Revised Version:August 31, 2012 Abstract An exact semidefinite linear programming (SDP) relaxation
More informationOptimization. Yuh-Jye Lee. March 28, Data Science and Machine Intelligence Lab National Chiao Tung University 1 / 40
Optimization Yuh-Jye Lee Data Science and Machine Intelligence Lab National Chiao Tung University March 28, 2017 1 / 40 The Key Idea of Newton s Method Let f : R n R be a twice differentiable function
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationNonsymmetric potential-reduction methods for general cones
CORE DISCUSSION PAPER 2006/34 Nonsymmetric potential-reduction methods for general cones Yu. Nesterov March 28, 2006 Abstract In this paper we propose two new nonsymmetric primal-dual potential-reduction
More informationHandout 8: Dealing with Data Uncertainty
MFE 5100: Optimization 2015 16 First Term Handout 8: Dealing with Data Uncertainty Instructor: Anthony Man Cho So December 1, 2015 1 Introduction Conic linear programming CLP, and in particular, semidefinite
More informationIntroduction to optimization
Introduction to optimization Geir Dahl CMA, Dept. of Mathematics and Dept. of Informatics University of Oslo 1 / 24 The plan 1. The basic concepts 2. Some useful tools (linear programming = linear optimization)
More informationModeling and Solving Nontraditional Optimization Problems Session 4b: Solver Selection
Modeling and Solving Nontraditional Optimization Problems Session 4b: Solver Selection Robert Fourer Industrial Engineering & Management Sciences Northwestern University AMPL Optimization LLC 4er@northwestern.edu
More informationConvex Optimization and Modeling
Convex Optimization and Modeling Convex Optimization Fourth lecture, 05.05.2010 Jun.-Prof. Matthias Hein Reminder from last time Convex functions: first-order condition: f(y) f(x) + f x,y x, second-order
More informationConvex optimization problems. Optimization problem in standard form
Convex optimization problems optimization problem in standard form convex optimization problems linear optimization quadratic optimization geometric programming quasiconvex optimization generalized inequality
More informationWhat s New in Active-Set Methods for Nonlinear Optimization?
What s New in Active-Set Methods for Nonlinear Optimization? Philip E. Gill Advances in Numerical Computation, Manchester University, July 5, 2011 A Workshop in Honor of Sven Hammarling UCSD Center for
More informationPrimal-Dual Geometry of Level Sets and their Explanatory Value of the Practical Performance of Interior-Point Methods for Conic Optimization
Primal-Dual Geometry of Level Sets and their Explanatory Value of the Practical Performance of Interior-Point Methods for Conic Optimization Robert M. Freund M.I.T. June, 2010 from papers in SIOPT, Mathematics
More informationContinuous Optimisation, Chpt 7: Proper Cones
Continuous Optimisation, Chpt 7: Proper Cones Peter J.C. Dickinson DMMP, University of Twente p.j.c.dickinson@utwente.nl http://dickinson.website/teaching/2017co.html version: 10/11/17 Monday 13th November
More informationFrom Convex Optimization to Linear Matrix Inequalities
Dep. of Information Engineering University of Pisa (Italy) From Convex Optimization to Linear Matrix Inequalities eng. Sergio Grammatico grammatico.sergio@gmail.com Class of Identification of Uncertain
More informationSecond Order Cone Programming Relaxation of Positive Semidefinite Constraint
Research Reports on Mathematical and Computing Sciences Series B : Operations Research Department of Mathematical and Computing Sciences Tokyo Institute of Technology 2-12-1 Oh-Okayama, Meguro-ku, Tokyo
More information3. Convex functions. basic properties and examples. operations that preserve convexity. the conjugate function. quasiconvex functions
3. Convex functions Convex Optimization Boyd & Vandenberghe basic properties and examples operations that preserve convexity the conjugate function quasiconvex functions log-concave and log-convex functions
More informationSF2822 Applied nonlinear optimization, final exam Wednesday June
SF2822 Applied nonlinear optimization, final exam Wednesday June 3 205 4.00 9.00 Examiner: Anders Forsgren, tel. 08-790 7 27. Allowed tools: Pen/pencil, ruler and eraser. Note! Calculator is not allowed.
More informationApplying a type of SOC-functions to solve a system of equalities and inequalities under the order induced by second-order cone
Applying a type of SOC-functions to solve a system of equalities and inequalities under the order induced by second-order cone Xin-He Miao 1, Nuo Qi 2, B. Saheya 3 and Jein-Shan Chen 4 Abstract: In this
More informationConvex functions. Definition. f : R n! R is convex if dom f is a convex set and. f ( x +(1 )y) < f (x)+(1 )f (y) f ( x +(1 )y) apple f (x)+(1 )f (y)
Convex functions I basic properties and I operations that preserve convexity I quasiconvex functions I log-concave and log-convex functions IOE 611: Nonlinear Programming, Fall 2017 3. Convex functions
More informationLecture 3. Optimization Problems and Iterative Algorithms
Lecture 3 Optimization Problems and Iterative Algorithms January 13, 2016 This material was jointly developed with Angelia Nedić at UIUC for IE 598ns Outline Special Functions: Linear, Quadratic, Convex
More informationOptimization. Yuh-Jye Lee. March 21, Data Science and Machine Intelligence Lab National Chiao Tung University 1 / 29
Optimization Yuh-Jye Lee Data Science and Machine Intelligence Lab National Chiao Tung University March 21, 2017 1 / 29 You Have Learned (Unconstrained) Optimization in Your High School Let f (x) = ax
More informationResearch Note. A New Infeasible Interior-Point Algorithm with Full Nesterov-Todd Step for Semi-Definite Optimization
Iranian Journal of Operations Research Vol. 4, No. 1, 2013, pp. 88-107 Research Note A New Infeasible Interior-Point Algorithm with Full Nesterov-Todd Step for Semi-Definite Optimization B. Kheirfam We
More informationMinimizing Cubic and Homogeneous Polynomials Over Integer Points in the Plane
Minimizing Cubic and Homogeneous Polynomials Over Integer Points in the Plane Robert Hildebrand IFOR - ETH Zürich Joint work with Alberto Del Pia, Robert Weismantel and Kevin Zemmer Robert Hildebrand (ETH)
More informationLecture 13: Constrained optimization
2010-12-03 Basic ideas A nonlinearly constrained problem must somehow be converted relaxed into a problem which we can solve (a linear/quadratic or unconstrained problem) We solve a sequence of such problems
More informationNonlinear Programming Models
Nonlinear Programming Models Fabio Schoen 2008 http://gol.dsi.unifi.it/users/schoen Nonlinear Programming Models p. Introduction Nonlinear Programming Models p. NLP problems minf(x) x S R n Standard form:
More informationA PREDICTOR-CORRECTOR PATH-FOLLOWING ALGORITHM FOR SYMMETRIC OPTIMIZATION BASED ON DARVAY'S TECHNIQUE
Yugoslav Journal of Operations Research 24 (2014) Number 1, 35-51 DOI: 10.2298/YJOR120904016K A PREDICTOR-CORRECTOR PATH-FOLLOWING ALGORITHM FOR SYMMETRIC OPTIMIZATION BASED ON DARVAY'S TECHNIQUE BEHROUZ
More informationNewton s Method. Ryan Tibshirani Convex Optimization /36-725
Newton s Method Ryan Tibshirani Convex Optimization 10-725/36-725 1 Last time: dual correspondences Given a function f : R n R, we define its conjugate f : R n R, Properties and examples: f (y) = max x
More informationThe Q Method for Symmetric Cone Programmin
The Q Method for Symmetric Cone Programming The Q Method for Symmetric Cone Programmin Farid Alizadeh and Yu Xia alizadeh@rutcor.rutgers.edu, xiay@optlab.mcma Large Scale Nonlinear and Semidefinite Progra
More information