Online generation via offline selection - Low dimensional linear cuts from QP SDP relaxation -
|
|
- Avice Murphy
- 5 years ago
- Views:
Transcription
1 Online generation via offline selection - Low dimensional linear cuts from QP SDP relaxation - Radu Baltean-Lugojan Ruth Misener Computational Optimisation Group Department of Computing Pierre Bonami Andrea Tramontani IBM Research CPLEX Team /03/07 Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 1 / 23
2 Deterministically solving non-convex QP QP : min x T Qx + c T x x s.t. Ax b, x [0, 1] N Prior work: Branch & Cut with relaxations RLT McCormick + ext., e.g. triangle ineq. Bonami et al. [2016] (QP) SDP/SOCP/Convex Dong [2016], Saxena et al. [2011], Buchheim and Wiegele [2013], Zheng et al. [2011], Bao et al. [2011], Anstreicher [2009], Chen and Burer [2012] LP relaxation of SDP (typically high dimensional) e.g. Qualizza et al. [2012], Sherali and Fraticelli [2002] Edge-concave Bao et al. [2009], Misener and Floudas [2012] This work: Off-line (learned) selection of low-dim. LP cuts from SDP Develop online strong low dimensional linear cuts; Offline cut selection via neural net estimator trained a priori ; Cheaply outer-approximate SDP esp. in combination with other low-dim cuts (RLT, triangle, edge-concave, Boolean quadric polytope). Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 2 / 23
3 SDP & RLT relaxations (Anstreicher, J Glob Optim, 2009) min x x T Qx + c T x Ax b, min x Q X + c T x Ax b, X xx T SDP relaxation x [0, 1] N X ii x i X ij x i x j 1 X ij x i 0 RLT relaxation X ij x j 0 x [0, 1] N X [0, 1] N N X ij = X ji, Q X is the matrix inner product Q X = N i,j=1 Q ij X ij, SDP Semidefinite programming, RLT Reformulation linearisation technique. Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 3 / 23
4 SDP & RLT relaxations (Anstreicher, J Glob Optim, 2009) min x Q X + c T x Ax b, X = xx T x [0, 1] N X [0, 1] N N min x Q X + c T x Ax b, X xx T SDP relaxation X ii x i X ij x i x j 1 X ij x i 0 RLT relaxation X ij x j 0 x [0, 1] N X [0, 1] N N X ij = X ji, Q X is the matrix inner product Q X = N i,j=1 Q ij X ij, SDP Semidefinite programming, RLT Reformulation linearisation technique. Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 3 / 23
5 SDP & RLT relaxations (Anstreicher, J Glob Optim, 2009) min x Q X + c T x Ax b, X = xx T x [0, 1] N X [0, 1] N N min x Q X + c T x Ax b, X xx T SDP relaxation X ii x i X ij x i x j 1 X ij x i 0 RLT relaxation X ij x j 0 x [0, 1] N X [0, 1] N N X ij = X ji, Q X is the matrix inner product Q X = N i,j=1 Q ij X ij, SDP Semidefinite programming, RLT Reformulation linearisation technique. Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 3 / 23
6 SDP & RLT relaxations (Anstreicher, J Glob Optim, 2009) min x Q X + c T x Ax b, X = xx T x [0, 1] N X [0, 1] N N min x Q X + c T x Ax b, X xx T SDP relaxation X ii x i X ij x i x j 1 X ij x i 0 RLT relaxation X ij x j 0 x [0, 1] N X [0, 1] N N X ij = X ji, Q X is the matrix inner product Q X = N i,j=1 Q ij X ij, SDP Semidefinite programming, RLT Reformulation linearisation technique. Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 3 / 23
7 SDP & RLT relaxations (Anstreicher, J Glob Optim, 2009) min x Q X + c T x Ax b, X = xx T x [0, 1] N X [0, 1] N N min x Q X + c T x Ax b, X xx T SDP relaxation X ii x i X ij x i x j 1 X ij x i 0 RLT relaxation X ij x j 0 x [0, 1] N X [0, 1] N N X ij = X ji, Q X is the matrix inner product Q X = N i,j=1 Q ij X ij, SDP Semidefinite programming, RLT Reformulation linearisation technique. Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 3 / 23
8 SDP & RLT relaxations: Point Packing (Anstreicher, J Glob Optim, 2009) Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 4 / 23
9 Schur s complement & Decomposition for SDP relaxations Schur s complement SDP Relaxation X = xx T relax X xx T 1 x Schur s complement X xx T T 0 x X Key Idea Consider smaller subsets We have X = xx T, so X S = x S xs T 1 x 1 x 2 x 3 x 4 x 1 X 11 X 12 X 13 X 14 x 2 X 21 X 22 X 23 X 24 x 3 X 31 X 32 X 33 X 34 0 = x 4 X 41 X 42 X 43 X 44 for all S 1, N, e.g. 1 x 1 x 2 x 3 x 1 X 11 X 12 X 13 x 2 X 21 X 22 X 23 0 x 3 X 31 X 32 X 33 Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 5 / 23
10 Sum-additive objective decomposition Recall Power set P n If finite set S has S = n elements, then S has P n = 2 n subsets. min x Q X + c T x Ax b, X = xx T x [0, 1] N X [0, 1] N N = min x S P n Q S X S + c T x Ax b, X = xx T x [0, 1] N X [0, 1] N N Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 6 / 23
11 Initial RLT relaxation min x Q X + c T x Ax b, X = xx T x [0, 1] N X [0, 1] N N min x Q X + c T x Ax b, X ij x i x j 1 i, j X ij x i 0 i, j X ij x j 0 i, j X ij = X ji i, j x [0, 1] N X [0, 1] N N x, X Cutting plane motivation x is feasible in the space of the original QP, For nonconvex QP, X may be infeasible in the original QP, I find LP solvers easier to use than SDP solvers, As in MIP & SAT, may want low-dimensional cutting planes. Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 7 / 23
12 A decomposition of the SDP relaxation for QP (1) QP SDP relaxation min Q X + x,x ct x s.t. Ax b, 1 x T 0, x X x [0, 1] N, X ii x i i (2) QP SDP relaxation at given point x min f (X x) = Q X X 1 x T s.t. 0, X ii x i i x X (3) Relaxed QP SDP relaxation at given x min f S (X S x S ) X S P n 1 x T s.t. S 0 S P n, X ii x i i, x S X S where P n = {S 1, N, S = n N} (*) and, f (X x) = Q X = Q S X S = f S (X S x S ). S P n S P n (4) nd-sdp sub-problems Parameters: n, S, Q S, x S S P n S (XS x S ) = min Q S X S X S 1 x S T s.t. 0, X ii x i i S x S X S f Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 8 / 23
13 A decomposition of the SDP relaxation for QP (1) QP SDP relaxation min Q X + x,x ct x s.t. Ax b, 1 x T 0, x X x [0, 1] N, X ii x i i (2) QP SDP relaxation at given point x min f (X x) = Q X X 1 x T s.t. 0, X ii x i i x X (3) Relaxed QP SDP relaxation at given x min f S (X S x S ) X S P n 1 x T s.t. S 0 S P n, X ii x i i, x S X S where P n = {S 1, N, S = n N} (*) and, f (X x) = Q X = Q S X S = f S (X S x S ). S P n S P n (4) nd-sdp sub-problems Parameters: n, S, Q S, x S S P n S (XS x S ) = min Q S X S X S 1 x S T s.t. 0, X ii x i i S x S X S f Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 8 / 23
14 A decomposition of the SDP relaxation for QP (1) QP SDP relaxation min Q X + x,x ct x s.t. Ax b, 1 x T 0, x X x [0, 1] N, X ii x i i (2) QP SDP relaxation at given point x min f (X x) = Q X X 1 x T s.t. 0, X ii x i i x X (*) We take n = 3, 4, 5 in our experiments. (3) Relaxed QP SDP relaxation at given x min f S (X S x S ) X S P n 1 x T s.t. S 0 S P n, X ii x i i, x S X S where P n = {S 1, N, S = n N} (*) and, f (X x) = Q X = Q S X S = f S (X S x S ). S P n S P n (4) nd-sdp sub-problems Parameters: n, S, Q S, x S S P n S (XS x S ) = min Q S X S X S 1 x S T s.t. 0, X ii x i i S x S X S Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 8 / 23 f
15 A decomposition of the SDP relaxation for QP (1) QP SDP relaxation min Q X + x,x ct x s.t. Ax b, 1 x T 0, x X x [0, 1] N, X ii x i i (2) QP SDP relaxation at given point x min f (X x) = Q X X 1 x T s.t. 0, X ii x i i x X (*) We take n = 3, 4, 5 in our experiments. (3) Relaxed QP SDP relaxation at given x min f S (X S x S ) X S P n 1 x T s.t. S 0 S P n, X ii x i i, x S X S where P n = {S 1, N, S = n N} (*) and, f (X x) = Q X = Q S X S = f S (X S x S ). S P n S P n (4) nd-sdp sub-problems Parameters: n, S, Q S, x S S P n S (XS x S ) = min Q S X S X S 1 x S T s.t. 0, X ii x i i S x S X S Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 8 / 23 f
16 Cut selection from nd-sdp sub-problems at given x Generate outer-approximate hyperplanes for each nd-sdp S P n : f S (XS x S ) = min Q S X S, X S 1 x T s.t. S 0, X ii x i i S x S X S (Given n, S, parametric on Q S, x S ) Selection of nd-sdp sub-problems to cut Combinatorial explosion! # sub-problems = ( N n), need quick optimal selection of a few sub-problems for generating hyperplanes Assume current sol. X, x with S sub-problem objective f S ( X S, x S ) Order/select top S (to cut off X, x via X S, x S ) by estimated objective improvement on X, ObjImpX(S): ( f S (XS x S ) f S ( X ) S, x S ) ˆf S (Q S, x S ) f S ( X S, x S ) = ˆf S (Q S, x S ) Q S X S, where ˆf S (Q S, x S ) is a fast estimator of f S ( x S, X S ). ( ObjImpX(S) ) Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 9 / 23
17 Generating cutting hyperplanes at given X S, x S for one nd-sdp sub-problem Generating hyperplanes Could generate separating hyperplane tangent to SDP cone. In practice, generate cuts from negative eigenvalues, Qualizza et al. [2012]: 1 x vk T T S v k = x S X S = vk T λ k v k = λ k < 0. Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 10 / 23
18 Data for learning estimator ˆf S (Q S, x S ) Estimator only as good as data is sampled Critical that sample space {Q S, x S } is uniform in important features for any learner to generalize well Features: x S (positioning), eigenvalues {λ i } of Q S (positive definiteness) Data sampling Uniform x S [0, 1] n Uniform Q S elements Uniform {λ i } and orthonormal basis Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 11 / 23
19 Neural network as estimator ˆf S (Q S, x S ) Q S x s = ˆf S (Q S, x S ) Why neural nets? ˆf S is a nonlinear regression mapping (collection of convex surfaces) Neural nets (NN): regression via trained hidden layers, no need to specify model Flexible model + lots of well sampled data low variance and bias Architecture (for 3-5D cases) 3-4 hidden layers with neurons tanh activation (well-scaled with our data) in hidden layers Trained by 5-fold cross-validation on 1M data pts. with scaled conjugate gradient Early stop on low gradient (10 5 ) Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 12 / 23
20 Engineering Non-linear activation function Non-linear activation function in the hidden layers Hyperbolic tangent (tanh) vs. Rectified linear unit (ReLU): tanh faster to train, tanh has a bounded output of [ 1, 1], tanh has a significantly positive derivative on the domain [ 4, 4], tanh is symmetric around 0. Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 13 / 23
21 Are the domain and co-domain bounds okay? Lemma. If all eigenvalues of a square matrix M are bounded within [ m, m] then any element in M is bounded within [ m, m]. Let M R n n with eigenvalues and eigenvectors λ i and v i for i 1, n, and let v ij be the j-th element of v i. Then the absolute value of element M ij on the i-th row and j-th column can be expressed as: M ij = v ki v jk λ k k 1,n v ki v jk λ k k 1,n k 1,n k 1,n ((v 2 ki + v 2 jk )/2) λ k ((v 2 ki + v 2 jk )/2)m = m Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 14 / 23
22 Neural network training (3D case) - Results on.5m test set 3D-SDP trained NN (9 inputs layer + 3 hidden layers x 50 neurons) Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 15 / 23
23 Neural network training (4D case) - Results on.5m test set 4D-SDP trained NN (14 inputs layer + 3 hidden layers x 64 neurons) Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 16 / 23
24 Neural network training (5D case) - Results on.5m test set 5D-SDP trained NN (20 inputs layer + 4 hidden layers x 64 neurons) Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 17 / 23
25 Cut selection in practice BoxQP spar rounds of cuts, n = 3, 100 S sub-problems selected by ObjImpX (S) (black lines) Better bound by few cuts After each round, overall bound improving as ObjImpX (S) across S Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 18 / 23
26 Cut selection in practice BoxQP spar rounds of cuts, n = 3, 100 S sub-problems selected by ObjImpX (S) (black lines) Limits - NN error After a few rounds, as ObjImpX (S) NN error: Incorrect selection of S where ObjImpX (S) < 0 Missed selection of S where ObjImpX (S) > 0 Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 19 / 23
27 Results for different problem sizes/densities (n = 3) Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 20 / 23
28 Conclusion Pluses Offline cut selection Good bounds with few low-dimensional linear cuts Easily integrate SDP-based linear cuts with other cut classes in Branch&Cut Minuses Weaker bounds then full SDP or convex-based relaxations Best complemented by other linear cutting planes (e.g. RLT-based) Limited to low-dimensionality cuts Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 21 / 23
29 References I Anstreicher. Semidefinite programming versus the reformulation linearization technique for nonconvex quadratically constrained quadratic programming. J Glob Optim, 43(2-3):471, Bao, Sahinidis, and Tawarmalani. Multiterm polyhedral relaxations for nonconvex, quadratically constrained quadratic programs. Optim Met Softw, 24(4-5):485, Bao, Sahinidis, and Tawarmalani. Semidefinite relaxations for quadratically constrained quadratic programming: A review and comparisons. Math Prog, 129(1):129, Bonami, Günlük, and Linderoth. Solving box-constrained nonconvex quadratic programs Buchheim and Wiegele. Semidefinite relaxations for non-convex quadratic mixed-integer programming. Math Prog, 141(1-2):435, Chen and Burer. Globally solving nonconvex quadratic programming problems via completely positive programming. Math Prog Comput, 4(1):33, Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 22 / 23
30 References II Dong. Relaxing nonconvex quadratic functions by multiple adaptive diagonal perturbations. SIAM J Optim, 26(3):1962, Misener and Floudas. Global optimization of mixed integer quadratically constrained quadratic programs (MIQCQP) through piecewise linear and edge-concave relaxations. Math Prog B, 136:155, Qualizza, Belotti, and Margot. Linear programming relaxations of quadratically constrained quadratic programs. In Lee and Leyffer, editors, Mixed Integer Nonlinear Programming, page Saxena, Bonami, and Lee. Convex relaxations of non-convex mixed integer quadratically constrained programs: Projected formulations. Math Prog, 130: 359, Sherali and Fraticelli. Enhancing RLT relaxations via a new class of semidefinite cuts. J Glob Optim, 22(1-4):233, Zheng, Sun, and Li. Convex relaxations for nonconvex quadratically constrained quadratic programming: matrix cone decomposition and polyhedral approximation. Math Prog, 129(2):301, Baltean-Lugojan, Misener et al. Online cuts by offline selection 2018/03/07 23 / 23
A Note on Representations of Linear Inequalities in Non-Convex Mixed-Integer Quadratic Programs
A Note on Representations of Linear Inequalities in Non-Convex Mixed-Integer Quadratic Programs Adam N. Letchford Daniel J. Grainger To appear in Operations Research Letters Abstract In the literature
More informationNonconvex Quadratic Programming: Return of the Boolean Quadric Polytope
Nonconvex Quadratic Programming: Return of the Boolean Quadric Polytope Kurt M. Anstreicher Dept. of Management Sciences University of Iowa Seminar, Chinese University of Hong Kong, October 2009 We consider
More informationComparing Convex Relaxations for Quadratically Constrained Quadratic Programming
Comparing Convex Relaxations for Quadratically Constrained Quadratic Programming Kurt M. Anstreicher Dept. of Management Sciences University of Iowa European Workshop on MINLP, Marseille, April 2010 The
More informationSemidefinite Relaxations for Non-Convex Quadratic Mixed-Integer Programming
Semidefinite Relaxations for Non-Convex Quadratic Mixed-Integer Programming Christoph Buchheim 1 and Angelika Wiegele 2 1 Fakultät für Mathematik, Technische Universität Dortmund christoph.buchheim@tu-dortmund.de
More informationSolving Box-Constrained Nonconvex Quadratic Programs
Noname manuscript No. (will be inserted by the editor) Solving Box-Constrained Nonconvex Quadratic Programs Pierre Bonami Oktay Günlük Jeff Linderoth June 13, 2016 Abstract We present effective computational
More informationA Binarisation Approach to Non-Convex Quadratically Constrained Quadratic Programs
A Binarisation Approach to Non-Convex Quadratically Constrained Quadratic Programs Laura Galli Adam N. Letchford February 2015 Abstract The global optimisation of non-convex quadratically constrained quadratic
More informationOn Valid Inequalities for Quadratic Programming with Continuous Variables and Binary Indicators
On Valid Inequalities for Quadratic Programming with Continuous Variables and Binary Indicators Hongbo Dong and Jeff Linderoth Wisconsin Institutes for Discovery University of Wisconsin-Madison, USA, hdong6,linderoth}@wisc.edu
More informationThe Trust Region Subproblem with Non-Intersecting Linear Constraints
The Trust Region Subproblem with Non-Intersecting Linear Constraints Samuel Burer Boshi Yang February 21, 2013 Abstract This paper studies an extended trust region subproblem (etrs in which the trust region
More informationSolving Mixed-Integer Nonlinear Programs
Solving Mixed-Integer Nonlinear Programs (with SCIP) Ambros M. Gleixner Zuse Institute Berlin MATHEON Berlin Mathematical School 5th Porto Meeting on Mathematics for Industry, April 10 11, 2014, Porto
More informationCutting Planes for RLT Relaxations of Mixed 0-1 Polynomial Programs
Cutting Planes for RLT Relaxations of Mixed 0-1 Polynomial Programs Franklin Djeumou Fomeni Konstantinos Kaparis Adam N. Letchford December 2013 Abstract The Reformulation-Linearization Technique (RLT),
More informationConvex Quadratic Relaxations of Nonconvex Quadratically Constrained Quadratic Progams
Convex Quadratic Relaxations of Nonconvex Quadratically Constrained Quadratic Progams John E. Mitchell, Jong-Shi Pang, and Bin Yu Original: June 10, 2011 Abstract Nonconvex quadratic constraints can be
More informationCutting Planes for RLT Relaxations of Mixed 0-1 Polynomial Programs
Cutting Planes for RLT Relaxations of Mixed 0-1 Polynomial Programs Franklin Djeumou Fomeni Konstantinos Kaparis Adam N. Letchford To Appear in Mathematical Programming Abstract The Reformulation-Linearization
More informationSemidefinite Programming Basics and Applications
Semidefinite Programming Basics and Applications Ray Pörn, principal lecturer Åbo Akademi University Novia University of Applied Sciences Content What is semidefinite programming (SDP)? How to represent
More informationarxiv: v1 [math.co] 7 Jun 2012
LINEAR PROGRAMMING RELAXATIONS OF QUADRATICALLY CONSTRAINED QUADRATIC PROGRAMS ANDREA QUALIZZA 1, PIETRO BELOTTI 2, AND FRANÇOIS MARGOT 1,3 arxiv:1206.1633v1 [math.co] 7 Jun 2012 Abstract. We investigate
More informationarxiv: v3 [math.oc] 13 Jul 2018
Globally solving Non-Convex Quadratic Programs via Linear Integer Programming techniques Wei Xia, Juan Vera 2 and Luis F. Zuluaga arxiv:5.02423v3 [math.oc] 3 Jul 208 Department of Industrial and Systems
More informationApproximation Algorithms
Approximation Algorithms Chapter 26 Semidefinite Programming Zacharias Pitouras 1 Introduction LP place a good lower bound on OPT for NP-hard problems Are there other ways of doing this? Vector programs
More informationCutting Planes for First Level RLT Relaxations of Mixed 0-1 Programs
Cutting Planes for First Level RLT Relaxations of Mixed 0-1 Programs 1 Cambridge, July 2013 1 Joint work with Franklin Djeumou Fomeni and Adam N. Letchford Outline 1. Introduction 2. Literature Review
More informationc 2000 Society for Industrial and Applied Mathematics
SIAM J. OPIM. Vol. 10, No. 3, pp. 750 778 c 2000 Society for Industrial and Applied Mathematics CONES OF MARICES AND SUCCESSIVE CONVEX RELAXAIONS OF NONCONVEX SES MASAKAZU KOJIMA AND LEVEN UNÇEL Abstract.
More informationStrong formulations for convex functions over nonconvex sets. Daniel Bienstock and Alexander Michalka Columbia University New York.
Strong formulations for convex functions over nonconvex sets Daniel Bienstock and Alexander Michalka Columbia University New York November, 2011 version WedOct102153432012 1 Introduction In this paper
More informationSeparation Techniques for Constrained Nonlinear 0 1 Programming
Separation Techniques for Constrained Nonlinear 0 1 Programming Christoph Buchheim Computer Science Department, University of Cologne and DEIS, University of Bologna MIP 2008, Columbia University, New
More informationCuts for Conic Mixed-Integer Programming
Cuts for Conic Mixed-Integer Programming Alper Atamtürk and Vishnu Narayanan Department of Industrial Engineering and Operations Research, University of California, Berkeley, CA 94720-1777 USA atamturk@berkeley.edu,
More informationOn Non-Convex Quadratic Programming with Box Constraints
On Non-Convex Quadratic Programming with Box Constraints Samuel Burer Adam N. Letchford July 2008 Abstract Non-Convex Quadratic Programming with Box Constraints is a fundamental N P-hard global optimisation
More informationSDP relaxation for a strategic pricing bilevel problem in electricity markets
SDP relaxation for a strategic pricing bilevel problem in electricity markets Marcia Fampa Instituto de Matemática-DCC and COPPE Universidade Federal do Rio de Janeiro C.P. 68530, 21945-970 Rio de Janeiro,
More informationMixed Integer Nonlinear Programming
Mixed Integer Nonlinear Programming IMA New Directions Short Course on Mathematical Optimization Jeff Linderoth and Jim Luedtke Department of Industrial and Systems Engineering University of Wisconsin-Madison
More informationObtaining Tighter Relaxations of Mathematical Programs with Complementarity Constraints
Obtaining Tighter Relaxations of Mathematical Programs with Complementarity Constraints John E. Mitchell, Jong-Shi Pang, and Bin Yu Original: February 19, 2011. Revised: October 11, 2011 Abstract The class
More informationInderjit Dhillon The University of Texas at Austin
Inderjit Dhillon The University of Texas at Austin ( Universidad Carlos III de Madrid; 15 th June, 2012) (Based on joint work with J. Brickell, S. Sra, J. Tropp) Introduction 2 / 29 Notion of distance
More information6-1 The Positivstellensatz P. Parrilo and S. Lall, ECC
6-1 The Positivstellensatz P. Parrilo and S. Lall, ECC 2003 2003.09.02.10 6. The Positivstellensatz Basic semialgebraic sets Semialgebraic sets Tarski-Seidenberg and quantifier elimination Feasibility
More informationSome Recent Advances in Mixed-Integer Nonlinear Programming
Some Recent Advances in Mixed-Integer Nonlinear Programming Andreas Wächter IBM T.J. Watson Research Center Yorktown Heights, New York andreasw@us.ibm.com SIAM Conference on Optimization 2008 Boston, MA
More informationFeasibility Pump for Mixed Integer Nonlinear Programs 1
Feasibility Pump for Mixed Integer Nonlinear Programs 1 Presenter: 1 by Pierre Bonami, Gerard Cornuejols, Andrea Lodi and Francois Margot Mixed Integer Linear or Nonlinear Programs (MILP/MINLP) Optimize
More informationOn global optimization with indefinite quadratics
On global optimization with indefinite quadratics Marcia Fampa Jon Lee Wendel Melo 21 November 2013 Abstract We present an algorithmic framework for global optimization problems in which the non-convexity
More informationDisjunctive Cuts for Mixed Integer Nonlinear Programming Problems
Disjunctive Cuts for Mixed Integer Nonlinear Programming Problems Pierre Bonami, Jeff Linderoth, Andrea Lodi December 29, 2012 Abstract We survey recent progress in applying disjunctive programming theory
More informationConvexification of Mixed-Integer Quadratically Constrained Quadratic Programs
Convexification of Mixed-Integer Quadratically Constrained Quadratic Programs Laura Galli 1 Adam N. Letchford 2 Lancaster, April 2011 1 DEIS, University of Bologna, Italy 2 Department of Management Science,
More informationFeed-forward Network Functions
Feed-forward Network Functions Sargur Srihari Topics 1. Extension of linear models 2. Feed-forward Network Functions 3. Weight-space symmetries 2 Recap of Linear Models Linear Models for Regression, Classification
More informationThe IMA Volumes in Mathematics and its Applications Volume 154
The IMA Volumes in Mathematics and its Applications Volume 154 For further volumes: http://www.springer.com/series/811 Institute for Mathematics and its Applications (IMA) The Institute for Mathematics
More informationCSC Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming
CSC2411 - Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming Notes taken by Mike Jamieson March 28, 2005 Summary: In this lecture, we introduce semidefinite programming
More informationA new family of facet defining inequalities for the maximum edge-weighted clique problem
A new family of facet defining inequalities for the maximum edge-weighted clique problem Franklin Djeumou Fomeni June 2016 Abstract This paper considers a family of cutting planes, recently developed for
More informationQuadratic reformulation techniques for 0-1 quadratic programs
OSE SEMINAR 2014 Quadratic reformulation techniques for 0-1 quadratic programs Ray Pörn CENTER OF EXCELLENCE IN OPTIMIZATION AND SYSTEMS ENGINEERING ÅBO AKADEMI UNIVERSITY ÅBO NOVEMBER 14th 2014 2 Structure
More informationConvex Relaxations of Non-Convex Mixed Integer Quadratically Constrained Programs: Projected Formulations
Anureet Saxena Pierre Bonami Jon Lee Convex Relaxations of Non-Convex Mixed Integer Quadratically Constrained Programs: Projected Formulations November 19, 008 Abstract A common way to produce a convex
More informationIBM Research Report. Convex Relaxations of Non-Convex Mixed Integer Quadratically Constrained Programs: Extended Formulations
RC24621 (W0808-038) August 15, 2008 Mathematics IBM Research Report Convex Relaxations of Non-Convex Mixed Integer Quadratically Constrained Programs: Extended Formulations Anureet Saxena Axioma Inc. 8800
More informationLifting for conic mixed-integer programming
Math. Program., Ser. A DOI 1.17/s117-9-282-9 FULL LENGTH PAPER Lifting for conic mixed-integer programming Alper Atamtürk Vishnu Narayanan Received: 13 March 28 / Accepted: 28 January 29 The Author(s)
More informationHeuristics for nonconvex MINLP
Heuristics for nonconvex MINLP Pietro Belotti, Timo Berthold FICO, Xpress Optimization Team, Birmingham, UK pietrobelotti@fico.com 18th Combinatorial Optimization Workshop, Aussois, 9 Jan 2014 ======This
More informationBCOL RESEARCH REPORT 07.04
BCOL RESEARCH REPORT 07.04 Industrial Engineering & Operations Research University of California, Berkeley, CA 94720-1777 LIFTING FOR CONIC MIXED-INTEGER PROGRAMMING ALPER ATAMTÜRK AND VISHNU NARAYANAN
More informationOn mathematical programming with indicator constraints
On mathematical programming with indicator constraints Andrea Lodi joint work with P. Bonami & A. Tramontani (IBM), S. Wiese (Unibo) University of Bologna, Italy École Polytechnique de Montréal, Québec,
More informationConvex Optimization M2
Convex Optimization M2 Lecture 8 A. d Aspremont. Convex Optimization M2. 1/57 Applications A. d Aspremont. Convex Optimization M2. 2/57 Outline Geometrical problems Approximation problems Combinatorial
More informationLecture Note 5: Semidefinite Programming for Stability Analysis
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State
More informationGlobally solving nonconvex quadratic programming problems via completely positive programming
Math. Prog. Comp. (2012) 4:33 52 DOI 10.1007/s12532-011-0033-9 FULL LENGTH PAPER Globally solving nonconvex quadratic programming problems via completely positive programming Jieqiu Chen Samuel Burer Received:
More informationMixed Integer Non Linear Programming
Mixed Integer Non Linear Programming Claudia D Ambrosio CNRS Research Scientist CNRS & LIX, École Polytechnique MPRO PMA 2016-2017 Outline What is a MINLP? Dealing with nonconvexities Global Optimization
More informationCutting Planes for Mixed Integer Programming
Carnegie Mellon University Research Showcase @ CMU Dissertations Theses and Dissertations 9- Cutting Planes for Mixed Integer Programming Andrea Qualizza Carnegie Mellon University, qualizza@cmu.edu Follow
More informationProjection methods to solve SDP
Projection methods to solve SDP Franz Rendl http://www.math.uni-klu.ac.at Alpen-Adria-Universität Klagenfurt Austria F. Rendl, Oberwolfach Seminar, May 2010 p.1/32 Overview Augmented Primal-Dual Method
More informationSparse Optimization Lecture: Basic Sparse Optimization Models
Sparse Optimization Lecture: Basic Sparse Optimization Models Instructor: Wotao Yin July 2013 online discussions on piazza.com Those who complete this lecture will know basic l 1, l 2,1, and nuclear-norm
More informationMonomial-wise Optimal Separable Underestimators for Mixed-Integer Polynomial Optimization
Monomial-wise Optimal Separable Underestimators for Mixed-Integer Polynomial Optimization Christoph Buchheim Claudia D Ambrosio Received: date / Accepted: date Abstract In this paper we introduce a new
More informationSummer School: Semidefinite Optimization
Summer School: Semidefinite Optimization Christine Bachoc Université Bordeaux I, IMB Research Training Group Experimental and Constructive Algebra Haus Karrenberg, Sept. 3 - Sept. 7, 2012 Duality Theory
More informationLift-and-Project Inequalities
Lift-and-Project Inequalities Q. Louveaux Abstract The lift-and-project technique is a systematic way to generate valid inequalities for a mixed binary program. The technique is interesting both on the
More informationStructured Problems and Algorithms
Integer and quadratic optimization problems Dept. of Engg. and Comp. Sci., Univ. of Cal., Davis Aug. 13, 2010 Table of contents Outline 1 2 3 Benefits of Structured Problems Optimization problems may become
More informationConvex Optimization. (EE227A: UC Berkeley) Lecture 6. Suvrit Sra. (Conic optimization) 07 Feb, 2013
Convex Optimization (EE227A: UC Berkeley) Lecture 6 (Conic optimization) 07 Feb, 2013 Suvrit Sra Organizational Info Quiz coming up on 19th Feb. Project teams by 19th Feb Good if you can mix your research
More informationValid Inequalities and Convex Hulls for Multilinear Functions
Electronic Notes in Discrete Mathematics 36 (2010) 805 812 www.elsevier.com/locate/endm Valid Inequalities and Convex Hulls for Multilinear Functions Pietro Belotti Department of Industrial and Systems
More informationMoments and Positive Polynomials for Optimization II: LP- VERSUS SDP-relaxations
Moments and Positive Polynomials for Optimization II: LP- VERSUS SDP-relaxations LAAS-CNRS and Institute of Mathematics, Toulouse, France Tutorial, IMS, Singapore 2012 LP-relaxations LP- VERSUS SDP-relaxations
More informationDevelopment of an algorithm for solving mixed integer and nonconvex problems arising in electrical supply networks
Development of an algorithm for solving mixed integer and nonconvex problems arising in electrical supply networks E. Wanufelle 1 S. Leyffer 2 A. Sartenaer 1 Ph. Toint 1 1 FUNDP, University of Namur 2
More informationPolynomial Solvability of Variants of the Trust-Region Subproblem
Polynomial Solvability of Variants of the Trust-Region Subproblem Daniel Bienstock Alexander Michalka July, 2013 Abstract We consider an optimization problem of the form x T Qx + c T x s.t. x µ h r h,
More informationLecture: Examples of LP, SOCP and SDP
1/34 Lecture: Examples of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:
More informationIntersection cuts for factorable MINLP
Zuse Institute Berlin Takustr. 7 14195 Berlin Germany FELIPE SERRANO 1 Intersection cuts for factorable MINLP 1 0000-0002-7892-3951 This work has been supported by the Research Campus MODAL Mathematical
More informationConic mixed-integer rounding cuts
Math. Program., Ser. A (2010) 122:1 20 DOI 10.1007/s10107-008-0239-4 FULL LENGTH PAPER Conic mixed-integer rounding cuts Alper Atamtürk Vishnu Narayanan Received: 24 March 2007 / Accepted: 6 May 2008 /
More informationAdvances in CPLEX for Mixed Integer Nonlinear Optimization
Pierre Bonami and Andrea Tramontani IBM ILOG CPLEX ISMP 2015 - Pittsburgh - July 13 2015 Advances in CPLEX for Mixed Integer Nonlinear Optimization 1 2015 IBM Corporation CPLEX Optimization Studio 12.6.2
More informationGlobal Quadratic Minimization over Bivalent Constraints: Necessary and Sufficient Global Optimality Condition
Global Quadratic Minimization over Bivalent Constraints: Necessary and Sufficient Global Optimality Condition Guoyin Li Communicated by X.Q. Yang Abstract In this paper, we establish global optimality
More informationGlobally Solving Nonconvex Quadratic Programming Problems via Completely Positive Programming
ARGONNE NATIONAL LABORATORY 9700 South Cass Avenue Argonne, Illinois 60439 Globally Solving Nonconvex Quadratic Programming Problems via Completely Positive Programming Jieqiu Chen and Samuel Burer Mathematics
More informationCHAPTER 2: QUADRATIC PROGRAMMING
CHAPTER 2: QUADRATIC PROGRAMMING Overview Quadratic programming (QP) problems are characterized by objective functions that are quadratic in the design variables, and linear constraints. In this sense,
More informationDisjunctive Inequalities: Applications and Extensions
Disjunctive Inequalities: Applications and Extensions Pietro Belotti Leo Liberti Andrea Lodi Giacomo Nannicini Andrea Tramontani 1 Introduction A general optimization problem can be expressed in the form
More informationBCOL RESEARCH REPORT 06.03
BCOL RESEARCH REPORT 06.03 Industrial Engineering & Operations Research University of California, Berkeley, CA CONIC MIXED-INTEGER ROUNDING CUTS ALPER ATAMTÜRK AND VISHNU NARAYANAN Abstract. A conic integer
More informationBCOL RESEARCH REPORT 15.04
BCOL RESEARCH REPORT 15.04 Industrial Engineering & Operations Research University of California, Berkeley, CA 94720-1777 Forthcoming in Mathematical Programming A SPATIAL BRANCH-AND-CUT METHOD FOR NONCONVEX
More informationMixed-Integer Nonlinear Programming
Mixed-Integer Nonlinear Programming Claudia D Ambrosio CNRS researcher LIX, École Polytechnique, France pictures taken from slides by Leo Liberti MPRO PMA 2016-2017 Motivating Applications Nonlinear Knapsack
More informationELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications
ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March
More informationI.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More informationIntroduction to Semidefinite Programming I: Basic properties a
Introduction to Semidefinite Programming I: Basic properties and variations on the Goemans-Williamson approximation algorithm for max-cut MFO seminar on Semidefinite Programming May 30, 2010 Semidefinite
More informationGLOBAL OPTIMIZATION WITH GAMS/BARON
GLOBAL OPTIMIZATION WITH GAMS/BARON Nick Sahinidis Chemical and Biomolecular Engineering University of Illinois at Urbana Mohit Tawarmalani Krannert School of Management Purdue University MIXED-INTEGER
More informationCS6375: Machine Learning Gautam Kunapuli. Support Vector Machines
Gautam Kunapuli Example: Text Categorization Example: Develop a model to classify news stories into various categories based on their content. sports politics Use the bag-of-words representation for this
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationRank-one Generated Spectral Cones Defined by Two Homogeneous Linear Matrix Inequalities
Rank-one Generated Spectral Cones Defined by Two Homogeneous Linear Matrix Inequalities C.J. Argue Joint work with Fatma Kılınç-Karzan October 22, 2017 INFORMS Annual Meeting Houston, Texas C. Argue, F.
More informationOutline. 1 Introduction. 2 Modeling and Applications. 3 Algorithms Convex. 4 Algorithms Nonconvex. 5 The Future of MINLP? 2 / 126
Outline 1 Introduction 2 3 Algorithms Convex 4 Algorithms Nonconvex 5 The Future of MINLP? 2 / 126 Larry Leading Off 3 / 126 Introduction Outline 1 Introduction Software Tools and Online Resources Basic
More informationMaximum-Entropy Sampling and the Boolean Quadric Polytope
Journal of Global Optimization manuscript No. (will be inserted by the editor) Maximum-Entropy Sampling and the Boolean Quadric Polytope Kurt M. Anstreicher the date of receipt and acceptance should be
More informationCSCI 1951-G Optimization Methods in Finance Part 10: Conic Optimization
CSCI 1951-G Optimization Methods in Finance Part 10: Conic Optimization April 6, 2018 1 / 34 This material is covered in the textbook, Chapters 9 and 10. Some of the materials are taken from it. Some of
More informationSeparating Doubly Nonnegative and Completely Positive Matrices
Separating Doubly Nonnegative and Completely Positive Matrices Hongbo Dong and Kurt Anstreicher March 8, 2010 Abstract The cone of Completely Positive (CP) matrices can be used to exactly formulate a variety
More information4. Algebra and Duality
4-1 Algebra and Duality P. Parrilo and S. Lall, CDC 2003 2003.12.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationLagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual
More informationInteger and Combinatorial Optimization: Introduction
Integer and Combinatorial Optimization: Introduction John E. Mitchell Department of Mathematical Sciences RPI, Troy, NY 12180 USA November 2018 Mitchell Introduction 1 / 18 Integer and Combinatorial Optimization
More informationLecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.
MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.
More informationA General Framework for Convex Relaxation of Polynomial Optimization Problems over Cones
Research Reports on Mathematical and Computing Sciences Series B : Operations Research Department of Mathematical and Computing Sciences Tokyo Institute of Technology 2-12-1 Oh-Okayama, Meguro-ku, Tokyo
More informationCSC321 Lecture 5: Multilayer Perceptrons
CSC321 Lecture 5: Multilayer Perceptrons Roger Grosse Roger Grosse CSC321 Lecture 5: Multilayer Perceptrons 1 / 21 Overview Recall the simple neuron-like unit: y output output bias i'th weight w 1 w2 w3
More informationDevelopment of the new MINLP Solver Decogo using SCIP - Status Report
Development of the new MINLP Solver Decogo using SCIP - Status Report Pavlo Muts with Norman Breitfeld, Vitali Gintner, Ivo Nowak SCIP Workshop 2018, Aachen Table of contents 1. Introduction 2. Automatic
More information4. Convex optimization problems
Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization
More informationConic optimization under combinatorial sparsity constraints
Conic optimization under combinatorial sparsity constraints Christoph Buchheim and Emiliano Traversi Abstract We present a heuristic approach for conic optimization problems containing sparsity constraints.
More informationThe Strength of Multi-row Aggregation Cuts for Sign-pattern Integer Programs
The Strength of Multi-row Aggregation Cuts for Sign-pattern Integer Programs Santanu S. Dey 1, Andres Iroume 1, and Guanyi Wang 1 1 School of Industrial and Systems Engineering, Georgia Institute of Technology
More informationThere are several approaches to solve UBQP, we will now briefly discuss some of them:
3 Related Work There are several approaches to solve UBQP, we will now briefly discuss some of them: Since some of them are actually algorithms for the Max Cut problem (MC), it is necessary to show their
More informationHeuristics and Upper Bounds for a Pooling Problem with Cubic Constraints
Heuristics and Upper Bounds for a Pooling Problem with Cubic Constraints Matthew J. Real, Shabbir Ahmed, Helder Inàcio and Kevin Norwood School of Chemical & Biomolecular Engineering 311 Ferst Drive, N.W.
More informationCSC Linear Programming and Combinatorial Optimization Lecture 12: The Lift and Project Method
CSC2411 - Linear Programming and Combinatorial Optimization Lecture 12: The Lift and Project Method Notes taken by Stefan Mathe April 28, 2007 Summary: Throughout the course, we have seen the importance
More informationConvex hull of two quadratic or a conic quadratic and a quadratic inequality
Noname manuscript No. (will be inserted by the editor) Convex hull of two quadratic or a conic quadratic and a quadratic inequality Sina Modaresi Juan Pablo Vielma the date of receipt and acceptance should
More informationRelaxations and Randomized Methods for Nonconvex QCQPs
Relaxations and Randomized Methods for Nonconvex QCQPs Alexandre d Aspremont, Stephen Boyd EE392o, Stanford University Autumn, 2003 Introduction While some special classes of nonconvex problems can be
More informationImproved quadratic cuts for convex mixed-integer nonlinear programs
Improved quadratic cuts for convex mixed-integer nonlinear programs Lijie Su a,b, Lixin Tang a*, David E. Bernal c, Ignacio E. Grossmann c a Institute of Industrial and Systems Engineering, Northeastern
More informationE5295/5B5749 Convex optimization with engineering applications. Lecture 5. Convex programming and semidefinite programming
E5295/5B5749 Convex optimization with engineering applications Lecture 5 Convex programming and semidefinite programming A. Forsgren, KTH 1 Lecture 5 Convex optimization 2006/2007 Convex quadratic program
More information