INTERIOR-POINT METHODS ROBERT J. VANDERBEI JOINT WORK WITH H. YURTTAN BENSON REAL-WORLD EXAMPLES BY J.O. COLEMAN, NAVAL RESEARCH LAB
|
|
- Berniece Holmes
- 5 years ago
- Views:
Transcription
1 1 INTERIOR-POINT METHODS FOR SECOND-ORDER-CONE AND SEMIDEFINITE PROGRAMMING ROBERT J. VANDERBEI JOINT WORK WITH H. YURTTAN BENSON REAL-WORLD EXAMPLES BY J.O. COLEMAN, NAVAL RESEARCH LAB
2 Outline 2 Introduction and Review of Problem Classes: LP, NLP, SOCP, SDP Formulating SOCPs as Smooth Convex NLPs Formulating SDPs as Smooth Convex NLPs Applications and Computational Results
3 Introduction and Review of Problem Classes: LP, NLP, SOCP, SDP 3
4 Traditional Families of Optimization Problems 4 Linear Programming (LP) minimize subject to c T x Ax = b x 0. Smooth Convex Nonlinear Programming (NLP) We assume that minimize f(x) subject to h i (x) = 0, i E, h i (x) 0, i I. h i s in equality constraints are affine; h i s in inequality constraints are concave; f is convex; All are twice continuously differentiable.
5 Two New Families of Optimization Problems 5 Second-Order Cone Programming (SOCP) minimize f T x subject to A i x + b i c T i x + d i, i = 1,..., m, Here, A i is a k i n matrix, b i is a k i -vector, c i is an n-vector, and d i is a scalar. Semidefinite Programming (SDP) minimize f(x) subject to h i (X) = 0, i E, X 0. Here, X is an n n symmetric matrix, h i is affine, and X 0 means that X is to be positive semidefinite.
6 Preview of Applications 6 Second-Order Cone Programming FIR Filter Design Antenna Array Optimization Structural Optimization Grasping Problems Semidefinite Programming SDP relaxations of combinatorial optimization problems such as MAX- CUT have been shown to be better than LP relaxations Eigenvalue optimization: maximize the minimum eigenvalue More later on some of these.
7 Interior-Point Algorithms 7 Interior-point methods were first developed in the mid 80 s for LP. Later they were extended to NLP, SOCP, and SDP. Extension to NLP follows closely the LP case. That is, is treated the same in both cases. The nonnegative-orthant cone, x 0, plays a fundamental role. For SOCP, a different cone is introduced, the Lorentz cone, and algorithms are derived using this cone in place of the nonnegative orthant cone. Similarly for SDP, a different cone, the cone of positive semidefinite matrices, plays a fundamental role in algorithm development.
8 Our Aim A Large Enclosing Superclass 8 Recently, SOCP and SDP have been unified under the banner of Conic Programming and software has appeared to solve problems from the union of the SOCP and SDP problem classes. The aim of our work is to show that SOCP and SDP can be included under the banner of NLP and solved with generic NLP software. This is a much more general setting.
9 Formulating SOCPs as Smooth Convex NLPs 9
10 SOCP as NLP 10 For SOCP, h i (x) = c T i x + d i A i x + b i is concave but not differentiable on {x : A i x + b i = 0}. Nondifferentiability should not be a problem unless it happens at optimality...
11 An Example 11 minimize ax 1 + x 2 ( 1 < a < 1) subject to x 1 x 2, Clearly, (x 1, x 2 ) = (0, 0). Dual feasibility: [ a 1 ] + [ d x1 dx 1 1 ] y = 0. An interior-point method must pick the correct value for d x 1 when x dx 1 = 0: 1 d x 1 dx = a. 1 x1 =0 Not possible a priori.
12 Smooth Alternative Formulations 12 Constraint formulation: φ(a i x + b i, c T i x + d i) 0, where φ(u, t) = t u. Not differentiable at u = 0. Smooth alternatives: i = 1,..., m t ɛ 2 + i u2 i 0, concave not equiv. t 2 u 2 0, t 0 nonconcave equiv. t u 2 /t 0, t > 0 concave equiv. strict interior Third suggestion is due to E.D. Andersen
13 Formulating SDPs as Smooth Convex NLPs 13
14 Formulating SDPs as NLPs 14 How to express minimize f(x) subject to h i (X) = 0, i E, X 0. as minimize f(x) subject to h i (x) = 0, i E, h i (x) 0, i I.
15 Characterizations of Semidefiniteness 15 The Definition of SDP Semiinfinite LP ξ T Xξ 0 ξ R n. Involves an infinite number of constraints. Smallest Eigenvalue Nonsmooth Convex NLP λ min (X) 0. The smallest eigenvalue as a function of the matrix X is concave, but it is not smooth as the following example shows: [ ] x1 y X =. y x 2 Eigenvalues are: λ 1 (X) = λ 2 (X) = ((x 1 + x 2 ) ) (x 1 x 2 ) 2 + 4y 2 /2 ((x 1 + x 2 ) + ) (x 1 x 2 ) 2 + 4y 2 /2.
16 Characterizations of Semidefiniteness Continued 16 Factorization Smooth Convex NLP For j = 1,..., n, let { Djj, where X = LDL d j (X) = T, X 0,, else. then X 0 if and only if d j (X) 0 j. Theorem 1. The functions d j are (1) concave, (2) continuous over the positive semidefinite matrices, and (3) infinitely differentiable over the positive definite matrices.
17 First and Second Derivatives Explicit Formulae 17 Fix j and partition X: X = j j Z y y T x. Assume that X 0. Then Z is nonsingular and d j (X) = x y T Z 1 y. Derivatives of d j are easily obtained from derivatives of And they are... f(y, Z) = y T Z 1 y.
18 Derivatives (Fletcher 85) 18 The derivatives of f(y, Z) are: f y i = y T Z 1 e i + e T i Z 1 y, 2 f y i y j = e T j Z 1 e i + e T i Z 1 e j, f z ij = y T Z 1 e i e T j Z 1 y, 2 f z ij z kl = y T Z 1 e k e T l Z 1 e i e T j Z 1 y + y T Z 1 e i e T j Z 1 e k e T l Z 1 y, 2 f y i z kl = y T Z 1 e k e T l Z 1 e i e T i Z 1 e k e T l Z 1 y. From these it is easy to check that f is convex and hence that d j is concave.
19 LOQO A Specific Interior-Point Algorithm for NLP 19 The problem: minimize f(x) subject to h i (x) = 0, i E, h i (x) 0, i I. is represented internally by LOQO as: minimize f(x) subject to h i (x) w i = 0, i E, h i (x) w i = 0, i I, x j g j + t j = 0, j = 1,..., n, w i + p i = 0, i E, w i 0, i E I, p i 0, i E, g j 0, j = 1,..., n, t j 0, j = 1,..., n.
20 Complementarity 20 Each nonnegative variable has a complementary dual variable: w i y i, p i q i, g i z i, t i s i. Using LOQO to solve SDPs LOQO preserves positivity of each nonnegative variable. It does not preserve positivity of h i. In fact, without further precaution, h i does indeed go negative. This is bad for SDP!
21 Step Shortening 21 Steps lengths can be shortened to guarantee that h i s remain positive. But then the algorithm could jam. Next theorem shows that simple jamming does not occur. To be precise, we consider a point ( x, w, ȳ, p, q, ḡ, z, t, s) satisfying the following conditions: (1) Nonnegativity: w 0, ȳ 0, p 0, q 0, ḡ 0, z 0, t 0, and s 0. (2) Complementary variable positivity: w + ȳ > 0, p + q > 0, ḡ + z > 0, and t + s > 0. (3) Free variable positivity: ḡ + t > 0.
22 Nonjamming Theorem 22 We are interested in a point where some of the w i s vanish. Let U = {i : w i = 0} and let B denote the other indices. Write A = ( h T ) T (and other matrices and vectors) in block form according to this partition [ ] B A = U Matrix A and other quantities are functions of the current point. We use the same letter with a bar over it to denote the value of these objects at the point ( x,..., s). Theorem 2. If the point ( x,..., s) satisfies conditions (1) (3), and Ū has full row rank, then the step direction w for w has a continuous extension to this point and w U = µy 1 U e U > 0 there.
23 An Example 23 We used LOQO to solve the following simple problem: minimize 3 j=1 X jj subject to X 12 = 1 X 13 = 1.5 X 23 = 2 X 0. Without step shortening, LOQO fails. With step shortening, it solves in less than 20 iterations.
24 Applications and Computational Results 24
25 Finite Impulse Response (FIR) Filter Design Low Pass Filter minimize ρ 25 where H(ν) = subject to 19 k= 5 ( 1 L L ( 1 (H(ν) 1) 2 dν) 1/2 ρ H h(k)e 2πikν, H H 2 (ν)dν) 1/2 ρ h(k) = Complex filter coefficients, i.e., decision variables
26 Specific Example 26 Ref: J.O. Coleman and D.P. Scholnik, U.S. Naval Research Laboratory, constraints 1880 variables 1648 time (iterations) LOQO 17.7 (33) SEDUMI(Sturm) 46.1 (18) MWSCAS99 paper available: engr.umbc.edu/ jeffc/pubs/abstracts/mwscas99socp.html Click here for an animation.
27 FIR Filter Design Woofer, Midrange, Tweeter minimize ρ 27 subject to where 1 0 (H w (ν) + H m (ν) + H t (ν) 1) 2 dν ɛ ( 1 W ( 1 M ( 1 H i (ν) = h i (0) + 2 T n 1 k=1 W M T H 2 w (ν)dν ) 1/2 ρ W = [.2,.8] H 2 m (ν)dν ) 1/2 ρ M = [.4,.6] [.9 H 2 t (ν)dν ) 1/2 ρ T = [.7,.3] h i (k) cos(2πkν), h i (k) = filter coefficients, i.e., decision variables i = w, m, t
28 28
29 29 Specific Example: Pink Floyd s Money filter length: n = 14 integral discretization: N = 1000 constraints 4 variables 43 time (secs) LOQO 79 MINOS 164 LANCELOT 3401 SNOPT 35 Ref: J.O. Coleman, U.S. Naval Research Laboratory, CISS98 paper available: engr.umbc.edu/ jeffc/pubs/abstracts/ciss98.html Click here for a demo
30 Wide-Band Antenna Array Weight Design SOCP 30 minimize subject to α (µ,ν) S A(µ, ν) 2 dµdν α, A(µ, ν) 10 25/20, (µ, ν) S where A(µ, ν) = k A(µ, ν) 10 45/20, (µ, ν) S 0 ν P A(µ m, ν) β m 2 dν 10 50/10, m β m = M k n c kn 2 ρ c kn e 2πi(kµ+nν) n m = 1,..., M c kn = complex-valued design weight for array element k at freq tap n S = subset of direction/freq pairs representing sidelobe S 0 = subset of sidelobe spelling NRL µ m = finite set of directions covering pass band ρ = Noise bound
31 Specific Example antennae in a linear array 21 taps on each array 671 Chebychev constraints to spell NRL constraints 6230 variables 624 time (iterations) LOQO 1049 (48) SEDUMI(Sturm) 573 (27) Ref: D.P. Scholnik and J.O. Coleman, U.S. Naval Research Laboratory, RADAR2000 paper available: engr.umbc.edu/ jeffc/pubs/papers/radar2kds/radar2000.html
32 32
33 Max-Cut Problem SDP 33 minimize C X e T CDiag(X) subject to X jj = 1/4, j = 1,..., n, X 0. where C is a symmetric incidence matrix; i.e., C ij = 1 if {i, j} is an arc and zero otherwise. Specific example: C is a arc incidence matrix for a randomly generated graph with 1.5 arcs/node on average. LOQO solves this problem in 27 iterations and 43.0 seconds.
34 Max-Min Eigenvalue SDP 34 minimize C X subject to X kk = 1, k = 1,..., n, X 0. Specific example: C is a random symmetric matrix. LOQO solves the problem in 27 iterations and 43.1 seconds.
35 Exploiting Sparsity 35 The computational results shown for SDP don t properly exploit sparsity. For SDP, one way to exploit sparsity is to solve the dual problem: maximize subject to k b k y k A (k) ij y k Z ij = C ij, k Z 0. Typically, C and each of the A (k) are sparse matrices. i, j = 1, 2,..., n The equality constraints then imply that Z must have a sparsity pattern that is the union of the sparsity patterns of these other matrices. Therefore, Z is often sparse too, and its sparsity pattern is known from the beginning. This sparsity is easy to exploit, although we haven t done that yet. Also, derivative calculatioins require Z 1 too.
On Solving SOCPs with an Interior-Point Method for NLP
On Solving SOCPs with an Interior-Point Method for NLP Robert Vanderbei and Hande Y. Benson INFORMS Nov 5, 2001 Miami Beach, FL Operations Research and Financial Engineering, Princeton University http://www.princeton.edu/
More informationNonlinear Programming Models: A Survey
Nonlinear Programming Models: A Survey Robert Vanderbei and David Shanno NJ-INFORMS Nov 16, 2000 Rutgers University Operations Research and Financial Engineering, Princeton University http://www.princeton.edu/
More informationSOLVING PROBLEMS WITH SEMIDEFINITE AND RELATED CONSTRAINTS USING INTERIOR-POINT METHODS FOR NONLINEAR PROGRAMMING
SOLVING PROBLEMS WITH SEMIDEFINITE AND RELATED CONSTRAINTS SING INTERIOR-POINT METHODS FOR NONLINEAR PROGRAMMING ROBERT J. VANDERBEI AND HANDE YRTTAN BENSON Operations Research and Financial Engineering
More informationLecture: Introduction to LP, SDP and SOCP
Lecture: Introduction to LP, SDP and SOCP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2015.html wenzw@pku.edu.cn Acknowledgement:
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More information6-1 The Positivstellensatz P. Parrilo and S. Lall, ECC
6-1 The Positivstellensatz P. Parrilo and S. Lall, ECC 2003 2003.09.02.10 6. The Positivstellensatz Basic semialgebraic sets Semialgebraic sets Tarski-Seidenberg and quantifier elimination Feasibility
More informationCSCI 1951-G Optimization Methods in Finance Part 10: Conic Optimization
CSCI 1951-G Optimization Methods in Finance Part 10: Conic Optimization April 6, 2018 1 / 34 This material is covered in the textbook, Chapters 9 and 10. Some of the materials are taken from it. Some of
More informationHW1 solutions. 1. α Ef(x) β, where Ef(x) is the expected value of f(x), i.e., Ef(x) = n. i=1 p if(a i ). (The function f : R R is given.
HW1 solutions Exercise 1 (Some sets of probability distributions.) Let x be a real-valued random variable with Prob(x = a i ) = p i, i = 1,..., n, where a 1 < a 2 < < a n. Of course p R n lies in the standard
More informationSemidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5
Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5 Instructor: Farid Alizadeh Scribe: Anton Riabov 10/08/2001 1 Overview We continue studying the maximum eigenvalue SDP, and generalize
More informationSemidefinite Programming Basics and Applications
Semidefinite Programming Basics and Applications Ray Pörn, principal lecturer Åbo Akademi University Novia University of Applied Sciences Content What is semidefinite programming (SDP)? How to represent
More informationELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications
ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March
More informationConvex Optimization of Graph Laplacian Eigenvalues
Convex Optimization of Graph Laplacian Eigenvalues Stephen Boyd Stanford University (Joint work with Persi Diaconis, Arpita Ghosh, Seung-Jean Kim, Sanjay Lall, Pablo Parrilo, Amin Saberi, Jun Sun, Lin
More informationPrimal-Dual Geometry of Level Sets and their Explanatory Value of the Practical Performance of Interior-Point Methods for Conic Optimization
Primal-Dual Geometry of Level Sets and their Explanatory Value of the Practical Performance of Interior-Point Methods for Conic Optimization Robert M. Freund M.I.T. June, 2010 from papers in SIOPT, Mathematics
More informationLecture Note 5: Semidefinite Programming for Stability Analysis
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State
More informationminimize x subject to (x 2)(x 4) u,
Math 6366/6367: Optimization and Variational Methods Sample Preliminary Exam Questions 1. Suppose that f : [, L] R is a C 2 -function with f () on (, L) and that you have explicit formulae for
More informationLinear Programming: Chapter 1 Introduction
Linear Programming: Chapter 1 Introduction Robert J. Vanderbei October 17, 2007 Operations Research and Financial Engineering Princeton University Princeton, NJ 08544 http://www.princeton.edu/ rvdb Resource
More informationIntroduction to Nonlinear Stochastic Programming
School of Mathematics T H E U N I V E R S I T Y O H F R G E D I N B U Introduction to Nonlinear Stochastic Programming Jacek Gondzio Email: J.Gondzio@ed.ac.uk URL: http://www.maths.ed.ac.uk/~gondzio SPS
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationOptimality, Duality, Complementarity for Constrained Optimization
Optimality, Duality, Complementarity for Constrained Optimization Stephen Wright University of Wisconsin-Madison May 2014 Wright (UW-Madison) Optimality, Duality, Complementarity May 2014 1 / 41 Linear
More informationApplications of Linear Programming
Applications of Linear Programming lecturer: András London University of Szeged Institute of Informatics Department of Computational Optimization Lecture 9 Non-linear programming In case of LP, the goal
More informationDimension reduction for semidefinite programming
1 / 22 Dimension reduction for semidefinite programming Pablo A. Parrilo Laboratory for Information and Decision Systems Electrical Engineering and Computer Science Massachusetts Institute of Technology
More informationAdditional Homework Problems
Additional Homework Problems Robert M. Freund April, 2004 2004 Massachusetts Institute of Technology. 1 2 1 Exercises 1. Let IR n + denote the nonnegative orthant, namely IR + n = {x IR n x j ( ) 0,j =1,...,n}.
More informationSEMIDEFINITE PROGRAM BASICS. Contents
SEMIDEFINITE PROGRAM BASICS BRIAN AXELROD Abstract. A introduction to the basics of Semidefinite programs. Contents 1. Definitions and Preliminaries 1 1.1. Linear Algebra 1 1.2. Convex Analysis (on R n
More information14. Duality. ˆ Upper and lower bounds. ˆ General duality. ˆ Constraint qualifications. ˆ Counterexample. ˆ Complementary slackness.
CS/ECE/ISyE 524 Introduction to Optimization Spring 2016 17 14. Duality ˆ Upper and lower bounds ˆ General duality ˆ Constraint qualifications ˆ Counterexample ˆ Complementary slackness ˆ Examples ˆ Sensitivity
More informationIII. Applications in convex optimization
III. Applications in convex optimization nonsymmetric interior-point methods partial separability and decomposition partial separability first order methods interior-point methods Conic linear optimization
More informationLecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.
MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.
More informationFast ADMM for Sum of Squares Programs Using Partial Orthogonality
Fast ADMM for Sum of Squares Programs Using Partial Orthogonality Antonis Papachristodoulou Department of Engineering Science University of Oxford www.eng.ox.ac.uk/control/sysos antonis@eng.ox.ac.uk with
More informationLecture 6: Conic Optimization September 8
IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions
More informationConvex Optimization Theory. Athena Scientific, Supplementary Chapter 6 on Convex Optimization Algorithms
Convex Optimization Theory Athena Scientific, 2009 by Dimitri P. Bertsekas Massachusetts Institute of Technology Supplementary Chapter 6 on Convex Optimization Algorithms This chapter aims to supplement
More informationAdvances in Convex Optimization: Theory, Algorithms, and Applications
Advances in Convex Optimization: Theory, Algorithms, and Applications Stephen Boyd Electrical Engineering Department Stanford University (joint work with Lieven Vandenberghe, UCLA) ISIT 02 ISIT 02 Lausanne
More informationConvex Optimization M2
Convex Optimization M2 Lecture 8 A. d Aspremont. Convex Optimization M2. 1/57 Applications A. d Aspremont. Convex Optimization M2. 2/57 Outline Geometrical problems Approximation problems Combinatorial
More information4. Convex optimization problems
Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationEE 227A: Convex Optimization and Applications October 14, 2008
EE 227A: Convex Optimization and Applications October 14, 2008 Lecture 13: SDP Duality Lecturer: Laurent El Ghaoui Reading assignment: Chapter 5 of BV. 13.1 Direct approach 13.1.1 Primal problem Consider
More informationN. L. P. NONLINEAR PROGRAMMING (NLP) deals with optimization models with at least one nonlinear function. NLP. Optimization. Models of following form:
0.1 N. L. P. Katta G. Murty, IOE 611 Lecture slides Introductory Lecture NONLINEAR PROGRAMMING (NLP) deals with optimization models with at least one nonlinear function. NLP does not include everything
More informationINTERIOR-POINT METHODS FOR NONLINEAR, SECOND-ORDER CONE, AND SEMIDEFINITE PROGRAMMING. Hande Yurttan Benson
INTERIOR-POINT METHODS FOR NONLINEAR, SECOND-ORDER CONE, AND SEMIDEFINITE PROGRAMMING Hande Yurttan Benson A DISSERTATION PRESENTED TO THE FACULTY OF PRINCETON UNIVERSITY IN CANDIDACY FOR THE DEGREE OF
More informationCONVERGENCE ANALYSIS OF AN INTERIOR-POINT METHOD FOR NONCONVEX NONLINEAR PROGRAMMING
CONVERGENCE ANALYSIS OF AN INTERIOR-POINT METHOD FOR NONCONVEX NONLINEAR PROGRAMMING HANDE Y. BENSON, ARUN SEN, AND DAVID F. SHANNO Abstract. In this paper, we present global and local convergence results
More informationRelaxations and Randomized Methods for Nonconvex QCQPs
Relaxations and Randomized Methods for Nonconvex QCQPs Alexandre d Aspremont, Stephen Boyd EE392o, Stanford University Autumn, 2003 Introduction While some special classes of nonconvex problems can be
More informationThe maximal stable set problem : Copositive programming and Semidefinite Relaxations
The maximal stable set problem : Copositive programming and Semidefinite Relaxations Kartik Krishnan Department of Mathematical Sciences Rensselaer Polytechnic Institute Troy, NY 12180 USA kartis@rpi.edu
More informationSelected Examples of CONIC DUALITY AT WORK Robust Linear Optimization Synthesis of Linear Controllers Matrix Cube Theorem A.
. Selected Examples of CONIC DUALITY AT WORK Robust Linear Optimization Synthesis of Linear Controllers Matrix Cube Theorem A. Nemirovski Arkadi.Nemirovski@isye.gatech.edu Linear Optimization Problem,
More informationPreliminaries Overview OPF and Extensions. Convex Optimization. Lecture 8 - Applications in Smart Grids. Instructor: Yuanzhang Xiao
Convex Optimization Lecture 8 - Applications in Smart Grids Instructor: Yuanzhang Xiao University of Hawaii at Manoa Fall 2017 1 / 32 Today s Lecture 1 Generalized Inequalities and Semidefinite Programming
More informationLECTURE 13 LECTURE OUTLINE
LECTURE 13 LECTURE OUTLINE Problem Structures Separable problems Integer/discrete problems Branch-and-bound Large sum problems Problems with many constraints Conic Programming Second Order Cone Programming
More informationA Parametric Simplex Algorithm for Linear Vector Optimization Problems
A Parametric Simplex Algorithm for Linear Vector Optimization Problems Birgit Rudloff Firdevs Ulus Robert Vanderbei July 9, 2015 Abstract In this paper, a parametric simplex algorithm for solving linear
More informationLecture 3 Interior Point Methods and Nonlinear Optimization
Lecture 3 Interior Point Methods and Nonlinear Optimization Robert J. Vanderbei April 16, 2012 Machine Learning Summer School La Palma http://www.princeton.edu/ rvdb Example: Basis Pursuit Denoising L
More informationConvex sets, conic matrix factorizations and conic rank lower bounds
Convex sets, conic matrix factorizations and conic rank lower bounds Pablo A. Parrilo Laboratory for Information and Decision Systems Electrical Engineering and Computer Science Massachusetts Institute
More informationConvex Functions and Optimization
Chapter 5 Convex Functions and Optimization 5.1 Convex Functions Our next topic is that of convex functions. Again, we will concentrate on the context of a map f : R n R although the situation can be generalized
More informationm i=1 c ix i i=1 F ix i F 0, X O.
What is SDP? for a beginner of SDP Copyright C 2005 SDPA Project 1 Introduction This note is a short course for SemiDefinite Programming s SDP beginners. SDP has various applications in, for example, control
More informationDEPARTMENT OF STATISTICS AND OPERATIONS RESEARCH OPERATIONS RESEARCH DETERMINISTIC QUALIFYING EXAMINATION. Part I: Short Questions
DEPARTMENT OF STATISTICS AND OPERATIONS RESEARCH OPERATIONS RESEARCH DETERMINISTIC QUALIFYING EXAMINATION Part I: Short Questions August 12, 2008 9:00 am - 12 pm General Instructions This examination is
More informationConvex Optimization Boyd & Vandenberghe. 5. Duality
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationThe Graph Realization Problem
The Graph Realization Problem via Semi-Definite Programming A. Y. Alfakih alfakih@uwindsor.ca Mathematics and Statistics University of Windsor The Graph Realization Problem p.1/21 The Graph Realization
More information4. Algebra and Duality
4-1 Algebra and Duality P. Parrilo and S. Lall, CDC 2003 2003.12.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone
More informationSparse Matrix Theory and Semidefinite Optimization
Sparse Matrix Theory and Semidefinite Optimization Lieven Vandenberghe Department of Electrical Engineering University of California, Los Angeles Joint work with Martin S. Andersen and Yifan Sun Third
More informationTutorial on Convex Optimization: Part II
Tutorial on Convex Optimization: Part II Dr. Khaled Ardah Communications Research Laboratory TU Ilmenau Dec. 18, 2018 Outline Convex Optimization Review Lagrangian Duality Applications Optimal Power Allocation
More informationLecture: Cone programming. Approximating the Lorentz cone.
Strong relaxations for discrete optimization problems 10/05/16 Lecture: Cone programming. Approximating the Lorentz cone. Lecturer: Yuri Faenza Scribes: Igor Malinović 1 Introduction Cone programming is
More informationConvex optimization problems. Optimization problem in standard form
Convex optimization problems optimization problem in standard form convex optimization problems linear optimization quadratic optimization geometric programming quasiconvex optimization generalized inequality
More informationThe moment-lp and moment-sos approaches
The moment-lp and moment-sos approaches LAAS-CNRS and Institute of Mathematics, Toulouse, France CIRM, November 2013 Semidefinite Programming Why polynomial optimization? LP- and SDP- CERTIFICATES of POSITIVITY
More informationA Distributed Newton Method for Network Utility Maximization, II: Convergence
A Distributed Newton Method for Network Utility Maximization, II: Convergence Ermin Wei, Asuman Ozdaglar, and Ali Jadbabaie October 31, 2012 Abstract The existing distributed algorithms for Network Utility
More informationLecture 15 Newton Method and Self-Concordance. October 23, 2008
Newton Method and Self-Concordance October 23, 2008 Outline Lecture 15 Self-concordance Notion Self-concordant Functions Operations Preserving Self-concordance Properties of Self-concordant Functions Implications
More informationDuality Theory of Constrained Optimization
Duality Theory of Constrained Optimization Robert M. Freund April, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 The Practical Importance of Duality Duality is pervasive
More informationLecture 8. Strong Duality Results. September 22, 2008
Strong Duality Results September 22, 2008 Outline Lecture 8 Slater Condition and its Variations Convex Objective with Linear Inequality Constraints Quadratic Objective over Quadratic Constraints Representation
More informationConic Linear Programming. Yinyu Ye
Conic Linear Programming Yinyu Ye December 2004, revised October 2017 i ii Preface This monograph is developed for MS&E 314, Conic Linear Programming, which I am teaching at Stanford. Information, lecture
More informationc 2000 Society for Industrial and Applied Mathematics
SIAM J. OPIM. Vol. 10, No. 3, pp. 750 778 c 2000 Society for Industrial and Applied Mathematics CONES OF MARICES AND SUCCESSIVE CONVEX RELAXAIONS OF NONCONVEX SES MASAKAZU KOJIMA AND LEVEN UNÇEL Abstract.
More informationLagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual
More informationResearch Reports on Mathematical and Computing Sciences
ISSN 1342-2804 Research Reports on Mathematical and Computing Sciences Doubly Nonnegative Relaxations for Quadratic and Polynomial Optimization Problems with Binary and Box Constraints Sunyoung Kim, Masakazu
More informationA Unified Theorem on SDP Rank Reduction. yyye
SDP Rank Reduction Yinyu Ye, EURO XXII 1 A Unified Theorem on SDP Rank Reduction Yinyu Ye Department of Management Science and Engineering and Institute of Computational and Mathematical Engineering Stanford
More informationThe Simplest Semidefinite Programs are Trivial
The Simplest Semidefinite Programs are Trivial Robert J. Vanderbei Bing Yang Program in Statistics & Operations Research Princeton University Princeton, NJ 08544 January 10, 1994 Technical Report SOR-93-12
More informationCO 250 Final Exam Guide
Spring 2017 CO 250 Final Exam Guide TABLE OF CONTENTS richardwu.ca CO 250 Final Exam Guide Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4,
More informationLecture: Duality.
Lecture: Duality http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/35 Lagrange dual problem weak and strong
More informationFacial reduction for Euclidean distance matrix problems
Facial reduction for Euclidean distance matrix problems Nathan Krislock Department of Mathematical Sciences Northern Illinois University, USA Distance Geometry: Theory and Applications DIMACS Center, Rutgers
More informationSemidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 4
Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 4 Instructor: Farid Alizadeh Scribe: Haengju Lee 10/1/2001 1 Overview We examine the dual of the Fermat-Weber Problem. Next we will
More informationMidterm Review. Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.
Midterm Review Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye (LY, Chapter 1-4, Appendices) 1 Separating hyperplane
More informationMathematical Optimization Models and Applications
Mathematical Optimization Models and Applications Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapters 1, 2.1-2,
More informationA NEW SECOND-ORDER CONE PROGRAMMING RELAXATION FOR MAX-CUT PROBLEMS
Journal of the Operations Research Society of Japan 2003, Vol. 46, No. 2, 164-177 2003 The Operations Research Society of Japan A NEW SECOND-ORDER CONE PROGRAMMING RELAXATION FOR MAX-CUT PROBLEMS Masakazu
More informationAssignment 1: From the Definition of Convexity to Helley Theorem
Assignment 1: From the Definition of Convexity to Helley Theorem Exercise 1 Mark in the following list the sets which are convex: 1. {x R 2 : x 1 + i 2 x 2 1, i = 1,..., 10} 2. {x R 2 : x 2 1 + 2ix 1x
More informationConvex Optimization in Classification Problems
New Trends in Optimization and Computational Algorithms December 9 13, 2001 Convex Optimization in Classification Problems Laurent El Ghaoui Department of EECS, UC Berkeley elghaoui@eecs.berkeley.edu 1
More informationCS675: Convex and Combinatorial Optimization Fall 2016 Convex Optimization Problems. Instructor: Shaddin Dughmi
CS675: Convex and Combinatorial Optimization Fall 2016 Convex Optimization Problems Instructor: Shaddin Dughmi Outline 1 Convex Optimization Basics 2 Common Classes 3 Interlude: Positive Semi-Definite
More informationAgenda. 1 Cone programming. 2 Convex cones. 3 Generalized inequalities. 4 Linear programming (LP) 5 Second-order cone programming (SOCP)
Agenda 1 Cone programming 2 Convex cones 3 Generalized inequalities 4 Linear programming (LP) 5 Second-order cone programming (SOCP) 6 Semidefinite programming (SDP) 7 Examples Optimization problem in
More informationGradient Descent. Ryan Tibshirani Convex Optimization /36-725
Gradient Descent Ryan Tibshirani Convex Optimization 10-725/36-725 Last time: canonical convex programs Linear program (LP): takes the form min x subject to c T x Gx h Ax = b Quadratic program (QP): like
More informationOn deterministic reformulations of distributionally robust joint chance constrained optimization problems
On deterministic reformulations of distributionally robust joint chance constrained optimization problems Weijun Xie and Shabbir Ahmed School of Industrial & Systems Engineering Georgia Institute of Technology,
More informationUsing Schur Complement Theorem to prove convexity of some SOC-functions
Journal of Nonlinear and Convex Analysis, vol. 13, no. 3, pp. 41-431, 01 Using Schur Complement Theorem to prove convexity of some SOC-functions Jein-Shan Chen 1 Department of Mathematics National Taiwan
More informationPENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP p.1/22
PENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP Michal Kočvara Institute of Information Theory and Automation Academy of Sciences of the Czech Republic and Czech Technical University
More informationGraph Partitioning Using Random Walks
Graph Partitioning Using Random Walks A Convex Optimization Perspective Lorenzo Orecchia Computer Science Why Spectral Algorithms for Graph Problems in practice? Simple to implement Can exploit very efficient
More informationAnalysis and synthesis: a complexity perspective
Analysis and synthesis: a complexity perspective Pablo A. Parrilo ETH ZürichZ control.ee.ethz.ch/~parrilo Outline System analysis/design Formal and informal methods SOS/SDP techniques and applications
More informationAlgorithm-Hardware Co-Optimization of Memristor-Based Framework for Solving SOCP and Homogeneous QCQP Problems
L.C.Smith College of Engineering and Computer Science Algorithm-Hardware Co-Optimization of Memristor-Based Framework for Solving SOCP and Homogeneous QCQP Problems Ao Ren Sijia Liu Ruizhe Cai Wujie Wen
More informationSolution to EE 617 Mid-Term Exam, Fall November 2, 2017
Solution to EE 67 Mid-erm Exam, Fall 207 November 2, 207 EE 67 Solution to Mid-erm Exam - Page 2 of 2 November 2, 207 (4 points) Convex sets (a) (2 points) Consider the set { } a R k p(0) =, p(t) for t
More informationConvex Optimization and l 1 -minimization
Convex Optimization and l 1 -minimization Sangwoon Yun Computational Sciences Korea Institute for Advanced Study December 11, 2009 2009 NIMS Thematic Winter School Outline I. Convex Optimization II. l
More informationConvex Optimization of Graph Laplacian Eigenvalues
Convex Optimization of Graph Laplacian Eigenvalues Stephen Boyd Abstract. We consider the problem of choosing the edge weights of an undirected graph so as to maximize or minimize some function of the
More informationLecture: Convex Optimization Problems
1/36 Lecture: Convex Optimization Problems http://bicmr.pku.edu.cn/~wenzw/opt-2015-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/36 optimization
More informationEE364 Review Session 4
EE364 Review Session 4 EE364 Review Outline: convex optimization examples solving quasiconvex problems by bisection exercise 4.47 1 Convex optimization problems we have seen linear programming (LP) quadratic
More informationGeometric problems. Chapter Projection on a set. The distance of a point x 0 R n to a closed set C R n, in the norm, is defined as
Chapter 8 Geometric problems 8.1 Projection on a set The distance of a point x 0 R n to a closed set C R n, in the norm, is defined as dist(x 0,C) = inf{ x 0 x x C}. The infimum here is always achieved.
More informationBCOL RESEARCH REPORT 07.04
BCOL RESEARCH REPORT 07.04 Industrial Engineering & Operations Research University of California, Berkeley, CA 94720-1777 LIFTING FOR CONIC MIXED-INTEGER PROGRAMMING ALPER ATAMTÜRK AND VISHNU NARAYANAN
More informationLMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009
LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix
More informationI.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More informationConvex Optimization & Parsimony of L p-balls representation
Convex Optimization & Parsimony of L p -balls representation LAAS-CNRS and Institute of Mathematics, Toulouse, France IMA, January 2016 Motivation Unit balls associated with nonnegative homogeneous polynomials
More informationPOLYNOMIAL OPTIMIZATION WITH SUMS-OF-SQUARES INTERPOLANTS
POLYNOMIAL OPTIMIZATION WITH SUMS-OF-SQUARES INTERPOLANTS Sercan Yıldız syildiz@samsi.info in collaboration with Dávid Papp (NCSU) OPT Transition Workshop May 02, 2017 OUTLINE Polynomial optimization and
More informationSemidefinite and Second Order Cone Programming Seminar Fall 2012 Project: Robust Optimization and its Application of Robust Portfolio Optimization
Semidefinite and Second Order Cone Programming Seminar Fall 2012 Project: Robust Optimization and its Application of Robust Portfolio Optimization Instructor: Farid Alizadeh Author: Ai Kagawa 12/12/2012
More informationLecture: Duality of LP, SOCP and SDP
1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:
More informationBenders Decomposition
Benders Decomposition Yuping Huang, Dr. Qipeng Phil Zheng Department of Industrial and Management Systems Engineering West Virginia University IENG 593G Nonlinear Programg, Spring 2012 Yuping Huang (IMSE@WVU)
More informationMassachusetts Institute of Technology 6.854J/18.415J: Advanced Algorithms Friday, March 18, 2016 Ankur Moitra. Problem Set 6
Massachusetts Institute of Technology 6.854J/18.415J: Advanced Algorithms Friday, March 18, 2016 Ankur Moitra Problem Set 6 Due: Wednesday, April 6, 2016 7 pm Dropbox Outside Stata G5 Collaboration policy:
More informationLinear and non-linear programming
Linear and non-linear programming Benjamin Recht March 11, 2005 The Gameplan Constrained Optimization Convexity Duality Applications/Taxonomy 1 Constrained Optimization minimize f(x) subject to g j (x)
More information