15. Conic optimization
|
|
- Arthur Parks
- 6 years ago
- Views:
Transcription
1 L. Vandenberghe EE236C (Spring 216) 15. Conic optimization conic linear program examples modeling duality 15-1
2 Generalized (conic) inequalities Conic inequality: a constraint x K where K is a convex cone in R m we will require that the cone K is proper: closed pointed: K ( K) = {} with nonempty interior: int K ; equivalently, K + ( K) = R m Notation (for proper K) x K y x y K x K y x y int K Conic optimization 15-2
3 Inequality notation we will use a different convention than in EE236B Vector inequalities: for x, y R m x > y, x y denote componentwise inequality x y, x y denote conic inequality for general (unspecified) proper cone K x K y, x K y denote conic inequality for specific K Matrix inequality: for X, Y S p X Y, X Y mean X Y is positive (semi-)definite Conic optimization 15-3
4 Properties of conic inequalities preserved by nonnegative linear combinations: if x y and u v, then αx + βu αy + βv α, β define a partial order of vectors x x x y z implies x z x y and y x imply y = x in general, not a total order (requires that x y or y x for all x, y) Conic optimization 15-4
5 Conic linear program minimize subject to c T x Ax b F x = g A R m n, F R p n ; without loss of generality, we can assume rank(f ) = p, [ A rank( F ] ) = n K is a proper cone in R m for K = R m +, problem reduces to regular linear program (LP) by defining K = K 1 K r, this can represent multiple conic inequalities A 1 x K1 b 1,..., A r x Kr b r Conic optimization 15-5
6 Outline conic linear program examples modeling duality
7 Norm cones K = { (x, y) R m 1 R x y } 1 y x 2 x 1 for the Euclidean norm this is the second-order cone (notation: Q m ) Conic optimization 15-6
8 Second-order cone program minimize c T x subject to B k x + d k 2 B k1 x + d k1, k = 1,..., r Conic LP formulation: express constraints as Ax K b K = Q m 1 Q m r, A = B 1 B 11. B r, b = d 1 d 11. d r B r1 d r1 (assuming B k, d k have m k 1 rows) Conic optimization 15-7
9 Vector notation for symmetric matrices vectorized symmetric matrix: for U S p vec(u) = 2 ( U11, U 21,..., U p1, U 22, U 32,..., U p2,..., U ) pp inverse operation: for u = (u 1, u 2,..., u n ) R n with n = p(p + 1)/2 mat(u) = 1 2 2u1 u 2 u p u 2 2up+1 u 2p 1. u p u 2p 1.. 2up(p+1)/2 coefficients 2 are added so that standard inner products are preserved: tr(uv ) = vec(u) T vec(v ), u T v = tr(mat(u) mat(v)) Conic optimization 15-8
10 Positive semidefinite cone S p = {vec(x) X S p +} = {x R p(p+1)/2 mat(x) } 1 z S 2 = 1 y { (x, y, z) 1 [ x y/ 2 y/ 2 z x ] } 1 Conic optimization 15-9
11 Semidefinite program with A ij, B i S p i minimize c T x subject to x 1 A 11 + x 2 A x n A 1n B 1... x 1 A r1 + x 2 A r2 + + x n A rn B r Conic LP formulation K = S p 1 S p 2 S p r A = vec(a 11 ) vec(a 12 ) vec(a 1n ) vec(a 21 ) vec(a 22 ) vec(a 2n ).. vec(a r1 ) vec(a r2 ) vec(a rn )., b = vec(b 1 ) vec(b 2 ). vec(b r ) Conic optimization 15-1
12 Exponential cone the epigraph of the perspective of exp x is a non-proper cone K = { } (x, y, z) R 3 ye x/y z, y > the exponential cone is K exp = cl K = K {(x,, z) x, z } 1 z 2 y x Conic optimization 15-11
13 Geometric program minimize subject to c T x log n i k=1 exp(a T ik x + b ik), i = 1,..., r Conic LP formulation minimize subject to c T x at ik x + b ik 1 z ik n i k=1 z ik 1, K exp, k = 1,..., n i, i = 1,..., r i = 1,..., m Conic optimization 15-12
14 Power cone Definition: for α = (α 1, α 2,..., α m ) > and m i=1 α i = 1 K α = { (x, y) R m + R y x α 1 1 } xα m m Examples for m = 2 α = ( 1 2, 1 2 ) α = ( 2 3, 1 3 ) α = ( 3 4, 1 4 ) y x 1 x x 1 x x 1 x 2 Conic optimization 15-13
15 Cones constructed from convex sets Inverse image of convex set under perspective K = {(x, y) R n R y >, x/y C} K {(, )} is a convex cone if C is a convex set cl K is proper if C has nonempty interior, does not contain straight lines Consequence any convex constraint x C can be represented as a conic inequality x C (x, 1) K (with minor modifications to make K proper) Conic optimization 15-14
16 Cones constructed from functions Epigraph of perspective of convex function K = {(x, y, z) R n R R y >, yf(x/y) z} K {(,, )} is a convex cone if f is convex cl K is proper if int dom f, epi f does not contain straight lines Consequence can represent any convex constraint f(x) t as a conic linear inequality f(x) t (x, 1, t) K (with minor modifications to make K proper) Conic optimization 15-15
17 Outline conic linear program examples modeling duality
18 Modeling software Modeling packages for convex optimization CVX, YALMIP (MATLAB) CVXPY, PICOS (Python) MOSEK Fusion (different platforms) assist in formulating convex problems by automating two tasks: verifying convexity from convex calculus rules transforming problem in input format required by standard solvers Related packages general-purpose optimization modeling: AMPL, GAMS Conic optimization 15-16
19 Modeling and conic optimization Convex modeling systems convert problems stated in standard mathematical notation to conic LPs in principle, any convex problem can be represented as a conic LP in practice, a small set of primitive cones is used (R n +, Qp, S p ) choice of cones is limited by available algorithms and solvers (see later) modeling systems implement set of rules for expressing constraints f(x) t as conic inequalities for the implemented cones Conic optimization 15-17
20 Examples of second-order cone representable functions convex quadratic f(x) = x T P x + q T x + r (P ) quadratic-over-linear function f(x, y) = xt x y with dom f = R n R + (assume / = ) convex powers with rational exponent f(x) = x α, f(x) = { x β x > + x for rational α 1 and β p-norm f(x) = x p for rational p 1 Conic optimization 15-18
21 Examples of SD cone representable functions matrix-fractional function f(x, y) = y T X 1 y with dom f = {(X, y) S n + R n y R(X)} maximum eigenvalue of symmetric matrix maximum singular value f(x) = X 2 = σ 1 (X) [ ti X X 2 t X T ti ] nuclear norm f(x) = X = i σ i(x) X t U, V : [ U X X T V ], 1 (tr U + tr V ) t 2 Conic optimization 15-19
22 Functions representable with exponential and power cone Exponential cone exponential and logarithm entropy f(x) = x log x Power cone increasing power of absolute value: f(x) = x p with p 1 decreasing power: f(x) = x q with q and domain R ++ p-norm: f(x) = x p with p 1 Conic optimization 15-2
23 Outline conic linear program examples modeling duality
24 Dual cone K = {y x T y for all x K} Properties (if K is a proper cone) K is a proper cone (K ) = K int K = {y x T y > for all x K, x } Dual inequality: x y means x K y for generic proper cone K Note: dual cone depends on choice of inner product Conic optimization 15-21
25 Examples R p +, Qp, S p are self-dual: K = K dual of norm cone is norm cone for dual norm dual of exponential cone K exp = { (u, v, w) R R R + u log( u/w) + u v } (with log(/w) = if w ) dual of power cone is K α = { (u, v) R m + R v (u 1 /α 1 ) α1 (u m /α m ) α m } Conic optimization 15-22
26 Primal and dual conic LP Primal (optimal value p ) minimize subject to c T x Ax b Dual (optimal value d ) maximize b T z subject to A T z + c = z Weak duality: p d (without exception) Conic optimization 15-23
27 Strong duality Main theorem: p = d if primal or dual problem is strictly feasible Other implications of strict feasibility if primal is strictly feasible, then dual optimum is attained (if d is finite) if dual is strictly feasible then primal optimum is attained (if p is finite) Compare with linear programming duality (K = R m + ) for an LP, only exception to strong duality is p = +, d = strong duality holds if primal or dual is feasible if optimal value is finite then it is attained (in primal and dual) Conic optimization 15-24
28 Example with finite nonzero duality gap Primal problem minimize x 1 subject to [ ] x1 x 1 x 2 x 1 1 optimal value p = Dual problem maximize subject to optimal value d = 1 z [ ] Z11 Z 12, z Z 12 Z 22 2Z 12 + z = 1, Z 22 = Conic optimization 15-25
29 Optimality conditions if strong duality holds, then x and z are optimal if and only if [ s ] = [ A T A ] [ x z ] + [ c b ] (1) s, z, z T s = Primal feasibility: block 2 of (1) and s Dual feasibility: block 1 of (1) and z Zero duality gap: inner product of (x, z) and (, s) gives z T s = c T x + b T z Conic optimization 15-26
30 Strong theorems of alternative Strict primal feasibility exactly one of the following two systems is solvable 1. Ax b 2. A T z =, z, z, b T z Strict dual feasibility if c R(A T ), exactly one of the following two systems is solvable 1. Ax K, Ax, c T x 2. A T z + c =, z K solution of one system is a certificate of infeasibility of the other system Conic optimization 15-27
31 Weak theorems of alternative Primal feasibility at most one of the following two systems is solvable 1. Ax b 2. A T z =, z, b T z < Dual feasibility at most one of the following two systems is solvable 1. Ax, c T x < 2. A T z + c =, z these are strong alternatives if a constraint qualification holds Conic optimization 15-28
32 Self-dual embeddings Idea embed primal, dual conic LPs into a single (self-dual) conic LP, so that: embedded problem is feasible, with known feasible points from the solution of the embedded problem, primal and dual solutions of original problem can be constructed, or certificates of primal or dual infeasibility Purpose: a feasible algorithm applied to the embedded problem can detect infeasibility in original problem does not require a phase I to find initial feasible points used by some interior-point solvers Conic optimization 15-29
33 Basic self-dual embedding minimize subject to s κ = A b AT c c T b T x z τ s, κ, z, τ a self-dual conic LP has a trivial solution (all variables zero) z T s + τκ = for all feasible points (follows from equality constraint) hence, problem is not strictly feasible Conic optimization 15-3
34 Optimality condition for embedded problem s κ = A b AT c c T b T x z τ s, κ, z, τ z T s + τκ = follows from self-dual property a (mixed) linear complementarity problem Conic optimization 15-31
35 Classification of nonzero solution let s, κ, x, z, τ be a nonzero solution of the self-dual embedding Case 1: τ >, κ = ŝ = s/τ, ˆx = x/τ, ẑ = z/τ are primal and dual solutions of the conic LPs, i.e., satisfy [ ŝ ] = [ A T A ] [ ˆx ẑ ] + [ c b ] ŝ, ẑ, ŝ T ẑ = Conic optimization 15-32
36 Classification of nonzero solution Case 2: τ =, κ > ; this implies c T x + b T z < if b T z <, then ẑ = z/( b T z) is a proof of primal infeasibility: A T ẑ =, b T ẑ = 1, ẑ if c T x <, then ˆx = x/( c T x) is a proof of dual infeasibility: Aˆx, c T ˆx = 1 Case 3: τ = κ = ; no conclusion can be made about the original problem Conic optimization 15-33
37 Extended self-dual embedding minimize subject to θ s κ = A T c q x A b q z c T b T q τ qx T qz T q τ x z τ θ + 1 s, κ, z, τ q x, q z, q τ chosen so that (s, κ, x, z, τ, θ) = (s, 1, x, z, 1, z T s + 1) is feasible, for some given s, x, z a self-dual conic LP Conic optimization 15-34
38 Optimality condition s κ = A T c q x A b q z c T b T q τ qx T qz T q τ x z τ θ + 1 s, κ, z, τ z T s + τκ = follows from self-dual property a (mixed) linear complementarity problem Conic optimization 15-35
39 Properties of extended self-dual embedding problem is strictly feasible by construction if s, κ, x, z, τ, θ satisfy equality constraint, then θ = s T z + κτ (take inner product with (x, z, τ, θ) of each side of the equality) at optimum, θ = and problem reduces to the embedding on p.15-3 classification of p also applies to solutions of extended embedding Conic optimization 15-36
40 Reference A. Ben-Tal and A. Nemirovski, Lectures on Modern Convex Optimization. Analysis, Algorithms, and Engineering Applications, (21). Conic optimization 15-37
Conic Linear Optimization and its Dual. yyye
Conic Linear Optimization and Appl. MS&E314 Lecture Note #04 1 Conic Linear Optimization and its Dual Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.
More informationELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications
ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March
More informationLMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009
LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix
More informationLecture: Introduction to LP, SDP and SOCP
Lecture: Introduction to LP, SDP and SOCP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2015.html wenzw@pku.edu.cn Acknowledgement:
More information18. Primal-dual interior-point methods
L. Vandenberghe EE236C (Spring 213-14) 18. Primal-dual interior-point methods primal-dual central path equations infeasible primal-dual method primal-dual method for self-dual embedding 18-1 Symmetric
More informationConvex Functions. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Convex Functions Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Definition convex function Examples
More informationAgenda. 1 Cone programming. 2 Convex cones. 3 Generalized inequalities. 4 Linear programming (LP) 5 Second-order cone programming (SOCP)
Agenda 1 Cone programming 2 Convex cones 3 Generalized inequalities 4 Linear programming (LP) 5 Second-order cone programming (SOCP) 6 Semidefinite programming (SDP) 7 Examples Optimization problem in
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More informationL. Vandenberghe EE236C (Spring 2016) 18. Symmetric cones. definition. spectral decomposition. quadratic representation. log-det barrier 18-1
L. Vandenberghe EE236C (Spring 2016) 18. Symmetric cones definition spectral decomposition quadratic representation log-det barrier 18-1 Introduction This lecture: theoretical properties of the following
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More information4. Algebra and Duality
4-1 Algebra and Duality P. Parrilo and S. Lall, CDC 2003 2003.12.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone
More informationNew stopping criteria for detecting infeasibility in conic optimization
Optimization Letters manuscript No. (will be inserted by the editor) New stopping criteria for detecting infeasibility in conic optimization Imre Pólik Tamás Terlaky Received: March 21, 2008/ Accepted:
More informationLecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.
MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.
More informationLecture 5. The Dual Cone and Dual Problem
IE 8534 1 Lecture 5. The Dual Cone and Dual Problem IE 8534 2 For a convex cone K, its dual cone is defined as K = {y x, y 0, x K}. The inner-product can be replaced by x T y if the coordinates of the
More informationCOURSE ON LMI PART I.2 GEOMETRY OF LMI SETS. Didier HENRION henrion
COURSE ON LMI PART I.2 GEOMETRY OF LMI SETS Didier HENRION www.laas.fr/ henrion October 2006 Geometry of LMI sets Given symmetric matrices F i we want to characterize the shape in R n of the LMI set F
More informationResearch overview. Seminar September 4, Lehigh University Department of Industrial & Systems Engineering. Research overview.
Research overview Lehigh University Department of Industrial & Systems Engineering COR@L Seminar September 4, 2008 1 Duality without regularity condition Duality in non-exact arithmetic 2 interior point
More informationSEMIDEFINITE PROGRAM BASICS. Contents
SEMIDEFINITE PROGRAM BASICS BRIAN AXELROD Abstract. A introduction to the basics of Semidefinite programs. Contents 1. Definitions and Preliminaries 1 1.1. Linear Algebra 1 1.2. Convex Analysis (on R n
More informationLecture 5. Theorems of Alternatives and Self-Dual Embedding
IE 8534 1 Lecture 5. Theorems of Alternatives and Self-Dual Embedding IE 8534 2 A system of linear equations may not have a solution. It is well known that either Ax = c has a solution, or A T y = 0, c
More informationApproximate Farkas Lemmas in Convex Optimization
Approximate Farkas Lemmas in Convex Optimization Imre McMaster University Advanced Optimization Lab AdvOL Graduate Student Seminar October 25, 2004 1 Exact Farkas Lemma Motivation 2 3 Future plans The
More informationE5295/5B5749 Convex optimization with engineering applications. Lecture 5. Convex programming and semidefinite programming
E5295/5B5749 Convex optimization with engineering applications Lecture 5 Convex programming and semidefinite programming A. Forsgren, KTH 1 Lecture 5 Convex optimization 2006/2007 Convex quadratic program
More informationLecture 18: Optimization Programming
Fall, 2016 Outline Unconstrained Optimization 1 Unconstrained Optimization 2 Equality-constrained Optimization Inequality-constrained Optimization Mixture-constrained Optimization 3 Quadratic Programming
More informationSecond-order cone programming
Outline Second-order cone programming, PhD Lehigh University Department of Industrial and Systems Engineering February 10, 2009 Outline 1 Basic properties Spectral decomposition The cone of squares The
More informationLECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE
LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE CONVEX ANALYSIS AND DUALITY Basic concepts of convex analysis Basic concepts of convex optimization Geometric duality framework - MC/MC Constrained optimization
More informationSemidefinite Programming Basics and Applications
Semidefinite Programming Basics and Applications Ray Pörn, principal lecturer Åbo Akademi University Novia University of Applied Sciences Content What is semidefinite programming (SDP)? How to represent
More informationLecture: Duality of LP, SOCP and SDP
1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:
More informationConvex Optimization & Lagrange Duality
Convex Optimization & Lagrange Duality Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Convex optimization Optimality condition Lagrange duality KKT
More informationCSCI : Optimization and Control of Networks. Review on Convex Optimization
CSCI7000-016: Optimization and Control of Networks Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one
More information14. Duality. ˆ Upper and lower bounds. ˆ General duality. ˆ Constraint qualifications. ˆ Counterexample. ˆ Complementary slackness.
CS/ECE/ISyE 524 Introduction to Optimization Spring 2016 17 14. Duality ˆ Upper and lower bounds ˆ General duality ˆ Constraint qualifications ˆ Counterexample ˆ Complementary slackness ˆ Examples ˆ Sensitivity
More informationA Brief Review on Convex Optimization
A Brief Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one convex, two nonconvex sets): A Brief Review
More informationA new primal-dual path-following method for convex quadratic programming
Volume 5, N., pp. 97 0, 006 Copyright 006 SBMAC ISSN 00-805 www.scielo.br/cam A new primal-dual path-following method for convex quadratic programming MOHAMED ACHACHE Département de Mathématiques, Faculté
More informationLECTURE 13 LECTURE OUTLINE
LECTURE 13 LECTURE OUTLINE Problem Structures Separable problems Integer/discrete problems Branch-and-bound Large sum problems Problems with many constraints Conic Programming Second Order Cone Programming
More informationLecture 6: Conic Optimization September 8
IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions
More information4. Convex optimization problems
Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization
More informationConvex Optimization Boyd & Vandenberghe. 5. Duality
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 17
EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 17 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory May 29, 2012 Andre Tkacenko
More informationLecture: Duality.
Lecture: Duality http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/35 Lagrange dual problem weak and strong
More informationSemidefinite Programming
Semidefinite Programming Basics and SOS Fernando Mário de Oliveira Filho Campos do Jordão, 2 November 23 Available at: www.ime.usp.br/~fmario under talks Conic programming V is a real vector space h, i
More informationContinuous Optimisation, Chpt 9: Semidefinite Problems
Continuous Optimisation, Chpt 9: Semidefinite Problems Peter J.C. Dickinson DMMP, University of Twente p.j.c.dickinson@utwente.nl http://dickinson.website/teaching/2016co.html version: 21/11/16 Monday
More informationI.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More information13. Convex programming
CS/ISyE/ECE 524 Introduction to Optimization Spring 2015 16 13. Convex programming Convex sets and functions Convex programs Hierarchy of complexity Example: geometric programming Laurent Lessard (www.laurentlessard.com)
More informationModule 04 Optimization Problems KKT Conditions & Solvers
Module 04 Optimization Problems KKT Conditions & Solvers Ahmad F. Taha EE 5243: Introduction to Cyber-Physical Systems Email: ahmad.taha@utsa.edu Webpage: http://engineering.utsa.edu/ taha/index.html September
More informationLecture: Convex Optimization Problems
1/36 Lecture: Convex Optimization Problems http://bicmr.pku.edu.cn/~wenzw/opt-2015-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/36 optimization
More informationLagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual
More informationSelected Examples of CONIC DUALITY AT WORK Robust Linear Optimization Synthesis of Linear Controllers Matrix Cube Theorem A.
. Selected Examples of CONIC DUALITY AT WORK Robust Linear Optimization Synthesis of Linear Controllers Matrix Cube Theorem A. Nemirovski Arkadi.Nemirovski@isye.gatech.edu Linear Optimization Problem,
More informationLinear and non-linear programming
Linear and non-linear programming Benjamin Recht March 11, 2005 The Gameplan Constrained Optimization Convexity Duality Applications/Taxonomy 1 Constrained Optimization minimize f(x) subject to g j (x)
More informationConvex Optimization. Newton s method. ENSAE: Optimisation 1/44
Convex Optimization Newton s method ENSAE: Optimisation 1/44 Unconstrained minimization minimize f(x) f convex, twice continuously differentiable (hence dom f open) we assume optimal value p = inf x f(x)
More informationLagrangian Duality Theory
Lagrangian Duality Theory Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapter 14.1-4 1 Recall Primal and Dual
More informationCanonical Problem Forms. Ryan Tibshirani Convex Optimization
Canonical Problem Forms Ryan Tibshirani Convex Optimization 10-725 Last time: optimization basics Optimization terology (e.g., criterion, constraints, feasible points, solutions) Properties and first-order
More informationExample: feasibility. Interpretation as formal proof. Example: linear inequalities and Farkas lemma
4-1 Algebra and Duality P. Parrilo and S. Lall 2006.06.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone of valid
More informationLecture 14: Optimality Conditions for Conic Problems
EE 227A: Conve Optimization and Applications March 6, 2012 Lecture 14: Optimality Conditions for Conic Problems Lecturer: Laurent El Ghaoui Reading assignment: 5.5 of BV. 14.1 Optimality for Conic Problems
More informationIMPLEMENTATION OF INTERIOR POINT METHODS
IMPLEMENTATION OF INTERIOR POINT METHODS IMPLEMENTATION OF INTERIOR POINT METHODS FOR SECOND ORDER CONIC OPTIMIZATION By Bixiang Wang, Ph.D. A Thesis Submitted to the School of Graduate Studies in Partial
More informationSparse Optimization Lecture: Dual Certificate in l 1 Minimization
Sparse Optimization Lecture: Dual Certificate in l 1 Minimization Instructor: Wotao Yin July 2013 Note scriber: Zheng Sun Those who complete this lecture will know what is a dual certificate for l 1 minimization
More informationSubgradient. Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes. definition. subgradient calculus
1/41 Subgradient Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes definition subgradient calculus duality and optimality conditions directional derivative Basic inequality
More informationLecture 8. Strong Duality Results. September 22, 2008
Strong Duality Results September 22, 2008 Outline Lecture 8 Slater Condition and its Variations Convex Objective with Linear Inequality Constraints Quadratic Objective over Quadratic Constraints Representation
More information6-1 The Positivstellensatz P. Parrilo and S. Lall, ECC
6-1 The Positivstellensatz P. Parrilo and S. Lall, ECC 2003 2003.09.02.10 6. The Positivstellensatz Basic semialgebraic sets Semialgebraic sets Tarski-Seidenberg and quantifier elimination Feasibility
More informationConvex Optimization. (EE227A: UC Berkeley) Lecture 6. Suvrit Sra. (Conic optimization) 07 Feb, 2013
Convex Optimization (EE227A: UC Berkeley) Lecture 6 (Conic optimization) 07 Feb, 2013 Suvrit Sra Organizational Info Quiz coming up on 19th Feb. Project teams by 19th Feb Good if you can mix your research
More informationConstrained optimization
Constrained optimization DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_fall17/index.html Carlos Fernandez-Granda Compressed sensing Convex constrained
More informationCSC Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming
CSC2411 - Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming Notes taken by Mike Jamieson March 28, 2005 Summary: In this lecture, we introduce semidefinite programming
More informationLecture: Examples of LP, SOCP and SDP
1/34 Lecture: Examples of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:
More informationLecture Note 5: Semidefinite Programming for Stability Analysis
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State
More informationThe proximal mapping
The proximal mapping http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes Outline 2/37 1 closed function 2 Conjugate function
More informationEE 227A: Convex Optimization and Applications October 14, 2008
EE 227A: Convex Optimization and Applications October 14, 2008 Lecture 13: SDP Duality Lecturer: Laurent El Ghaoui Reading assignment: Chapter 5 of BV. 13.1 Direct approach 13.1.1 Primal problem Consider
More informationSemidefinite Programming
Semidefinite Programming Notes by Bernd Sturmfels for the lecture on June 26, 208, in the IMPRS Ringvorlesung Introduction to Nonlinear Algebra The transition from linear algebra to nonlinear algebra has
More informationSolving conic optimization problems via self-dual embedding and facial reduction: a unified approach
Solving conic optimization problems via self-dual embedding and facial reduction: a unified approach Frank Permenter Henrik A. Friberg Erling D. Andersen August 18, 216 Abstract We establish connections
More informationAgenda. Interior Point Methods. 1 Barrier functions. 2 Analytic center. 3 Central path. 4 Barrier method. 5 Primal-dual path following algorithms
Agenda Interior Point Methods 1 Barrier functions 2 Analytic center 3 Central path 4 Barrier method 5 Primal-dual path following algorithms 6 Nesterov Todd scaling 7 Complexity analysis Interior point
More informationA semidefinite relaxation scheme for quadratically constrained quadratic problems with an additional linear constraint
Iranian Journal of Operations Research Vol. 2, No. 2, 20, pp. 29-34 A semidefinite relaxation scheme for quadratically constrained quadratic problems with an additional linear constraint M. Salahi Semidefinite
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationExtreme Abridgment of Boyd and Vandenberghe s Convex Optimization
Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The
More informationInterior-Point Methods
Interior-Point Methods Stephen Wright University of Wisconsin-Madison Simons, Berkeley, August, 2017 Wright (UW-Madison) Interior-Point Methods August 2017 1 / 48 Outline Introduction: Problems and Fundamentals
More informationContinuous Optimisation, Chpt 9: Semidefinite Optimisation
Continuous Optimisation, Chpt 9: Semidefinite Optimisation Peter J.C. Dickinson DMMP, University of Twente p.j.c.dickinson@utwente.nl http://dickinson.website/teaching/2017co.html version: 28/11/17 Monday
More informationWhat can be expressed via Conic Quadratic and Semidefinite Programming?
What can be expressed via Conic Quadratic and Semidefinite Programming? A. Nemirovski Faculty of Industrial Engineering and Management Technion Israel Institute of Technology Abstract Tremendous recent
More informationLecture: Cone programming. Approximating the Lorentz cone.
Strong relaxations for discrete optimization problems 10/05/16 Lecture: Cone programming. Approximating the Lorentz cone. Lecturer: Yuri Faenza Scribes: Igor Malinović 1 Introduction Cone programming is
More informationGlobal Optimization of Polynomials
Semidefinite Programming Lecture 9 OR 637 Spring 2008 April 9, 2008 Scribe: Dennis Leventhal Global Optimization of Polynomials Recall we were considering the problem min z R n p(z) where p(z) is a degree
More information4TE3/6TE3. Algorithms for. Continuous Optimization
4TE3/6TE3 Algorithms for Continuous Optimization (Duality in Nonlinear Optimization ) Tamás TERLAKY Computing and Software McMaster University Hamilton, January 2004 terlaky@mcmaster.ca Tel: 27780 Optimality
More information8. Geometric problems
8. Geometric problems Convex Optimization Boyd & Vandenberghe extremal volume ellipsoids centering classification placement and facility location 8 Minimum volume ellipsoid around a set Löwner-John ellipsoid
More informationAssignment 1: From the Definition of Convexity to Helley Theorem
Assignment 1: From the Definition of Convexity to Helley Theorem Exercise 1 Mark in the following list the sets which are convex: 1. {x R 2 : x 1 + i 2 x 2 1, i = 1,..., 10} 2. {x R 2 : x 2 1 + 2ix 1x
More informationPrimal-Dual Interior-Point Methods. Javier Peña Convex Optimization /36-725
Primal-Dual Interior-Point Methods Javier Peña Convex Optimization 10-725/36-725 Last time: duality revisited Consider the problem min x subject to f(x) Ax = b h(x) 0 Lagrangian L(x, u, v) = f(x) + u T
More informationConvex Optimization : Conic Versus Functional Form
Convex Optimization : Conic Versus Functional Form Erling D. Andersen MOSEK ApS, Fruebjergvej 3, Box 16, DK 2100 Copenhagen, Blog: http://erlingdandersen.blogspot.com Linkedin: http://dk.linkedin.com/in/edandersen
More informationFrom Convex Optimization to Linear Matrix Inequalities
Dep. of Information Engineering University of Pisa (Italy) From Convex Optimization to Linear Matrix Inequalities eng. Sergio Grammatico grammatico.sergio@gmail.com Class of Identification of Uncertain
More informationCopositive Plus Matrices
Copositive Plus Matrices Willemieke van Vliet Master Thesis in Applied Mathematics October 2011 Copositive Plus Matrices Summary In this report we discuss the set of copositive plus matrices and their
More information3. Convex functions. basic properties and examples. operations that preserve convexity. the conjugate function. quasiconvex functions
3. Convex functions Convex Optimization Boyd & Vandenberghe basic properties and examples operations that preserve convexity the conjugate function quasiconvex functions log-concave and log-convex functions
More informationLecture 17: Primal-dual interior-point methods part II
10-725/36-725: Convex Optimization Spring 2015 Lecture 17: Primal-dual interior-point methods part II Lecturer: Javier Pena Scribes: Pinchao Zhang, Wei Ma Note: LaTeX template courtesy of UC Berkeley EECS
More informationDuality. Geoff Gordon & Ryan Tibshirani Optimization /
Duality Geoff Gordon & Ryan Tibshirani Optimization 10-725 / 36-725 1 Duality in linear programs Suppose we want to find lower bound on the optimal value in our convex problem, B min x C f(x) E.g., consider
More informationShiqian Ma, MAT-258A: Numerical Optimization 1. Chapter 4. Subgradient
Shiqian Ma, MAT-258A: Numerical Optimization 1 Chapter 4 Subgradient Shiqian Ma, MAT-258A: Numerical Optimization 2 4.1. Subgradients definition subgradient calculus duality and optimality conditions Shiqian
More informationNonlinear Programming Models
Nonlinear Programming Models Fabio Schoen 2008 http://gol.dsi.unifi.it/users/schoen Nonlinear Programming Models p. Introduction Nonlinear Programming Models p. NLP problems minf(x) x S R n Standard form:
More informationLecture 7: Convex Optimizations
Lecture 7: Convex Optimizations Radu Balan, David Levermore March 29, 2018 Convex Sets. Convex Functions A set S R n is called a convex set if for any points x, y S the line segment [x, y] := {tx + (1
More informationLargest dual ellipsoids inscribed in dual cones
Largest dual ellipsoids inscribed in dual cones M. J. Todd June 23, 2005 Abstract Suppose x and s lie in the interiors of a cone K and its dual K respectively. We seek dual ellipsoidal norms such that
More informationAdvances in Convex Optimization: Theory, Algorithms, and Applications
Advances in Convex Optimization: Theory, Algorithms, and Applications Stephen Boyd Electrical Engineering Department Stanford University (joint work with Lieven Vandenberghe, UCLA) ISIT 02 ISIT 02 Lausanne
More informationMidterm Review. Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.
Midterm Review Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye (LY, Chapter 1-4, Appendices) 1 Separating hyperplane
More informationConvex Optimization. (EE227A: UC Berkeley) Lecture 28. Suvrit Sra. (Algebra + Optimization) 02 May, 2013
Convex Optimization (EE227A: UC Berkeley) Lecture 28 (Algebra + Optimization) 02 May, 2013 Suvrit Sra Admin Poster presentation on 10th May mandatory HW, Midterm, Quiz to be reweighted Project final report
More informationOptimization for Machine Learning
Optimization for Machine Learning (Problems; Algorithms - A) SUVRIT SRA Massachusetts Institute of Technology PKU Summer School on Data Science (July 2017) Course materials http://suvrit.de/teaching.html
More informationCourse Outline. FRTN10 Multivariable Control, Lecture 13. General idea for Lectures Lecture 13 Outline. Example 1 (Doyle Stein, 1979)
Course Outline FRTN Multivariable Control, Lecture Automatic Control LTH, 6 L-L Specifications, models and loop-shaping by hand L6-L8 Limitations on achievable performance L9-L Controller optimization:
More information3. Convex functions. basic properties and examples. operations that preserve convexity. the conjugate function. quasiconvex functions
3. Convex functions Convex Optimization Boyd & Vandenberghe basic properties and examples operations that preserve convexity the conjugate function quasiconvex functions log-concave and log-convex functions
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More informationminimize x x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x 2 u 2, 5x 1 +76x 2 1,
4 Duality 4.1 Numerical perturbation analysis example. Consider the quadratic program with variables x 1, x 2, and parameters u 1, u 2. minimize x 2 1 +2x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x
More informationLecture 3. Optimization Problems and Iterative Algorithms
Lecture 3 Optimization Problems and Iterative Algorithms January 13, 2016 This material was jointly developed with Angelia Nedić at UIUC for IE 598ns Outline Special Functions: Linear, Quadratic, Convex
More informationHomework 4. Convex Optimization /36-725
Homework 4 Convex Optimization 10-725/36-725 Due Friday November 4 at 5:30pm submitted to Christoph Dann in Gates 8013 (Remember to a submit separate writeup for each problem, with your name at the top)
More informationConvex Optimization in Classification Problems
New Trends in Optimization and Computational Algorithms December 9 13, 2001 Convex Optimization in Classification Problems Laurent El Ghaoui Department of EECS, UC Berkeley elghaoui@eecs.berkeley.edu 1
More informationConvex Optimization Theory. Chapter 5 Exercises and Solutions: Extended Version
Convex Optimization Theory Chapter 5 Exercises and Solutions: Extended Version Dimitri P. Bertsekas Massachusetts Institute of Technology Athena Scientific, Belmont, Massachusetts http://www.athenasc.com
More information