Solving linear and non-linear SDP by PENNON
|
|
- Katrina Flowers
- 6 years ago
- Views:
Transcription
1 Solving linear and non-linear SDP by PENNON Michal Kočvara School of Mathematics, The University of Birmingham University of Warwick, February 2010
2 Outline Why nonlinear SDP? PENNON the new generation Example: Nearest correlation matrix Example: Approximation by nonnegative splines Example: Sensor network localization
3 Semidefinite programming (SDP) generalized mathematical program min f(x) subject to g(x) 0 A(x) 0 A(x) (non)linear matrix operator R n S m (A(x) = A 0 + x i A i )
4 SDP notations S n... symmetric matrices of order n n A 0... A positive semidefinite A 0... A B 0 A, B := Tr(AB)... inner product on S n A[R n S m ]... linear matrix operator defined by A(y) := n y i A i i=1 with A i S m A [S m R n ]... adjoint operator defined by A (X) := [ A 1, X,..., A n, X ] T and satisfying A (X), y = A(y), X for all y R n
5 Primal-dual SDP pair inf C, X := Tr(CX) (P) X s.t. A (X) = b [ A i, X = b i, i = 1,...,n] X 0 sup b, y := b i y i y,s s.t. A(y) + S = C [ y i A i + S = C] S 0 (D) Weak duality: Feasible X, y, S satisfy C, X b, y = A(y) + S, X y i A i, X = S, X 0 duality gap nonnegative for feasible points
6 Linear Semidefinite Programming Vast area of applications... LP and CQP is SDP eigenvalue optimisation robust programming control theory relaxations of integer optimisation problems approximations to combinatorial optimisation problems structural optimisation chemical engineering machine learning many many others...
7 Why nonlinear SDP? Problems from Structural optimization Control theory Mathematical Programming with Equilibrium Constraints Examples below There are more but the researchers just don t know about...
8 Nonlinear SDP? The general nonlinear SDP (NSDP) problem min f(x) x R n subject to g i (x) 0, h i (x) = 0, A(x) 0 i = 1,..., n g i = 1,..., n h b R n and A : R n S m nonlinear, nonconvex
9
10
11 A380 Inboard Inner Leading Edge Ribs Impact of topological decisions all designs sizing-optimized Baseline Stiffeners at 45-13% -33% Optimized topology
12 Aim: Free Material Optimization Given an amount of material, boundary conditions and external load f, find the material (distribution) so that the body is as stiff as possible under f. The design variables are the material properties at each point of the structure. M. P. Bendsøe, J.M. Guades, R.B. Haber, P. Pedersen and J. E. Taylor: An analytical model to predict optimal material properties in the context of optimal structural design. J. Applied Mechanics, 61 (1994)
13 Free Material Optimization
14 FMO SL primal formulation FMO-problem (minimum volume formulation) min u R n,e 1,...,E m subject to m TrE i i=1 E i 0,ρ TrE i ρ, f u C A(E)u = f i = 1,..., m
15 FMO SL primal formulation FMO-problem with vibration/buckling constraint min u R n,e 1,...,E m subject to m TrE i i=1 E i 0, ρ TrE i ρ, f u C A(E)u = f A(E) + G(E, u) 0 i = 1,..., m
16 FMO SL primal formulation FMO-problem with vibration/buckling constraint min u R n,e 1,...,E m subject to m TrE i i=1 E i 0, ρ TrE i ρ, f u C A(E)u = f A(E) + G(E, u) 0 i = 1,..., m... nonlinear, non-convex semidefinite problem
17 PENNON collection PENNON (PENalty methods for NONlinear optimization) a collection of codes for NLP, (linear) SDP and BMI one algorithm to rule them all READY PENNLP PENSDP PENBMI AMPL, MATLAB, C/Fortran MATLAB/YALMIP, SDPA, C/Fortran MATLAB/YALMIP, C/Fortran NEW PENNON (NLP + SDP) extended AMPL, MATLAB
18 The problem Optimization problems with nonlinear objective subject to nonlinear inequality and equality constraints and semidefinite bound constraints: min f(x, Y) x,y 1 S p 1,...,Y k S p k subject to g i (x, Y) 0, i = 1,...,m g h i (x, Y) = 0, i = 1,...,m h λ i I Y i λ i I, i = 1,...,k. (NLP-SDP)
19 The problem Here x R n is the vector variable Y 1 S p 1,..., Y k S p k are the matrix variables, k symmetric matrices of dimensions p 1 p 1,..., p k p k we denote Y = (Y 1,..., Y k ) f, g i and h i are C 2 functions from R n S p 1... S p k to R λ i and λ i are the lower and upper bounds, respectively, on the eigenvalues of Y i, i = 1,...,k
20 The problem Any nonlinear SDP problem can be furmulated as NLP-SDP, using slack variables and (NLP) equality constraints: write as g(x) 0 g(x) = S S 0 element-wise
21 The algorithm Based on penalty/barrier functions ϕ g : R R and Φ P : S p S p : g i (x) 0 p i ϕ g (g i (x)/p i ) 0, Z 0 Φ P (Z) 0, Z S p. Augmented Lagrangian of (NLP-SDP): i = 1,...,m F(x,Y,u,U,U,p)=f(x,Y)+ m g i=1 u ip i ϕ g(g i (x,y)/p i ) + k i=1 U i,φ P(λ i I Y i ) + k i=1 U i,φ P (Y i λ i I) ; here u R mg and U i, U i are Lagrange multipliers.
22 The algorithm A generalized Augmented Lagrangian algorithm (based on R. Polyak 92, Ben-Tal Zibulevsky 94, Stingl 05): Given x 1, Y 1, u 1, U 1, U 1 ; p 1 i > 0, i = 1,...,m g and P > 0. For k = 1, 2,... repeat till a stopping criterium is reached: (i) Find x k+1 and Y k+1 s.t. x F(x k+1, Y k+1, u k, U k, U k, p k ) K (ii) u k+1 i = u k i ϕ g (g i(x k+1 )/p k i ), i = 1,..., m g (iii) p k+1 i U k+1 = D i A Φ P ((λ i I Y i ); U k i ), i = 1,..., k U k+1 i = D A Φ P ((Y i λ i I); U k i ), i = 1,..., k < p k i, i = 1,..., m g P k+1 < P k.
23 Interfaces How to enter the data the functions and their derivatives? Matlab interface AMPL interface
24 Matlab interface User provides six MATLAB functions: f... evaluates the objective function df... evaluates the gradient of objective function hf... evaluates the Hessian of objective function g... evaluates the constraints dg... evaluates the gradient of constraints hg... evaluates the Hessian of constraints
25 Matlab interface Matrix variables are treated as vectors, using the function svec : S m R (m+1)m/2 defined by a 11 a a 1m a a 2m svec.... sym a mm = (a 11, a 12, a 22,..., a 1m, a 2m, a mm ) T
26 Matlab interface Matrix variables are treated as vectors, using the function svec : S m R (m+1)m/2 defined by a 11 a a 1m a a 2m svec.... sym a mm = (a 11, a 12, a 22,..., a 1m, a 2m, a mm ) T Keep a specific order of variables, to recognize which are matrices and which vectors. Add lower/upper bounds on matrix eigenvalues. Sparse matrices available, sparsity maintained in the user defined functions.
27 AMPL interface AMPL does not support SDP variables and constraints. Use the same trick: Matrix variables are treated as vectors, using the function svec : S m R (m+1)m/2 defined by a 11 a a 1m a a 2m svec.... sym a mm = (a 11, a 12, a 22,..., a 1m, a 2m, a mm ) T Need additional input file specifying the matrix sizes and lower/upper eigenvalue bounds.
28 Example: nearest correlation matrix Find a nearest correlation matrix: min X n (X ij H ij ) 2 (1) i,j=1 subject to X ii = 1, X 0 i = 1,...,n
29 Example: nearest correlation matrix AMPL code: param h{1..21}; set ind within {1..21}; # Nonlinear SDP variables var x{1..21} default 0; minimize Obj: sum{i in 1..21} (x[i]-h[i])ˆ2; subject to l1{i in ind}: x[i] = 1; data; param h:=
30 Example: nearest correlation matrix For H ext = the eigenvalues of the correlation matrix are eigen =
31 Example: nearest correlation matrix The condition number of the nearest correlation matrix must be bounded. Add new variables y, z R and new cosntraints X zi (2) X yi (3) y κz (4) where κ is the required condition number. The constraints do not fit into our required NLP-SDP problem structure. Introduce two new (slack) matrix variables, say, P and Q, and replace (2) and (3) by X zi P = 0 X yi Q = 0 P 0 Q 0
32 Example: nearest correlation matrix More ellegant way: rewrite constraints (2) (3) as I X κi (5) assuming that y = κz and using the transormation of the variable X: z X = X. The new problem: min z, X n (z X ij H ij ) 2 (6) i,j=1 subject to z X ii = 1, I X κi i = 1,...,n
33 Example: nearest correlation matrix For X = the eigenvalues of the correlation matrix are eigen =
34 Example: nearest correlation matrix Cooperation with Allianz SE, Munich: Matrices of size up to Code PENCOR: C code, data in xml format feasibility analysis sensitivity analysis w.r.t. bounds on matrix elements
35 Example: Approximation by nonnegative splines Let f : [0, 1] R. Given its (noisy) function values b i, i = 1,..., n at points t i (0, 1). Find a smooth approximation of f by a cubic spline: 3 P(t) = P (i) (t) = P(i) k (t a i 1 ) k k=1 for a point t [a i 1, a i ], where 0 = a 0 < a 1 <... < a m = 1 are the knots and P (i) k (i = 1,..., m, k = 0, 1, 2, 3) the coefficients of the spline. Spline property: for i = 1,..., m 1 P (i+1) 0 P (i) 0 P(i) 1 (a i a i 1 ) P (i) 2 (a i a i 1 ) 2 P (i) 3 (a i a i 1 ) 3 = 0 (7) P (i+1) 1 P (i) 1 2P(i) 2 (a i a i 1 ) 3P (i) 3 (a i a i 1 ) 2 = 0 (8) 2P (i+1) 2 2P (i) 2 6P(i) 3 (a i a i 1 ) = 0. (9)
36 Example: Approximation by nonnegative splines The function f will be approximated by P in the least square sense: minimize n (P(t j ) b j ) 2 subject to (7),(8),(9). j=1 Now, f is assumed to be nonnegative, so P 0 is required.
37 Example: Approximation by nonnegative splines de Boor and Daniel 74: while approximation of a nonnegative function by nonnegative splines of order k gives errors of order h k, approximation by a subclass of nonnegative splines of order k consisting of all those whose B-spline coefficients are nonnegative may yield only errors of order h 2. Nesterov 2000: P (i) (t) nonnegative there exist two symmetric matrices ( ) ( ) X (i) xi y = i, S y i z (i) si v = i i v i w i such that P (i) 0 = (a i a i 1 )s i (10) P (i) 1 = x i s i + 2(a i a i 1 )v i (11) P (i) 2 = 2y i 2v i + (a i a i 1 )w i (12) P (i) 3 = z i w i (13) X (i) 0, S (i) 0. (14)
38 Example: Approximation by nonnegative splines We want to solve an NLP-SDP problem min P (i) k R j=1 i=1,...,m, k=0,1,2,3 subject to (7),(8),(9), i = 1,..., m (10) (14), i = 1,...,m n (P(t j ) b j ) 2 (15)
39 Example: Approximation by nonnegative splines Example, n = 500, m = 7, noisy data: Figure: Approximation by nonnegative splines: noisy data given in green, optimal nonnegative spline in red and an optimal spline ignoring the nonnegativity constraint in blue. The right-hand side figure zooms on the left valey.
40 Sensor network localization (Euclidean distance matrix completion, Graph realization) We have (in R 2 (or R d )) n points a i, anchors with known location m points x i, sensors with unknown location d ij known Euclidean distance between close points d ij = x i x j, (i, j) I x d kj = a k x j, (k, j) I a Goal: Find the positions of the sensors! Find x R 2 m such that x i x j 2 = d 2 ij, a k x j 2 = d 2 kj, (i, j) I x (k, j) I a
41 Sensor network localization Example, 4 anchors, 36 sensors
42 Sensor network localization Applications Wireless sensor network localization habitat monitoring system in the Great Duck Island detecting volcano eruptions industrial control in semiconductor manufacturing plants structural health monitoring military and civilian surveillance moving object tracking asset location Molecule conformation...
43 Sensor network localization Solve the least-square problem min x i x j 2 d 2 + x 1,...,x m (i,j) I x ij (i,j) I a to global minimum. This is an NP-hard problem. a k x j 2 d 2 kj
44 (P. Biswas and Y. Ye, 04) SDP relaxation Let X = [x 1 x 2... x n ] be a d n unknown matrix. Then x i x j 2 = (e i e j ) T X T X(e i e j ) [ ] a k x j 2 = (a k ; e j ) T Id X T [I d X](a k ; e j ) and the problem becomes (e i e j ) T X T X(e i e j ) = dij 2 ( ) (a k ; e j ) T Id X X T (a Y k ; e j ) = d 2 kj Y = X T X
45 SDP relaxation Now( relax Y = ) X T X to Y X T X, equivalent to Id X Z = X T 0 Y Relaxed problem: min 0 subject to Z 1:d,1:d = I d (0; e i e j ) T Z(0; e i e j ) = d 2 ij (i, j) I x (a k ; e j ) T Z(a k ; e j ) = d 2 kj (k, j) I a Z 0 Full SDP relaxation, FSDP (linear SDP)
46 SDP relaxation Equivalent formulation: min ((0; e i e j ) T Z(0; e i e j ) dij 2 )2 (i,j) I x + ((a k ; e j ) T Z(a k ; e j ) d 2 kj )2 (k,j) I a subject to Z 1:d,1:d = I d Z 0 Full SDP relaxation, FSDP (nonlinear SDP)
47 SDP relaxation Take the SDP solution as initial approximation for the original unconstrained nonconvex problem. Solve both by PENNON.
48 Sensor network localization Example, 9 anchors, 720 sensors
49 Sensor network localization Example, 9 anchors, 720 sensors Figure: SDP: variables, (4 4) LMIs
50 Other Applications, Availability polynomial matrix inequalities (with Didier Henrion) financial mathematics (with Ralf Werner) structural optimization with matrix variables and nonlinear matrix constraints (PLATO-N EU FP6 project) approximation by nonnegative splines approximation of arrival rate function of a non-homogeneous Poisson process (F. Alizadeh, J. Eckstein) sensor network localization (with Houduo Xi) detection of definite pairs of matrices (with F. Tisseur) Many other applications any hint welcome Free academic version of the code available Free downloadable MATLAB version available soon
Introducing PENLAB a MATLAB code for NLP-SDP
Introducing PENLAB a MATLAB code for NLP-SDP Michal Kočvara School of Mathematics, The University of Birmingham jointly with Jan Fiala Numerical Algorithms Group Michael Stingl University of Erlangen-Nürnberg
More informationPENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP p.1/22
PENNON A Generalized Augmented Lagrangian Method for Nonconvex NLP and SDP Michal Kočvara Institute of Information Theory and Automation Academy of Sciences of the Czech Republic and Czech Technical University
More informationI.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More informationPENNON A Generalized Augmented Lagrangian Method for Convex NLP and SDP p.1/39
PENNON A Generalized Augmented Lagrangian Method for Convex NLP and SDP Michal Kočvara Institute of Information Theory and Automation Academy of Sciences of the Czech Republic and Czech Technical University
More informationApplications of Linear Programming
Applications of Linear Programming lecturer: András London University of Szeged Institute of Informatics Department of Computational Optimization Lecture 9 Non-linear programming In case of LP, the goal
More informationPart IB Optimisation
Part IB Optimisation Theorems Based on lectures by F. A. Fischer Notes taken by Dexter Chua Easter 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after
More informationSemidefinite Programming
Semidefinite Programming Basics and SOS Fernando Mário de Oliveira Filho Campos do Jordão, 2 November 23 Available at: www.ime.usp.br/~fmario under talks Conic programming V is a real vector space h, i
More informationConvex Optimization M2
Convex Optimization M2 Lecture 8 A. d Aspremont. Convex Optimization M2. 1/57 Applications A. d Aspremont. Convex Optimization M2. 2/57 Outline Geometrical problems Approximation problems Combinatorial
More informationTutorial on Convex Optimization: Part II
Tutorial on Convex Optimization: Part II Dr. Khaled Ardah Communications Research Laboratory TU Ilmenau Dec. 18, 2018 Outline Convex Optimization Review Lagrangian Duality Applications Optimal Power Allocation
More informationPENNON User s Guide (Version 0.9)
PENNON User s Guide (Version.9) Michal Kočvara Michael Stingl June 11, 28 Contents 1 Installation 2 1.1 Unpacking................................... 2 1.2 Compilation..................................
More informationLinear and non-linear programming
Linear and non-linear programming Benjamin Recht March 11, 2005 The Gameplan Constrained Optimization Convexity Duality Applications/Taxonomy 1 Constrained Optimization minimize f(x) subject to g j (x)
More informationThe moment-lp and moment-sos approaches
The moment-lp and moment-sos approaches LAAS-CNRS and Institute of Mathematics, Toulouse, France CIRM, November 2013 Semidefinite Programming Why polynomial optimization? LP- and SDP- CERTIFICATES of POSITIVITY
More informationLecture Note 5: Semidefinite Programming for Stability Analysis
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State
More informationCSCI 1951-G Optimization Methods in Finance Part 10: Conic Optimization
CSCI 1951-G Optimization Methods in Finance Part 10: Conic Optimization April 6, 2018 1 / 34 This material is covered in the textbook, Chapters 9 and 10. Some of the materials are taken from it. Some of
More informationModule 04 Optimization Problems KKT Conditions & Solvers
Module 04 Optimization Problems KKT Conditions & Solvers Ahmad F. Taha EE 5243: Introduction to Cyber-Physical Systems Email: ahmad.taha@utsa.edu Webpage: http://engineering.utsa.edu/ taha/index.html September
More informationCSC Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming
CSC2411 - Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming Notes taken by Mike Jamieson March 28, 2005 Summary: In this lecture, we introduce semidefinite programming
More informationPENNON: Software for linear and nonlinear matrix inequalities
PENNON: Software for linear and nonlinear matrix inequalities Michal Kočvara 1 and Michael Stingl 2 1 School of Mathematics, University of Birmingham, Birmingham B15 2TT, UK and Institute of Information
More informationDuality in Linear Programs. Lecturer: Ryan Tibshirani Convex Optimization /36-725
Duality in Linear Programs Lecturer: Ryan Tibshirani Convex Optimization 10-725/36-725 1 Last time: proximal gradient descent Consider the problem x g(x) + h(x) with g, h convex, g differentiable, and
More informationLecture 6: Conic Optimization September 8
IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions
More informationSemidefinite and Second Order Cone Programming Seminar Fall 2012 Project: Robust Optimization and its Application of Robust Portfolio Optimization
Semidefinite and Second Order Cone Programming Seminar Fall 2012 Project: Robust Optimization and its Application of Robust Portfolio Optimization Instructor: Farid Alizadeh Author: Ai Kagawa 12/12/2012
More informationm i=1 c ix i i=1 F ix i F 0, X O.
What is SDP? for a beginner of SDP Copyright C 2005 SDPA Project 1 Introduction This note is a short course for SemiDefinite Programming s SDP beginners. SDP has various applications in, for example, control
More informationICS-E4030 Kernel Methods in Machine Learning
ICS-E4030 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 28. September, 2016 Juho Rousu 28. September, 2016 1 / 38 Convex optimization Convex optimisation This
More informationSupport Vector Machine (SVM) and Kernel Methods
Support Vector Machine (SVM) and Kernel Methods CE-717: Machine Learning Sharif University of Technology Fall 2014 Soleymani Outline Margin concept Hard-Margin SVM Soft-Margin SVM Dual Problems of Hard-Margin
More informationDuality. Lagrange dual problem weak and strong duality optimality conditions perturbation and sensitivity analysis generalized inequalities
Duality Lagrange dual problem weak and strong duality optimality conditions perturbation and sensitivity analysis generalized inequalities Lagrangian Consider the optimization problem in standard form
More informationConvex Optimization and SVM
Convex Optimization and SVM Problem 0. Cf lecture notes pages 12 to 18. Problem 1. (i) A slab is an intersection of two half spaces, hence convex. (ii) A wedge is an intersection of two half spaces, hence
More informationCOURSE ON LMI PART I.2 GEOMETRY OF LMI SETS. Didier HENRION henrion
COURSE ON LMI PART I.2 GEOMETRY OF LMI SETS Didier HENRION www.laas.fr/ henrion October 2006 Geometry of LMI sets Given symmetric matrices F i we want to characterize the shape in R n of the LMI set F
More informationLagrangian Duality Theory
Lagrangian Duality Theory Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapter 14.1-4 1 Recall Primal and Dual
More informationLecture: Introduction to LP, SDP and SOCP
Lecture: Introduction to LP, SDP and SOCP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2015.html wenzw@pku.edu.cn Acknowledgement:
More informationLMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009
LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix
More informationPENNON A Code for Convex Nonlinear and Semidefinite Programming
PENNON A Code for Convex Nonlinear and Semidefinite Programming Michal Kočvara Michael Stingl Abstract We introduce a computer program PENNON for the solution of problems of convex Nonlinear and Semidefinite
More informationWritten Examination
Division of Scientific Computing Department of Information Technology Uppsala University Optimization Written Examination 202-2-20 Time: 4:00-9:00 Allowed Tools: Pocket Calculator, one A4 paper with notes
More informationminimize x subject to (x 2)(x 4) u,
Math 6366/6367: Optimization and Variational Methods Sample Preliminary Exam Questions 1. Suppose that f : [, L] R is a C 2 -function with f () on (, L) and that you have explicit formulae for
More informationSupport Vector Machine (SVM) and Kernel Methods
Support Vector Machine (SVM) and Kernel Methods CE-717: Machine Learning Sharif University of Technology Fall 2015 Soleymani Outline Margin concept Hard-Margin SVM Soft-Margin SVM Dual Problems of Hard-Margin
More informationOptimization for Communications and Networks. Poompat Saengudomlert. Session 4 Duality and Lagrange Multipliers
Optimization for Communications and Networks Poompat Saengudomlert Session 4 Duality and Lagrange Multipliers P Saengudomlert (2015) Optimization Session 4 1 / 14 24 Dual Problems Consider a primal convex
More informationLagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual
More informationminimize x x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x 2 u 2, 5x 1 +76x 2 1,
4 Duality 4.1 Numerical perturbation analysis example. Consider the quadratic program with variables x 1, x 2, and parameters u 1, u 2. minimize x 2 1 +2x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x
More informationLagrange duality. The Lagrangian. We consider an optimization program of the form
Lagrange duality Another way to arrive at the KKT conditions, and one which gives us some insight on solving constrained optimization problems, is through the Lagrange dual. The dual is a maximization
More informationELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications
ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March
More informationMotivation. Lecture 2 Topics from Optimization and Duality. network utility maximization (NUM) problem:
CDS270 Maryam Fazel Lecture 2 Topics from Optimization and Duality Motivation network utility maximization (NUM) problem: consider a network with S sources (users), each sending one flow at rate x s, through
More informationE5295/5B5749 Convex optimization with engineering applications. Lecture 5. Convex programming and semidefinite programming
E5295/5B5749 Convex optimization with engineering applications Lecture 5 Convex programming and semidefinite programming A. Forsgren, KTH 1 Lecture 5 Convex optimization 2006/2007 Convex quadratic program
More informationProjection methods to solve SDP
Projection methods to solve SDP Franz Rendl http://www.math.uni-klu.ac.at Alpen-Adria-Universität Klagenfurt Austria F. Rendl, Oberwolfach Seminar, May 2010 p.1/32 Overview Augmented Primal-Dual Method
More informationResearch overview. Seminar September 4, Lehigh University Department of Industrial & Systems Engineering. Research overview.
Research overview Lehigh University Department of Industrial & Systems Engineering COR@L Seminar September 4, 2008 1 Duality without regularity condition Duality in non-exact arithmetic 2 interior point
More informationOptimality, Duality, Complementarity for Constrained Optimization
Optimality, Duality, Complementarity for Constrained Optimization Stephen Wright University of Wisconsin-Madison May 2014 Wright (UW-Madison) Optimality, Duality, Complementarity May 2014 1 / 41 Linear
More informationSupport Vector Machines: Maximum Margin Classifiers
Support Vector Machines: Maximum Margin Classifiers Machine Learning and Pattern Recognition: September 16, 2008 Piotr Mirowski Based on slides by Sumit Chopra and Fu-Jie Huang 1 Outline What is behind
More informationSemidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5
Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5 Instructor: Farid Alizadeh Scribe: Anton Riabov 10/08/2001 1 Overview We continue studying the maximum eigenvalue SDP, and generalize
More information- Well-characterized problems, min-max relations, approximate certificates. - LP problems in the standard form, primal and dual linear programs
LP-Duality ( Approximation Algorithms by V. Vazirani, Chapter 12) - Well-characterized problems, min-max relations, approximate certificates - LP problems in the standard form, primal and dual linear programs
More informationNetwork Localization via Schatten Quasi-Norm Minimization
Network Localization via Schatten Quasi-Norm Minimization Anthony Man-Cho So Department of Systems Engineering & Engineering Management The Chinese University of Hong Kong (Joint Work with Senshan Ji Kam-Fung
More informationISM206 Lecture Optimization of Nonlinear Objective with Linear Constraints
ISM206 Lecture Optimization of Nonlinear Objective with Linear Constraints Instructor: Prof. Kevin Ross Scribe: Nitish John October 18, 2011 1 The Basic Goal The main idea is to transform a given constrained
More informationNumerical Optimization. Review: Unconstrained Optimization
Numerical Optimization Finding the best feasible solution Edward P. Gatzke Department of Chemical Engineering University of South Carolina Ed Gatzke (USC CHE ) Numerical Optimization ECHE 589, Spring 2011
More informationConvexification of Mixed-Integer Quadratically Constrained Quadratic Programs
Convexification of Mixed-Integer Quadratically Constrained Quadratic Programs Laura Galli 1 Adam N. Letchford 2 Lancaster, April 2011 1 DEIS, University of Bologna, Italy 2 Department of Management Science,
More informationIntroduction to Convex Optimization
Introduction to Convex Optimization Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2018-19, HKUST, Hong Kong Outline of Lecture Optimization
More informationCS711008Z Algorithm Design and Analysis
CS711008Z Algorithm Design and Analysis Lecture 8 Linear programming: interior point method Dongbo Bu Institute of Computing Technology Chinese Academy of Sciences, Beijing, China 1 / 31 Outline Brief
More information10 Numerical methods for constrained problems
10 Numerical methods for constrained problems min s.t. f(x) h(x) = 0 (l), g(x) 0 (m), x X The algorithms can be roughly divided the following way: ˆ primal methods: find descent direction keeping inside
More informationCS-E4830 Kernel Methods in Machine Learning
CS-E4830 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 27. September, 2017 Juho Rousu 27. September, 2017 1 / 45 Convex optimization Convex optimisation This
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More informationConvex Optimization and Support Vector Machine
Convex Optimization and Support Vector Machine Problem 0. Consider a two-class classification problem. The training data is L n = {(x 1, t 1 ),..., (x n, t n )}, where each t i { 1, 1} and x i R p. We
More informationStrong duality in Lasserre s hierarchy for polynomial optimization
Strong duality in Lasserre s hierarchy for polynomial optimization arxiv:1405.7334v1 [math.oc] 28 May 2014 Cédric Josz 1,2, Didier Henrion 3,4,5 Draft of January 24, 2018 Abstract A polynomial optimization
More informationCopositive Programming and Combinatorial Optimization
Copositive Programming and Combinatorial Optimization Franz Rendl http://www.math.uni-klu.ac.at Alpen-Adria-Universität Klagenfurt Austria joint work with I.M. Bomze (Wien) and F. Jarre (Düsseldorf) IMA
More informationSemidefinite Programming Basics and Applications
Semidefinite Programming Basics and Applications Ray Pörn, principal lecturer Åbo Akademi University Novia University of Applied Sciences Content What is semidefinite programming (SDP)? How to represent
More informationA direct formulation for sparse PCA using semidefinite programming
A direct formulation for sparse PCA using semidefinite programming A. d Aspremont, L. El Ghaoui, M. Jordan, G. Lanckriet ORFE, Princeton University & EECS, U.C. Berkeley A. d Aspremont, INFORMS, Denver,
More informationIII. Applications in convex optimization
III. Applications in convex optimization nonsymmetric interior-point methods partial separability and decomposition partial separability first order methods interior-point methods Conic linear optimization
More informationAdditional Exercises for Introduction to Nonlinear Optimization Amir Beck March 16, 2017
Additional Exercises for Introduction to Nonlinear Optimization Amir Beck March 16, 2017 Chapter 1 - Mathematical Preliminaries 1.1 Let S R n. (a) Suppose that T is an open set satisfying T S. Prove that
More informationAgenda. Applications of semidefinite programming. 1 Control and system theory. 2 Combinatorial and nonconvex optimization
Agenda Applications of semidefinite programming 1 Control and system theory 2 Combinatorial and nonconvex optimization 3 Spectral estimation & super-resolution Control and system theory SDP in wide use
More informationPrimal-dual Subgradient Method for Convex Problems with Functional Constraints
Primal-dual Subgradient Method for Convex Problems with Functional Constraints Yurii Nesterov, CORE/INMA (UCL) Workshop on embedded optimization EMBOPT2014 September 9, 2014 (Lucca) Yu. Nesterov Primal-dual
More informationMIT Algebraic techniques and semidefinite optimization February 14, Lecture 3
MI 6.97 Algebraic techniques and semidefinite optimization February 4, 6 Lecture 3 Lecturer: Pablo A. Parrilo Scribe: Pablo A. Parrilo In this lecture, we will discuss one of the most important applications
More informationLecture 3 Interior Point Methods and Nonlinear Optimization
Lecture 3 Interior Point Methods and Nonlinear Optimization Robert J. Vanderbei April 16, 2012 Machine Learning Summer School La Palma http://www.princeton.edu/ rvdb Example: Basis Pursuit Denoising L
More informationSolving Dual Problems
Lecture 20 Solving Dual Problems We consider a constrained problem where, in addition to the constraint set X, there are also inequality and linear equality constraints. Specifically the minimization problem
More informationsubject to (x 2)(x 4) u,
Exercises Basic definitions 5.1 A simple example. Consider the optimization problem with variable x R. minimize x 2 + 1 subject to (x 2)(x 4) 0, (a) Analysis of primal problem. Give the feasible set, the
More informationLecture 18: Optimization Programming
Fall, 2016 Outline Unconstrained Optimization 1 Unconstrained Optimization 2 Equality-constrained Optimization Inequality-constrained Optimization Mixture-constrained Optimization 3 Quadratic Programming
More informationStructure of Valid Inequalities for Mixed Integer Conic Programs
Structure of Valid Inequalities for Mixed Integer Conic Programs Fatma Kılınç-Karzan Tepper School of Business Carnegie Mellon University 18 th Combinatorial Optimization Workshop Aussois, France January
More informationLecture #21. c T x Ax b. maximize subject to
COMPSCI 330: Design and Analysis of Algorithms 11/11/2014 Lecture #21 Lecturer: Debmalya Panigrahi Scribe: Samuel Haney 1 Overview In this lecture, we discuss linear programming. We first show that the
More informationOn the Local Quadratic Convergence of the Primal-Dual Augmented Lagrangian Method
Optimization Methods and Software Vol. 00, No. 00, Month 200x, 1 11 On the Local Quadratic Convergence of the Primal-Dual Augmented Lagrangian Method ROMAN A. POLYAK Department of SEOR and Mathematical
More informationReduced-order output feedback control design with specsdp, a code for linear / nonlinear SDP problems
Reduced-order output feedback control design with specsdp, a code for linear / nonlinear SDP problems J.B. Thevenet and P. Apkarian and D. Noll ONERA-CERT, Control System department, 2 Av. Edouard Belin,
More informationIntroduction to Semidefinite Programming I: Basic properties a
Introduction to Semidefinite Programming I: Basic properties and variations on the Goemans-Williamson approximation algorithm for max-cut MFO seminar on Semidefinite Programming May 30, 2010 Semidefinite
More informationPrimal-Dual Geometry of Level Sets and their Explanatory Value of the Practical Performance of Interior-Point Methods for Conic Optimization
Primal-Dual Geometry of Level Sets and their Explanatory Value of the Practical Performance of Interior-Point Methods for Conic Optimization Robert M. Freund M.I.T. June, 2010 from papers in SIOPT, Mathematics
More informationSOCP Relaxation of Sensor Network Localization
SOCP Relaxation of Sensor Network Localization Paul Tseng Mathematics, University of Washington Seattle IWCSN, Simon Fraser University August 19, 2006 Abstract This is a talk given at Int. Workshop on
More informationJeff Howbert Introduction to Machine Learning Winter
Classification / Regression Support Vector Machines Jeff Howbert Introduction to Machine Learning Winter 2012 1 Topics SVM classifiers for linearly separable classes SVM classifiers for non-linearly separable
More information4. Algebra and Duality
4-1 Algebra and Duality P. Parrilo and S. Lall, CDC 2003 2003.12.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone
More informationKernel Machines. Pradeep Ravikumar Co-instructor: Manuela Veloso. Machine Learning
Kernel Machines Pradeep Ravikumar Co-instructor: Manuela Veloso Machine Learning 10-701 SVM linearly separable case n training points (x 1,, x n ) d features x j is a d-dimensional vector Primal problem:
More informationRandomized Coordinate Descent Methods on Optimization Problems with Linearly Coupled Constraints
Randomized Coordinate Descent Methods on Optimization Problems with Linearly Coupled Constraints By I. Necoara, Y. Nesterov, and F. Glineur Lijun Xu Optimization Group Meeting November 27, 2012 Outline
More informationA notion of Total Dual Integrality for Convex, Semidefinite and Extended Formulations
A notion of for Convex, Semidefinite and Extended Formulations Marcel de Carli Silva Levent Tunçel April 26, 2018 A vector in R n is integral if each of its components is an integer, A vector in R n is
More informationSupport Vector Machine (SVM) & Kernel CE-717: Machine Learning Sharif University of Technology. M. Soleymani Fall 2012
Support Vector Machine (SVM) & Kernel CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2012 Linear classifier Which classifier? x 2 x 1 2 Linear classifier Margin concept x 2
More informationDuality. Geoff Gordon & Ryan Tibshirani Optimization /
Duality Geoff Gordon & Ryan Tibshirani Optimization 10-725 / 36-725 1 Duality in linear programs Suppose we want to find lower bound on the optimal value in our convex problem, B min x C f(x) E.g., consider
More informationLECTURE 13 LECTURE OUTLINE
LECTURE 13 LECTURE OUTLINE Problem Structures Separable problems Integer/discrete problems Branch-and-bound Large sum problems Problems with many constraints Conic Programming Second Order Cone Programming
More informationInterior Point Methods. We ll discuss linear programming first, followed by three nonlinear problems. Algorithms for Linear Programming Problems
AMSC 607 / CMSC 764 Advanced Numerical Optimization Fall 2008 UNIT 3: Constrained Optimization PART 4: Introduction to Interior Point Methods Dianne P. O Leary c 2008 Interior Point Methods We ll discuss
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationThe Lagrangian L : R d R m R r R is an (easier to optimize) lower bound on the original problem:
HT05: SC4 Statistical Data Mining and Machine Learning Dino Sejdinovic Department of Statistics Oxford Convex Optimization and slides based on Arthur Gretton s Advanced Topics in Machine Learning course
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 18
EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 18 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory May 31, 2012 Andre Tkacenko
More informationInterior Point Algorithms for Constrained Convex Optimization
Interior Point Algorithms for Constrained Convex Optimization Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Inequality constrained minimization problems
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationConvex Optimization. Newton s method. ENSAE: Optimisation 1/44
Convex Optimization Newton s method ENSAE: Optimisation 1/44 Unconstrained minimization minimize f(x) f convex, twice continuously differentiable (hence dom f open) we assume optimal value p = inf x f(x)
More informationMidterm Review. Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.
Midterm Review Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye (LY, Chapter 1-4, Appendices) 1 Separating hyperplane
More informationConvex Optimization. (EE227A: UC Berkeley) Lecture 6. Suvrit Sra. (Conic optimization) 07 Feb, 2013
Convex Optimization (EE227A: UC Berkeley) Lecture 6 (Conic optimization) 07 Feb, 2013 Suvrit Sra Organizational Info Quiz coming up on 19th Feb. Project teams by 19th Feb Good if you can mix your research
More informationProject Discussions: SNL/ADMM, MDP/Randomization, Quadratic Regularization, and Online Linear Programming
Project Discussions: SNL/ADMM, MDP/Randomization, Quadratic Regularization, and Online Linear Programming Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305,
More informationSupport Vector Machines
Support Vector Machines Support vector machines (SVMs) are one of the central concepts in all of machine learning. They are simply a combination of two ideas: linear classification via maximum (or optimal
More informationHomework Set #6 - Solutions
EE 15 - Applications of Convex Optimization in Signal Processing and Communications Dr Andre Tkacenko JPL Third Term 11-1 Homework Set #6 - Solutions 1 a The feasible set is the interval [ 4] The unique
More informationA solution approach for linear optimization with completely positive matrices
A solution approach for linear optimization with completely positive matrices Franz Rendl http://www.math.uni-klu.ac.at Alpen-Adria-Universität Klagenfurt Austria joint work with M. Bomze (Wien) and F.
More informationPOLYNOMIAL OPTIMIZATION WITH SUMS-OF-SQUARES INTERPOLANTS
POLYNOMIAL OPTIMIZATION WITH SUMS-OF-SQUARES INTERPOLANTS Sercan Yıldız syildiz@samsi.info in collaboration with Dávid Papp (NCSU) OPT Transition Workshop May 02, 2017 OUTLINE Polynomial optimization and
More informationLecture: Duality of LP, SOCP and SDP
1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:
More informationHilbert s 17th Problem to Semidefinite Programming & Convex Algebraic Geometry
Hilbert s 17th Problem to Semidefinite Programming & Convex Algebraic Geometry Rekha R. Thomas University of Washington, Seattle References Monique Laurent, Sums of squares, moment matrices and optimization
More information