Determinant maximization with linear. S. Boyd, L. Vandenberghe, S.P. Wu. Information Systems Laboratory. Stanford University


 Oscar Howard
 1 years ago
 Views:
Transcription
1 Determinant maximization with linear matrix inequality constraints S. Boyd, L. Vandenberghe, S.P. Wu Information Systems Laboratory Stanford University SCCM Seminar 5 February
2 MAXDET problem denition minimize c T x + log det G(x),1 subject to G(x) = G 0 + x 1 G 1 + +x m G m > 0 F(x) =F 0 +x 1 F 1 ++x m F m 0 { x2r m is variable { G i = G T i 2 R ll, F i = F T i 2 R nn { F (x) 0, G(x) > 0 called linear matrix inequalities { looks specialized, but includes wide variety of convex optimization problems { convex problem { tractable, in theory and practice { useful duality theory 2
3 Outline 1. examples of MAXDET probems 2. duality theory 3. interiorpoint methods 3
4 Special cases of MAXDET semidenite program (SDP) minimize c T x subject to F (x) =F 0 +x 1 F 1 ++x m F m 0,c r x opt F (x) 6 0 F (x) 0 LMI can represent many convex constraints linear inequalities, convex quadratic inequalities, matrix norm constraints,... linear program minimize c T x subject to a T x b i i; i =1;:::;n SDP with F (x) =diag (b, Ax) 4
5 analytic center of LMI minimize log det F (x),1 subject to F (x) = F 0 + x 1 F 1 + +x m F m > 0 { log det F (x),1 smooth, convex on fx j F (x) > 0g { optimal point x ac maximizes det F (x) { x ac called analytic center of LMI F (x) > 0 rx ac 5
6 Minimum volume ellipsoid around points nd min vol ellipsoid containing points x 1,..., x K 2 R n E ellipsoid E = fx jkax, bk 1g { center A,1 b { A = A T > 0, volume proportional to det A,1 minimize log det A,1 subject to A = A T > 0 kax i, bk 1; convex optimization problem in A, b (n + n(n +1)=2 vars) express constraints as LMI kax i, bk 1() I i=1;:::;k Ax i, b (Ax i, b) T
7 Maximum volume ellipsoid in polytope nd max vol ellips. in P = fx j a T i x b i; i E d s ellipsoid E = fby + d jkyk1g { center d { B = B T > 0, volume proportional to det B EP ()a T i (By + d) b i for all kyk 1 () sup a T By i + atd b i i kyk1 () kba i k + a T i d b i; i =1;:::;L convex constraint in B and d 7
8 maximum volume EP formulation as convex problem in variables B, d: maximize subject to B = B T > 0 kba i k + a Td b i i; i =1;:::;L log det B express constraints as LMI in B, d kba i k + a T i d b i () (b i, a Td)I i Ba i (Ba i ) T b i, a Td i hence, formulation as MAXDETproblem minimize log det B,1 subject to B> (b i,a Td)I i Ba i (Ba i ) T b i, a Td i ; i =1;:::;L 8
9 Experiment design estimate x from measurements y k = a T k x + w k; i =1;:::;N { a k 2fv 1 ;:::;v m g, v i given test vectors { w k IID N(0; 1) measurement noise { i = fraction of a k 's equal to v i { N m LS estimator: c x = error covariance 0 N X k=1 E( c x, x)( c x, x) T = 1 N a k a T k 1 C A,1 NX i=1 0 B m i v i v T i i=1 y k a k 1 C A,1 = 1 N E() optimal experiment design: choose i i 0; that make E() `small' mx i=1 i =1; { minimize max (E()) (Eoptimality) { minimize Tr E() (Aoptimality) { minimize det E() (Doptimality) all are MAXDET problems 9
10 Doptimal design minimize log det subject to i 0; mx i=1 mx i=1 i =1 0 B m i v i v T i i=1 i v i v T i > 0 1 C A,1 i =1;:::;m can add other convex constraints, e.g., { bounds on cost or time of measurements: c T i b i { no more than 80% of the measurements is concentrated in less than 20% of the test vectors bm=5c X i=1 [i] 0:8 ( [i] is ith largest component of ) 10
11 Positive denite matrix completion matrix A = A T { entries A ij, (i; j) 2N are xed { entries A ij, (i; j) 62 N are free positive denite completion choose free entries such that A>0(if possible) maximum entropy completion maximize subject to A>0 log det A property: (A,1 ) ij =0for i; j 62 N log det ij =,(A,1 ) ij ) 11
12 Moment problem there exists a probability distribution on R such that i = Et i ; i =1;:::;2n if and only if H() = ::: n,1 n 1 2 ::: n n n,1 n ::: 2n,2 2n,1 n n+1 ::: 2n,1 2n LMI in variables i hence, can solve maximize/minimize E(c 0 + c 1 t + +c 2n t 2n ) subject to i Et i i ; i =1;:::;2n over all probability distributions on R by solving SDP maximize/minimize c 0 + c c 2n 2n subject to i i i ; i =1;:::;2n H( 1 ;:::; 2n ) 0 12
13 Other applications { maximizing products of positive concave functions { minimum volume ellipsoid covering union or sum of ellipsoids { maximum volume ellipsoid in intersection or sum of ellipsoids { computing channel capacity in information theory { maximum likelihood estimation 13
14 MAXDET duality theory primal MAXDET problem minimize c T x + log det G(x),1 subject to G(x) =G 0 +x 1 G 1 ++x m G m > 0 F(x)=F 0 +x 1 F 1 ++x m F m 0 optimal value p? dual MAXDET problem maximize log det W, Tr G 0 W, Tr F 0 Z + l subject to Tr F i Z + Tr G i W = c i ; i =1;:::;m W >0; Z 0 variables W = W T 2 R ll, Z = Z T 2 R nn optimal value d? properties { p? d? (always) { p? = d? (usually) denition duality gap = primal objective, dual objective 14
15 Example: experiment design primal problem minimize subject to log det m X i=1 i =1 i 0; mx i=1 0 B m i v i v T i i=1 i v i v T i > 0 1 C A,1 i=1;:::;m dual problem maximize log det W subject to W = W T > 0 v T i Wv i 1; i =1;:::;m interpretation: W determines smallest ellipsoid with center at the origin and containing v i, i =1;:::;m 15
16 Central path: general general convex optimization problem f 0 ;C convex minimize f 0 (x) subject to x 2 C ' is barrier function for C { smooth, convex { '(x)!1as x(2 int central path x? (t) =argmin x2c (tf 0 (x) +'(x)) for t>0 16
17 Central path: MAXDET problem f 0 (x) = c T x + log det G(x),1 C = fx j F (x) 0g barrier function for LMI F (x) 0 '(x) = 8 >< >: log det F (x),1 if F (x) > 0 +1 otherwise MAXDET central path: x? (t) =argmin F (x) > 0 G(x) > 0 '(t; x), with '(t; x) =t c T x+ log det G(x),1 + log det F (x),1 example: SDP t =0 r x ac r t = 1,c c T x = p? 17
18 Pathfollowing for MAXDET properties of MAXDET central path { from x? (t), get dual feasible Z? (t), W? (t) { corresponding duality gap is n=t { x? (t)! optimal as t!1 pathfollowing algorithm given strictly feasible x, t 1 repeat 1. compute x? (t) using Newton's method 2. x := x? (t) 3. increase t until n=t < tol tradeo: large increase in t means { fast gap reduction (fewer outer iterations), but { many Newton steps to compute x? (t + ) (more Newton steps per outer iteration) 18
19 # Newton steps Complexity of Newton's method for selfconcordant functions denition: along a line Example: (K =2) (Nesterov & Nemirovsky, late 1980s) jf 000 (t)j Kf 00 (t) 3=2 '(t; x) =t(c T x+log det G(x),1 )+log det F (x),1 (t 1) complexity of Newton's method { theorem: #Newton steps to minimize '(t; x), starting from x (0) : #steps 10:7('(t; x (0) ), '? (t)) + 5 { empirically: #steps ('(t; x (0) ), '? (t)) '(t; x (0) ), '? (t) 19
20 Pathfollowing algorithm idea: choose t +, starting point c x for Newton alg. s.t. '(t + ; c x), '? (t + )= (bounds # Newton steps required to compute x? (t + )) in practice: use lower bound from duality '(t + ; x) c, '? (t + ) '(t + ; x) c + log det Z,1 + t log det W,1 + Tr G 0 W + Tr F 0 Z, l = '(t + ; x)+function c of W;Z 20
21 duality gap duality gap two extreme choices { xed reduction: c x = x? (t), t + = 1+ r 2=n t { predictor step along tangent of central path x? (t) x? (t + ) x? x? bx x? (t + ) x? (t) =10 xed reduction =50 xed reduction predictor predictor Newton iterations Newton iterations 21
22 Newton iterations Newton iterations Newton iterations Total complexity total number of Newton steps { upper bound: O p ( n log(1=)) { practice, xedreduction method: O p ( n log(1=)) { practice, with predictor steps: O (log(1=)) xed reduction predictor steps n l m one Newton step involves a leastsquares problem minimize F ~ 2 (v) + ~ G(v) F 2 F 22
23 Conclusion MAXDETproblem minimize c T x + log det G(x),1 subject to G(x) > 0; F (x) 0 arises in many dierent areas { includes SDP, LP, convex QCQP { geometrical problems involving ellipsoids { experiment design, max. likelihood estimation, channel capacity,... convex, hence can be solved very eciently software/paper available on ftp soon (anonymous ftp to isl.stanford.edu in /pub/boyd/maxdet) 23
Advances in Convex Optimization: Theory, Algorithms, and Applications
Advances in Convex Optimization: Theory, Algorithms, and Applications Stephen Boyd Electrical Engineering Department Stanford University (joint work with Lieven Vandenberghe, UCLA) ISIT 02 ISIT 02 Lausanne
More informationOptimization in. Stephen Boyd. 3rd SIAM Conf. Control & Applications. and Control Theory. System. Convex
Optimization in Convex and Control Theory System Stephen Boyd Engineering Department Electrical University Stanford 3rd SIAM Conf. Control & Applications 1 Basic idea Many problems arising in system and
More informationELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications
ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March
More informationAgenda. Interior Point Methods. 1 Barrier functions. 2 Analytic center. 3 Central path. 4 Barrier method. 5 Primaldual path following algorithms
Agenda Interior Point Methods 1 Barrier functions 2 Analytic center 3 Central path 4 Barrier method 5 Primaldual path following algorithms 6 Nesterov Todd scaling 7 Complexity analysis Interior point
More informationLecture 15 Newton Method and SelfConcordance. October 23, 2008
Newton Method and SelfConcordance October 23, 2008 Outline Lecture 15 Selfconcordance Notion Selfconcordant Functions Operations Preserving Selfconcordance Properties of Selfconcordant Functions Implications
More informationLECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE
LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE CONVEX ANALYSIS AND DUALITY Basic concepts of convex analysis Basic concepts of convex optimization Geometric duality framework  MC/MC Constrained optimization
More information8. Geometric problems
8. Geometric problems Convex Optimization Boyd & Vandenberghe extremal volume ellipsoids centering classification placement and facility location 8 1 Minimum volume ellipsoid around a set LöwnerJohn ellipsoid
More information4. Convex optimization problems
Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization
More information10. Unconstrained minimization
Convex Optimization Boyd & Vandenberghe 10. Unconstrained minimization terminology and assumptions gradient descent method steepest descent method Newton s method selfconcordant functions implementation
More informationPrimalDual InteriorPoint Methods. Ryan Tibshirani Convex Optimization /36725
PrimalDual InteriorPoint Methods Ryan Tibshirani Convex Optimization 10725/36725 Given the problem Last time: barrier method min x subject to f(x) h i (x) 0, i = 1,... m Ax = b where f, h i, i = 1,...
More informationLecture 14 Barrier method
L. Vandenberghe EE236A (Fall 201314) Lecture 14 Barrier method centering problem Newton decrement local convergence of Newton method shortstep barrier method global convergence of Newton method predictorcorrector
More informationLecture 6: Conic Optimization September 8
IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions
More informationRobust linear optimization under general norms
Operations Research Letters 3 (004) 50 56 Operations Research Letters www.elsevier.com/locate/dsw Robust linear optimization under general norms Dimitris Bertsimas a; ;, Dessislava Pachamanova b, Melvyn
More information15. Conic optimization
L. Vandenberghe EE236C (Spring 216) 15. Conic optimization conic linear program examples modeling duality 151 Generalized (conic) inequalities Conic inequality: a constraint x K where K is a convex cone
More informationLecture 8. Strong Duality Results. September 22, 2008
Strong Duality Results September 22, 2008 Outline Lecture 8 Slater Condition and its Variations Convex Objective with Linear Inequality Constraints Quadratic Objective over Quadratic Constraints Representation
More informationLecture: Convex Optimization Problems
1/36 Lecture: Convex Optimization Problems http://bicmr.pku.edu.cn/~wenzw/opt2015fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/36 optimization
More informationAgenda. Applications of semidefinite programming. 1 Control and system theory. 2 Combinatorial and nonconvex optimization
Agenda Applications of semidefinite programming 1 Control and system theory 2 Combinatorial and nonconvex optimization 3 Spectral estimation & superresolution Control and system theory SDP in wide use
More informationELE539A: Optimization of Communication Systems Lecture 6: Quadratic Programming, Geometric Programming, and Applications
ELE539A: Optimization of Communication Systems Lecture 6: Quadratic Programming, Geometric Programming, and Applications Professor M. Chiang Electrical Engineering Department, Princeton University February
More informationMotivation. Lecture 2 Topics from Optimization and Duality. network utility maximization (NUM) problem:
CDS270 Maryam Fazel Lecture 2 Topics from Optimization and Duality Motivation network utility maximization (NUM) problem: consider a network with S sources (users), each sending one flow at rate x s, through
More informationSparse Covariance Selection using Semidefinite Programming
Sparse Covariance Selection using Semidefinite Programming A. d Aspremont ORFE, Princeton University Joint work with O. Banerjee, L. El Ghaoui & G. Natsoulis, U.C. Berkeley & Iconix Pharmaceuticals Support
More informationInput: System of inequalities or equalities over the reals R. Output: Value for variables that minimizes cost function
Linear programming Input: System of inequalities or equalities over the reals R A linear cost function Output: Value for variables that minimizes cost function Example: Minimize 6x+4y Subject to 3x + 2y
More informationSemidefinite Programming
Semidefinite Programming Basics and SOS Fernando Mário de Oliveira Filho Campos do Jordão, 2 November 23 Available at: www.ime.usp.br/~fmario under talks Conic programming V is a real vector space h, i
More informationWHY DUALITY? Gradient descent Newton s method Quasinewton Conjugate gradients. No constraints. Nondifferentiable ???? Constrained problems? ????
DUALITY WHY DUALITY? No constraints f(x) Nondifferentiable f(x) Gradient descent Newton s method Quasinewton Conjugate gradients etc???? Constrained problems? f(x) subject to g(x) apple 0???? h(x) =0
More informationOptimization in Information Theory
Optimization in Information Theory Dawei Shen November 11, 2005 Abstract This tutorial introduces the application of optimization techniques in information theory. We revisit channel capacity problem from
More informationTutorial on Convex Optimization for Engineers Part II
Tutorial on Convex Optimization for Engineers Part II M.Sc. Jens Steinwandt Communications Research Laboratory Ilmenau University of Technology PO Box 100565 D98684 Ilmenau, Germany jens.steinwandt@tuilmenau.de
More information7. Statistical estimation
7. Statistical estimation Convex Optimization Boyd & Vandenberghe maximum likelihood estimation optimal detector design experiment design 7 1 Parametric distribution estimation distribution estimation
More informationPrimalDual InteriorPoint Methods for Linear Programming based on Newton s Method
PrimalDual InteriorPoint Methods for Linear Programming based on Newton s Method Robert M. Freund March, 2004 2004 Massachusetts Institute of Technology. The Problem The logarithmic barrier approach
More informationPrimalDual InteriorPoint Methods
PrimalDual InteriorPoint Methods Lecturer: Aarti Singh Coinstructor: Pradeep Ravikumar Convex Optimization 10725/36725 Outline Today: Primaldual interiorpoint method Special case: linear programming
More informationConvex Functions. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Convex Functions Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470  Convex Optimization Fall 201718, HKUST, Hong Kong Outline of Lecture Definition convex function Examples
More informationEE 227A: Convex Optimization and Applications October 14, 2008
EE 227A: Convex Optimization and Applications October 14, 2008 Lecture 13: SDP Duality Lecturer: Laurent El Ghaoui Reading assignment: Chapter 5 of BV. 13.1 Direct approach 13.1.1 Primal problem Consider
More informationLecture 5. Theorems of Alternatives and SelfDual Embedding
IE 8534 1 Lecture 5. Theorems of Alternatives and SelfDual Embedding IE 8534 2 A system of linear equations may not have a solution. It is well known that either Ax = c has a solution, or A T y = 0, c
More informationDuality in Linear Programs. Lecturer: Ryan Tibshirani Convex Optimization /36725
Duality in Linear Programs Lecturer: Ryan Tibshirani Convex Optimization 10725/36725 1 Last time: proximal gradient descent Consider the problem x g(x) + h(x) with g, h convex, g differentiable, and
More informationNewton s Method. Ryan Tibshirani Convex Optimization /36725
Newton s Method Ryan Tibshirani Convex Optimization 10725/36725 1 Last time: dual correspondences Given a function f : R n R, we define its conjugate f : R n R, Properties and examples: f (y) = max x
More informationSecondOrder Cone Program (SOCP) Detection and Transformation Algorithms for Optimization Software
and SecondOrder Cone Program () and Algorithms for Optimization Software Jared Erickson JaredErickson2012@u.northwestern.edu Robert 4er@northwestern.edu Northwestern University INFORMS Annual Meeting,
More informationConvex Optimization of Graph Laplacian Eigenvalues
Convex Optimization of Graph Laplacian Eigenvalues Stephen Boyd Stanford University (Joint work with Persi Diaconis, Arpita Ghosh, SeungJean Kim, Sanjay Lall, Pablo Parrilo, Amin Saberi, Jun Sun, Lin
More informationNewton s Method. Javier Peña Convex Optimization /36725
Newton s Method Javier Peña Convex Optimization 10725/36725 1 Last time: dual correspondences Given a function f : R n R, we define its conjugate f : R n R, f ( (y) = max y T x f(x) ) x Properties and
More informationAnalytic Center CuttingPlane Method
Analytic Center CuttingPlane Method S. Boyd, L. Vandenberghe, and J. Skaf April 14, 2011 Contents 1 Analytic center cuttingplane method 2 2 Computing the analytic center 3 3 Pruning constraints 5 4 Lower
More information10 Numerical methods for constrained problems
10 Numerical methods for constrained problems min s.t. f(x) h(x) = 0 (l), g(x) 0 (m), x X The algorithms can be roughly divided the following way: ˆ primal methods: find descent direction keeping inside
More informationKarushKuhnTucker Conditions. Lecturer: Ryan Tibshirani Convex Optimization /36725
KarushKuhnTucker Conditions Lecturer: Ryan Tibshirani Convex Optimization 10725/36725 1 Given a minimization problem Last time: duality min x subject to f(x) h i (x) 0, i = 1,... m l j (x) = 0, j =
More informationA Proof of the Converse for the Capacity of Gaussian MIMO Broadcast Channels
A Proof of the Converse for the Capacity of Gaussian MIMO Broadcast Channels Mehdi Mohseni Department of Electrical Engineering Stanford University Stanford, CA 94305, USA Email: mmohseni@stanford.edu
More informationMoments and Positive Polynomials for Optimization II: LP VERSUS SDPrelaxations
Moments and Positive Polynomials for Optimization II: LP VERSUS SDPrelaxations LAASCNRS and Institute of Mathematics, Toulouse, France Tutorial, IMS, Singapore 2012 LPrelaxations LP VERSUS SDPrelaxations
More informationLecture 4: Linear and quadratic problems
Lecture 4: Linear and quadratic problems linear programming examples and applications linear fractional programming quadratic optimization problems (quadratically constrained) quadratic programming secondorder
More informationIntroduction to optimization
Introduction to optimization Geir Dahl CMA, Dept. of Mathematics and Dept. of Informatics University of Oslo 1 / 24 The plan 1. The basic concepts 2. Some useful tools (linear programming = linear optimization)
More informationORIE 6326: Convex Optimization. QuasiNewton Methods
ORIE 6326: Convex Optimization QuasiNewton Methods Professor Udell Operations Research and Information Engineering Cornell April 10, 2017 Slides on steepest descent and analysis of Newton s method adapted
More informationContents Acknowledgements 1 Introduction 2 1 Conic programming Introduction Convex programming....
Pattern separation via ellipsoids and conic programming Fr. Glineur Faculte Polytechnique de Mons, Belgium Memoire presente dans le cadre du D.E.A. interuniversitaire en mathematiques 31 ao^ut 1998 Contents
More informationSecondorder cone programming
Outline Secondorder cone programming, PhD Lehigh University Department of Industrial and Systems Engineering February 10, 2009 Outline 1 Basic properties Spectral decomposition The cone of squares The
More informationApplied Lagrange Duality for Constrained Optimization
Applied Lagrange Duality for Constrained Optimization February 12, 2002 Overview The Practical Importance of Duality ffl Review of Convexity ffl A Separating Hyperplane Theorem ffl Definition of the Dual
More informationRankone LMIs and Lyapunov's Inequality. Gjerrit Meinsma 4. Abstract. We describe a new proof of the wellknown Lyapunov's matrix inequality about
Rankone LMIs and Lyapunov's Inequality Didier Henrion 1;; Gjerrit Meinsma Abstract We describe a new proof of the wellknown Lyapunov's matrix inequality about the location of the eigenvalues of a matrix
More informationLecture 1 Introduction
L. Vandenberghe EE236A (Fall 201314) Lecture 1 Introduction course overview linear optimization examples history approximate syllabus basic definitions linear optimization in vector and matrix notation
More informationSparse inverse covariance estimation with the lasso
Sparse inverse covariance estimation with the lasso Jerome Friedman Trevor Hastie and Robert Tibshirani November 8, 2007 Abstract We consider the problem of estimating sparse graphs by a lasso penalty
More informationDiffeomorphic Warping. Ben Recht August 17, 2006 Joint work with Ali Rahimi (Intel)
Diffeomorphic Warping Ben Recht August 17, 2006 Joint work with Ali Rahimi (Intel) What Manifold Learning Isn t Common features of Manifold Learning Algorithms: 11 charting Dense sampling Geometric Assumptions
More informationHW1 solutions. 1. α Ef(x) β, where Ef(x) is the expected value of f(x), i.e., Ef(x) = n. i=1 p if(a i ). (The function f : R R is given.
HW1 solutions Exercise 1 (Some sets of probability distributions.) Let x be a realvalued random variable with Prob(x = a i ) = p i, i = 1,..., n, where a 1 < a 2 < < a n. Of course p R n lies in the standard
More informationCopositive Programming and Combinatorial Optimization
Copositive Programming and Combinatorial Optimization Franz Rendl http://www.math.uniklu.ac.at AlpenAdriaUniversität Klagenfurt Austria joint work with M. Bomze (Wien) and F. Jarre (Düsseldorf) and
More informationThe proximal mapping
The proximal mapping http://bicmr.pku.edu.cn/~wenzw/opt2016fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes Outline 2/37 1 closed function 2 Conjugate function
More information14. Duality. ˆ Upper and lower bounds. ˆ General duality. ˆ Constraint qualifications. ˆ Counterexample. ˆ Complementary slackness.
CS/ECE/ISyE 524 Introduction to Optimization Spring 2016 17 14. Duality ˆ Upper and lower bounds ˆ General duality ˆ Constraint qualifications ˆ Counterexample ˆ Complementary slackness ˆ Examples ˆ Sensitivity
More informationLecture 7: Weak Duality
EE 227A: Conve Optimization and Applications February 7, 2012 Lecture 7: Weak Duality Lecturer: Laurent El Ghaoui 7.1 Lagrange Dual problem 7.1.1 Primal problem In this section, we consider a possibly
More informationCS 6820 Fall 2014 Lectures, October 320, 2014
Analysis of Algorithms Linear Programming Notes CS 6820 Fall 2014 Lectures, October 320, 2014 1 Linear programming The linear programming (LP) problem is the following optimization problem. We are given
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 15: Nonlinear optimization Prof. John Gunnar Carlsson November 1, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I November 1, 2010 1 / 24
More informationWhat can be expressed via Conic Quadratic and Semidefinite Programming?
What can be expressed via Conic Quadratic and Semidefinite Programming? A. Nemirovski Faculty of Industrial Engineering and Management Technion Israel Institute of Technology Abstract Tremendous recent
More informationA Distributed Newton Method for Network Utility Maximization, II: Convergence
A Distributed Newton Method for Network Utility Maximization, II: Convergence Ermin Wei, Asuman Ozdaglar, and Ali Jadbabaie October 31, 2012 Abstract The existing distributed algorithms for Network Utility
More information13. Convex programming
CS/ISyE/ECE 524 Introduction to Optimization Spring 2015 16 13. Convex programming Convex sets and functions Convex programs Hierarchy of complexity Example: geometric programming Laurent Lessard (www.laurentlessard.com)
More informationMVE165/MMG631 Linear and integer optimization with applications Lecture 5 Linear programming duality and sensitivity analysis
MVE165/MMG631 Linear and integer optimization with applications Lecture 5 Linear programming duality and sensitivity analysis AnnBrith Strömberg 2017 03 29 Lecture 4 Linear and integer optimization with
More informationNonsymmetric potentialreduction methods for general cones
CORE DISCUSSION PAPER 2006/34 Nonsymmetric potentialreduction methods for general cones Yu. Nesterov March 28, 2006 Abstract In this paper we propose two new nonsymmetric primaldual potentialreduction
More informationA SecondOrder PathFollowing Algorithm for Unconstrained Convex Optimization
A SecondOrder PathFollowing Algorithm for Unconstrained Convex Optimization Yinyu Ye Department is Management Science & Engineering and Institute of Computational & Mathematical Engineering Stanford
More informationLecture: Introduction to LP, SDP and SOCP
Lecture: Introduction to LP, SDP and SOCP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2015.html wenzw@pku.edu.cn Acknowledgement:
More informationConvex Optimization of Graph Laplacian Eigenvalues
Convex Optimization of Graph Laplacian Eigenvalues Stephen Boyd Abstract. We consider the problem of choosing the edge weights of an undirected graph so as to maximize or minimize some function of the
More informationDuality of LPs and Applications
Lecture 6 Duality of LPs and Applications Last lecture we introduced duality of linear programs. We saw how to form duals, and proved both the weak and strong duality theorems. In this lecture we will
More informationCopositive Programming and Combinatorial Optimization
Copositive Programming and Combinatorial Optimization Franz Rendl http://www.math.uniklu.ac.at AlpenAdriaUniversität Klagenfurt Austria joint work with I.M. Bomze (Wien) and F. Jarre (Düsseldorf) IMA
More informationLP Duality: outline. Duality theory for Linear Programming. alternatives. optimization I Idea: polyhedra
LP Duality: outline I Motivation and definition of a dual LP I Weak duality I Separating hyperplane theorem and theorems of the alternatives I Strong duality and complementary slackness I Using duality
More informationOptimization for Machine Learning
Optimization for Machine Learning (Problems; Algorithms  A) SUVRIT SRA Massachusetts Institute of Technology PKU Summer School on Data Science (July 2017) Course materials http://suvrit.de/teaching.html
More informationOn Conically Ordered Convex Programs
On Conically Ordered Convex Programs Shuzhong Zhang May 003; revised December 004 Abstract In this paper we study a special class of convex optimization problems called conically ordered convex programs
More informationA Review of Linear Programming
A Review of Linear Programming Instructor: Farid Alizadeh IEOR 4600y Spring 2001 February 14, 2001 1 Overview In this note we review the basic properties of linear programming including the primal simplex
More informationLecture 17: Primaldual interiorpoint methods part II
10725/36725: Convex Optimization Spring 2015 Lecture 17: Primaldual interiorpoint methods part II Lecturer: Javier Pena Scribes: Pinchao Zhang, Wei Ma Note: LaTeX template courtesy of UC Berkeley EECS
More informationYinyu Ye, MS&E, Stanford MS&E310 Lecture Note #06. The Simplex Method
The Simplex Method Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye (LY, Chapters 2.32.5, 3.13.4) 1 Geometry of Linear
More informationSelfConcordant Barrier Functions for Convex Optimization
Appendix F SelfConcordant Barrier Functions for Convex Optimization F.1 Introduction In this Appendix we present a framework for developing polynomialtime algorithms for the solution of convex optimization
More informationConvex Optimization and Modeling
Convex Optimization and Modeling Duality Theory and Optimality Conditions 5th lecture, 12.05.2010 Jun.Prof. Matthias Hein Program of today/next lecture Lagrangian and duality: the Lagrangian the dual
More informationMoments and Positive Polynomials for Optimization II: LP VERSUS SDPrelaxations
Moments and Positive Polynomials for Optimization II: LP VERSUS SDPrelaxations LAASCNRS and Institute of Mathematics, Toulouse, France EECI Course: February 2016 LPrelaxations LP VERSUS SDPrelaxations
More informationInterior Point Methods for LP
11.1 Interior Point Methods for LP Katta G. Murty, IOE 510, LP, U. Of Michigan, Ann Arbor, Winter 1997. Simplex Method  A Boundary Method: Starting at an extreme point of the feasible set, the simplex
More informationNonlinear Programming 3rd Edition. Theoretical Solutions Manual Chapter 6
Nonlinear Programming 3rd Edition Theoretical Solutions Manual Chapter 6 Dimitri P. Bertsekas Massachusetts Institute of Technology Athena Scientific, Belmont, Massachusetts 1 NOTE This manual contains
More informationRelaxations and Randomized Methods for Nonconvex QCQPs
Relaxations and Randomized Methods for Nonconvex QCQPs Alexandre d Aspremont, Stephen Boyd EE392o, Stanford University Autumn, 2003 Introduction While some special classes of nonconvex problems can be
More informationSemidefinite Programming, Combinatorial Optimization and Real Algebraic Geometry
Semidefinite Programming, Combinatorial Optimization and Real Algebraic Geometry assoc. prof., Ph.D. 1 1 UNM  Faculty of information studies Edinburgh, 16. September 2014 Outline Introduction Definition
More informationDual and primaldual methods
ELE 538B: LargeScale Optimization for Data Science Dual and primaldual methods Yuxin Chen Princeton University, Spring 2018 Outline Dual proximal gradient method Primaldual proximal gradient method
More informationNesterov s Optimal Gradient Methods
Yurii Nesterov http://www.core.ucl.ac.be/~nesterov Nesterov s Optimal Gradient Methods Xinhua Zhang Australian National University NICTA 1 Outline The problem from machine learning perspective Preliminaries
More informationCoordinate descent. Geoff Gordon & Ryan Tibshirani Optimization /
Coordinate descent Geoff Gordon & Ryan Tibshirani Optimization 10725 / 36725 1 Adding to the toolbox, with stats and ML in mind We ve seen several general and useful minimization tools Firstorder methods
More informationLecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem
Lecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem Michael Patriksson 00 The Relaxation Theorem 1 Problem: find f := infimum f(x), x subject to x S, (1a) (1b) where f : R n R
More informationLecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.
MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.
More informationA solution approach for linear optimization with completely positive matrices
A solution approach for linear optimization with completely positive matrices Franz Rendl http://www.math.uniklu.ac.at AlpenAdriaUniversität Klagenfurt Austria joint work with M. Bomze (Wien) and F.
More informationLinear Vector Optimization. Algorithms and Applications
Linear Vector Optimization. Algorithms and Applications Andreas Löhne MartinLutherUniversität HalleWittenberg, Germany ANZIAM 2013 Newcastle (Australia), February 4, 2013 based on Hamel, A.; Löhne,
More information1 Robust optimization
ORF 523 Lecture 16 Princeton University Instructor: A.A. Ahmadi Scribe: G. Hall Any typos should be emailed to a a a@princeton.edu. In this lecture, we give a brief introduction to robust optimization
More information Wellcharacterized problems, minmax relations, approximate certificates.  LP problems in the standard form, primal and dual linear programs
LPDuality ( Approximation Algorithms by V. Vazirani, Chapter 12)  Wellcharacterized problems, minmax relations, approximate certificates  LP problems in the standard form, primal and dual linear programs
More informationModule 04 Optimization Problems KKT Conditions & Solvers
Module 04 Optimization Problems KKT Conditions & Solvers Ahmad F. Taha EE 5243: Introduction to CyberPhysical Systems Email: ahmad.taha@utsa.edu Webpage: http://engineering.utsa.edu/ taha/index.html September
More informationLinear Algebra: Linear Systems and Matrices  Quadratic Forms and Deniteness  Eigenvalues and Markov Chains
Linear Algebra: Linear Systems and Matrices  Quadratic Forms and Deniteness  Eigenvalues and Markov Chains Joshua Wilde, revised by Isabel Tecu, Takeshi Suzuki and María José Boccardi August 3, 3 Systems
More informationInverses. Stephen Boyd. EE103 Stanford University. October 28, 2017
Inverses Stephen Boyd EE103 Stanford University October 28, 2017 Outline Left and right inverses Inverse Solving linear equations Examples Pseudoinverse Left and right inverses 2 Left inverses a number
More informationA CONIC DANTZIGWOLFE DECOMPOSITION APPROACH FOR LARGE SCALE SEMIDEFINITE PROGRAMMING
A CONIC DANTZIGWOLFE DECOMPOSITION APPROACH FOR LARGE SCALE SEMIDEFINITE PROGRAMMING Kartik Krishnan Advanced Optimization Laboratory McMaster University Joint work with Gema Plaza Martinez and Tamás
More informationLecture #21. c T x Ax b. maximize subject to
COMPSCI 330: Design and Analysis of Algorithms 11/11/2014 Lecture #21 Lecturer: Debmalya Panigrahi Scribe: Samuel Haney 1 Overview In this lecture, we discuss linear programming. We first show that the
More informationc 2005 Society for Industrial and Applied Mathematics
SIAM J. MATRIX ANAL. APPL. Vol. 27, No. 2, pp. 532 546 c 2005 Society for Industrial and Applied Mathematics LEASTSQUARES COVARIANCE MATRIX ADJUSTMENT STEPHEN BOYD AND LIN XIAO Abstract. We consider the
More informationEfficient Nonlinear Optimizations of Queuing Systems
Efficient Nonlinear Optimizations of Queuing Systems Mung Chiang, Arak Sutivong, and Stephen Boyd Electrical Engineering Department, Stanford University, CA 9435 Abstract We present a systematic treatment
More informationLecture 7 Duality II
L. Vandenberghe EE236A (Fall 201314) Lecture 7 Duality II sensitivity analysis twoperson zerosum games circuit interpretation 7 1 Sensitivity analysis purpose: extract from the solution of an LP information
More information10725/36725: Convex Optimization Prerequisite Topics
10725/36725: Convex Optimization Prerequisite Topics February 3, 2015 This is meant to be a brief, informal refresher of some topics that will form building blocks in this course. The content of the
More information1 The independent set problem
ORF 523 Lecture 11 Spring 2016, Princeton University Instructor: A.A. Ahmadi Scribe: G. Hall Tuesday, March 29, 2016 When in doubt on the accuracy of these notes, please cross chec with the instructor
More information