18. Primal-dual interior-point methods
|
|
- Mary Henry
- 5 years ago
- Views:
Transcription
1 L. Vandenberghe EE236C (Spring ) 18. Primal-dual interior-point methods primal-dual central path equations infeasible primal-dual method primal-dual method for self-dual embedding 18-1
2 Symmetric cone program primal and dual problem minimize c T x subject to Ax+s = b s maximize b T z subject to A T z +c = z inequalities are with respect to a symmetric cone optimality conditions s = A T A x z + c b (s,z), s T z = Primal-dual interior-point methods 18-2
3 Central path equations barrier function: we use the log-det barrier of lecture 17 φ(x) = logdetx a θ-normal barrier for K gradient is φ(x) = x 1 (see page 17-14) primal-dual central path equations s = A T A x z + c b (s,z), z = µ φ(s) = µs 1 last condition can be written symmetrically as s z = µe Primal-dual interior-point methods 18-3
4 Scaling scaling matrix: we call a nonsingular W a scaling matrix if multiplications with W and W T preserve the cone W intk = intk, W T intk = intk inverses are transformed as Wx 1 = (W T x) 1 scaled central path equations: for any scaling, central path is solution of s = A T A x z + c b (s,z), (W T s) (Wz) = µe Primal-dual interior-point methods 18-4
5 for a given pair (ŝ,ẑ), define where w satisfies ŝ = P(w)ẑ from page 17-21, Nesterov-Todd scaling W = W T = P(w 1/2 ) w = P(ẑ 1/2 ) ( ) 1/2 P(ẑ 1/2 )ŝ multiplications by W and W 1 map ŝ and ẑ to the same point: this implies that λ 2 2 = ŝ T ẑ W 1 ŝ = Wẑ = λ Primal-dual interior-point methods 18-5
6 NT scaling for nonnegative orthant W is a positive diagonal scaling ŝ1 /ẑ 1 W = P(w 1/2 ŝ2 /ẑ ) = ŝp /ẑ p scaling point is ( w = ŝ1 /ẑ 1, ) ŝ 2 /ẑ 2,..., ŝ p /ẑ p scaled ŝ, ẑ are λ = W T ŝ = Wẑ = ( ŝ1 ẑ 1, ŝ 2 ẑ 2,..., ŝ p ẑ p ) Primal-dual interior-point methods 18-6
7 NT scaling for second-order cone W is a hyperbolic Householder transformation W = P(w 1/2 ) = β(2vv T J), J = 1 I and β = wt J w, v = 2 1 wt J w w, w = w1/2 scaling point w can be computed from w = P(ẑ 1/2 ) ( ) 1/2 P(ẑ 1/2 )ŝ using the expressions for P and squareroot on pages 17-15, Primal-dual interior-point methods 18-7
8 NT scaling for positive semidefinite cone W is a symmetric congruence transformation Wy = vec (T 1/2 mat(y)t 1/2) where T = Ẑ 1/2( Ẑ 1/2 ŜẐ1/2) 1/2Ẑ 1/2 T = RR T with R computed as on page a simpler, nonsymmetric scaling is Wy = vec ( R T mat(y)r ), W T y = vec ( Rmat(y)R T) Primal-dual interior-point methods 18-8
9 Outline primal-dual central path equations infeasible primal-dual method primal-dual method for self-dual embedding
10 Basic primal-dual update suppose the current iterates are ŝ, ˆx, ẑ with ŝ, ẑ define µ = ŝ T ẑ/θ and compute the NT scaling matrix W for ŝ, ẑ compute s, x, z by linearizing the central path equation A T x c = + s A z b (W T s) (Wz) = σµe around ŝ, ˆx, ẑ, for some σ < 1 make an update (ŝ,ˆx) := (ŝ,ˆx)+α p ( x, s), ẑ := ẑ +α d z that preserves positivity of ŝ, ẑ Primal-dual interior-point methods 18-9
11 Linearized central path equation define λ = W T ŝ = Wẑ and ŝ A T ˆx ẑ c b r = A linearized central path equation s A T A x z = r second equation is linearization of λ ( W z +W T s ) = σµe λ λ (W T (ŝ+ s)) (W(ẑ + z)) = σµe Primal-dual interior-point methods 18-1
12 Path-following algorithm choose starting points ŝ, ˆx, ẑ with ŝ, ẑ 1. compute residuals and evaluate stopping criteria A T ˆx c r = ŝ A ẑ b terminate if r and ŝ T ẑ are sufficiently small 2. compute scaling matrix W associated with (ŝ, ẑ) and set λ := W T ŝ = Wẑ, µ := λt λ θ = ŝt ẑ θ Primal-dual interior-point methods 18-11
13 3. compute affine scaling direction by solving the linear equation s a A T A xa z a λ ( W z a +W T s a ) = λ λ = r 4. select barrier parameter σ = ( (ŝ+αp s a ) T (ẑ +α d z a ) ŝ T ẑ ) δ where δ is an algorithm parameter (a typical value is δ = 3) and α p = sup{α,1 ŝ+α s a } α d = sup{α,1 ẑ +α z a } Primal-dual interior-point methods 18-12
14 5. compute search direction by solving the linear equation 6. update iterates where s A T A x z = r λ (W z +W T s) = σµe λ λ (ˆx,ŝ) := (ˆx,ŝ)+min{1,.99α p }( x, s) ẑ := ẑ +min{1,.99α d } z α p = sup{α ŝ+α s }, α d = sup{α ẑ +α z } return to step 1 Primal-dual interior-point methods 18-13
15 interpretation and discussion step 3: affine scaling direction solves linearized central path equation with σ =, i.e., the linearized optimality conditions step 4 is a heuristic for choosing σ based on an estimate of the quality of the affine scaling direction σ is small if a step in the affine scaling direction gives a large reduction in ŝ T ẑ step 5: linear equation has same coefficient matrix as equation in step 3 if a direct method is used, we can reuse the factorization used in step 3, and solve the two equations at the cost of one Primal-dual interior-point methods 18-14
16 Mehrotra correction in step 5, solve s A T A x z = r λ (W z +W T s) = σµe λ λ (W T s a ) (W z a ) extra term on the r.h.s. is approximation of the second-order term in (W T (ŝ+ s)) (W(ẑ + z)) = σµe adding the correction typically saves a few iterations Primal-dual interior-point methods 18-15
17 Newton equations steps 3 and 5 reduce to equations A T A W T W x z = dx d z usually solved by eliminating z: A T W 1 W T A x = d x +A T W 1 W T d z a KKT system (see in BV for a discussion of solution methods) since W T W = P(w) = 2 φ(w) 1, A T W 1 W T A = A T 2 φ(w)a, the Hessian of the barrier function φ(b Ax) at the scaling point w Primal-dual interior-point methods 18-16
18 Quadratic cone program minimize (1/2)x T Qx+q T x subject to Ax+s = b s optimality conditions s = Q A T A x z + q b, (s,z), s T z = central path s = Q A T A x z + q b, (s,z), s z = µe Primal-dual interior-point methods 18-17
19 Path-following algorithm algorithm is almost identical to algorithm on page compute search directions from linearized central path equation; for example, step 5 becomes s Q A T A x z = r λ (W z +W T s) = σµe λ λ use equal primal and dual step sizes for example, in step 6, α p = α d = sup{α ŝ+α s,ẑ +α z } Primal-dual interior-point methods 18-18
20 Outline primal-dual central path equations infeasible primal-dual method primal-dual method for self-dual embedding
21 Extended self-dual embedding minimize (θ + 1)γ subject to s κ = A T c q x A b q z c T b T q τ qx T qz T q τ x z τ θ + θ +1 (s,κ,z,τ) θ is the logarithmic degree or rank of the cone q x, q z, q τ defined as q x q z q τ = θ +1 s T z +1 s 1 s, x, z are arbitrary with s, z AT c A b c T b T x z 1 Primal-dual interior-point methods 18-19
22 Optimality condition s κ = A T c q x A b q z c T b T q τ qx T qz T q τ x z τ γ + θ +1 (s,κ,z,τ), s T z +κτ = follows from self-dual property shows that γ = at optimum optimal solution gives nonzero solution of embedding of p.14-3 Primal-dual interior-point methods 18-2
23 Properties of extended self-dual embedding problem is strictly feasible; a strictly feasible point is given by (s, κ, x, z, τ, γ) = (s, 1, x, z, 1, st z +1 θ +1 ) (1) if s, κ, x, z, τ, γ satisfy equality constraint, then γ = st z +κτ θ +1 (take inner product with (x,z,τ,γ) on two sides of the equality) this is the extended embedding of page 14-34, but using variable γ instead of θ, and with a coefficient θ +1 in objective and r.h.s. Primal-dual interior-point methods 18-21
24 Central path for extended embedding s κ = A T c q x A b q z c T b T q τ qx T qz T q τ x z τ γ + θ +1 (s,κ,z,τ), s z = µe, κτ = µ inner product with (x,z,τ,γ) shows that on the central path γ = zt s+κτ θ +1 = µ initial point (1) is on the central path with µ = (s T z +1)/(θ +1) Primal-dual interior-point methods 18-22
25 Simplified central path equations s κ = AT c A b c T b T x z τ +µ q x q z q τ (s,κ,z,τ), s z = µe, κτ = µ we eliminated variable γ because γ = µ on the central path we removed the 4th equality, because it is implied by the first three (this follows by taking inner product with (x,z,τ)) can be seen as a shifted central path for the embedding on p.14-3 Primal-dual interior-point methods 18-23
26 Path-following algorithm choose starting points ŝ, ˆx, ẑ, with ŝ, ẑ ; set ˆκ := 1, ˆτ := 1 1. compute residuals and evaluate stopping criteria r = ŝ ˆκ AT c A b c T b T ˆx ẑ ˆτ terminate if r and ŝ T ẑ/τ 2 are sufficiently small, or an approximate certificate of primal or dual infeasibility has been found 2. compute scaling matrix W associated with (ŝ, ẑ) and set λ := W T ŝ = Wẑ, µ := ŝt ẑ + ˆκˆτ θ +1 Primal-dual interior-point methods 18-24
27 3. compute affine scaling direction by solving the linear equation s a κ a AT c A b c T b T x a z a τ a = r λ ( W z a +W T s a ) = λ λ, ˆκ τa + ˆτ κ a = ˆκˆτ 4. select barrier parameter σ := (1 α) δ where δ is an algorithm parameter (typical value is δ = 3) and α = sup{α,1 (ŝ,ˆκ,ẑ,ˆτ)+α( s a, κ a, z a, τ a ) } Primal-dual interior-point methods 18-25
28 5. compute search direction by solving the linear equation s κ AT c A b c T b T x z τ = (1 σ)r λ (W z +W T s) = σµe λ λ (W T s a ) (W z a ) ˆκ τ + ˆτ κ = σµ ˆκˆτ κ a τ a 6. update iterates (ŝ,ˆκ,ˆx,ẑ,ˆτ) := (ŝ,ˆκ,ˆx,ẑ,ˆτ)+min{1,.99α}( s, κ, x, z, τ) where α = sup{α,1 (ŝ,ˆκ,ẑ,ˆτ)+α( s, κ, z, τ) } return to step 1 Primal-dual interior-point methods 18-26
29 properties (without proof) step 3: affine scaling direction satisfies ŝ T z a +ẑ T s a = ŝ T ẑ, ˆκ τ a + ˆτ κ a = ˆκˆτ s T a z a + τ a κ a = step 5: search direction satisfies ŝ T z + ˆκ τ +ẑ T s+ ˆτ κ = (1 σ)(ŝ T ẑ + ˆκˆτ) s T z + τ κ = Primal-dual interior-point methods 18-27
30 discussion step 4: expression for σ is based on simplifiying σ = ( (ŝ+α sa ) T (ẑ +α z a )+(ˆκ+α κ a )(ˆτ +α τ a ) ŝ T ẑ + ˆκˆτ ) δ steps 5 and 6: gap and residual decrease linearly with α: µ + = (1 α(1 σ))µ, r + = (1 α(1 σ))r, if µ + and r + are the values of µ and r at the next iteration r = µq, with q defined on p (a multiple of the initial residual) in step 5, (1 σ)r = r +σµq: the equation is the linearization of the central path equation of p for barrier parameter σµ Primal-dual interior-point methods 18-28
31 Linear algebra complexity essentially the same as for the method on page eliminating τ, κ in steps 3 and 5 requires solution of an extra system A T A W T W x z = c b this increases the number of KKT systems solved per iteration to 3 (as opposed to 2 in the method on page 18-11) Primal-dual interior-point methods 18-29
32 References implementations of primal-dual algorithms based on Nesterov-Todd scaling J.F. Sturm, Implementation of interior-point methods for mixed semidefinite and second order cone optimization problems, Optimization methods and Software (22) an overview of Sedumi R.H. Tütüncü, K.C. Toh, M.J. Todd, Solving semidefinite-quadratic-linear programs using SDPT3, Mathematical Programming (23) an overview of SDPT3 CVXOPT (cvxopt.org) the conelp and coneqp solvers implement the algorithms in on page and page M.S. Andersen, J. Dahl, L. Vandenberghe, Interior-point methods for large-scale cone programming, in: S. Sra, S. Nowozin, S.J. Wright (eds.), Optimization for Machine Learning, 211 numerical results with the CVXOPT solvers on problems with structure Primal-dual interior-point methods 18-3
Agenda. Interior Point Methods. 1 Barrier functions. 2 Analytic center. 3 Central path. 4 Barrier method. 5 Primal-dual path following algorithms
Agenda Interior Point Methods 1 Barrier functions 2 Analytic center 3 Central path 4 Barrier method 5 Primal-dual path following algorithms 6 Nesterov Todd scaling 7 Complexity analysis Interior point
More information15. Conic optimization
L. Vandenberghe EE236C (Spring 216) 15. Conic optimization conic linear program examples modeling duality 15-1 Generalized (conic) inequalities Conic inequality: a constraint x K where K is a convex cone
More informationL. Vandenberghe EE236C (Spring 2016) 18. Symmetric cones. definition. spectral decomposition. quadratic representation. log-det barrier 18-1
L. Vandenberghe EE236C (Spring 2016) 18. Symmetric cones definition spectral decomposition quadratic representation log-det barrier 18-1 Introduction This lecture: theoretical properties of the following
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationResearch Note. A New Infeasible Interior-Point Algorithm with Full Nesterov-Todd Step for Semi-Definite Optimization
Iranian Journal of Operations Research Vol. 4, No. 1, 2013, pp. 88-107 Research Note A New Infeasible Interior-Point Algorithm with Full Nesterov-Todd Step for Semi-Definite Optimization B. Kheirfam We
More informationInterior-Point Methods
Interior-Point Methods Stephen Wright University of Wisconsin-Madison Simons, Berkeley, August, 2017 Wright (UW-Madison) Interior-Point Methods August 2017 1 / 48 Outline Introduction: Problems and Fundamentals
More informationConvex Optimization. Newton s method. ENSAE: Optimisation 1/44
Convex Optimization Newton s method ENSAE: Optimisation 1/44 Unconstrained minimization minimize f(x) f convex, twice continuously differentiable (hence dom f open) we assume optimal value p = inf x f(x)
More informationIII. Applications in convex optimization
III. Applications in convex optimization nonsymmetric interior-point methods partial separability and decomposition partial separability first order methods interior-point methods Conic linear optimization
More informationPrimal-Dual Interior-Point Methods. Javier Peña Convex Optimization /36-725
Primal-Dual Interior-Point Methods Javier Peña Convex Optimization 10-725/36-725 Last time: duality revisited Consider the problem min x subject to f(x) Ax = b h(x) 0 Lagrangian L(x, u, v) = f(x) + u T
More informationSecond-order cone programming
Outline Second-order cone programming, PhD Lehigh University Department of Industrial and Systems Engineering February 10, 2009 Outline 1 Basic properties Spectral decomposition The cone of squares The
More informationDuality revisited. Javier Peña Convex Optimization /36-725
Duality revisited Javier Peña Conve Optimization 10-725/36-725 1 Last time: barrier method Main idea: approimate the problem f() + I C () with the barrier problem f() + 1 t φ() tf() + φ() where t > 0 and
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationLecture 17: Primal-dual interior-point methods part II
10-725/36-725: Convex Optimization Spring 2015 Lecture 17: Primal-dual interior-point methods part II Lecturer: Javier Pena Scribes: Pinchao Zhang, Wei Ma Note: LaTeX template courtesy of UC Berkeley EECS
More informationLecture 15 Newton Method and Self-Concordance. October 23, 2008
Newton Method and Self-Concordance October 23, 2008 Outline Lecture 15 Self-concordance Notion Self-concordant Functions Operations Preserving Self-concordance Properties of Self-concordant Functions Implications
More informationLecture 14 Barrier method
L. Vandenberghe EE236A (Fall 2013-14) Lecture 14 Barrier method centering problem Newton decrement local convergence of Newton method short-step barrier method global convergence of Newton method predictor-corrector
More informationNonlinear Optimization for Optimal Control
Nonlinear Optimization for Optimal Control Pieter Abbeel UC Berkeley EECS Many slides and figures adapted from Stephen Boyd [optional] Boyd and Vandenberghe, Convex Optimization, Chapters 9 11 [optional]
More informationLecture: Algorithms for LP, SOCP and SDP
1/53 Lecture: Algorithms for LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:
More informationA PREDICTOR-CORRECTOR PATH-FOLLOWING ALGORITHM FOR SYMMETRIC OPTIMIZATION BASED ON DARVAY'S TECHNIQUE
Yugoslav Journal of Operations Research 24 (2014) Number 1, 35-51 DOI: 10.2298/YJOR120904016K A PREDICTOR-CORRECTOR PATH-FOLLOWING ALGORITHM FOR SYMMETRIC OPTIMIZATION BASED ON DARVAY'S TECHNIQUE BEHROUZ
More informationNew stopping criteria for detecting infeasibility in conic optimization
Optimization Letters manuscript No. (will be inserted by the editor) New stopping criteria for detecting infeasibility in conic optimization Imre Pólik Tamás Terlaky Received: March 21, 2008/ Accepted:
More informationInterior Point Algorithms for Constrained Convex Optimization
Interior Point Algorithms for Constrained Convex Optimization Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Inequality constrained minimization problems
More informationBarrier Method. Javier Peña Convex Optimization /36-725
Barrier Method Javier Peña Convex Optimization 10-725/36-725 1 Last time: Newton s method For root-finding F (x) = 0 x + = x F (x) 1 F (x) For optimization x f(x) x + = x 2 f(x) 1 f(x) Assume f strongly
More informationApproximate Farkas Lemmas in Convex Optimization
Approximate Farkas Lemmas in Convex Optimization Imre McMaster University Advanced Optimization Lab AdvOL Graduate Student Seminar October 25, 2004 1 Exact Farkas Lemma Motivation 2 3 Future plans The
More informationNonsymmetric potential-reduction methods for general cones
CORE DISCUSSION PAPER 2006/34 Nonsymmetric potential-reduction methods for general cones Yu. Nesterov March 28, 2006 Abstract In this paper we propose two new nonsymmetric primal-dual potential-reduction
More informationIMPLEMENTATION OF INTERIOR POINT METHODS
IMPLEMENTATION OF INTERIOR POINT METHODS IMPLEMENTATION OF INTERIOR POINT METHODS FOR SECOND ORDER CONIC OPTIMIZATION By Bixiang Wang, Ph.D. A Thesis Submitted to the School of Graduate Studies in Partial
More informationInterior Point Methods in Mathematical Programming
Interior Point Methods in Mathematical Programming Clóvis C. Gonzaga Federal University of Santa Catarina, Brazil Journées en l honneur de Pierre Huard Paris, novembre 2008 01 00 11 00 000 000 000 000
More information10-725/ Optimization Midterm Exam
10-725/36-725 Optimization Midterm Exam November 6, 2012 NAME: ANDREW ID: Instructions: This exam is 1hr 20mins long Except for a single two-sided sheet of notes, no other material or discussion is permitted
More informationMore First-Order Optimization Algorithms
More First-Order Optimization Algorithms Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapters 3, 8, 3 The SDM
More information2.098/6.255/ Optimization Methods Practice True/False Questions
2.098/6.255/15.093 Optimization Methods Practice True/False Questions December 11, 2009 Part I For each one of the statements below, state whether it is true or false. Include a 1-3 line supporting sentence
More informationLecture 14: Optimality Conditions for Conic Problems
EE 227A: Conve Optimization and Applications March 6, 2012 Lecture 14: Optimality Conditions for Conic Problems Lecturer: Laurent El Ghaoui Reading assignment: 5.5 of BV. 14.1 Optimality for Conic Problems
More information10 Numerical methods for constrained problems
10 Numerical methods for constrained problems min s.t. f(x) h(x) = 0 (l), g(x) 0 (m), x X The algorithms can be roughly divided the following way: ˆ primal methods: find descent direction keeping inside
More informationHomework 4. Convex Optimization /36-725
Homework 4 Convex Optimization 10-725/36-725 Due Friday November 4 at 5:30pm submitted to Christoph Dann in Gates 8013 (Remember to a submit separate writeup for each problem, with your name at the top)
More informationLecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.
MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.
More informationA full-newton step infeasible interior-point algorithm for linear programming based on a kernel function
A full-newton step infeasible interior-point algorithm for linear programming based on a kernel function Zhongyi Liu, Wenyu Sun Abstract This paper proposes an infeasible interior-point algorithm with
More informationWritten Examination
Division of Scientific Computing Department of Information Technology Uppsala University Optimization Written Examination 202-2-20 Time: 4:00-9:00 Allowed Tools: Pocket Calculator, one A4 paper with notes
More informationOn implementing a primal-dual interior-point method for conic quadratic optimization
On implementing a primal-dual interior-point method for conic quadratic optimization E. D. Andersen, C. Roos, and T. Terlaky December 18, 2000 Abstract Conic quadratic optimization is the problem of minimizing
More informationLecture 9 Sequential unconstrained minimization
S. Boyd EE364 Lecture 9 Sequential unconstrained minimization brief history of SUMT & IP methods logarithmic barrier function central path UMT & SUMT complexity analysis feasibility phase generalized inequalities
More informationInterior Point Methods. We ll discuss linear programming first, followed by three nonlinear problems. Algorithms for Linear Programming Problems
AMSC 607 / CMSC 764 Advanced Numerical Optimization Fall 2008 UNIT 3: Constrained Optimization PART 4: Introduction to Interior Point Methods Dianne P. O Leary c 2008 Interior Point Methods We ll discuss
More informationLecture 3. Optimization Problems and Iterative Algorithms
Lecture 3 Optimization Problems and Iterative Algorithms January 13, 2016 This material was jointly developed with Angelia Nedić at UIUC for IE 598ns Outline Special Functions: Linear, Quadratic, Convex
More informationAdvances in Convex Optimization: Theory, Algorithms, and Applications
Advances in Convex Optimization: Theory, Algorithms, and Applications Stephen Boyd Electrical Engineering Department Stanford University (joint work with Lieven Vandenberghe, UCLA) ISIT 02 ISIT 02 Lausanne
More informationLecture Note 5: Semidefinite Programming for Stability Analysis
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State
More informationLargest dual ellipsoids inscribed in dual cones
Largest dual ellipsoids inscribed in dual cones M. J. Todd June 23, 2005 Abstract Suppose x and s lie in the interiors of a cone K and its dual K respectively. We seek dual ellipsoidal norms such that
More informationAnalytic Center Cutting-Plane Method
Analytic Center Cutting-Plane Method S. Boyd, L. Vandenberghe, and J. Skaf April 14, 2011 Contents 1 Analytic center cutting-plane method 2 2 Computing the analytic center 3 3 Pruning constraints 5 4 Lower
More informationConic Linear Optimization and its Dual. yyye
Conic Linear Optimization and Appl. MS&E314 Lecture Note #04 1 Conic Linear Optimization and its Dual Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.
More informationConvex Optimization and l 1 -minimization
Convex Optimization and l 1 -minimization Sangwoon Yun Computational Sciences Korea Institute for Advanced Study December 11, 2009 2009 NIMS Thematic Winter School Outline I. Convex Optimization II. l
More informationInterior Point Methods for Mathematical Programming
Interior Point Methods for Mathematical Programming Clóvis C. Gonzaga Federal University of Santa Catarina, Florianópolis, Brazil EURO - 2013 Roma Our heroes Cauchy Newton Lagrange Early results Unconstrained
More informationInterior Point Methods for LP
11.1 Interior Point Methods for LP Katta G. Murty, IOE 510, LP, U. Of Michigan, Ann Arbor, Winter 1997. Simplex Method - A Boundary Method: Starting at an extreme point of the feasible set, the simplex
More informationPrimal-dual IPM with Asymmetric Barrier
Primal-dual IPM with Asymmetric Barrier Yurii Nesterov, CORE/INMA (UCL) September 29, 2008 (IFOR, ETHZ) Yu. Nesterov Primal-dual IPM with Asymmetric Barrier 1/28 Outline 1 Symmetric and asymmetric barriers
More informationIntroduction to Nonlinear Stochastic Programming
School of Mathematics T H E U N I V E R S I T Y O H F R G E D I N B U Introduction to Nonlinear Stochastic Programming Jacek Gondzio Email: J.Gondzio@ed.ac.uk URL: http://www.maths.ed.ac.uk/~gondzio SPS
More informationCS711008Z Algorithm Design and Analysis
CS711008Z Algorithm Design and Analysis Lecture 8 Linear programming: interior point method Dongbo Bu Institute of Computing Technology Chinese Academy of Sciences, Beijing, China 1 / 31 Outline Brief
More information1 Review of last lecture and introduction
Semidefinite Programming Lecture 10 OR 637 Spring 2008 April 16, 2008 (Wednesday) Instructor: Michael Jeremy Todd Scribe: Yogeshwer (Yogi) Sharma 1 Review of last lecture and introduction Let us first
More informationSMO vs PDCO for SVM: Sequential Minimal Optimization vs Primal-Dual interior method for Convex Objectives for Support Vector Machines
vs for SVM: Sequential Minimal Optimization vs Primal-Dual interior method for Convex Objectives for Support Vector Machines Ding Ma Michael Saunders Working paper, January 5 Introduction In machine learning,
More informationA Distributed Newton Method for Network Utility Maximization, II: Convergence
A Distributed Newton Method for Network Utility Maximization, II: Convergence Ermin Wei, Asuman Ozdaglar, and Ali Jadbabaie October 31, 2012 Abstract The existing distributed algorithms for Network Utility
More informationAlgorithms for Constrained Optimization
1 / 42 Algorithms for Constrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University April 19, 2015 2 / 42 Outline 1. Convergence 2. Sequential quadratic
More informationNewton s Method. Javier Peña Convex Optimization /36-725
Newton s Method Javier Peña Convex Optimization 10-725/36-725 1 Last time: dual correspondences Given a function f : R n R, we define its conjugate f : R n R, f ( (y) = max y T x f(x) ) x Properties and
More informationConvex Optimization : Conic Versus Functional Form
Convex Optimization : Conic Versus Functional Form Erling D. Andersen MOSEK ApS, Fruebjergvej 3, Box 16, DK 2100 Copenhagen, Blog: http://erlingdandersen.blogspot.com Linkedin: http://dk.linkedin.com/in/edandersen
More informationSolution Methods. Richard Lusby. Department of Management Engineering Technical University of Denmark
Solution Methods Richard Lusby Department of Management Engineering Technical University of Denmark Lecture Overview (jg Unconstrained Several Variables Quadratic Programming Separable Programming SUMT
More informationAn Infeasible Interior-Point Algorithm with full-newton Step for Linear Optimization
An Infeasible Interior-Point Algorithm with full-newton Step for Linear Optimization H. Mansouri M. Zangiabadi Y. Bai C. Roos Department of Mathematical Science, Shahrekord University, P.O. Box 115, Shahrekord,
More informationLecture 5. Theorems of Alternatives and Self-Dual Embedding
IE 8534 1 Lecture 5. Theorems of Alternatives and Self-Dual Embedding IE 8534 2 A system of linear equations may not have a solution. It is well known that either Ax = c has a solution, or A T y = 0, c
More information4TE3/6TE3. Algorithms for. Continuous Optimization
4TE3/6TE3 Algorithms for Continuous Optimization (Duality in Nonlinear Optimization ) Tamás TERLAKY Computing and Software McMaster University Hamilton, January 2004 terlaky@mcmaster.ca Tel: 27780 Optimality
More informationFull Newton step polynomial time methods for LO based on locally self concordant barrier functions
Full Newton step polynomial time methods for LO based on locally self concordant barrier functions (work in progress) Kees Roos and Hossein Mansouri e-mail: [C.Roos,H.Mansouri]@ewi.tudelft.nl URL: http://www.isa.ewi.tudelft.nl/
More informationFast Algorithms for SDPs derived from the Kalman-Yakubovich-Popov Lemma
Fast Algorithms for SDPs derived from the Kalman-Yakubovich-Popov Lemma Venkataramanan (Ragu) Balakrishnan School of ECE, Purdue University 8 September 2003 European Union RTN Summer School on Multi-Agent
More information4TE3/6TE3. Algorithms for. Continuous Optimization
4TE3/6TE3 Algorithms for Continuous Optimization (Algorithms for Constrained Nonlinear Optimization Problems) Tamás TERLAKY Computing and Software McMaster University Hamilton, November 2005 terlaky@mcmaster.ca
More informationSparse Optimization Lecture: Basic Sparse Optimization Models
Sparse Optimization Lecture: Basic Sparse Optimization Models Instructor: Wotao Yin July 2013 online discussions on piazza.com Those who complete this lecture will know basic l 1, l 2,1, and nuclear-norm
More informationUses of duality. Geoff Gordon & Ryan Tibshirani Optimization /
Uses of duality Geoff Gordon & Ryan Tibshirani Optimization 10-725 / 36-725 1 Remember conjugate functions Given f : R n R, the function is called its conjugate f (y) = max x R n yt x f(x) Conjugates appear
More informationSparse Matrix Theory and Semidefinite Optimization
Sparse Matrix Theory and Semidefinite Optimization Lieven Vandenberghe Department of Electrical Engineering University of California, Los Angeles Joint work with Martin S. Andersen and Yifan Sun Third
More informationPrimal-dual relationship between Levenberg-Marquardt and central trajectories for linearly constrained convex optimization
Primal-dual relationship between Levenberg-Marquardt and central trajectories for linearly constrained convex optimization Roger Behling a, Clovis Gonzaga b and Gabriel Haeser c March 21, 2013 a Department
More information11. Equality constrained minimization
Convex Optimization Boyd & Vandenberghe 11. Equality constrained minimization equality constrained minimization eliminating equality constraints Newton s method with equality constraints infeasible start
More informationCS-E4830 Kernel Methods in Machine Learning
CS-E4830 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 27. September, 2017 Juho Rousu 27. September, 2017 1 / 45 Convex optimization Convex optimisation This
More informationAdvanced Mathematical Programming IE417. Lecture 24. Dr. Ted Ralphs
Advanced Mathematical Programming IE417 Lecture 24 Dr. Ted Ralphs IE417 Lecture 24 1 Reading for This Lecture Sections 11.2-11.2 IE417 Lecture 24 2 The Linear Complementarity Problem Given M R p p and
More informationNonlinear Programming
Nonlinear Programming Kees Roos e-mail: C.Roos@ewi.tudelft.nl URL: http://www.isa.ewi.tudelft.nl/ roos LNMB Course De Uithof, Utrecht February 6 - May 8, A.D. 2006 Optimization Group 1 Outline for week
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 17
EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 17 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory May 29, 2012 Andre Tkacenko
More informationConvex Optimization. Prof. Nati Srebro. Lecture 12: Infeasible-Start Newton s Method Interior Point Methods
Convex Optimization Prof. Nati Srebro Lecture 12: Infeasible-Start Newton s Method Interior Point Methods Equality Constrained Optimization f 0 (x) s. t. A R p n, b R p Using access to: 2 nd order oracle
More informationA new primal-dual path-following method for convex quadratic programming
Volume 5, N., pp. 97 0, 006 Copyright 006 SBMAC ISSN 00-805 www.scielo.br/cam A new primal-dual path-following method for convex quadratic programming MOHAMED ACHACHE Département de Mathématiques, Faculté
More informationA FULL-NEWTON STEP INFEASIBLE-INTERIOR-POINT ALGORITHM COMPLEMENTARITY PROBLEMS
Yugoslav Journal of Operations Research 25 (205), Number, 57 72 DOI: 0.2298/YJOR3055034A A FULL-NEWTON STEP INFEASIBLE-INTERIOR-POINT ALGORITHM FOR P (κ)-horizontal LINEAR COMPLEMENTARITY PROBLEMS Soodabeh
More informationLinear Algebra and its Applications
Linear Algebra and its Applications 433 (2010) 1101 1109 Contents lists available at ScienceDirect Linear Algebra and its Applications journal homepage: www.elsevier.com/locate/laa Minimal condition number
More informationConvergence Analysis of Inexact Infeasible Interior Point Method. for Linear Optimization
Convergence Analysis of Inexact Infeasible Interior Point Method for Linear Optimization Ghussoun Al-Jeiroudi Jacek Gondzio School of Mathematics The University of Edinburgh Mayfield Road, Edinburgh EH9
More informationA Constraint-Reduced MPC Algorithm for Convex Quadratic Programming, with a Modified Active-Set Identification Scheme
A Constraint-Reduced MPC Algorithm for Convex Quadratic Programming, with a Modified Active-Set Identification Scheme M. Paul Laiu 1 and (presenter) André L. Tits 2 1 Oak Ridge National Laboratory laiump@ornl.gov
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Notes for EE227C (Spring 2018): Convex Optimization and Approximation Instructor: Moritz Hardt Email: hardt+ee227c@berkeley.edu Graduate Instructor: Max Simchowitz Email: msimchow+ee227c@berkeley.edu
More informationA priori bounds on the condition numbers in interior-point methods
A priori bounds on the condition numbers in interior-point methods Florian Jarre, Mathematisches Institut, Heinrich-Heine Universität Düsseldorf, Germany. Abstract Interior-point methods are known to be
More informationConstraint Reduction for Linear Programs with Many Constraints
Constraint Reduction for Linear Programs with Many Constraints André L. Tits Institute for Systems Research and Department of Electrical and Computer Engineering University of Maryland, College Park Pierre-Antoine
More informationInterior Point Methods for Linear Programming: Motivation & Theory
School of Mathematics T H E U N I V E R S I T Y O H F E D I N B U R G Interior Point Methods for Linear Programming: Motivation & Theory Jacek Gondzio Email: J.Gondzio@ed.ac.uk URL: http://www.maths.ed.ac.uk/~gondzio
More informationSolving conic optimization problems via self-dual embedding and facial reduction: a unified approach
Solving conic optimization problems via self-dual embedding and facial reduction: a unified approach Frank Permenter Henrik A. Friberg Erling D. Andersen August 18, 216 Abstract We establish connections
More information1 Computing with constraints
Notes for 2017-04-26 1 Computing with constraints Recall that our basic problem is minimize φ(x) s.t. x Ω where the feasible set Ω is defined by equality and inequality conditions Ω = {x R n : c i (x)
More informationExtreme Abridgment of Boyd and Vandenberghe s Convex Optimization
Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The
More information5 Handling Constraints
5 Handling Constraints Engineering design optimization problems are very rarely unconstrained. Moreover, the constraints that appear in these problems are typically nonlinear. This motivates our interest
More informationGeometric problems. Chapter Projection on a set. The distance of a point x 0 R n to a closed set C R n, in the norm, is defined as
Chapter 8 Geometric problems 8.1 Projection on a set The distance of a point x 0 R n to a closed set C R n, in the norm, is defined as dist(x 0,C) = inf{ x 0 x x C}. The infimum here is always achieved.
More informationSEMIDEFINITE PROGRAM BASICS. Contents
SEMIDEFINITE PROGRAM BASICS BRIAN AXELROD Abstract. A introduction to the basics of Semidefinite programs. Contents 1. Definitions and Preliminaries 1 1.1. Linear Algebra 1 1.2. Convex Analysis (on R n
More informationPrimal-Dual Interior-Point Methods. Ryan Tibshirani Convex Optimization /36-725
Primal-Dual Interior-Point Methods Ryan Tibshirani Convex Optimization 10-725/36-725 Given the problem Last time: barrier method min x subject to f(x) h i (x) 0, i = 1,... m Ax = b where f, h i, i = 1,...
More information5.5 Quadratic programming
5.5 Quadratic programming Minimize a quadratic function subject to linear constraints: 1 min x t Qx + c t x 2 s.t. a t i x b i i I (P a t i x = b i i E x R n, where Q is an n n matrix, I and E are the
More informationLecture 18: Optimization Programming
Fall, 2016 Outline Unconstrained Optimization 1 Unconstrained Optimization 2 Equality-constrained Optimization Inequality-constrained Optimization Mixture-constrained Optimization 3 Quadratic Programming
More informationSMCP Documentation. Release Martin S. Andersen and Lieven Vandenberghe
SMCP Documentation Release 0.4.5 Martin S. Andersen and Lieven Vandenberghe May 18, 2018 Contents 1 Current release 3 2 Future releases 5 3 Availability 7 4 Authors 9 5 Feedback and bug reports 11 5.1
More informationConvex Optimization. Lecture 12 - Equality Constrained Optimization. Instructor: Yuanzhang Xiao. Fall University of Hawaii at Manoa
Convex Optimization Lecture 12 - Equality Constrained Optimization Instructor: Yuanzhang Xiao University of Hawaii at Manoa Fall 2017 1 / 19 Today s Lecture 1 Basic Concepts 2 for Equality Constrained
More informationNewton s Method. Ryan Tibshirani Convex Optimization /36-725
Newton s Method Ryan Tibshirani Convex Optimization 10-725/36-725 1 Last time: dual correspondences Given a function f : R n R, we define its conjugate f : R n R, Properties and examples: f (y) = max x
More informationI.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More informationChapter 14 Linear Programming: Interior-Point Methods
Chapter 14 Linear Programming: Interior-Point Methods In the 1980s it was discovered that many large linear programs could be solved efficiently by formulating them as nonlinear problems and solving them
More informationMS&E 318 (CME 338) Large-Scale Numerical Optimization
Stanford University, Management Science & Engineering (and ICME) MS&E 318 (CME 338) Large-Scale Numerical Optimization 1 Origins Instructor: Michael Saunders Spring 2015 Notes 9: Augmented Lagrangian Methods
More informationCSCI 1951-G Optimization Methods in Finance Part 09: Interior Point Methods
CSCI 1951-G Optimization Methods in Finance Part 09: Interior Point Methods March 23, 2018 1 / 35 This material is covered in S. Boyd, L. Vandenberge s book Convex Optimization https://web.stanford.edu/~boyd/cvxbook/.
More informationLinear algebra issues in Interior Point methods for bound-constrained least-squares problems
Linear algebra issues in Interior Point methods for bound-constrained least-squares problems Stefania Bellavia Dipartimento di Energetica S. Stecco Università degli Studi di Firenze Joint work with Jacek
More informationLagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual
More informationImplementation of Warm-Start Strategies in Interior-Point Methods for Linear Programming in Fixed Dimension
Implementation of Warm-Start Strategies in Interior-Point Methods for Linear Programming in Fixed Dimension Elizabeth John E. Alper Yıldırım May 11, 2006 Abstract We implement several warm-start strategies
More information