Secondorder cone programming


 Benjamin Lang
 1 years ago
 Views:
Transcription
1 Outline Secondorder cone programming, PhD Lehigh University Department of Industrial and Systems Engineering February 10, 2009
2 Outline 1 Basic properties Spectral decomposition The cone of squares The arrowhead operator 2 Notation Optimality conditions barrier functions perturbed optimality The Newton system 3 4
3 Basic properties Spectral decomposition The cone of squares The arrowhead operator A new For u, v R n define: u v = u T v; u 1 v 2:n + v 1 u 2:n ). Theorem Properties of ) 1 Distributive law: u v + w) = u v + u w. 2 Commutative law: u v = v u. 3 The unit element is ι = 1; 0), i.e., u ι = ι u = u. 4 Using the notation u 2 = u u we have u u 2 v) = u 2 u v). 5 Power associativity: u p = u u is welldefined. 6 Associativity does not hold in general.
4 Basic properties Spectral decomposition The cone of squares The arrowhead operator Spectral decomposition Every vector u R n can be written as u = λ 1 c 1) + λ 2 c 2), where c 1) and c 2) are on the boundary of the cone, and c 1)T c 2) = 0 c 1) c 2) = 0 c 1) c 1) = c 1) c 2) c 2) = c 2) c 1) + c 2) = ι c 1), c 2) : Jordan frame λ 1, λ 2 : eigenvalues or spectral values: λ 1,2 u) = u 1 ± u 2:n 2 Naturally: u L λ 1,2 u) 0
5 Basic properties Spectral decomposition The cone of squares The arrowhead operator The cone of squares Theorem A vector x is in a second order cone i.e., x 1 x 2:n 2 ) if and only if it can be written as the square of a vector under the multiplication, i.e., x = u u. u F = λ λ2 2 = 2 u 2, u 2 = max { λ 1, λ 2 } = u 1 + u 2:n 2, u 1 = λ 1 1 c1) + λ 1 2 c2), u 1 2 = λ c1) + λ c 2), where u u 1 = u 1 u = ι and u 1 2 u 1 2 = u.
6 Basic properties Spectral decomposition The cone of squares The arrowhead operator The arrowhead operator Since the mapping v u v is linear, it can be represented with a matrix. u 1 u 2... u n u 2 u 1 Arr u) =...., u n u 1 Now we have u v = Arr u) v = Arr u) Arr v) ι. Quadratic representation: Q u = 2 Arr u) 2 Arr u 2), thus Q u v) = 2u u v) u 2 v is a quadratic function.
7 Primaldual interiorpoint methods: notation Outline Notation Optimality conditions barrier functions perturbed optimality The Newton system K = L n1 L nk, A = A 1,..., A k), x = x 1 ;... ; x k), s = s 1 ;... ; s k), c = c 1 ;... ; c k). With this notation we can write k Ax = A i x i, i=1 ) A T y = A 1 T y;,... ; A k T y. Arr u) and Q u are block diagonal matrices built from the blocks Arr u i) and Q u i, respectively.
8 Notation Optimality conditions barrier functions perturbed optimality The Newton system Duality and optimality Weak duality always holds Primal dual) strict feasibility implies strong duality and dual primal) solvability Under strong duality, the optimality conditions for second order conic optimization are Ax = b, x K A T y + s = c, s K x s = 0. An equivalent form of the complementarity condition is c T x b T y = x T s = 0.
9 Notation Optimality conditions barrier functions perturbed optimality The Newton system The central path using barrier functions If x int L, consider ) φx) = ln x 2 1 x 2:n 2 2 = ln λ 1 x) ln λ 2 x), Goes to if x is getting close to the boundary of the cone. Derivatives: φx) = 2 x 1; x 2:n ) T x 2 1 x 2:n 2 2 = 2 x 1) T, where the inverse is taken in the Jordan algebra.
10 Notation Optimality conditions barrier functions perturbed optimality The Newton system The central path Perturbed optimality conditions: Ax = b, x K A T y + s = c, s K where ι i = 1; 0;... ; 0) R n i. Newton system: x i s i = 2µι i, i = 1,..., k, A x = 0 A T y + s = 0, x i s i + x i s i = 2µι i x i s i, i = 1,..., k, where x = x 1 ;... ; x k ) and s = s 1 ;... ; s k ).
11 Newton system  rewritten Outline Notation Optimality conditions barrier functions perturbed optimality The Newton system A T A Arr s) I Arr x) y x s = 0 0 2µι x s, where ι = ι 1 ;... ; ι k ). Eliminating x and s: A Arr s) 1 Arr x) A T ) y = A Arr s) 1 2µι x s). Problems: Not symmetric May be singular Solution: symmetrization!
12 Outline min Q p 1c ) T Qp x) ) AQp 1 Qp x) = b Q p x K max b T y ) T AQp 1 y + Qp 1s = Q p 1c Q p 1s K Lemma If p int K, then 1 Q p Q p 1 = I. 2 The cone K is invariant, i.e., Q p K) = K. 3 The scaled and the original problems are equivalent.
13 Scaled optimality conditions Outline ) AQp 1 Qp x) = 0 ) T AQp 1 y + Qp 1 s = 0, Q p x) Q p 1 s ) + Q p x) Q p 1s ) = 2µι Q p x) Q p 1s ). Simplifies to A x = 0 A T y + s = 0, Q p x) Q p 1 s ) + Q p x) Q p 1s ) = 2µι Q p x) Q p 1s ). The last equation cannot be simplified!
14 The choice of p AHO: p = ι: does not provide a nonsingular Newton system HKM: p = s 1/2 or p = x 1/2, in which case Q p 1s = ι or Q p x = ι. Implemented in SDPT3. NT: Most popular one. p = Q x 1/2 Q x 1/2s) 1/2) 1/2 = Q s 1/2 Q s 1/2x) 1/2) 1/2. Simplifies to Q p x = Q p 1s. Implemented in SeDuMi, MOSEK, SDPT3.
15 Centrality measures µx, s) = k i=1 x it s i n i. w = w 1 ;... ; w k ), where w i = Q 1/2 x s i. i δ F x, s) := Q x 1/2 s µι F := k λ 1 w i ) µ) 2 + λ 2 w i ) µ) 2 i=1 δ x, s) := Q x 1/2 s µι 2 := max { λ 1w i ) µ, λ 2 w i ) µ } i=1,...,k δ x, s) := Qx 1/2 s µι) := µ min {λ 1w i ), λ 2 w 2 )}, i=1,...,k Neighbourhoods δ x, s) δ x, s) δ F x, s). N γ) := {x, y, s) strictly feasible : δx, s) γµx, s)}. δx, s) = δ F x, s): narrow neighbourhood δx, s) = δ x, s) wide neighbourhood
16 IPM for SOCP Theorem Shortstep IPM for SOCO) Choose γ = and ζ = Assume that we have a starting point x 0, y 0, s 0 ) N F γ). Compute the Newton step from the scaled Newton ) system. In every iteration, µ is decreased to 1 ζ k µ, i.e., θ = ζ k, and the stepsize is α = 1. This finds an εoptimal solution for the second order conic optimization problem with k second order cones in at most ) k 1 O log ε iterations. Independent of m, n!) The cost of one iteration is ) k O m 3 + m 2 n +. i=1 n 2 i
Approximate Farkas Lemmas in Convex Optimization
Approximate Farkas Lemmas in Convex Optimization Imre McMaster University Advanced Optimization Lab AdvOL Graduate Student Seminar October 25, 2004 1 Exact Farkas Lemma Motivation 2 3 Future plans The
More informationThe Q Method for Symmetric Cone Programmin
The Q Method for Symmetric Cone Programming The Q Method for Symmetric Cone Programmin Farid Alizadeh and Yu Xia alizadeh@rutcor.rutgers.edu, xiay@optlab.mcma Large Scale Nonlinear and Semidefinite Progra
More information15. Conic optimization
L. Vandenberghe EE236C (Spring 216) 15. Conic optimization conic linear program examples modeling duality 151 Generalized (conic) inequalities Conic inequality: a constraint x K where K is a convex cone
More informationSemidefinite Programming, Combinatorial Optimization and Real Algebraic Geometry
Semidefinite Programming, Combinatorial Optimization and Real Algebraic Geometry assoc. prof., Ph.D. 1 1 UNM  Faculty of information studies Edinburgh, 16. September 2014 Outline Introduction Definition
More informationLecture 17: Primaldual interiorpoint methods part II
10725/36725: Convex Optimization Spring 2015 Lecture 17: Primaldual interiorpoint methods part II Lecturer: Javier Pena Scribes: Pinchao Zhang, Wei Ma Note: LaTeX template courtesy of UC Berkeley EECS
More informationAgenda. Interior Point Methods. 1 Barrier functions. 2 Analytic center. 3 Central path. 4 Barrier method. 5 Primaldual path following algorithms
Agenda Interior Point Methods 1 Barrier functions 2 Analytic center 3 Central path 4 Barrier method 5 Primaldual path following algorithms 6 Nesterov Todd scaling 7 Complexity analysis Interior point
More informationPrimaldual pathfollowing algorithms for circular programming
Primaldual pathfollowing algorithms for circular programming Baha Alzalg Department of Mathematics, The University of Jordan, Amman 1194, Jordan July, 015 Abstract Circular programming problems are a
More informationLecture: Introduction to LP, SDP and SOCP
Lecture: Introduction to LP, SDP and SOCP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2015.html wenzw@pku.edu.cn Acknowledgement:
More informationInterior Point Methods for Convex Quadratic and Convex Nonlinear Programming
School of Mathematics T H E U N I V E R S I T Y O H F E D I N B U R G Interior Point Methods for Convex Quadratic and Convex Nonlinear Programming Jacek Gondzio Email: J.Gondzio@ed.ac.uk URL: http://www.maths.ed.ac.uk/~gondzio
More informationLecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.
MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.
More informationThe Q Method for Symmetric Cone Programming
The Q Method for Symmetric Cone Programming Farid Alizadeh Yu Xia October 5, 010 Communicated by F. Potra Abstract The Q method of semidefinite programming, developed by Alizadeh, Haeberly and Overton,
More informationInterior Point Methods for Linear Programming: Motivation & Theory
School of Mathematics T H E U N I V E R S I T Y O H F E D I N B U R G Interior Point Methods for Linear Programming: Motivation & Theory Jacek Gondzio Email: J.Gondzio@ed.ac.uk URL: http://www.maths.ed.ac.uk/~gondzio
More informationBasic Concepts in Linear Algebra
Basic Concepts in Linear Algebra Grady B Wright Department of Mathematics Boise State University February 2, 2015 Grady B Wright Linear Algebra Basics February 2, 2015 1 / 39 Numerical Linear Algebra Linear
More informationSparse Optimization Lecture: Basic Sparse Optimization Models
Sparse Optimization Lecture: Basic Sparse Optimization Models Instructor: Wotao Yin July 2013 online discussions on piazza.com Those who complete this lecture will know basic l 1, l 2,1, and nuclearnorm
More informationThe Q Method for SecondOrder Cone Programming
The Q Method for SecondOrder Cone Programming Yu Xia Farid Alizadeh July 5, 005 Key words. Secondorder cone programming, infeasible interior point method, the Q method Abstract We develop the Q method
More informationELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications
ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March
More informationSemidefinite Programming
Semidefinite Programming Basics and SOS Fernando Mário de Oliveira Filho Campos do Jordão, 2 November 23 Available at: www.ime.usp.br/~fmario under talks Conic programming V is a real vector space h, i
More informationLECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE
LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE CONVEX ANALYSIS AND DUALITY Basic concepts of convex analysis Basic concepts of convex optimization Geometric duality framework  MC/MC Constrained optimization
More informationA fullnewton step infeasible interiorpoint algorithm for linear programming based on a kernel function
A fullnewton step infeasible interiorpoint algorithm for linear programming based on a kernel function Zhongyi Liu, Wenyu Sun Abstract This paper proposes an infeasible interiorpoint algorithm with
More informationLecture 14 Barrier method
L. Vandenberghe EE236A (Fall 201314) Lecture 14 Barrier method centering problem Newton decrement local convergence of Newton method shortstep barrier method global convergence of Newton method predictorcorrector
More informationON POSITIVE SEMIDEFINITE PRESERVING STEIN TRANSFORMATION
J. Appl. Math. & Informatics Vol. 33(2015), No. 12, pp. 229234 http://dx.doi.org/10.14317/jami.2015.229 ON POSITIVE SEMIDEFINITE PRESERVING STEIN TRANSFORMATION YOON J. SONG Abstract. In the setting
More informationLecture 15 Newton Method and SelfConcordance. October 23, 2008
Newton Method and SelfConcordance October 23, 2008 Outline Lecture 15 Selfconcordance Notion Selfconcordant Functions Operations Preserving Selfconcordance Properties of Selfconcordant Functions Implications
More informationSecondorder cone programming
Math. Program., Ser. B 95: 3 51 (2003) Digital Object Identifier (DOI) 10.1007/s1010700203395 F. Alizadeh D. Goldfarb Secondorder cone programming Received: August 18, 2001 / Accepted: February 27,
More informationIMPLEMENTING THE NEW SELFREGULAR PROXIMITY BASED IPMS
IMPLEMENTING THE NEW SELFREGULAR PROXIMITY BASED IPMS IMPLEMENTING THE NEW SELFREGULAR PROXIMITY BASED IPMS By Xiaohang Zhu A thesis submitted to the School of Graduate Studies in Partial Fulfillment
More informationPrimalDual InteriorPoint Methods
PrimalDual InteriorPoint Methods Lecturer: Aarti Singh Coinstructor: Pradeep Ravikumar Convex Optimization 10725/36725 Outline Today: Primaldual interiorpoint method Special case: linear programming
More informationLecture 5. Theorems of Alternatives and SelfDual Embedding
IE 8534 1 Lecture 5. Theorems of Alternatives and SelfDual Embedding IE 8534 2 A system of linear equations may not have a solution. It is well known that either Ax = c has a solution, or A T y = 0, c
More information14. Duality. ˆ Upper and lower bounds. ˆ General duality. ˆ Constraint qualifications. ˆ Counterexample. ˆ Complementary slackness.
CS/ECE/ISyE 524 Introduction to Optimization Spring 2016 17 14. Duality ˆ Upper and lower bounds ˆ General duality ˆ Constraint qualifications ˆ Counterexample ˆ Complementary slackness ˆ Examples ˆ Sensitivity
More informationm i=1 c ix i i=1 F ix i F 0, X O.
What is SDP? for a beginner of SDP Copyright C 2005 SDPA Project 1 Introduction This note is a short course for SemiDefinite Programming s SDP beginners. SDP has various applications in, for example, control
More informationA CONIC DANTZIGWOLFE DECOMPOSITION APPROACH FOR LARGE SCALE SEMIDEFINITE PROGRAMMING
A CONIC DANTZIGWOLFE DECOMPOSITION APPROACH FOR LARGE SCALE SEMIDEFINITE PROGRAMMING Kartik Krishnan Advanced Optimization Laboratory McMaster University Joint work with Gema Plaza Martinez and Tamás
More information10 Numerical methods for constrained problems
10 Numerical methods for constrained problems min s.t. f(x) h(x) = 0 (l), g(x) 0 (m), x X The algorithms can be roughly divided the following way: ˆ primal methods: find descent direction keeping inside
More informationPrimalDual InteriorPoint Methods for Linear Programming based on Newton s Method
PrimalDual InteriorPoint Methods for Linear Programming based on Newton s Method Robert M. Freund March, 2004 2004 Massachusetts Institute of Technology. The Problem The logarithmic barrier approach
More informationProximal Methods for Optimization with Spasityinducing Norms
Proximal Methods for Optimization with Spasityinducing Norms Group Learning Presentation Xiaowei Zhou Department of Electronic and Computer Engineering The Hong Kong University of Science and Technology
More informationSelfConcordant Barrier Functions for Convex Optimization
Appendix F SelfConcordant Barrier Functions for Convex Optimization F.1 Introduction In this Appendix we present a framework for developing polynomialtime algorithms for the solution of convex optimization
More informationAdvanced Mathematical Programming IE417. Lecture 24. Dr. Ted Ralphs
Advanced Mathematical Programming IE417 Lecture 24 Dr. Ted Ralphs IE417 Lecture 24 1 Reading for This Lecture Sections 11.211.2 IE417 Lecture 24 2 The Linear Complementarity Problem Given M R p p and
More informationInteriorpoint methods Optimization Geoff Gordon Ryan Tibshirani
Interiorpoint methods 10725 Optimization Geoff Gordon Ryan Tibshirani Analytic center Review force field interpretation Newton s method: y = 1./(Ax+b) A T Y 2 A Δx = A T y Dikin ellipsoid unit ball of
More informationA CONIC INTERIOR POINT DECOMPOSITION APPROACH FOR LARGE SCALE SEMIDEFINITE PROGRAMMING
A CONIC INTERIOR POINT DECOMPOSITION APPROACH FOR LARGE SCALE SEMIDEFINITE PROGRAMMING Kartik Krishnan Sivaramakrishnan Department of Mathematics NC State University kksivara@ncsu.edu http://www4.ncsu.edu/
More informationProperties of Matrices and Operations on Matrices
Properties of Matrices and Operations on Matrices A common data structure for statistical analysis is a rectangular array or matris. Rows represent individual observational units, or just observations,
More informationThe momentlp and momentsos approaches
The momentlp and momentsos approaches LAASCNRS and Institute of Mathematics, Toulouse, France CIRM, November 2013 Semidefinite Programming Why polynomial optimization? LP and SDP CERTIFICATES of POSITIVITY
More informationLecture 6: Conic Optimization September 8
IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions
More informationEE 227A: Convex Optimization and Applications October 14, 2008
EE 227A: Convex Optimization and Applications October 14, 2008 Lecture 13: SDP Duality Lecturer: Laurent El Ghaoui Reading assignment: Chapter 5 of BV. 13.1 Direct approach 13.1.1 Primal problem Consider
More informationDeterminant maximization with linear. S. Boyd, L. Vandenberghe, S.P. Wu. Information Systems Laboratory. Stanford University
Determinant maximization with linear matrix inequality constraints S. Boyd, L. Vandenberghe, S.P. Wu Information Systems Laboratory Stanford University SCCM Seminar 5 February 1996 1 MAXDET problem denition
More informationLecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem
Lecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem Michael Patriksson 00 The Relaxation Theorem 1 Problem: find f := infimum f(x), x subject to x S, (1a) (1b) where f : R n R
More informationA SECOND ORDER MEHROTRATYPE PREDICTORCORRECTOR ALGORITHM FOR SEMIDEFINITE OPTIMIZATION
J Syst Sci Complex (01) 5: 1108 111 A SECOND ORDER MEHROTRATYPE PREDICTORCORRECTOR ALGORITHM FOR SEMIDEFINITE OPTIMIZATION Mingwang ZHANG DOI: 10.1007/s11440103179 Received: 3 December 010 / Revised:
More informationPrimalDual InteriorPoint Methods. Ryan Tibshirani Convex Optimization /36725
PrimalDual InteriorPoint Methods Ryan Tibshirani Convex Optimization 10725/36725 Given the problem Last time: barrier method min x subject to f(x) h i (x) 0, i = 1,... m Ax = b where f, h i, i = 1,...
More informationLinear Algebra review Powers of a diagonalizable matrix Spectral decomposition
Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Prof. Tesler Math 283 Fall 2016 Also see the separate version of this with Matlab and R commands. Prof. Tesler Diagonalizing
More informationOptimality, Duality, Complementarity for Constrained Optimization
Optimality, Duality, Complementarity for Constrained Optimization Stephen Wright University of WisconsinMadison May 2014 Wright (UWMadison) Optimality, Duality, Complementarity May 2014 1 / 41 Linear
More informationLinear Algebra Practice Problems
Linear Algebra Practice Problems Math 24 Calculus III Summer 25, Session II. Determine whether the given set is a vector space. If not, give at least one axiom that is not satisfied. Unless otherwise stated,
More informationLecture Note 1: Background
ECE5463: Introduction to Robotics Lecture Note 1: Background Prof. Wei Zhang Department of Electrical and Computer Engineering Ohio State University Columbus, Ohio, USA Spring 2018 Lecture 1 (ECE5463 Sp18)
More informationLecture: Convex Optimization Problems
1/36 Lecture: Convex Optimization Problems http://bicmr.pku.edu.cn/~wenzw/opt2015fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/36 optimization
More informationA class of Smoothing Method for Linear SecondOrder Cone Programming
Columbia International Publishing Journal of Advanced Computing (13) 1: 94 doi:1776/jac1313 Research Article A class of Smoothing Method for Linear SecondOrder Cone Programming Zhuqing Gui *, Zhibin
More informationConic Linear Programming. Yinyu Ye
Conic Linear Programming Yinyu Ye December 2004, revised October 2017 i ii Preface This monograph is developed for MS&E 314, Conic Linear Programming, which I am teaching at Stanford. Information, lecture
More informationELE539A: Optimization of Communication Systems Lecture 6: Quadratic Programming, Geometric Programming, and Applications
ELE539A: Optimization of Communication Systems Lecture 6: Quadratic Programming, Geometric Programming, and Applications Professor M. Chiang Electrical Engineering Department, Princeton University February
More informationAN EQUIVALENCY CONDITION OF NONSINGULARITY IN NONLINEAR SEMIDEFINITE PROGRAMMING
J Syst Sci Complex (2010) 23: 822 829 AN EQUVALENCY CONDTON OF NONSNGULARTY N NONLNEAR SEMDEFNTE PROGRAMMNG Chengjin L Wenyu SUN Raimundo J. B. de SAMPAO DO: 10.1007/s1142401080571 Received: 2 February
More informationFast ADMM for Sum of Squares Programs Using Partial Orthogonality
Fast ADMM for Sum of Squares Programs Using Partial Orthogonality Antonis Papachristodoulou Department of Engineering Science University of Oxford www.eng.ox.ac.uk/control/sysos antonis@eng.ox.ac.uk with
More informationNumerical Linear Algebra
Numerical Linear Algebra Direct Methods Philippe B. Laval KSU Fall 2017 Philippe B. Laval (KSU) Linear Systems: Direct Solution Methods Fall 2017 1 / 14 Introduction The solution of linear systems is one
More informationSome inequalities involving determinants, eigenvalues, and Schur complements in Euclidean Jordan algebras
positivity manuscript No. (will be inserted by the editor) Some inequalities involving determinants, eigenvalues, and Schur complements in Euclidean Jordan algebras M. Seetharama Gowda Jiyuan Tao February
More informationLinear Algebra. Workbook
Linear Algebra Workbook Paul Yiu Department of Mathematics Florida Atlantic University Last Update: November 21 Student: Fall 2011 Checklist Name: A B C D E F F G H I J 1 2 3 4 5 6 7 8 9 10 xxx xxx xxx
More informationThe Jordan Algebraic Structure of the Circular Cone
The Jordan Algebraic Structure of the Circular Cone Baha Alzalg Department of Mathematics, The University of Jordan, Amman 1194, Jordan Abstract In this paper, we study and analyze the algebraic structure
More informationA Distributed Newton Method for Network Utility Maximization, I: Algorithm
A Distributed Newton Method for Networ Utility Maximization, I: Algorithm Ermin Wei, Asuman Ozdaglar, and Ali Jadbabaie October 31, 2012 Abstract Most existing wors use dual decomposition and firstorder
More informationA Distributed Newton Method for Network Utility Maximization
A Distributed Newton Method for Networ Utility Maximization Ermin Wei, Asuman Ozdaglar, and Ali Jadbabaie Abstract Most existing wor uses dual decomposition and subgradient methods to solve Networ Utility
More informationALGEBRA QUALIFYING EXAM PROBLEMS LINEAR ALGEBRA
ALGEBRA QUALIFYING EXAM PROBLEMS LINEAR ALGEBRA Kent State University Department of Mathematical Sciences Compiled and Maintained by Donald L. White Version: August 29, 2017 CONTENTS LINEAR ALGEBRA AND
More informationFundamentals of Matrices
Maschinelles Lernen II Fundamentals of Matrices Christoph Sawade/Niels Landwehr/Blaine Nelson Tobias Scheffer Matrix Examples Recap: Data Linear Model: f i x = w i T x Let X = x x n be the data matrix
More informationAdvances in Convex Optimization: Theory, Algorithms, and Applications
Advances in Convex Optimization: Theory, Algorithms, and Applications Stephen Boyd Electrical Engineering Department Stanford University (joint work with Lieven Vandenberghe, UCLA) ISIT 02 ISIT 02 Lausanne
More informationSolution Methods. Richard Lusby. Department of Management Engineering Technical University of Denmark
Solution Methods Richard Lusby Department of Management Engineering Technical University of Denmark Lecture Overview (jg Unconstrained Several Variables Quadratic Programming Separable Programming SUMT
More informationIntroduction to optimization
Introduction to optimization Geir Dahl CMA, Dept. of Mathematics and Dept. of Informatics University of Oslo 1 / 24 The plan 1. The basic concepts 2. Some useful tools (linear programming = linear optimization)
More informationLecture #21. c T x Ax b. maximize subject to
COMPSCI 330: Design and Analysis of Algorithms 11/11/2014 Lecture #21 Lecturer: Debmalya Panigrahi Scribe: Samuel Haney 1 Overview In this lecture, we discuss linear programming. We first show that the
More informationThe Eigenvalue Problem: Perturbation Theory
Jim Lambers MAT 610 Summer Session 200910 Lecture 13 Notes These notes correspond to Sections 7.2 and 8.1 in the text. The Eigenvalue Problem: Perturbation Theory The Unsymmetric Eigenvalue Problem Just
More informationConjugate Gradient (CG) Method
Conjugate Gradient (CG) Method by K. Ozawa 1 Introduction In the series of this lecture, I will introduce the conjugate gradient method, which solves efficiently large scale sparse linear simultaneous
More informationKarushKuhnTucker Conditions. Lecturer: Ryan Tibshirani Convex Optimization /36725
KarushKuhnTucker Conditions Lecturer: Ryan Tibshirani Convex Optimization 10725/36725 1 Given a minimization problem Last time: duality min x subject to f(x) h i (x) 0, i = 1,... m l j (x) = 0, j =
More informationNewton s Method. Ryan Tibshirani Convex Optimization /36725
Newton s Method Ryan Tibshirani Convex Optimization 10725/36725 1 Last time: dual correspondences Given a function f : R n R, we define its conjugate f : R n R, Properties and examples: f (y) = max x
More informationImproving Performance of The Interior Point Method by Preconditioning
Improving Performance of The Interior Point Method by Preconditioning MidPoint Status Report Project by: Ken Ryals For: AMSC 663664 Fall 27Spring 28 6 December 27 Background / Refresher The IPM method
More informationMatrix Algebra, part 2
Matrix Algebra, part 2 MingChing Luoh 2005.9.12 1 / 38 Diagonalization and Spectral Decomposition of a Matrix Optimization 2 / 38 Diagonalization and Spectral Decomposition of a Matrix Also called Eigenvalues
More informationA study of search directions in primaldual interiorpoint methods for semidefinite programming
A study of search directions in primaldual interiorpoint methods for semidefinite programming M. J. Todd February 23, 1999 School of Operations Research and Industrial Engineering, Cornell University,
More informationFINITEDIMENSIONAL LINEAR ALGEBRA
DISCRETE MATHEMATICS AND ITS APPLICATIONS Series Editor KENNETH H ROSEN FINITEDIMENSIONAL LINEAR ALGEBRA Mark S Gockenbach Michigan Technological University Houghton, USA CRC Press Taylor & Francis Croup
More informationIntroduction to Numerical Analysis
Université de Liège Faculté des Sciences Appliquées Introduction to Numerical Analysis Edition 2015 Professor Q. Louveaux Department of Electrical Engineering and Computer Science Montefiore Institute
More informationFIXED POINT ITERATIONS
FIXED POINT ITERATIONS MARKUS GRASMAIR 1. Fixed Point Iteration for Nonlinear Equations Our goal is the solution of an equation (1) F (x) = 0, where F : R n R n is a continuous vector valued mapping in
More informationAn Introduction to Linear Matrix Inequalities. Raktim Bhattacharya Aerospace Engineering, Texas A&M University
An Introduction to Linear Matrix Inequalities Raktim Bhattacharya Aerospace Engineering, Texas A&M University Linear Matrix Inequalities What are they? Inequalities involving matrix variables Matrix variables
More informationSpectral radius, symmetric and positive matrices
Spectral radius, symmetric and positive matrices Zdeněk Dvořák April 28, 2016 1 Spectral radius Definition 1. The spectral radius of a square matrix A is ρ(a) = max{ λ : λ is an eigenvalue of A}. For an
More informationc 1995 Society for Industrial and Applied Mathematics Vol. 37, No. 1, pp , March
SIAM REVIEW. c 1995 Society for Industrial and Applied Mathematics Vol. 37, No. 1, pp. 93 97, March 1995 008 A UNIFIED PROOF FOR THE CONVERGENCE OF JACOBI AND GAUSSSEIDEL METHODS * ROBERTO BAGNARA Abstract.
More informationPrimaldual relationship between LevenbergMarquardt and central trajectories for linearly constrained convex optimization
Primaldual relationship between LevenbergMarquardt and central trajectories for linearly constrained convex optimization Roger Behling a, Clovis Gonzaga b and Gabriel Haeser c March 21, 2013 a Department
More informationGaussian Elimination without/with Pivoting and Cholesky Decomposition
Gaussian Elimination without/with Pivoting and Cholesky Decomposition Gaussian Elimination WITHOUT pivoting Notation: For a matrix A R n n we define for k {,,n} the leading principal submatrix a a k A
More informationPOWER flow studies are the cornerstone of power system
University of WisconsinMadison Department of Electrical and Computer Engineering. Technical Report ECE2. A Sufficient Condition for Power Flow Insolvability with Applications to Voltage Stability Margins
More informationLinear Equations and Matrix
1/60 ChiaPing Chen Professor Department of Computer Science and Engineering National Sun Yatsen University Linear Algebra Gaussian Elimination 2/60 Alpha Go Linear algebra begins with a system of linear
More informationUsing interior point methods for optimization in training very large scale Support Vector Machines. Interior Point Methods for QP. Elements of the IPM
School of Mathematics T H E U N I V E R S I T Y O H Using interior point methods for optimization in training very large scale Support Vector Machines F E D I N U R Part : Interior Point Methods for QP
More informationInexact primaldual pathfollowing algorithms for a special class of convex quadratic SDP and related problems
Inexact primaldual pathfollowing algorithms for a special class of convex quadratic SDP and related problems K.C. Toh, R. H. Tütüncü, and M. J. Todd February 8, 005 Dedicated to the memory of Jos Sturm:
More informationChap 2. Optimality conditions
Chap 2. Optimality conditions Version: 29092012 2.1 Optimality conditions in unconstrained optimization Recall the definitions of global, local minimizer. Geometry of minimization Consider for f C 1
More informationSecond Order Cone Programming Relaxation of Nonconvex Quadratic Optimization Problems
Second Order Cone Programming Relaxation of Nonconvex Quadratic Optimization Problems Sunyoung Kim Department of Mathematics, Ewha Women s University 111 Dahyundong, Sudaemoongu, Seoul 120750 Korea
More informationProjection methods in conic optimization
Projection methods in conic optimization Didier Henrion 1,2 Jérôme Malick 3 Abstract There exist efficient algorithms to project a point onto the intersection of a convex cone and an affine subspace. Those
More informationComputing regularization paths for learning multiple kernels
Computing regularization paths for learning multiple kernels Francis Bach Romain Thibaux Michael Jordan Computer Science, UC Berkeley December, 24 Code available at www.cs.berkeley.edu/~fbach Computing
More information1 Positive definiteness and semidefiniteness
Positive definiteness and semidefiniteness Zdeněk Dvořák May 9, 205 For integers a, b, and c, let D(a, b, c) be the diagonal matrix with + for i =,..., a, D i,i = for i = a +,..., a + b,. 0 for i = a +
More informationStrict diagonal dominance and a Geršgorin type theorem in Euclidean
Strict diagonal dominance and a Geršgorin type theorem in Euclidean Jordan algebras Melania Moldovan Department of Mathematics and Statistics University of Maryland, Baltimore County Baltimore, Maryland
More informationNumerical Methods I Solving Square Linear Systems: GEM and LU factorization
Numerical Methods I Solving Square Linear Systems: GEM and LU factorization Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATHGA 2011.003 / CSCIGA 2945.003, Fall 2014 September 18th,
More informationWhat can be expressed via Conic Quadratic and Semidefinite Programming?
What can be expressed via Conic Quadratic and Semidefinite Programming? A. Nemirovski Faculty of Industrial Engineering and Management Technion Israel Institute of Technology Abstract Tremendous recent
More informationSENSITIVITY ANALYSIS IN CONVEX QUADRATIC OPTIMIZATION: SIMULTANEOUS PERTURBATION OF THE OBJECTIVE AND RIGHTHANDSIDE VECTORS
SENSITIVITY ANALYSIS IN CONVEX QUADRATIC OPTIMIZATION: SIMULTANEOUS PERTURBATION OF THE OBJECTIVE AND RIGHTHANDSIDE VECTORS ALIREZA GHAFFARI HADIGHEH Department of Mathematics, Azarbaijan University
More informationSolving Dual Problems
Lecture 20 Solving Dual Problems We consider a constrained problem where, in addition to the constraint set X, there are also inequality and linear equality constraints. Specifically the minimization problem
More informationPart 5: Penalty and augmented Lagrangian methods for equality constrained optimization. Nick Gould (RAL)
Part 5: Penalty and augmented Lagrangian methods for equality constrained optimization Nick Gould (RAL) x IR n f(x) subject to c(x) = Part C course on continuoue optimization CONSTRAINED MINIMIZATION x
More informationConvexity of the Reachable Set of Nonlinear Systems under L 2 Bounded Controls
1 1 Convexity of the Reachable Set of Nonlinear Systems under L 2 Bounded Controls B.T.Polyak Institute for Control Science, Moscow, Russia email boris@ipu.rssi.ru Abstract Recently [1, 2] the new convexity
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 2
EE/ACM 150  Applications of Convex Optimization in Signal Processing and Communications Lecture 2 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory April 5, 2012 Andre Tkacenko
More informationThe Ongoing Development of CSDP
The Ongoing Development of CSDP Brian Borchers Department of Mathematics New Mexico Tech Socorro, NM 87801 borchers@nmt.edu Joseph Young Department of Mathematics New Mexico Tech (Now at Rice University)
More information