# Second-order cone programming

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Outline Second-order cone programming, PhD Lehigh University Department of Industrial and Systems Engineering February 10, 2009

2 Outline 1 Basic properties Spectral decomposition The cone of squares The arrowhead operator 2 Notation Optimality conditions barrier functions perturbed optimality The Newton system 3 4

3 Basic properties Spectral decomposition The cone of squares The arrowhead operator A new For u, v R n define: u v = u T v; u 1 v 2:n + v 1 u 2:n ). Theorem Properties of ) 1 Distributive law: u v + w) = u v + u w. 2 Commutative law: u v = v u. 3 The unit element is ι = 1; 0), i.e., u ι = ι u = u. 4 Using the notation u 2 = u u we have u u 2 v) = u 2 u v). 5 Power associativity: u p = u u is well-defined. 6 Associativity does not hold in general.

4 Basic properties Spectral decomposition The cone of squares The arrowhead operator Spectral decomposition Every vector u R n can be written as u = λ 1 c 1) + λ 2 c 2), where c 1) and c 2) are on the boundary of the cone, and c 1)T c 2) = 0 c 1) c 2) = 0 c 1) c 1) = c 1) c 2) c 2) = c 2) c 1) + c 2) = ι c 1), c 2) : Jordan frame λ 1, λ 2 : eigenvalues or spectral values: λ 1,2 u) = u 1 ± u 2:n 2 Naturally: u L λ 1,2 u) 0

5 Basic properties Spectral decomposition The cone of squares The arrowhead operator The cone of squares Theorem A vector x is in a second order cone i.e., x 1 x 2:n 2 ) if and only if it can be written as the square of a vector under the multiplication, i.e., x = u u. u F = λ λ2 2 = 2 u 2, u 2 = max { λ 1, λ 2 } = u 1 + u 2:n 2, u 1 = λ 1 1 c1) + λ 1 2 c2), u 1 2 = λ c1) + λ c 2), where u u 1 = u 1 u = ι and u 1 2 u 1 2 = u.

6 Basic properties Spectral decomposition The cone of squares The arrowhead operator The arrowhead operator Since the mapping v u v is linear, it can be represented with a matrix. u 1 u 2... u n u 2 u 1 Arr u) =...., u n u 1 Now we have u v = Arr u) v = Arr u) Arr v) ι. Quadratic representation: Q u = 2 Arr u) 2 Arr u 2), thus Q u v) = 2u u v) u 2 v is a quadratic function.

7 Primal-dual interior-point methods: notation Outline Notation Optimality conditions barrier functions perturbed optimality The Newton system K = L n1 L nk, A = A 1,..., A k), x = x 1 ;... ; x k), s = s 1 ;... ; s k), c = c 1 ;... ; c k). With this notation we can write k Ax = A i x i, i=1 ) A T y = A 1 T y;,... ; A k T y. Arr u) and Q u are block diagonal matrices built from the blocks Arr u i) and Q u i, respectively.

8 Notation Optimality conditions barrier functions perturbed optimality The Newton system Duality and optimality Weak duality always holds Primal dual) strict feasibility implies strong duality and dual primal) solvability Under strong duality, the optimality conditions for second order conic optimization are Ax = b, x K A T y + s = c, s K x s = 0. An equivalent form of the complementarity condition is c T x b T y = x T s = 0.

9 Notation Optimality conditions barrier functions perturbed optimality The Newton system The central path using barrier functions If x int L, consider ) φx) = ln x 2 1 x 2:n 2 2 = ln λ 1 x) ln λ 2 x), Goes to if x is getting close to the boundary of the cone. Derivatives: φx) = 2 x 1; x 2:n ) T x 2 1 x 2:n 2 2 = 2 x 1) T, where the inverse is taken in the Jordan algebra.

10 Notation Optimality conditions barrier functions perturbed optimality The Newton system The central path Perturbed optimality conditions: Ax = b, x K A T y + s = c, s K where ι i = 1; 0;... ; 0) R n i. Newton system: x i s i = 2µι i, i = 1,..., k, A x = 0 A T y + s = 0, x i s i + x i s i = 2µι i x i s i, i = 1,..., k, where x = x 1 ;... ; x k ) and s = s 1 ;... ; s k ).

11 Newton system - rewritten Outline Notation Optimality conditions barrier functions perturbed optimality The Newton system A T A Arr s) I Arr x) y x s = 0 0 2µι x s, where ι = ι 1 ;... ; ι k ). Eliminating x and s: A Arr s) 1 Arr x) A T ) y = A Arr s) 1 2µι x s). Problems: Not symmetric May be singular Solution: symmetrization!

12 Outline min Q p 1c ) T Qp x) ) AQp 1 Qp x) = b Q p x K max b T y ) T AQp 1 y + Qp 1s = Q p 1c Q p 1s K Lemma If p int K, then 1 Q p Q p 1 = I. 2 The cone K is invariant, i.e., Q p K) = K. 3 The scaled and the original problems are equivalent.

13 Scaled optimality conditions Outline ) AQp 1 Qp x) = 0 ) T AQp 1 y + Qp 1 s = 0, Q p x) Q p 1 s ) + Q p x) Q p 1s ) = 2µι Q p x) Q p 1s ). Simplifies to A x = 0 A T y + s = 0, Q p x) Q p 1 s ) + Q p x) Q p 1s ) = 2µι Q p x) Q p 1s ). The last equation cannot be simplified!

14 The choice of p AHO: p = ι: does not provide a nonsingular Newton system HKM: p = s 1/2 or p = x 1/2, in which case Q p 1s = ι or Q p x = ι. Implemented in SDPT3. NT: Most popular one. p = Q x 1/2 Q x 1/2s) 1/2) 1/2 = Q s 1/2 Q s 1/2x) 1/2) 1/2. Simplifies to Q p x = Q p 1s. Implemented in SeDuMi, MOSEK, SDPT3.

15 Centrality measures µx, s) = k i=1 x it s i n i. w = w 1 ;... ; w k ), where w i = Q 1/2 x s i. i δ F x, s) := Q x 1/2 s µι F := k λ 1 w i ) µ) 2 + λ 2 w i ) µ) 2 i=1 δ x, s) := Q x 1/2 s µι 2 := max { λ 1w i ) µ, λ 2 w i ) µ } i=1,...,k δ x, s) := Qx 1/2 s µι) := µ min {λ 1w i ), λ 2 w 2 )}, i=1,...,k Neighbourhoods δ x, s) δ x, s) δ F x, s). N γ) := {x, y, s) strictly feasible : δx, s) γµx, s)}. δx, s) = δ F x, s): narrow neighbourhood δx, s) = δ x, s) wide neighbourhood

16 IPM for SOCP Theorem Short-step IPM for SOCO) Choose γ = and ζ = Assume that we have a starting point x 0, y 0, s 0 ) N F γ). Compute the Newton step from the scaled Newton ) system. In every iteration, µ is decreased to 1 ζ k µ, i.e., θ = ζ k, and the stepsize is α = 1. This finds an ε-optimal solution for the second order conic optimization problem with k second order cones in at most ) k 1 O log ε iterations. Independent of m, n!) The cost of one iteration is ) k O m 3 + m 2 n +. i=1 n 2 i

### Approximate Farkas Lemmas in Convex Optimization

Approximate Farkas Lemmas in Convex Optimization Imre McMaster University Advanced Optimization Lab AdvOL Graduate Student Seminar October 25, 2004 1 Exact Farkas Lemma Motivation 2 3 Future plans The

### The Q Method for Symmetric Cone Programmin

The Q Method for Symmetric Cone Programming The Q Method for Symmetric Cone Programmin Farid Alizadeh and Yu Xia alizadeh@rutcor.rutgers.edu, xiay@optlab.mcma Large Scale Nonlinear and Semidefinite Progra

### 15. Conic optimization

L. Vandenberghe EE236C (Spring 216) 15. Conic optimization conic linear program examples modeling duality 15-1 Generalized (conic) inequalities Conic inequality: a constraint x K where K is a convex cone

### Semidefinite Programming, Combinatorial Optimization and Real Algebraic Geometry

Semidefinite Programming, Combinatorial Optimization and Real Algebraic Geometry assoc. prof., Ph.D. 1 1 UNM - Faculty of information studies Edinburgh, 16. September 2014 Outline Introduction Definition

### Lecture 17: Primal-dual interior-point methods part II

10-725/36-725: Convex Optimization Spring 2015 Lecture 17: Primal-dual interior-point methods part II Lecturer: Javier Pena Scribes: Pinchao Zhang, Wei Ma Note: LaTeX template courtesy of UC Berkeley EECS

### Agenda. Interior Point Methods. 1 Barrier functions. 2 Analytic center. 3 Central path. 4 Barrier method. 5 Primal-dual path following algorithms

Agenda Interior Point Methods 1 Barrier functions 2 Analytic center 3 Central path 4 Barrier method 5 Primal-dual path following algorithms 6 Nesterov Todd scaling 7 Complexity analysis Interior point

### Primal-dual path-following algorithms for circular programming

Primal-dual path-following algorithms for circular programming Baha Alzalg Department of Mathematics, The University of Jordan, Amman 1194, Jordan July, 015 Abstract Circular programming problems are a

### Lecture: Introduction to LP, SDP and SOCP

Lecture: Introduction to LP, SDP and SOCP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2015.html wenzw@pku.edu.cn Acknowledgement:

### Interior Point Methods for Convex Quadratic and Convex Nonlinear Programming

School of Mathematics T H E U N I V E R S I T Y O H F E D I N B U R G Interior Point Methods for Convex Quadratic and Convex Nonlinear Programming Jacek Gondzio Email: J.Gondzio@ed.ac.uk URL: http://www.maths.ed.ac.uk/~gondzio

### Lecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.

MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.

### The Q Method for Symmetric Cone Programming

The Q Method for Symmetric Cone Programming Farid Alizadeh Yu Xia October 5, 010 Communicated by F. Potra Abstract The Q method of semidefinite programming, developed by Alizadeh, Haeberly and Overton,

### Interior Point Methods for Linear Programming: Motivation & Theory

School of Mathematics T H E U N I V E R S I T Y O H F E D I N B U R G Interior Point Methods for Linear Programming: Motivation & Theory Jacek Gondzio Email: J.Gondzio@ed.ac.uk URL: http://www.maths.ed.ac.uk/~gondzio

### Basic Concepts in Linear Algebra

Basic Concepts in Linear Algebra Grady B Wright Department of Mathematics Boise State University February 2, 2015 Grady B Wright Linear Algebra Basics February 2, 2015 1 / 39 Numerical Linear Algebra Linear

### Sparse Optimization Lecture: Basic Sparse Optimization Models

Sparse Optimization Lecture: Basic Sparse Optimization Models Instructor: Wotao Yin July 2013 online discussions on piazza.com Those who complete this lecture will know basic l 1, l 2,1, and nuclear-norm

### The Q Method for Second-Order Cone Programming

The Q Method for Second-Order Cone Programming Yu Xia Farid Alizadeh July 5, 005 Key words. Second-order cone programming, infeasible interior point method, the Q method Abstract We develop the Q method

### ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications

ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March

### Semidefinite Programming

Semidefinite Programming Basics and SOS Fernando Mário de Oliveira Filho Campos do Jordão, 2 November 23 Available at: www.ime.usp.br/~fmario under talks Conic programming V is a real vector space h, i

### LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE

LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE CONVEX ANALYSIS AND DUALITY Basic concepts of convex analysis Basic concepts of convex optimization Geometric duality framework - MC/MC Constrained optimization

### A full-newton step infeasible interior-point algorithm for linear programming based on a kernel function

A full-newton step infeasible interior-point algorithm for linear programming based on a kernel function Zhongyi Liu, Wenyu Sun Abstract This paper proposes an infeasible interior-point algorithm with

### Lecture 14 Barrier method

L. Vandenberghe EE236A (Fall 2013-14) Lecture 14 Barrier method centering problem Newton decrement local convergence of Newton method short-step barrier method global convergence of Newton method predictor-corrector

### ON POSITIVE SEMIDEFINITE PRESERVING STEIN TRANSFORMATION

J. Appl. Math. & Informatics Vol. 33(2015), No. 1-2, pp. 229-234 http://dx.doi.org/10.14317/jami.2015.229 ON POSITIVE SEMIDEFINITE PRESERVING STEIN TRANSFORMATION YOON J. SONG Abstract. In the setting

### Lecture 15 Newton Method and Self-Concordance. October 23, 2008

Newton Method and Self-Concordance October 23, 2008 Outline Lecture 15 Self-concordance Notion Self-concordant Functions Operations Preserving Self-concordance Properties of Self-concordant Functions Implications

### Second-order cone programming

Math. Program., Ser. B 95: 3 51 (2003) Digital Object Identifier (DOI) 10.1007/s10107-002-0339-5 F. Alizadeh D. Goldfarb Second-order cone programming Received: August 18, 2001 / Accepted: February 27,

### IMPLEMENTING THE NEW SELF-REGULAR PROXIMITY BASED IPMS

IMPLEMENTING THE NEW SELF-REGULAR PROXIMITY BASED IPMS IMPLEMENTING THE NEW SELF-REGULAR PROXIMITY BASED IPMS By Xiaohang Zhu A thesis submitted to the School of Graduate Studies in Partial Fulfillment

### Primal-Dual Interior-Point Methods

Primal-Dual Interior-Point Methods Lecturer: Aarti Singh Co-instructor: Pradeep Ravikumar Convex Optimization 10-725/36-725 Outline Today: Primal-dual interior-point method Special case: linear programming

### Lecture 5. Theorems of Alternatives and Self-Dual Embedding

IE 8534 1 Lecture 5. Theorems of Alternatives and Self-Dual Embedding IE 8534 2 A system of linear equations may not have a solution. It is well known that either Ax = c has a solution, or A T y = 0, c

### 14. Duality. ˆ Upper and lower bounds. ˆ General duality. ˆ Constraint qualifications. ˆ Counterexample. ˆ Complementary slackness.

CS/ECE/ISyE 524 Introduction to Optimization Spring 2016 17 14. Duality ˆ Upper and lower bounds ˆ General duality ˆ Constraint qualifications ˆ Counterexample ˆ Complementary slackness ˆ Examples ˆ Sensitivity

### m i=1 c ix i i=1 F ix i F 0, X O.

What is SDP? for a beginner of SDP Copyright C 2005 SDPA Project 1 Introduction This note is a short course for SemiDefinite Programming s SDP beginners. SDP has various applications in, for example, control

### A CONIC DANTZIG-WOLFE DECOMPOSITION APPROACH FOR LARGE SCALE SEMIDEFINITE PROGRAMMING

A CONIC DANTZIG-WOLFE DECOMPOSITION APPROACH FOR LARGE SCALE SEMIDEFINITE PROGRAMMING Kartik Krishnan Advanced Optimization Laboratory McMaster University Joint work with Gema Plaza Martinez and Tamás

### 10 Numerical methods for constrained problems

10 Numerical methods for constrained problems min s.t. f(x) h(x) = 0 (l), g(x) 0 (m), x X The algorithms can be roughly divided the following way: ˆ primal methods: find descent direction keeping inside

### Primal-Dual Interior-Point Methods for Linear Programming based on Newton s Method

Primal-Dual Interior-Point Methods for Linear Programming based on Newton s Method Robert M. Freund March, 2004 2004 Massachusetts Institute of Technology. The Problem The logarithmic barrier approach

### Proximal Methods for Optimization with Spasity-inducing Norms

Proximal Methods for Optimization with Spasity-inducing Norms Group Learning Presentation Xiaowei Zhou Department of Electronic and Computer Engineering The Hong Kong University of Science and Technology

### Self-Concordant Barrier Functions for Convex Optimization

Appendix F Self-Concordant Barrier Functions for Convex Optimization F.1 Introduction In this Appendix we present a framework for developing polynomial-time algorithms for the solution of convex optimization

### Advanced Mathematical Programming IE417. Lecture 24. Dr. Ted Ralphs

Advanced Mathematical Programming IE417 Lecture 24 Dr. Ted Ralphs IE417 Lecture 24 1 Reading for This Lecture Sections 11.2-11.2 IE417 Lecture 24 2 The Linear Complementarity Problem Given M R p p and

### Interior-point methods Optimization Geoff Gordon Ryan Tibshirani

Interior-point methods 10-725 Optimization Geoff Gordon Ryan Tibshirani Analytic center Review force field interpretation Newton s method: y = 1./(Ax+b) A T Y 2 A Δx = A T y Dikin ellipsoid unit ball of

### A CONIC INTERIOR POINT DECOMPOSITION APPROACH FOR LARGE SCALE SEMIDEFINITE PROGRAMMING

A CONIC INTERIOR POINT DECOMPOSITION APPROACH FOR LARGE SCALE SEMIDEFINITE PROGRAMMING Kartik Krishnan Sivaramakrishnan Department of Mathematics NC State University kksivara@ncsu.edu http://www4.ncsu.edu/

### Properties of Matrices and Operations on Matrices

Properties of Matrices and Operations on Matrices A common data structure for statistical analysis is a rectangular array or matris. Rows represent individual observational units, or just observations,

### The moment-lp and moment-sos approaches

The moment-lp and moment-sos approaches LAAS-CNRS and Institute of Mathematics, Toulouse, France CIRM, November 2013 Semidefinite Programming Why polynomial optimization? LP- and SDP- CERTIFICATES of POSITIVITY

### Lecture 6: Conic Optimization September 8

IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions

### EE 227A: Convex Optimization and Applications October 14, 2008

EE 227A: Convex Optimization and Applications October 14, 2008 Lecture 13: SDP Duality Lecturer: Laurent El Ghaoui Reading assignment: Chapter 5 of BV. 13.1 Direct approach 13.1.1 Primal problem Consider

### Determinant maximization with linear. S. Boyd, L. Vandenberghe, S.-P. Wu. Information Systems Laboratory. Stanford University

Determinant maximization with linear matrix inequality constraints S. Boyd, L. Vandenberghe, S.-P. Wu Information Systems Laboratory Stanford University SCCM Seminar 5 February 1996 1 MAXDET problem denition

### Lecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem

Lecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem Michael Patriksson 0-0 The Relaxation Theorem 1 Problem: find f := infimum f(x), x subject to x S, (1a) (1b) where f : R n R

### A SECOND ORDER MEHROTRA-TYPE PREDICTOR-CORRECTOR ALGORITHM FOR SEMIDEFINITE OPTIMIZATION

J Syst Sci Complex (01) 5: 1108 111 A SECOND ORDER MEHROTRA-TYPE PREDICTOR-CORRECTOR ALGORITHM FOR SEMIDEFINITE OPTIMIZATION Mingwang ZHANG DOI: 10.1007/s1144-01-0317-9 Received: 3 December 010 / Revised:

### Primal-Dual Interior-Point Methods. Ryan Tibshirani Convex Optimization /36-725

Primal-Dual Interior-Point Methods Ryan Tibshirani Convex Optimization 10-725/36-725 Given the problem Last time: barrier method min x subject to f(x) h i (x) 0, i = 1,... m Ax = b where f, h i, i = 1,...

### Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition

Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Prof. Tesler Math 283 Fall 2016 Also see the separate version of this with Matlab and R commands. Prof. Tesler Diagonalizing

### Optimality, Duality, Complementarity for Constrained Optimization

Optimality, Duality, Complementarity for Constrained Optimization Stephen Wright University of Wisconsin-Madison May 2014 Wright (UW-Madison) Optimality, Duality, Complementarity May 2014 1 / 41 Linear

### Linear Algebra Practice Problems

Linear Algebra Practice Problems Math 24 Calculus III Summer 25, Session II. Determine whether the given set is a vector space. If not, give at least one axiom that is not satisfied. Unless otherwise stated,

### Lecture Note 1: Background

ECE5463: Introduction to Robotics Lecture Note 1: Background Prof. Wei Zhang Department of Electrical and Computer Engineering Ohio State University Columbus, Ohio, USA Spring 2018 Lecture 1 (ECE5463 Sp18)

### Lecture: Convex Optimization Problems

1/36 Lecture: Convex Optimization Problems http://bicmr.pku.edu.cn/~wenzw/opt-2015-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/36 optimization

### A class of Smoothing Method for Linear Second-Order Cone Programming

Columbia International Publishing Journal of Advanced Computing (13) 1: 9-4 doi:1776/jac1313 Research Article A class of Smoothing Method for Linear Second-Order Cone Programming Zhuqing Gui *, Zhibin

### Conic Linear Programming. Yinyu Ye

Conic Linear Programming Yinyu Ye December 2004, revised October 2017 i ii Preface This monograph is developed for MS&E 314, Conic Linear Programming, which I am teaching at Stanford. Information, lecture

### ELE539A: Optimization of Communication Systems Lecture 6: Quadratic Programming, Geometric Programming, and Applications

ELE539A: Optimization of Communication Systems Lecture 6: Quadratic Programming, Geometric Programming, and Applications Professor M. Chiang Electrical Engineering Department, Princeton University February

### AN EQUIVALENCY CONDITION OF NONSINGULARITY IN NONLINEAR SEMIDEFINITE PROGRAMMING

J Syst Sci Complex (2010) 23: 822 829 AN EQUVALENCY CONDTON OF NONSNGULARTY N NONLNEAR SEMDEFNTE PROGRAMMNG Chengjin L Wenyu SUN Raimundo J. B. de SAMPAO DO: 10.1007/s11424-010-8057-1 Received: 2 February

### Fast ADMM for Sum of Squares Programs Using Partial Orthogonality

Fast ADMM for Sum of Squares Programs Using Partial Orthogonality Antonis Papachristodoulou Department of Engineering Science University of Oxford www.eng.ox.ac.uk/control/sysos antonis@eng.ox.ac.uk with

### Numerical Linear Algebra

Numerical Linear Algebra Direct Methods Philippe B. Laval KSU Fall 2017 Philippe B. Laval (KSU) Linear Systems: Direct Solution Methods Fall 2017 1 / 14 Introduction The solution of linear systems is one

### Some inequalities involving determinants, eigenvalues, and Schur complements in Euclidean Jordan algebras

positivity manuscript No. (will be inserted by the editor) Some inequalities involving determinants, eigenvalues, and Schur complements in Euclidean Jordan algebras M. Seetharama Gowda Jiyuan Tao February

### Linear Algebra. Workbook

Linear Algebra Workbook Paul Yiu Department of Mathematics Florida Atlantic University Last Update: November 21 Student: Fall 2011 Checklist Name: A B C D E F F G H I J 1 2 3 4 5 6 7 8 9 10 xxx xxx xxx

### The Jordan Algebraic Structure of the Circular Cone

The Jordan Algebraic Structure of the Circular Cone Baha Alzalg Department of Mathematics, The University of Jordan, Amman 1194, Jordan Abstract In this paper, we study and analyze the algebraic structure

### A Distributed Newton Method for Network Utility Maximization, I: Algorithm

A Distributed Newton Method for Networ Utility Maximization, I: Algorithm Ermin Wei, Asuman Ozdaglar, and Ali Jadbabaie October 31, 2012 Abstract Most existing wors use dual decomposition and first-order

### A Distributed Newton Method for Network Utility Maximization

A Distributed Newton Method for Networ Utility Maximization Ermin Wei, Asuman Ozdaglar, and Ali Jadbabaie Abstract Most existing wor uses dual decomposition and subgradient methods to solve Networ Utility

### ALGEBRA QUALIFYING EXAM PROBLEMS LINEAR ALGEBRA

ALGEBRA QUALIFYING EXAM PROBLEMS LINEAR ALGEBRA Kent State University Department of Mathematical Sciences Compiled and Maintained by Donald L. White Version: August 29, 2017 CONTENTS LINEAR ALGEBRA AND

### Fundamentals of Matrices

Maschinelles Lernen II Fundamentals of Matrices Christoph Sawade/Niels Landwehr/Blaine Nelson Tobias Scheffer Matrix Examples Recap: Data Linear Model: f i x = w i T x Let X = x x n be the data matrix

### Advances in Convex Optimization: Theory, Algorithms, and Applications

Advances in Convex Optimization: Theory, Algorithms, and Applications Stephen Boyd Electrical Engineering Department Stanford University (joint work with Lieven Vandenberghe, UCLA) ISIT 02 ISIT 02 Lausanne

### Solution Methods. Richard Lusby. Department of Management Engineering Technical University of Denmark

Solution Methods Richard Lusby Department of Management Engineering Technical University of Denmark Lecture Overview (jg Unconstrained Several Variables Quadratic Programming Separable Programming SUMT

### Introduction to optimization

Introduction to optimization Geir Dahl CMA, Dept. of Mathematics and Dept. of Informatics University of Oslo 1 / 24 The plan 1. The basic concepts 2. Some useful tools (linear programming = linear optimization)

### Lecture #21. c T x Ax b. maximize subject to

COMPSCI 330: Design and Analysis of Algorithms 11/11/2014 Lecture #21 Lecturer: Debmalya Panigrahi Scribe: Samuel Haney 1 Overview In this lecture, we discuss linear programming. We first show that the

### The Eigenvalue Problem: Perturbation Theory

Jim Lambers MAT 610 Summer Session 2009-10 Lecture 13 Notes These notes correspond to Sections 7.2 and 8.1 in the text. The Eigenvalue Problem: Perturbation Theory The Unsymmetric Eigenvalue Problem Just

### Conjugate Gradient (CG) Method

Conjugate Gradient (CG) Method by K. Ozawa 1 Introduction In the series of this lecture, I will introduce the conjugate gradient method, which solves efficiently large scale sparse linear simultaneous

### Karush-Kuhn-Tucker Conditions. Lecturer: Ryan Tibshirani Convex Optimization /36-725

Karush-Kuhn-Tucker Conditions Lecturer: Ryan Tibshirani Convex Optimization 10-725/36-725 1 Given a minimization problem Last time: duality min x subject to f(x) h i (x) 0, i = 1,... m l j (x) = 0, j =

### Newton s Method. Ryan Tibshirani Convex Optimization /36-725

Newton s Method Ryan Tibshirani Convex Optimization 10-725/36-725 1 Last time: dual correspondences Given a function f : R n R, we define its conjugate f : R n R, Properties and examples: f (y) = max x

### Improving Performance of The Interior Point Method by Preconditioning

Improving Performance of The Interior Point Method by Preconditioning Mid-Point Status Report Project by: Ken Ryals For: AMSC 663-664 Fall 27-Spring 28 6 December 27 Background / Refresher The IPM method

### Matrix Algebra, part 2

Matrix Algebra, part 2 Ming-Ching Luoh 2005.9.12 1 / 38 Diagonalization and Spectral Decomposition of a Matrix Optimization 2 / 38 Diagonalization and Spectral Decomposition of a Matrix Also called Eigenvalues

### A study of search directions in primal-dual interior-point methods for semidefinite programming

A study of search directions in primal-dual interior-point methods for semidefinite programming M. J. Todd February 23, 1999 School of Operations Research and Industrial Engineering, Cornell University,

### FINITE-DIMENSIONAL LINEAR ALGEBRA

DISCRETE MATHEMATICS AND ITS APPLICATIONS Series Editor KENNETH H ROSEN FINITE-DIMENSIONAL LINEAR ALGEBRA Mark S Gockenbach Michigan Technological University Houghton, USA CRC Press Taylor & Francis Croup

### Introduction to Numerical Analysis

Université de Liège Faculté des Sciences Appliquées Introduction to Numerical Analysis Edition 2015 Professor Q. Louveaux Department of Electrical Engineering and Computer Science Montefiore Institute

### FIXED POINT ITERATIONS

FIXED POINT ITERATIONS MARKUS GRASMAIR 1. Fixed Point Iteration for Non-linear Equations Our goal is the solution of an equation (1) F (x) = 0, where F : R n R n is a continuous vector valued mapping in

### An Introduction to Linear Matrix Inequalities. Raktim Bhattacharya Aerospace Engineering, Texas A&M University

An Introduction to Linear Matrix Inequalities Raktim Bhattacharya Aerospace Engineering, Texas A&M University Linear Matrix Inequalities What are they? Inequalities involving matrix variables Matrix variables

### Spectral radius, symmetric and positive matrices

Spectral radius, symmetric and positive matrices Zdeněk Dvořák April 28, 2016 1 Spectral radius Definition 1. The spectral radius of a square matrix A is ρ(a) = max{ λ : λ is an eigenvalue of A}. For an

### c 1995 Society for Industrial and Applied Mathematics Vol. 37, No. 1, pp , March

SIAM REVIEW. c 1995 Society for Industrial and Applied Mathematics Vol. 37, No. 1, pp. 93 97, March 1995 008 A UNIFIED PROOF FOR THE CONVERGENCE OF JACOBI AND GAUSS-SEIDEL METHODS * ROBERTO BAGNARA Abstract.

### Primal-dual relationship between Levenberg-Marquardt and central trajectories for linearly constrained convex optimization

Primal-dual relationship between Levenberg-Marquardt and central trajectories for linearly constrained convex optimization Roger Behling a, Clovis Gonzaga b and Gabriel Haeser c March 21, 2013 a Department

### Gaussian Elimination without/with Pivoting and Cholesky Decomposition

Gaussian Elimination without/with Pivoting and Cholesky Decomposition Gaussian Elimination WITHOUT pivoting Notation: For a matrix A R n n we define for k {,,n} the leading principal submatrix a a k A

### POWER flow studies are the cornerstone of power system

University of Wisconsin-Madison Department of Electrical and Computer Engineering. Technical Report ECE-2-. A Sufficient Condition for Power Flow Insolvability with Applications to Voltage Stability Margins

### Linear Equations and Matrix

1/60 Chia-Ping Chen Professor Department of Computer Science and Engineering National Sun Yat-sen University Linear Algebra Gaussian Elimination 2/60 Alpha Go Linear algebra begins with a system of linear

### Using interior point methods for optimization in training very large scale Support Vector Machines. Interior Point Methods for QP. Elements of the IPM

School of Mathematics T H E U N I V E R S I T Y O H Using interior point methods for optimization in training very large scale Support Vector Machines F E D I N U R Part : Interior Point Methods for QP

### Inexact primal-dual path-following algorithms for a special class of convex quadratic SDP and related problems

Inexact primal-dual path-following algorithms for a special class of convex quadratic SDP and related problems K.C. Toh, R. H. Tütüncü, and M. J. Todd February 8, 005 Dedicated to the memory of Jos Sturm:

### Chap 2. Optimality conditions

Chap 2. Optimality conditions Version: 29-09-2012 2.1 Optimality conditions in unconstrained optimization Recall the definitions of global, local minimizer. Geometry of minimization Consider for f C 1

### Second Order Cone Programming Relaxation of Nonconvex Quadratic Optimization Problems

Second Order Cone Programming Relaxation of Nonconvex Quadratic Optimization Problems Sunyoung Kim Department of Mathematics, Ewha Women s University 11-1 Dahyun-dong, Sudaemoon-gu, Seoul 120-750 Korea

### Projection methods in conic optimization

Projection methods in conic optimization Didier Henrion 1,2 Jérôme Malick 3 Abstract There exist efficient algorithms to project a point onto the intersection of a convex cone and an affine subspace. Those

### Computing regularization paths for learning multiple kernels

Computing regularization paths for learning multiple kernels Francis Bach Romain Thibaux Michael Jordan Computer Science, UC Berkeley December, 24 Code available at www.cs.berkeley.edu/~fbach Computing

### 1 Positive definiteness and semidefiniteness

Positive definiteness and semidefiniteness Zdeněk Dvořák May 9, 205 For integers a, b, and c, let D(a, b, c) be the diagonal matrix with + for i =,..., a, D i,i = for i = a +,..., a + b,. 0 for i = a +

### Strict diagonal dominance and a Geršgorin type theorem in Euclidean

Strict diagonal dominance and a Geršgorin type theorem in Euclidean Jordan algebras Melania Moldovan Department of Mathematics and Statistics University of Maryland, Baltimore County Baltimore, Maryland

### Numerical Methods I Solving Square Linear Systems: GEM and LU factorization

Numerical Methods I Solving Square Linear Systems: GEM and LU factorization Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 September 18th,

### What can be expressed via Conic Quadratic and Semidefinite Programming?

What can be expressed via Conic Quadratic and Semidefinite Programming? A. Nemirovski Faculty of Industrial Engineering and Management Technion Israel Institute of Technology Abstract Tremendous recent

### SENSITIVITY ANALYSIS IN CONVEX QUADRATIC OPTIMIZATION: SIMULTANEOUS PERTURBATION OF THE OBJECTIVE AND RIGHT-HAND-SIDE VECTORS

SENSITIVITY ANALYSIS IN CONVEX QUADRATIC OPTIMIZATION: SIMULTANEOUS PERTURBATION OF THE OBJECTIVE AND RIGHT-HAND-SIDE VECTORS ALIREZA GHAFFARI HADIGHEH Department of Mathematics, Azarbaijan University

### Solving Dual Problems

Lecture 20 Solving Dual Problems We consider a constrained problem where, in addition to the constraint set X, there are also inequality and linear equality constraints. Specifically the minimization problem

### Part 5: Penalty and augmented Lagrangian methods for equality constrained optimization. Nick Gould (RAL)

Part 5: Penalty and augmented Lagrangian methods for equality constrained optimization Nick Gould (RAL) x IR n f(x) subject to c(x) = Part C course on continuoue optimization CONSTRAINED MINIMIZATION x

### Convexity of the Reachable Set of Nonlinear Systems under L 2 Bounded Controls

1 1 Convexity of the Reachable Set of Nonlinear Systems under L 2 Bounded Controls B.T.Polyak Institute for Control Science, Moscow, Russia e-mail boris@ipu.rssi.ru Abstract Recently [1, 2] the new convexity