L. Vandenberghe EE236C (Spring 2016) 18. Symmetric cones. definition. spectral decomposition. quadratic representation. log-det barrier 18-1
|
|
- Allan Strickland
- 5 years ago
- Views:
Transcription
1 L. Vandenberghe EE236C (Spring 2016) 18. Symmetric cones definition spectral decomposition quadratic representation log-det barrier 18-1
2 Introduction This lecture: theoretical properties of the following cones nonnegative orthant R p + = {x R p x k 0, k = 1,..., p} second-order cone Q p = {(x 0, x 1 ) R R p 1 x 1 2 x 0 } positive semidefinite cone S p = {x R p(p+1)/2 mat(x) 0} these cones are not only self-dual, but symmetric (also known as self-scaled) Symmetric cones 18-2
3 Outline definition spectral decomposition quadratic representation log-det barrier
4 Cones of squares the three basic cones can be expressed as cones of squares x 2 = x x for appropriately defined vector products x y nonnegative orthant: componentwise product x y = diag(x)y second-order cone: the product of x = (x 0, x 1 ) and y = (y 0, y 1 ) is x y = 1 2 [ x T y x 0 y 1 + y 0 x 1 ] positive semidefinite cone: symmetrized matrix product x y = 1 2 vec(xy + Y X) with X = mat(x), Y = mat(y ) Symmetric cones 18-3
5 Symmetric cones the vector products satisfy the following properties 1. x y is bilinear (linear in x for fixed y and vice-versa) 2. x y = y x 3. x 2 (y x) = (x 2 y) x 4. x T (y z) = (x y) T z except for the componentwise product, the products are not associative: x (y z) (x y) z in general Definition: a cone is symmetric if it is the cone of squares {x 2 = x x x R n } for a product x y that satisfies these four properties Symmetric cones 18-4
6 Classification symmetric cones are studied in the theory of Euclidean Jordan algebras all possible symmetric cones have been characterized List of symmetric cones the second-order cone the p.s.d. cone of Hermitian matrices with real, complex, or quaternion entries 3 3 positive semidefinite matrices with octonion entries Cartesian products of these primitive symmetric cones (such as R p + ) Practical implication can focus on Q p, S p and study these cones using elementary linear algebra Symmetric cones 18-5
7 Outline definition spectral decomposition quadratic representation log-det barrier
8 Vector product with each symmetric cone K we associate a bilinear vector product for R p +, Qp, S p we use the products on page 18-3 for a cone K = K 1 K N, with K i of one of the three basic types, (x 1,..., x N ) (y 1,..., y N ) = (x 1 y 1,..., x N y N ) we refer to the product associated with the cone K as the product for K Symmetric cones 18-6
9 Identity element Identity element: the element e that satisfies e x = x e = x for all x product for R p + : e = 1 = (1, 1,..., 1) product for Q p : e = ( 2, 0,..., 0) product for S p : e = vec(i) product for K 1 K N : the product of the N identity elements note we use the same symbol e for the identity element for each product Rank of the cone: θ = e T e is called the rank of K θ = p (K = R p +), θ = 2 (K = Q p ), θ = p (K = S p ) and θ = N θ i if K = K 1 K N and θ i is the rank of K i Symmetric cones 18-7
10 Spectral decomposition with each symmetric cone/product we associate a spectral decomposition x = θ λ i q i λ i are the eigenvalues of x; the eigenvectors q i satisfy q 2 i = q i, q i q j = 0 (i j), θ q i = e theory can be developed from properties of the vector product on page 18-4 we will define the decomposition by enumerating the symmetric cones Symmetric cones 18-8
11 Spectral decomposition for primitive cones Positive semidefinite cone (K = S p ) spectral decomposition of x follows from eigendecomposition of mat(x): mat(x) = p λ i v i vi T, q i = vec(v i vi T ) Second-order cone (K = Q p ) spectral decomposition of x = (x 0, x 1 ) R R p 1 is λ i = x 0 ± x 1 2 2, q i = 1 2 [ 1 ±y ], i = 1, 2 y = x 1 / x 1 2 if x 1 0, and y is an arbitrary unit-norm vector otherwise Symmetric cones 18-9
12 Spectral decomposition for composite cones Product cone (K = K 1 K N ) spectral decomposition follows from decomposition of different blocks example (K = K 1 K 2 ): decomposition of x = (x 1, x 2 ) is where x j = θ j [ x1 x 2 ] = θ 1 λ 1i [ q1i 0 ] + θ 2 λ 2i [ 0 q 2i ] λ ji q ji is the spectral decomposition of x j, j = 1, 2 Nonnegative orthant (K = R p + ) λ i = x i, q i = e i (ith unit vector), i = 1,..., n Symmetric cones 18-10
13 Some properties the eigenvectors are normalized ( q i 2 = 1) and nonnegative (q i K) eigenvectors are orthogonal: q T i q j = 0 for i j can be verified from the definitions, or from the properties on page 18-4 q T i q j = q T i (q j q j ) = (q i q j ) T q j = 0 e T q i = ( j q j) T q i = q T i q i = 1 e T x = θ λ i x K if and only λ i 0 for i = 1,..., θ x int K if and only λ i > 0 for i = 1,..., θ Symmetric cones 18-11
14 Trace, determinant, and norm tr x = θ λ i, det x = θ θ λ i, x F = ( λ 2 i ) 1/2 positive semidefinite cone (K = S p ) tr x = tr(mat(x)), det x = det(mat(x)), x F = mat(x) F second-order cone (K = Q p ) tr x = 2x 0, det x = 1 2 (x2 0 x T 1 x 1 ), x F = x 2 nonnegative orthant (K = R p + ) tr x = p x i, det x = p x i, x F = x 2 Symmetric cones 18-12
15 Powers and inverse powers of x can be defined in terms of the spectral decomposition x α = i λ α i q i (exists if λ α i is defined for all i) x α x β = x α+β x α x β = ( θ ) λ α i q i ( θ ) λ β i q i = θ λ α+β i q i = x α+β x is invertible if all λ i 0 (i.e., det x 0) inverse x 1 = θ λ 1 i q i satisfies x x 1 = x 1 x = e Symmetric cones 18-13
16 Expressions for inverse for invertible x (i.e., λ i 0 for i = 1,..., θ) nonnegative orthant (K = R p + ) x 1 = ( 1 x 1, 1 x 2,..., 1 x p ) second-order cone (K = Q p ) x 1 = 2 x T Jx Jx, J = [ I ] semidefinite cone (K = S p ) x 1 = vec ( mat(x) 1) Symmetric cones 18-14
17 Expressions for square root for nonnegative x (i.e., λ i 0 for i = 1,..., θ) nonnegative orthant (K = R p + ) x 1/2 = ( x1, x 2,..., x p ) second-order cone (K = Q p ) [ x 1/2 1 x0 = 2 (x 1/ x T Jx ) 1/2 x T x 1 Jx ] semidefinite cone (K = S p ) x 1/2 = vec (mat(x) 1/2) Symmetric cones 18-15
18 Outline definition spectral decomposition quadratic representation log-det barrier
19 Matrix representation of product since the product is bilinear, it can be expressed as x y = L(x)y L(x) is a symmetric matrix, linearly dependent on x nonnegative orthant: L(x) = diag(x) second-order cone L(x) = 1 [ x0 x T 1 2 x 0 I x 1 ] semidefinite cone: the matrix defined by L(x)y = 1 (XY + Y X), X = mat(x), Y = mat(y) 2 Symmetric cones 18-16
20 Quadratic representation the quadratic representation of x is the matrix P (x) = 2L(x) 2 L(x 2 ) (terminology is motivated by the property P (x)e = x 2 ) nonnegative orthant P (x) = diag(x) 2 second-order cone P (x) = xx T xt Jx J, J = 2 [ I ] positive semidefinite cone P (x)y = vec(xy X) where X = mat(x), Y = mat(y) Symmetric cones 18-17
21 Some useful properties Powers P (x) α = P (x α ) if x α exists P (x)x 1 = x if x is invertible Derivative of x 1 : for x invertible, P (x) 1 is the derivative of x 1, i.e., d (x + αy) 1 dα = P (x) 1 y α=0 Symmetric cones 18-18
22 Scaling with P (x) affine transformations y P (x)y have important properties if x is invertible, then P (x)k = K, P (x) int K = int K multiplication with P (x) preserves the conic inequalities if x and y are invertible, then P (x)y is invertible with inverse (P (x)y) 1 = P (x 1 )y 1 = P (x) 1 y 1 quadratic representation of P (x)y P (P (x)y) = P (x)p (y)p (x) hence also det (P (x)y) = (det x) 2 det y Symmetric cones 18-19
23 Distance to cone boundary for x 0 define σ x (y) = 0 if y 0 and ) σ x (y) = λ min (P (x 1/2 )y otherwise σ x (y) characterizes distance of x to the boundary of K in the direction y: x + αy 0 ασ x (y) 1 Proof: from the definition of P on page 18-17: P (x 1/2 )e = x therefore, with v = P (x 1/2 )y x + αy 0 P (x 1/2 )(e + αv) 0 e + αv αλ i (v) 0 Symmetric cones 18-20
24 Scaling point for a pair s, z 0, the point w = P (z 1/2 ) ( ) 1/2 P (z 1/2 )s satisfies w 0 and s = P (w)z the linear transformation P (w) preserves the cone and maps z to s equivalently, v = w 1/2 defines a scaling P (v) = P (w) 1/2 that satisfies P (v) 1 s = P (v)z Symmetric cones 18-21
25 Proof: we use the properties P (x)e = x 2, P (P (x)y) = P (x)p (y)p (x) if we define u = P (z 1/2 )s, we can write P (w) as P (w) = P (z 1/2 )P (u 1/2 )P (z 1/2 ) therefore P (w)z = P (z 1/2 )P (u 1/2 )e = P (z 1/2 )u = s Symmetric cones 18-22
26 Scaling point for nonnegative orthant Scaling point w = ( s1 /z 1, s 2 /z 2,..., s p /z p ) Scaling transformation: a positive diagonal scaling P (w) = s 1 /z s 2 /z s p /z p Symmetric cones 18-23
27 Scaling point for positive semidefinite cone Scaling point: w = vec(rr T ) R simultaneously diagonalizes mat(z) and mat(s) 1 : R T mat(z)r = R 1 mat(s)r T = Σ can be computed from two Cholesky factorizations and an SVD: if mat(s) = L 1 L T 1, mat(z) = L 2 L T 2, L T 2 L 1 = UΣV T then R = L 1 V Σ 1/2 = L 2 UΣ 1/2 Scaling transformation: a congruence transformation P (w)y = RR T mat(y)rr T Symmetric cones 18-24
28 Outline definition spectral decomposition quadratic representation log-det barrier
29 Log-det barrier Definition: the log-det barrier of a symmetric cone K is φ(x) = log det x = θ log λ i, dom φ = int K φ is logarithmically homogeneous with degree θ (i.e., the rank of K) nonnegative orthant (K = R p + ): φ(x) = p log x i second-order cone (K = Q p ): φ(x) = log(x 2 0 x T 1 x 1 ) + log 2 semidefinite cone (K = S p ): φ(x) = log det(mat x) composition K = K 1 K N : sum of the log-det barriers Symmetric cones 18-25
30 Convexity φ is a convex function Proof: consider arbitrary y and let λ i be the eigenvalues of v = P (x 1/2 )y the restriction g(α) = φ(x + αy) of φ to the line x + αy is g(α) = log det(x + αy) ( ) = log det P (x 1/2 )(e + αv) = log det x log det(e + αv) θ = log det x log(1 + αλ i ) (line 3 follows from page and page 18-21) hence restriction of φ to arbitrary line is convex Symmetric cones 18-26
31 Gradient and Hessian the gradient and Hessian of φ at a point x 0 are φ(x) = x 1, 2 φ(x) = P (x) 1 = P (x 1 ) Proof: continues from last page φ(x) T y = g (0) = θ λ i = tr(p (x 1/2 )y) = e T P (x 1/2 )y since this holds for all y, φ(x) = P (x 1/2 )e = x 1 the expression for the Hessian follows from page Symmetric cones 18-27
32 Dikin ellipsoid theorem for symmetric cones recall the definition of the Dikin ellipsoid at x 0: E x = {x + y y T 2 φ(x)y 1} = {x + y y T P (x) 1 y 1} x + y E x if and only if the eigenvalues λ i of P (x 1/2 )y satisfy θ λ 2 i 1 for symmetric cones the Dikin ellipsoid theorem E x K follows from θ λ 2 i 1 = min i λ i 1 therefore x + y E x implies σ x (y) 1 and x + y 0 Symmetric cones 18-28
33 Generalized convexity of inverse for all w 0, the function f(x) = w T x 1, dom f = int K is convex Proof: restriction g(α) = f(x + αy) of f to a line is ( ) 1 g(α) = w T (x + αy) 1 = w T P (x 1/2 )(e + αv) = w T P (x 1/2 )(e + αv) 1 = θ i 1 w T P (x 1/2 )q i 1 + αλ i λ i, q i are eigenvalues and eigenvectors of v = P (x 1/2 )y g(α) is convex because P (x 1/2 )K = K; therefore w T P (x 1/2 )q i 0 Symmetric cones 18-29
34 Self-concordance for all x 0 and all y, d dα 2 φ(x + αy) 2σ x (y) 2 φ(x) α=0 from page 18-20, with v = P (x 1/2 )y and λ i = λ i (v) θ σ x (y) = min{0, λ min } ( λ 2 i ) 1/2 = y x hence, inequality implies self-concordance inequality on page 16-2 this shows that the log-det barrier φ is a θ-normal barrier Symmetric cones 18-30
35 Proof: choose any σ > σ x (y); by definition of σ x we have σx + y 0 d dα 2 φ(x + αy) = d dα 2 φ(x ασx) + d dα 2 φ(x + α(σx + y)) from page 16-24, the first term on the r.h.s. (evaluated at α = 0) is σ d dt 2 φ(tx) = 2σ 2 φ(x) t=1 2nd term is 2 g(x) where g(u) = w T φ(u) and w = σx + y: g(u) = d φ(u + αw) dα, α=0 2 g(u) = d dα 2 φ(u + αw) α=0 2 g(u) 0 because from page 18-29, g(u) = w T u 1 is concave Symmetric cones 18-31
36 References J. Faraut and A. Korányi, Analysis on Symmetric Cones (1994). Yu. E. Nesterov and M. J. Todd, Self-scaled barriers and interior-point methods for convex programming, Mathematics of Operations Research (1997). L. Faybusovich, Euclidean Jordan algebras and interior-point algorithms, Positivity (1997). J. F. Sturm, Similarity and other spectral relations for symmetric cones, Linear Algebra and Its Applications (2000). S. H. Schmieta and F. Alizadeh, Extension of primal-dual interior point algorithms to symmetric cones, Math. Prog. (2003). F. Alizadeh and D. Goldfarb, Second-order cone programming, Math. Prog. (2003). Symmetric cones 18-32
The Jordan Algebraic Structure of the Circular Cone
The Jordan Algebraic Structure of the Circular Cone Baha Alzalg Department of Mathematics, The University of Jordan, Amman 1194, Jordan Abstract In this paper, we study and analyze the algebraic structure
More information15. Conic optimization
L. Vandenberghe EE236C (Spring 216) 15. Conic optimization conic linear program examples modeling duality 15-1 Generalized (conic) inequalities Conic inequality: a constraint x K where K is a convex cone
More informationLargest dual ellipsoids inscribed in dual cones
Largest dual ellipsoids inscribed in dual cones M. J. Todd June 23, 2005 Abstract Suppose x and s lie in the interiors of a cone K and its dual K respectively. We seek dual ellipsoidal norms such that
More informationA PREDICTOR-CORRECTOR PATH-FOLLOWING ALGORITHM FOR SYMMETRIC OPTIMIZATION BASED ON DARVAY'S TECHNIQUE
Yugoslav Journal of Operations Research 24 (2014) Number 1, 35-51 DOI: 10.2298/YJOR120904016K A PREDICTOR-CORRECTOR PATH-FOLLOWING ALGORITHM FOR SYMMETRIC OPTIMIZATION BASED ON DARVAY'S TECHNIQUE BEHROUZ
More information2.1. Jordan algebras. In this subsection, we introduce Jordan algebras as well as some of their basic properties.
FULL NESTEROV-TODD STEP INTERIOR-POINT METHODS FOR SYMMETRIC OPTIMIZATION G. GU, M. ZANGIABADI, AND C. ROOS Abstract. Some Jordan algebras were proved more than a decade ago to be an indispensable tool
More informationWe describe the generalization of Hazan s algorithm for symmetric programming
ON HAZAN S ALGORITHM FOR SYMMETRIC PROGRAMMING PROBLEMS L. FAYBUSOVICH Abstract. problems We describe the generalization of Hazan s algorithm for symmetric programming Key words. Symmetric programming,
More information18. Primal-dual interior-point methods
L. Vandenberghe EE236C (Spring 213-14) 18. Primal-dual interior-point methods primal-dual central path equations infeasible primal-dual method primal-dual method for self-dual embedding 18-1 Symmetric
More informationStrict diagonal dominance and a Geršgorin type theorem in Euclidean
Strict diagonal dominance and a Geršgorin type theorem in Euclidean Jordan algebras Melania Moldovan Department of Mathematics and Statistics University of Maryland, Baltimore County Baltimore, Maryland
More informationThe Q Method for Symmetric Cone Programming
The Q Method for Symmetric Cone Programming Farid Alizadeh Yu Xia October 5, 010 Communicated by F. Potra Abstract The Q method of semidefinite programming, developed by Alizadeh, Haeberly and Overton,
More informationA Primal-Dual Second-Order Cone Approximations Algorithm For Symmetric Cone Programming
A Primal-Dual Second-Order Cone Approximations Algorithm For Symmetric Cone Programming Chek Beng Chua Abstract This paper presents the new concept of second-order cone approximations for convex conic
More informationPrimal-dual path-following algorithms for circular programming
Primal-dual path-following algorithms for circular programming Baha Alzalg Department of Mathematics, The University of Jordan, Amman 1194, Jordan July, 015 Abstract Circular programming problems are a
More informationOPTIMIZATION OVER SYMMETRIC CONES UNDER UNCERTAINTY
OPTIMIZATION OVER SYMMETRIC CONES UNDER UNCERTAINTY By BAHA M. ALZALG A dissertation submitted in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY WASHINGTON STATE UNIVERSITY
More informationSupplement: Universal Self-Concordant Barrier Functions
IE 8534 1 Supplement: Universal Self-Concordant Barrier Functions IE 8534 2 Recall that a self-concordant barrier function for K is a barrier function satisfying 3 F (x)[h, h, h] 2( 2 F (x)[h, h]) 3/2,
More informationEE 546, Univ of Washington, Spring Proximal mapping. introduction. review of conjugate functions. proximal mapping. Proximal mapping 6 1
EE 546, Univ of Washington, Spring 2012 6. Proximal mapping introduction review of conjugate functions proximal mapping Proximal mapping 6 1 Proximal mapping the proximal mapping (prox-operator) of a convex
More informationLMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009
LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix
More informationELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications
ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 17
EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 17 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory May 29, 2012 Andre Tkacenko
More informationLecture 2: Linear Algebra Review
EE 227A: Convex Optimization and Applications January 19 Lecture 2: Linear Algebra Review Lecturer: Mert Pilanci Reading assignment: Appendix C of BV. Sections 2-6 of the web textbook 1 2.1 Vectors 2.1.1
More informationLecture Note 5: Semidefinite Programming for Stability Analysis
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State
More informationforms Christopher Engström November 14, 2014 MAA704: Matrix factorization and canonical forms Matrix properties Matrix factorization Canonical forms
Christopher Engström November 14, 2014 Hermitian LU QR echelon Contents of todays lecture Some interesting / useful / important of matrices Hermitian LU QR echelon Rewriting a as a product of several matrices.
More informationConvex Functions. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Convex Functions Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Definition convex function Examples
More informationAgenda. Interior Point Methods. 1 Barrier functions. 2 Analytic center. 3 Central path. 4 Barrier method. 5 Primal-dual path following algorithms
Agenda Interior Point Methods 1 Barrier functions 2 Analytic center 3 Central path 4 Barrier method 5 Primal-dual path following algorithms 6 Nesterov Todd scaling 7 Complexity analysis Interior point
More informationThe eigenvalue map and spectral sets
The eigenvalue map and spectral sets p. 1/30 The eigenvalue map and spectral sets M. Seetharama Gowda Department of Mathematics and Statistics University of Maryland, Baltimore County Baltimore, Maryland
More informationPRIMAL-DUAL INTERIOR-POINT METHODS FOR SELF-SCALED CONES
PRIMAL-DUAL INTERIOR-POINT METHODS FOR SELF-SCALED CONES YU. E. NESTEROV AND M. J. TODD Abstract. In this paper we continue the development of a theoretical foundation for efficient primal-dual interior-point
More informationUsing Schur Complement Theorem to prove convexity of some SOC-functions
Journal of Nonlinear and Convex Analysis, vol. 13, no. 3, pp. 41-431, 01 Using Schur Complement Theorem to prove convexity of some SOC-functions Jein-Shan Chen 1 Department of Mathematics National Taiwan
More informationKey words. Complementarity set, Lyapunov rank, Bishop-Phelps cone, Irreducible cone
ON THE IRREDUCIBILITY LYAPUNOV RANK AND AUTOMORPHISMS OF SPECIAL BISHOP-PHELPS CONES M. SEETHARAMA GOWDA AND D. TROTT Abstract. Motivated by optimization considerations we consider cones in R n to be called
More information10-725/36-725: Convex Optimization Prerequisite Topics
10-725/36-725: Convex Optimization Prerequisite Topics February 3, 2015 This is meant to be a brief, informal refresher of some topics that will form building blocks in this course. The content of the
More information1. Gradient method. gradient method, first-order methods. quadratic bounds on convex functions. analysis of gradient method
L. Vandenberghe EE236C (Spring 2016) 1. Gradient method gradient method, first-order methods quadratic bounds on convex functions analysis of gradient method 1-1 Approximate course outline First-order
More informationOn the game-theoretic value of a linear transformation on a symmetric cone
On the game-theoretic value of a linear transformation ona symmetric cone p. 1/25 On the game-theoretic value of a linear transformation on a symmetric cone M. Seetharama Gowda Department of Mathematics
More informationCOURSE ON LMI PART I.2 GEOMETRY OF LMI SETS. Didier HENRION henrion
COURSE ON LMI PART I.2 GEOMETRY OF LMI SETS Didier HENRION www.laas.fr/ henrion October 2006 Geometry of LMI sets Given symmetric matrices F i we want to characterize the shape in R n of the LMI set F
More informationInterior Point Methods: Second-Order Cone Programming and Semidefinite Programming
School of Mathematics T H E U N I V E R S I T Y O H F E D I N B U R G Interior Point Methods: Second-Order Cone Programming and Semidefinite Programming Jacek Gondzio Email: J.Gondzio@ed.ac.uk URL: http://www.maths.ed.ac.uk/~gondzio
More informationLecture: Introduction to LP, SDP and SOCP
Lecture: Introduction to LP, SDP and SOCP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2015.html wenzw@pku.edu.cn Acknowledgement:
More informationThe proximal mapping
The proximal mapping http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes Outline 2/37 1 closed function 2 Conjugate function
More informationLagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual
More informationControl System Analysis on Symmetric Cones
Submitted, 015 Conference on Decision and Control (CDC) http://www.cds.caltech.edu/~murray/papers/pm15-cdc.html Control System Analysis on Symmetric Cones Ivan Papusha Richard M. Murray Abstract Motivated
More informationSemidefinite Programming Basics and Applications
Semidefinite Programming Basics and Applications Ray Pörn, principal lecturer Åbo Akademi University Novia University of Applied Sciences Content What is semidefinite programming (SDP)? How to represent
More informationSome characterizations of cone preserving Z-transformations
Annals of Operations Research manuscript No. (will be inserted by the editor) Some characterizations of cone preserving Z-transformations M. Seetharama Gowda Yoon J. Song K.C. Sivakumar This paper is dedicated,
More informationSome inequalities involving determinants, eigenvalues, and Schur complements in Euclidean Jordan algebras
positivity manuscript No. (will be inserted by the editor) Some inequalities involving determinants, eigenvalues, and Schur complements in Euclidean Jordan algebras M. Seetharama Gowda Jiyuan Tao February
More informationA function(al) f is convex if dom f is a convex set, and. f(θx + (1 θ)y) < θf(x) + (1 θ)f(y) f(x) = x 3
Convex functions The domain dom f of a functional f : R N R is the subset of R N where f is well-defined. A function(al) f is convex if dom f is a convex set, and f(θx + (1 θ)y) θf(x) + (1 θ)f(y) for all
More informationThe Singular Value Decomposition
The Singular Value Decomposition Philippe B. Laval KSU Fall 2015 Philippe B. Laval (KSU) SVD Fall 2015 1 / 13 Review of Key Concepts We review some key definitions and results about matrices that will
More informationSymmetric Matrices and Eigendecomposition
Symmetric Matrices and Eigendecomposition Robert M. Freund January, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 Symmetric Matrices and Convexity of Quadratic Functions
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationIntroduction and Math Preliminaries
Introduction and Math Preliminaries Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Appendices A, B, and C, Chapter
More informationIII. Applications in convex optimization
III. Applications in convex optimization nonsymmetric interior-point methods partial separability and decomposition partial separability first order methods interior-point methods Conic linear optimization
More information2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian
FE661 - Statistical Methods for Financial Engineering 2. Linear algebra Jitkomut Songsiri matrices and vectors linear equations range and nullspace of matrices function of vectors, gradient and Hessian
More informationConvex Optimization. (EE227A: UC Berkeley) Lecture 6. Suvrit Sra. (Conic optimization) 07 Feb, 2013
Convex Optimization (EE227A: UC Berkeley) Lecture 6 (Conic optimization) 07 Feb, 2013 Suvrit Sra Organizational Info Quiz coming up on 19th Feb. Project teams by 19th Feb Good if you can mix your research
More informationLecture 8: Linear Algebra Background
CSE 521: Design and Analysis of Algorithms I Winter 2017 Lecture 8: Linear Algebra Background Lecturer: Shayan Oveis Gharan 2/1/2017 Scribe: Swati Padmanabhan Disclaimer: These notes have not been subjected
More informationM. Seetharama Gowda. J. Tao. G. Ravindran
Some complementarity properties of Z and Lyapunov-like transformations on symmetric cones M. Seetharama Gowda Department of Mathematics and Statistics, University of Maryland, Baltimore County, Baltimore,
More informationLecture: Examples of LP, SOCP and SDP
1/34 Lecture: Examples of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:
More informationLinear Algebra. Session 12
Linear Algebra. Session 12 Dr. Marco A Roque Sol 08/01/2017 Example 12.1 Find the constant function that is the least squares fit to the following data x 0 1 2 3 f(x) 1 0 1 2 Solution c = 1 c = 0 f (x)
More informationA Distributed Newton Method for Network Utility Maximization, II: Convergence
A Distributed Newton Method for Network Utility Maximization, II: Convergence Ermin Wei, Asuman Ozdaglar, and Ali Jadbabaie October 31, 2012 Abstract The existing distributed algorithms for Network Utility
More information2. Review of Linear Algebra
2. Review of Linear Algebra ECE 83, Spring 217 In this course we will represent signals as vectors and operators (e.g., filters, transforms, etc) as matrices. This lecture reviews basic concepts from linear
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 2
EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 2 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory April 5, 2012 Andre Tkacenko
More informationLINEAR ALGEBRA BOOT CAMP WEEK 4: THE SPECTRAL THEOREM
LINEAR ALGEBRA BOOT CAMP WEEK 4: THE SPECTRAL THEOREM Unless otherwise stated, all vector spaces in this worksheet are finite dimensional and the scalar field F is R or C. Definition 1. A linear operator
More informationReview of Linear Algebra
Review of Linear Algebra Definitions An m n (read "m by n") matrix, is a rectangular array of entries, where m is the number of rows and n the number of columns. 2 Definitions (Con t) A is square if m=
More informationWhat can be expressed via Conic Quadratic and Semidefinite Programming?
What can be expressed via Conic Quadratic and Semidefinite Programming? A. Nemirovski Faculty of Industrial Engineering and Management Technion Israel Institute of Technology Abstract Tremendous recent
More informationAn E cient A ne-scaling Algorithm for Hyperbolic Programming
An E cient A ne-scaling Algorithm for Hyperbolic Programming Jim Renegar joint work with Mutiara Sondjaja 1 Euclidean space A homogeneous polynomial p : E!R is hyperbolic if there is a vector e 2E such
More informationJordan-algebraic aspects of optimization:randomization
Jordan-algebraic aspects of optimization:randomization L. Faybusovich, Department of Mathematics, University of Notre Dame, 255 Hurley Hall, Notre Dame, IN, 46545 June 29 2007 Abstract We describe a version
More informationConvex Optimization M2
Convex Optimization M2 Lecture 8 A. d Aspremont. Convex Optimization M2. 1/57 Applications A. d Aspremont. Convex Optimization M2. 2/57 Outline Geometrical problems Approximation problems Combinatorial
More informationCOMP 558 lecture 18 Nov. 15, 2010
Least squares We have seen several least squares problems thus far, and we will see more in the upcoming lectures. For this reason it is good to have a more general picture of these problems and how to
More informationLinear Algebra: Matrix Eigenvalue Problems
CHAPTER8 Linear Algebra: Matrix Eigenvalue Problems Chapter 8 p1 A matrix eigenvalue problem considers the vector equation (1) Ax = λx. 8.0 Linear Algebra: Matrix Eigenvalue Problems Here A is a given
More informationLecture 15 Newton Method and Self-Concordance. October 23, 2008
Newton Method and Self-Concordance October 23, 2008 Outline Lecture 15 Self-concordance Notion Self-concordant Functions Operations Preserving Self-concordance Properties of Self-concordant Functions Implications
More informationLecture 17: Primal-dual interior-point methods part II
10-725/36-725: Convex Optimization Spring 2015 Lecture 17: Primal-dual interior-point methods part II Lecturer: Javier Pena Scribes: Pinchao Zhang, Wei Ma Note: LaTeX template courtesy of UC Berkeley EECS
More informationPermutation invariant proper polyhedral cones and their Lyapunov rank
Permutation invariant proper polyhedral cones and their Lyapunov rank Juyoung Jeong Department of Mathematics and Statistics University of Maryland, Baltimore County Baltimore, Maryland 21250, USA juyoung1@umbc.edu
More informationLinear algebra for computational statistics
University of Seoul May 3, 2018 Vector and Matrix Notation Denote 2-dimensional data array (n p matrix) by X. Denote the element in the ith row and the jth column of X by x ij or (X) ij. Denote by X j
More informationThe Algebraic Structure of the p-order Cone
The Algebraic Structure of the -Order Cone Baha Alzalg Abstract We study and analyze the algebraic structure of the -order cones We show that, unlike oularly erceived, the -order cone is self-dual for
More informationMore First-Order Optimization Algorithms
More First-Order Optimization Algorithms Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapters 3, 8, 3 The SDM
More informationAnalysis of Control Systems on Symmetric Cones
Analysis of Control Systems on Symmetric Cones Ivan Papusha Richard M. Murray Abstract It is well known that exploiting special structure is a powerful way to extend the reach of current optimization tools
More informationChapter 3 Transformations
Chapter 3 Transformations An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Linear Transformations A function is called a linear transformation if 1. for every and 2. for every If we fix the bases
More informationMatrix Rank Minimization with Applications
Matrix Rank Minimization with Applications Maryam Fazel Haitham Hindi Stephen Boyd Information Systems Lab Electrical Engineering Department Stanford University 8/2001 ACC 01 Outline Rank Minimization
More informationCSC Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming
CSC2411 - Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming Notes taken by Mike Jamieson March 28, 2005 Summary: In this lecture, we introduce semidefinite programming
More informationEE Applications of Convex Optimization in Signal Processing and Communications Dr. Andre Tkacenko, JPL Third Term
EE 150 - Applications of Convex Optimization in Signal Processing and Communications Dr. Andre Tkacenko JPL Third Term 2011-2012 Due on Thursday May 3 in class. Homework Set #4 1. (10 points) (Adapted
More informationN. L. P. NONLINEAR PROGRAMMING (NLP) deals with optimization models with at least one nonlinear function. NLP. Optimization. Models of following form:
0.1 N. L. P. Katta G. Murty, IOE 611 Lecture slides Introductory Lecture NONLINEAR PROGRAMMING (NLP) deals with optimization models with at least one nonlinear function. NLP does not include everything
More informationMatrix Mathematics. Theory, Facts, and Formulas with Application to Linear Systems Theory. Dennis S. Bernstein
Matrix Mathematics Theory, Facts, and Formulas with Application to Linear Systems Theory Dennis S. Bernstein PRINCETON UNIVERSITY PRESS PRINCETON AND OXFORD Contents Special Symbols xv Conventions, Notation,
More informationCSCI 1951-G Optimization Methods in Finance Part 10: Conic Optimization
CSCI 1951-G Optimization Methods in Finance Part 10: Conic Optimization April 6, 2018 1 / 34 This material is covered in the textbook, Chapters 9 and 10. Some of the materials are taken from it. Some of
More information3. Convex functions. basic properties and examples. operations that preserve convexity. the conjugate function. quasiconvex functions
3. Convex functions Convex Optimization Boyd & Vandenberghe basic properties and examples operations that preserve convexity the conjugate function quasiconvex functions log-concave and log-convex functions
More informationMath 3191 Applied Linear Algebra
Math 9 Applied Linear Algebra Lecture 9: Diagonalization Stephen Billups University of Colorado at Denver Math 9Applied Linear Algebra p./9 Section. Diagonalization The goal here is to develop a useful
More informationECON 4117/5111 Mathematical Economics
Test 1 September 29, 2006 1. Use a truth table to show the following equivalence statement: (p q) (p q) 2. Consider the statement: A function f : X Y is continuous on X if for every open set V Y, the pre-image
More informationAPPENDIX A. Background Mathematics. A.1 Linear Algebra. Vector algebra. Let x denote the n-dimensional column vector with components x 1 x 2.
APPENDIX A Background Mathematics A. Linear Algebra A.. Vector algebra Let x denote the n-dimensional column vector with components 0 x x 2 B C @. A x n Definition 6 (scalar product). The scalar product
More informationProblems in Linear Algebra and Representation Theory
Problems in Linear Algebra and Representation Theory (Most of these were provided by Victor Ginzburg) The problems appearing below have varying level of difficulty. They are not listed in any specific
More informationPositive and doubly stochastic maps, and majorization in Euclidean Jordan algebras
Positive and doubly stochastic maps, and majorization in Euclidean Jordan algebras M. Seetharama Gowda Department of Mathematics and Statistics University of Maryland, Baltimore County Baltimore, Maryland
More informationOn self-concordant barriers for generalized power cones
On self-concordant barriers for generalized power cones Scott Roy Lin Xiao January 30, 2018 Abstract In the study of interior-point methods for nonsymmetric conic optimization and their applications, Nesterov
More informationLinear Algebra: Characteristic Value Problem
Linear Algebra: Characteristic Value Problem . The Characteristic Value Problem Let < be the set of real numbers and { be the set of complex numbers. Given an n n real matrix A; does there exist a number
More informationSubgradient. Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes. definition. subgradient calculus
1/41 Subgradient Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes definition subgradient calculus duality and optimality conditions directional derivative Basic inequality
More informationGeometric problems. Chapter Projection on a set. The distance of a point x 0 R n to a closed set C R n, in the norm, is defined as
Chapter 8 Geometric problems 8.1 Projection on a set The distance of a point x 0 R n to a closed set C R n, in the norm, is defined as dist(x 0,C) = inf{ x 0 x x C}. The infimum here is always achieved.
More informationThe Q Method for Symmetric Cone Programmin
The Q Method for Symmetric Cone Programming The Q Method for Symmetric Cone Programmin Farid Alizadeh and Yu Xia alizadeh@rutcor.rutgers.edu, xiay@optlab.mcma Large Scale Nonlinear and Semidefinite Progra
More information10. Unconstrained minimization
Convex Optimization Boyd & Vandenberghe 10. Unconstrained minimization terminology and assumptions gradient descent method steepest descent method Newton s method self-concordant functions implementation
More informationBindel, Fall 2016 Matrix Computations (CS 6210) Notes for
1 Logistics Notes for 2016-08-29 General announcement: we are switching from weekly to bi-weekly homeworks (mostly because the course is much bigger than planned). If you want to do HW but are not formally
More informationOn Conically Ordered Convex Programs
On Conically Ordered Convex Programs Shuzhong Zhang May 003; revised December 004 Abstract In this paper we study a special class of convex optimization problems called conically ordered convex programs
More information1 Overview. 2 A Characterization of Convex Functions. 2.1 First-order Taylor approximation. AM 221: Advanced Optimization Spring 2016
AM 221: Advanced Optimization Spring 2016 Prof. Yaron Singer Lecture 8 February 22nd 1 Overview In the previous lecture we saw characterizations of optimality in linear optimization, and we reviewed the
More information3. Convex functions. basic properties and examples. operations that preserve convexity. the conjugate function. quasiconvex functions
3. Convex functions Convex Optimization Boyd & Vandenberghe basic properties and examples operations that preserve convexity the conjugate function quasiconvex functions log-concave and log-convex functions
More informationLecture: Duality of LP, SOCP and SDP
1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:
More informationDepartment of Social Systems and Management. Discussion Paper Series
Department of Social Systems and Management Discussion Paper Series No. 1262 Complementarity Problems over Symmetric Cones: A Survey of Recent Developments in Several Aspects by Akiko YOSHISE July 2010
More informationSecond-Order Cone Programming
Second-Order Cone Programming F. Alizadeh D. Goldfarb 1 Introduction Second-order cone programming (SOCP) problems are convex optimization problems in which a linear function is minimized over the intersection
More informationMATH 581D FINAL EXAM Autumn December 12, 2016
MATH 58D FINAL EXAM Autumn 206 December 2, 206 NAME: SIGNATURE: Instructions: there are 6 problems on the final. Aim for solving 4 problems, but do as much as you can. Partial credit will be given on all
More informationLecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.
MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More informationCompound matrices and some classical inequalities
Compound matrices and some classical inequalities Tin-Yau Tam Mathematics & Statistics Auburn University Dec. 3, 04 We discuss some elegant proofs of several classical inequalities of matrices by using
More informationSemidefinite Programming
Semidefinite Programming Notes by Bernd Sturmfels for the lecture on June 26, 208, in the IMPRS Ringvorlesung Introduction to Nonlinear Algebra The transition from linear algebra to nonlinear algebra has
More informationSparse Optimization Lecture: Basic Sparse Optimization Models
Sparse Optimization Lecture: Basic Sparse Optimization Models Instructor: Wotao Yin July 2013 online discussions on piazza.com Those who complete this lecture will know basic l 1, l 2,1, and nuclear-norm
More information