Conve functions 2{2 Conve functions f : R n! R is conve if dom f is conve and y 2 dom f 2 [0 1] + f ( + (1 ; )y) f () + (1 ; )f (y) (1) f is concave i
|
|
- Alicia Bradley
- 6 years ago
- Views:
Transcription
1 ELEC-E5420 Conve functions 2{1 Lecture 2 Conve functions conve functions, epigraph simple eamples, elementary properties more eamples, more properties Jensen's inequality quasiconve, quasiconcave functions log-conve and log-concave functions K-conveity
2 Conve functions 2{2 Conve functions f : R n! R is conve if dom f is conve and y 2 dom f 2 [0 1] + f ( + (1 ; )y) f () + (1 ; )f (y) (1) f is concave if ;f is conve conve concave neither `Modern' denition: f : R n! R [ f+1g (but not identically +1) f is conve if (1) holds as an inequality in R [ f+1g
3 Conve functions 2{3 Epigraph & sublevel sets The epigraph of the function f is epi f = f( t) j 2 dom f f () t g : f () epi f f conve function, epi f conve set The (-)sublevel set of f is C() = f 2 dom f j f () g : f conve ) sublevel sets are conve (converse false)
4 Conve functions 2{4 Dierentiable conve functions f dierentiable and conve () 8 0 : f () f ( 0 ) + rf ( 0 ) T ( ; 0 ) f () 0 f ( 0 )+rf ( 0 ) T ( ; 0 ) Interpretation 1st order Taylor appr. is a global lower bound on f supporting hyperplane to epi f: ( t) 2 epi f =) f () 2 6 rf ( 0) 4 ; T ; 0 t ; f ( 0 ) rf ( 0) ;1 3 5 f twice dierentiable and conve () r 2 f () 0
5 Conve functions 2{5 Simple eamples linear and ane functions: f () = a T + b conve quadratic functions: f () = T P+ 2q T + r with P = P T 0 any norm Eamples on R is conve on R + for 1, 0 concave for 0 1 log is concave, log is conve on R + e is conve jj, ma(0 ), ma(0 ;) are conve Z ;1 e;t2 dt is concave log
6 Conve functions 2{6 Elementary properties a function is conve i it is conve on all lines: f conve () f ( 0 + th) conve in t for all 0 h positive multiple of conve function is conve: f conve 0 =) f conve sum of conve functions is conve: f 1 f 2 conve =) f 1 + f 2 conve etends to innite sums, integrals: g( y) conve in =) Z g( y)dy conve
7 Conve functions 2{7 pointwise maimum: f 1 f 2 conve =) maff 1 () f 2 ()g conve (corresponds to intersection of epigraphs) f 2 () epi maff 1 f 2 g f 1 () pointwise supremum: f conve =) sup f conve 2A ane transformation of domain f conve ) f (A + b) conve
8 Conve functions 2{8 More eamples piecewise-linear functions: f () = mafa T i i + b ig is conve in (epi f is polyhedron) ma distance to any set, sup s2s k ; sk, is conve in f () = [1] + [2] + [3] is conve on R n ( [i] is the ith largest j ) f () = 0 Y i 1 C i A 1=n is concave on R n + f () = m X i=1 log(b i ; a T i );1 is conve on P = f j a T i < b i i = 1 ::: mg least-squares cost as functions of weights, is concave in w f (w) = inf X i w i (a T i ; b i) 2
9 Conve functions 2{9 Conve functions of matrices Tr X is linear in X more generally, Tr A T X = X i j A ij X ij = vec(a) T vec(x) log det X ;1 is conve on X = X T 0 Proof: let i be the eigenvalues of X ;1=2 0 HX ;1=2 0 f (t) = log det(x 0 + th) ;1 = log det X ;1 0 + log det(i + tx ;1=2 0 HX ;1=2 0 ) ;1 = log det X ;1 0 ; X i is a conve function of t log(1 + t i ) (det X) 1=n is concave on X = X T 0, X 2 R nn ma (X) is conve on X = X T Proof: ma (X) = sup y T Xy kyk=1 kxk = ma (X T 1=2 X) is conve on R nm Proof: kxk = sup u T Xv kuk=1 kvk=1
10 Conve functions 2 {10 Minimizing over some variables If h( y) is conve in and y, then is conve in f () = inf y h( y) corresponds to projection of epigraph, ( y t)! ( t) h( y) f () y
11 Conve functions 2 {11 Eample. If S R n is conve then (min) distance to S, is conve in dist( S) = inf k ; sk s2s Eample. If g() is conve, then is conve in y. f (y) = inffg() j A = yg Proof: nd B, C s.t. f j A = yg = fby + Cz j z 2 R k g so f (y) = inf z g(by + Cz) `Modern' proof: f (y) = inf z g() + h(a ; y) where is conve h(z) = 8 >< >: 0 if z = 0 +1 otherwise
12 Conve functions 2 {12 Composition one-dimensional case f () = h(g()) is conve if g conve h conve, nondecreasing g concave h conve, nonincreasing Eamples f () = ep g() is conve if g is conve f () = 1=g() is conve if g is concave, positive f () = g() p, p 1, is conve if g() conve, positive f 1,..., f n conve, then f () = ; X i conve on f j f i () < 0 i = 1 ::: ng log(;f i ()) is Proof: (dierentiable functions, 2 R) f 00 = h 00 (g 0 ) 2 + g 00 h 0
13 Conve functions 2 {13 Composition k-dimensional case f () = h(g 1 () ::: g k ()) with h : R k! R, g i : R n! R is conve if h conve, nondecreasing in each arg. g i conve h conve, nonincreasing in each arg. g i concave etc. Eamples f () = ma i g i () is conve if each g i is f () = log X i ep g i () is conve if each g i is Proof: (dierentiable functions, n = 1) f 00 = rh T g g 00 k 3 2 g g k T r 2 h g g k
14 Conve functions 2 {14 Jensen's inequality f : R n! R conve two points 2 [0 1] + f ( 1 + (1 ; ) 2 ) f ( 1 ) + (1 ; )f ( 2 ) more than two points f ( X i i 0 X i = 1 + i i i ) X i i f ( i ) continuous version most general form: p() 0 Z p()d = 1 + f (Z p()d) Z f ()p()d f (E ) E f () Interpretation: (zero mean) randomization, dithering increases average value of a conve function
15 Conve functions 2 {15 Applications Many (some people claim most) inequalities can be derived from Jensen's inequality Eample. Arithmetic-geometric mean inequality a b 0 ) p ab (a + b)=2 Proof. f () = log is concave on R + : 0 1 B (log a + log b) log + b 1 C A 2 2
16 Conve functions 2 {16 Quasiconve functions f : C! R, C a conve set, is quasiconve if every sublevel set S = f j f () g is conve. y S can have `locally at' regions f () f is quasiconcave if ;f is quasiconve, i.e., superlevel sets f j f () g are conve. A function which is both quasiconve and quasiconcave is called quasilinear. f conve (concave) ) f quasiconve (quasiconcave)
17 Conve functions 2 {17 Eamples f () = r jj is quasiconve on R f () = log is quasilinear on R + linear fractional function, f () = at + b c T + d is quasilinear on the halfspace c T + d > 0 k ; ak k ; bk f j k ; ak k ; bkg f () = is quasiconve on the halfspace f (a) = degree(a 0 + a 1 t + + a k t k ) on R k+1
18 Conve functions 2 {18 Properties f is quasiconve if and only if it is quasiconve on lines, i.e., f ( 0 + th) quasiconve in t for all 0 h. modied Jensen's inequality: f : C! R quasiconve if and only if y 2 C 2 [0 1] + f ( + (1 ; )y) maff () f(y)g f () y
19 Conve functions 2 {19 for f dierentiable, f quasiconve if and only if for all, y f (y) f () ) (y ; ) T rf () 0 S 1 rf () S 2 S 3 1 < 2 < 3 positive multiples f quasiconve 0 =) f quasiconve pointwise maimum f 1 f 2 quasiconve =) maff 1 f 2 g quasiconve (etends to supremum over arbitrary set) ane transformation of domain f quasiconve =) f (A + b) quasiconve
20 Conve functions 2 {20 projective transformation of domain 0 B f quasiconve =) f + b c T + d on c T + d > 0 1 C A quasiconve composition with monotone increasing function f quasiconve g monotone increasing =) g(f ()) quasiconve sums of quasiconve functions are not quasiconve in general f quasiconve in, y =) g() = inf y quasiconve in f ( y)
21 Conve functions 2 {21 Nested sets characterization f quasiconve ) sublevel sets S conve, nested, i.e., 1 2 ) S 1 S 2 converse: if T is a nested family of conve sets, then is quasiconve. f () = inff j 2 T g Engineering interpretation: T are specs, tighter for smaller
22 Conve functions 2 {25 Log-concave functions f : R n! R + is log-concave (log-conve) if log f is concave (conve) Log-conve ) conve concave ) log-concave `Modern' denition allows log-concave f to take on value zero, so log f takes on value ;1 Eamples normal density, f () = e ;(1=2)(; 0) T ;1 (; 0 ) erfc, f () = p 2 Z 1 e;t2 dt indicator function of conve set C: I C () = 8 >< >: 1 2 C 0 62 C
23 Conve functions 2 {26 Properties sum of log-concave functions not always log-concave (but sum of log-conve functions is log-conve) products (immediate) f g log-concave =) fg log-concave integrals f ( y) log-concave in y =) Z f ( y)dy log-concave convolutions f g log-concave =) Z f ( ; y)g(y)dy log-concave (immediate from the properties above)
24 Conve functions 2 {27 Log-concave probability densities Many common probability density functions are log-concave. Eamples normal ( 0) f () = 1 r(2) n det e;1 2 (;) T ;1 (;) eponential( i > 0) on R n + f () = 0 n Y i=1 i 1 C A e ;( n n ) uniform distribution on conve (bounded) set C f () = 8 >< >: 1= 2 C 0 62 C where is Lebesgue measure of C (i.e., length, area, volume...)
25 Conve functions 2 {31 K-conveity conve cone K R m induces generalized inequality K f : R n! R m is K-conve if 0 1 =) f ( + (1 ; )y) K f () + (1 ; )f (y) Eample. K is PSD cone (called matri conveity) let's show that f (X) = X 2 is K-conve on fxjx = X T g, i.e., for 2 [0 1], (X + (1 ; )Y ) 2 X 2 + (1 ; )Y 2 (1) for any u 2 R m, u T X 2 u = kxuk 2 is a (quadratic) conve fct of X, so u T (X + (1 ; )Y ) 2 u u T X 2 u + (1 ; )u T Y 2 u which implies (1)
Lecture 2: Convex functions
Lecture 2: Convex functions f : R n R is convex if dom f is convex and for all x, y dom f, θ [0, 1] f is concave if f is convex f(θx + (1 θ)y) θf(x) + (1 θ)f(y) x x convex concave neither x examples (on
More informationConvex functions. Definition. f : R n! R is convex if dom f is a convex set and. f ( x +(1 )y) < f (x)+(1 )f (y) f ( x +(1 )y) apple f (x)+(1 )f (y)
Convex functions I basic properties and I operations that preserve convexity I quasiconvex functions I log-concave and log-convex functions IOE 611: Nonlinear Programming, Fall 2017 3. Convex functions
More information3. Convex functions. basic properties and examples. operations that preserve convexity. the conjugate function. quasiconvex functions
3. Convex functions Convex Optimization Boyd & Vandenberghe basic properties and examples operations that preserve convexity the conjugate function quasiconvex functions log-concave and log-convex functions
More informationConvex Functions. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Convex Functions Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Definition convex function Examples
More information3. Convex functions. basic properties and examples. operations that preserve convexity. the conjugate function. quasiconvex functions
3. Convex functions Convex Optimization Boyd & Vandenberghe basic properties and examples operations that preserve convexity the conjugate function quasiconvex functions log-concave and log-convex functions
More informationConvex Functions. Wing-Kin (Ken) Ma The Chinese University of Hong Kong (CUHK)
Convex Functions Wing-Kin (Ken) Ma The Chinese University of Hong Kong (CUHK) Course on Convex Optimization for Wireless Comm. and Signal Proc. Jointly taught by Daniel P. Palomar and Wing-Kin (Ken) Ma
More informationConvex Functions. Pontus Giselsson
Convex Functions Pontus Giselsson 1 Today s lecture lower semicontinuity, closure, convex hull convexity preserving operations precomposition with affine mapping infimal convolution image function supremum
More informationLecture 1: Background on Convex Analysis
Lecture 1: Background on Convex Analysis John Duchi PCMI 2016 Outline I Convex sets 1.1 Definitions and examples 2.2 Basic properties 3.3 Projections onto convex sets 4.4 Separating and supporting hyperplanes
More informationA function(al) f is convex if dom f is a convex set, and. f(θx + (1 θ)y) < θf(x) + (1 θ)f(y) f(x) = x 3
Convex functions The domain dom f of a functional f : R N R is the subset of R N where f is well-defined. A function(al) f is convex if dom f is a convex set, and f(θx + (1 θ)y) θf(x) + (1 θ)f(y) for all
More informationSubgradient. Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes. definition. subgradient calculus
1/41 Subgradient Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes definition subgradient calculus duality and optimality conditions directional derivative Basic inequality
More informationShiqian Ma, MAT-258A: Numerical Optimization 1. Chapter 4. Subgradient
Shiqian Ma, MAT-258A: Numerical Optimization 1 Chapter 4 Subgradient Shiqian Ma, MAT-258A: Numerical Optimization 2 4.1. Subgradients definition subgradient calculus duality and optimality conditions Shiqian
More informationEconomics 205 Exercises
Economics 05 Eercises Prof. Watson, Fall 006 (Includes eaminations through Fall 003) Part 1: Basic Analysis 1. Using ε and δ, write in formal terms the meaning of lim a f() = c, where f : R R.. Write the
More informationConvex Optimization. 4. Convex Optimization Problems. Prof. Ying Cui. Department of Electrical Engineering Shanghai Jiao Tong University
Conve Optimization 4. Conve Optimization Problems Prof. Ying Cui Department of Electrical Engineering Shanghai Jiao Tong University 2017 Autumn Semester SJTU Ying Cui 1 / 58 Outline Optimization problems
More informationLecture 10. ( x domf. Remark 1 The domain of the conjugate function is given by
10-1 Multi-User Information Theory Jan 17, 2012 Lecture 10 Lecturer:Dr. Haim Permuter Scribe: Wasim Huleihel I. CONJUGATE FUNCTION In the previous lectures, we discussed about conve set, conve functions
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 17
EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 17 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory May 29, 2012 Andre Tkacenko
More informationMathematics 530. Practice Problems. n + 1 }
Department of Mathematical Sciences University of Delaware Prof. T. Angell October 19, 2015 Mathematics 530 Practice Problems 1. Recall that an indifference relation on a partially ordered set is defined
More informationThe proximal mapping
The proximal mapping http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes Outline 2/37 1 closed function 2 Conjugate function
More informationLMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009
LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix
More information1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics
1. Introduction Convex Optimization Boyd & Vandenberghe mathematical optimization least-squares and linear programming convex optimization example course goals and topics nonlinear optimization brief history
More information1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics
1. Introduction Convex Optimization Boyd & Vandenberghe mathematical optimization least-squares and linear programming convex optimization example course goals and topics nonlinear optimization brief history
More informationIE 521 Convex Optimization
Lecture 5: Convex II 6th February 2019 Convex Local Lipschitz Outline Local Lipschitz 1 / 23 Convex Local Lipschitz Convex Function: f : R n R is convex if dom(f ) is convex and for any λ [0, 1], x, y
More informationOptimality Conditions for Nonsmooth Convex Optimization
Optimality Conditions for Nonsmooth Convex Optimization Sangkyun Lee Oct 22, 2014 Let us consider a convex function f : R n R, where R is the extended real field, R := R {, + }, which is proper (f never
More informationA Tutorial on Convex Optimization
A Tutorial on Conve Optimization Haitham Hindi Palo Alto Research Center (PARC), Palo Alto, California email: hhindi@parc.com Abstract In recent years, conve optimization has become a computational tool
More informationOptimisation convexe: performance, complexité et applications.
Optimisation convexe: performance, complexité et applications. Introduction, convexité, dualité. A. d Aspremont. M2 OJME: Optimisation convexe. 1/128 Today Convex optimization: introduction Course organization
More informationCHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS. W. Erwin Diewert January 31, 2008.
1 ECONOMICS 594: LECTURE NOTES CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS W. Erwin Diewert January 31, 2008. 1. Introduction Many economic problems have the following structure: (i) a linear function
More informationOptimization and Optimal Control in Banach Spaces
Optimization and Optimal Control in Banach Spaces Bernhard Schmitzer October 19, 2017 1 Convex non-smooth optimization with proximal operators Remark 1.1 (Motivation). Convex optimization: easier to solve,
More information4. Convex optimization problems
Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization
More informationLecture 8: Basic convex analysis
Lecture 8: Basic convex analysis 1 Convex sets Both convex sets and functions have general importance in economic theory, not only in optimization. Given two points x; y 2 R n and 2 [0; 1]; the weighted
More informationCSCI : Optimization and Control of Networks. Review on Convex Optimization
CSCI7000-016: Optimization and Control of Networks Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one
More informationLecture: Convex Optimization Problems
1/36 Lecture: Convex Optimization Problems http://bicmr.pku.edu.cn/~wenzw/opt-2015-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/36 optimization
More informationA Brief Review on Convex Optimization
A Brief Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one convex, two nonconvex sets): A Brief Review
More informationSolutions Chapter 5. The problem of finding the minimum distance from the origin to a line is written as. min 1 2 kxk2. subject to Ax = b.
Solutions Chapter 5 SECTION 5.1 5.1.4 www Throughout this exercise we will use the fact that strong duality holds for convex quadratic problems with linear constraints (cf. Section 3.4). The problem of
More informationConvex Optimization Problems. Prof. Daniel P. Palomar
Conve Optimization Problems Prof. Daniel P. Palomar The Hong Kong University of Science and Technology (HKUST) MAFS6010R- Portfolio Optimization with R MSc in Financial Mathematics Fall 2018-19, HKUST,
More information1 Kernel methods & optimization
Machine Learning Class Notes 9-26-13 Prof. David Sontag 1 Kernel methods & optimization One eample of a kernel that is frequently used in practice and which allows for highly non-linear discriminant functions
More informationNonlinear Programming Models
Nonlinear Programming Models Fabio Schoen 2008 http://gol.dsi.unifi.it/users/schoen Nonlinear Programming Models p. Introduction Nonlinear Programming Models p. NLP problems minf(x) x S R n Standard form:
More information4. Convex optimization problems
Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization
More information8. Conjugate functions
L. Vandenberghe EE236C (Spring 2013-14) 8. Conjugate functions closed functions conjugate function 8-1 Closed set a set C is closed if it contains its boundary: x k C, x k x = x C operations that preserve
More informationConvex Functions and Optimization
Chapter 5 Convex Functions and Optimization 5.1 Convex Functions Our next topic is that of convex functions. Again, we will concentrate on the context of a map f : R n R although the situation can be generalized
More informationUNCONSTRAINED OPTIMIZATION PAUL SCHRIMPF OCTOBER 24, 2013
PAUL SCHRIMPF OCTOBER 24, 213 UNIVERSITY OF BRITISH COLUMBIA ECONOMICS 26 Today s lecture is about unconstrained optimization. If you re following along in the syllabus, you ll notice that we ve skipped
More informationConvex Optimization and Modeling
Convex Optimization and Modeling Duality Theory and Optimality Conditions 5th lecture, 12.05.2010 Jun.-Prof. Matthias Hein Program of today/next lecture Lagrangian and duality: the Lagrangian the dual
More informationCOURSE ON LMI PART I.2 GEOMETRY OF LMI SETS. Didier HENRION henrion
COURSE ON LMI PART I.2 GEOMETRY OF LMI SETS Didier HENRION www.laas.fr/ henrion October 2006 Geometry of LMI sets Given symmetric matrices F i we want to characterize the shape in R n of the LMI set F
More informationLinear and non-linear programming
Linear and non-linear programming Benjamin Recht March 11, 2005 The Gameplan Constrained Optimization Convexity Duality Applications/Taxonomy 1 Constrained Optimization minimize f(x) subject to g j (x)
More informationSubgradients. subgradients and quasigradients. subgradient calculus. optimality conditions via subgradients. directional derivatives
Subgradients subgradients and quasigradients subgradient calculus optimality conditions via subgradients directional derivatives Prof. S. Boyd, EE392o, Stanford University Basic inequality recall basic
More informationSemidefinite Programming Basics and Applications
Semidefinite Programming Basics and Applications Ray Pörn, principal lecturer Åbo Akademi University Novia University of Applied Sciences Content What is semidefinite programming (SDP)? How to represent
More informationConvex analysis and profit/cost/support functions
Division of the Humanities and Social Sciences Convex analysis and profit/cost/support functions KC Border October 2004 Revised January 2009 Let A be a subset of R m Convex analysts may give one of two
More informationN. L. P. NONLINEAR PROGRAMMING (NLP) deals with optimization models with at least one nonlinear function. NLP. Optimization. Models of following form:
0.1 N. L. P. Katta G. Murty, IOE 611 Lecture slides Introductory Lecture NONLINEAR PROGRAMMING (NLP) deals with optimization models with at least one nonlinear function. NLP does not include everything
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Notes for EE7C (Spring 018): Convex Optimization and Approximation Instructor: Moritz Hardt Email: hardt+ee7c@berkeley.edu Graduate Instructor: Max Simchowitz Email: msimchow+ee7c@berkeley.edu February
More informationHW1 solutions. 1. α Ef(x) β, where Ef(x) is the expected value of f(x), i.e., Ef(x) = n. i=1 p if(a i ). (The function f : R R is given.
HW1 solutions Exercise 1 (Some sets of probability distributions.) Let x be a real-valued random variable with Prob(x = a i ) = p i, i = 1,..., n, where a 1 < a 2 < < a n. Of course p R n lies in the standard
More informationSubgradients. subgradients. strong and weak subgradient calculus. optimality conditions via subgradients. directional derivatives
Subgradients subgradients strong and weak subgradient calculus optimality conditions via subgradients directional derivatives Prof. S. Boyd, EE364b, Stanford University Basic inequality recall basic inequality
More informationGeometric problems. Chapter Projection on a set. The distance of a point x 0 R n to a closed set C R n, in the norm, is defined as
Chapter 8 Geometric problems 8.1 Projection on a set The distance of a point x 0 R n to a closed set C R n, in the norm, is defined as dist(x 0,C) = inf{ x 0 x x C}. The infimum here is always achieved.
More informationLinear Algebra: Characteristic Value Problem
Linear Algebra: Characteristic Value Problem . The Characteristic Value Problem Let < be the set of real numbers and { be the set of complex numbers. Given an n n real matrix A; does there exist a number
More informationConvex Optimization in Communications and Signal Processing
Convex Optimization in Communications and Signal Processing Prof. Dr.-Ing. Wolfgang Gerstacker 1 University of Erlangen-Nürnberg Institute for Digital Communications National Technical University of Ukraine,
More informationConvex optimization problems. Optimization problem in standard form
Convex optimization problems optimization problem in standard form convex optimization problems linear optimization quadratic optimization geometric programming quasiconvex optimization generalized inequality
More informationsubject to (x 2)(x 4) u,
Exercises Basic definitions 5.1 A simple example. Consider the optimization problem with variable x R. minimize x 2 + 1 subject to (x 2)(x 4) 0, (a) Analysis of primal problem. Give the feasible set, the
More informationChapter 2 Convex Analysis
Chapter 2 Convex Analysis The theory of nonsmooth analysis is based on convex analysis. Thus, we start this chapter by giving basic concepts and results of convexity (for further readings see also [202,
More informationLecture 7: Weak Duality
EE 227A: Conve Optimization and Applications February 7, 2012 Lecture 7: Weak Duality Lecturer: Laurent El Ghaoui 7.1 Lagrange Dual problem 7.1.1 Primal problem In this section, we consider a possibly
More informationTranslative Sets and Functions and their Applications to Risk Measure Theory and Nonlinear Separation
Translative Sets and Functions and their Applications to Risk Measure Theory and Nonlinear Separation Andreas H. Hamel Abstract Recently defined concepts such as nonlinear separation functionals due to
More informationEC5555 Economics Masters Refresher Course in Mathematics September 2013
EC5555 Economics Masters Reresher Course in Mathematics September 3 Lecture 5 Unconstraine Optimization an Quaratic Forms Francesco Feri We consier the unconstraine optimization or the case o unctions
More information58 Appendix 1 fundamental inconsistent equation (1) can be obtained as a linear combination of the two equations in (2). This clearly implies that the
Appendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS Here we consider systems of linear constraints, consisting of equations or inequalities or both. A feasible solution
More informationCOM S 578X: Optimization for Machine Learning
COM S 578X: Optimization for Machine Learning Lecture Note 4: Duality Jia (Kevin) Liu Assistant Professor Department of Computer Science Iowa State University, Ames, Iowa, USA Fall 2018 JKL (CS@ISU) COM
More informationMatrix Support Functional and its Applications
Matrix Support Functional and its Applications James V Burke Mathematics, University of Washington Joint work with Yuan Gao (UW) and Tim Hoheisel (McGill), CORS, Banff 2016 June 1, 2016 Connections What
More informationDeterminant maximization with linear. S. Boyd, L. Vandenberghe, S.-P. Wu. Information Systems Laboratory. Stanford University
Determinant maximization with linear matrix inequality constraints S. Boyd, L. Vandenberghe, S.-P. Wu Information Systems Laboratory Stanford University SCCM Seminar 5 February 1996 1 MAXDET problem denition
More informationL. Vandenberghe EE236C (Spring 2016) 18. Symmetric cones. definition. spectral decomposition. quadratic representation. log-det barrier 18-1
L. Vandenberghe EE236C (Spring 2016) 18. Symmetric cones definition spectral decomposition quadratic representation log-det barrier 18-1 Introduction This lecture: theoretical properties of the following
More information9. Interpretations, Lifting, SOS and Moments
9-1 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC 2003 2003.12.07.04 9. Interpretations, Lifting, SOS and Moments Polynomial nonnegativity Sum of squares (SOS) decomposition Eample
More informationA : k n. Usually k > n otherwise easily the minimum is zero. Analytical solution:
1-5: Least-squares I A : k n. Usually k > n otherwise easily the minimum is zero. Analytical solution: f (x) =(Ax b) T (Ax b) =x T A T Ax 2b T Ax + b T b f (x) = 2A T Ax 2A T b = 0 Chih-Jen Lin (National
More informationMATH 409 Advanced Calculus I Lecture 7: Monotone sequences. The Bolzano-Weierstrass theorem.
MATH 409 Advanced Calculus I Lecture 7: Monotone sequences. The Bolzano-Weierstrass theorem. Limit of a sequence Definition. Sequence {x n } of real numbers is said to converge to a real number a if for
More informationExercises. Exercises. Basic terminology and optimality conditions. 4.2 Consider the optimization problem
Exercises Basic terminology and optimality conditions 4.1 Consider the optimization problem f 0(x 1, x 2) 2x 1 + x 2 1 x 1 + 3x 2 1 x 1 0, x 2 0. Make a sketch of the feasible set. For each of the following
More informationUnconstrained minimization: assumptions
Unconstrained minimization I terminology and assumptions I gradient descent method I steepest descent method I Newton s method I self-concordant functions I implementation IOE 611: Nonlinear Programming,
More informationExercise 1. Let f be a nonnegative measurable function. Show that. where ϕ is taken over all simple functions with ϕ f. k 1.
Real Variables, Fall 2014 Problem set 3 Solution suggestions xercise 1. Let f be a nonnegative measurable function. Show that f = sup ϕ, where ϕ is taken over all simple functions with ϕ f. For each n
More informationDefinition of convex function in a vector space
Convex functions Nonlinear optimization Instituto Superior Técnico and Carnegie Mellon University PhD course João Xavier TAs: Brian Swenson, Shanghang Zhang, Lucas Balthazar Convex functions Nonconvex
More information15. Conic optimization
L. Vandenberghe EE236C (Spring 216) 15. Conic optimization conic linear program examples modeling duality 15-1 Generalized (conic) inequalities Conic inequality: a constraint x K where K is a convex cone
More informationExamples of Dual Spaces from Measure Theory
Chapter 9 Examples of Dual Spaces from Measure Theory We have seen that L (, A, µ) is a Banach space for any measure space (, A, µ). We will extend that concept in the following section to identify an
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More informationConvex Optimization. Convex Analysis - Functions
Convex Optimization Convex Analsis - Functions p. 1 A function f : K R n R is convex, if K is a convex set and x, K,x, λ (,1) we have f(λx+(1 λ)) λf(x)+(1 λ)f(). (x, f(x)) (,f()) x - strictl convex,
More informationWinter Lecture 10. Convexity and Concavity
Andrew McLennan February 9, 1999 Economics 5113 Introduction to Mathematical Economics Winter 1999 Lecture 10 Convexity and Concavity I. Introduction A. We now consider convexity, concavity, and the general
More informationThe Erwin Schrodinger International Boltzmanngasse 9. Institute for Mathematical Physics A-1090 Wien, Austria
ESI The Erwin Schrodinger International Boltzmanngasse 9 Institute for Mathematical Physics A-090 Wien, Austria Lipschitz Image of a Measure{null Set Can Have a Null Complement J. Lindenstrauss E. Matouskova
More informationHandout 2: Elements of Convex Analysis
ENGG 5501: Foundations of Optimization 2018 19 First Term Handout 2: Elements of Convex Analysis Instructor: Anthony Man Cho So September 10, 2018 As briefly mentioned in Handout 1, the notion of convexity
More informationProbability and Random Processes
Probability and Random Processes Lecture 4 General integration theory Mikael Skoglund, Probability and random processes 1/15 Measurable xtended Real-valued Functions R = the extended real numbers; a subset
More information1. Sets A set is any collection of elements. Examples: - the set of even numbers between zero and the set of colors on the national flag.
San Francisco State University Math Review Notes Michael Bar Sets A set is any collection of elements Eamples: a A {,,4,6,8,} - the set of even numbers between zero and b B { red, white, bule} - the set
More information3.1 Convexity notions for functions and basic properties
3.1 Convexity notions for functions and basic properties We start the chapter with the basic definition of a convex function. Definition 3.1.1 (Convex function) A function f : E R is said to be convex
More informationSpringer-Verlag Berlin Heidelberg
SOME CHARACTERIZATIONS AND PROPERTIES OF THE \DISTANCE TO ILL-POSEDNESS" AND THE CONDITION MEASURE OF A CONIC LINEAR SYSTEM 1 Robert M. Freund 2 M.I.T. Jorge R. Vera 3 Catholic University of Chile October,
More informationON THE ARITHMETIC-GEOMETRIC MEAN INEQUALITY AND ITS RELATIONSHIP TO LINEAR PROGRAMMING, BAHMAN KALANTARI
ON THE ARITHMETIC-GEOMETRIC MEAN INEQUALITY AND ITS RELATIONSHIP TO LINEAR PROGRAMMING, MATRIX SCALING, AND GORDAN'S THEOREM BAHMAN KALANTARI Abstract. It is a classical inequality that the minimum of
More informationMath 172 Problem Set 5 Solutions
Math 172 Problem Set 5 Solutions 2.4 Let E = {(t, x : < x b, x t b}. To prove integrability of g, first observe that b b b f(t b b g(x dx = dt t dx f(t t dtdx. x Next note that f(t/t χ E is a measurable
More information1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics
1. Introduction mathematical optimization least-squares and linear programming convex optimization example course goals and topics nonlinear optimization brief history of convex optimization 1 1 Mathematical
More information1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics
1. Introduction Convex Optimization Boyd & Vandenberghe mathematical optimization least-squares and linear programming convex optimization example course goals and topics nonlinear optimization brief history
More information1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics
1. Introduction Convex Optimization Boyd & Vandenberghe mathematical optimization least-squares and linear programming convex optimization example course goals and topics nonlinear optimization brief history
More informationA notion of Total Dual Integrality for Convex, Semidefinite and Extended Formulations
A notion of for Convex, Semidefinite and Extended Formulations Marcel de Carli Silva Levent Tunçel April 26, 2018 A vector in R n is integral if each of its components is an integer, A vector in R n is
More information3 (Due ). Let A X consist of points (x, y) such that either x or y is a rational number. Is A measurable? What is its Lebesgue measure?
MA 645-4A (Real Analysis), Dr. Chernov Homework assignment 1 (Due ). Show that the open disk x 2 + y 2 < 1 is a countable union of planar elementary sets. Show that the closed disk x 2 + y 2 1 is a countable
More informationC.7. Numerical series. Pag. 147 Proof of the converging criteria for series. Theorem 5.29 (Comparison test) Let a k and b k be positive-term series
C.7 Numerical series Pag. 147 Proof of the converging criteria for series Theorem 5.29 (Comparison test) Let and be positive-term series such that 0, for any k 0. i) If the series converges, then also
More informationM17 MAT25-21 HOMEWORK 6
M17 MAT25-21 HOMEWORK 6 DUE 10:00AM WEDNESDAY SEPTEMBER 13TH 1. To Hand In Double Series. The exercises in this section will guide you to complete the proof of the following theorem: Theorem 1: Absolute
More informationStatic Problem Set 2 Solutions
Static Problem Set Solutions Jonathan Kreamer July, 0 Question (i) Let g, h be two concave functions. Is f = g + h a concave function? Prove it. Yes. Proof: Consider any two points x, x and α [0, ]. Let
More informationA : k n. Usually k > n otherwise easily the minimum is zero. Analytical solution:
1-5: Least-squares I A : k n. Usually k > n otherwise easily the minimum is zero. Analytical solution: f (x) =(Ax b) T (Ax b) =x T A T Ax 2b T Ax + b T b f (x) = 2A T Ax 2A T b = 0 Chih-Jen Lin (National
More informationEntrance Exam, Real Analysis September 1, 2017 Solve exactly 6 out of the 8 problems
September, 27 Solve exactly 6 out of the 8 problems. Prove by denition (in ɛ δ language) that f(x) = + x 2 is uniformly continuous in (, ). Is f(x) uniformly continuous in (, )? Prove your conclusion.
More information6.2 Fubini s Theorem. (µ ν)(c) = f C (x) dµ(x). (6.2) Proof. Note that (X Y, A B, µ ν) must be σ-finite as well, so that.
6.2 Fubini s Theorem Theorem 6.2.1. (Fubini s theorem - first form) Let (, A, µ) and (, B, ν) be complete σ-finite measure spaces. Let C = A B. Then for each µ ν- measurable set C C the section x C is
More informationOptimization Methods. Lecture 18: Optimality Conditions and. Gradient Methods. for Unconstrained Optimization
5.93 Optimization Methods Lecture 8: Optimality Conditions and Gradient Methods for Unconstrained Optimization Outline. Necessary and sucient optimality conditions Slide. Gradient m e t h o d s 3. The
More informationBellman function approach to the sharp constants in uniform convexity
Adv. Calc. Var. 08; (): 89 9 Research Article Paata Ivanisvili* Bellman function approach to the sharp constants in uniform convexity DOI: 0.55/acv-06-0008 Received February 9 06; revised May 5 06; accepted
More informationLecture: Duality of LP, SOCP and SDP
1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:
More informationDuality Uses and Correspondences. Ryan Tibshirani Convex Optimization
Duality Uses and Correspondences Ryan Tibshirani Conve Optimization 10-725 Recall that for the problem Last time: KKT conditions subject to f() h i () 0, i = 1,... m l j () = 0, j = 1,... r the KKT conditions
More informationIn English, this means that if we travel on a straight line between any two points in C, then we never leave C.
Convex sets In this section, we will be introduced to some of the mathematical fundamentals of convex sets. In order to motivate some of the definitions, we will look at the closest point problem from
More informationChapter 13. Convex and Concave. Josef Leydold Mathematical Methods WS 2018/19 13 Convex and Concave 1 / 44
Chapter 13 Convex and Concave Josef Leydold Mathematical Methods WS 2018/19 13 Convex and Concave 1 / 44 Monotone Function Function f is called monotonically increasing, if x 1 x 2 f (x 1 ) f (x 2 ) It
More information