Chapter 2: Preliminaries and elements of convex analysis
|
|
- Sherman Cook
- 5 years ago
- Views:
Transcription
1 Chapter 2: Preliminaries and elements of convex analysis Edoardo Amaldi DEIB Politecnico di Milano Website: Academic year Edoardo Amaldi (PoliMI) Optimization Academic year / 23
2 2.1 Basic concepts In R n with Euclidean norm x S R n is an interior point of S if ε > 0 such that B ε(x) = {y R n : y x < ε} S. x R n is a boundary point of S if, for every ε > 0, B ε(x) contains at least one point of S and one point of its complement R n \S. The set of all the interior points of S R n is the interior of S, denoted by int(s). The set of all boundary points of S is the boundary of S, denoted by (S). S R n is open if S = int(s); S is closed if its complement is open. Intuitively, a closed set contains all the points in (S). S R n is bounded if M > 0 such that x M for every x S. S R n closed and bounded is compact. Edoardo Amaldi (PoliMI) Optimization Academic year / 23
3 Property: A set S R n is closed if and only if every sequence {x i } i N S that converges, converges to x S. A set S R n is compact if and only if every sequence {x i } i N S admits a subsequence that converges to a point x S. Edoardo Amaldi (PoliMI) Optimization Academic year / 23
4 Existence of an optimal solution In general, when we wish to minimize a function f : S R n R, we only know that there exists a greatest lower bound (infimum), that is inf x S f(x) Theorem (Weierstrass): Let S R n be nonempty and compact set, and f : S R be continuous on S. Then there exists a x S such that f(x ) f(x) for every x S. Examples in which the result does not hold because: S is not closed, S is not bounded or f(x) is not continuous on S. Since the problem admits an optimal solution x S, we can write min x S f(x). Observation: this result holds in any vector space of finite dimension. Edoardo Amaldi (PoliMI) Optimization Academic year / 23
5 Cones and affine subspaces Consider any subset S R n Definition: cone(s) denotes the set of all the conic combinations of points of S, i.e., all the points x R n such that x = m i=1 α i x i with x 1,...,x m S and α i 0 per every i, 1 i m. Examples: polyedral cone generated by a finite number of vectors, ice cream cone in R 3 generated by an infinite number of vectors Definition: aff(s) denotes the smallest affine subspace that contains S. aff(s) coincides with the set of all the affine combinations of points in S, i.e., all the points x R n such that x = m i=1 α i x i with x 1,...,x m S and with m i=1 α i = 1, where α i R for every i, 1 i m. Examples: straight line containing two points in R 2, plane containing three points in general position in R 3 Edoardo Amaldi (PoliMI) Optimization Academic year / 23
6 2.2 Elements of convex analysis Definition: A set C R n is convex if αx 1 +(1 α)x 2 C x 1,x 2 C and α [0,1]. A point x R n is a convex combination of x 1,...,x m R n se x = m α i x i i=1 with α i 0 for every i, 1 i m, and m i=1 α i = 1. Property: If C i with i = 1,...,k are convex, then k i=1c i is convex. Edoardo Amaldi (PoliMI) Optimization Academic year / 23
7 Examples of convex sets 1) Hyperplane H = {x R n : p t x = β} with p 0. For x H, p t x = β implies H = {x R n : p t (x x) = 0} and hence H is the set of all the vectors in R n orthogonal to p. N.B.: H is closed since H = (H) 2) Closed half-spaces H + = {x R n : p t x β} and H = {x R n : p t x β} with p 0. 3) Feasible region X = {x R n : Ax b, x 0} of a Linear Program (LP) min c t x s.t. Ax b x 0 X is a convex and closed subset (intersection of m+n closed half-spaces if A R m n ). Edoardo Amaldi (PoliMI) Optimization Academic year / 23
8 Definition: The intersection of a finite number of closed half-spaces is a polyedron. N.B.: Also the set of optimal solutions of a LP is a polyhedron (just add c t x = z to the constraints, where z is the optimal value) and hence convex. Definition: The convex hull of S R n, denoted by conv(s), is the intersection of all convex sets containing S. conv(s) coincides with the set set of all convex combinations of points in S. Two equivalent characterizations (external/internal descriptions). Definition: Given a convex set C of R n, a point x C is an extreme point of C if it cannot be expressed as convex combination of two different points of C, that is implies that x 1 = x 2. x = αx 1 +(1 α)x 2 with x 1,x 2 C and α (0,1) Examples: convex sets with a finite and infinite number of extreme points. Edoardo Amaldi (PoliMI) Optimization Academic year / 23
9 Projection on a convex set Projection of a point on a subset to which it does not belong to. Lemma (Projection): Let C R n be a nonempty, closed and convex set, then for every y C there exists a unique x C at minimum distance from y. Moreover, x C is the closest point to y if and only if (y x ) t (x x ) 0 x C. Definition: x is the projection of y on C. Geometric illustration: Proof: Edoardo Amaldi (PoliMI) Optimization Academic year / 23
10 Separation theorem and consequences Geometrically intuitive but fundamental result. Theorem (Separating hyperplane) Let C R n be a nonempty, closed and convex set and y C, then there exists a p R n such that p t x < p t y for every x C. Thus there exists a hyperplane H = {x R n : p t x = β} with p 0 that separates y from C, i.e., such that C H = {x R n : p t x β} and y H (p t y > β) Geometric illustration: Proof: According to the Lemma, (y x ) t (x x ) 0 x C. Taking p = (y x ) 0 and β = (y x ) t x we have p t x β x C, while p t y β = (y x ) t (y x ) = y x 2 > 0 since y C. Edoardo Amaldi (PoliMI) Optimization Academic year / 23
11 Three important consequences: 1) Any nonempty, closed and convex set C R n is the intersection of all closed half-spaces containing it. Definition: Let S R n be nonempty and x (S) ( boundary w.r.t. aff(s) ), H = {x R n : p t (x x) = 0} is a supporting hyperplane of S at x if S H or S H +. 2) Supporting hyperplane: If C is a convex set in R n, then for every x (C) there exists a supporting hyperplane H at x, i.e., p 0 such that p t (x x) 0, for each x C. A convex set admits at least a supporting hyperplane at each boundary point. For nonconvex sets, such a hyperplane may not exist. Examples: cases with 1/ /0 supporting hyperplanes in a given boundary point x Proof sketch: Edoardo Amaldi (PoliMI) Optimization Academic year / 23
12 Central result of Optimization (also of Game theory) from which we will derive the optimality conditions for Nonlinear Programming. 3) Farkas Lemma: Let A R m n and b R m. Then x R n such that Ax = b and x 0 y R m such that y t A 0 t e y t b > 0. In this form, it provides an infeasibility certificate for a given linear system, but it is also known as the theorem of the alternative. Alternative: exactly one of the two systems Ax = b,x 0 and y t A 0 t,y t b > 0 is feasible. Geometric interpretation: b belongs to the convex cone generated by the columns A 1,...,A n of A, i.e., to cone(a)= {z R m : z = n j=1 α j A j,α 1 0,...,α n 0}, if and only if no hyperplane separating b from cone(a) exists. Alternative: b cone(a) or b cone(a) (hence hyperplane separating b from cone(a)) Edoardo Amaldi (PoliMI) Optimization Academic year / 23
13 Proof: Edoardo Amaldi (PoliMI) Optimization Academic year / 23
14 Application of Farkas Lemma: asset pricing in absence of arbitrage Single period, m assets are traded, n possible states (scenarios). Assets can be bought or sold short, i.e., with the promise to buy them back at the end. Portfolio y R m, with y i = amount invested in asset i. A negative value of y i indicates a short position. If p i is the price of a unit of asset i at the beginning, the portfolio cost is: p t y Consider A R m n with a ij = value of one euro invested in asset i if state j occurs. At the end all assets are sold (we receive a payoff of a ij y i ) and the short positions are covered (we pay a ij y i and hence have a payoff of a ij y i ). Portfolio value: v t = y t A Absence of arbitrage condition: any portfolio with nonnegative values for all states must have a nonnegative cost. Algebraically: y such that y t A 0 t and p t y < 0. Edoardo Amaldi (PoliMI) Optimization Academic year / 23
15 According to Farkas Lemma: No possibility of arbitrage exist ( y such that y t A 0 t and p t y < 0 ) if and only if q R n with q 0 such that the asset price vector p satisfies Aq = p. If the market is efficient, some state prices q i exist. N.B.: in general q is not unique. Edoardo Amaldi (PoliMI) Optimization Academic year / 23
16 2.2.2 Convex functions Definitions: A function f : C R defined on a convex set C R n is convex if f(αx 1 +(1 α)x 2 ) αf(x 1 )+(1 α)f(x 2 ) x 1,x 2 C and α [0,1], f is strictly convex if the inequality holds with < for x 1,x 2 C with x 1 x 2 and α (0,1). f is concave if f is convex. The epigraph of f : S R n R, denoted by epi(f), is the subset of R n+1 epi(f) = {(x,y) S R : f(x) y}. Let f : C R be convex, the domain of f is the subset of R n dom(f) = {x C : f(x) < + }. Edoardo Amaldi (PoliMI) Optimization Academic year / 23
17 Property: Let C R n be a (nonempty) convex set and f : C R be a convex function. For each real β (also for + ), the level set L β = {x C : f(x) β} and {x C : f(x) < β} is a convex subset of R n. The function f is continuous in the relative interior (with respect to aff(c)) of its domain. The function f is convex if and only if epi(f) is a convex subset of R n+1 (exercise 1.3). Edoardo Amaldi (PoliMI) Optimization Academic year / 23
18 Optimal solution of convex problems Consider min x C R n f(x) where C R n is a convex set and f is a convex function. Proposition: If C R n is convex and f : C R is convex, each local minimum of f on C is a global minimum. If f is strictly convex on C, then there exists at most one global minimum (the problem could be unbounded). Proof: Suppose x is a local minimum and x C such that f(x ) < f(x ). Since f is convex f(αx +(1 α)x ) αf(x )+(1 α)f(x ) < f(x ) α (0,1) contradicts the fact that x is a local minimum. If f is strictly convex and x 1 and x 2 are two global minima, the convexity of C implies 1 2 (x 1 +x 2 ) C and strict convexity of f implies f( 1 2 (x 1 +x 2 )) < 1 2 f(x 1 )+ 1 2 f(x 2 ). Thus x 1 and x 2 cannot be two global minima. Edoardo Amaldi (PoliMI) Optimization Academic year / 23
19 Characterizations of convex functions 1) Proposition: A continuously differentiable function (of class C 1 ) f : C R defined on an open and nonempty convex set C R n is convex if and only if f(x) f(x)+ t f(x)(x x) x,x C. f is strictly convex if and only if the inequality holds with > for every pair x,x C with x x. Definition: Directional derivative of f: lim α 0 + f(x+α(x x)) f(x) α = t f(x)(x x) Geometric interpretation: The linear approximation of f at x (1st order Taylor s expansion) bounds below f(x) and ( ) ( ) x H = { R n+1 : ( t x f(x) 1) = f(x)+ t f(x) x } y y is a supporting hyperplane of epi(f) in (x,f(x)), with epi(f) H. Edoardo Amaldi (PoliMI) Optimization Academic year / 23
20 2) Proposition: A twice continuously differentiable function (of class C 2 ) f : C R defined on an open and nonempty convex set C R n is convex if and only if the Hessian matrix 2 f(x) = ( 2 f x i x j ) is positive semidefinite at every x S. For C 2 functions, if 2 f(x) is positive definite x C then f(x) is strictly convex. N.B.: This condition is sufficient but not necessary: f(x) = x 4 is strictly convex but f (0) = 0. Definition: A symmetric matrix A n n is positive definite if y t Ay > 0 y R n with y 0, A symmetric matrix A n n is positive semidefinite if y t Ay 0 y R n. Equivalent definitions: based on the sign of the eigenvalues/principal minors of A or of the diagonal coefficients of specific factorizations of A (e.g., Cholesky factorization). Edoardo Amaldi (PoliMI) Optimization Academic year / 23
21 Subgradient of convex and concave functions Convex/concave (continuous) functions that are not everywhere differentiable, e.g. f(x) = x. Generalization of the concept of gradient for C 1 functions to piecewise C 1 functions. Definitions: Let C R n be a convex set and f : C R be a convex function on C a vector γ R n is a subgradient of f at x C if f(x) f(x)+γ t (x x) x C, the subdifferential, denoted by f(x), is the set of all the subgradients of f at x. Example: For f(x) = x 2, in x = 3 the only subgradient is γ = 6. Indeed, 0 (x 3) 2 = x 2 6x +9 implies for every x: f(x) = x 2 6x 9 = 9+6(x 3) = f(x)+6(x x) Edoardo Amaldi (PoliMI) Optimization Academic year / 23
22 Other examples: 1) For f(x) = x it is clear that: γ = 1 if x > 0, γ = 1 if x < 0, f(x) = [ 1,1] if x = 0. 2) Consider f(x) = min{f 1(x),f 2(x)} with f 1(x) = 4 x and f 2(x) = 4 (x 2) 2. Since f 2(x) f 1(x) for 1 x 4, { 4 x 1 x 4 f(x) = 4 (x 2) 2 otherwise γ = 1 for x (1,4), γ = 2(x 2) for x < 1 or x > 4, γ [ 1,2] at x = 1, γ [ 4, 1] at x = 4. Edoardo Amaldi (PoliMI) Optimization Academic year / 23
23 Properties: 1) A convex function f : C R admits at least a subgradient at every interior point x of C. In particular, if x int(c) then there exists γ R n such that H = {(x,y) R n+1 : y = f(x)+γ t (x x)} is a supporting hyperplane of epi(f) at (x,f(x)). N.B.: The existence of (at least) a subgradient at every point of int(c), with C convex, is a necessary and sufficient condition for f to be convex on int(c). 2) If f is a convex function and x C, f(x) is a nonempty, convex, closed and bounded set. 3) x is a (global) minimum of a convex function f : C R if and only if 0 f(x ). Edoardo Amaldi (PoliMI) Optimization Academic year / 23
Chapter 4: Unconstrained nonlinear optimization
Chapter 4: Unconstrained nonlinear optimization Edoardo Amaldi DEIB Politecnico di Milano edoardo.amaldi@polimi.it Website: http://home.deib.polimi.it/amaldi/opt-15-16.shtml Academic year 2015-16 Edoardo
More informationOptimality Conditions for Nonsmooth Convex Optimization
Optimality Conditions for Nonsmooth Convex Optimization Sangkyun Lee Oct 22, 2014 Let us consider a convex function f : R n R, where R is the extended real field, R := R {, + }, which is proper (f never
More informationLecture 6 - Convex Sets
Lecture 6 - Convex Sets Definition A set C R n is called convex if for any x, y C and λ [0, 1], the point λx + (1 λ)y belongs to C. The above definition is equivalent to saying that for any x, y C, the
More informationSWFR ENG 4TE3 (6TE3) COMP SCI 4TE3 (6TE3) Continuous Optimization Algorithm. Convex Optimization. Computing and Software McMaster University
SWFR ENG 4TE3 (6TE3) COMP SCI 4TE3 (6TE3) Continuous Optimization Algorithm Convex Optimization Computing and Software McMaster University General NLO problem (NLO : Non Linear Optimization) (N LO) min
More informationLECTURE 4 LECTURE OUTLINE
LECTURE 4 LECTURE OUTLINE Relative interior and closure Algebra of relative interiors and closures Continuity of convex functions Closures of functions Reading: Section 1.3 All figures are courtesy of
More informationIntroduction to Convex and Quasiconvex Analysis
Introduction to Convex and Quasiconvex Analysis J.B.G.Frenk Econometric Institute, Erasmus University, Rotterdam G.Kassay Faculty of Mathematics, Babes Bolyai University, Cluj August 27, 2001 Abstract
More informationLecture 5. Theorems of Alternatives and Self-Dual Embedding
IE 8534 1 Lecture 5. Theorems of Alternatives and Self-Dual Embedding IE 8534 2 A system of linear equations may not have a solution. It is well known that either Ax = c has a solution, or A T y = 0, c
More information3.10 Lagrangian relaxation
3.10 Lagrangian relaxation Consider a generic ILP problem min {c t x : Ax b, Dx d, x Z n } with integer coefficients. Suppose Dx d are the complicating constraints. Often the linear relaxation and the
More informationOptimization and Optimal Control in Banach Spaces
Optimization and Optimal Control in Banach Spaces Bernhard Schmitzer October 19, 2017 1 Convex non-smooth optimization with proximal operators Remark 1.1 (Motivation). Convex optimization: easier to solve,
More informationChapter 2 Convex Analysis
Chapter 2 Convex Analysis The theory of nonsmooth analysis is based on convex analysis. Thus, we start this chapter by giving basic concepts and results of convexity (for further readings see also [202,
More information4. Algebra and Duality
4-1 Algebra and Duality P. Parrilo and S. Lall, CDC 2003 2003.12.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone
More informationConvex Optimization Theory. Chapter 5 Exercises and Solutions: Extended Version
Convex Optimization Theory Chapter 5 Exercises and Solutions: Extended Version Dimitri P. Bertsekas Massachusetts Institute of Technology Athena Scientific, Belmont, Massachusetts http://www.athenasc.com
More informationCO 250 Final Exam Guide
Spring 2017 CO 250 Final Exam Guide TABLE OF CONTENTS richardwu.ca CO 250 Final Exam Guide Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4,
More informationLECTURE 12 LECTURE OUTLINE. Subgradients Fenchel inequality Sensitivity in constrained optimization Subdifferential calculus Optimality conditions
LECTURE 12 LECTURE OUTLINE Subgradients Fenchel inequality Sensitivity in constrained optimization Subdifferential calculus Optimality conditions Reading: Section 5.4 All figures are courtesy of Athena
More informationAppendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS
Appendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS Here we consider systems of linear constraints, consisting of equations or inequalities or both. A feasible solution
More informationLECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE
LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE CONVEX ANALYSIS AND DUALITY Basic concepts of convex analysis Basic concepts of convex optimization Geometric duality framework - MC/MC Constrained optimization
More informationConvex Functions. Pontus Giselsson
Convex Functions Pontus Giselsson 1 Today s lecture lower semicontinuity, closure, convex hull convexity preserving operations precomposition with affine mapping infimal convolution image function supremum
More informationConvex Analysis Background
Convex Analysis Background John C. Duchi Stanford University Park City Mathematics Institute 206 Abstract In this set of notes, we will outline several standard facts from convex analysis, the study of
More informationIn English, this means that if we travel on a straight line between any two points in C, then we never leave C.
Convex sets In this section, we will be introduced to some of the mathematical fundamentals of convex sets. In order to motivate some of the definitions, we will look at the closest point problem from
More informationHandout 2: Elements of Convex Analysis
ENGG 5501: Foundations of Optimization 2018 19 First Term Handout 2: Elements of Convex Analysis Instructor: Anthony Man Cho So September 10, 2018 As briefly mentioned in Handout 1, the notion of convexity
More informationConvex Analysis and Optimization Chapter 2 Solutions
Convex Analysis and Optimization Chapter 2 Solutions Dimitri P. Bertsekas with Angelia Nedić and Asuman E. Ozdaglar Massachusetts Institute of Technology Athena Scientific, Belmont, Massachusetts http://www.athenasc.com
More informationConvex Optimization and Modeling
Convex Optimization and Modeling Duality Theory and Optimality Conditions 5th lecture, 12.05.2010 Jun.-Prof. Matthias Hein Program of today/next lecture Lagrangian and duality: the Lagrangian the dual
More informationSummer School: Semidefinite Optimization
Summer School: Semidefinite Optimization Christine Bachoc Université Bordeaux I, IMB Research Training Group Experimental and Constructive Algebra Haus Karrenberg, Sept. 3 - Sept. 7, 2012 Duality Theory
More informationAppendix B Convex analysis
This version: 28/02/2014 Appendix B Convex analysis In this appendix we review a few basic notions of convexity and related notions that will be important for us at various times. B.1 The Hausdorff distance
More informationConvex Functions and Optimization
Chapter 5 Convex Functions and Optimization 5.1 Convex Functions Our next topic is that of convex functions. Again, we will concentrate on the context of a map f : R n R although the situation can be generalized
More informationLMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009
LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix
More informationConvex Optimization Theory
Convex Optimization Theory A SUMMARY BY DIMITRI P. BERTSEKAS We provide a summary of theoretical concepts and results relating to convex analysis, convex optimization, and duality theory. In particular,
More informationIntroduction to Convex Analysis Microeconomics II - Tutoring Class
Introduction to Convex Analysis Microeconomics II - Tutoring Class Professor: V. Filipe Martins-da-Rocha TA: Cinthia Konichi April 2010 1 Basic Concepts and Results This is a first glance on basic convex
More informationStructural and Multidisciplinary Optimization. P. Duysinx and P. Tossings
Structural and Multidisciplinary Optimization P. Duysinx and P. Tossings 2018-2019 CONTACTS Pierre Duysinx Institut de Mécanique et du Génie Civil (B52/3) Phone number: 04/366.91.94 Email: P.Duysinx@uliege.be
More information5.5 Quadratic programming
5.5 Quadratic programming Minimize a quadratic function subject to linear constraints: 1 min x t Qx + c t x 2 s.t. a t i x b i i I (P a t i x = b i i E x R n, where Q is an n n matrix, I and E are the
More informationLECTURE SLIDES ON BASED ON CLASS LECTURES AT THE CAMBRIDGE, MASS FALL 2007 BY DIMITRI P. BERTSEKAS.
LECTURE SLIDES ON CONVEX ANALYSIS AND OPTIMIZATION BASED ON 6.253 CLASS LECTURES AT THE MASSACHUSETTS INSTITUTE OF TECHNOLOGY CAMBRIDGE, MASS FALL 2007 BY DIMITRI P. BERTSEKAS http://web.mit.edu/dimitrib/www/home.html
More informationZero sum games Proving the vn theorem. Zero sum games. Roberto Lucchetti. Politecnico di Milano
Politecnico di Milano General form Definition A two player zero sum game in strategic form is the triplet (X, Y, f : X Y R) f (x, y) is what Pl1 gets from Pl2, when they play x, y respectively. Thus g
More informationExtreme Abridgment of Boyd and Vandenberghe s Convex Optimization
Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The
More informationConstrained Optimization Theory
Constrained Optimization Theory Stephen J. Wright 1 2 Computer Sciences Department, University of Wisconsin-Madison. IMA, August 2016 Stephen Wright (UW-Madison) Constrained Optimization Theory IMA, August
More information4. Convex Sets and (Quasi-)Concave Functions
4. Convex Sets and (Quasi-)Concave Functions Daisuke Oyama Mathematics II April 17, 2017 Convex Sets Definition 4.1 A R N is convex if (1 α)x + αx A whenever x, x A and α [0, 1]. A R N is strictly convex
More informationMath 341: Convex Geometry. Xi Chen
Math 341: Convex Geometry Xi Chen 479 Central Academic Building, University of Alberta, Edmonton, Alberta T6G 2G1, CANADA E-mail address: xichen@math.ualberta.ca CHAPTER 1 Basics 1. Euclidean Geometry
More informationLECTURE SLIDES ON CONVEX OPTIMIZATION AND DUALITY THEORY TATA INSTITUTE FOR FUNDAMENTAL RESEARCH MUMBAI, INDIA JANUARY 2009 PART I
LECTURE SLIDES ON CONVEX OPTIMIZATION AND DUALITY THEORY TATA INSTITUTE FOR FUNDAMENTAL RESEARCH MUMBAI, INDIA JANUARY 2009 PART I BY DIMITRI P. BERTSEKAS M.I.T. http://web.mit.edu/dimitrib/www/home.html
More informationConvex Analysis and Optimization Chapter 4 Solutions
Convex Analysis and Optimization Chapter 4 Solutions Dimitri P. Bertsekas with Angelia Nedić and Asuman E. Ozdaglar Massachusetts Institute of Technology Athena Scientific, Belmont, Massachusetts http://www.athenasc.com
More informationCSCI : Optimization and Control of Networks. Review on Convex Optimization
CSCI7000-016: Optimization and Control of Networks Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one
More informationIntroduction and Math Preliminaries
Introduction and Math Preliminaries Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Appendices A, B, and C, Chapter
More informationA Brief Review on Convex Optimization
A Brief Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one convex, two nonconvex sets): A Brief Review
More informationLINEAR PROGRAMMING II
LINEAR PROGRAMMING II LP duality strong duality theorem bonus proof of LP duality applications Lecture slides by Kevin Wayne Last updated on 7/25/17 11:09 AM LINEAR PROGRAMMING II LP duality Strong duality
More informationLecture 1: Introduction. Outline. B9824 Foundations of Optimization. Fall Administrative matters. 2. Introduction. 3. Existence of optima
B9824 Foundations of Optimization Lecture 1: Introduction Fall 2009 Copyright 2009 Ciamac Moallemi Outline 1. Administrative matters 2. Introduction 3. Existence of optima 4. Local theory of unconstrained
More informationConvex Sets. Prof. Dan A. Simovici UMB
Convex Sets Prof. Dan A. Simovici UMB 1 / 57 Outline 1 Closures, Interiors, Borders of Sets in R n 2 Segments and Convex Sets 3 Properties of the Class of Convex Sets 4 Closure and Interior Points of Convex
More informationOPTIMIZATION. joint course with. Ottimizzazione Discreta and Complementi di R.O. Edoardo Amaldi. DEIB Politecnico di Milano
OPTIMIZATION joint course with Ottimizzazione Discreta and Complementi di R.O. Edoardo Amaldi DEIB Politecnico di Milano edoardo.amaldi@polimi.it Website: http://home.deib.polimi.it/amaldi/opt-15-16.shtml
More informationEC9A0: Pre-sessional Advanced Mathematics Course. Lecture Notes: Unconstrained Optimisation By Pablo F. Beker 1
EC9A0: Pre-sessional Advanced Mathematics Course Lecture Notes: Unconstrained Optimisation By Pablo F. Beker 1 1 Infimum and Supremum Definition 1. Fix a set Y R. A number α R is an upper bound of Y if
More informationConvex Optimization and an Introduction to Congestion Control. Lecture Notes. Fabian Wirth
Convex Optimization and an Introduction to Congestion Control Lecture Notes Fabian Wirth August 29, 2012 ii Contents 1 Convex Sets and Convex Functions 3 1.1 Convex Sets....................................
More informationCHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS. W. Erwin Diewert January 31, 2008.
1 ECONOMICS 594: LECTURE NOTES CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS W. Erwin Diewert January 31, 2008. 1. Introduction Many economic problems have the following structure: (i) a linear function
More informationLecture Note 5: Semidefinite Programming for Stability Analysis
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State
More informationMATH 5720: Unconstrained Optimization Hung Phan, UMass Lowell September 13, 2018
MATH 57: Unconstrained Optimization Hung Phan, UMass Lowell September 13, 18 1 Global and Local Optima Let a function f : S R be defined on a set S R n Definition 1 (minimizers and maximizers) (i) x S
More informationJanuary 29, Introduction to optimization and complexity. Outline. Introduction. Problem formulation. Convexity reminder. Optimality Conditions
Olga Galinina olga.galinina@tut.fi ELT-53656 Network Analysis Dimensioning II Department of Electronics Communications Engineering Tampere University of Technology, Tampere, Finl January 29, 2014 1 2 3
More informationChapter 1. Preliminaries
Introduction This dissertation is a reading of chapter 4 in part I of the book : Integer and Combinatorial Optimization by George L. Nemhauser & Laurence A. Wolsey. The chapter elaborates links between
More informationExample: feasibility. Interpretation as formal proof. Example: linear inequalities and Farkas lemma
4-1 Algebra and Duality P. Parrilo and S. Lall 2006.06.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone of valid
More informationMathematics 530. Practice Problems. n + 1 }
Department of Mathematical Sciences University of Delaware Prof. T. Angell October 19, 2015 Mathematics 530 Practice Problems 1. Recall that an indifference relation on a partially ordered set is defined
More informationDivision of the Humanities and Social Sciences. Supergradients. KC Border Fall 2001 v ::15.45
Division of the Humanities and Social Sciences Supergradients KC Border Fall 2001 1 The supergradient of a concave function There is a useful way to characterize the concavity of differentiable functions.
More informationMathematical Background
A Mathematical Background Version: date: June 28, 2018 Copyright c 2018 by Nob Hill Publishing, LLC A.1 Introduction In this appendix we give a brief review of some concepts that we need. It is assumed
More informationCOURSE ON LMI PART I.2 GEOMETRY OF LMI SETS. Didier HENRION henrion
COURSE ON LMI PART I.2 GEOMETRY OF LMI SETS Didier HENRION www.laas.fr/ henrion October 2006 Geometry of LMI sets Given symmetric matrices F i we want to characterize the shape in R n of the LMI set F
More informationLecture 1: Introduction. Outline. B9824 Foundations of Optimization. Fall Administrative matters. 2. Introduction. 3. Existence of optima
B9824 Foundations of Optimization Lecture 1: Introduction Fall 2010 Copyright 2010 Ciamac Moallemi Outline 1. Administrative matters 2. Introduction 3. Existence of optima 4. Local theory of unconstrained
More informationShiqian Ma, MAT-258A: Numerical Optimization 1. Chapter 4. Subgradient
Shiqian Ma, MAT-258A: Numerical Optimization 1 Chapter 4 Subgradient Shiqian Ma, MAT-258A: Numerical Optimization 2 4.1. Subgradients definition subgradient calculus duality and optimality conditions Shiqian
More information6.254 : Game Theory with Engineering Applications Lecture 7: Supermodular Games
6.254 : Game Theory with Engineering Applications Lecture 7: Asu Ozdaglar MIT February 25, 2010 1 Introduction Outline Uniqueness of a Pure Nash Equilibrium for Continuous Games Reading: Rosen J.B., Existence
More informationHW1 solutions. 1. α Ef(x) β, where Ef(x) is the expected value of f(x), i.e., Ef(x) = n. i=1 p if(a i ). (The function f : R R is given.
HW1 solutions Exercise 1 (Some sets of probability distributions.) Let x be a real-valued random variable with Prob(x = a i ) = p i, i = 1,..., n, where a 1 < a 2 < < a n. Of course p R n lies in the standard
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Notes for EE7C (Spring 018): Convex Optimization and Approximation Instructor: Moritz Hardt Email: hardt+ee7c@berkeley.edu Graduate Instructor: Max Simchowitz Email: msimchow+ee7c@berkeley.edu February
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More informationLecture 2: Convex Sets and Functions
Lecture 2: Convex Sets and Functions Hyang-Won Lee Dept. of Internet & Multimedia Eng. Konkuk University Lecture 2 Network Optimization, Fall 2015 1 / 22 Optimization Problems Optimization problems are
More informationConvex Sets with Applications to Economics
Convex Sets with Applications to Economics Debasis Mishra March 10, 2010 1 Convex Sets A set C R n is called convex if for all x, y C, we have λx+(1 λ)y C for all λ [0, 1]. The definition says that for
More informationSome Properties of the Augmented Lagrangian in Cone Constrained Optimization
MATHEMATICS OF OPERATIONS RESEARCH Vol. 29, No. 3, August 2004, pp. 479 491 issn 0364-765X eissn 1526-5471 04 2903 0479 informs doi 10.1287/moor.1040.0103 2004 INFORMS Some Properties of the Augmented
More informationLecture 1: Background on Convex Analysis
Lecture 1: Background on Convex Analysis John Duchi PCMI 2016 Outline I Convex sets 1.1 Definitions and examples 2.2 Basic properties 3.3 Projections onto convex sets 4.4 Separating and supporting hyperplanes
More informationAssignment 1: From the Definition of Convexity to Helley Theorem
Assignment 1: From the Definition of Convexity to Helley Theorem Exercise 1 Mark in the following list the sets which are convex: 1. {x R 2 : x 1 + i 2 x 2 1, i = 1,..., 10} 2. {x R 2 : x 2 1 + 2ix 1x
More informationWhat can be expressed via Conic Quadratic and Semidefinite Programming?
What can be expressed via Conic Quadratic and Semidefinite Programming? A. Nemirovski Faculty of Industrial Engineering and Management Technion Israel Institute of Technology Abstract Tremendous recent
More informationCONSTRAINT QUALIFICATIONS, LAGRANGIAN DUALITY & SADDLE POINT OPTIMALITY CONDITIONS
CONSTRAINT QUALIFICATIONS, LAGRANGIAN DUALITY & SADDLE POINT OPTIMALITY CONDITIONS A Dissertation Submitted For The Award of the Degree of Master of Philosophy in Mathematics Neelam Patel School of Mathematics
More informationLecture 7 Monotonicity. September 21, 2008
Lecture 7 Monotonicity September 21, 2008 Outline Introduce several monotonicity properties of vector functions Are satisfied immediately by gradient maps of convex functions In a sense, role of monotonicity
More informationFunctions of Several Variables
Jim Lambers MAT 419/519 Summer Session 2011-12 Lecture 2 Notes These notes correspond to Section 1.2 in the text. Functions of Several Variables We now generalize the results from the previous section,
More informationGEORGIA INSTITUTE OF TECHNOLOGY H. MILTON STEWART SCHOOL OF INDUSTRIAL AND SYSTEMS ENGINEERING LECTURE NOTES OPTIMIZATION III
GEORGIA INSTITUTE OF TECHNOLOGY H. MILTON STEWART SCHOOL OF INDUSTRIAL AND SYSTEMS ENGINEERING LECTURE NOTES OPTIMIZATION III CONVEX ANALYSIS NONLINEAR PROGRAMMING THEORY NONLINEAR PROGRAMMING ALGORITHMS
More informationInequality Constraints
Chapter 2 Inequality Constraints 2.1 Optimality Conditions Early in multivariate calculus we learn the significance of differentiability in finding minimizers. In this section we begin our study of the
More informationMAT 419 Lecture Notes Transcribed by Eowyn Cenek 6/1/2012
(Homework 1: Chapter 1: Exercises 1-7, 9, 11, 19, due Monday June 11th See also the course website for lectures, assignments, etc) Note: today s lecture is primarily about definitions Lots of definitions
More informationLecture 1: Convex Sets January 23
IE 521: Convex Optimization Instructor: Niao He Lecture 1: Convex Sets January 23 Spring 2017, UIUC Scribe: Niao He Courtesy warning: These notes do not necessarily cover everything discussed in the class.
More informationConic Linear Programming. Yinyu Ye
Conic Linear Programming Yinyu Ye December 2004, revised October 2017 i ii Preface This monograph is developed for MS&E 314, Conic Linear Programming, which I am teaching at Stanford. Information, lecture
More informationA Geometric Framework for Nonconvex Optimization Duality using Augmented Lagrangian Functions
A Geometric Framework for Nonconvex Optimization Duality using Augmented Lagrangian Functions Angelia Nedić and Asuman Ozdaglar April 15, 2006 Abstract We provide a unifying geometric framework for the
More informationConic Linear Programming. Yinyu Ye
Conic Linear Programming Yinyu Ye December 2004, revised January 2015 i ii Preface This monograph is developed for MS&E 314, Conic Linear Programming, which I am teaching at Stanford. Information, lecture
More informationIE 521 Convex Optimization
Lecture 1: 16th January 2019 Outline 1 / 20 Which set is different from others? Figure: Four sets 2 / 20 Which set is different from others? Figure: Four sets 3 / 20 Interior, Closure, Boundary Definition.
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationChapter 1. Preliminaries. The purpose of this chapter is to provide some basic background information. Linear Space. Hilbert Space.
Chapter 1 Preliminaries The purpose of this chapter is to provide some basic background information. Linear Space Hilbert Space Basic Principles 1 2 Preliminaries Linear Space The notion of linear space
More informationProblem 1 (Exercise 2.2, Monograph)
MS&E 314/CME 336 Assignment 2 Conic Linear Programming January 3, 215 Prof. Yinyu Ye 6 Pages ASSIGNMENT 2 SOLUTIONS Problem 1 (Exercise 2.2, Monograph) We prove the part ii) of Theorem 2.1 (Farkas Lemma
More informationAN INTRODUCTION TO CONVEXITY
AN INTRODUCTION TO CONVEXITY GEIR DAHL NOVEMBER 2010 University of Oslo, Centre of Mathematics for Applications, P.O.Box 1053, Blindern, 0316 Oslo, Norway (geird@math.uio.no) Contents 1 The basic concepts
More informationLinear Programming. Larry Blume Cornell University, IHS Vienna and SFI. Summer 2016
Linear Programming Larry Blume Cornell University, IHS Vienna and SFI Summer 2016 These notes derive basic results in finite-dimensional linear programming using tools of convex analysis. Most sources
More informationLECTURE SLIDES ON CONVEX ANALYSIS AND OPTIMIZATION BASED ON LECTURES GIVEN AT THE MASSACHUSETTS INSTITUTE OF TECHNOLOGY CAMBRIDGE, MASS
LECTURE SLIDES ON CONVEX ANALYSIS AND OPTIMIZATION BASED ON LECTURES GIVEN AT THE MASSACHUSETTS INSTITUTE OF TECHNOLOGY CAMBRIDGE, MASS BY DIMITRI P. BERTSEKAS http://web.mit.edu/dimitrib/www/home.html
More informationLECTURE 10 LECTURE OUTLINE
LECTURE 10 LECTURE OUTLINE Min Common/Max Crossing Th. III Nonlinear Farkas Lemma/Linear Constraints Linear Programming Duality Convex Programming Duality Optimality Conditions Reading: Sections 4.5, 5.1,5.2,
More information4.6 Linear Programming duality
4.6 Linear Programming duality To any minimization (maximization) LP we can associate a closely related maximization (minimization) LP Different spaces and objective functions but in general same optimal
More informationConvex Feasibility Problems
Laureate Prof. Jonathan Borwein with Matthew Tam http://carma.newcastle.edu.au/drmethods/paseky.html Spring School on Variational Analysis VI Paseky nad Jizerou, April 19 25, 2015 Last Revised: May 6,
More informationMAT-INF4110/MAT-INF9110 Mathematical optimization
MAT-INF4110/MAT-INF9110 Mathematical optimization Geir Dahl August 20, 2013 Convexity Part IV Chapter 4 Representation of convex sets different representations of convex sets, boundary polyhedra and polytopes:
More information1 Directional Derivatives and Differentiability
Wednesday, January 18, 2012 1 Directional Derivatives and Differentiability Let E R N, let f : E R and let x 0 E. Given a direction v R N, let L be the line through x 0 in the direction v, that is, L :=
More informationIntroduction to Machine Learning Lecture 7. Mehryar Mohri Courant Institute and Google Research
Introduction to Machine Learning Lecture 7 Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu Convex Optimization Differentiation Definition: let f : X R N R be a differentiable function,
More informationConvexity, Duality, and Lagrange Multipliers
LECTURE NOTES Convexity, Duality, and Lagrange Multipliers Dimitri P. Bertsekas with assistance from Angelia Geary-Nedic and Asuman Koksal Massachusetts Institute of Technology Spring 2001 These notes
More informationLP Duality: outline. Duality theory for Linear Programming. alternatives. optimization I Idea: polyhedra
LP Duality: outline I Motivation and definition of a dual LP I Weak duality I Separating hyperplane theorem and theorems of the alternatives I Strong duality and complementary slackness I Using duality
More informationAnalysis and Linear Algebra. Lectures 1-3 on the mathematical tools that will be used in C103
Analysis and Linear Algebra Lectures 1-3 on the mathematical tools that will be used in C103 Set Notation A, B sets AcB union A1B intersection A\B the set of objects in A that are not in B N. Empty set
More informationI.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More informationLecture 8. Strong Duality Results. September 22, 2008
Strong Duality Results September 22, 2008 Outline Lecture 8 Slater Condition and its Variations Convex Objective with Linear Inequality Constraints Quadratic Objective over Quadratic Constraints Representation
More informationConvex analysis and profit/cost/support functions
Division of the Humanities and Social Sciences Convex analysis and profit/cost/support functions KC Border October 2004 Revised January 2009 Let A be a subset of R m Convex analysts may give one of two
More informationOn the convexity of piecewise-defined functions
On the convexity of piecewise-defined functions arxiv:1408.3771v1 [math.ca] 16 Aug 2014 Heinz H. Bauschke, Yves Lucet, and Hung M. Phan August 16, 2014 Abstract Functions that are piecewise defined are
More informationCHAPTER 4: Convex Sets and Separation Theorems
CHAPTER 4: Convex Sets and Separation Theorems 1 Convex sets Convexity is often assumed in economic theory since it plays important roles in optimization. The convexity of preferences can be interpreted
More information