Lecture 6 - Convex Sets
|
|
- Melvin McLaughlin
- 5 years ago
- Views:
Transcription
1 Lecture 6 - Convex Sets Definition A set C R n is called convex if for any x, y C and λ [0, 1], the point λx + (1 λ)y belongs to C. The above definition is equivalent to saying that for any x, y C, the line segment [x, y] is also in C. convex sets nonconvex sets Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 1 / 32
2 Examples of Convex Sets Lines: A line in R n is a set of the form where z, d R n and d 0. [x, y], (x, y) for x, y R n (x y)., R n. A hyperplane is a set of the form L = {z + td : t R}, H = {x R n : a T x = b} (a R n \{0}, b R) The associated half-space is the set H = {x R n : a T x b} Both hyperplanes and half-spaces are convex sets. Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 2 / 32
3 Convexity of Balls Lemma. Let c R n and r > 0. Then the open ball and the closed ball are convex. B(c, r) = {x R n : x c < r} B[c, r] = {x R n : x c r} Note that the norm is an arbitrary norm defined over R n. Proof. In class Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 3 / 32
4 l 1, l 2 and l balls l 1 l l Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 4 / 32
5 Convexity of Ellipsoids An ellipsoid is a set of the form E = {x R n : x T Qx + 2b T x + c 0}, where Q R n n is positive semidefinite, b R n and c R. Proof. Lemma: E is convex. Write E as E = {x R n : f (x) 0} where f (x) x T Qx + 2b T x + c. Take x, y E and λ [0, 1]. Then f (x) 0, f (y) 0. The vector z = λx + (1 λ)y satisfies z T Qz = λ 2 x T Qx + (1 λ) 2 y T Qy + 2λ(1 λ)x T Qy. x T Qy Q 1/2 x Q 1/2 y = x T Qx y T Qy 1 2 (xt Qx + y T Qy) z T Qz λx T Qx + (1 λ)y T Qy f (z) = z T Qz + 2b T z + c λx T Qx + (1 λ)y T Qy + 2λb T x + 2(1 λ)b T y + λc + (1 λ)c = λf (x) + (1 λ)f (y) 0, Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 5 / 32
6 Algebraic Operations Preserving Convexity Lemma. Let C i R n be a convex set for any i I where I is an index set (possibly infinite). Then the set i I C i is convex. Proof. In class Example: Consider the set P = {x R n : Ax b} where A R m n and b R m is convex. P is called a convex polyhedron and it is indeed convex. Why? Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 6 / 32
7 Algebraic Operations Preserving Convexity preservation under addition, cartesian product, forward and inverse linear mappings Theorem. 1. Let C 1, C 2,..., C k R n be convex sets and let µ 1, µ 2,..., µ k R. Then the set µ 1C 1 + µ 2C µ k C k is convex. 2. Let C i R k i, i = 1,..., m be convex sets. Then the cartesian product C 1 C 2 C m = {(x 1, x 2,..., x m) : x i C i, i = 1, 2,..., m} is convex. 3. Let M R n be a convex set and let A R m n. Then the set is convex. A(M) = {Ax : x M} 4. Let D R m be convex and let A R m n. Then the set is convex. A 1 (M) = {x R n : Ax D} Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 7 / 32
8 Convex Combinations Given m points x 1, x 2,..., x m R n, a convex combination of these m points is a vector of the form λ 1 x 1 + λ 2 x λ m x m, where λ 1, λ 2,..., λ m are nonnegative numbers satisfying λ 1 + λ λ m = 1. A convex set is defined by the property that any convex combination of two points from the set is also in the set. We will now show that a convex combination of any number of points from a convex set is in the set. Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 8 / 32
9 Convex Combinations Theorem.Let C R n be a convex set and let x 1, x 2,..., x m C. Then for any λ m, the relation m λ ix i C holds. Proof by induction on m. For m = 1 the result is obvious. The induction hypothesis is that for any m vectors x 1, x 2,..., x m C and any λ m, the vector m λ ix i belongs to C. We will now prove the theorem for m + 1 vectors. Suppose that x 1, x 2,..., x m+1 C and that λ m+1. We will show that z m+1 λ ix i C. If λ m+1 = 1, then z = x m+1 C and the result obviously follows. If λ m+1 < 1 then z = m m λ λ i ix i + λ m+1 x m+1 = (1 λ m+1 ) x i +λ m+1 x m+1. 1 λ m+1 }{{} v C and hence z = (1 λ m+1 )v + λ m+1 x m+1 C. v Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 9 / 32
10 The Convex Hull Definition. Let S R n. The convex hull of S, denoted by conv(s), is the set comprising all the convex combinations of vectors from S: { k } conv(s) λ i x i, : x 1, x 2,..., x k S, λ k. C conv(c) Figure: A nonconvex set and its convex hull Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 10 / 32
11 The Convex Hull The convex hull conv(s) is smallest convex set containing S. Proof. Lemma. Let S R n. If S T for some convex set T, then conv(s) T. Suppose that indeed S T for some convex set T. To prove that conv(s) T, take z conv(s). There exist x 1, x 2,..., x k S T (where k is a positive integer), and λ k such that z = k λ ix i. Since x 1, x 2,..., x k T, it follows that z T, showing the desired result. Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 11 / 32
12 We can assume that λ i > 0 for all i = 1, 2,..., k. If k n + 1, the result is proven. Otherwise, if k n + 2, then the vectors x 2 x 1, x 3 x 1,..., x k x 1, being more than n vectors in R n, are necessarily linearly dependent µ 2, µ 3,..., µ k not all zeros s.t. k i=2 µ i(x i x 1 ) = 0. Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 12 / 32 Carathéodory theorem Theorem. Let S R n and let x conv(s). Then there exist x 1, x 2,..., x n+1 S such that x conv ({x 1, x 2,..., x n+1 }), that is, there exist λ n+1 such that n+1 x = λ i x i. Proof. Let x conv(s). Then x 1, x 2,..., x k S and λ k s.t. x = k λ i x i.
13 Proof of Carathéodory Theorem Contd. Defining µ 1 = k i=2 µ i, we obtain that k µ i x i = 0, Not all of the coefficients µ 1, µ 2,..., µ k are zeros and k µ i = 0. There exists an index i for which µ i < 0. Let α R +. Then x = k λ i x i = k λ i x i + α k µ i x i = k (λ i + αµ i )x i. (1) We have k (λ i + αµ i ) = 1, so (1) is a convex combination representation iff λ i + αµ i 0 for all i = 1,..., k. (2). Since λ i > 0 for { all i, it follows that (2) is satisfied for all α [0, ε] where ε = min i:µi <0 λ i µ i }. Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 13 / 32
14 Proof of Carathéodory Theorem Contd. If we substitute { α = ε, then (2) still holds, but λ j + εµ j = 0 for j argmin µ } i. i:µ i <0 λ i This means that we found a representation of x as a convex combination of k 1 (or less) vectors. This process can be carried on until a representation of x as a convex combination of no more than n + 1 vectors is derived. Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 14 / 32
15 Example For n = 2, consider the four vectors ( 1 x 1 =, x 1) 2 = ( 1 2), x 3 = ( ( 2 2, x 1) 4 =, 2) and let x conv({x 1, x 2, x 3, x 4 }) be given by x = 1 8 x x x ( 13 ) 8 x 4 = Find a representation of x as a convex combination of no more than 3 vectors. In class Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 15 / 32
16 Convex Cones A set S is called a cone if it satisfies the following property: for any x S and λ 0, the inclusion λx S is satisfied. The following lemma shows that there is a very simple and elegant characterization of convex cones. Lemma. A set S is a convex cone if and only if the following properties hold: A. x, y S x + y S. B. x S, λ 0 λx S. Simple exercise Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 16 / 32
17 Examples of Convex Cones The convex polytope C = {x R n : Ax 0}, where A R m n. Lorentz Cone The Lorenz cone, or ice cream cone is given by {( ) } L n x = R n+1 : x t, x R n, t R. t nonnegative polynomials. set consisting of all possible coefficients of polynomials of degree n 1 which are nonnegative over R: K n = {x R n : x 1 t n 1 + x 2 t n x n 1 t + x n 0 t R} Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 17 / 32
18 The Conic Hull Definition. Given m points x 1, x 2,..., x m R n, a conic combination of these m points is a vector of the form λ 1 x 1 + λ 2 x λ m x n, where λ R m +. The definition of the conic hull is now quite natural. Definition. Let S R n. Then the conic hull of S, denoted by cone(s) is the set comprising all the conic combinations of vectors from S: { k } conv(s) λ i x i : x 1, x 2,..., x k S, λ R k +. Similarly to the convex hull, the conic hull of a set S is the smallest cone containing S. Lemma. Let S R n. If S T for some convex cone T, then cone(s) T. Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 18 / 32
19 Representation Theorem for Conic Hulls a similar result to Carathéodory theorem Conic Representation Theorem. Let S R n and let x cone(s). Then there exist k linearly independent vector x 1, x 2,..., x k S such that x cone ({x 1, x 2,..., x k }), that is, there exist λ R k + such that In particular, k n. x = k λ i x i. Proof very similar to the proof of Carathéodory theorem. See page 107 of the book for the proof. Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 19 / 32
20 Basic Feasible Solutions Consider the convex polyhedron. P = {x R n : Ax = b, x 0}, (A R m n, b R m ) the rows of A are assumed to be linearly independent. The above is a standard formulation of the constraints of a linear programming problem. Definition. x is a basic feasible solution (abbreviated bfs) of P if the columns of A corresponding to the indices of the positive values of x are linearly independent. Example.Consider the linear system: x 1 + x 2 + x 3 = 6 x 2 + x 4 = 3 x 1, x 2, x 3, x 4 0. Find all the basic feasible solutions. In class Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 20 / 32
21 Existence of bfs s Theorem.Let P = {x R n : Ax = b, x 0}, where A R m n and b R m. If P, then it contains at least one bfs. Proof. P b cone({a 1, a 2,..., a n }) where a i denotes the i-th column of A. By the conic representation theorem, there exist indices i 1 < i 2 <... < i k and k numbers y i1, y i2,..., y ik 0 such that b = k j=1 y i j a ij and a i1, a i2,..., a ik are linearly independent. Denote x = k j=1 y i j e ij. Then obviously x 0 and in addition A x = k y ij Ae ij = j=1 k y ij a ij = b. Therefore, x is contained in P and the columns of A corresponding to the indices of the positive components of x are linearly independent, meaning that P contains a bfs. j=1 Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 21 / 32
22 Topological Properties of Convex Sets Proof. Theorem.Let C R n be a convex set. Then cl(c) is a convex set. Let x, y cl(c) and let λ [0, 1]. There exist sequences {x k } k 0 C and {y k } k 0 C for which x k x and y k y as k. (*) λx k + (1 λ)y k C for any k 0. (**) λx k + (1 λ)y k λx + (1 λ)y. (*)+(**) λx + (1 λ)y cl(c). Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 22 / 32
23 The Line Segment Principle Theorem. Let C be a convex set and assume that int(c). Suppose that x int(c) and y cl(c). Then (1 λ)x + λy int(c) for any λ [0, 1). Proof. There exists ε > 0 such that B(x, ε) C. Let z = (1 λ)x + λy. We will show that B(z, (1 λ)ε) C. Let w B(z, (1 λ)ε). Since y cl(c), w 1 C s.t. w 1 y < (1 λ)ε w z. (3) λ Set w 2 = 1 1 λ (w λw 1). Then w 2 x = w λw1 1 λ x = 1 (w z) + λ(y w1) 1 λ 1 (3) ( w z + λ w1 y ) < ε, 1 λ Hence, since B(x, ε) C, it follows that w 2 C. Finally, since w = λw 1 + (1 λ)w 2 with w 1, w 2 C, we have that w C. Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 23 / 32
24 Convexity of the Interior Theorem. Let C R n be a convex set. Then int(c) is convex. Proof. If int(c) =, then the theorem is obviously true. Otherwise, let x 1, x 2 int(c), and let λ (0, 1). By the LSP, λx 1 + (1 λ)x 2 int(c), establishing the convexity of int(c). Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 24 / 32
25 Combination of Closure and Interior Lemma. Let C be a convex set with a nonempty interior. Then 1. cl(int(c)) = cl(c). 2. int(cl(c)) = int(c). Proof of 1. Obviously, cl(int(c)) cl(c) holds. To prove that opposite, let x cl(c), y int(c). Then x k = 1 k y + ( 1 1 k ) x int(c) for any k 1. Since x is the limit (as k ) of the sequence {x k } k 1 int(c), it follows that x cl(int(c)). For the proof of 2, see pages 109,110 of the book for the proof of Lemma 6.30(b). Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 25 / 32
26 Compactness of the Convex Hull of Convex Sets Proof. Theorem. Let S R n be a compact set. Then conv(s) is compact. M > 0 such that x M for any x S. Let y conv(s). Then there exist x 1, x 2,..., x n+1 S and λ n+1 for which y = n+1 λ ix i and therefore n+1 n+1 n+1 y = λ i x i λ i x i M λ i = M, establishing the boundedness of conv(s). To prove the closedness of conv(s), let {y k } k 1 conv(s) be a sequence converging to y R n. There exist x k 1, xk 2,..., xk n+1 S and λk n+1 such that n+1 y k = λ k i x k i. (4) Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 26 / 32
27 Proof Contd. By the compactness of S and n+1, it follows that {(λ k, x k 1, xk 2,..., xk n+1 )} k 1 has a convergent subsequence {(λ k j, x k j 1, xk j 2,..., xk j n+1 )} j 1 whose limit will be denoted by with λ n+1, x 1, x 2,..., x n+1 S Taking the limit j in (λ, x 1, x 2,..., x n+1 ) n+1 y kj = λ k j i x k j i, we obtain that y = n+1 λ ix i conv(s) as required. Example: S = {(0, 0) T } {(x, y) T : xy 1} Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 27 / 32
28 Closedness of the Conic Hull of a Finite Set Proof. Theorem. Let a 1, a 2,..., a k R n. Then cone({a 1, a 2,..., a k }) is closed. By the conic representation theorem, each element of cone({a 1, a 2,..., a k }) can be represented as a conic combination of a linearly independent subset of {a 1, a 2,..., a k }. Therefore, if S 1, S 2,..., S N are all the subsets of {a 1, a 2,..., a k } comprising linearly independent vectors, then cone({a 1, a 2,..., a k }) = N cone(s i ). It is enough to show that cone(s i ) is closed for any i {1, 2,..., N}. Indeed, let i {1, 2,..., N}. Then S i = {b 1, b 2,..., b m }, where b 1, b 2,..., b m are linearly independent. cone(s i ) = {By : y R m +}, where B is the matrix whose columns are b 1, b 2,..., b m. Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 28 / 32
29 Proof Contd. Suppose that x k cone(s i ) for all k 1 and that x k x. y k R m + such that x k = By k. (5) y k = (B T B) 1 B T x k. Taking the limit as k in the last equation, we obtain that y k ȳ where ȳ = (B T B) 1 B T x. ȳ R m +. Thus, taking the limit in (5), we conclude that x = Bȳ with ȳ R m +, and hence x cone(s i ). Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 29 / 32
30 Extreme Points Definition. Let S R n be a convex set. A point x S is called an extreme point of S if there do not exist x 1, x 2 S(x 1 x 2 ) and λ (0, 1), such that x = λx 1 + (1 λ)x 2. The set of extreme point is denoted by ext(s). For example, the set of extreme points of a convex polytope consists of all its vertices. 4 3 x 2 x 3 2 x x Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 30 / 32
31 Equivalence Between bfs s and Extreme Points Theorem. Let P = {x R n : Ax = b, x 0}, where A R m n has linearly independent rows and b R m. The x is a basic feasible solution of P if and only if it is an extreme point of P. Theorem 6.34 in the book. Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 31 / 32
32 Krein-Milman Theorem Theorem. Let S R n be a compact convex set. Then S = conv(ext(s)). Amir Beck Introduction to Nonlinear Optimization Lecture Slides - Convex Sets 32 / 32
Lecture 1: Convex Sets January 23
IE 521: Convex Optimization Instructor: Niao He Lecture 1: Convex Sets January 23 Spring 2017, UIUC Scribe: Niao He Courtesy warning: These notes do not necessarily cover everything discussed in the class.
More informationIE 521 Convex Optimization
Lecture 1: 16th January 2019 Outline 1 / 20 Which set is different from others? Figure: Four sets 2 / 20 Which set is different from others? Figure: Four sets 3 / 20 Interior, Closure, Boundary Definition.
More informationLMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009
LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix
More informationChapter 2: Preliminaries and elements of convex analysis
Chapter 2: Preliminaries and elements of convex analysis Edoardo Amaldi DEIB Politecnico di Milano edoardo.amaldi@polimi.it Website: http://home.deib.polimi.it/amaldi/opt-14-15.shtml Academic year 2014-15
More informationAssignment 1: From the Definition of Convexity to Helley Theorem
Assignment 1: From the Definition of Convexity to Helley Theorem Exercise 1 Mark in the following list the sets which are convex: 1. {x R 2 : x 1 + i 2 x 2 1, i = 1,..., 10} 2. {x R 2 : x 2 1 + 2ix 1x
More informationMath 341: Convex Geometry. Xi Chen
Math 341: Convex Geometry Xi Chen 479 Central Academic Building, University of Alberta, Edmonton, Alberta T6G 2G1, CANADA E-mail address: xichen@math.ualberta.ca CHAPTER 1 Basics 1. Euclidean Geometry
More informationIntroduction to Mathematical Programming IE406. Lecture 3. Dr. Ted Ralphs
Introduction to Mathematical Programming IE406 Lecture 3 Dr. Ted Ralphs IE406 Lecture 3 1 Reading for This Lecture Bertsimas 2.1-2.2 IE406 Lecture 3 2 From Last Time Recall the Two Crude Petroleum example.
More informationAppendix B Convex analysis
This version: 28/02/2014 Appendix B Convex analysis In this appendix we review a few basic notions of convexity and related notions that will be important for us at various times. B.1 The Hausdorff distance
More informationChapter 2 Convex Analysis
Chapter 2 Convex Analysis The theory of nonsmooth analysis is based on convex analysis. Thus, we start this chapter by giving basic concepts and results of convexity (for further readings see also [202,
More informationLinear Algebra Review: Linear Independence. IE418 Integer Programming. Linear Algebra Review: Subspaces. Linear Algebra Review: Affine Independence
Linear Algebra Review: Linear Independence IE418: Integer Programming Department of Industrial and Systems Engineering Lehigh University 21st March 2005 A finite collection of vectors x 1,..., x k R n
More informationAnalysis-3 lecture schemes
Analysis-3 lecture schemes (with Homeworks) 1 Csörgő István November, 2015 1 A jegyzet az ELTE Informatikai Kar 2015. évi Jegyzetpályázatának támogatásával készült Contents 1. Lesson 1 4 1.1. The Space
More informationFarkas Lemma. Rudi Pendavingh. Optimization in R n, lecture 2. Eindhoven Technical University. Rudi Pendavingh (TUE) Farkas Lemma ORN2 1 / 15
Farkas Lemma Rudi Pendavingh Eindhoven Technical University Optimization in R n, lecture 2 Rudi Pendavingh (TUE) Farkas Lemma ORN2 1 / 15 Today s Lecture Theorem (Farkas Lemma, 1894) Let A be an m n matrix,
More informationGEORGIA INSTITUTE OF TECHNOLOGY H. MILTON STEWART SCHOOL OF INDUSTRIAL AND SYSTEMS ENGINEERING LECTURE NOTES OPTIMIZATION III
GEORGIA INSTITUTE OF TECHNOLOGY H. MILTON STEWART SCHOOL OF INDUSTRIAL AND SYSTEMS ENGINEERING LECTURE NOTES OPTIMIZATION III CONVEX ANALYSIS NONLINEAR PROGRAMMING THEORY NONLINEAR PROGRAMMING ALGORITHMS
More informationCOURSE ON LMI PART I.2 GEOMETRY OF LMI SETS. Didier HENRION henrion
COURSE ON LMI PART I.2 GEOMETRY OF LMI SETS Didier HENRION www.laas.fr/ henrion October 2006 Geometry of LMI sets Given symmetric matrices F i we want to characterize the shape in R n of the LMI set F
More informationConvex Sets with Applications to Economics
Convex Sets with Applications to Economics Debasis Mishra March 10, 2010 1 Convex Sets A set C R n is called convex if for all x, y C, we have λx+(1 λ)y C for all λ [0, 1]. The definition says that for
More informationConvex Geometry. Carsten Schütt
Convex Geometry Carsten Schütt November 25, 2006 2 Contents 0.1 Convex sets... 4 0.2 Separation.... 9 0.3 Extreme points..... 15 0.4 Blaschke selection principle... 18 0.5 Polytopes and polyhedra.... 23
More informationExtreme points of compact convex sets
Extreme points of compact convex sets In this chapter, we are going to show that compact convex sets are determined by a proper subset, the set of its extreme points. Let us start with the main definition.
More information3. Linear Programming and Polyhedral Combinatorics
Massachusetts Institute of Technology 18.433: Combinatorial Optimization Michel X. Goemans February 28th, 2013 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the introductory
More informationAppendix A: Separation theorems in IR n
Appendix A: Separation theorems in IR n These notes provide a number of separation theorems for convex sets in IR n. We start with a basic result, give a proof with the help on an auxiliary result and
More informationAN INTRODUCTION TO CONVEXITY
AN INTRODUCTION TO CONVEXITY GEIR DAHL NOVEMBER 2010 University of Oslo, Centre of Mathematics for Applications, P.O.Box 1053, Blindern, 0316 Oslo, Norway (geird@math.uio.no) Contents 1 The basic concepts
More informationAppendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS
Appendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS Here we consider systems of linear constraints, consisting of equations or inequalities or both. A feasible solution
More informationOptimization and Optimal Control in Banach Spaces
Optimization and Optimal Control in Banach Spaces Bernhard Schmitzer October 19, 2017 1 Convex non-smooth optimization with proximal operators Remark 1.1 (Motivation). Convex optimization: easier to solve,
More informationBasic convexity. 1.1 Convex sets and combinations. λ + μ b (λ + μ)a;
1 Basic convexity 1.1 Convex sets and combinations AsetA R n is convex if together with any two points x, y it contains the segment [x, y], thus if (1 λ)x + λy A for x, y A, 0 λ 1. Examples of convex sets
More informationDiscrete Geometry. Problem 1. Austin Mohr. April 26, 2012
Discrete Geometry Austin Mohr April 26, 2012 Problem 1 Theorem 1 (Linear Programming Duality). Suppose x, y, b, c R n and A R n n, Ax b, x 0, A T y c, and y 0. If x maximizes c T x and y minimizes b T
More informationPOLARS AND DUAL CONES
POLARS AND DUAL CONES VERA ROSHCHINA Abstract. The goal of this note is to remind the basic definitions of convex sets and their polars. For more details see the classic references [1, 2] and [3] for polytopes.
More informationPractice Exam 1: Continuous Optimisation
Practice Exam : Continuous Optimisation. Let f : R m R be a convex function and let A R m n, b R m be given. Show that the function g(x) := f(ax + b) is a convex function of x on R n. Suppose that f is
More informationLocally convex spaces, the hyperplane separation theorem, and the Krein-Milman theorem
56 Chapter 7 Locally convex spaces, the hyperplane separation theorem, and the Krein-Milman theorem Recall that C(X) is not a normed linear space when X is not compact. On the other hand we could use semi
More informationClosedness of Integer Hulls of Simple Conic Sets
Closedness of Integer Hulls of Simple Conic Sets Diego A. Morán R., Santanu S. Dey June 7, 2013 Abstract Let C be a full-dimensional pointed closed convex cone in R m obtained by taking the conic hull
More informationDO NOT OPEN THIS QUESTION BOOKLET UNTIL YOU ARE TOLD TO DO SO
QUESTION BOOKLET EECS 227A Fall 2009 Midterm Tuesday, Ocotober 20, 11:10-12:30pm DO NOT OPEN THIS QUESTION BOOKLET UNTIL YOU ARE TOLD TO DO SO You have 80 minutes to complete the midterm. The midterm consists
More information3. Linear Programming and Polyhedral Combinatorics
Massachusetts Institute of Technology 18.453: Combinatorial Optimization Michel X. Goemans April 5, 2017 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the introductory
More informationContinuous Optimisation, Chpt 7: Proper Cones
Continuous Optimisation, Chpt 7: Proper Cones Peter J.C. Dickinson DMMP, University of Twente p.j.c.dickinson@utwente.nl http://dickinson.website/teaching/2017co.html version: 10/11/17 Monday 13th November
More informationNotes for Functional Analysis
Notes for Functional Analysis Wang Zuoqin (typed by Xiyu Zhai) November 6, 2015 1 Lecture 18 1.1 The convex hull Let X be any vector space, and E X a subset. Definition 1.1. The convex hull of E is the
More informationLinear Programming. Murti V. Salapaka Electrical Engineering Department University Of Minnesota, Twin Cities
Linear Programming Murti V Salapaka Electrical Engineering Department University Of Minnesota, Twin Cities murtis@umnedu September 4, 2012 Linear Programming 1 The standard Linear Programming (SLP) problem:
More informationp. 5 First line of Section 1.4. Change nonzero vector v R n to nonzero vector v C n int([x, y]) = (x, y)
Corrections for the first printing of INTRODUCTION TO NON- LINEAR OPTIMIZATION: THEORY, ALGORITHMS, AND AP- PLICATIONS WITH MATLAB, SIAM, 204, by Amir Beck 2 Last Updated: June 8, 207 p. 5 First line of
More informationConvex Optimization. (EE227A: UC Berkeley) Lecture 28. Suvrit Sra. (Algebra + Optimization) 02 May, 2013
Convex Optimization (EE227A: UC Berkeley) Lecture 28 (Algebra + Optimization) 02 May, 2013 Suvrit Sra Admin Poster presentation on 10th May mandatory HW, Midterm, Quiz to be reweighted Project final report
More informationLECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE
LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE CONVEX ANALYSIS AND DUALITY Basic concepts of convex analysis Basic concepts of convex optimization Geometric duality framework - MC/MC Constrained optimization
More informationChapter 1. Preliminaries. The purpose of this chapter is to provide some basic background information. Linear Space. Hilbert Space.
Chapter 1 Preliminaries The purpose of this chapter is to provide some basic background information. Linear Space Hilbert Space Basic Principles 1 2 Preliminaries Linear Space The notion of linear space
More informationProblem Set 6: Solutions Math 201A: Fall a n x n,
Problem Set 6: Solutions Math 201A: Fall 2016 Problem 1. Is (x n ) n=0 a Schauder basis of C([0, 1])? No. If f(x) = a n x n, n=0 where the series converges uniformly on [0, 1], then f has a power series
More informationProblem 1 (Exercise 2.2, Monograph)
MS&E 314/CME 336 Assignment 2 Conic Linear Programming January 3, 215 Prof. Yinyu Ye 6 Pages ASSIGNMENT 2 SOLUTIONS Problem 1 (Exercise 2.2, Monograph) We prove the part ii) of Theorem 2.1 (Farkas Lemma
More informationHandout 2: Elements of Convex Analysis
ENGG 5501: Foundations of Optimization 2018 19 First Term Handout 2: Elements of Convex Analysis Instructor: Anthony Man Cho So September 10, 2018 As briefly mentioned in Handout 1, the notion of convexity
More informationSummer School: Semidefinite Optimization
Summer School: Semidefinite Optimization Christine Bachoc Université Bordeaux I, IMB Research Training Group Experimental and Constructive Algebra Haus Karrenberg, Sept. 3 - Sept. 7, 2012 Duality Theory
More informationHelly s Theorem with Applications in Combinatorial Geometry. Andrejs Treibergs. Wednesday, August 31, 2016
Undergraduate Colloquium: Helly s Theorem with Applications in Combinatorial Geometry Andrejs Treibergs University of Utah Wednesday, August 31, 2016 2. USAC Lecture on Helly s Theorem The URL for these
More informationContents Real Vector Spaces Linear Equations and Linear Inequalities Polyhedra Linear Programs and the Simplex Method Lagrangian Duality
Contents Introduction v Chapter 1. Real Vector Spaces 1 1.1. Linear and Affine Spaces 1 1.2. Maps and Matrices 4 1.3. Inner Products and Norms 7 1.4. Continuous and Differentiable Functions 11 Chapter
More informationLECTURE 3 LECTURE OUTLINE
LECTURE 3 LECTURE OUTLINE Differentiable Conve Functions Conve and A ne Hulls Caratheodory s Theorem Reading: Sections 1.1, 1.2 All figures are courtesy of Athena Scientific, and are used with permission.
More informationConvex Optimization and an Introduction to Congestion Control. Lecture Notes. Fabian Wirth
Convex Optimization and an Introduction to Congestion Control Lecture Notes Fabian Wirth August 29, 2012 ii Contents 1 Convex Sets and Convex Functions 3 1.1 Convex Sets....................................
More informationMAT-INF4110/MAT-INF9110 Mathematical optimization
MAT-INF4110/MAT-INF9110 Mathematical optimization Geir Dahl August 20, 2013 Convexity Part IV Chapter 4 Representation of convex sets different representations of convex sets, boundary polyhedra and polytopes:
More informationA Review of Linear Programming
A Review of Linear Programming Instructor: Farid Alizadeh IEOR 4600y Spring 2001 February 14, 2001 1 Overview In this note we review the basic properties of linear programming including the primal simplex
More information3. Vector spaces 3.1 Linear dependence and independence 3.2 Basis and dimension. 5. Extreme points and basic feasible solutions
A. LINEAR ALGEBRA. CONVEX SETS 1. Matrices and vectors 1.1 Matrix operations 1.2 The rank of a matrix 2. Systems of linear equations 2.1 Basic solutions 3. Vector spaces 3.1 Linear dependence and independence
More informationKey words. Complementarity set, Lyapunov rank, Bishop-Phelps cone, Irreducible cone
ON THE IRREDUCIBILITY LYAPUNOV RANK AND AUTOMORPHISMS OF SPECIAL BISHOP-PHELPS CONES M. SEETHARAMA GOWDA AND D. TROTT Abstract. Motivated by optimization considerations we consider cones in R n to be called
More informationWeek 3: Faces of convex sets
Week 3: Faces of convex sets Conic Optimisation MATH515 Semester 018 Vera Roshchina School of Mathematics and Statistics, UNSW August 9, 018 Contents 1. Faces of convex sets 1. Minkowski theorem 3 3. Minimal
More informationHelly's Theorem and its Equivalences via Convex Analysis
Portland State University PDXScholar University Honors Theses University Honors College 2014 Helly's Theorem and its Equivalences via Convex Analysis Adam Robinson Portland State University Let us know
More informationIn English, this means that if we travel on a straight line between any two points in C, then we never leave C.
Convex sets In this section, we will be introduced to some of the mathematical fundamentals of convex sets. In order to motivate some of the definitions, we will look at the closest point problem from
More informationOperations that preserve the covering property of the lifting region
Operations that preserve the covering property of the lifting region Amitabh Basu and Joe Paat June 23, 2015 Abstract We contribute to the theory for minimal liftings of cut-generating functions. In particular,
More informationLECTURE SLIDES ON BASED ON CLASS LECTURES AT THE CAMBRIDGE, MASS FALL 2007 BY DIMITRI P. BERTSEKAS.
LECTURE SLIDES ON CONVEX ANALYSIS AND OPTIMIZATION BASED ON 6.253 CLASS LECTURES AT THE MASSACHUSETTS INSTITUTE OF TECHNOLOGY CAMBRIDGE, MASS FALL 2007 BY DIMITRI P. BERTSEKAS http://web.mit.edu/dimitrib/www/home.html
More informationNonlinear Programming Models
Nonlinear Programming Models Fabio Schoen 2008 http://gol.dsi.unifi.it/users/schoen Nonlinear Programming Models p. Introduction Nonlinear Programming Models p. NLP problems minf(x) x S R n Standard form:
More informationContinuity of convex functions in normed spaces
Continuity of convex functions in normed spaces In this chapter, we consider continuity properties of real-valued convex functions defined on open convex sets in normed spaces. Recall that every infinitedimensional
More information1 Overview. 2 Extreme Points. AM 221: Advanced Optimization Spring 2016
AM 22: Advanced Optimization Spring 206 Prof. Yaron Singer Lecture 7 February 7th Overview In the previous lectures we saw applications of duality to game theory and later to learning theory. In this lecture
More informationGeneral Convexity, General Concavity, Fixed Points, Geometry, and Min-Max Points
Mathematica Moravica Vol. 12-1 2008), 63 109 General Convexity, General Concavity, Fixed Points, Geometry, and Min-Max Points Milan R. Tasković Abstract. This paper continues the study of general convexity
More informationLECTURE SLIDES ON CONVEX OPTIMIZATION AND DUALITY THEORY TATA INSTITUTE FOR FUNDAMENTAL RESEARCH MUMBAI, INDIA JANUARY 2009 PART I
LECTURE SLIDES ON CONVEX OPTIMIZATION AND DUALITY THEORY TATA INSTITUTE FOR FUNDAMENTAL RESEARCH MUMBAI, INDIA JANUARY 2009 PART I BY DIMITRI P. BERTSEKAS M.I.T. http://web.mit.edu/dimitrib/www/home.html
More informationInequality Constraints
Chapter 2 Inequality Constraints 2.1 Optimality Conditions Early in multivariate calculus we learn the significance of differentiability in finding minimizers. In this section we begin our study of the
More informationSome Properties of Convex Hulls of Integer Points Contained in General Convex Sets
Some Properties of Convex Hulls of Integer Points Contained in General Convex Sets Santanu S. Dey and Diego A. Morán R. H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute
More informationThe Split Closure of a Strictly Convex Body
The Split Closure of a Strictly Convex Body D. Dadush a, S. S. Dey a, J. P. Vielma b,c, a H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, 765 Ferst Drive
More informationIntroduction to Koecher Cones. Michael Orlitzky
Introduction to Koecher Cones Extreme Points Definition 1. Let S be a convex set in some vector space. We say that the point x S is an extreme point of the set S if x = λx 1 + (1 λ)x 2 for λ (0, 1) implies
More informationExample: feasibility. Interpretation as formal proof. Example: linear inequalities and Farkas lemma
4-1 Algebra and Duality P. Parrilo and S. Lall 2006.06.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone of valid
More informationChapter 2. Convex Sets: basic results
Chapter 2 Convex Sets: basic results In this chapter, we introduce one of the most important tools in the mathematical approach to Economics, namely the theory of convex sets. Almost every situation we
More informationLecture: Cone programming. Approximating the Lorentz cone.
Strong relaxations for discrete optimization problems 10/05/16 Lecture: Cone programming. Approximating the Lorentz cone. Lecturer: Yuri Faenza Scribes: Igor Malinović 1 Introduction Cone programming is
More informationLecture 2 - Unconstrained Optimization Definition[Global Minimum and Maximum]Let f : S R be defined on a set S R n. Then
Lecture 2 - Unconstrained Optimization Definition[Global Minimum and Maximum]Let f : S R be defined on a set S R n. Then 1. x S is a global minimum point of f over S if f (x) f (x ) for any x S. 2. x S
More informationOn duality theory of conic linear problems
On duality theory of conic linear problems Alexander Shapiro School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia 3332-25, USA e-mail: ashapiro@isye.gatech.edu
More informationChapter 1. Preliminaries
Introduction This dissertation is a reading of chapter 4 in part I of the book : Integer and Combinatorial Optimization by George L. Nemhauser & Laurence A. Wolsey. The chapter elaborates links between
More informationAn Alternative Proof of Primitivity of Indecomposable Nonnegative Matrices with a Positive Trace
An Alternative Proof of Primitivity of Indecomposable Nonnegative Matrices with a Positive Trace Takao Fujimoto Abstract. This research memorandum is aimed at presenting an alternative proof to a well
More informationConvex Feasibility Problems
Laureate Prof. Jonathan Borwein with Matthew Tam http://carma.newcastle.edu.au/drmethods/paseky.html Spring School on Variational Analysis VI Paseky nad Jizerou, April 19 25, 2015 Last Revised: May 6,
More informationOn John type ellipsoids
On John type ellipsoids B. Klartag Tel Aviv University Abstract Given an arbitrary convex symmetric body K R n, we construct a natural and non-trivial continuous map u K which associates ellipsoids to
More informationNotes taken by Graham Taylor. January 22, 2005
CSC4 - Linear Programming and Combinatorial Optimization Lecture : Different forms of LP. The algebraic objects behind LP. Basic Feasible Solutions Notes taken by Graham Taylor January, 5 Summary: We first
More informationCover Page. The handle holds various files of this Leiden University dissertation
Cover Page The handle http://hdl.handle.net/1887/32076 holds various files of this Leiden University dissertation Author: Junjiang Liu Title: On p-adic decomposable form inequalities Issue Date: 2015-03-05
More information4. Algebra and Duality
4-1 Algebra and Duality P. Parrilo and S. Lall, CDC 2003 2003.12.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone
More informationA NICE PROOF OF FARKAS LEMMA
A NICE PROOF OF FARKAS LEMMA DANIEL VICTOR TAUSK Abstract. The goal of this short note is to present a nice proof of Farkas Lemma which states that if C is the convex cone spanned by a finite set and if
More informationOptimization methods NOPT048
Optimization methods NOPT048 Jirka Fink https://ktiml.mff.cuni.cz/ fink/ Department of Theoretical Computer Science and Mathematical Logic Faculty of Mathematics and Physics Charles University in Prague
More informationLecture 2: Convex Sets and Functions
Lecture 2: Convex Sets and Functions Hyang-Won Lee Dept. of Internet & Multimedia Eng. Konkuk University Lecture 2 Network Optimization, Fall 2015 1 / 22 Optimization Problems Optimization problems are
More informationOn the projection onto a finitely generated cone
Acta Cybernetica 00 (0000) 1 15. On the projection onto a finitely generated cone Miklós Ujvári Abstract In the paper we study the properties of the projection onto a finitely generated cone. We show for
More informationConvex Optimization Theory
Convex Optimization Theory A SUMMARY BY DIMITRI P. BERTSEKAS We provide a summary of theoretical concepts and results relating to convex analysis, convex optimization, and duality theory. In particular,
More informationPreliminary Version Draft 2015
Convex Geometry Introduction Martin Henk TU Berlin Winter semester 2014/15 webpage CONTENTS i Contents Preface ii WS 2014/15 0 Some basic and convex facts 1 1 Support and separate 7 2 Radon, Helly, Caratheodory
More informationLinear Vector Spaces
CHAPTER 1 Linear Vector Spaces Definition 1.0.1. A linear vector space over a field F is a triple (V, +, ), where V is a set, + : V V V and : F V V are maps with the properties : (i) ( x, y V ), x + y
More informationA Characterization of Polyhedral Convex Sets
Journal of Convex Analysis Volume 11 (24), No. 1, 245 25 A Characterization of Polyhedral Convex Sets Farhad Husseinov Department of Economics, Bilkent University, 6533 Ankara, Turkey farhad@bilkent.edu.tr
More informationLECTURE 4 LECTURE OUTLINE
LECTURE 4 LECTURE OUTLINE Relative interior and closure Algebra of relative interiors and closures Continuity of convex functions Closures of functions Reading: Section 1.3 All figures are courtesy of
More informationOptimization methods NOPT048
Optimization methods NOPT048 Jirka Fink https://ktiml.mff.cuni.cz/ fink/ Department of Theoretical Computer Science and Mathematical Logic Faculty of Mathematics and Physics Charles University in Prague
More informationSymmetric Matrices and Eigendecomposition
Symmetric Matrices and Eigendecomposition Robert M. Freund January, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 Symmetric Matrices and Convexity of Quadratic Functions
More informationOn the Properties of Positive Spanning Sets and Positive Bases
Noname manuscript No. (will be inserted by the editor) On the Properties of Positive Spanning Sets and Positive Bases Rommel G. Regis Received: May 30, 2015 / Accepted: date Abstract The concepts of positive
More informationResearch Division. Computer and Automation Institute, Hungarian Academy of Sciences. H-1518 Budapest, P.O.Box 63. Ujvári, M. WP August, 2007
Computer and Automation Institute, Hungarian Academy of Sciences Research Division H-1518 Budapest, P.O.Box 63. ON THE PROJECTION ONTO A FINITELY GENERATED CONE Ujvári, M. WP 2007-5 August, 2007 Laboratory
More informationIntegral Jensen inequality
Integral Jensen inequality Let us consider a convex set R d, and a convex function f : (, + ]. For any x,..., x n and λ,..., λ n with n λ i =, we have () f( n λ ix i ) n λ if(x i ). For a R d, let δ a
More informationVI. Convexity. the (closed) interval spanned by x and y. A subset K of X is called convex if it is closed under interval formation, i.e.
101 VI. Convexity Convex sets In Analysis, convex sets appear as sensible domains from which to single out solutions of certain problems by extremizing convex fl s. This chapter deals with the simple aspects
More informationON GENERALIZED-CONVEX CONSTRAINED MULTI-OBJECTIVE OPTIMIZATION
ON GENERALIZED-CONVEX CONSTRAINED MULTI-OBJECTIVE OPTIMIZATION CHRISTIAN GÜNTHER AND CHRISTIANE TAMMER Abstract. In this paper, we consider multi-objective optimization problems involving not necessarily
More informationKey words. Integer nonlinear programming, Cutting planes, Maximal lattice-free convex sets
ON MAXIMAL S-FREE CONVEX SETS DIEGO A. MORÁN R. AND SANTANU S. DEY Abstract. Let S Z n satisfy the property that conv(s) Z n = S. Then a convex set K is called an S-free convex set if int(k) S =. A maximal
More informationLecture 1: Introduction. Outline. B9824 Foundations of Optimization. Fall Administrative matters. 2. Introduction. 3. Existence of optima
B9824 Foundations of Optimization Lecture 1: Introduction Fall 2009 Copyright 2009 Ciamac Moallemi Outline 1. Administrative matters 2. Introduction 3. Existence of optima 4. Local theory of unconstrained
More informationHomework 5. Solutions
Homework 5. Solutions 1. Let (X,T) be a topological space and let A,B be subsets of X. Show that the closure of their union is given by A B = A B. Since A B is a closed set that contains A B and A B is
More informationUNDERGROUND LECTURE NOTES 1: Optimality Conditions for Constrained Optimization Problems
UNDERGROUND LECTURE NOTES 1: Optimality Conditions for Constrained Optimization Problems Robert M. Freund February 2016 c 2016 Massachusetts Institute of Technology. All rights reserved. 1 1 Introduction
More informationCutting planes from extended LP formulations
Cutting planes from extended LP formulations Merve Bodur University of Wisconsin-Madison mbodur@wisc.edu Sanjeeb Dash IBM Research sanjeebd@us.ibm.com March 7, 2016 Oktay Günlük IBM Research gunluk@us.ibm.com
More informationLecture 7: Positive Semidefinite Matrices
Lecture 7: Positive Semidefinite Matrices Rajat Mittal IIT Kanpur The main aim of this lecture note is to prepare your background for semidefinite programming. We have already seen some linear algebra.
More information1 Directional Derivatives and Differentiability
Wednesday, January 18, 2012 1 Directional Derivatives and Differentiability Let E R N, let f : E R and let x 0 E. Given a direction v R N, let L be the line through x 0 in the direction v, that is, L :=
More informationLecture 1: Introduction. Outline. B9824 Foundations of Optimization. Fall Administrative matters. 2. Introduction. 3. Existence of optima
B9824 Foundations of Optimization Lecture 1: Introduction Fall 2010 Copyright 2010 Ciamac Moallemi Outline 1. Administrative matters 2. Introduction 3. Existence of optima 4. Local theory of unconstrained
More informationOptimization Theory. A Concise Introduction. Jiongmin Yong
October 11, 017 16:5 ws-book9x6 Book Title Optimization Theory 017-08-Lecture Notes page 1 1 Optimization Theory A Concise Introduction Jiongmin Yong Optimization Theory 017-08-Lecture Notes page Optimization
More information