Introduction to Convex Analysis Microeconomics II - Tutoring Class
|
|
- Phoebe Leona Morris
- 5 years ago
- Views:
Transcription
1 Introduction to Convex Analysis Microeconomics II - Tutoring Class Professor: V. Filipe Martins-da-Rocha TA: Cinthia Konichi April Basic Concepts and Results This is a first glance on basic convex analysis results that are going to be used extensively during the course. Convexity is a very important concept in optimization theory since if it is assumed, necessary conditions for optimality become sufficient conditions. For an introduction to convex analysis as applied to optimization theory see Florenzano and Le Van (2001) and Izmailov and Solodov (2005). Borwein and Lewis (2000) and Bertsekas et al. (2003) are other useful references. For a throughout treatment see Rockafellar (1970). We begin by characterizing convex sets. Let E be a real vector space. Then the following definitions are appropriate: Definition 1 A set D E is said to be convex if for any x, y D the set {αx+(1 α)y : α [0, 1]} is contained in D. The point αx + (1 α)y is called convex combination of x and y (with parameter α). By induction, it is easily seen that D is convex if and only if p k=1 λ kx k D for every finite set {x 1,..., x p } of p elements of D and for every system of p nonnegative real coefficients {λ 1,..., λ p } such that p k=1 λ k = 1. Hence, a subset D of E is convex if and only if every convex combination of finitely many elements of D belongs to D. Some examples of convex sets are the open (or closed) balls on a vector space, the line segments, the vector space itself and the empty set. EPGE-FGV 1
2 x αx + (1 α)y y x y Figure 1: A convex and a non-convex set Lemma 2 Let Λ be an arbitrary set and {D λ } λ Λ be a family of convex subsets of E. Then D = λ Λ D λ is convex. Proof: Let x D and y D. Then x D λ and y D λ for all λ Λ. Since the sets D λ, λ Λ, are convex, αx + (1 α)y D λ for any α [0, 1] for all λ Λ. Hence, αx + (1 α)y D, that is D is convex. Then, on an optimization problem, for example, an arbitrary number of convex constraints, on the same space, turns out to be convex. With any arbitrary set C of E, we can associate another set, called the convex hull of C, denoted by co C, which is the intersection of all convex subsets of E containing C. By Lemma 2, the convex hull is convex. By the way, it is the smallest convex subset of E containing C. Lemma 3 Let D E be a convex set, then cld is convex 1 Proof: 2 Let x, y cld and α (0, 1). Then, there are sequences (x n ) n N and (y n ) n N in D, such that x n x and y n y. But, by the convexity of D αx n + (1 α)y n D, n N. Then αx n + (1 α)y n αx + (1 α)y implies αx + (1 α)y cld. C cl C co C Figure 2: The closure and the convex hull of a set C 1 Where cld is the closure of D. Remember that the closure of an arbitrary set B on a vector space is the set of limit points of sequences belonging to B. 2 For convenience, assume that E is a normed real vector space and take the usual convergence concept. 2
3 Definition 4 Let D E be a convex set. The function f : D R is convex in D when for any x D, y D e α [0, 1] we have: f(αx + (1 α)y) αf(x) + (1 α)f(y) The function f is said strictly convex if the inequality above is strict for all x y and α (0, 1). f(y) αf(x) + (1 α)f(y) f(αx + (1 α)y) f(x) x αx + (1 α)y y Figure 3: An illustration of a convex function Definition 5 Let D E be a convex set. The function f : D R is concave in D if ( f) is convex in D. Next lemma states an equivalent definition of concavity: Lemma 6 Let D E be a convex set, then f : D R is concave if and only if the set hypof := {(x, µ) D R : f(x) µ} is convex. Such set is called the hypograph of f. Proof: First, suppose that hypof is convex. Let x D and y D. Clearly, (x, f(x)) hypof and (y, f(y)) hypof. Because of the convexity of hypof, for all α [0, 1], we have: (αx + (1 α)y, αf(x) + (1 α)f(y)) = α(x, f(x)) + (1 α)(y, f(y)) hypof By the definition of hypof, we have: f(αx + (1 α)y) αf(x) + (1 α)f(y) 3
4 So f is concave. Conversely, suppose now that f is concave. Let (x, c 1 ) hypof and (x, c 2 ) hypof. Since f(x) c 1 and f(y) c 2, by the concavity of f, for all α [0, 1] we have: f(αx + (1 α)y) αf(x) + (1 α)f(y) αc 1 + (1 α)c 2 which means that: α(x, c 1 ) + (1 α)(y, c 2 ) = (αx + (1 α)y, αc 1 + (1 α)c 2 ) hypof Hence, the hypof is convex. We say that max f(x) subject to x D (1) is convex maximization problem when D E is a convex set and f : D R is concave in D. The importance of the convexity assumption can be seen in the following result. Theorem 7 Let D E be a convex set and f : D R a concave function in D. Then, every local maximum of problem 1 is global. Moreover, the set of elements that maximize the problem is convex. If f is strictly concave, the problem does not have more than one maximizer. Proof: Suppose by way of contradiction that x D is a local maximizer that is not global. Then, exists y D such that f(y) > f( x). Define x(α) = αy + (1 α) x. By the convexity of D, x(α) D for all α [0, 1]. By the concavity of f, for all α (0, 1], we have: f(x(α)) αf(y) + (1 α)f( x) = f( x) + α(f(y) f( x)) > f( x) Taking α > 0 sufficiently low, we can guarantee that the point x(α) is arbitrarily close to the point x and f(x(α)) > f( x). This contradicts the fact that x is a local maximizer of problem 1. Then, any local solution must be a global solution. Let S D be the set of (global) maximizers and v R the optimum value of the problem. Note that we have f(x) = v for any x S. For any x S, x S and α [0, 1], by the concavity of f, we have: f(αx + (1 α) x) αf(x) + (1 α)f( x) = α v + (1 α) v = v which implies that f(αx + (1 α) x) = v and then αx + (1 α) x S. Suppose now that f is strictly concave and that exist x S and x S, with x x. Let α (0, 1). Since x and x are global maximizers and αx + (1 α) x D by the convexity of D, it follows that: f(αx + (1 α) x) f(x) = f( x) = v 4
5 However, as f is strictly concave: f(αx + (1 α) x) > αf(x) + (1 α)f( x) = α v + (1 α) v = v (2) which is a contradiction. 2 Projection and Convex Sets Henceforth, let E be a real vector space equipped with an inner product, : E E R. Consider that : E R + is the norm generated by such inner product. Definition 8 Let B E be a nonempty set and let x 0 E be an arbitrary point. Then we define the distance of the point x 0 to the set B to be d B (x 0 ) : E R +, where d B (x 0 ) := inf x B x x 0 The set P B (x 0 ) := {x B : x x 0 = d B (x 0 )} is called the projection of x 0 on B. Note that, since B and 0, this function is well defined. It is easy to see that d B is continuous 3. If E is finite dimensional and B is a closed set, the minimum is attained. In fact, note that we can define a sequence in B whose distance from x 0 converges to d B (x 0 ). But this implies that this sequence of distances is bounded, which, by the way, implies that the sequence in B is bounded. Therefore, by the Bolzano- Weierstrass Theorem 4, this sequence admits a convergent subsequence. Thus, its limit point belongs to B (closed). Although, under closedness the minimum is not necessarily unique, adding convexity guarantees uniqueness. The geometrical intuition (E = R 2 ) for this result is that the distance is defined by a path which is orthogonal to the set. Therefore, two different projections imply two different paths that are orthogonal to the set. But, then we can define a triangle between those points. However, note that the convexity of B implies that the line segment joining the projections is in B. Hence we have a non-degenerate triangle with two right angles, a contradiction. Let s do the formal statement: Theorem 9 Let E be a finite dimensional real vector space, with a norm defined by an inner product. Let D E be a closed and convex set; and, fix x 0 E. Then 3 Let x, y E. Then x x x y + y x, x E by the triangle inequality inf x B x x x y + inf x B y x. By the other hand, y x y x + x x, x E. Therefore d B (x) d B (y) x y, i.e., d B is a Lipschitz function (thus is continuous). 4 Note that the assertion that a bounded sequence has a convergent subsequence may be invalid on an infinite dimensional space. For example, on R consider the sequence (x n ) n N such that x n = (y n,t ) t N and y n,t = 1 if t = n and 0 otherwise. It s easily seen that (x n ) n N is bounded, however it has no convergent subsequence (on the usual definition of convergence). 5
6 D DA = 2.5 DC = 2.5 A C H G GH = Figure 4: Geometrical intuition for the unique minimum 1. x P D (x 0 ) if and only if x D and x x 0, x y 0, y D; 2. there is a unique x D such that x x 0 = d D (x 0 ). Proof: Let s prove the first assertion and then use it to demonstrate the second one. Let x P D (x 0 ). Then, since D is convex, for any α (0, 1) and any y D\{ x} x x 0 (1 α) x + αy x 0 x x 0 2 (1 α) x + αy x x x 0, x x 0 x x 0 α x + αy, x x 0 α x + αy 0 x x 0, x x 0 x x 0, x x 0 α x + αy α x + αy, x x 0 α x + αy 0 x x 0, α x αy α x + αy, x x 0 α x + αy 0 x x 0, α x αy α x + αy, x x 0 α x + αy, α x + αy 0 2α x x 0, x y α 2 x y 2 0 Dividing both sides of the inequality above by 2α > 0 and letting α 0, we get x x 0, x y 0. On the other hand, let x D be such that x x 0, x y 0, y D. Note that, for any y D, x x 0, x y = x x x x 0, x 0 y. But, by the Cauchy-Schwartz inequality we have x x 0 x 0 y x x 0, x 0 y. Therefore, y D, 0 x x 0, x y x x 0 2 x x 0, x 0 y x x 0 x 0 y If x o D, then x x so x = x 0 P D (x 0 ). If x 0 / D then x x 0 x 0 y, y D, so x P D (x 0 ). Thus (1) is proved. Now, let x, x P D (x 0 ). Then, y D, x x 0, x y 0 and x x 0, x y 0. In particular, x x 0, x x 0 and x x 0, x x 0. Therefore 0 x x 0, x x x x 0, x x = x x, x x = x x 2 6
7 Hence, x = x. Therefore, for a convex D E, it is possible to define a function p D : E D such that P D (x) = {p D (x)}, x E. 3 Separating Hyperplane Theorems Before going to the results, let us define some objects. Definition 10 For a E\{0} and c R the set H(a, c) := {x E : a, x = c} is said to be a hyperplane. Note that E may be written as the union of two disjoint sets and the hyperplane. That is E = H(a, c) {x E : a, x < c} {x E : a, x > c}. Definition 11 Let B 1, B 2 E. The hyperplane H(a, c) is said to separate B 1 and B 2 if x B 1, a, x c a, y, y B 2 If both inequalities are strict, we say that H(a, c) strictly separates B 1 and B 2. In the geometric sense, separability means that a set is on one side of the hyperplane and the other set on the other side (see Figure 5). The next result is important for convex sets: Lemma 12 Let D E be a convex set and let H(a, c) be a hyperplane such that H(a, c) D =. Then D {x E : a, x < c} = or D {x E : a, x > c} =. Proof: Assume by way of contradiction that x, x D, x {x E : a, x < c} and x {x E : a, x > c}. Then, x = λx + (1 λ)x H(a, c), where λ = (0, 1). Therefore, x D H(a, c), contradiction. a,x c a,x a,x 3.1 Support Theorem The next lemma states that a point that does not belong to the closure of a convex set can be strict separated from that set. Lemma 13 (Minkowski Lemma) Let D E be a non-empty convex set, where E is finite dimensional. If x / cld, then there are a E\{0} and c R such that x H(a, c) and a, y > c, y D. 7
8 ... Figure 5: Strict and non-strict separability Proof: By Lemma 3, we know that cld is convex. By Theorem 9, there is a unique projection x = P D (x) cld. Let a = x x (a 0 since x / cld) and c = a, x. Then, y D, a, y = x x, y x x, x. But, by Theorem 9, x x, x y 0, y D. Therefore, a, y x x, x = x x 2 + x x, x = x x 2 + c > c, since x x. Definition 14 A hyperplane H(a, c) is said to support a set B if H(a, c) fr B 5 and B {x E : a, x < c} = or B {x E : a, x > c} =. If x H(a, c) fr B it is x a cl D Figure 6: An illustration of Minkowski Lemma 5 Where fr B is the boundary of B. Remember that x E belongs to fr B if and only if there are sequences on B and B c which both converge to x. Thus, fr B clb. 8
9 said that H(a, c) supports D at x. That is, a hyperplane supports a set if it contains at least one point of the set and if all the points of the set are on a same side of the hyperplane. Note that a set can admit more than one hyperplanes supporting it at the same point. Theorem 15 (Support Theorem) Let E be finite dimensional and D E be a nonempty convex set. If x fr D, then there are a E\{0} and c R such that x H(a, c) and a, y c, y D, i.e., the hyperplane H(a, c) supports D at x. Proof: Since x fr D, there is (x n ) n N such that x n x and x n / cld, n N. By Minkowski Lemma, for each n N, there are a n E\{0} and c n R such that x n H(a n, c n ). Note that a n, x n = c n an a n, x n = c n a n ( ) a Since (x n ) n N is convergent, it is bounded. Furthermore, n a n is bounded. Then, ( ) n N c by the Cauchy-Schwartz inequality, n a n is bounded. Therefore, there is an infinite ( ) n N ( ) subset N c N such that n a a n and n n N a n are convergent. Let a E and n N ( ) c R be the respective limit points of those subsequences. Since = 1, n N, we have a E\{0}. By continuity of the inner product, we have x H(a, c). By the other hand, since a n, y > c n, y D, n N, we have a, y c. a n a n 3.2 Separating Theorems Finally, the most important result of this material: Theorem 16 (Separating Hyperplane) Let E be finite dimensional and D 1, D 2 E be disjoint non-empty convex sets. Then there are a E\{0} and c R such that x D 1, a, x c a, y, y D 2 i.e., there is a hyperplane H(a, c) which separates D 1 and D 2. Proof: Define D = D 2 D 1. Then D is convex and 0 / D because D 1 D 2 =. If 0 / cl D we can apply Minkowski Lemma. Otherwise, if 0 fr D, we apply the Support Theorem. In both cases, there exist a E\{0} such that a, z 0 z D. Therefore, x D 1, a, x a, y, y D 2 9
10 In particular, the function a, is bounded from below in D 2 and bounded from above in D 1. Then, x D 1, a, x sup a, x inf a, x a, y, y D 2 x D 1 x D 2 Since D 1 and D 2 are non-empty, the last inequality is well defined and defining the result holds. c = sup x D 1 a, x + inf x D 2 a, x 2 If, in addition to being convex, we assume that the sets are closed and at least one of them is compact, then we have strict separation. The intuition is that under compacity the sets cannot be, at the same time, arbitrarily near and disjoint. Theorem 17 (Strict Separating Hyperplane) Let E be finite dimensional and D 1, D 2 E be disjoint, non-empty, closed and convex sets. In addition assume that D 1 is compact. Then there are a E\{0} and b, c R such that x D 1, a, x b < c a, y, y D 2 i.e., there is a hyperplane H(a, c) which separates D 1 and D 2. Proof: Since d D2 is continuous and D 1 is compact, by the Weierstrass Theorem x arg min{d D2 (x) : x D 1 } As D 1 is compact, D 2 is closed and D 1 D 2 =, we have d D2 (x ) > 0. Let y = P D2 (x ). Define a = y x 0. But by Theorem 9, we have 0 y x, y y, y D 2 a, y a, y, y D 2. Define c 2 = a, y. Note that, for any x D 1 we have y x d D2 (x) d D2 (x ) = x P D2 (x ) = x y x = P D1 (y ) Hence, we have a, x a, x, x D 1. Define c 1 = a, x. Note that c 2 c 1 = y x 2 > 0. Therefore, let b = (c 2 c 1 )/4 and c = (c 2 c 1 )/2. Then, we have x D 1, a, x b < c a, y, y D 2 10
11 4 Applications In this section, we will give an important application of the results seen above. Lemma 18 Assume E is finite dimensional and let D E be a convex set. If f : D R is concave and x intd, then the following set is non-empty: f(x) = {b E : f(y) + b, y x f(x), y D} Proof: By Lemma 6, hypof is convex. Then, by the Support Theorem, there is a (R E)\{0} and c R such that (f(x), x) H(a, c) and a, z c, z hypof. Let a = (a 1, a 2 ) where a 1 R and a 2 E. Analogously, let z = (z 1, z 2 ). For each z hypof we have a 1 z 1 + a 2, z 2 c. Since for all v < z 1, (v, z 2 ) hypof, it should be the case that a 1 0. Otherwise, for v sufficiently negative the inequality would be invalid. Now we have to prove that a 1 > 0. Assume by way of contradiction that a 1 = 0. Since x H(a, c), we have a 2, x = c. But for δ > 0 sufficiently small we have x = x+δa 2 D, and a 2, x = a 2, x + δ a 2 2 E c a 2 = 0, a contradiction because the Support Theorem guarantees that a 0. Therefore, a 1 > 0. Define b = a 2 /a 1 and ĉ = c/a 1. Since (f(y), y) hypof, we have f(y)+ b, y ĉ = f(x)+ b, x. Thus, f(y)+ b, y x f(x) b f(x). Note the implication of the Support Theorem as applied to concave functions. For each interior point of the function s domain 6, there is a concave programming problem whose solution is given by this point. In fact, for any x intd and b f(x) we have: x arg max x D {f(x ) + b, x } Hence, the supporting hyperplanes gives us a way to distort (rotate around a point) the graph of the objective function in such a way that any point may be turned into a global optimum. But then, one may think about rotating the function on a special point, a point which is a solution of a restricted optimization problem. The Lagrange Theorem builds upon this principle, but instead of using only the hypograph of the function, it uses the convex sets defined by the constraint functions. This way, it is possible to create restrictions on the hyperplane which can arise as support for the restricted optimum. Indeed, under some conditions, we can uniquely identify such hyperplane. Then we can solve the global optimization problem first. 6 If the function is assumed to be continuous over the entire domain, then the property is valid for all points. An important remark should be given (we omit the proof): A concave function is continuous at the interior of its domain. 11
12 References D.P. Bertsekas, A. Nedić, and A.E. Ozdaglar. Convex analysis and optimization. Athena Scientific Belmont, Mass, J.M. Borwein and A.S. Lewis. Convex analysis and nonlinear optimization. Springer Hong Kong, M. Florenzano and C. Le Van. Finite dimensional convexity and optimization. Springer, A. Izmailov and M. Solodov. Otimização volume 1: Condições de Otimalidade, Elementos de Análise Convexa e de Dualidade. IMPA, R.T. Rockafellar. Convex analysis. Princeton University Press,
Appendix B Convex analysis
This version: 28/02/2014 Appendix B Convex analysis In this appendix we review a few basic notions of convexity and related notions that will be important for us at various times. B.1 The Hausdorff distance
More informationConvex Analysis and Optimization Chapter 2 Solutions
Convex Analysis and Optimization Chapter 2 Solutions Dimitri P. Bertsekas with Angelia Nedić and Asuman E. Ozdaglar Massachusetts Institute of Technology Athena Scientific, Belmont, Massachusetts http://www.athenasc.com
More informationSet, functions and Euclidean space. Seungjin Han
Set, functions and Euclidean space Seungjin Han September, 2018 1 Some Basics LOGIC A is necessary for B : If B holds, then A holds. B A A B is the contraposition of B A. A is sufficient for B: If A holds,
More informationOptimality Conditions for Nonsmooth Convex Optimization
Optimality Conditions for Nonsmooth Convex Optimization Sangkyun Lee Oct 22, 2014 Let us consider a convex function f : R n R, where R is the extended real field, R := R {, + }, which is proper (f never
More informationConvex Analysis and Economic Theory Winter 2018
Division of the Humanities and Social Sciences Ec 181 KC Border Convex Analysis and Economic Theory Winter 2018 Supplement A: Mathematical background A.1 Extended real numbers The extended real number
More informationSome Background Math Notes on Limsups, Sets, and Convexity
EE599 STOCHASTIC NETWORK OPTIMIZATION, MICHAEL J. NEELY, FALL 2008 1 Some Background Math Notes on Limsups, Sets, and Convexity I. LIMITS Let f(t) be a real valued function of time. Suppose f(t) converges
More informationPOLARS AND DUAL CONES
POLARS AND DUAL CONES VERA ROSHCHINA Abstract. The goal of this note is to remind the basic definitions of convex sets and their polars. For more details see the classic references [1, 2] and [3] for polytopes.
More informationChapter 2 Convex Analysis
Chapter 2 Convex Analysis The theory of nonsmooth analysis is based on convex analysis. Thus, we start this chapter by giving basic concepts and results of convexity (for further readings see also [202,
More informationLecture 2: Convex Sets and Functions
Lecture 2: Convex Sets and Functions Hyang-Won Lee Dept. of Internet & Multimedia Eng. Konkuk University Lecture 2 Network Optimization, Fall 2015 1 / 22 Optimization Problems Optimization problems are
More informationA Geometric Framework for Nonconvex Optimization Duality using Augmented Lagrangian Functions
A Geometric Framework for Nonconvex Optimization Duality using Augmented Lagrangian Functions Angelia Nedić and Asuman Ozdaglar April 15, 2006 Abstract We provide a unifying geometric framework for the
More informationAppendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS
Appendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS Here we consider systems of linear constraints, consisting of equations or inequalities or both. A feasible solution
More information2. Dual space is essential for the concept of gradient which, in turn, leads to the variational analysis of Lagrange multipliers.
Chapter 3 Duality in Banach Space Modern optimization theory largely centers around the interplay of a normed vector space and its corresponding dual. The notion of duality is important for the following
More informationCourse 212: Academic Year Section 1: Metric Spaces
Course 212: Academic Year 1991-2 Section 1: Metric Spaces D. R. Wilkins Contents 1 Metric Spaces 3 1.1 Distance Functions and Metric Spaces............. 3 1.2 Convergence and Continuity in Metric Spaces.........
More informationLocally convex spaces, the hyperplane separation theorem, and the Krein-Milman theorem
56 Chapter 7 Locally convex spaces, the hyperplane separation theorem, and the Krein-Milman theorem Recall that C(X) is not a normed linear space when X is not compact. On the other hand we could use semi
More informationThe Relation Between Pseudonormality and Quasiregularity in Constrained Optimization 1
October 2003 The Relation Between Pseudonormality and Quasiregularity in Constrained Optimization 1 by Asuman E. Ozdaglar and Dimitri P. Bertsekas 2 Abstract We consider optimization problems with equality,
More informationChapter 2: Preliminaries and elements of convex analysis
Chapter 2: Preliminaries and elements of convex analysis Edoardo Amaldi DEIB Politecnico di Milano edoardo.amaldi@polimi.it Website: http://home.deib.polimi.it/amaldi/opt-14-15.shtml Academic year 2014-15
More informationCHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS. W. Erwin Diewert January 31, 2008.
1 ECONOMICS 594: LECTURE NOTES CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS W. Erwin Diewert January 31, 2008. 1. Introduction Many economic problems have the following structure: (i) a linear function
More informationHelly's Theorem and its Equivalences via Convex Analysis
Portland State University PDXScholar University Honors Theses University Honors College 2014 Helly's Theorem and its Equivalences via Convex Analysis Adam Robinson Portland State University Let us know
More informationIntroduction to Convex and Quasiconvex Analysis
Introduction to Convex and Quasiconvex Analysis J.B.G.Frenk Econometric Institute, Erasmus University, Rotterdam G.Kassay Faculty of Mathematics, Babes Bolyai University, Cluj August 27, 2001 Abstract
More informationl(y j ) = 0 for all y j (1)
Problem 1. The closed linear span of a subset {y j } of a normed vector space is defined as the intersection of all closed subspaces containing all y j and thus the smallest such subspace. 1 Show that
More informationMaths 212: Homework Solutions
Maths 212: Homework Solutions 1. The definition of A ensures that x π for all x A, so π is an upper bound of A. To show it is the least upper bound, suppose x < π and consider two cases. If x < 1, then
More informationConvex Analysis Background
Convex Analysis Background John C. Duchi Stanford University Park City Mathematics Institute 206 Abstract In this set of notes, we will outline several standard facts from convex analysis, the study of
More informationExtreme Abridgment of Boyd and Vandenberghe s Convex Optimization
Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The
More information4. Convex Sets and (Quasi-)Concave Functions
4. Convex Sets and (Quasi-)Concave Functions Daisuke Oyama Mathematics II April 17, 2017 Convex Sets Definition 4.1 A R N is convex if (1 α)x + αx A whenever x, x A and α [0, 1]. A R N is strictly convex
More informationL p Spaces and Convexity
L p Spaces and Convexity These notes largely follow the treatments in Royden, Real Analysis, and Rudin, Real & Complex Analysis. 1. Convex functions Let I R be an interval. For I open, we say a function
More informationMath 5210, Definitions and Theorems on Metric Spaces
Math 5210, Definitions and Theorems on Metric Spaces Let (X, d) be a metric space. We will use the following definitions (see Rudin, chap 2, particularly 2.18) 1. Let p X and r R, r > 0, The ball of radius
More informationA Unified Analysis of Nonconvex Optimization Duality and Penalty Methods with General Augmenting Functions
A Unified Analysis of Nonconvex Optimization Duality and Penalty Methods with General Augmenting Functions Angelia Nedić and Asuman Ozdaglar April 16, 2006 Abstract In this paper, we study a unifying framework
More informationIn English, this means that if we travel on a straight line between any two points in C, then we never leave C.
Convex sets In this section, we will be introduced to some of the mathematical fundamentals of convex sets. In order to motivate some of the definitions, we will look at the closest point problem from
More informationElements of Convex Optimization Theory
Elements of Convex Optimization Theory Costis Skiadas August 2015 This is a revised and extended version of Appendix A of Skiadas (2009), providing a self-contained overview of elements of convex optimization
More informationDivision of the Humanities and Social Sciences. Supergradients. KC Border Fall 2001 v ::15.45
Division of the Humanities and Social Sciences Supergradients KC Border Fall 2001 1 The supergradient of a concave function There is a useful way to characterize the concavity of differentiable functions.
More informationOn Semicontinuity of Convex-valued Multifunctions and Cesari s Property (Q)
On Semicontinuity of Convex-valued Multifunctions and Cesari s Property (Q) Andreas Löhne May 2, 2005 (last update: November 22, 2005) Abstract We investigate two types of semicontinuity for set-valued
More informationMath 341: Convex Geometry. Xi Chen
Math 341: Convex Geometry Xi Chen 479 Central Academic Building, University of Alberta, Edmonton, Alberta T6G 2G1, CANADA E-mail address: xichen@math.ualberta.ca CHAPTER 1 Basics 1. Euclidean Geometry
More informationBest approximations in normed vector spaces
Best approximations in normed vector spaces Mike de Vries 5699703 a thesis submitted to the Department of Mathematics at Utrecht University in partial fulfillment of the requirements for the degree of
More informationAnalysis Finite and Infinite Sets The Real Numbers The Cantor Set
Analysis Finite and Infinite Sets Definition. An initial segment is {n N n n 0 }. Definition. A finite set can be put into one-to-one correspondence with an initial segment. The empty set is also considered
More informationOn John type ellipsoids
On John type ellipsoids B. Klartag Tel Aviv University Abstract Given an arbitrary convex symmetric body K R n, we construct a natural and non-trivial continuous map u K which associates ellipsoids to
More information1 Directional Derivatives and Differentiability
Wednesday, January 18, 2012 1 Directional Derivatives and Differentiability Let E R N, let f : E R and let x 0 E. Given a direction v R N, let L be the line through x 0 in the direction v, that is, L :=
More informationProblem Set 2: Solutions Math 201A: Fall 2016
Problem Set 2: s Math 201A: Fall 2016 Problem 1. (a) Prove that a closed subset of a complete metric space is complete. (b) Prove that a closed subset of a compact metric space is compact. (c) Prove that
More informationNotes on Complex Analysis
Michael Papadimitrakis Notes on Complex Analysis Department of Mathematics University of Crete Contents The complex plane.. The complex plane...................................2 Argument and polar representation.........................
More informationEcon Lecture 3. Outline. 1. Metric Spaces and Normed Spaces 2. Convergence of Sequences in Metric Spaces 3. Sequences in R and R n
Econ 204 2011 Lecture 3 Outline 1. Metric Spaces and Normed Spaces 2. Convergence of Sequences in Metric Spaces 3. Sequences in R and R n 1 Metric Spaces and Metrics Generalize distance and length notions
More informationConvex Analysis and Economic Theory AY Elementary properties of convex functions
Division of the Humanities and Social Sciences Ec 181 KC Border Convex Analysis and Economic Theory AY 2018 2019 Topic 6: Convex functions I 6.1 Elementary properties of convex functions We may occasionally
More informationCHAPTER 7. Connectedness
CHAPTER 7 Connectedness 7.1. Connected topological spaces Definition 7.1. A topological space (X, T X ) is said to be connected if there is no continuous surjection f : X {0, 1} where the two point set
More informationLECTURE 4 LECTURE OUTLINE
LECTURE 4 LECTURE OUTLINE Relative interior and closure Algebra of relative interiors and closures Continuity of convex functions Closures of functions Reading: Section 1.3 All figures are courtesy of
More informationAliprantis, Border: Infinite-dimensional Analysis A Hitchhiker s Guide
aliprantis.tex May 10, 2011 Aliprantis, Border: Infinite-dimensional Analysis A Hitchhiker s Guide Notes from [AB2]. 1 Odds and Ends 2 Topology 2.1 Topological spaces Example. (2.2) A semimetric = triangle
More informationConvexity in R n. The following lemma will be needed in a while. Lemma 1 Let x E, u R n. If τ I(x, u), τ 0, define. f(x + τu) f(x). τ.
Convexity in R n Let E be a convex subset of R n. A function f : E (, ] is convex iff f(tx + (1 t)y) (1 t)f(x) + tf(y) x, y E, t [0, 1]. A similar definition holds in any vector space. A topology is needed
More informationREVIEW OF ESSENTIAL MATH 346 TOPICS
REVIEW OF ESSENTIAL MATH 346 TOPICS 1. AXIOMATIC STRUCTURE OF R Doğan Çömez The real number system is a complete ordered field, i.e., it is a set R which is endowed with addition and multiplication operations
More information15 Real Analysis II Sequences and Limits , Math for Economists Fall 2004 Lecture Notes, 10/21/2004
14.102, Math for Economists Fall 2004 Lecture Notes, 10/21/2004 These notes are primarily based on those written by Andrei Bremzen for 14.102 in 2002/3, and by Marek Pycia for the MIT Math Camp in 2003/4.
More information1 Topology Definition of a topology Basis (Base) of a topology The subspace topology & the product topology on X Y 3
Index Page 1 Topology 2 1.1 Definition of a topology 2 1.2 Basis (Base) of a topology 2 1.3 The subspace topology & the product topology on X Y 3 1.4 Basic topology concepts: limit points, closed sets,
More informationConvex Analysis and Optimization Chapter 4 Solutions
Convex Analysis and Optimization Chapter 4 Solutions Dimitri P. Bertsekas with Angelia Nedić and Asuman E. Ozdaglar Massachusetts Institute of Technology Athena Scientific, Belmont, Massachusetts http://www.athenasc.com
More informationSome Background Material
Chapter 1 Some Background Material In the first chapter, we present a quick review of elementary - but important - material as a way of dipping our toes in the water. This chapter also introduces important
More informationLecture 8: Basic convex analysis
Lecture 8: Basic convex analysis 1 Convex sets Both convex sets and functions have general importance in economic theory, not only in optimization. Given two points x; y 2 R n and 2 [0; 1]; the weighted
More informationA SET OF LECTURE NOTES ON CONVEX OPTIMIZATION WITH SOME APPLICATIONS TO PROBABILITY THEORY INCOMPLETE DRAFT. MAY 06
A SET OF LECTURE NOTES ON CONVEX OPTIMIZATION WITH SOME APPLICATIONS TO PROBABILITY THEORY INCOMPLETE DRAFT. MAY 06 CHRISTIAN LÉONARD Contents Preliminaries 1 1. Convexity without topology 1 2. Convexity
More informationGeometry and topology of continuous best and near best approximations
Journal of Approximation Theory 105: 252 262, Geometry and topology of continuous best and near best approximations Paul C. Kainen Dept. of Mathematics Georgetown University Washington, D.C. 20057 Věra
More informationOptimization and Optimal Control in Banach Spaces
Optimization and Optimal Control in Banach Spaces Bernhard Schmitzer October 19, 2017 1 Convex non-smooth optimization with proximal operators Remark 1.1 (Motivation). Convex optimization: easier to solve,
More informationReal Analysis Math 131AH Rudin, Chapter #1. Dominique Abdi
Real Analysis Math 3AH Rudin, Chapter # Dominique Abdi.. If r is rational (r 0) and x is irrational, prove that r + x and rx are irrational. Solution. Assume the contrary, that r+x and rx are rational.
More informationCONSTRAINT QUALIFICATIONS, LAGRANGIAN DUALITY & SADDLE POINT OPTIMALITY CONDITIONS
CONSTRAINT QUALIFICATIONS, LAGRANGIAN DUALITY & SADDLE POINT OPTIMALITY CONDITIONS A Dissertation Submitted For The Award of the Degree of Master of Philosophy in Mathematics Neelam Patel School of Mathematics
More informationLecture Notes in Advanced Calculus 1 (80315) Raz Kupferman Institute of Mathematics The Hebrew University
Lecture Notes in Advanced Calculus 1 (80315) Raz Kupferman Institute of Mathematics The Hebrew University February 7, 2007 2 Contents 1 Metric Spaces 1 1.1 Basic definitions...........................
More informationChapter 2 Metric Spaces
Chapter 2 Metric Spaces The purpose of this chapter is to present a summary of some basic properties of metric and topological spaces that play an important role in the main body of the book. 2.1 Metrics
More information8. Conjugate functions
L. Vandenberghe EE236C (Spring 2013-14) 8. Conjugate functions closed functions conjugate function 8-1 Closed set a set C is closed if it contains its boundary: x k C, x k x = x C operations that preserve
More informationIntroduction to Machine Learning Lecture 7. Mehryar Mohri Courant Institute and Google Research
Introduction to Machine Learning Lecture 7 Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu Convex Optimization Differentiation Definition: let f : X R N R be a differentiable function,
More informationBoundary Behavior of Excess Demand Functions without the Strong Monotonicity Assumption
Boundary Behavior of Excess Demand Functions without the Strong Monotonicity Assumption Chiaki Hara April 5, 2004 Abstract We give a theorem on the existence of an equilibrium price vector for an excess
More informationLECTURE 15: COMPLETENESS AND CONVEXITY
LECTURE 15: COMPLETENESS AND CONVEXITY 1. The Hopf-Rinow Theorem Recall that a Riemannian manifold (M, g) is called geodesically complete if the maximal defining interval of any geodesic is R. On the other
More informationEC 521 MATHEMATICAL METHODS FOR ECONOMICS. Lecture 1: Preliminaries
EC 521 MATHEMATICAL METHODS FOR ECONOMICS Lecture 1: Preliminaries Murat YILMAZ Boğaziçi University In this lecture we provide some basic facts from both Linear Algebra and Real Analysis, which are going
More informationIntroduction to Real Analysis Alternative Chapter 1
Christopher Heil Introduction to Real Analysis Alternative Chapter 1 A Primer on Norms and Banach Spaces Last Updated: March 10, 2018 c 2018 by Christopher Heil Chapter 1 A Primer on Norms and Banach Spaces
More informationWeek 3: Faces of convex sets
Week 3: Faces of convex sets Conic Optimisation MATH515 Semester 018 Vera Roshchina School of Mathematics and Statistics, UNSW August 9, 018 Contents 1. Faces of convex sets 1. Minkowski theorem 3 3. Minimal
More informationWeek 5 Lectures 13-15
Week 5 Lectures 13-15 Lecture 13 Definition 29 Let Y be a subset X. A subset A Y is open in Y if there exists an open set U in X such that A = U Y. It is not difficult to show that the collection of all
More informationConvex Analysis and Economic Theory Winter 2018
Division of the Humanities and Social Sciences Ec 181 KC Border Convex Analysis and Economic Theory Winter 2018 Topic 0: Vector spaces 0.1 Basic notation Here are some of the fundamental sets and spaces
More informationExtended Monotropic Programming and Duality 1
March 2006 (Revised February 2010) Report LIDS - 2692 Extended Monotropic Programming and Duality 1 by Dimitri P. Bertsekas 2 Abstract We consider the problem minimize f i (x i ) subject to x S, where
More informationNOTES ON VECTOR-VALUED INTEGRATION MATH 581, SPRING 2017
NOTES ON VECTOR-VALUED INTEGRATION MATH 58, SPRING 207 Throughout, X will denote a Banach space. Definition 0.. Let ϕ(s) : X be a continuous function from a compact Jordan region R n to a Banach space
More informationg 2 (x) (1/3)M 1 = (1/3)(2/3)M.
COMPACTNESS If C R n is closed and bounded, then by B-W it is sequentially compact: any sequence of points in C has a subsequence converging to a point in C Conversely, any sequentially compact C R n is
More informationConvex Sets. Prof. Dan A. Simovici UMB
Convex Sets Prof. Dan A. Simovici UMB 1 / 57 Outline 1 Closures, Interiors, Borders of Sets in R n 2 Segments and Convex Sets 3 Properties of the Class of Convex Sets 4 Closure and Interior Points of Convex
More informationChapter 2. Convex Sets: basic results
Chapter 2 Convex Sets: basic results In this chapter, we introduce one of the most important tools in the mathematical approach to Economics, namely the theory of convex sets. Almost every situation we
More informationDiscrete Geometry. Problem 1. Austin Mohr. April 26, 2012
Discrete Geometry Austin Mohr April 26, 2012 Problem 1 Theorem 1 (Linear Programming Duality). Suppose x, y, b, c R n and A R n n, Ax b, x 0, A T y c, and y 0. If x maximizes c T x and y minimizes b T
More informationLecture 3. Econ August 12
Lecture 3 Econ 2001 2015 August 12 Lecture 3 Outline 1 Metric and Metric Spaces 2 Norm and Normed Spaces 3 Sequences and Subsequences 4 Convergence 5 Monotone and Bounded Sequences Announcements: - Friday
More informationMath 273a: Optimization Subgradients of convex functions
Math 273a: Optimization Subgradients of convex functions Made by: Damek Davis Edited by Wotao Yin Department of Mathematics, UCLA Fall 2015 online discussions on piazza.com 1 / 20 Subgradients Assumptions
More informationThe general programming problem is the nonlinear programming problem where a given function is maximized subject to a set of inequality constraints.
1 Optimization Mathematical programming refers to the basic mathematical problem of finding a maximum to a function, f, subject to some constraints. 1 In other words, the objective is to find a point,
More informationMathematics for Economists
Mathematics for Economists Victor Filipe Sao Paulo School of Economics FGV Metric Spaces: Basic Definitions Victor Filipe (EESP/FGV) Mathematics for Economists Jan.-Feb. 2017 1 / 34 Definitions and Examples
More informationHamiltonian Mechanics
Chapter 3 Hamiltonian Mechanics 3.1 Convex functions As background to discuss Hamiltonian mechanics we discuss convexity and convex functions. We will also give some applications to thermodynamics. We
More informationMetric Spaces and Topology
Chapter 2 Metric Spaces and Topology From an engineering perspective, the most important way to construct a topology on a set is to define the topology in terms of a metric on the set. This approach underlies
More informationLocal strong convexity and local Lipschitz continuity of the gradient of convex functions
Local strong convexity and local Lipschitz continuity of the gradient of convex functions R. Goebel and R.T. Rockafellar May 23, 2007 Abstract. Given a pair of convex conjugate functions f and f, we investigate
More informationNonlinear Programming 3rd Edition. Theoretical Solutions Manual Chapter 6
Nonlinear Programming 3rd Edition Theoretical Solutions Manual Chapter 6 Dimitri P. Bertsekas Massachusetts Institute of Technology Athena Scientific, Belmont, Massachusetts 1 NOTE This manual contains
More information6.254 : Game Theory with Engineering Applications Lecture 7: Supermodular Games
6.254 : Game Theory with Engineering Applications Lecture 7: Asu Ozdaglar MIT February 25, 2010 1 Introduction Outline Uniqueness of a Pure Nash Equilibrium for Continuous Games Reading: Rosen J.B., Existence
More informationDO NOT OPEN THIS QUESTION BOOKLET UNTIL YOU ARE TOLD TO DO SO
QUESTION BOOKLET EECS 227A Fall 2009 Midterm Tuesday, Ocotober 20, 11:10-12:30pm DO NOT OPEN THIS QUESTION BOOKLET UNTIL YOU ARE TOLD TO DO SO You have 80 minutes to complete the midterm. The midterm consists
More informationDS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.
DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1
More informationOptimization Theory. A Concise Introduction. Jiongmin Yong
October 11, 017 16:5 ws-book9x6 Book Title Optimization Theory 017-08-Lecture Notes page 1 1 Optimization Theory A Concise Introduction Jiongmin Yong Optimization Theory 017-08-Lecture Notes page Optimization
More informationConvex Optimization Theory. Chapter 3 Exercises and Solutions: Extended Version
Convex Optimization Theory Chapter 3 Exercises and Solutions: Extended Version Dimitri P. Bertsekas Massachusetts Institute of Technology Athena Scientific, Belmont, Massachusetts http://www.athenasc.com
More informationREAL AND COMPLEX ANALYSIS
REAL AND COMPLE ANALYSIS Third Edition Walter Rudin Professor of Mathematics University of Wisconsin, Madison Version 1.1 No rights reserved. Any part of this work can be reproduced or transmitted in any
More informationPart III. 10 Topological Space Basics. Topological Spaces
Part III 10 Topological Space Basics Topological Spaces Using the metric space results above as motivation we will axiomatize the notion of being an open set to more general settings. Definition 10.1.
More informationAnalysis III. Exam 1
Analysis III Math 414 Spring 27 Professor Ben Richert Exam 1 Solutions Problem 1 Let X be the set of all continuous real valued functions on [, 1], and let ρ : X X R be the function ρ(f, g) = sup f g (1)
More informationDefinitions & Theorems
Definitions & Theorems Math 147, Fall 2009 December 19, 2010 Contents 1 Logic 2 1.1 Sets.................................................. 2 1.2 The Peano axioms..........................................
More information2 (Bonus). Let A X consist of points (x, y) such that either x or y is a rational number. Is A measurable? What is its Lebesgue measure?
MA 645-4A (Real Analysis), Dr. Chernov Homework assignment 1 (Due 9/5). Prove that every countable set A is measurable and µ(a) = 0. 2 (Bonus). Let A consist of points (x, y) such that either x or y is
More informationLECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE
LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE CONVEX ANALYSIS AND DUALITY Basic concepts of convex analysis Basic concepts of convex optimization Geometric duality framework - MC/MC Constrained optimization
More informationLECTURE SLIDES ON BASED ON CLASS LECTURES AT THE CAMBRIDGE, MASS FALL 2007 BY DIMITRI P. BERTSEKAS.
LECTURE SLIDES ON CONVEX ANALYSIS AND OPTIMIZATION BASED ON 6.253 CLASS LECTURES AT THE MASSACHUSETTS INSTITUTE OF TECHNOLOGY CAMBRIDGE, MASS FALL 2007 BY DIMITRI P. BERTSEKAS http://web.mit.edu/dimitrib/www/home.html
More informationMATH 131A: REAL ANALYSIS (BIG IDEAS)
MATH 131A: REAL ANALYSIS (BIG IDEAS) Theorem 1 (The Triangle Inequality). For all x, y R we have x + y x + y. Proposition 2 (The Archimedean property). For each x R there exists an n N such that n > x.
More informationConvex Geometry. Carsten Schütt
Convex Geometry Carsten Schütt November 25, 2006 2 Contents 0.1 Convex sets... 4 0.2 Separation.... 9 0.3 Extreme points..... 15 0.4 Blaschke selection principle... 18 0.5 Polytopes and polyhedra.... 23
More informationMATH 54 - TOPOLOGY SUMMER 2015 FINAL EXAMINATION. Problem 1
MATH 54 - TOPOLOGY SUMMER 2015 FINAL EXAMINATION ELEMENTS OF SOLUTION Problem 1 1. Let X be a Hausdorff space and K 1, K 2 disjoint compact subsets of X. Prove that there exist disjoint open sets U 1 and
More informationU e = E (U\E) e E e + U\E e. (1.6)
12 1 Lebesgue Measure 1.2 Lebesgue Measure In Section 1.1 we defined the exterior Lebesgue measure of every subset of R d. Unfortunately, a major disadvantage of exterior measure is that it does not satisfy
More informationHandout 2: Elements of Convex Analysis
ENGG 5501: Foundations of Optimization 2018 19 First Term Handout 2: Elements of Convex Analysis Instructor: Anthony Man Cho So September 10, 2018 As briefly mentioned in Handout 1, the notion of convexity
More informationEC9A0: Pre-sessional Advanced Mathematics Course. Lecture Notes: Unconstrained Optimisation By Pablo F. Beker 1
EC9A0: Pre-sessional Advanced Mathematics Course Lecture Notes: Unconstrained Optimisation By Pablo F. Beker 1 1 Infimum and Supremum Definition 1. Fix a set Y R. A number α R is an upper bound of Y if
More informationConvex Optimization Theory
Convex Optimization Theory A SUMMARY BY DIMITRI P. BERTSEKAS We provide a summary of theoretical concepts and results relating to convex analysis, convex optimization, and duality theory. In particular,
More informationLECTURE 3 LECTURE OUTLINE
LECTURE 3 LECTURE OUTLINE Differentiable Conve Functions Conve and A ne Hulls Caratheodory s Theorem Reading: Sections 1.1, 1.2 All figures are courtesy of Athena Scientific, and are used with permission.
More information