An Easy Path to Convex Analysis and Applications

Transcription

1 M & C Morgan & Claypool Publishers An Easy Path to Convex Analysis and Applications Boris S. Mordukhovich Nguyen Mau Nam SYNTHESIS LECTURES ON MATHEMATICS AND STATISTICS Steven G. Krantz, Series Editor

2

3 An Easy Path to Convex Analysis and Applications

4 Synthesis Lectures on Mathematics and Statistics Editor Steven G. Krantz, Washington University, St. Louis An Easy Path to Convex Analysis and Applications Boris S. Mordukhovich and Nguyen Mau Nam 2014 Applications of Affine and Weyl Geometry Eduardo García-Río, Peter Gilkey, Stana Nikčević, and Ramón Vázquez-Lorenzo 2013 Essentials of Applied Mathematics for Engineers and Scientists, Second Edition Robert G. Watts 2012 Chaotic Maps: Dynamics, Fractals, and Rapid Fluctuations Goong Chen and Yu Huang 2011 Matrices in Engineering Problems Marvin J. Tobias 2011 e Integral: A Crux for Analysis Steven G. Krantz 2011 Statistics is Easy! Second Edition Dennis Shasha and Manda Wilson 2010 Lectures on Financial Mathematics: Discrete Asset Pricing Greg Anderson and Alec N. Kercheval 2010

5 Jordan Canonical Form: eory and Practice Steven H. Weintraub 2009 iii e Geometry of Walker Manifolds Miguel Brozos-Vázquez, Eduardo García-Río, Peter Gilkey, Stana Nikčević, and Ramón Vázquez-Lorenzo 2009 An Introduction to Multivariable Mathematics Leon Simon 2008 Jordan Canonical Form: Application to Differential Equations Steven H. Weintraub 2008 Statistics is Easy! Dennis Shasha and Manda Wilson 2008 A Gyrovector Space Approach to Hyperbolic Geometry Abraham Albert Ungar 2008

6 Copyright 2014 by Morgan & Claypool All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means electronic, mechanical, photocopy, recording, or any other except for brief quotations in printed reviews, without the prior permission of the publisher. An Easy Path to Convex Analysis and Applications Boris S. Mordukhovich and Nguyen Mau Nam ISBN: ISBN: paperback ebook DOI /S00554ED1V01Y201312MAS014 A Publication in the Morgan & Claypool Publishers series SYNTHESIS LECTURES ON MATHEMATICS AND STATISTICS Lecture #14 Series Editor: Steven G. Krantz, Washington University, St. Louis Series ISSN Synthesis Lectures on Mathematics and Statistics Print Electronic

7 An Easy Path to Convex Analysis and Applications Boris S. Mordukhovich Wayne State University Nguyen Mau Nam Portland State University SYNTHESIS LECTURES ON MATHEMATICS AND STATISTICS #14 & M C Morgan & claypool publishers

8 ABSTRACT Convex optimization has an increasing impact on many areas of mathematics, applied sciences, and practical applications. It is now being taught at many universities and being used by researchers of different fields. As convex analysis is the mathematical foundation for convex optimization, having deep knowledge of convex analysis helps students and researchers apply its tools more effectively. e main goal of this book is to provide an easy access to the most fundamental parts of convex analysis and its applications to optimization. Modern techniques of variational analysis are employed to clarify and simplify some basic proofs in convex analysis and build the theory of generalized differentiation for convex functions and sets in finite dimensions. We also present new applications of convex analysis to location problems in connection with many interesting geometric problems such as the Fermat-Torricelli problem, the Heron problem, the Sylvester problem, and their generalizations. Of course, we do not expect to touch every aspect of convex analysis, but the book consists of sufficient material for a first course on this subject. It can also serve as supplemental reading material for a course on convex optimization and applications. KEYWORDS Affine set, Carathéodory theorem, convex function, convex set, directional derivative, distance function, Fenchel conjugate, Fermat-Torricelli problem, generalized differentiation, Helly theorem, minimal time function, Nash equilibrium, normal cone, Radon theorem, optimal value function, optimization, smallest enclosing ball problem, set-valued mapping, subdifferential, subgradient, subgradient algorithm, support function, Weiszfeld algorithm

9 vii e first author dedicates this book to the loving memory of his father SHOLIM MORDUKHOVICH ( ), a kind man and a brave warrior. e second author dedicates this book to the memory of his parents.

10

11 ix Contents Preface xi Acknowledgments xiii List of Symbols xv 1 Convex Sets and Functions Preliminaries Convex Sets Convex Functions Relative Interiors of Convex Sets e Distance Function Exercises for Chapter Subdifferential Calculus Convex Separation Normals to Convex Sets Lipschitz Continuity of Convex Functions Subgradients of Convex Functions Basic Calculus Rules Subgradients of Optimal Value Functions Subgradients of Support Functions Fenchel Conjugates Directional Derivatives Subgradients of Supremum Functions Exercises for Chapter Remarkable Consequences of Convexity Characterizations of Differentiability Carathéodory eorem and Farkas Lemma Radon eorem and Helly eorem

12 x 3.4 Tangents to Convex Sets Mean Value eorems Horizon Cones Minimal Time Functions and Minkowski Gauge Subgradients of Minimal Time Functions Nash Equilibrium Exercises for Chapter Applications to Optimization and Location Problems Lower Semicontinuity and Existence of Minimizers Optimality Conditions Subgradient Methods in Convex Optimization e Fermat-Torricelli Problem A Generalized Fermat-Torricelli Problem A Generalized Sylvester Problem Exercises for Chapter Solutions and Hints for Exercises Bibliography Authors Biographies Index

13 xi Preface Some geometric properties of convex sets and, to a lesser extent, of convex functions had been studied before the 1960s by many outstanding mathematicians, first of all by Hermann Minkowski and Werner Fenchel. At the beginning of the 1960s convex analysis was greatly developed in the works of R. Tyrrell Rockafellar and Jean-Jacques Moreau who initiated a systematic study of this new field. As a fundamental part of variational analysis, convex analysis contains a generalized differentiation theory that can be used to study a large class of mathematical models with no differentiability assumptions imposed on their initial data. e importance of convex analysis for many applications in which convex optimization is the first to name has been well recognized. e presence of convexity makes it possible not only to comprehensively investigate qualitative properties of optimal solutions and derive necessary and sufficient conditions for optimality but also to develop effective numerical algorithms to solve convex optimization problems, even with nondifferentiable data. Convex analysis and optimization have an increasing impact on many areas of mathematics and applications including control systems, estimation and signal processing, communications and networks, electronic circuit design, data analysis and modeling, statistics, economics and finance, etc. ere are several fundamental books devoted to different aspects of convex analysis and optimization. Among them we mention Convex Analysis by Rockafellar [26], Convex Analysis and Minimization Algorithms (in two volumes) by Hiriart-Urruty and Lemaréchal [8] and its abridge version [9], Convex Analysis and Nonlinear Optimization by Borwein and Lewis [4], Introductory Lectures on Convex Optimization by Nesterov [21], and Convex Optimization by Boyd and Vandenberghe [3] as well as other books listed in the bibliography below. In this big picture of convex analysis and optimization, our book serves as a bridge for students and researchers who have just started using convex analysis to reach deeper topics in the field. We give detailed proofs for most of the results presented in the book and also include many figures and exercises for better understanding the material. e powerful geometric approach developed in modern variational analysis is adopted and simplified in the convex case in order to provide the reader with an easy path to access generalized differentiation of convex objects in finite dimensions. In this way, the book also serves as a starting point for the interested reader to continue the study of nonconvex variational analysis and applications. It can be of interest from this viewpoint to experts in convex and variational analysis. Finally, the application part of this book not only concerns the classical topics of convex optimization related to optimality conditions and subgradient algorithms but also presents some recent while easily understandable qualitative and numerical results for important location problems.

14 xii PREFACE e book consists of four chapters and is organized as follows. In Chapter 1 we study fundamental properties of convex sets and functions while paying particular attention to classes of convex functions important in optimization. Chapter 2 is mainly devoted to developing basic calculus rules for normals to convex sets and subgradients of convex functions that are in the mainstream of convex theory. Chapter 3 concerns some additional topics of convex analysis that are largely used in applications. Chapter 4 is fully devoted to applications of basic results of convex analysis to problems of convex optimization and selected location problems considered from both qualitative and numerical viewpoints. Finally, we present at the end of the book Solutions and Hints to selected exercises. Exercises are given at the end of each chapter while figures and examples are provided throughout the whole text. e list of references contains books and selected papers, which are closely related to the topics considered in the book and may be helpful to the reader for advanced studies of convex analysis, its applications, and further extensions. Since only elementary knowledge in linear algebra and basic calculus is required, this book can be used as a textbook for both undergraduate and graduate level courses in convex analysis and its applications. In fact, the authors have used these lecture notes for teaching such classes in their universities as well as while visiting some other schools. We hope that the book will make convex analysis more accessible to large groups of undergraduate and graduate students, researchers in different disciplines, and practitioners. Boris S. Mordukhovich and Nguyen Mau Nam December 2013

15 xiii Acknowledgments e first author acknowledges continued support by grants from the National Science Foundation and the second author acknowledges support by the Simons Foundation. Boris S. Mordukhovich and Nguyen Mau Nam December 2013

16

17 xv List of Symbols R the real numbers R D.1; 1 the extended real line R C the nonnegative real numbers R > the positive real numbers N the positive integers co convex hull of aff affine hull of int interior of ri relative interior of span linear subspace generated by cone cone generated by K convex cone generated by dim dimension of closure of bd boundary of IB closed unit ball IB. NxI r/ closed ball with center Nx and radius r dom f domain of f epi f epigraph of f gph F graph of mapping F LŒa; b line connecting a and b d.xi / distance from x to.xi / projection of x to hx; yi inner product of x and y A adjoint/transpose of linear mapping/matrix A N. NxI / normal cone to at Nx ker A kernel of linear mapping A D F. Nx; Ny/ coderivative to F at. Nx; Ny/ T. NxI / tangent cone to at Nx F 1 horizon/asymptotic cone of F T F minimal time function defined by constant dynamic F and target set L.x; / Lagrangian Minkowski gauge of F F

18 xvi LIST OF SYMBOLS F.I / generalized projection defined by the minimal time function G F. Nx; t/ generalized ball defined by dynamic F with center Nx and radius t

19 C H A P T E R 1 Convex Sets and Functions 1 is chapter presents definitions, examples, and basic properties of convex sets and functions in the Euclidean space R n and also contains some related material. 1.1 PRELIMINARIES We start with reviewing classical notions and properties of the Euclidean space R n. e proofs of the results presented in this section can be found in standard books on advanced calculus and linear algebra. Let us denote by R n the set of all ntuples of real numbers x D.x 1 ; : : : ; x n /. en R n is a linear space with the following operations:.x 1 ; : : : ; x n / C.y 1 ; : : : ; y n / D.x 1 C y 1 ; : : : ; x n C y n /;.x 1 ; : : : ; x n / D.x 1 ; : : : ; x n /; where.x 1 ; : : : ; x n /;.y 1 ; : : : ; y n / 2 R n and 2 R. e zero element of R n and the number zero of R are often denoted by the same notation 0 if no confusion arises. For any x D.x 1 ; : : : ; x n / 2 R n, we identify it with the column vector x D Œx 1 ; : : : ; x n T, where the symbol T stands for vector transposition. Given x D.x 1 ; : : : ; x n / 2 R n and y D.y 1 ; : : : ; y n / 2 R n, the inner product of x and y is defined by hx; yi WD nx x i y i : e following proposition lists some important properties of the inner product in R n. Proposition 1.1 For x; y; z 2 R n and 2 R, we have: (i) hx; xi 0, and hx; xi D 0 if and only if x D 0. (ii) hx; yi D hy; xi. (iii) hx; yi D hx; yi. (iv) hx; y C zi D hx; yi C hx; zi. e Euclidean norm of x D.x 1 ; : : : ; x n / 2 R n is defined by q kxk WD x1 2 C : : : C x2 n :

20 2 1. CONVEX SETS AND FUNCTIONS It follows directly from the definition that kxk D p hx; xi. Proposition 1.2 For any x; y 2 R n and 2 R, we have: (i) kxk 0, and kxk D 0 if and only if x D 0. (ii) kxk D jj kxk. (iii) kx C yk kxk C kyk.the triangle inequality/. (iv) jhx; yij kxk kyk.the Cauchy-Schwarz inequality/. Using the Euclidean norm allows us to introduce the balls in R n, which can be used to define other topological notions in R n. Definition 1.3 e CLOSED BALL centered at Nx with radius r 0 and the CLOSED UNIT BALL of R n are defined, respectively, by IB. NxI r/ WD x 2 R n ˇˇ kx Nxk r and IB WD x 2 R n ˇˇ kxk 1 : It is easy to see that IB D IB.0I 1/ and IB. NxI r/ D Nx C rib. Definition 1.4 Let R n. en Nx is an INTERIOR POINT of if there is ı > 0 such that IB. NxI ı/ : e set of all interior points of is denoted by int. Moreover, is said to be OPEN if every point of is its interior point. We get that is open if and only if for every Nx 2 there is ı > 0 such that IB. NxI ı/. It is obvious that the empty set ; and the whole space R n are open. Furthermore, any open ball B. NxI r/ WD fx 2 R n j kx Nxk < rg centered at Nx with radius r is open. Definition 1.5 A set R n is CLOSED if its complement c D R n n is open in R n. It follows that the empty set, the whole space, and any ball IB. NxI r/ are closed in R n. Proposition 1.6 (i) e union of any collection of open sets in R n is open. (ii) e intersection of any finite collection of open sets in R n is open. (iii) e intersection of any collection of closed sets in R n is closed. (iv) e union of any finite collection of closed sets in R n is closed. Definition 1.7 Let fx k g be a sequence in R n. We say that fx k g CONVERGES to Nx if kx k Nxk! 0 as k! 1. In this case we write lim x k D Nx: k!1

21 1.1. PRELIMINARIES 3 is notion allows us to define the following important topological concepts for sets. Definition 1.8 Let be a nonempty subset of R n. en: (i) e CLOSURE of, denoted by or cl, is the collection of limits of all convergent sequences belonging to. (ii) e BOUNDARY of, denoted by bd, is the set n int. We can see that the closure of is the intersection of all closed sets containing and that the interior of is the union of all open sets contained in. It follows from the definition that Nx 2 if and only if for any ı > 0 we have IB. NxI ı/ \ ;. Furthermore, Nx 2 bd if and only if for any ı > 0 the closed ball IB. NxI / intersects both sets and its complement c. Definition 1.9 Let fx k g be a sequence in R n and let fk`g be a strictly increasing sequence of positive integers. en the new sequence fx k`g is called a SUBSEQUENCE of fx k g. We say that a set is bounded if it is contained in a ball centered at the origin with some radius r > 0, i.e., IB.0I r/. us a sequence fx k g is bounded if there is r > 0 with kx k k r for all k 2 N : e following important result is known as the Bolzano-Weierstrass theorem. eorem 1.10 Any bounded sequence in R n contains a convergent subsequence. e next concept plays a very significant role in analysis and optimization. Definition 1.11 We say that a set is COMPACT in R n if every sequence in contains a subsequence converging to some point in. e following result is a consequence of the Bolzano-Weierstrass theorem. eorem 1.12 A subset of R n is compact if and only if it is closed and bounded. For subsets, 1, and 2 of R n and for 2 R, we define the operations: 1 C 2 WD x C y ˇˇ x 2 1; y 2 2 ; WD x ˇˇ x 2 : e next proposition can be proved easily. Proposition 1.13 Let 1 and 2 be two subsets of R n. (i) If 1 is open or 2 is open, then 1 C 2 is open. (ii) If 1 is closed and 2 is compact, then 2 C 2 is closed.

22 4 1. CONVEX SETS AND FUNCTIONS Recall now the notions of bounds for subsets of the real line. Definition 1.14 have Let D be a subset of the real line. A number m 2 R is a LOWER BOUND of D if we x m for all x 2 D: If the set D has a lower bound, then it is BOUNDED BELOW. Similarly, a number M 2 R is an UPPER BOUND of D if x M for all x 2 D; and D is BOUNDED ABOVE if it has an upper bound. Furthermore, we say that the set D is BOUNDED if it is simultaneously bounded below and above. Now we are ready to define the concepts of infimum and supremum of sets. Definition 1.15 Let D R be nonempty and bounded below. e infimum of D, denoted by inf D, is the greatest lower bound of D. When D is nonempty and bounded above, its supremum, denoted by sup D, is the least upper bound of D. If D is not bounded below.resp. above/, then we set inf D WD 1.resp. sup D WD 1/. We also use the convention that inf ; WD 1 and sup ; WD 1. e following fundamental axiom ensures that these notions are well-defined. Completeness Axiom. For every nonempty subset D of R that is bounded above, the least upper bound of D exists as a real number. Using the Completeness Axiom, it is easy to see that if a nonempty set is bounded below, then its greatest lower bound exists as a real number. roughout the book we consider for convenience extended-real-valued functions, which take values in R WD.1; 1. e usual conventions of extended arithmetics are that a C 1 D 1 for any a 2 R, 1 C 1 D 1, and t 1 D 1 for t > 0. Definition 1.16 Let f W! R be an extended-real-valued function and let Nx 2 with f. Nx/ < 1. en f is CONTINUOUS at Nx if for any > 0 there is ı > 0 such that jf.x/ f. Nx/j < whenever kx Nxk < ı; x 2 : We say that f is continuous on if it is continuous at every point of. It is obvious from the definition that if f W! R is continuous at Nx (with f. Nx/ < 1), then it is finite on the intersection of and a ball centered at Nx with some radius r > 0. Furthermore, f W! R is continuous at Nx (with f. Nx/ < 1) if and only if for every sequence fx k g in converging to Nx the sequence ff.x k /g converges to f. Nx/.

23 1.2. CONVEX SETS 5 Definition 1.17 Let f W! R and let Nx 2 with f. Nx/ < 1. We say that f has a LOCAL MIN- IMUM at Nx relative to if there is ı > 0 such that f.x/ f. Nx/ for all x 2 IB. NxI ı/ \ : We also say that f has a GLOBAL/ABSOLUTE MINIMUM at Nx relative to if f.x/ f. Nx/ for all x 2 : e notions of local and global maxima can be defined similarly. Finally in this section, we formulate a fundamental result of mathematical analysis and optimization known as the Weierstrass existence theorem. eorem 1.18 Let f W! R be a continuous function, where is a nonempty, compact subset of R n. en there exist Nx 2 and Nu 2 such that f. Nx/ D inf f.x/ ˇˇ x 2 and f. Nu/ D sup f.x/ ˇˇ x 2 : In Section 4.1 we present some unilateral versions of eorem CONVEX SETS We start the study of convexity with sets and then proceed to functions. Geometric ideas play an underlying role in convex analysis, its extensions, and applications. us we implement the geometric approach in this book. Given two elements a and b in R n, define the interval/line segment Œa; b WD a C.1 /b ˇˇ 2 Œ0; 1 : Note that if a D b, then this interval reduces to a singleton Œa; b D fag. Definition 1.19 A subset of R n is CONVEX if Œa; b whenever a; b 2. Equivalently, is convex if a C.1 /b 2 for all a; b 2 and 2 Œ0; 1. Given! 1 ; : : : ;! m 2 R n, the element x D P m i! i, where P m i D 1 and i 0 for some m 2 N, is called a convex combination of! 1 ; : : : ;! m. Proposition 1.20 elements. A subset of R n is convex if and only if it contains all convex combinations of its

24 6 1. CONVEX SETS AND FUNCTIONS Figure 1.1: Convex set and nonconvex set. Proof. e sufficient condition is trivial. To justify the necessity, we show by induction that any convex combination x D P m i! i of elements in is an element of. is conclusion follows directly from the definition for m D 1; 2. Fix now a positive integer m 2 and suppose that every convex combination of k 2 N elements from, where k m, belongs to. Form the convex combination y WD mc1 X i! i ; mc1 X i D 1; i 0 and observe that if mc1 D 1, then 1 D 2 D : : : D m D 0, so y D! mc1 2. In the case where mc1 < 1 we get the representations mx i D 1 mc1 which imply in turn the inclusion and mx i 1 mc1 D 1; It yields therefore the relationships y D.1 mc1 / mx z WD mx i 1 mc1! i 2 : i 1 mc1! i C mc1! mc1 D.1 mc1 /z C mc1! mc1 2 and thus completes the proof of the proposition. Proposition 1.21 Let 1 be a convex subset of R n and let 2 be a convex subset of R p. en the Cartesian product 1 2 is a convex subset of R n R p.

25 1.2. CONVEX SETS 7 Proof. Fix a D.a 1 ; a 2 /; b D.b 1 ; b 2 / 2 1 2, and 2.0; 1/. en we have a 1 ; b and a 2 ; b e convexity of 1 and 2 gives us which implies therefore that a i C.1 /b i 2 i for i D 1; 2; a C.1 /b D a 1 C.1 /b 1 ; a 2 C.1 /b : us the Cartesian product 1 2 is convex. Let us continue with the definition of affine mappings. Definition 1.22 A mapping B W R n! R p is AFFINE if there exist a linear mapping A W R n! R p and an element b 2 R p such that B.x/ D A.x/ C b for all x 2 R n. Every linear mapping is affine. Moreover, B W R n! R p is affine if and only if B.x C.1 /y/ D B.x/ C.1 /B.y/ for all x; y 2 R n and 2 R : Now we show that set convexity is preserved under affine operations. Proposition 1.23 Let B W R n! R p be an affine mapping. Suppose that is a convex subset of R n and is a convex subset of R p. en B. / is a convex subset of R p and B 1./ is a convex subset of R n. Proof. Fix any a; b 2 B. / and 2.0; 1/. en a D B.x/ and b D B.y/ for x; y 2. Since is convex, we have x C.1 /y 2. en a C.1 /b D B.x/ C.1 /B.y/ D B.x C.1 /y/ 2 B. /; which justifies the convexity of the image B. /. Taking now any x; y 2 B 1./ and 2.0; 1/, we get B.x/; B.y/ 2. is gives us B.x/ C.1 /B.y/ D B x C.1 /y 2 by the convexity of. us we have x C.1 /y 2 B 1./, which verifies the convexity of the inverse image B 1./. Proposition 1.24 Let 1; 2 R n be convex and let 2 R. en both sets 1 C 2 and 1 are also convex in R n. Proof. It follows directly from the definitions.

26 8 1. CONVEX SETS AND FUNCTIONS Next we proceed with intersections of convex sets. Proposition 1.25 subset of R n. Let f g 2I be a collection of convex subsets of R n. en T 2I is also a convex Proof. Taking any a; b 2 T 2I, we get that a; b 2 for all 2 I. e convexity of each ensures that a C.1 /b 2 for any 2.0; 1/. us a C.1 /b 2 T 2I and the intersection T 2I is convex. Definition 1.26 Let be a subset of R n. e CONVEX HULL of is defined by co WD \ o nc ˇ C is convex and C : e next proposition follows directly from the definition and Proposition Figure 1.2: Nonconvex set and its convex hull. Proposition 1.27 e convex hull co is the smallest convex set containing. Proof. e convexity of the set co follows from Proposition On the other hand, for any convex set C that contains we clearly have co C, which verifies the conclusion. Proposition 1.28 For any subset of R n, its convex hull admits the representation n X m co D i a i ˇˇˇ mx o i D 1; i 0; a i 2 ; m 2 N :

27 1.2. CONVEX SETS 9 Proof. Denoting by C the right-hand side of the representation to prove, we obviously have C. Let us check that the set C is convex. Take any a; b 2 C and get px a WD ia i ; qx b WD ˇj b j ; j D1 where a i ; b j 2, i; ˇj 0 with P p i D P q j D1 ˇj D 1, and p; q 2 N. It follows easily that for every number 2.0; 1/, we have en the resulting equality a C.1 /b D px i C j D1 px ia i C qx.1 /ˇj b j : j D1 qx px.1 /ˇj D i C.1 / qx ˇj D 1 ensures that a C.1 /b 2 C, and thus co C by the definition of co. Fix now any a D P m ia i 2 C with a i 2 co for i D 1; : : : ; m. Since the set co is convex, we conclude j D1 by Proposition 1.20 that a 2 co and thus co D C. Proposition 1.29 e interior int and closure of a convex set R n are also convex. Proof. Fix a; b 2 int and 2.0; 1/. en find an open set V such that a 2 V and so a C.1 /b 2 V C.1 /b : Since V C.1 /b is open, we get a C.1 /b 2 int, and thus the set int is convex. To verify the convexity of, we fix a; b 2 and 2.0; 1/ and then find sequences fa k g and fb k g in converging to a and b, respectively. By the convexity of, the sequence fa k C.1 /b k g lies entirely in and converges to a C.1 /b. is ensures the inclusion a C.1 /b 2 and thus justifies the convexity of the closure. To proceed further, for any a; b 2 R n, define the interval Œa; b/ WD a C.1 /b ˇˇ 2.0; 1 : We can also define the intervals.a; b and.a; b/ in a similar way. Lemma 1.30 For a convex set R n with nonempty interior, take any a 2 int and b 2. en Œa; b/ int.

28 10 1. CONVEX SETS AND FUNCTIONS Proof. Since b 2, for any > 0, we have b 2 C IB. Take now 2.0; 1 and let x WD a C.1 /b. Choosing > 0 such that a C 2 IB gives us x C IB D a C.1 /b C IB a C.1 / C IB C IB D a C.1 / C.1 /IB C IB h a C 2 i IB C.1 / C.1 / : is shows that x 2 int and thus verifies the inclusion Œa; b/ int. Now we establish relationships between taking the interior and closure of convex sets. Proposition 1.31 Let R n be a convex set with nonempty interior. en we have: (i) int D and.ii/ int D int. Proof. (i) Obviously, int. For any b 2 and a 2 int, define the sequence fx k g by x k WD 1 k a C 1 1 b; k 2 N: k Lemma 1.30 ensures that x k 2 int. Since x k! b, we have b 2 int and thus verify (i). (ii) Since the inclusion int int is obvious, it remains to prove the opposite inclusion int int. To proceed, fix any b 2 int and a 2 int. If > 0 is sufficiently small, then c WD b C.b a/ 2, and hence b D 1 C a C 1 c 2.a; c/ int ; 1 C which verifies that int int and thus completes the proof. 1.3 CONVEX FUNCTIONS is section collects basic facts about general (extended-real-valued) convex functions including their analytic and geometric characterizations, important properties as well as their specifications for particular subclasses. We also define convex set-valued mappings and use them to study a remarkable class of optimal value functions employed in what follows. Definition 1.32 Let f W! R be an extended-real-valued function define on a convex set R n. en the function f is CONVEX on if f x C.1 /y f.x/ C.1 /f.y/ for all x; y 2 and 2.0; 1/: (1.1)

29 If the inequality in (1.1) is strict for x y, then f is STRICTLY CONVEX on. Given a function f W! R, the extension of f to R n is defined by 1.3. CONVEX FUNCTIONS 11 ef.x/ WD ( f.x/ if x 2 ; 1 otherwise: Obviously, if f is convex on, where is a convex set, then e f is convex on R n. Furthermore, if f W R n! R is a convex function, then it is also convex on every convex subset of R n. is allows to consider without loss of generality extended-real-valued convex functions on the whole space R n. Figure 1.3: Convex function and nonconvex function. Definition 1.33 e DOMAIN and EPIGRAPH of f W R n! R are defined, respectively, by dom f WD x 2 R n ˇˇ f.x/ < 1 and epi f WD.x; t/ 2 R n R ˇˇ x 2 R n ; t f.x/ D.x; t/ 2 R n R ˇˇ x 2 dom f; t f.x/ : Let us illustrate the convexity of functions by examples. Example 1.34 e following functions are convex: (i) f.x/ WD ha; xi C b for x 2 R n, where a 2 R n and b 2 R. (ii) g.x/ WD kxk for x 2 R n. (iii) h.x/ WD x 2 for x 2 R.

30 12 1. CONVEX SETS AND FUNCTIONS Indeed, the function f in (i) is convex since f x C.1 /y D ha; x C.1 /yi C b D ha; xi C.1 /ha; yi C b D.ha; xi C b/ C.1 /.ha; yi C b/ D f.x/ C.1 /f.y/ for all x; y 2 R n and 2.0; 1/: e function g in (ii) is convex since for x; y 2 R n and 2.0; 1/, we have g x C.1 /y D kx C.1 /yk kxk C.1 /kyk D g.x/ C.1 /g.y/; which follows from the triangle inequality and the fact that k uk D j j kuk whenever 2 R and u 2 R n. e convexity of the simplest quadratic function h in (iii) follows from a more general result for the quadratic function on R n given in the next example. Example 1.35 Let A be an n n symmetric matrix. It is called positive semidefinite if hau; ui 0 for all u 2 R n. Let us check that A is positive semidefinite if and only if the function f W R n! R defined by f.x/ WD 1 2 hax; xi; x 2 Rn ; is convex. Indeed, a direct calculation shows that for any x; y 2 R n and 2.0; 1/ we have f.x/ C.1 /f.y/ f x C.1 /y D 1.1 /ha.x y/; x yi: (1.2) 2 If the matrix A is positive semidefinite, then ha.x y/; x yi 0, so the function f is convex by (1.2). Conversely, assuming the convexity of f and using equality (1.2) for x D u and y D 0 verify that A is positive semidefinite. e following characterization of convexity is known as the Jensen inequality. eorem 1.36 A function f W R n! R is convex if and only if for any numbers i 0 as i D 1; : : : ; m with P m i D 1 and for any elements x i 2 R n, i D 1; : : : ; m, it holds that X m f i x i mx i f.x i /: (1.3) Proof. Since (1.3) immediately implies the convexity of f, we only need to prove that any convex function f satisfies the Jensen inequality (1.3). Arguing by induction and taking into account that for m D 1 inequality (1.3) holds trivially and for m D 2 inequality (1.3) holds by the definition of convexity, we suppose that it holds for an integer m WD k with k 2. Fix numbers i 0, i D 1; : : : ; k C 1, with P kc1 i D 1 and elements x i 2 R n, i D 1; : : : ; k C 1. We obviously have the relationship kx i D 1 kc1 :

31 1.3. CONVEX FUNCTIONS 13 If kc1 D 1, then i D 0 for all i D 1; : : : ; k and (1.3) obviously holds for m WD k C 1 in this case. Supposing now that 0 kc1 < 1, we get kx i 1 kc1 D 1 and by direct calculations based on convexity arrive at f kc1 X i x i D f h P k.1 kc1 / i ix i C kc1 x kc1 1 kc1.1 kc1 /f kx D.1 kc1 /f.1 kc1 / P k ix i 1 kc1 kx kc1 X D i f.x i /: C kc1 f.x kc1 / i x i C kc1 f.x kc1 / 1 kc1 i 1 kc1 f.x i / C kc1 f.x kc1 / is justifies inequality (1.3) and completes the proof of the theorem. e next theorem gives a geometric characterization of the function convexity via the convexity of the associated epigraphical set. eorem 1.37 A function f W R n! R is convex if and only if its epigraph epi f is a convex subset of the product space R n R. Proof. Assuming that f is convex, fix pairs.x 1 ; t 1 /;.x 2 ; t 2 / 2 epi f and a number 2.0; 1/. en we have f.x i / t i for i D 1; 2. us the convexity of f ensures that is implies therefore that f x 1 C.1 /x 2 f.x1 / C.1 /f.x 2 / t 1 C.1 /t 2 :.x 1 ; t 1 / C.1 /.x 2 ; t 2 / D.x 1 C.1 /x 2 ; t 1 C.1 /t 2 / 2 epi f; and thus the epigraph epi f is a convex subset of R n R. Conversely, suppose that the set epi f is convex and fix x 1 ; x 2 2 dom f and a number 2.0; 1/. en.x 1 ; f.x 1 //;.x 2 ; f.x 2 // 2 epi f. is tells us that x1 C.1 /x 2 ; f.x 1 / C.1 /f.x 2 / D x 1 ; f.x 1 / C.1 / x 2 ; f.x 2 / 2 epi f

32 14 1. CONVEX SETS AND FUNCTIONS and thus implies the inequality f x 1 C.1 /x 2 f.x1 / C.1 /f.x 2 /; which justifies the convexity of the function f. Figure 1.4: Epigraphs of convex function and nonconvex function. Now we show that convexity is preserved under some important operations. Proposition 1.38 Let f i W R n! R be convex functions for all i D 1; : : : ; m. en the following functions are convex as well: (i) e multiplication by scalars f for any > 0. (ii) e sum function P m f i. (iii) e maximum function max 1im f i. Proof. e convexity of f in (i) follows directly from the definition. It is sufficient to prove (ii) and (iii) for m D 2, since the general cases immediately follow by induction. (ii) Fix any x; y 2 R n and 2.0; 1/. en we have f1 C f 2 x C.1 /y D f1 x C.1 /y C f2 x C.1 /y which verifies the convexity of the sum function f 1 C f 2. f 1.x/ C.1 /f 1.y/ C f 2.x/ C.1 /f 2.y/ D.f 1 C f 2 /.x/ C.1 /.f 1 C f 2 /.y/; (iii) Denote g WD maxff 1 ; f 2 g and get for any x; y 2 R n and 2.0; 1/ that f i x C.1 /y fi.x/ C.1 /f i.y/ g.x/ C.1 /g.y/ for i D 1; 2. is directly implies that g x C.1 /y D max f 1 x C.1 /y ; f2 x C.1 /y g.x/ C.1 /g.y/; which shows that the maximum function g.x/ D maxff 1.x/; f 2.x/g is convex.

33 1.3. CONVEX FUNCTIONS 15 e next result concerns the preservation of convexity under function compositions. Proposition 1.39 Let f W R n! R be convex and let ' W R! R be nondecreasing and convex on a convex set containing the range of the function f. en the composition ' ı f is convex. Proof. Take any x 1 ; x 2 2 R n and 2.0; 1/. en we have by the convexity of f that f x 1 C.1 /x 2 f.x1 / C.1 /f.x 2 /: Since ' is nondecreasing and it is also convex, it follows that ' ı f x1 C.1 /x 2 D ' f x1 C.1 /x 2 ' f.x 1 / C.1 /f.x 2 / ' f.x 1 / C.1 /' f.x 2 / D.' ı f /.x 1 / C.1 /.' ı f /.x 2 /; which verifies the convexity of the composition ' ı f. Now we consider the composition of a convex function and an affine mapping. Proposition 1.40 Let B W R n! R p be an affine mapping and let f W R p! R be a convex function. en the composition f ı B is convex. Proof. Taking any x; y 2 R n and 2.0; 1/, we have f ı B x C.1 /y D f.b.x C.1 /y// D f B.x/ C.1 /B.y/ f B.x/ C.1 /f B.y/ D.f ı B/.x/ C.1 /.f ı B/.y/ and thus justify the convexity of the composition f ı B. e following simple consequence of Proposition 1.40 is useful in applications. Corollary 1.41 Let f W R n! R be a convex function. For any Nx; d 2 R n, the function 'W R! R defined by '.t/ WD f. Nx C td/ is convex as well. Conversely, if for every Nx; d 2 R n the function ' defined above is convex, then f is also convex. Proof. Since B.t/ D Nx C td is an affine mapping, the convexity of ' immediately follows from Proposition To prove the converse implication, take any x 1 ; x 2 2 R n, 2.0; 1/ and let Nx WD x 2, d WD x 1 x 2. Since '.t/ D f. Nx C td/ is convex, we have f x 1 C.1 /x 2 D f x2 C.x 1 x 2 / D './ D '.1/ C.1 /.0/ '.1/ C.1 /'.0/ D f.x 1 / C.1 /f.x 2 /; which verifies the convexity of the function f.

34 16 1. CONVEX SETS AND FUNCTIONS e next proposition is trivial while useful in what follows. Proposition 1.42 Let f W R n R p! R be convex. For. Nx; Ny/ 2 R n R p, the functions '.y/ WD f. Nx; y/ and.x/ WD f.x; Ny/ are also convex. Now we present an important extension of Proposition 1.38(iii). Proposition 1.43 Let f i W R n! R for i 2 I be a collection of convex functions with a nonempty index set I. en the supremum function f.x/ WD sup i2i f i.x/ is convex. Proof. Fix x 1 ; x 2 2 R n and 2.0; 1/. For every i 2 I, we have f i x1 C.1 /x 2 fi.x 1 / C.1 /f i.x 2 / f.x 1 / C.1 /f.x 2 /; which implies in turn that f x 1 C.1 /x 2 D sup f i x1 C.1 /x 2 f.x1 / C.1 /f.x 2 /: i2i is justifies the convexity of the supremum function. Our next intention is to characterize convexity of smooth functions of one variable. To proceed, we begin with the following lemma. Lemma 1.44 Given a convex function f W R! R, assume that its domain is an open interval I. For any a; b 2 I and a < x < b, we have the inequalities f.x/ f.a/ x a f.b/ f.a/ b a f.b/ f.x/ : b x Proof. Fix a; b; x as above and form the numbers t WD x a 2.0; 1/. en b a f.x/ D f a C.x a/ D f a C x a b a D f a C t.b a/ D f tb C.1 t/a : b a is gives us the inequalities f.x/ tf.b/ C.1 t/f.a/ and f.x/ f.a/ tf.b/ C.1 t/f.a/ f.a/ D t f.b/ f.a/ D x a f.b/ f.a/ ; b a which can be equivalently written as f.x/ f.a/ x a f.b/ f.a/ : b a

35 1.3. CONVEX FUNCTIONS 17 Similarly, we have the estimate f.x/ f.b/ tf.b/ C.1 t/f.a/ f.b/ D.t 1/ f.b/ f.a/ D x b f.b/ f.a/ ; b a which finally implies that f.b/ f.a/ b a and thus completes the proof of the lemma. f.b/ f.x/ b x eorem 1.45 Suppose that f W R! R is differentiable on its domain, which is an open interval I. en f is convex if and only if the derivative f 0 is nondecreasing on I. Proof. Suppose that f is convex and fix a < b with a; b 2 I. By Lemma 1.44, we have f.x/ f.a/ x a f.b/ f.a/ b a for any x 2.a; b/. is implies by the derivative definition that Similarly, we arrive at the estimate f 0.a/ f.b/ f.a/ b a f.b/ f.a/ : b a f 0.b/ and conclude that f 0.a/ f 0.b/, i.e., f 0 is a nondecreasing function. To prove the converse, suppose that f 0 is nondecreasing on I and fix x 1 < x 2 with x 1 ; x 2 2 I and t 2.0; 1/. en x 1 < x t < x 2 for x t WD tx 1 C.1 t/x 2 : By the mean value theorem, we find c 1 ; c 2 such that x 1 < c 1 < x t < c 2 < x 2 and f.x t / f.x 1 / D f 0.c 1 /.x t x 1 / D f 0.c 1 /.1 t/.x 2 x 1 /; f.x t / f.x 2 / D f 0.c 2 /.x t x 2 / D f 0.c 2 /t.x 1 x 2 /: is can be equivalently rewritten as tf.x t / tf.x 1 / D f 0.c 1 /t.1 t/.x 2 x 1 /;.1 t/f.x t /.1 t/f.x 2 / D f 0.c 2 /t.1 t/.x 1 x 2 /: Since f 0.c 1 / f 0.c 2 /, adding these equalities yields f.x t / tf.x 1 / C.1 t/f.x 2 /; which justifies the convexity of the function f.

36 18 1. CONVEX SETS AND FUNCTIONS Corollary 1.46 Let f W R! R be twice differentiable on its domain, which is an open interval I. en f is convex if and only if f 00.x/ 0 for all x 2 I. Proof. Since f 00.x/ 0 for all x 2 I if and only if the derivative function f 0 is nondecreasing on this interval. en the conclusion follows directly from eorem Example 1.47 Consider the function 8 < 1 if x > 0; f.x/ WD x : 1 otherwise: To verify its convexity, we get that f 00.x/ D 2 > 0 for all x belonging to the domain of f, which x3 is I D.0; 1/. us this function is convex on R by Corollary Next we define the notion of set-valued mappings (or multifunctions), which plays an important role in modern convex analysis, its extensions, and applications. Definition 1.48 We say that F is a SET-VALUED MAPPING between R n and R p and denote it by F W R n! R p if F.x/ is a subset of R p for every x 2 R n. e DOMAIN and GRAPH of F are defined, respectively, by dom F WD x 2 R n ˇˇ F.x/ ; and gph F WD.x; y/ 2 R n R p ˇˇ y 2 F.x/ : Any single-valued mapping F W R n! R p is a particular set-valued mapping where the set F.x/ is a singleton for every x 2 R n. It is essential in the following definition that the mapping F is set-valued. Definition 1.49 Let F W R n! R p and let ' W R n R p! R. e OPTIMAL VALUE or MARGINAL FUNCTION associated with F and ' is defined by.x/ WD inf '.x; y/ ˇˇ y 2 F.x/ ; x 2 R n : (1.4) roughout this section we assume that.x/ > 1 for every x 2 R n. Proposition 1.50 Assume that 'W R n R p! R is a convex function and that F W R n! R p is of convex graph. en the optimal value function in (1.4) is convex.

37 1.3. CONVEX FUNCTIONS 19 Figure 1.5: Graph of set-valued mapping. Proof. Take x 1 ; x 2 2 dom, 2.0; 1/. For any > 0, find y i 2 F.x i / such that It directly implies the inequalities '.x i ; y i / <.x i / C for i D 1; 2: '.x 1 ; y 1 / <.x 1 / C ;.1 /'.x 2 ; y 2 / <.1 /.x 2 / C.1 /: Summing up these inequalities and employing the convexity of ' yield ' x 1 C.1 /x 2 ; y 1 C.1 /y 2 '.x1 ; y 1 / C.1 /'.x 2 ; y 2 / <.x 1 / C.1 /.x 2 / C : Furthermore, the convexity of gph F gives us x1 C.1 /x 2 ; y 1 C.1 /y 2 D.x1 ; y 1 / C.1 /.x 2 ; y 2 / 2 gph F; and therefore y 1 C.1 /y 2 2 F.x 1 C.1 /x 2 /. is implies that x 1 C.1 /x 2 ' x1 C.1 /x 2 ; y 1 C.1 /y 2 <.x1 / C.1 /.x 2 / C : Letting finally! 0 ensures the convexity of the optimal value function. Using Proposition 1.50, we can verify convexity in many situations. For instance, given two convex functions f i W R n! R, i D 1; 2, let '.x; y/ WD f 1.x/ C y and F.x/ WD Œf 2.x/; 1/. en the function ' is convex and set gph F D epi f 2 is convex as well, and hence we justify the convexity of the sum.x/ D inf '.x; y/ D f 1.x/ C f 2.x/: y2f.x/

38 20 1. CONVEX SETS AND FUNCTIONS Another example concerns compositions. Let f W R p! R be convex and let B W R n! R p be affine. Define '.x; y/ WD f.y/ and F.x/ WD fb.x/g. Observe that ' is convex while F is of convex graph. us we have the convex composition.x/ D inf y2f.x/ '.x; y/ D f B.x/ ; x 2 R n : e examples presented above recover the results obtained previously by direct proofs. Now we establish via Proposition 1.50 the convexity of three new classes of functions. Proposition 1.51 Let ' W R p! R be convex and let B W R p! R n be affine. Consider the setvalued inverse image mapping B 1 W R n! R p, define f.x/ WD inf '.y/ ˇˇ y 2 B 1.x/ ; x 2 R n ; and suppose that f.x/ > 1 for all x 2 R n. en f is a convex function. Proof. Let '.x; y/ '.y/ and F.x/ WD B 1.x/. en the set gph F D.u; v/ 2 R n R p ˇˇ B.v/ D u is obviously convex. Since we have the representation f.x/ D inf '.y/; x 2 R n ; y2f.x/ the convexity of f follows directly from Proposition Proposition 1.52 For convex functions f 1 ; f 2 W R n! R, define the INFIMAL CONVOLUTION.f 1 f 2 /.x/ WD inf f 1.x 1 / C f 2.x 2 / ˇˇ x1 C x 2 D x and suppose that.f 1 f 2 /.x/ > 1 for all x 2 R n. en f 1 f 2 is also convex. Proof. Define ' W R n R n! R by '.x 1 ; x 2 / WD f 1.x 1 / C f 2.x 2 / and B W R n R n! R n by B.x 1 ; x 2 / WD x 1 C x 2. We have inf '.x 1 ; x 2 / ˇˇ.x1 ; x 2 / 2 B 1.x/ D.f 1 f 2 /.x/ for all x 2 R n ; which implies the convexity of.f 1 f 2 / by Proposition 1.51.

39 Definition RELATIVE INTERIORS OF CONVEX SETS 21 A function g W R p! R is called NONDECREASING COMPONENTWISE if xi y i for all i D 1; : : : ; p H) g.x 1 ; : : : ; x p / g.y 1 ; : : : ; y p / : Now we are ready to present the final consequence of Proposition 1.50 in this section that involves the composition. Proposition 1.54 Define h W R n! R p by h.x/ WD.f 1.x/; : : : ; f p.x//, where f i W R n! R for i D 1; : : : ; p are convex functions. Suppose that g W R p! R is convex and nondecreasing componentwise. en the composition g ı h W R n! R is a convex function. Proof. Let F W R n! R p be a set-valued mapping defined by en the graph of F is represented by F.x/ WD Œf 1.x/; 1/ Œf 2.x/; 1/ : : : Œf p.x/; 1/: gph F D.x; t 1 ; : : : ; t p / 2 R n R p ˇˇ ti f i.x/ : Since all f i are convex, the set gph F is convex as well. Define further ' W R n R p! R by '.x; y/ WD g.y/ and observe, since g is increasing componentwise, that inf '.x; y/ ˇˇ y 2 F.x/ D g f1.x/; : : : ; f p.x/ D.g ı h/.x/; which ensures the convexity of the composition g ı h by Proposition RELATIVE INTERIORS OF CONVEX SETS We begin this section with the definition and properties of affine sets. Given two elements a and b in R n, the line connecting them is Note that if a D b, then LŒa; b D fag. LŒa; b WD a C.1 /b ˇˇ 2 R : Definition 1.55 A subset of R n is AFFINE if for any a; b 2 we have LŒa; b. For instance, any point, line, and plane in R 3 are affine sets. e empty set and the whole space are always affine. It follows from the definition that the intersection of any collection of affine sets is affine. is leads us to the construction of the affine hull of a set. Definition 1.56 e AFFINE HULL of a set R n is aff WD \ C ˇˇ C is affine and C :

40 22 1. CONVEX SETS AND FUNCTIONS An element x in R n of the form mx x D i! i with mx i D 1; m 2 N; is called an affine combination of! 1 ; : : : ;! m. e proof of the next proposition is straightforward and thus is omitted. Proposition 1.57 e following assertions hold: (i) A set is affine if and only if contains all affine combinations of its elements. (ii) Let, 1, and 2 be affine subsets of R n. en the sum 1 C 2 and the scalar product for any 2 R are also affine subsets of R n. (iii) Let B W R n! R p be an affine mapping. If is an affine subset of R n and is an affine subset of R p, then the image B. / is an affine subset of R p and the inverse image B 1./ is an affine subset of R n. (iv) Given R n, its affine hull is the smallest affine set containing. We have n X m aff D i! i ˇˇˇ mx o i D 1;! i 2 ; m 2 N : (v) A set is a.linear/ subspace if and only if is an affine set containing the origin. Next we consider relationships between affine sets and (linear) subspaces. Lemma 1.58! 2. A nonempty subset of R n is affine if and only if! is a subspace of R n for any Proof. Suppose that a nonempty set R n is affine. It follows from Proposition 1.57(v) that! is a subspace for any! 2. Conversely, fix! 2 and suppose that! is a subspace denoted by L. en the set D! C L is obviously affine. e preceding lemma leads to the following notion. Definition 1.59 An affine set is PARALLEL to a subspace L if D! C L for some! 2. e next proposition justifies the form of the parallel subspace. Proposition 1.60 Let be a nonempty, affine subset of R n. en it is parallel to the unique subspace L of R n given by L D.

41 1.4. RELATIVE INTERIORS OF CONVEX SETS 23 Proof. Given a nonempty, affine set, fix! 2 and come up to the linear subspace L WD! parallel to. To justify the uniqueness of such L, take any! 1 ;! 2 2 and any subspaces L 1 ; L 2 R n such that D! 1 C L 1 D! 2 C L 2. en L 1 D! 2! 1 C L 2. Since 0 2 L 1, we have! 1! 2 2 L 2. is implies that! 2! 1 2 L 2 and thus L 1 D! 2! 1 C L 2 L 2. Similarly, we get L 2 L 1, which justifies that L 1 D L 2. It remains to verify the representation L D. Let D! C L with the unique subspace L and some! 2. en L D!. Fix any x D u! with u;! 2 and observe that! is a subspace parallel to. Hence! D L by the uniqueness of L proved above. is ensures that x 2! D L and thus L. e uniqueness of the parallel subspace shows that the next notion is well defined. Definition 1.61 e DIMENSION OF AN AFFINE SET ; R n is the dimension of the linear subspace parallel to. Furthermore, the DIMENSION OF A CONVEX SET ; R n is the dimension of its affine hull aff. To proceed further, we need yet another definition important in what follows. Definition 1.62 e elements v 0 ; : : : ; v m in R n, m 1, are AFFINELY INDEPENDENT if h X m i v i D 0; id0 mx id0 i i D 0 H) i D 0 for all i D 0; : : : ; m : It is easy to observe the following relationship with the linear independence. Proposition 1.63 e elements v 0 ; : : : ; v m in R n are affinely independent if and only if the elements v 1 v 0 ; : : : ; v m v 0 are linearly independent. Proof. Suppose that v 0 ; : : : ; v m are affinely independent and consider the system mx mx i.v i v 0 / D 0; i.e., 0 v 0 C i v i D 0; where 0 WD P m i. Since the elements v 0 ; : : : ; v m are affinely independent and P m id0 i D 0, we have that i D 0 for all i D 1; : : : ; m. us v 1 v 0 ; : : : ; v m v 0 are linearly independent. e proof of the converse statement is straightforward. Recall that the span of some set C, span C, is the linear subspace generated by C. Lemma 1.64 Let WD afffv 0 ; : : : ; v m g, where v i 2 R n for all i D 0; : : : ; m. en the span of the set fv 1 v 0 ; : : : ; v m v 0 g is the subspace parallel to.

42 24 1. CONVEX SETS AND FUNCTIONS Proof. Denote by L the subspace parallel to. en v 0 D L and therefore v i v 0 2 L for all i D 1; : : : ; m. is gives span v i v 0 ˇˇ i D 1; : : : ; m L: To prove the converse inclusion, fix any v 2 L and get v C v 0 2. us we have mx v C v 0 D i v i ; id0 mx i D 1: id0 is implies the relationship mx v D i.v i v 0 / 2 span v i v 0 ˇˇ i D 1; : : : ; m ; which justifies the converse inclusion and hence completes the proof. e proof of the next proposition is rather straightforward. Proposition 1.65 e elements v 0 ; : : : ; v m are affinely independent in R n if and only if its affine hull WD afffv 0 ; : : : ; v m g is m-dimensional. Proof. Suppose that v 0 ; : : : ; v m are affinely independent. en Lemma 1.64 tells us that the subspace L WD spanfv i v 0 j i D 1; : : : ; mg is parallel to. e linear independence of v 1 v 0 ; : : : ; v m v 0 by Proposition 1.63 means that the subspace L is m-dimensional and so is. e proof of the converse statement is also straightforward. Affinely independent systems lead us to the construction of simplices. Definition 1.66 Let v 0 ; : : : ; v m be affinely independent in R n. en the set m WD co v i ˇˇ i D 0; : : : ; m is called an m-simplex in R n with the vertices v i, i D 0; : : : ; m. An important role of simplex vertices is revealed by the following proposition. Proposition 1.67 Consider an m-simplex m with vertices v i for i D 0; : : : ; m. For every v 2 m, there is a unique element. 0 ; : : : ; m / 2 R mc1 C such that mx v D i v i ; id0 mx i D 1: id0