LECTURE 3 LECTURE OUTLINE Differentiable Conve Functions Conve and A ne Hulls Caratheodory s Theorem Reading: Sections 1.1, 1.2 All figures are courtesy of Athena Scientific, and are used with permission. 1
DIFFERENTIABLE CONVEX FUNCTIONS f(z) f()+ f() (z ) z Let C R n be a conve set and let f : R n R be differentiable over R n. (a) The function f is conve over C iff f(z) f()+(z ) f(), apple, z C (b) If the inequality is strict whenever = z, then f is strictly conve over C. 2
PROOF IDEAS αf() + (1 α)f(y) f(y) f() f(z)+( z) f(z) f(z) f(z)+(y z) f(z) z = α + (1 α)y (a) y f(z) f()+ f( + α(z ) f() α f()+(z ) f() + α(z ) z (b) 3
OPTIMALITY CONDITION Let C be a nonempty conve subset of R n and let f : R n R be conve and differentiable over an open set that contains C. Then a vector C minimizes f over C if and only if f( ) ( ) 0, apple C. Proof: If the condition holds, then f() f( )+( ) f( ) f( ), apple C, so minimizes f over C. Converse: Assume the contrary, i.e., minimizes f over C and f( ) ( ) < 0 for some C. By differentiation, we have lim α 0 f ( + α( ) f( ) α = f( ) ( ) < 0 ( so f + α( ) decreases strictly for su small α > 0, contradicting the optimality ciently of. Q.E.D. 4
PROJECTION THEOREM Let C be a nonempty closed conve set in R n. (a) For every z R n, there eists a unique minimum of f() = z 2 over all C (called the projection of z on C). (b) is the projection of z if and only if ( ) (z ) 0, apple C Proof: (a) f is strictly conve and has compact level sets. (b) This is just the necessary and su cient optimality condition f( ) ( ) 0, apple C. 5
TWICE DIFFERENTIABLE CONVEX FNS Let C be a conve subset of R n and let f : R n R be twice continuously differentiable over R n. (a) If 2 f() is positive semidefinite for all C, then f is conve over C. (b) If 2 f() is positive definite for all C, then f is strictly conve over C. (c) If C is open and f is conve over C, then 2 f() is positive semidefinite for all C. Proof: (a) By mean value theorem, for, y C f(y) =f()+(y ) f()+ 1 (y ) 2 f ( +α(y ) (y ) 2 for some α [0, 1]. Using the positive semidefiniteness of 2 f, we obtain f(y) f()+(y ) f(), From the preceding result, f is conve. apple, y C (b) Similar to (a), we have f(y) >f() +(y ) f() for all, y C with = y, and we use the preceding result. (c) By contradiction... similar. 6
CONVEX AND AFFINE HULLS Given a set X R n : A conve combination of elements of X is a vector of the form m i=1 α i i, where i X, α i m 0, and i=1 α i = 1. The conve hull of X, denoted conv(x), is the intersection of all conve sets containing X. (Can be shown to be equal to the set of all conve combinations from X). The a ne hull of X, denoted aff(x), is the intersection of all a ne sets containing X (an a ne set is a set of the form + S, where S is a subspace). A nonnegative combination of elements of X is a vector of the form m i=1 α i i, where i X and α i 0 for all i. The cone generated by X, denoted cone(x), is the set of all nonnegative combinations from X: It is a conve cone containing the origin. It need not be closed! If X is a finite set, cone(x) is closed (nontrivial to show!) 7
CARATHEODORY S THEOREM 4 2 1 X 2 cone(x) 1 conv(x) 0 (a) (b) 3 Let X be a nonempty subset of R n. (a) Every = 0 in cone(x) can be represented as a positive combination of vectors 1,..., m from X that are linearly independent (so m n). (b) Every / X that belongs to conv(x) can be represented as a conve combination of vectors 1,..., m from X with m n + 1. 8
PROOF OF CARATHEODORY S THEOREM (a) Let be a nonzero vector in cone(x), and let m be the smallest integer such that has the form m i=1 α i i, where α i > 0 and i X for all i =1,...,m. If the vectors i were linearly dependent, there would eist 1,..., m, with m i i =0 i=1 and at least one of the i is positive. Consider m (α i i ) i, i=1 where is the largest such that α i i 0 for all i. This combination provides a representation of as a positive combination of fewer than m vectors of X a contradiction. Therefore, 1,..., m, are linearly independent. (b) Use lifting argument: apply part (a) to Y = (, 1) X. 1 Y (, 1) 0 R n X 9
AN APPLICATION OF CARATHEODORY The conve hull of a compact set is compact. Proof: Let X be compact. We take a sequence in conv(x) and show that it has a convergent subsequence whose limit is in conv(x). By Caratheo { dory, a sequence in conv(x) can n+1 be epressed as i=1 αi k i k, where for all k and, k 0, k, and n+1 i α i i X i=1 αk i = 1. Since the sequence (α k 1,...,αn+1 k,k 1,..., n+1 k ) is bounded, it has a limit point (α1,...,α n+1, 1,..., n+1 ), n+1 which must satisfy i=1 α i = 1, and α i 0, i X for all i. The vector n+1 i=1 α { i i belongs to conv(x) n+1 and is a limit point of, showing that conv(x) is compact. i=1 αk i k i Q.E.D. Note that the conve hull of a closed set need not be closed! 10
MIT OpenCourseWare http://ocw.mit.edu 6.253 Conve Analysis and Optimization Spring 2012 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.