Subdifferentials of convex functions

Transcription

1 Subdifferentials of convex functions Jordan Bell Department of Matematics, University of Toronto April 21, 2014 Wenever we speak about a vector space in tis note we mean a vector space over R. If X is a topological vector space ten we denote by X te set of all continuous linear maps X R. X is called te dual space of X, and is itself a vector space. 1 1 Definition of subdifferential If X is a topological vector space, f : X [, ] is a function, x X, and λ X, ten we say tat λ is a subgradient of f at x if 2 f(y) f(x) + λ(y x), y X. Te subdifferential of f at x is te set of all subgradients of f at x and is denoted by f(x). Tus f is a function from X to te power set of X, i.e. f : X 2 X. If f(x), we say tat f is subdifferentiable at x. It is immediate tat if tere is some y suc tat f(y) =, ten { X f(x) = f(x) = f(x) >, x X. Tus, little is lost if we prove statements about subdifferentials of functions tat do not take te value. Teorem 1. If X is a topological vector space, f : X [, ] is a function and x X, ten f(x) is a convex subset of X. 1 In tis note, we are following te presentation of some results in Caralambos D. Aliprantis and Kim C. Border, Infinite Dimensional Analysis: A Hitciker s Guide, tird ed., capter 7. Tree oter sources for material on subdifferentials are: Jean-Paul Penot, Calulus Witout Derivatives, capter 3; Viorel Barbu and Teodor Precupanu, Convexity and Optimization in Banac Spaces, fourt ed., 2.2, pp ; and Jean-Pierre Aubin, Optima and Equilibria: An Introduction to Nonlinear Analysis, second ed., capter 4, pp =, =, and is nonsense; if a R, ten a = and a + =. 1

2 Proof. If λ 1, λ 2 f(x) and 0 t 1, ten of course (1 t)λ 1 + tλ 2 X. For any y X we ave f(y) = (1 t)f(y) + tf(y) (1 t)f(x) + (1 t)λ 1 (y x) + tf(x) + tλ 2 (y x) = f(x) + ( (1 t)λ 1 + tλ 2 ) (y x), sowing tat (1 t)λ 1 + tλ 2 f(x) and tus tat f(x) is convex. To say tat 0 f(x) is equivalent to saying tat f(y) f(x) for all y X and so f(x) = inf y X f(y). Tis can be said in te following way. Lemma 2. If X is a topological vector space and f : X [, ] is a function, ten x is a minimizer of f if and only if 0 f(x). 2 Convex functions If X is a set and f : X [, ] is a function, ten te epigrap of f is te set epi f = {(x, α) X R : α f(x)}, and te effective domain of f is te set dom f = {x X : f(x) < }. To say tat x dom f is equivalent to saying tat tere is some α R suc tat (x, α) epi f. We say tat f is finite if < f(x) < for all x X. If X is a vector space and f : X [, ] is a function, ten we say tat f is convex if epi f is a convex subset of te vector space X R. If X is a set and f : X [, ] is a function, we say tat f is proper if it does not take only te value and never takes te value. It is unusual to talk merely about proper functions rater tan proper convex functions; we do so to make clear ow convexity is used in te results we prove. 3 Weak-* topology Let X be a topological vector space and for x X define e x : X R by e x λ = λx. Te weak-* topology on X is te initial topology for te set of functions {e x : x X}, tat is, te coarsest topology on X suc tat for eac x X, te function e x : X R is continuous. Lemma 3. If X is a topological vector space, τ 1 is te weak-* topology on X, and τ 2 is te subspace topology on X inerited from R X wit te product topology, ten τ 1 = τ 2. 2

3 Proof. Let λ i X converge in τ 1 to λ X. For eac x X, te function e x : X R is τ 1 continuous, so e x λ i e x λ, i.e. λ i x λx. But for f i R X to converge to f R X means tat for eac x, we ave f i (x) f(x). Tus λ i converges to λ in τ 2. Tis sows tat τ 2 τ 1. Let x X, and let λ i X converge in τ 2 to λ X. We ten ave e x λ i = λ i x λx = e x λ; since λ i was an arbitrary net tat converges in τ 2, tis sows tat e x is τ 2 continuous. Tus, we ave sown tat for eac x X, te function e x is τ 2 continuous. But τ 1 is te coarsest topology for wic e x is continuous for all x X, so we obtain τ 1 τ 2. In oter words, te weak-* topology on X is te topology of pointwise convergence. We now prove tat at eac point in te effective domain of a proper function on a topological vector space, te subdifferential is a weak-* closed subset of te dual space. 3 Teorem 4. If X is a topological vector space, f : X (, ] is a proper function, and x dom f, ten f(x) is a weak-* closed subset of X. Proof. If λ f(x), ten for all y X we ave so, for any v X, using y = v + x, or, f(y) f(x) + λ(y x), f(v + x) f(x) + λv, λv f(x + v) f(x); tis makes sense because f(x) is finite. On te oter and, let λ X. If λv f(x + v) f(x) for all v X, ten λ(v x) f(v) f(x), i.e. f(v) f(x) + λ(v x), and so λ f(x). Terefore f(x) = {λ X : λv f(x + v) f(x)}. (1) v X Defining e v : X R for v X by e v λ = λv, for eac v X we ave e 1 v (, f(x + v) f(x)] = {λ X : λv f(x + v) f(x)}. Because e v is continuous, tis inverse image is a closed subset of X. Terefore, eac of te sets in te intersection (1) is a closed subset of X, and so f(x) is a closed subset of X. 3 cf. Caralambos D. Aliprantis and Kim C. Border, Infinite Dimensional Analysis: A Hitciker s Guide, tird ed., p. 265, Teorem

4 4 Support points If X is set, A is a subset of X, and f : X [, ] is a function, we say tat x X is a minimizer of f over A if f(x) = inf y A f(y), and tat x is a maximizer of f over A if f(x) = sup f(y). y A If A is a nonempty subset of a topological vector space X and x A, we say tat x is a support point of A if tere is some nonzero λ X for wic x is a minimizer or a maximizer of λ over A. Moreover, x is a minimizer of λ over A if and only if x is a maximizer of λ over A. Tus, if we know tat x is a support point of a set A, ten we ave at our disposal bot tat x is a minimizer of some nonzero element of X over A and tat x is a maximizer of some nonzero element of X over A. If x is a support point of A and A is not contained in te yperplane {y X : λy = λx}, we say tat A is properly supported at x. To say tat A is not contained in te set {y X : λy = λx} is equivalent to saying tat tere is some y A suc tat λy λx. In te following lemma, we sow tat te support points of a set A are contained in te boundary A of te set. Lemma 5. If X is a topological vector space, A is a subset of X, and x is a support point of A, ten x A. Proof. Because x is a support point of A tere is some nonzero λ X for wic x is a maximizer of λ over A: λx = sup λy. y A As λ is nonzero tere is some y X wit λy > λx. For any t > 0, (1 t)λx + tλy = λ((1 t)x + ty) = (1 t)λx + tλy > (1 t)λx + tλx = λx, ence if t > 0 ten (1 t)λx + ty A. But (1 t)x + ty x as t 0 and x A, sowing tat x A. Te following lemma gives conditions under wic a boundary point of a set is a proper support point of te set. 4 Lemma 6. If X is a topological vector space, C is a convex subset of X tat as nonempty interior, and x C C, ten C is properly supported at x. 4 Caralambos D. Aliprantis and Kim C. Border, Infinite Dimensional Analysis: A Hitciker s Guide, tird ed., p. 259, Lemma

5 Proof. Te Han-Banac separation teorem 5 tells us tat if A and B are disjoint nonempty convex subsets of X and A is open ten tere is some λ X and some t R suc tat λa < t λb, a A, b B. Ceck tat te interior of a convex set in a topological vector space is convex, and ence tat we can apply te Han-Banac separation teorem to {x} and C : as x belongs to te boundary of C it does not belong to te interior of C, so {x} and C are disjoint nonempty convex sets. Tus, tere is some λ X and some t R suc tat λy < t λx for all y C, from wic it follows tat λx λy for all y C, and λ 0 because of te strict inequality for te interior. As x C, tis means tat x is a maximizer of λ over C, and as λ 0 tis means tat x is a support point of C. But C is nonempty and if y C ten λx < λy, ence x is a proper support point of C. 5 Subdifferentials of convex functions If f : X (, ] is a proper function ten tere is some y X for wic f(y) <, and for f to ave a subgradient λ at x demands tat f(y) f(x) + λ(y x), and ence tat f(x) <. Terefore, if f is a proper function ten te set of x at wic f is subdifferentiable is a subset of dom f. We now prove conditions under wic a function is subdifferentiable at a point, i.e., under wic te subdifferential at tat point is nonempty. 6 Teorem 7. If X is a topological vector space, f : X (, ] is a proper convex function, x is an interior point of dom f, and f is continuous at x, ten f as a subgradient at x. Proof. Because f is convex, te set dom f is convex, and te interior of a convex set in a topological vector space is convex so (dom f) is convex. f is proper so it does not take te value, and on dom f it does not take te value, ence f is finite on dom f. But for a finite convex function on an open convex set in a topological vector space, being continuous at a point is equivalent to being continuous on te set, and is also equivalent to being bounded above on an open neigborood of te point. 7 Terefore, f is continuous on (dom f) and is bounded above on some open neigborood V of x contained in (dom f), say f(y) M for all y V. V (M, ) is an open subset of X R, and is contained in epi f. Tis sows tat epi f as nonempty interior. Since f(x) <, if ɛ > 0 ten (x, f(x) ɛ) epi f, and since f(x) > we ave (x, f(x)) epi f, and terefore (x, f(x)) epi f (epi f). We can now apply Lemma 6: epi f is a convex subset of te topological vector space X R wit nonempty interior and 5 Gert K. Pedersen, Analysis Now, revised printing, p. 65, Teorem Caralambos D. Aliprantis and Kim C. Border, Infinite Dimensional Analysis: A Hitciker s Guide, tird ed., p. 265, Teorem Caralambos D. Aliprantis and Kim C. Border, Infinite Dimensional Analysis: A Hitciker s Guide, tird ed., p. 188, Teorem

6 (x, f(x)) epi f (epi f), so epi f is properly supported at (x, f(x)). Tat is, Lemma 6 sows tat tere is some Λ (X R) suc tat Λ(x, f(x)) = sup Λ(y, α), (y,α) epi f and tere is some (y, α) epi f for wic Λ(x, f(x)) > Λ(y, α). Now, tere is some λ X and some β R = R suc tat Λ(y, α) = λy + βα for all (y, α) X R. Tus, tere is some nonzero λ X and some β R suc tat λx + βf(x) = sup λy + βα. (y,α) epi f If β > 0 ten te rigt-and side would be wile te left-and side is constant and <, so β 0. Suppose by contradiction tat β = 0. Ten λx λy for all y dom f, and as λ 0 tis means tat x is a support point of dom f, and ten by Lemma 5 we ave tat x (dom f), contradicting x (dom f). Hence β < 0, so λx + βf(x) λy + βf(y), y dom f, i.e., f(y) f(x) λ (y x), y dom f. β Furtermore, if y dom f ten f(y) =, for wic te above inequality is true. Terefore, f(y) f(x) λ β (y x) for all y X, sowing tat λ β is a subgradient of f at x. 6 Directional derivatives Lemma 8. If X is a vector space, f : X (, ] is a proper convex function, x dom f, v X, and 0 < <, ten f(x + v) f(x) f(x + v) f(x). Proof. We ave x + v = (x + v) + x, and because f is convex tis gives i.e. f(x + v) f(x + v) + f(x), f(x + v) f(x) (f(x + v) f(x)). 6

7 Dividing by, f(x + v) f(x) f(x + v) f(x). If f : X (, ] is a proper convex function, x dom f, and v X, ten te above lemma sows tat f(x + v) f(x) is an increasing function (0, ) (, ], and terefore tat f(x + v) f(x) lim 0 + exists; it belongs to [, ], and if tere is at least one > 0 for wic f(x + v) < ten te limit will be <. We define te one-sided directional derivative of f at x to be te function d + f(x) : X [, ] defined by 8 d + f(x + v) f(x) f(x)v = lim, v X. 0 + Lemma 9. If X is a topological vector space, f : X (, ] is a proper convex function, x (dom f), f is continuous at x, and v X, ten < d + f(x)v <. Proof. Because x (dom f), tere is some > 0 for wic x + v dom f and ence for wic f(x + v) <. Tis implies tat d + f(x)v <. Let > 0. By Teorem 7, te subdifferential f(x) is nonempty, i.e. tere is some λ X for wic f(y) f(x) + λ(y x) for all y X. Tus, for all v X we ave, wit y = x + v, i.e., f(x + v) f(x) + λ(v), λv f(x + v) f(x). Since tis difference quotient is bounded below by λv, its limit as 0 + is >, and terefore d + f(x)v >. 8 We are following te notation of Caralambos D. Aliprantis and Kim C. Border, Infinite Dimensional Analysis: A Hitciker s Guide, tird ed., p