Notes on Convexity. Roy Radner Stern School, NYU. September 11, 2006

Transcription

1 Notes on Convexit Ro Radner Stern School, NYU September, 2006 Abstract These notes are intended to complement the material in an intermediate microeconomic theor course. In particular, the provide a rigorous discussion of optimalit conditions for functions of one and several variables, including minimization of a convex (or maximization of a concave) function subject to a linear constraint. In the case of a function of a single variable, the notes should help the reader go somewhat beond the standard treatment, which tpicall deals onl with interior solutions and functions that are twice di erentiable. The notes should be accessible to somone with a good basic background in univariate and multivariate di erential and integral calculus. [Preliminar Draft - Do Not Quote] Preamble. The mathematical concept of convexit (along with the mirror concept of concavit) plas an important role in economic theor. These notes are intended to complement the material in an intermediate microeconomic theor course. The standard calculus treatment of optimization (" rst- and secondorder conditions") is justi ed in Corollar 4 and Propositions 6 and 7. The "method of Lagrange" for optimization subject to a linear constraint is justi ed in Proposition 9 and the subsequent remarks. However, the notes should help the reader go somewhat beond the standard treatment, which tpicall deals onl with interior solutions and functions that are twice di erentiable. The notes should be accessible to someone with a good basic background in univariate and multivariate di erential and integral calculus. Proofs of propositions that are ver eas are omitted. On the other hand, the proofs of some of the propositions require a more advanced background, such as is usuall covered in a course in "real analsis;" those propositions are indicated with an asterisk, and the proofs are also omitted. For a more advanced treatment of convexit see, for example, H. L. Roden, Real Analsis, Prentice Hall, Englewood Cli s, NJ, 988, 2nd ed.

2 Functions of One Variable. Properties of Convex and Concave Functions This section deals with real-valued functions of a real variable, de ned on a nite interval, a half-line, or the entire real line. A function is convex if for an numbers x; ; t, such that 0 t, f[t + ( t)x] tf() + ( t)f(x): () (See Fig. a.) The point t + ( t)x is said to be a convex combination of the points x and. A function is said to be strictl convex if the inequalit in () is strict for 0 < t <. (Fig. b.) A function f is (strictl) concave if ( f) is (strictl) convex. (Fig. c.) In what follows, properties of concave functions that follow immediatel from properties of convex functions will be omitted. A function that is both convex and concave is linear. Proposition If f ; :::; f N are convex, and b ; :::; b N are nonnegative numbers, then f = P n b nf n is convex. Proposition 2 If f is convex, and c is an number, then the set of points x such that f(x) c is either empt, a point, an interval, a half-line or the whole line. Proposition 3 If f is convex, x ; :::; x N ; are an numbers, and t ; :::; t N ; are an nonegative numbers such that P n t n =, then! X f t n x n X t n f(x n ): (2) n n Proof. The proof is b induction. The proposition is triviall true for N =, and b the de nition of convexit it is true for N = 2: Suppose it is true for N K. I shall show that it is then true for N = K +. Let c = KX t n ; b n = t n c ; then, noting that c = t K+, and using the convexit of f,!! K+ X KX f t n x n = f c b n x n + t K+ x K+ : B the induction hpothesis, cf! KX b n x n + t K+ f (x K+ ) : f! KX b n x n KX b n f (x n ) ; 2

3 and hence cf! KX KX b n x n + t K+ f (x K+ ) c b n f (x n ) + t K+ f (x K+ ) = K+ X t n f(x n ); which completes the proof. For a function f, de ne the left-hand derivative, f (x); at a point x b f (x) = lim h!0 h0 f(x + h) h f(x) : (3) Similarl, de ne the right-hand derivative, f + (x); at a point x b f + (x) = lim h!0 h0 f(x + h) h f(x) : (4) If the left- and right-hand derivatives are equal to each other at a point, then their common value is the derivative of f at the point, and one sas that the function is di erentiable there. Recall that a set is countable if it can be put into one-to-one correspondence with the integers. Proposition 4 (*) If a function is convex on an open interval (a; b), then it is continuous there. Its left- and right-hand derivatives exist at each point of (a; b), and are equal to each other except on a countable set. For an example of a convex function that is not di erentiable at ever point, consider N linear functions, f n, with di erent slopes, and let f(x) = max n ff n(x)g ; then f is convex and piece-wise linear, and is not di erentiable at the "kinks" where di erent "pieces" join. (Fig. 2.) Proposition 5 If a function is convex on an open interval (a; b), then () its left- and right-hand derivatives are each monotone nondecreasing functions in that interval, (2) at each point the left-hand derivative does not exceed the righthand derivative, and (3) x < z implies f + (x) f (z). Proof. The proof is based on the following lemma. Lemma 6 Let x < < z be three points in (a; b); then f() f(x) f(z) x z f(x) f(z) x z f() : (5) 3

4 Proof of Lemma. We ma write as a convex combination of x and z as follows: x z = z + x: (6) z x z x B the convexit of f, x z f() f(z) + z x z so f() f(x) f() f(x) x x f(x) = ( x)f(z) + (z )f(x) ; z x ( x)[f(z) f(x)] ; z x f(z) f(x) ; z x which veri es the rst inequalit. A smmetric calculation veri es the second inequalit, and completes the proof of the Lemma. To complete the proof of the proposition, rst let! x; then the rst inequalit in the Lemma implies that Similarl, letting! z we get f + (x) f(z) z f (z) [Cf. the de nitions (3) and (4).] Hence f(x) : x f(z) f(x) : z x f + (x) f (z); (7) which proves part (3) of the conclusion. Next, letting x! and z! we get f () f + (): Since this last holds for an point in (a; b), it also holds at z, so that f (z) f + (z); which veri es part (2) of the conclusion of the theorem. Putting this last together with (7), we have f + (x) f + (z): A smmetric argument shows that f (x) f (z); which completes the proof of the theorem. The last proposition shows that if a di erentiable function is convex on an interval, then its derivative is nondecreasing. The converse is also true. 4

5 Proposition 7 If a function f is di erentiable on the interior of an interval, and its derivative is nondecreasing, then it is convex there. Corollar 8 If a function is twice di erentiable on the interior of an interval, then it is convex there if and onl if its second derivative is nonnegative. Furthermore, if its second derivative is strictl positive on the interior of the interval, then the function is strictl convex there. Proof of the Proposition. Let x < z be two points in the interior of the interval, let t be a number such that 0 < t <, and let = tz + ( t)x: As in (6) we ma write = z x z z + x z x: (8) x Since f is di erentiable and nondecreasing, so Similarl, f() f(x) = Z f 0 (s)ds Z x x f 0 ()ds = f 0 ()( x); f() f(x) f 0 (): (9) x f(z) f() = Z z f 0 (s)ds Z z f 0 ()ds = f 0 ()(z ); and so Hence, b (9) and (0), f(z) z f() f 0 (): (0) f() f(x) f(z) x z f() : Multipling the last inequalit b ( x)(z ), and simplifing the resulting inequalit, we get (z x)f() ( x)f(z) + (z )f(x); x z f() f(z) + f(x); z x z x which last, b (6), can be rewritten in the form, f[tz + ( t)x] tf(z) + ( t)f(x); 5

6 thus completing the proof of the proposition. The proof of the corollar is left as an exercise for the reader. Note, however, that the second derivative of a function can be zero at a point in the interior of an interval, and et the function can be strictl convex in the interval. An example is provided b the function de ned b f(x) = x 4, which is strictl convex on the whole real line, and et f 00 (0) = 0..2 Application to Risk Aversion De ne a lotter to be a random variable whose outcomes ("pao s") are amounts of mone. A lotter, sa X, is determined b X = [x ; :::; x N ; p ; :::; p N ]; where N is a positive integer, x ; :::; x N are the possible pao s, and The expected pao is p n = PrfX = x n g; n = ; :::; N: EX = NX p n x n : () n= According to a well-known theor of decision-making under uncertaint, if a person is "rational," then her preferences among lotteries can be represented (scaled) b a function u, such that she prefers lotter X to lotter X 0 if and onl if NX NX Eu(X) = p n u(x n ) > p 0 nu(x 0 n) = Eu(X 0 ): n= The mathematical expectation, Eu(X), is called the expected utilit of the lotter X. A special case of a lotter is one in which the person receives a xed pao, sa, for sure. The person is said to be averse to risk (or risk-averse) if she would prefer receiving the amount of mone EX for sure to receiving the actual lotter X. This will happen if or u n= u(ex) > Eu(X);! NX p n x n > n= NX p n u(x n ): (2) Since the probabilities p n are nonnegative and their sum is, we see that riskaversion is equivalent to the condition that the utilit function u be strictl concave. [Questions: If the person s utilit function is linear, then one sas that the person is risk-neutral. Wh is this an apt description? What behavior is exhibited b a person whose utilit function is strictl convex?] n= 6

7 .3 Application to Optimization In this subsection we characterize where a convex function attains a minimum on a closed interval, S = [a; b], with a and b nite, and a < b. Some of the results will carr over immediatel to the case of maximizing a concave function. It will also be eas to extend the results to the cases in which S is a half-line or the whole real line. These extensions will be left to the reader. Henceforth, S = [a; b]. A fundamental proposition of real analsis implies that a convex function does attain a minimum on S. Proposition 9 (*). If a function is continuous on S, then it attains a minimum at a point in S. This Proposition, together with Proposition 4, impl the following corollar. Corollar 0 If a function is convex on S, and continuous at the endpoints of S, then it attains a minimum at a point in S. We now characterize those points at which a convex function attains a minimum on S. In general, a convex function ma attain a minimum at more than one point. However, it follows from Proposition 2 that the set of all such points forms an interval (possibl consisting of a single point.) Proposition If f is convex on S, and continuous at the endpoints of S, then it attains a minimum at a point in S if and onl if satis es one of the following three conditions: (i) a < < b and f () 0 f + (); (ii) = a and 0 f + (); (iii) = b and f () 0: Proposition is illustrated in Figures 3(i), 3(ii), and 3(iii). The following corollaries are an immediate consequence of the Proposition and Proposition 5, and are stated without proof. Corollar 2 If, in addtion to the hpotheses of the Propositon, f is di erentiable, then (i) reduces to (i 0 ) a < < b and f 0 () = 0: Corollar 3 If, in addtion to the hpotheses of Propositon, f is strictl convex, then the minimizing point is unique. Proof of the Proposition. First consider the case in which a < < b. If satis es (i), then b Proposition 5, for all x < ; f (x) 0 and f + (x) 0; 7

8 hence, f(x) is nonincreasing in x and f(x) f() for x <. A smmetric argument shows that f(z) f() for z >. Hence minimizes f on S. Conversel, suppose that minimizes f on S, and a < < b. If f + () < 0, then there exists a z such that: and hence f(z) z z < b; f() < 0; f(z) < f(); and so does not minimize f on S. A smmetric argument shows that if f () > 0, then does not minimize f on S, either. This completes the proof for case (i). Similar, but one-sided, arguments can be applied to cases (ii) and (iii), completing the proof of Proposition. A further corollar deals with the case in which f is twice-di erentiable, i.e., its second derivative is continuous on the interior of S. This corollar provides the justi cation for the so-called " rst-order" and "second-order" conditions for a minimum. Corollar 4 Suppose that, in addition to the hpotheses of Proposition, f satis es the following four conditions: (i) then there exist numbers c; d such that f 00 is continuous on (a; b); (ii) a < < b; (iii) f 0 () = 0; (iv) f 00 () > 0; a < c < < d < b; is the unique minimizer of f on [c; d]: If, in addition, f 00 (x) > 0 on all of (a; b), then is the unique minimizer of f on [a; b]. Proof of Corollar. Since f 00 () > 0 and f 00 is continuous, there exist numbers c; d such that a < c < < d < b; f 00 (x) > 0 for c < x < d: Hence, b Corollar 3, is the unique minimizer of f on [c; d]. If f 00 (x) > 0 on all of (a; b), then take c = a; d = b, which completes the proof of the Corollar. In the preceding corollar, f() is sometimes called a local minimum. Of course, if f is not convex everwhere in S, there ma be more than one local minimum, and the need not have the same value. 8

9 2 Functions of Severable Variables 2. Convex Sets and Functions In this section, some of the results of the previous section are extended to the case of functions of several variables. These results are applied to some problems in the theor of consumer preferences and the theor of cost minimization subject to constraints. In addition to the mathematics used in the previous section, we shall use some ver elementar ideas from linear algebra. Recall that N-dimensional Euclidean space, denoted b R N, is the set of vectors x = (x ; :::; x N ), where the coordinates, x n, are real numbers. In this notation, R = R is the real line. What follows is a summar of some elementar properties of R N. If x and are in R N, and c is in R, then de ne x + to be the vector with coordinates (x n + n ), and cx to be the vector with coordinates cx n. The distance between x and in R N is de ned b s X kx k = (x n n ) 2 : The (open) ball with center x and radius r > 0 is de ned b B(x; r) = j 2 R N ; kx k < r : A subset S of R N is said to be open if for ever point x in S there exists a number r > 0 such that B(x; r) is entirel contained in S. A subset S of R N is convex if for an vectors x; ; in R N, and an number t, such that 0 t, t + ( t)x 2 S: In the rest of this section, the set S will be assumed to be open and convex, unless something is said to the contrar. Note that in the real line, this amounts to assuming that S is an open interval, nite or in nite. A (real-valued) function f on S is convex if, for an vectors x; ; in S, and an number t, such that 0 t, n f[t + ( t)x] tf() + ( t)f(x): (3) Note that this last inequalit looks the same as () in Section, except that x and are in R N, not just R. Propositions and 3 of Section 3 remain true in this more general setting, and Proposition 2 takes the form: Proposition 5 If f is convex, and c is an number, then the set of points x such that f(x) c is a (possibl empt) convex set. The proof is essentiall the same as for Proposition 2, and is omitted. Man of the other propositions in Section. have analogues in R N. However, a serious treatment of those topics is beond the scope of these Notes. 9

10 Henceforth, unless otherwise noted, it will be assumed that the functions being considered have continuous second-order partial derivatives on the set S. The following proposition is an analogue of Corollar 8 in Section.. It will not be used in these notes, but it is stated without proof for the sake of completeness, because it has man applications in economic theor and econometrics. A smmetric N N matrix Q = ((q mn )) is said to be nonnegative semi-de nite if for an vector x = (x ; :::; x N ) in R N, X q mn x m x n 0: m;n In addition, Q is said to be positive de nite if the preceding inequalit is strict for an x 6= 0. For a function f de ned on S, de ne Q(x) to be the matrix of its second partial derivatives evaluated at the vector x, i.e., using a standard (if ambiguous) notation, it is the matrix with elements f mn (x) n : Note that "f is twice di erentiable on S" means that the functions f mn (x) are continuous on S. Proposition 6 (*) If f is twice di erentiable on S, then f is convex if and onl if the matrix Q(x) is nonnegative semi-de nite at ever point x of S. Furthermore, if Q(x) is everwhere positive de nite then f is strictl convex. 2.2 Application to Optimization Our rst result is an analogue of Corollar 2 in Section.3. Recall that the set S is open and convex, and that we are restricting attention to functions f that are twice di erentiable. Denote the partial derivative of f with respect to its mth argument b f m, i.e. f m m : Proposition 7 If f is convex, then minimizes f on S if and onl if f m () = 0 for each m = ; :::; N. Lemma 8 Fix z in S, and let T denote the set of all t such that tz + ( is in S. For t 2 T, de ne t) g(t) = f[tz + ( t)]; (4) then g is convex. 0

11 Proof of Lemma. Note that, since S is open and convex, T is an open interval (possibl in nite). Let t and t 2 be in T, and let and 2 be nonnegative numbers such that their sum is. Let x = ( t + 2 t 2 )z + ( t 2 t 2 ); then One easil veri es that g( t + 2 t 2 ) = f(x): Hence, since f is convex, x = [t z + ( t )] + 2 [t 2 z + ( t 2 )]: f(x) f[t z + ( t )] + 2 f[t 2 z + ( t 2 )] = g(t ) + 2 g(t 2 ); which completes the proof of the Lemma. To complete the proof of the Proposition, note that g 0 (t) = X m f m [tz + ( t)]: The rest of the proof now follows from a straightforward application of Corollar 2. I now turn to the problem of minimizing a convex function f on S subject to a linear constraint. (Two examples from microeconomic theor are presented in subsequent subsections.) Let L be a linear function on S, given b L(x) = X n b n x n ; (5) where at least one of the coe cients, b n, is di erent from zero. We wish to characterize the vectors that minimize f(x) subject to the constraint that L(x) = X n b n x n = c; (6) where c is an given number such that at least one vector in S satis es the constraint. Denote b K the set of points in S that satisf the constraint. Note that K is convex (an eas exercise), and since we have assumed that it is not empt, and S is open, it contains more than one point. A vector that minimizes f on K will be called optimal. Proposition 9 is optimal if and onl if there is a number such that f n () = b n for n = ; :::; N: (7)

12 Proof. Let be in K. If maximizes f on S (not just on K), then b the previous proposition we ma take = 0. Conversel, if = 0 then (again using the previous proposition) maximizes f on S, and hence on K. Hence, without loss of generalit we ma suppose that does not maximize f on S. If necessar, renumber the coordinates so that b N 6= 0. Solve the constraint for x N so that and de ne Let g(x ; :::; x N x N = c b N NX n= b N b n x n! " ) = f x ; :::; x N ; c ; (8) NX n=! # b n x n : S 0 = f(x ; :::; x N ) : for some x N ; (x ; :::; x N ; x N ) 2 Sg ; then S 0 is open in R N, and g is convex and twice di erentiable in S 0. The rst derivatives of g are given b bn g n (x ; :::; x N ) = f n (x) f N (x); n = ; :::; N ; where it is understood that in these last equations, x N is given b (8). The proof is now completed b appling the previous proposition, and taking b N = f N(x) b N : Remark. The optimal vector and the number both depend, in principle, on the parameter c of the constraint equation. In this context, has an interesting interpretation. With a slight abuse of notation, rewrite the optimalit condition of Proposition 9 as f n [(c)] = (c)b n for n = ; :::; N: The minimum of the function f on the constraint set K is then F (c) = f[(c)]: If the n are di erentiable with respect to c, then the derivative of F with respect to c is F 0 (c) = X n = X n f n [(c)] 0 n(c) (c)b n 0 n(c) = (c) X n b n 0 n(c): 2

13 Now note that di erentiating both sides of the constraint equation (6) with respect to c ields X b n n(c) 0 = ; and so we have the result: n Corollar 20 F 0 (c) = (c): (9) Thus (c) gives us the approximate rate at which a "small change " in the constant c changes the minimum value of the function f. Remark 2. De ne the function L b L() = f() [L() c]: Setting the derivatives of L with respect to the coordinates of equal to zero gives us the optimalit conditions of the proposition, and setting the derivative with respect to equal to zero gives us the constraint equation. This recipe is called the "method of Lagrange," and in this context is called the "Lagrange multiplier." The preceding discussion gives a set of conditions under which the recipe gives the right answer. Remark 3. Note that b n is the partial derivative of the constraint function L. This suggest how Proposition 9 can be generalized to cover the case of nonlinear functions L that satisf certain regularit conditions. 2.3 Application to Cost Minimization Suppose that a rm has several plants that can produce the same commodit (e.g., electricit). The cost of producing a quantit x n in plant n is C(x n ), so the total cost of producing the vector of outputs x = (x ; :::; x N ) is C(x) = X n C n (x n ): The rm wants to produce a total quantit q at minimum cost. Hence it wants to nd a vector x that minimizes C(x) subject to the constraint that X x n = q: n Of course, the output of each plant must be nonnegative. We consider here the case in which at an optimum the output of ever rm is strictl positive. Thus take S to be the set de ned b the strict inequalities: x n > 0; n = ; :::; N: Assume that each cost function C n is nonnegatve, twice di erentiable, strictl increasing, and strictl convex. It follows that C has the same properties. 3

14 Note that, if we assume further that C 0 n(0) = 0, then it is an eas exercise to show that, at the optimum, the output of ever plant will be strictl positive. Appling Proposition 9, one immediatel gets the optimalit condition: C 0 n( n ) = ; n = ; :::; N: The derivative C 0 n( n ) is called the marginal cost for plant n, so the optimalit condition can be paraphrased: "The total quantit q should be allocated to the plants so as to equalize their marginal costs." From Corollar 20, the value of the Lagrangean multiplier,, at the optimum is equal to the derivative of the minimum total cost with respect to the required quantit of total output. [The reader should consider how the optimalit conditions would be modi ed if at the optimum some plants have zero output. Note that this could happen without the assumption that, for each plant, C 0 n(0) = 0. The reader should also consider the case in which each plant has a xed capacit, i.e., an upper bound on output.] Example. Let C n (x n ) = c n x 2 n; then and so at an optimal, C 0 n(x n ) = 2c n x n ; C 0 n(0) = 0; 2c n n = ; n = ; 2c n q = X X n = ; 2 c n n n 2q = Pn : c n 2.4 Application to Consumer Choice Suppose that a consumer must choose a vector x of consumption (in a given period) subject to an expenditure constraint and nonnegativit constraints, X p n x n ; n x n 0; n = ; :::; N; where p n is the price of commodit n, and is the maxiumum feasible expenditure. Let u(x) denote the consumer s utilit from consuming the vector x, and suppose that the consumer is not satiated in the constraint set, so that she will spend up to the limit. Assume that the utilit function is concave and 4

15 twice di erentiable, so that u is convex. Suppose, also, that the maximum will occur at a point at which the consumption of ever commodit is strictl positive. (Ever commodit is a "necessit.") Thus we can take S to be the set de ned b the strict inequalities: x n > 0; n = ; :::; N; and the consumer s problem is to minimize u(x) on S subject to the linear constraint, X p n x n = : Appling Proposition 9, we get the condition that or, for all m; n, n u n (x) = p n for n = ; :::; N; u m (x) u n (x) = p m p n : The ratio u m (x)=u n (x) is called the "marginal rate of substition," i.e., it is the (in nitesimal) rate at which the consumer can substitute commodit m for commodit n, keeping her utilit constant. Hence the optimalit condition can be paraphrased: "for an two commodities, the marginal rate of substitution equals the price ratio." Finall, the reader should consider how the optimalit conditions would be modi ed if at the optimal consumption vector the consumption of one or more commodities would be zero. Exercise. Analze the example in which u(x) = X n a n ln x n ; where the parameters a n are strictll positive. Acknowledgment. I thank A. Radunskaa for helpful comments on an earlier draft. I am, of course, solel responsible for an errors. 5

16 Fig. a. A convex function Fig. b. A strictl convex function Fig. c. A strictl concave function

17 Fig. 2. A piecewise-linear convex function

18 a b Fig. 3 (i) a b Fig. 3 (ii) a Fig. 3 (iii) b