Constrained maxima and Lagrangean saddlepoints

Transcription

1 Division of the Humanities and Social Sciences Ec 181 KC Border Convex Analysis and Economic Theory Winter 2018 Topic 10: Constrained maxima and Lagrangean saddlepoints 10.1 An alternative As an application of the separating hyperplane theorem we present the following result due to Fan, Glicksberg, and Hoffman [1] Concave Alternative Theorem Let C be a nonempty convex subset of a vector space, and let f 1,..., f m : C R be concave. Letting f = (f 1,..., f m ): C R m, exactly one of the following is true. ( x C ) [ f( x) 0 ]. (1) Or (exclusive), ( p > 0 ) ( x C ) [ p f(x) 0 ]. (2) Proof : Clearly both cannot be true. Suppose (1) fails. Define A = {f(x) : x C} and let  be its decreasing hull,  = { y R m : ( x C ) [ y f(x) ] }. Since (1) fails, we see that A and R m ++ are disjoint. Consequently  and Rm ++ are disjoint. Now R m ++ is clearly convex, and observe that  is also convex. (The set A itself need not be convex see, for instance, Figure ) To see this, suppose y 1, y 2 Â. Then y i f(x i ), i = 1, 2. Therefore, for any λ (0, 1), λy 1 + (1 λ)y 2 λf(x 1 ) + (1 λ)f(x 2 ) f ( λx 1 + (1 λ)x 2 ), since each f j is concave. Therefore λy 1 + (1 λ)y 2 Â. Thus, by the Separating Hyperplane Theorem 8.5.1, there is a nonzero vector p R m properly separating  and Rm ++. We may assume p  p Rm ++. (3) By Exercise 8.2.4, p > 0. Let ε > 0, so that ε1 0, and note that (3) implies for every y Â, we have p y εp 1. Since ε may be taken arbitrarily small, we conclude that p y 0 for all y in Â. In particular, p f(x) 0 for all x in C. KC Border: for Ec 181, Winter 2018 src: SaddlePoint v ::15.56

2 KC Border Constrained maxima and Lagrangean saddlepoints Saddlepoints In this section we discuss the relation between constrained maxima of concave functions and saddlepoints of the so-called Lagrangean Definition Let f : X Y R. A point (x ; y ) 1 in X Y is a saddlepoint of f (over X Y ) if it satisfies f(x; y ) f(x ; y ) f(x ; y) for all x X, y Y. That is, (x ; y ) is a saddlepoint of f if x maximizes f( ; y ) over X and y minimizes f(x ; ) over Y. Saddlepoints of a function have the following nice interchangeability property Lemma (Interchangeability of saddlepoints) Let f : X Y R, and let (x 1 ; y 1 ) and (x 2 ; y 2 ) be saddlepoints of f. Then f(x 1 ; y 1 ) = f(x 2 ; y 1 ) = f(x 1 ; y 2 ) = f(x 2 ; y 2 ). Consequently (x 1 ; y 2 ) and (x 2 ; y 1 ) are also saddlepoints Exercise Prove Lemma Lagrangeans Saddlepoints play an important rôle in the analysis of constrained maximum problems via Lagrangean functions Definition Given f, g 1,..., g m : X R, the associated Lagrangean L: X Λ R is defined by L(x; λ) = f(x) + λ j g j (x) = f(x) + λ g(x), where Λ is an appropriate subset of R m. (Usually Λ = R m or R m +.) The components of λ are called Lagrange multipliers. 2 The first result is that saddlepoints of Lagrangeans are constrained maxima. This result requires no restrictions on the functions. 1 The use of a semicolon instead of a comma is to visually emphasize the differing roles of x and y, but mathematically it plays the same role as a comma. On occasion I may forget this convention and use a comma. Don t fret about it. 2 You may have noticed that I have departed from my usual convention of letting Greek letters denote scalars. This usage of λ to denote a vector of Lagrange multipliers is one of my many bad habits. v ::15.56 src: SaddlePoint KC Border: for Ec 181, Winter 2018

3 KC Border Constrained maxima and Lagrangean saddlepoints Theorem (Lagrangean saddlepoints are constrained maxima) Let X be an arbitrary set, and let f, g 1,..., g m : X R. Suppose that (x, λ ) is a saddlepoint of the Lagrangean L(x; λ) = f + λ g (over X R m +). That is, L(x; λ ) L(x ; λ ) L(x ; λ) x X, λ 0. (4) (4a) (4b) Then x maximizes f over X subject to the constraints g j (x) 0, j = 1,..., m, and furthermore λ jg j (x ) = 0, j = 1,..., m. (5) Proof : Inequality (4b) implies λ g(x ) λ g(x ) for all λ 0. By the Nonnegativity Test 0.1.1, g(x ) 0, so x satisfies the constraints. Setting λ = 0, we see that λ g(x ) 0. This combined with λ 0 and g(x ) 0 implies λ g(x ) = 0. Indeed it implies λ jg j (x ) = 0 for j = 1,..., m. Now note that (4a) implies f(x) + λ g(x) f(x ) for all x. Therefore, if x satisfies the constraints, g(x) 0, we have f(x) f(x ), so x is a constrained maximizer. Condition (5) implies that if the multiplier λ j is strictly positive, then the corresponding constraint is binding, g j (x ) = 0; and if a constraint is slack, g j (x ) > 0, then the corresponding multiplier satisfies λ j = 0. These conditions are sometimes called the complementary slackness conditions. The converse of Theorem is not quite true, but almost. To state the correct result we now introduce the notion of a generalized Lagrangean Definition A generalized Lagrangean L µ : X Λ R, where µ 0, is defined by L µ (x; λ) = µf(x) + λ j g j (x), where Λ is an appropriate subset of R m. Note that each choice of µ generates a different generalized Lagrangean. However for Λ = R m +, as long as µ > 0, a point (x; λ) is a saddlepoint of the generalized Lagrangean if and only if (x; λ/µ) is a saddlepoint of the Lagrangean. Thus the only case to worry about is µ = 0. The next results state that for concave functions satisfying a regularity condition, constrained maxima are saddlepoints of some generalized Lagrangean Theorem (Concave constrained maxima are nearly Lagrangean saddlepoints) Let C R n be convex, and let f, g 1,..., g m : C R be concave. Suppose x maximizes f subject to the constraints g j (x) 0, j = 1,..., m. Then there exist real numbers µ, λ 1,..., λ m 0, not all zero, such that (x ; λ ) is a saddlepoint of the generalized Lagrangean L µ. That is, µ f(x) + λ jg j (x) µ f(x ) + (6a) λ jg j (x ) µ f(x ) + (6b) λ j g j (x ) (6) KC Border: for Ec 181, Winter 2018 src: SaddlePoint v ::15.56

4 KC Border Constrained maxima and Lagrangean saddlepoints 10 4 for all x C and all λ 1,..., λ m 0. Furthermore λ jg j (x ) = 0. (7) Proof : Since x is a constrained maximizer there is no x C satisfying f(x) f(x ) > 0 and g(x) 0. Therefore the Concave Alternative implies the existence of nonnegative µ, λ 1,..., λ m, not all zero, satisfying µ f(x) + λ jg j (x) µ f(x ) for every x C. Evaluating this at x = x yields m λ jg j (x ) 0. But each term in this sum is the product of two nonnegative terms, so (7) holds. This in turn implies (6a). Given that g j (x ) 0 for all j, (7) also implies (6b) Corollary (Constrained maxima may be Lagrangean saddlepoints) Under the hypotheses of Theorem suppose in addition that Slater s Condition, ( x C ) [ g( x) 0 ], (S) is satisfied. Then µ > 0, and may be taken equal to 1. Consequently the pair ( x ; (λ 1,..., λ m) ) is a saddlepoint of the Lagrangean for x C, λ 0. That is, where L(x; λ) = f(x) + λ g(x). L(x; λ ) L(x ; λ ) L(x ; λ) x C, λ 0, (8) Proof : Suppose µ = 0. Then evaluating (6) at x = x yields λ g( x) 0, but g( x) > 0 implies λ j = 0, j = 1,..., m. Thus µ = 0 and λ j = 0, j = 1,..., m, a contradiction. Therefore µ > 0, and by dividing the Lagrangean by µ, we may take µ = 1. The remainder is then just Theorem Combining these results gives us the following The Saddlepoint Theorem Let C R n be convex, and let f, g 1,..., g m : C R be concave. Assume in addition that Slater s Condition, is satisfied. The following are equivalent. ( x C ) [ g( x) 0 ] (S) 1. The point x maximizes f over C subject to the constraints g j (x) 0, j = 1,..., m. v ::15.56 src: SaddlePoint KC Border: for Ec 181, Winter 2018

5 KC Border Constrained maxima and Lagrangean saddlepoints Then there exist real numbers λ 1,..., λ m 0 such that (x ; λ ) is a saddlepoint of the Lagrangean L. That is, f(x) + λ jg j (x) f(x ) + λ jg j (x ) f(x ) + λ j g j (x ) (9) for all x C and all λ 1,..., λ m 0. Furthermore λ jg j (x ) = 0, j = 1,..., m Remark Sometimes it hard to see the forest for the trees. The most important consequence of the Saddlepoint Theorem is that for the concave case satisfying Slater s condition, the constrained maximizer x of f is an unconstrained maximizer of the Lagrangean L( ; λ ). The rôle of a Lagrange multiplier is to act as a conversion factor between the g-values and the f-values that convert the constrained maximization problem to an unconstrained maximization problem Example Concavity is crucial for these results. Even in the classical case of an interior constrained maximizer of a smooth function, even when the Lagrange Multiplier Theorem applies, the constrained maximizer may not be an unconstrained maximizer of the Lagrangean. Sydsaeter [4] points out the following simple example. Let f(x, y) = xy and g(x, y) = 1 (x + y), so the Lagrangean is L(x, y; λ) = xy + λ(1 x y). Note that g is linear and so concave, but f is quasiconcave, but not concave, since it exhibits increasing returns. The point (x, y ) = (1, 1) maximizes f subject to the constraints g(x, y) 0, x 0, y 0. The point ( x, ȳ) = (0, 0) verifies Slater s condition. The multiplier λ = 2 satisfies the conclusion of the classical Lagrange multiplier theorem, namely the first order conditions L(x, y ; λ ) x = 2x y λ = 0 and But (x, y ) does not maximize L(x, y ; λ ) y L(x, y; 2) = xy + 2 2x 2y. = 2x y λ = 0. Indeed there is no unconstrained maximizer of the Lagrangean for any λ 0. KC Border: for Ec 181, Winter 2018 src: SaddlePoint v ::15.56

6 KC Border Constrained maxima and Lagrangean saddlepoints 10 6 Karlin [2, vol. 1, Theorem 7.1.1, p. 201] proposed the following alternative to Slater s Condition: ( λ > 0 ) ( x(λ) C ) [ λ g ( x(λ) ) > 0 ], which we may as well call Karlin s condition Theorem Let C R n be convex, and let g 1,..., g m : C R be concave. Then g satisfies Slater s Condition if and only it satisfies Karlin s Condition. Proof : Clearly Slater s Condition implies Karlin s. Now suppose g violates Slater s Condition. Then by the Concave Alternative Theorem , it must also violate Karlin s. The next example shows what can go wrong when Slater s Condition fails Example In this example, due to Slater [3], C = R, f(x) = x, and g(x) = (1 x) 2, so both f and g are concave. Note that Slater s Condition fails because g 0. The constraint set {g 0} is the singleton {1}. Therefore f attains a constrained maximum at x = 1. There is however no saddlepoint over R R + of the Lagrangean L(x; λ) = x λ(1 x) 2 = λ + (1 + 2λ)x λx 2. To see that L has no saddlepoint, consider an arbitrary ( x, λ) R R +. If λ = 0, then since L(x; 0) = x, no value for x maximizes L( ; 0). On the other hand if λ > 0, the first order condition for a maximizer at x is L = 0, or 1+2 λ 2 λ x = 0, x which implies x = 1 + 1/(2 λ) > 1. But for x > 1, L( x; ) is strictly decreasing, so no λ is a minimizer The rôle of Slater s Condition In this section we present a geometric argument that illuminates the rôle of Slater s Condition in the saddlepoint theorem. The saddlepoint theorem was proved by invoking the Concave Alternative Theorem , so let us return to the underlying argument used in its proof. In the framework of Theorem , define the function h: C R m+1 by h(x) = ( g 1 (x),..., g m (x), f(x) f(x ) ) and set A = {h(x) : x C} and  = {y R m+1 : ( x C ) [ y h(x) ] }. Then  is a convex set bounded in part by A. Figure depicts the sets A and  for Slater s example , where f(x) f(x ) is plotted on the vertical v ::15.56 src: SaddlePoint KC Border: for Ec 181, Winter 2018

7 KC Border Constrained maxima and Lagrangean saddlepoints 10 7 A f(x) f(x ) f(x) f(x ) 1 h(x ) g(x) 1  h(x ) g(x) A 1 1 Figure The sets A and  for Slater s example. axis and g(x) is plotted on the horizontal axis. Now if x maximizes f over the convex set C subject to the constraints g j (x) 0, j = 1,..., m, then h(x ) has the largest vertical coordinate among all the points in H whose horizontal coordinates are nonnegative. The semipositive m + 1-vector ˆλ = (λ 1,..., λ m, µ ) from Theorem is obtained by separating the convex set  and Rm It has the property that ˆλ h(x) ˆλ h(x ) for all x C. That is, the vector ˆλ defines a hyperplane through h(x ) that supports  at h(x ). It is clear in the case of Slater s example that the supporting hyperplane is a vertical line. The fact that the hyperplane is vertical means that µ (the multiplier on f) must be zero. If there is a non-vertical supporting hyperplane through h(x ), then µ is nonzero, so we can divide by it and obtain a full saddlepoint of the true Lagrangean. This is where Slater s condition comes in. In the one dimensional, one constraint case, Slater s Condition reduces to the existence of x satisfying g( x) > 0. This rules out having a vertical supporting line through h(x ). To see this, note that the vertical component of h(x ) is f(x ) f(x ) = 0. If g(x ) = 0, then the vertical line through h(x ) is simply the vertical axis, which cannot support Â, since h( x) A lies to the right of the axis. See Figure In Figure , the shaded area is included in Â. For instance, let C = (, 0], f(x) = x, and g(x) = x + 1. Then the set  is just {y R2 : y (0, 1)}. Later we shall see that if f and the g j s are linear, then Slater s Condition is not needed to guarantee a non-vertical supporting line. Intuitively, the reason for this is that for the linear programming problems considered, the set  is polyhedral, so even if g(x ) = 0, there is still a non-vertical line separating  and Rm The proof of this fact relies on some results on linear inequalities that will be proven KC Border: for Ec 181, Winter 2018 src: SaddlePoint v ::15.56

8 KC Border Constrained maxima and Lagrangean saddlepoints 10 8 f(x) f(x ) h(x ) g(x)  h( x) Figure Slater s condition rules out a vertical supporting line. later. It is subtle because Slater s condition rules out a vertical supporting line. In the linear case, there may be a vertical supporting line, but if there is, there is also a non-vertical supporting line that yields a Lagrangean saddlepoint. As a case in point, consider C = (, 0], f(x) = x, and g(x) = x. Then the set  is just {y R 2 : y 0}, which is separated from R 2 ++ by every semipositive vector Decentralization and Lagrange Multipliers This section is inspired by Uzawa [6]. Consider the case of a firm that can produce n different products, each according to a one-product concave production function f j of l inputs or factors. The firm has already contracted to have a quantity ω k of factor k available for use. (Alternatively, consider a country with n industries that sell their outputs on world markets and each factor is immobile and fixed in supply.) If the sale price of the j th output is p j > 0, then the firm (or country) faces the constrained maximization problem of maximizing the value of output, subject to the resource constraint on each factor: maximize v 1,...,v n n p j f j (v j ) subject to n v jk ω k, v jk 0, k = 1,..., l j = 1,..., n k = 1,..., l. The Lagrangean for this problem is: n l n L(v; µ) = p j f j (v j ) + µ k ω k v jk. k=1 By assumption, each f j is concave, so the objective function n p j f j (v j ) is concave. Moreover, the constraint function g k (v) := ω k n v jk is affine in v, and so concave. Note that as long as each ω k > 0, then Slater s Condition is satisfied. v ::15.56 src: SaddlePoint KC Border: for Ec 181, Winter 2018

9 KC Border Constrained maxima and Lagrangean saddlepoints 10 9 Therefore by the Saddlepoint Theorem , a point v solves the constrained maximization problem if and only if the there is a vector µ such that ( v; µ) is a saddlepoint of the Lagrangean over ( ) R l n + R l +. Now regroup the Lagrangean as ( ) ( ) l l l L(v; µ) = p 1 f 1 (v 1 ) µ k v 1k + + p n f n (v n ) µ k v nk + µ k ω k. k=1 By the Saddlepoint Theorem the constrained maximizer v is an unconstrained maximizer of the Lagrangean evaluated at µ = µ. But note that this implies that for each j = 1,..., n, l v j maximizes p j f j (v j ) µ k v jk. k=1 In other words the saddlepoint values of the Lagrange multipliers are factor wages such that each optimal v j unconstrainedly maximizes the profit at price p j and factor wages µ. These multipliers, sometimes known as shadow wages, allow the constrained problem to be decentralized as n independent profit maximization decisions. You should verify that the converse is true, namely if each v j unconstrainedly maximizes the profit at price p j and factor wages µ, and the market for inputs clears, that is, if ( n v j ω and each µ k ωk l v jk ) = 0, then v solves the constrained maximization problem. Thus the prices of of the outputs determine the wages of the factors as saddlepoint Lagrange multipliers. k=1 k= References [1] K. Fan, I. Glicksberg, and A. J. Hoffman Systems of inequalities involving convex functions. Proceedings of the American Mathematical Society 8: [2] S. Karlin Mathematical methods and theory in games, programming, and economics. New York: Dover. Reprint of the 1959 two-volume edition published by Addison Wesley. [3] M. L. Slater Lagrange multipliers revisited: A contribution to non-linear programming. Discussion Paper Math. 403, Cowles Commission. Reissued as Cowles Foundation Discussion Paper #80 in [4] K. Sydsaeter Letter to the editor on some frequently occurring errors in the economic literature concerning problems of maxima and minima. Journal of Economic Theory 9(4): DOI: / (74) KC Border: for Ec 181, Winter 2018 src: SaddlePoint v ::15.56

10 KC Border Constrained maxima and Lagrangean saddlepoints [5] H. Uzawa The Kuhn Tucker conditions in concave programming. In K. J. Arrow, L. Hurwicz, and H. Uzawa, eds., Studies in Linear and Nonlinear Programming, number 2 in Stanford Mathematical Studies in the Social Sciences, chapter 3, pages Stanford, California: Stanford University Press. [6] A note on the Menger Wieser theory of imputation. Zeitschrift für Nationalökonomie 18(3): DOI: /BF v ::15.56 src: SaddlePoint KC Border: for Ec 181, Winter 2018