Calculus an optimization These notes essentially correspon to mathematical appenix 2 in the text. 1 Functions of a single variable Now that we have e ne functions we turn our attention to calculus. A function f is i erentiable if it is continuous an "smooth" with no breaks or kinks. A erivative, f 0 (x), gives the slope or instantaneous rate of change in f (x). If f 0 (x) is i erentiable, then we can n the secon erivative, f 00 (x), too. The secon erivative is relate to the curvature of the original function as it tells how the rst erivative is changing. Table 1 provies some common rules of i erentiation. We now state some equivalency relations for concave functions. Theorem 1 Let D be a nonegenerate interval of real numbers on which f is twice continuously i erentiable. The following statements are equivalent: 1. f is concave 2. f 00 (x) 0 for all x 2 D 3. For all x 0 2 D : f (x) f x 0 + f 0 x 0 x x 0 4. If f 00 (x) < 0 for all x 2 D, then f is strictly concave. From our prior theorem relating convex an concave functions we also have a theorem for convex functions by reversing the inequalities an changing the wor "concave" to "convex". 2 Functions of several variables Many times we will work with functions of several variables. In consumer theory rms have utility functions which are functions of the quantities consume of various goos; in proucer theory rms have cost functions which are functions of the quantities of inputs use to prouce a goo. Since there are multiple variables, it is sometimes easier to think in terms of the slope of one particular variable, holing the other variables constant. De nition 2 Let y = f (x 1 ; :::; x n ). The partial erivative of f with respect to x i is e ne as: (x) lim h!0 f (x 1 ; :::; x i + h; :::; x n ) f (x 1 ; :::; x i ; :::; x n ) h (1) Constants, Sums Power rule Prouct rule Quotient rule Chain rule ln rule x () x (f (x) g (x)) = f 0 (x) g 0 (x) x (xn ) = nx n 1 x (f (x) g (x)) = f (x) g0 (x) + f 0 (x) g (x) x f(x) g(x) = g(x)f 0 (x) f(x)g 0 (x) (g(x)) 2 x (f (g (x))) = f 0 (g (x)) g 0 (x) ( ln x) x = x Table 1: Common rules of i erentiation 1
The rst thing to notice is that there are n partial erivatives. In essence, to n the n th partial erivative of a particular function simply treat the other n 1 variables as constants. If f (x 1 ; x 2 ) = x 2 1 + 3x 1 x 2 x 2 2, then we have: = 2x 1 + 3x 2 (2) = 3x 1 2x 2 If we collect the n partial erivatives into a row vector, then this will be the graient of the function. for our example the graient, e ne as rf (x) is: h i = 2x 1 + 3x 2 3x 1 2x 2 We can also consier the partial erivatives of the graient, as the secon partials of the original function shoul provie information about the curvature of the original function. Note that if one takes the secon partial erivatives of a row vector of length n then the result will be an nn matrix of secon partials. This matrix of secon partial erivatives is calle the Hessian matrix. Consiering our example of f (x 1 ; x 2 ) = x 2 1 + 3x 1 x 2 x 2 2, the Hessian matrix, H (x), is: " # @ 2 f @ 2 f @x H (x) = 1 2 3 = (4) 3 2 @ 2 f @ 2 f A theorem which we will be able to invoke later is Young s Theorem, which basically states that the orer in which the partial erivatives are taken oes not a ect the resulting secon partial erivative: Theorem 3 (Young s Theorem) For any twice i erentiable function f (x) @ 2 f = @2 f 2 = 2 (5) @x j @x j We can see in our example that = 3. While we will not get into the etails of curvature it shoul be helpful to state some results: Theorem 4 Let D be a convex subset of R n with a nonempty interior on which f is twice continuously i erentiable. The following statements are equivalent: 1. f is concave 2. H (x) is negative semie nite for all x in D. 3. For all x 0 2 D : f (x) f x 0 + rf x 0 x x 0 2.1 Homogeneous functions A homogeneous function are real-value functions which behave in particular ways as all variables are increase simultaneously an in the same proportion. De nition 5 A real-value function is calle homogeneous of egree k if f (tx) t k f (x) (6) If a function is homogeneous of egree 0, then increasing the variables in the same proportion will leave the value of the function unchange. If the function is homogeneous of egree 1, then increasing the variables in the same proportion will increase the value of the function in the same proportion (if all the variables are ouble, the value of the function oubles, etc.). So (3) 2
3 Optimization Suppose we have a single variable function f (x). Assume that f (x) is i erentiable. The function f (x) achieves a local maximum at x if f (x ) f (x) for all x in some neighborhoo of x. The function f (x) achieves a global maximum at x if f (x ) f (x) for all x in the omain of the function. The local maximum is unique if f (x ) > f (x) for all x in some neighborhoo of x an the global maximum is unique if f (x ) > f (x) for all x in the omain of the function. Similarly, the function achieves a local minimum at ex if f (ex) f (x) for all x in some neighborhoo of ex an a global mininum at ex if f (ex) f (x) for all x in the omain of the function. If the inequalities are strict then the minimum is unique. What follows are the rst-orer necessary conitions an secon-orer necessary conitions for the optima of a general twice continuously i erentiable function of one variable: Theorem 6 Let f (x) be a twice continuously i erentiable function of one variable. local interior: 1. maximum at x =) f 0 (x ) =) f 00 (x ) 0 2. minimum at ex =) f 0 (x ) =) f 00 (ex) 0 Then f (x) reaches a 3.1 Optima for real-value functions of multiple variables Suppose that D R n an let f : D! R be a twice continuously i erentiable real-value function of n variables. Intuitively, a local maximum is achieve at x if no small movement away from x results in the function increasing. The rst-orer necessary conitions for a local interior optima at x are as follows: Theorem 7 If the i erentiable function f (x) reaches a local interior maximum or minimum at x then x solves the system of simultaneous equations: (x ) (x ) ::: (x ) @x n (7) The secon-orer necessary conitions are as follows: Theorem 8 Let f (x) be twice continuously i erentiable. 1. If f (x) reaches a local interior maximum at x, then H (x ) is negative semie nite. 2. If f (x) reaches a local interior minimum at ex, then H (ex) is positive semie nite. Again, we will not get into the etails of negative an positive semie niteness, but the function will be strictly concave if the principal minors of the Hessian matrix alternate in sign, beginning with a negative sign. It will be strictly convex if the principal minors of the Hessian matrix are all positive. 3.2 Constraine optimization Thus far we have focuse on unconstraine optimization. However, in many problems there are constraints which must be met. They may be equality constraints, such that we are optimizing a particular function f x 1 ; x 2 subject to the equality constraint g x 1 ; x 2. They may be constraints as simple as nonnegativity constraints, so that x 1 0 an x 2 0. They may be more complex inequality constraints, such that we are optimizing the function f x 1 ; x 2 subject to g x 1 ; x 2 0. The function g x 1 ; x 2 may be linear or nonlinear. 3
3.2.1 Equality constraints Formally, an optimization with an equality constraint is set up as: max f (x 1 ; x 2 ) s.t. g (x 1 ; x 2 ) (8) x 1;x 2 The function which we are optimizing, f in this instance, is calle the objective function. The variables that are being chosen, x 1 an x 2 in this problem, are the choice variables. The function g (x 1 ; x 2 ) is calle the constraint function. With equality constraine optimization problems the problem can be solve by substitution. If we solve for one of the variables in the constraint function, say x 2, we woul have a new function: Substitute this irectly into the objective function an the problem becomes: x 2 = eg (x 1 ) (9) max x 1 f (x 1 ; eg (x 1 )) (10) an then we are maximizing a function of a single variable. We then set the rst erivative equal to zero an n the critical value for x 1. This rst orer conition is: (x 1; eg (x 1)) + (x 1; eg (x 1)) eg (x 1) x 1 (11) Once x 1 is known we n x 2 by using x 2 = eg (x 1). For simple problems this substitution metho works well; when the constraint functions are complex or there are multiple choice variables (or constraints) this metho becomes teious. 3.2.2 Lagrange s metho Solving unconstraine optimization problems is (relatively) easy. The iea that Lagrange ha was to turn constraine optimization problems into unconstraine optimization problems. Our problem from before is: max f (x 1 ; x 2 ) s.t. g (x 1 ; x 2 ) (12) x 1;x 2 Now, multiply the constraint by the variable (why? that comes later). A that prouct to the objective function an we have create a new function calle the Lagrangian function (or Lagrangian). The Lagrangian is: L (x 1 ; x 2 ; ) = f (x 1 ; x 2 ) + g (x 1 ; x 2 ) (13) The rst-orer necessary conitions for optimizing the Lagrangian are the set of partial erivatives set equal to zero: = (x 1; x 2) + @g (x 1; x 2) (14) = (x 1; x 2) + @g (x 1; x 2) (15) @ = g (x 1; x 2) (16) The iea of Lagrange s metho is that if we solve these three equations simultaneously for x 1, x 2, an we will have foun a critical point of f (x 1 ; x 2 ) along the constraint g (x 1 ; x 2 ). While we will not go through all of the etails to prove why this metho works, the book provies a nice iscussion. Also, as of now we o not know whether or not these critical values are maxima or minima, but some comments on this topic will be mae shortly. Note that Lagrange s metho can be use for any number of variables (n) an any number of constraints (m) so long as the number of constraints is less than the number of variables (m < n). There is still the question of whether or not a solution actually exists for a problem an whether or not the variable exists. While these are important questions a general iscussion of these concepts is more than is neee here. When we iscuss particular economic optimization problems we will iscuss these concepts in slightly more etail. The formal statement of Lagrange s theorem is here (note that is a vector of the variables): 4
Theorem 9 Let f (x) an g j (x), j = 1; :::; m, be continuously i erentiable real-value functions over some omain D R n. Let x be an interior point of D an suppose that x is an optimum (maximum or minimum) of f subject to the constraints, g j (x ), j = 1; :::; m. If the graient vectors rg j (x ), j = 1; :::; m are linearly inepenent, then there exist m unique numbers j, j = 1; :::; m such that (x ; ) = (x ) + mx j=1 @g j (x ) j for i = 1; :::; n (17) So this theorem guarantees us the existence of the variables. As for etermining whether or not the constraine function is attaining a maximum or minimum, one can check the eterminant of the borere Hessian of the Lagrangian function. The borere Hessian is simply the Hessian matrix of the Lagrangian function borere by the rst-orer partial erivatives of the constraint equation an a zero. The borere Hessian for a problem with two choice variables an one constraint is: 2 H = 4 L 3 11 L 12 g 1 L 21 L 22 g 2 5 g 1 g 2 0 where L ij = @2 L @x j an g i = @g. As a practical matter in this course we will not be concerne with etermining the sign of the eterminant of the borere Hessian matrix, but you are encourage to rea the text for more etail. 3.2.3 Inequality constraints In some problems we will have inequality constraints to conten with. Many times we assume that the choice variables must be nonnegative, or x i 0. Look at the following pictures an attempt to characterize the solution to the problem: f (x * )<0 f (x * )=0 f (x * )=0 x * =0 x * =0 x * >0 Case 1 Case 2 Case 3 In case 1 we have x an f 0 (x ) < 0. In case 2 we have x an f 0 (x ). In case 3 we have x > 0 an f 0 (x ). In all three of these cases we have x [f 0 (x )]. But this is not the only conition because in case 3 when ex we have ex [f 0 (ex)] but ex is NOT a maximum. So the necessary conitions we nee to have hol are: f 0 (x ) 0 (18) x [f 0 (x )] (19) x 0 (20) 5
The necessary conitions for a minimum are fairly similar to those of a maximum. We have: f 0 (x ) 0 (21) x [f 0 (x )] (22) x 0 (23) 3.2.4 Kuhn-Tucker conitions A general problem that we might encounter is: max f (x 1 ; x 2 ) s.t. g (x 1 ; x 2 ) 0 (24) x 1;x 2 This is a nonlinear programming problem, as a linear programming problem we have a linear function which is optimize subject to linear equality constraints. We make no such restrictions here. We can construct the Lagrangian to n: L (x 1 ; x 2 ; ) = f (x 1 ; x 2 ) + [g (x 1 ; x 2 )] : (25) The Kuhn-Tucker necessary conitions for this problem are: + @g (26) + @g (27) g (x 1 ; x 2 ) (28) 0; g (x 1 ; x 2 ) 0 (29) Now, there are many times in which we might restrict our choice variables x 1 an x 2 to be greater than or equal to 0. In those instances, our problem woul be: Technically, the Lagrangian is: max f (x 1; x 2 ) s.t. g (x 1 ; x 2 ) 0 (30) x 10;x 20 L (x 1 ; x 2 ; 1 ; 2 ; 3 ) = f (x 1 ; x 2 ) + 1 [g (x 1 ; x 2 )] + 2 x 1 + 3 x 2 (31) The Kuhn-Tucker necessary conitions for this type of problem are: 0; x 1 0; x 1 0; x 2 0; x 2 @ 1 0; 1 0; 1 @ 1 @ 2 0; 2 0; 2 @ 2 @ 3 0; 3 0; 3 @ 3 : Technically one woul have to check all these conitions to see which constraints are bining an which are not. For most consumer an rm problems that we will solve we will be able to "know" that x 1 > 0 an x 2 > 0 by looking at the objective function. 6