Week 4: Calculus and Optimization (Jehle and Reny, Chapter A2) Tsun-Feng Chiang *School of Economics, Henan University, Kaifeng, China September 27, 2015 Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 1 / 31
Alternatively, to see whether a multivariate function f is concave, we can check if the Hessian matrix of the function is negative semidefinite. Definition: Definiteness Let A be an n n symmetric matrix, then A is: positive definite if z T Az > 0 for all z 0 in R n, positive semidefinite if z T Az 0 for all z 0 in R n, negative definite if z T Az < 0 for all z 0 in R n, negative semidefinite if z T Az 0 for all z 0 in R n, and indefinite if z T Az > 0 for some z in R n, and z T Az < 0 for some other z in R n Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 2 / 31
Figure: (left) Negative definite function x 3 = x1 2 x 2 2 and (right) negative semidefinite function x 3 = (x 1 + x 2 ) 2 (Simon and Blume 1994, page 378-379) Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 3 / 31
With this in mind, the analog of a nonpositive second derivative of a function of one variable would be a negative semidefinite matrix of second-order partial derivatives (i.e., the Hessian matrix) of a function of many variables. We can now put together Theorem A2.1 and A2.3 to confirm this. Theorem A2.4 Slope, Curvature, and Concavity in Many Variables Let D be a convex subset of R n with a nonempty interior on which f is twice continuously differentiable. The following statements 1 to 3 are equivalent: 1. f is concave. 2. H(x) is negative semidefinite for all x in D. 3. For all x 0 D : f (x) f (x 0 ) + f (x 0 )(x x 0 ) Moreover, 4. If H(x) is negative definite for all x in D, then f is strictly concave (the converse is not true). Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 4 / 31
In a single-variable case, a necessary and sufficient condition for a function to be concave (convex) is that its second derivative not be rising (falling). In the multivariable case, we can note a necessary, but not a sufficient condition for concavity or convexity in terms of the signs of all "own" second partial derivative. Theorem A2.5 Concavity, Convexity, and Second-Order Own Partial Derivatives Let f : D R be a twice differentiable function. 1. If f is concave, then x, f ii (x) 0, i = 1,, n. 2. If f is convex, then x, f ii (x) 0, i = 1,, n. Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 5 / 31
Homogeneous Functions Homogeneous real-valued function arise quite often in microeconomic applications. In this section, we briefly consider functions of this type and use our calculus tools to establish some of their important properties. Definition A2.2 Homogeneous Functions A real-valued function f (x) is called homogeneous of degree k if f (tx) t k f (x) for all t > 0. Two special cases are worthy of note: f (x) is homogeneous of degree 1, or linear homogeneous, if f (tx) tf (x) for all t > 0; it is homogeneous of degree zero if f (tx) f (x) for all t > 0. Homogeneous function display very regular behavior as all variables are increased simultaneously and in the same proportion. When a Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 6 / 31
function is homogeneous of degree 1, for example, doubling or tripling all variables doubles or triples the value of the function. When homogeneous of degree zero, equiproportionate changes in all variables leave the value of the function unchanged. Example A2.3 The function f (x 1, x 2 ) Ax1 αx β 2, A > 0, α > 0, β > 0, is known as the Cobb-Douglas function. We now multiply all variables by the same factor t, f (tx 1, tx 2 ) A(tx 1 ) α (tx 2 ) β t α t β Ax α 1 x β 2 = t α+β f (x 1, x 2 ). According to the definition, the Cobb-Douglas is homogeneous of degree α + β > 0. If the coefficients are chosen so that α + β = 1, it is linear homogeneous. Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 7 / 31
The partial derivative of homogeneous functions are also homogeneous. The following theorem makes this clear. Theorem A2.6 Partial Derivative of Homogeneous Functions If f (x) is homogeneous of degree k, its partial derivative are homogeneous of degree k 1. Proof: Assume f (x) is homogeneous of degree k, Then f (tx) t k f (x) for all t > 0. Differentiate the left-hand side and right-hand side with respective to x i. The resulting derivatives are equal, f (tx) x i k 1 f (x) = t, x i for i = 1,, n, and t > 0, as we sought to show. Let verify this theorem for the Cobb-Douglas form. Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 8 / 31
Example A2.4 Let f (x 1, x 2 ) Ax1 αx β 2, and suppose α + β = 1 so that it is linear homogeneous. The partial derivative with respect to x 1 is f (x 1, x 2 ) x 1 = αax α 1 1 x β 2 Multiply both x 1 and x 2 by the factor t, and evaluate the partial derivative at (tx 1, tx 2 ). We obtain f (tx 1, tx 2 ) x 1 = αa(tx 1 ) α 1 (tx 2 ) β = t α+β 1 αax α 1 1 x β 2 = f (x 1, x 2 ) x 1, as required, because α + β = 1 and t α+β 1 = t 0 = 1. Finally, Euler s theorem, sometimes called the adding-up theorem, gives us an interesting way to completely characterize homogeneous functions. Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 9 / 31
Theorem A2.7 Euler s Theorem f (x) is homogeneous of degree k if and only if kf (x) = n i=1 f (x) x i x i for all x This theorem says a function is homogeneous if and only if it can always be written in terms of its own partial derivatives and the degree of homogeneity. Example A2.5 Let f (x 1, x 2 ) Ax1 αx β 2, and again suppose α + β = 1. The partial derivatives are f (x 1, x 2 ) x 1 = αax α 1 1 x β 2, f (x 1, x 2 ) = βax1 α x x β 1 2 2 Multiply the first by x 1, the second by x 2, add, and use the fact that Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 10 / 31
Example A2.5(Continued) α + β = 1 to get f (x 1, x 2 ) x 1 + f (x 1, x 2 ) x 2 = αax α 1 x 1 x 1 x β 2 x 1 + βax1 α x β 1 2 x 2 2 = (α + β)ax α 1 x β 2 = 1 f (x 1, x 2 ). just as we were promised by Euler s theorem. Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 11 / 31
A2.2 Unconstrained Optimization This section is devoted to the calculus approach to optimization problems, the most common form of problem in microeconomic theory. First we consider the function of a single variable, y = f (x), and assume it is differentiable. If we can find a point x or x, then x is a (unique) local maximum if f (x ) f (x) (f (x ) > f (x)) for all x x in some neighborhood of x. x is a (unique) global maximum if f (x ) f (x) (f (x ) > f (x)) for all x x in the domain of the function. x is a (unique) local minimum if f ( x) f (x) (f ( x) < f (x)) for all x x in some neighborhood of x. x is a (unique) global minimum if f ( x) f (x) (f ( x) < f (x)) for all x x in the domain of the function. Various types of optima are illustrated in Figure A2.4 (see the next slide). Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 12 / 31
Figure A2.4: Local and global optima In Figure A2.5 (see the next slide), we can see the characters of the maximum and the minimum, which can be presented by the first-order necessary condition (FONC) and the second-order necessary condition (SONC). Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 13 / 31
Figure A2.5: Necessary conditions for local optima Theorem A2.8 Necessary Conditions for Local (Interior) Optima in the Single-Variable Case Let f (x) be a twice continuously differentiable function of one variable. Then f (x) reaches a local 1. maximum at x f (x ) = 0 (FONC) f (x ) 0 (SONC) 2. minimum at x f ( x) = 0 (FONC) f ( x) 0 (SONC) Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 14 / 31
Real-Valued Functions of Several Variables Suppose that D R n, and let f : D R be a twice continuously differentiable real-valued function of n variables. Similar to the single-variable case, if we can find a point x or x, then x is a local maximum whenever there exists some ɛ > 0 such that f (x ) f (x) for all x B ɛ (x ). x is a global maximum if f (x ) f (x) for all x in the domain. x is a local minimum whenever there exists some ɛ > 0 such that f ( x) f (x) for all x B ɛ ( x). x is a global minimum whenever f ( x) f (x) for all x in the domain. These optima are unique if the inequalities hold strictly. As one might expect, the analogy to the derivative being zero at the optimum for functions of one variable will be the gradient vector must be zero at an optimum of a function of many variables. Thus, the single first-order equation f (x ) = 0 characterizing optima for functions of one variable Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 15 / 31
generalizes to the first-order system of n simultaneous equations, f (x ) = 0 characterizing the optima of functions of n variables. This gives us our first-order necessary condition for any (interior) optima of real-valued functions. Theorem A2.9 First-Order Necessary Condition for Local (Interior) Optima of Real-Valued Functions If the differentiable functions f (x) reaches a local (interior) maximum or minimum at x, then x solves the system of simultaneous equations, f (x ) x 1 = 0, f (x ) x 2 = 0, f (x ) x n = 0. Example A2.6 Let y = x 2 4x1 2 + 3x 1x 2 x2 2. To find a critical point of this function, take each of its partial derivatives: Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 16 / 31
Example A2.6 (Continued) f (x 1, x 2 ) x 1 = 8x 1 + 3x 2, f (x 1, x 2 ) x 2 = 1 + 3x 1 2x 2 We will have a critical point at a vector (x1, x 2 ) where both of these equal zero simultaneously. To find x1 and x 2, set each partial equal to zero: f (x 1, x 2 ) x 1 = 8x 1 + 3x 2 = 0, f (x 1, x 2 ) x 2 = 1 + 3x 1 2x 2 = 0 and solve this system for x1 and x 2. We can see the critical point at x1 = 3/7 and x 2 = 8/7. We do not yet know whether we have found a maximum or a minimum, though. For that we have to look at the second-order conditions. Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 17 / 31
Second-Order Conditions Once we have found a point where f (x ) = 0, we know we have a maximum (minimum) if the function is "locally concave" ("locally convex") there. Theorem A2.4 pointed out that curvature depends on the definiteness property of the Hessian of f. Intuitively, it appears that the function will be locally concave around x if H(x) is negative semidefinite, and will be locally convex if it is positive semidefinite. Intuition thus suggests the following second-order necessary condition for local (interior) optima. Theorem A2.10 Second-Order Necessary Condition for Local (Interior) Optima of Real-Valued Functions Let f (x) be twice continuously differentiable. 1. If f (x) reaches a local interior maximum at x, then H(x ) is negative semidefinite. 2. If f (x) reaches a local interior minimum at x, then H( x) is positive semidefinite. Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 18 / 31
Theorem A2.9 and A2.10 are both necessary conditions which help in locating potential maxima (or minima) of specific functions, but to verify that they actually maximize (or minimize) the function, we need sufficient conditions. Sufficient conditions for optima are more stringent than necessary conditions. Simply stated, sufficient conditions for interior optima are as follows: If f i (x ) = 0 for i = 1,, n and H(x ) is negative definite at x, then f (x) reaches a local maximum at x. If f i ( x) = 0 for i = 1,, n and H( x) is positive definite at x, then f (x) reaches a local minimum at x. The sufficient conditions require the point to be a critical point, and require the curvature conditions to hold in their strict forms. For example, when H(x ) is negative definite, the function will be strictly concave in some ball around x. Locating a critical point is easy. We simply set all first-order partial derivatives equal to zero and solve the system of n equations. Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 19 / 31
Determining whether the Hessian is negative or positive definite there will generally less easy. Various tests for determining the definiteness property of the Hessian key on the sign pattern displayed by the determinants of certain submatrices formed from it at the point (or region) in question. These determinants are called the principal minors of the Hessian. By the first through nth principal minors of H(x) at the point x, we mean the determinants D 1 (x) f 11 = f11, D 2 (x) f 11 f 12 f 11 f 1i f 21 f 22 D i(x)..... f i1 f ii f 11 f 1n D n (x)....., f n1 f nn where it is understood that f i1 is evaluated at x. Each is the determinant of a matrix resulting when the last (n i) rows and Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 20 / 31
columns of the Hessian H(x) are deleted, for i = 1,, n. The following theorem gives requirements on its principal minors sufficient to ensure definiteness of the Hessian. Theorem A2.11 Let f(x) be twice continuously differentiable, and let D i (x) be the ith-order principal minor of the Hessian matrix H(x). 1. If ( 1) i D i (x) > 0, i = 1,, n, then H(x) is negative definite. 2. If D i (x) > 0, i = 1,, n, then H(x) is positive definite. If condition 1 holds for all x in the domain, then f is strictly concave. If condition 2 holds for all x in the domain, then f is strictly convex. From Theorem A2.9 and A2.11, we can state the first- and second-order sufficient conditions for local interior optima. Theorem A2.12 Sufficient Conditions for Local Interior Optima of Real-Valued Functions Let f(x) be twice continuously differentiable, Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 21 / 31
Theorem A2.12 (Continued) 1. If f i (x ) = 0 and ( 1) i D i (x ) > 0, i = 1, 2,, n, then f (x) reaches a local maximum at x. 2 If f i ( x) = 0 and D i ( x) > 0, i = 1, 2,, n, then f (x) reaches a local minimum at x. Example A2.7 Check whether the critical point for the function y = x 2 4x1 2 + 3x 1x 2 x2 2 was a maximum or a minimum. We compute the second-order partials, 2 f x 2 1 = 8; 2 f x 1 x 2 = 2 f x 2 x 1 = 3; 2 f x 2 2 = 2 [ ] 8 3 and form the Hessian, H(x) =. In Example A2.6, we found 3 2 a critical point at x = (3/7, 8/7). Checking the principal minors, Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 22 / 31
Example A2.7 (Continued) D 1 (x) 8 = 8 < 0, D 2 (x) 8 3 3 2 = 16 9 = 7 > 0, Because these principal minors alternate in sign, beginning with negative, Theorem A2.12 tells us that x = (3/7, 8/7) is a local maximum. In this example the Hessian matrix was completely independent of x. We would therefore obtain the same alternating sign pattern on the principal minors regardless of where we evaluated them. In Theorem A2.11, we observed that this is sufficient to ensure that the function involved is strictly concave. In other words, it means there is only a hill in the graph and it is the only highest point. Indeed, from Figure A2.5, it is intuitively clear that any local maximum (minimum) of a concave (convex) function must also be a global maximum (minimum). Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 23 / 31
Theorem A2.13 (Unconstrained) Local-Global Theorem Let f be a twice continuously differentiable real-valued concave function on D. The following statements are equivalent, where x is an interior point of D: 1. f (x ) = 0. 2. f achieves a local maximum at x. 3. f achieves a global maximum at x. Proof: Clearly, 3 2, and by Theorem A2.9, 2 1. Hence, it remains only to show that 1 3. Because f is concave, Theorem A2.4 implies that for all x in the domain, f (x) f (x ) + f (x )(x x ). Then given 1, the inequality above implies that f (x) f (x ) Therefore, f reaches a global maximum at x. Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 24 / 31
Theorem A2.13 says that under convexity or concavity, any local optimum is a global optimum. Notice, however, that it is still possible that the lowest (highest) value is reached at more than one point in the domain. If we want the highest or lowest value of the function to be achieved at a unique point, we have to impose strict concavity or strict convexity. Theorem A2.14 Strict Concavity/Convexity and the Uniqueness of Global Optima 1. If x maximizes the strictly concave function f, then x is the unique global maximizer, i.e., f (x ) > f (x) x D, x x. 2. If x minimizes the strictly convex function f, then x is the unique global minimizer, i.e., f ( x) < f (x) x D, x x. Proof: We will prove 1. If x is a global maximizer of f but it is not unique, then there exists some other points x x such that f (x ) = f (x ). If we let x t = tx + (1 t)x, then strict concavity requires that Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 25 / 31
Theorem A2.14 (Continued) f (x t ) > tf (x ) + (1 t)f (x ) t (0, 1) Because f (x ) = f (x ), the requires that or, simply f (x t ) > tf (x ) + (1 t)f (x ) f (x t ) > f (x ) This, however, contradicts the assumption that x is a global maximizer of f. Thus, any global maximizer of a strictly concave function must be unique. From Theorem A2.9 and A2.14, we have the sufficient conditions for unique global optima. Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 26 / 31
Theorem A2.15 Sufficient Conditions for Unique Global Optima Let f (x) be twice continuously differentiable, 1. If f (x) is strictly concave and f i (x ) = 0, i = 1,, n, then x is the unique global maximizer of f (x). 2. If f (x) is strictly convex and f i ( x) = 0, i = 1,, n, then x is the unique global minimizer of f (x). Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 27 / 31
Constrained Optimization Scarcity is a pervasive fact of economic life. It is most commonly expressed as constraints on permissible value of economic variables. Agents, often consumers and firms, are then represented as seeking to do the best they can within the constraints they face. This is the type of problem we will regularly encounter. We need to modify our techniques of optimization and the terms that characterize the optima in such cases, accordingly. There are three basic types of constraints we will encounter. They are equality constraints, nonnegativity constraints, and, amore generally, any form of inequality constraint. We will derive methods for solving problems that involve each of them in turn. We will confine discussion to problems of maximization, simply noting the modifications (if any) for minimization problems. Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 28 / 31
Equality Constraints Consider choosing x 1 and x 2 to maximize f (x 1, x 2 ), when x 1 and x 2 must satisfy some particular relation to each other that we write in implicit form as g(x 1, x 2 ) = 0. Formally max x1,x 2 f (x 1, x 2 ) subject to g(x 1, x 2 ) = 0 Here f (x 1, x 2 ) is called the objective function. The x 1 and x 2 are called choice variables. The function g(x 1, x 2 ) is called the constraint. The set of all x 1 and x 2 that satisfy the constraint are sometimes called the constraint set or the feasible set. One way to solve this problem is by substitution. For example, suppose that g(x 1, x 2 ) = 0 can be written to isolate x 2 on one side as x 2 = g(x 1 ) Substitute the equation above into the objective function to replace x 2. This way, the two-variable constrained maximization problem can be rephrased as the single-variable problem with no constraints: Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 29 / 31
max x1 f (x 1, g(x 1 )) The usual first-order conditions require that we set the total derivative, f / x 1, equal to zero and solve for the optimal x1. That is, f (x1, g(x 1 )) + f (x 1, g(x 1 )) d g(x 1 )) = 0 (by chain rule) x 1 x 2 dx 1 When we have found x1, we plug it back to the constraint and find x2 = g(x 1 ). The pair (x 1, x 2 ) then solves the constrained problem, provided the appropriate second-order condition is also fulfilled. However, there more economic problems which involve more than two choice variables and more than one constrains. The substitution method is not well suited to these more complicated problems. The better way is using Lagrange s method. Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 30 / 31
First Midterm Exam Date: Scheduled on Monday, October 26th, 2015 Time: 9:00 am 11:30 am Location: To be Announced Coverage: Week 1 - Week 5 and a small part of Week 6 Rules: The exam is closed-book, closed-notes and in-class. Taking the in-class exam is only way for earning grades in this course fro any student. No scratch paper (will be provided by the proctor), calculator, cellphone and other electronic devices are allowed. Pens, correction tools and basic requirements for life (ex. water, tissue paper, and medicine) are allowed. Microeconomic Theory Week 4: Calculus and Optimization (Jehle and Reny, Chapter September A2) 27, 2015 31 / 31