II. An Application of Derivatives: Optimization

Similar documents
September Math Course: First Order Derivative

Review of Optimization Methods

Lecture 4: Optimization. Maximizing a function of a single variable

Economics 101A (Lecture 3) Stefano DellaVigna

EC /11. Math for Microeconomics September Course, Part II Lecture Notes. Course Outline

STUDY MATERIALS. (The content of the study material is the same as that of Chapter I of Mathematics for Economic Analysis II of 2011 Admn.

2015 Math Camp Calculus Exam Solution

School of Business. Blank Page

Suppose that f is continuous on [a, b] and differentiable on (a, b). Then

MAT 122 Homework 7 Solutions

where u is the decision-maker s payoff function over her actions and S is the set of her feasible actions.

Math 180, Exam 2, Practice Fall 2009 Problem 1 Solution. f(x) = arcsin(2x + 1) = sin 1 (3x + 1), lnx

EC /11. Math for Microeconomics September Course, Part II Problem Set 1 with Solutions. a11 a 12. x 2

Rules of Differentiation

Week 4: Calculus and Optimization (Jehle and Reny, Chapter A2)

MAXIMA AND MINIMA CHAPTER 7.1 INTRODUCTION 7.2 CONCEPT OF LOCAL MAXIMA AND LOCAL MINIMA

z = f (x; y) f (x ; y ) f (x; y) f (x; y )

Exam A. Exam 3. (e) Two critical points; one is a local maximum, the other a local minimum.

Math 265H: Calculus III Practice Midterm II: Fall 2014

Math 10C - Fall Final Exam

MATH Midterm 1 Sample 4

Basics of Calculus and Algebra

Lecture Notes for Chapter 12

Math Maximum and Minimum Values, I

x +3y 2t = 1 2x +y +z +t = 2 3x y +z t = 7 2x +6y +z +t = a

MA102: Multivariable Calculus

14 Increasing and decreasing functions

Taylor Series and stationary points

This exam will be over material covered in class from Monday 14 February through Tuesday 8 March, corresponding to sections in the text.

Mathematical Economics. Lecture Notes (in extracts)

Exam 3 MATH Calculus I

Constrained optimization.

CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS. W. Erwin Diewert January 31, 2008.

Mathematical Economics: Lecture 16

MATH The Chain Rule Fall 2016 A vector function of a vector variable is a function F: R n R m. In practice, if x 1, x n is the input,

Universidad Carlos III de Madrid

g(x,y) = c. For instance (see Figure 1 on the right), consider the optimization problem maximize subject to

Mathematic 108, Fall 2015: Solutions to assignment #7

ARE202A, Fall 2005 CONTENTS. 1. Graphical Overview of Optimization Theory (cont) Separating Hyperplanes 1

Partial Derivatives. w = f(x, y, z).

a x a y = a x+y a x a = y ax y (a x ) r = a rx and log a (xy) = log a (x) + log a (y) log a ( x y ) = log a(x) log a (y) log a (x r ) = r log a (x).

REVIEW OF DIFFERENTIAL CALCULUS

Matrix Theory and Differential Equations Homework 2 Solutions, due 9/7/6

CHAPTER 4: HIGHER ORDER DERIVATIVES. Likewise, we may define the higher order derivatives. f(x, y, z) = xy 2 + e zx. y = 2xy.

MATH Max-min Theory Fall 2016

ExtremeValuesandShapeofCurves

UNIVERSITY OF CAMBRIDGE

ENGI Partial Differentiation Page y f x

ECON2285: Mathematical Economics

0, otherwise. Find each of the following limits, or explain that the limit does not exist.

BEEM103 UNIVERSITY OF EXETER. BUSINESS School. January 2009 Mock Exam, Part A. OPTIMIZATION TECHNIQUES FOR ECONOMISTS solutions

Final Exam Study Guide

Solving and Graphing a Linear Inequality of a Single Variable

Mathematical Foundations II

Math 180, Final Exam, Fall 2007 Problem 1 Solution

DIFFERENTIAL EQUATIONS

Mathematics for Economics ECON MA/MSSc in Economics-2017/2018. Dr. W. M. Semasinghe Senior Lecturer Department of Economics

Sec. 14.3: Partial Derivatives. All of the following are ways of representing the derivative. y dx

Sign of derivative test: when does a function decreases or increases:

Maxima and Minima. (a, b) of R if

Lecture 20: Further graphing

Modern Optimization Theory: Concave Programming

Preliminary draft only: please check for final version

Institute of Computer Science

Name: Instructor: Lecture time: TA: Section time:

Math 2204 Multivariable Calculus Chapter 14: Partial Derivatives Sec. 14.7: Maximum and Minimum Values

Calculus and optimization

Sometimes the domains X and Z will be the same, so this might be written:

DEPARTMENT OF MATHEMATICS AND STATISTICS UNIVERSITY OF MASSACHUSETTS. MATH 233 SOME SOLUTIONS TO EXAM 2 Fall 2018

Calculus 2502A - Advanced Calculus I Fall : Local minima and maxima

Solutions to old Exam 3 problems

ARE211, Fall 2005 CONTENTS. 5. Characteristics of Functions Surjective, Injective and Bijective functions. 5.2.

2.3 Maxima, minima and second derivatives

2. Find the intervals where function is increasing and decreasing. Then find all relative extrema.

IMP 2007 Introductory math course. 5. Optimization. Antonio Farfán Vallespín

1 Calculus - Optimization - Applications

Lecture 9: Implicit function theorem, constrained extrema and Lagrange multipliers

Optimality Conditions

EconS 301. Math Review. Math Concepts

1. Introduction. 2. Outlines

AY Term 1 Examination November 2013 ECON205 INTERMEDIATE MATHEMATICS FOR ECONOMICS

Seminars on Mathematics for Economics and Finance Topic 5: Optimization Kuhn-Tucker conditions for problems with inequality constraints 1

Functions. A function is a rule that gives exactly one output number to each input number.

ECON 186 Class Notes: Optimization Part 2

1 Implicit Differentiation

The First Derivative Test for Rise and Fall Suppose that a function f has a derivative at every poin x of an interval A. Then

22: Applications of Differential Calculus

Functions of Several Variables

Final Exam 2011 Winter Term 2 Solutions

VANDERBILT UNIVERSITY. MATH 2300 MULTIVARIABLE CALCULUS Practice Test 1 Solutions

Math 1314 Lesson 24 Maxima and Minima of Functions of Several Variables

ARE211, Fall2015. Contents. 2. Univariate and Multivariate Differentiation (cont) Taylor s Theorem (cont) 2

39.1 Absolute maxima/minima

1. Sets A set is any collection of elements. Examples: - the set of even numbers between zero and the set of colors on the national flag.

Higher order derivative

3.5 Quadratic Approximation and Convexity/Concavity

Lecture 9 - Increasing and Decreasing Functions, Extrema, and the First Derivative Test

Optimization. Sherif Khalifa. Sherif Khalifa () Optimization 1 / 50

Math 118, Fall 2014 Final Exam

Limit. Chapter Introduction

Transcription:

Anne Sibert Autumn 2013 II. An Application of Derivatives: Optimization In this section we consider an important application of derivatives: finding the minimum and maximum points. This has important applications in economics, medicine, finance, physics and many other disciplines. A particularly simple example is one from economics. A firm s profit is equal to its revenue minus its costs, both of which are functions of its output. Thus, its profit is a function of its output and it wants to choose the output that will maximize its profit. The first subsection of this section considers the optimization of functions of a single variable. The second subsection extends the idea of differentiation to functions of two variables. It considers the optimization and constrained optimization of functions of two variables. Finally it briefly considers differentiation and optimization of functions of more than two variables. Upon completing this section, students should be able to Use the first- order conditions to find the critical values of functions of one variable or several variables. Explain the notions of concave and convex functions of a single variable and how concavity or convexity can be used to determine if a critical point is a maximizing or minimizing point. Explain the concept of local maxima and minima. Find the maximum and minimum of a continuous function of a single variable on a closed interval. Find partial derivatives, total derivatives and differentials. Explain the Envelope Theorem. Use the method of Lagrange multipliers to solve constrained optimization problems. Find Hessian and bordered Hessian matrices. Use the second derivative test for functions of one or several variables to determine if a critical point is a maximizing or minimizing point. 1

II.A. Optimization of Functions of a Single Variable This section considers locating the maximum or minimum value of a continuous function f ( x) on some interval of the real line. The function to be maximized is referred to as the objective function. II.A.1 Maximum and minimum points and first- order conditions Definition II.1. Let f be a function on some interval S R. Then x * is said to maximize (minimize) f if f x * ( ) f ( x), x S. [The upside- down A is a ( ) mathematical symbol that means for every.] Example II.1. If f x minimizes f on S. ( ) = x 2 and S = [ 0,1] then x = 1 maximizes f on S and x = 0 There can be more than one maximizing or minimizing point. If f ( x) = 4 on the real line, then every point on the real line is both a maximizing and a minimizing point. A maximizing or minimizing point also need not exist. The function f ( x) = x 2 has no maximizing point on the real line because it goes to infinity. The function f ( x) = x has no maximizing or minimizing point on any open interval. Suppose that we want to find the maximum of the function f ( x) = 2 + 4x x 2 on the real line. This function is drawn in Figure 1 below. 5 2.5 1-0.5 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Figure 1. The Derivative is Zero at a Maximum Let x * be the point where the function is maximized. Clearly it cannot be that f ' x * ( ) > 0. If this were the case f could be increased by increasing x above x *. 2

Likewise it cannot be that f ' x * decreasing x below x *. Thus, it must be that the f ' x * ( ) < 0. If this were the case f could be increased by ( ) = 0. This is seen in Figure 1. The curve is flat and its slope is zero at the point where it is maximized. This gives us a way to find the point where the function is maximized. Find the derivative and set it equal to zero. We have f '( x) = 4 2x = 0 x * = 2. This is consistent with what is seen in Figure 1. Suppose we want to find the minimum of the function f ( x) = 1 4x + x 2 on the real line. The function is graphed in Figure 2 below. This function clearly has no maximum as it goes off to infinity. It is seen that if we follow the procedure of setting the derivative equal to zero and solving we will end up with a minimum instead of a maximum. Indeed, the intuition from the last paragraph holds for finding minima as well as maxima. Let x * be the point where the function is minimized. Clearly it cannot be that f ' x * ( ) > 0. If this were the case f could be decreased by decreasing ( ) < 0. If this were the case f could be ( ) = 0. This is x below x *. Likewise it cannot be that f ' x * decreased by increasing x above x *. Thus, it must be that the f ' x * seen in Figure 2. The curve is flat and its slope is equal to zero at x *. Taking the derivative yields f ' x ( ) = 4 + 2x = 0 x * = 2 5 2.5-0.8-0.4 0 0.4 0.8 1.2 1.6 2 2.4 2.8 3.2 3.6 4-2.5-5 Figure 2. The Derivative is Zero at a Minimum We have the following result. Proposition II.1. Suppose that f is a differentiable function on an open interval S R. If x * maximizes or minimizes f on S then f ' x * ( ) = 0. 3

The condition that the first derivative is equal to zero is called the first- order condition. Points that satisfy the first- order condition are called critical points. Not all critical points, however, are maximizing or minimizing points. The function f ( x) = x 3, graphed in Figure 3 below, has a derivative equal to 3x 2. This derivative is equal to zero at x = 0, but the function is neither maximized nor minimized at this point. We call this type of critical point an inflection point. 40 30 20 10 4-3.2-2.4-1.6-0.8 0 0.8 1.6 2.4 3.2 4-10 -20 Figure 3. An Inflection Point Consider the more complicated function f ( x) = x 4 10x 2 + 9, graphed in Figure 4, below. 16 8 5-4 -3-2 -1 0 1 2 3 4 5-8 -16 Figure 4. Two Minima and a Local Maximum Taking the first derivative and setting it equal to zero yields f '( x) = 4x 3 20x = 0. Solving yields three critical points: x = 5,0, 5. Clearly x = 5 and x = 5 minimize f. The function does not have a maximum as it goes off to infinity. 4

However, its appearance suggests a more restrictive notion of a maximum or minimum. Definition II.2. If N is an open interval in S containing x, then N is said to be a neighborhood of x in S. Definition II.3. Let f be a function on some interval S R. Then f is said to have a local maximum (minimum) at x * if there exists a neighborhood N S of x such that f x * ( ) f ( x), x N. ( ) The function in Figure 4 has a local maximum at zero. II.A.2 Concave and convex Functions and the second- order conditions A strictly concave function of one variable has the following geometric property. Take any two points where the function is defined. Call them x 1 and x 2. Then draw a ( ( )) and P 2 = x 2, f ( x 2 ) ( ). If this line lies below the line between the points P 1 = x 1, f x 1 graph of the function, then the function is said to be strictly concave. This is shown in Figure 5, below. The red connecting line lies below the black function curve and, thus, the function is strictly concave. The function in Figure 1 is also strictly concave. y P1 P2 Figure 5. A Strictly Concave Function x x A (weakly) concave function has the geometric property that the line between any two points does not lie above the function. The function in Figure 6 below, which contains a linear segment, is concave, but not strictly concave. 5

y x Figure 6. A Function that is Concave but not Strictly Concave A strictly convex function has the following geometric property. Take any two points where the function is defined. Call them x 1 and x 2. Then draw a line between the ( ( )) and P 2 = x 2, f ( x 2 ) ( ). If this line lies above the graph of the points P 1 = x 1, f x 1 function, then the function is said to be strictly convex. This is shown in Figure 7, below. The red connecting line lies above the black function curve, thus, this function is strictly convex. The function in Figure 2 is also strictly convex. y P1 P2 x Figure 7. A Strictly Convex Function A (weakly) convex function has the property that the line between any two points does not lie below the curve. Similarly to Figure 6, convex functions can have parts that are straight lines. Note that the function f ( x) = 3x + 2, for example, is neither concave nor convex but it is both weakly concave and weakly convex. The geometric definitions can be written formally as follows. Definition II.4. A function f is said to be strictly concave on S if for every distinct pair x 0 S and x 1 S f α x 0 + ( 1 α )x 1 ( ) + ( 1 α ) f ( x 1 ) for every α ( 0,1). ( ) > α f x 0 6

Definition II.5. A function f is said to be strictly convex on S if for every distinct pair x 0 S and x 1 S f α x 0 + ( 1 α )x 1 ( ) + ( 1 α ) f ( x 1 ) for every α ( 0,1). ( ) < α f x 0 Definition II.6. A function is said to be (weakly) concave on S if for every pair x 0 S and x 1 S f α x 0 + ( 1 α )x 1 ( ) + ( 1 α ) f ( x 1 ) for every α ( 0,1). ( ) α f x 0 Definition II.7. A function f is said to be (weakly) convex on S if for every pair x 0 S and x 1 S f α x 0 + ( 1 α )x 1 ( ) + ( 1 α ) f ( x 1 ) for every α ( 0,1). ( ) α f x 0 We have the following results: Proposition II.2. Suppose x * S such that f '( x * ) = 0. Then f attains a unique maximum (minimum) on S at x * if f is strictly concave (strictly convex) on S. [The backwards E is mathematical notation for there exists.] Proposition II.3. Suppose that x * S such that f '( x * ) = 0. Then x * maximizes (minimizes) f on S if f is concave (convex) on S. Proposition II.4. The function f is concave (convex) on S if and only if f " ( ) 0 x in the interior of S. Proposition II.5. If f " < (>) 0 x in the interior of S then f is strictly concave (strictly convex) on S. The condition is Proposition II.5 is called the second- order condition. A solution to the first- order condition is a critical point. If the critical point satisfies, say, f " < 0 then it satisfies the second- order condition for a unique maximum. The function is strictly concave and hence it has a unique maximum at the critical point. Unfortunately, while satisfying the second- order condition is sufficient for the critical point to be a maximum, it is not necessary. It is possible for f to be strictly concave or strictly convex and for its second derivative to be equal to zero. The function f x ( ) = x 4 is strictly concave but f "( 0) = 0. Example II.2. f ( x) = x 2 + 4x 1. The first- order condition is f '( x) = 2x + 4 = 0. Solving yields x * = 2. From the graph of the equation, shown in Figure 8 below, it is 7

clear that the function is strictly concave and attains a unique maximum at the critical point. This is verified by checking the second- order: f '' x ( ) = 2 < 0. 10 7.5 5 2.5-0.4 0 0.4 0.8 1.2 1.6 2 2.4 2.8 3.2 3.6 4-2.5-5 Figure 8. A Strictly Concave Function with a Unique Maximum Example II.3: f ( x) = x 2 4x 1. The first- order condition is f '( x) = 2x 4 = 0. Solving yields x * = 2. From the graph of the equation in Figure 9, below, it is clear that the function is strictly convex and attains a unique minimum at the critical point. This is verified by checking the second- order: f '' x ( ) = 2 > 0. 5 2.5-0.4 0 0.4 0.8 1.2 1.6 2 2.4 2.8 3.2 3.6 4-2.5-5 Figure 9. A Strictly Convex Function with a Unique Minimum It is a sufficient condition for a critical point to be a maximum that the function is concave. But, it is not a necessary condition. The function f x ( ) = e x2 has a first derivative f '( x) = 2xe x2 that is equal to zero at zero. This bell- shaped function, shown in Figure 10, below, is not concave but it is maximized at zero. 8

1 0.75 0.5 0.25 5-4 -3-2 -1 0 1 2 3 4 5 Figure 10. This Function is Maximized at Zero Proposition II.6. Suppose that f ' x * maximum (minimum) at x * if f " x * ( ) = 0 for some x * S. Then f attains a local ( ) < (>) 0. If f "( x * ) = 0 then this second- order condition for a local maximum or minimum is inconclusive. The method for resolving matters in the inclusive case is to find the third derivative. If this is not equal to zero then the critical point is an inflection point. If it is equal to zero, find the fourth derivative. If the fourth derivative is strictly negative (positive) the inflection point is a local maximum (minimum). If it is equal to zero the test is inconclusive. To resolve matters in this case, find the fifth derivative. If this is not equal to zero the critical point is an inflection point. If it is equal to zero find the sixth derivative. If the sixth derivative is strictly negative (positive) the critical point is a local maximum (minimum). If it is equal to zero the test is inconclusive. The procedure can be repeated as necessary. 1.6 0.8-1 -0.75-0.5-0.25 0 0.25 0.5 0.75 1-0.8-1.6 Figure 11. Some Polynomial Functions 9

( ) = x 4 (drawn in red), f ( x) = x 5 (drawn in orange), ( ) = x 6 (drawn in yellow) and f ( x) = x 7 (drawn in green). All of these functions Figure 11 above, depicts f x f x have their first derivative equal to zero at zero. They all also have their second derivatives equal to zero at zero so the second- derivative test is inconclusive. Using the procedure just described it is easy to show that f x local mimina at zero, while f x ( ) = x 4 and f ( x) = x 6 have ( ) = x 5 and f ( x) = x 7 have inflection points at zero. The concept of a local maximum or minimum might seem restrictive. However, if a function only has one or a few local maxima then one only needs to compare these local maxima and pick out the one that is best. Then, compare this with what happens as x goes to the endpoints of the interval under consideration. For the function in Figure 4 (defined on the real line), we know that the unique local maximum is not a global maximum because the function goes to infinity as x goes to plus or minus infinity. This function has no global maximum. For the same reason, the two local minima or also the two (non- unique) global minima. If the interval under consideration is a closed interval we have the following result. Proposition II.7. If f is continuous and S R is a closed interval then f has both a maximizing and a minimizing value in S. Suppose that f is differentiable. To find, say, the maximum point on a closed interval, find the critical points and check which ones are local maxima. Compute the value of the function at these local maxima and at the end points to determine the global maximum. If f is not differentiable (think of the absolute value function of the last lecture) then we need to compare the local maxima, the end points and the points where f is not differentiable. II.B. Functions of Two (More) Variables In this section we consider functions of two variables. While the formal theory of functions of two variables is much more difficult than that of functions of one variable the practical application is usually fairly straightforward. The last subsection briefly considers optimization of functions of more than two variables. II.B.1 Derivatives of functions of two variables II.B.1.a. partial derivatives Let z = f ( x, y) be a function defined on a set S R 2. If we think of y as a constant then the derivative of z with respect to x is called the partial derivative of z with respect to x. It is usually denoted by z x = f / x or by f x ( x, y) or f 1 ( x, y). The partial derivative with respect to y is defined similarly. 10

Example II.4. If f ( x, y) = x 3 y + e x+y2, f x ( x, y) = 3x 2 y + e x+y2 and f y ( x, y) = x 3 + 2ye x+y2. If we think of y as a constant then the second derivative of z with respect to x is called the second partial derivative of z with respect to x. It is usually denoted by 2 z x 2 = 2 f / x 2 or by f xx x, y ( ) or f 11 ( x, y). The second partial derivative with respect to y is defined similarly. If we think of y as a constant, we can also find the derivative with respect to x of the partial derivative with respect to y. This cross second partial derivative is usually denoted by 2 z x y = 2 f x y or by f yx ( x, y) or f 21 ( x, y). Likewise if we think of x as a constant, we can also find the derivative with respect to y of the partial derivative with respect to x. This cross second 2 z partial derivative is usually denoted by y x = 2 f y x or by f xy ( x, y) or f 21 ( x, y). For the previous example the cross partial derivatives are, f xy ( x, y) = f yx ( x, y) = 3x 2 + 2ye x+y2. This result is true for all of the functions that you are ever likely to encounter; the two cross partial derivatives are equal to each other. This result is known as Schwarz Theorem. II.B.1.b implicit functions So far the functions we have encountered have been explicitly defined, but sometimes they are implicitly defined as the solution to an equation. For example, a function z = z x, y ( ) might be given as the solution to an equation such as We can then find the partial derivatives x 2 + y 2 + z 2 = 0. 2x + 2z z x = 0 z x = x z 2y + 2z z x = 0 z x = y z. II.B.2. Optimization of functions of two variables II.B.2.a first- order conditions Suppose that f x, y ( ) is defined on and has partial derivatives on S R 2. Analogously to the result in Proposition II.1, the first- order conditions amount to setting the partial derivatives equal to zero. 11

Proposition II.8 If f has a local maximum or minimum at a point x *, y * interior of S, then f x x *, y * ( ) = f y ( x *, y * ) = 0. ( ) in the The function f ( x, y) = xy x 2 y 2, graphed in Figure 12 below has a maximum at (0,0). This function is strictly concave. The one- variable strictly concave function in Figure 1 looked like the cross- section of an upside- down bowl. The two- variable strictly concave function below looks like an upside- down bowl. Figure 12. This Function has a Maximum The function f ( x, y) = x 2 + y 2 xy, graphed in Figure 13 below has a minimum at (0,0). This function is strictly convex. The one- variable strictly convex function in Figure 2 looked like the cross- section of a right side up bowl. The two- variable strictly convex function below looks like a right side up bowl. Figure 13 This Function has a Minimum at (0,0) 12

The function f ( x, y) = x 2 y 2, shown in Figure 14 below, has a saddle point at (0,0) (indicated in red). This is a critical point where the partial derivatives are equal to zero but there is neither a local maximum nor a local minimum. Figure 14. This Function has a Saddle Point at (0,0) II.B.2.b second- order conditions This subsection extends the notions of concave and convex functions to two- variable functions and provides the second- order conditions for maximizing or minimizing functions of two variables. Proposition II.9 Let f be a twice continuously differentiable function on an open ( ) such that f x ( x *, y * ) = f y ( x *, y * ) = 0, then ( ) is a local maximum if f xx ( x *, y * ) f yy ( x *, y * ) f xy ( x *, y * ) 2 > 0 and ( ) < 0. ( ) is a local minimum if f xx ( x *, y * ) f yy ( x *, y * ) f xy ( x *, y * ) 2 > 0 and ( ) > 0. ( ) is neither a local maximum nor a local minimum if ( ) f yy ( x *, y * ) f xy ( x *, y * ) 2 < 0. set S R 2. If there is a point x *, y * f x *, y * f xx x *, y * f x *, y * f xx x *, y * f x *, y * f xx x *, y * Example II.5 f ( x, y) = x 3 12xy + 8y 3. We have f x = 3x 2 12y = 0 and f y = 12x + 24y 2 = 0 ( x, y) = (0,0) or ( 2,1). Using the second derivative test 13

f xx = 6x, f yy = 48y and f xy = 12. Thus, f xx f yy f 2 xy = 288xy 144. At ( 0,0), f xx f yy f 2 xy < 0 no maximum or minimum. At (2,1) f xx f yy f xy 2 > 0 and f xx > 0 local minimum. Note that if f xx > ( < ) 0 then f yy > ( < ) 0. Note also that f xx f yy f xy 2 = f xx f xy f xy f yy H. The matrix of partial derivatives of a function, such as the two- dimensional matrix H above, is called the Hessian matrix of the function. II.B.3. The differential, the total derivative and the envelope theorem II.B.3.a the differential Suppose that we have a function z = f ( x, y). It can be useful to have a formula for the change in z. This is given by the differential II.B.3.b. the total derivative dz = δ z δ x dx + δ z δ y dy. ( ). A small change in t not only has Suppose that we have a function z = f x( t), y( t),t a direct effect on z, given by its partial derivative, but also indirect effects through changes in x and y. The total change in z resulting from an infinitely small change in t is given by the total derivative of f with respect to t. This is given by Suppose that z = f x, y,α dz dt = δ z dx δ x dt + δ z dy δ y dt + δ z δt. ( ) is a function of x, y and the parameter α. Suppose that ( α ), y * ( α ) f attains a maximum at some point x * We can write the first- order conditions as ( ) that depends on the parameter. ( ) = 0 ( ( ), y * ( α )α ) = 0. f x x * ( α ), y * ( α ),α f y x * α We can totally differentiate both sides with respect to the parameter α to find 14

( ) dx f xx x * ( α ), y * ( α ),α ( ) dx ( ) dy dα + f xy x * ( α ), y * ( α ),α ( ) dy ( ) = 0 dα + f xα ( x* α ), y * ( α ),α ( ) = 0. f xy x * ( α ), y * ( α )α dα + f yy x * ( α ), y * ( α )α dα + f yα x * ( α ), y * ( α )α This pair of equations can be solved to find dx dα = f yα f xy f xα f yy, dy H dα = f xα f xy f yα f xx, H where H is the Hessian matrix and the partial derivatives are evaluated at the maximum. If we want to find the signs of these expressions we only need to worry about the sign of the numerator, as we know that the determinant of the Hessian matrix is negative for a maximum. II.B.3.c envelope theorem We can also find the total derivative of the objective function z with respect to α, evaluated at the maximum. Using the first- order conditions, this is equal to dz dα = δ z δ x dx dα + δ z δ y dy dα + δ z δα = δ z δα. This result that, at an optimal point, the total derivative of the objective function with respect to a parameter is equal to the partial derivative of the objective function with respect to that parameter is called the envelope theorem. II.B.4. Constrained optimization: the method of Lagrange multipliers This section presents the method of Lagrange multipliers for solving constrained optimization problems. It begins with an explanation of where this method comes from and why it works. You need not be able to reproduce this explanation: it is important, however, that you be able to apply this methodology in a cookbook fashion. II.B.4.a. where Lagrange multipliers come from (optional) ( ) subject to the constraint that ( ) = 0. Suppose that a (local) maximum exists and that at that maximum one of Suppose that we want to maximize a function f x, y g x, y the derivatives, say the one with respect to y, does not vanish (that is, it does not equal zero). At the maximum the change in g is given by the differential δ g dg = δ g δ x dx + δ g dy dy = 0 δ y dx = δ x. δ g δ y If we solve the constraint to find y as a function of x we have that 15

y = h( x), where h' x ( ) = δ g δ x. δ g δ y The optimization problem can be reformulated as maximizing f x,h x respect to x. The first- order condition for this single variable problem δ g δ f δ x + δ f δ y h' ( x) = δ f δ x δ x δ f δ g δ y = 0. δ y Note that it is also true that Let Then we have δ g δ f δ y δ y δ f δ g δ y = 0. δ y λ δ f δ y. δ g δ y δ f δ x λ δ g δ x = 0 δ f δ y λ δ g δ y = 0 ( ( )) with These are the first- order conditions that we would get if we solved the unconstrained optimization problem of maximizing (1) L( x, y,λ) = f ( x, y) λg( x, y). (2) The variable λ is known as the Lagrange multiplier and the function L is known as the Lagrangian. II.B.4.b. recipe for optimization with Lagrange multipliers If you want to maximize f ( x, y) subject to g( x, y) = 0. (3) 16

The cookbook approach is to (i) form the Lagrangian function in equation (2); (ii) find the first- order conditions (1); solve the two first- order conditions to eliminate the Lagrange multiplier λ ; (iii) solve the result and equation (3) for x and y. Note that it is conventional to have a minus sign in the right- hand side in equation (2), but that it is not important in that if you use a plus sign instead and follow the procedure you will come up with the same x and y. Example II.6. Maximize ln x + 2ln y subject to 3x + y 12 = 0. The first step is to form the Lagrangian L = ln x + 2ln y - λ ( 3x + y 12). The second step is to find the first derivatives and set them equal to zero. We have 1/ x 3λ = 0 and 2 / y λ = 0. The third step is to solve these two equations to find y = 6x. The next step is to solve this equation and the constraint to find x = 4 / 3 and y = 8. To check the second order conditions, form the bordered Hessian matrix H L xx L xy g x L xy L yy g y g x g y 0 Proposition II.10. If there is a x *, y *,λ * ( ) = 0 and H > < L λ x *, y *,λ * ( ) such that L x ( x *, y *,λ * ) = L y ( x *, y *,λ * ) = ( ) then ( x *, y *,λ * ) is a local maximum ( ) 0 at x *, y *,λ * (minimum). Checking the second- order condition for Example II.6 yields H = 1 x 2 0 3 0 1 y 2 1 3 1 0 = 1 x 2 1 y 2 1 1 0 3 0 3 1 y 2 1 = 1 x 2 + 9 y 2 > 0. II.B.5. Optimization of functions of many variables Suppose that f ( x 1, x 2,..., x n ) is a function of n > 2 variables. Everything generalizes from the case of two variables in a straightforward manner. The only difficulty is that the second- order conditions become more ornate. Let 17

H 2 f 11 f 12 f 21 f 22, H 3 f 11 f 12 f 13 f 21 f 22 f 23 f 31 f 32 f 33,..., H n f 11 f 12... f 1n f 21 f 22... f 2n......... f n1 f n2... f nn be the principle minors of the Hessian matrix. Proposition II.11. Let f x 1, x 2,..., x n ( ) be a twice continuously differentiable function ( ) S such that f 1 ( P * ) =... = f n ( P * ) = 0, then on an open set S R n. If P * = x * * 1,..., x n f ( P * )is a local maximum on S if the determinants of the principle minors of > 0,..., ( 1) n H n > 0. its Hessian matrix alternate in sign at P * : f 11 < 0, H 2 f ( P * )is a local minimum on S if the determinants of the principle minors of its Hessian matrix are positive at P * : f 11 > 0, H 2 > 0,..., H n > 0. For the constrained optimization problem with n > 2 variables, let L L 11 L 12 g 1 11 L 12 L 13 g 1 H 2 L 21 L 22 g 2, H 3 L 21 L 22 L 23 g 2 g 1 g 2 0 L 31 L 32 L 33 g,..., 3 g 1 g 2 g 3 0 H n L 11 L 12... L 1n g 1 L 21 L 22... L 2n g 2............ L n1 L n2... L nn g n g 1 g 2... g n 0. Proposition II.12. If there is a P * x 1 *,..., x n *,λ * L n+1 P * ( ) = 0 then ( ) such that L 1 ( P * ) =... = L n ( P * ) = f ( P * )is a local maximum if at P * if H 2 > 0, H 3 < 0,..., 1 f ( P * )is a local minimum if at P * if H 2 < 0, H 3 < 0..., H n < 0. ( ) n H n > 0. 18

Problem Set 1. Find the critical points for the following functions defined on the real line and determine whether they are maxima, minima or inflection points. a. f x ( ) = x 2 + 3x 4 ( ) = x 3 + 2x 2 x + 3 b. f x 2. Find the maxima and minima of the following functions on [ 1,1]. a. f x ( ) = 2x + 3 b. f ( x) = x 2 + x c. f ( x) = x 1 2 + x + 1 2 d. f ( x) = e x e x e. f ( x) = ex e x e x + e x 3. Find the local maximum and local minimum points for f ( x) = x + 1 x the value of the function at these points. and find ( ) = x 2 e x2 on the real line. Explain your 4. Find the maxima and minima of f x reasoning. 5. Find all of the first- and second- order partial derivatives of the function ( ) = e x2 +xy+y 2. f x, y 6. Suppose that y is a function of x that satisfies x 2 + 2xy + y 2 = 1. Find dy dx. 7. Suppose that z is a function of x and y that satisfies x 5 3xy 2 + yz 2 z 3 = 1 and find δ z δ x and δ z δ y. 8. Maximize xy subject to x + y = 2, where x, y 0. 19

20