Constrained Optimization in Two Variables James K. Peterson Department of Biological Sciences and Department of Mathematical Sciences Clemson University November 17, 216 Outline Constrained Optimization What Does the Lagrange Multiplier Mean?
Let s look at the problem of finding the minimum or maimum of a function f (, y) subject to the constraint g(, y) = c where c is a constant. Let s suppose we have a point (, y) where an etremum value occurs and assume f and g are differentiable at that point. Then, at another point (, y) which satisfies the constraint, we have g(, y) = g(, y) + g(, y)( ) + gy (, y)(y y) + Eg (, y,, y) But g(, y) = g(, y) = c, so we have = g ( ) + g y (y y) + Eg (, y,, y) where we let g = g(, y) and g y = gy (, y). Now given an, we assume we can find a y values so that g(, y) = c in Br () for some r >. Of course, this value need not be unique. Let φ() be the function defined by φ() = {y g(, y) = c} For eample, if g(, y) = c was the function 2 + y 2 = 25, then φ() = ± 25 2 for 5 5. Clearly, we can t find a full circle Br () when = 5 or = 5, so let s assume the point (, y) where the etremum value occurs does have such a local circle around it where we can find corresponding y values for all values in Br (). So locally, we have a function φ() defined around so that g(, φ()) = c for Br (). Then we have = g ( ) + g y (φ() φ()) + Eg (, φ(),, φ()) Now divide through by to get φ() φ() = g + gy Eg (, φ(),, φ()) + Thus, φ() φ() gy = g Eg (, φ(),, φ()) and assuming g and g y, we can solve to find
φ() φ() = g gy 1 gy Eg (, φ(),, φ()) Eg (,φ(),,φ()) Since g is differentiable at (, φ()), as,. Thus, we have φ is differentiable and therefore also continuous with φ φ() φ() () = lim = g g y Now since both g and gy, the fraction g g too. This y means locally φ() is either strictly increasing or strictly decreasing; i.e. is a strictly monotone function. Let s assume φ () > and so φ is increasing on some interval ( R, + R). Let s also assume the etreme value is a minimum, so we know on this interval f (, y) f (, y) with g(, y) = c. This means f (, φ()) f (, φ() on ( R, + R). Now do an epansion for f to get f (, φ()) = f (, φ()) + f ( ) + f y (φ() φ()) + Ef (, φ(),, φ()) where f = f(, φ()) and fy = fy (, φ()). This implies f (, φ()) f (, φ()) = f ( ) + f y (φ() φ()) + Ef (, φ(),, φ()) Now we have assumed f (, φ()) f (, φ), so we have f ( ) + fy (φ() φ) + E f (, φ(),, φ) If > we find φ() φ() f + fy + E f (, φ(),, φ) Now take the limit as + to find f + f y φ
When <, we argue similarly to find Combining, we have f + f y φ () f + f y φ () which tells us at this etreme value we have the equation f + fy φ () =. This implies fy = f φ (). Net note [ ] [ ] f f { f( )} fy = f = f g φ () y = f [ ] g g gy g This says there is a scalar λ = f g so that (f )(, y) = λ (g)(, y) where g(, y) = c. The value f g is called the Lagrange Multiplier for this etremal Problem. This is the basis for the Lagrange Multiplier Technique for a constrained optimization problem. The value λ above is called a Lagrange Multiplier of the etremum problem we are trying to solve. We can do a similar sort of analysis in the case the etremum is a maimum too. Our analysis assumes that the the point (, y) where the etremum occurs is like an interior point in the {(, y) g(, y) = c}. That is, we assume for such an there is a interval Br () with any Br () having a corresponding value y = φ() so that g(, φ()) = c. So the argument does not handle boundary points such as the ±5 is our previous eample. To implement the Lagrange Multiplier technique, we define a new function H(, y, λ) = f (, y) + λ(g(, y) c) The critical points we see are the ones where the gradient of f is a multiple of the gradient of g for the reasons we discussed above. Note the λ we use here is the negative of the Lagrange Multiplier.
To find them, we set the partials of H equal to zero. = = f = λ g = = f y y = λ g = = g(, y) = c λ The first two lines are the statement that the gradient of f is a multiple of the gradient of g and the third line is the statement that the constraint must be satisfied. As usual, solving these three nonlinear equations gives the interior point solutions and we always must also include the boundary points that solve the constraints as well. Eample Etremize 2 + 3y subject to 2 + y 2 = 25. Solution Set H(, y, λ) = (2 + 3y) + λ( 2 + y 2 25). The critical point equations are then = 2 + λ(2) = y = 3 + λ(2y) = λ = 2 + y 2 25 =
Solution Solving for λ, we find λ = 1/ = 3/(2y). Thus, = 2y/3. Using this relationship in the constraint, we find (2y/3) 2 + y 2 = 25 or 13y 2 /9 = 25. Thus, y 2 = 225/13 or y = ±15/ 13). This means = ±1/ 13. We find f (1/ 13, 15/ 13) = 45/ 13 = 12.48 f ( 1/ 13, 15/ 13) = 45/ 13 = 12.48 f ( 5, ) = 1 f (5, ) = 1 f (, 5) = 15 f (, 5) = 15. So the etreme values here occur at the boundary points (, ±5). Eample Etremize sin(y) subject to 2 + y 2 = 1. Solution Set H(, y, λ) = (sin(y)) + λ( 2 + y 2 1). The critical point equations are then = y cos(y) + λ(2) = y = cos(y) + λ(2y) = λ = 2 + y 2 1 =
Solution Solving for λ in the first equation, we find λ = y cos(y)/2. Now use this in the second equation to find cos(y) 2y 2 cos(y)/2 =. Thus, cos(y)( y 2 /) =. The first case is cos(y) =. This implies y = π/2 + 2nπ. These are rectangular hyperbolas and the closest one to the origin is y = π/2 = 1.57 whose closest point to the origin is ( π/2, π/2) = (1, 25, 1, 25). This point is outside of the constraint set, so this case does not matter. The second case is 2 = y 2 which, using the constraint, implies 2 = 1/2. Thus there are four possibilities: (1/ 2, 1/ 2), (1/ 2, 1/ 2), ( 1/ 2, 1/ 2) and ( 1/ 2, 1/ 2). The other possibilities are the circle s boundary points (, ±1) and (±1, ). We find sin(1/ 2, 1/ 2) = sin( 1/ 2, 1/ 2) = sin(1/2) =.48 sin(1/ 2, 1/ 2) = sin( 1/ 2, 1/ 2) = sin(1/2) =.48 sin(±1, ) =, sin(, ±1) = So the etreme values occur here at the interior points of the constraint. Let s change our etremum problem by assuming the constraint constant is changing as a function of ; i.e. the problem is now find the etreme values of f (, y) subject to g(, y) = C(). We can use our tangent plane analysis like before. We assume the etreme value for constraint value C() occurs at an interior point of the constraint. We have g(, φ()) C() = Then φ() φ() = g + gy Eg (, φ(),, φ() C() C() + We assume the constraint bound function C is differentiable and so letting, we find g + g y φ () = C () Now assume (1, φ(1)) etremizes f for the constraint bound value C(1).
We have f (1, φ(1)) f (, φ()) = f (1 ) + f y (φ(1) φ()) +Ef (1, φ(1),, φ()) or f (1, φ(1)) f (, φ()) 1 φ(1) = f + fy φ() 1 ( Ef (1, φ(1),, φ()) + 1 ) Now assuming C() locally around, we can write f (1, φ(1)) f (, φ()) C(1) C() = C(1) C() 1 φ(1) f + fy φ() Ef (1, φ(1),, φ()) + 1 1 Now let Θ() be defined by Θ() = f (, φ()) when (, φ()) is the etreme value for the problem of etremizing f (, y) subject to g(, y) = C(). This is called the Optimal Value Function for this problem. Rewriting, we have Θ(1) Θ() C(1) C() = C(1) C() 1 ( φ(1) f + fy φ() 1 ) + Ef (1, φ(1),, φ()) 1 Now let 1. We obtain Θ(1) Θ() lim C () 1 C(1) C() = f + fy φ () Thus, the rate of change of the optimal value with respect to change in the constraint bound, dθ dc is well - defined and where C is the value C(). dθ dc (C) C () = f + f y φ ()
Assuming C (), we have dθ dc (C) = f C () + f y C () φ () But we know at the etreme value for that fy = g y g f. Thus, dθ dc (C) = f C () + g y f g C () φ () = f ( C 1 + g ) y () g φ () = f ( ) 1 g C g + gy φ () () But the Lagrange Multiplier λ for etremal value for is λ = f ( ) dθ 1 (C) = λ g + gy φ () dc C () But C () = g + gy φ (). Hence, we find dθ dc (C) = λ. g. So We see from our argument that the Lagrange Multiplier λ is the rate of change of optimal value with respect to the constraint bound. There are cases: sign(λ) = sign( f g ) = + + = + which implies the optimal value goes up if the constraint bound is perturbed. sign(λ) = sign( f g ) = = + which implies the optimal value goes up if the constrained bound is perturbed. sign(λ) = sign( f g ) = + = which implies the optimal value goes down if the constrained bound is perturbed. sign(λ) = sign( f g ) = + = which implies the optimal value goes down if the constrained bound is perturbed. Hence, we can interpret the Lagrange Multiplier as a price: the change in optimal value due to a change in constraint is essentially the loss of value eperienced due to the constraint change. For eample, if λ is very small, it says the rate of change of the optimal value with respect to constraint modification is small too. This implies the optimal value is insensitive to constraint modification.
Homework 37 37.1 Find the etreme values of sin(y) subject to 2 + 2y 2 = 1 using the Lagrange Multiplier Technique. 37.2 Find the etreme values of cos(y) subject to 2 + 2y 2 = 1 using the Lagrange Multiplier Technique. 37.3 If an 1 and bn 3, use an ɛ N proof to show 5an/bn 5/3. 37.4 If an 1 and bn 2, use an ɛ N proof to show 5anbn 1.