Higher order derivative

2

Î 3 á Higher order derivative Î 1 Å Iterated partial derivative Iterated partial derivative Suppose f has f/ x, f/ y and ( f ) = 2 f x x x 2 ( f ) = 2 f x y x y ( f ) = 2 f y x y x ( f ) = 2 f y y y 2 These are the second partial derivatives of f. Use simplified expression f/ x = f x, f/ y = f y x 2 = f xx, y x = f xy, x y = f yx, y 2 = f yy Example 1.1. (1) f(x,y) = e xy + x 2 y (2) f(x,y,z) = x 2 y + y 2 z + z 3 x 3

4 3 ì HIGHER ORDER DERIVATIVE (x 0, y 0 ) 3.1: Difference sol. (1) For f(x,y) = e xy + x 2 y f x = ye xy + 2xy, f y = xe xy + x 2 f xx = y 2 e xy + 2y, f yy = x 2 e xy f xy = f yx = e xy + xye xy + 2x (2) For f(x,y,z) = x 2 y + y 2 z + z 3 x f x = 2xy + z 3, f y = x 2 + 2yz, f z = y 2 + 3z 2 x f xx = 2y, f yy = 2z, f zz = 6xz f xy = f yx = 2x f xz = f zx = 3z 2 f yz = f zy = 2y Mixed partials mixed partial derivative. Derivatives such as f xy,f yx, f xz,f zx, f yz,f zy are mixed partials. Theorem 1.2. If f(x,y) is class C 2 Then x y = 2 f y x Also, x i x j = 2 f x j x i

1 ITERATED PARTIAL DERIVATIVE 5 Proof. Need more motivation. Recall for fixed y 0 Then second derivative must be lim ( f x ) y lim f x = f(x 0 + x,y 0 ) f(x 0,y 0 ). x = f(x 0 + x,y 0 + y) f(x 0,y 0 + y) f(x 0 + x,y 0 ) + f(x 0,y 0 ). y x Let s define the increment of f(correspond. to second derivative) by Fix y 0 and y and define S( x, y) = f(x 0 + x,y 0 + y) f(x 0 + x,y 0 ) f(x 0,y 0 + y) + f(x 0,y 0 ) g(x) = f(x,y 0 + y) f(x,y 0 ) Then S( x, y) = g(x 0 + x) g(x 0 ). Use MVT. There is x 0 x x 0 + x such that S( x, y) = g ( x) x. So S( x, y) = g ( x) x ( f = x ( x,y 0 + y) f ) x ( x,y 0) x Apply MVT for y again there is y 0 ỹ y 0 + y such that But / x y is continuous S( x, y) = 2 f y x ( x,ỹ) x y y x (x S( x, y) 0,y 0 ) = lim ( x, y) (0,0) x y l.h.s. depends on the order of x, y but r.h.s. is independent of x and y. So exchanging the role of x and y. Then x y (x S( x, y) 0,y 0 ) = lim ( x, y) (0,0) y x

6 3 ì HIGHER ORDER DERIVATIVE Example 1.3. Find mixed partial / y x of f(x,y) = xy 2 e x2 x /(x 2 +1). sol. By Thm 1.2, f yx may be computed instead of f xy. f yx = (2xy) x = 2y Computing f xy is more complicated then f yx. Example 1.4. Show the following when f belongs to C 3. 3 f x y z = 3 f z x y sol. We can change order of differentiation 3 f x y z = ( x y z = 2 x z ( f y = 3 f z x y ) = ( ) x z y ) = 2 ( f ) z x y Example 1.5. Find all the second derivatives of h(s,t) = g f(s,t) = g(x(s,t),y(s,t)) when g(x,y): R 2 R and f(s,t) = (x(s,t),y(s,t)): R 2 R all functions are class C 2

2 TAYLOR THEOREM 7 sol. Chain rule Then h s = g x x s + g y y s, h t = g x x t + g y y t h ss = (g x ) s x s + g x x ss + (g y ) s y s + g y y ss = (g xx x s + g xy y s )x s + g x x ss + (g yx x s + g yy y s )y s + g y y ss, h st = h ts = (g x ) t x s + g x x st + (g y ) t y s + g y y st = (g xx x t + g xy y t )x s + g x x st + (g yx x t + g yy y t )y s + g y y st, and In matrix form, h tt = (g x ) t x t + g x x tt + (g y ) t y t + g y y tt = (g xx x t + g xy y t )x t + g x x tt + (g yx x t + g yy y t )y t + g y y tt h s = Dg D s x, h ss = D(Dg) (D s x) 2 + Dg D ss x, etc. Î 2 Å Taylor theorem The linear approximation of f(x 0 + h) is f(x 0 ) + Df(x 0 )h and the error R = f(x 0 + h) f(x 0 ) Df(x 0 )h satisfies lim h 0 R x x 0 = 0 when f is differentiable at x 0. What if we want higher order approximation? Theorem 2.1 (Taylor theorem one variable-integral remainder). If f : R R has continuous n- th partial derivatives f(x 0 + h) = f(x 0 ) + f (x 0 )h + f (x 0 ) h 2 + 2 + + f(n) (x 0 ) h n + R n (h,x 0 ) n! Here R n (x 0,h) = x0 +h x 0 (x 0 + h τ) n f (n+1) (τ)dτ n!

8 3 ì HIGHER ORDER DERIVATIVE and R n (x 0,h)/h n 0 as h 0. Proof. x0 +h f(x 0 + h) = f(x 0 ) + f (τ)dτ. x 0 Recall integration by parts: udv = uv x 1 x 0 v du Integrate by parts, (with dτ = d(x 0 + h τ)) we have x0 f(x 0 + h) = f(x 0 ) + f +h (x 0 )h + f (τ)(x 0 + h τ)dτ. x 0 which is first order formula. Integrate by parts again, x0 +h x 0 = 1 2 f (τ)(x 0 + h τ)dτ x0 +h x 0 f (τ)d(x 0 + h τ) 2 = 1 2 f (x 0 )h 2 + 1 2 x0 +h x 0 f (τ)(x 0 + h τ) 2 dτ. We can repeat this k-times. Also the remainder satisfies R k hk+1 (k + 1)! M Theorem 2.2 (Taylor theorem 1st order). If f : R n R is C 1 differentiable at x 0, then f(x 0 + h) = f(x 0 ) + [Df(x 0 )] h + R 1 (x 0,h) where R 1 (h,x 0 )/ h 0 as h 0. Here, h = (h 1,...,h n ). In scalar form, we have n f(x 0 + h) = f(x 0 ) + h i f xi + R 1 (h,x 0 ) i=1

2 TAYLOR THEOREM 9 Theorem 2.3 (Taylor theorem 2nd order). If f : R n R is class C 3 n f(x 0 + h) = f(x 0 ) + h i f xi (x 0 ) + 1 n h i h j f xi x 2 j (x 0 ) + R 2 (h,x 0 ) i=1 i,j=1 where R 2 (x 0,h)/ h 2 0 as h 0. In matrix form, f(x 0 + h) = f(x 0 ) + ( f,, f h 1 ) x 1 x n. h n + 1 x 1 x 1 x 1 x 2 2 (h 2 f 1,,h n ) x 2 x 1 x 2 x 2 x n x 1 x n x 2 = f(x 0 ) + Df(x 0 )h + 1 2 ht D(Df)(x 0 )h + R 2. x 1 x n h 1 x 2 x n. h n x n x n +R 2(h,x 0 ) Proof. Let g(t) = f(x 0 +th) and use Taylor theorem in one variable and chain rule. Then and By Chain rule, g(1) = g(0) + g (0) + g (0) 2! R 2 = 1 0 (t 1) 2 g (t)dt 2! + R 2 n g (t) = f xi (x 0 + th)h i, n g (t) = f xi x j (x 0 + th)h i h j, i=1 i,j=1 and n g (t) = f xi x j x k (x 0 + th)h i h j h k i,j,k=1 n R 2 (h,x 0 ) = i,j,k=1 1 0 (1 t) 2 f xi x 2 j x k (x 0 + th)h i h j h k dt

10 3 ì HIGHER ORDER DERIVATIVE Then f(x 0 + h) = f(x 0 ) + n h i f xi (x 0 ) + 1 n h i h j f xi x 2 j (x 0 ) + R 2 (h,x 0 ) i=1 i,j=1 Here the integrand of R 2 (x,h) is continuous. So if h is small, it is less than M. So R 2 (h,x) M h 3. In other words R 2 (x,h) / h 2 0 as h 0. Example 2.4. Find 2nd order Taylor approximation of f(x,y) = e x+y near x 0 = (0,0). sol. Partials of f are f x (0,0) = f y (0,0) = e 0 = 1 f xx (0,0) = f xy (0,0) = f yy (0,0) = e 0 = 1 f(x,y) = 1 + x + y + 1 2 (x2 + 2xy + y 2 ) + R 2 As (x,y) (x 0,y 0 ), R 2 / (x,y) 2 0. Theorem 2.5. (1) Remainder of Taylor s theorem 2.2 is given by R 1 (h,x 0 ) = n i,j=1 1 0 (1 t)f xi x j (x 0 + th)h i h j dt = Here c is a point between x 0 and x 0 + h. (2) Remainder of Taylor s theorem 2.3 is R 2 (h,x 0 ) = n i,j,k=1 1 0 n i,j=1 (1 t) 2 f xi x 2 j x k (x 0 +th)h i h j h k dt = Here c is a point between x 0 and x 0 + h. 1 2 f x i,x j (c)h i h j n i,j,k=1 Proof. [MVT for integral] If h and g are continuous on [a,b] and g > 0 then for some c [a, b] the following holds b a b h(t)g(t)dt = h(c) g(t) dt a 1 3! f x i,x j,x k (c)h i h j h k

2 TAYLOR THEOREM 11 (1) From thm 2.2 we see n R 1 (h,x 0 ) = i,j=1 1 0 (1 t)f xi x j (x 0 + th)h i h j dt If we set h(t) = f xi x j (x 0 + th) and g(t) = (1 t)h i h j and use MVT. Then 1 0 (1 t)f xi x j (x 0 + th)h i h j dt 1 =f xi x j (c) (1 t)h i h j dt 0 = 1 2 f x i x j (c)h i h j (2) similar Second order Taylor formula gives second order approximation. Example 2.6. Find second order approx. of f(x,y) = sin xsin y. What is error if x 0.1, y 0.1. sol. partials of f(x,y) are f(0,0) = 0, f x (0,0) = 0, f y (0,0) = 0 f xx (0,0) = 0, f xy (0,0) = 1, f yy (0,0) = 0 Hence f(x,y) = 0 + 0 + 0 + 1 2 ( ) x 2 (0) + 2xy + y 2 (0) + R 2 So sinxsin y xy and the error is R 2 = 1 6 (x3 f xxx + 3x 2 yf xxy + 3xy 2 f xyy + y 3 f yyy ) (c1,c 2 ) 1 ((0.1) 3 + 3(0.1) 1 + 3(0.1) 1 + (0.1) 1) 6 8 6 (0.1)3 0.00134 Example 2.7. Find second order approx. of f(x,y) = e x cos y at (0,0).

12 3 ì HIGHER ORDER DERIVATIVE sol. partials of f(x,y) are f(0,0) = 0, f x (0,0) = 1, f y (0,0) = 0 f xx (0,0) = 1, f xy (0,0) = 0, f yy (0,0) = 1 Hence f(h) = 1 + h 1 + 1 2 ( ) h 2 1 h 2 2 + R 2 where R 2 / h 2 0 as h 0. Example 2.8. Find approx. value of (3.98 1) 2 /(5.97 3) 2 compare with exact value. sol. Let f = (x 1) 2 /(y 3) 2. Desired value is close to f(4,6) = 1. partials of f(x,y) are f(4,6) = 1, f x (4,6) = 2 3, f y(4,6) = 2 3, f xx (4,6) = 2 9, f xy(4,6) = 4 9, f yy(4,6) = 2 3 Hence linear approx. is while quadratic approx is 1 + 2 3 ( 0.02) 2 ( 0.03) = 1.00666 3 1+ 2 3 ( 0.02) 2 3 ( 0.03)+2 ( 0.02) 2 4 9 2 9 ( 0.02)( 0.03)+2 ( 0.03) 2 3 2 = 1.00674 exact value is 1.00675.(calculator value) Î 3 Å Extrema of real valued functions Local Max, Min Definition 3.1. f : R n R has local minimum at x 0 R n if there is a nhd V of x 0 such that f(x 0 ) f(x) for all x V. Similarly, f has a local

3 EXTREMA OF REAL VALUED FUNCTIONS 13 maximaum if there is a nhd V of x 0 such that f(x 0 ) f(x) for all x V. The point x 0 is called local, or relative extremum A critical point which is either local max or min is called a saddle. local max local min 3.2: Near extreme Definition 3.2. A point x 0 R n is critical point if f is not differentiable or f(x 0 ) = 0 = (0,...,0). Theorem 3.3 (First derivative test for local extrema). If f : U R n R is differentiable at x 0 R n and has a extreme then Df(x 0 ) = 0, i.e, x 0 is a critical point. Proof. Suppose f has local maximum at x 0. Then for any h R n, the function g(t) = f(x 0 + th) has a local minimum. Hence g (0) = D h f(x 0 ) = f(x 0 ) h = 0 Sine this holds for every h, f(x 0 ) = 0. i.e, x 0 is critical point of f. Example 3.4. Find max and min f = x 2 + y 2 Example 3.5. Find extrema f = x 2 y 2 Example 3.6. Find critical points of z = x 2 y + y 2 x. sol. z x = 2xy + y 2, z y = 2xy + x 2 Set them equal to 0, obtain x 2 = y 2, Then for x = y, we get 2y 2 +y 2 = 0. Hence we get (0,0). For x = y, again get x = y = 0. Now for x = y, z = 2x 3. Not a extreme. So saddle.

14 3 ì HIGHER ORDER DERIVATIVE saddle point 3.3: Near Saddle point z y x y z x y z x z = x 2 + y 2 z = x 2 y 2 z = x 2 y 2 3.4: Graphs Example 3.7. z = 2(x 2 + y 2 )e x2 y 2 sol. z x = [4x + 2( 2x)(x 2 + y 2 )]e (x2 +y 2 ) = 4x(1 x 2 y 2 )e (x2 +y 2 ) z y = 4y(1 x 2 y 2 )e (x2 +y 2 ) Solving obtain x = y = 0 or x 2 + y 2 = 1, The points on the crater s rim are local maximum

3 EXTREMA OF REAL VALUED FUNCTIONS 15 Second derivative test Definition 3.8. Hf(x 0 )(h) = 1 2 (h 1,,h n ) x 1 x 1 x 2 x 1 x n x 1 x 1 x 2 x 2 x 2 x n x 2 x 1 x n h 1 x 2 x n. h n x n x n This is called (homogeneous) Hesssian at (x 0,y 0 ). At critical point, DF(x 0 ) = 0. Hence f(x 0 + h) = f(x 0 ) + Hf(x 0 )(h) + R 2 (x 0,h) A quadratic function g : R n R is said to be positive definite if g(h) 0 for all h R n, and g(h) = 0 only when h = 0. It is said to be negative definite if g(h) 0 for all h R n, and g(h) = 0 only when h = 0. Remark 3.9. If g is quadratic, then g(λh 1,,λh n ) = λ 2 g(h 1,,h n ). Theorem 3.10. [Second derivative test] Suppose f is C 3 and (x 0,y 0 ) is a critical point of f. If the Hessian Hf(x 0 ) is positive definite, then x 0 is relative minimum. Similarly, if Hf(x 0 ) is negative definite, then x 0 is relative maximum. Example 3.11. Consider f(x,y) = x 2 + y 2. (0,0) is the critical point. We see f(x,y) = f(0,0) + (h 2 1 + h 2 2) + 0 Hf = h 2 1 + h2 2 This is positive definite, hence (0,0) is min. Lemma 3.12. If B = [b ij ] is n n real matrix and if H : R n R,(h 1,,h n ) 1 2 bij h i h j is positive-definite, then there is M > 0 such that for all h H(h) M h 2 Proof. For h = 1, set g(h) = H(h). Then g is continuous function on a

16 3 ì HIGHER ORDER DERIVATIVE closed set, hence have a positive minimum say M. Because H is quadratic, for any h 0. H(h) = H( h h h ) = H( h h ) h 2 = g( h h ) h 2 M h 2 Proof of Theorem 3.10. f(x 0 ) f(x 0 ) = H(x 0 )(h) + R 2 where R 2 / h 2 0. Since Hf(x 0 ) is positive definite, Hf(x 0 )(h) M h 2 and R 2 < M h 2. hence R 2 > 0 for 0 < h < δ. So a strict min. Example 3.13. (1) f(x,y) = x 4 + y 4 (2) h(x,y) = x 3 + y 3 sol. (1) One can check Hf(0) = h 2 1 + h2 2. So rel minimum. (2) (0,0) is the only critical point of f and D = 0 at (0,0). So the test fails. It is easy to check f 0 for all (x,y) and f(0,0) = 0. So (0,0) is local minimum (3) Again D = 0. h 0 if x 0, y 0 or x 0, y 0 Also, h 0 for x 0, y 0 or x 0, y 0. Hence (0,0) is a saddle of h. Determinant test for Positive definiteness Then how do we know Positive definiteness? Lemma 3.14. Let [ ] a b B = b c and H(h) = 1 2 [h 1,h 2 ] B [ ] h1 h 2 Then H(h) is positive definite iff a > 0 and det B = ac b 2 > 0.

3 EXTREMA OF REAL VALUED FUNCTIONS 17 Theorem 3.15 (Second derivative test). Suppose f is C 3 on an open subset U of R 2. A point (x 0,y 0 ) is a local min, if the following holds (1) f x (x 0,y 0 ) = f y (x 0,y 0 ) = 0. (critical point of f) (2) f xx (x 0,y 0 ) > 0 (3) D = f xx (x 0,y 0 )f yy (x 0,y 0 ) (f xy (x 0,y 0 )) 2 D is called the discriminant. If cond. (2) is replaced by f xx (x 0,y 0 ) < 0 then f has local max at (x 0,y 0 ). If If D < 0, (x 0,y 0 ) is saddle. Proof of pos. definiteness when D > 0 Hf = 1 2 [h 1,h 2 ] [ ] [ ] fxx f xy h1 f yx f yy h 2 [ ] a b Thus we need to check definiteness of the matrix of the form B = b c z = ax 2 + 2bxy + cy 2 ac b 2 > 0 a < 0 a > 0 z = ax 2 + 2bxy + cy 2 ac b 2 < 0 3.5: Graphs of quadratic functions [ ] a b Lemma 3.16. Let B = and H(h) = 1 2 b c ht Bh. Then H(h) is positivedefinite if and only if a > 0 and ac b 2 > 0.

18 3 ì HIGHER ORDER DERIVATIVE sol. We have [ ] [ ] ah1 + bh 2 h1 H(h) = 1 2 [h 1,h 2 ] bh 1 + ch 2 h 2 = 1 2 (ah2 1 + 2bh 1h 2 + ch 2 2 ) ) 2 + 1 2 ( ) H(h) = 1 2 a (h 1 + b a h 2 c b2 a h 2 2 Suppose H is positive definite, then setting h 2 = 0, we have a > 0. Setting h 1 = 0, we get ac b 2 > 0. Converse also hold. Similarly, we have negative definite if a < 0 and ac b 2 > 0. Example 3.17. Classify the critical points of the following form. (1) g(x,y) = 3x 2 + 6xy + 9y 2 (2) h(x,y) = 2x 2 + xy y 2 (3) k(x,y) = x 2 xy + 2y 2 sol. All the critical points are (0,0). For g, we see D = 3 9 3 2 = 18 > 0. Hence (0,0) is a local min of g. For h, we have D = ( 2)( 1) 1/4 = 7/4 > 0 and a = 2 < 0, we see h has local maximum at (0,0). For k, D = 2 1 1/4 = 3/4 > 0 and a = 1 > 0, hence k has local minimum at (0,0). Example 3.18. Locate relative maxima minima saddle of f(x,y) = log(x 2 + y 2 + 1) sol. f = 0 gives (0,0) as a critical point. Second derivatives are f xx (0,0) = 2 = f yy (0,0), f xy (0,0) = 0 Hence D = 2 2 = 4 > 0. Hence min. Example 3.19. The graph of g = 1/xy is a surface S. Find the point on S closest to (0,0).

3 EXTREMA OF REAL VALUED FUNCTIONS 19 Each point on the surface is (x,y,1/xy). Hence sol. d 2 = x 2 + y 2 + 1 x 2 y 2 We find the point which minimize f(x,y) = d 2 (x,y) rather than d. f x = 2x 2 x 3 y 2 = 0 f y = 2y 2 x 2 y 3 = 0. That is, x 4 y2 = 1 and x 2 y 4 = 1. From the first eq. we get y 2 = 1/x 4. Subst into second eq, we get x 6 = 1. So x = ±1 and y = ±1. All these four points are min. (d = 3) As x or y approaches, f. So f has min somewhere and no max. Example 3.20. Find the critical points of f(x,y) = (x 2 y 2 )e ( x2 y 2 )/2 and determine if they are local max. or min. or neither.

20 3 ì HIGHER ORDER DERIVATIVE sol. Critical points. f x = [2x x(x 2 y 2 )]e ( x2 y 2 )/2 = 0 f y = [ 2y y(x 2 y 2 )]e ( x2 y 2 )/2 = 0 We see Hence x[2 (x 2 y 2 )] = 0, y[ 2 (x 2 y 2 )] = 0. (x,y) = (0,0), (± 2,0), (0, ± 2) On the other hand, f xx = [2 5x 2 + x 2 (x 2 y 2 ) + y 2 ]e ( x2 y 2 )/2, f yy = [5y 2 2 + y 2 (x 2 y 2 ) x 2 ]e ( x2 y 2 )/2, f xy = xy(x 2 y 2 )e ( x2 y 2 )/2. Since D(0,0) = 4 (0,0) is a saddle. While D(± 2,0) = 16/e 2 > 0 and f xx (± 2,0) = 4/e. So (± 2,0) is local min. Since D(0, ± 2) = 16/e 2 > 0 and f xx (0, ± 2) = 4/e, (0, ± 2) is local max. Graph is as Fig 3.6. z y x 3.6: f(x,y) = (x 2 y 2 )e ( x2 y 2 )/2 Global maxima and Minima Definition 3.21. Suppose f D R n is real valued function. A point x 0 D is an absolute maximum if f(x 0 ) f(x) for all x A. Similarly, it is an

3 EXTREMA OF REAL VALUED FUNCTIONS 21 absolute minimum if f(x 0 ) f(x) for all x A. Strategy (1) Find all critical points (2) Find max or min of U(by parametrization) (3) Compute values at critical points (4) Compare all of the above. Example 3.22. Find maximum or minimum of f(x,y) = x 2 + y 2 x y + 1 in D = {(x,y) R 2 x 2 + y 2 1}. sol. Critical points of f. Since f x = 2x 1 = 0, f y = 2y 1 = 0, (1/2,1/2) is the only critical points. (By second derivative test f xx = 2, f xy = 0, f yy = 2 f xx f yy fxy 2 = 4 > 0, f xx = 2 > 0 (1/2,1/2) is minimum) Now on the boundary D: x 2 + y 2 = 1. Use parametrization x = cos t, y = sin t, 0 t 2π to f. g(t) = sin 2 t + cos 2 t sin t cos t + 1 = 2 sin t cos t See g (t) = cos t + sint = 0 hence t = π/4,5π/4 are critical points. We have to check the end points t = 0,2π also. Hence the values are g(0) = 1, g(π/4) = 2 2. g(5π/4) = 2 + 2, g(2π) = 1 Comparing, we see maximum is at t = 5π/4, (x,y) = ( 2/2, 2/2) and Min at π/4. Existence of max and min Definition 3.23. A set D R n is bounded if x M for all x D. It is Closed if it contains all boundary points. (fig 3.7 ) Example 3.24. (1) D = {(x,y) R 2 x 2 + y 2 1} (2) D 0 = D {(0,0)}

22 3 ì HIGHER ORDER DERIVATIVE A p q x 0 D ε(x 0) 3.7: A neighborhood D ǫ (x 0 ) of a boundary point x 0 contains contains both points of A and points not in A D M(0) A 3.8: A is contained in a neighborhood D M (0). (3) S = {(x,y) R 2 x 2 + y 2 = 1} Theorem 3.25 (Existence of maximum and minimum). If f : A R is continuous function defined on a closed bounded set A R n. Then there are x 0,x 1 A such that for all x A f(x 0 ) f(x) f(x 1 ) Î 4 Å Constrained Extrema and Lagrange multiplier Lagrange multiplier method Theorem 4.1 (Lagrange multiplier method). Assume f : R n R and g: R n R are C 1 class. The restriction of f to the level set S = {x R n g(x) = c} (written as f S ) has maximum or minimum at x 0 S where g(x 0 ) 0. Then there is a λ (Lagrange multiplier) such that f(x 0 ) = λ g(x 0 )

4 CONSTRAINED EXTREMA AND LAGRANGE MULTIPLIER23 g(x 0) f(x 0) x 0 x1 f(x, y) = M x 2 f(x, y) = M 1 g(x,y) = C f(x, y) = M 2 3.9: Lagrange multiplier method Proof. First recall that g(x 0 ) is perpendicular to the level surface S at x 0. If c(t) is any curve in S, then since f is constant along any curve, df(c(t))/dt t=0 = 0. 0 = df(c(t)) dt = f(x 0 ) c (0) t=0 Hence f(x 0 ) perpendicular to c (0). So perpend. to tangent plane of S at x 0 Hence f(x 0 ) and g(x 0 ) are parallel(orthogonal to the same plane). Hence for some λ, f(x 0 ) = λ g(x 0 ) holds. Theorem 4.2. if f has max or min at a point x 0 of S. Then f S. To solve it we solve system of equations with n + 1 variables f(x) = λ g(x) g(x) = c Or f x1 = λg x1. f xn = λg xn g(x) = c (3.1) Another interpretation of Lagrange multiplier method: Let h(x 1,,x n λ) = f(x 1,,x n ) λ[g(x 1,,x n ) c]. Lagrange multiplier method says to find

24 3 ì HIGHER ORDER DERIVATIVE extreme points of f S, we should find extreme points of h. That is equivalent to solving equations (3.1). Example 4.3. Find extrema of f(x,y) = x 2 +y 2 on the set S, where S is the line y = x + 1. See figure in the book, p229. Example 4.4. Find max of f(x,y) = x 2 y 2 on S : x 2 + y 2 = 1. (See figure) where the two level curves touch. sol. Since g(x,y) = x 2 + y 2 = 1 and f = (2x 2y), g = (2x,2y) the equation is f x (x,y) = λg x (x,y) 2x = λ2x f y (x,y) = λg y (x,y) 2y = λ2y g(x,y) = 1 x 2 + y 2 = 1 From the first equation we get x = 0 or λ = 1. If x = 0, we see from third equation y = ±1. If λ = 1 then y = 0 and x = ±1. Now f(0,1) = f(0, 1) = 1, f(1,0) = f( 1,0) = 1. Hence max is 1 min is 1. Example 4.5. Find max of f(x,y,z) = x + z subject to x 2 + y 2 + z 2 = 1. sol. By Lagrange multiplier 1 = 2xλ 0 = 2yλ 1 = 2zλ 1 = x 2 + y 2 + z 2 From first and third equation we see λ 0 and x = z. Hence from second equation y = 0. From fourth equation we obtain x = z = ±1/ 2. Hence (1/ 2,0,1/ 2) and ( 1/ 2,0, 1/ 2). 2/ 2 is max and 2/ 2 is min.

4 CONSTRAINED EXTREMA AND LAGRANGE MULTIPLIER25 Example 4.6. Find the maximum volume of rectangular box with fixed surface area 10m 2. sol. Let x, y, z be the dimension. Then volume is f(x,y,z) = xyz. But surface are is 10. Hence the condition g(x,y,z) = 2(xy + yz + zx) = 10 is the constraint. yz = λ(y + z), xz = λ(x + z), yx = λ(y + x), 5 = xy + yz + zx. Since x > 0, y > 0, z > 0, y + z 0, x + z 0. So eliminating λ we get yz/(y + z) = xz/(x + z). Hence x = y. Similarly, y = z and we see x = y = z = 5/3. i,e, f( 5/3, 5/3, 5/3) = (5/3) 3/2 are candidates for maximum or minimum. Surface S : xy + yz + zx = 5 is not bounded. If function value f(x,y,z) approaches 0 as any of x or y z approaches 0 or then (5/3) 3/2 is max. Example 4.7. Find max(min) of f(x,y) = x 2 y 2 on x 2 + y 2 1. sol. Critical points of f. Since f x = 2x = 0, f y = 2y = 0, (0,0) is the only critical point. f(0,0) = 0. But D = f xx f yy fxy 2 = 4 < 0 hence it is a saddle. We have seen in Example 4.4 that f has max and min 1 and 1 Several constraint Let S be the surface determined by the following equations: g 1 (x) = c 1, g k (x) = c k

26 3 ì HIGHER ORDER DERIVATIVE If f : R n R is C 1 class and has max or min on S then there exist λ 1,...,λ k such that f(x 0 ) = λ 1 g 1 (x 0 ) + + λ k g k (x 0 ) Example 4.8. Find extreme points of f = x + y + z subject to x 2 + y 2 = 2 and x + z = 1. sol. Constraints are g 1 = x 2 + y 2 2 = 0 and g 2 = x + z 1 = 0. Thus f = λ 1 g 1 + λ 2 g 2 Since g 1 = x 2 + y 2 2 g 2 = x + z 1 we obtain 1 = λ 1 2x + λ 2 1 1 = λ 1 2y + λ 2 0 1 = λ 1 0 + λ 2 1 0 = x 2 + y 2 2 0 = x + z 1 From third equation we obtain λ 2 = 1 and so λ 1 2x = 0 and λ 1 2y = 1. From second, we see λ 1 0, hence x = 0. Thus y = ± 2 and z = 1. Hence possible extrema are (0, ± 2,1). (0, 2,1) give max(0, 2,1) give min. Example 4.9. Find absolute maximum and minimum of f = xy on x 2 +y 2 1.

5 CONSTRAINED EXTREMA AND LAGRANGE MULTIPLIER27 sol. First find critical points. f x = y, f y = x (0,0) is the only critical point. Now consider on the unit circle g = x 2 + y 2 = 1. f = λ g (y,x) = λ(2x,2y) Thus we get y = 4λ 2 y, or λ = ±1/2 and y = ±x. So y = ±1/ 2 = x. Checking f values at these points we see f has max 1/2 and 1/2 is min. By checking second derivative, (0,0) is saddle Example 4.10. Find absolute maximum and minimum of f = 1 2 x2 + 1 2 y2 on 1 2 x2 + y 2 1. sol. First find critical points. f x = x, f y = y (0,0) is the only critical point. Now consider on the unit circle g = x 2 +y 2 = 1. Use Lagrange method.(recall one could use parametrization as before). Thus we get f = λ g (x,y) = λ(x,2y) x = λx y = 2λy 1 2 x2 + y 2 = 1 So (0, ±1) and (± 2,0). Checking f values at these points we see f has max 1 at (0,0).

28 3 ì HIGHER ORDER DERIVATIVE Î 5 Å Implicit function theorem Implicit function theorem-one variable Theorem 5.1. y = f(x) and f (x 0 ) 0. Then inverse function x = f 1 (y) exists near y 0 = f(x 0 ). We generalize this to higher dimensions. A special case first. Theorem 5.2. Suppose F : R n+1 R has continuous partials. Denote points in R n+1 by (x,z). assume F(x,z) satisfies F(x 0,z 0 ) = 0 and F z (x 0,z) 0. Then there is a ball U containing x 0 and V of z 0 such that z = g(x) for x U and satisfies F(x,g(x)) = 0. Moreover, evaluated at z = g(x). Dg = 1 D F x F(x,z) z Proof. Sketch only. Assume z is a function of x near x 0, take derivative of Fz(x 0,z) with resp. to x i. Then F + F z = 0. x i z x i So z = x i F x i F z. Example 5.3. Show that near (x, y, uy, v) = (1, 1, 1, 1) we can solve xu + yvu 2 = 2 xu 3 + y 2 v 4 = 2

5 IMPLICIT FUNCTION THEOREM 29 sol. Let F 1 = xu + yvu 2 2 F 2 = xu 3 + y 2 v 4 2 and check = = = F 1 F 1 u v F 2 F 2 u v x + 2yuv yu 2 3u 2 4y 2 v 3 3 1 3 4 = 9

30 3 ì HIGHER ORDER DERIVATIVE