8 Extremal Values & Higher Derivatives

Transcription

1 8 Extremal Values & Higher Derivatives Not covered in 2016\17 81 Higher Partial Derivatives For functions of one variable there is a well-known test for the nature of a critical point given by the sign of the second derivative Assuming f (a = 0 if f (a > 0 then a is a minimum, if f (a < 0 then a is a maximum This result can be generalised to functions of severable variable but to do this we have to consider higher derivatives in this setting Given a function f : U R where U is open in R n with partial derivatives d i f = f/ x i : U R, 1 i n, we can consider whether the partial derivatives themselves have partial derivatives Recall from the last Chaper Definition 1 If d i f = f/ x i : U R has a j-th partial derivative then this is written ( f f = 2 f x j x i x j x : U R i When j = i this can be written as 2 f ( x i 2 : U R These are all called the partial derivatives of f of order 2 Similarly, we can define partial derivatives of order q for any q 1 by induction on q (when they exist: q f x iq x i q 1 x i 2 x i 1 = ( q 1 f x iq x i q 1 x i 2 x i 1 Definition 2 If all of the q-th order partial derivatives of a function f : U R where U is open in R n exist and are continuous on U, then we say that f is a function of class C q, or a C q -function Remark By an earlier result if all the partial derivatives of a function exist the function need not be differentiable But if the derivatives exist and are continuous, ie it is a C 1 -function, then it is differentiable Further, differentiable implies continuous, hence if a function is of class C 1 then it is of class C 0 This argument can be repeated to show that if a function is of class C q the it is of class C q 1 for all q 1 Definition A function which is of class C q for all q 0 is said to be a C -function or a smooth function Example 4 is a smooth function on R 2 f (x = 1 x 2 +y

2 An important result from the last Chapter was Theorem 5 If a function f : U R where U is open in R n is of class C 2 then for all i and j 2 f x j x = 2 f i x i x j 82 The Hessian Matrix Definition 6 Given a scalar-valued C 2 -function f : U R where U is open in R n we define the Hessian matrix of f at a U to be the n n matrix Hf (a with (i,j-th entry 2 f (a/ x i x j This matrix is symmetric by Theorem 5 Recall a point a is critical if df a = 0 Definition 7 A critical point a U of f is non-degenerate when the Hessian matrix of f at a is non-singular (ie invertible Remark When a critical point is non-degenerate, the nature of the critical point is determined by the Hessian matrix To see this we consider higher directional derivatives Recall that given a C 1 -function f : U R where U is open in R n, and a unit vector v R n we have a directional derivative d v f (a = φ v(0 where φ v (t = f (a+vt In fact, by the Chain Rule d v f (a = f (a v = f x i (avi The function f has a local minimum at a if, and only if, the functions φ v (t = f (a+vt have a local minimum at t = 0 for all unit vectors v This occurs when φ v(0 > 0 for all unit vectors v Definition 8 With the above notation we call φ v(0 the second directional derivative of f in the direction v and write it as d 2 vf (a Thus f has a local minimum at a critical point a when d 2 vf (a > 0 for all unit vectors v R n Also f has a local maximum at a critical point a when d 2 vf (a < 0 for all unit vectors v R n If d 2 vf (a > 0 for some unit vector v R n and d 2 wf (a < 0 for another unit vector w R n then a is called a saddle point You can approach a along a path on which the value of f is minimal at a and along a different path on which the value of f is maximal at a 2

3 Theorem 9 If f : U R where U is open in R n is a C 2 -function then, for a U and a unit vector v R n, d 2 vf (a = j=1 2 f x j x i (avi v j = v T Hf (av Proof Exercise φ v(t = d dt = d dt = d dt = ( d dt φ v(t = d ( d dt dt f (a+vt ( ( f x (a+vt d ( a i +v i t by the Chain Rule i dt f x i (a+vtvi = v i d dt ( v i 2 f x j x i (a+vtvj, j=1 ( f x (a+vt i again by the Chain Rule The result follows on letting t 0 Definition 10 Assume A is a symmetric real matrix We say that A is positive definite iff v T Av > 0 for all non-zero vectors v, A is negative definite iff v T Av < 0 for all non-zero vectors v, A is indefinite if these exists v 0 : v T Av > 0 and there exists v 0 : v T Av < 0, otherwise A is nondefinite Note the nondefinite definition covers the cases such as v T Av 0 for all v with v T Av = 0 for some non-zero vector v Question What does it mean for a function if it s Hessian at a critical point is positive definite, negative definite or indefinite? Answer: If Hf (a is positive definite then d 2 vf (a = v T Hf (av > 0 for all unit vectors v and thus f has a local minimum at the critical point a If Hf (a is negative definite then d 2 vf (a = v T Hf (av < 0 for all unit vectors v and thus f has a local maximum at the critical point a If Hf (a is indefinite then d 2 vf (a = v T Hf (av > 0 for some unit vector v R n and d 2 wf (a = w T Hf (aw < 0 for another unit vector w R n and thus f has a saddle at the critical point a Question, Given a symmetric matrix A under what conditions is v T Av > 0 for all non-zero vectors v? When is v T Av < 0 for all non-zero vectors v? The answer is easily given if A is a 2 2 matrix

4 Theorem 11 Suppose M = ( a b b c If detm > 0 then M is positive definite if a > 0 and negative definite if a < 0 If detm < 0, then M is indefinite If detm = 0 then M is nondefinite Proof Exercise, but all results follow from ( a b (x,y b c ( ( x = a x+ by 2 + deta y a a y2 Notation To ease congestion in the writing we already have Extend this to higher derivatives by d j f (x = f x j (x d i,j f (x = 2 f x j x i (x I ve no idea why the order of the subscripts in d i,j is the reverse of the superscripts in 2 f/ x j x i but juts remember that it is so Example 12 Find the critical points of f (x = 1 x x y2 +xy +1x y +2, x R 2 and find if they are local minima, maxima or saddle points Solution f (x = ( x 2 6x+y +1, y 2 +x 1 T So we wish to solve x 2 6x+y +1 = 0 and 2x+y 2 = 0 Substitute y = 2x+2 into the first equation to get x 2 8x+15 = 0 which factorises as (x (x 5 = 0 This gives two critical points The second order partial derivatives are a 1 = (, 4 and a 2 = (5, 8 d 1,1 f (x = 2x 6, d 1,2 f (x = 1 and d 2,2 f (x = 1/2 Hence the Hessian matrix is Hf (x = ( 2x /2 4

5 Then Hf (a 1 = ( /2 and Hf (a 2 = ( /2 Since dethf (a 1 = 1 < 0 the matrix is indefinite and we have a saddle at a 1 Since dethf (a 2 = 1 > 0 and 4 > 0 the matrix is positive definite and we have a local minimum at a 2 The situation is more complicated for larger n Without proof, and only for purpose of answering questions we state some results from Linear Algebra If A is a real symmetric n n matrix then, it can be shown that, R n has an orthonormal basis of eigenvectors {v 1,v 2,,v n } satisfying Av i = λ i v i ; the λ i are the eigenvalues of A If all λ i > 0 then A is positive definite, if all λ i < 0 then A is negative definite If some λ k > 0 and some λ l < 0 then A is indefinite It may be reasonable for small n, ie n = or 4 to calculate the eigenvalues, even by hand For larger n this becomes infeasible and we don t, in fact, need the exact values of the eigenvalues, only their signs The following is a test for positive definiteness Definition 1 Let A = (a ij be a real, symmetric, n n matrix For 1 l n, form the l l matrix A l = (a ij 1 i,j l The A l are called the l l principal minors of A Proposition 14 Let A = (a ij be a real symmetric n n matrix Then A is positive definite if, and only if, deta l > 0 for all 1 l n Proof Not given (but see Appendix So if dethf (a l > 0 for all 1 l n then f has a local minimum at the critical point a Note that if M is an n n matrix then det( M = ( 1 n detm Hence Corollary 15 Then A is negative definite iff A is positive definite iff det( A l > 0, that is ( 1 l deta l > 0 for all 1 l n So if ( 1 l dethf (a l > 0 for all 1 l n then f has a local maximum at the critical point a Example Return to an earlier example, f : R R given by f (x = x 2 y 2 +z 2 +2x 4y +z Then d 1,1 f (x = y 2, d 1,2 f (x = 4xy, d 1, f (x = 0, d 2,2 f (x = x 2, d 2, f (x = 0 and d, f (x = 2 Hence Hf (x = f ( 2 2/,2 1/, 2 1 = y 2 4xy 0 4xy x

6 Itwasfoundinthatearlierexamplethatf hasonlyonecriticalpoint,a = ( 2 2/,2 1/, 2 1 T At that point Hf (a = This is non-singular and so a is a non-degenerate critical point Looking at the determinants of the principal minors ( ( det 2 > 0, det < 0 and dethf (a < 0 2 This shows that at a there is a saddle point Remarks The fundamental result for symmetric matrices is the following Theorem 16 If A is a real symmetric n n matrix then R n has an orthonormal basis of eigenvectors {v 1,v 2,,v n } satisfying Av i = λ i v i ; the λ i are the eigenvalues of A Further, there exits a matrix M such that A = MDM T where D is a diagonal matrix with the eigenvalues on the diagonal See Appendix for further details Proposition 17 Let A = (a ij be a real symmetric n n matrix A is positive definite if, and only if, all eigenvalues are positive, A is negative definite if, and only if, all eigenvalues are negative If some λ k > 0 and some λ l < 0 then A is indefinite Proof See Appendix ExampleReturntof (x = x 2 y 2 +z 2 +2x 4y+z,criticalvaluea = ( 2 2/,2 1/, 2 1 T and Hf (a = Then dethf (a < 0 But recall that the determinant of a matrix is the product of all eigenvalues So dethf (a = λ 1 λ 2 λ Then λ 1 λ 2 λ < 0 must have either all three < 0 or two > 0 and one < 0 But looking at the matrix we immediately see that 2 is an eigenvalue (with eigenvector e So we have two eigenvalues > 0 and one < 0 Hence Hf (a is indefinite and a saddle at f 6