1 The space of linear transformations from R n to R m :

Transcription

1 Math 540 Spring 20 Notes #4 Higher deriaties, Taylor s theorem The space of linear transformations from R n to R m We hae discssed linear transformations mapping R n to R m We can add sch linear transformations in the sal way (L + L 2 ) (x) = L (x) + L 2 (x). Similarly we can mltiply sch a linear transformation by a scalar. In this way, the set L (R n ; R m ) = flinear transformations from R n to R m g becomes a ector space. If we choose bases for R n and R m ; say the standard bases, then each element of L (R n ; R m ) has an m n matrix with respect to these bases. Since there are mn entries in sch a matrix, and they all can be chosen independently of each other, L (R n ; R m ) has dimension mn A basis is the set of m n matrices which are all zero except for a in one entry. 2 Second deriatie Recall that if f R n! R m is di erentiable at a point x 2 R n ; then Df (x) is a linear transformation from R n to R m Hence, for each x; Df (x) 2 L (R m ; R n ) From this we see that Df is a fnction from R n to L (R n ; R m ) We can then discss D (Df) ; or D 2 f; the second deriatie of f For each x 2 R n ; D 2 f (x) is a linear transformation from R n to L (R n ; R m ) Hence, for any 2 R n ; D 2 f (x) () 2 L (R n ; R m ) Therefore, for any w 2 R n ; D 2 f (x) () (w) 2 R m Recall that R n R n = f(; w) j and w are in R n g We can therefore consider D 2 f (x) as a linear transformation from R n R n to R m So instead of writing D 2 f (x) () (w) ; we write D 2 f (x) (; w) This linear transformation from R n to R m is called bilinear, becase it is linear as a fnction of for each xed w; and also as a fnction of w for each xed In other words, D 2 f (x) + 2 ; w + w 2 = D 2 f (x) ; w + D 2 f (x) 2 ; w + D 2 f (x) ; w 2 + D 2 f (x) 2 ; w 2

2 Now we will only consider the case m = Ths, Similarly, f R n! R For each x; Df (x) R n! R Df R n! L (R n ; R) For each x D 2 f (x) R n! L (R n ; R) Eqialently, D 2 f (x) R n R n! R () We wish to consider the natre of a general bilinear fnction L from R n R n to R Let e ; ; e n be the standard basis ectors of R n Then for each i and j; L (e i ; e j ) 2 R Let L (e i ; e j ) = a ij It will be simplest now to consider the case n = 2 Sppose that = c e + c 2 e 2 ; and w = d e + d 2 e 2 The bilinearity implies that L (; w) = L (c e + c 2 e 2 ; d e + d 2 e 2 ) = c d a + c d 2 a 2 + c 2 d a 2 + c 2 d 2 a 22 It trns ot that this eqals a a (c ; c 2 ) 2 a 2 a 22 d d 2 (Check by mltiplying this ot.) In this way, each L is associated with an n n matrix A In the case where L = D 2 f (x) ; it is shown in the text 2 f A = j If yo recall that for most fnctions, the order in which yo take partial deriaties doesn t matter, yo see that nder some assmptions on f; A is a symmetric matrix. Theorem in the text says that this will be tre if all of the second partial deriaties of f are continos. Example Let f (x; y) = x 2 y+xy 3 We will nd the standard matrix for D 2 f (; ) ; and check that the limit formla for deriatie works. For this fnction we hae Df (x; y) = 2xy + y 3 ; x 2 + 3xy 2 2

3 By this we mean that Also, Df (x; y) = D 2 f (x; y) has the matrix (2xy + y 3 ) (x 2 + 3xy 2 ) 2y 2x + 3y 2 2x + 3y 2 6xy (Notice f f ) D2 f (x; y) R n! L (R n ; R) Therefore, D 2 f (x) mst be a map from R n to R We saw sch a map before Df (x) maps R n to R Any element of L (R n ; R) can be written in the form x! bx where b is a n matrix; that is, a row ector. And any linear map L from R n to fall n-dimensional row ectorsg can be written asy! y T A for some n n matrix A It is shown in the text that if L = D 2 f (x) ; then A is the matrix of second partial deriaties of f; called the Hessian. This leads s to the eqation D 2 f (x; y) From this we get D 2 f (x; y) Hence, D 2 f (; ) p q p q = (; ) = (; ) 2 5 = (; ) 5 6 2y 2x + 3y 2 2x + 3y 2 6xy 2y 2x + 3y 2 2x + 3y 2 6xy p q p q = 2p + 5q + 5p We now check this last formla sing the de nition of deriatie. Howeer, it is a bit complicated to describe jst what is meant by the norm of a linear operator. It trns ot to be eqialent to discss the corresponding matrices. Once again the sp norm will be conenient. We wish to check that 2 5 Df (x; y) Df (; ) (x ; y ) 5 6 lim (x;y)!(;) x = 0 y 3

4 (Notice that in the nmerator we are dealing with row ectors.) We obtain 2xy + y 3 ; x 2 + 3xy 2 (3; 4) (2 (x ) + 5 (y ) ; 5 (x ) + 6 (y )) It is s cient to show that the ratio of the absolte ale of each component of the ector in this expression to the norm in the denominator tends to zero as (x; y)! (; ) The rst component is y 3 + 2xy 2x 5y + 4 A little algebra is necessary Since 2x = 2 (x ) + 2 and 5y = 5 (y ) + 5; we hae y 3 + (2 (x ) + 2) y 2x 5 (y ) = y 3 3y (x ) (2y 2) Frther, it trns ot that y 3 3y + 2 = (y ) 2 (y + 2) Hence if (x; y) 6= (; ) then the ratio of the absolte ale of the rst component of the nmerator to the denominator is (y ) 2 (y + 2) + 2 (x ) (y ) max fjx j ; jy jg ( (x ) 2 jy+2j+2jx j 2 jx j if jx j > jy j (y ) 2 [y+2]+2jy j 2 jy j if jx j jy j Both alternaties on the right tend to zero as as (x; y)! (; ) The second component can be handled similarly. It wold be a nice algebra exercise to do this. 3 Third deriatie Notice the pattern f R n! R; and for each x 2 R n (where f is di erentiable), Df (x) R n! R In other words, Df (x) 2 L (R n ; R) The linear transformation Df (x) has the standard matrix ( n) gien by the gradient, which is in R n Ths, Df R n! R n Df is not sally a linear transformation. As we explained, D 2 f (x) is a linear transformation from R n R n to R; and this linear transformation has the standard n n matrix gien aboe. Therefore, D 2 f R n! L (R n R n ; R) Hence, we expect that for each x; D 3 f (x) R n! L (R n R ; R) This will inole the third deriaties 3 k We will consider this frther below. First, we hae a reiew of Taylor series in one ariable. 4 Taylor series for f R! R First recall the general formla for a Taylor series in one ariable. Sppose that f R! R; and all deriaties of f exist at eery x 2 R If x 0 2 R; then the Taylor 4

5 series for f at x 0 is n=0 n! f (n) (x 0 ) (x x 0 ) n (2) Here f (n) is the n-th deriatie of f We hae the sal conentions that 0! = and f (0) = f This series may conerge for all x; or only for x in some interal containing x 0 (It obiosly conerges if x = x 0 ) And if it conerges for some x 6= x 0 ; it might not conerge to f (x) Examples of these possibilities will be gien in class. De nition If the series (2) conerges to f (x) in some neighborhood of x 0 ; then f is called analytic at x 0 Perhaps of een more importance is sing a nite sm of the terms in the Taylor series to approximate f on some interal containing x 0 This can sometimes be done een if f is not analytic at x 0 ; perhaps becase not all of the deriaties of f at x 0 are de ned. The theorem which allows s to gie sch approximations is called Taylor s theorem. To state Taylor s theorem we rst need a de nition. De nition 2 Sppose that f I R! R; where I is an open interal containing a point x 0 Sppose that r is a nonnegatie integer. We say that f is of class C r on I if the rst r deriaties, f; f 0, f 00,..., f (r) exist and are continos on I Theorem 3 Sppose that f I R! R where I is an open interal containing a point x 0 ; and f is of class C r on I Sppose that x and y are in I Then there is a c between x and y sch that f (y) f (x) = r n= n! f (n) (x) (y x) n + r! f (r) (c) (y x) r If r = ; then this is the mean ale theorem. As an example, let f (x) = jxj 5=2 ; and consider f (2) f ( ) Notice that f 0 (0) = f 00 (0) = 0; bt f 000 (0) doesn t exist. Also, f (x) = ( x) 5=2 if x < 0 We wish to nd c 2 ( ; 2) sch that f (2) f ( ) = f 0 ( ) (2 ( )) + 2 f 00 (c) (2 ( )) 2 2 5=2 = 5 2 ( ) (3) + 2 f 00 (c) (9) 5

6 If c > 0; then f 00 (c) = c=2 ; while if d = c; then f (d) = 2 2 ( d) =2 = f 00 (c) ; so f 00 is an een fnction. Therefore we can assme c > 0; and we want or 2 5= = c 2 ; p 8 c = 2 5= Since we assmed that c > 0; we hae to check that c < 2 That is easily done. p c < (8 + 7) = 5 5 Taylor s theorem of order 2 and qadratic forms. As pointed ot earlier, the mean ale theorem is a special case of Taylor s Theorem. Using the formla on page 353, in Theorem 6.3., we see that if f is di erentiable at eery x; then for any x 0 and x there is a c between x 0 and x sch that f (x) = f (x 0 ) + Df (c) (x x 0 ) Recall that Df (c) is a row ector, (the gradient). Extending by one more term, Theorem gies f (x) = f (x 0 ) + Df (x 0 ) (x x 0 ) + 2 D2 f (c) (x x 0 ; x x 0 ) (3) We need to explain the last term. From the theory aboe, we see that if we write x x 0 as a colmn ector, then the last term is of the form and A is the n n matrix (x x 0 ) T A (x x 0 ) 2 j Writing this ot, we hae the expression D 2 f (c) (x x 0 ; x x 0 ) = ((x x 0 ) ; (x x 0 ) 2 ; ; (x x 0 ) n ) A 0 (x x 0 ) (x x 0 ) 2 (x x 0 ) n C A (5) = n i;j=a ij (x x 0 ) i (x x 0 ) j (6) 6

7 Let s look again at n = 2 Let x x 0 = for scalars and Then the expression in (5) becomes Bt A is symmetric, so we get a 2 + a 2 + a 2 + a 22 2 a 2 + 2a 2 + a 22 2 Sch an expression is called a qadratic form. In the n dimensional case with (x x 0 ) = ; we get a 2 + a a nn 2 n + 2a a a (n )n n n ; which is again called a qadratic form. One of the chief qestions one asks abot a qadratic form is whether it is positie wheneer 6= 0 In that case it is called a positie de nite qadratic form. One major reason that qestion is important is its application in the next section of the text to maxima and minima of fnctions f R n R 6 Taylor s theorem This is Theorem 6.8.5, which was referred to aboe. The proof is somewhat complicated, and the longest proof in either Chapter 5 or Chapter 6. I will be content here to carry the expansion ot one more term than in (3), ths adding a third deriatie, and discssing the reslting expression. It is f (x) = f (x 0 )+Df (x 0 ) (x x 0 )+ 2 D2 f (x 0 ) (x x 0 ; x x 0 )+ 3! D3 f (c) (x x 0 ; x x 0 ; x x 0 ) The last term with D 3 f (c) ; is called the remainder term. Here c is a point on the line segment between x 0 and x Yo can tell from the last term that D 3 f (c) R n R n R n! R There are 3 third deriaties, 3 k (c) Ths, they won t t into a sqare matrix. We will 7

8 denote these deriaties by f abc (c) where a; b; and c are each one of xor y The third x deriatie term when n = 2, and x is ; trns ot to be y f xxx (c) (x x 0 ) 3 + 3f xxy (c) (x x 0 ) 2 (y y 0 ) + 3f xyy (c) (x x 0 ) (y y 0 ) 2 + f yyy (c) (y y 0 ) 3 3! (7) Can yo see what the third order deriatie wold be when n = 3? What abot the forth deriatie term for n = 2 and n = 3? 7 Maxima and Minima 7. Positie de nite qadratic forms. A qadratic form is a fnction Q R n! R of the form Q () = T A for some symmetric n n matrix A The releance of this to Taylor s Theorem is seen by looking at eqations (3) and (4). De nition 4 A symmetric matrix A is called positie de nite if Q () > 0 for eery 6= 0 in R n There are two particlarly sefl criteria for determining of A is positie de nite. These are from linear algebra, and won t be proed here. Theorem 5 A symmetric nn matrix A is positie de nite if either of the following conditions holds (i) All eigenales of A are positie (ii) All n pper left sbdeterminants of A are positie. An pper left sbdeterminant is one formed by deleting between zero and n of the last rows and colmns of A= This will be illstrated in class. If A is a symmetric matrix and A is positie de nite, then A is called negatie de nite. 8

9 7.2 Application to maxima and minima The eqation (3) aboe allows s to determine criteria garanteeing that a point x 0 is a local maximm or local minimm for the fnction f To apply it, we mst assme that f 2 C 2 There cannot be a local maximm at x 0 nless Df (x 0 ) = 0; for otherwise there is a nonzero directional deriatie in some direction e; which means that d dt f (x 0 + te) j t=0 6= 0; and so there are larger ales of f either for t positie or t negatie, and jtj small. De nition 6 x 0 is called a critical point of f if Df (x 0 ) = 0 We then repeat eqation (3) f (x) = f (x 0 ) + Df (x 0 ) (x x 0 ) + D 2 f (c) (x x 0 ; x x 0 ) Assming that Df (x 0 ) = 0, we get f (x) = f (x 0 ) + (x x 0 ) T D 2 f (c) (x x 0 ) (8) Theorem 7 If x 0 is a critical point of f and the matrix corresponding to D 2 f (x 0 ) is positie de nite, then x 0 is a local minimm for f If D 2 f (x 0 ) is negatie de nite, then x 0 is a local maximm. Proof. Sppose that x 0 is a critical point of f and A = D 2 f (x 0 ) is positie de nite. Then e T Ae > 0 for eery nit ector e 2 R n If is a nonzero ector in R n ; then e = is a nit ector. It follows that A is positie de nite if and only if e T Ae > 0 jjjj for eery nit ector e Bt the set of all nit ectors in A is a compact set. Hence, = min e T Ae > 0 jjej=j Becase D 2 f (x) is continos, it follows that there is a sch that if jjc xjj < ; then e T D 2 f (c) e > > 0 for eery nit ector e Hence 2 D2 f is also positie de nite. (i.e. the symmetric matrix corresponding to D 2 f (c) is positie de nite.) If jjx x 0 jj < then jjc x 0 jj <, becase c is on the line segment between x and x 0 Eqation (8) then implies that if 0 < jjx x 0 jj <, then f (x) > f (x 0 ) Hence x 0 is a local minimm for f The case of a minimm is similar. 9

10 8 Homework, de Feb. 2 at the beginning of class. Use the formla in the middle of page 359 to write ot completely the terms inoling second deriaties in the Taylor series arond x 0 = 0 of a fnction f R 3! R (That is, gie the expression corresponding to (7) aboe, which is the third deriatie term for a fnction from R 2 to R ) Then write ot completely the terms in the Taylor series arond 0 inoling the third deriatie for a fnction g R 3! R 2. pg. 386, # 9, c, 3. pg 386, #9 f # 7 b. 5. pg. 384, # 7 c, The answer is in the back of the book (page 708), bt yo need to show the calclations needed to get the answer. Referring to the answer in the book, yo need only consider the points where k = 0 and m = 0; n = j = 0; and n = ; j = 0 0