Several variables and partial derivatives

Transcription

1 Chapter 8 Several variables ad partial derivatives 8.1 Vector spaces, liear mappigs, ad covexity Note:??? lectures Vector spaces The euclidea space R has already made a appearace i the metric space chapter. I this chapter, we will exted the differetial calculus we created for oe variable to several variables. The key idea i differetial calculus is to approximate fuctios by lies ad liear fuctios. I several variables we must itroduce a little bit of liear algebra before we ca move o. So let us start with vector spaces ad liear fuctios o vector spaces. While it is commo to use x or the bold x for elemets of R, especially i the applied scieces, we use just plai x, which is commo i mathematics. That is, v R is a vector, which meas v = (v 1,v 2,...,v ) is a -tuple of real umbers. It is commo to write vectors as colum vectors, that is, 1 matrices: v = (v 1,v 2,...,v ) = v 1 v 2.. v We will do so whe coveiet. We call real umbers scalars to distiguish them from vectors. The set R has a so-called vector space structure defied o it. However, eve though we will be lookig at fuctios defied o R, ot all spaces we wish to deal with are equal to R. Therefore, let us defie the abstract otio of the vector space. Subscripts are used for may purposes, so sometimes we may have several vectors which may also be idetified by subscript such as a fiite or ifiite sequece of vectors y 1,y 2,... 7

2 8 CHAPTER 8. SEVERAL VARIABLES AND PARTIAL DERIVATIVES Defiitio Let X be a set together with operatios of additio, +: X X X, ad multiplicatio, : R X X, (we usually write ax istead of a x). X is called a vector space (or a real vector space) if the followig coditios are satisfied: (i) (Additio is associative) If u,v,w X, the u + (v + w) = (u + v) + w. (ii) (Additio is commutative) If u,v X, the u + v = v + u. (iii) (Additive idetity) There is a 0 X such that v + 0 = v for all v X. (iv) (Additive iverse) For every v X, there is a v X, such that v + ( v) = 0. (v) (Distributive law) (vi) (Distributive law) (vii) (Multiplicatio is associative) If a R, u,v X, the a(u + v) = au + av. If a,b R, v X, the (a + b)v = av + bv. (viii) (Multiplicative idetity) 1v = v for all v X. If a, b R, v X, the (ab)v = a(bv). Elemets of a vector space are usually called vectors, eve if they are ot elemets of R (vectors i the traditioal sese). If Y X is a subset that is a vector space itself with the same operatios, the Y is called a subspace or vector subspace of X. Example 8.1.2: A example vector space is R, where additio ad multiplicatio by a costat is doe compoetwise: if a R ad x,y R, the x + y := (x 1,x 2,...,x ) + (y 1,y 2,...,y ) = (x 1 + y 1,x 2 + y 2,...,x + y ), ax := a(x 1,x 2,...,x ) = (ax 1,ax 2,...,ax ). I this book we mostly deal with vector spaces that ca be ofte regarded as subsets of R, but there are other vector spaces useful i aalysis. Let us give a couple of examples. Example 8.1.3: A trivial example of a vector space (the smallest oe i fact) is just X = {0}. The operatios are defied i the obvious way. You always eed a zero vector to exist, so all vector spaces are oempty sets. Example 8.1.4: The space C([0,1],R) of cotiuous fuctios o the iterval [0,1] is a vector space. For two fuctios f ad g i C([0,1],R) ad a R we make the obvious defiitios of f + g ad a f : ( f + g)(x) := f (x) + g(x), (a f )(x) := a ( f (x) ). The 0 is the fuctio that is idetically zero. We leave it as a exercise to check that all the vector space coditios are satisfied. Example 8.1.5: The space of polyomials c 0 + c 1 t + c 2 t c m t m is a vector space, let us deote it by R[t] (coefficiets are real ad the variable is t). The operatios ca be defied i the same way as for fuctios above. Agai, a very easy exercise is to verify that the operatios satisfy

3 8.1. VECTOR SPACES, LINEAR MAPPINGS, AND CONVEXITY 9 the vector space structure. Furthermore, suppose we have two polyomials, oe of degree m ad oe of degree. Suppose that m for simplicity. The reader should verify that the vector space operatios satisfy ad (c 0 + c 1 t + c 2 t c m t m ) + (d 0 + d 1 t + d 2 t d t ) = (c 0 + d 0 ) + (c 1 + d 1 )t + (c 2 + d 2 )t (c m + d m )t m + d m+1 t m d t a(c 0 + c 1 t + c 2 t c m t m ) = (ac 0 ) + (ac 1 )t + (ac 2 )t (ac m )t m. Despite what it looks like, R[t] is ot equivalet to R for ay. I particular it is ot fiite dimesioal, we will make this otio precise i just a little bit. Oe ca make a fiite dimesioal vector subspace by restrictig the degree. For example, if we say P is the space of polyomials of degree or less, the we have a fiite dimesioal vector space. The space R[t] ca be thought of as a subspace of C(R,R). If we restrict the rage of t to [0,1], R[t] ca be idetified with a subspace of C([0,1],R). It is ofte better to thik of eve simpler fiite dimesioal vector spaces usig the abstract otio rather tha always R. It is possible to use other fields tha R i the defiitio (for example it is commo to use the complex umbers C), but let us stick with the real umbers. A fuctio f : X Y, whe Y is ot R, is ofte called a mappig or a map rather tha a fuctio Liear combiatios ad dimesio Defiitio Suppose X is a vector space, x 1,x 2,...,x k X are vectors, ad a 1,a 2,...,a k R are scalars. The a 1 x 1 + a 2 x a k x k is called a liear combiatio of the vectors x 1,x 2,...,x k. If Y X is a set the the spa of Y, or i otatio spa(y ), is the set of all liear combiatios of some fiite umber of elemets of Y. We also say Y spas spa(y ). Example 8.1.7: Let Y := {(1,1)} R 2. The spa(y ) = {(x,x) R 2 : x R}. That is, spa(y ) is the lie through the origi ad the poit (1,1). Example 8.1.8: Let Y := {(1,1),(0,1)} R 2. The spa(y ) = R 2, as ay poit (x,y) R 2 ca be writte as a liear combiatio (x,y) = x(1,1) + (y x)(0,1). If you wat a very fuky vector space over a differet field, R itself is a vector space over the ratioal umbers.

4 10 CHAPTER 8. SEVERAL VARIABLES AND PARTIAL DERIVATIVES A sum of two liear combiatios is agai a liear combiatio, ad a scalar multiple of a liear combiatio is a liear combiatio, which proves the followig propositio. Propositio Let X be a vector space. For ay Y X, the set spa(y ) is a vector space itself. If Y is already a vector space the spa(y ) = Y. Defiitio A set of vectors {x 1,x 2,...,x k } X is liearly idepedet, if the oly solutio to a 1 x 1 + a 2 x a k x k = 0 (8.1) is the trivial solutio a 1 = a 2 = = a k = 0. A set that is ot liearly idepedet, is liearly depedet. A liearly idepedet set B of vectors such that spa(b) = X is called a basis of X. For example the set Y of the two vectors i Example is a basis of R 2. If a vector space X cotais a liearly idepedet set of d vectors, but o liearly idepedet set of d + 1 vectors the we say the dimesio or dim X := d. If for all d N the vector space X cotais a set of d liearly idepedet vectors, we say X is ifiite dimesioal ad write dim X :=. Clearly for the trivial vector space, dim{0} = 0. We will see i a momet that ay vector space that is a subset of R has a fiite dimesio, ad that dimesio is less tha or equal to. If a set is liearly depedet, the oe of the vectors is a liear combiatio of the others. I other words, i (8.1) if a j 0, the we ca solve for x j x j = a 1 a j x a j 1 a j x j 1 + a j+1 a j x j a k a k x k. Clearly the the vector x j has at least two differet represetatios as liear combiatios of {x 1,x 2,...,x k }. Propositio If B = {x 1,x 2,...,x k } is a basis of a vector space X, the every poit y X has a uique represetatio of the form for some scalars a 1,a 2,...,a k. y = k a j x j Proof. Every y X is a liear combiatio of elemets of B sice X is the spa of B. For uiqueess suppose k k y = a j x j = b j x j,

5 8.1. VECTOR SPACES, LINEAR MAPPINGS, AND CONVEXITY 11 the k (a j b j )x j = 0. By liear idepedece of the basis a j = b j for all j. For R we defie e 1 := (1,0,0,...,0), e 2 := (0,1,0,...,0),..., e := (0,0,0,...,1), ad call this the stadard basis of R. We use the same letters e j for ay R, ad which space R we are workig i is uderstood from cotext. A direct computatio shows that {e 1,e 2,...,e } is really a basis of R ; it is easy to show that it spas R ad is liearly idepedet. I fact, x = (x 1,x 2,...,x ) = Propositio Let X be a vector space. (i) If X is spaed by d vectors, the dim X d. x j e j. (ii) dim X = d if ad oly if X has a basis of d vectors (ad so every basis has d vectors). (iii) I particular, dim R =. (iv) If Y X is a vector space ad dim X = d, the dim Y d. (v) If dim X = d ad a set T of d vectors spas X, the T is liearly idepedet. (vi) If dim X = d ad a set T of m vectors is liearly idepedet, the there is a set S of d m vectors such that T S is a basis of X. Proof. Let us start with (i). Suppose S = {x 1,x 2,...,x d } spas X, ad T = {y 1,y 2,...,y m } is a set of liearly idepedet vectors of X. We wish to show that m d. Write y 1 = d a k,1 x k, k=1 for some umbers a 1,1,a 2,1,...,a d,1, which we ca do as S spas X. Oe of the a k,1 is ozero (otherwise y 1 would be zero), so suppose without loss of geerality that this is a 1,1. The we ca solve x 1 = 1 y 1 a 1,1 d k=2 a k,1 a 1,1 x k. I particular {y 1,x 2,...,x d } spa X, sice x 1 ca be obtaied from {y 1,x 2,...,x d }. Next, y 2 = a 1,2 y 1 + d k=2 a k,2 x k.

6 12 CHAPTER 8. SEVERAL VARIABLES AND PARTIAL DERIVATIVES As T is liearly idepedet, we must have that oe of the a k,2 for k 2 must be ozero. Without loss of geerality suppose a 2,2 0. Proceed to solve for x 2 = 1 a 2,2 y 2 a 1,2 a 2,2 y 1 d k=3 a k,2 a 2,2 x k. I particular {y 1,y 2,x 3,...,x d } spas X. The astute reader will thik back to liear algebra ad otice that we are row-reducig a matrix. We cotiue this procedure. If m < d, the we are doe. So suppose m d. After d steps we obtai that {y 1,y 2,...,y d } spas X. Ay other vector v i X is a liear combiatio of {y 1,y 2,...,y d }, ad hece caot be i T as T is liearly idepedet. So m = d. Let us look at (ii). First otice that if we have a set T of k liearly idepedet vectors that do ot spa X, the we ca always choose a vector v X \ spa(t ). The set T {v} is liearly idepedet (exercise). If dim X = d, the there must exist some liearly idepedet set of d vectors T, ad it must spa X, otherwise we could choose a larger set of liearly idepedet vectors. So we have a basis of d vectors. O the other had if we have a basis of d vectors, it is liearly idepedet ad spas X. By (i) we kow there is o set of d + 1 liearly idepedet vectors, so dimesio must be d. For (iii) otice that {e 1,e 2,...,e } is a basis of R. To see (iv), suppose Y is a vector space ad Y X, where dim X = d. As X caot cotai d + 1 liearly idepedet vectors, either ca Y. For (v) suppose T is a set of m vectors that is liearly depedet ad spas X. The oe of the vectors is a liear combiatio of the others. Therefore if we remove it from T we obtai a set of m 1 vectors that still spa X ad hece dim X m 1. For (vi) suppose T = {x 1,...,x m } is a liearly idepedet set. We follow the procedure above i the proof of (ii) to keep addig vectors while keepig the set liearly idepedet. As the dimesio is d we ca add a vector exactly d m times Liear mappigs Defiitio A mappig A: X Y of vector spaces X ad Y is liear (or a liear trasformatio) if for every a R ad x,y X we have A(ax) = aa(x) A(x + y) = A(x) + A(y). We usually write Ax istead of A(x) if A is liear. If A is oe-to-oe a oto the we say A is ivertible ad we deote the iverse by A 1. If A: X X is liear the we say A is a liear operator o X. We write L(X,Y ) for the set of all liear trasformatios from X to Y, ad just L(X) for the set of liear operators o X. If a,b R ad A,B L(X,Y ), defie the trasformatio aa + bb (aa + bb)(x) = aax + bbx.

7 8.1. VECTOR SPACES, LINEAR MAPPINGS, AND CONVEXITY 13 If A L(Y,Z) ad B L(X,Y ), defie the trasformatio AB as ABx := A(Bx). Fially deote by I L(X) the idetity: the liear operator such that Ix = x for all x. It is ot hard to see that aa + bb L(X,Y ), ad that AB L(X, Z). I particular, L(X,Y ) is a vector space. It is obvious that if A is liear the A0 = 0. Propositio If A: X Y is ivertible, the A 1 is liear. Proof. Let a R ad y Y. As A is oto, the there is a x such that y = Ax, ad further as it is also oe-to-oe A 1 (Az) = z for all z X. So A 1 (ay) = A 1 (aax) = A 1( A(ax) ) = ax = aa 1 (y). Similarly let y 1,y 2 Y, ad x 1,x 2 X such that Ax 1 = y 1 ad Ax 2 = y 2, the A 1 (y 1 + y 2 ) = A 1 (Ax 1 + Ax 2 ) = A 1( A(x 1 + x 2 ) ) = x 1 + x 2 = A 1 (y 1 ) + A 1 (y 2 ). Propositio If A: X Y is liear the it is completely determied by its values o a basis of X. Furthermore, if B is a basis, the ay fuctio Ã: B Y exteds to a liear fuctio o X. We will oly prove this propositio for fiite dimesioal spaces, as we do ot eed ifiite dimesioal spaces. For ifiite dimesioal spaces, the proof is essetially the same, but a little trickier to write, so let us stick with fiitely may dimesios. Proof. Let {x 1,x 2,...,x } be a basis ad suppose A(x j ) = y j. The every x X has a uique represetatio x = b j x j for some umbers b 1,b 2,...,b. The by liearity Ax = A b j x j = b j Ax j = b j y j. The furthermore follows by defiig the extesio Ax = b jy j, ad otig that this is well defied by uiqueess of the represetatio of x. The ext propositio oly works for fiite dimesioal vector spaces. It is a special case of the so called rak-ullity theorem from liear algebra. Propositio If X is a fiite dimesioal vector space ad A: X X is liear, the A is oe-to-oe if ad oly if it is oto.

8 14 CHAPTER 8. SEVERAL VARIABLES AND PARTIAL DERIVATIVES Proof. Let {x 1,x 2,...,x } be a basis for X. Suppose A is oe-to-oe. Now suppose c j Ax j = A c j x j = 0. As A is oe-to-oe, the oly vector that is take to 0 is 0 itself. Hece, 0 = c j x j ad so c j = 0 for all j. Therefore, {Ax 1,Ax 2,...,Ax } is liearly idepedet. By a above propositio ad the fact that the dimesio is, we have that {Ax 1,Ax 2,...,Ax } spa X. As ay poit x X ca be writte as x = a j Ax j = A a j x j, so A is oto. Now suppose A is oto. As A is determied by the actio o the basis we see that every elemet of X has to be i the spa of {Ax 1,...,Ax }. Suppose A c j x j = c j Ax j = 0. By the same propositio as {Ax 1,Ax 2,...,Ax } spa X, the set is idepedet, ad hece c j = 0 for all j. This meas that A is oe-to-oe. If Ax = Ay, the A(x y) = 0 ad so x = y. We leave the proof of the ext propositio as a exercise. Propositio If X ad Y are fiite dimesioal vector spaces, the L(X,Y ) is also fiite dimesioal Covexity A subset U of a vector space is covex if wheever x,y U, the lie segmet from x to y lies i U. That is, if the covex combiatio (1 t)x +ty is i U for all t [0,1]. See Figure 8.1. Note that i R, every coected iterval is covex. I R 2 (or higher dimesios) there are lots of ocovex coected sets. For example the set R 2 \ {0} is ot covex but it is coected. To see this simply take ay x R 2 \ {0} ad let y := x. The (1/2)x + (1/2)y = 0, which is ot i the set. O the other had, the ball B(x,r) R (usig the stadard metric o R ) is always covex by the triagle iequality. Exercise 8.1.1: Show that i R ay ball B(x,r) for x R ad r > 0 is covex.

9 8.1. VECTOR SPACES, LINEAR MAPPINGS, AND CONVEXITY 15 U y x (1 t)x +ty Figure 8.1: Covexity. Example : Ay subspace V of a vector space X is covex. Example : A somewhat more complicated example is give by the followig. Let C([0, 1], R) be the vector space of cotiuous real valued fuctios o R. Let X C([0,1],R) be the set of those f such 1 f (x) dx 1 ad f (x) 0 for all x [0,1]. 0 The X is covex. Take t [0,1] ad ote that if f,g X the t f (x) + (1 t)g(x) 0 for all x. Furthermore 1 ( ) 1 1 t f (x) + (1 t)g(x) dx = t f (x) dx + (1 t) g(x) dx Note that X is ot a subspace of C([0,1],R). Propositio The itersectio two covex sets is covex. I fact, If {C λ } λ I is a arbitrary collectio of covex sets, the C := λ IC λ is covex. Proof. The proof is easy. If x,y C, the x,y C λ for all λ I, ad hece if t [0,1], the tx + (1 t)y C λ for all λ I. Therefore tx + (1 t)y C ad C is covex. Propositio Let T : V W be a liear mappig betwee two vector spaces ad let C V be a covex set. The T (C) is covex. Proof. Take ay two poits p,q T (C). The pick x,y C such that T (x) = p ad T (y) = q. As C is covex the for all t [0,1] we have tx + (1 t)y C, so is i T (C). T ( tx + (1 t)y ) = tt (x) + (1 t)t (y) = t p + (1 t)q

10 16 CHAPTER 8. SEVERAL VARIABLES AND PARTIAL DERIVATIVES For completeess, let us A very useful costructio is the covex hull. Give ay set S V of a vector space, defie the covex hull of S, by co(s) := {C V : S C, ad C is covex}. That is, the covex hull is the smallest covex set cotaiig S. Note that by a propositio above, the itersectio of covex sets is covex ad hece, the covex hull is covex. Example : The covex hull of 0 ad 1 i R is [0,1]. Proof: Ay covex set cotaiig 0 ad 1 must cotai [0,1]. The set [0,1] is covex, therefore it must be the covex hull Exercises Exercise 8.1.2: Verify that R is a vector space. Exercise 8.1.3: Let X be a vector space. Prove that a fiite set of vectors {x 1,...,x } X is liearly idepedet if ad oly if for every j = 1, 2,..., spa({x 1,...,x j 1,x j+1,...,x }) spa({x 1,...,x }). That is, the spa of the set with oe vector removed is strictly smaller. Exercise (Challegig): Prove that C([0, 1], R) is a ifiite dimesioal vector space where the operatios are defied i the obvious way: s = f + g ad m = f g are defied as s(x) := f (x) + g(x) ad m(x) := f (x)g(x). Hit: for the dimesio, thik of fuctios that are oly ozero o the iterval (1/+1, 1/). Exercise 8.1.5: Let k : [0,1] 2 R be cotiuous. Show that L: C([0,1],R) C([0,1],R) defied by 1 L f (y) := k(x,y) f (x) dx 0 is a liear operator. That is, show that L is well defied (that L f is cotiuous), ad that L is liear. Exercise 8.1.6: Let P be the vector space of polyomials i oe variable of degree or less. Show that P is a vector space of dimesio + 1. Exercise 8.1.7: Let R[t] be the vector space of polyomials i oe variable t. Let D: R[t] R[t] be the derivative operator (derivative i t). Show that D is a liear operator. Exercise 8.1.8: Let us show that Propositio oly works i fiite dimesios. Take R[t] ad defie the operator A: R[t] R[t] by A ( P(t) ) = tp(t). Show that A is liear ad oe-to-oe, but show that it is ot oto. Exercise 8.1.9: Prove Propositio Hit: A liear operator is determied by its actio o a basis. So give two bases {x 1,...,x } ad {y 1,...,y m } for X ad Y respectively, cosider the liear operators A jk that sed A jk x j = y k, ad A jk x l = 0 if l j.

11 8.1. VECTOR SPACES, LINEAR MAPPINGS, AND CONVEXITY 17 Exercise (Easy): Suppose X ad Y are vector spaces ad A L(X,Y ) is a liear operator. a) Show that the ullspace N := {x X : Ax = 0} is a vectorspace. b) Show that the rage R := {y Y : Ax = y for some x X} is a vectorspace. Exercise (Easy): Show by example that a uio of covex sets eed ot be covex. Exercise : Compute the covex hull of the set of 3 poits {(0,0),(0,1),(1,1)} i R 2. Exercise : Show that the set {(x,y) R 2 : y > x 2 } is a covex set. Exercise : Show that every covex set i R is coected usig the stadard topology o R. Exercise : Suppose K R 2 is a set such that the oly poit of the form (x,0) i K is the poit (0,0). Further suppose that there (0,1) K ad (1,1) K. The show that if (x,y) K the y > 0 uless x = 0.

12 18 CHAPTER 8. SEVERAL VARIABLES AND PARTIAL DERIVATIVES 8.2 Aalysis with vector spaces Note:??? lectures Norms Let us start measurig distace. Defiitio If X is a vector space, the we say a real valued fuctio is a orm if: (i) x 0, with x = 0 if ad oly if x = 0. (ii) cx = c x for all c R ad x X. (iii) x + y x + y for all x,y X (Triagle iequality). Before defiig the stadard orm o R, let us defie the stadard scalar dot product o R. For two vectors if x = (x 1,x 2,...,x ) R ad y = (y 1,y 2,...,y ) R, defie x y := x j y j. It is easy to see that the dot product is liear i each variable separately, that is, it is a liear mappig whe you keep oe of the variables costat. The Euclidea orm is the defied as x := x x = (x 1 ) 2 + (x 2 ) (x ) 2. It is easy to see that the Euclidea orm satisfies (i) ad (ii). To prove that (iii) holds, the key key iequality i the so-called Cauchy-Schwarz iequality that we have see before. As this iequality is so importat let us restate ad reprove it usig the otatio of this chapter. Theorem (Cauchy-Schwarz iequality). Let x,y R, the x y x y = x x y y, with equality if ad oly if the vectors are scalar multiples of each other. Proof. If x = 0 or y = 0, the the theorem holds trivially. So assume x 0 ad y 0. If x is a scalar multiple of y, that is x = λy for some λ R, the the theorem holds with equality: Next take x +ty, λy y = λ y y = λ y 2 = λy y. x +ty 2 = (x +ty) (x +ty) = x x + x ty +ty x +ty ty = x 2 + 2t(x y) +t 2 y 2.

13 8.2. ANALYSIS WITH VECTOR SPACES 19 If x is ot a scalar multiple of y, the x +ty 2 > 0 for all t. So the above polyomial i t is ever zero. From elemetary algebra it follows that the discrimiat must be egative: or i other words (x y) 2 < x 2 y 2. 4(x y) 2 4 x 2 y 2 < 0, Item (iii), the triagle iequality, follows via a simple computatio: x + y 2 = x x + y y + 2(x y) x 2 + y 2 + 2( x + y ) = ( x + y ) 2. The distace d(x,y) := x y is the stadard distace fuctio o R that we used whe we talked about metric spaces. I fact, o ay vector space X, oce we have a orm (ay orm), we defie a distace d(x,y) := x y that makes X ito a metric space (a easy exercise). Defiitio Let A L(X,Y ). Defie A := sup{ Ax : x X with x = 1}. The umber A is called the operator orm. We will see below that ideed it is a orm (at least for fiite dimesioal spaces). By liearity we get This implies that Ax A = sup{ Ax : x X with x = 1} = sup x X x. x 0 Ax A x. It is ot hard to see from the defiitio that A = 0 if ad oly if A = 0, that is, if A takes every vector to the zero vector. It is also ot difficult to see the orm of the idetity operator: Ix I = sup x X x 0 x = sup x X x 0 x x = 1. For fiite dimesioal spaces A is always fiite as we prove below. This also implies that A is cotiuous. For ifiite dimesioal spaces either statemet eeds to be true. For a simple example, take the vector space of cotiuously differetiable fuctios o [0,1] ad as the orm use the uiform orm. The fuctios si(x) have orm 1, but the derivatives have orm. So differetiatio (which is a liear operator) has ubouded orm o this space. But let us stick to fiite dimesioal spaces ow.

14 20 CHAPTER 8. SEVERAL VARIABLES AND PARTIAL DERIVATIVES Whe we talk about fiite dimesioal vector space, it is usually fie to just thik R, although if we have a orm, the orm might perhaps ot be the stadard euclidea orm. I the exercises, you ca prove that every orm is i some sese equivalet to the euclidea orm i that the topology it geerates is the same. Therefore, for simplicity we restrict our attetio to R with the euclidea orm. The followig propositios are stated for R, but work i ay fiite dimesioal vector space. See the exercises. Propositio (i) If A L(R,R m ), the A < ad A is uiformly cotiuous (Lipschitz with costat A ). (ii) If A,B L(R,R m ) ad c R, the A + B A + B, ca = c A. I particular, the operator orm is a orm o the vector space L(R,R m ). (iii) If A L(R,R m ) ad B L(R m,r l ), the BA B A. Proof. Let us start with (i). Let {e 1,e 2,...,e } be the stadard basis. Write x X, with x = 1, as x = c j e j. Sice e j e l = 0 wheever j l ad e j e j = 1, we fid c j = x e j x e j = 1. The Ax = c j Ae j c j Ae j Ae j The right had side does ot deped o x. We have foud a fiite upper boud idepedet of x, so A <. Next, for ay x,y R, A(x y) A x y If A <, the this says that A is Lipschitz with costat A. For (ii), let us ote that (A + B)x = Ax + Bx Ax + Bx A x + B x = ( A + B ) x. So A + B A + B.

15 8.2. ANALYSIS WITH VECTOR SPACES 21 Similarly Thus ca c A. Next ote (ca)x = c Ax ( c A ) x. c Ax = cax ca x. Hece c A ca. That we have a metric space follows pretty easily, ad is left to studet. For (iii) write BAx B Ax B A x. As a orm defies a metric, we have defied a metric space topology o L(R,R m ) so we ca talk about ope/closed sets, cotiuity, ad covergece. Propositio Let U L(R ) be the set of ivertible liear operators. (i) If A U ad B L(R ), ad the B is ivertible. (ii) U is ope ad A A 1 is a cotiuous fuctio o U. A B < 1 A 1, (8.2) To make sese of this o a simple example. Thik back to R 1, where liear operators are just umbers a. The operator a is ivertible (a 1 = 1/a) wheever a 0. Of course a 1/a is cotiuous. Whe > 1, the there are other oivertible operators tha just zero, ad i geeral thigs are a bit more difficult. Proof. Let us prove (i). First a straight forward computatio x = A 1 Ax A 1 Ax A 1 ( (A B)x + Bx ) A 1 A B x + A 1 Bx. Now assume x 0 ad so x 0. Usig (8.2) we obtai x < x + A 1 Bx, or i other words Bx 0 for all ozero x, ad hece Bx 0 for all ozero x. This is eough to see that B is oe-to-oe (if Bx = By, the B(x y) = 0, so x = y). As B is oe-to-oe operator from R to R it is oto ad hece ivertible. Let us look at (ii). Let B be ivertible ad ear A 1, that is (8.2) is satisfied. I fact, suppose A B A 1 < 1/2. The we have show above (usig B 1 y istead of x) B 1 y A 1 A B B 1 y + A 1 y 1/2 B 1 y + A 1 y,

16 22 CHAPTER 8. SEVERAL VARIABLES AND PARTIAL DERIVATIVES or So B 1 2 A 1. Now ote that B 1 y 2 A 1 y. A 1 (A B)B 1 = A 1 (AB 1 I) = B 1 A 1, ad B 1 A 1 = A 1 (A B)B 1 A 1 A B B 1 2 A 1 2 A B Matrices Fially let us get to matrices, which are a coveiet way to represet fiite-dimesioal operators. Suppose we have bases {x 1,x 2,...,x } ad {y 1,y 2,...,y m } for vector spaces X ad Y respectively. A liear operator is determied by its values o the basis. Give A L(X,Y ), Ax j is a elemet of Y. Therefore, defie the umbers {a i, j } as follows Ax j = m i=1 a i, j y i, ad write them as a matrix a 1,1 a 1,2 a 1, a 2,1 a 2,2 a 2, A = a m,1 a m,2 a m, Ad we say A is a m-by- matrix. Note that the colums of the matrix are precisely the coefficiets that represet Ax j. Let us derive the familiar rule for matrix multiplicatio. Whe z = c j x j, the Az = m i=1 c j a i, j y i,= m c j a i, j )y i, i=1( which gives rise to the familiar rule for matrix multiplicatio. There is a oe-to-oe correspodece betwee matrices ad liear operators i L(X,Y ). That is, oce we fix a basis i X ad i Y. If we would choose a differet basis, we would get differet matrices. This is importat, the operator A acts o elemets of X, the matrix is somethig that works with -tuples of umbers.

17 8.2. ANALYSIS WITH VECTOR SPACES 23 If B is a -by-r matrix with etries b j,k, the the matrix for C = AB is a m-by-r matrix whose i,kth etry c i,k is c i,k = a i, j b j,k. A liear mappig chagig oe basis to aother is the just a square matrix i which the colums represet basis elemets of the secod basis i terms of the first basis. We call such a liear mappig a chage of basis. Now suppose all the bases are just the stadard bases ad X = R ad Y = R m. If we recall the Cauchy-Schwarz iequality we ote that Az 2 = m ) 2 c j a i, j i=1( m i=1( 2)( ) j ) (a i, j ) (c 2 = I other words, we have a boud o the operator orm A m i=1 (a i, j ) 2. m (a i, j ) ) z i=1( 2 2. If the etries go to zero, the A goes to zero. I particular, if A if fixed ad B is chagig such that the etries of A B go to zero the B goes to A i operator orm. That is B goes to A i the metric space topology iduced by the operator orm. We have proved the first part of: Propositio If f : S R m is a cotiuous fuctio for a metric space S, the takig the compoets of f as the etries of a matrix, f is a cotiuous mappig from S to L(R,R m ). Coversely if f : S L(R,R m ) is a cotiuous fuctio the the etries of the matrix are cotiuous fuctios. The proof of the secod part is rather easy. Take f (x)e j ad ote that is a cotiuous fuctio to R m with stadard Euclidea orm (Note (A B)e j A B ). Such a fuctio recall from last semester that such a fuctio is cotiuous if ad oly if its compoets are cotiuous ad these are the compoets of the jth colum of the matrix f (x) Determiats It would be ice to have a easy test for whe is a matrix ivertible. This is where determiats come i. First defie the symbol sg(x) for a umber is defied by 1 if x < 0, sg(x) := 0 if x = 0, 1 if x > 0.

18 24 CHAPTER 8. SEVERAL VARIABLES AND PARTIAL DERIVATIVES Suppose σ = (σ 1,σ 2,...,σ ) is a permutatio of the itegers (1,2,...,), that is, a reorderig of (1,2,...,). Ay permutatio ca be obtaied by a sequece of traspositios (switchigs of two elemets). Call a permutatio eve (resp. odd) if it takes a eve (resp. odd) umber of traspositios to get from σ to (1,2,...,). It ca be show that this is well defied (exercise), i fact it is ot hard to show that sg(σ) := sg(σ 1,...,σ ) = p<q sg(σ q σ p ) (8.3) is 1 if σ is eve ad 1 if σ is odd. This fact ca be proved by otig that applyig a traspositio chages the sig, which is ot hard to prove by iductio o. The ote that the sig of (1,2,...,) is 1. Let S be the set of all permutatios o elemets (the symmetric group). Let A = [a i, j ] be a matrix. Defie the determiat of A Propositio (i) det(i) = 1. (ii) det([x 1 x 2 det(a) := sg(σ) σ S i=1 a i,σi. x ]) as a fuctio of colum vectors x j is liear i each variable x j separately. (iii) If two colums of a matrix are iterchaged, the the determiat chages sig. (iv) If two colums of A are equal, the det(a) = 0. (v) If a colum is zero, the det(a) = 0. (vi) A det(a) is a cotiuous fuctio. (vii) det [ a b c d ] = ad bc, ad det[a] = a. I fact, the determiat is the uique fuctio that satisfies (i), (ii), ad (iii). But we digress. By (ii), we mea that if we fix all the vectors x 1,...,x except for x j ad thik of the determiat as fuctio of x j, it is a liear fuctio, that is, if v,w R are two vectors, ad a,b R are scalars the det([x 1 x j 1 (av + bw) x j+1 x ]) = adet([x 1 x j 1 v x j+1 x ]) + bdet([x 1 x j 1 w x j+1 x ]). Proof. We go through the proof quickly, as you have likely see this before. (i) is trivial. For (ii), otice that each term i the defiitio of the determiat cotais exactly oe factor from each colum. Part (iii) follows by otig that switchig two colums is like switchig the two correspodig umbers i every elemet i S. Hece all the sigs are chaged. Part (iv) follows because if

19 8.2. ANALYSIS WITH VECTOR SPACES 25 two colums are equal ad we switch them we get the same matrix back ad so part (iii) says the determiat must have bee 0. Part (v) follows because the product i each term i the defiitio icludes oe elemet from the zero colum. Part (vi) follows as det is a polyomial i the etries of the matrix ad hece cotiuous. We have see that a fuctio defied o matrices is cotiuous i the operator orm if it is cotiuous i the etries. Fially, part (vii) is a direct computatio. Propositio If A ad B are matrices, the det(ab) = det(a)det(b). I particular, A is ivertible if ad oly if det(a) 0 ad i this case, det(a 1 ) = 1 det(a). Proof. Let b 1,b 2,...,b be the colums of B. The AB = [Ab 1 Ab 2 Ab ]. That is, the colums of AB are Ab 1,Ab 2,...,Ab. Let b j,k deote the elemets of B ad a j the colums of A. Note that Ae j = a j. By liearity of the determiat as proved above we have det(ab) = det([ab 1 Ab 2 Ab ]) = det ([ b j,1 a j Ab 2 Ab ]) = b j,1 det([a j Ab 2 Ab ]) = b j1,1b j2,2 b j, det([a j1 a j2 a j ]) 1 j 1, j 2,..., j ( ) = b j1,1b j2,2 b j, sg( j 1, j 2,..., j ) ( j 1, j 2,..., j ) S det([a 1 a 2 a ]). I the above, go from all itegers betwee 1 ad, to just elemets of S by otig that whe two colums i the determiat are the same the the determiat is zero. We the reorder the colums to the origial orderig ad obtai the sg. The coclusio follows by recogizig the determiat of B. The rows ad colums are swapped, but a momet s reflectio reveals it does ot matter. We could also just plug i A = I above. To prove the secod part of the theorem, suppose A is ivertible. The A 1 A = I ad cosequetly det(a 1 )det(a) = det(a 1 A) = det(i) = 1. If A is ot ivertible, the the colums are liearly depedet. That is, suppose γ j a j = 0,

20 26 CHAPTER 8. SEVERAL VARIABLES AND PARTIAL DERIVATIVES where ot all γ j are equal to 0. Without loss of geerality suppose γ 1 1. Take γ γ B := γ γ Applyig the defiitio of the derivative we see det(b) = γ 1 0. The det(ab) = det(a)det(b) = γ 1 det(a). The first colum of AB is zero, ad hece det(ab) = 0. Thus det(a) = 0. There are three types of so-called elemetary matrices. First for some j = 1,2,..., ad some λ R, λ 0, a matrix E defied by { e i if i j, Ee i = λe i if i = j. Give ay m matrix M the matrix EM is the same matrix as M except with the kth row multiplied by λ. It is a easy computatio (exercise) that det(e) = λ. Secod, for some j ad k with j k, ad λ R a matrix E defied by { e i if i j, Ee i = e i + λe k if i = j. Give ay m matrix M the matrix EM is the same matrix as M except with λ times the kth row added to the jth row. It is a easy computatio (exercise) that det(e) = 1. Fially for some j ad k with j k a matrix E defied by e i if i j ad i k, Ee i = e k if i = j, e j if i = k. Give ay m matrix M the matrix EM is the same matrix with jth ad kth rows swapped. It is a easy computatio (exercise) that det(e) = 1. Elemetary matrices are useful for computig the determiat. The proof of the followig propositio is left as a exercise. Propositio Let T be a ivertible matrix. The there exists a fiite sequece of elemetary matrices E 1,E 2,...,E k such that T = E 1 E 2 E k, ad det(t ) = det(e 1 )det(e 2 ) det(e k ).

21 8.2. ANALYSIS WITH VECTOR SPACES 27 Propositio Determiat is idepedet of the basis. I other words, if B is ivertible the, det(a) = det(b 1 AB). The proof is immediate. If i oe basis A is the matrix represetig a liear operator, the for aother basis we ca fid a matrix B such that the matrix B 1 AB takes us to the first basis, applies A i the first basis, ad takes us back to the basis we started with. Therefore, the determiat ca be defied as a fuctio o the space L(X) for some fiite dimesioal metric space X, ot just o matrices. We choose a basis o X, ad we ca represet a liear mappig usig a matrix with respect to this basis. We obtai the same determiat as if we had used ay other basis. It follows from the two propositios that det: L(X) R is a well-defied ad cotiuous fuctio Exercises Exercise 8.2.1: If X is a vector space with a orm, the show that d(x,y) := x y makes X a metric space. Exercise (Easy): Show that det(ab) = det(ba). Exercise 8.2.3: For R defie x := max{ x 1, x 2,..., x }, sometimes called the sup or the max orm. a) Show that is a orm o R (defiig a differet distace). b) What is the uit ball B(0,1) i this orm? Exercise 8.2.4: For R defie x 1 := sometimes called the 1-orm (or L 1 orm). a) Show that 1 is a orm o R (defiig a differet distace, sometimes called the taxicab distace). b) What is the uit ball B(0,1) i this orm? Exercise 8.2.5: Usig the euclidea orm o R 2. Compute the operator orm of the operators i L(R 2 ) give by the matrices: a) [ ] b) [ ] c) [ ] d) [ ] Exercise 8.2.6: Usig the stadard euclidea orm R. Show a) If A L(R,R ) is defied by Ax = ax for a vector a R. The the operator orm A = a. b) If B L(R,R) is defied by Bx = b x for a vector b R. The the operator orm B = b. Exercise 8.2.7: Suppose σ = (σ 1,σ 2,...,σ ) is a permutatio of (1,2,...,). a) Show that we ca make a fiite umber of traspositios (switchig of two elemets) to get to (1,2,...,). b) Show that the sig of sigma is well defied. That is, if it takes a eve umber of traspositios to get to (1,2,...,) it caot be achieved i a odd umber of traspositios. Ad vice versa. c) Usig the defiitio (8.3) show that σ is eve if sg(σ) = 1 ad σ is odd if sg(σ) = 1. x j,

22 28 CHAPTER 8. SEVERAL VARIABLES AND PARTIAL DERIVATIVES Exercise 8.2.8: Verify the computatio of the determiat for the three types of elemetary matrices. Exercise 8.2.9: Prove Propositio Exercise : a) Suppose D = [d i, j ] is a -by- diagoal matrix, that is, d i, j = 0 wheever i j. Show that det(d) = d 1,1 d 2,2 d,. b) Suppose A is a diagoalizable matrix. That is, there exists a matrix B such that B 1 AB = D for a diagoal matrix D = [d i, j ]. Show that det(a) = d 1,1 d 2,2 d,. Exercise : Take the vectorspace of polyomials R[t] ad the liear operator D L(R[t]) that is the differetiatio (we proved i a earlier exercise that D is a liear operator). Defie the orm o P(t) = c 0 + c 1 t + + c t as P := sup{ c j : j = 0,1,...,}. a) Show that P is a orm o R[t]. b) Show that D does ot have bouded operator orm, that is D =. Hit: cosider the polyomials t as teds to ifiity. The oe part of the proof of propositios ad eeded to state ad prove them for all fiite dimesioal vector spaces is to show that i fiite dimesioal vector spaces a liear operators always have fiite operator orm. Exercise : Let X be ay fiite dimesioal vector space with a orm. Let {x 1,x 2,...,x } be a basis. a) Show that the fuctio f : R R f (c 1,c 2,...,c ) = c 1 x 1 + c 2 x c x is cotiuous. b) Show that there exists a umber M such that if c = (c 1,...,c ) R with c 1 (stadard euclidea orm), the c 1 x 1 + c 2 x c x M (the orm o X). c) Show that there exists a umber B such that if x = c 1 x 1 + c 2 x c x, ad x = 1, the c j B. d) Use part (c) to show that if X ad Y are fiite dimesioal vector spaces ad A L(X,Y ), the A <. Exercise : Let X be ay fiite dimesioal vector space with a orm. Let {x 1,x 2,...,x } be a basis. Let c = (c 1,...,c ) R ad c be the stadard euclidea orm o R. a) Fid that there exist positive umbers m,m > 0 such that for m c c 1 x 1 + c 2 x c x M c. b) Use part (a) to show that of 1 ad 1 are two orms o X, the there exist positive umbers m,m > 0 (perhaps differet tha above) such that for all x X we have m x 1 x 2 M x 1. c) Now show that U X is ope i the metric defied by x y 1 if ad oly if it is ope i the metric defied by x y 2. I other words, covergece of sequeces, cotiuity of fuctios is the same i either orm. Hit: See previous exercise.

23 8.3. THE DERIVATIVE The derivative Note:??? lectures The derivative Recall that for a fuctio f : R R, we defied the derivative at x as lim h 0 f (x + h) f (x). h I other words, there was a umber a (the derivative of f at x) such that lim f (x + h) f (x) h 0 a h = lim f (x + h) f (x) ah h 0 h = lim f (x + h) f (x) ah = 0. h 0 h Multiplyig by a is a liear map i oe dimesio. That is, we thik of a L(R 1,R 1 ). We use this defiitio to exted differetiatio to more variables. Defiitio Let U R be a ope subset ad f : U R m. We say f is differetiable at x U if there exists a A L(R,R m ) such that f (x + h) f (x) Ah lim = 0. h 0 h R We write D f (x) := A, or f (x) := A, ad we say A is the derivative of f at x. Whe f is differetiable at all x U, we say simply that f is differetiable. For a differetiable fuctio, the derivative of f is a fuctio from U to L(R,R m ). Compare to the oe dimesioal case, where the derivative is a fuctio from U to R, but we really wat to thik of R here as L(R 1,R 1 ). The orms above must be i the right spaces of course. The orm i the umerator is i R m, ad the orm i the deomiator is R where h lives. Normally it is uderstood that h R from cotext. We will ot explicitly say so from ow o. We have agai cheated somewhat ad said that A is the derivative. We have ot show yet that there is oly oe, let us do that ow. Propositio Let U R be a ope subset ad f : U R m. Suppose x U ad there exist A,B L(R,R m ) such that The A = B. f (x + h) f (x) Ah f (x + h) f (x) Bh lim = 0 ad lim = 0. h 0 h 0

24 30 CHAPTER 8. SEVERAL VARIABLES AND PARTIAL DERIVATIVES Proof. So (A B)h (A B)h f (x + h) f (x) Ah ( f (x + h) f (x) Bh) = f (x + h) f (x) Ah f (x + h) f (x) Bh +. 0 as h 0. That is, give ε > 0, the for all h i some δ-ball aroud the origi ε > (A B)h = h (A B). For ay x with x = 1 let h = (δ/2)x, the < δ ad h = x. So (A B)x < ε. Takig the supremum over all x with x = 1 we get the operator orm A B ε. As ε > 0 was arbitrary A B = 0 or i other words A = B. Example 8.3.3: Let f : R 2 R 2 be defied by f (x,y) = ( f 1 (x,y), f 2 (x,y) ) := (1+x+2y+x 2,2x+ 3y+xy). Let us show that f is differetiable at the origi ad let us compute compute the derivative, directly usig the defiitio. The derivative is i L(R 2,R 2 ) so it ca be represeted by a 2 2 matrix [ a b c d ]. Suppose h = (h1,h 2 ). We eed the followig expressio to go to zero. f (h 1,h 2 ) f (0,0) (ah 1 + bh 2,ch 1 + dh 2 ) = (h 1,h 2 ) ((1 a)h1 + (2 b)h 2 + h 2 ) 2 ( ) (2 c)h1 + (3 d)h 2 + h 1 h 2. h h2 2 If we choose a = 1, b = 2, c = 2, d = 3, the expressio becomes h h2 1 h2 2 h 2 1 = h 1 + h2 2 = h 1. h h2 2 h h2 2 Ad this expressio does ideed go to zero as h 0. Therefore the fuctio is differetiable at the origi ad the derivative ca be represeted by the matrix [ ]. Example 8.3.4: If f (x) = Ax for a liear mappig A, the f (x) = A. This is easily see: f (x + h) f (x) Ah = A(x + h) Ax Ah = 0 = 0. Propositio Let U R be ope ad f : U R m be differetiable at p U. The f is cotiuous at p.

25 8.3. THE DERIVATIVE 31 Proof. Aother way to write the differetiability of f at p is to first write ad r(h) r(h) := f (p + h) f (p) f (p)h, must go to zero as h 0. So r(h) itself must go to zero. The mappig h f (p)h is liear mappig betwee fiite dimesioal spaces, it is therefore cotiuous ad goes to zero as h 0. Therefore, f (p + h) must go to f (p) as h 0. That is, f is cotiuous at p. Theorem (Chai rule). Let U R be ope ad let f : U R m be differetiable at p U. Let V R m be ope, f (U) V ad let g: V R l be differetiable at f (p). The F(x) = g ( f (x) ) is differetiable at p ad F (p) = g ( f (p) ) f (p). Without the poits where thigs are evaluated, this is sometimes writte as F = ( f g) = g f. The way to uderstad it is that the derivative of the compositio g f is the compositio of the derivatives of g ad f. That is, if A := f (p) ad B := g ( f (p) ), the F (p) = BA. Proof. Let A := f (p) ad B := g ( f (p) ). Take h R ad write q = f (p), k = f (p + h) f (p). Let r(h) := f (p + h) f (p) Ah = k Ah. The F(p + h) F(p) BAh = g( f (p + h) ) g ( f (p) ) BAh = g(q + k) g(q) B( k r(h) ) g(q + k) g(q) Bk + B r(h) g(q + k) g(q) Bk f (p + h) f (p) = k + B r(h). First, B is costat ad f is differetiable at p, so the term B r(h) goes to 0. Next as f is cotiuous at p, we have that as h goes to 0, the k goes to 0. Therefore g(q+k) g(q) Bk k goes to 0 because g is differetiable at q. Fially f (p + h) f (p) As f is differetiable at p, the term F(p+h) F(p) BAh f (p + h) f (p) Ah f (p+h) f (p) + Ah f (p + h) f (p) Ah + A. stays bouded as h goes to 0. Therefore, goes to zero, ad F (p) = BA, which is what was claimed.

26 32 CHAPTER 8. SEVERAL VARIABLES AND PARTIAL DERIVATIVES Partial derivatives There is aother way to geeralize the derivative from oe dimesio. We ca hold all but oe variables costat ad take the regular derivative. Defiitio Let f : U R be a fuctio o a ope set U R. If the followig limit exists we write f f (x 1,...,x j 1,x j + h,x j+1,...,x ) f (x) f (x + he j ) f (x) (x) := lim = lim. x j h 0 h h 0 h We call f x j (x) the partial derivative of f with respect to x j. Sometimes we write D j f istead. For a mappig f : U R m we write f = ( f 1, f 2,..., f m ), where f k are real-valued fuctios. The we defie f k x j (or write it as D j f k ). Partial derivatives are easier to compute with all the machiery of calculus, ad they provide a way to compute the derivative of a fuctio. Propositio Let U R be ope ad let f : U R m be differetiable at p U. The all the partial derivatives at p exist ad i terms of the stadard basis of R ad R m, f (p) is represeted by the matrix f 1 f x (p) 1 f 1 x (p) x (p) f 2 f x (p) 2 f 1 x (p) x (p) f m x 1 (p) f m x 2 (p)... f m x (p) I other words If h = c je j, the f (p)e j = f (p)h = m f k (p)e k. k=1 x j m k=1 c j f k x j (p)e k. Proof. Fix a j ad ote that f (p + he j ) f (p) h f (p)e j = f (p + he j ) f (p) f (p)he j h = f (p + he j) f (p) f (p)he j. he j

27 8.3. THE DERIVATIVE 33 As h goes to 0, the right had side goes to zero by differetiability of f, ad hece f (p + he j ) f (p) lim = f (p)e j. h 0 h Note that f is vector valued. So represet f by compoets f = ( f 1, f 2,..., f m ), ad ote that takig a limit i R m is the same as takig the limit i each compoet separately. Therefore for ay k the partial derivative f k f k (p + he j ) f k (p) (p) = lim x j h 0 h exists ad is equal to the kth compoet of f (p)e j, ad we are doe. Oe of the cosequeces of the theorem is that if f is differetiable o U, the f : U L(R,R m ) is a cotiuous fuctio if ad oly if all the f k x j are cotiuous fuctios. The coverse of the propositio is ot true. Just because the partial derivatives exist, does ot mea that the fuctio is differetiable. See the exercises Gradiet ad directioal derivatives Let U R be ope ad f : U R is a differetiable fuctio. We defie the gradiet as f (x) := f (x)e j. x j Suppose γ : (a,b) R R is a differetiable fuctio ad the image γ ( (a,b) ) U. Such a fuctio ad its image is sometimes called a curve, or a differetiable curve. Write γ = (γ 1,γ 2,...,γ ). Let g(t) := f ( γ(t) ). The fuctio g is differetiable. For purposes of computatio we idetify idetify L(R 1 ) with R, ad hece g (t) ca be computed as a umber: g (t) = f ( γ(t) ) γ (t) = f ( )dγ j γ(t) x j dt (t) = f dγ j x j dt. For coveiece, we sometimes leave out the poits where we are evaluatig as o the right had side above. Notice g (t) = ( f ) ( γ(t) ) γ (t) = f γ, where the dot is the stadard scalar dot product. We use this idea to defie derivatives i a specific directio. A directio is simply a vector poitig i that directio. So pick a vector u R such that u = 1. Fix x U. The defie γ(t) := x +tu.

28 34 CHAPTER 8. SEVERAL VARIABLES AND PARTIAL DERIVATIVES It is easy to compute that γ (t) = u for all t. By chai rule d [ ] f (x +tu) = ( f )(x) u, dt t=0 where the otatio dt d t=0 represets the derivative evaluated at t = 0. We also compute directly We obtai the directioal derivative, deoted by d [ ] f (x + hu) f (x) f (x +tu) = lim. dt t=0 h 0 h D u f (x) := d [ ] f (x +tu), dt t=0 which ca be computed by oe of the methods above. Let us suppose ( f )(x) 0. By Cauchy-Schwarz iequality we have D u f (x) ( f )(x). Equality is achieved whe u is a scalar multiple of ( f )(x). That is, whe u = ( f )(x) ( f )(x), we get D u f (x) = ( f )(x). The gradiet poits i the directio i which the fuctio grows fastest, i other words, i the directio i which D u f (x) is maximal The Jacobia Defiitio Let U R ad f : U R be a differetiable mappig. The defie the Jacobia of f at x as J f (x) := det ( f (x) ). Sometimes this is writte as ( f 1, f 2,..., f ) (x 1,x 2,...,x ). This last piece of otatio may seem somewhat cofusig, but it is useful whe you eed to specify the exact variables ad fuctio compoets used. The Jacobia J f is a real valued fuctio, ad whe = 1 it is simply the derivative. Whe f is C 1, the J f is a cotiuous fuctio. From the chai rule it follows that: J f g (x) = J f ( g(x) ) Jg (x).

29 8.3. THE DERIVATIVE 35 It ca be computed directly that the determiat tells us what happes to area/volume. Suppose we are i R 2. The if A is a liear trasformatio, it follows by direct computatio that the direct image of the uit square A([0,1] 2 ) has area det(a). Note that the sig of the determiat determies orietatio. If the determiat is egative, the the two sides of the uit square will be flipped i the image. We claim without proof that this follows for arbitrary figures, ot just the square. Similarly, the Jacobia measures how much a differetiable mappig stretches thigs locally, ad if it flips orietatio Exercises Exercise 8.3.1: Suppose γ : ( 1,1) R ad α : ( 1,1) R be two differetiable curves such that γ(0) = α(0) ad γ (0) = α (0). Suppose F : R R is a differetiable fuctio. Show that d F ( γ(t) ) = d F ( α(t) ). dt t=0 dt t=0 Exercise 8.3.2: Let f : R 2 R be give by f (x,y) = x 2 + y 2. Show that f is ot differetiable at the origi. Exercise 8.3.3: Defie a fuctio f : R 2 R by { xy if (x,y) (0,0), f (x,y) := x 2 +y 2 0 if (x,y) = (0,0). a) Show that partial derivatives f x ad f y exist at all poits (icludig the origi). b) Show that f is ot cotiuous at the origi (ad hece ot differetiable). Exercise 8.3.4: Defie a fuctio f : R 2 R by { x 2 y if (x,y) (0,0), f (x,y) := x 2 +y 2 0 if (x,y) = (0,0). a) Show that partial derivatives f x ad f y exist at all poits. b) Show that for all u R 2 with u = 1, the directioal derivative D u f exists at all poits. c) Show that f is cotiuous at the origi. d) Show that f is ot differetiable at the origi. Exercise 8.3.5: Suppose f : R R is oe-to-oe, oto, differetiable at all poits, ad such that f 1 is also differetiable at all poits. a) Show that f (p) is ivertible at all poits p ad compute ( f 1 ) ( f (p) ). Hit: cosider p = f 1 ( f (p) ). b) Let g: R R be a fuctio differetiable at q R ad such that g(q) = q. Suppose f (p) = q for some p R. Show J g (q) = J f 1 g f (p) where J g is the Jacobia determiat. Exercise 8.3.6: Suppose f : R 2 R is differetiable ad such that f (x,y) = 0 if ad oly if y = 0 ad such that f (0,0) = (1,1). Prove that f (x,y) > 0 wheever y > 0 ad f (x,y) < 0 wheever y < 0.

30 36 CHAPTER 8. SEVERAL VARIABLES AND PARTIAL DERIVATIVES Exercise 8.3.7: Suppose U R is ope ad f : U R is differetiable. Suppose that f has a local maximum at p U. Show that f (p) = 0, that is the zero mappig i L(R,R). Exercise 8.3.8: Suppose f : R 2 R is differetiable ad suppose that wheever x 2 +y 2 = 1, the f (x,y) = 0. Prove that there exists at least oe poit (x 0,y 0 ) such that f x (x 0,y 0 ) = f y (x 0,y 0 ) = 0. Exercise 8.3.9: Suppose f : R R is differetiable ad f (t) = 1 for all t (that is, we have a curve i the uit sphere). The show that for all t, treatig f as a vector we have, f (t) f (t) = 0.

31 8.4. CONTINUITY AND THE DERIVATIVE Cotiuity ad the derivative Note:??? lectures Boudig the derivative Let us prove a mea value theorem for vector valued fuctios. Lemma If ϕ : [a,b] R is differetiable o (a,b) ad cotiuous o [a,b], the there exists a t 0 (a,b) such that ϕ(b) ϕ(a) (b a) ϕ (t 0 ). Proof. By mea value theorem o the fuctio ( ϕ(b) ϕ(a) ) ϕ(t) (the dot is the scalar dot product agai) we obtai there is a t 0 (a,b) such that ( ϕ(b) ϕ(a) ) ϕ(b) ( ϕ(b) ϕ(a) ) ϕ(a) = ϕ(b) ϕ(a) 2 = (b a) ( ϕ(b) ϕ(a) ) ϕ (t 0 ) where we treat ϕ as a simply a colum vector of umbers by abuse of otatio. Note that i this case, if we thik of ϕ (t) as simply a vector, the by Exercise 8.2.6, ϕ (t) L(R,R ) = ϕ (t) R. By Cauchy-Schwarz iequality ϕ(b) ϕ(a) 2 = (b a) ( ϕ(b) ϕ(a) ) ϕ (t 0 ) (b a) ϕ(b) ϕ(a) ϕ (t 0 ). Recall that a set U is covex if wheever x,y U, the lie segmet from x to y lies i U. Propositio Let U R be a covex ope set, f : U R m a differetiable fuctio, ad a M such that f (x) M for all x U. The f is Lipschitz with costat M, that is for all x,y U. f (x) f (y) M x y Proof. Fix x ad y i U ad ote that (1 t)x +ty U for all t [0,1] by covexity. Next d [ f ( (1 t)x +ty )] = f ( (1 t)x +ty ) (y x). dt By mea value theorem above we get f (x) f (y) d [ f ( (1 t)x +ty )] f ( (1 t)x +ty ) y x M y x. dt