Differentiability. Alex Nita

Transcription

1 Differentibilit Ale Nit 1 Mtrices nd Liner Functions 11 Algebric Properties of Mtrices Let us define rel m n mtri s n rr of mn rel numbers ij consisting of m rows nd n columns, 11 1n A 11 m1 mn The inde i in ij indictes the ith row nd the inde j indictes the jth column For emple,,3 is the rel number in the second row, third column of A The word rel in the phrse rel mtri refers to the fct tht its entries i re rel numbers We could nlogousl define comple mtrices, quternionic mtrices, integer mtrices, etc Emple 11 The following is rel 3 mtri: while the following is rel 1 4 mtri: We cn define ddition of two mtrices A nd B of the sme dimension m n s we did with vectors, componentwise: 11 1n b 11 b 1n 11 + b 11 1n + b 1n A + B + 1 m1 mn b m1 b mn m1 + b m1 mn + b mn We cn lso define sclr multipliction of n m n rel mtri A b rel number : 11 1n 11 1n A 13 m1 mn m1 mn 1

2 Emple 1 For emple, here sum of 3 mtrices: nd here is sclr multiple of 3 mtri We cn lso multipl mtrices If A is n m n mtri nd B is n n p mtri m, n nd p m be different here, but note tht n ppers in the dimensions of both A nd B, in specific loctions!, then the product of A nd B, written AB, is n m p mtri whose entries c ij re the dot products of the ith row of A nd the jth column of B Tht is, if we write A in terms of row vectors, 1 A, m where 1 11,, 1n 1,, n m m1,, mn nd if we write B in terms of column vectors, B b1 b bp where b1 b 11 b n1, b b 1 b n, b1 b 1p b np then 1 b 1 1 b 1 b p b 1 b 1 b p AB m b 1 m b m b p 16 Emple 13 Let us see how this works when we multipl 3 mtri with 5 We should, of course, get 3 5 mtri Let 1 1 A , B Then A nd B re written in terms of their row nd column vectors s 1 1, 1 A 0, 3, 4 7 b1 b b3 b4 b5 B, 1, 4, 1 1 0, 0 3

3 We rell think of the b j s vectors, of course, so for emple b 1 7, Then, we hve b 1 1 b 1 b 1 3 b 1 4 b 1 5 AB b 1 b b 3 b 4 b 5 3 b 1 3 b 3 b 3 3 b 4 3 b 5 1, 1 7, 1, 1 1, 4 1, 1, 1 1, 1 1, 0 1, 1 0, 3 0, 7, 0, 1, 4 0,, 1 0, 1, 0 0, 0, 3, 4 7,, 4 1, 4, 4, 1, 4 1, 0, 4 0, Geometric Properties of Mtrices The significnce of mtrices is tht we cn use them to define liner functions, which, s the nme suggests, re functions tht generlize functions of single vrible whose grphs re lines, f m + b 17 We hve lred seen n emple of this in the liner eqution for plne, z m+n +b, which is the grph of the function z g, m + n + b This is ver geometric ide Let us eplin the geometric content of this now Recll the eqution of plne, + b + cz + d 0 18 We cn consider this plne either s the level surfce of function f : R 3 R, nmel f,, z + b + cz + d, or else, solving for z which gives z c b c d c, ssuming we cn do this, ie ssuming c 0, we cn consider it s the grph of function g : R R, nmel g, c b c d c Let us tke the ltter pproch for the moment In generl, the grph of function of the tpe g : R R g, m + n + b 19 will be plne If b 0, tht plne will not pss through the origin 0, 0, 0 it will pss insted through 0, 0, b, of course Our gol is to tke complicted function f : R R, whose grph m be curved nd twisted, nd locll pproimte its grph b the grph of liner function, tht is function of the tpe 19 g is clled liner becuse it is generliztion of the one-dimensionl line, m + b Now we hve two dimensions, so the nlog is z m + n + b More generll, function f : R n R will be clled liner if it is of the form f f 1,,, n m m + + m n n + b 110 Wht if the function is vector-vlued, tht is f : R n R m? Then f cn be described in terms of m component functions f i : R n R, f f 1, f,, f m 111 3

4 nd if ech of these component functions f i is liner, ie if ech f i is of the form 110, then we get sstem of liner epressions, f 1 f 1 1,,, n m m m 1n n + b 1 f f 1 1,,, n m m + + m n n + b 11 f m f 1 1,,, n m m1 1 + m m + + m mn n + b m With mtri nd vector nottion, we cn netl write the sstem 11 s f 1 m 11 m 1 m 1n 1 b 1 f m 1 m m n + b f 1 m m1 m m m mn n b m 113 If we write the component epression of f in 111 s column vector insted of row vector, then we cn even more netl write 11 s where f A + b 114 f 1 m 11 m 1 m 1n b 1 f f, A m 1 m m n, b b f 1 m m1 m m m mn A function f : R n R m of the form 114 will be clled liner function Note, however, tht in mn other books onl function of the tpe f A is clled liner, while function of the tpe f A + b is clled ffine Eercise 14 Show tht if A is n m n mtri, nd re column vectors in R n thought of s n 1 mtrices, nd nd b re rel numbers, then we hve the following distributivit lw: A + b A + ba b m 13 Emples of Liner Functions Emple 15 A ver simple, though not ver interesting, emple of liner function is the unction f : R 3 R, f,, z + 5 z 4, z + The two component functions of f re f 1,, z + 5 z 4 f,, z + nd the re both liner, so the function f is liner As we did in the generl cse, we cn write the sstem of epressions bove s single mtri epression: f1,, z f,, z 4 + f,, z z 4

5 Emple 16 the plne, Let f : R R be the rottion-followed-b--trnsltion function in f, cos θ sin θ + 5, sin θ + cos θ 4 This is rottion of the plne through n ngle θ followed b trnsltion b the vector/point 5, 4 We cn write this in mtri nottion s f1, cos θ sin θ 5 f, + f, sin θ cos θ 4 Let us briefl eplin the geometric resoning behind this epression Let s strt with rottion through n ngle θ, cll it R In the following picture of the unit circle, we rotte the two points nd ling on the circle, s well s the sum + ling outside of it, b n ngle θ, tht is we ppl rottion R to them R + R R R + 1 Now, the first thing to notice is tht ever point, in the plne cn be written s sum of sclr multiples of the coordinte vectors/points i 1, 0 nd j 0, 1, nmel b,, 0 + 0, 1, 0 + 0, 1 i + j so if we cn mnge to chrcterize wht the rottion does to i nd j, then we ll understnd wht it does to n point, provided of course tht R is liner, tht is it stisfies R + R + R R R for ll points/vectors nd nd ll rel numbers see Eercise 14 bove for the lgebric reson for linerit The intuitive geometric resoning for linerit is contined in the bove digrm: we d like to be ble to rotte the points nd to R nd R, then dd them to get R + R, nd we should get the sme point/vector s when we first dd nd nd then rotte the result to R + If R were liner, then knowing how R trets i nd j mens we know how it trets, i + j, becuse R, Ri + j Ri + Rj Let s now chrcterize R on i nd j Clerl, rottion through positive ngle θ sends the point i 1, 0 to the point cos θ, sin θ nd it sends the point j 0, 1 to the point 5

6 sin θ, cos θ, s cn be seen in the following digrm: Rj sin θ, cos θ j 0, 1 1 θ 1 Ri cos θ, sin θ θ i 1, 0 Tht is, Ri R1, 0 cos θ, sin θ Rj R0, 1 sin θ, cos θ Therefore, ssuming the linerit of R we hve tht R, Ri + j Ri + Rj cos θ, sin θ + sin θ, cos θ cos θ sin θ, sin θ + cos θ }{{}}{{} R 1, R, for ll points, in R Putting this in mtri nottion, R cn be described s mtri cos θ sin θ R 115 sin θ cos θ nd its effect on point/vector, in R is to left-multipl it, s column vector, b the mtri R: cos θ sin θ R sin θ cos θ The cumultive effect of the rottion R, s through θ π/6, is to rotte the entire plne: θ 6

7 Net, consider trnsltion T b point/vector v, b This is much simpler to understnd A trnsltion simpl dds the point/vector v to n given point,, T + v, +, b +, + b or, if we re going to write everthing in terms of column vectors, T + b The effect of T is thus to shift then entire plne R b the vector v For emple, if v 3/, 1/, the entire plne is shifted 3/ units to the right nd 1/ unit up: Lstl, let s combine the two The w to do this is to compose the two, first ppl the rottion, then ppl the trnsltion, cos θ sin θ f T R, ie f + sin θ cos θ b The picture is the following: 7

8 Emple 17 If l is line in the plne R pssing through the origin, then reflection bout l is function R : R R, which obviousl stisfies { if lies on l R if is orthogonl to l Note tht for to be orthogonl to l we must hve the entire line contining be perpendiculr to l for if tht is the cse, then n point on the line will be orthogonl to n point in the other line! Wht bout ever other point in the plne? Well, if, is n rbitrr point in R, then we cn decompose it into prllel nd perpendiculr components to l: In this cse, we must hve tht + R R + R + R if we suppose, s we must, tht R is liner Now, we know wht nd re Of course, l be entirel described b unit vector v, b, since ever other point on l is sclr multiple of v, tht is l {λv λ, λb λ is rel number} As consequence, vv nd vv, which mens R vv vv vv We cn describe this opertion s mtri, b noting tht v + b, so tht writing v nd s column vectors, we get R R vv + b b + b b + b 1 + b b + b 1 1 b b b 1 For emple, if l is the line contining the unit vector v 3/, 1/, ie cos π/6, sin π/6, then if we pick the point, s, 1/, 3/, it s reflection bout l will be R

9 so the picture of R v l R In sum, then, if we re going to reflect ll points in R bout given line l contining ll sclr mutliples of specific unit vector v, b, then the reflection R is liner function given in mtri terms b R 1 b b b 1 Differentibilit 1 The Totl Derivtive The thing we wnt to do now is to locll ppproimte complicted function f : R n R m b much simpler liner function This is the ide behind the totl derivtive of f t point in R n Formll, we s tht f is differentible t point R n if there eists n m n mtri m nd n here depend on the domin nd rnge of f! m 11 m 1n Df rel m n mtri 1 m m1 m mn clled the totl derivtive or the Jcobi mtri, which stisfies the following limit condition: f + h f Dfh lim 0 h 0 h Equivlentl, f must locll be pproimted b liner function, tht is where the error in the pproimtion f + h Dfh + f 3 Eh f + h liner pproimtion of the vlue of f t + h f + h Dfh + f stisfies Eh lim 0 4 h 0 h This lst sttement, 4, is obviousl the sme sttement s 9

10 Remrk 1 The epression Dfh denotes mtri multipliction Here, h h 1,, h n is vector in R n thought of s n n 1 column vector: m 11 m 1n h 1 m 11 h m 1n h n Dfh m m1 m mn h n m m1 h m mn h 1 Remrk Thus to s tht function f is differentible t point is equivlent to sing tht f hs totl derivtive there We shll see tht f m be prtill differentible, nd to hve directionl derivtives in ll directions, et not be differentible We will eplin this further below The Directionl Derivtive Suppose v is vector in R n nd is point in R n, nd let T : R R n be the trnsltionb-tv function T t + tv We s tht f : R n R m hs directionl derivtive t in the direction of v if the composition f T : R R m, f T t f + tv, is differentible t 0, in the sense tht the limit f v or D v f d dt f + tv t0 eists in R m lim t 0 f + tv f t 5 Remrk 3 Notice tht for ech fied nonzero t the difference f + tv f in the numertor is vector in R m, while 1/t is rel number, so the quotient f+tv f t is ctull sclr multipliction of the vector f + tv f b 1/t Another thing to notice is tht in the sum + tv we dded point to vector Since we hve emphsized blurring the lines between points nd vectors in R n, on ccount of lgebricll the re indistinguishble, t lest in R n, the sum mkes sense 3 The Prtil Derivtive Now suppose f is rel-vlued function, f : R n R The ith prtil derivtive of f t is the directionl derivtive of f t in coordinte direction, tht is in the direction of unit coordinte vector e i 0,, i,, 0, i or f i or D i f d dt f + te i t0 lim t 0 f + te i f t lim t 0 f 1,, i 1, i + t, i+1,, n f 1,, i,, n t 6 10

11 Remrk 4 The prcticl import of this definition will become cler in minute For now, notice tht the derivtive d dt t0 f + te i is n ordinr derivtive from Clc 1 It s the derivtive of the rel-vlued function of rel vrible f T : R R, f T t f + te i This mens tht ll the other coordintes, which we normll tret s vribles, since the m vr, re treted here s constnts Thus, if we lbel the vribles 1,, i,, n, ll the other j for j i re treted s constnts in n epression for f For emple, if f,, z z + + z, in the prtil derivtive with respect to the vribles nd z re treted s constnts, so we cn do wht we normll do when computing Clc 1 derivtive, pull the constnts out Here, for emple, we d hve z + 4 The Reltionship Between the Totl nd Directionl nd Prtil Derivtives Nobod wnts to compute n ctul limit, though the limit ide is etremel importnt theoreticll Luckil, we don t hve to here The directionl derivtive, though defined in terms of limit, is in fct computble in terms of mtri product! Theorem 5 If f : R n R m is differentible t in R n, then ll of its directionl derivtives t eist, nd for n choice of vector v in R n we hve D v f Dfv 7 The left-hnd side is limit, while the right-hnd side is mtri product, with v treted s column vector Proof: Since f is differentible t, fi v nd consider h tv for some sufficientl smll t R Appling the liner pproimtion 3 nd the linerit of the derivtive Df ie Df + b Df + bdf, cf Eercise 14 bove we get f + tv f tdfv f + tv f Dftv 8 nd ppling the limit 4 f + h f Dfh Eh Etv Etv Etv lim lim t 0 t t 0 t v v lim Etv Eh v lim v 0 v 0 t 0 tv h 0 h B 8 this mens nd hence f + tv f tdfv lim 0 t 0 t f + tv f f + tv f tdfv lim Dfv lim lim t 0 t t 0 t t 0 t f + tv f tdfv lim t 0 t 0 ie f + tv f D v f lim Dfv t 0 t 11

12 Emple 6 Suppose f : R 3 R is differentible t the point 1, 1, nd v 1, 4, is vector in R 3 If we lred knew the totl derivtive of f, s 3 Df then computing the directionl derivtive of f t in the direction of v would be es, nmel 3 f v Dfv Thus, if f is differentible t, the tsk is to find w to compute Df For then we cn compute ll directionl derivtives b simple mtri multipliction Luckil, we cn do this, using the previous theorem, provided we know the tht the totl derivtive Df hs rows consisting of the totl derivtives of the component functions, tht is f 1 D } f m {{} Df Df 1 Df m In this cse I clim tht the totl derivtive Df is the mtri of prtil derivtives of the component functions f i of f, n Df m m 1 n 9 To see this, tke rel-vlued function first, f : R n R for n vector-vlued function s bove is mde up of its m rel-vlued component functions nd look t the directionl derivtive in the ith coordinte direction e i 0,, 1,, 0 it hs 1 in the ith slot nd 0 everwhere else Letting Df m 1 m m n be the 1 n mtri defining the totl derivtive of f, nd noting tht the ith prtil derivtive is the directionl derivtive in the ith coordinte direction, we hve 0 i D ei f Dfe i m 1 m m n 1 0 m m 0 + m i 1 + m n 0 m i This is true for ech i 1,, n, so Df m 1 m m n 1 n 1

13 Now consider vector-vlued function f : R n R m, f f 1,, f m Then, ech of its component functions f i is rel-vlued function, nd the bove result pplies seprtel to ech, from which we get our result, 1 1 f 1 Df D f m Df 1 Df m 1 m 1 n m n The onl questionble thing bout this is the leglit of the second equlit, where we pulled the D inside the column vector It ws, in fct, legl, nd moreover our bilit to do this gives us nother useful w to decide the differentibilit of function Let us stte nd prove this result crefull Theorem 7 A function f : R n R m, f f 1,, f n, is differentible t if nd onl if ech of its component functions f i : R n R is differentible t In tht cse, we hve f 1 Df 1 Df D 10 f m Df m Tht is, to compute Df we cn just compute the 1 n derivtive mtrices of the f i first, which we know re of the form Df i i 1 i n, nd enter the result into the ith row of the lrger mtri Proof: This follows from the inequlities i n m 1 i n i for ll i, since if f is differentible t, then the limit eists, so the first inequlit bove implies tht the limit of zero eists in ech of the coordintes, nd so for ech of the coordinte functions Indeed, b tht limit we must hve tht Df i is the ith component function of Df Conversel, if the component functions re differentible t, then multipling the limit for f i b n nd using the second inequlit bove we hve tht the limit for f holds s well just choose the f i with mimum bsolute vlue, nd moreover we must hve tht Df i re the coordinte liner functionls of Df b the first inequlit Now tht we know how to compute Df if we know tht Df eists, we hve to nswer the question, How do we determine the eistence of Df? Well, we hve seen tht it boils down do determining the eistence of the m seprte totl derivtives of the component functions Df i The remining question, therefore, is, How do we determine the eistence of the m seprte totl derivtives Df i of the component functions f i? The nïve nswer is, Well, just compute the prtils i t of ech f i nd put them in mtri, unfortuntel, is not entirel correct It would be if we knew tht the prtils were lso continuous on neighborhood of the point, but not otherwise Here is n emple of wh the eistence of the prtils i t lone is not enough to conclude the eistence of Df we must lso hve their continuit: Emple 8 Consider the function f : R R given b f, 4, if, 0, 0 + 0, if, 0, 0 13

14 First, notice tht ll its directionl derivtives eist t the origin, for if v h, k is n vector in R, then the directionl derivtive D v f0 is computble directl: f0 + tv f0 th tk 0 D v f0 lim lim 1 t 0 t t 0 th 4 + tk t lim t 3 h k h t 0 t 3 t h 4 + k if k 0 k 0 if k 0 In prticulr, choosing v e 1 1, 0 nd v e 0, 1 shows tht it hs prtil derivtives 0,0 0,0 0 t the origin Outside the origin it is esil seen to be prtill differentible, nd its prtil derivtives eist everwhere on R, nd re given b , if, 0, 0 0, 0, if, 0, , if, 0, 0 0, 0, if, 0, 0 Thus, f is prtill differentible everwhere in R However, f is not differentible t the origin, in fct it is not even continuous there For notice tht on the prbol the function is constnt with vlue 1/: fh, h h4 h 4 1 so tht rbitrril close to the origin there re points for which f, 1/, while f0, 0 0 On the other hnd, long n stright line m the function stisfies f, m m 3 + m m + m so f pproches 0 long stright lines B one of our homework problems, however, ll differentible functions must be continuous, so we conclude tht f is not differentible t the origin We prove tht differentibilit implies continuit below! The problem here, of course, is tht the prtils For emple, line Check this! nd re not continuous t the origin pproches 0 long the prbol while it diverges to long the Remrk 9 The problem point 0, 0 isn t specil We could mke n point problem point, for emple 1, 5, b trnslting the bove emple function b 1, 5, ie b considering f, when, 1, 5 nd f0, 0 0, 0 OK, so now we know tht the mere eistence of the prtils i of f f 1,, f m isn t enough to ensure the eistence of Df Wht we need is the continuit of the prtils i on neighborhood of Let us prove this! Theorem 10 Let f : R n R m If ll the prtil derivtives i of f eist nd re continuous t, then f is differentible t 14

15 Proof: B Proposition 7 it is enough to prove this for the component functions f i of f Indeed, let f i be component function of f, nd suppose it s prtil derivtives ll eist nd re continuous in neighborhood of Then, since i moves onl in the jth coordinte direction, we need onl h j 0,, h j,, 0 in those directions B the definition of continuit of i, for n ε > 0 we choose there is δ > 0 such tht if h h j < δ then i i +h < ε h j n Let h be point in R n, so tht h h h n using our nottion bove B the Men Vlue Theorem from Clc 1, the continuit of f nd the eistence of the jth prtil implies the eistence of point + h j + t j e j between + h j nd + h j + e j such tht i f + h j f h j 11 +hi+t ie i Note: in the jth coordinte, keeping ll other coordintes fied, f j is rel-vlued function of single vrible, so this works Recll the MVT: If f is continuous on [, b] nd differentible on, b then there is point c between nd b such tht fb f f cb! As consequence, we hve i f i + h f i 1 i h < n i f i + h f i j1 j h j n i n i j1 j h j +h+tje i j1 j h i n i i +h+tje i h j j1 n j1 h ε where the first inequlit is from fctoring out h j nd then using the tringle inequlit, the second is b ppliction of 11 for ech j, nd the third b observing tht h h n h h n + + h h n n h Dividing the bove inequlit through b h j ε n h gives our desired inequlit, i f i + h f i 1 h We hve thus demonstrted the limit i f i + h f i 1 lim h 0 h i h < ε i h 0 which is the definition of differentibilit, nd moreover, in the course of the proof, we hve i lso shown tht Df i 1 s well! 15

16 Conclusion: If we know tht the component functions of f f 1,, f m re ech continuousl differentible on neighborhood of our point in R n, then we know tht the f i, nd therefore f itself, re differentible, nd the totl derivtive Df is in fct the m n mtri of prtil derivtives i! Emple 11 Let f : R 3 R be given b f,, z + z, e sin z + z Then f 1,, z + z nd f,, z e sin z + z re ech clerl continuousl differentible in ech prtil derivtive for emple, 1 is continuous on ll of Rn Therefore, f is differentible nd, s t 1, 1,, we hve Df1, 1, 1,1, 1,1, z 1,1, 1,1, 1,1, z 1,1, 1,1, 1 1,1, 1 1,1, e sin z + z 1,1, e sin z 1,1, e cos z + 1,1, 1 1 e sin + e sin e cos + 1 Moreover, if v 1, 4, is vector in R 3, then we cn compute the directionl derivtive of f t 1, 1, in the direction of v b simple mtri multipliction: D 1,4, f1, 1, Df1, 1, e sin + e sin e cos e sin + e cos 5 Further Properties of the Totl nd Prtil Derivtive Theorem 1 Chin Rule I Let f : R n R m nd g : R m R p be functions such tht g f is defined ie the imge of f is contined in the domin of g If f differentible t R n nd g is differentible t b f, then their composite g f : R n R p is differentible t nd their derivtive is mtri product, nmel the product of their two respective totl derivtives, Dg f Dg f Df 1 The components of the mtri Dg f in 1 m eplicitl be given b the formuls: g f i g i 1 1 b + + g i m m b 13 or, if we let z i : g i 1,, m nd k : f k 1,, n, z i z i z i m m 14 16

17 Theorem 13 Clirut: Equlit of Mied Prtil Derivtives If f : R n R m hs twice continuousl differentible prtil derivtives or equivlentl if for ll 1 i, j n the prtil derivtives f i nd f i eist on neighborhood of point nd re continuous t, then f i f i 15 for ll 1 i, j n Remrk 14 Filure of continuit t m led to inequlit of the mied prtils t Consider the function f : R R given b 3 3 f,, if, 0, 0 + 0, if, 0, 0 Then, , if, 0, 0 0, if, 0, , if, 0, 0 0, if, 0, , if, 0, 0 0, if, 0, 0 nd , if, 0, 0 0, if, 0, , if, 0, 0 0, if, 0, , if, 0, 0 0, if, 0, 0 Therefore, for, b 0 we hve 5b5 0,b b 4 b 5,0 4 nd consequentl f lim t 0 0,0 f lim t 0 0,0 0,t 0,0 t t,0 t 0,0 t 0 lim 1 t 0 t t 0 lim 1 t 0 t 17

18 nd so f 0,0 derivtives t 0, 0: f 0,0 The problem, of course, is the discontinuit of the second f f { , 4 if, 0, 0 0, if, 0, 0 { , 4 if, 0, 0 0, if, 0, 0 For emple, long the line we hve 1, so it pproches vlue of, while long the line 0 it sts constnt t 1, s noted bove f 18