In micro and macro classes, matrices of derivatives will often show up. In the. 1 x ( 2. f x x

Transcription

1 XI. INEAR AGEBRA: MARICES FOR DERIVAIVES I always thik of atrices as copact ways of epressig systes of equatios, deriaties, obseratios, ad such. Matri algebra proides the tools for hadlig these large systes ery efficietly. I icro ad acro classes, atrices of deriaties will ofte show up. I the siplest case, thik of a real-alued fuctio of ariables, f:r R. It has partial deriaties. he colu ector cosistig of all these partial deriaties is the gradiet ector, ( )= f M We talk about gradiets oly for fuctios ito R. More geerally, the atri of first deriaties of a fuctio f:r R, also called the Jacobia atri, is: f f f f ( ( )) f f f f f( )= Df ( )= = f ( ( )) M M O M M f ( f( ) ) he atri of first deriaties is. Goig back to the case for R -alued fuctios, you will ofte see istead of gradiets, the row ector (ad to those who clai there are o such thig, I say atri ) of partial deriaties: Df ( )= [ f ]=[ ( ) ] here oly differece is geoetric orietatio. ( ) αw α w Eaple: et ( p, w)= ( ( p, p, w), ( p, p, w) )=,. What is p p Dpw (, )? w p Eaple: Now let pw (, )=,. What is Dpw (, )? p p he secod iportat atri i theory classes i the atri of secod-deriaties ad cross-partials of a real-alued fuctio of ariables, f:r R. he Hessia atri is the whose ij-th etry is f i j. O the diagoal of this atri, we hae secod deriaties; off the diagoals, we hae cross-partials. Sice i j = j i, the ij-th etry of the atri is the sae as the ji-th etry, so the Hessia atri is always syetric. Suer 00 ath class otes, page 77

2 f D f ( ) ( )= = f D f ( ) M ( ) ( )= M O M = D f( ) Hessia atrices are coparable to the secod deriatie of a R a R fuctio, ad they will always be used to test the cocaity of a fuctio of ore tha oe ariable. You should get ery accustoed to fidig Hessias. α α Eaple: F( K, )= AK. Fid DF( K, ) with respect to all the iputs. ρ ρ ρ Eaple: U(, )= ( + ). Fid DU(, ). Eaple: U(,, 3)= α β γ 3 ad pw (, )=( α + β + γ ) ( α wp, β w p, γ w p3 ). Fid V( p, w)= U( ( p, w) ) ad the D ppv( p, w). Eaple: U(, )= + l ad pw (, )= ( wp, p p). Fid V( p, w) ad the DppV( p, w). he test for cocaity or coeity of a fuctio of oe ariable was whether the secod deriatie was egatie or positie. We see that for fuctios of ore tha oe ariable, there is a atri of secod deriaties. How do we idetify whether a atri is positie or egatie? he aswer is ot to look to look siply at each of the i. It is also ot sufficiet to esure that each idiidual eleet is egatie. Eaple: Is the fuctio f(, y)= y coe? O first ispectio, it looks like the product of two coe fuctios. It s coe i each ariable idiidually; that is, both (ow) secod deriaties are positie. he Hessia atri of this fuctio is: y 4y Df( y, )=. All the eleets of this atri are positie. Ad yet, cosider this: at 4y two poits o the aes, f( 0, )= f( 0, )= 0. At a coe cobiatio of those two poits, we fid that f ( 0, )+ f( 0)= 0 / 6 = f(, ). his is ot a coe fuctio. (It looks like a giat scalloped bowl.) his deostrates that we eed a ew way to defied positie ad egatie for atrices, at least as far as secod-deriatie tests are cocered. Suer 00 ath class otes, page 78

3 Defiitio: A atri A is called positie seidefiite if for all ectors A 0. R, the uber Defiitio: A atri A is called egatie seidefiite if for all ectors A 0. R, the uber Defiitio: A atri A is called positie defiite if for all ectors 0, the uber A> 0. Defiitio: A atri A is called positie defiite if for all ectors 0, the uber A> 0. R, R, Ay atri that is ot oe of these is called idefiite. Note that if M is a scalar (which is really just a atri), these defiitios correspod to the usual defiitios of weakly or strictly positie or egatie, sice chages the sig of othig, proided 0. he rules for cocaity, stated last week, are that a fuctio is cocae if ad oly if its Hessia atri is egatie seidefiite; strictly cocae if ad oly if its Hessia atri is egatie defiite. Wheeer we are aiizig a fuctio of ore tha oe ariable, we ust fid the Hessia atri of the futio ad cofir that it is egatie seidefiite i order to esure that we hae foud a aiu. he defiitios are geerally hard to work with, whe doig these tests. here is a set of rules for deteriig the sig ad defiiteess of a atri. If we hae a atri A, a k-th order pricipal subatri of A is a atri that results fro deletig k rows ad the sae k colus fro A. If we hae a 3 3 atri, a a a3 A = a a a3 a3 a3 a33 the we ca for three secod order pricipal subatrices: by deletig the first row ad first colu, by deletig the secod row ad secod colu, ad by deletig the third row ad third colu: a a a 3 a a a 3 a a a 3 a a 3 a a a3 a3 a33 = a a 3, a a a3 a3 a33 a3 a3 a = a a, = a a a3 33 a3 a3 a a a 33 a3 a3 a 33 he leadig pricipal subatrices of A are oly those pricipal subatrices fored by deletig the last k rows ad colus of atri. For the 3 3 atri described aboe, the first, secod, ad third order leadig pricipal subatrices are: Suer 00 ath class otes, page 79

4 a a a 3 a a [ a ],, ad: a a a a a 3 a3 a3 a 33 he deteriat of a k-th order pricipal subatri is called a k-th order pricipal ior. Here is the relatioship betwee the priciple iors ad the sig ad defiiteess of a atri: heore: A atri A is positie defiite if ad oly if all its leadig pricipal iors are strictly positie. heore: A atri A is positie seidefiite if ad oly if all its pricipal iors (ot just leadig!) are oegatie. heore: A atri A is egatie defiite if ad oly if its leadig pricipal iors alterate i sigs, with the sig of the k-th order leadig pricipal ior equal to ( ) k. heore: A atri A is egatie seidefiite if ad oly if all its pricipal iors (ot just leadig!) of order k equal zero or hae the sig of k. his is what you hae to do to test the cocaity or coeity of a fuctio of seeral ariables. Fidig pricipal subatrices of two-by-two ad three-by-three atrices is t terribly difficult. Oce it starts gettig to four ad fie ad ore diesioal, it s a real pai. Ufortuately, there s ot really a better way to deterie cocaity. Oly a particularly sadistic professor would ask you to test the cocaity of a fuctio of ore tha three ariables. Here are the siple rules for two ad three ariable cases (just for cocaity). Suppose your Hessia atri is : A = a a a a o show that the fuctio is strictly cocae, you eed to show that: a a a < 0 ad > 0 a a For just plai cocaity, you to show that both of those hold with weak iequality, ad additioally that a 0. For a fuctio of three ariables, you hae a 3 3 Hessia atri. For strict cocaity, you eed to cofir that the leadig pricipal iors hae the right sigs: a a a3 a a a < 0, > 0, ad: a a a3 < 0 a a a a a ( ) For plai cocaity, you eed to show that all the secod deriaties are opositie: a 0, a 0, a 33 0 Suer 00 ath class otes, page 80

5 Ad that all the secod-order pricipal iors are oegatie: a a3 a a3 a a 0, 0, 0 a a a a a a Fially, check that the deteriat of the atri itself is opositie. Eaple: et f(, y)= y. Check for cocaity or coeity (strict or otherwise). Eaple: et f(, y)= y. Check for cocaity or coeity (strict or otherwise). Eaple: et f(, y)= + y + y. Check for cocaity or coeity (strict or otherwise). α α Eaple: et F( K, )= AK. Fid the atri DF( K, ) ad check for cocaity or strict cocaity. ρ ρ ρ Eaple: et U(, )= ( + ). Fid the atri DU(, ) ad check for cocaity or strict cocaity. Eaple: et U(,, 3)= α β γ αw βw γw 3 ad pw (, )= (,, p( α+ β+ γ ) p( α+ β+ γ ) p( α+ β+ γ )). Fid the atri DppV( p, w) ad check for cocaity or strict cocaity. w p Eaple: et U(, )= + l ad pw (, )= ( p, ). Fid the atri p D V p, w pp ( ) ad check for cocaity or strict cocaity. Okay, eough about cocaity. et s talk briefly about eigealues ad eigeectors, which will be useful for checkig the stability of systes of differetial (or differece) equatios. I acro, you ight hae a atri that describes how seeral ariables i the ecooy eole, for istace: k t + a a kt = t + a a t where k t ad k t + describe the capital stock of the coutry at tie t ad at t+; t ad t + are real oey balaces. he atri A cosists of costats, or likely as ot, liear first-order approiatios of soe fuctios (reeber the aylor series?). If we wat to fid the alues of these two ariables two years i the future, we would use the forula iteratiely: k t + a a kt + a a a a kt = t + a a = t + a a a a t Ad the to fid the alue of the ariables years ito the future we just repeat this ultiplicatio ties: k t+ a a kt kt = t+ a a = A t t Suer 00 ath class otes, page 8

6 Does this settle dow at soe poit, or do the ariables keep growig foreer? Reeber fro earlier that if we ultiply ay (, ) by itself a uber of ties, it gets really sall, ad: 0 as I acroecooics we ight be woderig soethig ery siilar, ecept that the uber has bee replaces with a atri A. I order to see whether this atri coerges or ot, we hae to look at its eigealues. Gie a atri A, a scalar λ is called a eigealue or characteristic alue of A if there eists a ozero ector R (called the eigeector or characteristic ector) such that: A= λ Here are soe alteraties characterizatios of a eigealue. heore: he followig stateets are equialet:. λ is a eigealue of A.. ( A λi) = 0 has a solutio other tha = A λ I is sigular 4. A λi = 0. Matrices ofte hae ultiple eigealues. I fact, alost all atrices hae distict eigealues. Gie that a atri A has eigealues λ, λ, K, λ, the followig are true: λ = ( ) i= tr A ad: λ = A he large capital pi idicates product oer a buch of ariables, just as a large capital siga idicates the su. We ca ake a few obseratios (idirectly) fro these properties. First, a square atri A is iertible if ad oly if zero is ot a eigealue of A. Secod, if λ is a eigealue of A ad A is iertible, the λ is a eigealue of A. hird, if A ad B are both atrices, the the eigealues of AB are the sae as those of BA. he fourth characterizatio of a eigealue is usually the easiest to work with i order to sole for the. I the case, we hae that: A = A I a b a = λ b λ 0 = 0 c d c d λ his eas that the eigealues are roots to the quadratic equatio: ( a λ) ( d λ) cb = 0 λ ( a + d) λ + ( ad cb)= 0 Cosultig Sydsæter, Strø, ad Berck (page see!) for the quadratic forula, we ca write the solutios to the eigealue proble as: λ = tr( A)± ( tr( A) ) 4 det( A) i= Suer 00 ath class otes, page 8

7 his leads to a proble, that eigealues are ot ecessarily real ubers, sice the ter uder the radical is ot ecessarily positie. What we are iterested i is the odulus of ay cople eigealues. If we hae a cople uber z = + yi, where ad y are real scalars, the odulus or agitude of z is defied as z = + y. his is like the legth of the ector z i the cople plae. If a uber is strictly real, the its odulus is its absolute alue. All of this stuff is ecessary for this result, iportat for testig stability. heore: All eigealues of a square atri A hae oduli strictly less tha zero if ad oly if A 0 as. Corollary: If A or A, the A does ot coerge. his secod result cofirs the case whe is a scalar ad 0, the does ot coerge. Agai, we see soe relatioship betwee a deteriat of a atri ad the absolute alue of a scalar. he last thig o the ageda for this lecture is to do a proble workig with atrices. You will hae to do this frequetly i ecooetrics i fact, this eaple is the fudaetal priciple behid liear regressios. Soewhere out i the world, there is a relatioship betwee soe depedet ariable y i ad soe other ariables. We would like to describe this relatioship as a affie fuctio of ariables ad a costat, ore or less: y = β + β + β + + β + ε i 0 i i K i i Because we hae > obseratios, we ca t eactly sole this syste of equatios (we hae ore equatios tha ukows). herefore, we ll hae to say that soe of each alue of y i is eplaied by soe outside, uobserable thigs captured i the ter ε i that hae absolutely o bearig o aythig we re actually iterested i. We obsere the alues of each of the ij ad the y i. he questio is to fid alues of β j, so that we ca blae as little as possible of the outcoe of uobsered stuff o the ε i. First, though, let s write this proble i atri for: y y M y = M O M β 0 β M β ε + ε M ε ( + ) ( + ) he I rewrite this as: Y = X + β ε ( + ) ( + ) Here, beta ad epsilo are ectors. he proble will be that we wat to fid the alue of beta that iiizes the size of the ector epsilo. Recall that for a k arbitrary ector z R, its size or legth is defied as: Suer 00 ath class otes, page 83

8 z = z + z + K + z = z z Essetially, the proble is to sole: i ε ε β R ( ) k Ad thus begis our first eercise i workig with atrices. First of all, I reeber that iiizig a fuctio is the sae thig as iiizig a strictly ootoic trasforatio of that fuctio, so I square the objectie. Also, I ake a substitutio. i ε ε i β ( )= β β β ( ) ( R R Y X Y X ) he I ultiply out the ter i paretheses, keepig i id that the usual forula for squarig a ter does t work for atrices (sice atri ultiplicatio is ot coutatie, right?). i β β + β β β R ( Y Y Y X X Y X X ) Sice I hae a objectie fuctio that I wat to iiize with respect to the ector beta, I a goig to take a deriatie ad set it equal to zero. Here is a rule for takig the deriatie of a liear fuctio of a ector, whe the ector is trasposed: z z z z ( M)= M he the first order coditio for this proble is to fid β to sole: ε ( ε) = Y X Y X+ β X X+ β X X = 0 β Collectig ters ad trasposig, we wat to fid: X Y+ X Xβ = 0 X Xβ = X Y ( X X) ( X X) β = ( X X) ( X Y) X X X Y β = ( ) ( ) With a little atri algebraic aipulatio, we hae deried the epressio for the estiator of the coefficiets i least-squares, liear regressio odel. At least, we e foud a critical poit the proof that this is ideed a iiu is left to the reader. I soe ways, workig with systes of equatios writte i atri for is oly a bit ore coplicated that workig with a sigle ariable. Refereces: Harille, Matri algebra fro a statisticia s perspectie Greee, Ecooetric aalysis (Chapter ) Ees, Eleetary atri theory Suer 00 ath class otes, page 84