Lecture 2: Matrix Algebra

Transcription

1 Lecture 2: Mtrix lgebr Geerl. mtrix, for our purpose, is rectgulr rry of objects or elemets. We will tke these elemets s beig rel umbers d idicte elemet by its row d colum positio. mtrix is the ordered set. 2. Let ij R deote the elemet of mtrix which occupies the positio of the i th row d j th colum. The dimesio of the mtrix is defied or stted by idictig first the umber of rows d the the umber of colums. We will dopt covetio of idictig mtrix by cpitl letter d its elemets by the correspodig lower cse letter. Exmple Exmple 2. Exmple 3. Exmple mtrix is sid to be (i) squre if # rows = # colums d squre mtrix is sid to be (ii) symmetric if ij = ji i, j, i j. Exmple. The mtrix is squre but ot symmetric, sice 2 = 2 3 = 2. The squre mtrix is symmetric sice 2 = 2 = 2, 3 = 3 = 4, d 32 = 23 =

2 4. The priciple digol elemets of squre mtrix re give by the elemets ij, i = j. The priciple digol is the ordered -tuple (,..., ). The trce of squre mtrix is defied s the sum of the pricipl digol elemets. It is deoted tr() = i 5. digol mtrix is squre mtrix whose oly ozero elemets pper o the pricipl digol. 6. sclr mtrix is digol mtrix with the sme vlue i ll of the digol elemets. 7. idetity mtrix is sclr mtrix with oes o the digol. 8. trigulr mtrix is squre mtrix tht hs oly zeros either bove or below the pricipl digol. If the zeros re bove the digol, the the mtrix is lower trigulr d coversely for upper trigulr. Remrk: The followig ottios for idictig m mtrix re equivlet ij i,..., j,..., m m,,, or. m 9. If mtrix is of dimesio, the it is termed row vector,. ii there is oly oe row, the row idex is sometimes dropped d is writte.. Sice mtrix of dimesio is termed colum vector,. Likewise, sice there is oly oe colum, this is sometimes writte s. 2

3 lgebric Opertios o Mtrices. Equlity. Two mtrices sy d B, iji, biji,,, j,, m j,, m re sid to be equl iff ij = b ij i, j. 2. dditio d Subtrctio. Tke d B s bove with the sme dimesios we hve B m m b b m m b b m m 3. Sclr Multiplictio. Let k R. k kij m. b b B b b m m m m. 4. Multiplictio. Two mtrices d B c be multiplied to form B, oly if the colum dimesio of = row dimesio of B. If this coformbility requiremet is met, the it is possible to defie the product B. I words, the colum dimesio of the led mtrix must equl the row dimesio of the lg mtrix, for coformbility. Exmple If 23 d B, the B cot be defied, but B c be defied. I order to precisely preset the mechics of mtrix multiplictio, let us itroduce the ide of ier (dot) product of two -tuples of rel umbers. Suppose x, y R. The the ier product of x d y is defied by i i 2 2 i x y x y x y x y x y Note tht x x 2 d x y y x. Tht is, the dot product is commuttive. Give i x i m mtrix, let us ssocite the k th colum of with the ordered -tuple Moreover ssocite the j th row of with the ordered m-tuple jo j jm ok,,. k,,. k 3

4 2 3 Exmple = (2, 4) 20 = (0, 4, 5) With this ottio i hd, cosider two mtrices B m mk which re coformble for multiplictio i the order B. The product B is the give by B m mk Tht is if B = C, the c m b b b b b. jl ji il i Note it must be tht B. k Exmple B k k b b b 2 b 2 22 m i m b i i b m i m i i i i b i b i ik ik b b b b b b b b i 2 i b i i b 2i i 2 i 2 i b i i2 b 2i i2 b b b b Exmple B B

5 Exmple 3. Suppose tht is l row vector = = ( 2 ) d B col b vector B b. Hece we hve b b b b i b i i This is sclr d the opertio is termed sclr product. Note tht b = b. (The sclr product is sme s the ier product of 2 row vectors.) Moreover suppose tht b is well-defied d give by b b,, while B b. The product m b b b m m m b b bm bm 5. The opertio of dditio is both commuttive d ssocitive. We hve m. (Com. Lw) + B = B + (ssocitive) ( + B) + C = + (B + C) The opertio of multiplictio is ot commuttive but it does stisfy the ssocitive d distributive lws. (ssocitive) (Distributive) (B)C = (BC) (B + C) = B + C (B + C) = B + C To see tht B B cosider the exmple 2 B 0,

6 B 4 4 B Geerlly, whe we tke the product of mtrix d vector, we c write the result s c = b. I this exmple, the mtrix is by d the colum vectors c d b re by. This product c be iterpreted i two differet wys. Tkig the cse of 22 mtrix, we hve b First, this c be compct wy of writig the two equtios = + 3b 4 = 3 + 2b. ltertively, we c write the reltioship s lier combitio of the colums of 3 b I the geerl cse where is, we hve c = b = b + + b, where i is the ith colum of. Further, i the product C = B, ech colum of the mtrix C is lier combitio of the colums of where the coefficiets re the elemets i the correspodig colums of B. Tht is, C = B if d oly if c i = b i. 7. Trspose of Mtrix. The trspose of mtrix, deoted, is the mtrix formed by iterchgig the rows d colums of the origil mtrix. 6

7 Exmple. Let = ( 2) the 2 2 Exmple 2. Let , the Properties : (i) () = (obvious) (ii) ( + B) = + B Proof: Let + B = C, the c ij = ij + b ij. Let c ij deote elemet of C. Clerly, cij c ji ji b. Let b ji, be elemets of d B respectively such tht ij d b b ji ij cij ji b ji ij b. ij ji Thus, the elemets of C d + B re ideticl. ij ij (iii) (B) = B Proof: Let, B the m mk b b B b b k k b b B k b b k 0 0k b0 m B km b0 k m 7

8 m m b00 b0 0 B. b b 0k 0 0k 0 7. The Idetity d Null Mtrices.. idetity mtrix is squre mtrix with oes i its priciple digol d zeros elsewhere. idetity mtrix is deoted I. Properties: (i) Let be p. The we hve I = I P =. Proof: Exercise (ii) Let be p d B be p m. The we hve Ip B I B B. p p p pm p (iii) IIII I. I geerl, mtrix is termed idempotet, p terms whe it stisfies the property =. b. The ull mtrix, deoted [0] is mtrix whose elemets re ll zero. Subject to dimesiol coformbility we hve Properties: (i) + [0] = [0] + = (ii) [0] = [0] = [0]. Proofs: Exercise Remrk : If B = [0], it eed ot be true tht = [0] or B [0]. 8

9 Exmple B It is esy to show tht B = [0]. 8. Sums of Vlues.. Let i represet colum vector of oes. The the sum of the elemets of y vector x is give by i x = i'x. i b. If ll of the elemets of x re the sme d equl to k, the x = ki. Further, i x = i'ki = ki'i= k. c. Obviously, if is costt, the i i x i = i x = i'x. Settig = /, we obti the simple me _ x = i'x. i 9. Determits d Relted Cocepts.. determit is defied oly for squre mtrices. Whe tkig the determit of mtrix we ttch sig + or - to ech elemet: sig ttched to ij = sig (-) i+j. 9

10 Thus, for exmple, sig 2 = -, sig of 43 = -, d sig 3 = +. b. The determit of sclr x, x,is the mtrix itself. The determit of 2 2 mtrix, deoted or det, is defied s follows: Exmple c. The determit of rbitrry 2 mtrix c be foud vi the Lplce Expsio process. I order to itroduce this process, let us cosider some prelimiry defiitios. Let. Defiitio.The mior of the elemet ij, deoted M ij, is the determit of the submtrix formed by deletig the i th row d j th colum. Exmple. Let be 2 2, M Moreover M, M d M Exmple 2. Let be M Defiitio. The cofctor of the elemet ij deoted C ij is give by i j ij M. 2 3 Exmple Let be 3 3. The C

11 Defiitio. The priciple mior of the priciple digol elemet ii, deoted PM i is the determit of the submtrix formed by retiig oly the first i rows d first i colums. The order of PM i is its row = col. dimesio. Exmple PM PM PM3. 0. Lplce Expsio: Let be. The i ij C ij (expsio by j th col) C j ij ij (expsio by i th row) Note tht evetully cofctors degeerte to the 2 2 cse. Exmple 3 3. Expsio by 2d col C i 2 i 2 i Next cosider expsio vi the 3rd row c j 3j 3j Let s check the two terms to see if they re equl. The middle term of the secod expressio is the sme s the lst term of the first expressio. Checkig the remiig two terms, we hve the followig. I the first cse I the secod cse Thus, they re the sme. Exmple 2. is 3 3 d give by

12 I this cse it is esiest to expd vi the first col.. Properties of Determits (i) (ii) The iterchge of y two rows (or two col.) will chge the sig of the determit, but will ot chge its bsolute vlue. Exmples of (i) d (i) # # , B B 2 (iii) The multiplictio of y p rows (or col) of mtrix by sclr k will chge the vlue of the determit to k p. (iv) The dditio (subtrctio) of y multiple of y row to (from) other row will leve the vlue of the determit ultered, if the lier combitio is plced i the iitil (the trsformed) row slot. The sme holds true if we replce the word row by colum. (v) If oe row (col) is multiple of other row (col), the vlue of the determit will be zero. (vi) If d B re squre, the B = B. 2

13 Vector Spces, Lier Idepedece d Rk of Mtrix. s poited out bove, mtrix is termed col vector d mtrix is clled row vector. I geerl we hve Def. -compoet vector is ordered tuple of rel umbers writte s row or s col. The i, i,,, re termed the compoets of the vector. The elemets of such vector c be viewed s the coordites of poit i R or s the defiitio of the lie segmet coectig the origi d this poit. 2. R c be defied s the collectio of ll vectors. It is the -fold product of R. 3. The two bsic opertios defied for vectors re sclr multiplictio d dditio. Recll tht for vector, k = k k b, d tht + b =. b The set of ll possible scle multiples of vector is the lie through the zero vector d. y give scle multiple of is segmet of this lie. We c illustrte these cocepts geometriclly. k 2 (k > 0) (k < 0) 0 Figure 3

14 2 + b = c b Figure 2 To fid c = + b geometriclly, we move "" prllel to the tip of "b" or coversely. vector spce is collectio of vectors tht is closed uder the opertios of dditio d sclr multiplictio. Clerly R is vector spce. 4. Lier Combitios of Vectors d Bsis Vectors. Def. set of vectors sp vector spce if y vector i tht spce c be writte s lier combitio of the vectors i tht set. b. set of vectors spig vector spce which cotis the smllest umber of vectors is clled bsis. This set must be must be lierly idepedet. If the set were depedet, the some oe could be expressed s lier combitio of the others d it could be elimited. I this cse we would ot hve the smllest set. 4

15 Def. set of vectors m,, R is sid to be lierly depedet if there exist sclrs i R, ot ll zero such tht 2 2 m m 0,, 0 0. If the oly set of i 2 for which the bove holds is m 0 the the vectors,, m re sid to be lierly idepedet. 5. From this defiitio we c derive the followig result: Propositio. The vectors,, m from R re lierly depedet iff some oe of the vectors is lier combitio of the others. Proof Suppose oe vector, sy, is lier combitio of the others. The 2 2 m m. Thus, 2 2 m m 0 with 0. Hece, the set is lierly depedet. k 0. Thus, Suppose the set is lierly depedet. The the bove coditio is stisfied d k k k k k m. k k k k d k hs bee expressed s li. combitio of the other vectors. Remrk. If the set of vectors uder cosidertio hs but oe member R, the is lierly depedet if = 0 d is lierly idepedet, if 0. Here, lier depedece mes 0 such tht = 0 = 0. Now, if is ot lierly depedet we hve tht the oly for which = 0 is = 0. Thus, 0 0 d 0. Propositio 2. No set of lierly idepedet vectors c coti the zero vector. Proof: Suppose tht = O. Set 2 = = m =0 d =. The m 5

16 2 2 m m 0 d the set is lierly depedet. Propositio 3. y subset of set of lierly idepedet vectors is lierly idepedet. Proof: Suppose subset of lierly idepedet set,, m, is lierly depedet. Let this subset be, k,. The,, k ot ll zero such tht 2 2 k k 0. Set k+,, m = 0. The m m 0, i ot ll zero d we cotrdict lier idepedece. Propositio 4. y superset of set of lierly depedet vectors is lierly depedet. Proof: Suppose tht subset of,, m is lierly depedet. Let this subset be give by,, k. The,, k ot ll zero such tht k k k 0. Set,, m 0 d the result follows. We the hve the followig defiitio for the bsis of vector spce: Def. bsis for vector spce of dimesios is y set of lierly idepedet vectors i tht spce. Remrk: C you see why this defiitio is equivlet to the oe give bove? I R exctly idepedet vectors c form bsis for tht spce. Tht is, it tkes idepedet vectors to crete y other vector i R through lier combitio. Exmple. Let,b be lierly idepedet i R 2. Let c be y third vector. We c show tht c c be creted from lier combitio of d b. We oly eed to select d 2 such tht + 2 b = c. Tht is we wish to fid i + 2 b i = c i, for i =,2. Solvig these equtios for the i we hve = (b 2 c b c 2 )/(b 2 b 2 ) d 2 = ( c 2 c 2 )/ ( b 2 b 2 ). 6

17 It is possible to solve for the i if ( b 2 b 2 ) 0. This is true if / 2 b /b 2. Tht is, c ot be scle multiple of b. 6. Def. The rk of m mtrix, r, is defied s the lrgest # of lierly idepedet colums or rows. Propositio. Give m mtrix, we hve (i) r mi, m (ii) lrgest # li idep. col. = lrgest # li idep. rows. Propositio 2. The rk, r of m mtrix is equl to the order of the lrgest submtrix of whose determit is ozero. (By submtrix we me mtrix selected from by tkig out rows d colums of.) Remrk. If is d r = the 0 d the rows or col ech form set of lierly idepedet vectors. Moreover if 0, the, from Propositio, there re lierly idepedet rows (col) i. We hve 0 rows (col) of re li idep. r =. Exmple. Fid the rk of You should obti rk of Subspces. The set of ll lier combitios of set of vectors is clled the vector spce sped by those vectors. 2. s exmple, the spce sped by bsis for R is R. Moreover, if,b, d c re bsis for R 3 d d is fourth vector i R 3, the the spce sped by,b,c,d is R 3. Obviously, d is superfluous. 7

18 3. Cosider two vectors, b R 3, where 3 = b 3 = 0. It is cler tht d b c ot sp R 3, becuse ll lier combitios of d b will hve third coordite equl to zero. While d b do ot sp R 3, they do sp tht subspce of R 3, mely the set of ll vectors i R 3 which hve zero third coordite. This subspce is ple i R 3 d it is clled two dimesiol subspce of R 3. Geerlly, the spce sped by set of vectors i R hs t most dimesios. If this spce hs less th dimesios, it is clled subspce of R or hyperple i R. Iverse Mtrix. Def. Give squre mtrix, the iverse mtrix of, deoted -, is tht mtrix which stisfies - = - = I. Whe such mtrix exists, is sid to be osigulr. If - exists it is uique. Theorem. mtrix is osigulr iff r =. Remrk. Now we hve the followig equivlece. Let be 0 rows (col) of li. idep. r exists. 2. Computig the Iverse.. Let us begi by ssumig tht the mtrix we wish to ivert is mtrix with 0. b. Def. The cofctor mtrix of is give by C. C ij Def. The djoit mtrix of is give by dj = C. c. Computtio of Iverse: dj. Exmple: Let , C C ij 2 9 3, 8

19 2 3 dj / 25 3/ / / I 2 / 25 3/ / 25 / Key Properties of the Iverse Opertio. (B) - = B - -. Proof: B - - B = B - IB = I d BB - - = I. It follows tht B - - is the iverse of B. b. ( - ) - =. Proof: - = I d - = I. Thus, the result holds. c. (') - = ( - )'. Proof: - = - = I. Trsposig d otig tht I' = I, we hve ( - )'' = I = '( - )'. d. I - = I. Proof: II = I. 4.. Note tht B = 0 does ot imply tht = 0 or tht B = 0. If either or B is osigulr d B = 0, the the other mtrix is the ull mtrix. Tht is, the product of two o-sigulr mtrices cot be ull. Proof: Let 0 d B = 0. The - B = B = 0. b. For squre mtrices, it c be show tht B = B, so tht, i this cse, B = 0 if d oly if = 0, B = 0, or both. Lier Equtio Systems, the Iverse Mtrix d Crmer s Rule.. Cosider equtio system with ukows x i, i =,,. 2. 9

20 x x x d 2 2 x x x d x x x d 2 I mtrix ottio this system c be writte s x = d, x d, x d,. If 0, - exists d we c write x d where ij x d I x d x d. x Thus, if there is o lier depedece i the rows or colums of the coefficiet mtrix we c obti solutio to the equtio system. Sice - is uique if it exists, this solutio is uique. Hece, esy wy to test for existece d uiqueess of solutio to set of lier equtios is to determie whether the coefficiet mtrix hs ovishig determit. 2. This solutio gives us vlues of the solutio vribles, i terms of -, i vector form. formul kow s Crmer s Rule gives explicit solutios for ech x i. If 0, we hve 20

21 2 x d dj d C C C C d d x d C d C d C d C x x C di C di i i i i Thus, x C di j ij i. Cosider the term C di ij i. Recll C ij ij i (exps. by jth col.). Thus di C d d ij i j Usig this ottio, we obti for j =,,, x j = j. (Crmer s Rule) Remrk. This method does ot ivolve computtio of -.

22 Exmple 3x 4x 0 2 6x x x 0 6 x x x 2 2 Let's check this by computig - Check C 4 x x x C dj / 2 4 / / 2 3/ Chrcteristic Roots d Vectors I. Let D be mtrix. Does there exist sclr r d vector x 0 such tht Dx = rx? If so, the r is sid to be chrcteristic root of D. Rewritig, (Dx - rx) = 0, or (*) [D - ri]x = 0. 22

23 x is clled chrcteristic vector of D. Clerly, if x is solutio vector, the so is kx for y vlue of k. To remove the idetermicy, x is ormlized so tht x'x =. The solutio the cosists of r d the ukow elemets of x. 2. Equtio (*) defies the mtrix [D - ri]. This is clled the chrcteristic mtrix of D. For (*) to be true, it is ecessry tht D - ri = 0, give tht x 0. (To see tht this is true, let = [D - ri] d suppose to the cotrry tht x = 0, x 0, d 0. The - x = 0 d x = 0, so tht we hve cotrdictio.) This coditio is clled the chrcteristic equtio of D: (**) D - ri = The chrcteristic equtio is th degree polyomil i r which hs roots. If D is symmetric, the these roots re rel umbers. 4. Exmple. Let D = [D - ri] = 2 r 2 2. Tkig the determit of the ltter d r settig it equl to zero, we hve (2 - r)(- - r) -4 = 0. Whece, r 2 - r - 6 = 0. This is qudrtic i r (2 d degree polyomil). It hs the solutio r, r 2 = ( 4 6 ) / = /2 5/2 = 3, Give tht D - ri = 0, it is cler tht there re ifiity of x stisfyig (*), for ech chrcteristic root. We c obti uique chrcteristic vector by ormlizig s 2 x i i, for ech root. Goig bck to the exmple, we hve, for the first root r = 3, Note tht r, r 2 = [-b/2] [(b 2 4c) /2 /2], for r 2 + br +c = 0. 23

24 [D - ri] x x = x x = x + 2x 2 = 0 2x - 4x 2 = 0. Note tht equtio is just multiple of equtio 2. They re ot idepedet s expected. ll tht we c coclude from these is tht () x = 2x 2. If we impose the ormliztio costrit 2 (2) x + x 2 2 =, the () d (2) give us two equtios i two ukows. Solvig 2 (2x 2 ) 2 + x 2 2 = x = 2/(5) /2 d x 2 =/(5) /2. The chrcteristic vector uder this ormliztio is writte s v = (v, v 2 ) = ((2/(5) /2, /(5) /2 ). Usig the sme method for r 2 = -2 (I this cse, x 2 = -2x from the chrcteristic equtio), we c show tht v 2 = (-/(5) /2, 2/(5) /2 ). Figure illustrtes this procedure. The v i re tke o the uit circle. 2 We tke the positive root for ech x i or the egtive root for ech x i. 24

25 2 x 2 = -2x x 2 = x /2 v 2 v - 2 Figure 5. Geerl Results for Chrcteristic Roots d Vectors. Chrcteristic roots of symmetric mtrix re rel, but eed ot be distict. b. For symmetric mtrix, chrcteristic vectors correspodig to distict chrcteristic roots re pirwise orthogol. 3 If the chrcteristic roots of symmetric mtrix re distict, the they form bsis (orthoorml bsis ) for R. We hve v i v i = d v j v i = 0. It is covetiol to form the followig mtrix of chrcteristic vectors correspodig to Q = [v v ]. It is cler tht Q'Q = I, so tht Q' = Q -. Geerlly, we hve Def. The squre mtrix is orthogol if - = '. Thus, the mtrix of chrcteristic vectors is orthogol. From the chrcteristic equtio, we hve tht 3 If the roots re repetig, the it is still possible to fid orthogol chrcteristic vectors. We igore this detil here d ssume distict roots. 25

26 r Q = QR, where R r To see this, ote tht Q = [v v ] = [r v r v ] = QR. It follows tht (*) Q'Q = Q'QR = R. Coditio (*) is clled the digoliztio of. Tht is, we hve foud mtrix Q such tht the trsformtio Q'Q produces digol mtrix. I this cse, the mtrix digolizig is the correspodig mtrix of chrcteristic vectors d the digol mtrix is the mtrix with 's chrcteristic roots log the digol. It is lwys possible to digolize symmetric mtrix i this wy. s exercise, it is useful to work through this process for the umericl exmple provided bove. c. For squre mtrix, we hve i. The product of the chrcteristic roots is equl to the determite of the mtrix. ii. The rk of is equl to the umber of ozero chrcteristic roots. iii. The chrcteristic roots of 2 re the squres of the chrcteristic roots, but the chrcteristic vectors of both mtrices re the sme. iv. The chrcteristic roots of - re the reciprocl of the chrcteristic roots of, but the chrcteristic vectors of both mtrices re the sme. Geerl Results o the Trce of Mtrix. We defied the trce of squre mtrix s the sum of the digol elemets. 2. The followig results re esily show.. tr(c) = c(tr()). b. tr(') = tr(). c. tr(+b) = tr() + tr(b). 26

27 d. tr(i k ) = k. e. tr(b) = tr(b). Remrk: The product rule c be exteded to y cyclic permuttio i product. tr(bcd) = tr(bcd) = tr(cdb) = tr(dbc). 3. It c lso be show tht the trce of mtrix equls the sum of its chrcteristic roots. Qudrtic Forms qudrtic form is homogeeous polyomil of the secod degree. It tkes the form x'x, where is symmetric d by, d x is by. We hve x'x = ij x i x j. Now x'x is termed egtive defiite if it is egtive for ll x 0. The form d the mtrix re termed egtive defiite i this cse. The mtrix defiig the qudrtic form is clled the discrimite of the qudrtic form. The defiitios for positive defiite is logous with the iequlity sig reversig. The coditio give bove is ot computtiolly coveiet for determiig tht form is defiite. However, there re more coveiet equivlet coditios. The first ivolves the pricipl miors of. Propositio. d its qudrtic form re egtive defiite if d oly if pricipl miors of order i re of sig (-) i. Propositio sttes tht < 0, [ ij ] i,j =,2 > 0, [ ij ] i,j =,2.3 < 0. s exmple, cosider the mtrix 27

28 / 2 / 2. Is this defiite? Propositio 2. d its qudrtic form re positive defiite if d oly if pricipl miors of order i re of positive sig. Propositio 2 sys tht > 0, [ ij ] i,j =,2 > 0, [ ij ] i,j =,2.3 > 0,, [ ij ] i,j =, > 0. other equivlet coditio is give i Propositio 3. mtrix is egtive (positive) defiite if d oly if ll of its chrcteristic roots re egtive (positive). Remrk: Semidefiite mtrices re defied s bove with replcig >. 28