square atrix is oe that has the sae uber of rows as colus; that is, a atrix. he idetity atrix (deoted by I, I, or [] I ) is a square atrix with the property that for ay atrix, the product I equals. he idetity atrix has a ii = 1 ad a ij = 0 for all i j. I other words, it has oes o its diagoal ad zeroes everywhere else: 1 0 L 0 I = 0 1 0 M O M 0 0 L 1 diagoal atrix is a square atrix that has the etry a ij = 0 for all i j. he idetity atrix is also a diagoal atrix. Most square atrices, but ot all, are ivertible. Give a atrix, its iverse is ay other atrix B with the property that: B = I B = I he iverse of is usually deoted by 1. Here are soe geeral properties of trasposes ad iverses: + ( ) = ( B) = B ( B) = + B 1 1 ( ) = B 1 1 1 1 1 ( ) = B ( ) = ( ) Note that the order of ultiplicatio chages whe passig the traspose or iverse through paretheses. he iportat questio to ask kow is how to idetify whether a atrix has a iverse, ad if so, how to calculate the iverse. hikig back to scalars, all ubers except for zero have a (ultiplicative) iverse; aely, 1 x. y uber which has a (strictly) positive agitude has a iverse. here is a siilar cocept to easure the agitude of a atrix called its deteriat. he deteriat of a atrix, deoted by or det( ), easures the size of the area spaed i R by vectors defied as the rows of the atrix. For a 2 2 atrix, the deteriat is defied by: = a 11 a12 = aa 11 22 aa 12 21 a21 a22 he deteriat ca be iterpreted as the area of the parallelogra i the drawig o the right. For a 3 3 atrix, the deteriat ca be calculated as: x 2 ( a, a 21 22 ( a, a ) 11 12 x 1 Suer 2001 ath class otes, page 71
a a a = a a a a a a 11 12 13 21 22 23 31 32 33 = a a a a a a + a a 22 23 21 23 11 12 13 a32 a33 a31 a33 a a a 21 22 31 32 his deteriat ca be iterpreted as the area iside the paralleloped spaed by the rows of the atrix. his leds ituitio to what happes whe a two rows (or two colus, it turs out) of a atrix are idetical, or siply ultiples of oe aother. x 3 ( a, a 31 32 ( a, a 11 12 (,, a a a 21 22 23 x 2 I the two diesioal case, we ed up with two vectors that lie o top of each other. he space that they spa is a lie, ad a lie has area of zero. I the three diesioal case, two vectors beig ultiples of each other eas that the atrix describes a (flat) parallelogra at best, which has a volue of zero as well. I both cases the deteriat is zero, which ca be thought of as a atrix with o agitude. hese atrices will ot be ivertible. (Deteriat as a easure of how orthogoal or how dissiilar are vectors coprisig the atrix) More geerally, i order to calculate the deteriat of a square atrix, we pick oe row or colu to expad alog. Usually, people pick the top row uless soethig else looks uch easier. Each eleet of this row or colu is ultiplied by the cofactor atrix of that eleet. For a atrix, the cofactor of the eleet a ij is the deteriat of the ( 1) ( 1 ) atrix that results fro reovig the i-th row ad j-th colu of, ultiplied by ( 1) i+ j. hat is: a 11 L a1 j L a1 a11 L a1 j L a1 M M M M M M i j = a a a i1 L ij L i ij = ( 1) + ai1 L aij L ai M M M M M M a a a 1 L j L a a 1 L j L a Where the greyed-out eleets have bee deleted. It is usually easiest to reeber that the sigs of the ultiple o the deteriats follow this alteratig patter: Suer 2001 ath class otes, page 72
+ + L + + + L + M O M + + Here are soe otes o the useful properties of the deteriat of a (square) atrix: Rule: If all the eleets i a row (or colu) of are zero, the = 0. Rule: If two rows (or two colus) of are iterchaged, the deteriat chages sigs but the absolute value is uchaged. Rule: If all the eleets i a sigle row (or colu) of are ultiplied by a costat c, the the deteriat is ultiplied by c. Rule: If two of the rows (or colus) of are proportioal, the = 0. Rule: he value of reais uchaged if a ultiple of oe row (or oe colu) is added to aother row (or colu). Rule: =, B = B, ad + B + B t this poit, we are priarily iterested i deteriats because they idicate whether a atrix ca be iverted. Recall the case for scalars (the real ubers): a scalar ca be iverted if ad oly if it has strictly positive agitude, i other words, that x > 0. It is o coicidece that both deteriats ad absolute value use the sae sig there is a relatio betwee the two, but keep i id that deteriats ca still be egative. heore: atrix is ivertible if ad oly if 0 atrix with deteriat of zero is called sigular. Obviously, a osigular atrix is oe whose deteriat is ozero, ad thus is ivertible. Provided that the atrix is osigular, its iverse ca be calculated by this forula: 11 21 L 1 12 22 2 1 1 = M O M 1 2 L he atrix o the right had side is kow as the adjoit atrix of the atrix, ad is coprised of cofactors of the eleets of. Note though that the cofactor of the ij-th eleet of is i the ji-th etry of adj( ). hough the forula for coputig the iverse is uwieldy (ad coputatioally itese!), the versio of it for 2 2 atrices is relatively usable: = a b 1 1 d b = c d ad bc c a Suer 2001 ath class otes, page 73
his is probably the ost you ll be expected to do o your ow. Let s take a detour away fro atrices, back to vectors. Colu vectors (as vectors usually are) are siply 1 atrices. We ultiply vectors ties each other alog the sae rules that we ultiply atrices, ad we ca also ultiply vectors by atrices. Cosider soe vectors x v x v x v K x v 1, 2, 3,, R of the for x v i x1i, x2i, x3 i, K, xi. We say that a vector z v R is a liear cobiatio of x v x v K x v 1, 2,, if there exist real ubers α1, α2, K, α such that: v v v v v z = α x + α x + α x + K + α x 1 1 2 2 3 3 = ( ) his is like a covex cobiatio, except we o loger have the restrictio that all the weights add to zero. Exaple: 10 is a liear cobiatio of 6 2 1 ad 1 1. he vectors x v x v x v K x v 1, 2, 3,, R are called liearly depedet if there exist real ubers α1, α2, K, α ot all equal to zero such that: v v v v v = α x + α x + α x + K + α x 0 1 1 2 2 3 3 Equivaletly, they are liearly depedet if ad oly if oe of the vectors ca be writte as a liear cobiatio of the others. he vectors x v x v x v K x v 1, 2, 3,, R are called liearly idepedet if the are ot liearly depedet. Note: If >, the the vectors x v x v x v K x v,,,, R are liearly depedet. 1 2 3 I other words, you ca ever have ore liearly idepedet vectors tha the diesio of your space. If we have vectors x v x v K x v 1, 2,, R, with x v i = ( x1i, x2i, K, xi), we ca ake a atrix with each colu equal to oe of these vectors (or each row equal to the traspose of a vector). he vectors x v x v K x v 1, 2,, are liearly idepedet if ad oly if the deteriat of this atrix: x x L x 11 12 1 v v x x L x v x21 x22 x2 [ 1 2 ]= M O M x1 x2 L x is ot equal to zero. Suer 2001 ath class otes, page 74
he axiu uber of liearly idepedet rows (which is equal to the axiu uber of liearly idepedet colus) of a atrix is called the rak of, which is deoted by rak ( ). ( ) { } Rule: If is a atrix, the rak i,. Rule: If is a atrix ad x v = 0 v for soe x v R, x v 0 v the rak( )<. Rule: If is a atrix, the is ivertible if ad oly if rak( )=. Rak is useful for describig whether atrices are ivertible, but it will be iportat i ecooetrics. For istace, if you have affie equatios i ukows, it is ipossible to deterie the values of ore tha i {, } ukows, ad eve fewer if soe of the equatios are liear cobiatios of each other. I fact, the uber of variables we ca idetify is the rak of the atrix fored fro all the fuctios. Deteriat ad rak are two iportat properties of atrices. third property is the trace of the atrix. he trace of a atrix is defied as the su of all diagoal eleets. Here are soe rules for traces: tr + B tr tr B B B tr ( )= ( )+ ( ) )= tr( ) ( α)= α tr( ) tr( )= tr( ) Oly a square atrix has a proper iverses, though ay o-square atrix ay have a geeralized iverse, a left-iverse or a right-iverse. If is a atrix, the left-iverse (if it exists) is the atrix L such that: L = I he right-iverse (if it exists) is the atrix such that: R = I If the atrix is ot square, it ca have at ost oe of these (a right-iverse if <, ad a left-iverse if > ). If it is square, they are the sae if they exist. s tt turs out, a geeralized iverse exists if ad oly if the atrix is of full rak; that is, rak i, (all the rows or colus are liearly depedet). ( )= { } Fially, there are soe square atrices that occur frequetly, ad have special aes ad special properties. syetric atrix is oe which is the sae as its traspose, =. he Hessia atrix (the atrix of secod derivatives ad crosspartials) of a fuctio is always syetric. skew-syetric atrix is oe which is equal to the egative of its traspose, =. I thik I reeber ecouterig oe of these i ecooetrics. Idepotet atrices are oe that are the sae whe ultiplied by theselves as by the idetity atrix, 2 = =. hese are useful i etrics oe. Suer 2001 ath class otes, page 75
2 atrix is ivolutive if it is its ow iverse, = I, ad orthogoal if it produces the idetity atrix whe ultiplied by its iverse: = I. Refereces: Harville, Matrix algebra fro a statisticia s perspective Greee, Ecooetric aalysis (Chapter 2) Eves, Eleetary atrix theory Suer 2001 ath class otes, page 76