RMI University ENDIX MRIX GEBR INRDUCIN Mtrix lgebr is powerful mthemticl tool, which is extremely useful in modern computtionl techniques pplicble to sptil informtion science. It is neither new nor difficult, but prior to the development of the electronic computer ws thought to be too cumbersome for prcticl pplictions. In tody's computer ge lrge msses of dt re ccumulted nd mtrix lgebr is convenient nd concise wy of expressing lgorithms nd computer routines for the mnipultion of dt. he dvntges of mtrix lgebr my be set out s:. It provides systemtic mens of storing nd mnipulting lrge rrys of dt. Such dt my rnge from numericl coefficients of equtions to chrcters nd symbols relted to scnned digitl imges.. It provides mens of reducing lrge nd complicted systems of equtions to simple expressions, which cn be esily visulised nd nlysed. 3. It provides concise method of expressing lgorithms nd of directing computer execution of those lgorithms vi computer progrms. DEFINIINS. Mtrix mtrix is set of numbers or symbols rrnged in squre or rectngulr rry of m rows nd n columns s mn, 3 n 3 n 3 3 33 3n m m m3 mn 005, R.E. Dekin Notes on est Squres (005)
RMI University letter or symbol refers to the whole mtrix. In mny texts nd references, mtrices re denoted by boldfce type, ie, X Q W Mtrices my lso be indicted by pcing tilde (~) under symbol, ie, X Q W ~ ~ ~ ~ ~ Exmple 6 5 9 C 3 5 x 4 3 8 8 5 4, C nd x re ll mtrices. Note tht the mtrix x is row mtrix or row vector. Row mtrices or row vectors re usully denoted by lowercse letters.. Mtrix element Individul elements of mtrix re shown by lowercse letters where the subscripts i nd j th indicte the element lies t the intersection of the i row nd the column. he first subscript lwys refers to the row number nd the second to the column number. mn, N M column 3 j n 3 j n i i i3 i j in m m m3 m j mn B c h j Q j th row i nother wy of representing mtrix is by typicl element, for exmple i,,, n ns j,,, m 005, R.E. Dekin Notes on est Squres (005)
RMI University.3 Mtrix order mtrix is sid to be of order m by n (or m, n) where m is the number of rows nd n is the number of columns. he order of mtrix my be expressed in vrious wys ie,,, f f mn mn m n m n m n Exmple 6 5 9 C 3 5 x 3,3 3,, 4 8 5 4 4 3 8 Mtrix is of order (3,3), mtrix C is (3,) nd x is (,4). If mtrix is of order (,), it is clled sclr. 3 YES F MRICES 3. Squre Mtrix squre mtrix is mtrix with n equl number of rows nd columns. squre mtrix would be indicted by nd sid to be of order m. Squre mtrices hve principl or mm, leding digonl whose elements re for i g, m, s nd y lie on the leding digonl. 55, b c d e f g h i j k l m n o p q r s t u v w x y j. In mtrix below, order (5,5), elements, Specil cses of squre mtrices re symmetric nd skew-symmetric which re described below. 005, R.E. Dekin Notes on est Squres (005) 3
RMI University 3. Column Mtrix or Column Vector column mtrix or column vector is mtrix composed of only one column. Column vectors re usully designted by lowercse letters, for exmple b m, b b b b 3 m 3.3 Row Mtrix or Row Vector row mtrix or row vector is mtrix composed of only one row. Row vectors re usully designted by lowercse letters, for exmple b b b b b,n 3 n 3.4 Digonl Mtrix digonl mtrix is squre mtrix with ll "off-digonl" elements equl to zero D mm, d 0 0 0 0 d 0 0 0 0 d 0 33 0 0 0 d mm where d 0 for i j digonl mtrix my hve some digonl elements equl to zero. digonl mtrix is often shown in the form l D dig d, d, d,, d m 3 q 3.5 Sclr Mtrix sclr mtrix is digonl mtrix whose elements re ll equl to the sme sclr quntity 0 0 0 0 0 0 0 0 0 0 0 0 where 0 for i for i j j 005, R.E. Dekin Notes on est Squres (005) 4
RMI University Exmple 3 W 0 0 0 0 0 0 is (3,3) sclr mtrix 3.6 Identity or Unit Mtrix n identity or unit mtrix is digonl mtrix whose elements re ll equl to (unity). It is lwys referred to s I where I 0 0 0 0 0 0 0 0 0 0 0 0 Note tht ll the "off-digonl" elements re zero nd ll the elements of the leding digonl re unity. 3.7 Null or Zero Mtrix null or zero mtrix is mtrix whose elements re ll zero. It is denoted by boldfce 0. 3.8 ringulr Mtrix tringulr mtrix is squre mtrix whose elements bove, or below, but not including the leding digonl, re ll zero. Squre mtrices whose elements bove the leding digonl re zero re known s lower tringulr mtrices. mm, l l l 0 0 0 l l l 3 3 33 0 0 l l l l m m m3 mm 0 where l 0 for i< j Squre mtrices whose elements below the leding digonl re zero re known s upper tringulr mtrices. U mm, u u u u 3 m 0 0 0 u u u 3 m u u 33 3m 0 0 0 u mm where u 0 for i> j 005, R.E. Dekin Notes on est Squres (005) 5
RMI University Exmple 4 G 4 3 0 5 4 3 0 0 3 0 0 0 5 6 0 0 0 0 H 8 0 0 3 4 0 5 6 G nd H re both tringulr mtrices. G is n upper tringulr mtrix nd H is lower tringulr mtrix. ringulr mtrices of order n hve cn + nh non-zero elements. 3.9 Unit ower ringulr Mtrix his is specil cse of lower tringulr mtrix, in which ll the elements of the leding digonl re equl to unity. mm, 0 0 0 l l 0 0 l 3 3 l l l m m m3 0 where l l 0 for i< j for i j 3.0 Unit Upper ringulr Mtrix his is specil cse of n upper tringulr mtrix, in which ll the elements of the leding digonl re equl to unity. U mm, 0 u u u 3 m u 0 0 u 3 m u 3m 0 0 0 where u 0 for i> j u for i j 3. Bnded Mtrix bnded mtrix is ny squre mtrix in which the only non-zero elements occur in bnd bout the leding digonl. hus, if is to be bnded mtrix 0 when i j > k typicl bnded mtrix of order 4 is 005, R.E. Dekin Notes on est Squres (005) 6
RMI University 3 0 0 0 3 33 34 0 0 0 43 44 where 0 for i j > 4 MRIX ERINS 4. Equlity wo mtrices nd B re equl if nd only if they re the sme order nd b for ll i nd j. Mtrices of different order cnnot be equted. 4. ddition he sum of two mtrices nd B, of the sme order, is mtrix C of tht order whose elements re c + b for ll i nd j. Mtrices of different order cnnot be dded. he following lws of ddition hold true for mtrix lgebr: commuttive lw + B B+ ssocitive lw + ( B+ C) ( + B) + C + B+ C Exmple 5 8 3 3 4 B 5 4 3 + B 3,3 3,3 5 6 5 6 7 0 5 6 8 8 5 9 8 4.3 Sclr Multipliction Multipliction of mtrix by sclr k is nother mtrix B of the sme order whose elements re b k for ll i nd j. Exmple 6 8 3 3 4 B 5 6 40. 5. 05. 5. 0. 0. 5. 0. 30. he following lws relting to sclr multipliction hold true 005, R.E. Dekin Notes on est Squres (005) 7
RMI University k( + B) k+ kb ( k+ q) k+ q k( B) ( k) B ( k B) kq ( ) ( kq) 4.4 Sclr roduct In vector lgebr, it is customry to denote the sclr product of two vectors i+ j + 3 k nd b bi+ bj + b3 k, i, j nd k being unit vectors in the direction of the x, y nd z xes respectively, s 3 b b b b b b + + 3 Q 3 3 In mtrix lgebr, set of three simultneous equtions represented s x b or 3 3 3 3 33 x x x 3 mens tht ech element of the column vector b is the sclr product of ech row of by the column vector x, i.e., x + x + x b 3 3 x + x + x b 3 3 x + x + x b 3 3 33 3 3 b b b 3 4.5 Mtrix Multipliction For three mtrices, B nd C with their respective elements, b nd c, then B C implies c b ik kj k which sttes tht the element in row i nd column j of C is equl to the sclr product of row i of nd column j of B It is importnt to note tht for mtrix multipliction to be defined the number of columns of the first mtrix must be equl to the number of rows of the second mtrix. 005, R.E. Dekin Notes on est Squres (005) 8
RMI University s quick method of ssessing whether mtrix multipliction is defined, write down the mtrices to be multiplied with their ssocited rows nd columns, ie, B 4, 6, nd check tht the "inner numbers" re the sme. If they re the sme, then the multipliction is defined nd the product mtrix hs n order equl to the "outer numbers". B C 4, 6, 46, Remember, tht in ll cses, the first number of the mtrix order refers to the number of rows nd the second number refers to the number of columns. Exmple 7 3 0 nd B, 3, 4 6 4 5 3 3 N M Q C B 3,4 3,, 4 5 7 3 3 9 3 8 6 4 ( 5+ ) ( 3 5+ 9 ) ( 6 5+ 4 ) ( 7+ 3) ( 3 7+ 9 3) ( 6 7+ 4 3) ( + 8) ( 3 + 9 8) ( 6 + 4 8) ( 3+ ) ( 3 3+ 9 ) ( 6 3+ 4 ) 9 3 7 7 33 48 75 7 38 54 38 6 In forming the product B we sy tht B hs been pre-multiplied by, or tht hs been post-multiplied by B. he following reltionships regrding mtrix multipliction hold: I I BC ( ) ( BC ) BC with I the Identity mtrix (ssocitive lw) B ( + C) B+ C (distributive lw) ( + B) C C+ BC (distributive lw) 005, R.E. Dekin Notes on est Squres (005) 9
RMI University In these reltionships bove, the sequence of the mtrices is strictly preserved. Note tht in generl, the commuttive lw of lgebr does not hold for mtrix multipliction even if multipliction is defined in both orders, ie, B B in generl Exmple 8 is of order (,3), B is of order (3,), B is of order (,) nd B is of order (3,3). Even if both mtrices re squre nd of the sme order, the results will in generl not be the sme when the order of multipliction is reversed. B 4 6 3 8 5 9 7 6 98 50 55 94 46 6 7 88 nd nd B B 7 3 9 5 6 8 4 44 46 8 56 60 09 68 80 04 If the product of two mtrices nd B is equl to the null mtrix 0 then it does not follow tht either or B is zero Exmple 9 5 4 3 6 3 6 8 ; B 7 ; B 5 4 6 8 0 0 0 0 0 0 his differs from ordinry lgebr where if, for exmple, b 0 then either or b or both nd b re zero. Some prticulr results involving digonl mtrices re useful. If is squre mtrix nd D is digonl mtrix of the sme order, then. D cuses ech column of to be multiplied by the corresponding element j d jj of D.. D cuses ech row i of to be multiplied by the corresponding element d of D. ii 005, R.E. Dekin Notes on est Squres (005) 0
RMI University Exmple 0 D 3 33 3 3 33 α β γ 3 α β γ 33 α β γ 3 3 33 nd D nd D α 0 0 0 β 0 0 0 γ α α α 3 β β β 33 γ γ γ If digonl mtrix D nn, hs non-negtive elements d ii 0 then for p > 0 p D nn, N M d p nd for p > 0 nd q > 0 0 0 0 p d 0 0 0 d p nn Q 3 3 33 DD D p q p+ q nd in prticulr DD D 4.6 Mtrix rnspose he trnspose of mtrix of order (m,k) is (k,m) mtrix formed from by interchnging rows nd columns such tht row i of becomes column i of the trnsposed mtrix. he trnspose of is denoted by. here re vrious other nottions used to indicte the trnspose of mtrix, such s: t *,, nd. If B then b for ll i nd j ji Exmple B 3 3 3 4 0 6 4 5 7 0 B 3 4 0 3 3 6 4 7 5 0 005, R.E. Dekin Notes on est Squres (005)
RMI University he following reltionships hold true + Bf + B ( BC ) C B ( k) k c h If D is digonl mtrix, then D D If H is sclr mtrix, then H H If I is the Identity mtrix, then I I If x is column vector, then x xis non-negtive sclr tht is equl to the sum of If x is row vector, the squres of the vector components. then xx is symmetric mtrix (squre) of the sme order s the vector x. 4.7 Biliner nd Qudrtic forms If x is n (m,) vector of vribles, y n (n,) vector of vribles nd n (m,n) mtrix of constnts, the sclr function u xy is known s biliner form. If x is n (m,) vector of vribles nd B n (m,m) squre mtrix of constnts, the sclr function q xbx is known s qudrtic form. n exmple of qudrtic form is the sum of the squres of the weighted residuls; the function ϕ vwv, which is minimised in lest squres. 4.8 Mtrix Inverse Division is not defined in mtrix lgebr. In plce of division, the inverse of mtrix is introduced. his inverse, if it exists, hs the property I squre his reltionship defines the Cyley Inverse for squre mtrices only. squre mtrix whose determinnt is zero is singulr nd singulr mtrix does not hve n inverse. squre 005, R.E. Dekin Notes on est Squres (005)
RMI University mtrix whose determinnt is non-zero is non-singulr nd does hve n inverse. Furthermore, if the inverse exists it is unique. Rectngulr mtrices hve no determinnts nd so they re tken to be singulr but they my hve n inverse (such s Moore-enrose inverses), defined using Generlised Mtrix lgebr. hese "generlised inverses" re not used in these notes. Consider the mtrix eqution x b. If nd b re known, then x my be determined from x b. x is found in sense, by "dividing" b by, but in ctul fct x is determined by pre-multiplying both sides of the originl eqution by the inverse. For exmple x b x b (pre-multiply both sides by ) giving x b (since I nd Ix x ) he following rules regrding mtrix inverses hold: ( BC ) ( ) C B ( ) ( ) ( α ) ( α is sclr) α Mtrix inversion plys n importnt prt in lest squres, primrily in the solution of systems of liner equtions. If the order of is smll, sy (,) or (3,3), then mnul clcultion of the inverse is reltively simple. But s the order of increses, computer progrms or softwre products such s Microsoft's Excel or he MthWorks MB re the pproprite tools to clculte inverses nd solve systems of equtions. For (,) mtrix the inverse is simple nd my be computed from the following reltionship If then ( ) 005, R.E. Dekin Notes on est Squres (005) 3
RMI University 4.9 Mtrix Differentition. If every element of mtrix of order (m,n) is differentible function of (sclr) vrible u, then the derivtive d du is n (m,n) mtrix of derivtives d du mn, d du d du d du d du d du d du d du n d du n d du m m mn Exmple N M Q 3 N M 3u u d 6u 6u then 4 3 Q u 4u du u 6u u x x M 4u 3 d u then 3u M 4 du 3 3u u. For the mtrix product C B where the elements of the mtrices nd B re differentible functions of the (sclr) vrible u then dc du is given by dc d d db ( B) B + du du du du Note tht the sequence dopted in the product terms must be followed exctly, since for exmple, the derivtive of B is in generl not the sme s the derivtive of B. 3. If vector y m, represents m functions of the n elements of vrible vector x then the totl differentil dy is given by dy y d x x n, where the (m,) nd (n,) vectors dy nd d x contin differentils dy dy dy dy dx m, n, m nd M dx dx Mdx N n 005, R.E. Dekin Notes on est Squres (005) 4
RMI University nd the prtil derivtive y x is n (m,n) mtrix known s the Jcobin Mtrix y x y x y x y x y x y x y x y x n y x n y x m m m 4. he derivtive of the inverse is obtined from hence d dx I d di c h 0 dx dx d + 0 dx n d dx d dx 4.0 Differentition of Biliner nd Qudrtic forms For the biliner form u xy where is independent of both x nd y u u y nd x x y For the qudrtic form q xbxwhere B is independent of x q x x hese differentils re given without proof, but cn be verified in the following mnner let M x y x3, Mx, y,, nd y x 3 3, N M Q 3 3 005, R.E. Dekin Notes on est Squres (005) 5
RMI University then u xy x x x 3 3 3 y x + x + x 3 3 x + x + x 3 3 N My yx + yx + yx + yx + yx + yx y y 3 3 3 3 nd u x y + y y + y y + y yf 3 3 y nd u y x + x + x x + x + x 3 3 3 3 x Using similr methods, the prtil differentil for the qudrtic form q cn lso be verified. More explicit proofs cn be found in Mikhil (976, pp.457-460) nd Mikhil & Grcie (98, pp.3-34). 4. Mtrix rtitioning subset of elements from given mtrix is clled sub-mtrix nd mtrix prtitioning llows the mtrix to be written in terms of sub-mtrices rther thn individul elements. hus, the mtrix cn be prtitioned into sub-mtrices s follows mn, N M 3 4 n 3 4 n 3 3 33 34 3n 4 4 43 44 4n m m m3 m4 mn Q 005, R.E. Dekin Notes on est Squres (005) 6
RMI University Considering the verticl dotted line only, could be written s where is n (m,3) sub-mtrix nd is n m,( n 3 ) sub-mtrix. Similrly, considering the horizontl dotted line only N M Q where in this cse is (,n) sub-mtrix nd is n ( m ),n sub-mtrix. Considering both the horizontl nd verticl lines Q where is (,3) sub-mtrix, is (,( n 3 )) sub-mtrix, is n ((,3) submtrix nd is n (( m ),( n 3) ) sub-mtrix. m ) ll mtrix opertions outlined in the previous sections cn be performed on the sub-mtrices s if they re norml mtrix elements providing necessry precutions re exercised regrding dimensions. Exmple 3 rnsposing prtitioned mtrices nd 3 4 5 6 7 8 9 0 Q N M 5 9 6 0 3 7 4 8 Q 005, R.E. Dekin Notes on est Squres (005) 7
RMI University Exmple 4 Multiplying prtitioned mtrices the product is where 3,4 M N Q 3 4 5 B 4 M 3 N M Q nd,, B C B B ( B + B) ( B + B ), 7 7 5 5 3 4 3 6 B N M 4 Q N M 0 0 Q N M ; 5 Q 5 3 7 3 6 7 9 ; B 4 5 N M noting tht columns of xx must equl rows of B xx. he product is B C 6 53 50 4 5 Q 4 B 7 6 B 5 N M Q N M, 38 40 7 4 Q 5 SME SECI MRICES 5. Symmetric Mtrices symmetric mtrix is defined to be mtrix tht remins invrint when trnsposed, ie, Symmetric mtrices re lwys squre mtrices. For ny symmetric mtrix, the elements conform to the following ji 005, R.E. Dekin Notes on est Squres (005) 8
RMI University Exmple 5 3 5 7 3 9 5 9 33 3 7 3 For ny mtrix nd for ny symmetric mtrix B the mtrices 44,, B nd B re ll symmetric mtrices. In lest squres, we re often deling with symmetric mtrices. For exmple, the mtrix eqution N B W B uu un nn nu often ppers. B is n (n,u) mtrix of coefficients of the u,,,, unknowns in n equtions, W is n (n,n) weight mtrix (lwys symmetric) nd N is the (u,u) symmetric coefficient mtrix of the set of u norml equtions. 5. Skew-symmetric Mtrices In contrst to the bove, skew-symmetric (or nti-symmetric) mtrix is defined to be squre mtrix tht chnges sign when trnsposed, so tht nd the elements conform to the rule ji Note tht this definition mens tht the elements of the leding digonl cn only be zero. n exmple of skew-symmetric mtrix of order 3 is 0 b c b 0 d c d 0 005, R.E. Dekin Notes on est Squres (005) 9
RMI University Exmple 6 Skew-symmetric mtrices re found in some surveying nd geodesy pplictions. For instnce, 3D conforml trnsformtion from one orthogonl coordinte system (x,y,z) to nother (X,Y,Z) is defined by the mtrix eqution Q X Y Z λ R κφω x y z + X Y Z where λ is scle fctor, X, Y nd Z trnsltions between the coordinte origins nd R κφω is rottion mtrix derived by considering successive rottions ω, φ nd κ bout the x, y nd z xes respectively R κφω cc cs + ssc ss csc φ κ ω κ ω φ κ ω κ ω φ κ cs cc sss sc + css φ κ ω κ ω φ κ ω κ ω φ κ s s c c c φ ω φ ω φ Note tht cκss φ ω cosκ sinφ sin ω, nd x, y, z nd X, Y, Z refer to the xes of right-hnded orthogonl coordinte systems. Rottions ω, φ nd κ re considered s positive nticlockwise ccording to the "right-hnd-grip rule". In mny pplictions the rottion mtrix R κφω cn be simplified becuse ω, φ nd κ re smll (often less thn 3 ). In such cses, the sines of ngles re pproximtely equl to their rdin mesures, the cosines re pproximtely nd products of sines re pproximtely zero. his llows the rottion mtrix R κφω to be pproximted by R S R S κ κ φ ω φ ω his mtrix is sometimes referred to s skew-symmetric mtrix. lthough the elements of the upper-tringulr prt re the opposite sign of those in the lower tringulr prt, it does not conform to the definition bove, since the leding digonl elements re not zero. 005, R.E. Dekin Notes on est Squres (005) 0
RMI University Note tht R S cn be expressed s the sum of the identity mtrix I nd skew-symmetric mtrix S R S κ φ κ ω φ ω 0 0 Q 0 0 + 0 0 0 κ φ κ 0 ω φ Q ω I + S 0 5.3 Symmetry nd Skew-symmetry Every squre mtrix cn be uniquely decomposed into the sum of symmetric nd skewsymmetric mtrix. Consider the following where nd + c h c h + + + Sym Skew ( + ) is symmetric becuse Sym ( + ) ( + ) Sym ( ) is skew symmetric becuse Skew ( ) ( ) ( ) Skew Sym Skew 5.4 rthogonl Mtrix n orthogonl mtrix is squre mtrix stisfying the following two conditions:. the norms of ll its rows nd columns re equl to unity, nd. ny row is orthogonl to every other row in the mtrix nd similrly for the columns. hese two conditions imply tht if is n orthogonl mtrix, then I (I is the Identity mtrix) nd hence n orthogonl mtrix hs the very useful property tht its inverse mtrix is the sme s its trnspose mtrix, or (if is orthogonl) he terms norm nd orthogonl re pplicble to vector lgebr. he norm of vector is the mgnitude of the vector nd is the squre root of the product of the vector nd its trnspose. 005, R.E. Dekin Notes on est Squres (005)
RMI University ny row (or column) of mtrix hs ll the chrcteristics of vector, nd hence the norm of ny row (or column) of mtrix is the squre root of the product of the row (or column) by its trnspose. wo vectors re orthogonl if, nd only if, their sclr product is zero. Considering rows nd columns of the mtrix s vectors, then ny two mtrix rows (or columns) re orthogonl if their sclr product is zero. Exmple 7 Rottion mtrices re exmples of orthogonl mtrices. For exmple, consider point with f coordintes e,n in the est-north coordinte system. If the xes re rotted bout the origin by n ngle θ (mesured clockwise from north), will hve coordintes e, n rotted system equl to e ecosθ nsinθ n esinθ + ncosθ hese equtions cn be written in mtrix nottion s e n cosθ sinθ sinθ cosθ cosθ sinθ where R θ sinθ cosθ N M e Q e n n or R θ e N M Q is known s the rottion mtrix. he norms of the columns nd rows of R θ re unity since sin n in the θ + cos θ nd the columns nd rows re orthogonl since sinθ cosθ sinθ cosθ 0. Hence R θ is orthogonl nd its inverse is equl to its trnspose. his is useful in defining the trnsformtion from e, n encoordintes. re-multiplying both sides of the originl trnsformtion by the inverse R, θ to gives nd N M Q N M Q e e e e Rθ Rθ Rθ or Rθ since Rθ Rθ I n n n n e e nq n cosθ sinθ R θ sinθ cosθ e n since R θ Rθ 005, R.E. Dekin Notes on est Squres (005)
RMI University ENDIX REFERENCES Mikhil, E.M. 976, bservtions nd est Squres, IE Dun-Donelley, New York. Mikhil, E.M. nd Grcie, G. 98, nlysis nd djustment of Survey Mesurements. Vn Nostrnd Reinhold, New York. Willims, I.. 97, Mtrices for Scientists, Hutchinson &Co td, ondon. 005, R.E. Dekin Notes on est Squres (005) 3