A Mtrix Algebr Primer Mtrices, Vectors nd Sclr Multipliction he mtrix, D, represents dt orgnized into rows nd columns where the rows represent one vrible, e.g. time, nd the columns represent second vrible, e.g. mss. Ech intersection of row nd column hs numeric vlue clled n element, d r,c. Ech element represents prmeter whose vlue depends upon the row nd column vribles, e.g. counts. A column vector, v, cn be thought of s one-column mtrix. Unless otherwise noted, ll vectors re ssumed to be column vectors. Multipliction by sclr is strightforwrd. rnspose 0 8 D8 v 8 v 6 8 8 he trnspose, E, of the mtrix, D, is obtined by converting ech row into corresponding column. he trnspose, w, of column vector, v, is row vector. o sve spce in written mteril vector, v, might be defined using the nottion, v =. his llows it to be written horizontlly, s shown below, insted of consuming verticl spce. 8 ED 8 0 8 w v Vector Dot Product he dot product of two vectors is the sum of the element-by-element products. he result is sclr. 0 v 8 v s vv 0 8 8
Mtrix Multipliction Mtrix multipliction depends upon the ordering of the two mtrices. ht is, DE does not ordinrily give the sme result s ED. Additionlly, the number of columns of the left multiplicnd hs to be equl to the number of rows of the right multiplicnd. he number of rows of the product mtrix is equl to the number of rows of the left multiplicnd. he number of columns of the product mtrix is equl to the number of columns of the right multiplicnd. A useful mtrix nomenclture provides the number of rows nd columns below the mtrix symbols, (rows, columns). Note tht this cn be used to check row nd column restrictions. P E D P D E (, ) (, 7)( 7, ) ( 7, 7) ( 7, )(, 7) Using the bove mtrices, the esiest multipliction to perform by hnd is P. 8 0 8 0 8 5 0 0 6 8 0 0 0 3 6 8 P 8 8 0 0 80 6 3 6 8 6 3 6 80 0 0 8 6 3 0 0 0 8 6 0 0 5 hus, p,, is the dot product of the first row of D nd the first column of E. Likewise, p, is the dot product of the first row of D nd the second column of E, nd p, is the dot product of the second row of D nd the first column of E. It tkes lot more mth to compute P. 0 8 8 8 06 8 P 0 8 8 05
Mtrix Inversion here is no such thing s mtrix division. Insted one computes the inverse of mtrix nd multiplies by the inverse. he procedure used to invert mtrix is complicted nd t the "Primer" level not worth worrying bout. he inverse, I, of P, is given by the following. Note tht I P = P I =. (he symbol s used here does not denote cross-product.) 0.058 0.006 06 8 I P 0.006 0.060 8 05 0 IP PI 0 An inverse is only defined for squre mtrix. Not ll squre mtrices hve n inverse. Digonl Mtrix A digonl mtrix hs zero elements except long the mtrix digonl. An identity mtrix hs digonl of ll ones. 3 0 0 0 0 0 0 0 0 0 0 0 0 A I 0 0 5 0 0 0 0 0 0 0 0 0 0 0 Lest-Squres Mtrix Formlism Suppose tht set of dt re to qudrtic eqution, y = 0 + x + x. If the x-vlues chosen were 0,,, 3 nd, predictor vrible mtrix, X, cn be defined. Let the dt vector be denoted y nd coefficient vector by. 0. 0 0.05 0 3.88 9.6 3 9 y X 5.8 6 he lest-squres reltionship cn then be written using mtrix nottion. y X (5,) (5,3) (3,) Since X is not squre, the mtrix eqution hs to be solved using the following steps. 3
y X (5,) (5,3) (3,) X y X X (3,5) (5,) (3,5) (5,3) (3,) X X X y X X X X (3,5) (5,3) (3,5) (5,) (3,5) (5,3) (3,5) (5,3) (3,) X X X y (3,) (3,5) (5,3) (3,5) (5,) If you wnt to "follow long" with the lest-squres clcultion here re the ssocited numeric vlues. 9.99 X y 99.9 35.8 5 0 30 X X 0 30 00 30 00 35 0.8857 0.773 0.86 C X X 0.773.86 0.857 0.86 0.857 0.079 0.077 0.0009 0.988 he computed y-vlues, ŷ, nd the residuls, r = y - ŷ, re given below. yˆ 0.077.059.0 8.96 5.878 r 0.083 0.009 0. 0.99 0.0757 C is clled the vrince-covrince mtrix,. If this mtrix is multiplied by the vrince of the, σ, the digonl elements, c i,i re the vrince of the coefficients nd the off-digonl elements, c i,j, re the covrince of the coefficients. Note tht c i,j = c j,i, tht is, C is symmetric bout the digonl thus cov( 0, ) = cov(, 0 ). c 0, c, c 3,3 0 c, etc cov,.
he stndrd devition of the (in R the residul stndrd error) is obtined from the squred residuls nd the degrees of freedom (number of dt vlues minus the number of computed prmeters). s 5 yi yˆ i rr i 0.8 5 3 As noted bove, the digonl of C is used to compute coefficient errors. s s c 0.80.9 0.7 0, s s c 0.8.5 0.03, s s c 0.80.67 0.087 3,3 he Ht Mtrix he ht mtrix, H, is given by, H X X X X It is given this nme since it trnsforms the experimentl y-vlues into the computed lestsqures y-vlues. yˆ Hy he ht mtrix is useful for recognizing outliers in liner regression. Associted R Code See the R file, Liner Algebr Primer.R to see how ll of the bove computtions re chieved in R. 5