ANOVA Model and Matrx Computatons Notaton The followng notaton s used throughout ths chapter unless otherwse stated: N F CN Y Z j w W Number of cases Number of factors Number of covarates Number of levels of factor Value of the dependent varable for case Value of the jth covarate for case Weght for case Sum of weghts of all cases The Model A lnear model wth covarates can be wrtten n matrx notaton as Y = Xβ + ZC+ e () where Y N vector of values of the dependent varable X Desgn matrx N p < β Vector of parameters p 6 of ran q p 6 6 6 6 Z Matrx of covarates N CN C Vector of covarate coeffcents CN e Vector of error terms N
2 ANOVA Constrants To reparametrze equaton () to a full ran model, a set of non-estmable condtons s needed. The constrant mposed on non-regresson models s that all parameters nvolvng level of any factor are set to zero. For regresson model, the constrants are that the analyss of varance parameters estmates for each man effect and each order of nteractons sum to zero. The nteracton must also sum to zero over each level of subscrpts. For a standard two way ANOVA model wth the man effects α and β j, and nteracton parameter γ j, the constrants can be expressed as α = β = γ = γ = non regresson j 0 α = β = γ = γ j = 0 regresson where ndcates summaton. Computaton of Matrces XX Non-regresson Model The XX matrx contans the sum of weghts of the cases that contan a partcular combnaton of parameters. All parameters that nvolve level of any of the factors are excluded from the matrx. For a two-way desgn wth = 2 and 2 = 3, the symmetrc matrx would loo le the followng: α 2 β 2 β 3 γ 22 γ 23 α 2 N 2 N 22 N 23 N 22 N 23 β 2 N 2 0 N 22 0 β 3 N 3 0 N 23 γ 22 N 22 0 γ 23 N 23 The elements N or N j on the dagonal are the sums of weghts of cases that have level of α or level j of β. Off-dagonal elements are sums of weghts of cases cross-classfed by parameter combnatons. Thus, N 3 s the sum of weghts of
ANOVA 3 Regresson Model cases n level 3 of man effect β 3, whle N 22 s the sum of weghts of cases wth α 2 and β 2. A row of the desgn matrx X s formed for each case. The row s generated as follows: If a case belongs to one of the 2 to levels of factor, a code of s placed n the column correspondng to the level and 0 n all other columns assocated wth factor. If the case belongs n the frst level of factor, s placed n all the columns assocated wth factor. Ths s repeated for each factor. The entres for the nteracton terms are obtaned as products of the entres n the correspondng man effect columns. Ths vector of dummy varables for a case wll be denoted as d 6, =, K, NC, where NC s the number of columns n the reparametrzed desgn matrx. After the vector d s generated for case, the jth cell of XX s ncremented by dd jw =, K, and j. Checng and Adjustment for the Mean 6 6, where NC After all cases have been processed, the dagonal entres of XX are examned. Rows and columns correspondng to zero dagonals are deleted and the number of levels of a factor s reduced accordngly. If a factor has only one level, the analyss wll be termnated wth a message. If the frst specfed level of a factor s mssng, the frst non-empty level wll be deleted from the matrx for non-regresson model. For regresson desgns, the frst level cannot be mssng. All entres of XX are subsequently adjusted for means. The hghest order of nteractons n the model can be selected. Ths wll affect the generaton of XX. If none of these optons s chosen, the program wll generate the hghest order of nteractons allowed by the number of factors. If submatrces correspondng to man effects or nteractons n the reparametrzed model are not of full ran, a message s prnted and the order of the model s reduced accordngly. Cross-Product Matrces for Contnuous Varables Provsonal means algorthm are used to compute the adjusted-for-the-means crossproduct matrces.
4 ANOVA Matrx of Covarates ZZ The covarance of covarates m and l after case has been processed s 05 0 5 ZZ ml = ZZ ml - + w WZl - - ÊwjZlj WZm ÊwjZmj j= j= WW - where W s the sum of weghts of the frst cases. The Vector ZY The covarance between the mth covarate and the dependent varable after case has been processed s m05 m0 5 ZY = ZY + w W Y w Y W Z w Z j j m j mj j= j= WW The Scalar YY The corrected sum of squares for the dependent varable after case has been processed s 6 6 YY = YY + w WY wjyj j= WW 2 The Vector XY XY s a vector wth NC rows. The th element s N XY = Ywδ, =
ANOVA 5 where, for non-regresson model, δ = f case has the factor combnaton n column of XX; δ = 0 otherwse. For regresson model, δ = d 6, where d 6 s the dummy varable for column of case. The fnal entres are adjusted for the mean. Matrx XZ The (, m)th entry s N XZ m = Z m w δ = where δ has been defned prevously. The fnal entres are adjusted for the mean. Computaton of ANOVA Sum of Squares The full ran model wth covarates Y = Xβ + ZC+ e can also be expressed as Y = Xb + Xmbm + ZC+ e where X and b are parttoned as X= X Xm b and β= $ #. bm
6 ANOVA The normal equatons are then ZZ ZX ZX m X Z X X X Xm Xm Z Xm X Xm Xm $ # # $ C$ b$ # = b$ m ZY X Y X my $ # (2) The normal equatons for any reduced model can be obtaned by excludng those entres from equaton (2) correspondng to terms that do not appear n the reduced model. Thus, for the model excludng b m, Y = Xb + ZC+ e the soluton to the normal equaton s: $ ~ C ~ # = b $# ZZ ZX ZY X Z X X X Y $# (3) The sum of squares due to fttng the complete model (explaned SS) s ZY,, 6= $, $, $ X Y CZY $ b $ XY b $ mxmy X Y# = + + R Cb bm C b bm For the reduced model, t s ~ ~ ZY ~ ~ RCb, 6= C, b CZY bxy X Y$# = + m # $ The resdual (unexplaned) sum of squares for the complete model s RSS = YY RCb,, bm6 and smlarly for the reduced model. The total sum of squares s YY. The reducton n the sum of squares due to ncludng b m n a model that already ncludes b and C wll be denoted as Rb m C, b 6. Ths can also be expressed as m 6 m6 6 R b C, b = R C, b, b R C, b
ANOVA 7 There are several ways to compute Rb m C, b 6. The sum of squares due to the full model, as well as the sum of squares due to the reduced model, can each be calculated, and the dfference obtaned (Method ). R b m C, b C$ Z Y b$ X Y b$ ~ ~ 6= + + m X m Y C Z Y b X Y A sometmes computatonally more effcent procedure s to calculate R b m C, b b$ 6= m T m b$ m where b $ m are the estmates obtaned from fttng the full model and T m s the partton of the nverse matrx correspondng to b m (Method 2). $ # ZZ Ζ X ZX m X Z X X X Xm = X mz Xm X Xm Xm Tc Tc Tcm Tc T Tm Tmc Tm Tm $ # Model and Optons Notaton Let b be parttoned as b = M D $# = m m d M d M F F $ #
8 ANOVA where M m 6 m M D d d 6 D D 6 d C c 6 C C Vector of man effect coeffcents Vector of coeffcents for man effect M excludng m M ncludng only m through m Vector of nteracton coeffcents Vector of th order nteracton coeffcents Vector of coeffcents for the th of the th order nteractons D excludng d D ncludng only d through d d excludng d Vector of covarate coeffcents Covarate coeffcent C excludng c C ncludng only c through c Models Dfferent types of sums of squares can be calculated n ANOVA. Sum of Squares for Type of Effects Expermental and Herarchcal Covarates wth Man Effects Covarates after Man Effects Regresson Covarates Man Effects Interactons R6 C R MC d C, M, D 6 R4 9 RCM, 6 RCM, 6 R d C M 4 D 9 RCM 6 RM6 R d C M 4 D 9 6 RMCD, 6 R d C M 4 D 9 R CMD,
ANOVA 9 All sums of squares are calculated as descrbed n the ntroducton. Reductons n sums of squares 2RAB 67 are computed usng Method. Snce all cross-product matrces have been corrected for the mean, all sums of squares are adjusted for the mean. Sums of Squares Wthn Effects Default Expermental Covarates wth Man Effects Covarates after Man Effects Regresson Herarchcal Herarchcal and Covarates wth Man Effects or Herarchcal and Covarates after Man Effects Covarates Man Effects Interactons R c C R m C, M6 R d C, M, D, d R c MC, R m C, M6 same as default R c MC, R m M6 same as default 6 R m M6, C, D Rc 4 C 9 R m C, 4 M9 C 4, M9 R m 4 M9 R c MC,, D Rc R d C M D 6,,3 8 same as default same as default Reductons n sums of squares are calculated usng Method 2, except for specfcatons nvolvng the Herarchcal approach. For these, Method s used. All sums of squares are adjusted for the mean. Degrees of Freedom Man Effects df M F = = 6
0 ANOVA Man Effect 6 Covarates dfc = CN Covarate Interactons d r df r = number of lnearly ndependent columns correspondng to nteracton d r n XX Interactons d r df = number of ndependent columns correspondng to nteracton d r n XX Model F df = df + df + df Model M c r r=
ANOVA Resdual W df Model Total W Multple Classfcaton Analyss Notaton Y j n j W Value of the dependent varable for the th case n level j of man effect Sum of weghts of observatons n level j of man effect Number of nonempty levels n the th man effect Sum of weghts of all observatons Basc Computatons Mean of Dependent Varable n Level j of Man Effect n j Yj = Yj nj = Grand Mean Y = Yj W j
2 ANOVA Coeffcent Estmates 3 8 and 4 9 are obtaned as The computaton of the coeffcent for the man effects only model b j coeffcents for the man effects and covarates only model ~ b j prevously descrbed. Calculaton of the MCA Statstcs (Andrews, et al., 973) Devatons For each level of each man effect, the followng are computed: Unadjusted Devatons The unadjusted devaton from the grand mean for the jth level of the th factor: mj = Yj Y Devatons Adjusted for the Man Effects mj = bj bjnj W, where b = 0. j= 2 Devatons Adjusted for Man Effects and Covarates (Only for Models wth Covarates) 2 ~ ~ mj = bj bj nj W j= 2, where b ~ = 0.
ANOVA 3 ETA and Beta Coeffcents For each man effect, the followng are computed: ETA j j 3 8 2 = n Y Y YY j= 2 Beta Adjusted for Man Effects Beta n j m 2 j j= 2 4 9 YY = Beta Adjusted for Man Effects and Covarates Beta n j m 2 2 j j= 2 4 9 YY = Squared Multple Correlaton Coeffcents Man effects model Rm 2 = 6. R M YY Man effects and covarates model 2 Rmc = 6. R MC, YY The computatons of R(M), R(M,C), and YY are outlned prevously.
4 ANOVA Unstandardzed Regresson Coeffcents for Covarates Estmates for the C vector, whch are obtaned the frst tme covarates are entered nto the model, are prnted. Cell Means and Sample Szes Cell means and sample szes for each combnaton of factor levels are obtaned from the XY and XX matrces pror to correcton for the mean. 6 XY Y = =, K, CN XX 6 Means for combnatons nvolvng the frst level of a factor are obtaned by subtracton from margnal totals. Matrx Inverson References The Cholesy decomposton (Stewart, 973) s used to trangularze the matrx. If the tolerance s less than 0 5, the matrx s consdered sngular. Andrews, F., Morgan, J., Sonqust, J., and Klem, L. 973. Multple classfcaton analyss. Ann Arbor, Mch.: Unversty of Mchgan at Ann Arbor. Searle, S. R. 966. Matrx algebra for the bologcal scences. New Yor: John Wley & Sons, Inc. Searle, S. R. 97. Lnear models. New Yor: John Wley & Sons, Inc. Stewart, G. W. 973. Introducton to matrx computatons. New Yor: Academc Press.