UNIVERSITY OF NORTH CAROLINA Department of Statistics Chapel Hill} N. C. KIND OF MJLTICOLLII-lEARITY? AND UNBIASEDI-lESS AND RESTRICTED MONOTONICITY

Transcription

1 UNIVERSITY OF NORTH CAROLINA Department of Statistics Chapel Hill} N. C. MONOTONICITY OF THE POWER FUNCTIONS OF SOME TESTS FOR A PARTICULAR KIND OF MJLTICOLLII-lEARITY? AND UNBIASEDI-lESS AND RESTRICTED MONOTONICITY OF THE POWER FUNCTIONS OF THE STEP DOWN PROCEDURES 'OR MANOVA.ti'ID INDEPENDENCE by C. G. Khatri } April 1964 Contract~. AF-AFOSR This research was supported by the Mathematics nlvision of the Air Force Office of Scientific Research. Institute of Statistics Mimeo Series No ,.. ":'1""\1.."

2 MONOTONICIn' OP' THE POWER FUNCTIONS OF SOME TESTS FOR A PARTICULAR, KIND OF MJLTICOLLINEARITY, AND UNBIASEDNESS AND RESTRICTED MONOTONICITY OF THE POWER FUNCTIONS OF THE STEP DOWN PROCEDURES FOR MANOVA AND INDEPENDENCE by C. G. Khatri University of North Carolina and Gujarat University = == = = = = = = == = = == = ~ = = = = = = = = = = = = == = === = = = 1. Introduction: The problems of a particular kind of multicollinearity of means defined by Roy [7] and extended by the author [4, 5] to that of regression coefficients were reduced to a single form [5] in t he following way. Let the joint density function of the elements of the random matrices!: (p+q)x n, (n ~ p + q) and X: (p +q) X m be... where z: uxm is a known matrix of rank u(< m), Z: (p+q) X (p+q) is an unkno'wil N.- ~ covarian~e matrix and l3 (p+q)x u is an unknown regression matrix. Let us rj write (Y' ",1 and Then, to test the hypothesis of multicollinearity, against the procedures: ( -1) Ho: ~ 1. 2 =1'1 - ~12 E22 ~ 2 = }2 and 1 2: qx m. H o, given by alternative H(l3, f,.., 0), let us consider the following three test

3 2 (4) the likelihood ratio test [3, 5J whose acceptance region is p 7T (1 + c i ) :s )..1 ' a constant, i=l p (5) the trace test [3J whose acceptance region is 1: ci:s ~2' a constant; i=l and (6) the maximum root test [3, 4J whose acceptance region is a constant, where c l ~ c 2 ~.. ~ -1 roots of..eh~e ', h and ~e c p cl:s A3 ' are the characteristic (ch.) being the matrices of sums of products due to hypothesis and due to error, respectively, with (7),. h = [31 -!l!2(!2!2)-3: 2 ] [~+ 2C2(!2!~p-~2rl[~1-!1!2(!2!2)-~2]' and (8). e = 'xl~i - Jl!2(~2~2) -1 J2~i. For computational purposes, we may note that (7) can be rewritten as - S "...e We consider, in this paper, the step down procedures for two stages only. The results for more than two stages are similar. The step down procedures for MANOVA (refer J. Roy [6J) can be stated as follows: Let the first stage parameters be ~ 2 and the second stage parameters be ~l. 2 =.Pl - k2 h~ P2' Then, the null hypothesis for MANOVA is, and the alternative hypothesis is H( f2, ~9) = [H l ( El ~~)] U[H 2 ( El. 2 ~ 12) J. We propose below only three test criteria for testing (9), but many more can be stated depending on likelihood, trace and maximum root tests:

4 (10) (11) (12) An intersection procedure based on the likelihood ratio test, at each q stage, whose acceptance region is the intersection of II (1 + c 1.):s '\ 1 p j=l,j ) \ ' \,1 and \ being constants; and IT (1 + ci) :s i=l an intersection procedure based on the trace test, at each stage, q Z c 1. < \ 2 and j=l,j, ~ being constants; and acceptance region is the intersection of p Z c. < A_, A 2 and i=l ~ -"2 "1, an intersection procedure based on the maximum root test, whose at each stage, whose acceptance region is the intersection of c 1,l:S ~,3 and > c 1 -,q > c - p defined in (7) and (8). c1:s ~, \,3 and ~ being constants, where c 1, 1 ~ c 1, 2 ~.. are the characteristic roots of (32~2)(~2!e)-1 and c 1 ~ c 2 ~.. -1 are the ch. roots of, h~ with 2h and R. e as Now, the step down procedures at two stages for the independence of sets as considered by Roy and Bargmann [8] can be summarized as below: Let lsi = (~i ~2 ~3)' ~: (Pl + P2 + P3) X n, ~1: PlX n,,!2: P2x nand,33: P3x n have a joint density function as (13) MN(X; 0, Z ) "" rv '" where Jj 12 ~13 PI Z"3 P2,...,c is positive definite. Here the ~3 P3 P3 j-~:ut ;, stage parameters are k2 and the second stage parameters are (~3 and ~ 23)' That is

5 (14) H ( ~.. = 0 for i! j) a rv1.j ('V = H 01 ( k2 =2) n H 02 ( h3 =, E23 =,9) against the alternative H(~.. N1.J As before, we give below three test procedures: (15) ir i=l! 2) for i!j) = [H l ('&'2!.Q)] U[H 2 (Etf,g,E23!,Q)]. An intersection procedure based on the likelihood ratio test, at each p stage, whose acceptance region is the intersection of il2 (1 +w i.)<. 1,J - J= and (1 + w.) \,4 and 2 < '2, 4 being constants;,j - '2,4 ' (16) An intersection procedure based on the trace test, at each stage, whose p acceptance region is the intersection of ~ 2 W1. < ~ and,j - 'I 5 P3 t-, IMW. )..2.,5" j=l '.1:: w 2 j < '2 5 'I\'\ein g constants, and J=l', (17) an intersection procedure~ based on the maximum root test, at each stage, 4 \,4 whose acceptance region is the intersection of,,,, W2,1:S ~ 6' \ 6 and where w l 1 ~ UJ. 2 ~ ~ w 2, 1 2:: w 2 2 2:.. 2: w 2 J,P3 ~,6 being constants UJ.,1 :s \,6 and are the ch. roots of S S l "'1, h "'1, e -1 are the ch. roots of S2 h S2 with UJ., P2 rv, (V,e and (18) and ~2, e =t3.e3 - ~2,h. Results on the monotonicity of the test procedures when '1=0 and P3 = 0 in the above cases were first established by Roy and Mikhail [9, 10], Srivastava [11] and then by a different method due to Das Gupta, Anderson and Mudholkar [lj and Anderson and Das Gupta [2]. In this paper, we prove the monotonicity of the test procedures for multicollinearity and restricted monotonicity and unbiasedness of the step down procedures for MANOVA and independence. The orders of the identity matrix I and the null matrix 0 will be understood f'v tv by the context.

6 5 2. Multicollinearity and step down procedures for MANOVA: Lemma 1: The ch. roots of ShS-l and (X 2 X 2 ')(Y 2 YA)-1 are invariant under the rv...e {'1 '" ('J "'c.," -~ transformations (gl~ +,g2~) ~l' 23~2 kl' (Sl~l + JJ2r2) ~2 and!j3~ k2 where 21: PXP and -93: qxq are non-singular, El: noon " orthogonal matrices and 22: pxq. and ~2: nxn are Proof: Let,Bl = C,gl~h +,92:: 2),..;~, E 2 = g3~2 Sl' J l = (~h!l + E 2!2) ~2 and :J2 = JJ3~2 ~2' Then B2(~2'L2)-1 lt 2 = ~i ~2{!2~)-1c2 ~l ~l - jly2{~2!2) -~2 = gl~l~l +,92~2~1 - (~l}:l )!2(~2:2)-)c2~1 ' and ~lyi - Jl:t2(J2~P -~2i i = (gi*jl +,g2~) [~-!2(!2~P ""~2](21~~1 +,g2!2), = Gl[YlY 1 '- Y I Y 2 ' (Y2Y2') -ly2yl']g l ' rv "'... "'''' rlt' rj ~ N Moreover, the ch. roots of A~ are the ch. roots of BA except for some,~ '" tv zero ch. roots. (See Roy [7]). Using these results, the lemma 1 is obvious. Lemma 2: There exist matrices U: pxm, W: px{n-q), R: qxm and V: qxll '"... rj having a joint density function (20) MN(!!; fz,~) MN(~; 12, l) MN(~; ):2' J) MN{Y;, t), 1 ~ ( where ::.: pxm = (9 ij ), A ij = 0 for i F j, 8 ii i = 1, 2,... ) are the possibly non-zero ch. roots of~t-l.. = 0 for (i f j) and ~12 " (i=1,2, ) -,...,, J.J, 2J. -1 ) are the possibly nonzero ch. roots of (!1. 2~Z~ J3i. 2 rh.2 ' E 1. 2 = J::ll - J1.2 #;~ l1.2 ' and J12]2 is a diagonal matrix with diagonal elements ai which are the possibly nonzero ch. roots of

7 ( I -1) -1!2~Z~ 13 2 ~22 Moreover, c i ' s, the ch. roots of. hge,are the ch. roots of (~...:) (~i:)-1 and c l, j 's, the ch. roots of (~2~2)(.!2t2)-1, are the ch. roots of (RR')(VV,)-l. Proof: Let E =. :E' ' ""':er: '"'t = (~l 2 -i"" J3 = ";22' :E2 =~2 J22' Tl and T3 0' = (T-lt) Z)' = (a'. 2'), - l: pxm and..., N rv (V (Vl,_.- -1 and 02 = T t3 Z Then we can write tv rj 3 rj 2 (V 6 12 \ 2 : 3) such that ~l = ~ 1. 2 ' are symmetric matrices. Let -1 02: q,xm. Then 01 = T l t3 l 2Z IV N,.,.. tv.f'w ~l and,32 as (refer [3]), (20) where 11: PX.P, I2: q,xq" ~l: mxm are orthogonal matrices, ~ =(~;.J~) I ~l = (TIl'.): pxm, n l " = for i ~ j and n 1 2. are the possibly '", ~J, ~J, ~~ nonzero ch. roots of ~i,l = ~' pl 2 ~~: ~ 1'1. 2~ or,t:~:2!1. 2z,..Z~ ~i. 2 ',2'\2 = ~3~: q,xm, ~,: mxm is an orthogonal matrix and ~3 = (11 3, ij), 'n7( " =.J, ~J of ~ 11' "' (21) 30 =(;1) for i ~ j and a~ = ~32.. are the possibly nonzero ch. roots ~,~~ or J2.,2 or J2 ~~ ~2-1 J22 Let us apply the transformation =(~1 ;2);-J ~1 ~d!o =G1J =~1 ;2)~_1!- The jacobian of the transformation is J(X, Y; R, V ) IV,... ",0 rjo -1 and by lemma 1, the ch. roots of Sh S are invariant.,v.,je function of R and V can be written as e-jo NO (22) MN(V ; 0, I) MN(R;~, I) rjo N N ~o N rj = I zit (m+n) -v The joint density where 11' = (~i ~2) tv "".-

8 The matrix I - V'(VV,)-lV is idempotent of rank (n-q). Then we f""v l'v 'V ~.-- can write,6. - Y' (Y'L' )-lx = J-3 ~3 where k3: (n-q) x n is orthonormal 1 (1. e. j3 k3 =,!) such that (~3"'i' =. Let ~2 = ;L' (~,..; )'"2. Then (j3 ~2) is an orthogonal matrix. Using the transformation 7 the joint density function of!l' ~, ~,JIl and 2 2 can be written as MN(R;'fJ 2,I) (" rj "", -1 and c. IS, the ch. roots of ShS are the ch. roots of ~ (" "'e -1 1 L{ I+RI(VV')-~) L'(W..w,)-l, L = R l - W 2 (VV 1 )-2 R and t'-' tv"" "" "'" I'J -v ("ojl'''' 1 <V i'j ('oj tv tv '" MN(V;O,I), tv N tv the ch. roots of (~2~2)(!2!2)-1 are the ch. roots of -livdfrom (24), we may note;,the distribution of -»1 and 3 2 are independent normal. Hence the distribution of k, is normal with E(~) =]]1' To find the covari&~ce matrix, let. (i = 1, 2,..., p) be the i-th column of ""~ L ', Then it is easy to see that covariance matrix of. is ~ -~ I + R'(VV, )-lr and covariance matrix between. and '1 is zero~ 'V f'j rviv,..., ""~ IV~ Hence the distribution of L is rv Now (25) MN[L '; 1'1 ' ) I + R1 (VV I ) -1 R]. N ('V I ('V t"'tj tv t'v rv Using the transformation L[I + Rl(VV1)-1 R] tv tv f"'j. tv 'V ~ joint distribution of 21'!II' ~ and :!. as we get the (26) MN(El;2'~) MN(yl;~)!) MN(~;~2'!) MN(~;,~) 1 where E = 'fjl[i + Rl(VV 1 )-lrr2, and c. 's, the ch. roots of S S-l are ~j N N '" '" '" tv ~ ruh""e the ch, roots of (~l~i)(~l~])-1

9 8 Let us wr'ite (refer [3]), (27) where ~5 : noon. and = 0 for i ~ j f 3 : pxp are orthogonal matrices, and e: pxm = (0.. ), tv ('.' :1.J and e~. are the possibly nonzero ch. roots of S~1. :LJ. N <oj Finally, using the transformation (28) u (V and W = f 3 ' W l N '" '" 'we can write (26) as mentioned in the lemma 2. Thus, lemma 2 is established., ~heorem 1: An invariant test of multicollinearity for which the acceptance regi on is convex in each column vector of U for each set of fixed W, IV ('l fixed values of the other column vectors U has a power function Which is N monotonically increasing in each r. = ~l... 1.,:1.J. Proof: It follows from theorem 3 of Das Gupta, Anderson and Mudholkar [1] that for given R and V, the conditional probability of the acceptance r-j.-..j region monotonically decreases in each e~. (i = 1, 2,., m) are the :1.:1. -1 possibly nonzero ch. roots of 1 (~i ~l) {~+~,(yy,)-l ~ }, or!(~i :h)~ -2' where P =( I + R'(VV,)-lR ) is a symmetric matrix. Let ~f be the N I'V "",.yt'\j,..., matrix obtained from l'l by changing the diagonal elements of 211 from r. = 0 1 " to ~*l" = r* where rt > r i (i = 1, 2, ). Then :L, J.J., :1.:1. i J. - ~! ~!I ~l ~i is positive semi-definite at least, and, since n is ~ a non-singular symmetric matrix, we get P ~* ~*I P - P ~ ~I P tv ",1 ",1,..., <V ",,1 ",1 '" to be positive semi-definite, and so at. > 9 44 for each i, and conversely if J.: at. > e.., then rt > r.. Hence, for any fixed R and V, the conditional J.J. - J.J. :L - :L ~ N probability of the acceptance region decreases monotonically in each The theorem 1 now follows from the fact that the marginal distribution and :f, does not contain any r i rio R \ tv

10 Now let W k be the sum of all different products of (1+01)' (1+C 2 ),.., (l+c) taken k at a time (k = 1, 2,, p). Then by using theorem 1 p above and the lemmas 2 and :; of [1], we have the following corollaries: 9 ~j.j~tl,*,: The maximum root test given by (6) has a power function which is monotonically increasing in each r. ~ P A test having the acceptance region L: k=l has a power function which is monotonically increasing in each r.. ~ Q.Lo..l~: The power functj.on of the likelihood ratio test given by (4) increases as each r. increases. ~ r~!~i: as each r. increases. ~ Theorem 2: The power function of the trace test given by (5) increases An invariant test for which the acceptance region is sectionwise convex at each stage of the step down procedure for MANOVA is unbiased and moreover is monotonically increasing with respect to the ch. roots of the parameters at the i-th stage when the parameters of (i-l) stages are fixed and all the parameters above the i-th stage are zero. [This type of monotonicity will be called the restricted monotonic property.] This follows from theorem 1, lemma 2 and theorem :; of [1] for the two stages of the step dowm procedure for MANOVA. The generalization is immediate. Let W ij be the sum of all different products of (l+cl,l)'..., (l+c l ) taken j at a time (j = 1, 2,.., q). Then using theorem 2,,q we get the following corollaries: P r2;u.~: A test having the acceptance region L: ~ Wk:S IJ, and q k=l 1:: b. Wl.:s IJ, (~~O, b. ~O) of the step down procedure for ~ANOVA j=l J,J J at two stages is unbiased and has the restricted monotonic property.

11 10 11~: The test procedures given by (10), (11) and (12) are unbiased and each has the restricted monotonic property. It may be noted that the regions described above are all, in fact, not only conditionally sectionwise convex, but also conditionally sectionwise ellipsoids, so that, for proving the above results, we could as well have used Roy and Mikhail [9, 10]. 3. Step down procedure for independence: Let ' ; 3.1,2 = ~33 - z = I 1 (,J1.l ~12 )_l(r;~ol F - Z-2 (Z( Z') (Y - tv3.l, 2 1V13 rv 2 3 k2 re22 N 1 1- :NOVT, we use the transformation ~~l. ~2 X Y = Z- 2 X and,_ rjli lv l'",2 "'2.1"'2 1 Y3 = ~ X 3 in (13). Since the ch. roots of S. hsi-l (i = 1, 2) r.,:::;)., tv tv ~, tv, e given by (18) and (19) are invariant, we replace Y by X (i- (Vi (Vi - and then write the joint density function of.el' ~2 and,!3 as (29) MN%; B'~) MN(:2; El' 2) MN[!f3;!(~)',!l, where ~: P2XPI and!: P3x(Pl+P2) are defined above. Since " - <3i,W [(~) (isi )$2) J~~) and I - ~i (!h:si)-~ 1,2,3), are idempotent matrices of ranks (n - PI - P2) and (n - PI) respectively, we can write them respectively as ~l ~i and..93 ~3 where ~i: (n-pl-p2)xn and!'3: (n-pl)xn are orthon,ormal (1. e. ~i ~l = and '&3 ~3 =J)

12 such that 6. 1 ' (Xl' X 2 ') = 0 and 6.~ X 2 ' = O. Let us write t"v"..., 'V,..., I"\,I./I'V"..., ~()si 32) [(~~) (!Si.J}2) l-~ and ~4 = lsi%h) -!. Then (~l~) and (~3 '4) are orthogonal matrices. Now transform ~2 and ~3 by 11 Since the transformations are conditional in X, we have the 2 jacobian of the transformation J(~2' t:3; X!l' Ji2' :Y 1, $2) = JC.~2; )II' :12) J<"e3; ~l' :' 2) = 1, and so the density function of Jl' 2 1, ;G2' ;11 and.j 2 can be written as "'" XXI l!vl (X tv1""1 X')1. w~~ tv 2-1 w1e re G = I, the ch. roots of S S are,.., ( W(XX')2. WW'+WW' (Vl,hl"l,e ""2 ",1",1,...,2~ ""1"'1 the ch. roots of (~~) (li l 1:!i) -1, and the ch. roots of ~2, h~~~e are the ch. roots of Let us write ~V2~)(~1~i)-1 1. E(~l~l')~ = f 2 '1\lf f"w' IV rw l and FG=f:z;'l'l2 f 4 t'v fv r./ tv t"v ' where ' 1: P1Jq)1'!2: P 2 x:p 2 'I3: P 3 x:p 3 and!4: (P l +P 2 )x(p l +P 2 ) are orthogonal matrices, ~l = ('l'll,ij): P 2 x PI' ~l,ij = 0 for i f j and 2 '11 l,ii are the possibly nonzero ch. roots of [(~lj.sl.)!'1p] and ~2 =(~2,ij): P3 x (P l +P 2 ), ~2,ij = 0 for i ~ j and ~~,ii are the possibly nonzero ch. roots of (F IFG 2 ). Now, we apply the transformation fv.-v N and U 4 = f ~ ('V,v..,J V 2 f 4 '. ~ tv

13 Since the transformations are cc,nditional in ~l and l:!2' we get the jacobian of the transformation as J(~l' E 2,,!l' 12; 2 1, 2 2, By 24) = J(~l' 2 2 ; }21' B 2 ) J(!l' 2"2; ~3',E4) = 1. Hence the joint density function 12 of.el' ' 1' 2 2, 3 and ]4 is where the ch. roots of S S-l are the ch. roots of (~2V.2)( lv,i)-l and "'1,h""l, e the ch. roots of S S-l are the ch. roots of ~4~) ( 3~3rl. On rv2,hcj2,e account of (34), we may note the following which are derivable by the similar arguments as in Section 2. (35) If..31' ";2' ~ll' 212 and ~22 are fixed, then the power of the test due to the ch. roots of (~~)(~3~3)-1 increases with each ~2,ii and hence due to each ch. root of L:: l ( E - E.. ) ",,;>.1,2 "";33,w;>.1,2 (36) If,E13 = Q and. 23 = Q, and..31 is fixed, then the power of the test due to the ch. roots of (E2~)( 1~i)-1 increases with each 11 1, ii and hence due to each ch. root of re~~l(j'r 2 2 -!'2.1) From (35) and (36), we have the following theorem. Theorem 3: An invariant test for which the acceptance region is sectionwise convex at each stage of the step down procedure for independence is unbiased and has the restricted monotonic property. Let WI.,J be the sum of all different products of (1 + wi, 1), (1 + l.l~ 2)'..., (1 + IJ.l. ) taken j at a time (j = 1,2"..,1'2) and,.1.,1'2 W 2 k the sum of all different products of (1 + w 2 1), (1 + w 2 2)" d', (1 + ljj 2 1\!J?3) taken k 3, we have the following corollaries:,, at a time (k = 1, 2,...,1'3)' Then using theorem

14 13 P2 ~~.:L: A test having the acceptance region E b. WI. < ~ j=l J,J I>~ and L ~ W of the step down procedure for 2 k :s k=l ' independence at two stages is unbiased and has the restricted monotonic property. ~!:..~l~lt: The test procedures given by (15), (16) and (17) are unbiased and each has the restricted monotonic property. The same remarks (involving "the conditionally sectionwise ellipsoidal nature of the regions") apply here as at the end of section Ackno'VTledgement: I am extremely thankful to Professor S. N. Roy for suggesting the step down procedures and for helpful discussions with him in the course of preparation of this paper.

15 14 REFERENCES [1] Das Gupta, S., Anderson, T. W. and Mudholkar, G. S. (1964). Monotonicity of the power functions of some tests of the multivariate linear hypothesis. Ann. Math. Statist., 35, [2] Anderson, T. W. and Das Gupta, S., (1964). Monotonicity of the power functions of some tests of independence between two sets of variates. Ann. Math. Statist., 35, a [3] Khatri, C. G. (1960). On certain problems in multivariate analysis (multicollinearity of means and power series distributions). Ph.D. thesis, M. S. University of Baroda (India). [4] Khatri, C. G. (1961). Simultaneous confidence bounds on the departures from a particular kind of multicollinearity. of Statist. Math., 13, Annals of the Institute [5] Khatri, C. G. Non null distribution of the likelihood ratio for a particular kind of multicollinearity. (Sent for publication.) [6] Roy, J. (1958). Step-down procedure in Multivariate Analysis. Ann. Math. Statist., 29, [7] Roy, s. N. (1958). Some Aspects of Multivariate Analysis, Wiley, New York. [8] Roy, s. N. and Bargmann, R. E. (1958), Tests of multiple independence and the associated confidence bounds. Ann. Math. Statist., 29, [9] Roy, s. N. and Mikhail, W. F. (1961). On the monotonic character of the power functions of two multivariate tests. Ann. Math. Statist., 32,

16 15 [10] Roy, S. N. and Mikhail) W. F. (1964). Correction to and comment on section 4 of the paper "On the monotonic character of the power functions of two multivariate tests" by Roy, S. N. and Mikhail, W. F. (Ann. Math. Stat., 32, ), Ann. Math. Stat., 35, [11] Srivastava, J. N. (1962). On the monotonicity property of the three main tests for multivariate analysis of variance. Institute of Statistics, Mimeo Series No. 342, University of North Carolina.