I4ATIEMATIC*L SCIENCES LASSIFIED END. ~~ rf ~ O6 ~~~

Size: px

Start display at page:

Download "I4ATIEMATIC*L SCIENCES LASSIFIED END. ~~ rf ~ O6 ~~~"

Roger Higgins
5 years ago
Views:

1 I4ATIEMATIC*L SCIENCES F/S t2/t II END I rf 0 LASSIFIED O I..4

2 1.0 L l L 12.2 L I. : IIIO IIIII ilhii 6 MI( R t )FY RI Sot u 1 ION I I SI Cl IAN I NA k NAt I 1II I NI A 1 ANI A At A

4 LEEL / HI-.SQUARE TESTS FOR THE MULTINOMIALThISTRIBUTION / KHURSH EEDLALAM DEPARTMENT OF MATHEMATICAL SCIENCES, - j TECHNICAL fie T 6 D D C Research Supported by The Office of Nava l Research - Task NR C ac/ #14_75_c B Reproduction in whole or pare is permitted for any purposes of the U. S. Government. Distributj -,n of this document is unlimited. DISTRIBUT ION STATEMENT A Approved for public rel.as. Distribution Unlimited -

5 , -!._W r._ - -_-_ç_.- -,. _. _.! CHI-SQUARE TESTS FOR THE MULTINOMIAL DISTRIBUTION Khursheed Alam Clemson University 4 A simple proof of the asymptotic property of Chi-square tests, commonly used in the analysis of categorical data, is given for use as a note for instruction to first year graduate students. Key Words : Multinomial Distribution ; Chi-square Tests ; Contigency Tables. 1. Introduction The Chi-square tests associated with the niultinomial distribution are commonly used in the analysis of categorical data with reference to problems of specification, homogeneity of parallel samples, independence of attributes, etc. The asymptotic property of the tests, that is, the Chi-square distribution of the test statistics in large samp les is generall y known. However, it has been our observation that many applied statisticians tacitly accept the asymptotic result without satisfying themselve s with its proof. It is also true that nearly all the text books in use on elementary and higher statistics either omit the proo f or barely sketch jt. In this paper we outline a fairly simple proo f of the fundamental resu lt, for use as a note for the instruction to first-year graduate students and as a needed theory for the frequent application of Chi-square O -

6 2. tests by applied statisticians. The proof is essentially based on the contents of Chapters 5 and 6 of Rao (1966). Consider a multinomial distribution M (rz,p) with K cells, where p = denotes the vector of cell probabilities, I = fl 1 denotes the vector of cell frequencies resulting from n independent trials, i 1 p 1 = 1 and i i l n n. A general problem in judging goodness of fit is to test whether the cell probabilities are specified functions of a fewer number of parameters whose values may be unknown. Let the cell probabilities be given functions l K (t of an unknown vector e (t3 l * s 3 )s where r < K. To test the speci- r fication, it is a standard method to use the statistic (1.1) T = i l ( (s) ) 2 / np.(o) where t is a consistent estimator of, usually the maximum likelihood estimator. Next, assuming that the specification is true, consider the hypothesis that e is given by = g ( ) where g1,...,g are given functions, = (,..., t ) and s r. This hypothesis arises in a test of homogeneity of parallel samples and of independence in a contingency table. The statistic (1.2) T* = 5, K ((.) p ( )) 2 / p () is used to test the hypothesis, where denotes an estimate of and denotes the value of p.(3) as a function of, unde r the given hypothesis. It is shown below that T and T* are asymptotically distrib- uted for large n according to the Chi-square distribution under certain conditions. ( (S - A

2. Asymptotic Distribution of T and T* First, consider the specification that the multinomial cell probabilities 1tre given functions p l (e),...,p x (O) of an unknown vector = l1 1 8 r ) where r < K.

7 2. Asymptotic Distribution of T and T* First, consider the specification that the multinomial cell probabilities 1tre given functions p l (e),...,p x (O) of an unknown vector = l1 1 8 r ) where r < K. Let 0 0 denote the true value of e. We make the following assumptions: where (i) The functions (0) are continuous in 0, admitting first order partial derivatives which are continuous at e. (ii) Given 6 > 0, there exists >0 such that inf N(S) > 10 8 I N(s) = i 1 log Let M (8) = L log (p 1 (8 )/p 1 (e)). Consider the function N (e) on the sphere - l = 6. Since N(S) is continuous in 8, the infimum of N(s) is attained on the sphere. Therefore, in view of (ii), N(S) > for every point on the sphere. Since - converges in probability to as n it follows that M (8) 0 for all points on the sphere with probability approaching 1 as n + The log likelihood function is proportional to i n log p. (o). In view of (i) and the result given above, we have that for sufficiently large n, the likelihood function has a local maximum inside the open sphere l - l< at a point 0, say, which is a solution of the likelihood equation n. (2.l) i i p (e) J,r. Since 5 can be made arbitrarily small, the maximum likelihood estimator is a consistent solution of the likelihood equation. 4 -i - -.

8 I 4. Let I(S) = (M (S)) (M(9)) denote the information matrix of the mul tinomial distribution, where M(0) = ( v p (8) ap.(e) 3. is a K x r matrix, and let Z = M, where M = M (9 ) and n np (e ) n np (e) 1 1 K K = ( /npl8 J ) By the central limit theor n, the asymptotic distribution of is multi variate normal N(O,I- ), where denotes the null vector, I denotes the identity matrix and D = v p1 ( ) P K The asymptotic distribution of Z is N (Q,I), where I = 1(9 ). Note that M = 0. Substituting 9 for in (2.1), the jth equation can be written as K (2.2) i=l / p.( ) n. - np.(9 ) = K -i=l 3 In view of (i) we have r (2.3) p. (9) p. 2. (S. Sj ) Ị 3p.(8 ) + where fl 0 as 0 + Since 0 is a consistent estimator of 5, as shown above, the left side of (2.2) is asymptotically equivalent (converging in probab ility) to Z. Therefore, by the substitution of (2.3) on the right side of (2.2) we have that j a r D ),

9 r - 5. where means asymptotically equivalent to, and I denotes the Ljth element of 1. Hence z 1 ( 0 8 ) or (2.4) 1 Z / (0 e ) where I denotes a generalized inverse of I, given by I I I I, and is equal to I if I is non-singular. Let A be a symmetric matrix with real elements, and let X N (u, ), where means distributed as. If the covariance matrix is non singular then it is known that the quadratic form X A X x (non-central Chi-,5 square with v degrees of freedom and non centrality parameter ) if and only if A t is idempoten t, where v Rank A and & = A i.i. If is singular, then the given condition is only sufficient and v = Rank AE (see e.g. Graybill (1976), Theorem 4.7.1). If A E is a generalized inverse of z, given by =, then A :is idempotent and Rank A : = Rank. Therefore, X z x d 2, where v = Rank Z I 5 and S Let W = (W l?...,w K ) where W i = v i ( (e) - p ( 0 ))/ / (e ), i = From (2.3) we have that 1 (2.5) W,/n M (0 8 ) M I Z by (2.4 ) = M M. -.--I -- -

10 I 6. Note that M 1 M is idempotent. From (1.1) and (2.5) we have T ( - W) ( - W) (I - N 1 M ). Now, is asymptotically distributed as N (O, I tp ), (I M IM ) (I q p ) = I MIM is idemptotent and Rank (I-Ml 4 ) = Trace (I Ml M 4 ) = K Rank I = 3, say. Therefore, T is asymptotically distributed as. If 1 is of full rank then 5 = K r 1. Next, consider the hypothesis that is given by e g i = 1,...,r, as described in the previous section. Let v( ) = denote the r x s matrix of the derivatives, and let l*( ) denote the information matrix under the given hypothesis. Then (2.6) I *( ) = with 1 (3) being expressed as a function of. Let denote the true value of t. Similarly, as in the preceding we have that T* (MI M M( 1) 7 M ) where 7 = 7( ). The matrix (M1 M M (7 17) 7 N C (I b )= (MI M 17) 7 N ) is idempotent and Rank (MI M -Mv (7 Iv) N ) = Rank I -Rank ( 17) r, say.

- / - --- -- - --... --- - - 7. theny=r-s. refore, T* is a t cally distributed If l and are of thu nk We have st in that T and.

11 - / theny=r-s. refore, T* is a t cally distributed If l and are of thu nk We have st in that T and. T* are as ptotically distributed according to the Chi square distribution under the identifiability oorxlition (i) and tha continuity assu1 t.iofl (ii). Renark: For the goodness of fit test where the cell probabilities are o n- pletely specified we have K-i. In this case T x, asyn totica.uy. For testing luxogeneity of r parallel sanpies or independence of attributes in r x K c r tigency tables, we have y = (r-l) (K-i). Peferences. (1] Graybill, R. A. (1976). Tl Tl ory and Application of the Linear M de1. Wads orth Publishing Co., Bein nt, California. [2] Rao, C. R. (1966). Linear Statistical Inference and its Applications. r Wiley Publications in Statistics. I _ --

i SE 1Ar?. :. Ass. A om I AG I W I III. ) A L Nc AssIF:ED IuE Jr&?tnu DA E DEDnDI READ 4S RUC 1O 4S rs I - I I I J, Jl 1 I I! r BEFORE COMPLETI? G FORM 6P I, us(a. GOT ACC(SSION O. N10 4!C.P tw S A AL G 4I MSI t S I & s o Chi-square tests for the ul tinomia1 distribution Technical Report.

12 i SE 1Ar?. :. Ass. A om I AG I W I III. ) A L Nc AssIF:ED IuE Jr&?tnu DA E DEDnDI READ 4S RUC 1O 4S rs I - I I I J, Jl 1 I I! r BEFORE COMPLETI? G FORM 6P I, us(a. GOT ACC(SSION O. N10 4!C.P tw S A AL G 4I MSI t S I & s o Chi-square tests for the ul tinomia1 distribution Technical Report. 296 S C por AC * A SUt (.; $ IU,d, I 3 qca 4I 4 I 4 AMa A O AQCIL S5 t C. Noco14 7 c 345 : Clemson 7niversity Deot. of Mathematical Sciences / NR Clemson,_ South_Carolina QG AM t MtM SCI. &S,c Afl(A A ioi 4J hs( $ Ct AuE A o Aoo at ss 2. ULP OU A E office of Naval Research /, Code 436 Arlington O I G 44M5 S ADO ESS(,t r,om Cont oi g ftic.) S S(C. t 1 Y CLA SS. (.1 thu.pon Unclassified 1$.. CIC ASSI I CA1 CM SCMi ule O IA I A SUI u 3Pi 1 4 1M5N1 Od thui IPOft) Approved for publi : release; istr2.bution unlimited. S A?(t (.1,i,.e,act inh.,.d -n L cs, If IUsrsui f? si Rip.,) 4 u C 1 5 S (C AC CS C u:nu. n,i.i,a d f icsii W -d.n nfp Ap Aløe* Mul tinomial distributi- ns ; Chi-s ua :e test; Contir. ency table. A Sj U*C 1Sn i. i 1IJ1?.c.S. ds II 3 $iaaa Th.s paper gives a proof E the asymptotic prcperty cf the Chisquare tests assoc.ated with the uitincmiai fistr buticn, genera y used in the ana ysis of categcrica data, such as n tingency tables. DO 1473 t I 3 CY si s so ii S 4 _ S(CLI I :,. ji - - * Cu4 i J IA3( D.:.. Smii i j

P a g e 5 1 of R e p o r t P B 4 / 0 9

P a g e 5 1 of R e p o r t P B 4 / 0 9 J A R T a l s o c o n c l u d e d t h a t a l t h o u g h t h e i n t e n t o f N e l s o n s r e h a b i l i t a t i o n p l a n i s t o e n h a n c e c o n n e