Computation of Higher Order Moments from Two Multinomial Overdispersion Likelihood Models

Transcription

1 Computaton of Hgher Order Moments from Two Multnomal Overdsperson Lkelhood Models BY J. T. NEWCOMER, N. K. NEERCHAL Department of Mathematcs and Statstcs, Unversty of Maryland, Baltmore County, Baltmore, Maryland 21250, U.S.A. AND J. G. MOREL The Proctor & Gamble Company, Bometrcs and Statstcal Scences Department, Wnton Hll Busness Center, 6280 Center Hll Avenue, Cncnnat, Oho 45224, U.S.A. SUMMARY Computatonal detals and dervatons of hgher order moments from two multnomal overdsperson models are presented n ths report. In addton, we provde the analytcal forms for all frst through fourth order moments and ther matrx forms. Some key words: Drchlet-Multnomal dstrbuton; Fnte-Mxture dstrbuton; Moment calculatons; Multnomal overdsperson. 1. INTRODUCTION A key assumpton of multnomal data s that each count s comprsed of ndependent observatons. In many practcal stuatons however, the ndvdual observatons makng up the counts are correlated or clustered. Ths causes the data to exhbt more varablty than can be explaned by the multnomal model. Ths phenomenon s known as extra-varaton or overdsperson. There are several ways of analyzng overdspersed multnomal data. Lkelhood approaches nclude the Drchlet-Multnomal dstrbuton by Mosmann (1962 and the Fnte-Mxture dstrbuton by Morel & Nagaraj (1993 and Neerchal & Morel (1998. Ths report provdes computatonal detals and dervatons of hgher order moments from these two multnomal overdsperson models. Snce dervatons of the moments from these two dstrbutons requre the moments from a multnomal dstrbuton, we also provde the frst through fourth order moments from the multnomal dstrbuton. The computaton of these moments requre nontrval algebra and result n complcated expressons. Complete dervatons can be found n Newcomer ( THE MULTINOMIAL DISTRIBUTION Suppose T = (T 1, T 2,...,T s a k-dmensonal multnomal random varable wth parameters p = (p 1, p 2,...,p and cluster sze m. Let x (a = x(x 1 (x a + 1. Mosmann (1962 shows that the jont factoral moments of the multnomal dstrbuton are gven by [ ] µ (a1,a 2,...,a = E T (a 1 1 T (a 2 2 T (a ( a p a 1 1 pa 2 2 pa. (1 1

2 2 J. T. NEWCOMER, N. K. NEERCHAL AND J. G. MOREL Usng ths expresson, t can be shown that the frst through fourth order moments are gven by: ( E (T p, = 1, 2,...,k 1. ( E (T T j (2 p p j, j. ( E ( T 2 (2 p 2 + mp, = 1, 2,...,k 1. (v E (T T j T l (3 p p j p l, j l. (v E ( T 2 T j (3 p 2 p j + m (2 p p j, j. (v E ( T 3 (3 p 3 + 3m (2 p 2 + mp, = 1, 2,...,k 1. (v E (T T j T l T r (4 p p j p l p r, j l r. (v E ( T 2 T j T l (4 p 2 p j p l + m (3 p p j p l, j l. (x E ( T 2 Tj 2 (4 p 2 p 2 j + m (3 ( p 2 p j + p p 2 j + m (2 p p j, j. (x E ( T 3 T j (4 p 3 p j + 3m (3 p 2 p j + m (2 p p j, j. (x E ( T 4 (4 p 4 + 6m (3 p 3 + 7m (2 p 2 + mp, = 1, 2,...,k THE FINITE-MIXTURE DISTRIBUTION The Fnte-Mxture Dstrbuton by Morel & Nagaraj (1993 provdes a way to model categorcal data exhbtng overdsperson when the overdsperson s beleved to arse due to clumped multnomal samplng. Ths model can be generated as follows. Let Y, Y1 0, Y 2 0,...,Y m 0 be ndependent and dentcally dstrbuted k-dmensonal multnomal random varables wth parameters π = (π 1, π 2,...,π and cluster sze 1. Also let U 1, U 2,...,U m be ndependent and dentcally dstrbuted unform(0,1 random varables and let ρ be a real number, 0 < ρ < 1. We defne the random varable T as T = Y m I (U ρ + m Y 0 I (U > ρ, (2 where I( s the ndcator functon. Sectons 3 1 and 3 2 provde the frst through fourth order moments, ther matrx forms, and the dervatons for ths dstrbuton Fnte-Mxture Moments Suppose T = (T 1, T 2,...,T s dstrbuted as a Fnte-Mxture random varable wth parameters π = (π 1, π 2,...,π and ρ and cluster sze m. Let x (a = x(x 1 (x a + 1. The frst through fourth order moments are gven by: ( E (T π, = 1, 2,...,k 1. ( E (T T j (2 (1 ρ(1 + ρπ π j, j.

3 Multnomal Overdsperson Moments 3 ( E ( T 2 (2 (1 ρ(1 + ρπ 2 + m [ 1 + (m 1ρ 2] π, = 1, 2,...,k 1. (v E (T T j T l (3 (1 ρ 2 (1 + 2ρπ π j π l, j l. (v E ( T 2 T j (3 (1 ρ 2 (1 + 2ρπ 2 π j + m (2 (1 ρ [ 1 + ρ + (m 2ρ 2] π π j, j. (v E ( T 3 (3 (1 ρ 2 (1 + 2ρπ 3 + 3m (2 (1 ρ [ 1 + ρ + (m 2ρ 2] π 2 + m [ 1 + 3(m 1ρ 2 + (m 1(m 2ρ 3] π, = 1, 2,...,k 1. (v E (T T j T l T r (4 (1 ρ 3 (1 + 3ρπ π j π l π r, j l r. (v E ( T 2 T j T l (4 (1 ρ 3 (1 + 3ρπ 2 π j π l + m (3 (1 ρ 2 [ 1 + 2ρ + (m 3ρ 2] π π j π l, j l. (x E ( T 2 Tj 2 (4 (1 ρ 3 (1 + 3ρπ 2 πj 2 + m (3 (1 ρ 2 [ 1 + 2ρ + (m 3ρ 2] ( π πj 2 + π 2 π j + m (2 (1 ρ [ 1 + ρ + 2(m 2ρ 2] π π j, j. (x E ( T 3 T j (4 (1 ρ 3 (1 + 3ρπ 3 π j + 3m (3 (1 ρ 2 [ 1 + 2ρ + (m 3ρ 2] π 2 π j + m (2 (1 ρ [ 1 + ρ + 3(m 1ρ 2 + (m 1(m 2ρ 3] π π j, j. (x E ( T 4 (4 (1 ρ 3 (1 + 3ρπ 4 + 6m (3 (1 ρ 2 [ 1 + 2ρ + (m 3ρ 2] π 3 and + m (2 (1 ρ [ 1 + 7ρ + 18(m 2ρ 2 + 4(m 2(m 3ρ 3] π 2 + m [ 1 + 7(m 1ρ 2 + 6(m 1(m 2ρ 3 + (m 1(m 2(m 3ρ 4] π, = 1, 2,...,k 1. The expressons above allow us to wrte these moments n matrx form. Frst, defne T 2 = (T 2 1, T 2 2,...,T 2 T = (T 1 T 2, T 1 T 3, T 2 T 3,...,T 1 T,...,T k 2 T. Here T 2 s a vector of the squared counts and T s a vector contanng the unque off dagonal terms of TT. Smlarly defne, π 2 = (π 2 1, π 2 2,...,π 2, π = (π 1 π 2, π 1 π 3, π 2 π 3,...,π 1 π,...,π k 2 π, and π 3 = (π 3 1, π 3 2,...,π 3, π 4 = (π 4 1, π 4 2,...,π 4.

4 4 J. T. NEWCOMER, N. K. NEERCHAL AND J. G. MOREL Also letπ,π (2, andπ (3 be defned recursvely as follows. LetΠ 2 = (π 1 π 2, π 1 π 2, Π (2 2 = (π1 2π 2, π 1 π2 2, and Π (3 2 = π1 2π 2 + π 1 π2 2. The update equatons are gven by, ( Π Π r 1 π r dag(π r 1 r = π r π r 1, and Π (2 r = ( Π (3 (3 Π r = r 1 0 (r 1(r 2 2 ( (2 Π r 1 π r dag(πr 1 2 πrπ 2 r 1, 0 (r 1(r 2 2 π r Π r 1 r 1 π r π r 1 π r 1 + π2 rdag(π r 1 Π where 0 s s a s-dmensonal column vector of zeros, π r 1 denotes the frst r 1 elements of the vector π, and dag(x denotes a dagonal matrx wth the elements of x on the dagonals. The matrx forms of the frst through fourth order moments can be represented by: ( V ar (T [ 1 + (m 1ρ 2] dag(π ππ. ( Cov ( T,T 2 = 2m (2 (1 ρ [ 1 + ρ + (m 2ρ 2] dag(π 2 ππ 2 + m [ 1 + 3(m 1ρ 2 + (m 1(m 2ρ 3] dag(π ππ. ( V ar ( T 2 (4 (1 ρ 3 (1 + 3ρπ 2 π 2 + m (2 (1 ρ [ 1 + ρ + 2(m 2ρ 2] ππ + m (3 (1 ρ 2 [ 1 + 2ρ + (m 3ρ 2] ( ππ 2 + π 2 π + 4m (3 (1 ρ 2 [ 1 + 2ρ + (m 3ρ 2] dag(π 3 + 2m (2 (1 ρ [ 3ρ + 8(m 2ρ 2 + 2(m 2(m 3ρ 3] dag(π 2 + m [ 1 + 7(m 1ρ 2 + 6(m 1(m 2ρ 3 + (m 1(m 2(m 3ρ 4] dag(π (m (2 (1 ρ 2 2 π 2 π 2 + m 2 (1 + (m 1ρ 2 2 ππ +m 2 (m 1(1 ρ 2 (1 + (m 1ρ 2 ( ππ 2 + π 2 π., (v (v (v Cov ( T,T (2 (1 ρ [ 1 + ρ + (m 2ρ 2] Π 2ππ. Cov ( T 2, T = 2m (3 (1 ρ 2 [ 1 + 2ρ + (m 3ρ 2] Π (2 + m (2 (1 ρ [ 1 + ρ + 3(m 1ρ 2 + (m 1(m 2ρ 3] Π 2m (2 (1 ρ 2 [ (2m 3 + 2(2m 3ρ + (2m 2 8m + 9ρ 2] π 2 π 2m (2 (1 ρ [ 1 + ρ + (3m + 5ρ 2 + (m 2 3m + 3ρ 3] ππ. V ar ( T (3 (1 ρ 2 [ 1 + 2ρ + (m 3ρ 2] Π (3 + m (2 (1 ρ [ 1 + ρ + 2(m 2ρ 2] dag(π 2m (2 (1 ρ 2 [ (2m 3 + 2(2m 3ρ + (2m 2 8m + 9ρ 2] π π.

5 Multnomal Overdsperson Moments Dervaton For the Fnte-Mxture dstrbuton, Morel & Nagaraj (1993 show that the dstrbuton of T s a fnte mxture of multnomals. Ths allows us to wrte the probablty mass functon of T as P(T = t = k π P(X = t, where X ( = 1,...,k 1 s a multnomal random varable wth parameters p = (1 ρπ + ρe and cluster sze m, X k s a multnomal random varable wth parameters p k = (1 ρπ and cluster sze m, and e s the th column of the (k 1 (k 1 dentty matrx. Usng ths representaton, we can wrte the moment generatng functon of T as φ T (z = k π φ X (z, (3 where φ X (z s the moment generatng functon of X ( = 1,...,k whch are defned as above. Usng equaton (3 we can defne the mxed moments of T as E ( T a 1 1 T a = k ( π E X a 1 1 Xa (. (4 Therefore, the moments of T from a Fnte-Mxture dstrbuton can be computed usng equaton (4 and the moments from a multnomal dstrbuton, where X ( = 1,...,k are defned above. We provde the examples below to demonstrate the algebrac technques requred to compute these moments analytcally. As noted above, complete dervatons can be found n Newcomer (2008. ( E (T T j = = s j k π s E (X s X sj π s m (2 (1 ρπ (1 ρπ j + π m (2 [(1 ρπ + ρ](1 ρπ j + π j m (2 (1 ρπ [(1 ρπ j + ρ] ( + 1 π s m (2 (1 ρπ (1 ρπ j = π s m (2 (1 ρ 2 π π j + 2m (2 ρ(1 ρπ π j + m (2 (1 ρ 2 π π j π s m (2 (1 ρ 2 π π j (2 (1 ρ(1 + ρπ π j, j.

6 6 J. T. NEWCOMER, N. K. NEERCHAL AND J. G. MOREL (x E ( T 2 T 2 j = s j = k π s E ( XsX 2 sj 2 π s m (4 (1 ρ 4 π 2 πj 2 + m (3 (1 ρ 3 [ π 2 π j + π πj 2 ] + m (2 (1 ρ 2 π π j + π m (4 [(1 ρπ + ρ] 2 [(1 ρπ j ] 2 + m (3 [(1 ρπ + ρ] 2 (1 ρπ j + π m (3 [(1 ρπ + ρ] [(1 ρπ j ] 2 + m (2 [(1 ρπ + ρ](1 ρπ j + π j m (4 [(1 ρπ ] 2 [(1 ρπ j + ρ] 2 + m (3 [(1 ρπ ] 2 [(1 ρπ j + ρ] + π j m (3 (1 ρπ [(1 ρπ j + ρ] 2 + m (2 (1 ρπ [(1 ρπ j + ρ] ( + 1 π s m (4 (1 ρ 4 π 2 πj 2 + m (3 (1 ρ 3 [ π 2 π j + π πj] 2 + +m (2 (1 ρ 2 π π j = 4m (4 ρ(1 ρ 3 π 2 π 2 j + m (4 ρ 2 (1 ρ 2 π 2 π j + m (4 ρ 2 (1 ρ 2 π π 2 j+ + 3m (3 ρ(1 ρ 2 π 2 π j + 3m (3 ρ(1 ρ 2 π π 2 j + 2m (3 ρ 2 (1 ρπ π j + + 2m (2 ρ(1 ρπ π j + m (4 (1 ρ 4 π 2 π 2 j + m (3 (1 ρ 3 [ π 2 π j + π π 2 j] + + m (2 (1 ρ 2 π π j (4 (1 ρ 3 (1 + 3ρπ 2 π 2 j + m (3 (1 ρ 2 [ 1 + 2ρ + (m 3ρ 2] ( π π 2 j + π 2 π j, j. 4. THE DIRICHLET-MULTINOMIAL DISTRIBUTION The Drchlet-Multnomal dstrbuton, derved by Mosmann (1962, provdes another model for multnomal data exhbtng overdsperson. Here however, the cause of the overdsperson arses when the cell probabltes, P = (P 1, P 2,...,P, vary randomly accordng to a contnuous jont dstrbuton f(p. Condtonally, gven P, we assume T P s dstrbuted as a k- dmensonal multnomal random varable. The uncondtonal probablty mass functon of T s gven by P(T = t = m! t 1! t k! k j=1 p t j j f(pdp. (5 If we assume that f(p follows a Drchlet dstrbuton wth parameters (Cπ 1, Cπ 2..., Cπ, where C C(ρ = (1 ρ2 or ρ = C ρ 2 C+1, then t can be shown that (5 becomes k m! Γ(C j=1 P(T = t = Γ(t + Cπ t 1! t k! Γ(m + C k j=1 Γ(Cπ. (6 Sectons 4 1 and 4 2 provde the frst through fourth order moments, ther matrx forms, and the dervatons for ths dstrbuton.

7 Multnomal Overdsperson Moments Drchlet-Multnomal Moments Suppose T = (T 1, T 2,...,T s a k-dmensonal Drchlet-Multnomal random varable wth parameters ρ and π = (π 1, π 2,...,π and cluster sze m. Also, we let x (a = x(x 1 (x a + 1 and defne C C(ρ = (1 ρ2. The frst through fourth order moments are ρ 2 then gven by: ( E (T π, = 1, 2,...,k 1. ( E (T T j (2 C (C + 1 π π j, j. ( E ( T 2 (2 (Cπ + 1 (C + 1 π + mπ, = 1, 2,...,k 1. (v E (T T j T l (3 C 2 (C + 1 (C + 2 π π j π l, j l. (v E ( T 2 T j (3 C (Cπ + 1 (C + 1 (C + 2 π π j + m (2 C C + 1 π π j, j. (v E ( T 3 (3 (Cπ + 1(Cπ + 2 π + 3m (2(Cπ + 1 (C + 1(C + 2 C + 1 π + mπ = 1, 2,...,k 1. (v E (T T j T l T r (4 C 3 (C + 1(C + 2 (C + 3 π π j π l π r, j l r. (v E ( T 2 T j T l (4 C 2 (Cπ + 1 (C + 1 (C + 2(C + 3 π π j π l + + m (3 C 2 (C + 1 (C + 2 π π j π l, j l. (x E ( T 2 Tj 2 (4 C (Cπ + 1 (Cπ j + 1 (C + 1(C + 2(C + 3 π π j + + m (3C (Cπ + Cπ j + 2 (C + 1(C + 2 π π j + m (2 C C + 1 π π j, j. (x E ( T 3 T j (4 C (Cπ + 1(Cπ + 2 (C + 1 (C + 2(C + 3 π π j + 3m (3 C (Cπ + 1 (C + 1 (C + 2 π π j + + m (2 C C + 1 π π j, j. (x E ( T 4 (4 (Cπ + 1(Cπ + 2(Cπ + 3 π + (C + 1 (C + 2 (C m (3(Cπ + 1(Cπ + 2 π + 7m (2(Cπ + 1 (C + 1(C + 2 C + 1 π + mπ, = 1,...,k 1.

8 8 J. T. NEWCOMER, N. K. NEERCHAL AND J. G. MOREL The matrx forms of these moments are represented by: ( [ dag(π V ar (T 1 + (m 1 C+1] C ππ. ( Cov ( T,T 2 = 2m(2 C C m(c + 1 C + 2 [ 1 + ] (m 1 dag(π 2 ππ 2 (C + 1 [ (3m 2 (m 1(m (C + 1 (C ] dag(π ππ. ( V ar ( T 2 = + + m (4 C 3 (C + 1(C + 2(C + 3 π2 π 2 m (3 C 2 (m + C (C + 1(C + 2(C + 3 ππ 2 + π 2 π 4m (3 C 3 (C + 1(C + 2(C + 3 dag(π3 + m(2 C [(m 2(m 3 + 2(m 2(C (C + 2(C + 3] ππ (C + 1(C + 2(C m(2 C [10(m 2(m (m 2(C (C + 2(C + 3] dag(π 2 (C + 1(C + 2(C + 3 m[6(m 1(m 2(m (m 1(m 2(C + 3] + dag(π (C + 1(C + 2(C + 3 7m(m 1(C + 2(C (C + 1(C + 2(C dag(π (C + 1(C + 2(C + 3 (m (2 (1 ρ 2 2 π 2 π 2 + m 2 (1 + (m 1ρ 2 2 ππ +m 2 (m 1(1 ρ 2 (1 + (m 1ρ 2 ( ππ 2 + π 2 π. (v Cov ( T,T (2 C(m + C Π (C + 1(C + 2 2ππ. (v Cov ( T 2, T = 2m (3 C 2 (m + C (C + 1(C + 2(C + 3 Π (2 + m(2 C [2(m 2(m 3 + 3(m 2(C (C + 2(C + 3] Π (C + 1(C + 2(C + 3 2m(2 C 2 (m + C[2mC + 3(m C 1] (C π 2 π (C + 2(C + 3 2m(2 C(m + C[C(C m(2c + 3] (C ππ. (C + 2(C + 3

9 (v V ar ( T = Multnomal Overdsperson Moments 9 m (3 C 2 (m + C (C + 1(C + 2(C + 3 Π (3 + m(2 C [(m 2(m 3 + 2(m 2(C (C + 2(C + 3] dag(π (C + 1(C + 2(C + 3 2m(2 C 2 (m + C[2mC + 3(m C 1] (C π π. (C + 2(C + 3 where C = (1 ρ2 ρ 2 and Π, Π (2, and Π (3 are defned as above Dervaton In the case of the Drchlet-Multnomal dstrbuton, T P s dstrbuted as a k-dmensonal Multnomal random varable wth parameters P = (P 1,...,P and P vares randomly accordng to a Drchlet dstrbuton wth parameters (Cπ 1,...,Cπ, where C = ( 1 ρ 2 /ρ 2. Suppose P = (P 1,...,P s dstrbuted as a Drchlet random varable wth parameters Cπ 1,...,Cπ. The moments of P are gven by E ( P a 1 1 P a B(Cπ + a =, (7 B(Cπ where, for some vector α = (α 1,...,α, j=1 Γ(α j B(α = (. Γ j=1 α j Usng a condtonng argument, we can derve the formula for the generalzed factoral moments from a Drchlet-Multnomal dstrbuton from the generalzed factoral moments of a multnomal dstrbuton, ( ( E T (a 1 1 T (a a E ( P a 1 1 P a. (8 Equatons (7 and (8 allow us to compute the moments from a Drchlet-Multnomal dstrbuton. As before, we provde the followng examples to demonstrate the algebrac technques requred to compute these moments analytcally. (v E ( T 3 = E (T (3 (3 E(P E ( T 2 2E (T m (2(Cπ + 1 (C + 1 π + mπ 2mπ (3(Cπ + 1(Cπ + 2 π + 3m (2(Cπ + 1 (C + 1(C + 2 C + 1 π + mπ, = 1, 2,...,k 1. (v E (T T j T l T r (4 E(P P j P l P r (4 C 3 j l r. (C + 1(C + 2(C + 3 π π j π l π r,

10 10 J. T. NEWCOMER, N. K. NEERCHAL AND J. G. MOREL (v E ( T 2 ( T j T l = E T (2 T j T l + E (T T j T l = (4 E(P 2 P j P l + m (3 C 2 (C + 1 (C + 2 π π j π l (4 C 2 (Cπ + 1 (C + 1 (C + 2(C + 3 π π j π l + + m (3 C 2 (C + 1(C + 2 π π j π l,, j l. 5. COMPARISON OF MOMENTS FROM TWO MULTINOMIAL OVERDISPERSION MODELS Ths secton compares the thrd and fourth order moments from the fnte-mxture and drchlet-multnomal dstrbutons. Suppose T = (T 1, T 2, T 3 s a 4-category multnomal overdsperson model wth category probabltes π = (π 1 = 0.1, π 2 = 0.2, π 3 = 0.3. We assume that T follows ether a fnte-mxture or drchlet-multnomal dstrbuton. Fgure 1 depcts varous moments, varances, and covarances for T as a functon of the dsperson parameter ρ for these two dstrbutons. It s clear that the hgher order moments are dfferent for these two models. These dfferences can be a vtal tool n determnng whch dstrbuton fts a partcular dataset better, especally snce the frst two moments of these dstrbutons are the same. It s therefore possble that goodness-of-ft statstcs could be developed based on these hgher order moments. In addton, snce the frst two moments are the same, estmaton methods based on the frst two moments, such as generalzed estmatng equatons (GEE, provde the same estmate regardless of whch dstrbuton s thought to be the workng model. Newcomer et al. (2008 propose a novel extenson of GEE whch ncorporates the thrd and fourth order moments. Ths method therefore allows one to choose between the two models as a workng model and parameter estmates wll be dfferent based on the choce made. By ncorporatng ths addtonal nformaton nto the estmaton procedure we are able to obtan more effcent estmates than those from the usual GEE procedure. REFERENCES MOREL, J. & NAGARAJ, N. (1993. A fnte mxture dstrbuton for modelng multnomal extra varaton. Bometrka 80, MOSIMANN, J. E. (1962. On the compound multnomal dstrbuton, the multvarate β-dstrbuton, and correlaton among proportons. Bometrka 49, NEERCHAL, N. & MOREL, J. (1998. Large cluster results for two parametrc multnomal extra varaton models. Journal of the Amercan Statstcal Assocaton 93, NEWCOMER, J. (2008. Estmaton procedures for multnomal models wth overdsperson. Ph.D. Thess, Unversty of Maryland, Baltmore County. NEWCOMER, J., NEERCHAL, N. & MOREL, J. (2008. An effcent alternatve to GEE for multnomal overdspersed data. In preparaton for Bometrka.

11 Multnomal Overdsperson Moments 11 E(T Correlaton (ρ (a E(T 1 2 T Correlaton (ρ (b E(T 1 2 T 2T Correlaton (ρ (d Var(T 1 4 Correlaton (ρ (e Cov(T 1 2,T 2T Cov(T 1 2,T2 2 Correlaton (ρ (c Correlaton (ρ (e Fg. 1. Plots of varous thrd and fourth order moments as a functon of ρ for the fnte-mxture (red and drchletmultnomal (blue dstrbutons.