Fabio Rapallo. p x = P[x] = ϕ(t (x)) x X, (1)

Transcription

1 Frst Internatonal School on Algebrac Statstcs STID Menton (France) February, 17-18, 2003 Torc Statstcal Models. The Dacons Sturmfels algorthm for log-lnear models. Tutoral Fabo Rapallo 1 Theoretcal recalls In ths tutoral we show the practcal applcablty of the algorthm descrbed n the paper by Dacons & Sturmfels (1998) who apply the algebrac theory of torc deals to defne a Markov Chan Monte Carlo method for samplng from condtonal dstrbutons, through the noton of Markov bass. For some basc concepts from Computatonal Commutatve Algebra, such as polynomal deal, term-orderng and Gröbner bass, we refer to Kreuzer & Robbano (2000). Fundamental references for the applcatons of Commutatve Algebra to Probablty and Statstcs for fnte sample spaces are Pstone et al. (2001a), Chapter 6, and Pstone et al. (2001b). For the theory of log-lnear models, we can refer to Fenberg (1980) and Agrest (2002). Let X be a fnte set and let p x = P[x] = ϕ(t (x)) x X, (1) be a probablty model wth suffcent statstc T : X N s and such that the dstrbuton of a sample of ndependent and dentcally P-dstrbuted random varables X = (X 1,..., X N ) s of the form P N = ψ(t N ) where T N (X) = N T (X k ). (2) Ths means that the suffcent statstc of X s the sum of the suffcent statstcs of the one-dmensonal random varables X k, k = 1,..., N. We denote Y t = {(x 1,..., x N ) : T N (x 1,..., x N ) = t}, (3).e., the set of all samples wth fxed value t of the suffcent statstc T N. It s known that the dstrbuton of X gven {T N = t} s unform on Y t, n fact P N (x 1,..., x N T N = t] = ψ(t) = P N (x 1,..., x N T N = t]. (4) 1 k=1

2 We ntroduce the space { F t = f : X N : } f(x)t (x) = t x X, (5).e., the set of all frequency tables obtaned from samples wth value t of the suffcent statstc T N. Denotng by F the canoncal mappng from Y t to F t, the mage probablty of P N [ T N = t] nduced by F s H t (f) = P N [F 1 (f) T = t] = #{(x 1,..., x N ) F (x 1,..., x N ) = f} #Y t, (6) whch s by defnton the hypergeometrc dstrbuton on F t. Smple computatons show that H t (f) = N! (f(x)!) 1. (7) #Y t x X A log-lnear model defnes restrctons on the parameter space, through constrants on the p x s. Such restrctons are the mathematcal counterpart of statstcal notons, such as ndependence, condtonal ndependence, symmetry and others. It s common practce to test wether the observed data match wth the statstcal model or not. Ths can be done wth a goodness of ft test, computng the maxmum lkelhood estmate of the cell counts and then usng the Pearson s statstc C = x X (f obs (x) ˆf(x)) 2 ˆf(x) (8) where ˆf(x) s the maxmum lkelhood estmate of the count n x. Large values of the test statstc ndcate a departure from the null hypothess, so we have a one-tal test. The usual approach s the asymptotc one, whch nvolves ch-squared dstrbutons, but n many cases, especally when the table s sparse, the ch-squared approxmaton may not be adequate (for further detals on ths topc see, for example, Appendx IV of Fenberg (1980)). In the exact framework, the p value s defned as the probablty of the tables n F t havng a value of the test statstc C greater than or equal to that of the observed table. A frst example of exact computaton s the Fsher s exact test for 2 2 tables, whch computes the exact p value usng the move + n order to obtan all the tables + wth fxed margnal totals. Note that n general t s dffcult to fnd the number of tables n F t and to have a complete lst of all tables n F t. We can obtan approxmatons of test statstcs va Monte Carlo methods, drawng an d hypergeometrc sample of contngency tables n F t. The problem s then reduced to sample from the hypergeometrc dstrbuton on F t, or equvalently from the unform dstrbuton on Y t. Lterature suggests to avod the enumeraton problem va the use of Markov Chans Monte Carlo (MCMC) methods. In partcular we are nterested n Metropols Hastngs algorthm whch rests on a set of moves for constructng the relevant Markov chan. A revew on the Metropols Hastngs algorthm can be found n Chb & Greenberg (1995). 2

3 Defnton 1.1 A Markov bass of F t s a set of functons m 1,..., m L : X Z, called moves, such that for any 1 L m (x)t (x) = 0, (9) x X where T s the suffcent statstc, and for any f, f F t there exst a sequence of moves (m 1,..., m A ) and a sequence (ɛ j ) A j=1 wth ɛ j = ±1 such that f = f + A ɛ j m j (10) j=1 and for all 1 a A. a f + ɛ j m j 0 (11) j=1 The condton (9) mples that a move s a table wth nteger entres (even negatve) and such that the value of the suffcent statstc s constant for every table obtaned wth moves n {m 1,..., m L }. Note once agan the mportance of the lnearty condton for the suffcent statstc T. In partcular, f the suffcent statstc are the margns, then every move s a table wth null margns. For example, f we consder the margnal totals as suffcent statstc for the 3 3 tables, a move s m = (12) From ths defnton t s clear that Markov bass s the man tool to defne a randomwalk-lke Markov chan on F t. It s well known that a connected, reversble and aperodc Markov chan converges to the statonary dstrbuton. In practce, to obtan a sample from the dstrbuton of nterest σ(f) on F t, the Markov chan s performed as follows: (a) at the tme 0 the chan s n f; (b) choose a move m unformly n the Markov bass and ɛ = ±1 wth probablty 1/2 each ndependently from m; (c) f f + ɛm 0 then move the chan from f to f + ɛm wth probablty mn{σ(f + ɛm)/σ(f), 1}; n all other cases, stay at f. Let us recall here the basc convergence theorem for the Metropols Hastngs algorthm workng n our framework. Theorem 1.2 Let σ(f) be a postve functon on F t. Gven a Markov bass {m 1,... m L }, the Markov chan generated followng the descrbed algorthm s connected, reversble and aperodc on F t wth statonary dstrbuton proportonal to σ(f). 3

4 In order to navgate the set F t and to defne a MCMC algorthm for the computaton of the p value of the goodness of ft tests, we use the theory of torc deals, see Bgatt & Robbano (2001) for detals. We assocate an ndetermnate ξ to every sample pont and an ndetermnate y j to every component of the suffcent statstc and we defne the polynomal rngs over a feld K wth such ndetermnates. Consder a model wth suffcent statstc T and a Markov chan wth moves n a set M. The correspondence between moves and polynomals s gven by the followng rule. Decompose a move m nto ts postve and negatve parts m + m and defne the bnomal b m = ξ m+ ξ m. For example, the prevous move (12) corresponds to the bnomal n the polynomal rng K[ξ 11,..., ξ 33 ]. Defne the two deals b = ξ 12 ξ 21 ξ 23 ξ 32 ξ 11 ξ 2 22ξ 33 (13) I T = Ideal(ξ a ξ b T (a) = T (b)) (14) I M = Ideal(ξ m+ ξ m = 1,..., L) (15) The deal I T s a torc deal. The man result for fndng Markov bases (e.g. for showng the connectedness of the Markov chan) usng Commutatve Algebra, presented n Dacons & Sturmfels (1998), s the followng. Theorem 1.3 The Markov chan wth moves n M s connected f and only f I M = I T. Thus, n order to fnd the relevant Markov bases, we compute I T, the torc deal assocated to the suffcent statstc T, and we mpose I M = I T. Elmnaton-based algorthm A smple algorthm to compute the Gröbner bass of the torc deal I = I(τ 1,..., τ r ) s based on the elmnaton algorthm as follows. We frst consder the homomorphsm π : K[ξ 1,..., ξ r ] K[y 1,..., y s ] (16) defned by ξ τ for all X ; then, we consder the deal J n the polynomal rng K[ξ 1,..., ξ r, y 1,..., y s ] generated by the set of polynomals Usng the elmnaton theory, we have {ξ 1 τ 1,..., ξ r τ r }. (17) I = J K[ξ 1,..., ξ r ]. (18) A (reduced) Gröbner bass of I s obtaned computng a (reduced) Gröbner bass of J wth respect to a term-orderng of elmnaton for y 1,..., y s and takng the only polynomals not nvolvng the y s. 4

5 Saturaton-based algorthm Another method can be found n Bgatt et al. (1999). Such method s based on the theory of the saturaton and t s more effcent from the computatonal pont of vew. As the suffcent statstc s a lnear map T : N r N s, we can wrte ts matrx representaton A T. The saturaton algorthm follows these steps. Compute a bass v 1,..., v q of the kernel of T as vector space homomorphsm. As the elements of A T are nteger, such bass can be chosen wth nteger elements. Defne v = v + v, for = 1,..., q and defne the bnomals b = ξ v+ ξ v n the polynomal rng K[ξ]. Defne the deal I = Ideal(b 1,..., b q ). Saturate the deal I wth respect to the polynomal ψ = X ξ, the product of all the ξ ndetermnates,.e. compute the deal The deal I s the torc deal assocated to T. I = Elm(v, I + Ideal(ψv 1)) (19) The functon Torc of CoCoA (Capan et al. (2000)) uses ths algorthm and computes a bass of the torc deal startng from the matrx A T. Note that the theory of Markov bases does not need the noton of Gröbner bass. It s suffcent to consder the deals and set of generators. Gröbner bases wll be used below as computatonal tool. Moreover, the Markov bass does not depend from the observed value t of the suffcent statstc T, but only from ts functonal form. Remark 1.4 In the framework of log-lnear models we have models of the form s log m = λ (j) (20) where m s the mean of the -th cell and λ s are real parameters. For example, the ndependence model for two random varables X and Y wth I and J levels respectvely s j=1 log m j = λ + λ (X) + λ (Y ) j = 1,..., I, j = 1,..., J. (21) In terms of cell probabltes, s s p = N 1 m = N 1 exp(λ j ) = ζ j. (22) j=1 In such way, a statstcal model s defned through mplct relatons among the p s gven n mplct form va a set of power products. 2 Exercses The MCMC step of the algorthm s a classcal one, and you can use the Matlab programs smul and chsq. Note that n such programs we use a vector representaton also for the contngency tables and not a matrx representaton. Our choce s very useful for mult-way tables. j=1 5

6 Moreover, n order to transform a lst of polynomals nto a matrx, you can use the Co- CoA functon TorcToMat, possbly wth some further ASCII manpulaton for matchng the Matlab nput requrements. The log-lnear models presented here are extensvely dscussed n Rapallo (2002a) and Rapallo (2002b), both from the statstcal and the algebrac pont of vew. Exercse 1 The ndependence model for two-way tables has the form log m j = λ + λ (X) + λ (Y ) j (23) for = 1,..., I and j = 1,..., J. The components of the suffcent statstc are the row sums and the column sums of the table. Consder a 2 3 table. a) Wrte the power products defnng the model; b) usng the elmnaton algorthm, compute the Markov bass; c) observe that the Markov bass s a well known algebrac object,.e. t s the set of + all for any 2 2 mnor of the table and 0 otherwse (possbly modulo the + sgn); d) wrte the matrx representaton of the suffcent statstc and compute the same Markov bass wth the functon Torc; e) use the Markov bass n the numercal algorthm n order to test the ndependence model for the followng table The maxmum lkelhood estmate s 3 0 f obs = 4 1 (24) ˆf = (25) Exercse 2 Consder now the ndependence model, but for a 3 3 table wth a structural zero. A structural zero s a cell wth a pror zero probablty. Suppose that the structural zero be the (1, 1) cell. The log-lnear form of the model s agan log m j = λ + λ (X) + λ (Y ) j (26) but for (, j) (1, 1). The components of the suffcent statstc are the row sums and the column sums of the table. 6

7 a) Wrte the power products defnng the model; b) usng the elmnaton algorthm, compute the Markov bass; c) wrte the matrx representaton of the suffcent statstc and compute the same Markov bass wth the functon Torc; d) repeat c) consderng all the dagonal elements as structural zeros. Exercse 3 The quas-ndependence model s a log-lnear model for square tables whch consder ndependence except for the dagonal cells whch are ftted exactly. The log-lnear form of the model s log m j = λ + λ (X) + λ (Y ) j + δ I =j (27) for = 1,..., I and j = 1,..., I. The components of the suffcent statstc are the row sums, the column sums and the dagonal counts. a) Compute the Markov bass for the quas-ndependence model for the 3 3 tables and for the 4 4 tables (use the saturaton algorthm); b) use the 4 4 Markov bass n the numercal algorthm n order to test the ndependence model for the followng table f obs = The maxmum lkelhood estmate s (28) ˆf = (29) c) use the 3 3 table to compute the cardnalty of the reference set F t for the followng table f obs = (30)

8 Exercse 4 The quas-symmetry model s a log-lnear model for square tables whch consder the symmetry of the two varables, but t does not need the margnal homogenety. The log-lnear form of the model s log m j = λ + λ (X) + λ (Y ) j + λ (XY j ) (31) for = 1,..., I and j = 1,..., I, wth the constrants λ (XY j ) = λ (XY j ) for all, j = 1,..., I. The components of the suffcent statstc are the row sums, the column sums and the dagonal-opposte cells sums. a) Compute the Markov bass for the quas-symmetry model for the 3 3 tables and for the 4 4 tables (use the saturaton algorthm); b) use the 4 4 Markov bass n the numercal algorthm n order to test the ndependence model for the followng table f obs = The maxmum lkelhood estmate s (32) ˆf = (33) c) compare the Markov bases for the quas-ndependence model and for the quassymmetry model n the 4 4 case and n the 3 3 case. Exercse 5 Consder now a multdmensonal example. three varables wth three levels. The complete ndependence model for a) Wrte the log-lnear form of the model; b) compute the Markov bass for ths model, usng the saturaton algorthm; c) compute a Gröbner bass of the torc deal, a mnmal set of generators (wth MnGens) and compare the results; d) try to compute the Markov bass usng the elmnaton algorthm. 8

9 MCMC functon functon smul=smul(tab,mle,m,n,bs,step); %tab = the observed table; %mle = maxmum lkelhood estmate; %m = the matrx representaton of the moves; %N = number of MCMC replcates; %bs = number of burn-n steps; %step = reducton of correlaton step; nc=length(tab); p=0; c=zeros(n,1); chsqref=chsq(tab,mle); mm=-m; m=[m; mm]; [nmoves,ncm]=sze(m); numt=n*step+bs; for =1:numt tabp=zeros(1,nc); r=cel(nmoves*rand(1)); tabp=tab+m(r,:); f tabp>=zeros(1,nc) mhr=1; for j=1:nc f m(r,j)~=0 mhr=mhr*prod(1:tab(j))/prod(1:tabp(j)); alpha=rand(1); f mhr>=alpha tab=tabp; f (rem(,step)==0) & (>bs) c((-bs)/step)=chsq(tab,mle); f c((-bs)/step)>=chsqref p=p+1; p=p/n; smul=p; 9

10 References Agrest, A. (2002). Categorcal Data Analyss. New York: Wley, 2nd ed. Bgatt, A., La Scala, R. & Robbano, L. (1999). Computng torc deals. J. Symb. Comput. 27, Bgatt, A. & Robbano, L. (2001). Torc deals. Mat. Contemp. 21, Capan, A., Nes, G. & Robbano, L. (2000). CoCoA, a system for dong Computatons n Commutatve Algebra. Avalable va anonymous ftp from cocoa.dma.unge.t, 4th ed. Chb, S. & Greenberg, E. (1995). Understandng the Metropols Hastngs algorthm. Amer. Statst. 49, Dacons, P. & Sturmfels, B. (1998). Algebrac algorthms for samplng from condtonal dstrbutons. Ann. Statst. 26, Fenberg, S. (1980). The Analyss of Cross-Classfed Categorcal Data. Cambrdge: MIT Press. Kreuzer, M. & Robbano, L. (2000). Computatonal Commutatve Algebra 1. New York: Sprnger. Pstone, G., Rccomagno, E. & Wynn, H. P. (2001a). Algebrac Statstcs: Computatonal Commutatve Algebra n Statstcs. Boca Raton: Chapman&Hall/CRC. Pstone, G., Rccomagno, E. & Wynn, H. P. (2001b). Computatonal commutatve algebra n dscrete statstcs. In Algebrac Methods n Statstcs and Probablty, M. A. G. Vana & D. S. P. Rchards, eds., vol. 287 of Contemporary Mathematcs. Amercan Mathematcal Socety, pp Rapallo, F. (2002a). Algebrac Markov bases and MCMC for two-way contngency tables. Scand. J. Statst. In press. Rapallo, F. (2002b). Exact algebrac nference for rater agreement models. Preprnt 467, Unverstà d Genova, Genova. Submtted. 10