Clustering through Mixture Models

Transcription

1 lusterng through Mxture Models General referenes: Lndsay B.G. 995 Mxture models: theory geometry and applatons FS- BMS Regonal onferene Seres n Probablty and Statsts. MLahlan G.J. Basford K.E. 988 Mxture Models: Inferene and Applatons to lusterng Marel Dekker ew York. Fraley. Raftery A.E. 998 How Many lusters? Whh lusterng Method? Answers Va Model-Based luster Analyss The omputer Journal Applatons to Mroarray data: Yeung K.Y. Fraley. Murua A. Raftery A.E. and Ruzzo W. L. 200Model- Based lusterng and Data Transformaton for Gene Expresson Data Bonformats Examples that are jont work wth F. Bartolu bart@stat.unpg.t Dept. of Statsts Unversty of Peruga ITALY.

2 Issues: Relablty; arbtrarness natural lumpness of the data: brngng parttons and haraterst patterns wthn the doman of statstal nferene; substtute membershp wth membershp probabltes. Multple and ompoundng soures of expermental error: robustfaton towards anomales whle keepng an adequate degree of senstvty. Muh s unknown but some aspets are well known or objet of well defned hypotheses: ntegratng exploraton and substantve modelng. An approah based on multvarate normal mxtures and maxmum lkelhood may provde some answers 2

3 The Mxture Approah: data s a sze sample from R T ~ µ Σ + Γ eah profle omes from one of alternatve omponents 0 ; ontamnaton term Unform on data range or sparse and spheral absorbs anomalous profles -; regular omponents Model means and wthn omponent ovarane to varous degrees of spefty Γ Undata range or Γ µ σ I µ Σ µ 2 Σ Z S β σ 2 σ β 2 maybe Σ R p Σ S overage radus degree of ontamnaton Lnear re-param of means Assume equal better dsrmnaton of wthn-between varaton and model 3

4 A artoon ontamnaton robustfaton Lnear onstrants on mean patterns Z onst. mean pattern flterng Z 2 e.g. a snusodal trend n tme t Z 4 0 no systemat expresson flterng Dfferent weghts but same shape sze and orentaton Σ from S. e.g. spheral dagonal and unonstraned but other lasses are possble. Z I unonstraned means exploraton Z 3 e.g. a polynomal trend n temp t 4

5 Another artoon ontamnaton robustfaton Lnear onstrants on mean patterns Z onst. mean pattern flterng Z 4 0 no systemat expresson flterng Dfferent weghts Shape sze and orentaton Σ from S. e.g. modeled to share ertan features but not others or unonstraned. Z I unonstraned means exploraton Z 3 e.g. a polynomal trend n temp t 5

6 6 Log lkelhoods: Important: n prnple the s may ontan mssng values that wll end up n the ategory of unobserved data not n the nomplete lkelhood and wll be mputed by the EM algorthm next. + Σ Σ M T m f m l f l I Z Z f m R 2 'log 'log ' log ' ; ; ; '... {0} τ ϑ τ ϑ σ µ ϕ β ϕ β ϕ τ Unobserved omponent membershp vetors T-varate normal densty nomplete omplete

7 umeral maxmzaton va EM algorthm: E Usng the urrent parameter values ompute m E m ˆ' f ˆ τ dag ˆ f ˆ τ... M Substtute the urrent parameter values wth the maxmum of l M ϑ E l M ϑ m 'log f τ + m 'log Iterate untl onvergene. Intalzaton: 0 m... membershps from a k-means lusterng wth k-. Or other strateges dependene on ntalzaton s an ssue also here 7

8 Outomes from the last teraton: ˆ ˆ µ Σˆ ˆ pˆ Z S and m... ˆ β... possbly ˆ µ or σˆ Σˆ Estmated vetors of ondtonal prob s; membershp probabltes 2 S Estmated weghts Estmated mean patterns Estmated wthn-omponent varablty strutures Estmated ontamnaton parameters luster formaton: luster or threshold γ 0 luster resdual max{ˆ p max{ p * + th lassfor : pˆ ; γ} p * < γ } p * p * pˆ pˆ Ther dstrbuton s hgh end onentraton gves nterestng nfo on lumpness of the data n the ontext establshed by hoe of and onstrants spefaton 8

9 Frst applaton: Spellman et al expresson of yeast genes on a tme ourse overng 2+ ell yles. Log ratos; baselne unsynhronzed ulture. Selet 800 genes wth perod expresson profles. Halter et al restrt attenton to T2 equspaed tme ponts reoverng 2 ell yles and 696 profles wthout mssng values most of the varablty of the data loud s aptured by the frst two prnpal omponents; data do not appear lumpy. We use ths 696 x 2 data matrx but do not enter and standardze by row/gene profle. o mssng value mputaton; ontamnaton spheral normal; ommon wthn omponent ovarane struture. 9

10 Fts n frst applaton: K-means k8 ntalzaton for all mxture fts below Mx. Ft A: losest to k-means. -8 regular omponents plus ontamnaton. Unonstraned mean patterns. Spheral wthn-omp. ov. struture var. about mean pattern equal and unorr. over t s. Mx. Ft B: relaxaton of A; dagonal wthn-omp. ov. struture var. about mean pattern dfferent but unorr. over t s. [Mx. Ft : relaxaton of B; unonstraned wthn-omp. ov. struture var. about mean pattern dfferent and freely orr over t s]. Mx. Ft D: a restrton of B; mean patterns modeled as µ t t shft 2 β + β2t + β3 + β4tsn t perod β s ontnuously optmzed by EM optmzed at the outset over a grd 0

11 Seond applaton: Gash A.P. Spellman P.T. Kao.M. armel-harel O. Esen M.B. Storz G. Botsten D. Brown P.O. 200 Genom Expresson Programs n the Response of Yeast ells to Envronmental hanges Moleular Bology of the ell known and putatve genes on over 40 ondtons. We onentrate on a T8 tme ourse for heat shok 25 to 37 mnute Log ratos; baselnepoolng equal amounts of all expermental samples. The profles of 2509 genes 40.78% of the total have mssng values. We use ths 652x8 matrx wthout enterng and standardze by row/gene profle. Mssng value mputaton; ontamnaton unform on data range; allow for dfferent wthn omponent ovarane spefatons also dfferent

12 Fts n seond applaton: free means EEE ovaranes: -7 ommon wthn omponent ovarane struture unonstraned. free means and UUE ovaranes: -7 eah omponent has a ommon but not fxed orrelaton struture but dfferenes n overall varablty volume and dstrbuton over the tme ourse are allowed. many more also modelng means not presented 2