A Maximum Entropy Approach to Classifying Gene Array Data Sets

Transcription

1 Workshop on Data Minin for Genomics, First SIAM International Conference on Data Minin A Maximum Entropy Approach to Classifyin Gene Array Data Sets Shumei Jian, Chun Tan, Li Zhan and Aidon Zhan Department of Computer Science and Enineerin The State University of New York at Buffalo Buffalo, NY Murali Ramanathan Department of harmaceutics The State University of New York at Buffalo Buffalo, NY Abstract New technoloy such as DNA microarray can be used to determine simultaneously the expression levels of the thousands of enes which determine the function of all cells. Applyin this technoloy to investiate the ene-level responses to different dru treatments could provide deep insiht into the nature of many diseases as well as lead in the development of new drus. In this paper, we present a maximum entropy approach to classifyin ene array data sets. The experiments demonstrate the effectiveness of this approach. 1 Introduction Recently, DNA microarray technoloy has been developed which permits rapid, lare-scale screenin for patterns of ene expression, as well as analysis of mutations in key enes associated with cancer [9, 5, 15, 16, 23, 11, 12, 2, 20]. To use the arrays, labelled cdna is prepared from total messener RNA (mrna) of taret cells or tissues, and is hybridized to the array; the amount of label bound is an approximate measure of the level of ene expression. Thus ene microarrays can ive a simultaneous, semi-quantitative readout on the level of expression of thousands of enes. Just 4-6 such hih-density ene chips could allow rapid scannin of the entire human library for enes which are induced or repressed under particular conditions. By preparin cdna from cells or tissues at intervals followin some stimulus, and exposin each to replicate microarrays, it is possible to determine the identity of enes respondin to that stimulus, the time course of induction, and the deree of chane. Some methods have been developed usin both standard cluster analysis and new innovative techniques to extract, analyze and visualize ene expression data enerated from DNA microarrays. It has been found usin yeast data [13] that by clusterin ene expression data into roups, enes of similar function cluster toether and redundant representations of enes cluster toether. A similar tendency has been found in 1

2 humans. Data clusterin [1] was used to identify patterns of ene expression in human mammary epithelial cells rowin in culture and in primary human breast tumors. Clusters of coexpressed enes identified throuh manipulations of mammary epithelial cells in vitro also showed consistent patterns of variation in expression amon breast tumor samples. The enerated clusters are used to summarize enome-wide expression and to initiate supervised clusterin of enes into bioloically meaninful roups [10]. In [4], the authors present a stratey for the analysis of lare-scale quantitative ene-expression measurement data from time-course experiments. The approach takes advantae of cluster analysis and raphical visualization methods to reveal correlated patterns of ene expression from time series data. The coherence of these patterns suests an order that conforms to a notion of shared pathways and control processes that can be experimentally verified. The use of hih-density DNA arrays to monitor ene expression at a enome-wide scale constitutes a fundamental advance in bioloy. In particular, the expression pattern of all enes in Saccharomyces cerevisiae can be interroated usin microarray analysis, in which cdnas are hybridized to an array of each of the approximately 6000 enes in the yeast enome [14]. A key step in the analysis of ene expression data is the detection of roups that manifest similar expression patterns. The correspondin alorithmic problem is to cluster multicondition ene expression patterns. In [?], a novel clusterin alorithm is introduced for analysis of ene expression data in which an appropriate stochastic error model on the input has been defined. It has been proven that under certain conditions of the model, the alorithm recovers the cluster structure with hih probability. Multiple sclerosis (MS) is a chronic, relapsin, inflammatory disease. Interferon- ( ) has been the most important treatment for the MS disease for last decade [22]. The DNA microarray technoloy makes it possible to study the expression levels of thousands of enes simultaneously. In this paper, we present a maximum entropy approach to classifyin ene array data sets. In particular, we distinuish the healthy control, MS, IFN-treated patients based on the data collected from the DNA Array experiments. The ene expression levels are measured by the intensity levels of the correspondin array spots. The experiments demonstrate the effectiveness of this approach. This paper is oranized as follows. Section 2 introduces the maximum entropy model. Section 3, 4 and 5 describe the details of our approach on how to calculate features, probabilities and classification. Section 6 presents the experimental results. And finally, the conclusion is provided in Section 7. 2 Maximum Entropy Model Entropy is a measure of uncertainty of random variable [8, 18]. It represents the amount of information required on averae to describe the random variable. The entropy of a discrete random variable ( 2

3 X - E E E - E the set of which we ll call ) is defined by!#"%$'& defined as 012 3!#"%$'& it is a measure of the distance between two probability distributions. )* if and only if! (. The proof is simple.! (known as Kullback-Leibler diverence) is (! where ( is the probability. The relative entropy )*,+.- ) /+.- / and the equality occurs In data classification, the oal is to classify the data from all known information. The rinciple of Maximum Entropy [6] can be stated [18] as (1) Reformulate the different information sources as constraints to be satisfied by the taret estimate. (2) Amon all probability distributions that satisfy these constraints, choose the one that has the hihest entropy. One way to represent the known information is to encode it as features and impose some constraints on the value of those feature expectations [17]. Here a feature is a binary-valued functions on events (or data ) 8:9<;=*> 6@?BA. Given C features, the desired expectations can be formalized as where O DFE 8 9 (G8 9!IH JA?.K#?:L:L:L? C They must satisfy the observed expectation i.e. constraints. is the observed probability distribution in the trainin sample. D M 8:95 O DFE 8 9 DNM 8 9 (1) (G8:9!IH JA?.K#?:L:L:L? C The rinciple D E ofd Maximum M Entropy [6, 7, 17] recommends that we use SRT &FU R3V Q 3W where ZY 8:9 8:9? H ZA?.K#?:]:]:]3? C^. It can be shown [17] that ) _6 exists under the \[ /+Q expectation constraints and Q must have the form of! a` c b 9ed0f hji:k ml 9?n6po 9 orq (2) where ` is a constant and the 9 s are the model parameters. Each parameter 9 corresponds to exactly one feature 8:9 and can be viewed as a weiht for that feature. To find these weihts, an iterative alorithm Generalized Iterative Scalin (GIS) is used, which is uaranteed to convere to the solution [3, 17]. [3] shows that ) BO s+ 3 kut'v f l swx)* mo s+ kut l and "zyzu t'{} kut l.

4 c E ˆ Here is the sketch of the procedure. In our approach, Improved Iterative Scalin (IIS) alorithm [19] is used. where k~ l 9 JA DpM kut'v f l 9 kt l D E ƒ :9@ D E ƒ kt l (G8 9! v kut l f ( Š b kt l hibk ml 9 Œ 9ed0f, The maximum entropy model is simple and yet extremely eneral. It only imposes the constituent constraints without assumin anythin else. The feature functions can represent the detailed information accurately. Usin the maximum entropy, we can model very subtle dependencies amon variables. This is important and useful, especially in hih dimensions since all hih dimensional data are detailed information. By definin feature functions, we make reasonable, unspurious assumptions of the data. In our task of distinuishin healthy control people from MS patients, and MS patients from IFN treated patients, the information we have are the intensity values of about 4,000 enes for each identity. It is hard for human beins to look at the data and fiure out the hidden pattern of each class. It is important to develop a reliable alorithm to perform the task. In the next two sections, we first define feature functions then apply IIS to find the weihts for. Finally, a classifier is built based on these weihts. 3 Feature Definitions The feature functions are very important in applyin the maximum entropy theory. Bad features have no positive effects but causin noise and decreasin the classification precision. In the problem of classifyin healthy control and MS patients, how to transfer the ene intensity values to feature functions requires bioloy knowlede. Althouh the absolute intensity chanin values are important in classifyin patients, we believe the relative chane levels are more intrinsic. Also, different enes have different intensity value chanin levels. The intensity chane level alone by itself has no meanin. It varies with the the ene intensity chanin level (denoted <Ž ) for each patient and each ene. Our eneral formula is }Ž/ n n š œbž=œ Ÿ <Ž/. where n represents the intensity value for ene of person, is a parameter, and is the mean of the intensity values for ene for all patients. Since the <Ž s values are real numbers, we also bucket them into 21 predefined buckets }Ž by 4 [ [ (3)

5 A c [ }Ž n ª }Ž/ n Q «A:6 A<o }Ž/ n o A A:6 <Ž n wš A A:6 <Ž, n 0 ŠA Different bucketin strateies affect the performance. One more definition is needed before we define the feature functions. We divide all patients into three data classes : healthy control ( ), MS patients ( ) or IFN treated patients ( ± } ). Now for each ene, each chanin level bucket ²1³ and each class ², we define a feature functions 8 : µ2 n µ ; > 6@?BA to be 8 : µ2 µ ¹8»º¼,½?0¾m ²:š š À F½»²5À±Á  8(Ã3ž #œbáäœå ±ÆÃ'8Ç? Æ?È }Ž néu ²³ 6 à š œ:ž Ÿ ¾ œ Here we have multiple enes, multiple ene intensity value chanin levels, and multiple roups, each combination of them makes up one feature function. 4 robability The ultimate oal is to classify all kinds of people into different classes. We can treat the intensity value of each ene as the context to decide the patient class. Here, class ( ) has three values:, Context is defined as Ê Ë? ²³ and ± }. i.e. ene and its intensity value chanin level bucket. Which class the patient belons to depends on all the context information it has. In our situation, we adopt a probability model to describe it. If we can find the conditional probability j²³jà ¾3¾ ²1Ã3Á œ: G¾ for each class, we can claim that the patient belons to the class with the hihest probability based on the context information. where j²³à ¾3¾ j²ã3á œb ¾=? ²³jÀ ¾¾ ²Ã3Á œb ¾ [ 9 j²ãá œ: G¾? ²³jÀ ¾3¾ j²ã3á œb ¾ 9 H Ê? ²B Ë? ²³? ²: Weihts 9 s are overned by and Ê? ²B IÌ 9 hjibk ±l Í 9 Ê Ë? ²³ ²¼½ (4) 5

6 Ó Ó Ó c Ô Í kuî µ l 9 hjibk ±l 9 Í Thus, is a normalization constant, 9 s are the model parameters. Compare the format of equation (2) and (4), 8:9 s here are the feature functions we defined in the previous section with H Ë? ²³? ²B. Accordin to maximum entropy model, we can apply IIS (Improved Iterative Scalin [19]) to calculate 9 s. 9 s are viewed as weihts for 8 9. We call this process the trainin stae. There are two steps, the feature function induction and weiht evaluation [19]. In the feature function induction step, when a sinle candidate feature function is introduced, we calculate the reduction of the Kullback-Leiber diverence by adjustin the weiht of the candidate feature function while all the other parameters are kept constant. After one feature function is selected, all the weihts of the selected feature functions are recalculated. IIS (Improved Iterative Scalin) alorithm is adopted to calculate the model parameters. The loop stops when the lo-likely ain is less than the predefined threshold. The whole structure is shown in Fiure 1. ÏÐ ÑÒ Feature function space Select the next feature function which reduces the Kullback-Leibeler diverence most Evaluate weiht for each selected feature function Stop the process when the Lolikehood ain is less than predefined threshold Fiure 1: Trainin Structure. 6

7 [ ² c Ó Ó Ó [ 5 Classification In practice, amon all the 4132 enes for each person, not all enes have the same contribution in distinuishin the classes. Actually, most of them have little contribution. We need to select some enes which are more important than others in solvin the problem. To find those important enes, first, all enes are sorted by their deree of correlation, then the neihborhood analysis method is applied to extract the enes which are more correlated with the class distinction than other enes [21]. For all, we choose 88 enes for each identity. and } After trainin stae, classification can be preformed easily. Given a patient ¾, we first calculate the ene intensity chanin levels for all his enes, then construct the feature functions. From the trainin stae, we have weiht 9 for all 8:9 of each class ². We calculate j²³jà ¾3¾ ²Ã3Á œ: ¾ for all classes. Actually, only Ì 9 data, 9 is necessary since all the denominators are the same. Hiher for a class indicates hiher probability of the sample belon to that class. Finally we set the sample data to the class ² such that j²³à ¾3¾ ²Ã3Á œb ¾ is the hihest i.e. art &ÕU R3V µ Ö 9 hibķ l 9 H Ë? ²³? ²B The structure is shown in Fiure ÏÐÑÒ 2. Given patient s Calculate the <Ž for all enes For each class ² construct feature functions 8 9 H Ë? ²³? hjibķ ²B, then compute l Ì 9 9 usin 9 in the trainin stae. Assin the patient to class ² Ì 9 hjibķ l 9 is larest H Ë? ²³? ²B for which Fiure 2: Classification Structure. 7

8 6 Experimental Results The experiments are based on two different mix of the data sets: the MS IFN roup and the CONTROL MS roup. The MS IFN roup contains 14 MS samples and 14 IFN samples while the CONTROL MS roup contains 15 control samples and 15 MS samples. We perform the classification separately on each roup. For the MS IFN roup, in each experiment, we conduct 14 tests. In each test, we choose one different sample from the 14 MS samples and one different sample from the 14 IFN samples to make the test set, and use the other 26 samples as the trainin set. Thus each sample appears just once in the test set and the total number of samples we test is 28 which is the cardinality of the dataset. Similarly, for the CONTROL MS roup, in each experiment, we conduct 15 tests. In each test, we choose one different sample from the 15 control samples and one different sample 15 MS samples correspondinly as the test set, and use the other 28 samples as the trainin set. The total number of samples we test is 30. For each data set, we perform several experiments by adjustin the parameter to calculate chanin level CL in the formula Equation (3). In Table 1, we use the error classification number to evaluate the performance of our approach. We choose five different values varyin from 0.5 to 3 to perform five experiments on each data sets. As it can be observed from Table 1, different calculations of the chanin level will affect the testin result. Experiment# arameter t Error# of MS IFN(out of 28) Error# of CONTROL MS(out of 30) Table 1: Experiment results. 7 Conclusion In this paper, we have iven a maximum entropy approach to classifyin ene array data sets. In particular, we used the above approach to distinuish the healthy control, MS, IFN-treated patients based on the data collected from DNA Array experiments. To the best of our knowlede, the maximum entropy has not been used before to classify ene data. From our experiments, we demonstrated that the maximum entropy approach is a promisin approach to be used for classifyin ene array data sets. 8

9 8 References [1] Charles M. erou, Stefanie S. Jeffrey, Matt Van De Rijn, Christia A. Rees, Michael B. Eisen, Doulas T. Ross, Alexander eramenschikov, Cheryl F. Williams, Shirley X. Zhu, Jeffrey C. F. Lee, Deval Lashkari, Dari Shalon, at rick O. Brown, and David Bostein. Distinctive ene expression patterns in human mammary epithelial cells and breast cancers. roc. Natl. Acad. Sci. USA, Vol. 96(16): , Auust [2] D. Shalon, S.J. Smith,.O. Brown. A DNA microarray system for analyzin complex DNA samples usin two-color fluorescent probe hybridization. Genome Research, 6: , [3] J. N. Darroch and D. Ratcliff. Generalized iterative scalin for lo-linear models. The Annals for Mathematical Statistics, 43(5): , [4] G.S. Michaels, D.B. Carr, M. Askenazi, S. Fuhrman, X. Wen and R. Somoyi. Cluster Analysis and data visualization of lare-scale expression data. In ac Symposium of Biocomputin, volume 3, paes 42 53, [5] J. DeRisi, L. enland,.o. Brown, M.L. Bittner,.S. Meltzer, M. Ray, Y. Chen, Y.A. Su, J.M. Trent. Use of a cdna microarray to analyse ene expression patterns in human cancer. Nature Genetics, 14: , [6] E. T. Jaynes. Information theory and statistical methanics. hsics Reviews, 106: , [7] E. T. Jaynes. apers on robablity, Statistics, and Statistical hysis. R. Rosenkrantz, ed., D. Reidel ublishin Co., Dordrecht-Holland, [8] F. Jelinek. Statistical Methods for Speech Reconition. The MIT ress, [9] J.J. Chen, R. Wu,.C. Yan, J.Y. Huan, Y.. Sher, M.H. Han, W.C. Kao,.J. Lee, T.F. Chiu, F. Chan, Y.W. Chu, C.W. Wu, K. eck. rofilin expression patterns and isolatin differentially expressed enes by cdna microarray system with colorimetry detection. Genomics, 51: , [10] L.J. Heyer, S. Krulyak and S. Yooseph. Explorin Expression Data: Identification and Analysis of Coexpressed Genes. Genome Res, [11] M. Schena, D. Shalon, R.W. Davis,.O. Brown. Quantitative monitorin of ene expression patterns with a complementary DNA microarray. Science, 270: , [12] Mark Schena, Dari Shalon, Renu Heller, Andrew Chai, atrick O. Brown, and Ronald W. Davis. arallel human enome analysis: Microarray-based expression monitorin of 1000 enes. roc. Natl. Acad. Sci. USA, Vol. 93(20): , October [13] Michael B. Eisen, aul T. Spellman, atrick O. Brown and David Botstein. Cluster analysis and display of enome-wide expression patterns. roc. Natl. Acad. Sci. USA, Vol. 95: , [14] M.Q. Zhan. Lare-scale ene expression data analysis: a new challene to computational bioloists. Genome Res, [15] O. Ermolaeva, M. Rastoi, K.D. ruitt, G.D. Schuler, M.L. Bittner, Y. Chen, R. Simon,. Meltzer, J.M. Trent, M.S. Bouski. Data manaement and analysis for ene expression arrays. Nature Genetics, 20:19 23, [16] R.A. Heller, M. Schena, A. Chai, D. Shalon, T. Bedilion, J. Gilmore, D.E. Woolley, R.W. Davis. Discovery and analysis of inflammatory disease-related enes usin cdna microarrays. roc. Natl. Acad. Sci. USA, 94: , [17] A. Ratnaparkhi. A simple introduction to maximum entropy models for natural lanuae processin, [18] R. Rosenfeld. Adaptive Statistical Lanuae Modelin: A Maximum Entropy Approach. hd thesis, Carneie Mellon University, [19] S. ietra, V. ietra, and J. Lafferty. Inducin Features of Random Fields. IEEE Transactions attern Analysis and Machine Intellience, 19(4):1 13,

10 [20] S.M. Welford, J. Gre, E. Chen, D. Garrison,.H. Sorensen, C.T. Denny, S.F. Nelson. Detection of differentially expressed enes in primary tumor tissues usin representational differences analysis coupled to microarray hybridization. Nucleic Acids Research, 26: , [21] T.R. Golub, D.K. Slonim,. Tamayo, C. Huard, M. Gassenbeek, J.. Mesirov, H. Coller, M.L. Loh, J.R. Downin, M.A. Caliiuri, D.D. Bloomfield and E.S. Lander. Molecular classification of cancer: Class discovery and class prediction by ene expression monitorin. Science, Vol. 286(15): , October [22] V. Yon, S. Chabot, Q. Stuve and G. Williams. Interferon beta in the treatment of multiple sclerosis: mechanisms of action. Neuroloy, 51: , [23] V.R. Iyer, M.B. Eisen, D.T. Ross, G. Schuler, T. Moore, J.C.F. Lee, J.M. Trent, L.M. Staudt, Jr. J. Hudson, M.S. Bouski, D. Lashkari, D. Shalon, D. Botstein,.O. Brown. The transcriptional proram in the response of human fibroblasts to serum. Science, 283:83 87,