Module Based Neural Networks for Modeling Gene Regulatory Networks

Transcription

1 Module Based Neural Networks for Modeling Gene Regulatory Networks Paresh Chandra Barman, Std 1 ID: Term Project: BiS732 Bio-Network Department of BioSystems, Korea Advanced Institute of Science and Technology Daejeon, , Republic of Korea, pcbarman@yahoo.com ; pcbarman@neuron.kaist.ac.kr ABSTRACT. A new hybrid neural network model is proposed to model the gene regulatory networks. As we know most of a cell s activity is organized as a network of interacting modules: set of genes coregulated to respond to different conditions. This hybrid neural network consists with two parts: first part is for gene module extraction, which follows the Non-negative Matrix Factorization (NMF algorithm, and the second part is a Single Layer Perceptron (SLP to represent regulatory relationship between regulators and gene modules as a function of weight matrix. Our procedure identifies modules or cluster of semantic genes on the basis of their expression labels. The procedure follows the hypotheses M o d u l e Y r e g u l a t e s r e g u l a t o r X u n d e r c o n d i t i o n W. The proposed network e u k e m i a c a n c e r model has been tested to the L data. The network performance is investigated as the functions of the gene modules numbers, and also the slope of sigmoidal nonlinearity at the hidden neurons. Proposed the reverse engineering approach i.e., predicting gene expression profile for a given cell response. The developed model demonstrates a well defined gene regulatory network even for a smaller number of given experimental data point. 1 Introduction Systematic gene expression analyses provide comprehensive information about the transcriptional response to different environmental and developmental conditions. As these expression analysis technologies mature, biologists will be presented with accumulated data sets detailing the transcriptional response of a cell, tissue, or organism to many environmental, genetic, and developmental stimuli. In addition to elucidating the cellular response to such stimuli, these experimental results provide an opportunity to understand the regulatory pathways that underlie the observed gene expression patterns. While our ability to predict such regulatory pathways will remain rudimentary with limited data, as more data points are collected, we will be able to define ever more accurate predictions of the transcriptional regulatory pathway. It has been observed that most of the microarray data consists with limited number of experimental data points as compare to the number of genes. So it is very difficult to predict the regulatory path way by simply using the weighted matrices [1, 2]. In our model we try to solve these problems by defining a small numbers of modules of semantic genes according to their expression profile. The NMF network [3] of the proposed hybrid network provides the modeling or clustering the genes. Currently NMF algorithm is widely using for clustering the microarray data [4, 5, 6] or text data [7] we have extended the NMF algorithm to identify the gene regulatory network by adding a single layer perceptron. The probabilistic graphical models [8] identified the regulatory modules and their condition-specific regulators from gene expression data which provides a clear global view of functional modules, but one important limitation in this method is to define a proper number of modules. In our approaches this limitation is not too much important. In the proposed (NMF_SLP model a bipolar sigmoid function has used as a transfer function for signal transaction from NMF hidden layer to the input of SLP layer. The transfer function parameter η 1 has adjusted by minimizing the average Sum Squared Error (SSE of the trained SLP network. The regulative interaction matrix or weight matrix W r between the modules and the cells has been determined by using the gradient descent update rule. Finally cells response has been determined for a test set of gene expression profile. We have also modeled the reverse engineering i.e., for a given cell response predict or determine the corresponding gene expression profile. 1 To whom it will be correspondent : Paresh Chandra Barman, Computational Neuro Systems Lab, Dept. of BioSystems KAIST, Republic of Korea, pcbarman@yahoo.com

2 Term Project: BiS732 Bio-Network, Dept. of BioSystems, KAIST 2 Propose NMF_SLP Model for Modeling Gene Regulatory Network NMF_SLP model consists with two adaptive layers of the neural networks the basis block diagram has shown in figure 1. Input layer to apply the gene-expression profile Vn m, m represents the number of experimental time point or cell response, and n is the number of genes. The hidden layer (NMF feature extraction layer This is unsupervised adaptation layer. This layer extracts r number of modules from the input matrix V. The modules are semantic representation of the gene expression levels. The connection weight matrix W n r represents the r module vectors with dimension n, and the hidden layer response or module coefficient matrix r m represent the m time point s coefficient corresponding to r modules. The unsupervised adaptation of this layer is made by using the NMF algorithm (see Sect. 2.1 subject to minimizing the KL divergence among the input matrix V and the multiplication of weight matrix W and the coefficient matrix, i.e. ˆ. Fig. 1. The simple diagram of the NMF_SLP model for modeling gene regulatory network Classification or output (Single Layer Perceptron layer: This is supervised layer. This layer classifies the output of the hidden layer =f( into a given number of classes, on the basis of minimizing the error among the network output and target i.e., the well-known gradientdescent learning algorithm. Activation or transfer Functions: Two types of activation or transfer function have been used: function f for the hidden layer is the bisigmoid or bipolar sigmoid = ( ˆ = ˆ ( η + where Ĥ=W -1 V and the activation function for the output layer is unipolar sigmoid = ( ˆ = ( + η ˆ where Ô = U ; U c r is the hidden to output connection weight matrix, c is the number of document classes and r is the number of basis clusters. 2.1 Unsupervised Adaptation Rule For a given non-negative matrix V n m ; find non-negative factors, semantic modules W n r, and encoded matrix r m, such that: V W or

3 * "# & 6, % "# & * 6 / ˆ *!, ( +, 9 D? 5 ˆ C & B "$ D A 8 & B "$ 9 ' & D % 8 B "# Term Project: BiS732 Bio-Network, Dept. of BioSystems, KAIST ˆ ( = ˆ ( where r is chosen as r < nm n + m For our application purpose we make a single modification in the update rule of [3] [7]. In our case we also normalize the encoding matrix, like W, which are as follows, ˆ ( + ˆ ( ( = [ ], ( ( [ ˆ ( ] ˆ ( + ˆ ( + = ( ˆ ( + ( [ ˆ ( ], [ ˆ ( + ]# $ ( ( + ( -( + = ( + = ( + + D - - ij k ik kj Where Q A B. = A B = AB, and all the (. ij indicates that the noted division and multiplications are computed element by element. ( 2.2 Supervised Single Layer Perceptron Learning For this layer considering obtained by using equation 1, as the input for the SLP, find the net value Ô = W r *, and the output O by using equation 2. The Sum Square Error (SSE: E=(T-O 2 /2 where T is the target matrix. The weight matrix Wr by using the gradient descent update rule [8], such as = η ( ( 2 where k represents the iteration steps and η 2 learning rate constant. 2.3 Proposed Algorithm of Reverse Engineering Let us consider we for a certain gene expression profile we have already estimated the model parameters such as regulative interaction matrix Wr, transfer function parameters, and module weight matrix W, now if we have certain values of regulated cell response O r at a certain time t, then we can easily predict the corresponding gene expression profile by following algorithm: 1. Input: W, Wr, cell response (O r, η 1, η 2, Output gene expression profile V r. : ; < = : ; < = 2. determine O [ ] η O r O r 3. determine invwr O E F G C E F G C ˆ + η 4. determine [ ] 5. finally W V r 3 Data Set and Preprocessing Leukemia data: Todd Golub s genomic and computational approaches to cancer biology and cancer medicine represent seminal efforts in cancer microarray study design and analysis. Golub s 1999 [9] analysis of his leukemia data sets using hierarchical clustering (C and self-organizing map (SOM techniques is a first generation benchmark methodology for molecular classification. It is well know that acute leukemias may be classified as acute lymphoblastic leukemia (ALL originating from lymphoid precursors or acute myeloid leukemia (AML originating from myeloid precursors. ALL types can be further classified into Tlineage and B-lineage subtypes. Ramaswamy and Golub noted that distinct cellular Precursors likely account for the robust expression signatures that distinguish these

4 Term Project: BiS732 Bio-Network, Dept. of BioSystems, KAIST two cancers. [10] This distinction is critical for treatment planning but it is clinically difficult to determine. Golub applied a variety of clustering algorithms to systematically determine cell class without subjective analysis. ere we used this database to predict the module gene regulatory networks corresponding to cell response. In our experiment we have divided the data base into training (75% and testing (25% samples sets and make the data base non-negative by shifted the average above zero. The gene expression profile V is represented like figure 2. The graphical representation of the data base clustering by NMF has shown in figure 3. L L L L = L L M M M O M M M M M M O M L L Fig. 2. The simple example of the Gene expression profile V, where g s represent the genes, v i,j represents the expression levels of gene i, at experimental time point or corresponding cell C j, and C s represents the cells. 4 Experiments Apply the normalized V to the input of the NMF_SLP neural network. By using the NMF update equation 3 to 7, determine the modules W, where each column of W represents a module vector, (such as the i th column w i represents the i th module vector, i= 1, 2, r, where r is the number of modules and the coefficient matrix. Then using the transfer function as shown in equation 1, calculate the hidden layer output, and then adjust the perceptron layer connection weight matrix Wr using equation 8. After training the network for training set of gene expression profile, store both the basis cluster matrix W and SLP connection weight matrix Wr. Then for the test set of gene expression profile V ts (ts denotes the gene expression matrix for the test profile set, where n is same for both case, calculate = W -1 V ts. Using equation 1 calculate, then by using the trained weight matrix Wr and activation function equation 2 calculate the output or cell response O. We observe the NMF-SLP hybrid Neural Network s performance with respect to the following parameters and methods: We try to adjust the parameter η 1 and η 2 ; in our experiment we used η 1 =5 and η 2 =0.005 (as we considered in one of our previous work but the values may be dependent on experimental data set We determine different number of modules and investigate the corresponding regulative interaction matrix Wr (Fig. 5 and 7. Finally we observe the performance for various numbers of modules such as r= 3, 5, 7 (Fig. 8 shows the result. 5 Results We choose the bi-sigmoid transfer function from the output of NMF hidden layer to the input of SLP network and adjust the parameter η 1. Initially we consider the value of η 1 from 0.1 to 10 then observe the training and testing accuracy and also the SSE (result has not shown here of the SLP network. As we know each column of the coefficient matrix of the NMF hidden layer is positive and it is column-wise normalized, i.e., sum of each column is unit. So the distribution of each column is a random probability distribution. It has been also observed (Fig. 4 shows some example that the distribution of bipolar sigmoid function for an independent variable x i (with in the range from 0 x i 1 is fully distributed over the range from 0 to 1, if the value of parameter eta1=η 1 = 5. For the case of unipolar sigmoid function the distribution is always incomplete compare to the bipolar sigmoid function. Due to this reason we choose the bipolar sigmoid function as the transfer function from NMF hidden layer output to SLP input layer.

5 Fig.3. this figure represents Leukemia data set in the clustered format (using NMF clustering Algorithm; there are three categories of Leukemia cancer cells, ALL-B type (19 cells, ALL-T type (8 cells and AML type (11 cells. Fig.4. This figure represents training and testing accuracy of the NMF_SLP network with respect to the different values of parameter eta1 (from 0.1 to 50 of the bipolar sigmoid transfer function Figure 5 shows the gene modules for different numbers of modules. It has been observed that each modules represents different set of semantic or co-regulated gene groups. According to the significant labels of the semantic or co-regulation of the gene expression profile top-most 25 genes for each module has been shown in figure 6. Few of the genes shows significant expression label for each module or groups and the rest are insignificant. Although we have not tested but it is possible to reduce the dimension of module vectors by excluding insignificant expression label as a noise. Figure 5a, and 5b, it has been observed that one or two modules are not contains any significant gene expression labels we can exclude those modules from our final model. It has been observed from figure 7b and 7c that few connection weights are almost zero. Fore example Module 1 of Fig. 5b does not contains highly significant gene expression labels correspondingly the weight element W1 in figure 7b are almost zero, which means the regulative interaction for this module to the cell response is negligible or there are no interactions among those gene group and the corresponding cell or cell types. Figure 8 represents the regulated cells responses are maximum to the corresponding cell categories, which ensure the network function or performances. Even though in our experimental data set the number of samples is not large but it works well. From figure

6 7 is has been observed that for a certain categories of cell some of the modules are stimulating i.e. the value of interaction weight is positive, some has no interaction i.e., weight is zero, and some modules show repression interaction i.e., negative weight. For example in figure 7a (top most panels shows for cell categories one (T-type module one interact as stimulating, while the others show repression interaction. Fig.5a. 3 modules extracted by the NMF layer Fig.5b. 5 modules extracted by the NMF layer

7 Fig.5c. 7 modules extracted by the NMF layer Fig. 6a Top 25 genes according to the significant expression corresponding to each module

8 Fig. 6b Top 25 genes according to the significant expression corresponding to each module Fig. 6c Top 25 genes according to the significant expression corresponding to each module

9 Fig. 7a adjusted interaction or weight matrix between modules (W1, W2, W3 + 1 bias (W4 and 3 regulated cell types Fig. 7b adjusted interaction or weight matrix between modules (W1, W2, W3, W4, W5 + 1 bias (W6 and 3 regulated cell types

10 Fig. 7c adjusted interaction or weight matrix between modules (W1, W2, W3, W4, W5, W6, W7 + 1 bias (W8 and 3 regulated cell types Indices Test cells 1 ALL_5982_B-cell 2 ALL_7092_B-cell 3 ALL_R11_B-cell 4 ALL_R23_B-cell 5 ALL_17638_T-cell 6 ALL_22474_T-cell 7 AML_5 8 AML_6 9 AML_7 # of Module =7 Test cells B-type T-type AML-type # of Module =5 B-type T-type AML-type # of Module =3 B-type T-type AML-type Fig 8: a. Represents the test set of cells and b. represents the response for different cell types

11 6 Conclusion and Future Works The main focus point in this term project is to introduce a two-layer hybrid (NMF-SLP neural network for predicting the module based gene regulatory network. Where the first layer (NMF layer is adopted for the unsupervised module extraction and the supervised learning is implemented for the training of Single layer Perceptron (SLP neural networks for the adjusting the regulative interaction between the extracted gene modules and corresponding to regulated cell response or experimental time points. We used a bipolar sigmoid function as a transfer or activation function to transmit the hidden layer signal to the input of the SLP network to distribute the module effects in the wide area of the unit space so that the interaction effect of the modules are become distinguishable. Since the NMF hidden layer response is always positive i.e., 0 1; for the case of unipolar sigmoid transfer or activation function the distribution of the responded data imitated to the half of the unit space, i.e., 0.5 to 1. So the module coefficients are much more compact compare to the bipolar sigmoid function. It has been observed that if the number of modules become more than the number of sample categories some of the modules become insignificant and does not provide significant interactions to the regulated cell or cell groups, in some cases more than one module share common semantic gene groups they also share the same interaction to the regulated cell or cells group. So choosing the number of modules is little bit relax with in the limit of equation 3. Figure 8 it has also observed that the regulated cells responses are maximum to the corresponding cell categories, which ensure the network function or performances. Even though in our experimental data set the number of samples is not large but it works well. This work can be easily extended to predict the artificial or approximate gene expression profile. To valid this approach we need to observe for different types of micro array data. References 1. D. C. Weaver, C.T. Workman, G. D. Storm: Modeling Regulatory Networks with Weight Matrices, Pacific Symposium on Biocomputing 4: ( Jennifer allinan, Janet Wiles: Evolving Regulatory Networks Using an Artificial Genome, Asia-Pacific Bioinformatics Conference (APB D. D. Lee and. S. Seung: Learning the parts of objects by non-negative matrix factorization. Nature, 401(1999: Pascual-Montano, A.1,Carmona-Sáez, P.2, Pascual-Marqui, R.D.3, Tirado, F.1, Carazo, J.M. : Two-way clustering of gene expression profiles by sparse matrix factorization, Proceedings of the 2005 IEEE Computational Systems Bioinformatics Conference Workshops (CSBW 05, IEEE Guoli Wang, Andrew V Kossenkov and Michael F Ochs: LS-NMF: A modified non-negative matrix factorization algorithm utilizing uncertainty estimates, BMC Bioinformatics2006, 7: Philip M. Kim and Bruce Tidor: Subsystem Identification Through Dimensionality Reduction of Large-Scale Gene Expression Data, Genome Res : P.C. Barman, Nadeem Iqbal, Soo-Young Lee; Non-negative Matrix Factorization Based Text Mining: Feature Extraction and Classification, I. King et al. (Eds.: ICONIP 2006, Part II, LNCS 4233, pp , Springer- Verlag, ( Jacek M. Zurada,: Introduction to Artificial Neural Systems, Chapter 2 & 3, West Publishing Company ( T.R. Golub and D.K. Slonim et al. Molecular classification of cancer: Class discovery and class prediction by gene expression monitoring. Science, 286: , P. Tamayo and S. Ramaswamy. Expression Profiling of uman Tumors: Diagnostic and Research Applications. June 2002.