RNA Secondary Structure Prediction by MFT Neural Networks

Transcription

1 N econdary tructure Prediction by MFT Neural Network B. POLLONI, L. LOTOTO,. MOPUO,. ZNBONI Dipartimento di cienze dell Informazione Univerità degli tudi di Milano Via omelico 39, 035 Milano ITLY btract: - We propoe a neural network for the prediction of N econdary tructure. It i an MFT (mean field theory) network, a determinitic model with continuou activation value baed on the mean field approximation of the Boltzmann machine. The network learn from example to predict the econdary tructure of tn molecule which take the cloverleaf hape. The propoed network can alo be ued in the earch for gene coding tn molecule within DN equence. Key-Word: - N tructure prediction, mean field theory, neural network, learning from example. Introduction Protein are the catalyzer and building block of cell. The proce of producing protein i therefore a very important one in life. It conit in the trancription and tranlation of part of the DN (the coding region or gene) with N molecule, of which there exit different type, depending on the tak they have to accomplih. n N molecule conit of a long chain of element called ribonucleotide. Each ribonucleotide contain one of four poible bae: adenine (), guanine (), cytoine (), uracil (U). The bae' equence, called primary tructure, ditinguihe an N from another and univocally determine the way an N molecule will fold into it econdary tructure. Knowing the econdary tructure of an N molecule i of primary importance in order to undertand the role played by it in the ynthei of protein. Determining the econdary tructure of an N molecule uing technique uch a x-ray crytallography and nuclear magnetic reonance i difficult and cotly. For thi reaon mathematical model for the prediction of N econdary tructure have been developed. The N econdary tructure prediction problem ( N) i a computationally complex one: the number of poible tructure that can originate from a equence i exponential in N, the length (number of bai) of the equence. Heuritic and implification have yielded algorithm that are polynomial in N, but their predictive rate are low. mong the algorithm propoed for the N problem let u mention the combinatory one, uch a Pipa and McMahon' [8], tudnicka et al.' [], Papanicolau et al.' [6], and the recurive dynamic one uch a Zuker and tiegler [5] and Zuker' [4]. ecently a prediction technique baed on the ue of a neural network wa propoed by teeg [0]. In thi paper we further develop teeg approach o that it can be applied to real (and longer) N equence. The N problem not only i itelf a relevant one, but can alo be conidered a good ground for teting the validity of thi approach in molecular tructure prediction. The importance of predicting more complex tructure, uch a protein econdary and tertiary tructure, i indeed well known. The paper i organized a follow: in the next ection we formulate the problem. In ection 3 we dicu the problem repreentation on a neural network, the coding of input and output, and the network configuration. In ection 4 we preent our experiment and reult. Finally in ection 5 we give our concluion. Problem Formulation. N econdary tructure The econdary tructure of an N molecule i defined a it plane folding due to hydrogen bonding between complementary bae according to Waton and rick' model (-U, U-, -, -) [3]. The dangling pairing -U i alo frequent. We will call the et {-U, U-, -, -, -U, U-} of pairing extended Waton and rick model. n N molecule fold into a minimum free energy tructure. The econdary tructure of an N molecule i compoed of elementary ubtructure, ditinguihed in ingle and double tranded region. In the double tranded region, called tem, the molecule form a double helix of paired complementary bae. ingle tranded region include different type of loop (hairpin loop, bulge loop, internal loop, multiple loop) and dangling end. The minimum ize of a tem i two pair of bae. Figure depict ome of the ubtructure type. N molecule fold alo into tertiary tructure (e.g. peudoknot, pairing between ingle-tranded region), but the interaction which are reponible for them are weaker than thoe reponible for the econdary

2 tructure and will thu not be taken into account in thi paper. H D U M H ite for the peptide bond Fig.. ubtructure in cloverleaf pattern taken by a tn molecule. (: tem, H: hairpin loop, M: multiple loop, D: dangling end).. Mathematical Model n N molecule can be repreented [0] a a equence of ymbol,,., n where i i one of the four,,, U nucleotide. ubequence of can be called equence if thi doe not caue confuion. iven a equence, it econdary tructure can be repreented by an n-by-n matrix. ij i if the bae in poition i and j in the equence are paired. The econdary tructure can alo be repreented by a lit P of pair, where (i,j) i in P if and only if i and j are the indexe of two nucleotide paired according to the extended Waton and rick model. pairing can be referred to a i j. The ubequence from i to j i written [i,j]. ubequence i proper with repect to a econdary tructure P if for every paired element of the ubequence it partner i alo in the ubequence. If i j i a pair and i<r<j, then we ay i j urround r. Likewie i j urround r if it urround both r and. ubequence [i,j] i cloed with repect to a econdary tructure P if (i,j) i in P. pair p q or an element r in a proper ubequence [i,j] are acceible in [i,j] if they are not urrounded by any pair in [i,j] except poibly i j. cycle c i a et coniting of a cloing pair i j and all pair p q and unpaired element r acceible to it. n N molecule fold into it econdary tructure ubject to the following contraint [0]:. Pair do not overlap: if P contain (i,j), then P cannot contain (i,k) if k j, or (k,j) if k i.. For all i, (i,i) cannot be in P. H 3. Knot are not allowed: if h<i<j<k, then P cannot contain both (h,j) and (i,k). 4. No harp loop are allowed: if P contain (i,j), then i and j are at leat 4 bae apart. 5. Bae pair according to Waton and rick' extended rule: if P contain (i,j), then i and j are either and, or and, or and U, or U and, or and U, or U and. equence will take the econdary tructure P ubject to the biological contraint given above that ha minimum free energy econdary tructure P can be decribed a compoed by elementary ubtructure. The cycle of P are it loop and tacked pair. ccording to the formalim decribed above, a tem can be decribed by a equence of pair i j, (i+) (j-),., (i+h) (j-h). The other ubtructure can be decribed in an analogou way [0]. ll the poible pairing region of an N molecule can be viualized on a dot-plot matrix, a propoed by Maizel and Lenk [5]: the equence of nucleotide i reported along the two ide of a two-dimenional matrix, and a dot i put in the cell correponding to a pair of complementary bae. It i ufficient to conider the upper right triangular ubmatrix, thank to the immetricity of complementariety. andidate tem will how a egment perpendicular to the matrix' main diagonal. U U U U Fig.. Dot-plot matrix for UU..3 tructure and Free Energy The thermodynamical tability of an N molecule folded into it econdary tructure i meaured in term of free energy. mall ize N molecule have been hown to fold into minimum free energy tructure, wherea it i not known if for molecule of greater ize the tructure i the one with local free energy minima or that with global minimum free energy. Tinoco and Uhlenbeck [] olve the dilemma by hypotheizing a very important property for the prediction of a econdary tructure of an N molecule:

3 the free energy of a econdary tructure P i the um of the free energy value of P' cycle, E(P) = Σ i E(c i ) where E(P) i the free energy and c i are P' cycle. Each ubtructure contribute independently to the tructure' total free energy. ome ingle-tranded region, uch a the dangling end, though not cycle, alo give a mall contribution to the free energy. Much reearch ha been done on the problem of aigning free energy value to the ubtructure of N molecule. Experiment on ynthetic nucleotide (which are mall molecule) yielded etimate for uch value. The main double helix tabilizing energy i due to hydrogen interaction between neighbouring tacked pair. Thi energy, together with the one given by the bond between complementary bae, give the tacking energy. The mot table econdary tructure, thoe having the lowet free energy, are long chain of tacked pair, that i tem, which are the only type of cycle which contribute negative free energy to the tructure. The energy value of each complementary pair of bae depend both on the two pairing bae and on the pair of bae immediately next to them. Loop contribute poitive free energy, proportionally to their dimenion. Table with free energy value in kcal/mole were calculated by aler [9]. ll tn molecule hare a particular overall hape, called cloverleaf pattern, but the poition where the ubtructure occur can differ ignificantly []..4 MFT Neural Network In the literature many neural network model have been propoed. In [3] a model for the olution of contrained earch problem i propoed. The model i characterized by the exitence of a cot function, called energy E, which meaure contraint violation at the whole ytem level: M M M E = wij i j + λii i= j= i= where M i the number of node, i i the activation value of node i, w ij i the connection weight between node i and j, λ i i the threhold of node i. Function E mut not be confued with the free energy of the molecule econdary tructure. Hopfield [3] howed that the motion equation for the dynamic of a ymmetrical network take the network to a table tate. With aynchronou updating the network table tate are exactly the local minima of it energy function. uch tabilization proce, called relaxation, perform a parallel earch. The updating rule i a gradient decent one, therefore only locally optimal olution (local minima) can be found. To ecape local minima the Boltzmann machine [chie85] ha been developed, which exploit imulated annealing and ue the following tochatic updating rule: each unit k et it activation value to, regardle of the previou tate, with probability P( k ) = /(+e E k /T ), where T, called temperature, i a meaure of randomne in the ytem. drawback of thi model i that computation time grow ignificantly. lternatively one can ue continuo activation level, a in [4], [7]. In particular, the model propoed by Peteron and nderon [7] i a determinitic verion of the Boltzmann machine baed on Mean Field Theory (MFT). With thi method computation time are much lower than with a Boltzmann machine, at the price of ome approximation, which had no ignificant effect in our experiment. The node updating rule for an MFT network i: m i = tanh β wij m j j where β =/T. The imultaneou olution on all node to thee non linear equation can be obtained by iterating: new old m i = tanh β wij m j () j combined, if neceary, with a gradual lowering of T (imulated annealing). m i i really an etimate of < i >, the time average taken for the tate of unit i. In a Boltzmann machine learning i obtained with gradient decent to minimize relative entropy between the external ditribution that mut be learned, given by the example, and the network' tationary ditribution. In an MFT network the target i the ame, but it i obtained in way analogou to node updating. What i needed i the equivalent m ij of the correlation < i j > between connected unit ampled in the Boltzmann machine learning algorithm. Mean field annealing yield: m = + ij tanh β w jk mik tanh β wik m jk k k for which the ame iterative technique a for m i in equation () can be ued, but which i in the practice further approximated with the factorization [7]: m ij = m i m j. 3 n MFT Network for tn econdary tructure Prediction For the prediction of the econdary tructure of tn molecule we propoe a Hopfield neural network which ue mean field approximation to update it node both in the learning phae and in the intance reolution phae. The architecture of a Hopfield neural network offer the poibility of a natural mapping of the N problem onto it: every element of the econdary tructure ubmatrix i mapped onto a network' node. The

4 econdary tructure i thu ditributed on the network, with a node for every pair of nucleotide. The network ha M=N(N)/ node, where N i the number of nucleotide in the tn molecule. The node can take value in [0,] and each of them repreent the probability that the correponding nucleotide are paired in the econdary tructure. More preciely, an activation value cloe to mean that the correponding pair i attributed to a tem, wherea a value cloe to 0 mean that the correponding nucleotide are not paired in the econdary tructure. The node updating rule i the one given in equation (). The econdary tructure i incrementally build a the network evolve from tate to tate, until a table configuration i reached. The network doe not contain hidden node, all it node are both input and output node, ince the olution mut be read on all the ubmatrix forming the network. The compatibility of nucleotide pair to form a tem i implemented with excitatory connection. onflict, that i violation to biological contraint, between ubtructure are implemented with inhibitory connection between the node of network which repreent the ubtructure and with threhold. In thi application the minimum value of E mut correpond to the econdary tructure that doe not violate the biological contraint and i biochemically energetically optimal. 3. The Network Energy Function The network' energy function mut impoe the biological contraint and promote the biological goal, that i favor tem, the molecule tabilizing element. We ue a network in which only the upper right triangular matrix i repreented and mut therefore reformulate ome contraint expreed for the whole matrix in term of thi repreentation Fig. 3. From a matrix to the upper right triangular ubmatrix. Let u uppoe that bae and 4 are complementary and let u mark the matrix element (,4) with the ymbol. eferring to figure 3, note that ome element of row and column 4 (in gray in the figure) are outide of the triangular ubmatrix, but have their correponding element in it, that i in the element ymmetric to them with repect to the main diagonal. row (in fig. 3 repreented by ) of the whole matrix i mapped onto two egment, a vertical one and a horizontal one, in the ubmatrix. nalogouly for column (in fig. 3 repreented by ). We will call the et of ubmatrix element correponding to row i (column j) "row" i ("column" j). For each contraint and goal, which will be implemented on the network' weight and threhold, let u introduce the term in the energy function E which will enforce it (in the ummation, and vary repectively on the network' row and column). Let u begin with the contraint. ) Pair cannot overlap: if P contain (i,j), then it cannot contain: a) (i,k) if k j, that i there can be at mot one active element in a "row" a j j a j b) or (k,j) if k i, that i there can be at mot one active element in a "column" b j j b j ) bai cannot pair to itelf: for all i, (i,i) cannot be in P. The element (i,i) (repreented with ) are not repreented in the network. 3) Knot are not allowed: if h<i<j<k, then P cannot contain both (h,j) and (i,k): c c xy x= y= + N x= + y= + The above term are zero if and only if the correponding contraint i not violated, and they are greater than 0 otherwie. They are the yntactical term of the problem a oppoed to following term, which are to the term involved with the bio-chemitry of the phenomenon, that i with minimizing the free energy. 4) Bae pair according to the extended Waton and rick model and harp loop are not allowed: if P contain (i,j), then i and j are at leat 4 bae apart: + ϕ(, ) where ϕ(,) i a function of the nucleotide in poition and, of the ditance between the two nucleotide in the molecule' equence, and of whether the two nucleotide belong to a conitent candidate tem. tem i aid to be conitent if it binding energy i equal or greater than 5 or 6. The j j xy

5 function ϕ(,) i defined according to the following rule: a) high value i aigned to ϕ(,) when the nucleotide in poition and are not complementary, when the two nucleotide are not eparated by at leat 3 nucleotide, and when the two nucleotide are complementary, but do not belong to a conitent candidate tem. In order to compute the binding energy, a binding value i aigned to every pair in a tem, thi value being for pair -U, U-, -, or -, and for pair -U or U-. The binding energy of a tem i the um of the binding energie of it pair. We et 6 a the minimum energy value for a tem to be conitent. b) n intermediate value, but greater than 0, i aigned to ϕ(,) when the two nucleotide are a and a U. We are left now with the term for the goal to introduce: 5) tem, which contribute negatively to the tructure' free energy, "emerge" along the diagonal (i,j), (i+,j),, (i+n,jn) perpendicular to the main diagonal. To favor their forming, term are needed whoe contribute to the network' energy be the more negative the more adjacent node along uch diagonal are active. We conider neighbour until ditance 3 and, taking into account that ummation are over the ubmatrix, obtain: f g h, + f g h +,, + +, 3, , 3 The energy function implicitly define the connection weight and the threhold. Due to the regularity of the problem, the network' architecture i defined by few "global" parameter, a, b, c, f, g, h. From thee the network' connection weight can be eaily derived. The firt three term define inhibitory (negative) connection, which enforce the biological contraint, with weight a, b, and c, repectively, between element of the ame row, of the ame column, or of a knot. The fourth term et the node' threhold. The fifth term yield excitatory (poitive) connection that favor the forming of tem, with weight f, g, and h between element on the diagonal perpendicular to the main one. for now it i not poible to derive analytically from the table precie value for the global parameter. It i therefore neceary to learn them from example. well trained network, on the other hand, could ugget more accurate value for the ubtructure' free energie than thoe etimated thu far. 4 Experimental eult 4. Node Updating The network' node updating algorithm i run for n iteration. During each iteration, N(N)/ randomly choen node are updated according to equation (). If equilibrium i reached in le than n iteration, the algorithm halt. tability i checked by comparing the lat iteration. Both in the learning and in the tructure prediction phae (uually called generalization), n i et to 00, where thi value wa determined experimentally. The imulated annealing chedule i the following: initial temperature T init =000, then the temperature i varied linearly with the number of iteration by T=(T)/(n). 4. Learning Phae The network learned the external ditribution from a training et of 60 tn of the pecie eubacteria whoe econdary tructure i known, were taken from data bank ftp://ftp.embl-heidelberg.de/pub/databae/trna. Weight were initialized with random poitive and negative value, repectively for excitatory and inhibitory connection. Threhold for pair -U and U- were et to 70. The network wa trained by iterating on the training et until the it performance could not be further improved. raph how how the predictive accuracy increaed over the learning pae. accuracy % 7% 77% 8% no. of iteration raph. Percentage of example learned a a function of the number of iteration on the training et. More preciely, the percentage of example learned grew rapidly during the firt 0 iteration on the training et reaching 7%, it then improved, more lowly, to 8% with the next 3 iteration. 4.3 Tet Phae The validity of our method for predicting econdary tructure wa teted on 33 tn molecule, alo of pecie eubacteria, taken from the ame data bank. Of thee, 4 belong to bacterium E. oli. On the whole, in 79% of the cae (6/33) the ame olution a the one given in the data bank wa found. In 5% of the cae (5/33) a olution wa found which

6 atifie the biological contraint, but i not optimal, in 6% of the cae (/33) a wrong olution wa found. For E. oli of the 4 tructure were predicted correctly (about 86%). For the remaining molecule, tructure atifying the biological contraint but not optimal were predicted % 86% orrect olution 5% raph. Performance in the tet phae. We teted our network alo on the problem of earch DN for coding region and were able to locate tn coding gene within DN equence. 5 oncluion We further developed the approach propoed by teeg [0] for the prediction of N econdary tructure. We put more biological contraint into our neural network and were thu able to predict the tructure of whole tn equence, though of limited ize (75-77 bae), producing a further element in upport of the validity of thi approach. We implemented a imulation program for the network. The network wa trained with example of tn econdary tructure together with the correponding equence and it learned to propoe for unknown equence the correponding econdary tructure. The predicting ability increaed rapidly during training and reached a atifying performance. The reult can be conidered a poitive tet for the applicability of thi approach to more complex molecular tructure prediction problem, uch a protein econdary and tertiary tructure. firt goal could be that of deigning a network on a till more accurate biological model in order to tudy molecule of greater ize. 4% olution atifying the contraint but not optimal eneral reult eult for E. oli eference: [] ckley D.H., Hinton.E., and ejnowki T.J., learning algorithm for Boltzmann machine, ognitive cience, Vol. 9, 985, pp [] ttimonelli M., Peole., Quagliariello E., and accone., Principi di bioinformatica, nocchi, Napoli, 997. [3] Hopfield J.J., Neural network and phyical 6% Wrong olution ytem with emergent collective computational abilitie, Proc. Natl. cad. ci., U, Vol. 79, 98, pp [4] Hopfield J.J. and Tank D.W., Neural computation of deciion in optimization problem, Biological ybernetic Vol. 5, 985, pp [5]Maizel J.V.J. and Lenk.P., Enhanced graphic matrix analyi of nucleic acid and protein equence, Proc. Natl. cad. ci. U, Vol. 78, 98, pp [6] Papanicolau., ouy M., Ninio J., n energy model that predict the correct folding of both the tn and the 5 N molecule, Nucleic cid eearch, Vol., 984, pp [7] Peteron. and nderon J.., mean field theory learning algorithm for neural network, omplex ytem Vol., 987, pp [8] Pipa J.M. and McMahon J.E., Method for predicting N econdary tructure, Proc. Nat. cad. ci. U, Vol. 7, 975, pp [9] aler W., lobin mn equence: analyi of bae pairing and evolutionary implication, old pring Harbor ymp., Quant. Biol., Vol. 4, 977, pp [0] teeg E.W., Neural network, adaptive optimization, and N econdary tructure prediction, in Hunter L., rtificial intelligence and molecular biology, I Pre / The MIT Pre, Menlo Park alifornia U, ch. 3, 993. [] tudnicka.m., ahn.m., umming I.W., and aler Winton., omputer method for predicting the econdary tructure of ingle-tranded N, Nucleic cid eearch, Vol. 5, 978, pp [] Tinoco I., Uhlenbeck O.., and Levine M.D., Etimation of econdary tructure in ribonucleic acid, Nature, Vol. 30, 97, pp [3] Waton J. D. and rick F.H.., Molecular tructure of Nucleic cid. tructure for Deoxyriboe Nucleic cid, Nature (London), 7, 953, pp [4] Zuker M., On finding all uboptimal folding of an N molecule, cience, Vol. 44, 989, pp [5] Zuker M. and tiegler P., Optimal computer folding of large N equence uing thermo-dynamic and auxiliary information, Nucleic cid eearch, Vol. 9, 98, pp