Analysis of Group Coding of Multiple Amino Acids in Artificial Neural Network Applied to the Prediction of Protein Secondary Structure

Transcription

1 Analysis of Grou Coding of Multile Amino Acids in Artificial Neural Networ Alied to the Prediction of Protein Secondary Structure Zhu Hong-ie 1, Dai Bin 2, Zhang Ya-feng 1, Bao Jia-li 3,* 1 College of Life Science, 2 Deartment of Mathematics, 3 College of Medicine, Zheiang University, Hangzhou, , China, * To whom corresondence should be addressed. Tel: ; baol@zu.edu.cn ABSTRACT In this aer, a new method involving grou coding is introduced into Artificial Neural Networ in the rediction of the secondary structure of the roteins. This method, to a larger degree, tae advantage of information of amino acids grous which ossibly lays a significant role in determining the secondary structure of the articular osition. Exeriments are conducted to test the efficiency of this method. The result shows that the rediction accuracy is significantly imroved, comared to the former method using single residue coding. We further discuss the mechanism of grou coding in the model and discuss its further imrovement in secondary rediction and other significant fields on structure and function rediction of bio-macromolecule based on sequence analysis. KEY WORDS: grou coding; Artificial Neural Networ (ANN); secondary structure rediction INTRODUCTION Since Qian and Senowshi [1] successfully alied this imlement to the field of secondary structure rediction, imrovement of the basic method mushrooms, including four resects: (1) Coding mode, rofile coding mode can incororate evolution information into the model, which imroves the rediction accuracy. [2] (2) Structure of the model, several identical ANN are trained on the same data set, then the rediction results are combined, which can bring about higher accuracy [3]. Additionally, it structurally imroves the connecting weights-sharing of model erformance, so revents the over-fitting roblem effectively [5]. (3) Otimization of the training rocess, dynamic factors, dynamic adustment of train rate is used to accelerate the seed of convergence of the training rocess. (4) Integration of other algorithms to Neural Networ. Genetic Algorithm [4] and Bayesian Model have been integrated to ANN to imrove the rediction accuracy. Most of these methods merely consider the rotein molecule as discrete symbol sequence. Contiguous residues in sequences are studied as a whole unit, which taes fully advantage of information hiding in amino acids grou. This method is called grou coding. Grou coding exresses couling effect of olyetide, which lays a significant role in determining the secondary structure of the articular osition. In this aer, couling effect of amino acids grous are taen into analysis in the ANN model; different inds of grou coding is comared, and efficiency of this method is tested. The result shows that the rediction accuracy is significantly imroved. In addition, the mechanism of the grou coding in the model is exlained; its further imrovement as well as other significant fields on structure and function rediction of biomacromolecule based on sequence analysis is discussed. MATERIAL AND METHODS Data The rotein data used in the study were obtained from Broohaven Protein Databan (PDB), and the data sets, within which the redundant sequences were excluded, included 72 roteins which belong to the family of rotein inase-lie (PK-lie) according to the classification of the Structural Classification of Proteins (SCOP) database. Total amount of residuals is while the number of

2 Helix, Sheet and Coil was resectively 10311(41.5%), 3638(14.7%) and 10876(43.8%). Artificial Neural Networ Model Predicting secondary structure of rotein by artificial neural networ contains two rocesses training and redicting. In the training rocess the structure of ANN model is firstly determined, and the arameters are then modified using training set, finally the ANN model constructed. The structure of the model is shown in fig.1, which is comosed by inut layer I, hidden layer J and outut layer K. There are 17 units in the inut layer, 40 in hidden layer whose inut value is denoted as x i and outut value as x, and 3 in outut layer whose outut value is x. The connecting weight from inut unit i to hidden unit is denoted as ω i, and that from hidden unit and outut unit is ω. Helix Sheet Coil Outut Vector K Outut Layer J Hidden Layer I Inut Layer Fig.1 toologic architecture of neural networ Inut window The inut layer of the neural networ is a window consisted of 17 units each of which is called a osition, containing one of the 20 amino acids or sace. A osition is actually an array comosed by 21 units, each secially denoting one ind of amino acids or sace. Therefore, the unit in a osition an array is given 1 if and only if it is consist with the containment of the osition or is given 0. This coding strategy is called single residue coding as is shown in figure 2. Fig.2 The structure of inut window and osition array The ninth osition of the inut window is the center one. An inut window with one residue as its center osition is inut in the ANN shown in fig.1. The three units of the outut layer stands for α Helix (H), β Sheet (E) and Coil resectively. The secondary structure whose corresonding outut unit has the largest outut value is considered as the one of the center osition, which is also called winner-tae-all. Inut coding of the osition array Grou coding codes two or more contiguous ositions in an inut window. The dietide code

3 consists of two ositions, each of which contains 21 osition units; therefore there are 21 2 coding tyes. For each tye, only one unit is 1, and the rest are 0. The trietide code consists of three ositions, each of which contains 21 osition units, therefore there are 21 3 coding tyes. For each tye, only one unit is 1, and the rest are 0. In a window with 17 ositions, there are 16 dietide codes and 15 trietide ones. Combination of the osition array To imrove the rediction accuracy, the toology structure of the ANN could be modified, which is achieved by combination of different inds of grou coding. As is shown in fig.1, within an inut window, there are 17 ositions for single residue coding, 16 for dietide coding and 15 for trietide coding. When single residue and dietide coding combine, for a single inut window, all units of the 17 ositions of single residue coding and 16 ositions of dietide coding construct the inut layer together. Similarly, the combination of dietide and trietide coding, single residue and trietide coding, furthermore all the three coding method could be taen into analysis in the same way, as is shown in tab.1. Table 1 Seven different combination methods of the osition array Denotation M D T MD DT MT MDT Single Bieide Trieide Cross-validation Due to the fact that results varied with the roteins selected for the testing and training sets, the cross-validation method is adoted to overcome the roblem in which the 72 rotein data were divided into 7 grous, two with 11 roteins and five with 10 ones. In the oulation of the 7 grous, each of them served as the testing set in turn, while the rest of the subgrous served as training sets. Therefore, 7 rounds of test roceeded. Afterwards the average of the 7 results was calculated to estimate the overall rediction accuracy for each exeriment [5]. Measurements of accuracy In this study, the rediction accuracy of rotein secondary structure is defined as follow: Rhelix + RSheet + RCoil Q3 = (1) N Here, R helix is the number of correctly redicted residues of α Helix, R sheet is the number of correctly redicted residues of β Sheet and R coil is the number of correctly redicted residues of Coil. N is total number of residues. In addition, the correlation coefficients of α Helix could also be calculated as: C H = ( n ) ( u o ) H H H H ( n + u )( n + o )( + u )( + o ) H H H H H H H H In the equation, H is the number of correctly redicted α Helix, n H is the number of residues that are correctly identified as something other than helix, o H is the number of nonhelical residues that are redicted as helix, and u H is the number of helical residues that are missed by the algorithm. A similar calculation could be easily alied to the correlation coefficients of β Sheet C E and Coil C C. BP Artificial Neural Networ Model Neural units Neural units are the fundamental comonents constructing the neural networ. The total inut of the unit is the sum of all the roduct of the inut x i and the connecting weightsω i : (2)

4 net = ω ixi i The outut x of the unit is determined by stimulating function: x = f (net ) (4) In this aer Sigmoid function is adoted as the stimulating function: 1 f (net ) = ( ) (5) net + θ / θ0 1 + e The arameter θ 0 mainly determines the shae of Sigmoid function withθ being the threshold. The outut of this unit x is a real number between 0 and 1. Exected function In the training rocess of the neural networ, a set of samles are rovided to be trained, in which the exected outut of the th outut unit for every samle is given as t. When a grou of connecting weights are given denoted as ω i and ω,the actual outut x could be calculated. Usually, exected outut t is not equal to real outut y, with error: E = 1 2 ( t x ) (6) 2 (3) For the whole set of the samles, its error is: 1 E = ( t x ) 2 2 (7) Increase of the connecting weights The increase of the connecting weights is calculated by bac-forward roagation algorithm (BP algorithm). When a grou of connecting weights ω i and ω are initially given, the networ estimates the difference between the actual outut y and the exected outut t based on the exected function. If the difference does not reach the exected function E 0, there would be the changes Δω i and Δω to all the connecting weights ω i and ω. The rocess will be reeated until E reaches the exected E 0. The training rocess coordinates the changes of connecting weights Δω i 和 Δω according to the rule that E is decreasing fastest. Moreover BP algorithm firstly calculates the changes of the inut layer Δω i, and then the changes of the outut layer Δω i according to gradient decent method in order to minimize the variation of the whole training set. Changes of the connecting weights in outut layer According to gradient decent training method, the changes of connecting weights in outut layer K for every samle are: Δω = η E = η in which η is training efficiency, from (3): = From (6) and (4): ω x = x (9) = δ = ( t x ) = (t x ) f (net ) (10) (8)

5 Consequently, tae (9) and (10) to (8),changes of the connecting weights of the unit in outut layer K are: Δω = η (t x ) f (net )x (11) Changes of connecting weights in hidden layer Changes of connecting weights in hidden layer J are: Δω i = η E i = η in which η is training efficiency, from (3): i i (12) = ω i xi = x i (13) i i From (6) and (4): = = f (net ) = f (net ) (14) According to (14): = ω x = ω (15) Tae (15) and (10) to (14), then (14) and (13) to (12), the changes of the connecting weights for a unit in hidden J are: Δω i = ηδ ω f (net ) x i (16) As is shown by (16), BP algorithm first calculates Δω in the outut layer, obtainingδ, and then Δω i. This algorithm achieves the minimization of the networ s error through gradient decent, imlementing training rocess according to the nown outut results of the networ. By adusting free arameters such as the number of hidden units, the number of training samles, the degree of homology between the training and testing sets, the length of inut window of residues, the algorithm searches the error sace and tries to find the local minimum, which yields the best classification for the training samles. RESULTS From the outcome shown in Tab.2, it is reasonable to draw the following conclusions: (1) When dietide (D) or trietide (T) serve as the inut unit for the model, the rediction accuracy is 9% and 11.1% higher than that of the model with single residue (M) as inut unit resectively, the coefficients of the three secondary structures are imroved too, which shows that grou coding is effective for imroving rediction accuracy and coefficient, and the trietide model is better then the dietide model. (2) The model with single residue and dietide (MD) as inut unit gives rediction accuracy 8.7% lower than that with dietide (D) as inut unit. As for the model with trietide (T) as inut unit, when single residue is incororated (MT) as inut unit, the corresonding rediction accuracy declines 3.9%. Similarly, coefficients also decline. Considered from model structure, when single residue is incororated as inut unit, it functions comaratively with dietide and trietide, therefore, affects the outut and rediction results. What deserves mentioning is that the decline of accuracy and coefficients is most severe for the model with dietide as inut unit.

6 (3) When dietide and trietide (DT) serve as inut unit simultaneously, the corresonding rediction accuracy culminates as high as 64.3%, meanwhile, coefficients also have maximums. However, after single residue unit is incororated, rediction accuracy declines 6.2%, and the corresonding coefficients decline corresondently, which shows that information rovided by dietide unit and trietide unit is comlementary to some extent, while information rovided by single residue contradicts with that of the former two sometimes. Table 2 The Prediction Accuracy of Seven Models M D T MD MT DT MDT 52.2% 61.2% 63.3% 52.5% 59.4% 64.3% 58.1% C H C S C C *Note: The numbers in the rows of Helix, Sheet and coil reresent corresonding correlation coefficients. DISCUSSION change of connecting weight 20 residues of are selected from the amino acid residues of the 72 roteins randomly. The secondary structure corresonding to the 20 residues is: KHHKKKHKHHKKHHKEHHKK (The symbol H stands for αhelix; K stands for all the other inds of secondary structure excet for helix). Helix and other inds of structure aear alternately for ten times. The change of the 40 weights belonging to the unit corresonding to helix in the outut layer between the situation after the first training rocess and that after the last training rocess. The figure informs that excet for the 35 th weights which exhibits oosite feature, the others commonly tends to ee the same feature as that formed in the first few iterations, In other words, the initial larger weights always remain larger than initial smaller ones, what is more, the difference between weights are magnified. It could therefore be concluded that the attern of weights in outut layer forms in the initial stages of training rocess and in the later course; the attern would not change dramatically. This situation would result in that weight attern of hidden layer exerts maor function in the erformance of the whole model. Because the introduction of grou coding mainly changes the toology structure of inut and hidden layers, the outut layer would not diminish its effect. The comare aforementioned after training from the first osition. It is evident that the oosition of atterns of the two units has aeared at this time which means that if a weight in one unit is resectively larger, its counterart in the other unit tends to be smaller and vice versa. This trend is more significant after the training from the 20 ositions. The secondary structure corresonding to the first osition is coil. After the first training from the first osition, the weight attern of outut layer corresonding to helix is basically reversed to the outut attern of hidden layer. By contrast, the situation changed dramatically when the target of training is changed to Helix in the training from the second osition. In other words, the two aforementioned atterns are basically consistent with each other. The same analysis was conducted to the following 18 training rocesses, and the result is similar with aforementioned ones. The training rocess thus could be inferred from the above analyze. For every single unit in outut layer, when the ind of structure it stands for is consistent with training target (this ind of unit is termed as this first ind of units in outut layer while the other two are termed as the second ind of units.), its connecting weight attern would be consistent with outut attern of hidden layer which means larger outut would be multilied by larger connecting weight. This attern of weighted sum

7 would contribute to larger outut of this articular unit in outut layer. On the contrary, for the other two units, the situation is reverse. According to the winner-tae-all rincile, the articular ind of secondary structure that the first ind of unit in outut layer stands for would be the result of final rediction. Once the connecting weight attern of outut layer is formed (the attern is originally formed according to the randomization of connecting weight in hidden layer), this attern will remain unchanged basically. Furthermore,it will shae the connecting weight attern in hidden layer in the following training rocess and further shae its outut attern. Hidden units are the basic unit of this rocess, which means all the connecting weights in one unit will increase or decrease in the same manner. The final result is reresented by the significant value difference between the oututs of hidden units. This is similar to a ind of encoding strategy. The algorithm of training rocess could exlain the aforementioned extraolation. Firstly, based on eq.15 and eq.16, for connecting weights belonging to the same hidden unit, the differences among their change Δω i only exist in the inut x i. However, the inut to hidden layer in this model is only 0 or 1. The inut unit whose value is 0 and their corresonding connecting weight in hidden layer is not taen into account. Therefore, Δω i for other connecting weight belonging to the same hidden unit would be identical which demonstrates that all the connecting weight in one unit will change in the same manner. Secondly, for a hidden unit, besides x i mentioned above, the change Δω i is determined by two searated arts f (net ) and δ ω. Since connecting weights are initialized randomly, the difference of t f (net ) for all the units could be considered as no significant. The second art is δ ( ω). When being initialized, it is determined by connecting weight and other values of outut layer. In this articular model, this art could be further divided into three subarts, corresonding to the three units in outut layer. As to the first ind of unit in outut layer, δ is ositive. For larger connecting weights, the δ ω they contribute would be larger. On the other hand, their counterarts in the other two units are resectively smaller. Nevertheless, since the δ of these two units is negative, the δ ω they contribute would be still resectively larger than they do to other hidden units. This interesting strategy would finally lead to the common increase of the connecting weights belonging to this articular hidden unit, which further contribute to its larger outut. As to the former art of f (net ) in the same training rocess, since net continually increases, it would gradually decrease in the later iterations and finally converges. To the oosite, if the corresonding connecting weight of the hidden unit U in the first ind of unit in outut layer is smaller, its connecting weights would be continually decreased which leads to its final smaller outut. With this strategy, the connecting weight attern of outut layer shaes the connecting weight attern in hidden layer and further shae its outut attern. In the other resect, as is shown in eq.11, for a unit in outut layer, the differences of the change Δω among its connecting weights only exist in x which is the outut of hidden layer. According to the connecting weight attern of outut layer, the connecting weights of the first ind of units could be categorized into two sets: the set of larger ones and the set of smaller ones. The outut of units in hidden layer corresonding to the first set of connecting weights would be significantly increased in the first modification rocess, and as a matter of which, they would also significantly increase. On the other hand, since the outut of units in hidden layer corresonding to the second set of connecting

8 weights would be decreased in the first modification rocess, though they would also be increased, yet the amlitude would be much smaller than the first set. Similarly, the connecting weights of the second ind of units could be categorized into two sets: the set of larger ones and the set of smaller ones. The outut of units in hidden layer corresonding to the first set of connecting weights would be decreased in the first modification rocess, though they would be decreased, yet the amlitude would not be exected large. The outut of units in hidden layer corresonding to the second set of connecting weights would be significantly increased, and as a matter of which, they would be significantly decreased. With this strategy, outut layer maintains and further enhance its connecting weight attern. The roerty of indication and secification of amino acid grou In the training set, if some amino acids grou is corresondent to a articular secondary structure with relatively higher robability, the air of the amino acids grou and the secondary structure, is called a high frequency. For examle, if an amino acid grou AAA always corresonds to α Helix, the air of AAA and H ( AAA, H) is called a high frequency and the airs ( AAA, C) and ( AAA, S) is called low frequency. If some amino acids grou in training set is never corresonded to a secondary structure, the air of them is called a null frequency. The ercentage of high frequency in the roteins being redicted is highly corresondent to the level of the rediction accuracy which reflecting the close relationshi between secific amino acid grou and secondary structure. This articular roerty of amino acid grou is called the roerty of indication. Furthermore, null frequencies hardly aear in the roteins being redicted, which means most of the airs of amino acids grou and secondary structure being redicted have aeared in the training rocess. This condition as well as the roerty of indication is combined and commonly called the roerty of secification of amino acid grou. The roerty of secification of amino acids grou can be exlained by the function of every amino acids hysical and chemical roerties to determine the secondary structure, including the size and olarity of R grous, hydrohobic and hydrohilic ability. Different residues have different tendencies to form αhelix and βsheet, therefore, a articular etide is rone to form a certain secondary structure while others are liely to form a different one [6]. For illustrations, olyisoleucine whose R grous are large enough to form satial barrier can not formαhelix. Similarly, Proline has a distinctive cyclic structure. As a result, when roline aears in olyetide, αhelix will be broe [7]. In addition, inβturn which is classified as coil here often aear olar residues such as asaragines, asartate, glutamine, serine and residues that affect the formation of αhelix andβsheet such as roline and glycine [6]. Integrating the above three arts, grou coding has certain function in imroving the rediction accuracy. In the rocess of training or rediction, because of the roerty of secification of amino acid grous, contiguous amino acids are more liely to aear in secific combinations and arrangements rather random manner. In terms of ANN model structure, it means that merely a small ortion of connecting weights are used and modified. The combinations of these connecting weights are more liely to reflect the atterns of the connecting weights of hidden layer, which can imrove the rediction accuracy remarably when combined with the roerty of indication of amino acid grous. If the connecting weight attern memorized in the training rocess can not reflect the actual structure of the rotein being redicted, the rediction ability of grou coding will be undermined, which means that the function of grou coding deends on homology of data, for examle the rediction of structure of function site. Because these ositions exerience more evolution ressure,

9 their sequence and their structure tend to be conserved. As for rediction of common structure, it is helful to do some homology analysis before redict secondary structure using grou coding. Besides, when several identical networs redict a single sequence, the rediction accuracy can be imroved effectively. Due to the fact that homology and training order of the samles exert rofound influence on the final rediction accuracy,a different multi-model networs method is used, in which the models are trained on different data set, or they are trained in different orders of samles within the same data set. The rediction results are accomlished resectively by all the models, and are combined to yield the final result. ACKNOWLEDGEMENTS The authors gratefully acnowledge the financial suort of Student Research Training Program of Zheiang University of China. REFERENCE [1] Ning Qian, Terrence J. Senowsi, Predicting the Secondary Structure of Globular Proteins Using Neural Networ Models. J. Mol. Biol.heiang, 202, [2] Ruan Xiao-gang, Sun Hai-un. Research on Encode Influencing Protein Secondary Structure Prediction. Journal of Beiing University of Technology. 31(3), , 2005 [3] Hanxi ZHU, Iuo YOSHIHARA, Kunihito YAMAMORI. Prediction of Protein Secondary Structure by Multi Model Neural Networs, Proc International Joint Conference on Neural Networs (IJCNN 02), , 2002 [4] A J F Van Rooi, RP Johnson, LC Jain, Neural Networ Training Using Genetic Algorithm, World Scientific, 1996 [5] Yan Hua-Jun, Fu Yan, Zhang Yi, Li Yi-Chao. Neural Networ Method for Protein Secondary Structure Prediction, Comuter science, 30(2), 48-52, 2003 [6] Yan Long-fei, Sun Zhi-rong. Moleculer Structure of Protein. Tsinghua University Press [7] Wang Jing-yan, Zhu Sheng-geng, Xu Chang-fa. Biochemistry. Higher Education Press. 2002