Classification of DNA Sequences by a MLP and SVM Network

Transcription

1 Classification of DNA Sequences by a MLP and SVM Network TERJE KRISTENSEN FABIEN GUILLAUME Institute of Computer Science, Pattern solutions ltd. Nygårdsgaten 112, Bergen University College, Nygårdgaten 112 N-5020, Bergen, Norway N-5020, Bergen, Norway tkr@hib.no fabien.guillaume@student.hib.no Abstract - In this paper we show how a Multi-Layered Perceptron (MLP) neural network and a Support Vector Machine (SVM) are used to classify between eukaryotic and prokaryotic cells. The classification is based on their DNAsequences which are obtained from different databases available on Internet. The sequences are first pre-processed using a sliding window technique to obtain sub-sequence frequencies, and then normalised to make them comparable. Keywords: MLP, SVM, prokaryotic and eukaryotic DNAsequences, Java, sliding window I. INTRODUCTION One of the main research goals of molecular biology has been to determine a complete genetic description of any organism. In the Human Genome Project [4] the goal was to decipher the exact sequence of about 3 billion nucleotides in the 46 human chromosomes. An important part of the genome project is the computational processing of data [2]. The data first have to be organised into databases, and then analysed to see what information they contain. Since the birth of the Human Genome Project, sequence analysis as a computational method, has been used to infer biological information from the sequence data. The classical approach to analysing sequences is by sequence matching using either single or multiple alignment techniques [7,8]. With these techniques one seeks to determine whether sequences are significantly similar or not. Another approach is to use theories from neural computing or statistical learning theory to detect genetic information on the DNA sequences. Neural networks have been applied to various tasks such as automatic hyphenation of natural languages [11,12], edge detection [14], recognition of hand written Zip code and DNA sequence recognition [1,13]. A neural network trained by a Backpropagation algorithm may learn to categorise between different types of bacteria cells related to the structure of their DNA-sequence. Such a method is based on pattern recognition analysis, and is built on the assumption that some underlying characteristics of the DNA-sequence can be used to identify its bacteria type. Other neural network paradigms than a MLP network may also be used to analyse DNA sequences [18]. In this paper, however, we will focus on how to use both a MLP and a SVM network to distinguish between eukaryotic and prokaryrotic DNA-sequences on basis of their nucleotide frequency structure [8,9]. Cells can be divided into two major groups, prokaryotic and eukaryotic cells. All prokaryotic cells are uni-cellular organisms and consist mostly of bacteria. The genome of a prokaryotic cell consists of one double helix DNA strand, floating freely in the cell. This double helix strand is often circular. The genome of the bacterium E.Coli, for instance, consists of a circular strand of five and a half million bases. A nucleotide sequence can be viewed as a language based on an alphabet of four letters: A, G, C and T where the number of As is the same as the number of Ts, and the number of Cs is the same as the number of Gs. However, the relation of A(T) and G(C) can vary tremendously, and depends on the actual species that are studied. This fact can, for instance, be used in environmental research, where oil on the sea surface may contain many different types of species that can be identified on basis of their DNA sequence structure. Most eukaryotes are multicellular, but some are unicellular. The main difference between prokaryotic cells and eukaryotic cells is that eukaryotic cells contain a nucleus that is surrounded by a membrane. Prokaryotic cells do not have such a nucleus. In such cells the frequency distribution of pairs of nucleotides are different from those in prokaryotic cells [3]. II. SVM NETWORK Support Vector Machines (SVM) is a computationally efficient learning technique that is now being widely used in pattern recognition and classification problems [2]. This approach has been derived from some of the ideas of the statistical learning theory regarding controlling the generalization abilities of a learning machine [19, 20] In such a regime the machine learns an optimum hyper plane that classifies the given pattern. By use of kernel functions, the input feature space, by applications of a nonlinear function, can be transformed into a higher dimensional space where the optimum hyper plane may be learned. This gives a flexibility of using one of many learning models by changing the kernel functions. A. The SVM Classifier The basic idea of an SVM classifier is illustrated in Fig. 1. In the figure it is shown, in the simplest case, the data vectors (marked by X s and O s) may be separated by a hyper plane. In such a case, there may exist many separating hyper planes. Among them, the SVM classifier seeks the separating

2 hyper plane that produces the largest margin separation. In the more general case, in which the data points are not linearly separable in the input space, a non-linear transformation is used to map the data vectors into a highdimensional space (called feature space) prior to applying the linear maximum margin classifier. To avoid the potential pitfall of over-fitting in this higher dimensional space, SVM uses a kernel function in which the non-linear mapping is implicitly embedded. A function qualifies as a kernel function if it satisfies the Mercer's condition [20]. By use of a kernel function, the discriminant function in a SVM classifier has the following form N f(x) = i y i K(x i, x) + 0 (1) i 1 where K(-,-) is the kernel function, x i are the support vectors determined from the training data, y i is the class indicator e.g. +1 and -1 for a two class problem associated with each x, N is the number of supporting vectors determined during training and i are the Lagrangian multipliers function (RBF) kernel is given by K(x,y) = exp ( - x-y /2 2 ) (3) where > 0 is a constant that defines the kernel width. Both of these kernels satisfy the Mercer condition mentioned earlier. III. THE MLP NETWORK A Multi-Layered Perceptron (MLP) network generally contains three or more layers of processing units [10]. Fig. 1. A Support Vector Machine classification defined by a linear hyper plane that maximizes the separating margins between the classes. Support vectors are elements of the training set that lie either exactly on or inside the decision boundaries of the classifier. In essence, they consist of those training examples that are most difficult to classify. The SVM classifier uses these borderline examples to define the decision boundary between the two classes. B. The Kernel Functions The kernel function plays a central role of implicitly mapping the input vectors into a high-dimensional feature space, in which linear separability is achieved. The most commonly used kernel functions are the polynomial kernel given by: K(x,y) = (x T y + 1) p (2) where p > 0 is a constant. The Gaussian radial basis Fig. 2. A MLP network consisting of three layers of nodes.. Fig.2 shows the topology of a network containing three layers. The bottom layer constitutes the input layer which obtains its input from the environment. The middle layer contains hidden units. The hidden layer has de ability to solve non-linear separable problems. The top layer constitutes the output layer which gives the produced results from the network. Every layer is fully connected with the nearest layer, i.e. every unit in the first layer is connected to every unit in the second layer and every unit in the second layer is connected to every unit in the output layer. This is not a requirement, but most MLP networks are constructed like this. A. The MLP Classifier In the learning phase we present patterns to the network, and the weights are adjusted so that the produced outputs from the output nodes are equal to the target. In fact, we want the network to find a single set of weights that will satisfy all the (input, output) pairs presented to it. Neurons in one layer receive signals from neurons in the layer directly below and send signals to neurons in the layer directly above. Connections between neurons in the same layer are not allowed. Except for the input layer nodes, the network input to each node is the weighted sum of outputs of the nodes in the previous layer. Each node is activated in accordance with the input to the node and the activation of the node. The difference between

3 the calculated output and the target output is then calculated. The weights between the output layer, hidden layers and the input layer are adjusted by using this error function. The output of a node is calculated using the sigmoid function f(x) given by: f(x) = 1/(1 + e -x ) (4) The weighted sum S j is inserted into the sigmoid function, and the result is the output value from unit j: f(s j ) = 1/(1 + e -Sj ) (5) where S j = w ji a i (6) The error value of an output unit j can be calculated by: j = (t j - a j ) f (S j ) (7) where t j and a j are the target and output value of unit j, respectively. f is the derivative of the sigmoid function f, and S j the weighted input sum. The derivative of the sigmoid function is given by: f (S j ) = f (S j ) (1- f (S j )) (8) We notice that the derivative is expressed by the function itself which makes the error more rapidly to calculate. For a hidden node, the error value is calculated as: j = k w kj (f (S j )) (9) From the formula we see that the error of a processing unit in the hidden layer is computed by the upper layer. Finally, the weights can be adjusted by: w ji = j a i (10) where is the learning rate. The new weights are then given by: w ji (t+1) = w ji (t) + w ji (t) (11) where t represents a processing step. Usually, the momentum parameter β is introduced in the MLP network. It has been shown that the use of this additional parameter can be helpful in speeding up the convergence and avoiding local minima. Equation (11) can now be written as: w ji (t+1) = w ji (t) + j a i + β w ji (t) (12) In general, the outputs {a pi } (p is the current pattern) of the nodes in the output layer will not be the same as the target or desired values {t pi }. The system error or the cost function for the network is defined by: 1 E N 1 2 p (13) where N is the total number of patterns. A gradient search should be based on minimisation of the expression in equation (13). The weight updating rule then becomes as given in equation (12). For a more thorough discussion see references [10,15]. IV. i t pi a pi DNA DATAMINING Statistical analysis of several DNA sequences has shown that the distribution of nucleotides is far from random [17]. Some dinucleotide combinations in prokaryotic DNA sequences are more dominating than in eukaryotic cells. We will anticipate that this simple difference in data occurrence might be sufficient to allow species identification. We may then train, for instance, a MLP network, to use the differences in the nucleotide distribution to discriminate between eukaryotic and prokaryotic cells. We assume therefore that the identification of the DNA sequence is based solely on the frequency of nucleotide sub-sequences. A sliding window is used to count the number of nucleotide sub-sequences of the DNA sequence. In general, the size of the window may vary, from one base wide to a user defined number w. By choosing a window of length one, we simply count the number of the different bases of the DNA sequence. The result will be four different frequencies, one for each base. A window of length two will result in sixteen different ordered sub-sequences. The frequency of each sub-sequence is computed by counting the occurrence of each nucleotide pair in the DNA sequence. The number of triplets or codon units of the DNA sequence, may be estimated by using a window of three bases wide. This results in 4 3 or 64 ordered triplets. This is maybe the most relevant sub-sequence to study because the codon itself has important meanings in the DNA sequence. In general, a window of w bases results in 4 w sub-sequences of length w to be counted. For a sliding window of length two the frequency of sub-sequence AA is denoted as ƒ AA, for AC as ƒ AC and so on. These numbers are collected in a vector F n, where n denotes the number of DNA sub-sequences. For a sliding window of size two, n is equal to 16. The counting of the different nucleotide pairs is illustrated in Fig.3. In the figure the counting of the nucleotide pair AC is shown. After counting the pair AC, the window is moved one letter to the right to cover the next nucleotide pair. This is done until the end of the DNA sequence. 2 where β is the momentum and w ji (t) is the weight change from the previous processing step.

4 ACATGATGCTA... ACATGATGCTA... ACATGATGCTA... ACATGATGCTA... Fig. 3. A sliding window of two letters is used to count the occurrence of each ordered nucleotide pair. The DNA sequences obtained on the Internet have different sequence length. The frequency of the different nucleotide pairs have to be normalised to compare and present them to the MLP or SVM network. The normalisation condition of the frequency vector F n is given by equation 14. S F F n (14) n n Here F n means the Euclidean norm of the frequency vector. This non-linear transformation conserves the direction of the vector and enhances the differences among the input vectors. The geometric interpretation of the transformation is that the vector F n is moved onto the hyper unit sphere. V. FEATURE ANALYSIS Before the training started, a feature analysis of the training data was carried out. The training set consists of fifteen DNA sequences from each cell class. S n vectors from the two classes were aligned to see if one could find similarities between them. The mean relative frequency of corresponding sub-sequences in each class were computed and plotted in a graph. The main reason for doing this experiment was to make an analysis to see if the methodology used was adequate, and if there exist features that will make the DNA sequences different. The correlation between the mean relative frequency of eukaryotic and prokaryotic sub-sequences is given by the conventional linear correlation coefficient based on the usual covariance matrix. Matlab programs were developed for plotting the results of the comparison between eukaryotic and prokaryotic sequences in the training set, and a plot is shown in Fig. 4. The upper graph shows the correlation between the DNA sub-sequences when the size of the sliding window is two bases wide. The correlation coefficient r is then estimated to The lower graph shows a comparison between eukaryotic and prokaryotic sequences when the window size is three. The correlation coefficient r is now estimated to Fig. 4. The means of relative frequency for corresponding sub-sequences of size 2 and 3 are compared for eukaryotic and prokaryotic DNA sequences. Similar experiments were also carried out when the size of the window was greater than three. In fig. 5 we notice that the correlation coefficient drops when the window size increases. This means that by using longer sub-sequences, the feature difference between the two DNA classes is getting more substantial. The classification accuracy between the different DNA sequences should then increase. However, longer sub-sequences will tend to identify one specific DNA strand. This can be understood by letting the window size approach the length of the DNA sequence. The feature extraction should then be harder and make it more difficult for the network to generalise. A larger window will also generate more training data which in general will require more computer power and longer training times. VI. TRAINING Different DNA sequences were obtained from several DNA databases on the Internet. The experiments were performed under Window 7 on Intel Core i7-3610qm with 12GB of DDR3. A Java program has been developed to train the network using the javann (java Artificial Neural Network) software package created by the company Pattern Solutions AS in Norway [15]. In addition, the LIBSVM package created by Chih-Chung Chang and Chih-Jen Lin at the National Taiwan University has been used [21]. A

5 Graphic User Interface was created in addition to Java classes to read the EMBLM format and also export the results into Excel files. Fig. 6 shows the neural network which is used during the experiments. The EMBL format of a DNA sequence looks like the one given in Fig. 7. VII. PARAMETERS AND WINDOWS SIZE A. SVM experiments The SVM is trained using a polynomial kernel or a modified Radial Basis Function kernel K(x i,x j ), given by equation 15. K(x i,x j ) = e - xi xj, > 0 (15) Fig: 5. The correlation coefficient between eukaryotic and prokaryotic DNA sequences drops when the gliding window size increases. Procariotic Eucaryotic Output layer Hidden layer... Input p s1 p s2 p s3 p s4 p s5 p s6 p s7 p s8 p s9 p s10 p s11 p s12 p s13 p s14 p s15 p s16 Fig. 6. Classification of eukaryotic and prokaryotic DNA-sequences by a BP network. Each p xi is a relative frequency of a nucleotide pair. SQ Sequence 691 BP; 135 A; 243 C; 192 G; 121 T; 0 other; CCTGAACCCG GTGTCCCCGG GTGGGGGGTG GGGACGCCAC GGCCGAAGCA GCTAGCTCCG 60 TTCGTGATCC GGGAGCCTGG TGCCAGCGAG ACCTGGAATT TCCGGTCTGG TTGGTCTGGG 120 GCCCCGCGGA GCCAGGTTGA TACCCTCACC TCCCAACCCC AGGCCCTCGG ATGCCCAGAA 180 CCTGTAGGCC GCACCGTGGA CTTGTTCTTA ATCGAGGGGC ACTTCTACCC TAGCCGGGCC 240 CAGCCCCCGA GCAGTGCAGC CTCCCGAGTG CAGAGTGCAG CCCCTGCCCG CCCTGGCCCA 300 GCTGCCCATG TCTACCCTGC TGGATCCCAA GTAATGATGA TCCCTTCCCA GATCTCCTAC 360 CCAGCCTCCC AGGGGGCCTA CTACATCCCT GGACAGGGGC GTTCCACATA CGTTGTCCCG 420 ACACAGCAGT ACCCTGTGCA GCCAGGAGCC CCAGGCTTCT ATCCAGGTGC AAGCCCTACA 480 GAATTTGGGA CCTACGCTGG CGCCTACTAT CCAGCCCAAG GGGTGCAGCA GTTTCCCACT 540 GGCGTGGCCC CCGCCCCAGT TTTGATGAAC CAGCCACCCC AGATTGCTCC CAAGAGGGAG 600 CGTAAGACGA TCCGAATTCG AGATCCAAAC CAAGGAGGAA AGGATATCAC AGAGGAGATC 660 ATGTCTGGGG CCCGCACTGC CTCCACACCC A 691 Fig. 7. A database entry of a eukaryotic DNA sequence from the EMBL database on the Internet. The length of the DNA-sequence is 691 bases. The DNA sequence is a protein found in the human being.

6 Before training, two parameters need to be found, C and. C is the penalty parameter of the error term and is a kernel parameter. The C value used was equal 100, and was set to the inverse of the size of the number of input values. This means, for instance, that for an input file of sixteen inputs I equal 1/16 = However, these values for C and are currently not optimal. B. MLP experiments The network is trained according to the Backpropagation algorithm [10,18]. A set of S n vectors and their corresponding classification are presented to the network. During the training session the input to the network is alternately selected from the eukaryotic and prokaryotic cell class. After the training session is finished, the network is tested on a set of unknown S n vectors. The output is recorded, and the performance of the network computed. The MLP network used for window size two is shown in Fig 6. The input layer consists of 16 input nodes, one for each component of S n = (s 1, s 2,., s n ). Each p si given in Fig. 6 is a 'relative frequency' of each nucleotide pair of a DNA sequence. This is not the conventional relative frequency, because the relative frequencies do not sum up to 1. In this case the relative frequency is defined by the actual number of each nucleotide pair, divided by the square root of the total sum of pair frequency squared. C. Window sizes In Fig. 5 we notice that for sub-sequences of length two to four bases, the correlation coefficient does not vary much. This should mean that for a window of length 2, 3 or 4 bases wide, the classification performance should not differ very much. The experiments carried out may be a confirmation of this. For a window of size equal three two cases are to be considered. The reason for this is that three nucleotides correspond to a codon defined by the Genetic code. There are as many codons as there are permutations of three nucleotides, i.e. 4 3 = 64 possible codons, but only 22 of them are coding for amino acids and stop/start criteria. This also means that the Genetic code contains lots of redundancy. Table 1 shows the length of the frequency vector of different window sizes. The same input vectors were used in both the MLP and SVM network. The length of the Window size frequency vector (genetic 22 code) Table 1: The length of the frequency vector for different window sizes VIII. THE GENETIC CODE The DNA molecule contains genetic information in our cells. The genetic code is the language used by the cell to make different proteins by use of information given in the genes. A gene, which codes for a protein, is first transcribed to messenger RNA (mrna). Then the information in the DNA molecule is translated to the protein via its messenger RNA (mrna). The sequence of amino acids in the proteins is determined by of the DNA molecule. Three bases code for a certain amino acid. Such a base-triplet is called a codon. Since the DNA string consists of four different bases (A, C, G, T), there will be 4 3 = 64 different codons. Since the cell only uses 21 different amino acids to build proteins, most of the amino acids will correspond to several codons. So, there is a great deal of redundancy in this coding scheme. For instance, the codon ATG means start of a protein synthesis and will also at the same time code for the amino acid methionine. The codons TAA, TAG and TCA indicates all stop codons in the protein synthesis. A. Inclusion of the Genetic Code As explained in the section VI the representation of genetic code can be built into the neural network and the SVM experiments by use of a redefined frequency vector F 22, containing only the relative frequencies of 22 codon elements: The different codons and corresponding amino acids of the Genetic code are given in Table 2. Codons TTT,TTC TCA,TCG,CCT,CCC,CCA,CCG ACT,ACC,ACA ATG GCT,GCC,GCA, GCG TCT,TCC,TCA,TCG CCT,CCC,CCA,CCG ACT,ACC,ACA,ACG GCT,GCC,GCA,GCG TAT,TAC TAA,TAG,TGA CAT,CAC CAA,CAG AAT,AAC AAA,AAG GAT,GAC GAA,GAG TGT,TGC TGG CGT,CGC,CGA,CGG,AGA AGT,AGC GGT,GGC,GGA,GGG Amino Acid Phenylalanine Leucine Isoleucine START/Methionine Valine Serine Proline Threonine Alanine Tyrosine STOP Histidine Glutamine Asparagine Lysine Aspartic acid Flutamic acid Cysteine Tryotophan Arginine Serine Glycine Table 2: Different codons of the Genetic Code.

7 IX. EXPERIMENTS AND RESULTS By using a MLP or SVM network the classification task between the different DNA sequences might reduce the need for frequency aligning, and hence the classification time in general. In Fig. 5 we notice that for sub-sequences of length two to four bases, the correlation coefficient does not vary much. A. MLP results The training set consists of 30 DNA sequences distributed equally between eukaryotes and prokaryotes. In one experiment only the number of hidden neurons was varied. The optimal number was found to be 30 hidden nodes. In another experiment the learning rate and the momentum were varied. The training of the network is shown in Fig. 8. The number of iterations was set to The global error was then estimated to about 10 %. In general, such an error is too large. However, the experiments showed that the generalisation of the network was then optimal. More training of the network resulted in that some over-training happened. The best results were achieved for the learning rate = 0.07 and momentum = 0.1. These parameter values are very small which may indicate that the learning needs to be slow in order for the network to detect the detailed structures of the DNA sequences. WindowSize Training data Validation data (genetic code) 89,6 80,1 B. SVM results 3 86,2 84,6 2 89,6 80,7 Table 3: Results of using the MLP network Different types of kernels were selected in the experiments of the SVM network, with the Genetic code included or not. The best performance was achieved for a network with a polynomial kernel. The performance was 84.6 % for unseen DNA sequences. This looks more sensible compared to what we would expect before carrying out the experiments. Window size Kernel type Training data Validation data 4 linear polynomial RBF (genetic code) linear 86,2 76,9 3 (genetic code) polynomial 86,2 84,6 3 (genetic code) RBF 82, linear 96,6 65,4 3 polynomial 96,6 69,2 3 RBF 86,2 73,1 2 linear 89,7 84,6 2 polynomial 86,2 84,6 2 RBF 82,7 73,1 Table 4: Results of using the SVM network X. DISCUSSION Fig. 8. Training of the MLP neural network. The training time is iterations. The performance of the network has been tested on 26 unknown DNA sequences found on the Internet. The best prediction was about 85 % for a window of length three. The Genetic code was then not built into the training set. This does not seem to be what we expected, and the explanation is maybe that the MLP network needs more data to be properly trained on in this case. The results are given in Table 3. If we want an ANN or a SVM to fully automate the classification process of DNA sequences, we may get some problems to solve. In general, both the training and testing (performance) of the network will depend on the window size. By changing the window size, the topology of the neural network also must change. The number of input nodes of the network is dependent on the window size. For a window size of length w the number of input nodes is equal to 4 w. By changing the number of input nodes in a MLP network, the number of neurons of the hidden layer must be changed. However, a new network topology configuration in general, would also mean redefinition of the learning rate and momentum of the training algorithm. This will again influence the training time of the network. So, by changing the topology of the network, the different parameters should also be changed accordingly. A full automation of the categorisation of a MLP network for any size of sub-sequences, is a difficult task to solve. To be able to compare the results of the different experiment sin general, we may, for instance, plot the results in the 3D space

8 of different parameter values. We could, for instance, represent this as a point in 3D where the x-y-z axes are defined by the window size, learning rate and the number of hidden neurons, respectively. A more detailed analysis may also take into account the other parameters mentioned above. We could represent the different parameters as components of a vector in hyper space. However, the three parameters mentioned above are the most significant ones. The experiments could also be extended to use a sliding window of length greater than four, to see if the graph in Fig. 5 can be deduced from the experiments or not. The number of DNA sequences used in the training set is currently too small. The experiments definitely have to be scaled up to make the conclusion more valid. In addition, short sequences in the data set may contain too little information to be of any practical value in the analysis. Longer DNA sequences give better performance rate. Further work is required to confirm this properly. In a longer perspective, a more general solution of the classification problem of DNA sequences should be established as indicated in the discussion at the end of section V. This can be done by using, for instance, a parameter optimisation technique. In principle, such an approach may be similar to search in an n-dimensional space, where n is the number of parameters we want to optimize. One such optimization technique is Genetic algorithms that may be relevant in this case.. To automate the recognition process of DNA-sequences by a SVM network may be an easier task to carry out, since such a regime is more mathematically sound. However, the selection of the best kernel function adapted to the actual problem, may be a difficult task. XI. CONCLUSION AND FURTHER WORK A MLP and SVM network have been developed to categorise between eukaryotic and prokaryotic DNAsequences found in biological databases on the Internet. All the DNA-sequences have been represented in the EMBL format. Java programs have been developed to carry out the experiments for both networks. The experiments, so far, have shown that both a MLP and a SVM network are able to distinguish between prokaryotic and eukaryotic cells by using a simple feature extraction technique based on a gliding window. The recognition performance of unseen DNA-sequences was estimated to about 85 % for both the MLP and SVM network. An experimental approach has currently been used to determine the optimal values of the parameters used. However, in further work a separate regime based on, for instance, Particle Swarm Optimization techniques is planned to be used to find optimal parameter values of both ANN and SVM network. REFERENCES 1. Allex, C.F.,Shavlik,J.W., Blattner,F.R. Neural Network Input Representations that Produce Accurate Consensus Sequences fram DNA Fragment Assemblies. Bioinformatics 15, Burges, C.J. A Tutorial on Support Vector Machines for Pattern Recognition. Knowledge Discovery and Data Mining 2, Springer, Claverie, J Computational methods for the identification of genes in vertebrate genomic sequences. In Human Molecular Genetics, 6, 10, , Campbell,N., Reece, J. Biology. Addison Wesley chapter 28, New York, 2001, USA. 5. Collins,F., Galas, D. A new five-year plan for the U.S. Human Genome Project, Science, 262, 1993, Douzono,H., Hara, S., Noguchi,Y. An Application of Genetic Algorithm to DNA Sequencing by Oligonucleotide Hybridization. In Proceedings of IEEE International Joint Symposia on Intelligence and Systems, May Douzono,H., Hara, S., Noguchi,Y. A Design Method of DNA chips for SNP Analysis Using Self Organising Maps. In Proceedings of IEEE International Joint Conference on Neural Network, IJCNN 2001, Washington DC, 2001, USA. 8. Feng,D.F., Doolitle,R.F. Progressive sequence alignment of amino acid sequences and construction of phylogenetic trees from them. Methods in Enzymology, 266, Fickett, J.W., and Hatzigeorgiou, A.G..Eukaryotic promoter recognition. Genome Res., 1997,7, Haykin,S. Neural Networks. Prentice Hall, Kristensen,T. A Neural Network Approach to Hyphenating Norwegian. In Proceedings of IEEE International Joint Conference on Neural Networks, IJCNN 2000, Como, Italy. 12. Kristensen,T., Langmyhr,D. Two Regimes for Computer Hyphenation a Comparison. In Proceedings of IEEE International Joint Conference on Neural Networks, IJCNN 2001, Washington DC, USA. 13. Kristensen,T., Patel,R. Classification of Eukaryotic and Prokaryotic Cells by a Backpropagation Newtwork. In Proceedings of IEEE International Joint Conference on Neural Networks, IJCNN 2003, Portland, Oregon, USA. 14. Kristensen,T., Patel,R. Edge Detecting in a Lateral Inhibition Network. In Proceedings of IEEE World Congress on Computational Intelligence, WCCI 2002, Honolulu Hawai, USA. 15. Kristensen, T. javann: Java Artificial Neural Networks. Pattern solutions AS LeCun,Y., Boser,B., Denker,J.S., Henderson,R., Howard,E., Hubbard,W., Jackel,L.D. Backpropagation Applied to Handwritten Zip Code Recognition. In Neural Computation 1, Nusinov, R. Strong Preferences in Nucleotide Sequences of DNA Geometry. Journal of Molecular Evolution 20, Potamias,G., Papanikolaou,E., Hatzigeorgiou,A. Knowledge- Based TDNN Architectures for Features Recognition in DNA Sequences. In Proceedings of IEEE International Joint Conference on Neural Networks, IJCNN 2001, Washington DC, USA. 19. Vapnik, V.N. An Overview of statistical learning theory. In IEEE transactions on Neural Networks, September, Vapnik, V.N. Statistical learning theory. Wiley, New York,