Similarities Between Hidden Markov Models and Turing Machines, and Possible Applications Towards Bioinformatics

Transcription

1 Bonformatcs Fnal Proect, Fall 2000 Smlartes Between Hdden Markov Models and Turng Machnes, and Possble Applcatons Towards Bonformatcs Tyler Cheung Over the past fve or sx years, Hdden Markov Models (HMMs) have become an mportant tool for certan bonformatcal tasks, ncludng sequence and structural algnment. The advantage of an HMM s that t gves bonformatcsts, among other thngs, a rgorous mathematcal method, through probablty and combnatorcs, of algnng two or more sequences of protens and nuclec acds. However, mathematcal constructs known as "Turng Machnes" are very smlar to how HMMs are defned, and thus, we wsh to consder whether these machnes have any bonformatcal applcatons. HMMs can be brefly descrbed as a seres of states lnked by several probablstc paths. A typcal model often used for sequence algnment s descrbed n Krogh et al, 1994 (1). The states represent matches, gap nsertons, or gap deletons, and the match states also defne the probabltes for the occurrence of each amno acd. The arrangement of the states and the probabltes are often bult by a gven tranng set of amno acd sequences,.e. a known set of orthologs. If one partcular sequence n queston wants to be scored aganst the known tranng set, every possble path through the HMM that gves that sequence s determned and the total probablty, or the sum of the probabltes of all the paths, s calculated. Ths probablty then serves as the bass of the score. A dagram of the dfferent states and ther nterrelatons s gven n Eddy, 1996 (2): Page 1 of 5

2 In ths case, the M states are the match states wth the amno acd probabltes, the D states are the deleton states, and the I states are the nserton states. Each of the arrows denotng the paths also would be assgned a probablty. Ths would be a typcal HMM for a 2 or 3 amno acd sequence. A Turng Machne s smlar to ths n that t also s a set of states connected by paths. The Turng Machne was devsed by Brtsh mathematcan Alan Turng as a way of dealng wth the decdablty problem, or "Entschedungsproblem" (3). The mathematcan Kurt Gödel had shown logcally that certan mathematcal problems were unsolvable, and Turng's nventon was a practcal proof of that, but one whch also happened to be programmable and thus became the foundaton of modern computer scence (4). The machne can be loosely descrbed as a "black box" equpped wth a 1 dmensonal data tape, whch has three defned functons - read, wrte, and move. The box tself can assume any number of a fnte number of states, smlar to a Hdden Markov Model. A typcal Turng machne moves n a seres of tme steps - each step nvolves readng from the tape, assumng the state based on the read nput, movng along the tape, and wrtng. The tape tself contans an ntal set of symbols, and to whch are wrtten new symbols by the Turng Machne. The machne tself moves one slot on the tape wth each step. We can bascally defne each Turng Machne wth "quntuples," a set of fve values, statng the read nput, the current state, the new state, the drecton to move left or rght, and what to wrte. In mathematcal terms, Feynman proposed representng the quntuples as a seres of 5 varable sets, Q, S, Q, S, and D, subected to the three functons F, G, and D where Q = F(Q, S ) S = G(Q, S ) D = D(Q, S ) Q and S represent the ntal states and nput tape symbol, respectvely, whle Q and S represent the new state and the wrte symbol. D represents the drecton for the machne to move (5). A more ntutve way of dong ths s to draw a state dagram smlar to that of a Hdden Markov Model n fgure 1, as follows (5): A farly good llustraton of how a Turng Machne works s gven by Rchard Feynman n hs seres of lectures on computaton at Caltech n the 1980's (5). A verson of the machne he descrbed s shown here. Ths partcular machne checks for properly nested parentheses n a strng of arbtrary length. The state dagram s as follows: Page 2 of 5

3 We supply a tape wth the followng symbols: E ( ( ) ( ) ( ) ( ) ) ( ( ( ( ( ) ( ) ) ( ( ) ) ) ) ) ) ) E Ths s bascally a parentheses strng flanked by E symbols. We defne the machne to start at the leftmost E, and work ts way step by step to the rght, keepng the machne n the state R1. It gnores any left parentheses, "(",.e. wrtng left parentheses back nto ther orgnal slots, untl t hts a rght parenthess, ")", whereupon t replaces the ")" wth an "X", swtches nto state L1 and changes drecton back left. It replaces the frst "(" t sees wth another "X", swtches back to R1 and changes drecton back to the rght. Eventually, t wll reach ether the left E or the rght E, where we can tell whether the parentheses are properly nested by several of the followng ways: f t has ht the left E, we know that the parentheses strng s not properly nested, as ths sgnfes an unpared ")". We can replace that leftmost E wth a "0" to sgnfy "false". If t hts the rght E, then we have the machne swtch nto state L2 where t moves left. If t hts an "X," nothng happens, but f t hts a "(," t swtches nto state L3. When t hts the left E, t wrtes a 1 f n state L2, and a 0 f n state L3. Thus, we can tell f the parentheses were properly nested by lookng at the output at the leftmost E when the machne has fnshed. The fnshed state of the tape s thus: 0 X X X X X X X X X X X X X X X X X X X X X X X X X X ) ) E Turng machnes can also be used to defne varous mathematcal functons. An example of a smple adder s gven as follows: Page 3 of 5

4 Intal tape: A B C D The machne would move rght untl t ht a B, move left, and replace the dgt mmedately to the left wth another B and change nto a one of 2 states (R2, R3) representng the dgt t ust replaced, move rght untl t ht the C, and replace the dgt to the left wth a C, and assume one of three states representng the sum of that dgt and the one represented by the correspondng dgt of B (0, 1, 0 + carry - states R4, R5, and R6, respectvely). It moves rght, and wrtes that sum dgt nto the space drectly to the rght of the last C whle shftng exstng dgts to the rght of C further rght by 1 slot. It then moves left to the leftmost B to deal wth the next dgts. If t needs to carry a dgt, t assumes an alternate leftward state to sgnfy ths. Each new dgt sum s nserted nto the space drectly to the rght of the last C. We would, n theory, end up wth the tape A B B B B B C C C C C D The Turng Machne model s partcularly powerful, as once the basc mathematcal operatons are defned, one can do a varety of tasks wth them. In lght of ths, we can assume that Turng Machnes can be appled to certan common bonformatcs tasks. One possble use would be to mplement a smple Needleman-Wunsch scorng algorthm (6) by ncorporatng the addng machne descrbed above. In comparng two sample proten sequences, say ALWE and ALPN, we could have a tape as follows: X 0 A L W E X 1 A L P N X 2 X 3 X 4 A Turng Machne shuttlng back and forth could easly match up the dfferent pars of amno acds, and wrte the rows, n sequental order, of a Needleman-Wunsch smlarty matrx between X 2 and X 3. It could then sum up the smlarty matrx and wrte the resultng rows between X and X. If the rows are separated by a placeholder, you could even have the 3 4 machne zp back and forth among the rows n X and X, fnd the maxma of each row, and 3 4 wrte down possble algnment paths through the Needleman-Wunsch matrx on the tape after X. Another applcaton would be to smulate ndvdual paths through a HMM, wth 4 probablstc Turng Machnes descrbed n Leeuw et al (7). Such machnes have the functons F, G, or D of the quntuples set to generate a random values from the fnte set of possble Turng machne states or outputs. We could set up a generc nput tape to such a machne, such as X 1 X 2 X 3... X 20 for a random 20 amno acd sequence from a gven HMM confguraton, mnus any gaps that Page 4 of 5

5 mght be nserted, and run t through a Turng Machne wth a state dagram and quntuple functons defned to be equvalent to the HMM. The 1 letter abbrevaton for each amno acd, or a placeholder for gaps, could be nserted as the Turng machne steps over each X slot, thus generatng on the tape a random sequence from the HMM. Or, we could have a gven amno acd sequence on a tape,.e. T Y N G A A V X1 and we could somehow have a Turng Machne defned wth a HMM state dagram step through the sequence, evaluate all the possble paths through the state dagram, and calculate the sum of the probabltes, gvng the score. All ths leads to the obvous queston - what advantages do Turng Machnes offer over current sequence algnment technques? Wth the computers currently n use, a Turng Machne s a "vrtual" or emulated mplementaton wrtten on top of the exstng natve language of the computer. Thus, t s often a slower way of dong thngs rather than encodng programs wth smple logc nstructons wth assembly language, or wth languages such as C or perl. As noted above, however, ther smlarty wth HMMs could mean that t would be easy to couple them wth a HMM based algnment or proflng scheme. Also, the smularty of turng machnes wth natural bologcal obects that handle bts of nformaton s strkng - both usually operate on a 1-dmensonal nformaton storage system. What s a RNA polymerase but a Turng Machne defned as a coper wth separate read and wrte tapes? What s a rbosome but a translatng Turng Machne? The use of nuclec acd based Turng Machnes have already been touched on n Adleman, 1994 (8), and Shapro, 2000 (9). As the felds of bocomputng, proten desgn, proten engneerng, and structure predcton mprove, we may be able to desgn molecular Turng machnes able to perform sequence algnments drectly n the test tubes, or perhaps even n vvo sequence analyss or structure predcton. In other words, there may be a potental n bologcal systems for a natve or "hard" Turng Machne. Also, gven the nature of the unversal Turng Machne and the relatvely free-form nature of the machne, there are probably a multtude of bonformatcs applcatons overlooked n ths report and are ust watng to be dscovered. Works Cted 1. Krogh, A., Brown, B., Man, I.S., Solander, K., Haussler, D. (1994). "Hdden Markov Models n Computatonal Bology: Applcatons to Proten Modelng." J. Mol. Bol. 235: Eddy, S.R. (1996). "Hdden Markov Models." Curr. Opn. Struc. Bol. 6, Turng, A.M. "On the Computablty of Numbers, Wth an Applcaton to The Entschedungsproblem." Proc. of the London Mathematcal Socety Hopcraft, J.E. (1984). "Turng Machnes." Sc. Am Feynman, R.P. (1996). Feynman Lectures on Computaton. Eds. T. Hey and R. W. Allen. New York: Addson-Wesley. 6. Needleman, S. B. and Wunsch, C. D. (1971). "A general method applcable to the search for smlartes n the amno acd sequence of two protens." J. Mol. Bol. 48: Leeuw, K. de, Moore, E.F., Shannon, C.E., and Shapro, N. (1956). "Computablty by Probablstc Machnes." Annals of Mathematcal Studes (Automata Studes). 34: Adleman, L. (1994). "Molecular Computaton of Solutons to Combnatoral Problems." Scence. 266: Shapro, E. (2000). "A Mechancal Turng Machne: Blueprnt for a Bomolecular Computer." < Page 5 of 5