Hidden Markov Models. Seven. Three-State Markov Weather Model. Markov Weather Model. Solving the Weather Example. Markov Weather Model

Transcription

1 American Universiy of Armenia Inroducion o Bioinformaics June 06 Hidden Markov Models Seven Inroducion o Bioinformaics : /6 : /6 3 : /6 4 : /6 5 : /6 6 : /6 Fair Sae Sami Khuri Deparmen of Compuer Science San José Sae Universiy : /0 : /6 : /0 San José, CA 959 : /6 3 : /0 3 : /6 4 : /0 4 : /6 5 : /0 5 : /6 June 06 6 : / Loaded Sae 6 : /6 Fair Sae : /0 : /0 3 : /0 4 : /0 5 : /0 6 : / Loaded Sae Hidden Markov Models Russian Mahemaician Sain Peersburg Andrei Andreyevich Markov Markov Chain Homology Model profile Hidden Markov Model Vierbi Algorihm Forward Algorihm Backward Algorihm EM Algorihm hree-sae Markov Weaher Model We have hree saes: Rainy (R) Cloudy (C) Sunny (S) he weaher on any day is characerized by a single sae. Sae ransiion probabiliy marix: A Markov Weaher Model 0.4 Rainy Cloudy Sunny 0.8 Markov Weaher Model Rainy Cloudy Sunny 0.8 Compue he probabiliy of observing SSRRSCS given ha oday i is sunny (i.e., we are in sae S) Solving he Weaher Example Observaion seuence: O ( S, S, S, R, R, S, C, S) Using he chain rule we ge: PO ( model) PSSSRRSCS (,,,,,,, model) PSPS ( ) ( SPS ) ( SPR ) ( SPR ) ( R) PS ( RPC ) ( SPS ) ( C) π a a a a a a a ()(0.8) (0.)(0.4)(0.3)(0.)()

2 American Universiy of Armenia Inroducion o Bioinformaics June 06 Saes and ransiions he Disressed Suden Model Evaluaing Observaions Saring Sae of he Suden he probabiliy of observing a given seuence is eual o he produc of all observed ransiion probabiliies. Suppose ha: L: suden is in sae C: suden is in sae B: suden is in sae Sar he Model has a Sar Sae wih ransiion probabiliies of going o L, C, or B of / Behavior of hree Sudens Sar Suden : LLLCBCLLBBLL Suden :LCBLBBCBBBBL Suden 3 :CCCLCCCBCCCL Compuing Observed Seuences he probabiliy of observing a given seuence is eual o he produc of all observed ransiion probabiliies. Example: LLLCBCLLBBLL) /3 * 0. * 0. * 0. * 0.75 * 0. * 0.05 * 0. * 0.8 * 0.7 * * 0..4 *

3 American Universiy of Armenia Inroducion o Bioinformaics June 06 Compuing Observed Seuences LCBLBBCBBBBL) /3 * 0. * 0.75 * * 0.8 * 0.7 * 0. * 0.75 * 0.7 * 0.7 * 0.7 *.4406 * 0-5 CCCLCCCBCCCL) /3 * * * 0.05 * 0. * * * 0.75 * 0. * * * * 0-0 he Random Model he ull Hypohesis Sar Sae wih Random Model Sar Sar Sudens wih Random Model -6 Suden : LLLCBCLLBBLL.887 x 0-6 Suden :LCBLBBCBBBBL.887 x 0-6 Suden 3 :CCCLCCCBCCCL.887 x 0 Odds and Log Raios o deermine he significance of he resuls obained wih he 3 sudens, compare hem o he null model (random model) Odds Raio x Disressed Model) / x ull Model) Log Odds Log [ x Disressed Model) / x ull Model)] Likelihood Raios: Disressed Likelihood raios: Suden : Suden : Suden 3 : x 0 /.887 x 0 x x 0 /.887 x 0 y 4 x /.887 x 0 z Log likelihood raios: Suden log x Suden log y.94 Suden 3 log z

4 American Universiy of Armenia Inroducion o Bioinformaics June 06 he Successful Suden Model Sar Sudens wih Successful Model Sar Suden : LLLCBCLLBBLL Suden :LCBLBBCBBBBL Suden 3 :CCCLCCCBCCCL Oucomes wih Successful Model LLLCBCLLBBLL) /3 * 0.6 * 0.6 * 5 * 0.05 * 0.9 * 0.75 * 0.6 * 0.5 * 0.05 * 0.05 * * 0-7 LCBLBBCBBBBL) /3 * 5 * 0.05 * 0.05 * 0.5 * 0.05 * 0.9 * 0.05 * 0.05 * 0.05 * 0.05 * * 0-3 CCCLCCCBCCCL) /3 * * * 0.75 * 5 * * * 0.05 * 0.9 * * * * 0-7 Likelihood Raios: Successful Likelihood raios: Suden :.3669 x 0 Suden : Suden 3 : x 0.35 x /.887 x 0 z Log likelihood raios: Suden log x Suden log y -.03 Suden 3 log z /.887 x 0 x 3-6 /.887 x 0 y HMM Combined Model HMM Combined Model Successful Successful Sar End Disressed Disressed 7.4

5 American Universiy of Armenia Inroducion o Bioinformaics June 06 Hidden Markov Model Evaluaing Hidden Saes S S Sar End Sar End D D Given an observaion: LLLCBCLBBCL, find he seuence of saes which is he mos likely o have produced he observaion. Models of Seuences Consiss of saes (circles) and ransiions (arcs) labelled wih probabiliies. Saes have probabiliies of emiing an elemen of a seuence (or nohing). Arcs have ransiional probabiliies of moving from one sae o anoher. Sum of probabiliies of arcs ou of a sae mus be Self-loops are allowed. Markov Chain A seuence is said o be Markovian if he probabiliy of he occurrence of an elemen in a paricular posiion depends only on he previous elemens in he seuence. Order of a Markov chain depends on how many previous elemens influence he probabiliy: 0 h order: uniform probabiliy a every posiion s order: probabiliy depends only on he immediaely previous posiion. Simple Markov Model Example: Each sae emis (or, euivalenly, recognizes) a paricular number wih probabiliy, and each ransiion is eually likely. Begin Emi Emi Emi 4 Emi 3 End Probabilisic Markov Model ow, add probabiliies o each ransiion 0.5 Begin Emi Emi End 0.75 Emi Emi 3 Possible seuences:

6 American Universiy of Armenia Inroducion o Bioinformaics June 06 Probabilisic Markov Model We can compue he probabiliy of occurrence of any oupu seuence: Begin Emi Emi Emi Emi 3.0 End p (34) 0.5 * 0. * 0.75 * p (4) 0.5 * p (334) 0.5 * 0.75 * * Probabilisic Emission Define a se of emission probabiliies for elemens in he saes. Given an oupu seuence, where does i come from? Begin A (0.8) B() 0.9 C (0.) D (0.9) B (0.7) C(0.3) C (0.6) A(0.4) BCCD or BCCD?.0 End Hidden Markov Models Emission uncerainy means he seuence does no idenify a uniue pah. he saes are hidden : Begin A (0.8) B() 0.9 C (0.) D (0.9) B (0.7) C(0.3) C (0.6) A(0.4) BCCD or BCCD?.0 End Compuing Probabiliies Probabiliy of an oupu seuence is he sum of all he pahs ha can produce i: Begin A (0.8) B() 0.9 C (0.) D (0.9) B (0.7) C(0.3) C (0.6) A(0.4) p(bccd) (0.5 * * 0. * 0.3 * 0.75 * 0.6 * 0.8 * 0.9) + (0.5 * 0.7 * 0.75 * 0.6 * * 0.6 * 0.8 * 0.9) End he Dishones Casino (I) he Dishones Casino (II) : /6 : /6 3 : /6 4 : /6 5 : /6 6 : / : /0 : /0 3 : /0 4 : /0 5 : /0 6 : / : /6 : /6 3 : /6 4 : /6 5 : /6 6 : / : /0 : /0 3 : /0 4 : /0 5 : /0 6 : / Fair Sae Loaded Sae Fair Sae Loaded Sae If we see a seuence of rolls (he seuence of observaions) we do no know which rolls used a loaded die and which used a fair die A B Iniial Sae Sae ransiions Emissions 7.6

7 American Universiy of Armenia Inroducion o Bioinformaics June 06 he Urn and Ball Model (I) he Urn and Ball Model (II) urns conaining colored balls M disinc colors of balls Algorihm ha generaes Observed Seuence:. Pick iniial urn according o some random process.. Randomly pick a ball from he chosen urn, record is color and hen pu i back. 3. Randomly pick an urn 4. Repea seps and 3 An urn is seleced and hen a ball is seleced from he urn, is color is recorded, and he ball is pu back in he urn. Given he seuence of observed colors, can we guess from which urn each ball comes from? Looking for CpG Islands he Rariy CpG Islands Example: A CpG island in humans refers o he dinucleoide CG and no he basepair CG. he C of CpG is generally mehylaed o inacivae genes hence CpG is found around sar regions of many genes more ofen han elsewhere. Mehylaed C is easily muaed ino. CpG Island Crieria According o Gardiner-Garden and Fromer, CpG islands are commonly defined as regions of DA of a leas 00 bp in lengh, ha have a G+C conen above 50% ha have a raio of observed vs. expeced CpGs close o or above 0.6. Ses of CG repea elemens, usually found upsream of ranscribed regions of he genome. Looking for CpG Islands CpG islands are herefore rare in oher locaions CpG islands are generally a few hundred base pairs long Quesions:. Given a shor DA fragmen, does i come from a CpG island or no?. Given a long unannoaded seuence of DA, how do we find he CpG islands? 7.7

8 American Universiy of Armenia Inroducion o Bioinformaics June 06 Building an HMM for CpG Islands A se of human seuences were considered and 48 CpG islands were abulaed. wo Markov chain models were buil: One for he regions labeled as CpG islands (he + model or Model ) One for he remainder of he seuences (he - model or Model ). ransiion Probabiliies he ransiion probabiliies of each model were compued by: + c s a c + + s s c + ' s' s s c ' s' is he number of imes leer followed leer s in he plus model. a c he wo ransiion ables Log Odds Raio Given any seuence, we compue he log-odds raio o discriminae beween he wo models : x he + model) S( x) log x he model) L + xi xi log i ax i xi S(x)>0 means x is likely o be a CpG island. he raio is also called he log likelihood raio of ransiion probabiliies. a Log Likelihood Raios x he + model) S( x) log x he model) a L + xi xi log i ax i xi Looking for CpG Islands Given a long unannoaded seuence of DA, how do we find he CpG islands? he able s uni is he bi since base is used for he compuaion of he individual enries of he able. We can use a sliding window of size 00, for example, around each nucleoide in he seuence and use he previous able o score he log-odds. CpG islands would sand ou wih posiive values. 7.8

9 American Universiy of Armenia Inroducion o Bioinformaics June 06 Sliding Window Size he Island and he Sea How do we deermine he window size? CpG islands are of variable lenghs and migh have sharp boundaries. A beer approach is o build an HMM ha combines boh models. An HMM for CpG Islands (I) An HMM for CpG Islands (II) here are 8 saes, one for each nucleoide in a CpG island (+), and one for each nucleoide no in a CpG island (-). here wo saes for each oupu symbol. Example: is recognized or generaed by + or -. Wihin each group of saes, he group has he same behavior as he original Markov Model. An HMM for CpG Islands (III) he wo Pahs wih CGCG Assume he ransiions from a (+) nucleoide o a (-) are small. And ransiions from (-) nucleoides o (+) are also small. 7.9

10 American Universiy of Armenia Inroducion o Bioinformaics June 06 Swiching beween + & - Saes he maximum scoring pah receives a score of he mos likely sae pah is found o be C+G+C+G+. Given a much longer seuence, he derived opimal pah will swich beween he CpG and non-cpg saes. Applicaions of HMMs Generaing muliple seuence alignmens Modeling Proein Family discriminae beween seuences ha belong o a paricular family or conain a paricular domain vs. he ones ha do no. Sudy he model direcly he model may reveal somehing abou he common srucure of proeins wihin a family. Gene predicion Recognizing AG Eddy s oy Model inser saes mach saes delee saes HMM for Proein Family HMM: Begin and End Saes D D D 3 D 4 delee saes I 0 I I I 3 I 4 inser saes mach saes M 0 M M M 3 M 4 M 5 he general model wih Begin and End saes 7.0

11 American Universiy of Armenia Inroducion o Bioinformaics June 06 Family of Seuences If he emission probabiliies for he mach and inser saes are uniform over he 0 amino acids, he model will produce random seuences. If each sae emis one amino acid only, and ransiion probabiliies from one mach sae o he nex are one, hen he model will produce he same seuence. Somewhere beween he wo exreme cases we can se he parameers o obain a family of seuences (seuences ha are similar). he Goal Find a model, in oher words, a model lengh, and parameers, ha accuraely describes a family of proeins. he model will assign high probabiliies o proeins in he family of seuences ha i is designed for. Profile Hidden Markov Model Allowing gap penalies and subsiuions probabiliies o vary along he seuences reflecs biological realiy beer. Alignmens of relaed proeins have regions of higher conservaion, called funcional domains and regions of lower conservaion. Funcional domains have resised o change indicaing ha hey serve some criical funcion. Esimaing he Parameers In he HMM model of a proein family, he ransiion from: a mach sae o an inser sae corresponds o a gap open penaly an inser sae o iself corresponds o he gap exension penaly All applicaions of he HMM model sar wih raining or esimaing he parameers of he model using a se of raining seuences chosen from a proein family. Profile HMM From MSA Regular Expressions for MSA he given DA moif can be represened by a regular expression: [A][CG][AC][ACG]*A[G][GC] sar end Is his a good represenaion? he expression does no disinguish beween: GC - - AGG highly implausible seuence ACAC- - AC highly plausible seuence 7.

12 American Universiy of Armenia Inroducion o Bioinformaics June 06 Example: HMM for MSA (I) A Hidden Markov Model derived from he given alignmen. Example: HMM for MSA (II) Seuences, 3 and 5 have inserions of varying lenghs. So 3 ou of 5 seuences have inserions. Example: HMM for MSA (III) Example: HMM for MSA (IV) In he inserion sae we have: A:, C:, G:, :. Afer seuences,3,5 have made one inserion, we sill need more inserions for seuence. he oal number of ransiions back o he mach saes is 3. So here are 5 ransiions ou of he inserion sae. Compuing Probabiliy of Pah Compuing Probabiliy of Pah (II) ACACAC) 0.8 *.0 * 0.8 *.0 * 0.8 * 0.6 * 0.4 * 0.6 *.0 *.0 * 0.8 *.0 * CAACAC) *.0 * 0.8 *.0 * 0.8 * 0.6 * * 0.4 * 0.4 * 0.4 * * 0.6 *.0 *.0 * 0.8 *.0 *

13 American Universiy of Armenia Inroducion o Bioinformaics June 06 HMM: Compuing Log Odds Log Odd Scores of Seuences Using Log Odds of seuence S of lengh L log [ S ) / (5) L ] log S) L log (5) Example: Log Odds of ACACAC log (ACACAC)) 7 * log (5) 6.7 [naural logarihm] Log Odd Scores of Seuences HMM wih Log Odds of Each Base A seuence ha fis he moif very well has a high log-odds score. A seuence ha fis he null hypohesis beer has a negaive log-odds score. Emission of each base: log(p(base)) - log(5) ransiion probabiliies are convered o simple logs Log Odds Score of a Seuence SH3 Domain Example (I) Log Odds score of ACACAC An alignmen of 30 shor amino acids chopped ou of an alignmen of an SH3 domain. he shaded areas are he mos conserved and were chosen o be represened by he main (mach) saes and he unshaded area wih lower-case leers was chosen o be represened by an inser sae. [Kro98] 7.3

14 American Universiy of Armenia Inroducion o Bioinformaics June 06 SH3 Domain Example (II) SH3 Domain Example (III) he inser sae represens highly variable regions of he alignmen SH3 Domain Example (IV) SH3 Domain Example (V) 76/06 A profile HMM made from he alignmen. ransiion lines wih no arrow head are from lef o righ. ransiions wih probabiliy zero are no shown. hose wih very small probabiliy are shown as dashed lines. ransiion from an inser sae o iself are no shown; insead he probabiliy muliplied by 00 is shown in he diamond. he numbers in he circular delee saes are jus posiion numbers. SH3 Domain Example (VI) Markov Model Assumpions (I) Afer all 30 seuences have made one inserion each, here are 76 more inserions (number of amino acids in columns,3,4,5,6,7,8 of he inser sae) and here are 30 ransiions back o he mach saes. So here are a oal of ransiions ou he inser sae. self-loop) 76/06 and back o mach) 30/06. A se Q of saes, denoed by,,, An observable seuence, O: o,o,,o,,o An unobservable seuence, :,,,,, Firs order Markov model: P ( j i, k,...) P ( j i) 7.4

15 American Universiy of Armenia Inroducion o Bioinformaics June 06 Markov Model Assumpions (II) An iniial probabiliy disribuion: π i P ( i) i where π i i Saionary condiion: P ( j i) P ( j i) + l + l Sae ransiion Probabiliies Sae ransiion probabiliy marix: where:! a # a... a j... a # a # a... a j... a # A!!!!!! # a i a i... a ij... a i # # # "# ij!!!!!! a a... a j... a $ & & & & & & & & %& a j i) i, j a ij 0, i, j a ij, i j Hidden Markov Model : he number of hidden saes A se of saes: Q {,,, } M: he number of symbols A se of symbols: V {,,, M} A: he sae-ransiion probabiliy marix a i, j + j i) i, j B (or b): Emission probabiliy disribuion; k is a symbol from V: B j (k) o k j) j M he iniial sae disribuion π: π i P ( i) i he enire model λ, is given by: λ ( AB,, π) hree Basic Quesions. EVALUAIO given observaion O(o, o,,o ) and model λ ( AB,, π), efficienly compue: PO ( λ). Given wo models λ and λ, his can be used o choose he beer one. Use: Forward Algorihm or Backward Algorihm. DECODIG - given observaion O(o, o,,o ) and model λ find he opimal sae seuence (,,, ). Opimaliy crierion has o be decided (e.g. maximum likelihood) Use: Vierbi Algorihm 3. LEARIG given O(o, o,,o ), esimae model parameers λ ( AB,, π) ha maximize PO ( λ). Use: EM and Baum-Welch Algorihms Doubly Sochasic Process According o Rabiner and Juang: A Hidden Markov Model is a doubly sochasic process wih an underlying sochasic process which is no observable (i is hidden), bu can be only observed hrough anoher se of sochasic processes ha produce he seuence of observed symbols. (IEEE ASSP, January 986) HMM and Logarihms In a Hidden Markov Model here is no a one o one correspondence beween he saes and he symbols as is he case wih Markov Chains. Exensive muliplicaion operaions wih probabiliies ofen resul in underflows. Use logarihms: producs become sums. 7.5

16 American Universiy of Armenia Inroducion o Bioinformaics June 06 Scoring a Seuence All seuences will have a pah hrough he HMM. For mos seuences (excep very shor ones) here are a huge number of pahs hrough he model, mos of which will have very low probabiliy values. For a given observed seuence, we can approximae he oal probabiliy by he probabiliy of he mos likely pah. Vierbi: mehod for finding he mos likely pah. Vierbi: A Summary Similar o Dynamic Programming already sudied. Make a marix wih rows for seuence elemens and columns for saes in he model. Work row by row, calculaing he probabiliy for each sae o have emied ha elemen and puing ha probabiliy in a cell. When here are muliple pahs, selec he highes probabiliy and sore he seleced pah. Curren row uses resuls of previous row. Highes enry in he las row gives bes oal pah hrough back racking. hree Basic Quesions (I). EVALUAIO given observaion O(o, o,,o ) and model λ (A,B,π ), efficienly compue PO ( λ). Hidden saes complicae he evaluaion. Given wo models λ and λ, his can be used o choose he beer one.. DECODIG - given observaion O(o, o,,o ) and model λ find he opimal sae seuence (,,, ). Opimaliy crierion has o be decided (e.g. maximum likelihood) Explanaion of he daa. 3. LEARIG given O(o, o,,o ), esimae model parameers λ ( AB,, π) ha maximize PO ( λ). hree Basic Quesions (II) he Evaluaion Problem Given he observaion seuence O and he model λ, how do we efficienly compue O λ), he probabiliy of he observaion seuence, given he model? he Decoding Problem Given he observaion seuence O and he model λ, find he opimal sae seuence associaed O. Vierbi Algorihm finds he single bes seuence for he given observaion seuence O. he Learning Problem How can we adjus he model parameers o maximize he join probabiliy (likelihood)? hree Basic Quesions (III) he Evaluaion Problem Given he observaion seuence O and he model λ, how do we efficienly compue O λ), he probabiliy of he observaion seuence, given he model? Use: Forward Algorihm or Backward Algorihm. he Decoding Problem Given he observaion seuence O and he model λ, find he opimal sae seuence associaed wih O. Vierbi Algorihm finds he single bes seuence for he given observaion seuence O. Use: Vierbi Algorihm. he Learning Problem How can we adjus he model parameer o maximize he join probabiliy (likelihood)? Use: EM and Baum-Welch Algorihms. Soluion o Problem One (I) Problem: Compue o, o,,o λ) P ( O λ) O, λ) Bu: PO (, λ) PO (, λ) P ( λ) O, λ) o, λ) b (o )b (o )... b (o ) he summaion is over all pahs (,,, ) ha give O. We assume ha he observaions are independen Sami Khuri 7.6

17 American Universiy of Armenia Inroducion o Bioinformaics June 06 Soluion o Problem One (II) We also have: By replacing in: we have: O λ) π b ( ) ( ) ( )... o a b o a b 3 o 3 3 a λ) π a a... a 3 P ( O λ) O, λ) We have: muliplicaions and a maximum of sae seuences and O() calculaions. Complexiy: O( ). b ( o ) Soluion o Problem One (III) Since he complexiy is O( ), he brue force evaluaion of: P ( O λ) O, λ) by enumeraing all pahs ha generae O is no pracical. o efficienly compue o, o,,o λ), use he Forward Algorihm Sami Khuri Sami Khuri Forward Algorihm Define forward variable α () i as: α ( i) o, o,..., o, i λ) α () i is he probabiliy of observing he parial seuence ( o, o,..., o ) and landing in sae i a sage (sae is i). Recurrence Relaion:. Iniializaion:. Inducion: 3. erminaion: Complexiy: O ( ) α() i πibi( o) α+ ( j) α ( i) aij bj ( o + ) i PO ( λ) α ( i) i Saes Forward Procedure: Inducion a j α α 3 v a 3j a j a j + ime ( j) ( i) a b ( o ) + ij j + i j Already known from previous seps Saes Forward Procedure: erminaion 3 v a i a 3i a i a i - ime i PO ( λ) α ( i) i Use eiher Forward or Backward Algorihm o solve Problem One. Backward Algorihm Define backward variable β () i + + as: β () i o, o,..., o i, λ) β () i is he probabiliy of observing he parial seuence ( o knowing +, o+,..., o) ha we land in sae i a sage (in oher words, sae is i). 7.7

18 American Universiy of Armenia Inroducion o Bioinformaics June 06 Backward: Recurrence Relaion Backward Algorihm Recurrence Relaion of β () i Iniializaion: β () i Inducion: β () i a b ( o ) β ( j), i,,..., ij j + + j erminaion: Complexiy: O ( ) ( j j P O λ) π j b ( o ) β ( j) Saes ime o o.. o - o o + o +.. o - o Observaion Backward Algorihm: Remark Backward variable β () i is given by: β () i o, o,..., o i, λ) + + We noe ha, unlike he forward variable, here we know in which sae he process is a ime (sae i). he disincion is made o be able o combine he forward and backward variables o produce a useful resul. Using Forward and Backward (I) Compue he probabiliy of producing he enire observed seuence, O, wih he h symbol produced by sae i. O, i) i O) O) O, i) o, o,... o, o o,... o o, o,... o, i) o o, o,... o, i) o α ( i) β ( i) + +,... o, o, i), o, o, o,... o, i) We drop λ for convenience + +,... o,... o, o, o o, o,... o, i) i) Or: Using Forward and Backward (II) O, i) i O) O) α ( i) β ( i) O) O, i, λ) i O, λ) O, λ) α ( i) β ( i) O, λ) O) can be compued by using eiher he forward or backward algorihm. Soluion o Problem (I) We have o find a sae seuence: (,,, ), such he probabiliy of occurrence of he observed seuence: O(o, o,,o ) from he sae seuence, is greaer han or eual o any oher sae seuence. Find a pah * ( *, *,, * ) ha maximizes he likelihood: P (,,..., Oλ, ) 7.8

19 American Universiy of Armenia Inroducion o Bioinformaics June 06 Soluion o Problem (II) he Vierbi algorihm can be used o solve his problem. I is a modified forward algorihm. Insead of aking he sum of all possible pahs ha end up in a desinaion sae, he Vierbi algorihm picks and remembers he bes pah. Soluion o Problem (III) Use Dynamic Programming Define δ i ) max,,... i, o, o... o λ ) (,,... δ () i is he highes probabiliy pah ending in sae i a sep (ime ). By inducion we have: δ () j max[ δ () i a ] b ( o ) + ij j + i Vierbi Algorihm (I) Iniializaion: Recursion: δ () i π b( o ), i i i ψ () i 0 δ ( j) max[ δ ( i) a ] b ( o ) ij j i ψ ( j) argmax[ δ ( i) a ] ij i, j Vierbi Algorihm (II) erminaion: where * P P * * max[ δ ( i)] i arg max[ δ ( i)] i A maximum likelihood pah is given by: * ( *, *,, * ), where P (,,..., Oλ, ) * * ψ + ( + ),,,..., Saes Vierbi Algorihm (III). j i. δ k ( i) ( j) δ k. k- ime k - O O. O k- O k O - O Observaion racing back he opimal sae seuence max δ ( i) Soluion o Problem 3 Esimae λ ( AB,, π) o maximize PO ( λ) o analyic mehods exis because of complexiy Use an ieraive soluion. Expecaion Maximizaion: he EM algorihm. Le iniial model be λ 0. Compue new λ based on λ 0 and observaion O. 3. If log PO ( λ) log PO ( λ0 ) < DELAsop 4. Else se λ 0 λ and go o sep 7.9

20 American Universiy of Armenia Inroducion o Bioinformaics June 06 EM Special Case: Baum-Welch he Expecaion Maximizaion Algorihm is a very powerful general algorihm for probabilisic parameer esimaion. he Baum-Welch Algorihm is a special case of he Expecaion Maximizaion Algorihm. Parameer Esimaion for HMMs here are wo pars for specifying a Hidden Markov Model:. Design of he srucure (more of an ar) Deermining he saes Deermining he connecions of he saes. Assignmen of parameer values (a well-developed heory exiss) Deermining he ransiion and emission probabiliies. Assignmen of Parameer Values here are wo cases o consider when assigning parameer values o HMMs: Esimaion when he sae seuence is known Example: Locaion of CpG islands are already known Esimaion when he sae seuences are unknown. Esimaion wih Known Sae Pahs Esimaion of he parameers is sraighforward when seuence pahs are known. Coun he number of imes a paricular ransiion (denoed by A) or emission (denoed by B) is used in he raining se he maximum likelihood esimaions are: a kl Akl A l' kl ' b ( d) k Bk ( d) B ( d') d ' k he Dangers of Overfiing Pseudocouns o he Rescue When esimaing parameers, especially from a limied amoun of daa, here is a danger of overfiing: he model becomes very well adaped o he raining daa and does no generalize well o esing daa (new daa). a kl Akl A l' kl ' b ( d) k Bk ( d) B ( d') d ' k o avoid overfiing, add predeermined pseudocouns r & kl rk (d) o he numeraors of he ransiion esimaors: A kl is he number of ransiions k o l in he raining daa + r kl B k (d) is he number of emissions of d from sae k in he raining daa + r k (d) A B ( ) kl k d akl bk ( d) A Bk ( d' l' kl ' ' ' ) d 7.0

21 American Universiy of Armenia Inroducion o Bioinformaics June 06 Esimaion if Pahs are Unknown When pahs are unknown for raining seuences, we have no direc closed-form euaion for he esimaed parameer values. Ieraive procedures are used. he Baum-Welch algorihm (special case of he EM algorihm) has become he sandard mehod when pahs are unknown. he wo Seps of Baum-Welch he Baum-Welch Algorihm is based on he following observaion: If we knew he pahs, we could compue ransiion and emission probabiliies If we knew he ransiion and emission probabiliies, we could compue he pahs (for example: he mos probable pah) he algorihm alernaes beween he wo. Baum-Welch Ieraive Process he Baum-Welch Algorihm is basically an ieraive process ha alernaes beween he following wo seps: Esimae A and B kl k (d) by considering probable pahs for he raining seuence using he curren values of a kl and b k (d). [Expecaion] Derive new values by using above values in: [Maximizaion] A B kl k ( d) akl bk ( d) A Bk ( d') l' kl ' d ' Ierae unil some sopping crierion is reached. Baum-Welch a Work (I) a kl he probabiliy ha is used a posiion in he observed seuence O(o, o,,o ) is given by: α ( k) aklbl ( o + ) β+ ( l) k, + l O, λ) O, λ) hen he expeced number of imes ha a kl is used is obained by summing over all posiions and over all raining seuences: A j j j kl α ( k) a j klbl ( o + ) + ( l o ) ) β j Baum-Welch a Work (II) A j j j kl α ( k) aklbl ( o + + l j j P o ) ) β ( ( ) j α is he forward variable for raining seuence j (k) j β (k) is he backward variable for raining seuence j. Similarly, he expeced number of imes ha symbol d is emied from sae k in all he seuences is given by: j j Bk ( d) α ( k) β ( k) j j o ) j { o d} he inner sum is only over posiions for which he emied symbol is d. Baum-Welch Ieraion Use he newly compued expecaion values: A kl and B k (d) o calculae he new model ransiion and emission parameers : a kl Akl A l' We hen compue again A and (d) based on he new parameers and ierae once more Sami Khuri Sami Khuri kl ' b ( d) kl k B k Bk ( d) B ( d') d ' k 7.

22 American Universiy of Armenia Inroducion o Bioinformaics June 06 Baum-Welch Algorihm Iniializaion: Pick arbirary model parameers Recurrence: Se all he A and B variables o heir pseudocoun values r (or zero) For each seuence j,,n j Use forward algorihm o compue α (k) j Use backward algorihm o compue β (k) Add he conribuion of seuence j o A and B Compue he new model parameers Compue he new log likelihood of he model erminaion Sep erminaion: Sop when he change in he log likelihood is less han some predefined hreshold or he maximum number of ieraions is reached I can be shown ha he overall log likelihood is increased by he ieraion and ha he process converges o a local maximum. One of he challenges of designing HMMs: How good is ha local maximum? 7.