In this Chapter. Chap. 3 Markov chains and hidden Markov models. Probabilistic Models. Example: CpG Islands

Transcription

1 In ths Chpter Chp. 3 Mrov chns nd hdden Mrov models Bontellgence bortory School of Computer Sc. & Eng. Seoul Ntonl Unversty Seoul 5-74, Kore The probblstc model for sequence nlyss HMM (hdden Mrov model) or, ts smpler verson, Mrov model Does ths sequence belong to prtculr fmly? Assumng the sequence does come from some fmly, wht cn we sy bout ts nternl structure? Identfy lph hel or bet sheet regons n proten sequence. Ths slde fle s vlble onlne t Copyrght (c) 00 by SNU CSE Bontellgence b Copyrght (c) 00 by SNU CSE Bontellgence b Probblstc Models Emple: CpG Islnds Quntzed speech sgnl A C C G AA G Bologcl sequences Probblstc model - Word recognton - nguge understndng -Gene fndng - Gene/Proten fmles In the humn genome, CpG dnucleotdes re rrer thn would be epected from the ndependent probbltes of C nd G becuse of bologcl resons. C methylton T In the promoters or strt regons of mny genes, there mght est mny more CpGs thn elsewhere n the genome. C-methylton s usully suppressed n such res. These regons re clled CpG slnds ( few hundred to few thousnd bses long). Copyrght (c) 00 by SNU CSE Bontellgence b 3 Copyrght (c) 00 by SNU CSE Bontellgence b 4

2 CpG Islnds (Cont d) Two rsed questons Gven short stretch of genomc sequence, how would we decde f t comes from CpG slnd or not? Gven long pece of sequence, how would we fnd the CpG slnds n t, f there re ny? Mrov Chns Wht sort of model mght we use for CpG slnd regons? Dnucleotdes re mportnt. model n whch the probblty of symbol depends on the prevous symbol. Mrov chns (the smplest one) A collecton of sttes represented s node or verte Ech stte corresponds to prtculr resdue. Arrows between sttes. represents the trnston from one stte to nother. Copyrght (c) 00 by SNU CSE Bontellgence b 5 Copyrght (c) 00 by SNU CSE Bontellgence b 6 Mrov Chns for DNA Sequences Mrov Propertes Trnston probbltes st t s) The probblty of the sequence ),...,,...,,...,, ), ), )... ) Chn rule By the Mrov propertes,...,, ) ) ),...,, ) ) ) )... ) Trnston probbltes Copyrght (c) 00 by SNU CSE Bontellgence b 7 Copyrght (c) 00 by SNU CSE Bontellgence b 8

3 Modelng the Begnnng nd End of Sequences Mrov Chns for Dscrmnton Etr begn sttes Etr end stte Rel dt for the CpG slnd emple From set of humn DNA sequences, totl of 48 puttve CpG slnds were etrcted. Two Mrov chn models for the CpG slnd emple were derved. + model nd - model Trnston probbltes re estmted from dt s follows: s) Bs E t) te Modelng the length of sequences c + + st st c + t' st'. Mmum lelhood estmtors Copyrght (c) 00 by SNU CSE Bontellgence b 9 Copyrght (c) 00 by SNU CSE Bontellgence b 0 Mrov Chns for Dscrmnton (Cont d) elhood Rto Test Two resultng tbles Clculte the log-odds rto model + ) S( ) log model ) β + A tble for β (log lelhood rto) log lelhood Ech row sums to one. The tble s symmetrc. Copyrght (c) 00 by SNU CSE Bontellgence b Copyrght (c) 00 by SNU CSE Bontellgence b

4 The Dstrbuton of Scores, S() Etensons to the Prevous Model - Normlzed by the length of the sequence - Dr gry: +, lght gry: - - Resons of errors: ndequte models or mslbelng of the trnng sequences Copyrght (c) 00 by SNU CSE Bontellgence b 3 The second queston: Gven long pece of sequence, how would we fnd the CpG slnds n t, f there re ny? Use of Mrov chns Clculte the log-odds score for wndow of 00 nucleotdes round every nucleotde n the sequence. CpG slnds wll stnd out wth postve vlues. ndequte Wht f the CpG slnds hve shrp boundres. Why the wndow sze of 00? We need the more stsfctory model for ths queston. Copyrght (c) 00 by SNU CSE Bontellgence b 4 Etensons to the Prevous Model (Cont d) Smulte n one model the slnds n se of nonslnd genomc sequences Both the Mrov chns n one model smll It s no longer possble to tell wht stte the model ws n when ws generted just by loong t. hdden Mrov models Hdden Mrov Models Trnston probbltes of the sttes (Mrov propertes) l π l π ) Emsson probbltes of the symbols (genertve models) e ( b) b π ) Emple: the occsonlly dshonest csno Hdden sttes whch de The jont probblty of sequence nd pth π P (, π ) 0π e π ( ) π π + Copyrght (c) 00 by SNU CSE Bontellgence b 5 Copyrght (c) 00 by SNU CSE Bontellgence b 6

5 Emple: the Jont Probblty n CpG Islnds The probblty of sequence CGCG beng emtted by the stte (C +, G -, C -, G + ) n the model s 0, C G+ C G G C C G 0. In generl, we do not now the pth. estmte the pth. The most probble one the Vterb lgorthm Bsed on posteror dstrbuton over sttes The Underlyng Pth of Sttes Observed sequence decode the sequence of the underlyng sttes. From the speech recognton feld There my be mny stte sequences tht could gve rse to ny prtculr sequence of symbols. (C +, G +, C +, G + ), (C -, G -, C -, G - ), nd (C +, G -, C +, G - ) CGCG The most probble The lest probble Copyrght (c) 00 by SNU CSE Bontellgence b 7 Copyrght (c) 00 by SNU CSE Bontellgence b 8 The Most Probble Stte Pth Vterb Algorthm The most probble pth π * (one soluton) π * rgm π, π) Recursve constructon of the most probble pth The Vterb lgorthm v (): the probblty of the most probble pth endng n stte wth observton. v 0 (0) v l ( + ) e l ( + )m (v () l ) The logrthm of the probbltes s used becuse of computtonl resons. Copyrght (c) 00 by SNU CSE Bontellgence b 9 Copyrght (c) 00 by SNU CSE Bontellgence b 0

6 Emple: Vterb The model of CpG slnds (CGCG) Emple: Csno, Prt Generted rolls nd Vterb pth fndng Copyrght (c) 00 by SNU CSE Bontellgence b Copyrght (c) 00 by SNU CSE Bontellgence b The Probblty of Sequence The probblty of sequences Dscrmnton between CpG slnds nd non-slnd regons ) Σ π, π) Enumertng ll the possble π s mprctcl. One soluton s to use, π) nsted of ). Somewht strtlng but surprsngly good n mny cses. The vlue of ) cn lso be clculted by dynmc progrmmng slls. f ( ),...,, f ( + ) e ( l π l + ) ) f ( ) l Copyrght (c) 00 by SNU CSE Bontellgence b 3 The Forwrd Algorthm - The logrthm of the probbltes cn be used becuse of computtonl resons. - In ths cse, sclng of the probbltes s more pproprte. Copyrght (c) 00 by SNU CSE Bontellgence b 4

7 Another Queston The Bcwrd Algorthm The Vterb lgorthm fnd the most probble pth Forwrd lgorthm clculte the probblty of sequence We mght wnt to now wht the most probble stte s for n observton. The posteror probblty of stte t tme when the emtted sequence s nown. π ) Copyrght (c) 00 by SNU CSE Bontellgence b 5 The posteror probblty of stte t tme gven the observed sequence, π )., π )..., π ) f () Intlzton ( ):..., π ) , π ) π ) Copyrght (c) 00 by SNU CSE Bontellgence b 6 b () b () 0 for ll Recurson (,, ): b () Σ l l e l ( + )b l ( + ) Termnton: ) Σ l 0l e l ( )b l (), π ) π ) ) f ( ) b ( ) ) Emple: Csno, Prt 3 pp. 60, Fgure 3.6 Posteror Decodng When mny dfferent pths hve lmost the sme probblty s the most probble one. Another ppromton to the optml pth ˆ π rg m π ) Cn produce llegl pths. The posteror probblty of the functon g() t tme gven the observed sequence, G( ) π ) g( ). Copyrght (c) 00 by SNU CSE Bontellgence b 7 Copyrght (c) 00 by SNU CSE Bontellgence b 8

8 CpG Islnds Revsted Wht relly concerns us s whether bse s prt of n slnd or not. g() for {A +, C +, G +, T + } g() 0 for {A -, C -, G -, T - } G( ) s the posteror probblty ccordng to the model tht bse s n CpG slnd. Ths s not qute the most probble globl lbelng of gven sequence. Emple: Predcton of CpG Islnds By the Vterb decodng Flse negtves: Flse postves: 67 (by some post-processng) By posteror decodng Flse negtves: Flse postves: (by some post-processng) Some flse postves re rel CpG slnds. Flse negtves re perhps wrongly lbeled. It s possble tht more sophstcted model s needed. Copyrght (c) 00 by SNU CSE Bontellgence b 9 Copyrght (c) 00 by SNU CSE Bontellgence b 30 Emple: Csno, Prt 4 The model s chnged. The swtchng probblty from fr to loded 0.0 The Vterb decodng Never vsts the loded de stte. pp. 6, Fgure 3.7 Copyrght (c) 00 by SNU CSE Bontellgence b 3