Dshnes casn as an HMM N = 2, ={F,L} M=2, O = {h,} A = F B= [. F L F L 0.95 0.0 0] h 0.5 0. L 0.05 0.90 0.5 0.9 c Deva ubramanan, 2009 63
A generave mdel fr CpG slands There are w hdden saes: CpG and nn-cpg. Each sae s characerzed by emssn prbables f he 4 bases. Yu can see whch sae he mdel s, nly he emed bases are vsble. CpG A: C: G: T: Nn-CpG A: C: G: T: Hdden sae Observables c Deva ubramanan, 2009 64
Flerng r he frward cmpuan Gven an HMM mdel A,B,p, and an bservan sequence, can we fnd he ms lely hdden sae a me,? : flerng Observan sequence: h h Wha s he hdden sae here F r L? c Deva ubramanan, 2009 65
Flerng cnd. [ 0] 0 2 3 4 5 6 h h 0.95 Fs h:0.5 :0.5 0.05 0. L h:0. :0.9 0.9 Wha s he dsrbun f? nce, s 0 =F, we can say ha 0 =[0.95 0.05], based n he ransn prbables alne. Bu s ha all we nw? c Deva ubramanan, 2009 66
Mre flerng [ 0] 0 2 3 4 5 6 h h 0.95 Fs 0.05 0. L 0.9 We have als bserved h a me. Hw can we fld n n he assessmen f he dsrbun f? h:0.5 :0.5 h:0. :0.9 c Deva ubramanan, 2009 67
c Deva ubramanan, 2009 68 Flerng cnd. 0.0.05 0.05 0.50.95 0.95 L h h L F h h F 0.0.05 0.50.95 Therefre, =[0.99 0.0]
c Deva ubramanan, 2009 69 Flerng cmpuan - [p -p] F L......, s s s Recursvely cmpued
c Deva ubramanan, 2009 70 ummary: flerng n a b n s c T n n, Termnan :, Recursn :, Inalze :.,,..., Defne.,...,,,..., Fnd T, 0 0 Tme cmplexy On 2 T
mhng/pserr decdng 0 2 3 4 5 6 h h Quesn: can we re-esmae he dsrbun a where <, usng nfrman abu he bserved sequence up me? Tha s, wha s? c Deva ubramanan, 2009 7
c Deva ubramanan, 2009 72 Bacward cmpuan,...,,...,,..., c Frward cmpuan, Recursn :., Inalze :.,..., Defne, T N N T b a c N s Bacward cmpuan Tme cmplexy: On 2 T
serr decdng,..., c c Deva ubramanan, 2009 73
Full Decdng Gven HMM mdel A,B,p, and an bservan sequence, can we fnd he ms lely hdden sae sequence s s? argmax_{s s } s s c Deva ubramanan, 2009 74
c Deva ubramanan, 2009 75 The Verb algrhm n T s s b a n s s, 0,...,, max Recursn :, Inalze :,...,,,,..., max Cmpuanal cmplexy = OTn 2
Learnng an HMM: case Gven bservan sequences, and he crrespndng hdden sae sequences, can we fnd he ms lely mdel A,B,p whch generaed? F F F L L F F h h Tranng daa c Deva ubramanan, 2009 76
arameer esman Inal sae dsrbun Fracn f mes sae s sae n ranng daa Transn prbables a = number f ransns frm /number f ransns frm Emssn prbables b = number f mes s emed n sae /number f mes sae ccurs c Deva ubramanan, 2009 77
Learnng an HMM: case 2 Gven us he bservan sequences, can we fnd he ms lely mdel = A,B,p whch generaed? argmax... Annaed ranng daa s dffcul ge; s we wuld le derve mdel parameers frm bservable sequences. c Deva ubramanan, 2009 78
The EM algrhm. Guess a mdel 2. Use bservan sequence esmae ransn prbables, emssn prbables, and nal sae prbables. 3. Updae mdel 4. Repea 2 and 3 ll n change n mdel c Deva ubramanan, 2009 79
Re-esmang parameers Wha s he prbably f beng n sae a me and mvng sae, gven he curren mdel and he bservan sequence O?,, O, c Deva ubramanan, 2009 80
c Deva ubramanan, 2009 8 Usng frward and bacward cmpuan n n b a b a, O O +
Re-esmang a The ransn prbables a can be re-esmaed as fllws aˆ T T n ',, ' c Deva ubramanan, 2009 82
Inal sae prbables N, Expeced number f mes n sae Inal sae prbables are smply c Deva ubramanan, 2009 83
Emssn prbables ˆ b expeced number f mes n sae and bserve symbl expeced number f mes n sae bˆ T T c Deva ubramanan, 2009 84
The EM algrhm. Guess a mdel a, b, 2. Use bservan sequence esmae, and 3. Use hese esmaes recalculae ' a', b', ' 4. Repea 2 and 3 ll n change n mdel c Deva ubramanan, 2009 85
ummary f CpG sland HMM Gven a DNA regn x, Verb decdng predcs lcans f CpG slands n. Gven a nuclede x, Verb decdng ells wheher x s n a CpG sland n he ms lely sequence. serr decdng can assgn lcally pmal predcns f CpG slands. A fully annaed ranng daa se can be used esmae he generang HMM. Even whu annans, we can use he EM prcedure derve mdel parameers. c Deva ubramanan, 2009 86
Hw desgn an HMM fr a new prblem Archecure/plgy desgn: Wha are he saes, bservan symbls, and he plgy f he sae ransn graph? Learnng/Tranng: Fully annaed r parally annaed ranng daases arameer esman by maxmum lelhd r by EM Valdan/Tesng: Fully annaed esng daases erfrmance evaluan accuracy, specfcy and sensvy c Deva ubramanan, 2009 87
HMM mdel srucure Duran mdelng 0.95 0. 9 Fs h:0.5 :0.5 0.05 0. L h:0. :0.9 0.9 Wha s he prbably f sayng wh he far cn fr T me seps? c Deva ubramanan, 2009 88
Inheren lman f HMMs The duran n sae F fllws an expnenally decayng dsrbun called a gemerc dsrbun. X T F 0.95 T 0.05 The gemerc dsrbun gves much prbably shr sequences f Fs and Ls and lle medum and lng sequences f Fs and Ls. c Deva ubramanan, 2009 89
Duran mdelng T ban nn-gemerc lengh dsrbuns, we use an array f n F saes, as fllws: pp p p p p p F F F n=3 L n Ln n X L p p Generaed lengh dsrbun s a negave bnmal. c Deva ubramanan, 2009 90
Why des hs maer? Exn Inrn Gemerc ds Exn lengh ds L Lengh f say n Exn sae deermnes lengh f predced exns. Very shr exns are rare. mlarly fr nrns. Inrns shrer han 30 bp d n exs. c Deva ubramanan, 2009 9
Lengh dsrbuns f exns and nrns c Deva ubramanan, 2009 92
Generalzed HMMs sem-marv HMMs Each sae has a specfed lengh dsrbun. E I N self-ransns Exn Inrn generae exra symbls c a sae sar a =. Repea c he lengh f say d n curren sae frm dsrbun. Em d symbls n curren sae. c a new sae accrdng a marx and ransn a me +d c Deva ubramanan, 2009 93
Example Mulple symbls emed n each sae. One ne mappng beween symbls and hdden saes s ls n he generalzed HMM. c Deva ubramanan, 2009 94
c Deva ubramanan, 2009 95 Verb algrhm fr ghmms Jus le Verb fr HMMs, bu we use he enre say n sae nsead f a sae a a gven me. - s s --,, max max 0,, 0.. a l b f f r r rbably f ms lely pah endng a wh say f + n sae fllwng a say n sae