Hidden Markov Model. a ij. Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,..., sn

Transcription

1 Hdden Mrkov Model S S servon : 2... Ses n me : 2... All ses : s s2... s

2 2 Hdden Mrkov Model Con d Dscree Mrkov Model 2 z k s s s s s s Degree Mrkov Model

3 Hdden Mrkov Model Con d : rnson roly from S o S s s 3

4 Dscree Mrkov Model Exmple S : he weher s rny S2 : he weher s cloudy S3 : he weher s sunny A { } rny cloudy sunny rny cloudy sunny 4

5 Hdden Mrkov Model Exmple Con d Queson :How much s hs proly: Sunny-Sunny-Sunny-Rny-Rny-Sunny-Cloudy-Cloudy s s s s s s s s

6 Hdden Mrkov Model Exmple Con d he proly of eng n se n me = s Queson 2:he proly of syng n se S for d dys f we re n se S? s s s s d d d Dys 6

7 Dscree Densy HMM Componens : umer f Ses M : umer f upus A x : Se rnson roly Mrx B xm: upu ccurrence roly n ech se x: Inl Se roly A B : Se of HMM rmeers 7

8 hree Bsc HMM rolems Recognon rolem: Gven n HMM nd seuence of oservons wh s he proly? Se Decodng rolem: Gven model nd seuence of oservons wh s he mos lkely se seuence n he model h produced he oservons? rnng rolem: Gven model nd seuence of oservons how should we dus model prmeers n order o mxmze? 8

9 9 Frs rolem Soluon y y x y x z y z y x z y x We Know h: And

10 0 Frs rolem Soluon Con d Compuon rder : 2

11 Forwrd Bckwrd Approch 2 Compung Inlzon

12 Forwrd Bckwrd Approch Con d 2 Inducon : [ ] 3 ermnon : Compuon rder : 2 2

13 3 Bckwrd Vrle 2 Inlzon 2Inducon nd 2

14 4 Second rolem Soluon Fndng he mos lkely se seuence Indvdully mos lkely se : ] rg mx[ *

15 5 Ver Algorhm Defne : ] [ mx s he mos lkely se seuence wh hs condons : se me nd oservon o

16 Ver Algorhm Con d [mx ]. Inlzon 0 Is he mos lkely se efore se me - 6

17 7 Ver Algorhm Con d 2 ] rg mx[ ] mx[ 2 Recurson

18 Ver Algorhm Con d 3 ermnon: p * * 4Bckrckng: * mx[ rg mx[ ] ] * 2 8

19 9 hrd rolem Soluon rmeers Esmon usng Bum- Welch r Expecon Mxmzon EM Approch Defne:

20 hrd rolem Soluon Con d : Expeced vlue of he numer of umps from se : Expeced vlue of he numer of umps from se o se 20

21 2 hrd rolem Soluon Con d V o k k

22 22 Bum Auxlry Funcon Q 'log ' ' : ' Q Q f By hs pproch we wll rech o locl opmum

23 Resrcons f Reesmon Formuls M k k 23

24 Connuous servon Densy We hve mouns of DF nsed of k V k We hve M k C k k k d Mxure Coeffcens Averge Vrnce 24

25 Connuous servon Densy Mxure n HMM M M2 M2 M22 M3 M23 M3 M4 M32 M42 M33 M43 S S2 S3 Domnn Mxure: MxC k k k k 25

26 Connuous servon Densy Con d Model rmeers: A C M M K M K K : umer f Ses M : umer f Mxures In Ech Se K : Dmenson f servon Vecor 26

27 27 Connuous servon Densy Con d M k k k k C k k o k

28 28 Connuous servon Densy Con d k k k k o o k k roly of even h se nd k h mxure me

29 Se Duron Modelng S S roly of syng d mes n se : d d 29

30 Se Duron Modelng Con d HMM Wh cler duron d d. S. S 30

31 Se Duron Modelng Con d HMM consderon wh Se Duron : Selecng usng s Selecng d usng d Selecng servon Seuence 2 d usng 2 d n prcce we ssume he followng ndependence: 2 d Selecng nex se 2 usng rnson proles. We lso hve n ddonl 2 consrn: 0 d 3

32 rnng In HMM Mxmum Lkelhood ML Mxmum Muul Informon MMI Mnmum Dscrmnon Informon MDI 32

33 rnng In HMM Mxmum Lkelhood ML o o 2 o 3... o n * Mxmum[ r servon Seuence V 33 ]

34 34 rnng In HMM Con d Mxmum Muul Informon MMI log I v w w v w w I log log Muul Informon } { v

35 rnng In HMM Con d Mnmum Dscrmnon Informon MDI servon : 2 Auo correlon : R R R2 R R nf I Q : Q R I Q : olog o do o 35