Hidden Markov Models

Transcription

1 Machne Learnng for Sgnal Processng Hdden Markov Models Class Oc

2 Admnsrva HW2 due Tuesday Is everyone on he projecs page? Where are your projec proposals? 2

3 Recap: Wha s an HMM Probablsc funcon of a markov chan Models a dynamcal sysem Sysem goes hrough a number of saes Followng a Markov chan model On arrvng a any sae generaes observaons accordng o a sae-specfc specfc probably dsrbuon 3

4 A Though Expermen I jus called ou he 6 from he blue guy.. goa swch o paern 2.. Two shooers roll dce A caller calls ou he number rolled. We only ge o hear wha he calls ou The caller behaves randomly If he has jus called a number rolled by he blue shooer, hs nex call s ha of he red shooer 70% of he me Bu f he has jus called he red shooer, he has only a 40% probably of callng he red shooer agan n he nex call How do we characerze hs? 12 Oc /18797

5 A Though Expermen P(X blue e P(X red d The dos and arrows represen he saes of he caller When he s on he blue crcle he calls ou he blue dce When he s on he red crcle he calls ou he red dce The hsograms represen he probably dsrbuon of he numbers for he blue and red dce 5

6 A Though Expermen P(X blue e P(X red d When he caller s n any sae, he calls a number based on he probably dsrbuon of ha sae We call hese sae oupu dsrbuons A each sep, he moves from hs curren sae o anoher sae followng a probably dsrbuon We call hese ranson probables The caller s an HMM!!! 12 Oc /18797

7 Wha s an HMM HMMs are sascal models for (causal processes The model assumes ha he process canbenoneofa of a number of saes a any me nsan The sae of he process a any me nsan depends only on he sae a he prevous nsan (causaly, Markovan A each nsan he process generaes an observaon from a probably dsrbuon ha s specfc o he curren sae The generaed observaons are all ha we ge o see he acual sae of he process s no drecly observable Hence he qualfer hdden 7

8 Hdden Markov Models A Hdden Markov Model consss of wo componens A sae/ranson backbone ha specfes how many saes here are, and how hey can follow one anoher A se of probably dsrbuons, one for each sae, whch specfes he dsrbuon of all vecors n ha sae Markov chan Daa dsrbuons Ths can be facored no wo separae probablsc enes A probablsc Markov chan wh saes and ransons A se of daa probably dsrbuons, assocaed wh he saes 11755/18797

9 How an HMM models a process HMM assumed o be generang daa sae sequence sae dsrbuons observaon sequence 9

10 HMM Parameers The opology of he HMM Number of saes and allowed ransons E.g. here we have 3 saes and canno go from he blue sae o he red The ranson probables Ofen represened as a marx as here T j s he probably ha when n sae, he process wll move o j The probably of begnnng T a any sae s The complee se s represened as 12 Oc The 2010 sae oupu dsrbuons 11755/

11 HMM sae oupu dsrbuons The sae oupu dsrbuon s he dsrbuon of daa produced from any sae Typcally modelled as Gaussan Typcally modelled as Gaussan ( 0.5( 1 2 1, ; ( ( T x x d e x Gaussan s x P The paremeers are and More ypcally, modelled as Gaussan mxures 1, ; ( ( K j j j x Gaussan w s x P Oher dsrbuons may also be used E.g. hsograms n he dce case 0,,,, ; ( ( j j j j x Gaussan w s x P.g. sog a s e d ce case 12 Oc /18797

12 The Dagonal Covarance Marx Full covarance: all elemens are non-zero Dagonal covarance: off-dagonal elemens are zero -0.5(x- T -1 (x- (x - 2 / 2 2 For GMMs s frequenly assumed ha he feaure vecor dmensons are all ndependen of each oher Resul: The covarance marx s reduced o a dagonal form The deermnan of he dagonal marx s easy o compue 12

13 Three Basc HMM Problems Wha s he probably ha wll generae a specfc observaon sequence Gven a observaon sequence, how do we deermne whch observaon was generaed from whch sae The sae segmenaon problem How do we learn he parameers of he HMM from observaon sequences 13

14 Compung he Probably of an Observaon Sequence Two aspecs o producng he observaon: Progressng hrough a sequence of saes Producng observaons from hese saes 14

15 Progressng hrough saes HMM assumed o be generang daa sae sequence The process begns a some sae (red here From ha sae, makes an allowed ranson To arrve a he same or any oher sae From ha sae makes anoher allowed ranson And so on 15

16 Probably ha he HMM wll follow a parcular sae sequence P ( s, s, s,... P ( s P ( s s P ( s s P(s( 1 s he probably ha he process wll nally be n sae s 1 P(s s s he ranson probably bl of movng o sae s a he nex me nsan when he sysem s currenly n s Also denoed by T j earler 16

17 Generang Observaons from Saes HMM assumed o be generang daa sae sequence sae dsrbuons observaon sequence A each me generaes an observaon from he sae s n a ha me 17

18 Probably ha he HMM wll generae a parcular observaon sequence gven a sae sequence (sae sequence known P( o, o, o,... s, s, s,... P( os P( o s P( o s Compued from he Gaussan or Gaussan mxure for sae s 1 P(o s s he probably of generang observaon o when he sysem s n sae s 18

19 Proceedng hrough Saes and Producng Observaons HMM assumed o be generang daa sae sequence sae dsrbuons observaon sequence A each me produces an observaon and makes a ranson 19

20 Probably ha he HMM wll generae a parcular sae sequence and from, a parcular observaon sequence P ( o, o, o,..., s, s, s, P( o1, o2, o3,... s1, s2, s3,... P( s1, s2, s3,... P( os 1 1 P( o2 s2 P( o3 s3... P( s1 P( s2 s1 P( s3 s

21 Probably of Generang an Observaon Sequence The precse sae sequence s no known All possble sae sequences mus be consdered P ( o, o, o, all. possble sae. sequences P ( o, o, o,...,, s, s, s, all. possble sae. sequences P( os P( o s P( o s... P( s P( s s P( s s

22 Compung Effcenly Explc summng over all sae sequences s no racable A very large number of possble sae sequences Insead we use he forward algorhm A dynamc programmng echnque. 22

23 Illusrave Example Example: a generc HMM wh 5 saes and a ermnang sae. Lef o rgh opology P(s = 1 for sae 1 and 0 for ohers The arrows represen ranson for whch he probably s no 0 Noaon: P(s s = T j We represen P(o s = b ( for brevy 23

24 Dverson: The Trells Sae ndex s (s, -1 Feaure vecors (me The rells s a graphcal represenaon of all possble pahs hrough he HMM o produce a gven observaon The Y-axs represens HMM saes, X axs represens observaons Every edge n he graph represens a vald ranson n he HMM over a sngle me sep Every node represens he even of a parcular observaon beng generaed from a parcular sae 24

25 The Forward Algorhm ( s, P( x1, x2,..., x, sae( s ndex Sae n s (s, -1 me (s, s he oal probably of ALL sae sequences ha end a sae s a me, and all observaons unl x 25

26 The Forward Algorhm ( s, P( x1, x2,..., x, sae( s Sae n ndex (s,-1 s (s, Can be recursvely esmaed sarng from he frs me nsan (forward recurson (1,-1-1 ( s, ( s', 1 P ( s s' P ( x s s' me (s, can be recursvely compued n erms of (s,, ( he forward probables bl a me -1 26

27 The Forward Algorhm Toalprob ( s, T s ndex Sae n T me In he fnal observaon he alpha a each sae gves he probably of all sae sequences endng a ha sae General model: The oal probably of he observaon s he sum of he alpha values a all saes 27

28 The absorbng sae Observaon sequences are assumed o end only when he process arrves a an absorbng b sae No observaons are produced from he absorbng sae 28

29 The Forward Algorhm Toalprob ( s, T 1 absorbng ndex Sae n me ( s b absorbng, T 1 ( s', T P ( sabsorbng b s' s' Absorbng sae model: The oal probably s he alpha compued a he absorbng sae afer he fnal observaon 29 T

30 Problem 2: Sae segmenaon Gven only a sequence of observaons, how do we deermne whch h sequence of saes was followed n producng? 30

31 The HMM as a generaor HMM assumed o be generang daa sae sequence sae dsrbuons observaon sequence The process goes hrough a seres of saes and produces observaons from hem 31

32 Saes are hdden HMM assumed o be generang daa sae sequence sae dsrbuons observaon sequence The observaons do no reveal he underlyng sae 32

33 The sae segmenaon problem HMM assumed o be generang daa sae sequence sae dsrbuons observaon sequence Sae segmenaon: Esmae sae sequence gven observaons 33

34 Esmang he Sae Sequence Many dfferen sae sequences are capable of producng he observaon o Soluon: Idenfy he mos probable sae sequence The sae sequence for whch he probably of progressng hrough ha sequence and generang he observaon sequence s maxmum P( o, o, o,..., s, s, s,....e s maxmum

35 Esmang he sae sequence Once agan, exhausve evaluaon s mpossbly expensve Bu once agan a smple dynamc-programmng soluon s avalable P( o, o, o,..., s, s, s, P( os 1 1 P( o2 s2 P( o3 s3... P( s1 P( s2 s1 P( s3 s2... Needed: arg maxs s, s,... P ( o1 s1 P ( s1 P ( o2 s2 P ( s2 s1 P ( o3 s3 P ( s3 2, s

36 Esmang he sae sequence Once agan, exhausve evaluaon s mpossbly expensve Bu once agan a smple dynamc-programmng soluon s avalable P( o, o, o,..., s, s, s, P( os 1 1 P( o2 s2 P( o3 s3... P( s1 P( s2 s1 P( s3 s2... Needed: arg maxs s, s,... P ( o1 s1 P ( s1 P ( o2 s2 P ( s2 s1 P ( o3 s3 P ( s3 2, s

37 The HMM as a generaor HMM assumed o be generang daa sae sequence sae dsrbuons observaon sequence Each enclosed erm represens one forward ranson and a subsequen emsson 37

38 The sae sequence The probably of a sae sequence?,?,?,?,s x,s y endng d d ll b l a me, and producng all observaons unl o P(o 1..-1,?,?,?,?, s x, o,s y = P(o 1..-1,?,?,?,?, s x P(o s y P(s y s x The bes sae sequence ha ends wh s x,s y a wll have a probably equal o he probably of he bes sae sequence endng a -1 a s x mes P(o s y P(s y s x 38

39 Exendng he sae sequence sae sequence s x s y sae dsrbuons b observaon sequence The probably of a sae sequence?,?,?,?,s x,s y endng a me and producng observaons unl o P(o 1..-1,o,?,?,?,?, s x,s y = P(o 1..-1,?,?,?,?, s x P(o s y P(s y s x 39

40 Trells The graph below shows he se of all possble sae sequences hrough hs HMM n fve me nsans me 12 Oc /18797

41 The cos of exendng a sae sequence The cos of exendng a sae sequence endng a s x s only dependen on he ranson from s x o s y, and he observaon probably a s y P(o s y P(s y s x s y s x me 12 Oc /18797

42 The cos of exendng a sae sequence The bes pah o s y hrough s x s smply an exenson of he bes pah o s xbesp(o1..-1,?,?,?,?, s x P(o s y P(s y s x s y s x me 12 Oc /18797

43 The Recurson The overall bes pah o s y s an exenson of he bes pah o one of he saes a he prevous me s y me 12 Oc /18797

44 The Recurson Prob. of bes pah o s y = Max s???? x BesP(o 1..-1,?,?,?,?, s x P(o s y P(s y s x s y me 12 Oc /18797

45 Fndng he bes sae sequence The smple algorhm jus presened s called he VITERBI algorhm n he leraure Afer A.J.Verb, who nvened hs dynamc programmng algorhm for a compleely dfferen purpose: decodng error correcon codes! 45

46 Verb Search (cond. Inal sae nalzed wh pah-score = P(s 1 b 1 (1 me All oher saes have score 0 snce P(s = 0 for hem 12 Oc /18797

47 Verb Search (cond. P ( j Sae wh bes pah-score Sae wh pah-score < bes Sae whou a vald pah-score = max [P (-1 j b j (] Sae ranson probably, o j Score for sae j, gven he npu a me Toal pah-score endng up a sae j a me me 47

48 Verb Search (cond. P ( j = max [P (-1 j b j (] Sae ranson probably, o j Score for sae j, gven he npu a me Toal pah-score endng up a sae j a me me 48

49 Verb Search (cond. me 49

50 Verb Search (cond. me 50

51 Verb Search (cond. me 51

52 Verb Search (cond. me 52

53 Verb Search (cond. me 53

54 Verb Search (cond. me 54

55 Verb Search (cond. me 55

56 Verb Search (cond. THE BEST STATE SEQUENCE IS THE ESTIMATE OF THE STATE SEQUENCE FOLLOWED IN GENERATING THE OBSERVATION me 56

57 Problem3: Tranng HMM parameers We can compue he probably of an observaon, and he bes sae sequence gven an observaon, usng he HMM s parameers Bu where do he HMM parameers come from? They mus be learned from a collecon of observaon sequences 57

58 Learnng HMM parameers: Smple procedure counng Gven a se of ranng nsances Ieravely: 1. Inalze HMM parameers 2. Segmen all ranng nsances 3. Esmae ranson probables and sae oupu probably parameers by counng 58

59 Learnng by counng example Explanaon by example n nex few sldes 2-sae HMM, Gaussan PDF a saes, 3 observaon sequences Example shows ONE eraon How o coun afer sae sequences are obaned 59

60 Example: Learnng HMM Parameers We have an HMM wh wo saes s1 and s2. Observaons are vecors x j -h sequence, j-h vecor We are gven he followng hree observaon sequences And have already esmaed sae sequences Observaon 1 Tme sae S1 S1 S2 S2 S2 S1 S1 S2 S1 S1 Obs X a1 X a2 X a3 X a4 X a5 X a6 X a7 X a8 X a9 X a10 Observaon 2 Tme sae S2 S2 S1 S1 S2 S2 S2 S2 S1 Obs X b1 X b2 X b3 X b4 X b5 X b6 X b7 X b8 X b9 Observaon 3 Tme sae S1 S2 S1 S1 S1 S2 S2 S2 Obs X c1 X c2 X c3 X c4 X c5 X c6 X c7 X c8 60

61 Example: Learnng HMM Parameers Inal sae probables (usually denoed as : We have 3 observaons 2 of hese begn wh S1, and one wh S2 (S1 = 2/3, (S2 = 1/3 Observaon 1 Tme sae S1 S1 S2 S2 S2 S1 S1 S2 S1 S1 Obs X a1 X a2 X a3 X a4 X a5 X a6 X a7 X a8 X a9 X a10 Observaon 2 Tme sae S2 S2 S1 S1 S2 S2 S2 S2 S1 Obs X b1 X b2 X b3 X b4 X b5 X b6 X b7 X b8 X b9 Observaon 3 Tme sae S1 S2 S1 S1 S1 S2 S2 S2 Obs X c1 X c2 X c3 X c4 X c5 X c6 X c7 X c8 61

62 Example: Learnng HMM Parameers Transon probables: Sae S1 occurs 11 mes n non-ermnal locaons Of hese, s followed by S1 X mes I s followed by S2 Y mes P(S1 S1 = x/ 11; P(S2 S1 = y / 11 Observaon 1 Tme sae S1 S1 S2 S2 S2 S1 S1 S2 S1 S1 Obs X a1 X a2 X a3 X a4 X a5 X a6 X a7 X a8 X a9 X a10 Observaon 2 Tme sae S2 S2 S1 S1 S2 S2 S2 S2 S1 Obs X b1 X b2 X b3 X b4 X b5 X b6 X b7 X b8 X b9 Observaon 3 Tme sae S1 S2 S1 S1 S1 S2 S2 S2 Obs X c1 X c2 X c3 X c4 X c5 X c6 X c7 X c8 62

63 Example: Learnng HMM Parameers Transon probables: Sae S1 occurs 11 mes n non-ermnal locaons Of hese, s followed mmedaely by S1 6 mes I s followed by S2 Y mes P(S1 S1 = x/ 11; P(S2 S1 = y / 11 Observaon 1 Tme sae S1 S1 S2 S2 S2 S1 S1 S2 S1 S1 Obs X a1 X a2 X a3 X a4 X a5 X a6 X a7 X a8 X a9 X a10 Observaon 2 Tme sae S2 S2 S1 S1 S2 S2 S2 S2 S1 Obs X b1 X b2 X b3 X b4 X b5 X b6 X b7 X b8 X b9 Observaon 3 Tme sae S1 S2 S1 S1 S1 S2 S2 S2 Obs X c1 X c2 X c3 X c4 X c5 X c6 X c7 X c8 63

64 Example: Learnng HMM Parameers Transon probables: Sae S1 occurs 11 mes n non-ermnal locaons Of hese, s followed mmedaely by S1 6 mes I s followed mmedaely by S2 5 mes P(S1 S1 = x/ 11; P(S2 S1 = y / 11 Observaon 1 Tme sae S1 S1 S2 S2 S2 S1 S1 S2 S1 S1 Obs X a1 X a2 X a3 X a4 X a5 X a6 X a7 X a8 X a9 X a10 Observaon 2 Tme sae S2 S2 S1 S1 S2 S2 S2 S2 S1 Obs X b1 X b2 X b3 X b4 X b5 X b6 X b7 X b8 X b9 Observaon 3 Tme sae S1 S2 S1 S1 S1 S2 S2 S2 Obs X c1 X c2 X c3 X c4 X c5 X c6 X c7 X c8 64

65 Example: Learnng HMM Parameers Transon probables: Sae S1 occurs 11 mes n non-ermnal locaons Of hese, s followed mmedaely by S1 6 mes I s followed mmedaely by S2 5 mes P(S1 S1 = 6/ 11; P(S2 S1 = 5 / 11 Observaon 1 Tme sae S1 S1 S2 S2 S2 S1 S1 S2 S1 S1 Obs X a1 X a2 X a3 X a4 X a5 X a6 X a7 X a8 X a9 X a10 Observaon 2 Tme sae S2 S2 S1 S1 S2 S2 S2 S2 S1 Obs X b1 X b2 X b3 X b4 X b5 X b6 X b7 X b8 X b9 Observaon 3 Tme sae S1 S2 S1 S1 S1 S2 S2 S2 Obs X c1 X c2 X c3 X c4 X c5 X c6 X c7 X c8 65

66 Example: Learnng HMM Parameers Transon probables: Sae S2 occurs 13 mes n non-ermnal locaons Of hese, s followed mmedaely by S1 6 mes I s followed mmedaely by S2 5 mes P(S1 S1 = 6/ 11; P(S2 S1 = 5 / 11 Observaon 1 Tme sae S1 S1 S2 S2 S2 S1 S1 S2 S1 S1 Obs. X a1 X a2 X a3 X a4 X a5 X a6 X a7 X a8 X a9 X a10 Observaon 2 Tme sae S2 S2 S1 S1 S2 S2 S2 S2 S1 Obs X b1 X b2 X b3 X b4 X b5 X b6 X b7 X b8 X b9 Observaon 3 Tme sae S1 S2 S1 S1 S1 S2 S2 S2 Obs X c1 X c2 X c3 X c4 X c5 X c6 X c7 X c8 66

67 Example: Learnng HMM Parameers Transon probables: Sae S2 occurs 13 mes n non-ermnal locaons Of hese, s followed mmedaely by S1 5 mes I s followed mmedaely by S2 5 mes P(S1 S1 = 6/ 11; P(S2 S1 = 5 / 11 Observaon 1 Tme sae S1 S1 S2 S2 S2 S1 S1 S2 S1 S1 Obs X a1 X a2 X a3 X a4 X a5 X a6 X a7 X a8 X a9 X a10 Observaon 2 Tme sae S2 S2 S1 S1 S2 S2 S2 S2 S1 Obs X b1 X b2 X b3 X b4 X b5 X b6 X b7 X b8 X b9 Observaon 3 Tme sae S1 S2 S1 S1 S1 S2 S2 S2 Obs X c1 X c2 X c3 X c4 X c5 X c6 X c7 X c8 67

68 Example: Learnng HMM Parameers Transon probables: Sae S2 occurs 13 mes n non-ermnal locaons Of hese, s followed mmedaely by S1 5 mes I s followed mmedaely by S2 8 mes P(S1 S1 = 6/ 11; P(S2 S1 = 5 / 11 Observaon 1 Tme sae S1 S1 S2 S2 S2 S1 S1 S2 S1 S1 Obs X a1 X a2 X a3 X a4 X a5 X a6 X a7 X a8 X a9 X a10 Observaon 2 Tme sae S2 S2 S1 S1 S2 S2 S2 S2 S1 Obs X b1 X b2 X b3 X b4 X b5 X b6 X b7 X b8 X b9 Observaon 3 Tme sae S1 S2 S1 S1 S1 S2 S2 S2 Obs X c1 X c2 X c3 X c4 X c5 X c6 X c7 X c8 68

69 Example: Learnng HMM Parameers Transon probables: Sae S2 occurs 13 mes n non-ermnal locaons Of hese, s followed mmedaely by S1 5 mes I s followed mmedaely by S2 8 mes P(S1 S2 = 5 / 13; P(S2 S2 = 8 / 13 Observaon 1 Tme sae S1 S1 S2 S2 S2 S1 S1 S2 S1 S1 Obs X a1 X a2 X a3 X a4 X a5 X a6 X a7 X a8 X a9 X a10 Observaon 2 Tme sae S2 S2 S1 S1 S2 S2 S2 S2 S1 Obs X b1 X b2 X b3 X b4 X b5 X b6 X b7 X b8 X b9 Observaon 3 Tme sae S1 S2 S1 S1 S1 S2 S2 S2 Obs X c1 X c2 X c3 X c4 X c5 X c6 X c7 X c8 69

70 Parameers learn so far Sae nal probables, ofen denoed as (S1 = 2/3 = 0.66 (S2 = 1/3 = 0.33 Sae ranson probables P(S1 S1 = 6/11 = 0.545; P(S2 S1 = 5/11 = P(S1 S2 = 5/13 = 0.385; P(S2 S2 = 8/13 = Represened as a ranson marx A P ( S 1 S 1 P ( S 2 S P( S1 S2 P( S2 S Each row of hs marx mus sum o

71 Example: Learnng HMM Parameers Sae oupu probably for S1 There are 13 observaons n S1 Observaon 1 Tme sae S1 S1 S2 S2 S2 S1 S1 S2 S1 S1 Obs X a1 X a2 X a3 X a4 X a5 X a6 X a7 X a8 X a9 X a10 Observaon 2 Tme sae S2 S2 S1 S1 S2 S2 S2 S2 S1 Obs X b1 X b2 X b3 X b4 X b5 X b6 X b7 X b8 X b9 Observaon 3 Tme sae S1 S2 S1 S1 S1 S2 S2 S2 Obs X c1 X c2 X c3 X c4 X c5 X c6 X c7 X c8 71

72 Example: Learnng HMM Parameers Sae oupu probably for S1 There are 13 observaons n S1 Segregae hem ou and coun Compue parameers (mean and varance of Gaussan oupu densy for sae S1 oupu densy for sae S1 Tme sae S1 S1 S1 S1 S1 S1 Obs X a1 X a2 X a6 X a7 X a9 X a10 ( 0.5( exp (2 1 ( X X S X P T d Tme sae S1 S1 S1 Ob X X X b a a a a a a X X X X X X X X X X X X X Obs X b3 X b4 X b9 Tme c c c c b b X X X X X X T T T a a T a a X X X X X X X X sae S1 S1 S1 S1 Obs X c1 X c2 X c4 X c T c c T c c b b b b X X X X X X X X 12 Oc /18797

73 Example: Learnng HMM Parameers Sae oupu probably for S2 There are 14 observaons n S2 Observaon 1 Tme sae S1 S1 S2 S2 S2 S1 S1 S2 S1 S1 Obs X a1 X a2 X a3 X a4 X a5 X a6 X a7 X a8 X a9 X a10 Observaon 2 Tme sae S2 S2 S1 S1 S2 S2 S2 S2 S1 Obs X b1 X b2 X b3 X b4 X b5 X b6 X b7 X b8 X b9 Observaon 3 Tme sae S1 S2 S1 S1 S1 S2 S2 S2 Obs X c1 X c2 X c3 X c4 X c5 X c6 X c7 X c8 73

74 Example: Learnng HMM Parameers Sae oupu probably for S2 There are 14 observaons n S2 Segregae hem ou and coun Compue parameers (mean and varance of Gaussan oupu densy for sae S2 Tme sae S2 S2 S2 S2 1 T 1 P( X S2 exp 0.5( X 2 2 ( X 2 d Obs X a3 X a4 X a5 X a8 (2 2 Tme sae S2 S2 S2 S2 S2 S2 Obs X b1 X b2 X b5 X b6 X b7 X b8 Tme sae S2 S2 S2 S X a X Obs X c2 X c6 X c7 X c8 X b6 X a4 X b7 X a5 X b8 X a8 X c2 X b1 X c6 X b2 X c7 X b X T 1 a3 2 X a c8

75 We have learn all he HMM parmeers Sae nal probables, ofen denoed as (S1 = 0.66 (S2 = 1/3 = 0.33 Sae ranson probables A Sae oupu probables bl Sae oupu probably for S1 1 T 1 P( X S1 exp 0.5( X 1 1 ( X 1 d (2 1 Sae oupu probably for S2 1 T 1 P( X S2 exp 0.5( X 2 2 ( X 2 d (2 2 75

76 Updae rules a each eraon No.of observaon sequences ha sar a sae s ( s Toal no. of observaon sequences P( s j s obs : sae ( s 1.&.& sae ( 1 s 1 obs : sae( s. obs : sae( s j ( X ( X obs, 1 obs : sae( s. obs, X obs, obs : sae ( s obs T : sae( s. Assumes sae oupu PDF = Gaussan For GMMs, esmae GMM parameers from 12 Oc 2010collecon of observaons 11755/18797 a any sae 76 1

77 Tranng by segmenaon: Verb ranng Inal yes models Segmenaons Models Converged? no Inalze all HMM parameers Segmen all ranng observaon sequences no saes usng he Verb algorhm wh he curren models Usng esmaed sae sequences and ranng observaon sequences, reesmae he HMM parameers Ths mehod s also called a segmenal k-means learnng procedure

78 Alernave o counng: SOFT counng Expecaon maxmzaon Every observaon conrbues o every sae 78

79 Updae rules a each eraon Obs Obs P( s j ( s s P( sae( 1 s Obs Obs Toal no. of observaon sequences P sae( s Obs P( sae( s, sae( 1 s j Obs Obs ( X Obs, P ( sae( s Obs Obs P ( sae ( s Obs Obs P( sae ( s Obs ( X ( X ( Obs, Obs, Obs P( sae( s Obs Every observaon conrbues o every sae 79 T

80 Updae rules a each eraon Obs Obs P( s j ( s s P( sae( 1 s Obs Obs Toal no. of observaon sequences P sae( s Obs P( sae( s, sae( 1 s j Obs Obs ( X Obs, P ( sae( s Obs Obs P ( sae ( s Obs Obs P( sae ( s Obs ( X ( X ( Obs, Obs, Obs P( sae( s Obs Where dd hese erms come from? 80 T

81 P( sae( s Obs The probably ha he process was a s when generaed X gven he enre observaon Droppng he Obs subscrp for brevy P( sae( s X, X 2,..., X T P( sae( s, X1, X 2,..., X 1 T We wll compue P( sae ( s, x, x,..., x frs ( 1 2 T Ths s he probably ha he process vsed s a me whle producng he enre observaon 81

82 P( sae( s, x, x2,..., x 1 T The probably ha he HMM was n a parcular sae s when generang he observaon sequence s he probably ha followed a sae sequence ha passed hrough s a me s me 82

83 P( sae( s, x, x2,..., x 1 T Ths can be decomposed no wo mulplcave secons The secon of he lace leadng no sae s a me and he secon leadng ou of s me 83

84 The Forward Pahs The probably of he red secon s he oal probably of all sae sequences endng a sae s a me Ths s smply (s, Can be compued usng he forward algorhm s me 84

85 The Backward Pahs The blue poron represens he probably of all sae sequences ha began a sae s a me Lke he red poron can be compued usng a backward recurson me 85

86 ( s, P ( x The Backward Recurson ( 2 1, x,..., xt sae ( s (s, s (N, (s, Can be recursvely esmaed sarng from he fnal me me nsan (backward recurson +1 me ( s, ( s', 1 P ( s' s P ( x 1 s' ( 1 s' (s, s he oal probably of ALL sae sequences ha depar from s a me, and all observaons afer x (s,t = 1 a he fnal me nsan for all vald fnal saes 86

87 The complee probably ( s, ( s, P( x 1, x 2,..., xt, sae( s (N, (s,-1 s (s, (s 1, me 87

88 Poseror probably of a sae The probably ha he process was n sae s a me, gven ha we have observed he daa s obaned by smple normalzaon P( sae( s Obs P( sae( s, x1, x2,..., xt P( sae( s, x, x,..., x 1 2 T s' s' ( s, ( s, ( s', ( s', Ths erm s ofen referred o as he gamma erm and denoed by s, 88

89 Updae rules a each eraon Obs Obs P( s j ( s s P( sae( 1 s Obs Obs Toal no. of observaon sequences P sae( s Obs P( sae( s, sae( 1 s j Obs Obs ( X Obs, P ( sae( s Obs Obs P ( sae ( s Obs Obs P( sae ( s Obs ( X ( X ( Obs, Obs, Obs These have been found P( sae( s Obs 89 T

90 Updae rules a each eraon Obs Obs P( s j ( s s P( sae( 1 s Obs Obs Toal no. of observaon sequences P sae( s Obs P( sae( s, sae( 1 s j Obs Obs ( X Obs, P ( sae( s Obs Obs P ( sae ( s Obs Obs P( sae ( s Obs ( X ( X ( Obs, Obs, Obs P( sae( s Obs Where dd hese erms come from? 90 T

91 P( sae( s, sae( 1 s', x, x2,..., x 1 T s s me Oc /18797

92 P( sae( s, sae( 1 s', x, x2,..., x ( s, 1 T s s me Oc /18797

93 P( sae( s, sae( 1 s', x, x2,..., x ( s, P( s' s P( x 1 s' 1 T s s me Oc /18797

94 P( sae( s, sae( 1 s', x, x2,..., x 1 T ( s, P( s' s P( x 1 s' ( s', 1 s s me Oc /18797

95 The a poseror probably of ranson ( s, P( s' s P( x 1 s' ( s', 1 P ( sae ( s, sae ( 1 s' Obs ( s, P( s s P( x s ( s, s s The a poseror probably of a ranson gven an observaon 95

96 Updae rules a each eraon Obs Obs P( s j ( s s P( sae( 1 s Obs Obs Toal no. of observaon sequences P sae( s Obs P( sae( s, sae( 1 s j Obs Obs ( X Obs, P ( sae( s Obs Obs P ( sae ( s Obs Obs P( sae ( s Obs ( X ( X ( Obs, Obs, Obs These have been found P( sae( s Obs 96 T

97 Tranng whou explc segmenaon: Baum-Welch ranng Every feaure vecor assocaed wh every sae of every HMM wh a probably Sae Inal assocaon models Models Converged? yes probables no Probables compued usng he forward-backward algorhm Sof decsons aken a he level of HMM sae In pracce, he segmenaon based Verb ranng s much easer o mplemen and s much faser The dfference n performance beween he wo s small, especally f we have los of ranng daa

98 HMM Issues How o fnd he bes sae sequence: Covered How o learn HMM parameers: Covered How o compue he probably of an observaon sequence: Covered 98

99 Magc numbers How many saes: No nce auomac echnque o learn hs You choose For speech, HMM opology s usually lef o rgh (no backward ransons For oher cyclc processes, opology mus reflec naure of process No. of saes 3 per phoneme n speech For oher processes, depends on esmaed no. of dsnc saes n process 99

100 Applcaons of HMMs Classfcaon: Learn HMMs for he varous classes of me seres from ranng daa Compue probably of es me seres usng he HMMs for each class Use n a Bayesan classfer Speech recognon, o vson, gene e sequencng, characer recognon, ex mnng, opc deecon 100

101 Applcaons of HMMs Segmenaon: Gven HMMs for varous evens, fnd even boundares Smply fnd he bes sae sequence and he locaons where sae denes change Auomac speech segmenaon, ex segmenaon by opc, geneome segmenaon, 101

102 Implemenaon Issues For long daa sequences arhmec underflow s a problem Scores are producs of numbers ha are all less han 1 The Verb algorhm provdes a workaround work only wh log probables bl Mulplcaon changes o addon compuaonally faser oo Underflow almos compleely elmnaed For he forward algorhm complex normalzaon schemes mus be mplemened o preven underflow A some compuaonal expense Ofen no worh go wh Verb 102

103 Classfcaon wh HMMs HMM for Yes HMM for No P(Yes P(X Yes P(No P(X No Speech recognon of solaed words: Tranng: Collec ranng nsances for each word Learn an HMM for each word Recognon of an observaon X For each word compue P(X word Usng forward algorhm Alernaely, compue P(X,bes.sae.sequence word Compued usng he Verb segmenaon algorhm Compue P(word P(X word P(word = a pror probably of word Selec he word for whch P(word P(X word s hghes 103

104 Creang compose models HMM for Open HMM for Close HMM for Fle HMM for Open Fle HMM for Fle Close HMMs wh absorbng saes can be combned no composes E.g. ran models for open, close and fle Concaenae hem o creae models for open fle and fle close Can recognze open fle and fle close 12 Oc /18797

105 Model graphs HMM for open HMM for fle HMM for close Models can also be composed no graphs No jus lnearly Verb sae algnmen wll ell us whch porons of he graphs were vsed for an observaon X 105

106 Recognzng from graph u Trells for Open Fle vs. Close Fle u The VITERBI bes pah ells you wha was spoken Open Close Fle 106

107 Recognzng from graph u Trells for Open Fle vs. Close Fle u The VITERBI bes pah ells you wha was spoken Open Close Fle 107

108 Language probables can be ncorporaed HMM for open P(Open P(fle open P(Close P(fle close l HMM for fle HMM for close Transons beween HMMs can be assgned a probably Drawn from properes of he language Here we have shown Bgram probables 108

109 Ths s used n speech recognon Recognzng one of four lnes from charge of he lgh brgade Cannon o rgh of hem Cannon o lef of hem Cannon n fron of hem Cannon behnd hem Each word s an HMM o rgh of hem lef of hem Each word s an HMM M Cannon n fron behnd of hem hem 11755/18797

110 Graphs can be reduced somemes Recognzng one of four lnes from charge of he lgh brgade Graph reducon does no mpede recognon of wha was spoken P(rgh o rgh P(of rgh P(of lef P(cannon P(o cannon o lef P(hem of Cannon of hem P(n cannon n fron P(behnd cannon behnd 11755/18797 P(hem behnd

111 Speech recognon: An asde In speech recognon sysems models are raned for phonemes Acually rphones phonemes n conex Word HMMs are composed from phoneme HMMs Language HMMs are composed from word HMMs The graph s reduced usng auomaed echnques John McDonough alks abou WFSTs on Thursday 111