Hidden Markov Models for Speech Recognition. Bhiksha Raj and Rita Singh

Transcription

1 Hidden Markov Model for Speech Recogniion Bhikha Raj and Ria Singh

2 Recap: T 11 T 22 T 33 T 12 T 23 T 13 Thi rcre i a generic repreenaion of a aiical model for procee ha generae ime erie The egmen in he ime erie are referred o a ae The proce pae hrogh hee ae o generae ime erie The enire rcre may be viewed a one generalizaion of he DTW model we have diced h far

3 The HMM Proce The HMM model he proce nderlying he obervaion a going hrogh a nmber of ae For inance in prodcing he ond W i fir goe hrogh a ae where i prodce he ond UH hen goe ino a ae where i raniion from UH o AH and finally o a ae where i prodced AH W AH UH The re nderlying proce i he vocal rac here Which roghly goe from he configraion for UH o he configraion for AH

4 are abracion The ae are no direcly oberved Here ae of he proce are analogo o configraion of he vocal rac ha prodce he ignal We only hear he peech; we do no ee he vocal rac i.e. he ae are hidden The inerpreaion of ae i no alway obvio The vocal rac acally goe hrogh a coninm of configraion The model repreen all of hee ing only a fixed nmber of ae The model abrac he proce ha generae he daa The yem goe hrogh a finie nmber of ae When in any ae i can eiher remain a ha ae or go o anoher wih ome probabiliy When a any ae i generae obervaion according o a diribion aociaed wih ha ae

5 Hidden Markov Model A Hidden Markov Model coni of wo componen A ae/raniion backbone ha pecifie how many ae here are and how hey can follow one anoher A e of probabiliy diribion one for each ae which pecifie he diribion of all vecor in ha ae Markov chain Thi can be facored ino wo eparae probabiliic eniie A probabiliic Markov chain wih ae and raniion A e of daa probabiliy diribion aociaed wih he ae Daa diribion

6 HMM a a aiical model An HMM i a aiical model for a ime-varying proce The proce i alway in one of a conable nmber of ae a any ime When he proce vii in any ae i generae an obervaion by a random draw from a diribion aociaed wih ha ae The proce conanly move from ae o ae. The probabiliy ha he proce will move o any ae i deermined olely by he crren ae i.e. he dynamic of he proce are Markovian The enire model repreen a probabiliy diribion over he eqence of obervaion I ha a pecific probabiliy of generaing any pariclar eqence The probabiliie of all poible obervaion eqence m o 1

7 How an HMM model a proce HMM amed o be generaing daa ae eqence ae diribion obervaion eqence

8 The opology of he HMM HMM Parameer No. of ae and allowed raniion E.g. here we have 3 ae and canno go from he ble ae o he red The raniion probabiliie Ofen repreened a a marix a here Tij i he probabiliy ha when in ae i he proce will move o j The probabiliy of beginning a a pariclar ae The ae op diribion 0.6 T =

9 HMM ae op diribion The ae op diribion repreen he diribion of daa prodced from any ae In he previo lecre we ame he ae op diribion o be Gaian Albei largely in a DTW conex P( v) = Gaian( v; m C) = 1 2π C e 0.5( v m) T C 1 ( v m) In realiy he diribion of vecor for any ae need no be Gaian In he mo general cae i can be arbirarily complex The Gaian i only a coare repreenaion of hi diribion If we model he op diribion of ae beer we can expec he model o be a beer repreenaion of he daa

10 Gaian Mixre A Gaian Mixre i lierally a mixre of Gaian. I i a weighed combinaion of everal Gaian diribion P( v) = K 1 i= 0 w Gaian( v; m i v i any daa vecor. P(v) i he probabiliy given o ha vecor by he Gaian mixre K i he nmber of Gaian being mixed w i i he mixre weigh of he i h Gaian. m i i i mean and C i i i covariance The Gaian mixre diribion i alo a diribion I i poiive everywhere. The oal volme nder a Gaian mixre i 1.0. Conrain: he mixre weigh w i m all be poiive and m o 1 i C i )

11 Generaing an obervaion from a Gaian mixre ae diribion Fir draw he ideniy of he Gaian from he a priori probabiliy diribion of Gaian (mixre weigh) Then draw a vecor from he eleced Gaian

12 Gaian Mixre A Gaian mixre can repreen daa diribion far beer han a imple Gaian The wo panel how he hiogram of an nknown random variable The fir panel how how i i modeled by a imple Gaian The econd panel model he hiogram by a mixre of wo Gaian Cavea: I i hard o know he opimal nmber of Gaian in a mixre diribion for any random variable

13 wih Gaian mixre ae diribion The parameer of an HMM wih Gaian mixre ae diribion are: π he e of iniial ae probabiliie for all ae T he marix of raniion probabiliie A Gaian mixre diribion for every ae in he HMM. The Gaian mixre for he i h ae i characerized by K i he nmber of Gaian in he mixre for he i h ae The e of mixre weigh w ij 0<j<K i The e of Gaian mean m ij 0 <j<k i The e of Covariance marice C ij 0 < j <K i

14 Given an HMM: Three Baic HMM Problem Wha i he probabiliy ha i will generae a pecific obervaion eqence Given a obervaion eqence how do we deermine which obervaion wa generaed from which ae The ae egmenaion problem How do we learn he parameer of he HMM from obervaion eqence

15 Comping he Probabiliy of an Obervaion Seqence Two apec o prodcing he obervaion: Preceing hrogh a eqence of ae Prodcing obervaion from hee ae

16 Preceing hrogh ae HMM amed o be generaing daa ae eqence The proce begin a ome ae (red) here From ha ae i make an allowed raniion To arrive a he ame or any oher ae From ha ae i make anoher allowed raniion And o on

17 Probabiliy ha he HMM will follow a pariclar ae eqence P(...) = P( ) P( ) P( ) P( 1 ) i he probabiliy ha he proce will iniially be in ae 1 P( i i ) i he raniion probabiliy of moving o ae i a he nex ime inan when he yem i crrenly in i Alo denoed by Tij earlier

18 Generaing Obervaion from Sae HMM amed o be generaing daa ae eqence ae diribion obervaion eqence A each ime i generae an obervaion from he ae i i in a ha ime

19 Probabiliy ha he HMM will generae a pariclar obervaion eqence given a ae eqence (ae eqence known) P( o o o......) = P( o ) P( o ) P( o ) Comped from he Gaian or Gaian mixre for ae 1 P(o i i ) i he probabiliy of generaing obervaion o i when he yem i in ae i

20 Preceing hrogh Sae and Prodcing Obervaion HMM amed o be generaing daa ae eqence ae diribion obervaion eqence A each ime i prodce an obervaion and make a raniion

21 Probabiliy ha he HMM will generae a pariclar ae eqence and from i a pariclar obervaion eqence P( o o o......) = P( o1 o2 o ) P( ) = P( o 1 1) P( o2 2) P( o3 3)... P( 1) P( 2 1) P( 3 2)...

22 Probabiliy of Generaing an Obervaion Seqence If only he obervaion i known he precie ae eqence followed o prodce i i no known All poible ae eqence m be conidered P( o1 o2 o3...) = all. poible ae. eqence P( o o o......) = all. poible ae. eqence P( o ) P( o ) P( o )... P( ) P( ) P( )

23 Comping i Efficienly Explici mming over all ae eqence i no efficien A very large nmber of poible ae eqence For long obervaion eqence i may be inracable Fornaely we have an efficien algorihm for hi: The forward algorihm A each ime for each ae compe he oal probabiliy of all ae eqence ha generae obervaion nil ha ime and end a ha ae

24 Illraive Example Conider a generic HMM wih 5 ae and a erminaing ae. We wih o find he probabiliy of he be ae eqence for an obervaion eqence aming i wa generaed by hi HMM P( i ) = 1 for ae 1 and 0 for oher The arrow repreen raniion for which he probabiliy i no 0. P( i i ) = a ij We omeime alo repreen he ae op probabiliy of i a P(o i ) = b i () for breviy 91

25 Diverion: The Trelli Sae index α () -1 Feare vecor (ime) The relli i a graphical repreenaion of all poible pah hrogh he HMM o prodce a given obervaion Analogo o he DTW earch graph / relli The Y-axi repreen HMM ae X axi repreen obervaion Every edge in he graph repreen a valid raniion in he HMM over a ingle ime ep Every node repreen he even of a pariclar obervaion being generaed from a pariclar ae

26 The Forward Algorihm α () = P( x x... x ae() = λ) 1 2 Sae index α () -1 ime α () i he oal probabiliy of ALL ae eqence ha end a ae a ime and all obervaion nil x

27 The Forward Algorihm α () = P( x x... x ae() = λ) 1 2 Sae index α ( 1) α () Can be recrively eimaed aring from he fir ime inan (forward recrion) α ( 1 1) α () = α ( 1) P ( ) P( x ) -1 ime α () can be recrively comped in erm of α ( ) he forward probabiliie a ime -1

28 The Forward Algorihm Toalprob = α ( T ) Sae index T ime In he final obervaion he alpha a each ae give he probabiliy of all ae eqence ending a ha ae The oal probabiliy of he obervaion i he m of he alpha vale a all ae

29 Problem 2: The ae egmenaion problem Given only a eqence of obervaion how do we deermine which eqence of ae wa followed in prodcing i?

30 The HMM a a generaor HMM amed o be generaing daa ae eqence ae diribion obervaion eqence The proce goe hrogh a erie of ae and prodce obervaion from hem

31 Sae are Hidden HMM amed o be generaing daa ae eqence ae diribion obervaion eqence The obervaion do no reveal he nderlying ae

32 The ae egmenaion problem HMM amed o be generaing daa ae eqence ae diribion obervaion eqence Sae egmenaion: Eimae ae eqence given obervaion

33 Eimaing he Sae Seqence Any nmber of ae eqence cold have been ravered in prodcing he obervaion In he wor cae every ae eqence may have prodced i Solion: Idenify he mo probable ae eqence The ae eqence for which he probabiliy of progreing hrogh ha eqence and gen eraing he obervaion eqence i maximm i.e i maximm P( o o o......) =

34 Eimaing he ae eqence Once again exhaive evalaion i impoibly expenive B once again a imple dynamic-programming olion i available P( o o o......) P( o 1 1) P( o2 2) P( o3 3)... P( 1) P( 2 1) P( 3 2)... Needed: = arg max... P( o1 1) P( 1) P( o2 2) P( 2 1) P( o3 3) P( 3 2) 1 2 3

35 Eimaing he ae eqence Once again exhaive evalaion i impoibly expenive B once again a imple dynamic-programming olion i available P( o o o......) P( o 1 1) P( o2 2) P( o3 3)... P( 1) P( 2 1) P( 3 2)... Needed: = arg max... P( o1 1) P( 1) P( o2 2) P( 2 1) P( o3 3) P( 3 2) 1 2 3

36 The ae eqence The probabiliy of a ae eqence???? x y ending a ime i imply he probabiliy of???? x mliplied by P(o y )P( y x ) The be ae eqence ha end wih x y a will have a probabiliy eqal o he probabiliy of he be ae eqence ending a -1 a x ime P(o y )P( y x ) Since he la erm i independen of he ae eqence leading o x a -1

37 Trelli The graph below how he e of all poible ae eqence hrogh hi HMM in five ime inan ime 94

38 The co of exending a ae eqence The co of exending a ae eqence ending a x i only dependen on he raniion from x o y and he obervaion probabiliy a y y x ime 94

39 The co of exending a ae eqence The be pah o y hrogh x i imply an exenion of he be pah o x y x ime 94

40 The Recrion The overall be pah o x i an exenion of he be pah o one of he ae a he previo ime y x ime

41 The Recrion Bepah prob( y ) = Be (Bepah prob(? ) * P( y? ) * P(o y )) y x ime

42 Finding he be ae eqence Thi give a imple recrive formlaion o find he overall be ae eqence: 1. The be ae eqence X 1i of lengh 1 ending a ae i i imply i. The probabiliy C(X 1i ) of X 1i i P(o 1 i ) P( i ) 2. The be ae eqence of lengh +1 i imply given by (argmax Xi C(X i )P(o +1 j ) P( j i )) i 3. The be overall ae eqence for an erance of lengh T i given by argmax Xi j C(X Ti ) The ae eqence of lengh T wih he highe overall probabiliy 89

43 Finding he be ae eqence The imple algorihm j preened i called he VITERBI algorihm in he lierare Afer A.J.Vierbi who invened hi dynamic programming algorihm for a compleely differen prpoe: decoding error correcion code! The Vierbi algorihm can alo be viewed a a breadh-fir graph earch algorihm The HMM form he Y axi of a 2-D plane Edge co of hi graph are raniion probabiliie P( ). Node co are P(o ) A linear graph wih every node a a ime ep form he X axi A relli i a graph formed a he croprodc of hee wo graph The Vierbi algorihm find he be pah hrogh hi graph 90

44 Vierbi Search (cond.) Iniial ae iniialized wih pah-core = P( 1 )b 1 (1) All oher ae have core 0 ince P( i ) = 0 for hem ime 92

45 Vierbi Search (cond.) P () j Sae wih be pah-core Sae wih pah-core < be Sae wiho a valid pah-core = max [P i (-1) a ij b j ()] i Sae raniion probabiliy i o j Score for ae j given he inp a ime Toal pah-core ending p a ae j a ime ime 93

46 Vierbi Search (cond.) P () j = max [P i (-1) a ij b j ()] i Sae raniion probabiliy i o j Score for ae j given he inp a ime Toal pah-core ending p a ae j a ime ime 94

47 Vierbi Search (cond.) ime 94

48 Vierbi Search (cond.) ime 94

49 Vierbi Search (cond.) ime 94

50 Vierbi Search (cond.) ime 94

51 Vierbi Search (cond.) ime 94

52 Vierbi Search (cond.) THE BEST STATE SEQUENCE IS THE ESTIMATE OF THE STATE SEQUENCE FOLLOWED IN GENERATING THE OBSERVATION ime 94

53 Vierbi and DTW The Vierbi algorihm i idenical o he ring-maching procedre ed for DTW ha we aw earlier I compe an eimae of he ae eqence followed in prodcing he obervaion I alo give he probabiliy of he be ae eqence

54 Problem3: Training HMM parameer We can compe he probabiliy of an obervaion and he be ae eqence given an obervaion ing he HMM parameer B where do he HMM parameer come from? They m be learned from a collecion of obervaion eqence We have already een one echniqe for raining : The egmenal K-mean procedre

55 Modified egmenal K-mean AKA Vierbi raining The enire egmenal K-mean algorihm: 1. Iniialize all parameer Sae mean and covariance Traniion probabiliie Iniial ae probabiliie 2. Segmen all raining eqence 3. Reeimae parameer from egmened raining eqence 4. If no converged rern o 2

56 Iniialize Segmenal K-mean Ierae T1 T2 T3 T4 The procedre can be conined nil convergence Convergence i achieved when he oal be-alignmen error for all raining eqence doe no change ignificanly wih frher refinemen of he model

57 A Beer Techniqe The Segmenal K-mean echniqe niqely aign each obervaion o one ae However hi i only an eimae and may be wrong A beer approach i o ake a of deciion Aign each obervaion o every ae wih a probabiliy

58 The probabiliy of a ae The probabiliy aigned o any ae for any obervaion x i he probabiliy ha he proce wa a when i generaed x We wan o compe P ( ae( ) = x x2... xt ) P( ae( ) = x1 x2... x 1 T We will compe P ae( ) = x x... x ) fir ( 1 2 T Thi i he probabiliy ha he proce viied a ime while prodcing he enire obervaion )

59 Probabiliy of Aigning an Obervaion o a Sae The probabiliy ha he HMM wa in a pariclar ae when generaing he obervaion eqence i he probabiliy ha i followed a ae eqence ha paed hrogh a ime ime

60 Probabiliy of Aigning an Obervaion o a Sae Thi can be decompoed ino wo mliplicaive ecion The ecion of he laice leading ino ae a ime and he ecion leading o of i ime

61 Probabiliy of Aigning an Obervaion o a Sae The probabiliy of he red ecion i he oal probabiliy of all ae eqence ending a ae a ime Thi i imply α() Can be comped ing he forward algorihm ime

62 The forward algorihm α () = P( x x... x ae() = λ) 1 2 Sae index α ( 1) α () Can be recrively eimaed aring from he fir ime inan (forward recrion) α ( 1 1) α () = α ( 1) P ( ) P( x ) -1 ime λ repreen he complee crren e of HMM parameer

63 The Fre Pah The ble porion repreen he probabiliy of all ae eqence ha began a ae a ime Like he red porion i can be comped ing a backward recrion ime

64 The Backward Recrion β ( ) = P( x x... x ae( ) = λ) T β () β β ( N + 1) ( +1) Can be recrively eimaed aring from he final ime ime inan (backward recrion) +1 β () = β ( + ) P ( ) P( x ) ime β () i he oal probabiliy of ALL ae eqence ha depar from a ime and all obervaion afer x β(t) = 1 a he final ime inan for all valid final ae

65 The complee probabiliy α ( ) β ( ) = P( x x... x ae( ) = λ) 1 2 T β ( N + 1) α ( ) 1 β ( +1) α ( 1 1) ime = P( X ae( ) = λ)

66 Poerior probabiliy of a ae The probabiliy ha he proce wa in ae a ime given ha we have oberved he daa i obained by imple normalizaion Pae ( ( ) = X λ) = P( X ae( ) = λ) P( X ae( ) = λ) Thi erm i ofen referred o a he gamma erm and denoed by γ = α () β () α ( ) β ( )

67 Updae Rle Once we have he ae probabiliie (he gamma) he pdae rle are obained hrogh a imple modificaion of he formlae ed for egmenal K-mean Thi new learning algorihm i known a he Bam-Welch learning procedre Cae1: Sae op deniie are Gaian

68 Updae Rle = x x N 1 μ = x γ γ μ = x T x x N C ) ( ) ( 1 μ μ = T x x C ) ( ) ( γ μ μ γ Segmenal K-mean Bam Welch A imilar pdae formla reeimae raniion probabiliie The iniial ae probabiliie P() alo have a imilar pdae rle

69 Cae 2: Sae op deniie are Gaian Mixre When ae op deniie are Gaian mixre more parameer m be eimaed P( x ) = K 1 i= 0 w igaian( x; μ i C i ) The mixre weigh w i mean μ i and covariance C i of every Gaian in he diribion of each ae m be eimaed

70 Spliing he Gamma We pli he gamma for any ae among all he Gaian a ha ae Re-eimaion of ae parameer A poeriori probabiliy ha he h vecor wa generaed by he k h Gaian of ae h γ = P( ae( ) = X λ) P( k. Gaian ae( ) = x λ k )

71 Spliing he Gamma among Gaian A poeriori probabiliy ha he h vecor wa generaed by he k h Gaian of ae h γ = P( ae( ) = X λ) P( k. Gaian ae( ) = x λ k ) γ k = P( ae( ) = X λ) ( 2π ) ( 2π ) k k T 1 ( μk ) Ck ( μk ) 1 w e x x 1 2 k d C T 1 ( μk ) Ck ( μ k ) 1 w e x x 1 2 k d k C

72 Updaing HMM Parameer ~ μ ~ k w k = = γ γ γ γ j x k k k j ~ γ ~ μ ~ μ ( x )( x ) k k k C k = γ k T Noe: Every obervaion conribe o he pdae of parameer vale of every Gaian of every ae

73 Overall Training Procedre: Single Gaian PDF Deermine a opology for he HMM Iniialize all HMM parameer Iniialize all allowed raniion o have he ame probabiliy Iniialize all ae op deniie o be Gaian We ll revii iniializaion 1. Over all erance compe he fficien aiic T x 2. Ue pdae formlae o compe new HMM parameer 3. If he overall probabiliy of he raining daa ha no converged rern o ep 1 γ γ γ ( μ ) ( x x ) μ

74 An Implemenaional Deail Sep1 compe bffer over all erance Thi can be pli and parallelized U 1 U 2 ec. can be proceed on eparae machine = U U x x x γ γ γ = U U γ γ γ.. ) ( ) ( ) ( ) ( ) ( ) ( = U T U T T x x x x x x μ μ γ μ μ γ μ μ γ 1 U γ 1 U γ x 1 ) ( ) ( U T x x μ μ γ 2 U γ 2 U γ x 2 ) ( ) ( U T x x μ μ γ Machine 1 Machine 2

75 An Implemenaional Deail Sep2 aggregae and add bffer before pdaing he model = U U x x x γ γ γ = U U γ γ γ.. ) ( ) ( ) ( ) ( ) ( ) ( = U T U T T x x x x x x μ μ γ μ μ γ μ μ γ ~ μ γ γ k k k x = ( )( ) ~ ~ ~ C k k k k T k x x = γ μ μ γ ~ w k k j j = γ γ

76 An Implemenaional Deail Sep2 aggregae and add bffer before pdaing he model γ γ + γ.. ~ μ ~ k w k γ = + U U = x γ x + γ x + U U 1 1 T T T γ ( x μ ) ( x μ ) = γ ( x μ ) ( x μ ) + γ ( x μ ) ( x μ ) +.. = = γ γ γ γ j x k k k j U 1 ~ γ U 2 x ~ μ x ~ μ ( )( ) k k k C k = γ k Comped by machine 1 Comped by machine 2 T

77 Training for wih Gaian Mixre Sae Op Diribion Gaian Mixre are obained by pliing 1. Train an HMM wih (ingle) Gaian ae op diribion 2. Spli he Gaian wih he large variance Perrb he mean by adding and bracing a mall nmber Thi give 2 Gaian. Pariion he mixre weigh of he Gaian ino wo halve one for each Gaian A mixre wih N Gaian now become a mixre of N+1 Gaian 3. Ierae BW o convergence 4. If he deired nmber of Gaian no obained rern o 2

78 Spliing a Gaian μ ε μ+ε μ μ The mixre weigh w for he Gaian ge hared a 0.5w by each of he wo pli Gaian

79 Implemenaion of BW: nderflow Arihmeic nderflow i a problem α () = α ( 1) P( ) Px ( ) probabiliy erm probabiliy erm The alpha erm are a recrive prodc of probabiliy erm A increae an increaingly greaer nmber probabiliy erm are facored ino he alpha All probabiliy erm are le han 1 Sae op probabiliie are acally probabiliy deniie Probabiliy deniy vale can be greaer han 1 On he oher hand for large dimenional daa probabiliy deniy vale are ally mch le han 1 Wih increaing ime alpha vale decreae Wihin a few ime inan hey nderflow o 0 Every alpha goe o 0 a ome ime. All fre alpha remain 0 A he dimenionaliy of he daa increae alpha goe o 0 faer

80 Underflow: Solion One mehod of avoiding nderflow i o cale all alpha a each ime inan Scale wih repec o he large alpha o make re he large caled alpha i 1.0 Scale wih repec o he m of he alpha o enre ha all alpha m o 1.0 Scaling conan m be appropriaely conidered when comping he final probabiliie of an obervaion eqence

81 Implemenaion of BW: nderflow Similarly arihmeic nderflow can occr dring bea compaion β ( ) = β ( ' + 1) P( ' ) P( x + 1 ' ) ' The bea erm are alo a recrive prodc of probabiliy erm and can nderflow Underflow can be prevened by Scaling: Divide all bea erm by a conan ha preven nderflow By performing bea compaion in he log domain

82 Bilding a recognizer for iolaed word Now have all neceary componen o bild an HMM-baed recognizer for iolaed word Where each word i poken by ielf in iolaion E.g. a imple applicaion where one may eiher ay Ye or No o a recognizer and i m recognize wha wa aid

83 Iolaed Word Recogniion wih Aming all word are eqally likely Training Collec a e of raining recording for each word Compe feare vecor eqence for he word Train for each word Recogniion: Compe feare vecor eqence for e erance Compe he forward probabiliy of he feare vecor eqence from he HMM for each word Alernaely compe he be ae eqence probabiliy ing Vierbi Selec he word for which hi vale i highe

84 Ie Wha i he opology o e for he How many ae Wha kind of raniion rcre If ae op deniie have Gaian Mixre: how many Gaian?

85 HMM Topology For peech a lef-o-righ opology work be The Baki opology Noe ha he iniial ae probabiliy P() i 1 for he 1 ae and 0 for oher. Thi need no be learned Sae may be kipped

86 Deermining he Nmber of Sae How do we know he nmber of ae o e for any word? We do no really Ideally here hold be a lea one ae for each baic ond wihin he word Oherwie widely differing ond may be collaped ino one ae The average feare vecor for ha ae wold be a poor repreenaion For compaional efficiency he nmber of ae hold be mall Thee wo are conflicing reqiremen ally olved by making ome edcaed gee

87 Deermining he Nmber of Sae For mall vocablarie i i poible o examine each word in deail and arrive a reaonable nmber: S O ME TH I NG For larger vocablarie we may be forced o rely on ome ad hoc principle E.g. proporional o he nmber of leer in he word Work beer for ome langage han oher Spanih and Indian langage are good example where hi work a almo every leer in a word prodce a ond

88 How many Gaian No clear anwer for hi eiher The nmber of Gaian i ally a fncion of he amon of raining daa available Ofen e by rial and error A minimm of 4 Gaian i ally reqired for reaonable recogniion

89 Implemenaion of BW: iniializaion of alpha and bea Iniializaion for alpha: α (1) e o 0 for all ae excep he fir ae of he model. α (1) e o P(o 1 ) for he fir ae All obervaion m begin a he fir ae Iniializaion for bea: β ( T) e o 0 for all ae excep he erminaing ae. β ( ) e o 1 for hi ae All obervaion m erminae a he final ae

90 Iniializing Sae Op Deniy Parameer 1. Iniially only a ingle Gaian per ae amed Mixre obained by pliing Gaian 2. For Baki-opology a good iniializaion i he fla iniializaion Compe he global mean and variance of all feare vecor in all raining inance of he word Iniialize all Gaian (i.e all ae op diribion) wih hi mean and variance Their mean and variance will converge o appropriae vale aomaically wih ieraion Gaian pliing o compe Gaian mixre ake care of he re

91 Iolaed word recogniion: Final hogh All relevan opic covered How o compe feare from recording of he word We will no explicily refer o feare compaion in fre lecre How o e HMM opologie for he word How o rain for he word Bam-Welch algorihm How o elec he mo probable HMM for a e inance Comping probabiliie ing he forward algorihm Comping probabiliie ing he Vierbi algorihm Which alo give he ae egmenaion

92 Qeion?