The Concepts of Hidden Markov Model in Speech Recognition

Transcription

1 he Conceps of Hdden Mrov Model n Speech Recognon echncl Repor R99/9 Wleed H Adull nd ol K KsovDeprmen of Knowledge Engneerng L Informon Scence Deprmen Unversy of go ew Zelnd 999

2 he Conceps of Hdden Mrov Model n Speech Recognon Wleed H Adull nd ol K Ksov Knowledge Engneerng L Deprmen of Informon Scence Unversy of go ew Zelnd Inroducon Speech recognon feld s one of he mos chllengng felds h hve fced he scenss from long me he complee soluon s sll fr from rech he effors re concenred wh huge funds from he compnes o dfferen reled nd supporve pproches o rech he fnl gol hen pply o he enormous pplcons who re sll wng for he successful speech recognsers h re free from he consrns of speers voculres nd envronmen hs s s no n esy one due o he nerdscplnry nure of he prolem nd s reures speech percepon o e mpled n he recognser Speech Undersndng Sysems whch n urn srongly ponng o he use of nellgence whn he sysems he re echnues of recognsers whou nellgence re followng wde vrees of pproches wh dfferen clms of success y ech group of uhors who pu her fh n her fvoure wy However he sole echnue h gn he ccepnce of he reserchers o e he se of he r s Hdden Mrov Model HMM echnue HMM s greed o e he mos promsng one I mgh e used successfully wh oher echnues o mprove he performnce such s hyrdsng he HMM wh Arfcl eurl ewors A lgorhms hs doesn men h he HMM s pure from pproxmons h re fr from rely such s he successve oservon ndependence u he resuls nd he poenl of hs lgorhm s relle he modfcons on HMM e he urden of relesng from hese poorly represenve pproxmons hoppng for eer resuls 2

3 In hs repor we re gong o descre he c one of he HMM echnue wh he mn oulnes for successful mplemenon he represenon nd mplemenon of HMM vres n one wy or noher u he mn de s he sme s well s he resuls nd compuon coss s mer of preferences o choose one ur preference here s he one doped y Ferguson [] nd Rner e l [2]-[5] In hs repor we wll descre he Mrov Chn nd hen nvesgng very populr model n speech recognon feld he Lef-Rgh HMM opology he mhemcl formulon needed o e mplemened wll e fully explned s hey re crucl n uldng he HMM he promnen fcors n he desgn wll lso e dscussed Fnlly we conclude hs repor y some expermenl resuls o see he prccl oucomes of he mplemened model 2 Mrov Chns he HMM lgorhms re sclly nspred from he more hn 9 yers old mhemcl model nown s Mrov Chn o undersnd he ehvour of he Mrov Chn s dvsle o sr wh smple rel lfe exmple Le us consder smple weher forecs prolem nd ry o emule model h cn predc omorrow's weher sed on ody s condon In hs exmple we hve hree sonry ll dy weher whch could e sunny S cloudy C or Rny R From he hsory of he weher of he own under nvesgon we hve he followng le le- of proles of hvng cern se of omorrow's weher nd eng n cern condon ody: ody omorrow SunnyS CloudyC RnyR SunnyS 7 2 CloudyC RnyR le- Weher expecon proles 3

4 In hs cse wh we re loong for s he weher condonl proly omorrow/ody We relse h omorrow's weher depends on ody s condon s well s he prevous severl dys u we ccep he ssumpon of omorrow s weher depends only on ody s condon s s n conssency wh he frs order Mrov chn hs ssumpon s grely smplfyng he prolem of formulng he model even n he cul speech recognon cse nd we wll use when we come o cle he rel prolem We refer o he weher condons y se h re smpled nsn nd he prolem s o fnd he proly of weher condon of omorrow gven ody's condon / An cceple pproxmon for n nsns hsory s : / n / hs s he frs order Mrov chn s he hsory s consdered o e one nsn only he fne se dgrm of he weher prolsc le s shown n Fg Le us now s hs ueson: Gven ody s sunny S wh s he proly h he nex followng fve dys re S C C R nd S hvng he ove model? he nswer resdes n he followng formul usng frs order Mrov chn: S 2 S 3 C 4 C 5 R 6 S S 2 S/ S 3 C/ 2 S 4 C/ 3 C 5 R/ 4 C 6 S/ 5 R x 7 x 2 x 8 x 5 x he nl proly S s s ssumed h ody s sunny 7 S R C Fg Fne se represenon of weher forecs prolem 4

5 3 Hdden Mrov Model HMM he prculr prolem presened n he prevous secon he ses were oservle nd hey represened he weher condons S CR hey lso represened he oservon seuence hs nd of model formulon s very lmed due o he need of oservle se seuence whch s unnown n mos prolem he more generl cse s y consderng he se seuence o e hdden unoservle nd he oservons re prolsc funcons of he se hs noon mples he doule sochsc process More precsely he HMM s prolsc pern mchng echnue n whch he oservons re consdered o e he oupu of sochsc process nd consss of n underlyng Mrov chn I hs wo componens: fne se Mrov chn nd fne se of oupu proly dsruon he frs fruful nvesgon of HMM ws done y Bum e l [6]-[8] n he le 6s nd erly 7s he echnue ws ppled o he speech recognon feld y Ber [9] o undersnd he HMM we prefer o sr wh smple exmple nspred from h gven y Rner e l [3] Assume h we hve wo persons one dong n expermen nd he oher s n ousde oserver Le us consder h we hve urns ses numered from S o S nd n ech urn here re M coloured lls oservons dsrued n dfferen proporons Also we hve lc g elongs o ech urn ech g conns couners numered y hree numers hese numers re he curren urn numer S nd he followng wo urns numers S nd S 2 n proly proporons of 8 5 nd 5 respecvely he couners of he g elongng o he urn us efore he ls re crryng one of wo numers only; S - nd S n proles of 9 nd respecvely We ssume h he srng urn se s lwys urn S nd we end up n urn S he ls urn need no g s we sugges o sy her when we rech ll he end of he expermen We sr he expermen me y drwng ll from urn nd regser he colour hen reurn c o he urn hen drw couner from he correspondng urn g he expeced possle numers on he couners re: sy n urn or 2 move o he nex urn or 3 ump o he hrd urn We connue wh he sme procedure of drwng couner hen ll from he correspondng urn nd regserng he ll colours ll we rech se nd sy here ll he end of he expermen nsn 5

6 he oucome of hs expermen s seres of coloured lls oservons whch could e consdered s seuence of evens governed y he proly dsruon of he lls nsde ech urn nd y he couners exsng n ech g he ousde oserver hs no de ou whch urn ll ny nsn hs drwn from hdden ses wh he nows s only he oservon seuence of he coloured llsoservons Severl hngs could e concluded from hs expermen : he srng urn s lwys urn S 2 he urn whch hs een lef cn no e vsed gn e movng from lef o rgh drecon 3 Movemens re eher y one or wo urns o he rgh 4 he ls urn vsed s lwys urn S A chn of 5 urns ses s shown n Fg 2 y r g 2 y 2 r 2 g 2 3 y 3 r 3 g 3 4 y 4 r 4 g 4 5 y 5 r 5 g π { } Fg 2 Ses chn of he urn expermen usng 5 urns Ech numered crcle represens se nd he rrows shows he ses flow durng he whole process 6

7 Fg2 shows he noons whch we nen o use for he res of he repor nd hey re defned s follows: represens he proly of se rnson proly of eng n se S gven se S S / S w s he w symol ll colour proly dsruon n se S w s he lphe nd s he numer of symols n hs lphe π { } s he nl se proly dsruon In hs specl cse of ses chn opology π S { for for < he model s compleely defned y hese hree ses of prmeers nd π nd he model of ses nd M oservons cn e referred o y : A B π 2 where A { } B { w } nd M he model h we hve een descred s specl ype of HMM whch s normlly used n speech recognon I s clled Lef-Rgh HMM s derved from s wy of ehvour nd s opology movng from lef o rgh durng se rnson he reson for usng he L-R opology of HMM s due o s nheren srucure whch cn model he emporl flow of speech sgnl over me I mgh e no very ovous how he HMM reled o he speech sgnl modellng[] hs could e envsged y loong he speech producon mechnsm Speech s produced y he slow movemens of he rculory orgn he speech rculors ng up seuence of dfferen posons nd conseuenly producng he srem of sounds h form he speech sgnl Ech rculory poson could e represened y se of dfferen nd vryng duron Accordngly he rnson eween dfferen rculory posons ses cn e represened y A { } he oservons n hs cse re he sounds produced n ech poson nd due o he vrons n he evoluon of ech sound hs cn e lso represened y prolsc funcon B { w } he correspondence eween he model prmeers nd wh hey represen n he speech sgnl s no unue nd could e vewed dfferenly he mporn hng s o envsge he physcl menngs of he ses nd oservons n ech vew 7

8 4 HMM Consrns for Speech Recognon Sysems HMM could hve dfferen consrns dependng on he nure of he prolem h wned o e modelled he mn consrns needed n he mplemenon of speech recognsers cn e summrsed n he followng ssumpons[]: Frs order Mrov chn : In hs ssumpon he proly of rnson o se depends only on he curren se S / S - S -2 S w -n S z S / S 3 2 Sonry ses rnson hs ssumpon esfes h he ses rnson re me ndependen nd ccordngly we wll hve: S / S for ll 4 3 servons ndependence: hs ssumpon presumes h he oservons come ou whn cern se depend only on he underlyng Mrov chn of he ses whou consderng he effec of he occurrence of he oher oservons Alhough hs ssumpon s poor one nd dves from rely u wors fne n modellng speech sgnl hs ssumpon mples h: / p p / p 5 where p represens he consdered hsory of he oservon seuence hen we wll hve : / 6 4 Lef-Rgh opology consrn: for ll > 2 nd < 7 { for for π 8 S e π { } < 8

9 5 roly consrns: ur prolem s delng wh proles hen we hve he followng exr consrns: d π 9 If he oservons re dscree hen he ls negron wll e summon 5 he prncpl cses of HMM here re hree mn cses o e del wh o formule successful HMM hese re: Cse : Evluon Gven: model A B π redy o e used esng oservon seuence Acon: compue / ; he proly of he oservon seuence gven he model Cse 2: Decodng Gven: model A B π redy o e used esng or rnng oservon seuence Acon: rc he opmum se seuence Q h mos lely produce he gven oservons usng he gven model Cse 3: rnng Gven : model A B π redy o e used rnng oservon seuence where s he numer of exmples for rnng he model Acon: une he model prmeers o mxmse / 9

10 Cse s n evluon procedure s we re seeng o fnd he proly of producng gven oservon y gven model hs could e used o fnd ou he es model mong mny who produces he gven oservon Cse 2 s decodng procedure o deec or unhde he se seuence of gven oservon he oservons could e rnng exmples f we wn o sudy he ehvour of ech se from dfferen specs such s ses duron or specrl chrcerscs of ech se Some echnues ulse he se duron n her evluon procedure nd n hs cse he oservon wll e he es exmple o deec he ses duron Cse 3 s he rnng procedure o opmse he model prmeers o on he es model h represen cern se of oservons elongng o one spoen eny he wy s pved now o cle n mporn gol of our s nmely dervon of he mhemcl formuls o he hree prevous cses 5 Cse Formulon Le us e smple cse hen generlse o he complee one Consder h we hve 3 ses nd 5 oservons n process nd we wn o fnd / o expln he whole flow of he process he rells dgrm of Fg3 s of g help he se ech nsn s represened y smll crcle nd he rrows represen he se rnsons Se- 3 Ses rnsons Se- 2 Se servons Insns Fg3 rells Dgrm of 3 Ses nd 5 Insns L-R Model

11 From Fg3 we cn see ll he possles h he evens mgh e durng he whole process he doed lnes show one possly n whch /Q s compued y: 4 / 4 / / / 3 / 3 / 2 / 2 / / Q Q Q Q Q Q Q Q π π π π hs procedure hs o e done for ll possle ses seuences phs he superscrps of nd Q ndce he possly numer hen he proles of ll he phs hs o e summed o ge he overll proly of how lely he model produces he gven oservon seuence 5 / / / 5 / / p p Q Q Q where p s he numer of possle phs he ol numer of possles ncreses exponenlly wh he ncresng numer of ses nd oservon nsnces he Lef-Rgh opology s susnlly reducng he numer of possle phs over he full connecon opology ergodc models n whch every se could e reched from ny oher se ny nsn Furher reducon n he compuonl cos cn e cheved y he Forwrd-Bcwrd rocedure[2] hs echnue grely reduces he compuonl cos wh smple erve mhemcl formuls Acully s compound procedure composed of forwrd procedure nd cwrd procedure In he evluon cse we only need one of hem nd he forwrd procedure wll e our preference

12 5- Forwrd rocedure Inlly consder new forwrd proly vrle nsn nd se hs he followng formul: S / hs proly funcon could e solved for ses nd oservons ervely: Inlson 7 π 2 Inducon 8 Fg4 shows he nducon sep grphclly I s cler from hs fgure how se S nsn reched from possle ses nsn 3 ermnon / 9 hs sge s us sum of ll he vlues of he proly funcon over ll he ses nsn hs sum wll represen he proly of he gven oservons o e drven from he gven model h s how lely he gven model produces he gven oservons he proof of he ermnon formul wll e gven ler on 2

13 S S S S 3 S Fg4 Forwrd roly Funcon Rpresenon 5-2 Bcwrd rocedure hs procedure s smlr o he forwrd procedure u es no consderon he se flow s f n cwrd drecon from he ls oservon eny nsn ll he frs one nsn h mens h he ccess o ny se wll e from he ses h re comng us fer h se n me nd s shown n Fg5 o formule hs pproch le us consder he cwrd proly funcon β whch cn e defned s: β 2 / S 2 In nlogy o he forwrd procedure we cn solve for β n he followng wo seps: - Inlson: β 2 hese nl vlues for β s of ll ses nsn s rrrly seleced 3

14 2 Inducon β β 2 22 Euon 22 cn e well undersood wh help of Fg5 We re sll loong from lef o rgh n clculng he prl proly funcon β from o Fg5 shows hs ehvour clerly Even we re sll loong from lef o rgh n clculng he prl proly funcon from o However ech nsn we consder h we hve β nd we need o clcule me ; s f we re movng cwrd n me β β S 2 S 2 S 3 S 3 S Fg5 Bcwrd roly Funcon Rpresenon 5-3 Compung / from Forwrd nd Bcwrd roly Funcons he proly funcon of he model / cn e compued from oh nd β funcons Fg6 demonsres hs compuon grphclly A nsn he even of 4

15 eng n se nd movng o se nsn s clculed y whch ccouns for he ph ermnon n se he rnson o se s weghed y he produc A nsn he even of oservon seuence o he nsn srng from se S whle eng se S durng nsn s represened y he cwrd proly funcon β hen / s drecly concluded o e : / β 23 Susue 22 n 23 o ge: β 23 β S S S S 2 S S 3 3 S 3 S 3 S S - 2 Fg6 Forwrd - Bcwrd roly Funcons o fnd / 5

16 6 5-4 roof of ermnon Formul n Forwrd roly Funcon From 8 we hve : 8 Le - nd susue n 8 o ge: 8 8 From 23 we hve: 23 / β Le - n 23 o ge: 23 / β From 2 we hve: β Susue for β n 23 nd rerrnge he euon o ge: he erm nsde he sure rces s he sme s h n 8 susue nd you wll ge he fnl needed formul: / 9 24 /

17 5-2 Cse 2 Formulon hs cse dels wh he uncoverng he hdden ses of he model gven he oservon seuence nd he model hs mens h we hve o fnd he opml se seuence Q ssoced wh he gven oservon seuence presened o he model A B π he crer of opmly her s o serch for sngle es se seuence hrough modfed dynmc progrmmng echnue clled Ver Algorhm[3] We need o mxmse Q/ o deec he es se seuence hs could e cheved v mxmsng he on proly funcon Q/ usng o he Bysn Rule whch ses h: Q / Q / 25 / he denomnor hs nohng o shre n mxmsng Q/ s doesn nclude he se seuence fcor Q o go hrough he Ver Algorhm mehod le us defne he proly uny δ whch represens he mxmum proly long he es prole se seuence ph of gven oservon seuence fer nsns nd eng n se hs uny cn e defned mhemclly y: δ mx [ 2 S 2 2 / ] 26 he es se seuence s crced y noher funcon ψ he complee lgorhm cn e descred y he followng seps: Sep : Inlson δ π 27 ψ 28 Sep 2: Recurson δ mx[ δ ] 2 29 ψ rg mx[ δ ] 2 3 7

18 Sep 3: ermnon * * mx[ δ ] rg mx[ δ ] Sep 4: Bcrcng * ψ * I s cler h 29 of Ver recurson s smlr o 8 of forwrd nducon excep he nerchnge of summon y mxmson ne hng could e noed here s h Ver Algorhm cn lso e used o clcule he / pproxmely y consderng he use of * nsed hs s cceple s gves comprle resuls nd cn e usfed hrough he modfed euon 5 o do he summon on he mos prole se seuence whch hs he mor wegh mong ll he possle ses phs 5-3 Cse 3 Formulon hs cse s delng wh he rnng ssue whch s he mos dffcul one n ll he hree cses he s of hs cse s o dus he model prmeers A Bπ ccordng o cern opmly crer here re mny echnues o cheve he s of hs cse nd we wll descre here he well nown Bum-Welch Algorhm clled lso Forwrd Bcwrd Algorhm I s n erve mehod o rech he locl mxms of he proly funcon / Ech me he model prmeers re dused o ge new model whch s proved y Bum e l h he new model s eher eer or rech crcl condon whch he eron hs o e sopped s he locl mnm hs reched he model s lwys converge u he glol mxmson cn no e ssured Fg7 shows he non-lner opmson of hs prolem nd how he glol opmly seeng s dffcul o loce nd grely dependng on he nl pon of serch 8

19 / Locl Mxm Glol Mxm Locl Mxm Locl Mxm Fg7 pmum Serch ossles o go hrough he rnng procedure le us frs defne poseror proly funcon γ he proly of eng n se nsn gven he oservon seuence nd he model s: γ γ Snce nd from 23 hen γ S S β β β S β

20 2 Le us defne noher proly funcon ξ he proly of eng n se nsn nd gong o se nsn gven he model nd he oservon seuence ξ cn e mhemclly defned s: 4 Rule Byesn From 39 : ge o / y 39 of sdes oh Mulply 38 ξ ξ S S S S S S S S he rgh hnd sde of 4 cn e represened y he forwrd nd cwrd β funcons wh he help of Fg6 s follows: 4 S S β Susue 23 nd 4 n 39 nd rerrnge o ge: 42 β β ξ Also from 23 we cn hve: 43 β β ξ he relon eween γ nd ξ cn e esly deduced from her defnons : 44 ξ γ

21 2 ow f γ s summed over ll nsns excludng nsn we ge he expeced numer of mes h se S hs lef or he numer of mes hs se hs een vsed over ll nsns n he oher hnd f we sum ξ over ll nsns excludng we wll ge he expeced numer of rnsons h hve een mde from o From he ehvour of γ nd ξ he followng re-esmons of he model prmeers could e deduced: 47 S expeced numer nsnces n se nd hvng oservon nsns n se S expeced numer of 46 rnsons from S expeced numer of o S rnsons from S expeced numer of 45 S he srng se s nsns expeced numer of : ^ ^ ^ w w w β β γ γ β β γ ξ γ π Afer he re-esmon of he model prmeers we wll hve noher model ^ whch s more lely hn model producng oservon seuence hs mens h / / ^ > hs process of re-esmon cn e connued ll no mprovemen n / s reched h s we rech locl mxm

22 6 Dscree Hdden Mrov Model DHMM HMM modellng mehods ppled so fr re for process h hs dscree oservon seuence hese oservons could e he oucome ndces of Vecor Qunzon echnue VQ [4][5] VQ s echnue of cluserng me seres sgnl n our cse speech sgnl no cern numer of ns clusers Ech n represens he d elong o cern populon wh smlr or mnmum dfference specrl chrcerscs he cenre of grvy of ech n s ssgned o cern ndex nd consdered s he represenve of he cluser populon n ny process on he sgnl he long seuence of speech smples wll e represened y srem of ndces represenng frmes of dfferen wndow lenghs Hence VQ s consdered s process of redundncy removl whch mnmses he numer of s reured o denfy ech frme of speech sgnl VQ ws nlly used successfully wh Dynmc me Wrpng DW o recognse spoen words nd hen proved o e successful wh HMM s well he role of VQ n HMM s o prepre dscree symols from fne lphe Ech speech npu wll e unzed y he VQ reference ns Ech unzed npu wll e hen consdered s n oservon here re mny oher mehods o represen he oservons whch re no elong o he s of hs repor u very good reference o recommend s [6] he ype of HMM h models speech sgnls sed on VQ echnue o produce he oservons s clled Dscree Hdden Mrov Model DHMM I s effcen nd relle echnue whch hs comprle resuls o he more compuonl DW echnue In ddon he phones phonemes nd suwords could e modelled esly wh DHMM whle s very dffcul wh DW s he ler needs o deec he segmens oundry for comprson However VQ s responsle for loosng some nformon from he speech sgnl even when we ry o ncrese he codewords hs lose s due o he unzon error dsoron hs dsoron cn e reduced y ncresng he numer of codewords n he codoo u cnno e elmned 22

23 7 Connuous Hdden Mrov Model CHMM I s more sophsced mehodology o develop n mproved HMM model of he speech sgnl hs mehod even needs more memory hn DHMM o represen he model prmeers u s no sufferng from he dsoron prolem n he oher hnd needs more delere echnues o nlse he model s mgh dverge esly wh rndomly seleced nl prmeers In CHMM he model prmeers re lso π A nd B u hey re represened dfferenly he proly densy funcon pdf of cern oservons eng n se s consdered o e of Gussn Dsruon oher dsruons lso vld Le us consder o e nd hs he followng generl form: where: M c m m ℵ ; µ m U m c m : s he m-h mxure gn coeffcen n se 48 ℵ : s he pdf dsruon whch s consdered o e Gussn n our cse µ m : s he men of he m-h mxure n se U m : s he covrnce of he m-h mxure n se : s he oservon seuence of he feure vecors of dmenson d M : s he numer of mxures used : s he numer of ses he followng consrns hs o e fulflled o nsure he conssency of he model prmeers esmon m c M m c m m M hese consrns wll led o proper pdf normlson h s d for 5 23

24 he pdf of he oservons wll e of he form: 2π d / 2 U µ U 2 µ e 5 where prme superscrp here s referrng o he rnspose of mrx he covrnce mrx n 5 could e smplfed y usng dgonl mrx wh elemens represenng he vrnce of ech mxure hs pproxmon grely reduces he compuonl cos n spe of he necessy o ncrese he numer of mxures o me wor eer he reesmon formuls n mulmxure connuous densy HMM wll e s follows: ^ c m expeced nsnces of eng n se nd mxure m expeced nsnces of eng n se ^ c ^ µ ^ U m m m M m γ m γ m γ m γ m γ m µ γ m m µ m where γ m s he proly of eng n se wh m-h mxure nsn I s he sme s γ when m 24

25 he followng euon represens he modfed verson of 38 o me sule for mulmxure cse: c ; β mℵ µ m U m γ m M cm ; β ℵ µ m U m m 55 For he nl se nd he se rnson proly dsruons hey re he sme s for DHMM s n 45 nd 46 8 Mxure Densy Componens Esmon usng Mxmum Lelhood ML: he ML esmon s n opmson echnue h cn e used effcenly n esmng he dfferen componen of mulmxure models We re no gong hrough he mhemcl dervons of he ML u we only descre he mehod o e used n our s Le us frs me some defnons: : proly of eng n se gven oservon seuence c m : proly of eng n se wh mxure m gn coeffcen m : proly of eng n se wh mxure m nd gven Φw m : proly funcon of eng n mxure clss w m gven n se : s he ol numer of oservons n se m : s he numer of oservons n se wh mxure m : numer of ses M : numer of mxures n ech se 25

26 ow we re redy o mplemen he lgorhm hrough pplyng he followng seps: e severl versons of oservons of cern word sy dg zero spoen severl mes y mny speers 2 Apply Veer lgorhm o deec he ses of ech verson of he spoen word 3 u he whole oservons elongng o ech se from ll he versons of he spoen word no sepre cells ow we hve cells nd ech one represens he populon of cern se derved from severl oservon seuences of he sme word 4 Apply vecor unzon echnue o spl he populon of ech cell no M mxures nd geng w M clsses whn ech se 5 Usng he well nown sscl mehods o fnd he men µ m nd he covrnce U m of ech clss he gn fcor c m cn e clculed y: c m numer of oservons eng n se nd mxure m ol numer of oservons n se 56 6 Clcule Φw m from he followng formul: m Φ wm cm 57 ^ ^ 7 Fnd he nex esme of c µ nd U m from he formuls gven y ML : cˆ ˆ µ Uˆ ˆ m ˆ m m m Φ w M m m M cˆ Φ w Φ w cˆ m m m m ˆ m ℵ ; ˆ µ m m m m m m Uˆ m ˆ µ m ^ ˆ µ m

27 8 Compue he nex esme of usng he formul: Φˆ w m cˆ M n m cˆ ˆ n m ˆ n 63 9 IF Φ w Φˆ w ε HE ED m m ELSE Me he new vlue of Φw m eul he newly predced one Φ Φˆ w wm m G SE 7 where ε s very smll hreshold vlue 64 9 Implemenon Fcors here re severl fcors h my hve n one wy or noher effecs on he mplemened model We re gong o descre he more mporn fcors nd how o reduce her effecs 9- Sclng Fcor: he sclng fcor s mor ssue n mplemenng he HMM ecuse of he underflow h my esly occur when clculng he proly funcon / hs s due o he long seuence of mulplcons of less hn one vlues proly funcons For nsnce n usng he forwrd procedure o clcule n 8 we cn see esly how mny mulplcon of proly funcons we hve o me o clcule ny spoen enes he srgh forwrd echnue of sclng s sred y defnng he sclng coeffcen c[2]: c 65 ow le us compue from 8 nd hen mulply y c hs wll led o : 27

28 28 66 c Sme hng cn e done wh β o form he produc c β he re-esmon formul of 46 cn e rewren gn o nclude he sclng o ecome: 68 nd 68 where 67 ^ r r r c D c C D r C D C τ τ τ τ β β he numeror nd he denomnor of 67 conss of he produc c D C τ τ whch cn e fcored ou nd ren he orgnl euon of 46 hs sclng echnue cn lso e ppled successfully o 47 he sclng coeffcens cn e used o fnd log / y he followng mehod: Consder h we hve c for 23 nd we oned C from 68 hen from 65 we wll ge:

29 C c Usng 9 we wll hve : C c e he log of he ls wo erms : log[ ] log c 69 7 log c log[ ] 7 By usng log properes we cn on : 72 Euon 72 shows h log cn e compued u no s he ler wll e ou of he dynmc rnge of he compuer Ver Algorhm lso shows self her gn o e successful echnue n clculng log even whou oherng ou sclng prolem o follow Ver Algorhm le us ssume h: φ log[ π e he log of oh sdes of 29 nd use 72 o ge : φ mx[ φ ] log ] log[ ] ow log[ ] mx[ φ ] 9-2 Mulple servon Seuence fcor [2] he mn dsdvnge of Lef-Rgh opology of HMM s h he oservons cn no e concened no one srng nd sumed o he model for rnng hs s due o he one drecon lef-rgh move nd once se lef we cn no go c o Accordngly he model wll suc n he ls se fer pssng he frs oservon seuence nd no modellng possle for he oher seuences 29

30 3 he model hs o e modfed o ccep mulple seuence sumsson o llow he model o e rned y mny versons of he sme spoen eny Le us defne he se of oservons of he mulples of oservons of spoen eny y: ] [ ] [ where he gol of HMM s o mxmse / y dusng he prmeers of In mulple oservons / s defned y: K K 75 n more src wy 75 he mulple oservon seuence mplcon cn e done y normlsng he numerors nd denomnors 46 nd 47 y o ge: ^ ^ K K K K w β β η β β η he sme procedure of normlson could e used n he cse of connuous densy dsruon o fnd he prmeers of he model

31 9-3 Inl Model rmeers Esme fcor: When we nlly ry o uld n HMM model we normlly hve nohng u srems of oservons If we re forune hen we hve prmeers from old models whch s no normlly he cse o pu he nl model prmeers we hve o e creful s one mgh esly slp no dvergence wh d model nlson he prolem wh dscree oservons HMM s less effecve s we cn nlse he model prmeers wh rndom vlues u ng no consderon he consrns n 9 nd In connuous densy HMM CHMM he prolem s more serous nd he prmeers should e udcously seleced o ge rd of he dvergence fe Le us e he prolem n Lef-Rgh HMM opology nd sugges sfe wy o follow he prmeers h consue ny model re π A nd B For π s srgh forwrd nd nown o e lwys π [ ] of course hs s wh Lef-Rgh opology models For he ses rnson prmeers A[ ] he choce s lso flexle nd f we hve he opology of Fg2 hen A wll e he followng mrx for seven ses model: A he vlues of cn e seleced s : 94 4 nd 2 2 for for 6 for 7 hese vlues deduced from he fc h he oservons ends o sy n her curren se nd hve less endency o move o he nex se nd more less endency o ump he nex se Afer opmson we cn see h he oservons wned o sy n her 3

32 curren se s rue hs mply h > nd > 2 However he oservons mgh prefer o sy n he nex se or ump e > 2 or < 2 A more precse wy s y usng nl unform segmenon of ech uernce no he proposed numer of ses nd pply he followng lgorhm[]: he sule n progrmmng we use he noon nlson s found o e dependn on ses duron In hs cse he verge duron D of ech se s esmed y : D xk K n x n 79 where K s he numer of versons of n uernces n he rnng se s he numer of ses X n s he lengh of he n_h uernce n he rnng oservons hen he rnson mrx elemens re esmed y: For D D 8 2 For D D For c 8d 8e he formul s nferred from he fc h f here re K elemens of duron D n ech se hen here wll e only one rnson o he nex se Wh s lef now s he mos prolemc prmeers B { } hey hve o e very well nlsed In our cse we sugges he followng seps: 32

33 - Unformly segmen he uernces of ech spoen eny y ses 2 - e he men nd he covrnce of ech segmen 3 Consder he oservons follow Gussn densy proly dsruon Afer he prevous suggesons for nlsng he model prmeers we cn pply Ver Algorhm o exrc he opmum prmeers he echnue descred n hs secon s for unmodl sngle mxure dsruon o exend o mulmodl mulmxure dsruon he followngs re suggesed: Apply he sme procedures for π A B used n unmodl dsruon 2 Afer uncoverng he rel se seuence from Ver Algorhm ggrege he oservons of ll he versons of he spoen eny elongng o ech se n sepre cells 3 Use Vecor unzon echnue o cluser ech cell no severl mxures 4 pmse he cluserng y ny nown sscl echnue; such s mxmum lelhood nd expecon mxmson 5 Fnd he men nd he covrnce of ech cluser mxure he model s now complee 9-4 umer of Ses Fcor ne hng lef whch hs o e decded from he nl nsn of desgnng he model I s he opmum numer of ses needed o model he prolem here s no srgh forwrd nswer o hs reuremen he numer of ses s decded emprclly dependng on he nure of he prolem Some mes prevous experence ou he prolem s necessry or one hs o sugges dfferen numer of ses hen selec he one who gves he es resuls Also f we could defne he physcl menng of he ses we cn lm he numer of ses In soled words recognser he numer of ses re suggesed o e eween 4 nd 2 hs s usfed y ssumng h he ses re represenng he phonemes or he phones of he uernces In phonemes modellng he numer of ses re mosly ssumed o e 3 s he phonemes could e segmened no nl sle nd fnl ses 33

34 9-5 Se Duron Incorporon: he sc HMM does no e no consderon he se duron fcor n s modellng procedure hs s consdered s mor weness n he model snce he duron crry mporn nformon ou he emporl srucure of he speech sgnl ur duy now s o fnd some useful wy o nclude he duron whn he convenonl model Le us frs s hs ueson: Wh s he proly of eng n se for τ nsns? he nswer resdes n fndng he proly densy funcon p τ whch hs he defnon of : p τ S 2 S 3 S τ S τ S 8 π τ- - 8 ow we cn clcule he expeced duron n se y he followng euon: τ τ τp τ Usng 8 nd consderng π we ge : τ τ τ τ τ τ τ τ τ ow f we reurn o our frs exmple ou weher forecs nd s he ueson: Wh s he verge consecuve sunny cloudy nd rny dys? he nswer s y pplyng 83 usng he vlues of from le- o ge : 34

35 Sunny dys Cloudy dys Rny dys Unforunely hs duron dsruon s menngless when we ry o pply o speech recognon prolems herefore noher wy o ncorpore he duron hs o e consdered ne opon s o nclude he se duron n he model formuls hs needs reformulng he whole model prmeers[4] he model wors perfecly n hs cse u he prolem now s wh he vs ncrese n compuonl cos h mes he use of hs new model mprccl he oher opon s o use heursc echnue o nclude he duron o on comprle performnce s he correc heorecl duron ncluson wh very low compuonl nd sorge coss he se duron proly funcon p τ s esmed durng he model rnng cse nd my e defned s: p τ : s he proly of eng n se for τ duron he duron proly densy funcon s consdered o e Gussn wh 3 o 5 mxures Durng recognon he se duron re clculed from he crcng procedure n Ver Algorhm hen he log lelhood vlue s ncremened y he log of he duron proly vlue s elow: log[ ˆ ] log[ ] η log[ p τ ] 84 where η s sclng fcor Algorhm τ s he normlsed duron of eng n se s deeced y Ver 35

36 9-6 D Represenon Fcor: he rnng nd esng speech d re en from he: hp://kelogocnz/hyspeech/corpus he nl ses of d re dgs -9 spoen y 2 speers mles nd femles nd ech dg s spoen hree mes y ech speer Among hose words 42 uered dgs used for rnng nd 5 for esng he speech d n go Speech Corpus re smpled 225 Hz wh shor slence efore nd fer ech uernce he nex sep s o rnsform he me sgnl no Mel scle coeffcens he numer of coeffcen re seleced o e 26 2 mels nd 2 del mels wh one power nd s del Also expermens hve een done on 3 coeffcens whou consderng he dynmc ehvour of he sgnl he Mel scle coeffcens s exrced feures re seleced ecuse hey me no some exen he feure selecon n humn ers he Mel scle mehod consders he specrum re lnerly dsrued elow Hz nd logrhmclly ove h hs mes he fler ns movng on lner cenres elow Hz e 2 3 nd on logrhmc cenres over h e he very good chrcerscs of he Mel scle coeffcens s h hey llow he use of Euclden dsnce mesure n fndng he dsnce eween wo exmples hs s grely reduces he compuonl cos of procedures h depend on dsnce mesure le hose n VQ Resuls nd Conclusons In hs secon we re gong o show some expermenl resuls nd dscuss some useful conclusons - he frs expermen del wh he segmenon of spoen words no ses Fg8 shows dfferen versons of he spoen word zero y hree dfferen speers We cn see clerly how he me sgnl vred even for he sme word he ses re found y Ver Algorhm nd ssgned clerly o her correspondng segmens Also we cn see h he oservons re no lwys pssng hrough ll he ses h he model hs een desgned on In hs cse se 5 ws umped y dg zero oservons when hey were sumed o dg zero model 36

37 Fg8 Ses Assgnmen of Dg ZER resened o he ZER Se Model he nlyss uses 3 Mel scles coeffcens whou ng he dynmc coeffcens no consderon here re 6 ses deeced n ll he hree versons of he spoen dg ZER e nd 7 Se- Se-2 Se3 Se-4 Se-6 Se-7 37

38

39 -2 he second expermen del wh Mel scles coeffcens dsruon Fg9 Shows some dsruons of Mel scle coeffcens nd her dels of se one n spoen dg zero he power of he sgnl nd s del re represened y mel nd del mel he Mel coeffcens cpure he sle sgnl chrcerscs whle he dels cpure he dynmc chrcerscs Also we cn see from hs fgure he es f proly dsruon funcon pdf for ech coeffcen I s cler h some coeffcens le mel mel nd her dels re fr from eng represened y sngle pdf hs consoldes he need of mulmodl mulmxure represenon of he coeffcens In our model we pproxme he Mel scle coeffcens dsruon y 5 o 9 mxures Fg 9 Mel Scle Coeffcens Dsruon he hsogrm nd he es f norml pdf of melmel nd mel2 wh her dels es pdf mel del mel mel del mel mel 2 del mel 2 39

40 -3 he hrd expermen ws crred ou o show he correspondence eween he speech sgnl ses nd he specr hs relon gve us more undersndng nd confdence on he ehvour of he oservon vecors whn ech se Fg shows hs clerly nd we cn noce he dfference n specrl ehvour of dfferen ses Fg Shows he correspondence eween he me sgnl smples ses nd specrum of spoen dg zero Se- Se-2 Se-3 Se-4 Se-6 Se-7 4

41 -4 he fourh expermen ws ou he represenon of he se duron dsruon usng mulmodl mulmxure proly dsruon Fg shows he normlsed duron proly dsruon for unmodl nd mulmodl wh 3 mxures represenon he mulmodl pdf shows superory n represenng he dsruon Fg Mulmodl Represenon of Ses Duron he unmodl s poorly represenng he ses duron whle he mulmodl s smoohly follow up he duron dsruon even wh only hree mxures used mulmodl Se 4 cul dsruon unmodl Se 5 4

42 References : [] JD Ferguson Hdden Mrov Anlyss: An Inroducon n Hdden Mrov Models for Speech Insue of Defence Anlyses rnceon J 98 [2] S E Levnson L R Rner nd M M Sondh An Inroducon o he Applcon of he heory of rolsc Funcons of Mrov rocess o Auomc Speech Recognon he Bell Sysem echncl Journl vol 62 no4 pp35-73 Apr 983 [3] L R Rner B H Jung An Inroducon o Hdden Mrov Models IEEE ASS Mgzne pp 4 6 Jn 986 [4] L R Rner A uorl on Hdden Mrov Models nd Seleced Applcons n Speech Recognon vol 77 no 2 pp [5] L R Rner B H Jung Fundmenls of Speech Recognon rence Hll Englewood Clffs ew Jersey 993 [6] LE Bum nd ere Sscl Inference for rolsc Funcons of Fne Se Mrov Chns Ann Mh S vol 37 pp [7] L E Bum ere G Soules nd Wess A Mxmzon echnue ccurrng n he Sscl Anlyss of rolsc Funcons of Mrov Chns Ann Mh S vol 4 no pp [8] L E Bum An Ineuly nd Assoced Mxmzon echnue n Sscl Esmon for rolsc Funcons of Mrov rocesses roc Symp n Ineules vol 3 pp -7 Acdemc ress ew Yor nd London 972 [9] J K Ber he Drgon Sysem An vervew IEEE rns Acousc Speech nd Sgnl rocessng vol ASS-23 no pp Fe 975 [] F J wens Sgnl rocessng of Speech Mcmlln ress Ld London 993 [] E Hrorg Hdden Mrov Models Appled o Auomc Speech Recognon hd hess orwegn Insue of echnology rondhem Aug 99 [2] L E Bum nd J A Egon An nuly wh pplcons o sscl Esmon for rolsc Funcons of Mrov rocess nd o Model for Ecology Bull Amer Meeorol Soc vol73 pp [3] G D Forney he Ver Algorhm roc IEEE vol 6 pp Mr 973 [4] Y Lnde A Buzo R M Gry An ALgorhm for Vecor Qunzer Desgn IEEE rns on Comm vol CM-28 no pp

43 [5] R M Gry Vecor Qunzon IEEE ASS Mgzne vol no 2 pp [6] F Jelne Sscl Mehods for Speech Recognon he MI ress