Hybrid of Chaos Optimization and Baum-Welch Algorithms for HMM Training in Continuous Speech Recognition

Transcription

1 Inernaonal Conference on Inellgen Conrol and Informaon Processng Augus 3-5, - Dalan, Chna Hybrd of Chaos Opmzaon and Baum-Welch Algorhms for HMM ranng n Connuous Speech Recognon Somayeh Cheshom, Saeed Raha-Q, and Mohammad-R Akbarzadeh- Absrac In hs paper a new opmzaon algorhm based on Chaos Opmzaon algorhm(coa) combned wh radonal Baum Welch (BW) mehod s presened for ranng Hdden Markov Model (HMM) for Connues speech recognon. he BW algorhm easly rapped n local opmum, whch mgh deerorae he speech recognon rae, whle an mporan characer of COA s global search.so we can ge a globally opmal soluon or a leas sub-opmal soluon. In hs paper Chaos opmzaon algorhm was appled o he opmzaon of he nal value of HMM parameers n Baum-Welch algorhm. Expermenal resuls showed ha usng Chaos Opmzaon algorhm for HMM ranng (Chaos-HMM ranng) has a beer performance han usng oher heursc algorhms such as PSOBW and GAPSOBW. I. IRODUCIO IDDE Markov Models (HMMs) are wdely used for Hbologcal sequence analyss because of her ably o ncorporae bologcal nformaon n her srucure. s a naural and hghly robus sascal mehodology for auomac speech recognon. Some heursc algorhms such as Baum-Welch algorhm are developed o opmze he model parameers o descrbe he ranng observaon sequences. However, hese mehods are hll-clmbng algorhms and easy o converge o local opmal soluons, whch mgh deerorae he speech recognon rae[].he HMM parameers play mporan roles n a HMM based speech recognzer because hey can characerze he behavor of he speech segmens and drecly affec he sysem recognon accuracy. Anoher possbly s o use some sochasc search mehods. Genec Algorhm (GA)[], abu Search (S)[3] and Parcle Swarm Opmzaon (PSO)[] have been appled o HMM ranng n solaed and connuous word recognon. However, he convergence speed of hese algorhms s slow [4]. Chaos opmzaon algorhms s a novel mehod ha have many good properes such as pseudo-randomness, ergodcy and rregulary. Chaos can go hrough every sae n a ceran Manuscrp receved May 5,. S. Cheshom s wh he Islamc Azad unversy of Mashhad, Iran ( e-mal: somayeh.cheshom@ gmal.com). S. Raha-Q s wh he Islamc Azad unversy of Mashhad, Iran (e-mal: raha@mshdau.ac.r). Mohammad-R.Akbarzadeh- s wh he Ferdows Unversy of Mashhad, Iran (e-mal: akbarzadeh@eee.org). doman whou repeon. Wh hs advanage of chaos, we can apply n opmzaon calculaons. a new opmzaon algorhm ha s called he chaos opmzaon algorhms COA s presened n [5]. COA searches on he regulary of chaoc moons and can more easly escape from local mnma han sochasc opmzaon algorhms. By use of hese properes of chaos, an effecve hybrd opmzaon algorhm (CHAOS-BW) amed a fndng he global soluon or beer opmal soluons s proposed ha reasonably combnes he mers of he Chaos algorhm and he BW algorhm for he esmaon of he HMM model parameers. o evaluae he performance of hs mehod, we compare wh wo oher hybrd algorhms PSOBW[6], GAPSO-BW and radonal algorhm BW[]. II. HIDDE MARKOV MODEL HMM s a probably model used o represen he sasc propery of he sochasc process and s characerzed by model parameers. In order o defne an HMM compleely, followng elemens are needed:., he number of saes of he model. M, he number of mxures n each saes 3. A { a j }, he sae ranson probably marx { } aj P q + j q, j () where q s he sae a me and a j s he ranson probably from sae o sae j ha should sasfy he normal sochasc consrans as: and a,, j () j aj, (3) j 4. B { bj( O)}, he oupu probably dsrbuon where b j (O ) s a fne mxure of Gaussan dsrbuons assocaed wh sae j of he form: //$6. c IEEE 83

2 M (4) m b ( O ) c G( μ, Σ,O ) j Where O s he s he h observaon vecor, c s weghng coeffcen for he mh mxure n sae j, and G s he Gaussan dsrbuon wh mean vecor μ and covarance marx for he mh mxure componen n sae j.c should sasfy he sochasc consrans: U c γ (, ) M m γ ( j, m) γ ( j, mo ) μ γ ( j, m) γ,.( Oμ )( Oμ ) ' γ (, ) () () () And c, j, m M (5) Where π, a j, c, u andu are he model parameers of λ, γ ( j, m) s he probably of beng n sae j a me wh m h mxure componen accounng for O of he form M c, j (6) m 5. { j }, he nal sae dsrbuon ha used o descrbe he probably dsrbuon of he observaon symbol n he nal momen when whch mus sasfy π P(q ), () π herefore we can use he compac λ,, μ, Σ, π o denoe an HMM wh noaon ( Ac ) connuous denses. As menoned above, HMM s used o approxmae he probably of each observaon symbol exsng n he curren sae. We use he Chaos search algorhm o search he opmal model parameers. III. BAUM-WELCH ALGORIHM we usually work wh he Baum-Welch algorhm o esmae he model parameers λ ( Ac,, μ, Σ, π ). hs mehod s an erave algorhm based on expecaon maxmzaon ha searches for an opmal soluon.in addon o he Baum-Welch algorhm, s necessary o esmae he Alfa and Bea marces hanks o he forward and backward procedures. o compue he mos probable sae sequence, he Verb algorhm s he mos suable.he BW re-esmaon formulas are shown as follows[]: a j () α () β () β () α j j β () α π j α( j) a b O ( j) (8) (9) γ (, ) α() β() cgo (, μ, U) M ( j) α β() k cjkgo (, μjk, Ujk) (3) he forward varable s defned as he probably of he paral observaon sequence { O, O,, O } (unl me ) and sae S a me, wh he model, as α of he form α () πb O, α() j αj b( O) < <, j (4) And he backward varable s defned as he probably of he paral observaon sequence form + o he end, gven sae S a me and he model, as β of he form () β, α b ( O ) β () j <, j j + + j IV. CHAOS OPIMIZAIO ALGORIHM (5) Chaos ofen exss n nonlnear sysems. hese sysems are hghly sensve o he nal values so ha very small dfferences n he nal condons wll cause large dfferences n he long-me behavor of sysems. Chaos can go hrough all saes n ceran ranges whou repeon, he long-me moon of chaoc sysems shows some confuson and ypcal sochasc properes. Bu chaos s no really sochasc, has exquse srucure[5, 8, 9]. Wh hese characerscs of chaos, we can apply n Opmzaon 84

3 calculaon. hese chaos opmzaon algorhms n he referenced papers were all based on he Logsc map. he well-known Logsc map s wren as: zn+ f( μ, zn) μzn( zn) (6) Where μ s a conrol parameer, k,,,... and z s a varable, I s easy o fnd ha when μ4,he logsc map s oally n chaos sae and all values beween and excep he fxed pon (.5,.5,.5) are produced randomly by eraon. he smulaon resuls show ha when n >, he nal nformaon of he sysem s compleely los and Is oupu s lke a sochasc oupu. Moreover he probably densy funcon of he chaoc sequences for Logsc map s a Chebyshev-ype one wh very hgh densy near he wo ends of he chaoc varable nerval (, ) and low densy a mddle par, no unformly dsrbued a all. hs ype of densy dsrbuon may affec he global searchng capacy and compuaonal effcency of chaos algorhms remarkably.in [9] an mproved Chaoc opmzaon algorhm was proposed ha cus he pars near he wo ends of chaos varable nerval (, ) durng he chaos searchng. In hs paper we use hs opmzaon mehod o search he opmal model parameers λ. he basc process of chaos opmzaon algorhm generally ncludes wo major seps: A frs a sequence of chaoc pons are generaed based on he logsc map and mapped o a sequence of desgn pons n he orgnal desgn space by he carrer wave mehod. hen, he objecve funcons are calculaed and he pon wh he maxmum objecve funcon s chosen as he curren opmum. Secondly, he curren opmum s assumed o be close o he global opmum afer ceran eraons, and s vewed as he cener wh a lle chaoc perurbaon, and he global opmum s obaned hrough fne search. he above wo seps are repeaed unl some specfed convergence creron s sasfed, hen he global opmum s obaned. V. HE CHAOS-BW RAIIG APPROACH Classcal ranng mehod Baum Welch algorhm could only obans locally opmal soluon f s nal values no properly se, whch mgh decrease he recognon rae [] herefore, he selecon of he nal value s an mporan problem, as well as a roublesome problem. Chaos search algorhm has he funcon of globally lookng for he bes nal values of HMM parameers n Baum-Welch algorhm. I s fas and needs lm sorage because s parameers can be generae by a ceran formula. he seps of CHAOS-BW algorhm are shown a Fg :. nalzaon: Gve nal values n (,) whch have very small dfferences (no he fxed pons of Logsc map.e..5,.5,.5).se erave coun IRCIRM. Calculae he new chaoc varables (IRM ManIR) and map hem o opmzaon varables 3. Searchng for opmal soluon: Else le and calculae he value of he objecve funcon f, f f n++, IRC++ If f ( xn ( + )) f VI. EXPERIMEAL RESULS : For evaluang proposed mehod we used HK oolk 3.4 [] and changed s ranng funcons wh HCHAOS-BW algorhms o compare he expermenal resuls. we used FARSDA daase for connuous speech recognon ha has more han word and conans 6 pronuncaons ha pronounced by 3 speaker (each speaker senences). wo of pronounced senences by each speaker, are common beween all speakers o allow dalec comparson among speakers and overall here are 4 senences n daa. We used he pronounced senences by speakers for ranng and oher speakers for es.he samplng rae of speech sgnal s.5khz, frame me-lengh s 5ms and frame ransfer me s ms. We consder 39 dmenson Mel-frequency cepsral coeffcens (MFCC) ha conss of cepsral coeffcens plus h cepsral parameer and her frs and second dervaves as feaure vecor. Se for logsc map formula and choose nal Chaos varables randomly n range of (.,.).also se ManIR, maxirb and maxirc as, and 3 respecvely. we used lef o rgh HMM s wh 5 saes and ran hem for each phoneme. For evaluang performance of purposed mehod we compared s resuls wh 3 oher mehods PSOBW, ( + ), ( ( + )) X X n f f X n If ( IR max IRC) go o sep. 4. Se X as nal values for BW algorhm and erave coun IRB 5. Apply he BW algorhm.and IRB++ 6. If ( IRB max IRB). IRM++ go o sep 5 8. If ( IRRM ManIR) go o sep, oherwse ermnae Fg.. CHAOS-BW ranng algorhm 85

4 HGAPSOBW and radonal BW. All HMMs raned by four HMM ranng mehods. he average of logarhm of probably PO ( λ ) for each phoneme ( or each HMM) s repored n able. ABLE : HE AVERAGE LOG PROBABILIY FOR EACH MODEL RAIIG WIH EACH MEHOD Model BW BW-Chaos GAPSO_BW PSO_BW sp b p s j c~ h x d z r z~ s~ q f k g l m n v y a e o a~ u e~ he es accuracy and correcon rae n phone recognon for connuous speech by each of ranng mehod are repored n able. Calculaon mehod for correcon and accuracy s based on equaons and 8. In menoned equaons, s he phonemes coun, D s he deleon error, S s he subsuon error and I s he nseron error z ( n+ ) 4 z ( n)( z n ) () -D-S-I Acuracy rae % (8) Also he resul of embedded ranng for connuous speech (phone) recognon (ha ran all models smulaney) for models wh gaussan mxure has been shown n able 3. ABLE 3: CORRECIO AD ACCURACY RAE FOR COIUOUS SPEECH (PHOE) BY EMBEDDED RAIIG ranng H I S D corr acc Mehod BW HGAPSO-BW Chaos_BW VII. COCLUSIO In hs paper, we sudy a hybrd ranng algorhm (COABW) for he connuous HMM n connuous speech recognon based on he Chaos Opmzaon algorhm and he BW algorhm. From he heorecal pon of vew, can acheve he global or nearly global soluons for Comparng he resuls of COA_BW wh Baum-Welch, shows ha he COA_BW ranng approach acheves hgher average log probables and accuracy han oher hree algorhms n open se. Comparng wh GA-BW and PSO_BW ranng mehods, COA_BW ranng approach also acheves beer performance, and has rapd convergence. Also he resul of embedded ranng shown ha can acheves beer accuracy. ABLE. CORRECIO AD ACCURACY RAE FOR COIUOUS SPEECH (PHOE) ranng Mehod BW 643 PSOBW 643 HGAPSO-B 643 W CHAOS-BW 643 H I S D corr acc REFERECES [] Lpng Xue, J.Y., Zhen J,La Jang, "A Parcle Swarm Opmzaon for Hdden Markov Model ranng", n ICSP6 Proceedngs. [] C. W. Chau, S.K., C.K. Du, W.R. Fahrner, "OPIMIZAIO OF HMM BY A GEEIC ALGORIHM", n Proceedngs of he 99 IEEE Inernaonal Conference on Acouscs, Speech, and Sgnal Processng (ICASSP '9). [3] SOG-YI CHE, X.-D.M., JEG-SHYAG PA, "Opmzaon of HMM by he abu Search Algorhm". Journal of Informaon Scence and Engneerng, Vol., pp , 4. [4] B. H. Juang, L.R.R., "Hdden Markov models for speech recognon". echnomercs, Vol. 33, pp. 5-, 99. [5] W. Jang, B.L., "Opmzaon complex funcons by Chaos Search". Cybernecs and Sysems, Vol. 9, pp ,

5 [6] L. Xue, J.Y., Z.J,L. Jang. "A Parcle Swarm Opmzaon for Hdden Markov Model ranng". n ICSP. 6. [] L.R.Rabner, "A uoral on Hdden Markov Models and Seleced Applcaons n Speech Recognon". Proc. IEEE, 989. : p [8] L.H.juan, Z.H.m., M.L.hua, "A new opmzaon algorhm based on chaos". Zhejang Unversy SCIECE A, Vol., pp , 6. [9] D. Yang, G.L., G.Cheng, "On he effcency of chaos opmzaon algorhms for global opmzaon". Chaos, Solons and Fracals, Vol. 34, pp ,. [] X. Zhang, Y.W., Z.Zhao, "A Hybrd Speech Recognon ranng Mehod for HMM Based on Genec Algorhm and Baum Welch Algorhm". IEEE, : p [] S.Young, J.J., J.Odell, D. Ollason, P.Woodland,"he HK Book", verson 3.4 Dsrbued wh he HK oolk. 8