Speech recognition in noise by using word graph combinations

Size: px

Start display at page:

Download "Speech recognition in noise by using word graph combinations"

Dina Bradford
6 years ago
Views:

1 Proceedngs of 0 h Inernaonal Congress on Acouscs, ICA Augus 00, Sydney, Ausrala Seech recognon n by usng word grah cobnaons Shunsuke Kuraaa, Masaharu Kao and Tesuo Kosaka Graduae School of Scence and Engneerng, Yaagaa Unversy, Jonan Yonezawa-Cy, Yaagaa, Jaan PACS: 43.7.Ne ABSTRACT In he racce, he erforance of seech recognon syses s affeced by seech sgnals beng corrued wh varous background s n he envronen. In hs aer, we roose a new word grah cobnaon (WGC) aroach for seech-n- recognon. The a of hs work s o develo a ehod ha would ensure robus seech recognon under varous condons, and n arcular, under he adverse effec of envronenal and ulsve. For hs urose, we develoed a word grah cobnaon (WGC) echnque n whch boh connuous-xure hdden Markov odels (s) and dscree-xure hdden Markov odels (s) are beng used as acousc odels. I has been revously verfed ha a -based syse can ensure sgnfcan roveens n he seech recognon erforance under ulsve condons. We also showed ha he -based syse ndcaed beer erforance n hgh SNR condons and envronenal condons. On he grounds of he above enoned fndngs, we adoed a syse cobnaon aroach n whch boh a and a are used. Wh he roosed ehod, coleenary effecs can be ancaed because he and he exhb dfferen error rends. Aong he exsng cobnaon ehods, whch nclude recognzer ouu vong for error reducon (ROVER) and confuson nework cobnaon (CNC), n our work, we seleced he echnque of WGC. Unlke convenonal cobnaon aroaches, lke ROVER and CNC, he ng nforaon for all word hyoheses s well reserved n he WGC. In he seech recognon exerens we erfored, he roosed syse showed beer erforance han he ROVER-based syse or he baselne syse. In arcular, hs new syse showed coaravely hgher erforance under xed condons. INTRODUCTION In hs sudy, we a o rove seech recognon under nosy condons by usng he word grah cobnaon (WGC) echnque []. In he as sudes, a ulude of echnques has been roosed, all of whch are based on a cobnaon of syses exhbng dfferen error rends. These nclude recognzer ouu vong for error reducon (ROVER) [] and confuson nework cobnaon (CNC) [3], whch are boh oular echnques. The syse cobnaon echnque was orgnally roosed as a eans of cobnng syses fro ulle ses [4]. More secfcally, he a of hs cobnaon was o rove seech recognon erforance by cobnng he resuls fro ndvdual syses develoed by dfferen organzaons. In conras, we red o rove he seech recognon erforance by cobnng several acousc odels whch are all characerzed by dfferen error rends. Soe of our revous work was focused on rovng he erforance of seech-n- recognon hrough he cobnaon of ouus fro connuous-xure hdden Markov odel () and dscree-xure hdden Markov odel () by ROVER [5]. The revous work used he dfferences of he error rends n he resuls of and. uses a xure of dscree dsrbuons whch have a hgh degree of flexbly, and are execed o reresen colcaed shaes such as nosy condons. As a consequence, under ulsve condons, shows a hgher recognon rae han [6]. Moreover, seech recognon error rends dffer beween and because of dfferences n he acousc condons, for exale he sgnal-o- rao (SNR) and he ye of. Therefore, hs new aroach of usng boh and as acousc odels can rove he seech recognon erforance n varous envronens. Ths work consss of cobnng he and he usng he WGC echnque. The dfference beween ROVER and WGC les n he cobnaon of he seech recognon resuls, or he cobnaon of he neredae resuls. On he oher hand, an advanage of WGC over ROVER s ha suors rescorng on he negraed hyohess sace. Furherore, he ng of words s ke exlc n he srucure of he word grah. Therefore, WGC can exec o rovde for a sgnfcanly roved seech recognon erforance when coared o he ROVER syse. In order o verfy he effecveness of he roosed ehod, we erfored seech recognon exerens under varous condons. In he exerens, we used hree yes of s: envronenal sgnals whch were eher saonary or slow-varyng, ulsve n whch he ower and secral feaures ay radcally change whn a very shor e, and xed where he envronenal and he ulsve were xed arfcally. ICA 00

2 3-7 Augus 00, Sydney, Ausrala Proceedngs of 0h Inernaonal Congress on Acouscs, ICA 00 ACOUSTIC MODELS In hs aer, and are used as acousc odels for syse cobnaon. Ths secon rovdes a dealed descron of hese odels. Connuous-xure hdden Markov odels (s) In seech recognon, s have been wdely used as acousc odels, n whch he ouu robably densy s odeled by a xure of Gaussan dsrbuons as follows: ( ) w ( o, ). b o Ν µ () Here, N( µ, ) o s a Gaussan dsrbuon wh ean and covarance, whle µ w s he xure wegh of he h dsrbuon. In hs aer, he dagonal covarance was used n consderaon of couaonal cos. Dscree-xure hdden Markov odels (s) The s a ye of dscree hdden Markov odel (DHMM) ha was orgnally roosed by Takahash e al. o reduce couaonal cos n decodng rocesses [7]. More secfcally, wo yes of he s have been roosed. In he frs scalar-based quanzaon s beng used [7] and subvecor-based quanzaon n he second [8]. In hs aer, we have eloyed subvecor-based s. In he subvecor-based ehod, he feaure vecor s aroned no S o o,..., o,..., o and vecor quanzaon subvecors, [ ] s ( VQ ) codebooks are rovded for each subvecor. Subsequenly, he feaure vecor q ( o ) [ q ( o ),..., q ( o ),..., q ( o )] s s S o s quanzed as follows:, () where s( s) The ouu dsrbuon of he, ( ) S q o s he dscree sybol for he s h subvecor. he exresson: b ( ) w s( q s( o S) ) where s S b o, s gven by o ˆ, (3) w s he xure coeffcen for he h xure n sae, and ( ( o )) ˆ s q s s he robably of he dscree S sybol for he s h subvecor. In he reander of hs secon, we descrbe he ehod used for he esaon of he araeers based on he axu a oseror (MAP) creron. An esae of he axu lkelhood (ML) of he dscree robably s ( k) s calculaed wh he use of he followng exresson: s ( k) T γ δ T ( q ( o ), k) γ s where k s he ndex of he subvecor codebook and γ s he robably of he h xure coonen beng n sae s, (4) a e. If we assue ha he ror dsrbuon s reresened by he Drchle dsrbuon, he esae of he ˆ k based on he MAP creron s gven by s ( ) he followng exresson: τ 0 ˆ s s( k) ( k) + n s( k), τ + n where 0 s ( k) s he consraned ror value of he dscree robably and τ ndcaes he relave balance beween he corresondng ror value and he observed daa. In our exerens, τ was se o 0.0 based on he resuls of coarave exerens. Alhough boh he xure coeffcen and ranson robably can be esaed by eans of he MAP creron, only he ouu robably s beng esaed n hs aer. Coensaon of s To rove he robusness of seech recognon, a coensaon ehod for dscree dsrbuons s aled. I s ore lkely ha a sgnfcan reducon of he ouu robably wll aear under severe sach condons caused by unknown. Ths ehod can allevae he adverse effec of he unknown durng he decodng rocess. If ˆ o, n Eq. (3) s one of he subvecor robables, ( ( )) close o 0, he ouu robably, ( ) s q s s (5) bo, wll also be close o 0. In hs case, wll have a derenal effec on he decodng rocess, even f he e lengh of exosure o s shor. In he coensaon of s, a hreshold s se for he dscree robably, and he derenal effec of s reduced. The coensaon ehod can be descrbed as follows: f n Eq. (3) ˆ s ( qs( os) ) < dh, he ouu robably s se o dh, where dh s he hreshold value for he subvecor. The hreshold was se o n hs aer. WORD GRAPH COMBINATION (WGC) Overvew In hs secon, we descrbe he syse cobnaon aroach roosed for seech-n- recognon. I as o rase he seech recognon rae by cobnng he ouu of wh ha of. The syse cobnaon aroach has been roven o resul n sgnfcan roveens f he seech recognon resuls are subsanally dfferen beween syses. Snce he seech recognon resuls fro s and s are dfferen beween each oher under nosy condons, he erforance s execed o rove. The rocedure followed for WGC s rovded below.. An nu seech s decoded usng wo acousc odels (AM, AM), and a bgra language odel. Then, wo word grahs, WG and WG, are obaned by he decodng rocess.. The wo word grahs (WG, WG) are cobned o for one sngle word grah, WG. 3. The word grah WG s rescored usng he wo acousc C odels and a rgra language odel. In hs se, he wo scores obaned by AM and AM are erged o oban one sngle score. Furher deals of he scorng rocess are avalable n he nex secon. The rocedure of he WGC s shown n Fgure. C ICA 00

3 3-7 Augus 00, Sydney, Ausrala Proceedngs of 0h Inernaonal Congress on Acouscs, ICA 00 LM bgra LM rgra Inu seech Seech analyss Frs ass:bea search WG WGC WG WGc Second ass:rescorng Recognon resuls Fgure. Block dagra of he WGC echnque. Algorh of WGC AM AM AM AM Suose ha here are N word grahs, W, W,..., W, o be N cobned. If wo arcs, q n W and q n W, are equal, he wo word grahs, W and W, can be cobned as follows: W + W { q q + q q q } U{ q q W } U{ q q W }. Two equal arcs are defned as equal when hey have he sae word ID, sar e and end e. A dealed descron of he algorh behnd WGC has been rovded by Chen and Lee []. Scorng ehods In hs secon, wo yes of scorng ehods are descrbed. They are denoed by he ers "average score" and "weghed score". Accordng o he "average score" ehod, he erged score P for he h edge s gven by he exresson: N k P, (7) N k where N s he nuber of syses, and (6) k s he score for he h edge n syse k. In hs sudy, N has been se o wo. The "weghed score" of h edge s calculaed as follows: P ( α ) P + αp, (8) where P s he score calculaed by s, and P s he score rovded by s, whle α ndcaes he balance beween s and s. EXPERIMENTAL SETUP We used he "Jaanese Newsaer Arcle Senences" (JNAS) as ranng and es daa. More secfcally, wo ses of ranng daa were used; one for clean ranng, and he oher for ul-condon ranng [9]. The ranng daa se consss of 5,73 Jaanese senences uered by 0 ale seakers. For clean ranng, no had been added o he daa. On he oher hand, for ul-condon ranng, all hese uerances were dvded no 0 subses. No was added o four subses. In he reander of he daa, was arfcally added. Four yes of ran, crowd, car and exhbon hall were seleced and added o he uerances a a SNR of 0, 5, 0 and 5 db. We used four yes of acousc odels; clean condon s ( ), clean condon s ( ), ul-condon s ( ) and ul-condon s ( ) n our seech recognon exerens. The ranng ehod eloyed for each one of hese acousc odels s exlaned below. The s were raned by ML esaon usng he clean ranng daa. The s were raned by MAP esaon usng clean ranng daa. The nal odels used for her ranng were derved fro he converson of he s no s. The s were raned by ML esaon usng ul-condon ranng daa and s were used as nal odels. The s were raned by MAP esaon usng ul-condon ranng daa. In hs case, he nal odels were obaned by converng s no s. The hree yes of es ses used were as follows. Tes se for envronenal Four yes of saon, facory, sree crossng and elevaor hall were added o 00 senences uered by 0 ale seakers a a SNR of 0, 5, 0 and 5 db. These s were dfferen fro hose of he ranng daa. Tes se for ulsve The ulsve sgnals were added o 00 senences uered by 0 ale seakers. Three yes of ulsve were seleced fro he Real World Coung Parnersh (RWCP) daabase [0], naely: whsle3 blowng a whsle; clas handclas; bank hng a con bank. These sgnals were added o seech daa a nervals of sec and a SNR of 0 db. The SNR was calculaed as he average ower of he seech daa dvded by he axu ower of he ulsve. The axu ower was deerned fro ower values ha were calculaed fro he ulsve daa every 30 sec. Tes se for xed Nose sgnals fro above wo es ses were xed arfcally o ake a new es se. Four yes of s were reared a a SNR of 0 db as envronenal s. These s were xed wh hree ulsve s. Thus, welve yes of s were used for evaluaon of he roosed syse. The seech analyss condons are suarzed n Table. The srucure of and was 000-sae HMne (se of shared sae rhones), and he nuber of xure coonens was 6. Table suarzes he subvecor allocaon and he codebook sze for. Alhough and have been oed fro he able, all he codebooks were desgned n he sae anner. A wo-ass search decoder usng a bgra and a rgra was used n seech recognon. Decodng was erfored n he frs ass by eans of a one-ass algorh, n whch a fraesynchronous bea search algorh and a ree-srucured lexcon were aled. The bgra and rgra odels were raned usng 45 onhs of newsaers arcle senences. The raned language odels had 5 K word enres. In he case of ROVER, 50 dfferen ouus are cobned. These 50 ouus were obaned by varyng araeers such as language wegh and word nseron enaly. More secf- ICA 00 3

4 3-7 Augus 00, Sydney, Ausrala Proceedngs of 0h Inernaonal Congress on Acouscs, ICA 00 Salng frequency Quanzaon Frae lengh Frae erod Analyss wndow Feaure vecor Noralzaon Table. Seech analyss condons 6 khz 6 b 3 sec 8 sec Hang wndow MF (-), log ower+ + (oal of 39 densons) CMN Table. Codebook sze for each subvecor Paraeer logp C, C C 3, C 4 C 5, C 6 C 7, C 8 C 9, C 0 CB sze C, C Table 3. Values of language wegh and word nseron enaly for each acousc odel and condon Envronenal Iulsve Mxed language wegh ~ 4 nseron enaly -8 ~ -64 language wegh ~ 4 nseron enaly -8 ~ -48 language wegh ~ 33 nseron enaly -48 ~ -80 language wegh 8 ~ 34 nseron enaly -40 ~ -7 language wegh 6 ~ 8 nseron enaly 0 ~ -7 language wegh 4 ~ 30 nseron enaly -4 ~ -66 cally, fve dfferen values were used for he language wegh and fve values for he word nseron enaly. Naely, yes of araeer ses were reared for boh he s and he s. In he end, a oal of 50 ouus were cobned. The values of language wegh and nseron enaly n each condon and acousc odel are suarzed n Table 3. These araeers were se based on he resuls of ror exerens. RESULTS AND DISCUSSION Seech recognon exerens under envronenal condons The exerenal resuls of he cobnaon of he wh he under envronenal Table 4. WERs (%) under envronenal condons SNR (db) Before cobnaon Srucure cobnaon Ave Score cobnaon Whou weghng α 0.9 ROVER Ave WER (%) α Fgure. Relaon beween he araeer α and he WER (%) under envronenal condons condons are suarzed n Table 4. Here, he word error rae (WER) s rovded for each odel used. The er srucure cobnaon eans ha he srucures of wo word grahs were cobned, bu no he scores. On he oher hand, score cobnaon eans ha boh he srucures and scores were cobned. The seech recognon resuls of ROVER are rovded for coarson. Based on he resuls, we concluded ha boh he srucure and he score cobnaon are effecve. In he case of score cobnaon, he erforance acheved wh weghng was slghly beer han whou weghng. The bes erforance can be obaned a α 0.9 (see Fgure ). The reason why a large wegh should be aled o he s ha he erforance obaned wh he use hs odel was beer han whou cobnaon. Moreover, he erforance obaned wh he roosed syse was beer han he ROVER syse. On he ground of he above enoned resuls, we concluded ha he WGC aroach s effecve. 4 ICA 00

5 3-7 Augus 00, Sydney, Ausrala Proceedngs of 0h Inernaonal Congress on Acouscs, ICA 00 Nose Table 5. WER (%) under ulsve condons Before cobnaon Srucure cobnaon bank clas whsle Ave Score cobnaon Whou weghng α 0.9 ROVER bank clas whsle Ave Nose Table 6. WER (%) under xed condons Before cobnaon Srucure cobnaon bank clas whsle Ave Score cobnaon Whou weghng α 0.8 ROVER bank clas whsle Ave WER (%) α Fgure 3. Relaon beween he araeer α and he WER (%) under ulsve condons Seech recognon exerens under ulsve condons The exerenal resuls fro he cobnaon of he wh he under ulsve condons are suarzed n Table 5. The relaon beween he araeer α and WER obaned wh he score cobnaon ehod s shown n Fgure 3. We used clean condon odels, because he duraon of all he segens was very shor and each uerance was alos enrely free of under ulsve condons. Based on he exerenal resuls, we concluded ha wh he before cobnaon and he afer he score cobnaon we can oban aroxaely he sae WER value (4.80%). Under ulsve condons, he WGC ehod dd no resul n any roveen n he seech recognon erforance. Ths was arbued o he dfference n erforance obaned wh and ha of before cobnaon. If he syse could have known n advance ha he s ulsve, would be recoended o use s. However, he syse generally can no redc he condons. Therefore, he use of he WGC aroach s suable even f he seech recognon erforance does no rove under ulsve condons. WER (%) α Fgure 4. Relaon beween he araeer α and he WER (%) under xed condons Seech recognon exerens under xed condons The exerenal resuls fro he cobnaon of he wh he are suarzed n Table 6. The relaon beween he araeerα and WER s shown n Fgure 4. Each resul reresens an average WER of he four xed s cobned every e wh one of he ulsve s. Under xed condons, boh he srucure cobnaon and he score cobnaon are effecve as well as envronenal condons descrbed n he revous secon. By coarson, he WGC aroach was ore effecve under xed condons han under envronenal condons. Based on hese fndngs, we concluded ha he WGC echnque s effecve n varous envronens. SUMMARY OF SPEECH RECOGNITION EXPERIMENTS The exerenal resuls for he hree dfferen seech recognon odes (before cobnaon, srucure cobnaon and score cobnaon) are suarzed n Table 7. Based on hese resuls, we concluded ha boh he srucure cobnaon and he score cobnaon are effecve. In arcular, he ICA 00 5

6 3-7 Augus 00, Sydney, Ausrala Proceedngs of 0h Inernaonal Congress on Acouscs, ICA 00 Table 7. Suary of WERs (%) for hree recognon odes Before cobnaon Nose Envronenal.94. Iulsve Mxed Envronenal Srucure cobnaon.3.34 Iulsve Mxed Envronenal Score cobnaon Whou weghng Wh weghng.5.8 (α 0.9) Iulsve (α 0.9) Mxed (α 0.8) on Audo, Seech and Language Processng, 4, (006) 5 T. Kosaka, Y. Sao and M. Kao, Nosy seech recognon by usng ouu cobnaon of dscree-xure HMMs and connuous-xure HMMs In Proc. of Inerseech 009, (009) 6 T. Kosaka, M. Kaoh and M. Kohda, Robus seech recognon usng dscree-xure HMMs IEICE Transacons on Inforaon and Syses, E88-D, 8-88 (005) 7 S. Takahash, K. Akawa, and S. Sagayaa: Dscree xure HMM In Proc. of ICASSP97, (997) 8 S. Tsakalds, V. Dgalaks, and L. Neweyer, Effcen seech recognon usng subvecor quanzaon and dscree-xure HMMs In Proc. of ICASSP99, (999) 9 D. Pearce and H.-G. Hrsch: The AURORA exerenal fraework for he erforance evaluaon of seech recognon syses under nosy condons In Proc. of ICSLP000, vol.4,. 9-3 (000) 0 S. Nakaura, K. Hyane, F. Asano, T. Nshura, and T.Yaada: Acouscal sound daabase n real envronens for sound scene undersandng and hands-free seech recognon In Proc. of ICLRE, (000) bes ossble erforance could be acheved by usng he score cobnaon wh he score weghng ehod. CONCLUSIONS In hs aer, we roose a WGC ehod n whch acousc odels ha exhb dfferen error rends can be cobned n order o rove he erforance of seech-n- recognon. Based on exerenal resuls obaned under envronenal condons and xed condons, we concluded ha he WGC ehod can ensure sgnfcan roveens n he erforance of seech recognon. More secfcally, beer seech recognon erforance could be obaned wh he roosed cobnaon ehod han wh ROVER, he convenonal cobnaon ehod. Whle our exerenal resuls sugges ha he WGC aroach s effecve under varous condons, he cobnaon of weghed scores s also effecve. The nex se n hs lne of research would be o cobne he ROVER wh he WGC ehod n order o acheve furher roveens. In addon, we have sared workng on he auoac esaon of he weghng coeffcens used n he score cobnaon. REFERENCES I-F. Chen and L.-S. Lee, A new fraework for syse cobnaon based on negraed hyohess sace In Proc. of Inerseech-006, (006) J. G. Fscus, A os-rocessng syse o yeld reduced word error raes: Recognzer ouu vong error reducon (ROVER) In Proc. of IEEE Worksho on Auoac Seech Recognon and Undersandng, (997) 3 L. Mangu, E. Brll, and A. Solcke, Fndng consensus n seech recognon: word error nzaon and oher alcaons of confuson neworks Couer Seech and Language, 4, (000) 4 M.J.F. Gales, D.Y. K, P.C. Woodland, H.Y. Chan, D. Mrva, R. Snha and S.E. Traner, Progress n he CU- HTK broadcas news ranscron syse IEEE Trans. 6 ICA 00

A New Method for Computing EM Algorithm Parameters in Speaker Identification Using Gaussian Mixture Models

A New Method for Computing EM Algorithm Parameters in Speaker Identification Using Gaussian Mixture Models 0 IACSI Hong Kong Conferences IPCSI vol. 9 (0) (0) IACSI Press, Sngaore A New ehod for Comung E Algorhm Parameers n Seaker Idenfcaon Usng Gaussan xure odels ohsen Bazyar +, Ahmad Keshavarz, and Khaoon