Candidate Generation for Interactive Chinese Speech Recognition

Transcription

1 Canddat Gnraton for ntractv Chns Spch Rcognton Xnhu L 1 2 Xangdong Wang 1 Yulang Qan 1 Shouxun Ln 1 1 Ky Laboratory of ntllgnt nformaton Procssng nsttut of Computng Tchnology Chns Acadmy of Scncs Bng Chna 2 Graduat Unvrsty of Chns Acadmy of Scncs Bng Chna { lxnhu xdwang ylqan sxln}@ct.ac.cn Abstract Dspt progrss n automatc spch rcognton usr nvolvmnt and assstanc ar stll ndd to achv prformanc smlar to human n ral-world tasks. Though many ntractv spch rcognton systms usng canddat slcton ar dvlopd for Englsh and Japans smlar systms for Chns fac partcular dffcults du to charactrstcs of th Chns languag spcally at th canddat gnraton stag. n ths papr a mthod s proposd for canddat gnraton n ntractv Chns spch rcognton whch lsts all compttv canddats n forms of Chns charactrs n stad of words. Th man da s to algn tm-ovrlappd lnks n Chns word lattc nto clustrs basd on thr phontc smlarty and thn obtan canddats by splttng words attachd to lnks nto Chns charactrs. Exprmntal rsults dmonstratd that th canddats gnratd by th proposd mthod can lad to hgh prformanc n ntractv Chns spch rcognton. Kywords: spch rcognton rror corrcton canddat gnraton ntractv spch rcognton. 1. ntroducton Automatc spch rcognton has mad grat progrss ovr th past fw dcads. Howvr for tasks such as larg vocabulary contnuous spch rcognton (LVCSR) th stat-of-art tchnqu s stll far from ral-world applcaton. As a mattr of fact t sms vry dffcult for computr to achv prformanc smlar to human through th mprovmnt of rcognton tchnology n nar futur. Ths mpls that nvolvmnt and assstanc of human usrs ar ndd to achv hgh accuracy n spch rcognton. W rfr to spch rcognton wth nvolvmnt and assstanc of usrs as ntractv spch rcognton. A maor tchnqu n ntractv spch rcognton s rror corrcton or spch rpar va canddat slcton [1-4]. Undr th schm of canddat slcton th spch rcognton systm outputs not only th bst word squnc as rcognton rsult but also a lst of compttv canddats for ach word (not for a sntnc) and th usr can corrct th rcognton rrors by slctng rght canddats. For xampl a canddat lst for th uttranc w nd som hlp s shown n Fg. 1. Each word n th bst word squnc s sparatd and ts compttv canddats ar lstd blow t. W rfr to ach word and ts compttv canddats as a rcognton sgmnt. Th usr can corrct a rcognton rror by slctng th rght compttv canddat n th sgmnt. t s rportd that much mprovmnt n accuracy can b achvd by canddat slcton [4]. Bst squnc Compttv canddat Sgmnt WE NEED SUN HELP WLL LEAD SOME SON Fgur 1. Canddat lst for uttranc w nd som hlp. For th gnraton of compttv canddats most systms us th confuson ntwork (CN) algorthm [5-7] whch obtans th canddats by transformng word lattc nto CN. Durng th dcodng stag spch rcognton systms gnrat a word lattc as an ntrmdat rsult and sarch n th lattc for a bst word squnc as fnal rcognton rsult. A word lattc s a Drctd Acyclc Graph (DAG) whch contans a larg numbr of word hypothss dnotd by corrspondng lnks and thr assocatd lklhood scors. Fg. 2 gvs an xampl of an Englsh word lattc. To form th confuson ntwork lnks on a word lattc ar algnd accordng to tm nformaton and th lattc s transformd nto a lnar graph. Th transformaton s prformd by a clustrng procdur that groups tm ovrlappd lnks nto clustrs basd on thr phontc smlarty and word probablts whl prsrvng th prcdnc ordr of th lnks ncodd n th word lattc. Fg. 3 gvs th CN obtand by transformng th word lattc of Fg /09/$ EEE 583 1

2 SL VERY FNE T HAVE VEAL FAST SL MOVE VERY SL OFTEN T Fgur 2. An Englsh word lattc. HAVE MOVE T VEAL VERY FNE OFTEN FAST Fgur 3. CN of th word lattc n Fg. 2. Though thr ar many ntractv spch rcognton systms usng canddat slcton for Englsh and Japans [1-4] as to our knowldg thr ar no smlar systms for Chns spch yt. Th man challng for canddat slcton to b usd n Chns spch rcognton ls n that th CN algorthm cannot b usd for canddat gnraton du to charactrstc of th Chns languag. n Chns word lattc th word hypothss attachd to ach lnk may b a sngl Chns charactr or a strng of Chns charactrs (n gnral nclud 1-4 sngl Chns charactrs) and compttv rlatonshp may xst btwn two charactrs whch ar both parts of dffrnt word hypothss. Thrfor t s mpossbl to group all th word hypothss wth compttv rlatonshp nto a clustr by transformng Chns word lattc nto Chns CN. Fg. 4(a) gvs an xampl of Chns word lattc and Fg. 4(b) shows th Chns CN obtand by transformng th word lattc of Fg. 4(a). n Fg. 4(b) th word hypothss compts not only wth othr word hypothss n th sam clustr but also wth othr word hypothss n othr clustrs (lk bold word hypothss n th Fgur). n fact du to th charactrstc of Chns languag that both a sngl charactr and a strng of charactr can form a word t s dsrd by usrs that canddats can b lstd n trms of charactrs n stad of words whch cannot b ralzd by usng CN. n ordr to gnrat sutabl canddats for ntractv Chns spch rcognton consdrng th charactrstc of th Chns languag w propos an approach of canddat gnraton for ntractv Chns spch rcognton whch lsts all compttv canddats n forms of Chns charactrs n stad of words. Frst smlar to gnraton of CN tmovrlappd lnks n Chns word lattc ar algnd nto clustrs basd on thr phontc smlarty. And thn canddats ar obtand by splttng words attachd to lnks nto Chns charactrs. Exprmntal rsults show that th canddats gnratd by th proposd mthod can achv hgh covrag of corrct charactrs rsultng n hgh prformanc n ntractv Chns spch rcognton. Fgur 4. Chns word lattc and corrspondng CN. Th rst of th papr s organzd as follows. Scton 2 ntroducs th man da and procdur of th proposd mthod. n Scton 3 algorthms for canddat gnraton ar dscrbd n dtal. Exprmntal rsults of th proposd mthod ar prsntd n Scton 4. And fnally conclusons ar drawn and futur work ar dscussd n Scton Man da of th Proposd Approach 2.1. Prncpls of Canddat Gnraton n ordr to nabl usrs to quckly and ffcntly corrct rcognton rrors by canddat slcton th canddat should mt th followng prncpls: 1. Canddats wth compttv rlatonshp should b groupd nto th sam rcognton sgmnt. n ths cas th usr can fnd th rght compttv canddat wthn a small ara. 2. Th ordr of th rcognton sgmnts should b consstnt wth rcognton tm. 3. Th compttv canddats of ach rcognton sgmnt should b sortd by th valu of th postror probablts whch can b computd from word lattc. Ths mans th plausbl canddats ar locatd at uppr postons n th rcognton sgmnt. t can b sn that for Chns spch th CN gnratd from Chns word lattc dos not follow prncpls 1 and 3 snc compttv rlatonshp actually xsts btwn charactrs nstad of words n Chns Procdur of th Proposd Mthod n ordr to gnrat Chns canddats that mt th prncpls of canddat gnraton w propos a 584 2

3 Chns canddat gnraton mthod whch lsts canddats n charactrs. Frst an algnd ntwork s gnratd by algnng th lnks n Chns word lattc and thn charactr canddats ar gnratd by splttng Chns words n th algnd ntwork nto charactrs. Fg. 5 gvs an llustraton of Chns canddat gnraton. algnd clustrs n algnd ntwork and k dnots th st of lnks n th k-th clustr. (3) Chns canddats A Chns canddat lst s dnotd by C { C 0 C 1 C 2 C L } whch s th st of all rcognton sgmnts n Chns canddat whr C l { c0 c1 c2 ck} dnots th st of all canddats n th l-th rcognton sgmnt. Each c k C s dfnd as { W P } whr s th l word of canddat s th postror probablty c k of canddat. c k P ck ck ck W ck 3.2. Algorthm for Algnmnt of Chns Word Lattc n th frst stag th word lattc s algnd to form a lnar algn ntwork by clustrng lnks n th lattc. Th lnks whch mt th followng condtons ar clustrd nto on class: (1) th last Chns charactrs attachd to ach lnk ar phontcally smlar; (2) Th lnks ar tm-ovrlappd. Th algnmnt of Chns word lattc can b dscrbd as follows: 3. Algorthms for Canddat Gnraton for ntractv Chns Spch Rcognton 3.1. Assumpton and Dnotaton Frst w gv som assumptons and dnotatons usd by followng dscrpton. (1) Chns word lattc A Chns word lattc s dnotd by L=<N E> whr N { n0 n1 n2 n } s th st of all nods n Chns word lattc and E { J } s th st of all lnks n Chns word lattc. n N tn ( ) dnots th tm mark. Each k E s dfnd as { S F W A L } whr s k k k k k k F k th start nod of lnk s th nd nod of lnk W k s th word on th lnk k and ar th k k acoustc lklhood scor and th languag modl scor rspctvly. (2) Algnd ntwork An algnd ntwork s dnotd by { } whch s th st of all E E E E E A Fgur 5. Ovrvw of Chns canddat gnraton K A L S k k Stp-1: Calculat th postror probablty p() of ach lnk n th lattc usng forwardbackward algorthm [5]. Stp-2: Sort all lnks n E by th ordr of ncrasng tf) ( k and sort th lnks wth sam tf ( k ) by th ordr of ncrasng ts ( k ). Stp-3: ntalz algnd ntwork: for ach lnk E f ts ( ) 0 0. Stp-4: For ach lnk E 1 2 suppos that th prvous lnk s dnotd by 1 (a) f ts ( ) ts ( ) and tf ( ( ) 1 ) tf 1. (b) f E such that ts ( ) tf) ( 1 1. Also for ach lnk f SM( ) SM( ) 1 1 and \ whr SM( ) smc (() c ( ) ) ovrlap ( ) whr c( ) and c( ) ar th last Chns charactrs corrspondng to and sm (..) s th phontc smlarty btwn two charactrs computd from th most lkly phontc bas forms and ovrlap( ) s 585 3

4 dfnd as th tm ovrlap btwn and normalzd by th sum of thr lngths. (c) f K and K such that mn{ u ( El lk1 ts ( ) tf ( ) and )} ) u ( whr u() s th numbr of Chns charactrs attachd to th lnk. (d) f and K such that ts ( ) tf ( ) and mn{ u ( )} u ( ) K 1 1. () f and K such that K ts ( ) tf ( ) E 1 l lk and mn{ u ( )} u ( ) E 1 l lk H H K H whr H s dtrmnd by th followng quaton 1 H arg max{ SM( )} KH we ( ) H E H whr w ( H) s th numbr of lnks n H SM( ) rman th sam as bfor. Stp-5: n ach algnd clustr mrg all lnks wth th sam Chns word nto a sngl lnk whos postror probablty quals to th sum of postror probablts of all lnks mrgd. whr QW ( s th -th charactr n th ) Chns word attachd to lnk. (b) f u( ) > num lt th word of canddat c b W c = QW ( and th postror ) probablty of canddat c b Pc = p ( ) st C m num u ( ) C mnum u( ) c 01 u( ) 1. Stp-3: nn1 m m num. f n w( E A ) rturn to Stp-2; othrws nd procssng. Stp-4: n a rcognton sgmnt mrg all canddats wth th sam Chns charactr nto on sngl canddat whos postror probablty quals to th sum of postror probablts of all canddats mrgd. Fg. 6(b) shows charactr canddats gnratd by splttng th algnd ntwork of Fg. 6(a). As can b sn all th word hypothss splts nto Chns charactrs and Chns charactrs wth compttv rlatonshp ar groupd nto th sam rcognton sgmnt. Fgur 6(a) gvs th algnd ntwork gnratd by algnng th Chns word lattc of Fgur 4(a) Algorthm for Charactr Canddat Gnraton Basd on th algnd ntwork charactr canddat ar gnratd by splttng words nto charactrs and rordrng all canddats accordng to postror probablts. Th algorthm can b dscrbd as follows: Stp-1: ntalz: n=0 m =0. Stp-2: lt num = mn{ u ( )} u ( ) rmans th (a) f u = E n sam as bfor. For ach lnk n 1 2. ( ) num lt th word of canddat c b W c = QW ( ) and th postror probablty of canddat c b Pc = p ( ) st C C c 01 num1 m m Fgur 6. Exampl of Chns canddat gnraton 4. Exprmntal rsults To tst th ffctvnss of th proposd canddat gnraton mthod for ntractv Chns spch rcognton spch rcognton and canddat gnraton ar conductd on a contnuous Chns spch corpus known as th 863 Corpus. Th corpus contans data of 80 spakrs and 520 uttrancs ar avalabl for ach spakr. Data of 50 spakrs ar usd as tranng st and 450 sntncs xtractd from th rst data ar usd for tstng. MFCC faturs and tr-phon 586 4

5 HMMs ar usd for acoustc modl and th languag modl s a b-gram modl wth 120k words trand on 25M Popl s Daly Corpus. To masur th rror corrcton prformanc by usng th proposd algorthm to gnrat Chns canddats covrag rat of corrct charactrs n th top N canddats ar usd as mtrcs whch s also rfrrd to as charactr corrct rat. Th covrag rats for dffrnt numbrs of canddats gnratd by th proposd algorthm ar shown n Tabl 1. Th Top-N rprsnts th N-bst canddats and th Top-1 has no compttv canddats xcpt th bst word. t can b sn from th tabl that word corrctnss was mprovd whn th numbr of canddats ncrasd. Th word corrctnss fnally achvd was 94.06% and most rronous words could b corrctd whn th numbr of canddats was tn. Ths rsults show that th canddat gnratd by th proposd algorthm s suffcntly ffcnt to nabl a usr to corrct rcognton rrors. Tabl 1. Word corrctnss wth dffrnt numbr of canddats Numbr of Canddats Top % Top % Top % Top % all 94.06% Charactr Corrct Rat Acknowldgmnt Ths work s sponsord by Natonal Ky Tchnology R&D Program of Chna (Grant No. 2008BAH26B03). Rfrncs [1] Suhm B. Myrs B. & Wabl A. Dsgnng ntractv rror Rcovry Mthods for Spch ntrfacs Procdngs of ACM CH 1996 Workshop on Dsgnng th Usr ntrfac for Spch Rcognton applcatons. [2] Brnhard Suhm Emprcal Evaluaton of ntractv Multmodal Error Corrcton Proc. EEE Workshop on Spch rcognton and Undrstandng [3] Karat C. Halvrson C. Horn D. and Karat Pattrns of Entry and Corrcton n Larg Vocabulary Contnuous Spch Rcognton Systms Proc. CH [4] Ogata J. Goto M Spch rpar: quck rror corrcton ust by usng slcton opraton for spch nput ntrfacs. n: Proc. ntrspch pp [5] L. Mangu E. Brll and A. Stolck Fndng consnsus n spch rcognton: word rror mnzaton and othr applcaton of confuson ntwork Computr Spch and Languag vol.14 (4) pp [6] L. Mangu Fndng Consnsus n Spch Rcognton PhD Thss Johns Hopkns Unvrsty [7] J. Xu and Y.-X. Zhao mprovd confuson ntwork algorthm and shortst path sarch from word lattc CASSP 2005 vol.1 pp Conclusons n ths papr th prncpls of canddat gnraton ar dscussd and a canddat gnraton algorthm for ntractv Chns spch rcognton s proposd. Unlk mthods for Englsh spch whch ar not sutabl for Chns spch th proposd mthod lsts all compttv canddats n forms of Chns charactrs n stad of words. Th man da s to algn tm-ovrlappd lnks n Chns word lattc nto clustrs basd on thr phontc smlarty and thn obtan canddats by splttng words nto Chns charactrs. Exprmntal rsults hav shown that th canddat gnratd by th proposd algorthm can b usd to corrct most rcognton rrors. n futur work bas on th Chns canddat gnratd by th proposd algorthm w ntnd to us mor ntractv nformaton to mprov th rror corrcton prformanc of ntractv Chns spch rcognton

6 Ths full txt papr was pr rvwd at th drcton of EEE Communcatons Socty subct mattr xprts for publcaton n th WCNC 2008 procdngs. Numbr of rsolvd nods Fg ML wth Out-of-Rang ML Numbr of nods n th ntwork Th ffct of nod dgr n locaton dscovry nods rmans all th sam n th scnaro of th ntwork wth 24 nods. Th rason bhnd s that nods mult-hop away may b gographcally too far away from th unknown nod U so that thy ar out of th rang of any possbl locaton U mght rsd at. Thrfor 2 or 3 can b usd as a practcal sttng of h. C. Effct of avrag nod dgr Th numbr of addtonal nods locatd usng Out-of-Rang nformaton dpnds on th ntwork connctvty and topology. Fg 8 prsnts th numbr of rsolvd nods aftr locaton dscovry wth Out-of-Rang and wthout Out-of-Rang nformaton n a st of scnaros. Th total numbr of nods n th ntwork ncrass from 16 to 48 whl th numbr of rfrnc nods rmans to b 4. Th avrag nod dgr n ach scnaro s lstd n Tabl. t s worth notng that th avrag nod dgr dos not always ncras as th numbr of nods ncrass n th ntwork. Although aganst ntuton a furthr look at th topology shows that t could actually occur. t stms from th fact that nods ar unformly placd ovr th ntr ara. n th unform placmnt th ara s dvdd nto a numbr of clls and nods ar randomly placd wthn ach cll. A slght ncrasng of th numbr of nods can rsult n an addtonal cll wth only fw nods placd n t. TABLE AVERAGE NODE DEGREE g N A NETWORK OF n NODES n g Whn th ntwork has a total of 16 nods no othr nod can b locatd by mult-latraton or our schm snc th ntwork s parttond. No Out-of-Rang nformaton s usful n ths scnaro du to th lack of connctvty. Our schm rsolvs mor nods locaton than th basc mult-latraton schm as th connctvty ncrass. As th ntwork bcoms hghly connctd almost all nods can rsolv thr locaton usng th basc mult-latraton schm. As a rsult no Out-of-Rang nformaton s ndd n ths cass. Thrfor th proposd schm can rsolv a larg numbr of addtonal snsor nods locaton usng Out-of-Rang whn th ntwork s sparsly yt suffcntly connctd. V. CONCLUSON AND FUTURE WORK n mult-latraton basd localzaton schms for snsor ntworks ach unknown nod must gan suffcnt nformaton such as thr rfrnc nods n ordr to dscovr ts locaton. n undr watr snsor ntworks howvr t s not uncommon that an unknown snsor nod dos not hav thr or mor rfrnc nghborng nods du to th spars ntwork topology and lmtd avalablty of rfrnc nods. As a rsult many nods can not dscovr thr locatons usng th mult-latraton schm. n ths papr w propos to utlz th Out-of-Rang nformaton at both rfrnc nods and unknown nods to dtrmn snsor nods locaton n UWSNs whn suffcnt nformaton s not avalabl. t s shown that an unknown nod wth only two nghborng rsolvd nods can dtrmn ts locaton n crtan scnaros. Th smulaton rsults show that th proposd schm can sgnfcantly ncras th numbr of rsolvd snsor nods aftr locaton dscovry complts whn th ntwork connctvty s low. Th ncras can b as bg as 50% n som scnaros. W focusd on locatng mor nods n UWSN usng Out-of- Rang nformaton n ths papr. Th proposd schm howvr has th potntal to b appld to som othr ntworks such as ad hoc snsor ntworks. Ths ssus wll b furthr nvstgatd n th futur. ACKNOWLEDGMENTS Ths work s supportd by NSF awards and REFERENCES [1]. F. Akyldz D. Pompl and T. Mloda Undrwatr acoustc snsor ntworks: rsarch challngs Ad Hoc Ntworks (Elsvr) pp [2] J. Aspns T. Ern D. K. Goldnbrg A. S. Mors W. Whtly Y. R. Yang B. D. Andrson and P. N. Blhumur A thory of ntwork localzaton EEE Transactons on Mobl Computng vol. 5 no. 12 pp [3] J. Gbson d. Th Mobl Communcatons Handbook. EEE Prss [4] D. K. Goldnbrg P. Bhlr M. Cao J. Fang B. D. Andrson A. S. Mors and Y. R. Yang Localzaton n Spars Ntworks usng Swps n MobCom 06: Procdngs of th 12th annual ntrnatonal confrnc on Mobl computng and ntworkng (Nw York NY USA) ACM Prss Sptmbr [5] D. K. Goldnbrg A. Krshnamurthy W. C. Manss Y. R. Yang A. Young A. S. Mors A. Savvds and B. D. Andrson Ntwork Localzaton n Partally Localzabl Ntworks n Procdngs of N- FOCOM 2005 (Mam FL) March [6] J. Hdmann W. Y J. Wlls A. Syd and Y. L Rsarch Challngs and Applcatons for Undrwatr Snsor Ntworkng n Procdngs of th EEE Wrlss Communcatons and Ntworkng Confrnc (Las Vgas Nvada USA) pp EEE Aprl [7] N. Malhotra M. Krasnwsk C. Yang S. Bagch and W. Chappll Locaton Estmaton n Ad-Hoc Ntworks wth Drctonal Antnnas n Procdngs of th 25th EEE ntrnatonal Confrnc on Dstrbutd Computng Systms (CDCS) (Columbus Oho USA.) Jun [8] D. Nculscu and B. Nath Ad hoc postonng systm (APS) n Procdngs of EEE Global Communcatons Confrnc (GLOBECOM) [9] J. Rc SEAWEB ACOUSTC COMMUNCATON AND NAVGA- TON NETWORKS n Procdngs of th ntrnatonal Confrnc Undrwatr Acoustc Masurmnts: Tchnologs and Rsults (Hraklon Crt Grc) Jun-July [10] A. Savvds C. chh Han and M. B. Strvastava ynamc Fn-Grand Localzaton n Ad-Hoc Ntworks of Snsors n Procdngs of th ACM ntrnatonal Confrnc on Mobl Computng and Ntworkng (MOB- COM01) (Rom taly) July [11] A. Savvds H. Park and M. B. Srvastava Th bts and flops of th n-hop multlatraton prmtv for nod localzaton problms n Procdngs of th 1st ACM ntrnatonal workshop on Wrlss snsor ntworks and applcatons (WSNA) (Nw York NY USA) pp ACM Prss [12] G. Zhou T. H S. Krshnamurthy and J. A. Stankovc mpact of Rado rrgularty on Wrlss Snsor Ntworks n Procdngs of th Scond ntrnatonal Confrnc on Mobl Systms Applcatons and Srvcs (MobSys) Jun Authorzd lcnsd us lmtd to: Tamkang Unvrsty. Downloadd on July at 05:32 from EEE Xplor. Rstrctons apply.