DESIGN OF LOSS FUNCTIONS AND FEATURE TRANSFORMATION FOR MINIMUM CLASSIFICATION ERROR BASED AUTOMATIC SPEECH RECOGNITION MADHAVI VEDULA RATNAGIRI

Transcription

1 DESIGN OF LOSS FUNCTIONS AND FEATURE TRANSFORMATION FOR MINIMUM CLASSIFICATION ERROR BASED AUTOMATIC SPEECH RECOGNITION by MADHAVI VEDULA RATNAGIRI A dssron submd o h Grdu School Nw Brunswck Rugrs, Th S Unvrsy of Nw Jrsy In prl fulfllmn of h rqurmns For h dgr of Docor of Phlosophy Grdu Progrm n Elcrcl nd Compur Engnrng Wrn undr h drcon of Prof. Lwrnc Rbnr And pprovd by Nw Brunswck, Nw Jrsy Jnury 0

2 ABSTRACT OF THE DISSERTATION DESIGN OF LOSS FUNCTIONS AND FEATURE TRANSFORMATION FOR MINIMUM CLASSIFICATION ERROR BASED AUTOMATIC SPEECH RECOGNITION by MADHAVI VEDULA RATNAGIRI An uomc spch rcognon sysm hs wo mn componns, h fron nd fur procssng componn, followd by modl rnng componn. A wdly usd lgorhm for rnng modls s Mnmum Clssfcon Error (MCE) h dsgns h modl prmrs o mnmz rcognon rror. On pproch for dsgnng h fur procssng componn o mnmz h rror n rcognon s rnsformng h furs usng h MCE crron h s mos commonly usd only o sm h modl prmrs. Ps ffors h hv ngrd fur rnsformon no MCE rnng ssumd h Hddn Mrkov Modl s dsrbuons wr rprsnd by dgonl covrnc Gussn mxurs whn smng h modl prmrs; bu ssumd full covrnc Gussn mxurs whn smng h fur rnsformon mrx. W

3 rcfy hs dscrpncy n ssumpons, drv nd mplmn h MCE bsd fur rnsformon, usng dgonl covrnc Gussn mxurs. For dsgnng h modl prmrs, MCE mnmzs h rcognon rror usng sndrd sgmod loss funcon, whch s Przn wndow sm of h Bys rsk. Usng dffrn krnls for Przn Wndow smon w dvlopd nw loss funcons, vz. h Gussn Krnl bsd loss nd Gnrlzd Svg loss. Invsgon no h rcognon prformnc of hs loss funcons ld o h nroducon of nw lrg mrgn bsd loss funcon for MCE (LM-MCE) whr h rror s mnmzd nd mrgn mxmzd. Mnmzng h rror, ms o shf h dcson boundry so h wrongly clssfd okns mov o h sd of h corrc clss, (howvr som of hs okns could l on or clos o h boundry); whl ncrsng h mrgn bwn h corrcly clssfd okns nd h dcson boundry, rmovs som of hs okns wy from h boundry nd mprovs h robusnss of h clssfcon. Unlk prvous suds, hs ffor dos no rqur MCE rnng pror o mxmzng h mrgn nor dos rqur non-opml mnul ncrmns of h mrgn shfs; nd snc hs loss funcon s boundd, s no suscpbl o oulrs. Th dvlopmn of h nw LM-MCE loss funcon hs horcl bss n h Vpnk nd Chrvonnks (VC) hory nd ws dvlopd usng h Bys rsk formulon.

4 ACKNOWLEDGEMENT I m grful o my dvsor, Prof. Lwrnc Rbnr for hs vlubl suggsons, lrg hrdnss nd undrsndng. H gv m h frdom o xplor nd fnd h problm of nrs o m, provdd h sssnc I ndd o vlu my work nd hlp mov hngs forwrd. I would lk o hnk Prof. Bng-Hwng (Frd) Jung for hs suppor wh h publcon of my pprs nd shrng hs rch nsghs. I would lso lk o hnk Prof. Chn-Hu L for kng h m o rd h drfs of my ds, for hs commns nd consrucv crcsm. To Prof. Josph Wldr nd Prof. Ivn Mrsc, my comm mmbrs, I ow o hr rmrks h my prsnons h confrnc wr much pprcd by mny prs. I xnd my grud o Prof. Zorn Gc, h grdu drcor nd Ms. Brbr Klmkwcz for hlpng m nvg hrough h pprwork nd sssng m o sy on rck nd up o d wh h rqurmns of h progrm. I ow gr dl o my prns, prns-n-lw nd grndprns for hr lov nd suppor. I fl ndbd o my husbnd who sood by m n h hrds of ms, whn I myslf hd los ll hop nd now hnks o hm I snd mor confdn, cpbl womn hn I hv vr bn. To my kds whos ffcon nd nnocn oulook of lf hs brough m such oy nd hppnss nd gvn m whol nw prspcv on lf - Lov you lwys. v

5 Quos from IF by Rudyrd Kplng If you cn kp your hd whn ll bou you Ar losng hrs nd blmng on you; If you cn rus yourslf whn ll mn doub you, Bu mk llownc for hr doubng oo:. If you cn drm nd no mk drms your msr; If you cn hnk nd no mk houghs your m, If you cn m wh Trumph nd Dssr And r hos wo mposors us h sm: If you cn fll h unforgvng mnu Wh sxy sconds worh of dsnc run, Yours s h rh nd vryhng h s n, v

6 TABLE OF CONTENTS ABSTRACT OF THE DISSERTATION... ACKNOWLEDGEMENT... v Quos from IF by Rudyrd Kplng... v TABLE OF CONTENTS... v CHAPTER... INTRODUCTION.... Bckground.... Ouln of h Thss... 6 CHAPTER... 9 CLASSIFICATION TECHNIQUES Inroducon Mxmum Lklhood Esmon (MLE) Mxmum Muul Informon Esmon (MMIE) Mnmum Word Error (MWE)/ Mnmum Phon Error (MPE) Mnmum Clssfcon Error (MCE) Mnmzon of Clssfcon Error Usng Grdn Probbly Dscn Esmon of Hddn Mrkov Modls usng MCE... 7 CHAPTER 3... FEATURE TRANSFORMATION TECHNIQUES Lnr Trnsformons Prncpl Componn Anlyss (PCA) Lnr Dscrmnn Anlyss (LDA) Hroscdc Lnr Dscrmnn Anlyss (HLDA) Non Lnr Fur Trnsformon Usng Arfcl Nurl Nworks Ingrd Opmzon Sm-Td ML smon MCE Bsd Ingrd Fur Procssng nd Modl Dsgn... 9 CHAPTER MCE BASED FEATURE TRANSFORMATION MCE Bsd Fur Trnsformon Usng Dgonl Covrnc Exprmnl Rsuls Auror Dbs Comprson o Ohr Suds Don on Auror dbs Conclusons nd Fuur Work v

7 CHAPTER LOSS FUNCTIONS FOR MCE ESTIMATION - BASED ON THE PARZEN WINDOW ESTIMATES OF THE BAYES RISK Przn Wndow Esm of Bys Rsk Sndrd Sgmod Loss Funcon s Przn Wndow Esm Dvlopmn of Ohr MCE Loss Funcons Svg Loss Gussn Krnl Boundd nd Smoohd Approxmon of h Hng Loss Exprmnl Rsuls Gnrlzd Svg nd Svg Loss Gussn Krnl Conclusons CHAPTER LARGE MARGIN MINIMUM CLASSIFICATION ERROR (LM-MCE) LM-MCE Usng Mnully Incrmnd Mrgn Shfs LM-MCE Usng Sum of Shfd Sgmods Exprmnl Rsuls Conclusons CHAPTER SUMMARY, CONCLUSIONS AND SUGGESTIONS FOR FUTURE WORK APPENDIX A APPENDIX B... 9 REFERENCES CURRICILUM VITAE... 0 v

8 CHAPTER INTRODUCTION Work on uomc spch rcognon (ASR) sysms hs bn ongong for numbr of yrs now nd wh ch pssng yr mor powrful nd mor succssful sysms r bng dvlopd. Howvr, h on problm h hs conssnly bffld rsrchrs n hs fld, snc h m h frs spch rcognon sysms wr bul, s how o rsolv h dffrncs bwn h rnng smpls usd o dsgn nd buld spch rcognon sysm nd h s smpls h r nd o b rcognzd ou n h rl world. Mos succssful rcognon sysms r rnd on d h s dncl o h s d; hus lmng h dffrncs bwn h rn nd s ss. Wh pplcon of ASRs bcomng wd sprd, hs bcom mor urgn h ASRs prform wh hgh ccurcy n nvronmns nd s scnros h hy wr no rnd n, bcus ASRs cnno b rrnd vry m hr s chng n h bckground nos or for vry nw spkr. Somhow humns sm o do wh rlv s; nd h quson for uomd spch rcognon s how nd whch spc of gnrl rcognon sysm nds o b modfd o ncorpor dffrncs bwn rn nd s d - h furs h r usd o rn h spch modls, h yp of modls h r usd o chrcrz h furs or h modl dsgn lgorhm. Thr r lrg numbr of pprochs o cklng hs problm of msmch bwn h rn nd s ss nd hs docorl rsrch work conrbus o wo pprochs h wr dvlopd o rsolv h msmch

9 problm. On populr chnqu s modl dpon, whr h modls r rnd usng h old rnng d nd whn hr s chng n h nvronmn or spkr, smll ds h ncluds h chng s usd o rnsform h prvously rnd modls o ncorpor h nw chrcrscs. Th mn nd vrncs of h prvously rnd modls r rnsformd so h h modfd modls r mor lkly o gnr h nw s d. Th rsrch don n hs work on fur rnsformon xnds drcly o modl dpon. Th work on loss funcons rls o nohr mor rcn pproch o ddrssng h msmch problm vz. lrg mrgn (LM) clssfcon. Suds n LM clssfcon hv suggsd h ncrsng h mrgn bwn corrcly clssfd fur vcors (from h rnng s) nd h dcson boundry cn mprov h robusnss of h clssfcon, so h s vcors from h sm clss h hv slghly hghr prurbon sll fll whn h clss boundry. Ths hss dlvs no mor dl n ch of hs rs,.. fur rnsformon nd lrg mrgn clssfcon, wh spcfc rgrd o h mnmum clssfcon rror (MCE) rnng lgorhm.. Bckground Spch rcognon s prn clssfcon sk h rcognzs h word h ws spokn by cgorzng spch sgnl no on of h word clsss. Spch d s such s nvr usd for rcognon, bcus s oo lrg nd hs lo of rdundn nformon. Thus furs r xrcd from h spch sgnl h hv rlvn

10 3 nformon for clssfcon nd r compc so s o rduc compuonl nd sorg rqurmns. S of h r spch rcognon sysms hv fron nd fur procssng scon (Fg. ) h xrcs furs from h spch sgnl nd lso rnsforms h furs, usully o dcorrl hm. Th sscl proprs of furs blongng o word clss r gnrlly rprsnd by Gussn dsrbuons nd Spch sgnl Fur Exrcon Fur Obsrvon Vcor (X) Fur Trnsformon Trnsformd Fur Vcor (Y) Clssfr Modl Fur Procssng Modlng Fgur Schmc of gnrc uomc spch rcognon sysm. dcorrlng hs furs mks possbl o us only h dgonl covrnc s opposd o full covrnc mrx, hus furhr rducng h compuonl complxy. Th prmrs of h Gussn dsrbuons (lk h mn nd dgonl covrnc) r smd n h modlng scon, usng h furs from h fur procssng scon. Th modls hus rnd r hn usd o rcognz words from h s s, o vlu h ffccy of h rnng procss. A bckground on h sndrd pproch o xrcon nd rnsformon of furs nd n xplnon of h fundmnl Bys Dcson hory usd o cgorz fur vcor no clss r brfly xplnd n h nx wo scons. Th modl rnng lgorhms howvr r dld n Chpr s lsd n h Ouln scon (.).

11 4.. Sndrd Fur Exrcon nd Trnsformon Th mos commonly usd furs nd fur rnsformon chnqu h s sndrd n ll bsln spch rcognon sysms s h Ml-frquncy cpsrl coffcns (MFCC). MFCCs r xrcd from shor 5ms wndowd sgmns of spch, wh h wndow shfd vry 0 ms long h duron of h spch. Ths sgmns of spch r hn pssd hrough bnk of rngulr flrs. I ws found h spcng h cnr frquncy of h rngulr flrs bsd on h nurl ml-frquncy scl, sn n h cochl of h humn r, lds o br prformnc []. Th log of h spcrl furs hus xrcd r hn dcorrld usng Dscr Cosn rnsform (DCT) o obn h cpsrl coffcns. Usully h frs nd scond ordr drvvs of h cpsrl coffcns r lso compud for nformon bou h rlv chng bwn succssv cpsrl vlus []. Onc h MFCCs r xrcd, hs furs r usd o dsgn h spch modls n h modlng scon (Fg. ) nd for clssfyng h xrcd furs no h vrous modl clsss, durng h vluon phs, h Clsscl Bys Dcson hory [3] ws dvlopd... Clssfcon Usng Bys Rsk Bys Dcson Thory s fundmnl hory h uss h sscl dsrbuon of h furs o clssfy spch smpl no h rgh word clss by rducng h cos of clssfcon.

12 5 Consdr h hr r M word clsss, h gol s o dvlop clssfr wh dcson funcon h: Y C h ccurly mps clss lbl CC,C,...C M o vry fur vcor y Y, whr Y s h fur spc. Idlly w sk funcon h h ccurly clssfs ll fur vcors y o hr rgh clss; prcclly w choos funcon h o mnmz h rror n clssfcon. Assum h h fur vcors hv probbly dnsy p(y) hn h clssfcon rror or rsk s dfnd s R R(C /y)p(y)dy (.) Y whr R( C /y) s h condonl rsk nd dfnd s R( C M /y) P(C /y) (.) whr M s h ol numbr of modls, P(C /y) s h posror probbly nd s h cos of wrongly clssfyng n obsrvon from clss C no clss C. If h cos s dfnd s n (.3),, 0, (.3) h s, hr s no cos whn h vcor s clssfd o h rgh clss nd h cos s whn h vcor s wrongly clssfd, hn h condonl rsk s R( C /y) P(C /y) P(C /y) (.4) nd h rsk s mnmzd by h dcson rul

13 6 dcd C, f P(C /y) mx P(C /y),,...m (.5) hus h clss wh mxmum posror probbly s chosn s h word clss for h gvn fur vcor. Th rsk obnd usng h bov dcson rul (.5) s clld h Bys rsk, nd f modl cn b usd o dfn h ru posror probbly, h prmrs of h modl cn b smd by mnmzng h Bys rsk. Howvr, h Bys rsk cnno b mnmzd usng opmzon mhods bcus h cos funcon (.3) s dsconnuous nd h Bys dcson rul (.5) cnno b mplmnd bcus of h lck of knowldg of h ru posror probbly P(C /y). For nformon on h ru posror probbly compl knowldg of h sourc of spch s rqurd, whch cnno b obnd from h lmd moun of spch smpls h s usully vlbl. Thus h Bys Rsk rmns n unnbl horcl lowr bound nd hnc ohr clssfcon chnqus hv bn dvlopd ovr h yrs. In hs work w xplor vrous spcs of on such clssfcon chnqu clld h Mnmum Clssfcon Error (MCE).. Ouln of h Thss Bys dcson hory s prcclly unchvbl horcl lowr bound, so ohr clssfcon chnqus wr xplord by mny rsrchrs nd som of h succssful chnqus hv bn summrzd n Chpr. A dld xplnon of h Mnmum Clssfcon Error (MCE), modl dsgn chnqu s gvn bcus hs s h chnqu

14 7 shown o gv h bs rcognon ccurcy on mny spch rcognon sks nd hs bn xnsvly rsrchd n hs work. Th smon of h modl prmrs whn Hddn Mrkov Modls r usd o rprsn h dsrbuon of h furs s xplnd s wll. Chpr 3 rcps chnqus h hv bn nroducd ovr h yrs o dcorrl nd/or rnsform furs. Spcfclly chnqus h rnsform furs n o spc h would opmlly spr hm o mnmz h rcognon rror r xplord. Modl rnng sms h modl prmrs o mnmz h rror, whn h sm crron s usd o rnsform h furs, h furs nd h modl prmrs r opmlly smd nd s mor ffcv wy o dvlop spch rcognon sysm (ngrd opmzon s shown n Fg. ). Ingrd Opmzon Spch sgnl Fur Exrcon Fur Obsrvon Vcor (X) Fur Trnsformon Trnsformd Fur Vcor (Y) Clssfr Modl Fur Procssng Modlng Fgur Schmc of n uomc spch rcognon sysm wh s componns showng ngrd opmzon, whr h fur rnsformon nd clssfr modl smon s don usng h sm crron Th fur rnsformon chnqu dvlopd s pr of hs work rnsforms furs usng h MCE clssfcon chnqu for Hddn Mrkov Modls wh s dsrbuons h r Gussn mxurs wh dgonl covrnc. Th mhmcl

15 8 xposon of hs pproch long wh h xprmnl rsuls hv bn prsnd n Chpr 4. Chpr 5 llusrs h rsonng bhnd h xploron for nw loss funcons for MCE clssfcon nd nroducs wo nw loss funcons h hv bn dvlopd s pr of hs work. Chpr 6 dvlops nw lrg mrgn bsd loss funcon h opmlly mxmzs h dsnc bwn h corrcly clssfd vcor smpls nd h dcson boundry. Chpr 7 concluds wh summry of h work don s pr of hs docorl rsrch, h conclusons drwn from h rsuls h wr obsrvd nd ds for fuur work.

16 9 CHAPTER CLASSIFICATION TECHNIQUES. Inroducon On of h ky spcs of spch rcognon sysms s modlng h dsrbuon of fur vcors blongng o clss; hs nvolvs prsrvng h clss nformon, whl h sm m bng bl o spr h furs blongng o dffrn clsss. Th Bys rsk formulon provds dcson crron o ccurly cgorz furs no word clsss bsd on h fur dsrbuon. Howvr, h Bys dcson hory rqurs compl knowldg of h dsrbuon of furs nd hs cnno b obnd for mos sks bcus only lmd moun of spch d s ypclly vlbl; hus h Bys rsk formulon cnno b prcclly mplmnd. In ordr o dvlop rcognon sysms, clssfcon chnqus hv bn dvlopd h mploy ohr smon crr o modl h fur dsrbuon. Som of h succssful smon chnqus r brfly xplnd followd by dld xplnon of h sndrd MCE smon pproch, snc h s clssfcon chnqu rsrchd n hs work.

17 0.. Mxmum Lklhood Esmon (MLE) Th sndrd clssfcon chnqu s h Mxmum Lklhood smon (MLE) nd s bsd on h Bys dcson hory. As dscrbd n Chpr h Bys dcson hory dcds modl clss by mxmzng h posror probbly (.5). Snc h ru posror probbly cnno b obnd, MLE drvs h bs sm of posror probbly dsrbuon, whch s h dsrbuon of h clss gvn h furs h blong o [rfr Scon..], [4] [5]. Usng h Bys rul, h posror probbly cn b rwrn s shown blow. P(y/C )P(C ) P(C /y) (.) P(y) Snc P(y) s no dpndn on h clss, s gnord nd ssumng ll clsss r qully lkly, h posror probbly cn b mxmzd by mxmzng h clss condonl lklhood P(y/C ). Typclly Hddn Mrkov Modls (HMM) r consdrd o wll chrcrz h rnsn nd spcrl chrcrscs of spch nd hnc wdly usd o rprsn h clss condonl lklhood. To sm h HMM modl prmrs, h Expcon Mxmzon formulon nd s mplmnon usng h fs nd ffcn Bum-Wlch lgorhm [6], s consdrd s h sndrd nd usd s bsln o compr ll ohr clssfcon chnqus.

18 .. Mxmum Muul Informon Esmon (MMIE) Unlk h Bys dcson hory h ms o mnmz h clssfcon rror, Mxmum Muul Informon Esmon (MMIE) [7] mxmzs h muul nformon bwn h obsrvd fur squnc nd s corrspondng word squnc. Assum X x x...x,,... M s h squnc of furs xrcd from spch sgnl nd r h HMM modls rprsnng h word squnc, hn MMIE ms o sm h prmrs of h HMM modl h would mxmz h muul nformon bwn h modl nd s furs I (θ, y ) s shown blow P (x, ),x ) I ( log P(x )P( ) (.) whr s h s of HMM prmrs...3 Mnmum Word Error (MWE)/ Mnmum Phon Error (MPE) Mnmum word rror nroducd by Povy. l. [8] combns h MMIE crron wh word rror r. Word rror r s h prformnc mrc usd o s spch rcognon sysms. Whn rnng h spch rcognon modls h smon chnqus mnmz rror h snnc lvl howvr whn sng h rnd modls, word rror r s msurd usng h Lvnshn dsnc s usd. In ordr o brdg h gp bwn h snnc bsd smon chnqus nd h word bsd prformnc

19 mrc; obcv funcons h hv posror wghd probbly r usd s shown n (.3). F P (S/y)A(S, R) (.3) S whr S s h hypohss for n urnc whos obsrvon vcor s x nd R s h rfrnc rnscrpon. (S/y) P s h posror probbly of h urnc gvn h obsrvon vcor, s h modl prmr s. A(S, R) s vron of h word rror r lso clld h Rw ccurcy, nd hs should rprsn h Lvnshn dsnc whch s h numbr of words h wr ncorrcly rcognzd by subsuons, dlons or nsrons. Bcus of h dsconnuous nur of h Lvnshn dsnc, mny pproxmons o h Rw ccurcy hv bn dfnd, lk RwWordAccurcy. Mnmum Phon rror (MPE) vron of h Mnmum Word rror lso nroducd by Povy. l. [8] usd RwPhonAccurcy, s n pproxmon of h phon lvl Lvnshn dsnc, (hy found h phon lvl mprovd rcognon ccurcy mor h h word lvl). I ws dfnd s h numbr of corrcly rnscrbd phons (q) n hypohss S nd ws compud bsd on h xn of ovrlp n m of ch phon wh s rfrnc consdrd ovr ll possbl phon hypohss S. Th obcv funcon of hs pproch s shown blow RwPhonAccurcy(S) PhonAcc(q) q S whr whn q s h corrc lbl PhonAcc(q) ohrws (.4)

20 3 whr rprsns h conrbuon of ch phon drmnd by h xn of ovrlp n m o h rfrnc phons. Th obcv funcon usd for MPE ws p (x/s)p(s)rwphonaccurcy(s) F S MPE (.5) p s (x/s)p(s)..4 Mnmum Clssfcon Error (MCE) Mnmum Clssfcon Error s clssfcon chnqu h drcly mnmzs n mprcl rsk h s dffrn from h Bys rsk (hough hs bn shown [35] h undr crn ssumpons hs rsk cn convrg o h Bys rsk). MCE sms h modl prmrs by mnmzng connuous sgmod loss funcon usng h Grdn probbly dscn (GPD) opmzon pproch. Embddd no h loss funcon s msclssfcon msur, whch s msur of spron bwn hypohszd clss nd h nrs compng clsss, mkng hs pproch dscrmnv. Whl h loss funcon s mnmzd, h msclssfcon msur s mxmzd. As s gnrlly h cs, h spch furs r ssumd o b suffcnly chrcrzd by Hddn Mrkov Modls (HMMs) nd h follwong scon xplns h sndrd pproch [0, 9] o smon of h HMM modl prmrs n mor dl [9,0,7,35].

21 4. Mnmzon of Clssfcon Error Usng Grdn Probbly Dscn Consdr X o b h s of rnng vcors nd φ h s of modl prmrs h nd o b smd. L C b h h clss h X blongs o nd l C b h clss s bng clssfd no. Jus s n h cs of Bys rsk [rfr Chpr ], MCE lso ssgns zro cos o corrc clssfcon nd cos of o ncorrc clssfcon usng cos funcon s shown blow d (x) 0 (.6) In ordr o drmn f hr hs bn msclssfcon, MCE dfns d (x), msclssfcon msur nd lmng cs of d (x) s dfnd n (.7). d (X) g (X;φ) mx g (X;φ) (.7) whr g (X; φ) s h dscrmnn funcon. Dffrn dscrmnn funcons hv bn prvously suggsd [0], h on h s wdly usd s h log of h clss condonl lklhood, g (X;φ) log P X/φ (.8)

22 5 Thus f w consdr fur vcor from clss C whch s clssfd o clss C bcus P X/φ s grr hn X/φ P hn d (.7) wll b posv nd hus ssgnd cos of (.6). To mnmz h clssfcon rror r h cos funcon (.6) s o b mnmzd, howvr h cos funcon bng dsconnuous cnno b mnmzd by opmzon rouns, hnc h msclssfcon msur s mbddd no connuous sgmod (0-) loss funcon s shown blow. L (X;φ) xp( γd (X) ρ) (.9) nd h mprcl rsk s dfnd s R N M E Xl(X;φ) L (X;φ) p(x)dx XC (.0) whr M s h ol numbr of clsss. I cn b obsrvd from h bov quons (.9), (.0) h mnmzng h mprcl rror, mnmzs h loss funcon, whch n urn mxmzs h msclssfcon msur. Mxmzng h dscrmnn funcon for h currn clss whl mnmzng h funcon for nrs compng clss (.7) ncrss h spron bwn h clss ( ws shown h hs dsnc dfnd by h msclssfcon msur s n pproxmon o h Kullbck-Lblr dvrgnc []). Thus n mporn bnf o usng MCE s h whl mnmzs h cos of clssfcon, lso mxmzs h spron dsnc bwn clsss,. s dscrmnv by nur.

23 6 Th modl prmrs φ h mnmz h MCE mprcl cos r smd usng GPD. GPD upds h modl prmrs s shown n (.) whr h modl prmrs r updd long h ngv grdn of h loss funcon, hus choosng modl prmrs h lowr h vlu of h loss funcon. Wh rv procssng h locl mnmum cn b obnd nd f h nl modl prmrs r chosn udcously, hs locl mnmum cn b h globl mnmum. l (X;φ) φ τ φτ ε φ φφ τ (.) s h bov quon s h numbr of ron, ε s h lrnng r nd h drvv of h loss funcon s drmnd usng h chn rul s shown n (.). L g (X;φ) L d d g φ φ nd L γl (d )( L (d )) d d f = g f nd g log P X/φ φ φ (.) To mplmn h GPD s dscrbd bov, modl h chrcrzs h clss condonl lklhood, vz. P X/φ, s rqurd nd h mos commonly usd modls r h Hddn Mrkov Modls (HMMs). A smpl lf-rgh HMM s ypclly usd, so s o rprsn boh h m nd frquncy chrcrscs, whl h sm m kpng h

24 7 compuonl complxy mngbl. To g GPD srd HMM modl prmrs r nlzd usng h ML smon crron, whch s followd by h MCE smon s dscrbd blow..3 Esmon of Hddn Mrkov Modls usng MCE Whn usng Hddn Mrkov modls o chrcrz ch word/sub-word clss, s s gnrlly don for mos spch rcognon sysms, h dscrmnn funcon nd h clss-condonl lklhood r dfnd s shown n (.3). g X/φ log P X/φ log mx P X,q/φ T log P X,q/φ log π q q 0 q q b, (x ) q (.3) whr π q s h nl s probbly of h HMM modl corrspondng o clss 0 C, q 0 s h nl s, q q s h rnson probbly from s q o q, b q (x ) s h oupu s probbly dsrbuon, T s h numbr of m frms nd q s h opml s squnc. Usng h bov dfnon for HMM dnsy h s of modl prmrs h nd o b smd for n HMM r h nl s probbly π q, h rnson probbly 0 q q nd h prmrs of h oupu s dsrbuon b q (x ), whch s gnrlly rprsnd usng Gussn mxurs. I s only

25 8 h s dsrbuon prmrs h r smd bcus h ohr prmrs do no chng much durng h MCE smon procss [7]. Thus h grdn of h dscrmnn funcon, rqurd n (.) s obnd s shown blow. g logp X/φ φ φ logb q (x ) T δ(q ) φ (.4) Th Gussn mxur h rprsns h oupu s dsrbuon b q (x ) s dfnd s K b q (x ) c N x,λ ;μ (.5) k whr K s h ol numbr of Gussn mxurs, c s h wgh of ch Gussn mxur nd N dnos norml dsrbuon wh mn μ nd dgonl covrnc Λ nd dfnd n (.6). N x ) T (x μ ;μ,λ xp (x μ Λ ) (.6) / πn/ Λ n s h dmnsonly of X. Thus h s of prmrs h dfn h oupu s dsrbuon nd r smd s φ c,μ, Λ. In ordr o mnn h consrns of h HMM nd kp h covrnc mrx nd h mxur wghs posv

26 9 dfn nd lso o hndl h ssu of smll vrncs [0], h followng rnsformons for h mn (.7), vrnc (.8) nd mxur wgh (.9) r usd. μ μ (.7) Λ Λ logλ (.8) c logc (.9) Usng h bov rnsformon, h grdns wh rspc o h mn, vrnc nd mxur wgh r logb q (x ) μ c / πn/ Λ b q ) (x x Λ μ xp (x μ ) T Λ (x μ ) (.0) nd

27 0 logb c q (x ) / n/ π Λ b (x ) q x μ xp (x μ Λ ) T Λ (x μ ) (.) nd logb q (x ) c c / πn/ Λ b q xp (x ) (x μ ) T Λ (x μ ) (.) An ddonl consrn h h mxur wghs sum o on s mnnd by rnormlzng h wghs fr h MCE upd (.). Thus usng Grdn probbly dscn nd rv procssng, h modl prmrs h mnmz h clssfcon rror cn b smd.

28 CHAPTER 3 FEATURE TRANSFORMATION TECHNIQUES In h prvous chpr w consdrd on ky spc of spch rcognon sysms, whch ws h dsgn of h modls h chrcrz h fur vcors. Hr w consdr nohr ky spc nmly, fur rnsformon. Th rnsformon of furs s ssnl o rduc h numbr of prmrs h r smd n h modlng scon, so s o rduc compuonl complxy s wll s nsur rlbl sms. I cn lso b usd o mprov rcognon ccurcy. Mos rnsformon chnqus dcorrl furs so h whn usng Gussn mxurs o rprsn hm, dgonl covrncs would suffc nsd of full covrnc mrcs. For n NxN dgonl covrnc mrx only N rms r smd, on h ohr hnd for full covrnc mrx N*(N+) rms nd o b smd. Gvn lmd moun of d, mgh no b possbl o rlbly sm ll h cross rms of full-covrnc mrx. Thus consdrng only h dgonl lmns of covrnc mrx no only rducs compuon, bu lso nsurs h rms of h covrnc mrx smd r rlbl. Ohr rnsformon chnqus opmz furs o drcly or ndrcly mnmz h clssfcon rror. I s blvd h n opml spch rcognon sysm s obnd whn h fron nd procssng s ngrd wh h of h modlng scon.

29 So, rnsformon of h furs s don usng h sm clssfcon crr usd o rn h clssfr modls (s dscrbd n Chpr ). Som lnr rnsformon chnqus usd o dcorrl fur vcors, nonlnr rnsformon h uss Arfcl Nurl Nworks (ANNs) nd ngrd opmzon chnqus r furhr dscussd n hs chpr. 3. Lnr Trnsformons To dgonlz covrnc mrx, h furs r dcorrld usng lnr rnsformon chnqus lk Prncpl Componn Anlyss (PCA), Lnr Dscrmnn Anlyss (LDA), or Hroscdsc Lnr Dscrmnn Anlyss (HLDA). Ths chnqus hv shown o sgnfcnly mprov h rcognon prformnc [8], [3] comprd o h sndrd Dscr Cosn Trnsformon pproch (dscrbd n Chpr ). A bsc pproch o dgonlzng fur vcor s o consdr s gn dcomposon. L X b p dmnsonl fur vcor. Thn X cn b rprsnd by sum of lnrly ndpndn vcors X p ymm (3.) m whr m r h bss vcors of X nd ym Y s h orhogonl rnsformon. Usng mrx mnpulon, h rnsformd fur vcors cn b obnd s shown n (3.).

30 3 y m T m X whr m,...p Y Φ T X y Φ T ΣxΦ Λ x (3.) whr Φ s h lnr rnsformon mrx, Σ x s h covrnc of X nd Σ Y s h covrnc of Y. Th columns of Φ r h gn vcors m h spn h p- dmnsonl spc of X. Thus h rnsformd fur vcor Y hs dgonl covrnc mrx Λ x, whch s h gn vlu mrx. Ths pproch dos mprov rcognon ccurcy s shown by Lol.l. [], howvr s compuonlly xpnsv o fnd ll h gn vcor mrcs (xplnd n mor dl n Scon 3.3.). 3.. Prncpl Componn Anlyss (PCA) PCA s lnr rnsformon chnqu h rnsforms h furs no sub-spc usng h Krhunn-Lov (KL) rnsform so h h mn squrd rror bwn h rnsformd furs nd h orgnl furs s mnmzd. Th columns of h KL rnsformon mrx r gn vcors, h corrspond o h lrgs gn vlus of h covrnc mrx. In ohr words n PCA, only q of h p lnrly ndpndn bss vcors r consdrd. Xˆ q ymm (3.3) m In ordr for Xˆ o b good pproxmon of h orgnl fur vcors X, h mn squrd rror bwn X nd Xˆ s mnmzd nd h lds o h condon shown blow (rfr [] for mor dls).

31 4 p p E X - Xˆ λ T m m Σ x m (3.4) m q m q whr Σ x s h covrnc of X, λ m r h gn vlus. Incrsng h numbr of componns of h bss vcors q' h r consdrd, o rprsn h fur vcor, rducs h rror, bu ncrss h compuon nd vc vrs. Th slcon of q s hursc whr h bs rd off bwn compuon vs rcognon ccurcy s md Lnr Dscrmnn Anlyss (LDA) LDA compus gn vcors h mxmz h spron dsnc bwn clsss. Th clss sprbly s msurd usng h Fshr ndx wh-n clss vrnc w Σ o bwn clss vrnc Σ Σ - w Σ B whch s h ro of h B []. Th formr s mnmzd s h lr s mxmzd. q gn vcors h corrspond o q of h lrgs gn vlus of Σ - w Σ B r consdrd nd h fur vcors r procd ono h spc. Cmpbll showd n hs work [0] h h LDA sms of h vcors wr cully h ML sms undr h ssumpon h h clsss hd dffrn mns bu common covrnc mrx.

32 Hroscdc Lnr Dscrmnn Anlyss (HLDA) Kumr [] xndd h work by Cmpbll [0] by consdrng h bwn-clss vrnc o b dffrn or hroscdsc for dffrn clsss. In ordr o ncorpor clss-spcfc nformon nd lso b bl o modl ll h dmnsons of h fur vcor, h rnsformon mrx ws prond. Consdr pp lnr rnsformon mrx nd pron no wo mrcs, on connng ll h clss spcfc nformon nd s qp dmnsons, whl h ohr (p q) p mrx hs h nusnc dmnsons nd s common o ll clsss. Consdrng only h clss spcfc rnsformon wll proc vcors no dffrn sub-spcs, hus mkng comprson of probbls cross clsss no possbl. By rprsnng ll h dmnsons of h fur vcor, Kumr nsurd h h lklhood of h dffrn clsss cn b comprd nd by splng h mrx nd consdrng h clss dpndn pr, clss-spcfc chrcrscs wr rprsnd s wll. Y A T X whr A nd Σ y A μ y Σ 0 A g μ μ g 0 Σ g (3.5)

33 6 A s h qp dmnsonl clss-dpndn mrx, s h clss lbl nd Ag s h (p q) p nusnc mrx common o ll clsss. Th mn nd covrnc of h rnsformd furs r lso spl no on corrspondng o h clss-spcfc dmnsons μ, Σ nd h nusnc dmnson μ g, Σ g s shown n (3.5). H showd h h LDA sms of hs mn nd vrnc rms wr ML sms. 3. Non Lnr Fur Trnsformon Usng Arfcl Nurl Nworks Arfcl Nurl Nworks (ANNs) r somms usd o mp h fur vcors o hr rspcv clss lbls. Unlk h lnr rnsformon chnqus h spr clsss usng lnr hyprplns, ANNs cn dsgn non-lnr hyprplns. Th bck propgon lgorhm s commonly usd o sm h prmrs of h ANN. Th drwbck of ANNs s h h bck propgon lgorhm rqurs suprvsd lrnng h sms h wghs nd offs prmrs of h ANN by mnmzng h mn squrd rror bwn h oupu vlus h oupu nods nd h rg oupu nod vlus. Th oupu nods h corrspond o h clss h h fur vcor blongs o r ssgnd rg vlu of whl ll ohr nods r ssgnd rg vlus of 0 - so h oupu of n ANN, wll hv vlus clos o or 0, hghly non-gussn dsrbuon, hus mkng h mos common modl rprsnon lk Gussn dsrbuons or HMMs wh Gussn mxur ss, nppropr. Suds [3], [4]

34 7 comprng lnr rnsformon chnqus lk PCA nd LDA wh non-lnr ANN rnsformon hv shown mprovmn n rcognon ccurcy wh ANNs, whr hy wr bl o combn dffrn fur procssng lgorhms. 3.3 Ingrd Opmzon Fur rnsformon nd modl dsgn r ngrl prs of spch rcognon sysm nd h dsgn of hs wo spcs of h rcognon sysm should b ngrd nd opmzd usng h sm crron. Mny suds [5], [6], [7], [8], [9] hv ngrd h dsgn of furs nd clssfr modls (s shown n Fg. ) nd rpord mprovd prformnc n rcognon ccurcy. Gopnh [9] nd Gls [5] ndpndnly dvlopd pprochs o sm h modl nd h fur rnsformon mrx usng h Mxmum Lklhood (ML) crron. Us of ML for fur rnsformon hs bn possbl bcus of h prncpl dvlopd by Cmpbll [0] nd Kumr [] h h gn vcors obnd usng LDA/HLDA r ML sms. Th srs of suds by Bm. l. [3], [4], [5], [6] usd h MCE modl rnng lgorhm o sm h flr prmrs, h flr h s usd o xrc furs from spch sgnl. Wng. l.[8] comprd PCA nd LDA, wh ngrd opmzon usng MCE nd showd fur rnsformon usng MCE prformd br whn h rnsformd fur vcor dmnson wsn sgnfcnly lowr hn h orgnl fur dmnson. Rhm.l. [7], [8] usd MCE o sm h prmrs of n ANN h hy usd o non-lnrly rnsform furs. So mny fur procssng sks cn b

35 8 don usng MCE bcus of h us of grdn probbly dscn (GPD). Th bsc prms of ch of h bov mnond pprochs r furhr xplnd n h followng scons Sm-Td ML smon Ths s fur rnsformon pproch dvlopd by Gls [5] bsd on h ML modl smon crron. A lnr rnsformon of h fur vcors would rnsform h covrnc mrx s shown n (3.6). Y AX Σ y AΣ x A T (3.6) Rwrng x Σ usng (3.) nd suppos h rnsformon mrx A s h rnspos of h gn vcor mrx.. T hn h bov quon (3.6) gvs y Σ - Φ T Φ T Λ x Φ - Φ Λ x (3.7) Thus choosng A o b h rnspos of h gn vcor mrx gvs h opon of rprsnng full covrnc mrx Σ y s dgonl covrnc Λ x. Ths ws consdrd n sudy by Lol []. Typclly spch modl mploys hundrds of Gussn mxurs nd o compu h dgonl covrnc mrx x Λ corrspondng o ch Gussn mxur would rqur us s mny full rnsformon mrcs A. Compuonlly hs s mor vn xpnsv hn consdrng full covrnc mrcs for ll h Gussn mxurs. Gls

36 9 n hs formulon of h sm-d covrncs rprsnd h rnsformd covrnc mrx s combnon of dgonl componn s shown blow. Λx nd non-dgonl rnsform As Σ y AsΛ x T As (3.8) Insd of usng As o b h rnspos of h gn vcor mrx for ch spcfc Gussn mxur, h non-dgonl componn As cn b globl, whch would rqur h ls moun of sorg nd compuon, or spcfc o ch mxur, whch would mk h mos compuonlly xpnsv or nyhng n bwn, lk d ovr ll ss n modl. To sm A s, sm-d smon uss h drvon by Cmpbll [0] nd N. Kumr [] h h mn nd vrnc of h vcors procd usng LDA nd HLDA (.., h gn vcors of (3.7) mnmz h Fshr ndx) r cully h ML sms. Hnc Gls [5] modfd h Bum-Wlch lgorhm, gnrlly usd o compu h ML sms of h modl prmrs o sm h sm-d rnsformon mrx As s wll MCE Bsd Ingrd Fur Procssng nd Modl Dsgn Ohr uhors [6], [7], [8], [3] usd dffrn smon pproch nmly h Mnmum Clssfcon Error (MCE) o sm h modl prmrs s wll s xrc nd rnsform furs. MCE s dscrmnv rnng crron h mxmzs h spron bwn compng clsss, whl mnmzng smoohd rror funcon usng

37 30 grdn probbly dscn (GPD). Th us of GPD for Mnmum Clssfcon Error (MCE) rnng mks dpbl o smng h fur xrcon prmrs ncludng ANN prmrs (whn ANNs r usd for fur rnsformon) or h fur rnsformon mrx slf. Fur Exrcon Usng MCE Bm. l. n srs of suds [3], [4], [5],[6] usd MCE o sm h flr prmrs o xrc furs from h spch sgnl nd lso dgonl fur rnsformon mrx. Thy smd flr prmrs lk cnr frquncs, bndwdhs, gn c. Ths pproch mks h flr dsgn opml for clssfr dsgn, bu lso mks ncrsngly snsv o h rnng d. ANN Fur Trnsformon Dsgn Usng MCE Rhm nd L [7], [8] usd ANNs o rnsform h furs bfor rnng h modls nd usd MCE o sm h ANN prmrs nsd of h sndrd rror bckpropgon lgorhm. As mnond rlr, h bck propgon lgorhm bng suprvsd lgorhm rqurs rg vlus whch r usully or 0, hus h vlus h oupu nods hv hghly non-gussn dsrbuon nd mkng HMMs wh Gussn mxur ss nppropr s modls. Rhm nd L [7], [8] by combnng h ANN nd HMM dsgn llvd h problm of ssgnng rg vlus for h ANNs, hy smd h ANN prmrs by mnmzng h sgmod loss funcon usng GPD nd

38 3 hus dd no rqur h rg oupu nod vlus. Thy xndd hr work [8] o consdr mulpl ANNs for dffrn clsss, nsd of on ANN for ll clsss [7], bu fld o nsur h h probbly dsrbuons of fur vcors, procd no dffrn sub-spcs usng dffrn clss-dpndn rnsformons, cn b comprd. Fur Trnsformon Usng MCE Chnglvryn. l. [7] nd Wng.l [8] n hr work on ngrd opmzon usd h MCE modl rnng lgorhm o rnsform h sndrd ml-frquncy bsd cpsrl coffcns (MFCC). Chnglvryn. l. [7] smd s dpndn fur rnsformons (.. usd dffrn rnsformons for h dffrn ss n n HMM) nd fld o nsur h h clss condonl probbly (CCP) of dffrn ss cn b comprd cross h dffrn sub-spcs h h furs wr procd no. Comprson of h probbls s rqurd durng MCE s dscrmnv rnng whr h CCP of h hypohszd modl nd compng modls r usd. In h sng phs s wll, h CCPs r comprd nd h fur vcor s ssgnd o h clss wh hghs probbly. Wng. l. [8] n hr work on ngrd opmzon of furs nd modls usd globl fur rnsformon, so h CCP cn b comprd cross modls. Thy usd h Mhnolbs dsnc s msur of sprbly bwn clsss, nsd of h msclssfcon msur ( d ) whch hs bcom mor of sndrd. In boh hs suds by Wng. l [8] nd Chnglvryn. l. [7], hr ws mhmcl nconssncy bcus hy ssumd dgonl covrnc mrcs for h

39 3 Gussn mxurs whn smng h prmrs of h modl, whch s gnrlly h norm (s xplnd n Chpr du o rsons of compuon nd rlbly of sms) bu ssumd full covrnc mrcs whn drvng h sm for h rnsformon mrx. Furhr, s known fc h lnr rnsformon of Gussn dsrbuon wh full covrnc, dos no rnsform h furs, bu only scls h dsrbuon, hr by mkng h rnsformon mu. Th followng chpr dscrbs n dl MCE bsd fur rnsformon, whr h Gussn mxurs wh dgonl covrnc r consdrd whn smng boh h rnsformon mrx s wll s h clssfr modl prmrs. A globl fur rnsformon mrx h s common o ll ss nd modls s smd, hus mkng possbl o compr probbls. Usng GPD o sm h prmrs rqurs h nlzon (s xplnd o Chpr, Scon.) of h modl prmrs nd h rnsformon mrx whch s don usng h ngrd pproch of sm-d ML smon nroducd by Gls [5], nsd of nlzng h rnsform mrx by n dny s n h cs of Chnglvryn. l.[7] nd Wng. l.[8]. Th commonly usd, clss condonl probbly s consdrd s h dscrmnng funcon [0], s opposd o h Mhnolbs dsnc usd by Wng. l. [8] or h Eucldn dsnc usd by Torr. l [6].

40 33 CHAPTER 4 MCE BASED FEATURE TRANSFORMATION Indpndnly opmzng h fur procssng nd modl dsgn spcs of spch rcognon sysm mks for sub-opml sysm. Th ML nd MCE modl dsgn chnqus hv bn usd n h ps for fur rnsformon nd hs chnqus wr brfly dscussd n h prvous chpr. Th MCE pprochs hv shown proms, bu md h ssumpon h h Gussn mxurs usd full-covrnc mrcs whn smng h rnsformon mrx nd md h ssumpon of dgonl covrnc mrx whn smng h modl prmrs. W corrc hs dscrpncy n our opmzon procss nd drv h upd quons for h rnsformon mrx. Th rsuls of h mhmcl nlyss mplmnd on spch corpus r lso prsnd, hrn.

41 34 4. MCE Bsd Fur Trnsformon Usng Dgonl Covrnc Assum h X s h orgnl fur vcor whch s lnrly rnsformd, Y AX, whr A s non-sngulr globl n p rnsformon mrx, p s h dmnson of X nd n s h dmnson of Y. I s ssumd h A s globl rnsformon mrx h s common o ll clsss. For MCE bsd opmzon, h modl prmrs nd, n h cs of hs sudy, h fur rnsformon mrx, r compud by mnmzng h sgmod loss funcon. Th loss funcon s dfnd s shown blow L (Y,φ) xp(-d (Y,φ)) (4.) whr s h modl, γ s consn nd d s h msclssfcon msur. As xplnd n Chpr, h msclssfcon msur d s msur of h dsnc bwn h hypohszd modl nd h closs compng modl s shown gn n (4.). d g (Y,φ) mx g (Y,φ) (4.) whr g (Y, φ) s h dscrmnn funcon of clss C nd s dfnd s g (Y,φ) log P Y/ (4.3)

42 35 whr P Y/ s h clss condonl probbly. Assumng Hddn Mrkov Modls r usd o modl h clss dsrbuons, h clss condonl probbly s dfnd s T P Y/ log,q,q q b,q (y ) (4.4) 0 - whr s h modl,,q s h nl s probbly, q 0 s h nl s, 0,q q - s h rnson probbly from s q - o q nd b,q (y ) s h oupu dsrbuon of s q, s h currn frm nd T s h ol numbr of frms n h spch urnc. Ech s n h HMM s modld usng mxurs of Gussns hnc h oupu s dsrbuon s rprsnd s K b,q (y ) c N y, ;μ (4.5) k whr c s h wgh of ch Gussn mxur for modl nd mxur k, N() dnos norml dsrbuon wh mn μ nd dgonl covrnc nd K s h ol numbr of mxurs. Thus h s of modl prmrs h nds o b smd s φ c,μ, Λ, A, whch ncluds h Gussn mxur prmrs nd h rnsformon mrx. Th rnson probbly bwn ss s no consdrd bcus s known o no chng vry much du o MCE rnng [9]. Th prmrs of s φ r smd usng grdn probbly dscn (GPD) opmzon s shown n (4.6).

43 36 L (Y;φ) φ τ φτ ε φ φφ τ (4.6) whr τ s h τ h ron, ε s h lrnng r. Th drvv of h modl loss s drmnd s shown n Jung. l. [0] nd lso shown blow L Y,φ φ L g d d g φ (4.7) whr L d d g nd g φ γl ( L ) f = f logb T δ(q ),q φ ) (y (4.8) nd L s usd nsd of (Y; φ) L for convnnc. Y y, y,...y T s h rnsformd fur vcor y y,y,...y n s h fur vcor xrcd m nd s of dmnson n. For upd xprssons of mn, vrnc nd mxur wghs rfr Chpr or Jung.l [0], Chnglvryn. l. [7] or McDrmo [9]. No, h prmr upds gvn n Chpr nd pprs [0], [7] & [9] hv bn drvd undr h ssumpon h h covrnc mrcs for h Gussn mxurs r dgonl. Hr w drv h xprsson for updng h fur rnsformon mrx undr h sm ssumpon of dgonl covrnc.

44 37 I s wll known fc h lnr rnsformon of Gussn, whr h full covrnc mrx s consdrd, dos no modfy h Gussn dsrbuon bu only scls by h drmnn of h rnsformon mrx s shown n (4.9), n whch cs no compuon of n opml rnsformon mrx cn b don nd h ffc of h rnsformon mrx s bsorbd no h mn nd vrnc upds. T N y ;μ, xp y μ y μ / πn/ T, x, x, x xp Ax Aμ AΛ A T Ax Aμ / n/, x π AΛ A T A Nx,x,x ;μ, (4.9) whr μ Aμ,x, AΛ,x AT r h rnsformd mn nd full covrnc, whl μ,x nd,x r h mn nd full covrnc of h orgnl furs X. Snc s common prcc o consdr dgonl covrnc Gussns n ordr o rduc compuonl complxy nd for rlbl sms, h rnsformd Gussn dsrbuon should b wrn s shown n (4.0).

45 38 N y,λ ;μ xp, x Ax Aμ πn/ dg AΛ, x A T T dg AΛ, x A T /, x Ax Aμ (4.0) Assumng h X-domn furs hv bn prvously dgonlzd, so Λ,x s h dgonl covrnc n h X-domn nd fr lnr rnsformon wh A, h rnsformd dgonl covrnc Λ s dg AΛ,x AT. From h quon bov w s h by xplcly consdrng h dgonl covrnc ssumpon h ffc of h rnsformon mrx s no bsorbd no h mn nd vrnc sms. In ordr o ssfy h consrn h h rnsformd covrnc mrx rmns posv dfn w scl h rnsformon mrx s A loga, hn h mn nd vrnc r scld s wll, shown n (4.) nd (4.), rspcvly. μ Aμ,x A μ,x (4.) T Λ dg A Λ, x A (4.) For smng h rnsformon mrx usng grdn probbly dscn (4.8) h drvv of h oupu s dsrbuon wh rspc o h rnsformon A s compud s follows.

46 39 log b,q (y ) A K c N y k ;μ b,q (y ),Σ D D (4.3) whr D mn A mn Λ, x dg A Λ, x A T / AΛ, x A T mm mm dg AΛ, x A T (4.4) D mn, x mn U m Λ dgaλ, x A T U T dg A Λ, x A T U (x, x μ ) n U mn m dg AΛ, x A T m m (4.5) whr U A x μ, x, m nd n subscrps n (4.4) nd (4.5) r lmns of mrx or vcor, lowrcs s usd o rprsn mrx lmns nd mrx A hs bn subsud for A. Th drvon of hs quons s dld n h Appndcs A nd B. Du o h rnsformon A loga, w cnno nlz h rnsformon mrx by n dny mrx, nd nsd us ML bsd sm-d opmzon o nlz boh h rnsformon mrx nd h modl prmrs.

47 40 4. Exprmnl Rsuls Th MCE bsd upd of h fur rnsformon mrx drvd bov ws mplmnd on spch corpus. Frs, h sndrd Ml-frquncy cpsrl coffcns (MFCC) ncludng h nrgy rm, long wh h frs nd scond ordr m drvvs wr xrcd from h spch sgnl. 39 dmnsonl MFCC fur vcors xrcd from h spch sgnl wr hn rnsformd usng 39X39 dmnsonl globl rnsformon mrx, h ffccy of h rnsformon mrx sm usng h quon n (4.3) ws sd on h Auror dgs spch dbs. 4.. Auror Dbs Th Auror dbs uss h 8440 urncs from TIDg rcordngs whr h dgs lk h dgs of lphon numbr r spokn by ml nd fml spkrs wh dffrn ccns. Ths urncs r flrd usng h G.7 chrcrsc nd dffrn nos sgnls dffrn sgnl o nos ros (SNRs) r rfclly ddd. Two rnng ss r vlbl, h cln d s, whch hs only h TIDgs urncs wh no rfcl nos ddd o hm nd h mul-condon d s, whr cln s wll s nosy d r usd for rnng. In hs sudy, rnng ws don on h mulcondon ds; whr h 8440 urncs r spl no 0 subss, corrspondng o 4 nos scnros (subwy, bbbl, cr, xhbon), 5 SNR lvls (0 db, 5dB, 0dB, 5dB nd 0dB). Smlr o h Auror xprmnl frmwork [30], ol of HMM

48 4 modls wr rnd, word modls for ch of h dgs ( on, wo,..., nn ncludng oh nd zro ) nd slnc modl. Ech HMM hd 6 ss wh 3 mxurs pr s xcp h slnc modl whch hd 3 ss wh 6 mxurs (- h numbr of ss nd mxurs wr lgnd wh h bsln Auror ppr [30] so whn rsuls r comprd, vryhng ls bng qul, h bnf of hs pproch cn b vlud). Th rnd modls wr sd usng 3 dffrn s ss (ss A, B &C). In s s A, h spch smpls wr dffrn from h rnng s, howvr 4 nos squncs h r h sm s hos usd n h mul-condon rnng wr rfclly ddd, gn 5 dffrn SNRs, so h nos condons of hs s mch h rnng s. Th s s B uss h sm urncs s s A bu h 4 nos scnros (rsurn, sr, rpor, rn) ddd 5 dffrn nos lvls, r dffrn from h rnng. Fnlly s s C hs nos scnros ddd o, on s h sm s h ddd n S A (vz. subwy nos) nd h ohr s nos h s ddd o S B (vz. sr nos). Howvr boh h spch nd h nos n S C r flrd usng h MIRS chrcrsc bfor ddng h spch nd nos dffrn SNR lvls [30]. Tbl 4. shows h word ccurcy on h nr s s fr 9 rons of h GPD upd. Th sndrd MCE chnqu gvs word rror r rducon of 4.77% nd 9.66% for s A nd s B comprd o ML (rsuls from h Auror xprmnl ppr by Hrsch. l [30]). Whl MCE bsd fur rnsformon nd modl smon rducs h word rror r furhr, o 6.09% nd 7.4% comprd o ML. Howvr hs pproch s no robus o h lnr convoluon dsoron prsn n S C.

49 4 ML Esmon [] MCE ESTIMATION MCE Bsd Fur & Modl Esmon S A S B S C TABLE 4. Comprng h vrg word ccurcy for ML, MCE nd h MCE bsd Jon Opmzon for h nr s s As shown n Tbl 4., h word rror r rducon usng MCE bsd fur rnsformon s grr whn vlud on h cln s d, 50.68% nd 45.89% word rror r rducon rspcvly for ss A nd C comprd o ML. (Cln d s h TIDg urncs bfor h nos s ddd.) Th rlv rducon n word rror r du o h nroducon of h fur rnsformon cn b compud by comprng h word ccurcy of h sndrd MCE modl smon pproch nd h MCE bsd fur & modl smon, nd w found hr ws 0% rducon n word rror r whn ss A & B wr consdrd oghr (s s C wsn consdrd bcus hr ws rducon n rcognon ccurcy) nd 30% rducon for h cln d n Ss A & C, (snc s B uss h sm urncs s S A nd hnc h cln d s dncl o S A). Th rducon n rcognon ccurcy obsrvd n S C for boh MCE smon (column 3, Tbl 4.) nd MCE bsd modl & fur rnsformon (column 4, Tbl 4.) s bcus MCE, whch s dscrmnv rnng pproch s no robus o dsorons n nos. I s howvr robus o lnr dsorons n spch snc n ncrs s obsrvd n h rcognon ccurcy for S C of cln d s shown n Tbl

50 43 4. blow. Anohr rson h MCE bsd modl & fur rnsformon pproch dd poorly on h S C wh nosy spch d, s bcus whn ssumng dgonl covrnc mrcs h corrlon nformon s smply gnord nd hnc los snc h fur vcors r no dcorrld or rprsnd by ndpndn vcors s dscrbd n Chpr 3. ML Esmon [] MCE Esmon MCE Bsd Fur & Modl Esmon S A (sm s S B) S C TABLE 4. Comprng h vrg word ccurcy for ML, MCE nd h MCE bsd Jon Opmzon for Cln d To compr h MCE bsd fur & modl smon pproch o h sm-d pproch (whch s n ML-bsd fur nd modl opmzon pproch dvlopd by Gls [5]), w prsn h rsuls for ML smon [30] nd sm-d ML smon n Tbl 4.3. W s for nosy d, h h prformnc cully flls whn usng smd for Ss A nd C, h ncrs n prformnc for s B dfnd n rms of word rror r rducon s 3.55%. A smlr comprson bwn MCE nd MCE w/ fur rnsformon (. comprng columns 3 & 4 of Tbl 4.) shows rducon n prformnc for S C, bu n ncrs n prformnc for S A nd S B nd n rms of word rror r s rducon of 3.9% nd 9.43%. Dong h sm comprson for cln d (columns 4 & 5 of Tbl 4.3) h rducon n word rror r ws 6.89% nd

51 44 4.% for h ML cs comprd o 8.43% nd 33.6% for h MCE cs. Thus w s hr s sgnfcn mprovmn n prformnc whn usng MCE for fur rnsformon hn ML, n mos css. Thus vn hough MCE bsd rnsformon dos no dcorrl h furs nd hnc h corrlon nformon s no xplcly rprsnd by h dgonl covrncs, h rnsformon bsd on mnmzng loss nd usng MCE dscrmnv rnng, lds o grr mprovmn n rcognon prformnc. ML Esmon [] ML Esmon w/ Sm-Td ML Esmon Cln D [] ML Esmon w/ Sm- Td Cln D S A S B Sm s S A Sm s S A S C TABLE 4.3 Comprng h vrg word ccurcy for ML nd ML Bsd Sm-Td modlng for h nr s s nd us h cln d 4.. Comprson o Ohr Suds Don on Auror dbs Th Auror dbs ws dvlopd s sndrd o compr fur procssng chnqus h r robus o nos. In sp of s drwbcks of no hvng lvl vrons nd lnr dsorons h occur n nurlly nosy nvronmn, hr s vrbly du

52 45 o unmchd rnng nd sng scnros dffrn SNRs. In hs scon w dscuss som of h ohr suds h hv usd fur vcors robus o nos, nos modlng pprochs or ohr fur rnsformons for robus spch rcognon nd s how hr pprochs compr o h MCE bsd fur rnsformon. Th work by Srm. l. [4] combnd dffrn fur ss (ncludng mporl nd spcrl procssng fur ss) usng ANN non-lnr rnsformon. Thy usd rror-bck propgon o sm h ANN prmrs. Th rsuls of hs sudy wr prsnd o compr h rlv droron of h rcognon ccurcy bsd on h SNR lvls s shown n Tbl 4.4. Comprng hs rsuls o h obnd usng MCE bsd fur rnsformon (Row, Tbl 4.4). w s h rcognon ccurcy on cln d r comprbl, bu h rsuls of our sudy on nosy d rn s good s h of Srm. l s work, nd could b bcus of h lowr rcognon prformnc on Ts S C (rfr Tbl 4.). Rsuls for ndvdul s ss wrn prsnd n [4] o do br comprson. Cln 0 db 5 db 0 db 5 db 0 db Non-lnr fur rnsformon Srm. l.[4] MCE bsd fur nd modl dsgn TABLE 4.4 Comprng rsuls wh Srm. l. s work on non-lnr fur xrcon

53 46 Zhu. l.[3] n hr work combnd MFCCs wh furs xrcd usng four dffrn fron-nd procssng chnqus, nmly, vrbl frm r, pk solon, pk-o- vlly ro lockng nd hrmonc dmodulon (for dls pls rfr [3]). Ths fur procssng mhods wr shown o b mor robus o nos hn h sndrd MFCC furs. Th rsuls of hs sudy r prsnd n Tbl 4.5. Anohr sudy on h Auror dbs ws don by Droppo. l.[3], whr hy usd sro d.., boh cln nd nos dsord rnng d o sm corrcon vcors. So h mn nd vrncs of h modls r frs smd usng cln spch nd h nosy spch s hn usd o rn h corrcon vcors, o dvlop mppng bwn h nos cpsrum nd h cln cpsrum. In ordr o dcrs h dpndnc of hs lgorhm on h nos sscs n h rnng d nd for br prformnc on unmchd condons whr h nos n h s s s dffrn from h rn s hy mployd Nos Mn Normlzon [3]. Th rsuls of hr work r comprd o h prsn work on MCE smon nd MCE bsd fur rnsformon n Tbl 4.5. MCE Esmon MCE Bsd Fur nd Modl Dsgn Zhu. l. [3] Droppo. l. [3] S A S B S C TABLE 4.5 Comprng rsuls wh Zhu. l. s [3] nd Droppo. l. s [3] work I cn b sn from h comprsons n Tbl 4.5, h MCE bsd fur rnsformon gvs br rcognon ccurcy hn Zhu. l. s nos robus fron

54 47 nd procssng chnqus, xcp for h s C. Droppo. l s work [3] whr hy us sro d o sprly rprsn h nos chrcrscs gvs br rcognon rsul for h mchd rn nd s s (nmly S A rsuls). Howvr for h unmchd cs MCE bsd fur rnsformon gvs br rcognon prformnc. 4.3 Conclusons nd Fuur Work Th opmzon of h fur rnsformon mrx nd h HMM modls usng mnmum clssfcon rror shows sgnfcnly mprovd prformnc for no only cln d, bu lso for rfclly ddd nosy d. Th rlv rducon n word rror r ws found o b lrgr hn h sm-d ML smon for h mchd nos cs, bu smllr for h unmchd cs showng h fur rnsformon usng dscrmnv MCE rnng s br hn h ML pproch, bu loosng h corrlon nformon ffcs h rcognon ccurcy n unmchd condons. Ths pproch prformd br hn mos nos robus pprochs sd on h Auror dbs. By usng dgonl lmns of h covrnc mrx, som of h corrlon nformon s los nd h rnsformd fur vcor mgh no b n dqu pproxmon o h orgnl fur vcors nd y s prformnc s br hn mny ohr nos robus pprochs [4], [3], [3] h wr lso sd on h Auror dbs. Clss-dpndn fur rnsformons hv bn mployd n h ps for ffcv spch rcognon whr dffrn word clsss lk consonns nd vowls c, cn b

55 48 rnsformd dffrnly [5]. Ths pproch cnno b xndd o nclud dffrn clss dpndn rnsformons, bcus h comprson of clss condonl probbls cross dffrn sub-spc clsss cnno b don. Anohr dsdvng of hs pproch s h s compuonlly xpnsv snc rqurs mrx mulplcon o rnsform h vrnc (4.4),(4.5). Du o h loss of h corrlon nformon h us of lowr dmnson rnsformon mrx lds o sgnfcn rducon n rcognon ccurcy. Prvous suds hv usd MCE [33] nd Mnmum Vrfcon Error ( vron of MCE [34]) bsd fur rnsformon for dpon, bu consdrd full covrnc mrcs n hr upd quons. Adpon of h rnd HMM modls o vrons n spkr or nvronmn condons r wdly usd n spch rcognon. For fuur work, h prsn pproch of usng dgonl covrnc for MCE bsd fur rnsformon cn b xndd for modl dpon.

56 49 CHAPTER 5 LOSS FUNCTIONS FOR MCE ESTIMATION - BASED ON THE PARZEN WINDOW ESTIMATES OF THE BAYES RISK Th only loss funcon h s mos commonly usd for MCE s h sndrd sgmod loss funcon s dfnd n (.7). McDrmo s work [35] showd usng Przn wndow Esmon h mnmzon of h sndrd sgmod loss funcon could pproch Bys Rsk, whn crn ssumpons r ssfd. Th work prsnd n hs chpr nvsgd ohr funcons h ssfy h Przn wndow consrns o s f hy cn b usd o mprov h rcognon ccurcy for spch rcognon sks. 5. Przn Wndow Esm of Bys Rsk A smlr xposon o h don n Chpr for MCE s prly rpd hr; hs prsnon s grd o showng h h MCE loss funcon cn b obnd by consdrng h Przn wndow sm of h Bys rsk. So onc gn w consdr h cs of n M-clss clssfcon sk, h gol s o dvlop clssfr wh dcson

57 funcon h: X C h ccurly mps clss lbl C C,C,...C M 50 o vry fur vcor x X, whr X s h fur spc. Thus w sk funcon h h mnmzs h clssfcon rror. Th Bys clssfcon rror s dfnd s M R R(C /x)p(x)dx P(C /x)p(x)dx (5.) X X W ssum h h fur vcors hv probbly dnsy p(x) nd R(C /x) s h condonl rsk dfnd s, R(C M /x) P(C /x) (5.) whr P(C /x) s h posror probbly nd s h cos of wrongly clssfyng n obsrvon x from clss C no clss C. For MCE smon h cos funcon s shown n (5.3) s usd. x,λ d 0 (5.3) whr (.) s h ndcor funcon nd d x,λ s msclssfcon msur h xplos h dscrmnn nformon. I s lso msur of clss sprbly nd s dfnd s d x,λ g x, Λ x,λ mx g (5.4)

58 5 subsung for d from (5.4), h cos funcon cn b rwrn s n (5.5) x,λ mx g x,λ g 0 (5.5) If g x,λ s grr hn g x,λ (whch s h dscrmnn funcon for h hypohszd clss C ) hn X s wrongly clssfd o C, hnc x,λ 0 d nd s ssgnd pnly of [(5.3) & (5.5)]. So h rsk dfnd n rms of h msclssfcon msur d s M R d, xdx X x,λ 0 p C (5.6) Followng long h lns of work by McDrmo. l [35] h rsk cn b xprssd s M dx P(C ) M R P (C ) p x/c p(m /C )dm (5.7) X 0 whr X x X d (x,λ) 0 s h s of fur vcors h r ms-clssfd, P(C ) N / M N N /N, s h pror probbly, N s h numbr of frms h blong o h clss C nd N s h ol numbr of frms nd m d (x, Λ). Th scond quly n (5.7) s obnd by convrng h probbly dnsy from h fur domn x o h msclssfcon msur domn m [35], [38]. Th convrson o h

59 5 msur domn fcls h Przn wndow smon of h dnsy p(m /C ) s shown blow. N m d (xr, Λ) p N m /C K (5.8) N r h h whr m d (xr,λ) K s h Przn wndow or h krnl funcon of bndwdh h, h cnrd vry d pon d (xr, Λ) nd xr s mong h N d smpls hypohszd s blongng o clss C. Th rsulng rsk, xprssd n rms of h Przn wndow sm, s : N M m d (xr, Λ) M R N L K dm Ld (xr, Λ) N (5.9) N 0 r h h whr L(.) s h loss funcon. Ths mprcl rsk wll pproch Bys rsk s h Przn wndow sms convrg o h ru dnsy, whch cn b obnd by ncrsng h numbr of rnng smpls o nfny [3]. Th convrgnc o Bys rsk lso dpnds on mkng h rgh ssumpon bou h clss condonl lklhood nd on h opmzon lgorhm convrgng o h globl mnmum [35]. W s from (5.9) h usng dffrn krnl funcons, loss funcons h mnmz h mprcl rsk R N (L) cn b obnd nd f h bov mnond ssumpons r ssfd h mprcl rsk could pproch h Bys rsk.

60 53 5. Sndrd Sgmod Loss Funcon s Przn Wndow Esm For Przn wndow smon h krnl funcons h cn b usd hv o ssfy h followng wo consrns [3], K u 0 nd K(u)du u (5.0) To obn h sgmod loss funcon h s ypclly usd n MCE, L(d) (5.) d h krnl funcon shown n (5.) s usd. K(u) u (5.) u whr u m d h (xr, Λ) nd γ h. Ths krnl funcon ws obnd by smply kng h drvv of h loss funcon (5.). I s srghforwrd o s h hs funcon s posv for u 0,, hus ssfyng h frs krnl consrn gvn n (5.0). Th quon blow shows h h krnl dfnd n (5.) ssfs h scond krnl consrn s wll.

61 54 K u u du u u du u u (5.3). Thus mnmzng h sndrd sgmod loss funcon mnmzs h clssfcon rsk nd hs clssfcon rsk could pproch h Bys rsk, provdd h ssumpons dscrbd n Scon 5. r ssfd. Th krnl for h sndrd sgmod loss funcon (5.) s no h only funcon h ssfs h Krnl funcon. In h followng scon w xplor ohr funcons nd s hr prformnc on spch dbs o s f hy mprov rcognon prformnc. 5. Dvlopmn of Ohr MCE Loss Funcons 5.. Svg Loss On choc for loss funcon s h Svg loss (5.4) nroducd by Shrz. l. [36]. L(d) (5.4) d Shrz.l. [36] showd usng svg loss nd probbly lcon from sscs, h boundd loss funcon, wh convx rsk prforms clssfcon br hn convx loss funcons lk ls squrs, xponnl loss (usd for boosng), hng loss (usd n suppor vcor mchns) nd logsc rgrsson. (Shrz. l. showd hs for bnry