Unified Framework for Single Channel Speech Enhancement

Transcription

1 Unfe Framewor for Sngle Channel Speech Enhancement Ivan Tashev, Anrew Lovtt, an Alex Acero Mcrosoft Research, Mcrosoft Corporaton {vantash, anlovtt, Abstract In ths paper we escrbe a generc archtecture for sngle channel speech enhancement. We assume processng n frequency oman an suppresson base speech enhancement methos. The framewor conssts of a two stage voce actvty etector, nose varance estmator, a suppresson rule, an an uncertan presence of the speech sgnal mofer. The evaluaton corpus s a synthetc mxture of a clean speech (TIMIT atabase an n-car recore noses. Usng the framewor multple speech enhancement algorthms are tune for maxmum performance. We propose a formalze proceure for automate tunng of these algorthms. The optmzaton crteron s a weghte sum of the mean opnon score (PESQ-MOS, sgnalto-nose-rato (SNR, log-spectral stance (LSD, an mean square error (MSE. The propose framewor proves a complete speech enhancement chan an can be use for evaluaton an tunng of other suppresson rules an voce actvty etector algorthms.. Introucton Fgure. System esgn for the speech enhancement framewor propose n ths paper. Ase from capturng the esre speech sgnal, the mcrophones n telecommuncaton systems capture ambent nose an the speech sgnal corrupte by room reverberaton. As a result the capture speech sgnal has a ecrease qualty an thus the unerstanng requres hgher cogntve loa. Speech enhancement s a general term for sgnal processng algorthms amng to mprove varous propertes of the capture speech sgnal. Nose reucton (suppresson an cancellaton, echo reucton (cancellaton an suppresson an ereverberaton are typcal speech enhancement applcatons. The nose suppresson, echo resual suppresson, an some e-reverberaton algorthms nvolve the applcaton of a real value, tme varyng flter to the frequency-oman transform of a capture sgnal. In ths paper we propose an archtecture for a sngle channel speech enhancement framewor for suppresson base algorthms, an present a formalze proceure for the evaluaton an the tunng of nose suppressors for achevng maxmum performance. The optmalty crteron utlze s a weghte sum of several evaluaton parameters wth major contrbuton from the MOS compute usng the PESQ algorthm. As an llustraton of the framewor an the evaluaton proceure, we prove a comparson of the most commonly use suppresson rules where each one s optmze separately.. Speech enhancement framewor The overall archtecture of the speech enhancement framewor s shown n Fgure. Let s( nθ represent values from a fnte-uraton analog sgnal sample at a regular nterval θ. The corrupte sequence s represente by the atve observaton moel xn ( θ sn ( θ + n ( θ, where xnθ represents the observe sgnal at tme nex n, an nθ s atve ranom nose, whch s uncorrelate wth the orgnal sgnal. The goal of sgnal enhancement s to form an estmate sˆ( nθ of the unerlyng sgnal s( nθ base only on the observe sgnal xnθ. In many mplementatons where effcent real-tme performance s requre, the set of observatons s fltere usng the overlap-a metho of short-tme Fourer analyss an synthess. Tang the screte Fourer transform on wnowe frames yels K complex frequency bns

2 per frame: S + D. In ths paper the frame nex s omtte wherever possble. Nose reucton n ths framewor may be vewe as the applcaton of a suppresson rule, or nonnegatve real-value ( gan n, to each bn of the observe sgnal spectrum n, n orer to form an estmate S ˆ of the orgnal sgnal spectrum. In most cases the suppresson rule s a functon of the a pror an a posteror SNRs. They are efne [5] respectvely as: λs, ξ γ λ λ ( ere λ { E D } an λ { s E S } are the power spectra of the nose an speech sgnals. The estmaton of the suppresson rule assumes a mxture of speech an nose, whch s not always the case as pauses are an ntegral part of human speech. Therefore the suppresson rule s mofe to reflect ths mxture wth the probablty of the speech sgnal presence: Sˆ n n n n P. Ths spectral estmate s then nverse-transforme to obtan the tme-oman sgnal reconstructon. In a real value suppresson rule the phase of the estmate sgnal s the same as that of the nput sgnal []. 3. Voce actvty etectors an nose moels The suppresson rule estmaton s base on statstcal moels of the nose an speech sgnal. It s assume that the nose changes slower than the speech sgnal an can be moele as a zero mean Gaussan process. Thus the moel conssts of one parameter per frequency bn the nose varance λ. The nose varances are estmate an upate urng speech pauses, whch are etermne by a voce actvty etector (VAD. The most commonly use algorthms use statstcal methos an nee a nose moel to stngush the sgnal from the nose. Due to ths epenency a ual stage voce actvty etector s propose. The frst stage classfes auo frames by estmatng the probablty of the speech sgnal presence P n the current frame. In many cases the frst stage s sgnal energy base, an gves a bnary ecson: speech or nose. The sgnal energy s estmate after applyng K a flter W as L ( W.. The flter s optmal n the sense that t maxmzes the fference between the nose an speech segments: K ( W S ( W D K W arg max.., ( W where S an D are the average speech an nose magntue spectra respectvely. For typcal speech an nose spectra ths flter converges to a hgh pass flter wth a cut-off frequency aroun 5-3 z. Some of the stanarze weghtng functons, such as [], can be substtute as well. The nose floor can be aaptvely trace wth two fferent tme constants: one larger when the current level s hgher than the estmate an one smaller, when the level s lower than the estmate. Thus the mnmum level wll follow changes own qucly an up slowly: T ( n T ( n Lmn + L L > L mn up up Lmn. (3 T ( n T ( n Lmn + L L Lmn own own ere T s the frame uraton, up an own are the two tme constants, an ( n L mn s the estmate nose floor. The ecson for swtchng the state V (speech or nose s threshol base, usng two threshols own an up ( up > own : n n f L Lmn < own V f L Lmn > up (4 ( n V otherwse Regarless of ts smplcty, ths bnary ecson VAD classfes the sgnal well. Utlzng the per-frame VAD ecson a rough nose moel λ n s bult an upate only urng nose frames V : T λ λ λ. (5 + ( n ( n r ere r s the upatng tme constant, n s the average magntue of the nput sgnal across several neghborng bns. The rough nose moel s use by the seconary VAD, whch proves the speech sgnal presence probablty for each frequency bn P (. A statstcally base VAD s publshe n [9]. Assumng a Gaussan strbuton of the nose an speech sgnal, wth varances n λ, an n λs, respectvely, we conser two hypotheses: (speech plus nose an (nose only wth probablty ensty functons: p( exp (6 πλ, λ,

3 Table. Suppresson rules optmze n the framewor. Rule Ref. Formula MMMSE [] λ MMSE wth DDA Maxmum Lelhoo Spectral Subtracton Short Term MMMSE Short Term log-mmse [], ξ [5] + ξ [4] [3] ξ + + ξ p( exp (7 π ( λ, + λs, λ, + λs, whch yels a lelhoo rato p( γξ Λ exp. (8 p( + ξ + ξ Usng a frst orer MM the authors apply a hangover scheme for smoothng the lelhoo n tme: ( n a + aγ Γ Λ ( n, (9 a + aγ where a j s the probablty for swtchng from state Η to state Η j. The smoothe lelhoo for speech sgnal presence s P Η Η Λ. Γ > P ( Η < ( Η whch s ether compare wth certan threshol for bnary ecson or use to estmate the probablty: Λ P. ( +Λ The speech presence lelhoo for the entre frame s: Λ exp log Λ. ( K Then s apple a smoothng proceure as n equatons (9 an ( wth state change probabltes b an b. ξ + ξ πν ν ν ν [5] ( + ν I + ν I exp γ [6] JMAP SAP [7] MAP SAE [7] MMSE SP [7] ξ exp( t t + ξ t υ ξ ξ + ξ + ( + ξ ( + ξ γ ξ ξ + ξ + ( + ξ ( + ξ γ ξ + ν + ξ λ The speech presence probablty for the entre frame P s estmate accorng to (. The presente soft VAD s just an example as any publshe VADs whch prove a speech presence probablty per frequency bn can be use an evaluate n the presente framewor. The speech sgnal s sparse n frequency oman an even wth speech present n the current frame many frequency bns contan only nose. We can therefore upate the nose varance n the precse nose moel λ n : ( n T ( n λ, λ, + ( P ( P ( λ,. (3 4. Suppresson rules Utlzng the nown a pror an a posteror SNRs efne n ( an the magntue, the p goal s to estmate the suppresson rule whch s optmal base on the crtera. A selecton of the most commonly use suppresson rules are presente n Table. For a Gaussan strbuton of the nose an speech sgnals the optmal suppresson rule n the MMSE sense s the well nown Wener flter []: λs ξ. (4 λs + λ + ξ The estmaton of the pror SNR s not trval, usng the ML estmator ξ γ leas to: max, ( λ γ. (5 γ In the wely use form of equaton (5, the Wener suppresson rule requres only the nose varance λ. Ths Wener estmator ntrouces a storton to the estmate sgnal whch s calle muscal nose. Ths s the auble bubblng hear urng the pauses, whch s cause by the approxmaton n (5. To reuce the stortons n [3] Boll propose the spectral subtracton rule, whch s less aggressve an ntrouces less storton but suppresses less nose. Later McAulay an Malpass [4] erve a maxmum-lelhoo (ML spectral ampltue estmator uner the assumpton of a Gaussan nose an an orgnal sgnal characterze by a etermnstc waveform of unnown ampltue an phase. Ths suppresson rule s always greater than.5, whch completely elmnates the muscal nose, but reuces the amount of nose t suppresses. As an extenson of the unerlyng moel, Ephram an Malah [5] erve a mnmum mean-square error short-tme spectral ampltue estmator (ST MMSE

4 assumng that the Fourer expanson coeffcents of the orgnal sgnal an the nose may be moele as statstcally nepenent, zero-mean, Gaussan ranom varables. Ther suppresson rule s a functon of both the a pror an the a posteror SNRs. In Table I( an I( enote mofe Bessel functons of zero an ξ frst orer an ν γ. Ths spectral magntue + ξ estmator proves nose suppresson whle mantanng lower stortons an fewer artfacts. The shape of ths suppresson rule s shown n Fgure. The fact that humans hear n a logarthmc scale of the soun pressure level s use n [6] where Ephram an Malah erve a suppresson rule whch s optmal n the MMSE log-spectral ampltue sense (ST log- MMSE. Regarless of the qute fferent crteron for optmalty, ths suppresson rule s surprsngly smlar to the ST MMSE suppresson rule. The mean of the fference between the two rules s. B, an the maxmum fference s only.46 B for ξ [ 3, + 3 ] B an γ [ 3, + 3 ] B. In [7] Wolfe an Gosl erve three more suppresson rules, Jont Maxmum A Posteror Spectral Ampltue an Phase (JMAP SAP Estmator, Maxmum A Posteror Spectral Ampltue (MAP SA Estmator, an MMSE Spectral Power (MMSE SP Estmator. The erve suppresson rules are effcent to mplement for real-tme executon as they o not contan Bessel functons an exponents. Whle the ervaton s nterestng, the suppresson rules have qute smlar to ST-MMSE shape. 5. Pror SNR estmaton an uncertan presence of the speech sgnal The success of the Ephram an Malah suppresson rule s largely ue to the authors' ecson-recte approach (DDA for the estmaton of the a pror SNR ξ. For the n-th auo frame the DDA a pror SNR estmate ˆ ξ n s a geometrc weghtng of the SNR n the prevous an current frames: ˆ ( n ξ α + ( α.max,( γ n λ. (6 The ablty to estmate the a pror SNR enables the usage of equaton (4, maxmum lelhoo, an spectral subtracton rules n ther more precse forms. All suppresson rules so far were erve uner the assumpton that both speech an nose sgnals are present. The speech sgnal s sparse n both tme an Suppresson rule, B Fgure. ST-MMSE suppresson rule gan plotte aganst the a pror an a posteror values. frequency, an n ther paper McAulay an Malpass [4] use a mofer for the suppresson rule: P., (7 where n s the mofe suppresson rule, n s the estmate suppresson rule uner the assumpton of speech sgnal presence an P n s the probablty of a speech sgnal n the current frame. The ervaton of (7 s generc an oes not epen on any assumptons for specfc strbutons of the speech an nose. Ephram an Malah n [5] use an earler wor [8] to mofy the suppresson rule uner the uncertan presence of a speech sgnal: Λ (, q. (8 +Λ (, q where Λ ( Y, q s the generalze lelhoo rato: P( Λ ( Y, q μ (9 P( wth μ ( q / q an q s the probablty of only nose sgnal n the -th frequency bn. From equatons (8 an (9 t s seen that from the etecton an estmaton stanpont the suppresson rule mofer s just the probablty of a speech sgnal presence, whch s generalze n (7. 6. Practcal aspects a posteror SNR γ, B Ephram an Malah MMSE rule 3-3 Proper measures shoul be taen to prevent vson by zero; usually ths s mplemente by ang a small real number to the enomnator. In most runtme lbrares the exponents or logarthms have ther lmtatons for nput parameters an prove nval result beyon them. In such cases the argument value shoul be lmte to meet these lmtatons. For example: the exp( argument n a ouble precson mplementaton shoul be lmte to values below ~ a pror SNR ξ, B 3

5 Estmaton of the a pror SNR usng DDA converts the nose suppressor nto a system wth feebac. Ths rases stablty concerns when the suppresson gan has a value above one. Lmtng the suppresson rule to one guarantees the nose suppressor stablty. When the suppresson gan goes to zero t causes muscal nose. To reuce ths effect n practce the suppresson gans are lmte to a certan mnmal values an thus equaton (7 becomes: GUm + ( GUm P Gm + ( Gm ( ere G m s the mnmal gan for the suppresson rule an G Um s the mnmal gan for applyng the uncertan presence of the speech sgnal. These parameters are usually frequency nepenent. 7. Evaluaton an tunng The goal of speech enhancement an nose suppresson s not actually to suppress nose. We o not want MMSE estmaton, ML, or ST log-mmse estmaton of the speech sgnal. The actual goal s for humans to perceve the output as havng a better qualty,.e. we want to maxmze the perceptual soun qualty. The most commonly use crteron for perceptual speech qualty s the Mean Opnon Score (MOS []. Ths metho nvolves real humans an s thus slow an expensve. The Perceptual Evaluaton of Soun Qualty (PESQ, stanarze n [], s a sgnal processng algorthm whch proves an estmaton of MOS n a fast an effcent way. Recently publshe results show hgh correlaton between the PESQ an speech recognton rate [3]. Among the most common evaluaton parameters for nose suppressors s the mprovement n sgnal-to-nose-rato (SNR, efne as the proporton of the average power of the sgnal an nose frames. It s usually measure n B or n BC f t s C- weghte. When the clean speech sgnal s avalable (ether when usng synthetcally corrupte sgnals, or recore wth a close tal mcrophone other commonly use evaluaton parameters are the mean square error (MSE an the log-spectral stance (LSD. It s mportant that the tunng an evaluaton of statstcally base speech enhancement algorthms happen aganst a large corpus of ata. The speech enhancement framewor escrbe has a set of parameters that cannot be estmate rectly. These parameters are the aaptaton tme constants,,,,, probabltes ( own up r p, threshols ( own up ( a, a, b, b, geometry weghtngs ( α, VAD lmtng gans (, α, an G G. These parameters convert m Um Table. Optmzaton results for rules n Table. Rule PESQ SNR MSE LSD G m G Um Base lne MMSE MMSE DDA ML SS ST-MMSE ST log-mmse JMAP SAP MAP SAE MMSE SP the evaluaton an tunng process of each speech enhancement algorthm nto a multmensonal optmzaton problem to fn the best values of these parameters. The optmzaton metrc can be the weghte sum of several optmzaton crtera: Q wq. ( Once we have efne the optmzaton crteron the tunng proceure can use most of the algorthms for mathematcal optmzaton, such as the smplex metho an the entre varety of graent base approaches. It s common practce to splt the evaluaton corpus n three parts: the frst s use wth the optmzaton algorthm, whle the secon s evaluate after every teraton. The optmzaton proceure stops f several consecutve teratons show worse results aganst the secon set. The optmal soluton s the one whch was best wth regars to the secon set of ata. The fnal evaluaton s conucte wth the thr set, whch the optmzaton process not use. 8. Expermental results The presente framewor was use to tune an evaluate a speech enhancement algorthm for a communcaton system ntegrate nto a car. The evaluaton corpus was synthetcally generate. We measure the mpulse responses between several potental postons of the rver s mouth an a set of mcrophone postons. The measurements were one by playng a chrp sgnal through a hea an torso smulator an recorng the response wth the mcrophone. The same mcrophone was use for recorng noses n varous rvng contons such as a pare car, se street rvng, busy street rvng, an hghway rvng. These four basc scenaros were mofe by turnng the ar contoner on at varous levels. As a clean speech source we use the 63 utterances from TIMIT atabase [4]. Each utterance was convolve wth a ranomly selecte mpulse response an ranomly selecte nose segment was ae. Proper correcton for the Lombar effect was one on energy level base on the SNR. The corpus was ve n three parts wth the same strbuton of the SNRs. For tunng the parameters the steepest graent escent

6 Table 3. Optmal parameter values for the VAD own up own up algorthm was use. The optmzaton crteron Q was PESQ wth weght w.. The other three crtera (SNR n BC Q, MSE Q 3, an LSD on BC Q 4 ha weghts w., w 3., an w 4.. The optmzaton crteron s compute as the average result from all fles n the set. Each of the presente suppresson rules was tune separately usng sets one an two. The results from the evaluaton of the suppresson rules wth the thr set are presente n Table. The SNRs an LSDs are measure n BC. Overall the parameters of the two stage VAD converge to approxmately the same values an the parameters that vare the most were the mnmal gans. Table 3 recors the optmal values of the VAD parameters, whle the gan parameters for each suppresson rule are shown n Table. All tme constants n Table 3 are n secons. 9. Dscusson an conclusons The presente sngle channel framewor for suppresson rule base speech enhancement s a flexble tool for evaluaton an tunng of these algorthms. The propose formalze optmzaton proceure an evaluaton crteron s applcable for many speech enhancement algorthms an ther components. The VAD use s just an example an can be replace wth almost any of the publshe algorthms. It s ffcult to etermne how much mprovement the optmzaton process brngs, as t epens on the ntal conton. In all cases the proceure saves tme an resources over manual evaluaton an tunng. Usng a proper evaluaton corpus s crtcal for the successful tunng. The strbuton of the SNRs an the nose characterstcs shoul be as close as possble to the real contons. Overall the best performng algorthms wth the corpus we use are MAP SAE an MMSE DDA whch mprove PESQ wth.57 ponts. Among the best performers are JMAP SAP, MMSE SP, an ST log- MMSE. Increasng n LSD shows that all algorthms o not eal well wth lower magntues an SNRs. We evaluate several other nustral nose suppresson systems aganst the same corpus an they prove mprovement n the range of.7.8 PESQ ponts,.e. the propose archtecture an optmzaton methoology brng an auble mprovement n perceptual soun qualty. r p a a b References b α VAD α [] N. Wener, Extrapolaton, Interpolaton, an Smoothng of Statonary Tme Seres: Wth Engneerng Applcatons, Prncples of Electrcal Engneerng Seres. MIT Press, Cambrge, MA, 949. [] D. L. Wang an J. S. Lm, The unmportance of phase n speech enhancement, IEEE Trans. on Acoustcs, Speech, an Sgnal Processng, vol. ASSP-3, no. 4, pp , Aug. 98. [3] S. Boll, Suppresson of acoustc nose n speech usng spectral subtracton. IEEE Trans. on Acoustcs, Speech, an Sgnal Processng, vol. ASSP-6, pp. 3-, 975. [4] R. J. McAulay an M. L. Malpass, Speech enhancement usng a soft-ecson nose suppresson flter, IEEE Trans. on Acoustcs, Speech, an Sgnal Processng, vol. ASSP-8, no., pp , Apr. 98. [5] Y. Ephram, D. Malah, Speech enhancement usng a mnmum mean-square error short-tme spectral ampltue estmator, IEEE Trans. on Acoustcs, Speech, an Sgnal Processng, vol. ASSP-3, no. 6, pp. 9, Dec [6] Y. Ephram an D. Malah, Speech enhancement usng a mnmum mean-square error log-spectral ampltue estmator, IEEE Trans. on Acoustcs, Speech, an Sgnal Processng, vol. ASSP-33, no., pp , Apr [7] P. Wolfe, S. Gosll. Smple alternatves to the Ephram an Malah suppresson rule for speech enhancement. Proc. of th IEEE Worshop on Statstcal Sgnal Processng, ,. [8] D. Mleton, R. Esposto. Smultaneous Optmum Detecton an Estmaton of Sgnals n Nose. IEEE Trans. on Informaton Theory, vol. IT-4, No. 3, May 968. [9] J. Sohn, N. Km, W. Sung. A statstcal moel base voce actvty etector. IEEE Sgnal Processng Letters, vol. 6, No., pp. -3, January 999. [] ITU-T Recommenaton P.8. Methos for subjectve etermnaton of transmsson qualty. Geneva, Swtzerlan, 996. [] ITU-T Recommenaton P.86. Perceptual evaluaton of speech qualty (PESQ: an objectve metho for ento-en speech qualty assessment of narrow-ban telephone networs an speech coecs. Geneva, Swtzerlan,. [] ITU-T Recommenaton O.4. Psophometer for use on telephone-type crcuts. Geneva, Swtzerlan, 994. [3] P. Dng, J. ao. "Assessment of Correlaton between Objectve Measures an Speech Recognton Performance n the Evaluaton of Speech Enhancement". Proc. of Interspeech 8, Brsbane, Australa. [4] John S. Garofolo, et al. "TIMIT Acoustc-Phonetc Contnuous Speech Corpus", Lngustc Data Consortum, Phlaelpha, 993.