ON THE USE OF PHASE INFORMATION FOR SPEECH RECOGNITION. Baris Bozkurt and Laurent Couvreur

Transcription

1 ON THE USE OF PHASE NFOMATON FO SPEECH ECOGNTON Baris Bozkurt and Laurent Couvreur TCTS Lab, Faculté Polytechnique De Mons, nitialis Scientific Park, B-7000 Mons, Belgium, hone: , fax: , web: htt:// ABSTACT This study addresses the use of short-time hase sectra in automatic seech recognition (AS). Two recent studies [,2] have roosed two grou delay based sectral reresentations. Here we roose three new grou delay based reresentations and comare usefulness of all these reresentations in an AS exeriment. We show that two of the reresentations we roose erform better, contain equivalent or comlementary information to that of the ower sectrum and are otentially useful for imroving AS erformance.. NTODUCTON n most state-of-the-art AS systems, amlitude/ower sectrum has been the referred comonent of the Fourier Transform (FT) for feature extraction. However, recent studies on seech ercetion reort the imortance of hase information [3]. By its nature, the hase comonent of the FT sectrum is in a wraed form and the first derivative of the unwraed hase sectrum, i.e. the grou delay function, is much easier to study and rocess. The main difficulties in reliable hase sectrum estimation and unwraing are mostly related with the zeros of the signal's z-transform close to the unit circle, which cause sikes on the derivative of the hase sectrum (grou delay) []. Methods to remove these sikes are required in order to be able to use the FT hase information in AS. Two recent studies address this roblem and roose two grou delay based features: modified grou delay [] and roduct sectrum [2]. n this study, we introduce three new grou delay based reresentations and comare all five reresentations (and the ower sectrum for reference) in an AS exeriment. The results show that two of the reresentations that we roose rovide good results and contain equivalent or comlementary information to the ower sectrum that is otentially useful for imroving AS erformance. The sections are lanned as follows: section 2 resents the source of roblem in grou delay rocessing, the grou delay reresentations roosed in [,2] and the reresentations that we roose in this study. n section 3, we briefly define feature extraction rocedures for AS based on grou delay reresentations (the reader is referred to [2] for more detailed information). Section 4 is dedicated to AS exeriments and finally in section 5, we discuss the results. Figure : Geometric interretation of sikes in grou delay function at frequency locations close to a root/zero of the z- transform olynomial. 2. GOUP DELAY EPESENTATONS 2. Difficulties n Grou Delay Processing For a given discrete time digital signal x(n), the z-transform olynomial, X(z), can be exressed using an all-zero reresentation as: X ( z) N n 0 x( n) z n x(0) z N N + m ( z Z where Z m are the roots of the z-transform olynomial. The FT, which is simly the z-transform comuted on the unit circle, can be exressed as: N ( + ) ( ) jθ ( ω ) N ρe m jθ ( ω ) X ( ω ) x(0) ( ρe Z ) () where the radius ρ. Each factor in Eq. corresonds, in the z-lane, to a vector starting at Z m and ending at e j θ( ω ). As illustrated in Fig., the grou delay, i.e. the rate of change in the hase comonent, is very high at frequency bins very close to a root/zero of the z-transform olynomial and becomes ill-defined when the zero coincides with a frequency bin. For actual windowed seech signals, many zeros aear to be very close to the unit circle. The effect of zeros close to the unit circle can easily be observed both on amlitude sectra (as dis) and grou delay functions (as large sikes). For the amlitude sectrum a sectral enveloe with formant eaks is still observed/available with the resence of dis due to the zeros. Unlike amlitude sectrum, the grou delay function is effected to an extend that the sikes hide the actual vocal tract hase/grou delay information (see Fig. 2 for an examle). Therefore, it is necessary to find means of avoiding such sikes in the grou delay function for using it in feature extraction for AS systems. Below, we resent two methods from recent literature followed by our three methods for the estimation of grou delay reresentations, which can be further rocessed for feature extraction. m ) m

2 Figure 3: Power sectrum (PowerS) and grou delay reresentations for the seech signal frame in Fig Modified Grou Delay Function (MODGDF) [] The grou delay function can be exressed as: X Y Y τ 2 (2) X where X(ω) and Y(ω) denote to the FT of x(n) and nx(n) resectively, and refer to real and imaginary arts [2]. This formulation is quite useful since there is no need for hase unwraing, which is often referred to be roblematic, and grou delay can be directly comuted using the FT transform only. n [], the authors roose the so-called modified grou delay function (MODGDF), which is a modified version of Eq. 2: τ τ ω ( τ ) α mod ( ) τ X Y Y τ 2γ S( ω) where the term X( ω ) is relaced by its cestrally smoothed version S( ω ) in order to reduce sikes on the grou delay function. This is because the term X( ω ) in the denominator in Eq. 2 gets very small when there exists a zero very close to the unit circle. n addition, two new arameters are introduced: α and γ which need to be fine-tuned according to the environment. n all tests/lots of this study, we have set the arameters as in [], namely α0.4 and γ0.9. These smooth- ing arameters also reduce the effect of the sikes on the grou delay function to some extend. 2.3 Product Sectrum (PS) [2] n [2], another grou delay based reresentation is roosed. t is a version of Eq. 2 where the denominator, which is considered to be the source of sikes, is removed. The roduct sectrum Q( ω ) is defined as the roduct of the ower sectrum and the grou delay function: 2 Q X τ X Y Y Figure 4: Sectrogram lots of the noise-free utterance mah_4625a. Only the first half of the signal that to the digit utterance 46 is resented. 2.4 Grou Delay of GC-Synchronously Windowed Seech (GDGC) ecently, we have shown that smooth grou delay functions can be directly comuted from seech signals if windowing is aroriately [4]: window centred at glottal closure instant (GC) and smaller than two itch eriods. n our comutations, we use a Blackman window function though other tyes are ossible, e.g. Gaussian or Hanning-Poisson. Such a windowing oeration results in grouing the zeros across the unit circle but not on it, therefore avoiding most of the sikes. GDGC is referring to the grou delay function comuted using Eq. 2 on GC-synchronously windowed seech data. GC detection is achieved by rocessing the centre-of-gravity evolution signal obtained by shifting an analysis window on the seech signal as described in [5]. 2.5 Chir Grou Delay of GC-Synchronously Windowed Seech (CGDGC) After tests on noisy real seech data, we have observed that GC-synchronous windowing cannot guarantee a comletely zero-free region on the unit circle. The zeros related to the noise signal comonent may aear on the unit circle and introduce extra sikes. n addition, the size of window aears to be a roblem for very high itch seech: an error to include more than two itch eriods in the seech frame results in zeros due to eriodicity on the unit circle. For this reason a two-ste method is roosed: suression of zeros outside the unit circle (which are mainly due to the glottal flow comonent of seech [6]) and comutation of the hase derivative on a circle outside the unit circle from the remaining zeros using Eq.. The roots of a high degree olynomial can be efficiently obtained by searching for the eigenvalues of the associated comanion matrix [7]. The rocedure rovides enough accuracy to carry reliable sectral analysis. Unfortunately, such al-

3 Table : Performances of AS system for various feature extraction on the AUOA-2 task. esults are given in terms of word error rate (WE) in ercent. For every SN value, the rightmost value corresonds to the case where the actual feature extraction is combined with MFCC as described in section 4.3. Feature SN (db) Extraction MFCC MODGDF PS GDGC CGDGC CGDZP gorithm for estimating olynomial roots is comutationally demanding. n order to obtain a very smooth grou delay function with well-resolved formant eaks, it is useful to comute the grou delay function on a circle other than the unit circle in the z-lane. Equivalently, one can comute the grou delay function of a chired version of the signal, i.e. multilied by a decaying exonential whose daming arameter is solely related to the actual circle radius ρ. t results in the GC-synchronous chired grou delay function (CGDGC). n [6], we have shown that vocal tract formants can be estimated easily by tracking the eaks of CGDGC. The choice of the radius ρ of the z-transform comutation circle is a comromise between having smooth/blurred or detailed/siky sectral reresentation. The value ρ.2 is observed to be a good choice. 2.6 Chir Grou Delay of The Zero-Phase Version (CGDZP) Finally we roose another grou delay function, for which heavy comutation of olynomial roots is not necessary. Again the rocedure contains two stes: comutation of the zero-hase version of the signal (inverse FT of X( ω ) ) and comutation of the CGD on the circle with ρ.2 using the chir z-transform. Conversion to zero-hase guarantees that all of the zeros occur very close to the unit circle therefore the resulting chir grou delay reresentation is very smooth with wellresolved formant eaks. However, the hase information is destroyed for this case, therefore the reresentation contains only the information available in the amlitude sectrum but formant eak resolutions aear with higher resolution. 2.7 Comarison of Proosed Methods via Sectral Plots Fig. 2 resents a tyical time-domain seech signal and its grou delay function. As exected, the grou delay function comuted directly on the seech frame contains mainly sikes and resonance information cannot be observed. n Fig. 3, we resent the five grou delay based reresentations together with the ower sectrum for this seech frame. The formant eaks aear with high resolution in GDGC, CG- DGC and CGDZP. GDGC includes a sike at high frequencies due to a zero, which cannot be avoided by only GC-synchronous windowing. As more noise was added to signals, such sikes would be more frequent, therefore the robustness of GDGC to noise is rather low. Thanks to zero removal techniques and zero-hasing, CGDGC and CGDZP are more robust to noise. n Fig. 4, we also resent sectrogram lots obtained using the described grou delay reresentations as well as the classical ower sectrum. The formant tracks can be well observed on all of the sectrograms excet for MODGDF, and PS is very close to PowerS as already shown in Fig. of [3] and in Fig. 3 above. GDGC reresentation is vague to some level. This is mainly due to the fact that unvoiced frames include sikes with large amlitudes that force a low contrast on the lots. Actually, the grou delay functions comuted on unvoiced frames mostly do not contain resonance information but random sikes. GDGC and CGDGC are actually the two reresentations that really suffer from this roblem. These observations suggest that the reresentations have some otential in an AS framework. The main concern is if they can rovide comlementary information to the ower sectrum and imrove erformance. 3. COMPUTATON OF FEATUES FO AS The most common feature extraction for AS systems consists of comuting ower-based Mel-frequency cestral coefficients (MFCC) [9], that is, a Mel filterbank is alied to the ower sectrum and an inverse discrete cosine transform (DCT) is comuted on the logarithm of its oututs. The main reason for such rocessing is to cature the essential shae of the ower sectrum with a few coefficients well conditioned for attern recognition. A similar scheme can be alied to the grou delay functions in order to derive hase-based feature extractions for AS systems. The simlest aroach consists in relacing the ower sectrum in the MFCC algorithm by the grou delay reresentation comuted via one of the analysis techniques described in the revious section. n this work, we use a Mel filterbank with 24 triangular filters and 2 DCT coefficients are comuted for 30 ms frames shifted by 0 ms. Note that the logarithm is not alied on the oututs of the filterbank when fed with a hase sectrum. These coefficients are augmented with the frame log-energy and their (delta-)delta coefficients. We finally come u with six feature extractions: MFCC as a reference and five grou delay based methods.

4 4. AS system 4. AS EXPEMENTS The AS system that is considered in this work relies on the STUT toolkit [0]. t merely consists of three blocks. First, the feature extraction chos the discrete seech signal into overlaing frames and comutes for each frame a set of acoustic coefficients using one of the algorithms described in the revious sections. Next, the acoustic coefficient vectors are fed into the acoustic model that is here based on the Multi Layer Percetron (MLP) / Hidden Markov Models (HMM) aradigm []. n this framework, the honemes of the language under consideration are modelled by HMM's whose observation state robabilities are estimated as the oututs of a MLP. Such an acoustic model is trained beforehand in a suervised fashion on a large seech database containing a few hours of honetically segmented seech material. Finally, the word decoder searches for the most likely word sequence given the sequence of robability vectors for all the frames. Here, the search is constrained by a honetic lexicon and a word grammar, which together define all the authorized sequences of honemes. Here, the search is erformed as a oneass frame-synchronous Viterbi algorithm [9] without any runing constraints. 4.2 Seech Database The AUOA-2 database [8] was used in this work. t consists of connected English digit utterances samled at 8kHz. More exactly, we used the clean training set, which contains 8440 noise-free utterances soken by 0 male and female seakers, for building our acoustic models. These models were evaluated on the test set A. t has 4004 different noisefree utterances soken by 04 other seakers. t also contains the same utterances corruted by four tyes of real-world noises (subway, babble, car, exhibition hall) at various signalto-noise ratios (SN) ranging from 20dB to -5dB. During the recognition exeriments, the decoder is constrained by a lexicon reduced to the English digits and no grammar is alied. 4.3 Exerimental esults Tab. gives the word error rates (WE) for the AS system tested with the feature extractions described in section 3. Errors are counted in terms of word substitutions, deletions and insertions, and error rates are averaged over all noise tyes. The results are also rovided when combining MFCC feature extraction with the others (rightmost figure for every noise level and feature extraction). The combination is simly erformed by taking a weighted geometric average of the robability oututs of the combined acoustic models: λ λ 2 2 where 2, and 2 denote the combined robability and the robability rovided by the two combined acoustic model, resectively. The combination arameter λ takes its value in the range (0,) and is otimised for every combination. 5. DSCUSSONS AND CONCLUSONS Our main target in this study is to test if a hase/grou delay reresentation carries equivalent or comlementary information to that of the ower sectrum in the framework of feature extraction for AS systems. The results resented in Tab. shows that the grou delay reresentations CGDGC and CGDZP have this otential: the rightmost values comared to the MFCC-only results are in all cases lower excet for the extreme noise setting SN-5dB. n our in-detailed analysis, we have observed that the GDGC, which is the ure grou delay function comuted on GC-synchronous data without further rocessing, mainly suffers from window size roblems (including several itch eriods result in zeros on the unit circle). n addition, GDGC and CGDGC do not carry reliable information for unvoiced frames. The AUOA-2 task was chosen for its simlicity and ease of comarison to the already available results in [2]. Further exeriments will be erformed on other tasks in order to confirm the resent results about the usefulness of hase information for AS systems. EFEENCES []. M. Hegde, H. A. Murthy and V... Gadde, Continuous seech recognition using joint features derived from the modified grou delay function and MFCC, in Proc. CSLP, Jeju, Korea, Oct [2] D. Zhu and K. K. Paliwal, Product of ower sectrum and grou delay function for seech recognition, in Proc. CASSP, Montreal, Canada, May [3] K. K. Paliwal and L. D. Alsteris, Usefulness of hase sectrum in human seech ercetion, in Proc. EU- OSPEECH, Geneva, Switzerland, Se [4] B. Bozkurt, B. Doval, C. D'Alessandro and T. Dutoit, Aroriate windowing for grou delay analysis and roots of z-transform of seech signals, in Proc. EUSPCO, Vienna, Austria, Se [5] H. Kawahara, Y. Atake, and P. Zolfaghari, Accurate vocal event detection method based on a fixed-oint to weighted average grou delay, in Proc. CSLP, Beijing, China, Oct [6] B. Bozkurt, B. Doval, C. D'Alessandro and T. Dutoit, mroved differential hase sectrum rocessing for formant tracking, in Proc. CSLP, Jeju, Korea, Oct [7] G. H. Golub and C. F. Van Loan, Matrix Comutations, 3rd Edition, Johns Hokins University Press, 996. [8] H. G. Hirsch and D. Pearce, The AUOA exerimental framework for the erformance evaluation of seech recognition Systems under noisy conditions, in Proc. AS2000, Paris, France, Se [9] X. Huang, A. Acero and H.-W. Hon, Soken Language Processing: A guide to Theory, Algorithm, and System Develoment, Prentice Hall, 200. [0] J.-M. Boite, L. Couvreur, S. Duont and C. is, Seech Training and ecognition Unified Tool (STUT), htt://tcts.fms.ac.be/asr/roject/strut.

5 [] H. Bourlard and N. Morgan, Connectionist Seech ecognition: A Hybrid Aroach, Kluwer Academic Publisher, 994.