Robustness Techniques for Speech Recognition

Transcription

1 Robustness Techniques for peech Recognition Berin Chen, 2005 References: 1. X. Huang et a. poken Language Processing (2001). Chapter J. C. Junqua and J. P. Haton. Robustness in Automatic peech Recognition (1996), Chapters 5, T. F. Quatieri, Discrete-Time peech igna Processing (2002), Chapter 13

2 Introduction Cassification of peech Variabiity in Five Categories Pronunciation Variation Intra-speaker variabiity Linguistic variabiity Inter-speaker variabiity peaker-independency peaker-adaptation peaker-dependency Robustness Enhancement Variabiity caused by the environment Variabiity caused by the contet Contet-Dependent Acoustic Modeing peech - Berin Chen 2

3 Introduction (cont.) The Diagram for peech Recognition Acoustic Processing Linguistic Processing peech signa Feature Etraction Likeihood computation Linguistic etwork Decoding Recognition resuts Acoustic mode mode Language mode mode Leicon Importance of the robustness in speech recognition peech recognition systems have to operate in situations with uncontroabe acoustic environments The recognition performance is often degraded due to the mismatch in the training and testing conditions Varying environmenta noises, different speaker characteristics (se, age, diaects), different speaking modes (styistic, Lombard effect), etc. peech - Berin Chen 3

4 Introduction (cont.) If a speech recognition system s accuracy does not degrade very much under mismatch conditions, the system is caed robust AR performance is rather uniform for Rs greater than 25dB, but there is a very steep degradation as the noise eve increases 25dB Es Es 2.5 = 10 og 10 => = 10 E E 316 igna energy measured in time domain, e.g.: E s = 1 T 1 T * [] n s [] n s n = 0 Various noises eist in varying rea-word environments periodic, impusive, or wide/narrow band peech - Berin Chen 4

5 Introduction (cont.) Therefore, severa possibe robustness approaches have been deveoped to enhance the speech signa, its spectrum, and the acoustic modes as we Environmenta compensation processing (feature-based) Acoustic mode adaptation (mode-based) Inherenty robust acoustic features (both mode- and featurebased) Discriminative acoustic features peech - Berin Chen 5

6 peech - Berin Chen 6 The oise Types [ ] [ ] [ ] [ ] ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) { } ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) () ( ) ( ) ( ) ( ) () spectrum :, or power spectrum :, or cos 2 2 Re * = + = = + + = + = + = P P P P P H H H H H X H X m n m h m s m nn hh ss H X θ h[m] s[m] n[m] [m] A mode of the environment.

7 Additive oises Additive noises can be stationary or non-stationary tationary noises uch as computer fan, air conditioning, car noise: the power spectra density does not change over time (the above noises are aso narrow-band noises) on-stationary noises Machine gun, door sams, keyboard cicks, radio/tv, and other speakers voices (babbe noise, wide band nose, most difficut): the statistica properties change over time peech - Berin Chen 7

8 Additive oises (cont.) peech - Berin Chen 8

9 Convoutiona oises Convoutiona noises are mainy resuted from channe distortion (sometimes caed channe noises ) and are stationary for most cases Reverberation, the frequency response of microphone, transmission ines, etc. peech - Berin Chen 9

10 oise Characteristics White oise The power spectrum is fat nn =,a condition equivaent to different sampes being uncorreated, R nn [ m] = qδ [ m] White noise has a zero mean, but can have different distributions We are often interested in the white Gaussian noise, as it resembes better the noise that tends to occur in practice Coored oise ( ) q The spectrum is not fat (ike the noise captured by a microphone) Pink noise A particuar type of coored nose that has a ow-pass nature, as it has more energy at the ow frequencies and ros off at high frequency E.g., the noise generated by a computer fan, an air conditioner, or an automobie peech - Berin Chen 10

11 Musica oise oise Characteristics (cont.) Musica noise is short sinusoids (tones) randomy distributed over time and frequency That occur due to, e.g., the drawback of origina spectra subtraction technique and statistica inaccuracy in estimating noise magnitude spectrum Lombard effect A phenomenon by which a speaker increases his voca effect in the presence of background noise (the additive noise) When a arge amount of noise is present, the speaker tends to shout, which entais not ony a high ampitude, but aso often higher pitch, sighty different formants, and a different cooring (shape) of the spectrum The vowe portion of the words wi be overemphasized by the speakers peech - Berin Chen 11

12 Robustness Approaches

13 Three Basic Categories of Approaches peech Enhancement Techniques Eiminate or reduce the noisy effect on the speech signas, thus better accuracy with the originay trained modes (Restore the cean speech signas or compensate for distortions) The feature part is modified whie the mode part remains unchanged Mode-based oise Compensation Techniques Adjust (changing) the recognition mode parameters (means and variances) for better matching the testing noisy conditions The mode part is modified whie the feature part remains unchanged Inherenty Robust Parameters for peech Find robust representation of speech signas ess infuenced by additive or channe noise Both of the feature and mode parts are changed peech - Berin Chen 13

14 Assumptions & Evauations Genera Assumptions for the oise The noise is uncorreated with the speech signa The noise characteristics are fied during the speech utterance or vary very sowy (the noise is said to be stationary) The estimates of the noise characteristics can be obtained during non-speech activity The noise is supposed to be additive or convoutiona Performance Evauations Inteigibiity, quaity (subjective assessment) Distortion between cean and recovered speech (objective assessment) peech recognition accuracy peech - Berin Chen 14

15 pectra ubtraction (). F. Bo, 1979 A peech Enhancement Technique Estimate the magnitude (or the power) of cean speech by epicity subtracting the noise magnitude (or the power) spectrum from the noisy magnitude (or power) spectrum Basic Assumption of pectra ubtraction The cean speech s [ m] is corrupted by additive noise n[ m] Different frequencies are uncorreated from each other s [ m] and n[ m] are statisticay independent, so that the power spectrum of the noisy speech [ m] can be epressed as: P ( ) = P ( ) + P ( ) X To eiminate the additive noise: P ( ) = P ( ) P ( ) X We can obtain an estimate of P ( ) using the average period of M frames that known to be just noise: 1 M ( ) 1 Pˆ = P ( ),i i= 0 M frames peech - Berin Chen 15

16 pectra ubtraction (cont.) Probems of pectra ubtraction s [ m] and n[ m] are not statisticay independent such that the cross term in power spectrum can not be eiminated ( ) is possiby ess than zero Pˆ Introduce musica noise when ( ) P ( ) eed a robust endpoint (speech/noise/sience) detector P X peech - Berin Chen 16

17 pectra ubtraction (cont.) Modification: oninear pectra ubtraction () Pˆ P PX ( ) P ( ), if PX ( ) P ( ) ( ) = P ( ), otherwise ( ) and P ( ): smoothed noisy and noise spectrum X or Pˆ P φ P ( ) ( ) ( ) ( ) ( ) X φ, if PX > φ + β P ( ) = β P ( ), otherwise ( ) and P ( ) X : smoothed noisy and noise ( ): a non - inear function according to R spectrum peech - Berin Chen 17

18 pectra ubtraction (cont.) pectra ubtraction can be viewed as a fitering operation Power pectrum Pˆ ( ) = PX ( ) P ( ) P = ( ) ( ) P 1 X P ( ) = P X ( ) 1 + X 1 R ( ) 1 = P X ( ) ( R P ( ) P ( ) ( ) + P ( ) = P P ( ) ( ) (supposed that : instantane ous R ) P X ( ) P ( ) + P ( ) ) The time varying suppressio n fiter is given approimat ey by : H ( ) = R ( ) 1/ 2 pectrum Domain peech - Berin Chen 18

19 Wiener Fitering A peech Enhancement Technique From the tatistica Point of View The process [ m] is the sum of the random process s[ m] and the additive noise process n[ m] [ m] = s[ m] + n[ m] Find a inear estimate ŝ [ m] in terms of the process [ m] : Or to find a inear fiter h [ m] such that the sequence ŝ[ m] = [ m] h[ m] minimizes the epected vaue of ( ŝ[ m] s[ m] ) 2 [ m ] ŝ [ m ] oisy peech [ m ] s ˆ = [ m ] h [ m ] = A inear fiter h[n] = h [] [ m ] Cean peech peech - Berin Chen 19

20 Wiener Fitering (cont.) Minimize the epectation of the squared error (MME estimate) Minimize F k F h [ m] h[ ] [ m ] [] k s[ m] [ m k] = h[ ] [ m ] [ m k] k k = k = k = R s ss s s = 0 = E s [ m] [ m k] = h[ ] [ m ] [ m k] [ m] ( s[ m k] + n[ m k] ) = h[] [ m ] [ m k] s[ m] s[ m k] + s[ m] n[ m k] = h[ ] R[ k ] k = = [] k = h[] k R[] k Rs [ n] and R [ n] ( ) = H ( ) ( ) sequences of s n = = = k= = 2 k= and Take Fourier transform Take summation for k [] [ n] s [ m] n[ m] and are statisticay independent! : are respective y the autocorre ation peech - Berin Chen 20

21 Wiener Fitering (cont.) Minimize the epectation of the squared error (MME estimate) Q ss ( ) = H ( ) ( ) H ( ) = ss ( ) ( ) = ss ss ( ) ( ) + ( ) nn, is caed the noncausa Wiener fiter (where ( ) = ( ) + ( ) ss nn ) peech - Berin Chen 21

22 Wiener Fitering (cont.) The time varying Wiener Fiter aso can be epressed in a simiar form as the spectra subtraction H ( ) = ss = 1 + ( ) ss ( ) + ( ) P P nn P -1 ( ) 1 = 1 ( ) + R( ) = P ( ) ( ) + P ( ) -1, ( R ( ) = P P ( ) ( ) : instantane ous R ) vs. Wiener Fiter: 1. Wiener fiter has stronger attenuation at ow R region 2. Wiener fiter does not invoke an absoute threshoding 10 og P P ( ) ( ) peech - Berin Chen 22

23 Wiener Fitering (cont.) Wiener Fitering can be reaized ony if we know the power spectra of both the noise and the signa A chicken-and-egg probem Approach - I : Ephraim(1992) proposed the use of an HMM where, if we know the current frame fas under, we can use its mean spectrum as ( ) or P ( ) In practice, we do not know what state each frame fas into either Weight the fiters for each state by a posterior probabiity that frame fas into each state ss peech - Berin Chen 23

24 Approach - II : Wiener Fitering (cont.) The background/noise is stationary and its power spectrum can be estimated by averaging spectra over a known background region For the non-stationary speech signa, its time-varying power spectrum can be estimated using the past Wiener fiter (of previous frame) Pˆ H ( t, ) = PX ( t, ) H( t 1, ) Pˆ ( ) ( t, ) t, = Pˆ ( t, ) + P ( ), ( t : frameinde, H( ) : Wiener fiter) ~ P ( t, ) = P ( t, ) H( t, ) X The initia estimate of the speech spectrum can be derived from spectra subtraction ometimes introduce musica noise peech - Berin Chen 24

25 peech - Berin Chen 25 Wiener Fitering (cont.) Approach - III : ow down the rapid frame-to-frame movement of the object speech power spectrum estimate by appy tempora smoothing ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ),,,, ˆ, ˆ, in, ˆ repace to, use Then, ˆ 1 1, ~, α α P t P t P t H P t P t P t H t P t P t P t P t P + = + = + = ) ) ) )

26 Wiener Fitering (cont.) Cean peech oisy peech Enhanced oise peech Using Approach III α = 0.85 Other more compicate Wiener fiters peech - Berin Chen 26

27 The Effectives of Active oise peech - Berin Chen 27

28 Cepstra Mean ormaization (CM) A peech Enhancement Technique and sometimes caed Cepstra Mean ubtraction (CM) CM is a powerfu and simpe technique designed to hande convoutiona (Time-invariant inear fitering) distortions [ n] = s[ n] h[ n] ( ) ( ) H ( ) X = X = og H = og + og H = + H ( + H ) = C CH 1 T 1 = ( C t ) + CH = C CH CX = C + Time Domain pectra Domain Cepstra Domain 1 T 1 C = C t and CX + t = 0 t = 0 T T if the training and testing speech materias were recored from two different channes () 1 = C( + H(1) ) = C + CH(1), Testing : CX ( 2) = C( + H( 2 ) ) = C CH( 2 ) Training : CX + Log Power pectra Domain CX CX ( 1) CX ( 1) = C C ( 2 ) CX ( 2 ) = C C The spectra characteristics of the microphone and room acoustics thus can be removed! Can be eiminated if the assumption of zero-mean speech contribution! peech - Berin Chen 28

29 Cepstra Mean ormaization (cont.) ome Findings Interesting, CM has been found effective even the testing and training utterances are within the same microphone and environment Variations for the distance between the mouth and the microphone for different utterances and speakers Be carefu that the duration/period used to estimate the mean of noisy speech Why? Probematic when the acoustic feature vectors are amost identica within the seected time period peech - Berin Chen 29

30 Cepstra Mean ormaization (cont.) Performance For teephone recordings, where each ca has different frequency response, the use of CM has been shown to provide as much as 30 % reative decrease in error rate When a system is trained on one microphone and tested on another, CM can provide significant robustness peech - Berin Chen 30

31 Cepstra Mean ormaization (cont.) CM has been shown to improve the robustness not ony to varying channes but aso to the noise White noise added at different Rs ystem trained with speech with the same R (matched Condition) Cepstra deta and deta-deta features are computed prior to the CM operation so that they are unaffected. peech - Berin Chen 31

32 Cepstra Mean ormaization (cont.) From the other perspective We can interpret CM as the operation of subtracting a ow-pass tempora fiter d[ n], where a the T coefficients are identica and equa to 1, which is a high-pass tempora fiter T Aeviate the effect of conventiona noise introduced in the channe Tempora (Moduation) Frequency peech - Berin Chen 32

33 Cepstra Mean ormaization (cont.) Rea-time Cepstra ormaization CM requires the compete utterance to compute the cepstra mean; thus, it cannot be used in a rea-time system, and an approimation needs to be used Based on the above perspective, we can impement other types of high-pass fiters CX t = α CX t + α ( 1 ) CX t 1, ( CX t :cepstra mean) peech - Berin Chen 33

34 RATA Tempora Fiter Hyneck Hermansky, 1991 A peech Enhancement Technique RATA (Reative pectra) Assumption The inguistic message is coded into movements of the voca tract (i.e., the change of spectra characteristics) The rate of change of non-inguistic components in speech often ies outside the typica rate of change of the voca tact shape E.g. fi or sow time-varying inear communication channes A great sensitivity of human hearing to moduation frequencies around 4Hz than to ower or higher moduation frequencies Effect RATA uppresses the spectra components that change more sowy or quicky than the typica rate of change of speech peech - Berin Chen 34

35 RATA Tempora Fiter (cont.) The IIR transfer function 1 C ( ) ~ ( ) z z H z = = 0.1z C ( z) 1 MFCC stream 3 z 2z z c [ t] c ~ [ t] H(z) H(z) 4 ew MFCC stream Frame inde H(z) An other version z H ( z) = z z RATA has a peak at about 4Hz (moduation frequency) 2 z 4 c ~ [ t] = 0.98 c ~ [ t 1] c[ t] c[ t 1] 0.1 c[ t 3] c[ t 4] moduation frequency 100 Hz peech - Berin Chen 35

36 Retraining on Corrupted peech A Mode-based oise Compensation Technique Matched-Conditions Training Take a noise waveform from the new environment, add it to a the utterance in the training database, and retrain the system If the noise characteristics are known ahead of time, this method aow as to adapt the mode to the new environment with reativey sma amount of data from the new environment, yet use a arge amount of training data peech - Berin Chen 36

37 Retraining on Corrupted peech (cont.) Muti-stye Training Create a number of artificia acoustica environments by corrupting the cean training database with noise sampes of varying eves (30dB, 20dB, etc.) and types (white, babbe, etc.), as we as varying the channes A those waveforms (copies of training database) from mutipe acoustica environments can be used in training peech - Berin Chen 37

38 Mode Adaptation A Mode-based oise Compensation Technique The standard adaptation methods for speaker adaptation can be used for adapting speech recognizers to noisy environments MAP (Maimum a Posteriori) can offer resuts simiar to those of matched conditions, but it requires a significant amount of adaptation data MLLR (Maimum Likeihood Regression) can achieve reasonabe performance with about a minute of speech for minor mismatch. For severe mismatches, MLLR aso requires a arger amount of adaptation data peech - Berin Chen 38

39 igna Decomposition Using HMMs A Mode-based oise Compensation Technique Recognize concurrent signas (speech and noise) simutaneousy Parae HMMs are used to mode the concurrent signas and the composite signa is modeed as a function of their combined outputs Three-dimensiona Viterbi earch oise HMM (especiay for non-stationary noise) Computationay Epensive for both Training and Decoding! Cean speech HMM peech - Berin Chen 39

40 Parae Mode Combination (PMC) A Mode-based oise Compensation Technique By using the cean-speech modes and a noise mode, we can approimate the distributions obtained by training a HMM with corrupted speech peech - Berin Chen 40

41 Parae Mode Combination (cont.) The steps of tandard Parae Mode Combination (Log- orma Approimation) Cepstra domain μ Σ c c Σ μ Cean speech HMM s oisy speech HMM s μˆ c Σˆ c = C 1 μ c = C Σ ( C 1 c 1 ) T c μˆ = C μˆ c Σˆ = C Σˆ C Constraint: the estimate of variance is positive Log-spectra domain μ Σ μˆ Σˆ Σ ij μ = ep μ i = μ μ i ( + Σ 2) In inear spectra domain, the distribution is ognorma T j i ii [ ep( Σ ) 1] Because speech and noise are independent and additive in the inear spectra domain ij 1 Σˆ ii ( ˆ μi ) og( + 1) ˆ μi = og 2 ˆ μi 2 Σˆ ij = og ( ) Σˆ + 1 ˆ μ ˆ μ i ij j Σ Linear spectra domain μ Σ μ ˆ = gμ + μ~ ˆ 2 = g μˆ Σˆ Σ ~ + Σ oise HMM s μ~ ~ Σ Log-norma approimation (Assume the new distribution is ognorma) peech - Berin Chen 41

42 Parae Mode Combination (cont.) Modification-I: Perform the mode combination in the Log- pectra Domain (the simpest approimation) Log-Add Approimation: (without compensation of variances) ( ( ep μ ) ep( μ ) ˆ μ = og + ~ The variances are assumed to be sma A simpified version of Log-orma approimation Reduction in computationa oad Modification-II: Perform the mode combination in the Linear pectra Domain (Data-Driven PMC, DPMC, or Iterative PMC) Use the speech modes to generate noisy sampes (corrupted speech observations) and then compute a maimum ikeihood of these noisy sampes This method is ess computationay epensive than standard PMC with comparabe performance peech - Berin Chen 42

43 Parae Mode Combination (cont.) Modification-II: Perform the mode combination in the Linear pectra Domain (Data-Driven PMC, DPMC) Cean peech HMM oise HMM oisy peech HMM Cepstra domain Appy Monte Caro simuation to draw random cepstra vectors (for eampe, at east 100 for each distribution) Generating sampes Linear spectra domain Domain transform peech - Berin Chen 43

44 Parae Mode Combination (cont.) Data-Driven PMC peech - Berin Chen 44

45 Vector Tayor eries (VT) P. J. Moreno,1995 A Mode-based oise Compensation Technique VT Approach imiar to PMC, the noisy-speech-ike modes is generated by combining of cean speech HMM s and the noise HMM Unike PMC, the VT approach combines the parameters of cean speech HMM s and the noise HMM ineary in the ogspectra domain P = P P + P Power spectrum X X ( ) ( ) H ( ) ( ) = og( P ( ) PH ( ) + P ( ) ) P ( ) ( ) ( ) = og P PH 1 + P ( ) PH ( ) ogp ( ) ogp ( ) ogph ( ) = ogp ( ) + ogp ( ) + og1+ e = = + H + H + f H Log Power spectrum ( ) ( ) H + e on-inear function ( ) ( ) ( ) H, H,, wheref, H, = og1+ e + og1 Is a vector function peech - Berin Chen 45

46 peech - Berin Chen 46 Vector Tayor eries (cont.) The Tayor series provides a poynomia representation of a function in terms of the function and its derivatives at a point Appication often arises when noninear functions are empoyed and we desire to obtain a inear approimation The function is represented as an offset and a inear term ( ) ( ) ( )( ) ( )( ) ( ) ( )( ) ( ) n n n o f n f f f f R R f ! : =

47 Vector Tayor eries (cont.) Appy Tayor eries Approimation f df ( ) ( ) ( 0, H 0, 0 ),,,, ( H f 0 H d ) df ( 0, H 0, 0 ) ( ) df ( 0, H 0, 0 ) + + ( H H ) +... dh VT-0: use ony the 0th-order terms of Tayor eries VT-1: use ony the 0th- and 1th-order terms of Tayor eries f 0, H 0, 0 is the vector function evauated at a particuar vector point If VT-0 is used E u Σ [ X ] = E[ + H + f (,H, )] = u [ ( )] s + uh + E f,h, u + u + E[ f ( u,u,u )] s h s h n u + u + f ( u,u,u ) ( X s Σ s h + Σ ( ) h (if s h n and H To get the cean speech statistics is aso Gaussian) u Σ are independent) 0 u s Σ + g + s d If the channe fiter is inear - time invariant, 0-th order VT we can regard it as a bias (constant), g, in the og power spectrum domain f ( u,g,u ) ( X is aso Gaussian) s n 0 peech - Berin Chen 47

48 Vector Tayor eries (cont.) peech - Berin Chen 48

49 Retraining on Compensated Features A Mode-based oise Compensation Technique that aso Uses enhanced Features (processed by, CM, etc.) Combine speech enhancement and mode compensation peech - Berin Chen 49

50 Principa Component Anaysis Principa Component Anaysis (PCA) : Widey appied for the data anaysis and dimensionaity reduction in order to derive the most epressive feature Criterion: for a zero mean r.v. R, find k (k ) orthonorma vectors {e 1, e 2,, e k } so that (1) var(e 1 T )=ma 1 (2) var(e it )=ma i subject to e i e i-1 e 1 1 i k {e 1, e 2,, e k } are in fact the eigenvectors of the covariance matri (Σ ) for corresponding to the argest k eigenvaues Fina r.v y R k : the inear transform (projection) of the origina r.v., y=a T A=[e 1 e 2 e k ] data Principa ais peech - Berin Chen 50

51 Principa Component Anaysis (cont.) peech - Berin Chen 51

52 Principa Component Anaysis (cont.) Properties of PCA The components of y are mutuay uncorreated E{y i y j }=E{(e it ) (e jt ) T }=E{(e it ) ( T e j )}=e it E{ T } e j =e it Σ e j = λ j e it e j =0, if i j the covariance of y is diagona The error power (mean-squared error) between the origina vector and the projected is minimum =(e 1T )e 1 + (e 2T )e 2 + +(e kt )e k + +(e T )e =(e 1T )e 1 + (e 2T )e 2 + +(e kt )e k (ote : R ) error r.v : - = (e k+1t )e k+1 + (e k+2t )e k+2 + +(e T )e E((- ) T (- ))=E((e k+1t ) e k+1t e k+1 (e k+1t ))+ +E((e T ) e T e (e T )) =var(e k+1t )+ var(e k+2t )+ var(e T ) = λ k+1 + λ k+2 + +λ minimum peech - Berin Chen 52

53 PCA Appied in Inherenty Robust Features Appication 1 : the inear transform of the origina features (in the spatia domain) Origina feature stream t Frame inde z t = A T t A T A T A T A T The coumns of A are the first k eigenvectors of Σ transformed feature stream z t Frame inde peech - Berin Chen 53

54 peech - Berin Chen 54 PCA Appied in Inherenty Robust Features (cont.) Appication 2 : PCA-derived tempora fiter (in the tempora domain) The effect of the tempora fiter is equivaent to the weighted sum of sequence of a specific MFCC coefficient with ength L sid aong the frame inde quefrency Frame inde B 1 (z) B 2 (z) B n (z) L z k (1) z k (2) z k (3) Origina feature stream t The impuse response of B k (z) is one of the eigenvectors of the covariance for z k ( ) ( ) ( ) ( ) () ( ) () ( ) ( ) n m y m y m y m y K k K n k n n n K k K k K k K k ), ( ), (,2) (,1) ( ), ( ), (,2) (,1) ( ) (3, ) (3, (3,2) (3,1) ) (2, ) (2, (2,2) (2,1) ) (1, ) (1, (1,2) (1,1) 2 1 L L M M M M L M M L M M M M M M z k (n)=[ y k (n) y k (n+1) y k (n+2) y k (n+l-1)] T ( ) + = + = L n k n L μ zk z ( ) ( ) ( ) ( ) + = + = Σ L n T k k k k k n n L z z z z z μ μ ( ) ( ) ( ) n k n k T k z e 1, ˆ = The eement in the new feature vector From Dr. Jei-wei Hung

55 PCA Appied in Inherenty Robust Features (cont.) The frequency responses of the 15 PCA-derived tempora fiters From Dr. Jei-wei Hung peech - Berin Chen 55

56 PCA Appied in Inherenty Robust Features (cont.) Appication 2 : PCA-derived tempora fiter Mismatched condition Fiter ength L=10 Matched condition From Dr. Jei-wei Hung peech - Berin Chen 56

57 PCA Appied in Inherenty Robust Features (cont.) Appication 3 : PCA-derived fiter bank Power spectrum obtained by DFT h 1 h 2 h 3 h k is one of the eigenvectors of the covariance for k From Dr. Jei-wei Hung peech - Berin Chen 57

58 PCA Appied in Inherenty Robust Features (cont.) Appication 3 : PCA-derived fiter bank From Dr. Jei-wei Hung peech - Berin Chen 58

59 Linear Discriminative Anaysis Linear Discriminative Anaysis (LDA) Widey appied for the pattern cassification In order to derive the most discriminative feature Criterion : assume w j, μ j and Σ j are the weight, mean and covariance of cass j, j=1. Two matrices are defined as: Between - cass covariance : Within - cass covariance : Find W=[w 1 w 2 w k ] such that T W bw Wˆ = arg ma W T W W The coumns w j of W are the eigenvectors of w -1 B having the argest eigenvaues w w b = = j = 1 j = 1 w w j j Σ ( μ μ)( μ μ) j j j T peech - Berin Chen 59

60 Linear Discriminative Anaysis (cont.) The frequency responses of the 15 LDA-derived tempora fiters From Dr. Jei-wei Hung peech - Berin Chen 60

61 Minimum Cassification Error Minimum Cassification Error (MCE): Genera Objective : find an optima feature presentation or an optima recognition mode to minimize the epected error of cassification The recognizer is often operated under the foowing decision rue : C(X)=C i if g i (X,Λ)=ma j g j (X,Λ) Λ={λ (i) } i=1 M (M modes, casses), X : observations, g i (X,Λ): cass conditioned ikeihood function, for eampe, g i (X,Λ)=P(X λ (i) ) Traditiona Training Criterion : find λ (i) such that P(X λ (i) ) is maimum (Maimum Likeihood) if X C i This criterion does not aways ead to minimum cassification error, since it doesn't consider the mutua reationship between different casses For eampe, it s possibe that P(X λ (i) ) is maimum but X C i peech - Berin Chen 61

62 Minimum Cassification Error (cont.) Threshod τ P ( LR ( k ) KW k C k ) P LR ( k ) k ( ) KW k C k Type I error Type II error LR ( k ) Eampe showing histograms of the ikeihood ratio when keyword KW and KW k C k k C k LR ( k ) Type I error: Fase Rejection Type II error: Fase Aarm/Fase Acceptance peech - Berin Chen 62

63 Minimum Cassification Error (cont.) Minimum Cassification Error (MCE) (Cont.): One form of the cass miscassification measure : d d d i i i () i ( X ) g X, λ 1 M 1 ( ) ( ( () i + og ep g X, λ ) α ) = ( X ) 0 impies a miscassification (error = 1) ( X ) < 0 impies a correct cassification (error = 0) A continuous oss function is defined as foows : i ( X, Λ) = ( d ( X )) i X C where the sigmoid function ( d ) j i Cassifier performance measure : L i 1 = 1 + ep ( γ d + θ ) ( Λ) = E [ L( X, Λ) ] = ( X, Λ) δ( X C ) X X M i= 1 i i 1 α X C i peech - Berin Chen 63

64 Minimum Cassification Error (cont.) Using MCE in mode training : Find Λ such that Λ ˆ = arg min L Λ ( Λ) = arg min E [ L( X, Λ) ] Λ the above objective function in genera cannot be minimized directy but the oca minimum can be achieved using gradient decent agorithm ( ) L Λ wt+ = wt ε, w : an arbitrary parameter of w Using MCE in robust feature representation ˆ ( f ) f = arg mine L( f ( X ), Λ ) f ote: X 1 Λ f X [ ] : a transform of the originafeature X whie feature presentation is changed, the mode is aso changed accordingy peech - Berin Chen 64