A Multichannel Acoustic Echo Canceler Double-Talk Detector Based on a Normalized Cross-Correlation Matrix

Transcription

1 Acoustic Echo and Noise Control A Multichannel Acoustic Echo Canceler Double-Talk Detector Based on a Normalized Cross-Correlation Matrix JACOB BENESTY Bell Laboratories, Lucent Technologies, 700 Mountain Avenue, Murray Hill, NJ, USA jbenesty@bell-labs.com TOMAS GÄNSLER Agere Systems gaensler@agere.com Abstract. Multichannel acoustic echo cancellation is basically composed of two parts. One part is a multichannel system identification problem which is nontrivial to solve. The other part, which is also nontrivial, is the so-called double-talk detection problem. Near-end speech detection is based on a test statistic. Recently, a new double-talk test statistic based on the normalized cross-correlation vector was proposed for the single-channel case. Obviously, in the multichannel case, there are several solutions, but what should be the optimal one is not yet known. Then, a fundamental question arises: how do we deal optimally with multiple channels? In this paper, we generalize the idea of normalized cross-correlation vector (single-channel) to the matrix case (multichannel), derive a frequency-domain version, and show how to combine both the multichannel frequency-domain adaptive filter and the multichannel double-talk detector. INTRODUCTION Multichannel acoustic echo cancellation is basically composed of two parts. One part is a multichannel system identification problem which is nontrivial to solve; see, 2 for more details. The other part, which is also nontrivial, is the so-called double-talk detection problem 3. When the multiple echo paths are identified by a multichannel adaptive filter, a function should be included to freeze the adaptation whenever a near-end signal is detected, and thereby avoid the divergence of the adaptive algorithm. This is the role of a double-talk detector (DTD) 4. Consequently, a suitable DTD decision variable has to be found. An optimum decision variable ξ for double-talk detection should behave as follows: (i) if double-talk is not present, ξ T ; (ii) if double-talk is present, ξ<t. The threshold T must be a constant (in this application), independent of the data. Moreover, ξ must be insensitive to echo path variations when there is no double-talk. A A shorter version of this paper was presented at the International Workshop on Acoustic Echo and Noise Control, IWAENC 200. Optimum in the sense that for a given probability of false alarm (of double-talk), the probability of miss is minimized. decision variable that exhibits this behavior was proposed in 5 for the single-channel case, using a new normalized cross-correlation vector. In many acoustic signal processing problems such as voice activity detection, time delay estimation, source localization, double-talk detection, etc, multiple microphone signals are available. However, it is not obvious as to how all of this information should be taken into account for best performance. In the case of (multichannel) double-talk detection, there are at least three options. The first one is to consider only a single microphone and build a test statistic based on the normalized cross-correlation vector between the loudspeaker signals and the chosen microphone signal. This is the approach that was taken in 6, 7. However, this approach may increase the probability of miss since the microphones pick the near-end speech up with different levels and different SNRs, depending on the position of the talker and microphones. Furthermore, not all of this information is taken into account in the decision variable. The second option is to use a test statistic for each microphone, so that the decision made on one microphone is independent of the others. But a large amount of tests may not be easy to handle and we may have some contradictory results on the Vol. 3, No. 2, March-April 2002

2 J. Benesty, T. Gänsler presence of a near-end talker. In this paper, a third option is proposed which consists of using one test statistic that combines all the (spatially sampled) information of the microphone signals. This approach leads to a nice generalization of the normalized cross-correlation vector to the normalized cross-correlation matrix. By its nature, this method considers all the microphone signals equally. Other possibilities based on this approach can be derived but what will be the optimal choice is not yet known. Thus, multichannel double-talk detection is a very new topic of research as well as a challenging and an interesting problem. The organization of this paper is as follows. In section 2, a brief outline is given on multichannel acoustic echo cancellation and the multichannel Wiener-Hopf equation. In section 3, the normalized cross-correlation matrix is defined and it is shown how to apply it to double-talk detection. Section 4 gives a frequency-domain approach. In section 5, it is shown how to combine, in the frequency domain, echo cancellation and double-talk detection with the new approach. Finally, conclusions are given in section 6. 2 MULTICHANNEL ACOUSTIC ECHO CAN- CELLATION First, we assume that we have Q loudspeakers and microphones. We also assume that the system (room) is linear and time-invariant. Acoustic echo cancellation consists of identifying Q echo paths at each microphone so that in total, Q echo paths need to be estimated. We have output (microphone) signals (see figure for notation): y p (n) hqp T x q(n) + v p (n), () q p, 2,...,, superscript T denotes transpose of a vector or a matrix, h qp T h qp,0 h qp, h qp,l is the echo path of length L between loudspeaker q and microphone p, x q (n) xq (n) x q (n ) x q (n L + ) T, q, 2,..., Q, is the qth reference (loudspeaker) signal (also called the far-end speech), and v p is the near-end speech added at microphone p, assumed to be uncorrelated with the far-end speech. We define the error signal at time n for microphone p as e p (n) y p (n) ŷ p (n) y p (n) ĥ T qp x q(n), (2) q ĥ qp ĥ qp,0 ĥ qp, ĥ qp,l T are the model filters. It is more convenient to define an error signal vector for all the microphones: x (n) x 2 (n) x q (n) e p (n) e(n) y(n) ŷ(n) y(n) Ĥ T x(n), (3) y(n) H T x(n) + v(n), h h 2 h h 2 h 22 h 2 H , h Q h Q2 h Q v(n) v (n) v 2 (n) v (n) T, e(n) e (n) e 2 (n) e (n) T, y(n) y (n) y 2 (n) y (n) T, ŷ(n) ŷ (n) ŷ 2 (n) ŷ (n) T, ĥ ĥ 2 ĥ ĥ 2 ĥ 22 ĥ 2 Ĥ......, ĥ Q ĥ Q2 ĥ Q T x(n) x T (n) xt 2 (n) xt Q (n). Ad. FIR NL NL NL + ŷ p (n) y p (n) h p h 2p h qp Figure : Multichannel acoustic echo cancellation. (Ad. and NL stand respectively for adaptive and nonlinearity.) Having written the error signal, we now define the cost 96 ETT

3 A Multichannel Acoustic Echo Canceler Double-Talk Detector Based on a Normalized Cross-Correlation Matrix function J E{e T (n)e(n)} (4) E{e 2 p (n)} p J p. p E{ } denotes the statistical expectation operator. The minimization of (4) leads to the multichannel Wiener-Hopf equation: R xx Ĥ R xy, (5) R xx E{x(n)x T (n)} (6) is the covariance matrix of size (QL QL) of the reference signals x, and R xy E{x(n)y T (n)} (7) is the cross-correlation matrix of size (QL ) between x and y. It can easily be seen that the multichannel Wiener-Hopf equation (5) can be decomposed in independent Wiener- Hopf equations, each one corresponding to a microphone signal: R xx ĥ p r xyp, p, 2,...,, (8) ĥ p (resp. r xyp )isthepth column of matrix Ĥ (resp. R xy ). This result implies that minimizing J or minimizing each J p independently gives the same results. This makes sense from an identification point of view, since the identification of the impulse responses for one microphone is completely independent of the others. However, as far as double-talk is concerned, it is preferable to have a global and unique test statistic that takes into account the information of all the microphone signals, since the near-end speech is picked-up by all the microphones with different levels. Choosing one microphone signal and using a single test statistic based on this signal is not enough. On the other hand, using independent decision variables will be much harder to handle (computational complexity, inconsistency among the different tests,...). Thus, the approach taken here is to develop a unique test statistic by looking at the covariance matrix R yy of the microphone signals y. In many situations, the signals x q are generated from a unique source s, so that: x q (n) gq T s(n), q, 2,..., Q, (9) g q g q,0 g q, g q,l T is the impulse response between the source and microphone q in the transmission room in the case of a teleconferencing system 2. Therefore the signals x q are linearly related and we have the following Q(Q )/2 relations: xq T (n)g i xi T (n)g q, (0) i, q, 2,..., Q, i q. Now, consider the following vector: Q u q2 ζ qgq T ζ 2 g T ζ Q g T T, ζ q are arbitrary factors. We can verify using (0) that R xx u 0 QL,soR xx is not invertible. Vector u represents the nullspace of matrix R xx. The dimension of this nullspace depends of the number of channels and is equal to (Q 2)L + (for Q 2). So the problem becomes worse as Q increases. Thus, there is no unique solution to the problem and an adaptive algorithm will drift to any one of many possible solutions, which can be very different from the true desired solution ĥ qp h qp. These nonunique solutions are dependent on the impulse responses in the transmission room: ĥ p h p + β ζ q g q, () q2 ĥ qp h qp βζ q g, q 2,..., Q, (2) β is an arbitrary factor. This, of course, is intolerable because g q can change instantaneously, for example, as one person stops talking and another starts, 2. In the following, we suppose that the covariance matrix R xx is invertible. To almost guarantee that, we may add (for Q > ) a non-linear (NL) transformation (or add perceptually acceptable uncorrelated noise) to each input signal x q in order to reduce the coherence of the signals two-by-two 2. Other existing methods to reduce the coherence can be used 8, 9, 0,, 2, 3. 3 A NORMALIZED CROSS-CORRELATION MATRIX FOR MULTICHANNEL DOUBLE- TALK DETECTION For the single-channel case, it has been shown 5 that the so-called normalized cross-correlation vector is well suitable for DTD. In this section, we derive a decision variable based on a normalized cross-correlation matrix for the multichannel situation. Suppose that v 0 (no near-end speech). In this case: R yy E{y(n)y T (n)} H T R xx H. (3) Vol. 3, No. 2, March-April 2002

4 J. Benesty, T. Gänsler Since y(n) H T x(n),wehave: and (3) may be re-written as R xy E{x(n)y T (n)} R xx H (4) R yy R T xy R xx R xy. (5) Now, in general for v 0, R yy R T xy R xx R xy + R vv, (6) R vv E{v(n)v T (n)} (7) is the covariance matrix of the near-end speech v. Then from (5) and (6), the following decision statistic is proposed, ξ C xy E tr(c T xy C xy) tr(r T xy R xx R xy R yy ), (8) C xy R /2 xx R xy R /2 yy (9) is the normalized cross-correlation matrix between the two vectors x and y. Substituting (4) and (6) into (8), ξ trh T R xx H(H T R xx H + R vv ). (20) It is easily deduced from (20) that for v 0, ξ and for v 0, ξ<. So that, in the noiseless case, we can set the threshold T. For the single-channel case (Q ), (9) becomes the normalized cross-correlation vector between vector x and scalar y : c x y (σ 2 y R x x ) /2 r x y, (2) with σy 2 E{y 2 (n)}. The test statistic is now: ξ c x y E tr(c T x y c x y ) c x y 2 r T x y (σy 2 R x x ) r x y. (22) It can easily be seen that (22) can also be written as: h T ξ R x x h, (23) h T R x x h + σ 2 v so that for v 0, ξ and for v 0, ξ<. Thus, the normalized cross-correlation matrix is a natural and elegant extension of the normalized crosscorrelation vector 5 to the multichannel case. 4 A FREQUENCY-DOMAIN AROACH In this section, a frequency-domain DTD is derived that will be more useful in practice than its time-domain counterpart, because it is much more efficient from a computational complexity point of view. We define the block error signal (of length L) for microphone p as: e p (m) y p (m) ŷ p (m) y p (m) X q (m)ĥ qp, (24) p, 2,...,, q m is the block time index, X q (m) is a Toeplitz matrix, and e p (m) e p (ml) e p (ml + L ) T, y p (m) y p (ml) y p (ml + L ) T, X q (m) x q (ml) x q (ml + L ) T. It is well known that a Toeplitz matrix X q (m) can be transformed, by doubling its size, to a circulant matrix X C q (m) q (m) X q (m) X q (m) X q (m), X q (m) is also a Toeplitz matrix obtained by shifting rows of X q (m); the diagonal of C q (m) is arbitrary, but it is natural and customary to set it equal to the first sample in the previous block. It is also well known that a circulant matrix is easily decomposed as follows: C q (m) F D q (m)f, F is the Fourier matrix of size (2L 2L) and D q is a diagonal matrix whose elements are the discrete Fourier transform of the first column of C q (m). Before going to the frequency domain, we need to rewrite (24) in a different way: 0L e p (m) 0L y p (m) W q C q (m) ĥqp 0 L, (25) 98 ETT

5 A Multichannel Acoustic Echo Canceler Double-Talk Detector Based on a Normalized Cross-Correlation Matrix W 0L L 0 L L. 0 L L I L L Now, multiplying both sides of (25) by F, we obtain the frequency-domain error signal: Q e p (m) y p (m) G D q (m)ĥ qp, (26) q 0L e p (m) F, e p (m) 0L y p (m) F, y p (m) G FW F, ĥ qp F ĥqp 0 L. Minimizing the frequency-domain criterion J fd E{e H p (m)e p (m)} (27) p leads to the multichannel Wiener-Hopf equation in the frequency domain: S xx Ĥ S xy, (28) H denotes conjugate transpose and S xx E{D H (m)g D(m)}, D(m) D (m) D 2 (m) D Q (m), ĥ ĥ 2 ĥ ĥ 2 ĥ 22 ĥ 2 Ĥ......, ĥ Q ĥ Q2 ĥ Q S xy E{D H (m)y(m)}, Y(m) y (m) y 2 (m) y (m). Following the same philosophy as in section 3, we define the pseudo-coherence matrix: Ɣ xy S /2 xx S xy R /2 yy with R yy E{YH (m)y(m)}. For Q, Ɣ xy becomes: (29) γ x y S /2 x x s x y E /2 {y H (m)y (m)}, (30) which is also called the pseudo-coherence vector 6, 7. The difference between the true coherence and the pseudocoherence is a different normalization with respect to the signal y. We define the multichannel frequency-domain decision variable as: ξ fd Ɣ xy E tr(ɣ H xy Ɣ xy) tr(sxy H S xx S xy R yy ), (3) and it can be checked that for v 0, ξ fd and for v 0,0 ξ fd <. 5 COMBINATION OF MULTICHANNEL IDENTIFICATION AND DOUBLE-TALK DE- TECTION IN THE FREQUENCY DOMAIN In this part, it is shown how to combine both the adaptive identification of the echo paths and the near-end speech detection. Since the DTD will also be adaptive, two different multichannel model filters (one foreground and one background) will be used, like the two-path model 4. The background filter will be updated permanently to estimate the test statistic. Each time double-talk is detected, the adaptation of the foreground filter (for identification) will be halted. A multichannel frequency-domain adaptive algorithm can be easily derived from the Wiener-Hopf equation. In this section, only the algorithm is given 5, which is: Ŝ xx (m) λ f Ŝ xx (m ) + ( λ f )D H (m)d(m), (32) e f,p (m) y p (m) G D q (m)ĥ f,qp (m ) q y p (m) G D(m)ĥ f,p (m ), (33) ĥ f,p (m) (34) ĥ f,p (m ) + µg 2 Ŝ xx (m)dh (m)e f,p (m), p, 2,...,, the subscript f stands for foreground, λ f, 0 < λ f <, is an exponential forgetting factor, µ µ ( λ f ),0< µ 2, is the adaptation step size, and G 2 FW 2 F, IL L 0 W 2 L L. 0 L L 0 L L The decision variable should be estimated as follows: ˆξ 2 fd (m) trŝh xy (m)ŝ xx (m)ŝ xy (m) ˆR yy (m) trŝh xy (m)ĥ b (m) ˆR yy (m), (35) Vol. 3, No. 2, March-April 2002

6 J. Benesty, T. Gänsler Ŝ xy (m) λ b Ŝ xy (m ) + ( λ b )D H (m)y(m) (36) e b,p (m) y p (m) G D(m)ĥ b,p (m ) (37) ĥ b,p (m) (38) ĥ b,p (m ) + ( λ b )G 2 Ŝ xx (m)dh (m)e b,p (m) p, 2,...,, ˆR yy (m) λ b ˆR yy (m ) + ( λ b)y H (m)y(m), (39) subscript b stands for background, and λ b,0<λ b <, is an exponential forgetting factor. We must choose λ b <λ f (for a faster tracking of the background filter used in the DTD) in order that the DTD alerts the foreground filter before it diverges. Table summarizes the combination of both the multichannel acoustic echo canceler and the multichannel DTD in the frequency domain. Table : A frequency-domain multichannel acoustic echo canceler combined with a multichannel double-talk detector., number of microphones 0 <µ 2, normalized adaptation step size λ f,λ b, exponential windows T, threshold G FW F 0L L 0 W L L 0 L L I L L G 2 FW 2 F IL L 0 W 2 L L 0 L L 0 L L Spectral and correlation estimation Ŝ xx (m) λ f Ŝ xx (m ) + ( λ f )D H (m)d(m) Ŝ xy (m) λ b Ŝ xy (m ) + ( λ b )D H (m)y(m) ˆR yy (m) λ b ˆR yy (m ) + ( λ b)y H (m)y(m) 6 CONCLUSIONS In the literature, a lot of attention has been given to the identification part of multichannel acoustic echo cancellation but little has been done for near-end speech detection. Multichannel double-talk detection is not trivial and has to be investigated more deeply. In this paper, some possibilities to handle this problem have been proposed, centered on the use of the normalized cross-correlation matrix as the test statistic. This is a natural extension of the normalized cross-correlation vector used in the single-channel case which has been previously discussed 5. The advantage of the proposed approach is that all the microphone signals are taken into account in one decision variable. We also applied this method in the frequency domain and obtained a pseudo-coherence matrix. Finally, an efficient way to combine acoustic echo cancellation and double-talk detection in the frequency domain was proposed. Obviously, for future research there are still lots of things to study. One important aspect of this work that has to be carefully and judiciously investigated is an objective way to evaluate the different multichannel DTD algorithms and compare their performance. Multichannel double-talk detector (background filter) e b,p (m) y p (m) G D(m)ĥ b,p (m ) ĥ b,p (m) ĥ b,p (m ) + µ ( λ b )G 2 Ŝ xx (m)dh (m)e b,p (m) p, 2,..., ˆξ fd 2 (m) trŝh xy (m)ĥ b (m) ˆR yy (m) ˆξ fd (m) < T, double-talk, µ 0 ˆξ fd (m) T, no double-talk, µ µ ( λ f ) Multichannel acoustic echo canceler (foreground filter) e f,p (m) y p (m) G D(m)ĥ f,p (m ) ĥ f,p (m) ĥ f,p (m ) + µg 2 Ŝ xx (m)dh (m)e f,p (m) p, 2,..., ACKNOWLEDGMENT The authors would like to thank Dennis R. Morgan for his helpful suggestions. Manuscript received on December 9, 200. REFERENCES M. M. Sondhi, D. R. Morgan, and J. L. Hall. Stereophonic acoustic echo cancellation An overview of the fundamental problem. IEEE Signal rocessing Lett., Vol. 2, pages 48-5, Aug ETT

7 A Multichannel Acoustic Echo Canceler Double-Talk Detector Based on a Normalized Cross-Correlation Matrix 2 J. Benesty, D. R. Morgan, and M. M. Sondhi. A better understanding and an improved solution to the specific problems of stereophonic acoustic echo cancellation. IEEE Trans. Speech Audio rocessing, Vol. 6, pages 56-65, Mar J. H. Cho, D. R. Morgan, and J. Benesty. An objective technique for evaluating doubletalk detectors in acoustic echo cancelers. IEEE Trans. Speech Audio rocessing, Vol. 7, pages , Nov C. Breining,. Dreiseitel, E. Hänsler, A. Mader, B. Nitsch, H. uder, T. Schertler, G. Schmidt, and J. Tilp. Acoustic echo control, an application of very-high-order adaptive filters. IEEE Signal rocessing Mag., Vol. 6, pages 42-69, July J. Benesty, D. R. Morgan, and J. H. Cho. A new class of doubletalk detectors based on cross-correlation. IEEE Trans. Speech Audio rocessing, Vol. 8, pages 68-72, Mar J. Benesty, T. Gänsler, D. R. Morgan, M. M. Sondhi, and S. L. Gay. Advances in Network and Acoustic Echo Cancellation. Springer-Verlag, Berlin, Germany, T. Gänsler and J. Benesty. A frequency-domain double-talk detector based on a normalized cross-correlation vector. Signal rocessing, Vol. 8, pages , Aug M. Ali. Stereophonic echo cancellation system using timevarying all-pass filtering for signal decorrelation. In roc. IEEE ICASS, pages , A. Gilloire and V. Turbin. Using auditory properties to improve the behavior of stereophonic acoustic echo cancellers. In roc. IEEE ICASS, pages , T. Gänsler and. Eneroth. Influence of audio coding on stereophonic acoustic echo cancellation. In roc. IEEE ICASS, pages , 998. Y. Joncour and A. Sugiyama. A stereo echo canceller with pre-processing for correct echo path identification. In roc. IEEE ICASS, pages , S. Shimauchi and S. Makino. Stereo projection echo canceller with true echo path estimation. In roc. IEEE ICASS, pages , S. Shimauchi, Y. Haneda, S. Makino, and Y. Kaneda. New configuration for a stereo echo canceller with nonlinear preprocessing. In roc. IEEE ICASS, pages , K. Ochiai, T. Araseki, and T. Ogihara. Echo canceler with two echo path models. IEEE Trans. Commun., Vol. 25, pages , June J. Benesty and D. R. Morgan. Multi-channel frequencydomain adaptive filtering. In Acoustic Signal rocessing for Telecommunication, S. L. Gay and J. Benesty, Eds., Kluwer Academic ublishers, ch. 7, Vol. 3, No. 2, March-April 2002

8 J. Benesty, T. Gänsler 02 ETT