Adaptive filtering algorithms for acoustic echo and noise cancellation. Geert Rombouts

Transcription

1 Adaptive filtering algorithms for acoustic echo and noise cancellation Geert Rombouts 25th april 23

2 KATHOLIEKE UNIVERSITEIT LEUVEN FACULTEIT TOEGEPASTE WETENSCHAPPEN DEPARTEMENT ELEKTROTECHNIEK Kasteelpark Arenberg 1, 31 Leuven (Heverlee) Adaptive filtering algorithms for acoustic echo and noise cancellation Proefschrift voorgedragen tot het behalen van het doctoraat in de toegepaste wetenschappen door Geert ROMBOUTS Jury : Prof. dr. ir. E. Aernoudt, voorzitter Prof. dr. ir. M. Moonen, promotor Prof. dr. ir. D. Van Compernolle Prof. dr. ir. B. De Moor Prof. dr. ir. S. Van Huffel Prof. dr. ir. P. Sommen (TU Eindhoven) Prof. dr. ir. I. K. Proudler (King s College, UK) UDC 681.3*I12:534 April 23

3 Copyright Katholieke Universiteit Leuven - Faculteit Toegepaste Wetenschappen Arenbergkasteel, B-31 Heverlee Alle rechten voorbehouden. Niets uit deze uitgave mag vermenigvuldigd en/of openbaar gemaakt worden door middel van druk, fotocopie, microfilm, elektronisch of op welke andere wijze ook zonder voorafgaande schriftelijke toestemming van de uitgever. All rights reserved. No part of this publication may be reproduced in any form by print, photoprint, microfilm or any other means without written permission from the publisher. D/23/7515/13 ISBN Voor mijn grootmoeder, Maria Jonckers

4 4 Abstract In this thesis, we develop a number of algorithms for acoustic echo and noise cancellation. We derive a fast exact implementation for the affine projection algorithm (APA), and we also show that when using strong regularization the existing (approximating) fast techniques exhibit problems. We develop a number of algorithms for noise cancellation based on optimal filtering techniques for multi microphone systems. By using QR decomposition based techniques, a complexity reduction of a factor 5 to 1 is achieved compared to existing implementations. Finally, we show that instead of using a cascade of a noise cancellation system and an echo cancellation system, it is better to solve the combined problem as a global optimization problem. The aforementioned noise reduction techniques can be used to solve this optimization problem.

5 5 List of symbols B(k) : Right hand side in QRD RLS based noise reduction equation d(k) : Desired signal of an adaptive filter at time k ( d1 (k) d 2 (k) d 3 (k) ) : Desired signals for multiple right hand sides d(k) : Vector with recent desired signal samples δ : Regularisation parameter (diagonal loading) e(k) : Error signal of an adaptive filter ε{} : Expected value operator f(k) : Loudspeaker reference signal G : Givens rotation g i : Far end room paths h i : Near end room paths λ : Forgetting factor (weighting factor) λ n : Forgetting factor during noise only periods λ s : Forgetting factor during speech+noise periods M : Number of channels µ : Stepsize n(k) : Noise signal N : Number of filter taps per channel N aec : Number of taps in the AEC part in AENC Q(k) : Orthogonal matrix Q in a QR decomposition R(k) : Upper triangular matrix R in a QR decomposition Σ : Diagonal matrix in an SVD decomposition σ i : Singular value u(k) : Input vector with microphone signals and echo reference v(k) : Acoustical disturbance signal

6 6 v(k) : Vector with recent disturbance samples V (k) : Toeplitz matrix with disturbance signal w(k) : Filter coefficient vector. A subscript may specify the algorithm used. W (k) : Matrix of which the columns are filter vectors x(k) : Input signal x(k) : Input vector X(k) : Toeplitz matrix with input signal Ξ(k) : Input correlation matrix y(k) : Output of adaptive filter : Convolution symbol

7 Contents 1 Speech signal enhancement Overview Problem statement Nature of acoustical disturbances AEC, reference based noise reduction ANC, reference less noise reduction Combined AEC and ANC Applications The market Contributions Outline Adaptive filtering algorithms Introduction Normalized Least Mean Squares algorithm Recursive Least Squares algorithms Standard recursive least squares QRD updating QRD based RLS algorithm (QRD RLS)

8 8 CONTENTS QRD based least squares lattice (QRD LSL) RLS versus LMS Affine Projection based algorithms The affine projection algorithm APA versus LMS The Fast Affine Projection algorithm (FAP) Geometrical interpretation Conclusion APA regularization and Sparse APA for AEC APA regularization Diagonal loading Exponential weighting APA with sparse equations FAP and the influence of regularization Experimental results Regularization in multichannel AEC Conclusion Block Exact APA (BEAPA) for AEC Block Exact Fast Affine Projection (BEFAP) Block Exact APA (BEAPA) Principle Complexity reduction Algorithm specification Sparse Block Exact APA Derivation

9 CONTENTS Complexity reduction Algorithm specification Conclusion QRD RLS based ANC Introduction Unconstrained optimal filtering based ANC QRD based algorithm Speech+noise mode Noise only mode Residual extraction Initialization Algorithm description Trading off noise reduction vs. signal distortion Regularization Speech+noise mode Noise only mode Complexity Simulation results Conclusion Fast QRD LSL based ANC Preliminaries Modified QRD RLS based algorithm Speech+noise mode Noise only mode QRD LSL based algorithm

10 1 CONTENTS Per sample versus per vector classification LSL algorithm Transitions Transition from speech+noise to noise only mode Transition from a noise only to a speech+noise period Noise reduction vs. signal distortion trade off Regularization in QRD LSL based ANC Regularization using a noise buffer Mode dependent regularization Complexity Simulation results Conclusion Integrated noise and echo cancellation Introduction Optimal filtering based AENC Data driven approach QRD RLS based algorithm Speech+noise/echo updates Noise/echo only updates QRD LSL algorithm Regularized AENC Regularization using a noise/echo buffer Mode dependent regularization Performance Complexity Conclusion

11 CONTENTS 11 8 Conclusions 165

12 12 CONTENTS

13 Chapter 1 Speech signal enhancement A microphone often picks up acoustical disturbances together with a speaker s voice (which is the signal of interest). In this work, algorithms will be developed for techniques that allow for removing these disturbances from the speech signal before further processing it. 1.1 Overview In general, more than one type of disturbances will be present in a microphone signal, each of them requiring a specific enhancement approach. We will mainly focus on two classes of speech enhancement techniques, namely acoustic echo cancellation (AEC) (section 1.2.2) and acoustic noise cancellation (ANC) (section 1.2.3). For AEC, a whole range of algorithms exists, from computationally cheap to expensive, with of course a corresponding performance. We will focus on one of the intermediate types of algorithms, of which the performance and complexity can be tuned depending on the available computational power. We will describe some methods to increase noise robustness, we will show how existing fast implementations fail when their assumptions are violated, and we will derive a fast implementation which does not require any assumptions. For ANC, a class of promising state of the art techniques exists of which the characteristics could be complementary to the features of computationally cheaper (and commercially available) techniques. Existing algorithms for these techniques have a high numerical complexity, and hence are not suited for real time implementation. This observation motivates our work in the field of acoustic noise cancellation, and we describe a number of algorithms that are (several orders of magnitude) cheaper than 13

14 14 CHAPTER 1. SPEECH SIGNAL ENHANCEMENT existing implementations, and hence allow for real time implementation. Finally we will show that considering the combined problem of acoustic echo and noise cancellation as a global optimization problem leads to better results than using traditional cascaded schemes. The techniques which we use for ANC can easily be modified to incorporate AEC. The outline of this first chapter is as follows. After a problem statement in section 1.2, we will describe a number of applications in which acoustic echo and noise cancelling techniques prove useful in section 1.3. In section 1.4, an overview of commercially available applications in this field is given. In section 1.5 our own contributions are summarized. Section 1.6 gives an outline of the remains of the thesis. 1.2 Problem statement Nature of acoustical disturbances In many applications involving speech communication, it is difficult (expensive) to place microphones closely to the speakers. The microphone amplification then has to be large due to the large distance to the speech source. As a result, more environmental noise will be picked up than in the case where the microphones would be close to the speech source. For some of these disturbances, a reference signal may be available. For example a radio may be playing in the background while someone is making a telephone call. The electrical signal that is fed to the radio s loudspeaker can be used as a reference signal for the radio sound reaching the telephone s microphone. We will call the techniques that rely on the presence of a reference signal acoustic echo cancellation techniques (AEC), the reason for this name will become clear below. For other types of disturbances, no reference signal is available. Examples of such disturbances are the noise of a computer fan, people who are babbling in the room where someone is using a telephone, car engine noise,... Techniques that perform disturbance reduction where no reference signal is available will be called acoustic noise cancellation techniques (ANC) in this text. In some situations the above two noise reduction techniques should be combined with a third enhancement technique, namely dereverberation. Each acoustical environment has an impulse response, which results in a spectral coloration or reverberation of sounds that are recorded in that room. This reverberation is due to reflections of the sound against walls and objects, and hence has specific spatial characteristics, other than those of the original signal. The human auditory system deals with this effectively because it has the ability to concentrate on sounds coming from a certain direction, using information from both ears. If for example one would hear a signal

15 1.2. PROBLEM STATEMENT 15 recorded by only one microphone in a reverberant room, speech signals may easily become unintelligible. Of course also voice recognition systems that are trained on non reverberated speech will have difficulties handling signals that have been filtered by the room impulse response, and hence dereverberation is necessary. In this thesis, we will concentrate on algorithms for both classes of noise reduction (noise reduction with (AEC) and without (ANC) a reference signal). Dereverberation will not be treated here (we refer to [32, 4, 3] for dereverberation techniques) AEC, reference based noise reduction The most typical application of noise reduction in case a reference signal is available, is acoustic echo cancellation (AEC). As mentioned before, we will use the term AEC to refer to the technique itself, even though the disturbance which is reduced is not always strictly an echo. Single channel techniques. A teleconferencing setup consists of two conference rooms (see Figure 1.1) in both of which microphones and loudspeakers are installed. Near end room Far end Speech AEC Near end Speech Far end room Figure 1.1: Acoustic echo cancellation. The loudspeaker signal in the near end room is picked up by the microphone, and would be sent back to the far end room without an echo canceller, where the far end speaker would hear his own voice again (delayed by the communication setup). Sound picked up by the microphones in one room (called the far end speech and the far end room ) is reproduced by the loudspeakers in the other (near end) room. The task of an echo canceller is to avoid that the portion of the far end speech signal, which is picked up by the microphones in the near end room, is sent back to the far end. Hearing his own delayed voice will be very annoying to the far end speaker.

16 16 CHAPTER 1. SPEECH SIGNAL ENHANCEMENT A similar example is voice control of a CD player. The music itself then can be considered a disturbance (echo) to the voice control system. The loudspeaker signal in both cases is filtered by the room impulse response. This impulse response is the result of the sound being reflected and attenuated (in a frequency dependent way) by the walls and by objects in the room. Due to the nature of this process, the room acoustics can be modeled by a finite impulse response (FIR) filter. Nonlinear effects (mostly by loudspeaker imperfections) are not considered here. In an acoustic echo cancellation algorithm, a model of the room impulse response is identified. Since the conditions in the room may vary continuously (people moving around being an obvious example), the model needs to be updated continuously. This is done by means of adaptive filtering techniques. In the situation in Figure 1.2 the far end signal x(k) is filtered by the room impulse response, and then picked up by a microphone, together with the desired speech signal of the near end speaker. We consider digital signal processing techniques, hence A/D converted signals, i.e. discrete time signals and systems. At the same time, the loudspeaker signal x(k) is filtered by a model w(k) of the room impulse response w real, and subtracted from the microphone signal d(k) : e(k) = d(k) w T (k)x(k). During periods where the near end speaker is silent, the error (residual) signal e(k) may be used to update w(k), but when the near end speaker is talking, this signal would disturb the adaptation process. We assume that the room characteristics do not change too much during the periods in which near end speech is present, and the adaptation is frozen in these periods by a control algorithm in order to solve this problem. x(k) = [x(k)... x(k N+1)] w k Far End Signal e(k) + + d(k) Near end Speech Near end room Figure 1.2: Echo canceller : typical situation. In the acoustic echo canceller scheme, the adaptive filtering structure (see also Figure

17 1.2. PROBLEM STATEMENT ) is easily recognized. The input signal to this adaptive filter is the loudspeaker signal x(k) (the reference signal), the desired signal for the filter is the microphone signal d(k), and the error signal e(k) of the adaptive filter is used as the output signal for the AEC scheme. In practice, the length of the room acoustics (and by consequence also the impulse response length of the model w(k)) can easily be 2 filter taps (even for a rather low sampling frequency of 8 khz). This is the reason why people often use the celebrated and computationally cheap Normalized Least Mean Squares (NLMS) adaptive filter (see section 2.2), or even cheaper frequency domain derivatives of it for adapting w(k) [19, 18]. The disadvantage of NLMS is its often poor performance for non white input signals (like speech). While NLMS is a cheap algorithm, the Recursive Least Squares (RLS) algorithm (section 2.3) has a higher performance, and fast variants are indeed used for acoustic echo cancellation [9, 8, 2, 21]. However, due to its complexity, efforts have been done to find algorithms that combine the low complexity of NLMS with the performance of RLS. Most notably are the Fast Newton Transversal filter (FNTF) [42] and fast variants [26] of the Affine Projection Adaptive (APA) [43] filter (see section 2.4). In this thesis, we derive a number of contributions to the field of APA filtering. The performance advantage offered by these filters compared to NLMS, is due to a prewhitening structure that removes the correlation from the reference signal. As will be shown later, further signal processing may require multiple microphones (a microphone array) that pick up the sound in the room. The echo canceller structure then obviously has to be repeated for each of the microphones, as shown in Figure 1.3. The prewhitening stage, however, can be shared among the different microphones in X (k) Far End e (k) 1 e (k) d (k) 1 d (k) 2 Near end Speech Figure 1.3: Multi microphone acoustic echo canceller. The single channel setup can simply be repeated a multi microphone setup.

18 18 CHAPTER 1. SPEECH SIGNAL ENHANCEMENT An acoustic echo canceller never consists of the adaptive filter alone, but always requires some control logic. The adaptive filter is in practice never updated when near end speech is present, and only updated if there is far end signal available. The decision can e.g. be based upon measurements of the correlation of the residual signal e(k) with the loudspeaker signal. In this text, however, this control device will not be considered. All experiments have been done with a perfect control device, i.e. speech periods have been marked manually. In the acoustic echo canceller context, it is important that the decision device never allows the filter to adapt during a double talk period (when both far end and near end speaker active), since then the adaptation would be disturbed by the near end signal, and the coefficients would converge to wrong values. The other situation is less problematic : when a period in which only far end talk is present is labeled as double talk, the echo canceller would not adapt. If this would happen often, the overall convergence would just be somewhat slower. We refer to the literature [1, 25, 31, 45] from which a suitable implementation can be picked. Multichannel techniques. Multi channel techniques for acoustic echo cancellation [4, 28, 2, 41] should not be confused with multi microphone techniques. In a multi microphone setup, all adaptive filters have the same input signal (the mono loudspeaker signal), while in a multichannel setup, multiple loudspeakers (or reference signals) are used, see Figure 1.4. An application example is a stereo setup used for X (k) 1 X (k) 2 Far End Near end Speech + + d (k) 1 Figure 1.4: Multi channel acoustic echo canceller. The fundamental problem of stereophonic AEC tends to occur in this case, and decorrelation of the loudspeaker signals is necessary to achieve good performance

19 1.2. PROBLEM STATEMENT 19 teleconferencing in order to provide the listener with a comfortable spatial impression. While the extension of the single channel techniques to multiple microphones is trivial, multi channel AEC on the other hand is highly non trivial. A specific problem with multichannel echo cancellation is the non uniqueness [4, 2, 5, 24, 2] of the solution. This is sometimes referred to as the fundamental problem of stereophonic echo cancellation. Since all loudspeaker signals stem from the same sound source in the far end room, their joint correlation matrix may be rank deficient. As a result, there is not a single solution for a multi channel echo canceller, but a solution space. The echo canceller may find a solution for which the output signal is zero in the absence of near end speech, while the filter is not converged to the real room impulse response (see section 3.5). As a result, the slightest change in the far end room impulse response, may destroy the successful echo cancellation. For multichannel echo cancellation both a change in the transmitting and in the receiving room will have this effect. Even if this situation would not occur, still the problem becomes ill conditioned if both far end signals are correlated. This often results in a large sensitivity to noise that may be present in d(k), for example due to continuously present background noise in the near end room. This also indicates that proper measures should be used for the evaluation of different algorithms. One should not only look to the energy in the residual echo signal, because it can indeed be small or zero while the filter has not yet converged to the real echo path. For simulated environments, the room acoustics path is known, and hence the distance between this path and the echo canceller path can be plotted. While this is only feasible in artificial setups, it is the only correct way to evaluate the convergence behaviour of an echo canceller, especially in the multi channel case ANC, reference less noise reduction The signal picked up by the microphone will in realistic situations often also contain disturbance components for which no reference signal is available. Also for this case, multiple approaches to noise cancellation exist. Single channel techniques A microphone picks up a signal of interest, together with noise. Single microphone approaches to noise cancellation will try to estimate the spectral content of the noise (during periods where the signal of interest is absent), and assuming that the noise signal is stationary compensate for this spectrum in the spectrum of the microphone input signal whenever the signal of interest is present. The technique is commonly called spectral subtraction [16, 17]. Single channel approaches are known to perform poorly when the noise source is non stationary, and when the spectral content of the noise source and of the signal of interest are similar.

20 2 CHAPTER 1. SPEECH SIGNAL ENHANCEMENT Multi channel techniques In multi channel acoustic noise cancellation, a microphone array is used instead of a single microphone to pick up the signal. Apart from the spectral information also the spatial information can be taken into account. Different techniques that exploit this spatial information exist. In filter and sum beamforming [6], a static beam is formed into the (assumed known) direction of the (speech) source of interest (also called the direction of arrival). While filter and sum beamforming is about the cheapest multi channel noise suppression method, deviations in microphone characteristics or microphone placement will have a large influence on the performance, Since signals coming from other directions than the direction of arrival are attenuated, beamforming also provides a form of dereverberation of the signal. Generalized sidelobe cancellers (Griffiths Jim beamforming) [6] aim at reducing the response into directions of noise sources, with as a constraint a distortionless response towards the direction of arrival. The direction of arrival is required prior knowledge. A voice activity detector is required in order to discriminate between noise and speech+noise periods, such that the response towards the noise sources can be adapted during noise only periods. Griffiths Jim beamforming effectively is a form of constrained optimal filtering. A third method is unconstrained optimal filtering [12][13]. Here a MMSE optimal estimate of the signal of interest can be obtained, while no prior knowledge is required about geometry. A voice activity detector again is necessary and crucial to proper operation. The distortionless constraint towards the direction of arrival is not imposed here. A parameter can be used to trade off signal distortion against noise reduction. The contributions of this thesis in the field of acoustic noise reduction will be focused on this last method (chapters 5 and 6). Existing algorithms for unconstrained optimal filtering for acoustic noise reduction are highly complex compared to both other (beamforming based) methods, which implies that they are not suited for real time implementation. On the other hand, they are quite promising for certain applications, since they have different features than the beamforming based methods : filter and sum beamformers are well suited (and even optimal) for enhancing a localized speech source in a diffuse noise field, and generalized sidelobe cancellers are able to adaptively eliminate directional noise sources, but both of them rely upon a priori information about the geometry of the sensor array, the sensor characteristics, and the direction of arrival of the signal of interest. This means that the unconstrained optimal filtering technique is more robust against microphone placement and microphone characteristics, and that the direction of arrival is not required to be known a priori. Another advantage is that they can easily be used for combined AEC/ANC, as we show in chapter 7.

21 1.3. APPLICATIONS Combined AEC and ANC In many applications, techniques to cancel noise for which a reference signal exists (AEC) are often combined with techniques that do not use a reference signal (ANC), since both types of disturbances are often present. The order in which both signal processing blocks are applied to the signals is very important. In Figure 1.5, both options are shown. The upper scheme will first apply multichannel noise cancellation (no reference signal), and then echo cancellation. The advantage is that, since most referenceless noise reduction schemes make use of multiple microphones, only one echo canceller is needed. Moreover, in addition to the echo path, the echo canceller will have to model the variations in the noise cancellation block. The lower scheme in Figure 1.5 requires an echo canceller for each microphone, and these need to be robust against the noise that is still present in their input signals. In spite of the higher complexity of the second scheme, it is most often used because of its better performance compared to the first scheme. Apart from these combination schemes, a lot of different combination schemes are described in literature [1, 7, 37, 38, 6]. In this thesis, we will show that considering the combined problem as a global optimization problem leads to a better performance. We will describe how the unconstrained filtering techniques derived in the chapters about noise cancellation, can easily be adopted for solving the combined acoustic noise and echo cancellation problem. For echo paths of reasonable length, real time implementation of these techniques is possible with present day processors. 1.3 Applications Tele and videoconferencing As a first application example we consider teleconferencing. A number of people is meeting in two rooms. In each of these rooms, a microphone array and a loudspeaker are present. The loudspeaker reproduces the sound of the speakers in the other meeting room. The system can be expanded to have more loudspeakers, in order to give the conference participants a spatial impression of the reproduced sound. If no echo cancellation is applied, echo s and howling can occur. Echo paths can be as long as 2 msec, while a sampling speed of about 16 khz is required in order to have a high enough speech quality, resulting in echo path impulse responses of up to 3 taps. On the other hand, people talking in the background, a computer fan, air conditioning are all examples of disturbances that should be handled by means of noise cancellation. Often the echo cancellers in this type of applications could profit from algorithms as described in chapters 3 and 4, of which the convergence is less dependent on the input signal statistics than what is the case for NLMS. Also algorithms providing the

22 22 CHAPTER 1. SPEECH SIGNAL ENHANCEMENT From far end Interference Noise Speaker Desired signal Noise reduction Acoustic echo canceller To far end From far end Interference Noise Speaker Desired signal Acoustic echo canceller Noise reduction To far end Figure 1.5: Two methods to combine echo and noise cancellation.

23 1.3. APPLICATIONS 23 combined ANC and AEC approach in chapter 7 would increase the performance of a speech enhancement system for tele or videoconferencing. Note though that for larger auditoria the required number of filter taps is huge, and that complexity of the algorithms should be taken into account. Car applications In car applications such as voice control of a mobile phone or sound system, or hands free telephony, noise appears to be the most important problem. For engine noise or radio sound, a reference is available or can be derived, while wind and tyre noise, passengers talking to each other,... are disturbances without a reference signal. Acoustic paths in cars are much shorter (up to 256 impulse response taps), as compared to typical conference room impulse responses. Also in this case both ANC and AEC are required. Because of the limited length of the echo path, the algorithms in chapter 7 certainly become an option. Voice control Voice control technology can be found in consumer products, but also finds applications in making technology accessible for disabled people. Speech recognition systems are often trained with clean speech (without noise), because a lot of clean speech databases are available, although also databases are set up for specific noise situations (e.g. speech recognition in cars). A specific problem is voice control of a surround sound audio system, where a multichannel echo canceller is required in order to suppress the signal stemming from the five speakers after being picked up by the microphone. In this case, reference signals are available, and algorithm with a better performance for coloured signals than NLMS are required (chapters 3 and 4). Hearing aids Acoustic noise cancellation techniques are applied in the field of hearing aids and cochlear implants. It is known that merely amplifying a signal does not contribute to increasing the speech intelligibility, when background noises are present. Noise cancellation techniques can alleviate this problem, and at present 2 microphone hearing aids with noise cancellation technology are commercially available. The space (and hence the computational power) in a behind the ear device is limited, so most of the time cheap (adaptive beamforming) algorithms are used at present, but also these devices could benefit from the techniques in chapters 5 and 6. Selective volume control Techniques that are developed for acoustic echo cancelling, can also be applied in other fields. An example is a selective volume control

24 24 CHAPTER 1. SPEECH SIGNAL ENHANCEMENT device, which is used in e.g. discotheques to turn down the sound volume automatically if it exceeds the legal norms. In order to avoid that loud noises made by the crowd would result in lowering the amplifier s volume, an adaptive filter is used to retain only the sound from the loudspeakers in the signal that is picked up by a measurement microphone before the sound pressure level is calculated. A similar system is a volume control application in e.g. a train station, where the volume is automatically turned up if a train passes, or if the crowd is noisy, but which is not sensitive to the sound of the public address system s own loudspeakers. This kind of applications is even more demanding concerning filter lengths than ordinary echo cancelling in rooms. The legal norms about the maximum sound pressure level are given per frequency on the full audible frequency spectrum. This means that a sampling rate of 44 khz is required. So the required filter length is more than 1 filter taps. On the other hand, calculations could be done off line instead of in real time, and the music signals can be largely correlated. This again requires intermediate algorithms between NLMS (convergence depends on input signal statistics) and RLS. Recording A recording of e.g. an orchestra or a theatre play imposes different constraints. Microphones will not be placed in an array with an a priori known geometry, but they will be spread over the whole stage on which the performance takes place. The signal of interest does not originate from one specific direction. In dedicated theatres, the noise will mainly consist of the audience, but also scenario s with noise of air conditioning or heating systems (recordings in churches) are possible. 1.4 The market A large number of companies are currently offering products and services that are linked with the above mentioned speech enhancement techniques. While in high end devices for auditorium teleconferencing (price about 5 Euro) it is difficult to gather information on the type of algorithms used, data sheets about desktop conferencing consumer products often indicate that computationally cheap NLMS like or frequency domain derivatives of NLMS are used. Examples of companies are Spirit Corporation ( providing code libraries for acoustic echo and noise cancellation optimized for different types of DSP processors, and for the Microsoft Windows operating system. Polycom ( provides desktop teleconferencing solutions, and the performance data they publish (a convergence time of 1 4 sec) indicate the use of cheap adaptive filters. Larger systems are e.g. built by Clearone (

25 1.5. CONTRIBUTIONS 25 Another application is audio enhancement. Both the application CoolEdit (from Syntrillium, and SoundForge (from Sonic Foundry. contain signal enhancement modules providing single channel spectral subtraction techniques. Commercial voice command applications often use proprietary techniques based upon beamforming (e.g. with a microphone array on top of a computer monitor (Andrea Electronics, In hearing aids, the commercial state of the art devices use two microphones and Griffiths Jim beamforming based noise cancellation schemes. The importance of speech enhancement technology in the current market is also shown by the fact that in the most recent version of Microsoft Windows XP noise cancellation and echo cancellation features are built in the operating system ( It is clear that in the consumer telecommunications market, the demand for handfree mobile telephony a direct application of the techniques described here is high, because of security (and legal) issues concerning use of a mobile phone while driving. As an example : in 22, the worldwide sales of mobile phones has risen with 6%, 423,4 million devices were sold worldwide ( 1.5 Contributions From section 1.4, one can see that the commercially available applications are all based upon low complexity algorithms, obviously due to real time and cost constraints. For acoustic echo cancelling often more performant algorithms than NLMS based ones begin to be used, certainly in high end applications. The performance and the complexity of the APA based algorithms we have studied in this work can be tuned to use the available computational power. We provide some an alternative for obtaining noise robustness and derive an efficient frequency domain based algorithm, which does not contain any approximations (contrarily to existing implementations) One notices that the computational complexity of the newer (unconstrained adaptive filtering) algorithms for noise reduction prohibits their commercial application. Of course, with the rise of computational power over the years, in a decade from now these algorithms will also be applied, even in consumer electronics. In this text we will focus our attention to some of these new ( academic ) techniques, and we will derive new algorithms that have a (sometimes dramatically) reduced complexity compared to their predecessors, while keeping their performance at the same level. This should allow these more performant techniques to be considered for use in commercial applications in a much shorter time frame. The contributions to the field of speech enhancement which are treated in this text, can be subdivided into three major categories.

26 26 CHAPTER 1. SPEECH SIGNAL ENHANCEMENT The first category consists of signal enhancement techniques for acoustic noise reduction when a reference signal is available (AEC). The results consists of alternative regularization techniques for improving the noise robustness of acoustic echo cancellers based upon the affine projection algorithm (see further on in this text), and the Block Exact Affine Projection Algorithm (BEAPA), which is a fast frequency domain version of the affine projection algorithm with roughly the same complexity as BEFAP (see further on in this text), but without the need for the assumptions that need to be made for BEFAP. The results hereof are published in the conference papers [5, 47, 48, 49] and in the journal paper [55]. They will be treated in chapters 3 and 4. The second category focusses on MMSE based optimal filtering for acoustic noise reduction in case no reference signal is available (ANC). We proposed a QRD RLS and a QRD LSL based approach to unconstrained optimal filtering that achieves the same performance as existing (GSVD based) techniques, but with a complexity reduction of respectively one and two orders of magnitude. These results have been published in the papers [54, 52] and [56, 51]. We will treat them in chapters 5 and 6. Finally, combination of noise and echo cancelling is treated in chapter 7, and this result is in our paper [53]. 1.6 Outline 1. Speech signal enhancement 2. Adaptive filtering algorithms Introduction 3. APA regularization and Sparse APA 4. BEAPA for AEC Acoustic echo cancellation 5. QRD RLS based ANC 6. Fast QRD LSL based ANC Acoustic noise cancellation 7. Integrated noise and echo cancellation Echo and noise cancellation 8. Conclusions Figure 1.6: Outline of the text The outline of the text is depicted in Figure 1.6. Chapter 2 contains additional introductory material. Relevant adaptive filtering algorithms are reviewed, and the concept of signal flow graphs is explained briefly.

27 1.6. OUTLINE 27 Chapter 3 and 4 of the thesis focus on acoustic echo cancellation. More specifically in chapter 3 the importance of noise robustness in acoustic echo cancellers is reviewed, and some techniques are derived to implement this into fast affine projection algorithms. We also show that traditional fast implementations exhibit problems when strong regularization is applied. In chapter 4 a frequency domain block exact affine projection algorithm is derived which does not contain the approximations that are present in traditional fast affine projection schemes, while it has a complexity that is comparable to these schemes. Chapter 5 and 6 focus on acoustic noise cancellation techniques. In chapter 5 an unconstrained optimal filtering based noise cancellation algorithm is derived. This algorithm is based upon the QR decomposition (see section 2.3 for a definition). It obtains the same performance as existing algorithms for unconstrained optimal filtering, while its complexity is an order of magnitude lower. Chapter 6 builds upon the previous one to derive an even cheaper fast QRD based algorithm while again performance is maintained at the same level. In chapter 7 we discuss the combination of AEC and ANC, and show the performance advantage of using an integrated approach to acoustic noise and echo cancellation compared to traditional combination schemes. Chapter 8 finally, contains the overall conclusions of this work, as well as suggestions for further research.

28 28 CHAPTER 1. SPEECH SIGNAL ENHANCEMENT

29 Chapter 2 Adaptive filtering algorithms Adaptive filters will play an important role in this text. Therefore, we will devote a chapter to giving an overview of commonly used adaptive filtering techniques. In section 2.1 the general adaptive filtering setup and problem will be reviewed. The normalized least mean squares algorithm (NLMS) and the recursive least squares (RLS) algorithms will be reviewed in sections 2.2 and 2.3. An intermediate class of algorithms, both complexity and performance wise, can be derived from the affine projection algorithm (APA). APA will be introduced in section 2.4. In each section complete algorithm descriptions of these algorithms will be given for reference. Later on in this text, APA will be the main topic in chapters 3 and 4, where it will be used for acoustic echo cancellation. Chapters 5, 6 and 7 will mainly be based upon algorithms derived from RLS and fast versions thereof. 2.1 Introduction In this introduction we will give a short overview of the data representations that will be used in the remains of the chapter and the thesis. We will use both adaptive filtering configurations with single and multiple input and output channels. A single input, single output adaptive filtering setup is shown in Figure 2.1. An input signal x(k) is filtered by a filter w(k). The output from this filtering operation is subtracted from a desired signal d(k) and the resulting error signal e(k) is used to update the filter coefficients. The signals are assumed to be zero mean, and d(k) is a linearly filtered version of x(k) with zero mean noise added that is assumed to be independent of x(k). 29

30 3 CHAPTER 2. ADAPTIVE FILTERING ALGORITHMS x w d + y e Figure 2.1: Adaptive filter. The filter coefficients w are adapted such that e is minimized. All of the algorithms are based upon an overdetermined system of linear equations X(k)w(k) = ( d(k) d(k 1)... ) T, (2.1) where X(k) = x T (k) x T (k 1) x T (k 2)., x(k) = ( x(k) x(k 1)... x(k N + 1) ) T, which will be solved in the least squares sense, i.e. based on a LS criterion min w LS(k) The LS solution is given as ( d(k) d(k 1)... ) T X(k)w(k) 2. (2.2) w LS (k) = (X T (k)x(k)) 1 X T (k) ( d(k) d(k 1)... ) T. We will also use the MMSE criterion min ε{(d(k) w xt (k)w(k)) 2 }, (2.3) MMSE(k) where ε{ } is the expectation operator. The MMSE solution is given as w MMSE (k) = (ε{x(k)x T (k)}) 1 ε{x(k)d(k)}. In each time step k, a new equation will be added to (2.1), so at each time instant a new value for w(k) can be calculated. Since adaptivity is required in a changing

31 2.1. INTRODUCTION 31 environment, algorithms will be designed to forget old information. This can be achieved by exponentially weighting the rows of X(k) as it is usually done in the RLS algorithm, i.e. X(k) = x T (k) λx T (k 1) λ 2 x T (k 2)., or by only using the P most recent input vectors in X(k) : X(k) = x T (k) x T (k 1). x T (k P + 1). x x x W1 W2 W3 d + + e Figure 2.2: A multi channel adaptive filter. The input vector x(k) consists of the concatenation of the channel input vectors x i(k), and similarly the filter vector w(k) = ( w T 1 (k) w T 2 (k) w T 3 (k) ) T. In this text, we will also consider multichannel (multiple input) adaptive filters (see

32 32 CHAPTER 2. ADAPTIVE FILTERING ALGORITHMS Figure 2.2), where the input vectors x(k) will be defined as x(k) = x 1 (k). x 1 (k N + 1) x 2 (k) x 2 (k 1).. x M (k N + 1). (2.4) Similarly w(k) is then defined as a stacked version of the filter vectors w i (k) for i = 1...M : w(k) = w 1 (k) w 2 (k). w M (k). Here M will be the number of input channels of the adaptive filter, and N is the number of filter taps per input channel. Sometimes an alternative definition for the input vector will be used in which the input signals will be interlaced : x(k) = x 1 (k). x M (k) x 1 (k 1) x 2 (k 1). x M (k N + 1). (2.5) As a result, also the corresponding filter taps will be interlaced. Considering setups with multiple microphones, we will be solving least squares minimization problems that share the same left hand side matrix X(k), but have different right hand side vectors. They can be solved concurrently as one multiple right hand side least squares problem. In this case the columns of a matrix W (k) will be solutions to LS problems with the columns of a matrix D(k) as their respective right hand

33 2.2. NORMALIZED LEAST MEAN SQUARES ALGORITHM 33 sides. A system of equations analoguous to (2.1) can be written down : X(k)W (k) = D(k), (2.6) x T (k) x T (k 1) X(k) =., x(k) = ( x(k) x(k 1)... x(k N + 1) ) T, d T (k) d T (k 1) D(k) =., d T (k 1) d(k) = ( ) T d 1 (k) d 2 (k).... Note the structure of d(k) of which the components represent the different desired signal samples at time k. The least squares solution can be found from min D(k) X(k)W (k). (2.7) W (k) The corresponding MMSE criterion is min ε{d(k) x T (k)w (k)}. W (k) In the next sections we will give an overview of the different adaptive filtering techniques that will be used in this thesis. 2.2 Normalized Least Mean Squares algorithm One approach to solving (2.1) is the Least Mean Squares (LMS) algorithm. This algorithm is in fact a stochastic gradient descend method applied to the underlying MMSE criterium (2.3). The update equations for the filter coefficient vector w lms (k) are e(k + 1) = d(k + 1) x T (k + 1)w lms (k), w lms (k + 1) = w lms (k) + µx(k + 1)e(k + 1), (2.8) y(k + 1) = x T (k + 1)w lms (k + 1). (2.9) Here µ is a step size parameter. A full description is shown in Algorithm 1. In order to make the convergence behaviour independent of the input energy, often the Normalized Least Mean Squares (NLMS) algorithm is used, where the filter vector

34 34 CHAPTER 2. ADAPTIVE FILTERING ALGORITHMS Algorithm 1 LMS algorithm w lms = ;y = ; Loop (new input vector x and desired signal d in each step): e = d y w lms = w lms + µxe y = x T w lms update is divided by the input energy. The algorithm is given by e(k + 1) = d(k + 1) x T (k + 1)w nlms (k), w nlms (k + 1) = x(k + 1)e(k + 1) w nlms (k) + µ x T (k + 1)x(k + 1) + δ. (2.1) Here the δ is a regularization term. In NLMS it guarantees that the denominator can not become zero, but it also provides noise robustness (see section 2.4). Similar equations are obtained for the definitions (2.6). It can be shown that, for µ = 1 and δ =, the a posteriori error for NLMS, e post (k + 1) = d(k + 1) x T (k + 1)w nlms (k + 1), is zero, which means that for the NLMS algorithm the systems of equations (2.1) or (2.6) are effectively reduced to one single equation, namely the most recent one, and that this equation is solved exactly based on a minimum norm weight vector adaptation. NLMS is a computationally cheap algorithm with a complexity of 4N flops per sample 1, but it suffers from a slow convergence when non white input signals are applied. In practice often frequency domain variants of this algorithm are used in order to obtain an even lower complexity. An algorithm description of the time domain NLMS algorithm is given in Algorithm 2.. Algorithm 2 NLMS algorithm w nlms = ; y = Loop (new input vector x and desired signal d in each step): e = d y w nlms = w nlms + µ xe x T x+δ y = x T w nlms We also note here that if the LMS algorithm is to be calculated for multiple desired (right hand side) signals, the whole algorithm simply has to be repeated for each desired signal. In the NLMS algorithm the (small) cost of calculating the input energy 1 For complexity calculations in this text we will count an addition and a multiplication for 2 separate floating point operations

35 2.3. RECURSIVE LEAST SQUARES ALGORITHMS 35 can be shared : e T (k + 1) = d T (k + 1) x T (k + 1)W nlms (k), W nlms (k + 1) = W nlms (k) + µ x(k + 1)eT (k + 1) x T (k + 1)x(k + 1) + δ. (2.11) 2.3 Recursive Least Squares algorithms In this section, we will first review the standard recursive least squares algorithm, then the numerically more stable (and thus preferrable) QRD based RLS algorithm, and finally the fast QRD Least Squares Lattice algorithm Standard recursive least squares Instead of applying a stochastic gradient descent method (NLMS), the recursive least squares (RLS) algorithm solves system (2.6) or (2.1) in a least squares (LS) sense, i.e. based on the LS criterion (2.2), and does so by applying recursive updates to the solution calculated in the previous time step (cfr. newton-iterations on a quadratic error surface where the hessian reduces to a correlation matrix). For exponentially weighted RLS, the update equations are e rls (k + 1) = d(k + 1) x T (k + 1)w rls (k), Ξ 1 (k + 1) = 1 λ 2 Ξ 1 (k) 1 λ 2 Ξ 1 (k)x(k + 1)x T (k + 1) 1 λ 2 Ξ 1 (k) λ 2 x T (k + 1)Ξ 1 (k)x(k + 1) w rls (k + 1) = w rls (k) + Ξ 1 (k + 1)x(k + 1)e rls (k + 1), (2.12) where Ξ 1 (k) is the inverse correlation matrix (Ξ(k) = X T (k)x(k)). The first equation calculates the error at time instant k + 1, while the second equation is the filter coefficient update. Instead of doing an update in the direction of the input vector x(k) as in LMS, in (2.12) the input signal can be seen to be whitened because it is multiplied by the inverse correlation matrix. An algorithm description is provided in Algorithm 3. Again a regularization (or better : diagonal loading ) term can be added to the inverse correlation matrix w rls (k + 1) = w rls (k) + (X T (k + 1)X(k + 1) + δi) 1 x(k + 1)e rls (k + 1). Here I is the unity matrix. It is well known that this provides robustness to noise terms that could be present in d(k) [27].,