Robust Estimation Methods for Impulsive Noise Suppression in Speech

Transcription

1 Robust Estimation Methods for Impulsive Noise Suppression in Speech Mital A. Gandhi, Christelle Ledoux, and Lamine Mili Alexandria Research Institute Bradley Department of Electrical and Computer Engineering Virginia Polytechnic Institute and State University Abstract We discuss a new robust time domain filtering method that detects and reconstructs speech segments corrupted by impulsive noise. Robust statistical methods are very effective in the case of impulsive environments such as wireless communications and cellular phone applications. The speech signal may be corrupted by impulsive noise lasting several milliseconds. We utilize a robust estimator of covariance based on one-dimensional projections and sample median calculations to detect these impulsive segments. This method, called Projection Statistics, is a very computationally efficient algorithm to suppress the impulses. We estimate the missing segments of speech using the linear prediction technique whose parameters are estimated using a robust Schweppe-type Huber generalized maximum likelihood (GM) estimator. A robust estimator is needed since speech signals closely follow the Laplacian distribution rather than the Gaussian and edges from the impulses may be leftover in the signal. We provide preliminary simulation results from actual speech containing co-channel and fading interferences from cellular phones. 1. Introduction Degradations due to noise are ubiquitous in speech, ranging from applications in automatic recognition to human understandability. We work specifically with speech segments corrupted by impulsive noise. Co-channel and fading interferences in cellular phones introduce impulsive noise, making a robust solution to this problem a necessity [1]. Even though speech processing has been an area of active research, a satisfactory solution does not exist to suppress impulsive noise. The noise cannot be represented as white, zero-mean, Gaussian, as usually assumed. Impulsive noise is typically large amplitude spikes often overlapped over several hundred samples, which make it difficult to identify and suppress. Unlike other types of ambient noise, impulsive sources completely destroy the information content of the speech segment. Moving median filters have been suggested as a method of suppressing these pulses [2]. It is simply replacing the point in the center of the window by the median of the points currently included by the moving window. This window has to be larger than twice the pulse width though, which is often too large to maintain good signal quality, especially for noise corruption of around 40ms. Hence, even though the sample median is known to be a robust estimator, this method tends to decrease the quality of the speech signal. From a statistical viewpoint, the corruption of a signal by impulsive noise can be seen as the introduction of outliers among the data. Following this idea, filtering techniques have consisted of estimating an autoregressive (AR) model via the Huber M-estimator [3, 4]. Work has also been done where the speech signal is treated as a time series with missing data once the noise has been removed. The general iterative expectationmaximization (EM) algorithm has been used for estimation in such frameworks [5]. The technique we propose using Projection Statistics is able to (a) detect impulsive noise efficiently and (b) estimate replacement values for these outliers robustly via the autoregressive model. Projection Statistics was initiated by Donoho in 1982 [6] and applied to sparse non-linear regression model by Mili in power system state estimation [7]. Its statistical properties have been studied by Rousseeuw and van Zomeren [8]. The novelty in our work lies in the application of Projection Statistics to both detect and suppress the impulses. We realize that frequency domain approaches are often favored for noise filtering and speech enhancement, but we have observed smearing over time in the spectrogram of the corrupted speech. Our time domain approach provides better alignment in time of the PS values with the impulses to detect the noisy segments much more accurately. Our method based on Projection Statistics is also very computationally efficient compared to a frequency approach. The paper is organized as follows. Section 2 contains a discussion of background topics from signal processing and robust statistics, including breakpoint and bounded influence. We further discuss robust estimation theory and proposed framework in Section 3. The implemented procedure is discussed in Section 4, with simulation results in Section Background 2.1. Noise and Speech Models Impulsive noise is modeled via exponential functions mirroring each other about a discontinuity point, as shown in Figure 1. These spikes are clearly disturbing visually and audibly, as seen in Figure 2. The corrupted speech spectrogram, shown in Figure 3, also shows strong distortions in the form of vertical striations. The speech is sampled at 8KHz with impulses lasting upto 240 samples, representing cellular channel and fading intereferences quite well. The first of our contributions in this paper is to efficiently detect such noise via a time domain technique. For the estimation process, we use the very popular

2 Figure 1: Impulsive Noise Figure 3: Corrupted Speech Spectrogram 3. Robust Speech Processing 3.1. Autoregressive and Linear Regression Models We model the speech signal via the LPC technique, which uses the autoregressive model given by s n = n k=1 a k s n k + e k (1) Figure 2: Impulsive Noise Corrupted Speech LPC method to model speech [9, 10]. The second of our contributions in this paper is to apply Projection Statistics to the AR model by estimating the parameters of the model with a robust Huber Schweppe-type GM-estimator. The parameters are obtained over window lengths of around 30ms to obtain quasistationarity over segments of speech Breakdown Point and Bounded Influence Two properties in robust statistics are key to an estimator s bias stability: positive breakdown point and bounded influence under contamination. The breakdown point ǫ corresponds to the maximum fraction of outliers to which an estimator yields a finite maximum bias under contamination. In other words, the breakdown point can be treated as a rough upper bound on the fraction of outliers for which the estimator can be considered reliable [11, 12]. The asymptotic highest possible value for ǫ is 1 and is reached by the sample median. GM-estimators in 2 regression have bounded influence properties under infinitesimal contamination with a positive breakdown point. While the breakdown point of GM-estimators may not be high enough to directly handle outliers induced by the impulses, the positive breakdown point is still to our advantage since we only apply the estimator after removing the impulses using Projection Statistics, which do have a high breakdown point. This AR model can be casted into a linear regression form, as follows: z = Hx + e (2) where x is a n 1 vector containing parameters a k, k = 1,..., n z is a m 1 vector of samples over a window H is a m n observation matrix containing past samples e is a m 1 observation error vector The LPC technique consists of first estimating the parameters a k in Eq. 1 and then predicting the current output s n. The coefficients a k are commonly obtained by a least squares minimization on an error signal, yielding an optimal solution if the errors are Gaussian. We discuss how to robustly estimate a k in Section Robust Estimation of Covariance Mahalanobis Distances Each row h T i of the H matrix in Eq. 2 contains samples from the speech signal, identifying a point in the n-dimensional space called the factor space. The impulses in the speech signal induce outliers in this space. To pinpoint these outliers, we need to define in a robust way the standardized distance with respect to the bulk of the point cloud. The classic distances are Mahalanobis Distances (MD), given by Õ MD i = (h i h) T C 1 (h i h) where h = 1 m h i

3 and C = 1 m 1 (h i h)(h i h) T Unfortunately, the MDs are not robust because if one of the h i becomes arbitrarily large, the mean vector h will become arbitrarily large too and the covariance matrix C will explode, making these distances unreliable. We will see that the impulseinduced outliers in space of the observation matrix H can be identified by their respectively large Projection Statistics, as defined next Projection Statistics Carrying out one-dimensional projections provides a robust distance metric, yielding the so-called Projection Statistics [6, 7] obtained by the following procedure: Calculate m = median i,j (h i,j) v i = h i m, i = 1, 2,..., m For every direction v i, calculate where z i,j = y i,j = h jv T i L i = median j(y i,j) yi,j Li, j = 1, 2,..., m S i S i = median i y i median j(y j) m ν = Note that y (i,ν) is the ν th -ordered observation. Calculate the projection statistic of the i th datapoint, given by PS i = max j(z i,j) In our case, because the point cloud is centered on zero, m = 0, L i = 0, and median j(y j) = 0. Figure 4 shows the good performance of PS versus MD. When the redundancy m is less n than 5, it can be shown that the Projection Statistics follow a χ 2 n distribution if h is normally distributed [7]. If the redundancy is larger than 4, as in our case, it is the squared Projection Statistics that follows a χ 2 n distribution [11] Robust Parameter Estimation of AR Model Once the impulses have been removed, we need to replace the missing values in the signal. To this end, we use an autoregressive model of order 20 to reconstruct speech samples along with a robust GM-estimator to estimate the AR model parameters. The solution of the GM-estimator yielding the model parameters minimizes an objective J, expressed as J(x) = i where the weights w i are defined by ρ( ri ) w 2 i (3) w i = 1 for PS i b (4) = b/ps 2 i for PS i > b Figure 4: 97.5% confidence ellipses for MD and PS; 20% outliers The threshold b in Eq. 4 is typically set to χ 2 n, In Eq. 3, s is usually a robust scale estimator such as the median absolute deviation, defined as s = median i r i (5) Also, in Eq. 3, ρ( r i ) is the Huber function defined as ρ( ri ) = 1 2 ( ri ) 2, = c ri c 2 /2 for ri < c elsewhere A good choice for c would be 1.5 [7]. The function ψ( r i ) = ρ ( r i ) is useful in obtaining the weight function given by q( r i ) = ψ( r i ) r i. Minimizing the objective function yields the form of Eq. 6. The iteratively reweighted least squares (IRLS) solution to the problem is derived in subsequent equations. ψ( ri )w ih i = 0 (6) q( ri ) h i r i = 0 H T Qr = 0 H T Q(z Hx) = 0 The IRLS algorithm is then expressed as x (ν+1) = [H T Q (ν) H] 1 H T Q (ν) z (7) where the superscript ν denotes the iteration step and Q is a diagonal weight matrix defined by Q = diag[ q( ri s w i ) ] (8) Due to the AR model, an outlier in z i will also be one in h i. These outlying points (z i, h i) associated with the impulses are

4 Figure 5: Corrupted speech signal with corresponding Projection Statistic values termed bad leverage points [12, 7]. In least squares, all data points are given equal weight in Q; in contrast, we used Projection Statistics in Eq. 4 to obtain the weights for the points with respect to the point cloud robustly. The weight definition in Eq. 4 provides robustness against the bad leverage points by saturating the objective function when the weights are large enough such that the influence on paramter estimation is bounded. 4. Implemented Procedure 4.1. Impulse Detection We first discuss detection of the impulsive speech segments. A matrix H of an AR(4) containing consecutive values of the signal is built after the signal is segmented over 30ms windows. The dimension of each vector h i of H was chosen to be 4, which was empirically determined to yield good results. Each row of this matrix will have an associated Projection Statistic, which is compared to a threshold. For the given speech signal, we obtain Projection Statistics as shown in Figure 5. It is clear that the impulsive segments of speech are flagged with significantly higher PS values, representing outliers in the 4- dimensional space associated with the matrix H Prediction of the Missing Values We now discuss the speech reconstruction process. Again, each cluster of points deemed as impulsive noise is replaced with estimated values based on linear prediction with the parameter vector being obtained by (7). Accordingly, the following process was applied: Obtain a 120 sample signal segment immediately preceding the first outlier to be replaced. Formulate the z vector and H matrix using these 120 samples. The AR order was chosen to be 20. Estimate the parameter vector x. Note that the weight matrix may be initialized with an identity matrix and iteratively derived. The value for MAD in the process is Figure 6: Sample reconstructed segments of corrupted speech only estimated the first 4 times to avoid possible instability. Once x has been computed, excite the AR model with Laplacian noise, since speech follows closely the Laplacian distribution, to obtain the replacement samples. Gain factor problems may arise if one is not careful with such an AR process. Note also that the edges of the impulse are often not detected based on the comparison between the threshold and the Projection Statistics. Thus, we may see a rise in the filtered signal amplitude representing portions that were not replaced immediately followed by estimated lower amplitude values. To account for the edges, we followed a heuristic approach of replacing an additional 10 values preceding and following the first and last outliers. 5. Simulation Results 5.1. Temporal and Statistical Results A professionally recorded phrase back one message, containing simulated co-channel and fading interferences from cellular phones, was used in these simulations. We show four of the segments being reconstructed in Figure 6. It demonstrates the original spiky signal and the reconstruction. Also shown subsequently in Figure 7 is a plot of the associated projection statistics and final weights used in the parameter vector estimation for each of the signal samples over time corresponding to the third reconstructed segment in Figure 6. An appreciable suppression in impulsive noise can be noticed in Figure 8 when compared to the corrupted speech in Figure 2. The entire filtered speech signal is shown in Figure 9. Figures 10 and 11 allow us to better evaluate the improvement realized after filtering. Particularly, the overall and maximum errors in the clean/filtered signal pair have decreased appreciably in comparison to the clean/noisy signal pair Spectral Results We present an example from a specific reconstruction segment in the time domain (Figure 12) and its corresponding spectrum

5 Figure 7: Associated PS and weight values for segment 3 from reconstruction figure Figure 9: Comparison of original, noisy, and filtered Figure 8: Filtered signal (Figure 13). It is clear from the spectral image that the filtered signal (solid line with markers) follows the original speech spectrum (dotted line) much better when compared to the noisy signal spectrum (light solid line) Frequency versus Time Domain Approaches We proposed a time domain approach to the problem, as opposed to the frequency domain processing that has become almost default in many speech processing algorithms. First, Projection Statistics by itself is a fast algorithm as it does not require iterative calculations, making the spike detection computationally efficient. The estimation does take O(n) operation, but it leads to a reasonable solution at this computation level. We can obtain the FFT through O(n log n) operations; however, the spike detection and the more difficult task of reestimating the corrupted segment still remain and would add significant computational complexity, making the time domain approach appealing at this time. In addition to the computational advantage, the time domain is also more appealing than frequency from an algorithmic viewpoint of (a) noise detection and (b) speech estima- Figure 10: Histogram of errors between clean and filtered signals tion. Identifying the impulses along with the edges may not be straightforward task in frequency. While the large PS values align sharply with the spikes in the time domain, the strong vertical striations visible in the spectrogram (Figure 3) are slightly smeared and eventually misaligned with the impulses in time after about 1s. More study is needed to understand these distortions. Even though the impulse edges were difficult to identify in the time domain, at least the heuristic approach suggested earlier or fine-tuning the threshold are possible solutions. The misalignment in frequency prevents the use of such tricks to identify the edges. Finally, the robust GM-estimator presents a reasonable speech estimation solution. 6. Conclusion We have presented a time domain approach to suppress impulsive noise in speech, by using robust Projection Statistics and GM-estimators. The solution was based on an autoregressive model from robust statistics without assuming a distribution on the noise. Preliminary simulation results were presented to sup-

6 Figure 11: Histogram of errors between clean and noisy signals Figure 13: Spectral reconstruction sample Figure 12: Time domain reconstruction sample port the technique. We will use the work by Rubin [13] in future studies to implement a robust version of the EM algorithm for estimating missing speech data. It should also be noted from the results that the edges are still tracked and followed by the estimator. Future work may extend this autoregressive technique to incorporate a smoothing process to enhance the estimation. [6] M. Gasko and D. Donoho, Influential Observation in Data Analysis, American Stat. Assn., Proc. of the Business and Economic Statistics Section, pp , [7] L. Mili et. al., Robust Estimation Based on Projection Statistics, IEEE Trans. on Power Systems, Vol. 11, pp , [8] P. Rousseeuw and B. Van Zomeren, Robust Distances: Simulations and Cutoff Values, Directions in Robust Statistics and Diagnostics, Part II, edited by W. Stahel and S. Weisburg, Springer-Werlag, pp , [9] L. Rabiner and B. Juang, Fundamentals of Speech Recognition, New Jersey: Prentice-Hall, [10] J. Proakis et. al., Discrete-time Processing of Speech Signals, New York: Macmillan, [11] P. Rousseeuw and A. Leroy, Robust Regression and Outlier Detection, John Wiley and Sons, New Jersey, [12] F. Hampel et. al., Robust Statistics: Approach Based on Influence Functions, John Wiley and Sons, NY, [13] R. Little and D. Rubin, Statistical Analysis with Missing Data, John Wiley and Sons, New York. 7. References [1] P. Cardieri and T. Rappaport, Statistical Analysis of Cochannel Interference in Wireless Communications Systems, Wireless Communications and Mobile Computing, Vol. 1, No. 1, pp , January-March [2] S. Vaseghi, Advanced Signal Processing and Digital Noise Reduction, John Wiley and Sons, England, [3] S. Vaseghi and P. Rayner, Detection and Suppression of Impulsive Noise in Speech Communications, IEE Proc. of Communications, Speech, and Vision, pp , [4] K. Lee et. al., Robust Estimation of AR Parameters and its Application for Speech Enhancement, ICASSP, Vol. 1, pp , [5] F. Jelinek, Statistical Methods for Speech Recognition, The MIT Press, Cambridge, Massachusetts, 1997.