Efficient Use Of Sparse Adaptive Filters

Transcription

1 Efficient Use Of Sparse Adaptive Filters Andy W.H. Khong and Patrick A. Naylor Department of Electrical and Electronic Engineering, Imperial College ondon {andy.khong, Abstract We present a novel adaptive algorithm exploiting the sparseness of an impulse response for network echo cancellation. This sparseness-controlled improved proportionate normalized least mean square (SC-) algorithm is based on which allocates a step-size gain proportional to each filter coefficient. The proposed SC- algorithm achieves improved convergence over by estimating the sparseness of the impulse response and allocating gains for each stepsize such that a higher weighting is given to the proportionate term of the for sparse impulse responses. For a less sparse impulse response, a higher weighting will be allocated to the term. Simulation results presented show improved performance over the algorithm during convergence before and after an echo path change has been introduced. We also discuss the computational complexity of the proposed algorithm. xn ( ) h ˆ( n) en ( ) yn ( ) Fig.. yn ˆ( ) h( n) Schematic diagram of an echo canceller. wn ( ) I. INTRODUCTION Adaptive algorithms have been employed extensively in signal processing applications for system identification, channel equalization and prediction []. Classical adaptive algorithms with a uniform step-size across all filter coefficients, such as the normalized least-mean-square () algorithm, have slow convergence when estimating sparse impulse responses such as those that occur in some telecommunication networks. In such systems where traditional telephony equipment is connected to the packet switched network [], the echo path impulse response, which is typically of length 64-8 ms, exhibits an active region in the range of 8- ms duration and consequently, the impulse response is dominated by regions where magnitudes are close to zero, making the impulse response sparse. The inactive region is due to the presence of bulk delay caused by network propagation, encoding and jitter buffer delays [3]. Other applications for sparse system identification include wavelet identification using marine seismic signals [4] and geophysical seismic applications [5][6]. One of the first algorithms for network echo cancellation (NEC) is the proportionate (P) algorithm [7] where each filter coefficient is updated with an independent step-size that is proportional to the magnitude of that estimated filter coefficient. The P algorithm achieves fast initial convergence but subsequently slows down due to the slow convergence of coefficients having significantly smaller magnitudes. Subsequent improved versions such as the [8] and I [9] algorithms were proposed which achieve improved convergence by introducing a controlled mixture of proportionate (P) and non-proportionate () adaptation. Consequently, these algorithms perform better than P for sparse network impulse responses and in some cases for non-sparse acoustic impulse responses []. To reduce the computational complexity of P, the sparse partial update (SP) algorithm was proposed [] where, similar to the selective partial update (SPU- ) algorithm [], only taps corresponding to the M largest absolute values of the product between input signal and filter coefficients are selected for adaptation. An optimal step-size for P has been derived in [3] and employing an approximate μ-law function, it has been shown that the proposed segment P (SP) outperforms the P algorithm. Adaptive algorithms deployed for echo cancellation are expected to track any variations in the impulse response in order to achieve sufficient level of echo cancellation. The aim of this work is to develop an adaptive algorithm which utilizes the sparseness of an impulse response for NEC application. This is achieved by incorporating the sparseness measure [4][5] into such that the proportionate term in the update is allocated a higher weighting for increasing sparse systems. The result of this work is the sparseness-controlled (SC-) which is robust to the sparseness nature of the impulse response. This paper is organized as follows: In Section II, we review adaptive algorithms for sparse system identification. We then review and illustrate the sparseness of an impulse response in Section III while in Section IV we propose the SC- algorithm and discuss the computational complexity relative to the algorithm. Section V evaluates the performance of SC- while Section VI concludes our work /6/$. 375

2 II. SPARSE SYSTEM IDENTIFICATION We review the P and algorithms which have been proposed for sparse system identification. With reference to Fig., we first define filter coefficients and tap-input vector as ĥ(n) = [ĥ(n) ĥ(n)...ĥ (n)] T, () x(n) = [x(n) x(n )...x(n +)] T, () where is the adaptive filter length and the superscript T is defined as the transposition operator. The adaptive filter will model the unknown impulse response h(n) using the near-end signal y(n) =x T (n)h(n)+w(n), (3) where w(n) is the observation noise. For simplicity, we shall temporarily ignore the effects of w(n) in the description of algorithms. A. General formulation Sparse adaptive algorithms such as presented in [7][8] for network echo cancellation can be generalized using the following set of equations: T e(n) = y(n) x (n)ĥ(n ), (4) μq(n )x(n)e(n) ĥ(n) = ĥ(n ) + x T (n)q(n )x(n)+δ, (5) where μ is the adaptive step-size and δ is the regularization parameter. The diagonal control matrix Q(n ) = diag{q (n )... q (n )} (6) determines the step-size gain for each filter coefficient and is dependent on the specific algorithm as described below. B. The P algorithm The P algorithm [7] assigns higher step-sizes for coefficients with higher magnitude using a control matrix Q(n). Elements of this control matrix for P can be expressed as κ l (n) q l (n) = i= κ i(n), (7) { κ l (n) = max ρ } max{γ, ĥ(n)... ĥ (n) }, ĥl(n) (8) with l =,,..., being the tap-indices. The parameter γ, with a typical value of., prevents ĥl(n) from stalling during initialization stage where ĥ() = while ρ prevents coefficients from stalling when they are much smaller than the largest coefficient. The regularization parameter δ in (6) for P is taken as δ P = δ /, (9) where δ = σ x is the variance of the input signal [8]. It can be seen that for q l =, l, P is equivalent to. C. The algorithm An enhancement of P is the algorithm [8] which is a combination of P and with the relative significance of each controlled by a factor α. Elements of the control matrix Q(n) for are given by q l (n) = α ( + α) ĥl(n) +, l =,...,, () ĥ(n) + ɛ where ɛ is a small value and. is the l -norm operator. It can be seen from the second term of () that the magnitude of the estimated taps are normalized by the l -norm of ĥ. This shows that the weighting on the step-size for is dependent only on the relative scaling of the filter coefficients as oppose to their absolute values. Results presented in [8][6] have shown that good choices of α values are, -.5 and The regularization parameter δ in (6) for should be taken [8] as δ = α δ. () This choice of regularization ensures that the algorithm achieves the same steady-state normalized misalignment compared to that of the algorithm for the same stepsize, i.e., μ = μ. It can be seen that is equivalent to when α = while for α close to, behaves like P. III. SPARSENESS MEASURE The sparseness of an impulse response can be quantified by the sparseness measure [4][5] [ h(n) ξ(n) = ], h(n) () where h(n) is the l -norm of h(n). By considering impulse responses with various degrees of sparseness it can be shown that ξ(n). A. Sparseness of a delta function The sparseness of a length impulse defined by { ±v, n = n, h(n) =, n,n n, (3) where n defines the location of the impulse, can be computed from () by first considering h(n) = h l (n) = v, l= h(n) = h l (n) = v l= from which substituting into () gives ξ(n) =. 376

3 B. Sparseness of a signal with constant magnitude For h(n) of length having a constant magnitude ±v, we have h(n) = v = v, l= h(n) = v = v. l= Substituting these equations into (), the sparseness for h(n) is then ξ(n) =. C. Variation of ξ with sparseness Sparseness of impulse responses for NEC can be studied by generating synthetic impulses using random sequences. This can be achieved by first defining a vector u = [ p e /ψ e /ψ... e (u )/ψ] T, (4) where the p leading zeros models the length of the bulk delay and u = p is the length of the decaying window while ψ Z + is the decay constant which may be time-varying. For brevity of notation, we ignore the dependence of ψ on time index n. Defining a u vector b as a zero mean white Gaussian noise (WGN) sequence with variance σ b,the synthetic impulse response can then be expressed as B u u = diag{b}, [ ] p h(n) = p p u u + p, (5) u p B u u where the vector p ensures elements in the inactive region are small but non-zero and is an independent zero mean WGN sequence with variance σp. Figure shows an example set of impulse responses generated using (5) with σb =.87, σ p =.45 4, = 5 and p =3. Impulse responses with various degrees of sparseness are generated using decaying constants (a) ψ = 8, (b) ψ =, (c) ψ = 5 and (d) ψ = giving (a) ξ(n) =.845, (b)ξ(n) =.84, (c)ξ(n) =.674 and (d) ξ(n) =.57 respectively. Hence it can be seen that the sparseness of the impulse response increases with ξ(n). IV. THE SC- AGORITHM We propose the sparseness-controlled (SC- ) algorithm by taking into account the sparseness of an impulse response. As explained in Section I, it is known that the takes into account the slow convergence of the inactive region for the P algorithm. As a result, the fast convergence of is achieved by combining an term ( ( α)/() ) ( with the proportionate term (+α)( ĥ l (n) )/( ĥ +ɛ) ) where the relative significance of each term is controlled by the parameter α. As can be seen, the parameter α is chosen a priori for. The proposed SC- algorithm further improves the performance of by emphasizing the proportionate (a) (c) (b) (d) Fig.. Impulse responses controlled using (a) ψ =8,(b)ψ =,(c)ψ = 5 and (d) ψ = giving sparseness measure (a) ξ =.845, (b) ξ =.84, (c) ξ =.674 and (d) ξ =.57. term if the impulse response is significantly sparse. For relatively less sparse impulse responses, SC- allocates a higher weighting to the term. Similar to α, incorporation of ξ(n) into the algorithm can be achieved using two terms +ξ(n), for the proportionate term, and ξ(n), for the term in (). Direct incorporation of ξ(n) into () however is not feasible since (i) the impulse response h(n) is unknown and (ii) ξ(n) can be close to unity. The approach to (i) is to estimate ξ(n) using filter estimates obtained for each iteration giving [ ξ(n) = ĥ(n) ]. (6) ĥ(n) For n =,thel -norm of the filter coefficients ĥ(n) = and hence to prevent division by a small number or zero, this estimation can be computed for n. Forn<elements of the control matrix q l (n) is computed using (). Although it is highly unlikely that ξ(n) =, a high estimate of ξ(n) will introduce significant coefficient noise during adaptation since the effective step-size is increased. Hence to address (ii), we reduce the effect of ξ(n) on q l (n) using [ +.5 ξ(n)]/ and [.5 ξ(n)]/ for the proportionate and terms respectively. The computation of q l (n) for the SC- algorithm can then be expressed as q l (n) = [ ].5 ξ(n) α + [ ] +.5 ξ(n) ( + α) ĥ l (n) ĥ + ɛ (7) for n and l =,...,. The proposed SC- algorithm can be described by ()-(6) and (6), (7). 377

4 SC SC SC Fig. 3. Relative convergence of, and SC- using WGN input. Impulse response Fig. (b) is used. [α =.75, μ = μ =.3, μ SC =.7, SNR = 3 db]. A. Computational complexity We consider the computational complexity of the proposed SC- algorithm in terms of the number of multiplications required relative to the algorithm for each time iteration. The requires O(4) multiplications per iteration [9][6]. The additional complexity of the SC- algorithm compared to arises from the computation of ξ(n) and q l (n) given by (6) and (7). Given that /( ) can be computed off-line, and since ĥ(n) computation is already available from (), the remaining computation of (6) requires only + multiplications. For (7), 5 multiplications are required in addition to that for the. Hence, the complexity for the SC- is O(5). V. SIMUATION RESUTS The performance of SC- is compared with and in the context of system identification for network echo cancellation. This performance can be quantified using the normalized misalignment defined by η(n) = h(n) ĥ(n) h(n). (8) We assumed throughout our simulation that the length of the adaptive filter is equivalent to that of the unknown system. Any effects of undermodelling will introduce a bias to the normalized misalignment [7]. A. Simulations using WGN input In this first experiment we used impulse response as shown in Fig. (b) with = 5. A zero mean white Gaussian noise (WGN) sequence is used as the input signal while another zero mean WGN sequence w(n), as illustrated in Fig., is added to achieve a signal-to-noise ratio (SNR) of 3 db. The stepsizes for and are μ = μ = Fig. 4. Relative convergence of, and SC- using WGN input with echo path change at.5 s. Impulse response is changed from Fig. (a) to (b). [α =.75, μ = μ =.3, μ SC =.7, SNR = 3 db]. while the step-size for SC- is adjusted in order to achieve the same steady-state normalized misalignment as that for and. This corresponds to μ SC =.7. Similar to [6], we have used α =.75 for and SC-. Figure 3 shows convergence result where we note that SC- achieves improved normalized misalignment of approximately 7 db compared to that of the during initial convergence. Both and SC- achieve a higher rate of convergence than the as expected. In the second experiment, we used impulse responses as shown in Fig. (a) and (b) with = 5. As before, a zero mean WGN sequence is used as the input signal while an SNR of 3 db is used. The step-sizes are μ = μ =.3 and μ SC =.7 with α =.75 for and SC-. Figure 4 shows convergence result when an echo path change is introduced midway through the simulation where we changed the impulse response from a sparse to one which is relatively less sparse as shown in Fig. (a) to (b). We note from this result that SC- achieves improved normalized misalignment of approximately 6 db compared to that of the during initial convergence. After the echo path change at.5 s, a 3 db improvement in normalized misalignment can be seen for SC- compared to that of. As before, both and SC- achieve a higher rate of convergence (during initial phase and after echo path change) than the as expected. Figure 5 shows results when the initial impulse response is less sparse (Fig. (b)) compared to that after the echo path change (Fig. (a)). Step-size and α parameters are the same used for the previous experiment while the SNR is also at 3 db. From Fig. 5, the SC- algorithm achieves the highest rate of convergence with an approximately 7 and 3 db improvement in normalized misalignment over that for the during convergence before and after echo path 378

5 3 SC SC Fig. 5. Relative convergence of, and SC- using WGN input with echo path change at.5 s. Impulse response is changed from Fig. (b) to (a). [α =.75, μ = μ =.3, μ SC =.7, SNR = 3 db]. Speech SC 6 SC Fig. 6. Relative convergence of, and SC- using speech input with echo path change midway through simulation. Impulse response is changed from Fig. (a) to (b). [α =.75, μ = μ =.3, μ SC =.7, SNR = 3 db]. change respectively. B. Simulation using speech input Figure 6 shows results using speech input from a male speaker. The signal comprises 6 sentences from the same male speaker. In this experiment, a sparse impulse response is changed to one that is less sparse as shown in Fig. (a) to (b). As before, we have used μ = μ =.3, μ SC =.7in order for the algorithms to achieve the same steady-state normalized misalignment. We have also used α =.75 for the and SC- algorithms. An SNR of 3 db is used for this experiment. As before, the proposed SC- algorithm achieves the highest rate of convergence. Compared to the algorithm, SC- achieves an improvement of approximately.9 db normalized misalignment during initial convergence. VI. CONCUSION We have proposed the SC- algorithm for network echo cancellation. This algorithm estimates the sparseness of an impulse response and allocates a higher weighting to the proportionate term in the gain matrix for a relatively more sparse impulse response compared to one which is less sparse. Simulation results presented showed approximately.9 to 7 db improvement in convergence for the SC- algorithm compared to. REFERENCES [] S. Haykin, Adaptive Filter Theory, 4th ed., ser. Information and System Science. Prentice Hall,. [] M. M. Sondhi and D. A. Berkley, Silencing echoes on the telephone network, in Proc. IEEE, vol. 68, Aug. 98, pp [3] J. Radecki, Z. Zilic, and K. Radecka, Echo cancellation in IP networks, in Proc. Fourty-Fifth Midwest Symposium on Circuits and Systems, vol.,, pp. 9. [4] M. Boujida and J.-M. Boucher, Higher order statistics applied to wavelet identification of marine seismic signals, in Proc. Eur. Signal Process. Conf., 996. [5] Y.-F. Cheng and D. M. Etter, Analysis of an adaptive technique for modeling sparse systems, IEEE Trans on Acoustics, Speech, and Signal Processing, vol. 37, no., pp , Feb [6] E. A. Robinson and T. S. Durrani, Geophysical Signal Processing. Englewood Cliffs, NJ: Prentice-Hall, 986. [7] D.. Duttweiler, Proportionate normalized least mean square adaptation in echo cancellers, IEEE Trans. Speech Audio Processing, vol. 8, no. 5, pp , Sep.. [8] J. Benesty and S.. Gay, An improved P algorithm, in Proc. IEEE Int. Conf. Acoustics Speech Signal Processing, vol.,, pp [9] J. Cui, P. A. Naylor, and D. T. Brown, An improved algorithm for echo cancellation in packet-switched networks, in Proc. IEEE Int. Conf. on Signal Processing, vol. 4, May 4, pp [] R. Ahmad, A. W. Khong, and P. A. Naylor, Proportionate frequency domain adaptive algorithms for blind channel identification, in Proc. IEEE Int. Conf. Acoustics Speech Signal Processing, vol. 5, May 6, pp. V9 V3. [] H. Deng and M. Doroslovacki, New sparse adaptive algorithms using partial update, in Proc. IEEE Int. Conf. on Signal Processing, vol., May 4, pp [] K. Dogancay and O. Tanrikulu, Adaptive filtering algorithms with selective partial updates, IEEE Trans. Circuits Syst. II, vol. 48, no. 8, pp , Aug.. [3] H. Deng and M. Doroslovacki, Improving convergence of the P algorithm for sparse impulse response identification, IEEE Signal Processing ett., vol., no. 3, pp. 8 84, Mar. 5. [4] P. O. Hoyer, Non-negative matrix factorization with sparseness constraints, Journal of Machine earning Research, vol. 5, pp , Nov. 4. [5] J. Benesty, Y. A. Huang, J. Chen, and P. A. Naylor, Adaptive algorithms for the identification of sparse impulse responses, in Selected methods for acoustic echo and noise control, E. Hänsler and G. Schmidt, Eds. Springer, 6, ch. 5, pp [6] A. W. H. Khong, J. Benesty, and P. A. Naylor, An improved proportionate multi-delay block adaptive filter for packet-switched network echo cancellation, in Proc. European Signal Processing Conference, Sep. 5. [7] C. Breining, Control of a hands-free telephone set, Signal Processing, vol. 6, pp. 3 43,