A LOW COST COUPLED LMS ADAPTIVE ALGORITHM WITH AFFINE PROJECTION PERFORMANCE FOR AUTORREGRESSIVE INPUTS: MEAN WEIGHT ANALYSIS

Transcription

1 6th European Signal Processing Conerence (EUSIPCO 28), Lausanne, Sitzerland, August 25-29, 28, copyright by EURASIP A LOW COS COUPLED LMS ADAPIVE ALGORIHM WIH AFFINE PROJECION PERFORMANCE FOR AUORREGRESSIVE INPUS: MEAN WEIGH ANALYSIS Márcio H. Costa, Sérgio J.M. Almeida, and Guilherme H. Costa Depto de Engenharia Elétrica, Universidade Federal de Santa Catarina, Florianópolis, Brazil Escola de Engenharia e Arquitetura, Universidade Católica de Pelotas, Pelotas, Brazil Depto de Engenharia Mecânica, Universidade de Caxias do Sul, Caxias do Sul, Brazil costa@eel.usc.br; smelo@ucpel.tche.br; ghcosta@ucs.br ABSRAC his ork presents a lo cost ast convergence adaptive algorithm or autoregressive (AR) input signals. It basically consists on an NLMS adaptive algorithm coupled ith a linear adaptive predictor. Mean eight theoretical analysis shos that this adaptive structure, under AR conditions, provides a perormance quite similar to the ell knon Aine Proection algorithm. heoretical results are supported by Monte Carlo simulations shoing a very good matching or considerably correlated signals. his adaptive algorithm could be a good choice in control and iltering applications that require high adaptation speed and lo computational cost.. INRODUCION he Normalized Least Mean Square (NLMS) is a idely used adaptive algorithm in practical applications due to its robustness and ease o implementation. Hoever, its maor draback is its slo convergence speed or correlated input signals. he Aine Proection (AP) algorithm as originally proposed by Ozeki and Umeda in 984 [] as a solution or this problem. he AP algorithm updates the adaptive ilter eights in directions that are orthogonal to the last P input vectors. It has been shon that the AP algorithm converges much aster than NLMS or correlated inputs. he price or such perormance is a computational complexity o 2NPk inv P 2 multiply and accumulate operations, here N is the number o adaptive coeicients, P is the AP order and k inv is about 7 [2]. As a result, despite its good properties, real-time applications o the AP algorithm in multichannel and long-ilter problems are still restricted. Dierent approaches have been used to improve the convergence speed o adaptive ilters hile keeping a lo computational cost. Fast and eicient versions o the AP algorithm have been proposed. Hoever, such ast algorithms have a higher complexity than the NLMS and suer rom instability problems associated ith the matrix inversion process [2],[3]. In [4], Mboup et al. presented an adaptive structure here a linear adaptive predictor ilters the input and error signals used in the LMS update equation. he coupled adaptive ilter shoed aster speed hen compared ith the conventional LMS. Aterards, [5] presented a similar structure here the predictor is replaced by a copy o the adaptive ilter coeicients. As a result, this algorithm presents a aster speed ith a small additional complexity (5% greater). Hoever such algorithms are capable o astening the LMS velocity, the desired AP convergence characteristics and NLMS lo cost are not simultaneously obtained. his ork demonstrates, through analytical and simulation results, that or medium to high correlated autoregressive (AR) input signals the perormance o the AP algorithm can be achieved by using a very simple structure ith a very lo extra computational complexity hen compared to the conventional NLMS. his algorithm consists on an adaptive linear predictor that decorrelates the input regressor o the NLMS adaptive ilter. A mean eight theoretical analysis is provided in order to sho that this algorithm has approximately the same irst order behaviour as the AP algorithm. Simulation results corroborate the analysis providing the evidence that both algorithms also present the same excess mean square error. he main limitation o this algorithm is the use o very high correlated input signals. In such situation the algorithm may not converge. Finally, simulations ith a real acoustic signal sho its useulness in real conditions. 2. INPU SIGNAL MODEL In this ork the input signal u(n) is assumed to be a zeromean ide sense stationary AR process o order P. It can be described by P u( n) = au i ( n i) z( n) () here a i are the AR coeicients and z(n) is a ide sense stationary hite process ith variance r z. A set o N consecutive samples o () can be described by the olloing matrix notation u( n) = U( n) a z ( n) (2) here u(n)=[u(n) u(n-)... u(n-n)], a=[a a 2 a P ], z(n)=[z(n) z(n-)... z(n-n)], and U(n)=[u(n-) u(n-2) u(n-p)]. 3. HE AP ADAPIVE ALGORIHM he eight error update equation o the unit step-size AP algorithm (resulting in a scalar error) subected to an AR input can be ritten as [6]

2 6th European Signal Processing Conerence (EUSIPCO 28), Lausanne, Sitzerland, August 25-29, 28, copyright by EURASIP Assuming an AR input, an LMS based update strategy, a suiciently small step-size and enough time or achieving steady-state, the predictor coeicients a(n) tend to the AR parameters a, in such ay that lim n E{a(n) a [7]. In such case u (n) z(n). Figure Block diagram o the proposed algorithm. Figure 2 Block diagram o the linear predictor. v v Φ ( n) ( n) Φ( n) AP AP AP n = n e n Φ here v AP (n)= AP (n)- o, AP (n)=[ AP AP... APN- ] is the adaptive coeicient vector, o is the plant to be identiied, e AP (n)=-v AP (n)u(n)r(n), r(n) is the additive noise, Φ n = u n U n a n (4) and a AP (n) is the least squares estimate o the AR coeicients: ( n ) ( n ) ( n aap = U U ) U ( n ) u ( n ) (5) here U (n)u(n) is assumed o rank P; and a AP (n)=[a AP (n) a AP2 (n) a APP (n)]. It as demonstrated in [6] that the mean eight error behaviour o the AP can be analytically described by: E { ( n vap ) { AP N P 2 E v n (6) here E{ corresponds to expectation. 4. NLMS PLUS A LINEAR PREDICOR he proposed adaptive structure consists on including an adaptive linear predictor (S) to decorrelate the NLMS input signal. his structure is shon in Fig.. Fig.2 details the structure o the considered predictor S. From the block diagram shoed in Fig., the proposed eight error update equation is given by u ( n) v( n ) = v( n) µ e ( n) (7) u ( n) u ( n) here en = v ( n) u ( n) rn (8) v(n)=(n)- o, u (n)=[u (n) u (n-)... u (n-n)] and u (n) is the output o the adaptive predictor at iteration n. AP (3) 5. MEAN WEIGH BEHAVIOUR his section presents a theoretical analysis o the mean eight behaviour o the proposed algorithm in order to demonstrate its similarity ith the analytical model o the AP algorithm presented in [6]. he study o the statistical properties o the eight vector requires simpliications in order to make the problem mathematically tractable. hese simpliications are ell established in the adaptive ilter analysis area and, due to their complexity and length, proos ill be presented or reerenced as needed. For simplicity e assume that there is enough time or the adaptive predictor to achieve steady-state condition in order that u (n) z(n) [7]. Considering this, using (8) in (7) and taking the expectation e obtain E v n = E v n { { E r n z n u n v n z n (9) µ µ E z n z n z n z n Assuming independence beteen the input signal and the additive noise [7] e have ( n) r( n) z( n) z E E E = { r( n) = () z n z n z n z n Using (2) and () in (9) z( n) z ( n) v( n) E{ v( n ) = E{ v( n) µ E z ( n) z( n) () au ( n) v( n) z( n) µ E z ( n) z( n) Assuming a large number o coeicients e can use the averaging principle [8], and approximate () by E v n E v n { { µ E{ z ( n) z( n) E{ z( n) z ( n) v( n) µ E{ z ( n) z( n) E a U ( n) v( n) z( n) { (2) In appendix A, assuming v(n) has a Gaussian probability density unction [9], it is demonstrated that E{ au ( n) v( n) z( n ) = (3) Using (3) in (2) e obtain E{ v( n ) E{ v( n) (4) µe{ z ( n) z( n) E{ z( n) z ( n) v( n) he second expected value on the right hand side o (4) as already evaluated in []. Dierently o [], in this case z (n)z(n) has truly a chi-square distribution ith N degrees o reedom since z(n) is hite. As a result:

3 6th European Signal Processing Conerence (EUSIPCO 28), Lausanne, Sitzerland, August 25-29, 28, copyright by EURASIP { = ( N 2) E z n z n r (5) he last expectation in (4) can be obtained through the use o the independence theory [7] E{ z( n) z ( n) v( n) E{ z( n) z ( n) E{ v( n) (6) = re z { v( n) Using (5) and (6) in (4) e inally obtain a deterministic recursive equation or describing the mean eight behaviour o the ne algorithm: µ E{ v( n ) E{ v ( n) (7) N 2 6. COUPLED NLMS VERSUS AP ALGORIHMS he similarity beteen the AP and the coupled NLMS algorithms can be veriied comparing Eq. (7) ith the AP mean eight model, given in Eq. (6): E{ vap ( n ) E{ vap ( n) ( N P 2) (8) µ E{ v( n ) E{ v( n) N 2 Eq. (8) indicates that, or the same initialization, both algorithms must sho approximately the same mean eight behaviour i the step-size o the proposed algorithm is equated to: ( N 2) µ = (9) ( N P 2) or N>>P then µ. 7. SIMULAIONS his section presents comparisons among the proposed algorithm, the AP and the conventional NLMS algorithms. hree examples are provided. he irst to are didactical situations here the input signals are generated by irst and second order AR processes. In both cases, the plant impulse response is a tap normalized Hanning indo ( o o =). In the third example e consider a real acoustic impulse response and an input signal (certainly not corresponding to an AR process) sampled rom a real IBM-PC cooler. he common design parameters or all simulations are: 5 predictor iterations beore running the adaptive algorithms; predictor step-size µ P =.; ()=; regularization actor ε= -4 []; additive noise ith poer r r = -6. Example AR(): in this example e consider a correlated input signal generated by a irst order AR model given by u(n)=a u(n-)z(n), ith a =.9 and r z =.9. he step-size o the proposed algorithm is evaluated rom (9) (hich leads to µ=.) and the step-size o the conventional NLMS is set to µ NLMS =. he AP and the predictor orders are equal to the autoregressive order. Monte Carlo (MC) simulations consider 5 runs. Fig.3 shos the mean eight behaviour and Fig.4 the excess mean square error (EMSE) considering the three algorithms. z Example 2 AR(2): in this example the input signal is generated by a second order AR process given by u(n)=.2u(n-)-.85u(n-2)z(n) (poles radii equal to.922) ith r z = All other simulation conditions are the same as in the Example. Fig.5 shos the mean eight behaviour and Fig.6 the excess mean square error. Example 3 Acoustic noise: in this example the input signal is sampled (at 44.kHz) rom a real IBM- PC cooler acoustic noise. he plant impulse response corresponds to the irst 28 samples o a real acoustic response o a room. he adaptive ilter uses 28 coeicients; µ=µ NLMS =; predictor and AP orders are set to ; 3 runs ere perormed in the MC simulations. Fig.7 shos the mean eight behaviour and Fig.8 the excess mean square error. Only one in each ity samples is plotted in order to obtain smoother curves. Mean eight ( d ) Figure 3 Example. Mean eight behaviour o the 5 th coeicient. (a) AP (red); (b) conventional NLMS (green); (c) NLMS predictor (blue); and (d) analytical model (black) Figure 4 Example. Excess mean square error. (a) AP (red); (b)

4 6th European Signal Processing Conerence (EUSIPCO 28), Lausanne, Sitzerland, August 25-29, 28, copyright by EURASIP.6 = ( d ) Mean eight.8 Mean eight Figure 5 Example 2. Mean eight behaviour o the 5 th coeicient. (a) AP (red); (b) conventional NLMS (green); (c) NLMS predictor (blue); and (c) analytical model (black) x 4 Figure 7 Example 3. Mean eight behaviour o the 64 th coeicient. (a) AP (red); (b) conventional NLMS (green); and (c) NLMS predictor (blue) Figure 6 Example 2. Excess mean square error. (a) AP (red); (b) 8. DISCUSSION All simulation and analytical curves sho a quite good matching. he behaviour o the conventional NLMS as presented only in order to sho the gain o perormance (due to the hiten characteristics) o the analysed algorithms. he mean eight curves illustrate the similarity beteen the AP and the proposed algorithm behaviours, as ell as the validity o the theoretical analysis. he MC simulations sho good matching even in near real conditions, hen non-ar input signals are considered (Example 3). he excess mean square error simulations corroborate, under all assessed conditions, to the equivalence beteen the proposed structure and the AP algorithm. he proposed algorithm has a computational load o N2P9 multiply and accumulate (MAC) operations [2]. his amount is o the same order o the conventional NLMS (N8) and much less than the AP algorithm (2NP7P 2 ). able presents the main steps to implement the proposed algorithm. In spite o the good results o the proposed algorithm it as veriied that it shos stability problems or AR processes ith pole radius above.95 and highly correlated non x 4 Figure 8 Example 3. Excess mean square error. (a) AP (red); (b) AR signals. In such case ast versions o the AP algorithm should be used. As a result, beore practical application it is necessary a careul investigation about the input signal characteristics. A theoretical analysis o the excess mean square error behaviour ill be a topic o a uture ork. able : Proposed algorithm Equation (, P) (=, N) Comment u ( n) = [ u( n) u( n N )] Input vector u n u n au n i P = i ( ) Filtered input n u n u n N Filtered input u = ( n ) = ( n ) µ ( ) a a u n u n i i i P ( n) = ( n) ( ) vector Predictor update equation N i Error signal e d n u n i µ e ( n) ( n ) = ( n) u N n 2 u n k ε k = NLMS update equation

5 6th European Signal Processing Conerence (EUSIPCO 28), Lausanne, Sitzerland, August 25-29, 28, copyright by EURASIP 9. CONCLUSION his ork presented the mean eight statistical analysis o the NLMS adaptive algorithm hen the input regressor is iltered by a linear adaptive predictor. he obtained theoretical model, valid only or autoregressive signals, shos that its perormance is similar to the Aine Proection algorithm, or a properly chosen step-size. Monte Carlo simulations sho a very good match beteen both algorithms even or non-autoregressive inputs. his adaptive structure can be a good choice in control and iltering applications that require high adaptation speed and lo computational cost. ACKNOWLEDGMENS his ork as supported by the Brazilian Ministry o Science and echnology (CNPq) under grant 3866/25-8. APPENDIX A: Evaluation o E{a U (n)v(n)z(n) Assuming three Gaussian random vectors y =a U (n), y 2 =z(n) and y 3 =v(n) [9] they can be expanded as an orthonormal series [3] given by y = A y2 = B22 (A) y3 = C C22 C33 c here i =[ i- i-2 i-n ] or,2; E{ l-i k- i,l k =; E{ 2 -i =; c is a mean value vector and a, a 2, a 2,2 A = a 3, a 3,2 a 3,3 a 4, a 4,2 a 4,3 a 4,4 (A2) b2, b2 2, b2 2,2 B2 = b2 3, b2 3,2 b2 3,3 b2 4, b2,4,3 b2,4,3 b2 4,4 Premultiplying y 3 by y and postmultiplying the result by y 2 e obtain yyy = AC B 3 2 i 2 2 = ( AC) = ( AC) = ( A ) = 2 i i c B B B i 2 2 (A3) aking the expected value o (A3), using the independence beteen the orthonormal basis (E{ l-i k- i,l k =) and using (A) e come to (3). REFERENCES [] K. Ozeki and. Umeda, An adaptive iltering algorithm using orthogonal proection to an aine subspace and its properties, Electron. and Communic. in Japan, vol. 67- A(5), pp.9-27, Feb [2] S. L. Gay and S. avathia, he ast aine proection algorithm, in Proc. o Icassp 22, pp [3] H. Ding, A stable ast aine proection adaptation algorithm suitable or lo-cost processors, in Proc. o Icassp 2, pp [4] M. Mboupp, M. Bonnet and N. Bershad, LMS coupled adaptive prediction and system identiication: a statistical model and transient mean analysis, IEEE rans. Signal Proc., vol. 42(), pp , Oct [5] S. C. Douglas, A. Cichocki and S. Amar Selhitening algorithms or adaptive equalization and deconvolution, IEEE rans. Signal Proc., vol. 47(4), Apr [6] S. J. M. Almeida, J. C. M. Bermudez, N. J. Bershad and M. H. Costa, Statistical analysis o the aine proection algorithm or unity size and autoregressive input, IEEE rans. Circ. and Syst. - I, vol. 52, pp , 25. [7] S. Haykin, Adaptive Filter heory, second edition, Prentice-Hall, 99. [8] J. E. Mazo, On the independence theory o equalizer convergence, Bell Syst. ech. Journal, vol. 58, pp , May-June 979. [9] N. J. Bershad and Z. Q. Lian, On the probability density unction o the LMS adaptive ilter eights, Acoustics, Speech, and Signal Proc., vol. 37(), pp , 989. [] M. H. Costa and J. C. M. Bermudez, An improved model or the normalized LMS algorithm ith Gaussian inputs and large number o coeicients, in Proc. o Icassp 22, pp. -4. [] J. C. M. Bermudez and M.H. Costa, A statistical analysis o the epsilon-nlms and NLMS algorithms or correlated Gaussian Signals, Revista da Sociedade Brasileira de elecomunicações, vol. 2(2), pp. 7-3, 25. [2] Y. Lu, R. Foler, W. ian, Enhancing echo cancellation via estimation o delay, IEEE rans. Signal Proc., vol.53(), pp , 25. [3] N. J. Bershad, P. Celka and J. M. Vesin, Stochastic analysis o gradient adaptive identiication o nonlinear systems ith memory or Gaussian data and noisy input and output measurements, IEEE rans. Signal Proc., vol. 47(3), pp , 999.