Noise constrained least mean absolute third algorithm

Transcription

1 Noise constrained least mean absolute third algorithm Sihai GUAN 1 Zhi LI 1 Abstract: he learning speed of an adaptie algorithm can be improed by properly constraining the cost function of the adaptie algorithm. Besides the stabilization of the NCLMF algorithm is more complicated whose stability depends solely on the input power of the adaptie filter and the NCLMF algorithm with unbounded repressors is not mean square stability een for a small alue of the step-size. So in this paper a noise ariance constrained least mean absolute third (LMA) algorithm is inestigated. he noise constrained LMA (NCLMA) algorithm is obtained by constraining the cost function of the standard LMA algorithm to the third-order moment of the additie noise. And it can eliate a ariety of non-gaussian distribution of noise such as Rayleigh noise Binary noise and so on. he NCLMA algorithm is a type of ariable step-size LMA algorithm where the step-size rule arises naturally from the constraints. he main aim of this wor is first time to derie the NCLMA adaptie algorithm analyze its conergence behaior mean square error (MSE) mean-square deiation () and assess its performance in different noise enironments. Finally the eperimental results in system identification applications presented here illustrate the principle and efficiency of the NCLMA algorithm. Key words: Adaptie filter NCLMF noise constrained non-gaussian NCLMA. 1. Introduction Adaptie filter (AF) algorithm is frequently employed in equalization actie noise control acoustic echo cancellation biomedical engineering and many other fields [1][]. In many communication applications additie white Gaussian noise is an appropriate model for thermal and possibly interference noise and the noise power can be accurately estimated prior to channel tap estimation. Hence the inestigation of the benefits of incorporating noise ariance nowledge into an adaptie channel estimation algorithm is of some practical interest in recent decades noise as a constraint has been studied for eample the noise constrained LMS (NCLMS) algorithms [3][]. Besides although the NCLMS-type algorithms hae many adantages we may hae worse choice for system identification with noise constraint in some case where the measurement noise is non-gaussian [5]. So the noise constrained LMF (NCLMF) algorithm was proposed [6]. But the stabilization of the LMF algorithm is more complicated than that of the LMS algorithm whose stability for a gien step-size depends solely on the input power of the adaptie filter and the LMF algorithm with unbounded repressors is not mean square stability een for a small alue of the stepsize [7]. Significantly the LMA algorithm is built on the imization of the mean of the absolute error alue to the third power. he error function is a perfect cone function with respect to filter coefficients so there is no local imum for the LMA algorithm [8]. Zhi Li @163.com Sihai Guan gcihey@sina.cn 1 School of Electro-Mechanical Engineering Xidian Uniersity N0. South aibai Road Xi an China

2 Using a constraint would mae the step size in the LMA algorithm act as a ariable step size. For the research of ariable step-size LMA algorithm Guan S and Li Z proposed an NVSLMA algorithm [9]. In the NVSLMA algorithm fewer parameters need to be set. But this algorithm not used the additie noise ariance. Ultimately on the basis of wor in [6][8] we propose a noise constrained LMA (NCLMA) algorithm by constraining the cost function of the standard LMA algorithm to the third-order moment of the additie noise ariance. Besides the mean conergence steady state MSE and of the NCLMA algorithm are deried. he computational compleity of the NCLMA algorithm is analyzed theoretically. Besides its characteristics are ealuated in different noise enironments. In order to further illustrate that the NCLMA algorithm is superior to the LMA and NCLMF algorithms a number of simulation results are carried out to corroborate the theoretical findings and as epected improed performance is obtained through the use of this technique. he paper is organized as follows: the proposed NCLMA algorithm is introduced in Section. he performance of the NCLMA algorithm is eplored in Section 3. he numerical simulations are carried out in Section and the conclusion is drawn in Section 5.. he proposed NCLMA algorithm he output of an FIR channel in the presence of an additie noise can be written as d w (1) opt where w opt w0 w1 w corresponds to a channel/system impulse response with N N1 taps. N is the filter length. he input data ector of the unnown system at time instant and the obsered output signal are denoted by ( ) ( 1) ( 1) N and respectiely. is a zero-mean i.i.d. process with an arbitrary probability density function. It has a ariance of. We consider real-alued quantities for simplicity. So e d w () he cost function to be imized is gien by m E 0 w e for m (3) Minimization of the cost function (3) oer gies the optimal weight alue at time i.e. w wopt with w opt where is the alue of the cost function (3) ealuated at w w opt. Consequently the Lagrangian for this problem can be set up as w w 1 w w () he critical alues of Eq. () are w wopt for any γ. his situation may cause conergence problem. So a new augmented Lagrangian as follows: w w w (5) he uses of the γ and γλ in Eq. (5) ensure the unique critical alue of w is w w 0. In Eq. (5) the augmented Lagrangian is imized with respect to the opt d

3 weight w and maimized with respect to λ respectiely. By applying the Robbnis Munro method [3] the weight and λ are updated as follows: 1 w where α and β are the positie learning parameters. he cost function to be imized is gien by w w w (6) 1 w (7) 3 w E e (8) Substituting the results for deriaties in Eq. (6) and Eq. (7) and redefining α and β (replace α by α/3 and βγ by β/) we obtain the following update recursions: where sgn e 1 sgn e w w e (9) 1 (10) e (11) denotes the sign function of the ariable i.e. if then otherwise sgn e 1.. α β and γ are positie parameters proiding more control oer the performance of the proposed algorithm. As suggested in [3] after fiing the noal step-size α γ should be chosen as a alue as large as possible to obtain a fast conergence rate. In additions β should be chosen as a small alue to achiee a desired MSE and. sgn 1 e 3 If 0 in (11) as e we get what we call the zero noise 1 1 constrained LMA (ZNCLMA) algorithm which is similar to the NCLMA but results in a mismatch due to the fact that the true noise absolute third moment is replaced by zero. he ZNCLMA can be analyzed using the same approach used to analyze the NCLMA in the sequel. Also it can be obsered from Eq. (9) and Eq. (10) that the LMA algorithm [10] is recoered when γ=0. 3. Performances of the NCLMA algorithm In this section the mean behaior and the second order moments of the proposed NCLMA algorithm are analyzed. We mae the following assumptions to analyze the NCLMA algorithm [9][1]: A1: is a wide-sense stationary sequence of i.i.d. Gaussian random ariables with zero-mean and positie definite autocorrelation matri R. A: is independent of the input process. A3: For any time instant α and w are statistically independent. A: For any time instant and wopt w are statistically independent. A5: enconditioned ( ) on is Gaussian with zero mean and e e. A6: he statistical dependence of and can be neglected. A7: and are statistically independent. Note that when parameters are chosen so that the steady-state ariance of α or w is e e 0

4 small then A can be approimately satisfied he computation compleity Unlie the LMA algorithm the parameters α and λ required to calculate in NCLMA algorithm. So computational compleity of the NCLMA algorithm is much than the LMA algorithm. Furthermore there is no sgn required to operate in NCLMF algorithm. But there is a higher order power operation for the error in NCLMF algorithm. For conenience the computational compleity of the NCLMA algorithm and that of other eisting algorithms is listed in able 1. As can be seen from able 1 the computational compleity of NCLMF algorithm and NCLMA algorithm is approimate. able 1. he computational compleity of NCLMA LMA and NCLMF algorithms Algorithm Multiplications Comparisons Additions ( ) ( ) 3 LMA N+ 1 N NCLM F N+9 0 N+5 1 NCLM A N+7 1 N Mean weight behaior model Based on Eq. (9) defining the weight error ector Epression Eq. (11) can be rewritten as e 1 sgn e e (1) e 1 e e 1 e 1 e 1 n e-n n0 0 has an initial alue of 0. So Substituting Eq. (1) into Eq. (10) hen defining we get 1 n n n (13) e (1) e (15) 1 n n n 1 n n0 Now substituting Eq. (16) into Eq. (1) n e (16)

5 1 1 1 n 3 1 e e sgn e n n0 1 1 n 3 sgn 1 e e en e sgn e n0 Redefining the time-arying term as 1 n n n (17) e (18) We obtain the final epression for the weight error ector update equation e e e e (19) 1 sgn sgn Using the aboe mentioned assumptions it can be shown that the mean behaior of the weight error ector is goerned by the following recursion e e e e E 1 E E sgn E sgn (0) Note that Eq. (0) is identical to the mean weight alue epression of the standard LMA algorithm ecept for the last term which can be written as follows E e sgn e n 3 1 n e n e e e e n0 n0 E 1 sgn E 1 sgn n 3 1 n E 1 e sgn 1 E sgn n e e e e n0 n0 1 We readily note that 1 following relation n0 1 n 3 n e (1) is statistically independent of resulting in the 1 1 n n 3 e n e e en e e n0 n0 E 1 sgn E 1 E sgn () Substituting Eq. (1) and Eq. () into Eq. (0) we get E 1 e e e e E E sgn E sgn n 3 1 n E E 1 1 E sgn e n e e n0 n0 he distribution function of Y e conditioned on F y p e y y y 1 e ep( e e is ) d e under A5 then the probability density function of Y e conditioned on is shown as Eq. (5) p y y F y 1 y ep( ) y e e F y (3) () (5)

6 he distribution function ofy e conditioned on is F y under A5 F y p e y y y 1 e ep( e e ) d e then the probability density function ofy e conditioned on So and and So hus E 1 E 1 n0 p y y F y y ep( ) e e is shown as Eq. (7) (6) (7) E e (8) e 3 3 Ee (9) e E e sgn( e ) E E e sgn( e ) e R E 1 n 3 1 Ee n 1 1 n0 1 n 3 en n 3 1 en n0 n n 3 1 n E E 1 1 E sgn en e e n0 n0 1 1 E 1 E 1 E n0 n0 1 n 3 1 n en e R 1 e R E 3 e n E n0 1 I 1 1 E R n0 3 e n e Condition for the stability of the mean weight error ector (as Eq. (3)) is gien by (30) (31) (3)

7 n0 3 e n e ma where ma is the maimum eigenalue associated with R. In addition when range of α is the LMA algorithm. As seen from Eq. (3) we will get 3.3. Second-order moment analysis he MSE is gien by E 0 0 as (33) this. E e E E tr R E (3) where E. Pre-multiplying Eq. (1) by its transpose form yields 1 sgn sgn sgn sgn e e e e e e e e e sgn e e sgn e e sgn e e sgn e e e e e aing the epected alue and we also assume that N is sufficiently large so that E N because of an essentially ergodic assumption such that the time aerage oer the taps is equal to the ensemble aerage. E 1 e e e e e e E E sgn E sgn E sgn E e sgn e E e E e E e e e E E E sgn E E E e E E E E E E e e o further simplify Eq. (36) we can tae adantage of Eq. (13) and Eq. (17) in [11] i.e. E E (37) (35) (36)

8 E e E E 6 E 6 E E 6 N E 6( ) E 3( ) E N N Substituting Eq. (37) and Eq. (38) into Eq. (36) E 1 E E E E E E e e E E E e So 6 E E E 6( ) E 3( ) E N N N 1 E e 6 E E E 6( ) 3( ) N N N Eq. (0) will epress as Eq. (1) when discard. 1 N E e 6( ) E E N E E E o further simplify Eq. (6) we can tae adantage of Eq. (31) Eq. () Eq. (3) and Eq. () i.e. and and 1 1n 1 1 n0 1 n0 6 E e e e ( 1 n ) 1 1 ( ) (38) (39) (0) (1) () 6 y ep( )d y (3) 0 e y

9 So 1 E 1 n0 1 n 3 1 E en 1 6 n0 1 n 3 1 E e E n en 1 1 n e n e n n A e n e n0 n0 1 1 A B ( N ) 1 A e n ( ) e n e n 0 n n n n0 n0 1 1 A B N 1 15 E A e n 0 0 ( ) e n e n 0 n n n n0 n0 where A 1 1 and B 1 1 (7) (8). As β should be chosen as a small alue to achiee a desired MSE and in addition according to the eisting literature [3][6] we can now and when0. t Eq. (8) will epress as Eq. (9) when discard for t. 1 A 1 B A e n e n0 n0 1 1 A B ( N ) 1 A e n ( ) e n e n 0 n n n n0 n0 1 1 A B N 1 15 E A e n 0 0 ( ) e n e n 0 n n n n0 n0 So where 1 A D( ) e A B F( ) ( ) 6( N ) 1 A D( ) A B F( ) N 1 ( ) E A D ( ) 1 3 en n C ( ) D( ) C( ) and F( ) 15 C( ). n 0 en n0 n0 herefore we obtain an epression of the behaior of the NCLMA algorithm i.e. where 1 e e (9) (50) f N g N (51)

10 f N e A B F( ) (5) 1 ( ) 6( ) 1 ( ) A D e N A D ( ) A B F( ) g N 1 ( ) E e N A D ( ) (53) he term f N e influences the conergence rate of the algorithm. It can be noticed that the fastest conergence mode is obtained when the function from Eq. (5) reaches its imum. e ( ) A D (5) A B F( ) 6( N ) 1 A D( ) ( ) For the NCLMA algorithm to be stable in the mean-square sense its behaior must decrease monotonically with an increase of. e ( ) 0 A D (55) A B F( ) ( N ) 1 A D( ) ( ) Assug that the NCLMA algorithm has conerged to the steady state 1 (56) he is obtained by substituting Eq. (56) into Eq. (51) as follows A( ) B( ) F( ) N N A( ) D 1 E A B F( ) 6( ) 1 A D e N A D ( ) Finally the MSE is gien by incorporating Eq. (57) into Eq. (3) A( ) B( ) F( ) N ( ) 1 E N A D MSE A B F( ) 6( ) 1 A D e N A D ( ) In addition we can now 0 or MSE when In the real practical world and are unnown howeer we can estimate e by using equation Eq. (59) [13][1]. 1 e = pp + (1 ) a 0. where p Ee is a small positie number to aoid that the denoator of Eq. (59) is infinite when 0. Although in Eq. (59) both p is also unnownwe can estimate this parameter by using Eq. (60). e (57) (58) and (59) p p (1 ) e (60)

11 where he alue ofe is listed in able. able. he alue of for different distributes E Gaussian[15] Uniform[15] Binary[15] Rayleigh[16] Eponential[17] E Finally the NCLMA algorithm consists of Eq.(9)-(11) Eq.(5) and Eq.(59)-(60).. Simulation results his section presents simulations in the contet of system identification with arious noises when the system is stationary or non-stationary to illustrate the accuracy of the NCLMA algorithm. he length of the unnown coefficient ector is ; the unnown coefficient is w opt. he uncorrelated input signal is a Gaussian white noise signal with ariance and the correlated input signal is calculated by y 0.8y. In all the eperiments the coefficient ectors are initialized as a zero 1 1 ector. 10 log10 opt w w is used to measure the performance of algorithms. Fig. 1 compares the analytical (from Eq. (57)) and eperimental results of the for the proposed NCLMA algorithm (β=0.001and γ=1000) for input with Uniform noise in stationary enironment with an SNR=0 db close agreement between theory and simulation is obtained as depicted in Fig. 1. he unnown coefficient is w opt =[ ]. he results are obtained by Monte Carlo simulations with 10 independent running. Iteration number is o understand clearly the improement brought about the NCLMA algorithm (β=0.001and γ=1000) when the input the time-arying step-size is depicted in Fig. for input with Uniform noise in stationary enironment with an SNR=0 db. he unnown coefficient is w opt =[ ]. he time-arying step-size of the NCLMA algorithm is shown in Fig.. Similarly the time ariation of of the NCLMA algorithm is depicted in Fig. 3. As obsered from this figure rises sharply and then decreases swiftly to reach zero. he results are obtained by Monte Carlo simulations with 10 independent running. Iteration number is Fig. displays the learning cures of the LMA (μ=0.01) NCLMA (β=0.001and γ=10000) and ZNCLMA (β=0.001and γ=1000) algorithms when the noise statistic is Eponential distributed signal with for an SNR=10dB with y input in nonstationary enironment. he results are obtained by Monte Carlo simulations with 30 independent running. Iteration number is he unnown coefficient is w w 1 υ where y w L opt opt w opt =[ ] υ is an i.i.d. Gaussian sequence with m 0 and Here too the ecellent performance of the proposed NCLMA algorithm is maintained and therefore a consistency in performance is achieed by the proposed NCLMA algorithm. Net the choice of β and γ can also affect the performance of the NCLMA

12 algorithm. Results in Fig. 5 for different alues of β and γ show clearly that een in etreme cases. he system noise is Uniform distributed oer the interal (-3 3). he unnown coefficient is w opt =[ ] with input. he results are obtained by Monte Carlo simulations with 10 independent running. Iteration number is he cures of the NCLMA algorithm (β=0.001and γ=5000) with SNR=0dB. he performance of the NCLMA algorithm did not deteriorate much. he best performance was achieed by increasing β to 0.01 and decreasing γ to 1000 for a constant steady-state misadjustment obtained by the NCLMA algorithm. Fig.6 summarizes the performance of the proposed NCLMA algorithm (β=0.001 and γ=1000) in the fourth different noise enironments with an SNR of 0dB with y input in nonstationary enironment. he results are obtained by Monte Carlo simulations with 30 independent running. Iteration number is he unnown coefficient is wopt wopt 1 υ where w opt =[ ] is an i.i.d. Gaussian sequence with m 0 and From this figure that the best performance is obtained when the noise statistic is Rayleigh while the worst performance is obtained when the noise statistic is Uniform. Moreoer to further test its performance Fig.7 display the learning cures of the NCLMF (α=0.001 β= and γ=500) and NCLMA algorithms (β=0.001and γ=10000) with SNR= 0dB when the noise statistic is Gaussian white noise with υ m 0 and 1. he unnown coefficient is w opt =[ ]. he correlated input signal was gien by y 0.8y 1. Fig.8 display the learning cures of the NCLMA algorithms (β=0.001and γ=0000) and ZNCLMA (β= and γ=000) algorithms when the noise statistics are Eponential distributed signal with. he unnown coefficient is w w 1 υ where w opt =[ ] υ opt opt is an i.i.d. Gaussian sequence with m 0 and he correlated input signal was gien by y 0.8y 1. he results are obtained by Monte Carlo simulations with 10 independent running. Iteration number is he unnown coefficient is w opt =[ ]. he correlated input signal was gien by y 0.8y 1. Here too the ecellent performance of the proposed NCLMA (β=0.001 and γ=000) algorithm is maintained and therefore a consistency in performance is achieed by the proposed NCLMA algorithm. Moreoer to further test its performance the noise statistics are changed to Rayleigh distributed signal with for an SNR=0 db and as depicted from Fig. 9 a similar behaior as the former ones is obtained by the proposed NCLMA algorithm in this enironment. he results are obtained by Monte Carlo simulations with 10 independent running. Iteration number is he tuning parameters for this part of the ZNCLMA algorithm are β=0.003 and γ=100.

13 0 10 NCLMA-simulation NCLMA-theory Fig.1. learning cure of the NCLMA algorithm NCLMA ime arying step-size Fig.. Behaior of time-arying step-size of NCLMA algorithm NCLMA 0.0 Behaior of λ Fig.3. Behaior of of NCLMA algorithm

14 LMA NCLMA ZNCLMA Fig.. Learning cures for the LMA NCLMA and ZNCLMA algorithms NCLMA β= γ=0000 NCLMA β= γ=10000 NCLMA β=0.001 γ=5000 NCLMA β=0.01 γ= Fig.5. he effect of β and γ on the conergence behaior of the NCLMA algorithm NCLMA->Uniform noise NCLMA->Epential noise NCLMA->Binary noise NCLMA->Rayleigh noise Fig.6. Learning cures for the NCLMA algorithm in the fourth different noise enironments

15 NCLMF NCLMA Fig.7. learning cures of the NCLMF and NCLMA algorithms NCLMA ZNCLMA Fig.8. Behaior of time-ary step-size of NCLMA and ZNCLMA algorithms NCLMA ZNCLMA Fig.9. learning cures of the NCLMF and NCLMA algorithms.

16 5. Conclusion In this paper we proposed a noise constrained LMA-type algorithm (NCLMA) for FIR channel estimation and studied its performance both analytically and simulations. he NCLMA algorithm is obtained by constraining the cost function of the standard LMA algorithm to the third-order moment of the additie noise. he main aim of this wor is to derie the NCLMA algorithm analyze optimized α and assess the performance in steady-state in different noise enironments when the system is stationary or non-stationary. Compare with the NCLMF and LMA algorithms the NCLMA enjoys a superior performance. he techniques introduced here may be applicable to plant identification. Acnowledgments his wor was supported by National Natural Science Foundation of China ( ) and the State Key Laboratory of Intelligent Control and Decision of Comple Systems of Beijing Institute of echnology. References [1] H. Sayed. Adaptie Filters [M]. Hoboen N USA: ohn Wiley & Sons Inc [] P.S.R. Diniz Adaptie Filtering (Fourth Edi.) [M]. Springer US 013 [3] Wei Y Gelfand S B Krogmeier V. Noise-constrained least mean squares algorithm []. IEEE rans. Signal Process. 9(9) (001) [] R. Kwong and E. W. ohnston. A ariable step size LMS algorithm []. IEEE rans. Signal Process. 0(7) (199) [5] Zhao H Yu Y Gao S Zeng X He Z. A new normalized LMA algorithm and its performance analysis []. Signal Process. 105(1) (01) [6] Zerguine A Moinuddin M Imam S A A. A noise constrained least mean fourth (NCLMF) adaptie algorithm []. Signal Process. 91(1) (011) [7] Eweda E. Dependence of the Stability of the Least-mean Fourth Algorithm on arget Weights Non-Stationarity []. IEEE rans. Signal Process. 6(7) (01) [8] Lee Y H in D M Sang D K Cho S H. Performance of least-mean absolute third (LMA) adaptie algorithm in arious noise enironments []. Electron. Lett. 3(3) (1998) 1-3. [9] Guan S Li Z. Nonparametric ariable step-size LMA algorithm []. Circ. Syst. & Signal Process. 016:1-18. [10] Cho S Sang D K. Adaptie filters based on the high order error statistics[c]// Circuits and Systems IEEE Asia Pacific Conference on (1996) [11] Hubscher P I Bermudez C M Nascimento V H. A Mean-Square Stability Analysis of the Least Mean Fourth Adaptie Algorithm []. IEEE rans. Signal Process. 55(8) (007) [1] Cho S H Mathews V. racing analysis of the sign algorithm in nonstationary enironments []. IEEE rans. Acoustics Speech & Signal Processing 38(1) (1990)

17 [13] H.-C. Huang. Lee. A new ariable step-size NLMS algorithm and its performance analysis. IEEE rans. Signal Process. 60() (01) [1] Han-Sol Lee Seong-Eun Kim ae-woo Lee and Woo-in Song. A Variable Step- Size Diffusion LMS Algorithm for Distributed Estimation []. IEEE rans. Signal Process. 63(7) (015) [15] Eweda E Bershad N. Stochastic Analysis of a Stable Normalized Least Mean Fourth Algorithm for Adaptie Noise Canceling With a White Gaussian Reference []. IEEE rans. Signal Process. 60(1) (01) [16] K. Hirano Rayleigh Distribution (Wiley London 01) [17] Spiegel M R. Mathematical handboo of formulas and tables [M]. McGraw-Hill 01.