Normalized Minimum Error Entropy Algorithm with Recursive Power Estimation

Transcription

1 entropy Article Normalized Minimum Error Entropy Algorithm with Recursive Power Estimation Namyong Kim * and Kihyeon Kwon Division of Electronic, Information and Communication Engineering, Kangwon National University, Samcheo 45-7, Korea; weon@angwon.ac.r * Correspondence: namyong@angwon.ac.r; Tel.: Academic Editor: J.A. Tenreiro Machado Received: 6 March 6; Accepted: June 6; Published: 4 June 6 Abstract: The minimum error entropy (MEE) algorithm is nown to be superior in signal processing applications under impulsive noise. In this paper, based on the analysis of behavior of the optimum weight and the properties of robustness against impulsive noise, a normalized version of the MEE algorithm is proposed. The step size of the MEE algorithm is normalized with the power of input entropy that is estimated recursively for reducing its computational complexity. The proposed algorithm yields lower minimum MSE (mean squared error) and faster convergence speed simultaneously than the original MEE algorithm does in the equalization simulation. On the condition of the same convergence speed, its performance enhancement in steady state MSE is above 3 db. Keywords: MEE; step-size; normalization; recursive; power estimation. Introduction Wired or wireless communication channels are under multipath fading as well as impulsive noise from various sources [,]. The impulsive noise can cause large instantaneous errors and system failure so that enhanced signal processing algorithms for coping with such obstacles are needed. Most algorithms are designed based on the mean squared error (MSE) criterion, but it often fails in impulsive noise environments [3].One of the cost functions based on information theoretic learning (ITL), minimum error entropy (MEE) has been developed by Erdogmus [4]. As a nonlinear version of MEE, the decision feedbac MEE (DF-MEE) algorithm has been nown to yield superior performance under severe channel distortions and impulsive noise environments [5]. It also has been shown for shallow underwater communication channels that the DF-MEE algorithm has not only robustness against impulsive noise and severe multipath fading but can also be more improved by some modification of the ernel size [6]. One of the problems of the MEE algorithm is its heavy computational complexity caused by the computation of double summations for the gradient estimation of MEE algorithm at each iteration time. In the wor conducted by [7], a computation reducing method by the recursive gradient estimation of the DF-MEE has been proposed for practical implementation. Though those practical difficulties have been removed through the recursive method, theoretic analysis in depth on its optimum solutions and their behavior has not been carried out yet for further enhancement of the algorithm. In this paper, based on the analysis of behavior of optimum weight and some factors on mitigation of influence from large errors due to impulsive noise, we propose to employ a time-varying step size through normalization by the input power that is recursively estimated for effectiveness in computational complexity. The performance comparison with MEE will be discussed and experimented through simulation in equalization as well as in system identification problems with impulsive noise that can be encountered in experiments investigating physical phenomenon [8]. Entropy 6, 8, 39; doi:.339/e8739

2 Entropy 6, 8, 39 of 4 problems with impulsive noise that can be encountered in experiments investigating physical phenomenon [8]. Entropy 6, 8, 39 of 3. MSE Criterion and Related Algorithms. MSE Criterion and Related Algorithms The overall communication system model for this wor is described in Figure. The transmitter The overall sends communication a symbol d at system time model through for this the wor multipath is described channel indescribed Figure. in The z-transform, transmitter sends a symbol d at time through the multipath channel described in z-transform, Hpzq ř h z i i i, Hz () hz, and then impulsive noise n is added to the channel output to become the received i and then impulsive noise n is added to the channel output to become the received signal x so that the signal adaptive x system so that input the x adaptive contains system noise n input and intersymbol x contains interference noise n and (ISI) intersymbol caused by the interference channel s (ISI) multipath caused [9]. by the channel s multipath [9]. Figure.. Overall communication system model. x x ÿ hd h n i d i `. n. () i i () With the input X With the input rx X [, x x,,..., x x,..., j,..., x,..., x L` ] T s T and weight W and weight rw [ w,, w, w,,.., w,.., w,..., j,,..., w w L, ] T s T j L,, j, L of, of the tapped delay line (TDL) equalizer, the output y and the error e become the tapped delay line (TDL) equalizer, the output y and the error e become y W T T X, () y WX, () e d y d W T T e d y d WX. X. (3) (3) With the current weight W, a set of error samples and a set of input samples, the adaptive algorithms With the designed current according weight W to, a set of error samples and a set of input samples, the adaptive their own criteria such as MSE or MEE produce updated weight algorithms W ` with which designed theaccording adaptive system to their maes own criteria the nextsuch output as MSE y `. or MEE produce updated weight W Taing with which statistical the adaptive averagesystem Er s tomaes the error the power next output e, the y MSE. criterion is defined as Ere s. For practical reasons, instantaneous error power e can be used and the LMS (least mean square) Taing statistical average E[] to the error power e, the MSE criterion is defined as Ee [ ].For practical reasons, instantaneous error power e can be used and the LMS (least mean square)

3 Entropy 6, 8, 39 3 of 3 algorithm has been developed based on minimization of e [9].The minimization of e can be carried out by the steepest descent method utilizing the gradient of e as Be BW e X. (4) With Equation (4) and the step size µ LMS, the well-nown LMS algorithm is presented as W ` W µ LMS Be BW W ` µ LMS e X. (5) By letting the gradient Be BW be zero, we have the optimum condition of the LMS as e X. (6) Taing statistical average Er s to Equation (6) leads us to the optimum condition of the MSE criterion as Ere X s. (7) Inserting (3) into (6), we get the optimum weight of the LMS algorithm, W opt LMS as W opt LMS X X T d X. (8) The optimum weight W opt LMS in (8)might be expected to get wildly shay in impulsive noise situations since it has no protection measures from such impulses existing in the input vector X. When the effect of fluctuations in the input power levels is considered, the fact that the step size µ LMS of the LMS algorithm should be inversely proportional to the power of the input signal X leads to the normalized LMS algorithm (NLMS) where its step size is normalized by the squared norm of the input vector X, that is, µ NLMS { X [9]. One of the principal characteristics of the NLMS algorithm is that the parameter µ NLMS is dimensionless, whereas µ LMS has the dimensioning of inverse power as mentioned above. Therefore, we may view that the NLMS algorithm has an input power-dependent adaptation step size, so that the effect of fluctuations in the power levels of the input signal is compensated at the adaptation level. When we assume that in the steady state e and X are independent, the input vector X can be viewed as being normalized by its squared norm X in the NLMS algorithm as W ` W ` µ NLMS e X { X. Unlie the LMS or NLMS, the MEE algorithm based on the error entropy criterion is nown for its robustness against impulsive noise [6]. In the following section, the MEE algorithm will be analyzed with respect to its weight behavior under impulsive noise environments. 3. MEE Algorithm and Magnitude Controlled Input Entropy The MSE criterion is effective under the assumptions of linearity and Gaussianity since it uses only second order statistics of the error signal. When the noise is impulsive, a criterion considering all the higher order statistics of the error signal would be more appropriate. Error entropy as a scalar quantity provides a measure of the average information contained in a given error distribution. With N samples (sample size N) of error samples te, e,..., e N` u the distribution function of error, f E peq can be constructed based on Kernel density estimation as in Equation (9) []. f E peq G σ pe e N i q N i N` i N` σ? π expr pe e iq σ s. (9)

4 Entropy 6, 8, 39 4 of 3 Since the Shannon s entropy in [9] is hard to estimate and to minimize due to the integral of the logarithm of a given distribution function, Renyi s quadratic error entropy Hpeq has been effectively used in ITL methods as described in (). ż Hpeq logp f E peq deq. () When error entropy Hpeq in () is minimized, the error distribution f E peq of an adaptive system is contracted and all higher order moments are minimized [4]. Inserting (9) into () leads to the following Hpeq that can be interpreted as interactions among pairs of error samples where error samples act as physical particles.» Hpeq log N i N` j N` fi G? σ pe j e i qfl. () Since the Gaussian ernel G σ? pe j e i q is always positive and is an exponential decay function with the distance square, the Gaussian ernel may be considered to create a potential field. The sum of all pairs of interactions in the argument of log [.] in () is called information potential IP e [4]. IP e N i N` j N` G σ? pe j e i q. () Then, minimization of error entropy is equivalent to maximization of IP e. For the maximization of IP e, the gradient of () becomes BIP e BW σ N i N` j N` At the optimum state (BIP e {BW ), we have i N` j N` pe j e i q G σ? pe j e i q px j X i q. (3) pe j e i q G σ? pe j e i q px j X i q. (4) Since the term pe j e q implies how far the current error e is located from each error sample e j, we may define the error pair pe j e q as e j, which is generated from the error space E at each iteration time as in Figure. The term e j, can be considered to contain information of the extent of spread of error samples. Considering that entropy is a measure of how evenly energy is distributed or the range of positions of components of a system, we will refer to this information as error entropy (EE) in this paper for convenience. Similarly, the term px j X q indicates the distance between the current input vector X and another input vector X j in the input vector space. Therefore, with the following definition, we can say that X j, contains the information of the extent of spread of input vectors, that is, input entropy (IE). Liewise, we will refer to X j, as an IE vector in this paper. X j, px j X q. (5) Then, with the EE sample e j, and IE vector X j, Equation (4) can be rewritten as N i N` r j N` e j,i G σ? pe j,i qx j,i s. (6)

5 Entropy 6, 8, 39 5 of 3 Entropy 6, 8, 39 5 of 4 Figure Figure.. Error Error space space E and error entropy samples generated from error pairs. Similarly, the term ( X X ) indicates the distance between ř the current input vector X If we consider the sample-averaged and j operation N p q in (6) can be replaced with the another input vector X in the input vector space. Therefore, i N` with the following definition, we can j statistical average Er s or vice versa for practical reasons, the comparison between (6) and the say optimum that X condition contains information of the extent of spread of input vectors, that is, input entropy j, of the MSE criterion Ere X s in (7) provides insight that e of the MSE criterion (IE). can correspond Liewise, we towill EE sample refer to ex j,, and as an X IE vector in this paper. j, of MSE criterion can be related to G? σ pe j, qx j, as a ind of modified input entropy vector. We also see that the term G? σ pe j, qx j, in (6) implies that the magnitude of X j, is controlled by G? σ pe j, Xq. At ( Xthe occurrence X ). of a strong impulse in n j, j,e can (5) be located far away from e j so that the EE sample e j, has a very large value. Then, the Gaussian function output Then, G? with the EE sample e and IE vector X Equation (4) can be rewritten as σ pe j, q becomes a veryj, small value since its j, exponential is a decay function of e j,. In turn, the value of the IE vector X j, is reduced by the multiplication of G? σ pe j, q. In this regard, it is appropriate that the term G? σ pe j, qx [ e G ( e ) ] j, i σ j, i j, i N i j, in (6)is N j interpretedx as a=. magnitude-controlled version(6) of... N Defining Xj, as a magnitude controlled input entropy () in (7), this process can be described as in Figure 3. Entropy If 6, we 8, consider 39 the sample-averaged X operation () N i in (6) can be replaced with the j, G? σ pe j, qx j,. 6 of (7) 4 N statistical average E[] or vice versa for practical reasons, the comparison between (6) and the optimum condition of the MSE criterion Ee [ X ] in (7) provides insight that e of the MSE criterion can correspond to EE sample e j,, and X of MSE criterion can be related to G ( e ) X σ j, j, as a ind of modified input entropy vector. We also see that the term G ( e ) X in σ j, j, (6) implies that the magnitude of X is controlled by G ( e ). At the occurrence of a strong j, σ j, impulse in n, e can be located far away from e j so that the EE sample e has a very large j, value. Then, the Gaussian function output G ( e ) becomes a very small value since its σ j, exponential is a decay function of e. In turn, the value of the IE vector X is reduced by the j, j, multiplication of ( ) Figure 3. Magnitude controller for IE. G e. In this Figure regard, 3. Magnitude is appropriate controller for that IE. the term G ( e ) X in (6)is σ j, σ j, j, interpreted In an element as a magnitude-controlled expression, version of... Defining X as a magnitude controlled j, input entropy () in (7), this process can be described as in Figure 3. x x G ( e ) x G ( e )( x x ). j, j, G? σ σ pe j, j, qxj j,, σg? σ j, pe j, j qpx j x q. (8) (8) X G ( e ) X. j, σ j, j, (7) With X j,, the MEE algorithm becomes j, μ W W X. (9) W ` W ` µmee e j,i Xj,i. (9) MEE e ji, ji, σ N i N j N σ N i N` j N` The optimum condition in (6) can be rewritten as e X j, j,. () j N E[ ] We may observe that the MEE algorithm in () is very similar to (7) in the aspect of the error

6 Entropy 6, 8, 39 6 of 3 The optimum condition in (6) can be rewritten as Er j N` e j, X j, s. () We may observe that the MEE algorithm in () is very similar to (7) in the aspect of the error and input terms. One different part is that the MEE algorithm consists of summations of error entropy samples and input entropy vectors, while the LMS just has an error sample and an input vector. On the other hand, it can be noticed that Xj,i can eep the algorithm stable even at the occurrences of large error entropy that occurs mostly when the input is contaminated by impulse noise. The summation process over e j,i Xj,i can also mitigate the influence of impulses, but it does not contribute much to deterring the influence of large errors since even an impulse can dominate the averaging (summation) operation. 4. Recursive Power Estimation of The fixed step size of the MEE algorithm may mae the MEE require an understanding of the statistics of the input entropy prior to the adaptive filtering operation. This maes it hard in practice to choose an appropriate step size µ MEE that controls its learning speed and stability. Lie the approach of the normalized LMS that solves this ind of problem through normalization by the summed power of the current input samples as in [9,], we propose heuristically to normalize the step size by the summed power of the current element in (8) as µ NMEE ř µ xj, j N`. () Considering the fact that impulses can defeat the average operation as explained in Section 3, we can notice that the denominator may become large in an incident with impulsive noise; in turn, µ NMEE becomes a very small value, so that it may induce a very slow convergence. To avoid this ind of situation, we may adopt a sliding window as µ NMEE ř µ ř N xj,i i N` j N`. () However, this approach places a heavier computational complexity on the MEE algorithm. For reducing the burdensome computations, we need to trac the power recursively using a single-pole low-pass filter, i.e., Ppq βpp q ` p βq x, (3) j N` where β p ă β ă q controls the bandwidth and time constant of the system whose transfer function ř Tpzq with its input is given by xj, j N` j, z Tpzq p βq z β. (4) Then, the resulting algorithm that we will refer to in this paper as normalized MEE (NMEE) becomes µ W ` W ` Ppqσ N e j,i Xj,i. (5) i N` j N`

7 Entropy 6, 8, 39 7 of 3 On the other hand, the NLMS in (9) has been developed based on the principle of minimum disturbance that states the tap weight change of an adaptive filter from one iteration to the next, that is, the squared Euclidean norm (SEN) of the change in the tap-weight vector, W ` W should be minimal [9]. From that perspective, the effectiveness of the proposed NMEE algorithm can be analyzed based on the disturbance, SEN, at around the optimum state as SEN W ` W. (6) For the existing MEE algorithm of (9), σ N µmee SEN MEE µmee ř σ N ř ř ř i N` j N` i N` j N`l N` m N` ř ř e j,i X j,i e j,i e m,l Xj,i T Xm,l. (7) For the proposed NMEE algorithm, Then SEN NMEE ˆ µ SEN NMEE σ N Ppq ˆ ˆ µ σ N Ppq i N` j N` e j,i X j,i i N` j N`l N` m N` Comparison of SEN MEE in (7) and SEN NMEE in (9) leads to. (8) e j,i e m,l Xj,i T Xm,l. (9) SEN NMEE ą SEN MEE for Ppq ă µ µ MEE, (3) SEN NMEE SEN MEE for Ppq µ µ MEE, (3) SEN NMEE ă SEN MEE for Ppq ą µ µ MEE. (3) This result indicates that the proposed method is more suitable for the conventional MEE when the power is greater than µ{µ MEE, which means when a smaller µ MEE is demanded, such as when the input signal is contaminated with strong impulsive noise. On the other hand, it can be noticed that when the input signal is not large so that a bigger µ MEE can be employed for faster convergence, the proposed method may not be guaranteed to be better than the fixed step size MEE algorithm. On the other hand, we now there are a lot of step size selection methods for gradient-based algorithms, and we need to verify that this approach is the right one for the MEE problems. Considering that the proposed step size selection method is motivated and designed by the concept of the input power normalization as in the NLMS algorithm, it may be reasonable to investigate whether the input power normalization is effective in the MEE algorithm under impulsive noise. When we employ the squared norm of the input vector X ř, that is, x i in the MEE i N` algorithm (we will refer to this as NMEE for convenience), the squared Euclidean norm becomes

8 Entropy 6, 8, 39 8 of 3 µ ř ř SEN NMEE ř σ N x i i N` j N` i N` ř ř ř ř µ σ N e j,i X j,i e j,i e m,l Xj,i T Xm,l i N` j N`l N` m N`. ř x i i N` (33) Assuming the error entropy e j,i and X j,i µ SEN NMEE σ N i N` j N`l N` m N` are independent in the steady sate, (33) becomes e j,i e m,l X T j,i ř p N` xp X m,l ř q N` xq The SEN in (9) adopting the squared norm of the instead of X can be rewritten as µ SEN NMEE σ N i N` j N`l N` m N` Xj,i T e j,i e m,l Ppq. (34) X m,l Ppq. (35) Comparing the two SENs, (34) and (35), the in SEN NMEE is normalized by power ř Ppq, whereas the in SEN NMEE is normalized by the simply summed input power x p. This indicates that SEN NMEE might vary to some degree since the denominator p N` ř p N` xp containing impulsive noise can fluctuate from small values to large values due to strong impulses dominating the sum operation. From this analysis, the fact that the in SEN NMEE is normalized by power Ppq that uses the output of the magnitude controller cutting the outliers from strong impulses leads us to the argument that our proposed method is appropriate for impulsive noise situations. This will be tested in Section 5. As observed in (3), when the input signal is not in strong impulsive noise environments, the proposed method may not be better than the existing MEE algorithm. The effectiveness of the proposed NMEE algorithm under strong impulsive noise will be investigated in the following section. 5. Results and Discussion The simulation for observations of the optimum weight behavior of MEE algorithm is carried out in equalization of the multipath channel of Hpzq.6 `.93z `.6z []. The transmitted symbol d sent at time is randomly chosen from the symbol set td 3, d, d 3, d 4 3u(M 4). The impulsive noise n in () consists of the bacground white Gaussian noise (BWGN) and impulses (IM) with variance σbwgn and σ IN, respectively. The impulses are generated according to a Poisson process with its incident rate ε []. The distribution f N pn q of the impulses is ε f N pn q? expr σ BWGN π n σ BWGN n ε s ` expr bπpσ BWGN ` σ IN q pσ BWGN ` (36) σ INqs. The BWGN with σbwgn =. is added throughout the whole time to the channel output. The impulses are generated with variance σin = 5. The TDL equalizer has tap weights pl q. For the parameters for the MEE algorithm, the sample size N, the ernel size σ and convergence

9 Entropy 6, 8, 39 9 of 3 parameter µ MEE are,.7 and., respectively. The step size µ LMS for the LMS algorithm is.. All parameter values are selected when they produce the lowest minimum MSE in this simulation. Firstly, the weight traces will be investigated through simulation in order to verify the property of robustness against impulsive noise. The impulses are generated with ε =. for clear observation of the weight behavior. The impulse noise as depicted in Figure 4 is applied to the channel output in the Entropy 6, 8, 39 of 4 steady state, that is, after convergence. 5 Entropy 6, 8, 39 of 4-5 Volt - Volt Number of samples Figure 4. The impulse and bacground noise for the simulation for the behavior of optimum weight. Figure 4. The impulse and bacground noise for the simulation for the behavior of optimum weight. Figure 5 shows the learning curves of weight w 4, and w 5, (only two weights are chosen due to the -.6 page limitation). At around 5 samples, both reach their optima completely, and then they undergo the impulsive noise lie that in Figure 4. In Figure 5, it is observed that MEE and LMS have the same steady state weight values and each weight trace of MEE in the steady state shows no fluctuations remaining undisturbed under the strong impulses. This is obviously in contrast to the case of the LMS algorithm where traces of w 4, and w 5, have sharp perturbations at impulse occurrences and remain Figure 4. The impulse and bacground noise for the simulation for the behavior of optimum weight Number of samples. w4: LMS.8 w4: MEE w5: LMS perturbed for a long time.6 though graduallyw5: dying. MEE weight value weight value w4: LMS w4: MEE w5: LMS.6 w5: 4 MEE Number of samples (iterations). Figure 5. The behavior. of weight values of w 4, and w 5, with impulsive noise being added in the steady state We can notice that the -.6 optimum weight of MEE has averaging operations and in (3) has some differences when the weight update Equation 4 (8) 6 is compared. 8 Since the average operations can easily be defeated even by just one strong Number impulse, of samples we (iterations) can figure out that the dominant role of robustness against impulsive noise is the. Figure Secondly, 5. The the behavior effectiveness of weight of the values proposed of w Figure 5. The behavior of weight values of 4, and NMEE w algorithm (5) designed with the is 4, and w 5, with impulsive noise being added in the 5, with impulsive noise being added in the steady state. investigated through the learning performance comparison with the original MEE algorithm in (9) steady state. under the same impulsive noise with σ IN = 5 and ε =.3 as in the wor [5] in which the impulsive We can noise notice is used that in the all optimum time. The weight MSE learning of MEE has results averaging are shown operations in Figure and 6. in (3) has some The differences LMS algorithm when the converges weight very update slow Equation and stays (8) at is about compared. 8 db Since of MSE the in average the steady operations state. opt This can easily result be can defeated be explained even by from just the one expression strong impulse, of W we can LMS in (8) figure having out no that measures the dominant to protect role of it robustness against impulsive noise is the. from fluctuations from impulsive noise as discussed in Section 3. On the other hand, the MEE Secondly, the effectiveness of the proposed NMEE algorithm (5) designed with the is

10 Entropy 6, 8, 39 of 3 We can notice that the optimum weight of MEE has averaging operations and in (3) has some differences when the weight update Equation (8) is compared. Since the average operations can easily be defeated even by just one strong impulse, we can figure out that the dominant role of robustness against impulsive noise is the. Entropy Secondly, 6, 8, 39 the effectiveness of the proposed NMEE algorithm (5) designed with the of 4 is investigated through the learning performance comparison with the original MEE algorithm in (9) under algorithm the same (9) and impulsive its steady noise state with weight σin = undisturbed 5 and ε =.3 by as large in the error wor values [5] in that which may the be impulsive induced noise from excessive is used in noise all time. such The as MSE impulses. learning results are shown in Figure 6. log of MSE Number of samples (iterations) LMS. MEE. MEE. NMEE.6 Figure The Comparison of of MSE learning curves under impulsive noise (square: LMS with µ μ LMS =..,, circle: MEE MEEwith with μµ MEE. =.,, triangle: triangle: MEE MEE with with μ µ MEE. =.,, thic thic line: line: NMEE NMEE with LMS MEE MEE with β =.9 and µ = 6µ β.9 and μ 6μ MEE ). MEE ). The As for LMS the algorithm performance converges comparison very between slow andmee staysand at about NMEE 8 in db Figure of MSE 6, NMEE in theshows steadylower state. This minimum result MSE can beand explained faster convergence from the expression speed simultaneously. of W opt LMS in (8) having The difference no measures of convergence to protect itspeed from fluctuations is about 5 samples from impulsive and that noise of minimum as discussed MSE in is Section around 3. db. On When the other compared hand, the to the MEE condition algorithmof rigged the same with convergence the magnitude speed, controller the difference for IE converges in minimum about MSE is shown samplesto even be about under3 the db. strong This impulsive amount of noise. performance This result gap supports indicates thethat analysis the proposed that the method Xj, of tracing eeps the the algorithm power of (9) and its recursively steady state and weight using it undisturbed in normalization by large of error the step values size that is significantly may be induced effective fromin excessive the aspect noise of such performance as impulses. as well as computational complexity. As In Figure for the performance 7, the comparison power becomes between large MEE as and the NMEE algorithm in Figure 6, converges, NMEE shows and lower after minimum convergence, MSEthe andtrace faster shows convergence large variations, speed simultaneously. mostly above The6. difference The condition of convergence P ( ) 6speed in this about 5 samples that of minimum MSE is around db. When compared to the condition of the simulation implies that when NMEE is employed, the value μ according to (3) must be greater same convergence speed, the difference in minimum MSE is shown to be about 3 db. This amount of than 6μ performance MEE gap for better indicates performance. that the proposed The fact method that this of tracing is exactly the power in accordance of with recursively the choice and using μ 6μ MEE it indescribed normalization in Figure of the6 step justifies sizethe is significantly effectiveness effective of the proposed in the aspect method of performance by simulation. as well as computational complexity. In Figure 7, the power becomes large as the MEE algorithm converges, and after 45 convergence, the trace shows large variations, mostly above 6. The condition Ppq ą 6 in this 4 simulation implies that when NMEE is employed, the value µ according to (3) must be greater than 6µ 35 MEE for better performance. The fact that this is exactly in accordance with the choice µ 6µ MEE described in Figure 6 justifies the effectiveness of the proposed method by simulation. MICE power value Number of samples (iterations)

11 convergence, the trace shows large variations, mostly above 6. The condition P ( ) 6 in this simulation implies that when NMEE is employed, the value μ according to (3) must be greater than 6μ MEE for better performance. The fact that this is exactly in accordance with the choice Entropy μ μ 6, 8, 39 of 3 described in Figure 6 justifies the effectiveness of the proposed method by simulation. 6 MEE MICE power value Number of samples (iterations) Figure The The trace trace of of power power P() of P ( the ) MEE of the algorithm MEE algorithm under theunder same simulation the same simulation conditions Entropy 6, 8, 39 of 4 used for Figure 6. conditions used for Figure 6. In the same simulation environment, the MSE learning curves for two input power In the same simulation environment, the MSE learning curves for two input power normalization normalization approaches, NMEE and NMEE are compared in Figure 8. approaches, NMEE and NMEE are compared in Figure 8. log of MSE NMEE NMEE Number of samples (iterations) Figure 8. Learning curves of of NMEE and NMEE algorithms for for comparison of of the two methods of input input power normalization. As observed in in Figure 8, the 8, the input input power power normalization normalization approach approach for variable for variable step sizestep selection size selection for the MEE for the algorithm MEE algorithm shows different shows MSE different performances MSE performances according according to which to signal which power signal is power normalized. is normalized. When NMEE When isnmee employed is employed where the where magnitude the magnitude controls controls input entropy, input entropy, is used is forused power for normalization, power normalization, the MSE the learning MSE performance learning performance yields better yields steady better statesteady MSE of state above MSE db of above and faster db convergence and faster convergence speed by about speed by about samples than samples when than NMEE when is adopted, NMEE is inadopted, which the in squared norm of the unprocessed input X which the squared norm of the unprocessed is used for normalization. As discussed in Section 4, input X is used for normalization. As discussed in under strong impulsive noise, the power of can be the right choice for step size normalization Section for better 4, performance. under strong impulsive noise, the power of can be the right choice for step size normalization In system for identification better performance. applications of adaptive filtering as appeared in the wor [8], the desired signal In issystem derivedidentification by passing the applications white Gaussian of adaptive input through filtering the as unnown appeared system. in the The wor unnown [8], the desired system in signal this simulation is derived by is of passing lengththe 9. The white impulse Gaussian response input of through the unnown the unnown systemsystem. is chosenthe to unnown follow a triangular system in wave this form simulation that is is symmetric of length with 9. The respect impulse to the response centralof tap the point unnown [9,3]. system The TDL is chosen filter has to 9follow tap weights. a triangular The input wave signal form is that a white is symmetric Gaussianwith process respect withto zero the mean central and tap unit point variance. [9,3]. The same TDL filter impulsive has 9 noise tap weights. used in The Figure input 6, uncorrelated signal is a white with Gaussian the input, process is addedwith to the zero output mean ofand the unit unnown variance. system. The MSE same learning impulsive curves noise areused depicted in Figure in Figure 6, uncorrelated 9. with the input, is added to the output of the unnown system. MSE learning curves are depicted in Figure System identification MEE

12 unnown system in this simulation is of length 9. The impulse response of the unnown system is chosen to follow a triangular wave form that is symmetric with respect to the central tap point [9,3]. The TDL filter has 9 tap weights. The input signal is a white Gaussian process with zero mean and unit variance. The same impulsive noise used in Figure 6, uncorrelated with the input, is added to Entropy 6, 8, 39 of 3 the output of the unnown system. MSE learning curves are depicted in Figure System identification MEE NMEE log of MSE Number of samples (iterations) Figure Learning curves of of MEE and and NMEE for for system system identification. One can observe from Figure 8 that the proposed NMEE achieves lower steady-state MSE than the conventional MEE algorithm in the system identification problems as well. 6. Conclusions The MEE algorithm is nown to outperform MSE-based algorithms in most signal processing applications in an impulsive noise environment. The conventional MEE algorithm has a fixed step size so that it may require in practice to employ a time varying step size that appropriately controls its learning performance. Based on the analysis of the behavior of optimum weight and the role of in mitigation of influence from large error, it was found in this paper that the NMEE employing the step size normalized with the power of the current element can yield lower minimum MSE and faster convergence speed simultaneously. On the condition of the same convergence speed, the performance enhancement of 3 db in the equalization simulation leads us to conclude that the proposed method of recursive estimation of the power for normalization of the step size is significantly effective in both aspects of performance and computational complexity. Acnowledgments: This paper has been improved significantly by comments from three anonymous reviewers. We deeply appreciate it. Author Contributions: Namyong Kim derived the algorithm and finished the draft. And Kihyeon Kwon conducted the simulations, polished the language, and was in charge of technical checing. All authors have read and approved the final manuscript. Conflicts of Interest: The authors declare no conflict of interest. References. Kim, N.; Byun, H.; You, Y.; Kwon, K. Blind signal processing for impulsive noise channels. J. Commu. Netw., 4, [CrossRef]. Armstrong, J.; Shentu, J.; Chai, C.; Suraweera, H. Analysis of impulse noise mitigation techniques for digital television systems. In Proceedings of the 8th International OFDM-Worshop, 3, Hamburg, Germany, 4 5 September 3; pp Santamaria, I.; Poharel, P.; Principe, J. Generalized correlation function: Definition, properties, and application to blind equalization. IEEE Trans. Signal Process. 6, 54, [CrossRef] 4. Erdogmus, D.; Principe, J. An entropy minimization algorithm for supervised training of nonlinear systems. IEEE Trans. Signal Process., 5, [CrossRef] 5. Kim, N. Decision feedbac equalizer algorithms based on error entropy criterion. J. Internet Comput. Serv.,, 7 33.

13 Entropy 6, 8, 39 3 of 3 6. Kim, N. Performance analysis of entropy-based decision feedbac algorithms in wireless shallow-water communications. In Proceedings of KSII summer conference, Pyeongchang, Korea, 7 9 June ; pp Kim, N.; Andonova, A. Computationally efficient methods for decision feedbac algorithms based on minimum error entropy. Available online: Andonova.pdf (accessed on 3 June 6). 8. Wu, Z.; Peng, S.; Chen, B.; Zhao, H.; Principe, J. Proportionate minimum error entropy algorithm for sparse system identification. Entropy 5, 7, [CrossRef] 9. Hayin, S. Adaptive Filter Theory, 4th ed.; Prentice Hall: Upper Saddle River, NJ, USA,.. Parzen, E. On the estimation of a probability density function and the mode. Ann. Math. Stat. 96, 33, [CrossRef]. Chinaboina, R.; Ramiran, D.; Khan, H.; Usha, M.; Madhav, B.; Srinivas, K.; Ganesh, G. Adaptive algorithms for acoustic echo cancellation in speech processing. Int. J. Res. Rev. Appl. Sci., 7, Proais, J. Digital Communications, nd ed.; McGraw-Hill: New Yor, NY, USA, Kim, N. A least squares approach to escalator algorithms for adaptive filtering. J. Commu. Netw. 6, 8, [CrossRef] 6 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC-BY) license (