ANALYSIS OF STEADY-STATE EXCESS MEAN-SQUARE-ERROR OF THE LEAST MEAN KURTOSIS ADAPTIVE ALGORITHM

Transcription

1 ANALYSIS OF SEADY-SAE EXCESS MEAN-SQUARE-ERROR OF HE LEAS MEAN KUROSIS ADAPIVE ALGORIHM Junibakti Sanubari Department of Electronics Engineering, Satya Wacana University Jalan Diponegoro 5 60, Salatiga, Indonesia Phone: + (6) Fax: + (6) jsawitri@hotmail.com Web: ABSRAC In this paper, thverage of the steady state excess mean square error (ASEMSE) of the least mean kurtosis () adaptivlgorithm is theoretically derived. It is done by applying the energy conservation behavior of adaptive filters it is based on the n-th order correlations cumulants theory. By doing so, the behavior of the recently proposed can be predicted, so that it can be widely used. he behavior is compared with the various adaptivlgorithms. Our study shows that it is possible to adjust the performance of the. When the step size µ is carefully selected, the performance of the can outperform the. 1. INRODUCION raditionally, the coefficient of thdaptive filter is adjusted using the least mean square () algorithm. he algorithm is largely used in real-timdaptive filtering applications for communications, control engineering, processing of biological signals more recently active noise vibration controls [1]. It is very popular due to its simplicity. In the, the coefficient of thdaptive filter is adjusted so that the square of the error e between the output of thdaptive filter y the desired signal d is minimal[1]; minimizing n e. In applications, the input signal of thdaptive filter x is contaminated with additive noise v. By minimizing n e, we minimize the power of the noise; σv. No special consideration is taken regarding the detail statistical behavior of the noise. All parts of input signal are given an equal weighting. he obtained solution is more determined by largmplitude noise portion than that by small amplitude noise portion. he obtained solution is very much affected by largmplitude noise portion. Further improvements is hoped to be obtained when the statistical behavior of the noise is considered. hbove mentioned problems are encountered by assuming that remaining error e is Gaussian. Remembering the fact that the signal e is a summation of many stochastic processes the central limit theory [], assuming that e is Gaussian is reasonable. Since many stochastic processes with different probability density function (PDF) can have equal mean or variance no stochastic processes can have normalized kurtosis E e 4 γ 4 e E e 3 (1) except Gaussian process [3], in this paper we evaluate γe 4 to test the Gaussianity of e. We define the fourth-order cumulants of the error as E e 4 its variancs σ e. In this paper we study an adaptivlgorithm where is adjusted by forcing γe 4 3; which is named least mean kurtosis () [8]. hus, in algorithm, the coefficient of thdaptive filter is solved by forcing the error signal to be Gaussian. Many adaptivlgorithms that are based on higher order moments of the error signal due exist [4] [8] is one of such algorithm [8]. hoslgorithms have been shown to outperform in some important applications. he practical use of such algorithms, however, has been largely restricted due to lack of accuratnalytical models to predict their behavior. Since sompproximations have also been introduced to reduce the calculation complexity of the, it is interesting to study its analytic behavior to further open the possiblpplications of thlgorithm. he model of the behavior of the adaptivlgorithm with Gaussian input has been studied [9]. he behavior is studied for zero-mean contaminated input signals. Since the probability density function (PDF) of the noise is unknown, in this paper thverage of the steady-state excess mean square error (ASEMSE) of algorithm is studied for both the symmetrical the non-symmetrical PDF noises.. HE BASIC ADAPIVE SYSEM Consider a noisy measurements d that arise from the linear model d desired unknown pa- where the input data vector rameter vector 0 are 0 v () x x n 1 x n M 0 w 0 1 w 0 w 0 M (3) (4) respectively. he superscript denotes the vector or matrix transpose. v accounts for the noise signal the order of the system is M. Both v x are stochastic in nature. he so-called estimation error is defined as e d (5) where is the estimate of 0 at iteration n. Since (5) shows that indeed the error signal e is resulted from summing many stochastic process; d x n l ; for 0 l M, it is reasonable to assume that e is Gaussian. herefore, it is expected to gain improvements when the optimal solution is calculated by forcing γe 4 3.

2 By defining we can obtain e v 0 (6) (7) he estimation of 0 at time index n 1 is recursively determined using the gradient method [1] n 1 µ a J e n (8) he symbol indicates the first derivatives; i.e. f x f x is also called the error signal due to parameter mismatch the convergence constant is denoted as µ a. x. From the definition of q, 3. HE LEAS MEAN KUROSIS CRIERION Instead of using the definition of the normalized kurtosis of a stochastic in (1), this paper uses J e 3E e E e 4 (9) as the objective function to be minimized to obtain the optimal solution. It is selected, so that no unknown parameters are in the denominator. Sincll e has to bvailable to evaluate E e E e 4 it is impossible in adaptivlgorithm, in this paper the short average is used as an approximation E e 3! e 3 (10) E e! e βe n 1 β e n (11) Substituting thpproximation in (10) into (8), the updating is done using n 1 µ κ e (1) where µ 4µ a κ e e 3βe n 1 3β e n (13) or by using (7), we get where κ e v α 1 α α 3 " (14) α 1 e a 3βe a n 1 3β e a n α v 3βv n 1 3β v n α 3 4 v 6βv n 1 n 1 6β n v n (15) he selection of the positive constant β is not critical will be discussed later. he theoretical performance of the is evaluated in the following section. 4. HE SEADY SAE PERFORMANCE OF HE LEAS MEAN KUROSIS ALGORIHM One of the important performance of thdaptivlgorithm is the steady-state of the remaining error; lim n# q. Since the input is stochastic processes, we evaluate lim n# E q for all possible inputs. he evaluation is easily done by evaluating thverage of the steady-state excess mean-square-error (ASEMSE) which is defined as ASEMSE n# lim E $&% e '% ( (16) Under the often realistic assumption that [1]: 1. he noise sequence v is identical independent distributed (IID).. he noise v the input sequences x are independent each other. Utilizing those two assumptions (7) assuming that the noise is stationary the ASEMSE is given by: ASEMSE σv n# lim E (17) where σv is the variance of the noise sequence is defined in (7). 3. Generally, the PDF of the noise is unknown. In this paper, wssumed that the PDF of the noise v can be non-symmetrical; f v *) f v ; or symmetrical distributed; f v f v. 4. he PDF of the input signals x is IID. Utilizing the definition of in (6), (1) can be rewritten to be n 1 µ κ e (18) that can be taken its L norm. After somlgebraic manipulations, the relation in (19) can be obtained. %+% n 1 '%,% %,% -%+% µ κ e µ κ e he second term of (17) is solved by calculating E./%,% n 1 -%+% 0 When n E.1%+% '%,% 0 µ E κ e µ E. κ e, a steady state has been reached, so that (19) 0 (0) E 3%+% n 1 '%,% E 3%+% -%+% ; (1) showing that no further improvement can be gained. herefore, from (0) we get E 4 % '% X n 5 E 6 % µ κ e X n % X n 7 () where X n 8 tr R xx. he symbol tr indicates the trace of thutocorrelation matrix of the input signal x. Exping the right h side of (), we obtain µ E κ e µ E $ κ e X n ( (3)

3 KI > O I S Y Using (7), (14) the definition of α 1, α α 3 in (15), we can obtain κ e " v α 1 α α 3 9 (4) On the other h from the definition of κ e in (14) we can obtain κ e v α 1 : α α 3 (5) Using the facts that: 1. M is large. hus, based on the central limit theory [] (7), can be safely assumed to be zero mean Gaussian unity stard deviation (SD). hus all its odd moment its cumulant higher than two vanish.. v are independent 3. he number of samples is large; so that we can have E. v 0 E. v n 1 0 E. v n 0 σv E. e a 0 E. e a n 1 0 E. e a n 0 0 E. e a we will obtain E where κ e E. e 4 a 0 1E. v 0 E. e a 0 (6) E $ κ e ( 4E. e 6 a 0 γ 1 E. v 4 0 E. e a 0; γ 1 E. e 4 a 0 E. v 0 4E. v 6 0 γ E. v v n 1 0 E. e a n 1 0 γ 3 E. e a n 0 E. v v n 09 9β E. v v 4 n 1 0; 9β 4 E. v v 4 n 0 (7) γ β 1β 4 γ 60β 54β 18β 3 γ 3 60β 18β 3 54β 4 (8) Using the moment theorem [] that stated E x 4 3E x E x 6 15E x the cumulants theory [3], we get E κ e 6E. e a 0 1σv E. e a 0 (9) E $ κ e ( 60E. e a 0 6γ 1 σv E. e a 0 60σv γ m v E. e a 0 γ 3 E. e a 0 m v 4 0 9β m v (30) 9β 4 m v 6 where the N-th order moment of the noise signal v is defined as N@ 1A terms B C D E J N@ 1A terms B C D E m v N <=?> s q FG E HI v v n s v n q "ML IN (31) Z\[^],_ db WXS PRS P WXS PUVS PRQS π `bacd [^e,],f'g-hi c g-jlkmg'_/no rad Figure 1: he frequency response of the 0 th order system that was used in the simulation to obtain ASEMSE. Substituting (9) (30) into (17), we can obtaine ASEMSE as given in (3). Since the numerator of (3) is always positive, we have to limit µ as in (34) to be sure that the denominator of (3) is positive. By doing so we can be sure that ASEMSE in (3) is positive valid. his is consistent with the limitation of the step size µ that will be explained in the next section. Equation (3) shows that the selection of µ determine the resulted ASEMSE. ASEMSE was experimentally solved from (17) by generating contaminated signals to be applied as x. hose contaminated signals were generated by adding two signals. he first signal was generated by exciting an unknown model with a zero-mean stochastic process g the second signal was the noise v. he power of g was fixed to be 1.0 the power of v were adjusted to obtain Sp N 5 db in all experiments. o get unbiased modeling, the order of the unknown thdaptive systems were set to be 0. he frequency response of the 0 th order system that was used in experiment is shown in Fig. 1. We evaluated the figure of merit ρ using ρ ASEMSE ASEMSE (3) he ASEMSE is the ensemblverage of the steady state excess mean square error of the stard algorithm [10] ASEMSE is evaluated from (3). he value of ASEMSE ASEMSE were obtained from averaging 00 times independent experiments. We set various k µ µ µ (33) ρ q 1 means that ASEMSE r ASEMSE. On the other h ρ r 1 means that ASEMSE q ASEMSE. able 1 indicates that when we hope to achieve ASEMSE r ASEMSE, µ has to be selected lower than that of. Our experiments using various noises indicates that this is true for all kind of noises. has to utilize lower step size than that of, but since the time constant of is smaller than that of, can still be faster than. Furthermore our experiments show that the selection of β is not critical. π

4 ASEMSE σv µ. β $ 9m v β m v 6 1 4σv µ $ 60 6γ 1 σ v γ m v 4 0 ( 60σv 0 tr ts xx γ 3 m v 4 0 ( tr Xs xx (3) E lu n 1 -ut µ tr Xs xx $ 9β m v β 4 m v ( E 3u 'ut µ E e a. 1 4σv µ $ 60 6σv λ 1 λ m v λ 3 m ( tr tsv x 0 (33) µ $ 60 6γ 1 σ v γ m v 4 1 4σ v γ 3 m v 4 0 ( tr ts vwv (34) 5. HE SEP SIZE LIMI By substituting (14) (5) into (0), we will obtain equation (33). Based on (33), we have to limit µ as in (34). his limit is consistent with the pretending (3) to be positive. 6. HE SPEED OF CONVERGENCE Another important parameter to measure the performance of thdaptive filter is the convergence speed; showing how fast thlgorithm adapts when disturbances occur. Assuming that µ rxr 1 all terms of (33) containing µ can be neglected. hus from (0) we can write E./%,% n 1 -%+% 0 E./%,% -%+% 0 E µ κ e Using (9) we can obtain E.1%+% n 1 '%,% 0 Using the method in [1], we can obtain τ E.y%,% '%,% 0 1 µ. 6 1σv 0 svwv (36) µ 1 1 4σ v ws vwv (37) he convergence speed figure of merit is evaluate using k τ τ τ 1 4σ v (38) where τ is the time constant of the algorithm [1]. he resulted k τ for various conditions arlso included in tabel 1. k τ q 1 means that the time constant of algorithm τ r τ, meaning that will converges faster than that of. 7. CALCULAION COMPLEXIY able shows the required multiplications (mpx) additions (addt) to implement various adaptivlgorithms. he table shows that has the heaviest burden among, N. his fact indicates that more powerful processors have to be utilized for implementing the proposed algorithm than in case of, N. his complexity is compensated by the fact that the performance of the is the best among those threlgorithms. (35) Noise k µ ρ µ k τ ype Gaussian distributed Flat distributed Non symmetrical distributed able 1: he table of k µ, ρ µ k τ for various adaptivlgorithms various noises. All Sp N 5dB Method Formula Complexity z { n} 1~/ z { n~t} µ { n~ e{ n~ M } mpx M addt N z { n} 1~/ z { n~t} µvƒ n eƒ n X n 3M } mpx 3M addt z { n} 1~ z { n~x} µ ;{ n~ κ { e{ n~ ~ M } 7 mpx M } addt able : he total multiplications mpx additions addt for various adaptivlgorithms. 8. HE SIMULAION RESULS In the following experiment, the unknown model is excited by a zero-mean noise. he order of the unknown system was set to be 0, while the order of thdaptive system was also selected to be 0. We measured e { n~ as a function of time index n. We compare the result of the the conventional algorithm [1] methods. he level of the noise v{ n~ was adjusted so that S N ˆ 5 db. In the first experiment, we used Gaussian noise. hverage progression of e { n~ that was obtained from 00 independent runs is plotted in Fig.. We used the the algorithms to adapt the coefficient of the filter. We select k µ 10. he progression of e { n~ for both algorithms are plotted in Fig.. hose plots indicate that the algorithm converges faster than that of also to a lower error as it is expected in section II. In the second experiment, a non Gaussian noise was applied.

5 Figure : he progression of e when the noise is Gaussian using algorithms. Figure 3: he progression of e for contaminated non- Gaussian symmetrical PDF additive noise. he non Gaussian noise was generated using the rom noise generator with flat probability density function ˆ 1 Š f { x~9š 1. In the experiments wpplied S N ˆ 5 db. he obtained e { n~ as in Fig. 3 shows the same behavior as before. We obtain smaller final error by using than that when is used. Figure 4: he progression of e for contaminated non- Gaussian non-symmetrical PDF additive noise. In the third experiment, a non Gaussian noise with nonsymmetrical PDF was applied. he non Gaussian noise was generated using the rom noise generator with flat probability density function ˆ 1 Š f { x~3š 1. o get non-symmetrical PDF will only take signal that are positive use them as noise. In the experiments the power of the noise was adjusted to get S N ˆ 5 db. he obtained e { n~ as in Fig. 4 shows the same behavior as before. We obtain smaller final error by using than that when is used. All our other simulation results show the same behavior for both Gaussian impulsivdditive noise cases. We can always achieve smaller error by using. he algorithm convergences is faster than that when is used. Because of space limitation, those results are not presented here. In many applications, the statistical behavior of noise is unknown it could be impulsive. hus, finally we recommend that is widely used as an alternativdaptivlgorithm than the conventional. 9. CONCLUSIONS A theoretical analysis of the algorithm performance has been presented in this paper. he theoretical experimental study results indicate that can outperforms the conventional method; i.e for Gaussian non-gaussian noises by adjusting µ. Selected thppropriate µ is still studied will be reported later. Since calculation burden of the are not much different, can be considered as an alternativlgorithm to. Real applications also efficient hardware implementation of are still studied will be reported somewhere else. REFERENCES [1] S. Haykin, Adaptive Filter heory, nd Edition, Prentice Hall Inc, [] A. Papoulis, Rom Variables Stochastic Processes, Mc Graw Hill Inc, [3] C.L. Nikias M.R. Raghuveer, Bispectrum Estimation: A Digital Signal Processing Framework, Proceedings of the IEEE, vol. 75, no. 7, pp. 869 ˆ 891, July [4] J. A. F. Fonolossa, Adaptive system Identification Based on High Order Statistics, Proc. ICASSP 1991, pp ˆ 3440, oronto, Canada, [5] D. I. Pazaitis A. G. Constantinides, A novel kurtosis driven variable step-sizdaptivlgorithm, IEEE rans. Signal Processing, vol. 47, pp. 864 ˆ 87, no. 3, March [6] D. I. Pazaitis A. G. Constantinides, A novel kurtosis driven variable step-sizdaptivlgorithm, IEEE rans. Signal Processing, vol. 47, pp. 864 ˆ 87, no. 3, March [7] D.I. Pazaitis A.G. Constantinides, An Intelligent +F Algorithm, Proc 8 th IEEE Signal Processing Workshop, Statistical Signal Array Processing, pp. 486 ˆ 489, June [8] O. anrikulu, A.G. Constantinendes, Least Mean Kurtosis: A novel higher-order statistics based adaptive filtering algorithm, Electronics Letters, vol. 30, no. 3, pp. 189 ˆ 190, February [9] P.I. Hubsher J.C. Bermudez, A Model for the Behavior of the Least Mean Kurtosis () Adaptive Algorithm with Gaussian Input, Proc. International elecommunication Symposium - IS 000, Natal, Brasil. [10] Nabil R. Yousef Ali H Sayed, A Unified Approach to the Steady State racking Analyses of Adaptive Filters, IEEE rans. on Signal Processing, vol. 49, no., pp. 314 ˆ 34, Feb. 001.