5th Eurpea Sigal Prcessig Cferece (EUSIPCO 7), Pza, Plad, September 3-7, 7, cpyright by EURASIP MOIFIE LEAKY ELAYE LMS ALGORIHM FOR IMPERFEC ESIMAE SYSEM ELAY Jua R. V. López, Orlad J. bias, ad Rui Seara LINSE Circuits ad Sigal Prcessig Labratry epartmet f Electrical Egieerig Federal Uiversity f Sata Cataria 884-9 Flriaóplis SC Brazil E-mails: {jua, rlad, seara}@lise.ufsc.br ABSRAC his paper prpses a mdified leaky delayed least-mea-square (MLLMS) algrithm, aimig t circumvet algrithm istability prblems uder imperfect system delay estimates. I additi, a mdel fr the first ad secd mmets f the algrithm is prpsed. Such a mdel is btaied withut ivkig the idepedece thery ad csiderig a slw adaptati cditi. Numerical simulatis crrbrate the very gd agreemet betwee the results btaied with the Mte Carl methd ad thse frm the prpsed mdel fr clred Gaussia iputs.. INROUCION he least-mea-square (LMS) algrithm is e f the mst used adaptive algrithms due t its rbustess ad cmputatial simplicity [], []. he wide rage f applicatis i which the LMS algrithm ca be applied gives rise t differet versis f the stadard frm. Specifically, we aalyze here the case i which the errr sigal is ly available fr the adaptive algrithm with a certai time delay. I this cditi, the stadard versi f the LMS algrithm perates lger prperly, presetig pr cvergece r eve istability [3], [4]. circumvet such a prblem the stadard LMS algrithm is slightly chaged, resultig i the delayed LMS (LMS) algrithm [5]-[7]. his algrithm hwever reveals a csiderable degradati as a fucti f the system-delay value fr bth cvergece ad stability prperties. vercme such a prblem, a mdificati f the LMS algrithm has bee prpsed i [8]-[9], givig birth t the mdified LMS (MLMS) algrithm. Such a chage leads the mdified algrithm t w have characteristics f cvergece ad stability idetical t thse f the stadard LMS e, i.e., idepedet f the system delay if that delay is accurately estimated. Hwever, uder a imperfect delay estimate similar t the stadard LMS algrithm, the MLMS als presets istability. reestablish the algrithm cvergece we prpse the use f a leaky factr, resultig i the mdified leaky LMS algrithm (MLLMS). Orlad J. bias is als with the Electrical Egieerig ad elecm. ept., Regial Uiversity f Blumeau (FURB), Blumeau, SC, Brazil. his wrk was supprted i part by the Brazilia Natial Research Cucil fr Scietific ad echlgical evelpmet (CNPq) ad the Cmmittee fr Pstgraduate Curses i Higher Educati (CAPES). Ccerig the stchastic mdelig f the LMS algrithm, i [5] ad [6] a detailed aalysis fr the first ad secd mmets f the adaptive filter weights is preseted. Such a aalysis is accmplished uder the light f the well-kw idepedece thery (I). Hwever, whe I is applied t the case i which the delay is i the adaptati path, such a assumpti is lger apprpriate []. I [], a stchastic aalysis f the LMS algrithm withut ivkig I is discussed. I that paper, mdel expressis are btaied fr the first ad secd mmets f the adaptive weight vectr, predictig accurately the LMS algrithm behavir. the best f ur kwledge, i the pe literature there is mdel fcusig the MLMS algrithm. hus, this wrk als prpses a stchastic mdel fr such a algrithm icludig the leakage factr. I the same way as i []-[], I is t ivked here fr derivig the required mdel expressis, resultig i a mre accurate mdelig fr the MLLMS algrithm. he prpsed mdel is btaied csiderig slw adaptati cditi ad Gaussia data. Cmpariss betwee Mte Carl (MC) simulatis ad prpsed mdel predictis are preseted allwig t assess the accuracy f the btaied mdel with crrelated data.. LEAKY LMS ALGORIHM. Algrithm descripti I this secti, the leaky LMS (LLMS) algrithm is briefly detailed. Fig. illustrates the blck diagram f the LMS algrithm where the fllwig tati is used: w = [ w, w,l w, N ] ad w( ) = [ w( ) w( ) L wn ( )] are the legth-n vectrs f the plat ad adaptive weights, respectively. he system delay is give by ad its estimate, by. Variables d ( ) ad z ( ) dete the desired (r primary) sigal ad the measuremet ise, respectively. he latter is i.i.d, zer-mea with variace σ z ad ucrrelated with ay ther sigal i the system. I this aalysis the iput vectr is x( ) = [ x( ) x( ) L x ( N+ )], with { x()} beig a zer-mea Gaussia prcess with variace σ x. Vectr x ( ) represets the delayed iput sigal. I this wrk, we csider that vectrs w ad w ( ) have the same 7 EURASIP 365
5th Eurpea Sigal Prcessig Cferece (EUSIPCO 7), Pza, Plad, September 3-7, 7, cpyright by EURASIP dimesi. Nte hwever that ad may be differet, beig this case the mst cmm cditi fr these parameters i practical applicatis. Frm Fig. the errr sigal is give by e( ) = d( ) y( ) + z( ) () where d ( ) ad y ( ) are btaied as fllws: d ( ) = w x( ) = x ( ) w () ad y( ) = w ( ) x( ) = x ( ) w ( ). (3) Nw, substitutig (3) it () e btais e( ) = d( ) w ( ) x ( ) + z( ). (4) herefre, the weight update equati f the leaky LMS algrithm is fially give by [] w( + ) =ν w( ) +μe( ) x ( ) (5) where v = μγ ad γ is the leakage factr. Sice μ is always psitive ad μγ, the ν.. Mdelig f the mdified leaky LMS algrithm he MLLMS algrithm is btaied by icludig a cmpesati term Λ ( ) i the istataeus errr sigal give i (4). I this way, the cmpesated errr sigal is w expressed as [8] e( ) = d( ) w ( ) x ( ) Λ ( ) + z( ). (6) he term Λ ( ) is determied by efrcig (6) t be equal t the errr sigal f the stadard LMS algrithm [], which is give by e ( ) = d ( ) w ( ) x ( ) + z ( ). (7) LMS I the pe literature [9], such cmpesati assumes a perfect estimate fr the system delay; hwever, we csider here a imperfect estimate cditi. Frm (6) ad (7) the cmpesatig term is Λ ( ) = d( ) d( ) w ( ) x( ) (8) + w ( ) x( ) Sice the delay estimate is kw, the fllwig apprximati ca be csidered w ( ) x ( ) w ( ) x ( ). hus, ad Λ ( ) =ε ( ) + [ w ( ) w ( )] x ( ) (9) ε ( ) ( ) ( ). = d d () Frm (5), the differece [ w ( ) w ( )] is btaied, thereby (9) results i j x x w x Λ ( ) =μ v e( j) ( j) ( ) +ε ( ) + ( v ) ( ) ( ). () By substitutig () it () ad usig (6), the cmpesated errr sigal is e( ) = d( ) w ( ) x( ) j μ v e( j) x ( j) x( ) () ( v ) w ( ) x( ). Nte that i () e ( ) depeds bth curret ad past values; the latter is t preset i the case f the stadard LMS algrithm. he weight update expressi fr the MLLMS algrithm is give by w( + ) =ν w( ) +μe ( ) x ( ). (3) x( ) x( ) w w( ) LMS d () + y () Σ e- ( ) z() + Figure Blck diagram f the LMS algrithm. 3. ANALYSIS 3. Aalysis assumptis btai the prpsed mdel expressis, e assumes the fllwig aalysis csideratis: i) he crrelatis betwee iput vectrs at differet lags are much mre imprtat tha the crrelatis betwee the iput ad weight vectrs []-[]. ii) he prpsed mdel is derived by csiderig slw adaptati; thus, the terms affected by μ β fr β 3 are disregarded. iii) Sice the iput sigal is Gaussia, the furth-rder mmet f the iput sigal are btaied by usig the Mmet Factrig herem []. 3. Mea-weight behavir I this secti, a mdel expressi fr the first mmet f the adaptive weight vectr is derived. By substitutig () it (3) ad takig the expected value f bth sides f the resultig expressi, we get E[ w( + )] =ν E[ w( )] +μe[ x( ) d( )] μe[ x( ) x ( ) w( )] j μ E[ x( ) v e ( j) x ( j) x( )] μ( v ) E[ x( ) x ( ) w( )] +μe[ x( ) z( )]. (4) 7 EURASIP 366
5th Eurpea Sigal Prcessig Cferece (EUSIPCO 7), Pza, Plad, September 3-7, 7, cpyright by EURASIP By determiig d ( ) ad e ( j) frm () ad (), respectively, ad substitutig these expressis it (4), by usig (i)-(iii), ad after sme algebra, we btai E[ w( + )] = ve[ w( )] +μp μr E[ w( )] v R E w j v j j j + μ( ) [ ( )] { R R R tr[ R ]} μ + + { w ( v ) E[ w( j)] } { RR RR j R + R + j } j v j j + μ + + tr[ ] E[ w( j)]. (5) where p = E[ x ( ) d( )]. he iput autcrrelati matrices i (5) are btaied accrdig t the geeral frm R = E[ x ( α) x ( β)]. βα 3.3 Steady-state value f w ( ) ad leakage stabilizig effect By assumig algrithm cvergece, the steady-state value f the weight vectr is btaied frm the fllwig cditi: lim E[ w( + )] = lim E[ w( )] = lim E[ w( )] = lim E[ w( j)] lim E[ w( j)] = w. (6) By substitutig (6) it (5) ad csiderig a slw adaptati cditi, we btai w = [ R ( ) ]. +γ I + ν R p (7) Fr the MLLMS algrithm, the effect f leakage is mre evidet i the case f imperfect delay estimate ad, i particular, fr a white iput vectr. I this case, it is easy t verify that matrix R has ly e zer diagal (either the mai r secdary, depedig the value f the differece ). Fr istace, csider the tw-tap filter case with σ x =, =, ad =, which results i R = (8) havig iverse. Hwever, by addig the term γ ( I+μR ), we btai the matrix γ ( +μ) R +γ ( I+μ R ) = γ ( + μ) (9) whse iverse ca w be cmputed. Obviusly, the cditi umber f this matrix depeds γ ad μ values. Fr clred iputs, matrix R is sigular, but i geeral, it is prly cditied. I this case, such a characteristic is imprved by itrducig sme leakage. 3.4 Weight-errr vectr f w ( ) Nw, defiig the weight-errr vectr as, v( ) = w( ) w () ad substitutig () it (5), ad disregardig the terms ctaiig μ, (5) ca be writte as E[ v( + )] = ve[ v( )] μr E[ v( )] + μ( v ) R E[ v( )] () At this pit, we ca better uderstad the mtivati ad hw the MLLMS algrithm wrks. Let us csider () fr a simple case f a adaptive filter havig tw taps ad the fllwig cditis: white ise iput with σ x =, =, ad =. hus, = [ ( ) ( )]. E R x x = () By detig E[ v ( )] = [ ε( ) ε( )] ad expressig () i terms f the vectr cmpets, the weight update equati yields ε ( + ) =νε( ) (3) ε ( + ) =νε ( ) με ( ) Assumig that ε(), ε(), ad ν<, we btai ε ( ) =ν ε() ε ( ) =ν ε () ( ) μν ε ( ) (4) Nte that (4) teds t zer whe ad ν<. O the ther had, zer leakage gives ν=, leadig (4) t diverge whe ad ε(). 3.5 Learig curve ad secd mmet f w ( ) By applyig (9) i (6) ad after a simple algebra we btai a mdel expressi fr the learig curve f the MLLMS algrithm. hus, e ( ) = d ( ) w ( ) x( ) (5) w ( ) x( ) + w ( ) x( ) + z( ). Nw, substitutig () it (5), squarig ad takig the expected value f bth sides f the resultig expressi, we btai Ee [ ( )] = w R w tr R E[ w( ) w ] { } { R E w w } { R E w w } { R E w w } { R E w w } { RE w w } { R E w w } { RE w w } { R E w ) ( )] } + σz. + tr [ ( ) ( )] tr [ ( ) ] + tr [ ( )] + tr [ ( ) ( )] + tr [ ( ) ( )] tr [ ( ) ( )] +tr [ ( ) ( )] tr [ ( w (6) 7 EURASIP 367
5th Eurpea Sigal Prcessig Cferece (EUSIPCO 7), Pza, Plad, September 3-7, 7, cpyright by EURASIP By usig (i) i (6), ad defiig σ d ( ) where =w R w, we get d ξ ( ) = E[ e ( )] ad { R k } { R k } { R k } { R k } { R k } { R k } ξ ( ) =σ ( ) tr ( ) + tr ( ) tr ( ) + tr ( ) + tr ( ) tr ( ) { Rk } { Rk } z + tr ( ) tr ( ) + σ (7) k ( ) = E[ w( ) w ( )] = E[ w( ) w ( )] = E[ w( ) w ( )] k ( ) = E[ w( )] w k( ) = E[ w( )] w k ( ) = E[ w( )] w k ( ) = E[ w( ) w ( )] k ( ) = E[ w( ) w ( )] k ( ) = E[ w( ) w ( )] cmplete the derivati f (7) we must determie the secd mmet f w ( ). this ed, by ivkig the slw adaptati cditi, the required secd mmet ca be apprximated by E[ v( ) v ( )] E[ v( )] E[ v ( )] [3]. 4. SIMULAION RESULS I this secti, csiderig a system idetificati prblem, tw examples are preseted i rder t assess bth the ew algrithm (MLLMS) ad its prpsed aalytical mdel fr clred Gaussia iput data. Example : I this example, the plat is the legth-7 vectr w = [.39.89.6.559.339 5.46 7.568 9.355. 9.355 7.568 5.46.339.559.6.89.39], the system-delay value is = = 7, ad a crrelated iput sigal is used, which is btaied frm a AR() prcess give by x( ) = a x( ) + a x( ) + u( ) (8) where u ( ) is white ise with uit variace. he cefficiets f the AR prcess are a =.58 ad a =.75 with a eigevalue spread f the iput autcrrelati matrix equal t 63.. he maximum step-size value (experimetally determied) with which the MLLMS algrithm cverges is μ max =.. A value f μ=.5 is used here. Mte Carl (MC) simulatis are btaied frm averagig idepedet rus. Fig. shws curves btaied by MC simulati fr the MLLMS algrithm with γ = (which becmes the stadard MLMS algrithm), γ =., ad γ =.. Frm that figure, we tice a icrease f the steady-state as the γ value is icreased. Example : his example illustrates umerical simulatis cmparig the results btaied frm MC simulatis ad the prpsed mdel, csiderig = 7, = 6 ad γ =.. he plat ad the AR() prcess are the same as i Example. he maximum step size (experimetally determied) with which the algrithm cverges is μ max =.. he umerical results are determied by usig μ =.5 ad μ =.8 i Figs. 3 ad 4, respectively. MC simulatis are btaied frm averagig idepedet rus. Figs. 3(a) ad 4(a) shw the mea-weight behavir btaied by simulati ad the prpsed mdel. Observe the gd matchig betwee simulati ad mdel. Figs. 3(b) ad 4(b) depict the curves btaied frm simulati ad the prpsed mdel. Agai, te that the prpsed mdel satisfactrily matches the behavir f the learig curve btaied frm simulati. Ntice that i the case f a imperfect delay estimate the stadard MLMS algrithm (MLLMS with γ = ) diverges as demstrated i Secti 3.4 (which is t the case f the MLLMS due t the use f sme leakage factr). 5. CONCLUING REMARKS he mdified leaky LMS (MLLMS) algrithm is prpsed t circumvet the stability prblems uder a imperfect system delay estimate. Aalytical expressis fr the first mmet ad the learig curves f the MLLMS algrithm are derived. he prpsed mdel is btaied withut ivkig the classical idepedece assumpti ad csiderig a slw adaptati cditi. Numerical simulati results verify the effectiveess f bth the ew prpsed algrithm ad its aalytical mdel fr clred Gaussia iput data. - - -3-4 γ=. γ=. γ= -5..4.6.8...4.6.8. 4 Figure Example. curves f the MLLMS algrithm fr clred iput sigal with = 7, = 7, ad μ=.5. 7 EURASIP 368
5th Eurpea Sigal Prcessig Cferece (EUSIPCO 7), Pza, Plad, September 3-7, 7, cpyright by EURASIP.35.3.5.5 E[w()]...5 - -.5 -. -.5.5..5 -. 4..4.6.8...4.6.8. 4 (a) (b) Figure 3 Example. Clred iput sigal fr = 7, = 6, μ =.5, ad γ =.. (a) Evluti f E[w ()]. (Ragged-gray lie) MC simulati; (slid-black lie) prpsed mdel. (b) curves: (Ragged-gray lie) MC simulatis; (slid-black lie) prpsed mdel..35.3.5.5 E[ w()].. -.5 -.5 -. -.5.5..5 -. 4..4.6.8...4.6.8. 4 (a) (b) Figure 4 Example. Clred iput data usig = 7, = 6, μ =.8, ad γ =.. (a) Evluti f E[w ()]. (Ragged-gray lie) MC simulati; (slid-black lie) mdel. (b) curves: (Ragged-gray lie) MC simulatis; (slid-black lie) mdel. REFERENCES [] B. Widrw ad S.. Stears. Adaptive Sigal Prcessig, Pretice Hall, 998. [] S. Hayki, Adaptive Filter hery, Pretice Hall,. [3].. Falcer, Adaptive referece ech cacellati, IEEE ras. Cmmuicatis, vl. 3,. 9, pp. 83-94, Sep. 98. [4] J. G. Prakis, igital Cmmuicati, ed., New Yrk: McGraw-Hill, 989. [5] G. Lg, F. Lig, ad J. Prakis, he LMS with delayed cefficiet adaptati, IEEE ras. Acust., Speech, Sigal Prcess., vl. 37,. 9, pp. 397-45, Sep. 989. [6], Crrectis t the LMS with delayed cefficiet adaptati, IEEE ras. Acust., Speech, Sigal Prcess., vl. 4,., pp. 3-3, Ja. 99. [7] F. Laichi,. Abulasr, ad W. Steeaart, Effect f delay the perfrmace f the leaky LMS adaptive algrithm, IEEE ras. Sigal Prcess., vl. 45,. 3, pp. 8-83, Mar. 997. [8] R.. Pltma, Cversi f the delayed LMS algrithm it the LMS algrithm, IEEE Sigal Prcess. Lett., vl.,., pp. 3, ec. 995. 7 EURASIP [9] M. Rupp, Savig cmplexity f mdified filtered-x LMS ad delayed update LMS algrithms, IEEE ras. Circuits Syst. II: Aalg igit. Sigal Prcess., vl. 44,., pp. 45-48, Ja. 997. [] O. J. bias ad R. Seara, Leaky delayed LMS algrithm: aalysis fr Gaussia data ad delay mdelig errr, IEEE ras. Sigal Prcess., vl. 5,. 6, pp. 596-66, Ju. 4. [] O. J. bias, J. C. M. Bermudez, N. J. Bershad, ad R. Seara, Mea weight behavir f the filtered-x LMS algrithm, i Prc. IEEE It. Cf. Acustics, Speech, Sigal Prcessig (ICASSP), Seattle, WA, May 998, pp. 3545-3548. [] I. S. Reed, O a mmet therem fr cmplex Gaussia prcesses, IEEE ras. Ifrm. hery, vl. 8,. 3, pp. 94-95, Apr. 96. [3] N. J. Bershad, P. Celka, ad J. M. Vesi, Stchastic aalysis f gradiet adaptive idetificati f liear systems with memry fr Gaussia data ad isy iput ad utput measuremets, IEEE ras. Sigal Prcess., vl. 47,. 3, pp. 675-689, Mar. 999. 369