IMPROVED STEADY STATE ANALYSIS OF THE RECURSIVE LEAST SQUARES ALGORITHM

Transcription

1 IMPROVED STEADY STATE ANALYSIS OF THE RECURSIVE LEAST SQUARES ALGORITHM Muhammad Monuddn,2 Tareq Y. Al-Naffour 3 Khaled A. Al-Hujal 4 Electrcal and Computer Engneerng Department, Kng Abdulazz Unversty KAU), Kngdom of Saud Araba KSA), Emal: mmsansar@kau.edu.sa 2 Center of Excellence n Intellgent Engneerng Systems CEIES), KAU, KSA, Emal: mmsansar@kau.edu.sa 3 Electrcal Engneerng Department, Kng Abdullah Unversty of Scence and Technology, Kngdom of Saud Araba, Emal:tareq.alnaffour@kaust.edu.sa 4 Department of Electrcal Engneerng, Tabah Unversty, KSA, Emal:khujal@tabah.edu.sa ABSTRACT Ths paper presents a new approach for studyng the steady state performance of the Recursve Least Square RLS) adaptve flter for a crcularly correlated Gaussan nput. Earler methods have two major drawbacks: ) The energy relaton developed for the RLS s approxmate as we show later) and 2) The evaluaton of the moment of the random varable u 2 P, where u s nput to the RLS flter and P s the estmate of the nverse of nput covarance matrx by assumng that u and P are ndependent whch s not true). These assumptons could result n negatve value of the stead-state Excess Mean Square Error EMSE). To overcome these ssues, we modfy the energy relaton wthout mposng any approxmaton. Based on modfed energy relaton, we derve the steady-state EMSE and two upper bounds on the EMSE. For that, we derve closed from expresson for the aforementoned moment whch s based on fndng the cumulatve dstrbuton functon CDF) of the random varable of the form + u 2 D, where u s correlated crcular Gaussan nput and D s a dagonal matrx. Smulaton results corroborate our analytcal fndngs. Index Terms Adaptve Flters, RLS, Steady-state analyss, Mean square analyss, Excess Mean-Squares-Error. INTRODUCTION The RLS s one of the mportant algorthm from the adaptve flter s famly. Motvaton for the RLS adaptve flters reles on the fact that t provde a soluton to the least square error mnmzaton problem. The RLS algorthms are more costler than the basc famles such as the Leat mean squares famly LMS) but wth much faster convergence speed. Analyzng the performance of the RLS algorthm s not an easy task due to the presence of the nput covarance matrx and ts nverse whch depend on current and past nput regressors. As such, only a few works consdered the performance of the RLS and ts varants, 2, 3, 4, 5, 6, 7, 8, 9, 0,, 2. The smplest approach s based on the energy relaton 9, 3 whch wth the ad of separaton prncple 9) can be used to state that the EMSE s a functon of the moment E u 2 P, where u s the nput regressor and P s the estmate of the nverse of nput covarance matrx. Another separaton prncple s then used to wrte E u 2 P = T reu u P ). But ths approach s not rgorous as u and P are dependent. Other approaches use the dea of random matrx to study the performance of the RLS 0,. In addton to requrng a much more sophstcated machnery, these approaches are vald for flters relatvely larger szes. In ths work, we perform steady-state analyss of the RLS algorthm for correlated crcular Gaussan nputs. The approach s based on evaluatng the CDF and the moments of random varable of the form z = where u s a correlated crcular Gaussan vector and D s a dagonal matrx. To + u 2 D evaluate the CDF, we replace the step functon wth ts equvalent Fourer transform and employ complex ntegraton. Usng ths CDF, we evaluate n closed form the requred moment that appear n the steady-state mean-square analyss of RLS algorthm. 2. THE EXPONENTIALLY WEIGHTED RLS ALGORITHM The exponentally weghted RLS attempts to solve the followng problem mn N+) w Πw + y N H N w) Γ N y N H N w) w ) n a recursve manner. Here y N = d0), d),, dn) T s the measurements vector and H N s the N + M) For any matrx D, the quadratc form u 2 D s defned as u 2 D u Du where the notaton ) denotes conjugate transposton. = /8/$ IEEE 439 ICASSP 208

2 data matrx consstng of N + ) row regressor vectors u k and Γ N = dag{ N, N,...,, } s a dagonal weghtng matrx defned n terms of the forgettng factor 0 ). It can be shown that the soluton of ) s w N = P N H N Γ Ny N. The RLS allows us to obtan the soluton of ) n a recursve manner. Specfcally, we have where w = w + P u d) u w 2) P = P P u u P + u P u whch s ntalzed wth w = 0 and = Π. 3. THE CONVENTIONAL ENERGY RELATION FOR THE RLS In the system dentfcaton model, the measurement d) takes the form d) = u wo + v) where v) s an addtve nose and w o s the system coeffcents. We can thus defne the weght error vector as w = w o w and wrte 2) as 3) w = w P u e) 4) where e) = d) u w whch s called the estmaton error. Two other error measures are the a pror and a posteror estmaton errors defned by respectvely e a ) = u w, e p ) = u w. If we multply both sdes of 4) from the left by u we obtan e p ) = e a ) u 2 P e) 5) Next, by evaluatng the energes of both sdes of 4) wth the ad of 5) wll lead to the well known Energy Conservaton Relaton 9 gven by ea ) 2 ep ) 2 E u 2 = E P u 2 6) P Fnally, by usng 5) and the fact that e) = e a ) + v), we can show that σ 2 ve u 2 P + E u 2 P e a ) 2) = 2E e a ) 2 7) Ths relaton can be used to evaluate the steady-state EMSE whch s defned as ζ = lm E e a ) Approach of 9 and ts Drawbacks The approach of 9 hss followng drawbacks:. In the development of the energy relaton 7), t assumes lm E w 2 = lm E w 2 whch s not true because the norms w 2 and w 2 are not weghted by ther respectve P matrces. 2. It assumes that u and P are ndependent whch s not true as can be seen from 3) whch shows that P s a functon of u and P ). As a result, the moment E u 2 P s approxmated as lm E u 2 P = E u 2 lm EP = T rrp) 8) where T r represents the trace operator, P = lm EP and R = Eu u 3. It assumes that the expectaton of the nverse of P s equal to the nverse of expectaton of P,.e., P lm EP ) = )R 9) whch s also not true n general. By employng 8) and 9), the requred moment s approxmated as E u 2 P TraceRP) )M. Thus, the EMSE proposed by 9 s found to be ζ σ2 v )M 2 )M 0) The problem wth the above result s that t wll gve negatve values of EMSE for the scenaro of )M > 2 and a value of nfnty at )M = 2 whch are not realzable. In order to deal wth the above lmtatons, we propose to modfy the energy relaton whch s presented next. 4. OUR APPROACH: MODIFIED ENERGY RELATION FOR THE RLS In the proposed approach, we modfy the energy relaton by rewrtng w 2 n terms of. Ths s done by expressng the n terms of usng the relaton P = + u u whch modfes the term w 2 to w 2 = w 2 + e a ) 2 ) By usng the above, the modfed energy relaton takes the form w 2 + e a) 2 u 2 P = w 2 + e a ) 2 + e p) 2 u 2 P 2) Now, by replacng e p ) 2 by ts equvalent expresson n 5) and by assumng that lm w 2 = lm w 2 yelds v 2 u 2 P + u 2 P e a ) 2 = e a ) 2 + ) w 2 3) 440

3 Next, we evaluate the term w 2 = + u u whch results n w 2 usng the relaton = w 2 + w 2 u 4) u But w 2 u u s nothng but e p) 2. As a result, the energy relaton n 3) can be rewrtten as v 2 u 2 P + u 2 P e a ) 2 = e a ) 2 + ) w 2 + ) e p ) 2 5) In the ensung secton, we derve the steady-state EMSE and ts upper bounds usng the modfed energy relaton 5). Thus, wth the ad of the above defnton and the relaton gven n 5), the relaton 8) results n the followng expresson for the steady-state EMSE ζ = σ2 v + EZ 2EZ )EZ 2 ) + EZ + )EZ 2 2) where we have employed the separaton prncple for the e terms E a) 2 e + u and E a) 2 2 P + u. It can be seen 2 P ) 2 from 2) that the calculaton of EMSE requres the evaluaton of EZ and EZ 2 whch are provded n Appendx. 6. UPPER BOUNDS ON THE EMSE 5. PROPOSED STEADY-STATE EMSE In ths secton, wth the ad of separaton prncple, we evaluate the expresson for the steady-state EMSE of the RLS by usng the modfed energy relaton derved n 5). We start by takng expectaton on both sdes of 5) to arrve at σ 2 ve u 2 P + E u 2 P e a ) 2 = E e a ) 2 + )E w 2 + )E e p ) 2 6) Snce s a functon of {u j} for all j < and the ndependence assumpton dctates us that w s ndependent of u j for all j <, we conclude that w s ndependent of. Hence, the term E w 2 can be smplfed to E w 2 = E w 2 E = E w 2 R = E e a) 2 ) ) 7) where we have used the facts that E = R ) and E w 2 R = E e a) 2. Thus, the relaton n 6) can be set up as σ 2 ve u 2 P + E u 2 P e a ) 2 = + )E e a ) 2 + )E e p ) 2 8) To proceed further, we replace u 2 P n 8) by ts equvalent representaton 2 u 2 P = u 2 P + u 2 P = + u 2 P 9) At ths stage, we defne the followng random varable Z = + u 2 P 20) 2 Ths equvalent representaton s obtaned by pre and post multplyng 3) by u and u, respectvely. In ths secton, we am to derve upper bounds on the EMSE of the RLS wthout nvokng the separaton prncplefor two extreme scenaros: large values of as ) and small vaues of as 0). 6.. Upper Bound for Large Values of Startng wth 5) and by takng expectaton wth the ad of the result n 7 and the alternate expresson for u 2 P, we can show that σve u 2 2 P = + 2 )ζ + )E w 2 u 2 P + )E e p ) 2 + u 2 P ) 22) To obtan the upper bound on the EMSE, we drop the last two terms on the rght hand sde of the above havng the term ) whch s also a good approxmaton of the EMSE for larger values of as ) approaches to zero as approaches to unty. As a result, we arrve to the followng upper bound on the EMSE σve u 2 2 P ζ + 2 = σ2 vt rrp) 23) + 2 where we have used the fact that lm E u 2 = P T rrp) because P s ndependent of u Upper Bound for Small Values of We agan start wth 5) to develop the upper bound for smaller values of. By droppng the term ) w 2 and approxmatng the term ) as unty whch s true for small values of ), we can have the followng nequalty v 2 u 2 P + u 2 P e a ) 2 e a ) 2 + e p ) 2 24) Now, usng 5) and alternatve representaton of u 2 P gven n 9), the above relaton results n v 2 u 2 P 2 e a ) 2 + e a ) 2 u 2 P 25) 44

4 EMSE of 9 EMSE va Smulatons EMSE va Proposed Approach Proposed Upper Bound For Small EMSE va Smulatons EMSE va Proposed Approach EMSE db) 4 EMSE db) Fg.. EMSE of the RLS versus forgettng factor. Fg. 3. Upper Bound for small. EMSE db) Proposed Upper Bound For Large EMSE va Smulatons EMSE va Proposed Approach Fg. 2. Upper Bound for Large. Next, by takng expectaton on both sdes of the above gves us σ 2 ve u 2 P 2E e a ) 2 + E e a ) 2 u 2 P 26) Fnally, by droppng the second term from the rght hand sde of 26) and usng the fact that P s ndependent of u, we get the followng upper bound on the EMSE ζ σ2 vt rrp) 2 7. SIMULATION RESULTS 27) In smulatons, the steady-state performance of the RLS algorthm s nvestgated for an unknown system dentfcaton wth w o = , , , , T. The nose s zero mean..d wth varance σ 2 v = Input to the adaptve flter and to the unknown system s correlated crcular complex Gaussan havng correlaton R, j) = α j c 0 < α c < ). We frst compare our derved EMSE gven n 2)) wth the one obtaned va smulatons and the one proposed n 9 n Fg. 2. It can be depcted from the fgure that the proposed EMSE result has a good match wth the smulaton for larger values of say > 0.8) but t gves a poor estmate for smaller values of. Ths devaton s because of employng the separaton prncple whch s not vald for smaller values of. On the other hand, the EMSE usng 9 gves postve values only for larger forgettng factor.e., 9. Ths s because of the fact that the EMSE expresson gven n 9 becomes unrealstc negatve or nfnty) for )M 2. In contrast, our approach s vald for all values of and M. Next, n Fg. 2, the upper bound for large gven n 23)) s compared wth the EMSE va smulaton and usng 2). It can be depcted from the result and ts zoomed vew that the proposed upper bound s tght and very close to the smulaton result partcularly for large values of. Fnally, n Fg. 3, the upper bound for small gven n 27)) s plotted whch also shows that ths upper bound s also close to the EMSE va smulatons near small values of. 8. CONCLUSION In ths work, we analyze the RLS algorthm at steady state for correlated complex Gaussan nput and we evaluate ts steady-state EMSE. The novelty of the work resdes n three aspects: frst n the modfcaton of energy relaton, second n the evaluaton of the moment E u 2 P whch s based on the dervaton of a closed form expresson for the CDF and moment of random varable of the form + u 2 D, and thrd n dervaton of upper bounds on the EMSE for both large and small values of. Smulaton results show that, unlke the prevous work, EMSE va our approach s vald for a vde range of. Also, the derved upper bounds are very close to the smulatons n ther respectve ranges. APPENDIX: Moments of the Random Varable Z We evaluate the requred moments of Z by frst evaluatng ts CDF usng the approach of 4, 5 and are found to be and EZ = M EZ 2 = 2 M m= m= E2 )e fm M = f f m m e fm E 2 ) + E 3 ) 2 M = f f m m ) 28) 29) 442

5 9. REFERENCES E. Eleftherou and D. Falconer, Trackng propertes and steady-state performance of RLS adaptve flter algorthms, IEEE Transactons on Acoustcs, Speech and Sgnal Processng, vol. 34, no. 5, pp , L. L. L. Guo and P. Prouret, Performance analyss of the forgettng factor rls algorthm, Int. J. Adaptve Contr. Sgnal Processng, vol. 7, pp , P. S. R. Dnz and M. G. Squera, Analyss of the qrrls algorthm for colored-nput sgnals, Proc. IEEE Int. Conf. Acoust., Speech, Sgnal Process., T. Al-Naffour and B. Hassb, On the dstrbuton of ndefnte quadratc forms n gaussan random varables, n IEEE Internatonal Symposum on Informaton Theory, ISIT 2009, pp , T. Y. Al-Naffour, M. Monuddn, N. Ajeeb, B. Hassb, and A. L. Moustakas, On the Dstrbuton of Indefnte Quadratc Forms n Gaussan Random Varables, IEEE Transactons on Communcatons, vol. 64, no., pp , G. V. Moustakdes, Study of the transent phase of the forgettng factor rls, IEEE Transactons on Sgnal Processng, vol. 45, no. 0, pp , S. S. Haykn, Adaptve flter theory. Upper Saddle Rver, N.J.: Prentce Hall, S. M. Y. Engel and R. Mer, The kernel recursve leastsquares algorthm, IEEE Transactons on Sgnal Processng, vol. 52, no. 8, pp , Aug M. L. Chebolu and S. K. Jayaweera, Energy effcency analyss of an rls-based adaptve sgnal processng algorthm for energy aware sensor networks, Proceedngs of 2005 Internatonal Conference on Intellgent Sensng and Informaton Processng, Jan X.-D. Z. Z-Zhe Dng and Y.-T. Su, Study of the transent phase of the forgettng factor rls, IEEE Sgnal Processng Letters, vol. 4, pp. 4, A. H. Sayed, Adaptve Flters. Wley-IEEE Press, ed., Aprl A. Vakl and B. Hassb, A steltjes transform approach for analyzng the RLS adaptve flter, n th Annual Allerton Conference on Communcaton, Control, and Computng, pp , M. Pajovc and J. C. Presg, Performance analyss of the least squares based lt channel dentfcaton algorthm usng random matrx methods, 49th Annual Allerton Conference on Communcaton, Control, and Computng Allerton), pp , Montcello, G. Mateos and G. B. Gannaks, Dstrbuted recursve least-squares: Stablty and performance analyss, IEEE Transactons on Sgnal Processng, vol. 60, pp , Aprl T. Y. Al-Naffour and A. H. Sayed, Transent analyss of data normalzed adaptve flters, IEEE Transactons on Sgnal Processng, vol. 5, no. 3, pp , Mar