WINDOW OPTIMIZATION ISSUES IN RECURSIVE LEAST-SQUARES ADAPTIVE FILTERING AND TRACKING Tayb Sadiki,Mahdi Triki, Dirk T.M. Slock Eurcom Institut 9 rout ds Crêts, B.P. 93, 694 Sophia Antipolis Cdx, FRANCE Email: sadiki,triki,slock@urcom.fr ABSTRACT In this papr w considr tracking of an optimal filtr modld as a stationary vctor procss. W intrprt th Rcursiv Last- Squars RLS adaptiv filtring algorithm as a filtring opration on th optimal filtr procss and th intantanous gradint nois inducd by th masurmnt nois. Th filtring opration carrid out by th RLS algorithm dpnds on th window usd in th last-squars critrion. To arriv at a rcursiv LS algorithm rquirs that th window impuls rspons can b xprssd rcursivly output of an IIR filtr. In practic, only two popular window choics xist with ach on tuning paramtr: th xponntial wighting W-RLS and th rctangular window SWC-RLS. Howvr, th rctangular window can b gnralizd at a small cost for th rsulting RLS algorithm to a window with thr paramtrs GSW-RLS instad of just on, ncompassing both SWCand W-RLS as spcial cass. Sinc th complxity of SWC-RLS ssntially doubls with rspct to W-RLS, it is gnrally blivd that this incras in complxity allows for som improvmnt in tracking prformanc. W show that, with qual stimation nois, W-RLS gnrally outprforms SWC-RLS in causal tracking, with GSW-RLS still prforming bttr, whras for non-causal tracking SWC-RLS is by far th bst with GSW-RLS not bing abl to improv. Whn th window paramtrs ar optimizd for causal tracking MSE, GSW-RLS outprforms W-RLS which outprforms SWC-RLS. W also driv th optimal window shaps for causal and non-causal tracking of arbitrary variation spctra. It turns outs that W-RLS is optimal for causal tracking of AR paramtr variations whras SWC-RLS if optimal for non-causal tracking of intgratd whit jumping paramtrs, all optimal filtr paramtrs having proportional variation spctra in both cass.. INTRODUCTION Th RLS algorithm is on of th basic tools for adaptiv filtring. Th convrgnc bhavior of th RLS algorithm is now wll undrstood. Typically, th RLS algorithm has a fast convrgnc rat, and is not snsitiv to th ignvalu sprad of th corrlation matrix of th input signal. Howvr, whn oprating in a nonstationary nvironmnt, th adaptiv filtr has th additional task of tracking th variation in nvironmntal conditions. In this contxt, it has bn stablishd that adaptiv algorithms that xhibit good Eurcom s rsarch is partially supportd by its industrial partnrs: Bouygus Télécom, Fondation d Entrpris Group Cgtl, Fondation Haslr, Franc Télécom, Hitachi, Sharp, ST Microlctronics, Swisscom, Txas Instrumnts, Thals. convrgnc proprtis in stationary nvironmnts do not ncssarily provid good tracking prformanc in a non-stationary nvironmnt; bcaus th convrgnc bhavior of an adaptiv filtr is a transint phnomnon, whras th tracking bhavior is a stadystat proprty [, ]. On fundamntal non-stationary scnario involvs a timvarying systm in which th cross-corrlation btwn th input signal and th dsird rspons is tim-varying. This cas occurs in th systm idntification stup. To tak into account systm variation, two main variants of RLS algorithms xist. Th first introducs a forgtting factor, and lads to th xponntially Wightd RLS W-RLS approach. Th scond uss a Sliding rctangular Window SWC-RLS approach. In [3, 4], a gnralizd sliding window RLS GSW-RLS algorithm was introducd, that gnralizs th W-RLS and SWC-RLS algorithms. Th GSW-RLS uss a gnralizd window s Fig., which consists of an xponntial window with a discontinuity at dlay L. It can b sn that th xponntial and rctangular windows ar particular cass of th gnralizd window, for α = and α, λ =, rsp. In [3, 4], Fig.. Th gnralizd sliding window a tracking improvmnt for GSW-RLS was obsrvd for diffrnt systm variation modls AR, MA, and Random walk. On th on hand th initial portion of th window prmits to mphasiz th vry rcnt past which allows vry fast tracking. On th othr hand, th GSW-RLS algorithm solvs nvrthlss an ovrdtrmind systm of quations and hnc njoys th fast convrgnc proprtis of RLS algorithms. Anothr ffct of th xponntial tail of th GSW is rgularization. In fact, th rctangular window sampl covarianc matrix apparing in SWC-RLS can b particularly ill-conditiond compard to a sampl covarianc matrix basd on an xponntial window with compatibl tim constant. Finally, th GSW-RLS algorithm turns out to hav th sam structur and comparabl computational complxity as th SWC-RLS algorithm.
This papr is organizd as follows. In sction, a tracking analysis in th frquncy domain is prsntd. Uninformd and Informd Baysian approachs ar invstigatd rspctivly in sctions 3, and 4. Finally a discussion and concluding rmarks ar providd in sction 5.. TRACKING CHARACTERISTICS OF RLS ALGORITHMS W considr th classic adaptiv systm idntification problm s Fig.. Th adaptiv systm idntification is dsignd for dtrmining a typically linar FIR modl of th transfr function for an unknown, tim-varying digital or analog systm. systm variation is a zro-man, wid-sns stationary procss Hk o with a powr spctral dnsity matrix S HH, and if w invok th indpndnc assumption, in which Y k and H k ar assumd to b indpndnt this works bttr for th a priori rror, th EMSE can b xprssd in th following form: [ ] EMSE = tr E H k HT k R = tr R S H H df By stting th gradint of J k in 3 w.r.t. H k to zro, w hav F q Y k Yk T H k = F q Y k d k = F q Y k Y T k H o k + F q Y k n k.. Lt s dnot by F q = F q th dc transfr normalizd wighting window. As this window is gnrally low-pass, F q acts as F an avraging oprator, and w hav F q Y k Y T k R. Fig.. Systm idntification block diagram Th adaptiv systm idntification problm can b dscribd by: dk = Hk ot Y k + n k x k = Hk T Y k whr - n k is an iid Gaussian nois squnc n k N, σ n whr σ n is th Minimum Man Squard Error MMSE - H o k dnots th optimal Winr Filtr - H k rprsnts th adaptiv Filtr - k is th a postriori rror givn by: k = d k x k = H T k Y k + n k whr H k = H o k H k dnots th filtr dviation. In wightd RLS, th st of th N adaptiv filtr cofficints H k = [H,k H N,k ] T gts adaptd so as to minimiz rcursivly th Wightd Last Squars critrion J k = F q k = i f i k i 3 whr F z = i fi z i is th transfr function of th wighting window f i charactrizing th RLS algorithm, and q k = k. Thr ar a numbr of rfrncs daling with th prformanc of RLS algorithms in non-stationary nvironmnts [7, 6, 5, 4]. Th basic ida is to focus on th modl quality in trms of th output Excss MSE EMSE. W considr stationary optimal filtr variation modls, hnc th RLS algorithm will rach a stationary rgim to which w limit attntion. Th EMSE is dfind as: EMSE = E k σ n = E Yk T H k HT k Y k 4 in principl th a priori rror signal should b considrd for th EMSE, w shall stick to th a postriori rror signal to avoid th apparanc of a dlay in th notation. So, if w assum that th On th othr hand, as th optimal systm variation is indpndnt of th input signal in th systm id stup, w approximat: F qy k Y T k H o k F qy k Yk T F qhk o R F qh o k. Hnc th filtr dviation can b xprssd as: H k = Hk o H k = F q Hk o R F qyk n k and th EMSE bcoms: EMSE = Nσ n F jπf df 5 + F tr R S HH df Rmark that th EMSE can b brokn up into two trms: E st = Nσn F jπf df corrsponding to th stimation nois contribution; it can b intrprtd as th stimation accuracy in tim-invariant conditions, E lag = F jπf tr R S HH df rp- rsnting th stimation rror rsulting from low-pass filtring th systm variations lag nois, sinc in th causal window cas this mans lagging bhind. Th stimation and lag nois trms can also b intrprtd as th varianc and th bias of th conditional stimation problm, for a givn valu of th optimal filtr squnc. In fact, H k = F q Hk o R F qyk n k b
whr b = E H o Hk is th stimation bias. W gt R H H = E H o Hk HT k = b b T + σn f i R bias i varianc Thn w s that: E st = Nσ n i f i is th varianc componnt, E lag = tr [ R E ] H o bb T is th bias componnt.. 6 3. UNINFORMED APPROACH FOR RLS TRACKING ANALYSIS In an uninformd approach w assum that littl or no information about th systm variations is availabl for th dsign of th RLS algorithm. 3.. Uninformd Tracking Analysis of Causal RLS Algorithms From 5, w can s that th following window charactristics charactriz stimation and lag noiss rsp.: l = f i i = F charactrizing E st, E F f = F, calld paramtr tracking charactristic, charactrizing th lag nois. To compar th tracking ability of RLS with diffrnt wighting windows, w shall choos th windows paramtrs such that th diffrnt algorithms bhav idntically undr tim-invariant conditions. In fact, comparing adaptiv filtrs charactrizd by diffrnt valus of l barly maks any sns in th uninformd cas and it rsmbls comparing runnrs that spcializ in diffrnt distancs [8]. By normalizing th prformanc undr tim-invariant conditions, th tracking charactristic E F w dpnds only on th window shap. Sinc th complxity of RLS with a rctangular window ssntially doubls with rspct to an xponntial window, it is gnrally blivd that this incras in complxity allows for som improvmnt in tracking prformanc. In contrast to this intuition, Fig. 3 shows that th plot of th normalizd tracking charactristic of W-RLS lis blow that corrsponding to SWC-RLS. Thus, th tracking capability of W-RLS approach is bttr. This ffct can b attributd to a highr dgr of concntration of th xponntial window around i =, which rsults in a smallr stimation dlay, hnc smallr bias rror [8]. As w hav mntiond in th Introduction, th SW-RLS approach can b gnralizd at a small cost for th rsulting RLS algorithm to a window with thr paramtrs instad of just on. Compard to th Sliding and th Exponntial widows, th Gnralizd Sliding Window introducs two xtra dgrs of frdom. Th shap of th widow dpnds on th choic of ths dgrs of frdom. Thus, thy can b optimizd to minimiz th avrag paramtr tracking charactristic. In othr words, th window paramtrs ar chosn so as to min λ,α,l f subjct to k F df 7 f k = l whr f is th assumd bandwidth of th systm variations. In Fig. 3, w add th tracking charactristic of th GSW-RLS stimator, as a function of f = f o. As xpctd, th optimizd GSW- RLS outprforms th SWC-RLS and W-RLS approachs. E F.5..5..5 paramtr tracking charactristic SWC RLS W RLS GSW RLS..4.6 frq.8. x 3 Fig. 3. Paramtr tracking charactristic for W, SWC, GSW-RLS. 3.. Uninformd Tracking Analysis for Non-Causal RLS Algorithms Bias in th RLS algorithm is causd by two kinds of distortion: amplitud and phas distortions. Phas distortion can b considrably attnuatd by introducing a suitabl stimation dlay using non-causal filtring. With no information about th systm charactristics, a suitabl stimation dlay can b dtrmind as [8]: τ = k k f k, 8 th man of th f k considrd as a distribution. As bfor, undr idntical stimation nois, comparing th tracking capability of th non-causal RLS algorithms can b invstigatd by comparing what is calld in [8] th paramtr matching charactristic dfind as: Ẽ F w = τ F = τ F. Fig. 4 shows plots of normalizd matching charactristics of SWC- RLS, W-RLS, and GSW-RLS with optimizd window paramtrs algorithms. Th curvs show that in th non-causal adaptation cas, th optimal shap for th gnralizd window bcoms th rctangular on and in particular, rctangular windowing outprforms xponntial windowing. Th bttr paramtr matching proprtis of th SWC-RLS approach can b xplaind by th linarity of th associatd phas charactristic du to th window symmtry. Th dlay τ bcoms th cntr of th window and aftr dlay compnsation, thr is zro phas distortion lft. 4. INFORMED APPROACH FOR RLS TRACKING OPTIMIZATION Now w suppos th statistics of th systm variation to b availabl. In this cas, w can achiv an optimal tradoff btwn th stimation and lag noiss. Th optimal tradoff can b found by minimizing th EMSE: min F Nσ n df F + df F tr RS HH f subjct to F = 9
.5 paramtr matching charactristic SWC RLS W RLS GSW RLS whr γ k gts computd rcursivly as: γk+ = λγ k ακ λ L k, k < L, γ k+ = λγ k, k L. E F. To invstigat th tracking ability of th diffrnt RLS algorithms, w compar th minimum EMSE achivd by ach variant with optimizd paramtrs. Fig. 5 plots th curvs of minimizd EMSE as a function of th bandwidth f..5 35 3 Gnralizd Sliding window Exponntial window Rctangular window..4.6 frq.8. x 3 Fig. 4. Paramtr matching charactristic. 4.. Optimizd Causal Paramtric Windows, Sparabl Variation Spctrum Cas In a first instanc, w propos to invstigat a simpl but intrsting variation modl. W assum that th impuls rspons cofficints hav proportional Dopplr spctrum; i.. S HH = D S hh. To simplify, w suppos also that th scalar spctrum S hh is a flat low-pass spctrum; i.. S hh = f < f lswhr Th matrix D is arbitrary but if it wr diagonal dcorrlatd filtr cofficints th diagonal would rprsnt th powr dlay profil of th optimal filtr in wirlss channl trminology. With th sparabl modl, th Excss MSE for th causal adaptation cas can b xprssd as: Nσ n df F + tr R D df F. Th EMSE xprssions for th diffrnt RLS variants bcom, as a function of th windows paramtrs: EMSE SW CRLS = Nσ n L L + tr R D L f L sin πf k πl EMSE W RLS = Nσn λ + λ + tr R D λf λ π k= λ + λ arctan + λ tan πf λ EMSE GSW RLS = Nσn λ + αα L L + λ αλ L γ k + tr R D f γ + sin kπf πk k= Misadjustmnt 5 5 5..4.6.8..4.6.8 3 log f Fig. 5. EMSE min curvs for flat low-pass variations. This analysis shows that, for a flat low-pass spctrum, th xponntial window prforms bttr than th rctangular window, but also that th optimal gnralizd window prforms vn bttr. 4.. Optimizd Windows Fig. 6. Signal in nois problm. Considr minimizing th Excss MSE with rspct to th window cofficints. This problm can b intrprtd in trms of Winr filtring for a signal in nois problm, s Fig. 6, whr th dsird and nois signal spctra ar rspctivly S vv = σ v = Nσ n, and S dd = tr R S HH. Th causal Winr solution for such problm is F W inr = S dd S dd + σ v Th DC componnt of this Winr solution is F W inr = S dd S dd + σ v.. Now, sinc S dd is quit lowpass, w hav S dd >> σ v. Thus, for an accptabl SNR, F W inr. Thn, F opt F W inr = S dd. 3 S dd + σv
Th associatd Excss MSE is givn by: EMSE min = σ v F opt df = σ v f opt. 4 If w impos a causality constraint, th Winr solution bcoms: F c W inr σv = σ A 5 whr A z dnots th optimal prdiction rror filtr for th signal x = d + v, and σ th associat prdiction rror varianc. Using th sam argumnts as bfor, on can show that, for an accptabl SNR, FW c inr ; and thn F opt c F W c inr. Th associatd Excss MSE is givn by: EMSE c min = σ v F c opt df = σ v f c,opt. 6 4.3. Optimality Considrations for Classical Windows Th qustion w invstigat in this sction is For which optimal filtr variation modl is th xponntial or th rctangular window optimal?. To answr this qustion, w us th rvrs nginring tchniqu. W assum a sparabl optimal filtr variation spctrum with uncorrlatd optimal filtr cofficints. Equating th causal Winr solution and th xponntial window transfr function lads to a variation spctrum of th form: S hh z αz αz. 7 Thus, for AR drifting paramtrs, th xponntial window optimizs th tracking prformanc ovr th st of all causal windows. Similarly, by quating th non-causal Winr filtr with th cntrd rctangular window, on can show that SWC-RLS is optimal for a variation spctrum of th form: S hh z FRz F Rz 8 whr F Rz = z L z L dnots th cntrd sliding window transfr function. S hh z in 8 can b intrprtd to b th L z z spctrum of a procss g with spctrum S ggz = F Rz, lowpass filtrd by Hz = / F R z spctral factor. Th input g can b intrprtd as a whit jumping procss with mmory L/, or as whit nois viwd on th scal L/: g stays constant for L/ sampls, thn jumps to an uncorrlatd valu, stays constant again for L/ sampls and so on. On th othr hand, around frquncy zro F R jπf 3 π L f. 9 Thn, th low-pass filtr Hz can b intrprtd as an intgrator. 5. CONCLUDING REMARKS In this papr, w hav considrd tracking of a tim-varying systm modld as a stationary vctor procss. In th Uninformd approach, w hav invstigatd th tracking capability, by comparing th tracking and matching charactristics of th diffrnt RLS windows undr idntical stimation nois. In th Informd approach, w hav intrprtd th RLS algorithm as a filtring opration on th optimal filtr procss and th instantanous gradint nois. W hav shown th optimality of th xponntial window for AR drifting paramtrs, and of th sliding window for intgratd whit jumping paramtrs. An opn qustion rmains: how to stimat and optimiz simultanously th adaptiv filtr and window paramtrs? Altrnativly, on may opt for a twostp approach: Stp : non-causal SWC-RLS with short window, providing noisy but undistortd filtr stimats. Stp : Winr Kalman filtring to provid th optimal stimation nois/low-pass distortion compromis,.g. as in [9]. Such an approach allows for lss constraind optimal filtring, that can b optimizd, and tailord to individual and possibly corrlatd filtr cofficints, whras RLS has only on global window. 6. REFERENCES [] O.M. Machhi and N.J. Brshad. Adaptiv Rcovry of a Chirpd Sinusoid in Nois: I. Prformanc of th RLS Algorithm, IEEE Trans. Acoustics, Spch, and Signal Procssing, Vol.39, pp. 583 594, March 99. [] S. Haykin, A.H. Sayd, J.R. Zidlr, P. Y, P.C. Wi. Adaptiv Tracking of Linar Tim-Variant Systms by Extndd RLS Algorithms, IEEE Trans. Signal Procssing, Vol.45, pp. 8-8, May 997. [3] K. Maouch and D.T.M. Slock. Prformanc Analysis and FTF Vrsion of th Gnralizd Sliding Window Rcursiv Last-Squars GSWRLS Algorithm, In Proc. Asilomar Confrnc on Signals, Systms and Computrs, Vol., pp. 685 689, Pacific Grov, USA, Nov. 995. [4] K. Maouch. Algorithms ds Moindrs Carr és R écursifs Doublmnt Rapids : Application à l Idntification d R éponss Impulsionnlls Longus, PhD Thsis, Ecol National Sup ériur ds T él écommunications, Paris, March 996. [5] M. Montazri. Un Famill d Algorithms Adaptatifs Comprnant ls Algorithms NLMS t RLS. Application à l Annulation d Echo Acoustiqu, PhD Thsis, Univrsit é d Paris-Sud, U.F.R Scintifiqu d ORSAY, Spt. 994. [6] E. Elfthriou and D. Falconr. Tracking Proprtis and Stady Stat Prformanc of RLS Adaptiv Filtr Algorithms, IEEE Trans. Acoustics, Spch and Signal Procssing, Vol.34, No.5, pp. 97, Oct. 986. [7] M. Nidzwicki. First-Ordr Tracking Proprtis of Wightd Last Squars Estimators, IEEE Trans. Automatic Control, Vol.33, No., pp. 94 96, Jan. 988. [8] M. Nidzwicki. Idntification of Tim-Varying Procsss, Wily & Sons,. [9] M. Lnardi and D. Slock. Estimation of Tim-Varying Wirlss Channls and Application to th UMTS W-CDMA FDD Downlink, In Proc. Europan Wirlss EW, Flornc, Italy, Fb..