6. Comparison of NLMS-OCF with Existing Algorithms

Size: px

Start display at page:

Download "6. Comparison of NLMS-OCF with Existing Algorithms"

Clarence Stafford
6 years ago
Views:

1 6. Compariso of NLMS-OCF with Eistig Algorithms I Chaptrs 5 w drivd th NLMS-OCF algorithm, aalyzd th covrgc ad trackig bhavior of NLMS-OCF, ad dvlopd a fast vrsio of th NLMS-OCF algorithm. W also mtiod th istc of a class of algorithms quivalt to NLMS-OCF. h distiguishig charactristic of ths algorithms is that thy updat th wights basd o multipl iput sigal vctors. As mtiod i Chaptr, Ozki ad Umda's affi projctio algorithm (APA) is a wll-kow ampl of this class []. Algorithms such as th partial rak algorithm (PRA) ad gralizd optimal block algorithm (GOBA) also blog to this class. A cllt rviw of this APA class of algorithms was publishd rctly [3]. Svral othr algorithms, such as Nw Data-Rusig LMS (NDR-LMS) proposd by Schaufr ad Jkis [7] ad Dcorrlatig Algorithms (DA) proposd by Rupp [0], also us multipl iput vctors for adaptatio. I this chaptr, w compar th prformac of NLMS-OCF with th APA, NDR-LMS, ad DA algorithms. hs algorithms us multipl iput vctors for adaptatio. Howvr, thy us th iput vctors diffrtly for adaptatio ad hc yild diffrt prformacs. h covrgc rat of ths algorithms is cotrolld by th stp-siz costat µ. W aalyz th bhavior of ths algorithms for µ = du to two rasos. Firstly, µ = is th optimum stp siz to maimiz th covrgc rat. Scodly, th adaptatio corrspodig to µ = lads to a ic gomtric itrprtatio of th abov algorithms. Diffrt authors us diffrt otatios for variabls such as stp-siz, filtr ordr, ad umbr of vctors usd for adaptatio. o avoid cofusios du to otatios, w us th stadard otatios usd arlir i this dissrtatio whil discussig th diffrt algorithms. 6. NLMS Algorithm h wight updat quatio of th NLMS algorithm is giv by [] + = + (6.) µ 70

2 whr ŵ is th old stimat (a priori) for th plat paramtrs, is th iput vctor giv by ( ) N+ =, ad is th a priori stimatio rror giv by = ŵ (6.) d I th abov quatios, ( ) dots th traspos oprator. For ( 0,) updats th stimat for th wights µ, th NLMS algorithm ŵ so that th a postriori stimatio rror giv by d + = (6.3) is strictly lss tha th a priori stimatio rror. I particular, for µ =, th a postriori rror is rducd to zro. hat is, w + ˆ = d. (6.4) Sic (6.4) is usually a udr-dtrmid systm of quatios (o quatio with N ukows), thr is a maifold π of solutios w that rduc th a postriori rror to zro. h updat gratd by NLMS is th solutio that miimizs th orm of th icrmt i wights ˆ + +. Equivaltly, th w stimat for wights w ˆ + gratd by NLMS is th orthogoal projctio of th currt stimat ŵ oto th maifold π of solutios to (6.4). h NLMS algorithm has a vry low computatioal complity of Ο ( N). 6. Affi Projctio Algorithm Whil NLMS updats th wights basd oly o th currt iput vctor, APA updats th wights basd o a additioal M such vctors i th most rct past, amly o,,,,. For µ =, th AP algorithm updats th wights so that th a postriori M rrors k / + d k + = ar simultaously rducd to zro for k =,,, M. k ˆ + Equivaltly, th w wight stimat w gratd by APA is th orthogoal projctio of th currt stimat orthogoally projcts ŵ oto th itrsctio of th maifolds ŵ oto M k =0 π. k π, π,, π, i.. APA M 7

3 h wight adaptatio quatio of APA is giv by [] whr X = [ ] M + + [ XX] = µ X (6.5) is th matri formd by th currt ad past iput vctors usd for adaptatio ad [ ] / / M / = is th vctor formd by th associatd a priori stimatio rrors. Notic that wh M is chos to b zro, th APA adaptatio, show i (6.5), rducs to NLMS adaptatio, show i (6.). Aothr itrstig itrprtatio of APA has b put forth by Rupp [0]. h vctor usd to adapt th adaptiv filtr (quivaltly, th dirctio alog which th filtr cofficits ar updatd) is usually rfrrd to as th iformatio vctor. Rupp shows that for µ = APA adaptatio is quivalt to usig th compot of that is orthogoal to,,, as th iformatio vctor, whil NLMS uss itslf as th iformatio vctor. h wights ar adaptd so that th a postriori rror bcoms zro. Furthrmor, sic th adaptatio is orthogoal to M / +,,, th a postriori rrors for k =,,, M rmai th sam as thir corrspodig a priori rrors, which ar zro. Usig th compot of that is orthogoal to th past M vctors tds to dcorrlat th idividual lmts of th iformatio vctor, spcially if is a autorgrssiv (AR) procss of ordr at most M. Hc, Rupp argus, APA ca covrg fastr tha NLMS. It would b worth otig hr that this dos ot ma that th covrgc rat of APA is th sam as th covrgc rat of NLMS with whit (dcorrlatd) iput, or that o furthr improvmt i covrgc rat ca b ralizd by icrasig M byod th ordr of th AR procss. his is bcaus APA is ot mrly dcorrlatig th iformatio vctor; it is also isurig that th currt iformatio vctor is orthogoal to th M iput vctors i th most rct past. Whil th dcorrlatio itrprtatio hlps to plai th acclratio of covrgc corrspodig to a AR iput, it dos ot plai, for ampl, why APA covrgs fastr for a whit iput sigal that alrady grats dcorrlatd iformatio vctors (so that APA ca ot dcorrlat ay furthr!). h acclratd covrgc i th lattr cas ca, howvr, b plaid by th orthogoality proprty. If th iformatio vctor is orthogoal to th past M vctors, th M+ a postriori rrors ar simultaously forcd to zro. h dcorrlatio proprty dos ot guarat this; with k / + M 7

4 a dcorrlatd iformatio vctor, which is ot cssarily orthogoal to th past M vctors, oly th currt a postriori rror is st to zro. hus, th orthogoality proprty provids a huristic plaatio for th fastr covrgc of APA i th cas of a whit iput sigal. Our APA covrgc aalysis rsults, drivd i Chaptr 3, ad th simulatio rsults providd i Sctio 6.6 corroborat th fact that APA covrgs fastr tha NLMS for a whit iput. h AP algorithm as show i (6.5) has a computatioal complity of Ο ( NM ). A fast vrsio of th APA is also kow [9]. his fast AP (FAP) algorithm has a rducd computatioal complity of N + Ο( M) kow to hav umrical problms [].. Howvr, th fast trasvrsal filtr usd by FAP is 6.3 NLMS with Orthogoal Corrctio Factors h NLMS-OCF algorithm, as dos APA, adapts th wights basd o M vctors i th past, i additio to th currt iput vctor. Howvr, th past vctors ar chos diffrtly. NLMS- OCF uss a st of vctors that ar spacd D sampls apart, startig from th currt iput vctor. hat is, th NLMS-OCF adaptatio is basd o,,,,, whr D is th iput vctor dlay. APA is th spcial cas of NLMS-OCF corrspodig to D =. Hc, NLMS- OCF is cosidrd a gralizd AP algorithm. For µ =, th NLMS-OCF algorithm updats D D MD th wights so that th a postriori rrors k / + d k + = ar simultaously rducd to k zro for k =, D,, MD. Equivaltly, th w wight stimat w gratd by ˆ + NLMS is th orthogoal projctio of th currt stimat ŵ oto th itrsctio of th maifolds π, π D,, π MD, i.. NLMS-OCF orthogoally projcts ŵ oto M π. As w showd i Chaptr, for D =, th wight updat gratd by NLMS-OCF is idtical to th wight updat gratd by APA. NLMS-OCF allows for usig a iput vctor dlay largr tha uity. For larg valus of D, th iput vctors D k =0 D kd,,,, ar mor likly to b mutually ucorrlatd ad, hc, th iput vctors usd for wight updats td to spa a mor "divrs" subspac. his hlps to acclrat th covrgc udr most MD 73

5 circumstacs. Cosqutly, th NLMS-OCF algorithm ca provid fastr covrgc tha th APA for th sam umbr M of iput vctors usd for th wight updat. As w saw i arlir chaptrs, th NLMS-OCF basd o Gram-Schmidt orthogoalizatio [7] for computig OCF has a computatioal complity of ( NM ) Ο whil th fast NLMS-OCF that uss a lattic structur to grat th OCF has a computatioal complity of Ο ( NM ). 6.4 Nw Data-Rusig LMS Algorithm h ormalizd data-rusig LMS (NDR-LMS) algorithm proposd by Schaufr ad Jkis also uss th past iput vctors for adaptatio [7]. Howvr, th past iput vctors ar usd for adaptatio diffrtly. h NDR-LMS algorithm rpatdly russ iput vctors from prvious itratios, i additio to th iput vctor from th currt itratio, i ordr to grat gradit stimats ad to adapt th cofficit vctor alog ach of th gradits. I mathmatical trms, NDR-LMS updats th wights as show blow [7]: c ˆ 0 = (6.6) w For i = 0,,R-, rpat (6.7) c i+ = ci + 0, i + µ, i + + µ M, i M µ (6.7) w ˆ = (6.8) + c R whr ( c ) k, i d k i k k µ =. h paramtr R is th rus factor. his ida of data rusig was origially motivatd by th possibility to ploit th uusd cycls availabl o th procssor. Hc, R is usually chos dpdig o th umbr of uusd cycls availabl. As R approachs ifiity, th w wight stimat gratd by NDR-LMS approachs th w wight stimat gratd by APA [56]. A carful compariso of th NDR-LMS updat quatio, show i (6.6)-(6.8), with th NLMS-OCF adaptatio quatio rvals that NDR-LMS uss th currt ad past iput vctors dirctly for its wight updat, whil NLMS-OCF uss a orthogoal basis of th spac spad by th currt ad past iput vctors for its wight updat. I gomtric trms, NDR-LMS projcts ŵ durig ach itratio succssivly oto π, π,..., π ad this squtial M 74

6 projctio is rpatd R tims. his adaptatio procdur dos ot sur that th a postriori rrors at, -,, -M ar miimizd simultaously. Istad, th a postriori rrors ar miimizd squtially. Bcaus of th squtial miimizatio, whil miimizig th a postriori rror at k, th prviously miimizd a postriori rrors ar ot maitaid at zro. his causs th wight updat producd by NDR-LMS to dpd o th ordr i which th iput vctors ar usd. h NDR-LMS algorithm has a complity of Ο ( NMR). 6.5 Dcorrlatig Algorithms wo algorithms blogig to this family hav b proposd by Rupp [0]. hs algorithms wr motivatd by th dcorrlatig proprty of APA. Sic th dcorrlatio itrprtatio of APA is valid oly if th iput sigal is a AR procss, APA is modifid to prsrv th dcorrlatio proprty if th iput sigal is a movig avrag (MA) or ARMA procss, rsultig i th corrspodig dcorrlatig algorithms. Howvr, th modifid algorithms los th mor importat orthogoality proprty possssd by APA. I othr words, for µ = ths algorithms st oly th currt a postriori rror to zro. Hc, ths algorithms covrg slowr tha APA. h updat quatios of th dcorrlatig algorithm for MA iput ar giv blow [0]. Z b φ + = = [ φ φ φ ] [ Z Z ] = = d = w Z b +µ h first thr stps i (6.9) sstially attmpt to dcorrlat th MA procss. his dcorrlatio procss rsults i th "dcorrlatd" vctor φ, which is orthogoal to φ Z φ M (6.9) φ k for k =,,, M. DA adapts th wights alog φ such that th a postriori rror at is miimizd (zro if th stp siz is uity). Sic φ big orthogoal to φ for k =,,, M dos ot k cssarily ma that φ is orthogoal to for k =,,, M, th a postriori rrors at -, k 75

7 -,, -M ar ot cssarily zro. Hc, th dcorrlatig algorithms do ot possss th (dsirabl) orthogoality proprty. Cosqutly, o dos ot pct DA to outprform APA. h dcorrlatig algorithm, show i (6.9), has a low computatioal complity of ( ) N + Ο M. Sic th dcorrlatio procss ivolvs (tim-varyig) fdback, th dcorrlatio algorithm ca possibly bcom ustabl [0]. his is a major drawback of this class of algorithms. 6.6 Simulatio Rsults h prformac of th abov algorithms was compard usig MALAB simulatios for four diffrt iput procsss, amly big whit, first-ordr AR with its pol at 0.95, highly colord first-ordr MA with its zro at 0.95, ad rasoably colord first-ordr MA with its zro at 0.5. h systm to b idtifid i ach cas is modld usig a 3 st ordr FIR filtr. h tap wights of th systm ar chos such that th maimum tropy assumptio is satisfid. h adaptiv filtr ad th udrlyig systm ar assumd to hav th sam umbr of taps. W furthr assum that th masurmt ois is abst. h umbr M of additioal iput vctors (compard to NLMS) is chos to b 0 for ach algorithm. h stp siz µ is st to uity. W us dlay D of 3 for NLMS-OCF simulatios ad rus factor R of uity for NDR-LMS simulatios. All simulatios us soft iitializatio. All th adaptiv filtr cofficits ar iitializd to zro. All th rsults show ar obtaid by smbl avragig 5 idpdt ralizatios. Figur 6. shows th larig curvs of th algorithms corrspodig to whit iput. W obsrv that APA covrgs fastr tha NLMS ad that NLMS-OCF has th fastst covrgc amog all of th algorithms. h fact that APA covrgs fastr tha NLMS for whit iput cofirms that APA dos ot possss mrly th dcorrlatig proprty (but also th orthogoality proprty). Itrstigly, for whit iput, NDR-LMS ad DA hav th sam covrgc rat as APA. Whil NLMS covrgs at a rat of approimatly 0 db for vry 5 N itratios, NLMS- OCF covrgs at a rat of approimatly 0 db for vry 5 ( M +) N itratios. 76

8 Figur 6. Larig Curvs for Whit Iput: (a) NLMS, (b) APA, (c) NLMS-OCF, (d) NDR-LMS, ad () DA. Figur 6. shows th larig curvs for th AR iput. It is vidt that NLMS-OCF has th fastst covrgc ad that NDR-LMS has th slowst covrgc (but fastr tha NLMS) amog th algorithms big compard. Figur 6. Larig Curvs for AR Iput: (a) NLMS, (b) APA, (c) NLMS-OCF, (d) NDR-LMS, ad () DA. Figur 6.3 shows th larig curvs for th highly colord MA iput. W obsrv that APA, which is ot a "optimal" dcorrlatio algorithm for th MA iput, has a slightly fastr covrgc tha DA, which "optimally" dcorrlats th MA iput. NDR-LMS has th slowst covrgc (but fastr tha NLMS) for th MA iput as wll. W also s from Figur 6.3 that 77

9 NLMS-OCF is slowr tha APA. his suggsts that icrasig D dos ot hlp to acclrat th covrgc if th iput is a highly colord MA procss. Figur 6.3 Larig Curvs for Highly Colord MA Iput: (a) NLMS, (b) APA, (c) NLMS-OCF, (d) NDR-LMS, ad () DA. h larig curvs corrspodig to th rasoably colord MA iput ar show i Figur 6.4. W obsrv that NLMS-OCF has th fastst covrgc ad also that APA, DA, ad NDR- LMS hav comparabl covrgc rats. Figur 6.4 Larig Curvs for Rasoably Colord MA Iput: (a) NLMS, (b) APA, (c) NLMS-OCF, (d) NDR-LMS, ad () DA. 78

10 6.7 Coclusio W compard NLMS-OCF with diffrt adaptiv filtrig algorithms. Whil all of thm prform bttr tha NLMS, NLMS-OCF prforms bttr tha th rst for whit, AR, ad rasoably colord MA iputs. Whil APA ad DA yild comparabl prformacs for th whit ad MA iputs, APA outprforms DA for th AR iput. APA covrgs fastr tha NLMS v for th whit iput, suggstig that APA posssss ot oly th "dcorrlatio proprty" but also th orthogoality proprty. Ev though APA is ot th "optimal" dcorrlatig algorithm for MA iputs, APA prforms as wll as DA for th MA iput. Howvr, DA, which is th optimal dcorrlatig algorithm for MA iputs, dos ot prform as wll as APA for AR iputs. 79

Statistics 3858 : Likelihood Ratio for Exponential Distribution

Statistics 3858 : Likelihood Ratio for Exponential Distribution Statistics 3858 : Liklihood Ratio for Expotial Distributio I ths two xampl th rjctio rjctio rgio is of th form {x : 2 log (Λ(x)) > c} for a appropriat costat c. For a siz α tst, usig Thorm 9.5A w obtai