DISTRIBUTED PROCESSIG OVER ADAPTIVE ETWORKS Casso G Lopes and A H Sayed Department of Eectrca Engneerng Unversty of Caforna Los Angees, CA, 995 Ema: {casso, sayed@eeucaedu ABSTRACT Dstrbuted adaptve agorthms are proposed based on ncrementa and dffuson strateges Adaptaton rues that are sutabe for rng topooges and genera topooges are descrbed Both dstrbuted LMS and RLS mpementatons are consdered n order to endow a networ of nodes wth earnng abtes; thus resutng n a networ that s an adaptve entty n ts own rght ITRODUCTIO Dstrbuted and sensor networs are emergng as a formdabe technoogy for a varety of appcatons, rangng from precson agrcuture, to envronment surveance, to target ocazaton However, the advantages advocated by the use of dstrbuted and cooperatve processng [] demand adaptve processng capabtes at the dstrbuted nodes n order to be abe to cope wth tme varatons n the envronment and the networ In addton, the adaptve processors shoud enabe the networ to respond to such varatons n rea-tme and to adjust ts performance accordngy Inspred by ncrementa strateges [], we propose dstrbuted processng strateges over what we refer to as adaptve networs (eg, []) The proposed strateges requre the adaptve nodes to share nformaton ony ocay and to expot both spata and tempora nformaton n a cooperatve fashon Dfferent cooperaton poces w ead to dfferent dstrbuted agorthms Each node across an -node networ s assumed to have access to tme reazatons {d (), u,, of zero-mean random data {d, u, wth d () a scaar measurement and u, a regresson row vector; both at tme see Fgure The nodes shoud use the data to estmate some unnown common vector w o Rather than expect each node to functon ndependenty of the other nodes, the nodes w nstead coaborate wth each other n an adaptve manner n order to acheve three objectves: () mprove goba performance wth reduced communcaton; () aow the nodes to converge to the desred estmate wthout the need to share goba nformaton; () endow the networ wth earnng abtes Recenty the authors proposed a scheme [] that mpements a dstrbuted ncrementa gradent agorthm n whch an nta vector estmate s updated aong a coaboraton cyce over the networ Each oca fter updates the estmate receved from the prevous neghbor wth ts oca data and passes the estmate to the next node +, operatng over a coaboraton cyce - see Ths matera was based on wor supported n part by the atona Scence Foundaton under awards ECS-88 and ECS- The wor of Mr Lopes was aso supported by a feowshp from CAPES, Braz, under award 8/- Fg A dstrbuted networ wth nodes Fgure further ahead Ths approach requres mted communcatons and ncreases the networ autonomy [] The networ may aso earn at an enhanced pace as compared to a standard gradentdescent souton In ths paper, we propose a east-squares framewor that equps the nodes wth a RLS-type adaptaton rue, whe eepng the same cooperaton strategy, yedng to a dstrbuted and recursve eastsquares souton (drls) The resutng agorthm conveys an exact goba east-squares estmate, for the unnown vector w o, to every node n the networ Ths scheme further aows an aternatve drls mpementaton wth decreased communcaton requrements, savng energy compared wth ts exact counterpart It aso has the strng feature that n steady-state, both agorthms present smar performance n the mean-square error sense When more communcaton and energy resources are avaabe, the topoogy constrants mped by the aforementoned agorthms can be removed by resortng to a dffuson cooperatve scheme, where the adaptve processor at node updates ts estmate usng a avaabe estmates from the neghbors, as we as oca data and ts own past estmate see Fgure DISTRIBUTED ESTIMATIO We are nterested n estmatng an unnown vector w o from measurements coected at nodes n a networ (see Fg ) Each node has access to reazatons of zero-mean data {d, u, =,,, where d s a scaar and u s M We coect the
regresson and measurement data nto goba matrces: U = co{u, u,, u ( M) () d = co{d, d,, d ( ) () and pose the mnmum mean-square error estmaton probem: mn w J(w), where J(w) = E d Uw () The optma souton w o satsfes the norma equatons []: where R u = E U U = X = R du = R u w o () R u,, R du = E U d = X = R du, (5) Computng w o from () requres every node to have access to the goba statstca nformaton {R u, R du, thus dranng communcatons and computatona resources In [] we proposed a dstrbuted souton (ncrementa LMS) that aows cooperaton among nodes through mted oca communcatons, whe equppng the nodes wth adaptve mechansms to respond to tme-varatons n the underyng sgna statstcs ICREMETAL LMS ADAPTATIO In ths secton, we revew the dstrbuted ncrementa LMS agorthm [], whch s a startng pont for the ater deveopments The agorthm s obtaned as foows We start from the standard gradent-descent mpementaton w = w µ [ J(w )] () for sovng the norma equatons (), where µ > s a sutaby chosen step-sze parameter, w s an estmate for w o at teraton, and J( ) denotes the gradent vector of J(w) evauated at w For µ suffcenty sma we w have w w o as Ths teratve souton coud be apped at every node or centray at some centra node A dstrbuted verson can be motvated as foows The cost functon J(w) can be decomposed as: J(w) = X = J (w), where J (w) = E d u w () whch aows us to rewrte () as w = w µ " X = J (w )# (8) ow et be a oca estmate of wo at node and tme and assgn the nta condton w Then w can be evauated by teratng through the nodes n the foowng manner: = ψ() µ [ J (w )], =,, (9) At the end of the procedure (9), the ast node w contan the goba estmate w from (8), e, w Ths scheme st requres EMSE (db) 5 5 5 5 Incrementa Souton () Souton (9) 8 Iteraton Fg Excess mean square error (EMSE) performance for both the dstrbuted ncrementa souton () and the centrazed souton (9) at node a nodes to share the goba nformaton w A fuy dstrbuted souton can be acheved by resortng to ncrementa strateges, whch woud requre each node n (9) to evauate ts parta gradent J ( ) at ts oca estmate, nstead of w Ths approach eads to the ncrementa agorthm: = ψ() µ [ J ( ) ], =,, () Ths cooperatve scheme requres each node to communcate ony wth ts mmedate neghbor over a pre-defned path Moreover, t s an estabshed resut n optmzaton theory that the ncrementa souton () can outperform the souton (9) as ustrated n Fg The fgure compares the excess mean square error (EMSE) of both agorthms for a networ wth = nodes and usng Gaussan regressors wth R u, = I The bacground nose s whte and Gaussan wth σ v = The curves are obtaned by averagng over 5 experments wth µ = 5 ow usng nstantaneous approxmatons ˆR du, = d ()u, and ˆR u, = u,u, n (), and aowng for dfferent step-szes at dfferent nodes, eads to a dstrbuted ncrementa LMS agorthm, summarzed beow: 8 >< >: w = ψ() + µ u, w d () u, () wth =,, In ths agorthm, a weght estmate s crcuated through a path defned over the networ and updated by oca adaptve fters usng oca data see Fg node: {d (), u, node: {d (), u, node: - {d - (), u -, node: + {d + (), u +, node: {d (), u, ode, Tme ( ) Remote nput Loca nput ψ (from node -) (sensors) { d ( ), u, ( ( ) ) ψ + µ u d u ψ ( ) * ( ),, ( ) ψ Toward neghbor node + ode output (ready to use) Fg The structure of ncrementa LMS
EXACT DISTRIBUTED RLS ADAPTATIO We formuate n ths secton a east-squares souton for estmatng the unnown parameter vector w o At each tme nstant, the networ has access to the foowng space-tme data: y = d () d () d () 5 and H = u, u, u, 5 () We can then see an estmate for w o by sovng a reguarzed eastsquares probem of the form []: mn w w Πw + Y H w () where Π > s a reguarzaton matrx and Y and H coect a the data that are avaabe up to tme : Y = y y y 5 and H = H H H 5 () One coud aso ncorporate weghtng nto () to account for spata reevance, tempora reevance, and node reevance Here we contnue wthout weghtng n order to convey the man dea We are thus nterested n dervng a dstrbuted mpementaton of the east-squares souton Some reated wor has been recenty proposed where a goba east-squares souton s acheved ony approxmatey at each node, and the agorthm demands arge communcaton and energy resources [5] We proceed nstead as foows Assume that w s the souton to the foowng eastsquares (LS) probem usng the data that are avaabe up to tme : mn w Πw + Y H w w (5) We are nterested n updatng w to w by accountng for the ncomng data bocs y and H at tme An agorthm that updates w ncrementay s gven by: w ; P, P for = : e () = d () u, = ψ() + P, +u, P, u u,e (), P, = P, P,u, u,p, +u, P, u, end w ; P P, () Smary to the ncrementa LMS n Secton, agorthm () nduces a cyce across the networ, aong whch the estmate w s spatay updated by sequentay vstng every node once At each node, the estmate at tme s the LS souton that s based on the data bocs Y and H and the data coected aong the path up to that node, namey mn ψ ψ Πψ + Y H ψ = () where now Y = Y d () d () d () 5 and H = H u, u, u, 5 (8) At the end of the cyce, w contan the desred souton w If we start from = wth w = and P = Π and repeatedy appy (), then w be the souton to () The dstrbuted RLS (drls) agorthm () can be motvated as foows ote frst that the souton for () s gven by []: wth P, = Partton Y and H as foows: Y Y = and H = d () = P,H Y (9) Π + H H () H u, () Then a spata recurson for P, can be found by substtutng () nto (): P, = Π + H H = Π + H H + u,u, = P, + u,u, () whch, by appyng the matrx nverson emma, eads to P, = P, P,u,u, P, + u, P, u (), Substtutng () and () nto (9) we get: = P, H Y + u,d () = P, H {z Y = + P, u, whch eads to wth u,p, u, + u, P, u, P, u,u, P + u, P, u, H, = ψ() + P, d () Y {z = + u, P, u u,e () (), e () = d () u, (5) Groupng (), (), and (5) eads to () The agorthm structure s reatvey smpe and t can be understood as a standard east-squares souton unwrapped aong the coaboraton cyce However, the nodes are exposed to data wth dstnct spata and nose profes Ths varaton refects tsef n the performance of the agorthm, whch w be studed esewhere
5 LOW-COMMUICATIO DISTRIBUTED RLS ADAPTATIO The agorthm proposed n the prevous secton mpements exact RLS dstrbutvey, whereby the nodes share nformaton about the oca weght estmates { and the matrces {P, A ess compex souton that ony shares nformaton about the weght estmates can be obtaned by requrng the matrces {P, to evove ocay Ths strategy eads to: w for = : e () = d () u, P, = = ψ() + end w P, +u, P, u, u,e () P, P, u, u,p, +u, P, u, () To ustrate the operaton of both agorthms drls and ts owcommuncaton counterpart (LC-dRLS), we consder a networ wth = 5 nodes where each oca fter has M = taps The system evoves for teratons and the resuts are averaged over ndependent experments The steady-state meansquare error vaues are obtaned by averagng the ast 5 tme sampes Each node accesses tme-correated spatay ndependent Gaussan regressors u, wth correaton functons r () = σ u, (α ), =,, M, wth {α and {σ u, randomy chosen n [, ) and depcted n Fg The bacground nose v () has varance σ v, = across the networ ote n Fg 5 that both dstrbuted RLS agorthms have smar performance n the mean-square error sense, suggestng that the ow-communcaton mpementaton can be a qute compettve suboptma mpementaton DIFFUSIO LMS ADAPTATIO When more communcaton and energy resources are avaabe, we may tae advantage of the networ connectvty and devse more sophstcated peer-to-peer cooperaton rues We expore a smpe dffuson protoco (see Fg ) Each ndvdua node consuts ts peer nodes from the neghborhood ( ) and combnes ther past estmates {ψ ( ) ; ( ) wth ts own past estmate ψ ( ) The node generates an aggregated estmate φ ( ) and feeds t n ts oca adaptve fter Ths strategy can be expressed as foows: φ ( ) = f ψ ( ) ; ( ) = φ ( ) + µu, for some combner f ( ) d () u, φ ( ) () ode power profe σ U () Correaton Index α 9 8 5 8 8 8 ode ode Fg etwor statstca profe Steady state experments Fg A networ wth dffuson cooperaton strategy In ths wor we expore a smpe combnng rue n whch the aggregated estmate s generated by averagng oca and neghbors prevous estmates, e, X φ ( ) = a(, ) ψ ( ) = φ ( ) + µu, d () u, φ ( ) (8) MSE (db) 98 99 drls LC drls 8 ode Fg 5 MSE performance for both agorthms drls and LC-dRLS across the networ where a(, ) = /deg(), wth deg() denotng the degree of node (number of ncdent ns at ths node, ncudng tsef) Ths scheme expots networ connectvty eadng to more robust agorthms Furthermore, snce more nformaton s aggregated n the oca adaptve fter updates, ndvdua nodes can attan better earnng behavor when compared to the non-cooperatve case, provded that the combners f are we desgned To ustrate ths fact, we run a smuaton exampe wth a networ of = adaptve fters wth M = taps each The topoogy s presented n Fg, whe the networ statstca profe s presented n Fg 8 The neghborhood of a node s the set of nodes drecty connected to t, ncudng tsef
o coop Dffuson 5 5 8 9 EMSE per node (db) 5 5 5 8 9 ode Fg etwor topoogy (sef-oops omtted) for the dffuson LMS exampe etwor Statstcs 8 α σ u, 8 ode 8 σ v, 8 ode Fg 8 etwor statstca profe: regressors power and correaton ndces (eft) and nose power (rght) Fg Steady-state EMSE per node for dffuson LMS COCLUDIG REMARKS We have descrbed adaptve schemes to perform dstrbuted estmaton n a cooperatve fashon When communcaton and energy resources are scarce, the ncrementa LMS scheme may be used When more powerfu processors are avaabe at the nodes, dstrbuted RLS mpementatons wth mted cooperaton can be empoyed For genera topooges, and wth more energy and communcaton resources, one can resort to dffuson LMS strateges The dffuson technques can aso be extended to recursve eastsquares formuatons, whch we w examne esewhere 8 REFERECES EMSE goba (db) 5 5 5 5 o coop Dffuson 5 Tme [] D Estrn, G Potte and M Srvastava Intrumentng the word wth wreess sensor networs, Proc IEEE ICASSP, pp -, Sat Lae Cty, UT, May [] D Bertseas, A new cass of ncrementa gradent methods for eastsquares probems, SIAM J Optm, vo, no, pp 9-9, ov 99 [] C Lopes and A H Sayed, Dstrbuted adaptve ncrementa strateges: formuaton and performance anayss, Proc ICASSP, Tououse, France, May [] A H Sayed Fundamentas of Adaptve Fterng, Wey, J, [5] L Xao, S Boyd and S La, A scheme for robust dstrbuted sensor fuson based on average consensus, Fourth IPS, Los Angees, UCLA, Apr 5, pp - Fg 9 Dffuson LMS - Transent goba EMSE We compare the dffuson LMS wth the non-cooperatve case, n whch the adaptve fters evove ndependenty accessng oca data and consutng ther own past estmates ony We use the goba EMSE average, defned as ζ g = X = ζ (9) as a fgure of mert, where ζ s the oca EMSE at node Fgure 9 shows the goba earnng behavor and Fgure 9 presents the networ ndvdua EMSE profe n steady-state Despte some osses at a few nodes, as for ths case n node 9, on average, the entre system benefts from cooperaton