Analysis of incremental RLS adaptive networks with noisy links

Size: px

Start display at page:

Download "Analysis of incremental RLS adaptive networks with noisy links"

Douglas Eaton
5 years ago
Views:

1 Analysis of incremental RLS adaptive networs with noisy lins Azam Khalili, Mohammad Ali Tinati, and Amir Rastegarnia a) Faculty of Electrical and Computer Engineering, University of Tabriz Tabriz 51664, Iran a) a rastegar@ieee.org Abstract: In this paper, we study the effect of noisy lins on the steady state performance of incremental recursive least-squares (RLS) adaptive networs. In our analysis, using weighted spatial-temporal energy conservation approach, we arrive a variance relation which contains moments that represent the effects of noisy lins. We evaluate these moments and derive closed form expressions for the mean-square deviation (MSD) and excess mean-square error (EMSE) to explain the steady-state performance at each individual node. The derived expressions have good match with simulations. Keywords: incremental networs, distributed estimation, noisy lins Classification: Science and engineering for electronics References [1] C. G. Lopes and A. H. Sayed, Incremental adaptive strategies over distributed networs, IEEE Trans. Signal Process., vol. 55, no. 8, pp , Aug [2] A. H. Sayed and C. G. Lopes, Adaptive processing over distributed networs, IEICE Trans. Fundamentals, vol. E90 A, no. 8, pp , Aug [3] L. Li, J. A. Chambers, C. G. Lopes, and A. H. Sayed, Distributed estimation over an adaptive incremental networ based on the affine projection algorithm, IEEE Trans. Signal Process., vol. 58, no. 1, pp , Jan [4] C. G. Lopes and A. H. Sayed, Diffusion least-mean squares over adaptive networs: Formulation and performance analysis, IEEE Trans. Signal Process., vol. 56, no. 7, pp , July [5] N. Taahashi, I. Yamada, and A. H. Sayed, Diffusion least-mean squares with adaptive combiners: Formulation and performance analysis, IEEE Trans. Signal Process., vol. 8, no. 9, pp , Sept [6] F. S. Cattivelli, C. G. Lopes, and A. H. Sayed, Diffusion recursive leastsquares for distributed estimation over adaptive networs, IEEE Trans. Signal Process., vol. 56, no. 5, pp , May [7] A. H. Sayed, Fundamentals of adaptive filtering, Wiley, NJ,

2 1 Introduction The problem of estimating an unnown parameter from data collected by a set of nodes arises in several applications. Among the proposed solutions in the literature, adaptive networs are appealing solutions when the statistical information about the underlying processes of interest is not available. Cooperative processing in conjunction with adaptive filtering per node allows the networ to account for time variations in the signal statistics. Adaptive networs may be referred to as incremental networs or diffusion networs, depending on the manner by which the nodes communicate with each other. The incremental LMS [1], incremental RLS [2], incremental techniques based on the affine projection algorithm [3], are examples of adaptive networs with incremental mode of cooperation. When more communication and energy resources are available, a diffusion cooperative scheme can be applied. In these schemes each node updates its estimate using all available estimates from its neighbors, as well as data and its own past estimate [4, 5, 6]. In [1, 2, 3, 4, 5, 6], the communication lins between nodes are assumed to be ideal. In this paper we study the steady-state performance of incremental RLS adaptive networ with unreliable communications modeled by noisy lins. We first show that the performance of incremental RLS adaptive networ drastically deteriorates when lins between nodes are noisy. Then, we use the weighted energy conservation argument and derive closed-form expressions to explain the steady-state performance at each individual node. Simulation results are also presented to clarify the derived expressions. Notation: We adopt boldface letters for random quantities. Symbol * denotes complex conjugation (scalars) and Hermitian transpose (matrices). 2 Incremental RLS with noisy lins Consider a distributed networ which is deployed to estimate an unnown vector w o from measurements collected at nodes. Each node has access to time-realizations {d (i),u,i } of zero-mean spatial data {d, u }, where each d (i) is a scalar measurement and each u,i isa1 M row regression vector. At each time instant i, the networ has access to the following data y i =[d 1 (i) d 2 (i) d N (i)] T and H i =[u 1,i u 2,i u N,i ] T (1) Note that y i and H i are in fact snapshot matrices revealing the networ data status at time i. Collecting all the data (available up to time i) intoglobal matrices Y i and H i yields Y i =[y 0 y 1 y i ] T and H i =[H 0 H 1 H i ] T (2) Now we pose regularized weighted least-squares (LS) problem to estimate w o as follows ] min [λ i+1 w Πw + Y i H i w 2 (3) Wi w where Π is a regularization matrix, and the weighting matrix is given by [2] W i =diag{λ i D, λ i 1 D,...,λD,D} (4) 624

3 with a spatial weighting factor D =diag{γ 1,γ 2,...,γ N }, γ i 0 and (time) forgetting factor 0 λ 1. The solution of problem (3) is given by [2] where w i = P i H i W i Y i (5) P i = ( λ i+1 Π+H i W i H i ) 1 (6) In [2] the incremental RLS adaptive networ is introduced to estimate w o recursively in a distributed manner which can be summarized as follows ψ (i) 0 w i 1 ; P 0,i λ 1 P i 1 ψ (i) = ψ (i) 1 +Γu,i (d (i) u,i ψ (i) ] P,i 1 Γu,i u,ip,i 1 P,i = λ 1 [ w i ψ (i) N ; P i P N,i 1 ) (7) where ψ (i) is the local estimate of w o at node at time i and Γ is Γ= γ 1 λ 1 P,i 1 + λ 1 u,i P,i 1 u,i (8) The given algorithm in (7) wors as follows [2]: at each time i, the local estimate ψ (i) at node is the LS solution considering data blocs Y i 1 and H i 1 in addition to the data collected along the path. At the end of the cycle, ψ (i) N will contain precisely the desired solution w i. Note that in this implementation ψ (i) is transmitted to the next node in the path, while P,i is estimated locally and independent from the neighbor nodes. In the presence of noisy lins, ψ (i) in (7) is updates as ψ (i) = ψ (i) 1 + q,i +Γu,i e (i) Γu,i u,iq,i (9) where e (i) =d (i) u,i ψ (i) 1 and the M 1 vector q,i represents the time realization of lin noise between node 1and which is assumed to be additive, zero-mean with covariance matrix Q = E(q q ). No distributional assumptions are required on the noise sequence. The effect of noisy lins on the performance of incremental RLS adaptive networ is obvious from Fig. 1. (The simulation setup is described in section (4)). As it is clear from Fig. 1, the performance of distributed adaptive estimation algorithm drastically decreases when lins are noisy. Fig. 1. The MSD learning curve for incremental RLS. 625

4 3 Performance analysis Our analysis relies on the energy conservation approach of [7]. To carry out the performance analysis, we first need to assume a model for the data as is commonly done in the literature of adaptive algorithms. Thus, in the subsequent analysis we consider the following assumptions A.1. Linear model d (i) =u,i w o + v (i) where v (i) is white noise term with variance σ 2 v, and is independent of {d l(j), u l,j } for all l, j. A.2. u,i are spatially and temporary independent and {u } arise from a source with circular Gaussian distribution with covariance matrix R u,. A.3. q,i is independent of {u l,j, v l (j), q l,j } for all l, j. Note that in steady-state analysis, it is desired to evaluate the MSD and EMSE for every node which are defined as η ( ) E( ψ 1 2 ( ) )=E( ψ 1 2 I) (MSD) (10) ζ E( e a, ( ) 2 ( ) )=E( ψ 1 2 R u, ) (EMSE) (11) where e a, (i) u ψ(i) (i),i 1 and ψ w o ψ (i). By subtracting wo from both sides of update equation (9) we get ψ (i) = ψ (i) 1 q,i Γu,i e (i)+γu,i u,iq,i (12) Equating the weighted norm of both sides of the previous equation, tae expectation of both sides and using assumptions (A.1)-(A.3) we obtain E( ψ 2 Σ)=E( ψ 1 2 Σ )+σ2 v, E( u 2 ΓΣΓ)+E( u 2 ΓΣΓq u u q ) +E( q 2 Σ) E(q ΣΓu u q ) E(q u u ΓΣq )(13) where Σ Σ E(ΣΓu u + u u ΓΣ) + E( u 2 ΓΣΓu u ) (14) Recursion (13) is a variance relation that can be used to infer the steady-state performance at every node. Note that Σ is solely regressors-dependent and, therefore, decoupled from the weight error vector. Now using the eigndecomposition R u, = U Λ U, (where Λ is a diagonal matrix with the eigenvalues of R u, and U is unitary) we define the transformed quantities ψ U ψ, ψ 1 U ψ 1, u u U, q U q Σ U ΣU, Σ U Σ U, Γ U ΓU (15) Using the definitions in (15), Eqs.(13) and (14) can be rewritten as E( ψ 2 Σ )=E( ψ 1 2 Σ )+σ2 v, E( u 2 Γ ΣΓ )+E ( u 2 Γ ΣΓ q u u ) q +E( q 2 Σ ) E(q ΣΓu u q ) E(q u u Γ Σq )(16) Σ Σ E(Σ Γu u + u u Γ Σ) + E( u 2 Γ ΣΓ u u ) (17) 626

5 To proceed, we evaluate the moments in (16) and (17) which are E( q 2 Σ ) = E( u 2 Γ ΣΓ ) = Tr[ B Σ ] Λ Γ Σ Γ ] E( u 2 Γ ΣΓ q u u q ) = Tr[B (Λ Tr[Γ Σ ΓΛ ]+δλ Γ Σ ΓΛ )] E(q ΣΓu u q ) = Tr[Λ B Σ Γ] E( u 2 Γ Σ Γ u u ) = (Λ Tr[Γ Σ ΓΛ ]+δλ Γ Σ ΓΛ ) (18) where B = U Q U and δ = 1 for circular complex data and δ = 2 for real data. Replacing these moments, (16) and (17) can be rewritten as E( ψ 2 Σ )=E( ψ 1 2 Σ )+σ2 v, Tr[Λ Γ ΣΓ] + Tr[B Σ] +Tr[B (Λ Tr[Γ ΣΓΛ ]+δλ Γ ΣΓΛ )] 2Tr[Λ B ΣΓ] (19) Σ = Σ ( ΣΓΛ +Λ Γ Σ) + (Λ Tr[Γ ΣΓΛ ]+δλ Γ ΣΓΛ ) (20) Note from (20) that choosing Σ to be diagonal, will be diagonal Σ as well, suggesting a more compact notation. Thus, we introduce the M 1 vectors σ diag{σ} σ diag{σ } b diag{λ } (21) Using the diagonal notation [1, 2] we obtain linear relation σ = F σ where F is a M M matrix that includes statistics of local data as with c =diag{λ 1 As a result, expression (19) becomes where g is a row vector as F =(1 2β + δβ 2 )I + β2 b c T (22) } and β given by 1 λ, for λ 1 γ β = 1 1 λ, for smaller λ (23) γ 1 λ+(1 λ)m E( ψ 2 diag{σ} )=E( ψ 1 2 diag{σ } )+g σ (24) g = σ 2 v, β2 ct + (diag{b }) T F (25) Comparing (25) with g in [2] we can see that (diag{b }) T F explicitly accounts for the noisy lin effects. Thus, following the similar steps given in [1, 2] we can obtain the following expressions for MSD and EMSE where r diag{i} and η = a (I Π,1 ) 1 r, (MSD) (26) ζ = a (I Π,1 ) 1 b, (EMSE) (27) Π,l F +l 1 F +l...f N F 1...F 1, l =1,...,N (28) a g Π,2 + g +1 Π, g 2 Π,N + g 1 (29) 627

6 4 Simulation In this section we provide computer simulations comparing the theoretical to simulation results. To this aim we consider a networ with N = 15nodes where each local filter has M = 4 taps. Each node accesses independent Gaussian regressors u,i where their eigenvalue spread is ρ = 5. The observation noise variances σv, 2 and Tr[R u,] are shown in Fig. 2. We also assume Q =10 4 I M to model the covariance matrix of lin noise and λ = The system evolves for 2000 iterations and the steady-state values are obtained by averaging the last 200 time samples. The curves are obtained by averaging over 100 experiments with λ = In Fig. 3 the steady-state of MSD and EMSE are plotted. It is clear from Fig. 3 that there is a good match between simulations and theory which is an evidence that the derived expressions can describe the steady-state performance of incremental RLS adaptive networ with noisy lins. Fig. 2. The Observation noise profile, σ 2 v, and Tr(R u,). Fig. 3. The MSD and EMSE per node comparing simulation with theory. 5 Conclusion In this paper, we studied the effect of noisy lins on the performance of incremental RLS adaptive networs. Using weighted spatial-temporal energy conservation relation, we arrived a variance relation which contains moments that represent the effects of noisy lins. We evaluated these moments and derived closed-form expressions for the MSD and EMSE to explain the steadystate performance at each individual node. The derived expressions have good match with simulations. 628

A Quality-aware Incremental LMS Algorithm for Distributed Adaptive Estimation

International Journal of Automation and Computing 11(6), December 2014, 676-682 DOI: 10.1007/s11633-014-0838-x A Quality-aware Incremental LMS Algorithm for Distributed Adaptive Estimation Wael M. Bazzi