Trimmed Diffusion Least Mean Squares for Distributed Estimation

Transcription

1 Trimmed Diffusion Least Mean Squares for Distributed Estimation Hong Ji, Xiaohan Yang, Badong Chen School of Electronic and Information Engineering Xi an Jiaotong University Xi an , P.R. China Abstract We consider the problem of distributed estimation, where a set of nodes is required to collectively estimate networ parameters from noisy measurements. The problem is important when modeling a wide class of real-time sensor networs, where efficiency, robustness, and low power consumption are desired features. In this wor, we focus on diffusion-based adaptive solutions that capable to avoid undue influence from outliers, especially in the presence of impulsive noise or dysfunction of certain nodes. We motivate and propose trimmed diffusion least mean square (TDLMS) algorithm that selects normal neighborhood to update the system estimation. We provide performance analysis together with simulation results comparing with existing methods. Index Terms Adaptive networs, diffusion adaptation, distributed estimation, diffusion least mean square. I. INTRODUCTION Distributed estimation deals with the in-networ information extraction where the data collected at nodes distributed over a geographic region. Such algorithms are useful in several contexts, including communications and sensor networs. In the centralized solution to the problem, every node transmits their data to a fusion center for processing, and then receives the resulting estimate from that center. This approach enables the calculation of the global optimization, however has the disadvantage to fully rely on the fusion center. Moreover, centralizing all measurements in a single node may require large amounts of energy and communication resources. The distributed solutions on the other hand, every node only communicates with its closest neighbors and the processing is done locally, which achieve higher robustness, consume fewer resources and handle lighter loads in computation resources. Owing to these merits, distributed estimation has received more and more attention recently and been widely used in cognitive radios [1], environmental monitoring [2], and industrial automation [3]. Diffusion is one of the fundamental governing phenomena in the physical world. In previous studies, many distributed estimation algorithms have been proposed based on diffusion strategies, such as diffusion RLS [4, 5], diffusion LMS [6 9], and diffusion sparse LMS [10]. Prominent solution among these methods is the diffusion least mean square (DLMS) strategy that offers a simple but effective way to implement This wor was supported by the 973 project (2015CB351703) and NNSFC ( , ). distributed adaptive filtering over networs. This wor goes after the problem formulation of the DLMS algorithm. Most of the existing algorithms assume that the errors are Gaussian, and they may perform poorly when the data are non-gaussian, particularly in the case of large outliers (observations that significantly deviates from the bul of data, heavy tails). There is clear interest to develop robust estimators to wor properly in inpulsive environments. For instance, different types of artificial noise in electronic devices, atmospheric noise and lighting spies in natural phenomena, which can be described more accurately using heavy tailed non-gaussian noise models [11, 12]. The least trimmed squares (LTS) estimator, which minimizes the sum of the M smallest squared residuals, is a breathrough technique for robust estimators [13, 14]. In this wor, we will apply the LTS criterion into the problem of distributed adaptive filtering and develop a new algorithm named the trimmed diffusion least mean square (TDLMS), which can avoid undue influence from outliers and achieves better performance when the system is disturbed by impulsive noises. The rest of the paper is organized as follows. In section 2, we briefly introduce the problem formulation and the DLMS algorithm. The trimmed diffusion LMS (TDLMS) are then developed in Section 3. In Section 4, some encouraging simulation results are presented and finally, the conclusion is given in section 5. II. DLMS ALGORITHMS Consider a set of N nodes distributed over some geographic region. At every time instant i, every node taes a scalar measurement d (i) of some random process d (i) and a 1 M regression vector, µ,i, corresponding to a realization of a random process µ,i, which is correlated with d (i) by linear model: d (i) = µ,i ω o + ν (i) (1) where ν (i) represents bacground noise and is independent of µ,i for all and i, and independent of ν l (j) for l or i j. The common goal of all nodes is to identify ω o from the observations within node s neighbor set N in an adaptive manner. The neighbor set N is defined as the set of

2 nodes that have a direct lin with node, including node itself. The cost function of DLMS for each node is defined as J DLMS l N c l, E e l,i 2 l N c l, E d l (i) µ l,i ω 2 (2) where E denotes the expectation operator and c l, are possibly combination weights which satisfy: c l, = 0 if l / N, C1 = 1, 1 T C = 1 T (3) where C with N N non-negative real entries denotes the combination weight matrix and 1 denotes N 1 vector with unit entries. The diffusion strategy is then performed in two stages [6, 9, 15]: combination and adaptation. According to the different order of these two stages, the DLMS is classified into two algorithms: Combine-then-Adapt (CTA) DLMS and Adapt-then-Combine (ATC) DLMS [6, 9]. Replacing the statistical moments by local instantaneous approximations, start with ω l, 1 = 0 for all l, for each time i 0 and for each node, the ATC diffusion DLMS algorithm can be described as follows [6]. ψ,i = ω,i 1 + µ c l, µ l,i (d l(i) µ l,i ω,i 1 ) l N ω,i a l, ψ l,i (diffusion step) l N (4) where µ is a positive step-size chosen by node, denotes complex conjugate-transposition, ψ,i is an intermediate estimate of ω o of node at time instant i, and {a l, } is the weighting coefficients satisfying the condition: a l, = 0 if l / N, 1 T A = 1 T (5) where A is also a real non-negative N N matrix with individual entries {a l, }. Similarly, the CTA diffusion DLMS is: ψ,i 1 = a l, ω l,i 1 (diffusion step) l N ω,i = ψ,i 1 + µ c l, µ l,i l N (d l(i) µ l,i ψ,i 1 ) (6) III. TRIMMED DLMS ALGORITHMS In previous study, most of the diffusion distributed estimation algorithms assume that the noise follows Gaussian distribution. However, the Gaussianity of measurement noise is not guaranteed in many applications. In this wor, we focus on avoiding undue influence from large outliers, such as in the presence of impulsive noise or dysfunction of neighbor nodes, where the DLMS will lose its optimality. As mentioned in section 2, the cost function of DLMS minimize the sum of squared residuals from random measurements of all the neighbors. If however, one or a part of the neighbors data are recorded erroneously, the evaluated coefficients will be biased towards these leverage points. We hereby motivate and Fig. 1. At time i, every node taes measurement {d (i), µ,i }, only the trimmed neighborhood N will be employed to update the system estimation. propose a trimmed DLMS algorithm (see Fig.1), that we see an estimate of ω o only by combining P neighbors of its Q neighbors for each node (Q > P ). By doing so, we trimmed Q P observations with large error residuals and achieves an estimator with higher robustness and accuracy. The cost function of TDLMS for each node is defined as J T DLMS c l, E e l,i 2 l N l N c l, E d l (i) µ l,i ω 2 (7) where N is the node s neighbor set and Ñ is the selected neighbors of node that satisfies the condition Ñ N and e 2 j(1),i e 2 j(2),i e 2 j(p ),i e2 j(q ),i are ordered square errors, P is the trimming constant of node, i.e. j(1),..., j(p ) Ñ and j(1),..., j(q ) N. Analogously, { c l, } are non-negative combination coefficients meeting the condition: c l, = 0 if l / Ñ, 1 T C = 1 T where C is a N N matrix with individual entries { c l, }. The LTS criteria was introduced in [13], it is very robust to outliers and simultaneously possesses desirable asymptotic properties [13, 14, 16]. Accordingly, we obtain following Adapt-then- Combine (ATC) TDLMS and Combine-then-Adapt (CTA) TDLMS algorithms. ψ,i = ω,i 1 + µ ω,i ã l, ψ l,i l N ψ,i 1 ã l, ω l,i 1 l N ω,i = ψ,i 1 + µ c l, µ l N l,i (d l(i) µ l,i ω,i 1 ) (diffusion step) (diffusion step) (8) (9) l N c l, µ l,i (d l(i) µ l,i ψ,i 1 ) (10) The derivations are based on [6] and the details are omitted here.

3 IV. EXPERIMENTAL RESULTS In this section, we present simulation results to show the effectiveness of the proposed TDLMS. We focus on CTA TDLMS algorithm for demonstration purpose (the performance with ATC TDLMS can be tested similarly). We use the measures of the transient networ (mean-square deviation) and steady-state networ for performance comparison under alpha-stable noise, whose characteristic function is [17]: ψ γ,α (ω) = exp( γ ω α ), γ > 0, 0 < α 2 (11) The is defined as 1 K E ω,i ω o 2 (12) K =1 In the simulation, we consider a networ composed of 20 nodes and the length of the unnown parameter vector is 9, which is initialized as ω o = [0.1, 0.2, 0.3, 0.4, 0.5, 0.4, 0.3, 0.2, 0.1] T (13) We let each node connect to Q = 5 neighbor nodes and each node selects P neighbors to update its coefficients. The weights {ã l,i } and { c l,i } are set following [18] { 1/N, if l Ñ {ã l,i } = { c l,i } = (14) 0, otherwise The data µ,i and the noise ν,i are chosen as white Gaussian random sequences and the power of µ,i is uniformly set as 1. The results are averaged from 500 independent experiments. In Fig.2, we show how the values of the trimming constant affect the performance of TDLMS with the averaged convergence curves. We choose four different values: 5, 4, 3 and 2. When P = 5, the TDLMS reduces to DLMS. The stepsize is set at µ T DLMS = 0.02 and the system is trained for 2000 epochs which is long enough to guarantee it reaches the steady-state. The noise parameters are α = 1.5, γ = 0.1. As expected, when P gets larger, the algorithm will converge faster but its steady-state will get worse. From the figure we could probably say that P = 4 is a good choice and in the rest of the simulations we let P = 4. Next, we compare the performance of different algorithms, including LMS without cooperation, DLMS, TDLMS as well as global solutions GLMS and TGLMS [15, 19]. The trimming constant in TGLMS is experimentally set to 16. The step-sizes of the five algorithms are chosen as: µ NoCooperation = 0.05, µ DLMS = 0.01, µ T DLMS = 0.023, µ GLMS = , µ T GLMS = such that they produce roughly the same convergence speed. The results are illustrated in Fig. 3, from which we observe that: 1) the trimmed algorithms (TDLMS, TGLMS) perform better than non-trimmed counterparts (DLMS, GLMS) due to their robustness to outliers; 2) the global adaptive algorithms (GLMS, TGLMS) perform better than diffusion adaptive algorithms (DLMS, TDLMS) due to that global optimization but cost more resources. The P=2 P=3 P=4 P=5(DLMS) iteration Fig. 2. Transient networ of TDLMS for different trimming constants No cooperation DLMS TDLMS GLMS TGLMS iteration Fig. 3. Transient networ for different algorithms testing MSEs at final iteration are summarized in Table 1. The TDLMS algorithm is a little worse than TGLMS however saves a lot computational and communications resources, we argue it be the best choice among all these algorithms. We then let the exponential parameter α in the noise model vary within the range 1 to 2 at a step change 0.1, in order to test the performances of algorithms under noise of different distributions. The parameter settings are the same as in the previous simulation. The s averaged over the last 100 Algorithm No cooperation ± DLMS ± GLMS ± TDLMS ± e-05 TGLMS ± e-05 TABLE I AT FINAL ITERATION

4 No cooperation DLMS TDLMS GLMS TGLMS alpha Fig. 4. Steady-state networ for different α. iterations for different values are plotted in Fig. 4. As we can see that when α < 1.8, the trimmed algorithms perform better than non-trimmed counterparts while little worse when α 1.8. It is mainly because that when α comes near to 2, the noise will become Gaussian whose distribution is not longtailed (hence there are few outliers). Anyhow, the results is just a little inferior when noise follows Gaussian distribution. V. CONCLUSION In most practical situations, the real-world data obtained from the environment are often contaminated by outliers. The DLMS strategy, which is based on MSE criteria, offers an effective way to implement distributed adaptive filtering under the Gaussian assumption while performs poorly when the data are heavy tailed non-gaussian. To address this problem, we apply the idea of LTS to develop a new distributed adaptive filtering algorithm, namely the trimmed diffusion least mean square (TDLMS). Compared with the DLMS algorithm, TDLMS algorithm performs better under impulsive noise with alpha-stable distribution in the context of diffusion distributed estimation. Meanwhile, the proposed algorithm brought no significant increase in computational complexity. REFERENCES [1] Paolo Di Lorenzo, Sergio Barbarossa, and Ali H Sayed, Bio-inspired swarming for dynamic radio access based on diffusion adaptation, in Proc. 20th European Signal Processing Conference, [2] Xianghui Cao, Jiming Chen, Yang Xiao, and Youxian Sun, Building-environment control with wireless sensor and actuator networs: Centralized versus distributed, Industrial Electronics, IEEE Transactions on, vol. 57, no. 11, pp , [3] Jiming Chen, Xianghui Cao, Peng Cheng, Yang Xiao, and Youxian Sun, Distributed collaborative control for industrial automation with wireless sensor and actuator networs, Industrial Electronics, IEEE Transactions on, vol. 57, no. 12, pp , [4] Federico S Cattivelli, Cassio G Lopes, and Ali H Sayed, A diffusion rls scheme for distributed estimation over adaptive networs, in Signal Processing Advances in Wireless Communications, SPAWC IEEE 8th Worshop on. IEEE, 2007, pp [5] Federico S Cattivelli, Cassio G Lopes, and Ali H Sayed, Diffusion recursive least-squares for distributed estimation over adaptive networs, Signal Processing, IEEE Transactions on, vol. 56, no. 5, pp , [6] Federico S Cattivelli and Ali H Sayed, Diffusion lms strategies for distributed estimation, Signal Processing, IEEE Transactions on, vol. 58, no. 3, pp , [7] CassioG Lopes and Ali H Sayed, Distributed processing over adaptive networs, in Proc. adaptive sensor array processing worshop, 2006, pp [8] Cassio G Lopes and Ali H Sayed, Diffusion least-mean squares over adaptive networs., in ICASSP (3), 2007, pp [9] Cassio G Lopes and Ali H Sayed, Diffusion least-mean squares over adaptive networs: Formulation and performance analysis, Signal Processing, IEEE Transactions on, vol. 56, no. 7, pp , [10] Ying Liu, Chunguang Li, and Zhaoyang Zhang, Diffusion sparse least-mean squares over networs, Signal Processing, IEEE Transactions on, vol. 60, no. 8, pp , [11] Konstantinos N Plataniotis, Dimitrios Androutsos, and Anastasios N Venetsanopoulos, Nonlinear filtering of non-gaussian noise, Journal of Intelligent and Robotic Systems, vol. 19, no. 2, pp , [12] Binwei Weng and Kenneth E Barner, Nonlinear system identification in impulsive environments, Signal Processing, IEEE Transactions on, vol. 53, no. 7, pp , [13] Peter J Rousseeuw and Annic M Leroy, Robust regression and outlier detection, vol. 589, John Wiley & Sons, [14] Er-Wei Bai, A random least-trimmed-squares identification algorithm, Automatica, vol. 39, no. 9, pp , [15] Federico S Cattivelli and Ali H Sayed, Diffusion lms algorithms with information exchange, in Signals, Systems and Computers, nd Asilomar Conference on. IEEE, 2008, pp [16] Badong Chen, Xiaohan Yang, Hong Ji, Hua Qu, Nanning Zheng, and Jose C Principe, Trimmed affine projection algorithms, in Neural Networs (IJCNN), 2014 International Joint Conference on. IEEE, 2014, pp [17] Chrysostomos L Niias and Min Shao, Signal processing with alpha-stable distributions and applications, Wiley- Interscience, [18] Vincent Blondel, Julien M Hendricx, Alex Olshevsy, J Tsitsilis, et al., Convergence in multiagent coordi-

5 nation, consensus, and flocing, in IEEE Conference on Decision and Control. IEEE; 1998, 2005, vol. 44, p [19] Federico S Cattivelli and Ali H Sayed, Distributed detection over adaptive networs using diffusion adaptation, Signal Processing, IEEE Transactions on, vol. 59, no. 5, pp , 2011.