Information Weighted Consensus

Transcription

1 Informaton Weghted Consensus A. T. Kamal, J. A. Farrell and A. K. Roy-Chowdhury Unversty of Calforna, Rversde, CA Abstract Consensus-based dstrbuted estmaton schemes are becomng ncreasngly popular n sensor networks due to ther scalablty and fault tolerance capabltes. In a consensusbased state estmaton framework, multple neghborng nodes teratvely communcate wth each other, exchangng ther own local estmates of a target s state wth the goal of convergng to a sngle state estmate over the entre network. However, the state estmaton problem becomes challengng when a node has lmted observablty of the state. In addton, the consensus estmate s sub-optmal when the cross-covarances between the ndvdual state estmates across dfferent nodes are not ncorporated n the dstrbuted estmaton framework. The crosscovarance s usually neglected because the computatonal and bandwdth requrements for ts computaton grow exponentally wth the number of nodes. These lmtatons can be overcome by notng that, as the state estmates at dfferent nodes converge, the nformaton at each node becomes redundant. Ths fact can be utlzed to compute the optmal estmate by proper weghtng of the pror state and measurement nformaton. Motvated by ths dea, we propose nformaton-weghted consensus algorthms for dstrbuted maxmum a posteror parameter estmates, and ther extenson to the nformaton-weghted consensus flter ICF for state estmaton. We show both theoretcally and expermentally that the proposed methods asymptotcally approach the optmal centralzed performance. Smulaton results show that ICF s robust even when the optmalty condtons are not met and has low communcaton requrements. I. INTRODUCTION Dstrbuted estmaton schemes are becomng ncreasngly popular n the sensor networks communty due to ther scalablty for large networks and hgh fault tolerance. Unlke centralzed schemes, dstrbuted schemes usually rely on peer-to-peer communcaton between sensor nodes and the task of nformaton fuson s dstrbuted across multple nodes. In a sensor network, each sensor may get multple measurements of a target s state. The objectve of a dstrbuted estmaton framework s to mantan an accurate estmate of the target s state usng all the measurements n the network wthout requrng a centralzed node for nformaton fuson. Among many types of dstrbuted estmaton schemes, consensus algorthms [1] are schemes where each node, by teratvely communcatng wth ts network neghbors, can compute a functon of the measurements at each node e.g. average. The consensus estmates asymptotcally converge to the global result. In practce, only a lmted number of teratons can be performed due to lmted bandwdth Ths work was partally supported by ONR award N ttled Dstrbuted Dynamc Scene Analyss n a Self-Confgurng Multmodal Sensor Network. and target dynamcs. Thus, true convergence may not be always reached. In the presence of state dynamcs, usually a predctor-corrector model s used for state estmaton, where a state predcton s made from the pror nformaton and corrected usng new measurements. The Kalman Consensus Flter KCF [2] s a popular dstrbuted state estmaton framework based on the average consensus algorthm. KCF works well n stuatons where each node gets a measurement of the target. In a sensor network, a node mght have lmted observablty when t does not have any measurement of a target avalable n ts local neghborhood consstng of the node and ts mmedate network neghbors. Due to lmted observablty and lmted number of teratons, the node becomes nave about the target s state. A nave node contans less nformaton about the state. If a nave node s estmate s gven an equal weght n the nformaton fuson scheme as n KCF, the performance of the overall state estmaton framework may decrease. The effect of navety s severe n sparse networks where the total number of edges s much smaller than the maxmum possble number of edges. The Generalzed Kalman Consensus Flter GKCF [3], was proposed to overcome ths ssue by utlzng a weghtedaveragng consensus scheme where the prors of each node were weghted by ther covarance matrces. The reason that these dstrbuted schemes are usually sub-optmal s that the cross-covarances between the prors across dfferent nodes are not ncorporated n the estmaton framework. As the consensus progresses, the errors n the nformaton at each node become hghly correlated wth each other. Thus, to compute the optmal state estmate, the error cross-covarances cannot be neglected. However, t s dffcult to compute the cross-covarance n a dstrbuted framework. We note that n a consensus-based framework, the state estmates at dfferent nodes acheve reasonable convergence over multple teratons. At ths pont, each node contans almost dentcal/redundant nformaton. Ths fact can be utlzed to compute the optmal estmate n a dstrbuted framework wthout explctly computng the cross-covarances. Motvated by ths dea, we propose nformaton-weghted consensus algorthms for dstrbuted state and parameter estmaton whch are guaranteed to converge to the optmal centralzed estmates as the pror state estmates become equal at dfferent nodes.e., the total number of teratons approach to nfnty at the prevous tme step. We also show expermentally that even wth lmted number of teratons, the proposed algorthms acheve near-optmal performance.

2 The ssue of navety and optmalty s handled by proper nformaton weghtng of the pror and measurement nformaton. The communcaton bandwdth requrement s also low for the proposed methods. Related works Consensus algorthms [1] are one of the many types of dstrbuted nformaton fuson schemes. Consensus algorthms are protocols that are run ndvdually by each agent where each agent communcates wth just ts network neghbors and corrects ts own nformaton teratvely usng the nformaton sent by ts neghbors. The protocol, over multple teratons, ensures the convergence of all the agents n the network to a sngle consensus. Consensus algorthms have been extended to perform varous tasks n a network of agents such as lnear algebrac operatons lke SVD, least squares, PCA, GPCA [4], dstrbuted state and parameter estmaton frameworks such as the KCF [2], the GKCF [3] and the dstrbuted maxmum lkelhood estmator DMLE [5]. A detaled revew of dstrbuted state estmaton methods and comparsons wth centralzed and decentralzed approaches can be found n [6]. These dstrbuted state and parameter estmaton frameworks have been appled n varous felds ncludng camera networks for dstrbuted mplementatons of 3-D pont trangulaton, pose estmaton [4], trackng [7], acton recognton [7], [8], collaboratve trackng and camera control [9] etc. The ssue of lmted observablty of the ndvdual nodes has been consdered prevously n dstrbuted estmaton frameworks. In [10], the authors proposed a hybrd peerto-peer/herarchcal framework for state estmaton requrng fuson centers. Thus, the soluton was not fully dstrbuted. In ths paper, we propose a fully dstrbuted framework wthout the requrement of fuson centers. Average consensus: Revew Average consensus [1] s a popular dstrbuted algorthm for computng the arthmetc mean of some values. Suppose, there are N nodes and each node C has the state a. Usng average consensus, the average value of these states.e. 1 N N a can be computed n a dstrbuted manner. Here, a can be a scalar, a vector or a matrx. In average consensus algorthm, each node ntalzes ts consensus state as a 0 a. At the begnnng of teraton k, a node C sends ts prevous state a k1 to ts mmedate network neghbors C j N and smlarly receves the neghbors prevous states a k1 j. Then t updates ts own state as a k a k1 + ɛ j a k1. 1 j N a k1 By teratvely dong so, the values of the states at all the nodes converge to the average of the ntal values. Here ɛ s the rate parameter whch should be chosen between 0 and 1 max, where max s the maxmum degree of the network graph G. Usng a hgher value of ɛ would gve a hgher rate of convergence. However, choosng a value greater than or 1 equal to max would render the system unstable. II. PROBLEM FORMULATION Consder a sensor network wth N sensors. The communcaton n the network can be represented usng an undrected connected graph G = C, E. The set C = {C 1,..., C N } contans the vertces of the graph and represents the sensor nodes. The set E contans the edges of the graph whch represents the avalable communcaton channels between dfferent nodes. The set of nodes havng drect communcaton channel wth node C sharng an edge wth C s represented by N. The true state of the targets s represented by xt R p. For multple targets xt s the concatenaton of the ndvdual state vectors. A data assocaton scheme mght be necessary for multple targets. As our focus s on the dstrbuted state estmaton problem, we would assume that the data assocaton s gven. For smplcty of notaton, tme ndex t wll be dropped where the ssue under consderaton can be understood wthout t. Each node has a pror estmate of x as x R p. The error n the pror estmate at C s η = x x Rp wth covarance P R p p. The nformaton form of the estmators wll be used throughout ths paper. Thus, we wll have notatons n the nverse covarance form whch s also known as the nformaton/precson matrx. We denote the pror nformaton matrx of node C as W R p p, where W = P 1. 2 The observaton of node C s denoted by z R m wth nose covarance R R m m, where m s the length of the measurement vector at node C. The observatons from all the nodes are modeled as, Z = Hx + ν. 3 Here, Z = [z T 1, z T 2,..., z T N ]T R m and observaton matrx H = [H T 1, H T 2,..., H T N ]T R m p where, H R m p and m = N m. Observaton nose ν s assumed to be Gaussan wth ν N 0, R R m. The nverse of R R m m s denoted by B R m m. The measurements are assumed to be uncorrelated across nodes. Thus, the measurement nformaton matrx s block dagonal and can be expressed as, B B 2 B = B N Here, B = R 1 R m m. III. DISTRIBUTED MAP ESTIMATION DMAP In ths secton, frst we wll present the centralzed soluton for our problem and later wll derve the dstrbuted mplementaton of t.

3 A. Centralzed case The task n a centralzed a posteror estmaton process s to estmate the state x from the measurements Z and pror state x c wth nformaton matrx Wc. The centralzed maxmum a posteror MAP [11] estmate x + c and ts nformaton matrx W c + n nformaton form can be expressed as, 1 x + c = Wc + H T BH Wc x c + H T BZ, 5 W c + = Wc + H T BH. 6 Let us defne U = H T B H and u = H T B z. Due to the block dagonal structure of B, we have H T BH = H T BZ = H T B H = U, 7 H T B z = u. 8 Thus, we have the centralzed MAP estmate as, 1 x + c = Wc + U W c x c + u N 1 W = c N + U N W c N x c + u, 9 W W c + = c N + U. 10 B. Dervaton of Dstrbuted MAP Now, we wll derve the dstrbuted mplementaton of the centralzed MAP estmates of In the centralzed case, after the estmaton at tme t 1, we have the state estmate x + c t 1 that s used as a pror x c t for the estmaton at tme t. For the dstrbuted case, each node wll have ts own pror x t. Ideally, for all, x t should be equal to x c t. However, n practce, due to lmted number of consensus teratons at prevous tme steps, there may be some dscrepances among the prors n the dstrbuted case. Here, we wll derve the dstrbuted MAP estmaton framework for the case where the prors n the dstrbuted framework have converged to the pror n the centralzed framework. Later n Sec IV, we wll dscuss the mportance of ths case for consensus-based estmaton frameworks. Under ths condton, for all, we have x = x c and = Wc. Thus, from 9 and 10 we have W x + c = W + c = N W N + U W N + U 1 N W N x + u Intutvely, ths dvson by the number of nodes N s very mportant because when all the nodes have the same pror state, from the centralzed perspectve, the state nformaton Algorthm 1 Dstrbuted Maxmum A Posteror DMAP at C Input: Pror state estmate x, nformaton matrx W, observaton matrx H, measurement z, measurement nformaton matrx B, consensus rate parameter ɛ and total consensus teratons K. 1 Compute ntal nformaton matrx and vector V 0 1 N W + H T B H 19 v 0 1 N W x + H T B z 20 2 Perform average consensus on V 0 and v0 ndependently for k = 1 to K do a Send V k1 and v k1 to all neghbors j N b Receve V k1 j and v k1 j from all neghbors j N c Update: V k V k1 + ɛ V k1 j V k1 j N v k v k1 + ɛ j N v k1 j v k1 end for 3 Compute MAP estmate and Informaton matrx x + V K 1 v K 23 W + NV K 24 Output: MAP estmate x + and nformaton matrx W +. matrx should only be used once n the calculaton of the MAP estmate. However, n the dstrbuted case, f ths dvson s not performed, the pror nformaton gets N tmes more weght than t should. As a result, the estmator gets more based towards the pror states and gves less weght to the new measurement nformaton. Let, V 0 = W N + U, 13 v 0 = W N x + u. 14 Each node can compute V 0 and v 0 from the nformaton avalable to t.e. x, W, u, U and N. Then, each node communcates to ts neghbors wth ts own nformaton matrx V k R p p and nformaton vector v k R p, usng average consensus algorthm as descrbed n Sec I to asymptotcally compute the global averages of each of these two quanttes as N V0 lm k Vk =, 15 N N lm k vk = v0 N. 16 Therefore, from we have x + c = lm NV k 1 Nv k k = lm V k 1 v k 17 k W c + = lm k NVk 18 From 17 and 18 we can see that as k, the state estmate and nformaton matrx at each node converges to the optmal centralzed MAP estmate. The DMAP framework s summarzed n Algorthm 1.

4 IV. INFORMATION-WEIGHTED CONSENSUS FILTER ICF In the prevous secton, we derved a dstrbuted MAP estmator for the case where each node has the pror nformaton that equals to the pror n the centralzed framework. In ths secton, we wll extend the DMAP algorthm consderng state dynamcs. The state evoluton s modeled usng the followng lnear dynamcal model, xt + 1 = Φxt + γt. 25 Here Φ s the state propagaton matrx and process nose γt N 0, Q. For the centralzed case, where the state estmate at tme t s x + c t wth nformaton matrx W + c t, we have the Kalman flter [12] state propagaton equatons as, W c t + 1 = ΦW + c t 1 Φ T + Q 1, 26 x c t + 1 = Φx + c t. 27 Combnng ths wth our dstrbuted MAP estmator n Sec III-B, we get the nformaton-weghted consensus flter ICF. The approach s summarzed n Algorthm 2. At each tme step, f k, the DMAP estmator n Algorthm 1 guarantees that the prors for the next tme step at each node wll be equal to the optmal centralzed one. Ths n turn sets the optmalty condton for the next tme step. Ths guarantees the optmalty of Algorthm 2 wth k at each tme step. In realty, reachng true convergence may not be possble due to lmted number of consensus teratons. The number of teratons needed to reach a reasonable convergence depends on the network sze and number and poston of nave nodes n the network graph. In Sec V, we wll show expermentally that n case of only one or a few teratons, ICF s robust to small dscrepances between the state estmates across the nodes and acheves near optmal performance. In a practcal mplementaton scenaro, at system startup or for the frst few teratons n a nave node, V K n 32 can be 0 thus, not nvertble, f there s no pror or measurement nformaton avalable n the local neghborhood. At that stuaton, a node wll not perform step 4 and 5 n Algorthm 2 untl t receves non-zero nformaton from ts neghbors through step 3 or gets a measurement tself through step 1 yeldng V K to be non-zero. V. EXPERIMENTAL EVALUATION In ths secton, we evaluate the performance of the proposed ICF algorthm n a smulated envronment and compare t wth other methods: the Centralzed Kalman Flter CKF [12], the Kalman Consensus Flter KCF [2] and the Generalzed Kalman Consensus Flter GKCF [3]. We smulate a camera network n ths experment, where the state estmaton algorthms are used for trackng a target roamng wthn a space. The target s ntal state vector s random. The target s state vector s a 4D vector, wth the 2D poston and 2D velocty components. The Algorthm 2 ICF at node C at tme step t Input: Pror state estmate x t, pror nformaton matrx W t, observaton matrx H, consensus rate parameter ɛ, total consensus teratons K, state transton matrx Φ and process covarance Q. 1 Get measurement z and measurement nformaton matrx B 2 Compute ntal nformaton matrx and vector V 0 1 N W t + HT B H 28 v 0 1 N W tx t + HT B z 29 3 Perform average consensus on V 0 and v0 ndependently for k = 1 to K do a Send V k1 and v k1 to all neghbors j N b Receve V k1 j and v k1 j from all neghbors j N c Update: V k V k1 + ɛ V k1 j V k1 j N v k v k1 + ɛ j N v k1 j v k end for 4 Compute a posteror state estmate and nformaton matrx for tme t 5 Predct for next tme step t + 1 x + t VK 1 v K 32 W + t NVK 33 W t + 1 ΦW + t1 Φ T + Q 1 34 x t + 1 Φx+ t 35 Output: State estmate x + t and nformaton matrx W+ t. ntal speed s unformly pcked from 1020 unts per tme step, wth a random drecton unformly chosen from 0 to 2π. The targets evolve for 40 tme steps usng the target dynamcal model of 25. The state transton matrx Φ and process covarance Q are chosen as the followng Φ= , Q= The target randomly changes ts drecton and s reflected back when t reaches the grd boundary. A set of N = 5 camera sensors montor the area. The observatons are generated usng 3. The observaton matrx H and the communcaton adjacency matrx A are set as the followng [ ] H =, A= If a camera has a measurement, measurement nformaton matrx B = 0.01I 2 s used. Otherwse, B s set to 0I 2. The consensus rate parameter ɛ s set to 0.65/ max where max = 2. For the frst experment, the ntal pror state x 1 and pror covarance P 1 s set equal at each node. A dagonal

5 KCF at C 1 KCF at C 2 KCF at C 3 KCF at C 4 KCF at C 5 ICF at C 1 ICF at C 2 ICF at C 3 ICF at C 4 ICF at C 5 Fg. 1: Here the smulaton framework s shown along wth the trackng results for KCF and ICF for K = 2. The rectangular boxes represent the same smulaton area from each camera s perspectve. Wthn each rectangle, the blue trangles represent a camera s feld of vew FOV. The green lne represents the ground truth track and the blue dots represent the observatons at the ndvdual cameras. The state estmates of KCF and ICF are shown n black and red lnes respectvely. The gray ellpses depct the covarances of the estmates. It shows that even for K = 2 and the presence of navety, ICF performs sgnfcantly better than KCF. matrx s used for P 1 wth the man dagonal elements as {100, 100, 10, 10}. The ntal pror state x 1 s generated by addng zero-mean Gaussan nose of covarance P 1 to the ground truth state. The trackng results for KCF and ICF s shown n Fg. 1. In ths experment, the total number of consensus teratons K s set to 2. The cameras C 3, C 4 and C 5 s nave about the target s state for most of the tme steps. Compared to the state estmates of KCF, n all the cameras, especally n the nave nodes, the state estmates of ICF are much closer to the ground truth. As a measure of performance, we compute the estmaton error e, defned as the Eucldean dstance between the ground truth poston and the estmated posteror poston. The mean error ē s computed by averagng the errors over all cameras and tme steps. In ths experment, the mean error ē for KCF s and for ICF s Next, we compare the performance of KCF, GKCF and ICF wth CKF after convergence. The smulaton s run Method Mean error Standard devaton of error KCF K= GKCF K= ICF K= ICF K= ICF K= ICF K= CKF TABLE I: Mean and standard devaton of the errors of dfferent methods for dfferent total number of consensus teratons K. Mean Error, ē KCF GKCF ICF CKF Independent smulaton runs Fg. 2: Mean error of the converged estmates of dfferent algorthms at multple ndependent smulaton runs. It supports the theory that wth hgh number of consensus teratons e.g., K = 100, ICF approaches the optmal centralzed performance. 15 tmes wth dfferent randomly generated tracks. The convergence was assumed after 100 consensus teratons. The results of ths experment are shown n Fg 2. It s apparent from ths fgure that ICF performs better than KCF and GKCF and acheves optmal centralzed performance wth hgh number of consensus teratons. In Table I, the mean and standard devaton of the errors for each method n ths experment are shown.

6 Fnally, to show the robustness of ICF, we conduct an experment by relaxng the optmalty condton where the ntal pror states and covarances are dfferent at dfferent nodes. The pror states are ntalzed by addng Gaussan noses generated usng the correspondng pror covarance matrces to the ntal ground truth states. The ntal pror error across dfferent cameras are correlated wth correlaton coeffcent ρ = 0.5. The total number of consensus teratons K s vared from 1 to 20 wth ncrements of 1. A total of 15 cameras are used and the camera locatons, orentatons, network topologes and ground truth tracks are generated randomly. The results of ths experment s shown n Fg 3. The smulaton results are averaged over 400 ndependent smulaton runs. The mean error sold lnes and the standard devaton ±0.2σ wth dotted lnes for dfferent methods are shown usng dfferent colors. The results show that ICF acheves near-optmal performance even when the optmalty condtons are not met. Ths s because ICF s a consensus based approach and rrespectve of the ntal condton, after several tme steps or consensus teratons, the states reach a reasonable converge. ICF was proved to be optmal wth converged pror states. Thus, after a few tme steps t acheves near-optmal performance as the system approaches the optmalty condtons. Comparng Fg 2 and 3 we can see that the performance of KCF deterorated n the latter but the performance of GKCF and ICF was not affected much. As the number of cameras was ncreased from 5 to 15 n Fg. 3 wth the same number of neghbors per node, the number of nave nodes ncreased. Ths shows that unlke KCF, ICF handles the ssue of navety well. The ssue of navety n dstrbuted frameworks was one of the man motvatons for the dervaton of the ICF approach. ICF requres low communcaton bandwdth whch s half the requred bandwdth of GKCF and comparable wth that of KCF. The nformaton sent from each node to a neghbor at each teraton for varous methods s shown n Table II. Method KCF GKCF ICF Message content For 1 st consensus step: u R p, U R p p, x R p For addtonal consensus steps: x R p u R p, U R p p, x R p, W R p p v R p, V R p p TABLE II: Informaton sent at each consensus step VI. CONCLUSION In ths paper, we proposed nformaton-weghted consensus algorthms,.e., a dstrbuted maxmum a posteror DMAP estmaton framework for parameter estmaton and ts extenson to a nformaton-weghted consensus flter ICF for state estmaton. We showed both theoretcally and expermentally that ICF approaches the optmal centralzed performance even n the presence of nave nodes as the total number of consensus teratons K ncreases. Smulaton results showed that ICF s robust even when the optmalty Mean Error, ē and S.D. ±0.2σ KCF GKCF ICF CKF Number of Consensus Iteratons, K Fg. 3: Performance comparson of dfferent approaches by varyng the total number of consensus teratons, K. Each lne represents the mean error ē for each method. The dotted lnes represent the standard devaton ±0.2σ. The prors at t = 1 were set to be dfferent and correlated wth ρ = 0.5. Ths fgure shows that ICF s robust and acheves near-optmal performance even when the optmalty condtons are not met. condtons were not met and has near-optmal performance whle requrng low communcaton resources. REFERENCES [1] R. Olfat-Saber, J. A. Fax, and R. M. Murray, Consensus and cooperaton n networked mult-agent systems, n Proceedngs of the IEEE, 2007, p [2] R. Olfat-Saber, Kalman-consensus flter: Optmalty, stablty, and performance, n IEEE Conference on Decson and Control, Dec. 2009, pp [3] A. T. Kamal, C. Dng, B. Song, J. A. Farrell, and A. K. Roy- Chowdhury, A generalzed Kalman consensus flter for wde area vdeo networks, n IEEE Conference on Decson and Control, Dec. 2011, pp [4] R. Tron and R. Vdal, Dstrbuted computer vson algorthms, Sgnal Processng Magazne, IEEE, vol. 28, no. 3, pp , May [5] A. T. Kamal, J. A. Farrell, and A. Roy-Chowdhury, Optmal dstrbuted maxmum a posteror estmaton, n Intl. Conf. on Image Processng, [6] M. Taj and A. Cavallaro, Dstrbuted and decentralzed multcamera trackng, IEEE Sgnal Processng Magazne, vol. 28, no. 3, pp , May [7] B. Song, A. T. Kamal, C. Soto, C. Dng, J. A. Farrell, and A. K. Roy-Chowdhury, Trackng and actvty recognton through consensus n dstrbuted camera networks, IEEE Trans. on Image Processng, vol. 19, no. 10, pp , Oct [8] A. T. Kamal, B. Song, and A. K. Roy-Chowdhury, Belef consensus for dstrbuted acton recognton, n Intl. Conf. on Image Processng, Sept. 2011, pp [9] B. Song, C. Dng, A. T. Kamal, J. A. Farrell, and A. K. Roy- Chowdhury, Dstrbuted camera networks: Integrated sensng and analyss for wde area scene understandng, IEEE Sgnal Processng Magazne, vol. 28, no. 3, pp , May [10] R. Olfat-Saber and N. F. Sandell, Dstrbuted trackng n sensor networks wth lmted sensng range, Amercan Control Conference, June [11] S. M. Kay, Fundamentals of statstcal sgnal processng: estmaton theory. Upper Saddle Rver, NJ, USA: Prentce-Hall, Inc., [12] R. E. Kalman, A new approach to lnear flterng and predcton problems, Transactons of the ASME Journal of Basc Engneerng, vol. 82, no. Seres D, pp , 1960.