The Impact of Multi-group Multi-layer Network. Structure on the Performance of Distributed. Consensus Building Strategies

Transcription

1 AIAA Guidance, Navigation, and Control Conference August 011, Portland, Oregon AIAA The Impact of Multi-group Multi-layer Network Structure on the Performance of Distributed Consensus Building Strategies Yan Wan and Kamesh Namuduri and Swathik Akula and Murali Varanasi University of North Texas, Denton, TX, 7601 Multi-group multi-layer (MGML) structure is common to distributed sensor networks (DSNs) in many natural and engineered applications. Although network structure plays a crucial role in the performance of consensus building strategies in such networks, little work is done to establish the precise connection between the two. In this paper, we consider a structural approach to the consensus building problem in MGML DSNs that have wide applications in e.g., multi-vehicle systems. From among the possible network structures, we focus on bipartite graph structures and briefly discuss hierarchical structures due to their wide applicability in similar applications. We show that the consensus building dynamics in MGML structure can be easily mapped to canonical discrete-time first-order dynamics. We establish the exact conditions for consensus and derive a precise relationship between the consensus value and the degree distribution of the nodes in the bipartite graph. We also demonstrate that for subclasses of connectivity patterns, convergence time and simple characteristics of network topology can be captured by explicit algebra. Direct inference of the convergence behavior of consensus strategies from MGML DSN structure is the main contribution of this paper. The insights gained from our analysis facilitate the design and development of large-scale DSNs that meet specific performance criteria. Assistant Professor, Department of Electrical Engineering, and AIAA Member. Associate Professor, Department of Electrical Engineering. Masters Student, Department of Electrical Engineering. Professor, Department of Electrical Engineering. 1 Copyright 011 by the American Institute of Aeronautics and Astronautics, Inc. All rights reserved.

2 I. Introduction Consensus problem widely appears in natural and engineering applications, and has received extensive study (see e.g., [1,, 8, 16 18, 7, 8]). However, it has not been investigated much in the context of multi-group multi-layer (MGML) structures; i.e., networks composed of multiple groups, in each of which some nodes are designated as leaders and given the responsibility of communicating with other groups through certain layered communication schemes. Such MGML structure is commonly observed in nature and has great potential for engineered systems. For instance, MGML structure is typically observed in bird flocking as studied in e.g., [14, 15]. Similarly, in distributed sensor networks that are widely adopted for multi-vehicle network applications, properly designed MGML communication structure has the potential to be more energy efficient due to the function separation of sensors and the organized routing structures [3, 9]. Network structure has known to be crucial to the performance of consensus building [9 1, 18]. In designing MGML network structures for consensus building, one question is whether there exists a quantitative relationship between the network structure and its performance, especially in large-scale networks, where scalability is important. We address this issue by studying the network design problem for some common MGML structures. In order to represent the information flow in a MGML DSN, it is beneficial to describe a DSN using an equivalent graphical model that can abstract and capture its connectivity patterns. Examples of some graphical models that we are interested are illustrated in Fig. 1. Bipartite graph structure (Figure 1a) is widely used in communications and has been recently adopted for sensor network applications [6,1,]. As we will illustrate further in the paper, bipartite structure can be used to capture a wide arrange of -layer lead-follower communication topology. Hierarchical structure (Figure 1b), as a generalization of the bipartite structure, can capture multi-layer communication schemes. In this paper, we focus on the bipartite graph structure, and also briefly discuss the hierarchical structure at the end of the paper, the analysis of which is a simple generalization of the bipartite structure developed in great detail. Our objective is to investigate the effect of network structure on the convergence behavior of consensus building strategies. Specifically, we take a structural approach to study the research questions of interest in consensus building, including whether consensus can be reached by the network, what is the final consensus value, and how many iterations are needed to reach the consensus. Direct inference of the convergence behavior of the consensus strategies from simple features of DSN structures is the main contribution of this paper. These results in turn, lead to efficient and scalable strategies for designing DSNs (e.g., the selection of the number of leaders and network connections) to meet desirable performance requirements. It is worthwhile to note that our approach also suggests a way to investigate algebraic graph

3 theory-related aspects by working with indirect graphs rather than the graph directly associated with network dynamics. We show that this approach can expose some hidden features of graph structures and provide insightful analytical results on network dynamics that are not possible otherwise. C C3 C4 C v1 v v v4 v6 v v3 v4 v5 C4 v3 v5 C1 C3 v5 C1 C3 v6 v7 v7 v8 C C4 v10 C C4 v7 v4 v5 v6 v7 v8 v9 v10 v11 v1 v10 v11 v1 v9 v11 v1 v10 v8 v9 (a) Bipartite Graph Model (b) Hierarchical Model (c) Mesh Model (d) Semi-regular Graph Model Figure 1: Four sample networks, each consisting of 16 nodes (twelve member nodes V1 to V1, and four leader nodes, C1 to C4) forming four groups, are shown here. In each group, one node serves as the leader. Information flows in both directions i.e., from the agents to the leaders and from the leaders to the agents. In the remainder of this section, we formulate the research problem. In section II, we provide a brief review of relevant literature. In section III.A, we present the condition for reaching consensus and the consensus value that all sensor nodes finally agree to. We show that this information can be explicitly inferred from the DSN structure. In section III.B, we explore the estimation of convergence time through eigen-analysis, and the relationship between network structure and the performance of the consensus strategy for subclasses of connectivity patterns. A. Problem Formulation Consensus building refers to the process in which each sensor node starting with its initial estimate about a parameter of interest, arrives at a common value, through a localized iterative updating process. In each iteration, each node updates its estimate based on the information received from its neighbors and then shares its updated estimate with its neighbors. In this section, let us first describe the MGML structure of interest, and then the detailed consensus algorithm adopted in this study. We begin with a functional description of the network. Each sensor node in the network has the same functionality in the sense of sensing and information processing. However, in each group, some sensor nodes are designated as leaders or fusion centers. To facilitate our analysis, we remove the sensing functionality from fusion centers, and introduce the concept of virtual fusion centers (VFCs), whose responsibilities 3

4 are solely on establishing connections within and between the groups (as shown in Fig. ). The VFCs are represented by square nodes, sensors are represented by circles, and communication links are represented by edges. The functionality of a VFC can be implemented at any node as shown in the figure on the right. This representation is not limited to a small number of nodes. It can be applied to more complex and general network structures. Our focus in this paper is on networks whose structure can be mapped to a bipartite graph as shown in Fig. 1(a) (see [1] for a detailed description). Bipartite graphs, are also known as Tanner graphs [], in coding theory. We note that the bipartite structure can capture a wide arrange of -layer leader-follower communication relationships. Even if the DSN structure is not bipartite (see the example structures discussed in Section III.B), the separation of sensing functionality from fusion center through the introduction of VFCs can map the DSN structure to a bipartite one. Hence, our study developed in this paper is not limited to bipartite DSNs. The network structure of a Tanner graph is represented by its parity matrix. In this paper, we refer this parity matrix as routing matrix in the context of a DSN. Assuming that a DSN has m VFCs and n sensor nodes, its structure can be captured by a m n routing matrix H, in which the entry in (i, j) is set to 1 if there is communication link between the i th VFC and the j th sensor node. For instance, the routing matrix associated with Figure is H = If the degrees (i.e., the number of communication links) of all VFC nodes are the same, we say that the DSN is regular. FC1 S3 S4 FC VFC1 VFC S1 S3 S4 S Figure : a) An example of a regular DSN, b) its bipartite graph representation, c) the graph capturing the direct network structure associated with the consensus building dynamics Now let us describe the consensus building process. The motivation for the consensus building strategies that we study comes from the belief propagation (BP) concepts developed by Gallager in 1963 [6] with applications to channel coding and Pearl s algorithm developed in 1980s in artificial intelligence community. It begins with the sensors observing a common phenomenon by taking their own measurements. In each iteration, each VFC receives inputs from the senors connected to it, aggregates the information, and then S1 S3 S S4 4

5 updates all the sensor nodes with the new value it computed. The iterative algorithm converges to a final value and the consensus is reached by all agents at the same time. Each iteration in the consensus building process consists of two operations. 1) Forward operation: In the first half of the iteration, each sensor sends its own value to the VFC to which it is connected. ) Backward operation: In the second half of the iteration, each VFC updates the sensor nodes with its new result. A VFC may employ a linear or a non-linear process to aggregate its input based on factors such as the type and proximity of the sensor to the target. In this paper, let us assume that the collected values from all connected sensor nodes are aggregated by a simple averaging operation based on the number of sensors it is receiving information from. In another words, each VFC calculates the average of all observations it received and sends this new estimate to each sensor node that it is connected to. Each sensor node then updates its value by taking the average of new estimates it receives from the VFCs that it is connected to. We refer to this typical consensus building algorithm as the iterative averaging algorithm. We will show that the consensus building dynamics using this algorithm can be easily reformulated as a variant of the discrete-time first-order dynamics appeared widely in the consensus literature [1, 16 18]. However, our focus in this paper is on direct inference of consensus property from network structure H and in turn the effective design of structure for fast convergence, rather than on general results without structural insights. II. Background Consensus problems have been extensively studied in a multitude of fields, including sensor networking [1,, 7], distributed computing [13, 0], and decentralized control [17, 18], among others. Due to space limitations, we provide only a brief review of the most relevant literature with the aim to motive our approach. In a closely related work, Boyd et. al. studied the convergence condition and convergence time for a network of autonomous sensors [1, 7]. The authors formulate the design of network structure for fast convergence as an optimization problem and use the semi-definite programming approach to solve it numerically. Our work is different from theirs in mainly three folds. First, we study MGML DSNs that have inherent structures rather than a network of egalitarian sensor nodes without any group structure. Second, works [1, 7] consider the final consensus value as an average of all sensor nodes initial values. In reality, sensor nodes may contribute 5

6 unequally to the final consensus value, hence we allow the contribution of each sensor to the consensus value as a free design variable. Finally, we derive explicit and closed-form relationship between the network structure and its convergence behavior, rather than providing a numerical approach. Consensus problems have received huge attention from the decentralized control community, whose focus is mainly on building local controllers to achieve consensus among agents. The works in this field are to some extent originated from [6]. A tutorial paper [17] provides an overview of the literature in this domain for networks with first-order dynamics and Laplacian topology. Paper [16] presents general results on consensus condition, consensus value, and consensus performance, which establish the foundation of our study. Our contribution in this paper is the direct inference of consensus property from structure, which facilitates design of MGML structure to meet certain performance criterion. Along this line, Paper [18] which presents some interesting results about the dependence of the consensus value on initial conditions and network topology. A. Network Topology and Algebraic Graph Theory The network topology or its graphical equivalent has a significant impact on the performance of any consensus strategy, particularly on its convergence rate (see e.g. [9 1]). For example, in [9], Kar suggests that consensus algorithms converge faster on the Ramanujan s d-regular graphs than on Erdös-Renýi random graphs, and graphs exhibiting small-world properties. The number of iterations it takes for a consensus strategy to converge impacts communication overhead and the network lifetime. Although in areas such as the science of networks, network structure and its impact on performance has been previously investigated, there has been limited research work in evaluating the impact in a quantitative manner, and in network design for consensus strategies that exploit network structure. Some of the relevant works are summarized here. A common optimal scaling problem for networks was investigated in [19]. A design that gives structural insights for the limited resource allocation problem was given in [4]. For networks with tree structures, a structural approach to weight assignment to edges in such a way to maximize the algebraic connectivity was explored in [5, 5]. Algebraic graph theory provides very useful insights into network designs, due to the ability of eigen-analysis in connecting network structures and the dynamics. Examples of the results in this field can be found in [3,4]. From the above literature review, it can be observed that despite the existence of abundant literature on consensus building and the realization that network connectivity structure plays a crucial role in the 6

7 performance of consensus building strategies, there is very limited effort on directly relating network structure to the performance of consensus building strategies and the design of network connectivity patterns for consensus building. Our goal is to bridge this gap. Specifically, we investigate the tanner and then hierarchical graph representation of multiple-group DSN, and show that the consensus condition, the final consensus value, and performance of consensus building can be directly inferred from the network structure for broad classes of network topologies. These results allow us to explicitly design DSNs with connectivity patterns that meet the desirable performance specifications without using complicated numerical computations. III. Inferring Consensus from Network Structure In this section, we analyze the impact of network structure on consensus condition and convergence time for the MGML DSN model and the iterative averaging algorithm formulated in section III.A. We first present the general conditions for convergence in consensus building strategies for the completeness of presentation, and the consensus estimate from structure in section III.A. In section III.B, we discuss an important performance measure the time needed to reach consensus. We provide results that directly relate convergence time with DSN structure; these results provide rich insights into the design of DSNs. A. Condition for Convergence and Estimate of the Consensus Value in Consensus Building Strategies In order to analyze the convergence behavior of consensus strategies, let us first obtain the dynamics of sensor values, and then analyze the asymptotic behavior of this dynamics. In order to do that, we introduce a vector x[k] R n 1 to hold the values of all sensor nodes at cycle k. Recall in section I.A that during each cycle, two operations are involved: the forward and backward operations. According to the iterative averaging algorithm, after the forward operation, the values held at the VFCs are found to be y[k + 1] = K Hx[k], where the m m diagonal matrix K = [diag(h1 n 1 )] 1. Here, diag() represents the operation that places the vector in the parenthesis onto the main diagonal. For instance, diag(1 n 1 ) is a n n identity matrix. Similarly, the values at the sensors after the backward operation can be computed using x[k + 1] = K 1 H T y[k + 1], where the n n diagonal matrix K 1 = [diag(h T 1 m 1 )] 1. The effect of the two operations in one cycle can be captured by the following linear time-invariant (LTI) difference equation: x[k + 1] = Ax[k] = K 1 H T K Hx[k], (1) 7

8 where A is used to describe the system matrix. We note that even though the routing structure associated with H is bipartite, the direct network structure can be very complicated (see Figure c for an example). The eigenvalues of A provide rich information into the system dynamics, and in turn on the convergence behavior of the consensus strategy. We note that Equation 1 maps the consensus problem in bipartite networks to the typical discrete-time first-order consensus problem studied in the literature, and hence general results presented in e.g. [1, 16] can be used for our analysis. Different from many of these studies, the system matrix in Equation 1 is not necessarily symmetric. Here in Theorem 1, let us first describe the properties of eigenvalues associated with A, through the introduction of new symmetric matrix Â. Theorem 1 The eigenvalues of the system matrix A described in Equation (1) are real, simple, and reside between 0 and 1. At least one eigenvalue is equal to 1. Moreover, the multiplicity of the eigenvalue with a value 0 is greater than or equal to n m. Proof: Let us first examine the properties of the system matrix A. Because the row sum of [diag(h T 1 m 1 )] 1 H T is 1 and the row sum of [diag(h1 n 1 )] 1 H is also 1, we see that A is a stochastic matrix in the sense that the row sum of A is 1 and each entry of A is non-negative. Hence A has at least one eigenvalue at 1 with associated right eigenvector 1 n 1. Now let us show that the eigenvalues of matrix A are real, simple, and non-negative. To show this, let us introduce a symmetric matrix  = K 1 1 H T K HK 1 1, and show that the eigenvalues of A are the same as the eigenvalues of the symmetric matrix Â. Supposing that w i is a left eigenvector of A corresponding to the eigenvalue λ i, we have w T i K 1H T K H = w T i λ i. This expression can be rewritten as w T i K 1 1 K 1 1 H T K HK 1 1 = w T i K 1 1 λ i by multiplying K 1 1 on both sides. This equation informs us that for any eigenvalue λ i and left eigenvector wi T associated with A,  = K 1 1 H T K HK 1 1 has the same eigenvalue λ i with the left eigenvector w T i K 1 1. Since eigenvalues associated with a symmetric matrix  are all real and simple, the eigenvalues of A are also real and simple. Moreover, since  is positive semi-definite, the eigenvalues of A are non-negative. What is left to show is that the multiplicity of the eigenvalue at 0 is greater than or equal to n m. This is straightforward since the multiplicity of the eigenvalue 0 equals n rank(a) and the rank of A is less than or equal to the rank of H which is at most m. Theorem 1 shows the range of eigenvalues of the system matrix A. Since the eigenvalues are real and reside between 0 and 1, we can denote them as 1 = λ 1 λ... λ n 0. Moreover, let us denote the normalized left and right eigenvectors associated with λ i as wi T and v i, respectively. As we will show later, the eigenstructure of the system matrix A plays a crucial role in consensus building. In theorem, we 8

9 will give the necessary and sufficient condition for consensus, and present a quantitative description of the dependence of consensus value on network structure captured by the routing matrix H. We note that the results on consensus condition and consensus value in terms of system matrix A are well known [1, 16, 17]. Theorem is a slight different version of those results in terms of network structure H, instead of A. Theorem Consider a MGML DSN represented by a Tanner graph with routing matrix H. The consensus of the DSN is guaranteed if and only if there is at least a path between any pair of sensor nodes in the graph. The final consensus value that the DSN converges to is mh1 n m Hx(0), where x(0) R n 1 contains initial values of all sensor nodes in the DSN. Proof: Let us decompose the system matrix A and rewrite equation 1 as x[k] = n v i λ k i wi T x[0], () i=1 Equation informs us that only the terms having eigenvalue 1 make contribution to the final values of sensor nodes. Now, let us prove the if direction. Since all the sensor nodes are connected, we know that the system matrix A is irreducible. Moreover, the way A is generated informs us that the underlining Markov chain for A is ergodic a. Hence, from the Perron-Frobenius theorem [7], we know the following facts: 1) the dominant eigenvalue of A, λ 1, is 1 and strictly larger than the magnitudes of all other eigenvalues, and ) the right eigenvector associated with it, v 1, is 1 n 1. The two facts guarantee that the sensor nodes will reach the same value asymptotically. To prove the only if, let us note the fact that if sensor nodes are not connected, matrix A will have at least two eigenvalues at 1. The eigenvectors associated with all eigenvalues of 1 will contribute to the final consensus. Hence, the sensor nodes can not reach consensus. In the case that the consensus condition is satisfied, the final value of x[k] can be calculated as w 1 T x[0]. This means that the consensus value is a summation of each initial value scaled by the corresponding entry in the left eigenvector associated with the dominant eigenvalue 1. w 1 T = mh1 n m H, since it satisfies w T 1 A = w T 1. The proof is complete. It can be easily checked that Theorem gives the condition for convergence of the consensus strategy and the dependence of consensus value on initial values of all sensor nodes. It is interesting to notice that the column sum of H matrix (as a Ergodic implies both recurrent and aperiodic. 9

10 indicated by 1 1 m H) gives the weighting of each sensor node s contribution to the final consensus value. The column sum of H matrix is in fact the degree sequence of sensor nodes. Hence, the degrees of sensor nodes are the sole factors in determining the final consensus value. Interestingly, the degrees of VFCs do not impact the final consensus value. Theorem specify the dependence of final consensus value on the column sum of H. This simple relationship allows us to quickly design the routing matrix H to achieve certain consensus value. For instance, if certain sensors are known to measure data with high fidelity, or probe information with higher priority, it is reasonable to design the DSN in a way that such sensors have more contribution in the final consensus value. This inhomogeneous weighting can be achieved by assigning the important sensor nodes with higher degrees. Theorem naturally leads to the following corollary. Corollary 1 Consider a MGML DSN represented by a Tanner graph with routing matrix H. Given that the DSN can reach consensus, the final consensus value is the average of all initial values if the degrees of all sensor nodes are the same. Let us use a simple example to illustrate theorem. Example 1: In this example, we consider the DSN described by the Tanner graph shown in Figure 3. The routing matrix for this example is H= From theorem, we know that consensus can be reached as there exists at least a path between any two sensor nodes. Moreover, the consensus value is x(0) = 1 5 (x 1(0)+x (0)+x 3 (0)+x 4 (0)), where x 1 (0), x (0), x 3 (0), and x 4 (0) represent the initial values of sensor nodes. VFC1 VFC S1 S S3 S4 Figure 3: The Tanner graph representation of the DSN for Example 1 10

11 [ ] Alternatively, let us compute the consensus value numerically. Since K 1 = diag( 1 1/ 1 1 ) and K = 1/3 1/3 1/3 0 [ ] 1/6 5/1 1/6 1/4 diag( 1/3 1/ ), the system matrix A = K 1 H T K H =. The final consensus 1/3 1/3 1/ / 0 1/ value can be found by iteratively updating the dynamical equation x[k + 1] = Ax[k]. When the number 1/5 /5 1/5 1/5 1/5 /5 1/5 1/5 of iterations is large, lim k x(k) = lim k A k x(0) = x[0]. Hence, the consensus 1/5 /5 1/5 1/5 [ 1/5 ] /5 1/5 1/5 value is given by x( )= 1 5 (x 1(0) + x (0) + x 3 (0) + x 4 (0)), which matches the result directly obtained from theorem 1. B. Dependence of Convergence Time on Network Structure Convergence time is an important performance measure for consensus building strategies. In this section, we first review the relation between convergence time and the second largest eigenvalue of the system matrix A for the completeness of presentation [1, 16]. Then, we show that eigen-analysis provides an approach to directly infer convergence time from some very simple structural characteristics of a DSN, such as the number and degrees of sensor nodes and VFCs, etc. Theorem 3 Consider a DSN represented by a Tanner graph with routing matrix H. Supposing that the DSN can reach consensus, the convergence time (i.e., the number of iterations such that the difference between sensor value and final consensus value is within δ of its initial value) is log λ δ as δ 0, where λ 0 is the second largest eigenvalue associated with the system matrix A. Proof: From theorem, we know that the values of sensor nodes converge to x f = v 1 w 1 x[0]. There is only one eigenvalue with value 1, and all the other eigenvalues are real and have magnitude less than 1. The case that λ = 0 is trivial in that the consensus can be reached in one iteration. Let us focus on the case the λ 0. 11

12 x[k] = n v i λ k i wi T x[0] = x f + i=1 n v i λ k i wi T x[0]. (3) Clearly, the spectral radius of n i= v iλ i w T i is λ, which is equal to lim k ( n i= v iλ k i wt i ) 1 k. In the limit of δ 0, k = log λ δ for x[k] x f x 0 = δ. i= From the above analysis, we see that the convergence time is a function of the second largest eigenvalue λ of the system matrix A, which is obtained from the routing matrix H. While this quantitative result is important and well known, it does not directly tell us the convergence time from the DSN structure. In the rest of this section, we relate the structure of the routing matrix H to λ, and in turn to the convergence time. In theorem 4 and corollary, we will consider a general DSN structure first and then give the results for a regular DSN. Theorem 4 The second largest eigenvalue of the system matrix A associated with a general DSN model represented by a bipartite graph can be found as the maximum of i=1 (K 1 Hy) i, where y Rn 1 is subject to the following constraints: 1) n y i = 0 ) n y i = 1. Proof: First, recall in the proof of theorem 1 that the eigenvalues of A are the same as the eigenvalues of the symmetric matrix  = K 1 1 H T K HK 1 1. Moreover, the left eigenvectors of  equals wt K 1/ 1, where w T is a left eigenvector of A. Matrix  is symmetric. The Courant-Fischer theorem informs us that the second largest eigenvalue λ can be found as [4] λ = max x w T 1 K 1 1, x =1xT Âx (4) = max x w T 1 K 1 Let us introduce a new vector y = K 1 1 x. Since w T 1 K 1 1, x =1xT 1 HT K HK 1 1 x. aligns with 1 1 n K1 1, y is subject to the constraint that y K n 1. Moreover, since x = 1, we obtain K 1 1 y = 1. This result is straightforward. Theorem 4 is very useful in that it allows the calculation of the second largest eigenvalue directly from the routing matrix H, rather than through the eigen-analysis of the system matrix A. The theorem provides an approach to obtain λ from simple characteristics of the routing matrix H. In the following corollary, we give a similar result for regular DSNs. 1

13 Corollary Considering a regular DSN, the second largest eigenvalue of the system matrix A can be found as the maximum of 1 p i=1 (Hy) i, where p is the regular degree of VFCs, and where y Rn 1 is subject to the following constraints: 1) n y i = 0 ) n y i = 1. Proof: This result can be derived directly from theorem, with the observation that K equals 1 p I. Theorem 4 and corollary provide an approach to obtain the second largest eigenvalue from structure, and hence the convergence time for various subclasses of DSNs. Complementing the results in the consensus building literature [1,], we show that convergence time can be directly obtained from some simple structural characteristics for subclasses of DSNs. Theorem 5 considers a class of regular DSNs containing multiple groups, each of which has one leader that communicates to all other group leaders. Theorem 5 Consider a regular DSN that contains m groups. Each group has 1 VFC and k sensor nodes that communicate with it. In each group, there is one and only one node that all VFCs communicate with. The consensus time for this regular DSN is log k 1 δ. k+m 1 Proof: The DSN considered in the theorem has m VFCs with regular degree p = k + m 1 and n = mk sensor nodes. Without loss of generality, let us denote the sensor node with index 1 + k(i 1) as the one in group i that all VFCs communicate with. Moreover, for the convenience of presentation, let us introduce set S containing the indices of such sensor nodes for all groups, and set G i containing the indices of all sensor nodes in group i. In order to obtain λ, let us use a slightly revised version of corollary. Specifically, λ is the maximum of 1 i=1 (Hy) i p n i=1 K 1 1 y i i, where y is subject to the constraint: n y i = 0. Now let us try to obtain the relationship among y i using Lagrange multipliers. Specifically, let us show that 1) all entries of y corresponding to the sensor nodes inside a group except the one that all VFCs communicate to are the same, i.e., y j are equal for all j G i S for each i from 1 to m. Let us define the Lagrange operator L as L = = m n (Hy) i + α( K1 1 i y i ) + β( i=1 m ( i=1 j G i +S i=1 y j ) + α( n i=1 n i=1 K 1 1 i y i ) + β( K 1 1 i y i 1) (5) n i=1 K 1 1 i y i 1) 13

14 The optimal solution for (y i, α, β) is the one that satisfies L y l leads to m i=1,&l G i ( j G i +S = 0 for all l, L α = 0, and L β = 0. L y l = 0 y j ) + K 1 1 l α + βk 1 1 l y l = 0 (6) Equation 6 informs us that y j are equal for all j G i S for each i from 1 to m. Finally, let us obtain the maximum of i=1 (Hy) i n i=1 K 1 1 y i i. Using the constraint that n y i = 0, we obtain i=1 (Hy) i n y i = i=1 ( j G i +S y j) i/ S y i + m = i S y i i=1 ( j / S K 1 j y j + j G i S y j) i/ S y i + m i S y i since j S y j = j / S K 1 j y j. The upper bound of the right side of Equation 7 can be found as i=1 ( j / S K 1 j y j + j G i S y j) i/ S y i + m i=1 ( j / S K 1 j y j + j G i S y j), (8) i S y i i / S y i since y i for i S does not appear in the denumerator and is greater than 0. The equality holds when y i = 0 for all i S. Applying the constraints that n y i = 0, and y j are equal for all j G i S for each i from 1 to m, we can simplify the right side of Equation 8 as i=1 ( j / S K 1 j y j + j G i S y j) Hence, the maximum of 1 i=1 (Hy) i p n i=1 K 1 1 y i i i/ S y i i=1 (Hy) i n i=1 K 1 1 y i i, we obtain λ = k 1 k+m 1. = i=1 ( j G i S y j) i / S y i = (k 1) i / S y i i / S y i (7) = k 1 (9) = k 1. Since the second largest eigenvalue λ is the maximum of Theorem 5 says that for a common class of DSN that contains multiple groups, in each of which exists a leader that communicates to all the other leaders, the consensus time using the iterative averaging algorithm is only dependent on the number of groups and the number of nodes in each group. The results on the explicit relationship of DSN structure and the consensus time is very valuable due to the scalability and facilitation of design. Here, we illustrate the theorem in the following example concerning a DSN with only three groups. However, it is important to note that the value of the theorem resides in large-scale networks, for which the consensus time can be inferred directly from simple characteristics of DSN structures, without any complicated numerical computation. Example : A DSN consisting of three groups is shown in Figure 4a. Each group has one fusion center (leader) and three sensor nodes communicating with the fusion center. Each group leader communicates to 14

15 the other two group leaders. The Tanner graph representation is shown in Figure 4b with k = 4 and m = 3. According to theorem 5, the second largest eigenvalue is 0.5, and the consensus time is log 0.5 δ. Figure 4: a) The DSN with fusion centers considered in Example ; b) The tanner graph representation In the next theorem, we consider another class of DSN which is regular and containing two VFCs. The consensus time can be directly inferred from the regular degree and the number of sensor nodes in the DSN. The theorem is exemplified in Example 3. Theorem 6 Consider a DSN containing two VFCs with regular degree p and n sensor nodes. Supposing that the DSN can reach consensus, the consensus time is log n p δ. p Proof: For the convenience of presentation, we introduce set S containing the sensor nodes that are communicating with both VFCs, and set G i containing the sensor nodes only communicating with VFC i. Similar to the proof of theorem 5, we can show that y j are the same for j G i, for each i. Now let us calculate λ as the the maximum of 1 p i=1 (Hy) i n i=1 K 1 1 y i i and the condition that y j are the same for j G i, for each i, we obtain 1 i=1 (Hy) i p n yi. Using the constraint that n y i = 0, = 1 ( i S+G 1 y i ) + ( i S+G y i ) p i G 1 yi + i G yi + i S y i = 1 ( i G 1 S y i i/ S 0.5y i) + ( i G S y i i / S 0.5y i) p i G 1 yi + i G yi + i S y i 1 ( i G y i ) + ( i G 1 y i ) p i G 1 yi + i G yi = 1 (n p)( i G 1 yi + i G yi ) p i G 1 yi + i G yi = n p p (10) The equality holds when y i = 0 for all i S. The proof is complete. 15

16 Example 3: Consider a regular DSN that has two VFCs with regular degree 6 and 8 sensor nodes (shown in Figure 5). According to theorem 6, the second largest eigenvalue is 5, and the consensus time is therefore log 0.4 δ. Figure 5: The tanner graph representation for the DSN in Example 3. In the next theorem, we will consider an irregular DSN that has one and only one sensor node that communicates with all VFCs. Theorem 7 Consider a DSN that contains m 3 groups (see Figure 6). k sensor nodes that communicate with it. communicates with all VFCs. The consensus time for this DSN is log k δ. k+1 Proof: Each group has 1 VFC and There exists one and only one sensor node in the DSN that The DSN considered in the theorem has m VFCs and n = mk sensor nodes. Let us use G i to indicate the set, containing the indices of sensor nodes in group i. Without loss of generality, let us assume that the index of the sensor node communicating with all VFCs is 1, and the group containing this particular sensor node is group 1. Moreover, let us assume that the sensor nodes in group i are indexed from k(i 1)+1 to ki. 1 m i=1 Let us use the theorem 4 to obtain λ, which is the maximum of (K Hy) i n, where y is subject to the i=1 K 1 1 y i i constraint: n y i = 0. Using Lagrange multipliers, we can show that all entries of y corresponding to the sensor node inside a group G i {1} for each i from 1 to m are the same. This result is discussed in the proof of theorem 5. Using these constraints, we obtain 16

17 i=1 (K 1 Hy) i n y i = = i=1 (K 1 i j G i +{1} y j) i / {1} y i + m i {1} y i i=1 ( K 1 i j / {1} y j m + K 1 i j G i {1} y j) i G 1 {1} y i + i/ G 1 yi + m i {1} y i i=1 (K 1 i j G i {1} y j) i G 1 {1} y i + i/ G 1 y i = (K 1 1 j G yj) + m 1 {1} i= (K 1 i j G i y j ) i G 1 {1} y i + i / G 1 yi = ( 1 k )( i G 1 {1} y i) + ( 1 i= ( j G i y j ) ) i G 1 {1} y i + i / G 1 yi k+1 )( = ( (k 1) k )y + ( k k+1 ) i= y k(i 1)+1 (k 1)y + i= ky k(i 1)+1 Denoting y be X and i= y k(i 1)+1 be Y, the above equation becomes (k 1) k X + k (k+1) Y (k 1)X + ky = (k 1)[ 1 k ((k 1)X + ky ) + k (k+1)(k 1) Y Y ] (k 1)X + ky = k 1 k + k 1 + k k = k + 1 k k+1y (k 1)Y (k 1)X + ky k k+1y (k 1)Y ky The equality holds when y i G1 = 0. The proof is complete. Theorems 5, 6, and 7 demonstrate three classes of DSNs, whose consensus time can be directly obtained from connectivity structures using theorem 4. Using this approach, we can obtain similar results for other classes of DSNs, including irregular DSNs. We emphasize that by linking eigenvalues with the properties of the routing matrix H rather than the system matrix A directly, rich structural information is exposed to aid in the network design and analysis. The inference of eigenvalue from the structure of graphs (especially asymmetric graphs) is not a trivial task. (11) This work provides an interesting approach to expose hidden graph structure that is not obvious from the graph directly associated with network dynamics; such hidden structure may help to address algebraic graph problems with rich insight. These results also provide a direct approach to DSN design. First, the degrees for sensor nodes can be selected to reach desired consensus value. Consensus time can also be achieved through the DSN design. 17

18 S 3 S 31 S 4 S 41 S 4k 1 S m 1, S m 1,1 S m 1, k 1 S 3k 1 FC3 FC4.. FCm 1 S m, 1 Sm, S 1 S FC FCm S m,k 1 S k 1 FC1 S 11 S 1 S 1k 1 Figure 6: The DSN structure considered in theorem 7. For instance, for the class of DSNs considered in theorem 5, we can choose the number of VFCs for any convergence time predefined, by using the explicit relation between λ and the number of VFCs. Consider a network containing 1000 sensor nodes and the desired convergence time of 10 (associated with δ = 0.01). Suppose we adopt the structure according to theorem 5, we can easily calculate that m = 5 by solving for log as log m 1 λ δ k. The results are particularly useful when large-scale networks are 1000 m +m 1 considered for which existing solutions for computing convergence times such as the numerical semi-definite programming (SDP) can be very time-consuming [7]. Similarly, for the class of DSNs considered in theorem 6, we can also choose the regular degree of VFCs for any desired convergence time. The results presented so far are focused on bipartite graphical models. However, they can easily be extended to other graphical models and iterative consensus algorithm as well. For instance, let us consider a hierarchical tree structure (see Figure 1b). The system dynamics for this 3-layer hierarchical structure can easily be captured by x[k + 1] = K 1 H T 1 (K 3 H T K 4 H )K H 1 x[k], where H 1 and H are the routing matrices associated with the first and second layers respectively, and K 1 = [diag(h T 1 1)] 1, K = [diag(h 1 1)] 1, K 3 = [diag(h T 1)] 1, and K 4 = [diag(h 1)] 1. shown in Figure 1b can be represented by x[k + 1] = T [ ] T [ ( For instance, the dynamics of the hierarchical network ] ) x[k]. 18

19 Explicit results characterizing the convergence behavior, similar to those derived for the bipartite graphical models, can thus also be derived for hierarchical graphical models. We intend to investigate fully hierarchical graph models and other MGML structures in near future. Acknowledgment The first author would like to thank Dr. Sandip Roy for illuminating discussions. The work is supported by NSF under grant numbers and References 1 S. Boyd, P. Diaconis, and L. Xiao, Fast mixing markov chain on a graph, SIAM Review, vol. 46, no. 4, pp , 004. S. Boyd, A. Ghosh, B. Prabhakar, and D. Shah, Randomized gossip algorithms, IEEE Transactions on Information Theory, vol. 5, no. 6, pp , June F. Chung and L. Lu, Complex Graphs and Networks. Providence, RI: American Mathematical Society, F. R. K. Chung, Spectral graph theory. Providence, RI: American Mathematical Society, M. Fiedler, Absolute algebraic connectivity of trees, Linear and Multilinear Algebra, vol. 43, no. 1, pp , May R. G. Gallager, Low-density parity-check codes, MIT Press, , Discrete Stochastic Processes. Norwell, Massachusetts: Kluwer Academic Publishers, M. O. Jackson and B. Golub, Naive learning in social networks: convergence, influence and wisdom of crowds, FEEM working paper No , S. Kar and J. Moura, Consensus based detection in sensor networks: Topology optimization under practical constraints, First International Workshop on Information Theory in Sensor Networks (WITS), June V. Kolmogorov and M. Wainwright, On the optimality properties of tree-reweighted message-passing, Proc. Uncertainity Artificial Intell., July F. Kschischang, Codes defined on graphs, IEEE Sig. Proc. Mag.., vol. 41, no. 4, pp , Aug H.-A. Loeliger, J. Dauwels, J. Hu, S. Korl, L. Ping, and F. Kshischang, The factor graph approach to model-based signal processing, Proc. IEEE, vol. 95, no. 6, pp , June N. A. Lynch, Distributed Algorithms. San Mateo, CA: Springer, M. Nagy, A. Ákos, D. Biro, and T. Vicsek, Hierarchical group dynamics in pigeon flocks, Nature. 15 V. Quera, F. S. Beltran, and R. Dolado, Flocking behaviour: agent-based simulation and hierarchical leadership, Report, Wireless Sensor Network. 16 A. F. R. Olfati-Saber and R. M. Murry, Consensus and cooperation in networked multi-agent systems, Proceedings of the IEEE, vol. 95, no. 1, pp , January

20 17 W. Ren, R. W. Beard, and E. A. Atkins, Information consensus in multivehicle cooperative control, IEEE Control Systems Magazine, pp. 71 8, April S. Roy, A. Saberi, and K. Herlugson, A control-theoretic perspective on the design of agreement protocol, International Journal of Robust and Nonlinear Control, vol. 17, no , pp , December S. Roy, A. Saberi, and P. Petite, Scaling: a canonical design problem for networks, Proceedings of the 006 American Control Conference, pp , June S. Roy, Y. Wan, A. Saberi, and M. Xue, Designing linear distributed algorithms with memory for fast convergence, 48th IEEE Conference on Decision and Control, December M. Shukair and K. Namuduri, ldpc-like belief propagation algorithm for consensus building in wireless sensor network, 43rd Annual Conference on Information Sciences and Systems, 009. R. M. Tanner, A recursive approach to low complexity codes, IEEE Transactions on Information Theory, vol. 7, no. 5, pp , Sep S. Tarannum, S. Suraiya, D. S. Asha, and K. R. Venugopal, Dynamic hierarchical communication paradigm for wireless sensor networks: a centralized, energy effient approach, Journal of Artificial Societies and Social Simulation, vol. 13, no.. 4 Y. Wan, S. Roy, and A. Saberi, A new focus in the science of networks: towards methods for design, Proceedings of the Royal Society A, vol. 464, pp , March Y. Wan, S. Roy, X. Wang, A. Saberi, T. Yang, M. Xue, and B. Malek, On the structure of graph edge designs that optimize the algebraic connectivity, Proceedings of 47th IEEE Conference on Decision and Control, December C. W. Wu and L. O. Chua, Application of kronecker products to the analysis of systems with uniform linear coupling, IEEE Transaction on Circuit and Systems I, vol. 4, no. 10, pp , October L. Xiao and S. Boyd, Fast linear iterations for distributed averaging, Proc. 4nd IEEE Conf. on Decision and Control, pp , Dec T. Yang, S. Roy, Y. Wan, and A. Saberi, Constructing consensus controllers for networks with identical general multi-input multi-output linear agents, in Proceedings of AIAA 010 Guidance, Navigation, and Control Conference, Toronto, Ontario, August B. Zhou, L. H. Ngoh, B. S. Lee, and C. P. Fu, Hda: A hierarchical data aggregation scheme for sensor networks, Computer Communications, vol. 9, pp , November