THE notion of consensus or agreement in networks of multiple

Transcription

1 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 6, NO. 9, SEPTEMBER Consensus Algorithms and the Decomposition-Separation Theorem Sadegh Bolouki, Member, IEEE, and Roland P. Malhamé, Member, IEEE Abstract Convergence properties of time inhomogeneous Markov chain based discrete and continuous time linear consensus algorithms are analyzed. Provided that a so-called infinite jet-flow property is satisfied by the coupling chain, necessary conditions for both consensus and multiple consensus are established. A recent extension by Sonin of the classical Kolmogorov Doeblin decomposition-separation for homogeneous Markov chains to the inhomogeneous case is then employed to show that the obtained necessary conditions are also sufficient when a discrete time chain is in Class P, as defined by Touri and Nedić. It is also shown that Sonin s D-S Theorem leads to a rediscovery and generalization of the existing related consensus results in the literature. Finally, a geometric approach first developed by Shen is taken to extend the results of the discrete time case to the continuous time case. Index Terms Consensus, decomposition-separation theorem, distributed averaging algorithms, infinite jet-flow, inhomogeneous Markov chains. I. INTRODUCTION THE notion of consensus or agreement in networks of multiple agents is of great importance within various research communities as it arises in many real-world phenomena. In biology, consensus is linked with the emergent behavior of bird flocks, fish schools, etc. [] [3]. In robotics and control, consensus problems arise when seeking coordination and cooperation of mobile agents (e.g., robots and sensors) [4], [5]. In sociology, the emergence of a common language in primitive societies is a collective behavior within a complex system [6]. Consensus algorithms also have a rich history within computer science community [7], while formal study of consensus problems has been carried within the management science community (see [8] and references therein). Consensus as part of the convergence properties of distributed averaging algorithms have gained increasing attention in the past decade. It is believed that such linear algorithms were first introduced be DeGroot [8], where the author considered the time-invariant case, i.e., fixed coupling weights between any pair of agents. Later, more general cases were considered in [4], [5], [9] [6], where the authors mainly aimed at identifying sufficient conditions for consensus to occur, i.e., for states to Manuscript received September 24, 204; revised July 20, 205 and September 5, 205; accepted September 28, 205. Date of publication December, 205; date of current version August 26, 206. Recommended by Associate Editor A. Nedich. The authors are with the Department of Electrical Engineering, Polytechnique Montréal, Montreal, QC, H3T J4, Canada, ( sadegh.bolouki@ polymtl.ca; roland.malhame@polymtl.ca). Color versions of one or more of the figures in this paper are available online at Digital Object Identifier 0.09/TAC asymptotically converge to the same value. Beside consensus, multiple consensus has been the subject of many articles, e.g., [7] [2]. Multiple consensus refers to the case when each agent state converges, as time grows large, to an individual limit which may or may not be different from the individual limits of other agent states. A. State of the Art It is well known that the occurrence of (multiple) consensus in a distributed averaging algorithm is equivalent to (class-) ergodicity of the coupling chain, which is formed by a sequence of row-stochastic matrices, underlying the algorithm [9], [2], [22]. Erogodicity refers to the property that the backward product of row-stochastic matrices converges to a matrix with identical rows, while class-ergodicity refers to the existence of a limit (with no additional requirement on the limit) for such a backward product. Considering the work on linear consensus algorithms, [9] appears to provide the largest set of (class-) ergodic continuous time chains of row-stochastic matrices, while [20] and [2] lead the way to address the same problem for the discrete time case. Hendrickx and Tsitsiklis [9] proved class-ergodicity of continuous time cut-balanced chains and also obtained for cutbalanced chains, that ergodicity is equivalent to the infinite flow property (IFP), a notion first defined in [22]. On the other hand, Touri and Nedié [20] established for the discrete time cutbalanced chains, that strong aperiodicity (s.a.) guarantees classergodicity, while the s.a. property together with the infinite flow property are sufficient for ergodicity. The authors also considered random chains in Class P, a larger set of chains than the set of cut-balanced chains, and proved that within that class, the so-called weak aperiodicity (w.a.) property leads to class-ergodicity of a chain, while the w.a. property together with the infinite flow property are sufficient for ergodicity. In our earlier work [2], we fully characterized (class-)ergodicity of the set of balanced asymmetric via the so-called notion of absolute infinite flow property [22]. Fig. demonstrates the inclusion relations between the set of strongly aperiodic cutbalanced chains, the set of balanced asymmetric chains, and Class P for the discrete time case. B. Our Contributions As one of the objectives of this paper, the (class-)ergodicity results of the articles mentioned in the pervious section are generalized. More specifically, we characterize the set of (class-) ergodic chains within Class P for both discrete and continuous time cases IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See for more information.

2 2358 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 6, NO. 9, SEPTEMBER 206 Fig.. Inclusion relations between Class P, balanced asymmetric, and strongly aperiodic cut-balanced chains. The discrete time case is investigated in Sections II IV. Inspired by [22] and recalling the notion of jets in Markov chains [23], [24], the so-called infinite jet-flow property of chains is introduced in Section II-A, that will eventually result in novel necessary conditions for (class-)ergodicity of the chain in Section IV-A. Since infinite jet-flow property imposes a more restrictive condition on the chain than both the infinite flow and absolute infinite flow properties, the necessary conditions given in Section IV-A prove to be stronger than those provided in Theoremsand2of[22]. Based on the connections between the discrete time linear consensus algorithms and Sonin s D-S Theorem, explored in Section III, it is established in Section IV-B that the general necessary conditions derived in Section IV-A become also sufficient if the chain is in Class P. These results consequently subsume (class-)ergodicity results of [20] and [2]. As for the continuous time case, a geometric approach first introduced by Shen [25] is taken to investigate the asymptotic behavior of linear consensus algorithms in Section V. Particularly, it is shown in Section V-B that all chain in Class P are class-ergodic and, if and only if the infinite flow property is satisfied, ergodic. It generalizes the (class-)ergodicity results of [9] since cut-balanced chains form only a subset of Class P for the continuous time case. II. NOTIONS AND TERMINOLOGY In this paper, we deal with a general linear consensus algorithm in both discrete and continuous time. Let V ={,...,N be the set of agents. For the discrete time case, we consider an N-agent system with linear update equation x(t +)=A(t)x(t), t t 0. () In (), t indicates the discrete time index, t 0 0 denotes the initial time, x(t) =[x (t) x N (t)] is the vector of agent states, where prime ( ) indicates the transposition, A(t) is the matrix of coupling weights a ij (t), i, j N, and{a(t) is the coupling chain of the system. Coupling chain {A(t) is a sequence of (N N) row-stochastic matrices, i.e., all elements of each matrix A(t) are non-negative and each row of A(t) sums up to. Throughout the paper, for simplicity, we assume that t 0 =0. We also refer to a row-stochastic matrix as a stochastic matrix. Since each A(t) is stochastic, sequence {x(t), by definition, forms a backward Markov chain with transition chain {A(t). Notice that the evolution is described as a right hand multiplication by a column vector instead of the usual left hand multiplication by a row vector. We focus on the discrete time case up to Section IV, and leave the discussion on the continuous time case to Section V. If all components of states vector x(t) asymptotically converge to the same limit, irrespective of the initial conditions, global consensus, orsimply,consensus, is said to occur. Furthermore, if there exists a fixed partition of the N agents such that consensus occurs for the corresponding subvectors of x(t), thenmultiple consensus is said to occur. The subsets in the partition are then said to form consensus clusters. Itis well known the occurrence of consensus in dynamics () is equivalent to ergodicity of coupling chain {A(t) (see [9]), i.e., the property that for any fixed 0, backward product A(t)A(t )...A() (2) converges to a matrix with identical rows as t.furthermore, [2] and [22] establish that linear algorithm () induces multiple consensus if {A(t) is class-ergodic, i.e., if for any 0, backward product (2) converges as t grows large. For class-ergodic chains, set V can be partitioned into ergodic classes, whereby i, j Vbelong to the same ergodic class if the difference between the ith and jth rows of backward product (2) vanishes for any 0, ast. Under multiple consensus, the agent indices within the ergodic classes are the same as those within consensus clusters. We adopt the following notation throughout the paper. Letter t stands for either discrete or continuous time indices according to context. For an arbitrary vector v R N,and i N, v i denotes the ith element of v. The overline ( ) on a subset indicates complementation of the subset in the universal set of interest. In the following subsections, several notions that are crucial in the discrete time part of this work are introduced. A. Infinite Jet-Flow Property (IJFP) Inspired by [22] and [23] (as reported in [24]), in this section, we define a property of chains of stochastic matrices, herein called the infinite jet-flow property. This notion will help derive necessary conditions for (class-)ergodicity of any chain as well as equivalent conditions for (class-)ergodicity of a large class of chains. Definition : For a given V V= {,...,N,ajet J in V is defined by a sequence {J(t) of subsets of V.JetJ in V is called proper if J(t) V, t 0 (see Fig. 2). Moreover, for a jet J, jet-limit J denotes the limit of the sequence {J(t), as t grows large, if it exists in the sense that the sequence becomes constant after a finite time. When the elements of the

3 BOLOUKI AND MALHAMÉ: CONSENSUS ALGORITHMS AND THE DECOMPOSITION-SEPARATION THEOREM 2359 Then, if jet J is defined by J(t) ={ if t is even and J(t) = {, 2 if t is odd, then: U(J, V\J) =0, which means that the IJFP is not satisfied. However, for any jet J with a timeinvariant size, U t (J, V\J) /2 for any t 0, which results in U(J, V\J) =. Thus, the AIFP is satisfied. Keeping in mind that infinite flow is a weaker condition than infinite jet-flow, in the following, we report a property of chains from [20], namely weak aperiodicity (w.a.), which together with the IFP, becomes stronger than the IJFP (see Lemma below for details). Definition 6 [20]: A chain{a(t) of stochastic matrices is said to be weakly aperiodic if for some γ>0 and every distinct i, j Vand t 0, there exists l Vsuch that Fig. 2. Example of a proper jet J in V = {, 2, 3, 4, 5: J(0) = {, 2, 3, 5, J() = {, 5, J(2) = {2,... sequence are all identical to a subset S of V, the jet will be referred to as jet S. Definition 2: A tuple of jets (J,...,J c ) is a jet-partition of V,if(J (t),...,j c (t)) forms a partition of V for any t 0. Definition 3: Let a chain {A(t) of stochastic matrices be given. For any two jets J s and J k in V, U A (J s,j k ), or simply U(J s,j k ) when no ambiguity results, denotes the total interactions between the two jets over the infinite time interval U(J s,j k ) = Δ a ij (t)+ a ij (t). t=0 i J s (t+) j J k (t) i J k (t+) j J s (t) (3) Moreover, U A(t) (J s,j k ),orsimplyu t (J s,j k ), denotes the interactions between the two jets at time t U t (J s,j k ) = Δ a ij (t)+ a ij (t). i J s (t+) j J k (t) i J k (t+) j J s (t) Definition 4: The complement of a jet J in V, denoted by V\J or J is the jet defined by the set sequence {V \ J(t). Definition 5: Achain{A(t) of stochastic matrices has the infinite jet-flow property (IJFP) over V Vif, for every proper jet J in V, U(J, V \ J) is unbounded. If V = V,chain{A(t) is simply said to have the infinite jet-flow property. It is note worthy that the infinite jet-flow property imposes a stronger condition than both the absolute infinite flow property (AIFP) and the infinite flow property (IFP) as defined in [22]. Accordingto [22], a chain {A(t) of stochastic matrices is said to have the AIFP if U(J, V\J) is unbounded for every jet J in V with a time-invariant size. Moreover, it is said to have the IFP if U(J, V\J) is unbounded for every time-invariant jet J in V. Remember that there is no condition on the jets in the definition of the infinite jet-flow property, which makes it more restrictive. An example of a chain with the AIFP and without the IJFP is the following. Let: A(t) = or {{ if t is even 0 0 {{ if t is odd (4) a li (t)a lj (t) γa ij (t). (5) Definition 7 [20]: For a chain {A(t) of stochastic matrices, we define its infinite flow graph, G A (V,E), by an undirected graph of size N such that { E = (i, j) i j V, (a ij (t)+a ji (t)) =. t=0 The set of nodes of each connected component of G A (V,E) is called an island of {A(t). Notice that {A(t) has the infinite flow property if and only if G A (V,E) is connected, i.e., it has a single island. Lemma : Let a chain {A(t) of stochastic matrices be weakly aperiodic. Then, the infinite jet-flow property holds over each island of {A(t). In particular, in the presence of a single island, the infinite jet-flow property is satisfied. Proof: See Appendix A. The inclusion relations between sets of the chains with different properties are illustrated in Fig. 3. We show in the following example how Lemma can be used to verify if a chain satisfies the IJFP. Example : Let chain {A(t) be define by A(t) = t+ 0 t+ or t+ 0 t {{ if t is even 0 0 {{ if t is odd Then, {A(t) is weakly aperiodic for γ =, and also satisfies the IFP. Thus, according to Lemma, it also has the IJFP. B. Class P In [20], the authors introduced a class of chains of stochastic matrices called Class P. The chains of this class are of great importance in this paper as we fully characterize their (class-) ergodicity in Section IV-B. Furthermore, we carry out the continuous time counterpart results in Section V-B. Based on the work of Kolmogorov in [26], we know that for every chain {A(t) t 0 of stochastic matrices, there exists a sequence {π(t) t 0 of probability distribution vectors in R N, called an absolute probability sequence, such that π (t +)A(t) =π (t), t 0. (6)

4 2360 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 6, NO. 9, SEPTEMBER 206 Proof: See Appendix B. We again highlight that Fig. shows the inclusion relations between the three classes of chains defined in this subsection. III. A D-S THEOREM APPROACH Consider algorithm () and let {π(t) be an absolute probability sequence of {A(t) satisfying (6). We aim to form a chain {P (t) of stochastic matrices such that π i (t)p ij (t) =π j (t +)a ji (t), i, j V, t 0. (7) To do so, for every i, j Vand t 0,set { π j (t +)a ji (t)/π i (t) if π i (t) 0 p ij (t) = /N if π i (t) =0. Fig. 3. Inclusion relations between sets of the chains with IFP, AIFP, IJFP, and weak aperiodicity + IFP. Definition 8 [20]: Achain{A(t) is said to be in Class P if it admits an absolute probability sequence uniformly bounded away from zero, i.e., if there exists p > 0 such that π i (t) p, i V, t 0. It is not convenient in general to determine whether a chain belongs to Class P. In the following two lemmas, we state two important subclasses of Class P, previously defined in the literature, that help verify if a chain has class property P (Fig. ). Definition 9 [20]: Achain{A(t) of stochastic matrices is strongly aperiodic if there exists δ>0 such that a ii (t) δ, i V, t 0. Definition 0 [9]: Achain{A(t) of stochastic matrices is cut-balanced if for some K and any V Vand t 0 a ij (t) K a ij (t). j V j V i V i V Proposition [20, Lemma 9]: Strongly aperiodic, cutbalanced chains are in Class P. In fact, strongly aperiodic cut-balanced chains form a subclass of balanced asymmetric chains [2] which themselves are a subclass of Class P. Definition [2]: Achain{A(t) of stochastic matrices is said to be balanced asymmetric if there exists M such that for any V, V 2 Vofthe same cardinality, and any t 0 a ij (t) M a ij (t). j V 2 j V 2 i V i V Proposition 2: Balanced asymmetric chains are in Class P. The term strongly aperiodic is referred to as self-confident in [2]. We show in the following that (i) each matrix P (t), t 0 is stochastic, (ii) relation (7) is satisfied. (i) If π i (t) 0: p ij (t) = j j = π i(t) π i (t) = π j (t+)a ji (t) π i (t) = j π j(t+)a ji (t) π i (t) while if π i (t) =0,wehave: j p ij(t) = j /N =. (ii) If π i (t) 0, (7) is obviously satisfied, while if π i (t) =0, both sides of (7) become zero since 0=π i (t) = j π j (t +)a ji (t). Thus, we successfully formed chain {P (t) of stochastic matrices for which (7) is satisfied. Notice now that for any i V and t 0, from (7) π i (t +)=π i (t +) a ij (t) = j j = π j (t)p ji (t) j π i (t +)a ij (t) which implies that π (t +)=π (t)p (t),foranyt 0.Therefore, {π(t) forms the probability distribution vector of an inhomogeneous forward propagating Markov chain with transition chain {P (t). One can now take advantage of Sonin s D-S Theorem [24] to analyze the asymptotic behavior of the two chains {π(t) and {x(t) simultaneously. To employ the Sonin s D-S Theorem, one has to view π(t) as the vector of volumes m(t) in [24], while x(t) corresponds to the vector of concentrations α(t). Remark : Notice that Sonin in [24] starts with a forward propagating Markov chain {m(t) and then, a backward Markov chain α(t) follows (a forward-to-backward approach). In other words, the backward Markov chain {α(t) in [24] is not created independently. Thus, there would be no guarantee that one could use Sonin s D-S Theorem to characterize the asymptotic behavior of an arbitrary backward Markov chain. In our arguments above, thanks to the existence of an absolute

5 BOLOUKI AND MALHAMÉ: CONSENSUS ALGORITHMS AND THE DECOMPOSITION-SEPARATION THEOREM 236 probability sequence for any chain of stochastic matrices, we showed that the Sonin s forward-to-backward approach is in fact reversible, and therefore, the D-S Theorem would be applicable to all backward Markov chains. Let V (J s,j k ) denote the total flow between two arbitrary jets J s and J k in V over the infinite time interval, i.e., V(J s,j k )= r ij (t)+ r ij (t) t=0 i J k (t) j J s (t+) i J s (t) j J k (t+) where r ij (t) = Δ π i (t)p ij (t) =π j (t +)a ji (t). Recalling U from (3), we note that for every J s,j k in V, V (J s,j k ) U(J s,j k ). The following theorem on the limiting behavior of {x(t) and {π(t), is an immediate result of Sonin s D-S Theorem [24, Theorem ]. Theorem : Let linear averaging dynamics () with coupling chain {A(t) be given. Assume that {π(t) is an absolute probability sequence admitted by {A(t). Then, there exists an integer c, c N, and a decomposition of V into jetpartition (J 0,J,...,J c ), J k = {J k (t), 0 k c, such that irrespective of the initial conditions of (): (i) For every k, k c, there exist constants πk and x k, such that lim π i (t) =πk, t i J k (t) lim t x i t (t) =x k for every sequence {i t, i t J k (t). Furthermore lim t i J 0 (t) π i(t) =0. (ii) For every distinct k, s, 0 k, s c: V (J k,j s ) <. In the following section (Section IV-B), Theorem will be employed to characterize (class-)ergodicity properties of chains of stochastic matrices in Class P. IV. MAIN RESULTS IN DISCRETE TIME We now state our main results on (class-)ergodicity of discrete time chains of stochastic matrices. The first set of results consists of necessary conditions for (class-)ergodicity, while the second set fully characterizes (class-)ergodicity of chains in Class P. A. General Necessary Conditions Recalling the notions of infinite jet-flow property and islands from Definitions 4 and 7, the following theorem provides a necessary condition for class-ergodicity of an arbitrary chain. Theorem 2: Achain{A(t) of stochastic matrices is classergodic only if the infinite jet-flow property holds over each island of {A(t). Proof: On the contrary, assume that {A(t) is classergodic, yet there exists a proper jet J in an island I of {A(t) such that U A (J, I \ J) <. Recall from Definition, that by a proper jet in I, we mean J(t) I, t 0. Since U A (J, I \ J) is bounded and I is an island of {A(t), we conclude that U A (J, V\J) is bounded as well. Recalling the (8) definition of l -approximation from [8], a chain {B(t) is an l -approximation of chain {A(t) if for any i, j V a ij (t) b ij (t) <. t=0 We now form chain {B(t), anl -approximation of chain {A(t), by eliminating interactions between J and V\J at all times. From [8, Lemma ], l -approximations do not influence the ergodic classes of a chain. Therefore, {B(t) will remain class-ergodic with the same ergodic classes as {A(t). Also, the islands of {B(t) are the same as those of {A(t).Furthermore, U B (J, V\J) =0. Given two distinct arbitrary constants α and α 2, let states of a multi-agent system evolve via dynamics y(t +)=B(t)y(t), t 0, and be initialized at: y i (0) = α if i J(0), andy i (0) = α 2 otherwise. Since there is no interaction between J and V\J at any time, we conclude that for every t 0: y i (t) =α if i J(t), andy i (t) =α 2 otherwise. Since {B(t) is class-ergodic, lim t y i (t) exists for every i V. Obviously, in the infinite time interval [0, ), at least one of i J(t) and i J(t) must happen infinitely often (i.o.). If i J(t) happens i.o., then lim t y i (t) has to be α. Similarly, if i lies in J(t) i.o., we must have lim t y i (t) = α 2. Thus, exactly one of i J(t) and i J(t) can happen i.o. Therefore, in a finite time, i would lie in either J or J and stay there ever after. Thus, jet-limit J exists and is a proper subset of I, i.e., J I. Since the island structure is common for chains {A(t) and {B(t), I is also an island of {B(t),which implies that U B (J,I \ J ) is unbounded. This contradicts U B (J, I \ J) U B (J, V\J) =0, which completes the proof. Later in this section, we shall as well establish the sufficiency of the infinite jet-flow property over each island for classergodicity, provided {A(t) is in Class P. Notice now that the infinite flow property of {A(t), which is a necessary condition for ergodicity of {A(t) according to [22], [27], is equivalent to the existence of a single island. Thus, Theorem 2 immediately results in the following corollary. Corollary : Achain{A(t) of stochastic matrices is ergodic only if it has the infinite jet-flow property. From Fig. 3, Corollary provides a stronger necessary condition for ergodicity than [22, Theorem ] (necessity of the IFP for ergodicity) and [22, Theorem 2] (necessity of the AIFP for ergodicity), which is illustrated by the following example. Example 2: Let chain {A(t) be defined by A(t)= (t+) 2 (t+) 2 0 (t+) {{ if t is even or (t+) 0 (t+)2 2 (t+) {{ if t is odd If we define jet J by J(t) ={ if t is even and J(t) ={, 2 if t is odd, then: U(J, V\J) = t=0 (/(t +)2 ) <, which shows that {A(t) does not satisfy the infinite jet-flow property. Thus, from Corollary, {A(t) is not ergodic. On the other hand, as we show in the following, the same result cannot be concluded from Theorems and 2 of [22] since both the IFP and AIFP are satisfied. To prove that the IFP holds, one has to

6 2362 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 6, NO. 9, SEPTEMBER 206 show U(J, J) = for any time-invariant proper jet J. Out of six possible such jets, we only investigate the following three, and the other three can be treated in the same way since they are complements of these jets: ) J(t) ={: For any even time t: U t (J, J) = /(t + ) 2, which produces an unbounded sum. 2) J(t) ={2: Foranyt: U t (J, J) =, which produces an unbounded sum. 3) J(t) ={3: For any odd time t: U t (J, J) = /(t + ) 2, which produces an unbounded sum. To prove that the AIFP holds, one has to show unboundedness of U(J, J) for any jet J with a time-invariant size. We only investigate jets with time-invariant size, and jets of timeinvariant size 2 can be treated in the same way as their complements have time-invariant size. If jet J is time-invariant, U(J, J) = due the IFP, as proved above. Thus, assume that jet J is not time-invariant. Based on these assumptions, at least one of the following two cases must happen: ) There exist infinitely many times t such that J(t) ={: For any such t, since J(t) {,wehaveu t (J, J). Since it happens infinitely often, U(J, J) is unbounded. 2) There exist infinitely many times t such that J(t) ={3: Similar to the previous case, U(J, J) is unbounded. Thus, the AIFP holds, and this concludes the example. We now point out that the infinite jet-flow property is not sufficient for ergodicity in general. For instance, one can verify that the chain of Example is not ergodic, while it satisfies the infinite jet-flow property. Definition 2: AjetJ in V is called independent if the total influence of J on J is finite over the infinite time interval, i.e., t=0 i J(t+) j J(t) a ij (t) <. The following theorem, which is a generalization of Corollary, states yet another necessary condition for ergodicity of chain {A(t) of stochastic matrices. Two jets J and J 2 in V are called disjoint if J (t) J 2 (t) = for any t 0. Theorem 3: Achain{A(t) of stochastic matrices is ergodic only if no two disjoint independent jets in V exist. Proof: Assume on the contrary, that there exist two disjoint independent jets J and J 2 in V. Similar to the proof of Theorem 2, form chain {B(t), anl -approximation of {A(t), by eliminating the influence of J s on J s, s =, 2, at all times. Recall that {A(t) and {B(t) will share the same ergodic properties. Let states of a multi-agent system, y i (t), i N, evolve via dynamics y(t +)=B(t)y(t), t 0, and be initialized such that for every i J s (0) ( s =, 2), y i (0) = α s,whereα α 2. Then, for every t 0, wehave:y i (t) = α s, i J s (t) ( s =, 2). Since α α 2, consensus does not occur. Consequently, chain {B(t) and thus {A(t) could not possibly be ergodic. As an example where Theorem 3 applies, recall again {A(t) of Example. There, jet { and jet {3 are two disjoint independent jets in V = {, 2, 3. Thus, Theorem 3 implies that {A(t) is not ergodic. Remark 2: To see why Theorem 3 generalizes Corollary, we notice that without the IJFP, there exists a jet J such that U(J, V \ J) is bounded, which means that both jets J and V\J are independent. On the other hand, jet J and V\J are disjoint. Thus, infinite jet-flow imposes a weaker condition than the non-existence of any two disjoint independent jets. B. Convergence in Class P We now apply Theorem to Class P chains and find necessary and sufficient conditions for their (class-)ergodicity. For chains of Class P, it is immediately implied that in the jet decomposition of Theorem, there is no jet J 0.Otherwise, lim t i J 0 (t) π i(t) would be bounded away from zero by at least p, which is in contradiction with Theorem (i). Therefore, there is a jet-partition of V into jets J,...,J c, such that for every k =,...,c, lim t x it (t) =x k,forevery sequence {i t,wherei t J k (t). Proposition 3: Consider a multi-agent system with dynamics (), where chain {A(t) is in Class P. Then, the set of accumulation points of states is finite. Proof: Obvious if we note that {x k k c forms the set of accumulation points of states. Recalling the definitions of U and V from (3) and (8), we state a lemma followed by the main result of this subsection. Lemma 2: If {A(t) P, then for any two jets J and J 2 in V, V (J,J 2 )= ifand only if U(J,J 2 )=. Proof: The result is obvious if one notes that p U(J,J 2 ) V (J,J 2 ) U(J,J 2 ). Theorem 4: Achain{A(t) in Class P is class-ergodic if and only if the infinite jet-flow property holds over each island of {A(t). In case of class-ergodicity of {A(t), islands are the ergodic classes of {A(t) and constitute the jet limits in its Sonin s jet decomposition. Proof: We first assume that chain {A(t) in P is classergodic. Then, Theorem 2 implies that the IJFP holds over each island of the chain. We now show that if {A(t) P is classergodic, islands are the ergodic classes of {A(t). Let us call an agent i V a prime member of jet J k if i J k (t) infinitely often. Having defined the prime membership, there exists some Sonin s jet-decomposition of {A(t) such that each agent becomes the prime member of a unique jet. To obtain such a jet-decomposition, start with an arbitrary jet-decomposition and let any two jets with a common prime member merge. The merging process results in a Sonin s jet-decomposition with the desired property. Jets of such decomposition have the property that they become time-invariant after a finite time. Thus, the jet-limits exist for each jet and are ergodicity classes of {A(t). If i and j belong to the same jet-limit, they are in the same island since they are in the same ergodic class of {A(t) ([8], Lemma 2). Conversely, assume that i and j are neighbors in the infinite flow graph, i.e., t=0 (a ij(t)+a ji (t)) =.Ifi and j were to belong to different jet-limits J s, J k, then, U(J s,j k ) would be unbounded. Thus, based on Lemma 2, V (J s,j k ) would be unbounded as well, which contradicts Theorem (ii).

7 BOLOUKI AND MALHAMÉ: CONSENSUS ALGORITHMS AND THE DECOMPOSITION-SEPARATION THEOREM 2363 Therefore, every two neighbors in the infinite flow graph belong to the same jet-limit. Consequently, every i and j in the same island must be in the same jet-limit. To prove the sufficiency, let the IJFP hold over each island. Assume that (J,...,J c ) is a Sonin s jet-decomposition, and for any k =,...,c, lim t x it (t) =x k for every sequence {i t,wherei t J k (t).let I be an arbitrary island. We aim to show that, for every i I, lim t x i (t) exists. Keeping in mind that the aim is achieved if one of jets J,...,J c contains island I ever after a finite time, we take three following steps. Pick an arbitrary jet J k among J,...,J c. Step : We show that infinitely often we have I J k (t) = or I J k (t) =I. Indeed, assume instead that this behavior occurs only a finite number r of times, denoted t,...,t r.form a proper jet J in I such that J(t) =I J k (t), if t t i, i r. Notice that J(t) for t = t i, i r, can be any arbitrary proper subset of I. Since the IJFP holds over I, U(J, I \ J) is unbounded. On the other hand, except for a finite number of time indices t = t i, i r, U t (J, I \ J) U t (J k, V\J k ). This implies that U(J k, V\J k ) is unbounded, and, according to Lemma 2, so is V (J k, V\J k ). This is in contradiction with Theorem. Therefore, I J k (t) = or I happens infinitely often. This means that either one or both of the events I J k (t) = and I J k (t) =I occurs infinitely often. Step 2: We show that there are at most a finite number of times such that I J k (t) and I J k (t +). Indeed, denote ɛ = Δ 3 min { x s x l s l c. (9) Then, there exists T ɛ 0 such that x i (t) x l <ɛ, l =,...,c, i J l (t), t T ɛ. (0) For some given t T ɛ assume that: I J k (t) and I J k (t+). Therefore, there exists i I such that i J k (t) \ J k (t +). In view of (), (9), and (0), we then have a ij (t)(x j (t) x i (t)) ɛ. () j J k (t) On the other hand a ij (t)(x j (t) x i (t)) j J k (t) a ij (t) x j (t) x i (t) L j J k (t) j J k (t) a ij (t) (2) where L =max Δ i,j V x j (0) x i (0). Note that L remains an upper bound of x j (t) x i (t), t 0, since states are updated via a convex combination of previous states. Eqs. () and (2) imply that j J k (t) a ij(t) ɛ/l. Thus, since i I a lj (t) a ij (t) a ij (t) ɛ L. (4) l I j I j I j J k (t) Since U(I,V\I) <, inequality (3) can only occur for finitely many times t.this shows that if I J k (t) happens infinitely often, then there exists T such that I J k (t) for every t T. Consequently, lim t x i (t) exists, i I, and is equal to x k. Therefore, assume that for a fixed island I, I J k (t) happens only a finite number of times for every k, k c. Thus, from the result of Step, I J k (t) = must happen infinitely often for every k, k c. Step 3: We show that if I J k (t) = happens infinite times for every k, k c, the following contradiction occurs: For every k, k c, there exists T k 0 such that I J k (t) =, t T k. The proof is established by induction on k. With no loss of generality, assume that x < <x k. (k =): Recalling ɛ and T ɛ from (9) and (0), assume that for a fixed t T ɛ we have I J (t) = and I J (t +). Thus, there exists i I such that i J (t +)\ J (t). Therefore j J (t) a ij (t)(x j (t) x i (t)) ɛ. Noting that J (t) V\I, by repeating steps (2), (3), we conclude that there are at most finitely many times at which I J (t) = and I J (t +). This together with the fact that I J (t) = happens infinite times, shows that there exists T 0 such that I J (t) =, t T. k k ( <k c): Assume that for a fixed t max{t l l<k, wehavei J k (t) = and I J k (t + ). Thus, there exists i I such that i J k (t +)\ J k (t). Therefore a ij (t)(x j (t) x i (t)) ɛ. j k l= J l (t) Note once again that k l= J l (t) Ī, and repeat (2), (3) to show that for some T k 0: I J k (t) =, t T k. Corollary 2: Achain{A(t) P is ergodic if and only if it has the infinite jet-flow property. Since convergence of states in linear algorithm () occurs inside each jet J k, k c, for multiple consensus to occur(class-ergodicity of {A(t)), it suffices that for each jet of the D-S Theorem jet decomposition, its jet-limit exists. C. Relationship to Previous Work We now discuss how Theorem 4 and Corollary 2 subsume (class-)ergodicity results of [20], [2] that are to the best of our knowledge the most general results in the literature on (class-) ergodicity of discrete time chains of stochastic matrices. ) Weakly Aperiodic Chains in Class P [20]: Recall the definition of weak aperiodicity from Definition 6. Theorem 4 and Lemma immediately imply the following corollary which is the deterministic counterpart of Theorem 3 of [20]. Corollary 3: Every weakly aperiodic chain in Class P is class-ergodic, and the islands of the infinite flow graph associated with the chain constitute the ergodic classes. As a result, any weakly aperiodic chain in Class P with the infinite flow property is ergodic.

8 2364 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 6, NO. 9, SEPTEMBER 206 2) Strongly Aperiodic, Cut-Balanced Chains [20], [2]: Recalling the definitions of strong aperiodicity and cut-balance from Definitions 9 and 0, we report the following result from [20], [2] and show how it can be deduced from Theorem 4. Proposition 4 [20, Corollary 4], [2, Theorem 4]: If chain {A(t) is strongly aperiodic and cut-balanced, then it is classergodic and the islands form the ergodic classes of {A(t). Proof: Assume that {A(t) has strong aperiodicity and cut-balance properties with bounds δ and K, respectively. The chain being strongly aperiodic and cut-balanced, from Proposition or [20, Lemma 9], it is in Class P. Thus, from Theorem 4, it is sufficient to show that for an arbitrary island I and an arbitrary proper jet J in I, wehaveu(j, I \ J) = (that is the infinite jet flow property holds island-wise). Indeed, if jet-limit J exists, unboundedness of U(J, I \ J) is immediately implied from unboundedness of U(J,I \ J ) in view of the definition of islands. Otherwise, there are infinitely many instants t such that J(t) J(t +).Ateverysucht, thereexists i I such that i (J(t) \ J(t +)) (J(t +)\ J(t)). Therefore, recalling (4), U t (J, I \ J) a ii (t) δ. Since there are infinitely many such times, U(J, I \ J) is unbounded. 3) Balanced Asymmetric Chains [2]: Recall the definition of balanced asymmetry from Definition. (Class-)ergodicity of balanced asymmetric chains is characterized in the following proposition, as stated in [2]. We again show that one can easily derive this proposition from our results in this work. Proposition 5 [2]: Assume that a chain {A(t) of stochastic matrices is balanced asymmetric. Then, {A(t) is classergodic if and only if the absolute infinity property holds over each island of {A(t). Furthermore, in case of class-ergodicity, islands are the ergodic classes of {A(t). Consequently, {A(t) is ergodic if and only if it has the absolute infinite flow property. Proof: Let {A(t) be balanced asymmetric with bound M. From Proposition 2, we know that {A(t) P. Therefore, taking advantage of Theorem 4, it suffices to show that absolute infinite flow and infinite jet-flow properties are equivalent on each island. Obviously, the AIFP is implied by the IJFP. To prove the converse, let the AIFP hold over each island. Assume that I is an arbitrary island of {A(t) and J is an arbitrary jet in I. If the cardinality of jet J becomes time-invariant after a finite time, unboundedness of U(J, I \ J) is immediately implied from the AIFP over I. Otherwise, the cardinality of J increases infinitely often by at least. In this case, assume that for a fixed t 0,wehave J(t +) > J(t). For an arbitrary i J(t +),lett J(t +)be such that i T and T = J(t). Thus, by the balanced asymmetry property j J(t) a ij (t) l T M l T j J(t) j J(t) a lj (t) a lj (t) M Therefore a ij (t) J(t+) M i J(t+) j J(t) l J k (t+) j J(t) i J(t+) j J(t) a lj (t). a ij (t). (4) On the other hand a ij (t)= J(t +) i J(t+) j J(t) i J(t+) j J(t) a ij (t). (5) (4) and (5) together imply J(t +) a ij (t) +M J(t +) +M. (6) i J(t+) j J(t) Since (6) happens infinitely often, we conclude that U(J,V\J) is unbounded. Moreover U(J, V\J)+U(I \J, V\I)=U(J, I \J)+U(I,V\I) (7) and since U(I,V\I) is bounded because I is an island, unboundedness of U(J, V\J) implies that U(J, I \ J) =. This completes the proof. V. C ONTINUOUS TIME CASE In this section, we investigate linear consensus algorithms in continuous time. In particular, we aim to characterize (class- )ergodicity properties of chains in a continuous time version of Class P as will be defined later on. One may define a general linear consensus algorithm in continuous time as follows: ẋ = A(t)x(t),t t 0 (8) where x(t) again represents the vector of states, and {A(t) is the coupling chain of the system. It is assumed that each matrix of coupling chain A(t) has zero row sum and nonnegative off-diagonal elements. Moreover, each element a ij (t) of A(t), i, j V, is a measurable function which is bounded for any bounded time interval. These constraints suggest a view of {A(t) as the evolution of the intensity matrix of a time inhomogeneous Markov chain in continuous time. Let Φ(t, ), t, 0, represent the state transition matrix associated with dynamics (8), i.e., x(t) =Φ(t, )x(), t t 0. Note that Φ(t, ) is a stochastic matrix for every t t 0. More specifically, Φ i,j (t, ) can be considered as transition probability of a backward propagating inhomogeneous Markov chain. In particular, for every t 2 t 0,wehave Φ i,j (t 2,)= k Φ i,k (t 2,t )Φ k,j (t,) with the following set of conditions: (i) Φ i,j (t, ) 0, (ii) j Φ i,j(t, ) =, (iii) Φ i,j (t, t) =δ ij,whereδ ij is the Kronecker symbol. Coupling chain {A(t) is said to be ergodic if for every, Φ(t, ) converges to a matrix with equal rows as t. Similar to the discrete time case, ergodicity of {A(t) is equivalent to the occurrence of consensus in (8) irrespective of the initial conditions. Moreover, {A(t) is class-ergodic if for every, lim t Φ(t, ) exists, but with possibly distinct rows. Chain {A(t) is class-ergodic if and only if multiple consensus occurs

9 BOLOUKI AND MALHAMÉ: CONSENSUS ALGORITHMS AND THE DECOMPOSITION-SEPARATION THEOREM 2365 in (8) irrespective of the initial conditions. For simplicity, from now on we assume that t 0 =0. Recall that the state transition matrix associated with (8) can be expressed via the Peano-Baker series [28, Section.3] t Φ(t, ) =I N N + t + A(σ ) σ t A(σ )dσ + A(σ 2 ) σ 2 A(σ ) σ A(σ 2 )dσ 2 dσ A(σ 3 )dσ 3 dσ 2 dσ + (9) where I N N denotes the N N identity matrix. Remember that Φ(t, ) is invertible for every t 0. Similar to the discrete time case, based on [26], for every state transition matrix Φ(t, ), t, 0, there exists an absolute probability sequence {π(t), t 0, such that π () =π (t)φ(t, ), t, 0. (20) We use the following notation throughout this section. Φ i (t, ) and Φ i,j (t, ), i, j N, denote the ith column and the (i, j)th element (intersectionof ith row and jth column) of Φ(t, ) respectively, while Φ i (t, ) refers to the ith column of Φ (t, ) (the prime acts first), which is also the transpose of the ith row of Φ(t, ). A. A Geometric Approach Inspired by Shen [25], we take a geometric approach in this section to analyze the convergence properties of linear consensus dynamics (8). For chain {A(t), define polytope C t,, t 0 as the convex hull of points in R N corresponding to the columns of the transpose of associated state transition matrix Φ(t, ): { N N C t, = α i Φ i (t, ) α i 0 i Vand α i =. i= Notice that due to invertibility of Φ(t, ), its rows are linearly independent and therefore none of them lies in the convex hull of the rest. Thus, C t, is has exactly N vertices for any t 0, with Φ i (t, ) s comprising the N vertices. From [25, Proposition 5.], we know that for every t 2 t, we have: C t2, C t,. It means that for any fixed, polytopes C t, s form a monotone decreasing sequence of polytopes in R N as t grows. An example of these nested polytopes projected on a two-dimensional subspace of R N is depicted in Fig. 4. It is to be noted that ergodicity of {A(t) is equivalent to the nested polytopes converging to a point. The sequence of polytopes being monotone decreasing implies that, for every 0, lim t C t, exists and is also a polytope in R N.Let C be the limiting polytope with c vertices. It is clear that c N. One can show that the value of c is independent of [29]. Thus, let c be the constant value of c,andv,...,v c be the c vertices of limiting polytope C 0. Assume that {0 t is a sequence of agents, i.e., 0 t Vfor every t 0. i= Fig. 4. Example of nested polygons converging to a triangle. Theorem 5: Let an agent sequence {0 t t 0, 0 t V,besuch that the distance between Φ 0 t (t, 0) and set {v,...,v c, i.e., min Φ 0 t (t, 0) v i i c does not vanish as t grows. Then: lim inf t π 0t (t) =0,where {π(t) is an absolute probability sequence admitted by Φ(t, ), defined according to (20). Proof: For any t 0, let:z(t) =Φ Δ 0 t (t, 0). Notice that vector v i, i c, lies outside of the convex hull of v j s, j i. Letw i be a nearest point to v i, on the convex hull of v j s, j i. Forasmallɛ > 0, draw an affine hyperplane, distant ɛ from v i, crossing segment v i w i and orthogonal to it. For a sufficiently small ɛ, there must exist an increasing time sequence t,t 2,...,wherelim k t k =, such that v i and each z(t k ) lie on opposite sides of the affine hyperplane for every i, i c. Otherwise, the distance between {z(t) and set {v,...,v c would converge to zero. Notice in particular Furthermore, define min i c z(t k) v i >ɛ,k =, 2,... (2) ɛ Δ = 4 min { v i w i i c, ɛ Δ =min{ɛ,ɛ and for an arbitrary constant δ, 0 <δ<,let ɛ Δ = δɛ. We shall take advantage of the following arguments: Argument : SinceC t,0 converges to C 0 as t grows, there exists T 0 such that, for any t T, every point in C t,0 lies within an ɛ -distance of C 0. Argument 2: Recalling the time sequence {t k from earlier in the proof, there exist c agent sequences {i tk, i c, such that Φ i tk (t k, 0) converges to v i as k grows. Therefore, for some integer K>0 Φ i tk (t k, 0) v i <ɛ, k K. (22)

10 2366 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 6, NO. 9, SEPTEMBER 206 i c, letη Δ = j S i Φ i t,j(t, T ). SinceΦ(t, T ) is rowstochastic, it follows that j S i Φ i t,j(t, T )= η. Using (25), we now conclude that the value η(ɛ + ɛ)+( η)0 = η(ɛ + ɛ) is a lower bound for the distance from Φ i t (t, 0) to affine hyperplane m i. Thus, from (22), we must have: η(ɛ + ɛ) < 2ɛ. Therefore, remembering ɛ = ɛ /δ, we obtain: η< 2δ/( + δ) < 2δ, which results in (23) and (24). Hence, for every i, i c, from (23) and the fact that S 0 V\S i Φ it,j(t, T ) Φ it,j(t, T ) < 2δ. (26) j S 0 j S i Fig. 5. l i and m i are orthogonal to segment v i w i. For every i, i c, draw an affine hyperplane l i, parallel to the one drawn previously, distant ɛ from v i, crossing segment v i w i. Draw also an affine hyperplane m i, parallel to l i,onthe other side of v i, distant ɛ from v i (see Fig. 5).Let a fixed T max(t,t K ) belong to the time sequence {t k, and define for every i, i c S i Δ = { j V Φ j (T,0) lies in strip margined by l i,m i. Notice that S i s, i c, are non-empty since i T S i according to Argument 2 [see (22)]. Moreover, define S 0 = Δ V\ c j= Sj.SinceT belongs to the time sequence {t k, we know that z(t ) and v i lie on opposite sides of l i (notice ɛ<ɛ )for every i, i c. It means that 0 T S 0 and, therefore, S 0 is non-empty as well. One can also use Argument to show that S i s, i c, are disjoint. Thus, S 0,...,S c partition the agent set V.Lett T from the time sequence {t k be arbitrary but fixed. We claim that for every i, i c, the following two inequalities hold: Φ it,j(t, T ) < 2δ (23) j S i Φ it,j(t, T ) > 2δ. (24) j S i To prove the claim, we write Φ i t (t, 0) = Φ (T,0)Φ i t (t, T )= j V Φ it,j(t, T )Φ j(t,0) = j S i Φ it,j(t, T )Φ j (T,0)+ j S i Φ it,j(t, T )Φ j (T,0) (25) and use (25) to find a lower bound for the distance from Φ i t (t, 0) to affine hyperplane m i (see Fig. 5). For a fixed i, We notice that the vertices of C T are mapped to the vertices of C 0 under linear operator φ T,0 defined by φ T,0 (v) =Φ Δ (T,0)v, v R N.Letu i s, i c, be the vertices of C T where u i = φ T,0 (v i). Recalling lim k Φ i tk (t k, 0) = v i, the continuity of the inverse of operator φ T,0 results in lim k Φ i tk (t k,t)= u i. This, and (26) in limit, imply that j S 0(u i) j 2δ. Consequently, since any û C T (which is in principle a stochastic vector) can be written as a convex combination of u i s, i c,wemusthave û j 2δ, û C T. (27) j S 0 We know that for a sufficiently large time T >T, Φ i (t, T ) lies within the δ-distance of C T for every i V and t T.For arbitrary but fixed i V and t T,letû C T be such that Φ i (t, T ) û δ. Thus, from (27) Φ i,j (t, T ) (δ +û j ) ( S 0 +2)δ (N +2)δ. j S 0 j S 0 Hence, for any t T i V (28) j S 0 Φ i,j (t, T ) N(N +2)δ. (29) Recalling that 0 T S 0, (29) implies that for any t T Φ i,0t (t, T ) Φ i,j (t, T ) N(N +2)δ. i V i V j S 0 Thus π 0T (T )=π (t)φ 0T (t, T )= π i (t)φ i,0t (t, T ) i V Φ i,0t (t, T ) N(N +2)δ. (30) i V Noticing that δ was chosen arbitrarily and T arbitrarily large, by letting δ go to zero, we conclude from (30) that lim inf t π 0t (t) =0. B. Convergence in Class P A continuous time chain {A(t) is said to be in Class P if its associated state transition matrix Φ(t, ) admits an absolute

11 BOLOUKI AND MALHAMÉ: CONSENSUS ALGORITHMS AND THE DECOMPOSITION-SEPARATION THEOREM 2367 probability sequence uniformly bounded away from zero, i.e., in view of (20), if there exists p > 0 such that π i (t) p, i V, t 0. Let the infinite flow graph of a continuous time chain {A(t) be defined according to Definition 7 by replacing summation with integral. As a result of Theorem 5, we state the following result on the convergence properties of system (8) when the coupling chain is in Class P. Theorem: Let state transition matrix Φ(t, ), t, 0, associated with chain {A(t) be in Class P. Then, {A(t) is class-ergodic and its islands constitute its ergodic classes, i.e., consensus clusters of system (8). In particular, {A(t) is ergodic if and only if its satisfies the infinite flow property. The following theorem clarifies that Theorem 6 generalizes convergence results of [9, Theorem ] which, to the best of our knowledge, provides the most general existing result on the occurrence of (multiple) consensus in continuous time linear dynamics (8). Theorem 7: If transition chain {A(t) in (8) is cut-balanced, then state transition matrix Φ(t, ), t 0,isinClassP. Proof: See Appendix C. VI. CONCLUSION We considered a general linear distributed averaging algorithm in both discrete time and continuous time. Following [22], and recalling the notion of jets from [23], [24], we introduced a property of chains of stochastic matrices, more precisely, the infinite jet-flow property in the discrete time case. The latter property is shown to be a strong necessary condition for ergodicity of a chain. Moreover, for the chain to be class-ergodic, the infinite jet-flow property must hold over each connected component of the infinite flow graph, as defined in [20]. We then illustrated the close relationship between Sonin s D-S Theorem and convergence properties of linear consensus algorithms. By employing the D-S Theorem, we showed in the discrete time case that the necessary conditions found earlier are also sufficient in case the chain is in Class P [20]. We argued that the obtained equivalent conditions for ergodicity and class-ergodicity of chains in Class P can subsume the previous related results in the literature, [20], [2] in particular. A geometric approach was then introduced to investigate the convergence properties of continuous time linear consensus algorithms. The approach turned out to be a powerful method to extend our discrete time results to the continuous time case. In future work, we shall attempt an extension of our results to the case when the number of agents increases to infinity, although the D-S Theorem holds only if N is finite. APPENDIX A. Proof of Lemma Let {A(t) be weakly aperiodic, I be an arbitrary island of {A(t), and J be an arbitrary jet in I. If jet-limit J exists, since I is a connected component of the infinite flow graph, U(J,I \ J ) is unbounded. Consequently, U(J, I \ J) is unbounded and the lemma holds. Thus instead, assume that for jet J, the jet-limit does not exist. Therefore, for infinitely many times t, wemusthave:j(t +) J(t). Lett be fixed and J(t +) J(t). Thus, there exists i J(t +)\ J(t). From the weak aperiodicity property of {A(t) [see (5)], for every j J(t), there exists l Vsuch that γa ij (t) a li (t).a lj (t) min {a li (t),a lj (t) U t (J, V\J) where U t is defined in (4). The reason for the last inequality is that, whether l J(t +)or l J(t +), one of a li (t),a lj (t) appears in U t (J, V\J). Hence γa ij (t) J(t) U t (J, V\J). (3) j J(t) j J(t) On the other hand γa ij (t) =γ a ij (t) j J(t) = γ a ij (t) γ ( U t (J, V\J)). j J(t) Relations (3) and (32) imply (32) U t (J, V\J) γ/(γ + J(t) ) >γ/(γ + N). (33) Since (33) holds for infinitely many times t, U(J, V\J) = t=0 U t(j, V\J) is unbounded, and so is U(J, I \ J) (since J is a jet in I,andI is an island). B. Proof of Proposition 2 To prove Proposition 2, we shall use a rotational transformation technique [22, Section 3.]. To this aim, we first need to state the following lemma. Lemma 3: Let A be an (N N) balanced asymmetric matrix with bound M. Then, there exists a permutation matrix P N N such that the product PA is strongly aperiodic with bound δ =4/(MN 2 +4N 4). Proof: Form a bipartite-graph H(V, E) from A with N nodes in each part. Let V and V 2, each a copy of V, besetsof nodes of the two parts of H. Foreveryi V and j V 2, connect i to j if a ij δ =4/(MN 2 +4N 4).Wewishtoshow that H has a perfect matching. By Hall s Marriage Theorem [30, Theorem 5.2], it suffices to show that for every subset K V, we have D(K) K where D(K) ={j V 2 i Ks.t. (i, j) E. Indeed, assume that on the contrary, there exists K V such that k = D(K) < K = k. LetK = {c,...,c k and D(K) ={d,...,d k. DefineK K by K = {c,...,c k. We now have a ij <k (N k )δ δn 2 /4. (34) i K j D(K)

12 2368 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 6, NO. 9, SEPTEMBER 206 On the other hand a ij i K j D(K) i K\K j D(K) a ij =(k k ) i K\K j D(K) a ij (k k ) (k k )(N k )δ (N )δ. (35) Since K,D(K) V are of identical cardinalities, the balanced asymmetry property of A together with (34) and (35) imply (N )δ <δmn 2 /4, which contradicts definition of δ in the lemma. Therefore, H has a perfect matching and consequently, there exists a permutation such that a (i),i δ, i. Thus, the permutation matrix P with e (i) as its ith row, where e j denotes a row vector of length N with in the jth position and 0 in every other position, is such that the product PA is strongly aperiodic with δ. We now continue the proof of Proposition 2. Let {A(t) be a balanced asymmetric chain with bound M. Set: δ = 4/(MN 2 +4N 4). We recursively define sequence {P (t) of permutation matrices as follows: From Lemma 3, we know that there exists a permutation matrix P (0) such that the product P (0)A(0) is strongly aperiodic with δ. Find permutation matrix P (t), t, such that the product P (t)a(t)p (t ) is strongly aperiodic with δ. Note that the existence of P (t) is implied by Lemma 3, taking into account the fact that the product A(t)P (t ) is balanced asymmetric with bound M,since the columns of the product are a permutation of the columns of A(t), itself a balanced asymmetric matrix with bound M. Hence, if we define for every t 0, B(t) =P (t)a(t)p (t ), then, {B(t) has both the strong aperiodicity and balanced asymmetry properties. Since balanced asymmetry is stronger than cut-balance, chain {B(t) is both strongly aperiodic and cut-balanced. Thus, from [20], we conclude that chain {B(t) belongs to Class P. Furthermore, it is straightforward to show that if {π(t) is an absolute probability sequence adapted to chain {B(t),then{P (t )π(t),wherep ( ) = I N N,is an absolute probability sequence adapted to chain {A(t).This immediately implies that {A(t) P. C. Proof of Theorem 7 To prove Theorem 7, we need the following two lemmas. Assume that Φ(t, ), t 0, is the state transition matrix associated with (8) Lemma 4: For every j Vand 0: π j () inf Φ i,j (t, ). t Proof: i V Obvious, since for every t π j () =π(t)φ j (t, ) = i V π i (t)φ i,j (t, ) i V Φ i,j (t, ). Lemma 5: A state transition matrix Φ(t, ) associated with (8), is in Class P if and only if for every j V Φ i,j (t, ) > 0. inf t i V Proof: The only if part is an immediate result of Lemma 4, and the if part is a result of the way the existence of the absolute probability sequence can be obtained in [26] by always choosing to initialize agent probabilities on finite intervals with a uniform distribution. Now, let {A(t) be cut-balanced with bound K.Inview of Lemma 5, our aim is to show that: /N e Φ A (t, ) p e, for some p > 0, wheree =[,...,], and the inequality is to be understood element-wise. Assume that α =sup{ a Δ ii (t ) i V, t t. Notice that α exists since each a ii (t) is bounded for any bounded interval by the assumption. Let chain {B(t) be such that B(t )= A(t )+2αI, t t, wherei is the identity matrix. It is easy to verify that Φ B (t, ) =e 2α(t ) Φ A (t, ). Moreover, by construction, diagonal elements of B(t ), t t, are greater than or equal to α. Note that B(t )( t t) is not a stochastic matrix; instead each of its rows sums up to 2α. We calculate in the following, /N e Φ B (t, ). Therefore, from the Peano-Baker series (9), the expression t σ σ k N e B(σ ) B(σ 2 ) B(σ k )dσ k dσ (36) is of interest. Expression (36) is equal to (2α) k N e t B(σ ) 2α which is also equal to t (2α) k σ σ k σ B(σ ) N e 2α σ k B(σ 2 ) 2α B(σ k ) 2α dσ k dσ B(σ 2 ) 2α B(σ k) 2α dσ k dσ. (37) Note that B(t )/2α is a sequence of transition matrices which generates a Markov chain which is both strongly aperiodic and cut-balanced, and hence in Class P ([20, Lemma 9]). As a result, there exists a positive p such that B(σ ) N e 2α B(σ 2) 2α B(σ k) 2α p e. (38) Inequality (38) implies that expression (37), and consequently (36), is greater than or equal to ((2α) k p (t ) k )/k!. Thus, writing /N e Φ B (t, ) as a sum of expressions like (36), we conclude N e Φ B (t, ) (2α) k p (t ) k = p e 2α(t ). k! k=0 Thus, (/N )e Φ A (t, ) p e 2α(t ).e 2α(t ) = p, and from Lemma 5, the theorem is proved.

13 BOLOUKI AND MALHAMÉ: CONSENSUS ALGORITHMS AND THE DECOMPOSITION-SEPARATION THEOREM 2369 REFERENCES [] G. Flierl, D. Grünbaum, S. Levins, and D. Olson, From individuals to aggregations: The interplay between behavior and physics, J. Theoret. Biol., vol. 96, no. 4, pp , 999. [2] I. D. Couzin, J. Krause, N. R. Franks, and S. A. Levin, Effective leadership and decision-making in animal groups on the move, Nature, vol. 433, no. 7025, pp , [3] F. Cucker and S. Smale, Emergent behavior in flocks, IEEE Trans. Autom. Control, vol. 52, no. 5, pp , May [4] J. N. Tsitsiklis, D. P. Bertsekas, and M. Athans, Distributed asynchronous deterministic and stochastic gradient optimization algorithms, IEEE Trans. Autom. Control, vol. 3, no. 9, pp , Sep [5] A. Jadbabaie, J. Lin, and A. S. Morse, Coordination of groups of mobile autonomous agents using nearest neighbor rules, IEEE Trans. Autom. Control, vol. 48, no. 6, pp , Jun [6] F. Cucker, S. Smale, and D.-X. Zhou, Modeling language evolution, Found. Comput. Math., vol. 4, no. 3, pp , [7] N. A. Lynch, Distributed Algorithms. Burlington, MA, USA: Morgan Kaufmann, 996. [8] M. H. DeGroot, Reaching a consensus, J. Amer. Statist. Assoc., vol. 69, no. 345, pp. 8 2, 974. [9] S. Chatterjee and E. Seneta, Towards consensus: Some convergence theorems on repeated averaging, J. Appl. Probabil., vol. 4, no., pp , 977. [0] J. N. Tsitsiklis, Problems in Decentralized Decision Making and Computation, DTIC Doc., Tech. Rep., 984. [] D. P. Bertsekas and J. N. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods. Upper Saddle River, NJ, USA: Prentice-Hall, 989. [2] V. Blondel, J. Hendrickx, A. Olshevsky, and J. Tsitsiklis, Convergence in multiagent coordination, consensus, flocking, in Proc. 44th IEEE Conf. Decision and Control and European Control Conf. (CDC-ECC 2005), 2005, pp [3] L. Moreau, Stability of multiagent systems with time-dependent communication links, IEEE Trans. Autom. Control, vol. 50, no. 2, pp , Feb [4] J. M. Hendrickx and V. Blondel, Convergence of different linear and nonlinear vicsek models, in Proc. 7th Int. Symp. Mathematical Theory of Networks and Systems (MTNS2006), 2006, pp [5] J. M. Hendrickx, Graphs and Networks for the Analysis of Autonomous Agent Systems, Ph.D. dissertation, Ecole Polytechnique, Palaiseau, France, [6] S. Li, H. Wang, and M. Wang, Multi-Agent Coordination Using Nearest Neighbor Rules: A Revisit to Vicsek Model, arxiv preprint cs/040702, [7] J. Lorenz, A stabilization theorem for dynamics of continuous opinions, Physica A: Statist. Mechan. and its Applic., vol. 355, no., pp , [8] B. Touri and A. Nedić, On approximations and ergodicity classes in random chains, IEEE Trans. Autom. Control, vol. 57, no., pp , Nov [9] J. M. Hendrickx and J. N. Tsitsiklis, Convergence of type-symmetric and cut-balanced consensus seeking systems, IEEE Trans. Autom. Control, vol. 58, no., pp , Jan [20] B. Touri and A. Nedić, Product of random stochastic matrices, IEEE Trans. Autom. Control, vol. 59, no. 2, pp , Feb [2] S. Bolouki and R. Malhamé, Ergodicity and class-ergodicity of balanced asymmetric stochastic chains, in Proc. Eur. Control Conf. (ECC), 203, pp [22] B. Touri and A. Nedić, On backward product of stochastic matrices, Automatica, vol. 48, no. 8, pp , 202. [23] D. Blackwell, Finite non-homogeneous chains, Ann. Mathemat., pp , 945. [24] I. M. Sonin et al., The decomposition-separation theorem for finite nonhomogeneous markov chains and related problems, in Markov Processes and Related Topics: A Festschrift for Thomas G. Kurtz. Shaker Heights, OH: Institute of Mathematical Statistics, 2008, pp. 5. [25] J. Shen, A geometric approach to ergodic non-homogeneous Markov chains, Lecture Notes in Pure and Applied Mathematics, pp , [26] A. Kolmogoroff, Zur theorie der markoffschen ketten, Mathematische Annalen, vol. 2, no., pp , 936. [27] B. Touri and A. Nedić, On ergodicity, infinite flow, consensus in random models, IEEE Trans. Autom. Control, vol. 56, no. 7, pp , Jul. 20. [28] R. W. Brockett, Finite Dimensional Linear Systems. New York, NY, USA: Wiley, 970. [29] S. Bolouki, R. P. Malhamé, M. Siami, and N. Motee, Éminence Grise Coalition: On the Shaping of Public Opinion, arxiv preprint, 204. [30] J. A. Bondy and U. S. R. Murty, Graph Theory With Applications. London, U.K.: Macmillan, 976, vol. 6. Sadegh Bolouki (M 4) received the B.S. degree from Sharif University of Technology, Tehran, Iran, in 2008 and the Ph.D. degree from École Polytechnique de Montréal, Montreal, QC, Canada, in 204, both in electrical engineering. From January 204 to July 205, he was a research scholar with the Department of Mechanical Engineering and Mechanics, Lehigh University. Since August 205, he has been a postdoctoral scholar at the Coordinated Science Lab, University of Illinois at Urbana-Champaign. His research interests include the areas of decentralized and distributed control, opinion dynamics, consensus, and game theory. Roland P. Malhamé (S 82 M 92) received the B.S. degree from the American University of Beirut in 976, the M.S. degree from the University of Houston in 978, and the Ph.D. degree from Georgia Institute of Technology in 983, all in electrical engineering. After single year stays at the University of Quebec, and CAE Electronics Ltd., Montreal, QC, Canada, he joined École Polytechnique de Montréal, in 985, where he is currently Professor of Electrical Engineering. In 994, 2004, and 202, he was on sabbatical leave with Laboratoire des Signaux Systéms, Gif-sur-Yvette, France, and University of Rome Tor Vergata, Roma, Italy, respectively. His interest in statistical mechanics inspired approaches to the analysis and control of large scale systems has led him to contributions in the area of aggregate electric load modeling, and to the early developments of the theory of mean field games. His current research interests are in collective decentralized decision making schemes, and the development of mean field based control algorithms in the area of smart grids. From June 2005 to June 20, he headed GERAD, the Group for Research on Decision Analysis. Dr. Malhamé is an Associate Editor for the International Transactions on Operations Research.