Constrained Consensus and Optimization in Multi-Agent Networks arxiv: v2 [math.oc] 17 Dec 2008

Transcription

1 LIDS Report Constrained Consensus and Optiization in Multi-Agent Networks arxiv: v2 [ath.oc] 17 Dec 2008 Angelia Nedić, Asuan Ozdaglar, and Pablo A. Parrilo February 15, 2013 Abstract We present distributed algoriths that can be used by ultiple agents to align their estiates with a particular value over a network with tie-varying connectivity. Our fraework is general in that this value can represent a consensus value aong ultiple agents or an optial solution of an optiization proble, where the global objective function is a cobination of local agent objective functions. Our ain focus is on constrained probles where the estiate of each agent is restricted to lie in a different constraint set. To highlight the effects of constraints, we first consider a constrained consensus proble and present a distributed projected consensus algorith in which agents cobine their local averaging operation with projection on their individual constraint sets. This algorith can be viewed as a version of an alternating projection ethod with weights that are varying over tie and across agents. We establish convergence and convergence rate results for the projected consensus algorith. We next study a constrained optiization proble for optiizing the su of local objective functions of the agents subject to the intersection of their local constraint sets. We present a distributed projected subgradient algorith which involves each agent perforing a local averaging operation, taking a subgradient step to iniize its own objective function, and projecting on its constraint set. We show that, with an appropriately selected stepsize rule, the agent estiates generated by this algorith converge to the sae optial solution for the cases when the weights are constant and equal, and when the weights are tie-varying but all agents have the sae constraint set. We would like to thank the associate editor, three anonyous referees, and various seinar participants for useful coents and discussions. This research was partially supported by the National Science Foundation under CAREER grants CMMI and DMI , under grant ECCS , and by the DARPA ITMANET progra A. Nedić is with the Industrial and Enterprise Systes Engineering Departent, University of Illinois at Urbana-Chapaign, Urbana IL e-ail: angelia@uiuc.edu) A. Ozdaglar and P. A. Parrilo are with the Laboratory for Inforation and Decision Systes, Electrical Engineering and Coputer Science Departent, Massachusetts Institute of Technology, Cabridge MA, e-ails: asuan@it.edu, parrilo@it.edu)

2 1 Introduction There has been uch interest in distributed cooperative control probles, in which several autonoous agents collectively try to achieve a global objective. Most focus has been on the canonical consensus proble, where the goal is to develop distributed algoriths that can be used by a group of agents to reach a coon decision or agreeent on a scalar or vector value). Recent work also studied ulti-agent optiization probles over networks with tie-varying connectivity, where the objective function inforation is distributed across agents e.g., the global objective function is the su of local objective functions of agents). Despite uch work in this area, the existing literature does not consider probles where the agent values are constrained to given sets. Such constraints are significant in a nuber of applications including otion planning and alignent probles, where each agent s position is liited to a certain region or range, and distributed constrained ulti-agent optiization probles. In this paper, we study cooperative control probles where the values of agents are constrained to lie in closed convex sets. Our ain focus is on developing distributed algoriths for probles where the constraint inforation is distributed across agents, i.e., each agent only knows its own constraint set. To highlight the effects of different local constraints, we first consider a constrained consensus proble and propose a projected consensus algorith that operates on the basis of local inforation. More specifically, each agent linearly cobines its value with those values received fro the tie-varying neighboring agents and projects the cobination on its own constraint set. We show that this update rule can be viewed as a version of the alternating projection ethod where, at each iteration, the values are cobined using weights that are varying in tie and across agents, and projected on the respective constraint sets. We provide convergence and convergence rate analysis for the projected consensus algorith. Due to the projection operation, the resulting evolution of agent values has nonlinear dynaics, which poses challenges for the analysis of the algorith s convergence properties. To deal with the nonlinear dynaics in the evolution of the agent estiates, we decopose the dynaics into two parts: a linear part involving a tievarying averaging operation and a nonlinear part involving the error due to the projection operation. This decoposition allows us to represent the evolution of the estiates using linear dynaics and decouples the analysis of the effects of constraints fro the convergence analysis of the local agent averaging. The linear dynaics is analyzed siilarly to that of the unconstrained consensus update, which relies on convergence of transition atrices defined as the products of the tie-varying weight atrices. Using the properties of projection and agent weights, we prove that the projection error diinishes to zero. This shows that the nonlinear parts in the dynaics are vanishing with tie and, therefore, the evolution of agent estiates is alost linear. We then show that the agents reach consensus on a coon estiate in the liit and that the coon estiate lies in the intersection of the agent individual constraint sets. We next consider a constrained optiization proble for optiizing a global objective function which is the su of local agent objective functions, subject to a constraint set given by the intersection of the local agent constraint sets. We focus on distributed algoriths in which agent values are updated based on local inforation given by the 2

3 agent s objective function and constraint set. In particular, we propose a distributed projected subgradient algorith, which for each agent involves a local averaging operation, a step along the subgradient of the local objective function, and a projection on the local constraint set. We study the convergence behavior of this algorith for two cases: when the constraint sets are the sae, but the agent connectivity is tie-varying; and when the constraint sets X i are different, but the agents use unifor and constant weights in each step, i.e., the counication graph is fully connected. We show that with an appropriately selected stepsize rule, the agent estiates generated by this algorith converge to the sae optial solution of the constrained optiization proble. Siilar to the analysis of the projected consensus algorith, our convergence analysis relies on showing that the projection errors converge to zero, thus effectively reducing the proble into an unconstrained one. However, in this case, establishing the convergence of the projection error to zero requires understanding the effects of the subgradient steps, which coplicates the analysis. In particular, for the case with different constraint sets but unifor weights, the analysis uses an error bound which relates the distances of the iterates to individual constraint sets with the distances of the iterates to the intersection set. Related literature on parallel and distributed coputation is vast. Most literature builds on the seinal work of Tsitsiklis [26] and Tsitsiklis et al. [27] see also [3]), which focused on distributing the coputations involved with optiizing a global objective function aong different processors assuing coplete inforation about the global objective function at each processor). More recent literature focused on ulti-agent environents and studied consensus algoriths for achieving cooperative behavior in a distributed anner see [28], [12], [6], [21], [7], and [22, 23]). These works assue that the agent values can be processed arbitrarily and are unconstrained. Another recent approach for distributed cooperative control probles involve using gae-theoretic odels. In this approach, the agents are endowed with local utility functions that lead to a gae for with a Nash equilibriu which is the sae as or close to a global optiu. Various learning algoriths can then be used as distributed control schees that will reach the equilibriu. In a recent paper, Marden et al. [14] used this approach for the consensus proble where agents have constraints on their values. Our projected consensus algorith provides an alternative approach for this proble. Most closely related to our work are the recent papers [18, 17], which proposed distributed subgradient ethods for solving unconstrained ulti-agent optiization probles. These ethods use consensus algoriths as a echanis for distributing coputations aong the agents. The presence of different local constraints significantly changes the operation and the analysis of the algoriths, which is our ain focus in this paper. Our work is also related to increental subgradient algoriths ipleented over a network, where agents sequentially update an iterate sequence in a cyclic or a rando order [4, 15, 24, 13]. In an increental algorith, there is a single iterate sequence and only one agent updates the iterate at a given tie. Thus, while operating on the basis of local inforation, increental algoriths differ fundaentally fro the algorith studied in this paper where all agents update siultaneously). Furtherore, the work in [4, 15, 24, 13] assues that the constraint set is known by all agents in the syste, which is in a sharp contrast with the algoriths studied in this paper our priary interest 3

4 is in the case where the inforation about the constraint set is distributed across the agents). The paper is organized as follows. In Section 2, we introduce our notation and terinology, and establish soe basic results related to projection on closed convex sets that will be used in the subsequent analysis. In Section 3, we present the constrained consensus proble and the projected consensus algorith. We describe our ulti-agent odel and provide a basic result on the convergence behavior of the transition atrices that govern the evolution of agent estiates generated by the algoriths. We study the convergence of the agent estiates and establish convergence rate results for constant unifor weights. Section 4 introduces the constrained ulti-agent optiization proble and presents the projected subgradient algorith. We provide convergence analysis for the estiates generated by this algorith. Section 5 contains concluding rearks and soe future directions. 2 Notation, Terinology, and Basics A vector is viewed as a colun, unless clearly stated otherwise. We denote by x i or [x] i the i-th coponent of a vector x. When x i 0 for all coponents i of a vector x, we write x 0. We write x to denote the transpose of a vector x. The scalar product of two vectors x and y is denoted by x y. We use x to denote the standard Euclidean nor, x = x x. A vector a R is said to be a stochastic vector when its coponents a j are nonnegative and their su is equal to 1, i.e., a j = 1. A set of vectors {a 1,...,a }, with a i R for all i, is said to be doubly stochastic when each a i is a stochastic vector and ai j = 1 for all j = 1,...,. A square atrix A is said to be doubly stochastic when its rows are stochastic vectors, and its coluns are also stochastic vectors. We write dist x, X) to denote the standard Euclidean distance of a vector x fro a set X, i.e., dist x, X) = inf x x. x X We use P X [ x] to denote the projection of a vector x on a closed convex set X, i.e., P X [ x] = arg in x x. x X In the subsequent developent, the properties of the projection operation on a closed convex set play an iportant role. In particular, we use the projection inequality, i.e., for any vector x, P X [x] x) y P X [x]) 0 for all y X. 1) We also use the standard non-expansiveness property, i.e., P X [x] P X [y] x y for any x and y. 2) In addition, we use the properties given in the following lea. 4

5 Figure 1: Illustration of the relation between the projection error and feasible directions of a convex set at the projection vector. Lea 1 Let X be a nonepty closed convex set in R n. Then, we have for any x R n, a) P X [x] x) x y) P X [x] x 2 for all y X. b) P X [x] y 2 x y 2 P X [x] x 2 for all y X. Proof. a) Let x R n be arbitrary. Then, for any y X, we have P X [x] x) x y) = P X [x] x) x P X [x]) + P X [x] x) P X [x] y). By the projection inequality [cf. Eq. 1)], it follows that P X [x] x) P X [x] y) 0, iplying P X [x] x) x y) P X [x] x 2 for all y X. b) For an arbitrary x R n and for all y X, we have P X [x] y 2 = P X [x] x + x y 2 By using the inequality of part a), we obtain = P X [x] x 2 + x y 2 + 2P X [x] x) x y). P X [x] y 2 x y 2 P X [x] x 2 for all y X. Part b) of the preceding Lea establishes a relation between the projection error vector and the feasible directions of the convex set X at the projection vector, as illustrated in Figure 2. We next consider nonepty closed convex sets X i R n, for i = 1,...,, and an averaged-vector ˆx obtained by taking an average of vectors x i X i, i.e., ˆx = 1 x i for soe x i X i. We provide an error bound that relates the distance of the averagedvector ˆx fro the intersection set X = X i to the distance of ˆx fro the individual 5

6 sets X i. This relation, which is also of independent interest, will play a key role in our analysis of the convergence of projection errors associated with various distributed algoriths introduced in this paper. We establish the relation under an interior point assuption on the intersection set X = X i stated in the following: Assuption 1 Given sets X i R n, i = 1,...,, let X = X i denote their intersection. There is a vector x intx), i.e., there exists a scalar δ > 0 such that {z z x δ} X. We provide an error bound relation in the following lea. Lea 2 Let X i R n, i = 1,...,, be nonepty closed convex sets that satisfy Assuption 1. Let x i X i, i = 1,...,, be arbitrary vectors and define their average as ˆx = 1 xi. Consider the vector s R n defined by where s = ǫ = ǫ ǫ + δ x + δ ǫ + δ ˆx, distˆx, X j ), and δ is the scalar given in Assuption 1. a) The vector s belongs to the intersection set X = X i. b) We have the following relation ˆx s 1 ) ) x j x distˆx, X j ). δ As a particular consequence, we have distˆx, X) 1 δ ) x j x ) distˆx, X j ). Proof. a) We first show that the vector s belongs to the intersection X = X i. To see this, let i {1,..., } be arbitrary and note that we can write s as s = ǫ ǫ + δ x + δ ) ) ˆx P Xi [ˆx] + δ ǫ ǫ + δ P X i [ˆx]. 6

7 By the definition of ǫ, it follows that ˆx P Xi [ˆx] ǫ, iplying ) by the interior point assuption cf. Assuption 1) that the vector x + δ ˆx P ǫ Xi [ˆx] belongs to the set X, and therefore to the set X i. Since the vector s is the convex cobination of two vectors in the set X i, it follows by the convexity of X i that s X i. The preceding arguent is valid for an arbitrary i, thus iplying that s X. b) Using the definition of the vector s and the vector ˆx, we have ˆx s = ǫ 1 x j x ǫ + δ ǫ x j x. δ Substituting the definition of ǫ yields the desired relation. 3 Constrained Consensus In this section, we describe the constrained consensus proble. In particular, we introduce our ulti-agent odel and the projected consensus algorith that is locally executed by each agent. We provide soe insights about the algorith and we discuss its connection to the alternating projections ethod. We also introduce the assuptions on the ulti-agent odel and present key eleentary results that we use in our subsequent analysis of the projected consensus algorith. In particular, we define the transition atrices governing the linear dynaics of the agent estiate evolution and give a basic convergence result for these atrices. The odel assuptions and the transition atrix convergence properties will also be used for studying the constrained optiization proble and the projected subgradient algorith that we introduce in Section Multi-Agent Model and Algorith We consider a set of agents denoted by V = {1,..., }. We assue a slotted-tie syste, and we denote by x i k) the estiate generated and stored by agent i at tie slot k. The agent estiate x i k) is a vector in R n that is constrained to lie in a nonepty closed convex set X i R n known only to agent i. The agents objective is to cooperatively reach a consensus on a coon vector through a sequence of local estiate updates subject to the local constraint set) and local inforation exchanges with neighboring agents only). We study a odel where the agents exchange and update their estiates as follows: To generate the estiate at tie k+1, agent i fors a convex cobination of its estiate x i k) with the estiates received fro other agents at tie k, and takes the projection of this vector on its constraint set X i. More specifically, agent i at tie k + 1 generates its new estiate according to the following relation: [ ] x i k + 1) = P Xi a i jk)x j k), 3) 7

8 where a i = a i 1,...,a i ) is a vector of nonnegative weights. The relation in Eq. 3) defines the projected consensus algorith. The ethod can be interpreted as a ulti-agent algorith for finding a point in coon to the given closed convex sets X 1,...,X. Note that the proble of finding a coon point can be forulated as an unconstrained convex optiization proble of the following for: 1 iniize 2 x P X i [x] 2 subject to x R n. 4) In view of this optiization proble, the ethod can be interpreted as a distributed gradient algorith where each agent is assigned an objective function f i x) = 1 2 x P X i [x] 2. At each tie k + 1, an agent incorporates new inforation x j k) received fro soe of the other agents and generates a weighted su ai jk)x j k). Then, the agent updates its estiate by taking a step with stepsize equal to 1) along the negative gradient of its own objective function f i = 1 x P 2 X i 2 at x = ai j k)xj k). In particular, since the gradient of f i is f i x) = x P Xi [x] see Theore in Facchinei and Pang [10]), the update rule in Eq. 3) is equivalent to the following gradient descent ethod for iniizing f i : x i k + 1) = [ ]) a i j k)xj k) a i j k)xj k) P Xi a i j k)xj k). This view of the update rule otivates our line of analysis of the projected consensus ethod. In particular, otivated by the objective function of proble 4), we use xi k) x 2 with x X i as a Lyapunov function easuring the progress of the algorith see Section 3.6) Relation to Alternating Projections Method The ethod of Eq. 3) is related to the classical alternating or cyclic projection ethod. Given a finite collection of closed convex sets {X i } i I with a nonepty intersection i.e., i I X i ), the alternating projection ethod finds a vector in the intersection i I X i. In other words, the algorith solves the unconstrained proble 4). Alternating projection ethods generate a sequence of vectors by projecting iteratively on the sets either cyclically or with soe given order), see Figure 2a). The convergence behavior of these ethods has been established by Von Neuann [20] and Aronszajn [1] for the case when the sets X i are affine; and by Gubin et al. [11] when the sets X i are closed and convex. Gubin et al. [11] also have provided convergence rate results for a particular for of alternating projection ethod. Siilar rate results under different assuptions have also been provided by Deutsch [8], and Deutsch and Hundal [9]. The constrained consensus algorith [cf. Eq. 3)] generates a sequence of iterates for each agent as follows: at iteration k, each agent i first fors a linear cobination 1 We focus throughout the paper on the case when the intersection set X i is nonepty. If the intersection set is epty, it follows fro the definition of the algorith that the agent estiates will not reach a consensus. In this case, the estiate sequences {x i k)} ay exhibit oscillatory behavior or ay all be unbounded. 8

9 Figure 2: Illustration of the connection between the alternating/cyclic projection ethod and the constrained consensus algorith for two closed convex sets X 1 and X 2. In plot a), the alternating projection algorith generates a sequence {xk)} by iteratively projecting onto sets X 1 and X 2, i.e., xk + 1) = P X1 [xk)], xk + 2) = P X2 [xk + 1)]. In plot b), the projected consensus algorith generates sequences {x i k)} for agents i = 1,2 by first cobining the iterates with different weights and then projecting on respective sets X i, i.e., w i k) = ai j k)xj k) and x i k + 1) = P Xi [w i k)] for i = 1,2. of the other agent values x j k) using its own weight vector a i k) and then projects this cobination on its constraint set X i. Therefore, the projected consensus algorith can be viewed as a version of the alternating projection algorith, where the iterates are cobined with the weights varying over tie and across agents, and then projected on the individual constraint sets, see Figure 2b). We conclude this section by noting that the alternate projection ethod has uch ore structured weights than the weights we consider in this paper. As seen fro the assuptions on the agent weights in the following section, the analysis of our projected consensus algorith and the projected subgradient algorith introduced in Section 4) is coplicated by the general tie variability of the weights a i j k). 3.3 Assuptions Following Tsitsiklis [26] see also Blondel et al. [5]), we adopt the following assuptions on the weight vectors a i k), i {1,..., } and on inforation exchange. Assuption 2 Weights Rule) There exists a scalar η with 0 < η < 1 such that for all i {1,..., }, a) a i ik) η for all k 0. b) If a i j k) > 0, then ai j k) η. Assuption 3 Doubly Stochasticity) The vectors a i k) = a i 1 k),...,ai k)) satisfy: 9

10 a) a i k) 0 and ai jk) = 1 for all i and k, i.e., the vectors a i k) are stochastic. b) ai j k) = 1 for all j and k. Inforally speaking, Assuption 2 says that every agent assigns a substantial weight to the inforation received fro its neighbors. This guarantees that the inforation fro each agent influences the inforation of every other agent persistently in tie. In other words, this assuption guarantees that the agent inforation is ixing at a nondiinishing rate in tie. Without this assuption, inforation fro soe of the agents ay becoe less influential in tie, and in the liit, resulting in loss of inforation fro these agents. Assuption 3a) establishes that each agent takes a convex cobination of its estiate and the estiates of its neighbors. Assuption 3b), together with Assuption 2, ensures that the estiate of every agent is influenced by the estiates of every other agent with the sae frequency in the liit, i.e., all agents are equally influential in the long run. We now ipose soe rules on the agent inforation exchange. At each update tie t k, the inforation exchange aong the agents ay be represented by a directed graph V, E k ) with the set E k of directed edges given by E k = {j, i) a i j k) > 0}. Note that, by Assuption 2a), we have i, i) E k for each agent i and all k. Also, we have j, i) E k if and only if agent i receives the inforation x j fro agent j in the tie interval t k, t k+1 ). We next forally state the connectivity assuption on the ulti-agent syste. This assuption ensures that the inforation of any agent i influences the inforation state of any other agent infinitely often in tie. Assuption 4 Connectivity) The graph V, E ) is strongly connected, where E is the set of edges j, i) representing agent pairs counicating directly infinitely any ties, i.e., E = {j, i) j, i) E k for infinitely any indices k}. We also adopt an additional assuption that the intercounication intervals are bounded for those agents that counicate directly. In particular, this is stated in the following. Assuption 5 Bounded Intercounication Interval) There exists an integer B 1 such that for every j, i) E, agent j sends its inforation to a neighboring agent i at least once every B consecutive tie slots, i.e., at tie t k or at tie t k+1 or... or at latest) at tie t k+b 1 for any k 0. In other words, the preceding assuption guarantees that every pair of agents that counicate directly infinitely any ties exchange inforation at least once every B tie slots. 2 2 It is possible to adopt weaker connectivity assuptions for the ulti-agent odel as those used in the recent work [16]. 10

11 3.4 Transition Matrices We introduce atrices As), whose i-th colun is the weight vector a i s), and the atrices where Φk, s) = As)As + 1) Ak 1)Ak) for all s and k with k s, Φk, k) = Ak) for all k. We use these atrices to describe the evolution of the agent estiates associated with the algoriths introduced in Sections 3 and 4. The convergence properties of these atrices as k have been extensively studied and well-established see [26], [12], [29]). Under the assuptions of Section 3.3, the atrices Φk, s) converge as k to a unifor steady state distribution for each s at a geoetric rate, i.e., li k Φk, s) = 1 ee for all s. The fact that transition atrices converge at a geoetric rate plays a crucial role in our analysis of the algoriths. Recent work has established explicit convergence rate results for the transition atrices [18, 17]. These results are given in the following proposition without a proof. Proposition 1 Let Assuptions 2, 3, 4 and 5 hold. Then, we have the following: a) The entries [Φk, s)] i j of the transition atrices converge to 1 as k with a geoetric rate uniforly with respect to i and j, i.e., for all i, j {1,..., }, [Φk, s)]i j η B 0 1 η B 0 1 η B 0 ) k s B 0 for all s and k with k s. b) In the absence of Assuption 3b) [i.e., the weights a i k) are stochastic but not doubly stochastic], the coluns [Φk, s)] i of the transition atrices converge to a stochastic vector φs) as k with a geoetric rate uniforly with respect to i and j, i.e., for all i, j {1,...,}, [Φk, s)] i j φ j s) η B 0 1 η B 0 1 η B 0 ) k s B 0 for all s and k with k s. Here, η is the lower bound of Assuption 2, B 0 = 1)B, is the nuber of agents, and B is the intercounication interval bound of Assuption Convergence In this section, we study the convergence behavior of the agent estiates {x i k)} generated by the projected consensus algorith 3) under Assuptions 2 5. We write the update rule in Eq. 3) as x i k + 1) = a i jk)x j k) + e i k), 5) 11

12 where e i k) represents the error due to projection given by [ ] e i k) = P Xi a i j k)xj k) a i j k)xj k). 6) As indicated by the preceding two relations, the evolution dynaics of the estiates x i k) for each agent is decoposed into a su of a linear tie-varying) ter ai j k)xj k) and a nonlinear ter e i k). The linear ter captures the effects of ixing the agent estiates, while the nonlinear ter captures the nonlinear effects of the projection operation. This decoposition plays a crucial role in our analysis. As we will shortly see [cf. Lea 3d)], under the doubly stochasticity assuption on the weights, the nonlinear ters e i k) are diinishing in tie for each i, and therefore, the evolution of agent estiates is alost linear. Thus, the nonlinear ter can be viewed as a non-persistent disturbance in the linear evolution of the estiates. For notational convenience, let w i k) denote w i k) = a i j k)xj k). 7) In this notation, the iterate x i k + 1) and the projection error e i k) are given by x i k + 1) = P Xi [w i k)], 8) e i k) = x i k + 1) w i k). 9) In the following lea, we show soe relations for the sus xi k) x 2 and wi k) x 2, and xi k) x and wi k) x for an arbitrary vector x in the intersection of the agent constraint sets. Also, we prove that the errors e i k) converge to zero as k for all i. The projection properties given in Lea 1 and the doubly stochasticity of the weights play crucial roles in establishing these relations. Lea 3 Let the intersection set X = X i be nonepty, and let Doubly Stochasticity assuption hold cf. Assuption 3). Let x i k), w i k), and e i k) be defined by Eqs. 7) 9). Then, we have the following. a) For all x X and all k, we have i) x i k + 1) x 2 w i k) x 2 e i k) 2 for all i, ii) wi k) x 2 xi k) x 2, iii) wi k) x xi k) x. { } { } b) For all x X, the sequences wi k) x 2 and xi k) x 2 are onotonically nonincreasing with k. { } { } c) For all x X, the sequences wi k) x and xi k) x are onotonically nonincreasing with k. 12

13 d) The errors e i k) converge to zero as k, i.e., li k ei k) = 0 for all i. Proof. a) For any x X and i, we consider the ter x i k + 1) x 2. Since X X i for all i, it follows that x X i for all i. Since we also have x i k + 1) = P Xi [w i k)], we have fro Lea 1b) that x i k + 1) x 2 w i k) x 2 x i k + 1) w i k) 2 for all x X and k 0, which yields the relation in part a)i) in view of relation 9). By the definition of w i k) in Eq. 7) and the stochasticity of the weight vector a i k) [cf. Assuption 3a)], we have for every agent i and any x X, w i k) x = a i jk) x j k) x ) for all k 0. 10) Thus, for any x X, and all i and k, w i k) x 2 = a i jk) x j k) x ) 2 a i jk) x j k) x 2, where the inequality holds since the vector ai j k)xj k) x) is a convex cobination of the vectors x j k) x and the squared nor 2 is a convex function. By suing the preceding relations over i = 1,...,, we obtain w i k) x 2 a i jk) x j k) x 2 = x ajk)) i j k) x 2. Using the doubly stochasticity of the weight vectors a i k), i.e., ai j k) = 1 for all j and k [cf. Assuption 3b)], we obtain the relation in part a)ii), w i k) x 2 x i k) x 2 for all x X and k 0. Siilarly, fro relation 10) and the doubly stochasticity of the weights, we obtain for all x X and all k, w i k) x a i j k) xj k) x = x j k) x, thus showing the relation in part a)iii). { } b) For any x X, the nonincreasing properties of the sequences wi k) x 2 { } and xi k) x 2 follow by cobining the relations in parts a)i) ii). 13

14 c) Since x i k+1) = P Xi w i k)) for all i and k 0, using the nonexpansiveness property of the projection operation [cf. Eq. 2)], we have x i k + 1) x w i k) x for all x X i, all i and k. Suing the preceding relations over all i {1,..., } yields for all k, x i k + 1) x w i k) x for all x X. 11) The nonincreasing property of the sequences { wi k) x } and { xi k) x } follows fro the preceding relation and the relation in part a)iii). d) By suing the relations in part a)i) over i = 1,...,, we obtain for any x X, x i k + 1) x 2 w i k) x 2 e i k) 2 for all k 0. Cobined with the inequality wj k) x 2 xj k) x 2 of part a)ii), we further obtain e i k) 2 x i k) x 2 x i k + 1) x 2 for all k 0. Suing these relations over k = 0,..., s for any s > 0 yields s k=0 e i s) 2 x i 0) x 2 x i s + 1) x 2 x i 0) x 2. By letting s, we obtain k=0 e i k) 2 x i 0) x 2, iplying li k e i k) = 0 for all i. We next consider the evolution of the estiates x i k + 1) generated by ethod 3) over a period of tie. In particular, we relate the estiates x i k + 1) to the estiates x i s) generated earlier in tie s with s < k + 1 by exploiting the decoposition of the estiate evolution in Eqs. 5) 6). In this, we use the transition atrices Φk, s) fro tie s to tie k see Section 3.4). As we will shortly see, the linear part of the dynaics is given in ters of the transition atrices, while the nonlinear part involves cobinations of the transition atrices and the error ters fro tie s to tie k. Recall that the transition atrices are defined as follows: Φk, s) = As)As + 1) Ak 1)Ak) for all s and k with k s, 14

15 where Φk, k) = Ak) for all k, and each As) is a atrix whose i-th colun is the vector a i s). Using these transition atrices and the decoposition of the estiate evolution of Eqs. 5) 6), the relation between x i k + 1) and the estiates x 1 s),...,x s) at tie s k is given by k ) x i k + 1) = [Φk, s)] i j x j s) + [Φk, r)] i j e j r 1) + e i k). 12) r=s+1 Here we can view e j k) as an external perturbation input to the syste. We use this relation to study the steady-state behavior of a related process. In particular, we define an auxiliary sequence {yk)}, where yk) is given by yk) = 1 w i k) for all k. 13) Since w i k) = ai j k)xj k), under the doubly stochasticity of the weights, it follows that yk) = 1 x j k) for all k. 14) Furtherore, fro the relations in 12) using the doubly stochasticity of the weights, we have yk) = 1 x j s) + 1 k ) e j r 1). 15) r=s+1 We now show that the liiting behavior of the agent estiates x i k) is the sae as the liiting behavior of yk) as k. We establish this result using the assuptions on the ulti-agent odel of Section 3.3. Lea 4 Let the intersection set X = X i be nonepty. Also, let Assuptions 2, 3, 4, and 5 hold. We then have li k xi k) yk) = 0, li w i k) yk) = 0 for all i. k Proof. By Lea 3d), we have e i k) 0 as k for all i. Therefore, for any ǫ > 0, we can choose soe integer s such that e i k) ǫ for all k s and for all i. Using the relations in Eqs. 12) and 15), we obtain for all i and k s + 1, x i k) yk) = + k 1 r=s+1 [Φk 1, s)] i j 1 ) x j s) [Φk 1, r)] i j 1 ) e j r 1) + e i k 1) 1 15 ) e j k 1)

16 + [Φk 1, s)] i j 1 x j s) k 1 r=s+1 [Φk 1, r)] i j 1 e j r 1) + e i k 1) + 1 Using the estiates for [Φk 1, s)] i j 1 of Proposition 1a), we have x i k) yk) η B η B 0 k 1 r=s+1 1 η B 0 ) k 1 s B η B 0 1 η B 0 + e i k 1) η B 0 ) k 1 r B 0 x j s) e j k 1). e j r 1) e j k 1). 16) Since e i k) ǫ for all k s and for all i, fro the preceding inequality we obtain x i k) yk) η B 0 1 η B 0 +2ǫ 1 + η B 0 1 η B 0 1 η B 0 ) k 1 s B 0 Thus, by taking the liit superior as k, we see that li sup x i k) yk) 2ǫ 1 + η B 0 k 1 η B 0 x j s) η B 1 + 2ǫ. 0 ) B η B 1 + 2ǫ, 0 ) B 0 which by the arbitrary choice of ǫ, iplies li k x i k) yk) = 0 for all i. Consider now wi k) yk). By using w i k) = ai j k)xj k) [cf. Eq. 7)] and the stochasticity of the vector a i k), we have w i k) yk) a i jk) x j k) yk). By exchanging the order of the suations over i and j, and using the doubly stochasticity of a i k), we further obtain ) w i k) yk) a i jk) x j k) yk) = x j k) yk). 17) Since li k x j k) yk) = 0 for all j, we have li k w i k) yk) = 0, 16

17 iplying li k w i k) yk) = 0 for all i. We next show that the agents reach a consensus asyptotically, i.e., the agent estiates x i k) converge to the sae point as k goes to infinity. Proposition 2 Consensus) Let the set X = X i be nonepty. Also, let Assuptions 2, 3, 4, and 5 hold. For all i, let the sequence {x i k)} be generated by the projected consensus algorith 3). We then have for soe x X, li k xi k) x = 0 li w i k) x = 0, for all i. k Proof. The proof idea is to consider the sequence {yk)}, defined in Eq. 15), and show that it has a liit point in the set X. By using this and Lea 4, we establish the convergence of each w i k) and x i k) to x. To show that {yk)} has a liit point in the set X, we first consider the sequence distyk), X j ). Since x j k) X j for all j and k 0, we have distyk), X j ) yk) x j k). Taking the liit as k in the preceding relation and using Lea 4, we conclude li k distyk), X j ) = 0. 18) For a given x X, using Lea 3c), we have x i k) x x i 0) x for all k 0. This iplies that the sequence { xi k) x }, and therefore each of the sequences {x i k)} are bounded. Since for all i yk) x i k) yk) + x i k) for all k 0, using Lea 4, it follows that the sequence {yk)} is bounded. In view of Eq. 18), this iplies that the sequence {yk)} has a liit point x that belongs to the set X = X j. Furtherore, because li k w i k) yk) = 0 for all i, we conclude { that x is also a } liit point of the sequence {w i k)} for all i. Since the su sequence wi k) x 17

18 is nonincreasing by Lea 3c) and since each {w i k)} is converging to x along a subsequence, it follows that li k w i k) x = 0, iplying li k w i k) x = 0 for all i. Using this, together with the relations li k w i k) yk) = 0 and li k x i k) yk) = 0 for all i cf. Lea 4), we conclude li k xi k) x = 0 for all i. 3.6 Convergence Rate In this section, we establish a convergence rate result for the iterates x i k) generated by the projected consensus algorith 3) for the case when the weights are tie-invariant and equal, i.e., a i k) = 1/,..., 1/) for all i and k. In our ulti-agent odel, this case corresponds to a fixed and coplete connectivity graph, where each agent is connected to every other agent. We provide our rate estiate under an interior point assuption on the sets X i, stated in Assuption 1. We first establish a bound on the distance fro the vectors of a convergent sequence to the liit point of the sequence. This relation holds for constant unifor weights, and it is otivated by a siilar estiate used in the analysis of alternating projections ethods in Gubin et al. [11] see the proof of Lea 6 there). Lea 5 Let Y be a nonepty closed convex set in R n. Let {uk)} R n be a sequence converging to soe ỹ Y, and such that uk + 1) y uk) y for all y Y and all k. We then have uk) ỹ 2 distuk), Y ) for all k 0. Proof. Let Bx, α) denote the closed ball centered at a vector x with radius α, i.e., Bx, α) = {z z x α}. For each l, consider the sets S l = l k=0 ) B P Y [uk)], distuk), Y ). The sets S l are convex, copact, and nested, i.e., S l+1 S l for all l. The nonincreasing property of the sequence {uk)} iplies that uk +s) P Y [uk)] uk) P Y [uk)] for all k, s 0; hence, the sets S l are also nonepty. Consequently, their intersection l=0 S l is nonepty and every point y l=0 S l is a liit point of the sequence {uk)}. By assuption, the sequence {uk)} converges to ỹ Y, and therefore, l=0 S l = {ỹ}. Then, in view of the definition of the sets S l, we obtain for all k, uk) ỹ uk) P Y [uk)] + P Y [uk)] ỹ 2 distuk), Y ). 18

19 We now establish a convergence rate result for constant unifor weights. In particular, we show that the projected consensus algorith converges with a geoetric rate under the Interior Point assuption. Proposition 3 Let Assuptions 1, 2, 3, 4, and 5 hold. Let the weight vectors a i k) in algorith 3) be given by a i k) = 1/,...,1/) for all i and k. For all i, let the sequence {x i k)} be generated by the algorith 3). We then have x i k) x ) k x i 0) x 2 for all k 0, 4R 2 where x X is the liit of the sequence {x i k)}, and R = 1 δ xi 0) x with x and δ given in the Interior Point assuption. Proof. Since the weight vectors a i k) are given by a i k) = 1/,...,1/), it follows that w i k) = wk) = 1 x j k) for all i, [see the definition of w i k) in Eq. 7)]. For all k 0, using Lea 2b) with the identification x i = x i k) for each i = 1,...,, and ˆx = wk), we obtain distwk), X) 1 δ ) x j k) x ) distwk), X j ), where the vector x and the scalar δ are given in Assuption 1. Since x X, the sequence { xi k) x } is nonincreasing by Lea 3c). Therefore, we have xi k + 1) x xi 0) x for all k. Defining the constant R = 1 δ xi 0) x and substituting in the preceding relation, we obtain distwk), X) R = R ) distwk), X j ) wk) x j k + 1), 19) where the second relation follows in view of the definition of x j k + 1) [cf. Eq. 8)]. By Proposition 2, we have wk) x for soe x X as k. Furtherore, by Lea 3c) and the relation w i k) = wk) for all i and k, we have that the sequence { wk) x } is nonincreasing for any x X. Therefore, the sequence {wk)} satisfies the conditions of Lea 5, and by using this lea we obtain wk) x 2 distwk), X) for all k. 19

20 Cobining this relation with Eq. 19), we further obtain wk) x 2R wk) x i k + 1). Taking the square of both sides and using the convexity of the square function ) 2, we have wk) x 2 4R2 wk) x i k + 1) 2. 20) Since x i k + 1) = P Xi [wk)] for all i and k, using Lea 3a) with the substitutions x = x X and e i k) = x i k + 1) wk) for all i, we see that wk) x i k + 1) 2 wk) x 2 x i k + 1) x 2 for all k. Using this relation in Eq. 20), we obtain wk) x 2 4R2 wk) x 2 ) x i k + 1) x 2. Rearranging the ters and using the relation wk) x 2 xi k) x 2 [cf. Lea 3a) with wk) = w i k) and x = x], we obtain x i k + 1) x 2 which yields the desired result. 4 Constrained Optiization 1 1 ) x i k) x 2, 4R 2 We next consider the proble of optiizing the su of convex objective functions corresponding to agents connected over a tie-varying topology. The goal of the agents is to cooperatively solve the constrained optiization proble iniize subject to f i x) 21) x X i, 22) where each f i : R n R is a convex function, representing the local objective function of agent i, and each X i R n is a closed convex set, representing the local constraint set of agent i. We assue that the local objective function f i and the local constraint set X i are known to agent i only. 20

21 To keep our discussion general, we do not assue differentiability of any of the functions f i. Since each f i is convex over the entire R n, the function is differentiable alost everywhere see [2] or [25]). At the points where the function fails to be differentiable, a subgradient exists and can be used in the role of a gradient. In particular, for a given convex function F : R n R and a point x, a subgradient of the function F at x is a vector s F x) R n such that F x) + s F x) x x) Fx) for all x. 23) The set of all subgradients of F at a given point x is denoted by F x), and it is referred to as the subdifferential set of F at x. 4.1 Distributed Projected Subgradient Algorith We introduce a distributed subgradient ethod for solving proble 21) using the assuptions iposed on the inforation exchange aong the agents in Section 3.3. The ain idea of the algorith is the use of consensus as a echanis for distributing the coputations aong the agents. In particular, each agent i starts with an initial estiate x i 0) X i and updates its estiate. An agent i updates its estiate by cobining the estiates received fro its neighbors, by taking a subgradient step to iniize its objective function f i, and by projecting on its constraint set X i. Forally, each agent i updates according to the following rule: v i k) = a i j k)xj k) 24) x i k + 1) = P Xi [ v i k) α k d i k) ], 25) where the scalars a i j k) are nonnegative weights and the scalar α k > 0 is a stepsize. The vector d i k) is a subgradient of the agent i local objective function f i x) at x = v i k). We refer to the ethod 24)-25) as the projected subgradient algorith. To analyze this algorith, we find it convenient to re-write the relation for x i k+1) in an equivalent for. This for helps us identify the linear effects due to agents ixing the estiates [which will be driven by the transition atrices Φk, s)], and the nonlinear effects due to taking subgradient steps and projecting. In particular, we re-write the relations 24) 25) as follows: v i k) = a i j k)xj k) x i k + 1) = v i k) α k d i k) + φ i k) 26) φ i k) = P Xi [ v i k) α k d i k) ] v i k) α k d i k) ). 27) The evolution of the iterates is coplicated due to the nonlinear effects of the projection operation, and even ore coplicated due to the projections on different sets. In our subsequent analysis, we study two special cases: 1) when the constraint sets are the sae [i.e., X i = X for all i], but the agent connectivity is tie-varying; and 2) when the 21

22 constraint sets X i are different, but the agent counication graph is fully connected. In the analysis of both cases, we use a basic relation for the iterates x i k) generated by the ethod in 27). The relation is established in the following lea. Lea 6 Let Assuptions 2 and 3 and hold. Let {x i k)} be the iterates generated by the algorith 24)-25). We have for any z X = X i and all k 0, x i k + 1) z 2 x i k) z 2 + αk 2 φ i k) 2. d i k) 2 2α k fi v i k)) f i z) ) Proof. Since x i k +1) = P Xi [v i k) α k d i k)], it follows fro Lea 1b) and fro the definition of the projection error φ i k) in 27) that for any z X, x i k + 1) z 2 v i k) α k d i k) z 2 φ i k) 2. By expanding the ter v i k) α k d i k) z 2, we obtain v i k) α k d i k) z 2 = v i k) z 2 + α 2 k d ik) 2 2α k d i k) v i k) z). Since d i k) is a subgradient of f i x) at x = v i k), we have d i k) v i k) z) f i v i k)) f i z). By cobining the preceding relations, we obtain x i k + 1) z 2 v i k) z 2 + α 2 k d ik) 2 2α k fi v i k)) f i z) ) φ i k) 2. Since v i k) = ai j k)xj k), using the convexity of the nor square function and the stochasticity of the weights a i jk), j = 1,...,, it follows that v i k) z 2 a i jk) x j k) z 2. Cobining the preceding two relations, we obtain x i k + 1) z 2 a i j k) xj k) z 2 + αk 2 d ik) 2 2α k fi v i k)) f i z) ) φ i k) 2. By suing the preceding relation over i = 1,...,, and using the doubly stochasticity of the weights, i.e., ) a i j k) xj k) z 2 = a i j k) x j k) z 2 = x j k) z 2, we obtain the desired relation. 22

23 4.1.1 Convergence when X i = X for all i We first study the case when all constraint sets are the sae, i.e., X i = X for all i. The next assuption forally states the conditions we adopt in the convergence analysis. Assuption 6 a) The constraint sets X i are the sae, i.e, X i = X for a closed convex set X. b) The subgradient sets of each f i are bounded over the set X, i.e., there is a scalar L > 0 such that for all i, d L for all d f i x) and all x X. The subgradient boundedness assuption in part b) holds for exaple when the set X is copact see [2]). In proving our convergence results, we use a property of the infinite su of products of the coponents of two sequences. In particular, for a scalar β 0, 1) and a scalar sequence {γ k }, we consider the convolution sequence k l=0 βk l γ l = β k γ 0 + β k 1 γ βγ k 1 + γ k. We have the following result. Lea 7 Let 0 < β < 1 and let {γ k } be a positive scalar sequence. Assue that li k γ k = 0. Then k β k l γ l = 0. In addition, if k γ k <, then li k k l=0 k β k l γ l <. l=0 Proof. Let ǫ > 0 be arbitrary. Since α k 0, there is an index K such that α k ǫ for all k K. For all k K + 1, we have k β k l γ l = l=0 K β k l γ l + l=0 k l=k+1 β k l γ l ax 0 t K γ t K β k l + ǫ l=0 k l=k+1 β k l. Since k l=k+1 βk l 1 1 β and K l=0 it follows that for all k K + 1, β k l = β k + + β k K = β k K β K ) βk K 1 β, k l=0 β k l γ l ax γ β k K t 0 t K 1 β + ǫ 1 β. 23

24 Therefore, li sup k k β k l γ l l=0 ǫ 1 β. Since ǫ is arbitrary, we conclude that li sup k k l=0 βk l γ l = 0, iplying li k k β k l γ l = 0. Suppose now k γ k <. Then, for any integer M 1, we have M k ) M M l M β k l γ l = γ l β t 1 γ l 1 β, iplying that k=0 l=0 l=0 l=0 k ) β k l γ l k=0 l=0 t=0 1 1 β l=0 γ l <. Our goal is to show that the agent disagreeents x i k) x j k) converge to zero. To easure the agent disagreeents x i k) x j k) in tie, we consider their average 1 xj k), and consider the agent disagreeent with respect to this average. In particular, we define yk) = 1 x j k) for all k. In view of Eq. 26), we have yk + 1) = 1 v i k) α k l=0 d i k) + 1 φ i k). When the weights are doubly stochastic, since v i k) = ai jk)x j k), it follows that yk + 1) = yk) α k d i k) + 1 φ i k). 28) Under Assuption 6, the assuptions on the agent weights and connectivity stated in Section 3.3, and soe conditions on{ the stepsize α k }, the next lea studies the convergence properties of the sequences x i k) yk) for all i. Lea 8 Let Assuptions 2, 3, 4, 5, and 6 hold. Let {x i k)} be the iterates generated by the algorith 24)-25) and consider the auxiliary sequence {yk)} defined in 28). a) If the stepsize satisfies li k α k = 0, then li k xi k) yk) = 0 for all i. 24

25 b) If the stepsize satisfies k α2 k <, then α k x i k) yk) < for all i. k=1 Proof. a) Using the relations in 27) and the transition atrices Φk, s), we can write for all i, and for all k and s with k > s, x i k + 1) = [Φk, s)] i jx j s) k 1 + r=s k 1 r=s [Φk, r + 1)] i jα r d j r) α k d i k) [Φk, r + 1)] i jφ j r) + φ i k). Siilarly, using the transition atrices and relation 28), we can write for yk + 1) and for all k and s with k > s, yk + 1) = ys) 1 k 1 r=s α r d j r) α k d i k) + 1 k 1 r=s φ j r) + 1 φ j k). Therefore, since ys) = 1 xj s), we have for s = 0, x i k) yk) [Φk 1, 0)]i j 1 xj 0) k 2 + [Φk 1, r + 1)]i j 1 α r d j r) r=0 +α k 1 d i k 1) + α k 1 d j k 1) k 2 + [Φk 1, r + 1)]i j 1 φj r) r=0 + φ i k 1) + 1 φ j k 1). Using the estiate for [Φk, s)] i j 1 of Proposition 1, we have for all k s, [Φk, s)]i j 1 Cβk s for all i, j, with C = 2 1+η B 0 and β = ) 1 η B 1 1 η B 0 B 0 0. Hence, using this relation and the subgradient boundedness, we obtain for all i and k 2, k 2 x i k) yk) Cβ k 1 x j 0) + CL β k r α r + 2α k 1 L 25 r=0

26 k 2 +C β k r φ j r) + φ i k 1) + 1 r=0 φ j k 1). 29) We next show that the errors φ i k) satisfy φ i k) α k L for all i and k. In view of the relations in 27), since x j k) X j = X for all k and j, and the vector a i k) is stochastic for all i and k, it follows that v i k) X for all i and k. Furtherore, by the projection property in Lea 1b), we have for all i and k, x i k + 1) v i k) 2 v i k) α k d i k) v i k) 2 x i k + 1) v i k) α k d i k)) 2 αk 2 L2 φ i k) 2, where in the last inequality we use d i k) L see Assuption 6). It follows that φ i k) α k L for all i and k. By using this in relation 29), we obtain k 2 x i k) yk) Cβ k 1 x j 0) + 2CL β k r α r + 4α k 1 L. 30) By taking the liit superior in relation 30) and using the facts β k 0 recall 0 < β < 1) and α k 0, we obtain for all i, li sup x i k) yk) 2CL li sup k k r=0 k 2 β k r α r Finally, since 0 < β < 1 and li k α k = 0, by Lea 7 we have k 2 li k r=0 β k r α r = 0. In view of the preceding two relations, it follows that li k x i k) yk) = 0 for all i. b) By ultiplying the relation in 30) with α k, we obtain k 2 α k x i k) yk) Cα k β k 1 x j 0) + 2CL β k r α k α r + 4α k α k 1 L. r=0 r=0 By using α k β k 1 α 2 k + β2k 1) and 2α k α r α 2 k + α2 r for any k and r, we have k 2 α k x i k) yk) Cβ 2k 1) x j 0) +CAαk 2 +CL β k r αr 2 +2Lα2 k +α2 k 1 ), where A = xj 0) + L. Therefore, by suing and grouping soe of the 1 β) ters, we obtain ) α k x i k) yk) C β 2k 1) x j 0) k=1 k=1 26 r=0

27 + CAα 2 k + 2Lαk 2 + αk 1) ) k CL β k r αr. 2 k=1 k=1 r=0 In the preceding relation, the first ter is suable since 0 < β < 1. The second ter is suable since k α2 k <. The third ter is also suable by Lea 7. Hence, k=1 α k x i k) yk) <. Using Leas 6 and 8, we next show that the iterates x i k) converge to an optial solution when we use a stepsize converging to zero fast enough. Proposition 4 Let Assuptions 2, 3, 4, 5, and 6 hold. Let {x i k)} be the iterates generated by the algorith 24)-25) with the stepsize satisfying k α k = and k α2 k <. In addition, assue that the optial solution set X is nonepty. Then, there exists an optial point x X such that li k xi k) x = 0 for all i. Proof. Fro Lea 6, we have for z X and all k, x i k + 1) z 2 x j k) z 2 + αk 2 d i k) 2 2α k fi v i k)) f i z) ) φ i k) 2. By dropping the nonpositive ter on the right hand side, and by using the subgradient boundedness, we obtain x i k + 1) z 2 x j k) z 2 + αk 2 L2 2α k fi v i k)) f i yk)) ) 2α k fyk)) fz)). 31) In view of the subgradient boundedness and the stochasticity of the weights, it follows f i v i k)) f i yk)) L v i k) yk) L a i j k) xj k) yk), iplying, by the doubly stochasticity of the weights, that fi v i k)) f i yk)) ) L a i j k) x j k) yk) = L x j k) yk). By using this in relation 31), we see that for any z X, and all i and k, x i k + 1) z 2 x j k) z 2 + αkl α k L x j k) yk) 27