Distributed Online Optimization for Multi-Agent Networks with Coupled Inequality Constraints

Transcription

1 Distributed Online Optimization for Multi-Agent etworks with Coupled Inequality Constraints Xiuxian Li, Xinlei Yi, and Lihua Xie arxiv: v [math.oc] 5 May 08 Abstract his paper investigates the distributed online optimization problem over a multi-agent network subject to local set constraints and coupled inequality constraints, which has a large number of applications in practice, such as wireless sensor networks, power systems and plug-in electric vehicles. he same problem has been recently studied in [], where a primal-dual algorithm is proposed with a sublinear regret analysis based on the assumptions that the communication graph is balanced and an algorithm generated parameter is bounded. However, it is inappropriate to assume the boundedness of a parameter generated by the designed algorithm. o overcome these problems, a modified primal-dual algorithm is developed in this paper, which does not rest on any parameter s boundedness assumption. Meanwhile, unbalanced communication graphs are considered here. It is shown that in such cases the proposed algorithm still has the sublinear regret. Finally, the theoretical results are verified by a simulation example. Index erms Distributed online optimization, multi-agent networks, coupled inequality constraints. I. IRODUCIO With the rapid development of advanced technologies and low-cost devices, distributed optimization problems have recently attracted numerous attention from diverse communities, e.g., systems and control community, because a large number of practical problems boil down to distributed optimization problems over multi-agent networks, such as machine learning, statistical learning, sensor networks, resource allocation, formation control, and power systems [] [5]. Distinct from classic centralized optimization, distributed optimization involves multiple agents over a network which hold their individual private information, and usually, no centralized agents can access the entire information over the network. As such, an individual agent does not have adequate information to handle the optimization problem alone, and thus all agents need to exchange their local information in order to cooperatively solve a global optimization problem, see, for example, [6] [8]. his paper focuses on distributed online optimization. With regard to online optimization, it was first investigated for centralized scenario in machine learning community [9] []. In centralized online optimization, there exists a sequence of time-dependent convex objective cost functions, which is not known as a priori knowledge and only revealed gradually. o be specific, the cost function at current time slot is accessible X. Li and L. Xie are with School of Electrical and Electronic Engineering, anyang echnological University, 50 anyang Avenue, Singapore ( xiuxianli@ntu.edu.sg; elhxie@ntu.edu.sg). X. Yi is with the ACCESS Linnaeus Centre, Electrical Engineering, KH Royal Institute of echnology, 00 44, Stockholm, Sweden ( xinleiy@kth.se). only after the decision at current time instant is made. o measure the performance of online algorithms, it is conventional to compare the cost incurred by the algorithm through the sequential objective cost functions with the cost incurred by the best fixed decision in hindsight, i.e., the minimal cost that can be known offline, and the metric, as the difference between two costs, is called regret. In general, it is declared good for an online algorithm if the regret is sublinear. For example, the author in [9] has considered the online optimization problem subject to feasible set constraints, and an online subgradient projection algorithm was proposed. Later, the authors in [0], [] have further addressed the same problem as in [9]. Recently, a sequence of time-varying inequality constraints have been treated for the online optimization in []. As the emergence of complex tasks and extremely big data in modern life, a single agent in general cannot acquire enough information to perform a complicated task due to its limited sensing and computation ability and so on, based on which it is beneficial and preferable for a family of agents to accomplish an intricate mission in a cooperative manner. As a consequence, recent years have witnessed many research on distributed online optimization over multi-agent networks, such as [3] [], in which a collection of agents cooperatively deal with an online optimization problem. For example, distributed unconstrained online optimization problems have been considered in [3] by proposing an online subgradient descent algorithm with proportional-integral disagreement and in [4] by designing a distributed online subgradient pushsum algorithm. Also, distributed online optimization has been further studied with the development of a esterov based primal-dual algorithm [5], a variant of the Arrow-Hurwicz saddle point algorithm [6], a mirror descent algorithm [7], [8], a dual subgradient averaging algorithm [9], and a distributed primal-dual algorithm [0] when global/local set constraints exist. In addition, besides local feasible set constraints, local inequality constraints have been considered in [] with the design of a consensus-based adaptive primal-dual subgradient algorithm. As an application of distributed online optimization, smart grid networks have been discussed in [3]. More recently, a general constraint, i.e., a coupled inequality constraint, has been investigated in [] for distributed online optimization, where a distributed primal-dual algorithm is proposed and a sublinear regret is provided. It is known that coupled inequality constraints find a multitude of applications in reality, such as optimal wireless networking [], smart grid and plug-in electric vehicles [4], etc. However, a drawback in [] is to assume the boundedness of Lagrange multipliers, because the multipliers are automatically generated by the designed algorithm and thus should be proved to be bounded,

2 yet not appropriate given as an assumption. It should be noted that coupled inequality constraints have also been addressed for distributed optimization in [5] [9], but [] is the first one to consider distributed online optimization with coupled inequality constraints. his paper revisits distributed online optimization subject to coupled inequality constraints, where all involved functions, including objective and constraint functions, are revealed over time, and all agents are unaware of future information. o solve this problem, a different algorithm from [] is proposed, for which a sublinear regret can be obtained without the assumption on the boundedness of any relevant parameters generated by the designed algorithm. he contributions of this paper can be summarized as follows: ) In comparison with [], the results in this paper do not rely on the assumption that Lagrange multipliers generated by the proposed algorithm are bounded. ote that the removal of this assumption is nontrivial, thus reducing the conservatism that the aforementioned bounded assumption is employed in []; ) Balanced communication graphs have been used for all agents information exchange in []. In contrast, more general interaction graphs, i.e., unbalanced graphs, are considered in this paper for distributed online optimization. o cope with the imbalance of communication graphs, a push-sum idea [30] [36] is exploited for designing our algorithm in order to counteract the effect of graph s imbalance. he rest of this paper is structured as follows. Section II presents some preliminary knowledge and then formulate the considered problem. Section III provides the main results of this paper, and subsequently, a simulation example is provided for supporting the theoretical results in Section IV. In Section V, the regret and constraint violation analysis are given. Section VI concludes this paper. otations: Denote by [] := {,,,...,} the index set for a positive integer. he set ofn-dimensional vectors with nonnegative entries is denoted by R n +. Let col(z,...,z k ) be the concatenated column vector of z i R n,i [k]. Denote by and the standard Euclidean norm and l -norm, respectively. x and x,y denote the transpose of a vector x and the standard inner product of x,y R n, respectively. Let [z] + be the component-wise projection of a vectorz R n onto R n +. Let,0 be the compatible column vectors of all entries and 0, respectively. I is the identity matrix of compatible dimension. A. Graph heory II. PRELIMIARIES Denote by G t = (V,E t ) a simple graph at time slot t, where V = {,...,} is the node set and E t V V is the edge set at time instant t. An edge (j,i) E t means that node j can route information to node i at time step t, where j is called an in-neighbor of i and conversely, i is called an out-neighbor of j. Denote by + i,t = {j : (j,i) E t} and i,t = {j : (i,j) E t} the in-neighbor and out-neighbor sets of node i, respectively. It is assumed that i + i,t and i i,t for all i []. he graph is said balanced at time t if + i,t = i,t, where means the cardinality of a set, and the graph is said balanced if it is balanced at all times. he in-degree and out-degree of node i at time t are respectively defined by d + i,t = + i,t and d i,t = i,t. A directed path is a sequence of directed consecutive edges, and a graph is called strongly connected if there is at least one directed path from any node to any other node in the graph. he adjacency matrix A t = (a ij,t ) R at time t is defined by: a ij,t > 0 if (j,i) E t, and a ij,t = 0 otherwise. For the communication graph, the following assumptions are imposed in this paper. Assumption. For all t 0, the communication graph G t satisfies: ) here exists a constant 0 < a < which lower bounds all nonzero weights, that is, a ij,t a if a ij,t > 0. ) he adjacency matrix A t is column-stochastic, i.e., a ij,t = for all j [], and meanwhile, a ij,t for all i []. 3) here exists a constant Q > 0 such that the graph (V, l=0,...,q E t+l ) is strongly connected for all t. It is worth pointing out that Assumption is less conservative than that in [], where A t is assumed doubly stochastic, i.e., balanced graphs. B. Optimization heory he projection of a point x R n onto a closed convex set S R n is defined to be the point that has the shortest distance to x, that is, P S (x) := argmin y S x y, which satisfies the following basic properties (x P S (x)) (y P S (x)) 0, x R n, y S () P S (z ) P S (z ) z z, z,z R n. () For a convex function g : R n R, a subgradient of g at a point x R n is defined to be a vector s R n such that g(y) g(x) s (y x), y R n, (3) and the set of subgradients at x is called the subdifferential of g at x, denoted by g(x). When the function g is differentiable, then the subdifferential at any point only has a single element, which is exactly the gradient, denoted by g(x) at a point x. A function L : Ω Λ R, where Ω R n,λ R m, is called convex-concave if L(,λ) : Ω R is convex for every λ Λ and L(x, ) : Λ R is concave for each x Ω. For a convex-concave function L, a saddle point of L over Ω Λ is defined to be a pair (x,λ ) such that for all x Ω and λ Λ L(x,λ) L(x,λ ) L(x,λ ). (4) Given an optimization problem minf(x), s.t. g(x) 0, (5) x X where f(x) : R n R and g(x) : R n R m are convex functions, and X R n is a nonempty convex and closed set. ote that the inequality is understood componentwise. For

3 3 problem (5), which is usually called the primal problem, the Lagrangian function is defined by L(x,µ) = f(x)+µ g(x), (6) where µ is called the dual variable or Lagrange multiplier associated with the problem. hen, the Lagrangian dual problem is given as max q(µ), (7) µ R m + where q(µ) := min x X L(x,µ), called Lagrange dual function. Let f and q be the optimal values of (5) and (7), respectively. As is known, the weak duality q f is always true, and furthermore the strong duality q = f holds if a constraint qualification, such as Slater s condition, holds [37] [39]. C. Problem Formulation his section formulates the distributed online optimization problem. In this problem, there exists a sequence of timevarying global objective cost functions {f t (x)} t=0 which are not known in advance and just revealed gradually over time. At each time step t, the global cost function f t is composed of a group of local cost functions over a network with agents, i.e., f t (x) = f i,t (x i ), (8) where x := col(x,...,x ) with x i X i R ni, and f i,t : R ni R {± } is proper. After agent i [] makes a decision at time t, say x i,t, the cost function f i,t is only revealed to agent i and a cost f i,t (x i,t ) is incurred. hat is, each agent only gradually accesses the information of f i,t along with an incurred cost. In the meantime, there also exists a collection of proper functionsg i : R ni R m {± } m,i [] which impose global and coupled inequality constraints for the online optimization problem, that is, at each time step t it should satisfy g(x) := g i (x i ) 0, (9) where g i is only known by agent i for each i []. For brevity, let X = Π X i be the Cartesian product of X i s, and X := {x X : g(x) 0}, (0) which is assumed nonempty. he goal of the distributed online optimization is to reduce the total incurred cost over a finite time horizon > 0. Specifically, the aim is to design an algorithm such that Reg() := f i,t (x i,t ) f i,t (x i) () A function h(z) : R n R {± } is called proper if h(z) < + for at least one z and h(z) > for all z. is minimized, where () is called the regret for measuring the performance of designed algorithms, where x i is the i-th component of x = col(x,...,x ) and x := argmin x X f i,t (x i ), () that is, x is the best decision vector by knowing the full knowledge of f i,t,i [],t [,] as an a priori and without any communication restrictions among agents. ote that all inequalities and equalities throughout this paper are understood componentwise. Generally speaking, a proposed algorithm is announced good if the regret is sublinear with respect to, i.e., Reg() = o(), where o() means that lim o()/ = 0. Intuitively, the sublinearity of the regret guarantees that the averaged value of the global objective function over time horizon achieves the optimal value as goes to infinity. Moreover, as the distributed online optimization involves coupled inequality constraints g(x) 0, it is indispensable for the designed algorithm to eventually respect this kind of constraints. hat is, the following constraint violation Reg c [ () := ] g i (x i,t ) (3) should grow more slowly than. Mathematically, it should be ensured by the designed algorithm that Reg c () is also sublinear with respect to, i.e., Reg c () = o(). Remark. o facilitate the understanding of online optimization, a simple centralized online optimization problem is introduced here, called prediction from expert advice, which is well known in prediction theory [40]. In this problem, there is one decision maker or agent who has to make a decision among the advice of l given experts, and the decision maker is unaware of the loss corresponding to each expert s advice. he decision maker will only know an incurred loss between zero and one after committing his/her decision. his process is repeated over time, and at each time the different experts costs can be arbitrary, maybe even adversarial in which scenario the experts may attempt to mislead the decision maker. he purpose is for the decision maker to follow the best expert s advice in hindsight. his problem can be cast as a special case of centralized online optimization problems. o be specific, it is easy to see that the decision set, from which the decision maker can choose a decision, is the set of all distributions over l elements associated with l experts, i.e., X i = {x R l : l k= x k =,x k 0}. Assume that h t (k) is the cost of expert k at time step t, and denote by h t = col(h t (),...,h t (l)) the cost column vector. By selecting an expert according to the distribution x, the cost function at time slot t is the expected cost, i.e., f t (x) = h t x. hen the decision maker aims to minimize the total cost f t(x t ) over a finite horizon > 0, where x t is the decision made at time t. As a result, the experts problem is a special case of centralized online optimization problems without inequality constraints. +

4 4 o this end, some necessary assumptions on the online optimization problem are listed as follows. Assumption. ) he functionsf i,t andg i for alli [], t 0 are convex. ) All the sets X i,i [] are convex and compact. he first assumption above does not require each function to be differentiable. he second assumption has been widely employed in distributed optimization [5], [0], [], in which the compactness of all X i s can result in that there exist positive constants B x,b f and B g such that x B x, (4) f i,t (x) B f, t 0 (5) g i (x) B g, x X i, i []. (6) Furthermore, in light of the facts that f i,t,g i are all convex and X i s are compact, it can be concluded that there exist positive constants C f and C g such that for any x,y X i and i [],t 0 f i,t (x) f i,t (y) C f x y, (7) g i (x) g i (y) C g x y, (8) f i,t (x) C f, g i (x) C g. (9) ote that it has been implicitly postulated in (5)-(9) that the limit superiors of f i,t (x), g i (x), f i,t (x) and g i (x) as t goes to infinity are bounded for all x X i,i [], that is, they are not drastically influenced by time t at infinity. III. MAI RESULS his section presents the main results of this paper, including the algorithm design and the conclusions on its regret and constraint violation. o start with, the Lagrangian function L t : R n R m + R of the online optimization problem at time instant t is defined as L t (x,µ) = f i,t (x i )+µ g i (x i ), (0) where n is the dimension of x X, i.e., n := n i, and µ 0 is the dual variable or Lagrange multiplier vector of this problem. By defining L i,t (x i,µ) := f i,t (x i )+µ g i (x i ), () it is easy to see that L t (x,µ) = L i,t(x i,µ). For the centralized online optimization where only one centralized agent exists in the network and attempts to solve the optimization problem, a well-known algorithm is the so-called Arrow-Hurwicz-Uzawa saddle point algorithm or primal-dual algorithm [4] by virtue of leveraging subgradients of primal and dual variables of the Lagrangian function L t, explicitly given as x t+ = P X (x t α t s x,t ), µ t+ = [µ t +α t µ L t (x t,µ t )] +, () where α t is the stepsize, µ L t (x t,µ t ) = g i (x i,t ), (3) and s x,t is a subgradient of L t with respect to x at point (x t,µ t ), i.e., ( ( s x,t x f i,t (x i,t ) )+ ) x g i (x i,t ) µ t. (4) However, in the scenario of distributed online optimization, no centralized agent can access the full knowledge of f t (x) and g(x), which are only gradually revealed for each individual agent in the network. As a consequence, algorithm () is not applicable directly since each agent does not have an identicalµ t and does not know µ L t (x t,µ t ) at time slot t. As such, the authors in [] have proposed a modified algorithm based on (), i.e., x i,t+ = P Xi (x i,t α t s i,t ), [ µ i,t+ = a ij,t µ j,t +α t a ij,t y j,t ]+, y i,t+ = a ij,t y j,t +g i (x i,t+ ) g i (x i,t ), (5) where s i,t f i,t(x i,t ) + g i (x i,t ) a ij,tµ j,t, and y i,t is an auxiliary variable of agent i for tracking the function g i(x i,t )/. It is shown that algorithm (5) can ensure the sublinearity of both the regret and constraint violation. evertheless, a critical assumption is that µ i,t are bounded for all i [] and t, which is inappropriate since µ i,t is generated by the algorithm (5) and should be proved to be bounded. On the other hand, algorithm (5) is designed for balanced communication graphs among agents, yet not applicable for unbalanced interaction graphs which are more general and practical in engineering applications. As pointed out above, two challenges appear in this paper when handling problem (8)-(9): one is to consider unbalanced communication graphs, as shown in Assumption for A t, and the other is to eliminate the assumption on the boundedness of µ i,t for all i [] and t. o address the two issues, two strategies are respectively introduced in the sequel. Firstly, to deal with unbalanced communication graphs, there are generally four methods which are respectively the push-sum method [30] [36], the surplus -based method [4], the row-stochastic matrix method [43], and the epigraph method [44]. Among which, the push-sum approach is the most popular one, originally devised for average consensus problems over unbalanced graphs [30] [3]. For the other three methods, there are some shortcomings. Specifically, the surplus -based idea used in [4] is required to access global information since a parameter in the algorithm depends on communication weight matrices, while some network-size variables are introduced for each agent in [43], [44] which will incur extremely high computational complexity especially for large-scale networks. Based on the aforementioned discussion, in this paper we adopt the push-sum approach to handle the

5 5 imbalance of the communication graph among agents. ote that it is inevitable for each agent in the push-sum method to know its own out-degree, which is a result claimed in [45]. Actually, as pointed out in [35], the information on the out-degree for each individual agent can be known by virtue of bidirectional exchange of hello messages during only a single round of communication. Specifically, in view of the push-sum idea, algorithm (5) is redesigned as = a ij,t w j,t, ˆµ i,t = a ij,t µ j,t, ŷ i,t = a ij,t y j,t, x i,t+ = P Xi (x i,t α t s i,t+ ), [ ŷ ] i,t µ i,t+ = ˆµ i,t +α t wi,t+, + y i,t+ = ŷ i,t +g i (x i,t+ ) g i (x i,t ), (6) where s i,t+ is defined as s i,t+ f i,t (x i,t )+ g i (x i,t ) ˆµ i,t, (7) and w i,t R is a variable of agent i, aiming at removing the imbalance of the communication graph by, roughly speaking, tracking the right-hand eigenvector of A t associated with the eigenvalue. Secondly, it is often hard to guarantee the boundedness of µ i,t in algorithm (6), as in algorithm (5). o hinder the increase of a parameter or bound it, a quintessential method is to append some penalty functions or terms [], [37] [39], inspired by which an additional penalty term is designed to be incorporated into the update of µ i,t+ in order to impede the growth of µ i,t, that is, [ ( ŷi,t ˆµ )] i,t µ i,t+ = ˆµ i,t +α t wi,t+ β t w, (8) i,t+ + where β t is a stepsize to be determined. ote that there is another method to handle the boundedness of µ i,t, that is, performing projections on some bounded set M i for agent i, instead of on R m +, when updating µ i,t at each time slot, as done in [5], [46], but the computation of the set M i is usually difficult or requires global information of the network. At this stage, the proposed algorithm in this paper is summarized in Algorithm, called distributed online primaldual push-sum algorithm. With the above preparations, it is now ready to present the main results of this paper. heorem. Under Assumptions and, and letα 0 =,β 0 =, and for t α t =, β t = t tκ, (34) where κ is a constant satisfying 0 < κ < /4, then the regret () and constraint violation (3) can be upper bounded as Reg() = O( +κ ), (35) Reg c () = O( κ ), (36) Algorithm Distributed Online Primal-Dual Push-Sum Require: Set 4. Locally initialize w i,0 =, x i,0 X i, µ i,0 = 0 and y i,0 = g i (x i,0 ) for all i []. : If t =, then stop. Otherwise, update for each i []: = a ij,t w j,t, (9) ˆµ i,t = a ij,t µ j,t, ŷ i,t = a ij,t y j,t, (30) x i,t+ = P Xi (x i,t α t s i,t+ ), (3) [ ( ŷi,t ˆµ )] i,t µ i,t+ = ˆµ i,t +α t wi,t+ β t w, (3) i,t+ + y i,t+ = ŷ i,t +g i (x i,t+ ) g i (x i,t ), (33) : Increase t by one and go to Step. where h = O(h ) means that there exist a positive constant C such that h C h for two functions h,h. Proof. he proof can be found in Section V-B. Remark. It can be found from heorem that Reg() has a convergence rate almost at O( / ) when κ is sufficiently small, and meanwhilereg c () will reach a good convergence rate when κ is large enough. As a result, there should be a tradeoff for choosing κ such that both Reg() and Reg c () get good convergence speeds. In comparison with [], where the same problem as (8)-(9) has been studied recently, the sublinearity of Reg() and Reg c () in heorem is obtained depending on less conservative assumptions, that is, in this paper no assumptions on boundedness of µ i,t are employed while it is utilized in []. In addition, balanced communication graphs are considered in [], while more general interaction graphs, i.e., unbalanced graphs, are taken into account here. As discussed in Remark, the parameter κ can be specified for the same convergence rate for Reg() and Reg c () as follows. Corollary. In heorem, let κ = 5, (37) then the regret () and constraint violation (3) can be upper bounded by the same as Reg() = O( 9 0 ), (38) Reg c () = O( 9 0 ). (39) Proof. o achieve the same convergence rate for Reg() and Reg c (), it amounts to that +κ = κ, thus leading to κ = /5, which directly implies (38) and (39). As a special case of problem (8)-(9), the time-invariant online optimization problem, that is, f i,t (x) s are independent of time t for all i [] and are simply denoted by f i (x), can enjoy a stricter upper bound on Reg c (), as shown below.

6 6 heorem. For the time-invariant online optimization problem, if Assumptions and hold, and let α 0 =,β 0 =, and for t α t =, β t = t tκ, (40) where κ is a constant satisfying 0 < κ < /4, then the regret () and constraint violation (3) can be upper bounded as Reg() = O( +κ ), (4) Reg c () = O( 3 4 +κ ). (4) Proof. he proof can be found in Section V-C. IV. A SIMULAIO EXAMPLE his section applies Algorithm to the Plug-in Electric Vehicles (PEVs) charging problem [4], [7] in order to corroborate the algorithm s efficiency. he purpose of this PEVs charging problem is to seek an optimal overnight charging schedule for a collection of vehicles subject to some practical constraints, such as the limited charging rate for each vehicle and the overall maximal power that can be delivered by the whole network, etc. inequality constraints, meaning the whole networked power constraints. Reg()/ =50,Q=4 =50,Q= Fig.. Evolutions of Reg()/ with Q = 4 and Q = 9 for = 50 agents =50,Q=4 =50,Q= Reg c ()/ 0.06 (a) (b) Fig. 3. Evolutions of Reg c ()/ with Q = 4 and Q = 9 for = 50 agents. (c) Fig.. Schematic illustration of 4 switching graphs. As done in [7], a slightly modified only charging problem in [4] is taken into account here. hat is, the charging rate of each vehicle is permitted to be optimized at each time step, rather than making a decision on whether or not to charge the vehicle at some fixed charging rate. Formally, the charging problem at time slot t can be cast as f i,t (x i ) = c i,t x i in (8) and g i (x i ) = D i x i b/ in (9) with x i X i R ni being a local feasible set constraint for each i [], where X i is usually a compact convex polygon in charging problem. In this problem, the variable x i stands for the charging rate in specified time duration, andc i,t represents the unitary charging cost of vehicleiat time instant t, randomly chosen in [0,0] in the simulation. Also, (D ix i b/) 0 is the coupled (d) For the charging problem, as given in [4], [7], the dimension of x i for each individual agent is n i = 4, each local feasible set X i is confined by 97 inequalities, and the number of inequality constraints is m = 48. In this setup, let κ = 0., and different switching graphs are considered in this simulation along with the distinct number of agents. Specifically, Figs. and 3 show the evolutions of Reg()/ and Reg c ()/ for a group of = 50 vehicles when Q = 4 and Q = 9, respectively, in which the trajectories are tending to the origin, supporting Algorithm. Wherein, Q is given in Assumption for communication graphs, and for instance, four switching graphs in Fig. are employed here when Q = 4. It is worthwhile to notice that the value of Reg()/ in Fig. can be negative, which is reasonable because the inequality constraints are not always respected by x i,t. In addition, Figs. 5 and 6 give the trajectories of

7 7 Reg()/ and Reg c ()/ for a fixed communication graph, i.e., Q =, when = 50 and = 00, respectively, indicating the convergence of Algorithm in this scenario. Besides, observing Figs. 4 and 7, one can easily find that µ i,t and thus µ i,t are bounded in these simulations =50,Q=4 =50,Q= Fig. 4. Evolutions of averaged µ i, over agents. Lemma. Consider the sequences {w i,t } with w i,t R and {z i,t } with z i,t R m for i [],t, having the following dynamics: z i,t+ = a ij,t z j,t +ǫ i,t+, = z i,t+ = a ij,t w j,t, a ij,tz j,t, (43) where ǫ i,t is a perturbation for agent i at time slot t. Denote by z t = z i,t the averaged variable of z i,t s. If Assumption holds, then the following statement is true: z i,t+ z t 8 t λ r( t z 0 + λ t k ) ǫ k, k= where z 0 := col(z,0,...,z,0 ), ǫ k := col(ǫ,k,...,ǫ,k ), r := inf t=0,,... (min i [] [A t A 0 ] i ) with [ ] i being the i-th component of a vector, and λ (0,), satisfying r ( Q, λ ) Q Q =50,Q= =00,Q= 0. =50,Q= =00,Q= 0.0 Reg()/ Reg c ()/ Fig. 5. Evolutions of Reg()/ with = 50 and = 00 agents when Q =. V. PROOFS OF HE MAI RESULS his section gives the analysis of the regret () and constraint violation (3). In doing so, some lemmas are first provided and then the proofs of main results are presented. A. Useful Lemmas First, a result on perturbed push-sum algorithms is listed below, which is cited from [34] Fig. 6. Evolutions of Reg c ()/ with = 50 and = 00 agents when Q =. In the above lemma, the parameters r, λ can be better selected when the adjacency matrix A t is doubly stochastic, i.e., over balanced graphs, for all t. Please refer to [34] for more details. With Lemma in place, it is straightforward to see that (3) and (33) can be rewritten in the perturbed form (43) as µ i,t+ = ˆµ i,t +ǫ µi,t+, (44) y i,t+ = ŷ i,t +ǫ yi,t+, (45) where [ ( ŷi,t ˆµ )] i,t ǫ µi,t+ := ˆµ i,t +α t wi,t+ β t w ˆµ i,t, (46) i,t+ + ǫ yi,t+ := g i (x i,t+ ) g i (x i,t ). (47)

8 8 o move forward, let us, for notational simplicity, denote for all i [] and t µ i,t+ = ˆµ i,t, µ t = ỹ i,t+ = ŷi,t, ȳ t = µ i,t, y i,t. (48) =50,Q= =00,Q= Fig. 7. Evolutions of averaged µ i, over agents. For the purpose of facilitating the following analysis, it is helpful to present the preliminary results below. Lemma. If Assumption holds, then for all i [] and t 0 r w i,t, r, ȳ t B g. (49) Proof. First, w i,t r follows directly from the definition of r in Lemma once noting that w i,0 = for all i []. o prove w i,t, it is easy to see that (9) can be rewritten as w t+ = A t w t, (50) where w t := col(w,t,...,w,t ). By pre-multiplying on both sides of (50), one has that = w i,t for all t 0, which combines with the fact that w i,0 = for all i [] gives rise to that w i,t = for all t 0. Observing the fact that w i,t 0, it can be concluded that w i,t. ext, let us show that r by contradiction. If r >, then w i,t r >, contradicting w i,t =. Hence, r. Finally, it remains to prove ȳ t B g. In view of (33), one can obtain that y t+ = (A t I m )y t +G(x t+ ) G(x t ), (5) wherex t := col(x,t,...,x,t ), y t := col(y,t,...,y,t ), and G(x t ) := col(g (x,t ),...,g (x,t )). By pre-multiplying on both sides of (5), it can obtain that y i,t+ = y i,t + g(x t+ ) g(x t ), and thus it yields that ȳ t+ g(x t+ )/ = ȳ t g(x t )/. Combining with y i,0 = g i (x i,0 ) results in that ȳ t = g(x t )/ for all t, thereby implying that ȳ t B g by (6). his finishes the proof. At this point, it is necessary to provide the results for bounding y i,t and µ i,t, which are pivotal to the subsequent analysis. Lemma 3. Under Assumption, there exists a constant B y > 0 such that for all i [] and t where y i,t B y, ŷ i,t B y, (5) ˆµ i,t B µ i,t, µ i,t B µ i,t /a, (53) { B µ i,t := max wi,t+ B y β t r, w } i,t+b y r 3. (54) Proof. Let us first prove (5). In view of (45), it follows from Lemma that ỹ i,t+ ȳ t 8 ( t λ t y 0 + λ t k ǫ y,k ), (55) r k= where r,λ are given in Lemma, y 0 := col(y,0,...,y,0 ) and ǫ y,k := col(ǫ y,k,...,ǫ y,k ). It is easy to see that ǫ yi,t+ m ǫ yi,t+ mb g, where (6) has been used to obtain the last inequality. As a result, one has that t k= λt k ǫ y,k mb g /( λ), which together with (55) implies that ỹ i,t+ ȳ t is bounded. At this stage, the boundedness of ỹ i,t+ ȳ t and ȳ t (by Lemma ) yields that ỹ i,t+ is bounded, which together with the boundedness of w i,t in Lemma leads to that ŷ i,t is bounded. At this point, invoking (33), (6) and boundedness of ŷ i,t, it can be concluded that y i,t is bounded, that is, there exists B y > 0 such that y i,t B y for all i []. As a result, ŷ i,t = a ij,ty j,t a ij,t y j,t B y, thus finishing the proof of (5). What follows is the proof of (53). Let us first show that ˆµ i,t B µ i,t by induction. It is easy to see that ˆµ i,0 B i,0 due to µ i,0 = 0 for all i []. Assume now that it is true at time instant t for all i [], and it suffices to show that it remains true at time t+. At first step, it can be obtained that ( ŷi,t ˆµ i,t +α t wi,t+ β ) tˆµ i,t ( = α ) tβ t ˆµ i,t + α tŷ i,t wi,t+. (56) In the following, three different scenarios are considered, i.e., ) α t β t / >, ) α t β t / and β t /r >, and 3) β t /r. ). When α t β t / >, one has that αtβt < 0, and it then follows from (56) that ( ŷi,t ˆµ i,t +α t wi,t+ β ) tˆµ i,t α tŷ i,t wi,t+ α tb y r = α t B y r B y r 3, (57)

9 9 where we have used (5) and r to gain the second inequality, and α t, r to get the last inequality. herefore, in light of (3) and (57), one can have that µ i,t+ B y /r 3, thereby yielding that ˆµ i,t+ = a ij,t+ µ j,t+ B y r 3 a ij,t+ = w i,t+b y r 3, (58) where (9) has been used for obtaining the last equality, which further implies that ˆµ i,t+ B µ i,t+. ). When α t β t / and β t /r >, it is easy to verify that B µ i,t = B y /r 3. As a result, it can be deducted by (56) that ( ŷi,t ˆµ i,t +α t β ) tˆµ i,t = w i,t+ ( α tβ t + α tb y r ) wi,t+ B y r 3 ( α tβ t + α ) tr wi,t+ B y r 3 B y r 3, (59) where we have utilized β t /r > to have the last inequality. hus, as done in ) for the remaining part, it can be asserted that ˆµ i,t+ B µ i,t+. 3). When β t /r, it has that α t β t / α t β t /r due to α t, and B µ i,t = B y /(β t r ) in this case. Invoking (56) leads to that ( ŷi,t ˆµ i,t +α t wi,t+ β ) tˆµ i,t ( α ) tβ t wi,t+ B y β t r + α tb y r = B y β t r, (60) which, together with (3), gives rise to that µ i,t+ B y /β t r. Hence, it yields that ˆµ i,t+ = a ij,t+ µ j,t+ B y β t r a ij,t+ w i,t+b y β t+ r, (6) where β t+ < β t has been employed for getting the last inequality. Consequently, one can obtain that ˆµ i,t+ B µ i,t+. hrough discussions on the above three cases, it can be claimed that ˆµ i,t B µ i,t holds for all i [] and t 0, which, together with ˆµ i,t = a ij,tµ j,t a ii,t µ i,t, further results in that µ i,t B µ i,t /a ii,t B µ i,t /a. his completes the proof. Equipped with the above results, it is now ready to present the results on the disagreement of L t (x,µ) at different points. Lemma 4. Under Assumptions and, we have that for all x = col(x,...,x ) X and µ R m + L t (x t, µ t ) L t (x, µ t ) ( xi,t x i x i,t+ x i ) α t + α t (C f +C g B t ) +B g µ i,t+ µ t, (6) L t (x t,µ) L t (x t, µ t ) ( µi,t w i,t µ µ i,t+ µ ) α t +( µ +B t ) ỹ i,t+ ȳ t + α tby r 6 +B g µ i,t+ µ t + β t µ, (63) where x t = col(x,t,...,x,t ), and { By B t := max B } β t r, y r 3. (64) Proof. o show (6), it can be obtained by using () that for all x X x i,t+ x i x i,t x i α t s i,t+ = x i,t x i +α t s i,t+ α t s i,t+(x i,t x i ), (65) in which, in view of (7), the last term can be manipulated as α t s i,t+(x i,t x i ) = α t [ f i,t (x i,t )+ µ i,t+ g i (x i,t )](x i,t x i ) α t [f i,t (x i,t ) f i,t (x i )+ µ i,t+ (g i(x i,t ) g i (x i ))] = α t [L i,t (x i,t, µ t ) L i,t (x i, µ t ) +( µ i,t+ µ t ) (g i (x i,t ) g i (x i ))], (66) where the convexity of f i,t,g i (i.e., (3)) and () have been exploited for obtaining the inequality and the last equality, respectively. ote that s i,t+ C f +C g B t by (9), (7) and (53). Consequently, by combining (65) and (66), preforming summations over i [] leads to (6), thus ending the proof of (6). It only remains to show (63). o do so, making use of () can yield that for all µ R m + µ i,t+ µ ( ŷi,t ˆµ i,t µ+α t = ˆµ i,t µ +α t w i,t+ ŷi,t wi,t+ β tˆµ i,t ) β tˆµ i,t + α tŷ i,t w i,t+ (ˆµ i,t µ) α tβ tˆµ i,t (ˆµ i,t µ). (67)

10 0 For the first term in the last equality of (67), it can be concluded that ˆµ i,t µ = a ij,t (µ j,t w j,t µ) k= ( ) = a a ij,t(µ j,t w j,t µ) ik,t k= a ik,t = ( ) a ij,t a ik,t k= a µ j,t w j,t µ ik,t k= k= a ik,t a ij,t µ j,t w j,t µ µ j,t w j,t µ, (68) where (9) and (30) are utilized to obtain the first equality, the convexity of norm is applied to get the first inequality, and Assumption. is employed for procuring the last equality. Concerning the second term in the last inequality of (67), one can conclude that ŷi,t wi,t+ β tˆµ i,t ŷ i,t w i,t+ + ( By ) +(βt r B t ) β tˆµ i,t 4B y r 6, (69) where we have employed (5) and (53) to obtain the second inequality, and r,β t B t B y /r 3 for the last inequality. ote that β t B t B y /r 3 can be directly deducted from (64) using r and β t. As for the third term in the last equality of (67), one has that ŷ i,t wi,t+ (ỹi,t+ = (ˆµ i,t µ) ȳt ) (ˆµi,t µ) + ȳ t (ˆµ i,t µ t )+ ȳ t ( µ t µ) (B t + µ ) ỹ i,t+ ȳ t +B g µ i,t+ µ t +ȳ t ( µ t µ) = (B t + µ ) ỹ i,t+ ȳ t +B g µ i,t+ µ t + [L i,t(x t, µ t ) L i,t (x t,µ)], (70) where we have made use of (53) and (49) for getting the inequality, and () and ȳ t = g(x t )/ for the last equality. With regard to the fourth term in the last equality of (67), we consider the function h(z) := z / for z R m, which is convex. Using convexity, one can obtain that h(z ) h(z ) h(z )(z z ), z,z R m (7) which, by letting z = ˆµ i,t and z = µ, follows that ˆµ i,t w i,t+ µ ˆµ i,t(ˆµ i,t µ), (7) further implying that α tβ tˆµ i,t (ˆµ i,t µ) α tβ t (w i,t+ µ ˆµ i,t ) α t β t µ α t β t µ, (73) where in (49) has been used for having the last inequality. ow, by combining (68), (69), (70), (73) with (67), performing summations over i [] gives rise to (63), which completes the proof. B. Proof of heorem It is now ready for us to give the proof of heorem. By virtue of Lemma 4, it can be obtained that for all x X and µ R m + L t (x t,µ) L t (x, µ t ) = L t (x t,µ) L t (x t, µ t )+L t (x t, µ t ) L t (x, µ t ) ( x i,t x i x i,t+ x i ) α t + α t ( µ i,t w i,t µ µ i,t+ µ ) +B g (+) µ i,t+ µ t +( µ +B t ) ỹ i,t+ ȳ t + α t (C f +C g B t ) + α tby r 6 + β t µ. (74) Meanwhile, by letting x = x with x = col(x,...,x ) being given in (), it is easy to verify that L t (x t,µ) L t (x, µ t ) β t µ = f i,t (x i,t )+µ g i (x i,t ) f i,t (x i ) µ t f i,t (x i,t ) f i,t (x i ) +µ g i (x i ) β t µ g i (x i,t ) β t µ, (75) where the inequality is obtained by resorting to µ t 0 and g i(x i ) 0. For ease of exposition, define g e (µ) := µ g i (x i,t ) µ β t. (76)

11 hen, by selecting x = x, combining (74) with (75) yields that for all µ R m + f i,t (x i,t ) f i,t (x i)+g e (µ) ( x i,t x i x i,t+ x i ) α t }{{} =:S + ( µ i,t w i,t µ µ i,t+ µ ) α t }{{} =:S (µ) +B g (+) µ i,t+ µ t } {{ } =:S 3 + ( µ +B t ) ỹ i,t+ ȳ t } {{ } =:S 4(µ) +(Cf + B y α t +Cg r 6 ) }{{} =:S 5 } {{ } =:S 6 α t Bt. (77) In the following the termss i,i [6] are gradually analyzed. First, some calculations can lead to that S = α + x i, x i x i,+ x i α ( ) x i,t x i α t α t t= 4B x α, (78) where the non-positivity of the second term and x i B x, x i,t B x, i [],t 0 have been used for implying the inequality. Similarly, by letting µ = 0, one can have that S (0) = α + t= µ i, α ( α t α t ) µ i,+ µ i,t 4 B a α, (79) where we have made use of µ i,t B µ i,t /a B t/a B /a for obtaining the inequality. o bound S 3, invoking Lemma implies that 8 r µ i,t+ µ t ( λ t µ 0 + t λ t k ǫ µ,k ), (80) k= where ǫ µ,k := col(ǫ µ,k,...,ǫ µ,k ) with ǫ µi,k being defined in (46). In view of (), we have that ǫ µi,t+ m ǫ µi,t+ ŷ i,t α t m wi,t+ β tˆµ i,t α t mby r 3, (8) where, to obtain the last inequality, Lemma 3 has been applied along withr andβ t for allt 0. herefore, it follows that µ i,t+ µ t 8 λ r( λ) + 6 m B y r 4 which, together with the fact that results in k= t λ t k α k = µ i,t+ µ t k= t λ t k α k, (8) t λ t t=0 λ t α k t=0 λ α k α k, (83) 8 λ r( λ) + 6 m B y r 4 α k. (84) ( λ) hus, it directly follows from (84) that ( 8 λ S 3 B g (+) r( λ) + 6 m B y r 4 ( λ) α k ). (85) Similarly, to bound S 4 (µ) for µ = 0, it can be deducted by Lemma that ỹ i,t+ ȳ t 8 r ( λ t y 0 + t λ t k ǫ y,k ), (86) k= where ǫ y,k := col(ǫ y,k,...,ǫ y,k ) with ǫ yi,k being defined in (47). In light of (8), one has that ǫ yi,t+ m ǫ yi,t+ mc g x i,t+ x i,t. (87) Invoking () leads to that x i,t+ x i,t α t s i,t+, which together with (3), (7), (9), and (53) implies that ǫ yi,t+ ( mc g α t C f + B ) yc g β t r 3. (88)

12 hen, similar to (8)-(84), it can be obtained that ỹ i,t+ ȳ t 8λ y 0 r( λ) mb y C g r 4 ( λ) mcf C g r( λ) α k α k β k. (89) At this point, by using (89) and observing that B t B y /(β t r 3 ) and β t β for t, we obtain that S 4 (0) B y r 3 B y r 3 κ ỹ i,t+ ȳ t [ 8λ y0 κ r( λ) + 8 mb y C g κ r 4 ( λ) + 8 mc f C g κ r( λ) α k α k ]. (90) Also, with reference to the fact that B t B y /(β t r 3 ), it can be concluded that S 6 B y C g r 6 = B y C g r 6 α t β t B yc g r 6 ( + t κ t κ dt ) +κ = B ycg ( r 6 +4κ + +4κ < B ycg +κ r 6, (9) (+4κ) where the last inequality is obtained by applying /(+ 4κ) < 0 due to k (0,/4). ote that α k + t / dt = ( ) / = O( / ) and also k= α k = O( / ). hus, it is easy to verify that S = O( ), S (0) = O( +κ ), S 3 = O( ), S4 (0) = O( +κ ), S 5 = O( ), S6 = O( +κ ), which together with (77) and g e (0) = 0 completes the proof of (35) in heorem. In what follows it remains to show (36). ote that it has been proven that (77) holds for all µ R m +. It is straightforward to verify that function g e (µ), defined in (76), can achieve its maximal value [ β t ] g i (x i,t ) + ) (9) when µ = µ 0, where µ 0 := [ ] g i(x i,t ) β t which together with (77) results in f i,t (x i,t ) + α t + f i,t (x i )+ (Regc ()) β t ( x i,t x i x i,t+ x i ) α t, (93) ( µ i,t w i,t µ 0 µ i,t+ µ 0 ) +B g (+) + µ i,t+ µ t ( µ 0 +B t ) ỹ i,t+ ȳ t +(C f + B y r 6 ) α t +Cg α t Bt. (94) Simple manipulations lead to that for 4 and κ (0,/4) β t t κ dt = κ κ κ ( κ), (95) β t + t κ dt = κ κ κ κ κ, (96) which, together with (6), gives rise to µ 0 B g β t B g( κ) κ. (97) By resorting to the similar arguments to bound S i s in (77) and further applying (95)-(96), we can bound the right-hand terms of (94) as f i,t (x i,t ) f i,t (x i)+ (Regc ()) β t = O( +κ ). (98) Additionally, with reference to (4) and (7), one can have that = f i,t (x i,t ) f i,t (x i ) (f i,t (x i,t ) f i,t (x i)) C f x i,t x i C f B x. (99)

13 3 Inserting (99) to (98) gives that (Reg c ()) β t O( +κ )+4 3 C f B x O( 3 +κ )+ 43 C f B x κ κ = O( κ ), (00) where we have employed (96) to obtain the second inequality, and 3/ + κ < κ due to κ < /4 for the last one. Obviously, (00) is equivalent to (36). his completes the proof of heorem. C. Proof of heorem his section gives the proof of heorem when f i,t s are independent of time for all i [], denoted by f i in this section. Firstly, (4) can be proved using the same argument as heorem. o show (4), it is easy to see from () that In this scenario, define x = argmin x X f(x) = β t f i (x i ). (0) f i (x i ), (0) L(x,µ) = f(x)+µ g(x), (03) where g(x) = g i(x i ). ow, invoking the property of saddle points can imply that L(x,µ ) L(x,µ ) for all x X, where µ is an optimal dual variable, which is equivalent to f(x )+(µ ) g(x ) f(x t )+(µ ) g(x t ) (04) when letting x = x t := col(x,t,...,x,t ). hen, summing (04) over t gives rise to f i (x i,t ) f i (x i) (µ ) where we have used the fact that (µ ) g(x ) = 0. Inserting (05) into (98) yields that (Reg c ()) β t (µ ) g i (x i,t ), g i (x i,t ) = O( +κ ), (05) (06) which, combining with the fact that g i(x i,t ) [ g i(x i,t ) ], implies that + [ = 4 µ ] g i (x i,t ) µ + ( β t β t ) +O( +κ ) β t. (07) With reference to (96), it can be obtained by (07) that [ g i (x i,t ) ]+ µ β t = O( 3 +κ ). (08) By considering the components, one can gain that {[ g i (x i,t ) ]+ µ } β t i = O( 3 4 +κ ), (09) where { } i denotes the i-th component of a vector. Consider two cases for { } i in (09). ) If the scalar { } i is negative, then it directly follows by (96) that {[ ] } { µ g i (x i,t ) < + i } β t = O( κ ). ) If the scalar { } i is nonnegative, then combining (09) with (96) leads to {[ Consequently, it can be concluded that Reg c [ () ] g i (x i,t ) which completes the proof of heorem. ] } g i (x i,t ) = O( 3 4 +κ ). (0) + i + VI. COCLUSIO i = O( 3 4 +κ ), () his paper has investigated distributed online convex optimization problems over directed multi-agent networks subject to local set constraints and coupled inequality constraints. It is noted that the same problem has been studied in [] along with the design of an online primal-dual algorithm. However, the results in [] heavily depend on the boundedness of Lagrange multipliers generated by the proposed algorithm, which is not reasonable. o overcome the shortcoming, a modified distributed online primal-dual push-sum algorithm has been proposed, which has been proven to possess a sublinear regret and constraint violation. Moreover, unbalanced communication graphs have been considered for networked agents, which are more general. Finally, the algorithm s performance has been demonstrated by a numerical application. Future work can focus on further improving the convergence rates on Reg()/ and Reg c ()/. REFERECES [] F. Bullo, J. Cortés, and S. Martínez, Distributed Control of Robotic etworks: A Mathematical Approach to Motion Coordination Algorithms. Princeton University Press, 009. [] M. Rabbat and R. owak, Distributed optimization in sensor networks, in Proceedings of 3rd International Symposium on Information Processing in Sensor etworks, Berkeley, California, USA, 004, pp [3] J. sitsiklis, D. Bertsekas, and M. Athans, Distributed asynchronous deterministic and stochastic gradient optimization algorithms, IEEE ransactions on Automatic Control, vol. 3, no. 9, pp , 986. [4] S. Li and. Başar, Distributed algorithms for the computation of noncooperative equilibria, Automatica, vol. 3, no. 4, pp , 987.

14 4 [5] Y. Xu,. Han, K. Cai, Z. Lin, G. Yan, and M. Fu, A distributed algorithm for resource allocation over dynamic digraphs, IEEE ransactions on Signal Processing, vol. 65, no. 0, pp , 07. [6] A. edić and A. Ozdaglar, Distributed subgradient methods for multiagent optimization, IEEE ransactions on Automatic Control, vol. 54, no., pp. 48 6, 009. [7]. S. Aybat, Z. Wang,. Lin, and S. Ma, Distributed linearized alternating direction method of multipliers for composite convex consensus optimization, IEEE ransactions on Automatic Control, vol. 63, no., pp. 5 0, 08. [8] J. Xu, S. Zhu, Y. C. Soh, and L. Xie, A Bregman splitting scheme for distributed optimization over networks, IEEE ransactions on Automatic Control, in press, doi: 0.09/AC , 08. [9] M. Zinkevich, Online convex programming and generalized infinitesimal gradient ascent, in Proceedings of 0th International Conference on Machine Learning, Washington, DC, USA, 003, pp [0] E. Hazan, A. Agarwal, and S. Kale, Logarithmic regret algorithms for online convex optimization, Machine Learning, vol. 69, no. -3, pp. 69 9, 007. [] S. Shalev-Shwartz, Online learning and online convex optimization, Foundations and rends in Machine Learning, vol. 4, no., pp , 0. [] M. J. eely and H. Yu, Online convex optimization with time-varying constraints, arxiv preprint arxiv: , 07. [3] D. Mateos-únez and J. Cortés, Distributed online convex optimization over jointly connected digraphs, IEEE ransactions on etwork Science and Engineering, vol., no., pp. 3 37, 04. [4] M. Akbari, B. Gharesifard, and. Linder, Distributed online convex optimization on time-varying directed graphs, IEEE ransactions on Control of etwork Systems, vol. 4, no. 3, pp , 07. [5] A. edić, S. Lee, and M. Raginsky, Decentralized online optimization with global objectives and local communication, in Proceedings of American Control Conference, Chicago, IL, USA, 05, pp [6] A. Koppel, F. Y. Jakubiec, and A. Ribeiro, A saddle point algorithm for networked online convex optimization, IEEE ransactions on Signal Processing, vol. 63, no. 9, pp , 05. [7] S. Shahrampour and A. Jadbabaie, An online optimization approach for multi-agent tracking of dynamic parameters in the presence of adversarial noise, in Proceedings of American Control Conference, Seattle, USA, 07, pp [8], Distributed online optimization in dynamic environments using mirror descent, IEEE ransactions on Automatic Control, vol. 63, no. 3, pp , 08. [9] S. Hosseini, A. Chapman, and M. Mesbahi, Online distributed convex optimization on dynamic networks, IEEE ransactions on Automatic Control, vol. 6, no., pp , 06. [0] S. Lee, A. edić, and M. Raginsky, Stochastic dual averaging for decentralized online optimization on time-varying communication graphs, IEEE ransactions on Automatic Control, vol. 6, no., pp , 07. [] D. Yuan, D. W. C. Ho, and G. Jiang, An adaptive primaldual subgradient algorithm for online distributed constrained optimization, IEEE ransactions on Cybernetics, in press, doi: 0.09/CYB , 07. [] S. Lee and M. M. Zavlanos, On the sublinear regret of distributed primal-dual algorithms for online constrained optimization, arxiv preprint arxiv:705.8, 07. [3] X. Zhou, E. Dall Anese, L. Chen, and A. Simonetto, An incentive-based online optimization framework for distribution grids, IEEE ransactions on Automatic Control, in press, doi: 0.09/AC , 07. [4] R. Vujanic, P. M. Esfahani, P. J. Goulart, S. Mariéthoz, and M. Morari, A decomposition method for large scale MILPs, with performance guarantees and a power system application, Automatica, vol. 67, pp , 06. [5].-H. Chang, A. edić, and A. Scaglione, Distributed constrained optimization by consensus-based primal-dual perturbation method, IEEE ransactions on Automatic Control, vol. 59, no. 6, pp , 04. [6] D. Mateos-únez and J. Cortés, Distributed saddle-point subgradient algorithms with Laplacian averaging, IEEE ransactions on Automatic Control, vol. 6, no. 6, pp , 07. [7] A. Falsone, K. Margellos, S. Garatti, and M. Prandini, Dual decomposition for multi-agent distributed optimization with coupling constraints, Automatica, vol. 84, pp , 07. [8] I. otarnicola and G. otarstefano, Constraint coupled distributed optimization: Relaxation and duality Approach, arxiv preprint arxiv:7.09, 07. [9], A duality-based approach for distributed optimization with coupling constraints, in Proceedings of International Federation of Automatic Control World Congress, oulouse, France, 07, pp [30] D. Kempe, A. Dobra, and J. Gehrke, Gossip-based computation of aggregate information, in Proceedings of 44th Annual IEEE Symposium on Foundations of Computer Science, Cambridge, MA, USA, 003, pp [3] F. Bénézit, V. Blondel, P. hiran, J. sitsiklis, and M. Vetterli, Weighted gossip: Distributed averaging using non-doubly stochastic matrices, in Proceedings of IEEE International Symposium on Information heory, Austin, exas, USA, 00, pp [3] A. D. Domínguez-García and C.. Hadjicostis, Distributed strategies for average consensus in directed graphs, in Proceedings of 50th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC), Orlando, FL, USA, 0, pp [33] K. I. sianos, S. Lawlor, and M. G. Rabbat, Push-sum distributed dual averaging for convex optimization, in Proceedings of IEEE Annual Conference on Decision and Control, Maui, HI, USA, 0, pp [34] A. edić and A. Olshevsky, Distributed optimization over time-varying directed graphs, IEEE ransactions on Automatic Control, vol. 60, no. 3, pp , 05. [35], Stochastic gradient-push for strongly convex functions on timevarying directed graphs, IEEE ransactions on Automatic Control, vol. 6, no., pp , 06. [36] C. Xi and U. A. Khan, DEXRA: A fast algorithm for optimization over directed graphs, IEEE ransactions on Automatic Control, vol. 6, no. 0, pp , 07. [37] D. P. Bertsekas, A. edić, and A. E. Ozdaglar, Convex Analysis and Optimization. Athena Scientific, 003. [38] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge U.K.: Cambridge University Press, 004. [39] A. P. Ruszczyński, onlinear Optimization. Princeton University Press, 006. [40] E. Hazan, Introduction to online convex optimization, Foundations and rends in Optimization, vol., no. 3-4, pp , 06. [4] K. J. Arrow, L. Hurwicz, and H. Uzawa, Studies in Linear and on- Linear Programming. Stanford University Press, 958. [4] C. Xi and U. A. Khan, Distributed subgradient projection algorithm over directed graphs, IEEE ransactions on Automatic Control, vol. 6, no. 8, pp , 07. [43] C. Xi, V. S. Mai, R. Xin, E. H. Abed, and U. A. Khan, Linear convergence in optimization over directed graphs with row-stochastic matrices, IEEE ransactions on Automatic Control, in press, doi: 0.09/AC , 08. [44] P. Xie, K. You, R. empo, S. Song, and C. Wu, Distributed convex optimization with inequality constraints over time-varying unbalanced digraphs, IEEE ransactions on Automatic Control, in press, doi: 0.09/AC , 08. [45] J. M. Hendrickx and J.. sitsiklis, Fundamental limitations for anonymous distributed systems with broadcast communications, in Proceedings of 53rd Annual Allerton Conference on Communication, Control, and Computing, Monticello, IL, USA, 05, pp [46] M. Zhu and S. Martínez, On distributed convex optimization under inequality and equality constraints, IEEE ransactions on Automatic Control, vol. 57, no., pp. 5 64, 0.