Distributed Optimization over Networks Gossip-Based Algorithms

Transcription

1 Distributed Optimization over Networks Gossip-Based Algorithms Angelia Nedić ISE Department and Coordinated Science Laboratory University of Illinois at Urbana-Champaign

2 Outline Random Gossip for Averaging Distributed Algorithm based on Random Gossip 1

3 Distributed Computational Model: Self-organized Agents The model consists of a network of computing agents (nodes) that are connected and need to collectively optimize a network-wide objective function Agents can communicate only with immediate neighbors in the network Agents have individual objective functions that they do not reveal to each other Agents cooperate (share info.) with their neighbors Distributed load-balancing in a network Uplink power control in mobile cellular net f 1 (x 1,...,x n ) f 2 (x 1,...,x n ) Network objective: minimize 1 x=b i f i(x i ) f m (x 1,...,x n ) Network objective: minimize x X max 1 i m f i (x) 2

4 General Model Network of m agents represented by an undirected graph ([m], E ) where [m] = {1,..., m} and E is the edge set Each agent i has an objective function f i (x) known to that agent only Common constraint (closed convex) set X known to all agents f The problem can be: 1 (x 1,...,x n ) m minimize f i (x) subject to x X R n or i=1 minimize max 1 i m f i(x) subject to x X R n Distributed Self-organized Agent System f 2 (x 1,...,x n ) f m (x 1,...,x n ) 3

5 How Agents Manage to Optimize minimize m f i (x) subject to x X R n i=1 minimize max 1 i m f i(x) subject to x X R n Each agent i will generate its own estimate x i (t) of an optimal solution to the problem Each agent will update its estimate x i (t) by performing two steps: Consensus-like step (mechanism to align agents estimates toward a common point) Local gradient-based step (to minimize its own objective function) C. Lopes and A. H. Sayed, Distributed processing over adaptive networks, Proc. Adaptive Sensor Array Processing Workshop, MIT Lincoln Laboratory, MA, June A. H. Sayed and C. G. Lopes, Adaptive processing over distributed networks, IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, vol. E90-A, no. 8, pp , A. Nedić and A. Ozdaglar On the Rate of Convergence of Distributed Asynchronous Subgradient Methods for Multi-agent Optimization Proceedings of the 46th IEEE Conference on Decision and Control, New Orleans, USA, 2007, pp A. Nedić and A. Ozdaglar, Distributed Subgradient Methods for Multi-agent Optimization IEEE Transactions on Automatic Control 54 (1) 48-61,

6 Distributed Optimization Algorithm minimize m f i (x) subject to x X R n i=1 At time t, each agent i has its own estimate x i (t) of an optimal solution to the problem At time t + 1, agents communicate their estimates to their neighbors and update by performing two steps: Consensus-like step to mix their own estimate with those received from neighbors m w i (t + 1) = a ij x j (t) (a ij = 0 when j / N i ) j=1 Followed by a local gradient-based step x i (t + 1) = Π X [w i (t + 1) α(t) f i (w i (t + 1))] where Π X [y] is the Euclidean projection of y on X, f i is the local objective of agent i and α(t) > 0 is a stepsize Synchronous Updates 5

7 Drawbacks of synchronous updates All agents see the same time Imagine that each agent has its own clock and it updates at each tick of its clock Preceding algorithm requires that all nodes have synchronized clocks Hard to maintain/ensure for some networks Links have to be perfectly activated to transmit and receive information Communication protocols (require receiving ack-s ) Communications can be costly 6

8 Alternative Communication Network: Gossip Given a static undirected graph G = ([m], E) Each node may wake up with equal probability (1/m) Upon waking up, a node chooses one of its neighbors at random It contacts its neighbor j with probability P ij 7

9 Gossip: Information Exchange Upon agent i establishing connection with agent j, the agents talk They exchange their information Say at time t, i talks to j: i sends x i (t) to j, and j send x j (t) to i They perform some local updates Upon completing the updates: Both agents go to sleep 8

10 Random Gossip for Averaging This is a distributed algorithm for average consensus We consider the asynchronous model that does not require global synchronization of the network A node wakes up and communicates with a randomly selected neighbor Only a pair of nodes update at a given time The algorithm has been developed and analyzed by Boyd, Ghosh, Prabhakar, and Shah

11 Description of the Asynchronous Gossip Scheme Consider an undirected connected network (V, E) with m nodes and a set E of edges. We consider the gossip-based averaging. Each node has its own local Poisson clock that ticks with rate 1. Each node s clock ticks independently of the other nodes clocks At a tick of its clock, a node wakes up and selects randomly one of its neighbors The node and its selected neighbor exchange their information and perform an averaging update Upon update, both nodes go to sleep 10

12 The ticks of the local clocks can be modeled as single virtual Poisson clock that ticks with rate m. Let {Z k } be the Poisson process of the virtual clock tick-times The time is discretized according to {Z k }, since these are the only times when a change occurs in the agent (scalar) values x i (k) The inter-tick times {Z k+1 Z k } are i.i.d. exponentials of rate m Let I k be the index of the node that whose clock ticks at time Z k The variables {I k } are i.i.d. with uniform distribution over {1,..., m} Let J k be the index of a neighbor of I k that is selected randomly with probability p ij ; the matrix P = [p ij ] is row-stochastic 11

13 The nodes I k and J k exchange their current values x i (k) and x j (k) Both nodes update as follows x Ik (k + 1) = 1 ( xik (k) + x Jk (k) ) 2 x Jk (k + 1) = 1 ( xjk (k) + x Ik (k) ) 2 The other nodes do nothing x i (k + 1) = x i (k) for all i / {I k, J k }. NOTE: a value other than 1/2 can be used x Ik (k + 1) = γx Ik (k) + (1 γ)x Jk (k) x Jk (k + 1) = γx Jk (k) + (1 γ)x Ik (k) We will work with 1/2 12

14 The iteration update can be compactly written as x(k + 1) = A(k)x(k) where x(k) is a vector with components x i (k), and the matrix A(k) has the entries a ii (k) = 1, a Ik,J k (k) = a Jk I k (k) = 1 2, and else a ij = 0 Equivalently, A(k) is given by A(k) = W Ik J k, W ij = I 1 2 (e i e j )(e i e j ) T where e i is the unit vector with ith entry 1 and other entries 0 A(k) is a random matrix: A(k) = W ij with probability p ij /m Every realization of A(k) is symmetric and stochastic (hence, doubly stochastic) Furthermore, every realization W ij of A(k) is a projection matrix on the sub-space A matrix A is a projection matrix if A 2 = A S ij = {x R m x i = x j }. 13

15 Convergence and Bounds on Gossip-Averaging Time The random-gossip algorithm converges to the initial average with probability 1: lim x i(t) = 1 m x j (0) for all i, w/p 1. t m j=1 The same is true if the agent variables are vectors x i (0) R n - due to linearity of the gossip averaging process. The averaging time of the gossip algorithm (in terms of the number of clock ticks) is denoted by T ɛ,p and it is defined by { { } } x(k) x1 T ɛ,p = sup inf k Pr ɛ ɛ x(0) x(0) where x is the average of the initial values x 1 (0),..., x m (0). Proposition 5 [Boyd et al. 2006] The averaging time T ɛ,p of the gossip algorithm is bounded as follows: 0.5 ln ɛ 1 ln λ 1 2 (Ā) T 3 ln ɛ 1 ɛ,p ln λ 1 2 (Ā), where Ā = E[A(0)] and λ 2 (A) denotes the second largest absolute eigenvalue of a matrix A. 14

16 Proof Idea Looking at the second moments of the error y(k) = x(k) x1, we get E[ y(k + 1) 2 y(k)] = y(k) T E[A(k) T A(k)]y(k). Since A(k) is symmetric and projection matrix, we have E[A(k) T A(k)] = E[A 2 (k)] = E[A(k)]. Furthermore, the matrices A(k) are identically distributed, so that E[A(k)] = E[A(0)]. By letting Ā = E[A(0)], we have E[ y(k + 1) 2 y(k)] = y(k) T Āy(k). The matrix Ā is symmetric and positive-semidefinite, implying that all eigenvalues of Ā are real and nonnegative Furthermore, Ā is doubly stochastic (so 1 is its eigenvector of its largest eigenvalue 1). Since y(k) 1, it follows that y(k) T Āy(k) λ 2 (Ā) y(k) 2 Therefore E[ y(k + 1) 2 ] = λ 2 (Ā)E[ y(k) 2 ] λ k+1 2 (Ā) y(0) 2. Using y(0) 2 = x(0) 2 m x 2 x(0) 2, we obtain implying that E[ y(k + 1) 2 ] λ k+1 2 (Ā) x(0) 2, E[ y(k) 2 ] λ k x(0) 2 2 (Ā) for all k (1) 15

17 Using Markov s { Inequality, we obtain } { } x(k) x1 y(k) 2 Pr ɛ = Pr x(0) x(0) 2 ɛ2 1 ɛ 2 E[ y(k) 2 ] x(0) 2 This and Eq. (1) yield { x(k) x1 Pr x(0) ɛ } λ k 2 (Ā) After some simple algebra, it follows that for k K(ɛ) { } x(k) x1 Pr ɛ ɛ x(0) thus showing the upper bound on T ɛ,p 3 ln ɛ 1 ln λ 1 2 (Ā), The lower bound is a little more involving (see Boyd et al. 06) 16

18 Further Developments of Gossip Algorithm To improve the averaging time (mixing time of the chain P ) and to reduce the number of expected messages needed for ɛ-averaging, several variants of the gossip algorithm have been proposed In view of the communication cost, these algorithms scale better as a function of the number m of nodes Geographic gossip (Dimakis, Sarwate, and Wainwright 2007) Path-averaging gossip (Benezit, Dimakis, Thiran and Vetterli 2008) For a two-dimensional grid with n nodes (the matrix P reduces to the transition matrix of a random walk on a grid) The expected number of messages is Θ(m 2 ln ɛ 1 ) for gossip Θ(m 3/2 ln ɛ 1 ) for geographic gossip [T ɛ = Θ(m ln ɛ 1 )] Θ(m ln ɛ 1 ) for path-averaging gossip [T ɛ = Θ( m ln ɛ 1 )] 17

19 Gossip-Based Optimization Algorithm Convex problem: minimize m i=1 f i(x) subject to x X Each node has a local clock ticking at Poisson rate of 1 At a tick of its clock, a node wakes up and contacts its neighbor j with probability P ij Equivalent: A single virtual clock ticking at rate m Z t - the time of the t-th tick of the virtual clock Time is discretized according to the intervals [Z t 1, Z t ) - time t I t - the index of agent waking up at time t J t - the index of a neighbor of i contacted by agent i At time t, agents I t and J t exchange information and update The other agents do nothing 18

20 Agent i estimate at time t is x i (t) R n Agents that do not update: x i (t + 1) = x i (t) for i {I t, J t } Update at time t: for i {I t, J t }, v i (t) = 1 2 ( xit (t) + x Jt (t) ) x i (t + 1) = Π X [v i (t) α i (t)( f i (v i (t)) + ɛ i (t))] Agents I t and J t average their values and Each locally minimizes its objective f i at x = v i (t) ɛ i (t) is subgradient error of f i at x = v i (t) 19

21 Alternative Representation Compact representation of agent local-update relations: for all i, m v i (t) = [A(t)] ij x j (t) j=1 x i (t + 1) = Π X [v i (t) α i (t)( f i (v i (t)) + ɛ i (t))]χ {i {It,J t }} + (1 χ {i {It,J t }})v i (t) where A(t) = I 1 ( ) ( ) eit e Jt eit e Jt 2 and χ {i {It,J t }} is the indicator function of the event {i {I t, J t } We will consider two different stepsize selections Frequency-Based Stepsize: a random stepsize given by α i (t) = 1 Γ i (t) where Γ i (t) is the number of times agent i updates up to time t, inclusively Constant Stepsize: each agent chooses a value α i > 0 and uses it throughout the updates α i (t) = α i for all t 20

22 The matrix A(t) has some properties that are very useful in gaining insights into the algorithm s behavior A(t) is symmetric and stochastic - hence, doubly stochastic A(t) is projection matrix: A 2 (t) = A(t) E[A(t)] is positive semidefinite, symmetric, doubly stochastic The second largest eigenvalue of E[A(t)] is λ < 1 when the graph (V, E) is connected 21

23 Assumptions The set X is compact and convex. The functions f i are convex over an open set containing X. As a consequence, the subgradients of each f i are uniformly bounded f i (x) C for all x X and i The errors ɛ i (t) are with zero mean and bounded norm: for all i and t, E[ɛ i (t) F t 1, I t, J t ] = 0, E [ ] ɛ i (t) 2 F t 1, I t, J t ν for some ν > 0 where F t is the history of the method up to time t, F t = {x i 0, i V } {I τ, J τ, ɛ It (τ), ɛ Jt (τ), τ = 0,..., t 1}. 22

24 Key Approximation Result To study the performance of the algorithm, we first introduce the measure of disagreement among the agents estimates m E[ x i (t) ȳ(t) ] i=1 with ȳ(t) = 1 m Proposition: Under the assumption on the errors ɛ i (t), we have m x j (t) If the stepsize is defined through the relative frequency α i (t) = 1, then with Γ i (t) probability 1: t=1 1 t x i(t 1) ȳ(t 1) <, j=1 lim x i (t) ȳ(t) = 0. t If the stepsize is constant, α i (t) = α i > 0, then m E[ x i (t) ȳ(t) ] 2m ᾱ λ t u λ i=1 with u 0 = m i=1 E[ x i(0) ȳ(0) 2 ], and ᾱ = max i α i (C + ν) 23

25 Convergence with Frequency-Based Stepsize Proposition: Consider the random stepsize given by α i (t) = 1 Γ i (t) where Γ i (t) is the number of times agent i updates up to time t, inclusively Assumptions on X, f i s and the errors ɛ i (t) hold Then, the sequences {x i (t)}, i = 1,..., m, converge to the same optimal solution with probability 1 Proof Base Law of the iterated logarithms Γ i (t) tγ i lim = 0, t t 1/2+q where γ i is the probability that agent i updates: N i = {j [m] {i, j} E} γ i = 1 m 1 + P ji j Ni 24

26 Error Estimates: Gossip Communications over Static Network, Constant Stepsize, Strongly Convex Functions Each agent uses its own constant stepsize α i (t) = α i for all t 0 Assumptions on X, f i s and the subgradient errors ɛ i (t) hold Also, assume each f i is strongly convex with σ i There holds 2σ i α i < 1 for each i (each agent independently can ensure) Then, for the sequences {x i (t)}, i = 1,..., m, we have m E [ x i (t) x 2] q t i=1 m j=1 E [ x j (0) x 2] + 2mCC ( )+ γ ( ) X max α i γ i min α j γ ᾱ2 (C + ν) 2 2 2m j 1 q i j 1 q 1 λ + m C X is the diameter of X, q = max 1 i m {1 2γ i α i σ i }, ᾱ i = max i α i, γ = max i γ i, ( γ i is the probability that agent i updates: γ i = ) m j N(i) P ji The effect of network structure: through the eigenvalue λ (second largest for E[W (t)]) The error grows as m in the size of the network (at best) 25

27 Error Estimates: Time-Averaged Iterates Consider the constant stepsize α i (t) = α i Assumptions on X, f i s and the errors ɛ i (t) hold Then, for the time-averages z i (t) = 1 t t τ=1 x i(τ 1) of {x i (t)}, i = 1,..., m, we have lim sup E[f(z i (t))] f + m t 2 (ρ 1) CC X ( 2m +ᾱ (ρ + m) 1 λ + m ) 2 ρ (C + ν) 2, where f is the optimal value of the problem, ᾱ = max i α i, λ is the second largest eigenvalue of E[W (t)], ρ = max i{γ i α i } min i {γ i α i } The error grows as m 3/2 in the size m of the network ρ = 1 has an interesting aspect!!! 26

28 Bottlenecks Gossip-based algorithms may not be efficient in wireless media The main limitations include Extra overhead associated with the knowledge of its own location and the location of its neighbors Geographic gossip and path-averaging gossip increase the diversity of pairwise exchanges but give rise to congestion issues associated with sending messages along long routes Routing protocols require bidirectional information exchange and setting-up a two-way route successfully aggravates the problem The wireless media is inherently broadcast At cost of one transmission, several terminals can be reached All nodes in a range can update without complex routing and pairwise exchange operations 27

29 Broadcast-Based Algorithm We consider an asynchronous distributed algorithm that uses the random broadcast scheme as a mechanism to diffuse the information over the network asynchronously Broadcast-communication: T.C. Aysal, M.E. Yildriz, A.D. Sarwate, and A. Scaglione, Broadcast gossip algorithms for consensus, IEEE Transactions on Signal processing 57 (2009),

30 Random Broadcast Communications The communication graph G = ([m], E) is connected and undirected The actual links that are used at any instance of time are virtually directed Each node has a local clock ticking at Poisson rate of 1; equivalent to having a single virtual clock ticking at rate m At a tick of its clock, a node wakes up and broadcasts its information Z t - the time of the t-th tick of the virtual clock Time is discretized according to the intervals [Z t 1, Z t ) - time t I t - the index of agent waking up at time t; N It - the set of neighbors of I t At time t, agents in the set N It receive information and update The other agents do nothing - including the agent that broadcasted its information 29

31 Broadcast-Based Algorithm m minimize f i (x) subject to x X R n Agent i estimate at time t is x i (t) R n At time t, agent I t wakes up and broadcasts its estimate x i (t) Agent I t and j N It do not update: i=1 x j (t+1) = x j (t) for j {I t } N It Neighbors j N It of agent I t update: v j (t) = βx It (t) + (1 β)x j (t) x j (t + 1) = Π X [v j (t) α j (t)( f j (v j (t)) + ɛ j (t))] Each locally minimizes its objective f j at x = v j (t) ɛ j (t) is subgradient error of f j at x = v j (t) 30

32 Results Similar convergence statements and rate estimates can be obtained as for the random-gossip based algorithm AN Asynchronous Broadcast-Based Convex Optimization over a Network TAC,