Accelerated Dual Descent for Constrained Convex Network Flow Optimization

Transcription

1 52nd IEEE Conference on Decision and Contro December 10-13, Forence, Itay Acceerated Dua Descent for Constrained Convex Networ Fow Optimization Michae Zargham, Aejandro Ribeiro, Ai Jadbabaie Abstract We present a fast distributed soution to the capacity constrained convex networ fow optimization probem. Our soution is based on a distributed approximation of Newtons method caed Acceerated Dua Descent (ADD). Our agorithm uses a parameterized approximate inverse Hessian, which is computed using matrix spitting techniques and a Neumann series truncated after N terms. The agorithm is caed ADD-N because each update requires information from N-hop neighbors in the networ. The parameter N characterizes an expicit trade off between information dependence and convergence rate. Numerica experiments show that even for N1 and N2, ADD-N converges orders of magnitude faster than subgradient descent. I. INTRODUCTION The goa of this paper is to deveop a fast distributed soution to the capacity constrained convex networ fow optimization probem. Soutions to minimum cost networ fow probems have ong been used in operations research and transportation networs [1], [2]. The semina resuts of Rocafear in [3] tie the networ fow probem to min cut and other combinatoria probems such as shortest path, [4][Chapter 1]. Networ fow probems aso arise in routing probems for communication networs, [5] where there is a need for distributed soutions. For exampe, soving a networ fow probem is ey subprobem in the wireess routing and resource aocation probem in [6]. Dua subgradient descent is a distributed agorithm used to sove the convex networ fow optimization. Anaysis of subgradient methods for convex optimization probems can be found in [7] and [8] with the atter taing into account uncertainty in the networ structure. However, practica appicabiity of subgradient type agorithms is imited by exceedingy sow convergence rates, [9] and [10]. An aternative distributed agorithm based on the Gauss Seide method is present in [11]. Lie gradient descent, Gauss Seide is a first order method. Faster methods such as dua Newton s method, require goba information and the existence of a dua Hessian. The Acceerated Dua Descent (ADD) method is a distributed agorithm requiring information from an N-hop neighborhood in order to update the dua variabes, [12], [13]. The ADD method is based on Newtons Method and therefore requires the existence of the dua Hessian. We show that capacity constraints resut in a nonsmooth dua function. This research is supported by Army Research Lab MAST Coaborative Technoogy Aiance, AFOSR compex networs program, ARO P NS, NSF CAREER CCF , and NSF CCF , ONR MURI N and NSF-ECS Michae Zargham, Aejandro Ribero and Ai Jadbabaie are with the Department of Eectrica and Systems Engineering, University of Pennsyvania. Therefore, there are points where dua Hessian does not exist and Newton type methods such as ADD cannot be directy appied. In this wor we prove the existence of a generaized dua Hessian. We compute the generaized dua Hessian via oca information and use it to impement the ADD-N agorithm. Using ADD-N with the generaized Hessian is proven to resut in a descent direction guaranteeing convergence of the agorithm. We begin the paper in Section II with the forma definition of the networ fow optimization probem considered here. Networ connectivity is modeed as a directed graph and the goa of the networ is to support a singe information fow specified by incoming rates at an arbitrary number of sources and outgoing rates at an arbitrary number of sins. Using the capacity constraints to restrict to positive vaued fows recovers the standard impementation of the networ fow optimization probem in [4], [14]. Each edge of the networ is associated with a convex function that determines the cost of transmitting a unit of fow across it. The objective is to find the fows that minimize the sum of in costs by soving a convex optimization probem with inear constraints. In Section II-A, we show that the dua probem is nonsmooth and we prove the existence of a generaized dua Hessian consistent with [15]. In Section III, we construct the ADD-N update using a Neumann expansion for the Hessian inverse, [16][Section 5.8] truncated after N terms. Since our inverse Hessian approximation is an N degree matrix poynomia our descent direction can be computed using N- hop neighbor information. In Section III-A, we demonstrate how the ADD-N direction can be impemented using N one hop exchanges per dua. In Section IV, the ADD agorithm is shown to exhibit faster convergence rate reative to subgradient descent in numerica experiments. II. CONSTRAINED NETWORK OPTIMIZATION Consider a networ represented by a directed graph G (N, E) with node set N {1,..., n}, and edge set E {1,..., E}. The networ is depoyed to support a singe information fow specified by incoming rates b i > 0 at source nodes and outgoing rates b i < 0 at sin nodes. Rate requirements are coected in a vector b, which to ensure probem feasibiity has to satisfy n i1 bi 1. Our goa is to determine a fow vector x [x e ] e E, with x e denoting the amount of fow on edge e (i, j). Fow conservation impies that it must be Ax b, with A the n E node-edge incidence /13/$ IEEE 1037

2 matrix defined as 1 if edge j eaves node i, [A] ij if edge j enters node i, 0 otherwise. We define the reward as the negative of a sum of convex cost functions φ e (x e ) denoting the cost of x e units of fow traversing edge e. We assume that the cost functions φ e are stricty convex, surjective and twice continuousy differentiabe on the interva c e x e c e u where c e 0 and c e u 0 are the capacity constraints on edge e. The direction of fow can be forced to match the direction of the edge by choosing c e 0. The networ fow optimization probem is then defined as min subject to: f(x) E e1 φ e(x e ) Ax b x c u. Keeping the capacity constraints and duaizing the equaity constraint, we have the partia dua (1) E q(λ) min x φ e (x e ) + λ (Ax b) (2) e1 subject to: x c u. We separate this equation into an edge dimensiona sum of optimization probems q(λ) λ b + E e1 min x φ e(x e ) + x e ( λ e ) (3) subject to: c e x e c e u where λ e λ j λ i where e (i, j). Each one of these optimization probems is a constrained convex optimization oca to one edge aowing us to write the prima optimizers expicity x e ( λ e ) [ ] d dx e φe ( λ e ) We are therefore interested in soving c e u c e. (4) λ arg max q(λ) (5) λ and computing the prima variabes x x(λ ) which are guaranteed to be optima by strong duaity via Sater s condition, [10]. Considering (2), we can compute the gradient g(λ) q(λ) in terms of the prima optimizers, g(λ) Ax(λ) b (6) where x(λ) denotes the vector of x e ( λ e ). The projection in (4) causes (6) to become a non-smooth function. This wor focuses on addressing this non-smoothness. A. Non-smooth Newton Method Consider an index, an arbitrary initia vector λ 0 and define iterates λ generated by the foowing recursion λ +1 λ + α d for a 0, (7) where d is an ascent direction satisfying g d > 0, g g(λ ) and α is a given stepsize sequence. The Newton method is obtained by maing d in (7) the Newton step, g H d. The Newton step is expicity given by d H g for any where the dua Hessian H exists 1. To obtain an expression for the dua Hessian, consider given dua λ and prima x x(λ ) variabes, and consider the second order approximation of the prima objective centered at the current prima iterates x, ˆf(y) f(x ) + f(x ) (y x ) (8) (y x ) 2 f(x )(y x ). The prima optimization probem in (1) is now repaced by the minimization of the approximating function ˆf(y) in (8) subject to the constraints Ay b and y c u. This approximated probem is a quadratic program whose dua is a piece-wise quadratic function ˆq(λ ) 1 2 λ A [ 2 f(x ) A ] cu λ + p λ c + r (9) 1 [ λ 2 A 2 f(x ) ] c u 2 f(x c ) [ 2 f(x ) A ] cu λ The vector p and the constant r can be expressed in cosed form as functions of f(x ) and 2 f(x ), but they are irreevant for the discussion here. When ˆf(x ) is centered at x x(λ ), we have ˆf(x(λ )) f(x(λ )) and it foows that ˆq(λ ) q(λ ) for any λ where the impicit prima optimizers x x(λ ) are used. The important consequence of (9) is that we can compute the dua Hessian. When the f(x ) A λ c u, the projection is inactive and we recover the Hessian H A 2 f(x ) A (10) discussed in [13]. We define the set of unsaturated edges U {e E c e < x e (λ ) < c e u}. (11) The dua Hessian 2 q(λ ) is dependent on which edges e are unsaturated, i.e. e U. When edge e (i, j) saturates, we have that the ith eement of the gradient g i (λ ) is unaffected by changes in λ j so g(λ )/ λ j 0. Thus the Hessian eements [H ] i,j [H ] j,i are zero. We define the off diagona eements of the generaized Hessian H 2 q(λ ), according to 2 { q(λ ) [A λ j 2 f(x ) A ] ij if (i, j) U (12) λi 0 if (i, j) U 1 H denotes the Moore-Penrose pseudoinverse of the dua function s Hessian H H(λ ) and g 1 for a. 1038

3 and the diagona eements are the sums of the off diagonas [H ] ii j:(i,j) U [A 2 f(x ) A ] ij. (13) From the definition of f(x) in (1) it foows that the prima Hessian 2 f(x ) is a diagona matrix, which is positive definite by strict convexity of f(x). Therefore, its inverse exists and can be computed ocay. Further observe that L A 2 f(x ) A is a weighted version of the networ graph s Lapacian. One consequence of its Lapacian form is that H L is negative semi-definite. Another is that 1 is an eigenvector of H associated with eigenvaue 0 and that H is invertibe on the subspace 1. B. Existence of the Generaized Hessian We sti need to understand how the projection in (4) and resuting non-smoothness in (6) impacts the existence of this dua Hessian. Theorem 1. Properties of the Dua Gradient: 1) The function g : R n R n is Lipschitz Continuous. g(λ) g( λ) L λ λ λ, λ R n (14) 2) The function g : R n R n is Semi-smooth: im {V h} Exists h V g(λ+th),t 0 Rn (15) where g(λ) is the generaized Hessian defined { } g(λ) co im g( λ) λ λ, λ D g (16) where co{ } denotes convex hu and D g is the set of points on which g(λ) is differentiabe. Proof: From our assumptions on f(x) in Section II we now that φ e (x e ) is stricty convex, surjective and twice continuousy differentiabe, therefore the function [ d dx φ e] ( ) e is continuous. We denote the vector of these functions f ( ).The prima optima variabes are computed according to (4), the saturation operator ce u c e preserves continuity but the function is no onger differentiabe. Appying (6), we observe from g(λ λ) A ( x(λ) x( λ) ) (17) that continuity of x(λ) is sufficient for continuity of g(λ). In order to show g(λ) is Lipschitz we substitute the definition of x(λ) from (4). g(λ) g( λ) A( f ( λ) cu f ( λ) cu ) (18) Since f(x) is twice continuousy differentiabe and surjective, it foows that f is continuousy differentiabe. Let y(x) f (x) which means f(y(x)) x by the definition of the function inverse. We tae the gradient of this expression with respect to x yieding 2 f(y(x)) y(x) 1 by the chain rue. Since f(x) is stricty convex we have an expression for the gradient of f y(x) [ f ](x) [ f(y(x))] 1. (19) f (c ) γ c u c -1 f (x) f (c ) Fig. 1. The constant γ is the steepest gradient point of the function f (x) on its active domain, thus maing it a property of the prima objective f(x) and the capacity constraints. This image simpifies the probem to 1 dimension to mae it visuaizabe. Now we wi find the steepest possibe sope of f γ max y(x) (20) s.t. f( ) x f(c u ). We need not consider any x outside of the set J { f( ) x f(c u )} because using that fact that f is monotonic increasing we now that any such x yieds f (x) or f (x) c u. We can guarantee the existence of γ as defined in (20) by restricting the domain of y to the set J because any continuous function from a compact space into a metric space is bounded. Using (20) we can concude that ( f ( λ) cu f ( λ) cu ) γ λ λ (21) Recaing that λ Aλ for any λ and subsituting (21) bac into (18) we have g(λ) g( λ) γ A 2 λ λ (22) which competes the proof of part 1) with L γ A 2. To continue with part 2), we consider the points for which g(λ) is not differentiabe { [ d S λ : dx φ e] } e ( λe ) c e for any e or [ d dx φ e]. (23) e ( λe ) c e u, for any e For any λ S, we have g(λ) {H(λ)} as defined in (12) and (13) because g(λ) is differentiabe at those points. The imit as we approach a point in S depends which edges are saturating and on whether the direction of the imit is from inside or outside of the saturation region. From within the saturation region, λ {λ R n (i, j) U } [ ] im λ λ,λ R n S g( λ) 0 (24) but when outside the saturation region λ {λ R n (i, j) U } [ ] im λ λ,λ R n S g( λ) [H ] ij (25) ij ij u x 1039

4 with H as defined in (10). The set of generaized Hessians g(λ) defined in (16) aows [ g(λ)] i,j [[H ] ij, 0] (26) for any edge e (i, j) for which x e (λ) c e or x e (λ) c e u. By our construction of the set g(λ) in (24)-(26), for any matrix V g(λ) incuding λ S, the expression V h exists for a h R n. We concude that (15) hods, competing the proof. Theorem 1 characterizes the dua gradient g(λ), providing a set of generaized Hessian matrices at the points where g(λ) is not differentiabe. In our proof, equation, (26) it is shown that our choice of H(λ) defined in (12) and (13) is an extreme point in the set of generaized Hessians g(λ). Whie we have shown that the entire set of generaized Hessians exists our agorithm uses this extreme point for its direct reationship to the Hessian in the unconstrained (smooth) version of networ fow optimization probem anayzed in [13]. Theorem 1 aso specifies that the dua gradient g(λ) is Lipshitz continuous which is a buiding boc for our convergence anaysis. Theorem 2. Suppose g(λ) is semi-smooth at λ and a V g(λ) are non-singuar and g(λ) is Lipshitz continous, then the λ +1 λ V g(λ ) (27) is we defined and gobay convergent to the unique soution λ of g(λ) 0 (28) at a q-superinear rate. Theorem 2 is Newton s Method for piecewise quadratic functions taen from [15][Theorem 1.1]. It states that we can sove a peicewise quadratic optimization probem such as the one we have in (9) using Newton s method as ong as the generaized Hessian exists and is fu ran. In our case, it converges as ong as the inverse of the prima Hessian 2 f(x ) is fu ran and the reduced graph Ĝ {V, U } remains connected. Unfortunatey, we cannot guarantee Ĝ wi remain connected. However, the approximate Newton method, caed ADD deveoped in [13] when appied with an enhance spitting technique, aows us to sidestep this concern. Furthermore, unie the true Newton method, can be computed using ony oca information. III. NON-SMOOTH ACCELERATED DUAL DESCENT The ADD-N agorithm uses matrix spitting to generate an approximation of the Newton direction requiring information from no more than N hops away. We consider a finite number of terms of a suitabe Tayor s expansion representation of the Newton direction. At, spit the Hessian H B D, with diagona eements D and off diagona eements B where both D and B are nonnegative eementwise. To hande potentia singuarities we appy an enhanced spitting H B where 2D + I and B D + I + B (29) observing that is a sti a diagona matrix and B retains the structure of a weighted adjacency. Using our spitting, we can define the approximate Hessian inverse as defined in [13], H N r0 N r0 1 2 ( ( 1 2 ) B 1 r B )r (30) which expoits the Neumann series for the inverse of a matrix of the form (I X). Given the approximate Hessian inverse our approximate Newton direction is given H d where the acceerated dua descent is H g (31) λ +1 λ + α d (32) and α is the step size at. The Nth order approximation H adds a term of the form ( )N B to the N 1st order approximation. The sparsity pattern of this term is that of B N, which coincides with the N-hop neighborhood because it is the N th power of a weighted adjacency matrix. Thus, computation of the oca eements of the Newton step necessitates information from N hops away; see [13] for further detais. We thus interpret (31) as a famiy of approximations indexed by N that yieds Hessian approximations requiring information from N-hop neighbors in the networ. A. Agorithm Impementation The direct impementation of the dua update described in (32) can be computationay cumbersome. In practice we impement ADD-N as an N steps of the consensus d (r+1) Bd (r) + g (33) where the is started with d (0) g which foows ceary from (30). Further expanation of the equivaence of ADD-N and N step consensus methods can be found in [13]. Our practica impementation of ADD-N is given in Agorithm 1. We assume that node i eeps trac of the i th eement of node dimensiona variabes such as λ and the nonzero eements of the i th row of matrices such as H and the spitting matrices. We aso assume that the edge variabes such as x e are observabe by both nodes i and j ined by edge e (i, j). With this framewor, steps 3 and 10 are the ony ones which require communication with neighbors one hop neighbors ony. Therefore, each fu λ +1 λ costs N + 1 oca information exchanges. B. Convergence Theorem 3. Agorithm 1 with fixed step size α α > 0 sma enough converges gobay to the optima dua variabe λ for the dua probem in (9) and x x(λ ) is the optima soution to the origina constrained networ fow optimization probem, (1). 1040

5 Agorithm 1: Acceerated Dua Descent. 1 Initiaize dua: λ 0 2 for 0, 1, 2,... do 3 Compute differentias: λ Aλ 4 Update Primas: x f ( λ ) cu 5 Compute Constraint Vioation: g Ax b 6 Compute the Generaized Hessian: { 1/φ e (x [H ] ij e ) if e (i, j) U 0 ese f(x) Prima Objective 0.4 Proj. Subgrad. ADD ADD [H ] ii j [H ] ij 10 2 Prima Feasibiity 7 Compute Spitting: I 2 diag(h ) and B + H 8 Initiaize direction: d (0) 9 for r 0, 1,..., N 1 do 10 Update direction: d (r+1) 11 end g 12 Update dua: λ +1 λ + α d 13 end Bd (r) + g Proof: As per the Descent Lemma [17][A.24], it is sufficient to show that the approximate Hessian inverse H is negative definite for a to guarantee that the ADD-N defined in (31) and (32) converges to the optima dua variabe λ for some fixed step size α. We consider the individua terms of H N r0 T r as defined in (30). The matrix D is diagona positive semi-definite, therefore 2D + I is diagona and positive definite. Using the fact that L H is a Lapacian from (12) and (13) and the basic spitting L D B, the matrix B D +I+B is stricty diagonay dominant. Furthermore, by construction B is a nonnegative symmetric matrix. A symmetric, nonnegative, stricty diagonay dominant matrix is positive definite [16][Section 6.1]. We can concude that each term T r /2 /2 ( B /2 ) r /2 (34) is positive definite because its symmetric and a product of positive definite matrices. Since each term T r is positive definite the finite sum N r0 T r is positive definite, so from (30) we get that H is negative definite and we concude that the dua converges to λ. The prima probem (1) satisfies Sater s condition for strong duaity [10][Section 5.2.3], therefore there is a zero duaity gap, indicating that the prima optima x is the argument of the minimization in (2) when λ λ. Thus x x(λ ) computed according to (4) is the optima prima variabe, competing the proof. Theorem 3 guarantees that the ADD-N agorithm converges to the optima prima and dua variabes given an appropriatey chosen fixed step size. A reativey oose bound on the aowabe step size can be determined by considering the Ax b Fig. 2. ADD-1 and ADD-2 out perform projected subgradient by an order of magnitude on a sparse networ with 20 nodes and 35 edges with capacity constraints Descent Lemma [17][A.24]. In our case such a bound wi be inversey proportiona to the Lipschitz constant L defined in Theorem 1. In practice, the appropriate step size scae can be determined by tuning. We aso mae no theoretica caim about the convergence rate, which appears in practice to be consistent with the resuts in [13]. IV. NUMERICAL EXPERIMENTS Numerica experiments were run on a variety of probems ranging in scae from 10 to 100 nodes and 9 to 500 edges. Two types of networs were considered, uniformy randomy generated networs and worst case scenario ine graphs where one node is a sin and a other nodes are sources. The stricty convex objective function φ e (x e ) exp(x e ) + exp( x e ) can be thought of a prohibitive cost of heaviy oading a singe in which encourages the networ to mae use of a of its resources. Across the board we observe resuts consistent with the unconstrained probem in [13]. The ADD-N agorithm converges one to two orders of magnitude faster than the subgradient descent type agorithms for probems arge and sma, sparse and dense. Figure 2 is a typica exampe of the convergence behavior on a sparse networ. Sparse probems tae consideraby ong to sove due to sower mixing rates for the information spreading matrix B. Mixing rates of marov chains are discussed extensivey in [18]. In Figure 3 there is a significant improvement in convergence rate, the 1041

6 Ax b f(x) Prima Objective Prima Feasibiity Proj. Subgrad. ADD ADD Fig. 3. ADD-1 and ADD-2 out perform projected subgradient by an order of magnitude on a denser networ with 20 nodes and 100 edges with capacity constraints time required to reach a feasibiity of 10 4 goes from 800 s in the first exampe to ony 100. V. CONCLUSION The Acceerated Dua Descent (ADD) agorithm has been generaized to the capacity constrained networ fow optimization probem, through proof and computation of a generaized dua Hessian. This improvement expands the appicabiity of the ADD method. In particuar, combining the resuts for the capacitated networ fow probem with the stochastic generaization in [19] opens the door to networing appications via the bacpressure formuation, [20], [21]. We have used the ADD as a foundation for Acceerated bacpressure [22] which stabiizes queues in capacitated muti commodity communication networs with stochastic pacet arriva rates. In this wor, simuations demonstrated that N 1 and N 2, respectivey denoted as ADD-1 and ADD-2 perform best in practice. ADD-1 corresponds to Newton step approximations using information from neighboring nodes ony, whie ADD-2 requires information from nodes two hops away. Whie we observe a per improvement when increasing N, we do so at the cost of additiona communication. Since further increases in N yied diminishing returns with respect to convergence rate, we recommend the use of ADD-1 and ADD-2 which outperform gradient descent by upwards of two orders of magnitude. [2] E. Miandoabchi R. Farahani. Graph Theory for Operations Research and Management. IGI Goba, [3] R. T. Rocafear. Networ Fows and Monotropic Programming. Wiey, New Yor, NY, [4] D. P. Bertseas. Networ Optimization: Continuous and Discrete Modes. Athena Scientific, Bemont, MA, [5] Micha Piro and Deepanar Medhi. Routing, Fow, and Capacity Design in Communication and Computer Networs. Esevier, [6] Lin Xiao, M. Johansson, and S.P. Boyd. Simutaneous routing and resource aocation via dua decomposition. IEEE Transactions on Communications, 52(7): , [7] A. Nedic and A. Ozdagar. Distributed subgradient methods for mutiagent optimization. IEEE Transactions on Automatic Contro, 54, [8] I. Lobe and A. Ozdagar. Distributed subgradient methods for convex optimization over random networs,. IEEE Transactions on Automatic Contro, 56, [9] N. Z. Shor. Minimization Methods for Non-differentiabe Functions. Springer, [10] S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, Cambridge, UK, [11] Dimitri P. Bertseas and Didier E Baz. Distributed asynchronous reaxation methods for convex networ fow probems. SIAM Journa of Optimization and Contro, [12] M. Zargham, A. Ribeiro, A. Ozdagar, and A. Jadbabaie. Acceerated dua descent for networ optimization. In Proceedings of IEEE ACC, [13] M. Zargham, A. Ribeiro, A. Ozdagar, and A. Jadbabaie. Acceerated dua descent for networ optimization. IEEE Transactions on Automatic Contro, (submitted). [14] A. V. Godberg, E. Tardos, and R. E. Tarjan. Networ Fow Agorithms. Springer-Verag, Berin, [15] J. Sun and H. Kuo. Appying a newton method to stricty convex separabe networ quadratic programs. SIAM Journa of Optimization, 8, [16] R. Horn and C. R. Johnson. Matrix Anaysis. Cambridge University Press, New Yor, [17] D.P. Bertseas. Noninear Programming. Athena Scientific, Cambridge, Massachusetts, [18] S. Boyd, A. Ghosh, B. Prabhaar, and D. Shah. Gossip agorithms: Design, anaysis, and appications. In Proceedings of IEEE INFOCOM, [19] M. Zargham, A. Ribeiro, and A. Jadbabaie. Networ optimization under uncertainty. Proceedings of IEEE CDC, [20] L. Tassiuas and A. Ephremides. Stabiity properties of constrained queueing systems and scheduing poicies for maximum throughput in mutihop radio networs. IEEE Transactions on Automatic Contro, 37: , [21] M. J. Neey. Stochastic Networ Optimization with Appication to Communication and Queueing Systems. Morgan and Caypoo, [22] M. Zargham, A. Ribeiro, and A. Jadbabaie. Acceerated bacpressure agorithm. ArXiv: REFERENCES [1] J. B. Orin. Minimum convex cost dynamic networ fows. MATHE- MATICS OF OPERATIONS RESEARCH, May