A SIMPLE PARALLEL ALGORITHM WITH AN O(1/T ) CONVERGENCE RATE FOR GENERAL CONVEX PROGRAMS

Transcription

1 A SIMPLE PARALLEL ALGORITHM WITH AN O(/T ) CONVERGENCE RATE FOR GENERAL CONVEX PROGRAMS HAO YU AND MICHAEL J. NEELY Abstract. This paper considers convex programs with a general (possibly non-differentiable) convex objective function and Lipschitz continuous convex inequality constraint functions. A simple algorithm is developed and achieves an O(/t) convergence rate. Similar to the classical dual subgradient algorithm and the ADMM algorithm, the new algorithm has a parallel implementation when the objective and constraint functions are separable. However, the new algorithm has a faster O(/t) convergence rate compared with the best known O(/ t) convergence rate for the dual subgradient algorithm with primal averaging. Further, it can solve convex programs with nonlinear constraints, which cannot be handled by the ADMM algorithm. The new algorithm is applied to a multipath network utility maximization problem and yields a decentralized flow control algorithm with the fast O(/t) convergence rate. Key words. convex programs, parallel algorithms, convergence rates AMS subject classifications. 90C5, 90C30. Introduction. Fix positive integers n and m. Consider the general convex program: () () (3) minimize subject to f(x) g k (x) 0, k {,,..., m}, x X, where set X R n is a closed convex set; function f(x) is continuous and convex on X ; and functions g k (x), k {,,..., m} are convex and Lipschitz continuous on X. Note that the functions f(x), g (x),..., g m (x) are not necessarily differentiable. Denote the stacked vector of multiple functions g (x), g (x),..., g m (x) as g(x) = [ g (x), g (x),..., g m (x) ] T. The Lipschitz continuity of each gk (x) implies that g(x) is Lipschitz continuous on X. Throughout this paper, we use to denote the vector Euclidean norm and Lipschitz continuity is defined with respect to the Euclidean norm. The following assumptions are imposed on the convex program ()-(3): Assumption (Basic Assumptions). There exists a (possibly non-unique) optimal solution x X that solves the convex program ()-(3). There exists a constant β such that g(x ) g(x ) β x x for all x, x X, i.e., the Lipschitz continuous function g(x) has modulus β. Assumption (Existence of Lagrange Multipliers). The convex program ()-(3) has Lagrange multipliers attaining the strong duality. That is, there exists a Lagrange multiplier vector λ = [λ, λ,..., λ m] T 0 such that q(λ ) = min x X { f(x) : g k (x) 0, k {,,..., m} where q(λ) = min x X {f(x) + m k= λ kg k (x)} is the Lagrangian dual function of problem ()-(3). Using the methodology initiated in the current paper, we further developed a different primaldual type algorithm with the same O(/t) convergence rate in the extended work [4]. Department of Electrical Engineering, University of Southern California, Los Angeles, CA (yuhao@usc.edu, mjneely@usc.edu). },

2 Assumption is a mild assumption. For convex programs, Assumption is implied by the existence of a vector s X such that g k (s) < 0 for all k {,..., m}, called the Slater condition [, 5]. However, there are convex programs where Assumption holds but the Slater condition does not hold... New Algorithm. Consider the following algorithm described in Algorithm. The algorithm computes vectors x(t) X for iterations t {0,,,...}. Define the average over the first t > 0 iterations as x(t) = t t x(τ). The algorithm uses an initial guess vector that is represented as x( ) and that is chosen as any vector in X. The algorithm also uses vector variables Q(t) = [ Q (t),..., Q m (t) ] T in the computations. The main result of this paper is that, whenever the parameter α is chosen to satisfy α β /, the vector x(t) closely approximates a solution to the convex program and has an approximation error that decays like O(/t). Algorithm Let α > 0 be a constant parameter. Choose any x( ) X. Initialize Q k (0) = max{0, g k (x( ))}, k {,,..., m}. At each iteration t {0,,,...}, observe x(t ) and Q(t) and do the following: Choose x(t) as x(t) = argmin x X Update virtual queues via { f(x) + [ Q(t) + g(x(t )) ] T g(x) + α x x(t ) }, Q k (t + ) = max{ g k (x(t)), Q k (t) + g k (x(t))}, k {,,..., m}. Update the averages x(t) via x(t + ) = t + t t x(τ) = x(t) t + + x(t) t +. The variables x(t) are called primal variables. The variables Q(t) can be viewed as dual variables because they have a close connection to Lagrange multipliers. The variables Q(t) shall be called virtual queues because their update rule resembles a queueing equation. The virtual queue update used by Algorithm is related to traditional virtual queue and dual variable update rules. However, there is an important difference. A traditional update rule is Q k (t + ) = max{q k (t) + g k (x(t)), 0} [3, 6]. In contrast, the new algorithm takes a max with g k (x(t)), rather than a max with 0. The primal update rule for x(t) is also new and is discussed in more detail in the next subsection. Algorithm has the following desirable property: If the functions f(x) and g(x) are separable with respect to components or blocks of x, then the primal updates for x(t) can be decomposed into several smaller independent subproblems, each of which only involves a component or block of x(t)... The Dual Subgradient Algorithm and the Drift-Plus-Penalty Algorithm. The dual subgradient algorithm is a well known iterative technique that approaches optimality for certain strictly convex problems [3]. A modification of the dual subgradient algorithm that averages the resulting sequence of primal estimates

3 is known to solve general convex programs (without requiring strict convexity) and provides an O(/ t) convergence rate [5, 3, 7], where t is the number of iterations. Work [] improves the convergence rate of the dual subgradient algorithm to O(/t) in the special case when the objective function f(x) is strongly convex and second order differentiable and constraint functions g k (x) are second order differentiable and have bounded Jacobians. A recent work in [3] shows that the convergence rate of the dual subgradient algorithm with primal averaging is O(/t) without requiring the differentiability of f(x) and g k (x) but still requiring the strong convexity of f(x). (Further improvements are also possible under more restrictive assumptions, see [3].) The dual subgradient algorithm with primal averaging is also called the Drift-Plus-Penalty (DPP) algorithm. This is because it is a special case of a stochastic optimization procedure that minimizes a drift expression for a quadratic Lyapunov function [6]. One advantage of these dual subgradient and drift approaches is that the computations required at every iteration are simple and can yield parallel algorithms when f(x) and g k (x) are separable. Algorithm developed in the current paper maintains this simplicity on every iteration, but provides fast O(/t) convergence for general convex programs, without requiring strict convexity or strong convexity. For example, the algorithm has the O(/t) convergence rate in the special case of linear f(x). Algorithm is similar to a DPP algorithm, or equivalently, a classic dual subgradient algorithm with primal averaging, with the following distinctions:. The Lagrange multiplier ( virtual queue ) update equation for Q k (t) is modified to take a max with g k (x(t)), rather than simply project onto the nonnegative real numbers.. The minimization step augments the Q k (t) weights with g k (x(t )) values obtained on the previous step. These g k (x(t )) quantities, when multiplied by constraint functions g k (x), yield a cross-product term in the primal update. This cross term together with another newly introduced quadratic term in the primal update can cancel a quadratic term in an upper bound of the Lyapunov drift such that a finer analysis of the drift-plus-penalty leads to the fast O(/t) convergence rate. 3. A quadratic term, which is similar to a term used in proximal algorithms [9], is introduced. This provides a strong convexity pushback. The pushback is not sufficient to alone cancel the main drift components, but it cancels residual components introduced by the new g k (x(t )) weight..3. The ADMM Algorithm. The Alternating Direction Method of Multipliers (ADMM) is an algorithm used to solve linear equality constrained convex programs in the following form: (4) (5) (6) minimize f (x) + f (y) subject to Ax + By = c, x X, y Y. In [5, 3, 7], the dual subgradient algorithm with primal averaging is proven to achieve an ɛ-approximate solution with O(/ɛ ) iterations by using an O(ɛ) step size. We say that the dual subgradient algorithm with primal averaging has an O(/ t) convergence rate because an algorithm with O(/ t) convergence requires the same O(/ɛ ) iterations to yield an ɛ-approximate solution. However, the dual subgradient algorithm in [5, 3, 7] does not have vanishing errors as does an algorithm with O(/ t) convergence.

4 Define the augmented Lagrangian as L ρ (x, y, λ) = f (x) + f (y) + λ T ( Ax + By c ) + ρ Ax + By c. At each iteration t, the ADMM algorithm consists of the following steps: Update x(t) = argmin x X L ρ (x, y(t ), λ(t )). Update y(t) = argmin y Y L ρ (x(t), y, λ(t )). Update λ(t) = λ(t ) + ρ ( Ax(t) + By(t) c ). Thus, the ADMM algorithm yields a distributed algorithm where the updates of x and y only involve local sub-problems and is suitable to solve large scale convex programs in machine learning, network scheduling, computational biology and finance [4]. The best known convergence rate of ADMM algorithm for convex program with general convex f ( ) and f ( ) is recently shown to be O(/t) [6, 9]. An asynchronous ADMM algorithm with the same O(/t) convergence rate is studied in []. Note that we can apply Algorithm to solve the problem (4)-(6) after replacing the equality constraint Ax + By = c by two linear inequality constraints Ax + By c and Ax + By c. It can be observed that the algorithm yielded by Algorithm is also separable for x and y. In fact, the updates of x and y in Algorithm are fully parallel while the ADMM algorithm updates x and y sequentially. The remaining part of this paper shows that the convergence rate of Algorithm is also O(/t). However, a significant limitation of the ADMM algorithm is that it can only solve problems with linear constraints. In contrast, Algorithm proposed in this paper can solve general convex programs with non-linear constraints..4. Decentralized Multipath Network Flow Control Problems. Section 4 presents an example application to multipath network flow control problems. The algorithm has a queue-based interpretation that is natural for networks. Prior work on distributed optimization for networks is in [, 5,, 3, ]. It is known that the DPP algorithm achieves O(/ t) convergence for general networks [5], and several algorithms show faster O(/t) convergence for special classes of strongly convex problems [, 3]. However, multipath network flow problems fundamentally fail to satisfy the strong convexity property because they have routing variables that participate in the constraints but not in the objective function. The algorithm of the current paper does not require strong convexity. It easily solves this multi-path scenario with fast O(/t) convergence. The algorithm also has a simple distributed implementation and allows for general convex but nonlinear constraint functions.. Preliminaries and Basic Analysis. This section presents useful preliminaries in convex analysis and important facts of Algorithm... Preliminaries. Definition (Lipschitz Continuity). Let X R n be a convex set. Function h : X R m is said to be Lipschitz continuous on X with modulus L if there exists L > 0 such that h(y) h(x) L y x for all x, y X. Definition (Strongly Convex Functions). Let X R n be a convex set. Function h is said to be strongly convex on X with modulus α if there exists a constant α > 0 such that h(x) α x is convex on X. In fact, we do not need to replace the linear equality constraint with two inequality constraints. We can apply Algorithm to problem (4)-(6) directly by modifying the virtual queue update equations as Q k (t + ) = Q k (t) + g k (x(t)), k {,,..., m}. In this case, a simple adaption of the convergence rate analysis in this paper can establish the same O(/t) convergence rate. However, to simplify the presentation, this paper just considers the general convex program in the form of ()-(3) since any linear equality can be equivalently represented by two linear inequalities.

5 By the definition of strongly convex functions, it is easy to show that if h(x) is convex and α > 0, then h(x) + α x x 0 is strongly convex with modulus α for any constant x 0. Lemma (Theorem 6.. in [7]). Let h(x) be strongly convex on X with modulus α. Let h(x) be the set of all subgradients of h at point x. Then h(y) h(x) + d T (y x) + α y x for all x, y X and all d h(x). Lemma (Proposition B.4 (f) in [3]). Let X R n be a convex set. Let function h be convex on X and x opt be a global minimum of h on X. Let h(x) be the set of all subgradients of h at point x. Then, there exists d h(x opt ) such that d T (x x opt ) 0 for all x X. Corollary. Let X R n be a convex set. Let function h be strongly convex on X with modulus α and x opt be a global minimum of h on X. Then, h(x opt ) h(x) α xopt x for all x X. Proof. A special case when h is differentiable and X = R n is Theorem..8 in [8]. The proof for general h and X is as follows: Fix x X. By Lemma, there exists d h(x opt ) such that d T (x x opt ) 0. By Lemma, we also have h(x) h(x opt ) + d T (x x opt ) + α x xopt (a) h(x opt ) + α x xopt, where (a) follows from the fact that d T (x x opt ) 0... Properties of the Virtual Queues. Lemma 3. In Algorithm, we have. At each iteration t {0,,,...}, Q k (t) 0 for all k {,,..., m}.. At each iteration t {0,,,...}, Q k (t) + g k (x(t )) 0 for all k {,..., m}. 3. At iteration t = 0, Q(0) g(x( )). At each iteration t {,,...}, Q(t) g(x(t )). Proof.. Fix k {,,..., m}. Note that Q k (0) 0 by the initialization rule Q k (0) = max{0, g k (x( ))}. Assume Q k (t) 0 and consider time t+. If g k (x(t)) 0, then Q k (t + ) = max{ g k (x(t)), Q k (t) + g k (x(t))} Q k (t) + g k (x(t)) 0. If g k (x(t)) < 0, then Q k (t + ) = max{ g k (x(t)), Q k (t) + g k (x(t))} g k (x(t)) > 0. Thus, Q k (t + ) 0. The result follows by induction.. Fix k {,,..., m}. Note that Q k (0) + g k (x( )) 0 by the initialization rule Q k (0) = max{0, g k (x( ))} g k (x( )). For t, by the virtual queue update equation, we have Q k (t) = max{ g k (x(t )), Q k (t ) + g k (x(t ))} g k (x(t )), which implies that Q k (t) + g k (x(t )) For t = 0. Fix k {,,..., m}. Consider the cases g k (x( )) 0 and g k (x( )) < 0 separately. If g k (x( )) 0, then Q k (0) = max{0, g k (x( ))} = 0 and so Q k (0) g k (x( )). If g k (x( )) < 0, then Q k (0) = max{0, g k (x( ))} = g k (x( )). Thus, in both cases, we have Q k (0) g k (x( )). Squaring both sides and summing over k {,,..., m} yields Q(0) g(x( )).

6 For t. Fix k {,,..., m}. Consider the cases g k (x(t )) 0 and g k (x(t )) < 0 separately. If g k (x(t )) 0, then Q k (t) = max{ g k (x(t )), Q k (t ) + g k (x(t ))} Q k (t ) + g k (x(t )) (a) g k (x(t )) = g k (x(t )), where (a) follows from part. If g k (x(t )) < 0, then Q k (t) = max{ g k (x(t )), Q k (t ) + g k (x(t ))} g k (x(t )) = g k (x(t )). Thus, in both cases, we have Q k (t) g k (x(t )). Squaring both sides and summing over k {,,..., m} yields Q(t) g(x(t ))..3. Properties of the Drift. Recall that Q(t) = [ Q (t),..., Q m (t) ] T is the vector of virtual queue backlogs. Define L(t) = Q(t). The function L(t) shall be called a Lyapunov function. Define the Lyapunov drift as (7) (t) = L(t + ) L(t) = [ Q(t + ) Q(t) ]. Lemma 4. At each iteration t {0,,,...} in Algorithm, an upper bound of the Lyapunov drift is given by (8) (t) Q T (t)g(x(t)) + g(x(t)). Proof. The virtual queue update equations Q k (t + ) = max{ g k (x(t)), Q k (t) + g k (x(t))}, k {,,..., m} can be rewritten as (9) where { g k (x(t)) = Q k (t + ) = Q k (t) + g k (x(t)), k {,,..., m}, g k (x(t)), Q k (t) g k (x(t)), if Q k (t) + g k (x(t)) g k (x(t)) else Fix k {,,..., m}. Squaring both sides of (9) and dividing by yields: (Q k(t + )) = (Q k(t)) + ( gk (x(t)) ) + Qk (t) g k (x(t)) = (Q k(t)) + ( gk (x(t)) ) + Qk (t)g k (x(t)) + Q k (t) ( g k (x(t)) g k (x(t)) ) (a) = (Q k(t)) + (x(t)) ) + Qk (t)g k (x(t)) ( gk ( g k (x(t)) + g k (x(t)) )( g k (x(t)) g k (x(t)) ) = (Q k(t)) ( gk (x(t)) ) + Qk (t)g k (x(t)) + ( g k (x(t)) ) (Q k(t)) + Q k (t)g k (x(t)) + ( g k (x(t)) ), k.

7 where (a) follows from the fact that Q k (t) ( g k (x(t)) g k (x(t)) ) = ( g k (x(t)) + g k (x(t)) )( g k (x(t)) g k (x(t)) ), which can be shown by considering g k (x(t)) = g k (x(t)) and g k (x(t)) g k (x(t)). Summing over k {,,..., m} yields Q(t + ) Q(t) + Q T (t)g(x(t)) + g(x(t)). Rearranging the terms yields the desired result. 3. Convergence Rate Analysis of Algorithm. This section analyzes the convergence rate of Algorithm for the problem ()-(3). 3.. An Upper Bound of the Drift-Plus-Penalty Expression. Lemma 5. Let x be an optimal solution of the problem ()-(3). If α β in Algorithm, then for all t 0, we have (t) + f(x(t)) f(x ) + α [ x x(t ) x x(t) ] + [ g(x(t)) g(x(t )) ], where β is defined in Assumption. Proof. Fix t 0. Note that Lemma 3 implies that Q(t)+g(x(t )) is componentwise nonnegative. Hence, the function f(x) + [ Q(t) + g(x(t )) ] T g(x) is convex with respect to x on X. Since α x x(t ) is strongly convex with respect to x with modulus α, it follows that f(x) + [ Q(t) + g(x(t )) ] T g(x) + α x x(t ) is strongly convex with respect to x with modulus α. Since x(t) is chosen to minimize the above strongly convex function, by Corollary, we have (0) f(x(t)) + [ Q(t) + g(x(t )) ] T g(x(t)) + α x(t) x(t ) f(x ) + [ Q(t) + g(x(t )) ] T g(x ) +α x x(t ) α x x(t) }{{} 0 (a) f(x ) + α x x(t ) α x x(t), where (a) follows by using the fact that g k (x ) 0 for all k {,,..., m} and Q k (t) + g k (x(t )) 0 (i.e., part in Lemma 3) to eliminate the term marked by an underbrace. Note that u T u = [ u + u u u ] for any u, u R m. Thus, we have () g(x(t )) T g(x(t)) = [ g(x(t )) + g(x(t)) g(x(t )) g(x(t)) ].

8 Substituting () into (0) and rearranging terms yields f(x(t)) + Q T (t)g(x(t)) f(x ) + α x x(t ) α x x(t) α x(t) x(t ) + g(x(t )) g(x(t)) g(x(t )) g(x(t)) (a) f(x ) + α x x(t ) α x x(t) + ( β α) x(t) x(t ) g(x(t )) g(x(t)) (b) f(x ) + α x x(t ) α x x(t) g(x(t )) g(x(t)), where (a) follows from the fact that g(x(t )) g(x(t)) β x(t) x(t ), which further follows from the assumption that g(x) is Lipschitz continuous with modulus β; and (b) follows from the fact α β. Summing (8) with the above inequality yields (t) + f(x(t)) f(x ) + α [ x x(t ) x x(t) ] + [ g(x(t)) g(x(t )) ]. 3.. Objective Value Violations. Lemma 6. Let x be an optimal solution of the problem ()-(3) and β be defined in Assumption.. If α β in Algorithm, then for all t, we have t f(x(τ)) tf(x ) + α x x( ).. If α > β in Algorithm, then for all t, we have t f(x(τ)) tf(x ) + α x x( ) + α α β g(x ) Q(t). Proof. By Lemma 5, we have (τ) + f(x(τ)) f(x ) + α[ x x(τ ) x x(τ) ] + [ g(x(τ)) g(x(τ )) ] for all τ {0,,,...}. Summing over τ {0,,..., t } yields t t t (τ) + f(x(τ)) tf(x ) + α [ x x(τ ) x x(τ) ] + t [ g(x(τ)) g(x(τ )) ]. Recalling that (τ) = L(τ + ) L(τ) and simplifying summations yields t L(t) L(0) + f(x(τ)) tf(x ) + α x x( ) α x x(t ) + g(x(t )) g(x( )).

9 Rearranging terms; and substituting L(0) = Q(0) and L(t) = Q(t) yields t f(x(τ)) tf(x ) + α x x( ) α x x(t ) + g(x(t )) g(x( )) + Q(0) Q(t) () (a) tf(x ) + α x x( ) α x x(t ) + g(x(t )) Q(t), where (a) follows from the fact that Q(0) g(x( )), i.e., part 3 in Lemma 3. Next, we present the proof of both parts:. This part follows from the observation that the equation () can be further simplified as t f(x(τ)) (a) tf(x ) + α x x( ) + g(x(t )) Q(t) (b) tf(x ) + α x x( ), where (a) follows by ignoring the non-positive term α x x(t ) on the right side and (b) follows from the fact that Q(t) g(x(t )), i.e., part 3 in Lemma 3.. This part follows by rewriting the equation () as t f(x(τ)) tf(x ) + α x x( ) α x x(t ) + g(x(t )) g(x ) + g(x ) Q(t) =tf(x ) + α x x( ) α x x(t ) + g(x(t )) g(x ) + g T (x )[g(x(t )) g(x )] + g(x ) Q(t) (a) tf(x ) + α x x( ) α x x(t ) + g(x(t )) g(x ) + g(x ) g(x(t )) g(x ) + g(x ) Q(t) (b) tf(x ) + α x x( ) α x x(t ) + β x x(t ) + β g(x ) x x(t ) + g(x ) Q(t) =tf(x ) + α x x( ) ( α β)[ x x(t ) β α g(x ) β α + α β g(x ) Q(t) ]

10 (c) tf(x ) + α x x( ) α + α β g(x ) Q(t), where (a) follows from Cauchy-Schwarz inequality; (b) follows from the fact that g(x(t )) g(x ) β x x(t ), which further follows from the assumption that g(x) is Lipschitz continuous with modulus β; and (c) follows from the fact that α > β. Theorem (Objective Value Violations). Let x be an optimal solution of the problem ()-(3). If α β in Algorithm, for all t, we have where β is defined in Assumption. f(x(t)) f(x ) + α t x x( ), Proof. Fix t. By part in Lemma 6, we have Since x(t) = t that t f(x(τ)) tf(x ) + α x x( ) t f(x(τ)) f(x ) + α t t x x( ). t x(τ) and f(x) is convex, by Jensen s inequality it follows f(x(t)) t f(x(τ)). t The above theorem shows that the error gap between f(x(t)) and the optimal value f(x ) is at most O(/t). This holds for any initial guess vector x( ) X. Of course, choosing x( ) close to x is desirable because it reduces the coefficient α x x( ) Constraint Violations. Lemma 7. Let Q(t), t {0,,...} be the sequence generated by Algorithm. For any t, t Q k (t) g k (x(τ)), k {,,..., m}. Proof. Fix k {,,..., m} and t. For any τ {0,..., t } the update rule of Algorithm gives: Q k (τ + ) = max{ g k (x(τ)), Q k (τ) + g k (x(τ))} Q k (τ) + g k (x(τ)). Hence, Q k (τ + ) Q k (τ) g k (x(τ)). Summing over τ {0,..., t } and using Q k (0) 0 gives the result. Lemma 8. Let x be an optimal solution of the problem ()-(3) and λ be a Lagrange multiplier vector satisfying Assumption. Let x(t), Q(t), t {0,,...} be sequences generated by Algorithm. Then, t f(x(τ)) tf(x ) λ Q(t), t.

11 Proof. The proof is similar to a related result in [3] for the DPP algorithm. Define Lagrangian dual function q(λ) = min {f(x) + m x X k= λ kg k (x)}. For all τ {0,,...}, by Assumption, we have f(x ) = q(λ ) (a) f(x(τ)) + m λ kg k (x(τ)), k= where (a) follows the definition of q(λ ). Thus, we have f(x(τ)) f(x ) m λ kg k (x(τ)), τ {0,,...}. k= Summing over τ {0,,..., t } yields t t m f(x(τ)) tf(x ) λ kg k (x(τ)) =tf(x ) (a) tf(x ) k= m [ t λ k k= m λ kq k (t) k= (b) tf(x ) λ Q(t), ] g k (x(τ)) where (a) follows from Lemma 7 and the fact that λ k 0, k {,,..., m}; and (b) follows from the Cauchy-Schwarz inequality. Lemma 9. Let x be an optimal solution of the problem ()-(3) and λ be a Lagrange multiplier vector satisfying Assumption. If α > β in Algorithm, then for all t, the virtual queue vector satisfies Q(t) λ + α α x x( ) + α g(x ), β where β is defined in Assumption. Proof. Fix t. By part in Lemma 6, we have t f(x(τ)) tf(x ) + α x x( ) α + α β g(x ) Q(t). By Lemma 8, we have t f(x(τ)) tf(x ) λ Q(t). Combining the last two inequalities and cancelling the common term tf(x ) on both

12 sides yields Q(t) ( α x x( ) + ( Q(t) λ ) λ + α x x( ) + Q(t) λ + λ + α x x( ) + (a) Q(t) λ + α x x( ) + α α β g(x ) ) λ Q(t) α α g(x ) β α α β g(x ) α α β g(x ), where (a) follows from the basic inequality a + b + c a + b + c for any a, b, c 0. Theorem (Constraint Violations). Let x be an optimal solution of the problem ()-(3) and λ be a Lagrange multiplier vector satisfying Assumption. If α > β in Algorithm, then for all t, the constraint functions satisfy g k (x(t)) ( λ + α ) α x x( ) + t α g(x ), k {,,..., m}, β where β is defined in Assumption. Proof. Fix t and k {,,..., m}. Recall that x(t) = t g k (x(t)) (a) t g k (x(τ)) t (b) Q k(t) t Q(t) t (c) t ( λ + α x x( ) + t x(τ). Thus, α ) α g(x ), β where (a) follows from the convexity of g k (x), k {,,..., m} and Jensen s inequality; (b) follows from Lemma 7; and (c) follows from Lemma Convergence Rate of Algorithm. The next theorem summarizes the last two subsections. Theorem 3. Let x be an optimal solution of the problem ()-(3) and λ be a Lagrange multiplier vector satisfying Assumption. If α > β in Algorithm, then for all t, we have f(x(t)) f(x ) + α t x x( ), g k (x(t)) ( λ + α ) α x x( ) + t α g(x ), k {,,..., m}, β where β is defined in Assumption. In summary, Algorithm ensures error decays like O(/t) and provides an ɛ-approximate solution with convergence time O(/ɛ).

13 4. Application: Decentralized Network Utility Maximization. This section considers the application of Algorithm to decentralized multipath network utility maximization problems. 4.. Decentralized Multipath Flow Control. Network flow control can be formulated as the convex optimization of maximizing network utility subject to link capacity constraints [8]. In this view, many existing TCP protocols can be interpreted as distributed solutions to network utility maximization (NUM) problems [0]. In single path network flow control problems, if the utility functions are strictly convex, work [] shows that the dual subgradient algorithm (with convergence rate O(/ t)) can yield a distributed flow control algorithm. If the utility functions are only convex but not necessarily strictly convex, the DPP algorithm (or dual subgradient algorithm with primal averaging) can yield a distributed flow control algorithm with an O(/ t) convergence rate [5, 3, 7]. If utility functions are strongly convex, a faster network flow control algorithm with an O(/t) convergence rate is proposed in []. A recent work [3] shows that the distributed network flow control based on the DPP algorithm also has convergence rate O(/t) if utility functions are strongly convex. Other Newton method based distributed algorithms for network flow control with strictly convex utility functions are considered in []. However, in multipath network flow control problems, even if the utility function is strictly or strongly convex with respect to the source rate, it is no longer strictly or strongly convex with respect to path rates. Thus, many of the above algorithms requiring strict or strong convexity can no longer be applied. The DPP algorithm can still be applied but the convergence rate is only O(/ t). Distributed algorithms based on the primal-dual subgradient method, also known as the Arrow-Hurwicz- Uzawa subgradient method, have been considered in [0]. However, the convergence rate 3 of the primal-dual subgradient method for general convex programs without strong convexity is known to be O(/ t) [4]. As shown in the previous sections, Algorithm has an O(/t) convergence rate for general convex programs and yields a distributed algorithm if the objective function and constraint functions are separable. The next subsection applies Algorithm to the multipath network flow control problem. The resulting algorithm has structural properties and implementation characteristics similar to the subgradient-based algorithm of [0] and to the DPP algorithm of [3], but has additional decision variables due to the multipath formulation. (The algorithms in [0, 3] rely on strict and strong convexity of the objective function, respectively, and apply only to single path situations.) 4.. Decentralized Multipath Flow Control Based on Algorithm. Suppose there are S sources enumerated by S = {,,..., S} and L links enumerated by L = {,,..., L}. Each link l L has a link capacity of c l bits/slot. Each source sends data from a specific origin to a specific destination, and has multiple path options. The paths for each source can use overlapping links, and they can also overlap with paths of other sources. Further, two distinct sources can have identical paths. However, it is useful to give distinct path indices to paths associated with each source (even if their corresponding paths are physically the same). Specifically, for each source s, define P s as the set of path indices used by source s. The index sets 3 Similar to the dual subgradient method, the primal-dual subgradient method has an O(/ t) convergence rate for general convex programs in the sense that O(/ɛ ) iterations are required to obtain an ɛ-approximate solution. However, the primal-dual subgradient method does not have vanishing errors.

14 P,..., P S are disjoint and P P P S = K = {,,..., K}, where K is the number of path indices. For each source s, let U s (y s ) be a real-valued, concave, continuous and nondecreasing utility function defined for y s 0. This represents the satisfaction source s receives by communicating with a total rate of y s, where the total rate sums over all paths it used. Note that different paths can share links in common. Define D l K as the set of paths that use link l and E k L as the set of links used by path k. Let x = [x,..., x K ] T be the vector that specifies the flow rate on each path; and y = [y,..., y S ] T be the vector that specifies the rate of each source. The goal is to allocate flow rates on each path so that no link is overloaded and network utility is maximized. This multipath network utility maximization problem can be formulated as follows: S (3) maximize U s (y s ) (4) (5) (6) (7) subject to s= x k c l, l L, k D l y s = x k, s S, k P s 0 x k x max k, k K, 0 y s ys max, s S, where x max and y max are the allowed maximum path rate and maximum source rate, respectively. The expression (3) represents the network utility; inequality (4) specifies the link capacity constraints; and equality (5) enforces the definition of y s. The linear equality constraints (5) can be formally treated by writing each one as two linear inequality constraints. However, since the utility functions U s ( ) are nondecreasing and seek to maximize the y s values, it is clear that the above problem is equivalent to the following: (8) (9) (0) minimize subject to S U s (y s ) s= x k c l, l L, k D l y s x k, s S, k P s () 0 x k x max k, k K, () 0 y s ys max, s S. [ ] x Note that the constraints (9)-(0) are linear and can be written as A [ ] y c, where A is an (L+S) (K +S) matrix of which each entry is in {0, ±}. This 0 [ ] [ ] x c inequality constraint A can be treated as the inequality constraint y 0 [ ] [ ] x c () defined as g(z) = Az b 0 with z = and b = in the general y 0

15 convex program ()-(3). The box constraints ()-() can be treated as the set constraint (3) in the general convex program ()-(3). Thus, the multipath network utility maximization problem (8)-() is a special case of the general convex program ()-(3). The Lipschitz continuity of g(z) is summarized in the next lemma. Lemma 0. If the constraints (9)-(0) are written as g(z) 0, then we have. The function g(z) is Lipschitz continuous with modulus β S + K + d k, k K where d k is the length, i.e., number of hops, of path k K.. The function g(z) is Lipschitz continuous with modulus β (L + )K + S. Proof.. Recall that g(z) = Az b is Lipschitz continuous with modulus β = σ max (A), where σ max (A) is the maximum singular value of matrix A, and a simple upper bound of σ max (A) is the Frobenius norm given by A F = tr(a T A) = i,j A ij. Note that A can be written as [ R 0L S A = T where R is a {0, } matrix of size L K, 0 L S is an L S zero matrix, T is a {0, } matrix of size S K and I S is an S S identity matrix. Note that the (l, k)-th entry of R is if and only if path k uses link l; the (s, k)-th entry of T is if and only if path k is a path for source s, i.e, k P s. Matrix R has k K d k non-zero entries since each column has exactly d k non-zero entries. Matrix T has K non-zero entries since there are in total K network paths. Thus, matrix A in total has S + K + k K d k non-zero entries whose absolute values are equal to. It follows that β S + K + k K d k.. This part follows from the fact that the length of each path is at most L. Note that the second bound in the above lemma is more loose than the first one but holds regardless of the flow path configurations in the network. A straightforward application of Algorithm yields the following decentralized network flow control algorithm described in Algorithm. Similar to the flow control based on the dual subgradient algorithm, Algorithm is decentralized and can be easily implemented within the current TCP protocols [0].( By Theorem 3, if we choose α S + K + k K d k), then for any t, I S ], (3) (4) (5) S S U s (y s (t)) U s (ys) O(/t), s= s= x k (t) c l + O(/t), l L, k D l y s (t) x k (t) + O(/t), s S, k P s where (x, y ) is an optimal solution of the problem (8)-(). Note that if Algorithm has been run for a sufficiently long time and we are satisfied with the current

16 Algorithm Initialization: Let x k ( ) [0, x max k ], k K be arbitrary, y s ( ) [0, ys max ], s S be arbitrary, Q l (0) = max{0, k D l x k ( ) + c l }, l L, Y l (0) = Q l (0) + k D l x k ( ) c l, l L, R s (0) = max{0, k P s x k ( ) y s ( )}, s S and Z s (0) = R s (0) + y s ( ) k P s x k ( ), s S. Each link l s algorithm: At each time t {0,,...}, link l does the following:. Receive the path rates x k (t) that use link l. Update Q l (t) via: { Q l (t + ) = max } x k (t) + c l, Q l (t) + x k (t) c l. k D l k D l. The price of this link is given by Y l (t+) = Q l (t+)+ k D l x k (t) c l. 3. Communicate the link price Y l (t + ) to sources that use link l. Each source s s algorithm: At each time t {0,,...}, source s does the following:. Receive from the network the link prices Y l (t) for all links l that are used by any path of source s.. Update the path rates x k, k P s by { [ ] x k (t) = argmin Y l (t) Z s (t) xk + α(x k x k (t )) } 0 x k x max k l E k [ = x k (t ) ( ) ] x max k Y l (t) Z s (t), α 0 l E k where [z] b a = min{max{z, a}, b}. 3. Communicate the path rates x k (t), k P s to all links that are used by path k. 4. Update the source rate y s (t) by y s (t) = argmin 0 y s y max s { U s (y s ) + Z s (t)y s + α(y s y s (t )) }, which usually has a closed form solution for differentiable utilities by taking derivatives. 5. Update virtual queue R s (t) and source price Z s (t) locally by { R s (t + ) = max y s (t) + } x k (t), R s (t) + y s (t) x k (t), k P s k P s Z s (t + ) = R s (t + ) + y s (t) x k (t). k P s t t performance, then we can fix x = t x(τ) and y = t y(τ) such that the performance is still within O(/t) sub-optimality for all future time Decentralized Joint Flow and Power Control. Since Algorithm allows for general nonlinear convex constraint functions, it can also be applied to solve the joint flow and power control problem. In this case, the capacity of each link l is not fixed but depends concavely on a power allocation variable p l. Assuming each

17 link capacity is logarithmic in p l results in the following problem: (6) (7) (8) (9) (30) (3) maximize subject to S U s (y s ) s= L V l (p l ) l= x k log( + p l ), l L, k D l y s x k, s S, k P s 0 x k x max k, k K, 0 y s ys max, s S, 0 p l p max l, l L, where p l is the power allocated at link l, log( + p l ) is the corresponding link capacity as a function of p l, and V l (p l ) is the associated power cost (assumed to be a convex function of p l ). A decentralized joint flow and power control algorithm for this problem can be similarly developed by applying Algorithm. 5. Numerical Results. This section considers numerical experiments to verify the convergence rate results shown in this paper. 5.. Decentralized Multiplath Flow Control. Consider the simple multipath network flow problem described in Figure. Assume each link has capacity. Let y, y and y 3 be the data rates of source, and 3; x, x, x 3, x 4, x 5, x 6 and x 7 be the data rates of the paths indicated in the figure; and the network utility be maximizing log(y ) + log(y ) + log(y 3 ). The NUM problem can be formulated as follows: where R = maximize log(y ) + log(y ) + log(y 3 ) subject to Rx c, y Tx, 0 x i x max i, i {,,..., 4}, 0 y i yi max, i {,, 3},, c =, T = The optimal value to this NUM problem is f = To verify the convergence of Algorithm, Figure shows ( the values of objective and constraint functions yielded by Algorithm with α = K+S+ k K d k) + = 0 and x( ) = 0. (By writing constraints Rx c and y Tx in the compact form Az b, it can be checked that β = σ max (A) = If we choose a smaller α, e.g., α = β + = , then Algorithm converges even faster. In this simulation, we choose a loose α whose value can be easily estimated from Lemma 0 without.

18 knowing the detailed network topology.) We also compare our algorithm with the dual subgradient algorithm (with primal averaging) with step size 0.0 in [3]. (Or equivalently, the DPP algorithm with V = 00 in [5, 7, 3].) Recall that the dual subgradient algorithm in [5, 3, 7, 3] does not converge to an exact optimal solution but only converges to an approximate solution with an error level determined by the step size. In contrast, our algorithm can eventually converge to the exact optimality. Figure shows that Algorithm converges faster than the dual subgradient algorithm with primal averaging. To verify the convergence rate of Algorithm, Figure 3 plots f(x(t)) f, all constraint values, function /t, and bounds from Theorem 3 with both x-axis and y-axis in log 0 scales. It can be observed that the curves of f(x(t)) f and all the source rate constraint values are parallel to the curve of /t for large t. Note that all the link capacity constraints are satisfied early (i.e., negative), and hence are not drawn in log 0 scales. Figure 3 verifies that the error of Algorithm decays like O(/t) and suggests that it is actually Θ(/t) for this multipath NUM problem. x l 4 x 3 source l 8 source l x l l 3 l 5 x 4 x 5 l 7 x 6 x 7 l 9 l 6 source 3 Fig.. A simple multipath NUM problem with 3 sources and 7 paths. 5.. Decentralized Joint Flow and Power Control. Consider the joint flow and power control over the same network described in Figure. We assume the power cost of each link is given by V l (p l ) = 0.5p l. The optimal value of this joint flow and power control problem is f = To verify the convergence of Algorithm, Figure 4 shows the values of objective and constraint functions yielded by Algorithm with α = 0, x( ) = 0, y( ) = 0 and p( ) = 0. (In fact, by writing constraints Rx log( + p) and y Tx in the compact form g(z) 0, it can be checked that β =.59. If we choose a smaller α, e.g., α = β + = 4.86, then Algorithm converges even faster.) We also compare our algorithm with the dual subgradient algorithm (with primal averaging) with step size 0.0 in [5, 3, 7, 3]. Figure 4 shows that Algorithm converges faster than the dual subgradient algorithm with primal averaging. To verify the convergence rate of Algorithm, Figure 5 plots f(x(t)) f, all constraint values, function /t, and bounds from Theorem 3 with both x-axis and y-axis in log 0 scales. It can be observed that the curves of f(x(t)) f and all the source rate constraint values are parallel to the curve of /t for large t.

19 Source Rate Cons Value Objective Value Link Capacity Cons Value Iterations: t Iterations: t Iterations: t Fig.. The convergence of Algorithm and the dual subgradient algorithm with primal averaging for a multipath flow control problem Quadratic Programs. Consider the following quadratic program min s.t. x T Px + c T x x T Qx + d T x e x min x x max where P and Q are positive semidefinite to ensure the convexity of the quadratic program. We randomly generate a large scale example where x R 00, P R is diagonal with entries from uniform [0, 4], c R 00 with entries from uniform [ 5, 0], Q R is diagonal with entries from uniform [0, ], d R 00 with entries from uniform [, ], e is a scalar from uniform [4, 5], x min = 0 and x max =. Note that Algorithm and the dual subgradient algorithm (with primal averaging) can deal with general semidefinite positive matrices P and Q. However, if P and Q are diagonal or block diagonal, then the primal update in both algorithms can be decomposed into independent smaller problems and hence has extremely low complexity. To verify the convergence of Algorithm, Figure 6 shows the values of objective and constraint functions yielded by Algorithm with α = β +, where β the is Lipschitz modulus of the constraint function, and x( ) = x min. We also compare our algorithm with the dual subgradient algorithm (with primal averaging) with step

20 Flow Control: Convergence Rate of Algorithm v.s. O(/t) upper bound of constraint violations from Thm 3 f $! f(x(t)) upper bound of f $! f(x(t)) from Thm 3 Constraint/Objective Violations curves are parallel for large t 0-4 =t 0-5 source rate constraints (3 curves) Iterations: t Fig. 3. The convergence rate of Algorithm for a multipath flow control problem. size 0.0 in [5, 3, 7, 3]. Figure 6 shows that Algorithm converges faster than the dual subgradient algorithm with primal averaging. To verify the convergence rate of Algorithm, Figure 7 plots f(x(t)) f, function /t, and the bound from Theorem 3 with both x-axis and y-axis in log 0 scales. It can be observed that the curves of f(x(t)) f and all the source rate constraint values are parallel to the curve of /t for large t. Note that the constraint violation is not plotted since the constraint function is satisfied for all iterations as observed in Figure Conclusions. This paper proposes a novel but simple algorithm to solve convex programs with a possibly non-differentiable objective function and Lipschitz continuous constraint functions. The new algorithm has a parallel implementation when the objective function and constraint functions are separable. The convergence rate of the proposed algorithm is shown to be O(/t). This is faster than the O(/ t) convergence rate of the dual subgradient algorithm with primal averaging. The ADMM algorithm has the same O(/t) convergence rate but can only deal with linear equality constraint functions. The new algorithm is further applied to solve multipath network flow control problems and yields a decentralized flow control algorithm which converges faster than existing dual subgradient or primal-dual subgradient based flow control algorithms. The O(/t) convergence rate of the proposed algorithm is also verified by numerical experiments.

21 Objective Value Link Capacity Cons Value Source Rate Cons Value Joint Flow and Power Control: Convergence of Objective Values 6 4 dual subgradient with primal averaging optimal value Algorithm Iterations: t Joint Flow and Power Control: Convergence of Link Capacity Constraints Algorithm (9 curves) Iterations: t Joint Flow and Power Control: Convergence of Source Rate Constraints 4 3 dual subgradient with primal averaging(9 curves) dual subgradient with primal averaging (3 curves) 0 Algorithm (3 curves) Iterations: t Fig. 4. The convergence of Algorithm and the dual subgradient algorithm with primal averaging for a multipath joint flow and power control problem. REFERENCES [] M. S. Bazaraa, H. D. Sherali, and C. M. Shetty, Nonlinear Programming: Theory and Algorithms, Wiley-Interscience, 006. [] A. Beck, A. Nedic, A. Ozdaglar, and M. Teboulle, An O(/k) gradient method for network resource allocation problems, IEEE Transactions on Control of Network Systems, (04), pp [3] D. P. Bertsekas, Nonlinear Programming, Athena Scientific, second ed., 999. [4] S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, Distributed optimization and statistical learning via the alternating direction method of multipliers, Foundations and Trends in Machine Learning, 3 (0), pp.. [5] S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, 004. [6] B. He and X. Yuan, On the O(/n) convergence rate of the Douglas-Rachford alternating direction method, SIAM Journal on Numerical Analysis, 50 (0), pp [7] J.-B. Hiriart-Urruty and C. Lemaréchal, Fundamentals of Convex Analysis, Springer, 00. [8] F. P. Kelly, A. K. Maulloo, and D. K. Tan, Rate control for communication networks: Shadow prices, proportional fairness and stability, Journal of the Operational Research Society, 49 (998), pp [9] T.-Y. Lin, S.-Q. Ma, and S.-Z. Zhang, On the sublinear convergence rate of multi-block ADMM, Journal of the Operations Research Society of China, 3 (05), pp [0] S. H. Low, A duality model of TCP and queue management algorithms, IEEE/ACM Transactions on Networking, (003), pp [] S. H. Low and D. E. Lapsley, Optimization flow control I: basic algorithm and convergence, IEEE/ACM Transactions on Networking, 7 (999), pp

22 0 3 0 $! x $! x 0 Constraint/Objective Violations curves are parallel for large t link capacity constraints (9 curves) source rate constraints (3 curves) Iterations: t Fig. 5. The convergence rate of Algorithm for a multipath joint flow and power control problem. [] I. Necoara and V. Nedelcu, Rate analysis of inexact dual first-order methods application to dual decomposition, IEEE Transactions on Automatic Control, 59 (04), pp [3] A. Nedić and A. Ozdaglar, Approximate primal solutions and rate analysis for dual subgradient methods, SIAM Journal on Optimization, 9 (009), pp [4] A. Nedić and A. Ozdaglar, Subgradient methods for saddle-point problems, Journal of Optimization Theory and Applications, 4 (009), pp [5] M. J. Neely, Distributed and secure computation of convex programs over a network of connected processors, in DCDIS Conference Guelph, July 005. [6] M. J. Neely, Stochastic Network Optimization with Application to Communication and Queueing Systems, Morgan & Claypool Publishers, 00. [7] M. J. Neely, A simple convergence time analysis of drift-plus-penalty for stochastic optimization and convex programs, arxiv:4.079, (04). [8] Y. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course, Springer Science & Business Media, 004. [9] N. Parikh and S. Boyd, Proximal algorithms, Foundations and Trends in Optimization, (03), pp [0] W.-H. Wang, M. Palaniswami, and S. H. Low, Optimal flow control and routing in multi-path networks, Performance Evaluation, 5 (003), pp [] E. Wei and A. Ozdaglar, On the O(/k) convergence of asynchronous distributed alternating direction method of multipliers, in Proceedings of IEEE Global Conference on Signal and Information Processing, 03. [] E. Wei, A. Ozdaglar, and A. Jadbabaie, A distributed Newton method for network utility maximization I: algorithm, IEEE Transactions on Automatic Control, 58 (03), pp [3] H. Yu and M. J. Neely, On the convergence time of the drift-plus-penalty algorithm for

23 Quadratic Program: Convergence of Objective Values Objective Value Algorithm optimal value dual subgradient with primal averaging Constraint Value Iterations: t Quadratic Program: Convergence of Constraint Violations dual subgradient with primal averaging Algorithm Iterations: t Fig. 6. The convergence of Algorithm and the dual subgradient algorithm with primal averaging for a quadratic program. strongly convex programs, in Proceedings of IEEE Conference on Decision and Control (CDC), 05. [4] H. Yu and M. J. Neely, A primal-dual type algorithm with the O(/t) convergence rate for large scale constrained convex programs, in Proceedings of IEEE Conference on Decision and Control (CDC), 06.

24 0 4 Quadratic Program: Convergence Rate of Algorithm v.s. O(/t) f(x(t))! f $ upper bound of f(x(t))! f $ from Thm 3 curves are parallel for large t =t Fig. 7. The convergence rate of Algorithm for a quadratic program.