Joint Congestion Control and Routing Optimization: An Efficient Second-Order Distributed Approach

Transcription

1 IEEE/ACM TRANSACTIONS ON NETWORKING Joint Congestion Contro and Routing Optimization: An Efficient Second-Order Distributed Approach Jia Liu, Member, IEEE, Ness B. Shroff, Feow, IEEE, Cathy H. Xia, and Hanif D. Sherai Abstract Distributed joint congestion contro and routing optimization has received a significant amount of attention recenty. To date, however, most of the existing schemes foow a ey idea caed the bac-pressure agorithm. Despite having many saient features, the first-order subgradient nature of the bac-pressure based schemes resuts in sow convergence and poor deay performance. To overcome these imitations, in this paper, we mae a first attempt at deveoping a second-order joint congestion contro and routing optimization framewor that offers utiity-optimaity, queue-stabiity, fast convergence, and ow deays. Our contributions in this paper are three-fod: i we propose a new second-order joint congestion contro and routing framewor based on a prima-dua interior-point approach; ii we estabish utiity-optimaity and queue-stabiity of the proposed second-order method; and iii we show how to impement the proposed second-order method in a distributed fashion. Index Terms Second-order distributed agorithm, muti-hop routing, congestion contro. I. INTRODUCTION With the rapid integration of new appications and technoogies, recent years have witnessed a growing chaenge in maing communication networs wor more efficienty. To date, whie there exists a arge body of wor on optimizationbased dynamic joint congestion contro and routing poicy for both wireine and wireess networs see, e.g., ] 5], most of these schemes foow a ey idea caed the bac-pressure agorithm, which traces its roots to the ceebrated paper 6]. The enduring popuarity of the bac-pressure agorithm is primariy due to: i a provabe throughput optimaity, ii eegant cross-ayer extensions, and iii a distributed dynamic queue-ength differentia based routing poicy that stabiizes a queues in the networ. Researchers have aso uncovered a fundamenta connection between the bac-pressure based congestion contro and the Lagrangian dua decomposition framewor pus the subgradient method in cassica noninear optimization theory ], 3], where scaed queue-engths pay the roe of Lagrangian dua variabes and the queue-ength updates correspond to subgradient directions. This enightening Manuscript received March 6, 04; revised December 4, 04; and accepted February 4, 05. This wor has been supported in part by NSF grants CNS-44658, CNS-0700, IIS , ECCS-38, SES- 4094, CNS-47830, and CNS-343. Jia Liu and Ness B. Shroff are with the Department of Eectrica and Computer Engineering, The Ohio State University, Coumbus, OH, 430 USA e-mai: {iu.736, shroff.@osu.edu. Cathy H. Xia is with the Department of Integrated Systems Engineering, The Ohio State University, Coumbus, OH, 430 USA e-mai: xia.5@osu.edu. Hanif D. Sherai is with the Grado Department of Industria and Systems Engineering, Virginia Tech, Bacsburg, VA, 406 USA e-mai: hanifs@vt.edu. Digita Object Identifier 0.09/TNET insight has unified techniques that originated independenty from contro and optimization theory. However, despite a the saient features, the subgradient nature of the bac-pressure based congestion contro and routing schemes turns out to be a factor that pagues their performance in practice. Being a first-order method subgradients can be viewed as a first-order support of the dua function, bac-pressure based joint congestion contro and routing schemes negect the curvature of the objective function contour, which is characterized by the eigenvaue condition number of the Hessian matrix that usuay becomes increasingy i-conditioned as the iterates approach an optima soution 7]. Hence, it necessitates a sma update in each iteration ] 4], 8], which subsequenty sows down convergence and undermines the performance of optimization. This imitation motivates us to pursue a second-order design approach for distributed dynamic congestion contro and routing. The fundamenta rationae behind our approach is that, as in cassica noninear optimization theory 7], by considering the secondorder Hessian information in congestion contro anid routing, we can expect to aeviate the inherent i-conditioned behavior of first-order methods, thus eading to much faster convergence and hence better performance in practice. However, deveoping a dynamic distributed second-order congestion contro and routing poicy is highy chaenging and resuts in this area remain scarce. First, unie the reativey obvious queue-ength connection between the bac-pressure based agorithms, it remains uncear how one can utiize the insights from existing second-order networ optimization agorithms 9] ] to guide the design of an optima dynamic congestion contro and routing poicy. The main chaenge is that the existing wor in 9] ] are static schemes that operate with average rates and ony yied fixed aocation soutions, rather than dynamic poicies that are abe to evove with time instants to dynamicay aocate resources. Aso, their connection to observabe networ state information e.g., queue-ength is sti missing. Second, after constructing a second-order scheme, it remains a difficut tas to prove its utiity-optimaity and queue-stabiity defined formay in Section III. This is because the incorporation of the second-order Hessian information significanty compicates the computationa schemes and necessitates new theoretica approaches in performance anaysis. Lasty, how to impement the deveoped second-order scheme in a distributed fashion comparabe to first-order methods is sti an open question. Simiar to the second-order optimization agorithms in 9] ], one woud have to face the chaenges arising from decentraizing the Hessian and Lapacian matrix inverse computations.

2 IEEE/ACM TRANSACTIONS ON NETWORKING The ey contribution of this paper is that, for the first time, we successfuy deveop a second-order joint congestion contro and routing framewor to address the aforementioned technica difficuties and estabish an anaytica foundation that offers fast convergence and high performance. The main resuts and technica contributions of this paper are as foows: We propose a fast-converging second-order joint congestion contro and routing framewor based on a primadua interior-point approach with a simpe step-size contro strategy, such that the resutant scheme is we-suited for impementation in practice. Our prima-dua approach exposes a deep connection between observabe networ state information and the prima-dua interior-point optimization theory, which itsef is an active research fied in operations research today see, e.g., 3] for a survey. We estabish the utiity-optimaity and queue-stabiity of the proposed second-order framewor. Our theoretica anaysis unveis the fundamenta reason behind the fast convergence in the proposed second-order framewor. Interestingy, our anaytica resuts naturay ead to a utiity-optimaity and queue-ength trade-off reationship governed by the barrier parameter of the interior-point method. We compare this trade-off reationship to those in first-order methods and contrast their simiarities and differences, thus further advancing our understanding of both first- and second-order methods in networ optimization theory. We suggest severa approaches to impement the proposed second-order method in a distributed fashion. In particuar, for the distributed dua Newton direction computation the most chaenging part in our second-order method, we propose a new Sherman-Morrison-Woodbury SMW based iterative approach. We show that, on a L-in networ, the SMW-based approach obtains the precise soution in L iterations, rather than asymptoticay as in 9] ]. Coectivey, our resuts in this paper contribute to an exciting deveopment of a cross-ayer networ contro and optimization theory with second-order techniques. The remainder of this paper is organized as foows. In Section II, we review reated wors. Section III introduces the networ mode and probem formuation. Section IV presents the agorithm and performance anaysis of our second-order scheme. Section V deveops the principa components of the distributed computations. Section VI presents some numerica resuts, and Section VII concudes the paper. II. RELATED WORK In this section, we review the state-of-the-art of both firstand second-order methods that are cosey reated to this paper. As mentioned earier, there is a arge body of wor on first-order bac-pressure based joint congestion contro and routing e.g., ] 4], 8], 4] Among these wors, the scheme in 3] is the most reated and can be directy compared to our wor since it is aso a prima-dua based controer, where the prima and dua variabes are updated jointy hence reativey more convenient to impement in practice. Thans to the second-order structure, our approach requires a much ess conservative step-size seection, whie achieving a steeper negative Lyapunov drift rate and inducing a much faster three orders of magnitude numericay convergence than in 3]. On the other hand, the schemes in ], ], 4] can be categorized as dua-based controers, where an inner subprobem defined in terms of prima variabes needs to be soved for each fixed set of dua variabes. Thus, a counterpart of prima-dua step-size seection does not exist. However, simiar Lyapunov drift rate anaysis and numerica resuts aso indicate a sow convergence performance due to their first-order nature. In the second-order domain, recent centraized and distributed interior-point based methods for networ optimization can be found in 0] ], 5] 9]. In particuar, significant efforts have been made to decentraize the secondorder computations, incuding a Gaussian beief propagation technique in 6] 8] and a matrix-spitting approach in 9] for fow contro with fixed routing; and a consensusbased oca averaging scheme for minimum cost routing with fixed source rates in ]. In our previous wor 0], ], we deveoped distributed second-order methods for crossayer optimization joint fow contro, routing, and scheduing in both wireine and wireess networs. However, a these second-order methods operate with ong-term rates and do not consider queue evoution and stabiity. Moreover, they can be a categorized as the cassica barrier interior-point approach. Different from these previous wor, this paper is motivated by recent observations on the superior efficiency of the prima-dua interior-point approach compared to the barrier-based approach 3]. The main difference between the barrier interior-point approach and the prima-dua interiorpoint approach ies in different approaches in handing the so-caed perturbed KKT system cf. Eqs. 3 6 in the context of this paper. From the perspective of the perturbed KKT system handing, the barrier approach can be interpreted as a restricted version of the prima-dua approach see 0, Page 6]. Moreover, it has been widey observed in practice that the barrier-based interior-point approach is ess efficient than the prima-dua method 3], 0], which is due to the fact that the Newton-step obtained by the barrier-based approach tends to produce infeasibe prima soutions see 3, Sec ] for detais. Therefore, the deveopment of our prima-dua second-order method in this paper foows a more effective soution process. We aso note that, recenty, the authors of ] have proposed a second-order distributed agorithm to acceerate the convergence of the bac-pressure agorithm. Our wor differs from ] in the foowing ey aspects. First, the agorithm in ] ony considers routing without addressing congestion contro for end-to-end utiity optimaity. Second, the second-order acceeration in ] is appied ony in the dua domain to adjust bac-pressure differentias, whie the prima variabes routing decisions are determined by the soft bacpressure poicy in ], which is a waterfiing-type scheme for mutipexing different sessions on each in. The waterfiing eve does not have a cosed-form soution and has to be determined numericay. In contrast, our scheme adjusts both prima and dua variabes by taing second-order Hessian information into consideration. In each time-sot, the mutipexing fraction of each session on each in naturay foows from the Hessian inverse, which

3 LIUet a.: JOINT CONGESTION CONTROL AND ROUTING OPTIMIZATION: AN EFFICIENT SECOND-ORDER DISTRIBUTED APPROACH 3 Srcf Srcf 3 n n 3 Srcf n n 6 Dstf n 4 Fig.. An iustrative exampe of the networ mode. n 5 Dstf 3 n 7 Dstf can be determined in cosed-form distributivey cf. Eq. 35. III. NETWORK MODEL AND PROBLEM FORMULATION We first introduce the notation stye in this paper. We use bodface to denote matrices and vectors. We et A T denote the transpose of A. Diag {A,..., A N represents the boc diagona matrix with A,..., A N on its main diagona. We et A ij represent the entry in the i-th row and j-th coumn of A and et v m represent the m-th entry of v. We et I K denote the K-dimensiona identity matrix, and et K and 0 K denote the K-dimensiona vectors whose eements are a ones and zeros K may be omitted for brevity if the dimension is cear from the context. We et λ min {A and λ max {A denote the smaest and argest eigenvaues of A, respectivey. Networ mode: We consider a time-sotted communication networ system with time sot units being indexed by t 0,,,.... As shown in Fig., we represent the communication networ by a directed graph G {N, L, where N and L are the sets of nodes and ins, with N N and L L, respectivey. We assume that G is connected. There are F endto-end sessions in the networ, indexed by f,..., F. Each session f has a source node and a destination node, represented by Srcf, Dstf N, respectivey. To avoid triviaity, we assume that Srcf Dstf for a f. The data of session f trave from Srcf to Dstf through the networ, possiby via muti-hop and muti-path routing. Congestion contro: As in ], 3], we assume that the source node Srcf has a continuousy-bacogged transport ayer reservoir that contains session f s data, as iustrated in Fig.. Simiar to a vave, in each time-sot t, a transport ayer congestion controer determines the amount of data s f, to be reeased from this reservoir into a networ ayer source queue, where the data awaits to be routed to node Dstf through the networ. In other words, {s f, acts as the arriva process to the source queue. To contro the burstiness, we et s f, s max f, t. We et s f 0 denote the time-average rate at which data of session f is injected at Srcf under congestion T contro, i.e., s f im T T t0 s f,. Each session is associated with a utiity function U f s f, which represents the utiity gained by session f when data is injected at rate s f. We assume that U f is stricty concave, monotonicay increasing, and twice continuousy differentiabe. Routing: We et x f, 0 denote the rate offered to route session f s data in time-sot t at in, as shown in Fig. 3. We Transport Layer Reservoir Congestion Contro s f Source Node Source Queue Fig.. An iustrative exampe of source node congestion contro. On f x, t f q n, t ] ] Lin Lin f x, t ] Node n f q n, t ] f x, t ] Lin 3 f x, t 3 ] Fig. 3. An iustrative exampe of routing at an intermediate node. et x f T im T T t0 xf, represent the time-average routing rate of session f at in. We use s s,..., s F ] T and x f x f,..., xf L ]T to group a congestion contro and session f s routing rates. We denote the capacity of in as C and assume that it is fixed, which is an appropriate mode for wireine networs or wireess networs with orthogona channes and fixed transmission power see, e.g., 3], 4]. As in ], 3], 5], we define the networ capacity region as the argest set of congestion contro rates s such that there exists a routing poicy for which the time-average routing rates { x f, f satisfy the foowing constraints: x f x f + s f f n, f, n Dstf, f In x f, C,, t, where O n and I n represent the sets of outgoing and incoming ins at node n, respectivey; f n is an indicator function that taes the vaue if n Srcf and 0 otherwise. For convenience, we use a node-arc incidence matrix NAIM 6] A f R N L and a source vector b f R N to represent the networ topoogy. Let Tx and Rx denote the transmitting and receiving nodes of in, respectivey. The entries A f n and b f n, n Dstf, are defined as foows: if n Tx, { A f n if n Rx, b f if n Srcf, n 0 otherwise. 0 otherwise, Then, the constraint in can be compacty written as: A f x f s f b f 0, f,,..., F. Queue-stabiity: We assume that each node maintains a separate queue for each session f, as shown in Fig. 3. We et q f n, 0 represent the amount of data in session f s queue at node n at time t. Since data eave the networ upon reaching destinations, we have q f Dstf, 0, t. The evoution of q f n,, n Dstf, is given by: + q f n,t+] q f n, x f, + x f, + s f, f n, 3 On In where + max{0, and x f, is the actua routing rate. Note that x f, x f, since Tx may have ess than x f, amount of data to transmit. Let q q f n,, f, n Dstf] T group a queue engths at time t. In this paper, we

4 4 IEEE/ACM TRANSACTIONS ON NETWORKING adopt the foowing notion of queue-stabiity same as in 3]: Under a congestion contro and routing scheme, we say that the networ is stabe if the norm of steady-state queue-engths remains finite, i.e., im sup t q <. Probem formuation: In this paper, our goa is to deveop an optima joint congestion contro and routing scheme to maximize the tota utiity F f U f s f, subject to the networ capacity region constraints and that the networ is stabe. Putting together the modes presented earier yieds the foowing joint congestion contro and routing JCCR optimization probem: JCCR: Maximize U f s f f subject to A f x f s f b f 0, f, x f, C,, t, f 3 Stabiity of a networ queues, 4 x f, 0, f,, t; s f, 0, f, t. As mentioned earier, severa first-order schemes based on the bac-pressure idea 6] have been proposed e.g., ] 5] to sove Probem JCCR. However, the convergence of these firstorder schemes is sow, which coud ead to poor performance in practice. In what foows, we wi investigate a new secondorder joint congestion contro and routing framewor. IV. A SECOND-ORDER CONGESTION CONTROL AND ROUTING OPTIMIZATION FRAMEWORK In Section IV-A, we first present our second-order joint congestion contro and routing agorithm aong with the main resuts on utiity-optimaity and queue-stabiity. Then, in Section IV-C, we expain the design rationae of our second-order approach. Section IV-D focuses on performance anaysis and provides the proofs for the main theorems in Section IV-A. In Section IV-B, we discuss the ey insights and intuition reated to the resuts in Section IV-A. A. The Agorithm and Main Theoretica Resuts In this subsection, we present the main agorithm and associated theoretica resuts. First, we use y to denote a instantaneous joint congestion contro and routing decisions at time t, which are arranged in the foowing in-based order: y s, s F,, x, xf,,, x L, L,] xf T. We use M ] B A A L to group a networ topoogy information, where B and A are defined as B Diag{b,..., b F, and A Diag{ a,..., a F, and where in the definition of A, the vector a f is the -th coumn of the matrix A f in Probem JCCR i.e., A f a f, af,..., ] af L. Aso, we et N Diag { 0 T F, T F,..., T F R L+ L+F and c 0, C,..., C L ] T R L+. Then, it can be verified that and can be compacty written as My 0 in each time sot rather than on average and Ny c. Next, we define the foowing µ-scaed barrier augmented objective function: f µ y µ U f s f, f ogs f, f f og C f x f, ogx f,, 4 where µ > 0 is caed the barrier parameter its meaning wi be cear soon in Section IV-C. We et g f µ y and H f µ y denote the gradient vector and Hessian matrix of f µ evauated at y, respectivey. Next, we associate with Constraint the dua variabes p f n > 0, f, n Dstf, which pay the roe of prices charged to session f for using node n. Accordingy, et p f n, be the price in time sot t. For convenience, we et p p f n,, f, n Dstf]T group a dua variabes at time t and define a diagona matrix P Diag { p. Now, we define a diagona matrix Q Diag { My that refects the intended queue-ength evoution at time t. To see this, we can expand My to verify that each diagona entry of Q is of the form: On xf, + In xf, + s f, f n, which is amost identica to the actua queueength change in 3, except without the + projection and that a x-variabes are repaced by x-variabes. With this notation and given a stricty feasibe initia soution at t 0 i.e., My 0] < 0, Ny 0] < c, p 0] > 0, our second-order agorithm is iustrated in Agorithm. In Agorithm, the prima variabes y and dua variabes p are updated foowing the Newton directions y in 5 and p in 6, respectivey; both of which expoit not ony the first-order gradient information g, but aso the secondorder Hessian information H, hence the name second-order approach. Aso, the prima variabes y and dua variabes p are jointy updated in 7, thus being a prima-dua scheme. Compared to dua-based controers e.g., ], ], 4], 5], where a couped subprobem defined in terms of prima variabes is soved in each dua iteration, a prima-dua scheme is more convenient for impementation in practice. Note that Agorithm is a dynamic poicy that evoves with time instants and is based on the intended queue-ength evoution Q, which is an easiy observabe networ state in practice. Moreover, as opposed to first-order methods where queue-ength itsef is directy used as a price, Eq. 6 shows that our pricing scheme expoits Q, which corresponds to the intended change of queue-ength. Therefore, the change of queue-ength in our second-order method compared to the queue-ength vaue itsef in first-order bacpressure methods can be viewed as one-order higher in the queueength variation sense, hence providing another perspective to interpret the name second-order. Lasty, the step-size contro in 7 and 8 is used to ensure the utiity-optimaity resut and wi be further expained in Section IV-C. In 7 and 8, the parameter M > 0 coud be set to some upper bound of the average source session rate to reduce the burstiness. The choice of the parameter ɛ wi be addressed shorty in

5 LIUet a.: JOINT CONGESTION CONTROL AND ROUTING OPTIMIZATION: AN EFFICIENT SECOND-ORDER DISTRIBUTED APPROACH 5 Agorithm A second-order joint congestion contro and routing optimization agorithm for a given µ.. In time-sot t, determine the second-order prima congestion contro and routing and dua price Newton directions y and p as foows: y H M T Q P M g M T Q, 5 p MH MT P Q MH g + M T p Q + P ]. 6. Update prima and dua variabes jointy as: ] ] ] yt+] y y + π p t+] p p S M ɛ, 7 where π 0, ] is a constant step-size, and S M ɛ represents the projection onto the set Sɛ M defined as: { Sɛ M y, p ɛ y M, My ɛ,, 8 Ny c ɛ, p ɛ. where the constant ɛ > 0 can be made arbitrariy cose to zero and the constant M > 0 is used for burstiness reduction. Let t t + and go to Step. Theorem 3 beow. Note that, for ease of performance anaysis in this section, the prima-dua Newton directions in 5 and 6 are expressed in matrix form for now. Their expicit distributed computationa schemes wi be derived ater in Section V. The foowing theorem says that the time-average congestion contro rates and routing rates obtained under Agorithm can be made arbitrariy cose to the optima soution by increasing the barrier parameter µ. Theorem Utiity-optimaity. Let ȳ represent the optima average rate soution to Probem JCCR. Under Agorithm and for some given µ, if the step-size π scaes as O µ, then there exists some constant B 0, independent of µ such that im sup T T T t0 y ȳ B µ. For a time-varying positive definite matrix A i.e., the smaest eigenvaue { is{ stricty positive for a t, we et λ min {A inf t λmin A. The foowing resut expains why Agorithm enjoys a fast convergence. Theorem Lyapunov drift rate. If ȳ is outside of ȳ B µ, ȳ + B µ ], where ȳ and B are as defined in Theorem, then there is a negative Lyapunov drift that drives ȳ toward this interva, and the drift rate can be ower bounded by R λ min{h λ min{h M T Q PM. Particuary, R as µ. Theorem indicates that the second-order scaing term H M T Q P M in 5 is crucia to the convergence of Agorithm. Without this term repacing it by an identity matrix I, we essentiay rediscover a first-order bacpressure method with M T Q being the pressure differentia. Thans to this second-order scaing term, the puing force of the negative Lyapunov drift is strong, aowing our scheme to approach the desired region at east as fast as at a rate R that is insensitive to the objective function contour. TABLE I PERFORMANCE SCALINGS COMPARISONS. st-order st-order nd-order Prima-dua: 3] Dua: ], 5] Optimaity gap O µ O O V V Queue-ength Oµ OV OV Step-size O µ O V O V In contrast, the Lyapunov drift rate in first-order methods can be characterized by inf t {λ min {Diag{ U f s f,, f see, e.g., 3, Eq.3] and discussions thereafter, which is ceary sensitive to the objective function contour and coud be very sma i.e., induce staing. The next theorem states that, under Agorithm, the norm of steady-sate queue-engths q remains finite, and hence inducing queue-stabiity. Theorem 3 Queue-stabiity. Under Agorithm and for a given µ, etting ɛ O/µ, there exists a constant K < that scaes as Oµ such that im sup t q K. The proofs of Theorems,, and 3 wi be given in Section IV-D. In what foows, we first discuss the performance scaings and the design rationae of Agorithm. B. Performance Scaings of Agorithm Simiar to most first-order methods, Theorems and 3 impy a trade-off between optimaity gap and queue-ength hence deay. Specificay, athough having a fundamentay different agorithmic meaning, the barrier parameter µ in our secondorder method pays a simiar roe in characterizing the tradeoff compared to the subgradient step-size scaing factor of the first-order methods e.g., V in ], K in 3], and β in 5]. Thus, we summarize the performance scaings of firstand second-order methods in Tabe I a parameters in firstorder methods are standardized to V. First, we see that a schemes have the same inear queueength scaing. Second, the optimaity gap scaing in 3] and our wor are simiar due to the common prima-dua nature. However, the O V -scaing in 3] is achieved at a sower convergence performance and under a more restrictive stepsize scaing described next. For dua-based controers in ], 5], the optimaity gap scaes as O V. Athough this resut appears to be better at first gance, a coser oo reveas that such a direct comparison cannot be made. In ], 5], the gap is measured by in our notation f U f s f f U f s f, i.e., the gap of utiity vaue. In contrast, Theorem measures the gap by T T t0 y ȳ, i.e., the coseness to the optima soution. For convenience, we et ȳ T T t0 y. As shown in Fig. 4, for an increasing stricty concave function U, even if Uȳ is in order sense cose to Uȳ, it is sti uncear how sma ȳ ȳ is in genera, and this distance coud be arge e.g., if U og. In contrast, Theorem directy characterizes ȳ ȳ. Due to the concavity of U, a sma ȳ ȳ guarantees a near-optimaity in the objective vaue. For step-size scaing, we can see that, for the first-order prima-dua scheme in 3] to approach optimaity, the stepsize shoud scae as O V, which is much smaer than our O µ -scaing and impies a very sow convergence. For duabased controers ], ], 4], 5], athough there is no direct

6 6 IEEE/ACM TRANSACTIONS ON NETWORKING Uȳ Uȳ U Fig. 4. Iustration of a monotonicay increasing stricty concave utiity function U and the reationship between Uȳ Uȳ and ȳ ȳ. prima-dua step-size counterpart, the dua step-size scaing therein can be understood as O V, simiar to our O µ. However, this O V -scaing is obtained under the dua-based architecture, which is more cumbersome to impement due to the couped inner prima subprobem. Lasty, we remar that the O µ step-size scaing is not restrictive in impementations since it is ony a sufficient condition to estabish Theorem. Given that the proof of Theorem see Section IV-D is a imiting argument and the constant B in Theorem may not be tight, the choice of the hidden constant in our O µ -scaing does not have to be sma. C. The Rationae behind the Agorithmic Design Agorithm is inspired by, and mirrors, a prima-dua interior-point method for directy soving Probem JCCR in terms of average rates s f and x f. In what foows, we outine the main steps in our agorithmic design. Step A perturbed KKT system: As in standard interiorpoint methods 0], we first reformuate Probem JCCR by appying a ogarithmic function to a inequaity constraints and then accommodating them in the objective function to obtain the foowing barrier objective function to be minimized: ˆf 0 µ ȳ µ µ U f s f µ f og s f µ f f n Dstf f og On ȳ ȳ og C og x f x f In y f x f x f s f f n, Then, we can rewrite Probem JCCR as the foowing unconstrained optimization probem: R-JCCR: Minimize ˆf 0 µ ȳ, 9 where, as µ, the origina objective function of Probem JCCR dominates the barrier functions, and hence the soution of Probem R-JCCR approaches that of Probem JCCR asymptoticay 3], 0]. Next, we tae the first derivatives 0 of ˆf µ ȳ and set them equa to zero i.e., by way of the first-order Karush-Kuhn-Tucer KKT condition to obtain: 0 ˆf µ ȳ U s f s f µ s f x f x f s f 0, 0 µ OSrcf ISrcf ˆf 0 µ ȳ x f µ µ OTx ORx µc F f xf µx f + x f s f f Tx x f x f ITx IRx x f s f f Rx 0. In 0 and, with respect to the fina terms, we define dua variabes aso caed barrier mutipiers, see 3, Section 3.] as foows: ˆp f n µ On xf In xf s f f n. Note that when ȳ is stricty prima feasibe, we have ˆp f n > 0. The purpose of introducing the ˆp f n -variabes in is to render a perturbed KKT system, which enabes the subsequent queuing design and anaysis. Toward this end, we use the vector p ˆp f n, f, n Dstf] T to group a dua variabes. Aso, we et ˆfµ ȳ µ f µȳ cf. 4 for the definition of f µ. Substituting in 0 and and then using ˆf µ ȳ, aong with p and the property of M, we arrive at the foowing perturbed KKT system that contains stationarity ST, prima feasibiity PF, dua feasibiity DF, and perturbed compementary sacness CS conditions: ST: ˆf µ ȳ + M T p 0, PF: ȳ > 0, Mȳ < 0, DF: p > 0, CS: Diag {Mȳ p /µ. Compared to the standard form of KKT conditions 7], the ony difference in this perturbed KKT system is that the righthand side RHS of the CS condition is changed from 0 to µ. As a resut, as µ, the perturbed KKT point ȳ, p amost satisfies the standard KKT conditions, impying nearoptimaity. For more convenient agebraic derivations, we et p µ p absorb the µ-factor and wor with the foowing µ- scaed perturbed KKT system in the rest of the paper: µ-st: f µ ȳ + M T p 0, 3 µ-pf: ȳ > 0, Mȳ < 0, 4 µ-df: p > 0, 5 µ-cs: Diag {Mȳ p. 6 Step Second-order Newton s method: We wi now appy prima-dua-based Newton s method a second-order method to determine a prima-dua pair ȳ, p that satisfies the perturbed KKT conditions 3 6. Simpy speaing, for our probem, the prima-dua Newton s method wors as the foowing iterative search scheme starting from some initia feasibe soution ȳ 0, p 0 : ] ] ] ȳ+ ȳ p + p + π ȳ p, 0, 7 where π is a step-size; ȳ and p denote the prima and dua Newton directions, respectivey.

7 LIUet a.: JOINT CONGESTION CONTROL AND ROUTING OPTIMIZATION: AN EFFICIENT SECOND-ORDER DISTRIBUTED APPROACH 7 We first wor with the µ-st and µ-cs conditions, whie the µ-pf and µ-df conditions wi be handed ater expicity when determining the step-size. Note that finding a primadua pair ȳ, p that satisfies the µ-st and µ-cs conditions amounts to computing the roots of a noninear equaity system consisting of 3 and 6, which does not have anaytic soutions in genera and necessitates numerica methods. By using the Newton s method for root-finding 0], one can compute the Newton direction ȳ T, p T ] T as: ] ȳ H P M M T Q p ] g + M T p P Q + I ],8 where we et g f µ ȳ, H f µ ȳ, P Diag { p, and Q Diag { Mȳ. Note that, due to the perturbed KKT conditions, 8 is different from the Newton systems in existing second-order methods cf. 9, Eq.4], 0, Eq.8],, Eq.9]. Aso, directy soving 8 is undesirabe due to its compex structure. A better way for soving 8 is to derive a reduced inear system by Gaussian eimination to obtain assuming p > 0 and hence P is non-singuar, which can be ensured by the step-size contro described next: ȳ H M T Q P M g M T Q, 9 p MH MT P Q MH g + M T p Q + P ]. 0 Now, it is not difficut to recognize the structura simiarity between 5 6 and 9 0. Next, we hande the µ-pf and µ-df conditions by step-size contro. In standard prima-dua interior-point methods 3], the step-size contro is based on two rues: The first one is to satisfy prima-dua feasibiity by finding: π max π 0, ] ȳ + π ȳ ɛ, Mȳ + π ȳ ɛ, Nȳ + π ȳ c ɛ, p + π p ɛ,, where ɛ > 0 is some arbitrariy sma constant. Note that a fu Newton step is taen if π. The second step-size seection rue is to guarantee a decreasing residua: Let r µ ȳ, p g + M T p T, P Q T ] T be the residua of µ-st and µ-cs at ȳ i.e., the right-hand side RHS of 8. The second rue is to choose π to satisfy 3]: r µ ȳ +, p + < r µ ȳ, p. Under the step-size rues in and, the second-order convergence speed anaysis foow from standard prima-dua interior-point methods see 3, Chap. 5] and 7]. Step 3 Bac to Agorithm : Now, we can see that Agorithm indeed mimics the foregoing approach to adjust y in every time-sot, rather than the average rate ȳ. Moreover, Agorithm has a much simpified step-size seection rue: We do not require a deicate ine search to determine π as in and have the residuas r µ ȳ, p decrease as in, both of which are expensive to chec due to a arge number of gradient and constraint evauations in each time-sot. Rather, we use a fixed step-size π 0, ] and a projection to maintain prima-dua feasibiity a basic requirement in an interiorpoint method. Surprisingy, even with this much simpified and reaxed step-size rue, we are sti abe to show that the time-average of {y, p t0 converges to a bounded region around the optima soution as indicated in Theorem, which is exacty the goa of Probem JCCR. D. Proofs of the Main Theorems In this section, we provide setched proofs for the theorems in Section IV-A for better readabiity. More detaied proof derivations can be found in the appendices. Setch of the proof of Theorem. The main idea and ey steps for proving Theorem are as foows. First, we consider the one-sot drift of the foowing particuar choice of quadratic Lyapunov function: V y, p π y ȳ + p p, µ 3 π which can be interpreted as measuring the unscaed distance between a prima-dua iterate y, p and a perturbed KKT point ȳ, p satisfying 3 6. For simpicity, we et F and G be defined as foows: F H M T Q P M and G MH MT P Q. Then, after some agebraic derivations and upper-bounding see Appendices A-A and A-B for detaied derivations, we obtain the foowing reationship: V y, p V yt+], p t+] V y, p R y ȳ + πb + µ B + µ B 3, 3 where R λmin{h λ min{f > 0 see Eq. 55 in Appendix A-A for detaied derivations and B, B, and B 3 are some positive constants as defined in 59, 69, and 7 in Appendix A, respectivey. Aso, based on 59, 69, and 7 in Appendix A, we can concude that B, B, and B 3 are independent of µ. It can be seen from 3 that if π O/µ, we have V y t+], p t+] V y, p R y ȳ + B, µ where B αb + B + B 3 for some α > 0. Teescoping T via one-sot drift expressions for t 0,..., T yieds: V T y T ], p T ] V y0], p 0] R t0 y ȳ + T B. µ Next, dividing both sides by T R, rearranging terms, and T taing T to infinity, we have im sup T T t0 y ȳ B µ, where we et B B/R. Then, the proof is compete because when T is arge, we have T T t0 y ȳ a T T t0 y ȳ B, µ where inequaity a foows from the trianguar inequaity and the basic reationship between - and -norms. We note that the most chaenging step in the proof ies in the one-sot drift anaysis, where we repeatedy expoit the ey reationships in the perturbed KKT system in 3 6. We reegate the derivation detais to Appendix A. Proof of Theorem. First, from 3, we have that the drift rate R is given by: R λ min{h λ min {F inf t{λ min {H inf t {λ min {F.

8 8 IEEE/ACM TRANSACTIONS ON NETWORKING Reca that F is defined as F H M T Q P M. From 4, it is cear that a entries in H i.e., the Hessian matrix of f µ y grow to infinity as µ. However, from the µ-cs condition, where we have et p µ p to absorb the µ-factor, we have that Q P is independent of µ. Aso, the matrix M is determined by networ topoogy and independent of µ. Hence, we have that the term H dominates M T Q P M as µ grows to infinity. That is, F H as µ. Aso, from the strict convexity of f µ and the boundedness of y, we have that H is positive definite for a t, i.e., λ min {H > 0. Hence, we have λmin{h λ as min{f µ. Proof of Theorem 3. The basic idea to prove Theorem 3 is based on anayzing the one-sot drift of the foowing Lyapunov function: ˆV q q. For convenience, we et ŷ s, s F,, x, xf,,, x L, xf ] T L, group a source and actua routing rates. Note that ŷ y since x f. Then, the queuing dynamic can be written, xf, as q t+] q + Mŷ and the one-sot drift ˆV can be bounded as: ˆV q t+] q q T Mŷ + ŷt MT Mŷ q T Mŷ + yt MT My a q T My +NL max {C + yt MT My, 4 where inequaity a is due to 3, Lemma ]. Now, we et B 4 NL max {C + λ max{m T M sup t { y. Note that NL max {C and λ max {M T M are determined by the networ topoogy and sup t { y max{m, max C. As a resut, B 4 depends ony on the networ and is independent of µ. On the other hand, according to our step-size contro in 8 and that ɛ O µ, we have My β µ for some β > 0. Therefore, we have ˆV q T My + B 4 β µ qt + B 4 β µ F f n Dstf qf n + B 4. 5 So it foows that when F f n Dstf qf n µ β B 4 + ɛ, where ɛ > 0 is some constant, we have ˆV q ɛ, i.e., the first term in 5 dominates B 4 and resuts in a negative drift when the tota queue ength is arge. Next, we caim that the foowing reationship is true: im sup t ˆV q µ β B 4 + ɛ + B 4. 6 This caim can be shown by the foowing argument: First, suppose that ˆV q µ β B 4 + ɛ. From 5, we now that q T My 0, which further impies that ˆV q < B 4. As a resut, we have ˆV q t+] ˆV q + ˆV q µ β B 4 + ɛ + B 4, i.e., 6 is true. On the other hand, suppose that ˆV q > µ β B 4 + ɛ. From the basic reationship between - and -norms, we have ˆV q F f n Dstf qf n. This impies that if ˆV q > µ β B 4 + ɛ, we have ˆV q ɛ. This means that ˆV qt+] < ˆV q and that the sequence { ˆV q wi monotonicay decrease at a rate at east ɛ. Therefore, there exists a time t such that ˆV q t ] µ β B 4 + ɛ, and then the rest foows from the earier discussions in the case where ˆV q µ β B 4 +ɛ. Finay, we et K ] µ β B 4 + ɛ + B 4 and note that K scaes as Oµ. Then, the resut stated in the theorem foows by mutipying both sides of 6 by two and taing the square root. This competes the proof. So far, we have designed a second-order joint congestion contro and routing agorithm and estabished its utiityoptimaity and queue-stabiity. However, given the more compex computationa scheme in Agorithm, one question begs to be answered: Can we design a distributed agorithm based on the proposed second-order method? Moreover, athough it is convenient to express 5 and 6 in matrix equations, they are cumbersome to use and more expicit scaar-based expressions are desired for impementations in practice. These issues constitute the main discussions in the next section. V. SECOND-ORDER DISTRIBUTED ALGORITHM DESIGN In this section, our main goa is to decentraize the proposed second-order method in Section IV. Note that the main computationa compexity in 5 and 6 stems from the foowing two dense matrix inverse computations that require goba networ information: F G H M T Q M P, 7 MH MT P Q. 8 Thus, our effort in this section is centered around tacing these two chaenges. We first derive an aternative way for computing the prima and dua Newton directions in Section V-A. Next, we deveop distributed computationa schemes for the prima and dua Newton directions in Sections V-B and V-C, respectivey. A. Aternative Computationa Scheme for Newton Directions Our first step towards designing a second-order distributed method is to simpify the computationa schemes in 5 and 6. The rationae behind this simpification is due to the foowing observation: Whie 5 and 6 ceany express y t+] and p t+] in terms of y and p and enabe a the subsequent utiity-optimaity and queue-stabiity anaysis, they aso mae the computationa schemes unnecessariy more compex for practica impementations. Toward this end, we estabish the foowing emma that wi be usefu in Sections V-B and V-C: Lemma 4. The prima and dua Newton directions in 5 and 6 can be aternativey computed as foows: y H g + M T p t+], 9 p p t+] p, 30

9 LIUet a.: JOINT CONGESTION CONTROL AND ROUTING OPTIMIZATION: AN EFFICIENT SECOND-ORDER DISTRIBUTED APPROACH 9 where p t+] is obtained by starting from p and taing a fu Newton step i.e., π and can be computed as: ] p t+] G MH g + P. 3 The basic idea here is that, through the use of an auxiiary variabe p t+], we obtain simper expressions in 9 and 30. Ceary, 30 foows from the definition of p t+]. The expressions in 9 and 3 can be derived by soving for y and p from 8 changing the indices from to and repacing p + p by p t+]. With Lemma 4, we are now in a position to derive a distributed scheme for computing prima and dua Newton directions. B. Distributed Computation of the Prima Newton Direction The first advantage of using the new scheme in 9 is that, instead of having to dea with F, which is an unstructured and dense matrix, we are now faced with H, which has the foowing nice boc diagona structure: H Diag { S, X,,..., X L,, where S R F F is a diagona matrix defined as { S Diag µu f s f, + s, f,..., F, 3 f, and where X R F F is a symmetric matrix with entries defined as foows: δ X, f,f + x f if f f, 33 δ if f f, where δ C F f xf represents the unused in capacity of in in time-sot t. It then foows from the boc diagona structure of H that { H Diag S, X,,..., X L,. 34 We note that this boc diagona structure of the Hessian is exacty the same as that in 0, Section V-C] after repacing the ong-term average rates by instantaneous rates in each time-sot t. Due to the same structure as their counterparts in 0], S and X, can be computed in cosed-form by using Lemma 4 and Theorem 5 in 0]. Furthermore, by noting the simiarity in structure between 9 and the prima Newton direction scheme in 0, Eq. 9], we immediatey have the foowing resut for second-order congestion contro and routing update directions the proof mirrors that of 0, Theorem 6] and is omitted for brevity: Theorem 5. Let x be defined as in 0, Theorem 6]. Given dua prices p t+], the congestion contro and routing directions s f, and x f, can be computed in cosed-form using oca information at each source and each in as foows omitting time-sot indexes and t + ] for simpicity: s f s f µsf U f s f + s f p f Srcf µs f U f s, f, 35 f x f + x f f, f x f x + p f x f δ Tx pf Rx x f x x f δ + p f Tx pf Rx ],, f. 36 Remar. Theorem 5 has two interesting networing interpretations. First, the dua price differentia p f Tx pf Rx in 36 pays a simiar roe of the queuing bacog differentia in the bac-pressure schemes. The main difference is that x f i.e., to increase or decrease x f, is based on not ony the pressure differentia of session f, but that of a sessions in in. Moreover, unie the winner-tae-a poicy in the bac-pressure schemes i.e., the session with the argest bacog differentia uses up the in capacity, our secondorder approach is more democratic in that every session gets a share of the in capacity as indicated in 36. C. Distributed Computation of the Dua Newton Direction Reca that the dua Newton direction p can be computed by first soving for p t+] in 3. However, one technica chaenge remains: the term MH MT in G cf. 8 is a dense weighted Lapacian matrix 8]. Thus, it is intractabe to derive a distributed cosed-form expression for G. One possibe approach to hande this chaenge is to borrow the matrix-spitting idea from 9] ] to compute p t+] iterativey. This is because the term P Q in 8 is diagona and can be absorbed into MH MT. Thus, the matrix-spitting scheme in 0], ] can be adopted with some minor modifications. The most appeaing feature of the matrixspitting scheme is that it ony requires one-hop information exchange. However, the main drawbac of this approach is that the obtained soution is an approximation and ony converges to the precise vaue asymptoticay. This issue is even more pronounced in a time-sotted system as it entais a time-scae separation assumption. To overcome this imitation, in this paper, we propose a new iterative approach based on the Sherman-Morrison-Woodbury SMW matrix inversion. Sherman-Morrison-Woodbury matrix inversion approach. The basic idea of the SMW-based approach is that, instead of computing G indirecty by spitting, we directy update G using the SMW matrix inversion emma 7] restated as foows: Lemma 6 SMW matrix inversion. For any invertibe matrix Ω and vectors u, v of conformabe dimension, if + v T Ω u 0, then Ω + uv T can be computed as: Ω + uv T Ω Ω uv T Ω + v T Ω u. 37 Note that if Ω is nown and the target matrix can be written as Ω couped with a ran- update, then the formua in 37 provides a numericay cheap way to compute the resut by a ran- correction based on Ω.

10 0 IEEE/ACM TRANSACTIONS ON NETWORKING In what foows, we outine the ey steps of our SMW approach omitting time-sot index for simpicity. First, we decompose G as G BS B T P Q + A X A T 38 using the structures of M and H cf. 34. Note that the term BS B T P Q in 38 is diagona and can be written as: { BS B T P Q Diag p f n s f f n + µu f s f + s f On x f In f n, f F x f. 39 We et D BS B T P Q Diag { D,..., D F denote this diagona matrix. Then, D can be readiy computed at each source node in a distributed fashion. Next, we note that the term L A X A T in 38 can be further decomposed as: A X AT { { Diag x f a f a f T, f x u u T, where a f is as defined in Section IV-A and u f x a f T, f,..., F ] T. Note that the first term L Diag{xf a f a f T, f has exacty the same boc-diagona structure as in D. Thus, we can merge it with D to obtain a new boc-diagona matrix D Diag {D,..., D F, where each boc D f is of the form: D f D f + x f a f a f T. 40 Now, it is important to recognize from 40 that D f can be viewed as appying ran- updates L times to D f. This motivates us to start from D f and appy the SMW inversion L times to compute D f. Let D f, ] denote the intermediate resut before appying the -th SMW-correction. Aso, et D f,0] D f. Then, we have the foowing computationa scheme based on Lemma 6: For ins,..., L, D f,] D f, ] D f, ] xf + x f a f a f T D f, ] a f T D f, ] af. 4 Ceary, after L times of SMW-corrections, we achieve D Diag { D,..., D F. Next, it is important to recognize that G D x u u T, 4 which can be viewed as appying ran- updates L times for D. Given that D has just been computed, we can appy the SWM inversion emma another L times to compute G starting from D. Toward this end, et K ] denote the intermediate resut before appying the -th SMW-correction. Aso, et K 0] D. Then, we have the foowing computationa scheme: K ] K ] + K ] u u T K ] x u T K ] u,,..., L. 43 Finay, with the aforementioned L SMW-corrections in tota, we achieve the precise vaue of G, which can in turn be used to compute p t+] and p. We summarize the SMW-based approach in Agorithm. Agorithm SMW-based approach for dua Newton direction. Initiaization:. For each node, compute the corresponding entries in D Diag { D f, f using 39 and input the resut to the starting in. Main Iteration:. For a f,..., F, et D f,0] D f. For ins,..., L, update D f,], f, using 4. Let D Diag{D,L],..., D F,L]. 3. Let K 0] D. For ins,..., L, update K ] using 43. Let G K L] and stop. Remar. Since each SMW-correction ony invoves information ocay avaiabe at one in, the scheme can proceed foowing any pre-determined in ordering in a distributed fashion. Unie the matrix-spitting approach that converges asymptoticay, we require exacty L SWM-corrections to obtain the precise vaue of G. Thus, the SMW-based approach is far more efficient. However, since the SMW-based approach invoves a L ins, the scae of information exchange is arger than the -hop scae required by the matrix-spitting approach and depends on the networ diameter. To see this, without oss of generaity, et the ins be ordered such that they simpy foow the in abes,..., L. Consider the -th step in 4 or 43, which ony requires intermediate resut from the - st step and the oca x f -information at in. In the next step, in needs to send its computed resut to in +. Consider the extreme case where in and in + are separated from each other by the argest possibe number of hops in this networ. In this case, it is cear that the required number of hops for sending information from in to in + is equa to the networ diameter, i.e., the argest number of hops between any pair of nodes in the networ. Fortunatey, many communications networs in practice are constructed in a hierarchica fashion such that the networ diameter is sma. VI. NUMERICAL RESULTS In this section, we conduct numerica studies to verify the efficacy of our proposed second-order joint congestion contro and routing agorithm. To iustrate the detais of our second-order approach, we first use a sma five-node twosession networ exampe as shown in Fig. 5: there are two sessions in the networ: N to N3 and N4 to N. Each in in the networ has unit capacity. We use ogs f as the utiity function, i.e., the we-nown proportiona fairness metric 9]. We set µ 000, meaning that the accuracy

11 LIUet a.: JOINT CONGESTION CONTROL AND ROUTING OPTIMIZATION: AN EFFICIENT SECOND-ORDER DISTRIBUTED APPROACH s : N N3 f : N N f : N N s : N4 N f : N4 N f : N4 N Fig. 5. A five-node two-session networ. Fig. 6. The routing soutions for session N N3. Fig. 7. The routing soutions for session N4 N Session : N N3 Session : N4 N.. Source Rates nd order Source Rates.5 Prima dua st order Source Rates nd order Session : N N3 Session : N4 N Time Sots 0.5 Dua based st order Time Sots Fow : N N3 Fow : N4 N Iteration Fig. 8. Convergence process of the second-order agorithm µ 000. Fig. 9. Convergence process of the first-order methods V 000. Fig. 0. Convergence process of the second-order agorithm µ 50. Source Rates Session : N N3 Session : N4 N Prima dua st order Dua based st order Time Sots Error Error SMW Scheme Linear Scae Matrix Spitting Number of Iterations Log Scae SMW Scheme Matrix Spitting Number of Iterations Average tota queue ength First order Second order Mean arriva rate per session Fig.. Convergence process of the first-order methods V 50. Fig.. Performance comparison between the SMW and matrix-spitting inversion schemes. Fig. 3. Average tota queue-ength vs. mean arriva rate µ V 000. of the µ-cs condition is on the order of 0 3. First, the routing soutions for Sessions and are shown in Fig. 6 and Fig. 7, respectivey. The convergence behavior is iustrated in Fig. 8. It can be seen from Fig. 8 that the source rates not just the average source rates rapidy converge to the foowing pair s , s.047 in approximatey 5 iterations. This shows the efficiency of our proposed second-order agorithm. To compare the convergence performance with the firstorder schemes, we aso used the same networ exampe in Fig. 5 to experiment with both prima-dua 3] and dua based first-order schemes ], ]. For a fair comparison, both firstorder bac-pressure based schemes were started from the same prima and dua initia points. Targeting approximatey the same eve of accuracy, we set the step-size scaing factor, denoted as V, as V 000 see the discussions in Section IV-B. The convergence performances of both primadua and dua based first-order schemes are iustrated in Fig. 9. We can see from Fig. 9 that in order to achieve soutions with high accuracy, both the first-order prima-dua and dua based schemes converge after approximatey 5000 iterations. This shows that our second-order scheme converges at east three orders of magnitude faster than the first-order schemes. We can aso observe that the iterates of the prima-dua based scheme in the first-order domain evove ess abrupty as compared to the dua-based scheme, but aso converge more sowy. To see the impacts of µ and V on the second-order and first-order methods, we et µ 50, V 50, and run another experiment on the networ in Fig. 5. As shown in Fig. 0, we can see that when µ is smaer, our second-order scheme converges even faster ess than 0 iterations but at the cost of a arger optimaity gap. On the other hand, as shown in Fig., with V 50, the convergence of the first-order methods can be made faster approximatey 900 iterations, but this yet exhibits much arger fuctuations. In this case, we can sti observe that our second-order scheme converges amost two orders of magnitude faster than the first-order schemes. Again, we see that the iterates in the prima-dua based scheme evove ess abrupty with fewer fuctuations, but converge sower. However, regardess of the choice of first-order scheme and the vaue of V, the evoution of the iterates under both first-order schemes are much ess efficient compared to that obtained

12 IEEE/ACM TRANSACTIONS ON NETWORKING under our proposed second-order scheme. Next, we compare the convergence performance between our proposed SMW-based matrix inversion scheme with the matrix-spitting scheme 9] ] in computing dua price updates. For the netwro exampe in Fig. 5, the convergence processes of our SMW and the matrix-spitting schemes are iustrated in Fig. the top and bottom haves are in inear and og scaes, respectivey. As expected, for this 7-in exampe, the SWM scheme obtains the precise soution in 7 4 iterations. In contrast, due to its asymptotic nature, matrix-spitting taes over 400 iterations for the approximation error to reach the 0 6 precision, which is not very satisfactory even in this sma-sized networ exampe. Lasty, the simuation resuts of average tota queue-engths vs. mean arrive rates are iustrated in Fig. 3, where we can see that the deay performance of our second-order scheme significanty outperforms that of the first-order methods more than three orders of magnitude ower. This arge deay performance gap is a direct consequence of the sow convergence of the first-order methods. VII. CONCLUSION In this paper, we have deveoped a new second-order agorithmic framewor for joint congestion contro and routing optimization. Unie most joint congestion contro and routing methods in the iterature, our proposed agorithmic framewor fundamentay deviates from the cassica bac-pressure idea to offer not ony utiity-optimaity and queue-stabiity, but aso fast convergence and ow deay. Our main contributions in this paper are three-fod: i We have proposed a secondorder joint congestion contro and routing framewor based on a prima-dua interior-point approach that is we-suited for impementation in practice; ii we have rigorousy estabished the utiity-optimaity and queue-stabiity of the proposed second-order joint congestion contro and routing framewor; and iii we have proposed severa nove approaches for the distributed impementation of our second-order joint congestion contro and routing optimization agorithm. Coectivey, these resuts serve as an exciting first step toward an anaytica foundation for a second-order joint congestion contro and optimization theory that offers fast convergence performance. Second-order cross-ayer optimization for networ system sis an important and yet under-expored area. Future research topics may incude extending and generaizing our proposed second-order agorithmic framewor to appications in other networ systems, such as wireess networs with stochastic channe modes, coud computing resource aocations, and energy production scheduing in the smart eectric power grid. APPENDIX A PROOF OF THEOREM We first show a basic property of the dua sequence {p t0 that wi be usefu in proving Theorems. Lemma 7. For a given µ and under Agorithm, if p 0] <, then p < for a t. Proof. We prove Lemma 7 resut by induction. For t 0, the resut is triviay true by assumption. Suppose that at time sot t we have p < B < ; we wi show that p t+] is aso bounded. We et p t+] p + p, i.e., we et π. After some agebraic derivations, we have: p t+] MH MT P Q MH g + P ]. Now, we caim that p t+] is bounded. This is true because ] p t+] MH MT P Q MH g +P a ] MH MT MH g + P b λ min {MH MH + P MT c λ min {MH MT g λ min {H Mg + P,44 where a hods because of the strict feasibiity of y and p and hence P Q is a positive definite diagona matrix, which can ony increase the eigenvaues of MH MT ; b foows from trianguar inequaity and taing the smaest eigenvaue of MH MT and factoring it outside the norm; and c foows from factoring λ min {H outside the norm. Since g is continuous, the spectra radius ρh is bounded. Aso, since M is constructed by the NAIM of a connected graph, ρh is aso finite. As a resut, λ min {MH MT must be finite. Aso, since s f, and x f are stricty bounded away from zero due to the step-size seection rue, we have that g is bounded. Liewise, since p is aso stricty bounded away from 0, we have that P is bounded from above. Therefore, we can concude that the RHS of inequaity c is bounded, i.e., p t+] is bounded. Finay, note that p t+] πp + π p t+] π p + π p t+], where the inequaity foows again from trianguar inequaity. Hence, we concude that p t+] is aso bounded. This competes the proof. As mentioned in Section IV-D, the main idea and the ey steps for proving Theorem are based on the drift anaysis of the foowing Lyapunov function: V y, p y ȳ + p π µ 3 p, π The one-sot drift anaysis reveas the foowing ey reationship: V y, p V yt+], p t+] V y, p R y ȳ + µ B, where R and B are both some positive finite quantities independent of µ. Based on this reationship, the resut stated in Theorem foows from teescoping T via one-sot drifts and then etting T go to infinity. We begin with evauating the one-sot Lyapunov drift V y, p : V y, p V y t+], p t+] V y, p yt+] + y ȳ T yt+] y π 45 + pt+] µ 3 + p p T pt+] p. π 46 In what foows, we wi bound the two expressions in 45 and 46 in Appendices A-A and A-B, respectivey.

13 LIUet a.: JOINT CONGESTION CONTROL AND ROUTING OPTIMIZATION: AN EFFICIENT SECOND-ORDER DISTRIBUTED APPROACH 3 A. One-sot Lyapunov drift of 45 Note that 45 can be further expanded as: 45 y ȳ π F g M T Q ] T πf g M T Q ] y ȳ T F g M T Q 47 + π T g M T Q F g M T Q. 48 We first examine 47, which can be computed as foows: 47 y ȳ T F g M T Q a y ȳ T F g g M T p M T Q b y ȳ T F g g + M T Q M T Q y ȳ T F g g 49 y ȳ T F MT Q Q, 50 where a foows from the fact that g + M T p 0 i.e., the µ-st condition and b foows from the fact that p Q i.e., the µ-cs condition. Note that the foowing reationship foows from 49 and the convexity of f µ : 49 y ȳ T g g. 5 λ min {F By the Mean-Vaue Theorem, we have the foowing pair of reationships: f µ y fµ ȳ + g T y ȳ + y ȳ T Hỹ ] y ȳ, 5 f µ ȳ T f ȳ µ y + g y + ȳ T y Hỹ ] ȳ y. 53 In 5 and 53, Hỹ ] and Hỹ ] represent the matrices evauated at points ỹ and ỹ, where ỹ α y +α ȳ and ỹ α y +α ȳ, for some 0 α, α. Next, adding 5 and 53 yieds: g g T y ȳ y ȳ T Hỹ ]+Hỹ ] y ȳ λ min H y ȳ. 54 Combining 5 and 54, we concude that 49 R y ȳ, 55 where we et R λmin{h λ min{f. Noting that the µ-factors in λ min {H and λ min {F cance each other, we have that R is independent of µ. Now, we evauate the term in 50, which is non-positive because: 50 y ȳ T M T Diag { M y ȳ λ min FΓ λ min FΓ Diag { M y ȳ 0, 56 where Γ is defined as Γ inf t { In xf, +s f, f n On xf, In xf, + s f f n On xf,. By combining 55 and 56, we have that 47 R y ȳ. 57 Next, by upper-bounding the quadratic term 48, we have: π g 48 λ min {F M T Q a π g λ min {F g M T p M T Q b π g λ min {F g M T Q Q c π λ min {F g g + M T Q Q ], 58 where inequaity a utiizes the µ-st condition g +M T p 0 cf. 3; equaity b utiizes the µ-cs condition p Q cf. 6; and inequaity c foows from triange inequaity. Note that the µ-factors in 58 cance each other. Aso, due to the boundedness of y under the agorithmic design and the assumption that U f is differentiabe, we concude that 58 is upper-bounded by some constant. By etting B λ min sup{ g g + M T Q {F Q ] t cf. B in 3 and using 57 and 58, we have R y ȳ + πb. 60 So far, we have finished the one-sot drift anaysis for 45. B. One-sot Lyapunov drift of 46 Now, we move on to anayzing the other term 46 in the one-sot drift, which can be further expanded as foows: 46 µ 3 p p T G MH g + M T p + π µ 3 Q + P ] MH MH g + M T p Q + P ] T G 6 g + M T p Q + P ]. 6 Note that due to the H term in G, G scaes as Oµ. We first anayze 6, which can be further decomposed as: 6 a µ 3 p p T G MH g g µ 3 p p T G MH MT p p + µ 3 p p T G Q + P b µ 3 p p T G MH g g 63 + µ 3 p p T G Q + P, 64

14 4 IEEE/ACM TRANSACTIONS ON NETWORKING where a foows from subtracting the µ-st condition g + M T p 0 and then coecting terms; whie b hods because is positive semidefinite, which impies that G MH MT µ p p T G MH MT p p 0. To study the boundedness of 6, we begin with 64, which can be further computed as foows: 64 a µ 3 p p T G Q Q P b µ 3 p p T G M y y + µ 3 p p T G P P + P ] c µ 3 p p T G M y y, 65 where equaity a utiizes the µ-cs condition Q P I i.e., Q P, equaity b utiizes My Q, and inequaity c hods because µ p p T G P P µ Φλ min{g p p T p p 0, where we et Φ inf t,n,f {p f n, pf, n. Next, combining 65 with 63, we have 6 ] µ 3 p p T G M H g g y ȳ.66 By the vector-vaued Tayor expansion of g 30], we have g g + H ȳ y + o y ȳ, which further impies that H g g y ȳ o y ȳ. 67 Therefore, we have 66 µ 3 p p T G M y ȳ a y ȳ p p M µ 3, 68 λ min {G where inequaity a foows from Cauchy-Schwarz inequaity. From the boundedness resut of p in Lemma 7, we have that p p is bounded. From our contro scheme design, we now that the entries in y is upper bounded by min{m, C,. Aso, it can be shown that M F f n Dstf O n I n + f n, which is a networ-specific constant. Hence, we concude that 68 is upper-bounded by some constant. By etting B M { µ λ min {G sup y ȳ p p, 69 t cf. B in 3 where we eave a µ inside the denominator to cance out the µ-factors in p p and λ, we min{g have 66 µ B. As a resut, we can finay bound 6 as: µ B. 70 Lasty, we evauate 6. Noting that π 0, ], we have 6 µ 3 MH g +M T ] TG p Q +P MH g + M T ] p Q + P MH µ 3 λ min {G g +M T p Q + P a M H µ 3 λ min {G g y +MH MT p P b MH µ 3 λ min {G g y + MH MT p c + P ] µ 3 λ min {G MH g g y ȳ + MH g ȳ + MH MT p + P ] µ 3 λ min {G y ȳ M + MH g ȳ + MH MT p + P ], 7 where a uses Q My ; b is due to the trianguar inequaity; and c foows from the same argument in 67 and 68. Note that in 7, y ȳ M is upper-bounded due to the same argument used for defining B ; MH g ȳ is upper-bounded due to the µ-factor canceation between H and g ; and MH MT p is upper-bounded due to: i the boundedness of p from Lemma 7, and ii the µ- factor canceation between H and p. Aso, P diminishes as µ increases. Thus, from the above discussions, we concude that 7 is upper-bounded. By etting { y ȳ M B 3 µ λ sup min {G t + MH g ȳ MH MT p + P ], 7 we have 6 µ B 3. Finay, combining this with resuts in 57, 60, and 70, we arrive at the foowing resut for the one-sot drift anaysis: V y t+], p t+] V y, p R y ȳ + πb + µ B + µ B The remaining steps of the proof incude teescoping via T, rearranging, and taing imit over T, and can be found in Section IV-D. This competes the proof of Theorem. REFERENCES ] X. Lin and N. B. Shroff, Joint rate contro and scheduing in mutihop wireess networs, in Proc. IEEE CDC, Atantis, Paradise Isand, Bahamas, Dec. 006, pp ] M. J. Neey, E. Modiano, and C.-P. Li, Faireness and optima stochastic contro for heterogeneous networs, IEEE/ACM Trans. Netw., vo. 6, no., pp , Apr ] A. Eryimaz and R. Sriant, Joint congestion contro, routing, and MAC for stabiity and fairness in wireess networs, IEEE J. Se. Areas Commun., vo. 4, no. 8, pp , Aug. 006.

15 LIUet a.: JOINT CONGESTION CONTROL AND ROUTING OPTIMIZATION: AN EFFICIENT SECOND-ORDER DISTRIBUTED APPROACH 5 4] X. Lin and N. B. Shroff, The impact of imperfect scheduing on crossayer congestion contro in wireess networs, IEEE/ACM Trans. Netw., vo. 4, no., pp , Apr ] A. L. Stoyar, Maximizing queueing networ utiity subject to stabiity: Greedy prima-dua agorithm, Queueing Systems, vo. 50, no. 4, pp , ] L. Tassiuas and A. Ephremides, Stabiity properties of constrained queuing systems and scheduing poicies for maximum throughput in mutihop radio networs, IEEE Trans. Autom. Contro, vo. 37, no., pp , Dec ] M. S. Bazaraa, H. D. Sherai, and C. M. Shetty, Noninear Programming: Theory and Agorithms, 3rd ed. New Yor, NY: John Wiey & Sons Inc., ] A. Eryimaz and R. Sriant, Fair resource aocation in wireess networs using queue-ength-based scheduing and congestion contro, in Proc. IEEE INFOCOM, Miami, FL, Mar. 005, pp ] E. Wei, A. Ozdagar, and A. Jadbabaie, A distributed Newton method for networ utitity maximization, in Proc. IEEE Conference on Decision and Contro CDC, Atanta, GA, Dec. 5-7, 00. 0] J. Liu and H. D. Sherai, A distributed Newton s method for joint mutihop routing and fow contro: Theory and agorithm, in Proc. IEEE INFOCOM, Orando, FL, Mar. 5-30, 0, pp ] J. Liu, C. H. Xia, N. B. Shroff, and H. D. Sherai, Distributed crossayer optimization in wireess networs: A second-order approach, in Proc. IEEE INFOCOM, Turin, Itay, Apr. 4-9, 03. ] A. Jadbabaie, A. Ozdagar, and M. Zargham, A distributed Newton method for networ optimization, in Proc. IEEE Conference on Decision and Contro CDC, Shanghai, China, Dec. 6-8, ] A. Forsgren, P. E. Gi, and M. H. Wright, Interior methods for noninear optimization, SIAM Review, vo. 44, no. 4, pp , Oct ] M. J. Neey, E. Modiano, and C. E. Rohrs, Dynamic power aocation and routing for time varying wireess networs, IEEE J. Se. Areas Commun., vo. 3, no., pp , Jan ] A. Zymnis, N. Trichais, S. Boyd, and D. OŠNei, An interior-point method for arge scae networ utiity maximization, in Proc. Aerton Conference on Communication, Contro, and Computing, Monticeo, IL, Sep. 6-8, ] D. Bicson, Y. Toc, O. Shenta, and D. Doev, Poynomia inear programming with Gaussian beief propagation, in Proc. Aerton Conference on Communication, Contro, and Computing, Monticeo, IL, Sep. 3-6, 008, pp ] D. Bicson, Y. Toc, A. Zymnis, S. Boyd, and D. Doev, Distributed arge scae networ utiity maximization, in Proc. IEEE Internationa Symposium on Information Theory ISIT, Seou, Korea, Jun.8 Ju.3, 009, pp ] D. Bicson, Gaussian beief propagation: Theory and appication, Ph.D. dissertation, Hebrew University of Jerusaem, ] E. Wei, A. Ozdagar, and A. Jadbabaie, A distributed Newton method for networ utiity maximization i: Agorithm, IEEE Trans. Autom. Contro, vo. 58, no. 9, pp. 6 75, Sep ] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, UK: Cambridge University Press, 004. ] M. Zargham, A. Ribeiro, and A. Jadbabaie, Acceerated bacpressure agorithm, 03. Onine]. Avaiabe: ] A. Ribeiro, Stochastic soft bacpressure agorithms for routing and scheduing in wireess ad-hoc networs, in Proc. IEEE CAMSAP, Aruba, Dutch Anties, The Netherands, Dec. 3-6, ] M. Kodiaam and T. Nandagopa, Characterizing achievabe rates in muti-hop wireess mesh networs with orthogona channes, IEEE/ACM Trans. Netw., vo. 3, no. 4, pp , ], Characterizing the capacity region in muti-radio muti-channe wireess mesh networs, in Proc. ACM Mobicom, Coogne, Germany, Aug. 005, pp ] M. J. Neey, E. Modiano, and C. E. Rohrs, Power aocation and routing in mutibeam sateites with time-varying channes, IEEE/ACM Trans. Netw., vo., no., pp. 38 5, Feb ] M. S. Bazaraa, J. J. Jarvis, and H. D. Sherai, Linear Programming and Networ Fows, 4th ed. New Yor: John Wiey & Sons Inc., 00. 7] A. S. E-Bary, R. A. Tapia, T. Tsuchiya, and Y. Zhang, On the formation and theory of the Newton interior-point method for noninear programming, J. Optim. Theory App., vo. 89, pp , ] F. R. K. Chung, Spectra Graph Theory. Providence, RI: American Mathematica Society, ] X. Lin, N. B. Shroff, and R. Sriant, A tutoria on cross-ayer optimization in wireess networs, IEEE J. Se. Areas Commun., vo. 4, no. 8, pp , Aug ] W. Rudin, Ed., Principes of Mathematica Anaysis. New Yor, NY: McGraw-Hi, 976. Jia Liu S 03 M 0 received his Ph.D. degree in the Bradey Department of Eectrica and Computer Engineering at Virginia Tech, Bacsburg, VA in 00. He is currenty a Research Assistant Professor in the Department of Eectrica and Computer Engineering at The Ohio State University. His research focus is in the areas of optimization of communication systems and networs, theoretica foundations of cross-ayer optimization on wireess networs, design of agorithms, and information theory. Dr. Liu was a Member of Technica Staff at Be Labs, Lucent Technoogies in Beijing China from 999 to 003. Dr. Liu is a member of IEEE and SIAM. He is a reguar reviewer for major IEEE conferences and journas. He was a recipient of the Be Labs President God Award in 00, Chinese Government Award for Outstanding Ph.D. Students Abroad in 008, and the IEEE ICC 008 Best Paper Award. His received the Best Paper Runner-up Award in IEEE INFOCOM 0 and the Best Paper Runner-up Award in IEEE INFOCOM 03. He has served as a TPC member for IEEE INFOCOM since 00. Ness B. Shroff S 9 M 93 SM 0 F 07 received his Ph.D. degree in Eectrica Engineering from Coumbia University in 994. He joined Purdue university immediatey thereafter as an Assistant Professor in the schoo of Eectrica and Computer Engineering. At Purdue, he became Fu Professor of ECE in 003 and director of CWSA in 004, a university-wide center on wireess systems and appications. In Juy 007, he joined The Ohio State University, where he hods the Ohio Eminent Schoar endowed chair in Networing and Communications, in the departments of ECE and CSE. Dr. Shroff currenty serves as editor at arge at the IEEE/ACM Trans. on Networing and on the editoria boards of the IEEE Trans. on Contro of Networ Systems, IEEE Networ Magazine, and the Networing Science journa. He has chaired various conferences and worshops, and co-organized worshops for the NSF to chart the future of communication networs. Dr. Shroff is a Feow of the IEEE and an NSF CAREER awardee. He has received numerous best paper awards for his research. Dr. Shroff is among the ist of highy cited researchers from Thomson Reuters ISI and in Thomson Reuters Boo on The Word s Most Infuentia Scientific Minds in 04. He won the IEEE INFOCOM achievement award in 04. Cathy H. Xia is an associate professor in the Department of Integrated Systems Engineering at The Ohio State University, with a courtesy appointment from the Department of Computer Science and Engineering. She received her M.S. in Statistics and her Ph.D. in Operations Research, both from Stanford University. Before joining OSU, Dr. Xia had wored at IBM T.J. Watson Research Center as a senior research scientist where she has received numerous IBM research and practice awards. Her areas of research are in performance modeing and anaysis, queueing theory and stochastic networs, and distributed optimization with appications in computer networs, information and service systems. She is a co-editor of Springer boo Performance Modeing and Engineering, a coinventor for more than a dozen US and internationa patents, and has pubished over 60 peer-reviewed artices in journas and conference proceedings. She is a recipient of the Best Paper Runner-up Award in IEEE INFOCOM 03. Hanif D. Sherai is a University Distinguished Professor Emeritus in the Industria and Systems Engineering Department at Virginia Poytechnic Institute and State University. His areas of research interest are in mathematica optimization modeing, anaysis, and design of agorithms for speciay structured inear, noninear, and continuous and discrete nonconvex programs, with appications to transportation, ocation, engineering and networ design, production, economics, and energy systems. He has pubished over 33 refereed artices in various operations research journas and has co- authored nine boos. He is an eected member of the Nationa Academy of Engineering, a Feow of both INFORMS and IIE, and a member of the Virginia Academy of Science Engineering and Medicine.