On the Flow-level Dynamics of a Packet-switched Network

Transcription

1 On the Flow-level Dynamics o a Packet-switched Network Ciamac C. Moallemi and Devavrat Shah Abstract: The packet is the undamental unit o transportation in modern communication networks such as the Internet. Physical layer scheduling decisions are made at the level o packets, and packet-level models with exogenous arrival processes have long been employed to study network perormance, as well as design scheduling policies that more eiciently utilize network resources. On the other hand, a user o the network is more concerned with end-to-end bandwidth, which is allocated through congestion control policies such as TCP. Utility-based low-level models have played an important role in understanding congestion control protocols. In summary, these two classes o models have provided separate insights or low-level and packet-level dynamics o a network. In this paper, we wish to study these two dynamics together. We propose a joint low-level and packet-level stochastic model or the dynamics o a network, and an associated policy or congestion control and packet scheduling that is based on α-weighted policies rom the literature. We provide a luid analysis or the model that establishes the throughput optimality o the proposed policy, thus validating prior insights based on separate packet-level and lowlevel models. By analyzing a critically scaled luid model under the proposed policy, we provide constant actor perormance bounds on the delay perormance and characterize the invariant states o the system. Keywords and phrases: Flow-level model, Packet-level model, Congestion control, Scheduling, Utility maximization, Back-pressure maximum weight. 1. Introduction The optimal control o a modern, packet-switched data network can be considered rom two distinct vantage points. From the irst point o view, the atomic unit o the network is the packet. In a packet-level model, the limited resources o a network are allocated via the decisions on the scheduling o packets. Scheduling policies or packet-based networks have been studied across a long line o literature e.g., [30, 27, 24]. The insights rom this literature have enabled the design o scheduling policies that allow or the eicient utilization o the resources o a network, in the sense o maximizing the throughput o packets across the network, while minimizing the delay incurred by packets, or, equivalently, the size o the buers needed to queue packets in the network. Packet-level models accurately describe the mechanics o a network at a low level. However, they model the arrival o new packets to the network exogenously. In reality, the arrival o new packets is also under the control o the network designer, via rate allocation or congestion control decisions. Moreover, while eicient utilization o network resources is a reasonable objective, a network designer may also be concerned with the satisaction o end users o the network. Such objectives cannot directly be addressed in a packet-level model. C. C. Moallemi is at the Graduate School o Business, Columbia University. D. Shah is with Laboratory or Inormation and Decision Systems, Massachusetts Institute o Technology. This work was supported in parts by NSF projects CNS and TF Authors addresses: ciamac@gsb.columbia.edu, devavrat@mit.edu. 1 imsart-generic ver. 2009/02/27 ile: low-level-journal.tex date: October 1, 2012

2 2 C. C. Moallemi & D. Shah Flow-level models c. [12, 3] provide a dierent point o view by considering the network at a higher level o abstraction or, alternatively, over a longer time horizon. In a low-level model, the atomic unit o the network is a low, or user, who wishes to transmit data rom a source to a destination. Resource allocation decisions are made via the allocation o a transmission rate to each low. Each low generates utility as a unction o its rate allocation, and rate allocation decisions may be made so as to maximize a global utility unction. In this way, a network designer can address end user concerns such as airness. Flow-level models typically make two simpliying assumptions. The irst assumption is that, as the number o lows evolves stochastically over time, the rates allocated to lows are updated instantaneously. The rate allocation decision or a particular low is made in a manner that requires immediate knowledge o the demands o other lows or the limited transmission resources along the low s entire path. This assumption, reerred to as time-scale separation, is based on the idea that lows arrive and depart according to much slower processes than the mechanisms o the rate control algorithm. The second assumption is that, once the rate allocation decision is made, each low can transmit data instantaneously across the network at its given rate. In reality, each low generates discrete packets, and these packets must travel through queues to traverse the network. Moreover, the packet scheduling decisions within the network must be made in a manner that is consistent with and can sustain the transmission rates allocated to each low, and the induced packet arrival process must not result in the ineicient allocation o low level network resources. In this paper, our goal is to develop a stochastic model that jointly captures the packet-level and low-level dynamics o a network, without any assumption o time-scale separation. The contributions o this paper are as ollows: 1. We present a joint model where the dynamic evolution o lows and packets is simultaneous. In our model, it is possible to simultaneously seek eicient allocation o low level network resources buers while maximizing the high-level metric o end-user utility. 2. For our network model, we propose packet scheduling and rate allocation policies where decisions are made via myopic algorithms that combine the distinct insights o prior packetand low-level models. Packets are scheduled according to a maximum weight policy. The rate allocation decisions are completely local and distributed. Further, in long term i.e., under luid scaling, the rate control policy exhibits the behavior o a primal algorithm or an appropriate utility maximization problem. 3. We provide a luid analysis o the joint packet- and low-level model. This analysis allows us to establish stability o the joint model and the throughput optimality o our proposed control policy. 4. We establish, using a luid model under critical loading, a perormance bound on our control policy under the metric o minimizing the outstanding number o packets and lows in the network or, in other words, minimizing delay. We demonstrate that, or a class o balanced networks, our control policy perorms to within a constant actor o any other control policy. 5. Under critical loading, we characterize the invariant maniold o the luid model o our control policy, as well as establishing convergence to this maniold starting rom any initial state. These results, along with the method o Bramson [4], lead to the characterization o multiplicative state space collapse under heavy traic scaling. Further, we establish that the invariant states o the luid model are asymptotically optimal under a limiting control policy. imsart-generic ver. 2009/02/27 ile: low-level-journal.tex date: October 1, 2012

3 On the Flow-level Dynamics o a Packet-switched Network 3 In summary, our work provides a joint dynamic low- and packet-level model that captures the microscopic packet and macroscopic luid, low behavior o large packet-based communications network aithully. The perormance analysis o our rate control and scheduling algorithm suggests that the separate insights obtained or dynamic low-level models [12, 3] and or packet-level models [30, 27, 24] indeed continue to hold in the combined model. The balance o the paper is organized as ollows. In Section 1.1, we survey the related literature on low- and packet-level models. In Section 2, we introduce our network model. Our network control policy, which combines eatures o maximum weight scheduling and utility-based rate allocation is described in Section 3. A luid model is derived in Section 4. Stability or, throughput optimality o the network control policy is established in Section 5. The critically scaled luid model is described in Section 6. In Section 7, we provide perormance guarantees or balanced networks. The invariant states o the critically scaled luid model are described in Section 8. Finally, in Section 9, we conclude Literature Review Tassiulas and Ephremides [30] considered packet-level network model in the context o wireless radio network and proposed a class o maximum weight MW or back-pressure policies. Such policies assign a weight to every schedule, which is computed by summing the number o packets queued at links that the schedule will serve. At each instant o time, the schedule with the maximum weight will be selected. Tassiulas and Ephremides [30] establish that, in the context o multi-hop wireless networks, MW is throughput optimal. That is, the stability region o MW contains the stability region o any other scheduling algorithm. This work was subsequently extended to a much broader class o queueing networks by others e.g., [18, 7, 29, 6]. By allowing or a broader class o weight unctions, the MW algorithm can be generalized to the amily o so-called MW-α scheduling algorithms. These algorithms are parameterized by a scalar α 0,. MW-α can be shown to inherit the throughput optimality o MW [14, 22] or all values o α 0,. However, it has been observed experimentally that the average queue length or, delay under MW-α decreases as α 0 + [14]. Certain delay properties o this class o algorithms have been subsequently established under a heavy traic scaling [27, 6, 24]. Flow-level models have received signiicant recent attention in the literature. The work o Kelly, Maulloo, and Tan [12] utilizes such model or Internet and suggests link between rate-control algorithm and decentralized solutions o a deterministic utility maximization problem. This optimization problem seeks to maximize the utility generated by a rate allocation, subject to capacity constraints that deine a set o easible rates. This work was subsequently generalized to settings where lows stochastically depart and arrive [17, 8, 3], addressing the question o the stability o the resulting control policies. Fluid and diusion approximations o the resulting systems have been subsequently developed [13, 11, 32]. Under these stochastic models, lows are assumed to be allocated rate as per the optimal solution o the utility maximization problem instantaneously. Essentially, this time-scale separation assumption captures the intuition that the dynamics o the arrivals and departures o lows happens on a much slower time-scale than the dynamics o rate control algorithm. In reality, low arrivals/departures and rate control happen on the same time-scale. Various authors have considered this issue, in the context o understanding the stability o the stochastic imsart-generic ver. 2009/02/27 ile: low-level-journal.tex date: October 1, 2012

4 4 C. C. Moallemi & D. Shah low level models without the time-scale separation assumption [15, 10, 28, 20, 26]. Lin, Schro, and Srikant [15] assume a stochastic model o low arrivals and departures as well as the operation o a primal-dual algorithm or rate allocation. However, there are no packet dynamics present. Other work [10, 28, 20] has assumed that rate control or each type o low is a unction o a local Lagrange multiplier; and a separate Lagrange multiplier is associated with each link in the network. These multipliers are updated using a maximum weight-type policy. In this line o work, Lagrange multipliers are interpreted as queue lengths, but there are no actual packet-level dynamics present. Further, these models lack low-level dynamics as well. Thus, while overall this collection o work is closest to the results o this paper, it stops short o oering a complete characterization o a joint low- and packet-level dynamic model. We take note o recent work by Walton [31], which presents a simple but insightul model or joint low- and packet-level dynamics. In this model, each source generates packets by reacting to the acknowledgements rom its destination, and at each time instant, each source has at most one packet in light. Under a many-source scaling or a speciic network topology, it is shown that the network operates with rate allocation as per the proportional air criteria. This work provides important intuition about the relationship between utility maximization and the rate allocation resulting rom the packet-level dynamics in a large network. However, it is ar rom providing a comprehensive joint low- and packet-level dynamic model as well as eicient control policy. Some remarks are in order regarding the relation between analytic methods o this paper and those existing in the literature. At the core o the present paper, we utilize luid model techniques along with an appropriate Lyapunov unction in order to establish positive-recurrence o a Markovian description o the system along with the existence and attractiveness o the invariant maniold or the critical luid model. The latter is an important step towards establishing diusion approximations. At some level, these approaches are well known or more than a decade now and have been developed or related systems in the literature positive-recurrence in [15, 28] and critical luid model analysis [24]. The non-triviality usually lies in the ability to execute this program in the system at hand. In our setting, this requires totally dierent sets o analytic arguments due to presence o two dierent dynamics. This, or example, can be seen in one o the key results c. Lemma 3 that establishes appropriate drit condition or the model o this paper. Similarly, the optimality results we establish or the critical luid are quite distinct rom those known in the literature. 2. Network Model In this section, we introduce our network model. This model captures both the low-level and the packet-level aspects o a network, and will allow us to study the interplay between the dynamics at these two levels. In a nutshell, lows o various types arrive according to an exogenous process and seek to transmit some amount o data through the network. As in the standard congestion control algorithm, TCP, the lows generate packets at their ingress to the network. The packets travel to their respective destinations along links in the network, queueing in buers at intermediate locations. As each packet travels along its route, it is subject to physical layer constraints, such as medium access constraints, switching constraints, or constraints due to limited link capacity. A low departs once all o its packets have been sent. imsart-generic ver. 2009/02/27 ile: low-level-journal.tex date: October 1, 2012

5 On the Flow-level Dynamics o a Packet-switched Network 5 In this section, we ocus on the mechanics o the network that are independent o the network control policy. In Section 3, we will propose a speciic network control policy to be applied in the context o this model Network Structure Consider a network consisting o a set V o destination nodes, a set L o links, and a set F o low types. Each low type is identiied by a ixed given route starting at the source link s L and ending at the destination node d V. At a given time, multiple lows o a given type exist in the network, each low injects packets into the network. The network maintains buers or packets that are in transit across the network. At each link, there is a separate queue or the packets corresponding to each possible destination. Let E = L V denote the set o all such queues, with each e = l, v being the queue at link l or inal destination v. Traic in each queue is transmitted to the next hop along the route to the destination, and leaves the network when it reaches the destination. We deine the routing matrix R {0, 1} E E by setting R ee 1 i the next hop or queue e is queue e, and R ee 0 otherwise. Traic or a low o type enters the network in the ingress queue ι s, d E. Deine the ingress matrix Γ {0, 1} E F by setting Γ e 1 i ι = e, and Γ e 0 otherwise. We will assume that the routes are acyclic. In this case, we can deine the matrix Ξ I R 1 = I + R + R Under the acyclic routing assumption, Ξ e e = 1 i and only i a packet arriving at queue e will subsequently eventually pass through queue e Dynamics: Flow-Level In this section, we will describe in detail the stochastic model or dynamics o lows in the network. The system evolves in continuous time, with t [0, denoting time, starting at t = 0. For each low type F, let N t denote the number o lows o type active at time t. Flows o type arrive according to an independent Poisson process o rate ν. Flows o type receive an aggregate rate o service X t [0, C] at time t. Here, C > 0 is the maximal rate o service 1 that can be provided to any low type. The total rate o service X t is divided equally among the N t lows. As lows are serviced, packets are generated. The evolution o packets and lows proceeds according to the ollowing: Packets are generated by all the lows o type, in aggregate, as a time varying Poisson process o rate X t at time t. I N t = 0, then we require that X t = 0. When a packet is generated by a low o type, it joins the ingress queue ι E. When a packet is generated by a low o type, the low departs rom the network with a probability o 0 < µ < 1, independent o everything else. 1 Our assumption o a maximal service rate is a technical assumption necessary or the mathematical developments later on. This is not meant to model the capacity o links, or example. imsart-generic ver. 2009/02/27 ile: low-level-journal.tex date: October 1, 2012

6 6 C. C. Moallemi & D. Shah Thus, each low o type generates a number o packets that is distributed according to an independent geometric random variable with mean 2 1/µ, and the low departure process or lows o type is a Poisson process o rate µ X t at time t. We can summarize the low count process N by the transitions { N t + 1 at rate ν, N t N t 1 at rate µ X t. Deine the oered load vector ρ R F + by ρ ν /µ, or each low type. Without loss o generality, we will make the ollowing assumptions: 3 ρ > 0, i.e., we restrict attention to lows with a non-trivial loading. ρ < C1, i.e., we assume that the maximal service rate C is suicient or the load generated by any single low type. ΞΓρ > 0, i.e., we restrict attention to queues with a non-trivial loadings. Denote by A t the cumulative number o lows o type that have arrived in the time interval [0, t]. Denote the cumulative number o packets generated by lows o type in the time interval [0, t] by A t. We suggest that the reader take note o dierence between A and A. Let D t denote the cumulative number o lows o type that have departed in the time interval [0, t]. The evolution o the low count or low type over time can be written as N t = N 0 + A t D t Dynamics: Packet-Level As we have just described, lows generate packets which are injected into the network. These packets must traverse the links o the network rom source to destination. In this section, we describe the dynamics o packets in the network. We assume that each queue in the network is capable o transmitting at most 1 data packet per unit time. However, the collection o queues that can simultaneously transmit is restricted by a set o scheduling constraints. These scheduling constraints are meant to capture any limitations o the network due to scarce resources e.g., limited wireless bandwidth, limited link capacity, switching constraints, etc.. Formally, the scheduling constraints are described by the set S {0, 1} E. Under a permissible schedule π S, a packet will be transmitted rom a queue e E i and only i π e = 1. We assume that 0 S. We require that each queue e be served by some schedule, i.e., there exists a π S with π e = 1. Further, we assume that S is monotone: i σ S and σ {0, 1} E is such that σ e σ e or every queue e, then σ S. Finally, denote by Π {0, 1} E S the matrix with columns consisting o the elements o S. The results o this paper can be extended, using the same arguments, to a the setting where S {0, 1,..., L} E, or some inite integer L, and S is monotone. In this case, π e would be the total number o packets scheduled per unit time rom queue e, and we would allow 2 The assumption that µ < 1 is equivalent to requiring that 1/µ > 1, i.e., each low is expected to generate more than one packet. This is reasonable since we require lows to arrive with at least one packet and or there to be some variability in the number o packets associated with a low. 3 In what ollows, inequalities between vectors are to be interpreted component-wise. The vector 0 resp., 1 is the vector where every component is 0 resp., 1, and whose dimension should be inerred rom the context. imsart-generic ver. 2009/02/27 ile: low-level-journal.tex date: October 1, 2012

7 On the Flow-level Dynamics o a Packet-switched Network 7 scheduling o up to L packets. To keep the exposition simple, however, we shall work with the setting o L = 1 as described above. We assume that the scheduling o packets happens at every integer time. At a time τ Z +, let πτ S denote the scheduled queues or the time interval [τ, τ + 1. For each queue e, denote by Q e τ the length o the queue e immediately prior to the time τ i.e., beore scheduling happens. The queue length evolves, or times t [τ, τ + 1 according to 4 Q e t Q e τ π e τi {Qeτ >0} + F Γ e A t A τ + e E R e eπ e τi {Qe τ >0}. Here, or each low type, A τ is the cumulative number o packets generated by lows o type in the time interval [0, τ. The term π e τi {Qeτ >0} enorces an idling constraint, i.e., i queue e is scheduled but empty, no packet departs. Note that, over a time interval [τ, τ + 1, we assume the transmission o packets already present in the network occurs instantly at time τ, while the arrival o new packets to the network occurs continuously throughout the entire time interval. Finally, let S π τ denote the cumulative number o time slots during which the schedule π was employed up to and including time τ. Let Z e τ denote the cumulative idling time or queue e up to and including time τ. That is, Z e τ τ π e Sπ s S π s 1 I {Qes 1=0}. s=0 π S Then, the overall queue length evolution can be written in vector orm as where we deine the vectors Qτ + 1 = Q0 I R ΠSτ + I R Zτ + ΓAτ + 1, 3 Qt [ Q e t ] e E, At [ A t ] F, Sτ [ S π τ ] π S, Zτ [ Z e t ] e E. 3. MWUM Control Policy A network control policy is a rule that, at each point in time, provides two types o decisions: a the rate o service provided to each low, and b the scheduling o packets subject to the physical constraints in the network. In Section 2, we described the stochastic evolution o lows and packets in the network, taking as given the network control policy. In this section, we describe a control policy called the maximum weight utility maximization-α MWUM-α policy. MWUM-α takes as a parameter a scalar α 0,. The MWUM-α policy is myopic and based only on local inormation. Speciically, a low generates packets at rate that is based on the queue length at its ingress, and the scheduling o packets is decided as a unction o the eected queue lengths. At the low-level, rate allocation decisions are made according to a per low utility maximization problem. Each low chooses a rate so as to myopically maximize its utility as a unction o rate 4 I { } denotes the indicator unction. imsart-generic ver. 2009/02/27 ile: low-level-journal.tex date: October 1, 2012

8 8 C. C. Moallemi & D. Shah consumption, subject to a linear penalty or, price or consuming limited network resources. As in the case o α-air rate allocation, the utility unction is assumed to have a constant relative risk aversion o α. The price charged is a unction o the number o packets queued at the ingress queue associated with the low, raised to the α power. At the packet-level, packets are scheduled according to a maximum weight-α scheduling algorithm. In particular, each queue is assigned a weight that is a unction o number o packets in the queue and in downstream queues, each raised to the α power, and a schedule is picked which maximizes the total weight o all scheduled queues Control: Rate Allocation The irst control decision we shall consider is that o rate allocation, or, the determination o the aggregate rate o service X t, at time t, or lows o type. We will assume our network is governed by a variant o an α-air rate allocation policy. This is as ollows: Assume that each low o type is allocated a rate Y t 0 at time t by maximizing a per low utility unction that depends on the allocated rate, subject to a linear penalty or, cost or consuming resources rom the limited capacity o the network. In particular, we will assume a utility unction given a rate allocation o y 0 to an individual low o type o the orm V y y1 α 1 α, or some 5 α 0, \ {1}. This utility unction is popularly known as α-air in the congestion control literature [19], and has a constant relative risk aversion o α. The individual low will be assigned capacity according to Y t argmax y 0 V y Q α ι ty. Here, Q α i t represents a price or congestion signal. Intuitively, a low reacts to the congestion in the network or lack thereo through the length o the ingress or irst-hop queue ι. Then, i N t > 0, the aggregate rate X t allocated to all lows o type at time t is determined according to X t = N ty t = argmax x 0 x 1 α N α t 1 α Q α ι t x. I N t = 0, we require that X t = 0. Further, we will constrain the overall rate allocated to lows o type by the constant C. Thus, rate allocation is determined by the equation x 1 α N α argmax t Q α ι X t = x [0,C] 1 α t x i N t > 0, 0 otherwise. 5 Our results easily extend to the case where α = 1, by adopting the standard deinition V y log y in this case. We will restrict ourselves to the α 1 case in what ollows, however, to simpliy the presentation. imsart-generic ver. 2009/02/27 ile: low-level-journal.tex date: October 1, 2012

9 On the Flow-level Dynamics o a Packet-switched Network 9 Given the strictly concave nature o the objective in this optimization program, it is clear that the maximizer is unique and is given by { max{n t/q t, C} i Q t > 0, X t = 4 C otherwise. Note that while 4 does not explicitly depend on the parameter α, this parameter will impact downstream queue lengths through the scheduling mechanism that we will deine momentarily, hence the choice o α will indirectly inluence rate allocation decisions. Denote by X t the cumulative rate allocation to lows o type in the time interval [0, t], i.e., X t t 0 X s ds. X is Lipschitz continuous and dierentiable, since X is always bounded by C Control: Scheduling The second control decision that must be speciied is that o scheduling. We will assume the ollowing variation o the maximum weight or back-pressure policies introduced by Tassiulas and Ephremides [30]. At the beginning o each discrete-time slot τ Z +, a schedule πτ S is chosen according to the optimization problem ] πτ argmax π S = argmax π S [ π e Q α e τ R ee Q α e τ e E e E π I RQ α τ, where Q α τ [ Q α e τ ] is a vector o component-wise powers o queue lengths, immediately e E prior to time τ. In other words, the schedule πτ is chosen so as to maximize the summation o weights o queues served, where weight o a queue e E is given by [ I RQ α τ ]. Given that S e is monotone, there exists a π S that maximizes this weight and is such that π e = 0 i Q e τ = 0. We will restrict our attention to such schedules only. From this, it ollows that the objective value o the optimization program in 5 is always non-negative. By the discussion above, it is clear that the ollowing invariants are satisied: 1. For any schedule π and time τ, S π τ = S π τ 1 i π I RQ α τ < σ I RQ α τ, or some σ S. 2. For any queue e and time τ, Z e τ = 0. In other words, there is no idling. 4. Fluid Model In this section, we introduce the luid model o the our system. As we shall see, the allocation o rate to lows in the luid model resembles rate allocation o low-level models that has been popular in the literature [17, 3]. In that sense, our model on original time-scale operates at a packet-level granularity and on the luid-scale operates at a low-level granularity. 5 imsart-generic ver. 2009/02/27 ile: low-level-journal.tex date: October 1, 2012

10 10 C. C. Moallemi & D. Shah 4.1. Fluid Scaling and Fluid Model Equations In order to introduce the luid model o our network, we will consider the scaled version o the system. To this end, denote the overall system state at a time t 0 by Zt Qt, Z t, Nt, S t, Xt, At, Dt, At. Here, the components o the state Zt are the primitives introduced in Sections 2.2 and 2.3. That is, at times t R +, we have, Qt [ Q e t ] e E, where Q et is the length o queue e, Nt [ N t ] F, where N t is the number o lows o type, Xt [ X t ] F, where X t is the cumulative rate allocated to low type, At [ A t ] F, where A t is the cumulative arrival count o low type, Dt [ D t ] F, where D t is the cumulative departure count o low type, At [ A t ] F, where A t is the cumulative packet arrival count o low type, and, at times τ Z +, we have Zτ [ Z e τ ] e E, where Z eτ is the cumulative idleness or queue e, Sτ [ S π τ ] π S, where S πτ is the cumulative time schedule π is employed. Given a scaling parameter r R, r 1, deine the scaled system state as Z r t Q r t, Z r t, N r t, S r t, X r t, A r t, D r t, A r t. 6 Here, the scaled components are deined as Q r t r 1 Qrt, X r t r 1 Xrt, D r t r 1 Drt, N r t r 1 Nrt, A r t r 1 Art, A r t r 1 Art, Z r t r 1[ rt rt Z rt + rt rtz rt ], S r t r 1[ rt rt S rt + rt rts rt ]. In the above, the components Z and S are linearly interpolated or technical convenience only. Our interest is in understanding the behavior o Z r as r. Roughly speaking, in this limiting system the trajectories will satisy certain deterministic equations, called luid model equations. Solutions to these equations, which are deined below, will be denoted as luid model solutions. The ormal result is stated in Theorem 1. Deinition 1 Fluid Model Solution. Given ixed initial conditions q0 R E + and n0 R F +, or every time horizon T > 0, let FMST denote the set o all trajectories zt qt, zt, nt, st, xt, at, dt, at over the time interval [0, T ] such that: Z R E + R E + R F + R S + R F + R F + R F + R F +, 7 imsart-generic ver. 2009/02/27 ile: low-level-journal.tex date: October 1, 2012

11 On the Flow-level Dynamics o a Packet-switched Network 11 F1 All components o zt are uniormly Lipschitz continuous and thus dierentiable or almost every t 0, T. Such values o t are known as regular points. F2 For all t [0, T ], nt = n0 + at dt. F3 For all t [0, T ], at = νt. F4 For all t [0, T ], dt = diagµ xt. F5 For all t [0, T ], qt = q0 I R Πst + I R zt + Γat. F6 For all t [0, T ], at = xt. F7 For all t [0, T ], 1 st = t. F8 Each component o z, s, and x is non-decreasing. In addition, deine the set FMS α T to be the subset o trajectories in FMST that also satisy: F9 I t 0, T is a regular point, then or all F, x 1 α n α argmax t qι α x t = x [0,C] 1 α t x i n t > 0, ν /µ = ρ otherwise, where 6 x t x t. F10 I t 0, T is a regular point, then or all π S, ṡ π t = 0, i π I Rq α t < max σ S σ I Rq α t. F11 I t 0, T is a regular point and n t = 0 or some F, then q ι t = 0. F12 For all t [0, T ], zt = 0. Note that F1 F8 correspond to luid model equations that must be satisied under any scheduling policy, and, hence, are algorithm independent luid model equations. On the other hand, F9 F12 are particular to networks controlled under the MWUM-α policy. F9 captures the long-term eect o the rate allocation mechanism through the α-air utility maximization based policy. F10 captures the eect o short-term packet-level behavior induced by the scheduling algorithm. Speciically, the characteristics o the MW-α packet scheduling algorithm are captured by this equation. As explained in [10], in a static utility maximization ormulation where the network is trying to achieve allocation o resources to maximize the α-air utility subject to capacity constraint, the equation F9 turns out to be the primal step and F10 the dual step or a gradient algorithm attempting to solve the Lagrangian dual ormulation o the static optimization problem. In that sense, the luid-model above suggests that the system is attempting to solve such a network-wide utility maximization problem at each time instance Formal Statement We wish to establish luid model solutions as limit o the scaled system state process Z r as r. To this end, ix a time horizon T > 0. Let D[0, T ] denote the space o all unctions rom 6 We use the notation θt to denote d θt or θ : [0, T ] R. dt imsart-generic ver. 2009/02/27 ile: low-level-journal.tex date: October 1, 2012

12 12 C. C. Moallemi & D. Shah [0, T ] to Z, as deined in 7, that are right continuous with let limits RCLL. We will denote the Skorohod metric on this space by d, see Appendix B or details. Given a ixed scaling parameter r, consider the scaled system dynamics over interval [0, T ]. Each sample path Z r o the system state is RCLL, and hence is contained in the space D[0, T ]. The ollowing theorem ormally establishes the convergence o the scaled system process to a luid model solution o the orm speciied in Deinition 1. Theorem 1. Given a ixed time horizon T > 0, consider a collection o scaled system state processes {Z r : r 1} D[0, T ] under an arbitrary control policy. Suppose the initial conditions lim r Qr 0 = q0, are satisied with probability 1. Then, or any ε > 0, lim in P Z r FMS ε T r lim N r 0 = n0, 8 r = 1. Here, FMS ε T is an ε-lattening o the set FMST o luid model solutions, i.e., FMS ε T {x D[0, T ] : dx, y < ε, y FMST }. Additionally, under the MWUM-α control policy, we have that lim in P Z r FMS α r ε T = 1. Here, FMS α ε T is an ε-lattening o the set FMS α T o MWUM-α luid model solutions, i.e., FMS α ε T {x D[0, T ] : dx, y < ε, y FMS α T }. Theorem 1 can be established by ollowing a somewhat standard sequence o arguments c. [4, 25, 13]. First, the collection o measures corresponding to the collection o random processes {Z r : r 1} is shown to be tight. This establishes that limit points must exist. Next, it is established that each limit point must satisy the conditions o a luid model solution with probability 1. The tightness argument uses concentration properties o Poisson process along with the Lipschitz property o queue length process. A detailed argument is required to establish that, with probability 1, the conditions o luid model solution are satisied. The complete proo o Theorem 1 is presented in Appendix B. 5. System Stability In this section, we characterize the stability o a network under the MWUM-α policy. In particular, we shall see that the network evolves according to a Markov process is positive recurrent as long as the system is underloaded. In other words, the system is maximally stable. In order to construct the stability region under the MWUM-α policy, irst deine the set o packet arrival rates by { } Λ λ R E + : s R S + with λ Πs, 1 s 1. 9 Imagine that the network has no packet arrivals rom lows, but instead has packets arriving according to exogenous processes. Suppose that λ R E + is the vector o exogenous arrival rates, so imsart-generic ver. 2009/02/27 ile: low-level-journal.tex date: October 1, 2012

13 On the Flow-level Dynamics o a Packet-switched Network 13 that packets arrived to each queue e at rate λ e. Then, it is not diicult to see that the network would not be stable under any scheduling policy i λ / Λ. This is because there is at least one queue in the network that is loaded beyond its service capacity. Hence, the set Λ represents the raw scheduling capacity o the network. The set Λ can alternatively be described as ollows: Given a vector λ R E +, consider the linear program PRIMALλ minimize 1 s s subject to λ Πs, s R S +. Clearly λ Λ i and only i PRIMALλ 1. The quantity PRIMALλ is called the eective load o a system with exogenous packet arrivals at rate λ. Now, in our model, packets arrive to the network not through an exogenous process, but rather, they are generated by lows. As discussed in Section 2.2, each low type F generates packets at according to an oered load o ρ. The generated packets are injected into the network according to the ingress matrix Γ, and subsequently travel through the network along pre-determined paths speciied by the routing matrix R. Let λ R E + be the vector o implied loads on the scheduling network due to the packets generated by lows. It seems reasonable to relate λ and the vector ρ R F + o oered loads according to λ = Γρ + R λ. Equivalently, we deine λ ΞΓρ, where Ξ is rom 1. We deine the eective load Lρ o our network by Lρ PRIMALΞΓρ. Given the above discussion, it seems natural to suspect that the network s scheduling capacity allows it to operate eectively as long as Lρ 1. This motivates the ollowing deinition: Deinition 2 Admissibility. A vector ρ R F + o oered loads admissible i Lρ 1. Similarly, ρ is strictly admissible i Lρ < 1. Finally, ρ is critically admissible i Lρ = 1. We will establish system stability, or, more ormally, positive recurrence, when oered load is strictly admissible. To this end, recall that the system is completely described by the Z process. Under the MWUM-α policy, the evolution o all the components o Z is entirely determined by N, Q. The packet-scheduling decisions happen at discrete time, lows are served as per Poisson process with state-dependent rate, and arrival process are ixed rate Poisson process. Thereore, the Zτ at discrete-time instance τ Z + is Markov chain. We shall establish positive-recurrence o this Markov chain when system is under-loaded. The ollowing is the main result o this section: Theorem 2. Consider a network system with strictly admissible ρ operating under the MWUM-α policy. Then, the Markov chain Nτ, Qτ, τ Z +, is positive recurrent. It is worth noting that i Lρ > 1, then at least one o the queues in the network must be, on average, loaded beyond its capacity. Hence the network Markov process can not be positive recurrent or stable. The proo o Theorem 2, provided next in Section 5.1, uses a luid model approach. Dai [5], or example, developed an early instance o such an approach or a class o queueing networks. However, this result does not apply to the present setting, and a specialized analysis is needed. Conceptually, the luid model approach involves two steps: 1 derive strong stability o luid model, and 2 use strong stability to establish the positive recurrence o the original Markov chain. To this end, deine Lyapunov unction L α over the vector o low counts n = [n ] F R F + and the vector o queue imsart-generic ver. 2009/02/27 ile: low-level-journal.tex date: October 1, 2012

14 14 C. C. Moallemi & D. Shah lengths q = [q e ] e E R E + by L α n, q F n 1+α µ ρ α + e E q 1+α e. 10 The ollowing lemma, whose proo is ound in Section 5.2, provides a central argument towards establishing the strong stability o the luid model. This lemma will also be o use later in establishing the characterization and attractiveness o invariant maniold under critical loading. Lemma 3. Let n, q be, respectively, the low count process and the queue length process o a luid model solution in the set FMS α T. I Lρ 1, then or every regular point t 0, T, d dt L α nt, qt 0. Suppose urther that Lρ < 1. Then, there exist δ > 0 and T > 0 such that, or all T > T, i the initial conditions n0, q0 satisy L α n0, q0 = 1, then L α nt, qt 1 δ, or all t [T, T ] Proo o Theorem 2 The ollowing lemma provides a suicient condition or positive recurrence: Lemma 4. [21, Theorem 8.13] Let X be an irreducible, aperiodic discrete-time Markov chain on a countable state space X. Suppose there exists a unction V : X R +, constants A and ε > 0, and an integrable stopping time τ > 0 such that, or all x X with V x > A, E [V X τ X 0 = x] V x εe[τ X 0 = x]. I the set {x X : V x A} is inite and E[V X 1 X 0 = x] < or all x X, then X is positive recurrent and ergodic. Our proo o Theorem 2 relies on establishing the suicient condition or positive recurrence given by Lemma 4, using the stability o the luid model Lemma 3. To this end, note that under the MWUM-α policy, X τ Nτ, Qτ Z F + Z E + is a discrete-time Markov chain over integer times τ 0, which is irreducible and aperiodic rom every state, the null state is reachable and there is a positive probability o remaining at the null state. We shall use normed version o the Lyapunov unction L α, deined as ln, q L α n, q 1 1+α, or all n, q R F + R E +. The role o V in Lemma 4 will be played by l. Lemma 14 implies that there exists constants 0 < C 1 C 2 < so that, or all n, q, C 1 n, q ln, q C 2 n, q. 11 imsart-generic ver. 2009/02/27 ile: low-level-journal.tex date: October 1, 2012

15 On the Flow-level Dynamics o a Packet-switched Network 15 Further, or any κ > 0 and n, q, we have that lκn, κq = κln, q. Now, consider any sequence o initial states { x k N k 0, Q k 0 Z F + Z E + } : k N, such that x k as k. For each k, consider a system starting at the initial state x k. Denote the state o the kth system at time t 0 by X k t N k t, Q k t. Here, or non-integer values o t, we will utilize the continuous time description o the original Markov process. Indeed, the integer-time sampled version o this original continuous time Markov process is precisely what we are interested in. For each k N, deine the scaling actor r k l x k, and notice that r k. Fix a time horizon T > 0, and or the kth system, consider the scaled state process, deined or t [0, T ] as X r k t 1 r k X k r k t = 1 r k N k r k t, Q k r k t, The descriptor o the scaled system is given, or t [0, T ], by Z r k t rom 6. Let µ r k be its distribution on D[0, T ]. From 11, we have that, or any k, X r k 0 = X k 0 r k 1/C Since the set o scaled initial conditions is compact, there must exist a limit point and a convergence subsequence. Along this subsequence, by the analysis o Theorem 1 in particular, Lemma 17 the measures {µ rk } are tight, and thereore, there exists a measure µ that is a limit point. By restricting to a urther subsequence, we can assume, without loss o generality, that µ rk µ as k. That is, Z r k t t [0,T ] zt, as k, t [0,T ] with z FMS α T satisying the luid model equations. Given a luid model solution z o the orm 7, denote by n, q the low count and queue length components. Note that l X r k 0 = 1, or all k. By the continuity o l, we have that l n0, q0 = 1; that is L α n0, q0 = 1. Then, rom Lemma 3, there exist δ > 0 and T > 0 so that, or suiciently large T, l nt, qt 1 δ, or all t [T, T ]. 13 Deine the unctional F : D[0, T ] R + by zt F sup l nt, qt t [0,T ] t [T, 1 2 T +T ] Since F is a continuous, it ollows that Z r F k t zt t [0,T ] F t [0,T ], as k. 14 imsart-generic ver. 2009/02/27 ile: low-level-journal.tex date: October 1, 2012

16 16 C. C. Moallemi & D. Shah Further, rom 12, and using the act that the arrival processes are Poisson and the boundedness o the rate allocation policies, it ollows that or all t [0, T ] and all k N, [ ] E N r k t, Q rk t C 3 T, 15 or some constant C 3 > 0. From this, the uniorm integrability o F Z r k ollows. Subsequently, rom 13 and 14, it ollows that [ Z lim E r F k t k t [0,T ]] 1 δ. Equivalently, in terms o the unscaled state process X, we have that lim k [ 1 lx k E sup l t [T, 1 2 T +T /2] X l x k ] t X 0 = x k 1 δ. 16 To complete the proo o Theorem 2, deine stopping time τ that is uniormly distributed over integers in range [l X 0 T, l X 0 T + T /2]. Then, it ollows that τ is integrable and or all initial states x Z F + Z E + with lx large enough, E [ l X τ X 0 = x ] lx εe [τ X 0 = x], or appropriately chosen ε > 0. This satisies the conditions o Lemma 4, and hence completes the proo o Theorem Proo o Lemma 3 Suppose t 0, T is a regular point. We will start by establishing the irst part o Lemma 3: i Lρ 1, then d dt L α nt, qt 0. To this end, note that d dt L α nt, qt = 1 + α F n α t µ ρ α ṅ t + qe α t q e t e E }{{}. 17 q } {{ } n We consider the terms n and q separately. First, consider the term n. For each low type, we wish to show that n α t µ ρ α ṅ t qι α t ρ x t. 18 By F2 F4, ṅ t = ν µ x t. There are two cases. I n t = 0, then by F9, we have that x t = ρ, thus both sides o 18 are 0. I n t > 0, then n α t µ ρ α ṅ t = nα t ρ α ρ x t n α tx1 α 1 α nα t 1 α. 19 ρ1 α t imsart-generic ver. 2009/02/27 ile: low-level-journal.tex date: October 1, 2012

17 On the Flow-level Dynamics o a Packet-switched Network 17 Here, the inequality ollows rom the act that the unction gz z 1 α /1 α is concave. For a concave unction g, g yy x gy gx, here we have x = x t and y = ρ. Now, since x t is optimal or the rate allocation problem o F9 and ρ is easible, we have that n α tx1 α t 1 α qα ι tx t n α ρ1 α t 1 α qα ι tρ. 20 Combining 19 and 20, we have established 18. Then, we can sum 18 over all F, to obtain n [ Γ ρ xt ] q α t. 21 Now, consider the term q in 17. F5, F6, and F12 imply that From F7 and F10, [ I R Πṡt] q α t = Together 22 and 23, imply that qt = I R Πṡt + Γxt. 22 π S ṡ π tπ I Rq α t = max σ S σ I Rq α t. 23 q = [Γxt] q α t max σ S σ I Rq α t. 24 Now, combining 17, 21, and 24, we have that d dt L α nt, qt 1 + α [Γρ] q α t max σ S σ I Rq α t. 25 By the deinition o Lρ PRIMALΞΓρ, there exists some s R S + with 1 s = Lρ and Γρ I R Πs = π S s π I R π. This implies that Hence, by 25, [Γρ] q α t Lρ max σ S σ I Rq α t, d dt L α nt, qt 1 + α 1 Lρ max σ S σ I Rq α t. 26 In order to bound the right hand side o 26, we will argue that max σ S σ I Rq α t 1 q α t E To see this, consider a queue e 0 E with maximal length at time t, i.e., q e0 t = max e q e t. Deine e 1,..., e J E to be a sequence o distinct queues so that, or each 0 i < J, packets departing rom queue e i go to queue e i+1, and packets departing rom queue e J exist the network. By our assumption o acyclic routing, such a sequence exists, and since each queue is distinct, J + 1 E. imsart-generic ver. 2009/02/27 ile: low-level-journal.tex date: October 1, 2012

18 18 C. C. Moallemi & D. Shah Consider the distinct schedules π 0,..., π J S, where, or 0 i J, the schedule π i serves exactly the queue e i such schedules exist by the monotonicity assumption on the scheduling constraints. Clearly, these schedules have weights given by { q πi I Rq α α t = ei t qe α i+1 t i 0 i < J, qe α J t i i = J. Averaging over the J + 1 schedules, 1 J + 1 J i=0 π i I Rq α t = qα e 0 t J + 1. Since at least one schedule must have a weight that exceeds this average, max σ S σ I Rq α t qα e 0 t J + 1 max e E qe α t E Since max e E qα e t 1 q α t, E 27 ollows. Combining 26 and 27, we obtain, when Lρ 1, This establishes the irst part o Lemma 3. d dt L 1 q α t α nt, qt 1 + α 1 Lρ E To prove the second part o Lemma 3, we will consider two separate cases over initial conditions n0, q0 with Lα n0, q0 = 1: i 1 q 1+α 0 > ε 1. ii 1 q 1+α 0 ε 1. Here, ε 1 > 0 is a constant that will be determined shortly. For case i, rom the norm inequality in part i o Lemma 14, we have that α 1 1+α ε1 < q0 1+α q0 α, Thus, 1 q α 1+α 0 > ε 2 ε1. Due to F1, q is uniormly Lipschitz continuous, there exists T 1 > 0 such that 1 q α t ε 2 /2 or all t [0, T 1 ]. From 28, since Lρ < 1,we have that d dt L ε 2 α nt, qt 1 + α 1 Lρ 2 E 2 < 0, or all regular t 0, T 1 ]. Thereore, there exists δ 1 > 0 so that or all t T 1. L α nt, qt 1 δ1, 29 imsart-generic ver. 2009/02/27 ile: low-level-journal.tex date: October 1, 2012

19 On the Flow-level Dynamics o a Packet-switched Network 19 For case ii, the argument is more complicated. The basic insight is as ollows: i 1 q 1+α 0 is small, then q in 17 must be small as well. Further, a good raction o the lows will be allocated the maximal rate i.e., C, thereore n in 17 will be signiicantly negative. This will leads to strictly negative drit in L. We will now ormalize this intuition and make an appropriate choice o ε 1 > 0 along the way. First, by the Lipschitz continuity o q, there exists some τ 1 > 0 such that, or all t [0, τ 1 ], 1 q 1+α t 2ε 1. Note that rom 24, the act that 0 S, and xt C1, or all regular t 0, τ 1 ], By Jensen s inequality, Thereore, or regular t 0, τ 1 ], q [Γxt] q α t C1 q α t. 1+α 1 F 1 q α α 1 t F 1 q 1+α t. 1 q C F 1+α 2ε1 α 1+α. 30 Again using the act that 1 q 1+α t 2ε 1 or t [0, τ 1 ], it ollows that q e t ε 3 2ε 1 1/1+α, or all e E. Now, consider the set F { F : n 0 4Cε 3 }. By Lipschitz continuity, there exists some 0 < T 2 < τ 1 such that, or all F and all t [0, T 2 ], n t 2Cε 3. Then, by F9, x t = C or regular t 0, T 2 ]. Thus, i F, we have, or regular t 0, T 2 ], where we deine n α t µ ρ α ṅ t = nα t µ ρ α ρ x t nα t C ρ β 1 min F µ ρ α Finally, i / F, we have, or regular t 0, T 2 ], where we deine n α t µ ρ α ṅ t = nα t µ ρ α µ ρ α > 0. ρ x t 2Cα ρ 1 α β 2 max F ρ 1 α µ. C ρ β 1 n α t, 31 α 1+α ε1 β 2 2C α α 1+α ε1, 32 µ Using 17 and 30 32, it ollows that, or all t [0, T 2 ], d dt L α nt, qt 1 + α [ 1 C F 1+α + β2 F 2C α] α 1+α ε1 β 1 n α t. 33 F From 28, or all t 0, L α nt, qt 1. For t [0, T2 ], 1 qt 1+α 2ε 1. Suppose that ε 1 < 1/4. Then, n 1+α 1 2ε 1 1 t 1 1. max F F µ ρ α 4 max F µ ρ α imsart-generic ver. 2009/02/27 ile: low-level-journal.tex date: October 1, 2012

20 20 C. C. Moallemi & D. Shah or t [0, T 2 ]. From part i o Lemma 14, n α t max F F µ ρ α α 1+α β 3 > 0. Then, or regular t 0, T 2 ], d dt L α nt, qt 1 + αβ1 β 3 /2 < 0. It ollows that there exists δ 2 > 0 such that, or all t T 2, L α nt, qt 1 δ2. 34 Lemma 3 ollows rom 29 and 34 with δ = min{δ 1, δ 2 } and T = max{t 1, T 2 }. 6. Critical Loading We have established the throughput optimality o the system under the MWUM-α control policy, or any α 0, \ {1}. Thus, this entire amily o policies possesses good irst order characteristics. Further, there may be many other throughput optimal policies outside the class o MWUM-α policies. This naturally raises the question o whether there is a best choice o α, and how the resulting MWUM-α policy might compare against the universe o all other policies. In order to answer these questions, we desire a more reined analysis o policy perormance than throughput optimality. One way to obtain such an analysis is via the study o a critically loaded system, i.e., a system with critically admissible arrival rates. Under a critical loading, luid model solutions take non-trivial values over entire horizon. In contrast, or strictly admissible systems under throughput optimal policies, all luid trajectories go to 0 c. Lemma 3. We will employ the study o the luid model solutions o critically loaded systems as a tool or the comparative analysis o network control policies. In particular, given a vector o low counts, n R F +, and the vector o queue lengths, q R E +, deine the linear cost unction cn, q F n + q e = 1 [ Γ diagµ 1 n + q ]. 35 µ e E This cost unction is analogous to a minimum delay objective in a packet-level queueing network: a cost is incurred or each queued packet, and a cost is also incurred or each outstanding low, proportional to the number o packets that it is expected to generate. In this section, we establish undamental lower bounds that apply to the cost incurred in a critically loaded luid model under any scheduling policy. In Sections 7 and 8, we will compare these with the costs incurred by MWUM-α control policies. We shall ind that as α 0 +, the cost induced by the MWUM-α algorithms improves and becomes close to the algorithm independent lower bound imsart-generic ver. 2009/02/27 ile: low-level-journal.tex date: October 1, 2012