A Globally Stable Adaptive Congestion Control Scheme for Internet-Style Networks with Delay 1

Transcription

1 A Globally Stable Adaptive ongestion ontrol Scheme for Internet-Style Networks with Delay Tansu Alpcan 2 and Tamer Başar 2 (alpcan, tbasar)@control.csl.uiuc.edu Abstract In this paper, we develop, analyze and implement a congestion control scheme in a noncooperative game framework, where each user s cost function is composed of a pricing function proportional to the queueing delay experienced by the user, and a fairly general utility function which captures the user demand for bandwidth. Using a network model based on fluid approximations and through a realistic modeling of queues, we establish the existence of a unique equilibrium as well as its global asymptotic stability for a general network topology, where boundary effects are also taken into account. We also provide sufficient conditions for system stability when there is a bottleneck link shared by multiple users experiencing non-negligible communication delays. In addition, we study an adaptive pricing scheme using hybrid systems concepts. Based on these theoretical foundations, we implement a window-based, end-to-end congestion control scheme, and simulate it in ns-2 network simulator on various network topologies with sizable propagation delays. Methods Keywords: ontrol theory, Mathematical programming/optimization, Simulations, Economics. I. INTRODUTION Game theory provides a natural framework for developing pricing and congestion control mechanisms for the Internet. Users on the network can be modeled as players in a congestion control game where they choose their strategies or in this case flow rates. Players are noncooperative in terms of their demands for network resources, and have no specific information on other users strategies. A user s demand or utility for bandwidth is captured in a utility function, and may not be bounded. To compensate for this, one can devise a pricing function, proportional to the bandwidth usage of a user, in order to preserve the network resources and to provide an incentive for the user to implement end-to-end congestion control []. A useful concept in such a noncooperative congestion control game is that of Nash equilibrium [2] where each player minimizes his/her own cost (or maximize payoff) given all other players strategies. There is a rich literature on game theoretic analysis of flow control problems utilizing both cooperative [3] and noncooperative [4], [5], [6] frameworks. ongestion control schemes utilizing pricing schemes based on explicit feedback have been proposed by Kelly et al. [7], [8], and Gibbens et al. [9], and subsequent studies have further Research supported in part by the NSF grant R All correspondence should be forwarded to: Prof. Tamer Başar, oordinated Science Laboratory, University of Illinois, 38 West Main Street, Urbana, IL USA. Tel: (27) ; Fax: (27) ; tbasar@control.csl.uiuc.edu 2 oordinated Science Laboratory and Department of Electrical and omputer Engineering, University of Illinois, 38 West Main Street, Urbana, IL 68 USA. elaborated on this approach following its basic principles [], [], [2], [3]. Although the game theoretic approach provides a suitable framework for formulating and studying congestion and flow control problems in general networks, there are some inherent restrictions on implementable cost functions in the case of Internet-style networks. For example, the current structure of the Internet makes it difficult, if not impossible, for users to obtain detailed real time information on the state of the network and on other users. Therefore, users are bound to use indirect aggregate metrics that are available to them, such as packet drop rate or variations in the average round trip time (RTT) of packets in order to infer the current situation in the network. Packet drops, for example, are currently used by most widely deployed versions of the transfer control protocol (TP) as an indication of congestion. In this paper, however, we propose and analyze a pricing and congestion control scheme based on variations in the queueing delay a user experiences. A similar approach has been suggested in a version of TP, known as TP Vegas [4]. Although TP Vegas is more efficient than a widely used version of TP, TP Reno [5], the suggested improvements are empirical and based on experimental studies. The studies by Mo and Walrand [3] and by La and Ananatharam [2] also make use of an approach similar to the one in this paper; however, they are based on fairness and pricing concepts of Kelly. Furthermore, the former study employs only a narrow set of utility functions in describing user demands, while the latter does not take into account queue dynamics or the effect of boundaries and propagation delays on stability of the system. The noncooperative congestion control game introduced in this paper is characterized by a cost function for each user that is defined as the difference of pricing and utility functions. The pricing function is proportional to the queueing delay experienced by the user, whereas the utility function that quantifies the user demand for bandwidth belongs to a broad class of strictly increasing and strictly concave functions. Through a network model based on fluid approximations, and a realistic queueing model, we show the existence of a unique Nash equilibrium, under the assumption that the effect of a user s flow on congestion cost is vanishingly small, which holds especially if the number of users is large. Furthermore, we establish the global stability of the equilibrium under a general network topology. We also investigate stability of the system in a network with non-negligible propagation delays, and provide sufficient conditions for stability in the case of a

2 bottleneck node with multiple users. In addition, we study an adaptive pricing scheme using for adjusting the pricing parameter dynamically, and make use of hybrid (switched) system concepts for its analysis. Based on the theoretical foundations developed, we design a window-based, end-to-end congestion control scheme for Internet-style networks, which is TP-friendly [6]. This congestion control scheme is then simulated in Network Simulator 2 (ns-2) over Internet protocol (IP) for various network topologies. An earlier version of this paper without the adaptive pricing scheme and without a rigorous analysis of stability with boundary effects, can be found in [7]. The rest of the paper is organized as follows: The underlying network model and cost function are given in the next section. In Section III, the existence of a unique equilibrium and global stability of the system under a general network topology are established. Section IV generalizes the stability analysis of Section III to the case with delay, with a single bottleneck link. We consider in Section V an adaptive pricing scheme, and analyze it within a hybrid system framework. In Section VI we provide a realistic implementation of the congestion control scheme for IP networks. Section VII includes simulation results, and is followed by the concluding remarks of Section VIII. A. The Network Model II. THE MODEL We consider a general network model based on fluid approximations. Fluid models are widely used in addressing a variety of network control problems, such as congestion control [3], [5], [8], routing [5], [6], and pricing [7], [3], [9]. The topology of the network is characterized by a set of nodes N = {,..., N}, and a set of links L = {,..., L} connecting the nodes. In this network model, we make the natural assumption of connectivity, and let M := {,..., M} denote the set of active users. Each link l L has a fixed capacity l >, and an associated buffer size b l. For simplicity, each user is associated with a (unique) connection. Hence, the i th (i M) user corresponds to a unique connection between the source and destination nodes, s i, de i N, and we denote the corresponding route (path), which is a subset of L, by R i. The nonnegative flow, x i, sent by the i th user over this path R i satisfies the bounds x i x i,max. The upper bound, x i,max, on the i th user s flow rate may be a user specific physical limitation. It is possible to define a routing matrix, A, as in [7] that describes the relation between the set of routes R associated with the users (connections) and links l L : A l,i = {, if source i uses link l, if source i does not use link l i M and l L () Using this routing matrix A, the capacity constraints of the links are given by Ax, where x is the (M ) flow rate vector of the users and is the (L ) link capacity vector. If the aggregate sending rate of users whose flows pass through link l exceeds the capacity, l, of that link then the arriving packets are queued (generally on a first-come first-serve basis) in the buffer, b l, of the link with b l,max being the maximum buffer size. Furthermore, if the buffer of the link is full, incoming packets have to be dropped. Let the total flow on link l be given by x l := i:l R i x i. Thus, the buffer level at link l evolves in accordance with [ x l l ], if b l (t) = b l,max ḃ l (t) = x l l, if < b l (t) < b l,max (2) [ x l l ] +, if b l (t) = where ḃl(t) denotes ( b l (t)/ t), [.] + represents the function max(., ), and [.] represents the function min(., ). In addition, let ḃ := [ḃ,..., ḃl] be the (L ) link buffer rate vector, and O := [O,..., O L ] be defined as the (L ) flow loss rate vector at the links, where O l := { [ x l l ] +, if b l = b l,max, otherwise Taking the buffer dynamics and packet losses into account, we redefine the capacity constraints at the links as Ax(t) ḃ(t) O(t). (3) B. The ost (Objective) Function An important indication of congestion for internet-style networks is the variation in queueing delay, d, which is defined as the difference between the actual delay experienced by a packet, d a, and the fixed propagation delay of the connection, d p. If the incoming flow rate to a router exceeds its capacity, packets are queued (generally on a first-come first-serve basis) in the existing buffer of the router, leading to an increase in the RTT of packets. Hence, RTT on a congested path is larger than the base RTT, which is defined as the sum of propagation and processing delays on the path of a packet. The queueing delay at the l th link, d l, is a nonlinear function of the excess flow on that link, given by d l (x, t) = [ ] ( x l l ), if d l (t) = d l,max l ( x l l ), if < d l (t) < d l,max [ l ] + ( x l l ), if d l (t) = l in accordance with the buffer model described in (2), with d l,max := b l,max / l being the maximum possible queueing delay. Thus, the total queueing delay, D i, a user experiences is the sum of queueing delays on its path, namely D i (x, t) = l R i d l (x, t), i M, which we henceforth write as D i (t), i M. Let us define a cost function for each user as the difference between pricing and utility functions. The pricing function of the i th user is linear in x i for each fixed total queueing delay D i of the user, and is linear in D i with x i fixed (hence (4)

3 it is a bilinear function of x i and D i ). The utility function U i (x i ) is assumed to be strictly increasing, differentiable, and strictly concave; it basically describes the user s demand for bandwidth. Accordingly, we make use of variations in RTT to devise a congestion control and pricing scheme. The cost (objective) function for the i th user at time t is thus given by J i (x, t) = α i D i (t) x i U i (x i ), (5) which s/he wishes to minimize. In accordance with this objective, we consider a simple dynamic model of the network game where each user changes her/his flow rate in proportion with the gradient of her/his cost function with respect to her/his flow rate, ẋ i = J i (x)/ x i. Taking into consideration also the boundary effects, the update algorithm for the i th user thus is: [ ] dui (x i ) α i D i (t), if x i = x i,max dx i du ẋ i = i (x i ) α i D i (t), if < x i < x i,max (6) [ dx i ] + dui (x i ) α i D i (t), if x i = dx i where the effect of the i th user on the delay, D i (t), s/he experiences is ignored. This assumption can be justified for networks with a large number of users, where the effect of each user is vanishingly small. Furthermore, from a practical point of view, it is extremely difficult if not impossible for a user to estimate her/his own effect on queueing delay. III. STABILITY ANALYSIS In this section, we analyze the stability of the system described by (4) and (6). First, we investigate the simple case of a single link with a single user in order to gain further insight into the system. We then generalize the analysis to the case of multiple users with again a single link. Finally, we establish stability for a general network topology with multiple links and users. A. Stability for a Single Link with a Single User For a single user on a single link, the equations describing the dynamics of the system consist of the user algorithm, which is a simplified version of (6), and queueing delay equation for a single user derived from (4). For the time being, we ignore the effects of boundaries on the system, and write the dynamics as ẋ(t) = du(x) dx αd(t) d(t) = x (7) where d is the queueing delay, x is the user flow rate, and is the link capacity. The system (7) has a unique equilibrium point (x, d ) given by x = and d = (/α) du(x )/dx. Defining the Admittedly, in this case the assumption of an individual user not affecting the delay on a link is violated, but still this exercise is useful for the subsequent analysis dealing with the multiple users case. queueing delay and flow rate around the equilibrium point as d := d d and x := x x, we obtain the following equivalent system around the equilibrium: where the function g( x) is defined as x(t) = g( x) α d(t), d(t) = x, (8) g( x) := du(x) dx du(x ) dx. Note that >, if x < g( x) <, if x > (9) =, if x =, due to the fact that U(x) is strictly concave in x, and hence, (du(x)/dx) is strictly decreasing. The system (8) can be viewed as a generalized pendulum equation with g( x) as the friction term [2]. Let us introduce an energy-like Lyapunov function: V ( x, d) = α ( x)2 + ( d) 2, () which is positive definite. The derivative of V along the system trajectories is given by V ( x, d) = 2 g( x) x, α where the inequality follows from (9). Thus, V ( x, d) is negative semi-definite. Let S := {( x, d) R 2 : V ( x, d) = }. It follows from (9) that S = {( x, d) R 2 : x = }. Hence, for any trajectory of the system that belongs to S, we have x. It follows then directly from (8) and the fact that g() = that x x = d =. Therefore, the only solution that can stay identically in S is the zero solution, which corresponds to the unique equilibrium of the original system (7). Furthermore, it is globally asymptotically stable by LaSalle s invariance theorem [2]. This completes the proof of stability without the boundary effects. To account for boundary effects, let us introduce the feasible set Ω for the system defined by (4) and (6), as Ω = {(x, d) R 2 : x x max and d d max }, () where d max and x max are the finite upper-bounds on d and x respectively. Assume that x max >. First, we investigate the existence and uniqueness of an inner equilibrium point, < d < d max and < x = < x max, on the set Ω. From (4), (6), and the definition of Ω, the only possible equilibrium points on the boundaries (of Ω) are (, ) and (, d max ). learly, the former cannot be an equilibrium as ẋ > when d =. For the latter not to be an equilibrium point we choose α in such a way that it satisfies α > du(x) d max dx. (2) x=

4 Thus, under (2) the equilibrium queueing delay satisfies < d < d max, and the single-user single-link system admits a unique inner solution, (x, d ), on Ω. We next analyze the effect of the boundaries on system stability to gain further insight into the problem. A rigorous comprehensive study on the effect of boundaries will actually be undertaken in Section III- for a general network topology with multiple users. For this reason, we will simply outline the procedure here, without details. We argue that each time the trajectory of the system hits a boundary, it has to leave the boundary in finite time. onsider the boundary of Ω where d =. From (6), ẋ is positive, and by (4) the trajectory leaves this boundary as soon as x >. Likewise, when x = we have d <, and hence, the flow rate becomes positive in finite time. We proceed with the other two boundaries in a similar fashion. In the case x = x max >, d is positive due to (4). Since (du(x)/dx) is strictly decreasing in its argument and α satisfies the condition in (2), there exists a d < d max such that ẋ(x max, d ) <. Finally, on the boundary where d = d max, if x < then the trajectory immediately leaves it. Otherwise, from condition (2) ẋ <, and hence, the trajectory cannot stay on the boundary indefinitely. It was already established that V inside the set Ω. Furthermore, chattering of the trajectory is not possible since V along the boundaries as we will show in Section III-. We conclude, therefore, that the unique inner equilibrium point of the system given by (4) and (6) is globally asymptotically stable on the set Ω by LaSalle s invariance theorem. B. Stability for a Single Link with Multiple Users The analysis for a single link with multiple users is a fairly straightforward generalization of the single-link single-user case discussed above. The generalized system is given by ẋ i (t) = du i(x i ) α i d(t), i =,..., M, dx i M d(t) = x (3) i, where U (x ),..., U M (x M ) are strictly concave user utility functions, and the boundary point behavior is described by (4) and (6). Let us define the corresponding constraint set Ω as Ω = {(x, d) R M+ : x i x i,max, i M and d d max }, (4) where d max and x i,max are upper-bounds on d and x i respectively. We investigate the existence of a unique inner equilibrium point under the assumption x i,max >, i M. To simplify the notation, let p i (x i ) := du i (x i )/dx i, where p i is strictly decreasing in x i due to strict concavity of U i. Further define (α i d) where p i is the inverse of the (α i d) is strictly decreasing in both α i and d, and therefore P (α, d) is also decreasing in the pair (α, d). Ignoring the boundary effects, the equilibrium point of (3) is given by P (α, d) := M p i function p i. We note that p i x i = p i (α i d ), i M, (5) where d is solved from P (α, d ) := M p i (α i d ) =. (6) We now show that (x, d ) is indeed an inner equilibrium solution to (3) on the set Ω for some range of α values. Let the vector α belong to the compact set {α R M : α i,min α i α i,max, i M}. Define ɛ > to be arbitrarily small. It follows from the definition of P (α, d) that P (α max, ɛ) > for arbitrarily large values of α i,max i M. Furthermore, given d max one can find a vector α min such that P (α min, d max ) <, (7) Using the Intermediate Value Theorem, we conclude that there exists a unique d (d min, d max ) such that P (α, d ) =. In addition, (5) yields a unique x corresponding to d. We finally note that there cannot be an equilbrium point on the boundary of the set Ω since the system trajectory cannot stay on the boundary indefinitely as we will show in Section III-. Proposition III.. Let α i,min α i α i,max, i M, where the elements of the vector α max are arbitrarily large. If α min and d max satisfy the condition (7), then there exists a unique equilibrium solution, (x, d ), to the system (3) on the set Ω, which is further an inner point of Ω. Remark III.2. In the special case of logarithmic utility functions, U i (x i ) = u i log(x i +), condition (7) can be explicitly given as < + M M u i α i < d max. In addition, as d max the system admits a unique inner equilibrium solution for any finite value of α. We now define the system around the unique inner equilibrium point as in (8): x i (t) = g i ( x i ) α i d(t), i =,..., M, d(t) = M x i, (8) where the functions g i (x i ) are defined similarly as in the case of (9). Define next a Lyapunov function similar to the one of (): V ( x, d) = M α i ( x i ) 2 + ( d) 2. (9) The rest of the analysis without boundary effects is similar to the one in the case of a single link with a single user, and therefore it will not be carried out. In particular, V = M 2 α i g i ( x i ) x i, and is equal to zero only if x i = i d =. Since global stability under the boundary effects for a more general case will be provided in Theorem III.7 we also omit the boundary analysis in this special case of a single link.

5 . Stability for a General Network Topology with Multiple Users We now establish the stability of the system under a general network topology with multiple links, and with a general routing matrix A as defined in (). The generalized system is described by ẋ i (t) = du i(x i ) α i D i (t), i =,..., M, dx i d l (t) = x l, l =,..., L, l (2) with the boundary behavior given by (4) and (6). Define the feasible set Ω (as before) as Ω = {(x, d) R M+L : x i x i,max and d l d l,max, i, l}, where d l,max and x i,max are upper bounds on d l and x i, respectively. Define d max := [d,max,..., d L,max ]. We first investigate existence and uniqueness of an inner equilibrium on the set Ω under the assumption of x i,max > l, l. Toward this end, we assume that A is a full row rank matrix with M L, which is in fact no loss of generality as non-bottleneck links on the network have no effect on the equilibrium point, and can safely be left out. The study of existence follows lines similar to the ones in the case of a single link. Supposing that (2) admits an inner equilibrium and by setting ẋ i (t) and d l (t) equal to zero for all l and i one obtains A x = (2) f(α, x) = A T d, (22) where d := [d,..., d L ] T is the delay vector at the links, is the capacity vector introduced earlier, and the nonlinear vector function f is defined as f(α, x) := [ α du dx,..., ] T du M. (23) α M dx M Define X := {x R M : Ax = } as the set of flows, x, which satisfy (2). Multiplying (22) from left by A yields A f(α, x ) = AA T d. Since A is of full row rank, the square matrix AA T is full rank, and hence invertible. Thus, for a given flow vector x and pricing vector α, d(α, x) = (AA T ) Af(α, x), (24) is unique. From the definition of f, d(α, x) is a linear combination of p i (x i )/α i, and hence, strictly decreasing in α. Since the set X is compact, the continuous function d(α, x) admits a maximum value on the set X for a given α. Therefore, for each ɛ > one can choose the elements of α max sufficiently large such that < max x X d(α max, x) < ɛ. In addition, given X and d max, one can find α min such that < max x X d(α min, x) < d max, (25) Hence, we conclude that there is at least one inner equilibrium solution, (x, d ), on the set Ω, which satisfies (2) and (22). We next establish the uniqueness of the equilibrium. Suppose that there are two different equilibrium points, (x, d ) and (x 2, d 2). Then, from (2) it follows that A (x x 2) = (x x 2) T A T = Similarly, from (22) we have f(α, x ) f(α, x 2) = A T (d d 2). Multiplying this with (x x 2) T from left we obtain (x x 2) T [f(α, x ) f(α, x 2)] = We rewrite this as M (x i x 2i) [ dui (x i ) du i(x 2i ) ] =. α i dx i dx i Since U i s are strictly concave, each term (say the i-th one) in the summation is negative whenever x i x 2 i with equality holding only if x i = x 2i. Hence, we conclude that x has to be unique, that is x = x = x 2. From this, and (2), it immediately follows that D i, i =,..., M, are unique. This does not however immediately imply that d l, l =,..., L, are also unique, which in fact may not be the case if A is not full row rank. The uniqueness of d l s, however, follow from (24), where we obtain a unique d for a given equilibrium flow vector x : d = (AA T ) Af(α, x ). As a result, (x, d ), obtained from (2) and (22) constitutes a unique inner equilibrium point on the set Ω. Proposition III.3. Let α i,min α i α i,max, i M where the elements of the vector α max are arbitrarily large, and A be of full row rank. Given X, if a min and d max satisfy < max x X d(α min, x) < d max, where d(α, x) is defined in (24), then the system (2) has a unique equilibrium point, (x, d ), which is in the interior of the set Ω. Defining the delays at links, d l, and user flow rates, x i, around the equilibrium as d l := d l d l and x i := x i x i, respectively, for all l and i, we obtain the following system inside the set Ω and around the equilibrium: x i (t) = g i ( x i ) α i Di (t), i =,..., M, d l (t) = x i, l =,..., L, (26) l i:l R i where D i = l R i dl, and g i (.) is defined as in (9).

6 We define a positive definite Lyapunov function as a generalized version of (9): V ( x, d) = M α i ( x i ) 2 + L l ( d l ) 2 (27) The time derivative of V ( x, d) along the system trajectories is given by V ( x, d) M 2 = g i ( x i ) x i, α i where the inequality follows because g i ( x i ) x i i. Thus, V ( x, d) is negative semidefinite. Let S := {( x, d) R M+L : V ( x, d) = }. It follows as before that S = {( x, d) R M+L : x = }. Hence, for any trajectory of the system that belongs identically to the set S, we have x =. It follows directly from (26) and the fact that g i () = i that l= x = x = D i = i d l = l, where the last implication is due to the fact that D = A T d and the matrix A is of full row rank. Therefore, the only solution that can stay identically in S is the zero solution, which corresponds to the unique inner equilibrium of the original system. Let us redefine the feasible set in terms of the tilde d quantities, x and d, as follows: Ω := {( x, d) R M+L : x i x i x i,max x i and d l d l d l,max d l, i, l}. We note that the set Ω can also be defined through a set of (linear) inequalities h j ( x, d), j H := {,..., H}. Given X, let d max and α min be chosen such that (25) holds. Then, by Proposition III.3 the unique equilibrium point, (, ), is an inner solution on Ω, where h j (, ) < j H. We now investigate the effect of the boundaries given in Ω and described by (4) and (6). The system (26) can also be written compactly as ( ) ( ) x ż := = F := F(z), (28) d x d with the definition of F being obvious from (26). We have shown earlier that V when the system trajectory is strictly inside the set Ω. On the other hand, if the trajectory hits the boundary, then it stays on the boundary as long as the system gradient F(z) in (28) points toward the boundary. This is known as sliding mode behavior, which we define in this context as follows: Definition III.4. Let the gradient vector and the corresponding unit vector of the boundary surface {z : h(z) = } be given by n(z) := h(z) and n u (z) := n(z)/ n(z), respectively. Suppose that the trajectory of the system ż = F(z) is inside the surface, h(z) z. Then, the system exhibits sliding mode behavior on the boundary surface {z : h(z) = }, if and only if, n(z) F(z), where n F denotes the dot product of the vectors n and F. Furthermore, the gradient of the system trajectory during sliding mode is given by F s := F (F n u )n u. We next establish the relationship between the Lyapunov function V (z) in (27) and the sliding mode behavior of the trajectory along the boundaries. Proposition III.5. Assume that the system (28) has a unique inner equilibrium point, z, in the compact and convex feasible set Ω, and V (z) trajectory := z V (z) F(z), for all z such that h j (z) < j H, where the Lyapunov function V (z) is given by (27). The system trajectory exhibits sliding mode behavior along the j-th boundary {z : h j (z) = } j H, if and only if, V (z) sliding V (z) trajectory, (29) where V (z) sliding := z V (z) F s (z), z {z : h j (z) = }. Proof. It follows from the definition of F s that V sliding = z V (z) [F (F n u )n u ], (3) where we drop the arguments z and j to simplify the notation. From (27), z V (z) is given by [ 2 x z V (z) =,..., 2 x ] M, 2 d,..., 2 L d L. α α M On the other hand, by the definition of Ω, the vector n, of length M + L, has all zero entries except the k-th one, which is { sign( x k ), if k M n k = sign( d k ), if k L, where sign( ) is the sign function. Note that, n u = n by definition. Thus, on the boundary {z : h(z) = } the term z V (z) n u yields either x k or d, k and we obtain z V (z) n u. We now show the necessity: if the trajectory exhibits sliding mode behavior along the boundary, then we have n(z) F(z) and z V (z) n u. Hence, the result in (29) follows immediately from (3). Next, in order to show the sufficiency we note that (29) yields z V (z) (F n u )n u. Since z V (z) n u is already established on the boundary, we immediately obtain n F, which corresponds to sliding mode behavior of the trajectory. Now, as a corollary to this proposition, we have the following result. orollary III.6. onsider the system (28) on the feasible set Ω with the unique inner equilibrium point z. Furthermore, let

7 the time derivative of the Lyapunov function V of (27) be nonincreasing along the system trajectory without boundary effect. Then, the system trajectory exhibits sliding mode behavior, if and only if, the rate of decrease in V at the boundary point is less than or equal to the one without the boundary effect. From Proposition III.5, we have V for all ( x, d) Ω. It was already shown that the unique inner equilibrium solution, (x, d ), is the only invariant solution in S Ω. Thus, the asymptotic stability of the system follows from LaSalle s invariance theorem. This is captured in the following theorem. Theorem III.7. Let α min α α max, where α max is arbitrarily large, and A be of full row rank. Given X, suppose that a min and d max are chosen such that holds, and hence, the system < max x X d(α min, x) < d max, ẋ i (t) = du i(x i ) α i D i (t), i =,..., M, dx i d l (t) = x l, l =,..., L, l admits the unique inner equilibrium point (x, d ). Then, this system with its boundary point behavior described by (4) and (6) is globally asymptotically stable on the set Ω := {(x, d) R M+L : x i x i,max and d l d l,max, i, l}. (3) We note that the unique equilibrium point of the system is only an approximation to the Nash equilibrium since the effect of the i th user on the delay, D i (x, t), s/he experiences has been ignored. This approximation becomes more accurate as the number of users in the network increases. onsider the special case of α i = α j i, j M. Then, the unique equilibrium is asymptotically optimal as the number of users in the network increases, since it approximately minimizes the system problem min x Ω α x l d l U i (x i ), (32) l L i M which is similar to the one in [7]. Notice that the partial derivatives of the costs at the links, and not on the path of the i-th user, with respect to x i yield zero. Likewise, the utility function of each user depends only on that user s flow rate. Thus, the asymptotic optimality of the equlibrium point follows immediately from the equivalence of the first-order necessary conditions and the convexity of user objective (5) and system (32) functions. IV. STABILITY UNDER INFORMATION DELAY It was shown in Section III that the system described by (4) and (6) is globally asymptotically stable under a general network topology. We now investigate the global stability of the system under arbitrary propagation delays, which we are also going to refer to as information delay, and denote by r. First, we analyze the simple case of a single link with a single user to gain insight into the problem. Next, we generalize the analysis to a general network with a single bottleneck node and multiple users. We do not consider the case of multiple users on a general network topology with multiple links, since the problem in that case is quite intractable under arbitrary information delays. A. Stability for a Single Link with a Single User under Information Delay For the case of a single user on a single link, we modify equation (7) describing the system around the equilibrium by introducing a maximum propagation (information) delay r between the user and the link, which we assume to be a constant : ẋ(t) = du(x(t)) αd(t r) d(t) = dx x(t r) We again assume that α > du() d max dx (33), and x max >, so du() dx is the unique equilibrium of (33), that x =, d = α which is in the interior of the feasible set. Defining the queueing delay and flow rate around the unique inner equilibrium point as in the no delay case, we obtain the following equivalent system: x(t) = g( x(t)) α d(t r) d(t) = x(t r) (34) Notice that (34) is a set of delay differential equations. Such systems have been studied extensively in the literature; see, e.g. [2], [22]. Here we will make particular use of the methods presented in hapter 4.2 of [22]. From (34), we immediately have and x(t) = g( x(t)) α d(t + r) + α[ d(t + r) d(t r)], x(t r) = g( x(t r)) α d(t) + α We define a Lyapunov function 2r V ( x, d) = α ( x(t r))2 + ( d(t)) 2 + 2r t t+s x 2 (u r)du ds, x(t + s)ds. (35) which is positive definite. Taking the time-derivative of V along the system trajectories, we obtain V ( x, d) = 2 g( x(t r)) x(t r) α r 2r Using the simple algebraic inequality x(t r) x(t + s r)ds [ x 2 (t r) x 2 (t + s r)]ds 2 x(t r) x(t + s r) x 2 (t r) + x 2 (t + s r),

8 one can bound V above by V ( x, d) 2 4r g( x(t r)) x(t r) + α x2 (t r) Thus, V ( x, d) can be made negative semi-definite by imposing a condition on the maximum delay r. In this case, let S := {( x, d) Ω : V ( x, d) = }. It follows immediately that S = {( x, d) Ω : x = }. Hence, for any trajectory of the system that belongs to S, we have x. It also follows directly from (34), since g() =, that x x = d =. Therefore, the only solution that can stay identically in S is the zero solution, which corresponds to the unique equilibrium of the original system (7). We thus conclude that the system (34) is asymptotically stable by LaSalle s invariance theorem if the maximum delay r satisfies the condition where k is defined as k := r < 2α k, (36) inf x x max g( x) x. In order to gain further insight into this condition, we compute the parameter k for the specific case when the utility function is taken as the logarithmic one, that is U(x) = u log(x + ). In this case we obtain and hence k = g( x) = min x x max u x + u + = u x (x + )( + ) u (x + )( + ) = u (x max + )( + ). Hence, a safe bound on r is u r < 2α(x max + )( + ). Of course a better bound can be obtained if we know that the trajectory remains in a small neighborhood of the equilibrium,. This would very much be dependent on the application at hand. The analysis of the effect of boundaries on system stability is almost identical to the one of the case without delay, and we again make use of Proposition III.5. This now brings us to the following theorem, where again LaSalle s invariance theorem is invoked: Theorem IV.. Let α > the system du() d max dx, and x max >. Then, ẋ(t) = du(x(t)) αd(t r) dx d(t) = x(t r), with the unique inner equilibrium point (x =, d = du() α dx ) and boundary point behavior described by the delayed versions of (4) and (6) is globally asymptotically stable on the set Ω if the maximum delay, r, in the system satisfies the condition r < k 2α, where k := inf g( x)/ x, and x x x x =. max x B. Stability for a Single (Bottleneck) Link with Multiple Users under Information Delay We now generalize the preceding analysis of a single link with a single user to multiple users by introducing user specific maximum propagation delays r = [r,..., r M ] between the link and the users. We invoke the assumptions of Section III so that the system has a unique inner equilibrium point (x, d ) as characterized in Section III. Modifying the system equations (8) around this equilibrium point by introducing the associated maximum propagation delays, we obtain x i (t) = g i ( x i (t)) α i d(t ri ), i =,..., M d(t) = M x i (t r i ) (37) Following an approach similar to the one in the single user case with delay, one gets for the i th user x i (t r i ) = g i ( x i (t r i )) α i d(t) + α M i x j (t + s r j )ds. 2r i j= We again define a positive definite Lyapunov function: V ( x, d) = M ( x i (t r i )) 2 + ( α d(t)) 2 i t 2r i t+s x2 i (u r i)du ds. + M M (38) Taking the derivative of V along the system trajectories, we obtain V ( x, d) = M 2 g i ( x i (t r i )) x(t r i ) α i + M M 2r i j= 2 x i(t r i ) x j (t + s r j )ds + M M 2r i [ x 2 i (t r) x2 i (t + s r)]ds We bound the derivative V from above by V ( x, d) M 2 α i g i ( x i (t r i )) x i (t r i ) + 4Mr i x2 i (t r i ) This can be made negative semi-definite by imposing a condition on the maximum delay in the system, r max := max i r i. Let S := {( x, d) Ω : V ( x, d) = }. It follows as before that S = {( x, d) Ω : x = }. Hence, for any trajectory of the system that belongs identically to the set S, we have x =. It also follows directly from (37), and the fact that g i () = i, that x = x = d =,

9 where we have made use of the fact that the matrix A is of full row rank. Therefore, the only solution that can stay identically in S is the zero solution, which corresponds to the unique equilibrium of the original system. As a result, the system (37) is asymptotically stable by LaSalle s invariance theorem if the maximum delay in the system, r max, satisfies the condition r max < k min 2α max M, (39) where α max and k min are defined as α max := max α i i k min := min i inf x i xi xi,max x i g( x i ) x i (4) Notice that the bound on the maximum delay required for the stability of the system is affected by, among other things, the maximum pricing parameter and the capacity per user /M. The analysis of the boundary effects is identical to earlier ones, and therefore will be omitted. The following theorem now extends the results of Theorem IV. to the multi-user case. Theorem IV.2. Let the conditions in Theorem III.7 hold such that the system ẋ i (t) = du i(x i (t)) α i d(x, t r), i =,..., M, dx i d(t) = M x i (t r i ), with the boundary point behavior described by (4) and (6) admits a unique inner equilibrium point (x, d ). This system is globally asymptotically stable on the corresponding set Ω defined by (3), if the maximum delay, r max, in the system satisfies the condition r max < k min 2α max M, where α max and k min are defined in (4). V. AN ADAPTIVE PRIING SHEME In Sections III and IV we have shown that the pricing parameter α can be chosen such that the equilibrium point is feasible, d < d max, if x < x max. In other words, we have assumed either the equilibrium queue sizes (delays) are less than the maximum queue size (delay) or there are infinite buffers at all nodes. In fact, the buffer sizes on a real network are limited, and choosing the parameter α appropriately in a dynamic networking environment with varying numbers of users and changing preferences is not a trivial task. If α is chosen to be too high, then the proposed congestion control scheme becomes less robust. On the other hand, a too small α may lead to large queueing delays as well as packet losses. Furthermore, it is not possible to choose α in a centralized manner due to communication constraints. Given the capacity vector, it is possible to derive d max directly from maximum queue sizes at the network nodes. onsequently, the vector d max denotes a strict physical bound on the maximum queueing delays. The nonlinear vector function f is strictly decreasing in α by (23). Hence, from (24) the unique equilibrium d is strictly decreasing in α. Then, a natural strategy for making the equilibrium queue size at node l, d l, feasible is to increase α i for all users whose flow pass through link l. At the same time, those users will experience packet losses at node l if d l > d l, max, i.e. equilibrium queue size (delay) is larger than the maximum one. By making use of the packet losses experienced by the users as a signal to increase prices, we obtain a distributed, dynamic, and adaptive pricing scheme. Furthermore, due to its dynamic and adaptive nature, the adaptive pricing scheme improves robustness of the congestion control scheme to variations in network parameters like number of users, user preferences, link capacities, etc. A. Hybrid Modeling of the Adaptive Pricing Scheme We model the proposed adaptive pricing scheme as a switched (hybrid) system, and analyze its stability. In this context, a switched hybrid system can be defined as a continuoustime system with isolated discrete switching events [23]. Let us first consider for illustrative purposes the case of a single user on a single link with capacity. If the equilibrium queueing delay is larger than the maximum delay, d > d max, then the system trajectory will hit the boundary B := {(x, d) R 2 : x x max, and d = d max } at a point where x >. We again implicitly assume that x max >. Since the vector field points toward B, d >, the trajectory cannot leave the boundary, resulting in a sliding mode behavior on the set {(x, d) R 2 : x x max, and d = d max }. Then, the resulting vector field is a projection of the original system dynamics onto the boundary B. A simplified sketch of the sliding mode behavior is shown in Figure. x (x,d) d_max vector field system trajectory (x*,d*) Fig.. Sliding mode behavior of a single user on a single link with capacity. For a general network topology, let the boundary B g be d

10 defined as B g := {(x, d) R M+L : x i x i,max, and d l = d l,max, i, l} In the case the system trajectory hits B g, the set of users whose flows pass through the links l, where d l = d l,max, will experience packet losses. Let us define this set as M loss (t) := {i M : l R i, where d l (t) = d l,max, l L}. (4) Then, each user i M loss dynamically increases α i (t) by a fixed constant λ > in the case of a packet loss. Thus, the adaptive pricing scheme is defined by, i M, { λ α i (t), if (x, d) B g and i M loss (t) α i (t) =,, else (42) with system dynamics given in (2). Global asymptotic stability of the system (2) was shown in Theorem III.7 for an inner equilibrium point. We now argue that the system is globally asymptotically stable under the adaptive pricing scheme (42). onsider the case when the system trajectory hits the boundary B g and exhibits sliding mode behavior. Then, α(t) increases exponentially by (42) until the trajectory leaves the boundary, ḋ <. Hence, we have a sequence of equilibria (x (α), d (α)) for each value of α. This has to happen in finite time, say t, since by (2) there exists an α such that x l < l for all links l. Furthermore, at time t the unique equilibrium of the system, (x (α ), d (α )), becomes feasible by (24) as otherwise the trajectory cannot leave the boundary. For any feasible equilibrium point, we can define a sufficiently small level set, S l, for the Lyapunov function V given by (27), which does not intersect with B g. Each hit of the system trajectory to B g further shifts the equilibrium point by (24) and increases the size of the set S l. Then, by global asymptotic stability of the system (see Theorem III.7) the trajectory has to enter this invariant level set in finite time t t, and converge asymptotically to the unique equilibrium point. Thus, we have the following theorem summarizing this result: Theorem V.. Let A be full row rank, and let the boundary B g be given by B g := {(x, d) R M+L : x i x i,max, and d l = d l,max, i, l} due to the limited buffer sizes at the nodes of the network. Let the set of users experiencing packet losses be given by (4). Define the adaptive pricing scheme as { λ α i (t), if (x, d) B g and i M loss (t) α i (t) =,, else for all i M. Then, there exists a finite time t such that the sequence of unique equilibria (x (α), d (α)) of the system (2), indexed by α, converges to a feasible inner solution (x inner, d inner ) Ω, where Ω is as defined in (3). This equilibrium solution, corresponding to a specific value of α, is globally asymptotically stable on the set Ω. We note that the Theorem V. can be straightforwardly extended to capture the single bottleneck system with propagation delays described in (37). Remark V.2. The convergence of the system with adaptive pricing scheme occurs in two different time scales. In the fast time scale, the vector α(t) converges through the dynamics given by (42) to a specific value which is associated with an inner equilibrium point of Ω. With the value of α fixed as described, the system (2) then asymptotically converges in the slower time scale to this unique inner equilibrium point. Remark V.3. Since the implementation of the adaptation algorithm and the system will be in discrete-time, relocation of the equilibrium and increase in the pricing parameter α will occur in discrete jumps separated by RTT instead of a continuous shift. VI. AN IMPLEMENTATION OF THE ONGESTION ONTROL SHEME The user responses in Section II are based on a continuous time formulation. In reality, however, users update their flow rates only at discrete time instances corresponding to multiples of RTT. Hence, for implementation purposes, we discretize the reaction function of the i th user, and normalize it with respect to the RTT of the user. In addition, we need a specific utility function in order to quantify the user response in (6). Logarithmic utility functions are widely used in the literature not only because they have nice properties like strict concavity but also because they adequately capture several important concepts from economics, such as the law of diminishing returns. We choose the following utility function for i th user: U i (x i ) = u i log(x i + ), where u i is a user specific utility parameter. The optimal user response is, therefore, a discretized version of (6), and is given by [ [ u i x i (t + ) = x i (t) + κ i x i (t) + α ] ] + i d l (t), (43) l R i where κ i is a (user specific) step-size constant. The adaptive pricing scheme (described in Section V) for the i th user is given by { λ α i (t), if a packet loss occurs α i (t + ) =, (44) α i (t), otherwise where λ is chosen as.. The congestion control scheme characterized by the user response (43) is implemented in a Game (theory) Based ongestion ontrol (GB) protocol using the Network Simulator 2 (ns-2) [24]. The simulator ns-2 is chosen because it provides both a realistic environment for testing the proposed congestion control scheme and a level of abstraction for easy implementation. GB is a simple window based protocol for best-effort data traffic. It is devised as an end-to-end sliding window protocol [25], where the sender side adjusts

11 its window size according to the reaction function (43). For simplicity, receiver window size is chosen as one. We also implement a version with a simple slow start mechanism where the window size is increased by one per RTT until a packet loss is observed. The GB scheme is then extended to Adaptive GB (AGB) using the adaptive pricing (44). We next provide an overview on the GB and AGB schemes by summarizing the sender and receiver side functionalities. A. GB Protocol As one of the goals of GB protocol is compatibility with existing protocols, most of the functionality is on the sender side. Specifically, the sender side has the following functions: The sender puts sequence number and time stamp into the packet header. It estimates RTT and base RTT, where the latter is calculated as the minimum of the RTTs up to that point, by using the received acknowledgment (ack) packets. The estimation method for RTT is the same as the one in [26]. If a double ack is received, i.e. the same packet is acknowledged twice by the receiver, then it retransmits the packages beginning from the last acknowledged packet number. We note that this go back n scheme [25] is implemented for its simplicity. In fact, better mechanisms with receiver window size being larger than do exist. The sender updates the window size according to (43) using the current value of queueing delay, which is taken as the difference between the current RTT and the base RTT. The window size, W, is strictly positive. If no ack packet is received within, say 2 RT T, then sender retransmits previous packets beginning from the last acknowledged one, and reduces the window size. The receiver side, on the other hand, has the function of acknowledging received packets. If no packet is received for a specific time, say 4 RT T, the last received packet is acknowledged again. B. AGB Protocol In the AGB protocol, sender and receiver have the same functionality as in GB. However, in addition to standard functions, the sender also adjusts its pricing parameter α at each packet loss in accordance with the adaptive pricing scheme (44). VII. SIMULATIONS We simulate the proposed congestion control schemes, GB and AGB, on ns-2. The underlying protocol used for routing is the standard IP. Links and queues are chosen to be duplex and drop-tail, respectively. For simplicity, we fix the packet sizes to, bytes. In most of the cases queueing delays on the links are much smaller than the propagation delays that we choose. Hence, RTT s are approximately equal to twice the propagation delays. First, we simulate GB without a slow start mechanism in the simple single-user single-link case. The parameters in (43) are chosen as α = 3 and u =,. The buffer size is Flow Rate (bps) x Single Link Single User with various Delays RTT ms RTT 5ms RTT 2ms Fig. 2. A single user on a single link with RT T =, 5, and 2ms. This version of GB has no slow start mechanism. 5KB and propagation delay on the link is varied from 5ms to 25ms, and to ms. We observe in Figure 2 that as RTT gets too large, the system becomes unstable in accordance with the analysis in Section IV. Notice that it takes up to 7 seconds for the flow to reach its capacity in this simulation. Therefore, we use the slow start version of GB for the rest of the study. Flow Rate (bps) 4 x GB versus TP at a Bottleneck Node TP Flow GB Flow Total Flow Fig. 3. GB flow versus TP flow on a bottleneck link with ms propagation delay. We next explore the interaction between GB and TP on a single bottleneck link with ms propagation delay. GB is TP-friendly [] since its long-term rate does not exceed the one of the TP flow as observed in Figure 3. The fluctuation in the first two seconds is due to the slow start mechanism which requires a packet loss for termination. In the final simulation on a single bottleneck link, there are 2 identical users with parameters α = 5, u = 4,, and propagation delays are randomly chosen between 2ms and 5ms according to a uniform distribution. We observe flows of 3 specific users with respective propagation delays of 2ms, 5ms, and 5ms

12 2 x Multiple Users with random Delays at a Bottleneck Node User (2ms) User 2 (5ms) User 3 (5ms) 2 x 6 Flow Rates at Node 2, Medium Delay ase User 2 User 3 Total Flow.4 Flow Rate (bps) Flow Rate (bps) Fig. 4. Three out of 2 flows with various propagation delays (2ms, 5ms, and 5ms) sharing a 5Mbps bottleneck link. 5 5 Fig. 6. Flows of users 2, 3, and total flow at node 2 are observed for 5 seconds. in Figure 4. The system again converges to the equilibrium, however similar to TP, GB favors flows with smaller RTT as it is a window-based scheme. We then carry out a simulation with three users on a simple three node network topology with two 5Mbps links of 2ms propagation delay as shown in Figure 5. While flows of users and 2 pass through links and 2 respectively, the flow of user 3 passes through both links. ost parameters are chosen as α = 3 and u = 4,. User 3 is charged more than others through summation of queueing delays as s/he uses resources on both links. Thus, having the same utility parameter as others, s/he obtains a smaller fraction of the bandwidth. Figure 6 depicts the flow rates of users 2 and 3, as observed in node 2. Fig. 7. A Nam screenshot of the general (arbitrary) topology network. 5 x 5 3 Selected Flows in General Network Topology User User 2 User 3 Flow Rate (bps) Fig. 5. A Nam screenshot of the simple network. Links are symmetric, and have a capacity of 5Mbps with 2ms propagation delay. 5 We next simulate users with various routes and experiencing various information delays on a seven node arbitrary topology network (Figure 7) with all links except the one between nodes 5 and 6 having capacity of 5Mbps each. The link between nodes 5 and 6, on the other hand, has a capacity of Mbps. The links have equal propagation delays of 5ms each, except the links to nodes 7, 8, and 9, which have delays 5 5 Fig. 8. Three flows from nodes 7, 8, and 9 to node 6 are shown where these users are symmetric and have the following cost parameters: α = 3 and u = 2,.