Using Partial Differential Equations to Model TCP Mice and Elephants in Large IP Networks

Transcription

1 Using Partial Differential Equations to Model TCP Mice and Elephants in Large IP Netorks M.Ajmone Marsan, M.Garetto, P.Giaccone, E.Leonardi, E.Schiattarella, A.Tarello Dipartimento di Elettronica, Politecnico di Torino, Italy Abstract Fluid models of IP netorks have been recently proposed as a ay to break the scalability barrier of traditional discrete state-space models, both simulative (e.g., ns-2) and analytical (e.g., queues and Markov chains). Fluid models adopt an abstract deterministic description of the average netork dynamics through a set of ordinary differential equations that are then solved numerically, obtaining estimates of the time-dependent netork behavior. Hoever, an important limit of the fluid model approaches presented so far in the literature is their unnatural representation of scenarios comprising the short-lived TCP flos that dominate in today s Internet. In this paper e propose a ne fluid model approach in hich a different description of the dynamics of traffic sources is adopted, exploiting partial differential equations. This ne description of the source dynamics allos the natural representation of short-lived as ell as long-lived TCP connections, ith little sacrifice in the scalability of the model. In addition, the use of partial differential equations permits the description of distributions, instead of averages, thus providing better accuracy in the results. The comparison beteen the performance estimates obtained ith fluid models and ith ns simulations proves the accuracy of the proposed modeling approach. I. INTRODUCTION Traditional approaches to performance evaluation of telecommunication netorks in general, and of packet netorks in particular (e specifically refer to IP netorks in this paper), have normally relied on attempts to describe as closely as possible the dynamics of netork elements over a discrete state-space. We shall refer to these approaches as discrete models. Discrete models are quite a natural choice in light of the fact that the operations of traffic sources, of sitches, and of protocols, are normally governed by finitestate machines, hose dynamics determine the IP netork performance. Hoever, discrete models, requiring the description of the dynamics of the different netork elements over their discrete state-spaces (accounting for the dependencies induced by netork control algorithms, by end-to-end protocols, and by the traffic floing through several netork elements), suffer from limited scalability, thus alloing only the performance analysis of rather small netorking setups. This is the reason hy only toy topologies are normally considered in IP netork performance studies, and models almost invariably concentrate on a very limited subset of the netork protocol stack. This ork as supported by the Italian Ministry for Education, University and Research, ithin the TANGO project. Today, IP netorks have become extraordinarily large and complex, and, in order to predict the effects on performance of ne topologies or protocols, scalable modeling approaches are a must. To obtain scalability, researchers often model one netork element (a protocol layer or a router) ith a high degree of accuracy, hereas the remaining netork elements are either neglected, or modeled in a very coarse manner. This approach may sometimes produce acceptable results, but many cases exist here the interaction among netork elements is a focal point for performance issues (for example, in a satellite IP netork, the interaction beteen TCP and the ARQ protocol at layer 2 is crucial). The above comments apply to analytical models as ell as simulation. Indeed, traditional discrete models for simulation are based on a detailed packet-level description of the netork, and consequently suffer of scaling problems resulting from the groth of CPU times and memory requirements beyond the capability of available machines. On the analytical side, the application of discrete-state probabilistic models to the analysis of sizable portions of the Internet appears prohibitive, although such models (continuous-time or discrete-time Markov chains and queueing models in particular) have traditionally been the mathematical tool of choice in the netorking field. Netork Calculus [], [2] is able to cope ith full-size netork performance evaluation. It allos an abstract, deterministic, flo-level, orst-case analysis of the netork dynamics, thus greatly simplifying the study of complex netorking setups, but up to no has produced rather loose performance bounds under simplified traffic assumptions. A ne class of semi-analytical models has recently been introduced in the netorking arena, and today appears to be the most promising approach for scalable and accurate performance analysis of large IP netorks. This ne approach, that is often called fluid models, adopts an abstract deterministic description of the average netork dynamics through a set of differential equations [3], [4], [5], [6], [7], thus neglecting the short-term, packet-by-packet description of the stochastic netork dynamics. The resulting set of differential equations is then solved numerically, obtaining estimates of the timedependent netork behavior. The most attractive property of fluid models resides in the fact that their complexity (i.e., the number of differential equations to be solved) is independent of the number of TCP flos and of link capacities, hen considering traffic scenarios comprising only long-lived TCP flos (commonly called ele /4/$2. (C) 24 IEEE IEEE INFOCOM 24

2 phants ). In addition, fluid models have been recently proved to capture the limiting behavior of TCP elephants in single bottleneck topologies hen the number of TCP flos gros very large [3], [8], [9], []. An important limit of the fluid model approaches presented so far in the literature is their unnatural representation of scenarios comprising the short-lived TCP flos (commonly called mice ) that dominate in today s Internet. In this paper e develop a ne fluid model approach based on partial differential equations. This ne description of the source dynamics allos the natural representation of TCP mice as ell as elephants, ith no sacrifice in the scalability of the model. In addition, the use of partial differential equations permits the description of TCP indo distributions, instead of averages, thus providing better accuracy in the performance predictions. The rest of this paper is organized as follos. Section II overvies the fluid model of IP netorks originally proposed by Misra, Gong and Tosley, and Section III discusses other previous orks in the same area. Section IV describes the modeling methodology that e propose in this paper, based on partial differential equations, by first discussing the simplest version, and then progressively extending it, to cope ith finite indo sizes, fast recovery, drop-tail buffers, and TCP mice. Results are shon along the ay, and compared ith performance estimates generated by ns-2 simulations, so as to prove the accuracy of the proposed fluid model approach. Finally, Section V concludes the paper. II. THE MGT FLUID MODEL OF IP NETWORKS In [5], [6], [7], Misra, Gong and Tosley presented simple differential equations to describe the behavior of TCP elephants over netorks of IP routers adopting a RED (Random Early Detection []) active queue management (AQM) scheme. Their approach (that e name MGT) spurred several research efforts aiming at the application of various kinds of fluid models to the performance analysis of packet netorks. It is important to note that the equations of the MGT model heavily rely on the assumptions mentioned above (all TCP connections are elephants, and all IP routers adopt RED), and that the extension to mice and drop-tail routers may be not simple. Consider a netork comprising K router output interfaces, equipped ith FIFO buffers, and interfacing data channels at rate C (the extension to non-homogeneous data rates is straightforard). The netork is fed by I classes of long-lived TCP flos; all the elephants ithin the same class follo the same route through the netork, thus experiencing the same round-trip time (RTT), and the same average loss probability (ALP). At time t allbuffersareassumedtobeempty. Buffers drop packets according to their average occupancy, as dictated by a RED AQM scheme. A. TCP source evolution equations Consider the ith class of elephants; the temporal evolution of the average indo of TCP sources in the class, W i (t), is described by the folloing differential equation: dw i (t) dt R i (t) W i(t) λ i (t) () 2 here R i (t) is the average RTT for class i, and λ i (t) is the loss indicator rate experienced by TCP flos of class i. The differential equation is obtained by considering the fact that elephants can be assumed to alays be in congestion avoidance (CA) mode, so that the indo dynamics are close to AIMD (Additive Increase, Multiplicative Decrease). The indo increase rate in CA mode is approximatively linear, and corresponds to one packet per RTT. The indo decrease rate is proportional to the rate ith hich congestion indications are received by the source, and each congestion indication implies a reduction of the indo by a factor to. B. Netork evolution equations Q k (t) denotes the (fluid) level of the packet queue in the kth buffer at time t; the temporal evolution of the queue level is described by: dq k (t) A k (t)[ p k (t)] D k (t) dt here A k (t) represents the fluid arrival rate at the buffer, D k (t) the departure rate from the buffer (hich equals C, provided that Q k (t) > ) and the function p k (t) represents the instantaneous loss probability at the buffer, hich depends on the RED parameters. An explicit expression for p k (t) is given in [5] for RED buffers. If T k (t) denotes the instantaneous delay of buffer k at time t, e can rite T k (t) Q k (t)/c If F k indicates the set of elephants traversing buffer k, A i k (t) and Di k (t) are respectively the arrival and departure rates at buffer k referred to elephants in class i, it results: A k (t) ttk (t) t A i k(t), D k (a)da A k (a)da i F k ttk (t) D i k(a)da t A i k(a)da, hich means that the total amount of fluid arrived up to time t at the buffer leaves the buffer by time t T k (t), since the buffer is FIFO. By differentiating the last equation: ( Dk(t i T k (t)) dt ) k(t) A i dt k(t) C. Source-netork interactions Consider elephants in class i. Letk(h, i) be the hth buffer traversed by them along their path P i of length L i.thertt R i (t) perceived by elephants of class i satisfies the folloing expression: ) L i L i R i (t g i T k(h,i) (t k(h,i) ) g i T k(h,i) (t k(h,i) ) h h (2) /4/$2. (C) 24 IEEE IEEE INFOCOM 24

3 here g i is the total propagation delay experienced by elephants in class i, and t k(h,i) is the time hen the fluid injected at time t by the TCP source reaches the hth buffer along its path P i.wehave: t k(h,i) t k(h,i) T k(h,i) (t k(h,i) ) (3) The loss indicator rate is instead given by: λ i (t R i (t)) α W i(t) R i (t) pf i (t) (4) here W i (t)/r i (t) is the instantaneous emission rate of TCP sources, α is a calibration parameter, and p F i (t) is the instantaneous loss probability experienced by elephants in class i: Finally: p F i (t) A k (t) i L i h [ pk(h,i) (t k(h,i) ) ] rqkd i q(t) i q i e i W i (t) k R i (t) N i here e i k if buffer k is the first buffer traversed by elephants of class i, and otherise; rqk i is derived by the routing matrix, being rqk i if buffer k immediately follos buffer q along P i ; N i is the number of class i active flos. It can be observed that the MGT fluid model is extremely simple, requiring just one equation per class of elephants, thus being capable of scaling to quite large netork models. Hoever, e must also note that the description of TCP mice ith the MGT model is not natural, because (obviously) the start time of each mouse determines its indo dynamics over time. This aspect is not captured by (), and one equation has to be ritten for each mouse, as in [4]. This means that the independence of the fluid model complexity ith respect to the number of flos is lost. Moreover, the MGT model, due to the fact that it only describes the average dynamics, also has problems in coping ith drop-tail buffers. Finally, the calibration parameter in (4), hich is necessary to compensate for the use of the average indo size, instead of the indo size distribution, must be set according to an empirical process. III. PREVIOUS WORK ON FLUID MODELS Fluid models have been recently proposed [3], [4], [5], [6], [7] as a useful approach to estimate the performance of large IP netorks loaded ith TCP traffic. In particular, fluid models ere proposed as a viable alternative to packetbased simulators, since the complexity of fluid models (i.e., the number of equations to be solved) is independent of the number of TCP flos and of link capacities. To the best of our knoledge, fluid models ere first proposed in [5] to study the interaction beteen TCP elephants Equation (2) comprises the propagation delay g i in a single term, as if it ere concentrated only at the last hop. This is just for the sake of easier reading, since the inclusion of the propagation delay of each hop ould introduce just a formal modification in the recursive equation of t k(h,i). and a RED buffer in a packet netork consisting of just one bottleneck link. In [7] the authors have recently extended their model to consider general multi-bottleneck topologies comprising RED routers. As e have already observed, the equations reported in Section II briefly summarize the fluid model proposed in [7], hich constitute the starting point for our ork. This set of ordinary differential equations must be solved numerically, using standard discretization techniques. In [3], [4] an alternative fluid model is proposed to describe the dynamics of the average indo for TCP elephants traversing a netork of drop-tail routers. The behavior of such a netork is pulsing: congestion epochs in hich some buffers are overloaded (and overflo) are interleaved to periods of time in hich no buffer is overloaded, and no loss is experienced, due to the fact that previous losses forced TCP sources to reduce their sending rate. In such a setup, a careful analysis of the average TCP indo dynamics at congestion epochs is necessary, hereas sources can be simply assumed to increase their rate at constant speed beteen congestion epochs. This behavior allos the development of fluid equations and an efficient methodology to solve them. Ingenious queueing theory arguments are exploited to evaluate the loss probability during congestion epochs, and to study the synchronization effect among sources sharing the same bottleneck link. Also in this case the complexity of the fluid model analysis is independent of link capacities and the number of TCP flos. An extension that allos considering TCP mice is also proposed in [3], [4]. In this case, since the dynamics of TCP mice ith different size and/or different start times are different, each mouse must be described ith to differential equations; one representing the average indo evolution, and one describing the orkload evolution. As a consequence, one of the nicest properties of fluid models, the insensitivity of the complexity ith respect to the number of TCP flos, is lost. IV. MODELING A LARGE POPULATION OF TCP SOURCES The class of fluid models that e propose in this paper differs from previous proposals because, instead of describing just the evolution of the average indo size of TCP sources, e model the evolution of the indo size distribution for the TCP flo population. This major improvement in the representation of the TCP sources dynamics gives us the advantage of a greater model flexibility, hich: (i) allos TCP mice to be described in a ay such that the insensitivity of complexity ith respect to the number of TCP flos is maintained, (ii) permits the modeling of netorks in hich AQM routers coexist ith drop-tail routers. In other ords, rather than just describing the average TCP connection behavior, e try to statistically model the dynamics of the entire population of TCP flos sharing the same path. This approach leads to systems of partial derivatives differential equations, and produces more flexible models, hich scale independently of the number of TCP flos /4/$2. (C) 24 IEEE IEEE INFOCOM 24

4 In this section e first introduce the basic model for the TCP flo population. This basic model can be extended by adding several features, hich permit a progressively more accurate description of the behavior of TCP sources. Such extensions are described one by one for the sake of readability, but they can be combined at ill, to obtain models ith the desired level of accuracy and numerical complexity. A. Basic TCP sources To begin, consider a fixed number of TCP elephants. We use P i (, t) to indicate the number 2 of elephants of class i hose indo is at time t. For the sake of simplicity, e consider just one class of flos, and omit the index i from all variables. The source dynamics are described by the folloing equation, for : P(, t) λ(α, t) P(α, t) dα P(, t) here λ(, t) is the loss indication rate. A formal derivation of (5) is given in Appendix A. The intuitive explanation of the formula is the folloing. The time evolution of the population described by P (, t) is governed by to terms: (i) the integral accounts for the groth rate of P (, t) due to sources ith indo beteen and 2 that experience losses; (ii) the second term is the decrease rate of P (, t) due to sources increasing their indo ith rate /. The quantity λ(, t) can be computed by recalling (4): λ(, t) pf (t) (6) in hich the current indo of the sources that emitted the lost fluid approximates the indo value at hich those sources emitted this fluid. Intuitively, this loss model distributes the lost fluid over the entire population, proportionally to the indo size. Note that this loss model does not require any calibration parameter, contrary to the MGT model; indeed, statistics like the variance of TCP indos impact on netork stationary behavior: the MGT model only evaluates the mean value of TCP indos, hile this model evaluates their distribution. B. Accounting for the maximum indo size We no extend the basic model of (5) to account for the maximum indo size of TCP sources, that e denote by W max. It holds: P(, t) min(2,w max ) λ(α, t) P(α, t) dα λ(w max,t)p max (t)u( W max /2) (5) P(, t) (7) for <W max, here u( ) is the unit step function, and P max (t) is the number of TCP flos hose indo is exactly equal to W max. 2 P i (, t) is assumed to be a continuous function IR 2 IR due to the fluid nature of the model For P max (t) e can rite : dp max (t) dt P(, t) lim λ(w max,t)p max (t) (8) W max ith the boundary conditions: P (,t) and lim W max P (, t) P max (t) N. The derivation of (7) is very similar to that of (5). The first term in (7) is the contribution of all TCP sources hich experience losses at indo size beteen and 2 (W max if 2 exceeds it). The second term of (7) is the contribution of all TCP sources at maximum indo size that experience losses; note that this contribution exists only for indos greater than W max /2. The groth rate of P max (t) is obtained as the limit of the usual groth rate ( P(, t)/ )/ of P (, t). The decrease rate of P max (t) is simply λ(w max,t). C. Experiments ith RED In this subsection e discuss some numerical results referring to the mathematical model in (7). Before proceeding e notice that all the results shon in this paper ere obtained by solving numerically the model. For this purpose e applied standard discretization techniques; in particular, a first-order finite differences method for the sources equations and a second-order Runge-Kutta method for the queue equations. Consider the case of a single bottleneck link topology in hich a gentle version of the RED AQM algorithm (min th, maxth 6, pmax.,.) isimplemented, ith to classes of 8 TCP elephants saturating the link capacity (C Mbps), assuming a propagation delay equal to 3 ms. We compare the results of three different experiments, in hich the first elephant class (class ) has alays maximum indo size 64, hile the other class (class 2) has maximum indo size 64, 32 and 24. The packet size for this and all other experiments in this paper is bits. In Fig. e sho the indo size probability density function of elephants in class 2 predicted by our model. In Table I e compare the average indo size, the average queue length and the loss probability for the model and the ns simulator. Note that for loer W max, the average indo size of class 2 elephants is smaller; at the same time, the average indo size for class flos increases, so that the average indo size of all the 6 TCP elephants is roughly constant and equal to 2. A model ithout indo size clipping, like for example the one in [5], [6], [7], is capable of correctly estimating the average indo size of the 6 elephants, but fails in capturing the differences among classes ith different maximum indo size values. The results of ns simulations for the same setup, reported in Table I and in Fig. 2 for comparison, clearly sho that the fluid model is quite accurate. D. Considering Fast Recovery Neer versions of TCP (such as NeReno - see RFCs 2, 258, 2582) avoid halving the indo more than once for /4/$2. (C) 24 IEEE IEEE INFOCOM 24

5 W max Fluid model ns AQL ALP AWS AQL ALP AWS TABLE I MAXIMUM WINDOW SIZE W max AND AVERAGE WINDOW SIZE (AWS) (IN PACKETS) FOR CLASS 2 FLOWS, AVERAGE QUEUE LENGTH (AQL) (IN PACKETS) AND AVERAGE LOSS PROBABILITY (ALP) FOR THE EXPERIMENTS OF SECTION IV-C. Average Windo Size Distribution.4 W max W max 64 W max Windo Size Fig.. Fluid model: average indo size distribution for 8 TCP elephants traversing a single bottleneck link ith RED buffer, varying their maximum indo size; these TCP flos compete ith 8 other TCP elephants ith maximum indo size 64. Probability Probability Average Windo Size Distribution (NS) Windo Size W max 24 W max 64 W max 32 Fig. 2. ns simulator: same as Fig. RTT, even in the case of multiple losses. To model this fact, e divide the population P (, t), representing the number of TCP flos hose congestion indo is at time t, into classes: class L comprises all those sources that experienced losses during the last RTT, hile class O is composed by remaining sources 3 (P (, t) P L (, t)p O (, t)). We can rite: P O (, t) λ(α, t) P O(α, t) dα P O (, t) P L(, t) (9) 3 For the sake of simplicity, the equations in this section and in the rest of the paper do not consider the effect of the maximum indo size. Hoever, in all numerical results that are presented in this paper the effect of the maximum indo size is alays accounted for. P L (, t) λ(α, t) P O(α, t) dα P L(, t) P L (, t) () A formal derivation of (9) and () is reported in Appendix B. An intuitive explanation of the to equations can be provided as follos. In the right hand side of (9), the first to terms account for the decrease rate of the number of elephants of class O hose indo is at time t, due to: (i) sources in class O experiencing losses and moving to class L, (ii) sources in class O increasing their indo. The third term refers to the sources moving to class O from class L after experiencing a RTT ithout losses. In the right hand side of (), the first term accounts for the groth rate of the number of elephants of class L hose indo is at time t, due to sources in class O experiencing losses. The second and third terms account for the decrease rate due to: (i) sources moving to class O from class L after a RTT ithout losses, (ii) sources in class L increasing their indo. More general fluid equations describing TCP elephants and accounting for the TCP threshold mechanisms and for timeouts are reported in Appendix E. E. Modeling drop-tail buffers As e have already mentioned, a fluid model for the description of RED AQM schemes as originally proposed in [5]. RED matches quite ell the fluid modeling approach, since in RED buffers the loss probability is a smooth function of the queue length averaged over a rather long time indo. The case of drop-tail buffers is instead much more difficult to describe ith fluid models, since in this case the loss probability is a discontinuous function of the instantaneous queue size. Many studies have shon that the behavior of netorks carrying TCP traffic is pulsing: congestion epochs in hich some buffers are overloaded (and overflo) are interleaved to periods of time in hich traffic is lighter, buffers are not saturated, and no loss is experienced. Light traffic periods are the result of losses at the previous congestion epochs, that force TCP sources to reduce their emission rate. As a consequence, the loss processes experienced by TCP flos traversing drop-tail buffers are quite bursty. This burstiness induces a high degree of correlation (synchronization) among the dynamics of TCP sources sharing the same buffer. In addition, during congestion epochs, losses are not evenly distributed among TCP flos, but are more likely to affect TCP sources ith larger indo size. In this context, it is necessary to distinguish among sources ith different instantaneous indo size, hile at the same time accounting for the effects of the TCP fast recovery mechanism, hich prevents TCP sources from halving their indo several times ithin one round trip time. The level of detail in the description of the TCP sources dynamics adopted in this paper allos an easy description of /4/$2. (C) 24 IEEE IEEE INFOCOM 24

6 DT Queue size model NS Time [s] Fig. 3. Fluid model and ns simulator: queue size evolution for one bottleneck link ith data rate C Mbps, propagation delay 3 ms, fed by a droptail buffer ith capacity equal to packets, and traversed by 3 TCP elephants, ith maximum indo size 64 packets. AWS AQL ALP Fluid model ns TABLE II AVERAGE WINDOW SIZE (AWS), AVERAGE QUEUE LENGTH (AQL) AND AVERAGE LOSS PROBABILITY (ALP) FOR THE SAME SETUP OF FIG. 3 the time-dependent behavior of the packet loss probability: p k (t) max(,a k(t) C) I {Qk (t)b A k (t) k } () that is, the loss probability p k (t) equals (A k (t) C)/A k (t) (the relative difference beteen the instantaneous arrival rate and the service rate) only hen the buffer is full, being B k the capacity of buffer k, and I { } the indicator function. A different approach is used in [3] and [4] to describe the dynamics of the average indo size for TCP flos traversing a netork ith drop-tail buffers. In those papers, the loss indicator rate is obtained by applying queueing theory results hich are not internal to the fluid model. This approach is probably difficult to generalize to netorks including both drop-tail and AQM buffers. F. Experiments ith drop-tail buffers In this subsection e briefly comment some numerical results obtained ith our modeling approach in the case of drop-tail buffers. First, e consider the case of a single bottleneck link (ith data rate C Mbps, propagation delay 3 ms), traversed by just one class of 3 TCP elephants, ith maximum indo size 64 packets; the maximum buffer size is set to. The curves in Fig. 3 sho the queue size evolution over time. Our model captures the ell-knon oscillating behavior of TCP, hich as observed in simulation experiments as ell as measurements [2], [3]. The results of ns simulations are reported in Fig. 3 and Table II for comparison, and again sho that the fluid model is accurate. The second scenario e consider is a netork topology comprising to links, the first fed by a RED buffer, the second Average indo size TCP TCP TCP Time [s] Fig. 4. Fluid model: indo size evolution for three long-lived TCP flos ith interfering UDP traffic. 3 Average indo size (NS) TCP TCP TCP Time [s] Fig. 5. ns simulator: same as Fig. 4 fed by a drop-tail buffer 4. The links are crossed by five classes of elephants. To classes of TCP flos are single-hop (TCP crosses the first link, TCP crosses the second one), hile the other one (TCP2) crosses both links; the to links are also crossed by to interfering classes of CBR UDP flos (UDP3 crosses the RED buffer, UDP4 crosses the drop tail buffer). UDP3 is on in the time interval [, 3] s, UDP4 in the time interval [2, 4] s: hen the UDP flos are on, they consume about 4% of the bandidth of their link. Fig. 4 shos plots of the indo size evolution for the three TCP flo classes. When UDP3 starts, the indo size of the to TCP flo classes sharing the same link decreases; hen also UDP4 starts, the indo size of TCP decreases, and again that of TCP2 goes don, in favor of TCP. The indo size of TCP and TCP2 increases hen UDP3 ends, hile those of TCP and TCP2 increase hen UDP4 ends. The results of ns simulations for the same setup are reported in Fig. 5 for comparison, and once more sho that the fluid model is quite accurate (in addition, in Fig. 6 and 7 e overlap the curves of the model and the ns simulator for the TCP and TCP elephants). These results prove that our model can cope ith both controlled (TCP) and uncontrolled (UDP) long-lived flos, 4 It is orth observing that all previous applications of fluid models to packet netorks alays considered either RED buffers, or drop-tail buffers, but the to types of buffers ere never mixed, since the fluid models could not support this feature /4/$2. (C) 24 IEEE IEEE INFOCOM 24

7 Average indo size (TCP) Average indo size (TCP) model NS Time [s] Fig. 6. Overlap of TCP curves from Fig. 4 and model NS Time [s] Fig. 7. Overlap of TCP curves from Fig. 4 and 5. and is capable of predicting the TCP transient effects due to the presence of on-off interfering sources. G. Modeling TCP mice We no come to the very important issue of modeling TCP mice, hose dynamics are mostly, if not completely, due to the slo start algorithm, and in particular to the first slo start phase that is executed at the start of the TCP connection. For this reason, in order to model TCP mice, e model the initial slo start phase up to the first loss or to the first hit of the maximum indo size, and then e assume that flos stay in congestion avoidance for the rest of the connection lifetime. Let P s (, t, l) be the number of flos in slo start ith indo size and residual orkload l at time t. Analogously, P (, t, l) refers to flos in congestion avoidance. We can rite: P(, t, l) P(, t, l) P(, t, l) l λ(α, t) P(, t, l) l l P(α, t, l ) dα λ(α, t) P s(α, t, l ) dα (2) P s (, t, l) P s (, t, l) P s (, t, l) l P s (, t, l) l l λ(α, t) P s(α, t, l ) dα γ(t, l) (3) A formal proof of these equations is given in Appendix C. An intuitive explanation is as follos. In (2), the first to terms on the right hand side account for the decrease rate of P (, t, l) due to: (i) sources increasing their rate (first term), (ii) sources terminating because of null residual orkload (second term). The last three terms account for the groth rate of P (, t, l). The third term takes into account those sources ith previous residual orkload slightly greater than l, assuming at time t a value l. The fourth term represents those sources in congestion avoidance ith indo beteen and 2 and residual orkload l that experience a loss. They are added to P (, t, l) because their indo is halved (and becomes ) and their residual orkload goes back to l, as the lost unit of fluid must be retransmitted. Finally, the fifth term represents an increase similar to the fourth term, applied to sources in slo start: these sources, ith indo size beteen and 2 and residual orkload l, experience a loss and consequently move to a state in hich they are in congestion avoidance, their indo is and their residual orkload goes back to l. Equation (3) is very similar to (2), since the evolution of P s (, t, l) ith respect to the residual orkload (second and third terms) is the same, and the first term differs only for the fact that the indo groth is in this case exponential rather than linear. Moreover, the fourth term refers to sources moving into congestion avoidance because of a loss (similarly to the fifth term of (2)), and the last term accounts for nely activated TCP mice. Note that the representation of the TCP indo dynamics over the (t, ) space allos us to distinguish among TCP mice ith different instantaneous indo size, thus providing the correct level of detail for the analysis of this type of TCP flos. Indeed, TCP mice start in slo-start, ith indo, and then their indo evolves according to (2) and (3). The model of TCP mice can be simplified by assuming flo lengths to be exponentially distributed, ith average L. Thanks to the memoryless property of the exponential distribution, e can rite: P(, t) P(, t) ( p L(t)) L P(α, t) λ(α, t) dα P(α, t) α dα λ(α, t) P s(α, t) dα (4) /4/$2. (C) 24 IEEE IEEE INFOCOM 24

8 P s (, t) P s (, t) ( p L(t)) α P s(α, t) dα L λ(α, t) P s(α, t) dα γ(t) (5) here p L (t) is the average loss probability experienced by the flo, during its total active period. The formal derivation of the second term is reported in Appendix D. We can approximate p L (t) by using the same approach proposed in [5], [6] to evaluate the average loss probability in a RED queue; e obtain: p L (t) (t) L p L(t) (t) L pf (t) (6) being p F (t) the instantaneous loss probability, defined in (4), and (t) the average indo size at time t. We ish to stress the fact that (4)-(6) provide quite a poerful tool for an efficient representation of TCP mice, since a ide range of distributions (including those incorporating long range dependence) can be approximated ith a good degree of accuracy by a mixture of exponential distributions [4]. H. Randomness in fluid models As e have observed, fluid models provide a deterministic, phenomenological description of the netork behavior, thus departing from the common approaches of attempting a probabilistic description of the netork dynamics over a huge state space. Hoever, in netorking scenarios ith only TCP mice, the pure determinism of fluid models fails to provide useful information about the buffering phenomena ithin the netork: if buffers are underloaded, their corresponding fluid model is constantly empty, hereas if buffers are overloaded, their corresponding fluid model gros to infinity. That is, the intrinsic determinism of fluid models does not allo them to grasp the random fluctuations in the mice arrival pattern and size hich cause buffer occupancies to gro above zero ithout diverging. In order to overcome this limit of fluid models in the analysis of the performance of TCP mice, e believe that some randomness has to be introduced ithin fluid models. This can be done at several levels. The deterministic mice arrival rate γ(t) in (5) can be replaced by a Poisson counter ith average γ(t), thus making (5) a stochastic partial differential equation. The deterministic completion process of TCP connections can be replaced by an inhomogeneous Poisson process hose average at time t is represented by the second term of (4). Note that this term suggests that, due to retransmissions, the completion time of connections increases as their loss probability gros. Instead of assuming the orkload emitted by TCP sources to be a continuous deterministic fluid process ith rate W i (t)/r i (t), it can be taken to be a Poisson point process (possibly ith batch arrivals) ith the same rate. Fluid model ns mice elephants bottleneck AR ACT AWS AQL ALP (flos/s) (ms) (pck) (pck) TABLE III ARRIVAL RATES (AR), AVERAGE COMPLETION TIMES (ACT), AVERAGE WINDOW SIZE (AWS), AVERAGE QUEUE LENGTH (AQL) AND AVERAGE LOSS PROBABILITIES (ALP) FOR THE EXPERIMENTS OF SUBSECTION IV-I. Of course, this is only a preliminary attempt to introduce randomness in fluid models, so as to be able to study the behavior of TCP mice; e do not claim any optimality of this approach, and a deeper investigation is needed about the possible ays of introducing randomness ithin fluid models ithout losing the property of independence of complexity ith respect to the number of flos. I. Experiments ith mice In this subsection e discuss results for netorking scenarios comprising TCP mice. First, e consider a case in hich both mice and elephants coexist. Results refer to a single bottleneck link fed by a drop-tail buffer. The buffer size is equal to packets, the link capacity is C Mbps, the propagation delay beteen the TCP sources and the buffer is 3 ms. 2 TCP elephants are active, ith maximum indo size 64 packets, and coexist ith TCP mice, hose length is geometrically distributed ith mean 2 segments. The TCP mice arrival rate is set equal to, 2 and 4 connections/s. The queue size evolution is similar to that already presented in Fig. 3. The presence of elephants is crucial in order to saturate the link bandidth, because they consume the capacity that is not used by mice, hich are practically uncontrolled. Indeed, in Table III e can see that the average indo size for elephants decreases hen the arrival rate of mice increases. In the same table, e also report the average completion time (ACT) of mice flos, obtained from the average number of active mice ith Little s theorem. Table III and Fig. 8 also report the results of ns simulations for the same setup, for comparison: the fluid model is quite accurate in this case too. If infinite flos are removed from the scenario, as e discussed in the previous subsection, fluid models cannot provide useful information about the netork performance. Consider a single bottleneck link fed by a drop-tail buffer, ith capacity equal to 256 packets. The link capacity C is Mbps, hile the propagation delay beteen sources and buffer is 3 ms; there are 3 classes of TCP mice: they all have geometrically distributed size, 89% ith average packets, % ith average, and % ith average. The maximum indo size is set to 64 packets for all TCP sources. Experiments ith loads equal to.6,.8 and.95 sho that the buffer is alays empty, hile loads over /4/$2. (C) 24 IEEE IEEE INFOCOM 24

9 Average indo size of infinite flos Time [s] model ns model ns Fig. 8. Average indo size evolution for elephants competing ith mice on a single bottleneck link; the average indo size decreases hen the mice arrival rate gros from to 4 connections/s. Probability.... e-5 DT Queue Size Distribution model ith batched Poisson arrivals model ith Poisson arrivals NS e Queue Occupancy Fig. 9. Queue size distribution for single drop tail bottleneck, varying the random process modeling the orkload emitted by the TCP sources; comparison ith ns simulator. ALP AQL ACT Poisson 3.6 8, 23, 652 batched Poisson , 233, 892 ns , 274, 97 TABLE IV AVERAGE LOSS PROBABILITY (ALP), AVERAGE QUEUE LENGTH (AQL) AND AVERAGE COMPLETION TIMES (ACT) OF THE THREE CLASSES OF MICE FOR THE SETUP OF SUBSECTION IV-I, HAVING INTRODUCED RANDOM ELEMENTS. obviously saturate the buffer (plots are not reported here). If e introduce randomness in the fluid model, using the approach described in the previous subsection, e obtain the results shon in Fig. 9, hich refer to load.8: the buffer occupancy distribution still has a peak in, but the variance has increased. If e use a Poisson process to model the instants in hich packets (or, more precisely, units of fluid) are emitted by TCP sources, the results generated by the fluid model do not match ell the results obtained ith the ns simulator, as can be observed in Table IV and Fig. 9. Hoever, the performance predictions obtained ith the fluid model become extremely accurate hen the orkload emitted by TCP sources is taken to be a Poisson process ith batch arrivals, ith batch size distribution derived from the indo size distribution of TCP mice. This approach derives from recent results about the close relationship existing beteen the burstiness of the traffic generated by mice and their indo size [5]. J. Experiments ith a real topology In order to conclude our validation of the fluid models proposed in this paper, e present some results referring to a mesh netork topology (depicted in Fig. ), that mimics the Italian Academic and Research Netork (named GARR). The netork backbone comprises 4 core routers connected by 5 links ith rates equal to either Gbps or 2.5 Gbps, and propagation delay 3 ms. Each core router is connected to edge routers through 622 Mbps links, hose propagation delays are comprised beteen 5 and 25 ms. The total number of edge routers is 8. In addition, a 2.5 Gbps ms transoceanic link is connected to one of the four core routers. Link speeds: 2.5 Gb/s Gb/s 622 Mb/s Fig.. simulated GARR netork topology 255 elephants enter the netork through edge routers, and 35 elephants through the transoceanic link. In addition, 85 mice/s enter through edge routers, and 54 mice/s through the transoceanic link. The mice length is taken to be geometrically distributed ith average equal to 2 packets. All the mice and elephants are uniformly distributed among all the possible sources and destinations, generating about 35 classes of flos (for either mice or elephants). The numerical solution of the netork dynamics over a period of one minute implies the investigation of the dynamics of about.2 millions TCP flos, and requires just fe minutes over a standard GHz PC, thanks to the excellent scalability properties of the model. Figs. and 2 report the throughput and completion time performance for TCP elephants and mice, respectively, in decreasing order. Both cases of a deterministic fluid model and of a model ith randomness are considered. In this particular case, the presence of elephants makes the impact of randomness marginal. Hoever, if e remove the elephants, as e already sa, the randomness becomes necessary in order to observe interesting phenomena. V. CONCLUSIONS In this paper e have proposed a ne fluid model approach for the investigation of the performance of IP netorks loaded /4/$2. (C) 24 IEEE IEEE INFOCOM 24

10 Completion Time [s] Class Stochastic Deterministic Fig.. Average completion time per class, in decreasing order, for mice present in the GARR topology. Throughput [Mbps] Stochastic Deterministic Class Fig. 2. Average throughput per class, in decreasing order, for elephants present in the GARR topology [4] F.Baccelli, D.Hong, Flo Level Simulation of Large IP Netorks, IEEE Infocom 23, San Francisco, CA, March 23. [5] S.Misra, W.B.Gong, D. Tosley, Fluid-Based Analysis of a Netork of AQM Routers Supporting TCP Flos ith an Application to RED, ACM SIGCOMM 2, Stockholm, Seden, August 2. [6] C.V.Hollot, V.Misra, D.Tosley, W.B.Gong, On the Design of Improved Controllers for AQM Routers Supporting TCP Flos, IEEE Infocom 2, Anchorage, Alaska, USA, April 2. [7] Y.Liu, F.Lo Presti, V.Misra, D.Tosley, Fluid Models and Solutions for Large-Scale IP Netorks, ACM Sigmetrics 23, San Diego, CA, June 23. [8] S.Deb, S.Shakkottai, R.Srikant, Stability and Convergence of TCP-like Congestion Controllers in a Many-Flos Regime, IEEE Infocom 23, San Francisco, CA, March 23. [9] S.Shakkottai, R.Srikant, Ho Good are Deterministic Fluid Models of Internet Congestion Control? IEEE INFOCOM 22, Ne York, June 22. [] P.Tinnakornsrisuphap, A.Makoski, Limit Behavior of ECN/RED Gateays Under a Large Number of TCP Flos, IEEE Infocom 23, San Francisco, CA, March 23. [] S.Floyd, V.Jacobson, Random Early Detection Gateays for Congestion Avoidance, IEEE/ACM Transactions on Netorking, vol., n. 4, pp , August 993. [2] V.Jacobson, Congestion Avoidance and Control ACM Sigcomm 998, Vancouver, Canada, September 998. [3] L.Zhang, D.Clark, Oscillating Behavior of Netork Traffic: a Case Study Simulation, Internetorking: Research and Experience, vol., n. 2, pp. -2, 99. [4] A. Feldmann, W. Whitt, Fitting Mixtures of Exponentials to Long-Tail Distributions to Analyze Netork Performance Models, IEEE Infocom 97, Kobe, Japan, April 997. [5] M.Garetto, D.Tosley, Modeling, Simulation and Measurements of Queuing Delay Under Long-Tail Internet Traffic, ACM Sigmetrics 23, San Diego, CA, June 23. [6] J.Padhye, V.Firoiu, D.Tosley, J.Kurose, Modeling TCP Throughput: A Simple Model and its Empirical Validation, ACM Sigcomm 998, Vancouver, Canada, September 998. by TCP mice and elephants (as ell as UDP flos). Our approach exploits partial differential equations, thus permitting the description of distributions, instead of averages, hence achieving better accuracy in the results ith respect to previously proposed fluid modeling approaches. The performance estimates obtained ith our fluid models have been compared against ns simulations in the cases in hich the latter are feasible, proving both the accuracy and the scalability of the proposed modeling approach. In addition, e applied the proposed fluid modeling approach to a realistic netork, shoing that the solution of reasonable size netorks can be obtained ith limited computational complexity. Further ork on this topic ill include the investigation of larger netorking setups, the optimization of the numerical solution techniques, as ell as a more detailed investigation of the possible approaches to introduce randomness in the modeling paradigm, so as to improve the accuracy in the description of netorks here only mice are present. REFERENCES [] R.Cruz, A Calculus for Netork Delay, Part I: Netork Elements in Isolation IEEE/ACM Transaction on Information Theory, vol. 37, n., pp. 4-3, January 99. [2] J.Y Le Boudec, P.Thiran, Netork Calculus: a Theory of Deterministic Queuing Systems for the Internet, Springer Verlag LNCS 25, June 2. [3] F.Baccelli, D.Hong, Interaction of TCP Flos as Billiards, IEEE Infocom 23, San Francisco, CA, March 23. APPENDIX A. Proof of Eq. (5) - basic sources We ish to estimate the evolution of P (, t); e define v(, t) P(, t)/ as the probability density of the indo distribution at time t. Consider a small enough t such that R(t t). Let P be the number of sources ith indo at time t, but ith indo >at time t t. All the sources hich do not experience any loss indication during the interval [t, t t) increase their indo ith rate /. Among these sources, P includes only the ones ith initial indo t/, since they ill exceed by time t t. If e assume to model (locally) the loss indication process ith a Poisson process ith rate λ(, t), the probability that no losses are experienced during t is ( λ(α, t) t o( t)), then: P t/ P t ( λ(α, t) t o( t)) v(α, t)dα v(, t) (7) Let no P be the number of sources ith indo >at time t, but ith indo at time t t. P include only the sources (i) ith indo in the range (, 2 t/] at time t, and (ii) receiving a loss indication in the interval [t, t t). Note that the probability of receiving multiple loss /4/$2. (C) 24 IEEE IEEE INFOCOM 24

11 indications is o( t), hence negligible. Hence, P t/ λ(α, t) t v(α, t)dα o( t) P λ(α, t)v(α, t)dα (8) t Since P (, t t) P (, t) P P, e can find (5): P P P (, t) lim t t λ(α, t)v(α, t)dα v(, t) B. Proof of Eq.(9) and () - sources ith fast recovery mechanisms The proof is similar to the previous one. Let v O (, t) P O (, t)/ and v L (, t) P L (, t)/. Consider the sources of class O moving to class L during the interval [t, t t); among these, P OL ill have a indo 2 and ill contribute to increase P L (, t). Analogously to (A): P OL λ(α, t) tv O (α, t)dα The number of sources of class L exceeding by time t t is, analogously to (A): ( λ(α, t) t)v L (α, t)dα (9) P L t/ No consider the population of sources hich ill leave class L because an RTT is elapsed. We assume an exponential distribution of the departure time of each source from class L, ith average. Hence, the number of sources moving from class L to class O ill be: P LO P L (, t) t/, by observing that the number of sources that already left class L by the end of t ill be e t/ t/ o( t). No observe that the P LO, defined as the number of sources moving from class L to class O and exceeding indo, ill include sources counted in both P LO and P L. These source can be derived by P L, since P LO P L t/. No e are able to add all the possible contributions: P L P OL (, t) lim P L P LO P LO (2) t t By recalling (9), e can compute: t ( P L P LO ( t hose limit is: lim t t ) t/ ) P L ( t ) ( λ(α, t) t)v L (α, t)dα t/ v L (α, t)dα P L (, t) In other ords, P LO is negligible ith respect to P L. Hence, from (2) e find (): P L (, t) We can no estimate: λ(α, t)v O (α, t)dα P L(, t) P L (, t) P O P LO P LO (, t) lim P O P OL t t here P O are the sources in class O exceeding indo by the time interval t and P OL the sources moving from class O to class L, due to losses. It holds: P OL lim lim t t t t λ(, t) tv O (α, t)dα λ(α, t) P O (α, t)dα Analogously to P L, P O ( P O(, t)/ )/. Itcan be shon that P LO is negligible ith respect to P LO. Hence, e can obtain Eq. (9): P O (, t) λ(α, t) P O (α, t)dα P O (, t) R(T ) R(T ) P L(, t) C. Proofs of Eq. (2) and (3) - sources ith finite flos. The only terms hich need a formal proof are the ones hich model the orkload evolution. P is the number of sources hich enter P (, t, l) during a time interval of size t because their orkload has just decreased. P is given by all the sources ith indo beteen and, and residual orkload beteen l and l t/, being / t the instantaneous emission rate of sources ith indo. Formally, P Finally, α l t/ βl l t/ βl P 2 P (α, t, β)dαdβ β P β (, t, β)dβ ) P (, t, l) (, t, l t P lim t t P (, t, l) l To account for the sources hich stop their activity during the time interval of size t, it is enough to set l /4/$2. (C) 24 IEEE IEEE INFOCOM 24

12 D. Proofs of Eq. (4) and (5) - sources ith finite flos exponentially distributed. Regarding (4) and (5), e prove formally only the terms related to the contribute of the variation of the orkload on the population P (, t). Consider a time interval of size t and a source hich does not experience any loss ith indo ; the probability that this source stops, i.e. its residual life time is less than t, is equal to exp{ t/(l)} t/(l), thanks to the memoryless property. Then, the contribution of the sources stopping is: P lim t t α α P (α, t)dα L The final contribution is given by multiplying the previous formula for ( p L (t)) corresponding to the number of sources not experiencing any losses. E. Sources ith slo start and threshold mechanisms We report here ho an accurate model of sources employing slo start and threshold mechanisms can be developed ithin our modeling frameork. Let P (, t, s) be the number of flos ith indo and threshold s, ith s. Whenever a source experiences a loss signal, it halves its indo and sets its threshold equal to the indo. In this model, no memory is kept of the previous threshold before the experienced loss. We have to discriminate among three cases, depending on the relation beteen and s. We can derive: P (, t, s) α α β β s P (, t, s) α 2s α β s s β P (, t, s) α2s 2s α s β s β 2 P (, t, β)dβ β 2 P (, t, β)dβ β λ /2 (α, t) 2 P (α, t, β)dαdβ β λ /2 (α, t) 2 P (α, t, β)dαdβ β for s (2) s β 2 P (, t, β)dβ β λ /2 (α, t) 2 P (α, t, β)dαdβ β β β s λ /2 (α, t) 2 P (α, t, β)dαdβ β for s< 2s (22) s β 2 P (, t, β)dβ β λ /2 (α, t) 2 P (α, t, β)dαdβ β λ /2 (α, t) 2 P (α, t, β)dαdβ β for >2s (23) P (,t,s) P (,t,s) T TO 2s λ TO (α, t) 2 P (α, t, β)dαdβ α β β for s (24) here T TO is the timeout duration for the TCP sources, that is taken to be constant. Equation (2) models the case hen s. The population P (, t, s) is decreased by: (i) the sources in congestion avoidance, ith threshold less than and decreasing rate / (first term), (ii) the sources in slo start, ith threshold beteen and s, ith decreasing rate / (second term). To understand the meaning of the integrals, note that ( 2 P (, t, s)/ s) is the density function of sources ith indo and threshold s. λ /2 (, t) is the loss rate given by loss recognition mechanisms (like duplicate acks, etc.) for sources ith indo equal to, hose effect is to halve the indo and set the ne value of the threshold. The population is increased by the sources experiencing the loss signal, in to cases: (i) hen the indo is beteen and 2 (third term), (ii) hen the indo is less than and the threshold is larger than s (fourth term). Note that the integral ith respect to s can be solved analytically for all the four terms and then used in the numerical solver. No consider (22), for beteen s and 2s. The population of sources is decreased by all the sources (in congestion avoidance) ith threshold s (first term). The population is increased by: (i) sources ith indo and any threshold s (second term), (ii) sources ith indo beteen and 2s and threshold less than s (third term). Finally, (23) models the sources ith indos 2s. The population ill be decreased by: (i) sources increasing their indo (first term), or (ii) sources ith threshold s and indo experiencing losses (second term), since their ne threshold ill be set equal to s /2 >s, hich is outside the considered population. The population ill be decreased only by sources experiencing losses ith indo 2s and threshold greater than s. The population ith sources at indo is described by the boundary equation (24). We assume that the time taken by sources to gro their unit indo decays exponentially ith average T TO ; this explains the first term. The second term is due to sources ith indo 2s experiencing a timeout; their threshold ill be set equal to a value s, independently from the initial threshold. Indeed, λ TO (, t) is the rate at hich timeouts are experienced; timeouts reset the indo to one. Hence, the total loss rate is λ(, t) λ /2 (, t)λ TO (, t). The function λ TO, hich models the source reaction to losses, and the occurrence of timeouts, is affected by the version of TCP. One possible choice is given by the approximate model proposed in [6] for hich: λ TO (, t) min(, 3/)/p F (t) /4/$2. (C) 24 IEEE IEEE INFOCOM 24