Fast Evaluation of Ensemble Transients of Large IP Networks. University of Maryland, College Park CS-TR May 11, 1998.

Transcription

1 Fast Evaluation of Ensemble Transients of Large IP Networks Catalin T. Popescu A. Udaya Shankar Department of Computer Science University of Maryland, College Park CS-TR-9 May, 998 Abstract We extend a numerical approximate solution method (the Z-iteration) to time-dependent open networks of M/M// and M/M//K queues, and apply the method to obtain transient performance metrics of large IP networks. The method generates a set of coupled differential equations, one for each queue in the network. The equations are numerically unstable under certain conditions (e.g., large bandwidths and buers), and we present techniques to overcome this problem. The resulting numerical procedure is accurate and very fast. For example, a -second evolution for a -node network with high-speed links ( packets/sec) and large buers ( packets) was obtained in 8 minutes on an Ultra Sparc, whereas simulation would take days. Introduction Evaluating the performance of large IP networks is an important problem, but an adequate technique does not currently exist. Analytical approaches cannot handle many critical network features. Packet-level simulation is popular but it is extremely slow when bandwidths are high and buers are large, even for networks of moderate size. Flow-level simulation is faster but its accuracy is questionable [because it makes the unrealistic assumption that the interval between successive changes in the network-wide ow pattern is large enough so that steady state holds for most of the interval]. We take an alternative approach, based on transient metrics. We are interested in observing the behavior of a network when a perturbation is applied, such as a node/link failure, load change, routing change, etc. We would like to know, for example, which links become congested and how quickly, how long an overload can be maintained before the blocking probability exceeds a specic value, and such like. Time-dependent queuing systems are a natural modeling tool for investigating such issues. Their arrival and service rates can vary over time to reect external or feedback control, and solving them yields transient responses to network and workload changes. The diculty, of course, is that these systems are not tractable by traditional approaches. Analytical and simulation approaches fail for the same reasons mentioned above. Numerical solutions are unmanageable for realistic networks because the state space becomes enormous. This work is supported partially by ARPA contract number DABT9C7 and DoD contract number MDA997C to the University of Maryland. It should not be interpreted as representing the opinions or views of ARPA, DoD, or the U.S. Government.

2 The Z-iteration is an attempt to address this problem. It is a numerical approximate solution method that yields the time evolution of ensemble metrics (e.g. instantaneous queue size distribution) at a cost several orders cheaper than simulation. The rst version of the Z-iteration [, ] handled single multi-class multi-resource queues, adequate for analyzing connection-oriented networks with strict access control and resource reservation. In this paper, we develop another formulation that also handles networks of time-dependent queues, appropriate for modeling classical datagram networks. Figure illustrates an application of this Z-iteration. The network in Figure (a) has nodes, links, and 7 end-to-end user classes subject to shortest-hop routing. At time t = sec, the link marked by the arrow fails and routes are updated. Figure (b) shows the evolution of the instantaneous average queue size at the link from nodes to, obtained from Z-iteration and from simulation. The Z-iteration took 8 seconds, compared with. hours for packet-level simulation (averaging over runs) x Simulations runs The evolution of N at queue > 7 8 N in packets (a) (b) Figure : A network and example evolution The rest of the paper is organized as follows. Section describes how we model a datagram network by a network of time-dependent queues. Section describes the Z-iteration for single queues and for networks of queues. Section examines numerical stability issues. Examples are presented in Section. We conclude in Section. Network Model We consider a datagram network with some routing algorithm. The links have xed bandwidths and the routers allocate xed-size buers for each outgoing link. The trac consists of endto-end user classes, each dened by source node, destination node, and time-dependent packet generation rate. We assume xed-size packets. We focus on networks with high-speed links ( packets/sec) and large link buers ( packets), similar to the IP networks of today. We model each outgoing link of a node by a M/M//K queue with service rate equal to the link bandwidth and max queue size K equal to the link buer space. The routing between the queues is determined by the network topology, user classes, and routing tables. To illustrate, suppose node A has an outgoing link A to node B, and node B has outgoing links, B, B, and B. Then in the queueing model, the output of A can go to B, B, B,

3 or be absorbed within node B, as shown in Figure. The probability r A,B of going to B is given by the fraction of the total user class rates in A that are forwarded to B (note that the path of a user class is determined by the routing tables). For example, if the trac that arrives at queue A consists of two classes, one with rate pkts/sec and another with rate pkts/sec, and node B routes the rst class to B and the second class to B, than the routing probability for B is =( + ) = =8, for B is =( + ) = =8, and for B is. Queue A ra,b Queue B ra,b Queue B ra,b Queue B Node A Node B Figure : The queue model for a link from node A to node B. Z-iteration The Z-iteration is an ecient numerical approximation method that computes instantaneous ensemble metrics of time-dependent queuing systems. It is based on functional approximations of relationships between instantaneous metrics by the corresponding steady-state relationships. These approximations allow the evolution of the metrics to be dened by a small number of dierential equations, rather than the large number of Chapmann-Kolmogorov equations (which are as many as the maximum queue size). Table gives the notation we use for a queue. Instantaneous parameters refer to a queue with time-varying arrival and service rates. Steady-state parameters refer to a queue in steady state, with constant arrival and service rates.. Old Version The rst version of the Z-iteration was based on the following ow equation, obtainable from the Chapmann-Kolmogorov equations: dn dt = [ B] U () The idea is to express B and U in terms of N, resulting in a single scalar dierential equation for N. It turns out that the relationship between B and N is very well approximated by the relationship between steady-state B and steady-state N. The same holds for the relationship between U and N. Thus we want functions expressing B and U in terms of N. What is available, however, are functions expressing N, B, and U in terms of the steady-state trac intensity (= =). We denote these functions by F N (), F B (), and F U (), respectively. For example, for a M/M// queue we have [] N = F N () =

4 N B U z N B U F N () F B () F U () instantaneous arrival rate at time t instantaneous service rate at time t instantantaneous average queue size at time t instantaneous blocking probability at time t instantaneous utilization at time t instantaneous virtual trac intensity at time t (= =) steady-state arrival rate steady-state service rate steady-state trac intensity (= =) steady-state average number of customers steady-state blocking probability steady-state utilization function yielding N in terms of function yielding B in terms of function yielding U in terms of Table : Notation For a M=M==K queue, we have []: B = F B () = U = F U () = () N = F N () = B = F B () = U = F U () = (K + )(K+) (K+) K (K+) K () (K+) For M/M//, we can invert F N () and so obtain B and U in terms of N, specically, B = and U = N=(N + ). But in general, including the case of blocking queues, we cannot invert F N () analytically. So instead we inverted numerically, using another approximation as follows: U is computed from N assuming a non-blocking system. B is computed as the xed point of B = F B () and = U=( B) (obtained by equating the inow ( B) to the outow U). The resulting value of is simply the steady-state trac intensity value consistent with B and N. This approach works very well for M/M// and M/M/K/K queues, and for M/M//K queues when <. But it does not work for networks of these queues or for M/M//K queues when >.. New Version We now develop a formulation of the Z-iteration that eliminates the non-blocking assumption used in the numerical inversion operation above. This version works for open networks of M/M//K and M/M// queues. As mentioned above, formulas for N, B and U are usually in terms of. This suggests that we introduce an instantaneous version of, which we refer to as the instantaneous virtual trac

5 intensity, denoted by z, and develop a dierential equation for z rather than for N. Then N, U, and B can be approximated by N = F N (z) B = F B (z) U = F U (z) () Although z is ctitious, it has a natural interpretation: at any time t, it is the amount of trac intensity that if applied constantly would result in steady-state N, B, and U equal to N, B, and U, repsectively. In fact, z is just a more accurate version of the iterate that appears in the numerical xed-point inversion in the old version. Note that z is not equal to =. To obtain a dierential equation for z, we start with the dierential equation for N dn dt = [ B] U; Replacing dn=dt by (dn=dz)(dz=dt), dn=dz by df N (z)=dz, B by F B (z), and U by F U (z), we obtain dz dt = df N (z)=dz [( F B (z)) F U (z)] () Thus we have a scalar dierential equation whose solution yields the evolution of z. Plugging z into equation () yields evolutions of N, U, and B. Equation () can be instantiated for any type of M/M/ queue. For a M/M// queue, we obtain dz = ( z) ( z) () dt For a M/M//K queue, we obtain dz dt = ( z (K+) )( z K )( z) ( z) (7) ( z (K+) ) (K + ) z K ( z) Example (M/M//K) We apply the method to M/M//K queues driven by constant and. The queues are initially empty. The accuracy of the results is demonstrated by comparing against direct solution of the Chapman-Kolmogorov equations. Figure has plots of N and U for = :, = : and K = 7. Figure shows N and B for = :, = : and K =.. Networks of Queues We extend the Z-iteration to networks of M/M// and M/M//K queues. Here, a departure from queue i is routed to P queue j with a time-dependent probability r ij, and leaves the network with probability j r ij. The arrivals to queue i consist of external arrivals i (from outside the network) and departures from queues in the network routed to queue i. The total arrival rate of queue i, denoted i, is given by i = i + nx j= r ji j U j : (8) Any arriving customer can be blocked, except, of course, customers feeding back from queue i. The dierential equation for z i, the instantaneous virtual trac intensity of queue i, is

6 . Evolution of N.8 Evolution of U Chapman Kolmogorov.7... N U..... Chapman Kolmogorov Figure : Results for M/M//K queue with = :, = :, K = 7 obtained by appropriately modifying equation () or (7). If queue i is a M/M// queue, we have dz i dt = ( z i ) [ i + nx j=;j=i If queue i is a M/M//K queue, we have r ji j F Uj (z j ) i ( r ii )z i ] (9) dz i dt = ( z (Ki+) i )( z Ki i )( z i ) ( z (Ki+) i ) (K i + ) z Ki i ( z i ) [ i + ( nx j=;j=i r ji j F Uj (z j )) i ( r ii )z i ] () Example (network of M/M//K queues) We apply the method to networks of M/M//K queues driven by constant and. The networks are initially empty. The accuracy of the results is demonstrated by comparing against simulation. The rst example is the network shown in Figure (a). Evolutions of instantaneous average queue size for the two queues are shown in Figure (b) and (c). A second example, dealing with a tandem network with feedback, is shown in Figure. From experimentation we nd that the method works very well for open queueing networks but fails for closed queueing networks. To work well with closed queueing networks, we need the standard normalization constant (for computing B, U and N). Numerical Issues Recall that we model IP networks by networks of M/M//K queues. We are interested in IP networks with large link speeds ( packets/sec), large link buers ( packets), and high utilizations, which translates into large values for K,, and. This makes the system of dierential equations for z i extremely sti, giving rise to problems of numerical stability and

7 8 Evolution of N. Evolution of B 7.. B B... Chapman Kolmogorov. Chapman Kolmogorov Figure : Results for M/M//K queue with = :, = :, K = convergence. Applying regular solving methods to the dierential equations as stated above will produce an incorrect answer or no answer at all, and applying sti equation solvers will produce a correct answer but extremely slowly (for a good coverage of dierential equation solvers, see []). In this section, we describe how to overcome these problems for networks of M/M//K queues. Consider the dierential equation (). Observe that for high K ( ), we have z K for z < and z K for z > +, for some. Using this, equation () becomes j=;j=i dz r ji j min(z j ; )) i ( r ii )z i ] for z i < i j=;j=i r ji j min(z j ; )) i ( r ii )z i ] for z i > + as in equation () otherwise () The formulas for N, B and U are similarly modied. For example, for N we have: 8 > < dt = > : ( z i)[ i + P P n ( ( z i) n [ z i + ( i N i = 8< : zi z i K i + zi z i as in equation () for z i < for z i > + otherwise These modications are not sucient. We need to stop using equation () around z i = :. To do so, we rst nd an for which N, computed using z = and () is equalt to the N computed using z = + and (). That is, () K + + ( + ) = ( ) () which yields = =K. So we make the computation jump over the interval [ ; +] as follows: whenever =K i < z i < would hold, we set z i = + =K i ; whenever < z i < + =K i would hold, we set z i = =K i. Outside this interval, we continue using equation (). This is still not good enough. For large i and i, we get a lot of large uctuations when z i comes close to the value i = i. This is because dz i =dt becomes highly negative (positive) when z i is slightly higher (lower) than i = i. To overcome this problem, we exploit a monotonicity property. 7

8 λ =. K = µ =. r_- =. r_- =. Evolution of N λ =. K = 7 µ =. (a) Evolution of N.. N N. Simulation. Simulation (b) (c) Figure : A simple network and the results obtained for it. Consider the evolution of z i. At any moment t, z i tends to evolve monotonically (increasing or decreasing) from its current value to i = i. So we introduce a \slope" ag which indicates the sign of the slope of z i, i.e., positive when z i is increasing and negative when z i is decreasing. The initial value of the slope ag is determined by the sign of dz i (:)=dt. While = does not change, the ag does not change. At each time step in the numerical solution, if we obtain a negative dz i =dt and the slope ag is positive, we do the following: if i = i has decreased from its value at the previous time step, set the slope ag to negative, otherwise set dz i =dt to. We proceed in a similar way if we obtain a positive dz i =dt and the slope ag is negative. This strategy is also used when modifying z i around :. whenever =K i < z i < would hold, we set z i = + =K i only if the slope ag is plus; whenever < z i < + =K i would hold, we set z i = =K i only if the slope ag is negative. Finally, some implementation issues. We use the standard Runge-Kutta method, of xed order, choosing a proper step size (around ). We take care that step max i z i < maxz holds, so that the step can be smaller when some dz i =dt is large. The modications of the slope ags and the jump over : are handled outside the inner Runge-Kutta loop. The combination of all these techniques yields a very fast and accurate solver (see run times in following section). 8

9 Evolutions of IP Networks This section describes evolutions for two IP networks, one of nodes and one of nodes. We used the Transit-Stub model in the GT-ITM (Georgia Tech Internetwork Topology Models) [] to generate the network topoplogies. The network of nodes is displayed in Figure 7. Figures 8 and 9 show the evolution of the instantaneous queue size at various links. Each gure has two curves, one obtained by the Z-iteration and one by averaging simulation runs. The Z-iteration took 8 seconds of computation time on an Ultra Sparc, whereas each simulation run took minutes for a total of. hours for the simulations. The plots for links -7 and - in the gures clearly show that many more simulations need to be averaged to obtain decent condence intervals, probably around simulations which would require about hours (as opposed to the Z-iteration's 8 seconds). Even though the simulations in the other plots in the gures appear to be of adequate condence, this is a misleading conclusion resulting from the large scaling in these plots. For instance, plots of the blocking probabilities on these links would show variation similar to that in the plots for links -7 and -. The -node network is \shown" in Figure, with certain parts relevant to the evolutions highlighted. The subsequent gures show the evolution of the instantaneous queue size at various links, obtained by the Z-iteration. It required about 8 minutes of computation time. We did not attempt a simulation of this as that would have taken far too long (at least many days for one run). Conclusions and Future Work Queuing systems are a natural way of modeling computer networks and many other systems. One usually obtains steady-state metrics of queueing models, because this is relatively tractable for many kinds of queueing systems. But steady-state metrics do not oer answers to many interesting questions. Transient evolutions, on the other hand, can provide answers to most questions that one would like to ask in evaluating a system. They also allow for realistic modeling of critical events such as network routing updates, unlike steady-state models. Transient metrics are usually very hard to obtain, unmanageable by analytical methods and time-consuming by simulation. But the Z-iteration changes this premise, allowing very fast computation of some very useful transient metrics. It translates a queuing network with N nodes into a system of N coupled dierential equations. Looking at the results and the run time, we conclude that it oers the power of simulation at a fraction of cost for these transient metrics. One area of future work is to extend the results here to multi-class queueing networks. These would be needed for modeling QoS datagram networks (incorporating buer and bandwidth reservation), like the next generation of IP networks. Another area of future work is to develop a performance evaluation procedure. Using the Z-iteration, one can obtain evolutions of ensemble transient metrics. But this by itself does not amount to evaluation. We need to develop a collection of scenarios of adequate coverage for a given network, obtain their evolutions and resulting metrics, interpret the data, and reach conclusions. References [] Ken Calvert Ellen W. Zegura and S. Bhattacharjee. How to Model an Internetwork. In IEEE Infocom '9, San Francisco, CA, 99. [] L. Kleinrock. Queueing Systems, volume I and II. New York: Wiley, 97. 9

10 [] I. Matta and A.U. Shankar. Z-Iteration: A Simple Method for Throughput Estimation in Time-Dependent Multi-Class Systems. In ACM SIGMETRICS/PERFORMANCE '9, pages {, Ottawa, Canada, May 99. [] I. Matta and A.U. Shankar. Fast Time-Dependent Evaluation of Integrated Services Networks. Computer Networks and ISDN System { Special Issue on Modeling of Wired and Wireless ATM, 998. To appear. Preliminary version in Proc. IEEE ICNP '9, 99. [] J. Stoer and R. Bulirsch. Introduction to Numerical Analysis. Springer-Verlag, second edition, 99.

11 r_- =. λ =.8 K = 7 K = K = K = 7 µ =. µ =. µ =. µ =. r_- =. (a). Evolution of N 7 Evolution of N. N. N.. Simulation Simulation 7 (b) Evolution of N. (c) Evolution of N. N N. Simulation. Simulation (d) (e) Figure : A Tandem network with feedback, and its' metrics i

12 Network consists of nodes and links organized as backbone (nodes,,, ) and 8 LANs (attached via access links at nodes,, 7,,,,, ). Buffer capacity for outgoing link queues backbone/access link:, pkts LAN link:, pkts Bandwidth of links backbone/access link:, pkts/sec LAN link:, pkts/sec Workload 7 end-to-end connections, each with poisson traffic of pkts/sec Routing single shortest-hop route for each connection Perturbation event at time sec, the link indicated by arrow fails routes are recomputed, and changes in link queues are plotted. Figure 7: Network : \small" network of nodes.. x The evolution of N at queue > Simulation run. x Simulation runs The evolution of N at queue > 8 The evolution of N at queue > 7 Simulation runs.. N in packets. N in packets. N in packets Figure 8: Network : instantaneous queue size evolution for links -, - and -7. Simulation runs The evolution of N at queue > Simulation runs The evolution of N at queue > Simulation runs The evolution of N at queue > 7 N in packets N packets 8 N packets Figure 9: Network : instantaneous queue size evolution for links -, - and -7. ii

13 Backbone Backbone Backbone Backbone 9 Backbone The links indicated by arrows will fail at time sec. The labeled nodes will have the traffic rate of their outgoing connections modified at the same time. Figure : Network : overview.. The evolution of N at queue > The evolution of N at queue > 7 7 The evolution of N at queue 7 >. N in packets.8. N packets N Figure : Network : instantaneous queue size evolution for links -, -7 and 7-. iii

14 8 8.9 The evolution of N at queue 7 > The evolution of N at queue > 8 The evolution of N at queue > N in packets... N in packets N packets Figure : Network : instantaneous queue size evolution for 7-, - and -.. The evolution of N at queue >. The evolution of N at queue 9 > 89. The evolution of N at queue 89 > N in packets.. N in packets.. N in packets Figure : Network : instantaneous queue size evolution for links -, 9-89 and iv