Teecommunication Network Contro and Management (EE E694) Prof. A. A. Lazar Notes for the ecture of 7/Feb/95 by Huayan Wang (this document was ast LaT E X-ed on May 9,995) Queueing Primer for Muticass Optima Fow Contro In the previous ectures weintroduced the state dependent mode for optima decentraized ow contro in a muti-cass network, expained the concepts used there and outined an approach to sove the probem. We know that the probem has a neat soution which is a set of window poices. In this ecture we wi present a discussion of the appication of queueing theory to this probem. We consider a simpe case invoving two users (casses). We wi mainy cover three topics:. The equiibrium probabiities of a muti-cass M/M/ queue.. Norton's equivaent of the muti-cass system. 3. Soving Nash Equiibrium based on the Norton's equivaent. Equiibrium probabiities Reca that our state dependent ow contro probem is represented by Figure : <C N (FIFO) N < C whycu@ctr.coumbia.edu Figure : Mode for state dependent ow contro
EE E694: Lecture 4 The criterion used is : max E k T k Ek for user k, k. () We have to nd out the optima soution, which is the Nash equiibrium (see ecture 3 for the formuation of the probem). We focus on the midde queue of Figure. Note that there are two ows of trace feed into a botteneck queue, we ca it a muti-cass M/M/ queue. (FIFO) state X(x, x, x...) 3 Figure : The queueing system we have to contro Athough at rst ook it is quite simiar to an M/M/ queue, there is a signicant dierence which comes about because of packet dependent routing, i.e., the server has to distinguish between packets in order to decide where to route them. So the state of the muti-cass M/M/ queue is no onger simpy the numberofpackets in the system. In order to describe the state of a FCFS (rst come rst serve) queue with muti casses of trac it is necessary to specify the cass of the packets at each position. So the state of the queueing system of Figure is given by X (x x ::: x N ), where x i is the cass of the packet in position i and N is the tota number of packets in the queue. In our case here, we have ony two casses, then x i equas either or. We can think the state as a string. For exampe, one outcome of X ( :::) means the rst packet in the queueing system is cass packet, the second is cass, third cass, and so on. It is worthwhie noting that the string ength is variabe, and depends on how many packets are present inthe queueing system. The ength may gotoinnityifn is aowed to be innite. So soving these states expicity is very compicated. If we draw the state transition diagram, it is a two dimensiona Markov chain (see Figure 3). Note that in the state string, the eftmost one is going to depart, and an arriva goes to the rightmost position, then it is easy to write down the corresponding transition rates in Figure 3. Note that the state transition diagram often dispays singe transitions (bidirectiona transitions on the boundaries ony) and we certainy cannot expect the
EE E694: Lecture 4 3,,,,,,,,,,,,,,,,,,,, Figure 3: State transition diagram (fu information case) Markov chain to be reversibe. Let us ook at a pain M/M/ queue for a moment. Its state diagram has bidirectiona transitions between each pair of adjacent states. This Markov chain is reversibe and oca baance equations hod, which means rather than soving the goba baance equations for each state, we can draw vertica boundaries between each pair of adjacent states and equate the ow from eft to right across such a boundary to the ow from right to eft, which eads to a set of oca baance equations. Then it is very easy to get the product form soution of the M/M/ queue p n () n p, where p can be soved by the normaization equation. i- i i+ j- j j+ Goba baance boundary around a state Loca baance boundaries between states Figure 4: Baance boundaries for M/M/ queue Coming back to the state transition diagram of our probem, we cannot think of a vertica boundary between two states since there is ony one directiona ow to/from some states. And the diagram is so compicated (the number of states increases exponentiay with the number of packets) that it is dicut to sove
EE E694: Lecture 4 4 the goba baance equations. Note that if the queue is innitey arge, i.e., has unimited capacity, the goba baance equations are innite in number! To approach this kind of a probem (which arises very often in the network), a very usefu way is to PLUG in a GOOD GUESS. Quite naturay a good guess here is a product form soution. One way to obtain the product form soution is based on a method caed agebraic topoogica interpretation (ATI) of the product form soution (pease see reference [6] for more detai). This method expains why certain modes have a product form soution and why others do not, more importanty, itgives a constructive means for arriving at the product form soution. The basic idea is the reaization that the state transition diagrams of product form networks can be decomposed into an aggregation of eementary \buiding bocks" which do not interact with each other in a substantia way, andthuscanbesoved in isoation for the reative state probabiities. The product form soution, then, for a specic equiibrium state probabiity in terms of a reference probabiity is reay a recursion that patches together these buiding bock soutions aong a \path" from the state of interest back to the reference state which usuay is the state. To isoate the buiding bock and to construct the soution aong a path to a reference state is the essence of the product form soution. We wiook at this idea cosey using our probem as an exampe. Buiding bocks have a very important propertywhich is that they can be embedded into the overa state transition diagram without aecting the ow pattern of the rest of the diagram, except for a renormaization. It can be shown that a cycic ow, which is an isoated circuation in the form of a singe cyce is one kind of buiding bock. For exampe, consider the subset of the state transition diagram which consists of states f, (,),, (,)g, the state transition sequence! ( )!! ( )! is a cycic ow of ength four. By the property of buiding bocks, the conservation of this ow ateach adjacent state sti hods even when the edges are embedded into the overa state transition diagram. Thus the reative equiibrium probabiities for these states can be soved in isoation without considering the rest of the state transition diagram. Look at the rst two stages of our state transition diagram, we can separate them into six buiding bocks. Denote X as the equiibrium probabiity of state X, we can easiy write down the baance equations across the boundaries (see Figure 5). Then it is easy to sove the oca baance equations to get the equiibrium probabiities: :
EE E694: Lecture 4 5 (3), () π π () π π (5) (6) (3) π π (), (4) π π (), (7) (8) (5) π π (6) π π (7) π π Loca baance boundaries (4), (8) π π corresponding oca baance equations Figure 5: Buiding bocks for the state transition diagram By inspection, we can see that for a specic state, the product form soution coecient is just the product of the 's on the way towards the state from that specic state, over the product of corresponding numbers of 's. In the third stage, we can see there are two cycic ows of ength two (on the boundaries) and another two of ength six. For exampe f( )! ( )! ( )! ( )! ( )! ( )! ( )g is a cycic ow of ength six. It is not dicut to work out the equiibrium probabiities for that stage: : We can appy this method iterativey to sove a the states. One more thing we need to point out is that any pathbetween two states woud have produced the same resuts. It is not dicut to verify by an exampe (readers are encouraged to do it as an exercise). The idea is simiar to the concept of potentias in the circuit theory: Given a reference point, the potentia of each point on the circuit is xed no matter which oopyou choose to cacuate them. Now we are happy that our system has a nice product form soution. We can write down a the probabiities by inspection of the state diagram. The
EE E694: Lecture 4 6 average throughput, the average time deay, and other quantities of interest that characterize the queueing system of Figure can be computed in terms of these probabiities. But ook at the notation of our soution, we have to specify a the 's in the coecient since their super- and sub- scripts are a dierent! In the next section we wi show that for the purpose of computing averages needed in our optimization probem, it is sucient to consider the Norton's equivaent of this system. This approach reduces the notationa compexity of the equations invoved and masks the unnecessary detais and iuminates the structure of the soution to our probem. Norton's equivaent Simpy speaking, Norton theorem states that if we are interested in the endto-end properties (time deay and throughput) between two points S and D, we can repace the queueing network by a singe state dependent queue with no change in the statistic behavior between points S and D. Norton's equivaent (sometimes caed aggregation method) can provide an extremey vauabe simpication in most cases. The queueing mode described above provides a detaied state description. But as far as our controer concerned, since it is more reaistic to assume knowing how many packets are in the system (outstanding) rather than knowing the exact position of each packets in the system. As we mentioned in ecture 3, there is a trade o between information reduction and contro quaity. Based on this observation, we ook for a Norton's equivaent of the origina muti-cass queueing system by aggregating the appropriate states. We dene a set of states S( )by S( )fxj number of cass k packets k k g: Now we get a more compact state transition diagram: Let us work out the equiibrium probabiities and transition rates here. Denote p as the equiibrium probabiity ofstates( ), then Obviousy we have p X xs( ) x : p p p p p p + p p p :
EE E694: Lecture 4 7, /3 /,,, / /3, /3 /,, /, /3, Figure 6: State transition diagram after aggregation So the transition rate from S(,) to S(,) (or S(,)) is /, which makes sense because at state S(,), there are two packets in the system, the chance of each one of them getting served is one haf. Simiary, consider p + + 3 3 p 3 p 3 p the transition rate from S(,) to S(,) is /3, whie from S(,) to S(,) is /3. By iteration, we can show that where p is given by: p + Y ; ( ; i ) ( j )p () XN XN i p : It is not dicut to see the intuition behind the equation where the binomia coecient comes because there are so many ways from state S( )to reach reference state, and the products specify the path. Note that the state probabiities of S( ) are aso of product form. In the end the state transition diagram after aggregation has transitions for cass and arrivas respectivey, and ( +), ( + ) for departures respectivey (see Figure 6). Observe the state transition diagram after aggregation, there are bidirectiona transitions between each pair of adjacent states, we can write the oca Y j
EE E694: Lecture 4 8 baance equations directy from the diagram : p + + + p + (3) p + + + p + : (4) Note that we start with a network (Figure 7), and we get its Norton's equivaent (Figure 8). (FIFO) state X(x, x, x...) 3 Figure 7: The origina queueing system ν + ν + state (, ) Figure 8: Norton's equivaent for the origina queueing system It is important to note that because of the existence of product form soution of the origina network, its Norton's equivaent aso has a product form soution! Athough we soved the Norton's equivaent expicity in this case, we woud ike introduce the mathematica structure of the aggregation procedure and provide a probabiistic interpretation for Norton's equivaent since it is used so often in the communication network that it is important to understand it very we. Observing the aggregation from Figure 7 to Figure 8, actuay there is an estimation procedure invoved here: Estimate the rate at which the server processes cass or packets in state ( ) (or equivaenty to say knowing and ), which is exacty the idea of Norton's equivaent in the set up of muti-cass queueing network. [] showed that the Norton's equivaent of muti-cass queueing network can be interpretated as a conditiona estimate. In our exampe here, assume that ( )isknown, dene k E[ k There are a tota of and cass and packets in the system respectivey ] (5)
EE E694: Lecture 4 9 where k is the instantaneous departure rate of cass k packets, k (x k) k + ( + > ): (6) where () is the indicator function. This is the actua service rate seen by the cass k packets. For exampe, for M/M/ queue the actua service rate is (n >), where n is the state of M/M/ queue. Pug into equation 5, we get k E[ k j ] (7) k E[ ( + > ) j ] (8) + k + ( + > ) (9) which is exacty what we worked out before. Notice that the service rate in the equivaentnetworkisnow cass dependent, which is sort of counter-intuitive, but it exacty ts the idea of the packet dependent routing we mentioned before. Now we wi show that in terms of computing average network throughput and time deay, using Norton's equivaent wi yied the same resut. E XN XN XN XN ( + )p () ( + > )( + )p + + () XN XN XN XN ( + > ) p () (x >) X xs( ) x (3) X a x (x >) x : (4) The ast expression is exacty the average throughput of the system in terms of the fu information states. We can work out the average deay in the same manner. Up to now whatwe end up with is Figure 8. But the rea situation is that user k ony has the partia information k,not( ). So the next step is to nd the service rate of the system as far as user k is concerned. Without oss of generaity, et us consider user here. Use the same technique as above, dene ^ E[ jthere are tota of cass packets in the system]: (5)
EE E694: Lecture 4 Then we get an equivaent system based on oca information (decentraization): Figure 9: Decentraized equivaent mode of the origina system In summary,we reduce our probem by atwo step procedure: state aggregation X (, ) information reduction knowing ess decentraization fu information what controer knows Figure : Two step procedure of information reduction Notice that we aggregate a the information into the service rate ^,which is certainy a function of since it is the interference. It can be proved that ^ does not depend on (pease see [4] for the proof), so we can appy our contro poicy based on the estimation of ^.Intuitivey it shoud be true since from the point of view of user, sending Poisson packets into the system shoud not aect the response of the system according to the Markovian property of the arrivas. And practicay speaking, it is a reasonabe approximation of the system. In the rea network we can estimate the service rate by observing the number of outstanding packets and the acknowedgements received. Now we wi show that in terms of computing average network throughput and time deay, using Norton's equivaent (Figure 9) wi get the same resut. Denote the margina probabiity P P as the probabiity that there are cass N packets in the system. Then p p. As far as user is concerned, E XN ^ p (6) XN E[ j ]p (7) XN XN p p p (8)
EE E694: Lecture 4 XN XN p : (9) The ast expression is exacty the average throughput for user if we use the rst step Norton's equivaent (see Figure 8). Simiary we can work out the average time deay. So we showed that the mode of Figure 9 is indeed equivaent to the origina system in terms of statistic properties (average throughput and time deay). 3 Nash equiibrium We showed that the origina probem Figure is equivaent to considering the two separate queues, each of the form: κ κ <C k k k < N k Figure : Decentraized equivaent mode of the origina system We want to optimize the ow contro under the criterion: max E k T k Ek for user k, k,. Remember in ecture, we aready soved this probem in the M/M/ case. But note in this case the service rate is no onger a constant, here^ k is state dependent. Fortunatey it can be proved by inear programming that the optima ow contro poicy is sti window type. For user k, the optima ow contro is of the form: k k 8 < : c k k m k <L k k L k k >L k for some L k <N k, k m ck, k (See ecture 3). Nash equiibrium is a pair of windows! It is a very neat resut: whatever other users do, user k aways uses window poicy, but the window size depends on the network oad. Now we soved our probem. But there are sti ots of question marks eft. Athough it can be shown that Nash equiibrium is unique respect to one window poicy (if a other users are frozen), but dynamicy, the soution is not unique. Each user has to tune its own window size according to other users. So can we
EE E694: Lecture 4 compute the Nash equiibrium in this case? How canwe appy it in the rea networks? Either we have tohave a good estimator to estimate the rate, or we can write some agorithms to try to nd the optima soution. Peope did ots of work on it and there do exist some decentraized agorithms. But there is sti many probems associated with the agorithm: Is it convergent? Is it convergent to exacty Nash equiibrium? How to dea with od data due to deay... We wi tak about these in more detai in the next ecture. References [] M.-T. T. Hsiao and A. A. Lazar, \Botteneck Modeing and Decentraized Optima Fow Contro II. Individua Objectives," in Proceedings of the 9 th Conference on Information Sciences and Systems, (Batimore, MD), Mar. 985. [] M.-T. T. Hsiao and A. A. Lazar, \An Extension to Norton's Equivaent," vo. 5, pp. 4{4, 989. [3] M.-T. T. Hsiao and A. A. Lazar, \Optima Fow Contro of Muticass Queueing Networks with Partia Information," IEEE Trans. Autom. Contro, vo. 35, pp. 855{86, Juy 99. [4] M.-T. T. Hsiao and A. A. Lazar, \Optima Decentraized Fow Contro of Markovian Queueing Networks with Mutipe Controers," Performance Evauation, vo. 3, no. 3, pp. 8{4, 99. [5] Y. A. Koriis and A. A. Lazar, \On the Existence of Equiibria in Noncooperative Optima Fow Contro," in Proceedings of the ITC Sponsored Seminar, (Bangaore, India), pp. 43{5, Indian Institute of Science, November 993. [6] T. G. Robertazzi, Computer Networks and Systems { Queueing Theory and Performance Evauation. NewYork: Springer-Verag, 994.