A Queueing Model for Call Blending in Call Centers

Transcription

1 434 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 48, NO. 8, AUGUST 2003 A Queueing Model for Call Blending in Call Center Sandjai Bhulai and Ger Koole Abtrat Call enter that apply all blending obtain high-produtivity and high-ervie level by dynamially mixing inbound and outbound traffi. We how that agent hould be aigned to outbound all if the number of available agent exeed a ertain threhold. Thi ontrol poliy i optimal for equal ervie time ditribution and a very good approximation otherwie. Index Term Call blending, all enter, queueing model, threhold poliie. I. INTRODUCTION In thi note, we onider a queueing ytem with two type of job. The firt type of job ha a ontraint on the performane, i.e., the average waiting time ha to be below a ertain level. Next to thi time-ontrained type there i a eond type of job, available in an infinite quantity, for whih the objetive i to erve a many a poible. The arrival of the firt job type are determined by a Poion proe and the ervie time of both job type are independent exponentially ditributed. Both job type are erved by a ommon pool of erver under a nonpreemptive diipline. The quetion that we will anwer in thi note i how to hedule thee erver to maximize the throughput of type 2 job while atifying the waiting time ontraint on the type job. Sheduling a type job and delaying a type 2 job doe not hange the throughput, ine we fou on the long-term throughput. Therefore the quetion i, when a erver beome idle and there are no type job in the queue, whether thi erver hould tart erving a type 2 job or wait for a type job to arrive. The optimal deiion will be a funtion of the tate of the other erver. A typial appliation of our model i all blending in all enter. Modern all enter deal with a mixture of inoming and outgoing all. Of oure, there are ontraint on the waiting time of inoming all. The traditional olution i to aign all enter employee (often alled agent ) to either inoming or outgoing all. However, the all rate flutuate over the day and in order to handle the all during peak period uually a ubtantial number of agent need to be aigned to inoming all. Conequently, the produtivity of the agent, i.e., the fration of time that agent are buy, i low during other period. On the other hand, aigning fewer agent to inoming all inreae the produtivity, but lead to longer waiting time. Hene, there i a need to balane produtivity and waiting time. The olution i all blending, dynamially aigning agent either to inoming or outgoing traffi. The model that we tudy in thi note repreent exatly thi ituation there are agent, the time ontrained inoming all are modeled a type utomer, and the type 2 all repreent the baklog of outgoing all. The objetive i to obtain a imple model that provide inight into the ytem behavior. Moreover, the aim i to derive imple heduling poliie that an be implemented in all enter oftware. Manuript reeived January 3, 2002; revied July 9, Reommended by Aoiate Editor A. Giua. S. Bhulai i with the Bell Laboratorie, Luent Tehnologie, Murray Hill, NJ USA, and alo with the Faulty of Siene, Vrije Univeriteit Amterdam, 08 HV Amterdam, The Netherland ( bhulai@.vu.nl). G. Koole i with the Faulty of Siene, Vrije Univeriteit Amterdam, 08 HV Amterdam, The Netherland ( koole@.vu.nl). Digital Objet Identifier 0.09/TAC The ontribution of thi note i twofold. We propoe heduling poliie that keep part of the ervie apaity free for arriving time-ontrained type job. Thi poliy mixe the traffi from the two hannel uh that both the waiting time ontraint for type job i met, and the throughput of type 2 job i maximized. In ontrat to many other queueing model where idling i not optimal (ee [9] for a reent urvey), we how that thi poliy i optimal for equal ervie time ditribution and a very good approximation otherwie. The eond ontribution follow from the pratial relevane of all blending in all enter. Call blending an ignifiantly improve all enter performane in many ompanie, ompared to the traditional eparation of employee in group aigned to either inoming or outgoing traffi. However, all blending ha not reeived muh attention yet (ee [6] for a reent urvey on queueing model for all enter). In thi note, we preent a mathematial model for all blending and olve it. A the reulting poliy i eaily implemented, it ha the potential to be ued in workfore management oftware for all enter. The organization of thi note i a follow. In Setion II, we give the exat model formulation. In Setion III, we analyze the ae of equal ervie requirement. In Setion IV, we analyze the ae of different ervie requirement. II. MODEL AND FIRST RESULTS The exat model formulation i a follow. There are two type of traffi, type and type 2, having independent exponentially ditributed ervie requirement with rate and 2. Type job arrive aording to a Poion proe with rate, and there i an infinite waiting queue for job that annot be erved yet. There i an infinite upply of type 2 job. There are a total of idential erver. The long-term average waiting time of the type job hould be below a ontant. Waiting exlude the ervie time; if the repone time i to be onidered, then the average ervie time, =, hould be added to the average waiting time. The objetive for type 2 job i to maximize it throughput, i.e., to erve on average per unit of time a many type 2 job a poible, of oure at the ame time obeying the ontraint on the type waiting time. The following ontrol ation are poible. The moment a erver finihe ervie, or, more generally at any moment that a erver i idle, it an take one of the following three ation tart erving a type job (if one or more are waiting in the queue for ervie), tart erving a type 2 job, or remain idle. Note that in our model preemption of job in ervie i not allowed. When preemption i allowed the problem i trivial. The optimal poliy will aign all erver to type 2 job when no type job are preent in the ytem. When a type job arrive then it i learly optimal to interrupt the ervie of a type 2 job and to erve the type job. Hene, the waiting time ontraint i atified and the type 2 throughput i equal to 2 ( 0 =). Note that any work-onerving poliy that atifie the waiting time ontraint i optimal and ahieve the ame throughput. In pratie the job that an be preempted in all enter are meage. Therefore, it i benefiial for all enter to enourage utomer to end their requet by . Thi finihe the deription of our model. In the next two etion, we will deal with the ae = 2 and 6= 2, repetively. A firt quetion that ha to be anwered i whether it i at all poible to find a poliy that atifie the waiting time ontraint of the type traffi /03$ IEEE

2 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 48, NO. 8, AUGUST Lemma Let = =, and W q denote the tationary waiting time of type job. The ontraint W q i atifiable when ;, and are uh that W q = ( 0 )!( 0 ) 2 2 x x=0 x! ( 0 )!( 0 ) Proof It i lear that the waiting time for type job i minimized if we hedule no type 2 job at all, thu reduing the model to a tandard M/M/ queue. Note that the tability ondition = <i not uffiient. Hene, W q hould be alulated firt, whih i given by [3, Ex. 4.27] yielding the expreion in the lemma. From now on, we aume that ;, and are uh that W q. Cheking whether thi i the ae an be eaily done uing the aforementioned formula. While formulating the model we tated that an idle erver an hedule a type 2 or a type job (when available) at any moment. Due to the fat that we are onidering long-term average performane it i only optimal to hedule job at ompletion or arrival intant. Indeed, if it i optimal to keep a erver idle at a ertain intant, then thi remain optimal until the next event in the ytem. Thi follow diretly from the ontinuou-time Bellman equation (ee [8, Ch. ]). Therefore, it uffie to onider the ytem only at ompletion or arrival intant. Beaue of thi, and beaue of the fat that the maximum total rate i uniformly bounded by maxf ; 2g,we an ue the well-known uniformization tehnique (ee [8, Se..5]). Thi allow u to ue direte-time dynami programming to ompute performane meaure and to find the optimal poliy. However, our ytem i not a tandard Markov deiion proe (MDP), beaue of the different objetive for queue and queue 2. The form of the problem make it a ontrained MDP; maximize the type 2 throughput with a ontraint on the type waiting time. Contrained MDP an be olved uing variou tehnique. Here, we ue one that introdue the ontraint in the objetive uing a Lagrange multiplier. Under weak ondition, it an be een that the optimal poliy for a ertain Lagrange multiplier i optimal for the ontrained problem if the value of the ontraint under thi poliy attain exatly. From the theory on ontrained MDP it follow that thi poliy i tationary and randomize in at mot tate. For thi and other reult on ontrained MDP, ee []. III. EQUAL SERVICE REQUIREMENTS Let = = 2. Conider the event that a erver beome idle, and that there are one or more type job waiting. Then the ontroller ha to hooe between heduling a type or a type 2 job (or idling, but thi i evidently uboptimal). Giving priority to a type 2 job and delaying type job obviouly lead to higher waiting time. Delaying the proeing of a type 2 job doe not hange the performane for thi la, a we are intereted in the long-term throughput. Thi intuitive argument implie that, when a erver beome idle and a type job i waiting, it i optimal to aign thi type job to the erver. The following oupling argument how that thi i indeed true. Theorem 2 Suppoe that a erver beome idle while there are type job waiting in the queue. Then, the ation that hedule a type job i among the et of optimal ation. Proof Let be an arbitrary poliy whih repet the waiting time ontraint on the type job. Suppoe that there i a time intant, ay t, where a type 2 job i heduled, given that there i a type job waiting in the queue. Sine repet the waiting time ontraint on type job there will be a later time intant, ay t 2 where thi type job will be heduled. Now, onider the poliy 0 whih follow all ation of exept that it hedule a type job at t and a type 2 job at t 2. Note that thi interhange doe not hange the deiion epoh for 0, ine the ervie requirement i for both job type. The total number of type 2 utomer erved after t 2 i equal under both polie, thu alo the throughput. However, the average waiting time under poliy 0 i the ame or lower than under, ine the type job i erved earlier. Hene, the reult follow. We model the ytem a a (ontrained) Markov deiion proe. Thi onit of a deription of the tate pae, the poible ation, the tranition probabilitie, and the reward truture. Sine both type of job have equal ervie requirement, we need not ditinguih between type or type 2 job in ervie. Therefore, the tate of the ytem i ompletely deribed by the number of job in ervie plu the number of type job in the queue. Thu, the tate pae i X = 0.By Theorem 2, there annot be le than job in ervie while there are type job waiting in the queue. Hene, one an only take an ation in tate x 2Xif x<; otherwie a type job i automatially heduled. The poible ation are denoted by a =0;...; 0 x, orreponding to heduling a type 2 job. We denote the tranition rate of going from x to y (before taking any ation) by p(x; y). Then we have p(x; x 0 ) = minfx; g and p(x; x )=. After uh an event an ation an be hoen, if the new tate i below. If ation a i hoen in x (with a 0 x), then the ytem move to x a. Next, we uniformize the ytem (ee [8, Se..5]). For impliity, we aume that. (We an alway get thi by aling.) Uniformizing i equivalent to adding dummy tranition (from a tate to itelf) uh that the rate out of eah tate i equal to ; then we an onider the rate to be tranition probabilitie. The objetive are modeled a follow. If ation a i hoen, then a reward of a i reeived, for eah type 2 job that enter ervie; thi model the throughput. Due to the Poion arrival and uniformization, the average waiting time i obtained by taking the ot rate equal to the expeted waiting time of an arriving utomer. Thu, to obtain the average waiting ot W q, we an take the ot rate equal to [x 0 ] =, where [x] = maxf0;xg. Note that the ot rate in thi ae are equivalent to lump ot at eah epoh. The two objetive are merged into a ingle reward funtion uing a Lagrange parameter 2 IR (ee []). In order to tudy trutural propertie of optimal poliie, we define the dynami programming operator T for funtion f X! IR a follow Tf(x) = [x 0 ] y2fx;x;xg p(x; y) max a2f0g[f;...;0yg fa f (y a)g with the onvention that f;...;0 yg = ; if 0 y 0. Oberve how the waiting time ontraint i merged with the throughput by the Lagrange parameter. Alo, note the plae of the maximization here the ation depend on the tate hange that ourred. (The dynami programming operator an eaily be rewritten in the tandard formulation with a ingle overall maximization; ee [4, Ch. 5]), but thi would oniderably ompliate the notation). The long-term average optimal ation are a olution of the optimality equation (in vetor notation) h g = Th. Another way of obtaining them i through value iteration, by reurively defining V n = TV n, for arbitrary V 0.Forn!, the maximizing ation onverge to the optimal one. (For exitene and onvergene of olution and optimal poliie, we refer to [8]). We will next how that the optimal poliy i of threhold type. Theorem 3 There i a level, alled the threhold, uh that if x<, then the optimal ation i 0x.Ifx, then 0 i the optimal ation.

3 436 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 48, NO. 8, AUGUST 2003 Proof We firt derive ertain monotoniity propertie of the value funtion V n with V 0 = 0. Thi i eaier if we rewrite T a follow Tf(x) = [x 0 ] y2fx;x;xg p(x; y)u () f (y) with U () the -fold onvolution of the operator U, and U defined a maxff(x); f (x )g; if x< Uf(x) = f (x); otherwie It i obviou that both form are equivalent heduling a group of type 2 job at the ame time i equivalent to heduling them one by one. We ontinue with the fat that we are only intereted in 0. For = 0the ituation i a if the waiting time ontraint poe no real ontraint; for <0it pay to have a poliy that trie to dereae the type waiting time. For 0, the diret reward [x 0 ] = are dereaing onave (dv) a a funtion of x. The funtion V 0 =0 i alo dv. It i traightforward to how that V n i dv for all n (thi i equivalent to the reult derived in [5, Se. 3]). When applying U to f, we ee that it i optimal to hedule a type 2 job if and only if f (x) 0 f (x ).Iff i dv, then f (x) 0 f (x )inreae in x. Thu for eah f there i a threhold level at and above whih it i not optimal to hedule type 2 job, and below whih it i. Beaue the U operator i repeated time after eah event the threhold level will be reahed. (In fat, there will never be heduled more than one type 2 job, a the jump ize i maximal ). We need to tre that it an be the ae that it i both optimal to hedule a type 2 job and not to hedule one. Thi our if f (x) = f (x). In thi ae, two threhold poliie, with threhold level and, are both optimal, and all the poliie that randomize between thee two poliie. We will ee that, in general, to find a threhold poliy that atifie W q =, we need to randomize. Next, we alulate for a fixed threhold poliy that randomize between and it tationary type waiting time W q and it type 2 throughput. We aume that the poliy i uh that if a tranition from to our then with probability 0 a type 2 job i immediately heduled. Thi reult in a tranition rate p( ;) from to of p( ;) =( ). The lowet poible tate i, a the tate move immediately up to a oon a 0 i reahed. The other poitive tranition rate are p(x; x ) = for all x and p(x; x 0 ) = minf; xg for all x >. Thi reult in a birth death proe from whih the average waiting time and the throughput an be omputed a follow. Theorem 4 Let = =. Then the average waiting time a a funtion of the threhold and the randomization parameter i given by W q (;) = 0! ( 0 )!( 0 ) 2 q with q x0! 0! = x! ( 0 )!( 0 ) x= The throughput of type 2 job i given by (;) = q x=maxf;g x0! (x 0 )! 0! ( 0 )!( 0 ) 0 Proof Let u alulate the tationary probabilitie, whih are denoted by q x, for the birth death proe. It i readily een that (with = =) and q x = x0! q for all <x x! q x = x0!! x0 q for all x> Fig.. = 2 = 3, and =5. with x0! 0! q = x! ( 0 )!( 0 ) x= We define the probability of delay for thi (; ) poliy by 0! C (;) (; ) = q x = ( 0 )!( 0 ) q x= The waiting time under the threhold poliy i alo ompletely equivalent to the one without type 2 job W q (;) = C (;)(; ) ( 0 ) Thi i the formula for the type waiting time. Next we derive an expreion for the type 2 throughput, whih we will denote by. The throughput of type 2 i the total throughput minu the type throughput. Therefore (;) = minfx; gq x 0 x= = q x=maxf;g x0! (x 0 )! 0! ( 0 )!( 0 ) 0 The expreion maxf; g in the ummation limit i there to prevent ()! from appearing in ae =0. Fig. diplay the behavior of the waiting time of type job (line) and the throughput of type 2 job (dotted line) when varie. The ued value of the parameter are ==2; ==3, and =5. The figure hould be interpreted a follow to a given waiting time guarantee to type job, one an read the optimal threhold value in the figure. Next, one an read the throughput of type 2 job aoiated with under the poliy uing the threhold. It i intereting to note that the average waiting time inreae lowly, while the throughput inreae nearly linearly. In the all enter framework, one ould linearly inreae the produtivity of the agent, while very little additional waiting our. Numerial experiment indiate that the threhold poliy perform very well in ae of nonexponential ervie time. Hene, it eem that the threhold poliy i very robut. IV. UNEQUAL SERVICE REQUIREMENTS When 6= 2 the analyi i more ompliated. In thi ae we have to differentiate between type utomer and type 2 utomer.

4 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 48, NO. 8, AUGUST The optimal poliy will depend on thee different lae and an be very ompliated. From a pratial viewpoint, thee poliie an alo be diffiult to implement in all enter oftware. Therefore, we prefer to tudy impler poliie and we retrit ourelve to the la of threhold poliie. Thi hoie i partially upported by the intuitive reaoning behind Theorem 2. Numerial experiment indiate that thi theorem alo hold in ae of unequal ervie requirement. The retrition to the la of threhold poliie fore the poliie to be imple and appealing. Moreover, we will how by numerial omputation that threhold poliie are a good approximation to the optimal poliy. Let x denote the number of type job in ervie and in the queue, and let y denote the number of type 2 job in ervie. Let the threhold be fixed, and let u for the moment aume that we do not randomize in tate x y =. Then the tationary probabilitie q x;y are determined by the following et of equilibrium equation. For y = we have (( 0 ) 2 )q x; = q x; ( 0 ) q x; x 0 () (x 2 )q x; = q x; (x ) q x; 0 <x<0 (2) q 0; = q ; q ; (3) For 0 <y<, the et of equation beome (( 0 y) y 2 )q x;y = q x;y ( 0 y) q x;y (y ) 2 q x;y x 0 y (4) (x y 2 )q x;y = q x;y (x ) q x;y (y ) 2 q x;y 0 y<x<0 y (5) (( 0 y) )q 0y;y =(y ) 2 q 0y;y (0 y ) q 0y;y (0 y ) q 0y;y (6) Finally, for y =0,wehave ( )q x;0 = q x;0 q x;0 2 q x; x (7) (x )q x;0 = q x;0 (x ) q x;0 2q x; <x< (8) ( )q ;0 = 2q ; ( ) q ;0 (9) Thi infinite et of equilibrium equation an be numerially olved by uing the matrix-geometri approah developed by Neut [7] or by the petral expanion method a deribed by Chakka and Mitrani [2]. However, note that in the equation there i no flow from q x;y toward level y or higher when x>0y, ine job of type are given priority over type 2 job. Due to thi peial truture of the equation, we an olve thi part of the ytem analytially uing tandard reult from the theory of linear differene equation. The equilibrium equation are olved by olving the equation for y = and afterward working the way down from y = 0 to y =0. After olving thee equation, we obtain a finite et of equation till to be olved. However, thee equation an eaily be omputed numerially by applying the reurrene relation on the obtained olution. Theorem 5 The olution to (), (4), and (7) i given by q x;y = i=y K y;i0yz x0(0i) i 0 y ; x 0 y (0) The ontant z i for i = 0;...; are the olution to a homogeneou differene equation and are given by the firt equation hown at the bottom of the page. The ontant K y;i are given by the reurrene relation K y;i = (y ) 2 i( 2 ) 0 i z yi K y;i 0 y<; i 0 y () Proof Fou the attention on a partiular row y with 0 y. Conider the orreponding homogeneou equation aoiated with y and x>0y; thu, we are onidering the homogeneou part of (), (4), or (7). It olution i given by q x;y = q 0y;yz x0(0y) y, where z y i the root of the polynomial 0 (( 0 y) y 2 )z ( 0 y) z 2 By traightforward omputation, one how that one of the two root i larger than and, therefore, not ueful. The other root, whih i poitive and le than one, i given by the eond equation hown at the bottom of the page. Note that the root z 0;...;z are different from eah other. Now, we will how that (0) and () atify the differene equation. The ae y = ha already been onidered during the derivation of z, ine () i a homogeneou differene equation. Now, let 0 y< and onider (4) and (7) given by q x;y 0 (( 0 y) y 2)q x;y ( 0 y) q x;y (y ) 2 q x;y Subtitute expreion (0) into thi equation. Fix i 0 y and look at all fator z yi. Thi yield the following expreion K y;i z x0(0y0i) [ 0 (( 0 y) yi y 2 )z yi ( 0 y) z 2 yi Now ubtrat the following quantity from thi K y;iz x0(0y0i) yi (y ) 2 K y;iz x0(0y0i) yi 0 (( 0 y 0 i) (y i) 2 )z yi (0 y 0 i) z 2 yi z i = (( 0 i) i2) 0 (( 0 i) i2)2 0 4( 0 i) 2( 0 i) z y = (( 0 y) y2) 0 (( 0 y) y2)2 0 4( 0 y) 2( 0 y)

5 438 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 48, NO. 8, AUGUST 2003 Thi quantity i equal to zero, ine the term between the braket i the olution to the homogeneou differene equation for level y i. The reult of the ubtration i given by K y;i z x0(0y0i) 0i( yi 2 )z yi i z 2 yi whih i equal to (y ) 2K y;iz x0(0y0i) yi z x0(0y0i) yi [0(i( 2 ) 0 i z yi )K y;i (y ) 2K y;i] Subtituting () into thi equation yield zero, howing that (0) and () indeed are the olution to (), (4), and (7). At thi point (), (4), and (7) have been olved up to K 0;0;...;K ;0. We till have a finite number of equation to olve, whih an be done numerially a follow. Uing relation (2), (5), and (8), one an ompute the value of q x;y for 0 y with 0 y<x< 0 y expreed in the unknown ontant. Then the boundary ondition (3), (6), and (9) relate the ontant to eah other leaving only one ontant to be determined. Thi ontant i finally determined by the ondition q x;y x;y =. Now the omplete ytem i determined. One the probabilitie q x;y are known, the performane harateriti of Setion III for the two type of job an be eaily omputed. The probability of delay i given by C () () = y=0 x=0y q x;y = y=0 i=y K y;i0y 0 z i z i0y i Reall that [x] =maxf0;xg, and define W (x; y) reurively by W (x; y) = ( 0 y) y 2 [ ( 0 y) W (x 0 ;y) y 2W (x; [y 0 ] )] for x y, and 0 otherwie. Note that W (x; y) =(x y 0 ) = when = = 2. The waiting time of type job i given by W q () = x;y W (x; y)q x;y Finally, the throughput of type 2 job i given by () = 2 x;y yq x;y Note that the previou reult are for the ae with no randomization. When randomizing in tate xy =, the previou reult till hold with ome minor hange. Randomization alter the balane equation for the tate x y = and x y =. Sine thi doe not affet (), (4), and (7), Theorem 5 i till valid where the ontant K now depend on, the randomization probability. The finite et of equation an now be olved reurively a deribed before. Hene, the waiting time of type job and the throughput of type 2 job will alo depend on. Fig. 2 illutrate the behavior of the waiting time of type job (line) and the throughput of type 2 job (dotted line) when 6= 2. The hoen parameter in thi ae are ==2; =4=0; 2 =3=0, and =5. Again, we ee that the average waiting time inreae lowly, while the throughput inreae nearly linearly. Sine the threhold poliy doe not need to be the optimal poliy, it i intereting to numerially ompute the performane of the optimal poliy. Thi an be done by uing the dynami programming operator for a fixed value of the Lagrange parameter yielding a poliy. The waiting time of type job i obtained by iterating the dynami programming operator following poliy without the reward rate for heduling type 2 job. Similarly, the throughput of type 2 job i obtained by iterating the dynami program- Fig. 2. = 2 =4 0 =3 0, and =5. ming operator following poliy without the ot rate for waiting. For the fixed value of the Lagrange parameter, we thu obtain the optimal waiting time with the orreponding throughput (for reult on optimality, ee []). The graph of the optimal poliy i generated by omputing the waiting time with the orreponding throughput for variou value of the Lagrange parameter. Mathing the throughput with the throughput omputed by the threhold poliy yield a value of. The minimal waiting time that an be ahieved for that level i the waiting time omputed by the optimal poliy. Thi waiting time i denoted by a dot in the graph. It i intereting to note that the optimal poliy yield a performane very loe to the approximative threhold poliy. Extenive experiment how that for other parameter value the ame reult i obtained. The experiment indiate that the optimal poliy behave nearly a a threhold poliy, but minor differene our when x y i loe to the threhold value. REFERENCES [] E. Altman, Contrained Markov Deiion Proee. London, U.K. Chapman and Hall, 999. [2] R. Chakka and I. Mitrani, Heterogeneou multiproeor ytem with breakdown Performane and optimal repair trategie, Theoretial Comput. Si., vol. 25, pp. 9 09, 994. [3] R. B. Cooper, Introdution to Queueing Theory. Amterdam, The Netherland North-Holland, 98. [4] G. M. Koole, Stohati Sheduling and Dynami Programming. Amterdam, The Netherland CWI Trat 3. CWI, 995. [5], Strutural reult for the ontrol of queueing ytem uing eventbaed dynami programming, Queueing Syt., vol. 30, pp , 998. [6] G. M. Koole and A. Mandelbaum, Queueing model of all enter An introdution, Ann. Oper. Re., vol. 2, 2002, to be publihed. [7] M. F. Neut, Matrix-Geometri Solution in Stohati Model. Baltimore, MD John Hopkin Univ. Pre, 98. [8] M. L. Puterman, Markov Deiion Proee. New York Wiley, 994. [9] S. Stidham Jr., Analyi, deign, and ontrol of queueing ytem, Oper. Re., vol. 50, pp , 2002.