Optimal scheduling in call centers with a callback option

Transcription

1 Optimal cheuling in call center with a callback option Benjamin Legro, Ouali Jouini, Ger Koole To cite thi verion: Benjamin Legro, Ouali Jouini, Ger Koole. Optimal cheuling in call center with a callback option. Performance Evaluation, Elevier, 206, 95, pp.-40. <0.06/j.peva >. <hal > HAL I: hal Submitte on 3 Feb 206 HAL i a multi-iciplinary open acce archive for the epoit an iemination of cientific reearch ocument, whether they are publihe or not. The ocument may come from teaching an reearch intitution in France or abroa, or from public or private reearch center. L archive ouverte pluriiciplinaire HAL, et etinée au épôt et à la iffuion e ocument cientifique e niveau recherche, publié ou non, émanant e établiement eneignement et e recherche françai ou étranger, e laboratoire public ou privé.

2 Optimal Scheuling in Call Center with a Callback Option Benjamin Legro Ouali Jouini Ger Koole 2 Laboratoire Genie Inutriel, CentraleSupélec, Univerité Pari-Saclay, Grane Voie e Vigne, Chatenay-Malabry, France 2 VU Univerity Amteram, Department of Mathematic, De Boelelaan 08a, 08 HV Amteram, The Netherlan benjamin.legro@centralien.net ouali.jouini@centraleupelec.fr ger.koole@vu.nl Performance Evaluation. To appear, 205. Abtract We conier a call center moel with a callback option, which allow to tranform an inboun call into an outboun one. A elaye call, with a long anticipate waiting time, receive the option to be calle back. We aume a probabilitic cutomer reaction to the callback offer option. The objective of the ytem manager i to characterize the optimal call cheuling that minimize the expecte waiting an abanonment cot. For the ingle-erver cae, we prove that non-iling i optimal. Uing a Markov eciion proce approach, we prove for the two-erver cae that a threhol policy on the number of queue outboun call i optimal. For the multi-erver cae, we numerically characterize a witching curve of the number of agent reerve for inboun call. It i a function of the number of queue outboun call, the number of buy agent an the ientity of job in ervice. We alo evelop a Markov chain metho to evaluate the ytem performance meaure uner the optimal policy. We next conuct a numerical tuy to examine the impact of the policy parameter on the ytem performance. We oberve that the value of the callback offer i epecially important for congete ituation. It alo appear that the benefit of a reervation policy are more apparent in large call center, while they almot iappear in the extreme ituation of light or heavy workloa. We moreover oberve in mot cae that the callback offer houl be given upon arrival to any elaye call. However, if balking an abanonment are very high which help to reuce the workloa or if the overall treatment time pent to erve an outboun call i too large compare to that of an inboun one, there i a value in elaying the propoition of the callback offer. Keywor. Call center, callback option, routing optimization, queueing ytem, Markov chain, Markov eciion procee, witching curve, reervation policy, blening operation, performance meaure. Introuction Context an Motivation. Call center erve a the public face in variou area an inutrie: inurance companie, emergency center, bank, information center, help-ek, tele-marketing, jut to name a few. The ucce of call center i ue to the technological avance in information an communication ytem. The mot ue form of communication i the telephone. However, in the context of highly congete call center, the ue of alternative ervice channel can be propoe to cutomer o a to better match eman

3 an capacity. Alternative channel coul be , chat, blog, or potpone callback ervice. We focu on thi lat alternative. The iea i that cutomer, who are expecte to experience long waiting time, receive the option to be calle back later. Thi lea to a contact center with two channel, one for inboun call inboun, an another for outboun call outboun. The recent tuy of ICMI 203, bae on the analyi of 36 large contact center, report that 76% of them ue the outboun channel. The flexibility of the callback option come from the willingne of ome cutomer to accept future proceing. The call center can then make ue of thi opportunity to better manage arrival uncertainty, which in turn woul improve the ytem performance. An illutration of callback option benefit i provie in Figure. The figure give imulate performance meaure of a call center example with variou level for the ue of the callback option. We conier a non-iling ytem where inboun have a non-preemptive higher priority over outboun. We oberve that the expecte waiting time of inboun an outboun call are conierably improve by uing the callback option. For intance, the expecte waiting time of inboun coul be ivie by aroun 20 it ecreae from 8 minute an 55 econ to 23 econ while only 0% of arriving call chooe to be calle back. e e l l l el el l W l l l el el l W a Inboun b Outboun Figure : Effect of the callback option on performance arrival rate = 5.5, ervice rate = 0.2, number of agent = 28 The unpreicte an flexible call center environment offer the potential for a routing optimization that woul lea to a ignificant operational improvement. It i a non-expenive approach compare to taffing optimization Gan an Zhou, 2003; Akşin et al., One important quetion for manager in our context i how houl be the routing rule of job that woul enure non-exceive waiting time for both job type, i.e., upon a ervice completion, houl the agent hanle an inboun or an outboun call? when houl be propoe the callback offer? We are thee quetion uner a queueing moeling framework an a probabilitic cutomer reaction to the callback option. A call center where agent imultaneouly hanle inboun an outboun call i commonly referre to a call blening. The key itinction of call center problem with blening come from the fact that outboun tak have le urgency relative to inboun call. Blene operation problem have le to reearch on performance evaluation Bernett et al., 2002; Pichitlamken et al., 2003; Delaurier et al., 2007, taffing Pang an Perry, 204 an analyi of blening policie Gan et al., 2003; Bhulai an Koole, 2003; Armony an Maglara, 2004a; Armony an War, 200; Legro et al., 203, 205b. Becaue of the lack of ervice level 2

4 requirement on outboun, it i bet to give higher priority to inboun. Moreover, to reuce the number of inboun who may experience long waiting before ervice, one ha to guarantee that there i ufficient ilene in the ytem. In the patent of Duma et al. 996, bae on extenive imulation experiment, it i hown that blening inboun an outboun call an employing a threhol policy, enure that the outboun throughput rate i met while waiting time of inboun are very hort. It i alo hown that blening the two type of call in one pool require le agent than employing two itinct pool. Bhulai an Koole 2003 an Gan an Zhou 2003, prove thi optimal control, which i of threhol type, when the ervice rate of the two type of job are equal. More preciely, they how that it i optimal to cheule outboun tak only when no outboun are in the queue an the number of ile agent excee a certain threhol. In the cae of a callback option, thi policy can not be irectly applie. The reaon i that the above literature conier an infinite amount of non-priority job. In a call center with a callback option, the number of cutomer waiting to be calle back ha to be finite in orer to avoi infinite waiting. The routing policy houl then account for the length of the callback queue. Another ifference, compare to cae with claical infinite amount of outboun tak, i that inboun an outboun arrival are negatively correlate. Thi require further analyi, an may lea to ifferent managerial recommenation. Contribution. We conier a call center with a ingle cutomer type. A elaye call, with a long anticipate waiting time, receive the option to be calle back. We evelop a moeling that account for balking, abanonment, probabilitic cutomer reaction to a tate-epenent elay information, unequal ervice requirement for job type, an the eventual non-availability of a calle back cutomer. The objective of the ytem manager i to fin the optimal call cheuling policy that minimize the expecte operating cot of inboun an outboun. The control action concern the number of agent reerve for inboun an the ytem tate ituation at which the callback offer houl be propoe. We itinguih three main contribution. The firt contribution i relate to the agent reervation policy. We prove for the ingle-erver cae that non-iling i optimal. Uing a Markov eciion proce MDP approach, we prove for the two-erver cae with equal ervice requirement that a threhol policy on the number of queue outboun i optimal. Bae on the two-erver reult, we conjecture for the multi-erver cae that the optimal policy i of witch type. The number of agent to reerve for inboun epen on the number of queue outboun, the number of buy agent an the ientity of job in ervice. Moreover, we examine the impact of the ytem exogenou parameter on the agent reervation policy. We oberve, for example, that a reervation policy i not likely to be ue uner light or heavily loae ituation. The econ contribution i the performance analyi uner the optimal reervation policy. The performance meaure of interet are relate to the job type waiting time an abanonment. We evelop a controlle numerical approximation to obtain thee performance meaure for the general moeling. For variou particular cae, uing a Markov chain metho, we go further by proviing either exact numerical algorithm, or cloe-form expreion for the performance analyi. The thir contribution i the analyi of the impact of the policy parameter on performance. We erive 3

5 the firt an econ monotonicity reult in the number of agent for the performance meaure in the non-iling cae. Thee reult upport that the benefit of a reervation policy i more apparent in large call center. Moreover, in mot cae, the callback offer houl be given upon arrival to any elaye call. We prove thi reult in the non-iling cae uing firt orer monotonicity reult. However, if balking an abanonment are very high which help to reuce the workloa or if the overall treatment time pent to erve an outboun call i too large compare to that of an inboun one, there i a value in elaying the callback offer to all cutomer. Literature Review. There i a rich literature on the operation management in call center. We refer the reaer to the two urvey by Gan et al an Akşin et al For a backgroun on the pecific context of multi-channel call center, we refer the reaer to Chapter 7 in Koole 203. A mentione above, there are only few paper ealing with routing trategie in the context of a finite amount of callback. The firt two paper irectly areing the problem of the callback option are by Armony an Maglara 2004a,b. The author conier a moel in which cutomer are given a choice of whether to wait online for their call to be anwere or to leave a number an be calle back within a pecifie time or to immeiately balk. Upon arrival, cutomer are informe or know from prior experience of the expecte waiting time if they chooe to wait an the elay guarantee for the callback option. Their eciion i probabilitic an bae on thi information. Uner the heavy-traffic regime, Armony an Maglara 2004a evelop an etimation cheme for the anticipate real-time elay. They alo propoe an aymptotically optimal routing policy that minimize real-time elay ubject to a ealine on the potpone ervice moe. In Armony an Maglara 2004b, the author evelop an aymptotically optimal routing rule, characterize the unique equilibrium regime of the ytem, an propoe a taffing rule that pick the minimum number of agent that atifie a et of operational contraint on the performance of the ytem. To the contrary to Armony an Maglara 2004a,b, we account here for the feature of abanonment, unequal ervice requirement an the poible non-availability of an outboun call. Yet, our moeling i retricte to policie with trict non-preemptive priority for inboun. Armony an Maglara 2004a,b conier intea a tate-epenent priority policy. Two recent paper are by Kim et al. 202 an Duin et al Kim et al. 202 conier a call center moel with a callback option where the queue capacity for inboun i finite. A in our moeling, cutomer balking an abanonment are allowe. The author provie an efficient algorithm for calculating the tationary probabilitie of the ytem tate. Moreover, they erive the Laplace-Stieltje tranform of the ojourn time itribution of virtual cutomer. Duin et al. 203 conier a lightly ifferent moeling, where lot cutomer are calle back. There are two agent team, one that hanle in priority inboun, an another one that hanle in priority outboun. They compute the tationary probabilitie, an euce the ytem performance meaure. They alo numerically are the taffing iue for the two team. Our approach iffer from thoe in Armony an Maglara 2004a,b; Kim et al. 202; Duin et al. 203 ince we allow for agent reervation trategie. We alo allow to control the propoition of the callback offer, wherea in all above reference thi option i propoe to all cutomer. Other paper coniering finite 4

6 amount of outboun tak are Armony an War 200 an Gurvich et al They tuy call center that exercie cro-elling. The cro-elling phae i initiate by the agent an can thu be coniere a a type of outboun work in finite number. However, thee are le relate to our pecific context of callback. Structure of the paper. The remainer of thi paper i tructure a follow. In Section 2, we ecribe the call center moel with a callback option. In Section 3, we are the optimal routing problem for outboun call. In Section 4, we evaluate the performance meaure uner the optimal reervation policy. In Section 5, we ue the optimization an performance meaure reult to examine the impact of the policy parameter on performance. We then provie concluion an highlight future reearch irection. Part of the proof of the reult of the main paper are given in the appenice an the online upplement. 2 Moel Decription We conier a call center moele a a multi-erver queueing ytem with ientical, parallel erver agent. The call center hanle two type of job: inboun call type job or inboun initiate by cutomer, an outboun call type 2 job or outboun initiate by agent. Each agent can hanle both type of job. Type job requet for a real-time ervice, while type 2 job are cutomer with a potpone ervice. A job 2 cutomer i originally a job cutomer that ha choen to be calle back. The real-time ervice i more important in the ene that the waiting time of an inboun call houl be in the orer of econ or minute, wherea the potpone ervice coul be elaye for everal hour. Thi i the attractive apect for uing the callback option. It allow to create a flexibility by elaying ome of the workloa for future proceing, which woul improve the ytem performance. The arrival proce of inboun i aume to be a homogeneou Poion proce with rate λ. Inboun call arrive at a eicate firt come, firt erve FCFS queue with infinite capacity, enote by queue. We aume that the ervice time for inboun are i.i.. an exponentially itribute with rate µ. Cutomer in queue can be impatient. After entering the queue, a cutomer will wait a ranom length of time for ervice to begin. If ervice ha not begun by thi time, the cutomer will abanon. Time before abanonment for inboun are aume to be i.i.. an exponentially itribute with rate β. Becaue of the flexibility of type 2 job, the ytem manager allocate more capacity to real-time ervice. Type job have therefore a trict non-preemptive priority over type 2 job, which mean that if an agent i buy with a job 2, the agent will finih firt thi job before turning to a newly arrive job. The non-preemption priority rule i coherent with the common call center practice, where it i not appropriate to interrupt a converation with a low priority cutomer. In aition, we allow for agent reervation policie for inboun. In other wor, we allow an agent to remain ile when queue i empty an queue 2 i not. Thi may reuce the waiting time of future inboun arrival. For imilar multi-channel call center ituation, agent reervation policie have been hown to be efficient Bhulai an Koole, 2003; Legro et al., 203. If a cutomer accept to be calle back, he virtually join a FCFS queue, enote by queue 2. Due to the nature of the outboun eman, we conier for thi cutomer, the three poibilitie a follow. With probability r, he ha exactly the ame nee a the one he ha when he firt mae her call. In thi cae, the ervice time i aume to be exponentially itribute with rate µ imilarly to an inboun 5

7 cutomer. With probability r 2 r + r 2 > 0, he ha alreay reolve her problem or a part of it. Hence, her ervice time may be horter. We aume in thi cae that the ervice time i exponentially itribute with rate µ 2 µ 2 µ. Finally, with the remaining probability r r 2, the outboun cutomer i not available, an an agent will try again to call her back later on. To hanle uch a ituation, we aume that the agent pen a ranom uration aume to be exponentially itribute with rate µ 3. Thi uration correpon to the require time to leave a meage to the cutomer, an to place her back in the queue at the lat poition he will be calle back when he will again reach the firt poition uner the FCFS rule. Decription of the call back option. The tate of the ytem at a given time t i efine by four variable: x, y, 2, 3, where x i the number of inboun in queue or in ervice plu the number of outboun in ervice with the ame ervice time requirement a inboun ervice rate µ, y i the number of outboun in queue 2, 2 i the number of agent buy with outboun that require a fat ervice ervice rate µ 2, an 3 i the number of agent hanling non-available outboun ituation rate µ 3, for x, y 0 an 0 2, 3. Conier a newly arriving inboun call. If at leat one agent i available, the cutomer immeiately tart ervice. If all agent are buy an the number of waiting call in queue i trictly lower than a given threhol, enote by k N, a elay information i announce to the cutomer. The elay information i bae on the ytem tate. We o not retrict the moel to a pecific type of information: it coul be the length of queue, the expecte value or ome quantile of the waiting time, etc. The new inboun cutomer then react to the elay information. She either balk immeiately leave the ytem with probability α x,2, 3, or join queue with probability α x,2, 3 where he may abanon or tart ervice after ome time uration. We aume that the probability α x,2, 3 increae in the announce elay, i.e., α x+,2, 3 α x,2, 3, for x < + k, 0 2, 3. Note that the probability α x,2, 3 coul be choen contant for the cae with no elay information. È,, º É É Ê, Ê Y º,,» Y Figure 2: The callback option moel If the number of waiting call in queue i higher than or equal to k, the ytem provie a elay information a well a a callback option. Exceeing the threhol k capture the fact that cutomer are likely to experience too long waiting time in cae they woul requet for a real-time ervice. The elay information i ytem tate-epenent. Concretely, the new inboun cutomer have the following three poibilitie 6

8 upon her arrival: he balk immeiately leave the ytem with probability α x,2, 3, or he chooe the callback option an virtually join queue 2 with probability q x,2, 3, or he join queue with probability q x,2, 3 α x,2, 3, for x k, 0 2, 3. Again, we aume that α x+,2, 3 α x,2, 3 an q x+,2, 3 q x,2, 3 for x k an 0 2, 3. Alo, the quantitie α x,2, 3 an q x,2, 3 coul be choen contant for x k, 0 2, 3. In uch a cae, we will then imply write them a α or q to implify the preentation. An illutration of the moel i given in Figure 2. Problem formulation. Let u firt efine the performance meaure of interet. We enote by W, W 2 an W the ranom variable meauring the tationary waiting time of erve inboun in queue, the tationary waiting time of outboun in queue 2, an the unconitional tationary waiting time in the queue of an arbitrary job inboun or outboun, repectively. We alo enote by P a the tationary proportion of inboun that leave the ytem without ervice either by abanoning queue, or by balking upon arrival. The tationary proportion of inboun that balk upon arrival i efine a P b. We finally enote by ψ the tationary probability that a new inboun call become an outboun one. We conier an economic framework bae on the holing cot of job an 2, an the cot of lot call becaue of balking or abanonment. The objective of the ytem manager i to characterize the optimal routing policy which minimize the expecte ytem cot, enote by SC, an given by SC = γ EW + γ 2 EW 2 + γ 3 P a, where γ, γ 2 an γ 3 are the cot parameter, an where EZ i the expecte value of a given ranom variable Z. We aume that γ > γ 2 to give more importance to the waiting time of inboun than that of outboun. The control parameter for the call center manager are the threhol k for queue which characterize the callback option, an the agent reervation policy for inboun. For a given tate x, y, 2, 3 0 x < an y > 0, there are two poible action: the firt one i to erve an outboun call an move to tate x +, y, 2, 3 with probability r, or to tate x, y, 2 +, 3 with probability r 2, or to tate x, y, 2, 3 + with probability r r 2 ; the econ one i to keep the firt outboun in line in queue 2 an tay at tate x, y, 2, 3. The knowlege of the optimal action at each tate efine a function enote by cx, y, 2, 3. The curve of thi function eparate the tate where the optimal action i to erve an outboun call from thoe where it i optimal to keep an outboun call in queue 2. The function cx, y, 2, 3 efine therefore the agent reervation policy. It will be characterize in Section 3. A ummary of the moel notation i given in Table. The call center moel ecribe above i referre to a Moel G general moel. Becaue of it complexity, we efine ubmoel that correpon to variou pecial cae, for which it i eaier to oberve an prove inight. We enote by Moel A the ubmoel where outboun have the ame ervice rate a inboun an thee are available when they are calle back r = an r 2 = 0, by Moel B a ubmoel of Moel A where inboun are infinitely patient β = 0, by Moel C a particular cae of Moel B where the balking an callback parameter are aume to be contant for example when no information i given to arriving 7

9 Table : Moel notation Sytem tate ecription x Number of inboun in queue or in ervice plu number of outboun with the ame ervice requirement a inboun in ervice y Number of outboun in queue Number of agent hanling fat-erve outboun Number of agent hanling non-available outboun ituation Exogenou parameter λ Arrival rate of inboun Number of agent r Probability that an outboun call ha the ame ervice requirement a an inboun one r 2 Probability that an outboun call ha a horter ervice requirement than an inboun one r r 2 Probability that an outboun call in queue 2 i not available µ Service rate of inboun, an alo a part of outboun with the ame ervice requirement µ 2 Service rate for fat-erve outboun µ 3 Service rate for hanling non-available outboun β Abanonment rate for each inboun call in queue α x,2, 3 Probability that a new inboun call balk upon arrival q x,2, 3 Probability that an inboun call accept the callback offer upon arrival Control parameter k cx, y, 2, 3 Ψ P a EW, EW 2, EW Threhol on the length of queue, at which we tart to propoe the callback offer Curve for the agent reervation policy Performance Meaure Proportion of inboun that accept the callback offer Proportion of inboun that leave the ytem without ervice after a balking or an abanonment Expecte waiting time for erve inboun in queue, expecte waiting time for outboun in queue 2, an unconitional waiting time in the queue of an arbitrary job inboun or outboun, repectively cutomer. We alo efine Moel NI non-iling moel a ubmoel of Moel G where iling i not allowe i.e., the firt outboun call in queue 2 tart ervice a oon a an agent become available an queue i empty. An illutration of the ubmoel i epicte in Figure 3.»,» 0 É 0 / E º,, º, È,, È D' DE/ D D D Figure 3: The ubmoel Markov eciion proce approach. For Moel G, we formulate the routing problem a a Markov eciion proce MDP. Since we are coniering long-term average performance, it i optimal to cheule job at arrival, ervice completion or abanonment time. If it i optimal to keep a erver ile at a given time, then the action remain optimal until the next event in the ytem. Thi reult follow irectly from 8

10 the continuou-time Bellman equation Puterman 994, Chapter. Therefore, it uffice to conier the ytem only at arrival, ervice completion or abanonment time. Due to the call abanonment in queue, the total event rate i not boune. We therefore ue the traitional approach where we aume that queue ha a limite capacity N N 0. The parameter N i choen high enough to approximate the real ytem. The total event rate i then uniformly boune by λ + maxµ, µ 2, µ 3 + Nβ, an without lo of generality, we aume that it i equal to one. We next ue the well known uniformization technique Puterman 994, Chapter 8, which allow to apply icrete-time ynamic programming to characterize the optimal routing policy. The poible action for an agent jut after a ervice completion an queue i empty are either to remain ile, or to erve an outboun call if queue 2 i not empty. We chooe to formulate a 2-tep value function, in orer to eparate tranition an action an implify the involve expreion. We efine the equence U n x, y, 2, 3 an V n x, y, 2, 3 over n tep, for n, x, y 0 an 0 2, 3. For n 0, we have U n+ x, y, 2, 3 = γ x γ 2 y + λ [ 0 x <V n x +, y, 2, 3 + x <+k α x,2, 3 V nx +, y, 2, 3 + α x,2, 3 V nx, y, 2, 3 + γ 3 + +k x <+Nq x,2, 3 V n x, y +, 2, 3 + α x,2, 3 V n x, y, 2, 3 + γ 3 + q x,2, 3 α x,2, 3 V nx +, y, 2, 3 + x =+Nq N,2, 3 V n x, y +, 2, 3 + q N,2, 3 V n x, y, 2, 3 + γ 3 ] + βx V n x, y, 2, 3 + γ 3 + min 2 3, xµ V n x, y, 2, µ 2 V n x, y, 2, µ 3 V n x, y +, 2, 3 + λ βx min 2 3, xµ 2 µ 2 3 µ 3 Vn x, y, 2, 3, for x, y 0, an 0 2, 3, where x A i the inicator function of a ubet A, an V n+ x, y, 2, 3 = minr U n+ x +, y, 2, 3 + r 2 U n+ x, y, 2 +, 3 + r r 2 U n+ x, y, 2, 3 +, U n+ x, y, 2, 3, for y > 0 an 0 x < an V n+ x, y, 2, 3 = U n+ x, y, 2, 3 in the remaining cae. We chooe V 0 x, y, 2, 3 = U 0 x, y, 2, 3 = 0, for x, y 0, an The tranition at bounary tate x = N are choen uch that the monotonicity propertie of the value function are maintaine. The value of thi choice i proven in the proof of Theorem in Section 3.2. Another poibility to maintain the monotonicity propertie i to ue the moothe rate truncation a propoe by Bhulai et al. 204, however, thi woul imply a more complicate expreion of the value function in our etting. The long-term average optimal action can be obtaine through value iteration, by recurively evaluating V n uing Equation, for n 0. A n ten to infinity, the minimizing action converge to the optimal one Puterman, 994. For 0 x < an y > 0, the minimizing action i choen between keeping an outboun call in queue 2 or tarting the ervice of thi call. For x , we o not conier any control action becaue of the priority for inboun i.e., no poibility of having an ile agent while a call i 9

11 waiting in queue. 3 Optimal Agent Reervation Policy We conier the ingle, the two-erver an the multi-erver cae. For the multi-erver cae of Moel G, we firt prove a preliminary reult tating that when all agent are iling an queue 2 i not empty, then it i optimal to erve at leat the firt outboun call in line. A corollary of thi reult i that non-iling i optimal in the ingle-erver cae. In the two-erver cae, we prove in Theorem the optimal reervation policy for Moel A. It i a threhol policy on the number of waiting outboun in queue 2. For the multi-erver cae of Moel A an G, we conjecture that the optimal routing follow a tate-epenent threhol policy, i.e., a witching curve. For Moel A, the witching curve i only bae on the number of outboun in queue 2 an the number of buy agent. In aition to that, for Moel G, the optimal policy epen on the number of each job type in ervice. The reult for the multi-erver cae i intuitive an a tanar extenion, in MDP problem, of the prove reult in the ingle an two-erver cae. It i however very har to obtain a proof becaue of the growing imenionality of the unerlying tate pace an the problem et own by the eparture term. Thi proof i relate to a well known funamental queueing control problem, for which no rigorou proof oe exit yet. We believe that our proof for the two-erver cae houl give ome inication that woul motivate future reearch. Thi open quetion conit in howing the propagation of a monotonicity relation through the minimizing operator. In Remark inie the proof of Theorem in Appenix A, we provie the mathematical etail of what houl be proven to rigorouly obtain the multi-erver reult. It reuce to that for the well known routing problem in the heterogeneou multi-erver queue, where the objective i to fin a non-preemptive routing policy that minimize the long run average time in the ytem Hajek, 984; Lin an Kumar, 984; e Véricourt an Zhou, For a backgroun on thi quetion, we refer the reaer to Koole Preliminary Reult Propoition provie a preliminary reult for Moel G. Propoition In the multi-erver cae of Moel G, if all agent are iling an queue 2 i not empty, then it i optimal to erve at leat an outboun call. Proof. For γ 2 > 0, it i clear that an outboun call in queue 2 ha to be erve at one point. Otherwie, queue 2 woul contain an infinite number of outboun ue to the FCFS rule. Therefore, a policy which woul not erve an outboun call can not be optimal. We next prove that the bet ituation for the ervice of an outboun call i when all agent are iling. Serving an outboun call alway improve the performance of outboun whether thi outboun call i erve when all agent are iling or in another ituation. An outboun taken in ervice woul eteriorate the performance of inboun if new inboun arrive at a buy ytem while thi outboun call i till in ervice. The lowet value of the probability of uch an event i reache in the cae thi outboun call ha been taken in ervice when all agent are iling. Moreover, an outboun call ervice uration oe not epen on the ytem tate. Thu, erving an outboun call when all agent are iling improve the performance of outboun an ha the mallet probability to eteriorate 0

12 the performance meaure of inboun. Since all outboun ha to be erve at one point, an optimal tate-epenent policy force the ervice of outboun, if any, when all agent are ile. We next euce the optimal agent reervation policy for the ingle-erver cae of Moel G. Corollary In the ingle-erver cae of Moel G, the optimal agent reervation policy i the non-iling policy. The proof of Corollary irectly follow from Propoition. In Section of the online upplement, we propoe another proof of thi corollary for Moel A uing an MDP approach. 3.2 Two-erver Reult for Moel A In the two-erver cae, uing Propoition, we never encounter ituation for the optimal policy where the two agent are iling an at leat one outboun call i in queue 2. When one erver i buy, we prove in Theorem that the optimal policy in Moel A i of threhol type for the reervation of the other erver. Theorem In the two-erver cae for Moel A, when one agent i buy, there exit a threhol on the number of outboun in queue 2, at an beyon which it i optimal to erve the firt waiting outboun in line, an it i optimal to not erve outboun in the remaining cae. The proof i given in Appenix A. It i bae on the propagation of monotonicity reult of the value function a efine in Section 2. Thi type of proof i tanar in MDP problem Koole, Yet, our reult can not irectly follow from Koole 2007 for the following reaon. The exiting reult concern motly the ingle-erver-one-imenional cae. Le i oable in the multi-imenional cae for the propagation of the reult through the minimizing operator. Moreover, abanonment from queue i allowe here, a feature that often break the monotonicity propertie when pace truncation i require. We how in our proof that the monotonicty propertie are maintaine. Finally, the complexity of the proof come from the arrival term, which i pecific in our moel an require a pecial conieration, becaue the two queue are involve an the cutomer reaction i tate-epenent. 3.3 Multi-Server Conjecture Let u now comeback to the multi-erver cae. Uing the value function efine in Section 2, we conjecture that the optimal policy i of witch type. For both Moel A an G, we conuct a numerical tuy from which we euce the witching curve which eparate tate where it i optimal to erve an outboun call from thoe where it i not. We alo examine the impact of the ytem parameter on the reervation policy Switching Curve for Moel A For Moel A, we o not nee to itinguih between inboun an outboun in ervice. Let u rewrite the value function for Moel A µ = µ 2 = µ, r =. We have for n 0, U n+ x, y = γ x + + γ 2 y + λ [ 0 x< V n x +, y + x<+k α x V n x +, y + α x V n x, y + γ 3 + +k x<+n q x V n x, y + + α x V n x, y + γ 3 + q x α x V n x +, y + x=+n q N V nx, y + + q N V nx, y + γ 3 ] + βx + V n x, y + γ 3 + min, xµv n x, y + λ βx + min, xµ Vn x, y, for x, y 0,

13 with V n+ x, y = minu n+ x +, y, U n+ x, y, for y > 0 an 0 x < an V n+ x, y = U n+ x, y in the remaining cae. We chooe V 0 x, y = U 0 x, y = 0, for x, y 0. We conjecture that the optimal policy i a function of x number of call in ervice plu number of inboun in queue an y number of outboun in queue 2. Figure 4 give variou optimal witching curve to illutrate the impact of the ytem parameter on the optimal policy. The abcia axi in each figure repreent the overall number of job in the ytem number of outboun in queue 2 plu number of call in ervice an the orinate axi repreent the number of call in ervice. We only conier tate where 0 x <. For the remaining tate, the only poible action i to keep outboun in the queue. The optimal action can be rea from the figure. Conier a given point x + y, x 0 x < an y > 0. If thi point i trictly uner the curve, then it i optimal to erve an outboun call an therefore move from x+y, x to x++y, x+ = x+y, x+. If thi new point i trictly uner the curve then the optimal action i to erve another outboun call. We continue to take the eciion to erve by moving on a vertical line until we reach the curve. On the witching curve or above, the optimal action i to keep outboun in the queue. The value to chooe x + y in abcia intea of y i to oberve the evolution from a non-optimal point to the optimal one on a vertical line intea of a iagonal one. The curve in ahe line repreent the non-iling policy. We oberve that when x = 0 an y > 0, the optimal action i alway to erve an outboun call thi hol from Propoition. Given that the witching curve i increaing in x + y, it i an increaing tep function. It i given by cx + y = miny 0, x + y + x+y y + x+y y2 + + x+y y y0, 2 where y 0 < y < y 2 < < y y0. The parameter y 0,..., y y0 are the level that repreent the changing point of the witching curve. Uing Propoition, we have y 0 0. Equation 2 can be interprete a follow. Aume we have x + y job in the ytem x buy agent an y outboun in queue 2. If x + y < y, then it i optimal to have at mot y 0 tak in ervice, i.e., if x < y 0 we move from tate x, y to tate miny 0, x + y, y miny 0, x + y x, an if x y 0 we tay in tate x, y. If y x + y < y 2, then at mot y 0 + job houl be in ervice, i.e., if x < y 0 + we move from tate x, y to tate miny 0 +, x + y, y miny 0 +, x + y x, an if x y 0 + we tay in tate x, y, an o on. Finally, if y y y0, then at mot y 0 + y 0 = job houl be in ervice. In other wor, when x + y y y0, no agent are reerve for inboun an it i optimal to move from tate x, y to tate min, x + y, y min, x + y x. A qualitative interpretation of Equation 2 i that the more numerou queue outboun an the le buy are the agent, the more likely the optimal eciion woul be to erve an outboun call. Thi witch type policy in the multi-erver cae i a tanar extenion of the threhol policy in the two-erver cae. The new element in the multi-erver cae i that the eciion to erve an outboun call houl no longer only epen on the length of queue 2, ince more than one agent might be involve. For 2

14 e e e le l ll ll l e e e lel e e e a Impact of γ 2 λ = 4, q = 40%, α = β = 0 b Impact of q γ 2 = 0.05, λ = 4, α = β = 0 e e e e e e ll ll e e e c Impact of q +k+x λ = 4, γ 2 = 0.05, α = β = 0, x 0 ll l l e Impact of λ γ 2 = 0.05, q = 40%, α = β = 0 ll ll ll lel e e e e Impact of α λ = 4, γ 2 = 0.05, q = 40%, β = 0. lel e e e f Impact of α +x λ = 4, γ 2 = 0.05, q = 40%, β = 0, x 0 Figure 4: Optimal witching curve µ = 0.2, r =, = 28, γ =, k = 5, γ 3 = 0.5 a given ituation with x buy agent an x ile agent, the optimal policy i a threhol policy on the length of queue 2. Thi lea, a a conequence, to a witch type policy. We next examine the impact of the parameter on the reervation policy. In Propoition 2, we prove that the more importance i given to inboun an the le cutomer are likely to accept the callback offer, the higher houl be the reervation for inboun. Propoition 2 Conier two ituation with ientical arrival an eparture parameter λ, α x for x, β, an µ. The firt ituation ha the cot parameter γ, γ 2 an γ 3 an the econ one ha γ, γ 2 an γ 3. The callback parameter are contant for both ituation. They are q an q + q for the firt an econ ituation, repectively. 3

15 If γ γ, γ 2 γ 2, γ 3 γ 3, q 0, then the firt ituation require more reervation than the econ one. In other wor, the witching curve i lower for the firt ituation. The proof of thi propoition i given in Appenix B. The impact of the cot parameter γ 2 i illutrate in Figure 4a, i.e., the witching curve increae the reervation ecreae in γ 2. The oppoite i true when γ or γ 3 increae. Figure 4b illutrate the impact of a contant callback parameter q x = q for x + k. It how that the more cutomer are likely to accept the callback option, the higher i the witching curve le reervation for inboun. The ame obervation hol when q x i not contant Figure 4c. The key factor, wether the callback parameter i contant or not, i the proportion of outboun. A le intuitive obervation i that the witching curve i not monotone in the workloa, efine a λ/µ Figure 4. We oberve that reervation oe not happen in the extreme ituation of light or heavy workloa. For light workloa ituation, the ytem capacity i high enough, uch that both call type experience mall waiting time. Then, the reervation for inboun call oe not nee to be ubtantial. For high workloa ituation, queue i often long. Thu, a high proportion of cutomer woul chooe the callback option an join queue 2. Given that queue 2 i alo long, the ytem houl not further eteriorate the waiting of outboun by reerving agent for job. However, for an intermeiate ituation, with a moerate workloa, job 2 are le numerou, an o not therefore nee to have acce to all agent. The ytem may then conier agent reervation for job. Figure 4e reveal that the impact of the balking parameter α x an the abanonment parameter β are not imilar to that of the workloa. For high value of α x or β, the ytem capacity i high enough to achieve mall waiting time. However, the proportion of abanonment i high, o, the reervation for inboun nee to be important to avoi too much abanonment. For low value of α x or β, the reervation policy mainly epen on the workloa λ/µ ee Figure 4e an 4f Switching Curve for Moel G We now conier Moel G. Figure 5 an 6 illutrate the witching curve for the optimal policy in Moel G. Again, the curve in ahe line repreent the non-iling policy. A expecte, we oberve that the optimal eciion are not only bae on the number of outboun in queue 2 an the number of buy agent a for Moel A, but alo the ientity of the job in ervice. We itinguih three ifferent zone elimite by two witching curve. A firt witching curve i efine for the cae where all buy agent are buy with rate µ 2 = 3 = 0. Thi ituation i the wort for the occupancy of the agent, becaue µ 3 µ 2 µ. Thu, uner thi firt witching curve, for any tate with le buy agent or more outboun in queue 2, the optimal eciion i to erve an outboun call if any, i.e., we move from tate x+y , x to tate x++y , x = x+y , x A econ witching curve i efine for the cae were all buy agent are buy with rate µ 3 x = 2 = 0. Thi ituation i the bet for the occupancy of the agent. On an above thi econ witching curve, for any tate with more buy agent an le outboun in queue 2, the optimal eciion i to keep all outboun in queue 2. The orering µ 3 µ 2 µ jutifie that the firt witching curve i below the econ one. Even in the 4

16 e e e e Figure 5: Optimal witching curve λ = 3.8, q = 40%, α = β = 0, γ =, γ 2 = 0.05, k = 5, µ = 0.2, µ 2 =, µ 3 = 0, r = r 2 = /3, = 28 cae µ 3 = µ 2, the econ witching curve x = 2 = 0 i till higher than a witching curve where all buy agent are buy with rate µ 2 x = 3 = 0. The reaon i the high nee of erving outboun when all agent are buy with rate µ 3. If the agent are all hanling a non-available outboun ituation, they woul not reuce the number of outboun in the ytem, o, the nee for erving outboun oe not reuce. Yet, for ituation with mall number of cutomer in the ytem or high number of cutomer in queue 2, the two extreme witching curve correponing to 2 = 3 = 0 an x = 2 = 0 coincie. Therefore, there only exit a finite number of tate where the optimal eciion epen on the ientity of the job in ervice. Figure 6a reveal that the two extreme witching curve get cloer to one another a r, µ 2, or µ 3 increae. The reaon i the imilarity between the ervice requirement of inboun an outboun. Figure 6b reveal that a r +r 2 ecreae, the two extreme witching curve get higher, i.e., le agent reervation. The reaon i relate to the ifficulty of erving an outboun call. When agent are often hanling nonavailable outboun ituation, it i ifficult to reuce the length of queue 2, therefore, outboun houl benefit from more availability of the agent. e e e e e e <^ <^ e e a Example with r = 80%, r 2 = 5%, µ = 0.2, µ 2 = 0.5 an µ 3 = 0 b Example with r = 0%, r 2 = 0%, µ = 0.2, µ 2 = an µ 3 = 0 Figure 6: Optimal witching curve λ = 3.8, q = 40%, α = β = 0, γ =, γ 2 = 0.05, k = 5, = 28 Similarly to Moel A, ince the witching curve i increaing in x + y , it i an increaing tep 5

17 function. Given that agent hanle 3 ifferent type of job, we efine the 3 variable increaing function bx, 2, 3 which give the buyne of the agent team. Becaue the number of agent i finite, we aume without lo of generality that 0 bx, 2, 3. Thi buyne function correct the witching curve, efine for Moel A, into cx + y = miny 0, x + y bx,2, 3 b 0 + x+y+2+ 3 y bx,2, 3 b + x+y+2+ 3 y 2 bx,2, 3 b x+y+2+ 3 y y0 bx,2, 3 b y0 + x+y+2+ 3 y y0, where y 0 < y < y 2 < < y y0 an 0 < b 0 b b y0. The parameter y i, 0 i y 0, have the ame ignification a thoe for Moel A. The parameter b i, 0 i y 0, are the level of change of the buyne of the agent team. The value of the b i can be etermine uing value iteration. From the numerical experiment, we oberve that the value of the b i are ifferent than one only for mall value of i. Thi implie that the buyne of the agent team affect the optimal eciion only when the number of buy agent i low. The reaon i relate to the blocking rik for an inboun call. When mot of the agent are iling, the eciion to erve an outboun call woul mot likely not block the agent team. In uch a ituation, what affect the eciion i then the ientity of job in ervice. In the oppoite cae, when mot of the agent are buy, the ervice of an outboun call coul eaily lea to a blocking ituation waiting time for inboun call. In uch a ituation, what affect the eciion i then the total number of buy agent x an the length of queue 2 y, more than the ientity of the job in ervice. 4 Performance Analyi We compute the tationary performance meaure. In Section 4., we profit from the contant tranition rate an propoe an exact algorithm for Moel C. In Section 4.2, we provie a controlle approximation bae on value iteration for Moel A an G. In Section 4.3, we conier pecial cae of agent reervation for Moel C Section 4.3. an the non-iling cae for Moel A Section Thi allow to obtain cloe-form expreion for the boun of the performance meaure of Moel A an C. 4. Moel C We compute here EW, EW 2, P b an Ψ. Our approach i bae on the analyi of the unerlying Markov chain. We compute the tationary probabilitie of the ytem tate by olving a ytem of linear ifference equation. We o o by olving the involve homogeneou equation efine on the et of complex number. Although ome quantitie contain infinite ummation, we provie a metho that allow to o the exact computation within a finite number of calculation. Conier the tochatic proce {xt, yt, t 0}, where xt enote the number of call in queue job or in ervice job or 2; an yt enote that in queue 2 job 2 at a given time t 0. We have xt, yt {0,, 2,...}, for t 0. A inter-arrival an ervice time are exponentially itribute, {xt, yt, t 0} i a Markov chain. An illutration of thi Markov chain in given in Figure 7. We enote by p x,y the tationary probability to be in tate x, y, for x, y N. In what follow, we 6

18 l l l l ll l l ll Figure 7: Markov chain for Moel C q = q + α compute the tationary probabilitie, from which we thereafter euce the ytem performance meaure of interet. To implify the preentation of the analyi, we ivie it into the following 7 tep: Step. We provie the et of equilibrium equation relating the tationary probabilitie. Step 2. We implify the expreion of p x,y, for x + k an y 0, by expreing them a a function of only two tate probabilitie from the row y in the Markov chain. Step 3. We how how p x,y, for x +k an y 0, can be compute a a function of p +k,0, p +k,,, p +k,y. Step 4. We evaluate all tationary probabilitie for x 0 an y = 0 a a function of p 0,0. Step 5. For y 0, we evelop a recurrence metho to compute all tationary probabilitie of row y + in the Markov chain a a function of the previou row. Thu all tationary probabilitie can be erive a a function of p 0,0. Step 6. Although p 0,0 involve an infinite ummation, we provie a metho to compute it within a finite number of calculation. Step 7. We finally erive the ytem performance meaure a a function of the tationary probabilitie. The etail for each tep are given in Appenix C. 7

19 4.2 Moel A an G We compute here EW, EW 2, P a, an Ψ. We propoe a numerical metho bae on the iterative computation of the ynamic programming operator. For Moel A, auming the witch policy a efine in Section 3.3., the value function can be rewritten, for n 0, a V n+ x, y = γ x + + γ 2 y + λ [ 0 x< V n x +, y + x<+k α x V n x +, y + α x V n x, y + γ 3 + +k x<+n q x V n x, y + + γ 4 + α x V n x, y + γ 3 + q x α x V n x +, y + x=+n q N V n x, y + + γ 4 + q N V n x, y + γ 3 ] + βx + V nx, y + γ 3 [ + min, xµ y>0 x+y y,x y 0 + y <x+y y 2,x y y2 <x+y y 3,x y y y0 <x+y,x V n x, y ] + y>0 x+y y,x y 0 + y <x+y y 2,x y y2 <x+y y 3,x y y y0 <x+y,x V n x, y + λ βx + min, xµ Vnx, y, for x, y 0, with V 0 x, y = 0, for x, y 0. For Moel G, auming the witch policy a efine in Section 3.3.2, the value function can be rewritten, for n 0, a U n+ x, y, 2, 3 = γ x γ 2 y + λ [ 0 x <V n x +, y, 2, 3 + x <+k α x,2, 3 V nx +, y, 2, 3 + α x,2, 3 V nx, y, 2, 3 + γ 3 + +k x <+Nq x,2, 3 V n x, y +, 2, 3 + γ 4 + α x,2, 3 V n x, y, 2, 3 + γ 3 + q x,2, 3 α x,2, 3 V nx +, y, 2, 3 + x =+Nq N,2, 3 V n x, y +, 2, 3 + γ 4 + q N,2, 3 V n x, y, 2, 3 + γ 3 ] + βx V nx, y, 2, 3 + γ 3 [ + min 2 3, xµ y>0 x+y y,x y 0,bx, 2, 3 b 0 + y <x+y y 2,x y 0 +,bx, 2, 3 b + + y y0 <x+y ,x r V nx, y, 2, 3 + r 2 V nx, y, 2 +, 3 + r r 2 V nx, y, 2, y>0 x+y y,x y 0,bx, 2, 3 b 0 + y <x+y y 2,x y 0 +,bx, 2, 3 b ] + + y y0 <x+y ,x V nx, y, 2, 3 [ + 2 µ 2 y>0 x+y y,x y 0,bx, 2, 3 b 0 + y <x+y y 2,x y 0 +,bx, 2, 3 b + + y y0 <x+y ,x r V nx +, y, 2, 3 + r 2 V nx, y, 2, 3 + r r 2 V n x, y, 2, y>0 x+y y,x y 0,bx, 2, 3 b 0 + y <x+y y 2,x y 0 +,bx, 2, 3 b ] + + y y0 <x+y ,x V n x, y, 2, 3 [ + 3 µ 3 y>0 x+y y,x y 0,bx, 2, 3 b 0 + y <x+y y 2,x y 0 +,bx, 2, 3 b + + y y0 <x+y ,x r V n x +, y, 2, 3 + r 2 V n x, y, 2 +, 3 + r r 2 V n x, y, 2, 3 + y>0 x+y y,x y 0,bx, 2, 3 b 0 + y <x+y y 2,x y 0 +,bx, 2, 3 b ] + + y y0 <x+y ,x V n x, y, 2, 3 + λ βx min 2 3, xµ 2 µ 2 3 µ 3 Vn x, y, 2, 3, for x, y 0, an 0 2, 3, 8

20 with V 0 x, y, 2, 3 = 0, for x, y 0 an 0 2, 3. In both cae Moel A an G, the tanar way of obtaining the long-term performance meaure i through value iteration, by recurively evaluating V n, for n 0. A n ten to infinity, the ifference V n+ x, y, 2, 3 V n x, y, 2, 3 converge to the eire metric. Thu, we top the iteration until the following criterion i met max {V n+ x, y, 2, 3 V n x, y, 2, 3 } min {V n+ x, y, 2, 3 V n x, y, 2, 3 } < ϵ, x,y, 2, 3 x,y, 2, 3 for ome given mall ϵ. In what follow we precie the parameter in the value function which allow to compute the eire performance meaure. One can calculate the expecte number of cutomer in queue, ay EN, by letting γ =, γ 2 = 0, γ 3 = 0, γ 4 = 0 in the value function; the expecte number of cutomer in queue 2, ay EN 2, by letting γ = 0, γ 2 =, γ 3 = 0, γ 4 = 0; the proportion of cutomer who abanon the ytem, P a, by letting γ = 0, γ 2 = 0, γ 3 = /λ, γ 4 = 0; the proportion of cutomer who chooe the callback offer, Ψ, by letting γ = 0, γ 2 = 0, γ 3 = 0, γ 4 = /λ. Uing next the Little law, we obtain the expecte waiting time for erve cutomer in queue, EW = EN λ P a Ψ ; an the expecte waiting time in queue 2, EW 2 = EN2 λψ. 4.3 Special Cae We conier here ome pecial cae of agent reervation for Moel C an the non-iling cae for Moel A Special Reervation Cae for Moel C We efine for Moel C the threhol y 0 on the number of buy agent. If the number of buy agent i lower than or equal to y 0 y 0 an at leat one outboun call i in queue 2, then we erve thi outboun call. In the remaining cae, we o not erve outboun. Therefore, the witching curve of thi policy i cx + y = minx + y, y 0. Since the optimal action i to erve an outboun call when all agent are iling Propoition, the wort policy for outboun the bet cae for inboun conit of erving an outboun call only when all agent are iling. We refer to the latter a the highet reervation policy. It correpon to the cae y 0 =. A for the non-iling policy, it correpon to the cae y 0 =. The analyi of thi policy i a euce from that of Section 4.. In Corollary 2, we give cloe-form expreion for EW, P b an Ψ a a function of y 0. The proof i given in Section 2 of the online upplement. Corollary 2 For y 0, we have q Ψ = a α k a! a q α q a y 0 a y 0 y 0!! p 0,0 a α k, a q α 9