On the Scheduling of Operations in a Chat Contact Center

Transcription

1 On the Scheduling of Operations in a Chat Contact Center Benjamin Legros a Oualid Jouini b a PSB Paris School of Business, Department of Economics, 59 rue ationale, Paris, France b Laboratoire Genie Industriel, CentraleSupélec, Université Paris-Saclay, Grande Voie des Vignes, Chatenay-Malabry, France benjamin.legros@centraliens.net oualid.jouini@centralesupelec.fr Abstract The emerging chat channel allows a contact center agent to work simultaneously on many customers. This flexibility brings however new issues for the management of operations. The improvement in efficiency because of work simultaneity can be challenged by the new abandonment feature in service. In this paper, we consider a contact center with inbound and outbound chats. Using a Markov decision approach, we focus on the chat routing problem to service in order to find the best tradeoff between waiting in the ueue and abandonment. We provide reuired conditions under which the classical Least Busy First LBF or the Most Busy First MBF policies are optimal. For the case without abandonment, we provide the reuired conditions for the optimal policy to be of switch type. The optimal decision is state-dependent, based on the number of chats in the ueue and the number of chats in service. For the case with chat abandonment, we use a smoothed rate truncation method to numerically obtain the optimality of a switch type policy. We also evaluate the improvement in implementing this state-dependent reservation policy in comparison to the classical fixed threshold policy. A number of managerial insights are derived from the theoretical results. For instance, a fair routing is not necessarily a good routing in terms of the call center performance. Also, a state-dependent policy outperforms a fixed threshold one especially for the setting of large call centers with highly impatient customers. Keywords: Contact centers, ueueing systems, Markov decision process, chat messages, routing, multitasking, abandonment.

2 1 Introduction Context and motivation. ew advances in telecommunication technology are revolutionizing the way call centers interact with customers. Customer preferences are also evolving rapidly toward the use of new technology. In this context, there has been in recent years a surge of interest about new software products that make it possible for contact centers to offer assistance to online users via the chat channel. Chat systems allow customers to access an instant messaging system built into the call center website to interact with agents online. From an operational point of view, chats offer more interactivity compared to s which leads to a better efficiency for demand resolution and less waste of the capacity time on already solved problems. They also allow for less working time per customer than call conversations because of the multitasking possibility. Chats are then somehow combining the benefits of call and channels. Other advantages of chat systems are the features such as screen sharing and the ability to share files and data, which are particularly useful to computer companies, software companies, and e-retailers Cui and Tezcan, Despite its prevalence in practice 40% of call centers use chats as reported in ICMI 2013, very few related papers have been published. Due to their uniue features, we believe that chat service systems are and will be an important way to interact with customers. Organizations that run contact centers will take a higher care at this alternative communication channel to improve efficiency and reduce costs relative to for example the cost of servicing phone calls. In this paper, we consider a contact center with inbound and outbound chats and address the routing problem of chats to agents. The optimization problem consists of a tradeoff between waiting times and abandonments. Interactions between agents and customers in a chat contact center can be modeled as a ueueing system. The characteristics of this ueueing model is that agents can treat more than one chat at once, customers can abandon not only during the wait but also during the service for example due to a perceived long wait during the discussion, a customer can be served by more than one agent successively because the conversation is written and agents are anonymous. In contrast to the existing studies, we allow in our modeling for two important practical features. First, existing studies usually assume for simplicity that, for a given agent, the service rate per chat is decreasing in the number of simultaneously treated chats. However, this is not always the case in practice. A statistical study by Tan and etessine 2014 shows that the agents speed up behavior is observed and may be explained by a rushing effect. We allow 2

3 here the service rate per chat to be an arbitrary function of the number of simultaneously treated chats. As a conseuence, to obtain a better efficiency, we also relax the assumption of the classical lightest-load-first rule which says that the least busy agent should be chosen in priority for the service of a new customer. Second, we do not restrict the routing policy of chats to agents to the classical fixed threshold policy, where chats are automatically assigned to a given agent until a predefined maximum limit is reached. We instead allow for idling policies. A chat is not automatically routed to an agent because this may deteriorate the system performance. The decision then depends on the system state. The use of a statedependent policy is illustrated in Figure 1 for a Brazilian chat contact center. The figure shows the number of simultaneously treated chats for a given agent as a function of time the entity agent 1 may concern more than one agent since agents are anonymous. We observe from the figure that the number of simultaneously treated chats increases with the system workload the threshold is the highest during peak hours. Moreover, in our modeling we allow for the possibility of initiating outbound chats. Outbound chats are used in practice to increase agents occupancy during off-peaks periods, prospect for new customers, etc. Contrary to calls, because of the possibility of simultaneous treatment, an agent does not have to be completely available to initiate an outbound chat. Yet, the initiation of outbound chats leads to the risk of blocking agents which delays the treatment of incoming inbound chats. We study here the tradeoff between the degradation in the uality of service for inbound chats and the opportunity of serving more customers by doing outbound chats. dd e E d e e e d d d d d d d d d d e e dd dd Figure 1: Data from a Brazilian chat contact center Contributions. We focus on a chat routing problem where the call center manager seeks for the tradeoff between various conflicting objectives, namely, the waiting time of inbound chats in the ueue, the service time duration per chat, and the throughput of treated jobs. The main contributions of this paper are as follows. First, we provide reuired conditions under 3

4 which the classical Least Busy First LBF or the Most Busy First MBF policies are optimal. Second, we consider the case without abandonment and provide the reuired conditions for the optimal policy to be of switch type. The optimal decision is state-dependent, based on the number of chats in the ueue and the number of chats in service. ext, for the case with chat abandonment, we use a smoothed rate truncation method to numerically obtain the optimality of a switch type policy. We also evaluate the improvement in implementing this state-dependent reservation policy in comparison to the classical fixed threshold policy. From the theoretical results, we derive a set of managerial insights. We conclude that a fair routing is not necessarily a good routing in terms of the call center performance. Also, contrary to intuition, agent reservation may help to better manage congested situations. Another observation is that outbounds help in smoothing the demand. Finally, a state-dependent policy outperforms a fixed threshold one particularly in large call centers with highly impatient customers. Related literature. Let us now briefly review the literature related to this paper. There is a rich literature on the operations management in call centers. For a background on this literature, we refer the reader to two surveys by Gans et al and Akşin et al For a background on the specific context of multi-channel call centers, we refer the reader to Koole Although the growth of the use of chats, there are very few related papers in the literature. Tezcan and Zhang 2014 consider the objective of minimizing the staffing while providing a certain service level measured in terms of the proportion of customers who abandon the system in the long run. They propose effective routing policies based on a static planning linear programming, for both cases with observable or unobservable arrival rates. The modeling is however restricted to the case of decreasing service rates per chat and a fixed threshold for the maximum number of chats per agent. Luo and Zhang 2013 in turn fix the routing policy and focus on the staffing optimization by investigating a fluid limit approximation assuming infinitely patient customer chats. Again for the staffing problem, more effective solutions are provided in Cui and Tezcan 2016 using diffusion limits. A stream of literature related to this paper is also that of multitasking in ueueing systems. This literature is mainly related to multitasking in computer and telecommunication systems. The standard modeling is a processor sharing ueue where a single server can treat different tasks simultaneously. The framework in most of the existing studies is to use a measure-valued process. For instance, Gromoll et al use measure-valued descriptor for the analysis of processor sharing ueues that are not overloaded. Zhang et al propose a measurevalued fluid model and show that there exists a uniue associated fluid model solution. Further 4

5 references include Avi-Itzhak and Halfin 1988; Jean-Marie and Robert 1994; Puha et al The reminder of this paper is structured as follows. In Section 2, we describe the setting for the chat contact center problem. In Section 3, we focus on the optimization of the choice of the best agent to serve a chat. In Section 4, we optimize the service admission policy. In Section 5, we summarize the key operational findings of the article. Section 6 concludes the paper. 2 Setting We first describe the ueueing chat model. We then formulate the routing optimization problem of the system manager. Modeling. Consider a chat service system with inbound and outbound customer chats. Inbound chats inbounds arrive over time at an infinite capacity ueue according to a Poisson process with rate λ, while there is an infinite sufficiently large amount of stored outbound chats outbounds. Since inbounds reuire uasi-instantaneous answer, we assume a nonpreemptive priority for inbounds. Therefore, the possibility for an agent to initiate the service of an outbound is allowed only when the inbound ueue is empty. The system capacity consists of a homogeneous pool of s agents. Given the chat channel nature less reuired responsiveness than for calls, an agent has the ability to serve more than one customer simultaneously. An agent is said to be at level i if she is serving i customers i 0 at once. We assume that the service time of a customer inbound or outbound receiving service from an agent at level i is exponentially distributed with rate µ i, for i 1. The classical ordering assumption is µ 1 µ 2 µ K. It means that the more numerous tasks an agent is working on, the slower she would be on each task. In this paper, we do not restrict the modeling to a specific ordering of the service rates. For instance, the classical assumption is not appropriate for the agent behavior under rushing. It is observed in practice that a large number of simultaneous tasks may push the agents to be short and efficient in their responses so as to end the conversation as soon as possible. This corresponds also to real-time instructions given by the management as it is often the case for the voice channel. The service rates may therefore increase in the number of simultaneous chats. Also, they might be an increasing behavior due to a rushing effect, followed by a decreasing behavior due to loss of concentration or excessive workload Tezcan and Zhang,

6 Because of the text messaging interaction, a customer may abandon not only while waiting in the ueue, but also while being in service if she estimates that the interaction uality is bad. We assume that the abandonment time of inbounds from the ueue is exponentially distributed with rate γ, and that of inbounds or outbounds during the service is exponentially distributed with rate γ s. Since the conversation history is available, it is possible that a new agent takes over the remaining service without informing the customer. This flexibility together with the abandonment during service and the simultaneous treatment are the main specificities of the chat channel. They may allow, when appropriately used, to improve performance. This will be shown throughout the analysis of this paper. Routing optimization problem. The performance measures of interest in a chat system are the proportion of abandonments of inbounds from the ueue, denoted by P a, ; the proportion of abandonments of inbounds from service, denoted by P a,s ; the expected waiting time of inbounds in the ueue, denoted by EW ; and the expected service time. The system manager is interested in practice in finding the tradeoff between various conflicting objectives. First, as in a classical voice call center, the manager worries about the waiting time in the ueue of inbounds. Second, because of the work simultaneity particularity for chats, one may reduce the inbound ueue by assigning many waiting chats into service. A chat treatment duration can take too much time in such a case. It is then important to limit the service time duration per chat. Third, to avoid excessive abandonments from the ueue or service, the manager wants to maximize the number of treated inbounds and outbounds. Based on the above and denoting by λ S the expected rate of served inbounds and outbounds, the manager goal can be formulated as minimizing the following weighted cost summation, defined as M, λ λ S M = c 1 EW + c 2 ES + c 3, 1 λ where the coefficients c 1, c 2 and c 3 are the cost parameters that translate the relative importance between the three cost components of the objective function M. ote that without outbounds, the term proportional to c 3 is simply the proportion of abandonment from the ueue or from service, i.e., P a, + P a,s. With outbounds, however, this term and as a conseuence M can be negative. Our objective is to optimize the chat routing policy in order to minimize M. We consider 6

7 the set of all non-preemptive non-anticipating routing policies. The optimization problem consists of two uestions: i at any point in time we optimize the choice of the best agent to serve a chat, if any; ii at any point in time we optimize the chat service initiation, i.e., the choice of serving or not a chat. We allow for non-workconserving policies. This means that we may force a chat to stay in the ueue while an agent has the capacity to serve it. This is useful since a new customer entering service could harm the performance for the customers already in service. 3 Who is the best agent? In this section, we address the uestion of how to optimally distribute the chats in service among the agents. Consider an available chat to be routed to an agent. The most common routing rule, in practice as well as in the existing literature, is the Least Busy First LBF principle. It consists in euiprobably routing a chat to one of the agents currently serving the smallest number of chats. This rule is simple and fair between agents. Yet, it may not be the optimal policy to minimize M. It might even lead to bad performance in certain cases. Surprisingly, the opposite mechanism under the Most Busy First MBF policy could outperform LBF and could be even the optimal policy. In what follows we give a counterexample which shows that the classical LBF policy is not optimal. Counterexample to prove that LBF is not optimal. Consider a chat ueueing model with two agents where each agent has the ability to serve up to 3 chats, i.e., µ i = 0 for i 4. To simplify the analysis, we assume the same abandonment rate in the ueue and in service; γ s = γ = γ. Moreover, we do not allow for agent reservation for the initiation of outbound chats. We next compare between the performance of the two policies LBF and MBF in terms of the achieved value of M. A Markov chain approach is used to evaluate the analysis of the ueueing model. A state of the system is defined by the number of customers x in the system, x 0. For both policy cases, the transition rate from state x to state x + 1 is λ, from state x to state x 1 is 6µ 3 + xγ for x 6, from state 5 to state 4 is 2µ 2 + 3µ 3 + 5γ, and from state 1 to 0 is µ 1 + γ. Moreover under LBF, the transition rate from state 2 to state 1 is 2µ 1 + 2γ, from state 3 to state 2 is µ 1 + 2µ 2 + 3γ, and from state 4 to state 3 is 4µ 2 + 4γ, while under MBF the transition rate from state 2 to state 1 is 2µ 2 + 2γ, from state 3 to state 2 is 3µ 3 + 3γ and from state 4 to state 3 is 3µ 3 + µ 1 + 4γ. The Markov chains of these two models are depicted in Figures 2a 7

8 and 2b. From the transition rates, one may already observe that the most efficient routing rule to minimize M is not necessarily LBF. First, due to the rushing effect we could have µ 3 > µ 2 > µ 1, which makes the transition rates from state 2 to 1 and from 3 to 2 higher under MBF than LBF. Second, if µ 2 < 3µ 3+µ 1 4 which reflects a type of convexity, the transition rate from state 4 to 3 is higher under MBF than LBF. É É É É É É É É μ γ 2μ 2γ 2μ μ 3γ 4μ 4γ 3μ 2μ 5γ 6μ 6γ 6μ 7γ 6μ 8γ a LBF É É É É É É É É μ γ 2μ 2γ 3μ 3γ 3μ μ 4γ 3μ 2μ 5γ 6μ 6γ 6μ 7γ 6μ 8γ b MBF Figure 2: Markov chains Figure 3 illustrates the evolution of M as a function of µ 1. We observe that LBF is not always the best policy. LBF is more sensitive to the increasing of µ 1 than MBF, which makes the former worse than the latter for low values of µ 1. This finishes the counterexample description. edl ddl ddl >& D& ddl ddl ddl dl d d d d d d d d d d Figure 3: M as a function of µ 1 s = 2, µ 2 = 1.5, µ 3 = 1, λ = 2, γ = 0.5, c 1 = c 2 = 0, and c 3 = 1 Actually to minimize M, the optimal policy is not necessarily either MBF, or LBF. The 8

9 optimal rule is in general state-dependent and maximizes the average service rate per chat. By maximizing the average service rate per chat, the system also maximizes the flow from service which in turn minimizes M. Yet, for a given state defined by the number of chats in service, the optimal routing is often MBF or LBF. Proposition 1 and its corollary provide simple conditions under which LBF or MBF is optimal in a given state. We use a myopic approach to determine how the chats in service should be distributed among the agents to maximize the average service rate per chat. The optimization of the number of chats admitted in service will be addressed in Section 4. Proposition 1 Consider a state with k chats in service. For 1 x k, if the function µ x + xµ x is strictly increasing, then the optimal policy is MBF. If it is strictly decreasing, then the optimal policy is LBF. Corollary 1 For 1 x k, if the function µ x is strictly increasing and convex, then the optimal policy is MBF. If it is strictly decreasing and concave, then the optimal policy is LBF. Proof of the proposition and the corollary. The function µ x is defined on positive integers. Without loss of generality, we extend the support of this function to all positive real numbers by prolongation with the Lagrange interpolation polynomial. Consider k chats in service. We denote by x i the number of chats currently handled by agent i 0 x i k and 1 i s. We want to maximize x 1 µ x1 + x 2 µ x2 + + x s µ xs subject to the constraint x 1 + x x s = k. Let us define F x 1, x 2,, x s1 = x 1 µ x1 + x 2 µ x2 + + x s1 µ xs1 + k x 1 x 2 x s1 µ xs, where x 1, x 2,, x s1 [0, k]. First, we compute the critical points of F. We have F x i = µ xi + x i µ x i µ kx1 x 2 x s1 k x 1 x 2 x s1 µ kx 1 x 2 x s1, for 1 i s 1. Therefore, F x i = 0 is euivalent to µ xi µ xs + x i µ x i x s µ x s = 0, for i = 1, 2,, s 1. If µ x + xµ x is a strictly monotonous function in x for 0 x k, then µ x + xµ x = µ y + yµ y is euivalent to x = y. Thus, F has only one critical point given by x 1 = x 2 = = x s = k s, which corresponds to the LBF policy. 9

10 The Hessian matrix on the critical point is µ k/s + k s µ k/s The eigenvalues of this matrix are s 2µ k/s + k s µ k/s and 2µ k/s + k s µ k/s. If 2µ k/s + k s µ k/s > 0, then the critical point is a minimum and the best routing rule is MBF. If 2µ k/s + k s µ k/s < 0, then the critical point is a maximum and the best routing rule is LBF. This completes the proof of the proposition and its corollary. As mentioned above, the idea of the optimal choice is to maximize the total service rate with respect to a fixed number of chats in service. First, we derive the uniue critical point which corresponds to the LBF policy. So, LBF is always a relative extremum for the average service rate per chat. This relative extremum can be either a minimum or a maximum which makes LBF either a good or a bad policy. However, by relaxing the assumption that µ x + xµ x is strictly monotonous in x for 1 x k, the euation µ x + xµ x = µ y + yµ y may have other solutions than x = y. Therefore, other mechanisms different from LBF could lead to a critical point. By computing the eigenvalues on the other critical points, we can determine if another relative maximum of the average service rate per chat do exist. The choice for the optimal routing is then limited to the critical points LBF included which are relative maximums as well as MBF not a critical point in general. This reduces the number of computations compared to an exhaustive search on all the possible job routing possibilities. From now we assume that when k chats are admitted into service, they are distributed so as to maximize the service rate per chat. We denote by µ k the optimal service rate per chat when k chats are in service. The overall service rate is then kµ k. Since an agent cannot handle an infinite number of chats at a time, kµ k is bounded. This overall service rate is the so-called agents productivity. In the next section, we answer the routing optimization problem of serving a chat or keeping it in the ueue. 4 Should we initiate a chat service? Since the productivity of agents may reduce in the number of simultaneous tasks, it could be interesting to apply partial idling. We next apply a dynamic programming approach to 10

11 characterize the optimal routing policy. Recall that we assume that when a given number of chats are admitted into service, they are distributed so as to maximize the service rate per chat. Thus, one can see the number of chats in service as only one dimension of the problem. It is therefore euivalent to consider the multi-server setting as a single-server one. This possibility results from the particularity of the chat channel which allows for redistribution of jobs in service at any time among the agents. Our system is defined by the following states and actions. Let us denote by x x the number of chats in the ueue, and by y y the number of chats in service. For x, y 0, the transition rate from state x, y to x + 1, y is λ, the transition rate from state x+1, y to x, y is x+1γ, and when moving from state x, y +1 to x, y it is y + 1γ s + µ y+1 = y + 1d y+1, where d y denotes the departure rate per chat from an agent at level y, d y = µ y + γ s for y 1. The possible action for the agents team, just after a service completion, a chat arrival or an abandonment, is either to start the service of a chat, or not. The initiation is for an inbound chat if the inbound ueue is not empty, otherwise, the initiation is for an outbound chat. One major difficulty of the analysis is that the overall event rate is an unbounded function of actions and states because of abandonments. This does not therefore allow us to use the uniformization techniue which would transform our continuous time problem into a discrete one. The known standard solution is to use state space truncation limit the ueue size and the maximal number of tasks in service. State truncation has however the inconvenient to break the monotonicity properties of the value functions and performance measures. To overcome the limitations of this standard techniue, Bhulai et al propose a method that modifies the system rates by linearly smoothing them. The value of this approximation is that the convexity properties of the operators in Markov decision problems are maintained on the boundaries whereas they are not with a simple truncation. We divide the chat routing analysis into two parts. We first consider the case with no abandonments. We prove a set of structural properties from which we deduce the optimal policy. We also study the impact of the chat arrival rate on the parameters of the optimal policy. We then extend the analysis to the case with abandonments using the smoothed rate truncation method in order to compute the optimal policy. 11

12 4.1 Without abandonment We use the standard uniformization techniue Puterman, Without loss of generality, we assume that λ+maxyµ y = 1. We define the value function as follows: U 0 x, y = V 0 x, y = 0, V n x, 1 = 0 for x, n 0 and U n+1 x, y = c 1 λ x + c 2 λ y + λv nx + 1, y + yµ y V n x, y λ yµ y V n x, y 2 with V n+1 x, y = { min U n+1 0, y, U n+1 0, y + 1 c 3 λ if x = 0 minu n+1 x, y, U n+1 x 1, y + 1 if x > 0, for n, x, y 0. Without abandonment, the flow of served customer can be increased by initiating outbounds. This explains the term c 3 in the minimizing operator. The long-term average optimal actions can be obtained through value iteration, by recursively evaluating V n using Euation 2, for n 0. As n tends to, the minimizing actions converge to the optimal ones Puterman For x > 0, the minimizing action is chosen between keeping a chat in the ueue or starting the service of this chat. In general, the optimal policy depends on the form of the function µ y. Theorem 1 proves that the optimal policy is of switch type for increasing and concave cases of agent productivity. Theorem 1 If yµ y is increasing and concave in y, then the optimal policy is given by an increasing switching curve y = cx at or under which the optimal action is to serve a chat an inbound if x > 0, or an outbound if x = 0 and above which the optimal action is not to serve any chat. The proof in given in the appendix. The idea to obtain the long-run average optimal actions is to use the value iteration techniue introduced by Bellman 1957 and Howard 1960, by recursively evaluating V n using Euation 2, for n 0. In Theorem 1, we prove by induction on the value function that the optimal policy is of switch type. We prove that if some structural properties defining the switch type structure of the optimal policy are satisfied for V n, then these properties are satisfied for V n+1. They therefore hold for every n. As n tends to infinity, the optimal policy converges to the uniue average optimal policy, which is also of switch type. This convergence result is ensured by Theorem in Puterman 1994 since our problem satisfies the conditions of the theorem countable state set, finite set of actions and uniformizable system. The proof of convergence to the average optimal policy is an important 12

13 result in the MDP literature. It is based on showing that the iteration from V n to V n+1 is a contraction mapping, as stated in Theorem in Puterman This Theorem also proves that the optimal long-run policy is independent of the choice of V 0. More concretely, a switch type policy is characterized by the fact that if keeping a customer in the ueue is optimal in x, then keeping a customer is also optimal in x + 1. A sufficient condition for this is V n x, y + 1 V n x + 1, y 0 = V n x + 1, y + 1 V n x + 2, y 0. This condition is satisfied if V n x + 2, y + V n x, y + 1 V n x + 1, y + 1 V n x + 1, y 0. We now want to investigate the impact of the arrival rate on the reservation policy. Proposition 2 states that the reservation decreases with the arrival rate. In other words, at a state x, y x, y 0, if the optimal action is to keep a chat in the ueue for a given arrival rate λ 1, then with λ 2 λ 1 it is also optimal to keep a chat in the ueue. This is a confirmation of the observation made on the real data in the introduction. The more the system is congested, the more likely the decision would be to have more chats per agent. Proposition 2 Consider two identical situations except that the first has arrival rate λ 1 and the second has λ 2. If λ 1 λ 2, then the first situation reuires less reservation than the second one. The proof is given in the appendix. Similarly to Theorem 1, the proof is done by induction on n. The idea is to consider two systems with two different arrival rates and to prove that if it is optimal to serve a customer in a given state x, y in the system with the smallest arrival rate, then it is also optimal to serve a customer in state x, y in the other system. 4.2 With abandonment The previous definition of the value function does not allow to obtain a naturally uniformized system which would maintain the monotonicity properties to prove the optimal policy. We therefore use the smoothed rate truncation method to redefine the value function, as suggested in Bhulai et al The method proposes to truncate the number of states with a parameter and decrease the rate linearly in the number of states. Using this method, we can define 13

14 the relative value function associated to our model assuming λ + γ + γ s + µ max = 1 uniformization. We choose to formulate a 2-step value function. Let us denote by V n x, y the expected costs over n steps, for n, x, y 0. We have U n+1 x, y = c 1 λ x + c 2 λ y + + d y x d y + 1 λ x λ + V n x + 1, y + + y V n x, y 1 + γ s x γ s λ x λ + γ y γ γ y γ + c 3 y λ + y + x d y x d y + xvn x 1, y + c 3 λ 3 V n x, y, 4 with V n+1 x, y = { min U n+1 0, y, U n+1 0, y + 1 c 3 λ if x = 0 minu n+1 x, y, U n+1 x 1, y + 1 if x > 0, for n, x, y 0 and an arbitrary V 0 x, y for x, y 0. We choose V 0 x, y = U 0 x, y = 0 for x, y 0. With abandonment, the terms proportional with c 3 increase with abandonment from the ueue or from service and decrease with outbound initiation. All structural properties exhibited in Theorem 1 cannot be proven under a value iteration step. However, the main structural properties first and second order monotonicities in x and y and the supermodularity are maintained in the induction from U n to V n+1 when yd y is increasing and concave as shown in the appendix. We next numerically characterize the optimal chat routing policy. We consider the three following cases for the evolution of the service rates: Case 1: λ = 30, µ y = 4 0.1y for 1 y 21 and µ y = 42/y for y 21. Case 2: λ = 10, µ 1 = 0.1, µ 2 = 0.5, µ 3 = 0.9, µ 4 = 1.1, µ 5 = 1.2, µ 6 = 1.3, µ 7 = 1.4, µ y+8 = y for 0 y 15, and µ 15+y = 0 for y 0. Case 3: λ = 6, µ y = 0.1y for 1 y 8 and µ y = 0 for y 9. Case 1 corresponds to the condition of Theorem 1 where yµ y is increasing and concave. Case 2 corresponds to a case where the productivity of the agents team is first increasing then decreasing. Case 3 corresponds in turn to a case where the productivity is increasing and convex but limited to a capacity of 8 chats. For all cases, even for those where yd y is not increasing and concave Cases 2 and 3, the optimal policy is of switch type as illustrated in Figure 4. The chosen cases are likely to correspond to realistic situations for the evolution of the 14

15 service rate. Examples of service rate forms which do not lead to an increasing switching curve may exist. This corresponds for instance to the case where the productivity is decreasing, then increasing, then decreasing again. Such examples are however not likely to happen in real-life. dd dd dd dd dd dd e dd e d d d d dd dd dd dd dd d d d d dd dd dd dd dd a Case 1 b Case 2 dd dd dd e e d d d d d dd dd dd dd dd c Case 3 Figure 4: Optimal policy γ = γ s = 1, c 1 = 0.1, c 2 = 1 and c 3 = 0.01 We next compare between the performance of the optimal policy state-dependent with a fixed threshold policy simpler policy. In order to simplify the analysis, we do not consider the possibility of initiating outbounds. This will not change the form of the euations and the related interpretations. Let us first derive the closed-form expressions of the performance measures under the optimal routing policy. The switching curve of the optimal policy is defined by the parameters k 1, k 2,, k n where 0 k 1 < k 2 < < k n and n 1 n is the maximum number of chats allowed in service. When k chats are in the system: if k i < k k i+1 then i chats are in service and k i chats are in the ueue 1 i n 1, if k k 1 then k chats are in the ueue and no chats in service, and if k > k n then n chats are in service and k n chats are in the ueue. We use a continuous-time Markov chain approach. The system state, denoted by x, is defined by the number of chats in the system, x 0. The transition rate from x to x + 1 is λ, for x 0. For 0 < x k 1, the transition rate from x to x 1 is xγ. For k i < x k i+1 and 1 i n, the 15

16 transition rate from x to x 1 is i γ s + µ i + x iγ. For x > k n, the transition rate from x to x 1 is n γ s + µ n + x nγ. The stationary system state probabilities are denoted by p x. Using the convention that an empty product is one, and after some algebra, we obtain p 1 0 = + k 1 x=0 x=1 λ γ k 1 γ x n1 1 x! + k 1! n1 l=1 i=1 k l+1 k l j=1 k i+1 k i x=1 γ k 1 k 1! i1 l=1 k l+1 k l j=1 λ x+k n [k l + j lγ + lγ s + µ l ] λ x+k i [k l + j lγ + lγ s + µ l ] x [k i + j iγ + iγ s + µ i ] j=1. x [k n + j nγ + nγ s + µ n ] j=1 The last sum is infinite but can be rewritten using the incomplete gamma function, defined as Γz, y = p 1 0 = k 1 x=0 λ γ y 0 t z1 e t dz = z 1 y z e y 1 + y j, for z, y R. Therefore, j j=1 z + i x n1 1 x! + i=1 k i+1 k i x=1 γ k 1 k 1! i1 k l+1 k l l=1 j=1 k nn+n γ s+µn λ γ γ i=1 λ x+k i [k l + j lγ + lγ s + µ l ] λ kn k n n + n γ s+µ n γ e γ λ 1 + k 1! n1 k l+1 k l [k l + j lγ + lγ s + µ l ] γ k 1 l=1 j=1 Γ x [k i + j iγ + iγ s + µ i ] j=1 k n n + n γ s + µ n, λ γ γ. We now compute the expected number of customers in service denoted by EL S. We have n1 EL S = i i=1 k i+1 j=k i +1 p j + n n1 = p 0 i i=1 γ k 1 k 1! i1 + np 0 λkn l=1 j=k n+1 k l+1 k l j=1 p j k n n + n γ s+µ n γ γ k 1 k 1! n1 l=1 λ k i [k l + j lγ + lγ s + µ l ] k l+1 k l j=1 λ γ k n n+n γ s+µn γ k i+1 k i x=1 [k l + j lγ + lγ s + µ l ] e λ γ 1 λ x x [k i + j iγ + iγ s + µ i ] j=1 Γ k n n + n γ s + µ n, λ γ γ. Observing that jy j1 j j=1 x+i i=1 = x y + Γx, y 1 x y, the expected number of customers in 16

17 system, denoted by EL, may be written as EL = xp x x=1 k 1 λ = p 0 x x=1 + p 0λ k n + p 0λ kn λ γ x 1 x! + p 0 n1k i+1 k i i=1 x=1 γ k 1 k i1 1! l=1 k l+1 k l j=1 [ k n Γ k n n + n γ s+µ n γ, λ γ k n n + n γs+µn knn+n γ s+µn γ λ γ γ k 1 γ k 1 + Γ k n1 1! l=1 k n1 1! l=1 k l+1 k l j=1 k n n + n γ s+µ n γ k l+1 k l j=1 x + k i λ x+k i [k l + j lγ + lγ s + µ l ] γ j=1 k n n+n γs+µn λ γ γ [k l + j lγ + lγ s + µ l ], λ γ 1 knn+n γs+µn γ λ γ [k l + j lγ + lγ s + µ l ]. x [k i + j iγ + iγ s + µ i] e λ γ 1 ] The expected number in the ueue can be then computed as EL = ELEL S. Using Little law, the expected waiting time in the ueue is EW = 1 λ EL. The proportion of chats which abandon from the ueue is P a, = γ λ EL. The expected service time is ES = EL S λ1p a,. Finally, the proportion of chats which abandon from service is P a,s = γs λ EL S. The performance measure M = c 1 EW + c 2 ES + c 3 P a, + P a,s. For the numerical experiments, we choose µ i = 1/ i for 1 i 10 and µ i = 0 for i > 10. The least busy first routing rule is then optimal. Also, this allows us to have yd y increasing and concave in y for 1 y 10 s, and limits the number of chats per agent to 10. For a coherent comparison, the ratio ρ i = λ/sµ i is held constant, for 1 i 10. We want to minimize M with c 1 = c 2 = 0, c 3 = 1. This is euivalent to minimize the overall proportion of abandonment, M = P a, + P a,s. In Table 1, we give the performance measures under both the optimal fixed threshold policy threshold k and the optimal state-dependent policy. The last column gives the relative difference, defined as rd = Mstate-dependentMfixed Mfixed. We observe as expected that the state-dependent policy outperforms. This is particularly apparent for large call centers. There are two reasons for this observation. First, because of pooling, large call centers can afford high reservation levels, and reservation in turn allows for less abandonments from service. The second reason is related to the more significant concavity 17

18 Table 1: Performance comparison Optimal fixed threshold Optimal state-dependent λ γ γ s s k P a, P a,s M P a, P a,s M rd % 16.65% 16.72% 0.07% 16.65% 16.72% 0.00% % 16.65% 16.66% 0.01% 16.65% 16.66% 0.00% % 2.19% 2.60% 0.41% 2.19% 2.60% 0.00% % 16.18% 16.19% 0.00% 13.24% 13.24% % % 14.57% 15.68% 0.00% 11.09% 11.09% % % 2.17% 2.17% 0.00% 2.16% 2.16% -0.36% % 14.66% 15.37% 0.00% 9.41% 9.41% % % 16.56% 14.74% 0.00% 6.84% 6.84% % % 2.17% 2.17% 0.00% 2.15% 2.15% -0.85% of the function yµ y in y as s increases 1 y s 10. We have yµ y = y for 1 y 10 and s = 1, yµ y = yi 1 y y I 11 y j + 1 j + 1y + 10j + 1 j j j + 1I 10j+1 y 10j y I 91 y 100 for 1 y 100 and s = 10, yµ y = yi 1 y y I 101 y y I 901 y 1000 for 1 y 1000 and s = 100, where I A is the indicator function of a given subset A. Recall that as shown in the proof of Theorem 1, the concavity in y of yµ y encourages a strong reservation strategy. 5 Operational findings The theoretical results and numerical experiments presented in this article enable us to derive a series of insights. The insights derived below aim at assessing key managerial messages that may help contact center managers to enhance chat routing in practice. Observation 1 A fair routing is not necessarily a good routing. The observation directly follows from the comparison between MBF and LBF in Section 3. This is an important insight as it shows that the commonly used LBF principle may not lead to optimized performance measures. In particular, if the the service rate per chat is increasing and convex, then the optimal policy is MBF. In other words, if a rushing effect is observed agents are more efficient when they have more work to do, then it may be better not to be fair in the work distribution among the agents. 18

19 Observation 2 Reservation may help to better manage congested situations. Contrary to intuition, it may be counterproductive to serve a customer whenever an agent could do it work-conserving system. The reason is related to two phenomenons. First, increasing the number of chats in service may reduce the agents productivity. Second, the abandonment from the ueue may help to reduce the congestion of the contact center. Observation 3 Chat initiation outbound chats may be used as a tool to manage low inbound demand situations. The contact center performance decreases when the arrival rate of inbound calls decreases. One way to avoid this difficulty together with the reduction of agents idleness is to let agents initiate outbound chats. This strategy can be developed during off-peaks periods when agents are sufficiently available. Observation 4 A state-dependent policy outperforms a fixed threshold policy in large contact centers with abandonment. This observation is derived from the numerical experiments in Table 1. Although the idea to have a fixed threshold in the maximal number of chats per agent is attractive due to its simplicity, it may lead to a very bad performance. This is particularly more apparent in large call centers with highly impatient customers. The obtained results in Table 1 argue for developing routing where decisions are adjusted to the system state. 6 Concluding remarks We demonstrated the operational advantages of agents reservation in a chat contact center. For a setting including both inbound and outbound chats, we gave reuired conditions under which the LBF and MBF routing rules are optimal. We also showed that the optimal policy for agent reservation is of switch type based on the number of chats in the ueue and in service. We evaluated the improvement in implementing this state-dependent reservation policy in comparison to a fixed threshold policy. A number of managerial insights are derived from the theoretical results. For instance, a fair routing is not necessarily a good routing in terms of the call center performance. Also, a state-dependent policy outperforms a fixed threshold one especially for the setting of large call centers with highly impatient customers. 19

20 Several interesting areas of future research arise. It would be interesting to uestion the exponential assumption for service times. Since the chat conversation is a succession of answer and uestions, an Erlang or a Coxian distribution may more appropriately model the service duration. Another challenging and interesting direction is to develop and analyze a framework that combines chats and other channels such as calls or s. Also, applications where switching from textual chats to video chats is possible makes an agent no longer anonymous. It is therefore interesting to study the effect of this technology on performance. References Akşin, O., Armony, M., and Mehrotra, V The modern call-center: A multi-disciplinary perspective on operations management research. Production and Operations Management, 16: Avi-Itzhak, B. and Halfin, S Expected response times in a non-symmetric time sharing ueue with a limited number of service positions. Proceedings of ITC, 125.4:1 2. Bellman, R Dynamic programming. Princeton University Press, Princeton. Bhulai, S., Brooms, A., and Spieksma, F On structural properties of the value function for an unbounded jump Markov process with an application to a processor sharing retrial ueue. Queueing Systems, pages Cui, L. and Tezcan, T Approximations for chat service systems using many-server diffusion limits. Mathematics of Operations Research. Gans,., Koole, G., and Mandelbaum, A Telephone call centers: Tutorial, review, and research prospects. Manufacturing & Service Operations Management, 5: Gromoll, H. C., Robert, P., and Zwart, B Fluid limits for processor-sharing ueues with impatience. Mathematics of Operations Research, 332: Howard, R. A Dynamic programming and markov processes.. ICMI Extreme engagement in the multichannel contact center: Leveraging the emerging channels research Report and best practices guide. ICMI Research Report. Jean-Marie, A. and Robert, P On the transient behavior of the processor sharing ueue. Queueing Systems, 171-2: Koole, G Call Center Optimization. MG Books. Luo, J. and Zhang, J Staffing and control of instant messaging contact centers. Operations Research, 612: Puha, A. L., Stolyar, A. L., and Williams, R. J The fluid limit of an overloaded processor sharing ueue. Mathematics of Operations Research, 312: Puterman, M Markov Decision Processes. John Wiley and Sons. 20

21 Tan, T. F. and etessine, S When does the devil make work? an empirical study of the impact of workload on worker productivity. Management Science. Tezcan, T. and Zhang, J Routing and staffing in customer service chat systems with impatient customers. Operations Research, 624: Zhang, J., Dai, J., and Zwart, B Law of large number limits of limited processorsharing ueues. Mathematics of Operations Research, 344: Proof of Theorem 1 We define a class of functions F from 2 to R as follows: f F if for x, y 0, we have fx + 1, y fx, y, fx, y + 1 fx, y, 5 6 fx, y fx, y 2fx, y + 1, 7 fx, y + fx + 1, y + 1 fx + 1, y + fx, y + 1, 8 fx + 2, y + fx, y + 1 fx + 1, y + fx + 1, y y + 2µ y+2 yµ y fx + 2, y + fx, y y + 1µ y+1 yµ y fx + 1, y + 2 y + 2µ y+2 y + 1µ y+1 fx + 1, y + 1 fx, y fx + 2, y 2fx + 1, y Relations 5 and 6 define a class of increasing functions in x and in y. Relation 7 defines the convexity in y. Relation 8 is known as supermodularity. ote that while summing up Relations 8 and 9 we obtain fx, y+fx+2, y 2fx+1, y. Thus, if f F, it implies that f is convex in x and convex in y. Relation 9 defines that the function fx, y + 1 fx + 1, y is decreasing in x. To simplify the writing, we refer to as serve the decision action to serve a chat and by keep the decision action to keep a chat in the ueue. To prove that an increasing switching curve policy is optimal, we only have to show that Relation 9 or Relation 11 is true for U n, n 0. If Relation 9 is true for U n, then U n x + 2, y U n x + 1, y + 1 U n x + 1, y U n x, y + 1 for x, y 0. Thus, if U n x + 1, y U n x, y + 1 0, then U n x + 2, y U n x + 1, y for x, y 0. Conseuently, if serve is optimal in x + 1, y, then serve is also optimal in x + 2, y, and vice versa, if keep is optimal in x + 2, y, then keep is also optimal in x+1, y. Thus, Relation 9 for U n n 0 suffices to prove that a switching curve is optimal. 21

22 If Relation 11 is true for U n, then U n x+2, yu n x+1, y+1 U n x+1, y+1u n x, y+2 for x, y 0. Thus, if U n x+1, y +1U n x, y +2 0, then U n x+2, yu n x+1, y +1 0 for x, y 0. Conseuently, if serve is optimal in x + 1, y + 1, then serve is also optimal in x + 2, y, and vice versa, if keep is optimal in x + 2, y, then keep is also optimal in x + 1, y + 1. Thus, Relation 11 for U n n 0 also suffices to prove that a switching curve is optimal. ote that either Relation 9 or 11 proves that the switching curve is increasing. If the switching curve would not be, then there would exist a state x, y for which the optimal action is to serve a chat and a state x + 1, y for which the optimal action is to keep a chat. These decision actions do not follow Relation 9. In what follows we prove by induction on n that both V n and U n are in F. We divide the proof into the following 3 steps: Step 1. We show that V 0, U 0 F. Step 2. We prove that if for a given n, U n F then V n F. Step 3. We prove that if for a given n, V n F then U n+1 F. Step 1. For x, y 0, V 0 x, y = U 0 x, y = 0, then V 0, U 0 F. Step 2. Assume that for a given n 0, U n F. We only consider the cases where x > 0 and y 0. In the cases where x = 0 the proof arguments are identical. Relations 5 and 6: We have V n x, y U n x, y 12 V n x, y U n x 1, y If keep is optimal in x + 1, y, then V n x + 1, y = U n x + 1, y. Combining Ineuality 12 with Relation 5 for U n proves Relation 5 for V n. If serve is optimal in x + 1, y, then V n x + 1, y = U n x, y + 1. Combining Ineuality 13 with Relation 5 for U n proves Relation 5. Thus, relation 5 is true for V n. The proof of relation 6 for V n is identical. 22

23 Relations 7: We have 2V n x, y + 1 2U n x, y V n x, y + 1 2U n x 1, y V n x, y + 1 U n x, y U n x 1, y If keep is the optimal action at states x, y and x, y + 2, then V n x, y + V n x, y + 2 = U n x, y+u n x, y+2. Thus, combining Relation 7 for U n and Ineuality 14 proves Relation 7 for V n. If serve is the optimal action at states x, y and x, y + 2, then V n x, y + V n x, y + 2 = U n x 1, y U n x 1, y + 3. Thus, combining Relation 7 for U n and Ineuality 15 proves Relation 7 for V n. If serve is the optimal action at state x, y and keep is the optimal one at state x, y + 2, then V n x, y + V n x, y + 2 = U n x 1, y U n x, y + 2. Thus, combining Relation 8 supermodularity for U n and Ineuality 16 proves Relation 7 for V n. The case where keep would be the optimal action at state x, y and serve would be the optimal action at state x, y + 2 cannot be considered because it is in contradiction with the fact that the switching curve is increasing due to Relation 9. In all cases Relation 7 is true for V n. Relation 8: V n x + 1, y + V n x, y + 1 U n x + 1, y + U n x, y V n x + 1, y + V n x, y + 1 2U n x, y V n x + 1, y + V n x, y + 1 U n x, y U n x 1, y If keep is the optimal action at states x, y and x+1, y+1, then V n x, y+v n x+1, y+1 = U n x, y + U n x + 1, y + 1. Thus, combining Relation 8 for U n and Ineuality 17 proves Relation 8 for V n for x, y 0. If keep is the optimal action at state x, y and serve is the optimal action at state x + 1, y + 1, then V n x, y + V n x + 1, y + 1 = U n x, y + U n x, y + 2. Using the convexity of U n in y in Ineuality 18 for U n, we prove Relation 8 for V n. If serve is the optimal action at state x, y and keep is the optimal action at state x + 1, y + 1, then V n x, y + V n x + 1, y + 1 = U n x 1, y U n x + 1, y + 1. Using the 23

24 convexity of U n in x in Ineuality 18, we prove Relation 8 for V n, for 0 x s 1, y > 0. If serve is the optimal action at states x, y and x+1, y+1, then V n x, y+v n x+1, y+1 = U n x 1, y U n x, y + 2. Thus, combining Relation 8 for U n and Ineuality 19 proves Relation 8 for V n. In all cases, Relation 8 holds for V n. Relation 9: We have V n x + 1, y + V n x + 1, y + 1 U n x + 1, y + U n x + 1, y V n x + 1, y + V n x + 1, y + 1 U n x, y U n x + 1, y V n x + 1, y + V n x + 1, y + 1 U n x, y U n x, y If keep is the optimal action at states x+2, y and x, y+1, then V n x+2, y+v n x, y+1 = U n x + 2, y + U n x, y + 1. Thus, combining Relation 9 for U n and Ineuality 20 proves Relation 9 for V n. If serve is the optimal action at state x + 2, y and keep is the optimal action at state x, y + 1, then V n x + 2, y + V n x, y + 1 = U n x + 1, y U n x, y + 1. Ineuality 21 proves Relation 9 for V n. If serve is the optimal action at states x+2, y and x, y+1, then V n x+2, y+v n x, y+1 = U n x + 3, y 1 + U n x + 1, y 1. Thus, combining Relation 9 for U n and Ineuality 22 proves Relation 9. The case where keep would be the optimal action at state x + 2, y and serve would be the optimal action at state x, y + 1 cannot be considered, because it is in contradiction with Relations 9 and 11. If serve is the optimal action at state x, y + 1, then with Relation 9 serve is also the optimal action at state x + 1, y + 1. If keep is the optimal action at state x+2, y, then from Relation 11 keep is also the optimal action at state x+1, y +1. Keep and serve cannot be both optimal, therefore this case should not be considered. In all cases, Relation 9 is true for V n. Relation 10: ote that proving Relation 10 is similar to that of Relation 9. The only difference is when serve is optimal in x + 2, y. Assume serve is the optimal action at state x + 2, y and keep is the optimal action at 24