Equilibrium customer strategies in a single server Markovian queue with setup times

Transcription

1 Equilibrium customer strategies in a single server Markovian queue with setup times Apostolos Burnetas and Antonis Economou {aburnetas,aeconom}@math.uoa.gr Department of Mathematics, University of Athens Panepistemioupolis, Athens 5784, Greece Published in 2007 in Queueing Systems 56, The original publication is available at DOI 0.007/s Abstract. We consider a single server Markovian queue with setup times. Whenever this system becomes empty, the server is turned off. Whenever a customer arrives to an empty system, the server begins an exponential setup time to start service again. We assume that arriving customers decide whether to enter the system or balk based on a natural reward-cost structure, which incorporates their desire for service as well as their unwillingness to wait. We examine customer behavior under various levels of information regarding the system state. Specifically, before making the decision, a customer may or may not know the state of the server and/or the number of present customers. We derive equilibrium strategies for the customers under the various levels of information and analyze the stationary behavior of the system under these strategies. We also illustrate further effects of the information level on the equilibrium behavior via numerical experiments. Keywords: queueing, setup times, vacations, balking, continuous time Markov chain, equilibrium strategies, individual optimization, pricing, stationary distribution, difference equations, matrix analytic methods. Introduction Queues with removable servers also referred to as queueing systems with vacations deal with situations where the servers may be unavailable for serving customers over some intervals of time. Such situations are often incurred in real applications. For example, a server may be deactivated for economic reasons low traffic intensity and/or high stand-by costs, suffer random failures, go under preventive maintenance or attend to a secondary system. Due to their versatility and applicability, queueing systems with removable servers have been extensively studied. Detailed surveys are contained in Takagi 99 and Tian and Zhang Although the literature for the single server case is very rich, for the multiserver case the number of published papers is limited. A classification of most of them is presented in Artalejo and Lopez-Herrero A significant part of this literature is dedicated to the analysis of queues with setup times. In such models once a server is reactivated, a generally random time is required for setup before it can begin serving customers. Bischof 200, Choudhury 998, 2000, He and Jewkes 995 and the references therein consider various single server systems with setup times, while Borthakur and Choudhury 999 and Artalejo et al deal with some multiserver models. In particular, the performance evaluation for an M/M/ queue with server setups has been

2 carried in Adan and van der Wal 998,p.5-54 and its multiserver counterpart in Artalejo et al In the queueing literature there is also an emerging tendency to study systems from an economic viewpoint. More concretely, it is assumed that there exists a reward-cost structure for the customers of a given system, incorporating their desire for service as well as their dislike for waiting. The customers are allowed to make decisions as to whether to join or balk, buy priority or not etc. This can be viewed as a game among the customers. The basic problem is to find individual and social optimal strategies. These ideas go back at least to the pioneering works of Naor 969 and Edelson and Hildebrand 975 who studied equilibrium and socially optimal strategies for whether to join or balk in an M/M/ queue with a simple reward-cost structure. Naor 969 studied the case where each customer observes the queue length before his decision, while Edelson and Hildebrand 975 considered the unobservable case. These results were further refined and extended by several authors, see e.g. Yechiali 97, Hassin 986 and Chen and Frank 200, Moreover, several authors have investigated the same problem for various queueing systems incorporating many diverse characteristics such as priorities, reneging and jockeying, schedules and retrials etc. The fundamental results in this area with extensive bibliographical references can be found in the comprehensive monograph of Hassin and Haviv In the present paper we investigate the equilibrium customer behavior in a single server Markovian queue with setup times. The customers dilemma is whether to join the system or balk. We examine various cases with respect to the level of information available to customers before they make this decision. More specifically, at his arrival epoch a customer may or may not know the state of the server and/or the number of customers present. Therefore, four combinations emerge, ranging from no to full information. In each of the four cases we characterize customer equilibrium strategies, analyze the stationary behavior of the corresponding system and derive the social benefit for all customers. We also explore the effect of the information level on the equilibrium behavior and the social benefit via analytical and numerical comparisons. When no information is available to arriving customers, the stationary analysis turns out to be equivalent to that of the M/M/ system with server setups and no customer decisions, performed in Adan and van der Wal 998. However, in all other cases, to derive the stationary behavior this analysis must be generalized in different directions. To our knowledge, these generalizations are new. The paper is organized as follows. In Section 2, we describe the dynamics of the model, the reward-cost structure and the decision assumptions for the customers information levels. In Section 3 we consider the case where the customers know the number of customers in system before they decide whether to join or to balk. We distinguish two subcases depending on the additional information, or lack thereof, of the server state. We determine equilibrium threshold strategies and investigate the resulting stationary system behavior. In Section 4 we consider the unobservable case. We also consider the same two subcases, derive the corresponding mixed equilibrium strategies, and analyze the stationary behavior. Finally, in Section 5, we present several numerical experiments that demonstrate the effect of the information level to the various performance measures. 2. Model description We consider a single server queueing system with infinite waiting room in which customers arrive according to a Poisson process with rate. The service times of the customers are assumed to be exponentially distributed random variables with rate. The server is deactivated as soon as the queue becomes empty. When a new customer arrives at an empty system, a setup process 2

3 0, 0,, 0 2, 2, 0 n, n, 0 n +, n +, 0 Figure : Transition rate diagram of the original model starts for the server to be reactivated. The time required for setup is also exponentially distributed with rate. During the setup customers continue to arrive. We assume that interarrival times, service times and setup times are mutually independent. We represent the state at time t by the pair Nt, It, where Nt denotes the number of customers in the system and It denotes the state of the server. It is clear that the process {Nt, It : t 0} is a continuous time Markov chain with state space S = {n, i 0 i, n i}. The transition diagram is shown in Figure. In Figure we observe that starting from a state n,, n it is not possible to make transitions to states m, 0 without visiting state 0, 0 first. Due to this special structure, it is possible to compute explicitly the stationary distribution and several performance measures under various customer strategies. We are interested in the behavior of customers when they can decide whether to join or balk upon their arrival. To model the decision process, we assume that every customer receives a reward of R units for completing service. This may reflect his satisfaction and/or the added value of being served. On the other hand there exists a waiting cost of C units per time unit that the customer remains in the system in queue or in service. Customers are risk neutral and maximize their expected net benefit. We assume that R > C + C. This condition ensures that the reward for service exceeds the expected cost for a customer who finds the system empty. Otherwise, after the system becomes empty for the first time no customers will ever enter. Finally, the decisions are irrevocable: retrials of balking customers and reneging of entering customers are not allowed. Under the above framework we can think of the situation as a symmetric game among the customers since they are all indistinguishable. Denote the common set of strategies and the payoff function by S and F, respectively. More concretely let F a, b be the payoff of a customer that selects strategy a when everyone else selects strategy b. A strategy s e is a symmetric Nash equilibrium if F s e, s e F s, s e, for every s S. The intuition is that an equilibrium strategy is a best response against itself, i.e., if all customers agree to follow it no one can benefit by changing it. A strategy s is said to dominate strategy s 2 if F s, s F s 2, s, for every s S and for at least one s the inequality is strict. A strategy s is said to be weakly dominant if it dominates all other strategies in S. The notion of dominance is substantially stronger than that of equilibrium. In fact, while an equilibrium strategy exists in almost all situations, a weakly dominant strategy rarely does. In the next two sections we obtain equilibrium customer strategies for joining/balking. We distinguish four cases depending on the information available to customers at their arrival instant, before the decision is taken: Fully observable case: Customers observe Nt, It 3

4 Almost observable case: Customers observe only Nt Almost unobservable case: Customers observe only It Fully unobservable case: Customer do not observe the system state For the terminology, note that the information Nt = n, for n, implies that the system is in one of the two states {n, 0 and n, } or the single state {0, 0}, for n = 0}. On the other hand, the information It = i, for i = 0,, implies that the state belongs to the infinite set {n, i, n +, i,...}. It is for this reason that we refer to these cases as almost observable and almost unobservable, respectively. It must be emphasized that by employing this terminology, we don t imply that it is more critical to have information about Nt than about It. This is not true in general. For example in a system with very low setup rate and very high service rate it is intuitively expected that observing It is more important than observing Nt. We further discuss the value of information under the light of the numerical results in Section 5. From a methodological point of view, the observable cases are similar and are treated in parallel in Section 3. The unobservable cases are both treated in Section Equilibrium threshold strategies for the observable cases For the fully observable and almost observable cases we show that there exist equilibrium strategies of threshold type. Specifically there exist threshold levels such that an arriving customer enters the system if the number of customers present upon arrival does not exceed the specified thresholds. We begin with the fully observable case in which customers know the exact state of the system n, i upon arrival. A customer who joins the system when he observes state n, i has mean sojourn time equal to n+, thus his expected net benefit is + i R Cn + C i. Such a customer strictly prefers to enter if this value is positive and is indifferent between entering and balking if it equals zero. Assuming that customers break ties in favor of entering, the tagged customer enters if and only if n + R C i. We thus conclude the following. Theorem In the fully observable M/M/ queue with setup times there exist thresholds R n e 0, n e = C R,, 2 C such that the strategy observe Nt, It, enter if Nt n e It and balk otherwise is a unique equilibrium in the class of threshold strategies. Moreover, it is also a weakly dominant strategy. For the stationary analysis, note that if all customers follow the threshold strategy in 2 the system follows a Markov chain similar to that in Figure, with state space restricted to S fo = {n, 0 0 n n e 0 + } {n, n n e + } and identical transition rates. The transition diagram is depicted in Figure 2. 4

5 0, 0, 2, n e 0, n e 0 +, n e,, 0 2, 0 n e 0, 0 n e 0 +, 0 n e +, Figure 2: Transition rate diagram for the n e 0, n e threshold strategy The corresponding stationary distibution p fo n, i : n, i S fo is obtained as the unique positive normalized solution of the following system of balance equations. Define p0, 0 = p, 3 pn, 0 + = pn, 0, n =, 2,..., n e 0 4 pn e 0 +, 0 = pn e 0, 0 5 p, + = p, 0 + p2, 6 pn, + = pn, + pn, 0 + pn +,, n = 2, 3,..., n e 0 7 pn e 0 +, = pn e 0, + pn e 0 +, 0 8 pn, = pn,, n = n e 0 + 2,..., n e + 9 ρ =, σ = +. By iterating 4 and 9, taking into account 3 and 5, we obtain pn, 0 = σn p,, n = 0,,..., n e 0 0 pn e 0 +, 0 = σn e0 p, pn, = ρ n n e0 pn e 0 +,, n = n e 0 + 2,..., n e +. 2 From 7 it follows that pn, : n =, 2,..., n e 0 + is a solution of the nonhomogeneous linear difference equation with constant coefficients x n+ + x n + x n = pn, 0 = σn p,, n = 2, 3,..., n e 0, 3 where the last equation is due to 0. Using the standard approach for solving such equations, see e.g. Elaydi 999,p we consider the corresponding characteristic equation x 2 + x + = 0, which has two roots at and ρ. Then the general solution of the homogeneous version of 3 is x hom n = A n + B ρ n if ρ or x hom n = A n + B n n if ρ =. The general solution x gen n of 3 is given as x gen n = x hom n + x spec n, where x spec n is a specific solution of 3. Because the nonhomogeneous part σn p, of 3 is geometric with parameter σ, we can again use the standard approach to find a specific solution. To this end, we consider specific solutions of the form Cσ n if σ, ρ, Cnσ n if σ = or σ = ρ and ρ or Cn 2 σ n if σ = = ρ. For brevity in the exposition, we assume hereafter that σ ρ and both are different from, which 5

6 is the regular case. The other cases can be viewed as singular and the corresponding results can easily be derived by taking appropriate limits of the regular case results see Remark 4 below. For the regular case, substituting x n = Cσ n in 3 we obtain C = Hence, the general solution of 3 is given as + p,. 4 x gen n = A n + B ρ n + C σ n, n =, 2,..., n e where C is given by 4 and A, B are to be determined. From 5, for n = it follows that A + ρb = + p,. 6 Furthermore, substituting 5 in 6 it follows after some rather tedious algebra that A + ρ 2 + ρ B = p,. 7 Solving the system of 6 and 7 we obtain A = 0 and B = + p,, thus, from 5, + pn, = σn ρ n p,, n =, 2,..., n e We have thus expressed all stationary probabilities in terms of p,, in relations see 0-2 and 8. The remaining probability, p,, can be found from the normalization equation n e 0+ n=0 n e + pn, 0 + n= pn, =. After some algebraic simplifications, we can reduce the stationary probabilities in terms of ρ and σ and summarize the results in the following. Proposition Consider an M/M/ queue with setup times and σ ρ σ, in which the customers follow the threshold policy n e 0, n e. The stationary probabilities p fo n, i : n, i S fo are as follows p fo, = ρ σ ρ σ ρ σne0+ ] + ρσ ρ ρn e+2 σ ne 0+ ρ 9 p fo n, 0 = ρ σn p fo,, n = 0,,..., n e 0 20 p fo n e 0 +, 0 = p fo n, = p fo n, = ρ σ σne0+ p fo, 2 σ ρ σn ρ n p fo,, n =, 2,..., n e 0 22 σ ρ σne0+ ρ ne0+ ρ n ne0 p fo,, n = n e 0 +,..., n e

7 Because of the PASTA property, the probability that an arrival finds the system at state n e 0 +, 0 or n e +, and, therefore, balks, is equal to p fo n e 0 +, 0 + p fo n e +,. Hence the social benefit per time unit when all customers follow the threshold policy n e 0, n e equals S fo = R p fo n e 0 +, 0 p fo n e +, n e 0+ n e + C np fo n, 0 + np fo n,. 24 n=0 Remark. Proposition holds for the stationary distribution corresponding to any threshold policy n e 0, n e and not only to the individually optimal policy specified by 2. n= Remark 2. There is no need to solve from scratch the difference equations for the singular cases, when the conditions σ ρ σ do not all hold. The stationary probabilities in the cases can be found by considering limits in the formulae of Proposition 2. For example, assume that σ = ρ and we want to compute p fo n,, for n =, 2,..., n e 0. We can take limits in 22 as σ ρ and obtain p fo n, = nρ n p fo,, n =, 2,..., n e 0. Remark 3. Equation 24 can be considerably simplified, as the related sums involve only geometric series terms and can be expressed in closed form. We do not elaborate further on this point, as we only use S fo in numerical experiments. We next consider the almost observable case, where arriving customers only observe the number of customers in the system. Then, the mean sojourn time of a customer who finds n customers in the system is n+, where π I N 0 n is the probability that an arriving customer finds the server at state 0 inactive, given that there are n customers. Therefore, the expected benefit of such a customer, if he decides to enter, is equal to + π I N 0 n R Cn + Cπ I N 0 n. 25 We seek equilibrium strategies of threshold type. Hence, we must compute π I N 0 n, when all customers follow the same threshold strategy. Assume that all customers use the same threshold for entrance n e. The stationary distribution of the corresponding Markov chain is from Proposition 2 with n e 0 = n e = n e. Thus, the embedded probabilities π I N 0 n are equal to π I N 0 n = pn, 0 pn, 0 + pn, {n }, n = 0,,..., n e +, where {n } is the indicator function of the set {, 2,...}. Using the various forms of pn, i from 9-23 we obtain π I N 0 n = + + π I N 0 n e + = = ρ n ], n = 0,,... n e, 26 σ ρ ne + ] σ ρ σ ne+ ]. 27

8 In light of we introduce the function Cn + fx, n = R C + x + ρ n ], σ x 0, ], n = 0,, 2, which will allow us to prove the existence of equilibrium threshold strategies and derive the corresponding thresholds. Let f U n = f, n, f L n = f0, n, n = 0,, 2, It is easy to see that f U 0 = f L 0 = R C C > 0 because of. In addition, lim n f U n = lim n f L n =. Hence, there exists n U such that f U 0, f U,..., f U n U > 0 and f U n U The function fx, n is clearly increasing with respect to x for every fixed n, thus, f L n f U n, n = 0,, 2,.... In particular, f L n U + 0, while f L 0 > 0. Hence there exists n L n U such that f L n L > 0 and f L n L +,..., f L n U, f L n U We can now establish the existence of equilibrium threshold policies in the almost observable case. We have the following. Theorem 2 In the almost observable M/M/ queue with setup times all pure threshold strategies observe Nt, enter if Nt n e and balk otherwise, for n e = n L, n L +,..., n U, are equilibrium strategies. Proof. Consider a tagged customer at his arrival instant and assume that all other customers follow the same threshold strategy observe Nt, enter if Nt n e and balk otherwise for some fixed n e {n L, n L +,..., n U }. Then, π I N 0 n is given by If the tagged customer finds n n e customers and decides to enter, his expected net benefit is equal to Cn + R C + + ρ n ] = f U n > 0, σ because of 25,26,28,29 and 30. So in this case the customer prefers to enter. If the tagged customer finds n = n e + customers and decides to enter, his expected net benefit is R Cn e + 2 C ρ ] ne + + = f L n e + 0, σ because of 25,27,28,29 and 3. So in this case the customer prefers to balk. Remark 4. When n L < n U, there exist multiple equilibrium threshold strategies. This is observed generally in Follow-The-Crowd situations, where in equilibrium customers tend to adopt the behavior of other customers. In models where the space of strategies can be ordered e.g., when strategies can be parametrized by a single number we have a Follow-The-Crowd FTC resp., Avoid-The-Crowd ATC situation when one s optimal response to a strategy x adopted by all others is increasing resp., decreasing in x see e.g. Hassin and Haviv 997, and Hassin and Haviv 2003, p. 6. In the framework of our model, the first part R Cn+ 8

9 0, 0, 2, n e,, 0 q e n e +, q e 2, 0 n e, 0 n e +, 0 Figure 3: Transition rate diagram for the n e, q e mixed threshold strategy of the expression 25, for the expected benefit of a tagged customer who decides to enter when he observes n customers, depends only on the number of customers that are present in front of him and not on the policy of the other customers. On the other hand, the part Cπ I/N 0 n which depends on the policy adopted by the other customers is easily seen to decrease in n. We can then see, using 26-27, that a threshold best response of the tagged customer whenever it exists is increasing to the threshold policy followed by the other customers. This means that a tagged customer, whose expected net benefit is given by 25, is more willing to enter the system if the other customers use a higher threshold policy, i.e. he adopts the behavior of the others. This shows that we have an FTC situation. We can also seek equilibrium strategies in the more general class of mixed threshold strategies. A strategy of the form observe Nt and enter if Nt n e, enter with probability q e if Nt = n e and balk otherwise is referred to as a mixed threshold strategy of type n e, q e also as n e + q e -threshold strategy, see Hassin and Haviv 2003, p. 8. To find equilibrium strategies in this class, we must compute π I N 0 n, when all customers follow the same mixed threshold strategy. Proposition 2 is not anymore applicable, because the transition rates of the system have changed. More specifically the transition rates all remain the same except for transitions n e, 0 n e +, 0 and n e, n e +,. For these transitions the rates are now q e instead of. The system is now depicted in Figure 3. The stationary analysis of this system is analogous to that in Proposition 2 for the pure threshold policy, with minor modifications. The following formulas can be derived for the embedded probabilities π I N 0 n. π I N 0 n = + + π I N 0 n e = + q e + π I N 0 n e + = + q e + ρ n ], n = 0,,... n e, 32 σ ρ ne ], 33 σ ρ ne σ ] For the case n L < n U, the following theorem shows that there is one mixed threshold equilibrium policy between every two consecutive pure threshold equilibrium policies. Theorem 3 In the almost observable M/M/ queue with setup times all mixed threshold strategies of the form observe Nt and enter if Nt n e, enter with probability q e if Nt = n e and balk otherwise are equilibrium strategies for n e = n L +,..., n U and q e = ρ ne C σ R Cn e

10 Proof. Fix an n e {n L +,..., n U } and define the corresponding q e by 35, i.e. q e is the unique solution of fx, n e = 0. The quantity q e is a probability because fx, n e is continuous with respect to x and f0, n e f, n e = f L n e f U n e 0, because of We now consider a tagged customer at his arrival instant and assume that all other customers follow the same mixed threshold strategy n e, q e. If the tagged customer finds n n e customers and decides to enter, his expected net benefit is R Cn + C + + ρ n ] = f U n > 0, σ because of 25, 32 and 30. Therefore, the tagged customer prefers to enter. If the tagged customer finds n = n e customers and decides to enter, his expected net benefit is equal to R Cn e + C + q e + ρ ne ] = fq e, n e = 0, σ because of 25, 33 and 35. So in this case the tagged customer is indiferent between entering and balking. Any decision is optimal and in particular entering with probability q e is optimal. If the tagged customer finds n = n e + customers and decides to enter, his expected net benefit is R Cn e + 2 R Cn e + 2 = R Cn e + 2 C = f L n e + 0. C + q e + C ρ ne + ] σ ρ ne + ] σ ρ σ ne+ ] Indeed, the initial expression is valid because of 25 and 34. The inequality is due to q e. The first equality follows from 27 and the second from with 3, since n e + {n L +2,..., n U + }. Thus, the customer prefers to balk. The above argument shows that any mixed threshold strategy n e, q e with n e and q e satisfying the conditions of the proposition is a best response against itself, therefore, it is an equilibrium strategy. Remark 5. For an equilibrium strategy y there may be a best response z y such that z is strictly a better response against itself than y is. In this case, given that the customers start with strategy y, they may all adopt strategy z and then they will never return to y. In this sense, y is unstable or transient. If no such z exists then y is said to be an evolutionarily stable strategy ESS. In particular, an equilibrium strategy which is a unique best response to itself is ESS. Equilibrium ESS rule out unstable equilibrium strategies and are considered a useful refinement of the equilibrium concept. In the present model, if, as is typically the case, the inequalities in 30-3 are strict, then the equilibrium pure threshold strategies are ESS, since they are unique best responses against themselves. On the contrary, the equilibrium mixed threshold strategies are not ESS. Indeed, it is easy to see that if all customers start from an equilibrium threshold strategy n e, q e with q e 0,, as in Theorem 3, and deviate to any threshold strategy n e, q, then n e, q e is not a better response for an individual against n e, q 0

11 , 0, 0 q0, 0 q 2, q q0 q0 2, 0 q n, q n +, q q0 q0 n, 0 q0 n +, 0 Figure 4: Transition rate diagram for the q0, q mixed strategy than n e, q is. In the light of this observation, we can restrict our attention to pure threshold strategies only. Remark 6. A moment of reflection in the proofs of Theorems 2 and 3 shows that the only equilibrium strategies of pure/mixed threshold type are those specified there. However, there also exist equilibrium strategies of non-threshold type. For example consider the strategy observe Nt and enter if Nt n e + for some n e {n L, n L +,..., n U }. This is also an equilibrium. Indeed this strategy agrees with the n e threshold strategy for states in {n, i 0 i, i n n e + }. On the other hand, the states in {n, i 0 i, n > n e + } are transient under both strategies and therefore do not influence the arriving customer s net benefit. These irrational equilibrium strategies can be eliminated by considering the refined concept of subgame perfect equilibrium SPE for details see Hassin and Haviv 2002, and Hassin and Haviv 2003, p Equilibrium mixed strategies for the unobservable cases In this section we turn our attention to the unobservable cases, where arrivals do not observe the number of customers present. We will prove that there exist equilibrium mixed strategies. We begin with the almost unobservable case in which arriving customers observe the state i of the server at their arrival instant. A mixed strategy for a customer is specified by a vector q0, q, where qi is the probability of joining when the server is in state i. If all customers follow the same mixed strategy q0, q, then the system follows a Markov chain similar to that described in Figure, except that the arrival rate equals i = qi for states where the server is in state i. The state space S au for the almost unobservable case is identical to the original state space S and the transition diagram is illustrated in Figure 4. Let p au n, i : n, i S be the stationary distribution of the corresponding system. The balance equations are presented below. p0, 00 = p, 36 pn, 00 + = pn, 00, n =, 2, p, + = p, 0 + p2, 38 pn, + = pn, + pn, 0 + pn +,, n = 2, 3, To obtain the stationary probabilities pn, i we proceed as in Proposition 2. We set By iterating 37 we obtain ρ0 = 0 0, ρ =, σ0 = 0 +. pn, 0 = σ0 n p0, 0, n = 0,,....

12 From 39 it follows that pn,, n =, 2,... is a solution of the nonhomogeneous linear difference equation with constant coefficients x n+ + x n + x n = pn, 0 = σ0 n p0, 0, n = 2, 3,.... Employing the standard approach as in the proof of Proposition, we can find the general solution of this equation. Note that the roots of the characteristic equation are now and ρ. For brevity we consider again only the typical case where σ0 ρ. Note also that σ0 by definition. Using 36 and the normalization equation we obtain the appropriate constants and we have the following. Proposition 2 Consider an M/M/ queue with setup times and σ0 ρ, in which customers observe the state i of the server upon arrival and enter with probability qi, i.e., they follow the mixed policy q0, q. The system is stable if and only if ρ <. In this case, the stationary probabilities p au n, i : n, i S are p au n, 0 = p au n, = σ0 ρ σ0 n, n = 0,,... ρ + ρ0 40 σ0 ρρ0 ρ + ρ0σ0 ρ σ0n ρ n, n =, 2, Remark 7. Proposition 2 can be alternatively proved by employing matrix analytic methods. Indeed, by partitioning the state space as S = n=0ln, where l0 = {0, 0} and ln = {n, 0, n, }, n =, 2,..., we obtain a continuous time quasi-birth-death QBD process. If we partition its stationary distribution according to the levels by setting p au 0 = p au 0, 0 and p au n = p au n, 0, p au n,, n, we have that the stationary distribution is matrix geometric see e.g. Neuts 98 Theorem.7. in p.32. More specifically for the Markov chain of interest we have that where σ0 ρ p au 0 = ρ + ρ0 σ0 ρσ0 p au =, ρ + ρ0 p au n = p au R n R = ρ0 σ0 ρ0 0 ρ σ0 ρρ0 ρ + ρ0 is the so-called rate matrix of the chain. The rate matrix R in this case is explicitly computable, because the block matrices of this QBD are upper triangular. By diagonalizing R, we derive after some algebra 40 and 4. The social benefit per time unit when all customers follow a mixed policy q e 0, q e can now be easily computed as ρ S au = ρ + ρ0 q0 C0 + R C ρ0 + ρ + ρ0 q C0 + C R. 42 2

13 We now consider a customer who finds the server at state i upon arrival. His mean sojourn time is EN i]+ + i, where EN i] is the expected number of customers in system found by an arrival, given that the server is found at state i. The expected benefit of such a customer who decides to enter is R CEN i] + C i. 43 We thus need to compute EN i] when all customers follow the same mixed strategy, say q0, q. Assume that the system is stable under this strategy, i.e., q <. Then the probability π N I n i, that an arrival finds n customers in system, given that the server is found at state i is equal to π N I n i = pn, ii n = i, i +, k=i pk, ii, We substitute 40-4 in 44 and from EN i] = n=i nπ N I n, i we obtain EN 0] = EN ] = σ0 σ0 ρ ρ + σ Substituting into 43 we can identify mixed equilibrium strategies for the almost unobservable model. Theorem 4 In the almost unobservable M/M/ queue with setup times and <, there exists a unique mixed equilibrium strategy q e 0, q e observe It and enter with probability q e It where the vector q e 0, q e is given as follows. Case I: <. R C, 0, R C + C, C+ + C, 0, R C+ + C, C+ q e 0, q e =, C, R C+ R C+ + C, C+,, R C+ + C,. + C + C Case II:. q e 0, q e = R C, C R C+, R,, R,, R C + C, C+ C+ + C + C, C+ C+ + C,. + C Case III: <. q e 0, q e =,, R R C,, R C + C, C+ + C C+ + C,. 3

14 Proof. Consider a tagged customer who finds the server at state 0 upon arrival. If he decides to enter, his expected net benefit is R CEN 0] + C C = R σ0 C C0 + = R C. Therefore we have two cases: Case : C + C < R C+ + C. In this case if all customers who find the system empty enter with probability q e 0 =, then the tagged customer suffers a negative expected benefit if he decides to enter. Hence, q e 0 = does not lead to an equilibrium. Similarly, if all customers use q e 0 = 0 then the tagged customer receives a positive benefit from entering, thus q e 0 = 0 also cannot be part of an equilibrium mixed strategy. Therefore, there exists a unique q e 0, satisfying R Cq e0 + C = 0 for which customers are indifferent between entering and balking. This is given by q e 0 = R C. 47 C+ Case 2: + C < R. In this case, for every strategy of the other customers, the tagged customer has a positive expected net benefit if he decides to enter. Hence q e 0 =. 48 We next consider q e and tag a customer who finds the server at state upon arrival. If he decides to enter his expected net benefit is equal to R CEN ] + C = R ρ C σ0 C C0 + = R { C = C, in case R C C+, in case Therefore, to find q e in equilibrium, we must examine Cases and 2 separately and consider the following subcases in each: Case a: C + C < R C+ + C and C < C. q e 0, q e = Case b: C + C < R C+ + C and C C C. q e 0, q e = R C, 0. R C,. 4

15 Case c: C + C < R C+ + C and C < C. R q e 0, q e = C Case 2a: C+ + C < R and R < C + C+. q e 0, q e =, 0.,. Case 2b: C+ + C < R and C + C+ R C + C+ q e 0, q e =, Case 2c: C+ + C < R and C + C+ < R. q e 0, q e =,.. C R C+. By rearranging Cases a-2c as R varies from C + C to infinity, keeping the operating parameters,, and the waiting cost rate C fixed, we obtain Cases I-III in the theorem statement. It seems reasonable at first glance that the information that the server is deactivated makes the customers less willing to enter the system, since they have to wait for the server activation time, i.e. we might expect that q e 0 q e. However, this is not generally true, as Theorem 4 shows: In case I it is always true that q e 0 q e, in case III that q e 0 q e and in case II the situation varies. Theorem 4 quantifies several intuitively expected scenarios about the behavior of the customers when they are informed about the state of the server upon arrival. The crucial factor is the mean setup time in comparison with the mean service time and the mean busy period of the corresponding M/M/ queue without setup times. For example, consider a system with small mean setup time and concentrate on a tagged customer. If he is given the information that the server is deactivated, then he knows that he must wait for a setup time. On the other hand, he expects that few customers are ahead of him, because the system was empty at the beginning of the current setup time and the mean setup time is small. Case I of Theorem 4 shows that the second factor few waiting customers prevails, thus it is optimal for the tagged customer to enter. In fact, when the mean setup time is small Case I, a more extreme situation occurs. In order for any customer to enter when the server is active, the reward must be sufficiently high that all customers enter when the server is inactive. In other words, q e > 0 only if q e 0 =. On the other hand, when the mean setup time is moderate Case II or high Case III, things are different. Another consequence of Theorem 4 is the following corollary, for the limiting case of fast server reactivation. Corollary When, q e 0, q e =, 0, R C, 2C, C, R 2C R C, C + C,, R C + C,. 50 5

16 This limiting equilibrium policy can be interpreted as follows. When the system behaves as an ordinary M/M/ queue and the server is inactive if and only if no customers are present. Therefore, the information on the state of the server is tantamount to informing arriving customers whether the system is empty or not. The expected net benefit of a customer who finds the server at state 0 is equal to R C. Thus, when R C, he prefers to enter, i.e., q e 0 =. On the other hand, the server is at state if and only if there exists at least one customer in the system. In this case the customer s expected net benefit if he decides to enter is R C EN N > 0] +, where N is a random variable representing the number of customers in system found by an arrival in an M/M/ queue with state dependent arrival rate: q e 0 = when the queue is empty and q e otherwise. We can easily show that the conditional distribution of N given that N > 0 is geometric with parameter qe, therefore EN N > 0] = q e. Hence the expected net benefit from entering of an arriving customer who finds the server active is equal to R C C q e. If R < 2C then this quantity is always negative, thus the best response is 0 and we obtain q e = 0, the first branch of 50. For 2C R C + C and R > C + C we obtain the other two branches of 50 with analogous reasoning. Remark 8. In Theorem 4 it was assumed that <. For the opposite case an analogous result holds with the various cases simplified. Specifically, it can be shown following a similar analysis, that, when, there exists a unique mixed equilibrium strategy q e 0, q e observe It and enter with probability q e It where the vector q e 0, q e is given as follows. Case I: <. R C, 0, R C + C, C+ q e 0, q e =, 0, R, C, R R C+ C+ + C + C, C+ C+ + C,. + C Case II: q e 0, q e =, R, C C R C+, R C + C, C+, R + C C+ + C,. We finally consider the fully unobservable case, where the customers do not observe the state of the system at all. Here a mixed strategy for a customer is specified by the probability q of entering. The stationary distribution of the system state is given by Proposition 2 by taking q0 = q = q. The equilibrium behavior of the customers is described in the following. Theorem 5 In the fully unobservable M/M/ queue with setup times and <, there exists a unique mixed equilibrium strategy enter with probability q e, where q e is given by C, R C R q e = C + C, C + C 5, R C + C,. 6

17 Proof. We consider a tagged customer at his arrival instant. If he decides to enter his expected net benefit is R C EN ] + C PrI = 0], 52 where N represents the number of customers in the system and I the state of the server seen by the tagged customer. Using Proposition 2 we obtain EN ] = ρ ρ + σ σ and PrI = 0] = ρ, with ρ = q, σ = q q+. Hence, the expected net benefit in 52 is equal to R C q C. 53 When R C + C, C + C, we find that 53 has a unique root in 0, which gives the first branch of 54. When R C + C, the quantity in 53 is positive for every q, thus the best response is and the unique equilibrium point is q e =, which gives the second branch of 54. Remark 9. For the case the analog of Theorem 5 is that there exists a unique mixed equilibrium strategy enter with probability q e, where q e = C R C, 54 for R C + C,. 5. Numerical results In this section we present numerical experiments that show the effect of the information level as well as several parameters on the behavior of the system. Specifically, we are interested in the values of the equilibrium thresholds for the observable models and the values of the equilibrium entrance probabilities for the unobservable models as well as the social benefit per unit time when the customers follow equilibrium strategies. We first consider the fully and almost observable systems and explore the sensitivity of the equilibrium pure threshold policies with respect to the service reward R, arrival rate and setup rate by rescaling if necessary, we can assume without loss of generality that = C =. The results are presented in Figure 5. An interesting conjecture arises from this figure. We observe that in all three diagrams the range of thresholds for the almost observable case, {n L,..., n U } is contained inside the range between n e 0 and n e for the fully observable case. In other words, in the almost observable model the common threshold has an intermediate value between the two separate thresholds that the customers use when they are given the additional information on the server state Regarding the sensitivity in specific parameters, we can make the following observations. When the service reward R varies above the minimum level C + C required for entrance, the thresholds increase in a linear fashion, up to the integrality requirement. Under varying arrival rate, the fully observable thresholds remain fixed. This is expected from Theorem, since the arrival rate is irrelevant to the customer s decision when he has full state information. On the other hand, the almost observable threshold range increases with. This means that when an arriving customer is given information on the number of customers present, then he is more likely 7

18 to enter the system when the arrival rate is higher. The reason is that when the arrival rate is high, it is more likely that the server is active, therefore the expected delay from server activation is reduced, while the delay due to the customers already present is not affected. Finally, when the setup rate varies, all thresholds increase, except for n e which remains constant. This is certainly intuitive, because when the server activation is faster customers generally have a greater incentive to enter both in the fully and the almost observable case. In the second numerical experiment we turn to the almost and fully unobservable systems and explore the sensitivity of the equilibrium entrance probabilities. The results are shown in Figure 6. A general observation from this Figure is that, similarly to the previous experiment,the entrance probability in the fully unobservable model is always inside the interval formed by the two entrance probabilities in the almost unobservable case. However the relative ordering of q e 0 and q e varies. Therefore, when customers are not given the information about server state, they follow an intermediate strategy and join the queue with a probability between those that they would employ for the two separate cases, were these known. With regard to sensitivity, the entrance probabilities are increasing with respect to R, which is intuitive. Furthermore, they are nonincreasing with respect to, therefore customers are less inclined to enter the system as the arrival rate increases. This is in contrast to the observable cases, where thresholds are nondecreasing in. The reason for the difference is that here the information on the number in the system is not available, therefore, when is higher, arriving customers expect that the system is more loaded and are less willing to enter. With respect to the setup rate, the behavior of the entrance probabilities varies. While for the most part they are all increasing with, which is intuitive, there is a range of small values of in which q e is decreasing. The last numerical experiment is concerned with the social benefit under the equilibrium strategy for the different information levels. The results are presented in Figures 7, 8, 9. In the almost observable case we have seen that in general there are multiple pure equilibrium strategies, corresponding to thresholds n L, n L +,..., n U. For this reason we present the social benefit under the two extreme threshold values, n L, n U. In the figures we observe that the difference in social benefit is small between the fully and almost observable case, while there may be significant differences between the observable and the unobservable models. Thus, it may be argued that the customers as a whole are generally better off when upon arrival they are given the information on how many are already present and are left to decide whether to join the system or not, while the additional information on the server state is not very beneficial. In regard to the sensitivity in parameters, the social benefit is increasing with respect to the reward R and setup rate, both of which are intuitive. Regarding the arrival rate, the social benefit achieves a maximum for intermediate values of this parameter. The reason for this behavior is that when the arrival rate is small the system is rarely crowded, therefore as more customers arrive they are served and the social benefit improves. However as continues to increase a smaller percentage decides to enter and those who do are subjected to longer delays, which has a detrimental effect on the social benefit. 6. Conclusions and Extensions In this paper we considered the problem of analyzing customer behavior in equilibrium, in an M/M/ queue with server setups where customers decide whether to join the system upon arrival. We identified four cases with respect to the level of information provided to arriving customers and derived the equilibrium strategies for each case. We also discussed the sensitivity with respect to various parameters as well as the effect of the information level on the social benefit. The focus of this work was on equilibrium analysis. The social benefit for each information 8

19 level was determined, under the assumption that all customers decide independently and rationally whether to enter the system. On the other hand, one can think of situations where a central planner employs acceptance policies that maximize the social benefit, under the various levels of information on the system state. As is expected, such plans, which do not generally constitute equilibrium policies, may not be possible to enforce when customers are allowed to make independent decisions. However they are applicable when all customers are potentially willing to enter the queue without necessarily considering benefit versus costs, perhaps because of other hard constraints. Such situations may arise among others in traffic engineering and telecommunications. Another interesting extension would be to incorporate the present model in a profit maximizing framework, where the owner or manager of the system imposes an entrance fee. The problem is to find the optimal fee that maximizes the owner s total profit subject to the constraint that customers independently decide whether to enter or not and take into accout the fee, in addition to the service reward and waiting cost. On a higher layer of optimization, one might also consider problems of the optimal capacity level that maximizes profit, given that the entrance pricing is designed optimally. Acknowledgements The authors acknowledge support by EPEAEKII-Pythagoras, grant # 70/3/7388 European social fund and national resources. A. Burnetas was also supported by grant ELKE/70/4/7584 University of Athens. A. Economou was also supported by grants ELKE/70/4/645 University of Athens and MTM Spanish research project. References ] Adan, I. and J. van der Wal 998 Difference and Differential Equations in Stochastic Operations Research, Online Notes, URL: iadan/ 2] Artalejo, J.R. and M.J. Lopez-Herrero 2003 On the M/M/m queue with removable servers, in: Stochastic Point Processes, eds. S.K. Srinivasan and A. Vijayakumar Narosa Publishing House ] Artalejo, J.R., A. Economou and M.J. Lopez-Herrero 2005 Analysis of a multiserver queue with setup times, Queueing Systems 52, ] Bischof, W. 200 Analysis of M/G/-queues with setup times and vacations under six different service disciplines, Queueing Systems 39, ] Borthakur, A. and G. Choudhury 999 A multiserver Poisson queue with a general startup time under N-Policy, Calcutta Statistical Association Bulletin 49, ] Chen, H. and M. Frank 2004 Monopoly pricing when customers queue, IIE Transactions 36, ] Chen, H. and M. Frank 200 State dependent pricing with a queue, IIE Transactions 33, ] Choudhury, G. 998 On a batch arrival poisson queue with a random setup and vacation period, Computers and Operations Research 25, ] Choudhury, G An M X /G/ queueing system with a setup period and a vacation period, Queueing Systems 36, ] Edelson, N. M. and K. Hildebrand 975 Congestion tolls for Poisson queueing processes, Econometrica 43, ] Elaydi, S. N. 999 An Introduction to Difference Equations, Mathematics. Springer-Verlag, New York. 9

20 2] Hassin, R. 986 Consumer information in markets with random products quality: The case of queues and balking, Econometrica 54, ] Hassin, R. and M. Haviv 997 Equilibrium threshold strategies: the case of queues with priorities, Operations Research 45, ] Hassin, R. and M. Haviv 2002 Nash equilibrium and subgame perfection in observable queues, Annals of Operations Research 3, ] Hassin, R. and M. Haviv 2003 To Queue or Not to Queue: Equilibrium Behavior in Queueing Systems, Kluwer Academic Publishers, Boston. 6] He, Q.M. and E. Jewkes 995 Flow time in the MAP/G/ queue with customer batching and setup times, Stochastic Models, ] Naor, P. 969 The regulation of queue size by levying tolls, Econometrica 37, ] Takagi, H. 99 Queueing Analysis - A Foundation of Performance Evaluation, Vol. : Vacation and Priority Systems, New York, North-Holland. 9] Tian, N. and Z.G.Zhang 2006 Vacation Queueing Models: Theory and Applications, Springer- Verlag, New York. 20] Yechiali, U. 97 On optimal balking rules and toll charges in the GI/M/ queue, Operations Research 9,

21 thresholds a n e 0 n e n L n U R thresholds b n e 0 n e n L n U thresholds c n e 0 n e n L n U Figure 5: Equilibrium Thresholds for Observable and Almost Observable Systems. Sensitivity with Respect to: a R, for = 0.4, =, = 0.05, C = ; b, for =, = 0.05, C =, R = 25; c, for = 0.8 =, C =, R = 25 2

22 Entrance Prob a q0 q q e R Entrance Prob b q0 q q e Entrance Prob c q0 q q e Figure 6: Equilibrium Entrance Probabilities for Unobservable and Almost Unobservable Systems. Sensitivity with Respect to: a R, for = 0.9, =, = 0.5, C = ; b, for =, = 0.5, C =, R = 8; c, for = 0.9, =, C =, R = 0 22

23 5 F.O. A.O. n L A.O. n U A.U. F.U 0 S R Figure 7: Social Benefit for Different Information Levels. Sensitivity with Respect to: a R, for = 0.5, =, = 0.05, C = 23

24 0 F.O. A.O. n L A.O. n U A.U. F.U 8 S Figure 8: Social Benefit for Different Information Levels. =, = 0.05, C =, R = 25 Sensitivity with Respect to, for 24

25 8 7 F.O. A.O. n L A.O. n U A.U. F.U 6 5 S Figure 9: Social Benefit for Different Information Levels. = 0.9, =, C =, R = 0 Sensitivity with Respect to, for 25