Individual, Class-based, and Social Optimal Admission Policies in Two-Priority Queues

Individual, Class-based, and Social Optimal Admission Policies in Two-Priority Queues Feng Chen, Vidyadhar G. Kulkarni Department of Statistics and Operations Research, University of North Carolina at Chapel Hill, 210 Smith Building, CB#3260, Chapel Hill, NC 27599-3260 chenf@email.unc.edu vkulkarn@email.unc.edu Abstract: This paper considers the admission control problem for an M/M/1 queueing system serving two classes of customers. Class 1 customers have preemptive resume priority over class 2 customers. Within each class, the service is provided on a first-come, first-served basis. The system is controlled by accepting or rejecting arriving customers. There is a class-dependent reward and holding cost associated with each accepted customer. The goal is to minimize the expected total discounted net cost. We analyze and compare the optimal control policies under three criteria: individual optimization, class optimization, and social optimization. We show (i) the optimal policy is of either critical-number or switching-curve form under each optimization criterion, (ii) the class-optimal policy accepts more class 1 customers but less class 2 customers than the socially optimal policy, which has interesting socio-economic explanation, (iii) the individually optimal policy accepts more class 1 customers than the class-optimal policy, while it can accept either more or less class 2 customers than either of the other two optimal policies. Keywords: M/M/1 queue; two-priority; optimal admission control; value iteration; switching curve 1 Introduction We consider an M/M/1 queueing system serving two classes of customers. Class 1 customers have preemptive resume priority over class 2 customers. Within each class, the service is provided on a first-come, first-served basis. Class i customers arrive according to a Poisson process with parameter λ i, i = 1, 2. Every customer requires an i.i.d. exp(µ) service time (same for both classes). The system is controlled by accepting or rejecting arriving customers. There is a reward of r i associated with accepting a class i customer. An accepted class i customer incurs a waiting cost of h i per unit time spent in the system. All rewards and costs are continuously discounted with rate α > 0. This research is partially supported by NSF under grant DMI-0223117. The goal is to minimize the expected total 1

discounted net cost. Clearly, this problem arises naturally in control of admission to queues with multiple priorities. This paper is also motivated by the problem of outsourcing warranty repairs to outside vendors when items have priority in service. Consider a manufacturer that has a contract with a number of repair vendors for a fixed fee per repair. The manufacturer assigns an item to a repair vendor at the time of failure with the objective of minimizing the expected total discounted cost. Priority issue arises when the manufacturer offers a choice of warranties that specify different repair turnaround times. Repairs with shorter turnaround time are granted higher priority. To solve this dynamic allocation problem, we start with solving the admission control problem for a single vendor, which is the problem we are considering in this paper. The results of this single-vendor problem can then be used to derive index-based dynamic allocation policies for multiple vendors. (See Opp et al. [14]) Admission control for single class queueing systems is a well studied area. See Stidham [18] for a survey. The first quantitative model in this area is that of Naor [13], who studies an M/M/1 system with a single class of customers. He considers undiscounted reward and cost and the objective is to maximize the long-run average net reward per unit time. Naor considers only critical-number policies and shows that n S n I, where n S and n I are the critical numbers for social optimization and individual optimization, respectively. Yechiali [20][21] proves that for GI/M/1, GI/M/s systems the socially optimal policy has critical-number form. Thus Naor s restriction to critical-number policies is without loss of generality. Naor s result has been generalized by many authors. Among others, Knudsen [5] considers an M/M/s queue with state-dependent net benefit. Lippman and Stidham [9] study a birth-death process with general departure rate, random reward, with or without discounting and for a finite or infinite time horizon. Stidham [17] considers a GI/M/1 queue with random reward and general holding cost, with or without discounting. For other models of admission control problem for single-class queues, see Adiri and Yechiali [1], Stidham and Weber [19], and Rykov [16]. Admission control for multi-class queueing systems is another important research area. Models in this area can be classified into two categories based on whether or not service is prioritized based on class. In models without priorities, different classes are distinguished by different arrival rates, service rates, rewards, holding costs, etc. For papers in this category, among others, see Miller [11], Blanc and de Waal [2], Kulkarni and Tedijanto [7], and Nair and Bapma [12]. Among papers that consider service priorities, Mendelson and Whang [10] study a priority 2

pricing problem for a multi-class M/M/1 queueing system, where each customer decides by himself whether or not to join the system and at what priority level. Hassin [4] studies a bidding mechanism for a GI/M/1 queue without balking. Ha [3] considers the production control problem in a make-to-stock production system with two priority customer classes. To the best of our knowledge, the admission control problem for a multi-priority queue with the objective of minimizing expected total discounted cost has not been studied. In addition to individual and social optimization problems, the existence of multiple classes generates a third criterion for optimization, i.e., class optimization. We have not seen this aspect addressed in the literature. We analyze the optimal control policies under all 3 criteria: individual optimization, class optimization, and social optimization. Under individual optimization, each customer obtains the reward and pays the waiting cost by himself. A customer makes decision based on the objective of minimizing his own expected total discounted net cost. Under class optimization, there is a controller for each class. The controller of class i obtains the reward and pays the waiting cost generated by each class i customer. He decides whether to accept an arriving class i customer or not based on the objective of minimizing the expected total discounted net cost incurred by all class i customers. Under social optimization, there is a single controller for the whole system. The system controller obtains the reward and pays the waiting cost generated by every customer. He decides whether to accept an arriving customer or not based on the objective of minimizing the expected total discounted net cost incurred by all customers. We show that the optimal policy for each class under each optimization criterion is of either critical-number or switching-curve form. We compare among the optimal policies and show that a class 1 customer will join the system under individual optimization whenever he is accepted by the class-optimal policy. A class 1 customer will be accepted by the class-optimal policy whenever he is accepted by the socially optimal policy. Interestingly, the relationship between the class and socially optimal policies for class 2 is the exact opposite, i.e., a class 2 customer will be accepted by the socially optimal policy whenever he is accepted by the class-optimal policy. This unexpected result has interesting socio-economic explanation. It says that the lower priority class would prefer an outside controller that can set social policies that take into account the interests of all classes in a fair way. On the other hand the higher priority class would prefer to set policies for themselves since they can ignore the effect on the lower priority class. Numerical examples show that, for class 2, the relationship between the individually optimal policy and either of the other two optimal policies can be arbitrary, as 3

demonstrated by the figures in the Appendix. The remainder of the paper is organized as follows. Sections 2, 3, and 4 analyze the optimal policies under three criteria: individual optimization, class optimization, and social optimization, respectively. Section 5 discusses a special case where h 1 = h 2. Section 6 compares among the optimal policies and shows the numerical results. We end with the summary and possible extensions in Section 7. 2 Individual Optimization We consider individual optimization in this section. Clearly, the individually optimal policy for an arriving customer is to join the system if and only if his expected discounted net cost is less than or equal to zero. Denote the system state by (i, j), where i is the number of class 1 customers in the system and j is the number of class 2 customers in the system. We need the following lemma to derive the main result in Theorem 1. Lemma 1 Let X(t) be the number of customers in a M/M/1/k queue at time t with arrival rate λ and service rate µ. Let T = min{t 0 : X(t) = 0} and define φ i (α) = E(e αt X(0) = i). Then, φ i (α) is given by where φ i (α) = ui 1 uk 1 2 (u 2 (α + µ) µ) u i 2 uk 1 1 (u 1 (α + µ) µ) u2 k 1 (u 2 (α + µ) µ) u1 k 1, i = 0,..., k, (1) (u 1 (α + µ) µ) u 1 = 1 2λ (α + λ + µ + (α + λ + µ) 2 4λµ), u 2 = 1 2λ (α + λ + µ (α + λ + µ) 2 4λµ). (2) Proof: {X(t), t 0} is a birth-death process on state space S = {0, 1,..., k}. By Theorem 6.21 of Kulkarni [6], {φ i (α)} is the solution to φ 0 (α) = 1, µφ i 1 (α) (α + λ + µ)φ i (α) + λφ i+1 (α) = 0, i = 1, 3,..., k 1, µφ k 1 (α) (α + µ)φ k (α) = 0. (3) Solving the above system of equations yields (1). Theorem 1 Under the individual optimization criterion, an arriving class 1 customer who sees the system in state (i, j) joins the queue if and only if i < L I 1, where {, if L I h1 αr 1 1 = log(1 αr 1 h 1 )/ log µ µ+α, if h 1 > αr 1. (4) 4

An arriving class 2 customer who sees the system in state (i, j) joins the queue if and only if j < L I 2 (i), where L I 2(i) =, if h 2 αr 2 log h 2 αr 2 h 2 φ i (α) / log β, if h 2 > αr 2, i L I 1 (log h 2 αr 2 h 2 φ L I 1 (α) + (i LI 1 where φ i (α) is given in (1), β = to x. Furthermore, L I 2 (i) is decreasing in i. µ+α )(log µ ))/ log β, if h 2 > αr 2, i > L I 1, (5) µ α+µ+λ 1 (1 φ 1 (α)), x is the largest integer less than or equal Proof : First consider class 1 customers. Denote the sojourn time of a class 1 customer who joins the system in state (i, j) by X(i, j). Since class 1 customers have preemptive priority over class 2 customers, we have X(i, j) = X 1 + X 2 +... + X i+1, where X k, k = 1, 2,..., i + 1 are i.i.d. exp(µ) service times. So the class 1 customer s expected total discounted cost is Therefore, he joins the queue if and only if X(i,j) E( h 1 e αt dt) = h 1 0 α (1 ( µ µ + α )i+1 ). which is equivalent to i < L I 1, where LI 1 is defined in (4). h 1 α (1 ( µ µ + α )i+1 ) r 1, (6) Now consider class 2 customers. Denote the sojourn time of a class 2 customer who joins the system in state (i, j) by Y (i, j). We can decompose Y (i, j) into 3 periods. Period 1, denoted by T 1, is the time period for serving the first i L I 1 class 1 customers, if i > LI 1. Period 1 has length 0 if i L I 1. Note that no class 1 arrivals will be accepted during this period. Period 2, denoted by T 2, is the server s busy period for serving the remaining class 1 customers and the class 1 customers joining the system during this period, which ends when the first class 2 customer starts receiving service. Period 3, denoted by T 3, is the time period for serving the j + 1 class 2 customers and the class 1 customers joining the system during this period. Consider period T 1 first. If i L I 1, T 1 has length 0, thus E(e αt 1 ) = 1. If i > L I 1, T 1 is the sum of i L I 1 i.i.d. exp(µ) service times. Thus E(e αt 1 ) = ( µ α+µ )i LI 1. From Lemma 1 we know the LST of T 2 is given by (1) with k = L I 1. 5

Consider period T 3. T 3 = j+1 k=1 Z k, where Z k is the time period for serving the kth class 2 customer and the class 1 customers joining the system during this period. Let β = E(e αz 1 ). Using first-step analysis, one can show that β satisfies β = µ + λ 1 µ ( + λ 1 φ 1 (α)β). α + µ + λ 1 µ + λ 1 µ + λ 1 Solving for β, we have Since {Z k } are i.i.d., we have β = µ α + µ + λ 1 (1 φ 1 (α)). E(e αt 3 ) = (E(e αz 1 )) j+1 = β j+1. Thus E(e αy (i,j) ) = E(e αt 1 )E(e αt 2 )E(e αt 3 µ ) = ( α + µ )max{0,i LI 1 } φ min{i,l I 1 } (α)βj+1. Therefore, the expected total discounted cost for a class 2 customer joining the system in state (i, j) is Y (i,j) E( h 2 e αt dt) = h 2 0 α (1 ( µ α + µ )max{0,i LI 1 } φ min{i,l I 1 } (α)βj+1 ). He will join the system if and only if h 2 α (1 ( µ α + µ )max{0,i LI 1 } φ min{i,l I 1 } (α)βj+1 ) r 2, which is equivalent to j < L I 2 (i), where LI 2 (i) is defined in (5). i. Since T is stochastically increasing in i, φ i (α) is decreasing in i. Thus L I 2 (i) is decreasing in 3 Class Optimization We consider class optimization in this section. There is a controller for each class. The controller of class i decides whether to accept an arriving class i customer or not based on the objective of minimizing the expected total discounted net cost incurred by all class i customers, i = 1, 2. Consider the optimal policy for the controller of class 1 first. This is the standard single-class admission control problem studied by many authors. Among others, Stidham [17] considers a GI/M/1 queue with random rewards and general holding cost and shows that the optimal policy is of critical-number form. As a special case, we have 6

Theorem 2 The optimal policy for the controller of class 1 is a threshold policy, i.e., there exists a constant L C 1 such that an arriving class 1 customer is accepted if and only if i < LC 1. Now consider the optimal policy for the controller of class 2. Assume that the controller of class 1 applies his optimal policy and the controller of class 2 knows that. Let v(i, j) be the minimum expected total discounted cost for the controller of class 2 with initial state (i, j). Following Lippman [8], we uniformize the process by defining the uniform rate Λ = λ 1 +λ 2 +µ. Assuming, without loss of generality, Λ + α = 1, the optimality equations can be written as v(i, j) = T v(i, j) = C(j) + λ 1 T 1 v(i, j) + λ 2 T 2 v(i, j) + µt 3 v(i, j), (7) where C(j) = h 2 j, (8) T 1 v(i, j) = { v(i + 1, j), i < L C 1 v(i, j), i L C 1, (9) T 2 v(i, j) = min{ r 2 + v(i, j + 1), v(i, j)}, (10) and v(i 1, j), i 1, j 0 T 3 v(i, j) = v(0, j 1), i = 0, j 1 v(0, 0), i = 0, j = 0. Let V be the set of functions such that if v V, then (11) v is monotonically increasing in i, i.e., v(i, j) v(i + 1, j), (12) v is monotonically increasing in j, i.e., v(i, j) v(i, j + 1), (13) v is supermodular, i.e., v(i, j + 1) + v(i + 1, j) v(i, j) + v(i + 1, j + 1), (14) v is diagonally dominant in j, i.e., v(i, j + 1) + v(i + 1, j + 1) v(i + 1, j) + v(i, j + 2). (15) 7

It is worth noting that if v V, then v is convex in j, i.e., v(i, j + 1) v(i, j) v(i, j + 2) v(i, j + 1). (16) This follows by adding inequalities (14) and (15). We have Theorem 3 The optimal value function v V. Proof : Let v 0 (i, j) = 0, (i, j) S, and define, for n 0, v n+1 (i, j) = C(j) + λ 1 T 1 v n (i, j) + λ 2 T 2 v n (i, j) + µt 3 v n (i, j). Since α > 0, we know that v n v as n. (See Theorem 6.3.1 of Puterman [15].) It is easy to see that C(j) V. One can show that if v n V then T i v n V for i = 1, 2, 3. (See Lemma 3, 4, 5 and their proofs in the Appendix.) Now, clearly v 0 V and the above observation yields that if v n V then v n+1 V. Hence, by induction, v n V for all n. Therefore, by taking limits, v V, thus proving the theorem Theorem 4 The optimal policy for the controller of class 2 is characterized by a monotonically decreasing switching curve, i.e., for each i 0, there exists a threshold L C 2 (i), such that a class 2 arrival in state (i, j) is accepted if and only if j < L C 2 (i). Furthermore, LC 2 (i) is monotonically decreasing in i. Proof : From (10) we can see that a class 2 arrival in state (i, j) is accepted if and only if v(i, j + 1) v(i, j) r 2. (17) Let L C 2 (i) = min{j : v(i, j + 1) v(i, j) > r 2}. By using property (16), one can show that condition (17) is equivalent to j < L C 2 (i). For i 1 i 2, we have v(i 2, j + 1) v(i 2, j) v(i 1, j + 1) v(i 1, j), which follows from property (14). By definition of L C 2 (i 1), we have v(i 1, L C 2 (i 1) + 1) v(i 1, L C 2 (i 1)) > r 2, so v(i 2, L C 2 (i 1) + 1) v(i 2, L C 2 (i 1)) > r 2. By definition of L C 2 (i 2), we have L C 2 (i 1) L C 2 (i 2). Thus, L C 2 (i) is decreasing in i. 8

4 Social Optimization We consider social optimization in this section. There is a single controller for the whole system, he earns the rewards and pays the holding costs generated by all customers. Let v(i, j) be the minimum expected total discounted cost for the system controller with initial state (i, j). Using uniform rate Λ = λ 1 + λ 2 + µ, and assuming, without loss of generality, Λ + α = 1, the optimality equations can be written as v(i, j) = T v(i, j) = C(i, j) + λ 1 T1 v(i, j) + λ 2 T 2 v(i, j) + µt 3 v(i, j), (18) where C(i, j) = h 1 i + h 2 j, T 1 v(i, j) = min{ r 1 + v(i + 1, j), v(i, j)}, (19) T 2 is defined in (10) and T 3 is defined in (11). Let V be the set of functions such that if v V, then v satisfies (12) - (15), and v is diagonally dominant in i, i.e., v(i + 1, j) + v(i + 1, j + 1) v(i, j + 1) + v(i + 2, j), (20) v is increasing in the direction of (1, 1), i.e., v(i, j + 1) v(i + 1, j). (21) Notice that if v V, then v is convex in i, i.e., v(i + 1, j) v(i, j) v(i + 2, j) v(i + 1, j). (22) This follows by adding inequalities (14) and (20). We have Theorem 5 If h 1 h 2, the optimal value function v V. Proof : Since h 1 h 2, it can be easily shown that C(i, j) V. One can show that inequalities (12) - (15), (20), (21) are preserved under T 1, T 2, and T 3. (See Lemma 6, 7, 8 and their proofs in the Appendix.) The theorem follows from similar arguments as in the proof of Theorem 3. 9

Theorem 6 Assume h 1 h 2, then the socially optimal policy is characterized by two monotonically decreasing switching curves. (1) For each i 0, there exists a threshold L S 2 (i), such that a class 2 arrival in state (i, j) is accepted if and only if j < L S 2 (i). Furthermore, LS 2 (i) is monotonically decreasing in i. (2) For each j 0, there exists a threshold L S 1 (j), such that a class 1 arrival in state (i, j) is accepted if and only if i < L S 1 (j). Furthermore, LS 1 (j) is monotonically decreasing in j. Proof : Define L S 1 (j) = min{i : v(i + 1, j) v(i, j) > r 1 }, L S 2 (i) = min{j : v(i, j + 1) v(i, j) > r 2 }. The theorem follows from similar arguments as in the proof of Theorem 4. 5 A Special Case for Social Optimization We consider the special case where h 1 = h 2 under social optimization criterion in this section. When h 1 = h 2, the order of service will not affect the social welfare. So the priority can be ignored and the problem becomes a standard admission control problem with two classes differentiated by different arrival rates and rewards. One can apply the proof in Stidham [17] on both classes and show that the socially optimal policy depends only on the total number of customers in the system and is described by two critical numbers. We prove this result as a special case of Theorem 6 as follows. Lemma 2 If h 1 = h 2, then Lemma 6, 7, and 8 hold with (21) replaced by v(i, j + 1) = v(i + 1, j). (23) Proof : We only need to show that (23) is preserved under T 1, T 2, and T 3. For T 1, we have T 1 v(i, j + 1) = min{ r 1 + v(i + 1, j + 1), v(i, j + 1)} = min{ r 1 + v(i + 2, j), v(i + 1, j)} = T 1 v(i + 1, j), where the second equality follows from the fact that v(i+1, j +1) = v(i+2, j) and v(i, j +1) = v(i + 1, j). T 2 preserving (23) can be proved similarly. 10

For T 3, if i 1, then T 3 v(i, j + 1) = v(i 1, j + 1) = v(i, j) = T 3 v(i + 1, j). If i = 0, then T 3 v(0, j + 1) = v(0, j) = T 3 v(1, j). Since C(i, j) obviously satisfies (23), Lemma 2 implies that Theorem 5 and 6 still hold after replacing (21) with (23). Thus, we have Theorem 7 If h 1 = h 2, then there exist constants l 1, l 2 such that L S 1 (j) = l 1 j, (24) L S 2 (i) = l 2 i, (25) where l 1 l 2 if and only if r 1 r 2. Proof : Let l 1 = L S 1 (0). In order to prove (24), we only need to show that LS 1 (j+1) = LS 1 (j) 1 for any j 0. Let i = i + 1, we have L S 1 (j + 1) = min{i : v(i + 1, j + 1) v(i, j + 1) > r 1 } = min{i : v(i + 2, j) v(i + 1, j) > r 1 } = min{i 1 : v(i + 1, j) v(i, j) > r 1 } = min{i : v(i + 1, j) v(i, j)} 1 = L S 1 (j) 1, where the second equality follows from Lemma 2. (25) can be proved similarly by setting l 2 = L S 2 (0). We have l 1 = L S 1 (0) = min{i : v(i + 1, 0) v(i, 0) > r 1}, and l 2 = L S 2 (0) = min{j : v(0, j + 1) v(0, j) > r 2 } = min{j : v(j + 1, 0) v(j, 0) > r 2 }, where the second equality follows from Lemma 2. Therefore, l 1 l 2 if and only if r 1 r 2. 11

6 Comparison and Numerical Results We first compare among the optimal policies for class 1 customers. Under individual optimization criterion, the cost incurred by a class 1 customer joining the system is his own waiting cost (the internal effect). Under class optimization criterion, the entry of a class 1 customer also imposes additional waiting costs on the class 1 customers joining the system later (the external effect). Under social optimization criterion, the external effect is imposed on all class 2 customers as well as later class 1 customers. Thus, intuitively, the number of class 1 customers admitted to the system is the most under individual optimization, the second under class optimization, and the least under social optimization. This intuition is shown to be correct by the following theorem. Theorem 8 L S 1 (j) LC 1 LI 1, j 0, where the first inequality holds when h 1 h 2. For a GI/M/1 single-class queue with convex, nondecreasing holding cost rate, Stidham [17] proves that more customers are accepted by the individually optimal policy than by the socially optimal policy. As a special case of Stidham s result, we have L C 1 LI 1. Note that the socially optimal policy in Stidham s model corresponds to the class-optimal policy here. See Appendix for the proof of L S 1 (j) LC 1, j 0. Now consider the optimal policy for class 2 customers. The external effects of a class 2 customer are the same under class optimization and social optimization. Since a class-optimal policy admits more class 1 customers, the internal effect of a class 2 customer, is higher under class optimization. Therefore, the class-optimal policy admits less class 2 customers than the socially optimal policy does. This intuition is proved to be true in the following theorem. Theorem 9 Assume h 1 h 2, then L C 2 (i) LS 2 (i), i 0. It is worth noting that the comparisons between class-optimal and socially optimal policies for class 1 and class 2 give opposite results. This contrast has the following interesting social connotation. Suppose the whole society can be divided into two classes, people with power and people without power. If we define better as more customers get served, then the powerful people will prefer to optimize things within their own class, while the powerless people will be better off if a central decision maker can optimize the benefits for the society as a whole. Seen in this fashion, the result makes intuitive sense. Now compare the individually optimal policy with the other two optimal policies. Under individual optimization, a class 2 entry has no external effect, but it has more internal effect 12

than under class or social optimization, since the individually optimal policy admits the most class 1 customers. So the comparison results between L I 2 (i) and LC 2 (i) and between LI 2 (i) and L C 2 (i) depend on which effect is more dominant. We demonstrate the above results by numerical examples below. See Appendix for the figures. The numerical examples are computed by using standard value iteration algorithm. We approximate the infinite state space by assuming that no customers arrive when the total number of customers in the system reaches an upper bound B, which is much larger than the expected queue length. Thus the state space is S = {(i, j) : 0 i, j B}. The stopping criterion is max{ v n+1 (i, j) v n (i, j) : (i, j) S} 10 5, where v n (i, j) is the value function at the n th iteration. Figure 1 illustrates the optimal policies for class 1 customers with parameters α = 0.05, µ = 0.5, λ 1 = 0.44, λ 2 = 0.01, h 1 = 20, h 2 = 10, r 1 = 200, r 2 = 190. Figure 2-6 illustrate the optimal policies for class 2 customers under different arrival rates. Figure 2 uses the same parameters as used in Figure 1 and shows that L I 2 (i) LC 2 (i) LS 2 (i), i. Keeping the other parameters the same, Figure 3 uses λ 1 = 0.39, λ 2 = 0.06, and shows that L C 2 (i) LI 2 (i) L S 2 (i), i. Figure 4 uses λ 1 = 0.27, λ 2 = 0.18, and shows that L C 2 (i) LS 2 (i) LI 2 (i), i. Figure 5 uses λ 1 = 0.41, λ 2 = 0.04, and shows that L I 2 (i) LC 2 (i) for i 4, LI 2 (i) LC 2 (i) for i 5. Figure 6 uses λ 1 = 0.32, λ 2 = 0.13, and shows that L I 2 (i) LS 2 (i) for i 7, LI 2 (i) LS 2 (i) for i 8. We only change the arrival rates in the above examples. However, other numerical examples show that changing other parameters may also affect the relative position of L I 2 (i). Thus the relationship between the individually optimal policy and either of the other two optimal policies can be arbitrary depending on the parameters. 7 Conclusion and Extensions In this paper we have studied the admission control problem for a two-class M/M/1 queueing system with predetermined priorities. We analyze the optimal policies under three criteria, i.e., individual optimization, class optimization, and social optimization, and show that they are characterized by either critical numbers or monotone switching curves. We also compare among different policies and show that a higher priority class customer is accepted by the class-optimal policy whenever he is accepted by the socially optimal policy, while lower priority class customers are treated the opposite way. Comparing with either socially optimal or classoptimal policies, the individually optimal policy accepts more higher priority class customers, 13

while it may accept either more or less lower priority class customers. Several extensions of the work presented here can be considered. We expect that the analysis done here will extend to the undiscounted case with the objective of minimizing the average cost for class and social optimization and total cost for individual optimization. The structural results and comparison should continue to hold. The results for the individually optimal policy will in fact simplify considerably in the total cost case. The control limits reduce to L I 1 = µr 1 h 1 and L I 2 (i) = (µ λ 1)r 2 h 2 i. We have proved the results for the socially optimal policy under assumption h 1 h 2. Both intuition and numerical experiments suggest that the results are still true when h 1 < h 2. However, it remains to prove them rigorously. The structural results for the individually optimal policy and the class-optimal policy can be extended to an M/M/s queue with classdependent service rates by using similar approaches as in the proofs of Theorem 1-4. The extension of the results for the socially optimal policy is more complicated and requires future work. An M/M/1 queue with more than two priority classes is another possible extension. Appendix Lemma 3 If v V, then T 1 v V. Proof : (a) For (12), if i L C 1 2, then T 1 v(i, j) = v(i + 1, j) v(i + 2, j) = T 1 v(i + 1, j). If i = L C 1 1, then T 1 v(i, j) = v(i + 1, j) = T 1 v(i + 1, j). If i L C 1, then T 1 v(i, j) = v(i, j) v(i + 1, j) = T 1 v(i + 1, j). (b) For (13), if i L C 1 1, then T 1 v(i, j) = v(i + 1, j) v(i + 1, j + 1) = T 1 v(i, j + 1). If i L C 1, then T 1 v(i, j) = v(i, j) v(i, j + 1) = T 1 v(i, j + 1). 14

(c) For (14), if i L C 1 2, then T 1 v(i, j + 1) + T 1 v(i + 1, j) = v(i + 1, j + 1) + v(i + 2, j) v(i + 1, j) + v(i + 2, j + 1) = T 1 v(i, j) + T 1 v(i + 1, j + 1), where the inequality follows from (14) with i replaced by i + 1. If i = L C 1 1, then T 1 v(i, j + 1) + T 1 v(i + 1, j) = v(i + 1, j + 1) + v(i + 1, j) = T 1 v(i, j) + T 1 v(i + 1, j + 1). If i L C 1, then T 1 v(i, j + 1) + T 1 v(i + 1, j) = v(i, j + 1) + v(i + 1, j) v(i, j) + v(i + 1, j + 1) = T 1 v(i, j) + T 1 v(i + 1, j + 1). (d) For (15), if i L C 1 2, then T 1 v(i, j + 1) + T 1 v(i + 1, j + 1) = v(i + 1, j + 1) + v(i + 2, j + 1) v(i + 2, j) + v(i + 1, j + 2) = T 1 v(i + 1, j) + T 1 v(i, j + 2), where the inequality follows from (15) with i replaced by i + 1. If i = L C 1 1, then T 1 v(i, j + 1) + T 1 v(i + 1, j + 1) = v(i + 1, j + 1) + v(i + 1, j + 1) v(i + 1, j) + v(i + 1, j + 2) = T 1 v(i + 1, j) + T 1 v(i, j + 2), where the inequality follows from (16) with i replaced by i + 1. If i L C 1, then T 1 v(i, j + 1) + T 1 v(i + 1, j + 1) = v(i, j + 1) + v(i + 1, j + 1) v(i + 1, j) + v(i, j + 2) = T 1 v(i + 1, j) + T 1 v(i, j + 2). 15

Lemma 4 If v V, then T 2 v V. Proof : (a) For (12), denote by a the minimizing action in T 2 v(i + 1, j), where action 0 (1) refers to rejecting (accepting) a customer, i.e., T 2 v(i+1, j) = min{ r 2 +v(i+1, j+1), v(i+1, j)} = v(i + 1, j), if a = 0, and T 2 v(i + 1, j) = r 2 + v(i + 1, j + 1), if a = 1. If a = 0, then T 2 v(i, j) = min{ r 2 + v(i, j + 1), v(i, j)} v(i, j) v(i + 1, j) = T 2 v(i + 1, j). If a = 1, then T 2 v(i, j) r 2 + v(i, j + 1) r 2 + v(i + 1, j + 1) = T 2 v(i + 1, j). (b) For (13), the proof is similar to (a). (c) For (14), denote by a 1 (a 2 ) the minimizing action in T 2 v(i, j) (T 2 v(i + 1, j + 1)). If a 1 = a 2 = 0, then T 2 v(i, j + 1) + T 2 v(i + 1, j) = min{ r 2 + v(i, j + 2), v(i, j + 1)} + min{ r 2 + v(i + 1, j + 1), v(i + 1, j)} v(i, j + 1) + v(i + 1, j) v(i, j) + v(i + 1, j + 1) = T 2 v(i, j) + T 2 v(i + 1, j + 1), where the second inequality follows from (14). The case where a 1 = a 2 = 1 can be proved similarly. If a 1 = 1, a 2 = 0, then T 2 v(i, j + 1) + T 2 v(i + 1, j) v(i, j + 1) r 2 + v(i + 1, j + 1) = T 2 v(i, j) + T 2 v(i + 1, j + 1). If a 1 = 0, a 2 = 1, following the convention that an arriving customer is accepted when the system performance is indifferent between accepting and rejecting this customer, we have v(i, j) < r 2 + v(i, j + 1), r 2 + v(i + 1, j + 2) v(i + 1, j + 1). 16

The sum of the these two inequalities gives us v(i, j) + v(i + 1, j + 2) < v(i, j + 1) + v(i + 1, j + 1). (26) Replacing j by j + 1 in (14), we get v(i, j + 2) + v(i + 1, j + 1) v(i, j + 1) + v(i + 1, j + 2). (27) Summing up (14), (15) and (27), we get v(i, j + 1) + v(i + 1, j + 1) v(i, j) + v(i + 1, j + 2), which is a contradiction to (26). Therefore, the case where a 1 = 0, a 2 = 1 does not exist. (d) For (15), denote by a 1 (a 2 ) the minimizing action in T 2 v(i + 1, j) (T 2 v(i, j + 2)). If a 1 = a 2 = 0, then T 2 v(i, j + 1) + T 2 v(i + 1, j + 1) = min{ r 2 + v(i, j + 2), v(i, j + 1)} + min{ r 2 + v(i + 1, j + 2), v(i + 1, j + 1)} v(i, j + 1) + v(i + 1, j + 1)} v(i + 1, j) + v(i, j + 2) = T 2 v(i + 1, j) + T 2 v(i, j + 2), where the second inequality follows from (15). The case where a 1 = a 2 = 1 can be proved similarly. If a 1 = 1, a 2 = 0, then T 2 v(i, j + 1) + T 2 v(i + 1, j + 1) r 2 + v(i, j + 2) + v(i + 1, j + 1) = T 2 v(i + 1, j) + T 2 v(i, j + 2). If a 1 = 0, a 2 = 1, then v(i + 1, j) < r 2 + v(i + 1, j + 1), r 2 + v(i, j + 3) v(i, j + 2). The sum of the above two inequalities gives us v(i + 1, j) + v(i, j + 3) < v(i + 1, j + 1) + v(i, j + 2). (28) Replacing j by j + 1 in (15), we have v(i, j + 2) + v(i + 1, j + 2) v(i + 1, j + 1) + v(i, j + 3). (29) 17

Summing up (15), (27), and (29), we get v(i + 1, j + 1) + v(i, j + 2) v(i + 1, j) + v(i, j + 3), which is a contradiction to (28). Therefore the case where a 1 = 0, a 2 = 1 does not exist. Lemma 5 If v V, then T 3 v V. Proof : (a) For (12), if i 1, j 0, then T 3 v(i, j) = v(i 1, j) v(i, j) = T 3 v(i + 1, j). If i = 0, j 1, then If i = 0, j = 0, then T 3 v(0, j) = v(0, j 1) v(0, j) = T 3 v(1, j). T 3 v(0, 0) = v(0, 0) = T 3 v(1, 0). (b) For (13), the proof is similar to (a). (c) For (14), if i 1, j 0, then T 3 v(i, j + 1) + T 3 v(i + 1, j) = v(i 1, j + 1) + v(i, j) v(i 1, j) + v(i, j + 1) = T 3 v(i, j) + T 3 v(i + 1, j + 1), where the inequality follows from (14) with i replaced by i 1. If i = 0, j 1, then T 3 v(0, j + 1) + T 3 v(1, j) = v(0, j) + v(0, j) v(0, j 1) + v(0, j + 1) = T 3 v(0, j) + T 3 v(1, j + 1), where the inequality follows from (16) with j replaced by j 1 and i = 0. If i = 0, j = 0, then T 3 v(0, 1) + T 3 v(1, 0) = v(0, 0) + v(0, 0) v(0, 0) + v(0, 1) = T 3 v(0, 0) + T 3 v(1, 1). 18

(d) For (15), if i 1, j 0, then T 3 v(i, j + 1) + T 3 v(i + 1, j + 1) = v(i 1, j + 1) + v(i, j + 1) v(i, j) + v(i 1, j + 2) = T 3 v(i + 1, j) + T 3 v(i, j + 2), where the inequality follows from (15) with i replaced by i 1. If i = 0, j 1, then T 3 v(0, j + 1) + T 3 v(1, j + 1) = v(0, j) + v(0, j + 1) = T 3 v(1, j) + T 3 v(0, j + 2). If i = 0, j = 0, then T 3 v(0, 1) + T 3 v(1, 1) = v(0, 0) + v(0, 1) = T 3 v(1, 0) + T 3 v(0, 2). Lemma 6 If v V, then T 1 v V. Proof : (a) For (12) and (13), the proofs are similar to part (a) of the proof of Lemma 4. (b) Since (14) is symmetric with respect to i and j, the proof of T1 preserving (14) is the same as part (c) of the proof of Lemma 4 with r 2 replaced by r 1 and i, j interchanged, e.g., replace term v(i + 1, j) by v(i, j + 1). (c) For (15), denote by a 1 (a 2 ) the minimizing action in T 1 v(i + 1, j) ( T 1 v(i, j + 2)). If a 1 = a 2 = 0, then T 1 v(i, j + 1) + T 1 v(i + 1, j + 1) v(i, j + 1) + v(i + 1, j + 1)} v(i + 1, j) + v(i, j + 2) = T 1 v(i + 1, j) + T 1 v(i, j + 2), where the second inequality follows from (15). The case where a 1 = a 2 = 1 can be proved similarly. If a 1 = 1, a 2 = 0, then T 1 v(i, j + 1) + T 1 v(i + 1, j + 1) r 1 + v(i + 1, j + 1) + v(i + 1, j + 1) r 1 + v(i + 2, j) + v(i, j + 2) = T 1 v(i + 1, j) + T 1 v(i, j + 2), 19

where the second inequality follows from the sum of (15) and (20). If a 1 = 0, a 2 = 1, then T 1 v(i, j + 1) + T 1 v(i + 1, j + 1) r 1 + v(i + 1, j + 1) + v(i + 1, j + 1) v(i + 1, j) r 1 + v(i + 1, j + 2) = T 1 v(i + 1, j) + T 1 v(i, j + 2), where the second inequality follows from (16). (d) For (20), the proof is the same as part (d) of the proof of Lemma 4 with r 2 replaced by r 1 and i, j interchanged. (e) For (21), denote by a the minimizing action in T 1 v(i + 1, j). If a = 0, then T 1 v(i, j + 1) v(i, j + 1) v(i + 1, j) = T 1 v(i + 1, j). If a = 1, then T 1 v(i, j + 1) r 1 + v(i + 1, j + 1) r 1 + v(i + 2, j) = T 1 v(i + 1, j), where the second inequality follows from (21) with i replaced by i + 1. Lemma 7 If v V, then T 2 v V. Proof : T 2 preserving inequalities (12) - (15) has been proved in Lemma 4. The proof of T 2 preserving (20) is the same as part (c) of the proof of Lemma 6 with r 1 replaced by r 2 and i, j interchanged. For (21), denote by a the minimizing action in T 2 v(i + 1, j). If a = 0, then T 2 v(i, j + 1) v(i, j + 1) v(i + 1, j) = T 2 v(i + 1, j). If a = 1, then T 2 v(i, j + 1) r 2 + v(i, j + 2) r 2 + v(i + 1, j + 1) = T 2 v(i + 1, j). 20

Lemma 8 If v V, then T 3 v V. Proof : T 3 preserving inequalities (12) - (15) has been proved in Lemma 5. For (20), if i 1, j 0, then T 3 v(i + 1, j) + T 3 v(i + 1, j + 1) = v(i, j) + v(i, j + 1) v(i 1, j + 1) + v(i + 1, j) = T 3 v(i, j + 1) + T 3 v(i + 2, j), where the inequality follows from (20) with i replaced by i 1. If i = 0, j 1, then T 3 v(1, j) + T 3 v(1, j + 1) = v(0, j) + v(0, j + 1) v(0, j) + v(1, j) = T 3 v(0, j + 1) + T 3 v(2, j), where the inequality follows from (21). If i = 0, j = 0, then T 3 v(1, 0) + T 3 v(1, 1) = v(0, 0) + v(0, 1) v(0, 0) + v(1, 0) = T 3 v(0, 1) + T 3 v(2, 0). For (21), if i 1, then T 3 v(i, j + 1) = v(i 1, j + 1) v(i, j) = T 3 v(i + 1, j). If i = 0, then T 3 v(0, j + 1) = v(0, j) = T 3 v(1, j). Proof of Theorem 8: Since L S 1 (j) is decreasing in j, we just need to prove LS 1 (0) LC 1. Denote the socially optimal expected total discounted cost by v s (i, j). optimality equations can be written as When j = 0, the v s (i, 0) = h 1 i + λ 1 min{ r 1 + v s (i + 1, 0), v s (i, 0)} + λ 2 min{ r 2 + v s (i, 1), v s (i, 0)} + µv s ((i 1) +, 0). Then L S 1 (0) = min{i : vs (i + 1, 0) v s (i, 0) > r 1 }. (30) 21

Denote the class-optimal expected total discounted cost for controller 1 by v c (i), the optimality equations can be written as v c (i) = h 1 i + λ 1 min{ r 1 + v c (i + 1), v c (i)} + µv c ((i 1) + ). Then If we can prove then the theorem follows. L C 1 = min{i : vc (i + 1) v c (i) > r 1 }. (31) v c (i + 1) v c (i) v s (i + 1, 0) v s (i, 0), (32) Apply value iteration. Let v0 c(i) = vs 0 (i, 0) = 0, i, then (32) is satisfied at iteration 0. Suppose (32) is true at iteration n, i.e., vn c (i + 1) vc n (i) vs n (i + 1, 0) vs n (i, 0). If we can show it is also true at iteration n + 1 then (32) follows by induction and the convergence of value iteration. v c n+1 (i + 1) vc n+1 (i) = h 1 + λ 1 (min{ r 1 + v c n (i + 2), vc n (i + 1)} min{ r 1 + v c n (i + 1), vc n (i)}) + µ(v c n (i) vc n ((i 1)+ )), (33) and vn+1 s (i + 1, 0) vs n+1 (i, 0) = h 1 + λ 1 (min{ r 1 + vn s (i + 2, 0), vs n (i + 1, 0)} min{ r 1 + vn s (i + 1, 0), vs n (i, 0)}) + λ 2 (min{ r 2 + vn s (i + 1, 1), vs n (i + 1, 0)} min{ r 2 + vn s (i, 1), vs n (i, 0)}) + µ(v s n (i, 0) vs n ((i 1)+, 0)). (34) To simplify notation, let D1 s = min{ r 1 + vn s (i + 2, 0), vs n (i + 1, 0)} min{ r 1 + vn s (i + 1, 0), vs n (i, 0)}, D2 s = min{ r 2 + vn s (i + 1, 1), vs n (i + 1, 0)} min{ r 2 + vn s (i, 1), vs n (i, 0)}, D s 3 = v s n (i, 0) vs n ((i 1)+, 0), D c 1 = min{ r 1 + v c n(i + 2), v c n(i + 1)} min{ r 1 + v c n(i + 1), v c n(i)}, D c 3 = v c n(i) v c n((i 1) + ). Compare D s 1 and Dc 1 first. 22

Obviously v0 c is nondecreasing and convex in i. Following similar approach as in part (c) of the proof of Lemma 4, one can show that if v c n is nondecreasing and convex in i, so is vc n+1. Therefore, if v c n (i + 1) vc n (i) > r 1, then v c n (i + 2) vc n (i + 1) > r 1. By induction hypothesis, we also have v s n (i + 1, 0) vs n (i, 0) > r 1. So D c 1 = v c n(i + 1) v c n(i) v s n(i + 1, 0) + v s n(i, 0) = D s 1. If v c n (i + 1) vc n (i) r 1 and v c n (i + 2) vc n (i + 1) > r 1, then v s n (i + 2, 0) vs n (i + 1, 0) > r 1. So D c 1 = v c n (i + 1) ( r 1 + v c n (i + 1)) = r 1 v s n (i + 1, 0) min{ r 1 + v s n (i + 1, 0), vs n (i, 0)} = Ds 1. If v c n(i + 1) v c n(i) r 1, v c n(i + 2) v c n(i + 1) r 1, and v s n(i + 1, 0) v s n(i, 0) > r 1, then v s n(i + 2, 0) v s n(i + 1, 0) > r 1, which follows from (22). Thus D c 1 = vc n (i + 2) vc n (i + 1)) r 1 < v s n (i + 1, 0) vs n (i, 0) = Ds 1. If v c n(i + 1) v c n(i) r 1, v c n(i + 2) v c n(i + 1) r 1, v s n(i + 1, 0) v s n(i, 0) r 1, and v s n(i + 2, 0) v s n(i + 1, 0) > r 1, then D c 1 = v c n(i + 2) v c n(i + 1)) r 1 = v s n(i + 1, 0) ( r 1 + v s n(i + 1, 0)) = D s 1. If v c n (i + 1) vc n (i) r 1, v c n (i + 2) vc n (i + 1) r 1, v s n (i + 1, 0) vs n (i, 0) r 1, and v s n (i + 2, 0) vs n (i + 1, 0) r 1, then D c 1 = v c n(i + 2) v c n(i + 1)) v s n(i + 2, 0) v s n(i + 1, 0) = r 1 + v s n(i + 2, 0) ( r 1 + v s n(i + 1, 0)) = D s 1. Therefore, D c 1 Ds 1. Now consider D s 2. If v s n (i + 1, 1) vs n (i + 1, 0) r 2, then v s n (i, 1) vs n (i, 0) r 2. So D2 s = r 2 + vn s (i + 1, 1) ( r 2 + vn s (i, 1)) = vs n (i + 1, 1) vs n (i, 1) 0. If v s n(i + 1, 1) v s n(i + 1, 0) > r 2 and v s n(i, 1) v s n(i, 0) r 2, then D s 2 = vs n (i + 1, 0) ( r 2 + v s n (i, 1)) r 2 > 0, which follows from (21). 23

If v s n (i + 1, 1) vs n (i + 1, 0) > r 2 and v s n (i, 1) vs n (i, 0) > r 2, then D2 s = vs n (i + 1, 0) vs n (i, 0)) 0. Therefore, D s 2 0. By induction hypothesis, D c 3 Ds 3. Combining the above results, we have vn+1 c (i + 1) vc n+1 (i) vs n+1 (i + 1, 0) vs n+1 (i, 0), thus the theorem follows. Proof of Theorem 9: We follow similar approach as in the proof of Theorem 8. Denote the socially optimal expected total discounted cost by v s (i, j). The optimality equations are Then v s (i, j) = h 1 i + h 2 j + λ 1 min{ r 1 + v s (i + 1, j), v s (i, j)} v s (i 1, j), if i 1 + λ 2 min{ r 2 + v s (i, j + 1), v s (i, j)} + µ v s (0, j 1), if i = 0, j 1 v s (0, 0), if i = j = 0. L S 2 (i) = min{j : v s (i, j + 1) v s (i, j) > r 2 }. (35) Denote the class-optimal expected total discounted cost for controller 2 by v c (i, j), the optimality equations are Then v c (i, j) = h 2 j + λ 1 { v c (i + 1, j), if i < L C 1 v c (i, j), if i L C 1 If we can show then the theorem follows. + λ 2 min{ r 2 + v c (i, j + 1), v c (i, j)} + µ v c (i 1, j), if i 1 v c (0, j 1), if i = 0, j 1 v c (0, 0), if i = j = 0. L C 2 (i) = min{j : vc (i, j + 1) v c (i, j) > r 2 }. (36) v s (i, j + 1) v s (i, j) v c (i, j + 1) v c (i, j), i, (37) Apply value iteration. Let v c 0 (i, j) = vs 0 (i, j) = vs (i, j), i, j, then (37) is satisfied at iteration 0. Suppose (37) is true at iteration n, i.e., v s n(i, j + 1) v s n(i, j) v c n(i, j + 1) v c n(i, j). If we can prove it is also true at iteration n + 1 then (37) follows by induction and the convergence of value iteration. 24

We have vn+1 c (i, j + 1) vc n+1 (i, j) { v c = h 2 + λ n (i + 1, j + 1) vn(i c + 1, j), if i < L C 1 1 vn(i, c j + 1) vn(i, c j), if i L C 1 + λ 2 (min{ r 2 + vn(i, c j + 2), vn(i, c j + 1)} min{ r 2 + vn(i, c j + 1), vn(i, c j)}) vn(i c 1, j + 1) vn(i c 1, j), if i 1 + µ vn c (0, j) vc n (0, j 1), if i = 0, j 1 0, if i = j = 0. and vn+1 s (i, j + 1) vs n+1 (i, j) = h 2 + λ 1 (min{ r 1 + vn s (i + 1, j + 1), vs n (i, j + 1)} min{ r 1 + vn s (i + 1, j), vs n (i, j)}) + λ 2 (min{ r 2 + vn s (i, j + 2), vs n (i, j + 1)} min{ r 2 + vn s (i, j + 1), vs n (i, j)}) vn s(i 1, j + 1) vs n (i 1, j), if i 1 + µ vn s (0, j) vs n (0, j 1), if i = 0, j 1 0, if i = j = 0. To simplify notation, let D s 1 = min{ r 1 + v s n(i + 1, j + 1), v s n(i, j + 1)} min{ r 1 + v s n(i + 1, j), v s n(i, j)}, D2 s = min{ r 2 + vn(i, s j + 2), vn(i, s j + 1)} min{ r 2 + vn(i, s j + 1), vn(i, s j)}, vn(i s 1, j + 1) vn(i s 1, j), if i 1 D3 s = vn(0, s j) vn(0, s j 1), if i = 0, j 1 0, if i = j = 0, { D1 c v c = n (i + 1, j + 1) vn c (i + 1, j), if i < LC 1 vn c (i, j + 1) vc n (i, j), if i LC 1, D2 c = min{ r 2 + vn c (i, j + 2), vc n (i, j + 1)} min{ r 2 + vn c (i, j + 1), vc n (i, j)}, v D3 c n c (i 1, j + 1) vc n (i 1, j), if i 1 = vn c (0, j) vc n (0, j 1), if i = 0, j 1 0, if i = j = 0. Compare D1 s and Dc 1 first. If vn(i s + 1, j) vn(i, s j) > r 1, then vn(i s + 1, j + 1) vn(i, s j + 1) > r 1, which follows from (14). So D1 s = vn(i, s j + 1) vn(i, s j) vn(i, c j + 1) vn(i, c j) D1. c Since L S 1 (j) LC 1, j, class 1 arrivals in state (i, j) with i LC 1 are always rejected by the socially optimal policy, i.e., v s (i + 1, j) v s (i, j) > r 1, i L C 1. Since vs 0 (i, j) = vs (i, j), we have v s k (i, j) = vs (i, j), k 0. Hence, v s k (i + 1, j) vs k (i, j) > r 1, k 0, i L C 1. 25

If v s n (i + 1, j) vs n (i, j) r 1 and v s n (i + 1, j + 1) vs n (i, j + 1) > r 1, the above observation yields i < L C 1. So D s 1 = v s n(i, j + 1) ( r 1 + v s n(i + 1, j)) v s n(i, j + 1) + (v s n(i + 1, j + 1) v s n(i, j + 1)) v s n(i + 1, j) = v s n (i + 1, j + 1) vs n (i + 1, j) vc n (i + 1, j + 1) vc n (i + 1, j) = Dc 1. If v s n(i + 1, j) v s n(i, j) r 1, v s n(i + 1, j + 1) v s n(i, j + 1) r 1, then i < L C 1. So D1 s = r 1 + vn s (i + 1, j + 1) ( r 1 + vn s (i + 1, j)) = v s n (i + 1, j + 1) vs n (i + 1, j) vc n (i + 1, j + 1) vc n (i + 1, j) = Dc 1. Therefore, D1 s Dc 1. Now consider D2 s and Dc 2. If vn(i, s j + 1) vn(i, s j) > r 2, then vn(i, s j + 2) vn(i, s j + 1) > r 2, which follows from (16). By induction hypothesis, we have vn(i, c j + 1) vn(i, c j) > r 2 and vn(i, c j + 2) vn(i, c j + 1) > r 2. So D s 2 = vs n (i, j + 1) vs n (i, j) vc n (i, j + 1) vc n (i, j) = Dc 2. So If v s n (i, j+1) vs n (i, j) r 2 and v s n (i, j+2) vs n (i, j+1) > r 2. Then v c n (i, j+2) vc n (i, j+1) > r 2. D s 2 = v s n(i, j + 1) ( r 2 + v s n(i, j + 1)) = r 2 v c n(i, j + 1) min{ r 2 + v c n(i, j + 1), v c n(i, j)} = D c 2. If v s n (i, j + 1) vs n (i, j) r 2, v s n (i, j + 2) vs n (i, j + 1) r 2, v c n (i, j + 1) vc n (i, j) > r 2, then v c n (i, j + 2) vc n (i, j + 1) > r 2, which follows from (16). So D s 2 = v s n(i, j + 2) v s n(i, j + 1)) r 2 < v c n(i, j + 1) v c n(i, j) = D c 2. If v s n(i, j + 1) v s n(i, j) r 2, v s n(i, j + 2) v s n(i, j + 1) r 2, v c n(i, j + 1) v c n(i, j) r 2, and v c n(i, j + 2) v c n(i, j + 1) > r 2, then D s 2 = v s n (i, j + 2) vs n (i, j + 1)) r 2 = v c n (i, j + 1) ( r 2 + v c n (i, j + 1)) = Dc 2. 26

If v s n (i, j + 1) vs n (i, j) r 2, v s n (i, j + 2) vs n (i, j + 1) r 2, v c n (i, j + 1) vc n (i, j) r 2, and v c n (i, j + 2) vc n (i, j + 1) r 2, then D s 2 = r 2 + v s n(i, j + 2) ( r 2 + v s n(i, j + 1)) r 2 + v c n(i, j + 2) ( r 2 + v c n(i, j + 1)) = D c 2. Therefore, D s 2 Dc 2. By induction hypothesis, D s 3 Dc 3. Combining the above results, we have vn+1 s (i, j + 1) vs n+1 (i, j) vc n+1 (i, j + 1) vc n+1 (i, j), thus the theorem follows. 27

10 9 α=0.05, µ=0.5, λ 1 =0.44, λ 2 =0.01, h 1 =20, h 2 =10, r 1 =200, r 2 =190 L1I L1C L1S(j) 8 7 control limits 6 5 4 3 2 1 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 number of class 2 customers (j) Figure 1: L S 1 (j) LC 1 LI 1 control limits 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 α=0.05, µ=0.5, λ 1 =0.44, λ 2 =0.01, h 1 =20, h 2 =10, r 1 =200, r 2 =190 L2I(i) L2C(i) L2S(i) 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 number of class 1 customers (i) Figure 2: L I 2 (i) LC 2 (i) LS 2 (i) 28

control limits 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 α=0.05, µ=0.5, λ 1 =0.39, λ 2 =0.06, h 1 =20, h 2 =10, r 1 =200, r 2 =190 L2I(i) L2C(i) L2S(i) 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 number of class 1 customers (i) Figure 3: L C 2 (i) LI 2 (i) LS 2 (i) control limits 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 α=0.05, µ=0.5; λ 1 =0.27, λ 2 =0.18, h 1 =20, h 2 =10, r 1 =200, r 2 =190 L2I(i) L2C(i) L2S(i) 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 number of class 1 customers (i) Figure 4: L C 2 (i) LS 2 (i) LI 2 (i) 29

control limits 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 α=0.05, µ=0.5, λ 1 =0.41, λ 2 =0.04, h 1 =20, h 2 =10, r 1 =200, r 2 =190 L2I(i) L2C(i) L2S(i) 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 number of class 1 customers (i) Figure 5: L I 2 (i) LC 2 (i) for i 4, LI 2 (i) LC 2 (i) for i 5 control limits 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 α=0.05, µ=0.5, λ 1 =0.32, λ 2 =0.13, h 1 =20, h 2 =10, r 1 =200, r 2 =190 L2I(i) L2C(i) L2S(i) 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 number of class 1 customers (i) Figure 6: L I 2 (i) LS 2 (i) for i 7, LI 2 (i) LS 2 (i) for i 8 30

Acknowledgement We would like to thank Professor Eylem Tekin for her helpful comments and suggestions. References [1] I. Adiri, and U. Yechiali, Optimal priority-purchasing and price decisions in nonmonopoly and monopoly queues, Opns. Res. 22 (1974), 1051-1066. [2] J. Blanc, and P.R. De Waal, Optimal control of admission to a multiserver queue with two arrival streams, IEEE Trans. Automat. Control 37 (1992), 785-797. [3] A. Ha, Stock-rationing policy for a make-to-stock production system with two priority classes and backordering, Naval Res. Logist. 44 (1997), 457-472. [4] R. Hassin, Decentralized regulation of a queue, Mgmt. Sci. 41 (1995), 163-173. [5] N.C. Knudsen, Individual and social optimization in a multi-server queue with a general cost-benefit structure, Econometrica 40 (1972), 515-528. [6] V.G. Kulkarni, Modeling and analysis of stochastic systems, Chapman and Hall, New York, 1995. [7] V.G. Kulkarni, and T.E. Tedijanto, Optimal admission control of markov-modulated batch arrivals to a finite-capacity buffer, Stochastic Models 14 (1998), 95-122. [8] S. Lippman, Applying a new device in the optimization of exponential queueing systems, Opns. Res. 23 (1975), 687-710. [9] S. Lippman, and S. Stidham, Individual versus social optimization in exponential congestion systems, Opns. Res. 25 (1977), 233-247. [10] H. Mendelson, and S. Whang, Optimal incentive-compatible priority pricing for the m/m/1 queue, Opns. Res. 38 (1990), 870-883. [11] B. Miller, A queueing reward system with several customer classes, Mgmt. Sci. 16 (1969), 234-245. [12] S.K. Nair, and R. Bapna, An application of yield management for internet service providers, Nav. Res. Log. 48 (2001), 348-362. 31

[13] P. Naor, The regulation of queue size by levying tolls, Econometrica 37 (1969), 15-24. [14] M. Opp, V.G. Kulkarni, and K. Glazebrook, Outsourcing warranty repairs: Dynamic allocation, Nav. Res. Log. 52 (2005), 381-398. [15] M.L. Puterman, Markov decision processes, Wiley, New York, 1994. [16] V.V. Rykov, Monotone control of queueing systems with heterogeneous servers, Queueing Syst. 37 (2001), 391-403. [17] S. Stidham, Socially and individually optimal control of arrivals to a GI/M/1 queue, Mgmt. Sci. 24 (1978), 1598-1610. [18] S. Stidham, Optimal control of admission to a queueing system, IEEE Trans. Automat. Control 30 (1985), 705-713. [19] S. Stidham, and R.R. Weber, Monotonic and insensitive optimal policies for control of queues with undiscounted costs, Opns. Res. 87 (1989), 611-625. [20] U. Yechiali, On optimal balking rules and toll charges in a GI/M/1 queuing process, Opns. Res. 19 (1971), 349-370. [21] U. Yechiali, Customers optimal joining rules for the GI/M/s queue, Mgmt. Sci. 18 (1972), 434-443. 32