ON THE OPTIMALITY OF LEPT AND µc RULES FOR PARALLEL PROCESSORS AND DEPENDENT ARRIVAL PROCESSES

Transcription

1 ON THE OPTIMALITY OF LEPT AND µc RULES FOR PARALLEL PROCESSORS AND DEPENDENT ARRIVAL PROCESSES Arie Hordijk & Ger Koole* Dept. of Mathematics and Computer Science, Universit of Leiden P.O. Box 9512, 2300 RA Leiden, the Netherlands ABSTRACT In this paper we stud scheduling problems of multiclass customers on identical parallel processors. A new tpe of arrival process, called a Markov Decision Arrival Process, is introduced. This arrival process can be controlled and allows for an indirect dependence on the numbers of customers in the queues. As a special case we show the optimalit of LEPT and the µc-rule in the last node of a controlled tandem network for various cost structures. A unifing proof using dnamic programming is given. STOCHASTIC SCHEDULING; DYNAMIC PROGRAMMING; DEPENDENT ARRIVALS Classification: 90B22 primar, 90C40 secondar. Published in: Advances in Applied Probabilit 25: , INTRODUCTION In this paper we consider stochastic scheduling problems on s identical parallel processors. There are m classes of customers, each class has its own queue and the service time of a customer of class k, k = 1,..., m is exponentiall distributed with rate µ k. Arrivals occur according to a Markov Decision Arrival Process (MDAP). This arrival process generates customers with intensities that do not depend on the state of the queues. However, actions can be taken in the MDAP. The control, consisting of an action in the MDAP and the assignment of the servers, ma depend on the state of the MDAP and the numbers of customers in the queues. This induces a dependence between the arrival process and the state of the queues. The MDAP is especiall designed to model the arrivals in the last center of a controlled tandem network of service centers. With the MDAP we can also control the availabilit of the servers. Besides dependent arrival processes and machine breakdowns and repairs it is possible to model the usual independent arrivals and breakdowns and repairs. It is shown that LEPT (the polic that processes customers in the sstem in decreasing order of expected processing time) is optimal in the class of preemptive policies if the cost function satisfies two conditions; one is the monotonicit in the numbers of customers in the queues and the other requires that the service rate multiplied b the marginal cost of a customer is decreasing in the same order as the expected processing times. We verif these conditions for various objective functions such as the indicator functions of the makespan or the first time to an empt sstem. Hence LEPT stochasticall minimizes the makespan or bus period. We show that the µc-rule minimizes the expected weighted sum of customer completion times under the agreeabilit condition that LEPT and the µc-rule have the same priorit list. This result gives the optimalit of the µc-rule in the last node of a tandem network. We also analze the optimalit of the * Present address: C.W.I., Kruislaan 413, 1098 SJ Amsterdam 1

2 µc-rule without the agreeabilit condition and we give counterexamples showing the nonoptimalit of the µc-rule even in the case of one server. However, in some special cases it is optimal in expectation and in distribution. In section 2 we introduce the MDAP, which generates the arrival epochs and the server vacation times. Also we show that the MDAP can be used to model: phase tpe renewal processes, Markov modulated Poisson processes, finite numbers of independent arrivals (the release date model as in Weber [20] and Chang et al. [6]), the first center of a tandem sstem, and controlled availabilit of servers. We prove our results (including optimalit in stochastic sense for several models) using dnamic programming (dp) and uniformization. In section 3 we derive the dp recursion with respect to the discrete-time variable of the uniformized process. Also in section 3 we formulate the main lemma which gives two sufficient conditions guaranteeing that LEPT and, under the agreeabilit condition the µc-rule, are optimal. The proof of the lemma is given in section 6. In section 4 cost functions are studied which satisf the sufficient conditions. The cost functions of the first tpe are related to emptiness of the sstem, for example the length of the bus period and the makespan. We show that LEPT is optimal in stochastic sense. The cost functions of the second tpe have linear cost rates c j, j = 1,..., m. LEPT is optimal for this tpe of cost function when the agreeabilit condition holds, i.e. when LEPT coincides with the µc-rule (the polic that serves customers in decreasing order of µ j c j ). In section 5 the optimalit of the µc-rule without the agreeabilit condition is studied. In subsection 5.2 it is shown for the multiple server case that there is no unique optimal polic. We give an example for which SEPT is better than the LEPT for T small and vice versa for T large. In subsection 5.1 we suppose that there is onl one server. It is well-known that the µc-rule is optimal for one server and an independent arrival process (cf. Buukkoc et al. [5] and Baras et al. [3]). We show that this result also follows from our main lemma. Also a necessar and sufficient condition is given for the optimalit in distribution of the µc-rule. With the counterexample of Hordijk & Koole [8] we show that the µc-rule is not optimal in the last center of a tandem network, when the agreeabilit condition is not satisfied. The optimalit of LEPT with increasing generalit is shown for the makespan in the papers b Pinedo & Weiss [14], Van der Heden [7], Bruno et al. [4], Weiss & Pinedo [24], Weiss [23], Weber [20], Weber [21] and Chang et al. [6]. All these papers assume an arrival process which is independent of the numbers of customers in the queues, our extension is the MDAP as arrival process, which shows the optimalit of LEPT in the last service sstem of a tandem network. Results on the optimalit of the µc-rule for one server and an independent arrival process can be found in Pinedo [13], Baras et al. [2], Baras et al. [3], Buukkoc et al. [5], Shanthikumar & Yao [18] and Righter & Shanthikumar [15]. Assuming the agreeabilit condition the optimalit of the µc-rule for the multi-server case is proved in papers b Ross [17], Kämpke [10], Weber [22] and Chang et al. [6]. The extension of this paper is the possible dependence of the arrival process on the queue sizes. Likel the most interesting application is again the tandem network for which we show that the µc-rule is optimal in the last node. New is also our unifing proof using dnamic programming. 2

3 2. THE ARRIVAL PROCESS We start this section b introducing the MDAP and we illustrate the MDAP with three special cases. Then we introduce the MDAP with server vacations and we give two examples. Let us recall that there are m customer classes and each class has its own queue. There are s servers and µ j is the service rate of a customer in queue j. Without restriction of generalit we assume that µ 1 µ m, i.e. LEPT coincides with the Smallest Index Polic (SIP). With i j we denote the number of customers in queue j, i = (i 1,..., i m ) Definition. (Markov Decision Arrival Process) Let Λ be the, possibl countable, state space of a Markov Decision Process with transition intensities λ xa with x, Λ and a the action chosen in state x. The finite set of possible actions in x is denoted b A(x). An arrival in class k occurs with probabilit q k xa when action a is chosen in x and a transition from x to occurs. We assume that arrivals cannot happen simultaneousl, hence m qk xa 1 for all x, and a. This arrival process can model man interesting problems. We will give three special cases. In the first two cases we assume that there is one action in each state of the MDAP and we suppress its notation. Phase tpe renewal processes. Assume we have a renewal process with independent interarrival times of phase tpe, as discussed in Neuts [12]. Phase tpe distributions are defined as follows. We have a Markov process with m + 1 states, where state m + 1 is absorbing, the other m states are transient. The transition intensit from state x to is denoted b t x, α x is the probabilit that the sstem starts in state x. The time until absorption has a phase tpe distribution. Assume α m+1 = 0, i.e. there is no atom at 0. To model this renewal process with an MDAP, we take the parameters as follows: Λ = {1,..., m}, λ x = t x + t xm+1 α and q x = (t xm+1 α )/(t x + t xm+1 α ). Note that the MDAP is immediatel restarted when the state of the Markov process moves to m + 1. When this happens a new independent interarrival time is started, and with probabilit α the starting state is. Markov Modulated Poisson Process. An MMPP is governed b a Markov process with state space Λ and transition intensities λ x. When the sstem is in state x customers arrive with intensit µ x. As this does not change the arrival process we ma assume λ xx = 0 for all x. In order to model this process as an MDAP we take: { Λ = Λ, µx if x =, λ x = otherwise λ x and q x = { 1 if x =, 0 otherwise. The MMPP is often used as it is eas to implement. However, the departure process of the M M 1 queue with a finite buffer can not be modeled with it as a departure and a change of state cannot occur simultaneousl in the MMPP. In Asmussen & Koole [1] it is shown that an independent arrival process with multiple classes of customers can be approximated arbitraril closel b a MDAP with one action in each state. Controlled tandem sstem of two service centers. Consider two service centers in tandem. Suppose that there are m customer classes and that customers of class k arrive at queue k of the first center according to a Poisson process with rate λ k. After being served in the first center a customer of tpe k joins queue 3

4 k of the second center. There are s 1 respectivel s 2 servers available in the first respectivel the second center. After being served in the second center the customers leave the sstem. We are interested in the optimal assignment polic of the servers to the queues in the last center. Therefore we model the first center as MDAP. With i 1 j we denote the number of customers in queue j of center one, then Λ, the set of states of the arrival process, is equal to {(i 1 1,..., i 1 m), i k 0, k = 1,..., m}. The assignment action a in state x = (i 1 1,..., i 1 m) consists of the numbers of the queues which are served b the s 1 servers, sa a = (l 1,..., l s1 ). The transition rates are as follows: λ xa(x+ek ) = λ k and q xa(x+ek ) = 0, for all x, a and k = 1,..., m, λ xa(x ek ) = µ k and q xa(x ek ) = 1 if queue k is served when action a is chosen in state x, i.e. l k a. In the following section we will see that the minimizing action a in the first center ma depend on x as well as on i = (i 1,..., i m ), the numbers of customers in the queues of the second center. Similarl, the assignment of the servers in the second center ma depend on the number of customers in all the queues of both centers. Hence the optimalit of LEPT, and the µc-rule under the agreeabilit condition, as shown in section 3 is valid for the second center in a tandem sstem and the optimalit holds in the class of policies which ma depend on the information of all queue sizes of both centers. This optimalit result for the tandem sstem can be generalized to an network in which we focus on the optimal assignment of servers to queues in a labeled node. The condition on the network is that jobs leaving the labeled node do not enter the network. For the optimalit of the µc-rule in the last node of a network an extra condition (the agreeabilit condition) is necessar (see the counterexample in section 5.1). An extra condition is not needed in the case of a tandem sstem in which customers are assigned to queues is analzed. For this model it is shown in Hordijk & Koole [9] that the structure of the optimal polic in the second center is the same as in the case of one service center. The MDAP can also be used to model extra events which ma be stochasticall dependent on the arrival process. In this paper we do this for server vacations. Assume that an event with server k is the start of a vacation when the server is currentl working, and the end of a vacation if the server is on vacation. The state of the sstem is not completel described b x and i; the state of the servers is also necessar. Therefore we need extra state variables z 1,..., z s. We take z k = 1 if server k is working, 0 otherwise. In this paper we are interested in the assignment of the available servers, not in (optimall) scheduling their vacations. Therefore we take as state of the arrival process (x, z) instead of just x. With apologizes for the possible confusion we denote the extended state variable of arrival process again b x. The transition intensities and arrival probabilities of this new arrival process are easil found when the servers vacations are independent of the arrivals (we give an example below). Notice that the state of the MDAP defines the servers which are available uniquel. Hence a MDAP with server vacations can be modeled as: 2.2. Definition. (Markov Decision Arrival Process with server vacations) Let Λ be the, possibl countable, state space of a Markov Decision Process with transition intensities λ xa with x, Λ and a the action chosen in state x. The finite set of possible actions in x is denoted b A(x). An arrival in class k occurs with probabilit q k xa when action a is chosen in x and a transition from x to occurs. There are sets Λ 1,..., Λ s, Λ k Λ, such that server k is available iff the state of the process is in Λ k. 4

5 Arrivals and server vacations. To illustrate the MDAP with server vacations we give a simple example, with one arrival process, one customer class and one server. Assume that the arrivals have an exponential interarrival time with rate α 1 and that the server vacations are exponentiall distributed with rate α 2 if action 1 is chosen and rate α 3 if action 2 is chosen. We assume that the uptime of the server has rate α 4. To model the arrivals we use the first component of the state variable x of the MDAP. Let this component have one value, sa 1. When this component jumps from 1 to 1 an arrival occurs. To model the vacations we use the second component of the state, sa it has two values, 1 and 2, and we assume that the server is working iff the second component is 2. In order to define the appropriate MDAP, we take Λ = {(1, 1), (1, 2, 2 actions in state (1, 1), and 1 action in (1, 2). The transition rates are: λ (1,1)a(1,1) = λ (1,2)1(1,2) = α 1 for a = 1 or 2, λ (1,1)1(1,2) = α 2, λ (1,1)2(1,2) = α 3 and λ (1,2)1(1,1) = α 4. As non-zero arrival probabilities we take: q(1,1)a(1,1) 1 = q1 (1,2)1(1,2) = 1 for a = 1 or 2, and q1 (1,2)1(1,2) = 1. We take Λ 1 = {(1, 2, i.e. the server is available iff the state is (1, 2). In this example the server vacation times are independent of the arrival epochs. It is eas to change the λ s such that there is dependence between arrivals and server vacations. Note that the optimalit results of the next section remain true when there is dependence. Arrivals and controlled availabilit of servers. In this example the customers arrive independentl. However, the availabilit of servers can be controlled. The simplest model is when there is a decision to be made for each server, and each server has exponentiall distributed up and down times. Action 0 means that the server will go or sta on vacation, action 1 means that the server will be working the next period. The availabilit of a single server can be modeled in the following wa: Λ = {0, 1}, Λ 1 = {1}, λ 100 = λ 000 = λ 111 = λ 011 = λ, for an arbitraril chosen λ. The superposition of s MDAP s of this tpe gives again an MDAP. Hence we can control the availabilit of each of the servers. Costs can be associated with the availabilit of each server, for example one can model the case of additional servers with higher cost rates than the normal servers. When λ is taken large relative to the service rates the MDAP approximates the model in which we can instantaneousl change the set of working servers. It is possible to combine various MDAP s, the superposition of a finite number of MDAP s is again a MDAP. This implies that our optimalit results of the next section are valid under rather general conditions of the arrival process and server vacation times. B continuit arguments the hold for an arbitrar arrival process and generall distributed server vacation times. The are also true in the last node of a network of service centers. 5

6 3. OPTIMALITY OF LEPT In this section we consider a service center with a MDAP as arrival process. We recall that there are m customer classes and each class has its own queue. Without restriction of generalit we index the class numbers such that µ 1 µ m. With i j we denote the number of customers in queue j, i = (i 1,..., i m ). There are in total s servers and the available servers have to be assigned to waiting customers. We show in this section that an optimal polic assigns servers to customers according to priorities that are decreasing in their indices. Hence LEPT, which processes customers in decreasing order of expected processing time, is optimal. Note that a polic in our model with a MDAP contains also a rule which gives the actions to be taken in the MDAP. Our optimalit result does not specif this rule. We start this section with the derivation of the dnamic programming recursion. This recursion and the proof of the inequalities (3.2) and (3.3) for the value functions are given with respect to a discrete-time variable. We use the uniformization method, i.e. we consider the jump epochs of a Poisson process with parameter γ and we restrict the class of policies to policies that allow preemption onl at these jump epochs. Using continuit arguments as γ tends to infinit our optimalit results hold for a large class of policies in continuous time. A rigorous proof of the optimalit of LEPT in continuous time is given in Koole [11]. Let v(x,i) n denote the minimal expected cost at the nth jump epoch of the Poisson process, given the starting state is (x, i), where x is the state of the MDAP and i gives the queue sizes. We denote b v 0 the cost function. Our main result, lemma 3.1, sas that the value functions v n satisf the inequalities (3.1) and (3.2) if the hold for v 0. The conditions (3.1) and (3.2) on the cost functions are not new. The have been used b Weiss & Pinedo [24] and recentl b Chang et al. [6]. Since we want to uniformize our continuous-time model, the rates in each state of the MDAP have to be bounded for each action. Thus we assume λ xa γ for all x and a A(x). This is no severe restriction, as most applications and also the approximations of Asmussen & Koole [1] satisf this assumption. Now the rates of the (x, i) -process are bounded b γ + sµ m. In the uniformization we need that the parameter γ is bigger than this sum. However, in order to keep the notation simple we take without restriction of generalit γ equal to one. We continue b giving the dnamic programming equation of the discrete-time model. Let s(x) be the number of servers available in state x of the MDAP. B adding a transition from x to x without arrivals we can assume λ xa to be constant for different a. Take µ = µ m. An assignment action in (x, i) consists of the s(x) class numbers to which the servers are assigned. We allow that servers are idle, sa a server is assigned to a dumm class 0 with rate µ 0 = 0 if it is idle, and we assume that no more servers are assigned to a class then there are customers in that class (we call this an admissible server assignment). The value function becomes: { (x,i) = min ( m λ xa qxav j (,i+e n + (1 m j) v n+1 a q j xa)v n (,i) { s(x) ( min µ lk v(x,i e n l lk ) + (µ µ l k )v(x,i) n + 1,...,l s(x) ( 1 ) λ xa s(x)µ v(x,i) n, a A(x). + (3.1) 6

7 To make the action unique we assume that l 1 l s(x). Denote with i the number of customers in state i, i.e. i = m i j Lemma. If µ j1 w (x,i ej1 ) + (µ µ j1 )w (x,i) µ j2 w (x,i ej2 ) + (µ µ j2 )w (x,i) (3.2) for 0 < j 1 < j 2 and i j1, i j2 > 0, and w (x,i ej1 ) w (x,i) for i j1 > 0 (3.3) hold for the cost function w = v 0, then the hold for all v n. We remark that if µ 1 = 0 then (3.2) and (3.3) are contradicting. Therefore we assume µ 1 > 0. The proof of lemma 3.1 can be found in section 6. In Koole [11] a similar result is obtained for the slightl more general model with partial availabilit of servers. If the number of customers is s(x) + 1 or more and if j 1,..., j s(x) and j1, j 2,..., j s(x) with j 1 < j1 are admissible server assignments, then it follows from (3.2): s(x) s(x) µ jk v(x,i e n jk ) + (s(x)µ µ jk )v(x,i) n s(x) s(x) µ j 1 v(x,i e n j ) + µ jk v(x,i e n jk ) + (s(x)µ µ j1 µ jk )v(x,i) n. 1 k=2 k=2 Hence (3.2) gives that LEPT is optimal in the class of nonidling policies. Since (3.3) sas that an optimal polic is nonidling, we have shown that LEPT is optimal in the class of policies which ma have idling servers Theorem. LEPT minimizes for all T the expected costs at T for all cost functions satisfing (3.2) and (3.3). B adding immediate costs in the dp equation, which satisf the conditions (3.2) and (3.3), we obtain similar results for the total costs from 0 to T. 7

8 4. COST FUNCTIONS In this section we give examples of cost functions satisfing (3.2) and (3.3). Define j v 0 (x,i) = v0 (x,i+e j) v0 (x,i). Then the conditions in lemma 3.1 are equivalent to µ j1 j1 v 0 (x,i e j1 ) µ j 2 j2 v 0 (x,i e j2 ) if j 1 < j 2 and i j1, i j2 > 0, (4.1) and j v(x,i) 0 0 j. (4.2) Makespan. The simplest nontrivial cost function satisfing these conditions is: v 0 (x,i) = I { i }. The expected cost of this function is the probabilit that there are customers at jump epoch n. In case there are no arrivals we find from theorem 3.2 that LEPT minimizes the makespan stochasticall. Bus period. We can modif the dnamics of our sstem such that it remains empt once it becomes empt, b taking v n+1 (x,0) = min a{ λ xav(,0) n } + (1 λ xa )v(x,0) n. It is eas to see that lemma 3.1 still holds for this model. Hence we find that LEPT stochasticall minimizes the bus period. Makespan or bus period for a tandem sstem. We consider a tandem sstem and we are interested in the time epoch that the whole sstem becomes empt. Here we take v(x,i) 0 = I { i >0 or x >0}, where x is the state of the first center. Theorem 3.2 gives that LEPT in the second center stochasticall minimizes the time epoch of first emptiness of the whole sstem. Note that we have no result for the optimal polic in the first center. Makespan in the release date model. In the construction of Asmussen & Koole [1] the number of alread arrived customers follows from the state of the arrival process. In the case of dependent arrivals we can add an extra component to the state of the arrival process giving the number of arrivals, like we added variables indicating the availabilit of servers. Now let Λ k Λ denote the set of states for which the number of arrivals is k or more. B taking v(x,i) 0 = I { i >0 or x Λ k } we can stud the first time epoch after the kth arrival at which the sstem becomes empt. If there are no arrivals after the kth we have that LEPT stochasticall minimizes the makespan in the release date model. This result for an independent arrival process was shown in Weber [20] and Chang et al. [6]. Expected weighted number of customers. We get the expected weighted number of customers with weights, c 1,..., c m, if we take: v 0 (x,i) = m c ji j. It is eas to see that this function satisfies the inequalities (3.2) and (3.3) iff µ 1 c 1 µ m c m 0. (4.3) As in Chang et al. [6] we call (4.3) the agreeabilit condition. Thus under the agreeabilit condition, LEPT minimizes the expected weighted number of customers at an time. However, the µc-rule is known to be optimal in some models without agreeabilit condition. The obvious question is: in which models can we omit the assumption 0 < µ 1 µ m and also for which models is the µc-rule optimal in distribution? These questions will be addressed in the next section. 8

9 Expected weighted number of late customers; expected weighted sum of customer tardiness. In Chang et al. [6] it is shown that the basic inequalities (3.2) and (3.3) impl under the agreeabilit condition that LEPT is also optimal for these objective functions. With the same arguments it can be shown that these optimalit results are also true for a MDAP. Hence the hold in the last service center of a tandem network. 5. OPTIMALITY OF THE µc-rule In this section we consider the optimalit of the µc-rule for various preemptive stochastic scheduling models with a single (multiple) server and arrival processes which are independent (dependent) of the numbers of customers in the queues. We show that the µc-rule is optimal in expectation and in stochastic sense for some models. For other models we give counterexamples showing that the µc-rule is not optimal. We do not assume µ 1 µ m in this section. The results for one server are derived in the first subsection, the negative results in case of two or more servers can be found in the second subsection One server. In this subsection we consider two cases: Case 1. The arrival process is independent of the numbers of customers in the queues. For an independent arrival process we ma take a MDAP with one action in each state, as in section 2 we suppress its notation. Also server vacations or breakdowns and repairs of the server can be modeled as in section 2. Without restriction of generalit we can take λ x + µ 1. Then the dnamic recursion equation (3.1) reduces to: v n+1 (x,i) = ( m λ x qxv j (,i+e n + ( m j) 1 min l {µ l v n (x,i e l ) + (µ µ l)v n (x,i) } + (1 qx j ) ) v n (,i) + λ x µ)v n (x,i). It is eas to verif that the proof of 3.1 does not need the condition µ 1 µ m in case of one server and independent arrivals (see the remarks in the proof of section 6). Hence we find: 5.1. Lemma. If and µ j1 w (x,i ej1 ) + (µ µ j1 )w (x,i) µ j2 w (x,i ej2 ) + (µ µ j2 )w (x,i) (5.1) hold for the cost function w = v 0, then the hold for all v n. for j 1 < j 2 and i j1, i j2 > 0 w (x,i ej) w (x,i) for i j > 0 (5.2) The inequalities (5.1) and (5.2) completel determine the optimal polic. From (5.2) it follows that the server ma not idle when there are customers in the sstem. According to (5.1) the optimal polic processes customers in the sstem in decreasing order of their class number. 9

10 Without restriction of generalit we ma assume that the class numbers have a decreasing value of µc, i.e. µ 1 c 1 µ m c m. Next we consider different cost functions and we analze for which the inequalities (5.1) and (5.2) hold. Expected weighted number of customers. For this case of linear cost rates it is shown in Baras et al. [3], Buukkoc et al. [5] and Walrand [19] that the µc-rule is optimal. This result follows easil from lemma 5.1. Indeed, v 0 (x,i) = m c ji j satisfies the conditions (5.1) and (5.2). Thus the µc-rule, which processes customers in decreasing order of their class number, is optimal. B using a MDAP with server vacations we find that the result is also true in case of server breakdowns and repairs. Linear cost rates, optimalit in distribution. Since the probabilit of an event is the expectation of its indicator, the cost function I m { cjij>k} for different k can be used to find a polic which is optimal in distribution. We distinguish three cases. The µc-rule is optimal in distribution onl in the first case. For the other cases we give counterexamples showing nonoptimalit in distribution. 1. µ j, c j. In this case the I m { cjij>k} satisfies for all k the conditions (5.1) and (5.2). Indeed, we have v(x,i e 0 j1 ) v0 (x,i e j2 ) v0 (x,i), hence j 1 v(x,i e 0 j1 ) j 2 v(x,i e 0 j2 ). Together with µ j 1 µ j2 we have (5.1) and (5.2)and the stochastic optimalit of the µc-rule. In case there are no arrivals this result also follows from Righter & Shanthikumar [16], b taking, in their notation, f j (C j ) = c j I {Cj>T }. 2. j 1 < j 2 such that µ j1 < µ j2. Take m = 2, no arrivals, i = (1, 1), µ 1 = 1, c 1 = 5, µ 2 = 2 and c 2 = 2. The µc-rule gives priorit to class 1. However, a polic which minimizes for some t, IP(N 1 (t)c 1 +N 2 (t)c 2 6) with N k (t), k = 1, 2, the number of customers in queue k, serves the customer in class 2 first. 3. j 1 < j 2 such that c j1 < c j2. Take m = 2, no arrivals, i = (1, 1), µ 1 = 4, c 1 = 1, µ 2 = 1 and c 2 = 3. Again the µc-rule gives priorit to class 1, but a polic which minimizes for some t, IP(N 1 (t)c 1 +N 2 (t)c 2 2), serves the customer in class 2 first. Thus the µc-rule is stochasticall optimal if and onl if µ 1 µ m and c 1 c m. Note the contrast with the condition on the service rates in lemma 3.1. Here SEPT coincides with the µc-rule. Intuitivel, it seems that the µc-rule is stochastic optimal in case of dependent arrivals if LEPT and SEPT coincide, i.e. if µ 1 = = µ m. Expected weighted number of late customers; expected weighted sum of customer tardiness. Similar as in Chang et al. [6] we can show that the basic inequalities (5.1) and (5.2) impl the optimalit of the µc-rule with respect to these criteria. Case 2. The arrival process is dependent of the numbers of customers in the queues. In Hordijk & Koole [8] a counterexample is given for a tandem sstem of two centers and linear cost rates, such that the optimal polic with respect to the total expected cost does not follow the µc-rule in the last center. Hence there is a t such that the expected cost at t is not minimized b the µc-rule. This shows that in general the µc-rule is not optimal in the case of a dependent arrival process. For completeness we give here the counterexample. There are 3 customers present, one in the first queue of node 1 and one in each of the queues of node 2. The parameters of the exponential distributions and the holding cost rates are given in figure 1. 10

11 center 1 center µ c. Figure 1 µ 1 c It is shown in Hordijk & Koole [8] that the optimal polic starts with serving the customer in the second queue (the queue with rate µ = 1) of center 2. After this customer has left the service of the customer in the first queue starts in center 1 and the same in center Two or more servers In this subsection we consider a preemptive stochastic scheduling problem with two or more servers, i.e. two or more parallel machines, and an arrival process which is independent of the numbers of customers in the queues. We give a counterexample for which the optimal polic depends on the time horizon. Thus the µc-rule is not optimal in this problem. Hence, we conclude that also in the more general case of a dependent arrival process the µc- rule is not optimal. Counterexample. We consider the model with s = 2, m = 2, µ 1 = 2, µ 2 = 1 and c 1 = c 2 = 1. There are no arrivals, we start with 2 (1) customers in queue 1 (2), i.e. i 1 = 2 and i 2 = 1. The objective function is the expected number of customers at T. The onl work-conserving policies are LEPT which starts serving a customer of class 1 and the onl customer in class 2, and SEPT which starts with both class 1 customers. In the continuous-time model it is eas to compute the expected number of departures before time T, sa L, with the formula below. Let α 1 and α 2 be the service rates of the customers which are served first, α 3 is the service rate of the other customer. Note that α 3 < α 1 + α 2. α 1 α 1 + α 2 α 2 α 1 + α 2 T 0 L = T 0 T 0 T 0 (α 1 + α 2 )e (α1+α2)t dt+ (α 1 + α 2 )e (α1+α2)t (1 e α2(t t) )dt+ (α 1 + α 2 )e (α1+α2)t (1 e α1(t t) )dt+ (α 1 + α 2 )e (α1+α2)t (1 e α3(t t) )dt = 1 e (α1+α2)t e (α1+α2)t e α1t e α2t + 1 e (α1+α2)t α 1 + α 2 α 1 + α 2 α 3 e α3t (1 e (α1+α2 α3)t ). The terms on the first line in the last expression give the probabilit that the first departure takes place before T. The sum of the terms on the second line is equal to (1 e α1t )(1 e α2t ), which is the probabilit that both customers scheduled first finish their service before T. The last term concerns the customer scheduled last. 11

12 Using a small computer program we computed L for LEPT (α 1 = 1, α 2 = α 3 = 2) and SEPT (α 1 = α 2 = 2, α 3 = 1). For T small SEPT is better than LEPT as can be expected from the infinitesimal properties (for T = 0.1 we have L = for SEPT and L = for LEPT). However, for T larger, LEPT is better than SEPT (for T = 3 we have L = for SEPT vs. L = for LEPT). We found that SEPT (LEPT) is optimal for a small (large) time horizon, which is intuitivel clear. Thus neither LEPT nor SEPT is optimal for an time horizon T. 6. PROOF OF LEMMA 3.1 Write s instead of s(x). The proof goes b induction: we will show that µ j1 v n+1 (x,i e j1 ) + (µ µ j 1 )v n+1 (x,i) µ j2 v n+1 (x,i e j2 ) + (µ µ j 2 )v n+1 (x,i), assuming it holds up to n. We will treat the terms corresponding to arrivals and departures separatel. The term with the dumm transition follows immediatel from the induction hpothesis: µ j1 (1 λ xa sµ)v n (x,i e j1 ) + (µ µ j 1 )(1 λ xa sµ)v n (x,i) (3.2) µ j2 (1 λ xa sµ)v n (x,i e j2 ) + (µ µ j 2 )(1 λ xa sµ)v n (x,i) Consider the terms corresponding to arrivals. Note that (3.2) is equivalent to µ j1 v n (x,i e j1 ) + (µ j 2 µ j1 )v n (x,i) µ j 2 v n (x,i e j2 ), µ j 2 µ j1 0. Assume a is the optimal action in (x, i e j2 ). Then we have µ j1 min a { ( m λ xa qxav j (,i e n j1 +e + (1 m j) (µ j2 µ j1 ) min a µ j1 λ xa { ( m λ xa qxav j (,i+e n + (1 m j) q j xa)v n (,i e j1 ) q j xa)v n (,i) + ( m q j xa v n (,i e j1 +e + (1 m ) j) q j xa )v n (,i e j1 ) + (µ j2 µ j1 ) λ xa ( m q j xa v n (,i+e + (1 m ) (3.2) j) q j xa )v n (,i) µ j2 µ j2 min a λ xa ( m q j xa v n (,i e j2 +e + (1 m ) j) q j xa )v n (,i e j2 ) = { ( m λ xa qxav j (,i e n j2 +e + (1 m j) q j xa)v n (,i e j2 ) Note that we used here that µ j1 µ j2, but we do not need this inequalit when there are no actions to choose in the MDAP, i.e. if the arrivals are independent. Consider the terms corresponding to departures. 12.

13 First assume i s + 1. We distinguish three cases here. Case 1. There are customers with class numbers j1 js present in state i e j1 with js j 1. This means that j 1 does not belong to the optimal schedule of servers (recall that it follows from the induction hpothesis that the optimal schedule is SIP for the discrete-time horizon n) in i, i e j1 or i e j2. Because j 1 < j 2, the same schedule, sa j1,..., js, is optimal in i, i e j1 and i e j2. We have for 1 k s: µ j1 µ j k vn (x,i e j1 e j k ) + (µ µ j 1 )µ j k vn (x,i e j k )+ µ j1 (µ µ j k )vn (x,i e j1 ) + (µ µ j 1 )(µ µ j k )vn (x,i) (3.2) µ j2 µ j vn k (x,i e j2 e j ) + (µ µ j 2 )µ j vn k (x,i e j )+ k k µ j2 (µ µ j )vn k (x,i e j2 ) + (µ µ j 2 )(µ µ j )vn k (x,i). (6.1) Now equation (3.2) follows easil: { s s µ j1 min µ lk v(x,i e n l j1 e lk ) + (sµ µ lk )v(x,i e n j1 1,...,l s + { s s (µ µ j1 ) min µ lk v(x,i e n l lk ) + (sµ µ lk )v n 1,...,l s (x,i = s s { µj1 µ j k vn (x,i e j1 e j k ) + (µ µ j 1 )µ j k vn (x,i e j k ) } + { µj1 (µ µ j )vn k (x,i e j1 ) + (µ µ } (6.1) j 1 )(µ µ j )vn k (x,i) s s { µj2 µ j k vn (x,i e j2 e j k ) + (µ µ j 2 )µ j k vn (x,i e j k ) } + { µj2 (µ µ j )vn k (x,i e j2 ) + (µ µ } j 2 )(µ µ j )vn k (x,i) = { s s µ j2 min µ lk v(x,i e n l j2 e lk ) + (sµ µ lk )v(x,i e n j2 1,...,l s + { s s (µ µ j2 ) min µ lk v(x,i e n l lk ) + (sµ µ lk )v n 1,...,l s (x,i. Case 2. Assume that the optimal schedule serves all class j 1 but not all class j 2 customers in i. Then i e j2 and i have the same optimal schedule, serving customers of the classes j 1 and sa j 1 j. Thus, serving j 2, j 1,..., j is suboptimal in i and i e j1. Therefore { s s µ j1 min µ lk v(x,i e n l j1 e lk ) + (sµ µ lk )v(x,i e n j1 1,...,l s + { s s (µ µ j1 ) min µ lk v(x,i e n l lk ) + (sµ µ lk )v n 1,...,l s (x,i 13

14 { µj1 µ j vn k (x,i e j1 e j ) + (µ µ } j 1 )µ j vn k (x,i e j + ) k k µ j1 µ j2 v n (x,i e j1 e j2 ) + (µ µ j 1 )µ j2 v n (x,i e j2 ) + { µj1 (µ µ j )vn k (x,i e j1 ) + (µ µ } j 1 )(µ µ j )vn k (x,i) + µ j1 (µ µ j2 )v n (x,i e j1 ) + (µ µ j 1 )(µ µ j2 )v n (x,i). First serving class j 1 and then j 2 or in the reversed order makes no difference: µ j1 µ j2 v n (x,i e j1 e j2 ) + µ j 1 (µ µ j2 )v n (x,i e j1 ) + (µ µ j1 )µ j2 v n (x,i e j2 ) + (µ µ j 1 )(µ µ j2 )v n (x,i) = µ j2 µ j1 v n (x,i e j2 e j1 ) + µ j 2 (µ µ j1 )v n (x,i e j2 ) + (µ µ j2 )µ j1 v n (x,i e j1 ) + (µ µ j 2 )(µ µ j1 )v n (x,i). (6.2) Using (6.1) for j1,..., j and (6.2) gives the desired inequalit in the same wa as in case 1. Case 3. The optimal schedule serves all customers of class j 1 and class j 2 in i. Note that case 3 does not appear when there is onl one server (s = 1). Thus there are j1 j such that j1,..., js 2, j 1, j 2 is optimal in i; j1,..., j, j 1 is optimal in i e j2 and j1,..., j, j 2 is optimal in i e j1. Note that j 1 < j 2 < j. First we show: µ j1 µ j v(x,i e n j1 e j ) + µ j 1 (µ µ j )v(x,i e n j1 ) + (µ µ j1 )µ j1 v n (x,i e j1 ) + (µ µ j 1 )(µ µ j1 )v n (x,i) µ j2 µ j v n (x,i e j2 e j ) + µ j 2 (µ µ j )v n (x,i e j2 ) + (6.3) (µ µ j2 )µ j2 v n (x,i e j2 ) + (µ µ j 2 )(µ µ j2 )v n (x,i). Therefore we need the following inequalities (note that µ j1 µ j2 0): µ j1 µ j v n (x,i e j1 e j ) + (µ µ j 1 )µ j v n (x,i e j ) µ j2 µ j v n (x,i e j2 e j ) + (µ µ j 2 )µ j v n (x,i e j ) (µ j1 µ j2 )µ j v n (x,i e j ) + (µ j 1 µ j2 )(µ µ j )v n (x,i) (3.2) (3.2) (6.4) (6.5) (µ j1 µ j2 )µ j2 v n (x,i e j2 ) + (µ j 1 µ j2 )(µ µ j2 )v n (x,i) (2µ µ j1 µ j )µ j1 v n (x,i e j1 ) + (2µ µ j 1 µ j )(µ µ j1 )v n (x,i) (2µ µ j1 µ j )µ j2 v n (x,i e j2 ) + (2µ µ j 1 µ j )(µ µ j2 )v n (x,i) Summing (6.4), (6.5), (6.6) and subtracting (µ µ j2 )µ j v(x,i e n j ) + (µ µ j 2 )(µ µ j )v(x,i) n from both sides gives (6.3). Again we have (6.2). (3.2) (6.6) The relations (6.3) and (6.2) give the terms corresponding to departures of customers of queues j 1, j 2 and j. As customers of queues j 1,..., j s 2 are served in state i, i e j1 i e j2 we have b (3.2) µ j1 (µ k v n (x,i e j1 e k ) + (µ µ k)v n (x,i e j1 ) ) + (µ µ j 1 )(µ k v n (x,i e k ) + (µ µ k)v n (x,i) ) 14 and

15 µ j2 (µ k v(x,i e n j2 e k ) + (µ µ k)v(x,i e n j2 ) ) + (µ µ j 2 )(µ k v(x,i e n k ) + (6.7) (µ µ k )v(x,i) n ), k = j 1,..., js 2. Summing (6.3), (6.2) and (6.7) for k = j 1,..., j s 2 and adding the terms corresponding to arrivals gives (3.2). Second assume 2 i s. The following inequalities hold: µ j1 (2µ µ j2 )v n (x,i e j1 ) + (µ µ j 1 )(2µ µ j2 )v n (x,i) µ j2 (2µ µ j2 )v n (x,i e j2 ) + (µ µ j 2 )(2µ µ j2 )v n (x,i). µ j1 (µ j2 µ j1 )v n (x,i e j1 ) (3.3) µ j1 (µ j2 µ j1 )v n (x,i). Summing them and subtracting (µ µ j1 )(µ µ j2 + µ j1 )v(x,i) n from both sides gives: (3.2) µ j1 µv n (x,i e j1 ) + µ j 1 (µ µ j1 )v n (x,i e j1 ) + (µ µ j 1 ) 2 v n (x,i) µ j2 µv n (x,i e j2 ) + µ j 2 (µ µ j2 )v n (x,i e j2 ) + (µ µ j 2 ) 2 v n (x,i). Adding (6.2) to this inequalit gives the expression corresponding to departures from the queues j 1 and j 2. Together with (6.7) for each of the other i 2 queues and the relations corresponding to arrivals, this gives again (3.2). We continue with the proof of (3.3) which is much easier. The term corresponding to dumm transitions follows easil. Let a be the optimal action for the MDAP in (x, i). Then we have: min a { ( m λ xa qxav j (,i e n j1 +e + (1 m j) λ xa min a q j xa)v n (,i e j1 ) ( m q j xa v n (,i e j1 +e + (1 m ) (3.3) j) q j xa )v n (,i e j1 ) λ xa ( m q j xa v n (,i+e + (1 m ) j) q j xa )v n (,i) = { ( m λ xa qxav j (,i+e n + (1 m j) q j xa)v n (,i) Let j 1,..., j s be the optimal assignment in (x, i). If j 1 does not belong to this schedule, we have µ k v n (x,i e j1 e k ) + (µ µ k)v n (x,i e j1 ). (3.3) µ k v n (x,i e k ) + (µ µ k)v n (x,i) (6.8) for k = j 1,..., j s. Summing these relations gives the desired expression. If j 1 does belong to the optimal schedule in (x, i), sa j 1 = j s, take the suboptimal assignment j 1,..., j, 0 in (x, i e j1 ). Again (6.8) holds for k = j 1,..., j. For the last server we have v n (x,i e j1 ) (3.3) µ j1 v n (x,i e j1 ) + (µ µ j 1 v n (x,i). Summing these inequalities gives the expression for the suboptimal assignment. As the optimal schedule in (x, i e j1 ) gives a smaller l.h.s., we have the desired inequalit. 15

16 Acknowledgment A major part of the research for this paper was done in the spring of 1991 while the first author was on sabbatical leave at the Department of Industrial Engineering and Operations Research of the Universit of California at Berkele. He would like to thank Rhonda Righter, Sheldon Ross, George Shanthikumar and Jean Walrand for stimulating discussions on various topics. The second author likes to thank Rhonda Righter for her hospitalit and interest during his sta in October REFERENCES [1] S. Asmussen & G. Koole (1991). Marked point processes as limits of Markovian arrival streams. To appear in Journal of Applied Probabilit. [2] J.S. Baras, A.J. Dorse & A.M. Makowski (1985). Two competing queues with linear costs and geometric service requirements: the µc-rule is often optimal. Advances in Applied Probabilit 17: [3] J.S. Baras, D.-J. Ma & A.M. Makowski (1985). K competing queues with geometric service requirements and linear costs: the µc-rule is alwas optimal. Sstems & Control Letters 6: [4] J. Bruno, P. Downe & G.N. Frederickson (1981). Sequencing tasks with exponential service times to minimize the expected flow time or makespan. Journal of the ACM 28: [5] C. Buukkoc, P. Varaia & J. Walrand (1985). The cµ rule revisited. Advances in Applied Probabilit 17: [6] C.S. Chang, X. Chao, M. Pinedo, R.R. Weber (1992). On the optimalit of LEPT and cµ-rules for machines in parallel. Journal of Applied Probabilit 29: [7] L. van der Heden (1981). Scheduling jobs with exponential service times on non-identical processors so as to minimize expected makespan. Mathematics of Operations Research 6: [8] A. Hordijk & G. Koole (1992). The µc-rule is not optimal in the second node of the tandem queue: a counterexample. Advances in Applied Probabilit 24: [9] A. Hordijk & G. Koole (1992). On the assignment of customers to parallel queues. Probabilit in the Engineering and Informational Sciences 6: [10] T. Kämpke (1987). On the optimalit of static priorit policies in stochastic scheduling on parallel machines. Journal of Applied Probabilit 24: [11] G. Koole (1992). Stochastic scheduling and dnamic programming. Ph.D. thesis, Universit of Leiden. (on request available from the author) [12] M.F. Neuts (1981). Matrix-Geometric Solutions in Stochastic Models; an Algorithmic Approach. Johns Hopkins, Baltimore. [13] M. Pinedo (1983). Stochastic scheduling with stochastic release dates and due dates. Operations Research 31: [14] M. Pinedo & G. Weiss (1971). Scheduling of stochastic tasks on two parallel processors. Naval Research Logistics Quarterl 26: [15] R. Righter & J.G. Shanthikumar (1989). Scheduling multiclass single server queueing sstems to stochasticall maximize the number of successful departures. Probabilit in the Engineering and Informational Sciences 3: [16] R. Righter & J.G. Shanthikumar (1992). Extension of the bivariate characterization for stochastic orders. Advances in Applied Probabilit 24:

17 [17] S.M. Ross (1983). Introduction to Stochastic Dnamic Programming. Academic Press, New York. [18] J.G. Shanthikumar & D.D.W. Yao (1992). Multiclass queueing sstems: polmatroidal structure and optimal scheduling control. Operations Research 40: S293 S299. [19] J. Walrand (1988). An Introduction to Queueing Networks. Prentice-Hall, Englewood Cliffs. [20] R.R. Weber (1982). Scheduling jobs with stochastic processing requirements on parallel machines to minimize makespan or flowtime. Journal of Applied Probabilit 19: [21] R.R. Weber (1983). Scheduling stochastic jobs on parallel machines to minimize makespan or flowtime. In: R. Disne & T. Ott (eds.), Applied Probabilit - Computer Science: The Interface. Birkhauser, Boston. [22] R.R. Weber (1988). Stochastic scheduling on parallel processors and minimization of concave functions of completion times. In: W. Fleming & P.L. Lions (eds.), Stochastic Differential Sstems, Stochastic Control Theor and Applications 10. Springer-Verlag, New York. [23] G. Weiss (1982). Multiserver stochastic scheduling. In: M.A.H. Dempster et al. (eds.), Deterministic and Stochastic Scheduling. Reidel, Dordrecht. [24] G. Weiss & M. Pinedo (1980). Scheduling tasks with exponential service times on nonidentical processors to minimize various cost functions. Journal of Applied Probabilit 17: