On the Class of Quasi-Skip Free Processes: Stability & Explicit solutions when successively lumpable

Transcription

1 On the Class of Quasi-Skip Free Processes: Stability & Explicit solutions when successively lumpable DRAFT 2012-Nov-29 - comments welcome, do not cite or distribute without permission Michael N Katehakis Laurens Smit and Flora Spieksma December 3, 2012 Abstract In this article we study the class of quasi-skip-free (QSF) processes, a generalization of the quasi-birth-and-death (QBD) processes They are Markov process with states that can be specified by tuples of the form (k i)) where k represents the current level of the state i In addition, their probability transition law does not permit transitions to a state with level more than two (2) levels away from the current state s level in one direction Such processes have applications in many areas of applied probability comprising computer science, queuing theory, reliability and the theory of branching processes Further, in this article we derive stability conditions for QSF processes and provide a simple condition under which a QSF process is successively lumpable (SL-QSF) We use this successive lumpability property to derive explicit solutions and bounds for the steady state probabilities of general state space SL-QSFs, and to obtain a number of simplified derivations for results that are much more difficult to establish otherwise Finally, we obtain explicit solutions for the well known P H/M/n and M/P H/n queues Keywords and Phrases: model with backorder Successive lumpable, QBD processes, Inventory 1 Introduction A quasi-skip-free process is a Markov process with states that can be specified by tuples of the form (m i) where m represents the current level of the state and i (i = 1,, l m ) represents a state within the level m (m = 0, ±1, ±2, ) In addition, their probability transition law does not permit transitions to a state with Dept of Management Science and Information Systems, Rutgers Business School - Newark and New Brunswick, 1 Washington Park Newark, NJ mnk@rutgersedu Mathematisch Instituut, Universiteit Leiden Niels Bohrweg 1, 2333 CA, The Netherlands & Dept of Management Science and Information Systems, Rutgers Business School - Newark and New Brunswick, 1 Washington Park Newark, NJ lcs116@pegasusrutgersedu Mathematisch Instituut, Universiteit Leiden Niels Bohrweg 1, 2333 CA, The Netherlands spieksma@mathleidenunivnl

2 level more than two (2) units away from the current state s level in one direction Thus, their infinitesimal generator (transition rate matrix) has the form: D m 1 W m 1 U m 1,m U m 1,m+1 U m 1,m+2 Q = 0 D m W m U m,m+1 U m,m+2, (1) 0 0 D m+1 W m+1 U m+1,m+2 where in the above specification of Q we use the letters D, W, and U to describe down, within, and up transition rates, respectively, in relation to the current level m of a state (m i) Such processes have applications in many areas of applied probability including queuing theory, reliability, computer science and the theory of branching processes For example, skip-free processes, cf Neuts (1981), Brown et al (2010), have been used to model systems in many areas including queuing theory, cf Bright and Taylor (1995), cf Latouche and Ramaswami (1999), Riska and Smirni (2002), Artalejo et al (2010), retrial queues, cf Artalejo and Gómez-Corral (2008), chains that represent restart systems cf Katehakis and Veinott Jr (1987), Tong et al (2006) and Sonin (2011), chains that represent inventory systems with random lead times, reliability, cf Kapodistria (2011) computer science and the theory of branching processes Other related work on stability includes Koole and Spieksma (2001), Hordijk and Spieksma (1992), Hordijk and Spieksma (1989), Hordijk and Spieksma (1992), Hordijk and Spieksma (1989) For recent queuing applications we refer to Naoumov and Ipit (nd), Economou and Kapodistria (2010), Kapodistria (2011), Perel and Yechiali (2010), Li (2009) and the survey of Alfa and Ramaswami (2011) Latouche and Ramaswami (1999) (Chapter 13), treats homogenous QSF processes as QBDs They give a solution method to find the rate matrix in a homogenous QBD when U has the form cṙ where c is a column vector of size l and r a row vector of the same dimension and the elements of r sum up to one, and a approximation method when D has this structure and there is a lower finite bound on m (zero) Subsequently, several studies have modeled versions of quasi-skipfree (QSF) processes using the smaller framework of quasi-birth-and-death (QBD) processes For other algorithmic solutions for QBDs we refer to Viper - The Antiplagiarism Scanner In this article we study the QSF processes as a generalization of the QBD process In this paper we obtain explicit solutions for certain Quasi-Skip-Free (QSF) and Quasi-Birth-and-Death (QBD) Markov chains, under some simple additional conditions for the structure of its transition matrix Specifically, we will consider a process X(t) on statespace X with transition matrix of the structure of Eq (1) below, where for some fixed, finite or infinite integers: l m,, M 2, with 1 l m, < M 2, and m {, + 1,, M 2 } The rate sub-matrices D m are of dimension l m l m 1, the sub-matrices W m of dimension l m l m and the sub-matrices U m,k of dimension l m l k The transition matrix has the following form: 2

3 The main contribution are: i) we derive stability conditions for QSF processes, ii) we provide a simple condition under which a QSF process is successively lumpable (SL-QSF), iii) we use this successive lumpability property to derive explicit solutions and bounds for the steady state probabilities of general state space SL-QSFs, and to obtain a number of simplified derivations for results that are much more difficult to establish otherwise Finally, we obtain explicit solutions for the well known P H/M/n and M/P H/n queues Specifically, Proposition 1 provides indicative stability for a general QSF Lemmata 2 and 3 provide conditions for successive lumpability Theorem 1 provides balance equations for a sub level L m Theorem 2 provides a recursive relation for the rate matrix that can be used to express the steady state probabilities of states in a set in sub levels Theorem 3 provides an explicit solution when is finite Proposition 2 gives a limit solution when is infinite Proposition 3 gives an upper bound when M 2 is infinite In section 4 we specialize this results for a QBD process in Theorem 4 (a specialization of Theorem 2) and 5 which is a specialization of Theorem 3 In section 5 we show how this method can be applied to the PH/M/n and the M/PH/n 2 Definition and Stability of QSF Processes The block structure of Q allows one to relabel the elements of X and write it as X = M 2 m= {(m, 1), (m, 2),, (m, l m )} Clearly X can be partitioned into a (possibly infinite) sequence of mutually exclusive and exhaustive level sets: L m = {(m, 1), (m, 2),, (m, l m )} The QSF form of Eq (1) implies that when the current state of the process is (m, i) transitions are possible only to states of the form (k j), for k m 1 and j = 1,, l k, with corresponding transition rates d(m 1, j m, i), (when there is a down transition) w(m, j m, i) (when there is a within transition) and u(k j m, i), (when there is a up transition), for k > m Note that the diagonal elements of W m are the negative sum of all other elements in that row, ie, w(m i m i) = ξ(m i) where ξ(m i) = l m 1 j=1 d(m 1, j m i) + l m j=1,j i w(m j m i) + l k j=1 u(k j m i) We will use the notation π m = [π(m, 1),, π(m, l m )] to denote the steady state probabilities of states in level m and the vector π = [π,, π M 2 ] to denote the steady state probabilities over all states and the vector π m = [π,, π m ] Definition 21 For any fixed m we define the sub-level set of L m to be the set of states L m = m k= L k while the set L m = M 2 k=m L k is the super-level set of L m In Proposition 1 below we provide a condition for the ergodicity of the QSF process with a countable state space Note first that it suffices to establish the stability of the 3

4 discrete time embedded chain of X(t) denoted by X e (t) with transition probability matrix: Q e obtained from Q by replacing D m, W m and U m,k by De m, We m and Ue m,k respectively where the elements are defined as in Ross (1996), eg, w e (m, j m, i) = w(m, j m, i) / w(m, i m, i) if j i and otherwise w e (m, i m, i) = 0 For a fixed state (m, i) and fixed scalars m 1 and m 2 we introduce the quantities below; where the dependence of the first four on (m, i) and of the last on m, m 1, m 2 will be suppressed in the sequel for notational simplicity û(k) = û(k m, i) = l k j=1 u(k j m, i), ũ = ũ( m, i) = M 2 lk j=1 u(k j m, i), ˆd = ˆd( m, i) = l m 1 j=1 d(m 1, j m, i), ŵ = ŵ( m, i) = l m j=1 w(m, j m, i), ˆm = ˆm(m 1, m 2, m) = m 1 + m 2 m The quantities û e (k) = û e (k m, i), ˆd e = ˆd e ( m, i) and ŵ e = ŵ e ( m, i) corresponding to the discrete time embedded chain X e (t) are defined analogously Note that for the X e (t) process all transition probabilities out of state (m, i) sum up to 1, ie, û e (k)+ŵ e + ˆd e = 1 (2) We next state and prove the following proposition that gives a sufficient condition for a QSF process to be ergodic Proposition 1 A QSF process is ergodic if there exist finite numbers m 1, m 2 (with m 1 < m 2 ) such that Ineq (3) holds for all states (m, i), when m m 2 and Ineq (4) holds for all states (m, i), when m m 1 k= ˆm+1 ˆd e > (k ˆm)û e (k)+ ˆd m 1 e < (k m)û e (k), (3) m 2 (k m) û e (k)+(m 1 m) k=m 1 +1 û e (k) + ˆm k=m 2 +1 ( ˆm k)û e (k) (4) Proof We use an extension to an unbounded state space in two directions of Theorem 1 in Szpankowski (1988) to establish the ordinary ergodic stability using the Lyapunov function: m 1 m if m < m 1, V (m, i) = m m 2 if m > m 2, 0 otherwise for all states (m, i) X 4

5 We consider below first the case when m m 2 Indeed for this Lyapunov function the generalized drift of the process when X(t) = (m, i) is equal to: AV (m, i) =E ( V (X(t + 1)) V (X(t)) X(t) = (m, i) ) =E ( V (X(t + 1)) X(t) = (m, i) ) V (m, i) = (k m 2 )û e (k)+(m m 2 )ŵ e + (m 1 m 2 ) ˆd e (m m 2 ) = ((k m 2 )+(m 2 m))û e (k) ˆd e +(m m 2 )( û e (k)+ŵ e + ˆd e 1) = ɛ, (k m)û e (k) ˆd e where the last equality uses Eq (2) and the last inequality holds for all m m 2 by the assumption of Eq (3) When m m 1, the generalized drift is as follows: AV (m, i) =E ( V (X(t + 1)) V (X(t)) X(t) = (m, i) ) = = = m 1 (m 1 k) û e (k) + k=m (m 1 m + 1) ˆd e (m 1 m) m 1 (m 1 k)û e (k) + + (m 1 m)( m 1 ɛ, (m k)û e (k) + k=m 2 +1 (k m 2 ) û e (k) + (m 1 m)ŵ e (k m 2 ) û e (k) + (m m 1 ) û e (k) + ŵ e + ˆd e 1) k=m 2 +1 (k ˆm) û e (k) + (m m 1 ) k=m 2 +1 where the inequality holds for all m m 1 by the assumption of Eq (4) û e (k) + ˆd e û e (k) + ˆd e Remark 1 Proposition 1 provides sufficient (but not necessary) conditions for ergodic stability In section 51 we use different conditions to establish ergodicity 3 Successive Lumpable QSF Processes Following Katehakis and Smit (2012) it is easy to prove the following lemma 5

6 Lemma 1 For a QSF process, a state (m, ε(l m )) L m is an entrance state for L m if and only if the following is true for all (m + 1, i) L m+1 : d(m, j m + 1, i) = 0, if (m, j) (m, ε(l m )) (5) Proof The structure of the rate matrix Q implies that down transitions leaving the set L m+1 can only come from states in L m+1 Further, by Eq (5) the later type of transitions are possible only when they lead into state (m, ε(l m )) L m It is easy to see that Lemma 1 is equivalent with stating that matrix D m has a single nonzero column For any fixed n {,, M 2 }, let D n denote the partition {L n, L n+1,, L M2 } of X For a fixed n, the next lemma establishes that when all D m have a single non-zero column for all m n + 1 the QSF process is successively lumpable with respect to the partition D n Lemma 2 A QSF process is successively lumpable with respect to a partition D n if and only if D m contains a single non-zero column vector for all m = n + 1,, M 2 Proof It is a direct consequence of Lemma 1, that for a QSF process, a sub-level set L m has an entrance state (m, ε(l m )) if and only if D m+1 contains a single nonzero column vector This is true for all m n Further, from Lemma 1 we note that when D m+1 contains a single non-zero column vector then the set L m has an entrance state (m ε(l m )) Since L m = L m 1 L m it follows from the definition that the chain is successively lumpable: in the notation of Katehakis and Smit (2012), D 0 corresponds to L n and for m > n: D m corresponds to L m n Note that when a QSF process is successively lumpable with respect to a partition D n then it is successively lumpable with respect to a partition D m for all m > n A a QSF process that is successively lumpable with respect to D n for all n will be called strongly successively lumpable Note that the later condition is equivalent to the requirement that the QSF process is successively lumpable with respect to the partition D M1 Assumption A The QSF process has a transition rate matrix Q with the following properties: 1 There is a fixed n {,, M 2 1}, such that for all m = n + 1,, M 2, only the first column of sub-matrix D m may contain non-zero elements, ie, d(m 1, 1 m, i) > 0 for at least one (m i) L m and all its other elements are equal to zero 2 The matrix Q implies that the QSF process is ergodic Note the the first column of D m assumption is equivalent (by relabeling) to the existence of a single non-zero column of D m This assumption further implies, by Lemma 1, that state (m 1) is the entrance state of L m In the sequel whenever we write n we refer to the n of the above assumption We next need the following notation 6

7 The identity matrix I m of dimension l m l m The following vectors of dimension l m : i) The vector 0 m with 0 at every entry; ii) The vector 1 m with 1 at every entry; iii) The vector δ m with 1 as its first coordinate and 0 elsewhere The matrix Ũ m n = M 2 k=n+1 U mk 1 k δ m of dimension l m l m The scalar l m := m k= l k The identity matrix Ĩm of dimension l m l m The following vectors of dimension l m : i) The vector 0 m with 0 at every entry; ii) The vector 1 m with 1 at every entry Next, we fix m > n, with n as in assumption A-1, and consider the rates of transitions into the states (m i) of the set L m from states in L m = L m 1 L m Now, state (m 1) is the entrance state of L m The rates of these transitions will be denoted by γ(m j k i) There are two cases: i) The rates of transitions into the entrance state (m 1) are as follows: { w(m 1 m i) + ũ(m i), (k i) Lm, γ(m 1 k i) = u(m 1 k i) + ũ(k i), (k i) L m 1 (6) ii) The rates of transitions into other non-entrance states (m j), for j = 2,, l m, are as follows: w(m j m i), (k i) L m, γ(m j k i) = u(m j k i), (k i) L m 1, (7) 0, otherwise We next define the rate sub-matrices: γ(m 1, 1) γ(m l m, 1) A m = γ(m 1 m 1 l m 1 ) γ(m l m m 1 l m 1 ), and B m = γ(m 1 m 1) γ(m l m m 1) γ(m 1 m l m ) γ(m l m m l m ), Γ m = [ Am B m ] 7

8 Note that where A m denotes the l m 1 l m sub-matrix of Γ m with elements γ(m j k i), with k < m and B m denotes the l m l m sub-matrix of Γ m containing the elements γ(m j m i) Remark 2 Note that γ(m i m i) = (m j) L m\{(m i)} γ(m j m i) d(m 1, 1 m i) for all (m i) L m by either Eq (6) or Eq (7)) when m > This is true by its definition Note that γ(m i m i) can be seen as the (negative of) the transition rates out of state (m i) to states in L m only, ie, the rates of transitions into states in L m+1 are excluded due to the lumping procedure we will employ next in Theorem 1 The absolute value of the (m i) diagonal element of B m is greater or equal than the sum of the other elements in that row, and strictly greater for at least one, since the process X(t) is irreducible Thus, B m is irreducibly diagonally dominant and it has all off-diagonal entries nonpositive Therefore it follows from the Levy- Desplanques theorem cf Varga (1963) (p 85), or Varga (1976), that it is nonsingular and B 1 m > 0 Remark 3 Let 0 = L m = {(, 1),, (, l M1 ),, (m 1),, (m l m )} The lumped process on 0 has a rate matrix U 0 (defined in Katehakis and Smit (2012)) of size l m l m that can be written as: [Λ m Γ m ] where Λ m contains the rates of transitions into states of the set L m 1 (ie, it is a matrix of dimension m m 1 and the construction of the m l m matrix Γ m is done above following l Katehakis l and Smit (2012) Note that l we do not explicitly define the elements of the matrices Λ m as they are not explicitly used ithe sequel We next state and prove the following theorem for successively lumpable Markov chains which is a direct consequence of Theorem 2 of Katehakis and Smit (2012) within the context and notation of the present paper Specifically, for any fixed m, we consider the partition D = {L m, L m+1,, L M2 } of the state space X of the chain We note that the sets m are, within the present context, given by: m = {(m 0)} L m ; where (m 0) represents the lumped state Theorem 1 The following equality true for the steady state probabilities π m of X(t) for every m n: π m Γ m = 0 m (8) Proof By Lemma 2 we know that X(t) is successively lumpable with respect to D Let v L m denote the steady state probability vector of the lumped process on 0 = L m, cf Remark 3 By Proposition 1 of Katehakis and Smit (2012) (with k instead of 0) we know that for all k m: π(k i) = π(k, j)v L m(k i) (9) (k,j) L m 8

9 Further, since v L m is a steady state probability vector of the lumped process on 0 it is the normalized to 1 solution of the equation below: v L m[λ m 1 Γ m ] = 0 m (10) From Eq (9) we know π m = c v L m, where c = (k,j) L m π(k, j) is a nonnegative constant Eq (11) below follows by multiplying both sides of Eq (10) by c: π m [Λ m 1 Γ m ] = [ π m Λ m 1 π m Γ m ] = 0 m = [ 0 m 1 0 m ], (11) and the proof is complete We next obtain the recursive equation Eq (??) for the steady state probabilities π m = [π(m 1),, π(m l m )] of states in level L m in terms of the steady state probabilities π m 1 = [π,, π m 1 ] of all states in L m 1 To do this we first need to partition Γ m into Remark 4 From Eqs (6) and (7) we obtain: A m = Ũ m 1 m+1 + U m 1,m Ũ m+1 + U,m, (12) B m = Ũ m m+1 + W m (13) We denote: R 1 m = A m (B m ) 1 (14) We recursively define the matrices Rm k for k = 2,, m as follows: [ ] Rm k = Rm (k 1) 1 Ĩ m k Rm k 1 (15) of size l m k l m Theorem 2 The following relation holds for all k = 1,, m : where R k m is as defined in Eq(15) π m = π m k R k m, (16) Proof The proof is by induction For k = 1 we know by Theorem 1 that π m Γ m = 0 m We can rewrite this as: and thus: [ [π m 1, π m Am ] B m ] = 0 m, π m = π m 1 A m (B m ) 1 9

10 Suppose the statement is true for any m and for k 1 We first show that the statement holds for k: π m = π m (k 1) Rm k 1 = [π m (k 1), π m k ]Rm k 1 = [π m k Rm (k 1) 1, π m k ]Rm k 1 [ ] = π m k Rm (k 1) 1, I m k Rm k 1 = π m k R k m Thus the statement is true for k = 1,, m and therefore: We define the l M1 l M1 matrix S M 2 S M 2 = 1 + π m = π m k R k m by Eq (17) below m= +1 R m m 1 m 1 M1 (17) Note that S M 2 + Γ M1 is invertible, since a column of S M 2 (all columns are equal and have strictly positive elements) is linear independent of all columns of B M1, a stochastic rate matrix with determinant zero and rank l M1 1 We now state and prove the following theorem Theorem 3 Suppose is finite The following are true: i) for m > : ii) where π m = π R m m, (18) π = 1 M1 [S M 2 + B M1 ] 1 (19) Proof Eq (18) follows from Theorem 2 if we note that π = π Since the chain is ergodic we have π 1 M1 + M 2 m= +1 πm 1 m = 1, thus Eq (18) implies: π 1 + m= +1 m= +1 R m m 1 m = 1 Multiplying the left- and the right-handside of the above equation by 1 M1 gives: M π 1 + R m m 1 k 1 M1 = π S M 2 = 1 M1 10

11 Now using Theorem 2, where A M1 is an empty matrix (and π 1 an empty vector): π (B M1 ) = 0 M1 we conclude: and thus: π [ S M 2 + B M1 ] = 1 M1 π = 1 M1 [S M 2 + B M1 ] 1 In the next proposition we construct an estimate for the steady state distribution of states in L m1 when =, with m 1 > and m 1 M 2 We define a process X 1 (t) with state space X 1 = L m1 and transitions W m 1 U m 1,m 1 +1 U m 1,m 1 +2 U m 1,m 1 +3 D m 1+1 W m 1+1 U m 1+1,m 1 +2 U m 1+1,m+3 Q 1 = 0 D m 1+2 W m 1+2 U m 1+2,m 1 +3, (20) where w(m 1, j m 1, i) = w(m 1, j m 1, i) if j i and otherwise w(m 1, i m 1, i) = w(m 1, i m 1, i)+d(m 1 1, 1 m 1, i) We denote the steady state distribution of X 1 (t) as π m1 The following proposition holds for finite or infinite Proposition 2 The following is true for all (m i) with m > m 1 : π(m i) = lim π m1 (m i) m 1 and π m 1 m 1 = 1 m1 ( S M 2 m 1 + Γ m1 ) 1 constructed similarly to π in Theorem 3 Proof By assumption we know that X(t) is ergodic and therefore m lim 1 (m i) = 0 m 1 π Thus lim m1 1 π m 1 (m i) = 1, and since Q 1 has the same structure as Q for states in L m1 the proposition is true To get upper bounds for the steady state distribution of states in L m 2 when M 2 = and with m 2 < and m 2 we define a process X 2 (t) with statespace X 2 = L m 2 and transitions Q 2 = D m 2 2 W m 2 2 U m 2 2,m 2 1 U m 2 2,m 2 + Ũ m 2 2,m 2 0 D m 2 1 W m 2 1 U m 2 1,m 2 + Ũ m 2 1,m D m 2 W m 2 + Ũ m 2,m 2 We denote the steady state distribution of this process as π m2 (21) 11

12 Proposition 3 For all (m i) L m 2 the following is true: π(m i) π m2 (m i) Furthermore, π m2 +k(m i) is a decreasing function over k Proof The proof can be easily completed using Theorem 1 in which is used that: π(m i) = π(k j)v L m(m i), (k i) L m where in this case π m2 (m i) = v L m(m i) and to note that (k π(k j) < 1 i) L m Repeat this argument to get the sequence π(m i) π m2 +k(m i) π m2 +k 1(m i),, π m2 (m i) Remark 5 Consider a process ˆX on X such that transition matrix ˆQ has a block form similar to the transpose of Q ie, D m 1,m 2 W m 1 U m ˆQ = D m m 2 D m m 1 W m U m 0, (22) D m+1,m 2 D m+1,m 1 D m+1,m W m+1 U m+1 When we relabel state (m i) to ( m i), then D m k becomes U m k for all m and k and this is again a QSF process according to our definition 4 A special case of QSF processes: QBD processes A Quasi Birth and Death Process is a special case of a QSF process, where U m k = 0 for k m + 2 Therefore we rename in this section U m m+1 to U m All the proofs of the previous section for a QSF process also hold for a QBD process, but the algebra simplifies In the sequel we will assume that = 0 and that D m has a single non-zero column for m = 1, 2,, M 2, where M 2 is finite Further we assume that the matrix Q implies that the QBD process is ergodic The QBD process X(t) has the following transition rate matrix form: W 0 U D 2 W 2 U D 3 W 3 U Q = (23) D M 2 1 W M 2 1 U M D M 2 W M 2 In a QBD process Ũ m is as follows: Ũ m = U m 1 m 1δ m 12

13 L m 1 L m L m+1 m 1, 1 m, 1 m + 1, 1 m 1, lm 1 m, lm m + 1, lm+1 Figure 1: Graphical representation of a QBD process In a successively lumpable QBD process it is possible to directly compute R m as follows: R m = U m 1 (Ũ m + W m ) 1 (24) We now state and prove the following theorem for a QBD process Theorem 4 The following relation holds for the QBD process X(t): π m = π m 1 R m, for m = 1,, M 2 (25) Proof Note that in the present QBD case the matrices A m, B m (defined in Eq (12) and (13)) simplify to: [ ] U m 1 A m =, O m B m = Ũ m + W m where O m is a matrix of size l m 1 l m with 0 at every entry Now by Equation 14 and Theorem 2 we know: and because of the structure of A m we write: And thus: π m = π m 1 A m (B m ) 1, (26) π m 1 A m = π m 1 U m 1 π m = π m 1 A m (B m ) 1 = π m 1 U m 1 (Ũ m + W m ) 1 Remark 6 For each m = 1, 2,, M 2 the matrices R m are easy to compute; the computation only involves inversion of the l m l m matrix: Ũ m + W m and pre multiplication of the inverse by the l m 1 l m matrix U m 1 13

14 Theorem 5 The following are true for the successively lumpable QBD process X(t): i) for m > 0 : ii) where and π m = π 0 m k=1 π 0 = 1 0 [S M Ũ 0 + W 0] 1 S M 2 0 = m=1 k=1 R k, (27) (28) m R k (29) Proof Direct consequence of the QBD structure and Theorems 2 and 3 Remark 7 i) Note that Eq (25) implies that the following recursive relation holds for all ν = 0,, m 1: π m = π ν m k=ν+1 R k (30) ii) It is easy to see that the above defined π m and R m satisfy Eq (122) of Latouche and Ramaswami (1987) The matrices R m are solutions to Eq (1211) of the same book In a homogeneous successively lumpable QBD we denote for all m =,, M 2 : D m = D; W m = W ; and U m = U In addition, l m = l for all m; 1 and δ are of size l It is easy to see that now Ũ = U1 δ and R m = R = U(Ũ + W ) 1 for all m We state the following corollary Corrolary 1 When X(t) is a homogeneous successively lumpable QBD process, Equation (30) simplifies to: π m = π k [R] m k (31) This equality gives us: with where π m = π 0 [R] m (32) [ ] 1 π 0 = 1 S M Ũ 0 + W, (33) S M 2 0 = [R] m m=1 Remark 8 Without the assumption of = 0 a general successively lumpable QBD process, possibly infinite in two directions has the following expression for S M 2 : S M 2 = m m= +1 k= +1 R k 14

15 5 Queueing applications of successively lumpable QBD processes 51 The PH /M /n queueing system In this section we derive limit solutions for a P H/M/n queueing system Specifically we consider a system with n identical servers, where the service time of a customer is exponentially distributed with parameter µ Customers arrive in L (L < ) phases Customer arrivals are Poisson with Parameter λ m i for the i-th phase of the customer when there are m customers in line (It is also possible to consider the phases of the arrivals as single customers and in that service can only begin when all L people of a group arrived When there are m complete groups in the system and i customers of a new group waiting, the next customer will arrive according to a λ m i process) We model this queueing process as a QBD process X(t) on state space X = {L 0, L 1, } with L m = {(m 1),, (m L)} There are m customers in the system and the (m+1)- th customer is waiting for its i-th phase to occur in state (m i) In this QBD process, U m W m and D m are of size L L and are given below For m = 0, 1, : U m = λ m L 0 0 0, λ 0,1 λ 0, λ 0,2 λ 0,2 0 W 0 =, 0 0 λ 0,L 1 λ 0,L λ 0,L and for m = 1, 2, : (λ m 1 +µ m ) λ m (λ m 2 +µ m ) λ m 2 0 W m =, 0 0 (λ m L +µ m ) λ m L (λ m L +µ m ) D m = µ m µ m µ m µ m where µ m = m µ for m n 1, µ m = n µ for m n 15,

16 L 0 L 1 L 2 0, L µ 1 1, L µ 2 2, L µ 3 λ 2,L λ 0,2 λ 0,L λ 1,2 λ 1,L λ 2,2 0, 2 µ 1 1, 2 µ 2 2, 2 µ 3 λ 0,1 λ 1,1 λ 2,1 0, 1 µ 1 1, 1 µ 2 2, 1 µ 3 Figure 2: A P H/M/n queueing process Using Remark 5, we consider the process ˆX(t) with relabelled states (m i) to ( m i) for all (m i) we know by Lemma 2 that the process is a successively lumpable QBD, since then D has a single nonzero column By Theorem 4 we can now construct the rate matrix ˆR m for the process ˆX(t), the solution of ˆπ m = ˆπ m 1 ˆRm It is easy to see that R m the solution of π m = π m 1 R m is equal to ( ˆR m+1 ) 1 since ˆπ m = π m Therefore R m has the following form, for m = 2, 3, R m = 1 µ m λ m 1,1 λ m 1,1 0 0 µ m 1 λ m 1,2 +µ m 1 λ m 1,2 0 µ m 1 0 λ m 1,3 +µ m 1 0 µ m λ m 1,L +µ m 1, and R 1 : R 1 = 1 µ m 1 λ m 1 λ m λ m 2 λ m λ m λ m L Using Proposition 2 we can construct a limit for π(m i) by creating the process ˆX 1 (t) described in this proposition and the process ˆX(t) above We get: ˆπ 0 m 1 = 1 m1 [ S 0 m 1 + Ũ m 1 + W m 1] 1 (34) and ˆπ m m 1 = m 1 k= m 1 R k (35) 16

17 where S M 2 = 0 m= m 1 Now, by Proposition 2 and Remark 5 we get: m 1 k= m 1 R k lim ˆπ m m 1 1 ( m i) = π(m i) 52 The M /PH /n queueing process In a M/P H/n the service of a customer occurs in L phases, each exponentially distributed with parameter µ i for the i-th phase of the service Customers arrive according to a poisson process with parameter λ m i when there are m customers in the system and the served customer has gone through the first i phases of services The service of one customer has to be completed before another customer can start his first phase We model this queueing process as a QBD process X(t) on state space X = {L 0, L 1, } with L m = {(m 1),, (m L)} where state (m i) means that there are m customers in the waiting line system and that customer under service has gone through i phases of the service In this QBD process, U m W m and D m are of size L L and are given below For m = 0, 1, : U m = λ m λ m λ m λ m L, λ 0,1 µ 2 µ λ 0,2 µ 3 µ 3 0 W 0 = 0 0 λ 0,L 1 µ L µ L λ 0,L and for m 1 : λ m 1 µ 2 µ λ m 2 µ 3 µ 3 0 W m = 0 0 λ m L 1 µ L µ L λ m L µ 1 17

18 L 0 L 1 L 2 0, L λ 0,L 1, L λ 1,L 2, L λ 2,L µ L µ L µ L 0, 2 λ 0,2 1, 2 λ 1,2 2, 2 λ 2,2 µ 2 µ 1 µ 2 µ 1 µ 2 µ 1 0, 1 λ 0,1 1, 1 λ 1,1 2, 1 λ 2,1 Figure 3: A M/P H/n queueing process D m = µ Note that (m 1) is the entrance state of the set L m, because D m has a single nonzero column Now we can make a direct expression for R m : R m = U m 1 µ 2 µ λ m 2 λ m 2 µ 3 µ 3 0 λ m 3 0 λ m 3 µ 4 0 λ m L 0 0 λ m L µ 1 1 Using Proposition 3 we can construct upper bounds for π(m i) by creating the process X 2 (t) described in this proposition We get: π 0 m 2 = 1 0 [S m Ũ 0 + W 0] 1 (36) where S m 2 0 = m 2 m=1 k=1 m R k Remark 9 It is clear that the homogenous M/P H/n and the P H/M/n queueing systems have a very similar structure when the number of phases L is equal After relabeling the states of the P H/M/n process from (m i) to ( m i) and change the role of λ and µ the P H/M/n process is exactly the negative extension of the M/P H/n process 18

19 µ3 2, L µ2 1, L µ1 0, L 0, L λ0,l 1, L λ1,l 2, L λ2,l λ2,l λ2,2 λ1,2 λ0,2 µl µl µl µ3 2, 2 µ2 1, 2 µ1 0, 2 0, 2 λ0,2 1, 2 λ1,2 2, 2 λ2,2 λ2,1 λ1,l λ1,1 λ0,l λ0,1 µ2 µ1 µ2 µ1 µ2 µ1 µ3 2, 1 µ2 1, 1 µ1 0, 1 0, 1 λ0,1 1, 1 λ1,1 2, 1 λ2,1 References Alfa, A and Ramaswami, V (2011), Matrix analytic method: Overview and history, Wiley Encyclopedia of Operations Research and Management Science Artalejo, J, Economou, A and Lopez-Herrero, M (2010), The maximum number of infected individuals in sis epidemic models: Computational techniques and quasi-stationary distributions, Journal of computational and applied mathematics 233(10), Artalejo, J and Gómez-Corral, A (2008), Retrial queueing systems: a computational approach, Springer Verlag Bright, L and Taylor, P (1995), Calculating the equilibrium distribution in level dependent quasi-birth-and-death processes, Stochastic Models 11(3), Brown, M, Peköz, E and Ross, S (2010), Some results for skip-free random walk, Probability in the Engineering and Informational Sciences 24(04), Economou, A and Kapodistria, S (2010), Synchronized abandonments in a single server unreliable queue, European Journal of Operational Research 203(1), Hordijk, A and Spieksma, F (1989), Constrained admission control to a queueing system, Advances in Applied Probability pp Hordijk, A and Spieksma, F (1992), On ergodicity and recurrence properties of a markov chain with an application to an open jackson network, Advances in applied probability pp Kapodistria, S (2011), The M/M/1 queue with synchronized abandonments, Queueing systems 68(1), Katehakis, M N and Veinott Jr, A F (1987), The multi- armed bandit problem: Decomposition and computation, Math Oper Res 12, Katehakis, M and Smit, L (2012), A successive lumping procedure for a class of Markov chains, Probability in the Engineering and Informational Sciences 26(4), Koole, G and Spieksma, F (2001), On deviation matrices for birth death processes, Probability in the Engineering and Informational Sciences 15(02),

20 Latouche, G and Ramaswami, V (1987), Introduction to matrix analytic methods in stochastic modeling, Vol 5, Society for Industrial Mathematics Latouche, G and Ramaswami, V (1999), Introduction to matrix analytic methods in stochastic modeling, Vol 5, Society for Industrial Mathematics Li, Q (2009), Constructive computation in stochastic models with applications: the RG-Factorizations, Springer Naoumov, V and Ipit, I (nd), Analysis of a versatile multi-class delay-loss system with a superimposed Markovian arrival process Neuts, M (1981), Matrix-geometric solutions in stochastic models, The Johns Hopkins University Press, New York Perel, N and Yechiali, U (2010), Queues with slow servers and impatient customers, European Journal of Operational Research 201(1), Riska, A and Smirni, E (2002), M/g/1-type markov processes: A tutorial, Performance Evaluation of Complex Systems: Techniques and Tools pp Ross, S M (1996), Stochastic Processes, Wiley, New York Sonin, I (2011), Optimal stopping of Markov chains and three abstract optimization problems, Stochastics An International Journal of Probability and Stochastic Processes 83(4-6), Szpankowski, W (1988), Stability conditions for multidimensional queueing systems with computer applications, Operations Research 36(6), Tong, H, Faloutsos, C and Pan, J (2006), Fast random walk with restart and its applications Varga, R (1963), Matrix iterative analysis, Vol 767, Prentice-Hall Englewood Cliffs, NJ Varga, R (1976), On recurring theorems on diagonal dominance, Linear Algebra and its Applications 13(1),