Renewal processes and Queues

Transcription

1 Renewal processes and Queues Last updated by Serik Sagitov: May 6, 213 Abstract This Stochastic Processes course is based on the book Probabilities and Random Processes by Geoffrey Grimmett and David Stirzaker. Chapter 1. A renewal process is a generalization of the Poisson process. Chapter 11. A simple queue model is a birth-death process with constant birth and death rates. 1 Renewal function and excess life Let T =, T n = X X n, where X i are iid strictly positive random variables called inter-arrival times. A renewal process N(t gives the number of renewal events during the time interval (, t] {N(t n} = {T n t}, T N(t t < T N(t+1. Definition 1 The renewal function m(t := E(N(t. In other texts it is the function U(t = 1 + m(t called the renewal function. Put F (t = P(X 1 t and define convolutions F (t = 1 {t }, F k (t = F (k 1 (t udf (u. Using a recursion N(t = 1 {X1 t}(1 + Ñ(t X 1, where Ñ(t is a renewed copy of the initial process N(t we find and m(t = F (t + k=1 m(t udf (u, m(t = F k (t, U(t = F k (t. In terms of the Laplace-Stieltjes transforms ˆF (θ := e θt df (t we get Û(θ = e θt du(t = k= k= e θt df k (t = ˆF (θ k 1 = 1 ˆF (θ. Example 2 Poisson process = Markovian renewal process: F (t = 1 e λt. Since k= we find m(t = λt. ˆF (θ = λ θ + λ ˆm(θ = λ θ, Definition 3 The excess life time E(t := T N(t+1 t. The distribution of E(t is given by P(E(t > y = (1 F (t + y udu(u. (1 1

2 This follows from the following recursion for b(t := P(E(t > y ( ( b(t = E E(1 {E(t>y} X 1 = E 1 {E(t X1>y}1 {X1 t} + 1 {X1>t+y} = 1 F (t + y + b(t xdf (x. Theorem 4 Elementary renewal theorem: m(t/t 1/µ as t, where µ = E(X 1. Proof. Due to the Wald equation (see later we have Since T N(t+1 = t + E(t, it follows E(T N(t+1 = µe(n(t + 1 = µu(t. (2 U(t = µ 1 (t + E(E(t, so that U(t µ 1 t. Moreover, if P(X 1 a = 1 for some finite a, then U(t µ 1 (t + a and the assertion follows. If X 1 is unbounded, then consider truncated inter-arrival times min(x i, a with mean µ a and renewal function U a (t. It remains to observe that U a (t tµ 1 a, U a (t U(t, and µ a µ as a. Theorem 5 Law of large numbers: N(t/t a.s. 1/µ as t. Proof. Note that T N(t t < T N(t+1. Thus, if N(t >, T N(t N(t t N(t < T ( N(t N(t + 1 N(t Since N(t as t, it ramains to apply the classical law of large numbers T n /n µ. Theorem 6 Central limit theorem. If σ 2 = Var(X 1 is positive and finite, then N(t t/µ tσ2 /µ 3 d N(, 1 as t. Proof. The usual CLT implies ( Ta(t µa(t P σ x Φ(x as a(t. a(t Put a(t = t/µ + x tσ 2 /µ 3, and observe that on one hand ( N(t t/µ P x = P(N(t a(t = P(T a(t t, tσ2 /µ 3 and on the other hand, t µa(t x as t. We conclude σ a(t ( N(t t/µ P x Φ( x = 1 Φ(x. tσ2 /µ 3 2 Stopping times and Wald s equation Definition 7 Let M be a r.v. taking values in the set {1, 2,...}. We call it a stopping time with respect to the sequence X n of inter-arrival times, if {M m} σ{x 1,..., X m }, for all m = 1, 2,... Lemma 8 Let X 1, X 2,... be iid r.v. with finite mean µ, and let M be a stopping time with respect to the sequence X n such that E(M <. Then E(X X M = µe(m. 2

3 Proof. By dominated convergence ( E(X X M = E lim n n ( X i 1 {M i} = lim E n X i 1 {M i} = E(X i P(M i = µe(m. n Here we used independence between {M i} and X i, which follows from the fact that {M i} is the complimentary event to {M i 1} σ{x 1,..., X i 1 }. Example 9 Since M = N(t + 1 is a stopping time for X n : the Wald equation implies (2. {M m} = {N(t m 1} = {X X m > t}, Example 1 Observe that M = N(t is not a stopping time and in general E(T N(t µm(t. Indeed, for the Poisson process µm(t = t while T N(t = t C(t, where C(t is the current lifetime. 3 Stationary distribution Theorem 11 Renewal theorem. If X 1 is not arithmetic, then for any positive h U(t + h U(t µ 1 h, t. Example 12 Arithmetic case. A typical arithmetic case is obtained, if we assume that the set of possible values R X for the inter-arrival times X i satisfies R X {1, 2,...} and R X {k, 2k,...} for any k = 2, 3,..., implying µ 1. If again U(n is the renewal function, then U(n U(n 1 is the probability that a renewal event occurs at time n. A discrete time version of Theorem 11 claims that U(n U(n 1 µ 1. Theorem 13 Key renewal theorem. monotone function, then If X 1 is not arithmetic, µ <, and g : [, [, is a g(t udu(u µ 1 g(udu, t. Sketch of the proof. Using Theorem 11, first prove the assertion for indicator functions of intervals, then for step functions, and finally for the limits of increasing sequences of step functions. Theorem 14 If X 1 is not arithmetic and µ <, then Proof. Apply the key renewal theorem to (1. y lim P(E(t y = t µ 1 (1 F (xdx. Definition 15 Let X 1, X 2,... be independent positive r.v. such that X 2, X 3,... have the same distribution. If as before, T = and T n = X X n, then N d (t = max{n : T n t} is called a delayed renewal process. It is described by two distributions F (t = P(X i t, i 2 and F d (t = P(X 1 t. Theorem 16 The process N d (t has stationary increments: N d (s + t N d (s d = N d (t, if and only if y F d (y = µ 1 (1 F (xdx. In this case the renewal function is linear m d (t = t/µ and P(E d (t y = F d (y independently of t. 3

4 4 Renewal-reward processes Let (X i, R i, i = 1, 2,... be iid pairs of possibly dependent random variables: X i are positive inter-arrival times and R i the associated rewards. Cumulative reward process W (t = R R N(t, w(t = E(W (t. Theorem 17 Renewal-reward theorem. Suppose (X i, R i have finite means µ = E(X and E(R. Then W (t/t a.s. E(R/µ, w(t/t E(R/µ, t. 5 Regeneration technique for queues Three applications of Theorem 17 to a general queueing system. Customers arrive one by one, and the n-th customer spends time V n in the system before departing. Let Q(t be the number of customers in the system at time t with Q( =. Let T be the time of first return to zero Q(T =. This can be called a regeneration time as starting from time T the future process Q(T + t is independent of the past. Assuming the traffic is light so that P(T < = 1, we get a renewal process of regeneration times = T < T = T 1 < T 2 < T 3 <.... Write N i for the number of customers arriving during the cycle [T i 1, T i and put N = N 1. To be able to apply Theorem 17 we shall assume E(T <, E(N <, E(NT <. (3 (A The reward associate with the inter-arrival time X i = T i T i 1 is taken to be R i = Ti T i 1 Q(udu. Since R := R 1 NT, we have E(R E(NT <, and by Theorem 17, t 1 Q(udu a.s. E(R/E(T =: L the long run average queue length. (B The reward associate with the inter-arrival time X i is taken to be N i. In this case the cumulative reward process N(t is the number of customers arrived by time t, and by Theorem 17, N(t/t a.s. E(N/E(T =: λ the long run rate of arrival. (C Consider now the reward-renewal process with inter-arrival times N i, and the associated rewards S i defined as the total time spent in system by the customers arrived during [T i 1, T i, so that S := S 1 = V V N. Since E(S E(NT <, by Theorem 17, n 1 n V i a.s. E(S/E(N =: ν the long run average time spent by a customer in the system. Theorem 18 Little s law. Under the assumption (3 we have L = λν. Although it looks intuitively reasonable, it s a quite remarkable result, as the relationship is not influenced by the arrival process distribution, the service distribution, the service order, or practically anything else. Proof. The mean amount of customer time spent during the first cycle can be represented in two ways ( N ( T E V i = E Q(udu. Thus combining (A, (B, (C we get ( T L λν = E(R E(T E(T E(N E(N E E(S = ( Q(udu N = 1. E V i 4

5 x x x x x x x x T1 T2 T3 T4 T5 T6 T7 T8 S S S S S S S S M/M/1 queues The most common notation scheme annotates the queueing systems by a triple A/B/s, where A describes the distribution of inter-arrival times of customers, B describes the distribution of service times, and s is the number of servers. It is assumed that the inter-arrival and service times are two independent iid sequences (X i and (S i. Let Q(t be the queue size at time t. The simplest queue M/M/1 has exponential with parameter λ inter-arrival times and exponential with parameter µ service times (M stands for the Markov discipline. It is the only queue with Q(t forming a Markov chain. In this case Q(t is a birth-death process with the birth rate λ and the death rate µ for the positive states, and no deaths at state zero. The probabilities p n (t = P(Q(t = n satisfy the Kolmogorov forward equations { p n (t = (λ + µp n (t + λp n 1 (t + µp n+1 (t for n 1, p (t = λp (t + µp 1 (t, subject to the boundary condition p n ( = 1 {n=}. In terms of the Laplace transforms ˆp n (θ := e θt p n (tdt we obtain { µˆpn+1 (θ (λ + µ + θˆp n (θ + λˆp n 1 (θ =, for n 1, µˆp 1 (θ (λ + θˆp (θ = 1. Its solution is given by ˆp n (θ = θ 1 (1 α(θα(θ n, α(θ := (λ + µ + θ (λ + µ + θ 2 4λµ. 2µ Theorem 19 Stationarity. Let ρ = λ/µ be the traffic intensity. As t for all n { (1 ρρ n if ρ < 1, P(Q(t = n if ρ 1. The result asserts that the queue settles down into equilibrium if and only if the service times are shorter than the inter-arrival times on average. It follows from the fact that a stationary distribution (π, π 1,... must satisfy the following equations { πn+1 (1 + ρπ n + ρπ n 1 = for n 1, 7 M/G/1 queues π 1 ρπ =. In the M/G/1 queueing system customers arrive according to a Poisson process with intensity λ and the service times S i has a fixed but unspecified distribution (G for General. Let ρ = λe(s be the traffic intensity. 5

6 Theorem 2 If the first customer arrives at time T 1, define a typical busy period of the server as B = inf{t > : Q(t + T 1 = }. We have that P(B < = 1 if ρ 1, and P(B < < 1 if ρ > 1. The moment generating function φ(u = E(e ub satisfies the functional equation φ(u = E(e (u λ+λφ(us. Proof. Imbedded branching process. Call customer C 2 an offspring of customer C 1, if C 2 joins the queue while C 1 is being served. The offspring number has generating function h(u = ( (λs u j j E e λs = E(e Sλ(u 1. j! j= The mean offspring number is h (1 = ρ. Observe that the event (B < is equivalent to the extinction of the branching process, and the first assertion follows. The functional equation follows from the representation B = S + B B Z, where B i are iid busy time of the offspring customers and Z is the number of offspring with generating function h(u. Theorem 21 Stationarity. As t for all n { π P(Q(t = n n if ρ < 1, where j= π ju j = (1 ρ(1 u h(u h(u u, if ρ 1. Proof. Let D n be the number of customers in the system right after the n-th customer left the system. Denoting Z n the offspring number of the n-th customer, we get D n+1 = D n + Z n+1 1 {Dn>}. Clearly, D n forms a Markov chain with transition probabilities δ δ 1 δ 2... δ δ 1 δ 2... ( δ δ 1... (λs j δ..., δ j = P(Z = j = E e λs. j! The stationary distribution of this chain gives the stated results. Theorem 22 Suppose a customer joins the queue after some large time has elapsed. He will wait a period W of time before his service begins. If ρ < 1, then E(e uw = (1 ρu λ + u λe(e us. Proof. Suppose that a customer waits for a period of length W and then is served for a period of length S. Assuming that the length D of the queue on the departure of this customer is distributed according to the stationary distribution given in Theorem 21 we get E(u D h(u = (1 ρ(1 u h(u u. On the other hand, D conditionally on (W, S has a Poisson distribution with parameter λ(w + S: h(u (1 ρ(1 u h(u u = E(uD = E(e λ(w +S(u 1 = E(e λw (u 1 h(u, and E(e λ(u 1W = (1 ρλ(u 1 λu λe(e λ(u 1S. 6

7 Theorem 23 Heavy traffic. Let ρ = λd be the traffic intensity of the M/G/1 queue with M = M(λ and G = D(d concentrated at value d. For ρ < 1 let Q ρ be a r.v. with the equilibrium queue length distribution. Then (1 ρq ρ converges in distribution as ρ 1 to the exponential distribution with parameter 2. Proof. Due to Theorem 21 the moment generating function E(e s(1 ρqρ equals (writing u = e s(1 ρ π j e s(1 ρj = (1 ρ(e s(1 ρ e ρ(u 1 1 e s(1 ρ e = (1 ρ(e s(1 ρ 1 ρ(u 1 exp(s(1 ρ ρ(e s(1 ρ 1 1 j= which converges to 2 2 s as ρ 1. 8 G/M/1 queues Customers arrival times form a renewal process with inter-arrival times (X n, and the service times are exponentially distributed with parameter µ. The traffic intensity is ρ = (µe(x 1. An imbedded Markov chain. Consider the moment T n at which the n-th customer joins the queue, and let A n be the number of customers who are ahead of him in the system. Define V n as the number of departures from the system during the interval [T n, T n+1. Conditionally on A n and X n+1 = T n+1 T n, the r.v. V n has a truncated Poisson distribution P(V n = v A n = a, X n+1 = x = { (µx v v! e µx if v a, (µx i i a+1 i! e µx if v = a + 1. The sequence (A n satisfies A n+1 = A n + 1 V n and forms a Markov chain with transition probabilities 1 α α... 1 α α 1 α 1 α... ( (µx j 1 α α 1 α 2 α 2 α 1 α..., α j = E e µx. j! Theorem 24 If ρ < 1, then the chain (A n is ergodic with a unique stationary distribution π j = (1 ηη j, where η is the smallest positive root of η = E(e Xµ(η 1. If ρ = 1, then the chain (A n is null recurrent. If ρ > 1, then the chain (A n is transient. Unlike the case of M/G/1, the stationary distribution of (A n need not be the limiting distribution of Q(t. To see an example of this consider a deterministic arrival process with P(X = 1 = 1. Theorem 25 Let ρ < 1, and assume that the chain (A n is in equilibrium. Then the waiting time W of an arriving customer has an atom of size 1 η at zero and for x P(W > x = ηe µ(1 ηx. Proof. If A n >, then the waiting time of the n-th customer is W n = S 1 + S 2 + S S An, where S1 is the residual service time of the customer under service, and S 2, S 3,..., S An are the service times of the others in the queue. By the lack-of-memory property, this is the sum A n iid exponentials. Use the equilibrium distribution of A n to find that (( µ E(e uw = E((E(e us An (µ u(1 η An = E = µ u µ u µη µ(1 η = (1 η + η µ(1 η u. 7

8 9 G/G/1 queues Now the arrivals of customers form a renewal process with inter-arrival times X n have an arbitrary distribution. The service times S n have another fixed distribution. The traffic intensity is given by ρ = E(S/E(X. Lemma 26 Lindley s equation. Let W n be the waiting time of the n-th customer. Then W n+1 = max{, W n + S n X n+1 }. Imbedded random walk. Note that U n = S n X n+1 is a collection of iid r.v.. Denote by G(x = P(U n x their common distribution function. Define an imbedded random walk by Lemma 26 implies Σ =, Σ n = U U n. W n+1 = max{, U n, U n + U n 1,..., U n + U n U 1 } d = max{σ,..., Σ n }. (4 Note that E(U = E(S E(X, and E(U < is equivalent to ρ < 1. Theorem 27 Let F n (x = P(W n x. Then for x F n+1 (x = x F n (x ydg(y. There exists a limit F (x = lim n F n (x which satisfies the Wiener-Hopf equation F (x = x F (x ydg(y. Proof. If x then due to the Lindley equation and independence between W n and U n = S n X n+1 P(W n+1 x = P(W n + U n x = and the first part is proved. We claim that x P(W n x ydg(y F n+1 (x F n (x for all x and n 1. (5 If (5 holds, then the second result follows immediately. We prove (5 by induction. Observe that F 2 (x 1 = F 1 (x, and suppose that (5 holds for n = k 1. Then F k+1 (x F k (x = x (F k (x y F k 1 (x ydg(y. Theorem 28 If ρ < 1, then F is a non-defective distribution function. If ρ = 1 and G is not concentrated at one point or if ρ > 1, then F (x = for all x. Proof. In view of (4 we have F (x = P(Σ n x for all n if x. If E(U <, then P(Σ n > for infinitely many n = P(n 1 Σ n > i.o. = P(n 1 Σ n E(U > E(U i.o. = due to the LLN. Thus max{σ, Σ 1,...} is either zero or the maximum of only finitely many positive terms, and F is a non-defective distribution. Next suppose that E(U > and pick any x >. For n 2x/E(U P(Σ n x = P(n 1 Σ n E(U n 1 x E(U P(n 1 Σ n E(U E(U/2 = P(n 1 Σ n E(U/2. Since 1 F (x P(Σ n x, the weak LLN implies F (x =. In the case when E(U = we need a more precise measure of the fluctuations of Σ n. According to the law of the iterated logarithm the fluctuations of Σ n are of order O( n log log n in both positive and negative directions with probability 1, and so 1 F (x = P(Σ n > x for some n = 1 for any given x. 8

9 Definition 29 Define an increasing sequence of r.v. by L( =, L(n + 1 = min{k > L(n : Σ k > Σ L(n }. The L(n are called ladder points of the random walk Σ, these are the times when the random walk Σ reaches its new maximal values. Lemma 3 Let η = P(Σ n > for some n. The total number Λ of ladder points has a geometric distribution P(Λ = k = (1 ηη k, k =, 1, 2,.... Lemma 31 If ρ < 1, the equilibrium waiting time distribution has the Laplace-Stieltjes transform ˆF (θ = 1 η 1 ηe(e θy, where Y = Σ L(1 is the first ladder hight of the imbedded random walk. Proof. Follows from the representation in terms of Y j = Σ L(j Σ L(j 1 which are iid copies of Y = Y 1 : max{σ, Σ 1,...} = Σ L(Λ = Y Y Λ. 9