Renewal theory and its applications

Renewal theory and its applications Stella Kapodistria and Jacques Resing September 11th, 212 ISP

Definition of a Renewal process Renewal theory and its applications If we substitute the Exponentially distributed inter-arrival times of the Poisson process by any arbitrary sequence of iid r.v. {X 1, X 2,...} we can generalize the definition of the counting process. Definition If the sequence of nonnegative random variables {X 1, X 2,...} is independent and identically distributed, then the counting process {N(t), t } is said to be a renewal process. For any sequence of iid r.v. we can define a counting process as n N(t) = max{n : S n t} with X j = S n The definition implies: (i) N(t) (ii) N(t) is integer valued (iii) If s < t, then N(s) N(t) (iv) For s < t, N(t) N(s) equals the number of events in (s, t]. j=1

Definition of a Renewal process Poisson process Definition The counting process {N(t), t } is called a Poisson process with rate λ, if {X 1, X 2,...} are iid having the exponential distribution at rate λ. In this case we know that: (i) N(t) Poisson(λt) and m(t) = E[N(t)] = λt (ii) n j=1 X j Erlang(λ, n) and E[S n ] = n/λ (iii) By the Strong Law of Large Numbers (SLLN) it follows that, S n n 1 λ, n (w.p.1)

Distribution of N(t) Distribution of N(t) The distribution of N(t) can be obtained, at least in theory, by noting that N(t) n S n t Then, P[N(t) = n] = P[N(t) n] P[N(t) n + 1] = P[S n t] P[S n+1 t] Now since the random variables X i, i 1, are iid having a distribution F, it follows that S n = n j=1 X j is distributed as F n, the n-fold convolution of F with itself (Section 2.5). Therefore, we obtain P[N(t) = n] = F n (t) F n+1 (t)

Distribution of N(t) The mean-value function The mean-value function The mean-value function can be obtained by noting that Then, N(t) n S n t m(t) = E[N(t)] = = = P[N(t) n] n=1 P[S n t] n=1 F n (t) The function m(t) is known as the mean-value or the renewal function. n=1 Example Suppose m(t) = 2t. What is the distribution P[N(1) = n] =?

Distribution of N(t) The mean-value function The mean-value function There is one-to-one correspondence between the renewal process and its mean-value function! We define m(s) to be the Laplace-Stieltjes transform of m(t) Then we can prove that m(s) = m(s) = e st m (t)dt ψ(s) 1 ψ(s) with ψ(s) = E[e sx ] we have denoted the Laplace-Stieltjes transform of the inter-arrival times {X 1, X 2,...}.

Distribution of N(t) The mean-value function The mean-value function There is one-to-one correspondence between the renewal process and its mean-value function! We want to determine m(t) for t 1. We will prove a basic renewal equation as follows m(t) = E[N(t)] = = = t t = F (t) + E[N(t) X 1 = x]f (x)dx E[N(t) X 1 = x]f (x)dx + [1 + m(t x)]f (x)dx + t m(t x)f (x)dx t t E[N(t) X 1 = x]f (x)dx f (x)dx This last equation is called the renewal equation and can sometimes be solved to obtain the mean-value function.

Distribution of N(t) The mean-value function Proposition 7.2 E[S N(t)+1 ] = (m(t) + 1)µ Proposition (7.2) E[S N(t)+1 ] = (m(t) + 1)µ Remark (Wald s Equation) For any sequence {X 1, X 2,...} of iid r.v. with mean E[X ] = µ and a r.v. N independent from the sequence {X 1, X 2,...} we can easily prove that N E[ X i ] = E[N]E[X ] = E[N]µ i=1

Distribution of N(t) The mean-value function Proposition 7.2 E[S N(t)+1 ] = (m(t) + 1)µ Proposition (7.2) Proof. We define E[S N(t)+1 ] = g(t), then g(t) = = = t t = µ + E[S N(t)+1 ] = (m(t) + 1)µ E[S N(t)+1 X 1 = x]f (x)dx E[S N(t)+1 ) X 1 = x]f (x)dx + [x + g(t x)]f (x)dx + t g(t x)f (x)dx t t xf (x)dx E[S N(t)+1 X 1 = x]f (x)dx

Distribution of N(t) The mean-value function Proposition 7.2 E[S N(t)+1 ] = (m(t) + 1)µ Proposition (7.2) Proof. We define E[S N(t)+1 ] = g(t), then t g(t) = µ + g(t x)f (x)dx E[S N(t)+1 ] = (m(t) + 1)µ If we substitute g(t) = (m(t) + 1)µ yields which completes the proof. t m(t) = F (t) + m(t x)f (x)dx

Distribution of N(t) The mean-value function Proposition 7.2 E[S N(t)+1 ] = (m(t) + 1)µ Proposition (7.2) Proof. We define E[S N(t)+1 ] = g(t), then g(t) = µ }{{} k(t) + t E[S N(t)+1 ] = (m(t) + 1)µ g(t x)f (x)dx For any known function k(t) the renewal type equation has a unique solution: g(t)= k(t) + (m(t) = F (t) + t t k(t x)m (x)dx m(t x)f (x)dx) Then by setting k(t) = µ immediately yields the result.

Limit Theorems Limit Theorems Let {X 1, X 2,...} be a sequence of iid r.v. and we define the renewal process {N(t), t } as Let E[X j ] = µ. By the SLLN it follows that, N(t) = max{n : S n t} with n X j = S n j=1 S n n µ, n (w.p.1) Hence, S n as n. Thus, S n t for at most a finite number of values of n, and hence by definition N(t) must be finite. However, though N(t) < for each t it is true that, with probability 1, N( ) = lim t N(t) =

Limit Theorems Limit Theorems Proposition ( 7.1) With probability 1, N(t) t 1 µ as t Proof. First of all recall that E[S N(t) ] = E[N(t)]E[X ], hence by the SLLN Secondly, S N(t) /N(t) E[X ] = µ t (w.p.1) S N(t) t < S N(t)+1 S N(t) N(t) N(t) j=1 X j N(t) t N(t) < S N(t)+1 N(t) t N(t) < S N(t)+1 N(t) + 1 = N(t) + 1 N(t) N(t)+1 j=1 X j N(t) + 1 N(t) + 1 N(t)

Limit Theorems Limit Theorems Proposition ( 7.1) With probability 1, N(t) t 1 µ as t Proof. First of all recall that E[S N(t) ] = E[N(t)]E[X ], hence by the SLLN Secondly, S N(t) /N(t) E[X ] = µ t (w.p.1) S N(t) t < S N(t)+1 S N(t) N(t) t N(t) < S N(t)+1 N(t) N(t) µ j=1 X j t N(t) N(t) < S µ N(t)+1 N(t)+1 N(t) + 1 = N(t) + 1 N(t) j=1 X 1 j N(t) N(t) + 1 + 1 N(t)

Limit Theorems Limit Theorems Theorem (Elementary Renewal Theorem) m(t) t 1 µ as t Theorem (Central Limit Theorem for Renewal Process) lim P t with µ = E[X ] and σ 2 = Var[X ]. Remark [ ] N(t) t/µ < x = 1 x e x 2 /2 dx tσ2 /µ 3 2π Var[N(t)] lim = σ2 t t µ 3

Limit Theorems Example 7.7 Example 7.7 Suppose that potential customers arrive at a single-server bank in accordance with a Poisson process having rate λ. Furthermore, suppose that the potential customer will enter the bank only if the server is free when he arrives; if upon arrival the customer sees the bank teller occupied he will immediately leave. If we assume that the amount of time spent in the bank by an entering customer is a random variable having distribution G, then (a) what is the rate at which customers enter the bank? (b) what proportion of potential customers actually enter the bank? In answering these questions, suppose that at time a customer has just entered the bank.

Limit Theorems Example 7.9 Example 7.9 Consider the renewal process whose inter-arrival distribution is the convolution of two exponentials; that is, F = F 1 F 2, where F i (t) = 1 e µ i t, i = 1, 2. Imagine that each renewal corresponds to a new machine being put in use, and suppose that each machine has two componentsinitially component 1 is employed and this lasts an exponential time with rate µ 1, and then component 2, which functions for an exponential time with rate µ 2, is employed. When component 2 fails, a new machine is put in use (that is, a renewal occurs). Now consider the process {X (t), t } where X (t) is i if a type i component is in use at time t. Calculate (a) the probability that the machine in use at time t is using its first component. (b) the expected excess time E[Y (t)] := E[S N(t)+1 t]. (c) the mean-value function. In answering these questions, suppose that at time a component 1 has just been employed.

Limit Theorems Example 7.1 Example 7.1 Two machines continually process an unending number of jobs. The time that it takes to process a job on machine 1 is a Gamma random variable with parameters n = 4, λ = 2, whereas the time that it takes to process a job on machine 2 is Uniformly distributed between and 4. Approximate the probability that together the two machines can process at least 9 jobs by time t = 1.

Renewal Reward Processes Renewal Reward Processes Consider a renewal process {N(t), t } having inter-arrival times {X 1, X 2,...} and suppose that each time a renewal occurs we receive a reward. We denote by R n, the reward earned at the time of the n-th renewal. We assume that the R n, n 1, are iid r.v. If we let N(t) R n R(t) = n=1 then R(t) represents the total reward earned by time t. Let E[R] = E[R n ], E[X ] = E[X n ] Proposition (7.3) If E[R] < and E[X ] <, then R(t) a) w.p.1 lim t t = E[R] E[R(t)] E[X ] b) lim t t = E[R] E[X ]

Renewal Reward Processes Example 7.11 Example 7.11 Suppose that potential customers arrive at a single-server bank in accordance with a Poisson process having rate λ. However, suppose that the potential customer will enter the bank only if the server is free when he arrives. That is, if there is already a customer in the bank, then our arriver, rather than entering the bank, will go home. If we assume that the amount of time spent in the bank by an entering customer is a random variable having distribution G, and that each customer that enters makes a deposit and that the amounts that the successive customers deposit in the bank are iid r.v. having a common distribution H, then the rate at which deposits accumulate that is, lim t (total deposits by the time t)/t is given by E[deposits during a cycle] E[time of cycle] = µ H µ G + 1/λ where µ G + 1/λ is the mean time of a cycle, and µ H is the mean of the distribution H.

Renewal Reward Processes Example 7.16 Example 7.16 (The Average Age of a Renewal Process) Consider a renewal process having inter-arrival distribution F and define A(t) := t S N(t) at time t. We are interested in s lim A(t)dt = average value of age s s Assume that s A(t)dt represents our total earnings by time s: s lim A(t)dt E[reward during a renewal cycle] s s E[time of a renewal cycle] Now since the age of the renewal process a time t into a renewal cycle is just t, we have reward during a renewal cycle = X where X F is the time of the renewal cycle. Hence, we have that s average value of age = lim A(t)dt = E[X 2 ]/2 s s E[X ] tdt

Regenerative Processes Regenerative Processes Consider a stochastic process {X (t), t } with state space {, 1, 2,...}, having the property that there exist time points at which the process (probabilistically) restarts itself. That is, suppose that with probability one, there exists a time T 1, such that the continuation of the process beyond T 1 is a probabilistic replica of the whole process starting at. Note that this property implies the existence of further times T 2, T 3,..., having the same property as T 1. Such a stochastic process is known as a regenerative process. From the preceding, it follows that T 1, T 2,..., constitute the arrival times of a renewal process, and we shall say that a cycle is completed every time a renewal occurs. Example (1) A renewal process is regenerative, and T 1 represents the time of the first renewal. (2) A recurrent Markov chain is regenerative, and T 1 represents the time of the first transition into the initial state.

Regenerative Processes Regenerative Processes We are interested in determining the long-run proportion of time that a regenerative process spends in state j. To obtain this quantity, let us imagine that we earn a reward at a rate 1 per unit time when the process is in state j and at rate otherwise. That is, if I (s) represents the rate at which we earn at time s, then { 1, if X (s) = j I (s) =, if X (s) j and total reward earned by t = t I (s)ds Proposition (7.4) For a regenerative process, the long-run proportion of time in state j = E[amount of time in j during a cycle] E[time of a cycle]

Regenerative Processes Example 7.18 Example 7.18 Consider a positive recurrent continuous time Markov chain that is initially in state i. By the Markovian property, each time the process reenters state i it starts over again. Thus returns to state i are renewals and constitute the beginnings of new cycles. By Proposition 7.4, it follows that the long-run proportion of time in state j = E[amount of time in j during an i i cycle] E[T i,i ] where E[T i,i ] represents the mean time to return to state i. If we take j to equal i, then we obtain proportion of time in state i = 1/v i E[T i,i ]

Regenerative Processes Example 7.19 Example 7.19 (A Queueing System with Renewal Arrivals) Consider a waiting time system in which customers arrive in accordance with an arbitrary renewal process and are served one at a time by a single server having an arbitrary service distribution. If we suppose that at time the initial customer has just arrived, then {X (t), t } is a regenerative process, where X (t) denotes the number of customers in the system at time t. The process regenerates each time a customer arrives and finds the server free.

Regenerative Processes Alternating Renewal Processes Alternating Renewal Processes Consider a system that can be in one of two states: on or off. Initially it is on, and it remains on for a time Z 1 ; it then goes off and remains off for a time Y 1. It then goes on for a time Z 2 ; then off for a time Y 2 ; then on, and so on. We suppose that the random vectors (Z n, Y n ), n 1 are iid; but we allow Z n and Y n to be dependent. In other words, each time the process goes on, everything starts over again, but when it then goes off, we allow the length of the off time to depend on the previous on time. We are concerned with P on, the long-run proportion of time that the system is on E[Z] P on = E[Y ] + E[Z] = E[on] E[off] + E[on]

Regenerative Processes Example 7.23 Example 7.23 (The Age of a Renewal Process) Suppose we are interested in determining the proportion of time that the age of a renewal process is less than some constant c. To do so, let a cycle correspond to a renewal, and say that the system is on at time t if the age at t is less than or equal to c, and say it is off if the age at t is greater than c. In other words, the system is on the first c time units of a renewal interval, and off the remaining time. Hence, letting X denote a renewal interval, we have E[min(X, c)] proportion of time age is less than c = E[X ] P[min{X, c} > x]dx = E[X ] c P[X > x]dx = E[X ] c (1 F (x))dx = E[X ]

Regenerative Processes Example 7.23 Summary Renewal theory 1 Definition of a Renewal process 2 Distribution of N(t) The mean-value function 3 Limit Theorems Example 7.7 Example 7.9 Example 7.1 4 Renewal Reward Processes Example 7.11 Example 7.16 5 Regenerative Processes Example 7.18 Example 7.19 Alternating Renewal Processes Example 7.23

Regenerative Processes Example 7.23 Exercises Introduction to Probability Models Harcourt/Academic Press, San Diego, 9th ed., 27 Sheldon M. Ross Chapter 7 Sections 7.1, 7.2, 7.3, 7.4, 7.5 Exercises: 2, 4, 5, 1, 11, 12, 15, 19, 26, 32, 37, 44