Renewal theory and its applications

Size: px
Start display at page:

Download "Renewal theory and its applications"

Transcription

1 Renewal theory and its applications Stella Kapodistria and Jacques Resing September 11th, 212 ISP

2 Definition of a Renewal process Renewal theory and its applications If we substitute the Exponentially distributed inter-arrival times of the Poisson process by any arbitrary sequence of iid r.v. {X 1, X 2,...} we can generalize the definition of the counting process. Definition If the sequence of nonnegative random variables {X 1, X 2,...} is independent and identically distributed, then the counting process {N(t), t } is said to be a renewal process. For any sequence of iid r.v. we can define a counting process as n N(t) = max{n : S n t} with X j = S n The definition implies: (i) N(t) (ii) N(t) is integer valued (iii) If s < t, then N(s) N(t) (iv) For s < t, N(t) N(s) equals the number of events in (s, t]. j=1

3 Definition of a Renewal process Poisson process Definition The counting process {N(t), t } is called a Poisson process with rate λ, if {X 1, X 2,...} are iid having the exponential distribution at rate λ. In this case we know that: (i) N(t) Poisson(λt) and m(t) = E[N(t)] = λt (ii) n j=1 X j Erlang(λ, n) and E[S n ] = n/λ (iii) By the Strong Law of Large Numbers (SLLN) it follows that, S n n 1 λ, n (w.p.1)

4 Distribution of N(t) Distribution of N(t) The distribution of N(t) can be obtained, at least in theory, by noting that N(t) n S n t Then, P[N(t) = n] = P[N(t) n] P[N(t) n + 1] = P[S n t] P[S n+1 t] Now since the random variables X i, i 1, are iid having a distribution F, it follows that S n = n j=1 X j is distributed as F n, the n-fold convolution of F with itself (Section 2.5). Therefore, we obtain P[N(t) = n] = F n (t) F n+1 (t)

5 Distribution of N(t) The mean-value function The mean-value function The mean-value function can be obtained by noting that Then, N(t) n S n t m(t) = E[N(t)] = = = P[N(t) n] n=1 P[S n t] n=1 F n (t) The function m(t) is known as the mean-value or the renewal function. n=1 Example Suppose m(t) = 2t. What is the distribution P[N(1) = n] =?

6 Distribution of N(t) The mean-value function The mean-value function There is one-to-one correspondence between the renewal process and its mean-value function! We define m(s) to be the Laplace-Stieltjes transform of m(t) Then we can prove that m(s) = m(s) = e st m (t)dt ψ(s) 1 ψ(s) with ψ(s) = E[e sx ] we have denoted the Laplace-Stieltjes transform of the inter-arrival times {X 1, X 2,...}.

7 Distribution of N(t) The mean-value function The mean-value function There is one-to-one correspondence between the renewal process and its mean-value function! We want to determine m(t) for t 1. We will prove a basic renewal equation as follows m(t) = E[N(t)] = = = t t = F (t) + E[N(t) X 1 = x]f (x)dx E[N(t) X 1 = x]f (x)dx + [1 + m(t x)]f (x)dx + t m(t x)f (x)dx t t E[N(t) X 1 = x]f (x)dx f (x)dx This last equation is called the renewal equation and can sometimes be solved to obtain the mean-value function.

8 Distribution of N(t) The mean-value function Proposition 7.2 E[S N(t)+1 ] = (m(t) + 1)µ Proposition (7.2) E[S N(t)+1 ] = (m(t) + 1)µ Remark (Wald s Equation) For any sequence {X 1, X 2,...} of iid r.v. with mean E[X ] = µ and a r.v. N independent from the sequence {X 1, X 2,...} we can easily prove that N E[ X i ] = E[N]E[X ] = E[N]µ i=1

9 Distribution of N(t) The mean-value function Proposition 7.2 E[S N(t)+1 ] = (m(t) + 1)µ Proposition (7.2) Proof. We define E[S N(t)+1 ] = g(t), then g(t) = = = t t = µ + E[S N(t)+1 ] = (m(t) + 1)µ E[S N(t)+1 X 1 = x]f (x)dx E[S N(t)+1 ) X 1 = x]f (x)dx + [x + g(t x)]f (x)dx + t g(t x)f (x)dx t t xf (x)dx E[S N(t)+1 X 1 = x]f (x)dx

10 Distribution of N(t) The mean-value function Proposition 7.2 E[S N(t)+1 ] = (m(t) + 1)µ Proposition (7.2) Proof. We define E[S N(t)+1 ] = g(t), then t g(t) = µ + g(t x)f (x)dx E[S N(t)+1 ] = (m(t) + 1)µ If we substitute g(t) = (m(t) + 1)µ yields which completes the proof. t m(t) = F (t) + m(t x)f (x)dx

11 Distribution of N(t) The mean-value function Proposition 7.2 E[S N(t)+1 ] = (m(t) + 1)µ Proposition (7.2) Proof. We define E[S N(t)+1 ] = g(t), then g(t) = µ }{{} k(t) + t E[S N(t)+1 ] = (m(t) + 1)µ g(t x)f (x)dx For any known function k(t) the renewal type equation has a unique solution: g(t)= k(t) + (m(t) = F (t) + t t k(t x)m (x)dx m(t x)f (x)dx) Then by setting k(t) = µ immediately yields the result.

12 Limit Theorems Limit Theorems Let {X 1, X 2,...} be a sequence of iid r.v. and we define the renewal process {N(t), t } as Let E[X j ] = µ. By the SLLN it follows that, N(t) = max{n : S n t} with n X j = S n j=1 S n n µ, n (w.p.1) Hence, S n as n. Thus, S n t for at most a finite number of values of n, and hence by definition N(t) must be finite. However, though N(t) < for each t it is true that, with probability 1, N( ) = lim t N(t) =

13 Limit Theorems Limit Theorems Proposition ( 7.1) With probability 1, N(t) t 1 µ as t Proof. First of all recall that E[S N(t) ] = E[N(t)]E[X ], hence by the SLLN Secondly, S N(t) /N(t) E[X ] = µ t (w.p.1) S N(t) t < S N(t)+1 S N(t) N(t) N(t) j=1 X j N(t) t N(t) < S N(t)+1 N(t) t N(t) < S N(t)+1 N(t) + 1 = N(t) + 1 N(t) N(t)+1 j=1 X j N(t) + 1 N(t) + 1 N(t)

14 Limit Theorems Limit Theorems Proposition ( 7.1) With probability 1, N(t) t 1 µ as t Proof. First of all recall that E[S N(t) ] = E[N(t)]E[X ], hence by the SLLN Secondly, S N(t) /N(t) E[X ] = µ t (w.p.1) S N(t) t < S N(t)+1 S N(t) N(t) t N(t) < S N(t)+1 N(t) N(t) µ j=1 X j t N(t) N(t) < S µ N(t)+1 N(t)+1 N(t) + 1 = N(t) + 1 N(t) j=1 X 1 j N(t) N(t) N(t)

15 Limit Theorems Limit Theorems Theorem (Elementary Renewal Theorem) m(t) t 1 µ as t Theorem (Central Limit Theorem for Renewal Process) lim P t with µ = E[X ] and σ 2 = Var[X ]. Remark [ ] N(t) t/µ < x = 1 x e x 2 /2 dx tσ2 /µ 3 2π Var[N(t)] lim = σ2 t t µ 3

16 Limit Theorems Example 7.7 Example 7.7 Suppose that potential customers arrive at a single-server bank in accordance with a Poisson process having rate λ. Furthermore, suppose that the potential customer will enter the bank only if the server is free when he arrives; if upon arrival the customer sees the bank teller occupied he will immediately leave. If we assume that the amount of time spent in the bank by an entering customer is a random variable having distribution G, then (a) what is the rate at which customers enter the bank? (b) what proportion of potential customers actually enter the bank? In answering these questions, suppose that at time a customer has just entered the bank.

17 Limit Theorems Example 7.9 Example 7.9 Consider the renewal process whose inter-arrival distribution is the convolution of two exponentials; that is, F = F 1 F 2, where F i (t) = 1 e µ i t, i = 1, 2. Imagine that each renewal corresponds to a new machine being put in use, and suppose that each machine has two componentsinitially component 1 is employed and this lasts an exponential time with rate µ 1, and then component 2, which functions for an exponential time with rate µ 2, is employed. When component 2 fails, a new machine is put in use (that is, a renewal occurs). Now consider the process {X (t), t } where X (t) is i if a type i component is in use at time t. Calculate (a) the probability that the machine in use at time t is using its first component. (b) the expected excess time E[Y (t)] := E[S N(t)+1 t]. (c) the mean-value function. In answering these questions, suppose that at time a component 1 has just been employed.

18 Limit Theorems Example 7.1 Example 7.1 Two machines continually process an unending number of jobs. The time that it takes to process a job on machine 1 is a Gamma random variable with parameters n = 4, λ = 2, whereas the time that it takes to process a job on machine 2 is Uniformly distributed between and 4. Approximate the probability that together the two machines can process at least 9 jobs by time t = 1.

19 Renewal Reward Processes Renewal Reward Processes Consider a renewal process {N(t), t } having inter-arrival times {X 1, X 2,...} and suppose that each time a renewal occurs we receive a reward. We denote by R n, the reward earned at the time of the n-th renewal. We assume that the R n, n 1, are iid r.v. If we let N(t) R n R(t) = n=1 then R(t) represents the total reward earned by time t. Let E[R] = E[R n ], E[X ] = E[X n ] Proposition (7.3) If E[R] < and E[X ] <, then R(t) a) w.p.1 lim t t = E[R] E[R(t)] E[X ] b) lim t t = E[R] E[X ]

20 Renewal Reward Processes Example 7.11 Example 7.11 Suppose that potential customers arrive at a single-server bank in accordance with a Poisson process having rate λ. However, suppose that the potential customer will enter the bank only if the server is free when he arrives. That is, if there is already a customer in the bank, then our arriver, rather than entering the bank, will go home. If we assume that the amount of time spent in the bank by an entering customer is a random variable having distribution G, and that each customer that enters makes a deposit and that the amounts that the successive customers deposit in the bank are iid r.v. having a common distribution H, then the rate at which deposits accumulate that is, lim t (total deposits by the time t)/t is given by E[deposits during a cycle] E[time of cycle] = µ H µ G + 1/λ where µ G + 1/λ is the mean time of a cycle, and µ H is the mean of the distribution H.

21 Renewal Reward Processes Example 7.16 Example 7.16 (The Average Age of a Renewal Process) Consider a renewal process having inter-arrival distribution F and define A(t) := t S N(t) at time t. We are interested in s lim A(t)dt = average value of age s s Assume that s A(t)dt represents our total earnings by time s: s lim A(t)dt E[reward during a renewal cycle] s s E[time of a renewal cycle] Now since the age of the renewal process a time t into a renewal cycle is just t, we have reward during a renewal cycle = X where X F is the time of the renewal cycle. Hence, we have that s average value of age = lim A(t)dt = E[X 2 ]/2 s s E[X ] tdt

22 Regenerative Processes Regenerative Processes Consider a stochastic process {X (t), t } with state space {, 1, 2,...}, having the property that there exist time points at which the process (probabilistically) restarts itself. That is, suppose that with probability one, there exists a time T 1, such that the continuation of the process beyond T 1 is a probabilistic replica of the whole process starting at. Note that this property implies the existence of further times T 2, T 3,..., having the same property as T 1. Such a stochastic process is known as a regenerative process. From the preceding, it follows that T 1, T 2,..., constitute the arrival times of a renewal process, and we shall say that a cycle is completed every time a renewal occurs. Example (1) A renewal process is regenerative, and T 1 represents the time of the first renewal. (2) A recurrent Markov chain is regenerative, and T 1 represents the time of the first transition into the initial state.

23 Regenerative Processes Regenerative Processes We are interested in determining the long-run proportion of time that a regenerative process spends in state j. To obtain this quantity, let us imagine that we earn a reward at a rate 1 per unit time when the process is in state j and at rate otherwise. That is, if I (s) represents the rate at which we earn at time s, then { 1, if X (s) = j I (s) =, if X (s) j and total reward earned by t = t I (s)ds Proposition (7.4) For a regenerative process, the long-run proportion of time in state j = E[amount of time in j during a cycle] E[time of a cycle]

24 Regenerative Processes Example 7.18 Example 7.18 Consider a positive recurrent continuous time Markov chain that is initially in state i. By the Markovian property, each time the process reenters state i it starts over again. Thus returns to state i are renewals and constitute the beginnings of new cycles. By Proposition 7.4, it follows that the long-run proportion of time in state j = E[amount of time in j during an i i cycle] E[T i,i ] where E[T i,i ] represents the mean time to return to state i. If we take j to equal i, then we obtain proportion of time in state i = 1/v i E[T i,i ]

25 Regenerative Processes Example 7.19 Example 7.19 (A Queueing System with Renewal Arrivals) Consider a waiting time system in which customers arrive in accordance with an arbitrary renewal process and are served one at a time by a single server having an arbitrary service distribution. If we suppose that at time the initial customer has just arrived, then {X (t), t } is a regenerative process, where X (t) denotes the number of customers in the system at time t. The process regenerates each time a customer arrives and finds the server free.

26 Regenerative Processes Alternating Renewal Processes Alternating Renewal Processes Consider a system that can be in one of two states: on or off. Initially it is on, and it remains on for a time Z 1 ; it then goes off and remains off for a time Y 1. It then goes on for a time Z 2 ; then off for a time Y 2 ; then on, and so on. We suppose that the random vectors (Z n, Y n ), n 1 are iid; but we allow Z n and Y n to be dependent. In other words, each time the process goes on, everything starts over again, but when it then goes off, we allow the length of the off time to depend on the previous on time. We are concerned with P on, the long-run proportion of time that the system is on E[Z] P on = E[Y ] + E[Z] = E[on] E[off] + E[on]

27 Regenerative Processes Example 7.23 Example 7.23 (The Age of a Renewal Process) Suppose we are interested in determining the proportion of time that the age of a renewal process is less than some constant c. To do so, let a cycle correspond to a renewal, and say that the system is on at time t if the age at t is less than or equal to c, and say it is off if the age at t is greater than c. In other words, the system is on the first c time units of a renewal interval, and off the remaining time. Hence, letting X denote a renewal interval, we have E[min(X, c)] proportion of time age is less than c = E[X ] P[min{X, c} > x]dx = E[X ] c P[X > x]dx = E[X ] c (1 F (x))dx = E[X ]

28 Regenerative Processes Example 7.23 Summary Renewal theory 1 Definition of a Renewal process 2 Distribution of N(t) The mean-value function 3 Limit Theorems Example 7.7 Example 7.9 Example Renewal Reward Processes Example 7.11 Example Regenerative Processes Example 7.18 Example 7.19 Alternating Renewal Processes Example 7.23

29 Regenerative Processes Example 7.23 Exercises Introduction to Probability Models Harcourt/Academic Press, San Diego, 9th ed., 27 Sheldon M. Ross Chapter 7 Sections 7.1, 7.2, 7.3, 7.4, 7.5 Exercises: 2, 4, 5, 1, 11, 12, 15, 19, 26, 32, 37, 44

Poisson Processes. Stochastic Processes. Feb UC3M

Poisson Processes. Stochastic Processes. Feb UC3M Poisson Processes Stochastic Processes UC3M Feb. 2012 Exponential random variables A random variable T has exponential distribution with rate λ > 0 if its probability density function can been written

More information

Exponential Distribution and Poisson Process

Exponential Distribution and Poisson Process Exponential Distribution and Poisson Process Stochastic Processes - Lecture Notes Fatih Cavdur to accompany Introduction to Probability Models by Sheldon M. Ross Fall 215 Outline Introduction Exponential

More information

1 Some basic renewal theory: The Renewal Reward Theorem

1 Some basic renewal theory: The Renewal Reward Theorem Copyright c 27 by Karl Sigman Some basic renewal theory: The Renewal Reward Theorem Here, we will present some basic results in renewal theory such as the elementary renewal theorem, and then the very

More information

The exponential distribution and the Poisson process

The exponential distribution and the Poisson process The exponential distribution and the Poisson process 1-1 Exponential Distribution: Basic Facts PDF f(t) = { λe λt, t 0 0, t < 0 CDF Pr{T t) = 0 t λe λu du = 1 e λt (t 0) Mean E[T] = 1 λ Variance Var[T]

More information

Renewal processes and Queues

Renewal processes and Queues Renewal processes and Queues Last updated by Serik Sagitov: May 6, 213 Abstract This Stochastic Processes course is based on the book Probabilities and Random Processes by Geoffrey Grimmett and David Stirzaker.

More information

IEOR 6711: Stochastic Models I Fall 2012, Professor Whitt, Thursday, October 4 Renewal Theory: Renewal Reward Processes

IEOR 6711: Stochastic Models I Fall 2012, Professor Whitt, Thursday, October 4 Renewal Theory: Renewal Reward Processes IEOR 67: Stochastic Models I Fall 202, Professor Whitt, Thursday, October 4 Renewal Theory: Renewal Reward Processes Simple Renewal-Reward Theory Suppose that we have a sequence of i.i.d. random vectors

More information

Statistics 150: Spring 2007

Statistics 150: Spring 2007 Statistics 150: Spring 2007 April 23, 2008 0-1 1 Limiting Probabilities If the discrete-time Markov chain with transition probabilities p ij is irreducible and positive recurrent; then the limiting probabilities

More information

Solution: The process is a compound Poisson Process with E[N (t)] = λt/p by Wald's equation.

Solution: The process is a compound Poisson Process with E[N (t)] = λt/p by Wald's equation. Solutions Stochastic Processes and Simulation II, May 18, 217 Problem 1: Poisson Processes Let {N(t), t } be a homogeneous Poisson Process on (, ) with rate λ. Let {S i, i = 1, 2, } be the points of the

More information

Chapter 2. Poisson Processes. Prof. Shun-Ren Yang Department of Computer Science, National Tsing Hua University, Taiwan

Chapter 2. Poisson Processes. Prof. Shun-Ren Yang Department of Computer Science, National Tsing Hua University, Taiwan Chapter 2. Poisson Processes Prof. Shun-Ren Yang Department of Computer Science, National Tsing Hua University, Taiwan Outline Introduction to Poisson Processes Definition of arrival process Definition

More information

Continuous Time Markov Chains

Continuous Time Markov Chains Continuous Time Markov Chains Stochastic Processes - Lecture Notes Fatih Cavdur to accompany Introduction to Probability Models by Sheldon M. Ross Fall 2015 Outline Introduction Continuous-Time Markov

More information

MATH 56A: STOCHASTIC PROCESSES CHAPTER 6

MATH 56A: STOCHASTIC PROCESSES CHAPTER 6 MATH 56A: STOCHASTIC PROCESSES CHAPTER 6 6. Renewal Mathematically, renewal refers to a continuous time stochastic process with states,, 2,. N t {,, 2, 3, } so that you only have jumps from x to x + and

More information

Manual for SOA Exam MLC.

Manual for SOA Exam MLC. Chapter 10. Poisson processes. Section 10.5. Nonhomogenous Poisson processes Extract from: Arcones Fall 2009 Edition, available at http://www.actexmadriver.com/ 1/14 Nonhomogenous Poisson processes Definition

More information

The story of the film so far... Mathematics for Informatics 4a. Continuous-time Markov processes. Counting processes

The story of the film so far... Mathematics for Informatics 4a. Continuous-time Markov processes. Counting processes The story of the film so far... Mathematics for Informatics 4a José Figueroa-O Farrill Lecture 19 28 March 2012 We have been studying stochastic processes; i.e., systems whose time evolution has an element

More information

IEOR 6711, HMWK 5, Professor Sigman

IEOR 6711, HMWK 5, Professor Sigman IEOR 6711, HMWK 5, Professor Sigman 1. Semi-Markov processes: Consider an irreducible positive recurrent discrete-time Markov chain {X n } with transition matrix P (P i,j ), i, j S, and finite state space.

More information

TMA 4265 Stochastic Processes

TMA 4265 Stochastic Processes TMA 4265 Stochastic Processes Norges teknisk-naturvitenskapelige universitet Institutt for matematiske fag Solution - Exercise 9 Exercises from the text book 5.29 Kidney transplant T A exp( A ) T B exp(

More information

Queueing Theory I Summary! Little s Law! Queueing System Notation! Stationary Analysis of Elementary Queueing Systems " M/M/1 " M/M/m " M/M/1/K "

Queueing Theory I Summary! Little s Law! Queueing System Notation! Stationary Analysis of Elementary Queueing Systems  M/M/1  M/M/m  M/M/1/K Queueing Theory I Summary Little s Law Queueing System Notation Stationary Analysis of Elementary Queueing Systems " M/M/1 " M/M/m " M/M/1/K " Little s Law a(t): the process that counts the number of arrivals

More information

Math 180C, Spring Supplement on the Renewal Equation

Math 180C, Spring Supplement on the Renewal Equation Math 18C Spring 218 Supplement on the Renewal Equation. These remarks supplement our text and set down some of the material discussed in my lectures. Unexplained notation is as in the text or in lecture.

More information

Random Walk on a Graph

Random Walk on a Graph IOR 67: Stochastic Models I Second Midterm xam, hapters 3 & 4, November 2, 200 SOLUTIONS Justify your answers; show your work.. Random Walk on a raph (25 points) Random Walk on a raph 2 5 F B 3 3 2 Figure

More information

Solutions to Homework Discrete Stochastic Processes MIT, Spring 2011

Solutions to Homework Discrete Stochastic Processes MIT, Spring 2011 Exercise 6.5: Solutions to Homework 0 6.262 Discrete Stochastic Processes MIT, Spring 20 Consider the Markov process illustrated below. The transitions are labelled by the rate q ij at which those transitions

More information

Q = (c) Assuming that Ricoh has been working continuously for 7 days, what is the probability that it will remain working at least 8 more days?

Q = (c) Assuming that Ricoh has been working continuously for 7 days, what is the probability that it will remain working at least 8 more days? IEOR 4106: Introduction to Operations Research: Stochastic Models Spring 2005, Professor Whitt, Second Midterm Exam Chapters 5-6 in Ross, Thursday, March 31, 11:00am-1:00pm Open Book: but only the Ross

More information

2905 Queueing Theory and Simulation PART III: HIGHER DIMENSIONAL AND NON-MARKOVIAN QUEUES

2905 Queueing Theory and Simulation PART III: HIGHER DIMENSIONAL AND NON-MARKOVIAN QUEUES 295 Queueing Theory and Simulation PART III: HIGHER DIMENSIONAL AND NON-MARKOVIAN QUEUES 16 Queueing Systems with Two Types of Customers In this section, we discuss queueing systems with two types of customers.

More information

Solutions to Homework Discrete Stochastic Processes MIT, Spring 2011

Solutions to Homework Discrete Stochastic Processes MIT, Spring 2011 Exercise 1 Solutions to Homework 6 6.262 Discrete Stochastic Processes MIT, Spring 2011 Let {Y n ; n 1} be a sequence of rv s and assume that lim n E[ Y n ] = 0. Show that {Y n ; n 1} converges to 0 in

More information

Continuous-Time Markov Chain

Continuous-Time Markov Chain Continuous-Time Markov Chain Consider the process {X(t),t 0} with state space {0, 1, 2,...}. The process {X(t),t 0} is a continuous-time Markov chain if for all s, t 0 and nonnegative integers i, j, x(u),

More information

T. Liggett Mathematics 171 Final Exam June 8, 2011

T. Liggett Mathematics 171 Final Exam June 8, 2011 T. Liggett Mathematics 171 Final Exam June 8, 2011 1. The continuous time renewal chain X t has state space S = {0, 1, 2,...} and transition rates (i.e., Q matrix) given by q(n, n 1) = δ n and q(0, n)

More information

Continuous distributions

Continuous distributions CHAPTER 7 Continuous distributions 7.. Introduction A r.v. X is said to have a continuous distribution if there exists a nonnegative function f such that P(a X b) = ˆ b a f(x)dx for every a and b. distribution.)

More information

Performance Modelling of Computer Systems

Performance Modelling of Computer Systems Performance Modelling of Computer Systems Mirco Tribastone Institut für Informatik Ludwig-Maximilians-Universität München Fundamentals of Queueing Theory Tribastone (IFI LMU) Performance Modelling of Computer

More information

Data analysis and stochastic modeling

Data analysis and stochastic modeling Data analysis and stochastic modeling Lecture 7 An introduction to queueing theory Guillaume Gravier guillaume.gravier@irisa.fr with a lot of help from Paul Jensen s course http://www.me.utexas.edu/ jensen/ormm/instruction/powerpoint/or_models_09/14_queuing.ppt

More information

1 Delayed Renewal Processes: Exploiting Laplace Transforms

1 Delayed Renewal Processes: Exploiting Laplace Transforms IEOR 6711: Stochastic Models I Professor Whitt, Tuesday, October 22, 213 Renewal Theory: Proof of Blackwell s theorem 1 Delayed Renewal Processes: Exploiting Laplace Transforms The proof of Blackwell s

More information

M/G/1 queues and Busy Cycle Analysis

M/G/1 queues and Busy Cycle Analysis queues and Busy Cycle Analysis John C.S. Lui Department of Computer Science & Engineering The Chinese University of Hong Kong www.cse.cuhk.edu.hk/ cslui John C.S. Lui (CUHK) Computer Systems Performance

More information

ECE 313 Probability with Engineering Applications Fall 2000

ECE 313 Probability with Engineering Applications Fall 2000 Exponential random variables Exponential random variables arise in studies of waiting times, service times, etc X is called an exponential random variable with parameter λ if its pdf is given by f(u) =

More information

System Simulation Part II: Mathematical and Statistical Models Chapter 5: Statistical Models

System Simulation Part II: Mathematical and Statistical Models Chapter 5: Statistical Models System Simulation Part II: Mathematical and Statistical Models Chapter 5: Statistical Models Fatih Cavdur fatihcavdur@uludag.edu.tr March 20, 2012 Introduction Introduction The world of the model-builder

More information

E[X n ]= dn dt n M X(t). ). What is the mgf? Solution. Found this the other day in the Kernel matching exercise: 1 M X (t) =

E[X n ]= dn dt n M X(t). ). What is the mgf? Solution. Found this the other day in the Kernel matching exercise: 1 M X (t) = Chapter 7 Generating functions Definition 7.. Let X be a random variable. The moment generating function is given by M X (t) =E[e tx ], provided that the expectation exists for t in some neighborhood of

More information

MS&E 321 Spring Stochastic Systems June 1, 2013 Prof. Peter W. Glynn Page 1 of 10

MS&E 321 Spring Stochastic Systems June 1, 2013 Prof. Peter W. Glynn Page 1 of 10 MS&E 321 Spring 12-13 Stochastic Systems June 1, 2013 Prof. Peter W. Glynn Page 1 of 10 Section 3: Regenerative Processes Contents 3.1 Regeneration: The Basic Idea............................... 1 3.2

More information

Exercises Stochastic Performance Modelling. Hamilton Institute, Summer 2010

Exercises Stochastic Performance Modelling. Hamilton Institute, Summer 2010 Exercises Stochastic Performance Modelling Hamilton Institute, Summer Instruction Exercise Let X be a non-negative random variable with E[X ]

More information

Sampling Distributions

Sampling Distributions Sampling Distributions In statistics, a random sample is a collection of independent and identically distributed (iid) random variables, and a sampling distribution is the distribution of a function of

More information

Math Spring Practice for the final Exam.

Math Spring Practice for the final Exam. Math 4 - Spring 8 - Practice for the final Exam.. Let X, Y, Z be three independnet random variables uniformly distributed on [, ]. Let W := X + Y. Compute P(W t) for t. Honors: Compute the CDF function

More information

Chapter 8 Queuing Theory Roanna Gee. W = average number of time a customer spends in the system.

Chapter 8 Queuing Theory Roanna Gee. W = average number of time a customer spends in the system. 8. Preliminaries L, L Q, W, W Q L = average number of customers in the system. L Q = average number of customers waiting in queue. W = average number of time a customer spends in the system. W Q = average

More information

HITTING TIME IN AN ERLANG LOSS SYSTEM

HITTING TIME IN AN ERLANG LOSS SYSTEM Probability in the Engineering and Informational Sciences, 16, 2002, 167 184+ Printed in the U+S+A+ HITTING TIME IN AN ERLANG LOSS SYSTEM SHELDON M. ROSS Department of Industrial Engineering and Operations

More information

M/G/1 and M/G/1/K systems

M/G/1 and M/G/1/K systems M/G/1 and M/G/1/K systems Dmitri A. Moltchanov dmitri.moltchanov@tut.fi http://www.cs.tut.fi/kurssit/elt-53606/ OUTLINE: Description of M/G/1 system; Methods of analysis; Residual life approach; Imbedded

More information

P (L d k = n). P (L(t) = n),

P (L d k = n). P (L(t) = n), 4 M/G/1 queue In the M/G/1 queue customers arrive according to a Poisson process with rate λ and they are treated in order of arrival The service times are independent and identically distributed with

More information

Lecture 4a: Continuous-Time Markov Chain Models

Lecture 4a: Continuous-Time Markov Chain Models Lecture 4a: Continuous-Time Markov Chain Models Continuous-time Markov chains are stochastic processes whose time is continuous, t [0, ), but the random variables are discrete. Prominent examples of continuous-time

More information

MS&E 321 Spring Stochastic Systems June 1, 2013 Prof. Peter W. Glynn Page 1 of 7

MS&E 321 Spring Stochastic Systems June 1, 2013 Prof. Peter W. Glynn Page 1 of 7 MS&E 321 Spring 12-13 Stochastic Systems June 1, 213 Prof. Peter W. Glynn Page 1 of 7 Section 9: Renewal Theory Contents 9.1 Renewal Equations..................................... 1 9.2 Solving the Renewal

More information

Assignment 3 with Reference Solutions

Assignment 3 with Reference Solutions Assignment 3 with Reference Solutions Exercise 3.: Poisson Process Given are k independent sources s i of jobs as shown in the figure below. The interarrival time between jobs for each source is exponentially

More information

Interlude: Practice Final

Interlude: Practice Final 8 POISSON PROCESS 08 Interlude: Practice Final This practice exam covers the material from the chapters 9 through 8. Give yourself 0 minutes to solve the six problems, which you may assume have equal point

More information

Lecture 10: Semi-Markov Type Processes

Lecture 10: Semi-Markov Type Processes Lecture 1: Semi-Markov Type Processes 1. Semi-Markov processes (SMP) 1.1 Definition of SMP 1.2 Transition probabilities for SMP 1.3 Hitting times and semi-markov renewal equations 2. Processes with semi-markov

More information

RENEWAL PROCESSES AND POISSON PROCESSES

RENEWAL PROCESSES AND POISSON PROCESSES 1 RENEWAL PROCESSES AND POISSON PROCESSES Andrea Bobbio Anno Accademico 1997-1998 Renewal and Poisson Processes 2 Renewal Processes A renewal process is a point process characterized by the fact that the

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY Department of Electrical Engineering and Computer Science

MASSACHUSETTS INSTITUTE OF TECHNOLOGY Department of Electrical Engineering and Computer Science MASSACHUSETTS INSTITUTE OF TECHNOLOGY Department of Electrical Engineering and Computer Science 6.262 Discrete Stochastic Processes Midterm Quiz April 6, 2010 There are 5 questions, each with several parts.

More information

Class 11 Non-Parametric Models of a Service System; GI/GI/1, GI/GI/n: Exact & Approximate Analysis.

Class 11 Non-Parametric Models of a Service System; GI/GI/1, GI/GI/n: Exact & Approximate Analysis. Service Engineering Class 11 Non-Parametric Models of a Service System; GI/GI/1, GI/GI/n: Exact & Approximate Analysis. G/G/1 Queue: Virtual Waiting Time (Unfinished Work). GI/GI/1: Lindley s Equations

More information

ISyE 6761 (Fall 2016) Stochastic Processes I

ISyE 6761 (Fall 2016) Stochastic Processes I Fall 216 TABLE OF CONTENTS ISyE 6761 (Fall 216) Stochastic Processes I Prof. H. Ayhan Georgia Institute of Technology L A TEXer: W. KONG http://wwong.github.io Last Revision: May 25, 217 Table of Contents

More information

(b) What is the variance of the time until the second customer arrives, starting empty, assuming that we measure time in minutes?

(b) What is the variance of the time until the second customer arrives, starting empty, assuming that we measure time in minutes? IEOR 3106: Introduction to Operations Research: Stochastic Models Fall 2006, Professor Whitt SOLUTIONS to Final Exam Chapters 4-7 and 10 in Ross, Tuesday, December 19, 4:10pm-7:00pm Open Book: but only

More information

Part I Stochastic variables and Markov chains

Part I Stochastic variables and Markov chains Part I Stochastic variables and Markov chains Random variables describe the behaviour of a phenomenon independent of any specific sample space Distribution function (cdf, cumulative distribution function)

More information

1 Basic concepts from probability theory

1 Basic concepts from probability theory Basic concepts from probability theory This chapter is devoted to some basic concepts from probability theory.. Random variable Random variables are denoted by capitals, X, Y, etc. The expected value or

More information

Chapter 2. Poisson Processes

Chapter 2. Poisson Processes Chapter 2. Poisson Processes Prof. Ai-Chun Pang Graduate Institute of Networking and Multimedia, epartment of Computer Science and Information Engineering, National Taiwan University, Taiwan utline Introduction

More information

Solutions For Stochastic Process Final Exam

Solutions For Stochastic Process Final Exam Solutions For Stochastic Process Final Exam (a) λ BMW = 20 0% = 2 X BMW Poisson(2) Let N t be the number of BMWs which have passes during [0, t] Then the probability in question is P (N ) = P (N = 0) =

More information

The Transition Probability Function P ij (t)

The Transition Probability Function P ij (t) The Transition Probability Function P ij (t) Consider a continuous time Markov chain {X(t), t 0}. We are interested in the probability that in t time units the process will be in state j, given that it

More information

Adventures in Stochastic Processes

Adventures in Stochastic Processes Sidney Resnick Adventures in Stochastic Processes with Illustrations Birkhäuser Boston Basel Berlin Table of Contents Preface ix CHAPTER 1. PRELIMINARIES: DISCRETE INDEX SETS AND/OR DISCRETE STATE SPACES

More information

Contents Preface The Exponential Distribution and the Poisson Process Introduction to Renewal Theory

Contents Preface The Exponential Distribution and the Poisson Process Introduction to Renewal Theory Contents Preface... v 1 The Exponential Distribution and the Poisson Process... 1 1.1 Introduction... 1 1.2 The Density, the Distribution, the Tail, and the Hazard Functions... 2 1.2.1 The Hazard Function

More information

Continuous Time Processes

Continuous Time Processes page 102 Chapter 7 Continuous Time Processes 7.1 Introduction In a continuous time stochastic process (with discrete state space), a change of state can occur at any time instant. The associated point

More information

λ λ λ In-class problems

λ λ λ In-class problems In-class problems 1. Customers arrive at a single-service facility at a Poisson rate of 40 per hour. When two or fewer customers are present, a single attendant operates the facility, and the service time

More information

IEOR 6711: Stochastic Models I, Fall 2003, Professor Whitt. Solutions to Final Exam: Thursday, December 18.

IEOR 6711: Stochastic Models I, Fall 2003, Professor Whitt. Solutions to Final Exam: Thursday, December 18. IEOR 6711: Stochastic Models I, Fall 23, Professor Whitt Solutions to Final Exam: Thursday, December 18. Below are six questions with several parts. Do as much as you can. Show your work. 1. Two-Pump Gas

More information

GI/M/1 and GI/M/m queuing systems

GI/M/1 and GI/M/m queuing systems GI/M/1 and GI/M/m queuing systems Dmitri A. Moltchanov moltchan@cs.tut.fi http://www.cs.tut.fi/kurssit/tlt-2716/ OUTLINE: GI/M/1 queuing system; Methods of analysis; Imbedded Markov chain approach; Waiting

More information

IEOR 4106: Spring Solutions to Homework Assignment 7: Due on Tuesday, March 22.

IEOR 4106: Spring Solutions to Homework Assignment 7: Due on Tuesday, March 22. IEOR 46: Spring Solutions to Homework Assignment 7: Due on Tuesday, March. More of Chapter 5: Read the rest of Section 5.3, skipping Examples 5.7 (Coupon Collecting), 5. (Insurance claims)and Subsection

More information

Poisson Processes. Particles arriving over time at a particle detector. Several ways to describe most common model.

Poisson Processes. Particles arriving over time at a particle detector. Several ways to describe most common model. Poisson Processes Particles arriving over time at a particle detector. Several ways to describe most common model. Approach 1: a) numbers of particles arriving in an interval has Poisson distribution,

More information

QUEUING SYSTEM. Yetunde Folajimi, PhD

QUEUING SYSTEM. Yetunde Folajimi, PhD QUEUING SYSTEM Yetunde Folajimi, PhD Part 2 Queuing Models Queueing models are constructed so that queue lengths and waiting times can be predicted They help us to understand and quantify the effect of

More information

Chapter 5. Continuous-Time Markov Chains. Prof. Shun-Ren Yang Department of Computer Science, National Tsing Hua University, Taiwan

Chapter 5. Continuous-Time Markov Chains. Prof. Shun-Ren Yang Department of Computer Science, National Tsing Hua University, Taiwan Chapter 5. Continuous-Time Markov Chains Prof. Shun-Ren Yang Department of Computer Science, National Tsing Hua University, Taiwan Continuous-Time Markov Chains Consider a continuous-time stochastic process

More information

An Analysis of the Preemptive Repeat Queueing Discipline

An Analysis of the Preemptive Repeat Queueing Discipline An Analysis of the Preemptive Repeat Queueing Discipline Tony Field August 3, 26 Abstract An analysis of two variants of preemptive repeat or preemptive repeat queueing discipline is presented: one in

More information

Stochastic Processes

Stochastic Processes Stochastic Processes 8.445 MIT, fall 20 Mid Term Exam Solutions October 27, 20 Your Name: Alberto De Sole Exercise Max Grade Grade 5 5 2 5 5 3 5 5 4 5 5 5 5 5 6 5 5 Total 30 30 Problem :. True / False

More information

STAT 380 Markov Chains

STAT 380 Markov Chains STAT 380 Markov Chains Richard Lockhart Simon Fraser University Spring 2016 Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring 2016 1 / 38 1/41 PoissonProcesses.pdf (#2) Poisson Processes

More information

Stat410 Probability and Statistics II (F16)

Stat410 Probability and Statistics II (F16) Stat4 Probability and Statistics II (F6 Exponential, Poisson and Gamma Suppose on average every /λ hours, a Stochastic train arrives at the Random station. Further we assume the waiting time between two

More information

Stochastic process. X, a series of random variables indexed by t

Stochastic process. X, a series of random variables indexed by t Stochastic process X, a series of random variables indexed by t X={X(t), t 0} is a continuous time stochastic process X={X(t), t=0,1, } is a discrete time stochastic process X(t) is the state at time t,

More information

3. Poisson Processes (12/09/12, see Adult and Baby Ross)

3. Poisson Processes (12/09/12, see Adult and Baby Ross) 3. Poisson Processes (12/09/12, see Adult and Baby Ross) Exponential Distribution Poisson Processes Poisson and Exponential Relationship Generalizations 1 Exponential Distribution Definition: The continuous

More information

Regenerative Processes. Maria Vlasiou. June 25, 2018

Regenerative Processes. Maria Vlasiou. June 25, 2018 Regenerative Processes Maria Vlasiou June 25, 218 arxiv:144.563v1 [math.pr] 22 Apr 214 Abstract We review the theory of regenerative processes, which are processes that can be intuitively seen as comprising

More information

Figure 10.1: Recording when the event E occurs

Figure 10.1: Recording when the event E occurs 10 Poisson Processes Let T R be an interval. A family of random variables {X(t) ; t T} is called a continuous time stochastic process. We often consider T = [0, 1] and T = [0, ). As X(t) is a random variable

More information

RELATING TIME AND CUSTOMER AVERAGES FOR QUEUES USING FORWARD COUPLING FROM THE PAST

RELATING TIME AND CUSTOMER AVERAGES FOR QUEUES USING FORWARD COUPLING FROM THE PAST J. Appl. Prob. 45, 568 574 (28) Printed in England Applied Probability Trust 28 RELATING TIME AND CUSTOMER AVERAGES FOR QUEUES USING FORWARD COUPLING FROM THE PAST EROL A. PEKÖZ, Boston University SHELDON

More information

2905 Queueing Theory and Simulation PART IV: SIMULATION

2905 Queueing Theory and Simulation PART IV: SIMULATION 2905 Queueing Theory and Simulation PART IV: SIMULATION 22 Random Numbers A fundamental step in a simulation study is the generation of random numbers, where a random number represents the value of a random

More information

Introduction to Markov Chains, Queuing Theory, and Network Performance

Introduction to Markov Chains, Queuing Theory, and Network Performance Introduction to Markov Chains, Queuing Theory, and Network Performance Marceau Coupechoux Telecom ParisTech, departement Informatique et Réseaux marceau.coupechoux@telecom-paristech.fr IT.2403 Modélisation

More information

EE126: Probability and Random Processes

EE126: Probability and Random Processes EE126: Probability and Random Processes Lecture 19: Poisson Process Abhay Parekh UC Berkeley March 31, 2011 1 1 Logistics 2 Review 3 Poisson Processes 2 Logistics 3 Poisson Process A continuous version

More information

Final Solutions Fri, June 8

Final Solutions Fri, June 8 EE178: Probabilistic Systems Analysis, Spring 2018 Final Solutions Fri, June 8 1. Small problems (62 points) (a) (8 points) Let X 1, X 2,..., X n be independent random variables uniformly distributed on

More information

CDA5530: Performance Models of Computers and Networks. Chapter 3: Review of Practical

CDA5530: Performance Models of Computers and Networks. Chapter 3: Review of Practical CDA5530: Performance Models of Computers and Networks Chapter 3: Review of Practical Stochastic Processes Definition Stochastic ti process X = {X(t), t T} is a collection of random variables (rvs); one

More information

EE126: Probability and Random Processes

EE126: Probability and Random Processes EE126: Probability and Random Processes Lecture 18: Poisson Process Abhay Parekh UC Berkeley March 17, 2011 1 1 Review 2 Poisson Process 2 Bernoulli Process An arrival process comprised of a sequence of

More information

Computer Systems Modelling

Computer Systems Modelling Computer Systems Modelling Computer Laboratory Computer Science Tripos, Part II Lent Term 2010/11 R. J. Gibbens Problem sheet William Gates Building 15 JJ Thomson Avenue Cambridge CB3 0FD http://www.cl.cam.ac.uk/

More information

Introduction to Queueing Theory

Introduction to Queueing Theory Introduction to Queueing Theory Raj Jain Washington University in Saint Louis Saint Louis, MO 63130 Jain@cse.wustl.edu Audio/Video recordings of this lecture are available at: http://www.cse.wustl.edu/~jain/cse567-11/

More information

Systems Simulation Chapter 6: Queuing Models

Systems Simulation Chapter 6: Queuing Models Systems Simulation Chapter 6: Queuing Models Fatih Cavdur fatihcavdur@uludag.edu.tr April 2, 2014 Introduction Introduction Simulation is often used in the analysis of queuing models. A simple but typical

More information

UCSD ECE250 Handout #27 Prof. Young-Han Kim Friday, June 8, Practice Final Examination (Winter 2017)

UCSD ECE250 Handout #27 Prof. Young-Han Kim Friday, June 8, Practice Final Examination (Winter 2017) UCSD ECE250 Handout #27 Prof. Young-Han Kim Friday, June 8, 208 Practice Final Examination (Winter 207) There are 6 problems, each problem with multiple parts. Your answer should be as clear and readable

More information

Probability and Statistics Concepts

Probability and Statistics Concepts University of Central Florida Computer Science Division COT 5611 - Operating Systems. Spring 014 - dcm Probability and Statistics Concepts Random Variable: a rule that assigns a numerical value to each

More information

LECTURE #6 BIRTH-DEATH PROCESS

LECTURE #6 BIRTH-DEATH PROCESS LECTURE #6 BIRTH-DEATH PROCESS 204528 Queueing Theory and Applications in Networks Assoc. Prof., Ph.D. (รศ.ดร. อน นต ผลเพ ม) Computer Engineering Department, Kasetsart University Outline 2 Birth-Death

More information

SOLUTIONS IEOR 3106: Second Midterm Exam, Chapters 5-6, November 8, 2012

SOLUTIONS IEOR 3106: Second Midterm Exam, Chapters 5-6, November 8, 2012 SOLUTIONS IEOR 3106: Second Midterm Exam, Chapters 5-6, November 8, 2012 This exam is closed book. YOU NEED TO SHOW YOUR WORK. Honor Code: Students are expected to behave honorably, following the accepted

More information

Optional Stopping Theorem Let X be a martingale and T be a stopping time such

Optional Stopping Theorem Let X be a martingale and T be a stopping time such Plan Counting, Renewal, and Point Processes 0. Finish FDR Example 1. The Basic Renewal Process 2. The Poisson Process Revisited 3. Variants and Extensions 4. Point Processes Reading: G&S: 7.1 7.3, 7.10

More information

IEOR 3106: Second Midterm Exam, Chapters 5-6, November 7, 2013

IEOR 3106: Second Midterm Exam, Chapters 5-6, November 7, 2013 IEOR 316: Second Midterm Exam, Chapters 5-6, November 7, 13 SOLUTIONS Honor Code: Students are expected to behave honorably, following the accepted code of academic honesty. You may keep the exam itself.

More information

Statistics 253/317 Introduction to Probability Models. Winter Midterm Exam Monday, Feb 10, 2014

Statistics 253/317 Introduction to Probability Models. Winter Midterm Exam Monday, Feb 10, 2014 Statistics 253/317 Introduction to Probability Models Winter 2014 - Midterm Exam Monday, Feb 10, 2014 Student Name (print): (a) Do not sit directly next to another student. (b) This is a closed-book, closed-note

More information

Simulating events: the Poisson process

Simulating events: the Poisson process Simulating events: te Poisson process p. 1/15 Simulating events: te Poisson process Micel Bierlaire micel.bierlaire@epfl.c Transport and Mobility Laboratory Simulating events: te Poisson process p. 2/15

More information

IBM Almaden Research Center, San Jose, California, USA

IBM Almaden Research Center, San Jose, California, USA This article was downloaded by: [Stanford University] On: 20 July 2010 Access details: Access Details: [subscription number 731837804] Publisher Taylor & Francis Informa Ltd Registered in England and Wales

More information

We introduce methods that are useful in:

We introduce methods that are useful in: Instructor: Shengyu Zhang Content Derived Distributions Covariance and Correlation Conditional Expectation and Variance Revisited Transforms Sum of a Random Number of Independent Random Variables more

More information

M/M/1 Queueing System with Delayed Controlled Vacation

M/M/1 Queueing System with Delayed Controlled Vacation M/M/1 Queueing System with Delayed Controlled Vacation Yonglu Deng, Zhongshan University W. John Braun, University of Winnipeg Yiqiang Q. Zhao, University of Winnipeg Abstract An M/M/1 queue with delayed

More information

Performance Evaluation of Queuing Systems

Performance Evaluation of Queuing Systems Performance Evaluation of Queuing Systems Introduction to Queuing Systems System Performance Measures & Little s Law Equilibrium Solution of Birth-Death Processes Analysis of Single-Station Queuing Systems

More information

Simple queueing models

Simple queueing models Simple queueing models c University of Bristol, 2012 1 M/M/1 queue This model describes a queue with a single server which serves customers in the order in which they arrive. Customer arrivals constitute

More information

Point Process Control

Point Process Control Point Process Control The following note is based on Chapters I, II and VII in Brémaud s book Point Processes and Queues (1981). 1 Basic Definitions Consider some probability space (Ω, F, P). A real-valued

More information

E-Companion to Fully Sequential Procedures for Large-Scale Ranking-and-Selection Problems in Parallel Computing Environments

E-Companion to Fully Sequential Procedures for Large-Scale Ranking-and-Selection Problems in Parallel Computing Environments E-Companion to Fully Sequential Procedures for Large-Scale Ranking-and-Selection Problems in Parallel Computing Environments Jun Luo Antai College of Economics and Management Shanghai Jiao Tong University

More information

Lecture 7: Chapter 7. Sums of Random Variables and Long-Term Averages

Lecture 7: Chapter 7. Sums of Random Variables and Long-Term Averages Lecture 7: Chapter 7. Sums of Random Variables and Long-Term Averages ELEC206 Probability and Random Processes, Fall 2014 Gil-Jin Jang gjang@knu.ac.kr School of EE, KNU page 1 / 15 Chapter 7. Sums of Random

More information