STAT 380 Markov Chains

Transcription

1 STAT 380 Markov Chains Richard Lockhart Simon Fraser University Spring 2016 Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / 38 1/41 PoissonProcesses.pdf (#2)

2 Poisson Processes Particles arriving over time at a particle detector. Several ways to describe most common model. Approach 1: numbers of particles arriving in an interval has Poisson distribution, mean proportional to length of interval, numbers in several non-overlapping intervals independent. Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / 38 2/41 PoissonProcesses.pdf (2/41)

3 Notation, formal assumptions For s < t, denote number of arrivals in (s, t] byn(s, t). Model is 1 N(s, t) hasapoisson(λ(t s)) distribution. 2 For 0 s 1 < t 1 s 2 < t 2 s k < t k the variables N(s i, t i ); i =1,...,k are independent. Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / 38 3/41 PoissonProcesses.pdf (3/41)

4 Approach 2 Let 0 < S 1 < S 2 < be the times at which the particles arrive. Let T i = S i S i 1 with S 0 =0byconvention. Then T 1, T 2,... are independent Exponential random variables with mean 1/λ. Note P(T i > x) =e λx is called survival function of T i. Approaches 1 and 2 are equivalent. Both are deductions of a model based on local behaviour of process. Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / 38 4/41 PoissonProcesses.pdf (4/41)

5 Approach 3 Assume: 1 given all the points in [0, t] the probability of 1 point in the interval (t, t + h] is of the form λh + o(h) 2 given all the points in [0, t] the probability of 2 or more points in interval (t, t + h] is of the form o(h) All 3 approaches are equivalent. Ishow: 3implies1,1implies2and2implies3. First explain o, O. Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / 38 5/41 PoissonProcesses.pdf (5/41)

6 6/41 PoissonProcesses.pdf (6/41)

7 Landau s big and little O Notation: given functions f and g we write f (h) =g(h)+o(h) provided f (h) g(h) lim =0 h 0 h Aside: if there is a constant M such that lim sup f (h) g(h) h 0 h M we say f (h) =g(h)+o(h) Another form is f (h) =g(h)+o(h) means there is δ>0andm s.t. for all h <δ f (h) g(h) Mh Idea: o(h) is tiny compared to h while O(h) is (very) roughly the same size as h. Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / 38 7/41 PoissonProcesses.pdf (7/41)

8 Model 3 implies 1 Fix t, definef t (s) to be conditional probability of 0 points in (t, t + s] given value of process on [0, t]. Derive differential equation for f. Given process on [0, t] and0pointsin(t, t + s] probability of no points in (t, t + s + h] is f t+s (h) =1 λh + o(h) Given the process on [0, t] the probability of no points in (t, t + s] is f t (s). Using P(AB C) =P(A BC)P(B C) gives f t (s + h) =f t (s)f t+s (h) = f t (s)(1 λh + o(h)) Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / 38 8/41 PoissonProcesses.pdf (8/41)

9 3 implies 1 continued Now rearrange, divide by h to get f t (s + h) f t (s) h = λf t (s)+ o(h) h Let h 0andfind f t (s) = λf t (s) s Differential equation has solution f t (s) =f t (0) exp( λs) =exp( λs). Notice: survival function of exponential rv. Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / 38 9/41 PoissonProcesses.pdf (9/41)

10 General case Notation: N(t) =N(0, t). N(t) is a non-decreasing function of t. Let P k (t) =P(N(t) =k) Evaluate P k (t + h) byconditioningonn(s); 0 s < t and N(t) =j. Given N(t) =j probability that N(t + h) =k is conditional probability of k j points in (t, t + h]. So, for j k 2: P(N(t + h) =k N(t) =j, N(s), 0 s < t) =o(h). Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / 38 10/41 PoissonProcesses.pdf (10/41)

11 General case, continued For j = k 1wehave P(N(t + h) =k N(t) =k 1, N(s), 0 s < t) =λh + o(h) For j = k we have P(N(t + h) =k N(t) =k, N(s), 0 s < t) =1 λh + o(h) N is increasing so only consider j k. k P k (t + h) = P(N(t + h) =k N(t) =j)p j (t) j=0 = P k (t)(1 λh)+λhp k 1 (t)+o(h) Rearrange, divide by h and let h 0tget P k (t) = λp k(t)+λp k 1 (t) Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / 38 11/41 PoissonProcesses.pdf (11/41)

12 12/41 PoissonProcesses.pdf (12/41)

13 General case, continued III For k = 0 the term P k 1 is dropped and P 0(t) = λp 0 (t) Using P 0 (0) = 1 we get P 0 (t) =e λt Put this into the equation for k =1toget P 1(t) = λp 1 (t)+λe λt Multiply by e λt to see ( e λt P 1 (t)) = λ With P 1 (0) = 0 we get P 1 (t) =λte λt Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / 38 13/41 PoissonProcesses.pdf (13/41)

14 General case, continued IV For general k we have P k (0) = 0 and Check by induction that ( e λt P k (t)) = λe λt P k 1 (t) e λt P k (t) =(λt) k /k! Hence: N(t) haspoisson(λt) distribution. Similar ideas permit proof of P (N(s, t) =k N(u); 0 u s) = {λ(t s)}k e λ(t s) By induction can prove N has independent Poisson increments. k! Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / 38 14/41 PoissonProcesses.pdf (14/41)

15 15/41 PoissonProcesses.pdf (15/41)

16 Exponential Interarrival Times Suppose N is a Poisson Process. Define T 1, T 2,... to be the times between 0 and the first point, the first point and the second and so on. Fact: T 1, T 2,... are iid exponential rvs with mean 1/λ. Already did T 1 rigorously. The event T > t is exactly the event N(t) =0. So P(T > t) =exp( λt) which is the survival function of an exponential rv. Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / 38 16/41 PoissonProcesses.pdf (16/41)

17 Exponential Interarrival Times deduced IdocaseofT 1, T 2. Let t 1, t 2 be two positive numbers and s 1 = t 1, s 2 = t 1 + t 2. Consider event {t 1 < T 1 t 1 + δ 1 } {t 2 < T 2 t 2 + δ 2 }. This is almost the same as the intersection of the four events: N(0, t 1 ]=0 N(t 1, t 1 + δ 1 ]=1 N(t 1 + δ 1, t 1 + δ 1 + t 2 ]=0 N(s 2 + δ 1, s 2 + δ 1 + δ 2 ]=1 which has probability e λt 1 λδ 1 e λδ 1 e λt 2 λδ 2 e λδ 2 Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / 38 17/41 PoissonProcesses.pdf (17/41)

18 Less Rigor Divide by δ 1 δ 2. Let δ 1 and δ 2 go to 0 to get joint density of T 1, T 2 : λ 2 e λt 1 e λt 2 This is joint density of two independent exponential variates. Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / 38 18/41 PoissonProcesses.pdf (18/41)

19 More Rigor Find joint density of S 1,...,S k. Use change of variables to find joint density of T 1,...,T k. Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / 38 19/41 PoissonProcesses.pdf (19/41)

20 More Rigor, II First step: Compute P(0 < S 1 s 1 < S 2 s 2 < S k s k ) This is just the event of exactly 1 point in each interval (s i 1, s i ]for i =1,...,k 1(s 0 =0)andatleastonepointin(s k 1, s k ]. This event has probability k 1 {λ(s i s i 1 )e λ(s i s i 1 ) }( 1 e λ(s k s k 1 ) ) 1 Second step: write this in terms of joint cdf of S 1,...,S k. Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / 38 20/41 PoissonProcesses.pdf (20/41)

21 More Rigor, III, k =2 P(0 < S 1 s 1 < S 2 s 2 ) = F S1,S 2 (s 1, s 2 ) F S1,S 2 (s 1, s 1 ) Notice tacit assumption s 1 < s 2. Differentiate twice, that is, take to get Simplify to f S1,S 2 (s 1, s 2 )= Recall tacit assumption to get 2 s 1 s 2 2 s 1 s 2 λs 1 e λs 1 λ 2 e λs 2 (1 e λ(s 2 s 1 ) ) f S1,S 2 (s 1, s 2 )=λ 2 e λs 2 1(0 < s 1 < s 2 ) Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / 38 21/41 PoissonProcesses.pdf (21/41)

22 More Rigor, IV, change of variables Now compute the joint cdf of T 1, T 2 by This is F T1,T 2 (t 1, t 2 )=P(S 1 < t 1, S 2 S 1 < t 2 ) P(S 1 < t 1, S 2 S 1 < t 2 )= Differentiate twice to get = λ t1 0 t1 0 s1 +t 2 λ 2 e λs 2 ds 2 ds 1 s 1 ( ) e λs 1 e λ(s 1+t 2 ) ds 1 =1 e λt 1 e λt 2 + e λ(t 1+t 2 ) f T1,T 2 (t 1, t 2 )=λe λt 1 λe λt 2 This is joint density of 2 independent exponential rvs. Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / 38 22/41 PoissonProcesses.pdf (22/41)

23 Summary so far Have shown: Instantaneous rates model implies independent Poisson increments model Independent Poisson increments model implies independent exponential interarrivals. Next: show independent exponential interarrivals implies the instantaneous rates model. Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / 38 23/41 PoissonProcesses.pdf (23/41)

24 Exponential interarrivals implies rates Suppose T 1,... iid exponential rvs with means 1/λ. Define N t by N t = k if and only if T T k t T T k+1 Let A be the event N(s) =n(s); 0 < s t. We are to show and P(N(t, t + h] =1 N(t) =k, A) =λh + o(h) P(N(t, t + h] 2 N(t) =k, A) =o(h) If n(s) is a possible trajectory consistent with N(t) =k then n has jumps at points s 1 t 1, s 2 t 1 + t 2,...,s k t t k < t and at no other points in (0, t]. Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / :13:19 24/41 PoissonProcesses.pdf (24/41)

25 Continued So given N(s) =n(s); 0 < s t with n(t) =k we are essentially being given T 1 = t 1,...,T k = t k, T k+1 > t s k and asked the conditional probabilty in the first case of the event B given by t s k < T k+1 t s k + h < T k+2 + T k+1. Conditioning on T 1,...,T k irrelevant (independence). P(N(t, t + h] =1 N(t) =k, A)/h = P(B T k+1 > t s k )/h = P(B) he λ(t s k ) Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / :13:19 25/41 PoissonProcesses.pdf (25/41)

26 Continued The numerator may be evaluated by integration: P(B) = t sk +h t s k t s k +h s 1 λ 2 e λ(s 1+s 2 ) ds 2 ds 1 Let h 0togetthelimit as required. The computation of is similar. P(N(t, t + h] =1 N(t) =k, A)/h λ lim P(N(t, t + h] 2 N(t) =k, A)/h h 0 Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / :13:19 26/41 PoissonProcesses.pdf (26/41)

27 Properties of exponential rvs Convolution: IfX and Y independent rvs with densities f and g respectively and Z = X + Y then P(Z z) = Differentiating wrt z we get f Z (z) = z x f (x)g(y)dydx f (x)g(z x)dx This integral is called the convolution of densities f and g. If T 1,...,T n iid Exponential(λ) thens n = T T n has a Gamma(n,λ) distribution. Density of S n is for s > 0. f Sn (s) =λ(λs) n 1 e λs /(n 1)! Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / :13:19 27/41 PoissonProcesses.pdf (27/41)

28 Proof Then P(S n > s) =P(N(0, s] < n) = f Sn (s) = d ds P(S n s) = d ds {1 P(S n > s)} n 1 (λs) j e λs /j! j=0 = λ j=1 This telescopes to n 1 = λe λs n 1 { j(λs) j 1 (λs) j} e λs j=1 { (λs) j j! j! } (λs)j 1 + λe λs (j 1)! f Sn (s) =λ(λs) n 1 e λs /(n 1)! + λe λs Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / :13:19 28/41 PoissonProcesses.pdf (28/41)

29 Extreme Values Suppose X 1,...,X n are independent exponential rvs with means 1/λ 1,...,1/λ n Then Y =min{x 1,...,X n } has exponential distribution with mean Proof: 1 λ λ n P(Y > y) =P( kx k > y) = e λ k y = e λ k y Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / :13:19 29/41 PoissonProcesses.pdf (29/41)

30 Memoryless Property Suppose X has exponential distribution. Conditional distribution of X x given X x is also exponential. Proof: P(X x > y X x) = P(X > x + y, X x) P(X > x) = PX > x + y) P(X x) = e λ(x+y) e λx = e λy Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / :13:19 30/41 PoissonProcesses.pdf (30/41)

31 Hazard Rates Assume rv T > 0withdensityf and cdf F Hazard rate, or instantaneous failure rate, is This is just r(t) = lim δ 0 P(t < T t + δ T t) δ r(t) = f (t) 1 F (t) For an exponential random variable with mean 1/λ this is h(t) = λe λt e λt = λ The exponential distribution has constant failure rate. Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / :13:19 31/41 PoissonProcesses.pdf (31/41)

32 The Weibull distribution Weibull random variables have density, for t > 0, f (t λ, α) =λ(λt) α 1 e (λt)α. The corresponding survival function is, for t > 0, 1 F (t) =e (λt)α The hazard rate is r(t) =λ(λt) α 1 Hazard rate is increasing for α>1, decreasing for α<1. For α = 1 this is the exponential distribution. Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / :13:19 32/41 PoissonProcesses.pdf (32/41)

33 Weibull distribution, continued Since r(t) = we can integrate to find df (t)/dt 1 F (t) = d log(1 F (t)) dt 1 F (t) =exp{ t 0 r(s)ds}. So r determines F and f. Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / :13:19 33/41 PoissonProcesses.pdf (33/41)

34 Properties of Poisson Processes 1) If N 1 and N 2 are independent Poisson processes with rates λ 1 and λ 2, respectively, then N = N 1 + N 2 is a Poisson process with rate λ 1 + λ 2. 2) Suppose N is a Poisson process with rate λ. Supposeeachpointis marked with a label, say one of L 1,...,L r,independentlyofallother occurences. Suppose p i is the probability that a given point receives label L i. Let N i count the points with label i (so that N = N N r ). Then N 1,...,N r are independent Poisson processes with rates p i λ. Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / :13:19 34/41 PoissonProcesses.pdf (34/41)

35 Properties of Poisson Processes 3) Suppose U 1, U 2,... independent rvs, each uniformly distributed on [0, T ]. Suppose M is a Poisson(λT ) random variable independent of the U s. Let N(t) = M 1(U i t) Then N is a Poisson process on [0, T ] with rate λ. 1 Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / :13:19 35/41 PoissonProcesses.pdf (35/41)

36 Properties of Poisson Processes 4) Suppose N is a Poisson process with rate λ. Let S 1 < S 2 < be the times at which points arrive Given N(T )=ns 1,...,S n have the same distribution as the order statistics of a sample of size n from the uniform distribution on [0, T ]. 5) Given S n+1 = T, S 1,...,S n have the same distribution as the order statistics of a sample of size n from the uniform distribution on [0, T ]. Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / :13:19 36/41 PoissonProcesses.pdf (36/41)

37 Indications of some proofs 1) N 1,...,N r independent Poisson processes rates λ i, N = N i. Let A h be the event of 2 or more points in N in the time interval (t, t + h] Let B h be the event of exactly one point in N in the time interval (t, t + h]. Let A ih and B ih be the corresponding events for N i. Let H t denote the history of the processes up to time t; wecondition on H t. We are given: P(A ih H t )=o(h) and P(B ih H t )=λ i h + o(h). Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / :13:19 37/41 PoissonProcesses.pdf (37/41)

38 Indications of some proofs, II Note that A h r (B ih B jh ) i=1 A ih i j Since P(B ih B jh H t )=P(B ih H t )P(B jh H t ) =(λ i h + o(h))(λ j h + o(h)) = O(h 2 ) = o(h) we have checked one of the two infinitesimal conditions for a Poisson process. Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / :13:19 38/41 PoissonProcesses.pdf (38/41)

39 Indications of some proofs, III Next let C h be the event of no points in N in the time interval (t, t + h]. Let C ih be the same for N i. Then shows P(C h H t )=P( C ih H t ) = P(C ih H t ) = (1 λ i h + o(h)) =1 ( λ i )h + o(h) P(B h H t )=1 P(C h H t ) P(A h H t ) =( λ i )h + o(h) Hence N is a Poisson process with rate λ i. Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / :13:19 39/41 PoissonProcesses.pdf (39/41)

40 Indications of some proofs, IV 2) The infinitesimal approach used for 1 can do part of this. See text for rest. Events defined as in 1): Define event B ih : there is one point in N i in (t, t + h] Define event B h :there is exactly one point in any of the r processes, B ih is union of B h and the subset of A h where there are two or more points in N in (t, t + h] butexactlyoneislabeledi. Since P(A h H t )=o(h) P(B ih H t )=p i P(B h H t )+o(h) = p i (λh + o(h)) + o(h) = p i λh + o(h) Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / :13:19 40/41 PoissonProcesses.pdf (40/41)

41 Indications of some proofs, V Similarly, A ih is a subset of A h so P(A ih H t )=o(h) This shows each N i is Poisson with rate λp i. To get independence requires more work; see the text for the algebraic method which is easier. Richard Lockhart (Simon Fraser University) STAT 380 Markov Chains Spring / :13:19 41/41 PoissonProcesses.pdf (41/41)