Outline Continuous-time Markov Process Poisson Process Thinning Conditioning on the Number of Events Generalizations

Size: px

Start display at page:

Download "Outline Continuous-time Markov Process Poisson Process Thinning Conditioning on the Number of Events Generalizations"

June Frederica Mitchell
5 years ago
Views:

1 Expoetial Distributio ad Poisso Process Page 1 Outlie Cotiuous-time Markov Process Poisso Process Thiig Coditioig o the Number of Evets Geeralizatios

2 Radom Vectors utdallas Probability ad Stochastic Processes Overview of Radom Variables ad Stochastic Processes Discrete-time: {X : 0, Z}; Cotiuous-time: {X(t): t 0, t R} Discrete state-space if each X or X(t) has a coutable rage; Otherwise, Cotiuous state-space. Page 2 Ucoutable Rage Cotiuous Radom Variables Cotiuous State Space Ivetory Models X +1 = X + q D q, D R Browia Process Coutable Rage Discrete Radom Variables Discrete State Space Markov Chais Poisso Process Probability Discrete-time Cotiuous-time Stochastic Processes Differece betwee a ifiite-legth radom vector & a discrete-time stochastic process?» Similarity, they are both deoted as {X : 0, Z}.» Differece is i the treatmet of depedece: Radom vectors of oly short legths (2-3 rvs) ca be studied if the rvs are depedet; Or assume structured depedece through copulas. Otherwise, we work with idepedet ad idetically distributed sequeces. I discrete-time stochastic processes, assume Markov Property for limited depedece

3 Radom Vectors utdallas Stochastic Processes Page 3 Ucoutable Rage Cotiuous Radom Variables Cotiuous State Space Ivetory Models X +1 = X + q D q, D R Browia Process Coutable Rage Discrete Radom Variables Discrete State Space Markov Chais Poisso Process Probability Discrete-time Stochastic Processes Cotiuous-time Differece betwee a discrete-time stochastic process & cotiuous-time stochastic process? Similarity, limited depedece is still sought. Differece is i the cotiuity of the process i time:» Cotiuity is ot a issue for processes with a discrete state space» Whe the state space is cotiuous, we woder how X(t) & X(t + ε) differ.» For radom variables, almost sure cotiuity: P lim ε 0 ω Ω: X t = X t ε = 1. Ex: EURUSD: The exchage rate betwee Euro ad US dollar. The stochastic process EURUSD(t) ca be geerally cotiuous except whe FED or ECB take actio to move the exchage rate i a certai directio. Immediately after some bak board meetigs, EURUSD(t) ca jump. Cetral bak itervetios ca cause jumps.

4 Seekig Coveiece Page 4 Covetio: A jump that happes at time t is icluded i X(t) but ot i X t ε Two commo properties Idepedet icremets X t 1 X s 1 X(t 2 ) X(s 2 ) for disjoit itervals (s 1, t 1 ] ad (s 2, t 2 ]. Statioary icremets P(X(s + a) X(s) = x) = P(X(t + a) X(t) = x). Ex: Idepedet icremets imply P(X(t) = x t X(r) = x r for r (0, s]) = P(X(t) = x t X(s) = x s ), which is the Markov property. Cotiuity property Right-cotiuous evolutio P lim X t + ε = X t = 1, i.e., X t is right-cotiuous almost surely. t 0 Ex: Let X t, ω = U t (ω) ad X t + ε, ω = U t+ε (ω) for U t, U t+ε iid from U(0,1). Check for covergece i probability. P X t + ε X t 0.5 = Area of two triagles i uit square = 1 4 X t + ε does ot coverge X t i probability. So it does ot coverge almost surely. X(t) is ot right cotiuous. It is ot left-cotiuous either. Ca X(t) be used to model a stock price? A process with Idepedet & Statioary icremets, Right-cotiuous evolutio is a Lévy process. Lévy-Itô Decompositio: Lévy Process = Poisso Process + Browia Process + Martigale Determiistic drift + Process Liear i Time A Martigale satisfies E X(t) X(s) = x s = x s for s t.

Coutig Process Page 5 A coutig process {N(t): t 0} is a cotiuous-time stochastic process with atural umbers as its state-space ad N(t) is the umber of evets that occur over iterval (0, t].

5 Coutig Process Page 5 A coutig process {N(t): t 0} is a cotiuous-time stochastic process with atural umbers as its state-space ad N(t) is the umber of evets that occur over iterval (0, t]. Start with N(0)=0 N(t) is o-decreasig ad right-cotiuous. Assume idepedet ad statioary icremets: The umber of evets happeig over disjoit itervals are idepedet: N t 1 N s 1 N(t 2 ) N(s 2 ) or disjoit itervals (s 1, t 1 ] ad (s 2, t 2 ]. The umber of evets over two itervals of the same legth have the same distributio: P(N(s + a) N(s) = k) = P(N(t + a) N(t) = k). Ex: For a car retal agecy, let i be the umber of scratches o retured car i. The the associated coutig process, which yields the total umber of scratches i the first m retured cars, is N m = m i. N(m) is idexed by iteger m but we ca still ask if it has idepedet ad statioary icremets. Idepedece: Scratches geerally occur whe the cars are reted out to differet drivers ad drive o differet roads at differet times Statioary: Differet cars have retal history of beig drive uder similar coditios. Assumig idepedet ad statioary icremets for N(m) is equivalet to assumig iid for { 1, 2, }. This highlights the similarity betwee stochastic processes ad radom vectors.

6 Poisso Process: A special coutig process Page 6 A coutig process N(t) is a Poisso process with rate λ if i) N(0) = 0, ii) N(t) has idepedet ad statioary icremets, iii) P N t = = exp λ t! λ t A alterative defiitio of the process is useful. g h A fuctio g is said to be order of umber h ad writte as g o(h) if lim = 0. h 0 h A o(h) fuctio drops faster tha the liear fuctio h as we get closer to 0. Alterative defiitio of Poisso process, maitai properties i) ad ii) but replace iii) with iii a) P(N(h) = 1) = λh + o(h) ad iii b) P(N(h) = 0) = 1 λh + o(h). Aother alterative defiitio of Poisso process, maitai properties i) ad ii) but replace iii) with iii ) The iterevet times are iid with Expo(λ). Expoetial iterevet times X i = S i S i 1 Expo(λ) X 1 X 2 X 3 X i N t 0 S 1 S 2 S 3 i Gamma evet times S i = j=1 X j Gamma(i, λ) S i 1 S i S S t

7 iii) P N t = = exp λ t λ t /! iii ) P(N(h) = 1) = λh + o(h) & P(N(h) = 0) = 1 λh + o(h) Page 7 iii) iii ) P N h = 0 = exp λh = 1 λh + λh 2 2! λh 3 3! P N h = 1 = exp λh λh = λh + λh 2 + λh 3 iii ) iii) Let p i t = P(N t = i) 2! = 1 λh + o(h) λh 4 3! = λh + o(h) p 0 (t + h) = P(N(t) = 0, N(t + h) N(t) = 0) = P(N(t) = 0)P(N(h) = 0) = p 0 t p 0 (h) = p 0 (t)(1 λh + o(h)), which leads to d dt p p 0 (t + h) p 0 (t) o h 0 t = lim = λp h 0 h 0 t + lim h 0 h = λp 0(t) This differetial equatio has the solutio of the form p 0 (t) = c exp( λt). Usig the iitial coditio, p 0 (0) = 1, we obtai c = 1. Hece, p 0 (t) = exp( λt). p i t + h = P N t = i, N t + h N t = 0 + P N t = i 1, N t + h N t = 1 + P N t = i 2, N t + h N t = 2 + P N t = i 3, N t + h N t = 3 + = p i t 1 λh + o h + p i 1 t λh + o h + o h, which leads to, for i 1, d dt p p i (t + h) p i (t) o h i t + λp i t = lim + λp h 0 h i t = λp i 1 t + lim h 0 h = λp i 1 (t) What remais to show: this equatio has the solutio p i (t) = exp λt λt i /i!. See the lecture otes.

8 iii) P N t = = exp λ t λ t /! iii ) The iterevet times are iid with Expo(λ) Page 8 A algebraic equality used below: Differeces of Gammas is Poisso t 1 λx λt t λx λx λx P Gamma(, λ) t = e λdx = e λt + e λdx 0 1!! 0! = P Po(λ) = + P Gamma( + 1, λ) t It ca be established by itegratio by parts; see lecture otes. iii) iii ) P S 1 > t = P N t = 0 = exp λt so S 1 Gamma(1, λ) ad X 1 Expo(λ) P S > t = P N t {0,1,, 1} = 1 i=0 P(N(t) = i) = P N t = P(N(t) = i) iii ) iii) = e λt 1 + t 0 e λx λx i 1 i 1! = e λt + 0 t e λx λdx 0 t e λx = 1 0 t e λx λx 1 λdx 0 t e λx λx 1 1! λdx λx i i! λdx 1! λdx S Gamma(, λ) ad X Expo(λ) P(N(t) = 0) = P(X 1 > 0) = e λt so N t = 0 has the Poisso probability P N t = P N t = +P(N t + 1) P(N(t) = ) = P(N t ) P(N t + 1) = P(S t) P(S +1 t) = e λt λt! so N t = has Poisso probability

9 Thiig of a Poisso Process Page 9 S 1 S 2 S 3 S i 1 S i S Poisso process With probability p 1 With probability p 2 Some evets are gree, some are blue; with probabilities p 1, p 2 ad p 1 + p 2 = 1. All evets occur accordig to Poisso process. Occurrece of gree or of blue evets accordig to thiig of Poisso process. N 1 (t) is a the umber of type 1 evets; N 2 (t) is a the umber of type 2 evets. N 1 t + N 2 t = N(t) ad so [N 1 (t) = 1 N(t) = ] ad [N 2 (t) = 2 N(t) = ] are depedet P N i t = i N t = = P(Bi, p i = i ) for i {1,2} P N 1 t = 1 N t = P N 2 t = 2 N t = = P(Bi, p 1 = 1 )P(Bi, p 2 = 2 ) P N 1 t = 1, N 2 t = 2 N t = = P Bi, p 1 = 1 = P(Bi, p 2 = 2 ) Coditioal radom variables are depedet; ucoditioal oes ca still be idepedet. Ucoditioal radom variable N 1 t Po(λp 1 t). Similarly N 2 t Po(λp 2 t). P N 1 t = 1 = =1 = =1 C 1 p 1 1 p 1 λt λt 2 e! Idepedece: N 1 t N 2 t P N 1 t = 1 N t = P N t = = =1 P(Bi, p 1 = 1 )P N t = = e λp 1t λp 1t 1 1! e λp2t 1 =1 ( 1 )! p 2 λt 1 = e λp 1t λp 1t 1 P(N 1 (t) = 1, N 2 (t) = 2 ) = P(N 1 (t) = 1, N 2 (t) = 2 N(t) = ) P(N(t) = ) = P N 1 t = 1 N t = P N t = = C p 1 1 p 2 2 e λt λt 1+ 2 ( )! = e λp 1t λp 1t 1 e λp 2t λp 2t 2 = P N 1! 2! 1 t = 1 P(N 2 (t) = 2 ) 1!

10 Number of s Util a Coupo Ex: A graduate studet is iterested i Texas style horseback ridig but does ot have much moey to sped o this. o The studet erolls i livigsocial.com's Dallas list to receive a discout offer for horseback ridig. o Livigsocial seds offers for other evets such as pilotig, karate lessos, gu carriage licese traiig, etc. o Livigsocial icludes 1 discout offer i every ad that offer ca be for ay oe of the evets listed above. What is the expected umber of livigsocial s that the studet has to receive to fid a offer for horseback ridig? Probability p 1 is for each to iclude a discout offer for horseback ridig. Let N 1 be the idex of the first icludig a horseback ridig offer. The P(N 1 = 1 ) = p 1 1 p 1 1 1, N 1 is a geometric radom variable. Page 10 Ex: Livigsocial seds offers for 5 types of evets with probabilities p 1 + p 2 + p 3 + p 4 + p 5 = 1. What is the expected umber of livigsocial s that the studet has to receive to obtai a discout offer for each evet type? Let N i be the idex of the first icludig a discout offer for evet type i. A particular realizatio ca have [N 1 = 3, N 2 = 7, N 3 = 1, N 4 = 2, N 5 = 5],» the 1 st has a offer for evet 3,» the 2 d has a offer for evet 4,» the 3 rd has a offer for evet 1, 3 4» the 4 th ad 6 th must iclude offers for evets 1, 3 ad 4. 1» the 5 th has a offer for evet 5 5 2» the 7 th has a offer for evet 2 I this realizatio, the umber of s to edure to obtai a offer for each evet is max{3, 7, 1, 2, 5} = 7. I geeral, N = max{n 1, N 2, N 3, N 4, N 5 } for depedet N i s is the umber s to obtai a offer for each evet. We wat E(N), but N i s are depedet eve if they have margial pmf s that are geometric distributios. How to break the depedece?

11 Amout of Time util a Coupo for Each Cotiuous-time Aalog Page 11 X 3 X 4 X 1 Discrete-time: Number of s to obtai coupo i X 5 X 2 Cotiuous-time: Amout of time to obtai coupo i Suppose s are set at the rate of 1 per day. Number of s N t Po(1t) with iterevet time Y i Expo(1). N i t is the umber of coupos of type i obtaied by time t. N i t Po(1p i t) N i t is obtaied from N(t) via thiig so N 1 N 2 N 5 X i is the iterevet time for N i t process. X i Expo(1p i ) X i is the time betwee two s both cotaiig coupo i Amout of time util a coupo for each evet X = max X 1, X 2, X 3, X 4, X 5 for X 1 X 2 X 5. Relatig the umber of s to the time of the s. How may s are received i X amout of time? How much time does it take to receive N s? N(t = X) or X = Y i Proceedig i either way: E N X = 1E X = E(X) or E X = E Y i E N = 1E(N), so E N = E X. To fiish, we eed E X. 5 P X t = P X i t = (1 e pit ) E X = 0 P X t dt = e p it dt For the umber of evet types approachig ifiity, E X. For a sigle evet type, E(X) = 1.

12 Distributio of Evet Times Coditioed o Number of Evets Recall from order statistics: {X 1, X 2,, X } is a iid sequece, where X i f X, ad Y i is the ith smallest elemet i the sequece. The joit pdf of Y = [Y 1, Y 2,, Y ] is, for y 1 y 2 y, f Y (y 1, y 2,, y ) =! f X (y i ) Ex: What is the joit pdf of the order statistics vector Y if each of uderlyig radom variable X i U(0, t)? The pdf of X i is f X (x) = I x (0,t] /t. The the joit pdf of the order statistics vector Y is, for 0 < y 1 < y 2 < < y < t, 1 f Y y 1, y 2,, y =! t =! t Page 12 Regardless of the rate of Poisso, the coditioal joit pdf of ([S 1, S 2,, S ] N(t) = ) is, for 0 < s 1 < < s < t, f [S1,S 2,,S ] N(t)=(s 1, s 2,, s ) =! t First ote [S 1 = s 1, S 2 = s 2,, S = s, N(t) = ] iff [X 1 = s 1, X 2 = s 2 s 1,, X = s s 1, X +1 t s ]. Hece, f S1,S 2,,S,N(t) s 1, s 2,, s, = f X s 1 f X s 2 s 1 f X s s 1 1 F X t s = λ e λs 1λe λ s 2 s 1 λe λ s s 1 e λ t s = λ e λt. The goig forward mechaically f [S1,S 2,,S ] N(t)= s 1, s 2,, s = f S 1,S 2,,S,N t s 1, s 2,, s, λ e λt = P N t = (λt) e λt /! =! t. From the last two examples, the coditioal joit pdf of ([S 1, S 2,, S ] N(t) = ) ad the pdf of order statistics [Y 1, Y 2,, Y ] with a uderlyig iid X i U i (0, t] have the same distributio. We say S i N t = = Y i (U 1 (0, t],, U (0, t]) i distributio.

13 Expected Total Waitig Time for a Bus Page 13 Sum of order statistics = Sum of sample E Y i = E Ex: A DART bus departs from UTD to grocery markets every T miutes. o Passegers arrive accordig to Poisso process N(t) with rate λ ad wait for the departure. o E.g.,, the ith passeger that arrives over (0, t] arrives at S i ad waits for T S i. o The expected total waitig time is = E U i = T 2 U i N T N T N T E T S i = E E T S i N T = = E =0 = E T Y i U 1 0, T,, U 0, T P N T = =0 = E T Y i (U 1 (0, T],, U (0, T]) P(N T = ) =0 = T E Y i U 1 0, T,, U 0, T P(N T = ) = =0 =0 T T 2 P N T = = 1 2 TE N T = 1 2 TλT T S i N T = (λt) e λt! Arrival time S i N(t) = Y i (U 1 (0, T],, U (0, T]) Waitig time λt Area of the triagle is the expected total waitig time T

14 Geeralizatios Page 14 No-statioary Statioary Icremets: Nohomogeeous Poisso process. Let λ(t) be time depedet evet rate t The expected umber of evets over (0, t] is 0 λ x dx The umber of evets durig (0, t] has Po 0 t λ x dx Compoud Poisso process: A Poisso process N(t) ad a iid sequece {Y i } ca be combied to obtai the compoud Poisso process: X(t) = These geeralizatios are useful to model real-life processes Higher traffic itesity durig early eveig hours Batch arrivals of customers to a store (or buyig multiple uits i a sigle arrival) N t Y i

15 Summary Page 15 Cotiuous-time Markov Process Poisso Process Thiig Coditioig o the Number of Evets Geeralizatios

Limit Theorems. Convergence in Probability. Let X be the number of heads observed in n tosses. Then, E[X] = np and Var[X] = np(1-p).

Limit Theorems. Convergence in Probability. Let X be the number of heads observed in n tosses. Then, E[X] = np and Var[X] = np(1-p). Limit Theorems Covergece i Probability Let X be the umber of heads observed i tosses. The, E[X] = p ad Var[X] = p(-p). L O This P x p NM QP P x p should be close to uity for large if our ituitio is correct.