Generalized Semi- Markov Processes (GSMP)

Transcription

1 Geeralized Semi- Markov Processes (GSMP) Summary Some Defiitios Markov ad Semi-Markov Processes The Poisso Process Properties of the Poisso Process Iterarrival times Memoryless property ad the residual lifetime paradox Superpositio of Poisso processes

2 Radom Process Let (Ω, F, P) be a probability space. A stochastic (or radom) process {X(t)} is a collectio of radom variables defied o (Ω, F, P), idexed by t T (where t is usually time). X(t) is the state of the process. Cotiuous Time ad Discrete Time stochastic processes If the set T is fiite or coutable the {X(t)} is called discrete-time process. I this case t {0,,2, } ad we may referred to a stochastic sequece. We may also use the otatio {X k }, k=0,,2, Otherwise, the process is called cotiuous-time process Cotiuous State ad Discrete State stochastic processes If {X(t)} is defied over a coutable set, the the process is discrete-state, also referred to as chai. Otherwise, the process is cotiuous-state. Classificatio of Radom Processes Joit cdf of the radom variables X(t 0 ),,X(t ) (,..., ;,..., ) Pr ( ),..., ( ) Idepedet Process F x x t t = X t x X t x X Let X,,X be a sequece of idepedet radom variables, the (,..., ;,..., ) = ( ; )... ( ; ) F x x t t F x t F x t X 0 0 X 0 0 X Statioary Process (strict sese statioarity) The sequece {X } is statioary if ad oly if for ay τ R 0 (,..., ; + τ,..., + τ) = (,..., ;,..., ) F x x t t F x x t t X 0 0 X 0 0

3 Classificatio of Radom Processes Wide-sese Statioarity Let C be a costat ad g(τ) a fuctio of τ but ot of t, the a process is wide-sese statioary if ad oly if Ε { X () t } = C Markov Process ad The future of a process does ot deped o its past, oly o its preset { X ( tk+ ) xk+ X ( tk) = xk X ( t0) = x0} Pr ( ) ( ) Pr,..., Ε{ X () t X ( t+ τ )} = g( τ ) { X tk+ xk+ X tk xk} = = Also referred to as the Markov property Markov ad Semi-Markov Property The Markov Property requires that the process has o memory of the past. This memoryless property has two aspects: All past state iformatio is irrelevat i determiig the future (o state memory). How log the process has bee i the curret state is also irrelevat (o state age memory). The later implies that the lifetimes betwee subsequet evets (iterevet time) should also have the memoryless property (i.e., expoetially distributed). Semi-Markov Processes For this class of processes the requiremet that the state age is irrelevat is relaxed, therefore, the iterevet time is o loger required to be expoetially distributed.

4 Example Cosider the process X = X X k+ k k with Pr{X 0 =0}= Pr{X 0 =}= 0.5 ad Pr{X = 0}= Pr{X =}= 0.5 Is this a Markov process? NO Is it possible to make the process Markov? Defie k = X k- ad form the vector Z k = [X k, k ] T the we ca write Z X X Z 0 0 k+ k k+ = = = k+ k k Reewal Process A reewal process is a chai {N(t)} with state space {0,,2, } whose purpose is to cout state trasitios. The time itervals betwee state trasitios are assumed iid from a arbitrary distributio. Therefore, for ay 0 t t k ( ) ( t ) ( t ) ( ) N = 0 N N... N t k

5 Geeralized Semi-Markov Processes (GSMP) A GSMP is a stochastic process {X(t)} with state space X geerated by a stochastic timed automato ( X, E, Γ, f, p, G) X is the coutable state space E is the coutable evet set Γ(x) is the feasible evet set at state x. f(x, e): is state trasitio fuctio. p 0 is the probability mass fuctio of the iitial state G is a vector with the cdfs of all evets. The semi-markov property of GSMP is due to the fact that at the state trasitio istat, the ext state is determied by the curret state ad the evet that just occurred. 0 The Poisso Coutig Process Let the process {N(t)} which couts the umber of evets that have occurred i the iterval [0,t). For ay 0 t N = 0 N t N t... N t... t k ( ) ( ) ( ) ( ) 0 2 k t t 2 t 3 t k- t k (, ) = ( ) ( ) N t t N t N t k k k k Process with idepedet icremets: The radom variables N(t ), N(t,t 2 ),, N(t k-,t k ), are mutually idepedet. Process with statioary idepedet icremets: The radom variable N(t k-, t k ), does ot deped o t k-, t k but oly o t k -t k-

6 The Poisso Coutig Process Assumptios: At most oe evet ca occur at ay time istat (o two or more evets ca occur at the same time) A process with statioary idepedet icremets { ( k k) } ( k k ) Pr N t, t = = Pr N t t = Give that a process satisfies the above assumptios, fid P () t Pr N() t =, = 0,,2,... The Poisso Process Step : Determie P0 () t Pr{ N() t = 0} Startig from Pr{ N( t+ s ) = 0} = Pr N() t = 0 ad N( tt, + s) = 0 = Pr N() t = 0 Pr N s = 0 Statioary idepedet icremets { ( ) } P ( t+ s) = P ( t) P ( s) Lemma: Let g(t) be a differetiable fuctio for all t 0 such that g(0)= ad g(t) for all t >0. The for ay t, s 0 λt gt ( + s) = g() t g( s) g() t = e for some λ>0

7 The Poisso Process Therefore P0 () t Pr N() t = 0 = e λt Step 2: Determie P 0 (Δt) for a small Δt. 2 3 λδt ( λδt) ( λδt) Pr{ N( Δt) = 0} = e = λδ t ! 3! = λδ t+ o( Δt). Step 3: Determie P (Δt) for a small Δt. For =2,3, sice by assumptio o two evets ca occur at the same time As a result, for = P ( Δt) Pr{ N( Δt) = } = o( Δt) ( ) P ( Δt) Pr N( Δt) = = λδ t+ o Δt The Poisso Process Step 4: Determie P (t+δt) for ay P( t t) Pr{ N( t t) = } = P +Δ +Δ k () t Pk( Δt) k = 0 () t 0( Δt) () t ( Δt) ( Δt). λ t o( Δt) λ t o Δt. = P P + P P + o [ Δ + ] P () t [ Δ + ( )] P () t o( Δt) = + + Movig terms betwee sides, P ( t+δt) P ( t) ( ) () () o Δt = λp t + λp t + Δt Δt Takig the limit as Δt 0 dp () t = λp() t + λp () t dt.

8 The Poisso Process Step 4: Determie P (t+δt) for ay P( t t) Pr{ N( t t) = } = P +Δ +Δ k () t Pk( Δt) k = 0 () t 0( Δt) ( t) ( Δt) ( Δt). λ t o( Δt) λ t o Δt. = P P + P P + o [ Δ + ] P () t [ Δ + ( )] P () t o( Δt) = + + Movig terms betwee sides, P ( t+δt) P ( t) ( ) () () o Δt = λp t + λp t + Δt Δt Takig the limit as Δt 0. dp ( t) dt = λp() t + λp () t The Poisso Process Step 5: Solve the differetial equatio to obtai ( λt) λt P () t Pr { N() t = } = e, t 0, = 0,, 2,...! This expressio is kow as the Poisso distributio ad it full characterizes the stochastic process {N(t)} i [0,t) uder the assumptios that No two evets ca occur at exactly the same time, ad Idepedet statioary icremets ou should verify that E[ N () t ] = λt ad var [ Nt ()] = λt Parameter λ has the iterpretatio of the rate that evets arrive.

9 Properties of the Poisso Process: Iterevet Times Let t k- be the time whe the k- evet has occurred ad let V k deote the (radom variable) iterevet time betwee the kth ad k- evets. What is the cdf of V k, G k (t)? () t = Pr{ } = Pr{ > } = Pr{ 0 arrivals i the iterval [ tk, tk + t) } = Pr{ N () t = 0} G V t V t k k k = e λt Statioary idepedet icremets V k G() t = e λt Expoetial Distributio t k- N(t k-,t k- +t)=0 t k- + t Properties of the Poisso Process: Expoetial Iterevet Times The process {V k } k=,2,, that correspods to the iterevet times of a Poisso process is a iid stochastic sequece with cdf () t G = Pr V t = e The correspodig pdf is k t g() t = λe λ, t 0 Oe ca easily show that EV [ k ] ad λ var V λt = [ k ] 2 Therefore, the Poisso is also a reewal process = λ

10 Properties of the Poisso Process: Memoryless Property Let t k be the time whe previous evet has occurred ad let V deote the time util the ext evet. Assumig that we have bee at the curret state for z time uits, let be the remaiig time util the ext evet. V What is the cdf of? t k t k +z =V-z () t = Pr{ } = Pr { < > } Pr { V > z ad V < z+ t} = = Pr{ V > z} F t V z t V z G( t+ z) G( z) = G ( z) λt F () t = + e = G() t Pr Pr λ( t+ z) λz e + e = λz + e { z < V < z+ t} { V < z} Memoryless! It does ot matter that we have already spet z time uits at the curret state. Memoryless Property This is a uique property of the expoetial distributio. If a process has the memoryless property, the it must be expoetial, i.e., Pr V z+ t V > z = Pr V t Pr V t = e λt Poisso Process λ Expoetial Iterevet Times G(t)=-e -λt Memoryless Property

11 Superpositio of Poisso Processes Cosider a DES with m evets each modeled as a Poisso Process with rate λ i, i=,,m. What is the resultig process? Suppose at time t k we observe evet. Let be the time util the ext evet. Its cdf is G (t)=-exp{-λ t}. Let 2,, m deote the residual time util the ext occurrece of the correspodig evet. V j Their cdfs are: e j e λ V = Memoryless Property Gi i () t = e t Let * be the time util the ext evet (ay type). Therefore, we eed to fid * * = mi * () t = Pr{ } G t i t k j =V j -z j Superpositio of Poisso Processes Idepedece * * () t = Pr{ } = Pr{ mi{ i } t} = Pr{ mi{ i } > t} G t G { t t} = > > Pr,..., m m = Pr > i= () t * { t} = e i Λt = m t e λ i= i where Λ= λi The superpositio of m Poisso processes is also a Poisso process with rate equal to the sum of the rates of the idividual processes m i=

12 Superpositio of Poisso Processes Suppose that at time t k a evet has occurred. What is the probability that the ext evet to occur is evet j? Without loss of geerality, let j= ad defie m =mi{ i : i=2,,m}. Pr{ ext evet is } j = = y λ = Λ Pr = λy Λ y = λ e Λ e dy dy ( ) y y e λ Λ = Λ e dy ~-exp λit = exp Λ t i= 2 m where Λ= λi i= Residual Lifetime Paradox Suppose that buses pass by the bus statio accordig to a Poisso process with rate λ. A passeger arrives at the bus statio at some radom poit. How log does the passeger has to wait? Solutio : V b k p b k+ E[V]= /λ. Therefore, sice the passeger will (o average) arrive i the middle of the iterval, he has to wait for E[]=E[V]/2= /(2λ). But usig the memoryless property, the time util the ext bus is expoetially distributed with rate λ, therefore E[]=/λ ot /(2λ)! Solutio 2: Usig the memoryless property, the time util the ext bus is expoetially distributed with rate λ, therefore E[]=/λ. But ote that E[Z]= /λ therefore E[V]= E[Z]+E[]= 2/λ ot /λ! Z