Markov Processes. Fundamentals (1) State graph. Fundamentals (2) IEA Automation 2018

Transcription

1 Fundaental () Markov Procee Dr Ulf Jeppon Div of Indutrial Electrical Engineering and Autoation (IEA) Dept of Bioedical Engineering (BME) Faculty of Engineering (LTH), Lund Univerity Tranition in dicrete tie > Markov chain When tranition are tochatic event at arbitrary point in tie > Markov proce Continuou tie decription Fundaental (2) State graph Conider the tie interval Dt Probability for tate tranition: a Dt Paraeter a = intenity (e.g. production rate, repair rate) dienion /tie f r f r f r f 00 0 µ µ r 0 Markov procee

2 State tranition () P[ i (t)] = probability to be in tate i at tie t Probability for tranition fro i to during the tie Dt i a i Dt State tranition (2) P[ (t + Δt )] = P[ (t)] (prob no tranfer fro ) + +P[ (t )] a Δt + P[ 2 (t )] a 2 Δt + + +P[ (t)] a, Δt + P[ + (t )] a +, Δt + + +P[ (t )] a Δt (auing a total of tate) Probability of no tranfer Can be expreed a: (prob to leave the tate ) = = a Δt i i State tranition (3) P[ (t + Δt)] = P[ (t)] a i i Δt + + P[ i (t)]a i Δt ( =, 2,...) i + a t Markov procee 2

3 State tranition (4) P[ (t + Δt)] = ( ) + = P[ (t)] + a Δt + P[ i (t)]a i Δt i State tranition (5) P[ (t + Δt)] P[ (t)] = = P[ i (t)]a i Δt i= d dt P[ (t)] = i= P[ i (t)]a i Probability vector ( ) = ( ) p(t) = p,p 2,!,p = P[ (t)] P[ 2 (t)] P[ (t)] Note: a row vector! Su of p(t) =! State tranition (6) d p(t) = p(t) A dt Solution: p(t) = p(0) e At A i the generator or tranition atrix Note: Row u of A = 0, one eigenvalue alway = 0 Markov procee 3

4 Stationary olution 0 = p(t) A = p A ut ue the extra condition: u(p) = Interpretation: flow of event = rate probability In tationarity: the flow of event into i i equal to the flow of event out of i Siple exaple r 2 f broken Generator A = r f operating r f Siulation 2 tate yte Markov procee - Suary P[ i (t)] = prob. to be in the tate i at tie t p i a row vector a i the tranfer rate (a peed ) The generator A Su of p =, row u of A = 0 Stationarity: p(t)*a = (t -> inf) = p tat *A = 0 General: d p(t) = p(t) A dt Markov procee 4

5 The Poion proce Special cae of the Markov proce The prob. for an event during the tie interval Dt i l Dt Only event at a tie No eory: the nuber of event within [t,t 2 ] independent of the nuber of event within [t 3,t 4 ] (no overlap) Tie interval between event are independent and exponentially ditributed Average waiting tie / l A birth proce Decribe the arrival pattern Arrival rate l l l l 0 2 d p t) dt ( =,2,..., k) ( = p ( t) p ( t) Siple tranfer line a birth proce 0 - k- k k d p t) dt ( =,2,..., k) ( = p ( t) p ( t) The tranfer line generator ! ! 0 0 0!!!!!!!! A = ! k k ! 0 k k ! Markov procee 5

6 Tranfer line olution Tranfer line proce Input + 4 achine + output à 6 tate p 0 (t) = e 0 t p (t) = t e t Special cae l = l p (t) = (t)! e t k, t 0 Work in progre Generalied, birth procee can be ued to odel arrival into a yte. Auption infinite et of ob. p n (t) i the probability that exactly n ob have arrived in the interval [0,t]. At leat one ob arrived: p 0 (t) More than one ob: p 0 (t) p (t) etc. Exaple (Quetion 6.6) The arrival of article to a proce can be odelled a a Poion proce. Aue arrival rate i l = 2 in -. Calculate the probability that there will be ore than 3 ob in the yte after inute. Average nuber of ob [0,t] = work in progre = WIP= L = n= 0 n p n (t) =... = t Markov procee 6

7 Superpoition Decopoition 2 n Σ aebly = n Poion arrival plit up p p 2 p n p p 2 p n M M 2 M n Birth-death proce Birth-death generator k- k- k k k+ µ µ k µ k+ p k+ = k µ k+ p k k = 0,,2, !! µ ( + µ ) 0!! 0 ( 2 + ) 2 0!! A = 0 0!!!!!! µ k ( k + µ k ) k 0!!!! 0 µ k + ( k + + µ k + ) k + 0!!!!!!!! Foundation for queuing theory (next coure). µ repreent departure rate (or production rate). Markov procee 7

8 Two achine and one buffer State graph 00 0 achine M buffer B achine M f r f r f r f r (n,a,a 2 ) 00 0 µ µ 0 Probability vector Iolated achine efficiency p(t) = [ p 000, p 00,...,p N ] Iolated achine efficiency: Iolated achine production: r i e i = r i + f i ρ i = µ i e i Markov procee 8

9 Syte efficiencie Total yte production Machine : Machine 2: N E = p(n,,α 2 ) n=0 N α 2 =0 E 2 = p(n,α,) n= α =0 P = µ E = E 2 Average buffer ize n = N n= 0 n p(n,α,α 2 ) α = 0 α 2 = 0 Markov procee 9