Rcall: continuous-tim Makov chains Modling and Vification of Pobabilistic Systms Joost-Pit Katon Lhstuhl fü Infomatik 2 Softwa Modling and Vification Goup http://movs.wth-aachn.d/taching/ws-89/movp8/ Dcmb 4, 208 Joost-Pit Katon Modling and Vification of Pobabilistic Systms /2 Joost-Pit Katon Modling and Vification of Pobabilistic Systms 2/2 Ngativ xponntial distibution Dnsity of xponntial distibution Rcall: continuous-tim Makov chains Th dnsity of an xponntially distibutd.v. Y with at λ R >0 is: f Y (x) = λ λ x fo x > 0 and f Y (x) = 0 othwis Th cumulativ distibution of.v. Y with at λ R >0 is: F Y (d) = d 0 λ λ x dx = [ λ x d 0 = λ d. Th at λ R >0 uniquly dtmins an xponntial distibution. Vaianc and xpctation Lt.v. Y b xponntially distibutd with at λ R >0. Thn: Expctation E[Y = 0 x λ λ x dx = λ Vaianc Va[Y = 0 (x E[X)2 λ λ x dx = λ 2 Joost-Pit Katon Modling and Vification of Pobabilistic Systms /2 Continuous-tim Makov chain Continuous-tim Makov chain A CTMC is a tupl (S, P,, ι init, AP, L) wh (S, P, ι init, AP, L) is a DTMC, and : S R >0, th xit-at function Rcall: continuous-tim Makov chains Lt R(s, s ) = P(s, s ) (s) b th tansition at of tansition (s, s ) Intptation sidnc tim in stat s is xponntially distibutd with at (s). phasd altnativly, th avag sidnc tim of stat s is (s). Joost-Pit Katon Modling and Vification of Pobabilistic Systms 4/2
Rcall: continuous-tim Makov chains Tansint distibution CTMC smantics Enabldnss Th pobability that tansition s s is nabld in [0, t is R(s,s ) t. Stat-to-stat timd tansition pobability Th pobability to mov fom non-absobing s to s in [0, t is: R(s, s ) (s) Rsidnc tim distibution ( (s) t). Th pobability to tak som outgoing tansition fom s in [0, t is: t 0 (s) (s) x dx = (s) t Joost-Pit Katon Modling and Vification of Pobabilistic Systms 5/2 Joost-Pit Katon Modling and Vification of Pobabilistic Systms 6/2 Tansint distibution Tansint distibution of a CTMC Tansint stat pobability Lt X(t) dnot th stat of a CTMC at tim t R 0. Th pobability to b in stat s at tim t is dfind by: p s (t) = P{ X(t) = s } = s S P{ X(0) = s } P{ X(t) = s X(0) = s } Tansint distibution thom Tansint distibution Thom: tansint distibution as lina diffntial quation Th tansint pobability vcto p(t) = (p s (t),..., p sk (t)) satisfis: p (t) = p(t) (R ) givn p(0) wh is th diagonal matix of vcto. Thom: tansint distibution as lina diffntial quation Th tansint pobability vcto p(t) = (p s (t),..., p sk (t)) satisfis: p (t) = p(t) (R ) givn p(0) Poof: On th blackboad. wh is th diagonal matix of vcto. Joost-Pit Katon Modling and Vification of Pobabilistic Systms 7/2 Joost-Pit Katon Modling and Vification of Pobabilistic Systms 8/2
Tansint distibution Th tansint pobability vcto p(t) = (p s (t),..., p sk (t)) satisfis: p (t) = p(t) (R ) givn p(0). Solution using standad knowldg yilds: p(t) = p(0) (R ) t. Computing a matix xponntial Fist attmpt: us Taylo-Maclauin xpansion. This yilds p(t) = p(0) (R ) t ((R ) t) i = p(0) But: numical instability du to fill-in of (R ) i in psnc of positiv and ngativ ntis in th matix R. Joost-Pit Katon Modling and Vification of Pobabilistic Systms 9/2 Joost-Pit Katon Modling and Vification of Pobabilistic Systms 0/2 Lt CTMC C = (S, P,, ι init, AP, L) with S finit. Unifom CTMC CTMC C is unifom if (s) = fo all s S fo som R >0. [Goss and Mill, 984 Lt R >0 such that max s S (s). Thn unif(, C) is th tupl (S, P,, ι init, AP, L) with (s) = fo all s S, and: : xampl Lt R >0 such that max s S (s). Thn unif(, C) = (S, P,, ι init, AP, L) with (s) = fo all s S, and: P(s, s ) = (s) P(s, s ) if s s and P(s, s) = (s) P(s, s) + (s). P(s, s ) = (s) P(s, s ) if s s and P(s, s) = (s) P(s, s) + (s). It follows that P is a stochastic matix and unif(, C) is a CTMC. CTMC C and its unifomizd countpat unif(6, C) Joost-Pit Katon Modling and Vification of Pobabilistic Systms /2 Joost-Pit Katon Modling and Vification of Pobabilistic Systms 2/2
: intuition Lt R >0 such that max s S (s). Thn unif(, C) = (S, P,, ι init, AP, L) with (s) = fo all s S, and: P(s, s ) = (s) Intuition P(s, s ) if s s and P(s, s) = (s) P(s, s) + (s). Fix all xit ats to (at last) th maximal xit at occuing in CTMC C. Thus, is th shotst man sidnc tim in th CTMC C. Thn nomaliz th sidnc tim of all stats with spct to as follows:. plac an avag sidnc tim (s) by a shot (o qual) on, 2. dcas th tansition pobabilitis by a facto (s), and. incas th slf-loop pobability by a facto (s) That is, slow down stat s whnv (s) <. Joost-Pit Katon Modling and Vification of Pobabilistic Systms /2 Stong and wak bisimulation Joost-Pit Katon Modling and Vification of Pobabilistic Systms 4/2 Stong bisimulation on DTMCs Stong and wak bisimulation Pobabilistic bisimulation [Lasn & Skou, 989 Lt D = (S, P, ι init, AP, L) b a DTMC and R S S an quivalnc. Thn: R is a pobabilistic bisimulation on S if fo any (s, t) R:. L(s) = L(t), and 2. P(s, C) = P(t, C) fo all quivalnc classs C S/R wh P(s, C) = s C P(s, s ). Stong bisimulation on CTMCs Stong and wak bisimulation Pobabilistic bisimulation [Buchholz, 994 Lt C = (S, P,, ι init, AP, L) b a CTMC and R S S an quivalnc. Thn: R is a pobabilistic bisimulation on S if fo any (s, t) R:. L(s) = L(t), and 2. (s) = (t), and. P(s, C) = P(t, C) fo all quivalnc classs C S/R Fo stats in R, th pobability of moving by a singl tansition to som quivalnc class is qual. Pobabilistic bisimilaity Lt D b a DTMC and s, t stats in D. Thn: s is pobabilistically bisimila to t, dnotd s p t, if th xists a pobabilistic bisimulation R with (s, t) R. Th last two conditions amount to R(s, C) = R(t, C) fo all quivalnc classs C S/R. Pobabilistic bisimilaity Lt C b a CTMC and s, t stats in C. Thn: s is pobabilistically bisimila to t, dnotd s m t, if th xists a pobabilistic bisimulation R with (s, t) R. Joost-Pit Katon Modling and Vification of Pobabilistic Systms 5/2 Joost-Pit Katon Modling and Vification of Pobabilistic Systms 6/2
Wak bisimulation on DTMCs Stong and wak bisimulation Wak pobabilistic bisimulation [Bai & Hmanns, 996 Lt D = (S, P, ι init, AP, L) b a DTMC and R S S an quivalnc. Thn: R is a wak pobabilistic bisimulation on S if fo any (s, t) R:. L(s) = L(t), and 2. if P(s, [s R ) < and P(t, [t R ) <, thn: Wak bisimulation on DTMCs Stong and wak bisimulation Wak pobabilistic bisimulation [Bai & Hmanns, 996 Lt D = (S, P, ι init, AP, L) b a DTMC and R S S an quivalnc. Thn: R is a wak pobabilistic bisimulation on S if fo any (s, t) R:. L(s) = L(t), and 2. if P(s, [s R ) < and P(t, [t R ) <, thn: P(s, C) P(s, [s R ) = P(t, C) P(t, [t R ) fo all C S/R, C [s R = [t R. P(s, C) P(s, [s R ) = P(t, C) P(t, [t R ) fo all C S/R, C [s R = [t R.. s can ach a stat outsid [s R iff t can ach a stat outsid [t R.. s can ach a stat outsid [s R iff t can ach a stat outsid [t R. Pobabilistic wak bisimilaity Fo stats in R, th conditional pobability of moving by a singl tansition to anoth quivalnc class is qual. In addition, ith all stats in an quivalnc class C almost suly stay th, o hav an option to scap fom C. Joost-Pit Katon Modling and Vification of Pobabilistic Systms 7/2 Lt D b a DTMC and s, t stats in D. Thn: s is pobabilistically wak bisimila to t, dnotd s p t, if th xists a pobabilistic wak bisimulation R with (s, t) R. Joost-Pit Katon Modling and Vification of Pobabilistic Systms 8/2 Stong and wak bisimulation Wak bisimulation on DTMC: xampl Rachability condition Stong and wak bisimulation Rmak Consid th following DTMC: Th quivalnc lation R with S/R = { {s, s 2, s, s 4 }, {u, u 2, u } } is a wak bisimulation. This can b sn as follows. Fo C = { u, u 2, u } and s, s 2, s 4 with P(s i, [s i R ) < w hav: P(s, C) P(s, [s ) = /8 5/8 = /4 /4 = P(s 2, C) P(s 2, [s 2 ) = / = P(s 4, C) P(s 4, [s 4 ). It is not difficult to stablish s p s 2. Not: P(s, [s R ) =, but P(s 2, [s 2 R ) <. Both s and s 2 can ach a stat outsid [s R = [s 2 R. Th achability condition is ssntial to stablish s p s 2 and cannot b doppd: othwis s and s 2 would b wakly bisimila to an qually lablld absobing stat. Not that P(s, [s R ) =. Sinc s can ach a stat outsid [s as s, s 2 and s 4, it follows that s p s 2 p s p s 4. Joost-Pit Katon Modling and Vification of Pobabilistic Systms 9/2 Joost-Pit Katon Modling and Vification of Pobabilistic Systms 20/2
Stong and wak bisimulation Stong and wak bisimulation Wak bisimulation on CTMCs A usful lmma Wak pobabilistic bisimulation [Bavtti, 2002 Lt C = (S, P,, ι init, AP, L) b a CTMC and R S S an quivalnc. Thn: R is a wak pobabilistic bisimulation on S if fo any (s, t) R:. L(s) = L(t), and 2. R(s, C) = R(t, C) fo all C S/R with C [s R = [t R Wak pobabilistic bisimilaity Lt C b a CTMC and s, t stats in C. Thn: s is wak pobabilistically bisimila to t, dnotd s m t, if th xists a wak pobabilistic bisimulation R with (s, t) R. Lt C b a CTMC and R an quivalnc lation on S with (s, t) R, P(s, [s R ) < and P(t, [t R ) <. Thn: th following two statmnts a quivalnt:. fo all C S/R, C [s R = [t R : P(s, C) P(s, [s R ) = P(t, C) P(t, [t R ) and R(s, S \ [s R ) = R(t, S \ [t R ) 2. R(s, C) = R(t, C) fo all C S/R with C [s R = [t R. Poof: Lft as an xcis. Joost-Pit Katon Modling and Vification of Pobabilistic Systms 2/2 Joost-Pit Katon Modling and Vification of Pobabilistic Systms 22/2 Stong and wak bisimulation Wak bisimulation on CTMCs: xampl Poptis (without poof) Stong and wak bisimulation Stong and wak bisimulation in unifom CTMCs Fo all unifom CTMCs C and stats s, u in C, w hav: s m u iff s m u iff s p u. Fo any CTMC C, w hav: C m unif(, C) with max s S (s). Psvation of tansint pobabilitis Equivalnc lation R with S/R = { {s, s 2, s, s 4, s 5, s 6 }, {u, u 2, u, u 4, u 5 } } is a wak bisimulation on th CTMC dpictd abov. This can b sn as follows. Fo C = { u, u 2, u, u 4, u 5 }, w hav that all s-stats nt C with at 2. Th ats btwn th s-stats a not lvant. Joost-Pit Katon Modling and Vification of Pobabilistic Systms 2/2 Fo all CTMCs C with stats s, u in C and t R 0, w hav: s m u implis p s (t) = p u (t) wh p s (0) = s and p u (0) = u wh s is th chaactistic function fo stat s, i.., s (s ) = iff s = s. Joost-Pit Katon Modling and Vification of Pobabilistic Systms 24/2
Th tansint pobability vcto p(t) = (p s (t),..., p sk (t)) satisfis: p (t) = p(t) (R ) givn p(0). Standad knowldg yilds: p(t) = p(0) (R ) t. As unifomization psvs tansint pobabilitis, w plac R by its vaiant fo th unifomizd CTMC, i.., R. W hav: Thus: R(s, s ) = P(s, s ) (s) = P(s, s ) and = I. p(0) (R ) t = p(0) (P I ) t = p(0) (P I) t = p(0) t t P. Joost-Pit Katon Modling and Vification of Pobabilistic Systms 25/2 Joost-Pit Katon Modling and Vification of Pobabilistic Systms 26/2 p(t) = p(0) (R ) t = p(0) (P I ) t = p(0) (P I) t = p(0) t t P. Computing a matix xponntial Exploit Taylo-Maclauin xpansion. This yilds: p(0) t t P = p(0) t ( t) i P i t ( t)i = p(0) }{{} P i Poisson pob. As P is a stochastic matix, computing th matix xponntial P i is numically stabl. Intmzzo: Poisson distibution Poisson distibution Th Poisson distibution is a disct pobability distibution that xpsss th pobability of a givn numb i of vnts occuing in a fixd intval of tim [0, t if ths vnts occu with a known avag at and indpndntly of th tim sinc th last vnt. Fomally, th pdf is: t ( t)i f (i; t) = wh is th man of th Poisson distibution. Rmak Th Poisson distibution can b divd as a limiting cas to th binomial distibution as th numb of tials gos to infinity and th xpctd numb of succsss mains fixd. Joost-Pit Katon Modling and Vification of Pobabilistic Systms 27/2 Joost-Pit Katon Modling and Vification of Pobabilistic Systms 28/2
Tansint pobabilitis: xampl P = [ 0 0, = [ 2 and P = [ 0 Lt initial distibution p(0) = (, 0), and tim bound t=. Thn: p() = p(0) i = (, 0) 0! Pi [ 0 0 2 [ 0 + (, 0)! [ 2 0 + (, 0) 2! 9 +...... (0.40404, 0.595957) 2 2 Tuncating th infinit sum t ( t)i p(t) = p(0) P i Summation can b tuncatd a pioi fo a givn o bound ε > 0. Th o that is intoducd by tuncating at summand k ε is: t (t)i k ε t (t)i p(i) p(i) = t (t)i p(i) Statgy: choos k ε minimal such that: i=k ε+ t (t)i = t (t)i k ε t (t)i i=k ε + = k ε t (t)i (t)i t (t)i t = du to th fact that is a (Poisson) distibution ε Joost-Pit Katon Modling and Vification of Pobabilistic Systms 29/2 Joost-Pit Katon Modling and Vification of Pobabilistic Systms 0/2 Summay Summay Summay Main points Bisimila stats a qually lablld and thi cumulativ at to any quivalnc class coincids. Wak bisimila stats hav qual conditional pobabilitis to mov to som quivalnc class, and can ith both lav thi class o both can t. nomalizs th xit ats of all stats in a CTMC. tansfoms a CTMC into a wak bisimila on. Tansint distibution a obtaind by solving a systm of lina diffntial quations. Ths quations can b solvd convnintly on th unifomizd CTMC. Joost-Pit Katon Modling and Vification of Pobabilistic Systms /2 Joost-Pit Katon Modling and Vification of Pobabilistic Systms 2/2