Ative Dignosis Serge Hddd LSV, ENS Chn & CNRS & Inri, Frne Veos 16 Otoer the 6th 2016 joint work with Nthlie Bertrnd 2, Eri Fre 2, Sten Hr 1,2, Loï Hélouët 2, Trek Melliti 1, Sten Shwoon 1 (1) FSTTCS 2013 nd (2) FOSSACS 2014 1/39
Dignosis: rom ilures to ults Exmple: MYCIN, n expert system, tht used rtiiil intelligene to identiy teri using severe inetions (1975). 2/39
Dignosis: deteting ults Fult detetion: suield o ontrol engineering whih onerns itsel with monitoring system, identiying when ult hs ourred, nd pinpointing the type o ult nd its lotion. 3/39
Dignosis: prediting ults Enhning retivity: (see Foundtion o Dignosis nd Preditility in Proilisti Systems N. Bertrnd, H., E. Leuheux, FSTTCS14) 4/39
Ative dignosis: oring detetion Comining ontrol nd dignosis 5/39
Outline 1 Amiguity in Lelled Trnsition System (LTS) Ative dignosis in LTS From LTS to proilisti LTS Anlysis o tive dignosis in LTS 6/39
Oserving Lelled Trnsition System Sttes re unoservle. Events re either oservle or unoservle. Fults () re unoservle. q 1 q 2 q 0 d u q 3 q 4 q 5 An exeution sequene yields n oserved sequene. Let σ = q 0 uq 3 q 4 q 0 q 1 (q 2 ) ω. Then P(σ) = ω. We only onsider live nd onvergent systems: There is t lest n event rom ny stte. There is no ininite sequene o unoservle events rom ny rehle stte. 7/39
Clssiition o oserved sequenes An exeution sequene is ulty i it ontins ult otherwise it is orret. An oserved sequene σ is surely ulty i or ll σ P 1 (σ), σ is ulty. An oserved sequene σ is surely orret i or ll σ P 1 (σ), σ is orret. An oserved sequene σ is miguous i it is neither surely ulty nor surely orret. q 1 q 2 q 0 u d q 3 q 4 q 5 d ω is surely ulty: the ourrene o d implies the ourrene o. ω is surely orret: P 1 () = {q 0 uq 3 q 4 q 5 q 5 }. ω is miguous: P 1 ( ω ) = {q 0 uq 3 (q 4 ) ω, q 0 q 1 (q 2 ) ω }. 8/39
How to determine unmiguous sequenes? Build Bühi utomton s synhronized produt o the LTS with ult memory nd the LTS without ults. q 1 q 2 (q 4,q 4) (q 5,q 5) q 0 d (q 0,q 0) (q 0,q 5) u q 3 q 4 q 5 (q 2,q 4) (q 5,q 0) Determinize nd omplement it s: Street utomton with 2 O(n 2 log(n)) sttes where n is the numer o sttes o the LTS. Bühi utomton with 3 2n 2 sttes using the rekpoint onstrution o Miyno nd Hyshi pproprite or the initil Bühi utomton. 9/39
An optiml hrteriztion Build deterministi Bühi utomton whose sttes re triples (U, V, W ) with: U the set o possile sttes rehed y orret sequene; W the set o possile sttes rehed y n erliest ulty sequene; V the set o other possile sttes rehed y ulty sequenes. The epting sttes re (U, V, W ) with: U =, i.e. the oserved sequene is (nd will remin) surely ulty; W =, i.e. the erliest ulty sequenes re disrded. q 0 u q 1 q 2 d q 3 q 4 q 5 ({q 0 },, ) ({q 4 },,{q 2 }) d (,,{q 4 }) (,,{q 2,q 4 }) d ({q 0,q 5 },, ) (,,{q 0,q 5 }) ({q 5 },, ) (,,{q 5 }) The numer o sttes is t most 7 n. 10/39
A lower ound or miguity...,, l 0 l 1 l 2 l n 1 l n l n+1 d...,,,,d q 0 q 1 q 2 q n 1 q n q n+1,... r 0 r 1 r 2 r n 1 r n r n+1,, d Amiguous sequenes re either {, } k {, } n 1 d ω or {, } k {, } n 1 ω (with 0 k n 1). So deterministi utomton or miguity must hve (t lest) 2 n sttes rehle ter n events. 11/39
Outline Amiguity in Lelled Trnsition System (LTS) 2 Ative dignosis in LTS From LTS to proilisti LTS Anlysis o tive dignosis in LTS 12/39
Controllle LTS nd tive dignoser Events re lso prtitioned in ontrollle nd unontrollle events. Controllle events must e oservle. A ontroller orids ontrollle events depending on the urrent oserved sequene. An tive dignoser is ontroller suh tht the ontrolled LTS: is still live; does not ontin miguous sequenes. The dely o n tive dignoser is the mximl numer o event ourrenes etween exeution sequene is ulty nd n oserved sequene is surely ulty. 13/39
An exmple o tive dignoser The miguous sequenes re {, } ω. The (inite-stte) tive dignoser orids two onseutive. Its dely is 3 (t most n ourrene o ). q 0 q 1 q 2 Σ, Σ, Σ\{},, 14/39
Ative dignosis prolems The tive dignosis deision prolem, i.e. deide whether LTS is tively dignosle. The synthesis prolem, i.e. deide whether LTS is tively dignosle nd in the positive se uild n tive dignoser. The miniml-dely synthesis prolem, i.e. deide whether LTS is tively dignosle nd in the positive se uild n tive dignoser with miniml dely. 15/39
Bühi gmes A two-plyer (I nd II) Bühi gme is deined y: A grph (V, E) whose verties re owned y plyers with epting verties F ; In vertex v owned y plyer, he selets n edge (v, w) nd the gme goes on with w s urrent vertex. Plyer I wins i Plyer II is stuk in ded vertex or the ininite pth ininitely oten visits F. Gme prolems: Does there exists winning strtegy or Plyer I? In the positive se how to uild suh strtegy? Clssil results: The deision prolem is PTIME-omplete. In the positive se, there is positionl winning strtegy. 16/39
A Bühi gme or tive dignosis Verties o the gme The verties o Plyer I re the sttes o the Bühi utomton. The verties o Plyer II re pirs o sttes o the Bühi utomton nd (susets o) events o the LTS. The epting verties re the epting sttes o the Bühi utomton. Edges o the gme There is n edge ((U, V, W ), ((U, V, W ), Σ )) i Σ is suset o events (inluding the unontrollle ones) suh tht rom ll stte o U V W, there is n oserved sequene lelled y some e Σ. There is n edge (((U, V, W ), Σ ), ((U, V, W ), e) i e Σ. There is n edge (((U, V, W ), e), (U, V, W ) i there is trnsition (U, V, W ) e (U, V, W ) in the Bühi utomton. 17/39
Exmple o Bühi gme q 0 q 1 q 2, 0 1 ({q 0 },, ) ({q 0 },,{q 2 }) 4 5 ({q 0 },{q 1 }, ) (,,{q 2 }) ({q 0 },,{q 1 }) ({q 0 },{q 2 }, ) 2 3 0 (0,{,}) (0,) 1 (0,{,,}) (0,)... (0,{,}) (0,) 2 18/39
Results o this onstrution Correspondene etween prolems There is winning strtegy or Plyer I i nd only i there is n tive dignoser. The sttes o this tive dignoser re the sttes o the Bühi utomton. Consequenes The deision prolem is EXPTIME-omplete (the lower ound holds y redution rom sety gmes with prtil oservtion D. Berwnger nd L. Doyen FSTTCS 2008). The synthesis lgorithm yields n tive dignoser with 2 O(n) sttes. The previous synthesis lgorithm yields douly exponentil numer o sttes (M. Smpth, S. Lortune, nd D. Teneketzis, IEEE TAC 1998). For ll n N, there is LTS with n sttes suh tht ny tive dignoser requires 2 Ω(n) sttes. 19/39
A lower ound or the synthesis prolem...,, l 0 l 1 l 2 l n 1 l n l n+1... q 0 q 1 q 2 q n 1 q n q n+1,,,,d... r 0 r 1 r 2 r n 1 r n r n+1,, An tive dignoser must orid d (resp. ) i it hs oserved n (resp. ) n times eore. So n tive dignoser must hve (t lest) 2 n sttes rehle ter n oservle events., d d 20/39
Wht out miniml dely synthesis? Our synthesis lgorithm provides dely t most twie the miniml dely. For ll n N, there is LTS with n sttes suh tht ny tive dignoser with miniml dely requires 2 Ω(n log(n)) sttes. We hve designed synthesis lgorithm o n tive dignoser with miniml dely tht requires 2 O(n2) sttes. 21/39
Outline Amiguity in Lelled Trnsition System (LTS) Ative dignosis in LTS 3 From LTS to proilisti LTS Anlysis o tive dignosis in LTS 22/39
plts A proilisti lelled trnsition system (plts) is live LTS with trnsition proility mtrix P., 1 2,1 q 1 q 2, 1 2 q 0, 1 3 d, 1 2 u, 1 2 q 3 q 4,1, 1 3 q 5, 1 3,1 Without lels, plts is disrete time Mrkov hin. Without trnsition proilities, plts is LTS. 23/39
(Se) Dignosility A plts is dignosle i the set o sequenes yielding miguous oserved sequenes hs null mesure. A plts is sely dignosle i it is dignosle nd the set o orret sequenes hs positive mesure., 1 2,1 q 1 q 2, 1 2, 1 2,1 q 1 q 2, 1 2 q 0, 1 3 d, 1 2 q 0, 1 2 d, 1 2 u, 1 2 q 3 q 4,1, 1 3 q 5 u, 1 2 q 3 q 4,1, 1 3,1, 1 2 sely dignosle dignosle ut not sely dignosle 24/39
LTS A ontrollle proilisti lelled trnsition system (LTS) is live plts with integer weights on trnsitions. nd prtition etween ontrollle nd unontrollle events. An ontroller orids ontrollle events depending on the urrent oserved sequene. It n rndomly selet the oridden events. A ontroller must not introdue dedloks. Let C e LTS nd π e ontroller. Then C π is plts where the proility re otined y normliztion mong the llowed events. Controller π is (se) tive dignoser i C π is (sely) dignosle. 25/39
Illustrtion, 1 2,1 q 1 q 2, 1 2 A deterministi tive dignoser π: Forid two onseutive ter n. q 0 u, 1 2, 1 d, 1 3 2, 1 3 q 3 q 4,1 q 5, 1 3,1, 1 3 ε,q 4,Σ d,1 d, 1,1 2, 1 2, 1 3 ε,q 1,Σ,q 2,Σ,q, 2,Σ\{} 1 3, 1 2, 1 3 ε,q 0,Σ ε,q 5,Σ u, 1 2,1, 1 3, 1 2 ε,q 3,Σ,q 4,Σ,q 4,Σ\{},1, 1 3, 1 2 26/39
Ative proilisti dignosis The tive proilisti dignosis prolem sks whether there exists n tive dignoser π or C. The se tive proilisti dignosis prolem sks whether there exists se tive dignoser π or C. The synthesis prolems onsist in uilding (se) tive dignoser π or C in the positive se. 27/39
Outline Amiguity in Lelled Trnsition System (LTS) Ative dignosis in LTS From LTS to proilisti LTS 4 Anlysis o tive dignosis in LTS 28/39
Prtilly oserved Mrkov deision proess A prtilly oservle Mrkov deision proess (POMDP) is tuple M = Q, q 0, Os, At, T where: Q is inite set o sttes with q0 the initil stte; Os : Q O ssigns n oservtion O O to eh stte. At is inite set o tions; T : Q At Dist(Q) is prtil trnsition untion. q 0 1 3 2 3 q 1... q 2 Given sequene o oservtions, strtegy rndomly selets n tion to e perormed. Given strtegy, POMDP eomes (possily ininite) plts. 29/39
From LTS dignosis to POMDP prolems Let C e LTS nd its Bühi utomton B, M C is uilt s ollows. Sttes re pirs (l, q) with l stte o B nd q stte o C with Os(l, q) = l. Ations o M C re suset o events tht inludes the unontrollle events. Given some tion Σ, the trnsition proility o M C rom (l, q) to (l, q ) is: the sum o proilities o pths in C rom q to q ; lelled y unoservle events o Σ ; ending with n oservle event Σ suh tht l B l. The proility o ny suh pth is the produt o the individul step proilities. The ltter re then deined y the normliztion o weights w.r.t. Σ. When in C, some pth rehes stte where no event o Σ is possile, one rehes in M C n dditionl stte lost. 30/39
Illustrtion, 1 2,1 q 1 q 2, 1 2 q 0, 1 3 d, 1 2 u, 1 2 q 3 q 4,1, 1 3 q 5, 1 3,1 ({q 0},, ),q 0 Σ\{} Σ 1 2 1 2 ({q 4},,{q 2}),q 2... 1 lost ({q 4},,{q 2}),q 4... 31/39
Deidility o the tive dignosis prolem C is tively dignosle i there exists strtegy in M C suh tht: lmost surely (W = U = ) The existene o strtegy in POMDP or lmost surely stisying Bühi ojetive is deidle (Bier, Bertrnd, Größer, FoSSCS 2008). The proo in (Bertrnd, Genest, Gimert, LICS 2009) is more generl nd elegnt. Anlyzing the redution to the POMDP prolem, we get tht the tive dignosis prolem is EXPTIME-omplete. C is sely tively dignosle i there exists strtegy in M C suh tht: lmost surely (W = U = ); with positive proility U. 32/39
Belie-sed dignosers re not enough In our ontext, the elie is the urrent stte o the Bühi utomton. q 1 q 0,, q 3 q 4 q 2 The LTS is strightorwrdly dignosle ut it is not se. A se tive dignoser must perorm guess nd keep in memory one it: oridding ter n odd numer o oservtions; nd oridding ter n even numer o oservtions. 33/39
Finite-memory dignosers re not enough u r 0 q 2 q 1 q 0 r 1 r 2 An oserved sequene σ is surely ulty i σ Σ ω. An oserved sequene σ is surely orret i σ ( + ) ω. 34/39
Finite-memory dignosers re not enough u r 0 q 2 q 1 q 0 r 1 r 2 A se tive dignoser Pik ny sequene o positive integers {α i } i 1 suh tht i 1 1 > 0. 2 αi Let A = {, u,, } nd A = {, u,, }. Let π e the ontroller tht onsists in seleting, t instnt n, the n th suset in the ollowing sequene A α1 AA α2 A.... Then π is se tive dignoser: All oserved sequenes re either surely ulty or surely orret. The proility tht sequene is orret is 1 2 i 1 1 2 αi > 0. There is no inite-memory se tive dignoser. 35/39
From lind POMDP to se tive dignosis The existene o n ininite word epted y Bühi proilisti utomton with positive proility is undeidle (Bier, Bertrnd, Größer, Fosss 2008). The existene o winning strtegy with positive proility or Bühi ojetive in lind POMDP (i.e. without oservtion) is undeidle (Chtterjee, Doyen, Gimert, Henzinger, MFCS 2010). We redue the ltter prolem to se tive dignosility prolem. Corollry. The prolem whether, given POMDP M with susets o sttes F nd I, there exists strtegy π with P π (M = F ) = 1 nd P π (M = I) > 0, is undeidle. Oservtion: The existene o strtegy or eh ojetive is deidle. 36/39
Sheme o the redution s 2 s M v 2, p t 2 C, p, p, p v 1 i 2, p s 1, p i 1 v, p, p t r 0 u q, 0 r 1 r 2 i t 1, An oserved sequene σ is surely ulty i σ Σ ω. An oserved sequene σ is surely orret i σ (( + ) + ( + )) ω. 37/39
Restrition to inite-memory dignosers Oservtion A priori the inite-memory requirement does not ensure deidility. A deision proedure in EXPTIME: Computing the se elies tht ensure the existene o n tive dignoser surely yielding orret sequenes. Cheking the existene o dignoser tht ensure tive dignosility lmost surely nd rehing elie inluding se elie with positive proility. The tive dignoser only requires n dditionl oolen (or swithing its mode). The prolem is EXPTIME-hrd (using the sme redution s eore). 38/39
Contriutions Conlusion nd perspetives Strong improvement o the tive dignosis proedures or trnsition systems. Almost mthing lower ounds o the tive dignosis prolems or trnsition systems. Introdution o (se) tive dignosis prolems or proilisti systems. Anlysis o the prolems or proilisti systems using POMDP rmework. Perspetives Closing the gp etween lower nd upper ounds relted to the miniml dely synthesis prolem. Introduing the tive preditility prolem (nd other relted issues). Investigting urther POMDP prolems with multiple ojetives. Modelling nd nlyzing dignosis with stohsti gmes. 39/39