Stochastic Games with Time The value Min strategies Max strategies Determinacy Finite-state games Cont.-time Markov chains
|
|
- Edward Sparks
- 5 years ago
- Views:
Transcription
1 Games with Time Finite-state Masaryk University Brno GASICS 00 /39
2 Outline Finite-state stochastic processes. Games over event-driven stochastic processes. Strategies,, determinacy. Existing results for with reachability. Games over discrete-time. Games over continuous-time. Games over event-driven stochastic processes.. GASICS 00 /39
3 in formal verification Finite-state In general: the arena (game board) corresponds to the state-space of a given system; a state reacts to some events whose impact in uncertain; there are special control states where two players, controller and environment can choose some action whose impact may also be uncertain; Is there a strategy for the controller such that the system satisfies a certain property no matter what the environment does? GASICS 00 3/39
4 stochastic processes Finite-state e e 3 3 state-space can be (countably) infinite; each event is either discrete-time or continuous-time; this model is closely related to real-time probabilistic processes [Alur, Courcoubetis, Dill] and generalized semi-markov processes. GASICS 00 4/39
5 stochastic processes () Finite-state u e e l r restart e e d Suppose that e and e have densities f and f and restart is a discrete-time event which takes zero time. The event e awaited in l has actually been fired in u. How do we capture this formally? What is the semantics of a given event-driven stochastic process G? GASICS 00 5/39
6 stochastic processes (3) u e e l r restart e e d Finite-state A fully rigorous approach: define the associated Markov process M G (with uncountable state-space). Usually, the state-space of M G is formed by tuples of the form (s, t,..., t n ). This is not appropriate in our setting. Alternatively, the state-space of M may consist of computational histories of G. A lightweight approach: define a suitable probability space over the runs in G. GASICS 00 6/39
7 stochastic processes (4) Finite-state A run of a given event-driven stochastic process is an infinite sequence (s 0, t 0, e 0 ), (s, t, e ), (s, t, e ),... where t i is the time spent in s i and e i is the triggering event. A basic cylinder determined by a finite sequence (s 0, I 0, e 0 ),... (s n, I n, e n ) consists of all runs of the form (s 0, t 0, e 0 ),... (s n, t n, e n ),... where t i I i for all 0 i n. We define the probability of basic cylinders in the natural way. Thus, we obtain the probability space (Run, F, P). GASICS 00 7/39
8 Games over event-driven stochastic processes Finite-state We add special control states V V where player and player can choose successor states. The impact of this choice may be uncertain in general. Players decisions are timeless. 3 3 GASICS 00 8/39
9 Games over event-driven stochastic processes () e 0.3 Finite-state e e e GASICS 00 9/39
10 Games over event-driven stochastic processes (3) A history of a game G is a finite sequence of the form (s 0, t 0, e 0, v 0 ),..., (s n, t n, e n, v n ) Finite-state A strategy of player, where {, }, is a measurable function which to every history (s 0, t 0, e 0, v 0 ),..., (s n, t n, e n, v n ) such that v n V assigns a probability distribution over the set of actions enabled in v n. Let µ 0 be an initial probability distribution over the set of states of G. Then each pair σ, π of strategies for player and player determines a unique play of G, denoted by G σ,π, which is an event-driven stochastic process. GASICS 00 0/39
11 Games over event-driven stochastic processes (4) A run in a game G is an infinite sequence of the form (s 0, t 0, e 0, v 0 ),..., (s n, t n, e n, v n ),... Finite-state One can define (Borel) σ-algebra over the runs of G which is the least σ-algebra containing all basic cylinders. A Borel objective is Borel set of runs in G. Various Borel are definable by timed automata and linear-time logics. GASICS 00 /39
12 Finite-state Games over event-driven stochastic processes (5) Theorem Games over event-driven stochastic processes with Borel have a value. That is, sup inf σ π Pσ,π (R) = inf sup P σ,π (R) π σ where R Run is Borel. Thm. follows directly from the result of Maitra & Sudderth [998] (which relies on Martin s determinacy result for Blackwell ). Thm. implies the existence of ε-optimal strategies for both players, but not the existence of optimal strategies. One can use various formalisms (e.g., timed automata) to construct finite representations of time-dependant strategies. GASICS 00 /39
13 Finite-state Let G be a game and T a set of target nodes. reach(t) consists of all runs that visit a target node. reach t (T) consists of all runs that visit a target node before time t. The goal of player / is to maximize/minimize the probability of reach t (T) (or reach(t)). The problems of our interest. Do the players have optimal strategies? And of what type? Can we compute the value and ε-optimal strategies? How about win-lose of the form P ϱ (reach(t))? Are such determined? If so, what is the type of winning strategies? GASICS 00 3/39
14 over Also known as simple stochastic. There is only one discrete-time event e with delay. Recall that the state-space can be infinite. Finite-state GASICS 00 4/39
15 have a value Finite-state Theorem Let G = (V, E, (V, V, V ), Prob) be a game, T V target vertices. For every v V we have that sup σ inf π Pσ,π v (reach(t)) = inf sup π σ P σ,π v (reach(t)) GASICS 00 5/39
16 Finite-state have a value () Proof sketch. Let Γ : [0, ] V [0, ] V be a (monotonic) function defined by if v T; sup {α(v ) (v, v ) E} if v T and v V ; Γ(α)(v) = inf {α(v ) (v, v ) E} if v T and v V ; (v,v ) E Prob(v, v ) α(v ) if v T and v V. µγ(v) sup inf σ π Pσ,π v (reach(t)) inf sup π σ P σ,π v (reach(t)) the second inequality holds for all Borel ; the tuple of all sup inf σ π Pσ,π v (reach(t)) is a fixed-point of Γ. It cannot be that µγ(v) < inf π sup σ P σ,π v (reach(t)) For all ε > 0 and v V, there is a strategy ˆπ such that sup σ P σ,ˆπ v (reach(t)) µγ(v) + ε. GASICS 00 6/39
17 Minimizing strategies () Finite-state Definition 3 (Locally optimal minimizing strategy) Let G = (V, E, (V, V, V ), Prob) be a game. An edge (v, v ) E is value minimizing if val(v ) = min { val(ˆv) V (v, ˆv) E } A locally optimal minimizing strategy is a strategy which in every play selects only value minimizing edges. GASICS 00 7/39
18 Finite-state Minimizing strategies () Theorem 4 Every locally optimal min. strategy is an optimal min. strategy. Proof. Let v V be an initial vertex, and u V a target vertex. () After playing k rounds according to a locally optimal minimizing strategy, player can switch to ε-optimal minimizing strategies in the current vertices of the play. Thus, we always (for every k and ε > 0) obtain an ε-optimal minimizing strategy for v. () Let π be a locally optimal min. strategy which is not optimal. Then there is a strategy σ of player such that (reach(t)) = val(v) + δ, where δ > 0. P σ,π v This means that there is k N such that P σ,π v (reachk (T)) > val(v) + δ. Hence, if player switches to δ 4-optimal minimizing strategy after playing k rounds according to π, we do not obtain a δ 4-optimal minimizing strategy for v. GASICS 00 8/39
19 Minimizing strategies (3) Finite-state Corollary 5 (Properties of minimizing strategies.) In every finitely-branching game, there is an optimal minimizing MD strategy. Theorem 6 Every optimal min. strategy is a locally optimal min. strategy. Hence, if player has some optimal minimizing strategy, then she also has an MD optimal minimizing strategy. GASICS 00 9/39
20 Minimizing strategies (4) Theorem 7 Optimal minimizing strategies do not necessarily exist, and (ε-) optimal minimizing strategies may require infinite memory. Proof. Finite-state r v i s s s 3 s i 4 8 i GASICS 00 0/39
21 Maximizing strategies () Observation 8 A locally optimal maximizing strategy is not necessarily an optimal maximizing strategy. This holds even for finite-state MDPs. Finite-state Proof. v t GASICS 00 /39
22 Finite-state Maximizing strategies () Theorem 9 Let v V be a vertex with finitely many successors t,..., t n. Then there is i n such that val(v) does not change if all edges (v, t j ), where i j, are deleted from the game. Proof. v V (σ,π) t k = P(u) P(u)+P( ) if P(u) + P( ) > 0; 0 otherwise; t k v u V σ t k = inf π V (σ,π) t k V tk = sup σ V σ t k There must be some k such that V tk = val(v). We put i = k. GASICS 00 /39
23 Maximizing strategies (3) Theorem 0 Optimal maximizing strategies may not exist, even in finitely-branching MDPs. Finite-state v GASICS 00 3/39
24 Maximizing strategies (4) Theorem Optimal maximizing strategies may require infinite memory, even in finitely-branching. Finite-state ˆv d d d 3 d 4 d 5 v e e e 3 e 4 e 5 s s s 3 s 4 s 5 GASICS 00 4/39
25 Summary Finite-state Minimizing strategies: Optimal minimizing strategies may not exist. Optimal and ε-optimal minimizing strategies may require infinite memory. In finitely-branching, there are MD optimal minimizing strategies. Maximizing strategies: Optimal maximizing strategies may not exist, even in finitely-branching. Optimal maximizing strategies may require infinite memory. In finite-state, there are MD optimal maximizing strategies. GASICS 00 5/39
26 as a win-lose objective () Finite-state Let ϱ [0, ]. A strategy σ Σ is ( ϱ)-winning in v if for every π Π we have that P (σ,π) v (reach(t) ϱ). A strategy π Π is (<ϱ)-winning if for every σ Σ we have that P (σ,π) v (reach(t) < ϱ). Is there a winning strategy for one of the two players? GASICS 00 6/39
27 as a win-lose objective () Finite-state Theorem Turn-based stochastic with reachability are not necessarily determined. However, finitely-branching are determined. GASICS 00 7/39
28 as a win-lose objective (3) u s v Finite-state u v GASICS 00 8/39
29 Algorithms for finite-state MDP and Finite-state We show how to compute the values and optimal strategies for reachability in finite-state and MDPs. For finite-state MDPs we have that the values and optimal strategies are computable in polynomial time by linear programming; For finite-state we have that the values and optimal strategies are computable in polynomial space (for a fixed number of randomized vertices, the problem is in P [Gimbert, Horn, 008]); There are also algorithms for certain classes of infinite-state. GASICS 00 9/39
30 References Finite-state D.A. Martin. The of Blackwell Games. The Journal of Symbolic Logic, Vol. 63, No. 4 (Dec., 998), pp A. Maitra and W. Sudderth. Finitely Additive Games with Borel Measurable Payoffs. International Journal of Game Theory, Vol. 7 (998), pp M.L. Puterman. Markov Decision Processes, Wiley, 994. T. Brázdil, V. Brožek, V. Forejt, A. Kučera. in recursive Markov decision processes. A. Kučera. Turn-based Games. In Lectures in Game Theory for Computer Scientists, Cambridge. To appear. A. Condon. The Complexity of Games. Information and Computation, 96():03 4, 99. L.S. Shapley.. Proceedings of the National Academy of Sciences USA, 39:095 00, 953. H. Gimbert, F. Horn. Simple Games with Few Random Vertices Are Easy to Solve. Proc. FoSSaCS 008, pp. 5 9, LNCS 496, Springer, 008. GASICS 00 30/39
31 Games over cont.-time Finite-state All events are continuous and exponentially distributed, i.e., P(d e t) = e λt where λ > 0 is a rate of the event e. For simplicity, we assume that the impact of players choice is determined (i.e., the distributions associated to the available actions are Dirac). Some facts about exponential distribution: Let X E(λ). Then P(X t + t X t ) = P(X t). Let X E(λ) and Y E(κ). Then min(x, Y) E(λ + κ) and P(X < Y) = λ/(λ + κ). GASICS 00 3/39
32 Games over cont.-time () Finite-state e E(λ) e E(κ) 3 3 λ κ+λ e E(κ + λ) κ κ+λ 3 3 GASICS 00 3/39
33 Games over cont.-time (3) Finite-state GASICS 00 33/39
34 Games over cont.-time (4) Finite-state -bounded reachability have been so far studied mainly for time abstract strategies. Theorem 3 Let G = (V, E, (V, V, V ), Prob) be a game, T V target vertices. For every v V we have that sup σ inf π Pσ,π v (reach t (T)) = inf sup π σ P σ,π v (reach t (T)) where σ and π range over time abstract strategies. GASICS 00 34/39
35 Games over cont.-time (5) Finite-state Proof sketch. Let H be the set of all histories i : R(G) N 0 where a R(G) i(a) <. Let Γ : (H V [0, ]) (H V [0, ]) be a (monotonic) function defined by F i (t) v T Γ(H)(i, v) = sup a E(v) u V P(a)(u) H(i + Rate(a), u) v V \ T inf a E(v) u V P(a)(u) H(i + Rate(e), u) v V \ T Let µγ be the least fixed-point of Γ. of a given vertex v is equal to µγ(0, v). Observation: as a R(G) i(a) increases, F i (t) approaches zero (assuming the rates are bounded). Hence, Γ allows to compute ε-optimal strategies. GASICS 00 35/39
36 Games over cont.-time (6) Finite-state In general, optimal strategies do not exist. In finitely-branching, player is guaranteed to have an optimal CD strategy. In finitely-branching with bounded rates, player is guaranteed to have an optimal CD strategy. In finitely-branching uniform, both players have BCD optimal strategies that are effectively computable. GASICS 00 36/39
37 Games over cont.-time (7) Finite-state References: C. Baier, H. Hermanns, J.-P. Katoen, and B.R. Haverkort. Efficient computation of time-bounded reachability probabilities in uniform continuous-time Markov decision processes. Theoretical Computer Science, 345: 6, 005. M. Neuhäußer, M. Stoelinga, and J.-P. Katoen. Delayed nondeterminism in continuous-time Markov decision processes. In Proceedings of FoSSaCS 009, volume 5504 of LNCS, pages Springer, 009. T. Brázdil, V. Forejt, J. Krčál, J. Křetínský, and A. Kučera. Continuous-time stochastic -bounded reachability. In Proceedings of FST&TCS 009, pages 6 7, 009. M. Rabe and S. Schewe. Optimal time-abstract schedulers for CTMDPs and Markov. In Eighth Workshop on Quantitative Aspects of Programming Languages, 00. GASICS 00 37/39
38 Games over event-driven stochastic processes Finite-state Assuming that all events are continuous and the objective is encoded as a deterministic timed automaton, one can decide if player has an almost-sure winning strategy and compute a finite description of this strategy [Brázdil et al., Concur 00]. It is not easy to extend this result to the general model with both continuous and discrete events. GASICS 00 38/39
39 , open problems Finite-state Games over event-driven stochastic processes can model concurrent systems with stochastic delays that are not necessarily exponentially distributed. One can rely on rich theory of discrete-time stochastic and Markov processes with general state-space. Almost everything is open. New theoretical results can also bring efficient algorithms for solving the considered problems. GASICS 00 39/39
Randomness for Free. 1 Introduction. Krishnendu Chatterjee 1, Laurent Doyen 2, Hugo Gimbert 3, and Thomas A. Henzinger 1
Randomness for Free Krishnendu Chatterjee 1, Laurent Doyen 2, Hugo Gimbert 3, and Thomas A. Henzinger 1 1 IST Austria (Institute of Science and Technology Austria) 2 LSV, ENS Cachan & CNRS, France 3 LaBri
More informationContinuous-Time Stochastic Games with Time-Bounded Reachability
Continuous-Time Stochastic Games with Time-Bounded Reachability Tomáš Brázdil, Vojtěch Forejt 1, Jan Krčál, Jan Křetínský 2,, Antonín Kučera Faculty of Informatics, Masaryk University, Botanická 68a, 60200
More informationParity Objectives in Countable MDPs
Parity Objectives in Countable MDPs Stefan Kiefer, Richard Mayr, Mahsa Shirmohammadi, Dominik Wojtczak University of Oxford, UK University of Edinburgh, UK University of Liverpool, UK Abstract We study
More informationState Explosion in Almost-Sure Probabilistic Reachability
State Explosion in Almost-Sure Probabilistic Reachability François Laroussinie Lab. Spécification & Vérification, ENS de Cachan & CNRS UMR 8643, 61, av. Pdt. Wilson, 94235 Cachan Cedex France Jeremy Sproston
More informationOn Strong Determinacy of Countable Stochastic Games
On Strong Determinacy of Countable Stochastic Games Stefan Kiefer, Richard Mayr, Mahsa Shirmohammadi, Dominik Wojtczak University of Oxford, UK University of Edinburgh, UK University of Liverpool, UK arxiv:70405003v
More informationAlgorithmic Game Theory and Applications. Lecture 15: a brief taster of Markov Decision Processes and Stochastic Games
Algorithmic Game Theory and Applications Lecture 15: a brief taster of Markov Decision Processes and Stochastic Games Kousha Etessami warning 1 The subjects we will touch on today are so interesting and
More informationLimiting Behavior of Markov Chains with Eager Attractors
Limiting Behavior of Markov Chains with Eager Attractors Parosh Aziz Abdulla Uppsala University, Sweden. parosh@it.uu.se Noomene Ben Henda Uppsala University, Sweden. Noomene.BenHenda@it.uu.se Sven Sandberg
More informationA Survey of Partial-Observation Stochastic Parity Games
Noname manuscript No. (will be inserted by the editor) A Survey of Partial-Observation Stochastic Parity Games Krishnendu Chatterjee Laurent Doyen Thomas A. Henzinger the date of receipt and acceptance
More informationStrategy Synthesis for Markov Decision Processes and Branching-Time Logics
Strategy Synthesis for Markov Decision Processes and Branching-Time Logics Tomáš Brázdil and Vojtěch Forejt Faculty of Informatics, Masaryk University, Botanická 68a, 60200 Brno, Czech Republic. {brazdil,forejt}@fi.muni.cz
More informationA Survey of Stochastic ω-regular Games
A Survey of Stochastic ω-regular Games Krishnendu Chatterjee Thomas A. Henzinger EECS, University of California, Berkeley, USA Computer and Communication Sciences, EPFL, Switzerland {c krish,tah}@eecs.berkeley.edu
More informationAlternating nonzero automata
Alternating nonzero automata Application to the satisfiability of CTL [,, P >0, P =1 ] Hugo Gimbert, joint work with Paulin Fournier LaBRI, Université de Bordeaux ANR Stoch-MC 06/07/2017 Control and verification
More informationPerfect-information Stochastic Parity Games
Perfect-information Stochastic Parity Games Wies law Zielonka LIAFA, case 7014 Université Paris 7 2, Place Jussieu 75251 Paris Cedex 05, France zielonka@liafa.jussieu.fr Abstract. We show that in perfect-information
More informationReachability in Recursive Markov Decision Processes
Reachability in Recursive Markov Decision Processes Tomáš Brázdil, Václav Brožek, Vojtěch Forejt, Antonín Kučera Faculty of Informatics, Masaryk University, Botanická 68a, 6000 Brno, Czech Republic. Abstract
More informationControlling probabilistic systems under partial observation an automata and verification perspective
Controlling probabilistic systems under partial observation an automata and verification perspective Nathalie Bertrand, Inria Rennes, France Uncertainty in Computation Workshop October 4th 2016, Simons
More informationReachability in Continuous-Time Markov Reward Decision Processes
Reachability in Continuous-Time Markov Reward Decision Processes Christel Baier 1 Boudewijn R. Haverkort 2 Holger Hermanns 3 Joost-Pieter Katoen 4 1 Faculty of Computer Science Technical University of
More informationPlaying Stochastic Games Precisely
Playing Stochastic Games Precisely Taolue Chen 1, Vojtěch Forejt 1, Marta Kwiatkowska 1, Aistis Simaitis 1, Ashutosh Trivedi 2, and Michael Ummels 3 1 Department of Computer Science, University of Oxford,
More informationReachability-Time Games on Timed Automata (Extended Abstract)
Reachability-Time Games on Timed Automata (Extended Abstract) Marcin Jurdziński and Ashutosh Trivedi Department of Computer Science, University of Warwick, UK Abstract. In a reachability-time game, players
More informationPOLYNOMIAL SPACE QSAT. Games. Polynomial space cont d
T-79.5103 / Autumn 2008 Polynomial Space 1 T-79.5103 / Autumn 2008 Polynomial Space 3 POLYNOMIAL SPACE Polynomial space cont d Polynomial space-bounded computation has a variety of alternative characterizations
More informationRecursive Stochastic Games with Positive Rewards
Recursive Stochastic Games with Positive Rewards K. Etessami 1, D. Wojtczak 1, and M. Yannakakis 2 1 LFCS, School of Informatics, University of Edinburgh 2 Dept. of Computer Science, Columbia University
More informationA Note on the Approximation of Mean-Payoff Games
A Note on the Approximation of Mean-Payoff Games Raffaella Gentilini 1 1 University of Perugia, Italy Abstract. We consider the problem of designing approximation schemes for the values of mean-payoff
More informationTwo Views on Multiple Mean-Payoff Objectives in Markov Decision Processes
wo Views on Multiple Mean-Payoff Objectives in Markov Decision Processes omáš Brázdil Faculty of Informatics Masaryk University Brno, Czech Republic brazdil@fi.muni.cz Václav Brožek LFCS, School of Informatics
More informationThe Complexity of Nash Equilibria in Simple Stochastic Multiplayer Games *
The Complexity of Nash Equilibria in Simple Stochastic Multiplayer Games * Michael Ummels 1 and Dominik Wojtczak 2,3 1 RWTH Aachen University, Germany E-Mail: ummels@logic.rwth-aachen.de 2 CWI, Amsterdam,
More informationAutomata, Logic and Games: Theory and Application
Automata, Logic and Games: Theory and Application 2 Parity Games, Tree Automata, and S2S Luke Ong University of Oxford TACL Summer School University of Salerno, 14-19 June 2015 Luke Ong S2S 14-19 June
More informationOn Decision Problems for Probabilistic Büchi Automata
On Decision Problems for Probabilistic Büchi Automata Christel Baier a, Nathalie Bertrand a,b, and Marcus Größer a a Technische Universität Dresden, Germany b IRISA/INRIA Rennes, France Abstract. Probabilistic
More informationEntropy Games and Matrix Multiplication Games. EQINOCS seminar
Entropy Games and Matrix Multiplication Games Eugene Asarin Julien Cervelle Aldric Degorre Cătălin Dima Florian Horn Victor Kozyakin IRIF, LACL, IITP EQINOCS seminar 2016-05-11 A game of freedom The story
More informationOn Model Checking Techniques for Randomized Distributed Systems. Christel Baier Technische Universität Dresden
On Model Checking Techniques for Randomized Distributed Systems Christel Baier Technische Universität Dresden joint work with Nathalie Bertrand Frank Ciesinski Marcus Größer / 6 biological systems, resilient
More informationEdinburgh Research Explorer
Edinburgh Research Explorer Recursive Concurrent Stochastic Games Citation for published version: Etessami, K & Yannakakis, M 2006, 'Recursive Concurrent Stochastic Games'. in ICALP (2). Springer- Verlag
More informationDISTINGUING NON-DETERMINISTIC TIMED FINITE STATE MACHINES
DISTINGUING NON-DETERMINISTIC TIMED FINITE STATE MACHINES Maxim Gromov 1, Khaled El-Fakih 2, Natalia Shabaldina 1, Nina Yevtushenko 1 1 Tomsk State University, 36 Lenin Str.. Tomsk, 634050, Russia gromov@sibmail.com,
More informationSFM-11:CONNECT Summer School, Bertinoro, June 2011
SFM-:CONNECT Summer School, Bertinoro, June 20 EU-FP7: CONNECT LSCITS/PSS VERIWARE Part 3 Markov decision processes Overview Lectures and 2: Introduction 2 Discrete-time Markov chains 3 Markov decision
More informationProbabilistic Weighted Automata
Probabilistic Weighted Automata Krishnendu Chatterjee 1, Laurent Doyen 2, and Thomas A. Henzinger 3 1 Institute of Science and Technology (IST), Austria 2 Université Libre de Bruxelles (ULB), Belgium 3
More informationA note on the attractor-property of infinite-state Markov chains
A note on the attractor-property of infinite-state Markov chains Christel Baier a, Nathalie Bertrand b, Philippe Schnoebelen b a Universität Bonn, Institut für Informatik I, Germany b Lab. Specification
More informationModels for Efficient Timed Verification
Models for Efficient Timed Verification François Laroussinie LSV / ENS de Cachan CNRS UMR 8643 Monterey Workshop - Composition of embedded systems Model checking System Properties Formalizing step? ϕ Model
More informationA reduction from parity games to simple stochastic games
A reduction from parity games to simple stochastic games Krishnendu Chatterjee IST Austria (Institute of Science and Technology, Austria) krishnendu.chatterjee@ist.ac.at Nathanaël Fijalkow IST Austria
More informationSolving Partial-Information Stochastic Parity Games
Solving Partial-Information Stochastic Parity ames Sumit Nain and Moshe Y. Vardi Department of Computer Science, Rice University, Houston, Texas, 77005 Email: {nain,vardi}@cs.rice.edu Abstract We study
More informationNash Equilibria in Concurrent Games with Büchi Objectives
Nash Equilibria in Concurrent Games with Büchi Objectives Patricia Bouyer, Romain Brenguier, Nicolas Markey, and Michael Ummels LSV, CNRS & ENS Cachan, France {bouyer,brenguier,markey,ummels}@lsv.ens-cachan.fr
More informationLong-run Average Reward for Markov Decision Processes
Long-run Average Reward for Markov Decision Processes Based on a paper at CAV 2017 Pranav Ashok 1, Krishnendu Chatterjee 2, Przemysław Daca 2, Jan Křetínský 1 and Tobias Meggendorfer 1 August 9, 2017 1
More informationRobust Controller Synthesis in Timed Automata
Robust Controller Synthesis in Timed Automata Ocan Sankur LSV, ENS Cachan & CNRS Joint with Patricia Bouyer, Nicolas Markey, Pierre-Alain Reynier. Ocan Sankur (ENS Cachan) Robust Control in Timed Automata
More informationarxiv: v4 [cs.lo] 2 Oct 2010
Generalized Mean-payoff and Energy Games Krishnendu Chatterjee 1, Laurent Doyen 2, Thomas A. Henzinger 1, and Jean-François Raskin 3 1 IST Austria (Institute of Science and Technology Austria) 2 LSV, ENS
More informationMean-Payoff Games and the Max-Atom Problem
Mean-Payoff Games and the Max-Atom Problem Albert Atserias Universitat Politècnica de Catalunya Barcelona, Spain Elitza Maneva Universitat Politècnica de Catalunya Barcelona, Spain February 3, 200 Abstract
More informationLecture notes for Analysis of Algorithms : Markov decision processes
Lecture notes for Analysis of Algorithms : Markov decision processes Lecturer: Thomas Dueholm Hansen June 6, 013 Abstract We give an introduction to infinite-horizon Markov decision processes (MDPs) with
More informationMeasuring Permissivity in Finite Games
Measuring Permissivity in Finite Games Patricia Bouyer, Marie Duflot, Nicolas Markey, and Gabriel Renault 3 LSV, CNRS & ENS Cachan, France {bouyer,markey}@lsv.ens-cachan.fr LACL, Université Paris, France
More informationModel Checking Durational Probabilistic Systems
Model Checking Durational Probabilistic Systems (Extended abstract)? François Laroussinie 1 and Jeremy Sproston 2 1 Lab. Spécification & Verification, ENS Cachan CNRS UMR 8643, France 2 Dipartimento di
More informationThe priority promotion approach to parity games
The priority promotion approach to parity games Massimo Benerecetti 1, Daniele Dell Erba 1, and Fabio Mogavero 2 1 Università degli Studi di Napoli Federico II 2 Università degli Studi di Verona Abstract.
More informationTHE BIG MATCH WITH A CLOCK AND A BIT OF MEMORY
THE BIG MATCH WITH A CLOCK AND A BIT OF MEMORY KRISTOFFER ARNSFELT HANSEN, RASMUS IBSEN-JENSEN, AND ABRAHAM NEYMAN Abstract. The Big Match is a multi-stage two-player game. In each stage Player hides one
More informationThe State Explosion Problem
The State Explosion Problem Martin Kot August 16, 2003 1 Introduction One from main approaches to checking correctness of a concurrent system are state space methods. They are suitable for automatic analysis
More informationGames with Discrete Resources
Games with Discrete Resources Sylvain Schmitz with Th. Colcombet, J.-B. Courtois, M. Jurdziński, and R. Lazić LSV, ENS Paris-Saclay & CNRS IBISC, October 19, 217 1/12 Outline multi-dimensional energy parity
More informationLogic Model Checking
Logic Model Checking Lecture Notes 10:18 Caltech 101b.2 January-March 2004 Course Text: The Spin Model Checker: Primer and Reference Manual Addison-Wesley 2003, ISBN 0-321-22862-6, 608 pgs. the assignment
More informationQuasi-Weak Cost Automata
Quasi-Weak Cost Automata A New Variant of Weakness Denis Kuperberg 1 Michael Vanden Boom 2 1 LIAFA/CNRS/Université Paris 7, Denis Diderot, France 2 Department of Computer Science, University of Oxford,
More informationAutomated Verification and Strategy Synthesis for Probabilistic Systems
Automated Verification and Strategy Synthesis for Probabilistic Systems Marta Kwiatkowska 1 and David Parker 2 1 Department of Computer Science, University of Oxford, UK 2 School of Computer Science, University
More informationOn positional strategies over finite arenas
On positional strategies over finite arenas Damian Niwiński University of Warsaw joint work with Thomas Colcombet Berlin 2018 Disclaimer. Credits to many authors. All errors are mine own. 1 Perfect information
More informationInfinite Games. Sumit Nain. 28 January Slides Credit: Barbara Jobstmann (CNRS/Verimag) Department of Computer Science Rice University
Infinite Games Sumit Nain Department of Computer Science Rice University 28 January 2013 Slides Credit: Barbara Jobstmann (CNRS/Verimag) Motivation Abstract games are of fundamental importance in mathematics
More informationIntroduction. Büchi Automata and Model Checking. Outline. Büchi Automata. The simplest computation model for infinite behaviors is the
Introduction Büchi Automata and Model Checking Yih-Kuen Tsay Department of Information Management National Taiwan University FLOLAC 2009 The simplest computation model for finite behaviors is the finite
More informationApproximately Bisimilar Finite Abstractions of Stable Linear Systems
Approximately Bisimilar Finite Abstractions of Stable Linear Systems Antoine Girard Université Joseph Fourier Laboratoire de Modélisation et Calcul B.P. 53, 38041 Grenoble, France Antoine.Girard@imag.fr
More informationERICH GRÄDEL a AND IGOR WALUKIEWICZ b. address: address:
Logical Methods in Computer Science Vol. 2 (4:6) 2006, pp. 1 22 www.lmcs-online.org Submitted Feb. 27, 2006 Published Nov. 3, 2006 POSITIONAL DETERMINACY OF GAMES WITH INFINITELY MANY PRIORITIES ERICH
More informationNash Equilibria for Reachability Objectives in Multi-player Timed Games
Nash Equilibria for Reachability Objectives in Multi-player Timed Games Patricia Bouyer, Romain Brenguier, and Nicolas Markey LSV, ENS Cachan & CNRS, France {bouyer,brenguie,markey}@lsv.ens-cachan.fr Abstract.
More informationBounded Model Checking with SAT/SMT. Edmund M. Clarke School of Computer Science Carnegie Mellon University 1/39
Bounded Model Checking with SAT/SMT Edmund M. Clarke School of Computer Science Carnegie Mellon University 1/39 Recap: Symbolic Model Checking with BDDs Method used by most industrial strength model checkers:
More informationThe Complexity of Solving Stochastic Games on Graphs
The Complexity of Solving Stochastic Games on Graphs Daniel Andersson and Peter Bro Miltersen Department of Computer Science, Aarhus University, Denmark Abstract. We consider some well-known families of
More informationComplexity Bounds for Regular Games (Extended Abstract)
Complexity Bounds for Regular Games (Extended Abstract) Paul Hunter and Anuj Dawar University of Cambridge Computer Laboratory, Cambridge CB3 0FD, UK. paul.hunter@cl.cam.ac.uk, anuj.dawar@cl.cam.ac.uk
More informationGeneralized Parity Games
Generalized Parity Games Krishnendu Chatterjee 1, Thomas A. Henzinger 1,2, and Nir Piterman 2 1 University of California, Berkeley, USA 2 EPFL, Switzerland c krish@eecs.berkeley.edu, {tah,nir.piterman}@epfl.ch
More informationExpectations or Guarantees? I Want It All! A Crossroad between Games and MDPs
Expectations or Guarantees? I Want It All! A Crossroad between Games and MDPs V. Bruyère (UMONS) E. Filiot (ULB) M. Randour (UMONS-ULB) J.-F. Raskin (ULB) Grenoble - 0.04.04 SR 04 - nd International Workshop
More informationParameterized verification of many identical probabilistic timed processes
Parameterized verification of many identical probabilistic timed processes Nathalie Bertrand 1 and Paulin Fournier 2 1 Inria Rennes, France nathalie.bertrand@inria.fr 2 ENS Cachan Antenne de Bretagne,
More informationRecent results on Timed Systems
Recent results on Timed Systems Time Petri Nets and Timed Automata Béatrice Bérard LAMSADE Université Paris-Dauphine & CNRS berard@lamsade.dauphine.fr Based on joint work with F. Cassez, S. Haddad, D.
More informationThe Complexity of Ergodic Mean-payoff Games,
The Complexity of Ergodic Mean-payoff Games, Krishnendu Chatterjee Rasmus Ibsen-Jensen Abstract We study two-player (zero-sum) concurrent mean-payoff games played on a finite-state graph. We focus on the
More informationCS 208: Automata Theory and Logic
CS 28: Automata Theory and Logic b a a start A x(la(x) y(x < y) L b (y)) B b Department of Computer Science and Engineering, Indian Institute of Technology Bombay of 32 Nondeterminism Alternation 2 of
More informationAutomata, Logic and Games: Theory and Application
Automata, Logic and Games: Theory and Application 1. Büchi Automata and S1S Luke Ong University of Oxford TACL Summer School University of Salerno, 14-19 June 2015 Luke Ong Büchi Automata & S1S 14-19 June
More informationComplexity Bounds for Muller Games 1
Complexity Bounds for Muller Games 1 Paul Hunter a, Anuj Dawar b a Oxford University Computing Laboratory, UK b University of Cambridge Computer Laboratory, UK Abstract We consider the complexity of infinite
More informationParity Objectives in Countable MDPs
Parity Objectives in Countable MDPs Stefan Kiefer Richard Mayr Mahsa Shirmohammadi Dominik Wojtczak LICS 07, Reykjavik 0 June 07 Kiefer, Mayr, Shirmohammadi, Wojtczak Parity Objectives in Countable MDPs
More informationComputing Laboratory STOCHASTIC GAMES FOR VERIFICATION OF PROBABILISTIC TIMED AUTOMATA. Marta Kwiatkowska Gethin Norman David Parker CL-RR-09-05
Computing Laboratory STOCHASTIC AMES FOR VERIFICATION OF PROBABILISTIC TIMED AUTOMATA Marta Kwiatkowska ethin Norman David Parker CL-RR-09-05 Oxford University Computing Laboratory Wolfson Building, Parks
More informationNash Equilibrium for Upward-Closed Objectives Krishnendu Chatterjee Report No. UCB/CSD August 2005 Computer Science Division (EECS) University
Nash Equilibrium for Upward-Closed Objectives Krishnendu Chatterjee Report No. UCB/CSD-5-1407 August 2005 Computer Science Division (EECS) University of California Berkeley, California 94720 Nash Equilibrium
More informationVerification of Probabilistic Systems with Faulty Communication
Verification of Probabilistic Systems with Faulty Communication P. A. Abdulla 1, N. Bertrand 2, A. Rabinovich 3, and Ph. Schnoebelen 2 1 Uppsala University, Sweden 2 LSV, ENS de Cachan, France 3 Tel Aviv
More informationQuantitative Verification
Quantitative Verification Chapter 3: Markov chains Jan Křetínský Technical University of Munich Winter 207/8 / 84 Motivation 2 / 84 Example: Simulation of a die by coins Knuth & Yao die Simulating a Fair
More informationSemi-Automatic Distributed Synthesis
Semi-Automatic Distributed Synthesis Bernd Finkbeiner and Sven Schewe Universität des Saarlandes, 66123 Saarbrücken, Germany {finkbeiner schewe}@cs.uni-sb.de Abstract. We propose a sound and complete compositional
More informationModel Checking Probabilistic Timed Automata with One or Two Clocks
Model Checking Probabilistic Timed Automata with One or Two Clocks Marcin Jurdziński 1, François Laroussinie 2, and Jeremy Sproston 3 1 Department of Computer Science, University of Warwick, Coventry CV4
More informationA subexponential lower bound for the Random Facet algorithm for Parity Games
A subexponential lower bound for the Random Facet algorithm for Parity Games Oliver Friedmann 1 Thomas Dueholm Hansen 2 Uri Zwick 3 1 Department of Computer Science, University of Munich, Germany. 2 Center
More informationOptimal Control of MDPs with Temporal Logic Constraints
52nd IEEE Conference on Decision and Control December 10-13, 2013. Florence, Italy Optimal Control of MDPs with Temporal Logic Constraints Mária Svoreňová, Ivana Černá and Calin Belta Abstract In this
More informationThe Complexity of Stochastic Müller Games
The Complexity of Stochastic Müller Games Krishnendu Chatterjee Electrical Engineering and Computer Sciences University of California at Berkeley Technical Report No. UCB/EECS-2007-110 http://www.eecs.berkeley.edu/pubs/techrpts/2007/eecs-2007-110.html
More informationThe Odds of Staying on Budget
The Odds of Staying on Budget Christoph Haase 1 and Stefan Kiefer 2 1 Laboratoire Spécification et Vérification (LSV), CNRS & ENS de Cachan, France 2 Department of Computer Science, University of Oxford,
More informationAn Alternative Construction in Symbolic Reachability Analysis of Second Order Pushdown Systems
An Alternative Construction in Symbolic Reachability Analysis of Second Order Pushdown Systems Anil Seth CSE Department, I.I.T. Kanpur, Kanpur 208016, INDIA. seth@cse.iitk.ac.in Abstract. Recently, it
More informationAutomata-based Verification - III
COMP30172: Advanced Algorithms Automata-based Verification - III Howard Barringer Room KB2.20: email: howard.barringer@manchester.ac.uk March 2009 Third Topic Infinite Word Automata Motivation Büchi Automata
More informationBoundedness Games. Séminaire du LIGM, April 16th, Nathanaël Fijalkow. Institute of Informatics, Warsaw University Poland
Boundedness Games Séminaire du LIGM, April 16th, 2013 Nathanaël Fijalkow Institute of Informatics, Warsaw University Poland LIAFA, Université Paris 7 Denis Diderot France (based on joint works with Krishnendu
More informationBisimilarity of Probabilistic Pushdown Automata
Bisimilarity of Probabilistic Pushdown Automata Vojtěch Forejt 1, Petr Jančar 2, Stefan Kiefer 1, and James Worrell 1 1 Department of Computer Science, University of Oxford, UK 2 Department of Computer
More informationarxiv: v3 [cs.gt] 10 Apr 2009
The Complexity of Nash Equilibria in Simple Stochastic Multiplayer Games Michael Ummels and Dominik Wojtczak,3 RWTH Aachen University, Germany E-Mail: ummels@logicrwth-aachende CWI, Amsterdam, The Netherlands
More informationReal-Time Systems. Lecture 15: The Universality Problem for TBA Dr. Bernd Westphal. Albert-Ludwigs-Universität Freiburg, Germany
Real-Time Systems Lecture 15: The Universality Problem for TBA 2013-06-26 15 2013-06-26 main Dr. Bernd Westphal Albert-Ludwigs-Universität Freiburg, Germany Contents & Goals Last Lecture: Extended Timed
More informationValue Iteration. 1 Introduction. Krishnendu Chatterjee 1 and Thomas A. Henzinger 1,2
Value Iteration Krishnendu Chatterjee 1 and Thomas A. Henzinger 1,2 1 University of California, Berkeley 2 EPFL, Switzerland Abstract. We survey value iteration algorithms on graphs. Such algorithms can
More informationSynthesis weakness of standard approach. Rational Synthesis
1 Synthesis weakness of standard approach Rational Synthesis 3 Overview Introduction to formal verification Reactive systems Verification Synthesis Introduction to Formal Verification of Reactive Systems
More informationTrading Bounds for Memory in Games with Counters
Trading Bounds for Memory in Games with Counters Nathanaël Fijalkow 1,2, Florian Horn 1, Denis Kuperberg 2,3, and Micha l Skrzypczak 1,2 1 LIAFA, Université Paris 7 2 Institute of Informatics, University
More informationCHURCH SYNTHESIS PROBLEM and GAMES
p. 1/? CHURCH SYNTHESIS PROBLEM and GAMES Alexander Rabinovich Tel-Aviv University, Israel http://www.tau.ac.il/ rabinoa p. 2/? Plan of the Course 1. The Church problem - logic and automata. 2. Games -
More informationOptimal Bounds in Parametric LTL Games
Optimal Bounds in Parametric LTL Games Martin Zimmermann 1 Institute of Informatics University of Warsaw Warsaw, Poland Abstract Parameterized linear temporal logics are extensions of Linear Temporal Logic
More informationStrategy Logic. 1 Introduction. Krishnendu Chatterjee 1, Thomas A. Henzinger 1,2, and Nir Piterman 2
Strategy Logic Krishnendu Chatterjee 1, Thomas A. Henzinger 1,2, and Nir Piterman 2 1 University of California, Berkeley, USA 2 EPFL, Switzerland c krish@eecs.berkeley.edu, {tah,nir.piterman}@epfl.ch Abstract.
More informationProbabilistic model checking with PRISM
Probabilistic model checking with PRISM Marta Kwiatkowska Department of Computer Science, University of Oxford 4th SSFT, Menlo College, May 204 Part 2 Markov decision processes Overview (Part 2) Introduction
More informationStrategy Synthesis for Stochastic Games with Multiple Long-Run Objectives
Strategy Synthesis for Stochastic Games with Multiple Long-Run Objectives Nicolas Basset, Marta Kwiatkowska, Ufuk Topcu, and Clemens Wiltsche Department of Computer Science, University of Oxford, United
More informationIncompleteness Theorems, Large Cardinals, and Automata ov
Incompleteness Theorems, Large Cardinals, and Automata over Finite Words Equipe de Logique Mathématique Institut de Mathématiques de Jussieu - Paris Rive Gauche CNRS and Université Paris 7 TAMC 2017, Berne
More informationOn the complexity of infinite computations
On the complexity of infinite computations Damian Niwiński, Warsaw University joint work with Igor Walukiewicz and Filip Murlak Newton Institute, Cambridge, June 2006 1 Infinite computations Büchi (1960)
More informationProbabilistic verification and approximation schemes
Probabilistic verification and approximation schemes Richard Lassaigne Equipe de Logique mathématique, CNRS-Université Paris 7 Joint work with Sylvain Peyronnet (LRDE/EPITA & Equipe de Logique) Plan 1
More informationDeciding the Winner in Parity Games Is in UP co-up
c 1998 Elsevier Science B.V. Marcin Jurdziński, Deciding the Winner in Parity Games Is in UP co-up. Information Processing Letters 68(3): 119 124, November 1998. Personal use of this material for research/educational
More informationDepartment of Computer Science CS-RR-16-03
Department of Computer Science Compositional Strategy Synthesis for Stochastic ames with Multiple Objectives Nicolas Basset, Marta Kwiatkowska and Clemens Wiltsche CS-RR-6-03 Department of Computer Science,
More informationSolution Concepts and Algorithms for Infinite Multiplayer Games
Solution Concepts and Algorithms for Infinite Multiplayer Games Erich Grädel and Michael Ummels Mathematische Grundlagen der Informatik, RWTH Aachen, Germany E-Mail: {graedel,ummels}@logic.rwth-aachen.de
More informationQuantitative Solution of Omega-Regular Games
Quantitative Solution of Omega-Regular Games Luca de Alfaro Department of EECS UC Berkeley Berkeley, CA 9470, USA dealfaro@eecs.berkeley.edu Rupak Majumdar Department of EECS UC Berkeley Berkeley, CA 9470,
More informationOn Pure Nash Equilibria in Stochastic Games
On Pure Nash Equilibria in Stochastic Games Ankush Das, Shankara Narayanan Krishna, Lakshmi Manasa (B), Ashutosh Trivedi, and Dominik Wojtczak Department of Computer Science and Engineering, IIT Bombay,
More information