Infinite Games Sumit Nain Department of Computer Science Rice University 28 January 2013 Slides Credit: Barbara Jobstmann (CNRS/Verimag)
Motivation Abstract games are of fundamental importance in mathematics logic (particularly set theory and logic) economics computer science. In computer science, games are very well suited to model reactive, ongoing computation between a system and an adversarial environment. Sumit Nain (Rice University) Infinite Games 28 January 2013 2 / 32
Outline Define 2-player games formally. Basic concepts: winning conditions, strategies, determinacy. Show how to solve simple Reachability games. Overview of Buchi and Parity games with main results. Sumit Nain (Rice University) Infinite Games 28 January 2013 3 / 32
Two-Player Games Two-player games between Player 0 and 1 An infinite game G,φ consists of a game graph G and a winning condition φ. G defines the playground, in which the two players compete. φ defines which plays are won by Player 0. If a play does not satisfy φ, then Player 1 wins on this play. Sumit Nain (Rice University) Infinite Games 28 January 2013 4 / 32
Game Graphs A game graph or arena is a tuple G = S,S 0,T where: S is a finite set of states S 0 S is the set of Player-0 states S 1 = S \S 0 are the Player-1 states T S S is a transition relation (set of moves). We assume that each state has at least one successor. s 3 s 4 Player 0 Player 1 s 0 s 1 s 2 Sumit Nain (Rice University) Infinite Games 28 January 2013 5 / 32
Plays A play is an infinite sequence of states ρ = s 0 s 1 s 2 S ω such that for all i 0 s i,s i+1 T. It starts in s 0 and it is built up as follows: If s i S 0, then Player 0 chooses an edge starting in s i, otherwise Player 1 picks such an edge. Intuitively, a token is moved from state to state via edges: From S 0 -states Player 0 moves the token, from S 1 -states Player 1 moves the token. s 3 s 4 s 0 s 1 s 2 Sumit Nain (Rice University) Infinite Games 28 January 2013 6 / 32
Winning Condition Winning condition describes plays won by Player 0. A winning condition or objective φ is a subset of plays, i.e., φ S ω. We use logical conditions (e.g., LTL formulas) or automata theoretic acceptance conditions to describe φ. Example: alwayseventuallys for some state s S All plays that stay within a safe region F S are in φ. Games are named after their winning condition, e.g., Safety game, Reachability game, LTL game, Parity game,... Sumit Nain (Rice University) Infinite Games 28 January 2013 7 / 32
Types of Games Muller, Rabin, Streett, parity, Büchi objectives can be defined. Muller is the most general. Muller can be converted to parity (see Theorem 2.7). We will focus on parity and simpler conditions. Sumit Nain (Rice University) Infinite Games 28 January 2013 8 / 32
Types of Games Given a play ρ, we define Occ(ρ) = {s S i 0 : s i = s} Inf(ρ) = {s S i 0 j > i : s j = s} Given a set F S, Reachability Game φ = {ρ S ω Occ(ρ) F } Safety Game φ = {ρ S ω Occ(ρ) F} Büchi Game φ = {ρ S ω Inf(ρ) F } Co-Büchi Game φ = {ρ S ω Inf(ρ) F} Sumit Nain (Rice University) Infinite Games 28 January 2013 9 / 32
Types of Games Given a function p : S {0,1,...,d} Parity Game φ = {ρ S ω min s Inf(ρ) p(s) is even} 0 1 2 3 4 More general objectives can be converted into equivalent (larger) parity game. Sumit Nain (Rice University) Infinite Games 28 January 2013 10 / 32
Hierarchy Parity Büchi co-büchi Reachability Safety Sumit Nain (Rice University) Infinite Games 28 January 2013 11 / 32
Strategies A strategy for Player 0 from state s is a (partial) function f : S S 0 S specifying for any sequence of states s 0,s 1,...s k with s 0 = s and s k S 0 a successor state s j such that (s k,s j ) T. A play ρ = s 0 s 1... is compatible with strategy f if for all s i S 0 we have that s i+1 = f(s 0 s 1...s i ). (Definitions for Player 1 are analogous.) Given strategies f and g from s for Player 0 and 1, respectively. We denote by G f,g the (unique) play that is compatible with f and g. Sumit Nain (Rice University) Infinite Games 28 January 2013 12 / 32
Winning Strategies and Regions A strategy f for Player 0 from s is called a winning strategy if for all Player-1 strategies g from s, G f,g φ holds. Analogously, a Player-1 strategy g is winning if for all Player-0 strategies f, G f,g φ holds. Player 0 (resp. 1) wins from s if they have a winning strategy from s. Sumit Nain (Rice University) Infinite Games 28 January 2013 13 / 32
Winning Strategies and Regions A strategy f for Player 0 from s is called a winning strategy if for all Player-1 strategies g from s, G f,g φ holds. Analogously, a Player-1 strategy g is winning if for all Player-0 strategies f, G f,g φ holds. Player 0 (resp. 1) wins from s if they have a winning strategy from s. The winning regions of Player 0 and 1 are the sets W 0 = {s S Player 0 wins from s} W 1 = {s S Player 1 wins from s} Note each state s belongs at most to W 0 or W 1. Sumit Nain (Rice University) Infinite Games 28 January 2013 13 / 32
Questions About Games Solve a game (G,φ) with G = (S,S 0,T): Decide for each state s S if s W 0. If yes, construct a suitable winning strategy from s. Sumit Nain (Rice University) Infinite Games 28 January 2013 14 / 32
Questions About Games Solve a game (G,φ) with G = (S,S 0,T): Decide for each state s S if s W 0. If yes, construct a suitable winning strategy from s. Further interesting question: Optimize construction of winning strategy (e.g., time complexity) Optimize parameters of winning strategy (e.g., size of memory) Sumit Nain (Rice University) Infinite Games 28 January 2013 14 / 32
Example s 3 s 4 s 0 s 1 s 2 Safety game (G,F) with F = {s 0,s 1,s 3,s 4 }, i.e., Occ(ρ) F A winning strategy for Player 0 (from state s 0, s 3, and s 4 ): From s 0 choose s 3 and from s 4 choose s 3 A winning strategy for Player 1 (from state s 1 and s 2 ): From s 1 choose s 2, from s 2 choose s 4, and from s 3 choose s 4 Sumit Nain (Rice University) Infinite Games 28 January 2013 15 / 32
Example s 3 s 4 s 0 s 1 s 2 Safety game (G,F) with F = {s 0,s 1,s 3,s 4 }, i.e., Occ(ρ) F A winning strategy for Player 0 (from state s 0, s 3, and s 4 ): From s 0 choose s 3 and from s 4 choose s 3 A winning strategy for Player 1 (from state s 1 and s 2 ): From s 1 choose s 2, from s 2 choose s 4, and from s 3 choose s 4 W 0 = {s 0,s 3,s 4 }, W 1 = {s 1,s 2 } Sumit Nain (Rice University) Infinite Games 28 January 2013 15 / 32
Determinacy Recall: the winning regions are disjoint, i.e., W 0 W 1 = Question: Is every state winning for some player? A game (G,φ) with G = (S,S 0,E) is called determined if W 0 W 1 = S holds. Remarks: All automata theoretic games we discuss here are determined. There are games which are not determined (e.g., concurrent games: even/odd sum, paper-rock-scissors) Determinacy of infinite games is a deep question in logic! Sumit Nain (Rice University) Infinite Games 28 January 2013 16 / 32
Strategy Types In general, a strategy is a (partial) function f : S + S. Computable or recursive strategies: f is computable Finite-state strategies: f is computable with a finite-state automaton meaning that f has bounded information about the past (history). Memoryless or positional strategies: f only depends on the current state of the game (no knowledge about history of play) A memoryless strategy for Player 0 is a function f : S 0 S Sumit Nain (Rice University) Infinite Games 28 January 2013 17 / 32
Memoryless Determinacy In what situations do memoryless strategies suffice to win? Muller games both players need memory to win. Rabin games Player 0 does not need memory to win. Streett games Player 1 does not need memory to win. Parity games neither player needs memory to win. Parity game have memoryless determinacy! Every game that requires finite memory can be converted into a larger game with memoryless determinacy. Sumit Nain (Rice University) Infinite Games 28 January 2013 18 / 32
Finite State to Memoryless s 3 s 4 s 0 s 1 s 2 Objective: visit s 0 and s 4, i.e, {s 0,s 4 } Occ(ρ) Sumit Nain (Rice University) Infinite Games 28 January 2013 19 / 32
Finite State to Memoryless s 3 s 4 s 0 s 1 s 2 Objective: visit s 0 and s 4, i.e, {s 0,s 4 } Occ(ρ) Winning strategy for Player 0 from s 0, s 3 and s 4 : From s 0 to s 3, from s 4 to s 3, and from s 3 to s 0 on first visit, otherwise to s 4. s 0/s 3 s 1/ s 2/ s 4/s 3 s 3/s 0 m 0 m 1 s 0/s 3 s 1/ s 2/ s 3/s 4 s 4/s 3 Sumit Nain (Rice University) Infinite Games 28 January 2013 19 / 32
Extended Game s 0/s 3 s 1/ s 2/ s 4/s 3 s 3/s 0 m 0 m 1 s 0/s 3 s 1/ s 2/ s 3/s 4 s 4/s 3 Sumit Nain (Rice University) Infinite Games 28 January 2013 20 / 32
Extended Game s 0/s 3 s 1/ s 2/ s 4/s 3 s 3/s 0 m 0 m 1 s 0/s 3 s 1/ s 2/ s 3/s 4 s 4/s 3 m 0 s 3 s 4 s 0 s 1 s 2 Sumit Nain (Rice University) Infinite Games 28 January 2013 20 / 32
Extended Game s 0/s 3 s 1/ s 2/ s 4/s 3 s 3/s 0 m 0 m 1 s 0/s 3 s 1/ s 2/ s 3/s 4 s 4/s 3 m 0 s 3 s 4 m 1 s 3 s 4 s 0 s 1 s 2 s 0 s 1 s 2 Sumit Nain (Rice University) Infinite Games 28 January 2013 20 / 32
Extended Game s 0/s 3 s 1/ s 2/ s 4/s 3 s 3/s 0 m 0 m 1 s 0/s 3 s 1/ s 2/ s 3/s 4 s 4/s 3 m 0 s 3 s 4 m 1 s 3 s 4 s 0 s 1 s 2 s 0 s 1 s 2 Note: the strategy in the extended grame graph is memoryless. Sumit Nain (Rice University) Infinite Games 28 January 2013 20 / 32
Reachability Games Theorem In a reachability game (G,F) with G = (S,S 0,E) and F S, the winning regions W 0 and W 1 are computable. Both players have corresponding memoryless winning strategies. Proof. W 0 are all the states from which Player 0 can force a visit to F. Key Idea: define the i-attractor of Player 0, which are all the states from which Player 0 can force a visit to F in i steps. Then define the attractor of Player 0 as union of i-attractor for all i. Sumit Nain (Rice University) Infinite Games 28 January 2013 21 / 32
Reachability Games Theorem In a reachability game (G,F) with G = (S,S 0,E) and F S, the winning regions W 0 and W 1 are computable. Both players have corresponding memoryless winning strategies. Proof. W 0 are all the states from which Player 0 can force a visit to F. Key Idea: define the i-attractor of Player 0, which are all the states from which Player 0 can force a visit to F in i steps. Then define the attractor of Player 0 as union of i-attractor for all i. Claim (1): Attractor of Player 0 is computable Claim (2): W 0 = Attractor and W 1 = S \ Attractor Sumit Nain (Rice University) Infinite Games 28 January 2013 21 / 32
Forced Visit in Next Step ( Controlled Next ) Given F, compute the set of states CNext 0 (F) from which Player 0 can force a visit to F in the next step. CNext 0 (F) = {s S 0 s S : (s,s ) E s Attr i 0 (F)} {s S 1 s S : (s,s ) E s Attr i 0(F)} Sumit Nain (Rice University) Infinite Games 28 January 2013 22 / 32
Observations about CNext 0 (F) Player 0 has a memoryless winning strategy from CNext 0 (F) to reach F in the next step. Player 1 has a memoryless winning strategy from S \CNext 0 (F) to avoid F in the next step. S \CNext 0 (F) is a sub-game, i.e., every state has at least one successor. Sumit Nain (Rice University) Infinite Games 28 January 2013 23 / 32
Computing the Attractor Construction of i Attractor Attr i 0 (F): Attr 0 0(F) = F Attr i+1 0 (F) = Attr i 0 (F) CNext 0(Attr i 0 (F)) Attr 0 (F) = i=0 Attri 0(F)= S i=0 Attri 0(F) Sumit Nain (Rice University) Infinite Games 28 January 2013 24 / 32
Computing the Attractor Construction of i Attractor Attr i 0 (F): Attr 0 0(F) = F Attr i+1 0 (F) = Attr i 0 (F) CNext 0(Attr i 0 (F)) Attr 0 (F) = i=0 Attri 0(F)= S i=0 Attri 0(F) Since S is finite, there exists k S s.t. Attr k 0 (F) = Attrk+1 0 (F). The attractor of Player 0 can be computed as Attr 0 (F) = S i=0 Attr i 0 (F) Sumit Nain (Rice University) Infinite Games 28 January 2013 24 / 32
Computing the Attractor Construction of i Attractor Attr i 0(F): Attr 0 0 (F) = F Attr i+1 0 (F) = Attr i 0 (F) CNext 0(Attr i 0 (F)) Attr 0 (F) Claim (1): proved. = i=0 Attri 0 (F) = S i=0 Attri 0 (F) Sumit Nain (Rice University) Infinite Games 28 January 2013 25 / 32
Example s 5 s 6 s 7 s 3 s 4 s 0 s 1 s 2 Sumit Nain (Rice University) Infinite Games 28 January 2013 26 / 32
Example s 5 s 7 Attr 0 0 = {s 3,s 4 } s 6 s 3 s 4 s 0 s 1 s 2 Sumit Nain (Rice University) Infinite Games 28 January 2013 26 / 32
Example s 5 s 6 s 7 Attr 0 0 = {s 3,s 4 } Attr 1 0 = {s 0,s 3,s 4 } s 3 s 4 s 0 s 1 s 2 Sumit Nain (Rice University) Infinite Games 28 January 2013 26 / 32
Example s 5 s 6 s 7 Attr 0 0 = {s 3,s 4 } Attr 1 0 = {s 0,s 3,s 4 } Attr 2 0 = {s 0,s 3,s 4,s 7 } s 3 s 4 s 0 s 1 s 2 Sumit Nain (Rice University) Infinite Games 28 January 2013 26 / 32
Example s 5 s 6 s 7 Attr 0 0 = {s 3,s 4 } Attr 1 0 = {s 0,s 3,s 4 } Attr 2 0 = {s 0,s 3,s 4,s 7 } Attr 3 0 = {s 0,s 3,s 4,s 6,s 7 } s 3 s 4 Attr 4 0 = {s 0,s 3,s 4,s 5,s 6,s 7 } Attr 5 0 = Attr4 0 s 0 s 1 s 2 Sumit Nain (Rice University) Infinite Games 28 January 2013 26 / 32
Example s 5 s 6 s 7 Attr 0 0 = {s 3,s 4 } Attr 1 0 = {s 0,s 3,s 4 } Attr 2 0 = {s 0,s 3,s 4,s 7 } Attr 3 0 = {s 0,s 3,s 4,s 6,s 7 } s 3 s 4 Attr 4 0 = {s 0,s 3,s 4,s 5,s 6,s 7 } Attr 5 0 = Attr4 0 s 0 s 1 s 2 Sumit Nain (Rice University) Infinite Games 28 January 2013 26 / 32
Proving Claim 2 Claim (2): W 0 = Attr 0 (F) and W 1 = S \Attr 0 (F) Since we know that W 0 W 1 = holds, it is sufficient to prove Attr 0 (F) W 0 and S \Attr 0 (F) W 1. This follows easily from construction of Attr 0 (F) We can construct strategies, s.t. Player 0 reaches F in finitely many steps from Attr 0 (F) and Player 1 avoids F for infinitely many steps from S \Attr 0 (F). Sumit Nain (Rice University) Infinite Games 28 January 2013 27 / 32
Safety as Dual of Reachability Given a safety game (G,F) with G = (S,S 0,E), i.e., φ S = {ρ S ω Occ(ρ) F}, consider the reachability game (G,S \F), i.e., φ R = {ρ S ω Occ(ρ) (S \F) }. Then, S ω \φ R = {ρ S ω Occ(ρ) F}. Player 0 has a safety objective in (G,F). Player 1 has a reachability objective in (G,F). So, W 0 in the safety game (G,F) corresponds to W 1 in the reachability game (G,S \F). Sumit Nain (Rice University) Infinite Games 28 January 2013 28 / 32
Summary We know how to solve reachability and safety games by memoryless winning strategies. The strategies are Player 0: Decrease distance to F (called attractor strategy) Player 1: Stay outside of Attr 0 (F) In LTL, eventually(f) = reachability and always(f) = safety. Sumit Nain (Rice University) Infinite Games 28 January 2013 29 / 32
Exercise Given a reachability game (G,F) with G = (S,S 0,E) and F Q, give an algorithm that computes the attractor(f) of Player 0 in time O( E ). Sumit Nain (Rice University) Infinite Games 28 January 2013 30 / 32
Exercise Given a reachability game (G,F) with G = (S,S 0,E) and F Q, give an algorithm that computes the attractor(f) of Player 0 in time O( E ). Solution: Preprocessing: Compute for every state s S 1 outdegree out(s) Set n(s) := out(s) for each s S 1 To breadth-first search backwards from F with the following conventions mark all s F mark s S 0 if reached from marked state mark s S 1 if n(s) = 0, other set n(s) := n(s) 1. The marked vertices are the ones of Attr 0 (F). Sumit Nain (Rice University) Infinite Games 28 January 2013 30 / 32
Büchi Games φ = {ρ S ω Inf(ρ) F } Winning condition: The play must visit F infinitely often. Büchi games are determined. Memoryless strategies suffice for Büchi games. Can be solved in O(nm), where n is number of vertices and m is number of edges in game graph. co-büchi games are the dual of Büchi games. LTL equivalent for Büchi: always eventually F LTL equivalent for co-büchi: eventually always F Sumit Nain (Rice University) Infinite Games 28 January 2013 31 / 32
Parity Games Given a priority function p : S {0,1,...,d}, we have φ = {ρ S ω min s Inf(ρ) p(s) is even} Winning condition: The lowest priority visited infinitely often must be even. Parity games are determined. Memoryless strategies suffice for parity games. Solving them is in NP and conp. Is solving parity games in P? Major open problem in computer science. LTL games can be transformed into equivalent but larger parity games. Sumit Nain (Rice University) Infinite Games 28 January 2013 32 / 32