Multiagent Systems and Games

Transcription

1 Multiagent Systems and Games Rodica Condurache Lecture 5 Lecture 5 Multiagent Systems and Games 1 / 31

2 Multiagent Systems Definition A Multiagent System is a tuple M = AP, Ag, (Act i ) i Ag, V, v 0, τ, E where Ag = {0, 1,..., k} is the set of agents Act i is the set of actions of Agent i Ag V is the set of states in the system v 0 is the initial state τ : V 2 AP is the labeling function E : V Act 0 Act 1... Act k V is the transition function (I,I ), (G, ) (I, ) (I,R) init g r v 0 (G, ) v 1 Lecture 5 Multiagent Systems and Games 2 / 31

3 Concurrent Games Definition A concurrent game is a tuple G = M, (Θ i ) i Ag where M = AP, Ag, (Act i ) i Ag, V, v 0, τ, E is a multiagent system Θ i V ω is the objective of Agent i Ag A play in G is a sequence π = v 0v 1v 2... such that n 0, (a0, n a1, n..., ak) n s.t. (v n, (a0, n a1, n..., ak), n v n+1) E A history is a finite prefix of a play. Agent i wins (satisfies its objective) along a play π if π Θ i Lecture 5 Multiagent Systems and Games 3 / 31

4 Strategies Given a multiagent system M = AP, Ag, (Act i ) i Ag, V, v 0, τ, E, A strategy for Agent i is a is a mapping σ i : V (Act i V ) Act i Agent i sees only the states and his actions! A strategy profile is a tuple σ = (σ 0,..., σ k ) of strategies, one for each agent A play π = v 0v 1v 2... is compatible with a strategy σ i of Agent i if (a0, 0 a1, 0..., ak) 0 (a0, 1 a1, 1..., ak) 1... s.t. n 0, v n+1 = E(v n, (a0 n, an 1,..., an k )) and ai n = σ i (v 0 ai 0v 1ai 1v 2...v n) out(σ i ) is the set of plays compatible with σ i out( σ) = i Ag out(σ i) is the outcome of the profile σ - only one play! Lecture 5 Multiagent Systems and Games 4 / 31

5 Strategies Given a concurrent game G = M, (Θ i ) i Ag, A strategy σ i of Agent i is winning from v V if for any play π that starts in v and is compatible with σ i, π Θ i. The winning region if Agent i is the set W i (G) V of all vertices v s.t. Agent i has a winning strategy from v in G. Lecture 5 Multiagent Systems and Games 5 / 31

6 Particular Winning Conditions Let π = v 0v 1v 2... V ω. visit(π) = {v v l = v for some l 0} and inf(v 0v 1v 2...) = {v v l = v for infinitely many l 0} A winning condition is a set Θ i V ω. Particular cases: Reachability: For a set R V, Reach(R) = {π V ω visit(π) R } Safety: For a set S V, Safe(S) = {π V ω visit(π) S} Büchi: For a set T V, Buchi(T ) = {π V ω inf(π) T } cobüchi: For a set T V, cobuchi(t ) = {π V ω inf(π) T = } Parity: For a priority function p : V N, Parity(p) = {π V ω min{p(v) v inf(π)} is even} LTL: For an LTL formula ϕ, LTL(ϕ) = {π V ω trace(π), 0 = ϕ} Lecture 5 Multiagent Systems and Games 6 / 31

7 Two-players Zero-sum Turn-based Games Play only two players in turns and choose the next state Zero-sum : when one player wins, the other loses start v 0 v 2 v 1 v 3 v R A Two-players Zero-sum Turn-based Games is a tuple G = Ag = {A, B}, V = V A V B, v 0, E, Θ A where A is the protagonist and B is the adversary V A : the states controlled by Player A V B : the states controlled by Player B E V V Θ A : the objective of Player A Player B wants to avoid Θ A A strategy for Player A is a mapping σ A : V V A V Lecture 5 Multiagent Systems and Games 7 / 31

8 Games: Fundamental Questions Determinacy: Given a class Γ of games, is it the case that for any game G Γ, either Player A or Player B has a winning strategy? I.e., Does there exists a partition V = W A (G) W B (G)? Decision: Given a game G, does Player A (or Player B) have a winning strategy? I.e., Is the case that v 0 W A (G) (or v 0 W B (G))? Construction: Given a game G, how to construct a winning strategy? Complexity: Can a winning strategy be implemented algorithmically? What are the needed resources? Lecture 5 Multiagent Systems and Games 8 / 31

9 Turn-based Zero-sum Reachability Games Reachability Game: G = V = V A V B, v 0, E, reach(r) for R V v 1 v 7 v 0 v 2 v 4 v 5 v 3 v 6 Winning condition for Player A : visit(π) R Solution complexity: linear time in E Algorithm: Dual game: attractors Safety Lecture 5 Multiagent Systems and Games 9 / 31

10 Turn-based Zero-sum Reachability Games : Atractors Let E(v) = {w V (v, w) E} Inductively construct the set Attr A i (R) from which Player A can force to reach R in at most i steps Attr A 0 (R) :=R Attr A n+1(r) :=Attr A n (R) {v V A E(v) Attr A n (R) } {v V B E(v) Attr A n (R)} Attr A n (R) increases until reaching the fixed point Attr A n+1(r) = Attr A n (R). Let Attr A (R) be the fixed point and call it attractor of R for Player A Lecture 5 Multiagent Systems and Games 10 / 31

11 Turn-based Zero-sum Reachability Games : Attractor Example Example v 1 v 0 v 2 v 7 v 4 v 5 Player A : Player B : R = {v 6} v 3 v 6 Attr A 0 ({v 6}) = {v 6} Lecture 5 Multiagent Systems and Games 11 / 31

12 Turn-based Zero-sum Reachability Games : Attractor Example Example v 1 v 0 v 2 v 7 v 4 v 5 Player A : Player B : R = {v 6} v 3 v 6 Attr A 0 ({v 6}) = {v 6} Attr A 1 ({v 6}) = {v 6, v 5} Lecture 5 Multiagent Systems and Games 11 / 31

13 Turn-based Zero-sum Reachability Games : Attractor Example Example v 1 v 0 v 2 v 7 v 4 v 5 Player A : Player B : R = {v 6} v 3 v 6 Attr A 0 ({v 6}) = {v 6} Attr A 1 ({v 6}) = {v 6, v 5} Attr A 2 ({v 6}) = {v 6, v 5, v 4} Lecture 5 Multiagent Systems and Games 11 / 31

14 Turn-based Zero-sum Reachability Games : Attractor Example Example v 1 v 0 v 2 v 7 v 4 v 5 Player A : Player B : R = {v 6} v 3 v 6 Attr0 A ({v 6}) = {v 6} Attr1 A ({v 6}) = {v 6, v 5} Attr2 A ({v 6}) = {v 6, v 5, v 4} Attr3 A ({v 6}) = {v 6, v 5, v 4, v 3} Lecture 5 Multiagent Systems and Games 11 / 31

15 Turn-based Zero-sum Reachability Games : Attractor Example Example v 1 v 0 v 2 v 7 v 4 v 5 Player A : Player B : R = {v 6} v 3 v 6 Attr0 A ({v 6}) = {v 6} Attr1 A ({v 6}) = {v 6, v 5} Attr2 A ({v 6}) = {v 6, v 5, v 4} Attr3 A ({v 6}) = {v 6, v 5, v 4, v 3} Attr4 A ({v 6}) = {v 6, v 5, v 4, v 3, v 0} Lecture 5 Multiagent Systems and Games 11 / 31

16 Turn-based Zero-sum Reachability Games : Attractor Example Example v 1 v 0 v 2 v 7 v 4 v 5 Player A : Player B : R = {v 6} v 3 v 6 Attr0 A ({v 6}) = {v 6} Attr1 A ({v 6}) = {v 6, v 5} Attr2 A ({v 6}) = {v 6, v 5, v 4} Attr3 A ({v 6}) = {v 6, v 5, v 4, v 3} Attr4 A ({v 6}) = {v 6, v 5, v 4, v 3, v 0} Attr5 A ({v 6}) = {v 6, v 5, v 4, v 3, v 0, v 2} Lecture 5 Multiagent Systems and Games 11 / 31

17 Turn-based Zero-sum Reachability Games : Attractor Example Example v 1 v 0 v 2 v 7 v 4 v 5 Player A : Player B : R = {v 6} v 3 v 6 Attr0 A ({v 6}) = {v 6} Attr1 A ({v 6}) = {v 6, v 5} Attr2 A ({v 6}) = {v 6, v 5, v 4} Attr3 A ({v 6}) = {v 6, v 5, v 4, v 3} Attr4 A ({v 6}) = {v 6, v 5, v 4, v 3, v 0} Attr5 A ({v 6}) = {v 6, v 5, v 4, v 3, v 0, v 2} Attr6 A ({v 6}) = {v 6, v 5, v 4, v 3, v 0, v 2} = Attr A ({v 6}) Lecture 5 Multiagent Systems and Games 11 / 31

18 Turn-based Zero-sum Reachability Games: Winning Regions and Strategies Given a reachability game G where Player A targets R V, Let rank : V N { } s.t. { min{n v Attrn A (R)} rank(v) = for every v Attr A, either v R v V A \ R and w E(v) s.t. rank(w) < rank(v), or v V B \ R and w E(v) s.t. rank(w) < rank(v) if v Attr A (R) otherwise for every v Attr A, either v V A and w E(v), holds rank(w) =, or v V B and w E(v), holds rank(w) =, or Lecture 5 Multiagent Systems and Games 12 / 31

19 Turn-based Zero-sum Reachability Games: Winning Regions and Strategies Theorem In a reachability game G where Player A targets R V, the winning region of Player A is W A (G) = Attr A (R) Proof. Let v 0 Attr A (R). We construct a strategy σ A : V V A V for Player A s.t. v 0v 1...v l V V A, σ A (v 0v 1...v l ) = w s.t. w E(v l ) and rank(w) < rank(v l ) Then, for any infinite play v 0v 1v 2... out(σ A ), holds rank(v 0) > rank(v 1) > rank(v 2) >... Therefore, we reach rank(v l ) = 0 which means v l R σ A is winning for Player A Lecture 5 Multiagent Systems and Games 13 / 31

20 Turn-based Zero-sum Reachability Games: Winning Regions and Strategies Theorem In a reachability game G where Player A targets R V, the winning region of Player B is W B (G) = V \ Attr A (R) Proof. Let v 0 V \ Attr A (R). We construct a strategy σ B : V V B V for Player B s.t. v 0v 1...v l V V B, σ A (v 0v 1...v l ) = w s.t. w E(v l ) and rank(w) = Then, for any infinite play v 0v 1v 2... out(σ A ), holds rank(v l ) = l 0 Therefore, we never reach rank(v l ) = 0 v l R l 0 σ B is winning for Player B Lecture 5 Multiagent Systems and Games 14 / 31

21 Turn-Based Zero-sum Reachability Games Theorem There is a linear-time algorithm to solve Reachability games. Lecture 5 Multiagent Systems and Games 15 / 31

22 Concurrent Multiplayer Reachability Games The Game: G = AP, Ag, (Act i ) i Ag, V, v 0, τ, E, (reach(r i )) i Ag Decision Problem: Given the concurrent game G, does Player i have a winning strategy? Recall: Each Player i has a target set R i! Let Succ(v, a i ) = {w V (a 0,..., a i 1, a i+1,..., a k ) s.t. w = E(v, ā)} Compute the attractor using the formula: Attr0(R i i ) :=R i Attrn+1(R i i ) :=Attrn(R i i ) {v V a i Act i s.t. Succ(v, a i ) Attrn(R)} i Let Attr i (R i ) be the fixed point where Attrn+1(R i i ) := Attrn(R i i ) The winning region of Player i is W i (G) = Attr i (R i ) Lecture 5 Multiagent Systems and Games 16 / 31

23 Turn-based Zero-Sum Safety Game Safety Game: G = V = V A V B, v 0, E, safe(s) for S V v 1 v 7 v 0 v 2 v 4 v 5 v 3 v 6 Winning condition for Player A : visit(π) S Solution complexity: linear time in E Algorithm: Dual game: duailze + attractors Reachability Lecture 5 Multiagent Systems and Games 17 / 31

24 Safety Games Zero-sum Safety games are dual to reachability games Player A has a winning strategy in the safety game G = V = V A V B, v 0, E, Θ A = S iff Player A has a winning strategy in the reachability game G = V = V A V B, v 0, E, Θ B = V \ S In concurrent multiplayer games, we compute the winning region using the sets: X i 0 :=S X i n+1 :=X i n \ {v a i Act i, Succ(v, a i ) (V \ X i n) } From X i n, Player i has a strategy to stay in S at least for n rounds X i n decreases until reaching a fixed point X i n+1 = X i n Then, The winning region of Player i is W i (G) = X i n s.t. X i n+1 = X i n Lecture 5 Multiagent Systems and Games 18 / 31

25 Safety Games Zero-sum Safety games are dual to reachability games Player A has a winning strategy in the safety game G = V = V A V B, v 0, E, Θ A = S iff Player A has a winning strategy in the reachability game G = V = V A V B, v 0, E, Θ B = V \ S In concurrent multiplayer games, we compute the winning region using the sets: X i 0 :=S X i n+1 :=X i n \ {v a i Act i, Succ(v, a i ) (V \ X i n) } From X i n, Player i has a strategy to stay in S at least for n rounds X i n decreases until reaching a fixed point X i n+1 = X i n Then, The winning region of Player i is W i (G) = X i n s.t. X i n+1 = X i n Aletrnative: The opponents of Player i into a safety game has reachability objective V \ S i! Consider the opponents of Player i as a single Player B that plays tuples of actions. Compute Attr B (V \ S i ) Then, W i (G) = V \ Attr B (V \ S i ) Lecture 5 Multiagent Systems and Games 18 / 31

26 Turn-based Zero-Sum Büchi Game Büchi Game: G = V = V A V B, v 0, E, Büchi(T ) for T V v 0 v 1 v 2 v 3 v 4 v 5 v 6 v 7 v 8 Winning condition for Player A : Solution complexity: Algorithm: Dual game: inf(π) T Ptime iterated attractors co-büchi Lecture 5 Multiagent Systems and Games 19 / 31

27 Turn-based Zero-Sum Büchi Game : Attractors Let X V. Attr0 A (X ) :={v V A E(v) X } {v V B E(v) X } Attrn+1(X A ) :=Attrn A (X ) {v V A E(v) Attrn A (X ) } {v V B E(v) Attrn A (X )} Attrn A (X ) increases until Attrn+1(X A ) = Attrn A (X ) Let Attr A (X ) be the fixed point called attractor in at least one step Build Attr A (X ) from which Player A has a strategy to reach X in at least one move Lecture 5 Multiagent Systems and Games 20 / 31

28 Turn-based Zero-sum Büchi Games : Attractor Example Example v 0 v 1 v 2 v 3 v 4 v 5 Player A : Player B : T = {v 0, v 6} v 6 v 7 v 8 Attr A 0 ({v 0, v 6}) = {v 3, v 7} Note the difference from Reachability!!! Lecture 5 Multiagent Systems and Games 21 / 31

29 Turn-based Zero-sum Büchi Games : Attractor Example Example v 0 v 1 v 2 v 3 v 4 v 5 Player A : Player B : T = {v 0, v 6} v 6 v 7 v 8 Attr A 0 ({v 0, v 6}) = {v 3, v 7} Attr A 1 ({v 0, v 6}) = {v 3, v 7, v 6, v 8} Note the difference from Reachability!!! Lecture 5 Multiagent Systems and Games 21 / 31

30 Turn-based Zero-sum Büchi Games : Attractor Example Example v 0 v 1 v 2 v 3 v 4 v 5 Player A : Player B : T = {v 0, v 6} v 6 v 7 v 8 Attr0 A ({v 0, v 6}) = {v 3, v 7} Note the difference from Reachability!!! Attr1 A ({v 0, v 6}) = {v 3, v 7, v 6, v 8} Attr2 A ({v 0, v 6}) = {v 3, v 7, v 6, v 8, v 5} Lecture 5 Multiagent Systems and Games 21 / 31

31 Turn-based Zero-sum Büchi Games : Attractor Example Example v 0 v 1 v 2 v 3 v 4 v 5 Player A : Player B : T = {v 0, v 6} v 6 v 7 v 8 Attr0 A ({v 0, v 6}) = {v 3, v 7} Note the difference from Reachability!!! Attr1 A ({v 0, v 6}) = {v 3, v 7, v 6, v 8} Attr2 A ({v 0, v 6}) = {v 3, v 7, v 6, v 8, v 5} Attr3 A ({v 0, v 6}) = {v 3, v 7, v 6, v 8, v 5} Lecture 5 Multiagent Systems and Games 21 / 31

32 Turn-based Zero-sum Büchi Games : Iterated Attractor Z n = { T if n = 1 Attr A (Z n 1) T if n > 1 Z n is the set of final states from which Player A can force at least n visits of T The sequence Z 1, Z 2, Z 3... is decreasing Base case: T = Z 1 Z 2 = Attr A (Z 1 ) T Inductive step: use the monotonicity of Attr A () Z n Z n+1 Attr A (Z n) Attr A (Z n+1 ) Z n+1 = Attr A (Z n) T Attr A (Z n+1 ) T = Z n+2 Let Z be the limit of the sequence Z 1, Z 2, Z 3... : Z = n 1 Z n Lecture 5 Multiagent Systems and Games 22 / 31

33 Turn-based Zero-sum Büchi Games : Iterated Attractor Example Example v 0 v 1 v 2 v 3 v 4 v 5 Player A : Player B : T = {v 0, v 6} v 6 v 7 v 8 Z 1 = {v 0, v 6} Lecture 5 Multiagent Systems and Games 23 / 31

34 Turn-based Zero-sum Büchi Games : Iterated Attractor Example Example v 0 v 1 v 2 v 3 v 4 v 5 Player A : Player B : T = {v 0, v 6} v 6 v 7 v 8 Z 1 = {v 0, v 6} Attr A (Z 1) = {v 3, v 7, v 6, v 8, v 5} Lecture 5 Multiagent Systems and Games 23 / 31

35 Turn-based Zero-sum Büchi Games : Iterated Attractor Example Example v 0 v 1 v 2 v 3 v 4 v 5 Player A : Player B : T = {v 0, v 6} v 6 v 7 v 8 Z 1 = {v 0, v 6} Attr A (Z 1) = {v 3, v 7, v 6, v 8, v 5} Z 2 = {v 6} Lecture 5 Multiagent Systems and Games 23 / 31

36 Turn-based Zero-sum Büchi Games : Iterated Attractor Example Example v 0 v 1 v 2 v 3 v 4 v 5 Player A : Player B : T = {v 0, v 6} v 6 v 7 v 8 Z 1 = {v 0, v 6} Attr A (Z 1) = {v 3, v 7, v 6, v 8, v 5} Z 2 = {v 6} Attr A (Z 2) = {v 3, v 7, v 6, v 8, v 5} Lecture 5 Multiagent Systems and Games 23 / 31

37 Turn-based Zero-sum Büchi Games : Iterated Attractor Example Example v 0 v 1 v 2 v 3 v 4 v 5 Player A : Player B : T = {v 0, v 6} v 6 v 7 v 8 Z 1 = {v 0, v 6} Attr A (Z 1) = {v 3, v 7, v 6, v 8, v 5} Z 2 = {v 6} Attr A (Z 2) = {v 3, v 7, v 6, v 8, v 5} Z 3 = {v 6} = Z 2 Lecture 5 Multiagent Systems and Games 23 / 31

38 Turn-based Zero-sum Büchi Games : Winning Regions and Strategies Theorem In a Büchi game G where Player A targets T V infinitely often, the winning region of Player A is W A (G) = Attr A (Z ) Proof. From Attr A (Z ) \ Z, play σ 1 A to reach Z (similar to Reachability) Once in Z = Attr A (Z ) F Player A has a strategy σ 2 A from Z to go back to Z in at least one step Lecture 5 Multiagent Systems and Games 24 / 31

39 Turn-based Zero-sum Büchi Games : Winning Regions and Strategies Theorem In a Büchi game G where Player A targets T V infinitely often, the winning region of Player B is Proof. W B (G) = V \ Attr A (Z ) Let v V \ Attr A (Z ) Z n s.t. v Attr A (Z n) \ Attr A (Z n+1). Put rk(v) = n ( rk(v) = 1 if v Z 1) If v V B, v V \ Attr A (Z n+1) v E(v) s.t. v Z n+1 = Attr A (Z n) T if v Attr A (Z n) but v T rk(v ) < n with n 1 else v T Player B plays v! if v V A, v V \ Attr A (Z n+1) v E(v) s.t. v Z n+1 = Attr A (Z n) T Player A cannot avoid v s.t. rk(v ) < n or v T In In the resulting play, we cannot have infinitely often v E(v) s.t. rk(v ) < rk(v) There is a position m s.t. j m, v j T Player B wins Lecture 5 Multiagent Systems and Games 25 / 31

40 Turn-based Zero-sum Büchi Games : Winning Regions and Strategies - Example Example v 0 v 1 v 2 v 3 v 4 v 5 Player A : Player B : T = {v 0, v 6} v 6 v 7 v 8 Z = {v 6} W A = Attr A (Z ) = {v 3, v 7, v 6, v 8, v 5} W B = V \ W A = {v 0, v 1, v 2, v 4} Lecture 5 Multiagent Systems and Games 26 / 31

41 Concurrent Multiplayer Büchi Games The Game: G = AP, Ag, (Act i ) i Ag, V, v 0, τ, E, (Büchi(T i )) i Ag Decision Problem: Given the concurrent game G, does Player i have a winning strategy? Similar algorithm as in the turn-based zero-sum setting Player i is the protagonist (Player A in the turn-based setting) The other players form the antagonist (Player B) Adapt the Attractor formula: Attr i 0(X ) :={v V a i Act i s.t. Succ(v, a i ) X } Attr i n+1(x ) :=Attr A n (X ) {v V a i Act i s.t. Succ(v, a i ) Attr i n(x )} Then, the sequence Zn i is defined by { Zn i T i if n = 1 = Attr i (Zn 1) i T i if n > 1 Lecture 5 Multiagent Systems and Games 27 / 31

42 Turn-based Zero-Sum co-büchi Game co-büchi Game: G = V = V A V B, v 0, E, co-büchi(t ) for T V v 0 v 1 v 2 v 3 v 4 v 5 Winning condition for Player A : Solution complexity: Algorithm: Dual game: inf(π) T = Ptime dualize + itterated attractors Büchi Lecture 5 Multiagent Systems and Games 28 / 31

43 co-büchi Game In Zero-sum co-büchi games are dual to Büchi games Player A has a winning strategy in the co-büchi game G = V = V A V B, v 0, E, Θ A = co-büchi(t ) iff Player A has a winning strategy in the Büchi game G = V = V A V B, v 0, E, Θ B = Büchi(T ) Player B is the protagonist! In concurrent multiplayer game: Use duality The opponents of Player i have the objective Büchi(T i ) Consider the opponents as a single player A that plays tuples of actions Compute the winning region W A of Player A in the Büchi game Then, W i (G) = V \ W A Lecture 5 Multiagent Systems and Games 29 / 31

44 co-büchi Games : Winning Regions and Strategies - Example Example v 0 v 1 v 2 v 3 v 4 v 5 Player A : Player B : co-büchi : T = {v 5} Player B ( ) has Büchi objective T = {v 5} Lecture 5 Multiagent Systems and Games 30 / 31

45 co-büchi Games : Winning Regions and Strategies - Example Example v 0 v 1 v 2 v 3 v 4 v 5 Player A : Player B : co-büchi : T = {v 5} Player B ( ) has Büchi objective T = {v 5} Solve the Büchi game with B protagonist: Z 1 = {v 5} Attr B (Z 1) = {v 2, v 4, v 5} Z 2 = {v 5} = Z W B = Attr B (Z 1) = {v 2, v 4, v 5} Lecture 5 Multiagent Systems and Games 30 / 31

46 co-büchi Games : Winning Regions and Strategies - Example Example v 0 v 1 v 2 v 3 v 4 v 5 Player A : Player B : co-büchi : T = {v 5} Player B ( ) has Büchi objective T = {v 5} Solve the Büchi game with B protagonist: Z 1 = {v 5} Attr B (Z 1) = {v 2, v 4, v 5} Z 2 = {v 5} = Z W B = Attr B (Z 1) = {v 2, v 4, v 5} In co-buchi game: W A = V \ W B = {v 0, v 1, v 3} Lecture 5 Multiagent Systems and Games 30 / 31

47 Bibliography automata-games-verification-12/downloads/notes10.pdf Erich Grädel et al: Automata, Logics, and Infinite Games - A Guide to Current Research(available online) Lecture 5 Multiagent Systems and Games 31 / 31