Valentin Goranko Stockholm University. ESSLLI 2018 August 6-10, of 33

Transcription

1 ESSLLI 2018 course Logics for Epistemic and Strategic Reasoning in Multi-Agent Systems Lecture 4: Logics for temporal strategic reasoning with complete information Valentin Goranko Stockholm University ESSLLI 2018 August 6-10, 2018 Sofia University, Bulgaria 1 of 33

2 Outline of the lecture The logics of temporal coalitional strategic reasoning ATL/ATL* Logical decision problems for ATL: model checking. Logical decision problems for ATL: satisfiability and model building. Nesting of strategic operators Revocability of strategies. Strategy contexts. 2 of 33

3 The multi-agent logic of strategic reasoning ATL(*) Alternating-time Temporal Logic ATL(*): introduced by Alur, Henzinger, and Kupferman, during Extends propositional logic PL with: Temporal operators: X (next time), G (forever), U (until) Coalitional strategic path operators: A for any group of agents A. Formulae of the full logic ATL : ϕ := p ϕ ϕ 1 ϕ 2 A ϕ X ϕ Gϕ ϕ 1 U ϕ 2 Formulae of the fragment ATL: ϕ := p ϕ ϕ 1 ϕ 2 A X ϕ A Gϕ A ϕ 1 U ϕ 2 Remark: the computation tree logic CTL(*) can be regarded as a fragment of ATL(*), where: - the existential path quantifier E is identified with A, - the universal path quantifier A is identified with. One agent suffices. 3 of 33

4 Semantics of ATL intuitively A ϕ: The coalition A has a joint strategy to guarantee the satisfaction of the goal ϕ on every play enabled by that strategy. In particular: A X ϕ: The coalition A has a joint action that ensures an outcome (state) satisfying ϕ (This is exactly [A] ϕ in CL.) A Gϕ: The coalition A has a joint strategy to maintain forever outcomes satisfying ϕ, A ψ U ϕ: The coalition A has a joint strategy to eventually reach an outcome satisfying ϕ, while meanwhile maintaining the truth of ψ. Definable operators: A Fϕ := A U ϕ, meaning The coalition A has a joint strategy to eventually reach an outcome satisfying ϕ. [[A]] ϕ := A ϕ, meaning: The coalition A cannot prevent the satisfaction of ϕ. 4 of 33

5 Expressing properties in ATL: some examples Agent F Safe [[Environment]] F Safe If the agent has a strategy to eventually get the system into a safe state, then the environment cannot prevent the agent from reaching a safe state. ( I G Honest I F Rich) I (Honest U Rich) If I have a strategy to remain honest forever and I have a strategy to eventually become rich, then I have a strategy to remain honest until I become rich. ( Agent1 G Safe Agent2 F Goal) Agent1, Agent2 (Safe U Goal) If Agent 1 has a strategy to keep the system in safe states forever and Agent 2 has a strategy to eventually reach Goal, then Agent 1 and Agent 2 have a joint strategy to keep the system in safe states until Goal is reached. A F B G ϕ The coalition A has a joint strategy to eventually ensure that the coalition B cannot prevent ϕ from happening eventually. 5 of 33

6 Plays, Computations and Strategies Given a CGM M = A, St, Act, act, out, Prop, L and a state s St: A state s in M is a successor of the state s if there is an action profile (σ 1,..., σ n ) such that out(s; σ 1,..., σ n ) = s. The set of successors of s: succ(s). A s-play in M: an infinite sequence s 0, s 1,..., such that s 0 = s and s i+1 succ(s i ) for every i 0. A (perfect recall) strategy in M for an agent i N: a mapping f i : St + Act that assigns to every sequence of states s 0,..., s n an action f i ( s 0,..., s n ) act(s n, i). If the strategy only depends on the current state, it is a no recall (memoryless, positional) strategy. A joint strategy in M for a set of agents A: a family of strategies F A = {f i } i A. A joint strategy F A enables a play λ if that play can occur as a result of the players in A following their strategies in F A. 6 of 33

7 ATL semantics: formally Truth of a formula ψ at a state s of a CGM M: M, s ψ Defined by structural induction on formulae, via the clauses: M, s A X ϕ iff there exists a joint strategy F A = {f i } i A such that M, s 1 ϕ for every s-play s, s 1,... enabled by F A. M, s A Gϕ iff there exists a joint strategy F A = {f i } i A such that M, s i ϕ for every s-play s, s 1,... enabled by F A and i 0. M, s A ϕ U ψ iff there exists a joint strategy F A = {f i } i A such that for every s-play s, s 1,... enabled by F A there is i 0 for which M, s i ψ and for all j such that 0 j < i, M, s j ϕ. NB: For the semantics of ATL memoryless strategies suffice. 7 of 33

8 Deciding the truth of ATL formulae in a CGM: examples (wait,wait) (push,push) (push,wait) s 0 (push,wait) (wait,wait) (push,push) (wait, (wait, s 2 push) push) s 1 (wait,push) (wait,wait) (push,push) (wait,wait) (push,wait) (push,park) (park,push) (wait,park) (park,wait) s 4 s 3 (wait,wait) M, s 0? = Yin F pos3 No M, s 0? = Yin G (pos3 pos 4 ) Yes M, s 0? = Yin, Yang (( pos1 ) U pos 4 ) Yes. M, s 0? = Yin F Yang F pos4 No M, s 0? = Yin, Yang G( pos1 Yin, Yang X pos 1 ) Yes. 8 of 33

9 Deciding the truth of ATL formulae: exercises Two agents: 1 and 2. Two types of actions: a, b. (b,a) s 2 {p,q} (a,a) (a,b) s 3 {q} (a,a) (b,b) (b,b) (b,a) (b,b) (a,b) (b,a) s 1 {p} (a,a) (a,b) (a,a)(a,b)(b,b) s 4 {} (b,a) M, s 1? = 1 Fq 2 G q No M, s1? = 1 Gp 2 Gp No?? M, s 3 = F 2 X p Yes M, s2 = 1 G 1, 2 ( qup) Yes 9 of 33

10 (Addendum) Detectives and fugitives puzzles A fugitive is trying to run away from a detective, who is trying to catch the fugitive. There are N caves arranged in a line and the fugitive is hiding in one of them. Every midnight the fugitive must move from the current cave to one of the neighbouring caves. Every day the detective inspects one of the caves, of his choice. The detective can only catch the fugitive if he is in the inspected cave. 1. Model the scenario for a given N with a CGM M N. (Hint: assume that the fugitive and the detective are moving from one cave to another simultaneously.) 2. Does the detective have a strategy to eventually catch the fugitive, no matter where he is initially, if: N = 4? N = 5? N > 5? In each case: if not, why? If yes, what is the least number of moves within which the strategy is guaranteed to succeed? 3. Same questions, if every time the fugitive has the choice to either remain in the same cave or move to a neighbouring cave. 4. Same questions, if the caves are not arranged linearly, but are at the vertices of any neighbourhood graph. 5. Same questions, if two detectives work as a team. 10 of 33

11 Extending the semantics of ATL* Two types of formulae in ATL*: State formulae ϕ ::= p ϕ ϕ ϕ A γ, where A A and p Prop. Path formulae: γ ::= ϕ γ γ γ X γ Gγ γ U γ The semantics of state formulae: as in ATL. The semantics of path formulae: defined on paths (plays), as in LTL. ATL* is much more expressive and has more complex semantics. Strategies generally need memory. Example: a (Fp Fq). Nesting of strategic operators causes higher complexity and also some problems with the semantics. 11 of 33

12 Validity and satisfiability in ATL An ATL formula φ is: (logically) valid if M, s φ for every CGM M and a state s M. satisfiable if M, s φ for some CGM M and a state s M. 12 of 33

13 Axiomatizing the validities of ATL: local axioms Recall Pauly s axiomatization of CL, extending the classical propositional logic with the following axioms and rule: A-Maximality: X ϕ A X ϕ Safety: C X Liveness: C X Superadditivity: for any C 1, C 2 A such that C 1 C 2 = : ( C 1 X ϕ 1 C 2 X ϕ 2 ) C 1 C 2 X (ϕ 1 ϕ 2 ) C X -Monotonicity Rule: ϕ 1 ϕ 2 C X ϕ 1 C X ϕ 2 13 of 33

14 Axiomatizing the validities of ATL: fixpoint axioms The axiomatization of ATL extends to one for CL with the following fixed point axioms and rules for G and U : (FP G ) C Gϕ ϕ C X C Gϕ. (GFP G ) G(θ (ϕ C X θ)) G(θ C Gϕ), (FP U ) C ψ U ϕ ϕ (ψ C X C ψ U ϕ), (LFP U ) G((ϕ (ψ C X θ)) θ) G( C ψ U ϕ θ), plus the rule G-Necessitation: ϕ Gϕ. Completeness: VG and G. van Drimmelen (TCS 2006). 14 of 33

15 Solving the logical decision problems for ATL 15 of 33

16 Logical decision problems in ATL Local model checking: given an ATL formula ψ, a finite CGM M and a state s M, determine whether M, s ψ. Global model checking: given an ATL formula ψ and a finite CGM M, determine the set ψ M of states in M where ψ is true. Used for automated verification of formal specifications in open and multi-agent systems and synthesis of strategies and protocols. Satisfiability testing: given an ATL formula ψ, determine whether ψ is satisfiable, i.e., whether M, s ψ for some CGM M and a state s M. Constructive satisfiability testing: given an ATL formula ψ, determine whether ψ is satisfiable, and if so, construct a CGM M and a state s M such that M, s ψ. Used for synthesis of multi-agent systems and controllers from formal specifications. 16 of 33

17 Solving the model checking problem for ATL Alur, Henzinger, and Kupferman [JACM 2002] extend the labeling algorithm for global model checking for CTL to ATL and show that the model checking of ATL is PTIME-complete. The algorithm will be presented here. 17 of 33

18 The operator Pre recalled Consider a CGM M = A, St, Act, act, out, Prop, L Given a coalition C A, a state s, and a set X St, we say that C is effective for X at s, if C has a joint action at s that guarantees the outcome to be in X, no matter how the remaining agents act at s. We define Pre(M, C, X ) as the set of states at which the coalition C is effective for X. Formally: Pre(M, C, X ) := {s St α C α A\C out(s, α C, α A\C ) X } where α C denotes a tuple of actions, one for each agent in C. In particular, for a formula ϕ, Pre(M, C, ϕ M ) is precisely C X ϕ M. 18 of 33

19 The temporal operators as fixed points: C G The validity C Gϕ ϕ C X C Gϕ means that C Gϕ M is a fixed point of the operator G C,ϕ (Z) := ϕ M Pre(M, C, Z) The validity G(θ (ϕ C X θ)) G(θ C Gϕ) means that C Gϕ M is the greatest (post)-fixed point of G C,ϕ. Therefore: C Gϕ M can be computed by starting from Z = St and iteratively applying G C,ϕ until stabilization. It suffices to reach a stage where Z G C,ϕ (Z). Then G C,ϕ (Z) = Z will hold. 19 of 33

20 The temporal operators as fixed points: C U The validity C ψ U ϕ ϕ (ψ C X C ψ U ϕ) means that C ψ U ϕ M is a fixed point of the operator U C,ϕ,ψ (Z) := ϕ M ( ψ M Pre(M, C, Z)) The validity G((ϕ (ψ C X θ)) θ) G( C ψ U ϕ θ) means that C ψ U ϕ M is the least (pre)-fixed point of U C,ϕ,ψ. Therefore: C ψ U ϕ M can be computed by starting from Z = and iteratively applying U C,ϕ,ψ until stabilization. It suffices to reach a stage where U C,ϕ,ψ (Z) Z. Then U C,ϕ,ψ (Z) = Z will hold. 20 of 33

21 Algorithm for global model checking of ATL formulae 1: procedure GlobalMC(ATL)(M, ϕ) 2: case ϕ = p Prop : return {s St p L(s)} 3: case ϕ = ψ : return S \ ψ M 4: case ϕ = ψ 1 ψ 2 : return ψ 1 M ψ 2 M 5: case ϕ = A X ψ : return Pre(M, A, ψ M ) 6: case ϕ = A Gψ: ρ St; τ ψ M ; 7: while ρ τ do 8: ρ τ; τ Pre(M, A, ρ) ψ M 9: end while; return ρ 10: end case 11: case ϕ = A ψ 1 U ψ 2 : ρ ; τ ψ 2 M ; 12: while τ ρ do 13: ρ τ; τ ψ 2 M (Pre(M, A, ρ) ψ 1 M ) 14: end while; return ρ 15: end case 16: end procedure 21 of 33

22 Global model checking of ATL formulae: exercises (b,a) s 2 {p,q} (a,a) (a,b) s 3 {q} (a,a) (b,b) (b,b) (b,a) (b,b) (a,b) (b,a) s 1 {p} 1 Gp M = {s 1, s 2 } (a,a) (a,b) (a,a)(a,b)(b,b) s 4 {} (b,a) 2 Gp M = ( qup) M = {s 1, s 2 } 2 ( qup) M = {s 1, s 2, s 4 } 1 G 2 ( qup) M = {s 1, s 2, s 4 } 22 of 33

23 On model checking fair ATL and ATL* Alur, Henzinger, and Kupferman [JACM 2002] extend the model checking method for ATL to Fair ATL (ATL with fairness constraints) and to the full ATL and show that: - model checking of Fair ATL is PSPACE-complete - model-checking ATL is 2EXPTIME-complete (even in the special case of turn-based synchronous models). Furthermore, under assumptions of incomplete information and perfect memory, model checking of ATL becomes undecidable. 23 of 33

24 Solving the satisfiability problem for ATL and ATL* van Drimmelen [LICS 2003]: decidability (in EXPTIME) of the satisfiability problem (SAT) in ATL. See also VG and van Drimmelen [TCS 2006] for an algorithm for deciding SAT by employing alternating tree automata and bounding-branching model property. VG and Shkatov [ToCL 2010]: constructive and practically usable tableau-based method for deciding for ATL in EXPTIME. Implemented in 2013 by Amélie David at Univ. d Evry Val d Essonne EXPTIME lower bound for this problem comes from SAT for CTL. Hence, the EXPTIME-completeness of the problem. Schewe [ICALP 2008]: SAT for ATL is 2EXPTIME-complete. VG and Vester [AiML 2014]: lower complexities for satisfiability in flat fragments of ATL*. 24 of 33

25 Nesting of strategic operators Revocability of strategies Strategy contexts 25 of 33

26 Nesting strategies in the semantics of ATL (*): a problem? How to interpret intuitively nesting of strategic operators in ATL? E.g., what exactly does i X j ϕ mean? How about i G i ϕ? A strategy is a global conditional plan. So, is a formula like i G(ϕ i F ϕ) intuitively satisfiable? NB: when evaluating the goal Φ in a formula C Φ in a given CGM, the agents in C are no longer restricted by the strategy chosen for C to justify the truth of C Φ. I.e, the compositional semantics is opportunistic. Like in: These are my principles! If you do not like them... I can change them! (Attributed to Groucho Marx) 26 of 33

27 Nesting of strategies in the semantics of ATL: example Example: model M with a single agent: a b a q1 q2 q3 a not p p not p M, q 1 = i X p: at q 1 the agent i can achieve p in the next state. M, q 1 = i G i X p: at q 1 the agent i can make sure that she can always achieve p in the next state, but only by never achieving p. E.g., a bachelor has the strategy to always be able to marry tomorrow any girl he meets, but only if he stays unmarried forever. Thus: M, q 1 = i G( p i X p). 27 of 33

28 Nesting of coalitional strategic operators Consider the following claim, in the standard semantics for ATL: M, q = 1, 2 F 2, 3 Gp. This means that the agents 1, 2 have a collective strategy s 1, s 2 such that on every computation λ in M compatible with that strategy there is a state λ[n] such that M, λ[n] = 2, 3 Gp. However, the collective strategy s 1, s 2 does not feature in the evaluation of 2, 3 Gp at the state λ[n] in M. To justify the truth of 2, 3 Gp, any collective strategy t 2, t 3 for the agents 2, 3 can be selected, even though the strategy t 2 may deviate from the earlier chosen strategy s 2 for agent 2. Moreover, the effect of strategy s 1 is no longer taken into account when evaluating the truth of 2, 3 Gp. Even worse in ATL*, where e.g. A B γ B γ for any coalitions A, B. 28 of 33

29 Revocability and commitment of strategies What does having a strategy to achieve a (complex) goal mean exactly? Does it involve commitment? Can a strategy involve its own revocability? Can one commit not to commit? Some possible solutions: [Ågotnes,VG,Jamroga, TARK 07]:ATL with irrevocable strategies The idea: once chosen, strategies become irrevocable throughout the entire evaluation of the formula. The price: Imposes unnecessarily strong commitments when the goals are local (next-state) goals or eventualities. Yields a non-compositional semantics. A more flexible solution: mechanisms for dynamic strategic commitment and release. [ Ågotnes, VG, Jamroga, LOFT 08]. Also, Brihaye et al, LFCS 2009: ATL with strategy context. 29 of 33

30 Operators for strategic commitment and release The idea: extend the language with operators to control strategic commitments and release within the formulae. A solution: add operators commit C, and release C. Example: John F(Get academic job John John FGet married) Formal semantics: truth of formulae is evaluated relative to models that are dynamically updated by strategy contexts. 30 of 33

31 Model update semantics of ATL (, ) Strategy context: a tuple of strategies F A for a set of agents A. For a strategy context F A and a collective strategy f C we denote: F f C : the strategy context extending F with the strategies in f C corresponding to the agents not mentioned in F. F C: the strategy context obtained from F by removing the strategies corresponding to the agents in C. Update of a model M by a strategy context f C : the model M F C obtained by fixing the choices of C as per the strategies in f C. Interpretation of the new strategic operators: M F, q = C φ iff C has a collective strategy f C such that M (F f C ), λ = φ for all plays λ starting at q in M (F f C ) M F, q = C φ iff M (F C), λ = φ for all plays λ starting at q in M (F C). 31 of 33

32 ATL with strategy contexts Brihaye, Da Costa, Laroussinie, and Markey: ATL with strategy contexts and bounded memory (LFCS 2009) The idea is very similar: keep the model intact but evaluate the truth relative to an explicitly specified strategy context. Only the strategy context update is different. Update of contexts. Consider a joint strategy s A of agents A (the current strategy context), and a new joint strategy t B of agents B. Context update s A t B is the joint strategy f for the agents in A B such that: f [i] = t B [i] for i B and f [i] = s A [i] for i A \ B. That is, the new strategies from t B are added to the context, replacing the previous ones for the same agents. N. Troquard and D. Walther (JELIA2012): Satisfiability in ATL with strategy contexts is undecidable. 32 of 33

33 Some references T. Ågotnes, V. Goranko, W. Jamroga, ATL with irrevocable strategies, Proc. of TARK T. Ågotnes, V. Goranko, W. Jamroga, ATL with Strategic Commitment and Release, Proc. of LOFT T. Brihaye, A. Da Costa, F. Laroussinie, and N. Markey: ATL with strategy contexts and bounded memory, Proc. of LFCS 2009 N. Troquard and D. Walther: Satisfiability in ATL with strategy contexts is undecidable, Proc. of JELIA of 33