Specification and Verification of Multi-Agent Systems ESSLLI 2010 CPH

Transcription

1 Specification and Verification of Multi-Agent Systems Wojciech Jamroga 1 and Wojciech Penczek 2 1 Computer Science and Communication, University of Luxembourg wojtek.jamroga@uni.lu 2 Institute of Computer Science, PAS, and University of Podlasie, Poland penczek@ipipan.waw.pl ESSLLI 2010 CPH Overview. Multi agent systems (MAS) provide an important framework for formalizing various problems in computer science, artificial intelligence, game theory, social choice theory, etc. Modal logics are amongst the most suitable and versatile logical formalisms for specification and verification of computational systems. The course offers an introduction to some important developments in the area. We introduce modal logics used for specification of temporal, epistemic, and strategic properties of systems; then, we present model checking algorithms based on Binary Decision Diagrams (BDD) and satisfiability solving (SAT), and discuss the computational complexity of the model checking problem. We also consider symbolic (compact) representations of systems, and how the representations change the semantic and algorithmic side of model checking. Finally, we discuss some techniques that help to reduce the complexity and make verification feasible even for large systems. Background. This is an advanced course, and the participants are expected to have basic knowledge of modal logic and complexity theory. We will introduce all the relevant fundamental concepts, but only very briefly. Program. The course consists of 5 lectures, each 2 x 40 minutes long (with a 10 minutes break in between). A preliminary program is given below. 1. Reasoning about evolution of systems. Multi-agent systems. Modal logic, Kripke models, epistemic logic. Temporal logic, linear vs. branching time, synchronous vs. asynchronous product; reasoning about knowledge and time; mu-calculus. Basic complexity classes, basic complexity results. Statespace explosion. 2. Introduction to model checking for knowledge and time. Standard non-symbolic algorithms. Model generators for partial order reductions (interleaved interpreted systems), partial order reduction methods, introduction to symbolic model checking. Logics for real time. Demo (VERICS).

2 3. Specification of agents and their teams. Strategic logics, ATL. Basic model checking algorithm for ATL. Imperfect information scenarios. Demo (MCMAS). 4. Complexity of verification. Model checking complexity for explicit models, example complexity proofs. Complexity revisited: higher-order representations. Model checking for real time and knowledge. Timed automata, detailed region graphs, timed CTLK, standard non-symbolic model checking, complexity results. 5. Practical model checking. BDD-based and SAT-based approaches to model checking, bounded and unbounded model checking for CTLK, unbounded model checking for ATL. Demonstration and experimental results (VERICS). See the following individual lecture notes for references to reading material. Publications which may be of particular interest are: 1. R. Alur, T. A. Henzinger, and O. Kupferman. Alternating-time Temporal Logic. Journal of the ACM, 49: , Available at tah/publications/alternating-time temporal logic.html. 2. A. Lomuscio, W. Penczek. Logic Column 19: Symbolic Model Checking for Temporal-Epistemic Logics, CoRR abs/ : (2007). Available at About the lecturers. Wojtek Jamroga is a senior postdoc at the Individual and Collective Reasoning group at the University of Luxembourg. He works mainly on logical and game-theoretical aspects of multi-agent systems. His teaching record includes three courses at ESSLLI (2006, 2007, and 2008), and three courses at EASSS (European Agent Systems Summer School). Wojtek Penczek is the head of the Theory of Distributed Systems group at ICS PAS, Warsaw. He was a research fellow at the Technical University of Eindhoven and Manchester University; he also worked as a consultant at AT&T. His research is now focused on verification methods for (timed) distributed and multi-agent systems. He has been a project leader of the EC-founded project CRIT-2 and played a main role in several national projects. He has chaired and was a PC member of many conferences on Computer Science. He is the author of more than 100 conference and journal articles on temporal logic, verification methods, model checking, and formal theories of multi-agent systems. His teaching record, besides numerous regular undergraduate courses, includes two courses at EASSS (European Agent Systems Summer School). More course material, including updated notes and slides will be placed on the course website: wjamroga/courses/verification2010esslli/ 2

3 1 Reasoning about evolution of systems The first lecture serves as an introduction of some fundamental concepts that we are going to use throughout this course. 1.1 Multi-agent systems Multi-agent systems (MAS) are systems that involve several autonomous entities acting in the same environment. The entities are called agents. What is an agent? Despite numerous attempts to answer this question, there seems to be no conclusive definition. We assume that MAS are, most of all, a philosophical metaphor that induces a specific way of seeing the world, and makes us use agent-oriented vocabulary when describing the phenomena we are interested in. Thus, while some researchers present multi-agent systems as a new paradigm for computation or design, we believe that primarily multi-agent systems form a new paradigm for thinking and talking about the world, and assigning it a specific conceptual structure. The metaphor of a multi-agent system seems to build on the intuition that we are agents and that other entities we study can be just like us to some extent. The usual properties of agents, like autonomy, pro-activeness etc., seem to be secondary: they are results of an introspection rather than primary assumptions we start with. We note that this view of multi-agent systems comes close to the role of logic in both philosophy and computer science. Logic provides conceptual structures for modeling and reasoning about the world in a precise manner and, occasionally, it also provides tools to do it automatically. References: [51, 52]. 1.2 Modal logic Modal logic is an extension of classical logic with new operators (necessity) and (possibility): p means that p is true in every possible scenario, while p means that p is true in at least one possible scenario. Let PV be the set of atomic propositions. Models of modal logics are called Kripke models or possible world models, and include the set of possible worlds (or states) St, modal accessibility relation(s) R 1,..., R k St St, and interpretation of propositions V : PV 2 St. Now, for a model M and world q in M: M, q = i ϕ iff M, q = ϕ for all q such that qr i q M, q = i ϕ iff M, q = ϕ for some q such that qr i q The actual accessibility relations can capture various dimensions of the reality (and therefore gives rise to different kinds of modal logics): knowledge ( epistemic logic), beliefs ( doxastic logic), obligations ( deontic logic), actions ( dynamic logic), time ( temporal logic) etc. In particular, various aspects 3

4 of agents (and agent systems) can be naturally captured within this generic framework. References: [8]. 1.3 Reasoning about knowledge The basic epistemic logic which we consider here involves modalities for individual agent s knowledge K i, with K i ϕ interpreted as agent i knows that ϕ. Additionally, one can consider modalities for collective knowledge of groups of agents: mutual knowledge (E A ϕ: everybody in group A knows that ϕ ), common knowledge (C A ϕ: all the agents in A know that ϕ, and they know that they know it etc. ), and distributed knowledge (D A ϕ: if the agents could share their individual information, they would be able to recognize that ϕ ). Note that E A ϕ i A K iϕ, and hence the operators for mutual knowledge can be omitted from the language. The formal semantics for the logic is based on multi-agent epistemic models of the type St, 1,..., k, V, where St is a set of epistemic states, V is a valuation of propositions, and each i St St is an equivalence relation that defines the indistinguishability of states for agent i. Operators K i are provided with the usual Kripke semantics given by the clause: M, q = K i ϕ iff M, q = ϕ for all q such that q i q The accessibility relation corresponding to E A is defined as E A = i A i, and the semantics of E A becomes M, q = E A ϕ iff M, q = ϕ for all q such that q E A q. Common knowledge C A is given semantics in terms of the relation C A defined as the transitive closure of E A : M, q = C A ϕ iff M, q = ϕ for all q such that q C A q. Finally, distributed knowledge D A is given semantics in terms of the relation D A defined as i A i, following the same pattern: M, q = D A ϕ iff M, q = ϕ for all q such that q D A q. References: [15, 19, 47]. 1.4 Modal logics of time and action In this part of the lecture, we present a brief overview of logics that regard the dynamics of systems. That is, essentially, logics that focus on actions that can be performed by (or in) a system, and on the way in which the system can evolve over time. We consider two basic cases here: either actions are first-class citizens of the language, or we abstract from them and reason about change in general. 4

5 PDL. Propositional Dynamic Logic (PDL), which was primarily designed to reason about computer programs, is probably the most typical representative of logics with explicit actions. Actions are represented in the language with action labels α 1, α 2,... from a set Act. Complex action terms can be also constructed, for example sequential composition (α 1 ; α 2 ), nondeterministic choice (α 1 α 2 ), finite iteration (α ) etc. Now, [α]ϕ expresses the fact that ϕ is bound to hold after every execution of α, and α ϕ [α] ϕ says that ϕ holds after at least one possible execution of α. On the semantic side, we have Labeled Transition Systems M = St,, α1, α2..., V, where actions are modeled as (nondeterministic) α state transformations St St, and the following semantic clause: M, q = [α]ϕ iff M, q = ϕ for all q such that q α q. LTL. Temporal logics leave actions implicit, and instead focus on possible patterns of evolution. Typical temporal operators are: X ( in the next state ), G ( always from now on ), F ( sometime in the future ), and U ( until ). Models include one transition relation, and come in two versions. Linear time models define a total ordering on possible worlds ( time moments ), so that a model can be seen as a single temporal path λ with successive states λ[0], λ[1],... ; the semantics of Linear Temporal Logic (LTL) can be defined as follows: λ = p iff λ[0] V (p); λ = ϕ iff λ = ϕ, λ = ϕ ψ iff λ = ϕ and λ = ψ; λ = Xϕ iff λ[1.. ] = ϕ; λ = ϕuψ iff λ[i.. ] = ϕ for some i 0, and λ[j.. ] = ϕ for all 0 j i, with Fϕ Uϕ, and Gϕ F ϕ. CTL. Branching-time models, on the other hand, consist of a tree that encapsulates all possible evolutions of the system. Computation Tree Logic (CTL) extends LTL with path quantifiers E ( there is a path ), and A ( for every path ), with the following semantics: M, q = Eϕ iff there is a path λ in M, that starts from q, and M, λ = ϕ. Consequently, Aϕ E ϕ. There are two basic versions of Computation Tree Logic: in pure (or vanilla ) CTL every occurrence of a temporal operator is preceded by exactly one path quantifier, while no such restriction is imposed in CTL*. Note that in the case of vanilla CTL, there exists an alternative semantics given entirely in terms of satisfaction of formulae in states: M, q = p iff q V (p); M, q = ϕ iff M, q = ϕ; M, q = ϕ ψ iff M, q = ϕ and M, q = ψ; M, q = EXϕ iff there is a q-path λ such that M, λ[1] = ϕ; 5

6 M, q = EGϕ iff there is a q-path λ such that M, λ[i] = ϕ for every i 0; M, q = EϕUψ iff there is a q-path λ and i 0 such that M, λ[i] = ψ, and M, λ[j] = ϕ for every 0 j < i; Temporal and dynamic dimensions have been combined with other modalities, e.g. in the well-known BDI logics of beliefs, desires and intentions. Modal µ-calculus. The µ-calculus is an extension of propositional modal logic with the least and greatest fixpoint operators µ, ν. The main idea is to use denotational semantics of modal sentences, in which formulae are denoted by sets of states, and (unary) modal operators are interpreted as transformers of such sets. Then, for a transformer τ, formula µx.τ(x) refers to the least fixpoint of τ, that is, the smallest set of states Q such that τ(q) = Q. Analogously, νx.τ(x) denotes the greatest fixpoint of τ. The µ-calculus is usually built on top of PDL operators. Formally, its semantics can be given as below. [p] M = V M (p); [ ϕ] M = St M \ [ϕ] M, [ϕ ψ ] M = [ϕ] M [ϕ] M ; [[α]ϕ] M = {q q.q α q q [ϕ] M }; [µx.ϕ(x)] M = the smallest Q St such that [ϕ(x)] M[V (X):=Q] = Q; [νx.ϕ(x)] M = the largest Q St such that [ϕ(x)] M[V (X):=Q] = Q. An important theorem that greatly facilitates computing fixpoints states that, if τ is a monotonic transformer of state sets, then its least and greatest fixpoints exist, are unique, and can be obtained according to the following equations: [µx.ϕ(x)] M = {Q St [ϕ(x)] M[V (X):=Q] Q}; [νx.ϕ(x)] M = {Q St [ϕ(x)] M[V (X):=Q] Q}. Modal µ-calculus strictly embeds PDL, LTL, CTL and CTL*. References: [20, 14, 16]. 1.5 Combining time and knowledge CTLK. In this section we discuss how the interaction between knowledge and time can be captured in modal logic. To this end, we combine epistemic and temporal dimensions in a multi-modal logic. The main proposal that we are going to consider is CTLK, which is a straightforward combination of CTL and standard epistemic logic. The language of CTLK includes all the operators of CTL and epistemic logic. Moreover, its semantics is based on Kripke models which include both temporal and epistemic accessibility relations: M = St, R, 1,..., k, V. The semantics of CTLK is simply a union of both respective semantics: M, q = p iff q V (p); 6

7 M, q = ϕ iff M, q = ϕ; M, q = ϕ ψ iff M, q = ϕ and M, q = ψ; M, q = EXϕ iff there is a q-path λ such that M, λ[1] = ϕ; M, q = EGϕ iff there is a q-path λ such that M, λ[i] = ϕ for every i 0; M, q = EϕUψ iff there is a q-path λ and i 0 such that M, λ[i] = ψ, and M, λ[j] = ϕ for every 0 j < i; M, q = K a ϕ iff M, q = ϕ for every q such that q a q. Interpreted systems. A well known study of the interplay between knowledge and time has been presented in [15]. The proposal builds on a notion of global states, defined formally as follows. Firstly, each agent i Agt = {1,..., k} has a set of local states St i. Every global state is represented by a tuple of local states q 1,..., q k corresponding to each of the agents. Thus, the global state space St is (a subset of) the product St 1... St k. It is assumed that each agent has access only to its own local state, i.e.: q 1,..., q k i q 1,..., q k iff q i = q i. The temporal dimension is added to by considering runs, i.e., sequences of global states: r : N St 1... St k. A system is a set of such runs. An interpreted system is a system plus a valuation of propositions: R, V. Given an interpreted system I = R, V, a point in I is a pair r, m where r is a run and m N. Epistemic equivalence between points is defined as follows: r, m i r, m iff r(m) i r (m ). Now, all epistemic modalities can be interpreted as before, e.g.: I, r, m = K i ϕ iff I, r, m = ϕ for all r, m such that r, m i r, m. Moreover, the standard temporal operators of LTL can be interpreted, too: I, r, m = Xϕ iff I, r, m + 1 = ϕ, I, r, m = ϕuψ iff I, r, m = ψ for some m > m and I, r, m = ϕ for all m such that m m < m. Finally, the semantics of path quantifiers from CTL* can be defined in interpreted systems as follows: I, r, m = Eϕ iff for all r such that r [0..m] = r[0..m] we have I, r, m = ϕ. Interpreted systems have been applied to modeling of synchrony and asynchrony, perfect recall, message passing systems, knowledge bases, distributed systems etc. References: [15]. 7

8 l Epistemic logic PSPACE-complete PDL PSPACE-complete CTL EXPTIME-complete LTL PSPACE-complete CTL* 2EXPTIME-complete m, l Epistemic logic P-complete PDL P-complete CTL P-complete LTL PSPACE-complete CTL* PSPACE-complete Fig. 1. Basic complexity results: satisfiability (left) and model checking (right) 1.6 Complexity of verification In this lecture, we will consider both practical algorithms for verifying properties of MAS, and the theoretical complexity of these problems. Here is a short (and rather informal) description of the most relevant complexity classes. A formal introduction can be found in any textbook on complexity theory. P: problems solvable in polynomial time by a deterministic Turing machine, NP: problems solvable in polynomial time by a nondeterministic Turing machine, Σ P n / Π P n / P n : problems solvable in polynomial time with use of adaptive queries to an n-level oracle, PSPACE: problems solvable by queries to a multilevel oracle with unbounded height, EXPTIME: problems solvable in exponential time by a deterministic Turing machine. Of course, theoretical complexity has many deficiencies: it refers only to the worst (hardest) instance in the set, neglects coefficients in the function characterizing the complexity, etc. However, it often gives a good indication of the inherent hardness of the problem in terms of scalability. For low complexity classes, scaling up from small instances of the problem to larger instances is relatively easy. For high complexity classes, this is not the case anymore. Figure 1 presents some basic complexity results for the satisfiability and model checking problems. The input of the SAT problem is given as a formula of the logic, and its size is measured in the length of the formula. The input of the model checking problem is given as the formula and the model; the size of an input instance is measured as m l, where m is the number of transitions in the model, and l is the length of the formula. 8

9 2 Introduction to model checking for knowledge and time In this lecture we look at standard non-symbolic model checking algorithms for CTLK and LTLK. Then, we define partial order methods for reducing state spaces for LTLK J X and CTLKJ X. Finally, we make an introduction to symbolic model checking and define a logic for real time and knowledge. 2.1 Model checking CTLK by state labeling If we do not bother about the size of a model, then the simplest approach to CTLK model checking, called state labelling, can be used. In the lecture, we show a deterministic algorithm, based on state labelling, for determining whether a CTLK formula ϕ is true at a state s S in a finite model M = ((S, s 0, ), V ), of time complexity O( ϕ ( S + )). The algorithm is designed so that when it finishes, each state s of M is labelled with the subformulas of ϕ which are true at s. The algorithm operates in stages. The i-th stage handles all subformulas of ϕ of length i for i ϕ. Thus, at the end of the last stage each state will be labelled with all subformulas of ϕ which are true at it. References. The original state labelling algoritm for CTL was introduced in [10]. Further reading [43]. Other non-symbolic approaches. A translation of a fragment of LTLK to LTL is shown in [21]. Then, the model checker SPIN is used for verification. 2.2 Partial order reductions for LTLK X and CTL*K X We investigate partial order reductions for model checking of multi-agent systems by focusing on interleaved interpreted systems. We give notions of stutteringequivalence and stuttering-bisimulation, and prove that they preserve respectively LTLK X and CTL*K X. Finally, we present algorithms to reduce the size of the models before the model checking step and show they preserve LTLK X and CTL*K X properties. Our approach is applicable to Interleaved Interpreted Systems, defined as follows. We assume that MAS is composed of m agents Agt = {1,..., m}. By L i = {li 1, l2 i,..., lnli i } we denote possible local states, Act i = {ɛ i, a 1 i, a2 i,..., anai i } - actions, ɛ i - the special action. Let Act = i A Act i be the union of all the sets Act i, Agent(a) Agt contains all the agents i such that a Act i, P i : L i 2 Acti is a local protocol, s.t. ɛ P i (li k), for any lk i. By t i : L i Act i L i we mean an evolution (partial) function, where t i (l i, ɛ i ) = l i for each l i L i, a global state g = (l 1,..., l m ) is a tuple of local states for all the agents, and G is the set of the global states. The global interleaved evolution function is defined as follows: t(g, act 1,..., act m ) = g iff there exists an action a Act such that for all i Agent(a), act i = a and t i (g i, a) = g i, and for all 9

10 i Agt \ Agent(a), act i = ɛ i and t i (g i, act i ) = g i. By π = g 0 a 0 g 1 a 1 g 2... we a mean an interleaved path, if g i i gi+1 for i 0. Π (g) stand for the set of all interleaved paths originating from g. Let P V be a set of propositions. Definition 1 (Interleaved Interpreted Systems). A tuple M = (G, ι, Π, V ) is an interleaved interpreted system (IIS), where G is a set of global states, ι G is an initial (global) state s.t. each state in G is reachable from ι, Π = Π(g) is the set of all the interleaved paths originating from all g G states in G, and V : G 2 PV is a valuation function. References: The first approach to partial order reductions for LTLK can be found in [34]. Our lecture is based on the solution of [35]. 2.3 Fixed-point verification for CTLK Symbolic and non-symbolic model checking methods can exploit the fixed point characterization of CTLK formulas. In what follows by C A we denote the modality dual to C A, i.e., C A ϕ def = C A ϕ. Similarly, for E A and D A. We do not discuss here algorithms for the operators E A and D A as these are straightforward given an algorithm for K i. Then, labelling of the states with the subformulas or computation of OBDD representation of a CTLK formula uses the standard algorithms for computing the minimal and the maximal fixpoints as follows. EGϕ ϕ EXEGϕ, E(ϕUψ) ψ (ϕ E(ϕUψ)), C A ϕ ϕ E A C A ϕ. Let ϕ = {s S s = ϕ}. Then, we have: EGϕ = ϕ EXEGϕ, E(ϕUψ) = ψ ( ϕ EXE(ϕUψ) ) C A ϕ = ϕ E A C A ϕ ) We can easily define algorithms computing the following sets: pre(x) = {s S ( s X) s s }, indis i (X) = {s S ( s X) s i s }, and indis E A (X) = {s S ( s X) s E A s }, for each subset X S. Then, we have: EGϕ = ϕ pre( EGϕ ), E(ϕUψ) = ψ ( ϕ pre( E(ϕUψ) )), C A ϕ = ϕ indis E A ( C Aϕ ). Next, define three functions on 2 S, which fixed points are equal to respectively EGϕ, E(ϕUψ), and C A ϕ. 10

11 1. τ EGϕ (X) = ϕ pre(x), 2. τ E(ϕUψ) (X) = ψ ( ϕ pre(x)), 3. τ CA ϕ (X) = ϕ indise A (X). Since EGϕ is the maximal fixpoint of τ EGϕ (X), it can be computed as τegϕ k (S) for some finite k. Since E(ϕUψ) is the minimal fixpoint of τ E(ϕUψ) (X) it can be computed as τe(ϕuψ) l ( ) for some finite l. Similarly, for τ C A ϕ (X). The above characterization can be now used for defining a model checking algorithm mchk for the formulas of CTLK. mchk(m, ϕ) { if ϕ P V, then return V 1 (ϕ), if ϕ = ψ, then return S \ mchk(m, ψ), if ϕ = ϕ 1 ϕ 2, then return mchk(m, ϕ 1 ) mchk(m, ϕ 2 ), if ϕ = EXψ, then return mchk EX (M, ψ), if ϕ = K i ψ, then return mchk Ki (M, ψ), if ϕ = EGψ, then return mchk EG (M, ψ), if ϕ = E(φUψ), then return mchk EU (M, φ, ψ), if ϕ = Cψ, then return mchk C (M, ψ). } mchk EX (M, ψ){ X := mchk(m, ψ); Y := pre(x); return Y }; mchk EG (M, ψ){ X := mchk(m, ψ); Y := S; Z := ; while (Z Y ){ Z := Y ; Y := X pre(y )} return Y }; mchk C (M, ψ){ Y := mchk(m, ψ); Z := ; W := S; while (Z W ){ W := Z; Z := Y indis E A (Z))} return Z }; mchk EKi (M, ψ){ X := mchk(m, ψ); Y := indis i (X); return Y }; mchk EU (M, ψ 1, ψ 2 ){ X := mchk(m, ψ 1 ); Y := mchk(m, ψ 2 ); Z := ; W := S; while (Z W ){ W := Z; Z := Y (X pre(z))} return Z }; 11

12 2.4 Logic for real time and knowledge We present TECTLK, a logic to specify knowledge and real time in multi-agent systems. We start with introducing the concept of timed automata [4] and then define Real Time Interpreted Systems. 2.5 Timed Automata Let IR = [0, ) be a set of non-negative real numbers, IR + = (0, ) a set of positive real numbers, IN = {0, 1,...} a set of natural numbers, X a finite set of real variables, called clocks, x X, c IN, and {, <, =, >, }. The clock constraints over X are defined by the following grammar: cc := true x c cc cc. The set of all the clock constraints over X is denoted by C(X ). A clock valuation on X is a tuple v IR X. The value of the clock x in v is denoted by v(x). For a valuation v and δ IR, v + δ denotes the valuation v such that for all x X, v (x) = v(x)+δ. For a subset of clocks X X, v[x := 0] denotes the valuation v such that v (x) = 0 for all x X, and v (x) = v(x) for all x X \ X. The satisfaction relation = for a clock constraint cc C(X ) and v IR X is defined inductively as follows: v = true, v = (x c) iff v(x) c, v = (cc cc ) iff v = cc iff v = cc. For a clock constraint cc C(X ), by [[cc] we denote the set of all the clock valuations satisfying cc, i.e., [cc] = {v IR X v = cc}. Definition 2 (Timed automaton). A timed automaton is a tuple TA = (Z, L, l 0, E, X, I), where Z is a finite set of actions, L is a finite set of locations, l 0 L is an initial location, X is a finite set of clocks, E L Z C(X ) 2 X L is a transition relation, and I : L C(X ) is a location invariant function, assigning to each location l L a clock constraint defining the conditions under which TA may stay in l. 2.6 Clock Equivalence We use timed automata to interpret a logical language for real time and knowledge. Let TA = (Z, L, l 0, E, X, I) be a timed automaton. An instantaneous state of TA is a pair (l, v), where l L and v IR X. Definition 3. The dense state space of TA is a tuple (L IR X, q 0, ), where L IR X is a set of all the instantaneous states, q 0 = (l 0, v 0 ) is the initial state such that v 0 (x) = 0 for all x X and v 0 [I(l 0 )], and (L IR X ) (Z IR) (L IR X ) is the transition relation, defined by: a Action transition: for a Z, (l, v) (l, v ) iff ( cc C(X ))( X X ) such that l a,cc,x l E, v [cc], v = v[x := 0], and v [I(l )], 12

13 δ Time transition: for δ IR, (l, v) (l, v + δ) iff v, v + δ [I(l)]. For (l, v) L IR X, let (l, v) + δ denote (l, v + δ). A q 0 -run ρ of TA is a finite or infinite sequence of instantaneous states: δ q 0 a 0 q0 + δ 0 δ 0 1 a q1 q1 + δ 1 δ 1 2 q2... such that q i L IR X, a i Z, δ 0 0, and δ i IR + for each i IN \ {0}. Let TA be a timed automaton, C(TA) C(X ) be a non-empty set containing all the clock constraints occurring in all enabling conditions used in the transition relation E and all state invariants of TA. Moreover, let c max be the largest constant appearing in C(TA) and let fr(σ) (respectively σ ), σ IR, denote the fractional (respectively integral part) of σ. We define an equivalence relation in the set of all the clock valuations as follows. Definition 4 ([3]). For two clock valuations v, v IR X, v v if and only if the following conditions are met: 1. For all x X, v(x) > c max iff v (x) > c max, 2. For all x, y X, if v(x) c max and v(y) c max then a.) v(x) = v (x), b.) fr(v(x)) = 0 iff fr(v (x)) = 0, and c.) fr(v(x)) fr(v(y)) iff fr(v (x)) fr(v (y)). In other words the valuations are equivalent if they return values greater than c max for the same x and when their integral part is the same for any x, and the fractional parts are either both nil or preserve the order of any two clock values. The relation partitions IR X into zones, denoted by Z, Z, and so on. We will denote the set of all the zones by Z( X ). 2.7 Real Time Interpreted Systems Let Agt be a set of m agents such that each agent is modelled by a timed automaton TA i = (Z i, L i, li 0, E i, X i, I i ), for i = 1,..., m, TA = (Z, L, l 0, E, X, I) the parallel composition of all the agents, and l i : L L i be a function returning the location of agent i in a global location. Moreover, we take PV i to be a set of propositional variables containing the constant true (denoted by true) such that PV i PV j = for all i, j {1,..., m}, and PV = m i=1 PV i. In order to reason about multi-agent systems, where each agent is represented by a timed automaton, we assume the existence of a (local) valuation function V TAi : L i 2 PVi for each agent i. We further require that true V TAi (l) for each l L i. The (global) valuation function V TA : L 2 PV for the parallel composition, is defined by V TA ((l 1,..., l m )) = m i=1 V TA i (l i ). Given this, a real time interpreted system is defined as follows. Definition 5 (Real Time Interpreted System). A real time interpreted system is a tuple M = (Q, q 0,, 1,..., m, V ), where: 13

14 Q is a subset of L IR X such that all the instantaneous states in Q are reachable 1. q 0, and are defined as in Definition 3. i Q Q is an (equivalence) relation defined by (l, v) i (l, v ) iff l i (l) = l i (l ) and v v, for each agent i. V : Q 2 PV is a valuation function such that V ((l, v)) = V TA (l). In (discrete time) interpreted systems the definition of i is based on the equality of the local states for agent i in the two global states. The present definition extends that by assuming that not only the local locations of the agents are the same, but also the two clock valuations are in the same zone. To reason about MAS, we introduce TECTLK, a logic for knowledge and real time that is the fusion [8] of the two underlying languages: an existential fragment of TCTL for branching real time [3] and S5 n for the knowledge operators. 2.8 Logic TCTLK Let PV be a set of propositional variables containing the symbol true that represents the constant true, Agt a set of m agents, and I an interval in IR with integer bounds of the form [n, n ], [n, n ), (n, n ], (n, n ), (n, ), and [n, ), for n, n IN. Let p PV, i Agt, and A Agt. The set of TECTLK formulae is defined by the following grammar: ϕ := p p ϕ ϕ ϕ ϕ E(ϕU I ϕ) E(ϕR I ϕ) K i ϕ D A ϕ E A ϕ C A ϕ. Let f TA (q) denote the set of all q-runs for TA, that is, the set of all the runs in TA that start at the state q. In order to give a semantics to TECTLK, we introduce the notion of a dense δ path π ρ corresponding to a q 0 -run ρ = q 0 a 0 q0 + δ 0 δ 0 1 a q1 q1 + δ 1 1 δ q Let idx(ρ, r) be the greatest i IN such that Σ i 1 j=0 δ j r. Notice that for i = 0 we let Σ i 1 j=0 δ j = 0. So, for r δ 0, idx(ρ, r) = 0. A dense path π ρ corresponding to ρ is a mapping from IR to the set of states Q such that π ρ (r) = q i + r Σ i 1 j=0 δ j where i = idx(ρ, r). This can be done in a unique way because we assume that runs of a TA do not contain two consecutive action transitions. Definition 6 (Satisfaction). Let M = (Q, q 0,, 1,..., m, V ) be a real time interpreted system. M, q = ϕ denotes that ϕ is true at state q in M. M is omitted, if it is implicitly understood. The satisfaction relation = is defined inductively as follows: q = p iff p V (q), q = p iff p / V (q), q = ϕ ψ iff q = ϕ or q = ψ, 1 An instantaneous state q L IR X is reachable iff there is a q 0 run ρ in TA such that there exists an instantaneous state in ρ equal to q. 14

15 q = ϕ ψ iff q = ϕ and q = ψ, q = E(ϕU I ψ) iff ( ρ f TA (q))( r I)(π ρ (r) = ψ and ( r < r) π ρ (r ) = ϕ), q = E(ϕR I ψ) iff ( ρ f TA (q))( r I)(π ρ (r) = ψ or ( r < r) π ρ (r ) = ϕ), q = K i ϕ iff ( q Q)(q i q and q = ϕ), q = D A ϕ iff ( q Q)(q D A q and q = ϕ), q = E A ϕ iff ( q Q)(q E A q and q = ϕ), q = C A ϕ iff ( q Q)(q C A q and q = ϕ). In the next lecture on real time and knowledge we discuss bounded model checking for TECTLK formulae. References. Our presentation is based on [36] and [43]. 15

16 3 Specification of agents and their teams 3.1 Games and strategies Interactions between autonomous and rational agents acting strategically have been extensively studied in the field of game theory. The models used in game theory can be categorised into two types: non-cooperative games, in which the possible actions of individual players are taken as primitives, and coalitional (or cooperative) games, in which the possible joint actions of groups of players are taken as primitives. A standard model in non-cooperative game theory is that of a strategic game. In a strategic game, it is assumed that each agent chooses her future actions (her strategy) once and for all at the beginning of the game, and that all agents do this simultaneously. Formally, a strategic game is a tuple G = (Agt, {Σ i : i Agt}, o, St, { i : i Agt}) where Agt is the set of agents, St is a set of states, Σ i is the set of actions (or strategies) available to agent i, o associates an outcome state o(a 1,..., a n ) St with every tuple of actions (a 1,..., a n ) i Agt Σ i, and i St St is a preference relation (complete, reflexive, transitive) over the outcome states for agent i. Leaving out the preference relations, we get a strategic game form G = (Agt, {Σ i : i Agt}, o, St). Cooperation in games. Henceforth, a coalition is simply a group of agents. An effectivity function models the powers agents can obtain by forming coalitions. Given a set of states X, we can say that a coalition C is effective for X if C can cooperate to ensure that the outcome will lie in X. Formally, an effectivity function for a set of players Agt over a set of states St is a function E : 2 Agt 2 2St giving the sets of outcomes E(C) for which each coalition C is effective. A game frame induces an effective function as follows: X E G (C) σ C σ Agt\C o(σ C, σ Agt\C ) X The effectivity functions induced from game frames in this way are called α-effectivity functions in social choice theory. References: [39]. 3.2 Coalition Logic Modal logics of strategic ability form one of the fields where logic and game theory can successfully meet. Marc Pauly s Coalition Logic (CL) formalises reasoning about the abilities of coalitions. The language of CL extends propositional logic with a modality [C] for each coalition C. The intended meaning of [C]ϕ is that C can make the outcome of the game satisfy ϕ. Formally, the language is interpreted over coalition models M = (St, E, V ) 16

17 where St is a set of states, V : PV 2 St an interpretation of atomic propositions PV in the states, and E : St (2 Agt 2 2St ) is a function such that for each q St, E(q) is a playable effectivity function. The interpretation of the main construct of the language in a state q in a coalition model M is defined as follows: M, q = [C]φ iff φ M E(q)(C) where φ M = {q St : M, q = φ}. Since the coalition models being used are usually based on α-effectivity functions of some strategic game, formulae of Coalition Logic can be alternatively seen as statements about strategic game forms. References: [40]. 3.3 ATL Coalition Logic allows only to reason about agents outcomes in strategic (onestep) games. What if a game consists of multiple steps, like in the case of extensive form games? In this lecture, we focus on Alternating-time Temporal Logic (ATL), one of the most important logic of time and strategies that have emerged in recent years. ATL can be seen as a generalization of the branching time temporal logic CTL (Coalition Tree Logic), in which path quantifiers (E: there is a path, A: for every path ) are replaced with strategic quantifiers A. The formula A ϕ, where A is a coalition of agents, expresses the claim that A has a collective strategy to enforce the temporal property ϕ throughout the interaction. Moreover, ATL formulas include the usual temporal operators: X ( in the next state ), G ( always from now on ), F ( sometime in the future ), and U ( until ). Several semantics have been introduced for atl, of which the one based on concurrent game structures (CGS) is perhaps the most popular. A concurrent game structure M = Agt, St, Act, d, o, V includes: a nonempty finite set of agents Agt = {1,..., k}; a nonempty set of global states St of the system; a set of (atomic) actions Act; function d : Agt St 2 Act defining the set of actions available to an agent in a state; a valuation of propositions V : St 2 PV ; and o is a (deterministic) transition function which assigns the outcome state q = o(q, α 1,..., α k ) to state q and a tuple of actions α 1,..., α k that can be executed by Agt in q. Note that, as concurrent game structures do not allow to represent any kind of uncertainty, ATL can be seen as a logic for reasoning about agents with perfect information about the current state of the game. A strategy of agent a is a conditional plan that specifies what a is going to do for every possible situation. For an agent with no implicit recall of the past, a strategy can be thus represented by function s a : St Act such that s a (q) d a (q), i.e. a function that specifies a valid choice of action for each state of the system. For agents with perfect recall, this would be s a : St + Act such 17

18 that s a (q 0 q 1...q i ) d a (q i ), i.e. one that specifies a valid choice of action for each history of the game. A collective strategy s A for a group of agents A is a tuple of strategies, one per agent from A. Then, by out M (q, s A ) (or just out(q, s A )) we denote the set of all paths that may result from agents A executing strategy s A from state q onward. The semantics of strategic quantifiers is defined as follows: M, q = A ϕ iff there is a strategy s A, such that M, λ = ϕ for every λ out(q, s A ); M, λ = Xϕ iff λ[1.. ] = ϕ; M, λ = Gϕ iff λ[i.. ] = ϕ for all i 0; M, λ = ϕuψ iff λ[i.. ] = ϕ for some i 0, and λ[j.. ] = ϕ for all 0 j i. Additionally, the sometime modality can be defined as: Fϕ Uϕ. We observe that ATL embeds two important logics. First, the branching time logic CTL can be seen as a strategic logic with a very limited set of strategic quantifiers (as the CTL path quantifiers can be embedded in ATL with the following definitions: Eϕ Agt ϕ, Aϕ ϕ). Second, Coalition Logic can be seen as the next -fragment of ATL, with the following embedding of the central modality of CL: [A]ϕ A Xϕ. The above clauses define the semantics of the full version of alternating-time logic, usually called ATL* for historical reasons. It must be noted, however, that the typical variant of ATL, used in the literature, is restricted to formulae in which every temporal operator is immediately preceded by exactly one cooperation modality. We will usually refer to the restricted variant as vanilla ATL. Towards the end of the lecture, we briefly mention some complexity results regarding the satisfiability and model checking problems for ATL, and compare them with analogous results for temporal logics. Bisimulation in ATL. The concept of bisimulation is central in modal logic, and is for example very useful in order to study the key meta-logical property of expressiveness. Bisimulation is a relationship between semantic structures, and it is typically the case that bisimilar structures satisfy exactly the same formulae. Then, we immediately know something about the expressiveness of the logical language: if two structures are bisimilar and one of them has some property while the other has not, then that property is not expressible in the logical language. The following is an adaption [1] of the standard modal logic notion of a bisimulation to CGSs. First, let D(q) = d 1 (q) d k (q) denote the set of joint actions in q, and when a C D(q, C) let next M (q, a C ) = {o(q, ) : D(q), a i = b i for all i C} denote the set of possible next states in CGS M when coalition C choose actions a C. Given two concurrent game structures, M 1 = (Agt, St 1, Act 1, PV, V 1, d 1, o 1 ) and M 2 = (Agt, St 2, Act 2, PV, V 2, d 2, o 2 ), and a set of agents C Agt, a relation β St 1 St 2 is a C-bisimulation between M 1 and M 2, denoted M 1 C β M 2, iff for all states q 1 St 1 and q 2 St 2, we have that q 1 βq 2 implies the following: Local harmony V 1 (q 1 ) = V 2 (q 2 ); 18

19 Forth For all joint actions a 1 C D 1(q 1, C) for C, there exists a joint action a 2 C D 2(q 2, C) for C such that for all states q 2 next M2 (q 2, a 2 C ), there exists a state q 1 next M1 (q 1, a 1 C ) such that q 1βq 2 ; Back Likewise, for 1 and 2 swapped. A relation β is a bisimulation between M 1 and M 2, denoted M 1 β M 2, if β is a C-bisimulation between M 1 and M 2 for every C Agt. Using this notion of bisimulation, we can characterise the expressiveness of ATL as mentioned above. Here, it is relevant to take into account a semantic issue mentioned in lecture 3: the role of memory. Two different ways to define strategies are, first, as mappings from states to actions, and second, as mappings from sequences of states (histories) to actions. We call the first type memoryless strategies, modelling agents who base their actions on the current state only, and the second full strategies, modelling agents who base their strategies on the history of states. The semantics of ATL can be defined using either; by restricing the quantification ( there is a strategy.. ) in the semantic clauses to the appropriate class of strategies. We can now show that: For vanilla ATL, memory does not matter: the interpretation of a formula (its truth value in a state of a structure) is the same whether we use only memoryless strategies or whether we use full strategies. The language of vanilla ATL cannot discern between bisimilar CGSs (neither with memoryless nor with full strategies) For ATL* memory matters: the interpretation of a formula depends on the amount of recall available. Model checking ATL in explicit models. One of the main results concerning ATL states that its formulae can be model-checked in deterministic linear time, analogously to CTL. It is important to emphasize, however, that the result is relative to the number of transitions in the model and the length of the formula. The ATL model checking algorithm from [5] is presented in Figure 2. The algorithm employs the well-known fixpoint characterizations of strategictemporal modalities: A Gϕ ϕ A X A Gϕ A ϕ 1 Uϕ 2 ϕ 2 ϕ 1 A X A ϕ 1 Uϕ 2, and computes a winning strategy step by step (if it exists). That is, it starts with the appropriate candidate set of states ( for U and the whole set St for G), and iterates backwards over A s one-step abilities until the set gets stable. It is easy to see that the algorithm needs to traverse each transition at most once per subformula of ϕ. Note that it does not matter whether perfect recall or memoryless strategies are used: the algorithm is correct for the perfect recall semantics, but it always finds a memoryless strategy. Thus, for a vanilla ATL formula A γ, agents A have a perfect recall strategy to enforce γ iff they have a memoryless strategy to obtain it. 19

20 function mcheck(m, ϕ). ATL model checking. Returns the set of states in model M = Agt, St, PV, V, o for which formula ϕ holds. case ϕ PV : return V (p) case ϕ = ψ : return St \ mcheck(m, ψ) case ϕ = ψ 1 ψ 2 : return mcheck(m, ψ 1) mcheck(m, ψ 2) case ϕ = A Xψ : return pre(m, A, mcheck(m, ψ)) case ϕ = A Gψ : Q 1 := St; Q 2 := mcheck(m, ψ); Q 3 := Q 2; while Q 1 Q 2 do Q 1 := Q 2; Q 2 := pre(m, A, Q 1) Q 3 od; return Q 1 case ϕ = A ψ 1Uψ 2 : Q 1 := ; Q 2 := mcheck(m, ψ 1); Q 3 := mcheck(m, ψ 2); while Q 3 Q 1 do Q 1 := Q 1 Q 3; Q 3 := pre(m, A, Q 1) Q 2 od; return Q 1 end case function pre(m, A, Q). Auxiliary function; returns the exact set of states Q such that, when the system is in a state q Q, agents A can cooperate and enforce the next state to be in Q. return {q α A α Agt\A o(q, α A, α Agt\A ) Q} Fig. 2. The ATL model checking algorithm from [5] Theorem 1 ([5]). Model checking ATL is P-complete, and can be done in time O( M ϕ ), where M is given by the number of transitions in M. Proof (Sketch). Each case of the algorithm is called at most O( ϕ ) times and terminates after O( M ) steps [5]. The latter is shown by translating the model to a two-player game [5], and then solving the invariance game on it in polynomial time [6]. Hardness is shown by a reduction of reachability in And-Or-Graphs, which was shown to be P-complete in [22], to model checking the (constant) ATL-formula 1 Fp in a two player game. In each Or-state it is the turn of player 1 and in each And-state it is player 2 s turn [5]. References: [5, 18, 1]. 3.4 Strategic reasoning for imperfect information ATL does not take into account the epistemic limitations of the agents; it assumes that every agent has complete information about the global state of the system. The Alternating-time Temporal Epistemic Logic ATEL was introduced in [48] as a straightforward combination of the multi-agent epistemic logic and ATL in 20

21 order to formalize reasoning about the interaction of knowledge and abilities of agents and coalitions. ATEL enables specification of various modes and nuances of interaction between knowledge and strategic abilities, e.g.: A ϕ E A A ϕ, E A A ϕ C A A ϕ; i ϕ K i Agt \ {i} ϕ, etc. Models of ATEL are concurrent epistemic game structures (CEGS): M = Agt, St, Act, d, o, V, 1,..., k combining the CGS-based models for ATL and the multi-agent epistemic models. The semantics of ATEL takes the semantic clauses for ATL and those from epistemic logic, and combines them in a straightforward way. While ATEL extends smoothly both ATL and epistemic logic, and provides a rich formalism for reasoning about knowledge and strategic abilities, it also raises a number of conceptual problems. Most importantly, one would expect that an agent s ability to achieve property ϕ should imply that the agent has enough control and knowledge to identify and execute a strategy that enforces ϕ. A number of approaches have been proposed to overcome this problem. Most of the solutions agree that only uniform strategies (i.e., strategies that specify the same choices in indistinguishable states) are really executable. However, in order to identify a successful strategy, the agents must consider not only the courses of action, starting from the current state of the system, but also from states that are indistinguishable from the current one. There are many cases here, especially when group epistemics is concerned: the agents may have common, ordinary, or distributed knowledge about a strategy being successful, or they may be hinted the right strategy by a distinguished member (the leader ), a subgroup ( headquarters committee ) or even another group of agents ( consulting company ). Most existing solutions (ATL ir [45], ETSL [50]) treat only some of the cases (albeit in an elegant way), while others (ATOL [25], Feasible ATEL [26]) offer a more general treatment of the problem at the expense of an overblown logical language. Constructive Strategic Logic (CSL) [24], on the other hand, proposes a solution which we believe to be both intuitive, general and elegant. However, there is a price to pay here, too. In CSL, formulae are interpreted over sets of states rather than single states. We write M, Q = A ϕ to express the fact that A must have a strategy which is successful for all opening states from Q. New epistemic operators K i, E A, C A, D A for practical or constructive knowledge yield the set of states for which a single evidence (i.e., a successful strategy) should be presented (instead of checking if the required property holds in each of the states separately, like standard epistemic operators do). M, Q = A ϕ iff there is s A such that M, λ = ϕ for every λ q Q out(q, s A ); M, Q = K i ϕ iff M, {q q Q q K i q } = ϕ; M, Q = C A ϕ iff M, {q q Q q C A q } = ϕ; M, Q = E A ϕ iff M, {q q Q q E A q } = ϕ; M, Q = D A ϕ iff M, {q q Q q D A q } = ϕ. We point out that in CSL: 21

22 1. K a a ϕ refers to agent a having a strategy de re to enforce ϕ (i.e. having a successful uniform strategy and knowing the strategy); 2. K[a] a ϕ refers to agent a having a strategy de dicto to enforce ϕ (i.e. knowing only that some successful uniform strategy is available); 3. a ϕ expresses that agent a has a uniform strategy to enforce ϕ from the current state (but not necessarily even knows about it). Thus, K i i ϕ captures the notion of i s knowing how to play to achieve ϕ, while K[i] i ϕ refers to knowing only that a successful play is possible. This extends naturally to abilities of coalitions. References: [45, 24]. 22

23 4 Complexity of verification In this lecture, we present an overview of complexity results for model checking temporal and strategic logic. A comprehensive survey on the subject has been presented in [9]. We do not replicate the survey in these lecture notes, and instead refer the reader directly to the original article. References: [9]. Available at wjamroga/papers/mcheck09book.pdf. 23

24 5 Model checking for real time and knowledge We show that the TECTLK model checking problem is decidable, and we present an algorithm for bounded model checking based on a discretisation method. We exemplify the use of the technique by means of the Railroad Crossing System, a popular example in the multi-agent systems literature. 5.1 Epistemic Region Graph and Its Discretisation Any real time interpreted system is dense and hence infinite. To perform model checking efficiently, we consider an appropriately generated finite version of it. In particular, we use an epistemic region graph (ERG), defined as an extension of the region graph [3] augmented to include the relation i, for each agent i Agt. Let Agt be a set of m agents, where each agent is modelled via a timed automaton and TA = (Z, L, l 0, E, X, I) be the parallel composition of them. The epistemic region graph for the timed automaton TA is a tuple where M rg = (S, ι, rg, rg 1,..., rg m, V rg ) S L Z( X ) is a set of reachable states, called regions; note that each element of S is a pair (l, Z) where l is a location and Z is a zone. ι = (l 0, Z 0 ) is the initial region, where Z 0 = {v 0 }; recall that v 0 (x) = 0, for all x X, rg S (Z {τ}) S is defined by: τ Time transition: (l, Z) rg (l, Z ) iff there exist v Z and v Z such that δ a.) (l, v) (l, v ) for some δ IR +, and δ b.) if (l, v) (l, v ) δ (l, v ) and (l, Z ) S for some Z such that v Z, then either v v or v v, and c.) if v v, then v v + δ for each δ IR. a Action transition: For any a Z, (l, Z) rg (l, Z ) iff the following conditions hold: a.) (l, Z) is not boundary 2 and b.) either there exist v Z and v Z a such that (l, v) (l, v ) or there exist Z and v Z τ such that (l, Z) rg (l, Z ) and (l, v a ) (l, v ). rg i S S is a relation defined by (l, Z) i (l, Z ) iff l i (l) = l i (l ) and Z = Z, for each agent i. Obviously i is an equivalence relation. V rg : S 2 PV is a valuation function that extends V TA as follows V rg ((l, Z)) = V TA (l). The epistemic region graph preserves validity of TECTLK formulae. In order to encode it we will discretise the state space by using a discretisation method described in [53] and shortly reported below. 2 A region (l, Z) is boundary if for each δ IR +, v Z, (v v + δ). 24