Admissible Strategies for Synthesizing Systems

Transcription

1 Admissible Strategies for Synthesizing Systems Ocan Sankur Univ Rennes, Inria, CNRS, IRISA, Rennes Joint with Romain Brenguier (DiffBlue), Guillermo Pérez (Antwerp), and Jean-François Raskin (ULB)

2 (Multiplayer) Reactive Synthesis Example uncontrollable (see Epsiode II) G controllable R2D2: Reach the gate G without collision Ocan Sankur Assume-Admissible Synthesis / 22

3 (Multiplayer) Reactive Synthesis Example uncontrollable (see Epsiode II) G controllable R2D2: Reach the gate G without collision Ocan Sankur Assume-Admissible Synthesis / 22

4 (Multiplayer) Reactive Synthesis Example uncontrollable (see Epsiode II) G controllable R2D2: Reach the gate G without collision Ocan Sankur Assume-Admissible Synthesis / 22

5 (Multiplayer) Reactive Synthesis Example uncontrollable (see Epsiode II) G Formalism: multiplayer graph games C (collision) controllable R2D2: Reach the gate G without collision Ocan Sankur Can circle player avoid C? Assume-Admissible Synthesis / 22

6 (Multiplayer) Reactive Synthesis Example uncontrollable (see Epsiode II) G Formalism: multiplayer graph games C (collision) controllable R2D2: Reach the gate G without collision Ocan Sankur Can circle player avoid C? Assume-Admissible Synthesis / 22

7 (Multiplayer) Reactive Synthesis Example uncontrollable (see Epsiode II) G Formalism: multiplayer graph games C (collision) controllable R2D2: Reach the gate G without collision Ocan Sankur Can circle player avoid C? Assume-Admissible Synthesis / 22

8 (Multiplayer) Reactive Synthesis Example uncontrollable (see Epsiode II) G Formalism: multiplayer graph games C (collision) controllable R2D2: Reach the gate G without collision Ocan Sankur Can circle player avoid C? Assume-Admissible Synthesis / 22

9 (Multiplayer) Reactive Synthesis Example uncontrollable (see Epsiode II) G Formalism: multiplayer graph games C (collision) controllable R2D2: Reach the gate G without collision Can circle player avoid C? Winning strategy σ for a player: for all strategies τ for the adversary, (σ, τ ) satisfies his objective very robust Ocan Sankur Assume-Admissible Synthesis / 22

10 System composed of interacting components φ 1 φ 2 φ 3 (φ 4 depends on φ 1 ) φ 4 φ 5 φ 6 φ 7 Example: Each φ i is a robot with a given objective Ocan Sankur Assume-Admissible Synthesis / 22

11 Cooperative approach φ 1 φ 2 φ 3 φ 4 φ 5 φ 6 φ 7 (σ 1,..., σ 7 ) = φ 1 φ 2 φ 3 φ 4 φ 5 φ 6 φ 7 Analyze the whole system, and synthesize a big joint strategy Ocan Sankur Assume-Admissible Synthesis / 22

12 Cooperative approach φ 1 φ 2 φ 3 φ 4 φ 5 φ 6 φ 7 (σ 1,..., σ 7 ) = φ 1 φ 2 φ 3 φ 4 φ 5 φ 6 φ 7 Analyze the whole system, and synthesize a big joint strategy Not compositional, does not scale well Each component relies on a very specific behavior of others: σ 1 only works if others respect σ 2,..., σ 7 ; does not resist any change Ocan Sankur Assume-Admissible Synthesis / 22

13 Adversarial approach Each component controlled by a different player: consider worst case φ 1 φ 2 φ 3 φ 4 φ 5 φ 6 φ 7 Ocan Sankur Assume-Admissible Synthesis / 22

14 Adversarial approach Each component controlled by a different player: consider worst case φ 1 φ 2 φ 3 φ 4 φ 4 φ 5 φ 6 φ 7 Find σ 4 such that τ, σ 4, τ = φ 4 τ is any joint strategy of the surrounding components φ 4 has a winning strategy against other components seen as adversarial Ocan Sankur Assume-Admissible Synthesis / 22

15 Adversarial approach Each component controlled by a different player: consider worst case φ 1 φ 2 φ 3 φ 4 φ 5 φ 6 φ 7 Find σ 4 such that τ, σ 4, τ = φ 4 τ is any joint strategy of the surrounding components Find σ 5 such that τ, σ 5, τ = φ 5 Ocan Sankur Assume-Admissible Synthesis / 22

16 Adversarial approach Each component controlled by a different player: consider worst case φ 1 φ 2 φ 3 φ 4 φ 5 φ 6 φ 7 Find σ 5 such that τ, σ 5, τ = φ 5 σ 1 = φ 1, σ 2 = φ 2, σ 3 = φ 3, σ 4 = φ 4, σ 5 = φ 5, σ 6 = φ 6 and σ 7 = φ 7 Ocan Sankur Assume-Admissible Synthesis / 22

17 Adversarial approach φ 1 φ 2 φ 3 φ 4 φ 5 φ 6 φ 7 Find σ 5 such that τ, σ 5, τ = φ 5 σ 1 = φ 1, σ 2 = φ 2, σ 3 = φ 3, σ 4 = φ 4, σ 5 = φ 5, σ 6 = φ 6 and σ 7 = φ 7 Excellent solution, each component is extremely robust! But: In general, no winning strategy for all components Each component is synthesized against a hostile environment Ocan Sankur Assume-Admissible Synthesis / 22

18 Driver Example User a 1,a 2 Driver r 1,r 2 g 1,g 2 Scheduler A run is a sequence of actions of : User Driver Sched User Driver φ User : arbitrary, no objective, sends actions a 1, a 2 φ Driver : Receive a i eventually send r i (to Scheduler) and do not repeat r i until g i is seen φ Sched : (g 1 g 2 ) (mutex) r 1 send g 1 one or two steps later r 2 send g 2 two steps later Ocan Sankur Assume-Admissible Synthesis / 22

19 Driver Example User a 1,a 2 Driver r 1,r 2 g 1,g 2 Scheduler φ User : arbitrary, no objective, sends actions a 1, a 2 φ Driver : Receive a i eventually send r i (to Scheduler) and do not repeat r i until g i is seen φ Sched : (g 1 g 2 ) (mutex) r 1 send g 1 one or two steps later r 2 send g 2 two steps later No winning strategy for Sched: assume Driver does r 1 r 2 at each step Ocan Sankur Assume-Admissible Synthesis / 22

20 Driver Example User a 1,a 2 Driver r 1,r 2 g 1,g 2 Scheduler φ User : arbitrary, no objective, sends actions a 1, a 2 φ Driver : Receive a i eventually send r i (to Scheduler) and do not repeat r i until g i is seen φ Sched : (g 1 g 2 ) (mutex) r 1 send g 1 one or two steps later r 2 send g 2 two steps later No winning strategy for Driver: assume Sched never sends any g i Ocan Sankur Assume-Admissible Synthesis / 22

21 Pathological Strategies User a 1,a 2 Driver r 1,r 2 g 1,g 2 Scheduler No adversarial solution... Ocan Sankur Assume-Admissible Synthesis / 22

22 Pathological Strategies User a 1,a 2 Driver r 1,r 2 g 1,g 2 Scheduler A cooperative solution where players generate a unique scenario: ρ = (a 1 r 1 g 1 ) (a 2 r 2 ) ( g 2 ) (a 1 r 1 g 1 )... (everyone agrees to alternate between tasks 1 and 2) If Sched does not respect ρ, then Driver switches to (r 1 r 2 ) ω If Driver does not respect ρ, then Sched switches to ( g 1 g 2 ) ω these cases are losing for everyone Ocan Sankur Assume-Admissible Synthesis / 22

23 Pathological Strategies User a 1,a 2 Driver r 1,r 2 g 1,g 2 Scheduler A cooperative solution where players generate a unique scenario: ρ = (a 1 r 1 g 1 ) (a 2 r 2 ) ( g 2 ) (a 1 r 1 g 1 )... (everyone agrees to alternate between tasks 1 and 2) If Sched does not respect ρ, then Driver switches to (r 1 r 2 ) ω If Driver does not respect ρ, then Sched switches to ( g 1 g 2 ) ω these cases are losing for everyone Cooperative synthesis may generate a unique outcome outside of which no guarantee is given Ocan Sankur Assume-Admissible Synthesis / 22

24 Pathological Strategies User a 1,a 2 Driver r 1,r 2 g 1,g 2 Scheduler A cooperative solution where players generate a unique scenario: ρ = (a 1 r 1 g 1 ) (a 2 r 2 ) ( g 2 ) (a 1 r 1 g 1 )... (everyone agrees to alternate between tasks 1 and 2) If Sched does not respect ρ, then Driver switches to (r 1 r 2 ) ω If Driver does not respect ρ, then Sched switches to ( g 1 g 2 ) ω these cases are losing for everyone Cooperative synthesis may generate a unique outcome outside of which no guarantee is given The device driver that only works for one user pattern... Ocan Sankur Assume-Admissible Synthesis / 22

25 Summary and Motivations Two approaches Cooperative approach: 1) often provides solutions 2) does not scale, 3) strategies define a unique scenario, Adversarial approach: 1) excellent robust strategies 2) almost never finds a solution How to improve: If there is no winning strategy, then a good strategy should work as long as the other players are reasonable Cooperative relies too much on other components Adversarial does not rely on them enough Ocan Sankur Assume-Admissible Synthesis / 22

26 Summary and Motivations Two approaches Cooperative approach: 1) often provides solutions 2) does not scale, 3) strategies define a unique scenario, Adversarial approach: 1) excellent robust strategies 2) almost never finds a solution How to improve: If there is no winning strategy, then a good strategy should work as long as the other players are reasonable Cooperative relies too much on other components Adversarial does not rely on them enough Next: Assume-Guarantee rule Ocan Sankur Assume-Admissible Synthesis / 22

27 Back to the example User a 1,a 2 Driver r 1,r 2 g 1,g 2 Scheduler Assume-Guarantee Rule (Chatterjee, Henzinger 2006) We have σ Driver, σ Sched, σ User = φ Driver φ Sched φ User, σ Driver = φ Sched φ Driver, σ Sched = φ Driver φ Sched. Each component should satisfy φ i assuming other objectives are being satisfied Ocan Sankur Assume-Admissible Synthesis / 22

28 Back to the example User a 1,a 2 Driver r 1,r 2 g 1,g 2 Scheduler Assume-Guarantee Rule (Chatterjee, Henzinger 2006) We have σ Driver, σ Sched, σ User = φ Driver φ Sched φ User, σ Driver = φ Sched φ Driver, σ Sched = φ Driver φ Sched. Consider again the unique scenario (alternate between 1 and 2): ρ = (a 1 r 1 g 1 ) (a 2 r 2 ) ( g 2 ) (a 1 r 1 g 1 )... If Sched does not respect ρ, then Driver switches to (r 1 r 2 ) ω If Driver does not respect ρ, then Sched switches to ( g 1 g 2 ) ω these cases are losing for everyone Ocan Sankur Assume-Admissible Synthesis / 22

29 Back to the example User a 1,a 2 Driver r 1,r 2 g 1,g 2 Scheduler Assume-Guarantee Rule (Chatterjee, Henzinger 2006) We have σ Driver, σ Sched, σ User = φ Driver φ Sched φ User, σ Driver = φ Sched φ Driver, σ Sched = φ Driver φ Sched. Assume-Guarantee strategies may not be trying hard enough This paper: Rule to synthesize strategies that always try hard to satisfy its objectives Ocan Sankur Assume-Admissible Synthesis / 22

30 Admissible Strategies How would the players play if there is no winning strategy? Starters: Games with Boolean objectives Quantitative Games Ocan Sankur Assume-Admissible Synthesis / 22

31 Admissible Strategies How would the players play if there is no winning strategy? Some strategies are better than others. Goal: Understand rational behaviors of players, compare strategies,... Ocan Sankur Assume-Admissible Synthesis / 22

32 Admissible Strategies How would the players play if there is no winning strategy? Some strategies are better than others. Goal: Understand rational behaviors of players, compare strategies,... Admissibility in Boolean Games For a player with objective φ, strategy σ is dominated by σ for all strategies τ of the other players, if (σ, τ) = φ (σ, τ) = φ and for some strategy τ 0, (σ, τ 0 ) = φ and (σ, τ 0 ) = φ. σ is clearly a more clever choice than σ a non-dominated strategy is called admissible On graph games with Boolean objectives: Berwanger 2007, Faella 2009, Brenguier, Raskin, Sassolas 2014 Ocan Sankur Assume-Admissible Synthesis / 22

33 Admissible Strategies How would the players play if there is no winning strategy? Some strategies are better than others. Goal: Understand rational behaviors of players, compare strategies,... Admissibility in Boolean Games For a player with objective φ, strategy σ is dominated by σ for all strategies τ of the other players, if (σ, τ) = φ (σ, τ) = φ and for some strategy τ 0, (σ, τ 0 ) = φ and (σ, τ 0 ) = φ. σ is clearly a more clever choice than σ a non-dominated strategy is called admissible On graph games with Boolean objectives: Berwanger 2007, Faella 2009, Brenguier, Raskin, Sassolas 2014 A strict partial-order on strategies... Admissible = maximal Ocan Sankur Assume-Admissible Synthesis / 22

34 Examples of Dominated and Admissible Strategies Playing an admissible strategy trying to satisfy his/her objective G dominated by G Ocan Sankur Assume-Admissible Synthesis / 22

35 Examples of Dominated and Admissible Strategies Playing an admissible strategy trying to satisfy his/her objective G dominated by G Both strategies are admissible: err Ocan Sankur Assume-Admissible Synthesis / 22

36 Examples of Dominated and Admissible Strategies Playing an admissible strategy trying to satisfy his/her objective G dominated by G Both strategies are admissible: err Ocan Sankur Assume-Admissible Synthesis / 22

37 Examples of Dominated and Admissible Strategies Playing an admissible strategy trying to satisfy his/her objective G dominated by G Both strategies are admissible: err Ocan Sankur Assume-Admissible Synthesis / 22

38 Examples of Dominated and Admissible Strategies Playing an admissible strategy trying to satisfy his/her objective G dominated by G If there is a winning strategy, then admissible = winning err Ocan Sankur Assume-Admissible Synthesis / 22

39 Motivations Comparing strategies Rational behaviors If players do not know the strategies of other players? Nash equilibria requires the knowledge about everyone s strategies For controller synthesis Simplification: The strategy space is reduced if we remove the dominated strategies Better and more frequent solutions if irrational behaviors are discarded (see next) Good candidate for assume-guarantee reasoning for games Ocan Sankur Assume-Admissible Synthesis / 22

40 Motivations Comparing strategies Rational behaviors If players do not know the strategies of other players? Nash equilibria requires the knowledge about everyone s strategies For controller synthesis Simplification: The strategy space is reduced if we remove the dominated strategies Better and more frequent solutions if irrational behaviors are discarded (see next) Good candidate for assume-guarantee reasoning for games Next: How to compute / represent the set of admissible strategies And how to reason about possible behaviors when all players are admissible Ocan Sankur Assume-Admissible Synthesis / 22

41 Assume-Admissible Rule P: set of players Rule AA An assume-admissible (AA) strategy profile is (σ i ) i P such that: for all i P, σ i Adm i ; for all i P, σ i Adm i. σ i, σ i = φ i. Compute winning strategies under admissibility assumption Ocan Sankur Assume-Admissible Synthesis / 22

42 Example with the AA rule User a 1,a 2 Driver r 1,r 2 g 1,g 2 Scheduler φ Driver : Receive a i eventually send r i (to Scheduler) and do not repeat r i until g i is seen φ Sched : (g 1 g 2 ) (mutex) r 1 send g 1 one or two steps later; r 2 send g 2 two steps later Ocan Sankur Assume-Admissible Synthesis / 22

43 Example with the AA rule User a 1,a 2 Driver r 1,r 2 g 1,g 2 Scheduler φ Driver : Receive a i eventually send r i (to Scheduler) and do not repeat r i until g i is seen φ Sched : (g 1 g 2 ) (mutex) r 1 send g 1 one or two steps later; r 2 send g 2 two steps later Admissible strategies for Driver: do not emit a new request before the previous one has been granted if previous request has been granted, upon user s request, eventually emit a new request Ocan Sankur Assume-Admissible Synthesis / 22

44 Example with the AA rule User a 1,a 2 Driver r 1,r 2 g 1,g 2 Scheduler Admissible strategies for Driver: do not emit a new request before the previous one has been granted if previous request has been granted, upon user s request, eventually emit a new request Admissible strategies for Sched: if request r 2 (but not r 1 ) was made 2 steps ago, then grant g 2 ; if request r 1 (but not r 2 ) was made 2 steps ago, then grant g 1 ; if r 1 and r 2 were made 1 step ago, then grant g 1 ; otherwise behave arbitrarily (specification will fail anyway) Ocan Sankur Assume-Admissible Synthesis / 22

45 Example with the AA rule User a 1,a 2 Driver r 1,r 2 g 1,g 2 Scheduler Admissible strategies for Driver: do not emit a new request before the previous one has been granted if previous request has been granted, upon user s request, eventually emit a new request Admissible strategies for Sched: if request r 2 (but not r 1 ) was made 2 steps ago, then grant g 2 ; if request r 1 (but not r 2 ) was made 2 steps ago, then grant g 1 ; if r 1 and r 2 were made 1 step ago, then grant g 1 ; otherwise behave arbitrarily (specification will fail anyway) All combinations of these strategies are solutions of AA Ocan Sankur Assume-Admissible Synthesis / 22

46 Assume-Admissible Strategies for all i P, σ i Adm i ; for all i P, σ i Adm i. σ i, σ i = φ i. Advantages of AA: Definition is just like adversarial but with admissibility assumption: Provided strategies are winning against all admissible strategies Robust (assuming admissibility) gives a set of strategies for each component such that any combination satisfies all objectives Ocan Sankur Assume-Admissible Synthesis / 22

47 Assume-Admissible Strategies for all i P, σ i Adm i ; for all i P, σ i Adm i. σ i, σ i = φ i. Advantages of AA: Definition is just like adversarial but with admissibility assumption: Provided strategies are winning against all admissible strategies Robust (assuming admissibility) gives a set of strategies for each component such that any combination satisfies all objectives Rectangularity The set of solutions of AA is a Cartesian product of the set of the AA-winning strategies of each player Consequence: each player can choose his/her strategy separately No pathological strategy profile! Ocan Sankur Assume-Admissible Synthesis / 22

48 Main Results Assume-Admissible synthesis AA-synthesis for Büchi objectives is PTIME-complete AA-synthesis for Müller objectives is PSPACE-complete a Abstraction framework: AA-strategies for each player can be computed on an abstract game a Same complexity as solving zero-sum Müller games Each strategy can be computed separately, using possibly a different abstraction Algorithm overview Ocan Sankur Assume-Admissible Synthesis / 22

49 Assume-Admissible: Algorithm C 2 G 4 5 Circle: Avoid state C Rectangle: Reach G No player has a winning strategy Ocan Sankur Assume-Admissible Synthesis / 22

50 Assume-Admissible: Algorithm C 2 G 4 5 Circle player s admissible strategies: do not take any dotted edges Ocan Sankur Assume-Admissible Synthesis / 22

51 Assume-Admissible: Algorithm C 2 G 4 5 Rectangle player s admissible strategies: do not take dotted edges + infinitely often go to 1 so that Circle player can help Ocan Sankur Assume-Admissible Synthesis / 22

52 Assume-Admissible: Algorithm C 2 G 4 5 Overall any pair of admissible strategies are conform to this graph + Rectangle will eventually reach 1 Any pair of strategies in this graph with the above assumption are admissible and winning Ocan Sankur Assume-Admissible Synthesis / 22

53 Assume-Admissible Synthesis Algorithm Algorithm 1 For each player, identify states (Identify winning, maybe winning, and definitely losing states) 2 Remove players edges that decrease their own value 3 Compute the help states (state 1 in the example) 4 Solve the resulting game for each player Algorithm corresponds to one-step elimination procedure of [Brenguier, Raskin, Sassolas, LICS-CSL 2014] + PTIME algorithm for Büchi + Abstraction framework Ocan Sankur Assume-Admissible Synthesis / 22

54 Quick Idea of Abstractions To find a strategy for Rectangle Simplify the state space by merging states in the same blue classes C 2 G 4 5 Ocan Sankur Assume-Admissible Synthesis / 22

55 Quick Idea of Abstractions G 3 C5 Abstract Assume-Admissible Synthesis Use abstractions to compute Adm i Adm i Adm i. Then find σ i Adm i such that σ i Adm i, (σ i, σ i ) = φ i Abstract assume-admissible implies normal assume-admissible It is harder to satisfy, but easier to compute Ocan Sankur Assume-Admissible Synthesis / 22

56 Computing Approximations of Adm i (G) Reminder: To define Adm i (G), remove player-i s bad transitions Win Maybe Lose Ocan Sankur Assume-Admissible Synthesis / 22

57 Computing Approximations of Adm i (G) Reminder: To define Adm i (G), remove player-i s bad transitions Win Maybe Lose Ocan Sankur Assume-Admissible Synthesis / 22

58 Computing Approximations of Adm i (G) Abstract computation: we cannot (and don t want to) compute Win, Maybe, Lose exactly. To define Adm i (G), use under-approximations removes more transitions To define Adm i (G), use over-approximations removes less transitions Win Maybe Lose Ocan Sankur Assume-Admissible Synthesis / 22

59 Computing Approximations of Adm i (G) Abstract computation: we cannot (and don t want to) compute Win, Maybe, Lose exactly. To define Adm i (G), use under-approximations removes more transitions To define Adm i (G), use over-approximations removes less transitions Ocan Sankur Assume-Admissible Synthesis / 22

60 Abstract Assume-Admissible Abstract Algorithm Given game G, and its abstraction G a : 1 For all players i, compute Adm i, Adm i. 2 For each player i, solve G a [ Adm i, j Adm j ]. (restrict each player i to Adm i, and players j i to Adm j ) 3 Compute a winning strategy for i in the restricted game. 4 If each player i has a winning strategy: return the profile Otherwise no solution Ocan Sankur Assume-Admissible Synthesis / 22

61 Abstract Assume-Admissible Abstract Algorithm Given game G, and its abstraction G a : 1 For all players i, compute Adm i, Adm i. 2 For each player i, solve G a [ Adm i, j Adm j ]. (restrict each player i to Adm i, and players j i to Adm j ) 3 Compute a winning strategy for i in the restricted game. 4 If each player i has a winning strategy: return the profile Otherwise no solution A bit more complicated for ω-regular objectives Ocan Sankur Assume-Admissible Synthesis / 22

62 Conclusions Assume-Admissible Synthesis One can take all players objectives into account Strategies are robust: no pathological behaviors; rectangular profiles Admissibility is a simple form of rationality (not the strongest one) Abstract interpretation can be applied Ocan Sankur Assume-Admissible Synthesis / 22

63 Conclusions Assume-Admissible Synthesis One can take all players objectives into account Strategies are robust: no pathological behaviors; rectangular profiles Admissibility is a simple form of rationality (not the strongest one) Abstract interpretation can be applied Extensions Quantitative games Concurrent games, timed games Imperfect information Ocan Sankur Assume-Admissible Synthesis / 22