Peled, Vardi, & Yannakakis: Black Box Checking

Transcription

1 Peled, Vardi, & Yannakakis: Black Box Checking Martin Leucker Department of Computer Systems,, Sweden

2 Plan Preliminaries State identification and verification Conformance Testing Extended and Communicating Finite State Machines Related Topics Peled, Vardi, & Yannakakis:Black Box Checking 2

3 Motivation Combination of Model Checking and Testing Peled, Vardi, & Yannakakis:Black Box Checking 3

4 Assumptions n upper bound on number of states unique reset input alphabet Σ is know experiment is: input symbol from Σ or reset, output is action enabled or action not enabled when action enabled, system makes move. when not, system stays in current state no backtracking available system is deterministic Peled, Vardi, & Yannakakis:Black Box Checking 4

5 Büchi Automata fix Σ with Σ = p Büchi automaton: (S, S 0, Σ, δ, F ) implementation automaton B is Büchi automaton where S n S 0 = {s 0 } F = S deterministic?! implementation automaton as Mealy machine: λ(s, a) = { success fail if δ(s, a) defined otherwise Peled, Vardi, & Yannakakis:Black Box Checking 5

6 Büchi Automata II specification automaton P S = m intersection automaton: δ = {((s, s ), α, (t, t )) (s, α, t) δ B (s, α, t ) δ P } Peled, Vardi, & Yannakakis:Black Box Checking 6

7 Experiment Finite sequence α 1 α 2... α k 1 (Σ {reset}) such that there exists a sequence of states s 1 s 2... s k of B, with s 1 S0 B, and for each 1 j < k, either a j = reset and s j+1 = s 0, or (s j, α j, s j+1 ) δ B, or there is no t S B such that (s j, α j, t) δ B and s j+1 = s j Peled, Vardi, & Yannakakis:Black Box Checking 7

8 Games -player and -player ( -player is stupid) Positions: Configurations C C = C i W + W C W + error positions W no error present M (C i C ) L C M (C i C ) L C (c, a, c ), (c, a, c ) M implies c = c alternating game play: sequence from (CL CL ) C, conforming with moves winning: ending in W := W + W Peled, Vardi, & Yannakakis:Black Box Checking 8

9 Information sets partition on configurations notation: c 1 c 2 c 1 c 2 implies (c 1, a, c 1 ) M iff (c 2, a, c 2 ) M c 1 c 2 implies c 1 W + iff c 2 W + (also W, C i?) Peled, Vardi, & Yannakakis:Black Box Checking 9

10 Strategies Deterministic strategy: st : C (L {init}) M such that play is winning, when played according to strategy c 1 c 2 implies st (c 1, l) and st (c 2, l) agree on label Non-deterministic strategy: nst : C (L {init}) M such that if for c there is a play to W +, the play according to nst will end in W + Deterministic time complexity: length of the longest winning path in a deterministic strategy Peled, Vardi, & Yannakakis:Black Box Checking 10

11 Combination Lock Automata β 1 β 2 β 3 β 4 S 1 S 2 S 3 S 4 S 5 β 1 β 2 β 3 β 4 β 1 β 2 β 1 β 2 β 3 β 4 Peled, Vardi, & Yannakakis:Black Box Checking 11

12 Black Box Deadlock Detection Peled, Vardi, & Yannakakis:Black Box Checking 12

13 The setting Let A be the set of all automata with at most n states. Configurations: (A, s, h) A automaton in A s (current) state of A h history Initial configurations: (A, s 0, ɛ) Peled, Vardi, & Yannakakis:Black Box Checking 13

14 Moves -player: (A, s, h) a (A, s, h ) Either or a Σ s = δ(s, a) if defined s = s otherwise a = resetb, b Σ s = δ(s 0, b) if defined s = s 0 otherwise -player: (A, s, h) a (A, s, h ) a = success, if previous move was possible a = fail, else. Peled, Vardi, & Yannakakis:Black Box Checking 14

15 Winning regions The labels of moves of the -player is an experiment on A Let c 1 c 2, iff there are two plays ξ 1 and ξ 2 leading to c 1 and c 2, resp., and ξ 1 and ξ 2 induce the same experiment. W + contains only configurations with automata having a deadlock W contains only configurations with automata having a nodeadlock Peled, Vardi, & Yannakakis:Black Box Checking 15

16 A non-deterministic strategy The labels of moves of the -player is an experiment on A -player guesses a machine A a run of A of length at most n -player plays on A according to run. When deadlock state is reached, it tries out every a Σ. Note: -player has a (winning) strategy iff A has a deadlock. -player should choose B W + should contain configurations so that indeed every a has been checked Complexity: O(n + p) Peled, Vardi, & Yannakakis:Black Box Checking 16

17 A deterministic strategy -player checks systematically all sequences up to length n 1, starting from initial state. Note: -player has a winning strategy iff A has a deadlock. W should guarantee that all p n sequences have been checked Complexity: O(p n ) Peled, Vardi, & Yannakakis:Black Box Checking 17

18 Complexity At least p n 1. If not, then there is sequence β 1 β 2... β n 1 missing. Consider combination lock automaton for β 1 β 2... β n 1. Same game. But the latter has a deadlock. Peled, Vardi, & Yannakakis:Black Box Checking 18

19 Checking properties of Black Box Finite State Machines Peled, Vardi, & Yannakakis:Black Box Checking 19

20 Problem Given a specification Büchi automaton P with m states, and a black box implementation automaton B with no more than n states, check if there is a sequence accepted by both P and B. Peled, Vardi, & Yannakakis:Black Box Checking 20

21 Complexity At least p n 1. Proof: Weird, similar to the previous proof Peled, Vardi, & Yannakakis:Black Box Checking 21

22 Off-Line strategy Learn B Do model checking with B and P Complexity: Learning + model checking string of length 2n 1 distinguishes two machines try all p 2n 1 and collect output See [22] O(np 2n 1 ) Model checking: O(pmn) Peled, Vardi, & Yannakakis:Black Box Checking 22

23 On-the-fly strategy Non-deterministic: -player guesses path σ of P σ = σ 1 σ 2, σ i mn δ(s 0, σ 1 ) = δ(δ(s 0, σ 1 ), σ 2 ) = t F Black box B must allow σ 1 σ n+1 2 after reset Consider : B P (s 1, s 0 ) σ 1 (s 2, t) σ 2 (s 3, t) σ 2... (s n+3, t) Hence, cycle. Complexity: O(nmn) = O(n 2 m). Peled, Vardi, & Yannakakis:Black Box Checking 23

24 On-the-fly strategy II Deterministic: try out all combinations for σ 1, σ 2 Complexity: O(n 2 p 2mn m) p 2mn choices for path length bounded by mn n + 1 times O(nmnp 2mn ) Peled, Vardi, & Yannakakis:Black Box Checking 24

25 A strategy based on Learning and Testing Peled, Vardi, & Yannakakis:Black Box Checking 25

26 Using Conformance Testing procedure by Vasilevskii and Chow input: assumed model M and black box B Provides a string showing discrepancy, if there is one Automata Learning Algorithm by Angluin using membership queries and equivalence test answered by oracle, providing counter example Peled, Vardi, & Yannakakis:Black Box Checking 26

27 General procedure construct sequence M 1, M 2,... converging to B membership queries are answered by B for equivalence do check L(M i ) L(P ). If so, then it contains σ 1 σ ω 2. Experiment resetσ 1σ n+1 2 on B. if experiment succeeds, we have a word in L(B) L(P ). Error. if not, M i and B are not equivalent. Feed counter example to Angluin s alg. If L(M i ) L(P ) =, check for M i and B for equivalence, using VC alg. Peled, Vardi, & Yannakakis:Black Box Checking 27

28 Done Discussion Peled, Vardi, & Yannakakis:Black Box Checking 28