EECS 144/244: Fundamental Algorithms for System Modeling, Analysis, and Optimization

Transcription

1 EECS 144/244: Fundamental Algorithms for System Modeling, Analysis, and Optimization Discrete Systems Lecture: State-Space Exploration Stavros Tripakis University of California, Berkeley Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 1 / 52

2 ANALYSIS OF DISCRETE SYSTEMS: STATE-SPACE EXPLORATION Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 2 / 52

3 State-Space Exploration Goal: explore all reachable states of the system (modeled as a transition system). Main application: exhaustive verification by reachability analysis. Check that system is never in an incorrect state (safety) deadlock state state which does not satisfy a given (state) property e.g., train is at intersection but gate is not lowered autopilot is off but pilot thinks it is on... Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 3 / 52

4 State-Space Exploration Goal: explore all reachable states of the system (modeled as a transition system). Main application: exhaustive verification by reachability analysis. Check that system is never in an incorrect state (safety) deadlock state state which does not satisfy a given (state) property e.g., train is at intersection but gate is not lowered autopilot is off but pilot thinks it is on... Also the basis for checking liveness properties: every so often system does something useful. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 3 / 52

5 State-Space Exploration For finite-state systems, it can be done fully automatically! (in principle) Forms the basis for model-checking Turing award 2007: Clarke, Emerson, Sifakis. Established practice in the industry (mainly hardware, but increasingly also software). Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 4 / 52

6 Model-Checking and State-Space Exploration Model-checking: check a model M (transition system) against a specification φ: M = φ? Often φ is a temporal logic formula: If φ is an LTL formula: M = φ means σ = φ for every trace of M. If φ is a CTL formula: M = φ means s0 = φ for every initial state s 0 of M. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 5 / 52

7 Model-Checking and State-Space Exploration Model-checking: check a model M (transition system) against a specification φ: M = φ? Often φ is a temporal logic formula: If φ is an LTL formula: M = φ means σ = φ for every trace of M. If φ is a CTL formula: M = φ means s0 = φ for every initial state s 0 of M. State-space exploration: the basis for model-checking. E.g., Can build monitor M φ and reduce the LTL model-checking question to a state-space exploration question about the composition of M and M φ (we will revisit this when we talk about composition). Can extend symbolic reachability analysis (later in this lecture) to CTL model-checking. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 5 / 52

8 State-Space Exploration Algorithms Enumerative (also called explicit state ). Symbolic Bounded model-checking using SAT/SMT solvers. Symbolic reachability. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 6 / 52

9 An Enumerative Algorithm: Depth-First Search Assume given: Kripke structure (P, S, S 0, L, R). main: 1: V := ; /* V : set of visited states */ 2: for all s S 0 do 3: DFS(s); 4: end for DFS(s): 1: check s; /* is s a deadlock? is given p L(s)?... */ 2: V := V {s}; 3: for all s such that (s, s ) R do 4: if s V then 5: DFS(s ); /* recursive call */ 6: end if 7: end for Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 7 / 52

10 An Enumerative Algorithm: Depth-First Search Let s simulate the algorithm on this graph. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 8 / 52

11 An Enumerative Algorithm: Depth-First Search Quiz: Does the algorithm terminate? Does it visit all reachable states? Does it visit any unreachable states? What is the complexity of the algorithm? Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 9 / 52

12 Enumerative Methods Many algorithms: DFS, BFS, A*,... Many approaches to combat state-space explosion: partial-order reduction, symmetry reduction, bit-state hashing,... In-depth discussion: Computer-Aided Verification course (see also bibliography). Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 10 / 52

13 SYMBOLIC METHODS Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 11 / 52

14 Symbolic Methods: Why? The plague of exhaustive verification: state explosion. A chip with 100 flip-flops: (potentially reachable) states. That is states. Even if each state costs 1 bit to store, this still makes = 2 32 = 4, 294, 967, 296 exabytes... Symbolic methods aim to improve this. A seminal paper was subtitled Symbolic model checking: states and beyond. [Burch et al., 1990] is still less than 2 67, but a great leap forward at that time. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 12 / 52

15 Symbolic Representation of State Spaces Key idea: Instead of reasoning about individual states, reason about sets of states. How do we represent a set of states? Symbolic representation: Set = predicate. Set of states = predicate on state variables. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 13 / 52

16 Symbolic Representation of Sets of States Examples: 1. Assume 3 state variables, p, q, r, of type boolean. S 1 : p q Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 14 / 52

17 Symbolic Representation of Sets of States Examples: 1. Assume 3 state variables, p, q, r, of type boolean. S 1 : p q = {pqr, pqr, pqr, pqr, pqr, pqr} Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 14 / 52

18 Symbolic Representation of Sets of States Examples: 1. Assume 3 state variables, p, q, r, of type boolean. S 1 : p q = {pqr, pqr, pqr, pqr, pqr, pqr} 2. Assume 3 state variables, x, i, b, of types real, integer, boolean. How many states are in S 2? S 2 : x > 0 (b i 0) Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 14 / 52

19 Symbolic Representation of Transition Relations Key idea: Use a predicate on two copies of the state variables: unprimed (current state) + primed (next state). If x is the vector of state variables, then the transition relation R is a predicate on x and x : R( x, x ) e.g., R(x, i, b, x, i, b ) Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 15 / 52

20 Symbolic Representation of Transition Relations Examples: 1. Assume one state variable, p, of type boolean. R 1 : (p p ) ( p p ) Which transition relation does this represent? Is it a relation or a function (deterministic)? Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 16 / 52

21 Symbolic Representation of Transition Relations Examples: 1. Assume one state variable, p, of type boolean. R 1 : (p p ) ( p p ) Which transition relation does this represent? Is it a relation or a function (deterministic)? 2. Assume one state variable, n, of type integer. R 2 : n = n + 1 n = n Which transition relation does this represent? Is it a relation or a function (deterministic)? Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 16 / 52

22 Symbolic Representation of Kripke Structures Kripke structure: (P, S, S 0, L, R) Symbolic representation: (P, Init, Trans) where P = {x 1, x 2,..., x n }: set of (boolean) state variables, also also taken to be the atomic propositions. 1 Predicate Init( x) on vector x = (x 1,..., x n ) represents the set S 0 of initial states. Predicate Trans( x, x ) represents the transition relation R. 1 this is done for simplicity, the two could be separated Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 17 / 52

23 Symbolic Representation of Kripke Structures Kripke structure: (P, S, S 0, L, R) Symbolic representation: (P, Init, Trans) where P = {x 1, x 2,..., x n }: set of (boolean) state variables, also also taken to be the atomic propositions. 1 Predicate Init( x) on vector x = (x 1,..., x n ) represents the set S 0 of initial states. Predicate Trans( x, x ) represents the transition relation R. Basis of the language of NuSMV. 1 this is done for simplicity, the two could be separated Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 17 / 52

24 Example: NuSMV model MODULE inverter(input) VAR output : boolean; INIT output = FALSE TRANS next(output) =!input next(output) = output What is the Kripke structure defined by this NuSMV program? Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 18 / 52

25 Example: NuSMV model MODULE inverter(input) VAR output : boolean; INIT output = FALSE TRANS next(output) =!input next(output) = output What is the Kripke structure defined by this NuSMV program? What about P and L? Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 18 / 52

26 Example: Kripke Structure Represent this symbolically. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 19 / 52

27 FINITE-HORIZON REACHABILITY (a.k.a. BOUNDED MODEL-CHECKING) Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 20 / 52

28 Bounded Model-Checking Question: Can a bad state be reached in up to n steps (transitions)? i.e., does there exist a path s 0, s 1,..., s k where k n, s 0 S 0 and s k Bad, for some given set Bad. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 21 / 52

29 Bounded Model-Checking Question: Can a bad state be reached in up to n steps (transitions)? i.e., does there exist a path s 0, s 1,..., s k where k n, s 0 S 0 and s k Bad, for some given set Bad. Key idea: Reduce the above question to a SAT (satisfiability) problem. SAT problem NP-complete for propositional logic, generally undecidable for predicate logic. In practice, today s SAT solvers can handle formulas with thousands of variables. SMT (SAT Modulo Theory) solvers also making good progress. BMC exploits this. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 21 / 52

30 Bounded Model-Checking (BMC) Suppose I have predicates Init( x), Trans( x, x ), and Bad( x). How to use them for BMC? Bad state reachable in 0 steps iff SAT ( ) Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 22 / 52

31 Bounded Model-Checking (BMC) Suppose I have predicates Init( x), Trans( x, x ), and Bad( x). How to use them for BMC? Bad state reachable in 0 steps iff SAT ( Init( x) Bad( x) ) Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 22 / 52

32 Bounded Model-Checking (BMC) Suppose I have predicates Init( x), Trans( x, x ), and Bad( x). How to use them for BMC? Bad state reachable in 0 steps iff SAT ( Init( x) Bad( x) ) Bad state reachable in 1 step iff SAT ( ) Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 22 / 52

33 Bounded Model-Checking (BMC) Suppose I have predicates Init( x), Trans( x, x ), and Bad( x). How to use them for BMC? Bad state reachable in 0 steps iff SAT ( Init( x) Bad( x) ) Bad state reachable in 1 step iff SAT ( Init( x 0 ) Trans( x 0, x 1 ) Bad( x 1 ) ) Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 22 / 52

34 Bounded Model-Checking (BMC) Suppose I have predicates Init( x), Trans( x, x ), and Bad( x). How to use them for BMC? Bad state reachable in 0 steps iff SAT ( Init( x) Bad( x) ) Bad state reachable in 1 step iff SAT ( Init( x 0 ) Trans( x 0, x 1 ) Bad( x 1 ) )... Bad state reachable in n steps iff SAT ( ) Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 22 / 52

35 Bounded Model-Checking (BMC) Suppose I have predicates Init( x), Trans( x, x ), and Bad( x). How to use them for BMC? Bad state reachable in 0 steps iff SAT ( Init( x) Bad( x) ) Bad state reachable in 1 step iff SAT ( Init( x 0 ) Trans( x 0, x 1 ) Bad( x 1 ) )... Bad state reachable in n steps iff SAT ( Init( x 0 ) Trans( x 0, x 1 ) Trans( x n 1, x n ) Bad( x n ) ) Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 22 / 52

36 BMC Outer Loop 1: for all k = 0, 1,..., n do 2: φ := Init( x 0) Trans( x 0, x 1) Trans( x k 1, x k ) Bad( x k ); 3: if SAT(φ) then 4: print Bad state reachable in k steps ; 5: output solution as counter-example; 6: end if 7: end for 8: print Bad state unreachable up to n steps ; Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 23 / 52

37 SAT Solving SAT: given propositional logic formula, check whether it is satisfiable, and return a solution if it is. Usually formula given in CNF: Conjunctive Normal Form. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 24 / 52

38 CNF and DNF Literal: a variable x or its negation x. Clause: a disjunction of literals. E.g.: clause 1 : x + y clause 2 : x + z + w CNF: conjunction of clauses, i.e., conjunction of disjunctions of literals (also called POS - product of sums ). E.g.: (x + y) (x + z + w) DNF: disjunction of conjunctions of literals (also called SOP - sum of products ). E.g.: (xy) + (xzw) + Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 25 / 52

39 Translations to/from CNF and DNF Every formula can be trivially transformed into DNF. How? Are there more efficient ways to transform into DNF? (Hint: how easy is it to check whether a DNF formula is SAT? how hard is SAT?) Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 26 / 52

40 Translations to/from CNF and DNF Every formula can be trivially transformed into DNF. How? Are there more efficient ways to transform into DNF? (Hint: how easy is it to check whether a DNF formula is SAT? how hard is SAT?) NNF (Negation Normal Form): all negations are pushed into literals. E.g.: (x y) (z w) Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 26 / 52

41 Translations to/from CNF and DNF Every formula can be trivially transformed into DNF. How? Are there more efficient ways to transform into DNF? (Hint: how easy is it to check whether a DNF formula is SAT? how hard is SAT?) NNF (Negation Normal Form): all negations are pushed into literals. E.g.: (x y) (z w) ( (x y)) (z w) Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 26 / 52

42 Translations to/from CNF and DNF Every formula can be trivially transformed into DNF. How? Are there more efficient ways to transform into DNF? (Hint: how easy is it to check whether a DNF formula is SAT? how hard is SAT?) NNF (Negation Normal Form): all negations are pushed into literals. E.g.: (x y) (z w) ( (x y)) (z w) ( x y) (z w) Homework: write a procedure to convert any boolean expression to NNF. Given a formula in NNF, how to transform it into CNF? Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 26 / 52

43 Translations to/from CNF and DNF 1: CNF(φ): 2: if φ is a literal then 3: return φ; 4: else if φ is φ 1 φ 2 then 5: return CNF(φ 1 ) CNF(φ 2 ); 6: else if φ is φ 1 φ 2 then 7: return DistributeOr ( CNF(φ 1 ), CNF(φ 2 ) ) ; 8: else 9: error: φ not in NNF; 10: end if 1: DistributeOr(φ 1, φ 2 ): 2: if φ 1 is φ 11 φ 12 then 3: return DistributeOr(φ 11, φ 2 ) DistributeOr(φ 12, φ 2 ); 4: else if φ 2 is φ 21 φ 22 then 5: return DistributeOr(φ 1, φ 21 ) DistributeOr(φ 1, φ 22 ); 6: else 7: return φ 1 φ 2 ; /* both must be literals at this point */ 8: end if Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 27 / 52

44 Translations to/from CNF and DNF 1: CNF(φ): 2: if φ is a literal then 3: return φ; 4: else if φ is φ 1 φ 2 then 5: return CNF(φ 1 ) CNF(φ 2 ); 6: else if φ is φ 1 φ 2 then 7: return DistributeOr ( CNF(φ 1 ), CNF(φ 2 ) ) ; 8: else 9: error: φ not in NNF; 10: end if 1: DistributeOr(φ 1, φ 2 ): 2: if φ 1 is φ 11 φ 12 then 3: return DistributeOr(φ 11, φ 2 ) DistributeOr(φ 12, φ 2 ); 4: else if φ 2 is φ 21 φ 22 then 5: return DistributeOr(φ 1, φ 21 ) DistributeOr(φ 1, φ 22 ); 6: else 7: return φ 1 φ 2 ; /* both must be literals at this point */ 8: end if How large can CNF(φ) be in the worst-case? Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 28 / 52

45 Translations to/from CNF and DNF More efficient translations to CNF exist: they preserve satisfiability of the original formula, by adding extra variables. Example: ab + cd (a + c)(a + d)(b + c)(b + d) }{{} normal translation ab + cd (z 1 + z 2 )(z 1 + a)(z 1 + b)(z 2 + c)(z 2 + d) }{{} translation adding variables z 1, z 2 Idea: z 1 represents ab and z 2 represents cd. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 29 / 52

46 SAT Solving SAT: given propositional logic formula, check whether it is satisfiable, and return a solution if it is. Usually formula given in CNF: Conjunctive Normal Form. Solvers exploit this form, e.g.: Easy to detect conflicts (unsatisfiability). Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 30 / 52

47 SAT Solving Let φ be a formula in CNF. Let x 1, x 2,..., x n be the variables appearing in φ. Brute-force algorithm: Explore the tree of all assignments to x 1, x 2,..., x n. As soon as a partial assignment results in a conflict, prune the entire subtree and backtrack. Either a complete assignment that works is found, or formula is UNSAT. Worst-case performance: O(2 n ). Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 31 / 52

48 SAT Solving: Resolution From two clauses x + A and x + B, derive new clause A + B. Example: (x + y)(x + z) (x + y)(x + z)(y + z) Why? Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 32 / 52

49 SAT Solving: Resolution From two clauses x + A and x + B, derive new clause A + B. Example: (x + y)(x + z) (x + y)(x + z)(y + z) Why? A form of learning (new clauses). Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 32 / 52

50 SAT Solving: Resolution We can assume that every variable x appears both in positive form x and in negative form x in φ, that is, φ contains at least two clauses (x + ) and (x + ). What if it s not the case? Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 33 / 52

51 SAT Solving: Resolution We can assume that every variable x appears both in positive form x and in negative form x in φ, that is, φ contains at least two clauses (x + ) and (x + ). What if it s not the case? Davis-Putnam algorithm: for every variable x in φ do for every pair of clauses A : (x + ) and B : (x + ) in φ do resolve A and B and add resolvent to φ; check UNSAT (empty clause); end for remove from φ original clauses containing x or x; remove variables that don t appear both in positive and negative form; check SAT; end for Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 33 / 52

52 SAT Solving: Davis-Putnam Algorithm Examples (a means negation, i.e., a): From Sanjit Seshia. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 34 / 52

53 SAT Solving Modern SAT solvers use much more involved (and efficient) techniques. In-depth discussion: Computer-Aided Verification course. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 35 / 52

54 Back to BMC: Soundness vs. Completeness 1: for all k = 0, 1,..., n do 2: φ := Init( x 0) Trans( x 0, x 1) Trans( x k 1, x k ) Bad( x k ); 3: if SAT(φ) then 4: print Bad state reachable in k steps ; 5: output solution as counter-example; 6: end if 7: end for 8: print Bad state unreachable up to n steps ; BMC algorithm is sound in the following sense: if algorithm reports reachable then indeed a bad state is reachable if algorithm reports unreachable then a bad state is unreachable but only up to n steps. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 36 / 52

55 Back to BMC: Soundness vs. Completeness 1: for all k = 0, 1,..., n do 2: φ := Init( x 0) Trans( x 0, x 1) Trans( x k 1, x k ) Bad( x k ); 3: if SAT(φ) then 4: print Bad state reachable in k steps ; 5: output solution as counter-example; 6: end if 7: end for 8: print Bad state unreachable up to n steps ; BMC algorithm is sound in the following sense: if algorithm reports reachable then indeed a bad state is reachable if algorithm reports unreachable then a bad state is unreachable but only up to n steps. Can we make BMC complete? Reports unreachable iff bad states are unreachable (w.r.t. any bound). Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 36 / 52

56 Back to BMC: Soundness vs. Completeness 1: for all k = 0, 1,..., n do 2: φ := Init( x 0) Trans( x 0, x 1) Trans( x k 1, x k ) Bad( x k ); 3: if SAT(φ) then 4: print Bad state reachable in k steps ; 5: output solution as counter-example; 6: end if 7: end for 8: print Bad state unreachable up to n steps ; BMC algorithm is sound in the following sense: if algorithm reports reachable then indeed a bad state is reachable if algorithm reports unreachable then a bad state is unreachable but only up to n steps. Can we make BMC complete? Reports unreachable iff bad states are unreachable (w.r.t. any bound). Is this even possible in general? Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 36 / 52

57 Back to BMC: Soundness vs. Completeness 1: for all k = 0, 1,..., n do 2: φ := Init( x 0) Trans( x 0, x 1) Trans( x k 1, x k ) Bad( x k ); 3: if SAT(φ) then 4: print Bad state reachable in k steps ; 5: output solution as counter-example; 6: end if 7: end for 8: print Bad state unreachable up to n steps ; BMC algorithm is sound in the following sense: if algorithm reports reachable then indeed a bad state is reachable if algorithm reports unreachable then a bad state is unreachable but only up to n steps. Can we make BMC complete? Reports unreachable iff bad states are unreachable (w.r.t. any bound). Is this even possible in general? For finite-state systems? Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 36 / 52

58 Complete BMC: the trivial threshold 1: for all k = 0, 1,..., n do 2: φ := Init( x 0) Trans( x 0, x 1) Trans( x k 1, x k ) Bad( x k ); 3: if SAT(φ) then 4: print Bad state reachable in k steps ; 5: output solution as counter-example; 6: end if 7: end for 8: print Bad state unreachable up to n steps ; A finite-state Kripke structure K = (P, S, S 0, L, R) is essentially a finite graph. Can we turn BMC into a complete method for finite-state structures? How? Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 37 / 52

59 Complete BMC: the trivial threshold 1: for all k = 0, 1,..., n do 2: φ := Init( x 0) Trans( x 0, x 1) Trans( x k 1, x k ) Bad( x k ); 3: if SAT(φ) then 4: print Bad state reachable in k steps ; 5: output solution as counter-example; 6: end if 7: end for 8: print Bad state unreachable up to n steps ; A finite-state Kripke structure K = (P, S, S 0, L, R) is essentially a finite graph. Can we turn BMC into a complete method for finite-state structures? How? If we know S (the number of all possible states) then we can set n := S. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 37 / 52

60 Complete BMC: the trivial threshold 1: for all k = 0, 1,..., n do 2: φ := Init( x 0) Trans( x 0, x 1) Trans( x k 1, x k ) Bad( x k ); 3: if SAT(φ) then 4: print Bad state reachable in k steps ; 5: output solution as counter-example; 6: end if 7: end for 8: print Bad state unreachable up to n steps ; A finite-state Kripke structure K = (P, S, S 0, L, R) is essentially a finite graph. Can we turn BMC into a complete method for finite-state structures? How? If we know S (the number of all possible states) then we can set n := S. With 100 boolean variables, S = 2 100, so this doesn t work. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 37 / 52

61 Complete BMC: the Completeness Threshold Reachability diameter : maximum number of steps that it takes to reach all states. d := min{i s Reach : path s 0, s 1,..., s j in K : j i s 0 S 0 s j = s} where Reach is the set of reachable states of K. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 38 / 52

62 Complete BMC: the Completeness Threshold Reachability diameter : maximum number of steps that it takes to reach all states. d := min{i s Reach : path s 0, s 1,..., s j in K : j i s 0 S 0 s j = s} where Reach is the set of reachable states of K. d is a much better threshold than S. Why? Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 38 / 52

63 Complete BMC: the Completeness Threshold Reachability diameter : maximum number of steps that it takes to reach all states. d := min{i s Reach : path s 0, s 1,..., s j in K : j i s 0 S 0 s j = s} where Reach is the set of reachable states of K. d is a much better threshold than S. Why? d Reach S. It could in practice be much smaller. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 38 / 52

64 Complete BMC: the Completeness Threshold Reachability diameter : maximum number of steps that it takes to reach all states. d := min{i s Reach : path s 0, s 1,..., s j in K : j i s 0 S 0 s j = s} where Reach is the set of reachable states of K. d is a much better threshold than S. Why? d Reach S. It could in practice be much smaller. Problem: we don t know Reach, therefore how to compute d? Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 38 / 52

65 Complete BMC: the Completeness Threshold Recurrence diameter : length of the longest loop-free path. r := max{i path s 0, s 1,..., s i in K : s 0 S 0 0 j < k i : s j s k } where Reach is the set of reachable states of K. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 39 / 52

66 Complete BMC: the Completeness Threshold Recurrence diameter : length of the longest loop-free path. r := max{i path s 0, s 1,..., s i in K : s 0 S 0 0 j < k i : s j s k } where Reach is the set of reachable states of K. Then: d r. Why? Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 39 / 52

67 Complete BMC: the Completeness Threshold Recurrence diameter : length of the longest loop-free path. r := max{i path s 0, s 1,..., s i in K : s 0 S 0 0 j < k i : s j s k } where Reach is the set of reachable states of K. Then: d r. Why? How to compute r? Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 39 / 52

68 Complete BMC: the Completeness Threshold Recurrence diameter : length of the longest loop-free path. r := max{i path s 0, s 1,..., s i in K : s 0 S 0 0 j < k i : s j s k } where Reach is the set of reachable states of K. Then: d r. Why? How to compute r? Express it in propositional logic! ( r := max{i SAT Init( x 0 ) Trans( x 0, x 1 ) Trans( x i 1, x i ) i 1 i j=0 k=j+1 x j x k )} Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 39 / 52

69 EXHAUSTIVE SYMBOLIC REACHABILITY (a.k.a. SYMBOLIC MODEL-CHECKING) Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 40 / 52

70 Symbolic Reachability Recall: Set of states = predicate φ( x) on vector of state variables x. E.g.: Init(x, y, z) : x y Bad(x 1, x 2 ) : x 1 = crit x 2 = crit Transition relation = predicate Trans( x, x ) on state variables and next-state variables. E.g.: Trans(x, y, x, y ) : x = x + 1 (y = 0 y = 1) Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 41 / 52

71 Symbolic Reachability: Predicate Transformer Then: Successors can be computed by a predicate transformer : succ ( φ( x) ) := ( x : φ( x) Trans( x, x ) ) [ x x] x : φ( x) Trans( x, x ): successors of states in φ [ x x]: renames variables so that resulting predicate is over current state variables Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 42 / 52

72 Symbolic Reachability: Predicate Transformer Then: Successors can be computed by a predicate transformer : Example: succ ( φ( x) ) := ( x : φ( x) Trans( x, x ) ) [ x x] x : φ( x) Trans( x, x ): successors of states in φ [ x x]: renames variables so that resulting predicate is over current state variables φ = 0 x 5 Trans = x x x + 1 succ(φ) = ( x : 0 x 5 x x x + 1)[x x] Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 42 / 52

73 Symbolic Reachability: Predicate Transformer Then: Successors can be computed by a predicate transformer : Example: succ ( φ( x) ) := ( x : φ( x) Trans( x, x ) ) [ x x] x : φ( x) Trans( x, x ): successors of states in φ [ x x]: renames variables so that resulting predicate is over current state variables φ = 0 x 5 Trans = x x x + 1 succ(φ) = ( x : 0 x 5 x x x + 1)[x x] = ( x : 0 x 5 0 x 5 + 1)[x x] Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 42 / 52

74 Symbolic Reachability: Predicate Transformer Then: Successors can be computed by a predicate transformer : Example: succ ( φ( x) ) := ( x : φ( x) Trans( x, x ) ) [ x x] x : φ( x) Trans( x, x ): successors of states in φ [ x x]: renames variables so that resulting predicate is over current state variables φ = 0 x 5 Trans = x x x + 1 succ(φ) = ( x : 0 x 5 x x x + 1)[x x] = ( x : 0 x 5 0 x 5 + 1)[x x] = (0 x 6)[x x] Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 42 / 52

75 Symbolic Reachability: Predicate Transformer Then: Successors can be computed by a predicate transformer : Example: succ ( φ( x) ) := ( x : φ( x) Trans( x, x ) ) [ x x] x : φ( x) Trans( x, x ): successors of states in φ [ x x]: renames variables so that resulting predicate is over current state variables φ = 0 x 5 Trans = x x x + 1 succ(φ) = ( x : 0 x 5 x x x + 1)[x x] = ( x : 0 x 5 0 x 5 + 1)[x x] = (0 x 6)[x x] = 0 x 6 Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 42 / 52

76 Symbolic Reachability: Predicate Transformer succ ( φ( x) ) := ( x : φ( x) Trans( x, x ) ) [ x x] How do we implement succ in practice? In particular, how to do quantifier elimination automatically? In the case of propositional logic, quantifier elimination becomes a simple procedure: x : φ(x) Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 43 / 52

77 Symbolic Reachability: Predicate Transformer succ ( φ( x) ) := ( x : φ( x) Trans( x, x ) ) [ x x] How do we implement succ in practice? In particular, how to do quantifier elimination automatically? In the case of propositional logic, quantifier elimination becomes a simple procedure: x : φ(x) φ(x)[x 0] φ(x)[x 1] Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 43 / 52

78 Predicate Transformer: Another Example succ(p q) = ( p, q : p q Trans)[p p, q q] Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 44 / 52

79 Predicate Transformer: Another Example succ(p q) = ( p, q : p q Trans)[p p, q q] = ( p, q : p q p q )[p p, q q] Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 44 / 52

80 Predicate Transformer: Another Example succ(p q) = ( p, q : p q Trans)[p p, q q] = ( p, q : p q p q )[p p, q q] = (p q )[p p, q q] Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 44 / 52

81 Predicate Transformer: Another Example succ(p q) = ( p, q : p q Trans)[p p, q q] = ( p, q : p q p q )[p p, q q] = (p q )[p p, q q] = p q Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 44 / 52

82 Symbolic Reachability 1: Reachable := Init; 2: terminate := false; 3: repeat 4: tmp := Reachable succ(reachable); 5: if tmp Reachable then 6: terminate := true; 7: else 8: Reachable := tmp; 9: end if 10: until terminate 11: return Reachable; Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 45 / 52

83 Symbolic Reachability 1: Reachable := Init; 2: terminate := false; 3: repeat 4: tmp := Reachable succ(reachable); 5: if tmp Reachable then 6: terminate := true; 7: else 8: Reachable := tmp; 9: end if 10: until terminate 11: return Reachable; Does the algorithm terminate? Why? Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 45 / 52

84 Symbolic Reachability 1: Reachable := Init; 2: terminate := false; 3: repeat 4: tmp := Reachable succ(reachable); 5: if tmp Reachable then 6: terminate := true; 7: else 8: Reachable := tmp; 9: end if 10: until terminate 11: return Reachable; Does the algorithm terminate? Why? Quiz: modify the algorithm to make it check reachability of a set of bad states characterized by predicate Bad. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 45 / 52

85 Symbolic Reachability: checking for Bad states 1: Reachable := Init; 2: terminate := false; 3: error := false; 4: repeat 5: tmp := Reachable succ(reachable); 6: if tmp Reachable then 7: terminate := true; 8: else 9: Reachable := tmp; 10: end if 11: if SAT(Reachable Bad) then 12: error := true; 13: end if 14: until terminate or error 15: return (Reachable,error); Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 46 / 52

86 Symbolic Model-Checking: Example Let s model-check this symbolically! We want to check that all reachable states satisfy p q. In temporal logic parlance: CTL: AG(p q) LTL: (p q) Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 47 / 52

87 Symbolic Model-Checking: Implementation For finite-state systems, boolean variables can be used to encode state. All predicates then become boolean expressions. Efficient data structures for boolean expressions: BDDs (Binary Decision Diagrams) Efficient algorithms for implementing logical operations (conjunction, disjunction, satisfiability check) on BDDs. In-depth discussion: Computer-Aided Verification course. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 48 / 52

88 Example: BDD Can you guess which boolean expression this BDD represents? Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 49 / 52

89 Example: BDD Can you guess which boolean expression this BDD represents? x 2 x 3 x 4 + x 1 (x 2 x 3 x 4 + x 2 x 3 x 4 ) + x 2 x 3 x 4 + x 1 x 2 x 4 Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 49 / 52

90 Symbolic Model-Checking Homework: Assuming that the system only has boolean variables, express deadlock as a predicate on state (or next state) variables. Note: the predicate must have no quantifiers. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 50 / 52

91 Bibliography I Baier, C. and Katoen, J.-P. (2008). Principles of Model Checking. MIT Press. Barrett, C. W., Sebastiani, R., Seshia, S. A., and Tinelli, C. (2009). Satisfiability modulo theories. In Biere, A., Heule, M., van Maaren, H., and Walsh, T., editors, Handbook of Satisfiability, volume 185, pages IOS Press. Biere, A., Cimatti, A., Clarke, E. M., Strichman, O., and Zhu, Y. (2003). Bounded model checking. Advances in Computers, 58: Burch, J., Clarke, E., Dill, D., Hwang, L., and McMillan, K. (1990). Symbolic model checking: states and beyond. In 5th LICS, pages IEEE. Clarke, E., Grumberg, O., and Peled, D. (2000). Model Checking. MIT Press. Godefroid, P. and Wolper, P. (1991). Using partial orders for the efficient verification of deadlock freedom and safety properties. In 4th CAV. Holzmann, G. (1998). An analysis of bitstate hashing. In Formal Methods in System Design, pages Chapman & Hall. Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 51 / 52

92 Bibliography II Huth, M. and Ryan, M. (2004). Logic in Computer Science: Modelling and Reasoning about Systems. Cambridge University Press. Jain, H. and Clarke, E. M. (2009). Efficient SAT solving for non-clausal formulas using DPLL, graphs, and watched cuts. In 46th Annual Design Automation Conference, DAC 09, pages ACM. Robinson, J. (1965). A machine-oriented logic based on the resolution principle. Journal of the ACM, 12(1). Tseitin, G. S. (1968). On the complexity of derivations in the propositional calculus. Studies in Mathematics and Mathematical Logic, Part II: Valmari, A. (1990). Stubborn sets for reduced state space generation. LNCS 483. Zhang, L. and Malik, S. (2002). The quest for efficient boolean satisfiability solvers. In 14th International Conference on Computer Aided Verification, CAV 02, pages Stavros Tripakis: EECS 144/244 Discrete Systems State-Space Exploration 52 / 52