SAT based Abstraction-Refinement using ILP and Machine Learning Techniques

Transcription

1 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 1 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques Edmund Clarke James Kukula Anubhav Guta Ofer Strichman

2 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 2 Abstraction in Model Checking I Set of variables V = {x 1,..., x n }. Set of states S = D x1 D xn. Set of initial states I S. Set of transitions R S S. Transition system M = (S, I, R).

3 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 3 Abstract Model Abstraction Function h : S Ŝ ˆM = (Ŝ, Î, ˆR) I h h h h h Ŝ = {ŝ s. s S h(s) = ŝ}

4 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 4 Abstract Model Abstraction Function h : S Ŝ ˆM = (Ŝ, Î, ˆR) I I Î = {ŝ s. I(s) h(s) = ŝ}

5 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 5 Abstract Model Abstraction Function h : S Ŝ ˆM = (Ŝ, Î, ˆR) I I ˆR = {(ŝ 1, ŝ 2 ) s 1. s 2. R(s 1, s 2 ) h(s 1 ) = ŝ 1 h(s 2 ) = ŝ 2 }

6 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 6 Model Checking AG, is a non-temoral roositional formula resects h if for all s S, h(s) = s = ~ ~ ~ ~ ~ resects h

7 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 7 Model Checking AG, is a non-temoral roositional formula resects h if for all s S, h(s) = s = ~ ~ ~ ~ ~ ~ does not resect h

8 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 8 Preservation Theorem Let ˆM be an abstraction of M corresonding to the abstraction function h, and be a roositional formula that resects h. Then ˆM = AG M = AG ~ ~ ~

9 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 9 Converse of Preservation Theorem ˆM = AG M = AG ~ ~ ~ ~ ~ ~ Counterexamle is surious. Abstraction is too coarse.

10 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 10 Refinement h is a refinement of h if 1. s 1, s 2 S, h (s 1 ) = h (s 2 ) imlies h(s 1 ) = h(s 2 ). 2. s 1, s 2 S such that h(s 1 ) = h(s 2 ) and h (s 1 ) h (s 2 ). ~ ~ ~ ~ ~ ~

11 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 11 Refinement h is a refinement of h if 1. s 1, s 2 S, h (s 1 ) = h (s 2 ) imlies h(s 1 ) = h(s 2 ). 2. s 1, s 2 S such that h(s 1 ) = h(s 2 ) and h (s 1 ) h (s 2 ). ~ ~ ~ ~ ~ ~

12 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 12 Abstraction-Refinement 1. Generate an initial abstraction function h. 2. Build abstract machine ˆM based on h. Model check ˆM. If ˆM = ϕ, then M = ϕ. Return TRUE. 3. If ˆM = ϕ, check the counterexamle on the concrete model. If the counterexamle is real, M = ϕ. Return FALSE. 4. Refine h, and go to ste 2.

13 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 13 Abstraction Function Partition variables V into visible(v) and invisible(i) variables. V = {v 1,..., v k }. The artitioning defines our abstraction function h : S Ŝ. The set of abstract states is and the abstraction functions is Ŝ = D v1 D vk h(s) = (s(v 1 )... s(v k )) x1 x2 x3 x } 0 Refinement : Move variables from I to V. x1 x2 0

14 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 14 Building Abstract Model ˆM can be comuted efficiently if R is in functional form, e.g. sequential circuits. R(s, s ) = i( m j=1 x j = f x j (s, i)) ˆR(ŝ, ŝ ) = s I i( x j V ˆx j = f x j (ŝ, s I, i)) x1 x2 x3 x4 x5 x6 x1 x2 i1 i2 i3 i1 i2 i3 x3 x4 x5 x6

15 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 15 Checking the Counterexamle Counterexamle : ŝ 1, ŝ 2,... ŝ m Set of concrete aths for counterexamle : ψ m = { s 1... s m I(s 1 ) m 1 i=1 R(s i, s i+1 ) m i=1 h(s i ) = ŝ i } The right-most conjunct is a restriction of the visible variables to their values in the counterexamle. Counterexamle is surious ψ m is emty. Solve ψ m with a SAT solver.

16 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 16 Checking the Counterexamle Similar to BMC formulas, excet Path restricted to counterexamle. Also restrict values of (original) inuts that are assigned by counterexamle. If ψ m is satisfiable we found a real bug. If ψ m is unsatisfiable, refine.

17 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 17 Refinement Find largest index f (failure index), f < m such that ψ f is satisfiable. The set D of all states d f such that there is a concrete ath d 1...d f in ψ f is called the set of deadend states. Abstract Trace Concrete Trace } Dead end No concrete transition from D to a concrete state in the next abstract state.

18 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 18 Refinement Since there is an abstract transition from ŝ f to ŝ f+1, there is a non-emty set of transitions φ f from h 1 (ŝ f ) to h 1 (ŝ f+1 ). φ f = { s f, s f+1 R(s f, s f+1 ) h(s f ) = ŝ f h(s f+1 ) = ŝ f+1 } The set B of all states b f such that there is a transition b f, b f+1 in φ f is called the set of bad states. Abstract Trace Concrete Trace } Dead end Bad {

19 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 19 Refinement Dead States ~ ~ Bad States ~ ~ ~ ~

20 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 20 Refinement There is a surious transition from ŝ f to ŝ f+1. Surious transition because D and B lie in the same abstract state. Refinement : Put D and B is searate abstract states. d D, b B (h (d) h (b)) ~ ~ ~ ~ ~ ~

21 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 21 Refinement as Searation Let S = {s 1...s m } and T = {t 1...t n } be two sets of states (binary vectors) of size l, reresenting assignments to a set of variables W, W = l. (The state searation roblem) Find a minimal set of variables U = {u 1...u k }, U W, such that for each air of states (s i, t j ), 1 i m, 1 j n, there exists a variable u r U such that s i (u r ) t j (u r ). Let H denote the searating set for D and B. The refinement h is obtained by adding H to V. Proof : Since H searates D and B, for all d D, b B there exists u H s.t. d(u) b(u). Hence, h(d) h(b).

22 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 22 Refinement as Searation and Learning For systems of realistic size, It is not ossible to generate D and B, either exlicitly or symbolically. Comutationally exensive to searate large D and B. Generate samles for D(denoted S D ) and B(denoted S B ) and try to infer the searating variables from the samles. State of the art SAT solvers like Chaff can generate many samles in a short amount of time. Our algorithm is comlete because a counterexamle will eventually be eliminated in subsequent iterations.

23 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 23 Searation using Integer Linear Programming Searating S D from S B as an Integer Linear Programming (ILP) roblem: Min I i=1 v i subject to: ( s S D ) ( t S B ) 1 i I, s(v i ) t(v i ) v i 1 v i = 1 if and only if v i is in the searating set. One constraint er air of states, stating that at least one of the variables that searates the two states should be selected.

24 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 24 Examle s 1 = (0, 1, 0, 1) t 1 = (1, 1, 1, 1) s 2 = (1, 1, 1, 0) t 2 = (0, 0, 0, 1) Min 4 i=1 v i subject to: v 1 + v 3 1 /* Searating s 1 from t 1 / v 2 1 /* Searating s 1 from t 2 / v 4 1 /* Searating s 2 from t 1 / v 1 + v 2 + v 3 + v 4 1 /* Searating s 2 from t 2 / Otimal value of the objective function is 3, corresonding to one of the two otimal solutions (v 1, v 2, v 4 ) and (v 3, v 2, v 4 ).

25 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 25 Searation using Decision Tree Learning ILP-based searation: Minimal searation set Comutationally exensive Decision Tree Learning based searation: Non otimal Comutationally efficient

26 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 26 Decision Tree Learning Inut : Set of examles with classification. Each examle assigns values to a set of attributes. Outut : Decision Tree Each internal node is a test on some attribute. Each leaf corresonds to a classification. v1 0 1 v2 v

27 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 27 Searation using Decision Tree Learning Searating S D from S B as a Decision Tree Learning roblem: Attributes corresond to the invisible variables. The classifications are +1 and 1, corresonding to S D and S B, resectively. The examles are S D labeled +1, and S B labeled 1. Searating set : All the variables resent at an internal nodes of the decision tree. Proof: Let d S D and b S B. The decision tree will classify d as +1 and b as 1. So, there exists a node n in the decision tree, labeled with a variable v, such that d(v) b(v). By construction, v lies in the outut set.

28 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 28 Examle s 1 = (0, 1, 0, 1) t 1 = (1, 1, 1, 1) s 2 = (1, 1, 1, 0) t 2 = (0, 0, 0, 1) E = ((0, 1, 0, 1), +1), ((1, 1, 1, 0), +1), ((1, 1, 1, 1), 1), ((0, 0, 0, 1), 1) v1 0 1 v2 v Searating set : {v 1, v 2, v 4 }

29 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 29 Decision Tree Learning Algorithm DecT ree(examles, Attributes) ID3 Algorithm 1. Create a Root node for the tree. 2. If all examles are classified the same, return Root with this classification. 3. Let A = BestAttribute(Examles, Attributes). Label Root with attribute A. 4. Let Examles 0 and Examles 1 be subsets of Examles having values 0 and 1 for A, resectively. 5. Add a 0 branch to the Root ointing to subtree generated by Dectree(Examles 0, Attributes {A}).

30 6. Add a 1 branch to the Root ointing to subtree generated by Dectree(Examles 1, Attributes {A}). 7. return Root. The BestAttribute rocedure returns an attribute (which is a variable in our case) that causes the maximum reduction in entroy if the set is artitioned according to this variable.

31 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 30 Efficient Samling Direct search towards samles that contain more information. Iterative Algorithm. At each iteration, the algorithm finds new samles that are not searated by the current searating set. Let SeSet denote the searating set for the current set of samles. New samles that are not searated by SeSet are comuted by solving Φ(SeSet). = ψ f φ f v i SeSet v i = v i

32 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 31 Efficient Samling SeSet = ; i = 0; reeat forever { If Φ(SeSet) is satisfiable, derive d i and b i from solution; else exit; SeSet = Searating Set for { i j=0 {d j }, ij=0 {b j }}; i = i + 1; }

33 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 32 Exeriments NuSMV frontend. Cadence SMV. A ublic domain ILP solver. Chaff. Exeriments conducted on a 1.5GHz Athlon with 3Gb RAM running Linux. We used the IU family of circuits, which are various abstractions of an interface control circuit from Synosys.

34 Circuit SMV Samling - ILP Samling - DTL Eff. Sam. - DTL Time BDD(k) Time BDD(k) S L Time BDD(k) S L Time BDD(k) S L IU IU IU IU IU IU IU IU IU IU IU IU IU

35 Circuit SMV Samling - ILP Samling - DTL Eff. Sam. - DTL Time BDD(k) Time BDD(k) S L Time BDD(k) S L Time BDD(k) S L IU IU IU IU IU IU IU IU IU IU IU IU IU

36 SAT based Abstraction-Refinement using ILP and Machine Learning Techniques 33 Conclusions and Future Work Our algorithm outerforms standard model checking in both execution time and memory requirements. Exloit criteria other than size of searating set for characterizing a good refinement. Exlore other learning techniques.