Finding Minimal Unsatisfiable Subsets in Linear Temporal Logic using BDDs

Transcription

1 Finding Minimal Unsatisfiable Subsets in Linear Temporal Logic using BDDs Timothy Sergeant Supervisors: Rajeev Goré and Jimmy Thomson Australian National University Abstract. When writing specifications in linear temporal logic, it is common for human error to introduce unsatisfiability into the knowledge base, preventing any reasoning from being performed. Due to the complexity of this problem, such an error is difficult to diagnose by hand. This paper describes an algorithm which can be used to reduce an unsatisfiable PLTL specification into a minimal unsatisfiable subset of the input, thereby removing any formulae which do not directly contribute to the unsatisfiability. Reducing the input in this way allows the source of the error to be more easily diagnosed. The method presented uses binary decision diagrams (BDDs) as an efficient way to store satisfiability information about the input set, and requires only a single call to a BDD-based theorem prover to operate. 1 Introduction Linear Temporal Logic provides semantics which are well suited to specifying and verifying properties of reactive systems.for example, business processes can be expressed in Propositional Linear Temporal Logic (PLTL) in order to show that they correctly encode the desired behaviour. Such a specification would be built up out of many smaller formulae describing individual aspects of the system. While each of these may appear sensible when considered in isolation, it is common for clashes to occur when joining the formulae to form a complete specification. In this case, there is no possible valuation of variables which would conform with the faulty specification, and we say that the input is unsatisfiable. Due to the complexity of the interactions between PLTL formulae, and the level of abstraction involved with the PLTL representation, manually debugging such a faulty specification would be difficult. As it is likely that only a small subset of the input actually contributes to the unsatisfiability, it would be desirable to be able to focus on this subset by automatically discarding any formulae which do not directly contribute to the problem. The reduced search space would then greatly reduce the effort required to debug and fix the issue. More generally, we wish to be able to reduce an unsatisfiable set of input formulae to some subset that is as small as possible while still remaining unsatisfiable. An obvious naive approach would be to remove formulae one by one, calling a satisfiability solver each time to determine whether the remainder is

2 still unsatisfiable. However, this approach will quickly become impractical, as PLTL is known to be PSPACE-complete [1]. In this paper, we describe an alternate, more efficient method which can be used to minimise an unsatisfiable set of PLTL formulae. We use Binary Decision Diagrams (BDDs) as a data structure which provides an efficient representation of the satisfiability of the input set, together with a BDD-based theorem prover for PLTL. We are then able to adapt the method presented in [2] to PLTL, allowing a minimal unsatisfiable subset to be extracted with only a single call to the theorem prover. This approach is experimentally evaluated, and possible future improvements are suggested. The remainder of the paper is organised as follows. First, we introduce the syntax and semantics of PLTL in Section 2. In Section 3, we provide a precise definition of minimal unsatisfiability, and then describe our algorithm for reducing an input set of formulae to a minimal unsatisfiable subset in Section 4. Section 5 then experimentally evaluates this approach. Finally, we describe other related work in Section 6 and possible improvements in Section 7 and before concluding in Section 8. 2 Introduction to PLTL The definition of PLTL used here is based off that in [3] and [4]. 2.1 Syntax Let Atm be a set of propositional variables. Then, the set of well-defined PLTL formulae is defined as follows, where p Atm: φ ::= p φ φ ψ φ ψ φ ψ X φ F φ G φ φ U ψ φ B ψ 2.2 Semantics PLTL formulae are interpreted over an infinite sequence of states S = (s i ) i N, where each state is assigned a subset of Atm, representing the propositions which are true at that moment in time. Let ϑ be the interpretation function from states to subsets of Atm. We can then define a relation between a pair I = S, ϑ, a state s i and a formula φ as: I, s i p iff p ϑ(s i ) I, s i (φ ψ) iff I, s i φ or s i ψ (similarly for, and ) I, s i X φ iff I, s i+1 φ I, s i G φ iff for all j N, j i implies I, s j φ I, s i F φ iff there exists j N such that j i and I, s j φ I, s i φ U ψ iff there exists j N such that j i, I, s j ψ, and for all k N, j > k i implies I, s k φ I, s i φ B ψ iff I, s i φ U ψ or I, s i G φ

3 Then, we say that an interpretation I satisfies a formula φ if I, s 0 φ. Similarly, φ is said to be satisfiable if there exists an interpretation I which satisfies it, and is unsatisfiable otherwise. For convenience, we say that a set of formulae Γ is satisfiable (unsatisfiable) if φ Γ φ is satisfiable (unsatisfiable). Finally, note that F φ U φ and G φ B φ. Next, we briefly consider the notion of logical consequence. We say that φ is a logical consequence of Γ (Γ = φ), where φ and Γ are PLTL formulae, if any interpretation that satisfies Γ also satisfies φ (Let I be the class of all interpretations. Then I I.I, s 0 Γ I, s 0 φ). In this situation, an unsatisfiable knowledge base would prevent effective reasoning from being performed. An alternative scenario is where we want to check whether a knowledge base permits a certain formula to be true. In this case, we test whether Γ φ is satisfiable - an unsatisfiable result indicates that something is wrong with the specification. In both cases, debugging would be aided with a minimal unsatisfiable subset of the input. As a running example, consider the specification given in example 1 below, adapted from [5]. Each formula within the specification is annotated with an English description of the meaning it encodes. Example 1. Γ a ={G (req ((X gnt) (X X gnt))), (1) (Whenever a request is made, it must be granted in the following two timesteps) G (gnt X gnt), (2) (Requests cannot be granted at two consecutive timesteps) G (pause X ( gnt U resume)} (3) (If processing is paused, from the next timestep, no requests may be granted until processing is resumed) This specification is clearly faulty - although each formula can be understood on its own, when conjoined to form a knowledge base Γ, equations 1 and 2 conflict. However, this knowledge base by itself is still satisfiable - for example, consider an interpretation where no requests or grants are made at all. However, querying the knowledge base allows us to check whether certain situations are allowed, such as in example 2, where we test whether a request can be made at any point. Example 2. Γ b ={G (req ((X gnt) (X X gnt))), G (gnt X gnt), G (pause X ( gnt U resume), F req}

4 3 Minimal Unsatisfiability in PLTL We first consider the definition of minimal unsatisfiability used in classical propositional logic. Any classical propositional formula can be converted into a Conjunctive Normal Form (CNF) representation. This is the conjunction of a set of clauses, where each clause is the disjunction of a set of literals (for a Atm, a and a are both literals). Minimal unsatisfiability is then defined in terms of the Conjunctive Normal Form (CNF) representation of an input formula: Definition 1. Let be a CNF formula. is minimal unsatisfiable if is unsatisfiable and c. \ c i is satisfiable [2] This is extended to PLTL by considering a set of PLTL formulae instead of a set of CNF clauses. Definition 2. Let Γ be a set of PLTL formulae. Γ is a minimal unsatisfiable set if Γ is unsatisfiable, and φ Γ.Γ \ {φ} is satisfiable. Thus, a set of formulae might have several minimal unsatisfiable subsets, which may be different sizes. In this case, we consider a minimum unsatisfiable subset to be a subset with the least cardinality. Definition 3. A minimum unsatisfiable subset of Γ is a minimal unsatisfiable subset S of Γ such that for every unsatisfiable subset S of Γ, S S Note that minimum unsatisfiable subsets are not necessarily unique. In this paper, we are only concerned with the extraction of minimal unsatisfiable subsets from a set of input formulae we do not attempt to find minimum unsatisfiable subsets. 4 Reduction to minimal unsatisfiability We will introduce the method used to reduce a set of PLTL formulae into a minimal unsatisfiable subset in two parts. First, we will describe the general strategy, using a version of the method presented in [2] adapted for PLTL. Then, we will discuss specific details relating to implementing this method with BDDs. 4.1 General strategy Let Γ = {φ 1,, φ m } be the formula set we want to reduce to minimal unsatisfiability, and let X be the set of propositional variables in Γ. Further, let Γ i = Γ \ {φ i }. The first step in the process is to label each φ i with a set of propositional variables. We introduce a set of k = log(m+1) new variables, Y = {y 1,, y k }. By conjoining these k variables in different combinations of positive and negative literals, we can produce a total of 2 k tag formulae. For example, with k = 2, the resulting tags are {y 1 y 2, y 1 y 2, y 1 y 2, y 1 y 2 }. Let T i be the ith

5 such tag formula. Since 2 k m, we can thus produce a new set of formulae Γ by tagging each formula in Γ with some unique T i : Γ = {T 1 φ 1,, T m φ m } Example 3. We now augment the Γ b of example 2 with tag variables, giving the following Γ : Γ ={(y 0 y 1 ) (G (req ((X gnt) (X X gnt)))), (y 0 y 1 ) (G (gnt X gnt)), ( y 0 y 1 ) (G (pause X ( gnt U resume))), ( y 0 y 1 ) (F req)} Consider a valuation of the Y variables such that some T i evaluates to true. Then, the corresponding element of Γ (T i φ i ) also evaluates to true, regardless of the contents of φ i. This effectively allows us to remove one φ i at a time from Γ by assigning values to the Y variables. Example 4. Consider setting both y 0 and y 1 to true in the Γ of example 3. Then, T 1 evaluates to, while all other tag formulae evaluate to. Γ then simplifies down to the following: Γ α ={, G (gnt X gnt), G (pause X ( gnt U resume), F req} Next, consider the following theorem. We use ˆΓ = φ Γ φ to denote the conjunction of all members of Γ, and use X in the same sense as quantified boolean logic, where X.f(X, Y ) has the same models at f(x, Y ), but with all references to variables in X removed. Theorem 1. A set of formulae Γ is minimal unsatisfiable iff XΓ has exactly m models over variables Y. [2] Theorem 1 is directly translated into PLTL from its propositional equivalent in [2], we give a full proof of this theorem in Appendix A. Using this theorem, if we are able to count the models of XΓ, we are able to determine whether a given Γ is minimal unsatisfiable. Example 5. Consider the Γ given in example 3. We try each of the 4 possible valuations for Y variables in turn: α(t 1 ) = Γ α is unsatisfiable α(t 2 ) = Γ α is unsatisfiable α(t 3 ) = Γ α is satisfiable α(t 4 ) = Γ α is unsatisfiable

6 Thus, there are three valuations of Y such that ( XΓ ) α is true, so XΓ has three models. We can thus see that Γ is not minimal unsatisfiable. However, removing equation 3 from Γ would still result in a total of 3 models for XΓ. As removing one equation gives Γ = 3, this is now a minimal unsatisfiable set. We are also able to use the tag formulae to determine which φ i can be removed without affecting unsatisfiability. If Γ remains unsatisfiable after removing some φ i (by assigning the appropriate values to the tag variables), then φ i can be safely removed without affecting unsatisfiability. By combining these two ideas, we are able to produce a minimal unsatisfiable subset of Γ by repeatedly checking for unsatisfiability and then finding a formula to remove. We describe this in full in algorithm 1 below. This algorithm will currently require multiple calls to the theorem prover, however, the BDD-based method described in section 4.2 allows this to be reduced to a single call. Algorithm 1 Minimise(Γ ) while Number of solutions to XΓ Γ do for each i [1, Γ ] do Let α be a valuation such that α(t 1 ) = if Γ α unsatisfiable then Γ Γ \ φ i end if end for end while 4.2 Implementation with BDDs An implementation of this algorithm was built using BDDs and a PLTL prover (pltlbdd) based on the fixpoint method presented in [6]. Given a set of input formulae Γ, this prover outputs a BDD encoding which allows us to extract the satisfiability of any subset of Γ. Given W as output from the prover, we can determine whether some subset S of Γ is (un)satisfiable by checking whether W φ S φ is not (is ) Note that φ is the BDD-representation of φ that is given by the theorem prover. In order to extract this information from the BDD prover, certain optimisations had to be disabled. In particular, the prover was no longer permitted to turn G φ formulae into assumptions that were considered true at every state, as we needed to be able to query subsets of the input where those formulae were not included. We can thus construct the BDD Γ = W φ i Γ φ i T i and existentially quantify out everything other than the tag variables. We are then able to use standard BDD operations to find the number of models in terms of the tag

7 variables (in O(m)) time [2]), and to restrict the BDD to a certain valuation of the tag variables, allowing us to determine whether each Γ \ φ i is unsatisfiable. Iterating through Γ to find a formula to be remove can also be performed in linear time. This gives us everything required to implement the algorithm described in section 4.1. Note that the BDD X.Γ must be rebuilt every time a formula is removed, however, the caching performed by the BDD library reduces the amount of repetitive computation required. 5 Experimental Results We built an implementation of this algorithm (PLTL-MUP) on top of the pltlbdd prover described in section 4.2. The core algorithm was built in OCaml, using the BuDDy library for BDD operations. This implementation was then compared to two similar provers, TRP++UC and procmine. Our experimental method is described in Section 5.1, before results are presented and discussed in Section Experimental Method A set of 635 benchmark cases were chosen from several families, based off those used in [5]. In particular, the lift, genbuf and forobots are applicationoriented benchmarks and O1 formula and O2 formula evaluate how well large inputs can be handled. A number of randomly generated (pltl) benchmarks were also used. All of the test cases considered are unsatisfiable. We executed these benchmarks over a total of five prover configurations: PLTL-MUP This is our minimal unsatisfiability prover, running on top of pltlbdd with certain optimisations turned off. pltlbdd This is the BDD prover running with all optimisations turned on, in order to determine the performance impact of disabling these optimisations. procmine This is the method described in [4], which uses tableaux with backjumping to extract an unsatisfiable subset, and then uses repeated calls to the theorem prover to reduce this to a minimal unsatisfiable subset TRP++UC This is the method described in [5], which uses temporal resolution to construct a proof of unsatisfiability, and then traces back through the proof to find parts of the input which directly contribute to the unsatisfiability. Note that this method does not guarantee minimality (unlike PLTL-MUP and procmine), Hybrid During early testing, it was found that the primary bottleneck in PLTL-MUP was the call to the theorem prover. This configuration is an attempt to reduce this bottleneck with a hybrid approach which runs TRP++UC over the input to obtain a reduced (but not necessarily minimal) subset, before passing this through to PLTL-MUP for minimisation. All tests were performed on a Intel Core i5 3570K 3.4GHz processor. Execution of each test case was limited to 600 seconds of runtime and 6GB of memory

8 usage. If two test cases from a sub-family failed by running out of time or memory, all remaining test cases from that sub-family were skipped. Table 1 shows the total number of test cases in each family and the number of test cases from each family successfully executed by each prover. Some slight variation can be seen between executions within these results. For example, TRP++UC solved 17 genbuf examples while the hybrid approach solved 18, even though Hybrid calls TRP++UC internally. This is caused by a test case which took very close to the 600 second time limit successfully finishing in one instance, but failing in the other. Table 1. Number of successful test cases for each prover over each benchmark family Family Number Solved Name Total Size PLTL-MUP pltlbdd Procmine TRP++UC Hybrid forobots genbuf lift O1 formula O2 formula phltl pltl-12x pltl-12y pltl-5x pltl-5y Total Discussion of results These results show that TRP++UC gives better performance on these benchmarks than pltlbdd running with all optimisations turned on. When we turn off these optimisations in order to allow the minimisation procedure to run, performance of the prover decreases further. In almost all test cases which PLTL-MUP completed within the time limit, running time was dominated by time spent proving unsatisfiability, while minimisation generally completed in less than a second. Finally, procmine performed the worst of the three solvers on these benchmarks. However, note that TRP++UC solves a far simpler problem than PLTL-MUP, as TRP++UC does not guarantee that its results will be minimal. Due to this difference in approaches, a fair comparison cannot be drawn between the two programs. The hybrid approach provides a good middle ground between the performance of TRP++UC and the guaranteed minimality of PLTL-MUP. Performing this

9 test case has also allowed us to analyse how well TRP++UC reduces formulae. Table 2 groups each test case according to the four possible results from the hybrid theorem prover: 1. Successfully minimised by TRP++UC and then checked by PLTL-MUP 2. Reduced by TRP++UC and then minimised by PLTL-MUP 3. Failed (time or memory) within TRP++UC 4. Passed by TRP++UC, but failed by PLTL-MUP (so we are unable to determine whether the result from TRP++UCis minimal) It can be seen that over half of the test cases successfully completed by the hybrid approach were minimised further from the output of TRP++UC. Finally, figure 1 compares the output size of TRP++UC to the hybrid prover for each test case. The further below the y = x line a point lies, the more that test case was able to be reduced. It can be seen that the hybrid approach achieved a significant reduction for many of the test cases. Table 2. Breakdown of results from Hybrid theorem into each of the four possible states Minimised Minimised Failed Failed Family Total size TRP++UC PLTL-MUP TRP++UC PLTL-MUP forobots genbuf lift O1 formula O2 formula phltl pltl-12x pltl-12y pltl-5x pltl-5y Total Related Work While several approaches have been taken to extract minimal unsatisfiable subsets of classical propositional logic [2, 7], relatively little work has been done within this domain for linear temporal logic. Two such approaches have already been introduced. In [4], Awad et al. describe the method used within procmine, using tableaux and back-jumping and then performing multiple calls to the theorem prover to reduce the result to a minimal unsatisfiable subset. They focus heavily on the application of their method to business process modelling, and thus are able to

10 40 35 Size of Hybrid output Size of TRP++UC output Fig. 1. Reduction in TRP++UC output size performed by hybrid approach. Each point represents an individual test case optimise by considering domain knowledge and compliance rules individually, only considering their conjunction when necessary. The alternative approach taken by TRP++UC is described in [5]. Due to the use of Separated Normal Form, Schuppan s notion of an unsatisfiable subset is somewhat different to ours. As noted in section 5.1, Schuppan s approach does not guarantee minimality, making these results less useful for applications such as business process modelling than those given by PLTL-MUP and procmine. Schuppan extends his approach in [8] by annotating results with a set of time states at which each formula in the unsatisfiable subset is used. It is clear that this additional information would assist with debugging, however, without a guarantee of minimality it appears that this could add additional noise to the process of debugging. 7 Future Work A number of avenues for future work within this topic are available. Firstly, as discussed in section 5, it is possible that significant performance improvements could be made by attempting an initial coarse reduction in the size of the input before sending it through the full theorem prover and minimisation procedure. One possible way to achieve this would be to partition the formula into subsets operating on distinct sets of propositional atoms. Each subset can then be tested to see if it is unsatisfiable. If neither subset is unsatisfiable, they can be combined and tested as one large set. As soon as an unsatisfiable subset of the input is found, it can be sent through the remainder of the minimisation process. This procedure can be recursively applied to each subset in order to allow small unsatisfiable subsets to be easily located. The amount of work that is performed by repeatedly calling the theorem prover can be reduced, as pltlbdd allows the

11 fixpoint calculation to be started with an existing BDD. In this case, we can use the fixpoint of a subset of Γ to start the fixpoint calculation for Γ, and thus preventing this work from being recomputed. Performance could also be improved by employing early quantification techniques. Early quantification is used within symbolic satisfiability solvers for classical propositional logic, which also involves existentially quantifying a large conjunction of BDDs [9]. In order to make this operation more efficient, quantification is pushed down in to the conjuncts, so that the size of the intermediate BDDs is reduced. The same technique could be employed when calculating XΓ in our procedure. However, this is unlikely to give significant improvements until the main bottleneck in the theorem prover can be reduced. 8 Conclusion We have presented a method which can be used to find minimal unsatisfiable subsets of input formulae within PLTL. This method uses a BDD-based theorem prover to encode a satisfiability information of the input set, which allows the satisfiability of subsets of this input to be queried efficiently. We then augment the input set by tagging each formula so that we can simulate turning off a single formula at a time. By combining these two inputs with standard BDD procedures, we are able to determine whether the input is already unsatisfiable, and if not, find formulae which can be safely removed. We have implemented and experimentally evaluated this procedure against other similar approaches, finding that while our approach performs well for some inputs, it is held back by the need to disable optimisations performed by the theorem prover. However, a hybrid approach combining our work with TRP++UC provided an effective middle ground in terms of performance while still guaranteeing minimality. Future work to address this issue has been suggested, including performing preprocessing to reduce the amount of work done by the theorem prover.

12 Bibliography [1] Sistla, A.P., Clarke, E.M.: The complexity of propositional linear temporal logics. Journal of the ACM (JACM) 32(3) (1985) [2] Huang, J.: MUP: a minimal unsatisfiability prover. In: Design Automation Conference, Proceedings of the ASP-DAC Asia and South Pacific. Volume 1., IEEE (2005) [3] Hustadt, U., Konev, B.: TRP++ 2.0: A temporal resolution prover. In: Automated Deduction CADE-19. Springer (2003) [4] Awad, A., Goré, R., Thomson, J., Weidlich, M.: An iterative approach for business process template synthesis from compliance rules. In: Advanced Information Systems Engineering, Springer (2011) [5] Schuppan, V.: Extracting unsatisfiable cores for LTL via temporal resolution. arxiv preprint arxiv: (2012) [6] Marrero, W.: Using BDDs to decide CTL. In: Tools and Algorithms for the Construction and Analysis of Systems. Springer (2005) [7] Oh, Y., Mneimneh, M.N., Andraus, Z.S., Sakallah, K.A., Markov, I.L.: AMUSE: a minimally-unsatisfiable subformula extractor. In: Proceedings of the 41st annual Design Automation Conference, ACM (2004) [8] Schuppan, V.: Enhancing unsatisfiable cores for LTL with information on temporal relevance. In: Submitted to the Eleventh International Workshop on Quantitative Aspects of Programming Languages and Systems, QAPL. (2013) [9] Huang, J., Darwiche, A.: Toward good elimination orders for symbolic sat solving. In: Tools with Artificial Intelligence, ICTAI th IEEE International Conference on, IEEE (2004) A Proof of Theorem 1 The following proof adapts the proof given in [2] to PLTL. Firstly, we consider the intuition given in section 4.1 that an assignment to the Y variables allows us to disable a single formula from Γ at a time. Let p be a function mapping truth assignments for Y to the index of the corresponding tag formula such that for some truth assignment α, T p(α) α evaluates to 1, while all other tag formulae evaluate to 0. This has the effect of removing φ p(α) while retaining all other formulae. We thus obtain the following lemma: Lemma 1. ˆΓ α = { Γ p(α) ˆ ˆΓ if 1 p(α) m otherwise We now restate and prove Theorem 1:

13 Theorem 1. A set of formulae Γ is minimal unsatisfiable iff XΓ has exactly m models over variables Y. [2] Proof. Let α be one of the 2 k possible assignments for variables Y. We first assume that Γ is minimal unsatisfiable. Then, Γ is unsatisfiable ( X. ˆΓ = ) and for all i such that 1 i m, Γ i is satisfiable ( X. ˆΓ i = ). Thus, according to Lemma 1 we have { ( X. ˆΓ ) α = X.( Γ ˆ X. Γ p(α) ˆ = if 1 p(α) m α ) = X. ˆΓ = otherwise Thus, X. ˆΓ has exactly m models over variables Y. Next, we assume that X. ˆΓ has exactly m models. It follows that Γ is unsatisfiable, as otherwise we would have X. ˆΓ = X. ˆΓ i =, in which case there are 2 k > m models over Y for X. ˆΓ. Now, as Γ is unsatisfiable, ( X. ˆΓ ) α = for all p(α) > m. Thus, in order for there to be m models, each Γ i must be satisfiable (so that ( X. ˆΓ ) α = for 1 p(α) m). Hence, Γ is unsatisfiable, but every Γ i is satisfiable, and therefore Γ is minimal unsatisfiable..