Towards Higher-Order Superposition and SMT. Jasmin Blanchette

Transcription

1 Towards Higher-Order Superposition and SMT Jasmin Blanchette

2 Outline Counterexample Generation 1. Nitpick 2. Nunchaku Proof Search 3. Sledgehammer 4. Matryoshka

3 1. Nitpick 1. A (countermodel) finder 1. for Isabelle/HOL Joint work with Alexander Krauss and Tobias Nipkow

4

5 Architecture HOL FORL SAT Isabelle Nitpick.Kodkod...SAT solver

6 Translation fixed finite cardinalities:?? try all cards. K for base types first-order τ1!! τn! bool τ1!! τn! τ higher-order σ! τ A1 An A1 An A + constraint A A σ times{

7 Translation Con Con Con Con 3 Con 2 3 Con 2 0 Nil 0 Nil datatypes codatatypes p = F p p0 = (λx. False) pi+1 = F pi inductive preds. p = F p p0 = (λx. True) pi+1 = F pi coinductive preds.

8 2. Nunchaku 2. A modular model finder 2. for higher-order logic Ongoing joint work with Simon Cruanes, Andrew Reynolds, and Cesare Tinelli

9 multiple frontends Isabelle/HOL, TLAPS, Coq, Lean, multiple backends CVC4, Kodkod, Paradox, SMBC, Leon, Vampire, more precision by better approximations more efficiency by using better backends and by letting them enumerate cardinalities

10 Simplified Translation Pipeline 1. Monomorphize 2. Specialize 3. Polarize 4. Encode (co)inductive predicates 5. Encode (co)recursive functions 6. Encode higher-order functions

11 Actual Translation Pipeline $ nunchaku --print-pipeline Pipeline: ty_infer convert skolem fork { mono elim_infinite elim_copy elim_multi_eqns specialize elim_match elim_codata polarize unroll skolem elim_ind_pred elim_quant lift_undefined model_clean close {smbc id} mono elim_infinite elim_copy elim_multi_eqns specialize elim_match fork { elim_codata polarize unroll skolem elim_ind_pred elim_data lambda_lift elim_hof elim_rec intro_guards elim_prop_args fork { elim_types model_clean close {to_fo elim_ite conv_tptp paradox id} model_clean close {to_fo fo_to_rel kodkod id} } polarize unroll skolem elim_ind_pred lambda_lift elim_hof elim_rec intro_guards model_clean close {to_fo flatten {cvc4 id}} } }

12 OCaml for Translation Pipeline...

13 Encoding of higher-order functions A simple approach Replace HO σ τ by hσ,τ Introduce appσ,τ : hσ,τ σ τ and ασ,τ Add guards whenever app is used Add extensionality axiom

14 Encoding of higher-order functions An annoying imprecision x. f x = g x? f = g rec fact x = (if x > 0 then x fact (x 1) else 1) rec bad x = (if x = 666 then x else fact x) fact = bad?

15 Encoding of higher-order functions A more precise approach Also introduce protoσ,τ : hσ,τ σ If (h, protoσ,τ h) in ασ,τ, then protoσ,τ h gives default value f = g HAS_PROTO f

16 Encoding of higher-order functions A more precise approach HAS_PROTO h := a : α. γ(a) = (h, proto h) APP h s := if a : α. γ(a) = (h, s) then app h s else app h (proto h) asserting HAS_PROTO h

17 3. Sledgehammer 2. Automatic proof search 2. for Isabelle/HOL Joint work with Sascha Böhme, Jia Meng, Tobias Nipkow, Larry Paulson, Makarius Wenzel, and many others

18 Does there exist a function f from reals to reals such that for all x and y, f(x + y 2 ) f(x) y? let lemma = prove (`!f:real->real. ~(!x y. f(x + y * y) - f(x) >= y)`, REWRITE_TAC[real_ge] THEN REPEAT STRIP_TAC THEN SUBGOAL_THEN `!n x y. &n * y <= f(x + &n * y * y) - f(x)` MP_TAC THENL [MATCH_MP_TAC num_induction THEN SIMP_TAC[REAL_MUL_LZERO; REAL_ADD_RID] THEN REWRITE_TAC[REAL_SUB_REFL; REAL_LE_REFL; GSYM REAL_OF_NUM_SUC] THEN GEN_TAC THEN REPEAT(MATCH_MP_TAC MONO_FORALL THEN GEN_TAC) THEN FIRST_X_ASSUM(MP_TAC o SPECL [`x + &n * y * y`; `y:real`]) THEN SIMP_TAC[REAL_ADD_ASSOC; REAL_ADD_RDISTRIB; REAL_MUL_LID] THEN REAL_ARITH_TAC; X_CHOOSE_TAC `m:num` (SPEC `f(&1) - f(&0):real` REAL_ARCH_SIMPLE) THEN DISCH_THEN(MP_TAC o SPECL [`SUC m EXP 2`; `&0`; `inv(&(suc m))`]) THEN REWRITE_TAC[REAL_ADD_LID; GSYM REAL_OF_NUM_SUC; GSYM REAL_OF_NUM_POW] THEN REWRITE_TAC[REAL_FIELD `(&m + &1) pow 2 * inv(&m + &1) = &m + &1`; REAL_FIELD `(&m + &1) pow 2 * inv(&m + &1) * inv(&m + &1) = &1`] THEN ASM_REAL_ARITH_TAC]);; John Harrison

19 Does there exist a function f from reals to reals such that for all x and y, f(x + y 2 ) f(x) y? [1] f(x + y 2 ) f(x) y for any x and y (given) [2] f(x + n y 2 ) f(x) n y for any x, y, and natural number n (by an easy induction using [1] for the step case) [3] f(1) f(0) m + 1 for any natural number m (set n = (m + 1) 2, x = 0, y = 1/(m + 1) in [2]) [4] Contradiction of [3] and the Archimedean property of the reals John Harrison

20 intermediate properties manual generated automatically

21

22

23

24

25

26 Proof assistants Automatic provers = Isabelle Isabelle vs. Vampire well suited for large formalizations but require intensive manual labor Sledgehammer fully automatic but no proof management

27 superposition select lemmas + translate to FOL = Isabelle HOL reconstruct proof SMT

28 superposition SMT refutational resolution rule term ordering equality reasoning redundancy criterion E, SPASS, Vampire, refutational SAT solver + congruence closure + quantifier instantiation + other theories (e.g. LIA, LRA) CVC4, verit, Yices, Z3,

29 Other hammers in proof assistants pre-sledgehammer Otter in ACL2 Bliksem in Coq Gandalf in HOL98 DISCOUNT, SPASS, etc., in ILF Otter, SPASS, etc., in KIV LEO, SPASS, etc., in ΩMEGA E, Vampire, etc., in Naproche... post-sledgehammer HOLyHammer for HOLs MizAR for Mizar SMTCoq/CVC4Coq for Coq SMT integration in TLAPS...

30 x Traditional encodings of HOL in FOL are inefficient HOL FOL higher-order features (e.g., currying) Suc n app(suc, n) types (possibly polymorphic) Suc n t(suc(t(n, nat)), nat) t(app(t(suc, fun(nat, nat)), t(n, nat)), nat)

31 More efficient encodings of higher-order features are possible HOL Suc x map f [x] = [ f x] λ x y. y + x p (x = x) FOL suc(x) map(f, [x]) = [app(f, x)] c(plus) p(eq(x, x))

32 More efficient encodings of types are possible HOL Suc x Nil Cons x xs x y x = off FOL suc(x) nil cons(a, x, xs) x y t(x, state) = off

33 Upon success, proofs are translated to Isabelle one-line detailed (Isar)

34 One-line proofs lemma "length (tl xs) length xs" by (metis diff_le_self length_tl) proof method lemmas usually fast and reliable lightweight cryptic sometimes slow (several seconds) often cannot deal with theories

35 Detailed (Isar) proofs lemma "length (tl xs) length xs" proof - have " x1 x2. (x1 nat) - x2 - x1 = 0 - x2" by (metis comm_monoid_diff_class.diff_cancel diff_right_commute) hence "length xs length xs = 0" by (metis zero_diff) hence "length xs - 1 length xs" by (metis diff_is_0_eq) thus "length (tl xs) length xs" by (metis length_tl) qed faster than one-liners higher reconstruction success rate self-explanatory technically more challenging not always so self-explanatory

36

37 Sledgehammer really works Developing proofs without Sledgehammer is like walking as opposed to running. Tobias Nipkow Larry Paulson I have recently been working on a new development. Sledgehammer has found some simply incredible proofs. I would estimate the improvement in productivity as a factor of at least three, maybe five. Sledgehammers have led to visible success. Fully automated procedures can prove 47% of the HOL Light/Flyspeck libraries, with comparable rates in Isabelle. These automation rates represent an enormous saving in human labor. Thomas Hales

38 Isabelle s pros and cons, according to my students 11.5 Sledgehammer 4 Nitpick 4 Isar 2.5 automation 2 IDE 1 Quickcheck 1 set theory 1 schematic variables 1 structural induction 1 classical logic 1 function induction 1 infix operators 1 "qed auto" 5 goal/assumption handling 4 weak logic (props as types, types as terms) 3 Sledgehammer on lists, HO goals, or induction 1 automatic induction 1 Sledgehammer-generated Isar 1 arithmetic 1 Isar 1 opaque proofs 1 double quotes around inner syntax 1 underdeveloped "fset" 1 proof reuse 1 no hnf for statements, not even definitions 1 guaranteed computability 1 forward "apply" in assumptions (drule?) 1 error messages in inner syntax 1 ltac (Eisbach?) 1 cannot click on fun to see definition (?) 1 tooltips for built-in functions etc.

39 Sledgehammer's main 3 weaknesses Higher-order "lost in translation" No induction Explosive search space

40 4. Matryoshka 2. Higher-order automatic 2. provers for proof assistants Ongoing joint work with Alex Bentkamp, Pascal Fontaine, Johannes Hölzl, Rob Lewis, Stephan Schulz, Uwe Waldmann, and many others

41 Vision: Take the Hard Labor out of Vision: Interactive Verification Push button automation for proof assistants (e.g. Coq) based on efficient higher-order (HO) provers λσ Π {} Discover Proof Using HO Provers HO super- position prover HO SMT solver λ Σ Π {} 4

42 Application: A Verified EasyChair PC members cannot review papers if they have a conflict of interest Proof today: using assms proof induction case (Step s a) thus?case proof (cases a) case (Cact ca) show?thesis using Step pref_conflict_isrev reach.step by simp next case (Uact ua) show?thesis proof (cases ua) case (upref confid uid p paperid pref) thus?thesis using Step unfolding Uact upref isrev_def2 by (blast dest: pref_conflict_isrevnth reach.step) qed (insert Step, simp add: Uact isrev_def2 u_defs pref_conflict_isrevnth_def)+ next case (UUact uua) show?thesis using Step unfolding UUact isrev_def2 by (meson IO_Automaton.reach.Step pref_conflict_isrevnth) qed simp+ qed (simp add: istate_def) Induction Rule Simplifier Arithmetic Procedure General Reasoner First-Order Provers via SLEDGEHAMMER fully automatic 5

43 Application: A Verified EasyChair PC members cannot review papers if they have a conflict of interest Proof today: using assms proof induction case (Step s a) thus?case proof (cases a) case (Cact ca) show?thesis using Step pref_conflict_isrev reach.step by simp next case (Uact ua) show?thesis proof (cases ua) case (upref confid uid p paperid pref) thus?thesis using Step unfolding Uact upref isrev_def2 by (blast dest: pref_conflict_isrevnth reach.step) qed (insert Step, simp add: Uact isrev_def2 u_defs pref_conflict_isrevnth_def)+ next case (UUact uua) show?thesis using Step unfolding UUact isrev_def2 by (meson IO_Automaton.reach.Step pref_conflict_isrevnth) qed simp+ qed (simp add: istate_def) Induction Rule Simplifier Arithmetic Procedure General Reasoner First-Order Provers via SLEDGEHAMMER fully automatic 5

44 Application: A Verified EasyChair PC members cannot review papers if they have a conflict of interest Proof today: using assms proof induction case (Step s a) thus?case proof (cases a) case (Cact ca) show?thesis using Step pref_conflict_isrev reach.step by simp next case (Uact ua) show?thesis proof (cases ua) case (upref confid uid p paperid pref) thus?thesis using Step unfolding Uact upref isrev_def2 by (blast dest: pref_conflict_isrevnth reach.step) qed (insert Step, simp add: Uact isrev_def2 u_defs pref_conflict_isrevnth_def)+ next case (UUact uua) show?thesis using Step unfolding UUact isrev_def2 by (meson IO_Automaton.reach.Step pref_conflict_isrevnth) qed simp+ qed (simp add: istate_def) Induction Rule Simplifier Arithmetic Procedure General Reasoner First-Order Provers via SLEDGEHAMMER boilerplate manual hints fully automatic 6

45 Application: A Verified EasyChair PC members cannot review papers if they have a conflict of interest Proof after Matryoshka: using assms proof induction case (Step s a) thus?case proof (cases a) case (Cact ca) show?thesis using Step pref_conflict_isrev reach.step by simp next case (Uact ua) show?thesis proof (cases ua) case (upref confid uid p paperid pref) thus?thesis using Step unfolding Uact upref isrev_def2 by (blast dest: pref_conflict_isrevnth reach.step) qed (insert Step, simp add: Uact isrev_def2 u_defs pref_conflict_isrevnth_def)+ next case (UUact uua) show?thesis using Step unfolding UUact isrev_def2 by (meson IO_Automaton.reach.Step pref_conflict_isrevnth) qed simp+ qed (simp add: istate_def) Σ {} Discover Proof Using HO Provers λ Π Σ {} λ Π missing proof fully automatic 7

46 My Grand Challenge Create efficient proof calculi and higher-order provers targeting proof assistants and their applications to software and hardware development by fusing and extending two lines of research: automatic proving & interactive proving Scientific Objectives SO1. SO2. SO3. SO4. Extend superposition and SMT to higher-order logic Design practical methods and heuristics based on benchmarks Conceive stratified architectures to build higher-order provers Integrate our provers into proof assistants (Coq, Isabelle, TLA + ) 8

47 SO1 Higher-Order Superposition (λsup) A "counterexample": a = β (λx. a) (f a) > f a > a The problem is not so much λ as β. 9

48 SO1 Higher-Order Superposition (λsup) First-order rule: D' t t' C' s[u] s' (D' C' s[t'] s')σ SUP-Left where σ = mgu(t, u) u is not a variable tσ t'σ sσ s'σ (t t' )σ is strictly maximal in (D' t t' )σ and no selection (s s' )σ is maximal in (C' s s' )σ or selected 9

49 SO1 Higher-Order Superposition (λsup) First-order rule: D' t t' C' s[u] s' (D' C' s[t'] s')σ SUP-Left where σ = mgu(t, u) u is not a variable tσ t'σ sσ s'σ (t t' )σ is strictly maximal in (D' t t' )σ and no selection (s s' )σ is maximal in (C' s s' )σ or selected We need sequences of unifiers 9

50 SO1 Higher-Order Superposition (λsup) First-order rule: D' t t' C' s[u] s' (D' C' s[t'] s')σ SUP-Left where σ = mgu(t, u) u is not a variable tσ t'σ sσ s'σ (t t' )σ is strictly maximal in (D' t t' )σ and no selection (s s' )σ is maximal in (C' s s' )σ or selected We need sequences of unifiers We need higher-order term ordering We also want proof-assistant-style HO rewriting 9

51 SO1 Higher-Order Term Orderings First-order Well-foundedness Transitivity Stability under substitution FO subterm property Compat. with FO contexts Totality for ground terms KBO LPO 10

52 SO1 Higher-Order Term Orderings First-order KBO LPO CPO/ HORPO Higher-order λfkbo λflpo Well-foundedness Transitivity Stability under substitution FO subterm property Compat. with FO contexts Totality for ground terms HO subterm property Compat. with HO contexts?? () () 10

53 SO3 Stratified Architecture main loop FO rules FO formulas HO rules HO formulas Inspired by Nelson Oppen (SMT) Base FO provers: E & verit Some scientific challenges: How to exploit derived FO formulas and/or candidate models to guide HO quantifier instantiation? How to generate certificates for reconstruction in proof assistants? First-Order Prover (e.g. verit) Matryoshka Prover (e.g. verihot) 11

54 SO3 Higher-Order SMT FO QI main loop HO QI First-order quantifier instantiation (QI): E-matching (triggers) Model-based QI Conflict-guided instantiation Congruence closure with free variables We need to extend these strategies to higher-order logic SAT Solver (e.g. MiniSat) First-Order Prover (e.g. verit) Matryoshka Prover (e.g. verihot) 12

55 SO4 Connection with Proof Assistants SAT Solver (e.g. MiniSat) First-Order Prover (e.g. verit) Matryoshka Prover (e.g. verihot) Proof Assistant (e.g. Coq) 13

56 SO4 Connection with Proof Assistants Dependent Type Theory Classical Higher-Order Logic Set Theory verihot verihot verihot HO E HO E HO E Coq Isabelle/HOL TLA + Agda HOL4 Isabelle/ZF Lean HOL Light Mizar Matita PVS Rodin (Event-B) 14

57 The m a t λ r y o s h k a Team Scientific Leader: Senior Collaborator: Postdoctoral Researchers: Ph.D. Students: Jasmin Blanchette Pascal Fontaine Johannes Hölzl Rob Lewis Alex Bentkamp Daniel El Ouraoui Hans-Jörg Schurr Petar Vukmirović Adam Ncy Adam Adam Adam Ncy Ncy Adam Associated Members: Other Collaborators: Stephan Schulz Uwe Waldmann Haniel Barbosa Simon Cruanes Simon Robillard & more Stgt SB Ncy Ncy Gbg 15

58 Matryoshka in one Slide Grand Challenge & Outcome Risks λ m a t r y o s h k a Create efficient proof calculi (λsup & λsmt) and stratified higher-order provers (HO E & verihot) to dramatically improve automation in proof assistants Efficient HO automation is a long-standing open problem Proposed stratified architecture has never been tried Impact The project will recast the methods of automatic proving to reach the goals of interactive proving Interactive verification will become a cost-effective option for building software & hardware of the highest quality 16