Inference in First-Order Logic Dr. Richard J. Povinelli rev 1.0, 10/7/2001 Page 1
Objectives You should! be able to analyze and employ logical inference in first-order logic. rev 1.0, 10/7/2001 Page 2
Outline Reducing first-order inference to propositional inference Unification Generalized Modus Ponens Forward and backward chaining Logic programming Resolution rev 1.0, 10/7/2001 AIMA Slides Stuart Russell and Peter Norvig, 1998 Page 3
A brief history of reasoning 450B.C. Stoics: propositional logic, inference (maybe) 322B.C. Aristotle: syllogisms (inference rules), quantifiers 1565 Cardano: probability theory (propositional logic + uncertainty) 1847 Boole: propositional logic (again) 1879 Frege: first-order logic 1922 Wittgenstein: proof by truth tables 1930 Gödel: complete algorithm for FOL 1930 Herbrand complete algorithm for FOL (reduce to propositional) 1931 Gödel: complete algorithm for arithmetic 1960 Davis/Putnam practical algorithm for propositional logic 1965 Robinson: practical algorithm for FOL resolution rev 1.0, 10/7/2001 AIMA Slides Stuart Russell and Peter Norvig, 1998 Page 4
Proofs Sound inference: find α such that KB = α. Proof process is a search, operators are inference rules. Modus Ponens (MP) And-Introduction (AI) Universal Elimination (UE) τ must be a ground term (i.e., no variables) rev 1.0, 10/7/2001 AIMA Slides Stuart Russell and Peter Norvig, 1998 Page 5
Example proof Bob is a buffalo Pat is a pig Buffaloes outrun pigs Prove:Bob outruns Pat 1. Buffalo(Bob) 2. Pig(Pat) 3. x, y Buffalo(x) Pig(y) Faster(x,y) rev 1.0, 10/7/2001 AIMA Slides Stuart Russell and Peter Norvig, 1998 Page 6
Example proof Bob is a buffalo Pat is a pig Buffaloes outrun pigs Prove:Bob outruns Pat AI 1 & 2 1. Buffalo(Bob) 2. Pig(Pat) 3. x, y Buffalo(x) Pig(y) Faster(x,y) 4. Buffalo(Bob) Pig(Pat) rev 1.0, 10/7/2001 AIMA Slides Stuart Russell and Peter Norvig, 1998 Page 7
Example proof Bob is a buffalo Pat is a pig Buffaloes outrun pigs Prove:Bob outruns Pat AI 1 & 2 UE 3, {x/bob, y/pat} 1. Buffalo(Bob) 2. Pig(Pat) 3. x, y Buffalo(x) Pig(y) Faster(x,y) 4. Buffalo(Bob) Pig(Pat) 5. Buffalo(Bob) Pig(Pat) Faster(Bob, Pat) rev 1.0, 10/7/2001 AIMA Slides Stuart Russell and Peter Norvig, 1998 Page 8
Example proof Bob is a buffalo Pat is a pig Buffaloes outrun pigs Prove:Bob outruns Pat AI 1 & 2 UE 3, {x/bob, y/pat} MP 4 & 5 1. Buffalo(Bob) 2. Pig(Pat) 3. x, y Buffalo(x) Pig(y) Faster(x,y) 4. Buffalo(Bob) Pig(Pat) 5. Buffalo(Bob) Pig(Pat) Faster(Bob, Pat) 6. Faster(Bob,Pat) rev 1.0, 10/7/2001 AIMA Slides Stuart Russell and Peter Norvig, 1998 Page 9
CAT Prove It Given! 1. Giraffe(Tom)! 2. Lion(Fred)! 3. x, y Giraffe(x) Lion(y) Taller(x,y) Prove that Tom is bigger than Fred Work with a partner for 5 minutes. rev 1.0, 10/7/2001 Page 10
Universal instantiation (UI) Every instantiation of a universally quantified sentence is entailed by it: α Subst ({ v g}, α ) for any variable v and ground term g ( ) ( ) ( ) E. g., x King x Greedy x Evil x yields ( ) ( ) ( ) ( ) ( ) ( ) ( ( )) ( ( )) ( ( )) King John Greedy John Evil John King Richard Greedy Richard Evil Richard King Father John Greedy Father John Evil Father John rev 1.0, 10/7/2001 Page 11
Existential instantiation (EI) For any sentence α, variable v, and constant symbol k that does not appear elsewhere in the knowledge base: v α Subst E. g., x Crown x OnHead x, John yields y ( ) ({ v k }, α ) ( ) ( ) ( ) (, ) Crown C OnHead C John y ( ) y Another example: from x d x dy = x we obtain d e dy = e 1 1 provided C is a new constant symbol, called a Skolem constant 1 y provided e is a new constant symbol rev 1.0, 10/7/2001 Page 12
Existential instantiation cont. UI can be applied several times to add new sentences; the new KB is logically equivalent to the old EI can be applied once to replace the existential sentence; the new KB is not equivalent to the old, but is satisfiable iff the old KB was satisfiable rev 1.0, 10/7/2001 Page 13
Reduction to propositional inference Suppose the KB contains just the following: x King ( x ) Greedy ( x ) Evil ( x ) King ( John ) Greedy ( John ) Brother ( Richard, John ) Instantiating the universal sentence in all possible ways, we have King ( John ) Greedy ( John ) Evil ( John ) King ( Richard ) Greedy ( Richard ) Evil ( Richard ) King ( John ) Greedy ( John ) Brother ( Richard, John ) The new KB is propositionalized: proposition symbols are King ( John ), Greedy ( John ), Evil ( John ), King ( Richard ),etc. rev 1.0, 10/7/2001 Page 14
Reduction cont. Claim: a ground sentence* is entailed by new KB iff entailed by original KB Claim: every FOL KB can be propositionalized so as to preserve entailment Idea: propositionalize KB and query, apply resolution, return result Problem: with function symbols, there are infinitely many ground terms,! e.g., Father(Father(Father(John))) Theorem: Herbrand (1930). If a sentence α is entailed by an FOL KB, it is entailed by a finite subset of the propositional KB Idea: For n = 0 to do! create a propositional KB by instantiating with depth-n terms! see if α is entailed by this KB Problem: works if α is entailed, loops if α is not entailed Theorem: Turing (1936), Church (1936), entailment in FOL is semidecidable rev 1.0, 10/7/2001 Page 15
Problems with propositionalization Propositionalization seems to generate lots of irrelevant sentences. E.g., from ( ) ( ) ( ) ( ) ( ) (, ) x King x Greedy x Evil x King John y Greedy y Brother Richard John it seems obvious that Evil(John), but propositionalization produces lots of facts such as Greedy(Richard) that are irrelevant With p k-ary predicates and n constants, there are pn k instantiations! rev 1.0, 10/7/2001 Page 16
Search with primitive inference rules Operators are inference rules States are sets of sentences Goal test checks state to see if it contains query sentence AI, UE, MP is a common inference pattern Problem: branching factor huge, esp. for UE Idea: find a substitution that makes the rule premise match some known facts a single, more powerful inference rule rev 1.0, 10/7/2001 AIMA Slides Stuart Russell and Peter Norvig, 1998 Page 17
Unification A substitution σ unifies atomic sentences p and q if pσ = qσ p Knows(John, x) Knows (John, x) Knows (John, x) q Knows (John, Jane) Knows (y, OJ) Knows (y, Mother(y)) σ rev 1.0, 10/7/2001 AIMA Slides Stuart Russell and Peter Norvig, 1998 Page 18
Unification A substitution σ unifies atomic sentences p and q if pσ = qσ p Knows(John, x) Knows (John, x) Knows (John, x) q Knows (John, Jane) Knows (y, OJ) Knows (y, Mother(y)) σ {x/jane} {x/john; y/oj } {y/john;x/mother(john)} Idea: Unify rule premises with known facts, apply unifier to conclusion Example: if we know q and Knows(John, x) Likes(John, x) then we conclude! Likes(John, Jane)! Likes(John, OJ)! Likes(John, Mother(John)) rev 1.0, 10/7/2001 AIMA Slides Stuart Russell and Peter Norvig, 1998 Page 19
Generalized Modus Ponens (GMP) Example! p 1 = Faster(Bob,Pat)! p 2 = Faster(Pat,Steve) ( ) p 1, p2,, pn, p1 p2 pn q q, θ! p 1 p 2 q = Faster(x, y) Faster(y, z) Faster(x, z)! σ = {x/bob, y/pat, z/steve}! qσ = Faster(Bob, Steve) GMP used with KB of definite clauses (exactly one positive literal): either a single atomic sentence or (conjunction of atomic sentences) (atomic sentence) All variables assumed universally quantified where p θ = pθ for all i i i rev 1.0, 10/7/2001 AIMA Slides Stuart Russell and Peter Norvig, 1998 Page 20
Soundness of GMP Need to show that! p 1,, p n, (p 1 p n q)! qθ! provided that p i σ = p i σ for all i Lemma: For any definite clause p, we have p! pθ by UE 1. (p 1 p n q)! (p 1 p n q)θ = (p 1 θ p n θ) qθ ) 2. p 1,, p n! p 1 p n! p 1 θ p n θ 3. From 1 and 2, qθ follows by simple MP rev 1.0, 10/7/2001 AIMA Slides Stuart Russell and Peter Norvig, 1998 Page 21
Forward chaining When a new fact p is added to the KB! for each rule such that p unifies with a premise " if the other premises are known " then add the conclusion to the KB and continue chaining Forward chaining is data-driven! inferring properties and categories from percepts rev 1.0, 10/7/2001 AIMA Slides Stuart Russell and Peter Norvig, 1998 Page 22
Forward chaining example Add facts 1, 2, 3, 4, 5, 7 in turn. Number in [] = unification literal; indicates rule firing 1. Buffalo(x) Pig(y) Faster(x, y) 2. Pig(y) Slug(z) Faster(y, z) 3. Faster(x, y) Faster(y, z) Faster(x, z) 4. Buffalo(Bob) [1a, ] 5. Pig(Pat) [1b, ], [2a, ] 6. Faster(Bob, Pat) [3a, ], [3b, ] 7. Slug(Steve) [2b, ] 8. Faster(Pat, Steve) [3a, ], [3b, ] 9. Faster(Bob, Steve) [3a, ], [3b, ] rev 1.0, 10/7/2001 AIMA Slides Stuart Russell and Peter Norvig, 1998 Page 23
Backward chaining When a query q is asked! if a matching fact q is known, return the unifier! for each rule whose consequent q matches q attempt to prove each premise of the rule by backward chaining Some added complications in keeping track of the unifiers More complications help to avoid infinite loops Two versions: find any solution, find all solutions Backward chaining is the basis for logic programming, e.g., Prolog rev 1.0, 10/7/2001 AIMA Slides Stuart Russell and Peter Norvig, 1998 Page 24
Backward chaining example 1. Pig(y) Slug(z) Faster(y, z) 2. Slimy(z) Creeps(z) Slug(z) 3. Pig(Pat) 4. Slimy(Steve) 5. Creeps(Steve) rev 1.0, 10/7/2001 AIMA Slides Stuart Russell and Peter Norvig, 1998 Page 25
Completeness in FOL Procedure i is complete if and only if! KB - i α whenever KB! α Forward and backward chaining are complete for Horn KBs, but incomplete for general first-order logic Example from! PhD(x) HighlyQualified(x)! PhD(x) EarlyEarnings(x)! HighlyQualified(x) Rich(x)! EarlyEarnings(x) Rich(x) should be able to infer Rich(Me) rev 1.0, 10/7/2001 AIMA Slides Stuart Russell and Peter Norvig, 1998 Page 26
CAT Infer Rich(Me) Given! PhD(x) HighlyQualified(x)! PhD(x) EarlyEarnings(x)! HighlyQualified(x) Rich(x)! EarlyEarnings(x) Rich(x) Infer Rich(Me) using forward and backward chaining Work with a partner for 5 minutes. rev 1.0, 10/7/2001 Page 27
Completeness in FOL Were you able to infer Rich(Me)? Does a complete algorithm exist? rev 1.0, 10/7/2001 Page 28
Resolution Entailment in first-order logic is only semidecidable:! can find a proof of α if KB = α! cannot always prove that KB α Cf. Halting Problem: proof procedure may be about to terminate with success or failure, or may go on for ever Resolution is a refutation procedure:! to prove KB = ff, show that KB α is unsatisfiable Resolution uses KB, α in CNF (conjunction of clauses) Resolution inference rule combines two clauses to make a new one: Inference continues until an empty clause is derived (contradiction) rev 1.0, 10/7/2001 AIMA Slides Stuart Russell and Peter Norvig, 1998 Page 29
Resolution inference rule Basic propositional version: Full first-order version: For example rev 1.0, 10/7/2001 AIMA Slides Stuart Russell and Peter Norvig, 1998 Page 30
Conjunctive Normal Form Literal = (possibly negated) atomic sentence, e.g., Rich(Me) Clause = disjunction of literals, e.g., Rich(Me) Unhappy(Me) The KB is a conjunction of clauses Any FOL KB can be converted to CNF as follows: 1. Replace P Q by P Q 2. Move inwards, e.g., x P becomes x P 3. Standardize variables apart, e.g., x P x Q becomes x P y Q 4. Move quantifiers left in order, e.g., x P y Q becomes x y P Q 5. Eliminate by Skolemization (next slide) 6. Drop universal quantifiers 7. Distribute over e.g., (P Q) R becomes (P Q) (P R) rev 1.0, 10/7/2001 AIMA Slides Stuart Russell and Peter Norvig, 1998 Page 31
Skolemization x Rich(x) becomes Rich(G1) where G1 is a new Skolem constant More tricky when is inside Example Everyone has a heart x Person(x ) y Heart(y) Has(x, y) Incorrect:! x Person(x ) Heart(H1) Has(x, H1) Correct:! x Person(x ) Heart(H (x)) Has(x, H (x))! where H is a new symbol ( Skolem function ) Skolem function arguments: all enclosing universally quantified variables rev 1.0, 10/7/2001 AIMA Slides Stuart Russell and Peter Norvig, 1998 Page 32
Resolution proof To prove α:! negate it! convert to CNF! add to CNF KB! infer contradiction Example: to prove Rich(me), add Rich(me) to the CNF KB! PhD(x) HighlyQualified(x)! PhD(x) EarlyEarnings(x)! HighlyQualified(x) Rich(x)! EarlyEarnings(x) Rich(x) rev 1.0, 10/7/2001 AIMA Slides Stuart Russell and Peter Norvig, 1998 Page 33
Resolution proof rev 1.0, 10/7/2001 AIMA Slides Stuart Russell and Peter Norvig, 1998 Page 34