CMPS 217 Logic in Computer Science https://courses.soe.ucsc.edu/courses/cmps217/spring13/01 Lecture #17 1
The Complexity of FO-Truth on a Structure Structure A Complexity of Th(A) Structure of the natural numbers N = (N, +,, 0, 1) Undecidable Structure of the real numbers R = (R, +,, 0, 1) Decidable in EXPSPACE; PSPACE-hard Non-trivial finite structure PSPACE-complete 2
Reminder: Decidable and Undecidable Problems Definition: Let Q be a decision problem. Q is decidable (solvable) if there is an algorithm (Turing machine, via Church s Thesis) for solving the membership problem for the language associated with it. Q is undecidable (unsolvable) if no such algorithm exists 1 ( yes ) Input x Q? 0 ( no ) 3
Semi-Decidable Problems Definition: Let Q be a decision problem. Q is semi-decidable (recursively enumerable) if there is an algorithm (Turing machine, via Church s Thesis) such that if If x is a yes input to Q (i.e., a member of the language associated with Q), then the algorithm halts and outputs 1. If x is a no input to Q, then the algorithm does not halt. 1 ( yes ) Input x Q? 4
Semi-Decidable Problems The Halting Problem: Given a Turing machine M and an input x, does M halt on x? Fact: The Halting Problem is semi-decidable. Algorithm: Use a universal Turing machine to run M on x. Hilbert s 10 th Problem: Given a polynomial p(x 1,,x n ) with integer coefficients, does it have an all-integer solution? Fact: Hilbert s 10 th Problem is semi-decidable. Algorithm: Evaluate p(x 1,..,x n ) on every tuple a 1,,a n until you find a tuple such that p(a 1,,a n ) = 0. 5
Semi-Decidable Problems Theorem: Let Q be a decision problem. Then the following statements are equivalent: 1. Q is semi-decidable. 2. There is an algorithm that enumerates all yes inputs of Q (i.e., all members of the language associated with Q), and nothing else. Proof: (Sketch) 1. 2. Run the algorithm for the semi-decidability of Q on progressively more inputs and for progressively increasing intervals of time. 2. 1. Given an input, run the enumeration algorithm until the input is produced by the enumeration algorithm. 6
Decidable vs. Semi-Decidable Problems Theorem: Let Q be a decision problem. Then the following statements are equivalent: 1. Q is decidable. 2. Both Q and the complement Q c of Q are semi-decidable Proof: (Sketch) 1. 2. Easy exercise. 2. 1. Given an input, run the semi-decidability algorithm for Q and the semi-decidability algorithm for Q c in parallel. Exactly one of these two algorithms must terminate. If the semi-decidability algorithm for Q terminates, return 1; If the semi-decidability algorithm for Q c terminates, return 0. 7
Decidable vs. Semi-Decidable Problems Theorem: Let Q be a decision problem. Then the following statements are equivalent: 1. Q is decidable. 2. Both Q and the complement Q c of Q are semi-decidable Corollary: If Q is an undecidable problem, then at least one of Q and Q c is not semi-decidable. Corollary: 1. The complement of the Halting problem is not semi-decidable. 2. The complement of Hilbert s 10 th Problem is not semi-decidable. 8
The Finer Structure of Undecidable Problems Fact: Suppose that Q is an undecidable decision problem. Then exactly one of the following three possibilities holds: 1. Q is semi-decidable, but its complement Q c is not. 2. Q is not semi-decidable, but its complement Q c is. 3. Neither Q nor its complement Q c is semi-decidable. Examples: The Halting Problem is semi-decidable, but its complement is not. Total Turing Machine Problem: Given a Turing maching M, does it halt on every input? Neither the Total Turing Machine Problem nor its complement is semi-decidable. Neither Th(N) nor its complement is semi-decidable. 9
Validities and Finite Validities Recall the two main undecidability results about FO-logic: Theorem (Gödel, Church, Tarski 1930s): Let S be a signature containing a relation symbol of arity at least 2. The Validity Problem is undecidable, i.e., there is no algorithm to solve the following problem: given a FO-sentence ψ over S, is it valid? Theorem (Trakhtenbrot 1949): Let S be a signature containing a relation sumbol of arity at least 2. The Finite Validity Problem is undecidable, i.e., there is no algorithm to solve the following problem: given a FO-sentence ψ over S, is it finitely valid? Question: What can we say about the semi-decidability of these two problems? 10
Finite Validities Theorem: The Complement of the Finite Validity Problem is semi-decidable. Proof: Given a FO-sentence ψ, consider its negation ( ψ) and search for a finite structure A such that A ( ψ). If such a finite structure is found, then ψ is not finitely valid. Corollary: The Finite Validity Problem is not semi-decidable. Equivalently, there is no algorithm that enumerates all finitely valid FO-sentences. 11
Validities Theorem (Gödel s Completeness Theorem): The Validity Problem is semi-decidable. Equivalently, there is an algorithm that enumerates all valid FO-sentences. Note: The semi-decidability of the Validity Problem yields a proof procedure for proving all valid FO-sentences (the proof of a valid FO-sentence is the run of the algorithm for semi-decidability). Corollary: The complement of the Validity Problem is not semi-decidable. 12
The Undecidability of FO-Logic Problem Decidable Semi-decidable Semi-decidable Complement Validity No Yes No Finite Validity No No Yes Th(N) No No No 13
Validities Theorem (Gödel s Completeness Theorem): The Validity Problem is semi-decidable. Equivalently, there is an algorithm that enumerates all valid FO-sentences. In what follows, we will give a proof of this theorem for the special case of FO-sentences without the equality symbol =. The proof will be carried out in three steps. Step 1: We will prove Skolem s Theorem for FO-sentences. Step 2: We will prove Herbrand s Theorem for FO-sentences without the equality symbol = Step 3: We will show that the semi-decidablity of the Validity Problem for FO-sentences without the equality symbol follows from Step 1, Step 2, and the Compactness Theorem for Propositional Logic. 14
Skolem s Theorem and Herbrand s Theorem Skolem s Theorem: There is a polynomial-time algorithm such that, given a FO-sentence ϕ, it returns a Π 1 -sentence ϕ over a signature expanded with additional function symbols such that ϕ is satisfiable if and only if ϕ* is satisfiable. Moreover, if the equality symbol = does not occur in ϕ, then it does not occur in ϕ* either. Herbrand s Theorem: For every Π 1 -sentence ψ without the equality symbol =, there is a (perhaps infinite) set H(ψ) of propositional formulas such that ψ is satisfiable if and and only if H(ψ) is satisfiable. Moreover, there is an algorithm that, given ψ, it enumerates H(ψ) (i.e., it produces a list of all elements of H(ψ). 15
Skolem s Theorem and Herbrand s Theorem Fact: The Completeness Theorem for FO-sentences without = can be obtained by combining Skolem s Theorem, Herbrand s Theorem, and the Compactness Theorem for Propositional Logic. Let ϕ be a FO-sentence without =. We now have that: ϕ is valid if and only if ( ϕ) is unsatisfiable if and only if (by Skolem s Theorem) ( ϕ)* is unsatisfiable if and only if (by Herbrand s Theorem) H(( ϕ)*) is unsatisfiable if and only if (by the Compactness Theorem for Prop. Logic) there is a finite subset H 0 of H(( ϕ)*) that is unsatisfiable. 16
Quantifiers and Functions Consider a FO-sentence of the form x y θ, where θ is quantifier-free. Suppose that A is a structure such that A x y θ. This means that for every element a in A, there is an element b in A such that A, a, b θ (i.e. A, s θ, where s(x) =a, s(y) =b) Now, suppose that for every a in A, we select such a b. This means that there is a function f*: A A such that for every a in A, we have that A, a, f*(a) θ. (Note: If A is an infinite set, then this step uses the Axiom of Choice). In turn, this means that A f x θ(y/f(x)), where f is a new unary function symbol and θ(y/f(x)) is the quantifier-free formula obtained from ψ by replacing each occurrence of y by the term f(x). Conversely if A f x θ(y/f(x)), then A x y θ. Consequently, x y θ f x θ(y/f(x)). Note: f x ψ(y/f(x)) is a formula of Second-Order Logic. 17
Quantifiers and Functions So, we saw that x y θ f x θ(y/f(x)), where f is a new unary function symbol. This extends to sentence of the form x 1 x k y θ, namely, x 1 x k y θ f x 1 x k θ(y/f(x 1,,x k )), where f is new k-ary function symbol. This transformation plays an key role in the proof of Skolem s Theorem. 18
Skolem s Theorem Theorem: There is a polynomial-time algorithm such that, given a FO-sentence ϕ, it returns a Π 1 -sentence ϕ over a signature expanded with additional function symbols f 1, f 2,, f k such that ϕ f 1 f 2 f k ϕ*. In particular, ϕ is satisfiable if and only if ϕ* is satisfiable. Moreover, if the equality symbol = does not occur in ϕ, then it does not occur in ϕ* either. Proof: Step 1: Bring ϕ to prenex normal form. Step 2: If ϕ begins with, then apply repeatedly the transformation x 1 x k yθ f x 1 x k θ(y/f(x 1,,x k )). Step 3: If ϕ begins with, then apply the transformation z θ f w θ(z/f(w)) (to see that these two formulas are logically equivalent: - left to right: take a witness c for z and let f be the constant function f(w) = c. - right to left: if b is any element, then f(b) is a witness for z.) 19
Skolem s Theorem: Examples Example 1: x 1 y 1 x 2 θ f 1 x 1 x 2 θ(y 1 /f 1 (x 1 )) Example 2: x 1 x 2 y 1 y 2 x 3 y 3 θ f 1 x 1 x 2 y 2 x 3 y 3 θ(y 1 /f 1 (x 1,x 2 )) f 1 f 2 x 1 x 2 x 3 y 3 θ(y 1 /f 1 (x 1,x 2 ), y 2 /f 2 (x 1,x 2 ) f 1 f 2 f 3 x 1 x 2 x 3 θ(y 1 /f 1 (x 1,x 2 ), y 2 /f 2 (x 1,x 2 ),y 3 /f 3 (x 1,x 2,x 3 )) 20
Skolem s Theorem: Examples Example 3: x 1 y 1 x 2 θ f 1 w 1 y 1 x 2 θ(x 1 /f 1 (w 1 )) f 1 f 2 w 1 y 1 θ(x 1 /f 1 (w), x 2 /f 2 (w 1,y 1 )) 21
Towards Herbrand s Theorem Let ψ be a Π 1 -sentence without equality =. As a stepping stone to Herbrand s Theorem, we will show that the following statements are equivalent: ψ is satisfiable (i.e., there is a structure A such that A ψ) ψ is satisfiable by some Herbrand structure (i.e., there is a Herbrand structure A such that A ψ). Key idea behind Herbrand structures: Use the syntax of first-order logic to build structures. In particular, use the terms as elements of the universe of structures. 22
Herbrand Universe and Herbrand Structures Definition. Let ψ be a FO-sentence. The Herbrand Universe U(ψ) of ψ is the set of all possible terms obtained from the function symbols and the constant symbols occurring in ψ. If ψ has no function or constant symbols, then U(ψ) consists of a new constant symbol c and all terms obtained from c and the function symbols occurring in ψ. A Herbrand structure associated with ψ is a structure A such that the universe of A is the Herbrand universe U(ψ). the terms are interpreted on A by themselves there no restrictions on the relations of A. Thus, to define a Herbrand structure, it suffices to define its relations. 23
Herbrand Structures Example: Let ψ be the formula x (R(x,c) Ç R(x,f(d)) The Herbrand Universe is the following infinite set U(ψ) = { c, d, f(c), f(d), f(f(c)), f(f(d)), f(f(f(c))), } Herbrand Structure A = (U(ψ), R, f, c, d) with f (c) = f(c), f (d) = f(d), (more generally, f (t) = f(t)). R = {(c,c), (d,c)} Different Herbrand structures can be obtained by changing the relation R, while keeping everything else the same. For example, consider the Herbrand structure B in which the relation symbol R is interpreted by the relation R = {(c,d), (d,d), (d, f(d)), (f(f(c)), f(f(f(d)))} 24
Herbrand Structures Theorem A: Let ψ be a Π 1 -sentence without equality. Then the following statements are equivalent: 1. ψ is satisfiable. 2. ψ is satisfiable by some Herbrand structure. Proof: Only the direction 1. 2. is not obvious. Let f 1,,f n, R 1,,R m, c 1,,c k be the non-logical symbols occurring in ψ. Assume that there is a structure A = (A, f* 1,,f* n,r* 1,,R* m, c* 1,,c* k ) such that A ψ. Let B be the Herbrand structure with universe U(ψ) and with relations R 1,,R m defined as follows: if R i is a relation symbol of arity r and t 1,,t r are closed terms, then (t 1, t r ) R i if and only if A R* i (t A 1,,t A r), where t A j is the interpretation of the term t j on A. We will show that B ψ. 25
Herbrand Structures Lemma: Let θ be a quantifier-free formula with variables x 1, x n and without equality =. For all closed terms t 1,,t n, the following statements are equivalent: 1. B θ(x 1 /t 1,,x n /t n ) 2. A θ(x 1 /t A 1,,x n /t A n). Proof: By induction on the construction of quantifier-free formulas. The base case of atomic formulas is true because of the way B was defined. Note: This lemma fails if we equalities = are allowed. For example, A may satisfy c* 1 = f* 2 (c 3 ), while B does not. 26
Herbrand Structures Proof (continued): Since ψ is a Π 1 -sentence, it is of the form x 1 x n θ, where θ is quantifier-free. We have to show that B x 1 x n θ. Take n elements from the universe U(ψ) of B. They must be closed terms t 1,,t n. Since A x 1 x n θ, we have that A θ(x 1 /t A 1,,x n /t A n). Hence, by the Lemma, we have that B θ(x 1 /t 1,,x n /t n ). This completes the proof that B x 1 x n θ. 27
Illustration Example (continued): Let ψ be the formula x (R(x,c) Ç R(x,f(d)) The Herbrand Universe is the following infinite set U(ψ) = { c, d, f(c), f(d), f(f(c)), f(f(d)), f(f(f(c))), } ψ is satisfiable. For example, A ψ, where A = ({a}, f*, R*, a,a), f*(a) = a, and R* = {(a,a)}. Note that f*(f*(a)) = a, f*(f*(f*(a))) = a, etc. Thus, for every closed term t, we have that t A = a. Let B = (U(ψ), f, R, c, d) be the Herbrand model such that (t 1,t 2 ) R if and only if (t A 1,tA 2) R*. This means that R = {(t 1,t 2 ): t 1, t 2 are closed terms}. Clearly, B ψ. 28
Herbrand s Theorem Recall that our goal is to establish Herbrand s Theorem: For every Π 1 -sentence ψ without the equality symbol =, there is a (perhaps infinite) set H(ψ) of propositional formulas such that ψ is satisfiable if and and only if H(ψ) is satisfiable. Moreover, there is an algorithm that, given ψ, it enumerates H(ψ) (i.e., it produces a list of all elements of H(ψ). Definition: Let ψ is a Π 1 -sentence of the form x 1 x n θ, where θ is quantifier-free. The Herbrand expansion H(ψ) of ψ is the set H(ψ) = { θ(x 1 /t 1,,x n /t n ): t 1,, t n are in U(ψ) } 29
Herbrand Expansions Definition: Let ψ is a Π 1 -sentence of the form x 1 x n θ, where θ is quantifier-free. The Herbrand expansion H(ψ) of ψ is the set H(ψ) = { θ(x 1 /t 1,,x n /t n ): t 1,, t n are in U(ψ) } Note: On the face of it, H(ψ) is a set of quantifier-free sentences. However, it can be identified, and it will be identified, with a set of propositional formulas obtained from the sentences of H(ψ) by replacing each distinct atomic sentence R(t 1,,t m ) by a distinct propositional variable. Fact: There is an algorithm that, given a Π 1 -sentence ψ, it enumerates all members of the Herbrand expansion H(ψ) of ψ. 30
Herbrand Expansions Example 1: Let ψ be the formula x (R(c,x) Ç R(x,f(c)) H(ψ) contains the quantifier-free sentences: R(c,c) Ç R(c,f(c)) x/c R(c,f(c)) Ç R(f(c),f(c)) x/f(c) R(c,f(f(c)) Ç R(f(f(c)), f(c)) x/f(f(c)) As a set of propositional formulas, H(ψ) contains the formulas P 1 Ç P 2 P 2 Ç P 3 P 4 Ç P 5 31
Herbrand Expansions Example 2: Let ψ be the formula x (P(x) Ç P(f(c)) H(ψ) contains the quantifier-free sentences: P(c) Ç P(f(c)) x/c P(f(c)) Ç P(f(c)) x/f(c) P(f(f(c)) Ç P(f(c)) x/f(f(c)) As a set of propositional formulas, H(ψ) contains the formulas P 1 Ç P 2 P 2 Ç P 2 P 3 Ç P 2 32
Herbrand s Theorem Herbrand s Theorem: Let ψ be a Π 1 -sentence ψ without the equality symbol = and let H(ψ) be its Herbrand expansion. Then the following statements are equivalent ψ is satisfiable H(ψ) is satisfiable (as a set of propositional formulas) Proof: By Theorem A, ψ is satisfiable if and only if ψ is satisfiable by some Herbrand structure A. Recall that in defining a Herbrand structure A, we need only define the relations of A. This means that we only need to decide which atomic sentences R(t 1,,t n ) are true on A or, equivalently, we need to decide the truth values of the propositional variables occurring in H(ψ). It follows that the Herbrand structures that satisfy ψ are in a one-to-one correspondence with the satisfying truth assignments of H(ψ). In particular, ψ is satisfiable if and only if H(ψ) is satisfiable. 33