Interpolation theorems, lower bounds for proof. systems, and independence results for bounded. arithmetic. Jan Krajcek

Transcription

1 Interpolation theorems, lower bounds for proof systems, and independence results for bounded arithmetic Jan Krajcek Mathematical Institute of the Academy of Sciences Zitna 25, Praha 1, , Czech Republic Abstract A proof of the (propositional) Craig interpolation theorem for cut-free sequent calculus yields that a sequent with a cut-free proof (or with a proof with cut-formulas of restricted form; in particular, with only analytic cuts) with k inferences has an interpolant whose circuit-size is at most k. We give a new proof of the interpolation theorem based on a communication complexity approach which allows a similar estimate for a larger class of proofs. We derive from it several corollaries: 1. Feasible interpolation theorems for the following proof systems: (a) resolution. (b) a subsystem of LK corresponding to the bounded arithmetic theory S 2 2 (). (c) linear equational calculus. (d) cutting planes. 2. New proofs of the exponential lower bounds (for new formulas) (a) for resolution ([15]). (b) for the cutting planes proof system with coecients written in unary ([4]). 3. An alternative proof of the independence result of [43] concerning the provability of circuit-size lower bounds in the bounded arithmetic theory S2 2 () Mathematics Subject Classication. Primary 03F20, 03B05, 03F30; Secondary 68Q25. Partially supported by the US - Czechoslovak Science and Technology Program grant # 93025, and by the grant # of the AV CR. 1

2 In the other direction we show that a depth 2 subsystem of LK does not admit feasible monotone interpolation theorem (the so called Lyndon theorem), and that a feasible monotone interpolation theorem for the depth 1 subsystem of LK would yield new exponential lower bounds for resolution proofs of the weak pigeonhole principle. Introduction The interpolation theorem proved by Craig [11, 12] is a basic result in logic. It says that whenever an implication A?! B is valid then there is a third formula I, an interpolant, which contains only those symbols of the language occurring in both A and B and such that the two implications A?! I I?! B are both valid. The theorem holds for propositional logic as well as for the rst order logic but we shall conne our attention to propositional logic in this paper. The question of nding an interpolant for the implication is quite relevant to computational complexity theory. To see this let U and V be two disjoint N P-subsets f0; 1g. It is well known that there are sequences of propositional formulas A n (p 1 ; : : :; p n ; q 1 ; : : :; q tn ) and B n (p 1 ; : : :; p n ; r 1 ; : : :; r sn ) such that the size of A n and B n is n O(1) and and U \ f0; 1g n = f( 1 ; : : :; n ) 2 f0; 1g n j 9 1 ; : : :; tn A n (; ) holdsg V \ f0; 1g n = f( 1 ; : : :; n ) 2 f0; 1g n j 9 1 ; : : :; sn B n (; ) holdsg : The assumption that U \ V = ; is equivalent to the statement that the implications A n?! :B n are all tautologically valid. If I n (p) is an interpolant (hence only atoms p 1 ; : : :; p n occur in I n ) then the set [ W := f 2 f0; 1g n j I n () holds g n separates U from V : U W and W \ V = ; : Hence an estimate of the complexity of propositional interpolation formulas in terms of the complexity of an implication yields an estimate to the computational complexity of a set separating U from V. For example, if one could 2

3 always nd such an interpolant whose formula-size (or circuit-size; recall that a circuit-size of a formula I is the number of dierent subformulas occurring in it) is polynomial in the size of the implication then N P \ con P N C 1 =poly (or N P \ con P P=poly). This is because for U 2 N P \ con P we may take for V the complement of U and hence it must hold that W = U. This example can be understood as a conditional lower bound to the size of interpolants; it was rst noted by Mundici [30, 31, 32]. For predicate logic there are lower bounds in terms of recursion theory, see [28, 13], for other connections to computer science see [14]. The question we shall study in this paper is a bit dierent. Problem: Given a propositional proof system, estimate the circuit-size of an interpolant of an implication in terms of the size of the shortest proof of the implication. Presumably one gets dierent estimates for dierent proof systems and, in particular, not all proof systems should admit polynomial upper bounds. However, this is an open problem. The proof of Craig interpolation theorem [11, 12] via cut-elimination (see, for example, [45] or [20, 4.3]) shows that an implication whose cut-free proof in the sequent calculus has k steps has an interpolant with circuit-size at most k. The reason for studying this problem is that a good upper bound for a proof system P yields lower bounds on the size of P -proofs. In particular, a pair of N P-sets U and V inseparable by a set of small complexity yields a sequence of implications A n?! :B n which cannot have short P -proofs (as the assumed good interpolation yields feasible upper bound to the complexity of I n and hence of W ). This idea was discussed in Krajcek [19] but no lower bounds were obtained there in this way. Our interest in this question was renewed by a remark in Razborov [43] that the results underlying the unprovability results there are certain interpolation theorems for fragments of second order bounded arithmetic. It occurred to us that these interpolation theorems (and problems) are more rudimentary in the propositional setting, and that a suciently sharp estimate to the complexity of the interpolation theorem for resolution - together with the known relations of propositional proof systems to bounded arithmetic theories - might yield an alternative proof of the main result of [43]. We prove such an interpolation theorem (in fact, a polynomial bound for resolution follows already from the bound for cut-free systems via a translation of resolution refutations into cutfree derivations, see 6.1(second proof)). In fact, we give a new proof of the Craig interpolation theorem (as well as of the Lyndon version) allowing us to deduce in a new way exponential lower bounds to the size of proofs in various systems (a subsystem of LK stronger than the cut-free fragment, resolution, a version of cutting planes). We formulate a general, syntax-free, framework for which our proof of the interpolation theorem yields good bounds. 3

4 The paper is organized as follows. In the rst section we dene several proof systems (sequent calculus, resolution, linear equational calculus and cutting planes). In the second section we recall some communication complexity (Karchmer-Wigderson game) and we reformulate a bit the characterization of the circuit-size in terms of P LS-problems from [43]. In the third section we give a new proof of Craig interpolation theorem for cut-free sequent calculus. The proof applies in a general, syntax-free, context. This is formalized by the notion of semantic derivations dened in section 4. A general form of the interpolation theorem for semantic derivations is proved in section 5. In section 6 we deduce from it polynomial upper bounds for interpolation for resolution, a subsystem of sequent calculus relevant to bounded arithmetic, linear equational calculus and a variant of cutting planes. In section 7 we obtain new proofs of exponential lower bounds for some of these systems and in section 8 we give an alternative treatment of the proof of the main independence result of [43]. A question for which proof systems one can prove a non-trivial lower bound for interpolation is discussed in section 9. It is linked with two topics, the existence of optimal propositional proof systems relative to a given theory (studied earlier in [22]) and the question of implicit denability of inverse functions to one-way functions. We also prove there that the depth 2 subsystem of LK does not admit feasible monotone interpolation theorem and that the validity of such a theorem for the depth 1 subsystem would imply new exponential lower bounds to the resolution proofs of the weak pigeonhole principle. The reader is assumed to have some familiarity with the subjects involved, in particular with some basic notions of complexity theory. A familiarity with bounded arithmetic is assumed only in the last two sections. References to original papers are often accompanied by a reference to a place in [20] which oers a survey of basic results in the eld. A remark on notation: we denote n-tuples of numbers or bits simply a; b; x; y; : : : rather than a; : : :, and the elements or the bits of a are denoted a 1 ; a 2 ; : : :. Logarithm log is base 2. 1 Propositional proof systems The propositional language of the sequent calculus LK contains the following connectives: constants 0 (false) and 1 (true), the negation :, the conjunction V and the disjunction W. The negation is allowed only in front of atoms, the conjunction and the disjunction are of unbounded arity. The symbol :A denotes the formula obtained from the formula A by interchanging 0 and 1, W and V and p i and :p i. The size jaj of A is the number of occurrences of connectives and atoms in it. The depth dp(a) of A is the maximal nesting of W and V in A: 4

5 dp(0) = dp(1) = dp(p i ) = dp(:p i ) = 0 dp( W i A i) = dp( V i A i) = 1 + max i (dp(a i )). We shall adopt the following version of the sequent calculus LK. The particular modication is unimportant and used just for technical reasons, similarly as in [19]. We shall keep the name LK as well. Further information on LK can be found in [45] or [20, Section 4.3] (contains also information about resolution, Section 4.2, and cutting planes, Section 13.1). A cedent is is a nite (possibly empty) sequence of formulas denoted?; ; : : :. The basic object of LK is a sequent, an ordered pair of cedents written??!. A sequent is satised if at least one formula in is satised or at least one formula in? is falsied. In particular, the empty sequent cannot be satised. The inference rules are the following: 1. the initial sequents are: 2. the weak structural rules are:?! 1 :1?! 0?!?! :0 p?! p :p?! :p p; :p?!?! p; :p the exchange:??!? 0?! 0 where? 0 ; 0 are any permutations of?; the contraction:??!? 0?! 0 where? 0 ; 0 are obtained from?; by deleting any multiple occurrences of formulas the weakening: where? 0? and 0 3. the propositional rules are: V :introduction??!? 0?! 0 A;??! V Ai ;??!??! ; A 1 : : :??! ; A m??! ; V im A i where A is one of A i in the left rule. 5

6 W :introduction 4. the cut rule: A 1 ;??! : : : A m ;??! W im A i;??! where A is one of A i in the right rule.??! ; A A;??!??!??! ; A??! ; W i A i An LK-proof of a sequent S from the sequents S 1 ; : : :; S m is a sequence Z 1 ; : : :; Z k such that Z k = S and each Z i is either an initial one or from fs 1 ; : : :; S m g, or derived from the previous ones by an inference rule. A proof-graph of an LK-proof is a directed acyclic graph whose nodes are the sequents of and a directed edge goes from a hypothesis of a rule to its conclusion. Hence the initial sequents correspond to the leaves. A proof is tree-like if its proof-graph is a forest, i.e., if every sequent is a hypothesis of at most one inference. k() is the number of sequents in. The size of a proof is the sum of the sizes of the formulas in it (counting multiple occurrences of a formula separately). A resolution refutation of sequents S 1 ; : : :; S m which contain no W ; V is an LK-proof of the empty sequent from S 1 ; : : :; S m in which no W ; V occur. This is obviously (essentially) equivalent to the more usual denition of resolution with clauses and the resolution rule as a resolution clause can be represented by the sequent f:p i1 ; : : :; :p ia ; p j1 ; : : :; p jb g p i1 ; : : :; p ia?! p j1 ; : : :; p jb and the resolution rule by the cut rule (and vice versa). We shall freely slide between the two denitions of resolution. We dene a linear equational calculus (LEC) to be a proof system working with linear equations a 1 x 1 + : : : + a n x n = b over a eld F. F is either a nite eld or the eld of rationals Q. The rules allow to add two equations and to multiply an equation by an element of F. An LEC-refutation of equations E 1 ; : : :; E m is an LEC-derivation of the equation 0 = 1 from E 1 ; : : :; E m. The size of an equation is P i ka ik + kbk where k u v k is the sum of the absolute values of u and v if F = Q and kak = 1 for all a 2 F if F is nite. LEC is sound and complete (by Gauss elimination), if by completeness we mean that every system of equations unsolvable in F is refutable. When completeness is considered only w.r.t. the systems with no 0-1 solution then LEC 6

7 is complete only for the two-element eld F 2. To get such completeness also for other elds one would have to expand LEC to an equational logic working with general polynomials and based on ring axioms. However, not even all Boolean functions can be represented by a conjunction of linear equations and so LEC cannot be considered, even for F 2, as a full propositional proof system in the sense of [9]. An important example of a formula which can be so represented is the negation of the pigeonhole principle, formalizing that there is a bijection between f1; : : :; n + 1g and f1; : : :; ng. This formula is represented by the following set of equations (over any F ) with variables x ij, i = 1; : : :n + 1 and j = 1; : : :; n: X j X i x ij = 1 ; for all i x ij = 1 ; for all j : It is easy to see that there is an LEC-refutation of this set of size polynomial in n. A system stronger than the resolution system is the cutting planes proof system introduced in [10]. This system CP works with inequalities of the form a 1 x 1 +: : :a n x n b, where a i ; b 2 Z and x i represent truth values of atoms. CP has few obvious rules: adding two inequalities, multiplying an inequality by a positive constant, the division rule: a 1 x 1 + : : :a n x n b a 1 c x 1 + : : : an c x n d c be provided cja i, all i, and few initial inequalities: x 0,?x?1. CP is a refutation system which derives from an unsatisable system of inequalities the inequality 0 1. The term unsatisable means that the system has no 0-1 solution. It is sound and complete and polynomially simulates resolution, see [10] or [20, 13.1]. 2 Protocols for Karchmer-Wigderson game Karchmer-Wigderson game (see [16]) is played as follows. Let U; V f0; 1g n be two disjoint sets. The game is played by two players A and B. Player A receives u 2 U while B receives v 2 V. They communicate bits of information (following a protocol previously agreed on) until both players agree on the same i 2 f1; : : :; ng such that u i 6= v i. Their objective is to minimize (over all protocols) the number of bits they need to communicate in the worst case. This minimum is called the communication complexity of the game and it is denoted by C(U; V ). 7

8 We say that the Boolean function B(p 1 ; : : :; p n ) separates U from V if and only if B(x) = 1 holds (resp. = 0) for all x 2 U (resp. for all x 2 V ). The following is a rather simple but quite important result. Theorem 2.1 ([16]) Let U; V f0; 1g n be two disjoint sets. Then C(U; V ) is precisely the minimal depth of a formula with binary _; ^ separating U from V. We shall need a bit ner version of the theorem. For that we need to dene the notion of a protocol in a particular way. Denition 2.2 Let U; V f0; 1g n be two disjoint sets. A protocol for the game on the pair (U; V ) is a labelled directed graph G satisfying the following four conditions: 1. G is acyclic and has one source (the in-degree 0 node) denoted ;. The nodes with the out-degree 0 are leaves, all other are inner nodes. 2. All leaves are labelled by one of the following formulas: for some i = 1; : : :; n. u i = 1 ^ v i = 0 or u i = 0 ^ v i = 1 3. There is a function S(u; v; x) (the strategy) such that S assigns to a node x and a pair u 2 U and v 2 V the edge S(u; v; x) leaving from the node x. Every pair u 2 U and v 2 V denes for every node x a directed path P x uv in G from the node x to a leaf: P x uv = x 1 ; : : :; x h, where x 1 = x, the edge S(u; v; x i ) goes from x i to x i+1, and x h is a leaf. 4. For every u 2 U and v 2 V there is a set F (u; v) G satisfying: (a) ; 2 F (u; v) (b) x 2 F (u; v)! P x u;v F (u; v) (c) the label of any leaf from F (u; v) is valid for u; v. Such a set F is called the consistency condition. A protocol is called monotone i every leaf in it is labelled by one of the formulas u i = 1 ^ v i = 0, i = 1; : : :; n. The communication complexity of G is the minimal number t such that for every x 2 G the players (one knowing u and x, the other one v and x) decide whether x 2 F (u; v) and compute S(u; v; x) with at most t bits exchanged in the worst case. 8

9 This denition is a variant of the formulation from [43] using P LS-problems. We would like to replace the consistency condition 4. by a simpler one: for all u; v the label of the leaf in P ; u;v is valid for u; v. However, the example of a linear size branching program nding i n such that u i 6= v i for all dierent u; v shows that that is not enough. Important examples of protocols are protocols formed from a circuit (later we shall dene protocols from proofs). Assume that C is a circuit separating U from V. Reverse the edges in C, take for F (u; v) those subcircuits diering in the value on u and v, and dene the strategy and the labels of the leaves in an obvious way. This determines a protocol for the game on (U; V ) whose communication complexity is 2. The next theorem says that there is a similar converse construction. Theorem 2.3 ([43]) Let U; V f0; 1g n be two disjoint sets. Let G be a protocol for the game on U; V which has k nodes and the communication complexity t. Then there is a circuit C of size k2 O(t) separating U from V. Moreover, if G is monotone so is C. On the other hand, any circuit (monotone circuit) C of size m separating U from V determines a protocol (a monotone protocol) G with m nodes whose communication complexity is 2. Proof : Let G be a protocol from the game. The number of nodes reachable from x via the edges denes a cost of x. For any u; v, the set F (u; v) together with the cost function and with the neighborhood function given by the strategy is a P LS-problem. By [43, Thm. 3.1] there is a circuit separating U from V of size at most j [ u;v F (u; v) j 2 O(t) = k 2 O(t) : If the protocol is monotone so is the circuit. The second part of the statement was noted above. q.e.d. 3 The Craig interpolation theorem We dene an interpolant of a valid implication A(p; q)! B(p; r) 9

10 where p = (p 1 ; : : :; p n ) are the atoms occurring in both A and B, while q = (q 1 ; : : :; q s ) occur only in A and r = (r 1 ; : : :; r t ) only in B, to be any Boolean function I(p) such that both implications A(p; q)! (I(p) = 1) and (I(p) = 1)! B(p; r) are tautologically valid. If I(p) is dened by a formula (also denoted I) this means that both implications A! I and I! B are tautologies. In the calculus LK the implication A! B is represented by the sequent A?! B and, in general, the sequent A 1 ; : : :; A m?! B 1 ; : : :; B` represents the implication V i A i! W j B j. Craig [11, 12] proved that every tautologically valid implication has an interpolant. In fact, the argument via the cut-elimination (see [45] or [20, 4.3]) gives the following theorem (with the bound k() instead of k() O(1), in fact). We shall give a new proof of the theorem which will later allow some generalizations not oered by Craig's original proof. For completeness we recall the standard proof as well (the second proof below). Theorem 3.1 ([11, 12]) Let be a cut-free LK-proof of the sequent: A 1 (p; q); : : :; A m (p; q)?! B 1 (p; r); : : :; B`(p; r) with p = (p 1 ; : : :; p n ) the atoms occurring simultaneously in some A i and B j, and q = (q 1 ; : : :; q s ) and r = (r 1 ; : : :; r t ) all other atoms occurring only in some A i or in some B j respectively. Then there is an interpolant I(p) of the implication: ^ im A i?! _ j` whose circuit-size is at most k() O(1). Moreover, if the atoms p occur only positively in all A i or in all B j then there is a monotone interpolant whose monotone circuit-size is at most k() O(1). B` First proof: Dene two sets U; V f0; 1g n by: U = fu 2 f0; 1g n j 9q u 2 f0; 1g s ; ^ im A i (u; q u )g ^ V = fv 2 f0; 1g n j 9r v 2 f0; 1g t ; :B(v; r v )g : j` 10

11 Note that the fact that the sequent A 1 ; : : :; A m?! B 1 ; : : :; B` is tautologically valid is equivalent to the fact that the sets U, V are disjoint, and that any Boolean function separates U from V i it is an interpolant of the sequent. Using the proof we dene a particular protocol for the game on U; V. Assume that player A received u 2 U and B received v 2 V. Player A xes some q u 2 f0; 1g s such V V that im A i(u; q u ) holds and player B xes some r v 2 f0; 1g t for which j` :B j(v; r v ) holds. Exchanging some bits they will construct the path P = S 0 ; : : :; S h of sequents of satisfying the following conditions: 1. S 0 in the end-sequent. 2. S i+1 is an upper sequent of the inference giving S i. 3. S h is an initial sequent. 4. For any a = 0; : : :; h: if S a has the form: E 1 (p; q); : : :; E e (p; q)?! F 1 (p; r); : : :; F f (p; r) then V ie E i(u; q u ) holds while W jf F j(v; r v ) fails. Note that as the proof is cut-free and there are no :-rules, no formula in the antecedent (resp. the succedent) of a sequent in the proof contains an atom r i (resp. an atom q i ). To nd S a+1 they proceed as follows: (a) If S a was deduced by an inference with only one hypothesis, they put S a+1 to be that hypothesis and they exchange no bits. (b) If the inference yielding S a was the introduction of V ig D i to the succedent the player B, who thinks that V ig D i is false, sends to A dlog(g)e bits identifying one particular D i (v; r v ), i g, which is false. They take for S a+1 the upper sequent of the inference containing the minor formula D i. (c) If the inference yielding S a was the introduction of W ig D i to the antecedent then analogously with (b) player A identies to B a true D i (u; q u ) and they take for S a+1 the sequent containing that minor formula. Let S h be the initial sequent the players arrive at in the path P. It must be one of the following: p i! p i or :p i! :p i for some i = 1; : : :; n. This is because all other the initial sequents either contain an atom r i in the antecedent or an atom q i in the succedent, or violate the condition 4. from the denition of P. 11

12 If S h is the former then by 4. u i = 1 ^ v i = 0, if it is the latter then u i = 0 ^ v i = 1. Formally, the protocol G is dened as follows: the nodes are the sequents of, the strategy is given by the denition of S a+1 from S a and the sequent is in the consistency condition F (u; v) i it satises the condition 4. above. The communication complexity of G is dlog(g)e + 2 dlog(k())e + 2. By Theorem 2.3 there is a circuit of size k() O(1) separating U from V. Note that if the atoms p i occur only positively in the antecedent or in the succedent of the end-sequent then the players always arrive to an initial sequent of the form p i?! p i. This yields the monotone case. This concludes the rst proof. Second proof: Let S be a sequent in the cut-free proof. By induction on the number of inferences before S we dene explicitly the interpolant I S (p) for S (this makes sense as no r i occurs in an antecedent and no q i occurs in a succedent). If S is initial then I S is one of 0; 1; p i ; :p i (because the only initial sequents in where q i or r i occur are q i ; :q i?! and?! r i ; :r i which have interpolants 0 and 1 respectively). If S was derived from one hypothesis S 1 put I S := I (S1). If S was derived by the right V :introduction (resp. by the left W :introduction) from S 1 ; : : :; S g then put I S := V ig I(Si) (resp. W ig I(Si) ). The monotone case follows as then :p i cannot occur as the interpolant of an initial sequent. This concludes the second proof. q.e.d. Both proofs of Theorem 3.1 can be modied for the case when is not necessarily cut-free but no cut-formula contains atoms q and r at the same time. To maintain the condition that q (resp. r) do not occur in the succedent (resp. in the antecedent) we picture a cut-inference with the cut-formula D as :D;??! D;??!??! or??! ; D??! ; :D??! according to whether atoms q do or do not occur in D (this is equivalent to considering a partition of formulas in a sequent S into the ancestors of the formulas from the antecedent and from the succedent of the end-sequent, rather than just into the antecedent and the succedent of S itself). A modication of the rst proof is then straightforward as the truth-value of any cut-formula is known to one of the players and he can direct the path by 12

13 sending one bit. To modify the second proof note that if I 1 and I 2 are interpolants of the hypotheses of a cut-inference as above then I 1 _ I 2, respectively I 1 ^ I 2, is an interpolant of the lower sequent. Corollary 3.2 Let be an LK-proof of the sequent: A 1 (p; q); : : :; A m (p; q)?! B 1 (p; r); : : :; B`(p; r) with occurrences of atoms as shown. Assume that in no cut-formula some q i and r j occur simultaneously. Then there is an interpolant I(p) of the implication: ^ im A i?! _ j` whose circuit-size is at most k() O(1). Moreover, if the atoms p occur only positively in all A i or in all B j then there is a monotone interpolant whose monotone circuit-size is at most k() O(1). A third way how to obtain the corollary from Theorem 3.1 is provided by the second proof of Theorem Semantic derivations In the rst proof of Theorem 3.1 we have not used any particular syntactic property of formulas in an LK-proof. Rather the proof worked with the sets of truth assignments satisfying the formulas in. The following notion is intended to formalize a general situation in which a similar argument may work for proofs allowing some form of cut-rule. Denition 4.1 Let N be a xed natural number. 1. The semantic rule allows to infer from two subsets A; B f0; 1g N a third one: A B C i C A \ B. 2. A semantic derivation of the set C f0; 1g N from the sets A 1 ; : : :; A m f0; 1g N is a sequence of sets B 1 ; : : :; B k f0; 1g N such that B k = C, each B i is either one of A j or derived from two previous B i1 ; B i2 by the semantic rule. 3. Let X exp(f0; 1g N ) be a family of subsets of f0; 1g N. A semantic derivation B 1 ; : : :; B k is an X -derivation i all B i 2 X. B` 13

14 Recall that a lter of subsets of f0; 1g N is a family X closed upwards (A 2 X ^ B A! B 2 X ) and closed under intersection (A; B 2 X! A \ B 2 X ). The following lemma is obvious. Lemma 4.2 Let A 1 ; : : :; A m ; C 2 f0; 1g N. Then the following three conditions are equivalent: 1. C can be semantically derived from A 1 ; : : :; A m. 2. C can be semantically derived from A 1 ; : : :; A m in m? 1 steps. 3. C is in the smallest lter containing A 1 ; : : :; A m. It means that to have a non-trivial meaning of the length of semantic derivations we must restrict to X -derivations for some family X which itself is not a lter. For example, a family formed by subsets of f0; 1g N denable by disjunctions of literals yields a non-trivial notion. The following technical denition abstracts a property of sets of truth assignments used in the rst proof of Theorem 3.1. Denition 4.3 Let N = n + s + t be xed and let A f0; 1g N. Let u; v 2 f0; 1g n, q u 2 f0; 1g s and r v 2 f0; 1g t. Consider three tasks: 1. Decide whether (u; q u ; r v ) 2 A. 2. Decide whether (v; q u ; r v ) 2 A. 3. If (u; q u ; r v ) 2 A 6 (v; q u ; r v ) 2 A nd i n such that u i 6= v i. These tasks can be solved by two players, one knowing u; q u and the other one knowing v; r v. The communication complexity of A, CC(A), is the minimal number of bits they need to exchange in the worst case in solving any of these three tasks. Consider two more tasks: 4. If (u; q u ; r v ) 2 A and (v; q u ; r v ) =2 A either nd i n such that u i = 1 ^ v i = 0 or learn that there is some u 0 satisfying (u u 0 means V in u i u 0 i.) u 0 u ^ (u 0 ; q u ; r v ) =2 A 14

15 5. If (u; q u ; r v ) =2 A and (v; q u ; r v ) 2 A either nd i n such that u i = 1 ^ v i = 0 or learn that there is some v 0 satisfying v 0 v ^ (v 0 ; q u ; r v ) =2 A : (Note that the players are not required to nd u 0 and v 0 in 4. and 5. and that the two cases in each task are not necessarily exclusive.) The monotone communication complexity w.r.t. U of A, MCC U (A), is the minimal t CC(A) such that the task 4. can be solved communicating t bits in the worst case. The monotone communication complexity w.r.t. V of A, MCC V (A), is the minimal t CC(A) such that the task 5. can be solved communicating t bits in the worst case. In the example above, any set denable by a disjunction of literals has both the communication complexity and the monotone communication complexity at most dlog(n)e + 2. Note that proofs in any of the usual propositional calculi translate into semantic derivations: simply replace a sequent (a formula, an equation, etc.) by the set of its satisfying truth assignments. The inference rules translate into instances of the semantic rule as they are all sound. Let N = n + s + t be xed for the whole section. For A f0; 1g n+s dene the set A ~ by: [ ~A := f(a; b; c) j c 2 f0; 1g t g (a;b)2a where a; b; c range over f0; 1g n, f0; 1g s and f0; 1g t respectively, and similarly for B f0; 1g n+t dene B: ~ [ ~B := f(a; b; c) j b 2 f0; 1g s g : (a;c)2b 5 An interpolation theorem for semantic derivations Theorem 5.1 Let A 1 ; : : :; A m f0; 1g n+s and B 1 ; : : :; B` f0; 1g n+t. Assume that there is a semantic derivation = D 1 ; : : :; D k of the empty set ; = D k from the sets A1 ~ ; : : :; Am ~ ; B1 ~ ; : : :; B` ~ such that: CC(D i ) t 15

16 for all i k. Then the two sets U = fu 2 f0; 1g n j 9q u 2 f0; 1g s ; (u; q u ) 2 \ jm A j g and V = fv 2 f0; 1g n j 9r v 2 f0; 1g t ; (v; r v ) 2 \ j` B j g can be separated by a circuit of size at most (k + 2n)2 O(t). Moreover, if the sets A 1 ; : : :; A m satisfy the following monotonicity condition w.r.t. U: \ \ (u; q u ) 2 A j ^ u u 0! (u 0 ; q u ) 2 jm jm and MCC U (D i ) t for all \ i k, or if the sets B 1 ; : : :; B` \ satisfy: (v; r v ) 2 B j ^ v v 0! (v 0 ; r v ) 2 B j j` j` A j and MCC V (D i ) t for all i k, then there is a monotone circuit separating U from V of size at most (k + n)2 O(t). Proof : Let = D 1 ; : : :; D k be a semantic derivation of ; from ~ A 1 ; : : :; ~ B`. We shall construct a protocol G for the Karchmer-Wigderson game on U; V. Before we dene it formally we rst explain its idea in terms of two players constructing a path through. The two players A and B, one knowing (u; q u ) 2 T j A j and the other one knowing (v; r v ) 2 T j B j, attempt to construct a path P = S 0 ; : : :; S h through. S 0 = ; = D k, S a+1 is one of the two sets which are the hypotheses of the semantic inference yielding S a and S h 2 f ~ A 1 ; : : :; ~ B`g. Moreover, both tuples (u; q u ; r v ) and (v; q u ; r v ) are not in S a, a = 0; : : :; h. If the players know S a which was deduced in the inference: X S a then they rst determine whether (u; q u ; r v ) 2 X and (v; q u ; r v ) 2 X. There are three possible outcomes: 1. both (u; q u ; r v ) and (v; q u ; r v ) are in X 2. none of (u; q u ; r v ); (v; q u ; r v ) is in X 3. only one of (u; q u ; r v ); (v; q u ; r v ) is in X. Y 16

17 In the rst case none of the two tuples can be in Y and the players put S a+1 := Y. In the second case they take S a+1 := X. In the third case they stop constructing the path and enter a protocol aimed at nding i n such that u i 6= v i. Such i must exists as necessarily u 6= v. As none of the initial sets ~A 1 ; : : :; ~ B` avoids both (u; q u ; r v ); (v; q u ; r v ) the players must sooner or later enter the possibility 3. and nd i n such that u i 6= v i. Now we dene the protocol G formally. G has (k + 2n) nodes, the k steps of the derivation plus 2n additional nodes labelled by formulas u i = 1 ^ v i = 0 and u i = 0 ^ v i = 1, i = 1; : : :; n. The consistency condition F (u; v) consists of those D j such that (v; q u ; r v ) =2 D j and of those of the additional 2n nodes whose label is valid for the particular pair u; v. The strategy function (for D j derived from X and Y ) is dened as follows: 1. If (u; q u ; r v ) =2 D j then X if (v; q S(u; v; D j ) := u ; r v ) =2 X Y if (v; q u ; r v ) 2 X (and hence (v; q u ; r v ) =2 Y ). 2. If (u; q u ; r v ) 2 D j then the players use the protocol (whose existence is guaranteed by the denition of CC(D j )) for nding i n such that u i 6= v i. S(u; v; D j ) is then the one of the two nodes labelled by u i = 1 ^ v i = 0 and u i = 0 ^ v i = 1 whose label is valid for the pair u; v. Note that both the strategy function S(u; v; x) and the membership relation x 2 F (u; v) can be determined by the players exchanging at most O(t) bits. As G has (k + 2n) nodes, Theorem 2.3 yields the wanted circuit separating U from V and having the size at most (k + 2n) 2 O(t). The protocol requires a modication for the monotone case. Assume that the sets A 1 ; : : :; A m satisfy the monotonicity condition w.r.t. U and that MCC U (D i ) t for all i k (the case of the monotonicity w.r.t. V is analogous). The protocol has (k + n) nodes, the k steps of the derivation plus n additional nodes labelled by formulas u i = 1 ^ v i = 0, i = 1; : : :; n. The consistency condition F (u; v) is dened as before. The strategy function is dened in a bit dierent way. The players use the protocol for solving the task 4. from Denition 4.3. There are two possible outcomes: 1. They decide that the condition: 9u 0 u; (u 0 ; q u ; r v ) =2 D j is true for u; v. Then they put: X if (v; q S(u; v; D j ) := u ; r v ) =2 X Y if (v; q u ; r v ) 2 X. 17

18 2. They nd i n such that u i = 1^v i = 0. S(u; v; D j ) is then the additional node with the label u i = 1 ^ v i = 0. By the monotonicity condition imposed on A 1 ; : : :; A m, for every u 0 occurring above it holds: \ (u 0 ; q u ; r v ) 2 A j : This implies that the players have to nd sooner or later i n such that u i = 1 ^ v i = 0. By the assumption about the monotone communication complexity of all D j, both the relation x 2 F (u; v) and the function S(u; v; x) can be computed exchanging O(t) bits. As G has (k + n) nodes, Theorem 2.3 yields the wanted monotone circuit separating U from V and having the size at most (k+n)2 O(t). jm q.e.d. 6 Upper bounds for some interpolation theorems In this section we derive from Theorem 5.1 feasible bounds for interpolation theorems for resolution, a subsystem of LK relevant to bounded arithmetic, and for LEC and CP. Theorem 6.1 Assume that the set of clauses fa 1 ; : : :; A m ; B 1 ; : : :; B`g where: 1. A i fp 1 ; :p 1 ; : : :; p n ; :p n ; q 1 ; :q 1 ; : : :; q s ; :q s g, all i m 2. B j fp 1 ; :p 1 ; : : :; p n ; :p n ; r 1 ; :r 1 ; : : :; r t ; :r t g, all j ` has a resolution refutation with k clauses. Then the implication: ^ A i )?! (^ :B j ) im( j` (where W A i denotes the disjunction of the literals in A i and V :B j denotes the conjunction of the negations of the literals in B j ) has an interpolant I(p) whose circuit-size is kn O(1). Moreover, if all atoms p occur only positively in all A i, or if all p occur only negatively in all B j, then there is a monotone interpolant whose monotone circuit-size is kn O(1). First proof: Let = C 1 ; : : :; C k be a resolution refutation of A 1 ; : : :; B`. For a clause C denote by ~ C the subset of f0; 1g n+s+t of all those truth assignments satisfying C. Then ~ = ~ C1 ; : : :; ~ Ck is a semantic derivation of ; from ~ A1 ; : : :; ~ B`. 18

19 Obviously, for any clause C both the communication complexity and the monotone communication complexity of ~ C is at most CC( ~ C) dlog(n)e + 2. Hence Theorem 5.1 yields circuit of size (k + 2n) n O(1) k n O(1) Similarly for the monotone case. This concludes the rst proof. Second proof: We give a second proof of a slightly worse bound via a translation of resolution refutation into cut-free proofs. Assume that C 1 ; : : :; C k is a resolution refutation of clauses A 1 ; : : :; B`. We show that for every a k there are cedents? a ; a such that the following conditions hold: 1. Each formula in? a has the form either p i _ :p i or q i _ :q i. 2. Each formula in a has the form r i ^ :r i. 3. The sequent? a ; _ A 1 ; : : :; _ A m?! ^ :B 1 ; : : :;^ :B`; a ; _ C a has a cut-free LK-proof with O(a N) sequents, where N = n + s + t. W This is readily V established by induction on a. For C a 2 fa 1 ; : : :; B`g apply : left or : right (O(N) sequents), otherwise replace the cut inference yielding C a with the cut formula p i or q i by introduction of p i _:p i or q i _:q i respectively into the antecedent, and the cut inference with the cut formula r i by introduction r i ^ :r i to the succedent. These new formulas form the cedents? a and a respectively. By Theorem 3.1 the implication: ^?k ^ ^ A i )?! (^ _ :B j ) _ k im( j` has an interpolant W V I(p) of circuit size (kn) O(1). Now note that, as? k is a tautology while k is unsatisable, I(p) is, in fact, an interpolant for the implication ^ A i )?! (^ :B j ) im( j` as well. This concludes the second proof. q.e.d. The following statement extends the previous theorem to a larger class of LK-proofs. This class appears naturally in connection with bounded arithmetic (see [19, 2.2] or [20, ]). 19

20 Corollary 6.2 Let be an LK-proof of the sequent: A 1 (p; q); : : :; A m (p; q)?! B 1 (p; r); : : :; B`(p; r) with atoms p; q; r occurring as displayed and such that the formulas A i (resp. B j ) are literals or disjunctions (resp. conjunctions) of literals. Assume that satises: 1. is tree-like. 2. Every formula in has the depth at most two. 3. Every sequent in contains at most c depth 2 formulas, where c is an independent constant. Then there is an interpolant I(p) of the implication: ^ im A i?! _ j` whose circuit-size is at most k() O(c) n O(1). Moreover, if the atoms p occur only positively in all A i or in all B j then there is a monotone interpolant whose monotone circuit-size is k() O(c) n O(1). Proof : Assume that satises the hypothesis of the corollary. It was shown in [19, 2.2] (or see [20, ]) that can be transformed into a tree-like proof 0 with k( 0 ) = k() O(c) in which every formula is of depth 1. Furthermore, by [19, 1.2] (or see [20, ]) such 0 can be transformed into a resolution refutation 00 (which is not necessarily tree-like) of clauses representing the sequents?! A 1 ; : : :;?! A m and :B 1?!; : : :; :B`?!, and such that k( 00 ) = k( 0 ) O(1) = k() O(c). The corollary then follows from Theorem 6.1. Next we deduce interpolation theorems for LEC and CP. B j q.e.d. Theorem 6.3 Let E 1 (x; y); : : :; E m (x; y) and F 1 (x; z); : : :; F`(x; z) be a system of linear equations over a nite eld F in which occur only the displayed variables x = (x 1 ; : : :; x n ), y = (y 1 ; : : :; y s ) and z = (z 1 ; : : :; z t ). Assume that there is an LEC-refutation of the system with k() inferences. Then there is an interpolant I(x) of the implication: ^ im E i (x; y)?! _ j` whose circuit-size is at most k()n O(1). :F j (x; z) 20

21 Proof : Put N := n + s + t. For an equation C(x; y; z) denote by ~ C the subset of f0; 1g N of those tuples satisfying C. If = C 1 ; : : :; C k is an LEC-refutation then ~ C 1 ; : : :; ~ C k is a semantic refutation of ~ E 1 ; : : :; ~ F`. For any linear equation C the communication complexity of ~ C is at most O(log(n)). The theorem then follows from Theorem 5.1. q.e.d. If F = Q we do not get such an estimate on CC( ~ C) valid for all C. Rather we need also to incorporate the sizes of the coecients (see the denition of kak in section 1). Instead of this we prove an interpolation theorem for CP ; the case of LEC with F = Q is similar. Theorem 6.4 Let E 1 (x; y); : : :; E m (x; y); F 1 (x; z); : : :; F`(x; z) be a system of CP -inequalities in which occur only the displayed variables x = (x 1 ; : : :; x n ), y = (y 1 ; : : :; y s ) and z = (z 1 ; : : :; z t ). Let N := n + s + t. Assume that there is a CP -refutation of the system such that: 1. contains k() inferences. 2. Every coecient occurring in has the absolute value at most M. Then there is an interpolant I(x) of the implication: ^ im E i (x; y)?! _ j` :F j (x; z) whose circuit-size is at most k()(mn) O(1) (Mn) O(log n). Moreover, if all x i occur in all E 1 ; : : :; E m with non-negative coecients only, or if all x i occur in all F 1 ; : : :; F` only with non-positive coecients only, then there is a monotone interpolant whose monotone circuit-size is at most k()(mn) O(1) (Mn) O(log n). Proof : Assume that is a CP -refutation satisfying the hypothesis of the corollary. Every inequality D in has the form: a x + b y + c z d where a x abbreviates the scalar product a 1 x 1 + : : : + a n x n (and similarly b y and c z). Let ~ D be the set of assignments satisfying D. Assume that player A received u 2 f0; 1g n such that all E i (u; y u ) are satised for some y u, while B received v 2 f0; 1g n such that all F j (v; z v ) are satised for some z v. As in the proof of Theorem 6.1 it is sucient to estimate the (monotone) communication complexity of ~ D. 21

22 For the tasks to decide whether (u; y u ; z v ) 2 ~ D and (v; y u ; z v ) 2 ~ D it is sucient if A sends to B the numbers a u and b y u, and B sends to A a v and c z v. This needs at most 2 dlog(mn)e bits each. If (u; y u ; z v ) 2 ~ D 6 (v; y u ; z v ) 2 ~ D then necessarily a u 6= a v and the players nd i n such that u i 6= v i by binary search. Here dlog ne dlog(mn)e bits suce. This shows that for any ~ D the communication complexity is at most: CC( ~ D) O(log(MN) + log n log(mn)) : Theorem 5.1 implies that the implication has an interpolant of circuit-size at most: k() (MN) O(1) (Mn) O(log n) : For the monotone case assume that the variables x i occur only with non-negative coecients in all E 1 ; : : :; E m (the other case is analogous). The monotonicity condition of Theorem 5.1 ^ ^ u 0 u ^ E i (u; y u )?! E i (u 0 ; y u ) im is then satised. It is thus sucient to estimate MCC U ( ~ D). Assume (u; y u ; z v ) 2 ~ D while (v; y u ; z v ) =2 ~ D. Then a u > a v. Write the vector a as a dierence of two vectors with non-negative coecients, a = a 1?a 2. There are two possibilities 1. a 1 u > a 1 v. 2. a 1 u a 1 v ^ a 2 u < a 2 v. In the rst case the players use binary search to nd i n such that u i = 1 ^ v i = 0. This needs at most dlog ne dlog(mn)e bits. In the second case they know that for some u 0 u it holds a 2 u 0 a 2 v and hence also a u 0 a v and (u 0 ; y u ; z v ) =2 ~ D. To decide which case applies needs at most 2dlog(Mn)e bits. Hence im MCC U ( ~ D) O(log(MN) + log n log(mn)) : Theorem 5.1 yields the existence of a monotone interpolant with the monotone circuit-size at most k() (MN) O(1) (Mn) O(log n) : q.e.d. 22

23 7 Lower bounds for proof systems Assume that for a propositional proof system P we have a good interpolation theorem allowing, in particular, good estimates of the complexity of the monotone interpolants. Then an implication which cannot have a small monotone interpolant must have long P -proofs. A similar idea of lower bounds for proof systems was discussed in the context of counting principles in [19, Sect.5]. Non-trivial lower bounds to the circuit size are known for monotone circuits separating graphs with large cliques from those colorable by a small number of colors, see [39, 2, 1]. It is thus natural to use the implications determined by these two N P-sets as explained in the introduction. Similar implications were discussed in [43].? n Denition 7.1 Let n;!; 1 be natural numbers, and let 2 denote the set of two-element subsets of f1; : : :; ng. The set? Clique n;! (p; q) is a set of the n following formulas in the atoms p ij, fi; jg 2 2, and qui, u = 1; : : :;! and i = 1; : : :; n: (1a) W in q ui, all u!, (1b) :q ui _ :q vi, all u < v! and i = 1; : : :; n,? n (1c) :q ui _ :q vj _ p ij, all u < v! and fi; jg 2 2. The set? Color n; (p; r) is the set of the following formulas in the atoms p ij, n fi; jg 2 2 W, and ria, i = 1; : : :; n and a = 1; : : :; : (2a) a r ia, all i n, (2b) :r ia _ :r ib, all a < b and i n,? n (2c) :r ia _ :r ja _ :p ij, all a and fi; jg 2 2. The expression Clique n;!?! :Color n; is an abbreviation of the sequent whose antecedent consists of all formulas in (1 a-c) and whose succedent consists of the negations of the formulas in (2 a-c). Truth assignments to p ij can be identied with graphs on n vertices. Truth assignments to q ui such that Clique n;! (p; q) is satised can be identied with 1- to-1 maps from the set f1; : : :;!g onto a clique in graph p, and truth assignments to r ia such that Color n; (p; r) is satised can be identied with colorings of graph p by colors f1; : : :; g. Thus the set fp j 9q Clique n;! (p; q)g is the set of graphs with a clique of size!, while the set fp j 9r Color n; (p; r)g 23

24 is the set of graphs colorable by colors. Hence the sequent Clique n;!?! :Color n; is tautologically valid i <!. The following theorem just restates the bound from [1], replacing the class of graphs without a clique of size used in [1] by the smaller class of -colorable graphs (the bound to monotone circuits separating these two classes is what is actually proved in [1]). Theorem 7.2 ([1]) Assume that 3 <! and p! sequent Clique n;!?! :Color n; has no interpolant of the monotone circuit-size smaller than: 2 ( p ) : n 8 log n. Then the For the next statement note that all formulas in the set Clique n;! [Color n; are disjunctions of literals and thus can be identied with resolution clauses. A resolution clause fx i1 ; : : :; x ia ; :x j1 ; : : :; :x jb g can be represented by a CP - inequality x i1 + : : : + x ia? x j1 : : :? x jb 1? b : Hence the set Clique n;! [ Color n; can be considered also as a set of CP - inequalities in p; q; r. Corollary 7.3 Let n be suciently large and let = d p n e,! = + 1. Then: 1. Every resolution refutation of the clauses Clique n;! [ Color n; must have at least 2 (n 1 4 ) clauses. 2. Every CP -refutation of the clauses Clique n;! [ Color n; with all coecients in the absolute value M must have at least inequalities. 2 (n 4 1 ) M O(log n) In particular, if M 2 n then for a suciently small constant the number of inequalities is at least 2 n(1). Proof : By Theorem 6.1 a resolution refutation with k clauses would imply the existence of an interpolant with monotone circuit size kn O(1). The hypothesis of Theorem 7.2 is fullled and so it must hold: kn O(1) 2 (n 1 4 ) 24

25 and hence as well. k 2 (n 1 4 ) The second part is proved analogously using Theorem 6.4 in place of 6.1. By 6.4 and 7.2: k()(mn) O(1) (Mn) O(log n) 2 (n 1 4 )? n where N = 2 + n(s + t) = O(n2 ). This implies: and so k()m O(log n) n O(log n) 2 (n 1 4 ) k M 1 2(n 4 ) : O(log n) For M 2 n, suitably small, the right-hand side is 2 n(1). q.e.d. Note that by a suitable choice of we can get a lower bound of the form 2 (n 1 3? ), for arbitrary small > 0. 8 An independence result for the bounded arithmetic theory S 2 2 () The rst bounded arithmetic theory was introduced in [34]. Current research is centered around the theories dened in [5]. In this section we give a new presentation of the proof of the independence result for the theory S 2 2() obtained in [43]. For the denition of the theory as well as for the details of bounded arithmetic the reader should consult [5] or [20, Chpt. 5] (in particular, the language L() of S 2 2() contains, in fact, countably many unary predicates i ). In the latter can also be found details of various relations between the arithmetic systems and the propositional proof systems (in [20, Chpt. 9] in particular). We shall recall briey a translation of bounded L() formulas; [35] used it rst in a connection with bounded arithmetic. A bounded formula A(a; 1 ; : : :; k ) with the predicate parameters i and the number parameter a can be for every value a := N translated into a (log N)O(1) constant-depth, size 2 formula: the atomic sentence j 2 i translates into the atom p i j, a true (resp. false) rst-order atomic sentence translates into 1 (resp. into 0) and a bounded universal (resp. existential) quantier 8x < tb(x) resp. 9x < tb(x) translates into a conjunction (resp. a disjunction) of the 25

26 translations of B(x); x = 0; : : :; t? 1. We shall denote the translation of formula A for a = N by hai N (p 1 ; : : :; p k ). There are rather sophisticated relations between bounded arithmetic theories and propositional proof systems, see [8, 35, 22, 23, 26, 27, 19, 21] or [20, Chpts.9 and 11-15]. The class of rst-order bounded formulas in the language of bounded arithmetic (no predicates i ) is denoted b 1. We call a bounded L()-formula E 1 (; b 1) if it has the form 9 A, where A is a disjunction of conjunctions of atomic formulas and b 1-formulas. U 1 (; b 1)-formulas are dened dually, replacing 9 by 8 and a disjunction of conjunctions by a conjunction of disjunctions. The following theorem is known (see, for example, the simulation as proved in [21] or [20, Chpt. 9]). Theorem 8.1 Assume that 8x s(a)a(a; x; 1 ; : : :; k ) is a bounded U 1 ( 1 ; : : :; k ; b 1 )-formula and that 9y t(a)b(a; y; 1; : : :; k ) is a bounded E 1 ( 1 ; : : :; k ; b 1)-formula. Assume that the theory S 2 2() proves the sequent: 8x s(a) A(a; x; 1 ; : : :; k )?! 9y t(a) B(a; y; 1 ; : : :; k ) : Then for every N the propositional sequent: hai N;0 ; : : :; hai N;s(N)?! hbi N;0 ; : : :; hbi N;t(N) (where the formulas are built from atoms p 1 ; : : :; p k ) has an LK-proof N satisfying the following conditions: 1. is tree-like. 2. k( N ) = 2 (log N)O(1). 3. Every formula in N has depth at most Every sequent in N contains at most c depth 2 formulas (c an independent constant). We turn now our attention to the provability of circuit-size lower bounds in bounded arithmetic. Razborov [42, 43] studies a formalization of Boolean complexity methods in the bounded arithmetic theory V1 1 and in its fragments. In that formalization Boolean functions and circuits are coded by sets while Boolean inputs are coded by numbers. This allows to speak directly about exponential size circuits. In [42] he demonstrated that all major lower bounds to the circuit-size of restricted circuit models known at present can be also proved in V 1 1. On the other hand, in [43] he showed - under a cryptographic assumption about the existence of strong 26