Descriptive Complexity of Linear Equation Systems and Applications to Propositional Proof Complexity

Descriptive Complexity of Linear Equation Systems an Applications to Propositional Proof Complexity Martin Grohe RWTH Aachen University grohe@informatik.rwth-aachen.e Wie Pakusa University of Oxfor wie.pakusa@cs.ox.ac.uk Abstract We prove that the solvability of systems of linear equations an relate linear algebraic properties are efinable in a fragment of fixe-point logic with counting that only allows polylogarithmically many iterations of the fixe-point operators. This enables us to separate the escriptive complexity of solving linear equations from full fixe-point logic with counting by logical means. As an application of these results, we separate an extension of first-orer logic with a rank operator from fixe-point logic with counting, solving an open problem ue to Holm [21]. We then raw a connection from this work in escriptive complexity theory to graph isomorphism testing an propositional proof complexity. Answering an open question from [7], we separate the strength of certain algebraic graph-isomorphism tests. This result can also be phrase as a separation of the algebraic propositional proof systems Nullstellensatz an monomial PC. I. INTRODUCTION There are two quite ifferent perspectives on this work. The first is from escriptive complexity an finite moel theory: we prove that the solvability of systems of linear equations an relate linear algebraic properties are efinable in a fragment of fixe-point logic with counting (FPC) that only allows polylogarithmically many iterations of the fixepoint operators (POLYLOG-FPC). To place this result, it may be helpful to view FPC as an approximation of the complexity class PTIME an the fragment POLYLOG-FPC as the corresponing approximation of the complexity class NC consisting of all problems solvable in parallel polylogarithmic time. Our result allows us to separate the escriptive complexity of solving linear equations from full FPC by logical means. As an application of these results, we separate an extension of first-orer logic with a rank operator from fixe-point logic with counting, solving an open problem ue to Holm [21]. The secon perspective on our work is from graph isomorphism testing an propositional proof complexity. Answering an open question from [7], we separate the strength of certain algebraic graph-isomorphism tests. This result can also be phrase as a separation of the algebraic propositional proof systems Nullstellensatz an monomial PC. Maybe the key insight of our paper is that there is a connection between these two topics an that escriptive complexity can be use to answer the open problem on the The secon author was partially supporte by a DFG grant (PA 2962/1-1). 978-1-5090-3018-7/17/$31.00 2017 European Union graph isomorphism algorithms. In fact, this open problem was our starting point, an the paper is the result of solving it. In the following, we present both the escriptive complexity sie an the graph isomorphism/proof complexity sie in more etail, giving context an references. A. Descriptive Complexity of Solving Linear Equations Fixe-point logic with counting has playe a central role in escriptive complexity theory ever since Immerman [23] propose it as a caniate for a logic capturing polynomial time. Over the last ten years we have learne that the logic is significantly more expressive than previously thought, both in graph theoretic contexts [18], [20], [27] an, quite unexpectely, also in algebraic contexts [3], [12], [21]. A culmination of the work on efinability of algebraic properties is Anerson, Dawar, an Holm s [3] result that the solvability of linear programs is expressible in FPC. Of course this implies that the solvability of systems of linear equations is also expressible in FPC. This is not obvious, because typical algorithms like Gaussian elimination, choosing a pivot element in each step, cannot be escribe in a choiceless logic like FPC. An it shoul be note that this result only hols over the fiels of characteristic 0 an not over fiels of characteristic p 0, see [4], [13], [17]. We take a closer look at the logical resources require to express the solvability of linear equations. Our intuition was that since the computational complexity of this problem is in NC, it might be possible to express it in the fragment POLYLOG-FPC. This turne out to be true. The key step of the proof is to transform, by a logical interpretation, a given system of linear equations into an equivalent system where we can efine a linear orer on the variables an equations. We also prove a relate, quite powerful, technical result stating that simultaneous similarity of two sequences of efinable matrices can be expresse in counting logic with polylogarithmic quantifier epth. Very briefly, two matrix sequences are simultaneously similar if there is a single similarity transformation which pairwise relates corresponing matrices in the two sequences. We can now use an ol result ue to Immerman [22], [24] that separates the logics POLYLOG-FPC an FPC (actually, Immerman works with ifferent logics, but this result follows) to prove that certain logical systems base on solving systems of linear equations must be strictly weaker than FPC. As an

immeiate application, this shows that the extension of firstorer logic by a rank operator [13], which expresses the rank of matrices over the fiel of rationals, is strictly weaker than FPC, answering an open question from [21]. B. Algebraic Approaches to Graph Isomorphism In recent years, graph isomorphism algorithms base on generic mathematical programming an algebraic techniques have receive consierable attention (e.g. [5], [7], [8], [19], [30], [31]). The iea is to encoe a graph-isomorphism instance into a system of linear or polynomial equations or inequalities an then to solve this system by stanar (linear) algebraic means. The most basic of these approaches is to write, for two given graphs, an integer linear program (ILP) whose solutions correspon to the isomorphisms between the graphs an then stuy the LP-relaxation of this ILP. The solutions to this linear program are known as fractional isomorphisms. Note that solving the linear program gives us a soun, but not complete isomorphism test: if there is no solution, we know that the graphs are not isomorphic; if there is a solution we know nothing. It follows from a beautiful result ue to Tinhofer [36] via a corresponence between logic an the so-calle Weisfeiler-Leman algorithm ue to Immerman an Laner [25] that the linear program has a solution (that is, there is a fractional isomorphism between the two graphs) if, an only if, the graphs are inistinguishable in the 2-variable fragment of first-orer logic with counting. Atserias an Maneva [5] extene Tinhofer s result to a corresponence between the Sherali-Aams hierarchy of increasingly tighter LP-relaxations of the original ILP an the hierarchy of finitevariable fragments of first-orer logic with counting. In [7], Berkholz jointly with the first author of this paper, stuie an algebraic approach where an instance of the graph isomorphism problem is encoe by a system of linear an quaratic equations, which then can be solve by Gröbner basis techniques. The strength of this approach can best be stuie in the framework of propositional proof complexity, specifically for the algebraic proof systems polynomial calculus (PC) [11] an Hilbert s Nullstellensatz [6]. In [7], an intermeiate proof system between Nullstellensatz an PC, the monomial PC was introuce, an it was shown that this system precisely characterises the hierarchy of finite-variable fragments of first-orer logic with counting. Here the number of variables correspons to the egree of the polynomials use in a refutation in the proof system (proving that the original system of equations is not satisfiable). The question whether the monomial PC is stronger than Nullstellensatz was left open. Remarkably, this question is equivalent to the question whether the nonnegativity constraints on all variables in the linear programs of the Sherali-Aams hierarchy mentione above can be omitte. We answer this question by proving that monomial PC is strictly stronger. The proof is by reuctions to our escriptive-complexity theoretic results. Fining Nullstellensatz refutations amounts to solving systems of linear equations an hence can be expresse in finite-variable first-orer logic with counting an polylogarithmic quantifier epth. Monomial PC has the strength of full FPC when it comes to istinguishing graphs. Now we can use Immerman s result (POLYLOG-FPC < FPC) to separate the two. C. Outline We introuce the polylogarithmic restriction of fixe-point logic with counting (POLYLOG-FPC) in Section III. Formulas of POLYLOG-FPC can be translate into equivalent formulas of counting logic with polylogarithmic quantifier rank (Theorem III.1). An ol result ue to Immerman (Theorem III.1) yiels the separation of POLYLOG-FPC from full FPC. After we iscusse the encoing of linear-algebraic objects by means of finite relational structures in Section IV, we explore the power of POLYLOG-FPC with respect to linear-algebraic queries over the complex fiel in Section V. One of our main results is that POLYLOG-FPC can efine the solvability of linear equation systems (Theorem V.3). More generally, we show in Section VI that POLYLOG-FPC can istinguish between all sequences of matrices which are not simultaneously unitarily similar (Theorem VI.4). This implies, for instance, that a polylogarithmic number of iterations of the Weisfeiler- Lehman metho suffices in orer to istinguish graphs which are not cospectral. Finally, we come to the main application of our POLYLOG-FPC-efinability results in Section VII, where we separate the power of the two propositional proof systems Nullstellensatz an polynomial calculus with respect to the graph (non)-isomorphism problem (Theorem VII.2). II. PRELIMINARIES We assume that the reaer has a soli backgroun in logic. To fully unerstan an appreciate our results, familiarity with the ieas an techniques of finite moel theory will be necessary (see [15], [24], [29], [16]). We briefly review the logics most important here. In this paper, we only consier finite relational structures; a τ-structure A has a finite universe A an a relation R A A k for each k-ary relation symbol R in the vocabulary τ. The class of all (finite) τ-structures is enote by Str(τ). We fix an encoing of pairs (A, B) of τ-structures as structures A, B of some vocabulary τ pair, an we let Pair(τ) enote the class of all pairs of τ-structures encoe this way. We view graphs G = (V (G), E(G)) as {E}-structures with universe V (G) an binary ege relation E(G). We enote the class of all pairs of graphs, encoe as {E} pair -structures, by GraphPair. Let us briefly review first-orer logic FO. First-orer formulas of vocabulary τ, or FO[τ]-formulas, are built from atomic formulas Rx 1,..., x k for k-ary R τ an x = y using the usual Boolean connectives an existential an universal quantifiers ranging over the universe of a structure. We next introuce counting logic C which is the (syntactic) extension of FO that allows counting quantifiers m x ( there exist at least m values for x ) for each m. Note that C an FO are equally expressive, because m x can be expresse in FO using m quantifiers an m istinct variables for x. However, the quantifier rank of formulas in C may be significantly lower

than that of their equivalent counterparts in FO, because a counting quantifier m x only increases the quantifier rank by one. The same hols for the number of variables in a formula. By C k we enote the fragment of C consisting of all formulas with at most k (free or boun) variables. We write A k B if the structures A an B cannot be istinguishe by any sentence in C k, we write A r B if both structures cannot be istinguishe by any sentence of C of quantifier rank at most r, an we write A k r B if both structures cannot be istinguishe by any sentence of C k of quantifier rank at most r. Inflationary fixe-point logic IFP is the extension of FO by a fixe-point operator with an inflationary semantics. To simplify the presentation, we only consier a special case an let ϕ(x, x) be a formula that has a k-tuple x = (x 1,..., x k ) of free iniviual variables ranging over the universe of a structure an a free k-ary relation variable X ranging over k- ary relations on the universe. (The general case is that ϕ has aitional free variables.) For every structure A, we efine a sequence of relations X (i) A k, for i N, by X (0) = an X (i+1) = X (i) ϕ(x (i), x) A for all i N, (II.1) where ϕ(x (i), x) A enotes the set of all ā A k such that A satisfies ϕ if the relation variable X is interprete by X (i) an the iniviual variables in x are interprete by ā. Since we have X (0) X (1) X (2) A k an A is finite, the sequence reaches a fixe-point X (n) = X (n+1), which we enote by X ( ). We use the following syntax for the ifpoperator: [IFP X x. ϕ( x)] ( x), (II.2) In the structure A, this formula efines the relation X ( ). Example II.1. The following IFP-formula efines the transitive closure of a binary relation R: [IFP T xy. (Rxy z(t xz T zy))] (x, y). (II.3) Fixe-point logic with counting (FPC) is the extension of inflationary fixe-point logic by counting terms. Formulas of FPC are evaluate over the two-sorte extension of an input structure A by a linear orer of the size of the input structure. More precisely, we enote by A # the two-sorte extension of a τ-structure A = (A, R 1,..., R k ) by a linear orer ({0,..., n}, <) of length n + 1, where n = A, i.e. the two-sorte structure A # = (A, R 1,..., R k, {0,..., n}, <) where the universe of the first sort (also referre to as vertex sort) is A an the universe of the secon sort (also referre to as number sort or counting sort) is {0,..., n}. For both the vertex an the number sort we have a collection of type first-orer variables. We also allow relation variables X of mixe types, i.e. a relation variable X of type (k, l) N N ranges over relations R A k {0,..., n} l. In particular, we allow fixe-point operators over relations of mixe type. For example if X is of type (k, l) N N, then [IFP X x. ϕ( x)] ( x) is a formula of FPC where the tuple of variables x = x 1,..., x k+l has to be compatible with the type of the relation symbol X, that is x 1,..., x k are vertex variables an x k+1,..., x k+l are number variables. To relate the vertex an the number sort, fixe-point logic with counting allows the formation of counting terms to efine sizes of sets. More precisely, for each formula ϕ we can form a counting term s = [#x. ϕ] whose value s A {0,..., n} in a structure A of size n is the number of elements a A such that A ϕ(a). Logical interpretations: The logical counterpart of the notion of (algorithmic) reuctions is the notion of logical interpretations. Basically, an interpretation I transforms each structure A into a new structure B = I(A) an this transformation is efine by formulas in some logic L. For the applications in this paper it suffices to give a formal efinition for the case L = FO. Let σ, τ be vocabularies with τ = {S 1,..., S l } where S i is an s i -ary relation symbol. An FO-interpretation of Str(τ) in Str(σ) is a tuple I = (ϕ δ ( x), ϕ 1 ( x 1,..., x s1 ),..., ϕ l ( x 1,..., x sl )), where ϕ δ, ϕ 1,..., ϕ l FO(σ), an where x, x 1,... are tuples of pairwise istinct variables of the same length 1 (I is then calle a -imensional interpretation). Let FO[σ τ] enote the set of FO-interpretations of Str(τ) in Str(σ). The quantifier rank of I is the maximal quantifier rank of the firstorer formulas ϕ δ, ϕ 1,..., ϕ l. With every -imensional interpretation I FO[σ τ] we associate a mapping I Str(σ) Str(τ) as follows. For A Str(σ) we efine the τ-structure I(A) = B Str(τ) over the universe B = { b A A ϕ δ ( b)} by setting S B i = {( b 1,..., b si ) B si A ϕ i ( b 1,..., b si )} for each S i τ. The notion of an algorithmic reuction is the central tool in complexity theory to etermine the relative algorithmic complexity between problems. Analogously, one can use logical interpretations to analyse the relative escriptive complexity between problems. In particular, just as in the case of complexity classes, many logics are close uner certain notions of interpretations in a way similar as state in the following lemma for counting logic an FO-interpretations. Lemma II.2. Let I FO[σ τ] be a first-orer interpretation from Str(τ) in Str(σ) of imension 1 an quantifier rank r 0. For every sentence ψ C of counting logic of vocabulary τ with quantifier rank at most l 1 we can fin a sentence ψ I C of vocabulary σ with quantifier rank at most r + l such that for every σ-structure A we have A ψ I if, an only if, I(A) ψ. III. THE POLYLOGARITHMIC FRAGMENT OF FIXED-POINT LOGIC WITH COUNTING We introuce a restriction of fixe-point logic with counting where we boun the iterations of fixe-point operators by a polylogarithmic function. While fixe-point logic with counting (FPC) is tailore to expressing polynomial-time properties of finite structures, this polylogarithmic variant

(POLYLOG-FPC) is efine as a counterpart of the complexity class NC to capture problems which are efficiently parallelisable. To get more intuition, recall that Anerson an Dawar prove that FPC can express precisely those properties of finite structures that can be ecie by uniform families of symmetric circuits of polynomial size [2]. The iea of POLYLOG-FPC is to capture those properties of finite structures which can be ecie by uniform families of symmetric circuits of polynomial size an of polylogarithmic epth. To obtain polylogarithmic fixe-point logic with counting, enote by POLYLOG-FPC, we specify polylogarithmic bouns for the fixe-point operators in the formulas of FPC. More specifically, we use fixe-point operators only in the form [IFP logr (n) X x. ϕ( x)] ( x), for a constant r 1. The semantics of this formula is etermine as above except for that we stop the inflationary evaluation of the fixe point after at most log r (n) steps (no matter of whether the inflationary fixe point was reache). That is, the formula efines the relation X ( logr (n) ), as efine in (II.1). Many natural fixe-point processes converge after a polylogarithmic number of steps, so we have X ( logr (n) ) = X. An example is the formula of Example II.1 efining the transitive closure, which converges after a logarithmic number of steps. It is easy to see that every formula of POLYLOG-FPC can be evaluate in polylogarithmic parallel time an thus the ata complexity of POLYLOG-FPC is in NC. Conversely, it is not har to show that over orere structures, POLYLOG-FPC can express all problems that can be solve in NC (we always refer to the uniform version of NC). In other wors, POLYLOG-FPC captures NC on orere structures. This even hols in the absence of counting, since over orere structures one can use the fixe-point operators to simulate counting terms. A. Embeing into Counting Logic It is known that every formula ϕ FPC can be translate into a family of formulas (ϕ n ) n 1 C k, for some k 1, such that for all n 1, the formula ϕ n is equivalent to ϕ on structures of size at most n, see e.g. [32]. If we apply this embeing to POLYLOG-FPC, then we obtain formulas ϕ n with polylogarithmic quantifier rank. Theorem III.1. For every sentence ϕ POLYLOG-FPC there are constants c, k, l 1 such that for all n 2 there is a sentence ϕ n C k of quantifier rank at most c log l (n) that is equivalent to ϕ on structures of size at most n. B. Immerman s Lower Boun on the Quantifier Rank of Counting Logic Polylogarithmic fixe-point logic with counting is a strict fragment of full fixe-point logic with counting. This follows from an ol construction by Immerman [22]. Theorem III.2 ([22], see also [9], [24]). For sufficiently large m 1 there are graphs G m, H m with the following properties. (1) G m = H m = n, an n < m 1+log(m). (2) G m m H m. (3) G m 3 H m. In wors, the graphs G m an H m cannot be istinguishe by any sentence of counting logic C of quantifier rank at most m, although these graphs can be istinguishe by a sentence of the same logic when we allow unboune quantifier rank an only three variables. Moreover, the size of the graphs G m an H m is quasipolynomial in m. It follows from Immerman s result that FPC is more powerful than POLYLOG-FPC. In fact, a property that witnesses this separation is the graph isomorphism problem over the class of graphs G m an H m from Theorem III.2. More precisely, the class K = { G, H GraphPair G, H {G m, H m } for some m 1} is POLYLOG-FPC-efinable, but not K iso = { G m, G m m 1} { H m, H m m 1}. On the other han, K iso is efinable in FPC. To see this, first recall from Theorem III.1 that over structures of size n every formula of POLYLOG-FPC can be translate into an equivalent formula in counting logic C with a polylogarithmic boun on the quantifier rank (the exponent of the polylogarithmic function is fixe by the POLYLOG-FPCformula). The size of the graphs G m an H m is quasipolynomial in m. Note that if we apply a polylogarithmic function to a quasipolynomial function in m, then we obtain a polylogarithmic function in m which clearly grows slower than the linear function m. Hence the resulting sentences are not able to istinguish between Immerman s graphs G m an H m for large enough m 1. On the other han, it is known that FPC can express equivalence in three-variable counting logic, see e.g. [32]. This suffices to istinguish between G m an H m. Corollary III.3. POLYLOG-FPC < FPC. IV. LINEAR-ALGEBRAIC OBJECTS AS RELATIONAL STRUCTURES A central aim of this paper is to stuy the POLYLOG-FPCefinability of important linear-algebraic problems, such as the solvability of linear equation systems, over the fiel Q of rationals an the fiel C of complex numbers. In this section we iscuss how one can encoe linear-algebraic objects such as matrices, vectors, an so on, by finite relational structures. It will be convenient for us to work over the fiel C throughout. However, the input vectors an matrices to our linear-algebraic problems will always be rationals or Gaussian rationals, that is, complex numbers with rational real an imaginary parts. Note that a system of linear equations with rational coefficients always has a rational solution, an a linear system whose coefficients are Gaussian rationals has a solution in the Gaussian rationals. Complex numbers: We start to explain how to encoe complex numbers c = Re(c) + i Im(c) with Re(c), Im(c) Q. Let τ lin = {<} an let τ C = τ lin {N Re, D Re, N Im, D Im, S Re, S Im } where < is a binary relation symbol an where N Re, D Re, N Im, D Im are unary relation symbols, an where S Re, S Im are nullary preicates. We consier τ C -structures (A, <, N Re, D Re, N Im, D Im, S Re, S Im ) where < is a linear orer on A. Clearly, we can uniquely ientify such structures with structures over the universe

{0,..., n} with < being the natural orer. The preicates be the class of τ seqmat -structures representing such sequences. N Re, D Re, N Im, D Im represent the binary encoings of the Linear equation systems: We also nee an encoing for absolute values of the numerators an enominators of the real an imaginary part of c. For example, the numerator of the absolute value of the real part is encoe by N Re an is etermine as i NRe 2 i. The nullary preicates S Re, S Im are use to encoe the signs of Re(c) an Im(c), respectively. Of course, we require some simple consistency conitions, for example D Re D Im (non-zero enominators). We enote by K C the (first-orer efinable) class consisting of all linear equation systems with complex coefficients by finite relational structures. We can represent every such system in the form M x = b for an I J-coefficient matrix M I J C an an I-vector of constants b I C. We alreay saw above that we can encoe such pairs (M, b) by relational structures. Hence, we let τ les enote an appropriate vocabulary to encoe linear equation systems (M, b) an let K les be the class of τ les -structures which encoe linear equation systems. τ C -structures which encoe complex numbers. Matrices: Formally, matrices M (with complex V. LINEAR ALGEBRA IN POLYLOGARITHMIC FIXED-POINT coefficients) are mappings M I J C for two nonempty LOGIC WITH COUNTING inex sets I, J. In general we o not require that the inex sets I an J are orere. To stress this fact we sometimes speak of unorere matrices. To represent unorere matrices by structures we use the vocabulary τ Mat = {I, J, L, <, N Re, D Re, N Im, D Im, S Re, S Im } with unary relation symbols I, J, L, binary relation symbols <, S Re, S Im an 4-ary relation symbols N Re, D Re, N Im, D Im. We consier τ Mat -structures (A, I, J, L, <, N Re, D Re, N Im, D Im, S Re, S Im ) where < is a linear orer on L, where L, I, J, I J L = A, an where for every i I an j J, the structure (L, <, N Re (i, j,, ), D Re (i, j,, ), N Im (i, j,, ), D Im (i, j,, ), S Re (i, j), S Im (i, j)) encoes the (i, j)-th entry M(i, j) C of the matrix M I J C as a complex number as explaine in the preceing paragraph. We enote by K Mat the (first-orer efinable) class consisting of all τ Mat -structures which encoe matrices in this sense. We usually ientify matrices with their structural encoings. A matrix M K Mat, M I J C, is a square matrix if I = J. We ientify (column) vectors with mappings b I {0} C an row vectors with mappings b {0} J C. In particular vectors are structures in K Mat. We speak of an orere matrix M K Mat if I, J L, that is if the encoing of M provies a linear orer on the inex sets for the rows an columns. Families of matrices: For the sake of conciseness, we o not explicitly specify the following encoings. We consier pairs of matrices compatible with matrix aition, that is pairs of matrices M I J C an N I J C over the same inex sets I an J. Let τ pairmat(+) enote an appropriate vocabulary to encoe such pairs an let K pairmat(+) enote the class of τ pairmat(+) -structures representing such pairs. We further consier structures which encoe pairs of matrices M I J C an N J K C which are compatible with matrix multiplication. Let τ pairmat( ) enote an appropriate vocabulary to encoe such pairs an let K pair-mat( ) be the class of τ pairmat( ) -structures representing such matrix pairs. Moreover, we consier (orere) sequences of square matrices over a common inex set, that is sequences of the form M = (M 0,..., M s 1 ) where all M i are square matrices M i I I C. All matrices in this sequence are compatible with matrix aition an multiplication. In particular, since the sequence is orere, the prouct of the sequence is also well-efine. Let τ seqmat enote an appropriate vocabulary to encoe orere sequences of square matrices an let K seqmat We explore the expressive power of POLYLOG-FPC with respect to queries from linear algebra over the complex fiel C. We will see that many non-trivial properties, such as the solvability of linear equation systems an the coefficients of the characteristic polynomial, can be expresse in POLYLOG-FPC. Central ieas that unerly our approaches have been use, for example, also by Holm an Laubner [21], [28] in orer to obtain efinability results for the stronger logic FPC. As POLYLOG-FPC captures NC on orere structures, we can express every NC-eciable property over the numeric sort in POLYLOG-FPC. For instance, we can efine in POLYLOG-FPC the aition an multiplication of (sets of) complex numbers (in binary representation). Also, over orere inputs, we can efine a host of important problems from linear algebra in POLYLOG-FPC over C an over finite fiels, such as (iterate) matrix multiplication, computing the coefficients of the characteristic polynomial, computing the matrix rank, an also eciing the solvability of linear equation systems, see e.g. [26]. However, this oes not mean that the corresponing problems remain POLYLOG-FPC-efinable over unorere inputs. In fact, Atserias, Bulatov, an Dawar showe that the solvability of linear equation systems over finite fiels can not be efine even in full FPC [4]. Surprisingly, the picture changes if we consier queries from linear algebra over the complex fiel. Here it turns out that the above mentione queries are all efinable in POLYLOG-FPC also over unorere inputs. A. Matrix Arithmetic We first observe that matrix aition can be efine in POLYLOG-FPC. Consier a pair of matrices (M, N) K pairmat(+), M I J C an N I J C. Then we have that (M + N)(i, j) = M(i, j) + N(i, j). Hence, we only have to express the aition of complex numbers. As complex numbers are orere objects this is possible in POLYLOG-FPC, since POLYLOG-FPC can express every NCproperty of orere inputs. For us the most important observation is that the multiplication of two unorere complex matrices is POLYLOG-FPCefinable. Assume we have a representation of two matrices (M, N) K pair-mat( ), M I J C an N J K C, compatible for matrix multiplication. Recall that the (i, k)- th entry of the prouct (M N) is given as M(i, k) =

j J M(i, j) N(j, k). Again, the multiplication of the complex numbers M(i, j) N(j, k) is efinable in POLYLOG-FPC, because this is an NC-computable function. The interesting question, however, is how to evaluate the unorere sum of complex numbers in the above equation. Here we crucially rely on the counting mechanism of POLYLOG-FPC in orer to reuce this task to the evaluation of an orere sum of complex numbers. First we consier the POLYLOG-FPCefinable linear preorer on J that is efine as j j if (M(i, j) N(j, k)) (M(i, j ) N(j, k)), where is the orer on C efine as x y if Re(x) < Re(y) or Re(x) = Re(y) an Im(x) Im(y). The resulting equivalence classes j/ J/ consist of elements which contribute the same complex number to the above sum. If we enote the sizes of these classes by j/, then we get j J M(i, j) N(j, k) = j/ J/ j/ M(i, j) N(j, k). Since inuces a linear orer on J/ an since we can etermine the sizes of the equivalence classes j/ with counting terms we have inee reuce the evaluation of an unorere sum of complex numbers to the evaluation of an orere sum. In this way we obtain the POLYLOG-FPCefinability of matrix multiplication. The above generalises for proucts over orere sequences of square matrices. Inee, since POLYLOG-FPC can express proucts of pairs of matrices, we can evaluate in POLYLOG-FPC proucts (of polynomial length) using a stanar ivie-an-conquer approach. Let us also mention that the above technique to evaluate unorere sums of complex numbers in POLYLOG-FPC has many further applications. For example, it can be use to show that the trace tr(m) = i I M(i, i) of a square matrix M C I I is efinable in POLYLOG-FPC. B. Linear Equation Systems over the Complex Fiel We escribe a new, an surprisingly simple, POLYLOG-FPC-efinable reuction that transforms a given linear equation system with unorere sets of variables an equations into an equivalent linear equation system with orere sets of variables an equations (over the complex fiel C). Since the solvability of linear equation systems (over the fiel C) can be ecie in NC, it follows from this reuction that the solvability of an (unorere) linear equation system over C can be efine in POLYLOG-FPC. Let M x = b be a linear equation system for an I J- matrix M over C an an I-vector b over C. The first step is to transform this system into a (solvability-)equivalent linear equation system whose coefficient matrix is a Hermitian matrix, that is a square complex matrix that is equal to its own conjugate transpose. Recall that the conjugate transpose of a matrix S I I C is the matrix S which results from the transpose S T of S by taking the complex conjugates of all entries, for instance: ( i 1 + i 2 i 2 ) = ( i 2 + i 1 i 2 ). Let K = I J. Then the linear equation system N y = c over C with the Hermitian K K-coefficient matrix N an the K-vector c given as N = ( 0 M M 0 ), c = (b 0 ), is (solvability-)equivalent to the original system an can easily be obtaine from M x = b. From now on assume that M x = b is a linear equation system over C given by an Hermitian matrix M C I I an an I-vector b C I. The important consequence of M being Hermitian is the following well known fact from linear algebra. Recall that the kernel ker(m) of a matrix M C I J is the set of all c C J such that Mc = 0. Lemma V.1. If M C I I is Hermitian, then ker(m) = ker(m i ) for all i 1. Proof. This easily follows by inuction on i 1. The case i = 1 is trivial so let i = 2. Clearly, we have ker(m) ker(m 2 ) an in general ker(m) ker(m i ) for all i 1. On the other han, if M 2 a = 0, then also M Ma = 0 since M = M. Hence, also a M Ma = 0. This implies that (Ma) (Ma) = 0 which in turn implies that Ma = 0 an hence a ker(m). For i > 2 we have that M i a = 0 implies M i 1 (Ma) = 0. By inuction hypothesis, we have Ma ker(m), hence a ker(m 2 ) = ker(m). We remark that an analogous result for symmetric matrices over finite fiels oes not hol. The following lemma is the key step in our transformation of an unorere system of linear equations into an orere system. Lemma V.2. Let M C I I be Hermitian, b C I, m = I. The linear equation system M x = b is solvable if, an only if, b Span(Mb,..., M m b). Proof. For the backwar irection, note that if b = m i=1 z i M i b Span(Mb,, M m b), then c = m i=1 z i M i 1 b is a solution for the system Mx = b. For the forwar irection, assume that Mc = b for a solution c C I. The set {b, Mb,..., M m b} has size m + 1 an so it is linearly epenent. Hence, let z 0 b + z 1 Mb + + z m M m b = 0 for a non-zero z = (z 0,..., z m ) C m+1. Let i 0 be minimal such that z i 0. If i = 0, then we are one. Otherwise z i M i b+ + z m M m b = 0 for some i > 0. Now we make use of the fact that Mc = b. Then the former equation can be rewritten as z i M i+1 c + + z m M m+1 c = 0. We obtain M i+1 (z i c + + z m M m i c) = 0. By Lemma V.1 we conclue that M(z i c+ + z m M m i c) = 0. Again by making use of the fact that Mc = b we get z i b + z i+1 Mb + + z m M m i b = 0. Since z i 0 the claim follows. Let N enote the I {1,..., m}-matrix over C whose i-th column is the I-vector M i b. By the above, we conclue that the linear equation system M x = b is (solvability-)equivalent to the system N y = b whose columns are inexe by the orere set {1,..., m}. The lexicographical orer inuce by this orer on the rows of N is a linear orer up to uplicates of

equations. Hence, we obtain an equivalent system with orere sets of variables an equations. It follows from our earlier iscussion about the POLYLOG-FPC-efinability of matrix multiplication that our outline reuction can be expresse in POLYLOG-FPC. The solvability of the orere system can be efine in POLYLOG-FPC since this is an NC-property. Theorem V.3. There exists a POLYLOG-FPC-sentence ϕ of vocabulary τ les such that for all linear equation systems (M, b) K les we have (M, b) is solvable (M, b) ϕ. For later reference, we also state a version in terms of counting logic with polylogarithmic quantifier rank. Corollary V.4. There exists t 1 such that for all sufficiently large n 1 we can fin a sentence ϕ n C of vocabulary τ les such that qr(ϕ n ) log t (n), an such that for all linear equation systems (M, b) K les of size (M, b) n we have (M, b) is solvable (M, b) ϕ n. C. Coefficients of the Characteristic Polynomial We next show that also the coefficients of the characteristic polynomial of complex matrices can be efine in POLYLOG-FPC. This result has many interesting consequences. For instance, it follows that the eterminant an the rank of complex matrices are POLYLOG-FPC-efinable. The efinability of the characteristic polynomial for complex matrices has been establishe in [21] with respect to full FPC. The iea there is to show that Csanky s algorithm can be expresse in FPC. Our observation is that for the simulation of this algorithm a POLYLOG-FPC-formula suffices. Let us recall Csanky s algorithm. For etails we refer to [26]. Let M C I I be an I I-matrix over C. Let n = I. Then the characteristic polynomial χ M of M is χ M = et(x i I M) = s 0 x n s 1 x n 1 + s 2 x n 2 ± s n, where the coefficients s i can be obtaine via the following linear recurrences: s 0 = 1, s 1 = tr(m), an, more generally, s k = 1 k (s k 1 tr(m) s k 2 tr(m 2 ) + ± tr(m k )) s n = 1 n (s n 1 tr(m) s n 2 tr(m 2 ) + ± tr(m n )) = et(m). These relations can be obtaine from Newton s ientities for symmetric polynomials. Now, if we want to efine this system of linear recurrences in POLYLOG-FPC, then we only have to etermine the powers M i of the matrix M for all i n = I an then compute the corresponing traces tr(m i ), that is the sums over the iagonal entries of the matrices M i. It follows from our earlier observations that this is possible in POLYLOG-FPC. The question remains whether the unique solution of this system, which is the characteristic polynomial of the matrix M, can be efine in POLYLOG-FPC. Again, this follows from the fact that this unique solution can be compute in NC an the observation that the above system of linear recurrences is orere. To see this, we express the above system as a matrix equation L s = c for a lower triangular matrix L. Then the unique solution of the system is given as s = L 1 c. Hence, in orer to compute the solution it suffices to compute the inverse L 1 of the non-singular lower triangular matrix L. This can be one in NC by using a iviean-conquer strategy base on the following ientity which hols for all non-singular lower triangular square matrices L Mat n (C) an square matrices A, B, C Mat n/2 (C): for L = ( A 0 C B ), we have A L 1 = ( 1 0 B 1 CA 1 B 1 ). As mentione before, from this efinability result we can extract POLYLOG-FPC-efinability results for the eterminant, the matrix rank, matrix inverses, an so on, via the stanar reuctions, see for example [26]. For completeness, we recall the steps which reuce the computation of the rank of a matrix M C I J to the computation of a characteristic polynomial. The first step is to obtain a Hermitian matrix. Using similar arguments as in the proof of Lemma V.1 it is easy to check that the rank of the matrix M is the same as the rank of the matrix M M. Since M M is Hermitian, we get: rk(m) = rk(m M) = rk ((M M) 2 ). Finally, for matrices N C I I with the property rk(n) = rk(n 2 ), the matrix rank can be etermine using the following well-known result from linear algebra. Theorem V.5. Let N C I I with I = n an such that rk(n) = rk(n 2 ). Then rk(n) = n k where k 0 is maximal such that x k ivies the characteristic polynomial χ N (x). D. Application to First-orer Rank Logic In his issertation [21], Holm introuce an extension of first-orer logic by operators which can compute the rank of matrices over the rationals Q. He prove that this logic, enote by FOR Q, is containe in FPC an that over orere structures it captures the exact logspace counting hierarchy, a complexity class introuce by Allener an Ogihara [1] to capture the (algorithmic) complexity of eciing singularity for integer matrices. Holm leaves open whether FOR Q is a strict fragment of FPC. From our above efinability results it follows that Holm s logic FOR Q is containe in POLYLOG-FPC. Hence we get the following. Theorem V.6. FOR Q POLYLOG-FPC < FPC. VI. SIMULTANEOUS UNITARY SIMILARITY In this section we review a classical theorem of Specht from 1940 which provies a characterisation for the simultaneous unitary similarity problem for pairs of orere sequences of square matrices (of the same imension) over the fiel of complex numbers. Our motivation is to use this criterion to show that counting logic with polylogarithmic quantifier rank can istinguish between all pairs of (non-isomorphic) graphs which have ifferent linear-algebraic properties over the fiel of complex numbers.

Recall that the set C I I of all (I I)-square matrices over C forms a C-algebra. By GL I (C) we enote the multiplication group of all non-singular matrices S C I I. A non-singular matrix S C I I is calle unitary if its inverse is the conjugate transpose, that is, S 1 = S. Two matrices A, B C I I are similar if there exists an invertible matrix S GL I (C) such that SA = BS. We consier a refinement of this equivalence relation an say that two matrices A, B C I I are unitarily similar if we can fin a unitary matrix S GL I (C) such that SA = BS. We will show that if two graphs cannot be istinguishe by fixe-point logic with counting with a polylogarithmic number of iterations, or equivalently, by a formula of counting logic C of polylogarithmic quantifier rank, then all finite sequences of matrices efinable by a fixe first-orer interpretation over the graphs are (in fact simultaneously) relate via a unitary similarity transformation. This means, in particular, that all of these matrices are similar. Example VI.1. As a simple application, where we just nee to look at the ajacency matrices of the graphs, we see that POLYLOG-FPC can istinguish between graphs which are not cospectral, that is which have ifferent (multi-)sets of eigenvalues. Let s 1 an let M = (M 1,..., M s ) an N = (N 1,..., N s ) enote two sequences of square matrices M i, N i C I I with complex coefficients an with the same inex sets I. We say that the sequences M an N are simultaneously unitarily similar for short s.u.s., if there exists a single unitary matrix S GL I (C) such that SM i = N i S for all 1 i s. To see whether two orere families of complex matrices are s.u.s. we make use of the following characterisation by Specht. Let Σ s = {x 1, x 1,..., x s, x s} enote the alphabet consisting of the letters x i an x i for all 1 i s. For a finite wor x Σ <ω s over Σ s let x M C I I enote the matrix which results from x by replacing all letters x i an x i by the matrices M i an Mi, respectively, an by evaluating the resulting prouct. In particular, for the empty wor x = ε Σ <ω s we agree to set x M = i I. The matrices x N C I I for wors x Σ <ω s are efine in the same way. Two sequences M = (M 1,..., M s ) an N = (N 1,..., N s ) of complex matrices M i, N i C I I are trace equivalent if for all wors x Σ <ω s we have tr(x M ) = tr(x N ). Moreover, for k 1, the sequences M an N are k-trace equivalent if for all wors x Σ k s we have tr(x M ) = tr(x N ). Theorem VI.2 (Specht s Theorem [35], [37], see also [34]). Two sequences M = (M 1,..., M s ) an N = (N 1,..., N s ) of complex square matrices M i, N i C I I are simultaneously unitarily similar if, an only if, they are trace equivalent. Specht s Theorem provies us with a criterion to test whether a pair of matrix sequences is s.u.s. in POLYLOG-FPC. Inee, we only have to verify for every wor x Σ <ω s whether tr(x M ) = tr(x N ) hols or not. This comes own to efining proucts an traces of matrices which, as we saw earlier, we can o in POLYLOG-FPC. Specifically, we have to express the following steps in POLYLOG-FPC for every wor x Σ <ω s : first of all, we efine the conjugate transposes Mi an N i of all input matrices M i an N i, an seconly, we etermine the proucts x M an x N specifie by the wor x, an finally, we compare the traces of x M an x N. Basically all these steps are expressible in POLYLOG-FPC. However, there are two obvious problems. The most striking one is that there infinitely many wors x Σ <ω s for which we have to test whether tr(x M ) = tr(x N ) hols or not. In particular, the length of these wors x Σ <ω s is not boune which means that also the sizes of (the representations of) the entries of the matrices x M an x N can become arbitrarily large. Fortunately it is not necessary to check all wors in Σ <ω s. In fact, Pearcy [33] prove that the two sequences M an N of square matrices are trace equivalent if, an only if, they are 2n 2 -trace equivalent where n 1 enotes the imension of the square matrices M i, N i, that is size of the inex set I. Theorem VI.3 ([33], see also [34]). Two sequences M = (M 1,..., M s ) an N = (N 1,..., N s ) of complex square matrices M i, N i C I I of imension n = I are trace equivalent if, an only if, M an N are 2n 2 -trace equivalent. This means that we only have to consier proucts x M an x N for all wors x Σ 2n2 s. In particular, the length of such proucts is polynomially boune in the imension of the input matrices which also gives us a polynomial boun on the (representations of the) entries of the matrices x M an x N. From Section V we know that matrix proucts of polynomial length can be evaluate in POLYLOG-FPC. Also efining traces of matrices is possible in POLYLOG-FPC as this correspons to the summation over an (unorere) set of complex numbers. Unfortunately, there is a secon problem: there still are exponentially many wors x Σ 2n2 s which we have to check. Clearly, we cannot go through all wors explicitly in POLYLOG-FPC. However, since we o not aim for an POLYLOG-FPC-efinability result, this oes not cause any trouble. In fact what we can o is to make use of the embeing of POLYLOG-FPC into counting logic C (Theorem III.1) which gives us for sequences of matrices M an N of a fixe size an for every fixe wor x Σ 2n2 s a sentence in C with a polylogarithmic quantifier rank that expresses the trace conition tr(x M ) = tr(x N ) for x M an x N. We can then take the conjunction over all these sentences for x Σ 2n2 s, which oes not increase the quantifier rank, to obtain a efinition for the simultaneous similarity of the sequences M an N in C with polylogarithmic quantifier rank. Theorem VI.4. There are k, r 1 such that the following hols. Let M = (M 1,..., M s ) K seqmat an N = (N 1,..., N s ) K seqmat be sequences of complex square matrices over inex sets I an J of the same size, that is M i C I I an N i C J J an I = J, such that the size of M an N is n (where n 1 is large enough an measures

the size of the structures M an N ). If M k log r (n) N, then M an N are simultaneously unitarily similar. Theorem VI.4 basically says that in counting logic with polylogarithmic quantifier rank we can express all linearalgebraic properties of finite structures which are invariant uner unitary similarity (over the complex fiel). Let us give a more concrete application. We fix some constant l 1. Now consier a sequence of formulas ϕ = (ϕ 1 (x 1,..., x l, y 1,..., y l ),..., ϕ s (x 1,..., x l, y 1,..., y l )) of counting logic over graphs. Then this sequence efines in every graph G = (V, E) a sequence of square matrices G ϕ = (M ϕ 1,..., M ϕ s ) with entries {0, 1} C over the inex set V l in the obvious way: M ϕ i ( v, w) = 1, if, an only if, G ϕ i( v, w). Now what Theorem VI.4 says is that we can fin constants k (the imension) an r (the polylogarithmic exponent) such that whenever two graphs G an H of size n are equivalent with respect to all formulas of k-variable counting logic with quantifier rank log r (n), then the two sequences of matrices G ϕ an H ϕ are simultaneously unitarily similar. Finally, let us remark that our POLYLOG-FPC-efinability result in this section oes not hol over finite fiels. In fact, over finite fiels, similarity of matrices cannot be efine even in full FPC [4]. Furthermore, quite interestingly, the criterion of simultaneous similarity of matrices over finite fiels has been use by Dawar an Holm in [14] to efine an isomorphism test which is provably strictly stronger than the Weisfeiler-Lehman metho. Our results in this section show that this isomorphism test collapses to the usual Weisfeiler- Lehman metho if consiere over the complex fiel. VII. APPLICATION TO GRAPH ISOMORPHISM TESTING AND PROOF COMPLEXITY We now turn to the main application of our efinability results, which on a high level we alreay escribe in Section I-B. We start by introucing the relevant algebraic proof systems an the encoing of the graph isomorphism problem. (For etails, we refer to [7] an the references cite there.) A. Algebraic Proof Systems Let F be a fiel (in this paper, F will always be the fiel C of complex numbers). We consier polynomial equations over a set of variables X j, j J, ranging over F. We o not assume to have an orering on the inex set J of variables, that is we consier polynomial equations over unorere sets of variables. We enote by F[ X] the ring of (multivariate) polynomials in variables X j, j J, an with coefficients from the fiel F. For a multi-inex α J N we let the monomial X α be efine as X α = Π j J X α(j) j. Then polynomials f F[ X] can be written as f = α f α X α where the f α F are coefficients from the fiel F an such that f α 0 for finitely many α only. The egree eg(x α ) of a monomial X α is efine as α = j J α(j), an the egree eg(f) of a polynomial f = α f α X α is efine as the maximal egree of a monomial X α occurring in f with non-zero coefficient f α 0. Note that eg(f g) = eg(f)+eg(g) for all non-zero polynomials f, g F[ X] {0}. A polynomial equation is an equation of the form f = 0 for a polynomial f F[ X]. For better reaability, we usually omit the equality = 0 when we specify polynomial equations, that is we ientify polynomials f F[ X] with the corresponing polynomial equations f = 0. A system of polynomial equations is a set P = {f i i I} consisting of polynomials f i F[ X] for all i I where I is an (unorere) inex set. A solution of P is a common zero ā F J of all polynomials in P. In what follows, we only consier systems P = {f i i I} which contain for every variable X = X j, j J, the polynomial equation (X 2 X) = 0. Note that these axioms (X 2 X) = 0 enforce that each variable X = X j, j J, can only take values 0 or 1. The algebraic proof systems we shall introuce next can be use to refute a system P of equations, that is, to prove that it has no solution. Nullstellensatz Proof System: This proof system is base on Hilbert s Nullstellensatz, an important result from algebra saying that the non-solvability of a system P = {f i i I} is equivalent to the existence of polynomials g i F[ X], i I, such that i I g i f i = 1. The polynomials g i are calle a Nullstellensatz refutation for the system P. It can be shown that by the axioms X 2 X one can restrict to polynomials g i whose egree is linear in the number of variables. Hence, Hilbert s Nullstellensatz can be use to search effectively, but of course not efficiently, for proofs for the non-solvability of the polynomial system P. If we want to systematically search for Nullstellensatz refutations efficiently, that means in polynomial time, then we have to restrict the search space for the polynomials g i. The most important approach is to restrict the egree of the polynomials g i by a constant. Inee, if we fix 1, then we can check efficiently, namely in time n O(), whether there are polynomials g i F[ X] satisfying that the egree of all proucts g i f i is boune by, an i I g i f i = 1. Nullstellensatz proofs via linear equation systems: We can reuce the question of whether a system P of polynomial equations in n variables has a Nullstellensatz refutation of egree at most to the question of solving a system of linear equations in n O(). This not only shows that we can fin a refutation of fixe egree in polynomial time, but also will enable us to connect Nullstellensatz refutations to efinability in POLYLOG-FPC. The reuction goes back to [10]. As a first step, let us simplify the situation a little bit. In fact, the axioms (X 2 X) = 0 allow us to restrict to linearise polynomials. Formally, we efine the linearisation operator Lin F[ X] F[ X] as the F-linear operator uniquely etermine by Lin(X α ) = X β where β(j) = 1 if α(j) > 0 an β(j) = 0 if α(j) = 0. Then Lin respects aition an F-scalar multiplication, but not multiplication. For example Lin(X (X 1)) Lin(X) Lin(X 1). However, we obtain the following weak preservation ientity for multiplication. If f, g F[ X], then Lin(f g) = Lin(Lin(f) Lin(g)).