The real line: the equational compatibility problem and one-variable variants Hilbert s 10 th Problem

The real line: the equational compatibility problem and one-variable variants Hilbert s 10 th Problem George F. McNulty Abstract. Walter Taylor proved recently that there is no algorithm for determining of a finite set of equations whether it is topologically compatible with the real numbers R in the sense that it has a model with universe R and with basic operations which are all continuous with respect to the usual topology of the real line. Taylor s account used operations symbols suitable for the theory of rings together with three unary operation symbols intended to name trigonometric functions supplemented finally by a countably infinite list of constant symbols. In this note we show how to reduce this list of constant symbols to a single constant symbol. Introduction Let T be a topological space and let Γ be a set of equations. We will say that Γ is compatible with T if there is an algebra T with universe T whose basic operations are continuous such that Γ is true in T. By a signature we mean a system of operation symbols each equipped with a natural number to specify its rank (i.e. its number of arguments). Let σ be a countable computable signature. The Equational Compatibility Problem for T and σ Find an algorithm for determining of any finite set Γ of equations of signature σ whether Γ is compatible with T. We say that this compatibility problem is algorithmically unsolvable if no such algorithm exits. Walter Taylor proved in [13] that the equational compatibility problem is algorithmically unsolvable for the real line R and the signature of ring theory enhanced by three additional operation symbols of rank 1 and a countably infinite number of constant symbols. He notes that this countably infinite list of constant symbols can be replaced by finitely many. In fact, Taylor devised a method for establishing the algorithmic unsolvability of equational compatibility problems. His method has two parts: a topological finite determination theorem and the algorithmic unsolvability of an appropriate variant of Hilbert s 10 th Problem. Suppose that T is a topological space, that σ and σ are signatures with σ an expansion of σ, and that T is an algebra of signature σ with universe T and for which all the basic operations are continuous. We will say that the topological algebra T is finitely determined with respect to σ provided there is a finite set Σ of equations of signature σ which is compatible with an expansion of T and up to isomorphism (simultaneously algebraic and topological) T 1

2 GEORGE F. MCNULTY is the only topological algebra of signature σ which can be expanded to a topological model of Σ. We also say that T is finitely determined by Σ. For the second element of Taylor s method we formulate: Hilbert s 10 th Problem in n variables for the algebra T Find an algorithm for determining of any finite set Γ of equations in no more than n variables whether Γ has a solution in T. Taylor s method is predicated on establishing for a given topological algebra T of signature σ that it is finitely determined with respect to some σ by some set Σ of equations and that Hilbert s 10 th Problem in n variables for T is not solvable algorithmically. Then the algorithmic unsolvability of the Equational Compatibility Problem for T and σ can be deduced from these two results as follows. Expand the signature σ by n new constant symbols c 0, c 1,..., c n 1 to obtain the signature σ and for each finite set Γ of equations of signature σ in the variables x 0, x 1,..., x n 1 associate the set Γ of equations by substituting the constant symbol c i for the variable x i, for each i < n. Then Γ has a solution in T if and only if Σ Γ is compatible with T. This reduces Hilbert s 10 th Problem in n variables for the algebra T to the equational compatibility problem for T and σ. Walter Taylor carried out this method for the real line. Taylor s proof is based on the topological algebra R = R, +,,, 0, 1, sin where sin (χ) := sin( π 2 χ) for all real numbers χ, and where R is given the usual topology on the real line. We will reserve σ to denote the signature of this algebra. We also need two more unary operations on R, namely cos (χ) := cos( π 2 χ) and λ(χ) := cos (sin (χ)) for all real numbers χ. We use σ to denote the expansion of σ by two addition unary operation symbols. To distinguish the operation symbols of a signature from the basic operations of an algebra, we use boldface for the symbols. So our operation symbols will be +,,, 0, 1, sin, cos, λ. Many familiar equations hold in R and its expansion by cos and λ. Among them, Taylor distinguished the following set Σ: x + (y + z) (x + y) + z x + y y + x x + 0 x x + ( x) 0 x (y z) (x y) z x y y x x 1 x x (y + z) x y + x z cos (x + y) cos (x) cos (y) sin (x) sin (y) sin (1) 1 sin (x + y) sin (x) cos (y) + cos (x) sin y cos (0) 1 cos (sin (x)) (λ(x)) 2 cos (1) 0 In [13], Taylor proved

TAYLOR S ALGORITHMIC UNSOLVABILITY OF TOPOLOGICAL R-COMPATIBILITY 3 Taylor s Finite Determination Theorem for R. The topological algebra R is finitely determined with respect to σ by the set Σ of equations. and Hilbert s 10 th Problem for R. There is a natural number n so that Hilbert s 10 th Problem in n variables for R is algorithmically unsovlable. As Hilbert originally framed the problem (for the ring of integers), there was no finite bound on the number of variables under consideration. The resolution of Hilbert s 10 th Problem actually showed that the recursively enumerable (i.e. listable) sets and the Diophantine sets coincide. From this it follows that there must be a finite bound for Hilbert s 10 th Problem over the integers. This bound will be inherited automatically in Taylor s result for R. The best known bound is n = 9 due to Matiyasevich a detailed exposition of this result can be found in [6]. Actually, Taylor proved a stronger form of his Finite Determination Theorem, namely that cos is also determined by Σ. The set Σ does not determine, in the topological sense, the interpretation of λ(x) as cos (sin (χ)) for all real numbers χ, since choosing the negative square root, for appropriate reals, works equally well. However, adding the two equations λ(x + 2) λ(x) λ(0) 1 to Σ forces the selection of the nonnegative square root in all cases. In section 8.1 of [13], Taylor presents alternatives for R and for the set Σ. The alternative basic operations still include the ring operations but the trigonometric functions are replaced by functions which are piecewise rational. It is not clear that the methods we consider below can be extended to this case. In essence, Hilbert s 10 th Problem was formulated for the ring of integers. It was shown to be algorithmically unsolvable by way of a proof laid out in the work of Martin Davis, Hilary Putnam, and Julia Robinson and completed by Yuri Matiyasevich. Notable references here are the 1950 abstract of Davis [2] (see also [3]), the 1958 paper of Martin Davis and Hilary Putnam [4], the 1960 abstract of Julia Robinson [12], the groundbreaking 1961 work of Davis, Putnam, and Robinson [5], and finally Yuri Matiyasevich s 1970 achievement [7]. A fully detailed and high readable exposition of this result and related matters was given in 1993 by Yuri Matiyasevich [10]. On the other hand, Hilbert s 10 th Problem has long been known to have a positive solution, as a result of Tarski s decision method [14], when formulated for the ring of real numbers or the ring of complex numbers. The situation arising from the expansion of the ring of reals (or of complex numbers) by various exponential and trigonometric functions has attracted considerable attention. In particular, Matiyasevich [10] has been able to show that Hilbert s 10 th Problem in 1 variable for the algebra R, +,,, 0, 1, sin is algorithmically unsolvable. This result of Matiyasevich depends substantially on earlier work of Daniel Richardson [11] in 1968, B. F. Caviness [1] in 1970, and Paul Wang [15] from 1974. Indeed, Wang s result differs from that of Matiyasevich essentially only because Wang needed to include π as a constant of the algebra.

4 GEORGE F. MCNULTY It is inviting to attempt the modification of the path to Taylor s theorem along the lines suggested by the work of Richardson, Caviness, Wang, and Matiyasevich with the reward that we shall only need one additional constant symbol. We take up the invitation. There are two essentially different approaches. We could attempt to adapt Taylor s Finite Determination Theorem to apply to sin rather than sin or we could adapt the arguments of Wang, Caviness, and Richardson to apply to sin rather than to sin. We take the second approach. We establish Theorem 0. Hilbert s 10 th Problem in 1 variable for R, +,,, 0, 1, sin is algorithmically unsolvable. As shown in Section 4, we can apply Taylor s method to obtain Theorem 1. The equational compatibility problem for the real line R with signature σ, expanded by one additional constant symbol, is algorithmically unsolvable. The remainder of the paper is primarily devoted to proving Theorem 0. I would like to thank Walter Taylor for his careful reading of this paper and for his helpful advice. 1. The methods of Richardson, Wang, and Matiyasevich Richardson s 1968 paper contains a number of interesting results, among which we draw attention to the two which are directly relevant here. Richardson s First Theorem. There is a natural number n so that Hilbert s 10 th Problem in n variables for R = R, +,,, 0, 1, exp, sin, ln 2, π, q q Q is algorithmically unsolvable. Let ρ be the signature appropriate to the algebra R over the reals that occurs in the statement of Richardson s First Theorem. Richardson s Second Theorem. There is no algorithm which, given a term t(x) of signature ρ in no more than one variable, will determine if there is a real number r such that t R (r) is negative. Daniel Richardson s work was accomplished before Matiyasevich completed the resolution of Hilbert s 10 th Problem. Instead, Richardson used the result of Davis, Putnam, and Robinson [5] that the version of Hilbert s 10 th Problem over the integers for exponential Diophantine equations is algorithmically unsolvable. For this reason, Richardson considered an algebra over the reals which included among its basic operations the natural exponential function and, as a distinguished constant, the natural logarithm of 2. The latter was needed since the Davis-Putnam-Robinson exponential Diophantine equations involved 2 x and Richardson s method invoked partial derivatives. The other basic operations Richardson demanded over the reals were the ring operations of addition, multiplication, negation, 0, and 1, the sine function, π as a distinguished constant, and a distinguished constant for each rational. A careful reading of Richardson s work [11] shows that all those rationals (apart from 1) need not be distinguished as constants. B. F. Caviness [1], also

TAYLOR S ALGORITHMIC UNSOLVABILITY OF TOPOLOGICAL R-COMPATIBILITY 5 not yet aware of Matiyasevich s work, points out that in the event that Hilbert s 10 th Problem is algorithmically unsolvable (as it indeed is), then the exponential function and ln 2 can be dropped as basic operations from Richardson s results. Caviness also simplified some parts of Richardson s argument. By 1974, the time of Paul Wang s paper [15], the resolution of Hilbert s 10 th Problem by Matiyasevich was known. Let Wang was able to show R = R, +,,, 0, 1, sin, π, q q Q. Wang s Theorem. Hilbert s 10 th Problem in 1 variable for R is algorithmically unsolvable. In section 9.2 of his book Matiyasevich [10] presents an exposition of a version of these results. In the process he drops the rationals as distinguished constants and, more surprisingly, is also able to drop π as well, leaving only the ring operations and the sine function. Moreover, Matiyasevich gives a readable account with full details. However, it is Wang s Theorem to which we will appeal most directly. Except at one important point, the way in which π and the sine function enter into Richardson s original argument, and into its subsequent adaptations by Caviness and Wang, is in expressions like F (x 0,..., x n 1 ) = (n + 1) 2 [ p 2 (x 0,..., x n 1 ) + i<n(sin 2 πx i )k i (x 0,..., x n 1 ) where p(x 0,..., x n 1 ) and k i (x 0,..., x n 1 ) are certain polynomials with integer coefficients. Such expressions can be easily framed in terms of Taylor s sin instead of in terms of sin and π: [ F (x 0,..., x n 1 ) = (n+1) 2 p 2 (x 0,..., x n 1 ) + ] i<n(sin 2x i ) 2 k i (x 0,..., x n 1 ) 1. Now Richardson s Second Theorem and Wang s Theorem depend on two lemmas. Here is the first: The Richardson-Caviness Inequality Lemma. There is an algorithm which given any polynomial p(x 0,..., x n 1 ) with integer coefficients will produce n polynomials k 0 (x 0,..., x n 1 ),..., k n 1 (x 0,..., x n 1 ) also with integer coefficients such that these polynomials exceed 1 when evaluated at any n-tuple of reals, and such that there is a solution of p(x 0,..., x n 1 ) 0 over the integers if and only if there is an n-tuple of real numbers to which the term function denoted by F (x 0,..., x n 1 ) over the reals assigns a negative value. The proofs of this lemma found in [11, 1, 15] all carry over without significant change when framed in terms of sin in place of sin and π. The second lemma is the key to squeezing down to one variable. Call a system e 0, e 1, e 2,... of functions on R a system of approximate decoding functions if and only if given any natural number m, any real numbers χ 0,..., χ m 1, and any > 0, there is a real number η so that e i (η) χ i < for all i < m. ] 1

6 GEORGE F. MCNULTY Richardson s Approximate Decoding Lemma. Let h(x) = x sin x and g(x) = x sin x 3. Then h(g i (x)) i is a natural number = h(x), h(g(x), h(g(g(x)))),... is a system of approximate decoding functions. A very short argument, given by Wang in [15], is needed to deduce Wang s Theorem from the Richardson-Caviness Inequality Lemma and Richardson s Approximate Decoding Lemma. What is needed to prove our Theorem 0 is a modification of Richardson s Approximate Decoding Lemma. 2. How to make a system of approximate decoding functions Lemma. Let h(x) = x sin x and g(x) = x sin x 3. Then e i (x) i is a natural number := h(x), h(g(x), h(g(g(x)))),... is a system of approximate decoding functions. For each i < n we let e i (x) be the obvious term that denotes the corresponding decoding function e i in the algebra R. Proof. The claim below is key to proving the Lemma. Claim. Given any real numbers χ and ψ and any > 0, there is a real number η such that h(η) χ < g(η) = ψ Proof of the Claim. It is harmless to suppose that < 1. Now fix an integer k so large that 6(4k 1) 2 > 4 and 4k 1 > χ and 4k 2 > ψ Consider the closed interval [4k 1, 4k + 1]. Notice sin (4k 1) = sin π 2 (4k 1) = sin(2πk π 2 ) = sin( π 2 ) = 1 sin (4k + 1) = sin π 2 (4k + 1) = sin(2πk + π 2 ) = sin(π 2 ) = 1 So sin (x) maps [4k 1, 4k + 1] onto [ 1, 1]. Hence, the image of the interval [4k 1, 4k + 1] under the map h(x) must include the interval [ 4k + 1, 4k + 1]. This means that there is η 0 in [4k 1, 4k + 1] so that h(η 0 ) = χ, since 4k 1 > χ entails that 4k + 1 < χ < 4k + 1.

TAYLOR S ALGORITHMIC UNSOLVABILITY OF TOPOLOGICAL R-COMPATIBILITY 7 By continuity, h will map any sufficiently small interval about η 0 into the interval about χ of radius. Our argument depends on finding out how small sufficiently small must be. After a bit of reverse engineering, we take δ =. Let ν be any element of [η 0 δ, η 0 + δ]. According to the Mean Value Theorem, we pick ˆη in [η 0 δ, η 0 + δ] so that Now just observe This means h(ν) χ = h(ν) h(η 0 ) h(ν) h(η 0 ) h (ˆη) δ h (ˆη) δ = sin( π 2 ˆη) + π 2 ˆη cos(π 2 ˆη) δ ( 1 + π ) 2 (η 0 + δ) δ ( 1 + π ( )) 4k + 1 + 2 ( 1 + 2kπ + π ) 2 + π (4k + 2)π + 2 ( < 1 + 2kπ + π 2 + π ) 2 = (1 + 2kπ + π) = () = h(ν) χ < for any ν in [η 0 δ, η 0 + δ]. So it remains to find η in [η 0 δ, η 0 + δ] so that g(η) = ψ. Now the cubing function maps [η 0 δ, η 0 + δ] onto [(η 0 δ) 3, (η 0 + δ) 3 ]. Also, observe (η 0 + δ) 3 (η 0 δ) 3 = 6η 2 0δ + 2δ 3 6(4k 1) 2 δ = 6(4k 1) 2 = 6(4k 1)2 > 4 = 4

8 GEORGE F. MCNULTY Therefore, as ν ranges over [η 0 δ, η 0 + δ] we find that ν 3 ranges over an interval of length at least 4. In turn, this means that π 2 ν3 ranges over an interval of length at least 2π. Consequently, as ν ranges over [η 0 δ, η 0 + δ] we conclude that sin (ν 3 ) = sin( π 2 ν3 ) takes on all values between 1 and 1. Then g(ν) = ν sin (ν 3 ) has to take on all values between δ η 0 and η 0 δ. Recall that 4k 1 η 0. Since we have ψ < 4k 2, we know that ψ will lie between δ η 0 and η 0 δ and we can pick η in [η 0 δ, η 0 + δ] so that g(η) = ψ and h(η) χ <, as desired. This completes the proof of the Claim. Our whole line of reasoning in support of the Lemma can now be concluded by a straightforward induction on the natural number m to the effect that for all reals χ 0,..., χ m 1 and every > 0, there is η m so that e i (η m ) χ i < for all i < m. The base step holds vacuously. Here is the inductive step. Suppose reals χ 0,..., χ m are given along with > 0. The inductive hypothesis applied to the system χ 1,..., χ m yields η m so that e i (η m ) χ i < for all i with 1 i m. Use the Claim to obtain η m+1 so that h(η m+1 ) χ 0 < and g(η m+1 ) = η m. It follows that e 0 (η m+1 ) χ 0 = h(η m+1 ) χ 0 < e 1 (η m+1 ) χ 1 = e 0 (g(η m+1 )) χ 1 = e 0 (η m ) χ 1 < e 2 (η m+1 ) χ 2 = e 1 (g(η m+1 )) χ 2 = e 1 (η m ) χ 2 <. e m (η m+1 ) χ m = e m 1 (g(η m+1 )) χ m = e m 1 (η m ) χ m <, which is exactly what we need. We have devised a system of approximate decoding functions from just and sin. This completes the proof of the Lemma. 3. Just one unknown: The proof of Theorem 0 We repeat here Wang s argument, with one small change (as Wang allowed the rational number 1 2 ). Let n be a natural number so that Hilbert s 10 th Problem in n variables over the integers is algorithmically unsolvable. With each polynomial p(x 0,..., x n 1 ) with integer coefficients we will associate a term G(x) in the signature of R so that p(x 0,..., x n 1 ) 0 has a solution in the integers if and only if G(x) 0 has a solution in the real numbers. Moreover, there will be an algorithm which upon input of p(x 0,..., x n 1 ) will output the associated G(x). In this way Hilbert s 10 th Problem in one variable for

TAYLOR S ALGORITHMIC UNSOLVABILITY OF TOPOLOGICAL R-COMPATIBILITY 9 R will be reduced to Hilbert s 10 th Problem in n variables for the ring of integers, and Theorem 0 will be established. First, let H(x 0,..., x n 1 ) be the term [ 2(n + 1) 2 p 2 (x 0,..., x n 1 ) + ] i<n(sin 2x i ) 2 k i (x 0,..., x n 1 ) 1, where the terms k i (x 0,..., x n 1 ) come from the Richardson-Caviness Inequality Lemma. Notice that H(x 0,..., x n 1 ) = 2F (x 0,..., x n 1 )+1, where F (x 0,..., x n 1 ) also comes from the Richardson-Caviness Inequality Lamma. Finally, put G(x) = H(e 0 (x), e 1 (x),..., e n 1 (x)) = 2F (e 0 (x),..., e n 1 (x)) +1. Now it follows from the Lemma by the continuity of H that H(b 0,..., b n 1 ) < 0 for some b 0,..., b n 1 R if and only if H(e 0 (η),..., e n 1 (η)) < 0 for some η R. A similar equivalence prevails with the inequalities going in the other direction. First, suppose that a 0,..., a n 1 are integers so that p(a 0,..., a n 1 ) = 0. Then F (a 0,..., a n 1 ) = 1, which implies that H(a 0,..., a n 1 ) = 1 as well. Consequently, G(µ) is negative for some real number µ. On the other hand it is easy to see that H( π 4,..., π 4 ) must be positive. So G(ν) must be positive for some real number ν, by the Lemma and the continuity of H. By the Intermediate Value Theorem there must be a real number η so that G(η) = 0. Conversely, suppose that G(η) = 0. Then there are real numbers β 0,..., β n 1 so that 0 = 2F (β 0,..., β n 1 ) + 1. This means that F (β 0,..., β n 1 ) is negative. By the Richardson-Caviness Inequality Lemma, we conclude that p(x 0,..., x n 1 ) 0 has a solution in the integers. That s all that needed to be proved. 4. A proof of Theorem 1 Let σ be the signature with operation symbols +,,, 0, 1, sin, cos, λ, c where c is a new constant symbol. We reduce Hilbert s 10 th Problem in one variable for R to the equational compatibility problem for R and σ. According to Theorem 0 the former problem is algorithmically unsolvable, hence the compatibility problem must also be algorithmically unsolvable. Let Γ be any finite set of equations of the signature of R so that x is the only variable to occur in Γ. Let Γ result from Γ by substituting the constant symbol c for the variable x. We contend that Γ has a solution in R if and only if Σ Γ is compatible with R.

10 GEORGE F. MCNULTY The left-to-right implication is immediate from the natural interpretations for the operation symbols. The right-to-left implication is immediate since Σ determines R according to Taylor s Finite Determination Theorem. References [1] B. F. Caviness,On canonical forms and simplification. Journal of the ACM, 17 (1970) 385 396. [2] Martin Davis, Arithmetical problems and recursively enumerable predicates (abstract), Journal of Symbolic Logic, 15 (1950) 77 78. [3] Martin Davis, Arithmetical problems and recursively enumerable predicates, Journal of Symbolic Logic, 18 (1953) 33 41. [4] Martin Davis and Hilary Putnam, Reductions of Hilbert s Tenth Problem. Journal of Symbolic Logic, 23 (1958) 183 187. [5] Martin Davis, Hilary Putnam, and Julia Robinson, The decision problem for exponential Diophantine equations. Annals of Mathematics, Second Series, 74 (1961) 425 436. [6] James P. Jones, Universal Diophantine equations. Journal of Symbolic Logic, 47 (1982) 549 571. [7] Yuri V. Matiyasevič, Enumerable sets are Diophantine. Soviet Mathematics Doklady, 11 (1970) 354 358. [8] Yuri V. Matiyasevič, Diophantine representation of recursively enumerable predicates. In Actes du Congrès International des Mathématiciens (Nice 1970), 1 (1971) 235 238, Gauthier-Villars, Paris. [9] Yuri V. Matiyasevič, Some purely mathematical results inspired by mathematical logic. In Logic, Foundations of Mathematics, and Computability Theory, vol. 1 of Proceedings of the Fifth International congress of Logic, Methodology and Philosophy of Science, (Robert E. Butts and Jaakko Hintikka, eds) (1977) 121 127, D. Reidel Publishing Company, Dordrecht, Holland. [10] Yuri V. Matiyasevich, Hilbert s Tenth Problem, Foundations of Computing Series, MIT Press, Cambridge, Massachusetts, 1993. [11] Daniel Richardson, Some undecidable problems involving elementary functions of a real variable. Journal of Symbolic Logic,33 (1968) 514 520. [12] Julia Robinson, The undecidability of exponential Diophantine equations. Notices of the American Mathematical Society, 7 (1960) p. 75. [13] Walter Taylor, Equations on real intervals, Algebra Universalis, to appear. [14] Alfred Tarski, A decision method for elementary algebra and geometry, University of California Press, Berkeley and Los Angeles, 1951. [15] Paul S. Wang, The undecidability of the existence of zeros of real elementary functions. Journal of the Association of Computing Machines, 21 (1974) 586 589. Department of Mathematics University of South Carolina Columbia, SC 29208 USA E-mail address: mcnulty@math.sc.edu