Real Numbers and Computational Complexity Norbert Müller mueller@uni-trier.de FB IV Informatik Universität Trier Norbert Müller (Trier) Darmstadt, 2012-10-05 1 / 34
Introduction 1 Introduction 2 Computational complexity / efficient algorithms 3 Real vs. discrete computational complexity Norbert Müller (Trier) Darmstadt, 2012-10-05 2 / 34
Introduction Double precision: numbers as basic objects... generalization to R: BSS model (Blum/Shub/Smale): algebraic approach restricted to basic arithmetic comparison like as total functions ERA, exact real arithmetic: approach is algebraic and analytic starting from basic arithmetic comparison like as partial functions extendable via (restricted) operator for limits ERA is already implemented, BSS will remain a theoretical model uniform cost measure not realistic complexity connected to: long integers (Karatsuba, Toom/Cook, Schönhage/Str., Fürer) multiple precision floating point numbers (Brent) Norbert Müller (Trier) Darmstadt, 2012-10-05 3 / 34
Introduction Example for ERA: Solving differential equations Exact trajectory x 1 = ẋ 2 x 2 = ẋ 1 t 0 = 0 x 0 = (1, 0) 0.8 1 0.6 0.4 0.2 0.2 0 0.4 0.6 0.8 1 0 5 10 15 20 25 1 0.8 0.6 0.4 0.2 00.20.40.60.81 data structures for real numbers / sequences / functions... operators for Taylor sum, IVP solving... (t i, x i ) (a n,i ) n N := IVP(t i, x i ) f i := λt. a n,i t n x i+1 := f i (t i+1 ) Norbert Müller (Trier) Darmstadt, 2012-10-05 4 / 34
Introduction (theoretical) computer science computable analysis / complexity on the reals constructive mathematics numerical mathematics TTE: Type-2 theory of effectivity (Weihrauch, Brattka, Hertling, Ko, Rettinger, Ziegler, M.,...) domains (Blanck, Edalat, Escardo, Tucker,...) BSS-Modell Real RAM (Blum, Shub, Smale, Meer, Gassner...) Norbert Müller (Trier) Darmstadt, 2012-10-05 5 / 34
Introduction Models of computation for functions f : S S for non-countable S: S 1 = Σ Type-2-Turing machine infinite input tape, output tape write-once, infinite computations prefix of the output is determined by prefix of the input S 2 = [Σ Σ ] oracle Turing machine M with functions as oracles fixing oracle Φ S2, M Φ computes Γ(Φ) S 2 : input output { oracle Φ S 2 }} { [ }{{} Σ }{{} Σ ] queries answers Γ(Φ) S 2 { }} { [ }{{} Σ ] }{{} Σ ] input output machine queries answers oracle Norbert Müller (Trier) Darmstadt, 2012-10-05 6 / 34
Introduction From Σ to R using Type-2-Turing machines: use e.g. infinite decimal strings w for input and output w 1 = 1.41421356237... w 2 = 3.14159265359... w 3 = 2.71828182846... better: signed digits { 9, 8, 7,..., 1, 0, 1,..., 8, 9} alternatively use sequences w of finite decimals with delimiter w 4 = 1#1.4#1.41#1.414#1.4142#... Generalization for metric spaces (M, d): use a countable dense subset D in M and encode fast converging Cauchy sequences in D... Norbert Müller (Trier) Darmstadt, 2012-10-05 7 / 34
Introduction From S 2 = [Σ Σ ] to R using oracle Turing machines: interpret S = [Σ Σ ] as [N Q]: Φ S Γ(Φ) S { }} { { }} { [ }{{} N queries Q }{{} answers ] [ }{{} N input Q }{{} output ] i.e. Φ is a sequence (q n ) n N and Γ(Φ) is a sequence (q n ) n N n N n N f x q n Q q n Q a real function f is computed as follows: if x R is given by (qn ) n N with ( n) x q n 2 n then (q n ) must satisfy ( n) f (x) q n 2 n Norbert Müller (Trier) Darmstadt, 2012-10-05 8 / 34
Introduction Computable analysis (via representations ): R ϱ f 1 f 2 R ϱ R ϱ Q N Γ Q N 1 Γ Q N 2 Remember: Computable functions are continuous! Norbert Müller (Trier) Darmstadt, 2012-10-05 9 / 34
Computational complexity / efficient algorithms 1 Introduction 2 Computational complexity / efficient algorithms 3 Real vs. discrete computational complexity Norbert Müller (Trier) Darmstadt, 2012-10-05 10 / 34
Computational complexity / efficient algorithms Computational complexity t G (n) on a (compact) set G: Type-2-machines M: worst case (on strings w for x G) t G (n) = for the number of steps of M until the output is determined up to a precision 2 n oracle Turing { machines M: worst case (on all oracles Φ for a x G) t G (n) = for the number of steps of M for input n Results (almost) independent of machine model! addition, subtraction: computable in linear time multiplication: computable in time M(n) := n log(n)2 θ(log (n)) Norbert Müller (Trier) Darmstadt, 2012-10-05 11 / 34
Computational complexity / efficient algorithms division, rational functions, x q (with q Q): (using fix point algorithms...) sin(x), cos(x), e x : (using Taylor series) O(M(n)) O(M(n) n/ log n) π, e, ln(2), sin(x), cos(x), ln(x), e x,...: O(M(n) log n) (using arithmetic/geometric mean a = a+b 2 ; b = a b ) Norbert Müller (Trier) Darmstadt, 2012-10-05 12 / 34
Computational complexity / efficient algorithms important aspect: modulus of computation, lookahead, delay to determine output with precision n, we need to know input with precision d(n) Example: substitution of functions g and f : t g f (n) = O( t g (n) + t f (d g (n)) ) d g f (n) = d f (d g (n)) f with Lipschitz constant L delay d f (n) n + log 2 L possible Norbert Müller (Trier) Darmstadt, 2012-10-05 13 / 34
Computational complexity / efficient algorithms A complexity bound t : N N is called regular, if eventually t is monotonic increasing with then: t is at least linear: id O(t) 2 t(n) t(2n) β t(n) t is polynomially bounded: k t O(n k ) important closure properties: linear increase: sparse sums: t(c n) O(t) log 2 n k=0 t(2 k ) O(t) Norbert Müller (Trier) Darmstadt, 2012-10-05 14 / 34
Computational complexity / efficient algorithms example: t(n) = n α (log n) β (log log n) γ for arbitrary α 1, β, γ 0, α, β, γ R are regular! bounds now implicitly regular (hence polynomial)! Norbert Müller (Trier) Darmstadt, 2012-10-05 15 / 34
Computational complexity / efficient algorithms Important application: computation of fixpoints given f with fixpoint ξ = f (ξ) use iteration x i+1 f (x i ) starting in some x 0 lim x i = ξ if f appropriate f(x) Norbert Müller (Trier) Darmstadt, 2012-10-05 16 / 34
Computational complexity / efficient algorithms Case (1), f converges linear to ξ with L < 1 ξ x i L ξ x i 1 L i ξ x 0 To get x i with precision 2 n, we have i n/ log 2 (1/L) complexity n t f (i + c) i=1 ξ has complexity O(n t f (n)) f(x) Norbert Müller (Trier) Darmstadt, 2012-10-05 17 / 34
Computational complexity / efficient algorithms Case (2), f converges superlinear to ξ with α > 1: ξ x i L ξ x i 1 α c ξ x 0 α 2i To get x i with precision 2 n, we have i log α n complexity ξ has complexity O(t f (n)) log α n i=1 t f (α i + c) f(x) Norbert Müller (Trier) Darmstadt, 2012-10-05 18 / 34
Computational complexity / efficient algorithms Applications: f (x) = 2 x + a x 2 mit ξ = 1 a division is in O(M) mit ξ = a square root is in O(M) f (x) = x + a/x 2 Norbert Müller (Trier) Darmstadt, 2012-10-05 19 / 34
Computational complexity / efficient algorithms Application: Differentiation suppose f twice continuously differentiable f f (x + h) f (x) (x) h = f (x) f (ξ) = f (θ) (x ξ) f (θ) h choose h 2 2n f has complexity O(t f (n)). Norbert Müller (Trier) Darmstadt, 2012-10-05 20 / 34
Computational complexity / efficient algorithms Applications: inverse function, f f 1 (if f (x) 0 on interval [a, b] ): f 1 (a) is fixpoint of x f (x) a f (x) f 1 has complexity O(t f (n)) root finding, f ξ, f (ξ) = 0 (with simple root ξ) ξ = f 1 (0) root ξ has complexity O(t f (n)) numerical integration, f f f holomorphic on intervall [a, b] integral x a f (y)dy has complexity O(n2 t f (n)) Norbert Müller (Trier) Darmstadt, 2012-10-05 21 / 34
Real vs. discrete computational complexity 1 Introduction 2 Computational complexity / efficient algorithms 3 Real vs. discrete computational complexity Norbert Müller (Trier) Darmstadt, 2012-10-05 22 / 34
Real vs. discrete computational complexity Consider functionals like derivative : (f, x) f (x) integral : (f, x, y) y x f (t)dt maximum : (f, x, y) max{f (t) x t y} maxarg : (f, x, y) z with f (z) = maximum(f, x, y)} or operators like derive : f f integrate : f (y integral(f, 0, y)) maximize : f (y maximum(f, 0, y)) solve : f u with u (x) = f (x, u(x)), u(0) = 0 Norbert Müller (Trier) Darmstadt, 2012-10-05 23 / 34
Real vs. discrete computational complexity [Ko et al.]: non-uniform negative results like (grossly shortened...) f P+C[0, 1] with derive(f )(q) not computable for all q Q P = NP iff maximize(f ) is in P+C[0, 1] for all f P+C[0, 1] #P = FP iff integrate(f ) is in P+C[0, 1] for all f P+C[0, 1] P = PSPACE iff solve(f ) is in P+C[0, 1] for all f P+C[[0, 1] [ 1, 1]] Norbert Müller (Trier) Darmstadt, 2012-10-05 24 / 34
Consider Real vs. discrete computational complexity g a (x) = e 1 x(x 1) g b (x) = x 0 e 1 y(y 1) dy 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 g a, g b P+C [0, 1] and for all i > 0 g (i) a g a (0) = g a (1) = g b (0) = 0 g b (1) 0.073 > 0 (0) = g (i) a (1) = g (i) b (0) = g(i) b (1) = 0 Norbert Müller (Trier) Darmstadt, 2012-10-05 25 / 34
Real vs. discrete computational complexity Every computable ξ R is root of a strictly monotonic f P+C. in general: complexity of f does not tell anything about complexity of ξ! Norbert Müller (Trier) Darmstadt, 2012-10-05 26 / 34
Real vs. discrete computational complexity P = NP max 0 y 1 f (y) polytime for any f P+C...v00000......v100...000......v100...100......v100...010......v100...110......v1w0......v111...110... f L (...v0u) = c 1 (v) + c 2 (v) 1 2 g a(u) f L (...v1wu) = c 1 (v) + c 2 (v) g a (u), if w is a witness for v L f L (...v1wu) = c 1 (v), if w is not a witness for v L Norbert Müller (Trier) Darmstadt, 2012-10-05 27 / 34
Real vs. discrete computational complexity x 0 FP = #P f (y)dy polytime for any f P+C...v100...000......v100...100......v100...010......v100...110......v1w0......v111...110... f L (...v1wu) = c 1 (v) + c 2 (v) g a (u), if w is a witness for v L f L (...v1wu) = c 1 (v), if w is a witness for v L Norbert Müller (Trier) Darmstadt, 2012-10-05 28 / 34
Real vs. discrete computational complexity [M. et al]: non-uniform positive results like Let f be holomorphic on compact G and computable in polynomial time t(n) then derive(f ) is computable in time t(n) integrate(f ) is computable in time n 2 t(n) solve(f ) is computable in time n 2 t(n 2 ) Norbert Müller (Trier) Darmstadt, 2012-10-05 29 / 34
Real vs. discrete computational complexity suppose f holomorphic, f (x) = a k x k f is completely determined by arbitrary infinite set of values basic operators: f (a k ) and (a k ) f map polytime argumtents to polytime results for many real operators Φ the complexity of Φ(f ) is comparable to f s complexity! integration, differentiation, differential equations... via formal operations on coeffizients Norbert Müller (Trier) Darmstadt, 2012-10-05 30 / 34
Real vs. discrete computational complexity standard TTE approach for definition of complexity: use a metric/norm on functions spaces, (e.g. for C[0, 1]) use some dense subset D, (e.g. rational polygons, rational polynomials, box lists...) represent functions as limits of sequences in D ( f = lim p n for polygons/polynomials/boxes (p n ) n N ) usual condition for speed of convergence: f p n 2 n approximations are valid everywhere, but time complexity is exponential... 1.2 1 0.8 0.6 0.4 0.2 0 sqrt(x) 3-bit, 50 boxes -0.2 5-bit, 200 boxes 0 0.2 0.4 0.6 0.8 1 Norbert Müller (Trier) Darmstadt, 2012-10-05 31 / 34
Real vs. discrete computational complexity usual applications only need local approximations to functions! new computational model: Instead of n p n for approximating f with use (x, n) p n (x) with f p n 2 n (i.e. p n is a complete list of boxes f ) f (x) p n (x) 2 n (i.e. just one single box ) idea e.g. used in: Zhao/M. [CCA 2008]: Complexity of operators on compact sets M. [CCA 2009]: Enhancing imperative exact real arithmetic... Kawamura/Cook [STOC 2010]: Complexity theory for operators in analysis Norbert Müller (Trier) Darmstadt, 2012-10-05 32 / 34
Real vs. discrete computational complexity Kawamura/Cook: real functions as argument/result of an operator Γ: Φ 1,2 S { }} { [ (Q N) N }{{} Q }{{} ] queries/input answers/output Φ1 : (Q N) Q approximating values: f (q) Φ 1 (q, n) 2 n Φ 2 : N Q bounding the global modulus: x y Φ 2 (n) f (x) f (y) 2 n (q, n ) Q N n N (q, n) Q N n N form of results (here shortened and without explanation...): solve is FPSPACE 2-2 mf -complete Γ f Φ 1 (n, q ) Q Φ 2 (n ) Q Φ 1(n, q) Q Φ 2(n) Q until now: no definition for finer distinctions (eg. linear complexity) Norbert Müller (Trier) Darmstadt, 2012-10-05 33 / 34
Real vs. discrete computational complexity Thank you for your attention! Any questions or remarks? Norbert Müller (Trier) Darmstadt, 2012-10-05 34 / 34