Dynamics and Computation

1 Dynamics and Computation Olivier Bournez Ecole Polytechnique Laboratoire d Informatique de l X Palaiseau, France Dynamics And Computation

1 Dynamics and Computation over R Olivier Bournez Ecole Polytechnique Laboratoire d Informatique de l X Palaiseau, France Dynamics And Computation

1 Dynamics and Computation over R d Olivier Bournez Ecole Polytechnique Laboratoire d Informatique de l X Palaiseau, France Dynamics And Computation

1 Dynamics and Computation over R d Several View Points Olivier Bournez Ecole Polytechnique Laboratoire d Informatique de l X Palaiseau, France Dynamics And Computation

2 Menu Introduction Some Very Basic Models Recursive Analysis The General Purpose Analog Computer (GPAC) of C. Shannon

3 Sub-menu Introduction A question Motivation 1: Models of Computation Motivation 2: Effectivity in Analysis Motivation 3: Algebraic Complexity Motivation 4: Verification/Control

Consider you preferred function f : R R. 4

4 Consider you preferred function f : R R. Is f computable?

4 Consider you preferred function f : R R. Is f computable? Several notions of computability for real functions

4 Consider you preferred function f : R R. Is f computable? Several notions of computability for real functions Turing machine approach: Recursive Analysis.

4 Consider you preferred function f : R R. Is f computable? Several notions of computability for real functions Turing machine approach: Recursive Analysis. Continuous time analog models

4 Consider you preferred function f : R R. Is f computable? Several notions of computability for real functions Turing machine approach: Recursive Analysis. Continuous time analog models Blum Shub Smale machines

4 Consider you preferred function f : R R. Is f computable? Several notions of computability for real functions Turing machine approach: Recursive Analysis. Continuous time analog models Blum Shub Smale machines...

4 Consider you preferred function f : R R. Is f computable? Several notions of computability for real functions with various motivations:

4 Consider you preferred function f : R R. Is f computable? Several notions of computability for real functions with various motivations: computability theory

4 Consider you preferred function f : R R. Is f computable? Several notions of computability for real functions with various motivations: computability theory lower bounds / upper bounds

4 Consider you preferred function f : R R. Is f computable? Several notions of computability for real functions with various motivations: computability theory lower bounds / upper bounds verification control theory

4 Consider you preferred function f : R R. Is f computable? Several notions of computability for real functions with various motivations: computability theory lower bounds / upper bounds verification control theory...

5 Sub-menu Introduction A question Motivation 1: Models of Computation Motivation 2: Effectivity in Analysis Motivation 3: Algebraic Complexity Motivation 4: Verification/Control

6 Motivation 1: Models of Computation NACA Lewis Flight Propulsion Laboratory s Differential Analyser Question: What is the computational power of this machine?

7 Sub-menu Introduction A question Motivation 1: Models of Computation Motivation 2: Effectivity in Analysis Motivation 3: Algebraic Complexity Motivation 4: Verification/Control

8 Motivation 2: Effectivity in Analysis Question: Can we compute the maximum of a continuous function over a compact domain? A point on which it is maximal?

9 Sub-menu Introduction A question Motivation 1: Models of Computation Motivation 2: Effectivity in Analysis Motivation 3: Algebraic Complexity Motivation 4: Verification/Control

10 Motivation 3: Algebraic Complexity Question: What is the complexity of Newton s method?

11 Sub-menu Introduction A question Motivation 1: Models of Computation Motivation 2: Effectivity in Analysis Motivation 3: Algebraic Complexity Motivation 4: Verification/Control

12 Motivation 4: Verification/Control Model M made of a mixture of continuous/discrete parts. Specification φ (e.g. reachability property). Informal question: Can we avoid that? Formal question: M = φ?

13 Menu Introduction Some Very Basic Models Recursive Analysis The General Purpose Analog Computer (GPAC) of C. Shannon

Some Very Basic Models Dynamical Systems Turing Machines Sub-menu

Dynamical Systems over a continuous space H = (X, f ) Discrete-Time { y(t + 1) = f (y(t)) y(0) = x Continuous-Time { y = f (y(t)) y(0) = x

Some Very Basic Models Dynamical Systems Turing Machines Sub-menu

17 Turing machines Particular discrete-time dynamical systems. Elementary operations 1. move some head on working tape one position to the left or right 2. move some head on input tape or output tape right 3. write a symbol a Σ on the position under the head 4. compare the symbol under some head with the symbol a Σ

18 Menu Introduction Some Very Basic Models Recursive Analysis The General Purpose Analog Computer (GPAC) of C. Shannon

Sub-menu Recursive Analysis Computable Reals Computable Functions Over the Real Numbers Complexity

The Starting Point of Recursive Analysis The computable numbers may be described briefly as the real numbers whose expressions as a decimal are calculable by a finite means. Cited from A. M. Turing, On computable numbers, with an application to the Entscheidungsproblem. Proc. London Math. Soc. 42 (1936) 230-265. This part is mostly borrowed from Vasko Brattka s Tutorial, CIE 2005 20

21 The Field of Computable Real Numbers Definition A real number is computable if there exists a Turing machine (without input) which can produce its decimal expansion. Theorem (Rice54) The set R c of computable reals is a real algebraically closed field. The field R c contains all particular real numbers we ever met in analysis (e.g. 2, π, e, etc.). Q A R c R (A denotes the set of algebraic numbers). R c is countable. R c is not complete. In the Russian school (ex. Markov, Sanin, Kushner, Aberth) on focus on computability of functions f : R c R c.

Characterisations of computable real numbers Theorem Let x [0, 1]. The following are equivalent x is a computable real number. there exists a computable sequence of rational numbers (q n ) n N which converges rapidly to x, i.e q i x < 2 i. {q Q : q < x} is a recursive set. there exists a recursive set A N such that x = i A 2 i 1. x admits a computable continued fraction expansion, i.e. there is some computable f : N N such that x = 1 f (0) + 1 f (1)+ 1 f (2)+ 1 f (3)+.

23 We sketch the proof of {q Q : q < x} is a recursive set iff (q n ) n N with q i x < 2 i.. For each i determine some q i Q with q i < x and not q i + 2 i 1 < x. These properties are decidable and a suitable i always exists. It follows that q i < x q i + 2 i 1 and thus q i x < 2 i.. 1. Case x Q. It is easy to decide q < x for a given q Q. 2. Case x Q. For given q Q, we determine some i N such that q i x < 2 i and q i q > 2 i. Such an i exists because x Q. Then q < x iff q < q i, which is decidable. Remark: the proof of contains a non-constructive case distinction that cannot be removed!!

Sub-menu Recursive Analysis Computable Reals Computable Functions Over the Real Numbers Complexity

25 Recursive Analysis: Naive Idea Due to Turing, Grzegorczyk, Lacombe. Here presentation from Weihrauch. A tape represents a real number Each real number x is represented via its decimal representation M behaves like a Turing Machine Read-only one-way input tapes Write-only one-way output tape. M outputs a representation of f (x 1, x 2 ) from representations of x 1, x 2.

Theorem The function f : R R, x 3x is not computable with respect to the decimal representation. Main problem: distinguishing 1/3 = 0.3 ω from 0.3.

Recursive Analysis Due to Turing, Grzegorczyk, Lacombe. Here presentation from Weihrauch. A tape represents a real number Each real number x is represented via an infinite sequence (x n ) n Q, x n x 2 n. M behaves like a Turing Machine Read-only one-way input tapes Write-only one-way output tape. M outputs a representation of f (x 1, x 2 ) from representations of x 1, x 2.

Computable Reals A sequence (x n ) n Q of rationals is a ρ-name of a real number x if there exists three functions a, b, c from N to N such that for all n N, x n = ( 1) a(n) b(n) c(n) + 1, and x n x 2 n.

Computable Reals A sequence (x n ) n Q of rationals is a ρ-name of a real number x if there exists three functions a, b, c from N to N such that for all n N, x n = ( 1) a(n) b(n) c(n) + 1, and x n x 2 n. x R is computable if it has a computable ρ-name: i.e. if we can take a, b, and c computable.

Computable Reals A sequence (x n ) n Q of rationals is a ρ-name of a real number x if there exists three functions a, b, c from N to N such that for all n N, x n = ( 1) a(n) b(n) c(n) + 1, and x n x 2 n. x R is computable if it has a computable ρ-name: i.e. if we can take a, b, and c computable. A function f : R R is computable, if for all x, a Turing machine taking a ρ-name of x, produces a ρ-name of f (x).

Computable Reals A sequence (x n ) n Q of rationals is a ρ-name of a real number x if there exists three functions a, b, c from N to N such that for all n N, x n = ( 1) a(n) b(n) c(n) + 1, and x n x 2 n. x R is computable if it has a computable ρ-name: i.e. if we can take a, b, and c computable. A function f : R R is computable, if for all x, a Turing machine taking a ρ-name of x, produces a ρ-name of f (x). There are Turing machines M a,b,c a, M a,b,c b and M a,b,c c that computes a, b and c representing f (x) from representations of x.

29 Computable functions Theorem The following functions are computable The arithmetic operations +,,., : R R R. The absolute value function abs : R R, x x. The functions min, max : R R R. The constant functions R R, x c with c R c. The projections pr i : R n R, (x 1,, x n ) x i. All polynomials p : R n R with computable coefficients. The exponential function and the trigonometric functions exp, sin, cos : R R. The square root function and the logarithm function log on their natural domains.

Some Results Theorem If f : R R is computable then f : R R is continuous Some results: IVT: There is no general algorithm that determines the 0 of a continuous function f : [0, 1] R with f (0)f (1) < 0. Brower FPT: There exists some computable f : [0, 1] 2 [0, 1] 2 without computable fix point [Orevkov63]. There exists a computable f : [0, 1] R of class C 1, whose derivative is not computable [Myhill71]. There exists some computable f : [0, 1] [ 1, 1] R such that y = f (t, y), has no computable solution over any closed domain [PE-R79].

Sub-menu Recursive Analysis Computable Reals Computable Functions Over the Real Numbers Complexity

Complexity This is possible to define a notion of polynomial time computable real number and of polynomial time computable function [Ko91]

Complexity This is possible to define a notion of polynomial time computable real number and of polynomial time computable function [Ko91] One must change the representation (signed digit representation). x = ρ(a n a n 1 a 0.a 1 a 2 ) = n a i 2 i, i=1 where a i { 1, 0, 1}, and n 1 (with the additional property that a n 0, if n 0, and a n a n 1 {1 1, 11}, if nge1).

Complexity This is possible to define a notion of polynomial time computable real number and of polynomial time computable function [Ko91] One must change the representation (signed digit representation). x = ρ(a n a n 1 a 0.a 1 a 2 ) = n a i 2 i, i=1 where a i { 1, 0, 1}, and n 1 (with the additional property that a n 0, if n 0, and a n a n 1 {1 1, 11}, if nge1). In constrast to the binary representation, the signed digit represenation is redundant in a symmetric way.

Complexity This is possible to define a notion of polynomial time computable real number and of polynomial time computable function [Ko91] One must change the representation (signed digit representation). x = ρ(a n a n 1 a 0.a 1 a 2 ) = n a i 2 i, i=1 where a i { 1, 0, 1}, and n 1 (with the additional property that a n 0, if n 0, and a n a n 1 {1 1, 11}, if nge1). In constrast to the binary representation, the signed digit represenation is redundant in a symmetric way. The idea is that one requires that bit number n must be produced in time p(n) for some fixed polynomial p.

33 Some results Theorem (Friedman84) P = NP iff for all function f : [0, 1] R polynomial time computable, the maximum function g : [0, 1] R defined by g(x) = max{f (y) : 0 y x} for all x [0, 1] is polynomial time computable. Theorem (Ko83) P = PSPACE implies that for all function f : [0, 1] [ 1, 1] R that satisfies some Lipschitz condition f (x, z 1 ) f (x, z 2 ) L z 1 z 2 for L > 0, the unique solution y : [0, 1] R of ordinary differential equation y = f (x, y(x)), y(0) = 0 is polynomial time computable.

Numerical Problems vs Complexity Rough classification [Ko91] Numerical Problem Optimisation (functions) Ordinary Differential Equations Integration Optimisation (numbers) Roots (dim 1) Differentiation Complexity Class EXPSPACE-complete PSPACE-complete #P-complete NP-complete P-complete P (if computable) From slides of V. Brattka 34

35 Futher Readings Vasko Brattka, Computability and Complexity in Analysis, CIE 2005 Tutorial, online. Klaus Weihrauch, Computability and Complexity in Analysis, Springer, 2000. Ker-I Ko, Complexity Theory of Real Functions, Birkhäuser, 1991. Marian Pour-El and J. Ian Richards, Computability in Analysis and Physics, Springer 1989.

36 Menu Introduction Some Very Basic Models Recursive Analysis The General Purpose Analog Computer (GPAC) of C. Shannon

The General Purpose Analog Computer The GPAC A mathematical abstraction from Claude Shannon (1941) of the Differential Analyzers.

Sub-menu The General Purpose Analog Computer (GPAC) of C. Shannon Differential Analysers The GPAC Model

Differential analyzers Vannevar Bush s 1938 mechanical Differential Analyser Underlying principles: Lord Kelvin 1876. First ever built: V. Bush 1931 at MIT. Applications: from gunfire control up to aircraft design Intensively used during U.S. war effort. Electronic versions from late 40s, used until 70s

40 A Mechanical Integrator Bureau of Naval Personnel, Basic Machines and How They Work, 1964

A Differential Analyser in Mecano Douglas Rayner Hartree, 1933 Tim Robinson, 2004 41

Electronic Differential Analyzer Advertisements in Scientific American, March 1953.

Electronic Differential Analyzer Advertisements in Scientific American, March 1953. See also: Doug Coward s Analog Computer Museum http://dcoward.best.vwh.net/analog/

43 A Modern Electronic Integrator R C U - + V V (t) = 1/RC t 0 U(t)dt

Sub-menu The General Purpose Analog Computer (GPAC) of C. Shannon Differential Analysers The GPAC Model

Basic units of a GPAC A GPAC is a machine built using basic units k k A constant unit u v + u + v An adder unit u v { w (t) = u(t)v (t) w w(t 0 ) = α An integrator unit u v uv A multiplier unit Feedback connections are allowed. A function is GPAC-generated if it corresponds to the output of some unit of a GPAC.

Example: Generating cos and sin via a GPAC t -1 y 1 y 2 y 3 y 1 = y 3 & y 1 (0) = 1 y 2 = y 1 & y 2 (0) = 0 y 3 = y 2 & y 3 (0) = 0 y 1 = cos(t), y 2 = sin(t), y 3 = sin(t).

About Shannon s 41 paper Shannon s 41 characterization is incomplete. several problems about definitions, corrected by [PourEl-Richards74], [Lipshitz-Rubel87], [Graça-Costa03].

About Shannon s 41 paper Shannon s 41 characterization is incomplete. several problems about definitions, corrected by [PourEl-Richards74], [Lipshitz-Rubel87], [Graça-Costa03]. In short Shannon assumes that considered circuits have an output and that this output is unique. The result relating the GPAC with d.a. functions has a gap.

About Shannon s 41 paper Shannon s 41 characterization is incomplete. several problems about definitions, corrected by [PourEl-Richards74], [Lipshitz-Rubel87], [Graça-Costa03]. In short Shannon assumes that considered circuits have an output and that this output is unique. The result relating the GPAC with d.a. functions has a gap. 1 Figure 1: A circuit that admits no solutions as outputs. + 1 + y + i t Figure 2: A circuit that admits two distinct solutions as outputs.

48 Question How computability with GPAC compare to classical computability?