Probabilistic Recursion Theory and Implicit Computational Complexity

Transcription

1 Probabilistic Recursion Theory and Implicit Computational Complexity Sara Zuppiroli (Work with Ugo Dal Lago) ICTAC, Bucharest, September 17th 2014

2 Deterministic Machines M

3 Deterministic Machines x Σ M

4 Deterministic Machines x Σ M f(x) Σ

5 Deterministic Machines x Σ M

6 Probabilistic Machines x Σ M

7 Probabilistic Machines x Σ M D : Σ R

8 Probabilistic Machines x Σ M D : Σ R { P Σ = D. D = } D(s) 1. s Σ

9 Church-Kleene Characterization C

10 Church-Kleene Characterization C Turing Machines

11 Church-Kleene Characterization C R Turing Machines

12 Church-Kleene Characterization C Turing Machines R Basic Functions

13 Church-Kleene Characterization C Turing Machines R Basic Functions Composition

14 Church-Kleene Characterization C Turing Machines R Basic Functions Composition Primitive Recursion

15 Church-Kleene Characterization C Turing Machines R Basic Functions Composition Primitive Recursion Minimization

16 Church-Kleene Characterization C Turing Machines = R Basic Functions Composition Primitive Recursion Minimization

17 Church-Kleene Characterization PC

18 Church-Kleene Characterization PC Probabilistic Turing Machines

19 Church-Kleene Characterization PC? Probabilistic Turing Machines

20 Related Work Probability and Machines [DeLeeuwMooreShannonShapiro1956]; Probability and Automata [Rabin1963]; Probabilistic Turing Machines [Santos1969, Mann1973]; Probabilistic Turing Machines and Computational Complexity [Gill1977];...

21 Related Work Probability and Machines [DeLeeuwMooreShannonShapiro1956]; Probability and Automata [Rabin1963]; Probabilistic Turing Machines [Santos1969, Mann1973]; Probabilistic Turing Machines and Computational Complexity [Gill1977];... Emphasis: Reductionism

22 Probabilistic Recursive Functions: General Form f : N... N }{{} n times P N

23 Probabilistic Recursive Functions: General Form f : N n P N

24 Probabilistic Recursive Functions: Base Functions The Zero function z : N P N defined as z(n)(0) = 1 for every n N;

25 Probabilistic Recursive Functions: Base Functions The Zero function z : N P N defined as z(n)(0) = 1 for every n N; The Successor function s : N P N defined as: s(n)(n + 1) = 1 for every n N;

26 Probabilistic Recursive Functions: Base Functions The Zero function z : N P N defined as z(n)(0) = 1 for every n N; The Successor function s : N P N defined as: s(n)(n + 1) = 1 for every n N; Projection functions Π n m : N n P N defined as: Π n m(k 1,, k n )(k m ) = 1 for every positive n, m N such that 1 m n;

27 Probabilistic Recursive Functions: Base Functions The Zero function z : N P N defined as z(n)(0) = 1 for every n N; The Successor function s : N P N defined as: s(n)(n + 1) = 1 for every n N; Projection functions Π n m : N n P N defined as: Π n m(k 1,, k n )(k m ) = 1 for every positive n, m N such that 1 m n; The Random function r : N P N defined as: { 1/2 if y = x r(x)(y) = 1/2 if y = x + 1

28 Probabilistic Recursive Functions: Composition Given f : N P N and g : N P N, the composition f g : N P N is defined as ((f g)(x))(y) = z N g(x)(z) f(z)(y).

29 Probabilistic Recursive Functions: Composition Given f : N P N and g : N P N, the composition f g : N P N is defined as ((f g)(x))(y) = z N g(x)(z) f(z)(y). We are lifting f to a function from P N to P N, then composing it to g.

30 Probabilistic Recursive Functions: Composition Given f : N P N and g : N P N, the composition f g : N P N is defined as ((f g)(x))(y) = z N g(x)(z) f(z)(y). We are lifting f to a function from P N to P N, then composing it to g. Can be easily generalized to n-ary functions. Primitive recursion is just iterated composition, as usual.

31 Probabilsitic Recursive Functions: Minimization If f : N 2 N is a deterministic (say, total) function, the minimization of f is a unary, partial, function from N to N which on x returns the least y such that f(x, y) = 0, if such a y exists.

32 Probabilsitic Recursive Functions: Minimization If f : N 2 N is a deterministic (say, total) function, the minimization of f is a unary, partial, function from N to N which on x returns the least y such that f(x, y) = 0, if such a y exists. But if f is a probabilistic function, f(x, y) is a distribution which can assign 0 any probability between 0 and 1.

33 Probabilsitic Recursive Functions: Minimization If f : N 2 N is a deterministic (say, total) function, the minimization of f is a unary, partial, function from N to N which on x returns the least y such that f(x, y) = 0, if such a y exists. But if f is a probabilistic function, f(x, y) is a distribution which can assign 0 any probability between 0 and 1. How should we proceed?

34 Probabilsitic Recursive Functions: Minimization If f : N 2 N is a deterministic (say, total) function, the minimization of f is a unary, partial, function from N to N which on x returns the least y such that f(x, y) = 0, if such a y exists. But if f is a probabilistic function, f(x, y) is a distribution which can assign 0 any probability between 0 and 1. How should we proceed? Given a f : N k+1 P N, the probabilistic function µf : N k P N, the minimization of f, is defined as follows: ( ) µf(x)(y) = f(x, y)(0) f(x, z)(k). z<y k>0

35 Back to Church-Kleene PC Probabilistic Turing Machines PR Basic Functions Composition Primitive Recursion Minimization

36 PC = PR The inclusion PR PC is relatively easy.

37 PC = PR The inclusion PR PC is relatively easy. The usual proof-strategy works well. Roughly, all Kleene s programming primitives can be simulated by Turing Machines

38 PC = PR The inclusion PR PC is relatively easy. The usual proof-strategy works well. Roughly, all Kleene s programming primitives can be simulated by Turing Machines Proving the inclusion PC PR is more subtle.

39 PC = PR The inclusion PR PC is relatively easy. The usual proof-strategy works well. Roughly, all Kleene s programming primitives can be simulated by Turing Machines Proving the inclusion PC PR is more subtle. In the deterministic case, x N #STEPS M SIM M M(x) N

40 PC = PR The inclusion PR PC is relatively easy. The usual proof-strategy works well. Roughly, all Kleene s programming primitives can be simulated by Turing Machines Proving the inclusion PC PR is more subtle. In the deterministic case, x N #STEPS M SIM M M(x) N This does not in presence of probability: the number of required computation steps can be infinite, even if the result is a distribution summing to 1.

41 PC = PR The inclusion PR PC is relatively easy. The usual proof-strategy works well. Roughly, all Kleene s programming primitives can be simulated by Turing Machines Proving the inclusion PC PR is more subtle. In the deterministic case, x N #STEPS M SIM M M(x) N This does not in presence of probability: the number of required computation steps can be infinite, even if the result is a distribution summing to 1. Solution: work with computation trees, outputting each leaf in them with its own probability.

42 PC = PR x N BRANCHES M SIM M M(x) P N BRANCHES M is not just the minimization of a primitive recursive function! Kleene s normal form theorem does not follow!

43 Complexity A family of distributions {D i } i N is polytime samplable if D n is the output distribution of a polytime probabilistic algorithm on input 1 n.

44 Complexity A family of distributions {D i } i N is polytime samplable if D n is the output distribution of a polytime probabilistic algorithm on input 1 n. Can we characterize polytime samplable distributions by a (sub)algebra of PR? Implicit complexity offers some hints about characterizing deterministic polynomial time: Function algebra [BC92,Leivant93]; Type systems [LeivantMarion93,Hofmann98]; Subsystems of linear logic [Girard97,Lafont04].

45 Leivant s Tiering ε W k r W k W k c i W k W k Π k n W n1 W nn W nk {g i W s1 W sr W mi } 1 i l f W m1 W mp W l f (g 1,..., g l ) W s1 W sr W l g ε W W l {g a W k W W l } a Σ case(g ε, {g a} a Σ) W k W W l g ε W W k m > k {g a W k W m W W k } a Σ rec(g ɛ, {g a} a Σ) W m W W k

46 PPC = PT The inclusion PT PPC :

47 PPC = PT The inclusion PT PPC : All predicatively probabilistic recursive functions can be simulated by Turing Machines in polinomial time

48 PPC = PT The inclusion PT PPC : All predicatively probabilistic recursive functions can be simulated by Turing Machines in polinomial time Proving the inclusion PPC PT : x N POLY M SIM M M(x) P N Defining functions able to simulate a polytime PTM.

49 Tiering vs. Probabilistic Polynomial Time Theorem Tiering characterizes polytime computable probabilistic functions, thus polytime samplable distributions.

50 Conclusions A fresh look at (complexity bounded) probabilistic computation, getting rid of reductionism.

51 Conclusions A fresh look at (complexity bounded) probabilistic computation, getting rid of reductionism. The natural extension of Kleene s scheme works well, but the proof of completeness is harder. Known concepts can be characterized naturally, as special cases of our constructions: (Polytime) Samplable Distributions; Computable Real Numbers. Computable Distributions.

52 Conclusions A fresh look at (complexity bounded) probabilistic computation, getting rid of reductionism. The natural extension of Kleene s scheme works well, but the proof of completeness is harder. Known concepts can be characterized naturally, as special cases of our constructions: (Polytime) Samplable Distributions; Computable Real Numbers. Computable Distributions. Future Works. Kleene s recursion theorems? Quantum computability?

53 Thank you! Questions?