Solomonoff Induction Violates Nicod s Criterion

Transcription

1 Solomonoff Induction Violates Nicod s Criterion Jan Leike and Marcus Hutter ALT 15 6 October 2015

2 Outline The Paradox of Confirmation Solomonoff Induction Results Resolving the Paradox of Confirmation References

3 Motivation What does this green apple tell you about black ravens?

4 The Paradox of Confirmation Proposed by [Hempel, 1945]. H = all ravens are black

5 The Paradox of Confirmation Proposed by [Hempel, 1945]. H = all ravens are black H = all nonblack objects are nonravens

6 The Paradox of Confirmation Proposed by [Hempel, 1945]. H = all ravens are black H = all nonblack objects are nonravens Nicod s criterion: Something that is F and G confirms all F s are Gs = A nonblack nonraven confirms H

7 The Paradox of Confirmation Proposed by [Hempel, 1945]. H = all ravens are black H = all nonblack objects are nonravens Nicod s criterion: Something that is F and G confirms all F s are Gs = A nonblack nonraven confirms H Equivalence condition: Logically equivalent hypotheses are confirmed by the same evidence = A nonblack nonraven confirms H

8 The Paradox of Confirmation Proposed by [Hempel, 1945]. H = all ravens are black H = all nonblack objects are nonravens Nicod s criterion: Something that is F and G confirms all F s are Gs = A nonblack nonraven confirms H Equivalence condition: Logically equivalent hypotheses are confirmed by the same evidence = A nonblack nonraven confirms H Paradox?

9 Outline The Paradox of Confirmation Solomonoff Induction Results Resolving the Paradox of Confirmation References

10 Solomonoff Induction Let U be a universal monotone Turing machine. Solomonoff s universal prior [Solomonoff, 1964]: M(x) := p: U(p)=x... M is a probability distribution on X X 2 p

11 Solomonoff Induction Let U be a universal monotone Turing machine. Solomonoff s universal prior [Solomonoff, 1964]: M(x) := p: U(p)=x... M is a probability distribution on X X 2 p Solomonoff normalization: M norm (ɛ) := 1 and M(xa) M norm (xa) := M norm (x) b X M(xb) M norm is a probability distribution on X

12 Properties of Solomonoff Induction Observe (non-iid) data x <t := x 1 x 2... x t 1 X, predict arg max M(a x <t ) a X

13 Properties of Solomonoff Induction Observe (non-iid) data x <t := x 1 x 2... x t 1 X, predict arg max M(a x <t ) a X At most E + O( E) errors when observing data from a computable measure µ (E = errors of the predictor that knows µ) [Hutter, 2001]

14 Properties of Solomonoff Induction Observe (non-iid) data x <t := x 1 x 2... x t 1 X, predict arg max M(a x <t ) a X At most E + O( E) errors when observing data from a computable measure µ (E = errors of the predictor that knows µ) [Hutter, 2001] M merges with any computable measure µ [Blackwell and Dubins, 1962]: sup M(H x <t ) µ(h x <t ) 0 µ-a.s. as t H

15 Properties of Solomonoff Induction Observe (non-iid) data x <t := x 1 x 2... x t 1 X, predict arg max M(a x <t ) a X At most E + O( E) errors when observing data from a computable measure µ (E = errors of the predictor that knows µ) [Hutter, 2001] M merges with any computable measure µ [Blackwell and Dubins, 1962]: sup M(H x <t ) µ(h x <t ) 0 µ-a.s. as t H M is lower semicomputable, but M(xy x) is incomputable

16 Properties of Solomonoff Induction Observe (non-iid) data x <t := x 1 x 2... x t 1 X, predict arg max M(a x <t ) a X At most E + O( E) errors when observing data from a computable measure µ (E = errors of the predictor that knows µ) [Hutter, 2001] M merges with any computable measure µ [Blackwell and Dubins, 1962]: sup M(H x <t ) µ(h x <t ) 0 µ-a.s. as t H M is lower semicomputable, but M(xy x) is incomputable = M is really good at learning

17 Outline The Paradox of Confirmation Solomonoff Induction Results Resolving the Paradox of Confirmation References

18 Setup Alphabet = observations: X := {BR, BR, BR, BR}

19 Setup Alphabet = observations: X := {BR, BR, BR, BR} Hypothesis H = all ravens are black : H := {x X X x does not contain BR}

20 Setup Alphabet = observations: X := {BR, BR, BR, BR} Hypothesis H = all ravens are black : H := {x X X x does not contain BR} Data x <t drawn from a computable measure µ for t = 1, 2,...

21 Setup Alphabet = observations: X := {BR, BR, BR, BR} Hypothesis H = all ravens are black : H := {x X X x does not contain BR} Data x <t drawn from a computable measure µ for t = 1, 2,... M(H x <t ) is subjective belief in H at time step t

22 Setup Alphabet = observations: X := {BR, BR, BR, BR} Hypothesis H = all ravens are black : H := {x X X x does not contain BR} Data x <t drawn from a computable measure µ for t = 1, 2,... M(H x <t ) is subjective belief in H at time step t Confirmation and disconfirmation: µ(h) = 0 = t. M(H x <t ) = 0 µ-a.s. µ(h) = 1 = M(H x <t ) 1 µ-a.s.

23 Setup Alphabet = observations: X := {BR, BR, BR, BR} Hypothesis H = all ravens are black : H := {x X X x does not contain BR} Data x <t drawn from a computable measure µ for t = 1, 2,... M(H x <t ) is subjective belief in H at time step t Confirmation and disconfirmation: µ(h) = 0 = t. M(H x <t ) = 0 µ-a.s. µ(h) = 1 = M(H x <t ) 1 µ-a.s. Equivalence condition is satisfied.

24 Nicod s Criterion Question: Does a black raven confirm H: M(H x <t ) < M(H x <t BR)?

25 Nicod s Criterion Question: Does a black raven confirm H: M(H x <t ) < M(H x <t BR)? Question: Does a nonblack nonraven confirm H: M(H x <t ) < M(H x <t BR)?

26 Nicod s Criterion Question: Does a black raven confirm H: M(H x <t ) < M(H x <t BR)? Question: Does a nonblack nonraven confirm H: M(H x <t ) < M(H x <t BR)? Answer: Not always.

27 Solomonoff Induction and Nicod s Criterion Theorem (Counterfactual Black Raven Disconfirms H) Let x 1: H X be computable and x t BR infinitely often. = t N (with x t BR) s.t. M(H x <t BR) < M(H x <t )

28 Solomonoff Induction and Nicod s Criterion Theorem (Counterfactual Black Raven Disconfirms H) Let x 1: H X be computable and x t BR infinitely often. = t N (with x t BR) s.t. M(H x <t BR) < M(H x <t ) Theorem (Disconfirmation Infinitely Often for M) Let x 1: H be computable. = M(H x 1:t ) < M(H x <t ) infinitely often.

29 Solomonoff Induction and Nicod s Criterion Theorem (Counterfactual Black Raven Disconfirms H) Let x 1: H X be computable and x t BR infinitely often. = t N (with x t BR) s.t. M(H x <t BR) < M(H x <t ) Theorem (Disconfirmation Infinitely Often for M) Let x 1: H be computable. = M(H x 1:t ) < M(H x <t ) infinitely often. Theorem (Disconfirmation Finitely Often for M norm ) Let x 1: H be computable. = t 0 t > t 0. M norm (H x 1:t ) > M norm (H x <t ).

30 Solomonoff Induction and Nicod s Criterion Theorem (Counterfactual Black Raven Disconfirms H) Let x 1: H X be computable and x t BR infinitely often. = t N (with x t BR) s.t. M(H x <t BR) < M(H x <t ) Theorem (Disconfirmation Infinitely Often for M) Let x 1: H be computable. = M(H x 1:t ) < M(H x <t ) infinitely often. Theorem (Disconfirmation Finitely Often for M norm ) Let x 1: H be computable. = t 0 t > t 0. M norm (H x 1:t ) > M norm (H x <t ). Theorem (Disconfirmation Infinitely Often for M norm ) There is an (incomputable) x 1: H s.t. M norm (H x 1:t ) < M norm (H x <t ) infinitely often.

31 Outline The Paradox of Confirmation Solomonoff Induction Results Resolving the Paradox of Confirmation References

32 Resolving the Paradox of Confirmation I Solution: Reject Nicod s criterion! [Good, 1967, Jaynes, 2003, Vranas, 2004] Not all black ravens confirm H.

33 Resolving the Paradox of Confirmation II In the literature there are perhaps 100 paradoxes and controversies which are like this, in that they arise from faulty intuition rather than faulty mathematics. Someone asserts a general principle that seems to him intuitively right. Then, when probability analysis reveals the error, instead of taking this opportunity to educate his intuition, he reacts by rejecting the probability analysis. [Jaynes, 2003, p. 144]

34 Outline The Paradox of Confirmation Solomonoff Induction Results Resolving the Paradox of Confirmation References

35 References I Blackwell, D. and Dubins, L. (1962). Merging of opinions with increasing information. The Annals of Mathematical Statistics, pages Good, I. J. (1967). The white shoe is a red herring. The British Journal for the Philosophy of Science, 17(4): Hempel, C. G. (1945). Studies in the logic of confirmation (I.). Mind, pages Hutter, M. (2001). New error bounds for Solomonoff prediction. Journal of Computer and System Sciences, 62(4): Jaynes, E. T. (2003). Probability Theory: The Logic of Science. Cambridge University Press.

36 References II Solomonoff, R. (1964). A formal theory of inductive inference. Parts 1 and 2. Information and Control, 7(1):1 22 and Vranas, P. B. (2004). Hempel s raven paradox: A lacuna in the standard Bayesian solution. The British Journal for the Philosophy of Science, 55(3):