Partial Operator Induction with Beta Distribution. Nil Geisweiller AGI-18 Prague

Transcription

1 Partial Operator Induction with Beta Distribution Nil Geisweiller AGI-18 Prague 1

2 Problem: Combining Models from Different Contexts Theory: Solomonoff Operator Induction and Beta Distribution Practice: Inference Control Meta-Learning 2

3 Outline Problem: Combining Models from Different Contexts Theory: Solomonoff Operator Induction and Beta Distribution Practice: Inference Control Meta-Learning 3

4 Outline Problem: Combining Models from Different Contexts Theory: Solomonoff Operator Induction and Beta Distribution Practice: Inference Control Meta-Learning 4

5 Problem: Models from different contexts How to combine models obtained from different contexts? Large Contexts Underfit Small Contexts Overfit 5

6 Problem: Preserve Uncertainty 6

7 Problem: Preserve Uncertainty Exploration vs Exploitation (Thompson Sampling) 7

8 Problem: ImplicationLink ImplicationLink <TV> R S Second Order P(S R) Beta Distribution in disguise 8

9 Solution Bayesian Model Averaging / Solomonoff Operator Induction, modified to: 1. Support partial models 2. Produce a probability distribution estimate, rather than probability estimate. 3. Specialize for Beta distributions 9

10 Outline Problem: Combining Models from Different Contexts Theory: Solomonoff Operator Induction and Beta Distribution Practice: Inference Control Meta-Learning 10

11 Outline Problem: Combining Models from Different Contexts Theory: Solomonoff Operator Induction and Beta Distribution Practice: Inference Control Meta-Learning 11

12 Solomonoff Operator Induction Bayesian Model Averaging + Universal Distribution Probability Estimate: ˆP(A n+1 Q n+1 ) = j n+1 a j 0 i=1 O j (A i Q i ) where: Q i = i th question A i = i th answer O j = j th operator a j 0 = prior of jth operator 12

13 Beta Distribution Operator Specialization of Solomonoff Operator Induction OpenCog implication link ImplicationLink <TV> R S Class of parameterized operators O j p(a i Q i ) = if R j (Q i ) then p, if A i = A n+1 1 p, otherwise 13

14 Beta Distribution Probability Density Function: pdf α,β (x) = x α 1 (1 x) β 1 B(α, β) Beta Function: B x (α, β) = x 0 p α 1 (1 p) β 1 dp B(α, β) = B 1 (α, β) Conjugate Prior: pdf m+α,n m+β (x) x m (1 x) n m pdf α,β (x) 14

15 Artificial Completion O j p (A i Q i ) = if R j (Q i ) then p, if A i = A n+1 1 p, otherwise 15

16 Artificial Completion O j p,c (A i Q i ) = if R j (Q i ) then else C(A i Q i ) p, if A i = A n+1 1 p, otherwise 15

17 Second Order Solomonoff Operator Induction Probability Estimate: ˆP(A n+1 Q n+1 ) = j n+1 a j 0 i=1 O j (A i Q i ) Probability Distribution Estimate: ˆ cdf (An+1 Q n+1 )(x) = a j 0 O j (A n+1 Q n+1 ) x i=1 n O j (A i Q i ) 16

18 Combing Solomonoff Operator Induction and Beta Distributions ˆ cdf (An+1 Q n+1 )(x) j a j 0 r j B x (m j +α, n j m j +β)b(m j +α, n j m j +β) where n j = number of observations explained by j th model m j = number of true observations explained by j th model r j = likelihood of the unexplained data r j =??? 17

19 Combing Solomonoff Operator Induction and Beta Distributions ˆ cdf (An+1 Q n+1 )(x) j a j 0 r j B x (m j +α, n j m j +β)b(m j +α, n j m j +β) where n j = number of observations explained by j th model m j = number of true observations explained by j th model r j = likelihood of the unexplained data r j =??? 2 v (1 c) v = n n j = number of unexplained observations c = compressability parameter c = 1 explains remaining data c = 0 can t explain remaining data 17

20 Outline Problem: Combining Models from Different Contexts Theory: Solomonoff Operator Induction and Beta Distribution Practice: Inference Control Meta-Learning 18

21 Outline Problem: Combining Models from Different Contexts Theory: Solomonoff Operator Induction and Beta Distribution Practice: Inference Control Meta-Learning 19

22 Inference Control Meta-learning Learn how to reason efficiently 20

23 Inference Control Meta-learning Methodology: Learn how to reason efficiently 1. Solve sequence of problems (via reasoning) 20

24 Inference Control Meta-learning Learn how to reason efficiently Methodology: 1. Solve sequence of problems (via reasoning) 2. Store inference traces 20

25 Inference Control Meta-learning Learn how to reason efficiently Methodology: 1. Solve sequence of problems (via reasoning) 2. Store inference traces 3. Mine traces to discover patterns 20

26 Inference Control Meta-learning Learn how to reason efficiently Methodology: 1. Solve sequence of problems (via reasoning) 2. Store inference traces 3. Mine traces to discover patterns 4. Build control rules Implication <TV> And <inference-pattern> <rule> <good-inference> 20

27 Inference Control Meta-learning Learn how to reason efficiently Methodology: 1. Solve sequence of problems (via reasoning) 2. Store inference traces 3. Mine traces to discover patterns 4. Build control rules Implication <TV> And <inference-pattern> <rule> <good-inference> 5. Combine control rules to guide future reasoning 20

28 Combine Control Rules Implication <TV1> And <inference-pattern-1> deduction-rule <good-inference> Implication <TV2> And <inference-pattern-2> deduction-rule <good-inference> c = 1 = 21

29 Combine Control Rules Implication <TV1> And <inference-pattern-1> deduction-rule <good-inference> Implication <TV2> And <inference-pattern-2> deduction-rule <good-inference> c = 0.5 = 22

30 Combine Control Rules Implication <TV1> And <inference-pattern-1> deduction-rule <good-inference> Implication <TV2> And <inference-pattern-2> deduction-rule <good-inference> c = 0.1 = 23

31 Conclusion Contribution: Second Order Solomonoff Operator Induction Specialized for Beta Distribution Attempt to Deal with Partial Models 24

32 Conclusion Contribution: Second Order Solomonoff Operator Induction Specialized for Beta Distribution Attempt to Deal with Partial Models Future Work: Improve Likelihood of Unexplained Data More Experiments (Inference Control Meta-learning) 24

33 Conclusion Contribution: Second Order Solomonoff Operator Induction Specialized for Beta Distribution Attempt to Deal with Partial Models Future Work: Improve Likelihood of Unexplained Data More Experiments (Inference Control Meta-learning) Thank you! 24