Biointelligence Lab School of Computer Sci. & Eng. Seoul National University

Transcription

1 Artificial Intelligence Chater 19 easoning with Uncertain Information Biointelligence Lab School of Comuter Sci. & Eng. Seoul National University

2 Outline l eview of Probability Theory l Probabilistic Inference l Bayes Networks l Patterns of Inference in Bayes Networks l Uncertain Evidence l D-Searation l Probabilistic Inference in Polytrees C SNU CSE Biointelligence Lab 2

3 19.1 eview of Probability Theory 1/4 l andom variables V 1 V2... V k l Joint robability V v V v... V k v k Ex. B BAT_OK M MOVES L LIFTABLE G GUAGE Joint Probability True True True True True True True False True True False True True True False False C SNU CSE Biointelligence Lab 3

4 19.1 eview of Probability Theory 2/4 l Marginal robability Ex. å B b B M L G B b å B b M m B M L G B b M m l Conditional robability V V i j i V j V V j Ex. The robability that the battery is charged given that the arm does not move B True M False B True M False M False C SNU CSE Biointelligence Lab 4

5 19.1 eview of Probability Theory 3/4 Figure 19.1 A Venn Diagram C SNU CSE Biointelligence Lab 5

6 19.1 eview of Probability Theory 4/4 l Chain rule B L G M B L G M L G M G M M l Bayes rule V V i l set notation j k V... V V V V V Õ 1 2 k i i-1... i 1 j Vi Vi V V Abbreviation for V 1 V2... where V V V... j V V k { } 1 2 V k C SNU CSE Biointelligence Lab 6 1

7 19.2 Probabilistic Inference 19.2 Probabilistic Inference l We desire to calculate the robability of some variable V i has value v i given the evidence E e. [ ] e e True V e True V i i E E E C SNU CSE Biointelligence Lab 7 PQ 0.3 PQ 0.2 P Q 0.2 P Q 0.1 PQ 0.05 P Q 0.1 P Q 0.05 P Q 0.0 Examle [ ] Q P Q P Q Q + + [ ] Q P Q P Q Q Q Q Q Q

8 Statistical Indeendence l Conditional indeendence V V V i V j Vi V j V V : a set of variables Intuition: V i tells us nothing more about V than we already knew by knowing V j l Mutually conditional indeendence k 1 V2... V j V Õ Vi Vi -1 Vi V1 V V i 1 k Õ i 1 V V i l Unconditional indeendence When V is emty V V... V V V j 1 2 V k C SNU CSE Biointelligence Lab 8

9 19.3 Bayes Networks 1/2 l Directed acyclic grah DAG whose nodes are labeled by random variables l Characteristics of Bayesian networks Node V i is conditionally indeendent of any subset of nodes that are not descendents of V i given its arents V... V V Pa V V Õ 1 2 k i i i 1 l Prior robability k l Conditional robability table CPT C SNU CSE Biointelligence Lab 9

10 19.3 Bayes Networks 2/2 Bayes network about the block-lifting examle C SNU CSE Biointelligence Lab 10

11 19.4 Patterns of Inference in Bayes Networks 1/3 l Causal or to-down inference Ex. The robability that the arm moves given that the block is liftable L B M L M B L + M B L G M M B L B L + M B L B L chain rule M B L B + M B L B from the structure 0.9 * C SNU CSE Biointelligence Lab 11

12 19.4 Patterns of Inference in Bayes Networks 2/3 l Diagnostic or bottom-u inference Using an effect or symtom to infer a cause B G M Ex. The robability that the block is not liftable given that the arm does not move. M L 0. using causal reasoning 9525 L L M M L L M M M using Bayes rule L M M L L M M M using Bayes rule L M C SNU CSE Biointelligence Lab 12

13 l Exlaining away One evidence: M the arm does not move Additional evidence: B the battery is not charged Bayes rule 19.4 Patterns of Inference in Bayes Networks 3/ Patterns of Inference in Bayes Networks 3/3 B G M L L L B M M B L C SNU CSE Biointelligence Lab 13 B exlains M making L less certain 0.30< Bayes rule def. of conditional rob. structure of the Bayes network M B L B L B M M B L L B L B M M B M B L

14 19.5 Uncertain Evidence l We must be certain about the truth or falsity of the roositions they reresent. Each uncertain evidence node should have a child node about which we can be certain. Ex. Suose the robot is not certain that its arm did not move. < Introducing M : The arm sensor says that the arm moved We can be certain that that roosition is either true or false. < L B M instead of L B M Ex. Suose we are uncertain about whether or not the battery is charged. < Introducing G : Battery guage < L G M instead of L B M C SNU CSE Biointelligence Lab 14

15 19.6 D-Searation 1/3 l D-saaration: direction-deendent searation l l l Two nodes V i and V j are conditionally indeendent given a set of nodes E if for every undirected ath in the Bayes network between V i and V j there is some node V b on the ath having one of the following three roerties. V b is in E and both arcs on the ath lead out of V b V b is in E and one arc on the ath leads in to V b and one arc leads out. Neither V b nor any descendant of V b is in E and both arcs on the ath lead in to V b. V b blocks the ath given E when any one of these conditions holds for a ath. If all aths between V i and V j are blocked we say that E d-searates V i and V j C SNU CSE Biointelligence Lab 15

16 19.6 D-Searation 2/3 Figure 19.3 Conditional Indeendence via Blocking Nodes C SNU CSE Biointelligence Lab 16

17 19.6 D-Searation 3/3 L l Ex. IG L B by rules 1 and 3 < By rule 1 B blocks the only ath between G and L given B. < By rule 3 M also blocks this ath given B. IG L < By rule 3 M blocks the ath between G and L. IB L < By rule 3 M blocks the ath between B and L. l Even using d-searation robabilistic inference in Bayes networks is in general NP-hard. G B M C SNU CSE Biointelligence Lab 17

18 19.7 Probabilistic Inference in Polytrees 1/2 l Polytree A DAG for which there is just one ath along arcs in either direction between any two nodes in the DAG. C SNU CSE Biointelligence Lab 18

19 19.7 Probabilistic Inference in Polytrees 2/2 l A node is above Q The node is connected to Q only through Q s arents l A node is below Q The node is connected to Q only through Q s immediate successors. l Three tyes of evidences All evidence nodes are above Q. All evidence nodes are below Q. There are evidence nodes both above and below Q. C SNU CSE Biointelligence Lab 19

20 Evidence Above 1/2 l Bottom-u recursive algorithm l Ex. Q P5 P4 Q P5 P4 Q P6 P7 P5 P4 å P 6 P 7 å P6 P7 å P6 P7 å P6 P7 å P6 P7 Q P6 P7 P5 P4 P6 P7 P5 P4 Q P6 P7 P6 P7 P5 P4 Q P6 P7 P6 P5 P4 P7 P5 P4 Q P6 P7 P6 P5 P7 P4 Structure of The Bayes network d-searation d-searation C SNU CSE Biointelligence Lab 20

21 Evidence Above 2/2 l Calculating P7 P4 and P6 P5 P7 P4 å P7 P3 P4 P3 P4 å P7 P3 P4 P3 P 3 P 3 P6 P5 å P6 P1 P2 P1 P5 P2 P1 P 2 l Calculating P1 P5 Evidence is below Here we use Bayes rule P5 P1 P1 P1 P5 P5 C SNU CSE Biointelligence Lab 21

22 Evidence Below 1/ Q P P P P P12 P13 P14 P P P P P Q Q k P P P P Q Q k P P Q P P Q Q l Using a to-down recursive algorithm P12 P13 Q P12 P13 P9 Q 9 Q å P9 å P9 P12 P13 P9 P9 Q P9 Q P9 P8 Q P8 P12 P13 P9 P12 P9 P13 P9 å P8 d-searation C SNU CSE Biointelligence Lab 22

23 Evidence Below 2/2 P14 P11 Q P14 P11 P10 P10 Q å P10 å P10 P14 P10 P11 P10 P10 Q å P15 10 å P11 P15 P10 P11 P P15 P P P P P P P P P P P P P P P P P k P P P P P P P P P P P P P P P P C SNU CSE Biointelligence Lab 23

24 Evidence Above and Below Q { P5 P4}{ P12 P13 P14 P11} E + E E E Q E - E + E E E E E E - + E Q Q E Q Q k Q Q 2 k 2 d-searation We have calculated two robabilities already C SNU CSE Biointelligence Lab 24

25 A Numerical Examle 1/2 We want to calculate Q U Q U k U Q Q U Q å U P P Q P Bayes rule P Q P Q å P Q + P Q Q U P U P U P Q Q U k k 0.03 To determine k we need to calculate Q U C SNU CSE Biointelligence Lab 25

26 A Numerical Examle 2/2 Q U k U Q Q å U Q U P P Q P Bayes rule P Q P Q å P Q + P Q Q U P U P U Q U Q U k k Q U \ k 4.35 P Q Finally C SNU CSE Biointelligence Lab 26 k k 0.03 k k 0.20 Q U

27 Other methods for Probabilistic inference in Bayes Networks l Bucket elimination l Monte Carlo methods when the network is not a olytree l Clustering C SNU CSE Biointelligence Lab 27

28 Additional eadings 1/5 l [Feller 1968] Probability Theory l [Goldszmidt Morris & Pearl 1990] Non-monotonic inference through robabilistic method l [Pearl 1982a Kim & Pearl 1983] Message-assing algorithm l [ussell & Norvig ff] Polytree methods C SNU CSE Biointelligence Lab 28

29 Additional eadings 2/5 l [Shachter & Kenley 1989] Bayesian network for continuous random variables l [Wellman 1990] Qualitative networks l [Neaolitan 1990] Probabilistic methods in exert systems l [Henrion 1990] Probability inference in Bayesian networks C SNU CSE Biointelligence Lab 29

30 Additional eadings 3/5 l [Jensen 1996] Bayesian networks: HUGIN system l [Neal 1991] elationshis between Bayesian networks and neural networks l [Hecherman 1991 Heckerman & Nathwani 1992] PATHFINDE l [Pradhan et al. 1994] CPCSBN C SNU CSE Biointelligence Lab 30

31 Additional eadings 4/5 l [Shortliffe 1976 Buchanan & Shortliffe 1984] MYCIN: uses certainty factor l [Duda Hart & Nilsson 1987] POSPECTO: uses sufficiency index and necessity index l [Zadeh 1975 Zadeh 1978 Elkan 1993] Fuzzy logic and ossibility theory l [Demster 1968 Shafer 1979] Demster-Shafer s combination rules C SNU CSE Biointelligence Lab 31

32 Additional eadings 5/5 l [Nilsson 1986] Probabilistic logic l [Tversky & Kahneman 1982] Human generally loses consistency facing uncertainty l [Shafer & Pearl 1990] Paers for uncertain inference l Proceedings & Journals Uncertainty in Artificial Intelligence UAI International Journal of Aroximate easoning C SNU CSE Biointelligence Lab 32