Bayesian Networks. 22c:145 Artificial Intelligence. Bayesian Networks. Review of Probability Theory. Bayesian (Belief) Networks.

Transcription

1 ayesian Networks 22c:145 rtificial ntelligence ayesian Networks Reading: h 14. Russell & Norvig To do probabilistic reasoning, you need to know the joint probability distribution ut, in a domain with N propositional variables, one needs 2 N numbers to specify the joint probability distribution We want to exploit independences in the domain Two components: structure and numerical parameters 1 4 Review of Probability Theory Random Variables The probability that a random variable X has value val is written as P(X=val) P: domain [0, 1] Sums to 1 over the domain:» P(ing = true) = P(ing) = 0.2» P(ing = false) = P( ing) = Joint distribution: P(X1, X2,, Xn) Toothache Toothache avity avity ayesian (elief) Networks Set of random variables, each has a finite set of values Set of directed arcs between them forming acyclic graph, representing causal relation Every node, with parents 1,, n, has P( 1,, n ) specified Probability assignment to all combinations of values of random variables and provide complete information about the probabilities of its random variables. JP table for n random variables, each ranging over k distinct values, has k n entries! 2 5 Review of Probability Theory onditioning P() = P( ) P() + P( ) P( ) = P( ) + P( ) and are independent iff P( ) = P() P() P( ) = P() P( ) = P() and are conditionally independent given iff P(, ) = P( ) P(, ) = P( ) P( ) = P( ) P( ) Key dvantage The conditional independencies (missing arrows) mean that we can store and compute the joint probability distribution more efficiently How to design a elief Network? Explore the causal relations ayes Rule P( ) = P( ) P() / P() P(, ) = P(, ) P( ) / P( ) 3 6 1

2 Roads nspector Smith is waiting for and, who are driving (separately) to meet him. t is winter. His secretary tells him that has had an accident. He says, t must be that the roads are icy. bet that will have an accident too. should go to lunch. ut, his secretary says, guess better wait for. Roads nspector Smith is waiting for and, who are driving (separately) to meet him. t is winter. His secretary tells him that has had an accident. He says, t must be that the roads are icy. bet that will have an accident too. should go to lunch. ut, his secretary says, guess better wait for. H and W are dependent, 7 10 Roads nspector Smith is waiting for and, who are driving (separately) to meet him. t is winter. His secretary tells him that has had an accident. He says, t must be that the roads are icy. bet that will have an accident too. should go to lunch. ut, his secretary says, guess better wait for. Roads nspector Smith is waiting for and, who are driving (separately) to meet him. t is winter. His secretary tells him that has had an accident. He says, t must be that the roads are icy. bet that will have an accident too. should go to lunch. ut, his secretary says, guess better wait for. H and W are dependent, but conditionally independent given 8 11 Roads nspector Smith is waiting for and, who are driving (separately) to meet him. t is winter. His secretary tells him that has had an accident. He says, t must be that the roads are icy. bet that will have an accident too. should go to lunch. ut, his secretary says, guess better wait for. and in and have moved to. He wakes up to his sprinkler on. He looks at his neighbor s lawn

3 and in and in and have moved to. He wakes up to his sprinkler on. He looks at his neighbor s lawn and have moved to. He wakes up to his sprinkler on. He looks at his neighbor s lawn Given W, P(R) goes up and in and in and have moved to. He wakes up to his sprinkler on. He looks at his neighbor s lawn and have moved to. He wakes up to his sprinkler on. He looks at his neighbor s lawn Given W, P(R) goes up and P(S) goes down explaining away and in and have moved to. He wakes up to his sprinkler on. He looks at his neighbor s lawn nference in ayesian Networks Query Types Given a ayesian network, what questions might we want to ask? onditional probability query: P(x e) Maximum a posteriori probability: What value of x maximizes P(x e)? General question: What s the whole probability distribution over variable X given evidence e, P(X e)?

4 Using the joint distribution To answer any query involving a conjunction of variables, sum over the variables not involved in the query. Pr(d) = = Pr( = a = b = c) a dom( ) b dom( ) c dom( ) P() = P(=true, =true, P() P() P(,) P( ) Using the joint distribution To answer any query involving a conjunction of variables, sum over the variables not involved in the query. Pr(d) = = Pr( = a = b = c) a dom( ) b dom( ) c dom( ) Pr(d b) = Pr(b,d) Pr(b) = P() = P(=true, =true, P() = P( )P() P() P() P(,) P( ) P() P() P() = P(=true, =true, P() P() P(,) P() = P(,) P( )P() = P( ) P( ) P() = P( ) independent from given independent from given

5 Roads with Numbers t = true f= false P(=t) 0.7 P(=f) 0.3 P() = P(=true, =true, P() P() P() = P( )P() = P( ) P() = P( ) P( ) P() = P(,) P( ) independent from given independent from given The right-hand column in these tables is redundant, since we know the entries in each row must add to 1. N: the columns need NOT add to Roads with Numbers t = true f= false P(=t) 0.7 P(=f) 0.3 P() = P(=true, =true, P() P() P() = P(,) P( )P() = P( ) P() = P( ) P( ) P( ) P() = P( ) P( ) P()P() independent from given independent from given independent from 26 =t =f The right-hand column in these tables is redundant, since we know the entries in each row must add to 1. Note: the columns need NOT add to 1. P(W=t ) P(W=f ) Roads with Numbers t = true f= false P(=t) 0.7 P(=f) 0.3 P() = P(=true, =true, P() P() P() = P(,) P( )P() = P( ) P() = P( ) P( ) P( ) P() = P( ) P( ) P()P() independent from given independent from given independent from =t =f P(H=t ) P(H=f ) =t =f The right-hand column in these tables is redundant, since we know the entries in each row must add to 1. Note: the columns need NOT add to 1. P(W=t ) P(W=f )

6 Probability that es Probability of given P()=0.7 P()=0.7 P(H ) P(W ) P(H ) P(W ) P(W) = P( W) = P(W ) P() / P(W) = *0.7 / 0.59 = 0.95 We started with P() = 0.7; knowing that crashed raised the probability to Probability that es Probability of given P()=0.7 P()=0.7 P(H ) P(W ) P(H ) P(W ) P(W) = P(W ) P() + P(W ) P( ) = *0.7 + *0.3 = = 0.59 P(H W) = Probability of given Probability of given P()=0.7 P()=0.7 P(H ) P(W ) P(H ) P(W ) P( W) = P(H W) = P(H, W) + P(H, - W) = P(H W,)P( W) + P(H W, ) P( W) = P(H )P( W) + P(H ) P( W) = * *0.05 = We started with P(H) = 0.59; knowing that crashed raised the probability to

7 Prob of given and P()=0.7 ~ P(H ) ~ P(W ) P(H W, ~ ) = P(H ~) = H and W are independent given, so H and W are conditionally independent given 37 Excercises P(J, M,,, E ) =? P(M,, ) =? P(-M,, ) =? P(, ) =? P(M, ) =? P( J) =? 38 7