Cavities. Independence. Bayes nets. Why do we care about variable independence? P(W,CY,T,CH) = P(W )P(CY)P(T CY)P(CH CY)

Transcription

1 ayes nets S151 David Kauchak Fall Some material borrowed from: Sara Owsley Sood and others Independence Two variables are independent if knowing the values of one, does not give us inrmation about the other P(A,) = P(A)P() P(A ) = P(A) Variables can also be independent only when they are conditioned on another variable P(A, ) = P(A )P( ) P(A,) = P(A ) Why do we care about variable independence? avities P(W,Y,T,H) = P(W )P(Y)P(T Y)P(H Y) What independences are encoded (both unconditional and conditional)? 1

2 ayes nets ayes nets are a way of representing joint distributions Directed, acyclic graphs Nodes represent random variables Directed edges represent dependence Associated with each node is a conditional probability distribution P(X parents(x)) They encode dependences/independences avities P(Weather) Weather P(avity) P(Toothache avity) P(atch avity) What independences are encoded? avities Why all the fuss about independences? P(Weather) Weather P(avity) P(Weather) Weather P(avity) P(Toothache avity) P(atch avity) P(Toothache avity) P(atch avity) Weather is independent of all variables Toothache and atch are conditionally independent GIVEN avity Does this help us in storing the distribution? asic joint distribution 2 4 = 16 entries With independences? = 12 If we re sneaky: = 6 an be much more significant as number of variables increases! 2

3 avities avities P(Weather) Weather P(avity) P(Weather) Weather P(avity) P(Toothache avity) P(atch avity) P(Toothache avity) P(atch avity) P(W,T,Y,H) = P(W )P(T,Y,H W ) = P(W )P(Y W )P(T,H Y,W ) = P(W )P(Y W )P(H Y,W )P(T H,Y,W ) Independences? = P(W )P(Y)P(H Y)P(T Y) Graph alws us to figure out dependencies, and encodes the same inrmation as the joint distribution. Another Example Question: Is the family next door out? Variables that give inrmation about this question: DO: is the dog outside? FO: is the family out (away from home)? LO: are the lights on? P: does the dog have a bowel problem? H: can you hear the dog bark? Expit onditional Independence Which variables are directly dependent? Variables that give inrmation about this question: DO: is the dog outside? FO: is the family out (away from home)? LO: are the lights on? P: does the dog have a bowel problem? H: can you hear the dog bark? Are LO and DO independent? What if you know that the family is away? Are H and FO independent? What if you know that the dog is outside? 3

4 Some options ayesian Network Example lights (LO) depends on family out (FO) dog out (DO) depends on family out (FO) barking (H) depends on dog out (DO) dog out (DO) depends on bowels (P) random variables direct probabilistic influence What would the network ok like? joint probability distribution Graph structure represents direct influences between variables (an think of it as causality but it doesn't have to be) Three Types of Relationships Relationships and independence Linear a onverging Diverging A b a c b A A A c b a c family-out family-out bowel-problem family-out In which of these are A and dependent/independent? dog-out hear-bark dog-out light-on dog-out 4

5 d-separation dependence separation an active path in a N is a path that carries inrmation If we know A does that give us inrmation about? Two variables are dependent if there is an active path between them d-separation We can bck an active path by conditioning on the internal node () The dependence between A and is through A and are independent given A A A A A A A Which have active paths from A to? d-separation In this case, A and are already independent A d-separation sets of nodes X and Y are independent given a set of nodes Z if the sets are d-separated by Z Z d-separates X and Y if r all undirected paths from X to Y the path contains a chain with a node from Z in the middle X Z Y X Z Y the path contains a rk with a node from Z in the middle What happens when we condition on? Z X Y We make A and dependent! This is sometimes referred to as the explaining away phenomenon Any inverted rks (aka colliders) do not contain a node from Z X Y 5

6 Independence Independence What unconditional independences does this graph encode, i.e. what nodes are d-separated using an empty set? FO independent of P LO independent of P onditional Independence onditional Independence What conditional independences does this graph encode? H independent of FO, P, LO given DO DO independent of LO given FO LO independent of H, DO, P given FO 6

7 Markov lanket The Markov blanket of a node is: the parents the children the parents of the children A node is independent of all other nodes, given its Markov blanket How do these independences help? Question: Is the family next door out? Variables that give inrmation about this question: DO: is the dog outside? FO: is the family out (away from home)? LO: are the lights on? P: does the dog have a bowel problem? H: can you hear the dog bark? How many entries in the full joint distribution table? How do these independences help? How many in the ayes Net? Question: Is the family next door out? Variables that give inrmation about this question: DO: is the dog outside? FO: is the family out (away from home)? LO: are the lights on? P: does the dog have a bowel problem? H: can you hear the dog bark? 32 Joint Probability Table hb hb do do do do 7

8 How do Ns help? How do Ns help? onditional Probability Tables P(H DO) P(P) do do hb Joint Probability Table hb hb hb do do do do P(FO) vs onditional Probability Tables P(H DO) P(P) do do hb Joint Probability Table hb hb do do do do P(FO) vs P(DO FO,P) do do P(DO FO,P) do P(FO LO) P(FO LO) N Example Prior probability distribution (r root nodes) conditional probability tables: P(hild Parents) ayes nets: ompactness How many numbers are required to build a ayes Net For a oolean variable X with k oolean parents, how many rows in the PT? 2 k If each variable has no more than k parents and there are n nodes in the network, how many numbers required? n2 k How many numbers required to specify the full joint distribution? 2 n Why don t these sum to 1? 8

9 ayes nets: Intuitiveness Example: ar Diagnosis an you estimate P(do,,,, hb)? P(do,,, hb)? How about P( ) (lights out given that the family is out)? MammoNet: 88% accuracy Other medical networks Mostly manually generated PATHFINDER: pathogy MUNIN: neuromuscular disorders PS (omputer-based Patient ase Study): internal medicine 448 nodes 8,254 conditional probability values Automatically generated 100K to millions of nodes e.g. text processing 9

10 ARO1: Forecasting Oil Prices ARO1: Forecasting Oil Prices Asking questions about distributions We want to be able to ask questions about these probability distributions Given n variables, a query splits the variables into three sets: query variable(s) known/evidence variables unknown/hidden variables P(query evidence) if we had no hidden variables, we could just multiply all the values in the different PTs to answer this, we need to sum over the hiden variables! N Example p( hb, )? 10

11 p( hb, ) p( hb,) = p(,hb,) p(hb,) p(fo hb,) = α p(fo,hb,) = α p(fo,hb,,,do) do Evidence: H, LO Query: FO Hidden: P, DO = α p(fo)p()p( FO)p(do FO,)p(hb do) do Any optimizations? p( hb, ) p( hb,) = p(,hb,) p(hb,) p(fo hb,) = α p(fo,hb,) = α p(fo,hb,,,do) do Evidence: H, LO Query: FO Hidden: P, DO = α p(fo)p()p( FO)p(do FO,)p(hb do) do = α p(fo) p( FO) p() p(do FO,) p(hb do) do p(fo hb,) = α p(fo) p( FO) p() p(do FO,) p(hb do) do p(fo hb,) = α p(fo) p( FO) p() p(do FO,) p(hb do) do p(fo) p( FO) Any optimizations? p(fo) p( FO) Idea: calculate from the bottom up p() p( ) p() p( ) p(do FO,) p( do FO,) p(do FO,) p( do FO,) p(do FO,) p( do FO,) p(do FO,) p( do FO,) p(hb do) p(hb do) p(hb do) p(hb do) p(hb do) p(hb do) p(hb do) p(hb do) 11