Outline. CSE 473: Artificial Intelligence Spring Bayes Nets: Big Picture. Bayes Net Semantics. Hidden Markov Models. Example Bayes Net: Car

Transcription

1 CSE 473: rtificial Intelligence Spring 2012 ayesian Networks Dan Weld Outline Probabilistic models (and inference) ayesian Networks (Ns) Independence in Ns Efficient Inference in Ns Learning Many slides adapted from Dan Klein, Stuart Russell, ndrew Moore & Luke ettlemoyer 1 ayes Nets: ig Picture wo problems with using full joint distribution tables as our probabilistic models: Unless there are only a few variables, the joint is W too big to represent explicitly Hard to learn (estimate) anything empirically about more than a few variables at a time ayes nets: a technique for describing complex joint distributions (models) using simple, local distributions (conditional probabilities) More properly called graphical models We describe how variables locally interact Local interactions chain together to give global, indirect interactions Formally: ayes Net Semantics set of nodes, one per variable directed, acyclic graph CP for each node CP = Conditional Probability able Collection of distributions over, one for each combination of parents values ayes net = opology (graph) + Local Conditional Probabilities 1 n Hidden Markov Models N5 E 1 E 2 E 3 E 4 E N5 Example ayes Net: Car n HMM is defined by: Initial distribution: ransitions: Emissions: 1

2 Probabilities in Ns ayes nets implicitly encode joint distributions s a product of local conditional distributions o see what probability a N gives to a full assignment, multiply all the relevant conditionals together: Example: Independence N fair, independent coin flips: h 0.5 h 0.5 h 0.5 t 0.5 t 0.5 t 0.5 his lets us reconstruct any entry of the full joint Not every N can represent every joint distribution he topology enforces certain independence assumptions Compare to the exact decomposition according to the chain rule! Example: Coin Flips Independence N independent coin flips wo variables are independent if:? 1 2 n his says that their joint distribution factors into a product of two simpler distributions nother form: No interactions between variables: absolute independence We write: Independence Independence wo variables are independent if: wo variables are independent if: his says that their joint distribution factors into a product of two simpler distributions nother form: his says that their joint distribution factors into a product two simpler distributions nother form:? We write: We write: Independence is a simplifying modeling assumption Empirical joint distributions: at best close to independent What could we assume for {Weather, raffic, Cavity, oothache}? 2

3 Example: Independence? P warm 0.25 W P cold W warm sun 0.4 warm sun 0.15 warm rain 0.1 warm rain 0.10 cold sun 0.2 cold sun 0.45 W P cold rain 0.3 cold rain 0.30 sun 0.6 rain 0.4 Conditional Independence Unconditional (absolute) independence very rare (why?) Conditional independence is our most basic and robust form of knowledge about uncertain environments: What about fire, smoke, alarm? Conditional Independence re & independent? P( )? P(), Conditionally Independent Given C P(,C) = P( C) C = spots P()=(.25+.5)/2 =.375 C P( C) =.25 P(,C)=.25 P()=.75 P( )=( )/3 =.3333 Daniel S. Weld 15 Daniel S. Weld 17 Example: larm Network Variables : urglary : larm goes off M: Mary calls J: John calls E: Earthquake! How big is joint distribution? 2 n -1 = 31 parameters Example: larm Network P() +b b J P(J ) +a +j 0.9 +a j 0.1 a +j 0.05 a j 0.95 urglary John h calls larm Earthqk Mary calls M P(M ) +a +m 0.7 +a m 0.3 a +m 0.01 a m 0.99 E P(E) +e e Only 10 params E P(,E) +b +e +a b +e a b e +a b e a 0.06 b +e +a 0.29 b +e a 0.71 b e +a b e a

4 Example: raffic II Let s build a graphical model Variables : raffic R: It rains L: Low pressure D: Roof drips : allgame C: Cavity Changing ayes Net Structure he same joint distribution can be encoded in many different ayes nets nalysis question: given some edges, what other edges do you need to add? One answer: fully connect the graph etter answer: don t make any false conditional independence assumptions Example: Independence For this graph, you can fiddle with (the CPs) all you want, but you won t be able to represent any distribution in which the flips are dependent! Example: Coins Extra arcs don t prevent representing independence, just allow non-independence h 0.5 t 0.5 h 0.5 t 0.5 ll distributions h 0.5 h 0.5 h 0.5 t 0.5 t 0.5 t 0.5 dding unneeded arcs isn t wrong, it s just inefficient h h 0.5 t h 0.5 h t 0.5 t t 0.5 opology Limits Distributions Given some graph topology G, only certain joint distributions can be encoded he graph structure guarantees certain (conditional) independences (here might be more independence) dding arcs increases the set of distributions, but has several costs Full conditioning can encode any distribution Independence in a N Important question about a N: re two nodes independent given certain evidence? If yes, can prove using algebra (tedious in general) If no, can prove with a counter example Example: Question: are and independent? nswer: no. Example: low pressure causes rain, which causes traffic. Knowledge about may change belief in, Knowledge about may change belief in (via ) ddendum: they could be independent: how? 4

5 Causal Chains Common Parent his configuration is a causal chain : Low pressure : Rain : raffic nother basic configuration: two effects of the same parent re and independent? Is independent of given? : Project due : Forum busy : Lab full es! Evidence along the chain blocks the influence Common Parent Common Effect nother basic configuration: two effects of the same parent re and independent? re and independent given? : Project due Last configuration: two causes of one effect (v-structures) re and independent? es: the ballgame and the rain cause traffic, but they are not correlated Still need to prove they must be (try it!) : Forum busy : Lab full : Raining : allgame es! : raffic Observing the cause blocks influence between effects. Common Effect he General Case Last configuration: two causes of one effect (v-structures) re and independent? es: the ballgame and the rain cause traffic, but they are not correlated Still need to prove they must be (try it!) re and independent given? No: seeing traffic puts the rain and the ballgame in competition as explanation! his is backwards from the other cases Observing an effect activates influence between possible causes. : Raining : allgame : raffic ny complex example can be analyzed using these three canonical cases General question: in a given N, are two variables independent (given evidence)? Solution: analyze the graph 5

6 Reachability (D-Separation) Question: re and conditionally independent given evidence vars {}? es, if and separated by Look for active paths from to No active paths = independence! path is active if each triple is active: Causal chain C where is unobserved (either direction) Common cause C where is unobserved Common effect (aka v-structure) C where or one of its descendents is observed ll it takes to block a path is a single inactive segment ctive riples Inactive riples Example: Independent? es R Example: Independent? Example es es D L R Variables: R: Raining : raffic D: Roof drips S: I m sad Questions: R S D es es Summary ayes nets compactly encode joint distributions Guaranteed independencies of distributions can be deduced from N graph structure D-separation gives precise conditional independence guarantees from graph alone ayes net s joint distribution may have further (conditional) independence that is not detectable until you inspect its specific distribution 6