CS 473: Artificial Intelligence ayes Nets: Independence A ayes net is an efficient encoding of a probabilistic model of a domain ecap: ayes Nets Questions we can ask: Inference: given a fixed N, what is P( e)? epresentation: given a N graph, what kinds of distributions can it encode? ieter Fox Modeling: what N is most appropriate for a given domain? [hese slides were created by an Klein and Pieter Abbeel for CS188 Intro to AI at UC erkeley. All CS188 materials are available at http://ai.berkeley.edu.] ayes Nets epresentation Conditional Independences Probabilistic Inference Learning ayes Nets from ata Conditional Independence and are independent if and are conditionally independent given (Conditional) independence is a property of a distribution ayes Nets: Assumptions Assumptions we are required to make to define the ayes net when given the graph: P (xi x1 xi 1) =P (xi parents(i)) eyond above chain rule à ayes net conditional independence assumptions Often additional conditional independences hey can be read off the graph Important for modeling: understand assumptions made when choosing a ayes net graph Independence in a N Important question about a N: Are two nodes independent given certain evidence? If yes, can prove using algebra (tedious in general) If no, can prove with a counter example Question: are and necessarily independent? Answer: no. Example: low pressure causes rain, which causes traffic. can influence, can influence (via ) Addendum: they could be independent: how? 1
-separation: Outline -separation: Outline Study independence properties for triples Analyze complex cases in terms of member triples -separation: a condition / algorithm for answering such queries Causal Chains Causal Chains his configuration is a causal chain Guaranteed independent of? No! One example set of CPs for which is not independent of is sufficient to show this independence is not guaranteed. his configuration is a causal chain Guaranteed independent of given? Low pressure causes rain causes traffic, high pressure causes no rain causes no traffic : Low pressure : ain : raffic In numbers: P( +y +x ) = 1, P( -y - x ) = 1, P( +z +y ) = 1, P( -z -y ) = 1 : Low pressure : ain : raffic es! Evidence along the chain blocks the influence Common Cause his configuration is a common cause Guaranteed independent of? No! Common Cause his configuration is a common cause Guaranteed and independent given? : Project due One example set of CPs for which is not independent of is sufficient to show this independence is not guaranteed. : Project due Project due causes both forums busy and lab full : Forums busy : Lab full In numbers: P( +x +y ) = 1, P( -x -y ) = 1, P( +z +y ) = 1, P( -z -y ) = 1 : Forums busy : Lab full es! Observing the cause blocks influence between effects. 2
Common Effect he General Case Last configuration: two causes of one effect (v-structures) : aining : allgame Are and independent? es: the ballgame and the rain cause traffic, but they are not correlated Still need to prove they must be (try it!) Are and independent given? No: seeing traffic puts the rain and the ballgame in competition as explanation. his is backwards from the other cases : raffic Observing an effect activates influence between possible causes. he General Case eachability General question: in a given N, are two variables independent (given evidence)? Solution: analyze the graph Any complex example can be broken into repetitions of the three canonical cases ecipe: shade evidence nodes, look for paths in the resulting graph Attempt 1: if two nodes are connected by an undirected path not blocked by a shaded node, then they are not conditionally independent Almost works, but not quite Where does it break? Answer: the v-structure at doesn t count as a link in a path unless active L Active / Inactive Paths -Separation Question: Are and conditionally independent given evidence variables {}? es, if and d-separated by Consider all (undirected) paths from to No active paths = independence! A path is active if each triple is active: Causal chain A à à C where is unobserved (either direction) Common cause A ß à C where is unobserved Common effect (aka v-structure) A à ß C where or one of its descendents is observed All it takes to block a path is a single inactive segment Active riples Inactive riples Query: Check all (undirected!) paths between and If one or more active, then independence not guaranteed Otherwise (i.e. if all paths are inactive), then independence is guaranteed? 3
Example Example L es es es es Example Structure Implications Variables: : aining : raffic : oof drips S: I m sad Questions: es S Given a ayes net structure, can run d- separation algorithm to build a complete list of conditional independences that are necessarily true of the form his list determines the set of probability distributions that can be represented Computing All Independences opology Limits istributions Given some graph topology G, only certain joint distributions can be encoded he graph structure guarantees certain (conditional) independences {,,,,, } { } (here might be more independence) Adding arcs increases the set of distributions, but has several costs Full conditioning can encode any distribution {} 4
ayes Nets epresentation Summary ayes nets compactly encode joint distributions Guaranteed independencies of distributions can be deduced from N graph structure -separation gives precise conditional independence guarantees from graph alone A ayes net s joint distribution may have further (conditional) independence that is not detectable until you inspect its specific distribution epresentation ayes Nets Conditional Independences Probabilistic Inference Enumeration (exact, exponential complexity) Variable elimination (exact, worst-case exponential complexity, often better) Probabilistic inference is NP-complete Sampling (approximate) Learning ayes Nets from ata 5