Bayesian Networks 3 D-separation. D-separation

Transcription

1 ayesian Networks 3 D-separation 1 D-separation iven a graph, we would like to read off independencies The converse is easier to think about: when does an independence statement not hold? g. when can X influence Y? 2 1

2 When can X influence Y? D-separation Direct connection: Indirect connection: X Y Indirect causal effect: Indirect evidential effect: Y Z X ommon cause: Y Z X ommon effect (v-structure): X Z Y Note: Z is observed as evidence 3 xplaining way Let s take a closer look at the common effect case (also known as explaining away): Difficulty Intelligence rade When rade is not observed: You can t really say anything about the Intelligence of the student given the Difficulty of the course When rade is observed eg. a : Difficulty and Intelligence are not independent g. if we observe rade = and Difficulty = Low, we tend to believe Intelligence = Low 4 2

3 D-separation Difficulty Intelligence rade Letter What happens if we observe Letter = weak as evidence? Indicates that student had a low rade Intelligence and Difficulty are now not independent (as in the previous slide) 5 D-separation When influence can flow from X to Y via Z, we say that the trail X Z Y is active (otherwise it is blocked): ausal trail: ctive if and only if Z is not observed vidential trail: ctive if and only if Z is not observed ommon cause: ctive if and only if Z is not observed ommon effect: ctive if and only if either Z or one of Z s descendants is observed 6 3

4 D-separation ll the previous cases deal with 3 node trails. Suppose we have a longer trail: X 1 X 2... X n irst, ignore the arrows. We will designate that we don t care about the arrow direction by using X 1 X 2... X n 7 D-separation or influence to flow from X 1 to X n, it needs every two-edge trail along the trail to allow influence to flow ie. Take X i-1 X i X i+1 Put the original arrows back in, and it must match the patterns on the right Or one of the descendants of Z is observed 8 4

5 D-separation Difficulty Intelligence rade ST Letter (xamples) onsider the trail D I S If Z={ }, the trail is not active (D I not active) If Z={L} the trail is active If Z={L,I} the trail is not active (I blocks the trail I S) 9 D-separation D-separation: Let X, Y, Z be three sets of nodes in. We say that X and Y are d-separated given Z, denoted d-sep (X; Y Z), if there is no active trail between any node X X and Y Y given Z. Use I() to denote the set of independencies that correspond to d-separation: I() = {(X Y Z) : d-sep (X;Y Z)} This set is also called the set of global Markov independencies 10 5

6 D-separation xercises 11 D-separation Recipe To determine if (X Y ), ignore the directions of the arrows, find all paths between X and Y Now pay attention to the arrows. Determine if the paths are blocked according to the 3 cases If all the paths are blocked, X and Y are d- separated given Which means they are conditionally independent given 12 6

7 locked Paths X Y ase 1 X Y ase 2 X Y ase 3 Note: is not observed 13 D-separation xamples ( )? D H 14 7

8 D-separation xamples ( )? Yes. Notice the two (undirected) paths between and This path from to is blocked by (ase 1) This path from to is blocked by, which is not in the evidence set (ase 3) 15 D-separation xamples ( )? D H 16 8

9 D-separation xamples ( )? Yes This path from to is blocked by (ase 2) This path from to is blocked by, which is not an evidence node (ase 3) 17 D-separation xamples ( D )? D H 18 9

10 D-separation xamples ( D )? No ut this path from to D is not blocked. This is because (which is a descendant of ) is in the evidence set (ase 3) D D This path from to D is blocked by (not in evidence set) (ase 3) and by (ase 2) 19 D-separation xamples ( {, })? D H 20 10

11 D-separation xamples ( {, })? Yes This path from to is blocked by (ase 2) This path from to is blocked by (ase 2) 21 Soundness and ompleteness 22 11

12 Soundness and ompleteness Soundness: If X and Y are d-separated given Z, then we are guaranteed that they are in fact conditionally independent given Z under distribution P ormally: If a distribution P factorizes according to, then I() I(P) (See proof in text section ) 23 Soundness and ompleteness ompleteness (informally): d-separation detects all possible independencies ut independencies in which distribution? Need to be more specific. irst, we define faithfulness 24 12

13 Soundness and ompleteness distribution P is faithful to if, whenever (X Y Z) I(P), then d-sep (X;Y Z). (Informally) any independence in P is reflected in the d-separation properties of the graph. 25 Soundness and ompleteness What about this definition of completeness? or any distribution P that factorizes over, we have that P is faithful to ; that is, if X and Y are not d-separated given Z in, then X and Y are dependent in all distributions P that factorize over. This is false: some independencies cannot be read off from the graph structure 26 13

14 Soundness and ompleteness ounterexample: onsider a distribution P over and, where and are independent. One possible I-map for P is shown below: P( ) false false 0.4 false true 0.6 true false 0.4 true true Soundness and ompleteness Need a weaker definition of completeness Theorem 3.4: Let be a N structure. If X and Y are not d-separated given Z in, then X and Y are dependent in Z in some distribution P that factorizes over

15 Soundness and ompleteness or almost all parameterizations P of the graph, the d-separation test finds the independencies that hold for P. If we have a distribution P that satisfies more independencies than I() slight perturbation of the PDs will almost always eliminate these extra independencies Such independencies are typically rare / accidental 29 15