Undirected Graphical Models 4 ayesian Networks and Markov Networks 1 ayesian Networks to Markov Networks 2 1
Ns to MNs X Y Z Ns can represent independence constraints that MN cannot MNs can represent independence constraints that Ns cannot Given a N, how can we represent the distribution P as a parameterized Markov network? 3 Ns to MNs Proposition 4.7: Let be a ayesian network over X. Then P (X) is a Gibbs distribution defined by the factors where { X i } X X i X i P ( X i Parents( X i )) 4 2
Ns to MNs How do we create an undirected graph that is an I-map for the set of factors in the N? or each factor: dd an edge between X i and its parents dd an edge between all the parents of X i 5 Ns to MNs The moral graph M[G] of a ayesian network structure G over X is the undirected graph over X that contains an undirected edge between X and Y if: There is a directed edge between them (in either direction) Or X and Y are both parents of the same node ifficulty Intelligence ifficulty Intelligence Grade ST Grade ST Letter Moralized Letter 6 3
Ns to MNs Proposition 4.8: Let G be any ayesian network graph. The moralized graph M[G] is a minimal I-map for G. [proof omitted here] 7 Ns to MNs The addition of the moralizing edges to the MN H leads to the loss of independence information implied by the graph structure X Y X Y (X Y) lost Z Moralized Z ut this only happens if moralization adds new edges into the graph 8 4
Ns to MNs ayesian network G is moral if it contains no immoralities ie. for any pair of variables X, Y that share a child, there is a covering edge between X and Y overing edge X Y Z 9 Ns to MNs If the directed graph G is moral, then its moralized graph M[G] is a perfect map of G. [proof omitted] Unfortunately, very few directed graphs are moral 10 5
Markov Networks to ayesian Networks 11 MNs to Ns How do we find a N that is a minimal I- map for a MN? Much harder (both conceptually and computationally) Resulting N might be larger than the MN 12 6
MNs to Ns xample: ind a ayesian network I-map for the given Markov Network rom Section 3.4.1 in text: To find a minimal I-map for a distribution P: Pick a variable ordering or each variable X i in the ordering: ind some minimal subset U of {X 1,, X i-1 } to be X i s parents in G such that {X i {X 1,, X i-1 } - U U} Note: (X i {X 1,, X i-1 }) is trivially true 13 MNs to Ns xample: ind a ayesian network I-map for the given Markov Network Ordering:,,,,, 14 7
MNs to Ns xample: ind a ayesian network I-map for the given Markov Network Ordering:,,,,, ( not ), so must add as parent of 15 MNs to Ns xample: ind a ayesian network I-map for the given Markov Network Ordering:,,,,, ( not ), so must dd as parent of. ( {,}) 16 8
MNs to Ns xample: ind a ayesian network I-map for the given Markov Network Ordering:,,,,, nd so on Notice the edges added which result in a set of triangles. The graph is chordal (defn to follow) 17 MNs to Ns Let X 1 X 2 X 3 X 4 be a loop in a graph chord in the loop is an edge connecting X i and X j for two nonconsecutive nodes X i, X j. n undirected graph H is said to be chordal if any loop X 1 X 2 X k X 1 for k 4 has a chord 18 9
MNs to Ns Theorem 4.10: Let H be a Markov network structure, and let G be any ayesian network minimal I-map for H. Then G can have no immoralities. [Proof omitted] 19 MNs to Ns ny nontriangulated loop of length at least 4 in a ayesian network graph necessarily contains an immorality. orollary 4.3: Let H be a Markov network structure, and let G be any minimal I-map for H. Then G is necessarily chordal. 20 10
MNs to Ns Turning a MN to a N requires triangulation: adding enough edges to a graph to make it chordal. Leads to the loss of independence information When converting from N MN, the (moralizing) edges added are in some sense implicitly there (ie. each factor in the N involves a node and its parents) When converting from MN N, we can introduce a large number of edges which results in very large cliques 21 hordal Graphs 22 11
hordal Graphs Theorem 4.11: Let H be a nonchordal Markov network. Then there is no ayesian network G which is a perfect map for H (ie. I(H) = I(G)) [Proof omitted] 23 hordal Graphs ut is there a ayesian network G which is a perfect map for a chordal Markov network H? onsider connected graphs (disconnected graph case is straightforward) Need to show that any connected chordal graph H can be decomposed into a tree of cliques (a tree whose nodes are maximal cliques in H) Structure of the tree encodes independencies in H 24 12
hordal Graphs Let H be a connected undirected graph Let 1,.., k be the set of maximal cliques in H Let T be any tree-structured graph whose nodes correspond to the maximal cliques 1,, k H T 25 hordal Graphs Let i, j be two cliques in the tree that are directly connected by an edge efine S i,j = i j to be a sepset between i and j Let W <(i,j) (W <(j,i) ) be all of the variables that appear in any clique on the i ( j ) side of the edge ach edge decomposes X into three disjoint sets: W <(i,j), W <(j,i), and S i,j. 1 = {,,} 2 = {,,} Sepset = {, } W <(1,2) = {} W <(2,1) = {} 26 13
hordal Graphs We say that a tree T is a clique tree for H if: ach node corresponds to a clique in H, and each maximal clique in H is a node in T ach sepset S i,j separates W <(i,j) and W <(j,i) in H Note: ach separator S i,j renders its two sides conditionally independent in H 27 hordal Graphs Theorem 4.12: very undirected chordal graph H has a clique tree T. [Proof omitted] Theorem 4.13: Let H be a chordal Markov network. Then there is a ayesian network G such that I(H) = I(G).[Proof omitted] 28 14
hordal Graphs irected Graphical Models Undirected Graphical Models hordal Graphs 29 15