Lecture 12: May 09, Decomposable Graphs (continues from last time)

Transcription

1 596 Pat. Recog. II: Introduction to Graphical Models University of Washington Spring 00 Dept. of lectrical ngineering Lecture : May 09, 00 Lecturer: Jeff Bilmes Scribe: Hansang ho, Izhak Shafran(000). Decomposable Graphs (continues from last time) Theorem.. The following conditions are equivalent for an undirected graph G:. Decomposable graph;. hordal or triangulated graph; 3. very (α, β)-separator is complete. () () and () (3) were proved. We will start today s lecture by proving (3) (), or that if every (α, β)-separator is complete, then it is a chordal (or triangulated) graph. ([SLL996]) Proof. (3) (): ssume that every (α, β)-separator is complete. If G is complete, then it is decomposable by definition. However, if G is not complete, it has at least two non-adjacent vertices, α and β. Let be the (necessarily complete) minimal (α, β)-separator which divides V into two disjoint sets ) the α connected components in V \, ) [α] V \, the corresponding β connected components [β] V \, and 3) any other remaining vertices (lets name them D). We define the sets = [α] V \ D and B = [β] V \, so that so that, B, form a partition in V. n example is shown in figure.. is complete by assumption. The graph is decomposable if triple (, B, ) forms a decomposition of G and if the subgraphs G and G B are also decomposable. We already have a decomposition in (, B, ) since separates and B and is complete. To show that the subgraphs are also decomposable, we use induction. Let be the minimal (α, β ) separator in G for some α, β. If is a minimal (α, β )-separator in G (one possible choice of is shown in figure.), it is also the minimal separator in G. This is because no path from α to β would go through B witout intersecting, since separates from B and is complete. Therefore, is complete by assumption. But since V < n, by induction G is decomposable. similar argument can be used to show that the subgraph G B is also decomposable, and therefore G is decomposable. -

2 Lecture : May 09, 00 - igure.: = (α, β)-separator for G inally, the equivalence relation stated in theorem. is proved.. Junction Trees Now, we shall revisit the definition of junction tree and examine how it is related to the family of decomposable graphs. Definition.. Let be a collection of subsets of a finite set V and T a tree with as its node set. Then T is said to be a junction tree if any intersection of a pair, of sets in is contained in every node on the unique path in T between and. Junction tree of cliques is a collection ρ (set of subsets of nodes in V). Let τ be a tree with elements of ρ as the nodes. The τ is a junction tree if any intersection S ij = i j, i τ, j τ, or S i j k is true. Where k is any clique on the necessary path between i and j. B DG Moral Triangulated D B D B D G G G H I H I H I igure.: xamples of a DG, its moralized, and triangulated one igure. shows an original graph, the corresponding moralized graph, and one possible triangulation of the moralized graph. Note that the cliques of this last triangulated graph are {, B,, D}, {B,, D, }, {D,, G}, {B,, }, {, G, I}, and {,, H}

3 Lecture : May 09, 00-3 junction tree is such that in the graph of cliques (i.e., a graph where the cliques themselves are nodes in the graph), the junction tree is the maximum spanning tree. So in the following graph of cliques, the edges show the counts of the intersections of the cliques. We find the tree with maximum weight to get the junction tree (we will prove this later in the course). BD B BD 3 DG H IG igure.3: xample of the numbers of intersection igure.3 shows an example of the numbers of intersection, in which the numbers beside each node show the number of each connection. The junction tree is the graph of cliques and is the maximum spanning tree. So figure.4 shows a junction tree of the graph in figure.3. BD 3 BD B DG H IG igure.4: xample of a junction tree or some graphs (i.e., decomposable as we will see), junction trees may be constructed using its cliques as illustrated in figure.5. Junction trees have the property that probabilisitc inference is correct (as we will show later in the course). B D B BD D igure.5: xample of a junction tree s shown in figure.6, junction trees T need not be unique for a graph G.

4 Lecture : May 09, 00-4 B D igure.6: Various junction trees BD D D or BD D D Theorem.3. There exists a junction tree T of cliques for the graph G if and only if G is decomposable. This is one of the key theorems in this course. Let s prove it. Proof. The proof is by induction on number of cliques ([DLS999]). [] Junction tree of cliques exists decomposable: The theorem clearly holds for G with two cliques and. In such a case, triple (, B, S) of decomposable graph consists of S =, = \ S and B = \ S. S is complete and separates \ S with \ S. Suppose that the theorem holds for all graphs with at most k cliques and let G have k + cliques. ssume T is a junction tree of cliques for G. Take and adjacent in T. On cutting the link, T separates into two subtrees, T and T (since there can only be one path between two nodes in T). Let V i be the union of the nodes in T i for i =,, and let G i = G Vi. igure.7: Breaking the link (T {T, T }) [Q] Is G decomposable? The nodes in T i are then the cliques of G i, and T i is a junction tree for G. By the inductive assumption, G and G are decomposable. Let S =. The triple (V, V, S) will form a decomposition of G, if S is complete and separates V and V. We need to show. S is complete.;. S separates V from V ; To prove the first one, let v S = V V. There exists a clique i of G i for each i =,, with v i. The path from and passes through and. Therefore, v V V. But, V V. So, S = and complete.

5 Lecture : May 09, 00-5 igure.8: S = Now, to prove that S separates V and V, let s make the assumption that S does not separate V from V. If this is true, then there exists a path u, w, w,..., w k, v with each w i / S for all i (i.e., none of the nodes on this path can be in S under this assumption). w V V u S u igure.9: Under the assumption that S does not separates V and V, there is another path from u to v Then, there exists a clique containing the complete set u, w with where is a clique of T. Let = where i are cliques of T i and where we have = φ. However, note that we can t have which is equivalent to the condition that V, since u V. This implies that or equivalently that V. Therefore, w V and hence w V \ S. Repeating this argument for each w i along the path, we deduce that v V \ S. This contradicts the assumption. Therefore, S separates V and V. We have already shown that S is complete and from the inductive assumption, we know that the subgraphs from V and V are decomposable. Therefore, the triple (V, V, S) forms a decomposition and this completes the proof that every graph G for which a junction tree of cliques exists is decomposable. w V V u S u igure.0: V [] Decomposable junction tree of cliques exits : ssume that G is decomposable, and let (W, W, S) be a decomposition of G into proper decomposition subgraph G V, G V, where V i = W i S. Then at least one of V and V - say V has the form with and redefining V = where = \. Let i i satisfy S i. By hypothesis, we have a junction tree for T i for G i where, as before G i = G Vi. We form a tree T by linking T and T via and. Let v V. If v / V, then all cliques containing v are in, and so connected in T, hence in T. If v / V, then all cliques containing v are in, and so connected in T, hence in T. If v / V, then similarly for T because it is symmetric. nd if v S = V V, then v (. The cliques in i containing v are connected in T i, and include i. Since and are connected the result follows.

6 Lecture : May 09, 00-6 igure.: a tree T by linking T and T via and So, now we have a very important equivalence, decomposable chordal every (α, β)-separator complete junction tree of cliques exists. Note that the above proof also shows that an intersection S = between two neighboring nodes in a junction tree of cliques forms the minimal separator between two decomposable graph G, as illustrated in figure 0.7. s we shall see in later lectures, the cliques and the separators provide meaningful entities in inference computation. The cliques represent the joint (marginal) probability and the separators form the denominator in conditional probability distributions. [Q] Is S minimal? Yes. S is a minimal separator. igure.: Junction tree of cliques.3 Running Intersection Property (r.i.p) sequence of sets (,,..., k ) is said to have the running intersection property if, for all < j k, there is an i < j such that j (... j ) i. To interpret along the lines of predictive model, we have. H j = (... j ) represents history up to j;

7 Lecture : May 09, j \ H j = R j represents the residual or innovations in the run of cliques; 3. S j = H j j represents the separator set.; The r.i.p. says that j, S j i for some i < j igure.3: property of RIP well-ordered (e.g., breadth-first or depth-first order) ordering of the junction tree clqiues is sufficient for the r.i.p. B B BD B BD 3 B BD 4 5 BH BD igure.4: n example of well-ordered junction tree.4 Junction Tree onstruction lgorithm We can construct a junction tree from cliques of chordal graph order to satisfy r.i.p.

8 Lecture : May 09, 00-8 lgorithm: constructing a junction tree input:,,..., p (the cliques of a chorded graph satisfying RIP) step : associate each node in the graph with each clique step : for j =,..., p add an edge between i and j for some i < j such that S j i (S j = H j j ) or a junction tree, for any node v, cliques containing v must induce a connected tree, and there are two cases: ase : If v S j i, then v i and v j. Therefore it induces a tree which is necessarily connected and containing v. ase : If v S j, then v R j = j \ H j The proof of this can be found in Lauritzen s text. References [SLL996] S.L. LURITZN Graphical Models, Oxford Science Publications, 996 [DLS999] R.G. OWLL,.P. DVID, S.L. LURITZN, D.J. SPIGLHLTR Probabilistic Networks and xpert Systems, Springer Publications, 999