Algebraic Classification of Small Bayesian Networks

Transcription

1 GROSTAT VI, Menton IUT STID p. 1 Algebraic Classification of Small Bayesian Networks Luis David Garcia, Michael Stillman, and Bernd Sturmfels lgarcia@math.vt.edu Virginia Tech

2 GROSTAT VI, Menton IUT STID p. 2 Bayesian Networks Let G be a directed acyclic graph with n nodes. The nodes represent random variables, denoted X 1,..., X n. The arrows represent causal dependencies among the variables The joint probability distribution is defined as: p(x 1, x 2,..., x n ) = n p(x i x pai ) i=1 p(x 1, x 2, x 3, x 4 ) = p(x 4 )p(x 3 x 4 )p(x 2 x 3 )p(x 1 x 3 )

3 GROSTAT VI, Menton IUT STID p. 3 Conditional Independence Relations X 1 X 3 X X 1 X 2 X X 3 X 2 The set of directed global Markov relations, global(g), is the set of independent statements A B C, for any triple A, B, C of disjoint subsets of vertices of G such that A and B are d-separated by C. The set of directed local Markov relations of G is the set of independence statements local(g) = {X i nd(x i ) pa(x i ) : i = 1, 2,..., n},

4 GROSTAT VI, Menton IUT STID p. 4 Conditional Independence Relations (3) Local: {X 1 {X 2, X 4 } X 3, X 2 {X 1, X 4 } X 3 } Global: Local {X 4 {X 1, X 2 } X 3 }

5 GROSTAT VI, Menton IUT STID p. 5 Ideals, Varieties, and Independent Statements Let X 1,..., X n be discrete random variables, where X i takes values in [d i ] = {1, 2,..., d i }. Let D = [d 1 ] [d 2 ] [d n ] so that R D denotes the real vector space of n-dimensional tables of format d 1 d n. Let p u1 u 2 u n be an indeterminate representing the probability of X 1 = u 1, X 2 = u 2,..., X n = u n.

6 GROSTAT VI, Menton IUT STID p. 6 Ideals, Varieties, and Independent Statements (2) Let A B C be a conditional independence statement, where A, B and C are pairwise disjoint subsets of {X 1,..., X n }. Let I A B C denote de ideal of RD generated by the homogeneous quadratic polynomials p(a, b, c)p(a, b, c) p(a, b, c)p(a, b, c) for every pair a, a of instances of A, every pair b, b of instances of B, and for every instance c of C. p(a, b, c) denotes de marginalization of p(x 1,..., x n ) over the variables in the complement of A B C.

7 GROSTAT VI, Menton IUT STID p. 7 Ideals, Varieties, and Independent Statements (3) Let M be the independence model M = {A (1) B (1) C (1),..., A (m) B (m) C (m) }. Then I M = I A (1) B (1) C (1) + + I A (m) B (m) C (m) The independence variety is the set V (I M ) of common zeros in C D of the polynomials in I M. V (I M ) is the set of all d 1 d n -tables with complex number entries which satisfy the conditional independence statements in M. V (I M + p 1 ) is the subset of the probability simplex specified by the model M. Where p denotes the sum of all unknowns.

8 GROSTAT VI, Menton IUT STID p. 8 Ideals, Varieties, and Independent Statements (4) 2 local(g) = {X 1 {X 2, X 4 } X 3, X 2 {X 1, X 4 } X 3 } global(g) = local(g) {X 4 {X 1, X 2 } X 3 } Let d 1 = d 2 = d 4 = 2 and d 3 arbitrary The ideal I local(g) = I global(g) is generated by the 2 2-subdeterminants of the following d 3 matrices ( p11k1 p 11k2 p 12k1 p 12k2 ) ( p11k1 p 11k2 p 21k1 p 21k2 ) p 21k1 p 21k2 p 22k1 p 22k2 p 12k1 p 12k2 p 22k1 p 22k2 For each k [d 3 ], the corresponding quadratic binomials define the Segre Variety P 1 P 1 P 1 P 7 V (I M ) is the join of d 3 Segre varieties P 1 P 1 P 1 P 7.

9 GROSTAT VI, Menton IUT STID p. 9 Factorization Theorem Theorem (Factorization Theorem). Let G be a directed acyclic graph and P a probability distribution on V (G), the following are equivalent: DF: P admits a recursive factorization according to G DG: P obeys the Directed Global Markov Property DL: P obeys the Directed Local Markov Proterty For any integer r [n] and u i [d i ], denote the marginalization over the first r random variables as: p ++ +ur+1 u n := d 1 d 2 i 1 =1 i 2 =1 d r i r =1 p i1 i 2 i r u r+1 u n. Denote by p the product of all of these linear forms. The equation of p = 0 defines a hyperplane arrangement in R D. I local(g) is prime locally outside this hyperplane arrangement, and hence so is I global(g).

10 GROSTAT VI, Menton IUT STID p. 10 Factorization Theorem (2) Theorem. ( Ilocal(G) : p ) = ( I global(g) : p ) = ker(φ). The prime ideal ker(φ) is called the distinguished component. It is the set of all homogeneous polynomial functions on R D which vanish on all probabilitity distributions that factor according to G. Theorem. The following four subsets of the probability simplex coincide: V (I local(g) + p 1 ) = V (I global(g) + p 1 ) = V (kernel(φ)) = image(φ ).

11 GROSTAT VI, Menton IUT STID p. 11 Bayesian Networks on three random variables Theorem. For any Bayesian network G on three discrete random variables, the ideal I local(g) is prime, and it has a quadratic Gröbner basis. Graph Local/Global Markov property Independence ideal {2, 3}, 2 {1, 3}, 3 {1, 2} I Segre , 1 2 3, 1 {2, 3} I 1 {2,3} I I I 2 3

12 GROSTAT VI, Menton IUT STID p. 12 Bayesian Networks on four random variables Theorem. Of the 30 local Markov ideals on four random variables, 22 are always prime, five are not prime but always radical and three are not radical

13 GROSTAT VI, Menton IUT STID p. 13 A Prime example: Local(G) = {1 2 {3, 4}, 2 {1, 3} 4, 3 {2, 4}}. I local(g) is binomial in p ijkl with i {+, 2,..., d 1 }. The generators are p i1 j 2 k 1 lp i2 j 1 k 2 l p i1 j 1 k 1 lp i2 j 2 k 2 l, and p +j1 k 2 l 1 p +j2 k 1 l 2 p +j1 k 1 l 1 p +j2 k 2 l 2. The S-pairs within each group reduce to zero by the Gröbner basis property of the 2 2-minors of a generic matrix. The given set of irreducible quadrics is a reverse lexicographic Gröbner basis. The lowest variable is not a zero-divisor, and hence by symmetry none of the variables p ijkl is a zero-divisor. Since ( I local(g) : p ) = ker(φ), then I local(g) coincides with the prime ideal ker(φ).

14 GROSTAT VI, Menton IUT STID p. 14 A Radical example: I local(g) is the irredundant intersection of 2 d 2 1 primes. local(g) = {1 {3, 4} 2, 2 4 3}. I local(g) is a binomial ideal in p ijkl with i {+, 2, 3,..., d 1 }. The minimal primes are indexed by proper subsets of [d 2 ]. For each subset σ we introduce the monomial prime M σ = p +jkl : j σ, k [d 3 ], l [d 4 ] The complementary monomial m σ = j [d 2 ]\σ k [d 3 ] l [d 4 ] p +jkl, The ideal P σ = ( (I local(g) + M σ ) : m σ These ideals are prime, and the union of their varieties is irredundant and equals the variety of I local(g). ).

15 GROSTAT VI, Menton IUT STID p. 15 A Non-radical example: Network Local(G) = { 1 {3, 4} 2, 3 4 }. I local(g) is generated by p i1 jk 2 l 2 p i2 jk 1 l 1 p i1 jk 1 l 1 p i2 jk 2 l 2, and p ++k1 l 2 p ++k2 l 1 p ++k1 l 1 p ++k2 l 2. Let d 1 = d 2 = d 3 = 2 and d 4 = 3. I local(g) is generated by 33 quadratic polynomials in 24 unknowns. The degree reverse lexicographic Gröbner basis of this ideal consists of 123 polynomials of degree up to 8. I local(g) is the intersection of the distinguished component and the P -primary ideal Q = I 1 {3,4} 2 + P 2, where P is the prime ideal generated by the 12 linear forms p +jkl.

16 GROSTAT VI, Menton IUT STID p. 16 Global Markov Relations on Four nodes Theorem. Of the 30 global Markov ideals on four random variables, 26 are always prime, one is not prime but always radical and three are not radical

17 GROSTAT VI, Menton IUT STID p. 17 Global Markov Relations on Five nodes Theorem. Of the 301 global Markov ideals on five binary random variables, 220 are prime, 68 are radical but not prime, and 13 are not radical. # of components # of ideals mike/bayes/global5.html I global(g138 ) has 207 minimal primes, and 37 embedded primes. Each of the 207 minimal primary components are prime.

18 GROSTAT VI, Menton IUT STID p. 18 Hidden Variables and Higher Secant Varieties Let G be a BN on n discrete random variables and let P G = ker(φ) be its homogeneous prime ideal. The variables corresponding to the nodes r + 1,..., n are hidden, The observable probabilities are p i1 i 2 i r ++ + = j r+1 [d r+1 ] j r+2 [d r+2 ] j n [d n ] p i 1 i 2 i r j r+1 j r+2 j n. Let D = [d 1 ] [d r ] and R[D ] R[D] generated by p i1 i 2 i r Let π : R D R D denote the canonical linear epimorphism induced by the inclusion of R[D ] in R[D]. π(v 0 (P G )) π(v (P G )) 0 π(v (P G )) π(v (P G )) R D.