Probabilistic Reasoning in Bayesian Networks: A Relational Database Approach

Transcription

1 Probbilistic Resoning in Byesin Networks: A Reltionl Dtbse Approch S.K.M. Wong, D. Wu nd C.J. Butz Deprtment of Computer Science, University of Regin Regin Ssktchewn, Cnd S4S 0A2 Abstrct. Probbilistic resoning in Byesin networks is normlly conducted on junction tree by repetedly pplying the locl propgtion whenever new evidence is observed. In this pper, we suggest to tret probbilistic resoning s dtbse queries. We dpt method for nswering queries in dtbse theory to the setting of probbilistic resoning in Byesin networks. We show n effective method for probbilistic resoning without repeted ppliction of locl propgtion whenever evidence is observed. 1 Introduction Byesin networks [3] hve been well estblished s model for representing nd resoning with uncertin informtion using probbility. Probbilistic resoning simply mens computing the mrginl distribution for set of vribles, or the conditionl probbility distribution for set of vribles given evidence. A Byesin network is normlly trnsformed through morliztion nd tringultion into junction tree on which the probbilistic resoning is conducted. One of the most populr methods for performing probbilistic resoning is the so-clled locl propgtion method [2]. The locl propgtion method is pplied to the junction tree so tht the junction tree reches consistent stte, i.e., mrginl distribution is ssocited with ech node in the junction tree. Probbilistic resoning cn then be subsequently crried out on this consistent junction tree [2]. In this pper, by exploring the intriguing reltionship between Byesin networks nd reltionl dtbses [5], we propose new pproch for probbilistic resoning by treting it s dtbse query. This new pproch hs severl slient fetures. (1) It dvoctes using hypertree insted of junction tree for probbilistic resoning. By selectively pruning the hypertree, probbilistic resoning cn be performed by employing the locl propgtion method once nd cn then nswer ny queries without nother ppliction of locl propgtion. (2) The structure of fixed junction tree my be fvourble to some queries but not to others. By using the hypertree s the secondry structure, we cn dynmiclly prune the hypertree to obtin the best choice for nswering ech query. (3) Finlly, this dtbse perspective of probbilistic resoning provides mple opportunities for well developed techniques in dtbse theory, especilly

2 techniques in query optimiztion, to be dopted for chieving more efficient probbilistic resoning in prctice. The pper is orgnized s follows. We briefly review Byesin networks nd locl propgtion in Sect. 2. In Sect. 3, we first discuss the reltionship between hypertrees nd junction trees nd the notion of Mrkov network. We then present our proposed method. We discuss the dvntges of the proposed method in Sect. 4. We conclude our pper in Sect Byesin Networks nd the Locl Propgtion We use U = {x 1,..., x n } to represent set of discrete vribles. Ech x i tkes vlues from finite domin denoted V xi. We use cpitl letters such s X to represent subset of U nd its domin is denoted by V X. By XY we men X Y. We write x i = α, where α V xi, to indicte tht the vrible x i is instntited to the vlue α. Similrly, we write X = β, where β V X, to indicte tht X is instntited to the vlue β. For convenience, we write p(x i ) to represent p(x i = α) for ll α V xi. Similrly, we write p(x) to represent p(x = β) for ll β V X. A Byesin network (BN) defined over set U = {x 1,..., x n } of vribles is tuple (D, C). The D is directed cyclic grph (DAG) nd the C = {p(x i p(x i )) x i U} is set of conditionl probbility distributions (CPDs), where p(x i ) denotes the prents of node x i in D, such tht p(u) = n i=1 p(x i p(x i )). This fctoriztion of p(u) is referred to s BN fctoriztion. Probbilistic resoning in BN mens computing p(x) or p(x E = e), where X E =, X U, nd E U. The fct tht E is instntited to e, i.e., E = e, is clled the evidence. The DAG of BN is normlly trnsformed through morliztion nd tringultion into junction on which the locl propgtion method is pplied [1]. After the locl propgtion finishes its execution, mrginl distribution is ssocited with ech node in the junction tree. p(x) cn be esily computed by identifying node in the junction tree which contins X nd then do mrginliztion; the probbility of p(x E = e) cn be similrly obtined by first incorporting the evidence E = e into the junction tree nd then propgting the evidence so tht the junction tree reches n updted consistent stte from which the probbility p(x E = e) cn be computed in the sme fshion s we compute p(x). It is worth emphsizing tht this method for computing p(x E = e) needs to pply the locl propgtion procedure every time when new evidence is observed. Tht is, it involves lot of computtion to keep the knowledge bse consistent. It is lso worth mentioning tht it is not evident how to compute p(x) nd p(x E = e) when X is not subset of ny node in the junction tree [6]. A more forml technicl tretment on the locl propgtion method cn be found in [1]. The problem of probbilistic resoning, i.e., computing p(x E = e), cn then be equivlently stted s computing p(x, E) for the set X E of vribles [6]. Henceforth, probbilistic resoning mens computing the mrginl distribution p(w) for ny subset W of U.

3 3 Probbilistic Resoning s Dtbse Queries In this section, we show tht the tsk of computing p(x) for ny rbitrry subset X cn be conveniently expressed nd solved s dtbse query. 3.1 Hypertrees, Junction Trees, nd Mrkov Networks A hypergrph is pir (N, H), where N is finite set of nodes (ttributes) nd H is set of edges (hyperedges) which re rbitrry subsets of N [4]. If the nodes re understood, we will use H to denote the hypergrph (N, H). We sy n element h i in hypergrph H is twig if there exists nother element h j in H, distinct from h i, such tht ( (H {h i })) h i = h i h j. We cll ny such h j brnch for the twig h i. A hypergrph H is hypertree [4] if its elements cn be ordered, sy h 1, h 2,..., h n, so tht h i is twig in {h 1, h 2,..., h i }, for i = 2,..., n 1. We cll ny such ordering hypertree (tree) construction ordering for H. It is noted for given hypertree, there my exist multiple tree construction orderings. Given tree construction ordering h 1, h 2,..., h n, we cn choose, for ech i from 2 to N, n integer j(i) such tht 1 j(i) i 1 nd h j(i) is brnch for h i in {h 1, h 2,..., h i }. We cll function j(i) tht stisfies this condition brnching for the hypertree H with h 1, h 2,..., h n being the tree construction ordering. For given tree construction ordering, there might exist multiple choices of brnching functions. Given tree construction ordering h 1, h 2,..., h n for hypertree H nd brnching function j(i) for this ordering, we cn construct the following multiset: L(H) = {h j(2) h 2, h j(3) h 3,..., h j(n) h n }. The multiset L(H) is the sme for ny tree construction ordering nd brnching function of H [4]. We cll L(H) the seprtor set of the hypertree H. Let (N, H) be hypergrph. Its reduction (N, H ) is obtined by removing from H ech hyperedge tht is proper subset of nother hyperedge. A hypergrph is reduced if it equls its reduction. Let M N be set of nodes of the hypergrph (N, H). The set of prtil edges generted by M is defined to be the reduction of the hypergrph {h M h H}. Let H be hypertree nd X be set of nodes of H. The set of prtil edges generted by X is lso hypertree [7]. A hypergrph H is hypertree if nd only if its reduction is hypertree [4]. Henceforth, we will tret ech hypergrph s if it is reduced unless otherwise noted. It hs been shown in [4] tht given hypertree, there exists set of junction trees ech of which corresponds to prticulr tree construction ordering nd brnching function for this ordering. On the other hnd, given junction tree, there lwys exists unique corresponding hypertree representtion whose hyperedges re the nodes in the junction tree. Exmple 1. Consider the hypertree H shown in Fig 1 (i), it hs three corresponding junction trees shown in Fig 1 (ii), (iii) nd (iv), respectively. On the other hnd, ech of the junction trees in Fig 1 (ii), (iii) nd (iv) corresponds to the hypertree in Fig 1 (i). The hypertree in Fig 1 (v) will be explined lter.

4 b b b b b c d c d c d c d d (i) (ii) (iii) (iv) (v) Fig. 1. (i) A hypertree H, nd its three possible junction tree representtions in (ii), (iii), nd (iv). The pruned hypertree with respect to b, d is in (v). We now introduce the notion of Mrkov network. A (decomposble) Mrkov network (MN) [3] is pir (H, P), where () H = {h i i = 1,...,n} is hypertree defined over vrible set U where U = h i H h i with h 1,..., h n s tree construction ordering nd j(i) s the brnching function; together with (b) set P = {p(h) h H} of mrginls of p(u). The conditionl independencies encoded in H men tht the jpd p(u) cn be expressed s Mrkov fctoriztion: p(u) = p(h 1 ) p(h 2 )... p(h n ) p(h 2 h j(2) )... p(h n h j(n) ). (1) Recll the locl propgtion method for BNs in Sect. 2. After finishing the locl propgtion without ny evidence, we hve obtined mrginls for ech node in the junction tree. Since junction tree corresponds to unique hypertree, the hypertree nd the set of mrginls (for ech hyperedge) obtined by locl propgtion together define Mrkov network [1]. 3.2 An Illustrtive Exmple For Computing p(x) As mentioned before, the probbilistic resoning cn be stted s the problem of computing p(x) where X is n rbitrry set of vribles. Consider Mrkov network obtined from BN by locl propgtion nd let H be its ssocited hypertree. It is trivil to compute p(x) if X h for some h H, s p(x) = h X p(h). However, it is not evident how to compute p(x) in the cse tht X h. Using the Mrkov network obtined by locl propgtion, we use n exmple to demonstrte how to compute p(x) for ny rbitrry subset X U by selectively pruning the hypertree H. Exmple 2. Consider the Mrkov network whose ssocited hypertree H is shown in Fig 1 (i) nd its Mrkov fctoriztion s follows: p(bcd) = p(b) p(c) p(d) p() p(). (2) Suppose we wnt to compute p(bd) where the nodes b nd d re not contined by ny hyperedge of H. We cn compute p(bd) by mrginlizing out ll the

5 irrelevnt vribles in the Mrkov fctoriztion of the jpd p(bcd) in eqution (2). Notice tht the numertor p(c) in eqution (2) is the only fctor tht involves vrible c. (Grphiclly speking, the node c occurs only in the hyperedge c). Therefore, we cn sum it out s shown below. p(bd) =, c = p(b) p(c) p(d) p() p() p(b) p(d) p() = p() p() = p(b) p(d) p() p() p(c) c p(b) p(d). (3) p() Note tht p(b) p(d) p() = p(bd) nd this is Mrkov fctoriztion. The bove summtion process grphiclly corresponds to deleting the node c from the hypertree H in Fig 1 (i), which results in the hypertree in Fig 1 (iv). Note tht fter the vrible c hs been summed out, there exists p() both s term in the numertor nd term in the denomintor. The existence of the denomintor term p() is due to the fct tht is in L(H). Obviously, this pir of p() cn be cnceled. Therefore, our originl objective of computing p(bd) cn now be chieved by working with this pruned Mrkov fctoriztion p(bd) = p(b) p(d) p() whose hypertree is shown in Fig 1 (iv). 3.3 Selectively Pruning the Hypertree of the Mrkov Network The method demonstrted in Exmple 2 cn ctully be generlized to compute p(x) for ny X U. In the following, we introduce method for selectively pruning the hypertree H ssocited with Mrkov network to the exct portion needed for computing p(x). This method ws originlly developed in the dtbse theory for nswering dtbse queries [7]. Consider Mrkov network with its ssocited hypertree H nd suppose we wnt to compute p(x). In order to reduce the hypertree H to the exct portion tht fcilittes the computtion of p(x), we first mrk those nodes in X nd repet the following two opertions to prune the hypertree H until neither is pplicble: (op1): delete n unmrked node tht occurs in only one hyperedge; (op2): delete hyperedge tht is contined in nother hyperedge. We use H to denote the resulting hypergrph. Note tht the bove procedure possesses the Church-Rosser property, tht is, the finl result H is unique, regrdless of the order in which (op1) nd (op2) re pplied [4]. It is lso noted tht the opertors (op1) nd (op2) cn be implemented in liner time [7]. It is perhps worth mentioning tht (op1) nd (op2) re grphicl opertors pplying to H. On the other hnd, the method in [8] works with BN fctoriztion nd it sums out irrelevnt vribles numericlly. Let H 0, H 1,..., H j,..., H m represent the sequence of hypergrphs (not necessrily reduced) in the pruning process, where H 0 = H, H m = H, 1 j m, ech H j is obtined by pplying either (op1) or (op2) to H j 1.

6 Lemm 1. Ech hypergrph H i, 1 i m, is hypertree. Lemm 2. L(H ) L(H). Due to lck of spce, the detiled proofs of the bove lemms nd the following theorem will be reported in seprte pper. For ech h i H, there exists h j H such tht h i h j. We cn compute p(h i ) s p(h i ) = h j h p(h j ). In other words, the mrginl p(h i ) cn be i computed from the originl mrginl p(h) supplied with the Mrkov network H. Therefore, fter selectively pruning H, we hve obtined the hypertree H nd mrginls p(h i ) for ech h i H. Theorem 1. Let (H, P) be Mrkov network. Let H be the resulting pruned hypertree with respect to set X of vribles. The hypertree H ={h 1, h 2,..., h k } nd the set P = {p(h i ) 1 i k} of mrginls define Mrkov network. Theorem 1 indictes tht the originl problem of computing p(x) cn now be nswered by the new Mrkov network defined by the pruned hypertree H. It hs been proposed [5] tht ech mrginl p(h) where h H cn be stored s reltion in the dtbse fshion. Moreover, computing p(x) from H cn be implemented by dtbse SQL sttements [5]. It is noted tht the result of pplying (op1) nd (op2) to H lwys yields Mrkov network which is different thn the method in [8]. 4 Advntges One slient feture of the proposed method is tht it does not require ny repeted ppliction of locl propgtion. Since the problem of computing p(x E = e) cn be equivlently reduced to the problem of computing p(x, E), we cn uniformly tret probbilistic resoning s merely computing mrginl distributions. Moreover, computing mrginl distribution, sy, p(w), from the jpd p(u) defined by BN, cn be ccomplished by working with the Mrkov fctoriztion of p(u), whose estblishment only needs pplying the locl propgtion method once on the junction tree constructed from the BN. It is worth mentioning tht Xu in [6] reched the sme conclusion nd proved theorem similr to Theorem 1 bsed on the locl propgtion technique on the junction tree. Computing p(w) in our proposed pproch needs to prune the hypertree H to the exct portion needed for computing p(w) s Exmple 2 demonstrtes if W h for ny h H. A similr method ws suggested in [6] by pruning the junction tree insted. Using hypertrees s the secondry structure insted of junction trees hs vluble dvntges s the following exmple shows. Exmple 3. Consider the hypertree H shown in Fig 1 (i), it hs three corresponding junction trees shown in Fig 1 (ii), (iii) nd (iv), respectively. The method in [6] first fixes junction tree, for exmple, sy the one in Fig 1 (ii). Suppose we need to compute p(bd), the method in [6] will prune the junction tree so tht ny irrelevnt nodes will be removed s we do for pruning hypertree. However, in this

7 junction tree, nothing cn be pruned out ccording to [6]. In other words, p(bd) hs to be obtined by the following clcultion: p(bd) = p(b) p() c p(c) p(d) p(). However, if we prune the hypertree in Fig 1 (i), the resulting pruned hypertree is shown in Fig 1 (v), from which p(bd) cn be obtined by eqution (3). Obviously, the computtion involved is much less. Observing this, one might decide to dopt the junction tree in Fig 1 (iii) s the secondry structure. This chnge fcilittes the computtion of p(bd). However, in similr fshion, one cn esily be convinced tht computing p(bc) using the junction tree in (iii), computing p(cd) using the junction tree in (iv) suffer exctly the sme problem s computing p(bd) using junction tree in (ii). In other words, regrdless of the junction tree fixed in dvnce, there lwys exists some queries tht re not fvored by the pre-determined junction tree structure. On the other hnd, the hypertree structure lwys provides the optiml pruning result for computing mrginl [7]. 5 Conclusion In this pper, we hve suggested new pproch for conducting probbilistic resoning from the reltionl dtbse perspective. We demonstrted how to selectively reduce the hypertree structure so tht we cn void repeted ppliction of locl propgtion. This suggests possible dul purposes dtbse mngement systems for both dtbse storge, retrievl nd probbilistic resoning. ACKNOWLEDGMENT. The uthors would like to thnk one of the reviewers for his/her helpful suggestions nd comments. References [1] C. Hung nd A. Drwiche. Inference in belief networks: A procedurl guide. Interntionl Journl of Approximte Resoning, 15(3): , October [2] S.L. Luritzen nd D.J. Spiegelhlter. Locl computtion with probbilities on grphicl structures nd their ppliction to expert systems. Journl of the Royl Sttisticl Society, 50: , [3] J. Perl. Probbilistic Resoning in Intelligent Systems: Networks of Plusible Inference. Morgn Kufmnn Publishers, Sn Frncisco, Cliforni, [4] G. Shfer. An xiomtic study of computtion in hypertrees. School of Business Working Ppers 232, University of Knss, [5] S.K.M. Wong, C.J. Butz, nd Y. Xing. A method for implementing probbilistic model s reltionl dtbse. In Eleventh Conference on Uncertinty in Artificil Intelligence, pges Morgn Kufmnn Publishers, [6] H. Xu. Computing mrginls for rbitrry subsets from mrginl representtion in mrkov trees. Artificil Intelligence, 74: , [7] Mihlis Ynnkkis. Algorithms for cyclic dtbse schemes. In Very Lrge Dt Bses, 7th Interntionl Conference, September 9-11, 1981, Cnnes, Frnce, Proceedings, pges 82 94, [8] N. Zhng nd Poole.D. Exploiting cusl independence in byesin network inference. Journl of Artificil Intelligence Reserch, 5: , 1996.