Junction Tree Algorithm 1. David Barber

Transcription

1 Juntion Tr Algorithm 1 David Barbr Univrsity Collg London 1 Ths slids aompany th book Baysian Rasoning and Mahin Larning. Th book and dmos an b downloadd from Fdbak and orrtions ar also availabl on th sit. Fl fr to adapt ths slids for your own purposs, but plas inlud a link th abov wbsit.

2 A gnral purpos infrn algorithm (?) Appliability Effiiny Th JTA dals with marginal infrn in multiply-onntd struturs. Th JTA an handl both Blif and Markov Ntworks. Th omplxity of th JTA an b vry high if th graph is multiply onntd. Provids an uppr bound on th omputational omplxity. May b that thr ar som problms for whih muh mor ffiint algorithms xist than th JTA.

3 Cliqu Graph A liqu graph onsists of a st of potntials, φ 1 (X 1 ),..., φ n (X n ) ah dfind on a st of variabls X i. For nighbouring liqus on th graph, dfind on sts of variabls X i and X j, th intrstion X s = X i X j is alld th sparator and has a orrsponding potntial φ s (X s ). A liqu graph rprsnts th funtion φ (X ) s φ s(x s ) Exampl φ(x 1 )φ(x 2 ) φ(x 1 X 2 ) X 1 X 1 X 2 X 2

4 Markov Nt Cliqu Graph p(a, b,, d) = φ(a, b, )φ(b,, d) Z b a d (a) a, b, b, b,, d (b) Figur : (a) Markov ntwork φ(a, b, )φ(b,, d). (b) Cliqu graph rprsntation of (a). Cliqu potntial assignmnts Th sparator potntial may b st to th normalisation onstant Z. Cliqus hav potntials φ(a, b, ) and φ(b,, d).

5 Transformation φ(a, b, )φ(b,, d) p(a, b,, d) = Z By summing w hav Zp(a, b, ) = φ(a, b, ) d φ(b,, d), Zp(b,, d) = φ(b,, d) a φ(a, b, ) Multiplying th two xprssions, ( w hav Z 2 p(a, b, )p(b,, d) = φ(a, b, ) d φ(b,, d) ) ( φ(b,, d) a φ(a, b, ) ) In othr words p(a, b,, d) = = Z 2 p(a, b,, d) a,d p(a, b, )p(b,, d) p(, b) p(a, b,, d) Cliqu potntial assignmnts Th sparator potntial may b st to p(b, ). Cliqus hav potntials p(a, b, ) and p(b,, d). Th liqus and sparators ontain th marginal distributions.

6 Markov Cliqu Graph Th transformation φ(a, b, ) p(a, b, ) φ(b,, d) p(b,, d) Z p(, b) Th usfulnss of this rprsntation is that if w ar intrstd in th marginal p(a, b, ), this an b rad off from th transformd liqu potntial. JTA Th JTA is a systmati way of transforming th liqu graph potntials so that at th nd of th transformation th nw potntials ontain th marginals of th distribution. Th JTA will work by a squn of loal transformations Eah loal transformation will lav th Cliqu rprsntation invariant.

7 Absorption Considr nighbouring liqus V and W, sharing th variabls S in ommon. In this as, th distribution on th variabls X = V W is φ (V)φ (W) p(x ) = and our aim is to find a nw rprsntation φ (V) φ (W) p(x ) = ˆφ (V) ˆφ (W) ˆφ (V) ˆ ˆφ (W) ˆ in whih th potntials ar givn by ˆφ (V) = p(v), ˆφ (W) = p(w), ˆ = p(s) W an xpliitly work out th nw potntials as funtion of th old potntials: p(w) = p(x ) = φ (V)φ (W) V\S = φ (W) φ (V) V\S V\S and p(v) = W\S p(x ) = W\S φ (V)φ (W) = φ (V) W\S φ (W)

8 Absorption W say that th lustr W absorbs information from lustr V. First w dfin a nw sparator φ (S) = V\S φ (V) and rfin th W potntial using φ (V) φ (S) φ (W) φ (W) = φ (W) φ (S) Invarian Th advantag of this intrprtation is that th nw rprsntation is still a valid liqu graph rprsntation of th distribution sin φ (V)φ (W) φ (S) = φ φ (V)φ (W) (S) φ(s) φ (S) = φ (V)φ (W) = p(x )

9 Absorption Aftr W absorbs information from V thn φ (W) ontains th marginal p(w). Similarly, aftr V absorbs information from W thn φ (V) ontains th marginal p(v). Aftr th sparator S has partiipatd in absorption along both dirtions, thn th sparator potntial will ontain p(s). Proof φ (S) = W\S φ (W) = W\S φ (W)φ (S) = Continuing, w hav th nw potntial φ (V) givn by φ (V) = φ (V)φ (S) φ (S) = {W V}\S = φ (V) W\S φ (W)φ (S)/ φ (S) W\S φ (V)φ (W) = p(v) φ (W)φ (V) = p(s)

10 Absorption Shdul on a Cliqu Tr A 3 4 B C D E 9 8 F For a valid shdul, mssags an only b passd to a nighbour whn all othr mssags hav bn rivd. Mor than on valid shdul may xist.

11 Forming a Cliqu Tr x 1 x 1, x 3 x 2 x 3, x 2, (a) (b) p(x 1, x 2, x 3, ) = φ(x 1, )φ(x 2, )φ(x 3, ) Th liqu graph of this singly-onntd Markov ntwork is multiply-onntd, whr th sparator potntials ar all st to unity. For absorption to work, w nd a singly-onntd liqu graph.

12 Forming a Cliqu Tr p(x 1, x 2, x 3, ) = φ(x 1, )φ(x 2, )φ(x 3, ) Rxprss th Markov ntwork in trms of marginals. First w hav th rlations p(x 1, ) = p(x 1, x 2, x 3, ) = φ(x 1, ) φ(x 2, ) φ(x 3, ) x 2,x 3 x 2 x 3 p(x 2, ) = p(x 1, x 2, x 3, ) = φ(x 2, ) φ(x 1, ) φ(x 3, ) x 1,x 3 x 1 x 3 p(x 3, ) = p(x 1, x 2, x 3, ) = φ(x 3, ) φ(x 1, ) φ(x 2, ) x 1,x 2 x 1 x 2 Taking th produt of th thr marginals, w hav p(x 1, )p(x 2, )p(x 3, ) ( ) 2 = φ(x 1, )φ(x 2, )φ(x 3, ) φ(x 1, ) φ(x 2, ) φ(x 3, ) } x 1 x 2 x 3 {{ } p() 2

13 Forming a Cliqu Tr This mans that th Markov ntwork an b xprssd in trms of marginals as p(x 1, x 2, x 3, ) = p(x 1, )p(x 2, )p(x 3, ) p( )p( ) Hn a valid liqu graph is also givn by x 1, x 1, x 3, x 2, x 3, x 2, If a variabl (hr ) ours on vry sparator in a liqu graph loop, on an rmov that variabl from an arbitrarily hosn sparator in th loop. Providd that th original Markov ntwork is singly-onntd, on an always form a liqu tr in this mannr.

14 Juntion Tr ab d f g h Running Intrstion Proprty A Cliqu Tr is a Juntion Tr if, for ah pair of nods, V and W, all nods on th path btwn V and W ontain th intrstion V W. Any singly-onntd Markov Ntwork an b transformd into a Juntion Tr. Thanks to th running intrstion proprty, loal onsistny of marginals propagats to global marginal onsistny.

15 Blif Nt Markov Nt a b a b d d f f g h g h Moralisation Form a link btwn all unmarrid parnts.

16 Markov Nt Juntion Tr ab ab d f d f g h g h Form th liqu graph Idntify a maximal wight spanning tr of th liqu graph. (Th wight of th dg is th numbr of variabls in th sparator)

17 Absorption a b ab d d f f g h g h Assign potntials to JT liqus. φ (ab) = p(a)p(b)p( a, b), φ (d) = p(d)p( d, ) φ (f) = p(f ), φ (g) = p(g ), φ (h) = p(h ) All sparator potntials ar initialisd to unity. Not that in som instans it an b that a juntion tr liqu is assignd to unity. Carry out absorption using a valid shdul. Marginals an thn b rad of th transformd potntials.