A Differential Approach to Inference in Bayesian Networks

Transcription

1 Dierentil pproh to Inerene in Byesin Networks esented y Ynn Shen shenyn@mi.pitt.edu Outline Introdution Oeriew o lgorithms or inerene in Byesin networks (BN) oposed new pproh How to represent BN s multi-rite polynomil? How to nswer queries? How to represent polynomil using rithmeti iruits? How to generte rithmeti iruits? Conlusions 2

2 Bkground Byesin network Direted yli grph (DG) Conditionl proility tles (CPT) Reiew o three lsses o inerene lgorithms Conditioning Vrile elimintion Tree lustering BN with n nodes nd tree width w O (n ep(w)) in time nd spe 3 Introdution new pproh to inerene in BN The proility distriution o BN is represented s polynomil oilisti queries re nswered y eluting nd dierentiting the polynomil Polynomil is represented s n rithmeti iruit, whih n e eluted nd dierentited in time nd spe liner in its size. BN with n nodes nd tree width w, iruit n e uilt in O (n ep(w)) in time nd spe 4

3 Network Polynomil Let X e rile; U e its prents in BN Eidene inditors Network prmeters u Represent the onditionl proility i ~ e (eidene) otherwise P r ( u) By the Chin Rule, ( ) u~ u 5 Network Polynomil lse.5.5 B C ( ) u~ u B lse lse lse lse B C lse lse lse lse C ( ).5.2 6

4 Polynomil o Network N... u~ ( e ) ( e) u lse.5.5 B BN with n inry nodes n lse 2 Eidene Reple terms (instntitions) lse lse lse B B C lse lse lse lse C C e,,,,,, ) ( ( ) ).5.2. ( ). 7 Derities wrt. Eidene Inditors... /..4. Prtil derities o.4. How to ompute? Set inditor,.2 the network polynomil Conditioning on eent.. t eidene ( )

5 Derities wrt. Eidene Inditors For eery rile X nd eidene e in Byesin network, () e (,e X ) Where, e X denotes the suset o instntition e pertining to riles not ppering in X. Eidene e ( ) (, B ) (, ) 9 Derities wrt. Eidene Inditors (posterior mrginls) For eery rile X nd eidene e, X E : ( e) () e e ( e) ( e) () e () e () Eidene e () e... () e / Prtil derities o the network polynomil t eidene, ( )..5

6 Derities wrt. Eidene Inditors (posterior mrginls) For eery rile X nd eidene e: ( e X ) () e ( e X ) Eidene e () e () e ( e ) () () e () e Derities wrt. Network Prmeters nd Seond Prtil Derities For eery mily XU, nd eidenee, () e (, u, ) () u e u For eery pir o riles X, Y, nd eidene e, when X Y, 2 y () e (, y, e XY ) (2) For eery mily XU, riley, nd eidene e, 2 () e (, u, y, e ) (3) u Y u y For eery pir o milies XU, YV, nd eidene e, when u y, 2 u y u y e () e (, u, y,, ) (4) 2

7 How to Represent Polynomil Using n rithmeti Ciruit? n rithmeti iruit oer riles is rooted, direted yli grph. Le nodes: numeri onstnts or riles in Other nodes: multiplition nd ddition opertions Size: # o edges 3 n rithmeti Ciruit Emple B C 4

8 How to Dierentite the Ciruit? Two registers r() dr() p p i is the root node, I re other hildren o prent p. Initiliztion: dr() is initilized to zero eept or root where dr() p ( ) p is multiplition node, then Upwrd-pss: t node, ompute the lue ( o nd p ) store it in r() p is n ddition node, then Downwrd-pss: t node nd or eh prent p, inrement dr() y dr(p) i p is n ddition node; ) I is not the root node, nd hs prent p, y hin rule, dr( p) r( i p is multiplition node, where re the other hildren o p. p 5 Upwrd-pss.5.. Eidene

9 Downwrd-pss dr( p) Eidene.4 dr( p ) r( (.5.2) ) The Compleity o Dierentiting Ciruits Upwrd-pss: Time: liner in the iruit size Downwrd-pss Time is liner only when eh multiplition node hs ounded numer o hildren r r ( ) ( p) () I r, r () when r need two dditionl its () Time: per multipition node to # o gurntee the method tkes time whih is liner in the iruit size 8

10 How to Generte rithmeti Ciruit? Gol: generte the smllest iruit possile; Oer gurntees on the ompleity o iruits Two lsses o methods: Eploit the glol struture o BN Eploit the lol struture (the speii lues o onditionl proilities) 9 Ciruits tht Eploit Glol Struture Eh jointree emeds n rithmeti iruit tht omputes the network polynomil. ssuming we he jointree or the gien network, reer to Deinition 5 or generting iruits sed on jointrees. I network hs n nodes nd treewidth w, then the iruit ompleity is O(n ep(w)) I the jointree hs luster o lrge size, sy 4, then the emedded rithmeti iruit will e intrtle. 2

11 Ciruits tht Eploit Lol Struture I the onditionl proilities o the BN ehiit some lol struture: whether some proilities or (logil onstrint) whether some proilities in the sme tle re equl (ontet-speii independene)..5 lse.5 B C Eploit glol struture B lse lse B C lse lse lse lse Eploit lol struture lse lse C Ciruits tht Eploit Lol Struture (reduing the prolem to logil resoning) Three oneptul steps: Enoding multi-liner untion using propositionl theory Ftoring the propositionl enoding (logil orm d- DNNF, reer to [Drwihe 22]) Etrting n rithmeti iruit Logil onstrints led to signiint redutions in the size o iruits 22

12 Conlusions new pproh or inerene in Byesin networks whih is sed on eluting nd dierentiting rithmeti iruits Susumes the jointree pproh The ompleity o inerene is sensitie to oth the glol nd lol struture o Byesin networks 23