Artificial Intelligence CS 6364

Transcription

1 Artificil Intllignc CS 6364 rofssor Dn Moldovn Sction 3 Bysin Ntworks

2 Bysin Ntworks Dirctd grphs in which ch nod is nnottd with quntittiv probbility informtion. Th full spcifiction is:. A st of rndom vribls mks up th nods of th ntwork. Vribls my b discrt or continuous. 2. A st of dirctd links or rrows conncts pirs of nods. If thr is n rrow from nod X to nod Y, X is sid to b prnt to X 3. Th grph hs no dirctd cycls it is dirctd cyclic grph, or DAG 478

3 Empl Dntl world: Vribls: Toothch, Cvity, Ctch, Wthr Intuitiv id: n rrow btwn X nd Y mns tht X hs dirct influnc on Y 479

4 Empl 2 Nw burglr lrm is instlld t hom. It is firly rlibl t dtcting burglry, but it lso rsponds on occsion to minor rthquks. Thr r 2 nighbors: John nd Mry, who hv promisd to cll you t work whn thy hr th lrm. John lwys clls whn h hrs th lrm, but somtims confuss th tlphon ringing with th lrm clls nd clls thn, too. Mry liks rthr loud music nd somtims misss th lrm togthr. Givn th vidnc of who hs or hs not clld, w would lik to stimt th probbility of burglry. 480

5 Conditionl Distributions Dt Structurs: Conditionl robbility Tbl CT Ech row in CT contins th conditionl probbility of ch nod vlu for conditioning cs. A conditioning cs is combintion of vlus for th prsnt nods A minitur tomic vnt roprty: ch row must sum to. Tht is prticulrly importnt for discrt, non-binry vribls. A nod without prnt hs CT with only on row: th prior probbility of ch possibl vlu of th vribl. 48

6 Smntics of Bysin Ntwork Two wys to undrstnding th smntics of Bysin ntwork:. S th ntwork s rprsnttion of th joint probbility distribution 2. Viw th Bysin ntwork s n ncoding of collction of conditionl indpndnc sttmnts Thy r quivlnt. Th first viw hlps in undrstnding how to construct ntworks. Th scond viw hlps in dsigning infrnc procdurs. 482

7 Rprsnting th full joint distribution A Bysin Ntwork provids dscription of th domin. Evry ntry in th full joint probbility distribution joint cn b clcultd from th informtion in th ntwork. A Gnric ntry in th joint is th probbility of conjunction of prticulr ssignmnts to ch vribl, such s: X... X n bbrvitd s,..., n Th vlu for th ntry is givn by n,..., n i prnts X i i n whr prntsx i dnots th spcific vlus of th vribls in rntsx i Ech configurtion X... X n n rprsnts possibl world 483

8 Rprsnting th Joint Ech ntry in th joint is rprsntd by th product of th pproprit lmnts of th CT in th Bysin Ntwork th CTs provid dcomposd rprsnttion of th joint Empl: Comput th probbility tht th lrm hs soundd, but nithr burglry nor n rthquk hs occurrd, nd both John nd Mry cll Dnot: b burglry, rthquk, lrm hs soundd, j John, m Mry j m b j m b b

9 Th Joint for Empl 2 rt Th Joint Tbl for vribls b,,, j, m b j m rob

10 Th Joint for Empl 2 rt 2 Th Joint Tbl for vribls b,,, j, m b j m rob

11 Th Joint for Empl 2 rt W cn comput ny probbility E.g. b,,,j, m=rob0,,0,,0= b, =,b, / b, = / blu+grn. b j m rob = = = = = = = b j m rob = = = = = = = = = = = = = = = = = = = = = = = = = 0 487

12 ttrns of infrnc in Blif Nts W r intrstd in infrnc tht cn b computd dirctly from th ntwork, without th plicit computtion of n ponntilly long tbl. Considr spcil ntwork topology: th polytr A polytr is dirctd cyclic grph for which thr is just on pth long th undirctd grph rcs btwn ny two nods. Any nod connctd to nod Q will not b connctd to ny othr nod cpt through nod Q Th vidnc might includ vribls ssocitd with componnts bov Q E + or blow Q E -. Thn: roba,,a m E + = roba E + roba m E + A A m robb,,b k E-,Q= robb E-,Q robb k E-,Q Infrnc is bsd on Bys rul for ll n possibl worlds of vribl X: B Q B k robx= robxw robw + + robxw n robw n Thr r thr css: / No vidnc 2/ Evidnc bov th qury 3/ Evidnc blow th qury 488

13 Cs : No vidnc A qury vribl Q is givn, nd thr is no vidnc. Gol: comput Q Mthod: - / Comput th probbility of vribls A,, A m bov Q - 2/ Comput th probbility of ll possibl worlds w i bsd on A,, A m using: roba,,a m E + = roba E + roba m E + - Comput th probbility of Q using: robq= robqw robw + + robqw n robw n Empl: Q=B thn B=0.00 Q=E thn E = At nod A: B E A w T T = T F = F T = F F = A = = = thn A =

14 Cs : No vidnc cont_ Continu: Empl: B=0.00 E = At nod A: B E A w T T = T F = F T = F F = A = thn A = At nod J: A J w T F At nod M: A M w T J = F = J = M = = M =

15 Cs 2: Evidnc bov th qury A qury vribl Q is givn, nd Q is prt of vidnc QE + = 0 or ccording to vidnc b Q is t th top, thn Q is givn by th CT c Mthod: - / Comput th probbility of vribls A,, A m bov Q - 2/ Comput th probbility of ll possibl worlds w i bsd on A,, A m using: roba,,a m E + = roba E + roba m E + - Comput th probbility of Q using: robq= robqw robw + + robqw n robw n Empl: AB At nod A: B E A w T T =0.002 T F = F T = 0 F F = 0 AB = = = thn AB =

16 Empl 2 Empl2: MB At nod A: B E A w T T =0.002 T F = F T = 0 F F = 0 At nod M: AB = = = thn AB = A M w T F MB = = = MB = MB=

17 Cs 3: Evidnc blow th qury A qury vribl Q is givn, nd th vidnc E -. Th gol: comput QE - Mthod: Comput E Q Q Q E E Thus w nd to comput: / E - Q cs 2 2/ Q cs 3/ normliztion constnt Empl: BJM BJM = JB MB B At nod A: B E A w T T =0.002 T F = F T = 0 F F = 0 AB = = = thn AB =

18 Cs 3 Cont AB = AB = At nod J: A J w T F JB = JB = 0.5 Also B= 0.00 Now BJM = JB MB B= = = To comput w nd to comput lso BJM = J B M B B = = ; = At nod M: A M w T F MB = = = MB = BJM = <0.284, 0.76> 494

19 Ect infrnc Bsic tsk: comput th postrior probbility distribution for st of qury vribls, givn som obsrvd vnt X qury vribl, E st of vidnc vribls E,E 2,,E m nd prticulr obsrvd vnt Y non-vidnc vribls Y,Y 2,,Y l lso clld hiddn vribls Th complt st of vribls X={}E Y Typicl qury: X th postrior distribution 495

20 496 Empl Burglry ntwork: vnt JohnClls = tru MryClls = tru Ask if burglry hs occurrd: Burglry JohnClls=tru,MryClls=tru=<0.284,0.76> Infrnc by numrtion For th qury BurglryJohnClls=tru,MryClls=tru th hiddn vribls r Erthquk nd Alrm Th smntics of Bysin ntworks givs us: y y X X X,,, m j B j B m j B,,,,,,, m j b b m j b,, constnt?

21 497 Computing th ostrior m j b b m j c b b m j b,,, How do w comput this? m j b b m j b,, b=0.00 b= = =0.998 B E A w T T = T F = F T = 0 F F = 0 At nod A:

22 Eplntion b j, m b b, j m b=0.00 b=0.00 At nod A: 0.002= B E A w T T =0.002 T F = F T = 0 F F = 0 = = b, =0.95 b, =0.05 b= = b, =0.94 b, =

23 499 Finl Structur m j b b m j b,, A J T 0.9 F 0.05 A M T 0.7 F 0.0

24 Enumrt-Ask 500

25 50 Improvmnts m j b b m j b,, rptitions

26 502 Vribl Elimintion Algorithm Id: Elimint rptd clcultions. How? Do th clcultions onc nd sv th rsults for ltr us Evlut prssions in th right-to-lft ordr bottom-up in th tr m j B B m j B,, m m A f M j j A f J B E A J M fctors

27 How dos on comput th fctors??? A st of initil fctors: B f B E f E T 0.00 F T F B E A f A T T T 0.95 T T F 0.05 T F T 0.94 T F F 0.06 F T T 0.29 F T F 0.7 F F T 0.0 F F F 0.99 A J f J T T 0.9 T F 0. F T 0.05 F F 0.95 A M f M T T 0.7 T F 0.3 F T 0.0 F F 0.90 For Bj,m w know tht J=T nd M=T,thus f M m 0.70 A m 0.0 f J j 0.90 A j

28 Computing th fctors For B, f A A,B,E= mtri From f M A, f J A nd f A A,B,E w must sum out A! B E A f A T T T 0.95 T T F 0.05 T F T 0.94 T F F 0.06 f B, E f, B, E f f AJM A f, B, E f f A J J M M F T T 0.29 F T F 0.7 F F T 0.0 F F F 0.99 f A, B, E f f J M Us pointwis product 504

29 ointwis roduct - it is not mtri multipliction! - it is not lmnt-by-lmnt multipliction! ointwis product of two fctors f nd f 2 yilds nw fctor f whos vribls r th union of vribls f nd f 2. Suppos th two fctors hv vribls Y,,Y k in common. Thn w hv: f X,..., X f j, Y,..., Y X k,..., X, Z j,.., Z, Y,..., Y k f 2 Y,..., Y k, Z,.., Z 505

30 Computing ointwis roduct f X,..., X j, Y,..., Yk, Z,.., Z f X,..., X j, Y,..., Yk f2 Y,..., Yk, Z,.., Z If ll th vribls r binry thn f nd f 2 hv 2 j+k nd 2 k+l ntris rspctivly, nd th pointwis product hs 2 j+k+l ntris. Empl: Considr f A,B nd f 2 B,C A B f A,B B C 2 f B,C A B C 3 f A,B,C T T.3 T T.2 T T T.3.2 T F.7 T F.8 T T F.3.8 F T.9 F T.6 T F T.7.6 F F. F F.4 T F F.7.4 F T T.9.2 F T F.9.8 F F T..6 F F F

31 Th Burglry World f B, E f, B, E f f AJM A J f, B, E f f A f, B, E f f A J J M M M B E A f A T T T 0.95 T T F 0.05 T F T 0.94 T F F 0.06 F T T 0.29 F T F 0.7 F F T 0.0 F F F 0.99 A J f J A M f M T T 0.9 T T 0.7 = F T 0.0 F T 0.05 B E f AJM T T T F F T F F

32 Continu f EAJM B f f B, f f B, E AJM E AJM B E f AJM T T E f E B F EAJM T F T F = F T T F F F

33 Computing th nswr B j, m f B B f B EAJM B F EAJM B f B T 0.00 F T F

34 Elimintion-Ask 50

35 A Mthod for Constructing Bysin Ntworks n,..., prnts X n i i i dfins wht givn Bysin Ntwork mns It dos not plin how to build Bysin Ntwork such tht th rsulting joint distribution is good rprsnttion of givn domin Howvr, it implis crtin dditionl indpndnc rltionships tht cn b usd to guid th knowldg nginr in constructing th topology of th ntwork 5

36 52 Stps. Rwrit th joint distribution in trms of conditionl probbility using th product rul: 2. Rpt th procss, rducing ch conjunctiv probbility to conditionl probbility nd smllr conjunction n i i i n X prnts,...,,...,,...,,..., n n n n bcoms n i i i n n n n n 2 2,...,...,...,,...,,..., th chin rul

37 Smntics Bcus,..., n n n,..., n n2,...,... 2 n i i i,..., W hv: i i,..., rnts i i if rnts i {i-, } Nods must b lbld in ny ordr tht is consistnt with th prtil ordr implicit in th grph structur 53

38 Construction Ruls Th prnts of nod X i should contin ll nods X,,X i- tht dirctly influnc X i Th corrct ordr in which to dd nods: dd root cuss first, thn vribls thy influnc, nd so on, until rching th lvs tht hv no dirct cusl influnc on othr vribls Wht hppns if w chos th wrong ordr of dding nods? 54

39 roblmtic Ntworks If w dd th nods in th ordr: MryClls, JohnClls, Alrm, Burglry, Erthquk w obtin mor complictd ntwork: Th procss: Add MryClls no prnts 2 Add JohnClls if MryClls, probbly th lrm wnt off it mks it mor likly JohnClls JohnClls nds MryClls s prnt 3 Add Alrm if both John nd Mry cll, it is mor likly th lrm wnt off both MryClls nd JohnClls r prnts 55

40 Mor roblms 4 Add Burglry if w know th lrm stt, thn th cll from John or Mry might giv us informtion bout our phon ringing or Mry s music, but not bout th burglry Burglry Alrm, JohnClls, MryClls Burglry, Alrm only Alrm is prnt 5 Adding Erthquk: if th lrm is on, it is mor likly tht thr hs bn n rthquk. But if w know tht thr hs bn burglry, thn tht plins th lrm. Both Alrm nd Burglry r prnts 56

41 Wht is th roblm? W do not hv only cusl links! Thr r lso links from symptoms to Cuss! 57

42 Conditionl Indpndnc in Bysin Ntworks A nod is conditionlly indpndnt of its prdcssors, givn its prnts bcus: n,..., prnts X n i i Conditionl indpndnc is lso dicttd by topologicl smntics:. A nod is conditionlly indpndnt of its non-dscndnts, givn its prnts. Empl: JohnClls is indpndnt of Burglry nd Erthquk, givn th vlu of Alrm 2. A nod is conditionlly indpndnt of ll othr nods in th ntwork, givn its prnts, childrn nd childrn s prnts clld Mrkov blnkt i 58

43 Empls A nod X is conditionlly indpndnt of its non-dscndnts.g., th Z ij s givn its prnts th U i s shown in th gry r. b A nod X is conditionlly indpndnt of ll othr nods in th ntwork givn its Mrkov blnkt th gry r 59

44 Efficint Rprsnttion of Conditionl Distribution Evn whn nod hs only K prnts th CT nds O2 k mmbrs this is th worst-cs scnrio. Usully, rltions btwn prnts nd child r dscribd by cnonicl distribution tht fits som stndrd pttrn th CT cn b spcifid by nming th pttrn nd fw prmtrs Empl: dtrministic nods. A dtrministic nod hs its vlu spcifid ctly by th vlu of its prnts, with no uncrtinty - Empl: Cndin US Micn disjunction North-Amricn 520

45 Noisy-OR Uncrtin rltionship chrctrizd by noisy logicl rltionships. Noisy- OR rltion: gnrliztion of th logicl OR Empl In propositionl logic: Fvr = Cold Flu Mlri Noisy-OR: llows for uncrtinty bout th bility of ch prnt to cus th child to b tru. Th rltion btwn prnt nt child my b inhibitd.q. Cold = tru but Fvr = fls Th inhibition of ch prnt is indpndnt of th inhibition of ny othr prnt All possibl cuss r listd 52

46 Empl: robbilitis of individul inhibitions fvr fvr fvr cold, flu, mlri 0.6 cold, flu, mlri 0.2 cold, flu, mlri 0. From this informtion, th ntir CT cn b built Cold Flu Mlri Fvr Fvr F F F F F T F T F F T T = T F F T F T = T T F = T T T =

47 Clustring Algorithms If w wnt to comput postrior probbilitis for ll vribls in th ntwork th vribl limintion lgorithm is not fficint. Solution: Considr clustring lgorithms lso known s join tr lgorithms. Th id: Clustr individul mods of th ntwork to form clustr nods th rsulting ntwork is poly tr. 523