Probabilistic Reasoning; Graphical models

Transcription

1 Ovrvw. robablsc Rasonng/Graphcal modls. Imporanc Samplng 3. Markov Chan Mon Carlo: Gbbs Samplng 4. Samplng n prsnc of Drmnsm 5. Rao-Blackwllsaon 6. AND/OR mporanc samplng

2 Ovrvw. robablsc Rasonng/Graphcal modls. Imporanc Samplng 3. Markov Chan Mon Carlo: Gbbs Samplng 4. Samplng n prsnc of Drmnsm 5. Cus-basd Varanc Rducon 6. AND/OR mporanc samplng

3 robablsc Rasonng; Graphcal modls Graphcal modls: Baysan nwork consran nworks md nwork Qurs Eac algorhm usng nfrnc sarch and hybrds Graph paramrs: r-wdh cycl-cus w-cus

4 Qurs robably of vdnc or paron funcon var pa osror margnal blfs: Mos robabl Eplanaon n var j n n var j Z j j pa pa j j C * argma

5 Appromaon Snc nfrnc sarch and hybrds ar oo pnsv whn graph s dns; hgh rwdh hn: Boundng nfrnc: mn-buck and mn-clusrng Blf propagaon Boundng sarch: Samplng Goal: an anym schm 5

6 Ovrvw. robablsc Rasonng/Graphcal modls. Imporanc Samplng 3. Markov Chan Mon Carlo: Gbbs Samplng 4. Samplng n prsnc of Drmnsm 5. Rao-Blackwllsaon 6. AND/OR mporanc samplng

7 Ouln Dfnons and Background on Sascs Thory of mporanc samplng Lklhood wghng Sa-of-h-ar mporanc samplng chnqus 7

8 A sampl Gvn a s of varabls ={... n } a sampl dnod by S s an nsanaon of all varabls: S... n 8

9 How o draw a sampl? Unvara dsrbuon Eampl: Gvn random varabl havng doman {0 } and a dsrbuon = Task: Gnra sampls of from. How? draw random numbr r [0 ] If r < 0.3 hn s =0 Els s = 9

10 How o draw a sampl? Mul-vara dsrbuon L ={.. n } b a s of varabls Eprss h dsrbuon n produc form... n... n Sampl varabls on by on from lf o rgh along h ordrng dcad by h produc form. Baysan nwork lraur: Logc samplng 0

11 Samplng for rob. Infrnc Ouln Logc Samplng Imporanc Samplng Lklhood Samplng Choosng a roposal Dsrbuon Markov Chan Mon Carlo MCMC Mropols-Hasngs Gbbs samplng Varanc Rducon

12 Logc Samplng: No Evdnc Hnron 988 Inpu: Baysan nwork = { N } N- #nods T - # sampls Oupu: T sampls rocss nods n opologcal ordr frs procss h ancsors of a nod hn h nod slf:. For = 0 o T. For = 0 o N 3. sampl from pa

13 Logc samplng ampl No Evdnc // gnra sampl k.sampl.sampl 3.Sampl 4.Sampl 3 4 from from from 3 from

14 Logc Samplng w/ Evdnc Inpu: Baysan nwork = { N } N- #nods E vdnc T - # sampls Oupu: T sampls conssn wh E. For = o T. For = o N 3. sampl from pa 4. If n E and rjc sampl: 5. Goo Sp. 4

15 Logc Samplng ampl Evdnc : // gnra sampl k.sampl.sampl 3.Sampl 4. If 3 and sar from ohrws 5.Sampl 3 from from from 0 rjc sampl 4 3 from 4 3 5

16 Epcd valu and Varanc Epcd valu: Gvn a probably dsrbuon and a funcon g dfnd ovr a s of varabls = { n } h pcd valu of g w.r.. s E [ g ] g Varanc: Th varanc of g w.r.. s: Var [ g ] g E [ g ] 6

17 Mon Carlo Esma Esmaor: An smaor s a funcon of h sampls. I producs an sma of h unknown paramr of h samplng dsrbuon. Gvn..d. sampls S gˆ T T S h M oncarlo sma of g S S E T drawn from [g] s gvn by : 7

18 Eampl: Mon Carlo sma Gvn: A dsrbuon = g = 40 f quals 0 = 50 f quals. Esma E [g]= =47. Gnra k sampls from : 0000 gˆ 40# sampls 0 50# # sampls sampls

19 Ouln Dfnons and Background on Sascs Thory of mporanc samplng Lklhood wghng Sa-of-h-ar mporanc samplng chnqus 9

20 Imporanc samplng: Man da Eprss qury as h pcd valu of a random varabl w.r.. o a dsrbuon Q. Gnra random sampls from Q. Esma h pcd valu from h gnrad sampls usng a mon carlo smaor avrag. 0

21 Imporanc samplng for ˆ : ] [ \ Z Q z w T z w E z Q z E z Q z Q z z z Q z E Z L T Q Q z z whr z M oncarlo sma as : w can rwr Thn 0 0 sasfyng proposaldsrbuon L QZb a

22 roprs of IS sma of Convrgnc: by law of larg numbrs Unbasd. Varanc: ˆ T a. s w z. for T T Var E Q Q [ ˆ ] ˆ Var Q T N w z Var Q [ w z] T

23 roprs of IS sma of Man Squard Error of h smaor MSE Q ˆ E ˆ Q E ˆ [ ] Var ˆ Q Q ˆ Var Var Q Q [ w ] T Ths quany nclosd n h bracks s zro bcaus h pcd valu of h smaor quals h pcd valu of g

24 Esmang 4 E basd : Esma s ˆ ˆ Rao sma: IS. Ida : Esma numraor and dnomnaor by and 0 ohrws. z conans whch s f drac - dla funcon z b a L T k T k wz wz z z Q z E z Q z z E z z z k k k Q Q z z

25 roprs of h IS smaor for Convrgnc: By Wak law of larg numbrs as T Asympocally unbasd Varanc lm T E Hardr o analyz [ ] Lu suggss a masur calld Effcv sampl sz 5

26 Gnrang sampls from Q No rsrcons on how o Typcally prss Q n produc form: QZ=QZ QZ Z.QZ n Z..Z n- Sampl along h ordr Z..Z n Eampl: Z QZ =0.0.8 Z QZ Z = Z 3 QZ 3 Z Z =QZ 3 =0.50.5

27 Ouln Dfnons and Background on Sascs Thory of mporanc samplng Lklhood wghng Sa-of-h-ar mporanc samplng chnqus 7

28 Lklhood Wghng Fung and Chang 990; Shachr and o 990 Is an nsanc of mporanc samplng! Clampng vdnc+ logc samplng+ wghng sampls by vdnc lklhood Works wll for lkly vdnc! 8

29 Lklhood Wghng: Samplng Sampl n opologcal ordr ovr! Clamp vdnc Sampl pa pa s a look-up n CT! 9

30 Lklhood Wghng: roposal Dsrbuon 30 E E j j E E E E j j n E j j pa pa pa pa Q w Wghs pa E Q.. : : \ \ \ \ Gvn a sampl Q. Evdnc and Baysan nwork: Gvn a : Eampl Noc: Q s anohr Baysan nwork

31 Lklhood Wghng: Esmas 3 T w T ˆ Esma : zro ohrws and quals f ˆ ˆ ˆ T T g w g w Esma osror Margnals:

32 Lklhood Wghng Convrgs o ac posror margnals Gnras Sampls Fas Samplng dsrbuon s clos o pror spcally f E Laf Nods Incrasng samplng varanc Convrgnc may b slow Many sampls wh =0 rjcd 3

33 Ouln Dfnons and Background on Sascs Thory of mporanc samplng Lklhood wghng Error smaon Sa-of-h-ar mporanc samplng chnqus 33

34

35

36

37 absolu

38 Ouln Dfnons and Background on Sascs Thory of mporanc samplng Lklhood wghng Sa-of-h-ar mporanc samplng chnqus 38

39 roposal slcon On should ry o slc a proposal ha s as clos as possbl o h posror dsrbuon. smaor zro- varanc o hav a 0 ] [ ˆ z z Q z Q z z Q z z Q z Q z N T z w Var Var Z z Q Q

40 rfc samplng usng Buck Algorhm: Elmnaon Run Buck lmnaon on h problm along an ordrng o= N... Sampl along h rvrs ordrng:.. N A ach varabl rcovr h probably... - by rfrrng o h buck.

41 4 Buck Elmnaon 0 0 a a d b c c b b a d a c a b a a 0 0 d c b b a d c b a b a c a 0 Elmnaon Ordr: dbc Qury: D: E: B: C: A: d D b a d b a f b a d c b 0 c b c b f E b E D B c b f b a f a b c a f 0 a f A p a C a a c c B C c a f a c a f a b DAB EBC BAC CA A b a f D c b f E c a f B f C a A D E C B Buck Tr D E B C A Orgnal Funcons Mssags Tm and spac pw*

42 buck B: buck C: buck D: buck E: buck A: Buck lmnaon BE Algorhm lm-bl Dchr 996 CA a b BA DBA BC h B ADC h C AD h D A h E a Elmnaon opraor S 4 B C D E A

43 Samplng from h oupu of BE Dchr 00 buck B: S A a D dc c n h buck Sampl : B b QB a d B a d BA DBA BC B a b c buck C: CA h B ADC S A a D d n h buck Sampl :C c QC a d C A h B a d C buck D: h C AD S A a n h buck Sampl:D d QD a h C ad buck E: h D A Evdnc buck :gnor buck A: A h E A QA A h E A Sampl : A a QA S 43

44 Mn-bucks: local nfrnc Compuaon n a buck s m and spac ponnal n h numbr of varabls nvolvd Thrfor paron funcons n a buck no mn-bucks on smallr numbr of varabls Can conrol h sz of ach mn-buck yldng polynomal comply. S 44

45 Mn-Buck Elmnaon buck B: Σ B BC Mn-bucks Σ B BA DBA Spac and Tm consrans: Mamum scop sz of h nw funcon gnrad should b boundd by buck C: buck D: CA h B C h B AD BE gnras a funcon havng scop sz 3. So canno b usd. buck E: h C A buck A: A h E A h D A Appromaon of S 45 45

46 Samplng from h oupu of MBE buck B: BC BA DBA buck C: CA h B C buck D: h B AD buck E: buck A: h E A h C A h D A Samplng s sam as n BE-samplng cp ha now w consruc Q from a randomly slcd mnbuck S 46 46

47 IJG-Samplng Goga and Dchr 005 Irav Jon Graph ropagaon IJG A Gnralzd Blf ropagaon schm Ydda al. 00 IJG ylds br appromaons of E han MBE Dchr Kask and Mascu 00 Oupu of IJG s sam as mn-buck clusrs Currnly h bs prformng IS schm!

48 Currn Rsarch quson Gvn a Baysan nwork wh vdnc or a Markov nwork rprsnng funcon gnra anohr Baysan nwork rprsnng a funcon Q from a famly of dsrbuons rsrcd by srucur such ha Q s closs o. Currn approachs Mn-bucks Ijgp Boh Eprmnd bu nd o b jusfd horcally.

49 Algorhm: Approma Samplng Run IJG or MBE A ach branch pon compu h dg probabls by consulng oupu of IJG or MBE Rjcon roblm: Som assgnmns gnrad ar non soluons

50 k ˆ R ' UpdaQ N ˆ ˆE Q... z Gnra sampls z o k do For 0 ˆ... Q roposal Inal k N E urn End Q Q k Q z w E from E Z pa Z Q Z pa Z Q Z Q Z k k N j k k n n Adapv Imporanc Samplng

51 Adapv Imporanc Samplng Gnral cas Gvn k proposal dsrbuons Tak N sampls ou of ach dsrbuon Approma ˆ k k j Avg wgh jh proposal

52 Esmang Q'z ' Q Z Q' Z whr ach Q'Z Q' Z Z.. Z pa Z s smad by mporanc samplng -... Q' Z pa n Z n

53 Ovrvw. robablsc Rasonng/Graphcal modls. Imporanc Samplng 3. Markov Chan Mon Carlo: Gbbs Samplng 4. Samplng n prsnc of Drmnsm 5. Rao-Blackwllsaon 6. AND/OR mporanc samplng

54 Markov Chan 3 4 A Markov chan s a dscr random procss wh h propry ha h n sa dpnds only on h currn sa Markov ropry:... If - dos no dpnd on m homognous and sa spac s fn hn s ofn prssd as a ranson funcon aka ranson mar 54

55 Eampl: Drunkard s Walk a random walk on h numbr ln whr a ach sp h poson may chang by + or wh qual probably 3 D {0...} n n 0.5 n 0.5 ranson mar 55

56 Eampl: Wahr Modl ran ran ran sun ran D { rany sunny} rany sunny rany sunny ranson mar 56

57 Mul-Varabl Sysm } D dscr { 3 sa s an assgnmn of valus o all h varabls fn {... n } 57

58 Baysan Nwork Sysm Baysan Nwork s a rprsnaon of h jon probably dsrbuon ovr or mor varabls { 3 } { } 3 58

59 Saonary Dsrbuon Esnc If h Markov chan s m-homognous hn h vcor s a saonary dsrbuon aka nvaran or qulbrum dsrbuon aka fd pon f s nrs sum up o and sasfy: D Fn sa spac Markov chan has a unqu saonary dsrbuon f and only f: Th chan s rrducbl All of s sas ar posv rcurrn j j 59

60 Irrducbl A sa s rrducbl f undr h ranson rul on has nonzro probably of movng from o any ohr sa and hn comng back n a fn numbr of sps If on sa s rrducbl hn all h sas mus b rrducbl Lu Ch. pp. 49 Df... 60

61 Rcurrn A sa s rcurrn f h chan rurns o wh probably L M b h pcd numbr of sps o rurn o sa Sa s posv rcurrn f M s fn Th rcurrn sas n a fn sa chan ar posv rcurrn. 6

62 Saonary Dsrbuon Convrgnc Consdr nfn Markov chan: n n 0 If h chan s boh rrducbl and aprodc hn: lm n n Inal sa s no mporan n h lm Th mos usful faur of a good Markov chan s s fas forgfulnss of s pas Lu Ch.. 0 n 6

63 Aprodc Dfn d = g.c.d.{n > 0 s possbl o go from o n n sps}. Hr g.c.d. mans h gras common dvsor of h ngrs n h s. If d= for hn chan s aprodc osv rcurrn aprodc sas ar rgodc 63

64 Markov Chan Mon Carlo How do w sma.g.? Gnra sampls ha form Markov Chan wh saonary dsrbuon = Esma from sampls obsrvd sas: vsd sas 0 n can b vwd as sampls from dsrbuon T T lm T 64

65 MCMC Summary Convrgnc s guarand n h lm Inal sa s no mporan bu ypcally w hrow away frs K sampls - burn-n Sampls ar dpndn no..d. Convrgnc mng ra may b slow Th srongr corrlaon bwn sas h slowr convrgnc! 65

66 Gbbs Samplng Gman&Gman984 Gbbs samplr s an algorhm o gnra a squnc of sampls from h jon probably dsrbuon of wo or mor random varabls Sampl nw varabl valu on varabl a a m from h varabl s condonal dsrbuon:..... } \ n Sampls form a Markov chan wh saonary dsrbuon 66

67 Gbbs Samplng: Illusraon Th procss of Gbbs samplng can b undrsood as a random walk n h spac of all nsanaons of = rmmbr drunkard s walk: In on sp w can rach nsanaons ha dffr from currn on by valu assgnmn o a mos on varabl assum randomzd choc of varabls.

68 Ordrd Gbbs Samplr Gnra sampl + from : In shor for = o N: 68 \ sampld from N N N N N N rocss All Varabls In Som Ordr

69 Transon robabls n BN Gvn Markov blank parns chldrn and hr parns s ndpndn of all ohr nods \ \ Markov blank: markov pa ch pa markov pa j ch Compuaon s lnar n h sz of Markov blank! : j pa j ch j j j 69

70 Ordrd Gbbs Samplng Algorhm arl988 Inpu: E= Oupu: T sampls { } F vdnc E= nalz 0 a random. For = o T compu sampls. For = o N loop hrough varabls 3. + markov 4. End For 5. End For

71 Gbbs Samplng Eampl - BN {... 9} E { 9} 3 6 = 0 6 = 6 0 = = = = = = 5 0 7

72 Gbbs Samplng Eampl - BN {... 9} E { 9}

73 Answrng Qurs =? Mhod : coun # of sampls whr = hsogram smaor: T markov T T T Drac dla f-n Mhod : avrag probably mur smaor: Mur smaor convrgs fasr consdr smas for h unobsrvd valus of ; prov va Rao-Blackwll horm

74 Rao-Blackwll Thorm Rao-Blackwll Thorm: L random varabl s b composd of wo groups of varabls R and L. Thn for h jon dsrbuon RL and funcon g h followng rsul appls Var[ E{ g R L} Var[ g R] for a funcon of nrs g.g. h man or covaranc Caslla&Robr996 Lu. al horm maks a wak proms bu works wll n pracc! mprovmn dpnds h choc of R and L 74

75 Imporanc vs. Gbbs T T T Q g T g Q g T g ˆ ˆ ˆ w Gbbs: Imporanc:

76 Gbbs Samplng: Convrgnc Sampl from ` Convrgs ff chan s rrducbl and rgodc Inuon - mus b abl o plor all sas: f and j ar srongly corrlad =0 j =0 hn w canno plor sas wh = and j = All condons ar sasfd whn all probabls ar posv Convrgnc ra can b characrzd by h scond gn-valu of ranson mar 76

77 Gbbs: Spdng Convrgnc Rduc dpndnc bwn sampls auocorrlaon Skp sampls Randomz Varabl Samplng Ordr Employ blockng groupng Mulpl chans Rduc varanc covr n h n scon 77

78 Blockng Gbbs Samplr Sampl svral varabls oghr as a block Eampl: Gvn hr varabls YZ wh domans of sz group Y and Z oghr o form a varabl W={YZ} wh doman sz 4. Thn gvn sampl y z compu n sampl: y z w y z w Y Z + Can mprov convrgnc graly whn wo varabls ar srongly corrlad! - Doman of h block varabl grows ponnally wh h #varabls n a block! 78

79 Gbbs: Mulpl Chans Gnra M chans of sz K Each chan producs ndpndn sma m : 79 M m M ˆ K m K \ Tra m as ndpndn random varabls. Esma as avrag of m :

80 Gbbs Samplng Summary Markov Chan Mon Carlo mhod Glfand and Smh 990 Smh and Robrs 993 Trny 994 Sampls ar dpndn form Markov Chan Sampl from whch convrgs o Guarand o convrg whn all > 0 Mhods o mprov convrgnc: Blockng Rao-Blackwllsd 80

81 Ovrvw. robablsc Rasonng/Graphcal modls. Imporanc Samplng 3. Markov Chan Mon Carlo: Gbbs Samplng 4. Samplng n prsnc of Drmnsm 5. Rao-Blackwllsaon 6. AND/OR mporanc samplng

82 Samplng: rformanc Gbbs samplng Rduc dpndnc bwn sampls Imporanc samplng Rduc varanc Achv boh by samplng a subs of varabls and ngrang ou h rs rduc dmnsonaly aka Rao-Blackwllsaon Eplo graph srucur o manag h ra cos 8

83 Smallr Subs Sa-Spac Smallr sa-spac s asr o covr { 3 4} { } D 64 D 6 83

84 Smoohr Dsrbuon

85 Spdng Up Convrgnc Man Squard Error of h smaor: MSE Q BIAS Var Q In cas of unbasd smaor BIAS=0 MSE Q ˆ [ ˆ] VarQ [ ˆ] EQ EQ[ ] Rduc varanc spd up convrgnc! 85

86 Rao-Blackwllsaon 86 } { ~ ]} [ { } { } { ˆ ]} [ { } { ]} {var[ ]} [ { } { ]} [ ] [ { ~ } { ˆ g Var T l h E Var T h Var g Var l g E Var g Var l g E l g E Var g Var l h E l h E T g h h T g L R T T Lu Ch..3

87 Rao-Blackwllsaon Carry ou analycal compuaon as much as possbl - Lu =RL Imporanc Samplng: Var Q R L { } Var Q R L Q R { } Q R Lu Ch..5.5 Gbbs Samplng: auocovarancs ar lowr lss corrlaon bwn sampls f and j ar srongly corrlad =0 j =0 only nclud on fo hm no a samplng s 87

88 Blockng Gbbs Samplr vs. Collapsd Y Z Fasr Convrgnc Sandard Gbbs: y z y z z y Blockng: y z y z Collapsd: y y 3 88

89 Collapsd Gbbs Samplng Gnrang Sampls Gnra sampl c + from c : 89 \ c c c c C c c c c c C c c c c c C c c c c c C K K K K K K sampldfrom In shor for = o K:

90 Collapsd Gbbs Samplr Inpu: C E= Oupu: T sampls {c } F vdnc E= nalz c 0 a random. For = o T compu sampls. For = o N loop hrough varabls 3. c + C c \c 4. End For 5. End For

91 Calculaon Tm Compung c c \c s mor pnsv rqurs nfrnc Tradng #sampls for smallr varanc: gnra mor sampls wh hghr covaranc gnra fwr sampls wh lowr covaranc Mus conrol h m spn compung samplng probabls n ordr o b mffcv! 9

92 Eplong Graph roprs Rcall compuaon m s ponnal n h adjusd nducd wdh of a graph w-cus s a subs of varabl s.. whn hy ar obsrvd nducd wdh of h graph s w whn sampld varabls form a w-cus nfrnc s pw.g. usng Buck Tr Elmnaon cycl-cus s a spcal cas of w-cus Samplng w-cus w-cus samplng! 9

93 Wha If C=Cycl-Cus? c { 5 } E { 9} 5 9 can compu usng Buck Elmnaon compuaon comply s ON 93

94 Compung Transon robabls 94 : 0 : BE BE Compu jon probabls: Normalz:

95 Cus Samplng-Answrng Qurs Qury: c C c =? sam as Gbbs: ˆ c T T c c \ c compud whl gnrang sampl usng buck r lmnaon Qury: \C =? T c T compu afr gnrang sampl usng buck r lmnaon 95

96 Cus Samplng vs. Cus Condonng 96 c c T c coun c c T C D c C D c T Cus Condonng Cus Samplng c c C D c

97 Cus Samplng Eampl Esmang for samplng nod : Sampl Sampl Sampl

98 Cus Samplng Eampl 98 } { } { } { c c c Esmang 3 for non-sampld nod 3 :

99 CCS54 Ts Rsuls CCS54 n=54 C=5 E=3 CCS54 n=54 C=5 E= Cus Gbbs Cus Gbbs # sampls Tmsc MSE vs. #sampls lf and m rgh Ergodc =54 D = C=5 E=3 Eac Tm = 30 sc usng Cus Condonng 99

100 CCS79 Ts Rsuls CCS79 n=79 C=8 E=35 CCS79 n=79 C=8 E= Cus Gbbs Cus Gbbs # sampls Tmsc MSE vs. #sampls lf and m rgh Non-Ergodc drmnsc CT nry = 79 C = 8 <= D <=4 E = 35 Eac Tm = sc usng Cus Condonng 00

101 CCS360b Ts Rsuls CCS360b n=360 C= E=36 CCS360b n=360 C= E=36 Cus Gbbs Cus Gbbs # sampls Tmsc MSE vs. #sampls lf and m rgh Ergodc = 360 D = C = E = 36 Eac Tm > 60 mn usng Cus Condonng Eac Valus oband va Buck Elmnaon 0

102 Random Nworks RANDOM n=00 C=3 E=5-0 Cus Gbbs # sampls RANDOM n=00 C=3 E=5-0 Cus Gbbs Tmsc MSE vs. #sampls lf and m rgh = 00 D =C = 3 E = 5-0 Eac Tm = 30 sc usng Cus Condonng 0

103 Codng Nworks Cus Transforms Non-Ergodc Chan o Ergodc 3 4 Codng Nworks n=00 C=-4 u u u 3 u 4 0. IB Gbbs Cus p p p 3 p y y y 3 y Tmsc MSE vs. m rgh Non-Ergodc = 00 D = C = 3-6 E = 50 Sampl Ergodc Subspac U={U U U k } Eac Tm = 50 sc usng Cus Condonng 03

104 Non-Ergodc Halfndr HalFndr n=56 C=5 E= HalFndr n=56 C=5 E= 0. Cus Gbbs Cus Gbbs # sampls Tmsc MSE vs. #sampls lf and m rgh Non-Ergodc = 56 C = 5 <=D <= E = 0 Eac Tm = sc usng Loop-Cus Condonng 04

105 CCS360b - MSE cpcs360b N=360 E=[0-34] w*=0 MSE Gbbs IB C=6fw=3 C=48fw= Tm sc MSE vs. Tm Ergodc = 360 C = 6 D = Eac Tm = 50 mn usng BTE 05

106 Cus Imporanc Samplng Apply Imporanc Samplng ovr cus C T T w T Q c c T ˆ T w c c T c T w c T whr c s compud usng Buck Elmnaon Goga & Dchr 005 and Bdyuk & Dchr 006

107 Lklhood Cus Wghng LCS Z=Topologcal Ordr{CE} Gnrang sampl +: For If Z Z z Els z End If End For Z E do : z z Z z... z KL[C QC+ KL* Q] compud whl gnrang sampl usng buck r lmnaon can b mmozd for som numbr of nsancs K basd on mmory avalabl 07

108 ahfndr 08

109 ahfndr 09

110 Lnk 0

111 Summary Imporanc Samplng..d. sampls Unbasd smaor Gnras sampls fas Sampls from Q Rjc sampls wh zro-wgh Improvs on cus Gbbs Samplng Dpndn sampls Basd smaor Gnras sampls slowr Sampls from ` Dos no convrg n prsnc of consrans Improvs on cus

112 MSE CCS360b.E-0.E-03 cpcs360b N=360 LC=6 w*= E=5 LW AIS-BN Gbbs LCS IB.E-04.E Tm sc LW lklhood wghng LCS lklhood wghng on a cus

113 MSE CCS4b.0E-0.0E-03 cpcs4b N=4 LC=47 w*= E=8 LW AIS-BN Gbbs LCS IB.0E-04.0E Tm sc LW lklhood wghng LCS lklhood wghng on a cus 3

114 MSE Codng Nworks.0E-0.0E-0 codng N=00 =3 LC=6 w*= LW AIS-BN Gbbs LCS IB.0E-03.0E-04.0E Tm sc LW lklhood wghng LCS lklhood wghng on a cus 4