Ovrvw. robablsc Rasonng/Graphcal modls. Imporanc Samplng 3. Markov Chan Mon Carlo: Gbbs Samplng 4. Samplng n prsnc of Drmnsm 5. Rao-Blackwllsaon 6. AND/OR mporanc samplng
Ovrvw. robablsc Rasonng/Graphcal modls. Imporanc Samplng 3. Markov Chan Mon Carlo: Gbbs Samplng 4. Samplng n prsnc of Drmnsm 5. Cus-basd Varanc Rducon 6. AND/OR mporanc samplng
robablsc Rasonng; Graphcal modls Graphcal modls: Baysan nwork consran nworks md nwork Qurs Eac algorhm usng nfrnc sarch and hybrds Graph paramrs: r-wdh cycl-cus w-cus
Qurs robably of vdnc or paron funcon var pa osror margnal blfs: Mos robabl Eplanaon n var j n n var j Z j j pa pa j j C * argma
Appromaon Snc nfrnc sarch and hybrds ar oo pnsv whn graph s dns; hgh rwdh hn: Boundng nfrnc: mn-buck and mn-clusrng Blf propagaon Boundng sarch: Samplng Goal: an anym schm 5
Ovrvw. robablsc Rasonng/Graphcal modls. Imporanc Samplng 3. Markov Chan Mon Carlo: Gbbs Samplng 4. Samplng n prsnc of Drmnsm 5. Rao-Blackwllsaon 6. AND/OR mporanc samplng
Ouln Dfnons and Background on Sascs Thory of mporanc samplng Lklhood wghng Sa-of-h-ar mporanc samplng chnqus 7
A sampl Gvn a s of varabls ={... n } a sampl dnod by S s an nsanaon of all varabls: S... n 8
How o draw a sampl? Unvara dsrbuon Eampl: Gvn random varabl havng doman {0 } and a dsrbuon = 0.3 0.7. Task: Gnra sampls of from. How? draw random numbr r [0 ] If r < 0.3 hn s =0 Els s = 9
How o draw a sampl? Mul-vara dsrbuon L ={.. n } b a s of varabls Eprss h dsrbuon n produc form... n... n Sampl varabls on by on from lf o rgh along h ordrng dcad by h produc form. Baysan nwork lraur: Logc samplng 0
Samplng for rob. Infrnc Ouln Logc Samplng Imporanc Samplng Lklhood Samplng Choosng a roposal Dsrbuon Markov Chan Mon Carlo MCMC Mropols-Hasngs Gbbs samplng Varanc Rducon
Logc Samplng: No Evdnc Hnron 988 Inpu: Baysan nwork = { N } N- #nods T - # sampls Oupu: T sampls rocss nods n opologcal ordr frs procss h ancsors of a nod hn h nod slf:. For = 0 o T. For = 0 o N 3. sampl from pa
Logc samplng ampl 3 4 3 4 3 No Evdnc 3 3 4 4 3 // gnra sampl k.sampl.sampl 3.Sampl 4.Sampl 3 4 from from from 3 from 4 3 3 3
Logc Samplng w/ Evdnc Inpu: Baysan nwork = { N } N- #nods E vdnc T - # sampls Oupu: T sampls conssn wh E. For = o T. For = o N 3. sampl from pa 4. If n E and rjc sampl: 5. Goo Sp. 4
Logc Samplng ampl Evdnc : 3 0 3 4 3 4 3 // gnra sampl k.sampl.sampl 3.Sampl 4. If 3 and sar from ohrws 5.Sampl 3 from from from 0 rjc sampl 4 3 from 4 3 5
Epcd valu and Varanc Epcd valu: Gvn a probably dsrbuon and a funcon g dfnd ovr a s of varabls = { n } h pcd valu of g w.r.. s E [ g ] g Varanc: Th varanc of g w.r.. s: Var [ g ] g E [ g ] 6
Mon Carlo Esma Esmaor: An smaor s a funcon of h sampls. I producs an sma of h unknown paramr of h samplng dsrbuon. Gvn..d. sampls S gˆ T T S h M oncarlo sma of g S S E T drawn from [g] s gvn by : 7
Eampl: Mon Carlo sma Gvn: A dsrbuon = 0.3 0.7. g = 40 f quals 0 = 50 f quals. Esma E [g]=400.3+500.7=47. Gnra k sampls from : 0000 gˆ 40# sampls 0 50# # sampls sampls 40 4 506 0 46 8
Ouln Dfnons and Background on Sascs Thory of mporanc samplng Lklhood wghng Sa-of-h-ar mporanc samplng chnqus 9
Imporanc samplng: Man da Eprss qury as h pcd valu of a random varabl w.r.. o a dsrbuon Q. Gnra random sampls from Q. Esma h pcd valu from h gnrad sampls usng a mon carlo smaor avrag. 0
Imporanc samplng for ˆ : ] [ \ Z Q z w T z w E z Q z E z Q z Q z z z Q z E Z L T Q Q z z whr z M oncarlo sma as : w can rwr Thn 0 0 sasfyng proposaldsrbuon L QZb a
roprs of IS sma of Convrgnc: by law of larg numbrs Unbasd. Varanc: ˆ T a. s w z. for T T Var E Q Q [ ˆ ] ˆ Var Q T N w z Var Q [ w z] T
roprs of IS sma of Man Squard Error of h smaor MSE Q ˆ E ˆ Q E ˆ [ ] Var ˆ Q Q ˆ Var Var Q Q [ w ] T Ths quany nclosd n h bracks s zro bcaus h pcd valu of h smaor quals h pcd valu of g
Esmang 4 E basd : Esma s ˆ ˆ Rao sma: IS. Ida : Esma numraor and dnomnaor by and 0 ohrws. z conans whch s f drac - dla funcon z b a L T k T k wz wz z z Q z E z Q z z E z z z k k k Q Q z z
roprs of h IS smaor for Convrgnc: By Wak law of larg numbrs as T Asympocally unbasd Varanc lm T E Hardr o analyz [ ] Lu suggss a masur calld Effcv sampl sz 5
Gnrang sampls from Q No rsrcons on how o Typcally prss Q n produc form: QZ=QZ QZ Z.QZ n Z..Z n- Sampl along h ordr Z..Z n Eampl: Z QZ =0.0.8 Z QZ Z =0.0.90.0.8 Z 3 QZ 3 Z Z =QZ 3 =0.50.5
Ouln Dfnons and Background on Sascs Thory of mporanc samplng Lklhood wghng Sa-of-h-ar mporanc samplng chnqus 7
Lklhood Wghng Fung and Chang 990; Shachr and o 990 Is an nsanc of mporanc samplng! Clampng vdnc+ logc samplng+ wghng sampls by vdnc lklhood Works wll for lkly vdnc! 8
Lklhood Wghng: Samplng Sampl n opologcal ordr ovr! Clamp vdnc Sampl pa pa s a look-up n CT! 9
Lklhood Wghng: roposal Dsrbuon 30 E E j j E E E E j j n E j j pa pa pa pa Q w Wghs pa E Q.. : : \ \ \ \ Gvn a sampl Q. Evdnc and Baysan nwork: Gvn a : Eampl 3 3 3 3 Noc: Q s anohr Baysan nwork
Lklhood Wghng: Esmas 3 T w T ˆ Esma : zro ohrws and quals f ˆ ˆ ˆ T T g w g w Esma osror Margnals:
Lklhood Wghng Convrgs o ac posror margnals Gnras Sampls Fas Samplng dsrbuon s clos o pror spcally f E Laf Nods Incrasng samplng varanc Convrgnc may b slow Many sampls wh =0 rjcd 3
Ouln Dfnons and Background on Sascs Thory of mporanc samplng Lklhood wghng Error smaon Sa-of-h-ar mporanc samplng chnqus 33
absolu
Ouln Dfnons and Background on Sascs Thory of mporanc samplng Lklhood wghng Sa-of-h-ar mporanc samplng chnqus 38
roposal slcon On should ry o slc a proposal ha s as clos as possbl o h posror dsrbuon. smaor zro- varanc o hav a 0 ] [ ˆ z z Q z Q z z Q z z Q z Q z N T z w Var Var Z z Q Q
rfc samplng usng Buck Algorhm: Elmnaon Run Buck lmnaon on h problm along an ordrng o= N... Sampl along h rvrs ordrng:.. N A ach varabl rcovr h probably... - by rfrrng o h buck.
4 Buck Elmnaon 0 0 a a d b c c b b a d a c a b a a 0 0 d c b b a d c b a b a c a 0 Elmnaon Ordr: dbc Qury: D: E: B: C: A: d D b a d b a f b a d c b 0 c b c b f E b E D B c b f b a f a b c a f 0 a f A p a C a a c c B C c a f a c a f a b DAB EBC BAC CA A b a f D c b f E c a f B f C a A D E C B Buck Tr D E B C A Orgnal Funcons Mssags Tm and spac pw*
buck B: buck C: buck D: buck E: buck A: Buck lmnaon BE Algorhm lm-bl Dchr 996 CA a b BA DBA BC h B ADC h C AD h D A h E a Elmnaon opraor S 4 B C D E A
Samplng from h oupu of BE Dchr 00 buck B: S A a D dc c n h buck Sampl : B b QB a d B a d BA DBA BC B a b c buck C: CA h B ADC S A a D d n h buck Sampl :C c QC a d C A h B a d C buck D: h C AD S A a n h buck Sampl:D d QD a h C ad buck E: h D A Evdnc buck :gnor buck A: A h E A QA A h E A Sampl : A a QA S 43
Mn-bucks: local nfrnc Compuaon n a buck s m and spac ponnal n h numbr of varabls nvolvd Thrfor paron funcons n a buck no mn-bucks on smallr numbr of varabls Can conrol h sz of ach mn-buck yldng polynomal comply. S 44
Mn-Buck Elmnaon buck B: Σ B BC Mn-bucks Σ B BA DBA Spac and Tm consrans: Mamum scop sz of h nw funcon gnrad should b boundd by buck C: buck D: CA h B C h B AD BE gnras a funcon havng scop sz 3. So canno b usd. buck E: h C A buck A: A h E A h D A Appromaon of S 45 45
Samplng from h oupu of MBE buck B: BC BA DBA buck C: CA h B C buck D: h B AD buck E: buck A: h E A h C A h D A Samplng s sam as n BE-samplng cp ha now w consruc Q from a randomly slcd mnbuck S 46 46
IJG-Samplng Goga and Dchr 005 Irav Jon Graph ropagaon IJG A Gnralzd Blf ropagaon schm Ydda al. 00 IJG ylds br appromaons of E han MBE Dchr Kask and Mascu 00 Oupu of IJG s sam as mn-buck clusrs Currnly h bs prformng IS schm!
Currn Rsarch quson Gvn a Baysan nwork wh vdnc or a Markov nwork rprsnng funcon gnra anohr Baysan nwork rprsnng a funcon Q from a famly of dsrbuons rsrcd by srucur such ha Q s closs o. Currn approachs Mn-bucks Ijgp Boh Eprmnd bu nd o b jusfd horcally.
Algorhm: Approma Samplng Run IJG or MBE A ach branch pon compu h dg probabls by consulng oupu of IJG or MBE Rjcon roblm: Som assgnmns gnrad ar non soluons
k ˆ R ' UpdaQ N ˆ ˆE Q... z Gnra sampls z o k do For 0 ˆ... Q roposal Inal k N E urn End Q Q k Q z w E from E Z pa Z Q Z pa Z Q Z Q Z k k N j k k n n Adapv Imporanc Samplng
Adapv Imporanc Samplng Gnral cas Gvn k proposal dsrbuons Tak N sampls ou of ach dsrbuon Approma ˆ k k j Avg wgh jh proposal
Esmang Q'z ' Q Z Q' Z whr ach Q'Z Q' Z Z.. Z pa Z s smad by mporanc samplng -... Q' Z pa n Z n
Ovrvw. robablsc Rasonng/Graphcal modls. Imporanc Samplng 3. Markov Chan Mon Carlo: Gbbs Samplng 4. Samplng n prsnc of Drmnsm 5. Rao-Blackwllsaon 6. AND/OR mporanc samplng
Markov Chan 3 4 A Markov chan s a dscr random procss wh h propry ha h n sa dpnds only on h currn sa Markov ropry:... If - dos no dpnd on m homognous and sa spac s fn hn s ofn prssd as a ranson funcon aka ranson mar 54
Eampl: Drunkard s Walk a random walk on h numbr ln whr a ach sp h poson may chang by + or wh qual probably 3 D {0...} n n 0.5 n 0.5 ranson mar 55
Eampl: Wahr Modl ran ran ran sun ran D { rany sunny} rany sunny rany 0.9 0.5 sunny 0. 0.5 ranson mar 56
Mul-Varabl Sysm } D dscr { 3 sa s an assgnmn of valus o all h varabls fn + + 3 3 + {... n } 57
Baysan Nwork Sysm Baysan Nwork s a rprsnaon of h jon probably dsrbuon ovr or mor varabls + + 3 3 3 + { 3 } { } 3 58
Saonary Dsrbuon Esnc If h Markov chan s m-homognous hn h vcor s a saonary dsrbuon aka nvaran or qulbrum dsrbuon aka fd pon f s nrs sum up o and sasfy: D Fn sa spac Markov chan has a unqu saonary dsrbuon f and only f: Th chan s rrducbl All of s sas ar posv rcurrn j j 59
Irrducbl A sa s rrducbl f undr h ranson rul on has nonzro probably of movng from o any ohr sa and hn comng back n a fn numbr of sps If on sa s rrducbl hn all h sas mus b rrducbl Lu Ch. pp. 49 Df... 60
Rcurrn A sa s rcurrn f h chan rurns o wh probably L M b h pcd numbr of sps o rurn o sa Sa s posv rcurrn f M s fn Th rcurrn sas n a fn sa chan ar posv rcurrn. 6
Saonary Dsrbuon Convrgnc Consdr nfn Markov chan: n n 0 If h chan s boh rrducbl and aprodc hn: lm n n Inal sa s no mporan n h lm Th mos usful faur of a good Markov chan s s fas forgfulnss of s pas Lu Ch.. 0 n 6
Aprodc Dfn d = g.c.d.{n > 0 s possbl o go from o n n sps}. Hr g.c.d. mans h gras common dvsor of h ngrs n h s. If d= for hn chan s aprodc osv rcurrn aprodc sas ar rgodc 63
Markov Chan Mon Carlo How do w sma.g.? Gnra sampls ha form Markov Chan wh saonary dsrbuon = Esma from sampls obsrvd sas: vsd sas 0 n can b vwd as sampls from dsrbuon T T lm T 64
MCMC Summary Convrgnc s guarand n h lm Inal sa s no mporan bu ypcally w hrow away frs K sampls - burn-n Sampls ar dpndn no..d. Convrgnc mng ra may b slow Th srongr corrlaon bwn sas h slowr convrgnc! 65
Gbbs Samplng Gman&Gman984 Gbbs samplr s an algorhm o gnra a squnc of sampls from h jon probably dsrbuon of wo or mor random varabls Sampl nw varabl valu on varabl a a m from h varabl s condonal dsrbuon:..... } \ n Sampls form a Markov chan wh saonary dsrbuon 66
Gbbs Samplng: Illusraon Th procss of Gbbs samplng can b undrsood as a random walk n h spac of all nsanaons of = rmmbr drunkard s walk: In on sp w can rach nsanaons ha dffr from currn on by valu assgnmn o a mos on varabl assum randomzd choc of varabls.
Ordrd Gbbs Samplr Gnra sampl + from : In shor for = o N: 68 \ sampld from............ 3 3 N N N N N N rocss All Varabls In Som Ordr
Transon robabls n BN Gvn Markov blank parns chldrn and hr parns s ndpndn of all ohr nods \ \ Markov blank: markov pa ch pa markov pa j ch Compuaon s lnar n h sz of Markov blank! : j pa j ch j j j 69
Ordrd Gbbs Samplng Algorhm arl988 Inpu: E= Oupu: T sampls { } F vdnc E= nalz 0 a random. For = o T compu sampls. For = o N loop hrough varabls 3. + markov 4. End For 5. End For
Gbbs Samplng Eampl - BN {... 9} E { 9} 3 6 = 0 6 = 6 0 = 0 5 8 7 = 7 0 3 = 3 0 4 7 9 8 = 8 0 4 = 4 0 5 = 5 0 7
Gbbs Samplng Eampl - BN {... 9} E { 9} 3 6 0 0... 8 9 5 8 0... 8 9 4 7 9 7
Answrng Qurs =? Mhod : coun # of sampls whr = hsogram smaor: T markov T T T Drac dla f-n Mhod : avrag probably mur smaor: Mur smaor convrgs fasr consdr smas for h unobsrvd valus of ; prov va Rao-Blackwll horm
Rao-Blackwll Thorm Rao-Blackwll Thorm: L random varabl s b composd of wo groups of varabls R and L. Thn for h jon dsrbuon RL and funcon g h followng rsul appls Var[ E{ g R L} Var[ g R] for a funcon of nrs g.g. h man or covaranc Caslla&Robr996 Lu. al. 995. horm maks a wak proms bu works wll n pracc! mprovmn dpnds h choc of R and L 74
Imporanc vs. Gbbs T T T Q g T g Q g T g ˆ ˆ ˆ w Gbbs: Imporanc:
Gbbs Samplng: Convrgnc Sampl from ` Convrgs ff chan s rrducbl and rgodc Inuon - mus b abl o plor all sas: f and j ar srongly corrlad =0 j =0 hn w canno plor sas wh = and j = All condons ar sasfd whn all probabls ar posv Convrgnc ra can b characrzd by h scond gn-valu of ranson mar 76
Gbbs: Spdng Convrgnc Rduc dpndnc bwn sampls auocorrlaon Skp sampls Randomz Varabl Samplng Ordr Employ blockng groupng Mulpl chans Rduc varanc covr n h n scon 77
Blockng Gbbs Samplr Sampl svral varabls oghr as a block Eampl: Gvn hr varabls YZ wh domans of sz group Y and Z oghr o form a varabl W={YZ} wh doman sz 4. Thn gvn sampl y z compu n sampl: y z w y z w Y Z + Can mprov convrgnc graly whn wo varabls ar srongly corrlad! - Doman of h block varabl grows ponnally wh h #varabls n a block! 78
Gbbs: Mulpl Chans Gnra M chans of sz K Each chan producs ndpndn sma m : 79 M m M ˆ K m K \ Tra m as ndpndn random varabls. Esma as avrag of m :
Gbbs Samplng Summary Markov Chan Mon Carlo mhod Glfand and Smh 990 Smh and Robrs 993 Trny 994 Sampls ar dpndn form Markov Chan Sampl from whch convrgs o Guarand o convrg whn all > 0 Mhods o mprov convrgnc: Blockng Rao-Blackwllsd 80
Ovrvw. robablsc Rasonng/Graphcal modls. Imporanc Samplng 3. Markov Chan Mon Carlo: Gbbs Samplng 4. Samplng n prsnc of Drmnsm 5. Rao-Blackwllsaon 6. AND/OR mporanc samplng
Samplng: rformanc Gbbs samplng Rduc dpndnc bwn sampls Imporanc samplng Rduc varanc Achv boh by samplng a subs of varabls and ngrang ou h rs rduc dmnsonaly aka Rao-Blackwllsaon Eplo graph srucur o manag h ra cos 8
Smallr Subs Sa-Spac Smallr sa-spac s asr o covr { 3 4} { } D 64 D 6 83
Smoohr Dsrbuon 3 4 0-0. 0.-0. 0.-0.6 0-0. 0.-0. 0.-0.6 0. 0. 0. 0 00 0 0 0 0 00 0. 0 0 0 84
Spdng Up Convrgnc Man Squard Error of h smaor: MSE Q BIAS Var Q In cas of unbasd smaor BIAS=0 MSE Q ˆ [ ˆ] VarQ [ ˆ] EQ EQ[ ] Rduc varanc spd up convrgnc! 85
Rao-Blackwllsaon 86 } { ~ ]} [ { } { } { ˆ ]} [ { } { ]} {var[ ]} [ { } { ]} [ ] [ { ~ } { ˆ g Var T l h E Var T h Var g Var l g E Var g Var l g E l g E Var g Var l h E l h E T g h h T g L R T T Lu Ch..3
Rao-Blackwllsaon Carry ou analycal compuaon as much as possbl - Lu =RL Imporanc Samplng: Var Q R L { } Var Q R L Q R { } Q R Lu Ch..5.5 Gbbs Samplng: auocovarancs ar lowr lss corrlaon bwn sampls f and j ar srongly corrlad =0 j =0 only nclud on fo hm no a samplng s 87
Blockng Gbbs Samplr vs. Collapsd Y Z Fasr Convrgnc Sandard Gbbs: y z y z z y Blockng: y z y z Collapsd: y y 3 88
Collapsd Gbbs Samplng Gnrang Sampls Gnra sampl c + from c : 89 \............ 3 3 c c c c C c c c c c C c c c c c C c c c c c C K K K K K K sampldfrom In shor for = o K:
Collapsd Gbbs Samplr Inpu: C E= Oupu: T sampls {c } F vdnc E= nalz c 0 a random. For = o T compu sampls. For = o N loop hrough varabls 3. c + C c \c 4. End For 5. End For
Calculaon Tm Compung c c \c s mor pnsv rqurs nfrnc Tradng #sampls for smallr varanc: gnra mor sampls wh hghr covaranc gnra fwr sampls wh lowr covaranc Mus conrol h m spn compung samplng probabls n ordr o b mffcv! 9
Eplong Graph roprs Rcall compuaon m s ponnal n h adjusd nducd wdh of a graph w-cus s a subs of varabl s.. whn hy ar obsrvd nducd wdh of h graph s w whn sampld varabls form a w-cus nfrnc s pw.g. usng Buck Tr Elmnaon cycl-cus s a spcal cas of w-cus Samplng w-cus w-cus samplng! 9
Wha If C=Cycl-Cus? c 0 0 0 { 5 } E { 9} 5 9 can compu usng Buck Elmnaon 3 3 4 5 6 4 6 7 8 9 7 8 9 5 9 compuaon comply s ON 93
Compung Transon robabls 94 : 0 : 9 3 9 3 BE BE 7 5 4 9 8 3 6 0 0 0 9 3 3 9 3 3 9 3 9 3 Compu jon probabls: Normalz:
Cus Samplng-Answrng Qurs Qury: c C c =? sam as Gbbs: ˆ c T T c c \ c compud whl gnrang sampl usng buck r lmnaon Qury: \C =? T c T compu afr gnrang sampl usng buck r lmnaon 95
Cus Samplng vs. Cus Condonng 96 c c T c coun c c T C D c C D c T Cus Condonng Cus Samplng c c C D c
Cus Samplng Eampl Esmang for samplng nod : 4 5 3 6 3 0 5 5 5 9 9 9 Sampl Sampl Sampl 3 7 8 9 9 3 0 5 9 5 5 9 9 97
Cus Samplng Eampl 98 } { } { } { 9 3 5 3 3 3 5 3 3 9 5 3 5 9 5 3 5 c c c Esmang 3 for non-sampld nod 3 : 7 6 5 4 9 8 3 3 9 3 5 3 3 9 5 3 9 5 3 9 3
CCS54 Ts Rsuls CCS54 n=54 C=5 E=3 CCS54 n=54 C=5 E=3 0.004 Cus Gbbs 0.0008 Cus Gbbs 0.003 0.0006 0.00 0.0004 0.00 0.000 0 0 000 000 3000 4000 5000 # sampls 0 0 5 0 5 0 5 Tmsc MSE vs. #sampls lf and m rgh Ergodc =54 D = C=5 E=3 Eac Tm = 30 sc usng Cus Condonng 99
CCS79 Ts Rsuls CCS79 n=79 C=8 E=35 CCS79 n=79 C=8 E=35 0.0 Cus Gbbs Cus Gbbs 0.0 0.0 0.008 0.006 0.004 0.0 0.008 0.006 0.004 0.00 0.00 0 00 500 000 000 3000 4000 # sampls 0 0 0 40 60 80 Tmsc MSE vs. #sampls lf and m rgh Non-Ergodc drmnsc CT nry = 79 C = 8 <= D <=4 E = 35 Eac Tm = sc usng Cus Condonng 00
CCS360b Ts Rsuls CCS360b n=360 C= E=36 CCS360b n=360 C= E=36 Cus Gbbs Cus Gbbs 0.0006 0.0006 0.000 0.000 0.00008 0.00008 0.00004 0.00004 0 0 00 400 600 800 000 # sampls 0 3 5 0 0 30 40 50 60 Tmsc MSE vs. #sampls lf and m rgh Ergodc = 360 D = C = E = 36 Eac Tm > 60 mn usng Cus Condonng Eac Valus oband va Buck Elmnaon 0
Random Nworks RANDOM n=00 C=3 E=5-0 Cus Gbbs 0.0035 0.003 0.005 0.00 0.005 0.00 0.0005 0 0 00 400 600 800 000 00 # sampls 0.00 0.0008 0.0006 0.0004 0.000 0 RANDOM n=00 C=3 E=5-0 Cus Gbbs 0 3 4 5 6 7 8 9 0 Tmsc MSE vs. #sampls lf and m rgh = 00 D =C = 3 E = 5-0 Eac Tm = 30 sc usng Cus Condonng 0
Codng Nworks Cus Transforms Non-Ergodc Chan o Ergodc 3 4 Codng Nworks n=00 C=-4 u u u 3 u 4 0. IB Gbbs Cus p p p 3 p 4 0.0 y y y 3 y 4 0.00 0 0 0 30 40 50 60 Tmsc MSE vs. m rgh Non-Ergodc = 00 D = C = 3-6 E = 50 Sampl Ergodc Subspac U={U U U k } Eac Tm = 50 sc usng Cus Condonng 03
Non-Ergodc Halfndr HalFndr n=56 C=5 E= HalFndr n=56 C=5 E= 0. Cus Gbbs Cus Gbbs 0.0 0. 0.0 0.00 0.00 0.000 0 500 000 500 # sampls 0.000 3 4 5 6 7 8 9 0 Tmsc MSE vs. #sampls lf and m rgh Non-Ergodc = 56 C = 5 <=D <= E = 0 Eac Tm = sc usng Loop-Cus Condonng 04
CCS360b - MSE cpcs360b N=360 E=[0-34] w*=0 MSE 0.00005 0.0000 Gbbs IB C=6fw=3 C=48fw= 0.00005 0.0000 0.000005 0 0 00 400 600 800 000 00 400 600 Tm sc MSE vs. Tm Ergodc = 360 C = 6 D = Eac Tm = 50 mn usng BTE 05
Cus Imporanc Samplng Apply Imporanc Samplng ovr cus C T T w T Q c c T ˆ T w c c T c T w c T whr c s compud usng Buck Elmnaon Goga & Dchr 005 and Bdyuk & Dchr 006
Lklhood Cus Wghng LCS Z=Topologcal Ordr{CE} Gnrang sampl +: For If Z Z z Els z End If End For Z E do : z z Z z... z KL[C QC+ KL* Q] compud whl gnrang sampl usng buck r lmnaon can b mmozd for som numbr of nsancs K basd on mmory avalabl 07
ahfndr 08
ahfndr 09
Lnk 0
Summary Imporanc Samplng..d. sampls Unbasd smaor Gnras sampls fas Sampls from Q Rjc sampls wh zro-wgh Improvs on cus Gbbs Samplng Dpndn sampls Basd smaor Gnras sampls slowr Sampls from ` Dos no convrg n prsnc of consrans Improvs on cus
MSE CCS360b.E-0.E-03 cpcs360b N=360 LC=6 w*= E=5 LW AIS-BN Gbbs LCS IB.E-04.E-05 0 4 6 8 0 4 Tm sc LW lklhood wghng LCS lklhood wghng on a cus
MSE CCS4b.0E-0.0E-03 cpcs4b N=4 LC=47 w*= E=8 LW AIS-BN Gbbs LCS IB.0E-04.0E-05 0 0 0 30 40 50 60 Tm sc LW lklhood wghng LCS lklhood wghng on a cus 3
MSE Codng Nworks.0E-0.0E-0 codng N=00 =3 LC=6 w*= LW AIS-BN Gbbs LCS IB.0E-03.0E-04.0E-05 0 4 6 8 0 Tm sc LW lklhood wghng LCS lklhood wghng on a cus 4