Sampling Techniques for Probabilistic and Deterministic Graphical models

Transcription

1 Samplng ehnques for robabls and Deermns Graphal models ICS 76 Fall 04 Bozhena Bdyuk Rna Deher Readng Darwhe haper 5 relaed papers

2 Overvew. robabls Reasonng/Graphal models. Imporane Samplng 3. Markov Chan Mone Carlo: Gbbs Samplng 4. Samplng n presene of Deermnsm 5. Rao-Blakwellsaon 6. AND/OR mporane samplng

3 Markov Chan 3 4 A Markov han s a dsree random proess wh he propery ha he ne sae depends only on he urren sae Markov ropery:... If X - does no depend on me homogeneous and sae spae s fne hen s ofen epressed as a ranson funon aka ranson mar X 3

4 Eample: Drunkard s Walk a random walk on he number lne where a eah sep he poson may hange by + or wh equal probably 3 D X {0...} n n n ranson mar X 4

5 Eample: Weaher Model ran ran ran sun ran D X { rany sunny} rany sunny rany sunny ranson mar X 5

6 Mul-Varable Sysem X X X X } D X dsree { 3 sae s an assgnmen of values o all he varables fne {... n } 6

7 Bayesan Nework Sysem Bayesan Nework s a represenaon of he jon probably dsrbuon over or more varables X X + X + X X 3 X X { X X X 3 } { } 3 7

8 Saonary Dsrbuon Esene If he Markov han s me-homogeneous hen he veor πx s a saonary dsrbuon aka nvaran or equlbrum dsrbuon aka fed pon f s enres sum up o and sasfy: π π D X Fne sae spae Markov han has a unque saonary dsrbuon f and only f: he han s rreduble All of s saes are posve reurren j j 8

9 Irreduble A sae s rreduble f under he ranson rule one has nonzero probably of movng from o any oher sae and hen omng bak n a fne number of seps If one sae s rreduble hen all he saes mus be rreduble Lu Ch. pp. 49 Def... 9

10 Reurren A sae s reurren f he han reurns o wh probably Le M be he epeed number of seps o reurn o sae Sae s posve reurren f M s fne he reurren saes n a fne sae han are posve reurren. 0

11 Saonary Dsrbuon Convergene Consder nfne Markov han: n n 0 If he han s boh rreduble and aperod hen: π lm n n Inal sae s no mporan n he lm he mos useful feaure of a good Markov han s s fas forgefulness of s pas Lu Ch.. 0 n

12 Aperod Defne d g..d.{n > 0 s possble o go from o n n seps}. Here g..d. means he greaes ommon dvsor of he negers n he se. If d for hen han s aperod osve reurren aperod saes are ergod

13 Markov Chan Mone Carlo How do we esmae X e.g. Xe? Generae samples ha form Markov Chan wh saonary dsrbuon πxe Esmae π from samples observed saes: vsed saes 0 n an be vewed as samples from dsrbuon π π π δ lm π 3

14 MCMC Summary Convergene s guaraneed n he lm Inal sae s no mporan bu ypally we hrow away frs K samples - burn-n Samples are dependen no..d. Convergene mng rae may be slow he sronger orrelaon beween saes he slower onvergene! 4

15 Gbbs Samplng Geman&Geman984 Gbbs sampler s an algorhm o generae a sequene of samples from he jon probably dsrbuon of wo or more random varables Sample new varable value one varable a a me from he varable s ondonal dsrbuon: X X..... } X \ + n Samples form a Markov han wh saonary dsrbuon Xe 5

16 Gbbs Samplng: Illusraon he proess of Gbbs samplng an be undersood as a random walk n he spae of all nsanaons of X remember drunkard s walk: In one sep we an reah nsanaons ha dffer from urren one by value assgnmen o a mos one varable assume randomzed hoe of varables X.

17 Ordered Gbbs Sampler Generae sample + from : In shor for o N: 7 \ sampled from e X X e X X e X X e X X N N N N N N roess All Varables In Some Order

18 ranson robables n BN X Gven Markov blanke parens hldren and her parens X s ndependen of all oher nodes X \ \ X Markov blanke: markov X pa h markov pa X j h Compuaon s lnear n he sze of Markov blanke! : j pa X j h j j pa j 8

19 Ordered Gbbs Samplng Algorhm earl988 Inpu: X Ee Oupu: samples { } F evdene Ee nalze 0 a random. For o ompue samples. For o N loop hrough varables 3. + X markov 4. End For 5. End For

20 Gbbs Samplng Eample - BN X { X X... X 9} E { X 9} X X3 X6 X 0 X X 0 X X5 X8 X X X4 X7 X9 X X X

21 Gbbs Samplng Eample - BN X { X X... X 9} E { X 9} X X3 X6 0 0 X X X5 X8 0 X X4 X7 X9

22 Answerng Queres e? Mehod : oun # of samples where X hsogram esmaor: markov X X X δ Dra dela f-n Mehod : average probably mure esmaor: Mure esmaor onverges faser onsder esmaes for he unobserved values of X ; prove va Rao-Blakwell heorem

23 Rao-Blakwell heorem Rao-Blakwell heorem: Le random varable se X be omposed of wo groups of varables R and L. hen for he jon dsrbuon πrl and funon g he followng resul apples Var[ E{ g R L} Var[ g R] for a funon of neres g e.g. he mean or ovarane Casella&Rober996 Lu e. al heorem makes a weak promse bu works well n prae! mprovemen depends he hoe of R and L 3

24 Imporane vs. Gbbs Q g g e X Q X g X g e X e X e X ˆ ˆ ˆ w Gbbs: Imporane:

25 Gbbs Samplng: Convergene Sample from Xe Xe Converges ff han s rreduble and ergod Inuon - mus be able o eplore all saes: f X and X j are srongly orrelaed X 0 X j 0 hen we anno eplore saes wh X and X j All ondons are sasfed when all probables are posve Convergene rae an be haraerzed by he seond egen-value of ranson mar 5

26 Gbbs: Speedng Convergene Redue dependene beween samples auoorrelaon Skp samples Randomze Varable Samplng Order Employ blokng groupng Mulple hans Redue varane over n he ne seon 6

27 Blokng Gbbs Sampler Sample several varables ogeher as a blok Eample: Gven hree varables XYZ wh domans of sze group Y and Z ogeher o form a varable W{YZ} wh doman sze 4. hen gven sample y z ompue ne sample: + Can mprove onvergene grealy when wo varables are srongly orrelaed! - Doman of he blok varable grows eponenally wh he #varables n a blok! Z Y w z y w z y X

28 Gbbs: Mulple Chans Generae M hans of sze K Eah han produes ndependen esmae m : 8 M m M ˆ K m K e \ rea m as ndependen random varables. Esmae e as average of m e :

29 Gbbs Samplng Summary Markov Chan Mone Carlo mehod Gelfand and Smh 990 Smh and Robers 993 erney 994 Samples are dependen form Markov Chan Sample from X e whh onverges o Guaraneed o onverge when all > 0 Mehods o mprove onvergene: Blokng Rao-Blakwellsed X e 9

30 Overvew. robabls Reasonng/Graphal models. Imporane Samplng 3. Markov Chan Mone Carlo: Gbbs Samplng 4. Samplng n presene of Deermnsm 5. Rao-Blakwellsaon 6. AND/OR mporane samplng

31 Samplng: erformane Gbbs samplng Redue dependene beween samples Imporane samplng Redue varane Aheve boh by samplng a subse of varables and negrang ou he res redue dmensonaly aka Rao-Blakwellsaon Eplo graph sruure o manage he era os 3

32 Smaller Subse Sae-Spae Smaller sae-spae s easer o over X { X X X 3 X 4} X { X X } D X 64 D X 6 3

33 Smooher Dsrbuon X X X 3 X X X

34 Speedng Up Convergene Mean Squared Error of he esmaor: MSE MSE [ ] BIAS Var [ ] Q + In ase of unbased esmaor BIAS0 Q Q [ ] ˆ E [ ] [ ˆ] Var [ ˆ] Q EQ Q Redue varane speed up onvergene! 34

35 Rao-Blakwellsaon 35 } { ~ ]} [ { } { } { ˆ ]} [ { } { ]} {var[ ]} [ { } { ]} [ ] [ { ~ } { ˆ g Var l h E Var h Var g Var l g E Var g Var l g E l g E Var g Var l h E l h E g h h g L R X Lu Ch..3

36 Rao-Blakwellsaon Carry ou analyal ompuaon as muh as possble - Lu XR L Imporane Samplng: Var Q R L { } Var Q R L Q R { } Q R Lu Ch..5.5 Gbbs Samplng: auoovaranes are lower less orrelaon beween samples f X and X j are srongly orrelaed X 0 X j 0 only nlude one fo hem no a samplng se 36

37 Blokng Gbbs Sampler vs. Collapsed X Y Z Faser Convergene Sandard Gbbs: y z y z z y Blokng: y z y z Collapsed: y y 3 37

38 Collapsed Gbbs Samplng Generang Samples Generae sample + from : 38 \ e C e C e C e C K K K K K K sampled from In shor for o K:

39 Collapsed Gbbs Sampler Inpu: C X Ee Oupu: samples { } F evdene Ee nalze 0 a random. For o ompue samples. For o N loop hrough varables 3. + C \ 4. End For 5. End For

40 Calulaon me Compung \ e s more epensve requres nferene radng #samples for smaller varane: generae more samples wh hgher ovarane generae fewer samples wh lower ovarane Mus onrol he me spen ompung samplng probables n order o be meeffeve! 40

41 Eplong Graph roperes Reall ompuaon me s eponenal n he adjused ndued wdh of a graph w-use s a subse of varable s.. when hey are observed ndued wdh of he graph s w when sampled varables form a w-use nferene s epw e.g. usng Buke ree Elmnaon yle-use s a speal ase of w-use Samplng w-use w-use samplng! 4

42 Wha If CCyle-Cuse? { 5 } E { X 9} 5 9 an ompue usng Buke Elmnaon X X X3 X X3 X4 X5 X6 X4 X6 X7 X8 X9 X7 X8 X9 5 9 ompuaon ompley s ON 4

43 Compung ranson robables 43 : 0 : BE BE X X7 X5 X4 X X9 X8 X3 X α α α Compue jon probables: Normalze:

44 Cuse Samplng-Answerng Queres Query: C e? same as Gbbs: ˆ e \ e ompued whle generang sample usng buke ree elmnaon Query: X\C e? e e ompue afer generang sample usng buke ree elmnaon 44

45 Cuse Samplng vs. Cuse Condonng 45 e e oun e e e C D C D Cuse Condonng Cuse Samplng e e e C D

46 Cuse Samplng Eample Esmang e for samplng node X : X X4 X X5 X3 X Sample Sample Sample 3 X7 X8 X

47 Cuse Samplng Eample 47 } { } { } { Esmang 3 e for non-sampled node X 3 : X X7 X6 X5 X4 X X9 X8 X

48 CCS54 es Resuls CCS54 n54 C5 E3 CCS54 n54 C5 E Cuse Gbbs Cuse Gbbs # samples mese MSE vs. #samples lef and me rgh Ergod X54 DX C5 E3 Ea me 30 se usng Cuse Condonng 48

49 CCS79 es Resuls CCS79 n79 C8 E35 CCS79 n79 C8 E Cuse Gbbs Cuse Gbbs # samples mese MSE vs. #samples lef and me rgh Non-Ergod deermns C enry X 79 C 8 < DX <4 E 35 Ea me se usng Cuse Condonng 49

50 CCS360b es Resuls CCS360b n360 C E36 CCS360b n360 C E36 Cuse Gbbs Cuse Gbbs # samples mese MSE vs. #samples lef and me rgh Ergod X 360 DX C E 36 Ea me > 60 mn usng Cuse Condonng Ea Values obaned va Buke Elmnaon 50

51 Random Neworks RANDOM n00 C3 E5-0 Cuse Gbbs # samples RANDOM n00 C3 E5-0 Cuse Gbbs mese MSE vs. #samples lef and me rgh X 00 DX C 3 E 5-0 Ea me 30 se usng Cuse Condonng 5

52 Codng Neworks Cuse ransforms Non-Ergod Chan o Ergod 3 4 Codng Neworks n00 C-4 u u u 3 u 4 0. IB Gbbs Cuse p p p 3 p y y y 3 y mese MSE vs. me rgh Non-Ergod X 00 DX C 3-6 E 50 Sample Ergod Subspae U{U U U k } Ea me 50 se usng Cuse Condonng 5

53 Non-Ergod Halfnder HalFnder n56 C5 E HalFnder n56 C5 E 0. Cuse Gbbs Cuse Gbbs # samples mese MSE vs. #samples lef and me rgh Non-Ergod X 56 C 5 <DX < E 0 Ea me se usng Loop-Cuse Condonng 53

54 CCS360b - MSE ps360b N360 E[0-34] w*0 MSE Gbbs IB C6fw3 C48fw me se MSE vs. me Ergod X 360 C 6 DX Ea me 50 mn usng BE 54

55 Cuse Imporane Samplng Apply Imporane Samplng over use C w Q e e ˆ w e δ α w e e α where e s ompued usng Buke Elmnaon Gogae & Deher 005 and Bdyuk & Deher 006

56 Lkelhood Cuse Weghng LCS Zopologal Order{CE} Generang sample +: For If Z Z z Else z + + End If End For Z E do : z z Z e z +... z + KL[Ce QC] KL[Xe QX] ompued whle generang sample usng buke ree elmnaon an be memozed for some number of nsanes K based on memory avalable 56

57 ahfnder 57

58 ahfnder 58

59 Lnk 59

60 Summary Imporane Samplng..d. samples Unbased esmaor Generaes samples fas Samples from Q Reje samples wh zero-wegh Improves on use Gbbs Samplng Dependen samples Based esmaor Generaes samples slower Samples from Xe Does no onverge n presene of onsrans Improves on use 60

61 CCS360b MSE.E-0.E-03.E-04 ps360b N360 LC6 w* E5 LW AIS-BN Gbbs LCS IB.E me se LW lkelhood weghng LCS lkelhood weghng on a use 6

62 CCS4b.0E-0.0E-03 ps4b N4 LC47 w* E8 LW AIS-BN Gbbs LCS IB MSE.0E-04.0E me se LW lkelhood weghng LCS lkelhood weghng on a use 6

63 Codng Neworks.0E-0.0E-0 odng N00 3 LC6 w* LW AIS-BN Gbbs LCS IB MSE.0E-03.0E-04.0E me se LW lkelhood weghng LCS lkelhood weghng on a use 63