Lecture 13: Markov Chain Monte Carlo. Gibbs sampling

Transcription

1 Lecture 13: Markv hain Mnte arl Gibbs sampling Gibbs sampling Markv chains 1 Recall: Apprximate inference using samples Main idea: we generate samples frm ur Bayes net, then cmpute prbabilities using (weighted) cunts) But we may need a lt f wrk t get enugh samples (eg if the PDs are very extreme) Rejectin sampling and likelihd weighting are als specific t directed mdels 2

2 Recall: rward sampling P() = 5 ludy P(S) Sprinkler Rain P(R) 0 20 S R P() et Grass Each sample is cnstructed frm scratch! e saw tw alternatives fr incrprating the evidence hrw away samples that are incnsistent with it rce the evidence variables, but then weigh the samples In bth cases, after a sample is cnstructed, we start a new ne frm scratch 3 A different idea P() = 5 ludy P(S) Sprinkler Rain P(R) 0 20 S R P() et Grass Suppse we want t cmpute e generate ne sample, with the given evidence variables instantiated crrectly hen we keep changing it! If we are careful, we will get samples frm the crrect distributin

3 H G 1 Initializatin Gibbs sampling Set evidence variables, t the bserved values Set all ther variables t randm values (eg by frward sampling, unifrm sampling) his gives us a sample 2 Repeat (as much as wanted) Pick a nn-evidence variable unifrmly randmly Sample frm! " $# Keep all ther values: % &')(+*-, he new sample is % 3 Alternatively, yu can march thrugh the variables in sme predefined rder 5 hy Gibbs wrks in Bayes nets he key step is sampling accrding t!/ " 0 1# 2 Hw d we cmpute this? In Bayes nets, we knw that a variable is cnditinally independent f all ther given its Markv blanket (parents, children, spuses) 3!/ " 0 1# 2 MarkvBlanket S we need t sample frm 4 MarkvBlanket Let 56'7* :9 be the children f Yu will shw (next hmewrk) that: ;=<?> :@ <BA ;<1> MarkxBlanket )DE :@ <?A Parents )D JIK <1M Parents D 6

4 V Example P() = 5 ludy P(S) Sprinkler Rain P(R) 0 20 et Grass S R P() 1 Generate a first sample: N 2 Pick, sample it frm N :P JP Q Suppse we get 3 ur new sample is N JP 4 7 Analyzing Gibbs sampling nsider the variables -R Each pssible assignment f values t these variables is a state f the wrld, SK2U In Gibbs sampling, we start frm a given state V S U Based n this, we generate a new state, S U V depends nly n V! here is a well-defined prbability f ging frm V t V Gibbs sampling cnstructs a Markv chain ver the Bayes net

5 f V A Markv chain is defined by: A set f states Markv chains V 3 V X A starting distributin ver the set f states YX e ften put these in a vectr Z\[ A statinary transitin prbability ^]_]a` V/b # r cnvenience, we ften put these in a a c dec matrix P V X=f V gf hhhif V b f V b # jf Vb V V 9 Steady-state (statinary) distributin here will the chain be in 1 step? lk lkx m ^ lx In tw steps? Yn n X In steps? b b " b lx A statinary distributin is a distributin left invariant by the chain: Nte that sme chains can have mre than ne statinary distributin! 10

6 ] Detailed balance nsider the statinary distributin: V p V _ V V his can be viewed as a flw prperty: the flw ut f V has t be equal t the flw cming int V frm all states ne way t ensure this is t make flw equal between any pair f states: V a V V p V a V V his gives us a sufficient cnditin fr statinarity, called detailed balance (why) A Markv chain with this prperty is called reversible 11 Mnte arl Markv hain (MM) Suppse we want t sample data frm sme distributin e will set up a Markv chain which has the desired distributin as its statinary distributin! r this we wuld like the chain t have a unique statinary distributin, s that we can get samples frm it regardless f the starting distributin 12

7 Ergdicity An ergdic Markv chain is ne in which any state is reachable frm any ther state, and there are n strictly peridic cycles In such a chain, there is a unique statinary distributin, which can be btained as: qsrut b vxw b his is called equilibrium distributin Nte that the chain reaches the equilibrium distributin regardless f X 13 Sampling the equilibrium distributin e can sample just by running the chain a lng time: fr sme arbitrary Set V X p r zy, if V b V, sample a value V fr V b # based n V V Return V{ If y is large enugh, this will be a sample frm In practice, yu d like t have a rapidly mixing chain, ie ne that reaches the equilibrium quickly 14

8 m m m Implementatin issues he initial samples are influenced by the starting distributin, s they need t be thrwn away his is called the burn-in stage Because burn-it can take a while, we d like t draw several samples frm the same chain! Hwever, if we take samples,,! }, they will be highly crrelated Usually we wait fr burn-in, then take every c ~ sample, fr sme c sufficiently large his will ensure that the samples are (fr all practical purpses) uncrrelated 15 Gibbs sampling as MM ƒ, with e have a set f randm variables evidence variables e want t sample frm & Let be the variable t be sampled, currently set t, and ˆ m be the values fr all ther variables in he transitin prbability fr the chain is: V V p ˆ Š bviusly the chain is ergdic e want t shw that & is the statinary distributin 16

9 m Gibbs satisfies detailed balance V a V V & ˆ & ˆ & ˆ & V V V ˆ & ˆ & ˆ & K ˆ & ˆ Š (by chain rule) (backwards chain rule) 17