グラフィカルモデルによる推論確率伝搬法 (2) Kenji Fukumizu The Institute of Statistical Mathematics 計算推論科学概論 II (2010 年度, 後期 )

Size: px

Start display at page:

Download "グラフィカルモデルによる推論確率伝搬法 (2) Kenji Fukumizu The Institute of Statistical Mathematics 計算推論科学概論 II (2010 年度, 後期 )"

Ursula Leslie Warner
5 years ago
Views:

1 グラフィカルモデルによる推論確率伝搬法 Kenj Fukuzu he Insue of Sascal Maheacs 計算推論科学概論 II 年度後期

2 Inference on Hdden Markov Model

3 Inference on Hdden Markov Model Revew: HMM odel : hdden sae fne Inference Coue... for any Naïve couaon requres OK oeraons exonenal on he sequence lengh. K: nuber of hdden saes

4 Belef Proagaon on HMM BP for undreced ree reresenaon Clque oenals ψ ψ ψ ψ ψ ψ ψ ψ ψ ψ

5 Belef Proagaon on HMM Uward essage assng ψ s gven. A A: ranson arx A udae rule

6 Belef Proagaon on HMM Downward essage assng ψ A A udae rule

7 Belef Proagaon on HMM Margnals 7 Hence and s are gven. s arbrary

8 Forward-Backward Algorh Suary Forward-backward algorh algorh A A 8 Margnals lkelhood of any soohng

9 Forward-Backward Algorh Meanng of and 9 Proof: hoework

10 Forward-Backward Algorh he ordnary dervaon of algorh uses as defnons and derves he udae rules. Confr agan Confr agan by - -

11 Forward-Backward Algorh Couaonal cos of he forward-backward algorh cos s OK whch s lnear o he sequence lengh. Soohng flerng and redcon are done by he algorh; soohng: flerng: redcon: A

12 Forward-Backward Algorh Predcon and flerng are coued sequenally. For each e se he udae of wh he new observaon s suffcen. We do no need o access he older varables of s.

13 Mn-Suary Belef roagaon s alcable o he nference of HMM HMM s a ree BP s alcable. BP for soohng derves he forward-backward algorh. Soohng for all he hdden varables s done by he couaon of he lnear cos n he lengh. BP for redcon and flerng derve sequenal forward algorh.

14 Inference on Non-ree Grahs

15 Mehods for Non-ree Grahs ooy Belef Proagaon Alcaon of BP udaes o general grahs hough hey have loos. An aroxaon algorh. here s no heorecal guaranee for convergence or correcness. Juncon ree Algorh Proagaon algorh on he clque ree. Exacness of he resulng argnals are guaraneed whle he argnals are obaned only for he clques. Effcency of he algorh deends on he clque ree derved fro he orgnal grah.

16 ooy Belef Proagaon Murhy K. Wess. and Jordan M AGORIHM he udae rule s he sae as he BP for rees. j ne k k j j j j } \{ ψ j k j j he order of udaes s arbrary: an arbrary orderng sulaneous udaes ec. Reea he udaes unl soe convergence creron s sasfed. Coue all he aroxaed argnals by b b ne j j b

17 ooy Belef Proagaon here are no heorecal guaranees for convergence or correcness. Curren research ssue. In any raccal exales looy BP shows fas convergence and hgh accuracy. Decodng ehod of error correcng codes urbo-code 7

18 Juncon ree Algorh Basc dea: argnalzaon by elnaon Exale e e 8 argnalzaon s easy e ψ

19 Juncon ree Algorh e e New clques aears n he rocess of successve elnang varables. 9 e ψ argnalzng ou connecs and argnalzng ou connecs and

20 Juncon ree Algorh Skech of J algorh. Moralzaon. rangulaon.. Fnd all he clques. Make a juncon ree clque ree.. Proagae essages Margnal robables of all he clques.

21 Moralzaon Moralzaon: connec arens for each node. Make an undreced grah by reovng he drecons. No addonal condonal ndeendence relaons are suggesed by oralzaon.

22 rangulaon chordless rangulaon: ake a grah such ha here are no loos of lengh larger han whou chord. A chord s an edge ha connecs non-consecuve wo nodes n a loo. rangulaon guaranees he runnng nersecon roery of he juncon ree.

23 Juncon ree Fnd all he clques Make a juncon ree Juncon ree ree of he clques Each edge has a searaor nersecon of conneced nodes Runnng nersecon roery: f a varable aears n ulle nodes us aears n all neredae nodes n he ree.

24 Juncon ree Proagaon ψ C C C: clque Inal oenals ψ C C Run belef roagaon on he juncon ree gven by he roduc of oenals n C AB A B A ψ A A CA \ B C N A C A A ψ A AB A B B A B ψ B Afer he uward and downward udaes he argnals are gven by C ψ C C S φs S.

25 Juncon ree Proagaon Rearks: he runnng nersecon roery of he juncon ree ensures ha he roagaon rocedure gves argnal-ou. clque ree No rangulaed Runnng nersecon does no hold. he couaonal cos of J deends on he sze of clques. If he orgnal grah s colee here s no gan.

26 Mn-Suary Proagaon algorhs for non-ree grahs ooy BP: aroxaon algorh Drec alcaon of he BP udae rule for general grahs. Juncon ree algorh: exac argnalzaon for clques. Exenson of roagaon algorh BP-ye algorhs are obaned f he wo oeraons sasfy he dsrbuon law. e.g. ax-roduc ax-su

EP2200 Queuing theory and teletraffic systems. 3rd lecture Markov chains Birth-death process - Poisson process. Viktoria Fodor KTH EES

EP2200 Queuing theory and teletraffic systems. 3rd lecture Markov chains Birth-death process - Poisson process. Viktoria Fodor KTH EES EP Queung heory and eleraffc sysems 3rd lecure Marov chans Brh-deah rocess - Posson rocess Vora Fodor KTH EES Oulne for oday Marov rocesses Connuous-me Marov-chans Grah and marx reresenaon Transen and

グラフィカルモデルによる推論 確率伝搬法 (2) Kenji Fukumizu The Institute of Statistical Mathematics 計算推論科学概論 II (2010 年度, 後期 )

グラフィカルモデルによる推論確率伝搬法 (2) Kenji Fukumizu The Institute of Statistical Mathematics 計算推論科学概論 II (2010 年度, 後期 )