An Ising model on 2-D image

Transcription

1 School o Coputer Scence Approte Inerence: Loopy Bele Propgton nd vrnts Prolstc Grphcl Models Lecture 4, ov 7, 007 Receptor A Knse C Gene G Receptor B Knse D Knse E TF F 6 Gene H 7 8 Hetunndn Ksetty Redng: J-Chp. 5,6, KF-Chp. 8 An Isng odel on -D ge odes encode hdden norton ptchdentty. They receve locl norton ro the ge rghtness, color. Inorton s propgted though the grph over ts edges. Edges encode coptlty etween nodes. r or wter?? Erc ng

2 Why Approte Inerence? Why cn t we ust run uncton tree on ths grph? I grd, tree wdth tlest I ~O000, we hve clque wth 00 entres Erc ng 3 A recp: Fctor Grphs ψ ψ, Prents Undrected grph Drected grph Mrov rndo eld Byesn networ P = ψ ψ, Z P = P prents ctor grphs P = / Z F nterctons vrles Erc ng 4

3 3 Erc ng 5 M M M, ψ ψ Coptltes nterctons eternl evdence M ψ Bele Propgton BP Messge-updte Rules BP on trees lwys converges to ect rgnls c. Juncton tree lgorth Erc ng 6 Beles nd essges n FG eles essges = \ c c = \ \

4 Approte Inerence: Wht to pprote? Let us cll the ctul dstruton P We wsh to nd dstruton such tht s good pproton Recll the denton o KL-dvergence KL >=0 KL KL =0 = We cn thereore use KL s scorng uncton to decde good But, KL KL P = / Z = F log Erc ng 7 Whch KL? Coputng KLP requres nerence! But KL P cn e coputed wthout perorng nerence on P KL P = log P Usng = = H log log P E log P P = / Z F KL P = H E log/ Z F F = H log/ Z E log Erc ng 8 4

5 Optzton uncton KL P = H E log + log Z F F P, We wll cll F P, the Free energy * F P, P =? FP, >= FP,P *Gs Free Energy Erc ng 9 The Free Energy Let us loo t the ree energy E log cn e coputed we hve rgnls over ech F F P, = H E log F = H log s hrder! Requres suton over ll possle vlues Coputng F, s thereore hrd n generl. Approch : Approte F P, wth esy to copute F P, Erc ng 0 5

6 Esy ree energes Consder tree-structured dstruton d The prolty cn e wrtten s: = Htree = log + d log FTree = log + d log = F + F F67 + F78 F F5 F F6 F3 F7 nvolves suton over edges nd vertces nd s thereore esy to copute Erc ng Bethe Approton to Gs Free Energy For generl grph, choose F P, = log + d log Clled Bethe pproton ter the physcst Hns Bethe Equl to the ect Gs ree energy when the ctor grph s tree ote: Ths s not the se s the entropy o tree 3 4 F ethe = F F3 + + F67 + F78 F F5 F F6.... F 8 Erc ng 6

7 Bethe Approton Pros: Esy to copute, snce entropy ter nvolves su over prwse nd sngle vrles Cons: F P, = F y or y not e well connected to F P, ethe It could, n generl, e greter, equl or less thn F P, Optze ech 's. For dscrete ele, constrned opt. wth Lgrngn ultpler For contnuous ele, not yet generl orul ot lwys converge Erc ng 3 Mnzng the Bethe Free Energy L = F + + Set dervtve to zero Bethe γ { λ } \ Erc ng 4 7

8 Proo Erc ng 5 Erc ng 6 8

9 9 Erc ng 7 = = log log λ \ Bethe = BP We hd Identy to otn BP equtons: ep d λ + log ep λ Erc ng 8 A ed pont terton procedure tht tres to nze F ethe Strt wth rndo ntlzton o essges nd eles Whle not converged do At convergence, sttonrty propertes re gurnteed However, not gurnteed to converge! Loopy Bele Propgton = c c new \ = new \ \

10 0 Erc ng 9 Regon grphs It wll e useul to loo eplctly t the essges eng pssed Messges ro vrle to ctors Messges ro ctors to vrles Let us represent ths grphclly Erc ng 0 Sury so r = F Z P / = F E H P F log, + = d P F log log, ep d λ + log ep λ

11 Better pprotons? Recll tht Bethe pproton ws F ethe F 78 = F + F 5 3 F F F F 67 F.. F We could construct gger regons Rules: I regon ncludes ctor, t ust nclude the vertces s well Ech ctor nd verte ust pper n tlest one regon Assocte weght wth ech regon so tht ech verte nd ctor s counted ectly once Erc ng ^ F = F F 6 56 F 36 + F F F + F + F + F 3 7 Other regons? Erc ng

12 Regon-sed Approtons to the Gs Free Energy Kuch, 95 Ect: G[ q ] Regons: G[{ r r }] ntrctle Erc ng 3 Generlzed Bele Propgton Bele n regon s the product o: Locl norton ctors n regon Messges ro prent regons Messges nto descendnt regons ro prents who re not descendnts. Messge-updte rules otned y enorcng rgnlzton constrnts. Erc ng 4

13 Generlzed Bele Propgton Erc ng 5 Generlzed Bele Propgton Erc ng 6 3

14 Generlzed Bele Propgton [ 45][ ] 45 Erc ng 7 Generlzed Bele Propgton [ ] 45 ] [ Erc ng 8 4

15 Soe results Erc ng 9 Sury We dened n oectve uncton F or pprote nerence However, we ound tht optzng ths uncton ws hrd We rst pproted oectve uncton F to spler F ethe Mn o F ethe turned out to e ed ponts o BP Then we etended ths to ore coplcted pprotons The resultng lgorths coe under ly clled Generlzed Bele Propgton et clss, we wll cover other ethods o pprotons Erc ng 30 5