Exact Inference. Kayhan Batmanghelich

Transcription

1 xt Inrn Kyn tngli

2 roilisti Inrn n Lrning W now v opt rprsnttions o proility istriutions: rpil Mols M M sris uniqu proility istriution Typil tsks: Tsk 1: How o w nswr quris out M.g. M XY? W us inrn s n or t pross o oputing nswrs to su quris Tsk 2: How o w stit plusil ol M ro t? y tis I n t grp strutur n/or t prtrs i. W us lrning s n or t pross o otining point stit o M. ii. ut or ysin ty sk pm wi is tully n inrn prol. iii. Wn not ll vrils r osrvl vn oputing point stit o M n to o inrn to iput t issing t. ri MU

3 Qury 1: Liklioo Most o t quris on y sk involv vin vin is n ssignnt o vlus to st vrils in t oin Witout loss o gnrlity = { X k+1 X n } Siplst qury: oput proility o vin x!x = å" å tis is otn rrr to s oputing t liklioo o x 1 x k 1 k ri MU

4 Qury 2: onitionl roility Otn w r intrst in t onitionl proility istriution o vril givn t vin X X X = = å X = x tis is t postriori li in X givn vin Mrginliztion: t pross o suing out t unosrv or "on't r" vrils z is ll. Y = å Y Z = z z x ri MU

5 pplitions o postriori li rition : wt is t proility o n outo givn t strting onition g! " # t qury no is snnt o t vin ignosis: wt is t proility o iss/ult givn syptos g!#" t qury no n nstor o t vin Not: T irtionlity o inortion low twn vrils is not rstrit y t irtionlity o t gs in M. You will s or pplition uring Lrning unr prtil osrvtion?? ri MU

6 Qury 3: Most rol ssignnt l In tis qury w wnt to in t ost prol joint ssignnt M or so vrils o intrst l Su rsoning is usully pror unr so givn vin n ignoring t vlus o otr vrils z : M Y = rg x yî Y y = rg x yîy å z y z l tis is t xiu postriori onigurtion o y. ri MU

7 pplitions o M lssiition in ost likly ll givn t vin xplntion wt is t ost likly snrio givn t vin utionry not: T M o vril pns on its "ontxt"---t st o vrils n jointly quri y 1 y 2 y 1 y 2 xpl: M o Y 1? M o Y 1 Y 2? ri MU

8 oplxity o Inrn T: oputing X = x in M is N-r l Hrnss os not n w nnot solv inrn. It siply sys tt tr xist iiult inrn prols. It pns on t strutur. l l It iplis tt w nnot in gnrl prour tt works iintly or ritrry Ms or prtiulr ilis o Ms w n v provly iint prours ri MU

9 Lnsp o inrn lgorits xt inrn lgorits T liintion lgorit Mssg-pssing lgorit su-prout li propgtion T juntion tr lgorits pproxit inrn tniqus Stosti siultion / spling tos Mrkov in Mont rlo tos Vritionl lgorits ri MU

10 Vril liintion

11 Mrginliztion n liintion signl trnsution ptwy: Qury: Wt is t liklioo tt protin is tiv? = åååå y in oposition w gt = åååå nïv sution ns to nurt ovr n xponntil nur o trs ri MU

12 ååå å åååå = = liintion on ins Rrrnging trs irst su! tn "... 26

13 ååå ååå å = = p Now w n pror innrost sution Tis sution "liints" on vril ro our sution rgunt t "lol ost". X liintion on ins 27

14 åå åå å ååå = = = p p p X X liintion in ins Rrrnging n tn suing gin w gt ri MU

15 liintion in ins X X X X liint nos on y on ll t wy to t n w gt = å p oplxity: lvr liintion tks!#$ % vrsus!$ ' or t nïv ppro. ri MU

16 ssoitivity o trix ultiplition Roo 1 Roo 2 Roo 3 x 1 x 2 x T px 1 x T =px 1 TY 1 t=1 px t+1 x t px t+1 = ix t = j =M ij

17 ssoitivity o trix ultiplition Roo 1 Roo 2 Roo 3 x 1 x 2 x T px 5 = ix 1 = 1 = X px t+1 = ix t = j =M ij x 4 x 3 x 2 px 5 x 4 px 4 x 3 px 3 x 2 px 2 x 1 = 1 =[M 4 v] i

18 rt Is in ML: Mssg ssing ount t solirs tr's 1 o 1 or you 2 or you 3 or you 4 or you 5 or you 5 in you 4 in you 3 in you 2 in you 1 in you pt ro MKy 2003 txtook 32

19 rt Is in ML: Mssg ssing ount t solirs 2 or you tr's 1 o li: Must = 6 o us only s y inoing ssgs 3 in you pt ro MKy 2003 txtook 33

20 rt Is in ML: Mssg ssing ount t solirs 1 or you tr's 1 o li: Must = 6 o us li: Must = 6 o us only s y inoing ssgs 4 in you pt ro MKy 2003 txtook 34

21 Wt out gnrl?

22 rt Is in ML: Mssg ssing solir rivs rports ro ll rns o tr 3 r 7 r 1 o 11 r = pt ro MKy 2003 txtook 40

23 rt Is in ML: Mssg ssing solir rivs rports ro ll rns o tr 3 r 7 r = r pt ro MKy 2003 txtook 41

24 rt Is in ML: Mssg ssing solir rivs rports ro ll rns o tr 11 r = r 3 r pt ro MKy 2003 txtook 42

25 rt Is in ML: Mssg ssing solir rivs rports ro ll rns o tr 3 r 7 r 3 r li: Must 14 o us pt ro MKy 2003 txtook 43

26 rt Is in ML: Mssg ssing solir rivs rports ro ll rns o tr 3 r 7 r 3 r li: Must 14 o us wouln't work orrtly wit 'loopy' yli grp pt ro MKy 2003 txtook 44

27 or oplx ntwork oo w H Wt is t proility tt wks r lving givn tt t grss onition is poor? ri MU

28 Qury: N to liint: H Initil tors: oos n liintion orr: H Stp 1: onitioning ix t vin no i.. on its osrv vlu i.. : Tis stp is isoorpi to rginliztion stp: H g ~ p = = ~ = å = p ~ xpl: Vril liintion ri MU

29 Qury: N to liint: Initil tors: Stp 2: liint H g g 1 = = å g g g p xpl: Vril liintion ri MU

30 Qury: N to liint: Initil tors: Stp 3: liint g g H xpl: Vril liintion = å p ri MU

31 Qury: N to liint: Initil tors: Stp 4: liint H xpl: Vril liintion g g = å p ri MU

32 Qury: N to liint: Initil tors: Stp 5: liint H xpl: Vril liintion g g = å p 57

33 Qury: N to liint: Initil tors: Stp 6: liint H xpl: Vril liintion = å p g g ri MU

34 g g Qury: N to liint: Initil tors: Stp 7: liint H xpl: Vril liintion = å p ri MU

35 Qury: N to liint: Initil tors: Stp 8: Wrp-up H xpl: Vril liintion g g ~ p p = å = p p ~ = å p p ~ ri MU

36 grp liintion lgorit orliztion H H grp liintion Unrstning Vril liintion ri MU

37 rp liintion gin wit t unirt M or orliz N rp V n liintion orring I liint nxt no in t orring I Roving t no ro t grp onnting t rining nigors o t nos T ronstitut grp 'V ' Rtin t gs tt wr rt uring t liintion prour T grp-torti proprty: t tors rsult uring vril liintion r ptur y roring t liintion liqu ri MU

38 grp liintion lgorit Intrit trs orrspon to t liqus rsult ro liintion orliztion H H grp liintion Unrstning Vril liintion H ri MU

39 liintion liqus H H Mssgs snt: H g Mssgs snt: ri MU

40 rp liintion n rginliztion Inu pnny uring rginliztion vs. liintion liqu Sution <-> liintion Intrit tr <-> liintion liqu H g g ri MU

41 liqu tr Mssg ro on 1 to 2: Multiply ll inoing ssgs wit t lol tor n su ovr vrils tt r not sr g H = å p g ri MU

42 oplxity T ovrll oplxity is trin y t nur o t lrgst liintion liqu Wt is t lrgst liintion liqu? pur grp torti qustion Tr-wit k: on lss tn t sllst ivl vlu o t rinlity o t lrgst liintion liqu rnging ovr ll possil liintion orring goo liintion orrings l to sll liqus n n ru oplxity wt will ppn i w liint "" irst in t ov grp? in t st liintion orring o grp --- N-r à Inrn is N-r ut tr otn xist "ovious" optil or nr-opt liintion orring ri MU

43 xpls Str Tr ri MU

44 Mor xpl: Ising ol ri MU

45 Liittion o rour liintion Liittion H H ri MU

46 Our lgorit so r nswrs only on qury.g. on on no o w n to o oplt liintion or vry su qury? liintion º ssg pssing on liqu tr Mssgs n rus H g H H º ro liintion to Mssg ssing = å g p ri MU

47 ro liintion to Mssg ssing Our lgorit so r nswrs only on qury.g. on on no o w n to o oplt liintion or vry su qury? liintion º ssg pssing on liqu tr notr qury... g H Mssgs n r rus otrs n to roput ri MU

48 Sury T sipl liint lgorit pturs t ky lgoriti Oprtion unrlying proilisti inrn: --- Tt o tking su ovr prout o potntil untions Wt n w sy out t ovrll oputtionl oplxity o t lgorit? In prtiulr ow n w ontrol t "siz" o t suns tt ppr in t squn o sution oprtion. T oputtionl oplxity o t liint lgorit n ru to purly grptorti onsirtions. Tis grp intrprttion will lso provi ints out ow to sign iprov inrn lgorit tt ovro t liittion o liint. ri MU