xt Inrn Kyn tngli
roilisti Inrn n Lrning W now v opt rprsnttions o proility istriutions: rpil Mols M M sris uniqu proility istriution Typil tsks: Tsk 1: How o w nswr quris out M.g. M XY? W us inrn s n or t pross o oputing nswrs to su quris Tsk 2: How o w stit plusil ol M ro t? y tis I n t grp strutur n/or t prtrs i. W us lrning s n or t pross o otining point stit o M. ii. ut or ysin ty sk pm wi is tully n inrn prol. iii. Wn not ll vrils r osrvl vn oputing point stit o M n to o inrn to iput t issing t. ri Xing @ MU 2005-2015 2
Qury 1: Liklioo Most o t quris on y sk involv vin vin is n ssignnt o vlus to st vrils in t oin Witout loss o gnrlity = { X k+1 X n } Siplst qury: oput proility o vin x!x = å" å tis is otn rrr to s oputing t liklioo o x 1 x k 1 k ri Xing @ MU 2005-2015 3
Qury 2: onitionl roility Otn w r intrst in t onitionl proility istriution o vril givn t vin X X X = = å X = x tis is t postriori li in X givn vin Mrginliztion: t pross o suing out t unosrv or "on't r" vrils z is ll. Y = å Y Z = z z x ri Xing @ MU 2005-2015 4
pplitions o postriori li rition : wt is t proility o n outo givn t strting onition g! " # t qury no is snnt o t vin ignosis: wt is t proility o iss/ult givn syptos g!#" t qury no n nstor o t vin Not: T irtionlity o inortion low twn vrils is not rstrit y t irtionlity o t gs in M. You will s or pplition uring Lrning unr prtil osrvtion?? ri Xing @ MU 2005-2015 5
Qury 3: Most rol ssignnt l In tis qury w wnt to in t ost prol joint ssignnt M or so vrils o intrst l Su rsoning is usully pror unr so givn vin n ignoring t vlus o otr vrils z : M Y = rg x yî Y y = rg x yîy å z y z l tis is t xiu postriori onigurtion o y. ri Xing @ MU 2005-2015 7
pplitions o M lssiition in ost likly ll givn t vin xplntion wt is t ost likly snrio givn t vin utionry not: T M o vril pns on its "ontxt"---t st o vrils n jointly quri y 1 y 2 y 1 y 2 xpl: 0 0 0.35 M o Y 1? M o Y 1 Y 2? 0 1 0.05 1 0 0.3 1 1 0.3 ri Xing @ MU 2005-2015 8
oplxity o Inrn T: oputing X = x in M is N-r l Hrnss os not n w nnot solv inrn. It siply sys tt tr xist iiult inrn prols. It pns on t strutur. l l It iplis tt w nnot in gnrl prour tt works iintly or ritrry Ms or prtiulr ilis o Ms w n v provly iint prours ri Xing @ MU 2005-2015 9
Lnsp o inrn lgorits xt inrn lgorits T liintion lgorit Mssg-pssing lgorit su-prout li propgtion T juntion tr lgorits pproxit inrn tniqus Stosti siultion / spling tos Mrkov in Mont rlo tos Vritionl lgorits ri Xing @ MU 2005-2015 10
Vril liintion
Mrginliztion n liintion signl trnsution ptwy: Qury: Wt is t liklioo tt protin is tiv? = åååå y in oposition w gt = åååå nïv sution ns to nurt ovr n xponntil nur o trs ri Xing @ MU 2005-2015 25
ååå å åååå = = liintion on ins Rrrnging trs irst su! tn "... 26
ååå ååå å = = p Now w n pror innrost sution Tis sution "liints" on vril ro our sution rgunt t "lol ost". X liintion on ins 27
åå åå å ååå = = = p p p X X liintion in ins Rrrnging n tn suing gin w gt ri Xing @ MU 2005-2015 28
liintion in ins X X X X liint nos on y on ll t wy to t n w gt = å p oplxity: lvr liintion tks!#$ % vrsus!$ ' or t nïv ppro. ri Xing @ MU 2005-2015 29
ssoitivity o trix ultiplition Roo 1 Roo 2 Roo 3 x 1 x 2 x T px 1 x T =px 1 TY 1 t=1 px t+1 x t px t+1 = ix t = j =M ij 2 3 0.7 0.5 0 40.3 0.3 0.55 0 0.2 0.5
ssoitivity o trix ultiplition Roo 1 Roo 2 Roo 3 x 1 x 2 x T px 5 = ix 1 = 1 = X px t+1 = ix t = j =M ij 2 3 0.7 0.5 0 40.3 0.3 0.55 0 0.2 0.5 x 4 x 3 x 2 px 5 x 4 px 4 x 3 px 3 x 2 px 2 x 1 = 1 =[M 4 v] i
rt Is in ML: Mssg ssing ount t solirs tr's 1 o 1 or you 2 or you 3 or you 4 or you 5 or you 5 in you 4 in you 3 in you 2 in you 1 in you pt ro MKy 2003 txtook 32
rt Is in ML: Mssg ssing ount t solirs 2 or you tr's 1 o li: Must 2 + 1 + 3 = 6 o us only s y inoing ssgs 3 in you pt ro MKy 2003 txtook 33
rt Is in ML: Mssg ssing ount t solirs 1 or you tr's 1 o li: Must 1 + 1 + 4 = 6 o us li: Must 2 + 1 + 3 = 6 o us only s y inoing ssgs 4 in you pt ro MKy 2003 txtook 34
Wt out gnrl?
rt Is in ML: Mssg ssing solir rivs rports ro ll rns o tr 3 r 7 r 1 o 11 r = 7+3+1 pt ro MKy 2003 txtook 40
rt Is in ML: Mssg ssing solir rivs rports ro ll rns o tr 3 r 7 r = 3+3+1 3 r pt ro MKy 2003 txtook 41
rt Is in ML: Mssg ssing solir rivs rports ro ll rns o tr 11 r = 7+3+1 7 r 3 r pt ro MKy 2003 txtook 42
rt Is in ML: Mssg ssing solir rivs rports ro ll rns o tr 3 r 7 r 3 r li: Must 14 o us pt ro MKy 2003 txtook 43
rt Is in ML: Mssg ssing solir rivs rports ro ll rns o tr 3 r 7 r 3 r li: Must 14 o us wouln't work orrtly wit 'loopy' yli grp pt ro MKy 2003 txtook 44
or oplx ntwork oo w H Wt is t proility tt wks r lving givn tt t grss onition is poor? ri Xing @ MU 2005-2015 52
Qury: N to liint: H Initil tors: oos n liintion orr: H Stp 1: onitioning ix t vin no i.. on its osrv vlu i.. : Tis stp is isoorpi to rginliztion stp: H g ~ p = = ~ = å = p ~ xpl: Vril liintion ri Xing @ MU 2005-2015 53
Qury: N to liint: Initil tors: Stp 2: liint H g g 1 = = å g g g p xpl: Vril liintion ri Xing @ MU 2005-2015 54
Qury: N to liint: Initil tors: Stp 3: liint g g H xpl: Vril liintion = å p ri Xing @ MU 2005-2015 55
Qury: N to liint: Initil tors: Stp 4: liint H xpl: Vril liintion g g = å p ri Xing @ MU 2005-2015 56
Qury: N to liint: Initil tors: Stp 5: liint H xpl: Vril liintion g g = å p 57
Qury: N to liint: Initil tors: Stp 6: liint H xpl: Vril liintion = å p g g ri Xing @ MU 2005-2015 58
g g Qury: N to liint: Initil tors: Stp 7: liint H xpl: Vril liintion = å p ri Xing @ MU 2005-2015 59
Qury: N to liint: Initil tors: Stp 8: Wrp-up H xpl: Vril liintion g g ~ p p = å = p p ~ = å p p ~ ri Xing @ MU 2005-2015 60
grp liintion lgorit orliztion H H grp liintion Unrstning Vril liintion ri Xing @ MU 2005-2015 62
rp liintion gin wit t unirt M or orliz N rp V n liintion orring I liint nxt no in t orring I Roving t no ro t grp onnting t rining nigors o t nos T ronstitut grp 'V ' Rtin t gs tt wr rt uring t liintion prour T grp-torti proprty: t tors rsult uring vril liintion r ptur y roring t liintion liqu ri Xing @ MU 2005-2015 63
grp liintion lgorit Intrit trs orrspon to t liqus rsult ro liintion orliztion H H grp liintion Unrstning Vril liintion H ri Xing @ MU 2005-2015 64
liintion liqus H H Mssgs snt: H g Mssgs snt: ri Xing @ MU 2005-2015 65
rp liintion n rginliztion Inu pnny uring rginliztion vs. liintion liqu Sution <-> liintion Intrit tr <-> liintion liqu H g g ri Xing @ MU 2005-2015 66 1 2 2 1 3 3 4 4 5 5 6 6 7 7
liqu tr Mssg ro on 1 to 2: Multiply ll inoing ssgs wit t lol tor n su ovr vrils tt r not sr g H = å p g ri Xing @ MU 2005-2015 67
oplxity T ovrll oplxity is trin y t nur o t lrgst liintion liqu Wt is t lrgst liintion liqu? pur grp torti qustion Tr-wit k: on lss tn t sllst ivl vlu o t rinlity o t lrgst liintion liqu rnging ovr ll possil liintion orring goo liintion orrings l to sll liqus n n ru oplxity wt will ppn i w liint "" irst in t ov grp? in t st liintion orring o grp --- N-r à Inrn is N-r ut tr otn xist "ovious" optil or nr-opt liintion orring ri Xing @ MU 2005-2015 68
xpls Str Tr ri Xing @ MU 2005-2015 69
Mor xpl: Ising ol ri Xing @ MU 2005-2015 70
Liittion o rour liintion Liittion H H ri Xing @ MU 2005-2015 71
Our lgorit so r nswrs only on qury.g. on on no o w n to o oplt liintion or vry su qury? liintion º ssg pssing on liqu tr Mssgs n rus H g H H º ro liintion to Mssg ssing = å g p ri Xing @ MU 2005-2015 72
ro liintion to Mssg ssing Our lgorit so r nswrs only on qury.g. on on no o w n to o oplt liintion or vry su qury? liintion º ssg pssing on liqu tr notr qury... g H Mssgs n r rus otrs n to roput ri Xing @ MU 2005-2015 73
Sury T sipl liint lgorit pturs t ky lgoriti Oprtion unrlying proilisti inrn: --- Tt o tking su ovr prout o potntil untions Wt n w sy out t ovrll oputtionl oplxity o t lgorit? In prtiulr ow n w ontrol t "siz" o t suns tt ppr in t squn o sution oprtion. T oputtionl oplxity o t liint lgorit n ru to purly grptorti onsirtions. Tis grp intrprttion will lso provi ints out ow to sign iprov inrn lgorit tt ovro t liittion o liint. ri Xing @ MU 2005-2015 74