Machine Learning. Recap of Basic Prob. Concepts

Size: px

Start display at page:

Download "Machine Learning. Recap of Basic Prob. Concepts"

Godwin Thomas
5 years ago
Views:

1 Min Lrnin 0-70/5 70/ ll 2006 ril Mols II Inrn ri in Visit to si Sokin 2 Turulosis Lun nr ronitis Ltur 3 Otor Turulosis or nr 6 Ry Rsult 7 ysn 8 Rin:. 8. ook ri in R o si ro. onts Joint roility ist. on ultil vrils: I i 's r innnt: i i I i 's r onitionlly innnt s sri y M t joint n tor to silr routs.. i ri in 2 3 i

2 Mrkov Rno ils Strutur: n unirt r Mnin: no is onitionlly innnt o vry otr no in t ntwork ivn its irt niors Lol ontinny untions otntils n t liqus in t r oltly trin t joint ist. Y Y 2 iv orrltions twn vrils ut no xliit wy to nrt sls ri in 3 Rrsnttion n: n unirt ril ol rrsnts istriution n in y n unirt r n st o ositiv otntil untions y ssoit wit liqus o s.t. x K xn ψ x Z wr Z is known s t rtition untion: Z x K x n ψ x lso known s Mrkov Rno ils Mrkov ntworks T otntil untion n unrstoo s n ontinny untion o its runts ssinin "r-roilisti" sor o tir joint oniurtion. ri in 4 2

3 Ms r your ol rins nsity stition rtri n nonrtri tos Rrssion Linr onitionl ixtur nonrtri lssiition nrtiv n isriintiv ro s Q Y Q ri in 5 n inolt nloy o ril ols itur y Zouin rni n S Rowis ri in 6 3

4 roilisti Inrn W now v ot rrsnttions o roility istriutions: ril Mols M M sris uniqu roility istriution ow o w nswr quris out? W us inrn s n or t ross o outin nswrs to su quris ri in 7 Qury : Liklioo Most o t quris on y sk involv vin vin is n ssinnt o vlus to st vrils in t oin Witout loss o nrlity { k+ n } Silst qury: out roility o vin K x xk x Kx k tis is otn rrr to s outin t liklioo o ri in 8 4

5 Qury 2: onitionl roility Otn w r intrst in t onitionl roility istriution o vril ivn t vin tis is t ostriori li in ivn vin x x W usully qury sust Y o ll oin vrils {YZ} n "on't r" out t rinin Z: Y YZ z t ross o suin out t "on't r" vrils z is ll rinliztion n t rsultin y is ll rinl ro. z ri in 9 litions o ostriori li rition: wt is t roility o n outo ivn t strtin onition? t qury no is snnt o t vin inosis: wt is t roility o iss/ult ivn sytos? t qury no n nstor o t vin Lrnin unr rtil osrvtion ill in t unosrv vlus unr n "M" sttin or ltr T irtionlity o inortion low twn vrils is not rstrit y t irtionlity o t s in M roilisti inrn n oin vin or ll rts o t ntwork ri in 0 5

6 Qury 3: Most rol ssinnt In tis qury w wnt to in t ost rol joint ssinnt M or so vrils o intrst Su rsonin is usully ror unr so ivn vin n inorin t vlus o otr vrils z : M Y r x y r x y z y y z tis is t xiu ostriori oniurtion o y. ri in litions o M lssiition in ost likly ll ivn t vin xlntion wt is t ost likly snrio ivn t vin utionry not: T M o vril ns on its "ontxt"---t st o vrils n jointly quri xl: M o? M o Y? x y xy ri in 2 6

7 olxity o Inrn T: outin x in M is N-r rnss os not n w nnot solv inrn It ilis tt w nnot in nrl rour tt works iintly or ritrry Ms or rtiulr ilis o Ms w n v rovly iint rours ri in 3 ros to inrn xt inrn lorits T liintion lorit T juntion tr lorits ut will not ovr in til r roxit inrn tniqus Stosti siultion / slin tos Mrkov in Mont rlo tos Vritionl lorits will ovr in vn ML ourss ri in 4 7

8 8 ri in 5 sinl trnsution twy: Qury: y in oosition w t nïv sution ns to nurt ovr n xonntil nur o trs Wt is t liklioo tt rotin is tiv? Mrinliztion n liintion ri in 6 liintion on ins Rrrnin trs...

9 9 ri in 7 Now w n ror innrost sution Tis sution "liints" on vril ro our sution runt t "lol ost". liintion on ins ri in 8 liintion in ins Rrrnin n tn suin in w t

10 liintion in ins liint nos on y on ll t wy to t n w t olxity: st osts OVl i *Vl i+ ortions: Okn 2 or to nïv vlution tt sus ovr joint vlus o n- vrils On k ri in 9 Inrn on nrl M vi Vril liintion nrl i: Writ qury in t or L x i i xn x3 x2 i tis susts n "liintion orr" o ltnt vrils to rinliz Itrtivly Mov ll irrlvnt trs outsi o innrost su ror innrost su ttin nw tr Insrt t nw tr into t rout wr-u ri in 20 0

11 ri in 2 oo w Wt is t roility tt wks r lvin ivn tt t rss onition is oor? or olx ntwork ri in 22 Qury: N to liint: Initil tors: oos n liintion orr: St : onitionin ix t vin no i.. on its osrv vlu i.. : Tis st is isoori to rinliztion st: ~ ~ ~ δ rultory ntwork xl: Vril liintion

12 2 ri in 23 Qury: N to liint: Initil tors: St 2: liint out xl: Vril liintion ri in 24 Qury: N to liint: Initil tors: St 3: liint out xl: Vril liintion

13 3 ri in 25 Qury: N to liint: Initil tors: St 4: liint out xl: Vril liintion ri in 26 Qury: N to liint: Initil tors: St 5: liint out xl: Vril liintion

14 4 ri in 27 Qury: N to liint: Initil tors: St 6: liint out xl: Vril liintion ri in 28 Qury: N to liint: Initil tors: St 7: liint out xl: Vril liintion

15 5 ri in 29 Qury: N to liint: Initil tors: St 8: Wr-u xl: Vril liintion ~ ~ ~ ri in 30 Suos in on liintion st w out Tis rquirs ultilitions or vlu o x y y k w o k ultilitions itions or vlu o y y k w o Vl itions olxity is xonntil in nur o vrils in t intrit tor x k x k x y y x y y ' K K k i i k x i x y y x ' y K i i k Vl Vl Y i i Vl Vl Y olxity o vril liintion

16 6 ri in 3 orliztion r liintion Unrstnin Vril liintion r liintion lorit ri in 32 liintion liqus

17 Unrstnin Vril liintion r liintion lorit orliztion r liintion Intrit trs orrson to t liqus rsult ro liintion oo liintion orrins l to sll liqus n n ru olxity wt will n i w liint "" irst in t ov r? inin t otiu orrin is N-r ut or ny r otiu or nrotiu n otn uristilly oun lis to unirt Ms ri in 33 liqu tr ri in 34 7

18 ro liintion to Mss ssin Our lorit so r nswrs only on qury.. on on no o w n to o olt liintion or vry su qury? liintion ss ssin on liqu tr Msss n rus ri in 35 ro liintion to Mss ssin Our lorit so r nswrs only on qury.. on on no o w n to o olt liintion or vry su qury? liintion ss ssin on liqu tr notr qury... Msss n r rus otrs n to rout ri in 36 8

19 Skt o t Juntion Tr lorit T lorit onstrution o juntion trs --- sil liqu tr rotion o roilitis --- ss-ssin rotool Rsults in rinl roilitis o ll liqus --- solvs ll quris in sinl run nri xt inrn lorit or ny M olxity: xonntil in t siz o t xil liqu --- oo liintion orr otn ls to sll xil liqu n n oo i.. tin JT Mny wll-known lorits r sil ss o JT orwr-kwr Kln iltr lin Su-rout... ri in 37 ros to inrn xt inrn lorits T liintion lorit T juntion tr lorits ut will not ovr in til r roxit inrn tniqus Stosti siultion / slin tos Mrkov in Mont rlo tos Vritionl lorits ltr lturs ri in 38 9

20 Mont rlo tos rw rno sls ro t sir istriution Yil stosti rrsnttion o olx istriution rinls n otr xtions n roxit usin sl-s vrs N t [ x] x N sytotilly xt n sy to ly to ritrry ols llns: ow to rw sls ro ivn ist. not ll istriutions n trivilly sl? ow to k ttr us o t sls not ll sl r usul or qlly usul s n xl ltr? ow to know w'v sl nou? t ri in 39 xl: niv slin Slin: onstrut sls orin to roilitis ivn in N J lr xl: oos t rit slin squn Slin:< > suos it is ls 0. S or < > suos it is ls... 2 rquny ountin: In t sls rit /0</9 8/9> M J ri in 40 20

21 xl: niv slin Slin: onstrut sls orin to roilitis ivn in N. lr xl: oos t rit slin squn 3 wt i w wnt to out J? w v only on sl... JJ/<0 > J 4 wt i w wnt to out J? No su sl vill! JJ/ n not in. or ol wit unrs or or vrils rr vnts will vry r to rnr vou sls vn tr lon ti or slin M J ri in 4 Mont rlo tos on. irt Slin W v sn it. Vry iiult to oult i-insionl stt s Rjtion Slin rt sls lik irt slin only ount sls wi is onsistnt wit ivn vins.... Mrkov in Mont rlo MM ri in 42 2

22 Mrkov in Mont rlo Sls r otin ro Mrkov in o squntilly volvin istriutions wos sttionry istriution is t sir x is slin w v vril st to {x x 2 x 3... x N } t st on o t vrils i is slt t rno or orin to so ix squns t onitonl istriution i -i is out vlu x i is sl ro tis istriution t sl x i rls t rvious o i in. ri in 43 MM Mrkov-lnkt vril is innnt ro otrs ivn its rnts ilrn n ilrn s rnts. -srtion. i -i i M i is slin rt rno sl. vry st oos on vril n sl it y M s on rvious sl. M{ J M} M{ } ri in 44 22

23 MM To lult JM oos 0MJ s strt vins r M vrils r J. oos nxt vril s Sl y M 0 M J suos to ls. 0 0 M J oos nxt rno vril s sl ~0... ri in 45 olxity or roxit Inrn roxit Inrn will not r t xt roility istriution in init ti ut only los to t vlu. Otn u str tn xt inrn wn N is i n olx nou. In MM only onsir M ut not t wol ntwork. ri in 46 23

Exact Inference. Kayhan Batmanghelich

Exact Inference. Kayhan Batmanghelich xt Inrn Kyn tngli roilisti Inrn n Lrning W now v opt rprsnttions o proility istriutions: rpil Mols M M sris uniqu proility istriution Typil tsks: Tsk 1: How o w nswr quris out M.g. M XY? W us inrn s n