How to represent a joint, or a marginal distribution?

Size: px

Start display at page:

Download "How to represent a joint, or a marginal distribution?"

Gilbert Griffin
5 years ago
Views:

1 School o Cou Scinc obabilisic Gahical ols Aoia Innc on Calo hos ic ing Lcu 8 Novb Raing ic CU How o sn a join o a aginal isibuion? Clos-o snaion.g. Sal-bas snaion ic CU

2 on Calo hos Daw ano sals o h si isibuion il a sochasic snaion o a col isibuion aginals an oh cions can b aoia using salbas avags N [ ] N Asyoically ac an asy o aly o abiay ols Challngs how o aw sals o a givn is. no all isibuions can b ivially sal? how o ak b us o h sals no all sal a usul o qally usul s an al la? how o know w'v sal nough? ic CU al naiv saling Consuc sals accoing o obabiliis givn in a BN. J A J Ala al Choos h igh saling squnc SalingB< > suos i is als. Sa o. A < > suos i is als... 2 Fquncy couning In h sals igh JJ/</9 8/9>. ic CU

3 al naiv saling Consuc sals accoing o obabiliis givn in a BN. Ala al Choos h igh saling squnc 3 wha i w wan o cou JA? w hav only on sal... JAJA/A<0 >. J 4 wha i w wan o cou JB? No such sal availabl! JAJB/B can no b in. Fo a ol wih huns o o vaiabls a vns will b vy ha o gan vough sals vn a a long i o saling... A J ic CU on Calo hos con. Dic Saling W hav sn i. Vy iicul o oula a high-insional sa sac Rjcion Saling Ca sals lik ic saling only coun sals which is consisn wih givn vincs. Liklihoo wighing... Sal vaiabls an calcula vinc wigh. Only ca h sals which suo h vincs. akov chain on Calo CC oolis-hasing Gibbs ic CU

4 Rjcion saling Suos w wish o sal o is. ΠΠ'/Z. Π is iicul o sal bu Π' is asy o valua Sal o a sil is Rjcion saling * * * * ~ acc w.. Π' / k Cocnss iall [ Π' / k ] [ Π' / k ] Π' Π Π' ic CU Rjcion saling iall Using Nµσ q2 I o sal Nµσ 2 I I σ q cs σ by % an insional000 Th oial accanc a kσ q /σ /20000 Big was o sals! Aaiv jcion saling Using nvlo uncions o in ic CU

5 5 ic CU Unnoaliz ioanc saling Suos saling o is ha. Suos w can sal o a "sil" oosal isibuion insa. I oinas i.. > 0 whnv > 0 w can sal o an wigh Wha is h obl h? w ~ wh ic CU Noaliz ioanc saling Suos w can only valua ' α.g. o an RF. W can g aoun h nasy noalizaion consan α as ollows Now α ' ' ' L w w wh ~ wh ' α

Noaliz vs unnoaliz ioanc saling Unoaliz ioanc saling is unbias [ w ] Noaliz ioanc saling is bias.g. o w w Howv h vaianc o h noaliz ioanc sal is usually low in acic.

6 Noaliz vs unnoaliz ioanc saling Unoaliz ioanc saling is unbias [ w ] Noaliz ioanc saling is bias.g. o w w Howv h vaianc o h noaliz ioanc sal is usually low in acic. Also i is coon ha w can valua ' bu no.g. ' / o Bays n o '/Z o RF. ic CU Liklihoo wighing W now aly noaliz ioanc saling o a Bays n. Th oosal is gon o h uila BN wh w cla vinc nos an cu hi incoing acs. Call his. Th unnoaliz osio is '. w δ i So o i δ i i w g ˆ i i i w wh w /. ic CU

7 Liklihoo wighing algoih ic CU icincy o liklihoo wighing Th icincy o ioanc saling ns on how clos h oosal is o h ag. Suos all h vinc is a h oos. Thn an all sals hav wigh. Suos all h vinc is a h lavs. Thn is h io so any sals igh g sall wigh i h vinc is unlikly. W can us ac vsal o ak so o h vinc nos b oos insa o lavs bu h suling nwok can b uch o nsly connc. ic CU

8 Wigh saling obl o ioanc saling ns on how wll achs I is songly vaying an has a signiican ooion o is ass concna in a sall gion will b oina by a w sals * * *** No ha i h high-ob ass gion o alls ino h low-ob ass gion o h vaianc o / can b sall vn i h sals co o low-ob gion o an onially onous. Soluion Us havy ail. Wigh saling / w l l / l ic CU Wigh saling Saling ioanc saling SIR. Daw N sals o N / 2. Consucing wighs w w N w l / l 3. Sub-sal o { N } w.. w w N l aicula Filing... A scial wigh sal il sals o osio A A A ic CU

9 ic ing @ CU 2005-2009 7 Skch o aicl Fils Th saing oin Thus is sn by A squnial wigh sal Ti ua asun ua

9 9 ic CU Skch o aicl Fils Th saing oin Thus is sn by A squnial wigh sal Ti ua asun ua w ~ w sal o a iu ol w ~ wigh ic CU F o swiching SS Rcall ha h bli sa has O2 Gaussian os

10 0 ic CU F o swiching SS Ky ia i you knw h isc sas you can aly h igh Kalan il a ach i s. So o ach ol aicl sal o h io aly h KF using aas o S o h ol bli sa o g an aoiaion o Usul o onlin acking aul iagnosis c. ~ S S S ˆ s y ic CU Rao-Blackwllis saling Saling in high insional sacs causs high vaianc in h sia. RB ia sal so vaiabls an coniional on ha cou c valu o s analyically This has low vaianc bcaus o h iniy ~ va va va τ τ τ

11 ic CU ic CU Rao-Blackwllis saling Saling in high insional sacs causs high vaianc in h sia. RB ia sal so vaiabls an coniional on ha cou c valu o s analyically This has low vaianc bcaus o h iniy Hnc so is a low vaianc siao. ~ va va va τ τ τ va va τ τ τ

12 Suay on Calo hos Dic Saling Vy iicul o oula a high-insional sa sac Rjcion Saling Ca sals lik ic saling only coun sals which is consisn wih givn vincs. Liklihoo wighing... Sal vaiabls an calcula vinc wigh. Only ca h sals which suo h vincs. akov chain on Calo CC oolis-hasing Gibbs ic CU

Probabilistic Graphical Models

Probabilistic Graphical Models School o Cour Scinc robabilisic Grahical Mols Aroia Inrnc: Mon Carlo hos Eric ing Lcur 6 March 7 204 Raing: S class wbsi Eric ing @ CMU 2005-204 Aroachs o inrnc Eac inrnc algorihs Th liinaion algorih Mssag-assing