Lecture 2: Bayesian inference - Discrete probability models

Transcription

1 cu : Baysian infnc - Disc obabiliy modls

2 Many hings abou Baysian infnc fo disc obabiliy modls a simila o fqunis infnc Disc obabiliy modls: Binomial samling Samling a fix numb of ials fom a Bnoulli ocss A Bnoulli ocss is a sis of ials (y, y, ) o wh in ach ial h wo ossibl oucoms (succss and failu) h obabiliy of succss is consan = o wh h mmbs of h s of ossibl squncs y (),, y (M) all wih s succsss and f failus (s + f = M) a xchangabl Th numb of succsss, in n ials is binomial disibud n n ~ n!! n! n, 0,,, n

3 Hygomic samling Samling a fix numb n of ims (wihou lacmn) fom a fini s of N ims. Th fini s of ims o conains N = R ims of a scific y ( succss im) Th numb of succss ims, among h n samld ims is hygomic disibud R N R n N n ~, 0,,, min( n, R )

4 ascal samling Samling a andom numb of ials fom a Bnoulli ocss unil a dmind numb of succsss has bn obaind. Th numb of ials ndd is a andom vaiabl n wih a ascal o Ngaiv binomial disibuion ~ n n n n,, n,, Scial cas, whn = : Fis succss (Fs) disibuion ~ n n n, n,, Rlad o h Gomic disibuion ~ x x x, x 0,,

5 Th oisson ocss A couning ocss wih so-calld indndn incmns Th vns o b cound aas wih an innsiy () Th numb of vns aaing in h im inval (, ) is oisson disibud wih man d i. ~ d,, d,! 0,, Mos common cas: () (consan) and = 0. = (homognous ocss): ~,,! 0,,

6 Bays hom alid o disc obabiliy disibuions Wha is obsvd is wha (nomally) has a disc obabiliy disibuion. In Binomial samling w obsv h numb of succsss In Hygomic samling w obsv h numb of succss ims in h saml In ascal samling w obsv how many ims nd o b samld In a couning ximn w coun h numb of vns in a scifid im inval Hnc, h disc obabiliy disibuion alicabl uls h liklihood.

7 Bays hom on a vy gnic fom: Daa ; Daa wh is h obabiliy masu alicabl o h aam and ( ; Daa) is h liklihood of in ligh of h obsvd Daa. Hnc, fo binomial samling fo hygomic samling fo ascal samling fo couning in a homognous oisson ocss n n n, n N n N N n N,, n n n,!, ooionaliy consan: d d Daa ; ofn ;Daa

8 Exnding wih hy aams ( ) Daa, ψ ; Daa ψ Whn is coninuous and h obabiliy masu is Rimann- Siljs ingabl (h is a cumulaiv disibuion funcion) f Daa, ψ ; Daa f ψ wh f sands fo a obabiliy dnsiy funcion (is fom may vy wll dnd on h condiions ( and (, Daa) scivly)

9 Excis 3.4 You fl ha, h obabiliy of hads on a oss of a aicula coin is ih 0.4, 0.5 o 0.6. You io obabiliis a (0.4) = 0., (0.5) = 0.7 and (0.6) = 0.. You oss h coin h ims and obain hads onc and ails wic. Wha a h osio obabiliis? If you hn oss h coin h mo ims and onc again obain hads onc and ails wic, wha a h osio obabiliis? Also, comu h osio obabiliis by ooling h wo samls and vising h oiginal obabiliis jus onc; coma wih you vious answs. iklihoods fom fis saml: 0.4; Fis 0.5; Fis ; Fis

10 osio obabiliis: Fis 0.4 Fis ;Fis 0.4; Fis ; Fis ; Fis ; Fis ; Fis ; Fis ; Fis Fis Fis

11 iklihoods fom scond saml: 0.4;Scond 0.5;Scond ;Scond Ths a h sam valus as wih h fis saml sinc h saml oucoms a idnical.

12 osio obabiliis af scond saml; Scond,(Fis) ;Scond Fis 0.4;Scond 0.4 Fis 0.5;Scond 0.5 Fis 0.6;Scond 0.6 Fis 0.4;Scond 0.4 Fis 0.4;Scond 0.4 Fis 0.5;Scond 0.5 Fis 0.6;Scond 0.6 Fis 0.4 Scond,(Fis) Scond,(Fis) Scond,(Fis)

13 iklihoods fom oold samls: 0.4; oold 0.5; oold ; oold

14 osio obabiliis: oold 0.4 oold ;oold 0.4; oold ; oold ; oold ; oold ; oold ; oold ; oold oold oold

15 Comaison: 0.4 Scond,(Fis) oold Scond,(Fis) oold Scond,(Fis) oold Idnical suls! Excd? Scond,(Fis) ;Scond Fis 0.4;Scond 0.4 Fis 0.5;Scond 0.5 Fis 0.6;Scond 0.6 Fis ;Scond Fis ;Scond ;Fis 0.4; Fis ; Fis ; Fis 0.6 ;Scond ;Fis 4 3 3

16 oold 6 ;oold 0.4; oold ; oold ; oold Hnc h wo ways of comuing h osio obabiliis using boh samls a always idnical

17 Excis 3.33 Suos ha you fl ha accidns along a aicula sch of highway occu oughly accoding o a oisson ocss and ha h innsiy of h ocss is ih, 3 o 4 accidns wk. You io obabiliis fo hs h ossibl innsiis a 0.5, 0.45 and 0.30, scivly. If you obsv h highway fo a iod of h wks and accidns occu, wha a you osio obabiliis? iklihoods: ; 3; 4;,,, ! 33! 43! 33 43!

18 osio obabiliis:, 3 Faculis and 3 cancl ou ms 3 3! 3 3! 33! ! 34 4, 3, ,

19 diciv disibuions Fo an unknown aam of ins,, w would accoding o h subjciv inaion of obabiliy assign a io disibuion uon obaining daa lad o, comu a osio disibuion Th io and osio disibuions a usd o mak infnc abou h unknown xlanaoy infnc W may also b insd in diciv infnc, i.. dic daa lad o no y obaind Fo coss-scional daa h m dicion is mosly usd, whil fo im sis daa w ah us h m focasing.

20 y,, y M, b h s (fini o infini) of obsvd valus w may obain und condiions uld by h unknown. Th uncainy associad wih ach obsvaion i.. ha is valu/sa canno b known in advanc is modlld by ling h obsvd valu b h alisaion of a andom vaiabl y wih a obabiliy disibuion dnding on : ~ io-diciv disibuions y y f y k Th io-diciv disibuion of y is h s of maginal obabiliis obaind whn h dndncy on is ingad/summd ou by wighing h obabiliy mass funcion f (y ) wih h io disibuion of : ~ y y k f f ~ y y k k d if if k assums assums a numabl valus on a s of coninuous valus scal

21 osio-diciv disibuions Th osio-diciv disibuion of y is h s of maginal obabiliis obaind whn h dndncy on is ingad/summd ou by wighing h obabiliy mass funcion f (y ) wih h osio disibuion of givn an alady obaind s of obsvaions (Daa): ~ y y f k f Daa ~ y Daa y Daa k k d if assums a numabl s of valus if assums valus on a coninuous scal