Density estimation III.

Transcription

1 Lecure 6 esy esmao III. Mlos Hausrec mlos@cs..eu 539 Seo Square Oule Oule: esy esmao: Bomal srbuo Mulomal srbuo ormal srbuo Eoeal famly

2 aa: esy esmao {.. } a vecor of arbue values Objecve: ry o esmae e uerlyg rue robably srbuo over varables X X usg eamles rue srbuo samles X.. } { esmae X Saar assumos: Samles are eee of eac oer come from e same ecal srbuo fe X Beroull rals aa: a sequece of oucomes suc a ea al 0 Moel: robably of a ea robably of a al robably of a oucome of a co fl Beroull srbuo ML Soluo: ML - umber of eas a als resecvely

3 oseror srbuo oseror esy va Bayes rule - s e ror robably o ror Leloo of aa ormalzg facor Bea Cojugae coce of ror: Bea oseror srbuo * Bea Bea Bea Bea

4 Mamum a oseror robably Mamum a oseror esmae Selecs e moe of e oseror srbuo MA Bea Bea oce a arameers of e ror ac le cous of eas a als somemes ey are also referre o as ror cous MA Soluo: arg ma MA Bomal srbuo Eamle: a base co Oucomes: wo ossble values -- ea or al aa: a se of orer-eee oucomes We rea as a mul-se - umber of eas see Moel: robably of a ea robably of a al - umber of als see robably of a oucome Bomal srbuo

5 Mamum leloo ML esmae. Leloo of aa: Log-leloo l Cosa from e o of omzao ML ML Soluo: e same as for Beroull a w sequece of eamles oseror esy oseror esy ror coce Leloo oseror MA esmae ma arg MA va Bayes rule Bea Bea MA

6 Eece value of e arameer e resul s e same as for Beroull srbuo Eece value of e arameer recve robably of eve 0 Bea E E E Mulomal srbuo Eamle: Mul-way co oss roll of a ce aa: a se of rals reae as a mul-se Moel arameers: robably of aa leloo ML esmae: ML s.. - a umber of mes a oucome as bee see - robably of a oucome Mulomal srbuo

7 oseror esy a MA esmae Coce of e ror: rcle srbuo.... r r MA.. MA esmae: oseror esy.. r rcle s e cojugae coce for e mulomal Eece value e resul s aaous o e resul for bomal Rereses e recve robably of a eve r E 0 E Eecao base arameer esmae

8 Oer srbuos e same eas ca be ale o oer srbuos ycally we coose srbuos a beave well so a comuaos lea o ce soluos Eoeal famly of srbuos Cojugae coces for some of e srbuos from e eoeal famly: Bomal Bea Mulomal - rcle Eoeal Gamma osso Iverse Gamma Gaussa - Gaussa mea a Wsar covarace Oer srbuos Gamma srbuo: a b a b Eoeal srbuo: A secal case of Gamma for a b e b osso srbuo: b a a e b for [ 0 ] λ e λ λ for { 0 }

9 Gaussa ormal srbuo Gaussa: ~ arameers: - mea - saar evao esy fuco: e[ Eamle: ] arameer esmaes Logleloo l ML esmaes of e mea a varace: ML varace esmae s base E E Ubase esmae:

10 Mulvarae ormal srbuo Mulvarae ormal: arameers: - mea - covarace mar esy fuco: Eamle: e / / ~ arameer esmaes Logleloo ML esmaes of e mea a covaraces: Covarace esmae s base Ubase esmae: l E E

11 oseror of a mulvarae ormal Assume a ror o e mea a s ormally srbue: e e oseror of s ormally srbue / / e e * / / e / / oseror of a mulvarae ormal e e oseror of s ormally srbue e / /

12 Sequeal Bayesa arameer esmao Sequeal Bayesa aroac Uer e e esmaes of e oseror ca be comue cremeally for a sequece of aa os If we use a cojugae ror we ge bac e same oseror Assume we sl e aa e las eleme a e res e: A ew ror Eoeal famly Eoeal famly: all robably mass / esy fucos a ca be wre e eoeal ormal form a vecor of aural or caocal arameers a fuco referre o as a suffce sasc a fuco of s less mora a ormalzao cosa a aro fuco Oer commo form: [ ] e Z f Z { } Z e [ ] e A f A Z

13 Eoeal famly: eamles Beroull srbuo e e { } e Eoeal famly f e [ ] Z arameers?? Z?? Eoeal famly: eamles Beroull srbuo e e { } e Eoeal famly f e [ ] Z arameers oe e Z e

14 Eoeal famly: eamles Uvarae Gaussa srbuo Eoeal famly arameers e e???? Z [ ] e Z f ] e[ Eoeal famly: eamles Uvarae Gaussa srbuo Eoeal famly arameers e e / / / 4 e e Z [ ] e Z f ] e[

15 Eoeal famly For samles e leloo of aa s Imora: e mesoaly of e suffce sasc remas e same w e umber of samles [ ] e A e A e A Eoeal famly e leloo of aa s Omzg e leloo For e ML esmae mus ol e A l A 0 A l A

16 Rewrg e grae: Eoeal famly A Z e e { } A e { } { A } A e A E Resul: E For e ML esmae e arameers soul be ajuse suc a e eecao of e sasc s equal o e observe samle sascs { } Momes of e srbuo For e eoeal famly e - mome of e sasc corresos o e - ervave of A If s a comoe of e we ge e momes of e srbuo by ffereag s corresog aural arameer Eamle: Beroull e A e ervaves: A e e e e A e