Density estimation III.

Transcription

1 Lecure 4 esy esmao III. Mlos Hauskrec mlos@cs..edu 539 Seo Square Oule Oule: esy esmao: Mamum lkelood ML Bayesa arameer esmaes MP Beroull dsrbuo. Bomal dsrbuo Mulomal dsrbuo Normal dsrbuo Eoeal famly

2 Eoeal famly Eoeal famly: all robably mass / desy fucos a ca be wre e oeal ormal form f [ ] a vecor of aural or caocal arameers a fuco referred o as a suffce sasc a fuco of s less mora a ormalzao cosa a aro fuco { } d Oer commo form: [ ] f Eoeal famly: eamles Beroull dsrbuo { } Eoeal famly f [ ] Parameers

3 Eoeal famly: eamles Beroull dsrbuo { } Eoeal famly f [ ] Parameers oe e e Eoeal famly: eamles Uvarae Gaussa dsrbuo µ [ µ ] µ µ Eoeal famly f Parameers

4 Eoeal famly: eamles Uvarae Gaussa dsrbuo Eoeal famly Parameers µ µ / / µ / 4 µ f ] [ µ µ Eoeal famly For d samles e lkelood of daa s Imora: e dmesoaly of e suffce sasc remas e same for dffere samle szes a s dffere umber of eamles P

5 Eoeal famly e lkelood of daa s Omzg e lkelood For e ML esmae mus old l 0 l Eoeal famly Rewrg e grade: Resul: For e ML esmae e arameers sould be adjused suc a e ecao of e sasc s equal o e observed samle sascs { } d { } { } d d { } d E E

6 Momes of e dsrbuo For e oeal famly e k- mome of e sasc corresods o e k- dervave of If s a comoe of e we ge e momes of e dsrbuo by dffereag s corresodg aural arameer Eamle: Beroull ervaves: e e e e e e Cojugae rors For ay member of e oeal famly ere ess a ror: Suc a for eamles e oseror s Noe a: f P g u g

7 P Cojugae rors For ay member of e oeal famly ere ess a ror: Suc a for eamles e oseror s Noe a: Pror corresods o observaos w value. f g u g Noaramerc Meods Paramerc dsrbuo models are: resrced o secfc forms wc may o always be suable; Eamle: modellg a mulmodal dsrbuo w a sgle umodal model. Noaramerc aroaces: make few assumos abou e overall sae of e dsrbuo beg modelled.

8 Noaramerc Meods Hsogram meods: aro e daa sace o dsc bs w wds ad cou e umber of observaos eac b. N Ofe e same wd s used for all bs. acs as a smoog arameer. I a -dmesoal sace usg M bs eac dme-so wll requre M bs! Noaramerc Meods ssume observaos draw from a desy ad cosder a small rego R coag suc a P d R e robably a K ou of N observaos le sde R s BKNP ad f N s large K NP If e volume of R V s suffcely small s aromaely cosa over R ad us P V P V K NV

9 Noaramerc Meods: kerel meods Kerel esy Esmao: F V esmae K from e daa. Le R be a yercube cered o ad defe e kerel fuco Parze wdow k I follows a ad ece / / K 0 K N oerwse k N N k Noaramerc Meods: smoo kerels o avod dscoues because of sar boudares use a smoo kerel e.g. a Gaussa y kerel suc a acs as a smooer. wll work.

10 Noaramerc Meods: knn esmao Neares Negbour esy Esmao: f K esmae V from e daa. Cosder a yer-sere cered o ad le grow o a volume V* a cludes K of e gve N daa os. e K acs as a smooer Noaramerc vs Paramerc Meods Noaramerc models: More flebly o desy model s eeded Bu requre sorg e ere daase ad e comuao s erformed w all daa eamles. Paramerc models: Oce fed oly arameers eed o be sored ey are muc more effce erms of comuao Bu e model eeds o be cked advace

11 K-Neares-Negbours for Classfcao Gve a daa se w N k daa os from class C k ad we ave ad corresodgly Sce Bayes eorem gves K-Neares-Negbours for Classfcao K 3