Density estimation II

Transcription

1 CS 750 Mche Lerg Lecture 6 esty estmto II Mlos Husrecht mlos@tt.edu 539 Seott Squre t: esty estmto {.. } vector of ttrute vlues Ojectve: estmte the model of the uderlyg rolty dstruto over vrles X X usg emles true dstruto smles X.. } { estmte ˆ X

2 esty estmto yes of desty estmto: rmetrc the dstruto s modeled usg set of rmeters ˆ X X Emle: me d covrces of multvrte orml Estmto: fd rmeters descrg dt o-rmetrc he model of the dstruto utlzes ll emles As f ll emles were rmeters of the dstruto Emles: erest-eghor Model ML rmeter estmto ˆ X X Θ Mmum lelhood ML rg m ML ML.. t log log {.. } Log lelhood hs the sme mmum s lelhood Ideedet emles

3 Byes rmeter estmto Byes rmeter estmto Uses the osteror dstruto for rmeters osteror covers ll ossle rmeter vlues & ther weghts rmeter osteror t Lelhood rmeter ror ror o rmeter + t + = Cojugte choces: ror dstruto mtches the dt dstruto osteror s the sme tye rmeter estmto Other crter: Mmum osteror rolty MA mode of the osteror Model: MA ˆ X X Θ rg m Θ Eected vlue of the rmeter ˆ X X ΘEV Eectto te wth regrd to osteror me of the osteror EV E d MA 3

4 Beroull dstruto Model for rdom vrle wth two outcomes Rdom vrle: wo outcomes: 0 or struto: where s the rolty of = Emle: Co toss Outcomes: Hed = l =0 rolty of Hed ror dstruto Choce of ror: Bet dstruto Bet osteror dstruto s g Bet dstruto Bet Bet CS 750 Mche Lerg - Gmm fucto For teger vlues of Why to use Bet dstruto? Bet dstruto fts Beroull smle - cojugte choces 4

5 5 CS 750 Mche Lerg Bet dstruto Bet osteror dstruto * = Bet Bet Bet Bet

6 Boml dstruto = * + 3* Emle rolem: co fls where ech co fl c hve two results: hed or tl Outcome: - umer of heds see - umer of tls see trls Model: rolty of hed rolty of tl rolty of outcome: Boml dstruto Boml dstruto: models order deedet sequece of Beroull trls Boml dstruto: Boml dstruto Bm0 0.5 Mtchg ror: Bet dstruto 6

7 7 Multoml dstruto Emle: multle rolls of de wth 6 results Outcome: couts of occurreces of ossle outcomes of trls: Model rmeters: rolty dstruto: ML estmte: ML s.t. - umer of tmes outcome hs ee see - rolty of outcome Multoml dstruto osteror d MA estmte Choce of the ror: rchlet dstruto.... r r MA.. MA estmte: osteror desty.. r rchlet s the cojugte choce for the multoml smlg

8 rchlet dstruto: Assume: =3 rchlet dstruto r.. 3 Other dstrutos he sme des c e led to other dstrutos yclly we choose dstrutos tht ehve well so tht comuttos led to ce solutos Eoetl fmly of dstrutos Cojugte choces for some of the dstrutos from the eoetl fmly: Boml Bet Multoml - rchlet Eoetl Gmm osso Iverse Gmm Guss - Guss me d Wshrt covrce 8

9 Guss orml dstruto Guss: ~ rmeters: - me - stdrd devto esty fucto: e[ Emle: ] rmeter estmtes Loglelhood l log ML estmtes of the me d vrce: ˆ ˆ ˆ ML vrce estmte s sed E E ˆ Used estmte: ˆ ˆ 9

10 Multvrte orml dstruto Multvrte orml: ~ rmeters: - me - covrce mtr esty fucto: e d / / Emle: rttoed Guss strutos Multvrte Guss: Emle: recso mtr Wht re the dstrutos for mrgls d codtols? 0

11 Codtol desty: Codtols d Mrgls Mrgl esty: Codtols d Mrgls

12 rmeter estmtes Loglelhood ML estmtes of the me d covrces: Covrce estmte s sed Used estmte: ˆ ˆ ˆ ˆ log l ˆ ˆ ˆ E E ˆ ˆ ˆ osteror of the me of multvrte orml Assume ror o the me tht s ormlly dstruted: he the osteror of s ormlly dstruted d / / e e * / / d e / / d

13 3 CS 750 Mche Lerg osteror of the me of multvrte orml he the osteror of s ormlly dstruted e / / d CS 750 Mche Lerg Other dstrutos Gmm dstruto: Eoetl dstruto: A secl cse of Gmm for = osso dstruto: e e e for ] 0 [ for } 0 {

14 4 Other dstrutos Gmm dstruto: e for ] 0 [ E vr Sequetl Byes rmeter estmto Sequetl Byes roch Uder the d the estmtes of the osteror c e comuted cremetlly for sequece of dt ots If we use cojugte ror we get c the sme osteror Assume we slt the dt the lst elemet d the rest he: d Θ Θ Θ d Θ Θ Θ Θ A ew ror