CS 2750 Machine Learning Lecture 5. Density estimation. Density estimation

Size: px
Start display at page:

Download "CS 2750 Machine Learning Lecture 5. Density estimation. Density estimation"

Transcription

1 CS 750 Mache Learg Lecture 5 esty estmato Mlos Hausrecht mlos@tt.edu 539 Seott Square esty estmato esty estmato: s a usuervsed learg roblem Goal: Lear a model that rereset the relatos amog attrbutes the data.. } ata: { a vector of attrbute values Attrbutes: modeled by radom varables X { X wth X Xd} Cotuous or dscrete valued varables esty estmato: lear a uderlyg robablty dstrbuto model : X X X X from d

2 ata: esty estmato {.. } a vector of attrbute values Objectve: estmate the model of the uderlyg robablty dstrbuto over varables X X usg eamles true dstrbuto samles X.. } { estmate ˆ X esty estmato true dstrbuto samles X.. } { estmate ˆ X Stadard d assumtos: Samles are deedet of each other come from the same detcal dstrbuto fed X Ideedetly draw staces from the same fed dstrbuto

3 esty estmato yes of desty estmato: arametrc the dstrbuto s modeled usg a set of arameters ˆ X X Eamle: mea ad covaraces of a multvarate ormal Estmato: fd arameters descrbg data o-arametrc he model of the dstrbuto utlzes all eamles As f all eamles were arameters of the dstrbuto Eamles: earest-eghbor Learg va arameter estmato I ths lecture we cosder arametrc desty estmato Basc settgs: A set of radom varables X { X X Xd} A model of the dstrbuto over varables X wth arameters : ˆ X Eamle: Gaussa dstrbuto wth mea ad varace arameters ata {.. } Objectve: fd arameters such that X fts data the best 3

4 ML arameter estmato Model ˆ X X Θ ata {.. } Mamum lelhood ML Fd that mamzes the lelhood.. log-lelhood ML arg ma ˆ X X Θ log log arg ma ML Ideedet eamles log Bayesa arameter estmato he ML estmate cs just oe value of the arameter roblem: f there are two dfferet arameter values that are close terms of the lelhood usg oly oe of them may troduce a strog bas f we use t for eamle for redctos. Bayesa arameter estmato Remedes the lmtato of oe choce Uses the osteror dstrbuto for arameters osteror covers all ossble arameter values ad ther weghts arameter osteror ata Lelhood arameter ror 4

5 What does t do? Bayesa arameter estmato ror ad osteror covers all ossble arameter values ad ther weghts Assume: we have a model of wth a arameter Bayesa arameter estmato: ror o a arameter ML Estmate + ata + = ata + = osteror o a arameter Just oe value Bayesa arameter estmato Bayesa arameter estmato Uses the osteror dstrbuto for arameters osteror covers all ossble arameter values ad ther weghts arameter osteror How to use the osteror for modelg X? ˆ X X X Θ Θ dθ Θ ata Lelhood arameter ror 5

6 arameter estmato Other crtera: Mamum a osteror robablty MA mamze Θ mode of the osteror Yelds: oe set of arameters Θ MA Aromato: ˆ X X Θ MA Eected value of the arameter Θˆ E Θ mea of the osteror Eectato tae wth regard to osteror Θ Yelds: oe set of arameters Aromato: ˆ X X Θˆ arameter estmato. Co eamle. Co eamle: we have a co that ca be based Outcomes: two ossble values -- head or tal ata: a sequece of outcomes such that head tal 0 Model: robablty of a head robablty of a tal Objectve: We would le to estmate the robablty of a head from data ˆ 6

7 arameter estmato. Eamle. Assume the uow ad ossbly based co robablty of the head s ata: H H H H H H H H H H H H H H H Heads: 5 als: 0 What would be your estmate of the robablty of a head? ~? arameter estmato. Eamle Assume the uow ad ossbly based co robablty of the head s ata: H H H H H H H H H H H H H H H Heads: 5 als: 0 What would be your choce of the robablty of a head? Soluto: use frequeces of occurreces to do the estmate ~ hs s the mamum lelhood estmate of the arameter 7

8 robablty of a outcome ata: a sequece of outcomes such that head tal 0 Model: robablty of a head robablty of a tal Assume: we ow the robablty robablty of a outcome of a co fl Combes the robablty of a head ad a tal So that s gog to c ts correct robablty Gves for Gves for 0 Beroull dstrbuto robablty of a sequece of outcomes. ata: a sequece of outcomes such that head tal 0 Model: robablty of a head robablty of a tal Assume: a sequece of deedet co fls = H H H H ecoded as = 00 What s the robablty of observg the data sequece :? 8

9 robablty of a sequece of outcomes. ata: a sequece of outcomes such that head tal 0 Model: robablty of a head robablty of a tal Assume: a sequece of co fls = H H H H ecoded as = 00 What s the robablty of observg a data sequece : robablty of a sequece of outcomes. ata: a sequece of outcomes such that head tal 0 Model: robablty of a head robablty of a tal Assume: a sequece of co fls = H H H H ecoded as = 00 What s the robablty of observg a data sequece : lelhood of the data 9

10 robablty of a sequece of outcomes. ata: a sequece of outcomes such that head tal 0 Model: robablty of a head robablty of a tal Assume: a sequece of co fls = H H H H ecoded as = 00 What s the robablty of observg a data sequece : 6 Ca be rewrtte usg the Beroull dstrbuto: he goodess of ft to the data Learg: we do ot ow the value of the arameter Our learg goal: Fd the arameter that fts the data the best? Oe soluto to the best : Mamze the lelhood Ituto: more lely are the data gve the model the better s the ft ote: Istead of a error fucto that measures how bad the data ft the model we have a measure that tells us how well the data ft : Error 0

11 Mamum lelhood ML estmate. Lelhood of data: Mamum lelhood estmate ML arg ma Otmze log-lelhood the same as mamzg lelhood l log log log log log - umber of heads see - umber of tals see log Mamum lelhood ML estmate. Otmze log-lelhood l log log Set dervatve to zero Solvg l 0 ML Soluto: ML

12 Mamum lelhood estmate. Eamle Assume the uow ad ossbly based co robablty of the head s ata: H H H H H H H H H H H H H H H Heads: 5 als: 0 What s the ML estmate of the robablty of a head ad a tal? Mamum lelhood estmate. Eamle Assume the uow ad ossbly based co robablty of the head s ata: H H H H H H H H H H H H H H H Heads: 5 als: 0 What s the ML estmate of the robablty of head ad tal? Head: al: ML ML

13 3 Bayesa arameter estmato Uses the dstrbutos ror ad osteror over all ossble values of the arameter of the samlg dstrbuto Beroull: We ow that the lelhood s: How to choose the ror robablty? va Bayes theorem - s the ror robablty o ror Lelhood of data ormalzg factor osteror CS 750 Mache Learg ror dstrbuto Beta Beta Choce of ror: Beta dstrbuto Beta dstrbuto fts Beroull samle - cojugate choces Beta Why to use Beta dstrbuto? osteror dstrbuto s aga a Beta dstrbuto - a Gamma fucto! For teger values of

14 4 CS 750 Mache Learg Beta dstrbuto b a b a b a b a Beta osteror dstrbuto * = Beta Beta Beta Beta

15 5 osteror dstrbuto Beta osteror A cojugate ror to Beroull samle otce that arameters of the ror act le couts of heads ad tals sometmes they are also referred to as ror couts Beta Beta Mamum aosteror robablty MA Mamum a osteror estmate Selects the mode of the osteror dstrbuto Selects the model of the osteror rereseted as a Beta dstrbuto va Bayes rule ror Lelhood of data ormalzg factor ma arg MA Beta Beta

16 6 Mamum osteror robablty Mamum a osteror estmate Selects the mode of the osteror dstrbuto Assumes cojugate ror to Beroull samle MA MA Soluto: Beta Beta 0 log Mode of the osteror satsfes : CS 750 Mache Learg MA estmate eamle Assume the uow ad ossbly based co robablty of the head s ata: H H H H H H H H H H H H H H H Heads: 5 als: 0 Assume What s the MA estmate? 55 Beta

17 MA estmate eamle Assume the uow ad ossbly based co robablty of the head s ata: H H H H H H H H H H H H H H H Heads: 5 als: 0 Assume Beta 55 What s the MA estmate? MA 9 33 CS 750 Mache Learg MA estmate eamle ote that the ror ad data ft data lelhood are combed he MA ca be based wth large ror couts It s hard to overtur t wth a smaller samle sze ata: H H H H H H H H H H H H H H H Heads: 5 als: 0 Assume Beta 55 Beta 50 9 MA 33 9 MA 48 CS 750 Mache Learg 7

18 8 CS 750 Mache Learg Bayesa framewor redctve robablty of a outcome the et tral Equvalet to the eected value of the arameter eectato s tae wth resect to the osteror dstrbuto Beta 0 d 0 E d osteror desty CS 750 Mache Learg Eected value of the arameter How to calculate the eected value of Beta? d d Beta E 0 0 d 0 Beta d 0 ote: for teger values of

19 Eected value of the arameter Substtutg the results for the osteror: Beta We get E ote that the mea of the osteror s yet aother reasoable arameter choce: ˆ E CS 750 Mache Learg Bomal dstrbuto Eamle roblem: a based co Outcomes: two ossble values -- head or tal ata: a set of order-deedet outcomes for trals - umber of heads see - umber of tals see ca be calculated from the tral data!!! Model: robablty of a head robablty of a tal robablty of a outcome Objectve: We would le to estmate the robablty of a head Bomal dstrbuto ˆ 9

20 Bomal dstrbuto = * + 3* Eamle roblem: co fls where each co fl ca have two results: head or tal Outcome: - umber of heads see - umber of tals see trals Model: robablty of a head robablty of a tal robablty of a outcome: Bomal dstrbuto Bomal dstrbuto: models order deedet sequece of Beroull trals Bomal dstrbuto: Bomal dstrbuto 0

21 Mamum lelhood ML estmate. Lelhood of data: Log-lelhood!!! log log!!! log log l Costat from the ot of otmzato!!! ML ML Soluto: he same as for Beroull ad wth d sequece of eamles osteror desty osteror desty ror choce Lelhood osteror MA estmate ma arg MA va Bayes rule Beta Beta MA

22 Multomal dstrbuto Eamle: multle rolls of a dce wth 6 results Outcome: couts of occurreces of ossble outcomes of trals: Model arameters: robablty dstrbuto: ML estmate: ML θ s.t. - a umber of tmes a outcome has bee see!!!! θ - robablty of a outcome Multomal dstrbuto osteror ad MA estmate Choce of the ror: rchlet dstrbuto.... r r θ θ θ θ MA.. MA estmate: osteror desty.. r θ rchlet s the cojugate choce for the multomal samlg!!!! θ θ

23 rchlet dstrbuto: Assume: =3 rchlet dstrbuto r θ.. 3 Other dstrbutos he same deas ca be aled to other dstrbutos ycally we choose dstrbutos that behave well so that comutatos lead to ce solutos Eoetal famly of dstrbutos Cojugate choces for some of the dstrbutos from the eoetal famly: Bomal Beta Multomal - rchlet Eoetal Gamma osso Iverse Gamma Gaussa - Gaussa mea ad Wshart covarace 3

24 Gaussa ormal dstrbuto Gaussa: ~ arameters: - mea - stadard devato esty fucto: e[ Eamle: ] CS 750 Mache Learg arameter estmates Loglelhood l log ML estmates of the mea ad varace: ˆ ˆ ˆ ML varace estmate s based E E ˆ Ubased estmate: ˆ ˆ CS 750 Mache Learg 4

25 Multvarate ormal dstrbuto Multvarate ormal: ~ arameters: - mea - covarace matr esty fucto: e d / / Eamle: CS 750 Mache Learg arttoed Gaussa strbutos Multvarate Gaussa: Eamle: recso matr What are the dstrbutos for margals ad codtoals? a a b 5

26 arttoed Codtoals ad Margals Codtoal desty: Margal esty: arttoed Codtoals ad Margals 6

27 7 CS 750 Mache Learg arameter estmates Loglelhood ML estmates of the mea ad covaraces: Covarace estmate s based Ubased estmate: ˆ ˆ ˆ ˆ log l ˆ ˆ ˆ E E ˆ ˆ ˆ CS 750 Mache Learg osteror of a multvarate ormal Assume a ror o the mea that s ormally dstrbuted: he the osteror of s ormally dstrbuted d / / e e * / / d e / / d

28 8 CS 750 Mache Learg osteror of a multvarate ormal he the osteror of s ormally dstrbuted e / / d CS 750 Mache Learg Other dstrbutos Gamma dstrbuto: Eoetal dstrbuto: A secal case of Gamma for a= osso dstrbuto: b a a e b a b a b e b b! e for ] 0 [ for } 0 {

29 9 Other dstrbutos Gamma dstrbuto: b a a e b a b a for ] 0 [ CS 750 Mache Learg Sequetal Bayesa arameter estmato Sequetal Bayesa aroach Uder the d the estmates of the osteror ca be comuted cremetally for a sequece of data ots If we use a cojugate ror we get bac the same osteror Assume we slt the data the last elemet ad the rest he: d Θ Θ Θ d Θ Θ Θ Θ A ew ror

30 Eoetal famly Eoetal famly: all robablty mass / desty fuctos that ca be wrtte the eoetal ormal form f h e t Z a vector of atural or caocal arameters t a fucto referred to as a suffcet statstc h a fucto of t s less mortat Z a ormalzato costat a artto fucto Z h e t d Other commo form: f h e t A log Z A CS 750 Mache Learg Eoetal famly: eamles Beroull dstrbuto e log log elog e log Eoetal famly f h e t Z arameters? t? Z? h? CS 750 Mache Learg 30

31 3 CS 750 Mache Learg Eoetal famly: eamles Beroull dstrbuto Eoetal famly arameters ote log log e log t h Z e e t h Z f log e log e e CS 750 Mache Learg Eoetal famly: eamles Uvarate Gaussa dstrbuto Eoetal famly arameters e log e?? t? h? Z e t h Z f ] e[

32 3 CS 750 Mache Learg Eoetal famly: eamles Uvarate Gaussa dstrbuto Eoetal famly arameters e log e / / t / h log 4 e log e Z e t h Z f ] e[ CS 750 Mache Learg Eoetal famly For d samles the lelhood of data s Imortat: the dmesoalty of the suffcet statstc remas the same wth the umber of samles e A t h e A t h e A t h

33 33 CS 750 Mache Learg Eoetal famly he log lelhood of data s Otmzg the loglelhood For the ML estmate t must hold e log A t h l log A t h 0 A t l t A CS 750 Mache Learg Eoetal famly Rewrttg the gradet:

34 Eoetal famly Rewrttg the gradet: A log Z log h e t d t h e t d A h e t d A t h e t A d A E t Result: E t t For the ML estmate the arameters should be adjusted such that the eectato of the statstc t s equal to the observed samle statstcs CS 750 Mache Learg Momets of the dstrbuto For the eoetal famly he -th momet of the statstc corresods to the -th dervatve of A If s a comoet of t the we get the momets of the dstrbuto by dfferetatg ts corresodg atural arameter Eamle: Beroull e log log A log log e ervatves: A e log e e e A e CS 750 Mache Learg 34

35 Ed CS 750 Mache Learg Multvarate ormal dstrbuto Multvarate ormal: ~ arameters: - mea - covarace matr esty fucto: e d / / Eamle: CS 750 Mache Learg 35

36 arameter estmates Loglelhood l log ML estmates of the mea ad covaraces: ˆ ˆ ˆ ˆ Covarace estmate s based E ˆ E ˆ Ubased estmate: ˆ ˆ CS 750 Mache Learg ˆ ˆ Learg va arameter estmato I ths lecture we cosder arametrc desty estmato Basc settgs: A set of radom varables X { X X Xd} A model of the dstrbuto over varables X wth arameters ata.. } { Objectve: fd arameters ˆ that ft the data the best What s the best set of arameters? here are varous crtera oe ca aly here CS 750 Mache Learg 36

37 37 CS 750 Mache Learg arameter estmato. Mamum lelhood ML Mamum a osteror robablty MA Bayesa framewor use a osteror desty o otmzato mamze - reresets ror bacgroud owledge mamze Selects the mode of the osteror CS 750 Mache Learg osteror of a multvarate ormal Assume that we use oly a ror o the mea: A ror he the osteror s: ormally ML estmates of the mea ad covaraces: Covarace estmate s based Ubased estmate: ˆ ˆ ˆ ˆ ˆ ˆ ˆ E E ˆ ˆ ˆ e / / d

38 Loglelhood arameter estmates l log ML estmates of the mea ad covaraces: ˆ ˆ ˆ ˆ Ubased estmate: ˆ ˆ ˆ CS 750 Mache Learg Usuervsed learg ata: {.. } a vector of attrbute values e.g. the descrto of a atet o secfc target attrbute we wat to redct o outut y Objectve: lear descrbe relatos betwee attrbutes eamles yes of roblems: Clusterg Grou together smlar eamles esty estmato Model robablstcally the oulato of eamles CS 750 Mache Learg 38

39 Beta dstrbuto =0.5 =0.5 =.5 =.5 =.5 = CS 750 Mache Learg Eoetal famly Eoetal famly of dstrbutos θ b θ f θφ e c φ a φ arameters: θ - locato arameters φ - scalg arameters Eamle: d / / e CS 750 Mache Learg 39

40 Eamle: Beroull dstrbuto Co eamle: we have a co that ca be based Outcomes: two ossble values -- head or tal ata: a sequece of outcomes such that head tal 0 Model: robablty of a head robablty of a tal Objectve: We would le to estmate the robablty of a head ˆ robablty of a outcome Beroull dstrbuto CS 750 Mache Learg 40

CS 3710 Advanced Topics in AI Lecture 17. Density estimation. CS 3710 Probabilistic graphical models. Administration

CS 3710 Advanced Topics in AI Lecture 17. Density estimation. CS 3710 Probabilistic graphical models. Administration CS 37 Avace Topcs AI Lecture 7 esty estmato Mlos Hauskrecht mlos@cs.ptt.eu 539 Seott Square CS 37 robablstc graphcal moels Amstrato Mterm: A take-home exam week ue o Weesay ovember 5 before the class epes

More information

Parameter Estimation

Parameter Estimation arameter Estmato robabltes Notatoal Coveto Mass dscrete fucto: catal letters Desty cotuous fucto: small letters Vector vs. scalar Scalar: la Vector: bold D: small Hgher dmeso: catal Notes a cotuous state

More information

Generative classification models

Generative classification models CS 75 Mache Learg Lecture Geeratve classfcato models Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Data: D { d, d,.., d} d, Classfcato represets a dscrete class value Goal: lear f : X Y Bar classfcato

More information

Learning Graphical Models

Learning Graphical Models School of omuter Scece Statstcal learg wth basc grahcal models robablstc Grahcal Models -78 Lecture 7 Oct 8 7 Recetor A Recetor B ase ase D ase E 3 4 5 Erc g Gee G TF F 6 7 Gee H 8 Readg: J-ha. 56 F-ha.

More information

Density estimation II

Density estimation II CS 750 Mche Lerg Lecture 6 esty estmto II Mlos Husrecht mlos@tt.edu 539 Seott Squre t: esty estmto {.. } vector of ttrute vlues Ojectve: estmte the model of the uderlyg rolty dstruto over vrles X X usg

More information

Bayesian belief networks

Bayesian belief networks Lecture 14 ayesa belef etworks los Hauskrecht mlos@cs.ptt.edu 5329 Seott Square Desty estmato Data: D { D1 D2.. D} D x a vector of attrbute values ttrbutes: modeled by radom varables { 1 2 d} wth: otuous

More information

2. Independence and Bernoulli Trials

2. Independence and Bernoulli Trials . Ideedece ad Beroull Trals Ideedece: Evets ad B are deedet f B B. - It s easy to show that, B deedet mles, B;, B are all deedet ars. For examle, ad so that B or B B B B B φ,.e., ad B are deedet evets.,

More information

STK3100 and STK4100 Autumn 2017

STK3100 and STK4100 Autumn 2017 SK3 ad SK4 Autum 7 Geeralzed lear models Part III Covers the followg materal from chaters 4 ad 5: Sectos 4..5, 4.3.5, 4.3.6, 4.4., 4.4., ad 4.4.3 Sectos 5.., 5.., ad 5.5. Ørulf Borga Deartmet of Mathematcs

More information

CS 2750 Machine Learning. Lecture 5. Density estimation. CS 2750 Machine Learning. Announcements

CS 2750 Machine Learning. Lecture 5. Density estimation. CS 2750 Machine Learning. Announcements CS 750 Machne Learnng Lecture 5 Densty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square CS 750 Machne Learnng Announcements Homework Due on Wednesday before the class Reports: hand n before

More information

Part I: Background on the Binomial Distribution

Part I: Background on the Binomial Distribution Part I: Bacgroud o the Bomal Dstrbuto A radom varable s sad to have a Beroull dstrbuto f t taes o the value wth probablt "p" ad the value wth probablt " - p". The umber of "successes" "" depedet Beroull

More information

Probability and Statistics. What is probability? What is statistics?

Probability and Statistics. What is probability? What is statistics? robablt ad Statstcs What s robablt? What s statstcs? robablt ad Statstcs robablt Formall defed usg a set of aoms Seeks to determe the lkelhood that a gve evet or observato or measuremet wll or has haeed

More information

CHAPTER 6. d. With success = observation greater than 10, x = # of successes = 4, and

CHAPTER 6. d. With success = observation greater than 10, x = # of successes = 4, and CHAPTR 6 Secto 6.. a. We use the samle mea, to estmate the oulato mea µ. Σ 9.80 µ 8.407 7 ~ 7. b. We use the samle meda, 7 (the mddle observato whe arraged ascedg order. c. We use the samle stadard devato,

More information

STK3100 and STK4100 Autumn 2018

STK3100 and STK4100 Autumn 2018 SK3 ad SK4 Autum 8 Geeralzed lear models Part III Covers the followg materal from chaters 4 ad 5: Cofdece tervals by vertg tests Cosder a model wth a sgle arameter β We may obta a ( α% cofdece terval for

More information

CS 2750 Machine Learning. Lecture 8. Linear regression. CS 2750 Machine Learning. Linear regression. is a linear combination of input components x

CS 2750 Machine Learning. Lecture 8. Linear regression. CS 2750 Machine Learning. Linear regression. is a linear combination of input components x CS 75 Mache Learg Lecture 8 Lear regresso Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Learg Lear regresso Fucto f : X Y s a lear combato of put compoets f + + + K d d K k - parameters

More information

Parametric Density Estimation: Bayesian Estimation. Naïve Bayes Classifier

Parametric Density Estimation: Bayesian Estimation. Naïve Bayes Classifier arametrc Dest Estmato: Baesa Estmato. Naïve Baes Classfer Baesa arameter Estmato Suose we have some dea of the rage where arameters should be Should t we formalze such ror owledge hoes that t wll lead

More information

Machine learning: Density estimation

Machine learning: Density estimation CS 70 Foundatons of AI Lecture 3 Machne learnng: ensty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square ata: ensty estmaton {.. n} x a vector of attrbute values Objectve: estmate the model of

More information

Overview. Basic concepts of Bayesian learning. Most probable model given data Coin tosses Linear regression Logistic regression

Overview. Basic concepts of Bayesian learning. Most probable model given data Coin tosses Linear regression Logistic regression Overvew Basc cocepts of Bayesa learg Most probable model gve data Co tosses Lear regresso Logstc regresso Bayesa predctos Co tosses Lear regresso 30 Recap: regresso problems Iput to learg problem: trag

More information

Unsupervised Learning and Other Neural Networks

Unsupervised Learning and Other Neural Networks CSE 53 Soft Computg NOT PART OF THE FINAL Usupervsed Learg ad Other Neural Networs Itroducto Mture Destes ad Idetfablty ML Estmates Applcato to Normal Mtures Other Neural Networs Itroducto Prevously, all

More information

ρ < 1 be five real numbers. The

ρ < 1 be five real numbers. The Lecture o BST 63: Statstcal Theory I Ku Zhag, /0/006 Revew for the prevous lecture Deftos: covarace, correlato Examples: How to calculate covarace ad correlato Theorems: propertes of correlato ad covarace

More information

Probabilistic Graphical Models

Probabilistic Graphical Models School of Comuter Scece Probablstc rahcal Models Parameter Est. fully observed Bs Erc Xg X X X X Lecture 7 February 5 04 X X 3 X X X 3 X 3 Readg: KF-cha 7 X 4 X 4 Erc Xg @ CMU 005-04 Learg rahcal Models

More information

BASIC PRINCIPLES OF STATISTICS

BASIC PRINCIPLES OF STATISTICS BASIC PRINCIPLES OF STATISTICS PROBABILITY DENSITY DISTRIBUTIONS DISCRETE VARIABLES BINOMIAL DISTRIBUTION ~ B 0 0 umber of successes trals Pr E [ ] Var[ ] ; BINOMIAL DISTRIBUTION B7 0. B30 0.3 B50 0.5

More information

Lecture 9. Some Useful Discrete Distributions. Some Useful Discrete Distributions. The observations generated by different experiments have

Lecture 9. Some Useful Discrete Distributions. Some Useful Discrete Distributions. The observations generated by different experiments have NM 7 Lecture 9 Some Useful Dscrete Dstrbutos Some Useful Dscrete Dstrbutos The observatos geerated by dfferet eermets have the same geeral tye of behavor. Cosequetly, radom varables assocated wth these

More information

Chapter 5 Properties of a Random Sample

Chapter 5 Properties of a Random Sample Lecture 6 o BST 63: Statstcal Theory I Ku Zhag, /0/008 Revew for the prevous lecture Cocepts: t-dstrbuto, F-dstrbuto Theorems: Dstrbutos of sample mea ad sample varace, relatoshp betwee sample mea ad sample

More information

THE ROYAL STATISTICAL SOCIETY 2016 EXAMINATIONS SOLUTIONS HIGHER CERTIFICATE MODULE 5

THE ROYAL STATISTICAL SOCIETY 2016 EXAMINATIONS SOLUTIONS HIGHER CERTIFICATE MODULE 5 THE ROYAL STATISTICAL SOCIETY 06 EAMINATIONS SOLUTIONS HIGHER CERTIFICATE MODULE 5 The Socety s provdg these solutos to assst cadtes preparg for the examatos 07. The solutos are teded as learg ads ad should

More information

Density estimation III.

Density estimation III. Lecure 6 esy esmao III. Mlos Hausrec mlos@cs..eu 539 Seo Square Oule Oule: esy esmao: Bomal srbuo Mulomal srbuo ormal srbuo Eoeal famly aa: esy esmao {.. } a vecor of arbue values Objecve: ry o esmae e

More information

Lecture 3 Naïve Bayes, Maximum Entropy and Text Classification COSI 134

Lecture 3 Naïve Bayes, Maximum Entropy and Text Classification COSI 134 Lecture 3 Naïve Baes, Mamum Etro ad Tet Classfcato COSI 34 Codtoal Parameterzato Two RVs: ItellgeceI ad SATS ValI = {Hgh,Low}, ValS={Hgh,Low} A ossble jot dstrbuto Ca descrbe usg cha rule as PI,S PIPS

More information

Random Variables. ECE 313 Probability with Engineering Applications Lecture 8 Professor Ravi K. Iyer University of Illinois

Random Variables. ECE 313 Probability with Engineering Applications Lecture 8 Professor Ravi K. Iyer University of Illinois Radom Varables ECE 313 Probablty wth Egeerg Alcatos Lecture 8 Professor Rav K. Iyer Uversty of Illos Iyer - Lecture 8 ECE 313 Fall 013 Today s Tocs Revew o Radom Varables Cumulatve Dstrbuto Fucto (CDF

More information

Econometric Methods. Review of Estimation

Econometric Methods. Review of Estimation Ecoometrc Methods Revew of Estmato Estmatg the populato mea Radom samplg Pot ad terval estmators Lear estmators Ubased estmators Lear Ubased Estmators (LUEs) Effcecy (mmum varace) ad Best Lear Ubased Estmators

More information

STK4011 and STK9011 Autumn 2016

STK4011 and STK9011 Autumn 2016 STK4 ad STK9 Autum 6 Pot estmato Covers (most of the followg materal from chapter 7: Secto 7.: pages 3-3 Secto 7..: pages 3-33 Secto 7..: pages 35-3 Secto 7..3: pages 34-35 Secto 7.3.: pages 33-33 Secto

More information

å 1 13 Practice Final Examination Solutions - = CS109 Dec 5, 2018

å 1 13 Practice Final Examination Solutions - = CS109 Dec 5, 2018 Chrs Pech Fal Practce CS09 Dec 5, 08 Practce Fal Examato Solutos. Aswer: 4/5 8/7. There are multle ways to obta ths aswer; here are two: The frst commo method s to sum over all ossbltes for the rak of

More information

Lecture 3 Probability review (cont d)

Lecture 3 Probability review (cont d) STATS 00: Itroducto to Statstcal Iferece Autum 06 Lecture 3 Probablty revew (cot d) 3. Jot dstrbutos If radom varables X,..., X k are depedet, the ther dstrbuto may be specfed by specfyg the dvdual dstrbuto

More information

Bayes (Naïve or not) Classifiers: Generative Approach

Bayes (Naïve or not) Classifiers: Generative Approach Logstc regresso Bayes (Naïve or ot) Classfers: Geeratve Approach What do we mea by Geeratve approach: Lear p(y), p(x y) ad the apply bayes rule to compute p(y x) for makg predctos Ths s essetally makg

More information

Point Estimation: definition of estimators

Point Estimation: definition of estimators Pot Estmato: defto of estmators Pot estmator: ay fucto W (X,..., X ) of a data sample. The exercse of pot estmato s to use partcular fuctos of the data order to estmate certa ukow populato parameters.

More information

2SLS Estimates ECON In this case, begin with the assumption that E[ i

2SLS Estimates ECON In this case, begin with the assumption that E[ i SLS Estmates ECON 3033 Bll Evas Fall 05 Two-Stage Least Squares (SLS Cosder a stadard lear bvarate regresso model y 0 x. I ths case, beg wth the assumto that E[ x] 0 whch meas that OLS estmates of wll

More information

Ordinary Least Squares Regression. Simple Regression. Algebra and Assumptions.

Ordinary Least Squares Regression. Simple Regression. Algebra and Assumptions. Ordary Least Squares egresso. Smple egresso. Algebra ad Assumptos. I ths part of the course we are gog to study a techque for aalysg the lear relatoshp betwee two varables Y ad X. We have pars of observatos

More information

Bayesian Classification. CS690L Data Mining: Classification(2) Bayesian Theorem: Basics. Bayesian Theorem. Training dataset. Naïve Bayes Classifier

Bayesian Classification. CS690L Data Mining: Classification(2) Bayesian Theorem: Basics. Bayesian Theorem. Training dataset. Naïve Bayes Classifier Baa Classfcato CS6L Data Mg: Classfcato() Referece: J. Ha ad M. Kamber, Data Mg: Cocepts ad Techques robablstc learg: Calculate explct probabltes for hypothess, amog the most practcal approaches to certa

More information

Nonparametric Density Estimation Intro

Nonparametric Density Estimation Intro Noarametrc Desty Estmato Itro Parze Wdows No-Parametrc Methods Nether robablty dstrbuto or dscrmat fucto s kow Haes qute ofte All we have s labeled data a lot s kow easer salmo bass salmo salmo Estmate

More information

Chapter 3 Sampling For Proportions and Percentages

Chapter 3 Sampling For Proportions and Percentages Chapter 3 Samplg For Proportos ad Percetages I may stuatos, the characterstc uder study o whch the observatos are collected are qualtatve ature For example, the resposes of customers may marketg surveys

More information

Module 7. Lecture 7: Statistical parameter estimation

Module 7. Lecture 7: Statistical parameter estimation Lecture 7: Statstcal parameter estmato Parameter Estmato Methods of Parameter Estmato 1) Method of Matchg Pots ) Method of Momets 3) Mamum Lkelhood method Populato Parameter Sample Parameter Ubased estmato

More information

Special Instructions / Useful Data

Special Instructions / Useful Data JAM 6 Set of all real umbers P A..d. B, p Posso Specal Istructos / Useful Data x,, :,,, x x Probablty of a evet A Idepedetly ad detcally dstrbuted Bomal dstrbuto wth parameters ad p Posso dstrbuto wth

More information

D KL (P Q) := p i ln p i q i

D KL (P Q) := p i ln p i q i Cheroff-Bouds 1 The Geeral Boud Let P 1,, m ) ad Q q 1,, q m ) be two dstrbutos o m elemets, e,, q 0, for 1,, m, ad m 1 m 1 q 1 The Kullback-Lebler dvergece or relatve etroy of P ad Q s defed as m D KL

More information

X ε ) = 0, or equivalently, lim

X ε ) = 0, or equivalently, lim Revew for the prevous lecture Cocepts: order statstcs Theorems: Dstrbutos of order statstcs Examples: How to get the dstrbuto of order statstcs Chapter 5 Propertes of a Radom Sample Secto 55 Covergece

More information

X X X E[ ] E X E X. is the ()m n where the ( i,)th. j element is the mean of the ( i,)th., then

X X X E[ ] E X E X. is the ()m n where the ( i,)th. j element is the mean of the ( i,)th., then Secto 5 Vectors of Radom Varables Whe workg wth several radom varables,,..., to arrage them vector form x, t s ofte coveet We ca the make use of matrx algebra to help us orgaze ad mapulate large umbers

More information

Machine Learning. Tutorial on Basic Probability. Lecture 2, September 15, 2006

Machine Learning. Tutorial on Basic Probability. Lecture 2, September 15, 2006 Mache Learg -7/5 7/5-78, 78, all 6 Tutoral o asc robablty Erc g f Lecture, Setember 5, 6 Readg: Cha. &, C & Cha 5,6, TM What s ths? Classcal AI ad ML research gored ths heomea The roblem a eamle: you wat

More information

Lecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model

Lecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model Lecture 7. Cofdece Itervals ad Hypothess Tests the Smple CLR Model I lecture 6 we troduced the Classcal Lear Regresso (CLR) model that s the radom expermet of whch the data Y,,, K, are the outcomes. The

More information

Chapter 5 Properties of a Random Sample

Chapter 5 Properties of a Random Sample Lecture 3 o BST 63: Statstcal Theory I Ku Zhag, /6/006 Revew for the revous lecture Cocets: radom samle, samle mea, samle varace Theorems: roertes of a radom samle, samle mea, samle varace Examles: how

More information

Dr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur

Dr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur Aalyss of Varace ad Desg of Exermets-I MODULE II LECTURE - GENERAL LINEAR HYPOTHESIS AND ANALYSIS OF VARIANCE Dr Shalabh Deartmet of Mathematcs ad Statstcs Ida Isttute of Techology Kaur Tukey s rocedure

More information

Continuous Distributions

Continuous Distributions 7//3 Cotuous Dstrbutos Radom Varables of the Cotuous Type Desty Curve Percet Desty fucto, f (x) A smooth curve that ft the dstrbuto 3 4 5 6 7 8 9 Test scores Desty Curve Percet Probablty Desty Fucto, f

More information

UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS

UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Exam: ECON430 Statstcs Date of exam: Frday, December 8, 07 Grades are gve: Jauary 4, 08 Tme for exam: 0900 am 00 oo The problem set covers 5 pages Resources allowed:

More information

( ) = ( ) ( ) Chapter 13 Asymptotic Theory and Stochastic Regressors. Stochastic regressors model

( ) = ( ) ( ) Chapter 13 Asymptotic Theory and Stochastic Regressors. Stochastic regressors model Chapter 3 Asmptotc Theor ad Stochastc Regressors The ature of eplaator varable s assumed to be o-stochastc or fed repeated samples a regresso aalss Such a assumpto s approprate for those epermets whch

More information

Chapter 4 Multiple Random Variables

Chapter 4 Multiple Random Variables Revew for the prevous lecture: Theorems ad Examples: How to obta the pmf (pdf) of U = g (, Y) ad V = g (, Y) Chapter 4 Multple Radom Varables Chapter 44 Herarchcal Models ad Mxture Dstrbutos Examples:

More information

Density estimation. Density estimations. CS 2750 Machine Learning. Lecture 5. Milos Hauskrecht 5329 Sennott Square

Density estimation. Density estimations. CS 2750 Machine Learning. Lecture 5. Milos Hauskrecht 5329 Sennott Square Lecure 5 esy esmao Mlos Hauskrec mlos@cs..edu 539 Seo Square esy esmaos ocs: esy esmao: Mamum lkelood ML Bayesa arameer esmaes M Beroull dsrbuo. Bomal dsrbuo Mulomal dsrbuo Normal dsrbuo Eoeal famly Noaramerc

More information

Classification : Logistic regression. Generative classification model.

Classification : Logistic regression. Generative classification model. CS 75 Mache Lear Lecture 8 Classfcato : Lostc reresso. Geeratve classfcato model. Mlos Hausrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Lear Bar classfcato o classes Y {} Our oal s to lear to classf

More information

ENGI 4421 Joint Probability Distributions Page Joint Probability Distributions [Navidi sections 2.5 and 2.6; Devore sections

ENGI 4421 Joint Probability Distributions Page Joint Probability Distributions [Navidi sections 2.5 and 2.6; Devore sections ENGI 441 Jot Probablty Dstrbutos Page 7-01 Jot Probablty Dstrbutos [Navd sectos.5 ad.6; Devore sectos 5.1-5.] The jot probablty mass fucto of two dscrete radom quattes, s, P ad p x y x y The margal probablty

More information

Continuous Random Variables: Conditioning, Expectation and Independence

Continuous Random Variables: Conditioning, Expectation and Independence Cotuous Radom Varables: Codtog, xectato ad Ideedece Berl Che Deartmet o Comuter cece & Iormato geerg atoal Tawa ormal Uverst Reerece: - D.. Bertsekas, J.. Tstskls, Itroducto to robablt, ectos 3.4-3.5 Codtog

More information

Bayes Estimator for Exponential Distribution with Extension of Jeffery Prior Information

Bayes Estimator for Exponential Distribution with Extension of Jeffery Prior Information Malaysa Joural of Mathematcal Sceces (): 97- (9) Bayes Estmator for Expoetal Dstrbuto wth Exteso of Jeffery Pror Iformato Hadeel Salm Al-Kutub ad Noor Akma Ibrahm Isttute for Mathematcal Research, Uverst

More information

Chapter 4 Multiple Random Variables

Chapter 4 Multiple Random Variables Revew o BST 63: Statstcal Theory I Ku Zhag, /0/008 Revew for Chapter 4-5 Notes: Although all deftos ad theorems troduced our lectures ad ths ote are mportat ad you should be famlar wth, but I put those

More information

Lecture 3. Sampling, sampling distributions, and parameter estimation

Lecture 3. Sampling, sampling distributions, and parameter estimation Lecture 3 Samplg, samplg dstrbutos, ad parameter estmato Samplg Defto Populato s defed as the collecto of all the possble observatos of terest. The collecto of observatos we take from the populato s called

More information

ON BIVARIATE GEOMETRIC DISTRIBUTION. K. Jayakumar, D.A. Mundassery 1. INTRODUCTION

ON BIVARIATE GEOMETRIC DISTRIBUTION. K. Jayakumar, D.A. Mundassery 1. INTRODUCTION STATISTICA, ao LXVII, 4, 007 O BIVARIATE GEOMETRIC DISTRIBUTIO ITRODUCTIO Probablty dstrbutos of radom sums of deedetly ad detcally dstrbuted radom varables are maly aled modelg ractcal roblems that deal

More information

Parameter, Statistic and Random Samples

Parameter, Statistic and Random Samples Parameter, Statstc ad Radom Samples A parameter s a umber that descrbes the populato. It s a fxed umber, but practce we do ot kow ts value. A statstc s a fucto of the sample data,.e., t s a quatty whose

More information

= 2. Statistic - function that doesn't depend on any of the known parameters; examples:

= 2. Statistic - function that doesn't depend on any of the known parameters; examples: of Samplg Theory amples - uemploymet househol cosumpto survey Raom sample - set of rv's... ; 's have ot strbuto [ ] f f s vector of parameters e.g. Statstc - fucto that oes't epe o ay of the ow parameters;

More information

Parametric Density Estimation: Bayesian Estimation. Naïve Bayes Classifier

Parametric Density Estimation: Bayesian Estimation. Naïve Bayes Classifier arametrc Dest Estmato: Baesa Estmato. Naïve Baes Classfer Baesa arameter Estmato Suppose we have some dea of the rage where parameters θ should be Should t we formalze such pror owledge hopes that t wll

More information

Recall MLR 5 Homskedasticity error u has the same variance given any values of the explanatory variables Var(u x1,...,xk) = 2 or E(UU ) = 2 I

Recall MLR 5 Homskedasticity error u has the same variance given any values of the explanatory variables Var(u x1,...,xk) = 2 or E(UU ) = 2 I Chapter 8 Heterosedastcty Recall MLR 5 Homsedastcty error u has the same varace gve ay values of the eplaatory varables Varu,..., = or EUU = I Suppose other GM assumptos hold but have heterosedastcty.

More information

UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS

UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Postpoed exam: ECON430 Statstcs Date of exam: Jauary 0, 0 Tme for exam: 09:00 a.m. :00 oo The problem set covers 5 pages Resources allowed: All wrtte ad prted

More information

22 Nonparametric Methods.

22 Nonparametric Methods. 22 oparametrc Methods. I parametrc models oe assumes apror that the dstrbutos have a specfc form wth oe or more ukow parameters ad oe tres to fd the best or atleast reasoably effcet procedures that aswer

More information

Simulation Output Analysis

Simulation Output Analysis Smulato Output Aalyss Summary Examples Parameter Estmato Sample Mea ad Varace Pot ad Iterval Estmato ermatg ad o-ermatg Smulato Mea Square Errors Example: Sgle Server Queueg System x(t) S 4 S 4 S 3 S 5

More information

Multivariate Transformation of Variables and Maximum Likelihood Estimation

Multivariate Transformation of Variables and Maximum Likelihood Estimation Marquette Uversty Multvarate Trasformato of Varables ad Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Assocate Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 03 by Marquette Uversty

More information

STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. x, where. = y - ˆ " 1

STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. x, where. = y - ˆ  1 STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS Recall Assumpto E(Y x) η 0 + η x (lear codtoal mea fucto) Data (x, y ), (x 2, y 2 ),, (x, y ) Least squares estmator ˆ E (Y x) ˆ " 0 + ˆ " x, where ˆ

More information

Formulas and Tables from Beginning Statistics

Formulas and Tables from Beginning Statistics Fmula ad Table from Begg Stattc Chater Cla Mdot Relatve Frequecy Chater 3 Samle Mea Poulato Mea Weghted Mea Rage Lower Lmt Uer Lmt Cla Frequecy Samle Se µ ( w) w f Mamum Data Value - Mmum Data Value Poulato

More information

Section 2 Notes. Elizabeth Stone and Charles Wang. January 15, Expectation and Conditional Expectation of a Random Variable.

Section 2 Notes. Elizabeth Stone and Charles Wang. January 15, Expectation and Conditional Expectation of a Random Variable. Secto Notes Elzabeth Stoe ad Charles Wag Jauar 5, 9 Jot, Margal, ad Codtoal Probablt Useful Rules/Propertes. P ( x) P P ( x; ) or R f (x; ) d. P ( xj ) P (x; ) P ( ) 3. P ( x; ) P ( xj ) P ( ) 4. Baes

More information

Training Sample Model: Given n observations, [[( Yi, x i the sample model can be expressed as (1) where, zero and variance σ

Training Sample Model: Given n observations, [[( Yi, x i the sample model can be expressed as (1) where, zero and variance σ Stat 74 Estmato for Geeral Lear Model Prof. Goel Broad Outle Geeral Lear Model (GLM): Trag Samle Model: Gve observatos, [[( Y, x ), x = ( x,, xr )], =,,, the samle model ca be exressed as Y = µ ( x, x,,

More information

MIMA Group. Chapter 4 Non-Parameter Estimation. School of Computer Science and Technology, Shandong University. Xin-Shun SDU

MIMA Group. Chapter 4 Non-Parameter Estimation. School of Computer Science and Technology, Shandong University. Xin-Shun SDU Grou M D L M Chater 4 No-Parameter Estmato X-Shu Xu @ SDU School of Comuter Scece ad Techology, Shadog Uversty Cotets Itroducto Parze Wdows K-Nearest-Neghbor Estmato Classfcato Techques The Nearest-Neghbor

More information

A BAYESIAN APPROACH TO SHRINKAGE ESTIMATORS

A BAYESIAN APPROACH TO SHRINKAGE ESTIMATORS A BAYESIAN APPROACH TO SHRINKAGE ESTIMATORS Fle das Neves RIZZO Crstae Alvarega GAJO Deval Jaques de SOUZA Lucas Motero CHAVES ABSTRACT: Estmators obtaed by shrkg the least squares estmator are becomg

More information

Random Variate Generation ENM 307 SIMULATION. Anadolu Üniversitesi, Endüstri Mühendisliği Bölümü. Yrd. Doç. Dr. Gürkan ÖZTÜRK.

Random Variate Generation ENM 307 SIMULATION. Anadolu Üniversitesi, Endüstri Mühendisliği Bölümü. Yrd. Doç. Dr. Gürkan ÖZTÜRK. adom Varate Geerato ENM 307 SIMULATION Aadolu Üverstes, Edüstr Mühedslğ Bölümü Yrd. Doç. Dr. Gürka ÖZTÜK 0 adom Varate Geerato adom varate geerato s about procedures for samplg from a varety of wdely-used

More information

Lecture 02: Bounding tail distributions of a random variable

Lecture 02: Bounding tail distributions of a random variable CSCI-B609: A Theorst s Toolkt, Fall 206 Aug 25 Lecture 02: Boudg tal dstrbutos of a radom varable Lecturer: Yua Zhou Scrbe: Yua Xe & Yua Zhou Let us cosder the ubased co flps aga. I.e. let the outcome

More information

Analysis of Variance with Weibull Data

Analysis of Variance with Weibull Data Aalyss of Varace wth Webull Data Lahaa Watthaacheewaul Abstract I statstcal data aalyss by aalyss of varace, the usual basc assumptos are that the model s addtve ad the errors are radomly, depedetly, ad

More information

Set Theory and Probability

Set Theory and Probability Set Theory ad Probablty Dr. Bob Baley Set Theory We beg wth a toc of dscusso. A set ca be cosdered to be ay collecto of zero or more obects or ettes covered by the toc of dscusso. Examles: hadwrtte umerals

More information

1 Solution to Problem 6.40

1 Solution to Problem 6.40 1 Soluto to Problem 6.40 (a We wll wrte T τ (X 1,...,X where the X s are..d. wth PDF f(x µ, σ 1 ( x µ σ g, σ where the locato parameter µ s ay real umber ad the scale parameter σ s > 0. Lettg Z X µ σ we

More information

CS 2750 Machine Learning Lecture 8. Linear regression. Supervised learning. a set of n examples

CS 2750 Machine Learning Lecture 8. Linear regression. Supervised learning. a set of n examples CS 75 Mache Learg Lecture 8 Lear regresso Mlos Hauskrecht los@cs.tt.eu 59 Seott Square Suervse learg Data: D { D D.. D} a set of eales D s a ut vector of sze s the esre outut gve b a teacher Obectve: lear

More information

THE ROYAL STATISTICAL SOCIETY 2010 EXAMINATIONS SOLUTIONS GRADUATE DIPLOMA MODULE 2 STATISTICAL INFERENCE

THE ROYAL STATISTICAL SOCIETY 2010 EXAMINATIONS SOLUTIONS GRADUATE DIPLOMA MODULE 2 STATISTICAL INFERENCE THE ROYAL STATISTICAL SOCIETY 00 EXAMINATIONS SOLUTIONS GRADUATE DIPLOMA MODULE STATISTICAL INFERENCE The Socety provdes these solutos to assst caddates preparg for the examatos future years ad for the

More information

Mathematics HL and Further mathematics HL Formula booklet

Mathematics HL and Further mathematics HL Formula booklet Dploma Programme Mathematcs HL ad Further mathematcs HL Formula booklet For use durg the course ad the eamatos Frst eamatos 04 Mathematcal Iteratoal Baccalaureate studes SL: Formula Orgazato booklet 0

More information

Lecture Note to Rice Chapter 8

Lecture Note to Rice Chapter 8 ECON 430 HG revsed Nov 06 Lecture Note to Rce Chapter 8 Radom matrces Let Y, =,,, m, =,,, be radom varables (r.v. s). The matrx Y Y Y Y Y Y Y Y Y Y = m m m s called a radom matrx ( wth a ot m-dmesoal dstrbuto,

More information

Chapter 3 Experimental Design Models

Chapter 3 Experimental Design Models Chater 3 Exermetal Desg Models We cosder the models whch are used desgg a exermet. The exermetal codtos, exermetal setu ad the obectve of the study essetally determe that what tye of desg s to be used

More information

Channel Models with Memory. Channel Models with Memory. Channel Models with Memory. Channel Models with Memory

Channel Models with Memory. Channel Models with Memory. Channel Models with Memory. Channel Models with Memory Chael Models wth Memory Chael Models wth Memory Hayder radha Electrcal ad Comuter Egeerg Mchga State Uversty I may ractcal etworkg scearos (cludg the Iteret ad wreless etworks), the uderlyg chaels are

More information

Chapter 4 (Part 1): Non-Parametric Classification (Sections ) Pattern Classification 4.3) Announcements

Chapter 4 (Part 1): Non-Parametric Classification (Sections ) Pattern Classification 4.3) Announcements Aoucemets No-Parametrc Desty Estmato Techques HW assged Most of ths lecture was o the blacboard. These sldes cover the same materal as preseted DHS Bometrcs CSE 90-a Lecture 7 CSE90a Fall 06 CSE90a Fall

More information

CS 2750 Machine Learning. Lecture 7. Linear regression. CS 2750 Machine Learning. Linear regression. is a linear combination of input components x

CS 2750 Machine Learning. Lecture 7. Linear regression. CS 2750 Machine Learning. Linear regression. is a linear combination of input components x CS 75 Mache Learg Lecture 7 Lear regresso Mlos Hauskrecht los@cs.ptt.edu 59 Seott Square CS 75 Mache Learg Lear regresso Fucto f : X Y s a lear cobato of put copoets f + + + K d d K k - paraeters eghts

More information

{ }{ ( )} (, ) = ( ) ( ) ( ) Chapter 14 Exercises in Sampling Theory. Exercise 1 (Simple random sampling): Solution:

{ }{ ( )} (, ) = ( ) ( ) ( ) Chapter 14 Exercises in Sampling Theory. Exercise 1 (Simple random sampling): Solution: Chapter 4 Exercses Samplg Theory Exercse (Smple radom samplg: Let there be two correlated radom varables X ad A sample of sze s draw from a populato by smple radom samplg wthout replacemet The observed

More information

ESS Line Fitting

ESS Line Fitting ESS 5 014 17. Le Fttg A very commo problem data aalyss s lookg for relatoshpetwee dfferet parameters ad fttg les or surfaces to data. The smplest example s fttg a straght le ad we wll dscuss that here

More information

Introduction to Probability

Introduction to Probability Itroducto to Probablty Nader H Bshouty Departmet of Computer Scece Techo 32000 Israel e-mal: bshouty@cstechoacl 1 Combatorcs 11 Smple Rules I Combatorcs The rule of sum says that the umber of ways to choose

More information

Entropy, Relative Entropy and Mutual Information

Entropy, Relative Entropy and Mutual Information Etro Relatve Etro ad Mutual Iformato rof. Ja-Lg Wu Deartmet of Comuter Scece ad Iformato Egeerg Natoal Tawa Uverst Defto: The Etro of a dscrete radom varable s defed b : base : 0 0 0 as bts 0 : addg terms

More information

Lecture Notes Types of economic variables

Lecture Notes Types of economic variables Lecture Notes 3 1. Types of ecoomc varables () Cotuous varable takes o a cotuum the sample space, such as all pots o a le or all real umbers Example: GDP, Polluto cocetrato, etc. () Dscrete varables fte

More information

Law of Large Numbers

Law of Large Numbers Toss a co tmes. Law of Large Numbers Suppose 0 f f th th toss came up H toss came up T s are Beroull radom varables wth p ½ ad E( ) ½. The proporto of heads s. Itutvely approaches ½ as. week 2 Markov s

More information

Qualifying Exam Statistical Theory Problem Solutions August 2005

Qualifying Exam Statistical Theory Problem Solutions August 2005 Qualfyg Exam Statstcal Theory Problem Solutos August 5. Let X, X,..., X be d uform U(,),

More information

Chapter 14 Logistic Regression Models

Chapter 14 Logistic Regression Models Chapter 4 Logstc Regresso Models I the lear regresso model X β + ε, there are two types of varables explaatory varables X, X,, X k ad study varable y These varables ca be measured o a cotuous scale as

More information

A New Family of Transformations for Lifetime Data

A New Family of Transformations for Lifetime Data Proceedgs of the World Cogress o Egeerg 4 Vol I, WCE 4, July - 4, 4, Lodo, U.K. A New Famly of Trasformatos for Lfetme Data Lakhaa Watthaacheewakul Abstract A famly of trasformatos s the oe of several

More information

The expected value of a sum of random variables,, is the sum of the expected values:

The expected value of a sum of random variables,, is the sum of the expected values: Sums of Radom Varables xpected Values ad Varaces of Sums ad Averages of Radom Varables The expected value of a sum of radom varables, say S, s the sum of the expected values: ( ) ( ) S Ths s always true

More information

LINEAR REGRESSION ANALYSIS

LINEAR REGRESSION ANALYSIS LINEAR REGRESSION ANALYSIS MODULE V Lecture - Correctg Model Iadequaces Through Trasformato ad Weghtg Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur Aalytcal methods for

More information

Randomness and uncertainty play an important

Randomness and uncertainty play an important CHAPTER 3 Probablty, Statstcs, ad Iformato Theory Radomess ad ucertaty play a mportat role scece ad egeerg. Most spoke laguage processg problems ca be characterzed a probablstc framework. Probablty theory

More information

Point Estimation: definition of estimators

Point Estimation: definition of estimators Pot Estmato: defto of estmators Pot estmator: ay fucto W (X,..., X ) of a data sample. The exercse of pot estmato s to use partcular fuctos of the data order to estmate certa ukow populato parameters.

More information

Likelihood Ratio, Wald, and Lagrange Multiplier (Score) Tests. Soccer Goals in European Premier Leagues

Likelihood Ratio, Wald, and Lagrange Multiplier (Score) Tests. Soccer Goals in European Premier Leagues Lkelhood Rato, Wald, ad Lagrage Multpler (Score) Tests Soccer Goals Europea Premer Leagues - 4 Statstcal Testg Prcples Goal: Test a Hpothess cocerg parameter value(s) a larger populato (or ature), based

More information