Learning Bayesian belief networks
|
|
|
- Kenneth Hodges
- 6 years ago
- Views:
Transcription
1 Lectue 6 Leag Bayesa belef etwoks Mlos Hauskecht mlos@cs.ptt.edu 5329 Seott Squae Admstato Mdtem: Wedesday, Mach 7, 2004 I class Closed book Mateal coveed by Spg beak, cludg ths lectue Last yea mdtem o the web No ew homewok
2 Leag pobablty dstbuto Basc settgs: A set of adom vaables = {, 2, K, } A model of the dstbuto ove vaables wth paametes Θ Data D = D, D,.., D } { 2 N Obectve: fd paametes Θˆ that descbe the data the best Leag Bayesa belef etwoks: paametezatos as defed by the stuctue of etwok Leag of BBN Leag. Leag of paametes of codtoal pobabltes Leag of the etwok stuctue Vaables: Obsevable values peset evey data sample Hdde they values ae eve obseved data Mssg values values sometmes peset, sometmes ot Next: All vaables ae obsevable. Leag of paametes of BBN 2. Leag of the model (BBN stuctue
3 Leag of paametes of BBN Idea: decompose the estmato poblem fo the full ot ove a lage umbe of vaables to a set of smalle estmato poblems coespodg to paet-vaable codtoals. Example: Assume A,E,B ae bay wth ue, False values B A 4 estmato poblems E P(A B=,E= P(A B=,E=F P(A B,E P(A B=F,E= P(A B=F,E=F Assumpto that eables the decomposto: paametes of codtoal dstbutos ae depedet Estmates of paametes of BBN wo assumptos that pemt the decomposto: Sample depedece P( D Θ, ξ = P( Θ, ξ u= Paamete depedece N D u p( Θ D, ξ = p( θ D, ξ q = = Paametes of each codtoal (oe fo evey assgmet of values to paet vaables ca be leaed depedetly
4 Leag of BBN paametes. Example. Example: P(Peumoa Peumoa F?? P(HWBC Peum P F?? F?? Paleess Feve Cough Hgh WBC P(Pale Peum P(Feve Peum P(Cough Peum??? Leag of BBN paametes. Example. Data D (dffeet patet cases: Pal Fev Cou HWB Peu F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F Paleess Feve Peumoa Cough Hgh WBC
5 Estmates of paametes of BBN Much lke multple co toss o oll of a dce poblems. A smalle leag poblem coespods to the leag of exactly oe codtoal dstbuto Example: P( Feve Peumoa = Poblem: How to pck the data to lea? Estmates of paametes of BBN Much lke multple co toss o oll of a dce poblems. A smalle leag poblem coespods to the leag of exactly oe codtoal dstbuto Example: P( Feve Peumoa = Poblem: How to pck the data to lea? Aswe:. Select data pots wth Peumoa= (goe the est 2. Focus o (select oly values of the adom vaable defg the dstbuto (Feve 3. Lea the paametes of the codtoal the same way as we leaed the paametes of the based co o dce
6 Leag of BBN paametes. Example. Lea: P( Feve Peumoa = Step : Select data pots wth Peumoa= Pal Fev Cou HWB Peu F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F F Paleess Feve Peumoa Cough Hgh WBC Leag of BBN paametes. Example. Lea: Step : P( Feve Peumoa = Igoe the est Pal Fev Cou HWB Peu F F F F F F F F Paleess Feve Peumoa Cough Hgh WBC
7 Leag of BBN paametes. Example. Lea: P( Feve Peumoa = Step 2: Select values of the adom vaable defg the dstbuto of Feve Pal Fev Cou HWB Peu F F F F F F F F Paleess Feve Peumoa Cough Hgh WBC Leag of BBN paametes. Example. Lea: P( Feve Peumoa = Step 2: Igoe the est Fev F F Paleess Feve Peumoa Cough Hgh WBC
8 Leag of BBN paametes. Example. Lea: P( Feve Peumoa = Step 3a: Leag the ML estmate Fev F F Paleess Feve Peumoa Cough Hgh WBC P( Feve Peumoa = F Leag of BBN paametes. Bayesa leag. Lea: P( Feve Peumoa = Step 3b: Leag the Bayesa estmate Assume the po Fev F F θ Feve = Peumoa ~ Beta(3,4 Paleess Feve Peumoa Cough Hgh WBC Posteo: Peumoa ~ Beta(6,6 θ Feve =
9 Naïve Bayes model A specal (smple Bayesa belef etwok used as a geeatve classfe model Class vaable Y Attbutes ae depedet gve Y p( x Y =, Θ = p( x Leag: ML, Bayesa estmates of paametes Classfcato: gve x we eed to deteme the class Choose the class wth the maxmum posteo p( Y = x, Θ = k = = Y =, Θ p( Y = Θ p( x Y =, Θ p( Y = Θ p( x Y =, Θ Class Y 2.. Naïve Bayes wth Gaussas dstbutos Geeatve classfcato model p(, Y. Pos o classes Y p ( Y =, p ( Y = 2, p ( Y = 3,... p( Y Befoe: Jot class codtoal destes (fo x p ( x µ, Σ = exp ( x µ ( / 2 / 2 Σ x µ d 2 (2π Σ p (Y Now: Naïve Bayes - depedet class codtoal destes p(y 2 p( x µ, σ = exp ( x 2 µ Y (2π σ 2σ p( Y 2..
10 Naïve Bayes wth Gaussas dstbutos How to lea the geeatve model p(, Y. Pos o classes p ( Y =, p ( Y = 2, p ( Y = 3,...? 2. Class codtoal destes 2 p( x µ, σ = exp ( x 2 µ (2π σ 2σ p( Y 2.. Y Model selecto BBN has two compoets: Stuctue of the etwok (models codtoal depedeces A set of paametes (codtoal chld-paet dstbutos We aleady kow how to lea the paametes fo the fxed stuctue But how to lea the stuctue of the BBN? Alam? Buglay Alam Quake Buglay Quake Joh May Joh May
11 Leag the stuctue Ctea we ca choose to scoe the stuctue S Magal lkelhood maxmze P ( D S, ξ ξ - epesets the po kowledge Maxmum posteo pobablty maxmze P ( S D, ξ P ( S D, ξ = P ( D S, ξ P ( S P ( D ξ ξ How to compute magal lkelhood P ( D S, ξ? Leag of BBNs Notato: ages ove all possble vaables =,.., =,..,q ages ove all possble paet combatos k=,.., ages ove all possble vaable values Θ - paametes of the BBN Θ s a vecto of Θ epesetg paametes of the codtoal pobablty dstbuto; such that Θ = N N = - a umbe of staces the dataset whee paets of vaable take o values ad has value k = N - po couts (paametes of Beta ad Dchlet pos
12 Magal lkelhood Itegate ove all possble paamete settgs Θ P ( D S, ξ = P( D S, Θ, ξ p( Θ S, ξ dθ Usg the assumpto of paamete ad sample depedece P ( D S, ξ = q = = + N We ca use log-lkelhood scoe stead log P( D S, ξ = + N q + N log + log = = + N = Scoe s decomposable alog vaables!!! k Computg the magal lkelhood Fom the d assumpto: P( D S, Θ = N h= = h P( x paets, Θ Let = umbe of values that attbute x ca take q = umbe of possble paet combatos N = umbe of cases D whee x has value k ad paets wth values. = h x = k paets = q k = q k P (, Θ θ N N
13 Fom paamete depedece Pos fo p( Θ S, ξ Θ = ( Θ,..., Θ s a vecto of paametes; we use a Dchlet dstbuto wth paametes to epeset t P Computg the magal lkelhood ( Θ S, ξ = P( Θ,..., Θ S, ξ = Dchlet( Θ,..., Θ = p( Θ S, ξ = p( Θ S, ξ = = q Θ Computg the magal lkelhood Combe thgs togethe: P ( D S = P ( D S, Θ P ( Θ S dθ Θ Γ q ( N = Θ Θ k Γ q ( = = Θ N + dθ a + N k + N q = dθ
14 Leag the stuctue Lkelhood of data fo the BBN (stuctue ad paametes P( D S, Θ, ξ measues the goodess of ft of the BBN to data Magal lkelhood (fo the stuctue oly P ( D S, ξ Does ot measue oly a goodess of ft. It s: dffeet fo stuctues of dffeet complexty Icopoates pefeeces towads smple stuctues, mplemets Occam s azo!!!! Occam s Razo Why thee s a pefeece towads smple stuctues? Rewte magal lkelhood as P( D S, ξ = We kow that Θ Θ P( D S, Θ, ξ p( Θ S, ξ dθ Θ p( Θ S, ξ dθ p( Θ S, ξ dθ = Itepetato: moe complex stuctues thee ae moe ways how paametes ca be set badly he umeato: cout of good assgmets he deomato: cout of all assgmets
15 Appoxmatos of pobablstc scoes Appoxmatos of the magal lkelhood ad posteo scoes Ifomato based measues Akake cteo Bayesa fomato cteo (BIC Mmum descpto legth (MDL Reflect the tadeoff betwee the ft to data ad pefeece towads smple stuctues Example: Akake cteo. Maxmze: scoe( S = log P( D S, ΘML, ξ compl(s Bayesa fomato cteo (BIC Maxmze: scoe( S = log P( D S, ΘML, ξ compl(s logn 2 Optmzg the stuctue Fdg the best stuctue s a combatoal optmzato poblem A good featue: the scoe s decomposable alog vaables: q Γ Γ + = ( + ( N log P ( D S, ξ log log = = Γ ( + N Γ ( Algothm dea: Seach the space of stuctues usg local chages (addtos ad deletos of a lk Advatage: we do ot have to compute the whole scoe fom scatch Recompute the patal scoe fo the affected vaable
16 Optmzg the stuctue. Algothms Geedy seach Stat fom stuctue wth o lks Add a lk that yelds the best scoe mpovemet Metopols algothm (wth smulated aealg Local addtos ad deletos Avods beg tapped local optmal
Learning the structure of Bayesian belief networks
Lectue 17 Leanng the stuctue of Bayesan belef netwoks Mlos Hauskecht mlos@cs.ptt.edu 5329 Sennott Squae Leanng of BBN Leanng. Leanng of paametes of condtonal pobabltes Leanng of the netwok stuctue Vaables:
Bayesian belief networks
Lecture 14 ayesa belef etworks los Hauskrecht mlos@cs.ptt.edu 5329 Seott Square Desty estmato Data: D { D1 D2.. D} D x a vector of attrbute values ttrbutes: modeled by radom varables { 1 2 d} wth: otuous
( m is the length of columns of A ) spanned by the columns of A : . Select those columns of B that contain a pivot; say those are Bi
Assgmet /MATH 47/Wte Due: Thusday Jauay The poblems to solve ae umbeed [] to [] below Fst some explaatoy otes Fdg a bass of the colum-space of a max ad povg that the colum ak (dmeso of the colum space)
RECAPITULATION & CONDITIONAL PROBABILITY. Number of favourable events n E Total number of elementary events n S
Fomulae Fo u Pobablty By OP Gupta [Ida Awad We, +91-9650 350 480] Impotat Tems, Deftos & Fomulae 01 Bascs Of Pobablty: Let S ad E be the sample space ad a evet a expemet espectvely Numbe of favouable evets
Learning Bayesian belief networks
Lecture 4 Learning Bayesian belief networks Milos Hauskrecht milos@cs.pitt.edu 5329 Sennott Square Administration Midterm: Monday, March 7, 2003 In class Closed book Material covered by Wednesday, March
Generative classification models
CS 75 Mache Learg Lecture Geeratve classfcato models Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Data: D { d, d,.., d} d, Classfcato represets a dscrete class value Goal: lear f : X Y Bar classfcato
= y and Normed Linear Spaces
304-50 LINER SYSTEMS Lectue 8: Solutos to = ad Nomed Lea Spaces 73 Fdg N To fd N, we eed to chaacteze all solutos to = 0 Recall that ow opeatos peseve N, so that = 0 = 0 We ca solve = 0 ecusvel backwads
Bayesian belief networks
Lecture 19 ayesa belef etworks los Hauskrecht mlos@cs.ptt.edu 539 Seott Square Varous ferece tasks: robablstc ferece Dagostc task. from effect to cause eumoa Fever redcto task. from cause to effect Fever
Professor Wei Zhu. 1. Sampling from the Normal Population
AMS570 Pofesso We Zhu. Samplg fom the Nomal Populato *Example: We wsh to estmate the dstbuto of heghts of adult US male. It s beleved that the heght of adult US male follows a omal dstbuto N(, ) Def. Smple
Bayes (Naïve or not) Classifiers: Generative Approach
Logstc regresso Bayes (Naïve or ot) Classfers: Geeratve Approach What do we mea by Geeratve approach: Lear p(y), p(x y) ad the apply bayes rule to compute p(y x) for makg predctos Ths s essetally makg
Lecture 10: Condensed matter systems
Lectue 0: Codesed matte systems Itoducg matte ts codesed state.! Ams: " Idstgushable patcles ad the quatum atue of matte: # Cosequeces # Revew of deal gas etopy # Femos ad Bosos " Quatum statstcs. # Occupato
Machine learning: Density estimation
CS 70 Foundatons of AI Lecture 3 Machne learnng: ensty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square ata: ensty estmaton {.. n} x a vector of attrbute values Objectve: estmate the model of
Atomic units The atomic units have been chosen such that the fundamental electron properties are all equal to one atomic unit.
tomc uts The atomc uts have bee chose such that the fudametal electo popetes ae all equal to oe atomc ut. m e, e, h/, a o, ad the potetal eegy the hydoge atom e /a o. D3.33564 0-30 Cm The use of atomc
XII. Addition of many identical spins
XII. Addto of may detcal sps XII.. ymmetc goup ymmetc goup s the goup of all possble pemutatos of obects. I total! elemets cludg detty opeato. Each pemutato s a poduct of a ceta fte umbe of pawse taspostos.
The Exponentiated Lomax Distribution: Different Estimation Methods
Ameca Joual of Appled Mathematcs ad Statstcs 4 Vol. No. 6 364-368 Avalable ole at http://pubs.scepub.com/ajams//6/ Scece ad Educato Publshg DOI:.69/ajams--6- The Expoetated Lomax Dstbuto: Dffeet Estmato
Minimum Hyper-Wiener Index of Molecular Graph and Some Results on Szeged Related Index
Joual of Multdscplay Egeeg Scece ad Techology (JMEST) ISSN: 359-0040 Vol Issue, Febuay - 05 Mmum Hype-Wee Idex of Molecula Gaph ad Some Results o eged Related Idex We Gao School of Ifomato Scece ad Techology,
A DATA DRIVEN PARAMETER ESTIMATION FOR THE THREE- PARAMETER WEIBULL POPULATION FROM CENSORED SAMPLES
Mathematcal ad Computatoal Applcatos, Vol. 3, No., pp. 9-36 008. Assocato fo Scetfc Reseach A DATA DRIVEN PARAMETER ESTIMATION FOR THE THREE- PARAMETER WEIBULL POPULATION FROM CENSORED SAMPLES Ahmed M.
Bayesian Classification. CS690L Data Mining: Classification(2) Bayesian Theorem: Basics. Bayesian Theorem. Training dataset. Naïve Bayes Classifier
Baa Classfcato CS6L Data Mg: Classfcato() Referece: J. Ha ad M. Kamber, Data Mg: Cocepts ad Techques robablstc learg: Calculate explct probabltes for hypothess, amog the most practcal approaches to certa
Point Estimation: definition of estimators
Pot Estmato: defto of estmators Pot estmator: ay fucto W (X,..., X ) of a data sample. The exercse of pot estmato s to use partcular fuctos of the data order to estmate certa ukow populato parameters.
Fairing of Parametric Quintic Splines
ISSN 46-69 Eglad UK Joual of Ifomato ad omputg Scece Vol No 6 pp -8 Fag of Paametc Qutc Sples Yuau Wag Shagha Isttute of Spots Shagha 48 ha School of Mathematcal Scece Fuda Uvesty Shagha 4 ha { P t )}
Naïve Bayes MIT Course Notes Cynthia Rudin
Thaks to Şeyda Ertek Credt: Ng, Mtchell Naïve Bayes MIT 5.097 Course Notes Cytha Rud The Naïve Bayes algorthm comes from a geeratve model. There s a mportat dstcto betwee geeratve ad dscrmatve models.
Lecture 12: Spiral: Domain Specific HLS. Housekeeping
8 643 ectue : Spal: Doma Specfc HS James C. Hoe Depatmet of ECE Caege Mello Uvesty 8 643 F7 S, James C. Hoe, CMU/ECE/CACM, 7 Houseeepg You goal today: see a eample of eally hghlevel sythess (ths lectue
CS 2750 Machine Learning. Lecture 8. Linear regression. CS 2750 Machine Learning. Linear regression. is a linear combination of input components x
CS 75 Mache Learg Lecture 8 Lear regresso Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Learg Lear regresso Fucto f : X Y s a lear combato of put compoets f + + + K d d K k - parameters
Detection and Estimation Theory
ESE 54 Detecto ad Etmato Theoy Joeph A. O Sullva Samuel C. Sach Pofeo Electoc Sytem ad Sgal Reeach Laboatoy Electcal ad Sytem Egeeg Wahgto Uvety Ubaue Hall 34-935-473 (Lyda awe) jao@wutl.edu J. A. O'S.
CE 561 Lecture Notes. Optimal Timing of Investment. Set 3. Case A- C is const. cost in 1 st yr, benefits start at the end of 1 st yr
CE 56 Letue otes Set 3 Optmal Tmg of Ivestmet Case A- C s ost. ost st y, beefts stat at the ed of st y C b b b3 0 3 Case B- Cost. s postpoed by oe yea C b b3 0 3 (B-A C s saved st yea C C, b b 0 3 Savg
2.1.1 The Art of Estimation Examples of Estimators Properties of Estimators Deriving Estimators Interval Estimators
. ploatoy Statstcs. Itoducto to stmato.. The At of stmato.. amples of stmatos..3 Popetes of stmatos..4 Devg stmatos..5 Iteval stmatos . Itoducto to stmato Samplg - The samplg eecse ca be epeseted by a
Lecture 9 Multiple Class Models
Lectue 9 Multple Class Models Multclass MVA Appoxmate MVA 8.4.2002 Copyght Teemu Keola 2002 1 Aval Theoem fo Multple Classes Wth jobs the system, a job class avg to ay seve sees the seve as equlbum wth
Classification : Logistic regression. Generative classification model.
CS 75 Mache Lear Lecture 8 Classfcato : Lostc reresso. Geeratve classfcato model. Mlos Hausrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Lear Bar classfcato o classes Y {} Our oal s to lear to classf
Supervised learning: Linear regression Logistic regression
CS 57 Itroducto to AI Lecture 4 Supervsed learg: Lear regresso Logstc regresso Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 57 Itro to AI Data: D { D D.. D D Supervsed learg d a set of eamples s
Chapter 7 Varying Probability Sampling
Chapte 7 Vayg Pobablty Samplg The smple adom samplg scheme povdes a adom sample whee evey ut the populato has equal pobablty of selecto. Ude ceta ccumstaces, moe effcet estmatos ae obtaed by assgg uequal
Multivariate Transformation of Variables and Maximum Likelihood Estimation
Marquette Uversty Multvarate Trasformato of Varables ad Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Assocate Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 03 by Marquette Uversty
χ be any function of X and Y then
We have show that whe we ae gve Y g(), the [ ] [ g() ] g() f () Y o all g ()() f d fo dscete case Ths ca be eteded to clude fuctos of ay ube of ado vaables. Fo eaple, suppose ad Y ae.v. wth jot desty fucto,
CS 2750 Machine Learning. Lecture 5. Density estimation. CS 2750 Machine Learning. Announcements
CS 750 Machne Learnng Lecture 5 Densty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square CS 750 Machne Learnng Announcements Homework Due on Wednesday before the class Reports: hand n before
CS 3710 Advanced Topics in AI Lecture 17. Density estimation. CS 3710 Probabilistic graphical models. Administration
CS 37 Avace Topcs AI Lecture 7 esty estmato Mlos Hauskrecht mlos@cs.ptt.eu 539 Seott Square CS 37 robablstc graphcal moels Amstrato Mterm: A take-home exam week ue o Weesay ovember 5 before the class epes
MATH Midterm Solutions
MATH 2113 - Midtem Solutios Febuay 18 1. A bag of mables cotais 4 which ae ed, 4 which ae blue ad 4 which ae gee. a How may mables must be chose fom the bag to guaatee that thee ae the same colou? We ca
Question 1. Typical Cellular System. Some geometry TELE4353. About cellular system. About cellular system (2)
TELE4353 Moble a atellte Commucato ystems Tutoal 1 (week 3-4 4 Questo 1 ove that fo a hexagoal geomety, the co-chael euse ato s gve by: Q (3 N Whee N + j + j 1/ 1 Typcal Cellula ystem j cells up cells
Maximum Likelihood Estimation
Marquette Uverst Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Coprght 08 b Marquette Uverst Maxmum Lkelhood Estmato We have bee sag that ~
Introduction to local (nonparametric) density estimation. methods
Itroducto to local (oparametrc) desty estmato methods A slecture by Yu Lu for ECE 66 Sprg 014 1. Itroducto Ths slecture troduces two local desty estmato methods whch are Parze desty estmato ad k-earest
FUZZY MULTINOMIAL CONTROL CHART WITH VARIABLE SAMPLE SIZE
A. Paduaga et al. / Iteatoal Joual of Egeeg Scece ad Techology (IJEST) FUZZY MUTINOMIA CONTRO CHART WITH VARIABE SAMPE SIZE A. PANDURANGAN Pofesso ad Head Depatmet of Compute Applcatos Vallamma Egeeg College,
STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. x, where. = y - ˆ " 1
STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS Recall Assumpto E(Y x) η 0 + η x (lear codtoal mea fucto) Data (x, y ), (x 2, y 2 ),, (x, y ) Least squares estmator ˆ E (Y x) ˆ " 0 + ˆ " x, where ˆ
Bayesian Nonlinear Regression Models based on Slash Skew-t Distribution
Euopea Ole Joual of Natual ad Socal Sceces 05; www.euopea-scece.com Vol.4, No. Specal Issue o New Dmesos Ecoomcs, Accoutg ad Maagemet ISSN 805-360 Bayesa Nolea Regesso Models based o Slash Skew-t Dstbuto
CS 1675 Introduction to Machine Learning Lecture 12 Support vector machines
CS 675 Itroducto to Mache Learg Lecture Support vector maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Mdterm eam October 9, 7 I-class eam Closed book Stud materal: Lecture otes Correspodg chapters
Motion and Flow II. Structure from Motion. Passive Navigation and Structure from Motion. rot
Moto ad Flow II Sce fom Moto Passve Navgato ad Sce fom Moto = + t, w F = zˆ t ( zˆ ( ([ ] =? hesystemmoveswth a gd moto wth aslat oal velocty t = ( U, V, W ad atoalvelocty w = ( α, β, γ. Scee pots R =
Best Linear Unbiased Estimators of the Three Parameter Gamma Distribution using doubly Type-II censoring
Best Lea Ubased Estmatos of the hee Paamete Gamma Dstbuto usg doubly ype-ii cesog Amal S. Hassa Salwa Abd El-Aty Abstact Recetly ode statstcs ad the momets have assumed cosdeable teest may applcatos volvg
ρ < 1 be five real numbers. The
Lecture o BST 63: Statstcal Theory I Ku Zhag, /0/006 Revew for the prevous lecture Deftos: covarace, correlato Examples: How to calculate covarace ad correlato Theorems: propertes of correlato ad covarace
Allocations for Heterogenous Distributed Storage
Allocatos fo Heteogeous Dstbuted Stoage Vasleos Ntaos taos@uscedu Guseppe Cae cae@uscedu Alexados G Dmaks dmaks@uscedu axv:0596v [csi] 8 Feb 0 Abstact We study the poblem of stog a data object a set of
Support vector machines
CS 75 Mache Learg Lecture Support vector maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Learg Outle Outle: Algorthms for lear decso boudary Support vector maches Mamum marg hyperplae.
Module Title: Business Mathematics and Statistics 2
CORK INSTITUTE OF TECHNOLOGY INSTITIÚID TEICNEOLAÍOCHTA CHORCAÍ Semeste Eamatos 009/00 Module Ttle: Busess Mathematcs ad Statstcs Module Code: STAT 6003 School: School of Busess ogamme Ttle: Bachelo of
Chapter 4 (Part 1): Non-Parametric Classification (Sections ) Pattern Classification 4.3) Announcements
Aoucemets No-Parametrc Desty Estmato Techques HW assged Most of ths lecture was o the blacboard. These sldes cover the same materal as preseted DHS Bometrcs CSE 90-a Lecture 7 CSE90a Fall 06 CSE90a Fall
Bayes Decision Theory - II
Bayes Decso Theory - II Ke Kreutz-Delgado (Nuo Vascocelos) ECE 175 Wter 2012 - UCSD Nearest Neghbor Classfer We are cosderg supervsed classfcato Nearest Neghbor (NN) Classfer A trag set D = {(x 1,y 1 ),,
Exponential Generating Functions - J. T. Butler
Epoetal Geeatg Fuctos - J. T. Butle Epoetal Geeatg Fuctos Geeatg fuctos fo pemutatos. Defto: a +a +a 2 2 + a + s the oday geeatg fucto fo the sequece of teges (a, a, a 2, a, ). Ep. Ge. Fuc.- J. T. Butle
Model Fitting, RANSAC. Jana Kosecka
Model Fttg, RANSAC Jaa Kosecka Fttg: Issues Prevous strateges Le detecto Hough trasform Smple parametrc model, two parameters m, b m + b Votg strateg Hard to geeralze to hgher dmesos a o + a + a 2 2 +
Unsupervised Learning and Other Neural Networks
CSE 53 Soft Computg NOT PART OF THE FINAL Usupervsed Learg ad Other Neural Networs Itroducto Mture Destes ad Idetfablty ML Estmates Applcato to Normal Mtures Other Neural Networs Itroducto Prevously, all
The Linear Probability Density Function of Continuous Random Variables in the Real Number Field and Its Existence Proof
MATEC Web of Cofeeces ICIEA 06 600 (06) DOI: 0.05/mateccof/0668600 The ea Pobablty Desty Fucto of Cotuous Radom Vaables the Real Numbe Feld ad Its Estece Poof Yya Che ad Ye Collee of Softwae, Taj Uvesty,
1. A real number x is represented approximately by , and we are told that the relative error is 0.1 %. What is x? Note: There are two answers.
PROBLEMS A real umber s represeted appromately by 63, ad we are told that the relatve error s % What s? Note: There are two aswers Ht : Recall that % relatve error s What s the relatve error volved roudg
CS 2750 Machine Learning Lecture 5. Density estimation. Density estimation
CS 750 Mache Learg Lecture 5 esty estmato Mlos Hausrecht mlos@tt.edu 539 Seott Square esty estmato esty estmato: s a usuervsed learg roblem Goal: Lear a model that rereset the relatos amog attrbutes the
Detection and Estimation Theory
ESE 54 Detecton and Etmaton Theoy Joeph A. O Sullvan Samuel C. Sach Pofeo Electonc Sytem and Sgnal Reeach Laboatoy Electcal and Sytem Engneeng Wahngton Unvety 411 Jolley Hall 314-935-4173 (Lnda anwe) jao@wutl.edu
VECTOR MECHANICS FOR ENGINEERS: Vector Mechanics for Engineers: Dynamics. In the current chapter, you will study the motion of systems of particles.
Seeth Edto CHPTER 4 VECTOR MECHNICS FOR ENINEERS: DYNMICS Fedad P. ee E. Russell Johsto, J. Systems of Patcles Lectue Notes: J. Walt Ole Texas Tech Uesty 003 The Mcaw-Hll Compaes, Ic. ll ghts eseed. Seeth
Recent Advances in Computers, Communications, Applied Social Science and Mathematics
Recet Advaces Computes, Commucatos, Appled ocal cece ad athematcs Coutg Roots of Radom Polyomal Equatos mall Itevals EFRAI HERIG epatmet of Compute cece ad athematcs Ael Uvesty Cete of amaa cece Pa,Ael,4487
such that for 1 From the definition of the k-fibonacci numbers, the firsts of them are presented in Table 1. Table 1: First k-fibonacci numbers F 1
Scholas Joual of Egeeg ad Techology (SJET) Sch. J. Eg. Tech. 0; (C):669-67 Scholas Academc ad Scetfc Publshe (A Iteatoal Publshe fo Academc ad Scetfc Resouces) www.saspublshe.com ISSN -X (Ole) ISSN 7-9
Feature Selection: Part 2. 1 Greedy Algorithms (continued from the last lecture)
CSE 546: Mache Learg Lecture 6 Feature Selecto: Part 2 Istructor: Sham Kakade Greedy Algorthms (cotued from the last lecture) There are varety of greedy algorthms ad umerous amg covetos for these algorthms.
Trace of Positive Integer Power of Adjacency Matrix
Global Joual of Pue ad Appled Mathematcs. IN 097-78 Volume, Numbe 07), pp. 079-087 Reseach Ida Publcatos http://www.publcato.com Tace of Postve Itege Powe of Adacecy Matx Jagdsh Kuma Pahade * ad Mao Jha
ECON 482 / WH Hong The Simple Regression Model 1. Definition of the Simple Regression Model
ECON 48 / WH Hog The Smple Regresso Model. Defto of the Smple Regresso Model Smple Regresso Model Expla varable y terms of varable x y = β + β x+ u y : depedet varable, explaed varable, respose varable,
THREE-PARAMETRIC LOGNORMAL DISTRIBUTION AND ESTIMATING ITS PARAMETERS USING THE METHOD OF L-MOMENTS
RELIK ; Paha 5. a 6.. THREE-PARAMETRIC LOGNORMAL DISTRIBUTION AND ESTIMATING ITS PARAMETERS USING THE METHOD OF L-MOMENTS Daa Bílová Abstact Commo statstcal methodology fo descpto of the statstcal samples
Ordinary Least Squares Regression. Simple Regression. Algebra and Assumptions.
Ordary Least Squares egresso. Smple egresso. Algebra ad Assumptos. I ths part of the course we are gog to study a techque for aalysg the lear relatoshp betwee two varables Y ad X. We have pars of observatos
Likelihood Ratio, Wald, and Lagrange Multiplier (Score) Tests. Soccer Goals in European Premier Leagues
Lkelhood Rato, Wald, ad Lagrage Multpler (Score) Tests Soccer Goals Europea Premer Leagues - 4 Statstcal Testg Prcples Goal: Test a Hpothess cocerg parameter value(s) a larger populato (or ature), based
Estimation of Parameters of the Exponential Geometric Distribution with Presence of Outliers Generated from Uniform Distribution
ustala Joual of Basc ad ppled Sceces, 6(: 98-6, ISSN 99-878 Estmato of Paametes of the Epoetal Geometc Dstbuto wth Pesece of Outles Geeated fom Ufom Dstbuto Pavz Nas, l Shadoh ad Hassa Paza Depatmet of
Support vector machines II
CS 75 Mache Learg Lecture Support vector maches II Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Learl separable classes Learl separable classes: here s a hperplae that separates trag staces th o error
MATH /19: problems for supervision in week 08 SOLUTIONS
MATH10101 2018/19: poblems fo supevisio i week 08 Q1. Let A be a set. SOLUTIONS (i Pove that the fuctio c: P(A P(A, defied by c(x A \ X, is bijective. (ii Let ow A be fiite, A. Use (i to show that fo each
STK4011 and STK9011 Autumn 2016
STK4 ad STK9 Autum 6 Pot estmato Covers (most of the followg materal from chapter 7: Secto 7.: pages 3-3 Secto 7..: pages 3-33 Secto 7..: pages 35-3 Secto 7..3: pages 34-35 Secto 7.3.: pages 33-33 Secto
By the end of this section you will be able to prove the Chinese Remainder Theorem apply this theorem to solve simultaneous linear congruences
Chapte : Theoy of Modula Aithmetic 8 Sectio D Chiese Remaide Theoem By the ed of this sectio you will be able to pove the Chiese Remaide Theoem apply this theoem to solve simultaeous liea cogueces The
Lecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model
Lecture 7. Cofdece Itervals ad Hypothess Tests the Smple CLR Model I lecture 6 we troduced the Classcal Lear Regresso (CLR) model that s the radom expermet of whch the data Y,,, K, are the outcomes. The
Dimensionality Reduction and Learning
CMSC 35900 (Sprg 009) Large Scale Learg Lecture: 3 Dmesoalty Reducto ad Learg Istructors: Sham Kakade ad Greg Shakharovch L Supervsed Methods ad Dmesoalty Reducto The theme of these two lectures s that
ENGI 4421 Propagation of Error Page 8-01
ENGI 441 Propagato of Error Page 8-01 Propagato of Error [Navd Chapter 3; ot Devore] Ay realstc measuremet procedure cotas error. Ay calculatos based o that measuremet wll therefore also cota a error.
ENGI 4421 Joint Probability Distributions Page Joint Probability Distributions [Navidi sections 2.5 and 2.6; Devore sections
ENGI 441 Jot Probablty Dstrbutos Page 7-01 Jot Probablty Dstrbutos [Navd sectos.5 ad.6; Devore sectos 5.1-5.] The jot probablty mass fucto of two dscrete radom quattes, s, P ad p x y x y The margal probablty
Binary classification: Support Vector Machines
CS 57 Itroducto to AI Lecture 6 Bar classfcato: Support Vector Maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 57 Itro to AI Supervsed learg Data: D { D, D,.., D} a set of eamples D, (,,,,,
Investigation of Partially Conditional RP Model with Response Error. Ed Stanek
Partally Codtoal Radom Permutato Model 7- vestgato of Partally Codtoal RP Model wth Respose Error TRODUCTO Ed Staek We explore the predctor that wll result a smple radom sample wth respose error whe a
Point Estimation: definition of estimators
Pot Estmato: defto of estmators Pot estmator: ay fucto W (X,..., X ) of a data sample. The exercse of pot estmato s to use partcular fuctos of the data order to estmate certa ukow populato parameters.
3. Basic Concepts: Consequences and Properties
: 3. Basc Cocepts: Cosequeces ad Propertes Markku Jutt Overvew More advaced cosequeces ad propertes of the basc cocepts troduced the prevous lecture are derved. Source The materal s maly based o Sectos.6.8
Chapter 4 Multiple Random Variables
Revew for the prevous lecture: Theorems ad Examples: How to obta the pmf (pdf) of U = g (, Y) ad V = g (, Y) Chapter 4 Multple Radom Varables Chapter 44 Herarchcal Models ad Mxture Dstrbutos Examples:
8.1 Hashing Algorithms
CS787: Advaced Algorthms Scrbe: Mayak Maheshwar, Chrs Hrchs Lecturer: Shuch Chawla Topc: Hashg ad NP-Completeess Date: September 21 2007 Prevously we looked at applcatos of radomzed algorthms, ad bega
Consumer theory. A. The preference ordering B. The feasible set C. The consumption decision. A. The preference ordering. Consumption bundle
Thomas Soesso Mcoecoomcs Lecte Cosme theoy A. The efeece odeg B. The feasble set C. The cosmto decso A. The efeece odeg Cosmto bdle x ( 2 x, x,... x ) x Assmtos: Comleteess 2 Tastvty 3 Reflexvty 4 No-satato
6.867 Machine Learning
6.867 Mache Learg Problem set Due Frday, September 9, rectato Please address all questos ad commets about ths problem set to 6.867-staff@a.mt.edu. You do ot eed to use MATLAB for ths problem set though
Chapter 14 Logistic Regression Models
Chapter 4 Logstc Regresso Models I the lear regresso model X β + ε, there are two types of varables explaatory varables X, X,, X k ad study varable y These varables ca be measured o a cotuous scale as
Dimensionality reduction Feature selection
CS 750 Mache Learg Lecture 3 Dmesoalty reducto Feature selecto Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 750 Mache Learg Dmesoalty reducto. Motvato. Classfcato problem eample: We have a put data
QR Factorization and Singular Value Decomposition COS 323
QR Factorzato ad Sgular Value Decomposto COS 33 Why Yet Aother Method? How do we solve least-squares wthout currg codto-squarg effect of ormal equatos (A T A A T b) whe A s sgular, fat, or otherwse poorly-specfed?
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Exam: ECON430 Statstcs Date of exam: Frday, December 8, 07 Grades are gve: Jauary 4, 08 Tme for exam: 0900 am 00 oo The problem set covers 5 pages Resources allowed:
X X X E[ ] E X E X. is the ()m n where the ( i,)th. j element is the mean of the ( i,)th., then
![X X X E[ ] E X E X. is the ()m n where the ( i,)th. j element is the mean of the ( i,)th., then](/thumbs/83/87696552.jpg "X X X E[ ] E X E X. is the ()m n where the ( i,)th. j element is the mean of the ( i,)th., then")
Secto 5 Vectors of Radom Varables Whe workg wth several radom varables,,..., to arrage them vector form x, t s ofte coveet We ca the make use of matrx algebra to help us orgaze ad mapulate large umbers
This may involve sweep, revolution, deformation, expansion and forming joints with other curves.
5--8 Shapes ae ceated by cves that a sface sch as ooftop of a ca o fselage of a acaft ca be ceated by the moto of cves space a specfed mae. Ths may volve sweep, evolto, defomato, expaso ad fomg jots wth
Lecture 3. Sampling, sampling distributions, and parameter estimation
Lecture 3 Samplg, samplg dstrbutos, ad parameter estmato Samplg Defto Populato s defed as the collecto of all the possble observatos of terest. The collecto of observatos we take from the populato s called
TESTS BASED ON MAXIMUM LIKELIHOOD
ESE 5 Toy E. Smth. The Basc Example. TESTS BASED ON MAXIMUM LIKELIHOOD To llustrate the propertes of maxmum lkelhood estmates ad tests, we cosder the smplest possble case of estmatg the mea of the ormal
ˆ SSE SSE q SST R SST R q R R q R R q
Bll Evas Spg 06 Sggested Aswes, Poblem Set 5 ECON 3033. a) The R meases the facto of the vaato Y eplaed by the model. I ths case, R =SSM/SST. Yo ae gve that SSM = 3.059 bt ot SST. Howeve, ote that SST=SSM+SSE
Objectives. Learning Outcome. 7.1 Centre of Gravity (C.G.) 7. Statics. Determine the C.G of a lamina (Experimental method)
Ojectves 7 Statcs 7. Cete of Gavty 7. Equlum of patcles 7.3 Equlum of g oes y Lew Sau oh Leag Outcome (a) efe cete of gavty () state the coto whch the cete of mass s the cete of gavty (c) state the coto
18.413: Error Correcting Codes Lab March 2, Lecture 8
18.413: Error Correctg Codes Lab March 2, 2004 Lecturer: Dael A. Spelma Lecture 8 8.1 Vector Spaces A set C {0, 1} s a vector space f for x all C ad y C, x + y C, where we take addto to be compoet wse
1. Overview of basic probability
13.42 Desg Prcples for Ocea Vehcles Prof. A.H. Techet Sprg 2005 1. Overvew of basc probablty Emprcally, probablty ca be defed as the umber of favorable outcomes dvded by the total umber of outcomes, other
9 U-STATISTICS. Eh =(m!) 1 Eh(X (1),..., X (m ) ) i.i.d
9 U-STATISTICS Suppose,,..., are P P..d. wth CDF F. Our goal s to estmate the expectato t (P)=Eh(,,..., m ). Note that ths expectato requres more tha oe cotrast to E, E, or Eh( ). Oe example s E or P((,
Bayes method for low rank tensor estimation
Joual of Physcs: Cofeece Sees PAPER OPEN ACCESS Bayes method fo low ak teso estmato To cte ths atcle: Taj Suzuk ad Hesho Kaagawa 2016 J. Phys.: Cof. Se. 699 012020 Vew the atcle ole fo updates ad ehacemets.
1 Solution to Problem 6.40
1 Soluto to Problem 6.40 (a We wll wrte T τ (X 1,...,X where the X s are..d. wth PDF f(x µ, σ 1 ( x µ σ g, σ where the locato parameter µ s ay real umber ad the scale parameter σ s > 0. Lettg Z X µ σ we
Suppose we have observed values t 1, t 2, t n of a random variable T.
Sppose we have obseved vales, 2, of a adom vaable T. The dsbo of T s ow o belog o a cea ype (e.g., expoeal, omal, ec.) b he veco θ ( θ, θ2, θp ) of ow paamees assocaed wh s ow (whee p s he mbe of ow paamees).
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Postpoed exam: ECON430 Statstcs Date of exam: Jauary 0, 0 Tme for exam: 09:00 a.m. :00 oo The problem set covers 5 pages Resources allowed: All wrtte ad prted