Soft k-means Clustering. Comp 135 Machine Learning Computer Science Tufts University. Mixture Models. Mixture of Normals in 1D
|
|
- Helena Ward
- 6 years ago
- Views:
Transcription
1 Comp 35 Machn Larnng Computr Scnc Tufts Unvrsty Fall 207 Ron Khardon Th EM Algorthm Mxtur Modls Sm-Suprvsd Larnng Soft k-mans Clustrng ck k clustr cntrs : Assocat xampls wth cntrs p,j ~~ smlarty b/w cntr and x j R-calculat mans as wghtd avrag of xampls n clustr Untl convrgnc Mxtur Modls Motvatd by soft k-mans W dvlop a gnratv modl for clustrng: Assum thr ar k clustrs Clustrs ar not rqurd to hav th sam numbr of ponts And not rqurd to hav th sam shap Mxtur of Normals n D Rpat for =,...,N ck clustr Id z from dscrt dstrbuton wth paramtrs p,p 2,...,p k Not: z 2 {, 2,...,k} ck th xampl x from normal dstrbuton wth paramtrs µ z, z Exampl: whn z =3usngµ 3 and 3 Gvn a datast gnratd by ths procss th clustrng task s to dntfy th paramtrs {p j,µ j, j} j =,...,k Mxtur of Normals n D Maxmum lklhood stmaton Rpat for =,...,N ck clustr Id z from dscrt dstrbuton wth paramtrs p,p 2,...,p k Not: z 2 {, 2,...,k} ck th xampl x from normal dstrbuton wth paramtrs µ z, z Exampl: whn z =3usngµ 3 and 3 To smplfy analyss n class w assum 8j, p j =/k and 8j, j =, ar known and that th x ar dmnsonal Frst analyz assumng z ar known Convnnt notaton: rprsnt th numbr z as a unt vctor bt squnc Exampl: k=4 z =) 000 z =2) 000 z =3) 000 z =4) 000 Notaton: z,j s j th bt wthn z z =2) 000 ) z,2 = z,3 =0
2 Maxmum lklhood stmaton Frst analyz assumng z ar known Th Complt Data ncluds all th x,z Maxmum lklhood stmaton Th Lklhood p(z )p(x z,µ z ) Data =(x,z ), (x 2,z 2 ),...,(x N,z N ) L = Y = Y (/k) p 2 2 (x µz )2 = Y (/k) p 2 2 j z,j(x µj)2 Notaton trck: xactly on trm rmans from th sum! Maxmum lklhood stmaton Maxmum lklhood stmaton L = Y (/k) p LogL = µ j =...=0 ) µ j = z,j x z,j 2 2 j z,j(x j µj)2 X X 2 2 z,j (x µ j ) 2 Ths s not surprsng. Why? Frst analyz assumng z ar known Th Complt Data ncluds all th x,z Data =(x,z ), (x 2,z 2 ),...,(x N,z N ) Th Obsrvd Data ncluds all th Data = x,x 2...,x N à Cannot us prvous stmat. What s th lklhood n ths cas? x Maxmum lklhood stmaton Th EM Algorthm Th Obsrvd Data ncluds z all th Data = x,x 2...,x N Maxmum lklhood prscrbs that w should optmz: p(obsrvd) = p(x,...,x N ) = X X X... p(x,...,x N,z,...,z N ) z z 2 z N Th Equaton for th lklhood nds to sum out (margnalz) ovr th z. No smpl closd form. x A gnral algorthm for maxmzng lklhood whn w hav hddn random varabls Th algorthm has a smpl form whn appld to mxtur modls W wll constran ourslvs to that smpl form And wll mnton th gnral schm of th EM algorthm brfly 2
3 Th EM Algorthm EM s an tratv algorthm Intalz probablty modl p us p to calculat an mprovd modl p St p =p Untl no furthr mprovmnt EM Algorthm for Mxtur Modls [E] Calculat usng p f,j = E p(z X,{µ 0`})[z,j ]=p(z = j {µ 0`},Data) [M] Estmat p paramtrs usng max lklhood soluton of th complt data by rplacng th unknown z,j by f,j EM for Mxturs n D EM for Mxturs n D [E] Calculat f,j = E p(z X,{µ 0`})[z,j ]=p(z = j {µ 0`},Data) f,j = p((z = j) and x ) p(x ) = p((z = j) and x ) ` p((z = `) and x ) = (/k) p `(/k) p 2 2 (x µ0 j )2 2 2 (x µ0`)2 [M] Estmat paramtrs usng max lklhood rplacng th unknown z,j by µ j = z,j x z ),j µ 00 j = f,j x f,j f,j Frst part holds for any mxtur modl. EM for Mxturs n D [E] Calculat for all,j f,j = (/k) p `(/k) p [M] Calculat for all j µ 00 j = f,j x f,j 2 2 (x µ0 j )2 2 2 (x µ0`)2 Gnral form of EM Dfn an auxlary functon Q(p,p ) Rlatv to obsrvd varabls O and hddn varabls H Q(p 0,p 00 )=E p0 (H O)[log p 00 (H, O)] Assgn for all j: µ 0 j = µ 00 j 3
4 Th EM Algorthm EM s an tratv algorthm Th EM Algorthm EM s an tratv algorthm Intalz probablty modl p us p to calculat an mprovd modl p St p =p Untl no furthr mprovmnt Intalz probablty modl p ck p so as to maxmz Q(p,p ) St p =p Untl no furthr mprovmnt EM Algorthm for Mxtur Modls [E] Calculat usng p f,j = E p(z X,{µ 0`})[z,j ]=p(z = j {µ 0`},Data) [M] Estmat p paramtrs usng max lklhood rplacng th unknown z,j by Usng th sam mthodology on any mxtur modl (not just Gaussan) ylds th sam tmplat. f,j Sm-Suprvsd Naïv Bays Modl Naïv Bays: robablstc modl wth strong smplfyng assumptons Illustratng applcaton: txt catgorzaton whr w hav data for (documnt,labl ) What f w hav many documnts but labls for only a fw of thm? Can th unlabld documnts hlp? Sm-Suprvsd Naïv Bays Modl What f w hav many documnts but labls for only a fw of thm? Can th unlabld documnts hlp? Bfor xplorng ths quston w wll dvlop th EM algorthm for ths modl whr th labls ar not known Rcall: Naïv Bays Modl Each class nducs a dstrbuton ovr faturs. Faturs ar condtonally ndpndnt gvn th class In ths slds I us th modl wth bnary faturs 4
5 Rcall: Naïv Bays Modl Rcall: Maxmum Lklhood p(z = j) =p j p(x,` = class j) =q j,` p(x class j) = Ỳ q x,` j,` ( q ( x,`) p(z = j and x )=p j Y p(z and x )= Y j " ` p j Y ` q x,` j,` ( q ( x,`) q x,` j,` ( q ( x,`) # z,j p j = p(z = j) = numbr of xampls wth class j numbr of xampls q j,` = p(x,` = z = j) = num of x wth class j and x,` = numbr of xampls wth class j Naïv Bays as Mxtur Modl EM Algorthm Rpat for =,...,N ck clustr Id z from dscrt dstrbuton wth paramtrs p,p 2,...,p k ck th xampl x from Nav Bays dstrbuton wth paramtrs q z Complt Data Lklhood L = Y " Y Y p j q x,` j,` ( j ` q ( x,`) Log Lklhood # z,j LogL = X X z,j (log p j + X` x,` log q j,` +( x,`) log( q ) j EM Algorthm EM Algorthm Maxmum Lklhood for complt data LogL = X X z,j (log p j + X` x,` log q j,` +( x,`) log( q ) j [w alrady solvd ths a fw lcturs ago] p j = z,j N q j,` = z,jx,` z,j E Stp: Calculatng f,j f,j = E p 0 (Z X)[z,j ]= p0 (z = j and x ) c p0 (z = c and x ) = p0 j Q` q0x,` j,` ( q0 ( x,`) c p0 c Q` q0x,` c,` ( q0 ( x,`) c,`) 5
6 EM Algorthm for Naïv Bays Calculat: Calculat: f,j = p 00 j = f,j N q 00 j,` = p0 j Q` q0x,` c p0 c f,jx,` f,j Assgn: p ßp and q ß q j,` ( q0 ( x,`) Q` q0x,` c,` ( q0 ( x,`) c,`) Sm-Suprvsd Naïv Bays Modl Naïv Bays for txt catgorzaton What f w hav many documnts but labls for only a fw of thm? Can th unlabld documnts hlp? Us EM: for xampls whr z s known us f,j =z,j nstad of stmatng t Nothng ls changs n th algorthm! 20 nwgroups data 20 nwgroups data 00% 90% 80% 0000 unlabld documnts No unlabld documnts 00% 90% 80% 3000 labld documnts 600 labld documnts 300 labld documnts 40 labld documnts 40 labld documnts 70% 70% Accuracy 60% 50% 40% Accuracy 60% 50% 40% 30% 30% 20% 20% 0% 0% 0% Numbr of Labld Documnts [From Ngam t all MLJ 999.] 0% Numbr of Unlabld Documnts [From Ngam t all MLJ 999.] Summary EM s a gnral algorthmc framwork for nfrnc wth hddn random varabls It taks a smpl form for mxtur modls altrnatng btwn stmatng fractonal mmbrshps and usng ths n maxmum lklhood calculatons. Gnral drvaton through th Q(p,p ) functon s applcabl n mor complx modls. Mxtur modl asly gnralzs to captur sm-suprvsd larnng 6
10/7/14. Mixture Models. Comp 135 Introduction to Machine Learning and Data Mining. Maximum likelihood estimation. Mixture of Normals in 1D
Comp 35 Introducton to Machn Larnng and Data Mnng Fall 204 rofssor: Ron Khardon Mxtur Modls Motvatd by soft k-mans w dvlopd a gnratv modl for clustrng. Assum thr ar k clustrs Clustrs ar not rqurd to hav
More informationReview - Probabilistic Classification
Mmoral Unvrsty of wfoundland Pattrn Rcognton Lctur 8 May 5, 6 http://www.ngr.mun.ca/~charlsr Offc Hours: Tusdays Thursdays 8:3-9:3 PM E- (untl furthr notc) Gvn lablld sampls { ɛc,,,..., } {. Estmat Rvw
More informationOutlier-tolerant parameter estimation
Outlr-tolrant paramtr stmaton Baysan thods n physcs statstcs machn larnng and sgnal procssng (SS 003 Frdrch Fraundorfr fraunfr@cg.tu-graz.ac.at Computr Graphcs and Vson Graz Unvrsty of Tchnology Outln
More informationAn Overview of Markov Random Field and Application to Texture Segmentation
An Ovrvw o Markov Random Fld and Applcaton to Txtur Sgmntaton Song-Wook Joo Octobr 003. What s MRF? MRF s an xtnson o Markov Procss MP (D squnc o r.v. s unlatral (causal: p(x t x,
More informationCHAPTER 7d. DIFFERENTIATION AND INTEGRATION
CHAPTER 7d. DIFFERENTIATION AND INTEGRATION A. J. Clark School o Engnrng Dpartmnt o Cvl and Envronmntal Engnrng by Dr. Ibrahm A. Assakka Sprng ENCE - Computaton Mthods n Cvl Engnrng II Dpartmnt o Cvl and
More informationLogistic Regression I. HRP 261 2/10/ am
Logstc Rgrsson I HRP 26 2/0/03 0- am Outln Introducton/rvw Th smplst logstc rgrsson from a 2x2 tabl llustrats how th math works Stp-by-stp xampls to b contnud nxt tm Dummy varabls Confoundng and ntracton
More information2. Grundlegende Verfahren zur Übertragung digitaler Signale (Zusammenfassung) Informationstechnik Universität Ulm
. Grundlgnd Vrfahrn zur Übrtragung dgtalr Sgnal (Zusammnfassung) wt Dc. 5 Transmsson of Dgtal Sourc Sgnals Sourc COD SC COD MOD MOD CC dg RF s rado transmsson mdum Snk DC SC DC CC DM dg DM RF g physcal
More informationAnalyzing Frequencies
Frquncy (# ndvduals) Frquncy (# ndvduals) /3/16 H o : No dffrnc n obsrvd sz frquncs and that prdctd by growth modl How would you analyz ths data? 15 Obsrvd Numbr 15 Expctd Numbr from growth modl 1 1 5
More informationGrand Canonical Ensemble
Th nsmbl of systms mmrsd n a partcl-hat rsrvor at constant tmpratur T, prssur P, and chmcal potntal. Consdr an nsmbl of M dntcal systms (M =,, 3,...M).. Thy ar mutually sharng th total numbr of partcls
More informationIntroduction to logistic regression
Itroducto to logstc rgrsso Gv: datast D { 2 2... } whr s a k-dmsoal vctor of ral-valud faturs or attrbuts ad s a bar class labl or targt. hus w ca sa that R k ad {0 }. For ampl f k 4 a datast of 3 data
More informationFrom Structural Analysis to FEM. Dhiman Basu
From Structural Analyss to FEM Dhman Basu Acknowldgmnt Followng txt books wr consultd whl prparng ths lctur nots: Znkwcz, OC O.C. andtaylor Taylor, R.L. (000). Th FntElmnt Mthod, Vol. : Th Bass, Ffth dton,
More informationA Note on Estimability in Linear Models
Intrnatonal Journal of Statstcs and Applcatons 2014, 4(4): 212-216 DOI: 10.5923/j.statstcs.20140404.06 A Not on Estmablty n Lnar Modls S. O. Adymo 1,*, F. N. Nwob 2 1 Dpartmnt of Mathmatcs and Statstcs,
More informationEM and Structure Learning
EM and Structure Learnng Le Song Machne Learnng II: Advanced Topcs CSE 8803ML, Sprng 2012 Partally observed graphcal models Mxture Models N(μ 1, Σ 1 ) Z X N N(μ 2, Σ 2 ) 2 Gaussan mxture model Consder
More informationte Finance (4th Edition), July 2017.
Appndx Chaptr. Tchncal Background Gnral Mathmatcal and Statstcal Background Fndng a bas: 3 2 = 9 3 = 9 1 /2 x a = b x = b 1/a A powr of 1 / 2 s also quvalnt to th squar root opraton. Fndng an xponnt: 3
More informationST 524 NCSU - Fall 2008 One way Analysis of variance Variances not homogeneous
ST 54 NCSU - Fall 008 On way Analyss of varanc Varancs not homognous On way Analyss of varanc Exampl (Yandll, 997) A plant scntst masurd th concntraton of a partcular vrus n plant sap usng ELISA (nzym-lnkd
More informationPhysics 256: Lecture 2. Physics
Physcs 56: Lctur Intro to Quantum Physcs Agnda for Today Complx Numbrs Intrfrnc of lght Intrfrnc Two slt ntrfrnc Dffracton Sngl slt dffracton Physcs 01: Lctur 1, Pg 1 Constructv Intrfrnc Ths wll occur
More information??? Dynamic Causal Modelling for M/EEG. Electroencephalography (EEG) Dynamic Causal Modelling. M/EEG analysis at sensor level. time.
Elctroncphalography EEG Dynamc Causal Modllng for M/EEG ampltud μv tm ms tral typ 1 tm channls channls tral typ 2 C. Phllps, Cntr d Rchrchs du Cyclotron, ULg, Blgum Basd on slds from: S. Kbl M/EEG analyss
More informationSCITECH Volume 5, Issue 1 RESEARCH ORGANISATION November 17, 2015
Journal of Informaton Scncs and Computng Tchnologs(JISCT) ISSN: 394-966 SCITECH Volum 5, Issu RESEARCH ORGANISATION Novmbr 7, 5 Journal of Informaton Scncs and Computng Tchnologs www.sctcrsarch.com/journals
More informationDiscrete Shells Simulation
Dscrt Shlls Smulaton Xaofng M hs proct s an mplmntaton of Grnspun s dscrt shlls, th modl of whch s govrnd by nonlnar mmbran and flxural nrgs. hs nrgs masur dffrncs btwns th undformd confguraton and th
More informationIntroduction to logistic regression
Itroducto to logstc rgrsso Gv: datast D {... } whr s a k-dmsoal vctor of ral-valud faturs or attrbuts ad s a bar class labl or targt. hus w ca sa that R k ad {0 }. For ampl f k 4 a datast of 3 data pots
More informationEcon107 Applied Econometrics Topic 10: Dummy Dependent Variable (Studenmund, Chapter 13)
Pag- Econ7 Appld Economtrcs Topc : Dummy Dpndnt Varabl (Studnmund, Chaptr 3) I. Th Lnar Probablty Modl Suppos w hav a cross scton of 8-24 yar-olds. W spcfy a smpl 2-varabl rgrsson modl. Th probablty of
More informationLecture 23 APPLICATIONS OF FINITE ELEMENT METHOD TO SCALAR TRANSPORT PROBLEMS
COMPUTTION FUID DYNMICS: FVM: pplcatons to Scalar Transport Prolms ctur 3 PPICTIONS OF FINITE EEMENT METHOD TO SCR TRNSPORT PROBEMS 3. PPICTION OF FEM TO -D DIFFUSION PROBEM Consdr th stady stat dffuson
More informationCOMPLEX NUMBER PAIRWISE COMPARISON AND COMPLEX NUMBER AHP
ISAHP 00, Bal, Indonsa, August -9, 00 COMPLEX NUMBER PAIRWISE COMPARISON AND COMPLEX NUMBER AHP Chkako MIYAKE, Kkch OHSAWA, Masahro KITO, and Masaak SHINOHARA Dpartmnt of Mathmatcal Informaton Engnrng
More information8-node quadrilateral element. Numerical integration
Fnt Elmnt Mthod lctur nots _nod quadrlatral lmnt Pag of 0 -nod quadrlatral lmnt. Numrcal ntgraton h tchnqu usd for th formulaton of th lnar trangl can b formall tndd to construct quadrlatral lmnts as wll
More informationLecture 3: Phasor notation, Transfer Functions. Context
EECS 5 Fall 4, ctur 3 ctur 3: Phasor notaton, Transfr Functons EECS 5 Fall 3, ctur 3 Contxt In th last lctur, w dscussd: how to convrt a lnar crcut nto a st of dffrntal quatons, How to convrt th st of
More informationThe Hyperelastic material is examined in this section.
4. Hyprlastcty h Hyprlastc matral s xad n ths scton. 4..1 Consttutv Equatons h rat of chang of ntrnal nrgy W pr unt rfrnc volum s gvn by th strss powr, whch can b xprssd n a numbr of dffrnt ways (s 3.7.6):
More informationFakultät III Wirtschaftswissenschaften Univ.-Prof. Dr. Jan Franke-Viebach
Unvrstät Sgn Fakultät III Wrtschaftswssnschaftn Unv.-rof. Dr. Jan Frank-Vbach Exam Intrnatonal Fnancal Markts Summr Smstr 206 (2 nd Exam rod) Avalabl tm: 45 mnuts Soluton For your attnton:. las do not
More informationEconomics 600: August, 2007 Dynamic Part: Problem Set 5. Problems on Differential Equations and Continuous Time Optimization
THE UNIVERSITY OF MARYLAND COLLEGE PARK, MARYLAND Economcs 600: August, 007 Dynamc Part: Problm St 5 Problms on Dffrntal Equatons and Contnuous Tm Optmzaton Quston Solv th followng two dffrntal quatons.
More informationMATH 829: Introduction to Data Mining and Analysis The EM algorithm (part 2)
1/16 MATH 829: Introducton to Data Mnng and Analyss The EM algorthm (part 2) Domnque Gullot Departments of Mathematcal Scences Unversty of Delaware Aprl 20, 2016 Recall 2/16 We are gven ndependent observatons
More informationChapter 6 Student Lecture Notes 6-1
Chaptr 6 Studnt Lctur Nots 6-1 Chaptr Goals QM353: Busnss Statstcs Chaptr 6 Goodnss-of-Ft Tsts and Contngncy Analyss Aftr compltng ths chaptr, you should b abl to: Us th ch-squar goodnss-of-ft tst to dtrmn
More informationSeptember 27, Introduction to Ordinary Differential Equations. ME 501A Seminar in Engineering Analysis Page 1. Outline
Introucton to Ornar Dffrntal Equatons Sptmbr 7, 7 Introucton to Ornar Dffrntal Equatons Larr artto Mchancal Engnrng AB Smnar n Engnrng Analss Sptmbr 7, 7 Outln Rvw numrcal solutons Bascs of ffrntal quatons
More informationExternal Equivalent. EE 521 Analysis of Power Systems. Chen-Ching Liu, Boeing Distinguished Professor Washington State University
xtrnal quvalnt 5 Analyss of Powr Systms Chn-Chng Lu, ong Dstngushd Profssor Washngton Stat Unvrsty XTRNAL UALNT ach powr systm (ara) s part of an ntrconnctd systm. Montorng dvcs ar nstalld and data ar
More informationFakultät III Univ.-Prof. Dr. Jan Franke-Viebach
Unv.Prof. r. J. FrankVbach WS 067: Intrnatonal Economcs ( st xam prod) Unvrstät Sgn Fakultät III Unv.Prof. r. Jan FrankVbach Exam Intrnatonal Economcs Wntr Smstr 067 ( st Exam Prod) Avalabl tm: 60 mnuts
More informationSemi-Supervised Learning
Sem-Supervsed Learnng Consder the problem of Prepostonal Phrase Attachment. Buy car wth money ; buy car wth wheel There are several ways to generate features. Gven the lmted representaton, we can assume
More informationYou already learned about dummies as independent variables. But. what do you do if the dependent variable is a dummy?
CHATER 5: DUMMY DEENDENT VARIABLES AND NON-LINEAR REGRESSION. Th roblm of Dummy Dpndnt Varabls You alrady larnd about dumms as ndpndnt varabls. But what do you do f th dpndnt varabl s a dummy? On answr
More informationFrom Elimination to Belief Propagation
School of omputr Scinc Th lif Propagation (Sum-Product lgorithm Probabilistic Graphical Modls (10-708 Lctur 5, Sp 31, 2007 Rcptor Kinas Rcptor Kinas Kinas X 5 ric Xing Gn G T X 6 X 7 Gn H X 8 Rading: J-hap
More informationJournal of Theoretical and Applied Information Technology 10 th January Vol. 47 No JATIT & LLS. All rights reserved.
Journal o Thortcal and Appld Inormaton Tchnology th January 3. Vol. 47 No. 5-3 JATIT & LLS. All rghts rsrvd. ISSN: 99-8645 www.att.org E-ISSN: 87-395 RESEARCH ON PROPERTIES OF E-PARTIAL DERIVATIVE OF LOGIC
More informationIntroduction to Arithmetic Geometry Fall 2013 Lecture #20 11/14/2013
18.782 Introduction to Arithmtic Gomtry Fall 2013 Lctur #20 11/14/2013 20.1 Dgr thorm for morphisms of curvs Lt us rstat th thorm givn at th nd of th last lctur, which w will now prov. Thorm 20.1. Lt φ:
More informationProblem Set #2 Due: Friday April 20, 2018 at 5 PM.
1 EE102B Spring 2018 Signal Procssing and Linar Systms II Goldsmith Problm St #2 Du: Friday April 20, 2018 at 5 PM. 1. Non-idal sampling and rcovry of idal sampls by discrt-tim filtring 30 pts) Considr
More informationClustering gene expression data & the EM algorithm
CG, Fall 2011-12 Clusterng gene expresson data & the EM algorthm CG 08 Ron Shamr 1 How Gene Expresson Data Looks Entres of the Raw Data matrx: Rato values Absolute values Row = gene s expresson pattern
More informationHydrogen Atom and One Electron Ions
Hydrogn Atom and On Elctron Ions Th Schrödingr quation for this two-body problm starts out th sam as th gnral two-body Schrödingr quation. First w sparat out th motion of th cntr of mass. Th intrnal potntial
More informationMaximum Likelihood Estimation
Maxmum Lkelhood Estmaton INFO-2301: Quanttatve Reasonng 2 Mchael Paul and Jordan Boyd-Graber MARCH 7, 2017 INFO-2301: Quanttatve Reasonng 2 Paul and Boyd-Graber Maxmum Lkelhood Estmaton 1 of 9 Why MLE?
More information6.3.4 Modified Euler s method of integration
6.3.4 Modfed Euler s method of ntegraton Before dscussng the applcaton of Euler s method for solvng the swng equatons, let us frst revew the basc Euler s method of numercal ntegraton. Let the general from
More informationSection 11.6: Directional Derivatives and the Gradient Vector
Sction.6: Dirctional Drivativs and th Gradint Vctor Practic HW rom Stwart Ttbook not to hand in p. 778 # -4 p. 799 # 4-5 7 9 9 35 37 odd Th Dirctional Drivativ Rcall that a b Slop o th tangnt lin to th
More informationΑ complete processing methodology for 3D monitoring using GNSS receivers
7-5-5 NATIONA TECHNICA UNIVERSITY OF ATHENS SCHOO OF RURA AND SURVEYING ENGINEERING DEPARTMENT OF TOPOGRAPHY AORATORY OF GENERA GEODESY Α complt procssng mthodology for D montorng usng GNSS rcvrs Gorg
More informationMachine learning: Density estimation
CS 70 Foundatons of AI Lecture 3 Machne learnng: ensty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square ata: ensty estmaton {.. n} x a vector of attrbute values Objectve: estmate the model of
More informationOn Parameter Estimation of the Envelope Gaussian Mixture Model
Australan Communcatons Thory Worsho (AusCTW) On Paramtr Estmaton of th Envlo Gaussan Mtur Modl Lnyun Huang Dartmnt of ECSE Monash Unvrsty, Australa Lnyun.Huang@monash.du Y Hong Dartmnt of ECSE Monash Unvrsty,
More informationLecture 14. Relic neutrinos Temperature at neutrino decoupling and today Effective degeneracy factor Neutrino mass limits Saha equation
Lctur Rlc nutrnos mpratur at nutrno dcoupln and today Effctv dnracy factor Nutrno mass lmts Saha quaton Physcal Cosmoloy Lnt 005 Rlc Nutrnos Nutrnos ar wakly ntractn partcls (lptons),,,,,,, typcal ractons
More informationComputing and Communications -- Network Coding
89 90 98 00 Computing and Communications -- Ntwork Coding Dr. Zhiyong Chn Institut of Wirlss Communications Tchnology Shanghai Jiao Tong Univrsity China Lctur 5- Nov. 05 0 Classical Information Thory Sourc
More informationLogistic Regression. CAP 5610: Machine Learning Instructor: Guo-Jun QI
Logstc Regresson CAP 561: achne Learnng Instructor: Guo-Jun QI Bayes Classfer: A Generatve model odel the posteror dstrbuton P(Y X) Estmate class-condtonal dstrbuton P(X Y) for each Y Estmate pror dstrbuton
More informationON THE COMPLEXITY OF K-STEP AND K-HOP DOMINATING SETS IN GRAPHS
MATEMATICA MONTISNIRI Vol XL (2017) MATEMATICS ON TE COMPLEXITY OF K-STEP AN K-OP OMINATIN SETS IN RAPS M FARAI JALALVAN AN N JAFARI RA partmnt of Mathmatcs Shahrood Unrsty of Tchnology Shahrood Iran Emals:
More informationThe exam is closed book, closed notes except your one-page cheat sheet.
CS 89 Fall 206 Introducton to Machne Learnng Fnal Do not open the exam before you are nstructed to do so The exam s closed book, closed notes except your one-page cheat sheet Usage of electronc devces
More informationQuantifying Uncertainty
Partcle Flters Quantfyng Uncertanty Sa Ravela M. I. T Last Updated: Sprng 2013 1 Quantfyng Uncertanty Partcle Flters Partcle Flters Appled to Sequental flterng problems Can also be appled to smoothng problems
More informationA Probabilistic Characterization of Simulation Model Uncertainties
A Proalstc Charactrzaton of Sulaton Modl Uncrtants Vctor Ontvros Mohaad Modarrs Cntr for Rsk and Rlalty Unvrsty of Maryland 1 Introducton Thr s uncrtanty n odl prdctons as wll as uncrtanty n xprnts Th
More informationAddition of angular momentum
Addition of angular momntum April, 07 Oftn w nd to combin diffrnt sourcs of angular momntum to charactriz th total angular momntum of a systm, or to divid th total angular momntum into parts to valuat
More informationLECTURE 6 TRANSFORMATION OF RANDOM VARIABLES
LECTURE 6 TRANSFORMATION OF RANDOM VARIABLES TRANSFORMATION OF FUNCTION OF A RANDOM VARIABLE UNIVARIATE TRANSFORMATIONS TRANSFORMATION OF RANDOM VARIABLES If s a rv wth cdf F th Y=g s also a rv. If w wrt
More informationTHE ARIMOTO-BLAHUT ALGORITHM FOR COMPUTATION OF CHANNEL CAPACITY. William A. Pearlman. References: S. Arimoto - IEEE Trans. Inform. Thy., Jan.
THE ARIMOTO-BLAHUT ALGORITHM FOR COMPUTATION OF CHANNEL CAPACITY Wllam A. Pearlman 2002 References: S. Armoto - IEEE Trans. Inform. Thy., Jan. 1972 R. Blahut - IEEE Trans. Inform. Thy., July 1972 Recall
More informationIV. First Law of Thermodynamics. Cooler. IV. First Law of Thermodynamics
D. Applcatons to stady flow dvcs. Hat xchangrs - xampl: Clkr coolr for cmnt kln Scondary ar 50 C, 57,000 lbm/h Clkr? C, 5 ton/h Coolr Clkr 400 C, 5 ton/h Scondary ar 0 C, 57,000 lbm/h a. Assumptons. changs
More informationMath 656 Midterm Examination March 27, 2015 Prof. Victor Matveev
Math 656 Mdtrm Examnatn March 7, 05 Prf. Vctr Matvv ) (4pts) Fnd all vals f n plar r artsan frm, and plt thm as pnts n th cmplx plan: (a) Snc n-th rt has xactly n vals, thr wll b xactly =6 vals, lyng n
More informationThe Fourier Transform
/9/ Th ourr Transform Jan Baptst Josph ourr 768-83 Effcnt Data Rprsntaton Data can b rprsntd n many ways. Advantag usng an approprat rprsntaton. Eampls: osy ponts along a ln Color spac rd/grn/blu v.s.
More informationAddition of angular momentum
Addition of angular momntum April, 0 Oftn w nd to combin diffrnt sourcs of angular momntum to charactriz th total angular momntum of a systm, or to divid th total angular momntum into parts to valuat th
More informationSearch sequence databases 3 10/25/2016
Sarch squnc databass 3 10/25/2016 Etrm valu distribution Ø Suppos X is a random variabl with probability dnsity function p(, w sampl a larg numbr S of indpndnt valus of X from this distribution for an
More informationAbstract Interpretation: concrete and abstract semantics
Abstract Intrprtation: concrt and abstract smantics Concrt smantics W considr a vry tiny languag that manags arithmtic oprations on intgrs valus. Th (concrt) smantics of th languags cab b dfind by th funzcion
More information2F1120 Spektrala transformer för Media Solutions to Steiglitz, Chapter 1
F110 Spktrala transformr för Mdia Solutions to Stiglitz, Chaptr 1 Prfac This documnt contains solutions to slctd problms from Kn Stiglitz s book: A Digital Signal Procssing Primr publishd by Addison-Wsly.
More informationThat is, we start with a general matrix: And end with a simpler matrix:
DIAGON ALIZATION OF THE STR ESS TEN SOR INTRO DUCTIO N By th us of Cauchy s thorm w ar abl to rduc th numbr of strss componnts in th strss tnsor to only nin valus. An additional simplification of th strss
More informationExpectation Maximization Mixture Models HMMs
-755 Machne Learnng for Sgnal Processng Mture Models HMMs Class 9. 2 Sep 200 Learnng Dstrbutons for Data Problem: Gven a collecton of eamples from some data, estmate ts dstrbuton Basc deas of Mamum Lelhood
More informationParametric fractional imputation for missing data analysis. Jae Kwang Kim Survey Working Group Seminar March 29, 2010
Parametrc fractonal mputaton for mssng data analyss Jae Kwang Km Survey Workng Group Semnar March 29, 2010 1 Outlne Introducton Proposed method Fractonal mputaton Approxmaton Varance estmaton Multple mputaton
More informationHeisenberg Model. Sayed Mohammad Mahdi Sadrnezhaad. Supervisor: Prof. Abdollah Langari
snbrg Modl Sad Mohammad Mahd Sadrnhaad Survsor: Prof. bdollah Langar bstract: n ths rsarch w tr to calculat analtcall gnvalus and gnvctors of fnt chan wth ½-sn artcls snbrg modl. W drov gnfuctons for closd
More informationGaussian Mixture Models
Lab Gaussan Mxture Models Lab Objectve: Understand the formulaton of Gaussan Mxture Models (GMMs) and how to estmate GMM parameters. You ve already seen GMMs as the observaton dstrbuton n certan contnuous
More informationLecture 10 Support Vector Machines II
Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the fake-test data; fxed
More informationRoadmap. XML Indexing. DataGuide example. DataGuides. Strong DataGuides. Multiple DataGuides for same data. CPS Topics in Database Systems
Roadmap XML Indxing CPS 296.1 Topics in Databas Systms Indx fabric Coopr t al. A Fast Indx for Smistructurd Data. VLDB, 2001 DataGuid Goldman and Widom. DataGuids: Enabling Qury Formulation and Optimization
More informationContinuous probability distributions
Continuous probability distributions Many continuous probability distributions, including: Uniform Normal Gamma Eponntial Chi-Squard Lognormal Wibull EGR 5 Ch. 6 Uniform distribution Simplst charactrizd
More informationThe Geometry of Logit and Probit
The Geometry of Logt and Probt Ths short note s meant as a supplement to Chapters and 3 of Spatal Models of Parlamentary Votng and the notaton and reference to fgures n the text below s to those two chapters.
More informationComputing Correlated Equilibria in Multi-Player Games
Computng Correlated Equlbra n Mult-Player Games Chrstos H. Papadmtrou Presented by Zhanxang Huang December 7th, 2005 1 The Author Dr. Chrstos H. Papadmtrou CS professor at UC Berkley (taught at Harvard,
More informationNote on EM-training of IBM-model 1
Note on EM-tranng of IBM-model INF58 Language Technologcal Applcatons, Fall The sldes on ths subject (nf58 6.pdf) ncludng the example seem nsuffcent to gve a good grasp of what s gong on. Hence here are
More informationA Propagating Wave Packet Group Velocity Dispersion
Lctur 8 Phys 375 A Propagating Wav Packt Group Vlocity Disprsion Ovrviw and Motivation: In th last lctur w lookd at a localizd solution t) to th 1D fr-particl Schrödingr quation (SE) that corrsponds to
More informationThe Expectation-Maximization Algorithm
The Expectaton-Maxmaton Algorthm Charles Elan elan@cs.ucsd.edu November 16, 2007 Ths chapter explans the EM algorthm at multple levels of generalty. Secton 1 gves the standard hgh-level verson of the algorthm.
More informationToday s logistic regression topics. Lecture 15: Effect modification, and confounding in logistic regression. Variables. Example
Today s stc rgrsson tocs Lctur 15: Effct modfcaton, and confoundng n stc rgrsson Sandy Eckl sckl@jhsh.du 16 May 28 Includng catgorcal rdctor crat dummy/ndcator varabls just lk for lnar rgrsson Comarng
More informationElements of Statistical Thermodynamics
24 Elmnts of Statistical Thrmodynamics Statistical thrmodynamics is a branch of knowldg that has its own postulats and tchniqus. W do not attmpt to giv hr vn an introduction to th fild. In this chaptr,
More informationxp(x µ) = 0 p(x = 0 µ) + 1 p(x = 1 µ) = µ
CSE 455/555 Sprng 2013 Homework 7: Parametrc Technques Jason J. Corso Computer Scence and Engneerng SUY at Buffalo jcorso@buffalo.edu Solutons by Yngbo Zhou Ths assgnment does not need to be submtted and
More information10-701/ Machine Learning, Fall 2005 Homework 3
10-701/15-781 Machne Learnng, Fall 2005 Homework 3 Out: 10/20/05 Due: begnnng of the class 11/01/05 Instructons Contact questons-10701@autonlaborg for queston Problem 1 Regresson and Cross-valdaton [40
More informationLucas Test is based on Euler s theorem which states that if n is any integer and a is coprime to n, then a φ(n) 1modn.
Modul 10 Addtonal Topcs 10.1 Lctur 1 Prambl: Dtrmnng whthr a gvn ntgr s prm or compost s known as prmalty tstng. Thr ar prmalty tsts whch mrly tll us whthr a gvn ntgr s prm or not, wthout gvng us th factors
More informationQuasi-Classical States of the Simple Harmonic Oscillator
Quasi-Classical Stats of th Simpl Harmonic Oscillator (Draft Vrsion) Introduction: Why Look for Eignstats of th Annihilation Oprator? Excpt for th ground stat, th corrspondnc btwn th quantum nrgy ignstats
More informationMLE and Bayesian Estimation. Jie Tang Department of Computer Science & Technology Tsinghua University 2012
MLE and Bayesan Estmaton Je Tang Department of Computer Scence & Technology Tsnghua Unversty 01 1 Lnear Regresson? As the frst step, we need to decde how we re gong to represent the functon f. One example:
More informationSpace of ML Problems. CSE 473: Artificial Intelligence. Parameter Estimation and Bayesian Networks. Learning Topics
/7/7 CSE 73: Artfcal Intellgence Bayesan - Learnng Deter Fox Sldes adapted from Dan Weld, Jack Breese, Dan Klen, Daphne Koller, Stuart Russell, Andrew Moore & Luke Zettlemoyer What s Beng Learned? Space
More information3.1 ML and Empirical Distribution
67577 Intro. to Machne Learnng Fall semester, 2008/9 Lecture 3: Maxmum Lkelhood/ Maxmum Entropy Dualty Lecturer: Amnon Shashua Scrbe: Amnon Shashua 1 In the prevous lecture we defned the prncple of Maxmum
More informationStanford University CS359G: Graph Partitioning and Expanders Handout 4 Luca Trevisan January 13, 2011
Stanford Unversty CS359G: Graph Parttonng and Expanders Handout 4 Luca Trevsan January 3, 0 Lecture 4 In whch we prove the dffcult drecton of Cheeger s nequalty. As n the past lectures, consder an undrected
More informationSundials and Linear Algebra
Sundials and Linar Algbra M. Scot Swan July 2, 25 Most txts on crating sundials ar dirctd towards thos who ar solly intrstd in making and using sundials and usually assums minimal mathmatical background.
More informationCh. 24 Molecular Reaction Dynamics 1. Collision Theory
Ch. 4 Molcular Raction Dynamics 1. Collision Thory Lctur 16. Diffusion-Controlld Raction 3. Th Matrial Balanc Equation 4. Transition Stat Thory: Th Eyring Equation 5. Transition Stat Thory: Thrmodynamic
More informationChemical Physics II. More Stat. Thermo Kinetics Protein Folding...
Chmical Physics II Mor Stat. Thrmo Kintics Protin Folding... http://www.nmc.ctc.com/imags/projct/proj15thumb.jpg http://nuclarwaponarchiv.org/usa/tsts/ukgrabl2.jpg http://www.photolib.noaa.gov/corps/imags/big/corp1417.jpg
More information1- Summary of Kinetic Theory of Gases
Dr. Kasra Etmad Octobr 5, 011 1- Summary of Kntc Thory of Gass - Radaton 3- E4 4- Plasma Proprts f(v f ( v m 4 ( kt 3/ v xp( mv kt V v v m v 1 rms V kt v m ( m 1/ v 8kT m 3kT v rms ( m 1/ E3: Prcntag of
More information22/ Breakdown of the Born-Oppenheimer approximation. Selection rules for rotational-vibrational transitions. P, R branches.
Subjct Chmistry Papr No and Titl Modul No and Titl Modul Tag 8/ Physical Spctroscopy / Brakdown of th Born-Oppnhimr approximation. Slction ruls for rotational-vibrational transitions. P, R branchs. CHE_P8_M
More informationFEFF and Related Codes
FEFF and Rlatd Cods Anatoly Frnl Profssor Physcs Dpartmnt, Yshva Unvrsty, w Yor, USA Synchrotron Catalyss Consortum, Broohavn atonal Laboratory, USA www.yu.du/faculty/afrnl Anatoly.Frnl@yu.du FEFF: John
More informationProbability Theory (revisited)
Probablty Theory (revsted) Summary Probablty v.s. plausblty Random varables Smulaton of Random Experments Challenge The alarm of a shop rang. Soon afterwards, a man was seen runnng n the street, persecuted
More informationSupport Vector Machines. Vibhav Gogate The University of Texas at dallas
Support Vector Machnes Vbhav Gogate he Unversty of exas at dallas What We have Learned So Far? 1. Decson rees. Naïve Bayes 3. Lnear Regresson 4. Logstc Regresson 5. Perceptron 6. Neural networks 7. K-Nearest
More informationBasic Statistical Analysis and Yield Calculations
October 17, 007 Basc Statstcal Analyss and Yeld Calculatons Dr. José Ernesto Rayas Sánchez 1 Outlne Sources of desgn-performance uncertanty Desgn and development processes Desgn for manufacturablty A general
More informationTotal Least Squares Fitting the Three-Parameter Inverse Weibull Density
EUROPEAN JOURNAL OF PURE AND APPLIED MATHEMATICS Vol. 7, No. 3, 2014, 230-245 ISSN 1307-5543 www.jpam.com Total Last Squars Fttng th Thr-Paramtr Invrs Wbull Dnsty Dragan Juć, Darja Marovć Dpartmnt of Mathmatcs,
More informationThe Gaussian classifier. Nuno Vasconcelos ECE Department, UCSD
he Gaussan classfer Nuno Vasconcelos ECE Department, UCSD Bayesan decson theory recall that we have state of the world X observatons g decson functon L[g,y] loss of predctng y wth g Bayes decson rule s
More informationShortest Paths in Graphs. Paths in graphs. Shortest paths CS 445. Alon Efrat Slides courtesy of Erik Demaine and Carola Wenk
S 445 Shortst Paths n Graphs lon frat Sls courtsy of rk man an arola Wnk Paths n raphs onsr a raph G = (V, ) wth -wht functon w : R. Th wht of path p = v v v k s fn to xampl: k = w ( p) = w( v, v + ).
More informationEEC 686/785 Modeling & Performance Evaluation of Computer Systems. Lecture 12
EEC 686/785 Modlng & Prformanc Evaluaton of Computr Systms Lctur Dpartmnt of Elctrcal and Computr Engnrng Clvland Stat Unvrsty wnbng@.org (basd on Dr. Ra Jan s lctur nots) Outln Rvw of lctur k r Factoral
More information