Introduction. Modeling Data. Approach. Quality of Fit. Likelihood. Probabilistic Approach
|
|
- Martina Elliott
- 5 years ago
- Views:
Transcription
1 Introducton Modelng Data Gven a et of obervaton, we wh to ft a mathematcal model Model deend on adutable arameter traght lne: m + c n Polnomal: a + a + a + L+ a n Choce of model deend uon roblem Aroach Degn a fgure-of-mert (or ftne) functon Meaure agreement between model and data mall value mean good ft Adut the model arameter to mnme the ftne functon Gve the bet-ft arameter Qualt of Ft If obervaton no, we can never ft eactl We wh to etmate he qualt of the ft of the model he uncertant n the etmate of the bet ft arameter Probabltc Aroach Concentrate on D outut for now Inut obervaton: Model form: { }.. { }.. f ( ; + : Model arameter : Redual noe term Dtrbuton () Lkelhood Probablt of obervaton gven the model arameter O ( f ( ; ) Prob. of all data ( f ( ; ) he lkelhood of the data gven the arameter
2 Model arameter etmaton We would lke the arameter whch are mot lkel gven the data data) data) data) unknown, but fed Mame Pror PDF of aram Eamle: Lne fttng Ft traght lne to data {, } m + c f ( ; { m, c}) Aume normal error on wth.d. ( ) ( ;, ) ( data m, ( ( m + ) Eamle: Lne fttng ( data m, log data m, ( ( m + ) log ( ( m + ) log data m, ( ( m + ) + cont { m, c} arg mn F( m, ( ( m + ) Lne Fttng { m, c} arg mn F( m, Let df dm df dc, ( ( m + ) ( ( ( m + ) ( m c etc m c ( ( m + ) m m Uncertant he arameter that mame data onl gve u a ngle value o nformaton about uncertant Often we need the full df data) data) data) Hard to comute! Dtrbuton of arameter ( data) Z Z data) a normalaton contant Z d Can be comuted eactl for ome cae Otherwe can be aromated b amlng where necear Z data ) are random amle from
3 Mean Mean of ), µ ) ˆ ( data) d d Z ˆ data ) Z Z data ) Eamle: Lne Fttng log data m, ( ( m + ) + cont m c If cont K ( ˆ) K( ˆ ) + cont K data) ( : ˆ, ) Etmatng Parameter PDF Error Proagaton ot alwa ea to comute dtrbuton eactl Common aromaton Error roagaton Monte-Carlo etmate Ueful when we have an analtc form for the comutng arameter from the data F({, }) F ({, }) For ntance, for lne fttng m c If vare b Error Proagaton F ({, }) then hu Gauan noe on caue noe on vare b aro. wth varance wth varance F F Monte-Carlo Method Ued when no analtc functon avalable Algorthm: For k..m Add noe to each orgnal meaurement Etmate otmal arameter, k Parameter PDF aromated b PDF of { k } otal varance of F 3
4 Qualt of Ft How well doe the model actuall ft the data? Qualt of Ft How well doe the model actuall ft the data? Probabl not a ver good ft Deend on uncertant on meaurement Probabl not a ver good ft Deend on uncertant on meaurement Qualt of Ft Model form: f ( ; + ( f ( ; ) Aume normal error on wth.d. ( f ( ; ) log + cont If each Ch-quared Dtrbuton drawn from a zero mean, unt normal PDF, then the PDF of M known a χ ( M : ) M ha a mean of and a varance of Ch quare PDF chdf(m,) n MatLab Ch-quared Dtrbuton Ch-quare and the Gamma Functon For large, χ (M:) (M:, ) A Gauan wth mean and varance Ch-quared ormal 5 Gamma Functon Incomlete Gamma Functon P( a, ) Γ( a) CDF of Ch-quare: Γ( z) a t t z t t t e dt t e dt M P ( χ < M ) P(, ) χ n! Γ( + n)
5 Qualt of Ft ( f ( ; ) log + cont Margnalaton ) ( M ( f ( ; ) χ wth d.o.f., ) Let If P P M M < If then P P(χ < M ) (CDF of χ at M ).99 then model a M > reaonable ft.9999 the model a oor ft chcdf(m,) n MatLab, ) ( ) ), ) d ( ) ) ) d ( ), ) d Eamle: Lne fttng If we are onl ntereted n the gradent, m m, c data) ( m, c : ( mˆ, cˆ), ) m data) ( m, c : ( mˆ, cˆ), ) dc m data) ( m; mˆ, ) m m General Lnear Model Conder model of form M f ( ) w f ( ) + ( :, ) Lne fttng : M, f (),f () Polnomal degree d fttng : M d +, f () General Lnear Model Data: {, } Contruct M Degn Matr D { D } D f ( ) / Otmal weght w gven b oluton to Dw r r / General Lnear Model Perform VD on D D UWV wˆ VW Covarance of r I U w VW U IUW w VW V r V 5
6 Model electon uoe we have everal oble model Eg lnear, quadratc, cubc etc How do we elect the bet? P(data aram) alone nuffcent More comle model gve lower redual Akake Informaton Crtera mle method of model electon For each model fnd otmal arameter,, and comute AIC log + d d the number of ndeendent arameter elect model gvng the mallet AIC Baean Model electon uoe M ndee varou oble model We eek the mot robable model to ft to ome data We thu wh to fnd M to mame P( data M ) P( M ) P ( M data) P( data) If all model equall lkel, mame P( data M ) contant Baean Model electon uoe model M ha arameter, P ( data M ) data, M ) M ) d (he `Evdence for model M) Unfortunatel, th often moble to comute analtcall Varou aromaton rooed ee the lterature 6
AP Statistics Ch 3 Examining Relationships
Introducton To tud relatonhp between varable, we mut meaure the varable on the ame group of ndvdual. If we thnk a varable ma eplan or even caue change n another varable, then the eplanator varable and
More informationImprovements on Waring s Problem
Imrovement on Warng Problem L An-Png Bejng 85, PR Chna al@nacom Abtract By a new recurve algorthm for the auxlary equaton, n th aer, we wll gve ome mrovement for Warng roblem Keyword: Warng Problem, Hardy-Lttlewood
More informationSpecification -- Assumptions of the Simple Classical Linear Regression Model (CLRM) 1. Introduction
ECONOMICS 35* -- NOTE ECON 35* -- NOTE Specfcaton -- Aumpton of the Smple Clacal Lnear Regreon Model (CLRM). Introducton CLRM tand for the Clacal Lnear Regreon Model. The CLRM alo known a the tandard lnear
More informationEstimation of a proportion under a certain two-stage sampling design
Etmaton of a roorton under a certan two-tage amng degn Danutė Kraavcatė nttute of athematc and nformatc Lthuana Stattc Lthuana Lthuana e-ma: raav@tmt Abtract The am of th aer to demontrate wth exame that
More informationIterative Methods for Searching Optimal Classifier Combination Function
htt://www.cub.buffalo.edu Iteratve Method for Searchng Otmal Clafer Combnaton Functon Sergey Tulyakov Chaohong Wu Venu Govndaraju Unverty at Buffalo Identfcaton ytem: Alce Bob htt://www.cub.buffalo.edu
More informationAdditional File 1 - Detailed explanation of the expression level CPD
Addtonal Fle - Detaled explanaton of the expreon level CPD A mentoned n the man text, the man CPD for the uterng model cont of two ndvdual factor: P( level gen P( level gen P ( level gen 2 (.).. CPD factor
More information2.3 Least-Square regressions
.3 Leat-Square regreon Eample.10 How do chldren grow? The pattern of growth vare from chld to chld, o we can bet undertandng the general pattern b followng the average heght of a number of chldren. Here
More informationThe multivariate Gaussian probability density function for random vector X (X 1,,X ) T. diagonal term of, denoted
Appendx Proof of heorem he multvarate Gauan probablty denty functon for random vector X (X,,X ) px exp / / x x mean and varance equal to the th dagonal term of, denoted he margnal dtrbuton of X Gauan wth
More information+, where 0 x N - n. k k
CO 745, Mdterm Len Cabrera. A multle choce eam has questons, each of whch has ossble answers. A student nows the correct answer to n of these questons. For the remanng - n questons, he checs the answers
More informationEstimation of Finite Population Total under PPS Sampling in Presence of Extra Auxiliary Information
Internatonal Journal of Stattc and Analy. ISSN 2248-9959 Volume 6, Number 1 (2016), pp. 9-16 Reearch Inda Publcaton http://www.rpublcaton.com Etmaton of Fnte Populaton Total under PPS Samplng n Preence
More informationMULTIPLE REGRESSION ANALYSIS For the Case of Two Regressors
MULTIPLE REGRESSION ANALYSIS For the Cae of Two Regreor In the followng note, leat-quare etmaton developed for multple regreon problem wth two eplanator varable, here called regreor (uch a n the Fat Food
More informationRockefeller College University at Albany
Rockefeller College Unverst at Alban PAD 705 Handout: Maxmum Lkelhood Estmaton Orgnal b Davd A. Wse John F. Kenned School of Government, Harvard Unverst Modfcatons b R. Karl Rethemeer Up to ths pont n
More informationAdaptive Centering with Random Effects in Studies of Time-Varying Treatments. by Stephen W. Raudenbush University of Chicago.
Adaptve Centerng wth Random Effect n Stde of Tme-Varyng Treatment by Stephen W. Radenbh Unverty of Chcago Abtract Of wdepread nteret n ocal cence are obervatonal tde n whch entte (peron chool tate contre
More informationDesign of Recursive Digital Filters IIR
Degn of Recurve Dgtal Flter IIR The outut from a recurve dgtal flter deend on one or more revou outut value, a well a on nut t nvolve feedbac. A recurve flter ha an nfnte mule reone (IIR). The mulve reone
More informationStatistical Properties of the OLS Coefficient Estimators. 1. Introduction
ECOOMICS 35* -- OTE 4 ECO 35* -- OTE 4 Stattcal Properte of the OLS Coeffcent Etmator Introducton We derved n ote the OLS (Ordnary Leat Square etmator ˆβ j (j, of the regreon coeffcent βj (j, n the mple
More informationTeam. Outline. Statistics and Art: Sampling, Response Error, Mixed Models, Missing Data, and Inference
Team Stattc and Art: Samplng, Repone Error, Mxed Model, Mng Data, and nference Ed Stanek Unverty of Maachuett- Amhert, USA 9/5/8 9/5/8 Outlne. Example: Doe-repone Model n Toxcology. ow to Predct Realzed
More informationA REVIEW OF ERROR ANALYSIS
A REVIEW OF ERROR AALYI EEP Laborator EVE-4860 / MAE-4370 Updated 006 Error Analss In the laborator we measure phscal uanttes. All measurements are subject to some uncertantes. Error analss s the stud
More informationMachine learning: Density estimation
CS 70 Foundatons of AI Lecture 3 Machne learnng: ensty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square ata: ensty estmaton {.. n} x a vector of attrbute values Objectve: estmate the model of
More informationCHAPTER 9 LINEAR MOMENTUM, IMPULSE AND COLLISIONS
CHAPTER 9 LINEAR MOMENTUM, IMPULSE AND COLLISIONS 103 Phy 1 9.1 Lnear Momentum The prncple o energy conervaton can be ued to olve problem that are harder to olve jut ung Newton law. It ued to decrbe moton
More informationConfidence intervals for weighted polynomial calibrations
Confdence ntervals for weghted olynomal calbratons Sergey Maltsev, Amersand Ltd., Moscow, Russa; ur Kalambet, Amersand Internatonal, Inc., Beachwood, OH e-mal: kalambet@amersand-ntl.com htt://www.chromandsec.com
More informationSimple Linear Regression Analysis
LINEAR REGREION ANALYSIS MODULE II Lecture - 5 Smple Lear Regreo Aaly Dr Shalabh Departmet of Mathematc Stattc Ida Ittute of Techology Kapur Jot cofdece rego for A jot cofdece rego for ca alo be foud Such
More informationComplete Variance Decomposition Methods. Cédric J. Sallaberry
Comlete Varance Decomoston Methods Cédrc J. allaberry enstvty Analyss y y [,,, ] [ y, y,, ] y ny s a vector o uncertan nuts s a vector o results s a comle uncton successon o derent codes, systems o de,
More informationPattern Classification
attern Classfcaton All materals n these sldes were taken from attern Classfcaton nd ed by R. O. Duda,. E. Hart and D. G. Stork, John Wley & Sons, 000 wth the ermsson of the authors and the ublsher Chater
More informationModal Reconstruction Methods Pros and Cons. Jim Schwiegerling, Ph.D. Department of Ophthalmology and Optical Sciences The University of Arizona
odal Reconstructon ethods Pros and Cons Jm Schwegerlng Ph.D. Department of Ophthalmolog and Optcal Scences The Unverst of Arzona Introducton Elevaton Data Cornea Topograph (usuall Proflometr Interferometr
More informationCHAPTER-5 INFORMATION MEASURE OF FUZZY MATRIX AND FUZZY BINARY RELATION
CAPTER- INFORMATION MEASURE OF FUZZY MATRI AN FUZZY BINARY RELATION Introducton The basc concept of the fuzz matr theor s ver smple and can be appled to socal and natural stuatons A branch of fuzz matr
More informationChapter 6 The Effect of the GPS Systematic Errors on Deformation Parameters
Chapter 6 The Effect of the GPS Sytematc Error on Deformaton Parameter 6.. General Beutler et al., (988) dd the frt comprehenve tudy on the GPS ytematc error. Baed on a geometrc approach and aumng a unform
More informationChapter 11. Supplemental Text Material. The method of steepest ascent can be derived as follows. Suppose that we have fit a firstorder
S-. The Method of Steepet cent Chapter. Supplemental Text Materal The method of teepet acent can be derved a follow. Suppoe that we have ft a frtorder model y = β + β x and we wh to ue th model to determne
More informationMixture of Gaussians Expectation Maximization (EM) Part 2
Mture of Gaussans Eectaton Mamaton EM Part 2 Most of the sldes are due to Chrstoher Bsho BCS Summer School Eeter 2003. The rest of the sldes are based on lecture notes by A. Ng Lmtatons of K-means Hard
More information11.5 MAP Estimator MAP avoids this Computational Problem!
.5 MAP timator ecall that the hit-or-mi cot function gave the MAP etimator it maimize the a oteriori PDF Q: Given that the MMS etimator i the mot natural one why would we conider the MAP etimator? A: If
More informationLinear Form of the Radiative Transfer Equation Revisited. Bormin Huang
Lnear Form of the Radate Tranfer Equaton Reted Bormn Huang Cooerate Inttute for Meteorologcal Satellte Stude, Sace Scence and Engneerng Center Unerty of Wconn Madon The 5 th Internatonal TOVS Study Conference
More informationMultiple Regression Analysis
Multle Regresson Analss Roland Szlág Ph.D. Assocate rofessor Correlaton descres the strength of a relatonsh, the degree to whch one varale s lnearl related to another Regresson shows us how to determne
More informationMIMA Group. Chapter 2 Bayesian Decision Theory. School of Computer Science and Technology, Shandong University. Xin-Shun SDU
Group M D L M Chapter Bayesan Decson heory Xn-Shun Xu @ SDU School of Computer Scence and echnology, Shandong Unversty Bayesan Decson heory Bayesan decson theory s a statstcal approach to data mnng/pattern
More informationLogistic Regression Maximum Likelihood Estimation
Harvard-MIT Dvson of Health Scences and Technology HST.951J: Medcal Decson Support, Fall 2005 Instructors: Professor Lucla Ohno-Machado and Professor Staal Vnterbo 6.873/HST.951 Medcal Decson Support Fall
More informationThe Fundamental Theorem of Algebra. Objective To use the Fundamental Theorem of Algebra to solve polynomial equations with complex solutions
5-6 The Fundamental Theorem of Algebra Content Standards N.CN.7 Solve quadratc equatons wth real coeffcents that have comple solutons. N.CN.8 Etend polnomal denttes to the comple numbers. Also N.CN.9,
More informationConfidence intervals for the difference and the ratio of Lognormal means with bounded parameters
Songklanakarn J. Sc. Technol. 37 () 3-40 Mar.-Apr. 05 http://www.jt.pu.ac.th Orgnal Artcle Confdence nterval for the dfference and the rato of Lognormal mean wth bounded parameter Sa-aat Nwtpong* Department
More information8/25/17. Data Modeling. Data Modeling. Data Modeling. Patrice Koehl Department of Biological Sciences National University of Singapore
8/5/17 Data Modelng Patrce Koehl Department of Bologcal Scences atonal Unversty of Sngapore http://www.cs.ucdavs.edu/~koehl/teachng/bl59 koehl@cs.ucdavs.edu Data Modelng Ø Data Modelng: least squares Ø
More informationSupervised Learning. Neural Networks and Back-Propagation Learning. Credit Assignment Problem. Feedforward Network. Adaptive System.
Part 7: Neura Networ & earnng /2/05 Superved earnng Neura Networ and Bac-Propagaton earnng Produce dered output for tranng nput Generaze reaonaby & appropratey to other nput Good exampe: pattern recognton
More informationImprovements on Waring s Problem
Improvement on Warng Problem L An-Png Bejng, PR Chna apl@nacom Abtract By a new recurve algorthm for the auxlary equaton, n th paper, we wll gve ome mprovement for Warng problem Keyword: Warng Problem,
More informationWeb-Mining Agents Probabilistic Information Retrieval
Web-Mnng Agents Probablstc Informaton etreval Prof. Dr. alf Möller Unverstät zu Lübeck Insttut für Informatonssysteme Karsten Martny Übungen Acknowledgements Sldes taken from: Introducton to Informaton
More informationStatistical analysis using matlab. HY 439 Presented by: George Fortetsanakis
Statstcal analyss usng matlab HY 439 Presented by: George Fortetsanaks Roadmap Probablty dstrbutons Statstcal estmaton Fttng data to probablty dstrbutons Contnuous dstrbutons Contnuous random varable X
More informationSmall signal analysis
Small gnal analy. ntroducton Let u conder the crcut hown n Fg., where the nonlnear retor decrbed by the equaton g v havng graphcal repreentaton hown n Fg.. ( G (t G v(t v Fg. Fg. a D current ource wherea
More informationHarmonic oscillator approximation
armonc ocllator approxmaton armonc ocllator approxmaton Euaton to be olved We are fndng a mnmum of the functon under the retrcton where W P, P,..., P, Q, Q,..., Q P, P,..., P, Q, Q,..., Q lnwgner functon
More informationOptimal inference of sameness Supporting information
Optmal nference of amene Supportng nformaton Content Decon rule of the optmal oberver.... Unequal relablte.... Equal relablte... 5 Repone probablte of the optmal oberver... 6. Equal relablte... 6. Unequal
More informationMATH 829: Introduction to Data Mining and Analysis The EM algorithm (part 2)
1/16 MATH 829: Introducton to Data Mnng and Analyss The EM algorthm (part 2) Domnque Gullot Departments of Mathematcal Scences Unversty of Delaware Aprl 20, 2016 Recall 2/16 We are gven ndependent observatons
More informationThe Essential Dynamics Algorithm: Essential Results
@ MIT maachuett nttute of technology artfcal ntellgence laboratory The Eental Dynamc Algorthm: Eental Reult Martn C. Martn AI Memo 003-014 May 003 003 maachuett nttute of technology, cambrdge, ma 0139
More informationa new crytoytem baed on the dea of Shmuley and roved t rovably ecure baed on ntractablty of factorng [Mc88] After that n 999 El Bham, Dan Boneh and Om
Weak Comote Dffe-Hellman not Weaker than Factorng Koohar Azman, azman@ceharfedu Javad Mohajer mohajer@harfedu Mahmoud Salmazadeh alma@harfedu Electronc Reearch Centre, Sharf Unverty of Technology Deartment
More information3 Implementation and validation of analysis methods
3 Implementaton and valdaton of anal method 3. Preface When mplementng new method bacall three cae can be dfferentated: - Implementaton of offcal method (nternatonall approved, valdated method, e.g. method
More informationCS 2750 Machine Learning. Lecture 5. Density estimation. CS 2750 Machine Learning. Announcements
CS 750 Machne Learnng Lecture 5 Densty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square CS 750 Machne Learnng Announcements Homework Due on Wednesday before the class Reports: hand n before
More informationWhy Monte Carlo Integration? Introduction to Monte Carlo Method. Continuous Probability. Continuous Probability
Introducton to Monte Carlo Method Kad Bouatouch IRISA Emal: kad@rsa.fr Wh Monte Carlo Integraton? To generate realstc lookng mages, we need to solve ntegrals of or hgher dmenson Pel flterng and lens smulaton
More informationSTK3100 and STK4100 Autumn 2018
SK3 ad SK4 Autum 8 Geeralzed lear models Part III Covers the followg materal from chaters 4 ad 5: Cofdece tervals by vertg tests Cosder a model wth a sgle arameter β We may obta a ( α% cofdece terval for
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analyss of Varance and Desgn of Exerments-I MODULE III LECTURE - 2 EXPERIMENTAL DESIGN MODELS Dr. Shalabh Deartment of Mathematcs and Statstcs Indan Insttute of Technology Kanur 2 We consder the models
More informationDiagnostics in Poisson Regression. Models - Residual Analysis
Dagnostcs n Posson Regresson Models - Resdual Analyss 1 Outlne Dagnostcs n Posson Regresson Models - Resdual Analyss Example 3: Recall of Stressful Events contnued 2 Resdual Analyss Resduals represent
More informationPhysicsAndMathsTutor.com
PhscsAndMathsTutor.com phscsandmathstutor.com June 005 5. The random varable X has probablt functon k, = 1,, 3, P( X = ) = k ( + 1), = 4, 5, where k s a constant. (a) Fnd the value of k. (b) Fnd the eact
More informationCIS526: Machine Learning Lecture 3 (Sept 16, 2003) Linear Regression. Preparation help: Xiaoying Huang. x 1 θ 1 output... θ M x M
CIS56: achne Learnng Lecture 3 (Sept 6, 003) Preparaton help: Xaoyng Huang Lnear Regresson Lnear regresson can be represented by a functonal form: f(; θ) = θ 0 0 +θ + + θ = θ = 0 ote: 0 s a dummy attrbute
More informationMaximum Likelihood Estimation of Binary Dependent Variables Models: Probit and Logit. 1. General Formulation of Binary Dependent Variables Models
ECO 452 -- OE 4: Probt and Logt Models ECO 452 -- OE 4 Mamum Lkelhood Estmaton of Bnary Dependent Varables Models: Probt and Logt hs note demonstrates how to formulate bnary dependent varables models for
More informationMachine Learning for Signal Processing Linear Gaussian Models
Machne Learnng for Sgnal Processng Lnear Gaussan Models Class 7. 30 Oct 204 Instructor: Bhksha Raj 755/8797 Recap: MAP stmators MAP (Mamum A Posteror: Fnd a best guess for (statstcall, gven knon = argma
More informationLogistic regression with one predictor. STK4900/ Lecture 7. Program
Logstc regresson wth one redctor STK49/99 - Lecture 7 Program. Logstc regresson wth one redctor 2. Maxmum lkelhood estmaton 3. Logstc regresson wth several redctors 4. Devance and lkelhood rato tests 5.
More informationReview: Fit a line to N data points
Revew: Ft a lne to data ponts Correlated parameters: L y = a x + b Orthogonal parameters: J y = a (x ˆ x + b For ntercept b, set a=0 and fnd b by optmal average: ˆ b = y, Var[ b ˆ ] = For slope a, set
More information7. Algorithms for Massive Data Problems
July 4, 009 7 Mave Data Samlng on the fly 7 Algorthm for Mave Data Problem Mave Data, Samlng h chater deal wth mave data roblem where the nut data (a grah, a matrx or ome other object) too large to be
More informationA total variation approach
Denosng n dgtal radograhy: A total varaton aroach I. Froso M. Lucchese. A. Borghese htt://as-lab.ds.unm.t / 46 I. Froso, M. Lucchese,. A. Borghese Images are corruted by nose ) When measurement of some
More informationApproximate Inference: Mean Field Methods
School of Comuter Scence Aromate Inference: Mean Feld Methods Probablstc Grahcal Models 10-708 Lecture 17 Nov 12 2007 Recetor A Knase C Gene G Recetor B X 1 X 2 Knase D Knase X 3 X 4 X 5 TF F X 6 Gene
More informationLecture 10 Support Vector Machines. Oct
Lecture 10 Support Vector Machnes Oct - 20-2008 Lnear Separators Whch of the lnear separators s optmal? Concept of Margn Recall that n Perceptron, we learned that the convergence rate of the Perceptron
More informationGenerative classification models
CS 675 Intro to Machne Learnng Lecture Generatve classfcaton models Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Data: D { d, d,.., dn} d, Classfcaton represents a dscrete class value Goal: learn
More informationSTK3100 and STK4100 Autumn 2017
SK3 ad SK4 Autum 7 Geeralzed lear models Part III Covers the followg materal from chaters 4 ad 5: Sectos 4..5, 4.3.5, 4.3.6, 4.4., 4.4., ad 4.4.3 Sectos 5.., 5.., ad 5.5. Ørulf Borga Deartmet of Mathematcs
More informationThe conjugate prior to a Bernoulli is. A) Bernoulli B) Gaussian C) Beta D) none of the above
The conjugate pror to a Bernoull s A) Bernoull B) Gaussan C) Beta D) none of the above The conjugate pror to a Gaussan s A) Bernoull B) Gaussan C) Beta D) none of the above MAP estmates A) argmax θ p(θ
More informationPattern Classification (II) 杜俊
attern lassfcaton II 杜俊 junu@ustc.eu.cn Revew roalty & Statstcs Bayes theorem Ranom varales: screte vs. contnuous roalty struton: DF an DF Statstcs: mean, varance, moment arameter estmaton: MLE Informaton
More informationPredictive Analytics : QM901.1x Prof U Dinesh Kumar, IIMB. All Rights Reserved, Indian Institute of Management Bangalore
Sesson Outlne Introducton to classfcaton problems and dscrete choce models. Introducton to Logstcs Regresson. Logstc functon and Logt functon. Maxmum Lkelhood Estmator (MLE) for estmaton of LR parameters.
More informationIndependent Component Analysis
Indeendent Comonent Analyss Mture Data Data that are mngled from multle sources May not now how many sources May not now the mng mechansm Good Reresentaton Uncorrelated, nformaton-bearng comonents PCA
More informationFinite Mixture Models and Expectation Maximization. Most slides are from: Dr. Mario Figueiredo, Dr. Anil Jain and Dr. Rong Jin
Fnte Mxture Models and Expectaton Maxmzaton Most sldes are from: Dr. Maro Fgueredo, Dr. Anl Jan and Dr. Rong Jn Recall: The Supervsed Learnng Problem Gven a set of n samples X {(x, y )},,,n Chapter 3 of
More informationMachine Learning for Signal Processing Linear Gaussian Models
Machne Learnng for Sgnal rocessng Lnear Gaussan Models lass 2. 2 Nov 203 Instructor: Bhsha Raj 2 Nov 203 755/8797 HW3 s up. Admnstrva rojects please send us an update 2 Nov 203 755/8797 2 Recap: MA stmators
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analyss of Varance and Desgn of Experment-I MODULE VII LECTURE - 3 ANALYSIS OF COVARIANCE Dr Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur Any scentfc experment s performed
More informationError Bars in both X and Y
Error Bars n both X and Y Wrong ways to ft a lne : 1. y(x) a x +b (σ x 0). x(y) c y + d (σ y 0) 3. splt dfference between 1 and. Example: Prmordal He abundance: Extrapolate ft lne to [ O / H ] 0. [ He
More informationLecture outline. Optimal Experimental Design: Where to find basic information. Theory of D-optimal design
v I N N O V A T I O N L E C T U R E (I N N O l E C) Lecture outlne Bndng and Knetc for Expermental Bologt Lecture 8 Optmal degn of experment The problem: How hould we plan an experment uch we learn the
More informationMaximum Likelihood Estimation of Binary Dependent Variables Models: Probit and Logit. 1. General Formulation of Binary Dependent Variables Models
ECO 452 -- OE 4: Probt and Logt Models ECO 452 -- OE 4 Maxmum Lkelhood Estmaton of Bnary Dependent Varables Models: Probt and Logt hs note demonstrates how to formulate bnary dependent varables models
More informationLecture 20: Hypothesis testing
Lecture : Hpothess testng Much of statstcs nvolves hpothess testng compare a new nterestng hpothess, H (the Alternatve hpothess to the borng, old, well-known case, H (the Null Hpothess or, decde whether
More informationANSWERS. Problem 1. and the moment generating function (mgf) by. defined for any real t. Use this to show that E( U) var( U)
Econ 413 Exam 13 H ANSWERS Settet er nndelt 9 deloppgaver, A,B,C, som alle anbefales å telle lkt for å gøre det ltt lettere å stå. Svar er gtt . Unfortunately, there s a prntng error n the hnt of
More informationA Kernel Particle Filter Algorithm for Joint Tracking and Classification
A Kernel Partcle Flter Algorthm for Jont Tracng and Clafcaton Yunfe Guo Donglang Peng Inttute of Informaton and Control Automaton School Hangzhou Danz Unverty Hangzhou Chna gyf@hdueducn Huaje Chen Ane
More informationMeasurement Indices of Positional Uncertainty for Plane Line Segments Based on the ε
Proceedngs of the 8th Internatonal Smposum on Spatal ccurac ssessment n Natural Resources and Envronmental Scences Shangha, P R Chna, June 5-7, 008, pp 9-5 Measurement Indces of Postonal Uncertant for
More informationDiscriminative classifier: Logistic Regression. CS534-Machine Learning
Dscrmnatve classfer: Logstc Regresson CS534-Machne Learnng 2 Logstc Regresson Gven tranng set D stc regresson learns the condtonal dstrbuton We ll assume onl to classes and a parametrc form for here s
More informationA Tutorial on Data Reduction. Linear Discriminant Analysis (LDA) Shireen Elhabian and Aly A. Farag. University of Louisville, CVIP Lab September 2009
A utoral on Data Reducton Lnear Dscrmnant Analss (LDA) hreen Elhaban and Al A Farag Unverst of Lousvlle, CVIP Lab eptember 009 Outlne LDA objectve Recall PCA No LDA LDA o Classes Counter eample LDA C Classes
More informationThe Gaussian classifier. Nuno Vasconcelos ECE Department, UCSD
he Gaussan classfer Nuno Vasconcelos ECE Department, UCSD Bayesan decson theory recall that we have state of the world X observatons g decson functon L[g,y] loss of predctng y wth g Bayes decson rule s
More informationBatch Reinforcement Learning
Batch Renforcement Learnng Alan Fern * Baed n part on lde by Ronald Parr Overvew What batch renforcement learnng? Leat Square Polcy Iteraton Ftted Q-teraton Batch DQN Onlne veru Batch RL Onlne RL: ntegrate
More informationBIO Lab 2: TWO-LEVEL NORMAL MODELS with school children popularity data
Lab : TWO-LEVEL NORMAL MODELS wth school chldren popularty data Purpose: Introduce basc two-level models for normally dstrbuted responses usng STATA. In partcular, we dscuss Random ntercept models wthout
More information15-381: Artificial Intelligence. Regression and cross validation
15-381: Artfcal Intellgence Regresson and cross valdaton Where e are Inputs Densty Estmator Probablty Inputs Classfer Predct category Inputs Regressor Predct real no. Today Lnear regresson Gven an nput
More informationLinear Feature Engineering 11
Lnear Feature Engneerng 11 2 Least-Squares 2.1 Smple least-squares Consder the followng dataset. We have a bunch of nputs x and correspondng outputs y. The partcular values n ths dataset are x y 0.23 0.19
More informationThe Bellman Equation
The Bellman Eqaton Reza Shadmehr In ths docment I wll rovde an elanaton of the Bellman eqaton, whch s a method for otmzng a cost fncton and arrvng at a control olcy.. Eamle of a game Sose that or states
More informationBOOTSTRAP METHOD FOR TESTING OF EQUALITY OF SEVERAL MEANS. M. Krishna Reddy, B. Naveen Kumar and Y. Ramu
BOOTSTRAP METHOD FOR TESTING OF EQUALITY OF SEVERAL MEANS M. Krshna Reddy, B. Naveen Kumar and Y. Ramu Department of Statstcs, Osmana Unversty, Hyderabad -500 007, Inda. nanbyrozu@gmal.com, ramu0@gmal.com
More informationStatistical inference for generalized Pareto distribution based on progressive Type-II censored data with random removals
Internatonal Journal of Scentfc World, 2 1) 2014) 1-9 c Scence Publshng Corporaton www.scencepubco.com/ndex.php/ijsw do: 10.14419/jsw.v21.1780 Research Paper Statstcal nference for generalzed Pareto dstrbuton
More informationBayesian Variable Selection and Computation for Generalized Linear Models with Conjugate Priors
Bayean Analy (2008 3, Number 3, pp. 585 614 Bayean Varable Selecton and Computaton for Generalzed Lnear Model wth Conjugate Pror Mng-Hu Chen, Lan Huang, Joeph G. Ibrahm and Sungduk Km Abtract. In th paper,
More informationGeneralized Linear Methods
Generalzed Lnear Methods 1 Introducton In the Ensemble Methods the general dea s that usng a combnaton of several weak learner one could make a better learner. More formally, assume that we have a set
More informationLesson 16: Basic Control Modes
0/8/05 Lesson 6: Basc Control Modes ET 438a Automatc Control Systems Technology lesson6et438a.tx Learnng Objectves Ater ths resentaton you wll be able to: Descrbe the common control modes used n analog
More informatione i is a random error
Chapter - The Smple Lnear Regresson Model The lnear regresson equaton s: where + β + β e for,..., and are observable varables e s a random error How can an estmaton rule be constructed for the unknown
More informationEngineering Risk Benefit Analysis
Engneerng Rsk Beneft Analyss.55, 2.943, 3.577, 6.938, 0.86, 3.62, 6.862, 22.82, ESD.72, ESD.72 RPRA 2. Elements of Probablty Theory George E. Apostolaks Massachusetts Insttute of Technology Sprng 2007
More informationStatistics MINITAB - Lab 2
Statstcs 20080 MINITAB - Lab 2 1. Smple Lnear Regresson In smple lnear regresson we attempt to model a lnear relatonshp between two varables wth a straght lne and make statstcal nferences concernng that
More informationFuzzy approach to solve multi-objective capacitated transportation problem
Internatonal Journal of Bonformatcs Research, ISSN: 0975 087, Volume, Issue, 00, -0-4 Fuzzy aroach to solve mult-objectve caactated transortaton roblem Lohgaonkar M. H. and Bajaj V. H.* * Deartment of
More informationLINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity
LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 31 Multcollnearty Dr. Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur 6. Rdge regresson The OLSE s the best lnear unbased
More informationLinear Approximation with Regularization and Moving Least Squares
Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...
More informationJoint Source Coding and Higher-Dimension Modulation
Jont Codng and Hgher-Dmenon Modulaton Tze C. Wong and Huck M. Kwon Electrcal Engneerng and Computer Scence Wchta State Unvert, Wchta, Kana 676, USA {tcwong; huck.kwon}@wchta.edu Abtract Th paper propoe
More information( ) [ ( k) ( k) ( x) ( ) ( ) ( ) [ ] ξ [ ] [ ] [ ] ( )( ) i ( ) ( )( ) 2! ( ) = ( ) 3 Interpolation. Polynomial Approximation.
3 Interpolaton {( y } Gven:,,,,,, [ ] Fnd: y for some Mn, Ma Polynomal Appromaton Theorem (Weerstrass Appromaton Theorem --- estence ε [ ab] f( P( , then there ests a polynomal
More informationDevelopment of Pedotransfer Functions for Saturated Hydraulic Conductivity
Open Journal of Modern Hydrology, 03, 3, 54-64 http://dx.do.org/0.436/ojmh.03.3309 Publhed Onlne July 03 (http://www.crp.org/journal/ojmh) Development of Pedotranfer Functon for Saturated Hydraulc Conductvty
More information