Lets start off with a visual intuition
|
|
|
- Percival Barrett
- 6 years ago
- Views:
Transcription
1 Naïve Bayes Classifier (pages on text book) Lets start off with a visual intuition Adapted from Dr. Eamonn Keogh s lecture UCR 1
2 Body length Data 2 Alligators Mouth size Crocodiles
3 Body length Data 3 Suppose we had a lot of data. We could use that data to build a histogram. Below is one built for the body length feature: Crocodiles Alligators Mouth size
4 We can summarize these histograms as two normal distributions. 4
5 Suppose we wish to classify a new animal that we just found. Its body length is 3 units. How can we classify it? One way to do this is, given the distributions of that feature, we can analyze which class is more probable: Crocodile or Alligator. More formally: p c j d = probability of class c j, given tat we observed d 5 3
6 p c j d = probability of class c j, given tat we observed d p Alligator body lengt = 3 p Crocodile body lengt = 3 = 10 (10 + 2) = = 2 (10 + 2) = Body length =
7 p c j d = probability of class c j, given tat we observed d p Alligator body lengt = 7 p Crocodile body lengt = 7 = 3 (3 + 9) = = 9 (3 + 9) = Body length =
8 p c j d = probability of class c j, given tat we observed d p Alligator body lengt = 5 p Crocodile body lengt = 5 = 6 (6 + 6) = 0. 5 = 6 (6 + 6) = Body length =
9 Naïve Bayes Classifier This visual intuition describes a simple Bayes classifier commonly known as: Naïve Bayes Simple Bayes Idiot Bayes While going through the math, keep in mind the basic idea: Given a new unseen instance, we (1) find its probability of it belonging to each class, and (2) pick the most probable. 9
10 P c j d Posterior probability Bayes Theorem Likelihood P c j d = P d c j P c j P(d) = probability of instance d being in class c j Class prior probability Predictor prior probability P d c j = probability of generating instance d given class c j P(c j ) = probability of occurrence of class c j P(d) = probability of instance d occurring 10
11 Suppose we have another binary classification problem with the following two classes: c 1 = male, and c 2 = female We now have a person called Morgan. How do we classify them as male or female? Morgan Fairchild What is the probability of being called Morgan given that you are a male? What is the probability of being a male? P male morgan = P morgan male P male P(morgan) Morgan Freeman 11 What is the probability of being called Morgan
12 Suppose this individual on your left (Morgan) was arrested for money laundering. Is Morgan male or female? Assume we are given the following database of names. We can then apply Bayes rule. 12 Name Morgan Reid Morgan Morgan Everaldo Francis Jennifer Sex Male Female Male Female Male Male Female
13 P c j d = P d c j P c j P(d) Name Morgan Reid Morgan Morgan Everaldo Francis Jennifer Sex Male Female Male Female Male Male Female P female morgan = /8 P male morgan = /8 = /8 = /8 Money launderer Morgan is more likely to be male. 13
14 Naïve Bayes Classifier Both examples that we looked at considered only a single feature (i.e., body length and name) What if we have several features? Name Over 6ft Eye color Hair style Sex 14 Morgan Yes Blue Long Female Bob No Brown None Male Vincent Yes Brown Short Male Amanda No Brown Short Female Reid No Blue Short Male Lauren No Purple Long Female Elisa Yes Brown Long Female
15 Naïve Bayes Classifier Naïve Bayes assumes that all features are independent (i.e., they have independent distributions). The probability of class c j generating instance d can then be estimated as: P d c j = P d 1 c j P d 2 c j P(d n c j ) 15 Probability of class c j generating the observed value for feature 1 Probability of class c j generating the observed value for feature 2
16 Naïve Bayes Classifier Suppose we have Amanda s data: Name Over 6ft Eye color Hair style Sex Amanda No Brown Short? P Amanda c j = P over6ft = No c j P eyecolor = Brown c j P(air = Sort c j ) 16
17 A visual representation: Naïve Bayes Classifier Each class implies certain features with certain probabilities p(d 1 c j ) c j p(d 3 c j ) All tables can be computed with a single scan of our dataset: Very efficient! 17 Sex Male Female Over 6ft Yes 0.8 No 0.2 Yes 0.01 No 0.99 Sex Male Female p(d 2 c j ) Eye color Brown 0.67 Blue 0.33 Brown 0.45 Blue 0.55 Sex Male Female Hair None 0.1 Short 0.8 Long 0.1 None 0.05 Short 0.15 Long 0.8
18 Naïve Bayes Classifier Naïve Bayes is not sensitive to irrelevant features. For instance, if we were trying to classify a person s sex based on a set of features that included eye color (which is irrelevant to their gender): P Reid c j = P eye = blue c j P wears makeup = Yes c j P Reid Female = 9,000/10,000 9,500/10,000 P Reid Male = 9,007/10,000 1/10,000 Nearly identical! The more data, the better! 18
19 Naïve Bayes Classifier Naïve Bayes works with any number of classes or features: c j p(d 1 c j ) Sex Crocodile Alligator T-Rex Friendly Yes 0.4 No 0.6 Yes 0.3 No 0.7 Yes 0.05 No 0.95 p(d 2 c j ) p(d 3 c j ) Sex Crocodile Alligator Color Black 0.1 Brown 0.2 Green 0.7 Black 0.2 Brown 0.3 Green 0.5 Black T-Rex Brown 0.2 Green 0.6
20 Naïve Bayes Classifier - Problems Naïve Bayes assumes all features are independent (often bad!) c j 20 Sex Crocodile Alligator T-Rex Friendly Yes 0.4 No 0.6 Yes 0.3 No 0.7 Yes 0.05 No 0.95 p(d 1 c j ) p(d 2 c j ) p(d 3 c j ) Sex Crocodile Alligator T-Rex Color Black 0.1 Brown 0.2 Green 0.7 Black 0.2 Brown 0.3 Green 0.5 Black 0.2 Brown 0.2 Green 0.6
21 Naïve Bayes Classifier - Problems What if one of the probabilities is zero? Ops. P d c j = P d 1 c j P d 2 c j P d n c j = This is not uncommon, especially when the number of instances is small Solution: m-estimate 21
22 m-estimate To avoid trouble when a probability P d 1 c j = 0, we fix its prior probability and the number of samples to some non-zero value beforehand Think of it as adding a bunch of fake instances before we start the whole process If we create m > 0 fake samples of feature X with value of x, and we assign a prior probability p to them, then posterior probabilities are obtained as: P X = x c j = # X=x,c j +mp #(c j )+m 22
23 Naïve Bayes Pros Can be used for both continuous and discrete features. Can be used for binary or multi-class problems. It is robust to isolated noise/outliers. Missing values are easily handled. They are simply ignored. It is robust to irrelevant attributes. 23
24 Naïve Bayes Cons Correlated features degrade performance. If we have the same feature multiple times in the dataset, we can change classification results. When m-estimate is used, the choice of m s and p s is very arbitrary. 24
25 Naïve Bayes Class exercise Predict if Bob will default his loan 25 Bob Home owner: No Marital status: Married Job experience: 3 Home owner Marital Status Job experience (1-5) Defaulted Yes Single 3 No No Married 4 No No Single 5 No Yes Married 4 No No Divorced 2 Yes No Married 4 No Yes Divorced 2 No No Married 3 Yes No Married 3 No Yes Single 2 Yes
26 Naïve Bayes Class exercise (1) Bob Home owner: No Marital status: Married Job experience: 3 P Y = No = 7/10 P Home owner = No Y = No = 4 7 P Marital status = Married Y = No = 4 7 P Job experience = 3 Y = No = 2 7 Home owner Marital Status Job experience (1-5) Defaulted Yes Single 3 No No Married 4 No No Single 5 No Yes Married 4 No No Divorced 2 Yes No Married 4 No Yes Divorced 2 No No Married 3 Yes No Married 3 No Yes Single 2 Yes 26 P Bob will NOT default = =
27 Naïve Bayes Class exercise (2) Bob Home owner: No Marital status: Married Job experience: 3 P Y = Yes = 3/10 P Home owner = No Y = Yes = 1 3 P Marital status = Married Y = Yes = 1 3 P Job experience = 3 Y = Yes = 1 3 Home owner Marital Status Job experience (1-5) Defaulted Yes Single 3 No No Married 4 No No Single 5 No Yes Married 4 No No Divorced 2 Yes No Married 4 No Yes Divorced 2 No No Married 3 Yes No Married 3 No Yes Single 2 Yes 27 P Bob will default = =
28 Naïve Bayes Class exercise (3) Bob Home owner: No Marital status: Married Job experience: 3 P Bob will NOT default P Bob will default = = Predict: BOB WILL NOT DEFAULT Home owner Marital Status Job experience (1-5) Defaulted Yes Single 3 No No Married 4 No No Single 5 No Yes Married 4 No No Divorced 2 Yes No Married 4 No Yes Divorced 2 No No Married 3 Yes No Married 3 No Yes Single 2 Yes 28
Uncertainty. Yeni Herdiyeni Departemen Ilmu Komputer IPB. The World is very Uncertain Place
Uncertainty Yeni Herdiyeni Departemen Ilmu Komputer IPB The World is very Uncertain Place 1 Ketidakpastian Presiden Indonesia tahun 2014 adalah Perempuan Tahun depan saya akan lulus Jumlah penduduk Indonesia
DATA MINING: NAÏVE BAYES
DATA MINING: NAÏVE BAYES 1 Naïve Bayes Classifier Thomas Bayes 1702-1761 We will start off with some mathematical background. But first we start with some visual intuition. 2 Grasshoppers Antenna Length
Lecture VII: Classification I. Dr. Ouiem Bchir
Lecture VII: Classification I Dr. Ouiem Bchir 1 Classification: Definition Given a collection of records (training set ) Each record contains a set of attributes, one of the attributes is the class. Find
Data Mining Classification: Basic Concepts and Techniques. Lecture Notes for Chapter 3. Introduction to Data Mining, 2nd Edition
Data Mining Classification: Basic Concepts and Techniques Lecture Notes for Chapter 3 by Tan, Steinbach, Karpatne, Kumar 1 Classification: Definition Given a collection of records (training set ) Each
Behavioral Data Mining. Lecture 2
Behavioral Data Mining Lecture 2 Autonomy Corp Bayes Theorem Bayes Theorem P(A B) = probability of A given that B is true. P(A B) = P(B A)P(A) P(B) In practice we are most interested in dealing with events
The Naïve Bayes Classifier. Machine Learning Fall 2017
The Naïve Bayes Classifier Machine Learning Fall 2017 1 Today s lecture The naïve Bayes Classifier Learning the naïve Bayes Classifier Practical concerns 2 Today s lecture The naïve Bayes Classifier Learning
Econometrics I Lecture 7: Dummy Variables
Econometrics I Lecture 7: Dummy Variables Mohammad Vesal Graduate School of Management and Economics Sharif University of Technology 44716 Fall 1397 1 / 27 Introduction Dummy variable: d i is a dummy variable
Gaussian and Linear Discriminant Analysis; Multiclass Classification
Gaussian and Linear Discriminant Analysis; Multiclass Classification Professor Ameet Talwalkar Slide Credit: Professor Fei Sha Professor Ameet Talwalkar CS260 Machine Learning Algorithms October 13, 2015
Learning from Data 1 Naive Bayes
Learning from Data 1 Naive Bayes Copyright David Barber 2001-2004. Course lecturer: Amos Storkey a.storkey@ed.ac.uk Course page : http://www.anc.ed.ac.uk/ amos/lfd/ 1 2 1 Why Naive Bayes? Naive Bayes is
Notes on Discriminant Functions and Optimal Classification
Notes on Discriminant Functions and Optimal Classification Padhraic Smyth, Department of Computer Science University of California, Irvine c 2017 1 Discriminant Functions Consider a classification problem
What does Bayes theorem give us? Lets revisit the ball in the box example.
ECE 6430 Pattern Recognition and Analysis Fall 2011 Lecture Notes - 2 What does Bayes theorem give us? Lets revisit the ball in the box example. Figure 1: Boxes with colored balls Last class we answered
Introduction to Machine Learning CMU-10701
Introduction to Machine Learning CMU-10701 23. Decision Trees Barnabás Póczos Contents Decision Trees: Definition + Motivation Algorithm for Learning Decision Trees Entropy, Mutual Information, Information
DATA MINING LECTURE 10
DATA MINING LECTURE 10 Classification Nearest Neighbor Classification Support Vector Machines Logistic Regression Naïve Bayes Classifier Supervised Learning 10 10 Illustrating Classification Task Tid Attrib1
MATH 2560 C F03 Elementary Statistics I Lecture 1: Displaying Distributions with Graphs. Outline.
MATH 2560 C F03 Elementary Statistics I Lecture 1: Displaying Distributions with Graphs. Outline. data; variables: categorical & quantitative; distributions; bar graphs & pie charts: What Is Statistics?
Decision Tree And Random Forest
Decision Tree And Random Forest Dr. Ammar Mohammed Associate Professor of Computer Science ISSR, Cairo University PhD of CS ( Uni. Koblenz-Landau, Germany) Spring 2019 Contact: mailto: Ammar@cu.edu.eg
Statistical Programming with R
Statistical Programming with R Lecture 3: Matrices Bisher M. Iqelan biqelan@iugaza.edu.ps Department of Mathematics, Faculty of Science, The Islamic University of Gaza 2017-2018, Semester 1 Matrices A
Advanced classifica-on methods
Advanced classifica-on methods Instance-based classifica-on Bayesian classifica-on Instance-Based Classifiers Set of Stored Cases Atr1... AtrN Class A B B C A C B Store the training records Use training
Bayesian Learning. Artificial Intelligence Programming. 15-0: Learning vs. Deduction
15-0: Learning vs. Deduction Artificial Intelligence Programming Bayesian Learning Chris Brooks Department of Computer Science University of San Francisco So far, we ve seen two types of reasoning: Deductive
Probabilistic modeling. The slides are closely adapted from Subhransu Maji s slides
Probabilistic modeling The slides are closely adapted from Subhransu Maji s slides Overview So far the models and algorithms you have learned about are relatively disconnected Probabilistic modeling framework
Latent Variable Models
Latent Variable Models Stefano Ermon, Aditya Grover Stanford University Lecture 5 Stefano Ermon, Aditya Grover (AI Lab) Deep Generative Models Lecture 5 1 / 31 Recap of last lecture 1 Autoregressive models:
15-381: Artificial Intelligence. Decision trees
15-381: Artificial Intelligence Decision trees Bayes classifiers find the label that maximizes: Naïve Bayes models assume independence of the features given the label leading to the following over documents
Examination Artificial Intelligence Module Intelligent Interaction Design December 2014
Examination Artificial Intelligence Module Intelligent Interaction Design December 2014 Introduction This exam is closed book, you may only use a simple calculator (addition, substraction, multiplication
Machine Learning for Signal Processing Bayes Classification and Regression
Machine Learning for Signal Processing Bayes Classification and Regression Instructor: Bhiksha Raj 11755/18797 1 Recap: KNN A very effective and simple way of performing classification Simple model: For
MATH 10 INTRODUCTORY STATISTICS
MATH 10 INTRODUCTORY STATISTICS Tommy Khoo Your friendly neighbourhood graduate student. Week 1 Chapter 1 Introduction What is Statistics? Why do you need to know Statistics? Technical lingo and concepts:
Lecture 28 Chi-Square Analysis
Lecture 28 STAT 225 Introduction to Probability Models April 23, 2014 Whitney Huang Purdue University 28.1 χ 2 test for For a given contingency table, we want to test if two have a relationship or not
Introduction to Machine Learning
Introduction to Machine Learning CS4375 --- Fall 2018 Bayesian a Learning Reading: Sections 13.1-13.6, 20.1-20.2, R&N Sections 6.1-6.3, 6.7, 6.9, Mitchell 1 Uncertainty Most real-world problems deal with
Introduction to Machine Learning
Uncertainty Introduction to Machine Learning CS4375 --- Fall 2018 a Bayesian Learning Reading: Sections 13.1-13.6, 20.1-20.2, R&N Sections 6.1-6.3, 6.7, 6.9, Mitchell Most real-world problems deal with
MATH 1150 Chapter 2 Notation and Terminology
MATH 1150 Chapter 2 Notation and Terminology Categorical Data The following is a dataset for 30 randomly selected adults in the U.S., showing the values of two categorical variables: whether or not the
Lecture 7 Decision Tree Classifier
Machine Learning Dr.Ammar Mohammed Lecture 7 Decision Tree Classifier Decision Tree A decision tree is a simple classifier in the form of a hierarchical tree structure, which performs supervised classification
Randomized Decision Trees
Randomized Decision Trees compiled by Alvin Wan from Professor Jitendra Malik s lecture Discrete Variables First, let us consider some terminology. We have primarily been dealing with real-valued data,
Chapter 2: Tools for Exploring Univariate Data
Stats 11 (Fall 2004) Lecture Note Introduction to Statistical Methods for Business and Economics Instructor: Hongquan Xu Chapter 2: Tools for Exploring Univariate Data Section 2.1: Introduction What is
CS6220: DATA MINING TECHNIQUES
CS6220: DATA MINING TECHNIQUES Matrix Data: Prediction Instructor: Yizhou Sun yzsun@ccs.neu.edu September 21, 2015 Announcements TA Monisha s office hour has changed to Thursdays 10-12pm, 462WVH (the same
Lecture 9: Naive Bayes, SVM, Kernels. Saravanan Thirumuruganathan
Lecture 9: Naive Bayes, SVM, Kernels Instructor: Outline 1 Probability basics 2 Probabilistic Interpretation of Classification 3 Bayesian Classifiers, Naive Bayes 4 Support Vector Machines Probability
1. True/False (40 points) total
Name: 1 2 3 4 5 6 7 total 40 20 45 45 20 15 15 200 UMBC CMSC 671 Final Exam December 20, 2009 Please write all of your answers on this exam. The exam is closed book and has seven problems that add up to
Bayesian Methods: Naïve Bayes
Bayesian Methods: aïve Bayes icholas Ruozzi University of Texas at Dallas based on the slides of Vibhav Gogate Last Time Parameter learning Learning the parameter of a simple coin flipping model Prior
Bayes Rule. CS789: Machine Learning and Neural Network Bayesian learning. A Side Note on Probability. What will we learn in this lecture?
Bayes Rule CS789: Machine Learning and Neural Network Bayesian learning P (Y X) = P (X Y )P (Y ) P (X) Jakramate Bootkrajang Department of Computer Science Chiang Mai University P (Y ): prior belief, prior
Bayesian Networks. Distinguished Prof. Dr. Panos M. Pardalos
Distinguished Prof. Dr. Panos M. Pardalos Center for Applied Optimization Department of Industrial & Systems Engineering Computer & Information Science & Engineering Department Biomedical Engineering Program,
Machine Learning 2nd Edition
INTRODUCTION TO Lecture Slides for Machine Learning 2nd Edition ETHEM ALPAYDIN, modified by Leonardo Bobadilla and some parts from http://www.cs.tau.ac.il/~apartzin/machinelearning/ The MIT Press, 2010
Chapter 26: Comparing Counts (Chi Square)
Chapter 6: Comparing Counts (Chi Square) We ve seen that you can turn a qualitative variable into a quantitative one (by counting the number of successes and failures), but that s a compromise it forces
Contingency Tables. Safety equipment in use Fatal Non-fatal Total. None 1, , ,128 Seat belt , ,878
Contingency Tables I. Definition & Examples. A) Contingency tables are tables where we are looking at two (or more - but we won t cover three or more way tables, it s way too complicated) factors, each
CLUe Training An Introduction to Machine Learning in R with an example from handwritten digit recognition
CLUe Training An Introduction to Machine Learning in R with an example from handwritten digit recognition Ad Feelders Universiteit Utrecht Department of Information and Computing Sciences Algorithmic Data
Data preprocessing. DataBase and Data Mining Group 1. Data set types. Tabular Data. Document Data. Transaction Data. Ordered Data
Elena Baralis and Tania Cerquitelli Politecnico di Torino Data set types Record Tables Document Data Transaction Data Graph World Wide Web Molecular Structures Ordered Spatial Data Temporal Data Sequential
Generative Learning. INFO-4604, Applied Machine Learning University of Colorado Boulder. November 29, 2018 Prof. Michael Paul
Generative Learning INFO-4604, Applied Machine Learning University of Colorado Boulder November 29, 2018 Prof. Michael Paul Generative vs Discriminative The classification algorithms we have seen so far
2011 Pearson Education, Inc
Statistics for Business and Economics Chapter 3 Probability Contents 1. Events, Sample Spaces, and Probability 2. Unions and Intersections 3. Complementary Events 4. The Additive Rule and Mutually Exclusive
Intuition Bayesian Classification
Intuition Bayesian Classification More ockey fans in Canada tan in US Wic country is Tom, a ockey ball fan, from? Predicting Canada as a better cance to be rigt Prior probability P(Canadian=5%: reflect
ECON Interactions and Dummies
ECON 351 - Interactions and Dummies Maggie Jones 1 / 25 Readings Chapter 6: Section on Models with Interaction Terms Chapter 7: Full Chapter 2 / 25 Interaction Terms with Continuous Variables In some regressions
Machine Learning. Naïve Bayes classifiers
10-701 Machine Learning Naïve Bayes classifiers Types of classifiers We can divide the large variety of classification approaches into three maor types 1. Instance based classifiers - Use observation directly
Lecture 9: Probabilistic IR The Binary Independence Model and Okapi BM25
Lecture 9: Probabilistic IR The Binary Independence Model and Okapi BM25 Trevor Cohn (Slide credits: William Webber) COMP90042, 2015, Semester 1 What we ll learn in this lecture Probabilistic models for
CS6220: DATA MINING TECHNIQUES
CS6220: DATA MINING TECHNIQUES Matrix Data: Prediction Instructor: Yizhou Sun yzsun@ccs.neu.edu September 14, 2014 Today s Schedule Course Project Introduction Linear Regression Model Decision Tree 2 Methods
CS 188: Artificial Intelligence. Outline
CS 188: Artificial Intelligence Lecture 21: Perceptrons Pieter Abbeel UC Berkeley Many slides adapted from Dan Klein. Outline Generative vs. Discriminative Binary Linear Classifiers Perceptron Multi-class
Generative classifiers: The Gaussian classifier. Ata Kaban School of Computer Science University of Birmingham
Generative classifiers: The Gaussian classifier Ata Kaban School of Computer Science University of Birmingham Outline We have already seen how Bayes rule can be turned into a classifier In all our examples
Research Methods in Political Science I
Research Methods in Political Science I 6. Linear Regression (1) Yuki Yanai School of Law and Graduate School of Law November 11, 2015 1 / 25 Today s Menu 1 Introduction What Is Linear Regression? Some
Machine Learning for natural language processing
Machine Learning for natural language processing Classification: Naive Bayes Laura Kallmeyer Heinrich-Heine-Universität Düsseldorf Summer 2016 1 / 20 Introduction Classification = supervised method for
COS 424: Interacting with Data. Lecturer: Dave Blei Lecture #11 Scribe: Andrew Ferguson March 13, 2007
COS 424: Interacting with ata Lecturer: ave Blei Lecture #11 Scribe: Andrew Ferguson March 13, 2007 1 Graphical Models Wrap-up We began the lecture with some final words on graphical models. Choosing a
Chapter 2 PROBABILITY SAMPLE SPACE
Chapter 2 PROBABILITY Key words: Sample space, sample point, tree diagram, events, complement, union and intersection of an event, mutually exclusive events; Counting techniques: multiplication rule, permutation,
3.2 Intoduction to probability 3.3 Probability rules. Sections 3.2 and 3.3. Elementary Statistics for the Biological and Life Sciences (Stat 205)
3.2 Intoduction to probability Sections 3.2 and 3.3 Elementary Statistics for the Biological and Life Sciences (Stat 205) 1 / 47 Probability 3.2 Intoduction to probability The probability of an event E
Bayesian Classification Methods
Bayesian Classification Methods Suchit Mehrotra North Carolina State University smehrot@ncsu.edu October 24, 2014 Suchit Mehrotra (NCSU) Bayesian Classification October 24, 2014 1 / 33 How do you define
Module 03 Lecture 14 Inferential Statistics ANOVA and TOI
Introduction of Data Analytics Prof. Nandan Sudarsanam and Prof. B Ravindran Department of Management Studies and Department of Computer Science and Engineering Indian Institute of Technology, Madras Module
Statistical Theory 1
Statistical Theory 1 Set Theory and Probability Paolo Bautista September 12, 2017 Set Theory We start by defining terms in Set Theory which will be used in the following sections. Definition 1 A set is
Data Mining Classification: Basic Concepts, Decision Trees, and Model Evaluation
Data Mining Classification: Basic Concepts, Decision Trees, and Model Evaluation Lecture Notes for Chapter 4 Part I Introduction to Data Mining by Tan, Steinbach, Kumar Adapted by Qiang Yang (2010) Tan,Steinbach,
Regression of Inflation on Percent M3 Change
ECON 497 Final Exam Page of ECON 497: Economic Research and Forecasting Name: Spring 2006 Bellas Final Exam Return this exam to me by midnight on Thursday, April 27. It may be e-mailed to me. It may be
Today. Statistical Learning. Coin Flip. Coin Flip. Experiment 1: Heads. Experiment 1: Heads. Which coin will I use? Which coin will I use?
Today Statistical Learning Parameter Estimation: Maximum Likelihood (ML) Maximum A Posteriori (MAP) Bayesian Continuous case Learning Parameters for a Bayesian Network Naive Bayes Maximum Likelihood estimates
Time Series Classification
Distance Measures Classifiers DTW vs. ED Further Work Questions August 31, 2017 Distance Measures Classifiers DTW vs. ED Further Work Questions Outline 1 2 Distance Measures 3 Classifiers 4 DTW vs. ED
7 Classification: Naïve Bayes Classifier
CSE4334/5334 Data Mining 7 Classifiation: Naïve Bayes Classifier Chengkai Li Department of Computer Siene and Engineering University of Texas at rlington Fall 017 Slides ourtesy of ang-ning Tan, Mihael
Data Mining and Machine Learning (Machine Learning: Symbolische Ansätze)
Data Mining and Machine Learning (Machine Learning: Symbolische Ansätze) Learning Individual Rules and Subgroup Discovery Introduction Batch Learning Terminology Coverage Spaces Descriptive vs. Predictive
Data Mining: Data. Lecture Notes for Chapter 2. Introduction to Data Mining
Data Mining: Data Lecture Notes for Chapter 2 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1 10 What is Data? Collection of data objects
Chapter 2. Reductions and NP. 2.1 Reductions Continued The Satisfiability Problem (SAT) SAT 3SAT. CS 573: Algorithms, Fall 2013 August 29, 2013
Chapter 2 Reductions and NP CS 573: Algorithms, Fall 2013 August 29, 2013 2.1 Reductions Continued 2.1.1 The Satisfiability Problem SAT 2.1.1.1 Propositional Formulas Definition 2.1.1. Consider a set of
Chapter 10: Chi-Square and F Distributions
Chapter 10: Chi-Square and F Distributions Chapter Notes 1 Chi-Square: Tests of Independence 2 4 & of Homogeneity 2 Chi-Square: Goodness of Fit 5 6 3 Testing & Estimating a Single Variance 7 10 or Standard
Lecture 7 Time-dependent Covariates in Cox Regression
Lecture 7 Time-dependent Covariates in Cox Regression So far, we ve been considering the following Cox PH model: λ(t Z) = λ 0 (t) exp(β Z) = λ 0 (t) exp( β j Z j ) where β j is the parameter for the the
CS4705. Probability Review and Naïve Bayes. Slides from Dragomir Radev
CS4705 Probability Review and Naïve Bayes Slides from Dragomir Radev Classification using a Generative Approach Previously on NLP discriminative models P C D here is a line with all the social media posts
Generative Model (Naïve Bayes, LDA)
Generative Model (Naïve Bayes, LDA) IST557 Data Mining: Techniques and Applications Jessie Li, Penn State University Materials from Prof. Jia Li, sta3s3cal learning book (Has3e et al.), and machine learning
An Introduction to Machine Learning
An Introduction to Machine Learning L2: Instance Based Estimation Alexander J. Smola Statistical Machine Learning Program Canberra, ACT 0200 Australia Alex.Smola@nicta.com.au Tata Institute, Pune, January
CS 188: Artificial Intelligence Fall 2008
CS 188: Artificial Intelligence Fall 2008 Lecture 23: Perceptrons 11/20/2008 Dan Klein UC Berkeley 1 General Naïve Bayes A general naive Bayes model: C E 1 E 2 E n We only specify how each feature depends
General Naïve Bayes. CS 188: Artificial Intelligence Fall Example: Overfitting. Example: OCR. Example: Spam Filtering. Example: Spam Filtering
CS 188: Artificial Intelligence Fall 2008 General Naïve Bayes A general naive Bayes model: C Lecture 23: Perceptrons 11/20/2008 E 1 E 2 E n Dan Klein UC Berkeley We only specify how each feature depends
ZERO INFLATED POISSON REGRESSION
STAT 6500 ZERO INFLATED POISSON REGRESSION FINAL PROJECT DEC 6 th, 2013 SUN JEON DEPARTMENT OF SOCIOLOGY UTAH STATE UNIVERSITY POISSON REGRESSION REVIEW INTRODUCING - ZERO-INFLATED POISSON REGRESSION SAS
EE (082) Ch. II: Intro. to Signals Lecture 2 Dr. Wajih Abu-Al-Saud
Classification of Signals Some important classifications of signals Analog vs. Digital signals: as stated in the previous lecture, a signal with a magnitude that may take any real value in a specific range
Notes for Math 324, Part 12
72 Notes for Math 324, Part 12 Chapter 12 Definition and main properties of probability 12.1 Sample space, events We run an experiment which can have several outcomes. The set consisting by all possible
9/12/17. Types of learning. Modeling data. Supervised learning: Classification. Supervised learning: Regression. Unsupervised learning: Clustering
Types of learning Modeling data Supervised: we know input and targets Goal is to learn a model that, given input data, accurately predicts target data Unsupervised: we know the input only and want to make
CSC 411: Lecture 09: Naive Bayes
CSC 411: Lecture 09: Naive Bayes Class based on Raquel Urtasun & Rich Zemel s lectures Sanja Fidler University of Toronto Feb 8, 2015 Urtasun, Zemel, Fidler (UofT) CSC 411: 09-Naive Bayes Feb 8, 2015 1
20 Unsupervised Learning and Principal Components Analysis (PCA)
116 Jonathan Richard Shewchuk 20 Unsupervised Learning and Principal Components Analysis (PCA) UNSUPERVISED LEARNING We have sample points, but no labels! No classes, no y-values, nothing to predict. Goal:
A first model of learning
A first model of learning Let s restrict our attention to binary classification our labels belong to (or ) We observe the data where each Suppose we are given an ensemble of possible hypotheses / classifiers
Introduction to Machine Learning Midterm Exam
10-701 Introduction to Machine Learning Midterm Exam Instructors: Eric Xing, Ziv Bar-Joseph 17 November, 2015 There are 11 questions, for a total of 100 points. This exam is open book, open notes, but
Machine Learning 4. week
Machine Learning 4. week robability and Conditional robability ayes Theorem Naïve ayes Classifier Umut ORHN, hd. robability The term shows the occurring likelihood of each situation in a random process
Bayes Formula. MATH 107: Finite Mathematics University of Louisville. March 26, 2014
Bayes Formula MATH 07: Finite Mathematics University of Louisville March 26, 204 Test Accuracy Conditional reversal 2 / 5 A motivating question A rare disease occurs in out of every 0,000 people. A test
A.I. in health informatics lecture 2 clinical reasoning & probabilistic inference, I. kevin small & byron wallace
A.I. in health informatics lecture 2 clinical reasoning & probabilistic inference, I kevin small & byron wallace today a review of probability random variables, maximum likelihood, etc. crucial for clinical
Generative Classifiers: Part 1. CSC411/2515: Machine Learning and Data Mining, Winter 2018 Michael Guerzhoy and Lisa Zhang
Generative Classifiers: Part 1 CSC411/2515: Machine Learning and Data Mining, Winter 2018 Michael Guerzhoy and Lisa Zhang 1 This Week Discriminative vs Generative Models Simple Model: Does the patient
CISC 4631 Data Mining
CISC 4631 Data Mining Lecture 06: ayes Theorem Theses slides are based on the slides by Tan, Steinbach and Kumar (textbook authors) Eamonn Koegh (UC Riverside) Andrew Moore (CMU/Google) 1 Naïve ayes Classifier
Dimensionality reduction
Dimensionality Reduction PCA continued Machine Learning CSE446 Carlos Guestrin University of Washington May 22, 2013 Carlos Guestrin 2005-2013 1 Dimensionality reduction n Input data may have thousands
WELCOME! Lecture 13 Thommy Perlinger
Quantitative Methods II WELCOME! Lecture 13 Thommy Perlinger Parametrical tests (tests for the mean) Nature and number of variables One-way vs. two-way ANOVA One-way ANOVA Y X 1 1 One dependent variable
Scribe to lecture Tuesday March
Scribe to lecture Tuesday March 16 2004 Scribe outlines: Message Confidence intervals Central limit theorem Em-algorithm Bayesian versus classical statistic Note: There is no scribe for the beginning of
Bayesian inference. Fredrik Ronquist and Peter Beerli. October 3, 2007
Bayesian inference Fredrik Ronquist and Peter Beerli October 3, 2007 1 Introduction The last few decades has seen a growing interest in Bayesian inference, an alternative approach to statistical inference.
Machine Learning. Yuh-Jye Lee. March 1, Lab of Data Science and Machine Intelligence Dept. of Applied Math. at NCTU
Machine Learning Yuh-Jye Lee Lab of Data Science and Machine Intelligence Dept. of Applied Math. at NCTU March 1, 2017 1 / 13 Bayes Rule Bayes Rule Assume that {B 1, B 2,..., B k } is a partition of S
Supervised Learning! Algorithm Implementations! Inferring Rudimentary Rules and Decision Trees!
Supervised Learning! Algorithm Implementations! Inferring Rudimentary Rules and Decision Trees! Summary! Input Knowledge representation! Preparing data for learning! Input: Concept, Instances, Attributes"
Naïve Bayes classification
Naïve Bayes classification 1 Probability theory Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. Examples: A person s height, the outcome of a coin toss
Lecture 8: Classification
1/26 Lecture 8: Classification Måns Eriksson Department of Mathematics, Uppsala University eriksson@math.uu.se Multivariate Methods 19/5 2010 Classification: introductory examples Goal: Classify an observation
Machine Learning! in just a few minutes. Jan Peters Gerhard Neumann
Machine Learning! in just a few minutes Jan Peters Gerhard Neumann 1 Purpose of this Lecture Foundations of machine learning tools for robotics We focus on regression methods and general principles Often
Introduction to Bayesian Learning. Machine Learning Fall 2018
Introduction to Bayesian Learning Machine Learning Fall 2018 1 What we have seen so far What does it mean to learn? Mistake-driven learning Learning by counting (and bounding) number of mistakes PAC learnability
CPSC 340: Machine Learning and Data Mining. More PCA Fall 2016
CPSC 340: Machine Learning and Data Mining More PCA Fall 2016 A2/Midterm: Admin Grades/solutions posted. Midterms can be viewed during office hours. Assignment 4: Due Monday. Extra office hours: Thursdays
Naïve Bayes, Maxent and Neural Models
Naïve Bayes, Maxent and Neural Models CMSC 473/673 UMBC Some slides adapted from 3SLP Outline Recap: classification (MAP vs. noisy channel) & evaluation Naïve Bayes (NB) classification Terminology: bag-of-words
Bayesian Classifiers and Probability Estimation. Vassilis Athitsos CSE 4308/5360: Artificial Intelligence I University of Texas at Arlington
Bayesian Classifiers and Probability Estimation Vassilis Athitsos CSE 4308/5360: Artificial Intelligence I University of Texas at Arlington 1 Data Space Suppose that we have a classification problem The