Topics. Concept Learning. Concept Learning Task. Concept Descriptions
|
|
|
- Kerrie Walton
- 6 years ago
- Views:
Transcription
1 Topics Concept Learning Sattiraju Prabhakar CS898O: Lecture#2 Wichita State University Concept Description Using Concept Descriptions Training Examples Concept Learning Algorithm: Find-S 1/22/2006 ML2006_ConceptLearning 2 Concept Descriptions Concept Learning Task Input: (training_example 1, binary_value) (training_example 2, binary_value) (training_example n, binary_value) Output: Target_Function that describes a concept 1/22/2006 ML2006_ConceptLearning 3 1/22/2006 ML2006_ConceptLearning 4 1
2 Concept Description What is a Concept? Concept describes a set of objects or events It is a subset of a complete description of a larger set of objects What is Concept Description? It is a representation of concept using a notation Pictorially Understanding a Concept Set of Objects Described by the Concept Set of All Objects 1/22/2006 ML2006_ConceptLearning 5 1/22/2006 ML2006_ConceptLearning 6 Representation of Concepts We use a sequence of (attribute, value) pairs Method: Identify set of all objects (S) to which the concept belongs to. Let S indicate the set of all objects Let S c indicate the set of all objects described by the concept Identify all the attributes that describe the set of objects, S A = {a 1, a 2,, a n } Domains of attributes, D = {d 1, d 2,, d n } Identify the subdomains of D that describe concept D c = {d 1, d 2,, d n } Concepts Example1 Example: Oranges S = Set of all fruit S c = set of oranges A = {Color, Taste, Shape, Size, Skin} Domain D D(Color) = {Red, Orange, Blue, Green} D(Taste) = {Bitter, Sweet, No_Taste} D(Shape) = {Spherical, Long, Oval, Cone} D(Size) = {Small, Medium, Large} D(Skin) = {Thin, Thick, No_Skin} Concept of Orange = <Color = {Orange}, Taste = {Bitter, Sweet}, Shape = {Spherical}, Size = {Medium}, Skin = {Thick}> 1/22/2006 ML2006_ConceptLearning 7 1/22/2006 ML2006_ConceptLearning 8 2
3 Exercise Using Concept Descriptions Write the concept description for Arch. Consider S to be the set of all configuration of three elements. 1/22/2006 ML2006_ConceptLearning 9 1/22/2006 ML2006_ConceptLearning 10 Test Example How does a machine use the concept? Concept Description Performance System /- Example Given the concept description for Orange, is the following example (test) an orange (this is an indirect way of asking whether it is an object covered by the concept) Example: <Color = red, Taste = sweet, Shape = spherical, Size = small, Skin = thin> : Test Example is a member of objects defined by the concept description -: Test example is not a member 1/22/2006 ML2006_ConceptLearning 11 1/22/2006 ML2006_ConceptLearning 12 3
4 Method for Performance System 1. C = concept description 2. = Test Example 3. Select the next a i A 1. concept_domain_values Domain_Values(C, a i ) 2. example_domain_values Domain_Values(, a i ) 3. If (example_domain_values concept_domain_values) 1. Go to 3 4. Else return - 4. return Exercise Using the Arch concept description, find whether the following (test) example is covered by the concept description or not. 1/22/2006 ML2006_ConceptLearning 13 1/22/2006 ML2006_ConceptLearning 14 Concept Learning Task Training Examples Training Examples Learning Algorithm Concept Description 1/22/2006 ML2006_ConceptLearning 15 1/22/2006 ML2006_ConceptLearning 16 4
5 Training Examples: EnjoySport Training Example: Cells Exa mpl e Sky Air Temp Humid ity Wind Wate r Foreca st Enjoy Sport 1 Sunny Warm Normal Strong Warm Same Yes 2 Sunny Warm High Strong Warm Same Yes Positive Negative Negative Positive 3 Rainy Cold High Strong Warm Change No 4 Sunny Warm High Strong Cool Change Yes 1/22/2006 ML2006_ConceptLearning 17 Attributes: <tails, nuclei, color, wall> Examples: (<2, 2, dark, thin>, 1) (<2, 1, light, thin>, 0) (<1, 2, dark, thick>, 0) (<2, 2, light, thin>, 1) 1/22/2006 ML2006_ConceptLearning 18 Training Examples: IsAnimal Exercise: Arch cat (<4, fur, meow>, 1) dog (<4, fur, bark>, 1) rose (<0, thorns, 0>, 0) tree (<1, scales, 0>, 0) Attributes: {Number_of_limbs, Skin_covering, Make_sound} Draw Training examples for Arch 1/22/2006 ML2006_ConceptLearning 19 1/22/2006 ML2006_ConceptLearning 20 5
6 Concept Learning Algorithm: Find-S Main Idea When we start we do not know target function (Concept Description) We derive target function Incrementally in a number of steps At each step, we come up with an approximate solution It is an approximation of the target function (the desired final solution) We call each approximate solution as a hypothesis We keep refining until we cover all the training examples 1/22/2006 ML2006_ConceptLearning 21 1/22/2006 ML2006_ConceptLearning 22 Hypothesis What is a hypothesis? Hypothesis is an approximation of final solution The final solution is target function (concept description) Since we cannot arrive at final solution in one step, hypothesis allows us to store intermediate solutions What is an approximate solution? It is not exactly the final solution But it has some features of the final solution Some features may not be correct Representation of Hypothesis Many possible representations Here h is conjunction of constraints on attributes Each constraint can be a specific value (e.g., Water = Warm) a don t care value (e.g., Water =?) no value allowed (e.g., Water = 0) Example: Sky AirTemp Humid Wind Water Forecst <Sunny?? Strong? Same> 1/22/2006 ML2006_ConceptLearning 23 1/22/2006 ML2006_ConceptLearning 24 6
7 General, Specific Hypotheses General Specific Relation Most General hypothesis It represents all objects That is, all possible training examples are covered Representation: <?,?,?,?,?,?> Most Specific hypothesis: It does not represent any training examples or any objects Representation: <0, 0, 0, 0, 0, 0> There are hypotheses at different levels of generality (or specificity) Examples: General: <Sunny, Warm,?, Strong,?,?> Specific: <Sunny, Warm, High, Strong,?, Same> 1/22/2006 ML2006_ConceptLearning 25 1/22/2006 ML2006_ConceptLearning 26 General Specific Relation Natural Structure: General Specific Intuition for Search Strategies: By using this relationship, we can exhaustively search through the hypothesis space without explicitly enumerating every hypothesis Formalization of Natural Structure for Search If h2 is more general than h1 & if h1 classifies a set of instances as positive Then h2 classifies this set and additional instances as positive Definition: Satisfies: For any instance x in and hypothesis h in H, x satisfies h if and only if h(x) = 1 1/22/2006 ML2006_ConceptLearning 27 1/22/2006 ML2006_ConceptLearning 28 7
8 Formalizing more_general_than Relation Definition: more_general_than_or_equal_to: Let h j and h k be boolean-valued functions defined over. Then h j is more_general_than_or_equal_to h k (written h j g h k ) if and only if ( x )[(h k (x) = 1) (h j (x) = 1)] Formalization (contd) Definition: strictly_more_general_than: h j > g h k if and only if (h j g h k ) Λ (h k g h j ) Definition: more_specific_than: h j is more specific than h k when h k is more general than h j 1/22/2006 ML2006_ConceptLearning 29 1/22/2006 ML2006_ConceptLearning 30 Find-S Algorithm: Visualization Find-S Algorithm: Using Functional Description Main Idea: Navigate through hypothesis space Goal: Arrive at most specific hypothesis Covers all positive training examples (instances) Eliminates all negative training examples Algorithm: Start with most specific hypothesis, h Select next training example, e If e is positive, generalize h to include e If e is negative, ignore it 1/22/2006 ML2006_ConceptLearning 31 1/22/2006 ML2006_ConceptLearning 32 8
9 Mechanism for Generalizing A hypothesis is a conjunction of constraints h = c 1 c 2 c n Here c i = (attribute, value) A training example, x = <(att, val), (att, val), > Algorithm: For each attribute, a: If value(a, x) does not satisfy value(a, h) Generalize value(a, h) to include value(a, x) Otherwise, do nothing Example x = (<Sky =sunny, AirTemp= warm, Humidity=high, Wind=strong, Water= warm, Forecast=same>, ) h = <Sky =sunny, AirTemp= warm, Humidity=normal, Wind=strong, Water= warm, Forecast=same> Only Humidity does not cover x The generalized hypothesis: h = <Sky =sunny, AirTemp= warm, Humidity={normal, high}, Wind=strong, Water= warm, Forecast=same> 1/22/2006 ML2006_ConceptLearning 33 1/22/2006 ML2006_ConceptLearning 34 Find-S Algorithm h most-specific hypothesis For each training example, x If x is positive For each attribute constraint ac in h If constraint ac is not satisfied by x» Replace ac in h by immediately more general constraint that is satisfied by x Exercises Learn the concept of arch Learn the concept of cell Learn the concept of animal 1/22/2006 ML2006_ConceptLearning 35 1/22/2006 ML2006_ConceptLearning 36 9
10 Analysis Convergence to Correct Target Concept Find-S cannot determine Preferring most specific hypothesis Not clear about extent of generalization What about inconsistent examples Gives errors 1/22/2006 ML2006_ConceptLearning 37 10
Concept Learning through General-to-Specific Ordering
0. Concept Learning through General-to-Specific Ordering Based on Machine Learning, T. Mitchell, McGRAW Hill, 1997, ch. 2 Acknowledgement: The present slides are an adaptation of slides drawn by T. Mitchell
Concept Learning Mitchell, Chapter 2. CptS 570 Machine Learning School of EECS Washington State University
Concept Learning Mitchell, Chapter 2 CptS 570 Machine Learning School of EECS Washington State University Outline Definition General-to-specific ordering over hypotheses Version spaces and the candidate
Overview. Machine Learning, Chapter 2: Concept Learning
Overview Concept Learning Representation Inductive Learning Hypothesis Concept Learning as Search The Structure of the Hypothesis Space Find-S Algorithm Version Space List-Eliminate Algorithm Candidate-Elimination
Concept Learning. Space of Versions of Concepts Learned
Concept Learning Space of Versions of Concepts Learned 1 A Concept Learning Task Target concept: Days on which Aldo enjoys his favorite water sport Example Sky AirTemp Humidity Wind Water Forecast EnjoySport
Version Spaces.
. Machine Learning Version Spaces Prof. Dr. Martin Riedmiller AG Maschinelles Lernen und Natürlichsprachliche Systeme Institut für Informatik Technische Fakultät Albert-Ludwigs-Universität Freiburg riedmiller@informatik.uni-freiburg.de
Introduction to machine learning. Concept learning. Design of a learning system. Designing a learning system
Introduction to machine learning Concept learning Maria Simi, 2011/2012 Machine Learning, Tom Mitchell Mc Graw-Hill International Editions, 1997 (Cap 1, 2). Introduction to machine learning When appropriate
Concept Learning. Berlin Chen Department of Computer Science & Information Engineering National Taiwan Normal University.
Concept Learning Berlin Chen Department of Computer Science & Information Engineering National Taiwan Normal University References: 1. Tom M. Mitchell, Machine Learning, Chapter 2 2. Tom M. Mitchell s
Concept Learning. Berlin Chen References: 1. Tom M. Mitchell, Machine Learning, Chapter 2 2. Tom M. Mitchell s teaching materials.
Concept Learning Berlin Chen 2005 References: 1. Tom M. Mitchell, Machine Learning, Chapter 2 2. Tom M. Mitchell s teaching materials MLDM-Berlin 1 What is a Concept? Concept of Dogs Concept of Cats Concept
Concept Learning.
. Machine Learning Concept Learning Prof. Dr. Martin Riedmiller AG Maschinelles Lernen und Natürlichsprachliche Systeme Institut für Informatik Technische Fakultät Albert-Ludwigs-Universität Freiburg Martin.Riedmiller@uos.de
Outline. [read Chapter 2] Learning from examples. General-to-specic ordering over hypotheses. Version spaces and candidate elimination.
![Outline. [read Chapter 2] Learning from examples. General-to-specic ordering over hypotheses. Version spaces and candidate elimination.](/thumbs/81/84340607.jpg "Outline. [read Chapter 2] Learning from examples. General-to-specic ordering over hypotheses. Version spaces and candidate elimination.")
Outline [read Chapter 2] [suggested exercises 2.2, 2.3, 2.4, 2.6] Learning from examples General-to-specic ordering over hypotheses Version spaces and candidate elimination algorithm Picking new examples
[read Chapter 2] [suggested exercises 2.2, 2.3, 2.4, 2.6] General-to-specific ordering over hypotheses
![[read Chapter 2] [suggested exercises 2.2, 2.3, 2.4, 2.6] General-to-specific ordering over hypotheses](/thumbs/86/94586808.jpg "[read Chapter 2] [suggested exercises 2.2, 2.3, 2.4, 2.6] General-to-specific ordering over hypotheses")
1 CONCEPT LEARNING AND THE GENERAL-TO-SPECIFIC ORDERING [read Chapter 2] [suggested exercises 2.2, 2.3, 2.4, 2.6] Learning from examples General-to-specific ordering over hypotheses Version spaces and
Question of the Day? Machine Learning 2D1431. Training Examples for Concept Enjoy Sport. Outline. Lecture 3: Concept Learning
Question of the Day? Machine Learning 2D43 Lecture 3: Concept Learning What row of numbers comes next in this series? 2 2 22 322 3222 Outline Training Examples for Concept Enjoy Sport Learning from examples
Lecture 2: Foundations of Concept Learning
Lecture 2: Foundations of Concept Learning Cognitive Systems II - Machine Learning WS 2005/2006 Part I: Basic Approaches to Concept Learning Version Space, Candidate Elimination, Inductive Bias Lecture
Machine Learning 2D1431. Lecture 3: Concept Learning
Machine Learning 2D1431 Lecture 3: Concept Learning Question of the Day? What row of numbers comes next in this series? 1 11 21 1211 111221 312211 13112221 Outline Learning from examples General-to specific
CSCE 478/878 Lecture 2: Concept Learning and the General-to-Specific Ordering
Outline Learning from eamples CSCE 78/878 Lecture : Concept Learning and te General-to-Specific Ordering Stepen D. Scott (Adapted from Tom Mitcell s slides) General-to-specific ordering over ypoteses Version
Fundamentals of Concept Learning
Aims 09s: COMP947 Macine Learning and Data Mining Fundamentals of Concept Learning Marc, 009 Acknowledgement: Material derived from slides for te book Macine Learning, Tom Mitcell, McGraw-Hill, 997 ttp://www-.cs.cmu.edu/~tom/mlbook.tml
EECS 349: Machine Learning
EECS 349: Machine Learning Bryan Pardo Topic 1: Concept Learning and Version Spaces (with some tweaks by Doug Downey) 1 Concept Learning Much of learning involves acquiring general concepts from specific
Answers Machine Learning Exercises 2
nswers Machine Learning Exercises 2 Tim van Erven October 7, 2007 Exercises. Consider the List-Then-Eliminate algorithm for the EnjoySport example with hypothesis space H = {?,?,?,?,?,?, Sunny,?,?,?,?,?,
BITS F464: MACHINE LEARNING
BITS F464: MACHINE LEARNING Lecture-09: Concept Learning Dr. Kamlesh Tiwari Assistant Professor Department of Computer Science and Information Systems Engineering, BITS Pilani, Rajasthan-333031 INDIA Jan
Topics. Bayesian Learning. What is Bayesian Learning? Objectives for Bayesian Learning
Topics Bayesian Learning Sattiraju Prabhakar CS898O: ML Wichita State University Objectives for Bayesian Learning Bayes Theorem and MAP Bayes Optimal Classifier Naïve Bayes Classifier An Example Classifying
Outline. Training Examples for EnjoySport. 2 lecture slides for textbook Machine Learning, c Tom M. Mitchell, McGraw Hill, 1997
Outline Training Examples for EnjoySport Learning from examples General-to-specific ordering over hypotheses [read Chapter 2] [suggested exercises 2.2, 2.3, 2.4, 2.6] Version spaces and candidate elimination
Data Mining and Machine Learning
Data Mining and Machine Learning Concept Learning and Version Spaces Introduction Concept Learning Generality Relations Refinement Operators Structured Hypothesis Spaces Simple algorithms Find-S Find-G
Computational Learning Theory
1 Computational Learning Theory 2 Computational learning theory Introduction Is it possible to identify classes of learning problems that are inherently easy or difficult? Can we characterize the number
Computational Learning Theory
Computational Learning Theory [read Chapter 7] [Suggested exercises: 7.1, 7.2, 7.5, 7.8] Computational learning theory Setting 1: learner poses queries to teacher Setting 2: teacher chooses examples Setting
Module 12. Machine Learning. Version 2 CSE IIT, Kharagpur
Module 12 Machine Learning Lesson 34 Learning From Observations 12.2 Concept Learning Definition: The problem is to learn a function mapping examples into two classes: positive and negative. We are given
Decision Tree Learning
Topics Decision Tree Learning Sattiraju Prabhakar CS898O: DTL Wichita State University What are decision trees? How do we use them? New Learning Task ID3 Algorithm Weka Demo C4.5 Algorithm Weka Demo Implementation
Machine Learning. Computational Learning Theory. Eric Xing , Fall Lecture 9, October 5, 2016
Machine Learning 10-701, Fall 2016 Computational Learning Theory Eric Xing Lecture 9, October 5, 2016 Reading: Chap. 7 T.M book Eric Xing @ CMU, 2006-2016 1 Generalizability of Learning In machine learning
Learning Classification Trees. Sargur Srihari
Learning Classification Trees Sargur srihari@cedar.buffalo.edu 1 Topics in CART CART as an adaptive basis function model Classification and Regression Tree Basics Growing a Tree 2 A Classification Tree
Decision Tree Learning
Topics Decision Tree Learning Sattiraju Prabhakar CS898O: DTL Wichita State University What are decision trees? How do we use them? New Learning Task ID3 Algorithm Weka Demo C4.5 Algorithm Weka Demo Implementation
Decision Tree Learning and Inductive Inference
Decision Tree Learning and Inductive Inference 1 Widely used method for inductive inference Inductive Inference Hypothesis: Any hypothesis found to approximate the target function well over a sufficiently
Tutorial 6. By:Aashmeet Kalra
Tutorial 6 By:Aashmeet Kalra AGENDA Candidate Elimination Algorithm Example Demo of Candidate Elimination Algorithm Decision Trees Example Demo of Decision Trees Concept and Concept Learning A Concept
Machine Learning. Computational Learning Theory. Le Song. CSE6740/CS7641/ISYE6740, Fall 2012
Machine Learning CSE6740/CS7641/ISYE6740, Fall 2012 Computational Learning Theory Le Song Lecture 11, September 20, 2012 Based on Slides from Eric Xing, CMU Reading: Chap. 7 T.M book 1 Complexity of Learning
Bayesian Learning Features of Bayesian learning methods:
Bayesian Learning Features of Bayesian learning methods: Each observed training example can incrementally decrease or increase the estimated probability that a hypothesis is correct. This provides a more
Imagine we ve got a set of data containing several types, or classes. E.g. information about customers, and class=whether or not they buy anything.
Decision Trees Defining the Task Imagine we ve got a set of data containing several types, or classes. E.g. information about customers, and class=whether or not they buy anything. Can we predict, i.e
Introduction to Machine Learning
Outline Contents Introduction to Machine Learning Concept Learning Varun Chandola February 2, 2018 1 Concept Learning 1 1.1 Example Finding Malignant Tumors............. 2 1.2 Notation..............................
Computational Learning Theory
Computational Learning Theory Sinh Hoa Nguyen, Hung Son Nguyen Polish-Japanese Institute of Information Technology Institute of Mathematics, Warsaw University February 14, 2006 inh Hoa Nguyen, Hung Son
Computational Learning Theory
09s1: COMP9417 Machine Learning and Data Mining Computational Learning Theory May 20, 2009 Acknowledgement: Material derived from slides for the book Machine Learning, Tom M. Mitchell, McGraw-Hill, 1997
The Bayesian Learning
The Bayesian Learning Rodrigo Fernandes de Mello Invited Professor at Télécom ParisTech Associate Professor at Universidade de São Paulo, ICMC, Brazil http://www.icmc.usp.br/~mello mello@icmc.usp.br First
Introduction. Decision Tree Learning. Outline. Decision Tree 9/7/2017. Decision Tree Definition
Introduction Decision Tree Learning Practical methods for inductive inference Approximating discrete-valued functions Robust to noisy data and capable of learning disjunctive expression ID3 earch a completely
Learning Decision Trees
Learning Decision Trees Machine Learning Fall 2018 Some slides from Tom Mitchell, Dan Roth and others 1 Key issues in machine learning Modeling How to formulate your problem as a machine learning problem?
Learning Decision Trees
Learning Decision Trees Machine Learning Spring 2018 1 This lecture: Learning Decision Trees 1. Representation: What are decision trees? 2. Algorithm: Learning decision trees The ID3 algorithm: A greedy
The Naïve Bayes Classifier. Machine Learning Fall 2017
The Naïve Bayes Classifier Machine Learning Fall 2017 1 Today s lecture The naïve Bayes Classifier Learning the naïve Bayes Classifier Practical concerns 2 Today s lecture The naïve Bayes Classifier Learning
Linear Classifiers: Expressiveness
Linear Classifiers: Expressiveness Machine Learning Spring 2018 The slides are mainly from Vivek Srikumar 1 Lecture outline Linear classifiers: Introduction What functions do linear classifiers express?
Introduction to ML. Two examples of Learners: Naïve Bayesian Classifiers Decision Trees
Introduction to ML Two examples of Learners: Naïve Bayesian Classifiers Decision Trees Why Bayesian learning? Probabilistic learning: Calculate explicit probabilities for hypothesis, among the most practical
Bayesian Classification. Bayesian Classification: Why?
Bayesian Classification http://css.engineering.uiowa.edu/~comp/ Bayesian Classification: Why? Probabilistic learning: Computation of explicit probabilities for hypothesis, among the most practical approaches
EECS 349: Machine Learning Bryan Pardo
EECS 349: Machine Learning Bryan Pardo Topic: Concept Learning 1 Concept Learning Much of learning involves acquiring general concepts from specific training examples Concept: subset of objects from some
Classification Using Decision Trees
Classification Using Decision Trees 1. Introduction Data mining term is mainly used for the specific set of six activities namely Classification, Estimation, Prediction, Affinity grouping or Association
Artificial Intelligence. Topic
Artificial Intelligence Topic What is decision tree? A tree where each branching node represents a choice between two or more alternatives, with every branching node being part of a path to a leaf node
Decision Trees Part 1. Rao Vemuri University of California, Davis
Decision Trees Part 1 Rao Vemuri University of California, Davis Overview What is a Decision Tree Sample Decision Trees How to Construct a Decision Tree Problems with Decision Trees Classification Vs Regression
Decision Trees. Tirgul 5
Decision Trees Tirgul 5 Using Decision Trees It could be difficult to decide which pet is right for you. We ll find a nice algorithm to help us decide what to choose without having to think about it. 2
Machine Learning Alternatives to Manual Knowledge Acquisition
Machine Learning Alternatives to Manual Knowledge Acquisition Interactive programs which elicit knowledge from the expert during the course of a conversation at the terminal. Programs which learn by scanning
Classification: Rule Induction Information Retrieval and Data Mining. Prof. Matteo Matteucci
Classification: Rule Induction Information Retrieval and Data Mining Prof. Matteo Matteucci What is Rule Induction? The Weather Dataset 3 Outlook Temp Humidity Windy Play Sunny Hot High False No Sunny
Introduction to Machine Learning
Introduction to Machine Learning Concept Learning Varun Chandola Computer Science & Engineering State University of New York at Buffalo Buffalo, NY, USA chandola@buffalo.edu Chandola@UB CSE 474/574 1 /
Decision Tree Learning - ID3
Decision Tree Learning - ID3 n Decision tree examples n ID3 algorithm n Occam Razor n Top-Down Induction in Decision Trees n Information Theory n gain from property 1 Training Examples Day Outlook Temp.
Machine Learning 2007: Slides 1. Instructor: Tim van Erven Website: erven/teaching/0708/ml/
Machine 2007: Slides 1 Instructor: Tim van Erven (Tim.van.Erven@cwi.nl) Website: www.cwi.nl/ erven/teaching/0708/ml/ September 6, 2007, updated: September 13, 2007 1 / 37 Overview The Most Important Supervised
Machine Learning: Exercise Sheet 2
Machine Learning: Exercise Sheet 2 Manuel Blum AG Maschinelles Lernen und Natürlichsprachliche Systeme Albert-Ludwigs-Universität Freiburg mblum@informatik.uni-freiburg.de Manuel Blum Machine Learning
B555 - Machine Learning - Homework 4. Enrique Areyan April 28, 2015
- Machine Learning - Homework Enrique Areyan April 8, 01 Problem 1: Give decision trees to represent the following oolean functions a) A b) A C c) Ā d) A C D e) A C D where Ā is a negation of A and is
ARTIFICIAL INTELLIGENCE. Supervised learning: classification
INFOB2KI 2017-2018 Utrecht University The Netherlands ARTIFICIAL INTELLIGENCE Supervised learning: classification Lecturer: Silja Renooij These slides are part of the INFOB2KI Course Notes available from
Rule Generation using Decision Trees
Rule Generation using Decision Trees Dr. Rajni Jain 1. Introduction A DT is a classification scheme which generates a tree and a set of rules, representing the model of different classes, from a given
T Machine Learning: Basic Principles
Machine Learning: Basic Principles Laboratory of Computer and Information Science (CIS) Department of Computer Science and Engineering Helsinki University of Technology (TKK) Autumn 2007 Outline Elements
Bayesian Learning. Artificial Intelligence Programming. 15-0: Learning vs. Deduction
15-0: Learning vs. Deduction Artificial Intelligence Programming Bayesian Learning Chris Brooks Department of Computer Science University of San Francisco So far, we ve seen two types of reasoning: Deductive
Lecture 3: Decision Trees
Lecture 3: Decision Trees Cognitive Systems II - Machine Learning SS 2005 Part I: Basic Approaches of Concept Learning ID3, Information Gain, Overfitting, Pruning Lecture 3: Decision Trees p. Decision
Decision Tree Learning Mitchell, Chapter 3. CptS 570 Machine Learning School of EECS Washington State University
Decision Tree Learning Mitchell, Chapter 3 CptS 570 Machine Learning School of EECS Washington State University Outline Decision tree representation ID3 learning algorithm Entropy and information gain
EECS 349:Machine Learning Bryan Pardo
EECS 349:Machine Learning Bryan Pardo Topic 2: Decision Trees (Includes content provided by: Russel & Norvig, D. Downie, P. Domingos) 1 General Learning Task There is a set of possible examples Each example
Applied Logic. Lecture 4 part 2 Bayesian inductive reasoning. Marcin Szczuka. Institute of Informatics, The University of Warsaw
Applied Logic Lecture 4 part 2 Bayesian inductive reasoning Marcin Szczuka Institute of Informatics, The University of Warsaw Monographic lecture, Spring semester 2017/2018 Marcin Szczuka (MIMUW) Applied
Lecture Notes in Machine Learning Chapter 4: Version space learning
Lecture Notes in Machine Learning Chapter 4: Version space learning Zdravko Markov February 17, 2004 Let us consider an example. We shall use an attribute-value language for both the examples and the hypotheses
Lecture 3: Decision Trees
Lecture 3: Decision Trees Cognitive Systems - Machine Learning Part I: Basic Approaches of Concept Learning ID3, Information Gain, Overfitting, Pruning last change November 26, 2014 Ute Schmid (CogSys,
Decision Trees. Danushka Bollegala
Decision Trees Danushka Bollegala Rule-based Classifiers In rule-based learning, the idea is to learn a rule from train data in the form IF X THEN Y (or a combination of nested conditions) that explains
CS 380: ARTIFICIAL INTELLIGENCE MACHINE LEARNING. Santiago Ontañón
CS 380: ARTIFICIAL INTELLIGENCE MACHINE LEARNING Santiago Ontañón so367@drexel.edu Summary so far: Rational Agents Problem Solving Systematic Search: Uninformed Informed Local Search Adversarial Search
T Machine Learning: Basic Principles
Machine Learning: Basic Principles Bayesian Networks Laboratory of Computer and Information Science (CIS) Department of Computer Science and Engineering Helsinki University of Technology (TKK) Autumn 2007
CS 6375 Machine Learning
CS 6375 Machine Learning Decision Trees Instructor: Yang Liu 1 Supervised Classifier X 1 X 2. X M Ref class label 2 1 Three variables: Attribute 1: Hair = {blond, dark} Attribute 2: Height = {tall, short}
Algorithms for Classification: The Basic Methods
Algorithms for Classification: The Basic Methods Outline Simplicity first: 1R Naïve Bayes 2 Classification Task: Given a set of pre-classified examples, build a model or classifier to classify new cases.
Naïve Bayes Classifiers
Naïve Bayes Classifiers Example: PlayTennis (6.9.1) Given a new instance, e.g. (Outlook = sunny, Temperature = cool, Humidity = high, Wind = strong ), we want to compute the most likely hypothesis: v NB
Data Mining Part 4. Prediction
Data Mining Part 4. Prediction 4.3. Fall 2009 Instructor: Dr. Masoud Yaghini Outline Introduction Bayes Theorem Naïve References Introduction Bayesian classifiers A statistical classifiers Introduction
Bayesian Learning. Reading: Tom Mitchell, Generative and discriminative classifiers: Naive Bayes and logistic regression, Sections 1-2.
Bayesian Learning Reading: Tom Mitchell, Generative and discriminative classifiers: Naive Bayes and logistic regression, Sections 1-2. (Linked from class website) Conditional Probability Probability of
Course 395: Machine Learning
Course 395: Machine Learning Lecturers: Maja Pantic (maja@doc.ic.ac.uk) Stavros Petridis (sp104@doc.ic.ac.uk) Goal (Lectures): To present basic theoretical concepts and key algorithms that form the core
Decision Trees. Gavin Brown
Decision Trees Gavin Brown Every Learning Method has Limitations Linear model? KNN? SVM? Explain your decisions Sometimes we need interpretable results from our techniques. How do you explain the above
Machine Learning
Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University October 11, 2012 Today: Computational Learning Theory Probably Approximately Coorrect (PAC) learning theorem
Machine Learning Recitation 8 Oct 21, Oznur Tastan
Machine Learning 10601 Recitation 8 Oct 21, 2009 Oznur Tastan Outline Tree representation Brief information theory Learning decision trees Bagging Random forests Decision trees Non linear classifier Easy
Decision Tree Learning. Dr. Xiaowei Huang
Decision Tree Learning Dr. Xiaowei Huang https://cgi.csc.liv.ac.uk/~xiaowei/ After two weeks, Remainders Should work harder to enjoy the learning procedure slides Read slides before coming, think them
Typical Supervised Learning Problem Setting
Typical Supervised Learning Problem Setting Given a set (database) of observations Each observation (x1,, xn, y) Xi are input variables Y is a particular output Build a model to predict y = f(x1,, xn)
Classification and Regression Trees
Classification and Regression Trees Ryan P Adams So far, we have primarily examined linear classifiers and regressors, and considered several different ways to train them When we ve found the linearity
Lecture 9: Bayesian Learning
Lecture 9: Bayesian Learning Cognitive Systems II - Machine Learning Part II: Special Aspects of Concept Learning Bayes Theorem, MAL / ML hypotheses, Brute-force MAP LEARNING, MDL principle, Bayes Optimal
weather, cloudy, rainy, sunny, snowy, windy
Halifax County Schools 1st Six Weeks Goal 2 Weather The learner will make observations and build an understanding of weather concepts. 2.01 Observe and report daily weather changes throughout the year.
Machine Learning in Bioinformatics
Machine Learning in Bioinformatics Arlindo Oliveira aml@inesc-id.pt Data Mining: Concepts and Techniques Data mining concepts Learning from examples Decision trees Neural Networks 1 Typical Supervised
Classification: Decision Trees
Classification: Decision Trees These slides were assembled by Byron Boots, with grateful acknowledgement to Eric Eaton and the many others who made their course materials freely available online. Feel
Machine Learning
Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University October 11, 2012 Today: Computational Learning Theory Probably Approximately Coorrect (PAC) learning theorem
Math 1312 Lesson 1: Sets, Statements, and Reasoning. A set is any collection of objects. These objects are called the elements of the set.
Math 1312 Lesson 1: Sets, Statements, and Reasoning A set is any collection of objects. hese objects are called the elements of the set. A is a subset of B, if A is "contained" inside B, that is, all elements
Decision Tree Learning
Decision Tree Learning Berlin Chen Department of Computer Science & Information Engineering National Taiwan Normal University References: 1. Machine Learning, Chapter 3 2. Data Mining: Concepts, Models,
Mining Classification Knowledge
Mining Classification Knowledge Remarks on NonSymbolic Methods JERZY STEFANOWSKI Institute of Computing Sciences, Poznań University of Technology COST Doctoral School, Troina 2008 Outline 1. Bayesian classification
Data classification (II)
Lecture 4: Data classification (II) Data Mining - Lecture 4 (2016) 1 Outline Decision trees Choice of the splitting attribute ID3 C4.5 Classification rules Covering algorithms Naïve Bayes Classification
CMSC 422 Introduction to Machine Learning Lecture 4 Geometry and Nearest Neighbors. Furong Huang /
CMSC 422 Introduction to Machine Learning Lecture 4 Geometry and Nearest Neighbors Furong Huang / furongh@cs.umd.edu What we know so far Decision Trees What is a decision tree, and how to induce it from
Question of the Day. Machine Learning 2D1431. Decision Tree for PlayTennis. Outline. Lecture 4: Decision Tree Learning
Question of the Day Machine Learning 2D1431 How can you make the following equation true by drawing only one straight line? 5 + 5 + 5 = 550 Lecture 4: Decision Tree Learning Outline Decision Tree for PlayTennis
Plant Reproduction - Pollination
Plant Adaptations Plant Reproduction - Pollination Pollination is the process where male genetic information (called pollen) from one flower is transferred to the female part of another flower. This is
Lecture 5: Efficient PAC Learning. 1 Consistent Learning: a Bound on Sample Complexity
Universität zu Lübeck Institut für Theoretische Informatik Lecture notes on Knowledge-Based and Learning Systems by Maciej Liśkiewicz Lecture 5: Efficient PAC Learning 1 Consistent Learning: a Bound on
Decision T ree Tree Algorithm Week 4 1
Decision Tree Algorithm Week 4 1 Team Homework Assignment #5 Read pp. 105 117 of the text book. Do Examples 3.1, 3.2, 3.3 and Exercise 3.4 (a). Prepare for the results of the homework assignment. Due date
My Journal By: Copyright 2015 A Dab of Glue Will Do
My Journal By: My Journal By: My Journal By: My Journal By: Science Notebook Contract This is my Science Notebook for the 20 school year: I promise to take care of my notebook. use one page at a time.
Computational Learning Theory (VC Dimension)
Computational Learning Theory (VC Dimension) 1 Difficulty of machine learning problems 2 Capabilities of machine learning algorithms 1 Version Space with associated errors error is the true error, r is
Mining Classification Knowledge
Mining Classification Knowledge Remarks on NonSymbolic Methods JERZY STEFANOWSKI Institute of Computing Sciences, Poznań University of Technology SE lecture revision 2013 Outline 1. Bayesian classification
Chapter 3: Decision Tree Learning
Chapter 3: Decision Tree Learning CS 536: Machine Learning Littman (Wu, TA) Administration Books? New web page: http://www.cs.rutgers.edu/~mlittman/courses/ml03/ schedule lecture notes assignment info.