A Comparison of EMG and EEG Signals for Prostheses Control using Decision Tree

Size: px
Start display at page:

Download "A Comparison of EMG and EEG Signals for Prostheses Control using Decision Tree"

Transcription

1 Technology Volume, Issue, July-September, 3, pp. -8, IASTER 3 Online: , Print: A Comparison of EMG and EEG Signals for Prostheses Control using Decision Tree V. V. Ramalingam, S. Mohan, 3 V. Sugumaran Research Scholar, Bharathiar University, Coimbatore, India Professor & Dean - Department of CSE, Sri Eshwar College of Engineering, Coimbatore, India 3 Associate Professor - SMBS, VIT University, Chennai Campus, Chennai, India, ABSTRACT In spite of availability of various approaches, the control of prosthetic limb would be more effective if it is based on Electromyogram (EMG) signals from remnant muscles and Electroencephalogram (EEG). The analysis of these signals depends on various factors such as amplitude, time and frequency domain properties. EEG signals are obtained from the eperiments conducted in Biomedical laboratory using 7 different subjects with four different hand movements viz., finger open (fopen), finger close (fclose), clock wise wrist rotation (cw) and counter clock wise wrist rotation (ccw). The EMG dataset for the same conditions were obtained from the NINAPRO DATABASE, a resource for bio robotics community of hand movements. The statistical features were etracted for both EMG and EEG signals and classified using decision tree (C4.5) algorithm. The comparison of classification accuracies for both EMG and EEG signals is presented. Keywords: Classification, Decision tree, Electromyogram (EMG) signals, Electroencephalogram (EEG) signals, Statistical feature.. INTORDUCTION The main requirement of prosthetic arm is that it should be as near as possible to a natural arm. The artificial arm can either be mechanical, electrical or myoelectric. Myoelectric control is based on the electromyogram (EMG), which is a measure of neuromuscular activity detected directly from the muscle or from the skin surface. A myoelectric control system maps a set of features drawn from the myoelectric signal to a particular function, such as fleion of a prosthetic wrist. In this type of system, a user with the potential for naturally-evoked movement control []. There are some difficulties in etracting sufficient information from the EMG for prosthetic control like electrode placement, electrode type, skin and the muscle. The Autoregressive model will overcome the electrode placement noise []. The EEG signal is highly comple; it is one of the most common sources of information used to study brain function and neurological disorders (Agarwal, Gotman, Flanagan, & Rosenblatt, 998; Adeli, Zhou, & Dadmehr, 3; Hazarika, Chen, Tsoi, & Sergejew, 997). In general, interpretation of EEG signals is very comple and thus it demands lot of epertise. Computers have been proposed to solve this problem and thus, automated systems to recognize electroencephalographic changes. The long-term EEG recordings for proper evaluation and treatment of neurological diseases and prevention of the possibility of the analyst missing (or misreading) information (Agarwal et al., 998; Adeli et al., 3; Hazarika et al., 997). Hence, it is very clear

2 International Journal of Research in Computer Applications & Information Technology, Volume-, Issue-, July-September, 3, (O) (P) that there are only few literature available to guide for selecting the suitable signals for the identification of hand movements. An attempt has been made with EMG and EEG signals with statistical features decision tree based classification for the above problem. The paper is organized as follows, the section, deals with system architecture and data acquisition. In section 3, Feature etraction concepts. In section 4, brief description of classifier (Decision tree) is given. In section 5, result and discussion of application of the decision tree trained by EMG and EEG signals are presented. Finally, in section 6 the drawn conclusions are emphasized.. SYSTEM ARCHITECHTURE AND DATA ACQUISITION The EMG and EEG signals were acquired after proper skin preparations and amplified before being filtered and sampled. The preprocessed signals were used to etract the features and fed into the classifier as shown in Fig.. EMG and EEG ACQUISITION Artifacts Removal Data Filtering Feature Etraction Classification. Data Acquisition Fig.. System Architecture of prosthetic arm The Electroencephalogram (EEG) eperimental setup primarily focus on data from 7 healthy subjects while performing four different classes viz., fopen, fclose, cw and ccw. The EEG signals were collected from the subject s four different hand movements. The data in the EEG database were obtained by the following procedure: The subject was made to sit on an adjustable chair, instructed to have electrodes (C3, C4, CZ, FZ and PZ) with conductive gel medium on scalp surface. Initial signal artifacts due to head motion will be generally ignored in the analysis. Eperiment will be scheduled based on specific time series with respect to the classes. Signals from neurons are acquired with the help of five electrodes which in turn connected with Electroencephalogram device with the frequency ranges from 8 to 3 Hz. In order to avoid muscle fatigue and its influence on the EEG signal, minutes of rest are allowed between the training sequences Fig.a Time domain EEG Signals -5 5

3 International Journal of Research in Computer Applications & Information Technology, Volume-, Issue-, July-September, 3, (O) (P) Fig.b Time domain EMG signals 5 NINAPRO database consists of EMG data from the upper limbs of 7 subjects while performing 5 finger, hand and wrist movements. The database is publicly available to download in standard ASCII format [3]. The EMG was collected from a subject s forearm skin while performing a number of movements of interest, or producing force patterns of interest. Out of 5 movements, in the study only four finger movements fopen, fclose, cw and ccw were considered. The data in the NINAPRO database was acquired by the following procedure: The subject sat comfortably on an adjustable chair, in front of a table with a large screen. The EMG electrodes, data glove and inclinometer were worn on the right hand. The subjects were presented with short movies appearing on the screen and were asked to simply replicate the movements depicted in the movies as accurately as possible. After the training phase, a sequential series of ten repetitions of each class of movements was presented to the subject while data are recorded. Each movie lasts five seconds and three seconds of rest are allowed inbetween movements. In order to avoid muscle fatigue and its influence on the EMG signal, 5 minutes of rest was allowed between the training sequences. 3. FEATURE EXTRACTION From the EMG and EEG signals, descriptive statistical parameters such as mean, median, mode, kurtosis, skewness, standard error, standard deviation, minimum, maimum, sum, and range were computed to serve as features. They are named as statistical features here. Brief descriptions about the etracted features are given below. (a) Standard error: Standard error is a measure of the amount of error in the prediction of y for an individual in the regression, where and y are the sample means and n is the sample size. y y Standard error of the predicted, Y y y n (b) Standard deviation: This is a measure of the effective energy or power content of the EEG and EMG signal. The following formula was used for computation of standard deviation. 3

4 Technology, Volume-, Issue-, July-September, 3, (O) (P) Standard Deviation nn ( ) (c) Sample variance: It is variance of the signal points and the following formula was used for computation of sample variance. Sample Variance nn ( ) (d) Kurtosis: Kurtosis indicates the flatness or the spikiness of the signal. Its value is very low for normal condition of the EEG and EMG signal and high for faulty condition of the EEG and EMG due to the spiky nature of the signal. Kurtosis 4 n( n ) i 3( n ) ( n )( n )( n 3) s ( n )( n 3) where s is the sample standard deviation. (e) Skewness: Skewness characterizes the degree of asymmetry of a distribution around its mean. The following formula was used for computation of skewness. Skewness n n i s 3 (f) Range: It refers to the difference in maimum and minimum signal point values for a given signal. (g) Minimum value: It refers to the minimum signal point value in a given EEG and EMG signal. Therefore, it can be used to detect faulty EEG and EMG signal condition. (h) Maimum value: It refers to the maimum signal point value in a given signal. (i) Sum: It is the sum of all feature values for each sample. 4. DECISION TREE (C4.5) A decision tree is a tree based knowledge methodology used to represent classification rules [4-6]. A standard tree induced with C4.5 consists of a number of branches, one root, a number of nodes and a number of leaves. One branch is a chain of nodes from root to a leaf; and each node involves one attribute. The occurrence of an attribute in a tree provides the information about the importance of the associated attribute. The procedure of forming the Decision Tree and eploiting the same for feature selection is eplained by Sugumaran. 4.. Information Gain and Entropy Reduction Information gain is the epected reduction in entropy caused by portioning the samples according to this feature. Entropy is a measure of homogeneity of the set of eamples. Information gain measures how well a given attribute separates the training eamples according to their target classification. The measure is used to select among the candidate features at each step while growing the tree. 4

5 Technology, Volume-, Issue-, July-September, 3, (O) (P) Information gain (S, A) of a feature A relative to a collection of eamples S, is defined as: Gain ( S, A) where, Entropy ( S) v Value( A) Sv Entropy ( Sv ) S Values (A) is the set of all possible values for attribute A, S v is the subset of S for which feature A has value v (i.e., S v = {s S A(s) = v}). Note the first term in the equation for Gain is just the entropy of the original collection S and the second term is the epected value of the entropy after S is partitioned using feature A. The epected entropy described by the second term is the direct sum of the entropies of each subset S v, weighed by the fraction of samples S v / S that belong to S v. Gain (S,A) is therefore the epected reduction in entropy caused by knowing the value of feature A. c Entropy( S) P i log P i Entropy is given by i Where, c is the number of classes and p i is the proportion of S belonging to class i. Fig. 3a. Decision tree of EEG signals Fig. 3b. Decision tree of EMG signals 5

6 Technology, Volume-, Issue-, July-September, 3, (O) (P) RESULT AND DISCUSSION The data sets described in section. were taken and statistical parameters namely mean, median, standard deviation, variance, skewness, kurtosis etc., were computed for each signal for EMG and EEG signals as eplained in section 3. Then C4.5 algorithm was used to perform the dimensionality reduction. A decision tree was developed as shown in Fig. 3a and Fig. 3b. and therefrom the good features that contribute well for the classification were selected following footsteps of Sugumaran [5]. For EEG signals, out of 6 features (5 channels, each statistical features), the selected statistical features are C3mean, C3standard error, C4maimum, CZmedian, CZrange, FZstandard error, FZkurtosis, and FZminimum. Similarly, for EMG signals, out of 96 features (8 channels, each statistical features), the selected statistical features are C5kurtosis, C5minimum, and C6skewness. With these features the classification accuracy was computed using C4.5 decision tree algorithm. The minimum number of objects required to form a class (M) was varied from to 7(total number of signals per class) and the corresponding classification accuracies were noted down. The value of M which gives maimum classification accuracy was fied and confidence factor was varied from to in steps of.. The variation of classification accuracy with respect to confidence factor and minimum number of objects is shown Fig. 4. Top two graphs shows for EEG signals and bottom two graphs are for EMG signals. The best classification accuracy of 4.74% for EEG signals was achieved with M value of 8 and confidence factor of.. While EMG signals could produce a best classification accuracy of 38.88% with M value of 8 and confidence factor of.. Fig. 4. Training of Classifiers 6

7 Technology, Volume-, Issue-, July-September, 3, (O) (P) The misclassification information is best epressed with the help of confusion matri. For the present study, with best parameters the developed confusion matrices are shown in Fig.5. The interpretation of the confusion matri is as follows: The diagonal elements in the confusion matri (Refer Fig.5) show the number of correctly classified instances. In the first row, the first element shows number of data points that belong to fopen class and classified by the classifier as fopen. In the first row, the second element shows the number of data points belonging to fopen class but misclassified as fclose. In the first row, the third element shows the number of fopen data points misclassified as cw. In the first row, the fourth element shows the number of fopen data points misclassified as ccw and so on. Fig.5 Confusion matri of decision tree 6. CONCLUSION The prosthetic arm using decision tree based approach is a possibility. In the present study an attempt is made to compare the performances of EMG and EEG signals. For this purpose descriptive statistical parameters were used as features and decision tree was used as a classifier. The EEG signals give a classification accuracy of 4.74% while the EMG signals give a classification accuracy of 38.88%. From the above result and discussion one can conclude that EEG signals is better suited for prosthetic arm. The signal processing of EEG and EMG signals is a comple task and requires sophisticated techniques to yield better classification accuracy. 7

8 Technology, Volume-, Issue-, July-September, 3, (O) (P) ACKNOWLEDGEMENTS The EEG signals were eperimentally obtained from Bio-medical Department in S.R.M University, Kattankulathur, Chennai and EMG dataset collected from NINAPRO DATABASE A RESOURCE FOR BIOROBOTICS COMMUNITY. REFERENCES [] Angkoon Phinyomark, Chusak Limsakul and Pornchai Phukpattaranont, A Novel EMG Feature Etraction for Tolerance of Interference, Journal of 3th Annual Sysposium on Computational Science and Engineering, pp.47-43, 9. [] P.C Doerschuk, D.E. Gustafson and A.S. Willsky, Upper Etremly Limb Discrimination Using EMG Signal Analysis. IEEE Transactions on Biomedical Enigineerinig, Vol. 3, No. PP.8-9, 983. [3] Manfredo Atzori, el. Al., Building the Ninapro Database: A Resource for the Bio robotics Community. IEEE International Conference on Biomedical Robotics and Biomechatronics, pp ,. [4] J. R. Quinlan. Induction of Decision Trees, Machine Learning, volume, pp. 8-6, 986. [5] V. Sugumaran, V. Muralidharan, K.I. Ramachandran, Feature selection using Decision Tree and classification through Proimal Support Vector Machine for fault diagnostics of roller bearing, Mechanical Systems and Signal Processing, Volume, Issue, February 7, Pages [6] N. R. Sakthivel, V. Sugumaran, Binoy. B. Nair, Comparison of decision tree-fuzzy and rough set-fuzzy methods for fault categorization of mono-block centrifugal pump, Mechanical Systems and Signal Processing, Volume 4, Issue 6, August, Pages

Condition Monitoring of Single Point Cutting Tool through Vibration Signals using Decision Tree Algorithm

Condition Monitoring of Single Point Cutting Tool through Vibration Signals using Decision Tree Algorithm Columbia International Publishing Journal of Vibration Analysis, Measurement, and Control doi:10.7726/jvamc.2015.1003 Research Paper Condition Monitoring of Single Point Cutting Tool through Vibration

More information

HMM and IOHMM Modeling of EEG Rhythms for Asynchronous BCI Systems

HMM and IOHMM Modeling of EEG Rhythms for Asynchronous BCI Systems HMM and IOHMM Modeling of EEG Rhythms for Asynchronous BCI Systems Silvia Chiappa and Samy Bengio {chiappa,bengio}@idiap.ch IDIAP, P.O. Box 592, CH-1920 Martigny, Switzerland Abstract. We compare the use

More information

Decision Trees / NLP Introduction

Decision Trees / NLP Introduction Decision Trees / NLP Introduction Dr. Kevin Koidl School of Computer Science and Statistic Trinity College Dublin ADAPT Research Centre The ADAPT Centre is funded under the SFI Research Centres Programme

More information

Myoelectrical signal classification based on S transform and two-directional 2DPCA

Myoelectrical signal classification based on S transform and two-directional 2DPCA Myoelectrical signal classification based on S transform and two-directional 2DPCA Hong-Bo Xie1 * and Hui Liu2 1 ARC Centre of Excellence for Mathematical and Statistical Frontiers Queensland University

More information

CRITICAL EXPONENT ANALYSIS APPLIED TO SURFACE EMG SIGNALS FOR GESTURE RECOGNITION

CRITICAL EXPONENT ANALYSIS APPLIED TO SURFACE EMG SIGNALS FOR GESTURE RECOGNITION Metrol. Meas. Syst., Vol. XVIII (2011), No. 4, pp. 645-658 METROLOGY AND MEASUREMENT SYSTEMS Index 330930, ISSN 0860-8229 www.metrology.pg.gda.pl CRITICAL EXPONENT ANALYSIS APPLIED TO SURFACE EMG SIGNALS

More information

Modeling of Surface EMG Signals using System Identification Techniques

Modeling of Surface EMG Signals using System Identification Techniques Modeling of Surface EMG Signals using System Identification Techniques Vishnu R S PG Scholar, Dept. of Electrical and Electronics Engg. Mar Baselios College of Engineering and Technology Thiruvananthapuram,

More information

6.867 Machine learning

6.867 Machine learning 6.867 Machine learning Mid-term eam October 8, 6 ( points) Your name and MIT ID: .5.5 y.5 y.5 a).5.5 b).5.5.5.5 y.5 y.5 c).5.5 d).5.5 Figure : Plots of linear regression results with different types of

More information

Direct Estimation of Wrist Joint Angular Velocities from Surface EMGs by Using an SDNN Function Approximator

Direct Estimation of Wrist Joint Angular Velocities from Surface EMGs by Using an SDNN Function Approximator Direct Estimation of Wrist Joint Angular Velocities from Surface EMGs by Using an SDNN Function Approximator Kazumasa Horie 1, Atsuo Suemitsu 2, Tomohiro Tanno 1, and Masahiko Morita 1 1 University of

More information

Machine Learning 3. week

Machine Learning 3. week Machine Learning 3. week Entropy Decision Trees ID3 C4.5 Classification and Regression Trees (CART) 1 What is Decision Tree As a short description, decision tree is a data classification procedure which

More information

Predictive Analytics on Accident Data Using Rule Based and Discriminative Classifiers

Predictive Analytics on Accident Data Using Rule Based and Discriminative Classifiers Advances in Computational Sciences and Technology ISSN 0973-6107 Volume 10, Number 3 (2017) pp. 461-469 Research India Publications http://www.ripublication.com Predictive Analytics on Accident Data Using

More information

Fault Diagnosis of Helical Gear Box using Decision Tree through Vibration Signals

Fault Diagnosis of Helical Gear Box using Decision Tree through Vibration Signals International Journal of Performability Engineering Vol. 9, No. 2, March 2013, pp.221-233. RAMS Consultants Printed in India Fault Diagnosis of Helical Gear Box using Decision Tree through Vibration Signals

More information

Statistical Modeling of EMG Signal

Statistical Modeling of EMG Signal International Journal of Engineering and Applied Sciences (IJEAS) Statistical Modeling of EMG Signal Naufel B. Mohammed, Atta A. Saeed, Avan M. Ahmed Abstract The aim of this research is to measure and

More information

An Algorithm of Wavelets for the Pretreatment of EMG Biomedical Signals

An Algorithm of Wavelets for the Pretreatment of EMG Biomedical Signals Contemporary Engineering Sciences, Vol. 3, 2010, no. 6, 285-294 An Algorithm of Wavelets for the Pretreatment of EMG Biomedical Signals Youssef BENTALEB youssef bentaleb2003@yahoo.fr Said EL HAJJI elhajji@fsr.ac.ma

More information

Electroencephalogram Based Causality Graph Analysis in Behavior Tasks of Parkinson s Disease Patients

Electroencephalogram Based Causality Graph Analysis in Behavior Tasks of Parkinson s Disease Patients University of Denver Digital Commons @ DU Electronic Theses and Dissertations Graduate Studies 1-1-2015 Electroencephalogram Based Causality Graph Analysis in Behavior Tasks of Parkinson s Disease Patients

More information

Engine gearbox fault diagnosis using empirical mode decomposition method and Naïve Bayes algorithm

Engine gearbox fault diagnosis using empirical mode decomposition method and Naïve Bayes algorithm Sādhanā Vol., No. 7, July 17, pp. 113 1153 DOI 1.17/s1-17-7-9 Ó Indian Academy of Sciences Engine gearbox fault diagnosis using empirical mode decomposition method and Naïve Bayes algorithm KIRAN VERNEKAR,

More information

Introduction to Supervised Learning. Performance Evaluation

Introduction to Supervised Learning. Performance Evaluation Introduction to Supervised Learning Performance Evaluation Marcelo S. Lauretto Escola de Artes, Ciências e Humanidades, Universidade de São Paulo marcelolauretto@usp.br Lima - Peru Performance Evaluation

More information

Lecture 2. Judging the Performance of Classifiers. Nitin R. Patel

Lecture 2. Judging the Performance of Classifiers. Nitin R. Patel Lecture 2 Judging the Performance of Classifiers Nitin R. Patel 1 In this note we will examine the question of how to udge the usefulness of a classifier and how to compare different classifiers. Not only

More information

Impeller Fault Detection for a Centrifugal Pump Using Principal Component Analysis of Time Domain Vibration Features

Impeller Fault Detection for a Centrifugal Pump Using Principal Component Analysis of Time Domain Vibration Features Impeller Fault Detection for a Centrifugal Pump Using Principal Component Analysis of Time Domain Vibration Features Berli Kamiel 1,2, Gareth Forbes 2, Rodney Entwistle 2, Ilyas Mazhar 2 and Ian Howard

More information

Finger Motion Classification by Forearm Skin Surface Vibration Signals

Finger Motion Classification by Forearm Skin Surface Vibration Signals The Open Medical Informatics Journal, 2010, 4, 31-40 31 Open Access Finger Motion Classification by Forearm Skin Surface Vibration Signals Wenwei Yu,1, Toshiharu Kishi #,1, U. Rajendra Acharya 2, Yuse

More information

Artificial Neural Networks Francesco DI MAIO, Ph.D., Politecnico di Milano Department of Energy - Nuclear Division IEEE - Italian Reliability Chapter

Artificial Neural Networks Francesco DI MAIO, Ph.D., Politecnico di Milano Department of Energy - Nuclear Division IEEE - Italian Reliability Chapter Artificial Neural Networks Francesco DI MAIO, Ph.D., Politecnico di Milano Department of Energy - Nuclear Division IEEE - Italian Reliability Chapter (Chair) STF - China Fellow francesco.dimaio@polimi.it

More information

Classification of Mental Tasks from EEG Signals Using Spectral Analysis, PCA and SVM

Classification of Mental Tasks from EEG Signals Using Spectral Analysis, PCA and SVM BULGARIAN ACADEMY OF SCIENCES CYBERNETICS AND INFORMATION TECHNOLOGIES Volume 18, No 1 Sofia 2018 Print ISSN: 1311-9702; Online ISSN: 1314-4081 DOI: 10.2478/cait-2018-0007 Classification of Mental Tasks

More information

EEC 686/785 Modeling & Performance Evaluation of Computer Systems. Lecture 11

EEC 686/785 Modeling & Performance Evaluation of Computer Systems. Lecture 11 EEC 686/785 Modeling & Performance Evaluation of Computer Systems Lecture Department of Electrical and Computer Engineering Cleveland State University wenbing@ieee.org (based on Dr. Raj Jain s lecture

More information

Masking Covariance for Common Spatial Pattern as Feature Extraction

Masking Covariance for Common Spatial Pattern as Feature Extraction Masking Covariance for Common Spatial Pattern as Feature Extraction H. Asyraf 1, M. I. Shapiai 1, 2, N. A. Setiawan 3, W. S. N. S. Wan Musa 1 1 Electronic System Engineering, MJIIT. 2 Centre of Artificial

More information

ECE521 Lecture7. Logistic Regression

ECE521 Lecture7. Logistic Regression ECE521 Lecture7 Logistic Regression Outline Review of decision theory Logistic regression A single neuron Multi-class classification 2 Outline Decision theory is conceptually easy and computationally hard

More information

Induction on Decision Trees

Induction on Decision Trees Séance «IDT» de l'ue «apprentissage automatique» Bruno Bouzy bruno.bouzy@parisdescartes.fr www.mi.parisdescartes.fr/~bouzy Outline Induction task ID3 Entropy (disorder) minimization Noise Unknown attribute

More information

Decision Trees Part 1. Rao Vemuri University of California, Davis

Decision Trees Part 1. Rao Vemuri University of California, Davis Decision Trees Part 1 Rao Vemuri University of California, Davis Overview What is a Decision Tree Sample Decision Trees How to Construct a Decision Tree Problems with Decision Trees Classification Vs Regression

More information

Learning Decision Trees

Learning Decision Trees Learning Decision Trees Machine Learning Spring 2018 1 This lecture: Learning Decision Trees 1. Representation: What are decision trees? 2. Algorithm: Learning decision trees The ID3 algorithm: A greedy

More information

Classifier Evaluation. Learning Curve cleval testc. The Apparent Classification Error. Error Estimation by Test Set. Classifier

Classifier Evaluation. Learning Curve cleval testc. The Apparent Classification Error. Error Estimation by Test Set. Classifier Classifier Learning Curve How to estimate classifier performance. Learning curves Feature curves Rejects and ROC curves True classification error ε Bayes error ε* Sub-optimal classifier Bayes consistent

More information

Notes on Discriminant Functions and Optimal Classification

Notes on Discriminant Functions and Optimal Classification Notes on Discriminant Functions and Optimal Classification Padhraic Smyth, Department of Computer Science University of California, Irvine c 2017 1 Discriminant Functions Consider a classification problem

More information

Issues and Techniques in Pattern Classification

Issues and Techniques in Pattern Classification Issues and Techniques in Pattern Classification Carlotta Domeniconi www.ise.gmu.edu/~carlotta Machine Learning Given a collection of data, a machine learner eplains the underlying process that generated

More information

Selected Algorithms of Machine Learning from Examples

Selected Algorithms of Machine Learning from Examples Fundamenta Informaticae 18 (1993), 193 207 Selected Algorithms of Machine Learning from Examples Jerzy W. GRZYMALA-BUSSE Department of Computer Science, University of Kansas Lawrence, KS 66045, U. S. A.

More information

Classification Using Decision Trees

Classification Using Decision Trees Classification Using Decision Trees 1. Introduction Data mining term is mainly used for the specific set of six activities namely Classification, Estimation, Prediction, Affinity grouping or Association

More information

Bayesian Classification of Single-Trial Event-Related Potentials in EEG

Bayesian Classification of Single-Trial Event-Related Potentials in EEG Bayesian Classification of Single-Trial Event-Related Potentials in EEG Jens Kohlmorgen Benjamin Blankertz Fraunhofer FIRST.IDA Kekuléstr. 7, D-12489 Berlin, Germany E-mail: {jek, blanker}@first.fraunhofer.de

More information

Decision Tree Analysis for Classification Problems. Entscheidungsunterstützungssysteme SS 18

Decision Tree Analysis for Classification Problems. Entscheidungsunterstützungssysteme SS 18 Decision Tree Analysis for Classification Problems Entscheidungsunterstützungssysteme SS 18 Supervised segmentation An intuitive way of thinking about extracting patterns from data in a supervised manner

More information

Induction of Decision Trees

Induction of Decision Trees Induction of Decision Trees Peter Waiganjo Wagacha This notes are for ICS320 Foundations of Learning and Adaptive Systems Institute of Computer Science University of Nairobi PO Box 30197, 00200 Nairobi.

More information

Data Mining. Preamble: Control Application. Industrial Researcher s Approach. Practitioner s Approach. Example. Example. Goal: Maintain T ~Td

Data Mining. Preamble: Control Application. Industrial Researcher s Approach. Practitioner s Approach. Example. Example. Goal: Maintain T ~Td Data Mining Andrew Kusiak 2139 Seamans Center Iowa City, Iowa 52242-1527 Preamble: Control Application Goal: Maintain T ~Td Tel: 319-335 5934 Fax: 319-335 5669 andrew-kusiak@uiowa.edu http://www.icaen.uiowa.edu/~ankusiak

More information

CSE 352 (AI) LECTURE NOTES Professor Anita Wasilewska. NEURAL NETWORKS Learning

CSE 352 (AI) LECTURE NOTES Professor Anita Wasilewska. NEURAL NETWORKS Learning CSE 352 (AI) LECTURE NOTES Professor Anita Wasilewska NEURAL NETWORKS Learning Neural Networks Classifier Short Presentation INPUT: classification data, i.e. it contains an classification (class) attribute.

More information

Decision Tree Learning and Inductive Inference

Decision Tree Learning and Inductive Inference Decision Tree Learning and Inductive Inference 1 Widely used method for inductive inference Inductive Inference Hypothesis: Any hypothesis found to approximate the target function well over a sufficiently

More information

Notes on Machine Learning for and

Notes on Machine Learning for and Notes on Machine Learning for 16.410 and 16.413 (Notes adapted from Tom Mitchell and Andrew Moore.) Learning = improving with experience Improve over task T (e.g, Classification, control tasks) with respect

More information

The exam is closed book, closed notes except your one-page (two sides) or two-page (one side) crib sheet.

The exam is closed book, closed notes except your one-page (two sides) or two-page (one side) crib sheet. CS 189 Spring 013 Introduction to Machine Learning Final You have 3 hours for the exam. The exam is closed book, closed notes except your one-page (two sides) or two-page (one side) crib sheet. Please

More information

CHAPTER-17. Decision Tree Induction

CHAPTER-17. Decision Tree Induction CHAPTER-17 Decision Tree Induction 17.1 Introduction 17.2 Attribute selection measure 17.3 Tree Pruning 17.4 Extracting Classification Rules from Decision Trees 17.5 Bayesian Classification 17.6 Bayes

More information

STA 414/2104, Spring 2014, Practice Problem Set #1

STA 414/2104, Spring 2014, Practice Problem Set #1 STA 44/4, Spring 4, Practice Problem Set # Note: these problems are not for credit, and not to be handed in Question : Consider a classification problem in which there are two real-valued inputs, and,

More information

Learning Decision Trees

Learning Decision Trees Learning Decision Trees Machine Learning Fall 2018 Some slides from Tom Mitchell, Dan Roth and others 1 Key issues in machine learning Modeling How to formulate your problem as a machine learning problem?

More information

2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media,

2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising

More information

Neural Networks Introduction

Neural Networks Introduction Neural Networks Introduction H.A Talebi Farzaneh Abdollahi Department of Electrical Engineering Amirkabir University of Technology Winter 2011 H. A. Talebi, Farzaneh Abdollahi Neural Networks 1/22 Biological

More information

Decision trees. Special Course in Computer and Information Science II. Adam Gyenge Helsinki University of Technology

Decision trees. Special Course in Computer and Information Science II. Adam Gyenge Helsinki University of Technology Decision trees Special Course in Computer and Information Science II Adam Gyenge Helsinki University of Technology 6.2.2008 Introduction Outline: Definition of decision trees ID3 Pruning methods Bibliography:

More information

Data Mining Project. C4.5 Algorithm. Saber Salah. Naji Sami Abduljalil Abdulhak

Data Mining Project. C4.5 Algorithm. Saber Salah. Naji Sami Abduljalil Abdulhak Data Mining Project C4.5 Algorithm Saber Salah Naji Sami Abduljalil Abdulhak Decembre 9, 2010 1.0 Introduction Before start talking about C4.5 algorithm let s see first what is machine learning? Human

More information

The Quadratic Entropy Approach to Implement the Id3 Decision Tree Algorithm

The Quadratic Entropy Approach to Implement the Id3 Decision Tree Algorithm Journal of Computer Science and Information Technology December 2018, Vol. 6, No. 2, pp. 23-29 ISSN: 2334-2366 (Print), 2334-2374 (Online) Copyright The Author(s). All Rights Reserved. Published by American

More information

Machine Learning to Automatically Detect Human Development from Satellite Imagery

Machine Learning to Automatically Detect Human Development from Satellite Imagery Technical Disclosure Commons Defensive Publications Series April 24, 2017 Machine Learning to Automatically Detect Human Development from Satellite Imagery Matthew Manolides Follow this and additional

More information

arxiv: v2 [quant-ph] 21 Nov 2014

arxiv: v2 [quant-ph] 21 Nov 2014 Myoelectric Control of Artificial Limb Inspired by Quantum Information Processing Michael Siomau Physics Department, Jazan University, P.O. Box 114, 45142 Jazan, Kingdom of Saudi Arabia Ning Jiang Department

More information

EXAM IN STATISTICAL MACHINE LEARNING STATISTISK MASKININLÄRNING

EXAM IN STATISTICAL MACHINE LEARNING STATISTISK MASKININLÄRNING EXAM IN STATISTICAL MACHINE LEARNING STATISTISK MASKININLÄRNING DATE AND TIME: June 9, 2018, 09.00 14.00 RESPONSIBLE TEACHER: Andreas Svensson NUMBER OF PROBLEMS: 5 AIDING MATERIAL: Calculator, mathematical

More information

CS6375: Machine Learning Gautam Kunapuli. Decision Trees

CS6375: Machine Learning Gautam Kunapuli. Decision Trees Gautam Kunapuli Example: Restaurant Recommendation Example: Develop a model to recommend restaurants to users depending on their past dining experiences. Here, the features are cost (x ) and the user s

More information

CHAPTER 5 EEG SIGNAL CLASSIFICATION BASED ON NN WITH ICA AND STFT

CHAPTER 5 EEG SIGNAL CLASSIFICATION BASED ON NN WITH ICA AND STFT 69 CHAPTER 5 EEG SIGNAL CLASSIFICATION BASED ON NN WITH ICA AND STFT 5.1 OVERVIEW A novel approach is proposed for Electroencephalogram signal classification using Artificial Neural Network based on Independent

More information

MIDTERM: CS 6375 INSTRUCTOR: VIBHAV GOGATE October,

MIDTERM: CS 6375 INSTRUCTOR: VIBHAV GOGATE October, MIDTERM: CS 6375 INSTRUCTOR: VIBHAV GOGATE October, 23 2013 The exam is closed book. You are allowed a one-page cheat sheet. Answer the questions in the spaces provided on the question sheets. If you run

More information

Learning Classification Trees. Sargur Srihari

Learning Classification Trees. Sargur Srihari Learning Classification Trees Sargur srihari@cedar.buffalo.edu 1 Topics in CART CART as an adaptive basis function model Classification and Regression Tree Basics Growing a Tree 2 A Classification Tree

More information

ARTIFICIAL NEURAL NETWORKS گروه مطالعاتي 17 بهار 92

ARTIFICIAL NEURAL NETWORKS گروه مطالعاتي 17 بهار 92 ARTIFICIAL NEURAL NETWORKS گروه مطالعاتي 17 بهار 92 BIOLOGICAL INSPIRATIONS Some numbers The human brain contains about 10 billion nerve cells (neurons) Each neuron is connected to the others through 10000

More information

Classification with Perceptrons. Reading:

Classification with Perceptrons. Reading: Classification with Perceptrons Reading: Chapters 1-3 of Michael Nielsen's online book on neural networks covers the basics of perceptrons and multilayer neural networks We will cover material in Chapters

More information

Principal Component Analysis

Principal Component Analysis Principal Component Analysis Introduction Consider a zero mean random vector R n with autocorrelation matri R = E( T ). R has eigenvectors q(1),,q(n) and associated eigenvalues λ(1) λ(n). Let Q = [ q(1)

More information

Extraction of Sleep-Spindles from the Electroencephalogram (EEG)

Extraction of Sleep-Spindles from the Electroencephalogram (EEG) Extraction of Sleep-Spindles from the Electroencephalogram (EEG) Allan Kardec Barros Bio-mimetic Control Research Center, RIKEN, 2271-13 Anagahora, Shimoshidami, Moriyama-ku, Nagoya 463, Japan Roman Rosipal,

More information

Dan Roth 461C, 3401 Walnut

Dan Roth   461C, 3401 Walnut CIS 519/419 Applied Machine Learning www.seas.upenn.edu/~cis519 Dan Roth danroth@seas.upenn.edu http://www.cis.upenn.edu/~danroth/ 461C, 3401 Walnut Slides were created by Dan Roth (for CIS519/419 at Penn

More information

Vibration Signals Analysis and Condition Monitoring of Centrifugal Pump

Vibration Signals Analysis and Condition Monitoring of Centrifugal Pump Technical Journal of Engineering and Applied Sciences Available online at www.tjeas.com 2013 TJEAS Journal-2013-3-12/1081-1085 ISSN 2051-0853 2013 TJEAS Vibration Signals Analysis and Condition Monitoring

More information

Machine Learning & Data Mining

Machine Learning & Data Mining Group M L D Machine Learning M & Data Mining Chapter 7 Decision Trees Xin-Shun Xu @ SDU School of Computer Science and Technology, Shandong University Top 10 Algorithm in DM #1: C4.5 #2: K-Means #3: SVM

More information

Machine Learning, Midterm Exam: Spring 2009 SOLUTION

Machine Learning, Midterm Exam: Spring 2009 SOLUTION 10-601 Machine Learning, Midterm Exam: Spring 2009 SOLUTION March 4, 2009 Please put your name at the top of the table below. If you need more room to work out your answer to a question, use the back of

More information

EEG- Signal Processing

EEG- Signal Processing Fatemeh Hadaeghi EEG- Signal Processing Lecture Notes for BSP, Chapter 5 Master Program Data Engineering 1 5 Introduction The complex patterns of neural activity, both in presence and absence of external

More information

Informal Definition: Telling things apart

Informal Definition: Telling things apart 9. Decision Trees Informal Definition: Telling things apart 2 Nominal data No numeric feature vector Just a list or properties: Banana: longish, yellow Apple: round, medium sized, different colors like

More information

Single-trial P300 detection with Kalman filtering and SVMs

Single-trial P300 detection with Kalman filtering and SVMs Single-trial P300 detection with Kalman filtering and SVMs Lucie Daubigney and Olivier Pietquin SUPELEC / UMI 2958 (GeorgiaTech - CNRS) 2 rue Edouard Belin, 57070 Metz - France firstname.lastname@supelec.fr

More information

Principal Component Analysis, A Powerful Scoring Technique

Principal Component Analysis, A Powerful Scoring Technique Principal Component Analysis, A Powerful Scoring Technique George C. J. Fernandez, University of Nevada - Reno, Reno NV 89557 ABSTRACT Data mining is a collection of analytical techniques to uncover new

More information

Linear Classification. CSE 6363 Machine Learning Vassilis Athitsos Computer Science and Engineering Department University of Texas at Arlington

Linear Classification. CSE 6363 Machine Learning Vassilis Athitsos Computer Science and Engineering Department University of Texas at Arlington Linear Classification CSE 6363 Machine Learning Vassilis Athitsos Computer Science and Engineering Department University of Texas at Arlington 1 Example of Linear Classification Red points: patterns belonging

More information

Data Mining. CS57300 Purdue University. Bruno Ribeiro. February 8, 2018

Data Mining. CS57300 Purdue University. Bruno Ribeiro. February 8, 2018 Data Mining CS57300 Purdue University Bruno Ribeiro February 8, 2018 Decision trees Why Trees? interpretable/intuitive, popular in medical applications because they mimic the way a doctor thinks model

More information

Plant-wide Root Cause Identification of Transient Disturbances with Application to a Board Machine

Plant-wide Root Cause Identification of Transient Disturbances with Application to a Board Machine Moncef Chioua, Margret Bauer, Su-Liang Chen, Jan C. Schlake, Guido Sand, Werner Schmidt and Nina F. Thornhill Plant-wide Root Cause Identification of Transient Disturbances with Application to a Board

More information

Support Vector Regression Based S-transform for Prediction of Single and Multiple Power Quality Disturbances

Support Vector Regression Based S-transform for Prediction of Single and Multiple Power Quality Disturbances European Journal of Scientific Research ISSN 145-216X Vol.34 No.2 (29), pp.237-251 EuroJournals Publishing, Inc. 29 http://www.eurojournals.com/ejsr.htm Support Vector Regression Based S-transform for

More information

Decision Tree Learning Lecture 2

Decision Tree Learning Lecture 2 Machine Learning Coms-4771 Decision Tree Learning Lecture 2 January 28, 2008 Two Types of Supervised Learning Problems (recap) Feature (input) space X, label (output) space Y. Unknown distribution D over

More information

Decision Tree Learning

Decision Tree Learning Decision Tree Learning Berlin Chen Department of Computer Science & Information Engineering National Taiwan Normal University References: 1. Machine Learning, Chapter 3 2. Data Mining: Concepts, Models,

More information

EEL 851: Biometrics. An Overview of Statistical Pattern Recognition EEL 851 1

EEL 851: Biometrics. An Overview of Statistical Pattern Recognition EEL 851 1 EEL 851: Biometrics An Overview of Statistical Pattern Recognition EEL 851 1 Outline Introduction Pattern Feature Noise Example Problem Analysis Segmentation Feature Extraction Classification Design Cycle

More information

day month year documentname/initials 1

day month year documentname/initials 1 ECE471-571 Pattern Recognition Lecture 13 Decision Tree Hairong Qi, Gonzalez Family Professor Electrical Engineering and Computer Science University of Tennessee, Knoxville http://www.eecs.utk.edu/faculty/qi

More information

Decision T ree Tree Algorithm Week 4 1

Decision T ree Tree Algorithm Week 4 1 Decision Tree Algorithm Week 4 1 Team Homework Assignment #5 Read pp. 105 117 of the text book. Do Examples 3.1, 3.2, 3.3 and Exercise 3.4 (a). Prepare for the results of the homework assignment. Due date

More information

.. Cal Poly CSC 466: Knowledge Discovery from Data Alexander Dekhtyar.. for each element of the dataset we are given its class label.

.. Cal Poly CSC 466: Knowledge Discovery from Data Alexander Dekhtyar.. for each element of the dataset we are given its class label. .. Cal Poly CSC 466: Knowledge Discovery from Data Alexander Dekhtyar.. Data Mining: Classification/Supervised Learning Definitions Data. Consider a set A = {A 1,...,A n } of attributes, and an additional

More information

A Novel Rejection Measurement in Handwritten Numeral Recognition Based on Linear Discriminant Analysis

A Novel Rejection Measurement in Handwritten Numeral Recognition Based on Linear Discriminant Analysis 009 0th International Conference on Document Analysis and Recognition A Novel Rejection easurement in Handwritten Numeral Recognition Based on Linear Discriminant Analysis Chun Lei He Louisa Lam Ching

More information

Article from. Predictive Analytics and Futurism. July 2016 Issue 13

Article from. Predictive Analytics and Futurism. July 2016 Issue 13 Article from Predictive Analytics and Futurism July 2016 Issue 13 Regression and Classification: A Deeper Look By Jeff Heaton Classification and regression are the two most common forms of models fitted

More information

CS145: INTRODUCTION TO DATA MINING

CS145: INTRODUCTION TO DATA MINING CS145: INTRODUCTION TO DATA MINING 4: Vector Data: Decision Tree Instructor: Yizhou Sun yzsun@cs.ucla.edu October 10, 2017 Methods to Learn Vector Data Set Data Sequence Data Text Data Classification Clustering

More information

Neural Networks and Ensemble Methods for Classification

Neural Networks and Ensemble Methods for Classification Neural Networks and Ensemble Methods for Classification NEURAL NETWORKS 2 Neural Networks A neural network is a set of connected input/output units (neurons) where each connection has a weight associated

More information

Comparison of Shannon, Renyi and Tsallis Entropy used in Decision Trees

Comparison of Shannon, Renyi and Tsallis Entropy used in Decision Trees Comparison of Shannon, Renyi and Tsallis Entropy used in Decision Trees Tomasz Maszczyk and W lodzis law Duch Department of Informatics, Nicolaus Copernicus University Grudzi adzka 5, 87-100 Toruń, Poland

More information

Sparse Kernel Machines - SVM

Sparse Kernel Machines - SVM Sparse Kernel Machines - SVM Henrik I. Christensen Robotics & Intelligent Machines @ GT Georgia Institute of Technology, Atlanta, GA 30332-0280 hic@cc.gatech.edu Henrik I. Christensen (RIM@GT) Support

More information

10-701/ Machine Learning - Midterm Exam, Fall 2010

10-701/ Machine Learning - Midterm Exam, Fall 2010 10-701/15-781 Machine Learning - Midterm Exam, Fall 2010 Aarti Singh Carnegie Mellon University 1. Personal info: Name: Andrew account: E-mail address: 2. There should be 15 numbered pages in this exam

More information

Sound Recognition in Mixtures

Sound Recognition in Mixtures Sound Recognition in Mixtures Juhan Nam, Gautham J. Mysore 2, and Paris Smaragdis 2,3 Center for Computer Research in Music and Acoustics, Stanford University, 2 Advanced Technology Labs, Adobe Systems

More information

Decision Trees. Each internal node : an attribute Branch: Outcome of the test Leaf node or terminal node: class label.

Decision Trees. Each internal node : an attribute Branch: Outcome of the test Leaf node or terminal node: class label. Decision Trees Supervised approach Used for Classification (Categorical values) or regression (continuous values). The learning of decision trees is from class-labeled training tuples. Flowchart like structure.

More information

Predictive Modeling: Classification. KSE 521 Topic 6 Mun Yi

Predictive Modeling: Classification. KSE 521 Topic 6 Mun Yi Predictive Modeling: Classification Topic 6 Mun Yi Agenda Models and Induction Entropy and Information Gain Tree-Based Classifier Probability Estimation 2 Introduction Key concept of BI: Predictive modeling

More information

Data Mining. 3.6 Regression Analysis. Fall Instructor: Dr. Masoud Yaghini. Numeric Prediction

Data Mining. 3.6 Regression Analysis. Fall Instructor: Dr. Masoud Yaghini. Numeric Prediction Data Mining 3.6 Regression Analysis Fall 2008 Instructor: Dr. Masoud Yaghini Outline Introduction Straight-Line Linear Regression Multiple Linear Regression Other Regression Models References Introduction

More information

Advanced Methods for Fault Detection

Advanced Methods for Fault Detection Advanced Methods for Fault Detection Piero Baraldi Agip KCO Introduction Piping and long to eploration distance pipelines activities Piero Baraldi Maintenance Intervention Approaches & PHM Maintenance

More information

Holdout and Cross-Validation Methods Overfitting Avoidance

Holdout and Cross-Validation Methods Overfitting Avoidance Holdout and Cross-Validation Methods Overfitting Avoidance Decision Trees Reduce error pruning Cost-complexity pruning Neural Networks Early stopping Adjusting Regularizers via Cross-Validation Nearest

More information

SUPERVISED LEARNING: INTRODUCTION TO CLASSIFICATION

SUPERVISED LEARNING: INTRODUCTION TO CLASSIFICATION SUPERVISED LEARNING: INTRODUCTION TO CLASSIFICATION 1 Outline Basic terminology Features Training and validation Model selection Error and loss measures Statistical comparison Evaluation measures 2 Terminology

More information

Rule Generation using Decision Trees

Rule Generation using Decision Trees Rule Generation using Decision Trees Dr. Rajni Jain 1. Introduction A DT is a classification scheme which generates a tree and a set of rules, representing the model of different classes, from a given

More information

ECE521 week 3: 23/26 January 2017

ECE521 week 3: 23/26 January 2017 ECE521 week 3: 23/26 January 2017 Outline Probabilistic interpretation of linear regression - Maximum likelihood estimation (MLE) - Maximum a posteriori (MAP) estimation Bias-variance trade-off Linear

More information

Principles of Pattern Recognition. C. A. Murthy Machine Intelligence Unit Indian Statistical Institute Kolkata

Principles of Pattern Recognition. C. A. Murthy Machine Intelligence Unit Indian Statistical Institute Kolkata Principles of Pattern Recognition C. A. Murthy Machine Intelligence Unit Indian Statistical Institute Kolkata e-mail: murthy@isical.ac.in Pattern Recognition Measurement Space > Feature Space >Decision

More information

Automated Statistical Recognition of Partial Discharges in Insulation Systems.

Automated Statistical Recognition of Partial Discharges in Insulation Systems. Automated Statistical Recognition of Partial Discharges in Insulation Systems. Massih-Reza AMINI, Patrick GALLINARI, Florence d ALCHE-BUC LIP6, Université Paris 6, 4 Place Jussieu, F-75252 Paris cedex

More information

Classification and Prediction

Classification and Prediction Classification Classification and Prediction Classification: predict categorical class labels Build a model for a set of classes/concepts Classify loan applications (approve/decline) Prediction: model

More information

ISSN: (Online) Volume 2, Issue 2, February 2014 International Journal of Advance Research in Computer Science and Management Studies

ISSN: (Online) Volume 2, Issue 2, February 2014 International Journal of Advance Research in Computer Science and Management Studies ISSN: 2321-7782 (Online) Volume 2, Issue 2, February 2014 International Journal of Advance Research in Computer Science and Management Studies Research Article / Paper / Case Study Available online at:

More information

Human Activity Recognition and Estimation of Calorie Expenditure - A Data Analytic Approach

Human Activity Recognition and Estimation of Calorie Expenditure - A Data Analytic Approach Human Activity Recognition and Estimation of Calorie Expenditure - A Data Analytic Approach Mushtaque Ahamed A Dr. Snehanshu Saha PESIT-Bangalore South Campus December 4, 2015 Mushtaque Ahamed A Dr. Snehanshu

More information

Chapter 14 Combining Models

Chapter 14 Combining Models Chapter 14 Combining Models T-61.62 Special Course II: Pattern Recognition and Machine Learning Spring 27 Laboratory of Computer and Information Science TKK April 3th 27 Outline Independent Mixing Coefficients

More information

CSE 546 Midterm Exam, Fall 2014

CSE 546 Midterm Exam, Fall 2014 CSE 546 Midterm Eam, Fall 2014 1. Personal info: Name: UW NetID: Student ID: 2. There should be 14 numbered pages in this eam (including this cover sheet). 3. You can use an material ou brought: an book,

More information