Ensemble Based on Data Envelopment Analysis
|
|
- Gloria Harper
- 5 years ago
- Views:
Transcription
1 Enseble Based on Data Envelopent Analysis So Young Sohn & Hong Choi Departent of Coputer Science & Industrial Systes Engineering, Yonsei University, Seoul, Korea Tel) , Fax) E-ail: Abstract. There has been uch research to evaluate the efficiency of various data fusion/enseble approaches. However, when cobining individual classifiers for fusion or enseble purposes, typically only isclassification rate has been considered as a perforance easure. This ight be risky especially when the class distribution is skewed or when the costs associated with both Type I and II errors are significantly different fro each other. For this kind of situation, consideration of additional perforance easures such as sensitivity, specificity, false negative or positive errors are needed. In this paper, we propose to use DEA in order to find the weights involved in ulti-attributes perforances of each classifier as an eleent of a data enseble algorith. This algorith is expected to serve general purposes of classification.. Introduction Data ining is the process of extracting valid, previously unknown, and ultiately coprehensible inforation fro large databases and using it to ake crucial business decisions. Effective ining necessary inforation and knowledge fro a large database has been recognized as a key research topic by any practitioners in the field of data-based arketing. Algoriths often used for data ining can be classified into one of the following areas: artificial neural network, achine learning, and classical statistical odels. It has been reported that the classification accuracy of the individual algorith can be iproved by cobining the results of several classifiers. Data fusion techniques try to cobine classification results obtained fro several single classifiers and are known to iprove the classification accuracy when soe results of relatively uncorrelated classifiers are cobined. Data enseble cobines various results obtained fro a single classifier fitted repeatedly based on several bootstrap resaples. The resulting perforance is known to be ore stable than that of a single classifier. There has been uch research to evaluate the efficiency of various data fusion/enseble approaches. However, when cobining individual classifiers for fusion or enseble purposes, typically only one attribute, that is isclassification rate, has been considered as a perforance easure. This ight be risky especially when the class distribution is skewed or when the costs associated with both Type I and II errors are significantly different fro each other. For this kind of situation, consideration of additional perforance easures such as sensitivity, specificity, false
2 negative or positive errors are needed. Then it becoes a ulti-attribute decision aking proble in ters of finding class inforation for a given case based on different weights on various perforance easures. Subsequent question raised is how to find such weights. In this paper, we propose to use DEA in order to find the weights involved in ultiattributes perforances of each classifier as an eleent of a data enseble algorith. Data Envelopent Analysis (DEA) has been frequently applied to assess the efficiency of several decision aking units (DMU) which have ultiple inputs as well as outputs. By way of DEA, one can find the efficiency score of each DMU and can figure out the set of efficient DMUs based on the set of non-doinated solution. In addition, DEA provides inefficient DMUs with the bencharking point and has advantages over alternative paraetric approaches such as regression or ratio analysis [2]. Organization of this paper is as follows. In section 2, we briefly suarize the established data enseble techniques along with related literature. In section 3, we introduce the enseble based on DEA. In section 4, we illustrate an exaple of the proposed ethod using the road traffic accident data. In section 5 we suarize our findings. 2. Literature Review We first review data enseble literature. Enseble algoriths can be divided into two types: those that adaptively change the distribution of the bootstrap training set based on the perforance of previous classifiers, as in Boosting ethods or Arcing (Adaptive resapling and Cobining) and those that do not, as in Bagging (Bootstrap AGGregatING)). Bagging (Bootstrap AGGregatING) algorith introduced by Breian [] votes classifiers generated by different bootstrap saples. A bootstrap saple is generated by uniforly sapling N instances fro the training set with replaceent. Detailed procedure is as follows: Step. Suppose f is a classifier, producing an M-vector output with (one) Step 2. To bag and M- (zero), at the input point x. f, we draw bootstrap saples T = ( t t,, t ) each of size N with replaceent fro the training data., 2 L N Step 3. Classify input point x to the class k with largest vote in f k bagging as follows. f k bagging M k = f () M =
3 The basic idea of Bagging is to reduce the deviation of several classifiers by voting the classified results due to bootstrap resaples. Arcing (Adaptive Resapling and Cobining) is designed exclusively for classification proble, which is developed by Freund and Shapire [3] in the nae of boosting but Breian renaed it as arcing. Basic idea of it is alost like bagging which tries to reduce the deviation of several classifiers by voting the classified results due to bootstrap resaples. Arcing takes unequal probability bootstrap saples, that is the probability of a training exaple being sapled is not unifor, but depends on the training error of previous predictors. In General, the classification procedure of arcing (Adaboost M) can be suarized as follows: Step. Saple with replaceent fro T with probabilities P (i) (where P (i)=/n and i,2, L, N = ) (=,, M) and construct the classifier f using the resapled set T of size N. Step 2. Classify T using f and let d(i)= if exaple i is incorrectly classified, else d(i)=0. ε Step 3. Calculate the and as follows. β ε = N i = ( ε P ( i ) d ( i ), β = ε Step 4. Update probabilities P + (i) by using the following forula. P d ( i) P ( i) β ( i) = d ( i P ( i) β + ) Step 5. Let =+ and go to Step if <M. Step 6. Take a weighted vote of the classifications, with weights log (β ). ) (2) (3) Quinlan [7] reported results of applying both bagging and boosting by decision trees (C4.5) on 8 data sets. Although boosting generally increases accuracy ore than bagging, it also produces severe degradation on soe data sets. The authors' further experient showed that such deterioration in general perforance of boosting is resulted fro over-fitting a large nuber of trials that allow the coposite classifier to becoe very coplex. Instead of using the fixed weight for the vote of classifier, they suggested using the voting weight of each classifier to vary in response to the confidence with which the instance is classified. Trials over a diverse collection of datasets under their suggestion reduced the downside of classification accuracy and also led to slightly better results on ost of the datasets considered. Opitz and Maclin [6] presented an epirical evaluation of Bagging and Boosting as ethods for creating an enseble of neural networks and decision-tree with 4 data sets. The authors found out that Bagging is appropriate for ost probles, but when properly applied, Boosting ay produce even larger gains in accuracy. Their results
4 also showed that the advantages and disadvantages of both Bagging and Boosting depend only on the doain to which they are applied, instead of the type of classifier. Hansen [4] copared five eta achine learning ethods which eploy neural networks as an enseble eber: three fro enseble ethods (Siple, Bagging and Adaboos and two fro ixture expert ethods (XuME and Dynco). The epirical results showed that the cooperative error function of Dynco is superior to the copetitive error function of the others. Kohavi and Wolpert [5] proposed bias and variance decoposition for isclassification error, when there are only two levels of class. The authors showed how estiating the ters in the decoposition using frequency counts leads to biased estiates and explained how to get unbiased estiators to overcoe such ajor shortcoings as obtaining potentially negative variance. They then gave soe exaples of the bias-variance tradeoff using two achine learning algoriths applied to data available in several UC-Irvine repository. 3. Enseble Based on DEA When ultiple classifiers are obtained, we suggest to cobine those results using the weight reflecting ulti-attribute perforance easures such as sensitivity, specificity, bias and variance of isclassification rate defined as follows: (Nuber of observations that predict the event correctly) Sensitivity = (Nuber of observations that represent the event ) (4) (Nuber of observations that predict the event 0 correctly) Specificity = (Nuber of observations that represent the event 0 ) (5) bias 2 [ 2 P(YF = y x ) P(YH = y x )] (6) 2 y Y 2 var iance = x P(YH y x ) (7) 2 y Y where ( Y y x) P F = is the probability that the outcoe of a given case with input x is y while ( Y y x) P H = is the probability that the outcoe of a given case with input x is classified as y. As a eans to obtain an individual weight for each perforance easure, DEA is proposed. Data Envelopent Analysis (DEA) has been frequently applied to assess the efficiency of several decision aking units (DMU) which have ultiple inputs as well as outputs. In our case, individual classifier is considered as DMU while we consider outputs as the four perforance easures of each classifier along with constant inputs. By way of DEA, one can find the efficiency score of each DMU and can figure out the set of efficient DMUs based on the set of non-doinated solution.
5 In addition, DEA provides inefficient DMUs along with bencharking points and has advantages over alternative paraetric approaches such as regression or ratio analysis [2]. We suggest DEA enseble as follows: Step. Choose the achine learning algorith to be used as a classifier. Step 2. Generate training set T by sapling fro T with replaceent, where the probability for sapling training case i is P(i). Step 3. Construct the classifier f using the resapled set T of size N. f Step 4. Evaluate the classifier in ters of ulti-attributes y where it represents the rth output variable of th classifier. In this study, we consider r four attributes sensitivity y r (y), specificity (y2), the squared bias of isclassification (y3) and the variance of isclassification (y4). Then for each unit o we want to find the best weight v i that axiize the weighted output by solving the following atheatical prograing odel: ax h s. t o = r= s r= v s r v v r r y y r r for =, > 0, s =,2,3,4,M L (8) Each DMU o is assigned the highest possible efficiency score that the constraints allow fro the available data by choosing the appropriate virtual ultipliers (weights) for the outputs. Let h o * denote the optial value of h o where 0 h o *. One can say that h o *= and the copleentary slackness conditions of linear prograing are et if and only if unit o is efficient relative to other units considered. On the other hand, if h o *<, then this unit is considered as inefficient which could not achieve a higher rating relative to the reference set to which it is being copared. = ( h, L, h M ) Step 5. Once a set of the efficiency scores of M classifiers h is found,
6 noralize it to ˆ = ( hˆ, hˆ,, hˆ 2 L M ) M h = h so that ˆ = and take a weighted vote of the classifications, with weights ĥ as follows. M f = hˆ f (9) bagging = 4. Nuerical Exaple In this section, we apply the proposed DEA enseble algorith to the actual data for illustration. Sohn and Shin [9] used individual algoriths such as neural network and decision tree to classify the severity of road accidents occurred in Seoul, Korea in 996. Input variables used for classification of two levels of severity (bodily injury and property daage) are road width, shape of car-body, accident category, speed before the accident, violent drive, and protective device. Detailed levels of these input variables are displayed in Table. These variables were selected using decision tree and all turned out to have better explanatory power than variables representing weather conditions. A saple of 564 accidents was taken and 60% of the were used for training while rest of the was used for validation, respectively. Correct classification rates obtained by both classification odels were not significantly different. In order to increase the classification perforance, we use DEA enseble introduced in the previous section. Table Input Variable Description Death Major Injury Minor Injury Injury Report Property Daage Car Alone
7 Accident Type Man to Car Car to Car Velocity Below 20k Below 0k Road Width Car Shape Service Area Over 6 Unknown Bus Figure Decision Tree based on First Bootstrap Resapled Training Data Set We generate 0 Bootstrap resaples and fit decision tree for each resaple. Figure shows a result of fitted tree. This fitted tree classifier is evaluated in ters of sensitivity, specificity, bias and variance of isclassification rate. Obtained values can be used for outputs for DEA. Suarized values for all 0 fitted tree classifiers are displayed in Table 2. We use SAS/OR [8] to solve atheatical prograing for DEA and efficiency score obtained for each classifier is suarized in Table 3. Apparently, classifiers 2,5,6,8 and 9 turn out to be efficient and accordingly their weights are higher than the rest of the reaining classifiers. We then apply these weights to the results of individual classifiers to obtain DEA enseble outcoe. Finally, DEA enseble is copared to a singletree result with respect to ultiple perforance easures.
8 Table 2 Perforance Measures of Each Bootstrap Classifier Table 3 Efficiency Score and Weight for Each Classifier DEA enseble appears to be better in ters of sensitivity and variance. In our case, there was not uch variation aong the ten efficiency scores and therefore the weights were alost even. Note that when the weights are the sae, DEA enseble would be equivalent to Bagging. Table 4 Coparisons of the Classification Results between DEA Enseble and Decision Tree
9 6. Conclusion In this paper, we suggest DEA enseble, which reflects ore than one perforance easures for voting. This is a generalized version of Bagging and is expected to have axiu utilization when there is uch variation in the efficiency scores of individual classifiers. One drawback of the proposed DEA enseble has, as the other enseble is that the cobined rule cannot be explicitly stated. DEA enseble concept suggested can be extended to Arcing by reflecting ultiple perforances when assigning new selection probabilities. This is left as further study areas. References [] Breian, L., Bagging, Boosting, and C4.5, ftp://ftp.stat.berkeley.edu/pub/users /breian, 996. [2] Charnes, A., Cooper, W. W., and Rhodes, E., Measuring the efficiency of decision aking units, European Journal of Operational Research, 2(6): , 978. [3] Freund, Y. and Shapire, R. E., A decision-theoretic generalization of on-line learning and an application to boosting, Journal of Coputer and Syste Sciences, 55:9-39, 995. [4] Hansen, J. V., Cobining predictors: coparison of five eta achine learning ethods, Inforation Sciences, 9:9-05, 999. [5] Kohavi, R., Wolpert, D. H., Bias plus variance decoposition for zero-one loss function, Proceedings of the Thirteenth International Conference on Machine Learning, 996. [6] Opitz, D. W. and Maclin, R. F., An epirical evaluation of bagging and boosting for artificial neural networks, International conference on neural networks, 3:40-405, 997. [7] Quinlan, J. R., Bagging, Boosting, and C4.5 In Proceedings of the Thirteenth National Conference on Artificial Intelligence, 996. Available on line at: [8] SAS/OR Manual, SAS Institute, Cary, NC, USA, 992. [9] Sohn, S. Y. and Shin, H. W., Pattern Recognition for Road Traffic Accident Severity in Korea, Ergonoics, 44():07-7, 200.
Combining Classifiers
Cobining Classifiers Generic ethods of generating and cobining ultiple classifiers Bagging Boosting References: Duda, Hart & Stork, pg 475-480. Hastie, Tibsharini, Friedan, pg 246-256 and Chapter 10. http://www.boosting.org/
More informationPattern Recognition and Machine Learning. Learning and Evaluation for Pattern Recognition
Pattern Recognition and Machine Learning Jaes L. Crowley ENSIMAG 3 - MMIS Fall Seester 2017 Lesson 1 4 October 2017 Outline Learning and Evaluation for Pattern Recognition Notation...2 1. The Pattern Recognition
More informationKernel Methods and Support Vector Machines
Intelligent Systes: Reasoning and Recognition Jaes L. Crowley ENSIAG 2 / osig 1 Second Seester 2012/2013 Lesson 20 2 ay 2013 Kernel ethods and Support Vector achines Contents Kernel Functions...2 Quadratic
More informationLecture 12: Ensemble Methods. Introduction. Weighted Majority. Mixture of Experts/Committee. Σ k α k =1. Isabelle Guyon
Lecture 2: Enseble Methods Isabelle Guyon guyoni@inf.ethz.ch Introduction Book Chapter 7 Weighted Majority Mixture of Experts/Coittee Assue K experts f, f 2, f K (base learners) x f (x) Each expert akes
More informationPattern Recognition and Machine Learning. Artificial Neural networks
Pattern Recognition and Machine Learning Jaes L. Crowley ENSIMAG 3 - MMIS Fall Seester 2017 Lessons 7 20 Dec 2017 Outline Artificial Neural networks Notation...2 Introduction...3 Key Equations... 3 Artificial
More informationIntelligent Systems: Reasoning and Recognition. Perceptrons and Support Vector Machines
Intelligent Systes: Reasoning and Recognition Jaes L. Crowley osig 1 Winter Seester 2018 Lesson 6 27 February 2018 Outline Perceptrons and Support Vector achines Notation...2 Linear odels...3 Lines, Planes
More informationA Smoothed Boosting Algorithm Using Probabilistic Output Codes
A Soothed Boosting Algorith Using Probabilistic Output Codes Rong Jin rongjin@cse.su.edu Dept. of Coputer Science and Engineering, Michigan State University, MI 48824, USA Jian Zhang jian.zhang@cs.cu.edu
More informationIntelligent Systems: Reasoning and Recognition. Artificial Neural Networks
Intelligent Systes: Reasoning and Recognition Jaes L. Crowley MOSIG M1 Winter Seester 2018 Lesson 7 1 March 2018 Outline Artificial Neural Networks Notation...2 Introduction...3 Key Equations... 3 Artificial
More informationE0 370 Statistical Learning Theory Lecture 6 (Aug 30, 2011) Margin Analysis
E0 370 tatistical Learning Theory Lecture 6 (Aug 30, 20) Margin Analysis Lecturer: hivani Agarwal cribe: Narasihan R Introduction In the last few lectures we have seen how to obtain high confidence bounds
More informationSupport Vector Machine Classification of Uncertain and Imbalanced data using Robust Optimization
Recent Researches in Coputer Science Support Vector Machine Classification of Uncertain and Ibalanced data using Robust Optiization RAGHAV PAT, THEODORE B. TRAFALIS, KASH BARKER School of Industrial Engineering
More informationNon-Parametric Non-Line-of-Sight Identification 1
Non-Paraetric Non-Line-of-Sight Identification Sinan Gezici, Hisashi Kobayashi and H. Vincent Poor Departent of Electrical Engineering School of Engineering and Applied Science Princeton University, Princeton,
More informationSupport Vector Machines MIT Course Notes Cynthia Rudin
Support Vector Machines MIT 5.097 Course Notes Cynthia Rudin Credit: Ng, Hastie, Tibshirani, Friedan Thanks: Şeyda Ertekin Let s start with soe intuition about argins. The argin of an exaple x i = distance
More informationA MESHSIZE BOOSTING ALGORITHM IN KERNEL DENSITY ESTIMATION
A eshsize boosting algorith in kernel density estiation A MESHSIZE BOOSTING ALGORITHM IN KERNEL DENSITY ESTIMATION C.C. Ishiekwene, S.M. Ogbonwan and J.E. Osewenkhae Departent of Matheatics, University
More informationPattern Recognition and Machine Learning. Artificial Neural networks
Pattern Recognition and Machine Learning Jaes L. Crowley ENSIMAG 3 - MMIS Fall Seester 2016 Lessons 7 14 Dec 2016 Outline Artificial Neural networks Notation...2 1. Introduction...3... 3 The Artificial
More informationBoosting with log-loss
Boosting with log-loss Marco Cusuano-Towner Septeber 2, 202 The proble Suppose we have data exaples {x i, y i ) i =... } for a two-class proble with y i {, }. Let F x) be the predictor function with the
More informationA note on the multiplication of sparse matrices
Cent. Eur. J. Cop. Sci. 41) 2014 1-11 DOI: 10.2478/s13537-014-0201-x Central European Journal of Coputer Science A note on the ultiplication of sparse atrices Research Article Keivan Borna 12, Sohrab Aboozarkhani
More informationMSEC MODELING OF DEGRADATION PROCESSES TO OBTAIN AN OPTIMAL SOLUTION FOR MAINTENANCE AND PERFORMANCE
Proceeding of the ASME 9 International Manufacturing Science and Engineering Conference MSEC9 October 4-7, 9, West Lafayette, Indiana, USA MSEC9-8466 MODELING OF DEGRADATION PROCESSES TO OBTAIN AN OPTIMAL
More informationTesting equality of variances for multiple univariate normal populations
University of Wollongong Research Online Centre for Statistical & Survey Methodology Working Paper Series Faculty of Engineering and Inforation Sciences 0 esting equality of variances for ultiple univariate
More informationModel Fitting. CURM Background Material, Fall 2014 Dr. Doreen De Leon
Model Fitting CURM Background Material, Fall 014 Dr. Doreen De Leon 1 Introduction Given a set of data points, we often want to fit a selected odel or type to the data (e.g., we suspect an exponential
More informationE. Alpaydın AERFAISS
E. Alpaydın AERFAISS 00 Introduction Questions: Is the error rate of y classifier less than %? Is k-nn ore accurate than MLP? Does having PCA before iprove accuracy? Which kernel leads to highest accuracy
More informationFoundations of Machine Learning Boosting. Mehryar Mohri Courant Institute and Google Research
Foundations of Machine Learning Boosting Mehryar Mohri Courant Institute and Google Research ohri@cis.nyu.edu Weak Learning Definition: concept class C is weakly PAC-learnable if there exists a (weak)
More informationSupport Vector Machines. Goals for the lecture
Support Vector Machines Mark Craven and David Page Coputer Sciences 760 Spring 2018 www.biostat.wisc.edu/~craven/cs760/ Soe of the slides in these lectures have been adapted/borrowed fro aterials developed
More informationInspection; structural health monitoring; reliability; Bayesian analysis; updating; decision analysis; value of information
Cite as: Straub D. (2014). Value of inforation analysis with structural reliability ethods. Structural Safety, 49: 75-86. Value of Inforation Analysis with Structural Reliability Methods Daniel Straub
More informationESTIMATING AND FORMING CONFIDENCE INTERVALS FOR EXTREMA OF RANDOM POLYNOMIALS. A Thesis. Presented to. The Faculty of the Department of Mathematics
ESTIMATING AND FORMING CONFIDENCE INTERVALS FOR EXTREMA OF RANDOM POLYNOMIALS A Thesis Presented to The Faculty of the Departent of Matheatics San Jose State University In Partial Fulfillent of the Requireents
More informationSupport Vector Machines. Maximizing the Margin
Support Vector Machines Support vector achines (SVMs) learn a hypothesis: h(x) = b + Σ i= y i α i k(x, x i ) (x, y ),..., (x, y ) are the training exs., y i {, } b is the bias weight. α,..., α are the
More informationA Simple Regression Problem
A Siple Regression Proble R. M. Castro March 23, 2 In this brief note a siple regression proble will be introduced, illustrating clearly the bias-variance tradeoff. Let Y i f(x i ) + W i, i,..., n, where
More informatione-companion ONLY AVAILABLE IN ELECTRONIC FORM
OPERATIONS RESEARCH doi 10.1287/opre.1070.0427ec pp. ec1 ec5 e-copanion ONLY AVAILABLE IN ELECTRONIC FORM infors 07 INFORMS Electronic Copanion A Learning Approach for Interactive Marketing to a Custoer
More informationRegularization and Averaging of the Selective Naïve Bayes classifier
Regularization and Averaging of the Selective Naïve Bayes classifier Marc Boullé Abstract The Naïve Bayes classifier has proved to be very effective on any real data applications. Its perforances usually
More informationUNIVERSITY OF TRENTO ON THE USE OF SVM FOR ELECTROMAGNETIC SUBSURFACE SENSING. A. Boni, M. Conci, A. Massa, and S. Piffer.
UIVRSITY OF TRTO DIPARTITO DI IGGRIA SCIZA DLL IFORAZIO 3823 Povo Trento (Italy) Via Soarive 4 http://www.disi.unitn.it O TH US OF SV FOR LCTROAGTIC SUBSURFAC SSIG A. Boni. Conci A. assa and S. Piffer
More informationPrincipal Components Analysis
Principal Coponents Analysis Cheng Li, Bingyu Wang Noveber 3, 204 What s PCA Principal coponent analysis (PCA) is a statistical procedure that uses an orthogonal transforation to convert a set of observations
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Notes for EE227C (Spring 2018): Convex Optiization and Approxiation Instructor: Moritz Hardt Eail: hardt+ee227c@berkeley.edu Graduate Instructor: Max Sichowitz Eail: sichow+ee227c@berkeley.edu October
More informationWhat is Probability? (again)
INRODUCTION TO ROBBILITY Basic Concepts and Definitions n experient is any process that generates well-defined outcoes. Experient: Record an age Experient: Toss a die Experient: Record an opinion yes,
More informationBayes Decision Rule and Naïve Bayes Classifier
Bayes Decision Rule and Naïve Bayes Classifier Le Song Machine Learning I CSE 6740, Fall 2013 Gaussian Mixture odel A density odel p(x) ay be ulti-odal: odel it as a ixture of uni-odal distributions (e.g.
More informationCOS 424: Interacting with Data. Written Exercises
COS 424: Interacting with Data Hoework #4 Spring 2007 Regression Due: Wednesday, April 18 Written Exercises See the course website for iportant inforation about collaboration and late policies, as well
More informationBayesian Approach for Fatigue Life Prediction from Field Inspection
Bayesian Approach for Fatigue Life Prediction fro Field Inspection Dawn An and Jooho Choi School of Aerospace & Mechanical Engineering, Korea Aerospace University, Goyang, Seoul, Korea Srira Pattabhiraan
More information1 Bounding the Margin
COS 511: Theoretical Machine Learning Lecturer: Rob Schapire Lecture #12 Scribe: Jian Min Si March 14, 2013 1 Bounding the Margin We are continuing the proof of a bound on the generalization error of AdaBoost
More informationFault Diagnosis of Planetary Gear Based on Fuzzy Entropy of CEEMDAN and MLP Neural Network by Using Vibration Signal
ITM Web of Conferences 11, 82 (217) DOI: 1.151/ itconf/2171182 IST217 Fault Diagnosis of Planetary Gear Based on Fuzzy Entropy of CEEMDAN and MLP Neural Networ by Using Vibration Signal Xi-Hui CHEN, Gang
More informationThe Algorithms Optimization of Artificial Neural Network Based on Particle Swarm
Send Orders for Reprints to reprints@benthascience.ae The Open Cybernetics & Systeics Journal, 04, 8, 59-54 59 Open Access The Algoriths Optiization of Artificial Neural Network Based on Particle Swar
More informationMachine Learning Basics: Estimators, Bias and Variance
Machine Learning Basics: Estiators, Bias and Variance Sargur N. srihari@cedar.buffalo.edu This is part of lecture slides on Deep Learning: http://www.cedar.buffalo.edu/~srihari/cse676 1 Topics in Basics
More informationRademacher Complexity Margin Bounds for Learning with a Large Number of Classes
Radeacher Coplexity Margin Bounds for Learning with a Large Nuber of Classes Vitaly Kuznetsov Courant Institute of Matheatical Sciences, 25 Mercer street, New York, NY, 002 Mehryar Mohri Courant Institute
More informationA Self-Organizing Model for Logical Regression Jerry Farlow 1 University of Maine. (1900 words)
1 A Self-Organizing Model for Logical Regression Jerry Farlow 1 University of Maine (1900 words) Contact: Jerry Farlow Dept of Matheatics Univeristy of Maine Orono, ME 04469 Tel (07) 866-3540 Eail: farlow@ath.uaine.edu
More informationFeature Extraction Techniques
Feature Extraction Techniques Unsupervised Learning II Feature Extraction Unsupervised ethods can also be used to find features which can be useful for categorization. There are unsupervised ethods that
More informationAn improved self-adaptive harmony search algorithm for joint replenishment problems
An iproved self-adaptive harony search algorith for joint replenishent probles Lin Wang School of Manageent, Huazhong University of Science & Technology zhoulearner@gail.co Xiaojian Zhou School of Manageent,
More informationA LOSS FUNCTION APPROACH TO GROUP PREFERENCE AGGREGATION IN THE AHP
ISAHP 003, Bali, Indonesia, August 7-9, 003 A OSS FUNCTION APPROACH TO GROUP PREFERENCE AGGREGATION IN THE AHP Keun-Tae Cho and Yong-Gon Cho School of Systes Engineering Manageent, Sungkyunkwan University
More informationZISC Neural Network Base Indicator for Classification Complexity Estimation
ZISC Neural Network Base Indicator for Classification Coplexity Estiation Ivan Budnyk, Abdennasser Сhebira and Kurosh Madani Iages, Signals and Intelligent Systes Laboratory (LISSI / EA 3956) PARIS XII
More informationExtension of CSRSM for the Parametric Study of the Face Stability of Pressurized Tunnels
Extension of CSRSM for the Paraetric Study of the Face Stability of Pressurized Tunnels Guilhe Mollon 1, Daniel Dias 2, and Abdul-Haid Soubra 3, M.ASCE 1 LGCIE, INSA Lyon, Université de Lyon, Doaine scientifique
More informationInteractive Markov Models of Evolutionary Algorithms
Cleveland State University EngagedScholarship@CSU Electrical Engineering & Coputer Science Faculty Publications Electrical Engineering & Coputer Science Departent 2015 Interactive Markov Models of Evolutionary
More informationThis model assumes that the probability of a gap has size i is proportional to 1/i. i.e., i log m e. j=1. E[gap size] = i P r(i) = N f t.
CS 493: Algoriths for Massive Data Sets Feb 2, 2002 Local Models, Bloo Filter Scribe: Qin Lv Local Models In global odels, every inverted file entry is copressed with the sae odel. This work wells when
More informationGrafting: Fast, Incremental Feature Selection by Gradient Descent in Function Space
Journal of Machine Learning Research 3 (2003) 1333-1356 Subitted 5/02; Published 3/03 Grafting: Fast, Increental Feature Selection by Gradient Descent in Function Space Sion Perkins Space and Reote Sensing
More informationAn Improved Particle Filter with Applications in Ballistic Target Tracking
Sensors & ransducers Vol. 72 Issue 6 June 204 pp. 96-20 Sensors & ransducers 204 by IFSA Publishing S. L. http://www.sensorsportal.co An Iproved Particle Filter with Applications in Ballistic arget racing
More informationAn Introduction to Meta-Analysis
An Introduction to Meta-Analysis Douglas G. Bonett University of California, Santa Cruz How to cite this work: Bonett, D.G. (2016) An Introduction to Meta-analysis. Retrieved fro http://people.ucsc.edu/~dgbonett/eta.htl
More informationQualitative Modelling of Time Series Using Self-Organizing Maps: Application to Animal Science
Proceedings of the 6th WSEAS International Conference on Applied Coputer Science, Tenerife, Canary Islands, Spain, Deceber 16-18, 2006 183 Qualitative Modelling of Tie Series Using Self-Organizing Maps:
More informationKernel-Based Nonparametric Anomaly Detection
Kernel-Based Nonparaetric Anoaly Detection Shaofeng Zou Dept of EECS Syracuse University Eail: szou@syr.edu Yingbin Liang Dept of EECS Syracuse University Eail: yliang6@syr.edu H. Vincent Poor Dept of
More informationComputational and Statistical Learning Theory
Coputational and Statistical Learning Theory Proble sets 5 and 6 Due: Noveber th Please send your solutions to learning-subissions@ttic.edu Notations/Definitions Recall the definition of saple based Radeacher
More informationLogLog-Beta and More: A New Algorithm for Cardinality Estimation Based on LogLog Counting
LogLog-Beta and More: A New Algorith for Cardinality Estiation Based on LogLog Counting Jason Qin, Denys Ki, Yuei Tung The AOLP Core Data Service, AOL, 22000 AOL Way Dulles, VA 20163 E-ail: jasonqin@teaaolco
More informationDEPARTMENT OF ECONOMETRICS AND BUSINESS STATISTICS
ISSN 1440-771X AUSTRALIA DEPARTMENT OF ECONOMETRICS AND BUSINESS STATISTICS An Iproved Method for Bandwidth Selection When Estiating ROC Curves Peter G Hall and Rob J Hyndan Working Paper 11/00 An iproved
More informationOBJECTIVES INTRODUCTION
M7 Chapter 3 Section 1 OBJECTIVES Suarize data using easures of central tendency, such as the ean, edian, ode, and idrange. Describe data using the easures of variation, such as the range, variance, and
More informationSPECTRUM sensing is a core concept of cognitive radio
World Acadey of Science, Engineering and Technology International Journal of Electronics and Counication Engineering Vol:6, o:2, 202 Efficient Detection Using Sequential Probability Ratio Test in Mobile
More informationINTELLECTUAL DATA ANALYSIS IN AIRCRAFT DESIGN
INTELLECTUAL DATA ANALYSIS IN AIRCRAFT DESIGN V.A. Koarov 1, S.A. Piyavskiy 2 1 Saara National Research University, Saara, Russia 2 Saara State Architectural University, Saara, Russia Abstract. This article
More informationBest Arm Identification: A Unified Approach to Fixed Budget and Fixed Confidence
Best Ar Identification: A Unified Approach to Fixed Budget and Fixed Confidence Victor Gabillon Mohaad Ghavazadeh Alessandro Lazaric INRIA Lille - Nord Europe, Tea SequeL {victor.gabillon,ohaad.ghavazadeh,alessandro.lazaric}@inria.fr
More informationAnalyzing Simulation Results
Analyzing Siulation Results Dr. John Mellor-Cruey Departent of Coputer Science Rice University johnc@cs.rice.edu COMP 528 Lecture 20 31 March 2005 Topics for Today Model verification Model validation Transient
More informationEffective joint probabilistic data association using maximum a posteriori estimates of target states
Effective joint probabilistic data association using axiu a posteriori estiates of target states 1 Viji Paul Panakkal, 2 Rajbabu Velurugan 1 Central Research Laboratory, Bharat Electronics Ltd., Bangalore,
More informationSupport Vector Machines. Machine Learning Series Jerry Jeychandra Blohm Lab
Support Vector Machines Machine Learning Series Jerry Jeychandra Bloh Lab Outline Main goal: To understand how support vector achines (SVMs) perfor optial classification for labelled data sets, also a
More informationThe Weierstrass Approximation Theorem
36 The Weierstrass Approxiation Theore Recall that the fundaental idea underlying the construction of the real nubers is approxiation by the sipler rational nubers. Firstly, nubers are often deterined
More informationSoft Computing Techniques Help Assign Weights to Different Factors in Vulnerability Analysis
Soft Coputing Techniques Help Assign Weights to Different Factors in Vulnerability Analysis Beverly Rivera 1,2, Irbis Gallegos 1, and Vladik Kreinovich 2 1 Regional Cyber and Energy Security Center RCES
More informationExperimental Design For Model Discrimination And Precise Parameter Estimation In WDS Analysis
City University of New York (CUNY) CUNY Acadeic Works International Conference on Hydroinforatics 8-1-2014 Experiental Design For Model Discriination And Precise Paraeter Estiation In WDS Analysis Giovanna
More informationare equal to zero, where, q = p 1. For each gene j, the pairwise null and alternative hypotheses are,
Page of 8 Suppleentary Materials: A ultiple testing procedure for ulti-diensional pairwise coparisons with application to gene expression studies Anjana Grandhi, Wenge Guo, Shyaal D. Peddada S Notations
More informationEstimating Parameters for a Gaussian pdf
Pattern Recognition and achine Learning Jaes L. Crowley ENSIAG 3 IS First Seester 00/0 Lesson 5 7 Noveber 00 Contents Estiating Paraeters for a Gaussian pdf Notation... The Pattern Recognition Proble...3
More informationTracking using CONDENSATION: Conditional Density Propagation
Tracking using CONDENSATION: Conditional Density Propagation Goal Model-based visual tracking in dense clutter at near video frae rates M. Isard and A. Blake, CONDENSATION Conditional density propagation
More informationSoft-margin SVM can address linearly separable problems with outliers
Non-linear Support Vector Machines Non-linearly separable probles Hard-argin SVM can address linearly separable probles Soft-argin SVM can address linearly separable probles with outliers Non-linearly
More information3.3 Variational Characterization of Singular Values
3.3. Variational Characterization of Singular Values 61 3.3 Variational Characterization of Singular Values Since the singular values are square roots of the eigenvalues of the Heritian atrices A A and
More informationPh 20.3 Numerical Solution of Ordinary Differential Equations
Ph 20.3 Nuerical Solution of Ordinary Differential Equations Due: Week 5 -v20170314- This Assignent So far, your assignents have tried to failiarize you with the hardware and software in the Physics Coputing
More information1 Proof of learning bounds
COS 511: Theoretical Machine Learning Lecturer: Rob Schapire Lecture #4 Scribe: Akshay Mittal February 13, 2013 1 Proof of learning bounds For intuition of the following theore, suppose there exists a
More informationElliptic Curve Scalar Point Multiplication Algorithm Using Radix-4 Booth s Algorithm
Elliptic Curve Scalar Multiplication Algorith Using Radix-4 Booth s Algorith Elliptic Curve Scalar Multiplication Algorith Using Radix-4 Booth s Algorith Sangook Moon, Non-eber ABSTRACT The ain back-bone
More informationGeneralized Augmentation for Control of the k-familywise Error Rate
International Journal of Statistics in Medical Research, 2012, 1, 113-119 113 Generalized Augentation for Control of the k-failywise Error Rate Alessio Farcoeni* Departent of Public Health and Infectious
More informationCh 12: Variations on Backpropagation
Ch 2: Variations on Backpropagation The basic backpropagation algorith is too slow for ost practical applications. It ay take days or weeks of coputer tie. We deonstrate why the backpropagation algorith
More informationPattern Classification using Simplified Neural Networks with Pruning Algorithm
Pattern Classification using Siplified Neural Networks with Pruning Algorith S. M. Karuzzaan 1 Ahed Ryadh Hasan 2 Abstract: In recent years, any neural network odels have been proposed for pattern classification,
More informationProbability Distributions
Probability Distributions In Chapter, we ephasized the central role played by probability theory in the solution of pattern recognition probles. We turn now to an exploration of soe particular exaples
More informationCompression and Predictive Distributions for Large Alphabet i.i.d and Markov models
2014 IEEE International Syposiu on Inforation Theory Copression and Predictive Distributions for Large Alphabet i.i.d and Markov odels Xiao Yang Departent of Statistics Yale University New Haven, CT, 06511
More informationNOTES AND CORRESPONDENCE. Two Extra Components in the Brier Score Decomposition
752 W E A T H E R A N D F O R E C A S T I N G VOLUME 23 NOTES AND CORRESPONDENCE Two Extra Coponents in the Brier Score Decoposition D. B. STEPHENSON School of Engineering, Coputing, and Matheatics, University
More informationFairness via priority scheduling
Fairness via priority scheduling Veeraruna Kavitha, N Heachandra and Debayan Das IEOR, IIT Bobay, Mubai, 400076, India vavitha,nh,debayan}@iitbacin Abstract In the context of ulti-agent resource allocation
More informationarxiv: v3 [cs.ds] 22 Mar 2016
A Shifting Bloo Filter Fraewor for Set Queries arxiv:1510.03019v3 [cs.ds] Mar 01 ABSTRACT Tong Yang Peing University, China yangtongeail@gail.co Yuanun Zhong Nanjing University, China un@sail.nju.edu.cn
More informationREDUCTION OF FINITE ELEMENT MODELS BY PARAMETER IDENTIFICATION
ISSN 139 14X INFORMATION TECHNOLOGY AND CONTROL, 008, Vol.37, No.3 REDUCTION OF FINITE ELEMENT MODELS BY PARAMETER IDENTIFICATION Riantas Barauskas, Vidantas Riavičius Departent of Syste Analysis, Kaunas
More information1 Generalization bounds based on Rademacher complexity
COS 5: Theoretical Machine Learning Lecturer: Rob Schapire Lecture #0 Scribe: Suqi Liu March 07, 08 Last tie we started proving this very general result about how quickly the epirical average converges
More informationBest Procedures For Sample-Free Item Analysis
Best Procedures For Saple-Free Ite Analysis Benjain D. Wright University of Chicago Graha A. Douglas University of Western Australia Wright s (1969) widely used "unconditional" procedure for Rasch saple-free
More informationIdentical Maximum Likelihood State Estimation Based on Incremental Finite Mixture Model in PHD Filter
Identical Maxiu Lielihood State Estiation Based on Increental Finite Mixture Model in PHD Filter Gang Wu Eail: xjtuwugang@gail.co Jing Liu Eail: elelj20080730@ail.xjtu.edu.cn Chongzhao Han Eail: czhan@ail.xjtu.edu.cn
More informationPAC-Bayes Analysis Of Maximum Entropy Learning
PAC-Bayes Analysis Of Maxiu Entropy Learning John Shawe-Taylor and David R. Hardoon Centre for Coputational Statistics and Machine Learning Departent of Coputer Science University College London, UK, WC1E
More informationOn Constant Power Water-filling
On Constant Power Water-filling Wei Yu and John M. Cioffi Electrical Engineering Departent Stanford University, Stanford, CA94305, U.S.A. eails: {weiyu,cioffi}@stanford.edu Abstract This paper derives
More informationWhen Short Runs Beat Long Runs
When Short Runs Beat Long Runs Sean Luke George Mason University http://www.cs.gu.edu/ sean/ Abstract What will yield the best results: doing one run n generations long or doing runs n/ generations long
More informationW-BASED VS LATENT VARIABLES SPATIAL AUTOREGRESSIVE MODELS: EVIDENCE FROM MONTE CARLO SIMULATIONS
W-BASED VS LATENT VARIABLES SPATIAL AUTOREGRESSIVE MODELS: EVIDENCE FROM MONTE CARLO SIMULATIONS. Introduction When it coes to applying econoetric odels to analyze georeferenced data, researchers are well
More informationPattern Recognition and Machine Learning. Artificial Neural networks
Pattern Recognition and Machine Learning Jaes L. Crowley ENSIMAG 3 - MMIS Fall Seester 2016/2017 Lessons 9 11 Jan 2017 Outline Artificial Neural networks Notation...2 Convolutional Neural Networks...3
More informationDepartment of Electronic and Optical Engineering, Ordnance Engineering College, Shijiazhuang, , China
6th International Conference on Machinery, Materials, Environent, Biotechnology and Coputer (MMEBC 06) Solving Multi-Sensor Multi-Target Assignent Proble Based on Copositive Cobat Efficiency and QPSO Algorith
More informationStochastic Subgradient Methods
Stochastic Subgradient Methods Lingjie Weng Yutian Chen Bren School of Inforation and Coputer Science University of California, Irvine {wengl, yutianc}@ics.uci.edu Abstract Stochastic subgradient ethods
More informationMulti-view Discriminative Manifold Embedding for Pattern Classification
Multi-view Discriinative Manifold Ebedding for Pattern Classification X. Wang Departen of Inforation Zhenghzou 450053, China Y. Guo Departent of Digestive Zhengzhou 450053, China Z. Wang Henan University
More informationLONG-TERM PREDICTIVE VALUE INTERVAL WITH THE FUZZY TIME SERIES
Journal of Marine Science and Technology, Vol 19, No 5, pp 509-513 (2011) 509 LONG-TERM PREDICTIVE VALUE INTERVAL WITH THE FUZZY TIME SERIES Ming-Tao Chou* Key words: fuzzy tie series, fuzzy forecasting,
More informationIAENG International Journal of Computer Science, 42:2, IJCS_42_2_06. Approximation Capabilities of Interpretable Fuzzy Inference Systems
IAENG International Journal of Coputer Science, 4:, IJCS_4 6 Approxiation Capabilities of Interpretable Fuzzy Inference Systes Hirofui Miyajia, Noritaka Shigei, and Hiroi Miyajia 3 Abstract Many studies
More informationDeep Boosting. Abstract. 1. Introduction
Corinna Cortes Google Research, 8th Avenue, New York, NY Mehryar Mohri Courant Institute and Google Research, 25 Mercer Street, New York, NY 2 Uar Syed Google Research, 8th Avenue, New York, NY Abstract
More informationUsing a De-Convolution Window for Operating Modal Analysis
Using a De-Convolution Window for Operating Modal Analysis Brian Schwarz Vibrant Technology, Inc. Scotts Valley, CA Mark Richardson Vibrant Technology, Inc. Scotts Valley, CA Abstract Operating Modal Analysis
More informationFast Structural Similarity Search of Noncoding RNAs Based on Matched Filtering of Stem Patterns
Fast Structural Siilarity Search of Noncoding RNs Based on Matched Filtering of Ste Patterns Byung-Jun Yoon Dept. of Electrical Engineering alifornia Institute of Technology Pasadena, 91125, S Eail: bjyoon@caltech.edu
More informationUnderstanding Machine Learning Solution Manual
Understanding Machine Learning Solution Manual Written by Alon Gonen Edited by Dana Rubinstein Noveber 17, 2014 2 Gentle Start 1. Given S = ((x i, y i )), define the ultivariate polynoial p S (x) = i []:y
More information