Tree Structured Classifier
|
|
- Angelica Walsh
- 5 years ago
- Views:
Transcription
1 Tree Structured Classifier Reference: Classificatin and Regressin Trees by L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stne, Chapman & Hall, 98. A Medical Eample (CART): Predict high risk patients wh will nt survive at least 3 days n the basis f the initial 2-hur data. 9 variables are measured during the first 2 hurs. These include bld pressure, age, etc. A tree structure classificatin rule is as fllws: Is the minimum systlic bld pressure ver the initial 2 hur perid > 9? yes n Is sinus tachycardia present? Is age > 2.5 yes yes n n Lw risk High risk High risk Lw risk
2 Dente the feature space by X. The input vectr X X cntains p features X, X 2,..., X p, sme f which may be categrical. Tree structured classifiers are cnstructed by repeated splits f subsets f X int tw descendant subsets, beginning with X itself. Definitins: nde, terminal nde (leaf nde), parent nde, child nde. The unin f the regins ccupied by tw child ndes is the regin ccupied by their parent nde. Every leaf nde is assigned with a class. A query is assciated with class f the leaf nde it lands in. Ntatin: A nde is dented by t. Its left child nde is dented by t L and right by t R. The cllectin f all the ndes is dented by T ; and the cllectin f all the leaf ndes by T. A split is dented by s. The set f splits is dented by S. 2
3 X split X X 2 X X 2 X X 3 split 2 split 3 X X 5 X X X 2 X 3 X X 5 X X 7 X X 3 X X 5 X 8 X X 2 X 3 X X 5 X split 2 2 X 7 X 8 3 3
4 The Three Elements The cnstructin f a tree invlves the fllwing three elements:. The selectin f the splits. 2. The decisins when t declare a nde terminal r t cntinue splitting it. 3. The assignment f each terminal nde t a class. In particular, we need t decide the fllwing:. A set Q f binary questins f the frm {Is X A?}, A X. 2. A gdness f split criterin Φ(s, t) that can be evaluated fr any split s f any nde t. 3. A stp-splitting rule.. A rule fr assigning every terminal nde t a class.
5 Standard Set f Questins The input vectr X = (X,X 2,...,X p ) cntains features f bth categrical and rdered types. Each split depends n the value f nly a unique variable. Fr each rdered variable X j, Q includes all questins f the frm fr all real-valued c. {Is X j c?} Since the training data set is finite, there are nly finitely many distinct splits that can be generated by the questin {Is X j c?}. If X j is categrical, taking values, say in{, 2,...,M}, then Q cntains all questins f the frm {Is X j A?}. A ranges ver all subsets f {, 2,...,M}. The splits fr all p variables cnstitute the standard set f questins. 5
6 Gdness f Split The gdness f split is measured by an impurity functin defined fr each nde. Intuitively, we want each leaf nde t be pure, that is, ne class dminates. Definitin: An impurity functin is a functin φ defined n the set f all K-tuples f numbers (p,...,p K ) satisfying p j, j =,..., K, j p j = with the prperties:. φ is a maimum nly at the pint ( K, K,..., K ). 2. φ achieves its minimum nly at the pints (,,..., ), (,,,..., ),..., (,,...,, ). 3. φ is a symmetric functin f p,..., p K, i.e., if yu permute p j, φ remains cnstant.
7 Definitin: Given an impurity functin φ, define the impurity measure i(t) f a nde t as i(t) = φ(p( t),p(2 t),...,p(k t)), where p(j t) is the estimated prbability f class j within nde t. Gdness f a split s fr nde t, dented by Φ(s, t), is defined by Φ(s, t) = i(s, t) = i(t) p R i(t R ) p L i(t L ), where p R and p L are the prprtins f the samples in nde t that g t the right nde t R and the left nde t L respectively. 7
8 Define I(t) = i(t)p(t), that is, the impurity functin f nde t weighted by the estimated prprtin f data that g t nde t. The impurity f tree T, I(T) is defined by I(T) = t T I(t) = t T i(t)p(t). Nte fr any nde t the fllwing equatins hld: Define p(t L ) + p(t R ) = p(t) p L = p(t L )/p(t), p R = p(t R )/p(t) p L + p R = I(s, t) = I(t) I(t L ) I(t R ) = p(t)i(t) p(t L )i(t L ) p(t R )i(t R ) = p(t)(i(t) p L i(t L ) p R i(t R )) = p(t) i(s, t) 8
9 Pssible impurity functin:. Entrpy: K j= p j lg p j. If p j =, use the limit lim pj p j lg p j =. 2. Misclassificatin rate: ma j p j. 3. Gini inde: K j= p j( p j ) = K j= p2 j. Gini inde seems t wrk best in practice fr many prblems. The twing rule: At a nde t, chse the split s that maimizes p L p R 2 p(j t L ) p(j t R ). j 9
10 Estimate the psterir prbabilities f classes in each nde: The ttal number f samples is N and the number f samples in class j, j K, is N j. The number f samples ging t nde t is N(t); the number f samples with class j ging t nde t is N j (t). K j= N j(t) = N(t). N j (t L ) + N j (t R ) = N j (t). Fr a full tree (balanced), the sum f N(t) ver all the t s at the same level is N. Dente the prir prbability f class j by π j. The prirs π j can be estimated frm the data by N j /N. Smetimes prirs are given befre-hand. The estimated prbability f a sample in class j ging t nde t is p(t j) = N j (t)/n j. p(t L j) + p(t R j) = p(t j). Fr a full tree, the sum f p(t j) ver all t s at the same level is.
11 The jint prbability f a sample being in class j and ging t nde t is thus: p(j,t) = π j p(t j) = π j N j (t)/n j. The prbability f any sample ging t nde t is: p(t) = K p(j,t) = j= K π j N j (t)/n j. j= Nte p(t L ) + p(t R ) = p(t). The prbability f a sample being in class j given that it ges t nde t is: p(j t) = p(j,t)/p(t). Fr any t, K j= p(j t) =. When π j = N j /N, we have the fllwing simplificatin: p(j t) = N j (t)/n(t). p(t) = N(t)/N. p(j, t) = N j (t)/n.
12 Stpping Criteria A simple criteria: stp splitting a nde t when ma s S I(s,t) < β, where β is a chsen threshld. The abve stpping criteria is unsatisfactry. A nde with a small decrease f impurity after ne step f splitting may have a large decrease after multiple levels f splits. 2
13 Class Assignment Rule A class assignment rule assigns a class j = {,...,K} t every terminal nde t T. The class assigned t nde t T is dented by κ(t). Fr - lss, the class assignment rule is: κ(t) = arg ma j p(j t). The resubstitutin estimate r(t) f the prbability f misclassificatin, given that a case falls int nde t is r(t) = ma j Dente R(t) = r(t)p(t). p(j t) = p(κ(t) t). The resubstitutin estimate fr the verall misclassificatin rate R(T) f the tree classifier T is: R(T) = t T R(t). 3
14 Prpsitin: Fr any split f a nde t int t L and t R, Prf: Dente j = κ(t). R(t) R(t L ) + R(t R ). p(j t) = p(j,t L t) + p(j,t R t) = p(j t L )p(t L t) + p(j t R )p(t R t) = p L p(j t L ) + p R p(j t R ) Hence, p L ma j p(j t L ) + p R ma j p(j t R ) r(t) = p(j [ t) ] p L map(j t L ) + p R map(j t R ) j j = p L ( map(j t L )) + p R ( map(j t R )) j j = p L r(t L ) + p R r(t R ) Finally, R(t) = p(t)r(t) p(t)p L r(t L ) + p(t)p R r(t R ) = p(t L )r(t L ) + p(t R )r(t R ) = R(t L ) + R(t R )
15 Digit Recgnitin Eample (CART) The digits are shwn by different n-ff cmbinatins f seven hrizntal and vertical lights. Each digit is represented by a 7-dimensinal vectr f zers and nes. The ith sample is i = ( i, i2,..., i7 ). If ij =, the jth light is n; if ij =, the jth light is ff. Digit
16 The data fr the eample are generated by a malfunctining calculatr. Each f the seven lights has prbability. f being in the wrng state independently. The training set cntains 2 samples generated accrding t the specified distributin. A tree structured classifier is applied. The set f questins Q cntains: Is j =?, j =, 2,..., 7. The twing rule is used in splitting. The pruning crss-validatin methd is used t chse the right sized tree. Classificatin perfrmance: The errr rate estimated by using a test set f size 5 is.3. The errr rate estimated by crss-validatin using the training set is.3. The resubstitutin estimate f the errr rate is.29. The Bayes errr rate is.2. There is little rm fr imprvement ver the tree classifier.
17 Y X5= N Y X= N Y X2= N 2 Y X= Y X2= N Y X= N 7 3 Y X= N Y X3= N 8 Y X3= N 5 9 Accidently, every digit ccupies ne leaf nde. In general, ne class may ccupy any number f leaf ndes and ccasinally n leaf nde. X and X 7 are never used. 7
18 Wavefrm Eample (CART) Three functins h (τ), h 2 (τ), h 3 (τ) are shifted versins f each ther, as shwn in the figure. h h 3 h Each h j is specified by the equal-lateral right triangle functin. Its values at integers τ = 2 are measured. 8
19 The three classes f wavefrms are randm cnve cmbinatins f tw f these wavefrms plus independent Gaussian nise. Each sample is a 2 dimensinal vectr cntaining the values f the randm wavefrms measured at τ =, 2,..., 2. T generate a sample in class, a randm number u unifrmly distributed in [, ] and 2 randm numbers ɛ, ɛ 2,..., ɛ 2 nrmally distributed with mean zer and variance are generated. j = uh (j) + ( u)h 2 (j) + ɛ j, j =,..., 2. T generate a sample in class 2, repeat the abve prcess t generate a randm number u and 2 randm numbers ɛ,..., ɛ 2 and set j = uh (j) + ( u)h 3 (j) + ɛ j, j =,..., 2. Class 3 vectrs are generated by j = uh 2 (j) + ( u)h 3 (j) + ɛ j, j =,..., 2. Eample randm wavefrms are shwn belw. 9
20 Class Class Class
21 3 randm samples are generated using prir prbabilities ( 3, 3, 3 ) fr training. Cnstructin f the tree: The set f questins: {Is j c?} fr c ranging ver all real numbers and j =,..., 2. Gini inde is used fr measuring gdness f split. The final tree is selected by pruning and crssvalidatin. Results: The crss-validatin estimate f misclassificatin rate is.29. The misclassificatin rate n a separate test set f size 5 is.28. The Bayes classificatin rule can be derived. Applying this rule t the test set yields a misclassificatin rate f.. 2
22 <=2. <=.8 2<=. 7<=. <=2.5 <=2. 7<=.9 <=. 5<=.9 <=
23 Advantages f the Tree-Structured Apprach Handles bth categrical and rdered variables in a simple and natural way. Autmatic stepwise variable selectin and cmpleity reductin. It prvides an estimate f the misclassificatin rate fr a query sample. It is invariant under all mntne transfrmatins f individual rdered variables. Rbust t utliers and misclassified pints in the training set. Easy t interpret. 23
24 Variable Cmbinatins Splits perpendicular t the crdinate aes are inefficient in certain cases. Use linear cmbinatins f variables: Is a j j c? The amunt f cmputatin is increased significantly. Price t pay: mdel cmpleity increases. 2
25 Missing Values Certain variables are missing in sme training samples. Often ccurs in gene-epressin micrarray data. Suppse each variable has 5% chance being missing independently. Then fr a training sample with 5 variables, the prbability f missing sme variables is as high as 92.3%. A query sample t be classified may have missing variables. Find surrgate splits. Suppse the best split fr nde t is s which invlves a questin n X m. Find anther split s n a variable X j, j m, which is mst similar t s in a certain sense. Similarly, the secnd best surrgate split, the third, and s n, can be fund. 25
Pattern Recognition 2014 Support Vector Machines
Pattern Recgnitin 2014 Supprt Vectr Machines Ad Feelders Universiteit Utrecht Ad Feelders ( Universiteit Utrecht ) Pattern Recgnitin 1 / 55 Overview 1 Separable Case 2 Kernel Functins 3 Allwing Errrs (Sft
More informationChapter 3: Cluster Analysis
Chapter 3: Cluster Analysis } 3.1 Basic Cncepts f Clustering 3.1.1 Cluster Analysis 3.1. Clustering Categries } 3. Partitining Methds 3..1 The principle 3.. K-Means Methd 3..3 K-Medids Methd 3..4 CLARA
More informationx 1 Outline IAML: Logistic Regression Decision Boundaries Example Data
Outline IAML: Lgistic Regressin Charles Suttn and Victr Lavrenk Schl f Infrmatics Semester Lgistic functin Lgistic regressin Learning lgistic regressin Optimizatin The pwer f nn-linear basis functins Least-squares
More informationBiplots in Practice MICHAEL GREENACRE. Professor of Statistics at the Pompeu Fabra University. Chapter 13 Offprint
Biplts in Practice MICHAEL GREENACRE Prfessr f Statistics at the Pmpeu Fabra University Chapter 13 Offprint CASE STUDY BIOMEDICINE Cmparing Cancer Types Accrding t Gene Epressin Arrays First published:
More informationLecture 2: Supervised vs. unsupervised learning, bias-variance tradeoff
Lecture 2: Supervised vs. unsupervised learning, bias-variance tradeff Reading: Chapter 2 STATS 202: Data mining and analysis September 27, 2017 1 / 20 Supervised vs. unsupervised learning In unsupervised
More informationPart 3 Introduction to statistical classification techniques
Part 3 Intrductin t statistical classificatin techniques Machine Learning, Part 3, March 07 Fabi Rli Preamble ØIn Part we have seen that if we knw: Psterir prbabilities P(ω i / ) Or the equivalent terms
More informationLHS Mathematics Department Honors Pre-Calculus Final Exam 2002 Answers
LHS Mathematics Department Hnrs Pre-alculus Final Eam nswers Part Shrt Prblems The table at the right gives the ppulatin f Massachusetts ver the past several decades Using an epnential mdel, predict the
More informationLecture 2: Supervised vs. unsupervised learning, bias-variance tradeoff
Lecture 2: Supervised vs. unsupervised learning, bias-variance tradeff Reading: Chapter 2 STATS 202: Data mining and analysis September 27, 2017 1 / 20 Supervised vs. unsupervised learning In unsupervised
More informationCN700 Additive Models and Trees Chapter 9: Hastie et al. (2001)
CN700 Additive Mdels and Trees Chapter 9: Hastie et al. (2001) Madhusudana Shashanka Department f Cgnitive and Neural Systems Bstn University CN700 - Additive Mdels and Trees March 02, 2004 p.1/34 Overview
More informationCOMP 551 Applied Machine Learning Lecture 4: Linear classification
COMP 551 Applied Machine Learning Lecture 4: Linear classificatin Instructr: Jelle Pineau (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/cmp551 Unless therwise nted, all material psted
More informationCOMP 551 Applied Machine Learning Lecture 11: Support Vector Machines
COMP 551 Applied Machine Learning Lecture 11: Supprt Vectr Machines Instructr: (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/cmp551 Unless therwise nted, all material psted fr this curse
More informationNAME: Prof. Ruiz. 1. [5 points] What is the difference between simple random sampling and stratified random sampling?
CS4445 ata Mining and Kwledge iscery in atabases. B Term 2014 Exam 1 Nember 24, 2014 Prf. Carlina Ruiz epartment f Cmputer Science Wrcester Plytechnic Institute NAME: Prf. Ruiz Prblem I: Prblem II: Prblem
More information3.4 Shrinkage Methods Prostate Cancer Data Example (Continued) Ridge Regression
3.3.4 Prstate Cancer Data Example (Cntinued) 3.4 Shrinkage Methds 61 Table 3.3 shws the cefficients frm a number f different selectin and shrinkage methds. They are best-subset selectin using an all-subsets
More informationIAML: Support Vector Machines
1 / 22 IAML: Supprt Vectr Machines Charles Suttn and Victr Lavrenk Schl f Infrmatics Semester 1 2 / 22 Outline Separating hyperplane with maimum margin Nn-separable training data Epanding the input int
More informationk-nearest Neighbor How to choose k Average of k points more reliable when: Large k: noise in attributes +o o noise in class labels
Mtivating Example Memry-Based Learning Instance-Based Learning K-earest eighbr Inductive Assumptin Similar inputs map t similar utputs If nt true => learning is impssible If true => learning reduces t
More informationHypothesis Tests for One Population Mean
Hypthesis Tests fr One Ppulatin Mean Chapter 9 Ala Abdelbaki Objective Objective: T estimate the value f ne ppulatin mean Inferential statistics using statistics in rder t estimate parameters We will be
More informationResampling Methods. Cross-validation, Bootstrapping. Marek Petrik 2/21/2017
Resampling Methds Crss-validatin, Btstrapping Marek Petrik 2/21/2017 Sme f the figures in this presentatin are taken frm An Intrductin t Statistical Learning, with applicatins in R (Springer, 2013) with
More informationSUPPLEMENTARY MATERIAL GaGa: a simple and flexible hierarchical model for microarray data analysis
SUPPLEMENTARY MATERIAL GaGa: a simple and flexible hierarchical mdel fr micrarray data analysis David Rssell Department f Bistatistics M.D. Andersn Cancer Center, Hustn, TX 77030, USA rsselldavid@gmail.cm
More informationCOMP 551 Applied Machine Learning Lecture 5: Generative models for linear classification
COMP 551 Applied Machine Learning Lecture 5: Generative mdels fr linear classificatin Instructr: Herke van Hf (herke.vanhf@mail.mcgill.ca) Slides mstly by: Jelle Pineau Class web page: www.cs.mcgill.ca/~hvanh2/cmp551
More informationChapter 3 Kinematics in Two Dimensions; Vectors
Chapter 3 Kinematics in Tw Dimensins; Vectrs Vectrs and Scalars Additin f Vectrs Graphical Methds (One and Tw- Dimensin) Multiplicatin f a Vectr b a Scalar Subtractin f Vectrs Graphical Methds Adding Vectrs
More informationDifferentiation Applications 1: Related Rates
Differentiatin Applicatins 1: Related Rates 151 Differentiatin Applicatins 1: Related Rates Mdel 1: Sliding Ladder 10 ladder y 10 ladder 10 ladder A 10 ft ladder is leaning against a wall when the bttm
More informationDepartment of Economics, University of California, Davis Ecn 200C Micro Theory Professor Giacomo Bonanno. Insurance Markets
Department f Ecnmics, University f alifrnia, Davis Ecn 200 Micr Thery Prfessr Giacm Bnann Insurance Markets nsider an individual wh has an initial wealth f. ith sme prbability p he faces a lss f x (0
More informationBootstrap Method > # Purpose: understand how bootstrap method works > obs=c(11.96, 5.03, 67.40, 16.07, 31.50, 7.73, 11.10, 22.38) > n=length(obs) >
Btstrap Methd > # Purpse: understand hw btstrap methd wrks > bs=c(11.96, 5.03, 67.40, 16.07, 31.50, 7.73, 11.10, 22.38) > n=length(bs) > mean(bs) [1] 21.64625 > # estimate f lambda > lambda = 1/mean(bs);
More informationT Algorithmic methods for data mining. Slide set 6: dimensionality reduction
T-61.5060 Algrithmic methds fr data mining Slide set 6: dimensinality reductin reading assignment LRU bk: 11.1 11.3 PCA tutrial in mycurses (ptinal) ptinal: An Elementary Prf f a Therem f Jhnsn and Lindenstrauss,
More informationDistributions, spatial statistics and a Bayesian perspective
Distributins, spatial statistics and a Bayesian perspective Dug Nychka Natinal Center fr Atmspheric Research Distributins and densities Cnditinal distributins and Bayes Thm Bivariate nrmal Spatial statistics
More informationResampling Methods. Chapter 5. Chapter 5 1 / 52
Resampling Methds Chapter 5 Chapter 5 1 / 52 1 51 Validatin set apprach 2 52 Crss validatin 3 53 Btstrap Chapter 5 2 / 52 Abut Resampling An imprtant statistical tl Pretending the data as ppulatin and
More informationAP Statistics Notes Unit Two: The Normal Distributions
AP Statistics Ntes Unit Tw: The Nrmal Distributins Syllabus Objectives: 1.5 The student will summarize distributins f data measuring the psitin using quartiles, percentiles, and standardized scres (z-scres).
More informationMATHEMATICS SYLLABUS SECONDARY 5th YEAR
Eurpean Schls Office f the Secretary-General Pedaggical Develpment Unit Ref. : 011-01-D-8-en- Orig. : EN MATHEMATICS SYLLABUS SECONDARY 5th YEAR 6 perid/week curse APPROVED BY THE JOINT TEACHING COMMITTEE
More informationCOMP 551 Applied Machine Learning Lecture 9: Support Vector Machines (cont d)
COMP 551 Applied Machine Learning Lecture 9: Supprt Vectr Machines (cnt d) Instructr: Herke van Hf (herke.vanhf@mail.mcgill.ca) Slides mstly by: Class web page: www.cs.mcgill.ca/~hvanh2/cmp551 Unless therwise
More informationChecking the resolved resonance region in EXFOR database
Checking the reslved resnance regin in EXFOR database Gttfried Bertn Sciété de Calcul Mathématique (SCM) Oscar Cabells OECD/NEA Data Bank JEFF Meetings - Sessin JEFF Experiments Nvember 0-4, 017 Bulgne-Billancurt,
More informationEngineering Decision Methods
GSOE9210 vicj@cse.unsw.edu.au www.cse.unsw.edu.au/~gs9210 Maximin and minimax regret 1 2 Indifference; equal preference 3 Graphing decisin prblems 4 Dminance The Maximin principle Maximin and minimax Regret
More informationSupport-Vector Machines
Supprt-Vectr Machines Intrductin Supprt vectr machine is a linear machine with sme very nice prperties. Haykin chapter 6. See Alpaydin chapter 13 fr similar cntent. Nte: Part f this lecture drew material
More informationThe blessing of dimensionality for kernel methods
fr kernel methds Building classifiers in high dimensinal space Pierre Dupnt Pierre.Dupnt@ucluvain.be Classifiers define decisin surfaces in sme feature space where the data is either initially represented
More informationSPH3U1 Lesson 06 Kinematics
PROJECTILE MOTION LEARNING GOALS Students will: Describe the mtin f an bject thrwn at arbitrary angles thrugh the air. Describe the hrizntal and vertical mtins f a prjectile. Slve prjectile mtin prblems.
More informationMidwest Big Data Summer School: Machine Learning I: Introduction. Kris De Brabanter
Midwest Big Data Summer Schl: Machine Learning I: Intrductin Kris De Brabanter kbrabant@iastate.edu Iwa State University Department f Statistics Department f Cmputer Science June 24, 2016 1/24 Outline
More informationMath 10 - Exam 1 Topics
Math 10 - Exam 1 Tpics Types and Levels f data Categrical, Discrete r Cntinuus Nminal, Ordinal, Interval r Rati Descriptive Statistics Stem and Leaf Graph Dt Plt (Interpret) Gruped Data Relative and Cumulative
More informationWhat is Statistical Learning?
What is Statistical Learning? Sales 5 10 15 20 25 Sales 5 10 15 20 25 Sales 5 10 15 20 25 0 50 100 200 300 TV 0 10 20 30 40 50 Radi 0 20 40 60 80 100 Newspaper Shwn are Sales vs TV, Radi and Newspaper,
More informationChapter 8: The Binomial and Geometric Distributions
Sectin 8.1: The Binmial Distributins Chapter 8: The Binmial and Gemetric Distributins A randm variable X is called a BINOMIAL RANDOM VARIABLE if it meets ALL the fllwing cnditins: 1) 2) 3) 4) The MOST
More informationENSC Discrete Time Systems. Project Outline. Semester
ENSC 49 - iscrete Time Systems Prject Outline Semester 006-1. Objectives The gal f the prject is t design a channel fading simulatr. Upn successful cmpletin f the prject, yu will reinfrce yur understanding
More information1b) =.215 1c).080/.215 =.372
Practice Exam 1 - Answers 1. / \.1/ \.9 (D+) (D-) / \ / \.8 / \.2.15/ \.85 (T+) (T-) (T+) (T-).080.020.135.765 1b).080 +.135 =.215 1c).080/.215 =.372 2. The data shwn in the scatter plt is the distance
More informationPre-Calculus Individual Test 2017 February Regional
The abbreviatin NOTA means Nne f the Abve answers and shuld be chsen if chices A, B, C and D are nt crrect. N calculatr is allwed n this test. Arcfunctins (such as y = Arcsin( ) ) have traditinal restricted
More informationEnhancing Performance of MLP/RBF Neural Classifiers via an Multivariate Data Distribution Scheme
Enhancing Perfrmance f / Neural Classifiers via an Multivariate Data Distributin Scheme Halis Altun, Gökhan Gelen Nigde University, Electrical and Electrnics Engineering Department Nigde, Turkey haltun@nigde.edu.tr
More informationPreparation work for A2 Mathematics [2017]
Preparatin wrk fr A2 Mathematics [2017] The wrk studied in Y12 after the return frm study leave is frm the Cre 3 mdule f the A2 Mathematics curse. This wrk will nly be reviewed during Year 13, it will
More informationAdministrativia. Assignment 1 due thursday 9/23/2004 BEFORE midnight. Midterm exam 10/07/2003 in class. CS 460, Sessions 8-9 1
Administrativia Assignment 1 due thursday 9/23/2004 BEFORE midnight Midterm eam 10/07/2003 in class CS 460, Sessins 8-9 1 Last time: search strategies Uninfrmed: Use nly infrmatin available in the prblem
More informationSupport Vector Machines and Flexible Discriminants
12 Supprt Vectr Machines and Flexible Discriminants This is page 417 Printer: Opaque this 12.1 Intrductin In this chapter we describe generalizatins f linear decisin bundaries fr classificatin. Optimal
More informationChapter Summary. Mathematical Induction Strong Induction Recursive Definitions Structural Induction Recursive Algorithms
Chapter 5 1 Chapter Summary Mathematical Inductin Strng Inductin Recursive Definitins Structural Inductin Recursive Algrithms Sectin 5.1 3 Sectin Summary Mathematical Inductin Examples f Prf by Mathematical
More informationComputational modeling techniques
Cmputatinal mdeling techniques Lecture 4: Mdel checing fr ODE mdels In Petre Department f IT, Åb Aademi http://www.users.ab.fi/ipetre/cmpmd/ Cntent Stichimetric matrix Calculating the mass cnservatin relatins
More informationA New Evaluation Measure. J. Joiner and L. Werner. The problems of evaluation and the needed criteria of evaluation
III-l III. A New Evaluatin Measure J. Jiner and L. Werner Abstract The prblems f evaluatin and the needed criteria f evaluatin measures in the SMART system f infrmatin retrieval are reviewed and discussed.
More informationthe results to larger systems due to prop'erties of the projection algorithm. First, the number of hidden nodes must
M.E. Aggune, M.J. Dambrg, M.A. El-Sharkawi, R.J. Marks II and L.E. Atlas, "Dynamic and static security assessment f pwer systems using artificial neural netwrks", Prceedings f the NSF Wrkshp n Applicatins
More informationElements of Machine Intelligence - I
ECE-175A Elements f Machine Intelligence - I Ken Kreutz-Delgad Nun Vascncels ECE Department, UCSD Winter 2011 The curse The curse will cver basic, but imprtant, aspects f machine learning and pattern recgnitin
More informationSTATS216v Introduction to Statistical Learning Stanford University, Summer Practice Final (Solutions) Duration: 3 hours
STATS216v Intrductin t Statistical Learning Stanfrd University, Summer 2016 Practice Final (Slutins) Duratin: 3 hurs Instructins: (This is a practice final and will nt be graded.) Remember the university
More informationmaking triangle (ie same reference angle) ). This is a standard form that will allow us all to have the X= y=
Intrductin t Vectrs I 21 Intrductin t Vectrs I 22 I. Determine the hrizntal and vertical cmpnents f the resultant vectr by cunting n the grid. X= y= J. Draw a mangle with hrizntal and vertical cmpnents
More informationLead/Lag Compensator Frequency Domain Properties and Design Methods
Lectures 6 and 7 Lead/Lag Cmpensatr Frequency Dmain Prperties and Design Methds Definitin Cnsider the cmpensatr (ie cntrller Fr, it is called a lag cmpensatr s K Fr s, it is called a lead cmpensatr Ntatin
More informationPSU GISPOPSCI June 2011 Ordinary Least Squares & Spatial Linear Regression in GeoDa
There are tw parts t this lab. The first is intended t demnstrate hw t request and interpret the spatial diagnstics f a standard OLS regressin mdel using GeDa. The diagnstics prvide infrmatin abut the
More information, which yields. where z1. and z2
The Gaussian r Nrmal PDF, Page 1 The Gaussian r Nrmal Prbability Density Functin Authr: Jhn M Cimbala, Penn State University Latest revisin: 11 September 13 The Gaussian r Nrmal Prbability Density Functin
More informationSection 5.8 Notes Page Exponential Growth and Decay Models; Newton s Law
Sectin 5.8 Ntes Page 1 5.8 Expnential Grwth and Decay Mdels; Newtn s Law There are many applicatins t expnential functins that we will fcus n in this sectin. First let s lk at the expnential mdel. Expnential
More informationHomology groups of disks with holes
Hmlgy grups f disks with hles THEOREM. Let p 1,, p k } be a sequence f distinct pints in the interir unit disk D n where n 2, and suppse that fr all j the sets E j Int D n are clsed, pairwise disjint subdisks.
More informationFlipping Physics Lecture Notes: Simple Harmonic Motion Introduction via a Horizontal Mass-Spring System
Flipping Physics Lecture Ntes: Simple Harmnic Mtin Intrductin via a Hrizntal Mass-Spring System A Hrizntal Mass-Spring System is where a mass is attached t a spring, riented hrizntally, and then placed
More informationFlipping Physics Lecture Notes: Simple Harmonic Motion Introduction via a Horizontal Mass-Spring System
Flipping Physics Lecture Ntes: Simple Harmnic Mtin Intrductin via a Hrizntal Mass-Spring System A Hrizntal Mass-Spring System is where a mass is attached t a spring, riented hrizntally, and then placed
More informationWYSE Academic Challenge Regional Mathematics 2007 Solution Set
WYSE Academic Challenge Reginal Mathematics 007 Slutin Set 1. Crrect answer: C. ( ) ( ) 1 + y y = ( + ) + ( y y + 1 ) = + 1 1 ( ) ( 1 + y ) = s *1/ = 1. Crrect answer: A. The determinant is ( 1 ( 1) )
More informationIn SMV I. IAML: Support Vector Machines II. This Time. The SVM optimization problem. We saw:
In SMV I IAML: Supprt Vectr Machines II Nigel Gddard Schl f Infrmatics Semester 1 We sa: Ma margin trick Gemetry f the margin and h t cmpute it Finding the ma margin hyperplane using a cnstrained ptimizatin
More informationSURVIVAL ANALYSIS WITH SUPPORT VECTOR MACHINES
1 SURVIVAL ANALYSIS WITH SUPPORT VECTOR MACHINES Wlfgang HÄRDLE Ruslan MORO Center fr Applied Statistics and Ecnmics (CASE), Humbldt-Universität zu Berlin Mtivatin 2 Applicatins in Medicine estimatin f
More informationProbability, Random Variables, and Processes. Probability
Prbability, Randm Variables, and Prcesses Prbability Prbability Prbability thery: branch f mathematics fr descriptin and mdelling f randm events Mdern prbability thery - the aximatic definitin f prbability
More informationSection 6-2: Simplex Method: Maximization with Problem Constraints of the Form ~
Sectin 6-2: Simplex Methd: Maximizatin with Prblem Cnstraints f the Frm ~ Nte: This methd was develped by Gerge B. Dantzig in 1947 while n assignment t the U.S. Department f the Air Frce. Definitin: Standard
More informationSource Coding and Compression
Surce Cding and Cmpressin Heik Schwarz Cntact: Dr.-Ing. Heik Schwarz heik.schwarz@hhi.fraunhfer.de Heik Schwarz Surce Cding and Cmpressin September 22, 2013 1 / 60 PartI: Surce Cding Fundamentals Heik
More informationCOMP9444 Neural Networks and Deep Learning 3. Backpropagation
COMP9444 Neural Netwrks and Deep Learning 3. Backprpagatin Tetbk, Sectins 4.3, 5.2, 6.5.2 COMP9444 17s2 Backprpagatin 1 Outline Supervised Learning Ockham s Razr (5.2) Multi-Layer Netwrks Gradient Descent
More informationBuilding to Transformations on Coordinate Axis Grade 5: Geometry Graph points on the coordinate plane to solve real-world and mathematical problems.
Building t Transfrmatins n Crdinate Axis Grade 5: Gemetry Graph pints n the crdinate plane t slve real-wrld and mathematical prblems. 5.G.1. Use a pair f perpendicular number lines, called axes, t define
More informationOF SIMPLY SUPPORTED PLYWOOD PLATES UNDER COMBINED EDGEWISE BENDING AND COMPRESSION
U. S. FOREST SERVICE RESEARCH PAPER FPL 50 DECEMBER U. S. DEPARTMENT OF AGRICULTURE FOREST SERVICE FOREST PRODUCTS LABORATORY OF SIMPLY SUPPORTED PLYWOOD PLATES UNDER COMBINED EDGEWISE BENDING AND COMPRESSION
More informationSticiGui Chapter 4: Measures of Location and Spread Philip Stark (2013)
SticiGui Chapter 4: Measures f Lcatin and Spread Philip Stark (2013) Summarizing data can help us understand them, especially when the number f data is large. This chapter presents several ways t summarize
More informationChapter 15 & 16: Random Forests & Ensemble Learning
Chapter 15 & 16: Randm Frests & Ensemble Learning DD3364 Nvember 27, 2012 Ty Prblem fr Bsted Tree Bsted Tree Example Estimate this functin with a sum f trees with 9-terminal ndes by minimizing the sum
More information3. Classify the following Numbers (Counting (natural), Whole, Integers, Rational, Irrational)
After yu cmplete each cncept give yurself a rating 1. 15 5 2 (5 3) 2. 2 4-8 (2 5) 3. Classify the fllwing Numbers (Cunting (natural), Whle, Integers, Ratinal, Irratinal) a. 7 b. 2 3 c. 2 4. Are negative
More informationThermodynamics and Equilibrium
Thermdynamics and Equilibrium Thermdynamics Thermdynamics is the study f the relatinship between heat and ther frms f energy in a chemical r physical prcess. We intrduced the thermdynamic prperty f enthalpy,
More information4th Indian Institute of Astrophysics - PennState Astrostatistics School July, 2013 Vainu Bappu Observatory, Kavalur. Correlation and Regression
4th Indian Institute f Astrphysics - PennState Astrstatistics Schl July, 2013 Vainu Bappu Observatry, Kavalur Crrelatin and Regressin Rahul Ry Indian Statistical Institute, Delhi. Crrelatin Cnsider a tw
More informationSequential Allocation with Minimal Switching
In Cmputing Science and Statistics 28 (1996), pp. 567 572 Sequential Allcatin with Minimal Switching Quentin F. Stut 1 Janis Hardwick 1 EECS Dept., University f Michigan Statistics Dept., Purdue University
More informationB. Definition of an exponential
Expnents and Lgarithms Chapter IV - Expnents and Lgarithms A. Intrductin Starting with additin and defining the ntatins fr subtractin, multiplicatin and divisin, we discvered negative numbers and fractins.
More informationHiding in plain sight
Hiding in plain sight Principles f stegangraphy CS349 Cryptgraphy Department f Cmputer Science Wellesley Cllege The prisners prblem Stegangraphy 1-2 1 Secret writing Lemn juice is very nearly clear s it
More informationData Mining: Concepts and Techniques. Classification and Prediction. Chapter February 8, 2007 CSE-4412: Data Mining 1
Data Mining: Cncepts and Techniques Classificatin and Predictin Chapter 6.4-6 February 8, 2007 CSE-4412: Data Mining 1 Chapter 6 Classificatin and Predictin 1. What is classificatin? What is predictin?
More informationModule 3: Gaussian Process Parameter Estimation, Prediction Uncertainty, and Diagnostics
Mdule 3: Gaussian Prcess Parameter Estimatin, Predictin Uncertainty, and Diagnstics Jerme Sacks and William J Welch Natinal Institute f Statistical Sciences and University f British Clumbia Adapted frm
More informationTuring Machines. Human-aware Robotics. 2017/10/17 & 19 Chapter 3.2 & 3.3 in Sipser Ø Announcement:
Turing Machines Human-aware Rbtics 2017/10/17 & 19 Chapter 3.2 & 3.3 in Sipser Ø Annuncement: q q q q Slides fr this lecture are here: http://www.public.asu.edu/~yzhan442/teaching/cse355/lectures/tm-ii.pdf
More information15-381/781 Bayesian Nets & Probabilistic Inference
15-381/781 Bayesian Nets & Prbabilistic Inference Emma Brunskill (this time) Ariel Prcaccia With thanks t Dan Klein (Berkeley), Percy Liang (Stanfrd) and Past 15-381 Instructrs fr sme slide cntent, and
More informationNUMBERS, MATHEMATICS AND EQUATIONS
AUSTRALIAN CURRICULUM PHYSICS GETTING STARTED WITH PHYSICS NUMBERS, MATHEMATICS AND EQUATIONS An integral part t the understanding f ur physical wrld is the use f mathematical mdels which can be used t
More informationLocalized Model Selection for Regression
Lcalized Mdel Selectin fr Regressin Yuhng Yang Schl f Statistics University f Minnesta Church Street S.E. Minneaplis, MN 5555 May 7, 007 Abstract Research n mdel/prcedure selectin has fcused n selecting
More informationA proposition is a statement that can be either true (T) or false (F), (but not both).
400 lecture nte #1 [Ch 2, 3] Lgic and Prfs 1.1 Prpsitins (Prpsitinal Lgic) A prpsitin is a statement that can be either true (T) r false (F), (but nt bth). "The earth is flat." -- F "March has 31 days."
More informationAgenda. What is Machine Learning? Learning Type of Learning: Supervised, Unsupervised and semi supervised Classification
Agenda Artificial Intelligence and its applicatins Lecture 6 Supervised Learning Prfessr Daniel Yeung danyeung@ieee.rg Dr. Patrick Chan patrickchan@ieee.rg Suth China University f Technlgy, China Learning
More informationELT COMMUNICATION THEORY
ELT 41307 COMMUNICATION THEORY Matlab Exercise #2 Randm variables and randm prcesses 1 RANDOM VARIABLES 1.1 ROLLING A FAIR 6 FACED DICE (DISCRETE VALIABLE) Generate randm samples fr rlling a fair 6 faced
More informationChapter 2 GAUSS LAW Recommended Problems:
Chapter GAUSS LAW Recmmended Prblems: 1,4,5,6,7,9,11,13,15,18,19,1,7,9,31,35,37,39,41,43,45,47,49,51,55,57,61,6,69. LCTRIC FLUX lectric flux is a measure f the number f electric filed lines penetrating
More informationWork, Energy, and Power
rk, Energy, and Pwer Physics 1 There are many different TYPES f Energy. Energy is expressed in JOULES (J 419J 4.19 1 calrie Energy can be expressed mre specifically by using the term ORK( rk The Scalar
More informationInternal vs. external validity. External validity. This section is based on Stock and Watson s Chapter 9.
Sectin 7 Mdel Assessment This sectin is based n Stck and Watsn s Chapter 9. Internal vs. external validity Internal validity refers t whether the analysis is valid fr the ppulatin and sample being studied.
More informationCompressibility Effects
Definitin f Cmpressibility All real substances are cmpressible t sme greater r lesser extent; that is, when yu squeeze r press n them, their density will change The amunt by which a substance can be cmpressed
More informationM thematics. National 5 Practice Paper D. Paper 1. Duration 1 hour. Total marks 40
N5 M thematics Natinal 5 Practice Paper D Paper 1 Duratin 1 hur Ttal marks 40 Yu may NOT use a calculatr Attempt all the questins. Use blue r black ink. Full credit will nly be given t slutins which cntain
More informationANSWER KEY FOR MATH 10 SAMPLE EXAMINATION. Instructions: If asked to label the axes please use real world (contextual) labels
ANSWER KEY FOR MATH 10 SAMPLE EXAMINATION Instructins: If asked t label the axes please use real wrld (cntextual) labels Multiple Chice Answers: 0 questins x 1.5 = 30 Pints ttal Questin Answer Number 1
More informationLeast Squares Optimal Filtering with Multirate Observations
Prc. 36th Asilmar Cnf. n Signals, Systems, and Cmputers, Pacific Grve, CA, Nvember 2002 Least Squares Optimal Filtering with Multirate Observatins Charles W. herrien and Anthny H. Hawes Department f Electrical
More informationEquilibrium of Stress
Equilibrium f Stress Cnsider tw perpendicular planes passing thrugh a pint p. The stress cmpnents acting n these planes are as shwn in ig. 3.4.1a. These stresses are usuall shwn tgether acting n a small
More informationPart One: Heat Changes and Thermochemistry. This aspect of Thermodynamics was dealt with in Chapter 6. (Review)
CHAPTER 18: THERMODYNAMICS AND EQUILIBRIUM Part One: Heat Changes and Thermchemistry This aspect f Thermdynamics was dealt with in Chapter 6. (Review) A. Statement f First Law. (Sectin 18.1) 1. U ttal
More informationLecture 24: Flory-Huggins Theory
Lecture 24: 12.07.05 Flry-Huggins Thery Tday: LAST TIME...2 Lattice Mdels f Slutins...2 ENTROPY OF MIXING IN THE FLORY-HUGGINS MODEL...3 CONFIGURATIONS OF A SINGLE CHAIN...3 COUNTING CONFIGURATIONS FOR
More informationFloating Point Method for Solving Transportation. Problems with Additional Constraints
Internatinal Mathematical Frum, Vl. 6, 20, n. 40, 983-992 Flating Pint Methd fr Slving Transprtatin Prblems with Additinal Cnstraints P. Pandian and D. Anuradha Department f Mathematics, Schl f Advanced
More informationStats Classification Ji Zhu, Michigan Statistics 1. Classification. Ji Zhu 445C West Hall
Stats 415 - Classificatin Ji Zhu, Michigan Statistics 1 Classificatin Ji Zhu 445C West Hall 734-936-2577 jizhu@umich.edu Stats 415 - Classificatin Ji Zhu, Michigan Statistics 2 Examples f Classificatin
More informationExam #1. A. Answer any 1 of the following 2 questions. CEE 371 October 8, Please grade the following questions: 1 or 2
CEE 371 Octber 8, 2009 Exam #1 Clsed Bk, ne sheet f ntes allwed Please answer ne questin frm the first tw, ne frm the secnd tw and ne frm the last three. The ttal ptential number f pints is 100. Shw all
More informationThe Kullback-Leibler Kernel as a Framework for Discriminant and Localized Representations for Visual Recognition
The Kullback-Leibler Kernel as a Framewrk fr Discriminant and Lcalized Representatins fr Visual Recgnitin Nun Vascncels Purdy H Pedr Mren ECE Department University f Califrnia, San Dieg HP Labs Cambridge
More informationM thematics. National 5 Practice Paper E. Paper 1. Duration 1 hour. Total marks 40
N5 M thematics Natinal 5 Practice Paper E Paper 1 Duratin 1 hur Ttal marks 40 Yu may NOT use a calculatr Attempt all the questins. Use blue r black ink. Full credit will nly be given t slutins which cntain
More information