An Improved Support Vector Machine Using Class-Median Vectors *
|
|
- Kristian Kelly
- 5 years ago
- Views:
Transcription
1 A Improved Support Vector Mache Usg Class-Meda Vectors Zhezhe Kou, Jahua Xu, Xuegog Zhag ad Lag J State Ke Laborator of Itellget Techolog ad Sstems Departmet of Automato, Tsghua Uverst, Bejg , P.R.C. zhezhekou00@mals.tsghua.edu.c Abstract Support vector mache bulds the fal dscrmat fucto o ol a small part of the trag samples, hch ma make the decso rule too sestve to ose ad outlers. Ispred b the dea of cetral support vector mache or C, e preset a mproved method based o the class-meda, called Meda Support Vector Mache or M ths paper. The epermet results sho M s a promsg ad robust algorthm, especall he outlers are far from the class-ceter. Keords Patter Classfcato, Support Vector Mache, Meda Support Vector Mache 1 Itroducto Support vector mache or s a e patter recogto techque developed b Dr. Vapk ad hs co-researchers [1-3]. The basc dea of s to desg a lear classfer th mamal classfcato marg, hle mmzg the trag error. Mamzg the marg plas the role of capact cotrol so that the learg mache ll ot ol have small emprcal rsk but also hold good geeralzato ablt [1-3]. Usuall the fal dscrmat fucto of ol depeds o part of the trag samples, hch are called support vectors. Ths propert makes the fal decso fucto sestve to some certa specfc samples the set. Thus the decso fucto obtaed b ma be badl cotorted f the samples are polluted b ose ad outlers ad some of these specfc samples are ufortuatel cosdered as support vectors, hch s ofte true the practcal applcatos. Ths problem has bee attractg more ad more atteto [4-6]. I [5] Zhag proposed a Ths ork s supported b Natoal Natural Scece Foudato of Cha(Project umber ). modfed verso amed cetral support vector mache or C, hch ot ol takes the advatage of capact cotrol but also troduces the class ceter to overcome the eakess of. It as proved effectve ad promsg some practcal cases [5-6]. Hoever, C ma ot alas ork. I some cases, especall he the outlers stad far from the class ceter, the mea vector ll be pulled aa from the ceter b these outlers. But the class meda s a more robust represetato of the class tha the class ceter. I ths paper e preset a method, hch follos the mportat dea of ad C but tres to prevet the classfer from becomg too sestve to outlers b buldg the classfer o both the class-meda vectors ad the support vectors. The method s referred to as Meda Support Vector Mache or M. Epermets o to data sho ts usefuless ad advatages over ad also over C for certa cases. The remag part of the paper s arraged the follog a. The dea ad algorthm of M s preseted secto 2 ad secto 3. Secto 4 dscusses the M o-separable cases ad secto 5 apples the dea of kerel to make M olear. A smplfed mplemetato of M s gve secto 6 to utlze prevous algorthms for. Secto 7 llustrates some epermet results to evaluate M. More dscussos ad coclusos are gve secto 8. 2 Meda Support Vector Mache I ths paper the class meda s defed as a vector hose each compoet s the meda of correspodg compoets of all samples. I practce f the umber of the samples s odd, compoets of the class meda are the meda values, otherse ts compoets are the average of the to mddle values. Whe outlers occur the sample set, the could chage the meda lttle, but affect the class-ceter severel. As sho - 1 -
2 Fg.1, f there ests o outler, the cetral vector s close to meda vector. Hoever he a outler occurs the cetral vector s pulled far from the true cetral vector but the meda vector ol chages lttle. So t s cocluded that class-meda ll be more robust tha class-ceter, especall he the outlers are located far from class-ceter. class-medas to the classfcato boudar are: b ( b) d (5a) b ( b) d (5b) here 1, 1 are the class labels of ad. The e ca formulate our verso of marg as: ( ) d d d (6) Fg.1: Dagram of ceter ad meda Lke ad C, M s to fd a optmal separatg hperplae ( ) b 0 (1) accordg to the sample set (, ), 1,...,, R d, { 1, 1} (2) The decso fucto s the f ( ) sg{( ) b} (3) For the sake of smplct, frst e cosder learl separable cases. Accordg to [1-3], for the lear classfer to be optmal, all samples (2) should be correctl classfed b (3) ad the separato marg should be mamzed. The former requremet ca be rtte as: b,,, [( ) ] ξ > 0 1 L (4) here ξ 0 s a slack varable that cotrols the trag errors. Ths requremet guaratees the emprcal rsk ca be mmzed. The latter requremet performs the role of capact cotrol, hch makes the learg mache geeralze ell [1-3]. I, the separato marg s defed as the dstaces from the earest samples to the classfcato boudar. Actuall t s ths defto of separato marg that decdes the famous propert that the traed classfer depeds o ol a small part of the trag samples. To avod ths, e defe aother kd of marg, that s, the dstaces from the to class medas to the separato boudar. Fgure 2 llustrates the basc dea of M. Deote the meda of class 1 as, ad the meda of class -1 as. The dstaces from the Fgure 2. Meda separato marg We call the marg defed b (6) the meda separato marg ad call the classfer to mamze ths marg as Meda Support Vector Mache or M. It s ot eas to mamze (6) drectl, so follog the scheme used stadard, e ormalze the umerator of meda separato marg d to 1,.e. ( ) 1 (7) The reaso h e ca do ths s that the magtude of ca be scaled arbtrarl thout affectg the classfcato result. Thus e costruct the optmzato problem of our M method as, m ψ ( ) ( ) (8) 2 2 subject to costrats (4) ad (7). Ths form s smlar to the prme problem of th oe more costrat. The optmzato goal of (8) ca be eplaed as mamzg the meda marg hle keepg all the trag data ot ol correctl classfed but also be aa from the separato hperplae. We ca ame the to hperplaes ( ) b ±ξ as the separato boudares ad call the rego betee them boudar zoe
3 3 The Dual Form of M Follog the smlar scheme th, the Lagrage fucto of the prme problem ca be rtte as: L p 1 2 ( ) α 1 { [( ) b] ξ} { ( ) 1} (9) β Ad the dual form of the above problem s: ma L D εα β 1 1 α 2, j 1 j ( β ) (10) ( α j β ), j 1 subject to: α 0 (11a) 1 α 0, 1,... (11b) hch s a quadratc programmg problem of α, β. The soluto eght vector s the lear combato of the trag samples α β ( ) 1 (12) Ad the decso fucto s f ( ) sg{( ) b } (13) hch the threshold b ca be decded from a sample for hch the equalt (4) holds true or be calculated b the to class medas. Smlarl as the case, from the Küh-Tucker codto e ca also fd that ol the α correspodg to those samples, hch make the equalt hold (4), s o-zero. These samples are also the oes that are earest to the separato boudar, correspodg to the support vectors. From (12) e ca see that the fal classfer of M s decded b both the support vectors ad the to class-meda vectors. Ths ca lead to a smplfed mplemetato of M dscussed secto 6. 4 Cosderatos for Noseparable Cases Whe cosderg the oseparable case, aga follog the deas, e troduce slack varables to codto (4) to clude the samples that volate ths codto: [ ( ) b] ξ ξ> 0, ξ 0, 1, L, (14) ad defe the objectve fucto as: mmze 1 ψ (, ξ ) ( ) C ξ (15) 2 1 Parameter C cotrols pealt o the errors ad also cotrols the trade-off betee the classfcato formato cotaed the medas of the samples ad the dvdual samples ear the boudar. For ths e problem, the dual problem turs to be same as defed b (10) ad (11) epect that codto (11b) s modfed to: 0 α < C,,... (16) 5 Kerel Verso 1 Lke e ca also appl the dea of kerel to, make M olear, b substtutg ( ) th, some kerel K( ) to realze the mappg from attrbute space to the hgh-dmeso feature space. I ths case, the fal decso fucto ll be f ( ) sg{ K( ) b } sg{ K (, ) K ( α β, 1 β K (, ) b } ) (17) Here e should pa especall atteto to ad. Ca the meda vectors attrbute space be used drectl here? From secto 2 t ca be see that the basc dea of usg class medas to buld support vector mache s that the class meda s less sestve to outlers ad thus ca be a more robust represetato of the class. Sce ths s true the attrbute space, e beleve that the class-meda attrbute space ca be a good represetatve of the sample. At the same tme, the medas the feature space are usuall uko because the feature space ma ot be or eed ot to be ko. Therefore t ould be reasoable ad feasble to drectl use the meda vectors attrbute space here. That s, use the mage of the meda vectors attrbute space as the medas feature space. Thus the M ca be mplemeted olear case a smplfed a. 6 A smplfed mplemetato of M From the epresso of the soluto of (12), e ca fd out that the fal decso fucto ca be cosdered as the combato of to parts, the support - 3 -
4 vectors ad the class-meda vectors. Ths fact leads to a smplfed mplemetato of M, hch ca be obtaed through the modfcato of the stadard result. That s, after gettg the eght of a stadard classfer, e ca adjust the hperplae b addg the dfferece vector betee the to class-medas th certa factor, e.g., e ( 1 λ ) λ( ) (18) here s the eght obtaed b stadard ad 0 λ 1 s a costat cotrollg the balace betee the classfcato formato cotaed the SVs ad the class medas. B takg ths short cut, prevous algorthms for ca be utlzed. I practce, λ ca be selected accordg the a pror estmato about our belefs the specfc samples ad the class medas. For eample, f there s severe ose sample set, a larger ca be chose to make the class medas pla a more mportat role. If λ 0, the stadard ca be obtaed. I practce, sce s scaled stadard, ma eed scalg to be comparable th. Whe kerel s adopted for the olear trasform, e ca use the mage of the meda vectors attrbute space as the meda feature space such that f ( ) sg{(1 λ) α K (, ) (19) 1 svm λ( K ( ) K ( )) b } Ths s our smplfed mplemetato of kerel verso of M. same as the oe obtaed he the trag set s clea. Note that here e dd ot attempt to detf ad remove the outler, but smpl made the result less sestve to outlers b takg the class-meda formato to accout. It ca also be see that he the outler s ot far from class-ceters, M ad C acheve smlar performace. (a) outler (b) (c) (d) Fg. 3: A comparso epermet of, C ad M. Pluses ad crcles deote the trag samples of to classes. (a) the stadard result at the estece of outlers; (b) the stadard result ecludg the outlers; (c) ad (d) the hperplaes of C ad M he the outlers est, respectvel. 7 Epermet Results ad Aalss I order to evaluate the performace of M ad compare M th ad C, e desg to to data sets. Epermet results ad aalss are preseted as follos. As sho Fg. 3, the frst eample s to compare the performace of MSV th that of C ad uder the estece of outlers ot far from the class ceter. I Fg.3 (a), there are to outlers the trag set ad the separato hperplae s obvous ot the optmal oe. Fg.3 (b) shos the optmal result he the trag set s oseless ad the separato hperplae chages a lot from the oe (a). Fg.3 (c) s the result obtaed b C, usg ts smplfed mplemetato[5]. Fg.3 (d) llustrates the result obtaed b M, usg ts smplfed mplemetato, th the same λ as (c). We ca fd that although the outlers est the trag set, the optmal result of M s almost the (a) (b) Fg. 4: A comparso epermet betee C ad M. Pluses ad crcles deote the trag samples of to classes. (a) ad (b) are the results of C ad M he the outlers are far from class-ceters, respectvel. Fgure 4 shos the epermet results of the comparso betee C ad M uder the estece of outlers far from the class-ceter. Fg.4 (a) shos the result obtaed b C ad Fg.4 (b) - 4 -
5 the result b M. It ca be see that he the outlers are far from class ceter, the chage the class-ceter greatl so that the results obtaed b C s far from the reall optmal oe. But the class meda s less sestve to such outlers ad therefore the result s more relable. So t ca be cocluded that M s a more robust algorthm. socet orkshop:, Ne York: The Isttute of Electrcal ad Electrocs Egeergs, Ic. pp [6] X. Zhag, H. Ke, Cetral support vector maches ad ts applcato to cacer classfcato. Submtted to IEEE Tras. o Neural Netorks. [7] Steve Gu, Support Vector Maches for Classfcato ad Regresso, ISIS Techcal Report, Uverst of Southampto, Dscusso ad Cocluso Support vector mache or, as a e techque for patter recogto, possesses good geeralzato ad thus has bee recevg more ad more atteto. Hoever, some authors observed that some cases here there est severe ose ad outlers the trag data, makes ts result too sestve to fe specfc samples ad thus less relable. As a step toard a more robust ad more practcall applcable modfcato of, e proposed meda support vector mache or M. We edeavor to combe the class-meda formato th the stadard to make the algorthm more robust. Further, accordg to the form of the M s soluto, a smpler mplemetato s troduced, hch s smpl the combato of the stadard SVs ad the class medas b a factor. Epermets llustrate the advatage of M over stadard ad also over C for certa cases. Hoever, further aalss of the method s deserved, ad e are also trg to appl t for some practcal problems. 9 Ackoledgemet The authors ould lke to thak Dr. JaHua Xu ad Ha Ke for ther helpful dscusso. Refereces [1] Vapk V N. Statstcal Learg Theor, Ne-York, Joh Wle & Sos, [2] Vapk V N. The Nature of Statstcal Learg Theor, NY: Sprger-Verlag, 1995 [3] X. Zhag, Itroducto to statstcal learg theor ad support vector maches, Acta Automatca Sca, 2000, 26(1):32-42( Chese). [4] B. Boser, I. Guo, V. Vapk. A trag algorthm for optmal marg classfers, preseted at the 5th Aual Workshop o Computatoal Learg Theor, Pttsburgh: ACM Press, [5] X. Zhag, Usg class-ceter vectors to buld supportvector maches, Yu-He Hu, Ja Larse, etc. Neural Netorks for Sgal Processg IX, Proceedgs of the 1999 IEEE sgal processg - 5 -
Support vector machines II
CS 75 Mache Learg Lecture Support vector maches II Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Learl separable classes Learl separable classes: here s a hperplae that separates trag staces th o error
More informationCS 1675 Introduction to Machine Learning Lecture 12 Support vector machines
CS 675 Itroducto to Mache Learg Lecture Support vector maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Mdterm eam October 9, 7 I-class eam Closed book Stud materal: Lecture otes Correspodg chapters
More informationBinary classification: Support Vector Machines
CS 57 Itroducto to AI Lecture 6 Bar classfcato: Support Vector Maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 57 Itro to AI Supervsed learg Data: D { D, D,.., D} a set of eamples D, (,,,,,
More informationKernel-based Methods and Support Vector Machines
Kerel-based Methods ad Support Vector Maches Larr Holder CptS 570 Mache Learg School of Electrcal Egeerg ad Computer Scece Washgto State Uverst Refereces Muller et al. A Itroducto to Kerel-Based Learg
More informationSupervised learning: Linear regression Logistic regression
CS 57 Itroducto to AI Lecture 4 Supervsed learg: Lear regresso Logstc regresso Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 57 Itro to AI Data: D { D D.. D D Supervsed learg d a set of eamples s
More informationAn Introduction to. Support Vector Machine
A Itroducto to Support Vector Mache Support Vector Mache (SVM) A classfer derved from statstcal learg theory by Vapk, et al. 99 SVM became famous whe, usg mages as put, t gave accuracy comparable to eural-etwork
More informationSupport vector machines
CS 75 Mache Learg Lecture Support vector maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Learg Outle Outle: Algorthms for lear decso boudary Support vector maches Mamum marg hyperplae.
More informationCS 2750 Machine Learning. Lecture 8. Linear regression. CS 2750 Machine Learning. Linear regression. is a linear combination of input components x
CS 75 Mache Learg Lecture 8 Lear regresso Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Learg Lear regresso Fucto f : X Y s a lear combato of put compoets f + + + K d d K k - parameters
More informationIntroduction to local (nonparametric) density estimation. methods
Itroducto to local (oparametrc) desty estmato methods A slecture by Yu Lu for ECE 66 Sprg 014 1. Itroducto Ths slecture troduces two local desty estmato methods whch are Parze desty estmato ad k-earest
More informationA CLASSIFICATION OF REMOTE SENSING IMAGE BASED ON IMPROVED COMPOUND KERNELS OF SVM
A CLASSIFICAION OF REMOE SENSING IMAGE BASED ON IMPROVED COMPOUND KERNELS OF SVM Jag Zhao Wal Gao * Zl Lu Gufe Mou L Lu 3 La Yu College of Iformato ad Electrcal Egeerg Cha Agrcultural Uverst Beg P. R.
More informationLinear Regression Linear Regression with Shrinkage. Some slides are due to Tommi Jaakkola, MIT AI Lab
Lear Regresso Lear Regresso th Shrkage Some sldes are due to Tomm Jaakkola, MIT AI Lab Itroducto The goal of regresso s to make quattatve real valued predctos o the bass of a vector of features or attrbutes.
More informationA handwritten signature recognition system based on LSVM. Chen jie ping
Iteratoal Coferece o Computatoal Scece ad Egeerg (ICCSE 05) A hadrtte sgature recogto sstem based o LSVM Che je pg Guagx Vocatoal ad echcal College, departmet of computer ad electroc formato egeerg, ag,
More informationSolving Constrained Flow-Shop Scheduling. Problems with Three Machines
It J Cotemp Math Sceces, Vol 5, 2010, o 19, 921-929 Solvg Costraed Flow-Shop Schedulg Problems wth Three Maches P Pada ad P Rajedra Departmet of Mathematcs, School of Advaced Sceces, VIT Uversty, Vellore-632
More informationStudy on a Fire Detection System Based on Support Vector Machine
Sesors & Trasducers, Vol. 8, Issue, November 04, pp. 57-6 Sesors & Trasducers 04 by IFSA Publshg, S. L. http://www.sesorsportal.com Study o a Fre Detecto System Based o Support Vector Mache Ye Xaotg, Wu
More informationRegression and the LMS Algorithm
CSE 556: Itroducto to Neural Netorks Regresso ad the LMS Algorthm CSE 556: Regresso 1 Problem statemet CSE 556: Regresso Lear regresso th oe varable Gve a set of N pars of data {, d }, appromate d b a
More informationAnalysis of Lagrange Interpolation Formula
P IJISET - Iteratoal Joural of Iovatve Scece, Egeerg & Techology, Vol. Issue, December 4. www.jset.com ISS 348 7968 Aalyss of Lagrage Iterpolato Formula Vjay Dahya PDepartmet of MathematcsMaharaja Surajmal
More informationFunctions of Random Variables
Fuctos of Radom Varables Chapter Fve Fuctos of Radom Varables 5. Itroducto A geeral egeerg aalyss model s show Fg. 5.. The model output (respose) cotas the performaces of a system or product, such as weght,
More information0/1 INTEGER PROGRAMMING AND SEMIDEFINTE PROGRAMMING
CONVEX OPIMIZAION AND INERIOR POIN MEHODS FINAL PROJEC / INEGER PROGRAMMING AND SEMIDEFINE PROGRAMMING b Luca Buch ad Natala Vktorova CONENS:.Itroducto.Formulato.Applcato to Kapsack Problem 4.Cuttg Plaes
More informationCSE 5526: Introduction to Neural Networks Linear Regression
CSE 556: Itroducto to Neural Netorks Lear Regresso Part II 1 Problem statemet Part II Problem statemet Part II 3 Lear regresso th oe varable Gve a set of N pars of data , appromate d by a lear fucto
More information15-381: Artificial Intelligence. Regression and neural networks (NN)
5-38: Artfcal Itellece Reresso ad eural etorks NN) Mmck the bra I the earl das of AI there as a lot of terest develop models that ca mmc huma thk. Whle o oe ke eactl ho the bra orks ad, eve thouh there
More informationGenerative classification models
CS 75 Mache Learg Lecture Geeratve classfcato models Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Data: D { d, d,.., d} d, Classfcato represets a dscrete class value Goal: lear f : X Y Bar classfcato
More informationComparison of SVMs in Number Plate Recognition
Comparso of SVMs Number Plate Recogto Lhog Zheg, Xaga He ad om Htz Uversty of echology, Sydey, Departmet of Computer Systems, {lzheg, sea, htz}@t.uts.edu.au Abstract. Hgh accuracy ad hgh speed are two
More informationRadial Basis Function Networks
Radal Bass Fucto Netorks Radal Bass Fucto Netorks A specal types of ANN that have three layers Iput layer Hdde layer Output layer Mappg from put to hdde layer s olear Mappg from hdde to output layer s
More informationPrincipal Components. Analysis. Basic Intuition. A Method of Self Organized Learning
Prcpal Compoets Aalss A Method of Self Orgazed Learg Prcpal Compoets Aalss Stadard techque for data reducto statstcal patter matchg ad sgal processg Usupervsed learg: lear from examples wthout a teacher
More informationABOUT ONE APPROACH TO APPROXIMATION OF CONTINUOUS FUNCTION BY THREE-LAYERED NEURAL NETWORK
ABOUT ONE APPROACH TO APPROXIMATION OF CONTINUOUS FUNCTION BY THREE-LAYERED NEURAL NETWORK Ram Rzayev Cyberetc Isttute of the Natoal Scece Academy of Azerbaa Republc ramrza@yahoo.com Aygu Alasgarova Khazar
More informationFeature Selection: Part 2. 1 Greedy Algorithms (continued from the last lecture)
CSE 546: Mache Learg Lecture 6 Feature Selecto: Part 2 Istructor: Sham Kakade Greedy Algorthms (cotued from the last lecture) There are varety of greedy algorthms ad umerous amg covetos for these algorthms.
More informationUnsupervised Learning and Other Neural Networks
CSE 53 Soft Computg NOT PART OF THE FINAL Usupervsed Learg ad Other Neural Networs Itroducto Mture Destes ad Idetfablty ML Estmates Applcato to Normal Mtures Other Neural Networs Itroducto Prevously, all
More informationResearch on SVM Prediction Model Based on Chaos Theory
Advaced Scece ad Techology Letters Vol.3 (SoftTech 06, pp.59-63 http://dx.do.org/0.457/astl.06.3.3 Research o SVM Predcto Model Based o Chaos Theory Sog Lagog, Wu Hux, Zhag Zezhog 3, College of Iformato
More informationInvestigating Cellular Automata
Researcher: Taylor Dupuy Advsor: Aaro Wootto Semester: Fall 4 Ivestgatg Cellular Automata A Overvew of Cellular Automata: Cellular Automata are smple computer programs that geerate rows of black ad whte
More informationPart 4b Asymptotic Results for MRR2 using PRESS. Recall that the PRESS statistic is a special type of cross validation procedure (see Allen (1971))
art 4b Asymptotc Results for MRR usg RESS Recall that the RESS statstc s a specal type of cross valdato procedure (see Alle (97)) partcular to the regresso problem ad volves fdg Y $,, the estmate at the
More informationEstimation of Stress- Strength Reliability model using finite mixture of exponential distributions
Iteratoal Joural of Computatoal Egeerg Research Vol, 0 Issue, Estmato of Stress- Stregth Relablty model usg fte mxture of expoetal dstrbutos K.Sadhya, T.S.Umamaheswar Departmet of Mathematcs, Lal Bhadur
More informationBlock-Based Compact Thermal Modeling of Semiconductor Integrated Circuits
Block-Based Compact hermal Modelg of Semcoductor Itegrated Crcuts Master s hess Defese Caddate: Jg Ba Commttee Members: Dr. Mg-Cheg Cheg Dr. Daqg Hou Dr. Robert Schllg July 27, 2009 Outle Itroducto Backgroud
More informationComparison of Dual to Ratio-Cum-Product Estimators of Population Mean
Research Joural of Mathematcal ad Statstcal Sceces ISS 30 6047 Vol. 1(), 5-1, ovember (013) Res. J. Mathematcal ad Statstcal Sc. Comparso of Dual to Rato-Cum-Product Estmators of Populato Mea Abstract
More informationLine Fitting and Regression
Marquette Uverst MSCS6 Le Fttg ad Regresso Dael B. Rowe, Ph.D. Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Coprght 8 b Marquette Uverst Least Squares Regresso MSCS6 For LSR we have pots
More informationDimensionality Reduction and Learning
CMSC 35900 (Sprg 009) Large Scale Learg Lecture: 3 Dmesoalty Reducto ad Learg Istructors: Sham Kakade ad Greg Shakharovch L Supervsed Methods ad Dmesoalty Reducto The theme of these two lectures s that
More informationFourth Order Four-Stage Diagonally Implicit Runge-Kutta Method for Linear Ordinary Differential Equations ABSTRACT INTRODUCTION
Malasa Joural of Mathematcal Sceces (): 95-05 (00) Fourth Order Four-Stage Dagoall Implct Ruge-Kutta Method for Lear Ordar Dfferetal Equatos Nur Izzat Che Jawas, Fudzah Ismal, Mohamed Sulema, 3 Azm Jaafar
More informationCHAPTER VI Statistical Analysis of Experimental Data
Chapter VI Statstcal Aalyss of Expermetal Data CHAPTER VI Statstcal Aalyss of Expermetal Data Measuremets do ot lead to a uque value. Ths s a result of the multtude of errors (maly radom errors) that ca
More informationLinear regression (cont.) Linear methods for classification
CS 75 Mache Lear Lecture 7 Lear reresso cot. Lear methods for classfcato Mlos Hausrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Lear Coeffcet shrae he least squares estmates ofte have lo bas but hh
More informationBayes (Naïve or not) Classifiers: Generative Approach
Logstc regresso Bayes (Naïve or ot) Classfers: Geeratve Approach What do we mea by Geeratve approach: Lear p(y), p(x y) ad the apply bayes rule to compute p(y x) for makg predctos Ths s essetally makg
More informationResearch Article A New Derivation and Recursive Algorithm Based on Wronskian Matrix for Vandermonde Inverse Matrix
Mathematcal Problems Egeerg Volume 05 Artcle ID 94757 7 pages http://ddoorg/055/05/94757 Research Artcle A New Dervato ad Recursve Algorthm Based o Wroska Matr for Vadermode Iverse Matr Qu Zhou Xja Zhag
More informationBeam Warming Second-Order Upwind Method
Beam Warmg Secod-Order Upwd Method Petr Valeta Jauary 6, 015 Ths documet s a part of the assessmet work for the subject 1DRP Dfferetal Equatos o Computer lectured o FNSPE CTU Prague. Abstract Ths documet
More informationCorrelation and Regression Analysis
Chapter V Correlato ad Regresso Aalss R. 5.. So far we have cosdered ol uvarate dstrbutos. Ma a tme, however, we come across problems whch volve two or more varables. Ths wll be the subject matter of the
More informationLecture 7: Linear and quadratic classifiers
Lecture 7: Lear ad quadratc classfers Bayes classfers for ormally dstrbuted classes Case : Σ σ I Case : Σ Σ (Σ daoal Case : Σ Σ (Σ o-daoal Case 4: Σ σ I Case 5: Σ Σ j eeral case Lear ad quadratc classfers:
More information9.1 Introduction to the probit and logit models
EC3000 Ecoometrcs Lecture 9 Probt & Logt Aalss 9. Itroducto to the probt ad logt models 9. The logt model 9.3 The probt model Appedx 9. Itroducto to the probt ad logt models These models are used regressos
More information6. Nonparametric techniques
6. Noparametrc techques Motvato Problem: how to decde o a sutable model (e.g. whch type of Gaussa) Idea: just use the orgal data (lazy learg) 2 Idea 1: each data pot represets a pece of probablty P(x)
More informationECONOMETRIC THEORY. MODULE VIII Lecture - 26 Heteroskedasticity
ECONOMETRIC THEORY MODULE VIII Lecture - 6 Heteroskedastcty Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur . Breusch Paga test Ths test ca be appled whe the replcated data
More informationMAX-MIN AND MIN-MAX VALUES OF VARIOUS MEASURES OF FUZZY DIVERGENCE
merca Jr of Mathematcs ad Sceces Vol, No,(Jauary 0) Copyrght Md Reader Publcatos wwwjouralshubcom MX-MIN ND MIN-MX VLUES OF VRIOUS MESURES OF FUZZY DIVERGENCE RKTul Departmet of Mathematcs SSM College
More informationStudy of Correlation using Bayes Approach under bivariate Distributions
Iteratoal Joural of Scece Egeerg ad Techolog Research IJSETR Volume Issue Februar 4 Stud of Correlato usg Baes Approach uder bvarate Dstrbutos N.S.Padharkar* ad. M.N.Deshpade** *Govt.Vdarbha Isttute of
More informationNonlinear Blind Source Separation Using Hybrid Neural Networks*
Nolear Bld Source Separato Usg Hybrd Neural Networks* Chu-Hou Zheg,2, Zh-Ka Huag,2, chael R. Lyu 3, ad Tat-g Lok 4 Itellget Computg Lab, Isttute of Itellget aches, Chese Academy of Sceces, P.O.Box 3, Hefe,
More informationDimensionality reduction Feature selection
CS 750 Mache Learg Lecture 3 Dmesoalty reducto Feature selecto Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 750 Mache Learg Dmesoalty reducto. Motvato. Classfcato problem eample: We have a put data
More informationA COMPARATIVE STUDY OF THE METHODS OF SOLVING NON-LINEAR PROGRAMMING PROBLEM
DAODIL INTERNATIONAL UNIVERSITY JOURNAL O SCIENCE AND TECHNOLOGY, VOLUME, ISSUE, JANUARY 9 A COMPARATIVE STUDY O THE METHODS O SOLVING NON-LINEAR PROGRAMMING PROBLEM Bmal Chadra Das Departmet of Tetle
More informationTESTS BASED ON MAXIMUM LIKELIHOOD
ESE 5 Toy E. Smth. The Basc Example. TESTS BASED ON MAXIMUM LIKELIHOOD To llustrate the propertes of maxmum lkelhood estmates ad tests, we cosder the smplest possble case of estmatg the mea of the ormal
More informationSTA 108 Applied Linear Models: Regression Analysis Spring Solution for Homework #1
STA 08 Appled Lear Models: Regresso Aalyss Sprg 0 Soluto for Homework #. Let Y the dollar cost per year, X the umber of vsts per year. The the mathematcal relato betwee X ad Y s: Y 300 + X. Ths s a fuctoal
More informationTo use adaptive cluster sampling we must first make some definitions of the sampling universe:
8.3 ADAPTIVE SAMPLING Most of the methods dscussed samplg theory are lmted to samplg desgs hch the selecto of the samples ca be doe before the survey, so that oe of the decsos about samplg deped ay ay
More informationArithmetic Mean and Geometric Mean
Acta Mathematca Ntresa Vol, No, p 43 48 ISSN 453-6083 Arthmetc Mea ad Geometrc Mea Mare Varga a * Peter Mchalča b a Departmet of Mathematcs, Faculty of Natural Sceces, Costate the Phlosopher Uversty Ntra,
More informationApplied Mathematics and Computation
Appled Mathematcs ad Computato 215 (2010) 4198 4202 Cotets lsts avalable at SceceDrect Appled Mathematcs ad Computato joural homepage: www.elsever.com/locate/amc Improvemet estmatg the populato mea smple
More informationOvercoming Limitations of Sampling for Aggregation Queries
CIS 6930 Approxmate Quer Processg Paper Presetato Sprg 2004 - Istructor: Dr Al Dobra Overcomg Lmtatos of Samplg for Aggregato Queres Authors: Surajt Chaudhur, Gautam Das, Maur Datar, Rajeev Motwa, ad Vvek
More informationGeneralized Minimum Perpendicular Distance Square Method of Estimation
Appled Mathematcs,, 3, 945-949 http://dx.do.org/.436/am..366 Publshed Ole December (http://.scrp.org/joural/am) Geeralzed Mmum Perpedcular Dstace Square Method of Estmato Rezaul Karm, Morshed Alam, M.
More informationESTIMATION OF MISCLASSIFICATION ERROR USING BAYESIAN CLASSIFIERS
Producto Systems ad Iformato Egeerg Volume 5 (2009), pp. 4-50. ESTIMATION OF MISCLASSIFICATION ERROR USING BAYESIAN CLASSIFIERS PÉTER BARABÁS Uversty of Msolc, Hugary Departmet of Iformato Techology barabas@t.u-msolc.hu
More informationCOMPROMISE HYPERSPHERE FOR STOCHASTIC DOMINANCE MODEL
Sebasta Starz COMPROMISE HYPERSPHERE FOR STOCHASTIC DOMINANCE MODEL Abstract The am of the work s to preset a method of rakg a fte set of dscrete radom varables. The proposed method s based o two approaches:
More informationPGE 310: Formulation and Solution in Geosystems Engineering. Dr. Balhoff. Interpolation
PGE 30: Formulato ad Soluto Geosystems Egeerg Dr. Balhoff Iterpolato Numercal Methods wth MATLAB, Recktewald, Chapter 0 ad Numercal Methods for Egeers, Chapra ad Caale, 5 th Ed., Part Fve, Chapter 8 ad
More informationGender Classification from ECG Signal Analysis using Least Square Support Vector Machine
Amerca Joural of Sgal Processg, (5): 45-49 DOI:.593/.asp.5.8 Geder Classfcato from ECG Sgal Aalyss usg Least Square Support Vector Mache Raesh Ku. rpathy,*, Ashutosh Acharya, Sumt Kumar Choudhary Departmet
More informationA Robust Total Least Mean Square Algorithm For Nonlinear Adaptive Filter
A Robust otal east Mea Square Algorthm For Nolear Adaptve Flter Ruxua We School of Electroc ad Iformato Egeerg X'a Jaotog Uversty X'a 70049, P.R. Cha rxwe@chare.com Chogzhao Ha, azhe u School of Electroc
More informationMachine Learning. knowledge acquisition skill refinement. Relation between machine learning and data mining. P. Berka, /18
Mache Learg The feld of mache learg s cocered wth the questo of how to costruct computer programs that automatcally mprove wth eperece. (Mtchell, 1997) Thgs lear whe they chage ther behavor a way that
More informationModel Fitting, RANSAC. Jana Kosecka
Model Fttg, RANSAC Jaa Kosecka Fttg: Issues Prevous strateges Le detecto Hough trasform Smple parametrc model, two parameters m, b m + b Votg strateg Hard to geeralze to hgher dmesos a o + a + a 2 2 +
More informationUnimodality Tests for Global Optimization of Single Variable Functions Using Statistical Methods
Malaysa Umodalty Joural Tests of Mathematcal for Global Optmzato Sceces (): of 05 Sgle - 5 Varable (007) Fuctos Usg Statstcal Methods Umodalty Tests for Global Optmzato of Sgle Varable Fuctos Usg Statstcal
More informationSummary of the lecture in Biostatistics
Summary of the lecture Bostatstcs Probablty Desty Fucto For a cotuos radom varable, a probablty desty fucto s a fucto such that: 0 dx a b) b a dx A probablty desty fucto provdes a smple descrpto of the
More informationPROJECTION PROBLEM FOR REGULAR POLYGONS
Joural of Mathematcal Sceces: Advaces ad Applcatos Volume, Number, 008, Pages 95-50 PROJECTION PROBLEM FOR REGULAR POLYGONS College of Scece Bejg Forestry Uversty Bejg 0008 P. R. Cha e-mal: sl@bjfu.edu.c
More informationAn Indian Journal FULL PAPER ABSTRACT KEYWORDS. Trade Science Inc. Research on scheme evaluation method of automation mechatronic systems
[ype text] [ype text] [ype text] ISSN : 0974-7435 Volume 0 Issue 6 Boechology 204 Ida Joural FULL PPER BIJ, 0(6, 204 [927-9275] Research o scheme evaluato method of automato mechatroc systems BSRC Che
More informationBayes Estimator for Exponential Distribution with Extension of Jeffery Prior Information
Malaysa Joural of Mathematcal Sceces (): 97- (9) Bayes Estmator for Expoetal Dstrbuto wth Exteso of Jeffery Pror Iformato Hadeel Salm Al-Kutub ad Noor Akma Ibrahm Isttute for Mathematcal Research, Uverst
More informationDelay-independent Fuzzy Hyperbolic Guaranteed Cost Control Design for a Class of Nonlinear Continuous-time Systems with Uncertainties 1)
Vol.31, No.5 ACTA AUTOMATICA SINICA September, 5 Delay-depedet Fuzzy Hyperbolc Guarateed Cost Cotrol Desg for a Class of Nolear Cotuous-tme Systems wth Ucertates 1) LUN Shu-Xa 1, ZHANG Hua-Guag 1,3 1 (Key
More informationA New Method for Decision Making Based on Soft Matrix Theory
Joural of Scetfc esearch & eports 3(5): 0-7, 04; rtcle o. JS.04.5.00 SCIENCEDOMIN teratoal www.scecedoma.org New Method for Decso Mag Based o Soft Matrx Theory Zhmg Zhag * College of Mathematcs ad Computer
More informationCHAPTER 4 RADICAL EXPRESSIONS
6 CHAPTER RADICAL EXPRESSIONS. The th Root of a Real Number A real umber a s called the th root of a real umber b f Thus, for example: s a square root of sce. s also a square root of sce ( ). s a cube
More informationDKA method for single variable holomorphic functions
DKA method for sgle varable holomorphc fuctos TOSHIAKI ITOH Itegrated Arts ad Natural Sceces The Uversty of Toushma -, Mamhosama, Toushma, 770-8502 JAPAN Abstract: - Durad-Kerer-Aberth (DKA method for
More informationAlgorithms behind the Correlation Setting Window
Algorths behd the Correlato Settg Wdow Itroducto I ths report detaled forato about the correlato settg pop up wdow s gve. See Fgure. Ths wdow s obtaed b clckg o the rado butto labelled Kow dep the a scree
More informationMULTIDIMENSIONAL HETEROGENEOUS VARIABLE PREDICTION BASED ON EXPERTS STATEMENTS. Gennadiy Lbov, Maxim Gerasimov
Iteratoal Boo Seres "Iformato Scece ad Computg" 97 MULTIIMNSIONAL HTROGNOUS VARIABL PRICTION BAS ON PRTS STATMNTS Geady Lbov Maxm Gerasmov Abstract: I the wors [ ] we proposed a approach of formg a cosesus
More informationChapter 11 Systematic Sampling
Chapter stematc amplg The sstematc samplg techue s operatoall more coveet tha the smple radom samplg. It also esures at the same tme that each ut has eual probablt of cluso the sample. I ths method of
More informationPinaki Mitra Dept. of CSE IIT Guwahati
Pak Mtra Dept. of CSE IIT Guwahat Hero s Problem HIGHWAY FACILITY LOCATION Faclty Hgh Way Farm A Farm B Illustrato of the Proof of Hero s Theorem p q s r r l d(p,r) + d(q,r) = d(p,q) p d(p,r ) + d(q,r
More informationThird handout: On the Gini Index
Thrd hadout: O the dex Corrado, a tala statstca, proposed (, 9, 96) to measure absolute equalt va the mea dfferece whch s defed as ( / ) where refers to the total umber of dvduals socet. Assume that. The
More information13. Parametric and Non-Parametric Uncertainties, Radial Basis Functions and Neural Network Approximations
Lecture 7 3. Parametrc ad No-Parametrc Ucertates, Radal Bass Fuctos ad Neural Network Approxmatos he parameter estmato algorthms descrbed prevous sectos were based o the assumpto that the system ucertates
More informationNew Schedule. Dec. 8 same same same Oct. 21. ^2 weeks ^1 week ^1 week. Pattern Recognition for Vision
ew Schedule Dec. 8 same same same Oct. ^ weeks ^ week ^ week Fall 004 Patter Recogto for Vso 9.93 Patter Recogto for Vso Classfcato Berd Hesele Fall 004 Overvew Itroducto Lear Dscrmat Aalyss Support Vector
More informationAnalysis of VMSS Schemes for Group Key Transfer Protocol
Aalss of VMSS Schemes for Group Ke Trasfer Protocol Chg-Fag Hsu, Sha Wu To cte ths verso: Chg-Fag Hsu, Sha Wu. Aalss of VMSS Schemes for Group Ke Trasfer Protocol. Chg- Hse Hsu; Xuahua Sh; Valeta Salapura.
More informationL5 Polynomial / Spline Curves
L5 Polyomal / Sple Curves Cotets Coc sectos Polyomal Curves Hermte Curves Bezer Curves B-Sples No-Uform Ratoal B-Sples (NURBS) Mapulato ad Represetato of Curves Types of Curve Equatos Implct: Descrbe a
More informationCan we take the Mysticism Out of the Pearson Coefficient of Linear Correlation?
Ca we tae the Mstcsm Out of the Pearso Coeffcet of Lear Correlato? Itroducto As the ttle of ths tutoral dcates, our purpose s to egeder a clear uderstadg of the Pearso coeffcet of lear correlato studets
More informationBayesian Classification. CS690L Data Mining: Classification(2) Bayesian Theorem: Basics. Bayesian Theorem. Training dataset. Naïve Bayes Classifier
Baa Classfcato CS6L Data Mg: Classfcato() Referece: J. Ha ad M. Kamber, Data Mg: Cocepts ad Techques robablstc learg: Calculate explct probabltes for hypothess, amog the most practcal approaches to certa
More information( ) = ( ) ( ) Chapter 13 Asymptotic Theory and Stochastic Regressors. Stochastic regressors model
Chapter 3 Asmptotc Theor ad Stochastc Regressors The ature of eplaator varable s assumed to be o-stochastc or fed repeated samples a regresso aalss Such a assumpto s approprate for those epermets whch
More informationChapter 2 Supplemental Text Material
-. Models for the Data ad the t-test Chapter upplemetal Text Materal The model preseted the text, equato (-3) s more properl called a meas model. ce the mea s a locato parameter, ths tpe of model s also
More informationDynamic Analysis of Axially Beam on Visco - Elastic Foundation with Elastic Supports under Moving Load
Dyamc Aalyss of Axally Beam o Vsco - Elastc Foudato wth Elastc Supports uder Movg oad Saeed Mohammadzadeh, Seyed Al Mosayeb * Abstract: For dyamc aalyses of ralway track structures, the algorthm of soluto
More informationCubic Nonpolynomial Spline Approach to the Solution of a Second Order Two-Point Boundary Value Problem
Joural of Amerca Scece ;6( Cubc Nopolyomal Sple Approach to the Soluto of a Secod Order Two-Pot Boudary Value Problem W.K. Zahra, F.A. Abd El-Salam, A.A. El-Sabbagh ad Z.A. ZAk * Departmet of Egeerg athematcs
More informationMechanics of Materials CIVL 3322 / MECH 3322
Mechacs of Materals CVL / MECH Cetrods ad Momet of erta Calculatos Cetrods = A = = = A = = Cetrod ad Momet of erta Calculatos z= z A = = Parallel As Theorem f ou kow the momet of erta about a cetrodal
More informationOn generalized fuzzy mean code word lengths. Department of Mathematics, Jaypee University of Engineering and Technology, Guna, Madhya Pradesh, India
merca Joural of ppled Mathematcs 04; (4): 7-34 Publshed ole ugust 30, 04 (http://www.scecepublshggroup.com//aam) do: 0.648/.aam.04004.3 ISSN: 330-0043 (Prt); ISSN: 330-006X (Ole) O geeralzed fuzzy mea
More informationAnalyzing Fuzzy System Reliability Using Vague Set Theory
Iteratoal Joural of Appled Scece ad Egeerg 2003., : 82-88 Aalyzg Fuzzy System Relablty sg Vague Set Theory Shy-Mg Che Departmet of Computer Scece ad Iformato Egeerg, Natoal Tawa versty of Scece ad Techology,
More informationEvaluating Polynomials
Uverst of Nebraska - Lcol DgtalCommos@Uverst of Nebraska - Lcol MAT Exam Expostor Papers Math the Mddle Isttute Partershp 7-7 Evaluatg Polomals Thomas J. Harrgto Uverst of Nebraska-Lcol Follow ths ad addtoal
More informationObjectives of Multiple Regression
Obectves of Multple Regresso Establsh the lear equato that best predcts values of a depedet varable Y usg more tha oe eplaator varable from a large set of potetal predctors {,,... k }. Fd that subset of
More informationFACE RECOGNITION BASED ON OPTIMAL KERNEL MINIMAX PROBABILITY MACHINE
Joural of heoretcal ad Appled Iformato echolog 8 th Feruar 03. Vol. 48 No.3 005-03 JAI & LLS. All rghts reserved. ISSN: 99-8645.jatt.org E-ISSN: 87-395 FACE RECOGNIION BASED ON OPIMAL KERNEL MINIMAX PROBABILIY
More informationV. Rezaie, T. Ahmad, C. Daneshfard, M. Khanmohammadi and S. Nejatian
World Appled Sceces Joural (Mathematcal Applcatos Egeerg): 38-4, 03 ISS 88-495 IDOSI Publcatos, 03 DOI: 0.589/dos.wasj.03..mae.99940 A Itegrated Modelg Based o Data Evelopmet Aalyss ad Support Vector Maches:
More informationLecture 16: Backpropogation Algorithm Neural Networks with smooth activation functions
CO-511: Learg Theory prg 2017 Lecturer: Ro Lv Lecture 16: Bacpropogato Algorthm Dsclamer: These otes have ot bee subected to the usual scruty reserved for formal publcatos. They may be dstrbuted outsde
More informationA tighter lower bound on the circuit size of the hardest Boolean functions
Electroc Colloquum o Computatoal Complexty, Report No. 86 2011) A tghter lower boud o the crcut sze of the hardest Boolea fuctos Masak Yamamoto Abstract I [IPL2005], Fradse ad Mlterse mproved bouds o the
More informationOrdinary Least Squares Regression. Simple Regression. Algebra and Assumptions.
Ordary Least Squares egresso. Smple egresso. Algebra ad Assumptos. I ths part of the course we are gog to study a techque for aalysg the lear relatoshp betwee two varables Y ad X. We have pars of observatos
More informationi 2 σ ) i = 1,2,...,n , and = 3.01 = 4.01
ECO 745, Homework 6 Le Cabrera. Assume that the followg data come from the lear model: ε ε ~ N, σ,,..., -6. -.5 7. 6.9 -. -. -.9. -..6.4.. -.6 -.7.7 Fd the mamum lkelhood estmates of,, ad σ ε s.6. 4. ε
More information