Adaptive and Iterative Least Squares Support Vector Regression Based on Quadratic Renyi Entropy
|
|
- Britney Wilkinson
- 5 years ago
- Views:
Transcription
1 daptve and Iteratve Least Squares Support Vector Regresson Based on Quadratc Ren Entrop Jngqng Jang, Chu Song, Haan Zhao, Chunguo u,3 and Yanchun Lang Coege of Mathematcs and Computer Scence, Inner Mongoa Unverst for atonates, ongao Inner Mongoa 843, Chna Coege of Computer Scence and echnoog, Jn Unverst, Changchun 3, Chna 3 atona Laborator of Pattern Recognton, Insttute of utomaton, Chnese cadem of Scences, Bejng 8, Chna jangjngqng@ahoo.com.cn bstract n adaptve and teratve LSSVR agorthm based on quadratc Ren entrop s presented n ths paper. LS-SVM oses the sparseness of support vector whch s one of the mportant advantages of conventona SVM. he proposed agorthm overcomes ths drawback. he quadratc Ren entrop s the evauatng crteron for workng set seecton, and the sze of workng set s determned at the process of teraton adaptve. he regresson parameters are cacuated b ncrementa earnng and the cacuaton of nversng a arge scae matrx s avoded. So the runnng speed s mproved. hs agorthm reserves we the sparseness of support vector and mproves the earnng speed.. Introducton he support vector machne (SVM s a nove earnng method that s constructed based on statstca earnng theor. he support vector machne has been studed wde snce t was presented n 995. It has been apped to pattern recognton broad and ts exceent performance has been shown n functon regresson probems. ranng a standard support vector machne requres the souton of a arge-scae quadratc programmng probem. hs s a dffcut probem when the number of the sampes exceeds a few thousands. Man agorthms for tranng the SVM have been studed. Sukens [] suggested a east squares support vector machne (LSSVM n whch the nequat constrans were repaced b equat constrans []. B ths wa, sovng a quadratc programmng was converted nto sovng near equatons. So the effcenc of tranng SVM s mproved great and the dffcut of tranng SVM s decreased. But a the tranng sampes are seected as support vectors and the sparseness of support vectors s destroed n LSSVM. LSSVM nvoves computng the nverse of a matrx n the process of tranng. It costs a ot of tme and space to computng the nverse matrx for arge scae tranng sampes. hs restrcts LSSVM apped on arge scae probems. Some researchers have nvoved n studng on the sparseness of support vectors and some resuts have been presented. Sukens[] proposed a prunng scheme based on support vector spectrum. he basc dea s to sort the support vector accordng to the correspondng Lagrange mutper and prune some support vectors wth the smaer Lagrange mutper. Recomputed usng the rest support vectors t the performance of the earnng machne decreased. hs method computed the Lagrange mutper for a the tranng sampes at begnnng so the cost on tme and space s arge. u [3] presented an adaptve and teratve agorthm for LSSVM tranng. he number of support vectors seected b ths method s decded adaptve through the ncrementa and nverse earnng. nd t s much smaer than the number of tranng sampes. However, the ftness for functon regresson s approach to the standard LSSVM. Espnoza [4-5] present a fxed-sze LSSVM agorthm and apped t to oad forecastng. he sze of support vectors was fxed he authors are gratefu to the support of Inner Mongoa atona Scence Foundaton (Grant o.787 and Inner Mongoa Unverst for atonates Scentfc Research Project (Grant o. MDB73, YB76, MDK73 Correspondng author, E-ma: cang@ju.edu.cn uthorzed censed use mted to: ISIUE OF UOMIO CS. Downoaded on March 5, 9 at : from IEEE Xpore. Restrctons app.
2 forehand and the maxmum quadratc Ren entrop was used to seect the support vectors nto workng set. ccordng to u[3] and Espnoza [4-5], combnng the Ren entrop wth ncrementa earnng agorthm, an adaptve and teratve LSSVM agorthm for regresson (LSSVR based on Ren entrop s presented n ths paper.. Least squares support vector regresson ccordng to [], et us consder a gven tranng set of sampes {, } wth the th nput datum n x x R and the th output datum R. he am of support vector machne mode s to construct the regresson functon takes the form: f ( x, w w ϕ ( x + b ( where the nonnear mappng ϕ ( maps the nput data nto a hgher dmensona feature space. In east squares support machne for functon regresson the foowng optmzaton probem s formuated mn J ( w, e w w + γ e ( w, e subject to the equat constrants w ϕ ( x + b + e,,..., (3 hs corresponds to a form of rdge regresson. he Lagrangan s gven b L( w, b, e, J ( w, e { w ϕ ( x + b + e } (4 wth Lagrange mutpers k. he condtons for the optmat are L w ϕ ( x L (5 b L γ e e L w ϕ ( x + b + e for,...,. fter emnatng e and w, we coud have the souton b the foowng near equatons b (6 Ω + γ I where [,..., ], [,...,], [,..., ] and the Mercer condton Ωkj ϕ ( xk ϕ( x j ψ( xk, x j k, j,..., (7 s apped. Set Ω + γ I. For s a smmetrc and postvedefnte matrx, exsts. Sovng the near equatons (6 we obtan the souton ( b b (8 Substtutng w n Eq. ( wth the frst equaton of Eqs. (5 and usng Eq. (7 we have f ( x, w ( x ψ ( x, x + b (9 where and b are the souton to Eqs(6. he kerne functon ψ ( can be chosen as (a near functon ψ ( x, x x x (b ponoma functon d ψ ( x, x ( x x + (c rada bass functon ( x, x exp{ x x / } 3. Incrementa earnng σ In order to obtan the regresson functon n Eq.(9, t needs to compute and b. nd computng s the ke to compute them. It costs arge on tme and space whe the number of tranng sampe s arge. Lu [6] presented an onne LSSVM earnng agorthm for functon and cassfcaton. hs agorthm avods computng the nverse of a arge matrx. he ncrement earnng part of the agorthm s used n our paper. he set whch eements (some of tranng sampes are used to construct the cassfer s caed workng set. Denotes as. ccordng to Eqs(6, set Ω+ γ I ( where s the number of sampes n current workng set. Eq. (8 can be rewrtten as b ( b ( where (,...,. he regresson functon s ψ ( x, x, + b hen a new comng sampe ( x +, + s added to the current workng set, we coud cacuate the parameters accordng to Eq. ( b ( + b + + ( where (,...,,, + ( +, + (, s constructed b added a ne and a coumn to +. hat s q + q s uthorzed censed use mted to: ISIUE OF UOMIO CS. Downoaded on March 5, 9 at : from IEEE Xpore. Restrctons app.
3 where q ( Ω, +, Ω, +,..., Ω, +, s Ω+, + + γ. hen the regresson functon changes to + +, ψ ( x, x + b + ccordng to the agorthm n reference Lu [6], the matrx n Eq. ( coud be cacuated from matrx +, that s q [ sq q] q + (3 + In ths wa the cacuaton for the nverse of a argescae matrx coud be avoded. 4. Ren Entrop Entrop s the measure of the degree of the sstem randomzaton. It s reated to the underng denst dstrbuton of the sampe. he arger the entrop s the more nformaton nvoves n sampe and the better the randomzaton of sampe s. Ren [7-8] gave the defnton for denst dstrbuton functon. he Ren entrop of order (, of a contnuous probabt denst functons p (x s defned n Eq. (4: H R og p ( x dx (4 hs paper s focus on Ren s quadratc entrop,, because ths eads to an mportant computatona smpfcaton obtaned for Gaussan kernes. he expresson of Ren quadratc entrop s gven b Eq.(5. og p ( x dx (5 H R p ( x dx can be estmated b [9] p ( x dx pˆ ( x dx ( x, x j ψ Ω j (6 where each vector has each eement equa to /. So the quadratc Ren entrop can be estmated b Eq.(7 H R og( ψ ( x, x j (7 j 5. daptve and teratve east squares support vector regresson based on Ren entrop here s no theor to nstruct the number of support vectors whch were seected nto workng set. n teratve agorthm s presented n ths paper and the number of support vectors n workng set s determned n the process of teraton. he quadratc Ren entrop s the evauatng crteron for seectng the support vector and the sampe that wth bg quadratc Ren entrop s seected nto workng set. Suppose the tranng set n { s s ( x,, x R, R,,,, } he regresson functon s ψ ( x, x + b where s the workng set whose eements are support vectors used to construct regresson functon. and b are decded b the workng set. Frst, carr out ntazaton. he frst sampes are seected to form the nta workng set. Compute and b usng Eq.(8 and compute. nd then repeat the foowng operaton t the stop crteron s reached: for the sampes s x, whch are not n ( workng set, put them nto workng set respectve and form the temporar workng set ˆ. Compute the quadratc Ren entrop for each temporar workng set. he bggest quadratc Ren entrop s seected and the correspondng sampe s j s put nto workng set. Compute and b usng the new b ncrementa earnng agorthm. Steps of the proposed agorthm are as foows: Intazaton: Set θ s the precson n tranng and testng, the precson n stop crteron s ε. Set {( x,,...,( x, } and cacuate anatca. Cacuate and b accordng to Eq.(8 and obtan the regresson functon. ( whe the stop crteron s fase do ( for,..., do (3 f s then (4 ˆ { s } // ˆ s temporar workng set for added sampe s (5 accordng to Eq.(7, compute the quadratc Ren entrop H ( ˆ R for temporar workng set ˆ (6 end f (7 end for (8 fnd the maxmum Ren entrop n H ( ˆ R, denote j arg max{ H ( ˆ R } (9 { s j } ( cacuate and b usng the sampes n workng set b ncrementa earnng ( cacuate the current object functon Q uthorzed censed use mted to: ISIUE OF UOMIO CS. Downoaded on March 5, 9 at : from IEEE Xpore. Restrctons app.
4 ( end whe (3 usng the sampes n workng set and, b to form the regresson functon In ths agorthm, the stop crteron s reated to the objectve functon. he objectve functon s + Q( w, e w γ s e s where w ϕ ( x, e. he meanng of s γ the defned stop crteron s that the procedure ends when the reatve error of objectve vaues n the two adjacent teratons s smaer than a gven precsonε Q current Qast ε Qast where Q current, Q are the objecton functon vaue ast for ths teraton and ast teraton respectve. 6. umerca Experment he experments are mpemented on a DELL PC, whch utzes a.8ghz Pentum IV processor wth 5MB memor. he OS s Mcrosoft ndows XP operatng sstem. the programs are comped under Mcrosoft s Vsua C In order to examne the effcenc of the proposed agorthm and compare wth LSSVR agorthm, numerca experments are performed usng three knds of data sets. he frst knd of data set s composed of the smp eementar functons whch ncude sn( x 5 x + 5 x x + 3 x x + hese functons are used to test the regresson abt for the known functon. he second knd of data set s composed of Macke-Gass (MG sstem and sampe functon sn c( x. he MG sstem s a bood ce reguaton mode estabshed n 977 b Macke and Gass. It s a chaos sstem dx a x( t τ b x( t dt + x ( t τ descrbed n [], where τ 7 a. b. Δt t (,4. he embedded dmensons are n 4,6, 8 respectve. he sampe functon s sn( x snc( x x 5 x 5 x x he thrd knd of data s spra functon. n RBF kerne functon ψ ( x, x exp( x x /(σ j j s empoed n these three knds functon. he parameters used n ths agorthm are showed n abe. he comparson between LSSVR and our agorthm are showed n abe, where the thrd coumn s the number of support vectors, the forth coumn s the seconds for tranng, and the ffth and seventh coumns are the regresson accurac for tranng and testng, respectve. he regresson accurac s a rato that s the number of sampes whose reatve error e s smaer than θ to the number of sampes n the workng set (testng set. he sxth and eghth coumns are the mean square error for tranng and testng, respectve. abe. Parameters used n agorthm data γ σ θ ε sn square 5... cube snc MG MG MG spra 5... It can be seen from abe that the earnng speed of our agorthm s much faster than LSSVR. Moreover, the number of support vectors s ess than that obtaned b LSSVR for the smar regresson accurac. abe 3 shows the rato of spendng tme and seected support vectors for the proposed agorthm to that for LSSVM. It can be seen that the tranng tme spent on the proposed agorthm s.59%--5.4% of standard LS-SVM, and the number of support vector s.85%--6.54% of tranng sampe. Fgure -4 show the dstrbuton of seected support vectors on sn, cubc, snc and spra functon. he back spots denote the support vectors. It can be seen that the dstrbuton of the support vector s homogeneous n the whoe sampe space and the dstrbuton refects the propert of the sampe space. uthorzed censed use mted to: ISIUE OF UOMIO CS. Downoaded on March 5, 9 at : from IEEE Xpore. Restrctons app.
5 abe. Comparson between the proposed agorthm (RLSSVR and standard LSSVR data *n gorthm name # of SVs ran tme (CPU s ccurac (tran% MSE (tran ccurac (test% MSE (test sn LSSVR e e-9 3 RLSSVR e e-8 square LSSVR e e-9 3 RLSSVR e e-6 cube LSSVR e e-9 3 RLSSVR e e-6 snc LSSVR e e- 3 RLSSVR e e-8 MG 4 LSSVR e-8.49e RLSSVR e-7 9.7e-7 MG 6 LSSVR e e RLSSVR e-7 4.3e-7 MG 8 LSSVR e-9 3.8e RLSSVR e-6.94e-6 spra LSSVR e e-4 5 RLSSVR e e- abe 3. he rato of support vector and speedng tme for the proposed agorthm to that for LSSVR data sn square cube snc MG 4 MG 6 MG 8 spra Rato of support vector 4.%.53%.4%.47%.97%.58%.85% 6.54% Rato of spendng tme 8.58%.69%.8%.8%.59% 5.7%.7% 5.4% Fgure. Support vectors for sn functon Fgure. Support vectors for cubc functon uthorzed censed use mted to: ISIUE OF UOMIO CS. Downoaded on March 5, 9 at : from IEEE Xpore. Restrctons app.
6 Fgure 3. Support vectors for snc functon Fgure 4. Support vectors for spra functon 7. Concuson and dscusson n adaptve and teratve LS-SVR agorthm based on quadratc Ren entrop s presented. the tranng sampes are support vectors n LS-SVM. So, LS-SVM oses the sparseness of support vector whch s one of the mportant advantages of conventona SVM. he proposed agorthm overcomes ths drawback. he quadratc Ren entrop s the evauatng crteron for workng set seecton, and the sze of workng set s determned at the process of teraton adaptve. For the entrop s a measure of sstem randomzaton, the bg entrop means that the sstem s randomzed we. he workng set wth bg entrop coud refect the aw of the sampe set. LS- SVM constructed b ths workng set has better generazaton. he regresson parameters are cacuated b ncrementa earnng and the cacuaton of nversng a arge scae matrx s avoded. So the runnng speed s mproved. e expermented on severa datasets. he tranng tme spent on the proposed agorthm s.59%--5.4% of standard LS- SVM, and the number of support vector s.85% % of tranng sampe for the smar regresson accurac and sma mean square error. hs agorthm reserves we the sparseness of support vector and mproves the earnng speed. References [] Sukens, J.. K., Vandewae, J. Least squares support vector machne cassfers, eura Processng Letter, 9(999, pp [] Sukens, J..K., Lukas, L., andewae, J. Sparse approxmaton usng east squares support vector machnes, In Proceedng of the IEEE Internatona Smposum on Crcuts and Sstems (ISCS, (, pp [3] u, C. G. Stud on Generazed Chromosome Genetc gorthm and Iteratve Least Squares Support Vector Machne Regresson, doctora dssertaton, Jn unverst, (6 [4] Espnoza, M., Sukens, J..K., Moor, B.D. Load forecastng usng fxed-sze east squares support vector machnes. 8th Internatona orkshop on rtfca eura etworks, I 5, Lecture otes n Computer Scence, 35(5, pp. 8-6 [5] Espnoza, M., Sukens, J..K., Moor, B.D. Fxed-sze east squares support vector machnes: a arge scae appcaton n eectrca oad forecastng. Computatona Management Scence, 3(6, pp. 3 9 [6] Lu, J. H., Chen, J.P. et a. Onne SL-SVM for functon and cassfcaton. Journa of Unverst of Scence and echnoog, (5 (3, pp,73-77 [7] Ren,. On measures of entrop and nformaton. Proceedngs of the Fourth Berkee Smposum on Mathematcs, Statstcs and Probabt, Unverst of Caforna Press, Berkee, C, (96, pp [8] Ren,. Introducton a a theore de nformaton. Cacu des probabtes. Dunod, Pars, (966 [9] Groam, M. Orthogona seres denst estmaton and the kerne egenvaue probem. eura Computaton, 4(3 (, pp []Fake G.., Lawrence S. Effcent SVM Regresson ranng wth SMO. Machne Learnng, 46 (, pp uthorzed censed use mted to: ISIUE OF UOMIO CS. Downoaded on March 5, 9 at : from IEEE Xpore. Restrctons app.
Research on Complex Networks Control Based on Fuzzy Integral Sliding Theory
Advanced Scence and Technoogy Letters Vo.83 (ISA 205), pp.60-65 http://dx.do.org/0.4257/ast.205.83.2 Research on Compex etworks Contro Based on Fuzzy Integra Sdng Theory Dongsheng Yang, Bngqng L, 2, He
More informationMARKOV CHAIN AND HIDDEN MARKOV MODEL
MARKOV CHAIN AND HIDDEN MARKOV MODEL JIAN ZHANG JIANZHAN@STAT.PURDUE.EDU Markov chan and hdden Markov mode are probaby the smpest modes whch can be used to mode sequenta data,.e. data sampes whch are not
More informationA finite difference method for heat equation in the unbounded domain
Internatona Conerence on Advanced ectronc Scence and Technoogy (AST 6) A nte derence method or heat equaton n the unbounded doman a Quan Zheng and Xn Zhao Coege o Scence North Chna nversty o Technoogy
More informationExample: Suppose we want to build a classifier that recognizes WebPages of graduate students.
Exampe: Suppose we want to bud a cassfer that recognzes WebPages of graduate students. How can we fnd tranng data? We can browse the web and coect a sampe of WebPages of graduate students of varous unverstes.
More informationDeriving the Dual. Prof. Bennett Math of Data Science 1/13/06
Dervng the Dua Prof. Bennett Math of Data Scence /3/06 Outne Ntty Grtty for SVM Revew Rdge Regresson LS-SVM=KRR Dua Dervaton Bas Issue Summary Ntty Grtty Need Dua of w, b, z w 2 2 mn st. ( x w ) = C z
More informationNONLINEAR SYSTEM IDENTIFICATION BASE ON FW-LSSVM
Journa of heoretca and Apped Informaton echnoogy th February 3. Vo. 48 No. 5-3 JAI & LLS. A rghts reserved. ISSN: 99-8645 www.jatt.org E-ISSN: 87-395 NONLINEAR SYSEM IDENIFICAION BASE ON FW-LSSVM, XIANFANG
More informationLecture Notes on Linear Regression
Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume
More informationImage Classification Using EM And JE algorithms
Machne earnng project report Fa, 2 Xaojn Sh, jennfer@soe Image Cassfcaton Usng EM And JE agorthms Xaojn Sh Department of Computer Engneerng, Unversty of Caforna, Santa Cruz, CA, 9564 jennfer@soe.ucsc.edu
More informationNeural network-based athletics performance prediction optimization model applied research
Avaabe onne www.jocpr.com Journa of Chemca and Pharmaceutca Research, 04, 6(6):8-5 Research Artce ISSN : 0975-784 CODEN(USA) : JCPRC5 Neura networ-based athetcs performance predcton optmzaton mode apped
More informationSupport Vector Machines. Vibhav Gogate The University of Texas at dallas
Support Vector Machnes Vbhav Gogate he Unversty of exas at dallas What We have Learned So Far? 1. Decson rees. Naïve Bayes 3. Lnear Regresson 4. Logstc Regresson 5. Perceptron 6. Neural networks 7. K-Nearest
More informationPredicting Model of Traffic Volume Based on Grey-Markov
Vo. No. Modern Apped Scence Predctng Mode of Traffc Voume Based on Grey-Marov Ynpeng Zhang Zhengzhou Muncpa Engneerng Desgn & Research Insttute Zhengzhou 5005 Chna Abstract Grey-marov forecastng mode of
More informationNumerical integration in more dimensions part 2. Remo Minero
Numerca ntegraton n more dmensons part Remo Mnero Outne The roe of a mappng functon n mutdmensona ntegraton Gauss approach n more dmensons and quadrature rues Crtca anass of acceptabt of a gven quadrature
More informationIDENTIFICATION OF NONLINEAR SYSTEM VIA SVR OPTIMIZED BY PARTICLE SWARM ALGORITHM
Journa of Theoretca and Apped Informaton Technoogy th February 3. Vo. 48 No. 5-3 JATIT & LLS. A rghts reserved. ISSN: 99-8645 www.att.org E-ISSN: 87-395 IDENTIFICATION OF NONLINEAR SYSTEM VIA SVR OPTIMIZED
More informationXin Li Department of Information Systems, College of Business, City University of Hong Kong, Hong Kong, CHINA
RESEARCH ARTICLE MOELING FIXE OS BETTING FOR FUTURE EVENT PREICTION Weyun Chen eartment of Educatona Informaton Technoogy, Facuty of Educaton, East Chna Norma Unversty, Shangha, CHINA {weyun.chen@qq.com}
More informationAssociative Memories
Assocatve Memores We consder now modes for unsupervsed earnng probems, caed auto-assocaton probems. Assocaton s the task of mappng patterns to patterns. In an assocatve memory the stmuus of an ncompete
More informationSupplementary Material: Learning Structured Weight Uncertainty in Bayesian Neural Networks
Shengyang Sun, Changyou Chen, Lawrence Carn Suppementary Matera: Learnng Structured Weght Uncertanty n Bayesan Neura Networks Shengyang Sun Changyou Chen Lawrence Carn Tsnghua Unversty Duke Unversty Duke
More informationNested case-control and case-cohort studies
Outne: Nested case-contro and case-cohort studes Ørnuf Borgan Department of Mathematcs Unversty of Oso NORBIS course Unversty of Oso 4-8 December 217 1 Radaton and breast cancer data Nested case contro
More informationModule 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Verson ECE IIT, Kharagpur Lesson 6 Theory of Quantzaton Verson ECE IIT, Kharagpur Instructonal Objectves At the end of ths lesson, the students should be able to:
More informationLINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity
LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 31 Multcollnearty Dr. Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur 6. Rdge regresson The OLSE s the best lnear unbased
More informationThe Study of Teaching-learning-based Optimization Algorithm
Advanced Scence and Technology Letters Vol. (AST 06), pp.05- http://dx.do.org/0.57/astl.06. The Study of Teachng-learnng-based Optmzaton Algorthm u Sun, Yan fu, Lele Kong, Haolang Q,, Helongang Insttute
More informationA New Refinement of Jacobi Method for Solution of Linear System Equations AX=b
Int J Contemp Math Scences, Vol 3, 28, no 17, 819-827 A New Refnement of Jacob Method for Soluton of Lnear System Equatons AX=b F Naem Dafchah Department of Mathematcs, Faculty of Scences Unversty of Gulan,
More informationChapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems
Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons
More informationOn the Equality of Kernel AdaTron and Sequential Minimal Optimization in Classification and Regression Tasks and Alike Algorithms for Kernel
Proceedngs of th European Symposum on Artfca Neura Networks, pp. 25-222, ESANN 2003, Bruges, Begum, 2003 On the Equaty of Kerne AdaTron and Sequenta Mnma Optmzaton n Cassfcaton and Regresson Tasks and
More informationShort-Term Load Forecasting for Electric Power Systems Using the PSO-SVR and FCM Clustering Techniques
Energes 20, 4, 73-84; do:0.3390/en40073 Artce OPEN ACCESS energes ISSN 996-073 www.mdp.com/journa/energes Short-Term Load Forecastng for Eectrc Power Systems Usng the PSO-SVR and FCM Custerng Technques
More informationLINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity
LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 30 Multcollnearty Dr. Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur 2 Remedes for multcollnearty Varous technques have
More informationLOW BIAS INTEGRATED PATH ESTIMATORS. James M. Calvin
Proceedngs of the 007 Wnter Smulaton Conference S G Henderson, B Bller, M-H Hseh, J Shortle, J D Tew, and R R Barton, eds LOW BIAS INTEGRATED PATH ESTIMATORS James M Calvn Department of Computer Scence
More informationAn Improved multiple fractal algorithm
Advanced Scence and Technology Letters Vol.31 (MulGraB 213), pp.184-188 http://dx.do.org/1.1427/astl.213.31.41 An Improved multple fractal algorthm Yun Ln, Xaochu Xu, Jnfeng Pang College of Informaton
More information10-701/ Machine Learning, Fall 2005 Homework 3
10-701/15-781 Machne Learnng, Fall 2005 Homework 3 Out: 10/20/05 Due: begnnng of the class 11/01/05 Instructons Contact questons-10701@autonlaborg for queston Problem 1 Regresson and Cross-valdaton [40
More informationDynamic Analysis Of An Off-Road Vehicle Frame
Proceedngs of the 8th WSEAS Int. Conf. on NON-LINEAR ANALYSIS, NON-LINEAR SYSTEMS AND CHAOS Dnamc Anass Of An Off-Road Vehce Frame ŞTEFAN TABACU, NICOLAE DORU STĂNESCU, ION TABACU Automotve Department,
More informationLinear Classification, SVMs and Nearest Neighbors
1 CSE 473 Lecture 25 (Chapter 18) Lnear Classfcaton, SVMs and Nearest Neghbors CSE AI faculty + Chrs Bshop, Dan Klen, Stuart Russell, Andrew Moore Motvaton: Face Detecton How do we buld a classfer to dstngush
More informationWhich Separator? Spring 1
Whch Separator? 6.034 - Sprng 1 Whch Separator? Mamze the margn to closest ponts 6.034 - Sprng Whch Separator? Mamze the margn to closest ponts 6.034 - Sprng 3 Margn of a pont " # y (w $ + b) proportonal
More informationGrover s Algorithm + Quantum Zeno Effect + Vaidman
Grover s Algorthm + Quantum Zeno Effect + Vadman CS 294-2 Bomb 10/12/04 Fall 2004 Lecture 11 Grover s algorthm Recall that Grover s algorthm for searchng over a space of sze wors as follows: consder the
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analyss of Varance and Desgn of Exerments-I MODULE III LECTURE - 2 EXPERIMENTAL DESIGN MODELS Dr. Shalabh Deartment of Mathematcs and Statstcs Indan Insttute of Technology Kanur 2 We consder the models
More informationWAVELET-BASED IMAGE COMPRESSION USING SUPPORT VECTOR MACHINE LEARNING AND ENCODING TECHNIQUES
WAVELE-BASED IMAGE COMPRESSION USING SUPPOR VECOR MACHINE LEARNING AND ENCODING ECHNIQUES Rakb Ahmed Gppsand Schoo of Computng and Informaton echnoogy Monash Unversty, Gppsand Campus Austraa. Rakb.Ahmed@nfotech.monash.edu.au
More information6.3.7 Example with Runga Kutta 4 th order method
6.3.7 Example wth Runga Kutta 4 th order method Agan, as an example, 3 machne, 9 bus system shown n Fg. 6.4 s agan consdered. Intally, the dampng of the generators are neglected (.e. d = 0 for = 1, 2,
More informationInexact Newton Methods for Inverse Eigenvalue Problems
Inexact Newton Methods for Inverse Egenvalue Problems Zheng-jan Ba Abstract In ths paper, we survey some of the latest development n usng nexact Newton-lke methods for solvng nverse egenvalue problems.
More informationThe line method combined with spectral chebyshev for space-time fractional diffusion equation
Apped and Computatona Mathematcs 014; 3(6): 330-336 Pubshed onne December 31, 014 (http://www.scencepubshnggroup.com/j/acm) do: 10.1164/j.acm.0140306.17 ISS: 3-5605 (Prnt); ISS: 3-5613 (Onne) The ne method
More informationOn the Power Function of the Likelihood Ratio Test for MANOVA
Journa of Mutvarate Anayss 8, 416 41 (00) do:10.1006/jmva.001.036 On the Power Functon of the Lkehood Rato Test for MANOVA Dua Kumar Bhaumk Unversty of South Aabama and Unversty of Inos at Chcago and Sanat
More informationRelevance Vector Machines Explained
October 19, 2010 Relevance Vector Machnes Explaned Trstan Fletcher www.cs.ucl.ac.uk/staff/t.fletcher/ Introducton Ths document has been wrtten n an attempt to make Tppng s [1] Relevance Vector Machnes
More informationLower Bounding Procedures for the Single Allocation Hub Location Problem
Lower Boundng Procedures for the Snge Aocaton Hub Locaton Probem Borzou Rostam 1,2 Chrstoph Buchhem 1,4 Fautät für Mathemat, TU Dortmund, Germany J. Faban Meer 1,3 Uwe Causen 1 Insttute of Transport Logstcs,
More informationThe Application of BP Neural Network principal component analysis in the Forecasting the Road Traffic Accident
ICTCT Extra Workshop, Bejng Proceedngs The Appcaton of BP Neura Network prncpa component anayss n Forecastng Road Traffc Accdent He Mng, GuoXucheng &LuGuangmng Transportaton Coege of Souast Unversty 07
More informationApplication of support vector machine in health monitoring of plate structures
Appcaton of support vector machne n heath montorng of pate structures *Satsh Satpa 1), Yogesh Khandare ), Sauvk Banerjee 3) and Anrban Guha 4) 1), ), 4) Department of Mechanca Engneerng, Indan Insttute
More informationKernel Methods and SVMs Extension
Kernel Methods and SVMs Extenson The purpose of ths document s to revew materal covered n Machne Learnng 1 Supervsed Learnng regardng support vector machnes (SVMs). Ths document also provdes a general
More informationSparse Training Procedure for Kernel Neuron *
Sparse ranng Procedure for Kerne Neuron * Janhua XU, Xuegong ZHANG and Yanda LI Schoo of Mathematca and Computer Scence, Nanng Norma Unversty, Nanng 0097, Jangsu Provnce, Chna xuanhua@ema.nnu.edu.cn Department
More informationNUMERICAL DIFFERENTIATION
NUMERICAL DIFFERENTIATION 1 Introducton Dfferentaton s a method to compute the rate at whch a dependent output y changes wth respect to the change n the ndependent nput x. Ths rate of change s called the
More informationSystems of Equations (SUR, GMM, and 3SLS)
Lecture otes on Advanced Econometrcs Takash Yamano Fall Semester 4 Lecture 4: Sstems of Equatons (SUR, MM, and 3SLS) Seemngl Unrelated Regresson (SUR) Model Consder a set of lnear equatons: $ + ɛ $ + ɛ
More informationLecture 10 Support Vector Machines II
Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the fake-test data; fxed
More informationADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING
1 ADVANCED ACHINE LEARNING ADVANCED ACHINE LEARNING Non-lnear regresson technques 2 ADVANCED ACHINE LEARNING Regresson: Prncple N ap N-dm. nput x to a contnuous output y. Learn a functon of the type: N
More informationLECTURE 21 Mohr s Method for Calculation of General Displacements. 1 The Reciprocal Theorem
V. DEMENKO MECHANICS OF MATERIALS 05 LECTURE Mohr s Method for Cacuaton of Genera Dspacements The Recproca Theorem The recproca theorem s one of the genera theorems of strength of materas. It foows drect
More informationSupport Vector Machines
CS 2750: Machne Learnng Support Vector Machnes Prof. Adrana Kovashka Unversty of Pttsburgh February 17, 2016 Announcement Homework 2 deadlne s now 2/29 We ll have covered everythng you need today or at
More informationIV. Performance Optimization
IV. Performance Optmzaton A. Steepest descent algorthm defnton how to set up bounds on learnng rate mnmzaton n a lne (varyng learnng rate) momentum learnng examples B. Newton s method defnton Gauss-Newton
More informationChapter 13: Multiple Regression
Chapter 13: Multple Regresson 13.1 Developng the multple-regresson Model The general model can be descrbed as: It smplfes for two ndependent varables: The sample ft parameter b 0, b 1, and b are used to
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analyss of Varance and Desgn of Experment-I MODULE VIII LECTURE - 34 ANALYSIS OF VARIANCE IN RANDOM-EFFECTS MODEL AND MIXED-EFFECTS EFFECTS MODEL Dr Shalabh Department of Mathematcs and Statstcs Indan
More informationBoundary Value Problems. Lecture Objectives. Ch. 27
Boundar Vaue Probes Ch. 7 Lecture Obectves o understand the dfference between an nta vaue and boundar vaue ODE o be abe to understand when and how to app the shootng ethod and FD ethod. o understand what
More informationDevelopment of whole CORe Thermal Hydraulic analysis code CORTH Pan JunJie, Tang QiFen, Chai XiaoMing, Lu Wei, Liu Dong
Deveopment of whoe CORe Therma Hydrauc anayss code CORTH Pan JunJe, Tang QFen, Cha XaoMng, Lu We, Lu Dong cence and technoogy on reactor system desgn technoogy, Nucear Power Insttute of Chna, Chengdu,
More informationThe Order Relation and Trace Inequalities for. Hermitian Operators
Internatonal Mathematcal Forum, Vol 3, 08, no, 507-57 HIKARI Ltd, wwwm-hkarcom https://doorg/0988/mf088055 The Order Relaton and Trace Inequaltes for Hermtan Operators Y Huang School of Informaton Scence
More informationPhysics 5153 Classical Mechanics. Principle of Virtual Work-1
P. Guterrez 1 Introducton Physcs 5153 Classcal Mechancs Prncple of Vrtual Work The frst varatonal prncple we encounter n mechancs s the prncple of vrtual work. It establshes the equlbrum condton of a mechancal
More informationEstimation: Part 2. Chapter GREG estimation
Chapter 9 Estmaton: Part 2 9. GREG estmaton In Chapter 8, we have seen that the regresson estmator s an effcent estmator when there s a lnear relatonshp between y and x. In ths chapter, we generalzed the
More informationLecture 12: Discrete Laplacian
Lecture 12: Dscrete Laplacan Scrbe: Tanye Lu Our goal s to come up wth a dscrete verson of Laplacan operator for trangulated surfaces, so that we can use t n practce to solve related problems We are mostly
More informationQuadratic speedup for unstructured search - Grover s Al-
Quadratc speedup for unstructured search - Grover s Al- CS 94- gorthm /8/07 Sprng 007 Lecture 11 001 Unstructured Search Here s the problem: You are gven a boolean functon f : {1,,} {0,1}, and are promsed
More informationwe have E Y x t ( ( xl)) 1 ( xl), e a in I( Λ ) are as follows:
APPENDICES Aendx : the roof of Equaton (6 For j m n we have Smary from Equaton ( note that j '( ( ( j E Y x t ( ( x ( x a V ( ( x a ( ( x ( x b V ( ( x b V x e d ( abx ( ( x e a a bx ( x xe b a bx By usng
More informationErrors for Linear Systems
Errors for Lnear Systems When we solve a lnear system Ax b we often do not know A and b exactly, but have only approxmatons  and ˆb avalable. Then the best thng we can do s to solve ˆx ˆb exactly whch
More informationOptimization of JK Flip Flop Layout with Minimal Average Power of Consumption based on ACOR, Fuzzy-ACOR, GA, and Fuzzy-GA
Journa of mathematcs and computer Scence 4 (05) - 5 Optmzaton of JK Fp Fop Layout wth Mnma Average Power of Consumpton based on ACOR, Fuzzy-ACOR, GA, and Fuzzy-GA Farshd Kevanan *,, A Yekta *,, Nasser
More informationAn Iterative Modified Kernel for Support Vector Regression
An Iteratve Modfed Kernel for Support Vector Regresson Fengqng Han, Zhengxa Wang, Mng Le and Zhxang Zhou School of Scence Chongqng Jaotong Unversty Chongqng Cty, Chna Abstract In order to mprove the performance
More informationCS 3710: Visual Recognition Classification and Detection. Adriana Kovashka Department of Computer Science January 13, 2015
CS 3710: Vsual Recognton Classfcaton and Detecton Adrana Kovashka Department of Computer Scence January 13, 2015 Plan for Today Vsual recognton bascs part 2: Classfcaton and detecton Adrana s research
More informationActive Learning with Support Vector Machines for Tornado Prediction
Actve Learnng wth Support Vector Machnes for Tornado Predcton Theodore B. Trafas, Indra Adranto, and Mchae B. Rchman Schoo of Industra Engneerng, Unversty of Okahoma, 0 West Boyd St, Room 4, Norman, OK
More informationUsing T.O.M to Estimate Parameter of distributions that have not Single Exponential Family
IOSR Journal of Mathematcs IOSR-JM) ISSN: 2278-5728. Volume 3, Issue 3 Sep-Oct. 202), PP 44-48 www.osrjournals.org Usng T.O.M to Estmate Parameter of dstrbutons that have not Sngle Exponental Famly Jubran
More informationVQ widely used in coding speech, image, and video
at Scalar quantzers are specal cases of vector quantzers (VQ): they are constraned to look at one sample at a tme (memoryless) VQ does not have such constrant better RD perfomance expected Source codng
More information[WAVES] 1. Waves and wave forces. Definition of waves
1. Waves and forces Defnton of s In the smuatons on ong-crested s are consdered. The drecton of these s (μ) s defned as sketched beow n the goba co-ordnate sstem: North West East South The eevaton can
More informationLogistic Regression. CAP 5610: Machine Learning Instructor: Guo-Jun QI
Logstc Regresson CAP 561: achne Learnng Instructor: Guo-Jun QI Bayes Classfer: A Generatve model odel the posteror dstrbuton P(Y X) Estmate class-condtonal dstrbuton P(X Y) for each Y Estmate pror dstrbuton
More informationLecture 20: November 7
0-725/36-725: Convex Optmzaton Fall 205 Lecturer: Ryan Tbshran Lecture 20: November 7 Scrbes: Varsha Chnnaobreddy, Joon Sk Km, Lngyao Zhang Note: LaTeX template courtesy of UC Berkeley EECS dept. Dsclamer:
More informationReactive Power Allocation Using Support Vector Machine
Reactve Power Aocaton Usng Support Vector Machne M.W. Mustafa, S.N. Khad, A. Kharuddn Facuty of Eectrca Engneerng, Unverst Teknoog Maaysa Johor 830, Maaysa and H. Shareef Facuty of Eectrca Engneerng and
More informationExponential Type Product Estimator for Finite Population Mean with Information on Auxiliary Attribute
Avalable at http://pvamu.edu/aam Appl. Appl. Math. ISSN: 193-9466 Vol. 10, Issue 1 (June 015), pp. 106-113 Applcatons and Appled Mathematcs: An Internatonal Journal (AAM) Exponental Tpe Product Estmator
More informationA DIMENSION-REDUCTION METHOD FOR STOCHASTIC ANALYSIS SECOND-MOMENT ANALYSIS
A DIMESIO-REDUCTIO METHOD FOR STOCHASTIC AALYSIS SECOD-MOMET AALYSIS S. Rahman Department of Mechanca Engneerng and Center for Computer-Aded Desgn The Unversty of Iowa Iowa Cty, IA 52245 June 2003 OUTLIE
More informationDesign and Optimization of Fuzzy Controller for Inverse Pendulum System Using Genetic Algorithm
Desgn and Optmzaton of Fuzzy Controller for Inverse Pendulum System Usng Genetc Algorthm H. Mehraban A. Ashoor Unversty of Tehran Unversty of Tehran h.mehraban@ece.ut.ac.r a.ashoor@ece.ut.ac.r Abstract:
More informationLearning Theory: Lecture Notes
Learnng Theory: Lecture Notes Lecturer: Kamalka Chaudhur Scrbe: Qush Wang October 27, 2012 1 The Agnostc PAC Model Recall that one of the constrants of the PAC model s that the data dstrbuton has to be
More informationThe Entire Solution Path for Support Vector Machine in Positive and Unlabeled Classification 1
Abstract The Entre Souton Path for Support Vector Machne n Postve and Unabeed Cassfcaton 1 Yao Lmn, Tang Je, and L Juanz Department of Computer Scence, Tsnghua Unversty 1-308, FIT, Tsnghua Unversty, Bejng,
More informationMACHINE APPLIED MACHINE LEARNING LEARNING. Gaussian Mixture Regression
11 MACHINE APPLIED MACHINE LEARNING LEARNING MACHINE LEARNING Gaussan Mture Regresson 22 MACHINE APPLIED MACHINE LEARNING LEARNING Bref summary of last week s lecture 33 MACHINE APPLIED MACHINE LEARNING
More informationDr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analyss of Varance and Desgn of Experment-I MODULE VII LECTURE - 3 ANALYSIS OF COVARIANCE Dr Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur Any scentfc experment s performed
More informationOn an Extension of Stochastic Approximation EM Algorithm for Incomplete Data Problems. Vahid Tadayon 1
On an Extenson of Stochastc Approxmaton EM Algorthm for Incomplete Data Problems Vahd Tadayon Abstract: The Stochastc Approxmaton EM (SAEM algorthm, a varant stochastc approxmaton of EM, s a versatle tool
More informationMeasurement Indices of Positional Uncertainty for Plane Line Segments Based on the ε
Proceedngs of the 8th Internatonal Smposum on Spatal ccurac ssessment n Natural Resources and Envronmental Scences Shangha, P R Chna, June 5-7, 008, pp 9-5 Measurement Indces of Postonal Uncertant for
More informationDelay tomography for large scale networks
Deay tomography for arge scae networks MENG-FU SHIH ALFRED O. HERO III Communcatons and Sgna Processng Laboratory Eectrca Engneerng and Computer Scence Department Unversty of Mchgan, 30 Bea. Ave., Ann
More informationA Network Intrusion Detection Method Based on Improved K-means Algorithm
Advanced Scence and Technology Letters, pp.429-433 http://dx.do.org/10.14257/astl.2014.53.89 A Network Intruson Detecton Method Based on Improved K-means Algorthm Meng Gao 1,1, Nhong Wang 1, 1 Informaton
More informationA General Column Generation Algorithm Applied to System Reliability Optimization Problems
A Genera Coumn Generaton Agorthm Apped to System Reabty Optmzaton Probems Lea Za, Davd W. Cot, Department of Industra and Systems Engneerng, Rutgers Unversty, Pscataway, J 08854, USA Abstract A genera
More informationChapter 6 Hidden Markov Models. Chaochun Wei Spring 2018
896 920 987 2006 Chapter 6 Hdden Markov Modes Chaochun We Sprng 208 Contents Readng materas Introducton to Hdden Markov Mode Markov chans Hdden Markov Modes Parameter estmaton for HMMs 2 Readng Rabner,
More informationMore metrics on cartesian products
More metrcs on cartesan products If (X, d ) are metrc spaces for 1 n, then n Secton II4 of the lecture notes we defned three metrcs on X whose underlyng topologes are the product topology The purpose of
More informationA New Evolutionary Computation Based Approach for Learning Bayesian Network
Avalable onlne at www.scencedrect.com Proceda Engneerng 15 (2011) 4026 4030 Advanced n Control Engneerng and Informaton Scence A New Evolutonary Computaton Based Approach for Learnng Bayesan Network Yungang
More informationEfficient, General Point Cloud Registration with Kernel Feature Maps
Effcent, General Pont Cloud Regstraton wth Kernel Feature Maps Hanchen Xong, Sandor Szedmak, Justus Pater Insttute of Computer Scence Unversty of Innsbruck 30 May 2013 Hanchen Xong (Un.Innsbruck) 3D Regstraton
More informationFuzzy Boundaries of Sample Selection Model
Proceedngs of the 9th WSES Internatonal Conference on ppled Mathematcs, Istanbul, Turkey, May 7-9, 006 (pp309-34) Fuzzy Boundares of Sample Selecton Model L. MUHMD SFIIH, NTON BDULBSH KMIL, M. T. BU OSMN
More informationComposite Hypotheses testing
Composte ypotheses testng In many hypothess testng problems there are many possble dstrbutons that can occur under each of the hypotheses. The output of the source s a set of parameters (ponts n a parameter
More informationNon-linear Canonical Correlation Analysis Using a RBF Network
ESANN' proceedngs - European Smposum on Artfcal Neural Networks Bruges (Belgum), 4-6 Aprl, d-sde publ., ISBN -97--, pp. 57-5 Non-lnear Canoncal Correlaton Analss Usng a RBF Network Sukhbnder Kumar, Elane
More informationDynamic Programming. Preview. Dynamic Programming. Dynamic Programming. Dynamic Programming (Example: Fibonacci Sequence)
/24/27 Prevew Fbonacc Sequence Longest Common Subsequence Dynamc programmng s a method for solvng complex problems by breakng them down nto smpler sub-problems. It s applcable to problems exhbtng the propertes
More informationChapter 11: Simple Linear Regression and Correlation
Chapter 11: Smple Lnear Regresson and Correlaton 11-1 Emprcal Models 11-2 Smple Lnear Regresson 11-3 Propertes of the Least Squares Estmators 11-4 Hypothess Test n Smple Lnear Regresson 11-4.1 Use of t-tests
More informationPop-Click Noise Detection Using Inter-Frame Correlation for Improved Portable Auditory Sensing
Advanced Scence and Technology Letters, pp.164-168 http://dx.do.org/10.14257/astl.2013 Pop-Clc Nose Detecton Usng Inter-Frame Correlaton for Improved Portable Audtory Sensng Dong Yun Lee, Kwang Myung Jeon,
More informationA PROBABILITY-DRIVEN SEARCH ALGORITHM FOR SOLVING MULTI-OBJECTIVE OPTIMIZATION PROBLEMS
HCMC Unversty of Pedagogy Thong Nguyen Huu et al. A PROBABILITY-DRIVEN SEARCH ALGORITHM FOR SOLVING MULTI-OBJECTIVE OPTIMIZATION PROBLEMS Thong Nguyen Huu and Hao Tran Van Department of mathematcs-nformaton,
More informationChapter - 2. Distribution System Power Flow Analysis
Chapter - 2 Dstrbuton System Power Flow Analyss CHAPTER - 2 Radal Dstrbuton System Load Flow 2.1 Introducton Load flow s an mportant tool [66] for analyzng electrcal power system network performance. Load
More informationApplication of Particle Swarm Optimization to Economic Dispatch Problem: Advantages and Disadvantages
Appcaton of Partce Swarm Optmzaton to Economc Dspatch Probem: Advantages and Dsadvantages Kwang Y. Lee, Feow, IEEE, and Jong-Bae Par, Member, IEEE Abstract--Ths paper summarzes the state-of-art partce
More information1 Convex Optimization
Convex Optmzaton We wll consder convex optmzaton problems. Namely, mnmzaton problems where the objectve s convex (we assume no constrants for now). Such problems often arse n machne learnng. For example,
More informationSupport Vector Machines
Separatng boundary, defned by w Support Vector Machnes CISC 5800 Professor Danel Leeds Separatng hyperplane splts class 0 and class 1 Plane s defned by lne w perpendcular to plan Is data pont x n class
More informationGeneralized Linear Methods
Generalzed Lnear Methods 1 Introducton In the Ensemble Methods the general dea s that usng a combnaton of several weak learner one could make a better learner. More formally, assume that we have a set
More information