A MESHSIZE BOOSTING ALGORITHM IN KERNEL DENSITY ESTIMATION
|
|
- Barnard Hampton
- 5 years ago
- Views:
Transcription
1 A eshsize boosting algorith in kernel density estiation A MESHSIZE BOOSTING ALGORITHM IN KERNEL DENSITY ESTIMATION C.C. Ishiekwene, S.M. Ogbonwan and J.E. Osewenkhae Departent of Matheatics, University of Benin, Benin City, Nigeria ABSTRACT This paper proposes a new algorith for boosting in kernel density estiation (KDE). This algorith enjoys the property of a bias reduction technique like other existing boosting algoriths and also enjoys the property of less function evaluations when copared with other boosting schees. Nuerical exaples are used and copared with existing algorith and the findings are coparatively interesting. Keywords: Boosting, kernel density estiation, bias reduction, boosting algorith. INTRODUCTION Kernel density estiation (KDE) is a process of constructing a density function fro a given set of data. The process depends on the choice of the soothing paraeter which is not easily found (Silveran, 1986; Jones and Signorini, 1997; and and Jones, 1995). As a result of this proble of finding a soothing paraeter, the idea of boosting in KDE is born. The area of boosting is relatively new in kernel density estiation. It was first proposed by Schapire (1990). Other contributors are Freund (1995), Freund and Schapire (1996), Schapire and Singer (1999). Boosting is a eans of iproving the perforance of a "weak learner" in the sense that given sufficient data, it would be guaranteed to produce an error rate which is better than "rando guessing". Mazio and Taylor (004) proposed an algorith in which a kernel density classifier is boosted by suitably re-weighting the data. This weight placed on the kernel estiator is a ratio of the log function in which the denoinator is a leave-one-out estiate of the density function. Also, Mazio and Taylor s (004) work theoretically gave an explanation and finally showed how boosting is able to reduce the bias in the asyptotic ean integrated squared error (AMISE) of Silveran (1986). In section, we shall see how the Mazio and Taylor's (004) algorith is a bias reduction technique. Section 3 shall show a proposed eshsize algorith and its liitations. Nuerical exaples Journal of Science and Technology, Vol. 8, No., August,
2 A eshsize boosting algorith in kernel density estiation are used to justify this algorith in section 4 and copare the results with that of Mazio and Taylor (004). Algorith on boosting kernel density estiates and bias reduction Throughout this paper, we shall assue our data to be univariate. The algorith of Mazio and Taylor (004) is briefly suarized in algorith 1. Algorith 1 Step 1: Given {x 1, i=1. n}, initialize 1 =1/n Step : Select h (the soothing paraeter). Step 3: For =1,, M, obtain a weighted kernel estiate ( i) n x x fˆ i = k (.1) i= 1 h h where x can be any value within the range of the x 1 s, k is the kernel function and w is a weight function; and then update the weights according to ( i) = ( i) + 1 ˆ Step 4: Provide output as M = 1 f ˆ ( x ) ( x ) fˆ i + log ( 1 ) (.) f ( xi ) renoralized to integrate to unity For full ipleentation of this algorith see Marzio and Taylor (004). Boosting as a Bias Reduction in Kernel Density Estiation Suppose we want to estiate f(x) by a ultiplicative estiate. e also suppose that we use only "weak" estiates which are such that h does not tend to zero as n. Let us use a population version instead of saple in which our weak learner, for h >0 is given by x y fˆ = 1 ( y) K f ( y)dy (.3) h h where 1 (y) is taken to be 1. e shall take our kernel function to be Gaussian (since all distributions tend to be noral as n, the saple size, becoes large through central liit theory (Towers, 00). The first approxiation in the Taylor's series, valid for h < 1 provided that the derivatives of f(x) are properly behaved, is fˆ ( 1) = f h f + and so we observe the usual bias of order 0(h ) of and and Jones (1995). If we now let ( ) ˆ x = f( ) ( x+ zh) 4 + ( h ) f ( x zh)dz f f ˆ = k( z) f ( x+ zh) + h ( ) 1 h f = + 0 f h This gives an overall estiator at the second step as 4 h f ( ) h 1 ˆ ( )( ) ˆ f f 1 x f( ) = f 1 + h f f 4 0( h ) f + (.4) 4 ( h ) (.5) which is clearly of order four and so we can see a bias reduction fro order two to order four. Proposed eshsize algorith in boosting In this section, we shall see how the leave-oneout estiator of the (.) in the weight function can be replaced by a eshsize estiator due to the tie coplexity involved in the leave-one-out estiator. In the leave-one-out estiator, we require (n+(n-1)).n function evaluations of the density for each boosting step. Thus, we are using a eshsize in its place. The only liitation on this eshsize algorith is that we ust first deterine the quantity 1/nh so as to know what the eshsize that would be placed on the weight function of (.) would be. The need to use a eshsize in place of the leave-one-out lies on the 70 Journal of Science and Technology, Vol. 8, No., August, 008
3 A eshsize boosting algorith in kernel density estiation fact that boosting is like the steepest-descent algorith in unconstrained optiization and thus a good substitute that approxiates the leave-one-out estiate of the function (Duffy and Helbold, 000; Taha, 1971; Ratsch et al., 000; Mannor et al., 001). The new eshsize algorith is stated as: Algorith Step 1: Given {x 1, i=1. n}, initialize 1 =1/n, initialize Step : Select h (the soothing paraeter). Step 3: For =1,, M, i) Get fˆ n = ( i) x xi k i= 1 h h where x can be any value within the range of the x 1 s, k is the kernel function and w is a weight function ii) Update ( i) = ( i) esh +1 + Step 4: Provide output M = 1 f ˆ ( x ) and noralize to integrate to unity Table 4.1 Showing bias reduction e can see that the weight function uses a eshsize instead of the leave-one-out log ratio function of Mazio and Taylor (004). The nuerical verification of this algorith would be seen in section 4. NUMERICAL RESULTS In this section, we shall use three sets of data (data 1 is the lifespan of a car battery in years, data is the nuber of written words without istakes in every 100 words by a set of students and data 3 is the scar length of patients randoly sapled. See Ishiekwene and Osewenkhae, 006) which were suspected to be noral to illustrate algoriths 1 and and copare their results. BASIC prograing language is used to generate the results for a specified h (Ishiekwene and Osewenkhae (006) and their corresponding graphs shown in figures 1a 1c for data 1(n=40), figures a c for data (n=64) while figures 3a 3c is for data 3(n=110). Also, table 4.1 shows the bias reduction we discussed in section for the three sets of data used. e can clearly see the bias reduction between the noral kernel estiator and the boosted kernel estiator. e also copared our boosted kernel estiator with the fourth-order kernel estiator of Jones and Signorini (1997) which is arguably the best soothing paraeter choice and an equivalent of Ishiekwene and Osewenkhae s (006) higher-order choice at order four. CONCLUSION Fro the graphs in figures 1A 3C for the three sets of data, we can clearly see that our proposed NORMAL (conventional order two) H4 (as in Jones and Signorini 1997 and Ishiekwene and Osewenkhae 006) BOOSTED (proposed schee) Bias Var AMISE Bias var AMISE Bias var AMISE Data Data Data Journal of Science and Technology, Vol. 8, No., August,
4 A eshsize boosting algorith in kernel density estiation.50e-01.00e-01 F D P 5.00E-0.00E E E E E E E E E+00 Fig. 1A (Leave-one out Algoriths).50E-01.00E-01.00E-0 Fig. A (Leave-One-Out Algoriths).50E-01.00E E E Fig. 1B (Meshsize Algoriths) Fig. B (Meshsize Algoriths) F D P.50E-01.00E E-0 LEAVE-ONE-OUT MESHSIZE Fig. 1C (Coparison of both Algoriths).50E-01.00E-01 LEAVE-ONE-OUT MESHSIZE F D P 5.00E-0 Fig. C; (Coparison of Both Algoriths) 7 Journal of Science and Technology, Vol. 8, No., August, 008
5 A eshsize boosting algorith in kernel density estiation Fig. 3A (Leave-One-Out Algoriths) P D F Fig. 3B (Meshsize Algoriths) LEAVE-ONE-OUT MESHSIZE Fig. 3C (Coparison of both Algoriths) algorith copares favorably if not arguably better than that of Mazio and Taylor (004). Also, the results of Table 4.1 has revealed the bias reduction targeted by both algoriths. Since our algorith does the sae task with fewer function evaluations and in fact better in ters of tie coplexity, we are recoending it for use in boosting in KDE. REFERENCES Duffy, N. and Helbold, D. (000). Potential bosters? Advances in Neural info. Proc. Sys. 1: Freund, Y. (1995). Boosting a weak learning algorith by ajority. Info Cop. 11: Freund, Y. and Schapire, R. (1996). Experients with a new boosting algorith. In achine learning: proceedings of the 1 th Intl. conf. Ed. L. Saite Ishiekwene, C. C. and Osewenkhae, J. E. (006). A coparison of fourth order window width selectors in KDE (A Univarate case). ABACUS 33(A): Jones, M.C. and Signorini, D.F. (1997). A coparison of Higher-order Bias Kernel Density Estiators. JASA 9: Mannor, S., Meir, R. and Mendelson, S. (001). On consistency of boosting algoriths. (Unpublished Manuscript). Mazio, D. M. and Taylor, C.C. (004). Boosting Kernel Density estiates: A bias reduction technique? Bioetrika 91: Ratsch, G. Mika, S., Scholkopf, B. and Muller, K. K. (000). Construction boosting algoriths fro SVM's: An application to-one-class classification. GMD Tech Report No. 1. Schapire, R. (1990). The strength of weak learnability. Machine learn. 5: Schapire, R. and Singer, Y. (1999). Iproved boosting algorith using confidence rated prediction. Machine learn. 37: Journal of Science and Technology, Vol. 8, No., August,
6 A eshsize boosting algorith in kernel density estiation Silveran, B.. (1986). Density Estiation for Statistics and Data analysis. Chapan and Hall, London; Taha, H. A. (1971). Operations Research: An Introduction Prentice Hall, New Jersey; Towers, S. (00). Kernel Probability density estiation ethods. In; M. halley, L. Lyons (Eds), Advanced Statistical techniques in particle Physics, Durha,England, and, M.P. and Jones, M.C. (1995). Kernel Soothing. Chapan and Hall/CRC, London; enxin, J. (001). Soe theoretical aspects of boosting in the presence of noisy data. Nueral Coputation 14(10): Journal of Science and Technology, Vol. 8, No., August, 008
Machine Learning Basics: Estimators, Bias and Variance
Machine Learning Basics: Estiators, Bias and Variance Sargur N. srihari@cedar.buffalo.edu This is part of lecture slides on Deep Learning: http://www.cedar.buffalo.edu/~srihari/cse676 1 Topics in Basics
More informationCombining Classifiers
Cobining Classifiers Generic ethods of generating and cobining ultiple classifiers Bagging Boosting References: Duda, Hart & Stork, pg 475-480. Hastie, Tibsharini, Friedan, pg 246-256 and Chapter 10. http://www.boosting.org/
More informationBoosting with log-loss
Boosting with log-loss Marco Cusuano-Towner Septeber 2, 202 The proble Suppose we have data exaples {x i, y i ) i =... } for a two-class proble with y i {, }. Let F x) be the predictor function with the
More informationEnsemble Based on Data Envelopment Analysis
Enseble Based on Data Envelopent Analysis So Young Sohn & Hong Choi Departent of Coputer Science & Industrial Systes Engineering, Yonsei University, Seoul, Korea Tel) 82-2-223-404, Fax) 82-2- 364-7807
More informationIntelligent Systems: Reasoning and Recognition. Perceptrons and Support Vector Machines
Intelligent Systes: Reasoning and Recognition Jaes L. Crowley osig 1 Winter Seester 2018 Lesson 6 27 February 2018 Outline Perceptrons and Support Vector achines Notation...2 Linear odels...3 Lines, Planes
More informationE. Alpaydın AERFAISS
E. Alpaydın AERFAISS 00 Introduction Questions: Is the error rate of y classifier less than %? Is k-nn ore accurate than MLP? Does having PCA before iprove accuracy? Which kernel leads to highest accuracy
More informationCh 12: Variations on Backpropagation
Ch 2: Variations on Backpropagation The basic backpropagation algorith is too slow for ost practical applications. It ay take days or weeks of coputer tie. We deonstrate why the backpropagation algorith
More informationFoundations of Machine Learning Boosting. Mehryar Mohri Courant Institute and Google Research
Foundations of Machine Learning Boosting Mehryar Mohri Courant Institute and Google Research ohri@cis.nyu.edu Weak Learning Definition: concept class C is weakly PAC-learnable if there exists a (weak)
More informationA Smoothed Boosting Algorithm Using Probabilistic Output Codes
A Soothed Boosting Algorith Using Probabilistic Output Codes Rong Jin rongjin@cse.su.edu Dept. of Coputer Science and Engineering, Michigan State University, MI 48824, USA Jian Zhang jian.zhang@cs.cu.edu
More information1 Proof of learning bounds
COS 511: Theoretical Machine Learning Lecturer: Rob Schapire Lecture #4 Scribe: Akshay Mittal February 13, 2013 1 Proof of learning bounds For intuition of the following theore, suppose there exists a
More informationDEPARTMENT OF ECONOMETRICS AND BUSINESS STATISTICS
ISSN 1440-771X AUSTRALIA DEPARTMENT OF ECONOMETRICS AND BUSINESS STATISTICS An Iproved Method for Bandwidth Selection When Estiating ROC Curves Peter G Hall and Rob J Hyndan Working Paper 11/00 An iproved
More informationBiostatistics Department Technical Report
Biostatistics Departent Technical Report BST006-00 Estiation of Prevalence by Pool Screening With Equal Sized Pools and a egative Binoial Sapling Model Charles R. Katholi, Ph.D. Eeritus Professor Departent
More informationPattern Recognition and Machine Learning. Artificial Neural networks
Pattern Recognition and Machine Learning Jaes L. Crowley ENSIMAG 3 - MMIS Fall Seester 2016 Lessons 7 14 Dec 2016 Outline Artificial Neural networks Notation...2 1. Introduction...3... 3 The Artificial
More informationModel Fitting. CURM Background Material, Fall 2014 Dr. Doreen De Leon
Model Fitting CURM Background Material, Fall 014 Dr. Doreen De Leon 1 Introduction Given a set of data points, we often want to fit a selected odel or type to the data (e.g., we suspect an exponential
More informationPattern Recognition and Machine Learning. Artificial Neural networks
Pattern Recognition and Machine Learning Jaes L. Crowley ENSIMAG 3 - MMIS Fall Seester 2017 Lessons 7 20 Dec 2017 Outline Artificial Neural networks Notation...2 Introduction...3 Key Equations... 3 Artificial
More informatione-companion ONLY AVAILABLE IN ELECTRONIC FORM
OPERATIONS RESEARCH doi 10.1287/opre.1070.0427ec pp. ec1 ec5 e-copanion ONLY AVAILABLE IN ELECTRONIC FORM infors 07 INFORMS Electronic Copanion A Learning Approach for Interactive Marketing to a Custoer
More informationE0 370 Statistical Learning Theory Lecture 6 (Aug 30, 2011) Margin Analysis
E0 370 tatistical Learning Theory Lecture 6 (Aug 30, 20) Margin Analysis Lecturer: hivani Agarwal cribe: Narasihan R Introduction In the last few lectures we have seen how to obtain high confidence bounds
More informationKernel Methods and Support Vector Machines
Intelligent Systes: Reasoning and Recognition Jaes L. Crowley ENSIAG 2 / osig 1 Second Seester 2012/2013 Lesson 20 2 ay 2013 Kernel ethods and Support Vector achines Contents Kernel Functions...2 Quadratic
More informationSupport Vector Machines. Machine Learning Series Jerry Jeychandra Blohm Lab
Support Vector Machines Machine Learning Series Jerry Jeychandra Bloh Lab Outline Main goal: To understand how support vector achines (SVMs) perfor optial classification for labelled data sets, also a
More informationŞtefan ŞTEFĂNESCU * is the minimum global value for the function h (x)
7Applying Nelder Mead s Optiization Algorith APPLYING NELDER MEAD S OPTIMIZATION ALGORITHM FOR MULTIPLE GLOBAL MINIMA Abstract Ştefan ŞTEFĂNESCU * The iterative deterinistic optiization ethod could not
More informationSupport Vector Machine Classification of Uncertain and Imbalanced data using Robust Optimization
Recent Researches in Coputer Science Support Vector Machine Classification of Uncertain and Ibalanced data using Robust Optiization RAGHAV PAT, THEODORE B. TRAFALIS, KASH BARKER School of Industrial Engineering
More informationNon-Parametric Non-Line-of-Sight Identification 1
Non-Paraetric Non-Line-of-Sight Identification Sinan Gezici, Hisashi Kobayashi and H. Vincent Poor Departent of Electrical Engineering School of Engineering and Applied Science Princeton University, Princeton,
More informationStochastic Subgradient Methods
Stochastic Subgradient Methods Lingjie Weng Yutian Chen Bren School of Inforation and Coputer Science University of California, Irvine {wengl, yutianc}@ics.uci.edu Abstract Stochastic subgradient ethods
More informationThe Methods of Solution for Constrained Nonlinear Programming
Research Inventy: International Journal Of Engineering And Science Vol.4, Issue 3(March 2014), PP 01-06 Issn (e): 2278-4721, Issn (p):2319-6483, www.researchinventy.co The Methods of Solution for Constrained
More informationSharp Time Data Tradeoffs for Linear Inverse Problems
Sharp Tie Data Tradeoffs for Linear Inverse Probles Saet Oyak Benjain Recht Mahdi Soltanolkotabi January 016 Abstract In this paper we characterize sharp tie-data tradeoffs for optiization probles used
More informationExtension of CSRSM for the Parametric Study of the Face Stability of Pressurized Tunnels
Extension of CSRSM for the Paraetric Study of the Face Stability of Pressurized Tunnels Guilhe Mollon 1, Daniel Dias 2, and Abdul-Haid Soubra 3, M.ASCE 1 LGCIE, INSA Lyon, Université de Lyon, Doaine scientifique
More informationA Simplified Analytical Approach for Efficiency Evaluation of the Weaving Machines with Automatic Filling Repair
Proceedings of the 6th SEAS International Conference on Siulation, Modelling and Optiization, Lisbon, Portugal, Septeber -4, 006 0 A Siplified Analytical Approach for Efficiency Evaluation of the eaving
More informationESTIMATING AND FORMING CONFIDENCE INTERVALS FOR EXTREMA OF RANDOM POLYNOMIALS. A Thesis. Presented to. The Faculty of the Department of Mathematics
ESTIMATING AND FORMING CONFIDENCE INTERVALS FOR EXTREMA OF RANDOM POLYNOMIALS A Thesis Presented to The Faculty of the Departent of Matheatics San Jose State University In Partial Fulfillent of the Requireents
More informationKeywords: Estimator, Bias, Mean-squared error, normality, generalized Pareto distribution
Testing approxiate norality of an estiator using the estiated MSE and bias with an application to the shape paraeter of the generalized Pareto distribution J. Martin van Zyl Abstract In this work the norality
More information1 Bounding the Margin
COS 511: Theoretical Machine Learning Lecturer: Rob Schapire Lecture #12 Scribe: Jian Min Si March 14, 2013 1 Bounding the Margin We are continuing the proof of a bound on the generalization error of AdaBoost
More informationResearch in Area of Longevity of Sylphon Scraies
IOP Conference Series: Earth and Environental Science PAPER OPEN ACCESS Research in Area of Longevity of Sylphon Scraies To cite this article: Natalia Y Golovina and Svetlana Y Krivosheeva 2018 IOP Conf.
More informationUNIVERSITY OF TRENTO ON THE USE OF SVM FOR ELECTROMAGNETIC SUBSURFACE SENSING. A. Boni, M. Conci, A. Massa, and S. Piffer.
UIVRSITY OF TRTO DIPARTITO DI IGGRIA SCIZA DLL IFORAZIO 3823 Povo Trento (Italy) Via Soarive 4 http://www.disi.unitn.it O TH US OF SV FOR LCTROAGTIC SUBSURFAC SSIG A. Boni. Conci A. assa and S. Piffer
More informationApproximation in Stochastic Scheduling: The Power of LP-Based Priority Policies
Approxiation in Stochastic Scheduling: The Power of -Based Priority Policies Rolf Möhring, Andreas Schulz, Marc Uetz Setting (A P p stoch, r E( w and (B P p stoch E( w We will assue that the processing
More informationSoft Computing Techniques Help Assign Weights to Different Factors in Vulnerability Analysis
Soft Coputing Techniques Help Assign Weights to Different Factors in Vulnerability Analysis Beverly Rivera 1,2, Irbis Gallegos 1, and Vladik Kreinovich 2 1 Regional Cyber and Energy Security Center RCES
More informationInspection; structural health monitoring; reliability; Bayesian analysis; updating; decision analysis; value of information
Cite as: Straub D. (2014). Value of inforation analysis with structural reliability ethods. Structural Safety, 49: 75-86. Value of Inforation Analysis with Structural Reliability Methods Daniel Straub
More informationC na (1) a=l. c = CO + Clm + CZ TWO-STAGE SAMPLE DESIGN WITH SMALL CLUSTERS. 1. Introduction
TWO-STGE SMPLE DESIGN WITH SMLL CLUSTERS Robert G. Clark and David G. Steel School of Matheatics and pplied Statistics, University of Wollongong, NSW 5 ustralia. (robert.clark@abs.gov.au) Key Words: saple
More informationAN EFFICIENT CLASS OF CHAIN ESTIMATORS OF POPULATION VARIANCE UNDER SUB-SAMPLING SCHEME
J. Japan Statist. Soc. Vol. 35 No. 005 73 86 AN EFFICIENT CLASS OF CHAIN ESTIMATORS OF POPULATION VARIANCE UNDER SUB-SAMPLING SCHEME H. S. Jhajj*, M. K. Shara* and Lovleen Kuar Grover** For estiating the
More informationEstimation of the Mean of the Exponential Distribution Using Maximum Ranked Set Sampling with Unequal Samples
Open Journal of Statistics, 4, 4, 64-649 Published Online Septeber 4 in SciRes http//wwwscirporg/ournal/os http//ddoiorg/436/os4486 Estiation of the Mean of the Eponential Distribution Using Maiu Ranked
More informationCOS 424: Interacting with Data. Written Exercises
COS 424: Interacting with Data Hoework #4 Spring 2007 Regression Due: Wednesday, April 18 Written Exercises See the course website for iportant inforation about collaboration and late policies, as well
More informationBayes Decision Rule and Naïve Bayes Classifier
Bayes Decision Rule and Naïve Bayes Classifier Le Song Machine Learning I CSE 6740, Fall 2013 Gaussian Mixture odel A density odel p(x) ay be ulti-odal: odel it as a ixture of uni-odal distributions (e.g.
More informationPAC-Bayes Analysis Of Maximum Entropy Learning
PAC-Bayes Analysis Of Maxiu Entropy Learning John Shawe-Taylor and David R. Hardoon Centre for Coputational Statistics and Machine Learning Departent of Coputer Science University College London, UK, WC1E
More informationIntelligent Systems: Reasoning and Recognition. Artificial Neural Networks
Intelligent Systes: Reasoning and Recognition Jaes L. Crowley MOSIG M1 Winter Seester 2018 Lesson 7 1 March 2018 Outline Artificial Neural Networks Notation...2 Introduction...3 Key Equations... 3 Artificial
More informationThis model assumes that the probability of a gap has size i is proportional to 1/i. i.e., i log m e. j=1. E[gap size] = i P r(i) = N f t.
CS 493: Algoriths for Massive Data Sets Feb 2, 2002 Local Models, Bloo Filter Scribe: Qin Lv Local Models In global odels, every inverted file entry is copressed with the sae odel. This work wells when
More informationRandomized Recovery for Boolean Compressed Sensing
Randoized Recovery for Boolean Copressed Sensing Mitra Fatei and Martin Vetterli Laboratory of Audiovisual Counication École Polytechnique Fédéral de Lausanne (EPFL) Eail: {itra.fatei, artin.vetterli}@epfl.ch
More informationN-Point. DFTs of Two Length-N Real Sequences
Coputation of the DFT of In ost practical applications, sequences of interest are real In such cases, the syetry properties of the DFT given in Table 5. can be exploited to ake the DFT coputations ore
More informationResearch Article A New Biparametric Family of Two-Point Optimal Fourth-Order Multiple-Root Finders
Applied Matheatics, Article ID 737305, 7 pages http://dx.doi.org/10.1155/2014/737305 Research Article A New Biparaetric Faily of Two-Point Optial Fourth-Order Multiple-Root Finders Young Ik Ki and Young
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Notes for EE7C (Spring 018: Convex Optiization and Approxiation Instructor: Moritz Hardt Eail: hardt+ee7c@berkeley.edu Graduate Instructor: Max Sichowitz Eail: sichow+ee7c@berkeley.edu October 15,
More informationAn Improved Particle Filter with Applications in Ballistic Target Tracking
Sensors & ransducers Vol. 72 Issue 6 June 204 pp. 96-20 Sensors & ransducers 204 by IFSA Publishing S. L. http://www.sensorsportal.co An Iproved Particle Filter with Applications in Ballistic arget racing
More informationBoosting kernel density estimates: A bias reduction technique?
Biometrika (2004), 91, 1, pp. 226 233 2004 Biometrika Trust Printed in Great Britain Boosting kernel density estimates: A bias reduction technique? BY MARCO DI MARZIO Dipartimento di Metodi Quantitativi
More informationFigure 1: Equivalent electric (RC) circuit of a neurons membrane
Exercise: Leaky integrate and fire odel of neural spike generation This exercise investigates a siplified odel of how neurons spike in response to current inputs, one of the ost fundaental properties of
More informationIAENG International Journal of Computer Science, 42:2, IJCS_42_2_06. Approximation Capabilities of Interpretable Fuzzy Inference Systems
IAENG International Journal of Coputer Science, 4:, IJCS_4 6 Approxiation Capabilities of Interpretable Fuzzy Inference Systes Hirofui Miyajia, Noritaka Shigei, and Hiroi Miyajia 3 Abstract Many studies
More informationTesting equality of variances for multiple univariate normal populations
University of Wollongong Research Online Centre for Statistical & Survey Methodology Working Paper Series Faculty of Engineering and Inforation Sciences 0 esting equality of variances for ultiple univariate
More informationSupport Vector Machines MIT Course Notes Cynthia Rudin
Support Vector Machines MIT 5.097 Course Notes Cynthia Rudin Credit: Ng, Hastie, Tibshirani, Friedan Thanks: Şeyda Ertekin Let s start with soe intuition about argins. The argin of an exaple x i = distance
More informationSupport Vector Machines. Maximizing the Margin
Support Vector Machines Support vector achines (SVMs) learn a hypothesis: h(x) = b + Σ i= y i α i k(x, x i ) (x, y ),..., (x, y ) are the training exs., y i {, } b is the bias weight. α,..., α are the
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Notes for EE227C (Spring 2018): Convex Optiization and Approxiation Instructor: Moritz Hardt Eail: hardt+ee227c@berkeley.edu Graduate Instructor: Max Sichowitz Eail: sichow+ee227c@berkeley.edu October
More informationRobustness and Regularization of Support Vector Machines
Robustness and Regularization of Support Vector Machines Huan Xu ECE, McGill University Montreal, QC, Canada xuhuan@ci.cgill.ca Constantine Caraanis ECE, The University of Texas at Austin Austin, TX, USA
More informationProbabilistic Machine Learning
Probabilistic Machine Learning by Prof. Seungchul Lee isystes Design Lab http://isystes.unist.ac.kr/ UNIST Table of Contents I.. Probabilistic Linear Regression I... Maxiu Likelihood Solution II... Maxiu-a-Posteriori
More informationProbability Distributions
Probability Distributions In Chapter, we ephasized the central role played by probability theory in the solution of pattern recognition probles. We turn now to an exploration of soe particular exaples
More informationA note on the multiplication of sparse matrices
Cent. Eur. J. Cop. Sci. 41) 2014 1-11 DOI: 10.2478/s13537-014-0201-x Central European Journal of Coputer Science A note on the ultiplication of sparse atrices Research Article Keivan Borna 12, Sohrab Aboozarkhani
More informationTracking using CONDENSATION: Conditional Density Propagation
Tracking using CONDENSATION: Conditional Density Propagation Goal Model-based visual tracking in dense clutter at near video frae rates M. Isard and A. Blake, CONDENSATION Conditional density propagation
More informationRecovering Data from Underdetermined Quadratic Measurements (CS 229a Project: Final Writeup)
Recovering Data fro Underdeterined Quadratic Measureents (CS 229a Project: Final Writeup) Mahdi Soltanolkotabi Deceber 16, 2011 1 Introduction Data that arises fro engineering applications often contains
More informationOn a few Iterative Methods for Solving Nonlinear Equations
On a few Iterative Methods for Solving Nonlinear Equations Gyurhan Nedzhibov Laboratory of Matheatical Modelling, Shuen University, Shuen 971, Bulgaria e-ail: gyurhan@shu-bg.net Abstract In this study
More informationBayesian Approach for Fatigue Life Prediction from Field Inspection
Bayesian Approach for Fatigue Life Prediction fro Field Inspection Dawn An and Jooho Choi School of Aerospace & Mechanical Engineering, Korea Aerospace University, Goyang, Seoul, Korea Srira Pattabhiraan
More informationAn improved self-adaptive harmony search algorithm for joint replenishment problems
An iproved self-adaptive harony search algorith for joint replenishent probles Lin Wang School of Manageent, Huazhong University of Science & Technology zhoulearner@gail.co Xiaojian Zhou School of Manageent,
More informationEfficient Filter Banks And Interpolators
Efficient Filter Banks And Interpolators A. G. DEMPSTER AND N. P. MURPHY Departent of Electronic Systes University of Westinster 115 New Cavendish St, London W1M 8JS United Kingdo Abstract: - Graphical
More informationQuantum Chemistry Exam 2 Take-home Solutions
Cheistry 60 Fall 07 Dr Jean M Standard Nae KEY Quantu Cheistry Exa Take-hoe Solutions 5) (0 points) In this proble, the nonlinear variation ethod will be used to deterine an approxiate solution for the
More informationOn Constant Power Water-filling
On Constant Power Water-filling Wei Yu and John M. Cioffi Electrical Engineering Departent Stanford University, Stanford, CA94305, U.S.A. eails: {weiyu,cioffi}@stanford.edu Abstract This paper derives
More informationSoft-margin SVM can address linearly separable problems with outliers
Non-linear Support Vector Machines Non-linearly separable probles Hard-argin SVM can address linearly separable probles Soft-argin SVM can address linearly separable probles with outliers Non-linearly
More informationUse of PSO in Parameter Estimation of Robot Dynamics; Part One: No Need for Parameterization
Use of PSO in Paraeter Estiation of Robot Dynaics; Part One: No Need for Paraeterization Hossein Jahandideh, Mehrzad Navar Abstract Offline procedures for estiating paraeters of robot dynaics are practically
More informationNeural Networks Trained with the EEM Algorithm: Tuning the Smoothing Parameter
eural etworks Trained wit te EEM Algorit: Tuning te Sooting Paraeter JORGE M. SATOS,2, JOAQUIM MARQUES DE SÁ AD LUÍS A. ALEXADRE 3 Intituto de Engenaria Bioédica, Porto, Portugal 2 Instituto Superior de
More informationLecture 12: Ensemble Methods. Introduction. Weighted Majority. Mixture of Experts/Committee. Σ k α k =1. Isabelle Guyon
Lecture 2: Enseble Methods Isabelle Guyon guyoni@inf.ethz.ch Introduction Book Chapter 7 Weighted Majority Mixture of Experts/Coittee Assue K experts f, f 2, f K (base learners) x f (x) Each expert akes
More informationKernel-Based Nonparametric Anomaly Detection
Kernel-Based Nonparaetric Anoaly Detection Shaofeng Zou Dept of EECS Syracuse University Eail: szou@syr.edu Yingbin Liang Dept of EECS Syracuse University Eail: yliang6@syr.edu H. Vincent Poor Dept of
More informationComputational and Statistical Learning Theory
Coputational and Statistical Learning Theory TTIC 31120 Prof. Nati Srebro Lecture 2: PAC Learning and VC Theory I Fro Adversarial Online to Statistical Three reasons to ove fro worst-case deterinistic
More informationVariational Adaptive-Newton Method
Variational Adaptive-Newton Method Mohaad Etiyaz Khan Wu Lin Voot Tangkaratt Zuozhu Liu Didrik Nielsen AIP, RIKEN, Tokyo Abstract We present a black-box learning ethod called the Variational Adaptive-Newton
More informationFast Montgomery-like Square Root Computation over GF(2 m ) for All Trinomials
Fast Montgoery-like Square Root Coputation over GF( ) for All Trinoials Yin Li a, Yu Zhang a, a Departent of Coputer Science and Technology, Xinyang Noral University, Henan, P.R.China Abstract This letter
More informationAnalyzing Simulation Results
Analyzing Siulation Results Dr. John Mellor-Cruey Departent of Coputer Science Rice University johnc@cs.rice.edu COMP 528 Lecture 20 31 March 2005 Topics for Today Model verification Model validation Transient
More informationA Theoretical Analysis of a Warm Start Technique
A Theoretical Analysis of a War Start Technique Martin A. Zinkevich Yahoo! Labs 701 First Avenue Sunnyvale, CA Abstract Batch gradient descent looks at every data point for every step, which is wasteful
More informationShannon Sampling II. Connections to Learning Theory
Shannon Sapling II Connections to Learning heory Steve Sale oyota echnological Institute at Chicago 147 East 60th Street, Chicago, IL 60637, USA E-ail: sale@athberkeleyedu Ding-Xuan Zhou Departent of Matheatics,
More informationTwo New Unbiased Point Estimates Of A Population Variance
Journal of Modern Applied Statistical Methods Volue 5 Issue Article 7 5--006 Two New Unbiased Point Estiates Of A Population Variance Matthew E. Ela The University of Alabaa, ela@baa.ua.edu Follow this
More informationLogLog-Beta and More: A New Algorithm for Cardinality Estimation Based on LogLog Counting
LogLog-Beta and More: A New Algorith for Cardinality Estiation Based on LogLog Counting Jason Qin, Denys Ki, Yuei Tung The AOLP Core Data Service, AOL, 22000 AOL Way Dulles, VA 20163 E-ail: jasonqin@teaaolco
More informationBernoulli Wavelet Based Numerical Method for Solving Fredholm Integral Equations of the Second Kind
ISSN 746-7659, England, UK Journal of Inforation and Coputing Science Vol., No., 6, pp.-9 Bernoulli Wavelet Based Nuerical Method for Solving Fredhol Integral Equations of the Second Kind S. C. Shiralashetti*,
More informationSPECTRUM sensing is a core concept of cognitive radio
World Acadey of Science, Engineering and Technology International Journal of Electronics and Counication Engineering Vol:6, o:2, 202 Efficient Detection Using Sequential Probability Ratio Test in Mobile
More informationLower Bounds for Quantized Matrix Completion
Lower Bounds for Quantized Matrix Copletion Mary Wootters and Yaniv Plan Departent of Matheatics University of Michigan Ann Arbor, MI Eail: wootters, yplan}@uich.edu Mark A. Davenport School of Elec. &
More informationNonmonotonic Networks. a. IRST, I Povo (Trento) Italy, b. Univ. of Trento, Physics Dept., I Povo (Trento) Italy
Storage Capacity and Dynaics of Nononotonic Networks Bruno Crespi a and Ignazio Lazzizzera b a. IRST, I-38050 Povo (Trento) Italy, b. Univ. of Trento, Physics Dept., I-38050 Povo (Trento) Italy INFN Gruppo
More informationUfuk Demirci* and Feza Kerestecioglu**
1 INDIRECT ADAPTIVE CONTROL OF MISSILES Ufuk Deirci* and Feza Kerestecioglu** *Turkish Navy Guided Missile Test Station, Beykoz, Istanbul, TURKEY **Departent of Electrical and Electronics Engineering,
More informationP016 Toward Gauss-Newton and Exact Newton Optimization for Full Waveform Inversion
P016 Toward Gauss-Newton and Exact Newton Optiization for Full Wavefor Inversion L. Métivier* ISTerre, R. Brossier ISTerre, J. Virieux ISTerre & S. Operto Géoazur SUMMARY Full Wavefor Inversion FWI applications
More informationMeta-Analytic Interval Estimation for Bivariate Correlations
Psychological Methods 2008, Vol. 13, No. 3, 173 181 Copyright 2008 by the Aerican Psychological Association 1082-989X/08/$12.00 DOI: 10.1037/a0012868 Meta-Analytic Interval Estiation for Bivariate Correlations
More informationRademacher Complexity Margin Bounds for Learning with a Large Number of Classes
Radeacher Coplexity Margin Bounds for Learning with a Large Nuber of Classes Vitaly Kuznetsov Courant Institute of Matheatical Sciences, 25 Mercer street, New York, NY, 002 Mehryar Mohri Courant Institute
More informationNyström Method vs Random Fourier Features: A Theoretical and Empirical Comparison
yströ Method vs : A Theoretical and Epirical Coparison Tianbao Yang, Yu-Feng Li, Mehrdad Mahdavi, Rong Jin, Zhi-Hua Zhou Machine Learning Lab, GE Global Research, San Raon, CA 94583 Michigan State University,
More informationInteractive Markov Models of Evolutionary Algorithms
Cleveland State University EngagedScholarship@CSU Electrical Engineering & Coputer Science Faculty Publications Electrical Engineering & Coputer Science Departent 2015 Interactive Markov Models of Evolutionary
More informationAccuracy at the Top. Abstract
Accuracy at the Top Stephen Boyd Stanford University Packard 264 Stanford, CA 94305 boyd@stanford.edu Mehryar Mohri Courant Institute and Google 25 Mercer Street New York, NY 002 ohri@cis.nyu.edu Corinna
More informationTEST OF HOMOGENEITY OF PARALLEL SAMPLES FROM LOGNORMAL POPULATIONS WITH UNEQUAL VARIANCES
TEST OF HOMOGENEITY OF PARALLEL SAMPLES FROM LOGNORMAL POPULATIONS WITH UNEQUAL VARIANCES S. E. Ahed, R. J. Tokins and A. I. Volodin Departent of Matheatics and Statistics University of Regina Regina,
More informationDepartment of Electronic and Optical Engineering, Ordnance Engineering College, Shijiazhuang, , China
6th International Conference on Machinery, Materials, Environent, Biotechnology and Coputer (MMEBC 06) Solving Multi-Sensor Multi-Target Assignent Proble Based on Copositive Cobat Efficiency and QPSO Algorith
More informationBootstrapping Dependent Data
Bootstrapping Dependent Data One of the key issues confronting bootstrap resapling approxiations is how to deal with dependent data. Consider a sequence fx t g n t= of dependent rando variables. Clearly
More informationComparison of Stability of Selected Numerical Methods for Solving Stiff Semi- Linear Differential Equations
International Journal of Applied Science and Technology Vol. 7, No. 3, Septeber 217 Coparison of Stability of Selected Nuerical Methods for Solving Stiff Sei- Linear Differential Equations Kwaku Darkwah
More informationTABLE FOR UPPER PERCENTAGE POINTS OF THE LARGEST ROOT OF A DETERMINANTAL EQUATION WITH FIVE ROOTS. By William W. Chen
TABLE FOR UPPER PERCENTAGE POINTS OF THE LARGEST ROOT OF A DETERMINANTAL EQUATION WITH FIVE ROOTS By Willia W. Chen The distribution of the non-null characteristic roots of a atri derived fro saple observations
More informationComplexity reduction in low-delay Farrowstructure-based. filters utilizing linear-phase subfilters
Coplexity reduction in low-delay Farrowstructure-based variable fractional delay FIR filters utilizing linear-phase subfilters Air Eghbali and Håkan Johansson Linköping University Post Print N.B.: When
More informationVI. Backpropagation Neural Networks (BPNN)
VI. Backpropagation Neural Networks (BPNN) Review of Adaline Newton s ethod Backpropagation algorith definition derivative coputation weight/bias coputation function approxiation exaple network generalization
More informationComparing Probabilistic Forecasting Systems with the Brier Score
1076 W E A T H E R A N D F O R E C A S T I N G VOLUME 22 Coparing Probabilistic Forecasting Systes with the Brier Score CHRISTOPHER A. T. FERRO School of Engineering, Coputing and Matheatics, University
More informationA method to determine relative stroke detection efficiencies from multiplicity distributions
A ethod to deterine relative stroke detection eiciencies ro ultiplicity distributions Schulz W. and Cuins K. 2. Austrian Lightning Detection and Inoration Syste (ALDIS), Kahlenberger Str.2A, 90 Vienna,
More information