X/94 $ IEEE 1894
|
|
- Kerrie Hampton
- 6 years ago
- Views:
Transcription
1 I Implementing Radial Basis Functions Using Bump-Resistor Networks John G. Harris University of Florida EE Dept., 436 CSE Bldg 42 Gainesville, FL harris@j upit er.ee.ufl.edu Abstract- Radial Basis Function (RBF) networks provide a powerful learning architecture for neural networks [6]. We have implemented a RBF network in analog VLSI using the concept of bump-resistors. A bump-resistor is a nonlinear resistor whose conductance is a Gaussian-like function of the difference of two other voltages. The width of the Gaussian basis functions may be continuously varied so that the aggregate interpolating function varies from a nearest-neighbor lookup, piecewise constant function to a globally smooth function. The bump-resis tor methodology extends to arbitrary dimensions while still preserving the radiality of the basis functions. The feedforward network architecture needs no additional circuitry other than voltage sources and the 1D bump-resistors. A nine-transistor variation of the Delbruck bump circuit is used to compute the Gaussian-like basis functions [2]. Below threshold the current output fits a Gaussian extremely well, see Figure 1. Figure 3 shows that the shape of the function deviates from the Gaussian shape above threshold. The width of the bump can to be varied by almost an order of magnitude (see Figure 4). The Delbruck bump circuit is shown in Figure A follower aggregation network shown in Figure 5 computes an average of the inputs voltages ci weighted by conductance values gi [4]: The bump current is used to control the conductance of the resistors in Figure 5 such that gi = G(x - ti) where ti is the voltage representing the center location of each bump-resistor. The circuit now computes: This normalized RBF or partition of unity form has been used by Moody and Darken in learning simula- BT Gaussian C iit 1 4 : : : : : : : : : a OS Y-TO Figure 1: Measured current from the Delbruck bump circuit (circles) and Gaussian fit (straightline). tions [5]. Some researchers have claimed that RBF networks show improved performance using this formulation [SI. Anderson, Platt and Kirk previously demonstrated the use of follower aggregation in an analog RBF chip [l]. The standard RBF form (without normalization) can be computed by holding the output to virtual ground and measuring the current. A one-dimensional eight-node circuit has been successfully fabricated and tested. Results from fitting 4 points using two different values of U are shown in Figure 6. The left plot shows the small U response which approximates a nearest neighbor lookup and the right plot shows the much smoother result for a large U. There are several strategies for extending this network to multiple dimensions. One is simply to construct a multi-dimensional bump function to use as the conductance function G(x) in Figure 5. For example, Kirk has designed a suitable multidimensional bump function by multiplying together several 1D Delbruck bump functions [3]. An alternate strategy used by Anderson, Platt and Kirk [l] X/94 $ IEEE 1894
2 : : : : : : : : : I ) U X M : : : : : : : ; : I b U X M Figure 6: Measured chip curva from fitting four data points Figure 7: Two-dimensional network 1895
3 o.m 4 I Od la W O Figure 4: Plot of the measured standard deviation of the bump function vs. voltage control knob. Figure 5: Follower aggregation network 1896
4 Vb = X-T (V) Figure 3: Normalized current from the bump circuit (circles) fit to Gaussians with identical variance. without worry of any problems with local minima. Learning the center locations (ti) and the widths of the Gaussians (ui) is a more difficult problem. REFERENCES [l] J. Anderson, J. C. Platt, and D. Kirk. An analog VLSI chip for radial basis functions. In J. Hanson, J. Cowan, and C. L. Giles, editors, Neural Information Processing Systems, pages Morgan-Kaufman, Palo Alto, [2] T. Delbruck. Bump circuits for computing similarity and disimilarity of analog voltages. In International Joint Conference on Neural Networks, Seattle, WA., July [3] D. Kirk, D. Kerns, K. Fleischer, and A. Barr. Analog VLSI implementation of multi-dimensional gradient descent. In J. Hanson, J. Cowan, and C. L. Giles, editors, Neural Information Processing Systems, pages Morgan-Kaufman, Palo Alto, [4] C. Mead. Analog VLSI and Neural Systems. Addison-Wesley, [5] J. Moody and C. Darken. Fast learning in net- works of locally-tuned processing units. Neural Computation, 1(2): , [6] T. Poggio and F. Girosi. Regularization algorithms for learning that are equivalent to multilayer networks. Science, 247: , [7] D. Scott. Multivariate Density Estimation. Wiley, [8] H.W. Werntges. Partitions of unity improve neural function approximators. In Proc. IEEE Intl. Conf. on Neural Networks, pages , San Francisco, CA,, Feb vol 2,. 1897
5 Figure 2: The Delbruck bump circuit. The voltage VB controls the width of the bump. is to use current summation to sum the squares of the voltage differences. A circuit must then be used to exponentiate the current and convert the result to a voltage. Rather than build more complex circuitry, we choose to use the physics of resistors in series to combine the dimensions. Figure 7 shows the the network in two dimensions (extension to further dimensions is straightforward). In two dimensions, each branch has two resistors with conductances of G(x) and G(y). The effective conductance H(z,y) is: (3) Typically we choose G(x) to be a Gaussian function since the Gaussian is the only function that can create a radial function by multiplying 1D functions. However, in this case the Gaussian is the wrong choice since H(x,y) will be nonradial. Instead, we choose G with the following form: Now the series combination is: 1 G(x) = (4) 1 + (x/c)2 which is radial. In many applications, radial functions are not strictly necessary. In fact, the density estimation literature reveals that some nonradial functions have been shown to be optimal under certain criteria [7]. We are also studying resistors that are controlled by sigmoidal functions. A feedforward RBF network is sufficient for many applications. A microprocessor can be used to implement a learning network and download the weights to the chip. On-chip learning will drastically speed up the learning process. Learning the coefficients ci is a straightforward process that can be performed with gradient descent 1898
Radial Basis Function Networks. Ravi Kaushik Project 1 CSC Neural Networks and Pattern Recognition
Radial Basis Function Networks Ravi Kaushik Project 1 CSC 84010 Neural Networks and Pattern Recognition History Radial Basis Function (RBF) emerged in late 1980 s as a variant of artificial neural network.
More informationNotes on Regularization and Robust Estimation Psych 267/CS 348D/EE 365 Prof. David J. Heeger September 15, 1998
Notes on Regularization and Robust Estimation Psych 67/CS 348D/EE 365 Prof. David J. Heeger September 5, 998 Regularization. Regularization is a class of techniques that have been widely used to solve
More informationJ. Lazzaro, S. Ryckebusch, M.A. Mahowald, and C. A. Mead California Institute of Technology Pasadena, CA 91125
WINNER-TAKE-ALL NETWORKS OF O(N) COMPLEXITY J. Lazzaro, S. Ryckebusch, M.A. Mahowald, and C. A. Mead California Institute of Technology Pasadena, CA 91125 ABSTRACT We have designed, fabricated, and tested
More informationCHAPTER IX Radial Basis Function Networks
Ugur HAICI - METU EEE - ANKARA 2/2/2005 CHAPTER IX Radial Basis Function Networks Introduction Radial basis function (RBF) networks are feed-forward networks trained using a supervised training algorithm.
More informationIn the Name of God. Lectures 15&16: Radial Basis Function Networks
1 In the Name of God Lectures 15&16: Radial Basis Function Networks Some Historical Notes Learning is equivalent to finding a surface in a multidimensional space that provides a best fit to the training
More informationA Parallel Gradient Descent Method for Learning in Analog VLSI Neural Networks
A Parallel Gradient Descent Method for Learning in Analog VLSI Neural Networks J. Alspector R. Meir'" B. Yuhas A. Jayakumar D. Lippet Bellcore Morristown, NJ 07962-1910 Abstract Typical methods for gradient
More informationPlan of Class 4. Radial Basis Functions with moving centers. Projection Pursuit Regression and ridge. Principal Component Analysis: basic ideas
Plan of Class 4 Radial Basis Functions with moving centers Multilayer Perceptrons Projection Pursuit Regression and ridge functions approximation Principal Component Analysis: basic ideas Radial Basis
More informationEffects of Moving the Centers in an RBF Network
IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 13, NO. 6, NOVEMBER 2002 1299 Effects of Moving the Centers in an RBF Network Chitra Panchapakesan, Marimuthu Palaniswami, Senior Member, IEEE, Daniel Ralph,
More informationApplication of a radial basis function neural network for diagnosis of diabetes mellitus
Application of a radial basis function neural network for diagnosis of diabetes mellitus P. Venkatesan* and S. Anitha Tuberculosis Research Centre, ICMR, Chennai 600 031, India In this article an attempt
More informationIN neural-network training, the most well-known online
IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 10, NO. 1, JANUARY 1999 161 On the Kalman Filtering Method in Neural-Network Training and Pruning John Sum, Chi-sing Leung, Gilbert H. Young, and Wing-kay Kan
More informationE techniques for identification of linear time-invariant
568 IEEE TRANSACTIONS ON NEURAL NETWORKS. VOL. 2. NO. 6. NOVEMBER 1991 Brief Papers A General Regression Neural Network Donald F. Specht Abstract-This paper describes a memory-based network that provides
More informationARTIFICIAL INTELLIGENCE LABORATORY. and CENTER FOR BIOLOGICAL INFORMATION PROCESSING. A.I. Memo No August Federico Girosi.
MASSACHUSETTS INSTITUTE OF TECHNOLOGY ARTIFICIAL INTELLIGENCE LABORATORY and CENTER FOR BIOLOGICAL INFORMATION PROCESSING WHITAKER COLLEGE A.I. Memo No. 1287 August 1991 C.B.I.P. Paper No. 66 Models of
More informationA Robust PCA by LMSER Learning with Iterative Error. Bai-ling Zhang Irwin King Lei Xu.
A Robust PCA by LMSER Learning with Iterative Error Reinforcement y Bai-ling Zhang Irwin King Lei Xu blzhang@cs.cuhk.hk king@cs.cuhk.hk lxu@cs.cuhk.hk Department of Computer Science The Chinese University
More informationIntroduction to Neural Networks: Structure and Training
Introduction to Neural Networks: Structure and Training Professor Q.J. Zhang Department of Electronics Carleton University, Ottawa, Canada www.doe.carleton.ca/~qjz, qjz@doe.carleton.ca A Quick Illustration
More informationStable Adaptive Momentum for Rapid Online Learning in Nonlinear Systems
Stable Adaptive Momentum for Rapid Online Learning in Nonlinear Systems Thore Graepel and Nicol N. Schraudolph Institute of Computational Science ETH Zürich, Switzerland {graepel,schraudo}@inf.ethz.ch
More informationThe Power of Approximating: a Comparison of Activation Functions
The Power of Approximating: a Comparison of Activation Functions Bhaskar DasGupta Department of Computer Science University of Minnesota Minneapolis, MN 55455-0159 email: dasgupta~cs.umn.edu Georg Schnitger
More informationChapter 8: Generalization and Function Approximation
Chapter 8: Generalization and Function Approximation Objectives of this chapter: Look at how experience with a limited part of the state set be used to produce good behavior over a much larger part. Overview
More informationFrequency Selective Surface Design Based on Iterative Inversion of Neural Networks
J.N. Hwang, J.J. Choi, S. Oh, R.J. Marks II, "Query learning based on boundary search and gradient computation of trained multilayer perceptrons", Proceedings of the International Joint Conference on Neural
More informationIntroduction to Machine Learning Spring 2018 Note Neural Networks
CS 189 Introduction to Machine Learning Spring 2018 Note 14 1 Neural Networks Neural networks are a class of compositional function approximators. They come in a variety of shapes and sizes. In this class,
More informationCSE 190: Reinforcement Learning: An Introduction. Chapter 8: Generalization and Function Approximation. Pop Quiz: What Function Are We Approximating?
CSE 190: Reinforcement Learning: An Introduction Chapter 8: Generalization and Function Approximation Objectives of this chapter: Look at how experience with a limited part of the state set be used to
More informationGeneralization and Function Approximation
Generalization and Function Approximation 0 Generalization and Function Approximation Suggested reading: Chapter 8 in R. S. Sutton, A. G. Barto: Reinforcement Learning: An Introduction MIT Press, 1998.
More informationModule 12. Machine Learning. Version 2 CSE IIT, Kharagpur
Module 12 Machine Learning Lesson 39 Neural Networks - III 12.4.4 Multi-Layer Perceptrons In contrast to perceptrons, multilayer networks can learn not only multiple decision boundaries, but the boundaries
More informationEM-algorithm for Training of State-space Models with Application to Time Series Prediction
EM-algorithm for Training of State-space Models with Application to Time Series Prediction Elia Liitiäinen, Nima Reyhani and Amaury Lendasse Helsinki University of Technology - Neural Networks Research
More informationComputation of Smooth Optical Flow in a Feedback Connected Analog Network
Computation of Smooth Optical Flow in a Feedback Connected Analog Network Alan Stocker Institute of Neuroinformatics University and ETH Zürich Winterthurerstrasse 90 80 Zürich, Switzerland Rodney Douglas
More informationCOMS 4771 Introduction to Machine Learning. Nakul Verma
COMS 4771 Introduction to Machine Learning Nakul Verma Announcements HW1 due next lecture Project details are available decide on the group and topic by Thursday Last time Generative vs. Discriminative
More informationA Logarithmic Neural Network Architecture for Unbounded Non-Linear Function Approximation
1 Introduction A Logarithmic Neural Network Architecture for Unbounded Non-Linear Function Approximation J Wesley Hines Nuclear Engineering Department The University of Tennessee Knoxville, Tennessee,
More informationArtificial Neural Networks (ANN) Xiaogang Su, Ph.D. Department of Mathematical Science University of Texas at El Paso
Artificial Neural Networks (ANN) Xiaogang Su, Ph.D. Department of Mathematical Science University of Texas at El Paso xsu@utep.edu Fall, 2018 Outline Introduction A Brief History ANN Architecture Terminology
More informationNeural networks. Chapter 20, Section 5 1
Neural networks Chapter 20, Section 5 Chapter 20, Section 5 Outline Brains Neural networks Perceptrons Multilayer perceptrons Applications of neural networks Chapter 20, Section 5 2 Brains 0 neurons of
More informationImplementation of Carry Look-Ahead in Domino Logic
Implementation of Carry Look-Ahead in Domino Logic G. Vijayakumar 1 M. Poorani Swasthika 2 S. Valarmathi 3 And A. Vidhyasekar 4 1, 2, 3 Master of Engineering (VLSI design) & 4 Asst.Prof/ Dept.of ECE Akshaya
More informationWhen can Deep Networks avoid the curse of dimensionality and other theoretical puzzles
When can Deep Networks avoid the curse of dimensionality and other theoretical puzzles Tomaso Poggio, MIT, CBMM Astar CBMM s focus is the Science and the Engineering of Intelligence We aim to make progress
More informationCHAPTER 2 NEW RADIAL BASIS NEURAL NETWORKS AND THEIR APPLICATION IN A LARGE-SCALE HANDWRITTEN DIGIT RECOGNITION PROBLEM
CHAPTER 2 NEW RADIAL BASIS NEURAL NETWORKS AND THEIR APPLICATION IN A LARGE-SCALE HANDWRITTEN DIGIT RECOGNITION PROBLEM N.B. Karayiannis Department of Electrical and Computer Engineering University of
More informationOld painting digital color restoration
Old painting digital color restoration Michail Pappas Ioannis Pitas Dept. of Informatics, Aristotle University of Thessaloniki GR-54643 Thessaloniki, Greece Abstract Many old paintings suffer from the
More informationAn Analog-digital CMOS circuit for motion detection based on direction-selective neural networks
An Analog-digital CMOS circuit for motion detection based on direction-selective neural networks Masato Koutani, Tetsuya Asai, and Yoshihito Amemiya Department of Electrical Engineering, Hokkaido University
More informationNeural networks. Chapter 19, Sections 1 5 1
Neural networks Chapter 19, Sections 1 5 Chapter 19, Sections 1 5 1 Outline Brains Neural networks Perceptrons Multilayer perceptrons Applications of neural networks Chapter 19, Sections 1 5 2 Brains 10
More informationSecond-order Learning Algorithm with Squared Penalty Term
Second-order Learning Algorithm with Squared Penalty Term Kazumi Saito Ryohei Nakano NTT Communication Science Laboratories 2 Hikaridai, Seika-cho, Soraku-gun, Kyoto 69-2 Japan {saito,nakano}@cslab.kecl.ntt.jp
More informationAdaptive Control of a Class of Nonlinear Systems with Nonlinearly Parameterized Fuzzy Approximators
IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 9, NO. 2, APRIL 2001 315 Adaptive Control of a Class of Nonlinear Systems with Nonlinearly Parameterized Fuzzy Approximators Hugang Han, Chun-Yi Su, Yury Stepanenko
More informationAn Error-Entropy Minimization Algorithm for Supervised Training of Nonlinear Adaptive Systems
1780 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 50, NO. 7, JULY 2002 An Error-Entropy Minimization Algorithm for Supervised Training of Nonlinear Adaptive Systems Deniz Erdogmus, Member, IEEE, and Jose
More informationbelow, kernel PCA Eigenvectors, and linear combinations thereof. For the cases where the pre-image does exist, we can provide a means of constructing
Kernel PCA Pattern Reconstruction via Approximate Pre-Images Bernhard Scholkopf, Sebastian Mika, Alex Smola, Gunnar Ratsch, & Klaus-Robert Muller GMD FIRST, Rudower Chaussee 5, 12489 Berlin, Germany fbs,
More informationCS599 Lecture 2 Function Approximation in RL
CS599 Lecture 2 Function Approximation in RL Look at how experience with a limited part of the state set be used to produce good behavior over a much larger part. Overview of function approximation (FA)
More informationElectric Load Forecasting Using Wavelet Transform and Extreme Learning Machine
Electric Load Forecasting Using Wavelet Transform and Extreme Learning Machine Song Li 1, Peng Wang 1 and Lalit Goel 1 1 School of Electrical and Electronic Engineering Nanyang Technological University
More informationGaussian Processes for Regression. Carl Edward Rasmussen. Department of Computer Science. Toronto, ONT, M5S 1A4, Canada.
In Advances in Neural Information Processing Systems 8 eds. D. S. Touretzky, M. C. Mozer, M. E. Hasselmo, MIT Press, 1996. Gaussian Processes for Regression Christopher K. I. Williams Neural Computing
More informationFrom CDF to PDF A Density Estimation Method for High Dimensional Data
From CDF to PDF A Density Estimation Method for High Dimensional Data Shengdong Zhang Simon Fraser University sza75@sfu.ca arxiv:1804.05316v1 [stat.ml] 15 Apr 2018 April 17, 2018 1 Introduction Probability
More informationΣ N (d i,p z i,p ) 2 (1)
A CLASSICAL ALGORITHM FOR AVOIDING LOCAL MINIMA D Gorse and A Shepherd Department of Computer Science University College, Gower Street, London WC1E 6BT, UK J G Taylor Department of Mathematics King s College,
More informationNEAR-OPTIMAL FLIGHT LOAD SYNTHESIS USING NEURAL NETS
NEAR-OPTIMAL FLIGHT LOAD SYNTHESIS USING NEURAL NETS Michael T. Manry, Cheng-Hsiung Hsieh, and Hema Chandrasekaran Department of Electrical Engineering University of Texas at Arlington Arlington, Texas
More informationConstructing Transportable Behavioural Models for Nonlinear Electronic Devices
Constructing Transportable Behavioural Models for Nonlinear Electronic Devices David M. Walker*, Reggie Brown, Nicholas Tufillaro Integrated Solutions Laboratory HP Laboratories Palo Alto HPL-1999-3 February,
More informationNN V: The generalized delta learning rule
NN V: The generalized delta learning rule We now focus on generalizing the delta learning rule for feedforward layered neural networks. The architecture of the two-layer network considered below is shown
More informationAdvanced statistical methods for data analysis Lecture 2
Advanced statistical methods for data analysis Lecture 2 RHUL Physics www.pp.rhul.ac.uk/~cowan Universität Mainz Klausurtagung des GK Eichtheorien exp. Tests... Bullay/Mosel 15 17 September, 2008 1 Outline
More informationKeywords- Source coding, Huffman encoding, Artificial neural network, Multilayer perceptron, Backpropagation algorithm
Volume 4, Issue 5, May 2014 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Huffman Encoding
More information18.6 Regression and Classification with Linear Models
18.6 Regression and Classification with Linear Models 352 The hypothesis space of linear functions of continuous-valued inputs has been used for hundreds of years A univariate linear function (a straight
More informationPATTERN CLASSIFICATION
PATTERN CLASSIFICATION Second Edition Richard O. Duda Peter E. Hart David G. Stork A Wiley-lnterscience Publication JOHN WILEY & SONS, INC. New York Chichester Weinheim Brisbane Singapore Toronto CONTENTS
More informationArtificial Neural Networks" and Nonparametric Methods" CMPSCI 383 Nov 17, 2011!
Artificial Neural Networks" and Nonparametric Methods" CMPSCI 383 Nov 17, 2011! 1 Todayʼs lecture" How the brain works (!)! Artificial neural networks! Perceptrons! Multilayer feed-forward networks! Error
More informationComments. Assignment 3 code released. Thought questions 3 due this week. Mini-project: hopefully you have started. implement classification algorithms
Neural networks Comments Assignment 3 code released implement classification algorithms use kernels for census dataset Thought questions 3 due this week Mini-project: hopefully you have started 2 Example:
More informationA Bayesian Local Linear Wavelet Neural Network
A Bayesian Local Linear Wavelet Neural Network Kunikazu Kobayashi, Masanao Obayashi, and Takashi Kuremoto Yamaguchi University, 2-16-1, Tokiwadai, Ube, Yamaguchi 755-8611, Japan {koba, m.obayas, wu}@yamaguchi-u.ac.jp
More informationAutomatic Noise Recognition Based on Neural Network Using LPC and MFCC Feature Parameters
Proceedings of the Federated Conference on Computer Science and Information Systems pp 69 73 ISBN 978-83-60810-51-4 Automatic Noise Recognition Based on Neural Network Using LPC and MFCC Feature Parameters
More informationArtificial Neural Network Simulation of Battery Performance
Artificial work Simulation of Battery Performance C.C. O Gorman, D. Ingersoll, R.G. Jungst and T.L. Paez Sandia National Laboratories PO Box 58 Albuquerque, NM 8785 Abstract Although they appear deceptively
More informationNeural Control for Rolling Mills: Incorporating Domain Theories to Overcome Data Deficiency
Neural Control for Rolling Mills: Incorporating Domain Theories to Overcome Data Deficiency Martin Roscheisen Computer Science Dept. Munich Technical University 8 Munich 40, FRG Reimar Hofmann Computer
More informationOn Learning µ-perceptron Networks with Binary Weights
On Learning µ-perceptron Networks with Binary Weights Mostefa Golea Ottawa-Carleton Institute for Physics University of Ottawa Ottawa, Ont., Canada K1N 6N5 050287@acadvm1.uottawa.ca Mario Marchand Ottawa-Carleton
More informationCSE 190: Reinforcement Learning: An Introduction. Chapter 8: Generalization and Function Approximation. Pop Quiz: What Function Are We Approximating?
CSE 190: Reinforcement Learning: An Introduction Chapter 8: Generalization and Function Approximation Objectives of this chapter: Look at how experience with a limited part of the state set be used to
More informationMultilayer Perceptron Learning Utilizing Singular Regions and Search Pruning
Multilayer Perceptron Learning Utilizing Singular Regions and Search Pruning Seiya Satoh and Ryohei Nakano Abstract In a search space of a multilayer perceptron having hidden units, MLP(), there exist
More informationNeural Networks. Chapter 18, Section 7. TB Artificial Intelligence. Slides from AIMA 1/ 21
Neural Networks Chapter 8, Section 7 TB Artificial Intelligence Slides from AIMA http://aima.cs.berkeley.edu / 2 Outline Brains Neural networks Perceptrons Multilayer perceptrons Applications of neural
More informationMultilayer Neural Networks
Multilayer Neural Networks Multilayer Neural Networks Discriminant function flexibility NON-Linear But with sets of linear parameters at each layer Provably general function approximators for sufficient
More informationCh.6 Deep Feedforward Networks (2/3)
Ch.6 Deep Feedforward Networks (2/3) 16. 10. 17. (Mon.) System Software Lab., Dept. of Mechanical & Information Eng. Woonggy Kim 1 Contents 6.3. Hidden Units 6.3.1. Rectified Linear Units and Their Generalizations
More informationRadial-Basis Function Networks. Radial-Basis Function Networks
Radial-Basis Function Networks November 00 Michel Verleysen Radial-Basis Function Networks - Radial-Basis Function Networks p Origin: Cover s theorem p Interpolation problem p Regularization theory p Generalized
More informationOptimal transfer function neural networks
Optimal transfer function neural networks Norbert Jankowski and Włodzisław Duch Department of Computer ethods Nicholas Copernicus University ul. Grudziądzka, 87 Toru ń, Poland, e-mail:{norbert,duch}@phys.uni.torun.pl
More informationArtificial Neural Networks. Edward Gatt
Artificial Neural Networks Edward Gatt What are Neural Networks? Models of the brain and nervous system Highly parallel Process information much more like the brain than a serial computer Learning Very
More informationFeed-forward Network Functions
Feed-forward Network Functions Sargur Srihari Topics 1. Extension of linear models 2. Feed-forward Network Functions 3. Weight-space symmetries 2 Recap of Linear Models Linear Models for Regression, Classification
More informationModel of a Biological Neuron as a Temporal Neural Network
Model of a Biological Neuron as a Temporal Neural Network Sean D. Murphy and Edward W. Kairiss Interdepartmental Neuroscience Program, Department of Psychology, and The Center for Theoretical and Applied
More informationINTERNATIONAL COMPUTER SCIENCE INSTITUTE
-. How Receptive Field Parameters Affect eural Learning Stephen M. Omohundro Barlett W. Mel TR-91-010 January, 1991 ITERATIOAL COMPUTER SCIECE ISTITUTE 1947 Center Street, Suite 600 Berkeley, California
More informationComparing linear and non-linear transformation of speech
Comparing linear and non-linear transformation of speech Larbi Mesbahi, Vincent Barreaud and Olivier Boeffard IRISA / ENSSAT - University of Rennes 1 6, rue de Kerampont, Lannion, France {lmesbahi, vincent.barreaud,
More informationModeling High-Dimensional Discrete Data with Multi-Layer Neural Networks
Modeling High-Dimensional Discrete Data with Multi-Layer Neural Networks Yoshua Bengio Dept. IRO Université de Montréal Montreal, Qc, Canada, H3C 3J7 bengioy@iro.umontreal.ca Samy Bengio IDIAP CP 592,
More informationA Fast Stochastic Error-Descent Algorithm for Supervised Learning and Optimization
A Fast Stochastic Error-Descent Algorithm for Supervised Learning and Optimization Gert Cauwenberghs California Institute of Technology Mail-Code 128-95 Pasadena, CA 91125 E-mail: gert(qcco. cal tech.
More informationRadial Basis Functions for Process Control Lyle H. Ungar Tom Johnson Richard D. De Veaux University ofpennsylvania Voice Processing Corp. Princeton Un
Radial Basis Functions for Process Control Lyle H. Ungar Tom Johnson Richard D. De Veaux University ofpennsylvania Voice Processing Corp. Princeton University Abstract Radial basis function èrbfsè neural
More informationMultilayer Feedforward Networks. Berlin Chen, 2002
Multilayer Feedforard Netors Berlin Chen, 00 Introduction The single-layer perceptron classifiers discussed previously can only deal ith linearly separable sets of patterns The multilayer netors to be
More informationNONLINEAR CLASSIFICATION AND REGRESSION. J. Elder CSE 4404/5327 Introduction to Machine Learning and Pattern Recognition
NONLINEAR CLASSIFICATION AND REGRESSION Nonlinear Classification and Regression: Outline 2 Multi-Layer Perceptrons The Back-Propagation Learning Algorithm Generalized Linear Models Radial Basis Function
More informationModification of Hard-Limiting Multilayer Neural Networks for Confidence Evaluation
Modification of Hard-Limiting Multilayer Neural Networks for Confidence Evaluation Robert Eigenmann and Josef A. Nossek nstitute for Network Theory and Circuit Design Munich University of Technology 8333
More informationNeural Networks Lecture 4: Radial Bases Function Networks
Neural Networks Lecture 4: Radial Bases Function Networks H.A Talebi Farzaneh Abdollahi Department of Electrical Engineering Amirkabir University of Technology Winter 2011. A. Talebi, Farzaneh Abdollahi
More informationENTROPIC ANALYSIS AND INCREMENTAL SYNTHESIS OF MULTILAYERED FEEDFORWARD NEURAL NETWORKS
International Journal of Neural Systems, Vol. 8, Nos. & 6 (October/December, 997) 67 69 c World Scientific Publishing Company ENTROPIC ANALYSIS AND INCREMENTAL SYNTHESIS OF MULTILAYERED FEEDFORWARD NEURAL
More informationDeep Learning. Convolutional Neural Network (CNNs) Ali Ghodsi. October 30, Slides are partially based on Book in preparation, Deep Learning
Convolutional Neural Network (CNNs) University of Waterloo October 30, 2015 Slides are partially based on Book in preparation, by Bengio, Goodfellow, and Aaron Courville, 2015 Convolutional Networks Convolutional
More informationMulti-Layer Boosting for Pattern Recognition
Multi-Layer Boosting for Pattern Recognition François Fleuret IDIAP Research Institute, Centre du Parc, P.O. Box 592 1920 Martigny, Switzerland fleuret@idiap.ch Abstract We extend the standard boosting
More informationCSE 417T: Introduction to Machine Learning. Final Review. Henry Chai 12/4/18
CSE 417T: Introduction to Machine Learning Final Review Henry Chai 12/4/18 Overfitting Overfitting is fitting the training data more than is warranted Fitting noise rather than signal 2 Estimating! "#$
More informationLecture 13 - Handling Nonlinearity
Lecture 3 - Handling Nonlinearit Nonlinearit issues in control practice Setpoint scheduling/feedforward path planning repla - linear interpolation Nonlinear maps B-splines Multivariable interpolation:
More informationArtificial Neural Networks 2
CSC2515 Machine Learning Sam Roweis Artificial Neural s 2 We saw neural nets for classification. Same idea for regression. ANNs are just adaptive basis regression machines of the form: y k = j w kj σ(b
More informationDifferent Criteria for Active Learning in Neural Networks: A Comparative Study
Different Criteria for Active Learning in Neural Networks: A Comparative Study Jan Poland and Andreas Zell University of Tübingen, WSI - RA Sand 1, 72076 Tübingen, Germany Abstract. The field of active
More informationRelating Real-Time Backpropagation and. Backpropagation-Through-Time: An Application of Flow Graph. Interreciprocity.
Neural Computation, 1994 Relating Real-Time Backpropagation and Backpropagation-Through-Time: An Application of Flow Graph Interreciprocity. Francoise Beaufays and Eric A. Wan Abstract We show that signal
More informationRobust Multiple Estimator Systems for the Analysis of Biophysical Parameters From Remotely Sensed Data
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 43, NO. 1, JANUARY 2005 159 Robust Multiple Estimator Systems for the Analysis of Biophysical Parameters From Remotely Sensed Data Lorenzo Bruzzone,
More informationARTIFICIAL NEURAL NETWORKS گروه مطالعاتي 17 بهار 92
ARTIFICIAL NEURAL NETWORKS گروه مطالعاتي 17 بهار 92 BIOLOGICAL INSPIRATIONS Some numbers The human brain contains about 10 billion nerve cells (neurons) Each neuron is connected to the others through 10000
More informationA Course in Machine Learning
A Course in Machine Learning Hal Daumé III Copyright 2012 Hal Daumé III http://ciml.info This book is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy
More informationIterative Construction of Sparse Polynomial Approximations
Iterative Construction of Sparse Polynomial Approximations Terence D. Sanger Massachusetts Institute of Technology Room E25-534 Cambridge, MA 02139 tds@ai.mit.edu Richard S. Sutton GTE Laboratories Incorporated
More informationNeural networks. Chapter 20. Chapter 20 1
Neural networks Chapter 20 Chapter 20 1 Outline Brains Neural networks Perceptrons Multilayer networks Applications of neural networks Chapter 20 2 Brains 10 11 neurons of > 20 types, 10 14 synapses, 1ms
More informationLecture 2. G. Cowan Lectures on Statistical Data Analysis Lecture 2 page 1
Lecture 2 1 Probability (90 min.) Definition, Bayes theorem, probability densities and their properties, catalogue of pdfs, Monte Carlo 2 Statistical tests (90 min.) general concepts, test statistics,
More informationLECTURE NOTE #NEW 6 PROF. ALAN YUILLE
LECTURE NOTE #NEW 6 PROF. ALAN YUILLE 1. Introduction to Regression Now consider learning the conditional distribution p(y x). This is often easier than learning the likelihood function p(x y) and the
More informationAn Adaptive Neural Network Scheme for Radar Rainfall Estimation from WSR-88D Observations
2038 JOURNAL OF APPLIED METEOROLOGY An Adaptive Neural Network Scheme for Radar Rainfall Estimation from WSR-88D Observations HONGPING LIU, V.CHANDRASEKAR, AND GANG XU Colorado State University, Fort Collins,
More information10 NEURAL NETWORKS Bio-inspired Multi-Layer Networks. Learning Objectives:
10 NEURAL NETWORKS TODO The first learning models you learned about (decision trees and nearest neighbor models) created complex, non-linear decision boundaries. We moved from there to the perceptron,
More informationUpper Bounds on the Number of Hidden Neurons in Feedforward Networks with Arbitrary Bounded Nonlinear Activation Functions
224 IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 9, NO. 1, JANUARY 1998 Upper Bounds on the Number of Hidden Neurons in Feedforward Networks with Arbitrary Bounded Nonlinear Activation Functions Guang-Bin
More informationAbstract. In this paper we propose recurrent neural networks with feedback into the input
Recurrent Neural Networks for Missing or Asynchronous Data Yoshua Bengio Dept. Informatique et Recherche Operationnelle Universite de Montreal Montreal, Qc H3C-3J7 bengioy@iro.umontreal.ca Francois Gingras
More informationOn Inverse Sigmoid Functions
Syracuse University SURFACE Electrical Engineering and Computer Science Technical Reports College of Engineering and Computer Science 6-15-1993 On Inverse Sigmoid Functions Anil Ravindran Menon Syracuse
More informationIntroduction to Machine Learning
Introduction to Machine Learning Neural Networks Varun Chandola x x 5 Input Outline Contents February 2, 207 Extending Perceptrons 2 Multi Layered Perceptrons 2 2. Generalizing to Multiple Labels.................
More informationNeural Network Weight Space Symmetries Can Speed up Genetic Learning
Neural Network Weight Space Symmetries Can Speed up Genetic Learning ROMAN NERUDA Λ Institue of Computer Science Academy of Sciences of the Czech Republic P.O. Box 5, 187 Prague, Czech Republic tel: (4)665375,fax:(4)8585789
More informationOn Optimal Physical Synthesis of Sleep Transistors
On Optimal Physical Synthesis of Sleep Transistors Changbo Long, Jinjun Xiong and Lei He {longchb, jinjun, lhe}@ee.ucla.edu EE department, University of California, Los Angeles, CA, 90095 ABSTRACT Considering
More informationoutput dimension input dimension Gaussian evidence Gaussian Gaussian evidence evidence from t +1 inputs and outputs at time t x t+2 x t-1 x t+1
To appear in M. S. Kearns, S. A. Solla, D. A. Cohn, (eds.) Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 999. Learning Nonlinear Dynamical Systems using an EM Algorithm Zoubin
More information