Radial Basis Function Networks: Algorithms
|
|
- Damon Kelley
- 6 years ago
- Views:
Transcription
1 Radial Basis Function Networks: Algorithms Introduction to Neural Networks : Lecture 13 John A. Bullinaria, The RBF Maing 2. The RBF Network Architecture 3. Comutational Power of RBF Networks 4. Training an RBF Network 5. Unsuervised Otimization of the Basis Functions 6. Finding the Outut Weights
2 The Radial Basis Function (RBF) Maing We are working within the standard framework of function aroximation. We have a set of N data oints in a multi-dimensional sace such that every D dimensional inut vector x i = { x : i = 1,..., D} has a corresonding K dimensional target outut t = { tk : k = 1,..., K}. The target oututs will generally be generated by some underlying functions g k ( x ) lus random noise. The goal is to aroximate the g k ( x ) with functions y k ( x ) of the form y M ( x) = w φ ( x) k k = 0 We shall concentrate on the case of Gaussian basis functions φ ( x) = ex x 2σ µ 2 2 in which we have basis centres { µ } and widths { σ }. Naturally, the way to roceed is to develo a rocess for finding the aroriate values for M, { w k }, { µ i } and { σ }. L13-2
3 The RBF Network Architecture We can cast the RBF Maing into a form that resembles a neural network: 1 K oututs y k weights w k 1 M basis functions φ (x i, µ i, σ ) weights µ i 1 D inuts x i The hidden to outut layer art oerates like a standard feed-forward MLP network, with the sum of the weighted hidden unit activations giving the outut unit activations. The hidden unit activations are given by the basis functions φ ( x, µ, σ ), which deend on the weights { µ i, σ } and inut activations { x i } in a non-standard manner. L13-3
4 Comutational Power of RBF Networks Intuitively, we can easily understand why linear suerositions of localised basis functions are caable of universal aroximation. More formally: Hartman, Keeler & Kowalski (1990, Neural Comutation, vol.2, ) rovided a formal roof of this roerty for networks with Gaussian basis functions in which the widths { σ } are treated as adustable arameters. Park & Sandberg (1991, Neural Comutation, vol.3, ; and 1993, Neural Comutation, vol.5, ) showed that with only mild restrictions on the basis functions, the universal function aroximation roerty still holds. As with the corresonding roofs for MLPs, these are existence roofs which rely on the availability of an arbitrarily large number of hidden units (i.e. basis functions). However, they do rovide a theoretical foundation on which ractical alications can be based with confidence. L13-4
5 Training RBF Networks The roofs about comutational ower tell us what an RBF Network can do, but nothing about how to find all its arameters/weights { w k, µ i, σ }. Unlike in MLPs, in RBF networks the hidden and outut layers lay very different roles, and the corresonding weights have very different meanings and roerties. It is therefore aroriate to use different learning algorithms for them. The inut to hidden weights (i.e. basis function arameters { µ i, σ }) can be trained (or set) using any of a number of unsuervised learning techniques. Then, after the inut to hidden weights are found, they are ket fixed while the hidden to outut weights are learned. Since this second stage of training involves ust a single layer of weights { w k } and linear outut activation functions, the weights can easily be found analytically by solving a set of linear equations. This can be done very quickly, without the need for a set of iterative weight udates as in gradient descent learning. L13-5
6 Basis Function Otimization One maor advantage of RBF networks is the ossibility of choosing suitable hidden unit/basis function arameters without having to erform a full non-linear otimization of the whole network. We shall now look at several ways of doing this: 1. Fixed centres selected at random 2. Orthogonal least squares 3. K-means clustering These are all unsuervised techniques, which will be articularly useful in situations where labelled data is in short suly, but there is lenty of unlabelled data (i.e. inuts without outut targets). Next lecture we shall look at how we might get better results by erforming a full suervised non-linear otimization of the network instead. With either aroach, determining a good value for M remains a roblem. It will generally be aroriate to comare the results for a range of different values, following the same kind of validation/cross validation methodology used for otimizing MLPs. L13-6
7 Fixed Centres Selected At Random The simlest and quickest aroach to setting the RBF arameters is to have their centres fixed at M oints selected at random from the N data oints, and to set all their widths to be equal and fixed at an aroriate size for the distribution of data oints. Secifically, we can use normalised RBFs centred at { µ } defined by φ ( x) = ex x 2σ µ 2 2 where { µ } { x } and the σ are all related in the same way to the maximum or average distance between the chosen centres µ. Common choices are σ d 2M = max or σ = 2d ave which ensure that the individual RBFs are neither too wide, nor too narrow, for the given training data. For large training sets, this aroach gives reasonable results. L13-7
8 Orthogonal Least Squares A more rinciled aroach to selecting a sub-set of data oints as the basis function centres is based on the technique of orthogonal least squares. This involves the sequential addition of new basis functions, each centred on one of the data oints. At each stage, we try out each otential Lth basis function by using the N L other data oints to determine the networks outut weights. The otential Lth basis function which leaves the smallest residual outut sum squared outut error is used, and we move on to choose which L+1th basis function to add. This sounds wasteful, but if we construct a set of orthogonal vectors in the sace S sanned by the vectors of hidden unit activations for each attern in the training set, we can calculate directly which data oint should be chosen as the next basis function. To get good generalization we generally use validation/cross validation to sto the rocess when an aroriate number of data oints have been selected as centres. L13-8
9 K-Means Clustering A otentially even better aroach is to use clustering techniques to find a set of centres which more accurately reflects the distribution of the data oints. The K-Means Clustering Algorithm icks the number K of centres in advance, and then follows a simle re-estimation rocedure to artition the data oints { x } into K disoint sub-sets S containing N data oints to minimize the sum squared clustering function J K x µ = 1 S = 2 where µ is the mean/centroid of the data oints in set S given by = 1 µ N S x Once the basis centres have been determined in this way, the widths can then be set according to the variances of the oints in the corresonding cluster. L13-9
10 Dealing with the Outut Layer Given the hidden unit activations φ ( x, µ, σ ) are fixed while we determine the outut weights { w k }, we essentially only have to find the weights that otimise a single layer linear network. As with MLPs we can define a sum-squared outut error measure 2 2 ( k k ) k 1 E = y ( x ) t but here the oututs are a simle linear combination of the hidden unit activations, i.e. y k M ( x ) = w φ ( x ) = 0 k. At the minimum of E the gradients with resect to all the weights w ki will be zero, so E w ki M = wkφ ( x ) tk φi( x ) = = 0 0 and linear equations like this are well known to be easy to solve analytically. L13-10
11 Comuting the Outut Weights Our equations for the weights are most conveniently written in matrix form by defining matrices with comonents (W) k = w k, (Φ) = φ (x ), and (T) k = {t k }. This gives Φ T T ( Φ W T) = 0 and the formal solution for the weights is W T = Φ T in which we have the standard seudo inverse of Φ T 1 Φ ( Φ Φ) Φ which can be seen to have the roerty Φ Φ = I. We see that the network weights can be comuted by fast linear matrix inversion techniques. In ractice we tend to use singular value decomosition (SVD) to avoid ossible ill-conditioning of Φ, i.e. Φ T Φ being singular or near singular. T L13-11
12 Overview and Reading 1. We began by defining Radial Basis Function (RBF) maings and the corresonding network architecture. 2. Then we considered the comutational ower of RBF networks. 3. We then saw how the two layers of network weights were rather different and different techniques were aroriate for training each of them. 4. We first looked at several unsuervised techniques for carrying out the first stage, namely otimizing the basis functions. 5. We then saw how the second stage, determining the outut weights, could be erformed by fast linear matrix inversion techniques. Reading 1. Bisho: Sections 5.2, 5.3, 5.9, 5.10, Haykin: Sections 5.4, 5.9, 5.10, 5.13 L13-12
Feedback-error control
Chater 4 Feedback-error control 4.1 Introduction This chater exlains the feedback-error (FBE) control scheme originally described by Kawato [, 87, 8]. FBE is a widely used neural network based controller
More informationUncorrelated Multilinear Principal Component Analysis for Unsupervised Multilinear Subspace Learning
TNN-2009-P-1186.R2 1 Uncorrelated Multilinear Princial Comonent Analysis for Unsuervised Multilinear Subsace Learning Haiing Lu, K. N. Plataniotis and A. N. Venetsanooulos The Edward S. Rogers Sr. Deartment
More informationSolved Problems. (a) (b) (c) Figure P4.1 Simple Classification Problems First we draw a line between each set of dark and light data points.
Solved Problems Solved Problems P Solve the three simle classification roblems shown in Figure P by drawing a decision boundary Find weight and bias values that result in single-neuron ercetrons with the
More informationPrincipal Components Analysis and Unsupervised Hebbian Learning
Princial Comonents Analysis and Unsuervised Hebbian Learning Robert Jacobs Deartment of Brain & Cognitive Sciences University of Rochester Rochester, NY 1467, USA August 8, 008 Reference: Much of the material
More informationRecent Developments in Multilayer Perceptron Neural Networks
Recent Develoments in Multilayer Percetron eural etworks Walter H. Delashmit Lockheed Martin Missiles and Fire Control Dallas, Texas 75265 walter.delashmit@lmco.com walter.delashmit@verizon.net Michael
More informationE( x ) = [b(n) - a(n,m)x(m) ]
Exam #, EE5353, Fall 0. Here we consider MLPs with binary-valued inuts (0 or ). (a) If the MLP has inuts, what is the maximum degree D of its PBF model? (b) If the MLP has inuts, what is the maximum value
More informationAI*IA 2003 Fusion of Multiple Pattern Classifiers PART III
AI*IA 23 Fusion of Multile Pattern Classifiers PART III AI*IA 23 Tutorial on Fusion of Multile Pattern Classifiers by F. Roli 49 Methods for fusing multile classifiers Methods for fusing multile classifiers
More informationUniformly best wavenumber approximations by spatial central difference operators: An initial investigation
Uniformly best wavenumber aroximations by satial central difference oerators: An initial investigation Vitor Linders and Jan Nordström Abstract A characterisation theorem for best uniform wavenumber aroximations
More informationApproximating min-max k-clustering
Aroximating min-max k-clustering Asaf Levin July 24, 2007 Abstract We consider the roblems of set artitioning into k clusters with minimum total cost and minimum of the maximum cost of a cluster. The cost
More informationMultilayer Perceptron Neural Network (MLPs) For Analyzing the Properties of Jordan Oil Shale
World Alied Sciences Journal 5 (5): 546-552, 2008 ISSN 1818-4952 IDOSI Publications, 2008 Multilayer Percetron Neural Network (MLPs) For Analyzing the Proerties of Jordan Oil Shale 1 Jamal M. Nazzal, 2
More informationGeneration of Linear Models using Simulation Results
4. IMACS-Symosium MATHMOD, Wien, 5..003,. 436-443 Generation of Linear Models using Simulation Results Georg Otte, Sven Reitz, Joachim Haase Fraunhofer Institute for Integrated Circuits, Branch Lab Design
More informationSession 5: Review of Classical Astrodynamics
Session 5: Review of Classical Astrodynamics In revious lectures we described in detail the rocess to find the otimal secific imulse for a articular situation. Among the mission requirements that serve
More informationECE 534 Information Theory - Midterm 2
ECE 534 Information Theory - Midterm Nov.4, 009. 3:30-4:45 in LH03. You will be given the full class time: 75 minutes. Use it wisely! Many of the roblems have short answers; try to find shortcuts. You
More informationA Simple Weight Decay Can Improve. Abstract. It has been observed in numerical simulations that a weight decay can improve
In Advances in Neural Information Processing Systems 4, J.E. Moody, S.J. Hanson and R.P. Limann, eds. Morgan Kaumann Publishers, San Mateo CA, 1995,. 950{957. A Simle Weight Decay Can Imrove Generalization
More informationE( x ) [b(n) - a(n, m)x(m) ]
Homework #, EE5353. An XOR network has two inuts, one hidden unit, and one outut. It is fully connected. Gie the network's weights if the outut unit has a ste actiation and the hidden unit actiation is
More informationResearch of power plant parameter based on the Principal Component Analysis method
Research of ower lant arameter based on the Princial Comonent Analysis method Yang Yang *a, Di Zhang b a b School of Engineering, Bohai University, Liaoning Jinzhou, 3; Liaoning Datang international Jinzhou
More informationDETC2003/DAC AN EFFICIENT ALGORITHM FOR CONSTRUCTING OPTIMAL DESIGN OF COMPUTER EXPERIMENTS
Proceedings of DETC 03 ASME 003 Design Engineering Technical Conferences and Comuters and Information in Engineering Conference Chicago, Illinois USA, Setember -6, 003 DETC003/DAC-48760 AN EFFICIENT ALGORITHM
More informationState Estimation with ARMarkov Models
Deartment of Mechanical and Aerosace Engineering Technical Reort No. 3046, October 1998. Princeton University, Princeton, NJ. State Estimation with ARMarkov Models Ryoung K. Lim 1 Columbia University,
More information4. Score normalization technical details We now discuss the technical details of the score normalization method.
SMT SCORING SYSTEM This document describes the scoring system for the Stanford Math Tournament We begin by giving an overview of the changes to scoring and a non-technical descrition of the scoring rules
More informationConvex Optimization methods for Computing Channel Capacity
Convex Otimization methods for Comuting Channel Caacity Abhishek Sinha Laboratory for Information and Decision Systems (LIDS), MIT sinhaa@mit.edu May 15, 2014 We consider a classical comutational roblem
More informationSession 12 : Monopropellant Thrusters
Session 12 : Monoroellant Thrusters Electrothermal augmentation of chemical rockets was the first form of electric roulsion alied in sace vehicles. In its original imlementation, resistojets were used
More informationFuzzy Automata Induction using Construction Method
Journal of Mathematics and Statistics 2 (2): 395-4, 26 ISSN 1549-3644 26 Science Publications Fuzzy Automata Induction using Construction Method 1,2 Mo Zhi Wen and 2 Wan Min 1 Center of Intelligent Control
More informationNeural Networks Lecture 4: Radial Bases Function Networks
Neural Networks Lecture 4: Radial Bases Function Networks H.A Talebi Farzaneh Abdollahi Department of Electrical Engineering Amirkabir University of Technology Winter 2011. A. Talebi, Farzaneh Abdollahi
More informationAn Investigation on the Numerical Ill-conditioning of Hybrid State Estimators
An Investigation on the Numerical Ill-conditioning of Hybrid State Estimators S. K. Mallik, Student Member, IEEE, S. Chakrabarti, Senior Member, IEEE, S. N. Singh, Senior Member, IEEE Deartment of Electrical
More informationDynamic System Eigenvalue Extraction using a Linear Echo State Network for Small-Signal Stability Analysis a Novel Application
Dynamic System Eigenvalue Extraction using a Linear Echo State Network for Small-Signal Stability Analysis a Novel Alication Jiaqi Liang, Jing Dai, Ganesh K. Venayagamoorthy, and Ronald G. Harley Abstract
More informationarxiv: v1 [physics.data-an] 26 Oct 2012
Constraints on Yield Parameters in Extended Maximum Likelihood Fits Till Moritz Karbach a, Maximilian Schlu b a TU Dortmund, Germany, moritz.karbach@cern.ch b TU Dortmund, Germany, maximilian.schlu@cern.ch
More informationOn Wald-Type Optimal Stopping for Brownian Motion
J Al Probab Vol 34, No 1, 1997, (66-73) Prerint Ser No 1, 1994, Math Inst Aarhus On Wald-Tye Otimal Stoing for Brownian Motion S RAVRSN and PSKIR The solution is resented to all otimal stoing roblems of
More informationNotes on Instrumental Variables Methods
Notes on Instrumental Variables Methods Michele Pellizzari IGIER-Bocconi, IZA and frdb 1 The Instrumental Variable Estimator Instrumental variable estimation is the classical solution to the roblem of
More informationBest approximation by linear combinations of characteristic functions of half-spaces
Best aroximation by linear combinations of characteristic functions of half-saces Paul C. Kainen Deartment of Mathematics Georgetown University Washington, D.C. 20057-1233, USA Věra Kůrková Institute of
More informationCOMPARISON OF VARIOUS OPTIMIZATION TECHNIQUES FOR DESIGN FIR DIGITAL FILTERS
NCCI 1 -National Conference on Comutational Instrumentation CSIO Chandigarh, INDIA, 19- March 1 COMPARISON OF VARIOUS OPIMIZAION ECHNIQUES FOR DESIGN FIR DIGIAL FILERS Amanjeet Panghal 1, Nitin Mittal,Devender
More informationCombining Logistic Regression with Kriging for Mapping the Risk of Occurrence of Unexploded Ordnance (UXO)
Combining Logistic Regression with Kriging for Maing the Risk of Occurrence of Unexloded Ordnance (UXO) H. Saito (), P. Goovaerts (), S. A. McKenna (2) Environmental and Water Resources Engineering, Deartment
More informationA Bound on the Error of Cross Validation Using the Approximation and Estimation Rates, with Consequences for the Training-Test Split
A Bound on the Error of Cross Validation Using the Aroximation and Estimation Rates, with Consequences for the Training-Test Slit Michael Kearns AT&T Bell Laboratories Murray Hill, NJ 7974 mkearns@research.att.com
More information1 University of Edinburgh, 2 British Geological Survey, 3 China University of Petroleum
Estimation of fluid mobility from frequency deendent azimuthal AVO a synthetic model study Yingrui Ren 1*, Xiaoyang Wu 2, Mark Chaman 1 and Xiangyang Li 2,3 1 University of Edinburgh, 2 British Geological
More informationUncorrelated Multilinear Discriminant Analysis with Regularization and Aggregation for Tensor Object Recognition
TNN-2007-P-0332.R1 1 Uncorrelated Multilinear Discriminant Analysis with Regularization and Aggregation for Tensor Object Recognition Haiing Lu, K.N. Plataniotis and A.N. Venetsanooulos The Edward S. Rogers
More informationFig. 21: Architecture of PeerSim [44]
Sulementary Aendix A: Modeling HPP with PeerSim Fig. : Architecture of PeerSim [] In PeerSim, every comonent can be relaced by another comonent imlementing the same interface, and the general simulation
More informationUncorrelated Multilinear Discriminant Analysis with Regularization and Aggregation for Tensor Object Recognition
Uncorrelated Multilinear Discriminant Analysis with Regularization and Aggregation for Tensor Object Recognition Haiing Lu, K.N. Plataniotis and A.N. Venetsanooulos The Edward S. Rogers Sr. Deartment of
More informationChapter 1 Fundamentals
Chater Fundamentals. Overview of Thermodynamics Industrial Revolution brought in large scale automation of many tedious tasks which were earlier being erformed through manual or animal labour. Inventors
More informationNumerical Linear Algebra
Numerical Linear Algebra Numerous alications in statistics, articularly in the fitting of linear models. Notation and conventions: Elements of a matrix A are denoted by a ij, where i indexes the rows and
More informationRecursive Estimation of the Preisach Density function for a Smart Actuator
Recursive Estimation of the Preisach Density function for a Smart Actuator Ram V. Iyer Deartment of Mathematics and Statistics, Texas Tech University, Lubbock, TX 7949-142. ABSTRACT The Preisach oerator
More informationDistributed Rule-Based Inference in the Presence of Redundant Information
istribution Statement : roved for ublic release; distribution is unlimited. istributed Rule-ased Inference in the Presence of Redundant Information June 8, 004 William J. Farrell III Lockheed Martin dvanced
More informationPARTIAL FACE RECOGNITION: A SPARSE REPRESENTATION-BASED APPROACH. Luoluo Liu, Trac D. Tran, and Sang Peter Chin
PARTIAL FACE RECOGNITION: A SPARSE REPRESENTATION-BASED APPROACH Luoluo Liu, Trac D. Tran, and Sang Peter Chin Det. of Electrical and Comuter Engineering, Johns Hokins Univ., Baltimore, MD 21218, USA {lliu69,trac,schin11}@jhu.edu
More informationFinite Mixture EFA in Mplus
Finite Mixture EFA in Mlus November 16, 2007 In this document we describe the Mixture EFA model estimated in Mlus. Four tyes of deendent variables are ossible in this model: normally distributed, ordered
More informationESTIMATION OF THE OUTPUT DEVIATION NORM FOR UNCERTAIN, DISCRETE-TIME NONLINEAR SYSTEMS IN A STATE DEPENDENT FORM
Int. J. Al. Math. Comut. Sci. 2007 Vol. 17 No. 4 505 513 DOI: 10.2478/v10006-007-0042-z ESTIMATION OF THE OUTPUT DEVIATION NORM FOR UNCERTAIN DISCRETE-TIME NONLINEAR SYSTEMS IN A STATE DEPENDENT FORM PRZEMYSŁAW
More informationPCA fused NN approach for drill wear prediction in drilling mild steel specimen
PCA fused NN aroach for drill wear rediction in drilling mild steel secimen S.S. Panda Deartment of Mechanical Engineering, Indian Institute of Technology Patna, Bihar-8000, India, Tel No.: 9-99056477(M);
More informationEvaluating Circuit Reliability Under Probabilistic Gate-Level Fault Models
Evaluating Circuit Reliability Under Probabilistic Gate-Level Fault Models Ketan N. Patel, Igor L. Markov and John P. Hayes University of Michigan, Ann Arbor 48109-2122 {knatel,imarkov,jhayes}@eecs.umich.edu
More informationChater Matrix Norms and Singular Value Decomosition Introduction In this lecture, we introduce the notion of a norm for matrices The singular value de
Lectures on Dynamic Systems and Control Mohammed Dahleh Munther A Dahleh George Verghese Deartment of Electrical Engineering and Comuter Science Massachuasetts Institute of Technology c Chater Matrix Norms
More informationEstimation of the large covariance matrix with two-step monotone missing data
Estimation of the large covariance matrix with two-ste monotone missing data Masashi Hyodo, Nobumichi Shutoh 2, Takashi Seo, and Tatjana Pavlenko 3 Deartment of Mathematical Information Science, Tokyo
More informationSTA 250: Statistics. Notes 7. Bayesian Approach to Statistics. Book chapters: 7.2
STA 25: Statistics Notes 7. Bayesian Aroach to Statistics Book chaters: 7.2 1 From calibrating a rocedure to quantifying uncertainty We saw that the central idea of classical testing is to rovide a rigorous
More informationSimple geometric interpretation of signal evolution in phase-sensitive fibre optic parametric amplifier
Simle geometric interretation of signal evolution in hase-sensitive fibre otic arametric amlifier A.A. REDYUK,,,* A.E. BEDNYAKOVA,, S.B. MEDVEDEV, M.P. FEDORUK,, AND S.K. TURITSYN,3 Novosibirsk State University,
More informationFigure : An 8 bridge design grid. (a) Run this model using LOQO. What is the otimal comliance? What is the running time?
5.094/SMA53 Systems Otimization: Models and Comutation Assignment 5 (00 o i n ts) Due Aril 7, 004 Some Convex Analysis (0 o i n ts) (a) Given ositive scalars L and E, consider the following set in three-dimensional
More informationA Recursive Block Incomplete Factorization. Preconditioner for Adaptive Filtering Problem
Alied Mathematical Sciences, Vol. 7, 03, no. 63, 3-3 HIKARI Ltd, www.m-hiari.com A Recursive Bloc Incomlete Factorization Preconditioner for Adative Filtering Problem Shazia Javed School of Mathematical
More informationLinear diophantine equations for discrete tomography
Journal of X-Ray Science and Technology 10 001 59 66 59 IOS Press Linear diohantine euations for discrete tomograhy Yangbo Ye a,gewang b and Jiehua Zhu a a Deartment of Mathematics, The University of Iowa,
More informationMATH 2710: NOTES FOR ANALYSIS
MATH 270: NOTES FOR ANALYSIS The main ideas we will learn from analysis center around the idea of a limit. Limits occurs in several settings. We will start with finite limits of sequences, then cover infinite
More informationOptimal array pattern synthesis with desired magnitude response
Otimal array attern synthesis with desired magnitude resonse A.M. Pasqual a, J.R. Arruda a and P. erzog b a Universidade Estadual de Caminas, Rua Mendeleiev, 00, Cidade Universitária Zeferino Vaz, 13083-970
More informationDownloaded from jhs.mazums.ac.ir at 9: on Monday September 17th 2018 [ DOI: /acadpub.jhs ]
Iranian journal of health sciences 013; 1(): 56-60 htt://jhs.mazums.ac.ir Original Article Comaring Two Formulas of Samle Size Determination for Prevalence Studies Hamed Tabesh 1 *Azadeh Saki Fatemeh Pourmotahari
More informationHidden Predictors: A Factor Analysis Primer
Hidden Predictors: A Factor Analysis Primer Ryan C Sanchez Western Washington University Factor Analysis is a owerful statistical method in the modern research sychologist s toolbag When used roerly, factor
More informationGeneral Linear Model Introduction, Classes of Linear models and Estimation
Stat 740 General Linear Model Introduction, Classes of Linear models and Estimation An aim of scientific enquiry: To describe or to discover relationshis among events (variables) in the controlled (laboratory)
More informationLIMITATIONS OF RECEPTRON. XOR Problem The failure of the perceptron to successfully simple problem such as XOR (Minsky and Papert).
LIMITATIONS OF RECEPTRON XOR Problem The failure of the ercetron to successfully simle roblem such as XOR (Minsky and Paert). x y z x y z 0 0 0 0 0 0 Fig. 4. The exclusive-or logic symbol and function
More informationLearning Sequence Motif Models Using Gibbs Sampling
Learning Sequence Motif Models Using Gibbs Samling BMI/CS 776 www.biostat.wisc.edu/bmi776/ Sring 2018 Anthony Gitter gitter@biostat.wisc.edu These slides excluding third-arty material are licensed under
More informationCoding Along Hermite Polynomials for Gaussian Noise Channels
Coding Along Hermite olynomials for Gaussian Noise Channels Emmanuel A. Abbe IG, EFL Lausanne, 1015 CH Email: emmanuel.abbe@efl.ch Lizhong Zheng LIDS, MIT Cambridge, MA 0139 Email: lizhong@mit.edu Abstract
More informationOn Fractional Predictive PID Controller Design Method Emmanuel Edet*. Reza Katebi.**
On Fractional Predictive PID Controller Design Method Emmanuel Edet*. Reza Katebi.** * echnology and Innovation Centre, Level 4, Deartment of Electronic and Electrical Engineering, University of Strathclyde,
More informationAdaptive estimation with change detection for streaming data
Adative estimation with change detection for streaming data A thesis resented for the degree of Doctor of Philosohy of the University of London and the Diloma of Imerial College by Dean Adam Bodenham Deartment
More informationStatics and dynamics: some elementary concepts
1 Statics and dynamics: some elementary concets Dynamics is the study of the movement through time of variables such as heartbeat, temerature, secies oulation, voltage, roduction, emloyment, rices and
More informationLearning Vector Quantization (LVQ)
Learning Vector Quantization (LVQ) Introduction to Neural Computation : Guest Lecture 2 John A. Bullinaria, 2007 1. The SOM Architecture and Algorithm 2. What is Vector Quantization? 3. The Encoder-Decoder
More informationThe non-stochastic multi-armed bandit problem
Submitted for journal ublication. The non-stochastic multi-armed bandit roblem Peter Auer Institute for Theoretical Comuter Science Graz University of Technology A-8010 Graz (Austria) auer@igi.tu-graz.ac.at
More informationCHAPTER-II Control Charts for Fraction Nonconforming using m-of-m Runs Rules
CHAPTER-II Control Charts for Fraction Nonconforming using m-of-m Runs Rules. Introduction: The is widely used in industry to monitor the number of fraction nonconforming units. A nonconforming unit is
More informationCryptanalysis of Pseudorandom Generators
CSE 206A: Lattice Algorithms and Alications Fall 2017 Crytanalysis of Pseudorandom Generators Instructor: Daniele Micciancio UCSD CSE As a motivating alication for the study of lattice in crytograhy we
More informationPositive decomposition of transfer functions with multiple poles
Positive decomosition of transfer functions with multile oles Béla Nagy 1, Máté Matolcsi 2, and Márta Szilvási 1 Deartment of Analysis, Technical University of Budaest (BME), H-1111, Budaest, Egry J. u.
More informationOutline. EECS150 - Digital Design Lecture 26 Error Correction Codes, Linear Feedback Shift Registers (LFSRs) Simple Error Detection Coding
Outline EECS150 - Digital Design Lecture 26 Error Correction Codes, Linear Feedback Shift Registers (LFSRs) Error detection using arity Hamming code for error detection/correction Linear Feedback Shift
More informationAn Algorithm of Two-Phase Learning for Eleman Neural Network to Avoid the Local Minima Problem
IJCSNS International Journal of Comuter Science and Networ Security, VOL.7 No.4, Aril 2007 1 An Algorithm of Two-Phase Learning for Eleman Neural Networ to Avoid the Local Minima Problem Zhiqiang Zhang,
More informationSpectral Analysis by Stationary Time Series Modeling
Chater 6 Sectral Analysis by Stationary Time Series Modeling Choosing a arametric model among all the existing models is by itself a difficult roblem. Generally, this is a riori information about the signal
More informationLecture 21: Quantum Communication
CS 880: Quantum Information Processing 0/6/00 Lecture : Quantum Communication Instructor: Dieter van Melkebeek Scribe: Mark Wellons Last lecture, we introduced the EPR airs which we will use in this lecture
More informationFor q 0; 1; : : : ; `? 1, we have m 0; 1; : : : ; q? 1. The set fh j(x) : j 0; 1; ; : : : ; `? 1g forms a basis for the tness functions dened on the i
Comuting with Haar Functions Sami Khuri Deartment of Mathematics and Comuter Science San Jose State University One Washington Square San Jose, CA 9519-0103, USA khuri@juiter.sjsu.edu Fax: (40)94-500 Keywords:
More informationTopic: Lower Bounds on Randomized Algorithms Date: September 22, 2004 Scribe: Srinath Sridhar
15-859(M): Randomized Algorithms Lecturer: Anuam Guta Toic: Lower Bounds on Randomized Algorithms Date: Setember 22, 2004 Scribe: Srinath Sridhar 4.1 Introduction In this lecture, we will first consider
More informationIntroduction to MVC. least common denominator of all non-identical-zero minors of all order of G(s). Example: The minor of order 2: 1 2 ( s 1)
Introduction to MVC Definition---Proerness and strictly roerness A system G(s) is roer if all its elements { gij ( s)} are roer, and strictly roer if all its elements are strictly roer. Definition---Causal
More informationFE FORMULATIONS FOR PLASTICITY
G These slides are designed based on the book: Finite Elements in Plasticity Theory and Practice, D.R.J. Owen and E. Hinton, 1970, Pineridge Press Ltd., Swansea, UK. 1 Course Content: A INTRODUCTION AND
More informationImproved Identification of Nonlinear Dynamic Systems using Artificial Immune System
Imroved Identification of Nonlinear Dnamic Sstems using Artificial Immune Sstem Satasai Jagannath Nanda, Ganaati Panda, Senior Member IEEE and Babita Majhi Deartment of Electronics and Communication Engineering,
More informationA Qualitative Event-based Approach to Multiple Fault Diagnosis in Continuous Systems using Structural Model Decomposition
A Qualitative Event-based Aroach to Multile Fault Diagnosis in Continuous Systems using Structural Model Decomosition Matthew J. Daigle a,,, Anibal Bregon b,, Xenofon Koutsoukos c, Gautam Biswas c, Belarmino
More informationLearning Vector Quantization
Learning Vector Quantization Neural Computation : Lecture 18 John A. Bullinaria, 2015 1. SOM Architecture and Algorithm 2. Vector Quantization 3. The Encoder-Decoder Model 4. Generalized Lloyd Algorithms
More informationA M,ETHOD OF MEASURING THE RESISTIVITY AND HALL' COEFFICIENT ON LAMELLAE OF ARBITRARY SHAPE
'. ' 220 HILlS TECHNICAL REVIEW VOLUME 20 A M,ETHOD OF MEASURING THE RESISTIVITY AND HALL' COEFFICIENT ON LAMELLAE OF ARBITRARY SHAE 621.317.331:538.632.083 Resistivity and Hall-coefficient measurements
More informationProof: We follow thearoach develoed in [4]. We adot a useful but non-intuitive notion of time; a bin with z balls at time t receives its next ball at
A Scaling Result for Exlosive Processes M. Mitzenmacher Λ J. Sencer We consider the following balls and bins model, as described in [, 4]. Balls are sequentially thrown into bins so that the robability
More informationComputer arithmetic. Intensive Computation. Annalisa Massini 2017/2018
Comuter arithmetic Intensive Comutation Annalisa Massini 7/8 Intensive Comutation - 7/8 References Comuter Architecture - A Quantitative Aroach Hennessy Patterson Aendix J Intensive Comutation - 7/8 3
More informationNeural network models for river flow forecasting
Neural network models for river flow forecasting Nguyen Tan Danh Huynh Ngoc Phien* and Ashim Das Guta Asian Institute of Technology PO Box 4 KlongLuang 220 Pathumthani Thailand Abstract In this study back
More informationPeriod-two cycles in a feedforward layered neural network model with symmetric sequence processing
PHYSICAL REVIEW E 75, 4197 27 Period-two cycles in a feedforward layered neural network model with symmetric sequence rocessing F. L. Metz and W. K. Theumann Instituto de Física, Universidade Federal do
More informationAKRON: An Algorithm for Approximating Sparse Kernel Reconstruction
: An Algorithm for Aroximating Sarse Kernel Reconstruction Gregory Ditzler Det. of Electrical and Comuter Engineering The University of Arizona Tucson, AZ 8572 USA ditzler@email.arizona.edu Nidhal Carla
More informationSHAPE OPTOMIZATION OF H-BEAM FLANGE FOR MAXIMUM PLASTIC ENERGY DISSIPATION
The Fourth China-Jaan-Korea Joint Symosium on Otimization of Structural and Mechanical Systems Kunming, Nov. 6-9, 2006, China SHAPE OPTOMIZATION OF H-BEAM FLANGE FOR MAXIMUM PLASTIC ENERGY DISSIPATION
More informationVariable Selection and Model Building
LINEAR REGRESSION ANALYSIS MODULE XIII Lecture - 38 Variable Selection and Model Building Dr. Shalabh Deartment of Mathematics and Statistics Indian Institute of Technology Kanur Evaluation of subset regression
More informationTopology Optimization of Three Dimensional Structures under Self-weight and Inertial Forces
6 th World Congresses of Structural and Multidiscilinary Otimization Rio de Janeiro, 30 May - 03 June 2005, Brazil Toology Otimization of Three Dimensional Structures under Self-weight and Inertial Forces
More informationFrequency-Weighted Robust Fault Reconstruction Using a Sliding Mode Observer
Frequency-Weighted Robust Fault Reconstruction Using a Sliding Mode Observer C.P. an + F. Crusca # M. Aldeen * + School of Engineering, Monash University Malaysia, 2 Jalan Kolej, Bandar Sunway, 4650 Petaling,
More informationA New Method of DDB Logical Structure Synthesis Using Distributed Tabu Search
A New Method of DDB Logical Structure Synthesis Using Distributed Tabu Search Eduard Babkin and Margarita Karunina 2, National Research University Higher School of Economics Det of nformation Systems and
More informationUsing a Computational Intelligence Hybrid Approach to Recognize the Faults of Variance Shifts for a Manufacturing Process
Journal of Industrial and Intelligent Information Vol. 4, No. 2, March 26 Using a Comutational Intelligence Hybrid Aroach to Recognize the Faults of Variance hifts for a Manufacturing Process Yuehjen E.
More informationThe analysis and representation of random signals
The analysis and reresentation of random signals Bruno TOÉSNI Bruno.Torresani@cmi.univ-mrs.fr B. Torrésani LTP Université de Provence.1/30 Outline 1. andom signals Introduction The Karhunen-Loève Basis
More informationGeneralized Coiflets: A New Family of Orthonormal Wavelets
Generalized Coiflets A New Family of Orthonormal Wavelets Dong Wei, Alan C Bovik, and Brian L Evans Laboratory for Image and Video Engineering Deartment of Electrical and Comuter Engineering The University
More informationConvolutional Codes. Lecture 13. Figure 93: Encoder for rate 1/2 constraint length 3 convolutional code.
Convolutional Codes Goals Lecture Be able to encode using a convolutional code Be able to decode a convolutional code received over a binary symmetric channel or an additive white Gaussian channel Convolutional
More informationAR PROCESSES AND SOURCES CAN BE RECONSTRUCTED FROM. Radu Balan, Alexander Jourjine, Justinian Rosca. Siemens Corporation Research
AR PROCESSES AND SOURCES CAN BE RECONSTRUCTED FROM DEGENERATE MIXTURES Radu Balan, Alexander Jourjine, Justinian Rosca Siemens Cororation Research 7 College Road East Princeton, NJ 8 fradu,jourjine,roscag@scr.siemens.com
More informationSTK4900/ Lecture 7. Program
STK4900/9900 - Lecture 7 Program 1. Logistic regression with one redictor 2. Maximum likelihood estimation 3. Logistic regression with several redictors 4. Deviance and likelihood ratio tests 5. A comment
More informationSTABILITY ANALYSIS TOOL FOR TUNING UNCONSTRAINED DECENTRALIZED MODEL PREDICTIVE CONTROLLERS
STABILITY ANALYSIS TOOL FOR TUNING UNCONSTRAINED DECENTRALIZED MODEL PREDICTIVE CONTROLLERS Massimo Vaccarini Sauro Longhi M. Reza Katebi D.I.I.G.A., Università Politecnica delle Marche, Ancona, Italy
More informationAnalysis of Group Coding of Multiple Amino Acids in Artificial Neural Network Applied to the Prediction of Protein Secondary Structure
Analysis of Grou Coding of Multile Amino Acids in Artificial Neural Networ Alied to the Prediction of Protein Secondary Structure Zhu Hong-ie 1, Dai Bin 2, Zhang Ya-feng 1, Bao Jia-li 3,* 1 College of
More informationOn parameter estimation in deformable models
Downloaded from orbitdtudk on: Dec 7, 07 On arameter estimation in deformable models Fisker, Rune; Carstensen, Jens Michael Published in: Proceedings of the 4th International Conference on Pattern Recognition
More informationFinding Shortest Hamiltonian Path is in P. Abstract
Finding Shortest Hamiltonian Path is in P Dhananay P. Mehendale Sir Parashurambhau College, Tilak Road, Pune, India bstract The roblem of finding shortest Hamiltonian ath in a eighted comlete grah belongs
More information