Staying on the path Fred Rothganger
|
|
|
- Andrea Hunter
- 5 years ago
- Views:
Transcription
1 Staying on the path Fred Rothganger Sandia National Laboratories is a multi-mission laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy s National Nuclear Security Administration under contract DE-AC04-94AL SAND NO XXXXP
2 Problems Neuro-inspired computing faces three key challenges: 1) Reducing energy per bit how to produce reliable results with unreliable hardware? 2) Creating algorithms to solve problems currently only solved by the human brain. 3) Porting algorithms between substantially different architectures. Computing with dynamical systems offers potential solutions to all three problems.
3 Nature computes with dynamical systems Biological systems respond to a problem by making internal changes to their dynamics. This can be called learning. Homeostat a biologically-inspired computer W. Ross Ashby, pioneer in cybernetics, 1947 New dynamics Old dynamics Homeostatic limit 3
4 Dynamical systems can solve numerical problems ff XX, pp = 1 pp ee XX + ppp XX = 0 Easy solutions Nonlinear systems Complex polynomial systems Eigenvalue problems Special hardware (xbar, spiking chip) could track the manifold. X 0 p 1 Hard solutions By construction, easy and hard solutions are always on the same manifold. Unlike an ODE, f always tells us distance to the manifold. 4
5 Eigenvalue problem A square matrix of arbitrary complex values D diagonal matrix of random values. These are initial eigenvalues λ, and columns of the identity matrix are the associated eigenvectors x. Definition of eigenpair (x,λ): AAAA = λλxx Homotopy formulation: ff xx, λλ, pp = 1 pp DD + ppaa xx λλxx 1 xx HH xx = 00 Jacobian: ff xx, λλ, pp = 1 pp DD + ppaa λλ xx AA DD xx 2xx TT 0 0 Gradient of f 2 : xx, λλ, pp = 2( ) TT ff Secant method for prediction. Gradient descent with backtracking line search for correction. 5
6 Digital computers are dynamical systems Imagine a trillion-dimensional hypercube (0,1,1) (1,1,1) (0,0,1) (1,0,1) The answer is another corner. The problem is some corner in this space. (0,0,0) (0,1,0) (1,0,0) (1,1,0) In between, the computer jumps from one corner to another based on its logic. 6
7 Leave the edges of the binary hypercube? (0,1,1) (1,1,1) (0,0,1) (1,0,1) (0,1,0) (1,1,0) (0,0,0) (1,0,0) 7
8 Backup 8
9 FlakyFloat Compiled CLAPACK with g++ Replaced FORTRAN REAL type with C++ class FlakyFloat Sign 31 Exponent 23 Mantissa 0 b bits damaged Damage methods: 1. Set to 0 2. Set to random value 9
10 LAPACK under degradation (zeros) 1E+1 1E+0 1E-1 1E-2 1E-3 1E-4 Epsilon Size 10 Size 100 Size E-5 1E-6 1E-7 1E
11 LAPACK vs. Continuation (zeros) 1E+0 1E-1 1E-2 1E-3 LAPACK 1E-4 Continuation 1E-5 1E
12 LAPACK vs. Continuation (random) 1E+1 1E+0 1E-1 1E-2 1E-3 LAPACK 1E-4 Continuation 1E-5 1E LAPACK crashed immediately in SLAMCH, so waited to turn on noise until after first call. 12
13 LAPACK zeros vs. random 1E+1 1E+0 1E-1 1E-2 1E-3 LAPACK zero 1E-4 LAPACK noise 1E-5 1E
14 Conclusion LAPACK is faster and more precise than continuation Possibly due to naïve implementation Continuation is more resilient to noise and loss of precision Next steps: Actually implement continuation as a dynamical system Small modification to gradient descent Attractor towards p=1 Suitable for memristor crossbar matrices are fixed, or in some cases linearly combined. Develop spiking version Test on hardware We are assembling a neuromorphic testbed with one of everything: TrueNorth (at LLNL), SpiNNaker, STPU, etc. 14
15 N2A A neural programming language Algorithm N A Network Model (LIF Neurons, Synapses with delay) Neural Compiler STPU Backend Internal Simulator C Backend Xyce Backend STPU FPGA hardware C++ compiler Netlist file Executable program Xyce 15
16 N2A Unified modeling framework Brain has Stem Cell Hippocampus has has Neuron Synapse Dentate Gyrus CA3 has has has is Progenitor Cell becomes Granule Cell is connects Mossy Terminal is connects is Pyramidal Cell Attributes Equations (a type of attribute) Name = Hippocampus CA3 pyramidal call Organism = Vertebrata Neurotransmitter released = Glutamate Dendrite Length = um V = (G * (V_rest - V) + I_inj) / C G =
Linear Regression, Neural Networks, etc.
Linear Regression, Neural Networks, etc. Gradient Descent Many machine learning problems can be cast as optimization problems Define a function that corresponds to learning error. (More on this later)
Neuron. Detector Model. Understanding Neural Components in Detector Model. Detector vs. Computer. Detector. Neuron. output. axon
Neuron Detector Model 1 The detector model. 2 Biological properties of the neuron. 3 The computational unit. Each neuron is detecting some set of conditions (e.g., smoke detector). Representation is what
Artificial Neural Networks. Q550: Models in Cognitive Science Lecture 5
Artificial Neural Networks Q550: Models in Cognitive Science Lecture 5 "Intelligence is 10 million rules." --Doug Lenat The human brain has about 100 billion neurons. With an estimated average of one thousand
3 Detector vs. Computer
1 Neurons 1. The detector model. Also keep in mind this material gets elaborated w/the simulations, and the earliest material is often hardest for those w/primarily psych background. 2. Biological properties
22c145-Fall 01: Neural Networks. Neural Networks. Readings: Chapter 19 of Russell & Norvig. Cesare Tinelli 1
Neural Networks Readings: Chapter 19 of Russell & Norvig. Cesare Tinelli 1 Brains as Computational Devices Brains advantages with respect to digital computers: Massively parallel Fault-tolerant Reliable
Neural networks. Chapter 19, Sections 1 5 1
Neural networks Chapter 19, Sections 1 5 Chapter 19, Sections 1 5 1 Outline Brains Neural networks Perceptrons Multilayer perceptrons Applications of neural networks Chapter 19, Sections 1 5 2 Brains 10
CS:4420 Artificial Intelligence
CS:4420 Artificial Intelligence Spring 2018 Neural Networks Cesare Tinelli The University of Iowa Copyright 2004 18, Cesare Tinelli and Stuart Russell a a These notes were originally developed by Stuart
COMP304 Introduction to Neural Networks based on slides by:
COMP34 Introduction to Neural Networks based on slides by: Christian Borgelt http://www.borgelt.net/ Christian Borgelt Introduction to Neural Networks Motivation: Why (Artificial) Neural Networks? (Neuro-)Biology
ARTIFICIAL NEURAL NETWORKS گروه مطالعاتي 17 بهار 92
ARTIFICIAL NEURAL NETWORKS گروه مطالعاتي 17 بهار 92 BIOLOGICAL INSPIRATIONS Some numbers The human brain contains about 10 billion nerve cells (neurons) Each neuron is connected to the others through 10000
(Feed-Forward) Neural Networks Dr. Hajira Jabeen, Prof. Jens Lehmann
(Feed-Forward) Neural Networks 2016-12-06 Dr. Hajira Jabeen, Prof. Jens Lehmann Outline In the previous lectures we have learned about tensors and factorization methods. RESCAL is a bilinear model for
A Study of Complex Deep Learning Networks on High Performance, Neuromorphic, and Quantum Computers
A Study of Complex Deep Learning Networks on High Performance, Neuromorphic, and Quantum Computers Thomas E. Potok, Ph.D. Computational Data Analytics Group Oak Ridge National Laboratory ORNL is managed
Neural networks. Chapter 20. Chapter 20 1
Neural networks Chapter 20 Chapter 20 1 Outline Brains Neural networks Perceptrons Multilayer networks Applications of neural networks Chapter 20 2 Brains 10 11 neurons of > 20 types, 10 14 synapses, 1ms
Magnetic tunnel junction beyond memory from logic to neuromorphic computing WANJUN PARK DEPT. OF ELECTRONIC ENGINEERING, HANYANG UNIVERSITY
Magnetic tunnel junction beyond memory from logic to neuromorphic computing WANJUN PARK DEPT. OF ELECTRONIC ENGINEERING, HANYANG UNIVERSITY Magnetic Tunnel Junctions (MTJs) Structure High density memory
Fitting a Tensor Decomposition is a Nonlinear Optimization Problem
Fitting a Tensor Decomposition is a Nonlinear Optimization Problem Evrim Acar, Daniel M. Dunlavy, and Tamara G. Kolda* Sandia National Laboratories Sandia is a multiprogram laboratory operated by Sandia
NEUROMORPHIC COMPUTING WITH MAGNETO-METALLIC NEURONS & SYNAPSES: PROSPECTS AND PERSPECTIVES
NEUROMORPHIC COMPUTING WITH MAGNETO-METALLIC NEURONS & SYNAPSES: PROSPECTS AND PERSPECTIVES KAUSHIK ROY ABHRONIL SENGUPTA, KARTHIK YOGENDRA, DELIANG FAN, SYED SARWAR, PRIYA PANDA, GOPAL SRINIVASAN, JASON
Post Von Neumann Computing
Post Von Neumann Computing Matthias Kaiserswerth Hasler Stiftung (formerly IBM Research) 1 2014 IBM Corporation Foundation Purpose Support information and communication technologies (ICT) to advance Switzerland
On one Application of Newton s Method to Stability Problem
Journal of Multidciplinary Engineering Science Technology (JMEST) ISSN: 359-0040 Vol. Issue 5, December - 204 On one Application of Newton s Method to Stability Problem Şerife Yılmaz Department of Mathematics,
Scalability of Partial Differential Equations Preconditioner Resilient to Soft and Hard Faults
Scalability of Partial Differential Equations Preconditioner Resilient to Soft and Hard Faults K.Morris, F.Rizzi, K.Sargsyan, K.Dahlgren, P.Mycek, C.Safta, O.LeMaitre, O.Knio, B.Debusschere Sandia National
, b = 0. (2) 1 2 The eigenvectors of A corresponding to the eigenvalues λ 1 = 1, λ 2 = 3 are
Quadratic forms We consider the quadratic function f : R 2 R defined by f(x) = 2 xt Ax b T x with x = (x, x 2 ) T, () where A R 2 2 is symmetric and b R 2. We will see that, depending on the eigenvalues
Linear discriminant functions
Andrea Passerini passerini@disi.unitn.it Machine Learning Discriminative learning Discriminative vs generative Generative learning assumes knowledge of the distribution governing the data Discriminative
Artifical Neural Networks
Neural Networks Artifical Neural Networks Neural Networks Biological Neural Networks.................................. Artificial Neural Networks................................... 3 ANN Structure...........................................
Artificial Neural Networks" and Nonparametric Methods" CMPSCI 383 Nov 17, 2011!
Artificial Neural Networks" and Nonparametric Methods" CMPSCI 383 Nov 17, 2011! 1 Todayʼs lecture" How the brain works (!)! Artificial neural networks! Perceptrons! Multilayer feed-forward networks! Error
New Delhi & Affiliated to VTU, Belgavi ) Oorgaum, Kolar Dist ,Karnataka
Design and Implementation of Logic Gates and Adder Circuits on FPGA Using ANN Neelu Farha 1., Ann Louisa Paul J 2., Naadiya Kousar L S 3., Devika S 4., Prof. Ruckmani Divakaran 5 1,2,3,4,5 Department of
Adiabatic quantum computation a tutorial for computer scientists
Adiabatic quantum computation a tutorial for computer scientists Itay Hen Dept. of Physics, UCSC Advanced Machine Learning class UCSC June 6 th 2012 Outline introduction I: what is a quantum computer?
Push-pull PDV analysis
Push-pull PDV analysis Sub-fringe data reduction PDV workshop August 6-7, 27 Daniel H. Dolan and Scott Jones Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,
Review of Linear Algebra
Review of Linear Algebra Definitions An m n (read "m by n") matrix, is a rectangular array of entries, where m is the number of rows and n the number of columns. 2 Definitions (Con t) A is square if m=
Balance of Electric and Diffusion Forces
Balance of Electric and Diffusion Forces Ions flow into and out of the neuron under the forces of electricity and concentration gradients (diffusion). The net result is a electric potential difference
EE04 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb 23, Sasidharan Sreedharan
EE04 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb 23, 2012 Sasidharan Sreedharan www.sasidharan.webs.com 3/1/2012 1 Syllabus Artificial Intelligence Systems- Neural Networks, fuzzy logic,
Quantum Hall Effect in Vanishing Magnetic Fields
Quantum Hall Effect in Vanishing Magnetic Fields Wei Pan Sandia National Labs Sandia is a multi-mission laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department
Compartmental Modelling
Modelling Neurons Computing and the Brain Compartmental Modelling Spring 2010 2 1 Equivalent Electrical Circuit A patch of membrane is equivalent to an electrical circuit This circuit can be described
Math 411 Preliminaries
Math 411 Preliminaries Provide a list of preliminary vocabulary and concepts Preliminary Basic Netwon s method, Taylor series expansion (for single and multiple variables), Eigenvalue, Eigenvector, Vector
Nonlinearity of Joints in Structural Dynamics of Weapons Systems
Nonlinearity of Joints in Structural Dynamics of Weapons Systems Daniel Brown AWE Dan Segalman Sandia National Laboratories Sandia National Laboratories is a multi-program laboratory managed and operated
Assignment # 4. K-WTA neural networks are relatively simple. N neurons, with all the connection strengths equal to -1 (with no self-connections).
Farrukh Jabeen COM 58 Mathematical Methods In AI Assignment # 4 This problem asks you to analyze a K-WTA network (K winner take all). First read the K-WTA reference article, then answer the questions below.
Marr's Theory of the Hippocampus: Part I
Marr's Theory of the Hippocampus: Part I Computational Models of Neural Systems Lecture 3.3 David S. Touretzky October, 2015 David Marr: 1945-1980 10/05/15 Computational Models of Neural Systems 2 Marr
Large Scale Data Analysis Using Deep Learning
Large Scale Data Analysis Using Deep Learning Linear Algebra U Kang Seoul National University U Kang 1 In This Lecture Overview of linear algebra (but, not a comprehensive survey) Focused on the subset
2015 Todd Neller. A.I.M.A. text figures 1995 Prentice Hall. Used by permission. Neural Networks. Todd W. Neller
2015 Todd Neller. A.I.M.A. text figures 1995 Prentice Hall. Used by permission. Neural Networks Todd W. Neller Machine Learning Learning is such an important part of what we consider "intelligence" that
Spiking Neural P Systems with Anti-Spikes as Transducers
ROMANIAN JOURNAL OF INFORMATION SCIENCE AND TECHNOLOGY Volume 14, Number 1, 2011, 20 30 Spiking Neural P Systems with Anti-Spikes as Transducers Venkata Padmavati METTA 1, Kamala KRITHIVASAN 2, Deepak
Lecture 8: Sequential Networks and Finite State Machines
Lecture 8: Sequential Networks and Finite State Machines CSE 140: Components and Design Techniques for Digital Systems Spring 2014 CK Cheng, Diba Mirza Dept. of Computer Science and Engineering University
Neural networks. Chapter 20, Section 5 1
Neural networks Chapter 20, Section 5 Chapter 20, Section 5 Outline Brains Neural networks Perceptrons Multilayer perceptrons Applications of neural networks Chapter 20, Section 5 2 Brains 0 neurons of
Part 8: Neural Networks
METU Informatics Institute Min720 Pattern Classification ith Bio-Medical Applications Part 8: Neural Netors - INTRODUCTION: BIOLOGICAL VS. ARTIFICIAL Biological Neural Netors A Neuron: - A nerve cell as
Neural Networks. Chapter 18, Section 7. TB Artificial Intelligence. Slides from AIMA 1/ 21
Neural Networks Chapter 8, Section 7 TB Artificial Intelligence Slides from AIMA http://aima.cs.berkeley.edu / 2 Outline Brains Neural networks Perceptrons Multilayer perceptrons Applications of neural
Multilayer Perceptron
Outline Hong Chang Institute of Computing Technology, Chinese Academy of Sciences Machine Learning Methods (Fall 2012) Outline Outline I 1 Introduction 2 Single Perceptron 3 Boolean Function Learning 4
(a) II and III (b) I (c) I and III (d) I and II and III (e) None are true.
1 Which of the following statements is always true? I The null space of an m n matrix is a subspace of R m II If the set B = {v 1,, v n } spans a vector space V and dimv = n, then B is a basis for V III
Neuromorphic computing with Memristive devices. NCM group
Neuromorphic computing with Memristive devices NCM group Why neuromorphic? New needs for computing Recognition, Mining, Synthesis (Intel) Increase of Fault (nanoscale engineering) SEMICONDUCTOR TECHNOLOGY
The Perceptron. Volker Tresp Summer 2014
The Perceptron Volker Tresp Summer 2014 1 Introduction One of the first serious learning machines Most important elements in learning tasks Collection and preprocessing of training data Definition of a
Neural networks: Associative memory
Neural networs: Associative memory Prof. Sven Lončarić sven.loncaric@fer.hr http://www.fer.hr/ipg 1 Overview of topics l Introduction l Associative memories l Correlation matrix as an associative memory
An overview of the. package for non-conformal mesh tying or simple contact problems
An overview of the package for non-conformal mesh tying or simple contact problems Glen Hansen 2012 Trilinos User Group Meeting Oct. 30, 2012 SAND 2012-9202P (UUR) Sandia National Laboratories is a multi-program
Parallel Simulation of Subsurface Fluid Flow
Parallel Simulation of Subsurface Fluid Flow Scientific Achievement A new mortar domain decomposition method was devised to compute accurate velocities of underground fluids efficiently using massively
1. A discrete-time recurrent network is described by the following equation: y(n + 1) = A y(n) + B x(n)
Neuro-Fuzzy, Revision questions June, 25. A discrete-time recurrent network is described by the following equation: y(n + ) = A y(n) + B x(n) where A =.7.5.4.6, B = 2 (a) Sketch the dendritic and signal-flow
A = 3 1. We conclude that the algebraic multiplicity of the eigenvalues are both one, that is,
65 Diagonalizable Matrices It is useful to introduce few more concepts, that are common in the literature Definition 65 The characteristic polynomial of an n n matrix A is the function p(λ) det(a λi) Example
Chapter 1. Introduction
Chapter 1 Introduction Symbolical artificial intelligence is a field of computer science that is highly related to quantum computation. At first glance, this statement appears to be a contradiction. However,
Applications of Memristors in ANNs
Applications of Memristors in ANNs Outline Brief intro to ANNs Firing rate networks Single layer perceptron experiment Other (simulation) examples Spiking networks and STDP ANNs ANN is bio inpsired inpsired
Section 4.1 The Power Method
Section 4.1 The Power Method Key terms Dominant eigenvalue Eigenpair Infinity norm and 2-norm Power Method Scaled Power Method Power Method for symmetric matrices The notation used varies a bit from that
RE-ENGINEERING COMPUTING WITH NEURO- MIMETIC DEVICES, CIRCUITS, AND ALGORITHMS
RE-ENGINEERING COMPUTING WITH NEURO- MIMETIC DEVICES, CIRCUITS, AND ALGORITHMS Kaushik Roy Abhronil Sengupta, Gopal Srinivasan, Aayush Ankit, Priya Panda, Xuanyao Fong, Deliang Fan, Jason Allred School
Uncertainty Quantification for Machine Learning and Statistical Models
Uncertainty Quantification for Machine Learning and Statistical Models David J. Stracuzzi Joint work with: Max Chen, Michael Darling, Stephen Dauphin, Matt Peterson, and Chris Young Sandia National Laboratories
A Framework for Modeling and Optimizing Dynamic Systems Under Uncertainty
A Framework for Modeling and Optimizing Dynamic Systems Under Uncertainty Bethany Nicholson John D. Siirola Center for Computing Research Sandia National Laboratories Albuquerque, NM USA FOCAPO January
Lecture 5: Recurrent Neural Networks
1/25 Lecture 5: Recurrent Neural Networks Nima Mohajerin University of Waterloo WAVE Lab nima.mohajerin@uwaterloo.ca July 4, 2017 2/25 Overview 1 Recap 2 RNN Architectures for Learning Long Term Dependencies
Perceptron. (c) Marcin Sydow. Summary. Perceptron
Topics covered by this lecture: Neuron and its properties Mathematical model of neuron: as a classier ' Learning Rule (Delta Rule) Neuron Human neural system has been a natural source of inspiration for
CHAPTER 11. A Revision. 1. The Computers and Numbers therein
CHAPTER A Revision. The Computers and Numbers therein Traditional computer science begins with a finite alphabet. By stringing elements of the alphabet one after another, one obtains strings. A set of
Multiphysics modeling of thermal batteries
Multiphysics modeling of thermal batteries Scott A. Roberts, Ph.D. Thermal/Fluid Component Sciences Department Sandia National Laboratories, Albuquerque, NM The Future of Munitions Batteries Workshop Army
Repeated Eigenvalues and Symmetric Matrices
Repeated Eigenvalues and Symmetric Matrices. Introduction In this Section we further develop the theory of eigenvalues and eigenvectors in two distinct directions. Firstly we look at matrices where one
CN2 1: Introduction. Paul Gribble. Sep 10,
CN2 1: Introduction Paul Gribble http://gribblelab.org Sep 10, 2012 Administrivia Class meets Mondays, 2:00pm - 3:30pm and Thursdays, 11:30am - 1:00pm, in NSC 245A Contact me with any questions or to set
Neural Networks 1 Synchronization in Spiking Neural Networks
CS 790R Seminar Modeling & Simulation Neural Networks 1 Synchronization in Spiking Neural Networks René Doursat Department of Computer Science & Engineering University of Nevada, Reno Spring 2006 Synchronization
Machine Learning and Data Mining. Multi-layer Perceptrons & Neural Networks: Basics. Prof. Alexander Ihler
+ Machine Learning and Data Mining Multi-layer Perceptrons & Neural Networks: Basics Prof. Alexander Ihler Linear Classifiers (Perceptrons) Linear Classifiers a linear classifier is a mapping which partitions
Lecture 6. Notes on Linear Algebra. Perceptron
Lecture 6. Notes on Linear Algebra. Perceptron COMP90051 Statistical Machine Learning Semester 2, 2017 Lecturer: Andrey Kan Copyright: University of Melbourne This lecture Notes on linear algebra Vectors
What s New in Isorropia?
What s New in Isorropia? Erik Boman Cedric Chevalier, Lee Ann Riesen Sandia National Laboratories, NM, USA Trilinos User s Group, Nov 4-5, 2009. SAND 2009-7611P. Sandia is a multiprogram laboratory operated
New Quantum Transport Results in Type-II InAs/GaSb Quantum Wells
New Quantum Transport Results in Type-II InAs/GaSb Quantum Wells Wei Pan Sandia National Laboratories Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation,
Chapter 9: The Perceptron
Chapter 9: The Perceptron 9.1 INTRODUCTION At this point in the book, we have completed all of the exercises that we are going to do with the James program. These exercises have shown that distributed
Simple Neural Nets For Pattern Classification
CHAPTER 2 Simple Neural Nets For Pattern Classification Neural Networks General Discussion One of the simplest tasks that neural nets can be trained to perform is pattern classification. In pattern classification
The N3XT Technology for. Brain-Inspired Computing
The N3XT Technology for Brain-Inspired Computing SystemX Alliance 27..8 Department of Electrical Engineering 25.4.5 2 25.4.5 Source: Google 3 25.4.5 Source: vrworld.com 4 25.4.5 Source: BDC Stanford Magazine
An Empirical Comparison of Graph Laplacian Solvers
An Empirical Comparison of Graph Laplacian Solvers Kevin Deweese 1 Erik Boman 2 John Gilbert 1 1 Department of Computer Science University of California, Santa Barbara 2 Scalable Algorithms Department
Hydrogeology of Deep Borehole Disposal for High-Level Radioactive Waste
SAND2014-18615C Hydrogeology of Deep Borehole Disposal for High-Level Radioactive Waste Geological Society of America Annual Meeting October 20, 2014 Bill W. Arnold, W. Payton Gardner, and Patrick V. Brady
Sample questions for Fundamentals of Machine Learning 2018
Sample questions for Fundamentals of Machine Learning 2018 Teacher: Mohammad Emtiyaz Khan A few important informations: In the final exam, no electronic devices are allowed except a calculator. Make sure
Neural Networks Introduction CIS 32
Neural Networks Introduction CIS 32 Functionalia Office Hours (Last Change!) - Location Moved to 0317 N (Bridges Room) Today: Alpha-Beta Example Neural Networks Learning with T-R Agent (from before) direction
The Perceptron. Volker Tresp Summer 2016
The Perceptron Volker Tresp Summer 2016 1 Elements in Learning Tasks Collection, cleaning and preprocessing of training data Definition of a class of learning models. Often defined by the free model parameters
22.3. Repeated Eigenvalues and Symmetric Matrices. Introduction. Prerequisites. Learning Outcomes
Repeated Eigenvalues and Symmetric Matrices. Introduction In this Section we further develop the theory of eigenvalues and eigenvectors in two distinct directions. Firstly we look at matrices where one
Discriminative Direction for Kernel Classifiers
Discriminative Direction for Kernel Classifiers Polina Golland Artificial Intelligence Lab Massachusetts Institute of Technology Cambridge, MA 02139 polina@ai.mit.edu Abstract In many scientific and engineering
Modeling and Simulation with ODE for MSE
Zentrum Mathematik Technische Universität München Prof. Dr. Massimo Fornasier WS 6/7 Dr. Markus Hansen Sheet 7 Modeling and Simulation with ODE for MSE The exercises can be handed in until Wed, 4..6,.
Lecture 5: Logistic Regression. Neural Networks
Lecture 5: Logistic Regression. Neural Networks Logistic regression Comparison with generative models Feed-forward neural networks Backpropagation Tricks for training neural networks COMP-652, Lecture
Variations. ECE 6540, Lecture 02 Multivariate Random Variables & Linear Algebra
Variations ECE 6540, Lecture 02 Multivariate Random Variables & Linear Algebra Last Time Probability Density Functions Normal Distribution Expectation / Expectation of a function Independence Uncorrelated
Synaptic Devices and Neuron Circuits for Neuron-Inspired NanoElectronics
Synaptic Devices and Neuron Circuits for Neuron-Inspired NanoElectronics Byung-Gook Park Inter-university Semiconductor Research Center & Department of Electrical and Computer Engineering Seoul National
ECE 407 Computer Aided Design for Electronic Systems. Simulation. Instructor: Maria K. Michael. Overview
407 Computer Aided Design for Electronic Systems Simulation Instructor: Maria K. Michael Overview What is simulation? Design verification Modeling Levels Modeling circuits for simulation True-value simulation
Machine Learning. Neural Networks. (slides from Domingos, Pardo, others)
Machine Learning Neural Networks (slides from Domingos, Pardo, others) For this week, Reading Chapter 4: Neural Networks (Mitchell, 1997) See Canvas For subsequent weeks: Scaling Learning Algorithms toward
Neural Networks Introduction
Neural Networks Introduction H.A Talebi Farzaneh Abdollahi Department of Electrical Engineering Amirkabir University of Technology Winter 2011 H. A. Talebi, Farzaneh Abdollahi Neural Networks 1/22 Biological
Lecture 7 Artificial neural networks: Supervised learning
Lecture 7 Artificial neural networks: Supervised learning Introduction, or how the brain works The neuron as a simple computing element The perceptron Multilayer neural networks Accelerated learning in
Last update: October 26, Neural networks. CMSC 421: Section Dana Nau
Last update: October 26, 207 Neural networks CMSC 42: Section 8.7 Dana Nau Outline Applications of neural networks Brains Neural network units Perceptrons Multilayer perceptrons 2 Example Applications
Phase Response Curves, Delays and Synchronization in Matlab
Phase Response Curves, Delays and Synchronization in Matlab W. Govaerts and B. Sautois Department of Applied Mathematics and Computer Science, Ghent University, Krijgslaan 281-S9, B-9000 Ghent, Belgium
Iterative Reweighted Least Squares
Iterative Reweighted Least Squares Sargur. University at Buffalo, State University of ew York USA Topics in Linear Classification using Probabilistic Discriminative Models Generative vs Discriminative
Hopfield Neural Network and Associative Memory. Typical Myelinated Vertebrate Motoneuron (Wikipedia) Topic 3 Polymers and Neurons Lecture 5
Hopfield Neural Network and Associative Memory Typical Myelinated Vertebrate Motoneuron (Wikipedia) PHY 411-506 Computational Physics 2 1 Wednesday, March 5 1906 Nobel Prize in Physiology or Medicine.
Eigenspaces. (c) Find the algebraic multiplicity and the geometric multiplicity for the eigenvaules of A.
Eigenspaces 1. (a) Find all eigenvalues and eigenvectors of A = (b) Find the corresponding eigenspaces. [ ] 1 1 1 Definition. If A is an n n matrix and λ is a scalar, the λ-eigenspace of A (usually denoted
Homework 1. Part(a) Due: 15 Mar, 2018, 11:55pm
ENGG1203: Introduction to Electrical and Electronic Engineering Second Semester, 2017 18 Homework 1 Due: 15 Mar, 2018, 11:55pm Instruction: Submit your answers electronically through Moodle. In Moodle,
EE263: Introduction to Linear Dynamical Systems Review Session 5
EE263: Introduction to Linear Dynamical Systems Review Session 5 Outline eigenvalues and eigenvectors diagonalization matrix exponential EE263 RS5 1 Eigenvalues and eigenvectors we say that λ C is an eigenvalue
Multilayer Perceptrons (MLPs)
CSE 5526: Introduction to Neural Networks Multilayer Perceptrons (MLPs) 1 Motivation Multilayer networks are more powerful than singlelayer nets Example: XOR problem x 2 1 AND x o x 1 x 2 +1-1 o x x 1-1
Parallel Tensor Compression for Large-Scale Scientific Data
Parallel Tensor Compression for Large-Scale Scientific Data Woody Austin, Grey Ballard, Tamara G. Kolda April 14, 2016 SIAM Conference on Parallel Processing for Scientific Computing MS 44/52: Parallel
AN INTRODUCTION TO NEURAL NETWORKS. Scott Kuindersma November 12, 2009
AN INTRODUCTION TO NEURAL NETWORKS Scott Kuindersma November 12, 2009 SUPERVISED LEARNING We are given some training data: We must learn a function If y is discrete, we call it classification If it is
Dynamical Representation
Dynamical systems Dynamical Representation I Dynamical systems originated (Newton, Leibnitz) to describe the motion of objects. I But today, they are widely employed to understand (and model) an enormous
TYPES OF MODEL COMPRESSION. Soham Saha, MS by Research in CSE, CVIT, IIIT Hyderabad
TYPES OF MODEL COMPRESSION Soham Saha, MS by Research in CSE, CVIT, IIIT Hyderabad 1. Pruning 2. Quantization 3. Architectural Modifications PRUNING WHY PRUNING? Deep Neural Networks have redundant parameters.
Backpropagation and Neural Networks part 1. Lecture 4-1
Lecture 4: Backpropagation and Neural Networks part 1 Lecture 4-1 Administrative A1 is due Jan 20 (Wednesday). ~150 hours left Warning: Jan 18 (Monday) is Holiday (no class/office hours) Also note: Lectures
Addressing Challenges in Neuromorphic Computing with Memristive Synapses
Addressing Challenges in Neuromorphic Computing with Memristive Synapses Vishal Saxena 1, Xinyu Wu 1 and Maria Mitkova 2 1 Analog Mixed-Signal and Photonic IC (AMPIC) Lab 2 Nanoionic Materials and Devices
Stochastic Networks Variations of the Hopfield model
4 Stochastic Networks 4. Variations of the Hopfield model In the previous chapter we showed that Hopfield networks can be used to provide solutions to combinatorial problems that can be expressed as the