Addressing Challenges in Neuromorphic Computing with Memristive Synapses
|
|
- Samuel Jenkins
- 6 years ago
- Views:
Transcription
1 Addressing Challenges in Neuromorphic Computing with Memristive Synapses Vishal Saxena 1, Xinyu Wu 1 and Maria Mitkova 2 1 Analog Mixed-Signal and Photonic IC (AMPIC) Lab 2 Nanoionic Materials and Devices Lab ECE Department, Boise State University, ID ORNL, 2012 June Boise 30, State 2016 University 1
2 Agenda Introduction Synapse Devices and Challenges Mixed-Signal Neural Learning Architecture Realizing Multibit Synapses Conclusion 2
3 Reinventing Computing Von Neumann computing in the nano-cmos regime + Massive transistor count, but. Device variability Dark silicon issue Not well suited for processing massively parallel data streams and pattern recognition tasks that come easy to a brain. US Office of Science and Technology (OSTP) RFI, June 2015: Create a new type of computer that can proactively interpret and learn from data, solve unfamiliar problems using what it has learned, and operate with the energy efficiency of the human brain DOE, Brain Initiative, AFOSR, DARPA, SRC 3
4 Brain-Inspired Neuromorphic System Our Current Research Focus Source: DARPA SyNAPSE Program 4
5 Revisiting Biological Inspiration A B C V spk,in Synapse Inhibitive Connection V spk, out Neuron t 1 t 2 t 3 Synapse Dendrites From Previous Neurons Neuron V mem V thr Axon V mem Neuron V spk,in V mem Soma Pre Post Δt<0 Δw Pre Post Δt>0 0 Δt Δt = t post t pre Synaptic placiticity Synapse V spk,in V spk, out V spk, out t fire Emerging understanding of neural computation from Computational Neuroscience Hierarchical spiking neural networks with localized learning Synapses strengthen or weaken by relative firing times of the pre- and postneurons Plasticity rules: Spike-Timing Dependent Plasticity (STDP) 5
6 Neuromorphic Computing with RRAMs WTA bus Output Layer Forward spike Backward spike Pre-synaptic neuron ReRAM synapse Post-synaptic neuron Input Layer WTA bus To achieve biology-like synapse density, emerging nanoscale devices are under consideration Resistive RAM (RRAM) or Memristors Chalcogenide-based Conductive Bridge RAM (CBRAM) devices from Mitkova Group at Boise State Three necessary criterions for the realization of neuromorphic hardware capable of deep learning: (1) non-volatility of the trained weights, (2) massive synaptic density, and (3) ultra-low-power operation. 6
7 CBRAM I-V Characteristics I-V Sweep Resistance 7
8 Stochastic CBRAM Behavior V TH_Set Distributions R ON Distributions The CBRAM switching behavior is stochastic The analog synapse value depends upon the electroforming process Weak bridge-> more intermediate resistance values The analog resistance value relaxes in few minutes to hours 8
9 Spike Timing Dependent Plasticity (STDP) 0.2 A V pre V post Δt>0 t V pre V post Δt<0 t B ΔGmr (µs) 0.1 Pre 0 Post Δt<0-0.1 Δw or ΔG mr Pre Post Δt>0 V net =V post -V pre t V net =V post -V pre t Δt Δt (µs) Δt = t post t pre v p v n t v p v n t t pre t post t pre t post Source: X. Wu, V. Saxena and K. Zhu, Homogeneous Spiking Neuromorphic System for Real-World Pattern Recognition, IEEE Journal on Emerging and Selected Topics in Circuits and Systems, Accepted. 9
10 CBRAM Synapse Challenges Variability: Device characteristics such as the switching threshold voltages are variable from device to device Switching characteristics depend upon the initial forming step and compliance current Resistive Load: Thousands of devices in parallel present a difficult (low resistance) load to drive by the neuron circuit Resolution: Challenging to realize long-term persistence of weights for more than 1-bit resolution in CBRAM devices 10
11 CMOS Integrate and Fire Neuron V thr V den C mem Spike Generator V leaky M leaky SW 2 SW 1 Ф d =0 A v Ф d =1 1 V mem Ф 1 Ф 2 V refr V rest Ф f =0 1 SW 3 Ф d V axon Ф i Phase Ф f Bus V tch Controller Interface Ф c V cpr V mode V rst Asynchronous integrated and fire operation Accommodates RRAM Provides current summing node Sustain a fixed voltage for synapses Dynamic biasing for large current drive STDP compatible STDP-compatible spike generator In-situ STDP learning Winner-takes-all bus for effective local decision making Single contact point to a synapse WTA Bus V wtab Source: X. Wu, V. Saxena and K. Zhu, Homogeneous Spiking Neuromorphic System for Real-World Pattern Recognition, IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 5(3),
12 Novel Tri-mode Operation A Leaky Integrator B Voltage Buffer C Discharging V leaky V refr V rest V leaky V refr V rest V leaky V refr V rest V refr V refr M leaky Фd=1 V refr A v V mem V thr M leaky Фd=1 A v M leaky Фd=1 V refr A v V thr C mem SW 2 SW 1 1 Фd=0 Фf=0 1 SW 3 V axon V thr C mem SW 2 SW 1 Фd=1 0 0 Фf=1 SW 3 V axon V thr C mem SW 2 SW 1 Фd=1 0 Фf=0 1 SW 3 V axon V den V refr A v V mem Vcpr Фd V den V refr A v V mem Vcpr Фd V den V refr A v V mem Vcpr Фd Spike Generator Фi Фf Phase Bus V tch Ф1 Controller Interface V mode Ф2 V rst Фc Spike Generator Фi Фf Phase Bus V tch Ф1 Controller Interface V mode Ф2 V rst Фc Spike Generator Фi Фf Phase Bus V tch Ф1 Controller Interface V mode Ф2 V rst Фc V wtab V wtab V wtab A first CMOS neuron that is compatible with RRAMs and drives a large load Several resistive devices in parallel load the neuron Source: X. Wu, V. Saxena and K. Zhu, Homogeneous Spiking Neuromorphic System for Real-World Pattern Recognition, IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 5(3),
13 Simulation: Output Spikes Output Spikes Parameter Symbol Unit Min Max Step Positive amplitude α + mv α + τ + Negative amplitude α - mv Positive pulse width τ + ns Negative pulse width τ - ns Negative pulse slope s mv/ns α - slope τ - Source: X. Wu, V. Saxena and K. Zhu, A CMOS Spiking Neuron for Dense Memristor-Synapse Connectivity for Brain-Inspired Computing, International Joint Conference on Neural Networks (IJCNN),
14 Test Chip interfaced with CBRAM Devices A B Pre-spike V pre Post-spike V post V a+ tail + +V t,p Δt Over threshold potential created by STDP spike-pair V a- tail - -V t,m Net potential across synapse V net = V post - V pre Mixed-signal IC with 180-nm CMOS Neurons connected with CBRAM Synapses A first RRAM compatible STDP learning Neuron Power consumed in the neuron is 9.3pJ/spike/synapse (in terms of 10,000 synapses each having around 1MΩ resistance). Source: X. Wu, V. Saxena, K. Zhu and S. Balagopal, A CMOS Spiking Neuron for Brain-Inspired Neural Networks With Resistive Synapses and In Situ Learning, IEEE TCAS-II, 62(11), pp
15 Associative Learning Experiment Pre- Neuron 1 t 1 Pre-spike Neurons Output Layer IFN #1 IFN #2 Pre- Neuron 2 Resistive Synapses t 1 +Δt Post-Neuron IFN #3 Post-spike Input Layer t 2 Pre-Neuron 1(IFN #1) "Sight of Food" Pre-Neuron 2 (IFN #2) "Sound of Bell" Post-Neuron (IFN #3) "Salivation" Time (ms) Source: X. Wu, V. Saxena, K. Zhu and S. Balagopal, A CMOS Spiking Neuron for Brain-Inspired Neural Networks With Resistive Synapses and In Situ Learning, IEEE TCAS-II, 62(11), pp
16 Total Current (ua) Current (µa) Dynamic Resistive Load Driving 97% driving 10,000 x 1MΩ synapses , Synapses Driving Baseline ,000 1, Number of Synapses 10,000 Synapses 1,000 Synapses 100 Synapses 0 56µA 13µA Time (us) Source: X. Wu, V. Saxena and K. Zhu, Homogeneous Spiking Neuromorphic System for Real-World Pattern Recognition, IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 5(3),
17 Spike-based Pattern Recognition Circuit... WTA Image i j Input Network i j Input Neuron Matrix i j n Synaptic Network n Output Neurons Source: X. Wu, V. Saxena and K. Zhu, Homogeneous Spiking Neuromorphic System for Real-World Pattern Recognition, IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 5(3), 2015, in press. 17
18 Asynchronous WTA Bus Interface V leaky V refr V rest V thr Inhibitory connection Ф Excitatory connection d =1 C mem M leaky SW 2 1 SW 1 Ф d =0 Ф f =0 1 SW 3 V axon V den A v V mem V cpr Ф d Explicit one-one inhibitory connections Inhibitory connection Excitatory connection Spike Generator Implicit inhibition via shared WTA bus WTA Bus Ф i Phase Ф f Bus V tch Controller Interface Ф 1 Ф 2 Ф c V wtab V mode V rst V dd V mode Tri-state Buffer Delay V wtab Φ d Implicit inhibition via shared WTA bus V cpr V tch V rst Φ c D S Q Φ f Source: X. Wu, V. Saxena and K. Zhu, Homogeneous Spiking Neuromorphic System for Real-World Pattern Recognition, IEEE Journal on Emerging and Selected Topics in Circuits and Systems, Accepted. 18
19 Neuron # Neuron # Neuron # Demo: Handwrite Digits Classification First 20 Train (original sequence) Test (class-by-class) Input Spike Trains Output Spike Trains without Learning Output Spikes Train with Learning Conductance Evolution in Synapses (µs) Time (µs) Time (µs) Time (µs) Time (µs) 10 0 Source: X. Wu, V. Saxena and K. Zhu, Homogeneous Spiking Neuromorphic System for Real-World Pattern Recognition, IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 5(3),
20 Parallel Multibit Synapses Emerging spike-based deep learning architectures require at least 4-bits of synaptic resolution. The devices fundamentally have bistable nonvolatile states with stochastic switching behavior Recent approach: Use several of these device in parallel Potentially multiple resistance states due to variation in individual threshold voltages Input Layer Forward spike Current integration Compound Multibit Synapse Backward spikes Output Neuron 20
21 Multibit Synapses: Dendritic Processing Dendritic processing to include sensitivity to time-overlap between the pre- and post-synaptic spikes Shift in time or pulse voltage in the back-propagating spikes Simulations exhibit remarkable STDP learning behavior; experiments underway with physical devices X. Wu and V. Saxena, Realizing Multibit Synapses with Bistable RRAM Devices, Unpublished result. 21
22 Conclusion Nanoscale emerging memory devices present opportunity for realizing non von Neumann computing Mixed-Signal architectures for spiking neural networks can realize a versatile neuromorphic computing motif Further need to characterize device capabilities and limitations for large scale integration Algorithm/architecture development to be driven ground up by emerging device capabilities 22
23 References 1. X. Wu, V. Saxena and K. Zhu, Homogeneous Spiking Neuromorphic System for Real-World Pattern Recognition, IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 5(3), 2015, in press. 2. X. Wu, V. Saxena and K. Zhu, A CMOS Spiking Neuron for Dense Memristor-Synapse Connectivity for Brain- Inspired Computing, International Joint Conference on Neural Networks (IJCNN), 2015, Accepted. 3. X. Wu, V. Saxena and K. A. Campbell, Energy-efficient STDP-based learning circuits with memristor synapses, SPIE Machine Intelligence and Bio-inspired Computation: Theory and Applications VIII, X. Wu, V. Saxena and K. Zhu, A CMOS Spiking Neuron for Brain-Inspired Neural Networks with Resistive Synapses and In-Situ Learning, IEEE Transactions on Circuits and Systems II: Express Briefs, In final review. 5. X. Wu and V. Saxena, Associative Learning on CMOS-Memristor Spiking Neuromorphic Chip, IEEE Transactions on Nanotechnology, In 2 nd round review. 23
24 Questions? 24
Neuromorphic computing with Memristive devices. NCM group
Neuromorphic computing with Memristive devices NCM group Why neuromorphic? New needs for computing Recognition, Mining, Synthesis (Intel) Increase of Fault (nanoscale engineering) SEMICONDUCTOR TECHNOLOGY
More informationNEUROMORPHIC COMPUTING WITH MAGNETO-METALLIC NEURONS & SYNAPSES: PROSPECTS AND PERSPECTIVES
NEUROMORPHIC COMPUTING WITH MAGNETO-METALLIC NEURONS & SYNAPSES: PROSPECTS AND PERSPECTIVES KAUSHIK ROY ABHRONIL SENGUPTA, KARTHIK YOGENDRA, DELIANG FAN, SYED SARWAR, PRIYA PANDA, GOPAL SRINIVASAN, JASON
More informationThe N3XT Technology for. Brain-Inspired Computing
The N3XT Technology for Brain-Inspired Computing SystemX Alliance 27..8 Department of Electrical Engineering 25.4.5 2 25.4.5 Source: Google 3 25.4.5 Source: vrworld.com 4 25.4.5 Source: BDC Stanford Magazine
More informationRE-ENGINEERING COMPUTING WITH NEURO- MIMETIC DEVICES, CIRCUITS, AND ALGORITHMS
RE-ENGINEERING COMPUTING WITH NEURO- MIMETIC DEVICES, CIRCUITS, AND ALGORITHMS Kaushik Roy Abhronil Sengupta, Gopal Srinivasan, Aayush Ankit, Priya Panda, Xuanyao Fong, Deliang Fan, Jason Allred School
More informationMachine Learning. Neural Networks. (slides from Domingos, Pardo, others)
Machine Learning Neural Networks (slides from Domingos, Pardo, others) For this week, Reading Chapter 4: Neural Networks (Mitchell, 1997) See Canvas For subsequent weeks: Scaling Learning Algorithms toward
More informationA 68 Parallel Row Access Neuromorphic Core with 22K Multi-Level Synapses Based on Logic- Compatible Embedded Flash Memory Technology
A 68 Parallel Row Access Neuromorphic Core with 22K Multi-Level Synapses Based on Logic- Compatible Embedded Flash Memory Technology M. Kim 1, J. Kim 1, G. Park 1, L. Everson 1, H. Kim 1, S. Song 1,2,
More informationArtificial Neural Network and Fuzzy Logic
Artificial Neural Network and Fuzzy Logic 1 Syllabus 2 Syllabus 3 Books 1. Artificial Neural Networks by B. Yagnanarayan, PHI - (Cover Topologies part of unit 1 and All part of Unit 2) 2. Neural Networks
More informationMemory and computing beyond CMOS
Memory and computing beyond CMOS Dipartimento di Elettronica, Informazione e Bioingegneria Politecnico di Milano daniele.ielmini@polimi.it Outline 2 Introduction What is CMOS? What comes after CMOS? Example:
More informationCISC 3250 Systems Neuroscience
CISC 3250 Systems Neuroscience Systems Neuroscience How the nervous system performs computations How groups of neurons work together to achieve intelligence Professor Daniel Leeds dleeds@fordham.edu JMH
More informationSynaptic Devices and Neuron Circuits for Neuron-Inspired NanoElectronics
Synaptic Devices and Neuron Circuits for Neuron-Inspired NanoElectronics Byung-Gook Park Inter-university Semiconductor Research Center & Department of Electrical and Computer Engineering Seoul National
More informationNovel VLSI Implementation for Triplet-based Spike-Timing Dependent Plasticity
Novel LSI Implementation for Triplet-based Spike-Timing Dependent Plasticity Mostafa Rahimi Azghadi, Omid Kavehei, Said Al-Sarawi, Nicolangelo Iannella, and Derek Abbott Centre for Biomedical Engineering,
More informationA brain-inspired neuromorphic architecture for robust neural computation
A brain-inspired neuromorphic architecture for robust neural computation Fabio Stefanini and Giacomo Indiveri Institute of Neuroinformatics University of Zurich and ETH Zurich BIC Workshop @ ISCA40 June
More informationArtificial Neural Network
Artificial Neural Network Contents 2 What is ANN? Biological Neuron Structure of Neuron Types of Neuron Models of Neuron Analogy with human NN Perceptron OCR Multilayer Neural Network Back propagation
More informationMachine Learning. Neural Networks. (slides from Domingos, Pardo, others)
Machine Learning Neural Networks (slides from Domingos, Pardo, others) For this week, Reading Chapter 4: Neural Networks (Mitchell, 1997) See Canvas For subsequent weeks: Scaling Learning Algorithms toward
More informationMagnetic tunnel junction beyond memory from logic to neuromorphic computing WANJUN PARK DEPT. OF ELECTRONIC ENGINEERING, HANYANG UNIVERSITY
Magnetic tunnel junction beyond memory from logic to neuromorphic computing WANJUN PARK DEPT. OF ELECTRONIC ENGINEERING, HANYANG UNIVERSITY Magnetic Tunnel Junctions (MTJs) Structure High density memory
More informationLast update: October 26, Neural networks. CMSC 421: Section Dana Nau
Last update: October 26, 207 Neural networks CMSC 42: Section 8.7 Dana Nau Outline Applications of neural networks Brains Neural network units Perceptrons Multilayer perceptrons 2 Example Applications
More informationNeural networks. Chapter 19, Sections 1 5 1
Neural networks Chapter 19, Sections 1 5 Chapter 19, Sections 1 5 1 Outline Brains Neural networks Perceptrons Multilayer perceptrons Applications of neural networks Chapter 19, Sections 1 5 2 Brains 10
More informationAn Introductory Course in Computational Neuroscience
An Introductory Course in Computational Neuroscience Contents Series Foreword Acknowledgments Preface 1 Preliminary Material 1.1. Introduction 1.1.1 The Cell, the Circuit, and the Brain 1.1.2 Physics of
More informationA Hybrid CMOS/Memristive Nanoelectronic Circuit for Programming Synaptic Weights
A Hybrid CMOS/Memristive Nanoelectronic Circuit for Programming Synaptic Weights Arne Heittmann and Tobias G. Noll Chair of Electrical Engineering and Computer Systems RWTH Aachen University -52062 Aachen,
More informationRevision: Neural Network
Revision: Neural Network Exercise 1 Tell whether each of the following statements is true or false by checking the appropriate box. Statement True False a) A perceptron is guaranteed to perfectly learn
More informationData Mining Part 5. Prediction
Data Mining Part 5. Prediction 5.5. Spring 2010 Instructor: Dr. Masoud Yaghini Outline How the Brain Works Artificial Neural Networks Simple Computing Elements Feed-Forward Networks Perceptrons (Single-layer,
More informationIntroduction. Previous work has shown that AER can also be used to construct largescale networks with arbitrary, configurable synaptic connectivity.
Introduction The goal of neuromorphic engineering is to design and implement microelectronic systems that emulate the structure and function of the brain. Address-event representation (AER) is a communication
More informationCenter for Spintronic Materials, Interfaces, and Novel Architectures. Spintronics Enabled Efficient Neuromorphic Computing: Prospects and Perspectives
Center for Spintronic Materials, Interfaces, and Novel Architectures Spintronics Enabled Efficient Neuromorphic Computing: Prospects and Perspectives KAUSHIK ROY ABHRONIL SENGUPTA, KARTHIK YOGENDRA, DELIANG
More informationApplications of Memristors in ANNs
Applications of Memristors in ANNs Outline Brief intro to ANNs Firing rate networks Single layer perceptron experiment Other (simulation) examples Spiking networks and STDP ANNs ANN is bio inpsired inpsired
More informationMachine Learning. Neural Networks. (slides from Domingos, Pardo, others)
Machine Learning Neural Networks (slides from Domingos, Pardo, others) Human Brain Neurons Input-Output Transformation Input Spikes Output Spike Spike (= a brief pulse) (Excitatory Post-Synaptic Potential)
More informationSTDP Learning of Image Patches with Convolutional Spiking Neural Networks
STDP Learning of Image Patches with Convolutional Spiking Neural Networks Daniel J. Saunders, Hava T. Siegelmann, Robert Kozma College of Information and Computer Sciences University of Massachusetts Amherst
More informationNeural Networks. Chapter 18, Section 7. TB Artificial Intelligence. Slides from AIMA 1/ 21
Neural Networks Chapter 8, Section 7 TB Artificial Intelligence Slides from AIMA http://aima.cs.berkeley.edu / 2 Outline Brains Neural networks Perceptrons Multilayer perceptrons Applications of neural
More informationAnalog CMOS Circuits Implementing Neural Segmentation Model Based on Symmetric STDP Learning
Analog CMOS Circuits Implementing Neural Segmentation Model Based on Symmetric STDP Learning Gessyca Maria Tovar, Eric Shun Fukuda,TetsuyaAsai, Tetsuya Hirose, and Yoshihito Amemiya Hokkaido University,
More information2015 Todd Neller. A.I.M.A. text figures 1995 Prentice Hall. Used by permission. Neural Networks. Todd W. Neller
2015 Todd Neller. A.I.M.A. text figures 1995 Prentice Hall. Used by permission. Neural Networks Todd W. Neller Machine Learning Learning is such an important part of what we consider "intelligence" that
More information3/10/2013. Lecture #1. How small is Nano? (A movie) What is Nanotechnology? What is Nanoelectronics? What are Emerging Devices?
EECS 498/598: Nanocircuits and Nanoarchitectures Lecture 1: Introduction to Nanotelectronic Devices (Sept. 5) Lectures 2: ITRS Nanoelectronics Road Map (Sept 7) Lecture 3: Nanodevices; Guest Lecture by
More informationNeural networks. Chapter 20. Chapter 20 1
Neural networks Chapter 20 Chapter 20 1 Outline Brains Neural networks Perceptrons Multilayer networks Applications of neural networks Chapter 20 2 Brains 10 11 neurons of > 20 types, 10 14 synapses, 1ms
More informationEE04 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb 23, Sasidharan Sreedharan
EE04 804(B) Soft Computing Ver. 1.2 Class 2. Neural Networks - I Feb 23, 2012 Sasidharan Sreedharan www.sasidharan.webs.com 3/1/2012 1 Syllabus Artificial Intelligence Systems- Neural Networks, fuzzy logic,
More informationLecture 14 Population dynamics and associative memory; stable learning
Lecture 14 Population dynamics and associative memory; stable learning -Introduction -Associative Memory -Dense networks (mean-ield) -Population dynamics and Associative Memory -Discussion Systems or computing
More informationNeuromorphic Network Based on Carbon Nanotube/Polymer Composites
Neuromorphic Network Based on Carbon Nanotube/Polymer Composites Andrew Tudor, Kyunghyun Kim, Alex Ming Shen, Chris Shaffer, Dongwon Lee, Cameron D. Danesh, and Yong Chen Department of Mechanical & Aerospace
More informationNeural networks. Chapter 20, Section 5 1
Neural networks Chapter 20, Section 5 Chapter 20, Section 5 Outline Brains Neural networks Perceptrons Multilayer perceptrons Applications of neural networks Chapter 20, Section 5 2 Brains 0 neurons of
More informationPost Von Neumann Computing
Post Von Neumann Computing Matthias Kaiserswerth Hasler Stiftung (formerly IBM Research) 1 2014 IBM Corporation Foundation Purpose Support information and communication technologies (ICT) to advance Switzerland
More informationNeural Networks Introduction
Neural Networks Introduction H.A Talebi Farzaneh Abdollahi Department of Electrical Engineering Amirkabir University of Technology Winter 2011 H. A. Talebi, Farzaneh Abdollahi Neural Networks 1/22 Biological
More informationCS:4420 Artificial Intelligence
CS:4420 Artificial Intelligence Spring 2018 Neural Networks Cesare Tinelli The University of Iowa Copyright 2004 18, Cesare Tinelli and Stuart Russell a a These notes were originally developed by Stuart
More informationConsider the following spike trains from two different neurons N1 and N2:
About synchrony and oscillations So far, our discussions have assumed that we are either observing a single neuron at a, or that neurons fire independent of each other. This assumption may be correct in
More informationA nanoparticle-organic memory field-effect transistor behaving as a programmable spiking synapse
A nanoparticle-organic memory field-effect transistor behaving as a programmable spiking synapse F. Alibart,. Pleutin, D. Guerin, K. Lmimouni, D. Vuillaume Molecular Nanostructures & Devices group, Institute
More informationOutline. Neural dynamics with log-domain integrator circuits. Where it began Biophysics of membrane channels
Outline Neural dynamics with log-domain integrator circuits Giacomo Indiveri Neuromorphic Cognitive Systems group Institute of Neuroinformatics niversity of Zurich and ETH Zurich Dynamics of Multi-function
More informationImplementing Synaptic Plasticity in a VLSI Spiking Neural Network Model
Implementing Synaptic Plasticity in a VLSI Spiking Neural Network Model Johannes Schemmel, Andreas Grübl, Karlheinz Meier and Eilif Mueller Abstract This paper describes an area-efficient mixed-signal
More informationNeuromorphic architectures: challenges and opportunites in the years to come
Neuromorphic architectures: challenges and opportunites in the years to come Andreas G. Andreou andreou@jhu.edu Electrical and Computer Engineering Center for Language and Speech Processing Johns Hopkins
More informationInstituto Tecnológico y de Estudios Superiores de Occidente Departamento de Electrónica, Sistemas e Informática. Introductory Notes on Neural Networks
Introductory Notes on Neural Networs Dr. José Ernesto Rayas Sánche April Introductory Notes on Neural Networs Dr. José Ernesto Rayas Sánche BIOLOGICAL NEURAL NETWORKS The brain can be seen as a highly
More informationUsing Variable Threshold to Increase Capacity in a Feedback Neural Network
Using Variable Threshold to Increase Capacity in a Feedback Neural Network Praveen Kuruvada Abstract: The article presents new results on the use of variable thresholds to increase the capacity of a feedback
More informationLinear Regression, Neural Networks, etc.
Linear Regression, Neural Networks, etc. Gradient Descent Many machine learning problems can be cast as optimization problems Define a function that corresponds to learning error. (More on this later)
More informationLecture 4: Feed Forward Neural Networks
Lecture 4: Feed Forward Neural Networks Dr. Roman V Belavkin Middlesex University BIS4435 Biological neurons and the brain A Model of A Single Neuron Neurons as data-driven models Neural Networks Training
More informationBiological Modeling of Neural Networks:
Week 14 Dynamics and Plasticity 14.1 Reservoir computing - Review:Random Networks - Computing with rich dynamics Biological Modeling of Neural Networks: 14.2 Random Networks - stationary state - chaos
More informationIntroduction Biologically Motivated Crude Model Backpropagation
Introduction Biologically Motivated Crude Model Backpropagation 1 McCulloch-Pitts Neurons In 1943 Warren S. McCulloch, a neuroscientist, and Walter Pitts, a logician, published A logical calculus of the
More informationFrom Spin Torque Random Access Memory to Spintronic Memristor. Xiaobin Wang Seagate Technology
From Spin Torque Random Access Memory to Spintronic Memristor Xiaobin Wang Seagate Technology Contents Spin Torque Random Access Memory: dynamics characterization, device scale down challenges and opportunities
More informationArtificial Neural Networks The Introduction
Artificial Neural Networks The Introduction 01001110 01100101 01110101 01110010 01101111 01101110 01101111 01110110 01100001 00100000 01110011 01101011 01110101 01110000 01101001 01101110 01100001 00100000
More informationHow to do backpropagation in a brain
How to do backpropagation in a brain Geoffrey Hinton Canadian Institute for Advanced Research & University of Toronto & Google Inc. Prelude I will start with three slides explaining a popular type of deep
More informationLarge-scale neural modeling
Large-scale neural modeling We re acquiring brain data at an unprecedented rate Dendritic recording Serial Scanning EM Ca ++ imaging Kwabena Boahen Stanford Bioengineering boahen@stanford.edu Goal: Link
More informationA Study of Complex Deep Learning Networks on High Performance, Neuromorphic, and Quantum Computers
A Study of Complex Deep Learning Networks on High Performance, Neuromorphic, and Quantum Computers Thomas E. Potok, Ph.D. Computational Data Analytics Group Oak Ridge National Laboratory ORNL is managed
More informationArtificial Neural Networks. Q550: Models in Cognitive Science Lecture 5
Artificial Neural Networks Q550: Models in Cognitive Science Lecture 5 "Intelligence is 10 million rules." --Doug Lenat The human brain has about 100 billion neurons. With an estimated average of one thousand
More informationCOMP9444 Neural Networks and Deep Learning 2. Perceptrons. COMP9444 c Alan Blair, 2017
COMP9444 Neural Networks and Deep Learning 2. Perceptrons COMP9444 17s2 Perceptrons 1 Outline Neurons Biological and Artificial Perceptron Learning Linear Separability Multi-Layer Networks COMP9444 17s2
More informationLecture 7 Artificial neural networks: Supervised learning
Lecture 7 Artificial neural networks: Supervised learning Introduction, or how the brain works The neuron as a simple computing element The perceptron Multilayer neural networks Accelerated learning in
More informationNovel Devices and Circuits for Computing
Novel Devices and Circuits for Computing UCSB 594BB Winter 2013 Lecture 4: Resistive switching: Logic Class Outline Material Implication logic Stochastic computing Reconfigurable logic Material Implication
More informationarxiv: v1 [cs.et] 20 Nov 2014
Liquid State Machine with Dendritically Enhanced Readout for Low-power, Neuromorphic VLSI Implementations Subhrajit Roy, Student Member, IEEE, Amitava Banerjee and Arindam Basu, Member, IEEE arxiv:4.5458v
More informationCharacterizing the firing properties of an adaptive
Characterizing the firing properties of an adaptive analog VLSI neuron Daniel Ben Dayan Rubin 1,2, Elisabetta Chicca 2, and Giacomo Indiveri 2 1 Department of Bioengineering, Politecnico di Milano, P.zza
More informationThis document is downloaded from DR-NTU, Nanyang Technological University Library, Singapore.
This document is downloaded from DR-NTU, Nanyang Technological University Library, Singapore. Title Liquid state machine with dendritically enhanced readout for low-power, neuromorphic VLSI implementations
More informationStructure and Measurement of the brain lecture notes
Structure and Measurement of the brain lecture notes Marty Sereno 2009/2010!"#$%&'(&#)*%$#&+,'-&.)"/*"&.*)*-'(0&1223 Neurons and Models Lecture 1 Topics Membrane (Nernst) Potential Action potential/voltage-gated
More informationPerceptron. (c) Marcin Sydow. Summary. Perceptron
Topics covered by this lecture: Neuron and its properties Mathematical model of neuron: as a classier ' Learning Rule (Delta Rule) Neuron Human neural system has been a natural source of inspiration for
More informationarxiv: v1 [cs.ne] 30 Mar 2013
A Neuromorphic VLSI Design for Spike Timing and Rate Based Synaptic Plasticity Mostafa Rahimi Azghadi a,, Said Al-Sarawi a,, Derek Abbott a, Nicolangelo Iannella a, a School of Electrical and Electronic
More informationINTRODUCTION TO NEURAL NETWORKS
INTRODUCTION TO NEURAL NETWORKS R. Beale & T.Jackson: Neural Computing, an Introduction. Adam Hilger Ed., Bristol, Philadelphia and New York, 990. THE STRUCTURE OF THE BRAIN The brain consists of about
More informationNeural Networks and the Back-propagation Algorithm
Neural Networks and the Back-propagation Algorithm Francisco S. Melo In these notes, we provide a brief overview of the main concepts concerning neural networks and the back-propagation algorithm. We closely
More informationNiobium oxide and Vanadium oxide as unconventional Materials for Applications in neuromorphic Devices and Circuits
Niobium oxide and Vanadium oxide as unconventional Materials for Applications in neuromorphic Devices and Circuits Martin Ziegler, Mirko Hansen, Marina Ignatov, Adrian Petraru, and Hermann Kohlstedt Nanoelektronik,
More informationQuasi Analog Formal Neuron and Its Learning Algorithm Hardware
Quasi Analog Formal Neuron and Its Learning Algorithm Hardware Karen Nazaryan Division of Microelectronics and Biomedical Devices, State Engineering University of Armenia, 375009, Terian Str. 105, Yerevan,
More informationMemristive Tunneling Devices: From Device Principles to Neuromorphic Applications
Memristive Tunneling Devices: From Device Principles to Neuromorphic Applications Martin Ziegler, A. Petraru, R. Soni, and H. Kohlstedt AG Nanoelektronik Technische Fakultät Christian-Albrechts-Universität
More informationLiquid Computing in a Simplified Model of Cortical Layer IV: Learning to Balance a Ball
Liquid Computing in a Simplified Model of Cortical Layer IV: Learning to Balance a Ball Dimitri Probst 1,3, Wolfgang Maass 2, Henry Markram 1, and Marc-Oliver Gewaltig 1 1 Blue Brain Project, École Polytechnique
More information22c145-Fall 01: Neural Networks. Neural Networks. Readings: Chapter 19 of Russell & Norvig. Cesare Tinelli 1
Neural Networks Readings: Chapter 19 of Russell & Norvig. Cesare Tinelli 1 Brains as Computational Devices Brains advantages with respect to digital computers: Massively parallel Fault-tolerant Reliable
More informationSignal, donnée, information dans les circuits de nos cerveaux
NeuroSTIC Brest 5 octobre 2017 Signal, donnée, information dans les circuits de nos cerveaux Claude Berrou Signal, data, information: in the field of telecommunication, everything is clear It is much less
More informationMethods for Estimating the Computational Power and Generalization Capability of Neural Microcircuits
Methods for Estimating the Computational Power and Generalization Capability of Neural Microcircuits Wolfgang Maass, Robert Legenstein, Nils Bertschinger Institute for Theoretical Computer Science Technische
More informationSupporting Online Material for
www.sciencemag.org/cgi/content/full/319/5869/1543/dc1 Supporting Online Material for Synaptic Theory of Working Memory Gianluigi Mongillo, Omri Barak, Misha Tsodyks* *To whom correspondence should be addressed.
More informationLearning Spatio-Temporally Encoded Pattern Transformations in Structured Spiking Neural Networks 12
Learning Spatio-Temporally Encoded Pattern Transformations in Structured Spiking Neural Networks 12 André Grüning, Brian Gardner and Ioana Sporea Department of Computer Science University of Surrey Guildford,
More informationThis document is downloaded from DR-NTU, Nanyang Technological University Library, Singapore.
This document is downloaded from DR-NTU, Nanyang Technological University Library, Singapore. Title Small-signal neural models and their applications Author(s) Basu, Arindam Citation Basu, A. (01). Small-signal
More information! Charge Leakage/Charge Sharing. " Domino Logic Design Considerations. ! Logic Comparisons. ! Memory. " Classification. " ROM Memories.
ESE 57: Digital Integrated Circuits and VLSI Fundamentals Lec 9: March 9, 8 Memory Overview, Memory Core Cells Today! Charge Leakage/ " Domino Logic Design Considerations! Logic Comparisons! Memory " Classification
More information1. HP's memristor and applications 2. Models of resistance switching. 4. 3D circuit architectures 5. Proposal for evaluation framework
OUTL LINE 1. HP's memristor and applications 2. Models of resistance switching 3. Volatility speed tradeo ffs 4. 3D circuit architectures 5. Proposal for evaluation framework HP S MEMRISTOR memristor =
More informationNeuromorphic computing using non-volatile memory
Advances in Physics: X ISSN: (Print) 2374-6149 (Online) Journal homepage: http://www.tandfonline.com/loi/tapx20 Neuromorphic computing using non-volatile memory Geoffrey W. Burr, Robert M. Shelby, Abu
More informationCMOS Digital Integrated Circuits Lec 13 Semiconductor Memories
Lec 13 Semiconductor Memories 1 Semiconductor Memory Types Semiconductor Memories Read/Write (R/W) Memory or Random Access Memory (RAM) Read-Only Memory (ROM) Dynamic RAM (DRAM) Static RAM (SRAM) 1. Mask
More information(Feed-Forward) Neural Networks Dr. Hajira Jabeen, Prof. Jens Lehmann
(Feed-Forward) Neural Networks 2016-12-06 Dr. Hajira Jabeen, Prof. Jens Lehmann Outline In the previous lectures we have learned about tensors and factorization methods. RESCAL is a bilinear model for
More informationExercise 15 : Cable Equation
Biophysics of Neural Computation : Introduction to Neuroinformatics WS 2008-2009 Prof. Rodney Douglas, Kevan Martin, Hans Scherberger, Matthew Cook Ass. Frederic Zubler fred@ini.phys.ethz.ch http://www.ini.uzh.ch/
More informationProbabilistic Models in Theoretical Neuroscience
Probabilistic Models in Theoretical Neuroscience visible unit Boltzmann machine semi-restricted Boltzmann machine restricted Boltzmann machine hidden unit Neural models of probabilistic sampling: introduction
More informationCCS050M12CM2 1.2kV, 50A Silicon Carbide Six-Pack (Three Phase) Module Z-FET TM MOSFET and Z-Rec TM Diode
CCS5M12CM2 1.2kV, 5A Silicon Carbide Six-Pack (Three Phase) Module Z-FET TM MOSFET and Z-Rec TM Diode Features Ultra Low Loss Zero Reverse Recovery Current Zero Turn-off Tail Current High-Frequency Operation
More informationSynaptic dynamics. John D. Murray. Synaptic currents. Simple model of the synaptic gating variable. First-order kinetics
Synaptic dynamics John D. Murray A dynamical model for synaptic gating variables is presented. We use this to study the saturation of synaptic gating at high firing rate. Shunting inhibition and the voltage
More informationESE 570: Digital Integrated Circuits and VLSI Fundamentals
ESE 570: Digital Integrated Circuits and VLSI Fundamentals Lec 19: March 29, 2018 Memory Overview, Memory Core Cells Today! Charge Leakage/Charge Sharing " Domino Logic Design Considerations! Logic Comparisons!
More informationModeling of Retinal Ganglion Cell Responses to Electrical Stimulation with Multiple Electrodes L.A. Hruby Salk Institute for Biological Studies
Modeling of Retinal Ganglion Cell Responses to Electrical Stimulation with Multiple Electrodes L.A. Hruby Salk Institute for Biological Studies Introduction Since work on epiretinal electrical stimulation
More informationNew Delhi & Affiliated to VTU, Belgavi ) Oorgaum, Kolar Dist ,Karnataka
Design and Implementation of Logic Gates and Adder Circuits on FPGA Using ANN Neelu Farha 1., Ann Louisa Paul J 2., Naadiya Kousar L S 3., Devika S 4., Prof. Ruckmani Divakaran 5 1,2,3,4,5 Department of
More informationCARNEGIE MELLON UNIVERSITY DEPARTMENT OF ELECTRICAL AND COMPUTER ENGINEERING DIGITAL INTEGRATED CIRCUITS FALL 2002
CARNEGIE MELLON UNIVERSITY DEPARTMENT OF ELECTRICAL AND COMPUTER ENGINEERING 18-322 DIGITAL INTEGRATED CIRCUITS FALL 2002 Final Examination, Monday Dec. 16, 2002 NAME: SECTION: Time: 180 minutes Closed
More information1.2 kv 16 mω 1.8 mj. Package. Symbol Parameter Value Unit Test Conditions Notes 117 V GS = 20V, T C
CAS1H12AM1 1.2 kv, 1A Silicon Carbide Half-Bridge Module Z-FET TM MOSFET and Z-Rec TM Diode Not recommended for new designs. Replacement part: CAS12M12BM2 Features Ultra Low Loss Zero Turn-off Tail Current
More informationHow to outperform a supercomputer with neuromorphic chips
How to outperform a supercomputer with neuromorphic chips Kwabena Boahen Stanford Bioengineering boahen@stanford.edu Telluride Acknowledgements Paul Merolla John Arthur Joseph Kai Lin Hynna BrainsInSilicon.stanford.edu
More informationEdge of Chaos Computation in Mixed-Mode VLSI - A Hard Liquid
Edge of Chaos Computation in Mixed-Mode VLSI - A Hard Liquid Felix Schürmann, Karlheinz Meier, Johannes Schemmel Kirchhoff Institute for Physics University of Heidelberg Im Neuenheimer Feld 227, 6912 Heidelberg,
More informationDesign of Basic Logic Gates using CMOS and Artificial Neural Networks (ANN)
AMSE JOURNALS-2016-Series: Advances D; Vol. 21; N 1; pp 54-65 Submitted July 2016; Revised Oct. 11, 2016, Accepted Nov. 15, 2016 Design of Basic Logic Gates using CMOS and Artificial Neural Networks (ANN)
More informationMachine Learning and Data Mining. Multi-layer Perceptrons & Neural Networks: Basics. Prof. Alexander Ihler
+ Machine Learning and Data Mining Multi-layer Perceptrons & Neural Networks: Basics Prof. Alexander Ihler Linear Classifiers (Perceptrons) Linear Classifiers a linear classifier is a mapping which partitions
More informationtec UNIVERSITY OF PENNSYLVANIA ELECTRICAL ENGINEERING DEPARTMENT 7C., Ouarterly Report
UNIVERSITY OF PENNSYLVANIA ELECTRICAL ENGINEERING DEPARTMENT tec 2 8 1994 7C., Ouarterly Report COGNITIVE NETWORKS FOR ATR: THE ROLES OF BIFURCATION AND CHAOS Period Covered: 10/01/94-1/01/95 Submitted
More informationUsing a Hopfield Network: A Nuts and Bolts Approach
Using a Hopfield Network: A Nuts and Bolts Approach November 4, 2013 Gershon Wolfe, Ph.D. Hopfield Model as Applied to Classification Hopfield network Training the network Updating nodes Sequencing of
More informationMM74C90 MM74C93 4-Bit Decade Counter 4-Bit Binary Counter
4-Bit Decade Counter 4-Bit Binary Counter General Description The MM74C90 decade counter and the MM74C93 binary counter and complementary MOS (CMOS) integrated circuits constructed with N- and P-channel
More informationARTIFICIAL INTELLIGENCE. Artificial Neural Networks
INFOB2KI 2017-2018 Utrecht University The Netherlands ARTIFICIAL INTELLIGENCE Artificial Neural Networks Lecturer: Silja Renooij These slides are part of the INFOB2KI Course Notes available from www.cs.uu.nl/docs/vakken/b2ki/schema.html
More informationArtificial Intelligence
Artificial Intelligence Jeff Clune Assistant Professor Evolving Artificial Intelligence Laboratory Announcements Be making progress on your projects! Three Types of Learning Unsupervised Supervised Reinforcement
More informationTemporal Pattern Analysis
LIACS Leiden Institute of Advanced Computer Science Master s Thesis June 17, 29 Temporal Pattern Analysis Using Reservoir Computing Author: Ron Vink Supervisor: Dr. Walter Kosters 1 Contents 1 Introduction
More information