Lecture 14 Population dynamics and associative memory; stable learning
|
|
- Verity Jones
- 5 years ago
- Views:
Transcription
1 Lecture 14 Population dynamics and associative memory; stable learning -Introduction -Associative Memory -Dense networks (mean-ield) -Population dynamics and Associative Memory -Discussion
2 Systems or computing and inormation processing Brain Computer Distributed architecture 10 (10 proc. Elements/neurons) No separation o processing and memory memory CPU input Von Neumann architecture 1 CPU 10 (10 transistors)
3 motor cortex visual cortex association cortex to motor output
4 1mm neurons 3 km wires Signal: action potential (spike) action potential
5 Systems or computing and inormation processing Brain 1mm neurons 3 km wires Distributed architecture neurons 4 10 connections/neurons No separation o processing and memory
6 - recognize/understand images: pattern recognition Noisy image Associative memory/ collective computation Full image Brain-style computation
7 Lecture 14 Population dynamics and associative memory; stable learning -Introduction -Associative Memory -Dense networks (mean-ield) -Population dynamics and Associative Memory -Final discussion
8 Associative memory Elementary pixel S i = +1 S i = -1 w i = +1 w i = +1 w i = -1 dynamics S i ( t 1) sgn w S i t Hopield model Sum over all interactions with i
9 Associative memory interactions w i p i p Prototype p 1 Prototype p 2 S i ( t 1) sgn Sum over all prototypes dynamics w S i t Hopield model Sum over all interactions with i
10 Associative memory interactions w i p i p This rule is very good or random patterns Sum over all prototypes It does not work well or correlated patters Prototype p 1 S i ( t 1) sgn dynamics w S i t Hopield model Sum over all interactions with i
11 Associative memory Interacting neurons Prototype p 1 Hopield model Finds the closest prototype i.e. maximal overlap (similarity) m Computation - without CPU, - without explicit memory unit
12 Hebbian Learning k pr e w i i post When an axon o cell repeatedly or persistently takes part in iring cell i, then s eiciency as one o the cells iring i is increased Hebb, local rule - simultaneously active (correlations)
13 Hebbian Learning Elementary pixel S i = +1 S i = -1 w i = +1 w i = +1 w i = -1
14 Hebbian Learning item memorized
15 Hebbian Learning Recall: Partial ino item recalled
16 Lecture 14 Population dynamics and associative memory; stable learning -Introduction -Associative Memory So ar: neuron=spin=pixel=binary=on/o BUT: what about more realistic spiking neurons?
17 Lecture 14 Population dynamics and associative memory; stable learning -Introduction -Associative Memory -Dense networks (mean-ield) -Population dynamics and Associative Memory -Stable learning
18 Populations o spiking neurons t? I(t) population dynamics? t population activity A( t) n( t; t N t t)
19 Neuron # Activity in a populations o neurons A [Hz] input -low rate -high rate time [ms] 200 Neuron # Population neurons - 20 percent inhibitory - randomly connected u [mv] time [ms] 200
20 Homogeneous network (I&F) Consider 1 neuron
21 Homogeneous network (I&F) Assumption o Stochastic spike arrival: network o exc. neurons, total spike arrival rate A(t) d C u dt All synapses have identical weight I I 0 k I( t) I ion k k I(t) Membrane equation Synaptic current pulses o shape J 0 ( t tk ) N EPSC R I(t)
22 Homogeneous network (I&F) Assumption o Stochastic spike arrival: network o exc. neurons, total spike arrival rate A(t) C d dt u k I ion k I(t) membrane equation I ( t) I o I noise I I 0 R I(t) I 0 A 0 Population activity
23 Analysis o Homogeneous Population Step 1: Single neuron property: Inect noise current I I 0 I(t) Hodgkin-Huxley d C u dt k I ion k I(t) Leaky I&F d dt u ( u u ) R I( t) rest I ( t) I o I noise Measure requency with noise I ( t) I o I noise g ( I 0 ) I 0
24 Analysis o Homogeneous Population Step 2 : consider 1 neuron in the network I ext All neurons receive the same input ( mean ield ): - input rom network - external input I I0 I0 A0 I ext Population activity Mean input rom network prop. to population activity
25 Step 3: assume Stationary State/Asynchronous State A 0 I g( 0 ) All neurons are the same A(t)=const Step 4: close equation calculate A0 A 0 g( I0) g( A0 I ext ) I0 A0 I ext 1 A0 I0 I ext typical mean ield (Curie Weiss) A 0 g( I 0 ) requency (single neuron) Blackboard
26 Exercise (some time ago): ind stationary state A(t)=const I0 A0 I ext 1 A0 I0 I ext ully connected coupling J0/N typical mean ield A 0 requency (single neuron) g( I 0 ) close equations calculate A0
27 Lecture 14 Population dynamics and associative memory; stable learning -Introduction -Associative Memory review -Dense networks (mean-ield) -Population dynamics and Associative Memory
28 Back to Associative memory -Possible with spiking neurons -Calculation: mean-ield -Prototypes = random patterns Interacting neurons Computation - without CPU, - without explicit memory unit
29 Associative memory - simple model -Rate model/population activity -Calculation: mean-ield -Prototypes = random patterns Interacting neurons rate model Single-neuron iring rate Ai g( wi A ) population rate g( I 0 ) w i N 2 max p i p
30 Associative memory For comparison with Spin S=+/-1 1 m N p S Prototype p 1 Task: Find the prototype with maximal overlap m 1 N p (2 +/-1 max 1) Blackboard
31 Associative memory g ( I 0 ) Input current g( h ) 0 Input potential Prototype p 1 m 1 N p (2 max 1) overlap 2 1 m max p N N +/-1 p with prob /-1
32 Associative memory main idea requency (single neuron) m g( I 0 ) Pattern recognized Prototype p 1 m Pattern not recalled Task: Find the prototype with maximal overlap 1 N p (2 max 1) +/-1
33 Conclusion - Associative memory -Possible with spiking neurons -Calculation: mean-ield -Prototypes = random patterns Interacting neurons Computation - without CPU, - without explicit memory unit
34 Associative memory interactions w i p i p This rule is very good or random Patterns (a=0.5) Prototype p 1 Sum over all prototypes It does not work well or correlated patters or low activity random Patterns (e.g., a=0.1) wi ( pi c)( p a) e.g., in each pattern exactly 10 percent o neurons are active
35 Associative memory 1 wi K( pi, p ) N Locally stored inormation Prototype p 1 Blackboard or low activity random Patterns (e.g., a=0.1) wi ( pi c)( p a) e.g., in each pattern exactly 10 percent o neurons are active
36 Associative memory 1 wi K( pi, p ) N Blackboard Locally stored inormation wi ( pi c)( p a) e.g., a=0.1 means in each pattern exactly 10 percent o neurons are active Lz ( ) h( z) K( z, z ') ( s) A( z ', t s) ds z ' N A( z ') g( h( z '))
37 Associative memory For a given pattern only 2 populations! A requency (single neuron) g( h ) 0 Blackboard h in h act Lz ( ) h( z) K( z, z ') ( s) A( z ', t s) ds z ' N 1 1 wi K( pi, p ) ( pi c)( p a) N N A( z ') g( h( z '))
38 Lecture 14 Population dynamics and associative memory; stable learning -Introduction -Associative Memory -Dense networks (mean-ield) -Population dynamics and Associative Memory -Final discussion
39 Nearly the end: what can I improve or the students next year? Integrated exercises? Miniproect? Overall workload?(4 credit course = 6hrs per week) Background/Prerequisites? -Physics students -SV students -Math students
40 Exam: -written exam, 23.6 rom 8:15-11:15 - miniproects counts 1/3 towards inal grade For written exam: -bring 1 page A5 o own handwritten notes The end
41 The end
42 Exercise now: Associative memory g( I 0 ) i g ( wi w i ) N 2 max p i p i g( p m i ) Prototype p 1 Assume 4 patterns. At time t=0, overlap with Pattern 3, no overlap with other patterns. discuss temporal evolution (assume that patterns are orthogonal) m 1 N p (2 max 1)
43 The end
Biological Modeling of Neural Networks:
Week 14 Dynamics and Plasticity 14.1 Reservoir computing - Review:Random Networks - Computing with rich dynamics Biological Modeling of Neural Networks: 14.2 Random Networks - stationary state - chaos
More informationApplication of Associative Matrices to Recognize DNA Sequences in Bioinformatics
Application of Associative Matrices to Recognize DNA Sequences in Bioinformatics 1. Introduction. Jorge L. Ortiz Department of Electrical and Computer Engineering College of Engineering University of Puerto
More informationMath in systems neuroscience. Quan Wen
Math in systems neuroscience Quan Wen Human brain is perhaps the most complex subject in the universe 1 kg brain 10 11 neurons 180,000 km nerve fiber 10 15 synapses 10 18 synaptic proteins Multiscale
More informationStorage Capacity of Letter Recognition in Hopfield Networks
Storage Capacity of Letter Recognition in Hopfield Networks Gang Wei (gwei@cs.dal.ca) Zheyuan Yu (zyu@cs.dal.ca) Faculty of Computer Science, Dalhousie University, Halifax, N.S., Canada B3H 1W5 Abstract:
More informationDecoding. How well can we learn what the stimulus is by looking at the neural responses?
Decoding How well can we learn what the stimulus is by looking at the neural responses? Two approaches: devise explicit algorithms for extracting a stimulus estimate directly quantify the relationship
More informationConsider the following spike trains from two different neurons N1 and N2:
About synchrony and oscillations So far, our discussions have assumed that we are either observing a single neuron at a, or that neurons fire independent of each other. This assumption may be correct in
More informationNeuronal Dynamics: Computational Neuroscience of Single Neurons
Week 5 part 3a :Three definitions of rate code Neuronal Dynamics: Computational Neuroscience of Single Neurons Week 5 Variability and Noise: The question of the neural code Wulfram Gerstner EPFL, Lausanne,
More informationComputational Intelligence Lecture 6: Associative Memory
Computational Intelligence Lecture 6: Associative Memory Farzaneh Abdollahi Department of Electrical Engineering Amirkabir University of Technology Fall 2011 Farzaneh Abdollahi Computational Intelligence
More informationUsing Variable Threshold to Increase Capacity in a Feedback Neural Network
Using Variable Threshold to Increase Capacity in a Feedback Neural Network Praveen Kuruvada Abstract: The article presents new results on the use of variable thresholds to increase the capacity of a feedback
More informationMachine Learning. Neural Networks. (slides from Domingos, Pardo, others)
Machine Learning Neural Networks (slides from Domingos, Pardo, others) For this week, Reading Chapter 4: Neural Networks (Mitchell, 1997) See Canvas For subsequent weeks: Scaling Learning Algorithms toward
More informationBasic elements of neuroelectronics -- membranes -- ion channels -- wiring. Elementary neuron models -- conductance based -- modelers alternatives
Computing in carbon Basic elements of neuroelectronics -- membranes -- ion channels -- wiring Elementary neuron models -- conductance based -- modelers alternatives Wiring neurons together -- synapses
More informationBiological Modeling of Neural Networks
Week 4 part 2: More Detail compartmental models Biological Modeling of Neural Networks Week 4 Reducing detail - Adding detail 4.2. Adding detail - apse -cable equat Wulfram Gerstner EPFL, Lausanne, Switzerland
More informationAddressing Challenges in Neuromorphic Computing with Memristive Synapses
Addressing Challenges in Neuromorphic Computing with Memristive Synapses Vishal Saxena 1, Xinyu Wu 1 and Maria Mitkova 2 1 Analog Mixed-Signal and Photonic IC (AMPIC) Lab 2 Nanoionic Materials and Devices
More informationAn Introductory Course in Computational Neuroscience
An Introductory Course in Computational Neuroscience Contents Series Foreword Acknowledgments Preface 1 Preliminary Material 1.1. Introduction 1.1.1 The Cell, the Circuit, and the Brain 1.1.2 Physics of
More informationCISC 3250 Systems Neuroscience
CISC 3250 Systems Neuroscience Systems Neuroscience How the nervous system performs computations How groups of neurons work together to achieve intelligence Professor Daniel Leeds dleeds@fordham.edu JMH
More informationEffects of Interactive Function Forms and Refractoryperiod in a Self-Organized Critical Model Based on Neural Networks
Commun. Theor. Phys. (Beijing, China) 42 (2004) pp. 121 125 c International Academic Publishers Vol. 42, No. 1, July 15, 2004 Effects of Interactive Function Forms and Refractoryperiod in a Self-Organized
More informationNeuromorphic computing with Memristive devices. NCM group
Neuromorphic computing with Memristive devices NCM group Why neuromorphic? New needs for computing Recognition, Mining, Synthesis (Intel) Increase of Fault (nanoscale engineering) SEMICONDUCTOR TECHNOLOGY
More informationLecture 4: Feed Forward Neural Networks
Lecture 4: Feed Forward Neural Networks Dr. Roman V Belavkin Middlesex University BIS4435 Biological neurons and the brain A Model of A Single Neuron Neurons as data-driven models Neural Networks Training
More informationMemories Associated with Single Neurons and Proximity Matrices
Memories Associated with Single Neurons and Proximity Matrices Subhash Kak Oklahoma State University, Stillwater Abstract: This paper extends the treatment of single-neuron memories obtained by the use
More informationDeconstructing Actual Neurons
1 Deconstructing Actual Neurons Richard Bertram Department of Mathematics and Programs in Neuroscience and Molecular Biophysics Florida State University Tallahassee, Florida 32306 Reference: The many ionic
More informationSynaptic Devices and Neuron Circuits for Neuron-Inspired NanoElectronics
Synaptic Devices and Neuron Circuits for Neuron-Inspired NanoElectronics Byung-Gook Park Inter-university Semiconductor Research Center & Department of Electrical and Computer Engineering Seoul National
More informationAnnales UMCS Informatica AI 1 (2003) UMCS. Liquid state machine built of Hodgkin-Huxley neurons pattern recognition and informational entropy
Annales UMC Informatica AI 1 (2003) 107-113 Annales UMC Informatica Lublin-Polonia ectio AI http://www.annales.umcs.lublin.pl/ Liquid state machine built of Hodgkin-Huxley neurons pattern recognition and
More information7 Rate-Based Recurrent Networks of Threshold Neurons: Basis for Associative Memory
Physics 178/278 - David Kleinfeld - Fall 2005; Revised for Winter 2017 7 Rate-Based Recurrent etworks of Threshold eurons: Basis for Associative Memory 7.1 A recurrent network with threshold elements The
More informationActivity Driven Adaptive Stochastic. Resonance. Gregor Wenning and Klaus Obermayer. Technical University of Berlin.
Activity Driven Adaptive Stochastic Resonance Gregor Wenning and Klaus Obermayer Department of Electrical Engineering and Computer Science Technical University of Berlin Franklinstr. 8/9, 187 Berlin fgrewe,obyg@cs.tu-berlin.de
More informationHopfield Networks. (Excerpt from a Basic Course at IK 2008) Herbert Jaeger. Jacobs University Bremen
Hopfield Networks (Excerpt from a Basic Course at IK 2008) Herbert Jaeger Jacobs University Bremen Building a model of associative memory should be simple enough... Our brain is a neural network Individual
More informationHopfield Neural Network and Associative Memory. Typical Myelinated Vertebrate Motoneuron (Wikipedia) Topic 3 Polymers and Neurons Lecture 5
Hopfield Neural Network and Associative Memory Typical Myelinated Vertebrate Motoneuron (Wikipedia) PHY 411-506 Computational Physics 2 1 Wednesday, March 5 1906 Nobel Prize in Physiology or Medicine.
More informationA. The Hopfield Network. III. Recurrent Neural Networks. Typical Artificial Neuron. Typical Artificial Neuron. Hopfield Network.
III. Recurrent Neural Networks A. The Hopfield Network 2/9/15 1 2/9/15 2 Typical Artificial Neuron Typical Artificial Neuron connection weights linear combination activation function inputs output net
More informationExercises. Chapter 1. of τ approx that produces the most accurate estimate for this firing pattern.
1 Exercises Chapter 1 1. Generate spike sequences with a constant firing rate r 0 using a Poisson spike generator. Then, add a refractory period to the model by allowing the firing rate r(t) to depend
More informationSampling-based probabilistic inference through neural and synaptic dynamics
Sampling-based probabilistic inference through neural and synaptic dynamics Wolfgang Maass for Robert Legenstein Institute for Theoretical Computer Science Graz University of Technology, Austria Institute
More informationMachine Learning. Neural Networks. (slides from Domingos, Pardo, others)
Machine Learning Neural Networks (slides from Domingos, Pardo, others) For this week, Reading Chapter 4: Neural Networks (Mitchell, 1997) See Canvas For subsequent weeks: Scaling Learning Algorithms toward
More informationMachine Learning. Neural Networks
Machine Learning Neural Networks Bryan Pardo, Northwestern University, Machine Learning EECS 349 Fall 2007 Biological Analogy Bryan Pardo, Northwestern University, Machine Learning EECS 349 Fall 2007 THE
More informationSupporting Online Material for
www.sciencemag.org/cgi/content/full/319/5869/1543/dc1 Supporting Online Material for Synaptic Theory of Working Memory Gianluigi Mongillo, Omri Barak, Misha Tsodyks* *To whom correspondence should be addressed.
More informationSynaptic Input. Linear Model of Synaptic Transmission. Professor David Heeger. September 5, 2000
Synaptic Input Professor David Heeger September 5, 2000 The purpose of this handout is to go a bit beyond the discussion in Ch. 6 of The Book of Genesis on synaptic input, and give some examples of how
More informationSystems Biology: A Personal View IX. Landscapes. Sitabhra Sinha IMSc Chennai
Systems Biology: A Personal View IX. Landscapes Sitabhra Sinha IMSc Chennai Fitness Landscapes Sewall Wright pioneered the description of how genotype or phenotypic fitness are related in terms of a fitness
More informationThe Bayesian Brain. Robert Jacobs Department of Brain & Cognitive Sciences University of Rochester. May 11, 2017
The Bayesian Brain Robert Jacobs Department of Brain & Cognitive Sciences University of Rochester May 11, 2017 Bayesian Brain How do neurons represent the states of the world? How do neurons represent
More informationMarr's Theory of the Hippocampus: Part I
Marr's Theory of the Hippocampus: Part I Computational Models of Neural Systems Lecture 3.3 David S. Touretzky October, 2015 David Marr: 1945-1980 10/05/15 Computational Models of Neural Systems 2 Marr
More informationThe Hebb rule Neurons that fire together wire together.
Unsupervised learning The Hebb rule Neurons that fire together wire together. PCA RF development with PCA Classical Conditioning and Hebbʼs rule Ear A Nose B Tongue When an axon in cell A is near enough
More information15 Grossberg Network 1
Grossberg Network Biological Motivation: Vision Bipolar Cell Amacrine Cell Ganglion Cell Optic Nerve Cone Light Lens Rod Horizontal Cell Retina Optic Nerve Fiber Eyeball and Retina Layers of Retina The
More informationNeocortical Pyramidal Cells Can Control Signals to Post-Synaptic Cells Without Firing:
Neocortical Pyramidal Cells Can Control Signals to Post-Synaptic Cells Without Firing: a model of the axonal plexus Erin Munro Department of Mathematics Boston University 4/14/2011 Gap junctions on pyramidal
More informationStructure of Brain at Small, Medium and Large Scales. Laval University 4 May 2006
Structure of Brain at Small, Medium and Large Scales Helmut Kröger Laval University 4 May 2006 Collaboration: Dr. Alain Destexhe,, Neurosciences Intégratives et Computationnelles, CNRS, Paris Dr. Igor
More informationA. The Hopfield Network. III. Recurrent Neural Networks. Typical Artificial Neuron. Typical Artificial Neuron. Hopfield Network.
Part 3A: Hopfield Network III. Recurrent Neural Networks A. The Hopfield Network 1 2 Typical Artificial Neuron Typical Artificial Neuron connection weights linear combination activation function inputs
More informationBo Deng University of Nebraska-Lincoln UNL Math Biology Seminar
Mathematical Model of Neuron Bo Deng University of Nebraska-Lincoln UNL Math Biology Seminar 09-10-2015 Review -- One Basic Circuit By Kirchhoff's Current Law 0 = I C + I R + I L I ext By Kirchhoff s Voltage
More informationHopfield Neural Network
Lecture 4 Hopfield Neural Network Hopfield Neural Network A Hopfield net is a form of recurrent artificial neural network invented by John Hopfield. Hopfield nets serve as content-addressable memory systems
More informationHow do biological neurons learn? Insights from computational modelling of
How do biological neurons learn? Insights from computational modelling of neurobiological experiments Lubica Benuskova Department of Computer Science University of Otago, New Zealand Brain is comprised
More information7 Recurrent Networks of Threshold (Binary) Neurons: Basis for Associative Memory
Physics 178/278 - David Kleinfeld - Winter 2019 7 Recurrent etworks of Threshold (Binary) eurons: Basis for Associative Memory 7.1 The network The basic challenge in associative networks, also referred
More information1 R.V k V k 1 / I.k/ here; we ll stimulate the action potential another way.) Note that this further simplifies to. m 3 k h k.
1. The goal of this problem is to simulate a propagating action potential for the Hodgkin-Huxley model and to determine the propagation speed. From the class notes, the discrete version (i.e., after breaking
More informationNeural Network with Ensembles
Neural Network with Ensembles Ernst Kussul, Member, IEEE, Oleksandr Makeyev, Member, IEEE, Tatiana Baidyk, and Daniel Calderon Reyes Abstract Pattern recognition systems usually have a relatively small
More informationCOMP304 Introduction to Neural Networks based on slides by:
COMP34 Introduction to Neural Networks based on slides by: Christian Borgelt http://www.borgelt.net/ Christian Borgelt Introduction to Neural Networks Motivation: Why (Artificial) Neural Networks? (Neuro-)Biology
More informationA Learning Theory for Reward-Modulated Spike-Timing-Dependent Plasticity with Application to Biofeedback
A Learning Theory for Reward-Modulated Spike-Timing-Dependent Plasticity with Application to Biofeedback Robert Legenstein, Dejan Pecevski, Wolfgang Maass Institute for Theoretical Computer Science Graz
More informationArtificial neural networks
Artificial neural networks B. Mehlig, Department of Physics, University of Gothenburg, Sweden FFR35/FIM70 Artificial Neural Networks Chalmers/Gothenburg University, 7.5 credits 3 Course home page Teachers
More informationAdaptation in the Neural Code of the Retina
Adaptation in the Neural Code of the Retina Lens Retina Fovea Optic Nerve Optic Nerve Bottleneck Neurons Information Receptors: 108 95% Optic Nerve 106 5% After Polyak 1941 Visual Cortex ~1010 Mean Intensity
More informationTemporal Pattern Analysis
LIACS Leiden Institute of Advanced Computer Science Master s Thesis June 17, 29 Temporal Pattern Analysis Using Reservoir Computing Author: Ron Vink Supervisor: Dr. Walter Kosters 1 Contents 1 Introduction
More informationOutline. Neural dynamics with log-domain integrator circuits. Where it began Biophysics of membrane channels
Outline Neural dynamics with log-domain integrator circuits Giacomo Indiveri Neuromorphic Cognitive Systems group Institute of Neuroinformatics niversity of Zurich and ETH Zurich Dynamics of Multi-function
More informationIn the Name of God. Lecture 9: ANN Architectures
In the Name of God Lecture 9: ANN Architectures Biological Neuron Organization of Levels in Brains Central Nervous sys Interregional circuits Local circuits Neurons Dendrite tree map into cerebral cortex,
More informationBME 5742 Biosystems Modeling and Control
BME 5742 Biosystems Modeling and Control Hodgkin-Huxley Model for Nerve Cell Action Potential Part 1 Dr. Zvi Roth (FAU) 1 References Hoppensteadt-Peskin Ch. 3 for all the mathematics. Cooper s The Cell
More informationCMSC 421: Neural Computation. Applications of Neural Networks
CMSC 42: Neural Computation definition synonyms neural networks artificial neural networks neural modeling connectionist models parallel distributed processing AI perspective Applications of Neural Networks
More informationA Model for Real-Time Computation in Generic Neural Microcircuits
A Model for Real-Time Computation in Generic Neural Microcircuits Wolfgang Maass, Thomas Natschläger Institute for Theoretical Computer Science Technische Universitaet Graz A-81 Graz, Austria maass, tnatschl
More informationChapter 37 Active Reading Guide Neurons, Synapses, and Signaling
Name: AP Biology Mr. Croft Section 1 1. What is a neuron? Chapter 37 Active Reading Guide Neurons, Synapses, and Signaling 2. Neurons can be placed into three groups, based on their location and function.
More informationAssociative Memories (I) Hopfield Networks
Associative Memories (I) Davide Bacciu Dipartimento di Informatica Università di Pisa bacciu@di.unipi.it Applied Brain Science - Computational Neuroscience (CNS) A Pun Associative Memories Introduction
More informationA realistic neocortical axonal plexus model has implications for neocortical processing and temporal lobe epilepsy
A realistic neocortical axonal plexus model has implications for neocortical processing and temporal lobe epilepsy Neocortical Pyramidal Cells Can Send Signals to Post-Synaptic Cells Without Firing Erin
More informationFast classification using sparsely active spiking networks. Hesham Mostafa Institute of neural computation, UCSD
Fast classification using sparsely active spiking networks Hesham Mostafa Institute of neural computation, UCSD Artificial networks vs. spiking networks backpropagation output layer Multi-layer networks
More informationIntroduction. Previous work has shown that AER can also be used to construct largescale networks with arbitrary, configurable synaptic connectivity.
Introduction The goal of neuromorphic engineering is to design and implement microelectronic systems that emulate the structure and function of the brain. Address-event representation (AER) is a communication
More informationArtificial Neural Networks. Q550: Models in Cognitive Science Lecture 5
Artificial Neural Networks Q550: Models in Cognitive Science Lecture 5 "Intelligence is 10 million rules." --Doug Lenat The human brain has about 100 billion neurons. With an estimated average of one thousand
More informationHierarchy. Will Penny. 24th March Hierarchy. Will Penny. Linear Models. Convergence. Nonlinear Models. References
24th March 2011 Update Hierarchical Model Rao and Ballard (1999) presented a hierarchical model of visual cortex to show how classical and extra-classical Receptive Field (RF) effects could be explained
More informationMethods for Estimating the Computational Power and Generalization Capability of Neural Microcircuits
Methods for Estimating the Computational Power and Generalization Capability of Neural Microcircuits Wolfgang Maass, Robert Legenstein, Nils Bertschinger Institute for Theoretical Computer Science Technische
More informationInstituto Tecnológico y de Estudios Superiores de Occidente Departamento de Electrónica, Sistemas e Informática. Introductory Notes on Neural Networks
Introductory Notes on Neural Networs Dr. José Ernesto Rayas Sánche April Introductory Notes on Neural Networs Dr. José Ernesto Rayas Sánche BIOLOGICAL NEURAL NETWORKS The brain can be seen as a highly
More informationNeural Networks Lecture 6: Associative Memory II
Neural Networks Lecture 6: Associative Memory II H.A Talebi Farzaneh Abdollahi Department of Electrical Engineering Amirkabir University of Technology Winter 2011. A. Talebi, Farzaneh Abdollahi Neural
More informationHigh-conductance states in a mean-eld cortical network model
Neurocomputing 58 60 (2004) 935 940 www.elsevier.com/locate/neucom High-conductance states in a mean-eld cortical network model Alexander Lerchner a;, Mandana Ahmadi b, John Hertz b a Oersted-DTU, Technical
More informationFast neural network simulations with population density methods
Fast neural network simulations with population density methods Duane Q. Nykamp a,1 Daniel Tranchina b,a,c,2 a Courant Institute of Mathematical Science b Department of Biology c Center for Neural Science
More informationProcessing of Time Series by Neural Circuits with Biologically Realistic Synaptic Dynamics
Processing of Time Series by Neural Circuits with iologically Realistic Synaptic Dynamics Thomas Natschläger & Wolfgang Maass Institute for Theoretical Computer Science Technische Universität Graz, ustria
More informationBiosciences in the 21st century
Biosciences in the 21st century Lecture 1: Neurons, Synapses, and Signaling Dr. Michael Burger Outline: 1. Why neuroscience? 2. The neuron 3. Action potentials 4. Synapses 5. Organization of the nervous
More informationUsing a Hopfield Network: A Nuts and Bolts Approach
Using a Hopfield Network: A Nuts and Bolts Approach November 4, 2013 Gershon Wolfe, Ph.D. Hopfield Model as Applied to Classification Hopfield network Training the network Updating nodes Sequencing of
More informationSTDP Learning of Image Patches with Convolutional Spiking Neural Networks
STDP Learning of Image Patches with Convolutional Spiking Neural Networks Daniel J. Saunders, Hava T. Siegelmann, Robert Kozma College of Information and Computer Sciences University of Massachusetts Amherst
More informationNE 204 mini-syllabus (weeks 4 8)
NE 24 mini-syllabus (weeks 4 8) Instructor: John Burke O ce: MCS 238 e-mail: jb@math.bu.edu o ce hours: by appointment Overview: For the next few weeks, we will focus on mathematical models of single neurons.
More informationArtificial Neural Network and Fuzzy Logic
Artificial Neural Network and Fuzzy Logic 1 Syllabus 2 Syllabus 3 Books 1. Artificial Neural Networks by B. Yagnanarayan, PHI - (Cover Topologies part of unit 1 and All part of Unit 2) 2. Neural Networks
More informationHow to read a burst duration code
Neurocomputing 58 60 (2004) 1 6 www.elsevier.com/locate/neucom How to read a burst duration code Adam Kepecs a;, John Lisman b a Cold Spring Harbor Laboratory, Marks Building, 1 Bungtown Road, Cold Spring
More informationMachine Learning and Adaptive Systems. Lectures 5 & 6
ECE656- Lectures 5 & 6, Professor Department of Electrical and Computer Engineering Colorado State University Fall 2015 c. Performance Learning-LMS Algorithm (Widrow 1960) The iterative procedure in steepest
More informationModelling stochastic neural learning
Modelling stochastic neural learning Computational Neuroscience András Telcs telcs.andras@wigner.mta.hu www.cs.bme.hu/~telcs http://pattern.wigner.mta.hu/participants/andras-telcs Compiled from lectures
More informationStructure and Measurement of the brain lecture notes
Structure and Measurement of the brain lecture notes Marty Sereno 2009/2010!"#$%&'(&#)*%$#&+,'-&.)"/*"&.*)*-'(0&1223 Neurons and Models Lecture 1 Topics Membrane (Nernst) Potential Action potential/voltage-gated
More informationLecture 2: Learning from Evaluative Feedback. or Bandit Problems
Lecture 2: Learning from Evaluative Feedback or Bandit Problems 1 Edward L. Thorndike (1874-1949) Puzzle Box 2 Learning by Trial-and-Error Law of Effect: Of several responses to the same situation, those
More informationEmergence of resonances in neural systems: the interplay between adaptive threshold and short-term synaptic plasticity
Emergence of resonances in neural systems: the interplay between adaptive threshold and short-term synaptic plasticity Jorge F. Mejias 1,2 and Joaquín J. Torres 2 1 Department of Physics and Center for
More informationCN2 1: Introduction. Paul Gribble. Sep 10,
CN2 1: Introduction Paul Gribble http://gribblelab.org Sep 10, 2012 Administrivia Class meets Mondays, 2:00pm - 3:30pm and Thursdays, 11:30am - 1:00pm, in NSC 245A Contact me with any questions or to set
More informationLiquid Computing in a Simplified Model of Cortical Layer IV: Learning to Balance a Ball
Liquid Computing in a Simplified Model of Cortical Layer IV: Learning to Balance a Ball Dimitri Probst 1,3, Wolfgang Maass 2, Henry Markram 1, and Marc-Oliver Gewaltig 1 1 Blue Brain Project, École Polytechnique
More informationHow to do backpropagation in a brain
How to do backpropagation in a brain Geoffrey Hinton Canadian Institute for Advanced Research & University of Toronto & Google Inc. Prelude I will start with three slides explaining a popular type of deep
More informationNeural Networks. Hopfield Nets and Auto Associators Fall 2017
Neural Networks Hopfield Nets and Auto Associators Fall 2017 1 Story so far Neural networks for computation All feedforward structures But what about.. 2 Loopy network Θ z = ቊ +1 if z > 0 1 if z 0 y i
More informationNEUROMORPHIC COMPUTING WITH MAGNETO-METALLIC NEURONS & SYNAPSES: PROSPECTS AND PERSPECTIVES
NEUROMORPHIC COMPUTING WITH MAGNETO-METALLIC NEURONS & SYNAPSES: PROSPECTS AND PERSPECTIVES KAUSHIK ROY ABHRONIL SENGUPTA, KARTHIK YOGENDRA, DELIANG FAN, SYED SARWAR, PRIYA PANDA, GOPAL SRINIVASAN, JASON
More informationCausality and communities in neural networks
Causality and communities in neural networks Leonardo Angelini, Daniele Marinazzo, Mario Pellicoro, Sebastiano Stramaglia TIRES-Center for Signal Detection and Processing - Università di Bari, Bari, Italy
More informationNeural networks. Chapter 19, Sections 1 5 1
Neural networks Chapter 19, Sections 1 5 Chapter 19, Sections 1 5 1 Outline Brains Neural networks Perceptrons Multilayer perceptrons Applications of neural networks Chapter 19, Sections 1 5 2 Brains 10
More informationCompartmental Modelling
Modelling Neurons Computing and the Brain Compartmental Modelling Spring 2010 2 1 Equivalent Electrical Circuit A patch of membrane is equivalent to an electrical circuit This circuit can be described
More informationA biologically realistic cleanup memory: Autoassociation in spiking neurons
Available online at www.sciencedirect.com Cognitive Systems Research 12 (2011) 84 92 www.elsevier.com/locate/cogsys A biologically realistic cleanup memory: Autoassociation in spiking neurons Action editor:
More informationMathematical Methods for Neurosciences. ENS - Master MVA Paris 6 - Master Maths-Bio ( )
Mathematical Methods for Neurosciences. ENS - Master MVA Paris 6 - Master Maths-Bio (2014-2015) Etienne Tanré - Olivier Faugeras INRIA - Team Tosca November 26th, 2014 E. Tanré (INRIA - Team Tosca) Mathematical
More informationMemory capacity of networks with stochastic binary synapses
Memory capacity of networks with stochastic binary synapses Alexis M. Dubreuil 1,, Yali Amit 3 and Nicolas Brunel 1, 1 UMR 8118, CNRS, Université Paris Descartes, Paris, France Departments of Statistics
More informationMotivation Subgradient Method Stochastic Subgradient Method. Convex Optimization. Lecture 15 - Gradient Descent in Machine Learning
Convex Optimization Lecture 15 - Gradient Descent in Machine Learning Instructor: Yuanzhang Xiao University of Hawaii at Manoa Fall 2017 1 / 21 Today s Lecture 1 Motivation 2 Subgradient Method 3 Stochastic
More informationInformation Theory and Neuroscience II
John Z. Sun and Da Wang Massachusetts Institute of Technology October 14, 2009 Outline System Model & Problem Formulation Information Rate Analysis Recap 2 / 23 Neurons Neuron (denoted by j) I/O: via synapses
More informationCSE/NB 528 Final Lecture: All Good Things Must. CSE/NB 528: Final Lecture
CSE/NB 528 Final Lecture: All Good Things Must 1 Course Summary Where have we been? Course Highlights Where do we go from here? Challenges and Open Problems Further Reading 2 What is the neural code? What
More informationModel of a Biological Neuron as a Temporal Neural Network
Model of a Biological Neuron as a Temporal Neural Network Sean D. Murphy and Edward W. Kairiss Interdepartmental Neuroscience Program, Department of Psychology, and The Center for Theoretical and Applied
More informationarxiv:quant-ph/ v1 17 Oct 1995
PHYSICS AND CONSCIOUSNESS Patricio Pérez arxiv:quant-ph/9510017v1 17 Oct 1995 Departamento de Física, Universidad de Santiago de Chile Casilla 307, Correo 2, Santiago, Chile ABSTRACT Some contributions
More informationSingle neuron models. L. Pezard Aix-Marseille University
Single neuron models L. Pezard Aix-Marseille University Biophysics Biological neuron Biophysics Ionic currents Passive properties Active properties Typology of models Compartmental models Differential
More informationComputational Neuroscience Introduction Day
Computational Neuroscience Introduction Day 9.30am Introduction (C.Machens) 10am M1 (C.Machens) 10.15am M2 (V. Hakim) 10.40 break 11.00 Matching Law (S. Deneve) 14.00 Student presentations 11.20 Rescorla-Wagner
More informationWeek 4: Hopfield Network
Week 4: Hopfield Network Phong Le, Willem Zuidema November 20, 2013 Last week we studied multi-layer perceptron, a neural network in which information is only allowed to transmit in one direction (from
More information