How to outperform a supercomputer with neuromorphic chips

Transcription

1 How to outperform a supercomputer with neuromorphic chips Kwabena Boahen Stanford Bioengineering boahen@stanford.edu Telluride Acknowledgements Paul Merolla John Arthur Joseph Kai Lin Hynna BrainsInSilicon.stanford.edu Collaborators Bert Shi Rajit Manohar Support Packard Foundation NSF CAREER & BITS ONR YIP & MURI CRCNS - NIMH R01 Pioneer Award - NIH K Boahen, May

2 Dendritic recording Serial Scanning EM Two-photon LM Hausser et al 1997 Denk et al 2005 Reid et al 2005 Levels of Investigation Analyze Simulate Experiment Churchland & Sejnowski

3 Cell Compartment u τ Ion-channel 1 u du dt ( V ) ( V) = α ( V ) β ( V ) ( ) ( V ) u u V u = τ 1 = α V + β V α ( V ) α V + β V ( ) ( ) ( ) ( ) Shenoy et al Blue Gene Supercomputer Two spiral galaxies Point mass approx. Hubble Telescope 1999 F j = Gmj i m r i 2 ij Jun Makino GRAPE6 Supercomputer Law of gravity 3

4 2 years 5 years Neurogrid (chips) Neurocore (neurons) Total (neurons) M 8 8 1K 1K 64M Speed (teraflops) ,200 α ( V ) 1 u u β ( V ) du u ( V) u = dt τ ( V ) 1 τ ( V ) = α V + β V α ( V ) u ( V) = α V + β V ( ) ( ) ( ) ( ) Decoded Image Ion channel Transistor Spike Output Zaghloul & Boahen

5 Limitations of neuromorphic chips as simulation platforms Specialized circuits Neuronal properties are fixed Hardwired connections Synaptic organization is fixed Proposed solutions Implement Hodgkin-Huxley Huxley model Describes various ion-channels Implement softwires User can specify connectivity Boahen & Andreou 1992 Ion-channel population Huguenard & McCormick 1992 Steady-state level Time constant (ms) Transistor circuit u H A first in silicon! V O V C u V C I T u L Hynna & Boahen

6 Ion channel Transistor Transistors and ion-channels are analogs Gating-particles and electrons both overcome their energy barriers at exponential rates. This observation yields a compact silicon model We modulate the transistor s s barrier-height the same way the ion-channel channel s s is modulated. Energy Closed Open State Energy Source Drain Channel First-order kinetics 1 u α ( V ) β ( V ) Time-constant τ 1 ( V ) = α ( V ) + β ( V ) u Differential equation du = α V dt u V u = τ ( )( 1 ) β ( ) 1 ( ) ( )( u u V ) V Steady-state u ( V ) = α α ( V ) ( V ) + β ( V ) Opening rate log(u) Closing rate 6

7 e κ V o e κv c 0 e κv o e κv c Data from Huguenard and McCormick, 1992 Soft wires Chip 2 Computer STDP Chip CPLD RAM post pre USB RAM Chip 1 Silicon cortex [Douglas et al 99] IFAT [Andreou[ et al 01] Neurotrope [Taba & Boahen 03] Arthur & Boahen

8 Hardwired Neurons Neurons (n) ( Synaptic sites (n( 2 ) Softwired Neurons RAM (nf( nf) Addr Fanout: f = 3 Plus Arbors Neurons RAM (n)( Addr f = 8 Reduce cost from O(n 2 ) to O(n) ) by exploiting: Sparse connectivity ( (f << n) Local connectivity ( (f clustered) Cable-like like arbor and binary synapses BioE332 s s 1,280-neuron simulator Computer USB STDP Chip Demo CPLD RAM Students perform nine labs on: Synapses and neurons Spiking, adaptation, bursting Synapse-Neuron Interaction Phase-response curve, locking Inhibitory Networks Rhythms, synchrony, delays Excita-Inhibitory Networks Entrainment, binding Synaptic Plasticity Spike-timing dependent plasticity Enhancing synchrony Episodic Memory Storing and recalling patterns 8

9 1-D D grid: 32,768 neurons on 4 chips Multichip model of cortical orientation hypercolumns Shih, Boahen et al 2005 Even Odd neurons 1 cm 8,192 neurons TSMC 0.25 um 9.8 mm 2 ; 3 mw Steerable orientation Even/odd, on/off Bus d pitch Grid T = (2ns inch) d n bus d pitch pitch chip T link = (2ns inch) d pitch Bus capacity drops as more chips are added Grid s s capacity remains the same (expandable) 9

10 Neuroscope pixels4pixels bit SRAM TSMC 0.18um mm Lin, Boahen et al Delay-insensitive link 63M spikes/sec! Lin, Boahen et al Word-serial format 10

11 Lin, Boahen et al Delivered Filtered Programming packet Address-event packet 11

12 Neurogrid RAM Cortex Vertical RAM Resistors Horizontal Arbor Connecting 6000 synapses On-chip RAM: Vertical projections (f = 6) Off-chip RAM: Horizontal projections (f = 10) Resistive mesh: Adjustable electro- tonic spread (f ~100) Layered projections Forward Backward Lateral Two-compartment abstraction Modulation Luscher et al. 01 Projections target apical or basal dendrites Terminate in distinct cortical layers Apical input modulates pyramidal cell s response to basal input Back-propagating spikes recruit Ca current if dendrite is depolarized 12

13 Multi- versus two-compartment L4 L3 L5 Conductances Mainen & Sejnowski 96 Same ion-channel types & densities Only morphology differed Two-compartment captured this Visual areas MT Backward V1 Forward Society for Neuroscience 2005 Van Essen & Fellerman 1991 Sillito

14 Summary: We have developed three enabling technologies for Neurogrid Analog VLSI for real-time simulation Models ion-channel populations Digital VLSI for programmability Specifies synaptic connections Grid network for expandability Relays spikes from chip to chip Putting supercomputers on neuroscientists desks with real-time cortex-scale simulation is feasible in 5yrs. Circuit analysis Assuming N1 is in saturation: du dt uh ( ) = V κvo uv κvc ul C Ids0e e e Ids0e e du I I = ( 1 ) dt CU CU O H C L e u e u ds0 κv u ds0 κv u T u = uv uh e Ids κ I α( VO) = e and β( V ) = e CU 0 V u ds0 κv u C T CUT O H C L Rates are exponentially dependent if V O and V C vary linearly with V mem Slopes should be complementary V O increases while V C decreases for activation variable The reverse is true for an inactivation variable T 14