Neuromorphic computing with Memristive devices. NCM group

Transcription

1 Neuromorphic computing with Memristive devices NCM group

2 Why neuromorphic? New needs for computing Recognition, Mining, Synthesis (Intel) Increase of Fault (nanoscale engineering) SEMICONDUCTOR TECHNOLOGY CHALLENGES Saturation of clock frequency + Energy consumption Von Neumann bottleneck Shift toward a new paradigm for computation BIO-INSPIRED COMPUTING to match the brain performances (low power consumption, fault tolerant, performances for RMS)

3 Supercomputer resources Purely digital NNET directions 1 11 neurons 1 15 synapses Emerging nanotechnologies New architecture concepts and integration strategies Custom IC, Mix analog/digital Multichip approach, i.e. with conventional technologies

4 The super computer approach As a matter of comparison, supercomputers have today the capacity of tens of petaflop/s (with an energy consumption in the range of MW) when the biological brain is estimated to be in the range of petaflop/s (with an energy consumption around 1W)

5 The custom IC approach Purely CMOS approach: how to design efficiently neuromorphic circuits with conventionnal technology

6 The nanotechnology approach V I Govoreanu, IEDM211) It is compatible with back end process on top of a CMOS substrate Lee, Nat.Mat.211 Torrezan, Nanotechnology 211)

7 NNET: two different stream Biological neural network OUTPUT Artificial neural network NEURON: Summation Thresholding Output activation INPUT Lots of new discoveries in neurosciences (characterization tools, non invasive probing, ) Better understanding of NNets organization, dynamics and principle SYNAPSES: Input weighting Weight adaptation

8 Practically, we can do: Pattern classification, Clustering, Prediction x 1 x 2 x 3 x 4 Input (+1 or -1) Two classes Output sgn() y= +1 if -1 if Multiple classes NNET basic principle x 1 x 2 x 3 x 4 x 5 input Synaptic weight output y 1 y 2 y 3 y m x 5 Post-neurons x n x n Pre-neurons Non linearly separable ensembles

9 NNet: basic principle 28x28 pixel array 1 classes 784 symapses 256x256 pixel array 1 classes synapses

10 Nano for artificial NNet x 1 input Synaptic weight output x 2 y 1 x 3 y 2 x 4 y 3 x 5 y m x n x 1 x 2 x 3 x 4 x 5.1 Current (A) y 1 y 2 y 3 y 4 y 5 Footprint 4F devices/cm 2 Voltage (V) -1,5, 1,5 -.1

11 Pattern classification bias x x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 x 9 pre-synaptic neurons (inputs) Input set of patterns X Desired output: d = +1 Input set of patterns T Desired output: d = -1 w + w - S S I + I - y post-synaptic neuron (output) w + w 1 + w 2 + w 3 + w 4+ w 5+ w 6+ w 7+ w 8+ w 9 + y + Learning rule: w ± i = ±α d (p) y (p) w - w - w - 2 w - 3 w 4- w 5- w - 6 w - 7 w - 8 w memristors x x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 x 9 y -

12 Pattern classification +V switch v pre-synaptic voltage x =+1 v x =-1 v post-synaptic voltage d =+1 v d =+1 4-step pulse sequence Training in parallel of all the devices -V switch t t t t A Input set of patterns X Desired output: d = +1 Input set of patterns T Desired output: d = -1 y + x x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 x 9 y - B bias x x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 x 9 pre-synaptic neurons (inputs) +V switch v v v v w + w - -V switch t t voltage across memristors t t S I + y I - Learning rule: S postsynaptic neuron (output) w i ± = ±α d (p) y (p)

13 Pattern classification A B V training =.9V V training = 1V Number of pattern # Initial 1 epochs V training =.9V 7 epochs V training =1V X T ( 2mV) I + - I - (A) Training events # Figure 4

14 Nano for bio-mimetic time today

15 Nano for bio-mimetic S D N n neurotransmiter (NT) activated at spike n Recovery of NT with a time t d I f N ) n ( n V OUT charge of N n holes in the NP at spike n Pentacene thin film I n f ( Nn) Gold nanoparticles Discharge of the holes with a time t d

16 Nano for bio-mimetic 1Khz 1Khz TiO2 TiO2-x Capacity (F) 1E-11 1E-12 1E-3 Conductance (S) 1E-4.1 Current (A) -1,5-1, -,5,,5 1, 1,5 V bias Voltage (V) -1,5, 1,5 -.1

17 Nano for bio-mimetic 4,µ R axon = 2k OFF State,2,4 Pt 2,µ, 2,µ R axon = 5k TiO2-x Pt TiO2 EPSC (A) 1,µ, 2,µ 1,µ, 2,µ 1,µ, R axon = 1k R axon = 15k (Alibart, Adv. Funct. Mat, 211 EPSP (V) 2 1,2,4 A

18 By mechanical mask Nano in between: STP to LTP with memristive devices By EBL X-electrodes sizes: 1µm, 5nm, 2nm, 1nm, 8nm, 4nm, 32nm, 23nm

19 Nano in between: STP to LTP with memristive devices b) ) 1,µ C I (V) Pt (25nm) TE Ag 2 S (6nm) Ag (25nm) BE I [ A ] 5,µ B SiO 2 Si, D A V [ V ] c) Reset 1µm 1µm Natural Relaxation (After 5min)

20 STP to LTP with memristive devices Tunable volatility: key parameters Number of pulses cumulative effect V pulse [V] I C =1uA; Vsw=,44V R max t [s]

21 Nano in between: STP to LTP with memristive devices Ic=1 A Ic=25 A Ic=8 A G 1s [ ms ] 1.1 1E-3 G max = G 1s G max > G 1s = G ON Volatility G max [ ms ] Bio-inspired model La Barbera et al., ACSnano

22 Nano in between 22 How neuro can help computing INPUTS OUTPUTS Renforcement STP LTP

23 conclusions Design of new materials at the nanoscale (resistive, capacitive, ionics, ) Understanding of dynamical systems (not only static, but dynamic memory) Engineering of more complex systems (synapse to neuron to network)