A 68 Parallel Row Access Neuromorphic Core with 22K Multi-Level Synapses Based on Logic- Compatible Embedded Flash Memory Technology

Transcription

1 A 68 Parallel Row Access Neuromorphic Core with 22K Multi-Level Synapses Based on Logic- Compatible Embedded Flash Memory Technology M. Kim 1, J. Kim 1, G. Park 1, L. Everson 1, H. Kim 1, S. Song 1,2, S. Lee 2 and C. H. Kim 1 1 Dept. of ECE, University of Minnesota 2 Anaflash Inc.

2 Outline Background Logic-Compatible eflash based Synapse Neuromorphic Core Design 65nm Test Chip Results Conclusions 2

3 Artificial Neural Network (ANN) Multi-layer perceptron (MLP) Unit perceptron Inputs Weights X 0 X 1 X 2 Output X 0 X 1 W0 W1 Integrator Ʃ Activation function Output X 2 W2 X i X i Wi Input layer Hidden layer Output layer MLP: input/hidden/output layers Unit perceptron: multiply-accumulate operation + activation function 3

4 ANN Digital vs Analog Implementation Digital implementation Analog Implementation X i X 0 Multiplier W 0 W i Weights X 0 W 0 BL X i Adder Threshold W i I Sum I Ref Accumulator Comparator Pros : Digital CMOS Issues : Large area, large power consumption Output Sense Amp. Output Pros : Current summation replaces complex digital blocks Issues: Sensitive to PVT variation, requires good synaptic device 4

5 Memory Options for Synapse Circuit This work Device SRAM MRAM RRAM PCRAM eflash Cell Configuration M 1 FG M 2 NW NW M 3 D S Nonvolatile? No Yes Yes Yes Yes Tunable? No No No Yes Yes Logic Compatible? Multi level Weights? Yes Not yet No No Yes No No No Yes Yes Area/bit 150F 2 40F 2 60F 2 40F 2 ~500F 2 5

6 Dual-poly vs. Single-poly eflash Dual-Poly eflash CG FG Single-Poly eflash FG M 1 FG NW M 3 D S D S D S D S D PW NW P-SUB NW FG : Floating Gate CG : Control Gate M 1 M 3 M 2 M 2 S NW (W M1 >> W M2, W M3 ) Needs additional masks to form floating gate (FG) 6

7 Proposed Synapse: Logic Compatible eflash Dual-Poly eflash CG FG Single-Poly eflash FG M 1 FG NW M 3 D S D S D S D S D PW NW P-SUB NW FG : Floating Gate CG : Control Gate M 1 M 3 M 2 M 2 S NW (W M1 >> W M2, W M3 ) Floating gate: Back-to-back connected gate Logic-compatible nonvolatile memory solution Program verify allows precise weight programming 7

8 Proposed Synapse: Logic Compatible eflash 2.0V X FG: Floating Gate X 2.0V 0V W BL : 0.6V I CELL X W X W I CELL µA µA µA µA µA µa Cell current proportional to X W (=0µA, 5µA, or µa) BL voltage pinned at 0.6V during read operation Cell Current (µa) nm,1.0V,25 C, X(VRD)=0.8V VFG V VFG V VFG 0.046V X=1, W=2 X=1, W=1 X=1, W=0 or X=0, W=0,1,2 BL Voltage (V) 8

9 Proposed Synapse: Logic Compatible eflash 2.0V X FG: Floating Gate Even BL : 0.6V X W I 0 W 1 X W 0 X W 1 I CELL0 I CELL1 I CELL0 I CELL1 W 0 W 1 X 2.0V 0V Positive Weight Odd BL : 0.6V Negative Weight µA µa -µa µA 5µA -5µA µA 0µA 0µA µA 0µA 5µA µa 0µA µa Negative Weights Positive Weights Even and odd bitline pair realizes 5 level weights 9

10 Detailed Erase and Program Operations Erase Operation RWL(0V) PWL(0V) W M1 = 8xW M2,3 M 1 WWL(HV) EWL(0V) CSL(0V) M 2 e - ~0V BL(0V) OFF OFF RWL(VDD) PWL(HV) ~HV WWL(HV) EWL(0V) CSL(VDD) Programmed BL A (0V) BL B (VDD) ~HV e - e - Program-Inhibited via self-boosting FN tunneling utilized for erase and program Program inhibition of unselected cells via self-boosting Cell A Program Operation ON OFF Cell B OFF OFF [8] S. Song, JSSC 2013 (UMN)

11 Proposed Bitline Pair Implementation WL<0> 1 Even BL Odd BL WL<1> WL<2> WL<67> W P W P W P. Th P W N W N W N. Th N Spiking criteria W P -(-Th P ) > W N +(+Th N ) Output generated in a single current summation and thresholding cycle I EXC I INH W P Postive weight W N Th Negative weight Threshold Output Sensing neuron 11

12 68 Row x 160 Column Core Architecture High Voltage Switch WRITE SCAN & BL DRIVER WRITE SCAN RWL BL S 1 Pixel-IN SCAN & Level Shifter Control VPP1~4 Charge Pump HVS HVS HVS HVS Ref. Gen. WWL67 PWL67 5T eflash Array WWL0 PWL0 BL0 BL159 Verify/Sensing Neuron Circuit (Core Devices) SPIKE-OUT SCAN 5T eflash array, high voltage switches, BL sensing circuits Input data loaded on to 64 read wordlines, 4 rows for threshold RWL PWL WWL EWL CSL V REG - + FG EV VEV BL EV M1 M2 V REG FG OD VOD SPIKE-OUT SCAN BL OD V REF S1 M3 S2 I REF PWL FGIN WWL EWL CSL M 1 M 3 M 2 S 2 5T Eflash Unit Cell 12

13 Inference and Verify Operations Inference mode 2-cycle current verify mode WRITE SCAN WRITE SCAN WRITE SCAN RWL PWL BL EV BL OD RWL PWL BL EV BL OD RWL PWL BL EV BL OD FG EV FG OD FG EV FG OD FG EV FG OD WWL EWL CSL Pass-TR V REG - + I EV V REG On - + I OD On WWL EWL CSL V REG - + V REG On - + Off WWL EWL CSL V REG - + V REG Off - + On V EV Off V OD SA(Sense - + V REF Amp.) SPIKE-OUT SCAN Off I REF Inference mode: Compares BL pair currents Verify mode: Compare BL currents with a common ref. current V EV - + Off V OD V REF SPIKE-OUT SCAN On I REF V EV - + On V OD V REF SPIKE-OUT SCAN I REF Off 13

14 65nm Die Photo and Feature Summary BL Sensing 10µm WL Drivers w/ HVS EFlash Based Synapse Array WL Drivers w/ HVS BL Driver 600µm Technology 65nm CMOS Circuit Area 10 X 600 μm 2 VDD (Core, IO) # of Neurons # of Synapses Throughput Power 1.0V / 2.5V K (=68x320) 1.28G pixels/s per core (tread : 50ns) 15.9µW (per neuron) 14

15 Program and Program Inhibition Characteristics Cell Current (µa) Avg. current of 0 cells, 25 C, VRD=0.8V, VBL=0.6V PGM Bias 7à8.8V (0.1V Step) Pulse Count (20µs width) Program mode Cell Current (µa) PGM Bias 8.8à7V (0.1V Step) Pulse Count (20µs width) Program inhibition mode 15

16 Multi-level Program Sequence Weight 0 PGM & 0 Verify Weights 1,2 PGM & Verify Weight 1 PGM & Verify 8.8V 7.4V 7.1V 7.1V 7.1V 7.1V Cell Current (µa) µs 40µs 5µs 5µs 20 Weight 1,2 Weight 2 Weight 1 0 Weight 0 Program bias: 8.8V à 7.4V à 7.1V Target cell current: 0µA, 5µA, and µa 16

17 Cell Current (µa) Programmed Cell Current Variation Weight 0,1,2 Cell Current Variation, 25 C, VRD=0.8V, VBL=0.6V BL # Weight 2 Cells Weight 1 Cells Weight 0 Cells Cell current variation less than 0.8µA after program-verify operations Data shows that a higher number of levels is possible Cell Current (µa) WL # Weight 2 cells Weight 1 cells 2 Weight 0 cells

18 Number of Program Pulses Cell Current (µa) C, VRD=0.8V, VBL=0.6V Weight 2 target Weight 1 target Program Pulse Count Initial cell current variation is 8µA Weight 2 and weight 1 cells completely programmed after 20 and 30 pulses, respectively. 18

19 MNIST Digit Recognition Accuracy Results MNIST dataset: Handwritten Digit images Training Inference 256 axon signals 16X16 Pixel data Learning algorithm (RBM) Synaptic weights Neuromorphic core (Test chip) Linear classifier (Spike comparison w/ trained ref.) Hidden units: Spike set Match Digit recognition eflash-based core Final results Recognition accuracy is 91.8% for K MNIST test images, which is close to the software model s 93.8% accuracy # of Recognitions Image 0 Image 1 Image 2 Image 3 Image 9 Image 8 Image 7 Image 6 Image 4 Image Output Neuron Index # of Recognitions

20 Retention Test Results Cell Current (µa) K Pre-cycles, T bake =150 C Weight 2 (mean) ±3σ Weight 1 (mean) Weight 0 (mean) Excellent retention characteristics Reprogramming is not necessary A higher number of levels is possible Before baking Bake Time (hrs) 20

21 Comparison Table This work ISSCC 18 [6] ISSCC 18 [4] ISSCC 18 [5] IEDM 17 [2] IEDM 17 [3] Application Handwritten digit recognition Handwritten digit recognition Handwritten digit recognition Machine learning classifier Computing in memory Handwritten digit recognition Technology 65nm 65nm 65nm 65nm 150nm 180nm Voltage 1.0V 1.0V 1.0V 1.0V Non volatile? YES (Eflash) YES (ReRAM) NO (SRAM) NO (SRAM) 1.8V YES (ReRAM) 2.7V YES (eflash) Logic Compatible? YES NO YES YES NO NO Program-verify? YES NO NO NO NO YES Weight Resolution 2.3 Bits (5 levels) 3 Bits 1 Bit 1 Bit 2 Bits N/A # of Currents Summed Up 68 Cells 14 Cells 30 Cells 4 Cells 2 Cells N/A [2] W. Chen, et al., IEDM, [3] X.Guo, et al., IEDM, [4] W.Khwa, et al., ISSCC, [5] S. Gonugondia, et al., ISSCC, [6] W.Chen, et al., ISSCC,

22 Conclusions A logic-compatible 5T eflash based neuromorphic core demonstrated in 65nm CMOS Key features Non-volatile weight storage in a logic CMOS process Precise multi-level weights enabled by program verify 68 row parallel access Test chip results show 91.8% digit recognition accuracy 22