ESE566A Modern System-on-Chip Design, Spring 2017 ECE 566A Modern System-on-Chip Design, Spring 2017 Class Project: CNN hardware accelerator design

Transcription

1 ECE 566A odern System-on-Chp Desgn, Sprng 2017 Class Project: CNN hardware accelerator desgn 1. Overvew Background knowledge Convolutonal neural network bref ntroducton CNN summarzed n 4 steps NIST Dataset Introducton of CNN source code C/C Code structure Explanaton of each layer Forward propagaton process Error back propagaton process atlab reference resource Possble optmzatons usng parallelsm n referenced C/C++ code How to get started Requred Project Delverables Basc functonalty Performance expectaton Report expectaton Project submsson Acknowledgement Appendx A... 13

2 1. Overvew Convolutonal neural networks have been wdely employed for mage recognton applcatons because of ther hgh accuracy, whch they acheve by emulatng how our own bran recognzes objects. The possblty of makng our electronc devces recognze ther surroundngs have spawned a vast number potental of useful applcatons, ncludng vdeo survellance, moble robot vson, mage search n data centers, and more. The ncreasng usage of such applcatons n moble platforms and data centers have led to a hgher demands for methods that can compute these computatonalnsenstve networks n a fast and power effcent way. One such method s by usng applcaton specfc hardware accelerators. Ths project wll explore the desgn and mplementaton of convolutonal neural networks (CNNs) n hardware wth the ntenton of mprovng energy effcency over tradtonal mplementaton n software on a general-purpose CPU. The overall goal s to buld an energy effcent hardware accelerator that mplements the forward propagaton n CNN to recognze the NIST handwrtng dataset. 2. Background knowledge 2.1 Convolutonal neural network bref ntroducton Convolutonal Neural Networks (CNN), s a type of advanced artfcal neural network. It dffers from regular neural networks n terms of the flow of sgnals between neurons. Typcal neural networks pass sgnals along the nput-output channel n a sngle drecton, wthout allowng sgnals to loop back nto the network. Ths s called a forward feed. Whle forward feed networks were successfully employed for mage and text recognton, t requred all neurons to be connected, resultng n an overly-complex network structure. The cost of complexty grows when the network has to be traned on large datasets whch, coupled wth the lmtatons of computer processng speeds, result n grossly long tranng tmes. Hence, forward feed networks have fallen nto dsuse from manstream machne learnng n today s hgh resoluton, hgh bandwdth, mass meda age. A new soluton was needed. In 1986, researchers Hubel and Wesel were examnng a cat s vsual cortex when they dscovered that ts receptve feld comprsed sub-regons whch were layered over each other to cover the entre vsual feld. These layers act as flters that process nput mages, whch are then passed on to subsequent layers. Ths proved to be a smpler and more effcent way to carry sgnals. In 1998, Yann LeCun and Yoshua Bengo tred to capture the organzaton of neurons n the cat s vsual cortex as a form of artfcal neural net, establshng the bass of the frst CNN. 1

3 2.2 CNN summarzed n 4 steps There are four man steps n CNN: convoluton, subsamplng, actvaton and full connectedness. 1) Step 1: convoluton Fg.1 The 4 key layers of a CNN The frst layers that receve an nput sgnal are called convoluton flters. Convoluton s a process where the network tres to label the nput sgnal by referrng to what t has learned n the past. If the nput sgnal looks lke prevous mages t has seen before, the reference sgnal wll be mxed nto, or convolved wth, the nput sgnal. The resultng output sgnal s then passed on to the next layer, as what Fg.2 shows. Fg.2 convoluton Convoluton has the nce property of beng translatonal nvarant. Intutvely, ths means that each convoluton flter represents a feature of nterest (e.g whskers, fur), and the CNN algorthm learns whch features comprse the resultng reference (.e. cat). The output sgnal strength s not dependent on where the features are located, but smply 2

4 whether the features are present. Hence, a cat could be sttng n dfferent postons, and the CNN algorthm would stll be able to recognze t. 2) Step 2: Subsamplng Inputs from the convoluton layer can be smoothened to reduce the senstvty of the flters to nose and varatons. Ths smoothng process s called subsamplng, and can be acheved by takng averages or takng the maxmum over a sample of the sgnal. Examples of subsamplng methods (for mage sgnals) nclude reducng the sze of the mage, or reducng the color contrast across red, green, blue (RGB) channels. 3) Step 3: Actvaton Fg.3 Subsamplng The actvaton layer controls how the sgnal flows from one layer to the next, emulatng how neurons are fred n our bran. Output sgnals whch are strongly assocated wth past references would actvate more neurons, enablng sgnals to be propagated more effcently for dentfcaton. CNN s compatble wth a wde varety of complex actvaton functons to model sgnal propagaton, the most common functon beng the Rectfed Lnear Unt (ReLU), whch s favored for ts faster tranng speed. 4) Step4: Fully connected The last layers n the network are fully connected, meanng that neurons of precedng layers are connected to every neuron n subsequent layers. Ths mmcs hgh level reasonng where all possble pathways from the nput to output are consdered. 5) Step 5: Loss (Durng tranng step) When tranng the neural network, there s addtonal layer called the loss layer. Ths layer provdes feedback to the neural network on whether t dentfed nputs correctly, 3

5 and f not, how far off ts guesses were. Ths helps to gude the neural network to renforce the rght concepts as t trans. Ths s always the last layer durng tranng. 2.3 NIST Dataset The NIST database of handwrtten dgts, avalable from ths page, has a tranng set of 60,000 examples, and a test set of 10,000 examples. It s a subset of a larger set avalable from NIST. The dgts have been sze-normalzed and centered n a fxed-sze mage. It s a good database for people who want to try learnng technques and pattern recognton methods on real-world data whle spendng mnmal efforts on preprocessng and formattng. The orgnal black and whte (blevel) mages from NIST were sze normalzed to ft n a 20x20 pxel box whle preservng ther aspect rato. The resultng mages contan grey levels as a result of the ant-alasng technque used by the normalzaton algorthm. the mages were centered n a 28x28 mage by computng the center of mass of the pxels, and translatng the mage so as to poston ths pont at the center of the 28x28 feld. In ths project, we usng the followng tranng and test datasets. tran-mages-dx3-ubyte.gz: tranng set mages ( bytes) tran-labels-dx1-ubyte.gz: tranng set labels (28881 bytes) t10k-mages-dx3-ubyte.gz: test set mages ( bytes) t10k-labels-dx1-ubyte.gz: test set labels (4542 bytes) 3. Introducton of CNN source code C/C++ Ths CNN code s a basc convoluton neural network, manly used for handwrtten numeral recognton. The dataset selected for tranng test s NIST handwrtten dgtal lbrary. Ths CNN manly ncludes a basc mult-layer convoluton network framework, convoluton layer, subsamplng (poolng) layer, and fully connected sngle-layer neural network output layer, but wthout other CNN mportant concepts such as Dropout, ReLu, etc. Ths convolutonal network has fve layers. The man structures contan convoluton layer, poolng layer, convoluton layer, poolng layer and fully connected sngle-layer neural layer (output layer), as Fg.4 shows. 4

6 Fg.4 The overvew structure of CNN source code 3.1 Code structure cnn.cpp cnn.h: CNN network functon and structure mnst.cpp mnst.h: nst dataset process functon and data structure mat.cpp mat.h: atrc functon, convoluton process, 180 degree rotaton, etc. man.cpp man: Functon and test functon 3.2 Explanaton of each layer (1) convoluton layer Input s the gray mage, and the gray mage s convolved wth sx 5 5 templates respectvely. And then sx convolutonal mage wll be obtaned. Also each pxel n the mage should be added wth a weght and passed through an actvaton functon. And n the fnal, the output wll be obtaned. Tps for actvaton functon: 5

7 Fg.5 Sgmod and Tanh functon dagram In the neural network, there are two man reasons why the actvaton functon should be used: frst, to clamp the data wthn a certan range (such as Sgmod functon to compress the data between -1 and 1). t should not be too hgh nor too low. Second, to ntroduce the nonlnear factor. Because the expresson ablty of the lnear model s not enough. The Sgmod and Tanh are two mostly used actvaton functons (Sgmod and Tanh functons are shown n Fg.5) n the conventonal neural network. In the source code, t uses Sgmod actvaton functon. (2) samplng layer and S4 (Poolng layer) samplng layer are also called poolng layer. Poolng layer s manly used for reduce the dmenson of data whch are used for processng. The commonly used poolng strateges are max poolng and average poolng max poolng: Select the maxmum number of pxels n the current block to represent the current local block average poolng: select the average value of the pxels n the current block to represent the current local block (3) convoluton layer Ths convoluton layer s a fully connected convoluton layer. The convoluton formula for the output s as below: 6 I j = φ ( W j I + b j ), j = 1 12 =1 6

8 I represents the mage, W represents the convoluton template, b represents bas, φ represents the actvaton functon, represents the nput mage order number ( = 1 6), and j represents the output mage order number (j = 1 6) In the convoluton layer, t can be seen that the nputs are mage, and the outputs are mage. The requred tranng parameters are convoluton templates w and 12 bas b (each bas assocated wth each template s same) (4) Output layer : Go through the poolng layer S4, mage wll be obtaned. Expand all the mages nput one dmenson, we wll get a vector of = 192 bts. The output layer s composed of 192 bts nput and 10 bts fully connected sngle-layer neural network. It contans 10 neurons, and each neuron are connected wth 192 bts nput, whch means each neuron has 192 bts nput and 1 bt output. The processng formula s shown as below: 192 I j = φ ( W j I + b j ), j = 1 10 =1 j represents the order number of output neuron, and I represents the order number of nput. Therefore, n ths layer, there are weght w, and 10 bas b. 3.3 Forward propagaton process The forward propagaton process actually refers to nput mage data and get the output. Here we wll ntroduce the convoluton layer (or ) forward propagaton process (, S4 and layers are omtted because the process of them are qute strat-forward). The convoluton layer (or ) has 6 convoluton templates. Each template has 6 outputs for 1 nput, whch means t wll have 6 output. The mage convoluton formula s shown below: c r (c 1) (r 1) g(x, y) = w(s, t) f (x s, y t ), c, r = 2k s=0 t=0 3.4 Error back propagaton process The error back propagaton method s the bass of neural network learnng. Actually t s the process that use the gradent descent method to compute the weght of mnmum error. 7

9 The updatng formula of gradent descent method s shown as below: W n+1 = W n W n W n = η E e n W n = η E n e u n u n W n = ηδn X n u n = W n X n δ n = E e n u n Here, W represents weght, E represents the error energy, n represents nth teraton, η represents the learnng rate, Y represents output, and δ represents the local gradent. Backward propagaton datapath s used for CNN tranng. However, n ths project, you don t need to mplement backward propagaton datapath n the CNN hardware accelerator desgn. Because the man functonalty for ths desgn s to do the handwrtng dataset classfcaton whch actually s the CNN test part not tranng part. If you are nterested n the entre pcture of ths CNN code ncludng the backward propagaton. You can read the Appendx A The analyss of the back propagaton process n ths CNN code. 3.5 atlab reference resource Ths referenced C/C++ code provded n gthub s the C/C++ mplementaton of DeepLearnngToolBox (atlab), f you want to learnng ths CNN work n deep, such as analyzng the ntermedate results of each layer, you can download DeepLearnngToolBox n followng webste and have fun wth t Possble optmzatons usng parallelsm n referenced C/C++ code A vast amount of the computaton requred by a CNN can be parallelzed. Thus, n order to acheve the processng of the network t s mportant that these potental parallelsms are dentfed and exploted. The most obvous beng: 1. The convoluton of a matrx n n usng a k k kernel conssts of (n k + 1) (n k + 1) convoluton operaton, whch each can be done n parallel. Thus convolutng the whole matrx could potentally take only the tme t takes to perform one convoluton operaton. 8

10 2. The subsamplng/poolng operaton can also be parallelzed by poolng all of the ndvdual sub-matrces at the same tme. 3. The computaton of each of the ndvdual feature maps and ther correspondng subsamplng/poolng. 4. It s also possble to parallelze the computaton of the feature maps that take more than one matrx as nput. Ths s the case n the subsequent layers after the frst. 5. The actvaton of each neuron n the fully connected layer. One opton s to parallelze them by creatng a bnary tree multpler, where you have n unts compute the product of the nput and ts respectve weght, then you use n unts to add two of the results each, 2 and so on untl you have a sngle value. Ths wll reduce the tme t takes from n tme to log 2 n tme f they can all be done n parallel. 5. How to get started You are expected to accept the lab assgnment n the lnk by clck the button Accept ths assgnment. If you are the frst tme to use gthub, you should generate your own publc key and add t to your gthub account, or you wll have permsson dened when you gt clone the repostory form our gthub classroom. When you add the ssh publc key, you can follow the lnk below: And then create a folder n your own lnux server account and gt clone your lab assgnment repostory. There are the command lnes you may refer below: % mkdr name_folder % cd name_folder % gt clone gt@gthub.com:wustl-ese566/class-project-team#.gt % cd class-project-team# / (.e., cd class-project-team1/) Ths repostory class-project-team#/ contans three folders: CNN_Referenced_Code/CNN: a folder that contans the C/C++ source code of CNN desgn CNN_Referenced_Code/nst: a folder that contans the NIST handwrtng dataset CNN_Accelerator: a folder contans that testbench, and your own Verlog code should be n ths folder 9

11 CNN_Referenced_Code/CNN folder have 8 fles: cnn.cpp: each layer defnton and tranng, test functon and etc. cnn.h: varables of each layer n ths CNN and functon clam mnst.cpp: nst dataset mage and label read functon mnst.h: varables of nst dataset mage and functon clam mat.cpp atrc computaton, convoluton computaton, 180 degree rotaton, max computaton and summaton etc. mat.h: varables defnton and functon clam of mat.cpp man.cpp man: tranng functon and test functon nst.bn: 16bts fxed pont weght matrx dataset CNN_Referenced_Code/nst folder have 4 fles, these 4 fles are downloaded from nst handwrtng dataset. CNN_Accelerator folder have 1 fle: CNN_tb.v: ths s an ncompleted testbench, t wll provde the nterface for feedng n test mage and weght matrx Please be sure to source the class setup scrpt usng the followng command before complng your source code: module add ese Requred Project Delverables 6.1 Basc functonalty Refer to CNN C/C++ code and paper to buld your own CNN hardware accelerator. Your CNN hardware accelerator desgn should acheve the forward propagaton functon of CNN to accomplsh the handwrtng dataset classfcaton. For the CNN functonalty evaluaton, we wll randomly feed n a handwrtng mage and the accelerator your team have bult should compute the rght classfcaton of the mage whch s fed n. (Don t worry about correct predcton rate. We wll use a 100% predctable mage we have tested to evaluate your work) Note: we wll provde a 16 bts fxed pont weght matrx whch contans, and weght values. 6.2 Performance expectaton 10

12 Although there s no specfc performance targets n the project as long as you are able to demonstrate the basc functonalty (as dscussed n Secton 6.1), t s mportant to demonstrate the sellng pont of your desgn, be t low latency, hgh throughput, low power, or small area. You want to be able to demonstrate sgnfcant desgn efforts and technques n your project to optmze for one or more performance specfcatons. You can refer to the paper we revewed n class (DanNao and Eyerss) and the optmzaton technques suggested n Secton 4 for nspraton for key feature of your accelerator desgn such as ppelnng, matrx tlng, or other datapath optmzaton. You should organze your project presentaton and report to hghlght the desgn features n your hardware accelerator, as f you are wrtng a reference paper and gvng a conference presentaton to showcase your desgn wth compellng results and rgorous analyss. To report credble speed, power, and area numbers, you need to go through all the steps you dd n lab1: Use Synopsys VCS to comple the Verlog source code of CNN hardware accelerator Use Desgn Compler to do the synthess of CNN hardware accelerator Use Cadence Encounter to do the place and route of CNN hardware accelerator To evaluate your desgn, we need all the source Verlog codes, the test bench, and all the Tcl desgn scrpt you use for synthess and place-and-route. We wll re-run the smulaton wth randomly chosen test mage and check the classfcaton output. We wll also revew your code to verfy the clamed desgn features. Each feature wll pont that contrbute to your fnal grades. Also we wll smulate your desgn at the clock frequency specfed n your report and verfy ts error-free operaton. We wll look at the waveform analyss and all synthess reports of your desgn. Teams that are able to acheve the best specfcaton n the class (fastest executon tme, lowest power consumpton, or smallest slcon area) wll get extra bonus ponts. 6.3 Report expectaton Each team should turn n a well-organzed detaled report to explan your work. In the report, you should show the mplementaton detals and datapath optmzaton method and analyss featured n your accelerator desgn. In the appendx, you should nclude screen shots of crtcal waveform and synthess reports and gve explanatons of them, as well as screen shots of the fnal physcal layout after place-and-route. In the end, you should paste your code and provde detaled comments for your code. 11

13 7. Project submsson Please submt your lab assgnment on Gthub. You are expected to submt your report, all hardware desgn module fles, and class_project_dc.tcl fll. If you modfy the test benches or create new ones, submt them too. To submt your job, execute the followng command: (Note: the frst two commands just need to be done once for the entre semester.) % gt confg --global user.name your_user_name % gt confg --global user.emal your_emal_for_gthub % cd drectory_of_your_lab_assgnment/class-project-team#/ % gt add CNN_Accelerator/all module Verlog fle % gt add CNN_Accelerator/class_project_dc.tcl % gt add class-project-team#-report.pdf % gt commt -m your commts % gt push -u orgn master You also should submt anythng else you thnk may help us understand your code and result. Please do not submt fles lke complng result(smv) or smulaton data(.vpd). 8. Acknowledgement [1] [2] [3] 12

14 Appendx A The analyss of the back propagaton process n ths CNN code (1) Output layer (sngle-layer neural network) error s defned as the dfference between actual output and expected output. N E e n = 1 2 (d y ) 2 Here, d represents the expected output, y represents actual output, represents the output bt. In ths network, the output s 10 bts. Therefore, N = 10. The dervatve of error energy wth respect to weght: E e W = E e j y y j j u u j j w j 192 u j = W j y S4 + y j = φ(u j ) N b j E e = 1 2 (d y j ) 2 E e W j = 1 φ (u j ) y S4 13

15 E e b j = 1 φ (u j ) 1 In ths source code, t uses Sgmod actvaton functon, so the dervatve s: The local gradent s: φ (u j ) = y (1 y ) δ = E e u = E e y y u = 1 φ (u j ) (2) The output layer whch s followed by the poolng layer S4 Snce there s no weght n ths layer, we don t need to update weght. But we need to pass the error energy to the next layer. Therefore, we have to compute the local gradent δ. The defnton of ths s shown as below: δ S4 = E e S4 u j S4 Here, j ndcates the pxel s order number of output mage. There are = 192 output pxels n S4, so j = And the local gradent δ of output layer s computed already: δ S4 = E e S4 u S4 = E S4 ej j y S4 y S4 j j u S4 j δ = E e u = E e y y j u = 1 φ (u j ) S4 10 = E ej u 192 j = j W j y S4 j + E ej δ S4 j = E S4 ej y S4 y S4 j j u j S4 = 10 E e u u y S4 y S4 j j u J S4 = φ (u j S4 ) Snce poolng layer does not have actvaton functon, therefore, the dervatve of φ s 1. Then, 10 δ j S4 = δ From the above formula, we could compute the local gradent δ whch pass from output layer to poolng layer S4. We can see that the local gradent δ value passed to the output pxel j of the poolng layer s actually the sum of the weghts correspondng to the local gradent δ value of the followng layer output. 10 δ W j W j b 14

16 (3) convoluton layer, whch connected the followng poolng layer In order to compute the parameters, the output of S4 and layers are expanded to one-dmenson vector. Therefore, all pxels could be labeled wth and j. And we use m(x,y) to ndcate the coordnate of the pxel n the m th output template. The local gradent δ s defned as: δ m(x,y) = E e = Eem(x,y) y m(x,y) y m(x,y) The error energy delvered to the pxel s equal to the sum of the pxel error energy assocated wth ths pxel. Here, ndcates all pxels n samplng neghborhood Θ of m(x,y). E em(x,y) S4 = E e ϵϑm(x,y) Snce we use average poolng method, the output of S4 s the average of all pxels n the neghborhood of current pxel. Here, S ndcates the number of the pxels n the neghborhood Θ. In ths code we use 2 2 samplng block, therefore, S=4. u S4 = 1 S m(x,y)ϵϑ therefore, the gradent delvered from S4 to s: δ m(x,y) = E S4 e = 1 S y ϵϑm(x,y) m(x,y) ϵϑm(x,y) δ S4 y m(x,y) φ (u m(x,y) ) y m(x,y) S4 = E e S4 u ϵϑm(x,y) u S4 y m(x,y) Next we update weght n layer usng the local gradent δ: y m(x,y) layer totally has 6 12 = templates. Frst, we defne n = 1 6, m = 1 12 ndcates the label of templates. s, t ndcates the poston of parameter n ths template. u m(x,y) N = W nm(s,t) n c r s=0 t=0 y (c 1) n(x s 15

17 E e = E e W nm(s,t) E e W nm w h u m x=0 y=0 m(x,y) W nm(x,y) w h = δ m(x,y) m x=0 y=0 y (c 1) n(x s = correlaton(δ m, y n ) = conv(δ m, rotate180(y n )) m E e w m b m = δ m(x,y) h x=0 y=0 Smlarly, we can get weght updatng formula of layer. Here =6, N=1, and y ndcates the nput mage. δ m(x,y) E e W nm(s,t) = E e = 1 S y ϵϑm(x,y) m(x,y) ϵϑm(x,y) u m(x,y) w δ = E e h y m(x,y) φ (u m(x,y) ) N c = E e = W nm(s,t) n u m x=0 y=0 m(x,y) E e W nm r s=0 t=0 W nm(x,y) u ϵϑm(x,y) u y m(x,y) y m(x,y) l y (c 1) n(x s w h = δ m(x,y) m x=0 y=0 l y (c 1) n(x s = correlaton(δ m, y l n ) = conv(δ m, rotate180(y l n )) m E e w m b m = δ m(x,y) h x=0 y=0 (4) Poolng layer whch connected followng convoluton layer Here, n ndcates the order number (n = 1 6) of output mage of current layer, and m ndcates the order number (m = 1 12) of output mage of current layer. δ n(x,y) = E e = E en(x,y) y n(x,y) y n(x,y) u n(x,y) 16

18 δ n(x,y) δ n(x,y) = E en(x,y) c r = E em(x s (c 1) = m s=0 t=0 c r E (c 1) em(x s y m s=0 t=0 m(x,y) c r E (c 1) em(x s u m s=0 t=0 (c 1) m(x s δ n(x,y) u m(x,y) c r = W nm(s,t) s=0 t=0 r = δ m(x s (c 1) s=0 t=0 Therefore, the local gradent δ of nth mage s: m c y m(x,y) u (c 1) m(x s y n(x,y) y (c 1) n(x s δ n = correlaton(δ m, W nm ) m W n,m(s,t) y m(x,y) y n(x,y) y n(x,y) 17