CELLULAR NEURAL NETWORKS & APPLICATIONS TO IMAGE PROCESSING. Vedat Tavsanoglu School of EEIE SOUTH BANK UNIVERSITY LONDON UK

Transcription

1 CELLULAR NEURAL NETWORKS & APPLICATIONS TO IMAGE PROCESSING Vedat Tavsanoglu School of EEIE SOUTH BANK UNIVERSITY LONDON UK

2 Outline What is CNN? Architecture of CNN Analogue Computing with CNN Advantages of CNN The CNN Universal Machine CNNUM) Application Potential CNN in Image Processing Feature Etraction For Character Recognition Using Gabor-type Filters Implemented By Cellular Neural Networks 2

3 What is CNN? L. O. Chua & L. Yang, 988. Generic definition of Cellular Neural Networks CNN): N-dimensional array of mainly identical dynamical systems, called cells, which satisfy two properties: Interactions are local within a finite radius r. All state variables are continuous-valued signal. The local interconnection pattern, called cloning template or synaptic law, may be static or dynamic, linear or nonlinear, instantaneous or delayed. 3 Leon Chua and Lin Yang, Cellular Neural Networks: Theory and Applications, IEEE Trans. on Circuit and Systems, vol. 35, no.0, October 988.

4 4 Architecture of CNN

5 Local Connectivity CNN is made of a massive aggregate of regularly spaced circuit clones, called cells. Each cell communicate with each other directly only through its nearest neighbours. This locality of connections local connectivity) between the units is the main difference between CNNs and other dynamical neural networks. 5

6 Cloning Template The matri that defines the connections between a cell and its neighbouring cells is called the CLONING TEMPLATE. Sliding the template on the CNN plane clones the same connection pattern for each cell. 6

7 r-neighbourhood CNN The order of local connectivity of cells, r, around the centre cell represent its neighbourhood. Figures below represent 33, 55 and 77 cloning templates or -, 2- and 3-neighbourhood. 7 r= r=2 r=3

8 The basic circuit unit of a CNN: The Cell Each cell is made of a linear capacitor, a nonlinear voltage-controlled current source, and a few resistive linear circuit elements 8

9 Equations of CNN Now define: uij ij ij ij yij ij C d dt ij u, v, The equation for the cell i,j): v ij Aij, kl ykl Bij, klukl ) R kl The eternal input to the cell is typically assumed to be constant over a certain operation interval. The total input current to the cell = weighted sum of control inputs) + weighted sum of feedback outputs)+ a constant bias term I) v y I 9

10 Equations of CNN Without loss of generality, R and C values can be set to. Thus: d dt ij A y B u ) ij kl ij, kl kl ij, kl kl I dstate dt ij state A output B input ) ij ij, kl kl ij, kl neighbour kl kl I 0

11 Equations of CNN The only nonlinear element in each cell is a piecewise-linear voltage controlled voltage source with the characteristic: output ij f state ), for i, ij y i, j) f i, j)) 2 j N

12 The block diagram of a cell 2

13 3 CELLULAR NEURAL NETWORKS & APPLICATIONS TO Implementation of a 2-Cell CNN ) ) ) ) ) pf sf t g sf pf,2 ) ) 2 )) i t t t f i i i

14 Trajectory of Cell States, 2 of 2-Cell CNN For the choice s.2, p 2, 0) 0.4, 20) 0., A 4.04, the circuit shows the following chaotic behaviour: f / 4 4

15 In Practice Most commonly: regular 2D lattice topology neighbourhood size: or 2 space-invariant template and bias binary output in steady state -: white; +: black) Using a CNN: specify cloning template i.e. CNN function) set input u ij ) and initial state ij 0)) read out binary) solution in steady state 5

16 6 Analogue Computing with CNN

17 What is Analogue Computing Using CNN Data representation with voltages and currents or optical signals, etc.) Operations: addition: currents obeying Kirchoff s law multiplication: voltage-controlled current sources other: e.g., ep) using a diode Memory: charges on capacitors Data path: single wire vs. bus) 7

18 Analog Computing: + and - 8

19 9 Advantages of CNN

20 Advantages of CNN CNN s continuous time feature allows real-time signal processing. Its local interconnection feature makes it suitable for VLSI implementation in comparison to NN. Conventional digital computation methods run into a serious speed bottleneck due to their serial nature. As a computation model, CNNs are well suited for highspeed parallel signal processing. Equivalent computing power: TeraOps Direct input/output to/from cells e.g. optical sensors) 20

21 2 The CNN Universal Machine

22 The CNN Universal Machine T. Roska & L. O. Chua 2, 993 Spatio-temporal cellular processor Combining analogue and logic operations Local and global logic and memory Stored analogic programs Software/hardware environment for application development 22 2 Tamas Roska and Leon Chua, The CNN Universal Machine: An Analogic Array Computer, IEEE Trans. on Circuit and Systems II, vol. 40, no. 3, March 993.

23 The CNN Universal Machine architecture as an analogic array computer The CNN Universal Machine is an "analogic" 3D array computer, a stored program, with a CNN array embedded. It contains, however, additional units: local continuous analogue) and logic memory, local analogue and logic units as well as a global analogic programming unit GAPU). Hence, continuous valued spatio-temporal dynamics is embedded in a logic structure, locally and globally. Using this new architecture, a new class of algorithms is emerging, the computational compleity issues are redefined. Impressive eamples show the practical relevance and the ease of use of this profoundly new idea of computing, including CNN Universal Chips with trillion operations per second. 23

24 The CNN Universal Machine 24

25 The CNN Universal Machine 25 Analogic Lab. In Budapest:

26 The CNN Chip project Funding: ONR Chip design: UC Berkeley USA), CNM Seville, Spain), HUT Helsinki, Finland) Chip manufacturing: USA, France, Sweden Hardware/software development environment: SzTAKI Budapest, Hungary) 26

27 27 Application Potential

28 Application Potential 28 Signal Processing Image Processing 2-D Filtering Feature etraction and classification Motion detection and estimation Collision Avoidance Object counting and size estimation Path tracking Analysis of 3-D Comple Surfaces Detecting minima and maima Detecting areas with gradients that eceed a given treshold Modelling of biological and physical systems

29 29 CNN in Image Processing

30 Features of CNN Image Processing 30

31 Etracting The Edges of a Diamond and a Square 3 Leon Chua and Lin Yang, Cellular Neural Networks: Theory and Applications, IEEE Trans. on Circuit and Systems, vol. 35, no.0, October 988.

32 Etracting The Corners of a Diamond and a Square 32

33 Feature Etraction of the Chinese Characters 33

34 Smoothing 34

35 Connected Component Detector 35

36 Connected Component Detector

37 Edge Detector 37

38 Edge Detector 38

39 Grayscale Contour Detector 39

40 Halftoning 40

41 Halftoning 4

42 Hole Filler 42

43 Local Maima Detector 43

44 Small Object Remover 44

45 Grayscale to Binary Treshold 45

46 Shadow Detector 46

47 Shadow Detector 47

48 Shadow Detector 48

49 A Character Recognition System V : Vertical CCD H : Horizontal CCD D: North-Eastern Direction Diagonal CCD D2: South-Eastern Direction Diagonal CCD LS : Left SD RS : Right SD US : Up SD DS : Down SD 49

50 50 Feature Etraction For Character Recognition Using Gabor-type Filters Implemented By Cellular Neural Networks

51 Outline A Preview of CNN Gabor-Type Filters -D Gabor Filters -D CNN Gabor-Type Filters 2-D Gabor Filters 2-D CNN Gabor-Type Filters Feature Etraction from Handwritten Characters 5 Orientation Map Algorithm Constructing the Orientation Map Application to Handwritten Characters Handwritten Characters and their FFTs Discussion on the correct selection of filter parameters Conclusion

52 -D Gabor Filters For -D signals, the impulse response h) of a Gabor filter is a comple eponential function modulated by the Gaussian: h ) g ) g ) e j e 2 2 / 2 2 ) : angular frequency : standart deviation of the Gaussian 52

53 -D CNN Filters 53 To filter a -D image of N piels, [Shi] introduced a -D CNN array of N cells where the state vn) at the nth cell satisfies: v n) a v n k) bu n) k k where n is the spatial integer variable and ) denotes time derivative; A : feedback cloning template; a k k 2 b : feedforward cloning template. Shi, B.E. Gabor-Type Filtering in Space and Time with Cellular Neural Networks, IEEE Trans. CAS-I, vol 45, pp.2-32, February 998.

54 -D CNN Low-Pass Filters [Shi] also gave a resistive grid implementation based on [Shi & Chua] 2 of -D low-pass CNN Gabor-Type filter: dv n) 2 [ v n ) v n)] G [ v n ) v n)] G v n) G0 C u n) dt dv n) 2 2 v n) v n ) 2 ) v n) v n ) u n) dt 2 A a a a a 2 ), 2 k 0 b k Shi, B.E., Chua, L. Resistive Grid Image Filtering: Input/Output Analysis via CNN Framework, IEEE Trans. CAS-I, vol 39, No 7, pp , July 992.

55 -D CNN Low-Pass Filters The spatial impulse and frequency responses of the - D CNN low-pass filter: h LP n) cosh e 4 n 2 / 2) H LP e j ) V e U e j j ) ) cos 55

56 -D CNN Gabor-Type BP) Filter A -D CNN Gabor-type BP) filter tuned to frequency w 0 can be obtained from a -D CNN low-pass filter by replacing the feedback cloning template: A a k k with jk a ke k 0 A A 2 jk j j a e e e k ) k If the original low-pass filter template is stable, then the corresponding Gabor-type filter template is also stable. 56

57 -D CNN Gabor-Type Filter The spatial impulse and frequency responses of the CNN Gabor-type filter are: h H BP BP e 4 n j 0n jn 0 ) ) n e j ) h LP H n LP e e j 0 ) ) e cos 0 ) 57

58 2-D Gabor Filters 2-D Gabor filters are used as preprocessors for various tasks in computer vision applications owing to their orientation selectivity property. One of these applications is feature etraction for character recognition [Watanabe et al.] Watanabe, M., Hamamato, Y., Yasuda, T., Tomita, S., Normalization Techniques of Handwritten Numerals for Gabor Filters, Document Analysis and Recognition, vol, pp , 997.

59 2-D Gabor Filters A 2-D Gabor filter is described by the spatial impulse response: h, y) g, y) e j y) y g, y) e y 2 2 ), y : : angular spatial frequency standart deviation of the Gaussian 59

60 60 CELLULAR NEURAL NETWORKS & APPLICATIONS TO 2-D Gabor Filters The output v,y) of the filter h,y) to an image u,y) is obtained through the convolution sum: 2 2 2, )) ) 2 ) ) ), 2 y y y j y y y e e y u, ), ), ),,, y y y y h y u y v

61 2-D CNN Filters To filter a 2-D image of MN piels, [Shi] introduced a 2-D CNN array of MN cells where the state vm,n) at the m,n)th cell satisfies: v m, n) A b a k, l a k, l k, l k, l : v m k, n l) bu m, n) feedback cloning template : feedforward cloning template 6

62 2-D CNN Low-Pass Filters A circuit implementation of this CNN: 62

63 63 CELLULAR NEURAL NETWORKS & APPLICATIONS TO 2-D CNN Low-Pass Filters From KCL 0 ), ), ), )], ), [ )], ), [ )], ), [ )], ), [ 2 0 n m u dt n m dv C G n m v G n m v n m v G n m v n m v G n m v n m v G n m v n m v ), ), ), ), ), ), ) 4 ), ), 2 2 n m u n m v n m v n m v n m v n m v dt n m dv n m v 2 2, 0,,,0 0,0,0, 0,,, ; 0 0 ) b a a a a a a a a a a A l k kl

64 2-D CNN Gabor-Type BP) Filters A 2-D CNN Gabor-type BP) filter tuned to frequency w 0,w y0 ) can be obtained from a 2-D CNN low-pass filter by replacing the feedback cloning template: a k l k A, l, with j k l ) a k, le k l 0 y 0 A, 64

65 65 CELLULAR NEURAL NETWORKS & APPLICATIONS TO 2-D CNN Gabor-Type Filters In the case of 2-D band-pass CNN Gabor filter tuned to the centre frequency w o,w yo ), feedback and feedforward cloning templates are given by: ) b e e e e A y y j j j j

66 66 CELLULAR NEURAL NETWORKS & APPLICATIONS TO 2-D CNN Gabor-Type Filters Frequency response of the CNN Gabor-type filter: ) 2cos ) 2cos 4 ), ), ), y y j j j j j j BP y y y e e U e e V e e H

67 2-D CNN Gabor-Type Filters Orientation Selectivity Properties Gabor filters are orientation selective and respond maimally to edges which are oriented at an angle: tan y / ) θ is defined to be the angle between the horizontal ais and the line perpendicular to the edge, which is related to the radius of spatial frequency r : r cos, y r sin 67

68 2-D CNN Gabor-Type Filters Frequency Response 68 r centre frequency band-with

69 2-D CNN Gabor-Type Filters Frequency Response 69 r centre frequency band-with

70 Feature Etraction from Handwritten Characters We wish to eploit the orientation selectivity property of CNN Gabor-type filters in the recognition of handwritten characters. To this end, we will device a system that detects the total orientation of the image at all possible angles. This will be achieved by first finding the maimum dominant) orientation at each piel and then by summing them over all piels to obtain the total orientation. This device will be called the ORIENTATION MAP. 70

71 Feature Etraction from Handwritten Characters 7 Orientation Map Set up a filter bank of 8 filters where kth filter represents an orientation defined by: k =k/8, k=0,,7. Store the filter bank outputs in 8 MN matrices. Compare piel by piel the values stored in 8 matrices. Store the maimum output for each piel in a matri in the same piel position with its orientation info. This is called Dominant Orientation Matri DOM). Count the number of piels in DOM with same orientation. The orientation map is then defined as the graph of these numbers versus orientations of 8 different angles. The vertical and horizontal aes represent the total orientation and the angle of orientation, respectively.

72 Feature Etraction from Handwritten Characters Algorithm Constructing the Orientation Map MN 2 MN 2 3 MN M N COMPARATOR DOMINANT ORIENTATION MATRIX 3 Total Dominant Orientation 8 MN 8 72 FILTER BANK OF 8 CNN GABOR-TYPE FILTERS FILTERED IMAGES ORIENTATION COUNTER

73 Application to Handwritten Characters We use a filter bank of n p =8 filters to detect the dominant orientations. The first task is to assign values to the parameters and r of this system. To this end, we prepare the circle pattern because the circle consists of edges with all orientations distributed equally. We should epect to obtain an almost flat orientation map when input pattern is a circle. Therefore such an input sets a good eample of a test pattern for finding appropriate values for r and. 73

74 Gabor-type Filter Output for the letter O 74

75 Application to Handwritten Characters 75

76 Gabor-type Filter Output for the letter A 76

77 Application to Handwritten Characters 77

78 Application to Handwritten Characters 78

79 Application to Handwritten Characters 79

80 Application to Handwritten Characters 80

81 Handwritten Characters and their FFTs 8

82 Handwritten Characters and their FFTs 82

83 Discussion on the correct selection of filter parameters The parameter r is the radius of spatial frequency and controls the location of Gabor filter centre frequency w 0,w y0 ). The parameter determines the spread of the CNN Gabor filter frequency response along both and +90 directions, which are the same due to the circular symmetry of the filter. Small selection of makes the filter narrow and selective which yields better results. It is easily seen from FFTs of handwritten characters that most of the energy is localised at lower frequencies. Therefore values of r should be chosen small enough to capture most of the energy on the frequency plane. 83

84 Conclusion In this study feature etraction from handwritten characters has been carried out using Gabor-type filters implemented by CNN s. An orientation map is used which converts the filter output to a suitable form of etracted features. Filtering is investigated using different parameter values and optimum parameter values have been discussed. The result of this study will be used in the recognition of handwritten characters. 84