PULSE-COUPLED networks (PCNs) of integrate-and-fire

Transcription

1 1018 IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 15, NO. 5, SEPTEMBER 2004 Grouping Synchronization in a Pulse-Coupled Network of Chaotic Spiking Oscillators Hidehiro Nakano, Student Member, IEEE, and Toshimichi Saito, Senior Member, IEEE Abstract This paper studies a pulse-coupled network consisting of simple chaotic spiking oscillators (CSOs). If a unit oscillator and its neighbor(s) have (almost) the same parameter values, they exhibit in-phase synchronization of chaos. As the parameter values dfer, they exhibit asynchronous phenomena. Based on such behavior, some synchronous groups appear partially in the network. Typical phenomena are veried in the laboratory via a simple test circuit. These phenomena can be evaluated numerically by using an effective mapping procedure. We then apply the proposed network to image segmentation. Using a lattice pulse-coupled network via grouping synchronous phenomena, the input image data can be segmented into some sub-regions. Index Terms Chaos, grouping, integrate-and-fire, pulse-coupled network (PCN), spiking neuron, synchronization. I. INTRODUCTION PULSE-COUPLED networks (PCNs) of integrate-and-fire neurons (IFNs) are a type of articial neural network [1] [8]. The PCNs exhibit various synchronous and asynchronous phenomena [2], [3] and are applicable to associative memories [4], [6], image processing [7], [8], etc. The PCNs can be realized by simple electric circuits [9]. In the published literature, IFNs with periodic behavior have been the main focus. On the other hand, we have presented chaotic spiking oscillators (CSOs) that can output a chaotic spike-train [10] [13]. The CSO can be regarded as a higher order IFN that can exhibit chaos and rich burcation phenomena; the CSO has richer dynamics than usual IFNs. Connecting plural CSOs using each spike-train, a chaotic pulse-coupled network (CPCN) can be constructed. The pulse-coupling method of the CPCN is based on and is simpler than that of PCN in [4], [7], and [8]. The CPCN can exhibit various chaos synchronous phenomena that may be applicable to image processing [12]. Our CSO is concerned with a novel resonate-and-fire neuron (RFN) model [14]. Study of CSOs having rich dynamics may contribute to the study of basic nonlinear phenomena and flexible engineering applications including image processing [15], pattern recognition [16], and pulse-based communications [17], [18]. This paper studies synchronous phenomena in a CPCN consisting of simple CSOs. In Section II, as a preparation, we introduce the basic dynamics of the single CSO presented in [11]. Manuscript received June 2, 2003; revised December 12, This work was supported by JSPS.KAKENHI under Grant H. Nakano is with the Department of Computer Science and Media Engineering, Musashi Institute of Technology, Tokyo, Japan ( nakano@ic.cs.musashi-tech.ac.jp). T. Saito is with the Department of Electronics, Electrical, and Computer Engineering, Hosei University, Tokyo, Japan ( tsaito@k.hosei.ac.jp). Digital Object Identier /TNN The CSO can be implemented easily, and typical chaotic behavior is veried in the laboratory. In Section III, we present the CPCN having a local connection structure. Each CSO is connected with the neighbor CSOs. First, we consider a CPCN consisting of two CSOs. If both the CSOs have (almost) the same parameter values, they exhibit in-phase synchronization of chaos. As the parameter values dfer, they exhibit asynchronous phenomena. Next, we consider a ladder CPCN consisting of four CSOs. If a CSO and its neighbor CSO(s) have (almost) the same parameter values, they exhibit in-phase synchronization of chaos. As the parameter values dfer, they exhibit asynchronous phenomena. Based on such behavior, some synchronous groups appear partially in the CPCN. Typical phenomena are veried in the laboratory with a simple test circuit. In Section IV, we introduce a normal form equation for the CPCN in order to extract essential parameters. By defining a coincident spike rate between the CSOs, synchronous phenomena can be evaluated numerically. In order to more efficiently calculate this rate, we introduce a mapping procedure. This map can be described precisely by using exact piecewise solutions. In Section V, we apply the CPCN to image segmentation. For the input, we prepare a lattice CPCN where the parameter of each CSO corresponds to each pixel value of the input. By the grouping synchronous phenomena, the input image data can be segmented into some sub-regions. We show typical simulation results. This paper provides basic experimental and analysis results for a PCN of simple chaotic oscillators. These results contribute to the study of basic nonlinear phenomena. Our CPCN has a simple local connection structure and can exhibit various grouping synchronous patterns depending on the network parameters. This means that the CPCN has rich functionality and may be developed into flexible applications such as image processing systems. II. CSO Fig. 1 shows a CSO. The CSO will be a unit element of the CPCN. The two capacitors and the two linear voltage-controlled current sources (VCCSs) construct a linear circuit [11]. The circuit dynamics is described by the following equation the firing switch is open. for (1) We assume that (1) has unstable complex characteristic roots, where (2) /04$ IEEE

2 NAKANO AND SAITO: GROUPING SYNCHRONIZATION IN A PCN OF CSOs 1019 Fig. 1. CSO. Fig. 3. Chaotic attractor in a unit CSO (C ' 1:0 nf, C ' 47 nf, 1=g ' 100 k, V ' 1:0 V, E '00:3 V, V '0V ' 8:0 V). (a) Phase space attractor. (b) Time-domain waveform. Fig. 4. CPCN having a local connection structure. The circles and solid lines represent the CSOs and the couplings, respectively. Fig. 2. Implementation example of a CSO. In this case, the capacitor voltages can vibrate below the firing threshold voltage. If the capacitor voltage reaches, the pulse-generator (PG) outputs a single firing spike that closes the firing switch. Then, is reset to the base voltage instantaneously while cannot change instantaneously (3) Repeating in this manner, the CSO generates a firing spike-train for where and are high- and low-voltage levels, respectively. In this paper, for simplicity, we assume that all the circuit elements are ideal and define the switching dynamics as an ideal model. 1 Fig. 2 shows an implementation example of the CSO. The linear VCCSs are realized by operational transconductance ampliers (OTAs, NJM13600). The conductance value, which controls the self-running angular frequency, can be adjusted by means of, and. We have confirmed an approximate relation. The PG and the firing switch 1 This definition is a routine in the electrical engineering. (4) are realized by one comparator (LM339), two analog switches (4066), one capacitor, and one resistor. Let the time constant be sufficiently small and let at. If the capacitor voltage reaches the firing threshold voltage, the comparator closes the switch and the voltage charges up to in a short time. The voltage closes the firing switch and is reset to the base voltage in a short time. Then the comparator opens the switch and returns to with the sufficiently small time constant. These switching dynamics can satisfy (3) and (4), approximately. In order to visualize the very narrow firing spikes, we have used a monstable multivibrator (MM, 4538) which does not affect the dynamics of the CSO. Fig. 3 shows a typical chaotic attractor in the laboratory measurements. [11] has presented corresponding chaotic attractor in numerical simulations and shown theoretical evidence for the chaos generation. III. CPCN Fig. 4 shows a CPCN having a local connection structure. The circuit dynamics is described by (5) (6) where denotes the index of the CSO. Each CSO is connected with the neighbor CSOs. Fig. 5 illustrates the connection method. If the capacitor voltage reaches the

3 1020 IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 15, NO. 5, SEPTEMBER 2004 Fig. 6. Implementation example of a PG for a CPCN. Fig. 5. Switching rule of the CPCN. The ith and jth CSOs are connected to each other. firing threshold voltage, the th CSO outputs the firing spike, and is reset to the base voltage instantaneously. At the same time, the firing spike is input to the th CSO (the neighbor of the th CSO) instantaneously. At this moment, the capacitor voltage is higher than the refractory threshold voltage, the th CSO outputs the firing spike, and is reset to instantaneously. If is less than, the th CSO does not output the firing spike and is unaffected. In the case where the th CSO has other neighbor(s), the firing spike is input to the neighbor(s) instantaneously. By this instantaneous propagation, some of the CSOs output the firing spikes simultaneously. Repeating in this manner, each CSO generates a firing spike-train or otherwise and where denotes the index of the neighbor(s) of the th CSO. Equations (5) (7) give the dynamics of the CPCN. Fig. 6 shows an implementation example of the PG of the th CSO, where. Let the time constant be sufficiently small, thus, firing spikes can be propagated between the CSOs with a sufficiently short time delay. Let at. The switch is closed when reaches the firing threshold voltage. The switch is closed when is higher than the refractory threshold voltage. The switch is closed when the firing spike is applied from the th CSO. 2 If the switch is closed, or both the switches and are closed simultaneously, the voltage charges up to in a short time. The voltage closes the firing switch and is reset to the base voltage in a short time. At the same time, the voltage is input to the th CSO with a short time delay. Then returns to with the sufficiently small time 2 This circuit can deal with the other neighbors by connecting multiple switches in parallel, as shown in Fig. 6. (7) constant. These switching dynamics can satisfy (6) and (7), approximately. Fig. 7 shows typical experimental results for a CPCN of two CSOs:, and. If both the CSOs have (almost) the same parameter values, they exhibit in-phase synchronization of chaos, as shown in Fig. 7(a). Since both the CSOs have (almost) the same trajectories, they always output coincident firing spikes. As the conductance value of the second CSO varies, the synchronous phenomena are broken down to asynchronous phenomena, as shown in Fig. 7(b). Since both the CSOs have dferent trajectories, they output both coincident and incoincident firing spikes. Fig. 8 shows typical experimental results for a ladder CPCN of four CSOs:,,, and. If all the CSOs have (almost) the same parameter values, they exhibit in-phase synchronization of chaos, as shown in Fig. 8(a). Note that all the CSOs have (almost) the same trajectories independently of the number of the neighbors, and always output coincident spikes. As the conductance values of the third and fourth CSOs vary, the synchronous phenomena change into grouping synchronous phenomena, as shown in Fig. 8(b). The first and second CSOs form an in-phase synchronous group. The third and fourth CSOs form another in-phase synchronous group. The two groups do not synchronize to each other. Fig. 9 shows the time-domain waveforms. In the figure, at, reaches, is higher than, and both and are lower than. Then, both and jump to simultaneously, and both and move continuously. At, reaches and the other states are higher than. Then, all the states jump to simultaneously. However, because of parameter dferences, trajectories of the third and fourth CSOs part from those of the first and second CSOs. This means that CSOs in the same group can have (almost) the same trajectories after synchronization is achieved. However, in the rare case where states of either group are close to in the firing moment of the other group, the synchronization may be broken down because of the influences such as noise and/or parameter mismatches. In the figure, at, reaches, is lower than, and is higher than. Then, moves continuously and jumps to ; the synchronization of the first and second CSOs is broken down. In

4 NAKANO AND SAITO: GROUPING SYNCHRONIZATION IN A PCN OF CSOs 1021 Fig. 7. Typical experimental results for a CPCN of two CSOs (C ' 1:0 nf, C ' 47 nf, V ' 1:0 V, E '00:3 V, V ' 0:5 V, V '0V ' 8:0 V). (a) In-phase synchronization of chaos (1=g ' 1=g ' 100 k).(b) Asynchronous phenomena (1=g ' 100 k, 1=g ' 133 k).(c) Network topology. a likewise manner, the synchronization of the third and fourth CSOs may be broken down. However, CSOs having (almost) the same parameters can repeat the synchronous firings and the synchronization can rapidly recover. Therefore, we observe the grouping synchronous phenomena in the laboratory, as shown in Fig. 8(b). IV. NUMERICAL ANALYSIS In this section, we investigate synchronous phenomena associated with the proposed CPCN. Using dimensionless variables and parameters Fig. 8. Typical experimental results for a ladder CPCN of four CSOs (C ' 1:0 nf, C ' 47 nf, V ' 1:0 V, E '00:3 V, V ' 0:5 V, V ' 0V ' 8:0 V). (a) In-phase synchronization of chaos (1=g ' 1=g ' 1=g ' 1=g ' 100 k). (b) Grouping synchronization (1=g ' 1=g ' 100 k, 1=g ' 1=g ' 133 k). (c) Network topology. Equations (5) (7) are transformed into (9) (11), respectively (9) (10) or otherwise. and (11) (8) Fig. 10 shows the normalized trajectories. Except for the selfrunning angular frequency, the CPCN has three parameters:

5 1022 IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 15, NO. 5, SEPTEMBER 2004 Fig. 9. Trajectories for a ladder CPCN of four CSOs. Fig. 10. Normalized trajectories for a CPCN of two CSOs. the damping, the base, and the refractory threshold. For simplicity we fix these parameters th CSOs always output coincident spikes. The case implies that the th and th CSOs output no coincident spike. In order to more efficiently calculate this rate, we introduce a return map. First, we define the following objects: For, (9) has the following exact piecewise solution. (12) where (, ) denotes an initial state vector at. Using this solution, trajectories can be calculated precisely [10]. Fig. 11 shows typical synchronous and asynchronous phenomena in numerical simulations. The numerical data in Fig. 11(a) and (b) correspond to the laboratory data in Fig. 7(a) and (b), respectively. Also, in the numerical simulations, we have veried grouping synchronous phenomena corresponding to the laboratory data in Fig. 8. Let be the th time when either of the th or th CSO outputs a firing spike. Let be the number of spikes of for, and let be the number of spikes of for, where, and and are nonnegative integers. We then define the following coincident spike rate between the th and th CSOs: (14) Let us consider trajectories starting from at (see Fig. 10). As the trajectories start from, either trajectory of the CSOs must reach the threshold at some finite time, and the trajectories return to. We can then define a return map (15) This map is given analytically. The firing time is given by (16) (13) where is a positive-minimum root of Calculating this rate, we can numerically evaluate the synchronous phenomena. The case implies that the th and (17)

6 NAKANO AND SAITO: GROUPING SYNCHRONIZATION IN A PCN OF CSOs 1023 Fig. 11. Typical numerical results for a CPCN of two CSOs ( = 0:07, q = 00:3, a = 0:5). (a) In-phase synchronization of chaos ( = = 1). (b) Asynchronous phenomenon ( =1, =0:75). TABLE I COINCIDENT SPIKE RATE FOR A LADDER CPCN OF four CSOS ( =0:07, q = 00:3, a =0:5, m =10, n =10 ). THE PARAMETER VALUES OF (a) AND (b) CORRESPOND TO THOSE OF FIG. 8(a) AND (b), RESPECTIVELY Fig. 12. Coincident spike rate for a CPCN of two CSOs ( =0:07, q = 00:3, a =0:5, =1, m =10, n =10 ). A and B correspond to Fig. 7(a) and (b), respectively. The root can be solved by the Newton Raphson method. The return points are given by,, and, where for simplicity of analysis we set the parameters as follows: for for (18) Since the return map (15) is given analytically, the coincident spike rate (13) can be calculated rapidly. Fig. 12 shows the coincident spike rate for the CPCN of two CSOs:,, and.as approaches 1, the rate approaches 1 gradually. Table I shows the coincident spike rate for the ladder CPCN of four CSOs:, We have set random initial states for each CSO for the calculation of the coincident spike rate. In the calculation of (13), the values of have converged reasonably in about iterations. As the parameter varies, the rate changes, as shown in Fig. 13. The rate is similar to Fig. 12. On the other hand, we have confirmed that the rates and hold the constant value 100 [%] independently of the parameter value. In order to investigate effects of parameter perturbation, we consider the basic case where denotes parameter perturbation of the fourth CSO. Fig. 14 shows the coincident spike rate for parameter perturbation. If the perturbation is not 0, the rate is not 100 [%].

7 1024 IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 15, NO. 5, SEPTEMBER 2004 Fig. 13. Coincident spike rate for a ladder CPCN of four CSOs ( =0:07, q = 00:3, m =10, n =10). C and D correspond to Fig. 8(a) and (b), respectively. Fig. 14. Effects of a parameter perturbation ( =0:07, q = 00:3, =0:75, m =10, n =10 ). D corresponds to Fig. 8(b). (a) a =0:5. (b) a =0:2. However, is satisfied for a sufficiently small value, we can evaluate that the second and third CSOs are in dferent groups, and the third and fourth CSOs are in the same group. V. IMAGE SEGMENTATION In this section, we apply a CPCN to image segmentation. Let the input image data be binary and pixels, as shown in Fig. 15(a). For the input, we prepare a lattice CPCN consisting of CSOs. Each CSO is allocated on each pixel and connected with the nearest neighbors, as shown in Fig. 15(b). The parameter value of each CSO corresponds to each pixel value. We have set a random initial state for each CSO. If a CSO and its neighbor CSO(s) have the same parameter values, they exhibit in-phase synchronization of chaos. As the parameter values Fig. 15. Image segmentation using a CPCN ( =0:07, q = 00:3, a =0:5). (a) Input image data. (b) Allocation of each CSO. The number denotes the CSO index. (c) Synchronous groups appearing for the input. (d) Output spike-trains of each group. dfer, they exhibit asynchronous phenomena. Based on such behavior, five synchronous groups appear partially in the CPCN, as shown in Fig. 15(c). That is, the binary image input is segmented into five sub-regions. In the simulation, transient time to the grouping synchronization approximates (in the normalized time). Fig. 15(d) shows the output spike-trains of each group. These synchronous phenomena can be evaluated by the coincident spike rate, as shown in Table II. In the table, we have chosen two representative CSOs from a group. Note that any coincident spike rates between CSOs in the same group are a constant value 100 [%]. Each CSO in groups 2 to 5 has the same parameter values. However, CSOs in dferent groups do not synchronize to each other. Also, we have veried such grouping synchronous phenomena for a gray scale image input [12].

8 NAKANO AND SAITO: GROUPING SYNCHRONIZATION IN A PCN OF CSOs 1025 TABLE II COINCIDENT SPIKE RATE FOR THE SIMULATION IN FIG. 15( =0:07, q = 00:3, a =0:5, m =10, n =10 ) Fig. 16. Image segmentation using a PCN of basic periodic oscillators (a = 0:9). Such grouping is dficult for basic periodic oscillator networks. Fig. 16 shows a typical simulation result using a PCN of basic integrate-and-fire oscillators described by for or otherwise. and (19) The input image data and the allocation of each oscillator are the same as those of Fig. 15(a) and (b), respectively. The parameter value of each oscillator corresponds to each pixel value. We have set a random initial state for each oscillator. In this simulation, group 1 and the other groups exhibit a kind of multiphase synchronization to each other. Groups 2 to 5 exhibit in-phase synchronization through group 1. These groups are not separated. If basic periodic oscillator networks are applied to the image segmentation, globally inhibitory couplings are necessary [8], [15]. However, our CPCN cannot exhibit multiphase synchronization because of the chaotic behavior of each CSO. Even all the CSOs start from the almost same initial state, CSOs in dferent groups cannot have the same trajectories because of sensitivity to the initial states in chaos. Therefore, our CPCN can realize the image segmentation without global couplings. The couplings of the CPCN can be simpler than those of existing basic periodic oscillator networks for the image segmentation. VI. CONCLUSION We have studied synchronous phenomena in a CPCN. The CPCN exhibits grouping synchronous phenomena characterized by partial in-phase synchronization of chaos. Constructing a simple test circuit, we have veried typical phenomena in the laboratory. We have evaluated these phenomena numerically by using a mapping procedure. Based on the grouping synchronous phenomena, we have applied the CPCN to image segmentation. In future work, we are considering the following. For simplicity, we have fixed some parameters. We should analyze the synchronous phenomena and related burcation phenomena for a wider parameter region. We have confirmed grouping synchronous phenomena in the CPCN both experimentally and numerically. We should analyze characteristic of transient time to synchronization in more detailed. The grouping synchronous phenomena are broken down because of the influences such as noise and/or parameter mismatches. We should analyze the phenomena including these influences in more detailed. Also, we should implement a large scale CPCN and investigate synchronous phenomena and their stability. We have obtained fundamental results for image segmentation using the CPCN. We should apply the CPCN to various natural image data and evaluate its performance. REFERENCES [1] J. P. Keener, F. C. Hoppensteadt, and J. Rinzel, Integrate-and-fire models of nerve membrane response to oscillatory input, SIAM J. Appl. Math., vol. 41, pp , [2] R. E. Mirollo and S. H. Strogatz, Synchronization of pulse-coupled biological oscillators, SIAM J. Appl. Math., vol. 50, pp , [3] E. Catsigeras and R. Budelli, Limit cycles of a bineuronal network model, Physica D, vol. 56, pp , [4] E. M. Izhikevich, Weakly pulse-coupled oscillators, FM interactions, synchronization, and oscillatory associative memory, IEEE Trans. Neural Networks, vol. 10, pp , May [5] G. Lee and N. H. Farhat, The burcating neuron network 1, Neural Netw., vol. 14, pp , [6], The burcating neuron network 2: An analog associative memory, Neural Netw., vol. 15, pp , [7] J. J. Hopfield and A. V. M. Herz, Rapid local synchronization of action potentials: Toward computation with coupled integrate-and-fire neurons, Proc. Nat. Acad. Sci., vol. 92, no. 15, pp , [8] S. R. Campbell, D. Wang, and C. Jayaprakash, Synchrony and desynchrony in integrate-and-fire oscillators, Neural Comput., vol. 11, pp , [9] T. Asai and Y. Amemiya, Frequency-and temporal-domain neural competition in analog integrate-and-fire neurochips, in Proc. IJCNN, 2002, pp [10] K. Mitsubori and T. Saito, Mutually pulse-coupled chaotic circuits by using dependent switched capacitors, IEEE Trans. Circuits Syst. I, vol. 47, pp , July 2000.

9 1026 IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 15, NO. 5, SEPTEMBER 2004 [11] H. Nakano and T. Saito, Basic dynamics from a pulse-coupled network of autonomous integrate-and-fire chaotic circuits, IEEE Trans. Neural Networks, vol. 13, pp , Jan [12], Synchronization in a pulse-coupled network of chaotic spiking oscillators, in Proc. MWSCAS, vol. I, 2002, pp [13] Y. Takahashi, H. Nakano, and T. Saito, A hyperchaotic circuit family including a dependent switched capacitor, in Proc. Int. Symp. Circuits and Systems, vol. III, 2003, pp [14] E. M. Izhikevich, Resonate-and-fire neurons, Neural Netw., vol. 14, pp , [15] K. Chen and D. Wang, A dynamically coupled neural oscillator network for image segmentation, Neural Netw., vol. 15, pp [16] L. O. Chua and L. Yang, Cellular neural networks: applications, IEEE Trans. Circuits Syst. I, vol. TCAS1 35, pp , Oct [17] T. Stojanovsky, L. Kocarev, and U. Palitz, Driving and synchronization by chaotic impulses, Phys. Rev. E, vol. 54, pp , [18] L. M. Pecora, T. L. Carroll, G. A. Johnson, and D. J. Mar, Fundamentals of synchronization in chaotic systems, concepts and applications, Chaos, vol. 7, no. 4, pp , Toshimichi Saito (M 88 SM 00) received the B.E., M.E, and Ph.D. degrees in electrical engineering, all from Keio University, Yokohama, Japan, in 1980, 1982, and 1985, respectively. He is currently a Professor with the Department of Electronics, Electrical, and Computer Engineering, Hosei University, Tokyo, Japan. His current research interests include analysis and synthesis of nonlinear circuits, chaos and burcation, articial neural networks, power electronics, and digital communication. Dr. Saito is a Member of the INNS and IEICE. Hidehiro Nakano (S 02) received the B.E., M.E., and Ph.D. degrees in electrical engineering, all from Hosei University, Tokyo, Japan, in 1999, 2001, and 2004, respectively. He is currently a Research Assistanr with the Department of Computer Science and Media Engineering, Musashi Institute of Technology, Tokyo, Japan. His research interests include chaotic circuits and neural networks. Dr. Nakano is a Student Member of the INNS and IEICE.