Dynamics of Cortical Columns Self-Organization of Receptive Fields

Transcription

1 Preprint, LNCS 3696, 31 37, ICANN 2005 Dynamics of Cortical Columns Self-Organization of Receptive Fiels Jörg Lücke 1,2 an Jan D. Bouecke 1 1 Institut für Neuroinformatik, Ruhr-Universität, Bochum, Germany 2 Gatsby Computational Neuroscience Unit, UCL, Lonon WC1N 3AR, UK Abstract. We present a system of ifferential equations which abstractly moels neural ynamics an synaptic plasticity of a cortical macrocolumn. The equations assume inhibitory coupling between minicolumn activities an Hebbian type synaptic plasticity of afferents to the minicolumns. If input in the form of activity patterns is presente, selforganization of receptive fiels (RFs) of the minicolumns is inuce. Self-organization is shown to appropriately classify input patterns or to extract basic constituents form input patterns consisting of superpositions of subpatterns. The latter is emonstrate using the bars benchmark test. The ynamics was motivate by the more explicit moel suggeste in [1] but represents a much compacter, continuous, an easier to analyze ynamic escription. Keywors: cerebral cortex, cortical columns, non-linear ynamics, selforganization, receptive fiels 1 Introuction The minicolumn is believe to be the smallest neural moule consisting of roughly a hunre neurons which are stacke orthogonal to the cortical surface. Axons an enrites of pyramial cells in the same minicolumn bunle together an are assume to be strongly interconnecte [2, 3]. Connectivity of inhibitory cells suggests inhibition between the minicolumns. Minicolumns combine together to what is calle a macrocolumn or segregate [4]. Minicolumns of a macrocolumn receive input from the same source, e.g. a patch of the boy surface, but in the ault brain they react ifferently to ifferent types of stimuli, i.e., the minicolumns possess ifferent RFs. At birth afferents to cortical columns are foun to be relatively unspecific (see, e.g., [5]). The subsequent specialization is believe to be mainly riven by synaptic plasticity an to crucially epen on sensory input. In this paper a ynamical system is presente which moels the neural ynamics of minicolumn activities in a macrocolumn an the specialization of minicolumnar RFs on the basis of Hebbian plasticity.

2 2 Activity of Minicolumns Instea of explicitly moeling the neural connectivity within a macro- an minicolumn [1] we consier an abstract ynamics of the activity p α of minicolumn α in a macrocolumn of k minicolumns 3 : t p ( α = a p α pα h(p) p 2 ) α + κ Iα + σ η t, (1) where a is a time constant, κ the coupling strength to external input I α, an where σ 2 is the variance of zero-mean Gaussian white noise 4 η t. h is a function of the activities of all k minicolumns of the macrocolumn p = (p 1,..., p k ). Dynamics (1) is a simple choice for moeling mini- an macrocolumn properties. An abstract erivation of (1) an a non-linear analysis of its ynamical properties can be foun in [6]. The ifferent summans on the right-han-sie (rhs) of (1) can be consiere as moeling ifferent neuro-ynamical aspects: a (p α ) 2 moels self-excitation by excitatory interconnectivity within a minicolumn. a (p α ) 3 moels negative fee-back ue to neural refraction times which naturally limit the minicolumn activity. a p α h(p) moels inhibition by the minicolumns of the macrocolumn. The function h(p) is greater than zero. Note that because of the multiplication with p α this term cannot rive the activity to non-biological negative values. If we choose as inhibition function h(p) = ν max {p β}, it can be shown that β=1,...,k for zero input an zero noise the system possesses exponentially many stationary points [6]. ν (0, 1) plays the role of a bifurcation parameter. For ν (0, 1 2 ) there are (3 k 2 k + 1) an for ν ( 1 2, 1) there are 2k stationary points. In the point of structural instability, 1 2 = ν c, 2 k k 1 non-trivial stable stationary points loose their stability in subcritical bifurcations. Analytical expressions for all stationary points an for their stabilities can be erive [6]. The system qualitatively reprouces the bifurcations observe in the explicit moel efine in [1]: if ν is increase from a value ν < 1 2 to ν > 1 2 an if the macrocolumn has been in its symmetric stable state (all minicolumns are equally active) the minicolumns are eactivate via a process of symmetry breakings 5 (compare [1]). For non-zero input the symmetry is broken on the basis of small ifferences between the inputs I α to the minicolumns. The smallest value of ν for which the eactivation of a minicolumn occurs is a measure for the input strength relative to the other inputs. 3 p α(t) can be thought of as the fraction of neurons in minicolumn α that have spike uring a short fixe time-interval aroun t. 4 which is taken to be ifferent for each α 5 Note that for symmetry breakings infinitesimal perturbations are require, e.g., using a non-zero noise term with very small stanar eviation.

3 3 Self-Organization of Receptive Fiels We operate the system with oscillating inhibitory gain factor ν. An oscillation or ν-cycle starts with a ν = 0 interval uring which the system stabilizes the symmetric stable stationary state uner the influence of noise (σ > 0). Subsequently, ν is increase from a value ν min to a maximal value ν max (see Fig.1). The input to the minicolumns originates from a set of input units p E 1 to pe N (p E j [0, 1]) an is meiate by afferent connections R αj : I α = N j=1 R αjp E j. The afferent fibers we take to be subject to synaptic plasticity of the form: ( ) N t R αj = E p α p E j ( Ep α p E l )R αj H(χ A(t)), (2) l=1 where H(x) = 0 for x 0 an H(x) = 1 for x > 0 is a step function an where E is a synaptic growth factor. The positive term on the rhs of (2) moels Hebbian type synaptic plasticity. It is only greater than zero if minicolumn α an input unit j are simultaneously active. The negative term insures that j R αj converges to one for all α. The RFs, R α = (R α1,...,r αn ), are only moifie if the over-all activity A(t) = k α=1 p α falls below a threshol χ which ensures that learning takes place only after a number of minicolumns have been eactivate. The system is sketche in Fig.1 an the complete ynamics now reas: { 0 if t < T init ν(t) = (ν max ν min ) t T init, (3) T init + ν min if t T init ( ) N t p α = a p α p α ν(t) max {p β} p 2 α + κ R αj p E j + σ η t, (4) β=1,...,k j=1 ( ) N t R αj = E p α p E j ( Ep α p E l )R αj H(χ A(t)), E = ǫ N, (5) l=1 where t = mo(t, T), i.e., t = t nt where n is the greatest integer satisfying t nt 0. ǫ is the relative synaptic growth factor. Equations (4) an (5) are a system of non-linear ifferential equations couple to an oscillation given by (3). In simulations the oscillation is chosen to be slow compare to the ynamics of p α. We stuy the system behavior by exposing it to ifferent kins of input. From a given atabase with ifferent input patterns P [0, 1] N we present a ranomly chosen pattern P o uring each ν-cycle, i.e., P o efines the values of the input units for the uration of a ν-cycle, p E = P o. An input pattern P is, for visualization purposes, isplaye as two-imensional grey-level image. Before we can start simulating the ynamics we have to choose a suitable set of parameters. To choose a consier an isolate minicolumn without selfexcitation ( t p = a p3 ). In this case we expect that, e.g., an activity p = 1.00 rapily ecays to a value close to zero, e.g. p = 0.05, in about 1ms (the orer of magnitue of action potentials an refraction times). For the activity levels

4 p E 1 p E 2 p E N R 1 R 2 R 3 p 1 p 2 p 3 Inhibition Fig. 1. Sketch of a macrocolumn with k = 3 minicolumns. Interactions are inicate using arrows. The inhibition between minicolumns is visualize using a symbolic inhibitory neuron. p(0ms) = 1.00 an p(1ms) = 0.05 we get a 200ms 1. Note that this is a very coarse estimate ue to the arbitrariness in choosing the activities but one obtains the orer of magnitue of a. The value of the coupling κ is taken to be only a small fraction of the value for a, κ = 1.0ms 1, an stanar eviation σ of the Gaussian white noise is taken to be only a fraction of κ, σ = 0.12ms 1. For the oscillation of ν (3) we choose a perio length of T = 25ms an a time of T init = 2ms with ν = 0 an aitional noise to reset the ynamics. After initialization ν is increase from ν min = 0.3 to a value ν max = 0.55 which is slightly greater than the critical value ν c = 0.5. For the ynamics of Hebbian plasticity (5) we choose ǫ = 0.2 an a threshol of χ = Simulations Equations (3) to (5) can now be numerically simulate (e.g., using the Euler metho for stochastic ifferential equations). In the first experiment we use the set of 42 input patterns isplaye in Fig. 2A (compare [1]). By simulating ynamics (3) to (5) with k = 6 an parameters as given above we get RF selforganization as can be observe in Fig.2B. After ranom initialization the RFs specialize to ifferent classes of input patterns. If we have fewer minicolumns than major classes exist, we get coarser RFs (see Fig. 2C) an if we have more minicolumns we get RFs with higher specialization egrees (see Fig.2D). In the secon experiment we use the bars test [7] in orer to emonstrate the system s ability of learning a istribute coe for input consisting of subpattern superpositions. We operate the system using the same parameters as in

5 R 1 R 2 R 3 R 4 R 5 R 6 R 1 R 2 R 3 R 4 R 5 R 6 Fig.2. In A the set of input patterns is isplaye (N = 16 16). During each ν- cycle one ranomly chosen pattern of this set is presente. In B the moification of the RFs of a macrocolumn with k = 6 minicolumns is isplaye. After 1000 ν-cycles six ifferent pattern classes are represente. The RFs egree of specialization further increases thereafter to a final egree. C RF specialization (after 250 ν-cycles) if an abstract macrocolumn with k = 3 minicolumns is use with the same input. D RF specialization (after ν-cycles) if a macrocolumn with k = 9 is use. the previous experiment an we use a bars test with b = 8 bars. Each bar occurs in an image with probability 1 4 (see Fig.3A). As can be seen in Fig. 3B, RF self-organization results in a representation of all bars. In 200 consiere simulations with k = 10 minicolumns a bars test with above parameters require less than 600 ν-cycles in 50% of the cases to represent all bars (less than 410 in 20% an less than 950 ν-cycles in 80% of the simulations). The system foun a correct representation for all bars in all simulations an is robust against various perturbations to the bars test. Note that the results for the bars test show an improvement compare to the explicit system presente in [1] which requires more ν-cycles for the same bars test. Thus ynamics (3) to (5) represent not only an abstraction but, at least in the here iscusse bars test, also an improvement of the explicit ynamics in [1] (also compare [8]). Note that alreay the system presente in [1] has on the basis of extensive measurements shown to be highly competitive to all other systems suggeste to solve the bars test. 5 Conclusion On the basis of recurrent activity in cortical minicolumns, oscillatory inhibitory coupling between the minicolumns, an phase couple Hebbian synaptic plasticity of afferents we erive a system of couple ifferential equations which moels self-organization of RFs of cortical minicolumns. Self-organization allows a macrocolumn to represent input using istribute activity of its minicolumns relative to an oscillation. The moel combines most often inepenently is-

6 Input patterns: R 1 R 2 R 3 R 4 R 5 R 6 R 7 R 8 R 9 R 10 Fig.3. A A selection of 22 typical input patterns (N = 16 16) of a bars test with 8 ifferent four pixel wie bars. B Typical example of the self-organization of the RFs of a macrocolumn with k = 10 minicolumns. During each ν-cycle a ranomly generate input pattern of the upper type is presente. cusse aspects of neural information processing an is functionally competitive in a stanar benchmark test for feature extraction. Acknowlegment. Funing by the Gatsby Charitable Founation is gratefully acknowlege. References 1. J. Lücke an C. von er Malsburg. Rapi processing an unsupervise learning in a moel of the cortical macrocolumn. Neural Computation, 16: , V. B. Mountcastle. The columnar organization of the neocortex. Brain, 120: , D. P. Buxhoeveen an M. F. Casanova. The minicolumn an evolution of the brain. Brain, Behavior an Evolution, 60: , O. V. Favorov an M. Diamon. Demonstration of iscrete place-efine columns, segregates, in cat SI. Journal of Comparative Neurology, 298:97 112, K. E. Schmit, D. S. Kim, W. Singer, T. Bonhoeffer, an S. Löwel. Functional specificity of long-range intrinsic an interhemispheric connections in the visual cortex of strabismic cats. Journal of Neuroscience, 17: , J. Lücke. Dynamics of cortical columns sensitive ecision making. ICANN 2005, accepte. 7. P. Föliák. Forming sparse representations by local anti-hebbian learning. Biological Cybernetics, 64: , J. Lücke. Hierarchical self-organization of minicolumnar receptive fiels. Neural Networks, Special Issue New Developments in Self-Organizing Systems, 17/8 9: , 2004.