F.P. Battaglia 1. Istituto di Fisica. Universita di Roma, La Sapienza, Ple Aldo Moro, Roma and. S. Fusi

Transcription

1 partially structured synaptic transitions F.P. Battaglia 1 Istituto di Fisica Universita di Roma, La Sapienza, Ple Aldo oro, Roma and S. Fusi INFN, Sezione dell'istituto Superiore di Sanita, Viale Regina Elena 299, Roma September 7, 1995 Abstract We show that stochastic learning of attractors can take place in a situation in which either only potentiation or only depression of synaptic ecacies are caused in a structured Hebbian way. In each case the transition in the opposite sense take place at random, but occurs only upon presentation of a stimulus. The outcome is an associative memory with the palimpsest property. It is shown that structured potentiation produces more eective learning than structured depression, i.e. it creates a network with a much higher number of retrievable memories. Introduction Unsupervised learning of uncorrelated stimuli in attractor networks was recently described [1] as a stochastic process on the distribution of synaptic values characterizing the network. In this approach synapses have a nite number of stable states (ecacies). Learning is schematized as a random walk among them. Probability of each step is determined by the activities of the two neurons connected by the synapse. Neuronal states are in turn driven by external stimuli, represented as a stream of words, or patterns, each consisting of N 0's or 1's each. Patterns are uncorrelated with each other and have a given coding level f, which is the average fraction of neurons driven by a stimulus. Stimuli drive synaptic transitions by means of the activity correlations of the two neurons connected by each synapse. If both neurons on a synapse have high activity there is a non-zero probability for the synapse to transit into an higher ecacy state; anticorrelation produces transitions depressing the synaptic ecacies; two inactive neurons leave the synapse unchanged. Synaptic modications induced by more recently presented patterns stack upon those produced by older patterns. As more patterns are presented, traces of older ones gradually fade away. After the learning of a given pattern, the network can learn other patterns without destroying the retrieval of the rst one. Thus, the network reaches and permain into a steady state, with new patterns learned starting from an asymptotic distribution of synaptic weights. Always the same number of patterns is stored in memory, ready for retrieval. Retrievable patterns are the most recently learned ones. If we consider other learning schemes, in which we start from a distribution of connection weights dierent from the asymptotic one, we obtain other memory behaviours: in some situations we could have the network storing the rst patterns presented, with more strength than the following ones, and the memory of the former lasts for more time than the memory of the latters. With certain parameters choice, the network is not capable of learning (and forgetting), when it has reached the 1 Present Address: S.I.S.S.A. { Via Beirut, 2-4, Trieste 1

2 primacy phenomenon [2]). Here we concentrate on the steady state memory, when only a constant number of the newest patterns is stored in the network (the recency phenomenon [2]) The maximum number of such patterns represents the storage capacity of the network. It is the number of the most recent stimuli that can be recalled. In [1] this limit is investigated by checking the bitwise stability of a recalled pattern by a signalto-noise computation. In other words, after the learning of p uncorrelated patterns in the stream, the rst pattern is presented to the network for retrieval. The synaptic input generated by this pattern, given the resulting learned synaptic matrix, is checked to see if it reproduces the originally learned pattern. It turns out that, with xed learning parameters and a stream of uncorrelated patterns, a network cannot store any patterns without errors. Allowing the coding level f and the transition probabilities to vary appropriately with the number of neurons N as N 1, one can achieve optimal storage and retrieve as many as (N= log N) 2 patterns. Weaker learning requirements In [1] the structure of the correlations in a learned pattern determines the probability of both synaptic potentiation and depression. A synapse is potentiated (transits to a stable synaptic state of higher ecacy) with probability q + if the neurons connected by the synapse have correlated activities. If the two neurons have anticorrelated activities, depression takes place with probability q?. Here we report that learning and recall of patterns are possible even when the requirements on the learning process are weaker. Such weaker conditions could be that only the synaptic transitions in one of the two possible directions (potentiation or depression) depend on the correlations of neural activities in the stimuli. The transitions in the other direction could be random. In fact, in both cases the qualitative behaviour of the network in learning and retrieval are maintained. Two learning scenarios with examples of weaker requirements are: 1. A synapse J ij is potentiated with probability q +, if i = j = 1 (both neurons connected by the synapse are active). The depression (transition to a stable synaptic state of lower ecacy) takes place randomly, with probability z, independently of activity values, during the presentation of a pattern. 2. A synapse J ij is depressed (or pruned) with probability q?, if the two neurons connected by it have anticorrelated activities. Potentiation takes place at random, independently of activity values, with probability r, during the presentation of a pattern. It should be underlined that in both options the random transition have been coupled to the presentation of the patterns, even though they are partially independent of their structure. This restriction is introduced to avoid the appearance of an additional time scale. If any type of transition becomes independent of the presentation of stimuli, the synaptic states can no longer be considered stable and their transition probability determines a time scale in which memory is destroyed spontaneously. Following a long enough interval in which no pattern is presented to the network, the synaptic structure will become completely dominated by the random transitions. Possible biological underpinning for the modeled possibilities will be discussed at the end. In what follows the scope is limited to synapses with two stable values 0,1 and neurons with 0,1 activities. As in ref. [1], most eects of interest appear already in this reduced context. It is found that in both cases the network is able to learn, in the sense that patterns that have been learned in the past can be associatively recalled. The learning process produces a palimpsest [2, 3, 4, 5]. If the learning parameters are xed, no patterns can be stored, due to correlations in synaptic structure 2

3 learning parameters, i.e. allowing f; q + (q? ); z(r) to decrease with increasing N, one can achieve a network which learns continuously and can maintain a much higher number of distinct patterns. But here the two scenarios part ways: In scenario 1, when potentiation is structured the capacity can become as large as (N= log N) 2 patterns. While for scenario 2, no manipulation of the parameters can make the capacity larger than N=(log N) 2. The theoretical framework The framework is the same as that of ref. [1]: the network is presented a stream of p uncorrelated stimuli. Since the synapses are assumed to have a nite number of states, after the removal of each stimulus, the synaptic ecacies converge rapidly to one of the stable states. So, on long time scales, the learning process can be seen as a walk among the stable states. In fact this is a random walk for two reasons: 1. the stimuli are assumed uncorrelated, so each synapse will see a random sequence of pairs of activities ( i ; j ) on the two neurons connected by it [1, 6] 2. The transition from one stable synaptic state to another may itself be stochastic [1, 2]. There are at least two possible sources of synaptic stochasticity: the dynamics of the synaptic modication and the random uctuations in the spike rates of the two neurons connected by the synapse. This stochastic process is described by a conditional probability distribution function for the synaptic values p (; ~ J ), giving the probability of nding the synaptic value J following the presentation of p uncorrelated patterns, conditional on the appearance of neural activities, ~ in the presentation of the rst of the p patterns. Since the synapses are assumed to have a nite number of states the stochastic process is generically ergodic and leads to an asymptotic distribution after a large enough number of presentations. This distribution is taken to be the initial distribution upon which learning takes place. Also the conditional distribution p J(; ) ~ is driven towards the asymptotic form, as p 1, by the presentation of p uncorrelated patterns following the rst one, the one that produced the conditioning. The dynamics of the conditional X distribution is p J(; ) ~ = 1 K(; )( ~ p?1 ) KJ ; (1) K=0;1 where is the stochastic transition matrix of the process: KJ is the probability of going from state K to state J. For instance consider the model of [1] in which the network is presented sparse patterns (f is the mean fraction of active neurons), the synapse has two stable states, and both depression and potentiation are structured: = 1? f(1? f)q? f(1? f)q? (2) f 2 q + 1? f 2 q + f 2 q + is the probability of potentiation (the probability of having two active neurons produces the f 2 factor), and f(1? f)q? is the probability of depression. Given p J(; ~ ), one computes the statistics of the synaptic inputs among neurons upon the presentation, for retrieval, of the rst of the p patterns. The mean synaptic input into neurons which had activity during the presentation of 1 is: 3

4 hh 1 i i = h N j=1 J ij (p) 1 j i = ~ ~ J=0;1 J ~ p J(; ~ ) (3) where ~ is the probability that stimulus number 1 induced the state ~ on a neuron. For uncorrelated and sparse patterns we have: 1 = f, 0 = 1? f (see ref. [1]). Retrieval is possible if the distribution of h i is such that there exists a threshold which separates the neural inputs of the neurons which had been active in the learned pattern (hh i i 1 ) from those of the neurons which had been quiescent (hh i i 0 ). So we dene the signal of pattern 1 as the distance between the averages of the two distributions: S = hh p i i 1? hh p i i 0 = X ~ X ~ J=0;1 The uctuations of the two neural inputs are estimated by: R 2 () = h(h p i? hh p i i ) 2 i J ~ [ p J(1; ~ )? p J(0; ~ )]: (4) If the random variables h p i are gaussian, then total noise is: R 2 = 1 2 [R2 (1) + R 2 (0)] Each term is given by [1]: h(h p i? hh p i i ) 2 i = * 1 N 2 j;j6=i k;k6=i N? 1 N 2 X X + J ij 1 j 1 k X ~ X ~ XX (N? 1)(N? 2) N 2?* 1 N X j;j6=i J ij 1 j J 2 2 ~ p (; ~ J )? hh p i i A 2 + ~ ~ 0 ~ ~ 0 XX J = JJ 0 ~ ~ 0 p 0(; JJ ; ~ ~ 0 ) A (5) J 0 1 The rst term is computed as if the synaptic values were uncorrelated. The second term is due to correlations between J ij and J ik, which have a neuron in common. p 0(; ~ JJ ; ~ 0 ) is the distribution of J ij ; J ik conditional on the appearance of the values ; ; ~ ~ 0 on neurons i, j, k, in the rst pattern. The evolution of the conditional distribution, following the presentation of 1 can be fully described in terms of the eigen-values and the eigen-vectors of. We have (see e.g. [7]): p J = 1 J + p?1 X K 1 K(; ~ )u K v J (6) where 1 J is the asymptotic distribution (the term corresponding to the unitary eigen-value), which is the palimpsest on top of which the new pattern 1 is learned. is the subleading eigen-value of ; 1 K(; ) ~ is the conditional distribution produced on top of the palimpsest, immediately following the presentation of 1. This is the extent of `one shot' learning. u K, v J are, respectively, the right and left eigen-vectors of corresponding to. emory maintenance is tested by evaluating the signal to noise ratio upon presentation of one of the patterns presented in the past as a stimulus for retrieval. Then assuming a gaussian distribution for the signals, one uses the criterion that if the square of the signal to noise ratio S 2 =R 2 satises 4

5 as N becomes large, retrieval is assured. See e.g. ref. [1, 8]. S =R > log N (7) Structured potentiation and random depression In this scenario (no. 1) the transition matrix is = 1? z z f 2 q + 1? f 2 q + (8) in which the rst row (column) corresponds to the initial (nal) synaptic value 1. z is the probability of leaving the upper state (depression) following the presentation of the stimulus. f 2 q + is the probability of potentiation: it occurs with probability q + when both neurons on the synapse have high activity (f 2 factor) during the presentation of the stimulus. Note that the probabilities of unstructured transitions, z and 1? z intervene together with the structured ones. This is a consequence of our assumption that unstructured depressions appear only during the presentation of a stimulus. This matrix has the eigen-values: The asymptotic synaptic distribution is 1 = 1 = 1? z? f 2 q + < 1 (9) 1 = (p + ; p? ) = f 2 q + z ; z + f 2 q + z + f 2 q + where p + and p? are the probabilities of nding 1 or 0 synapses, respectively, in the asymptotic palimpsest. It has no memory. The expression for the signal is (eq. 4 and [1]) The uncorrelated part of noise is (10) S = p?1 q +p? f (11) R 2 = f N [p +? p + f + O(f p?1 )] (12) It is evident form Eq. 5 that the correlation term is O(1) in N when N is large. Therefore it will dominate in the limit N 1. With xed learning parameters one has, for Signal/Noise ratio S R 2 2(p?1) cost: (13) and the condition (S=R) 2 log N is never fullled, not even for one single pattern stored. Actually the Weisbuch and Fogelman-Soulie condition (7) is a very strict constraint. It implies that pattern can be retrieved, with no errors at all, with probability 1, in the limit N 1 [8]. The stochastic nature of learning dynamics is such that it cannot satisfy this constraint for any number of stored patterns. This is mainly because of correlations induced in the synaptic structure. If f tends in an appropriate fashion to 0 when N 1, correlations become negligible, and the Signal/Noise computations can proceed with the uncorrelated noise term alone, see e.g.[1]. 5

6 f N pf 5 (14) We note that this condition is looser than the corresponding one in [1] ((f=n) pf 3 ), where both potentiation and depression are structured. This is due to the fact that here only upward transition generate correlations in the synaptic structure, and each of them requires two neurons to be active, implying a factor of f 2. For example, if we choose f log N=N and p N 3 =(log N) 4 condition is satised and we can consider the uncorrelated term only. To leading order in f: The condition (S=R) 2 log N reads p c < S R 2 = 2(p?1) N f (q +p? ) 2 p + (15) 1?2 log log N f (q + p? ) 2 : (16) (log N) p + provided N f (q+ p? ) 2 (log N) p + > 1 (17) This last condition should be satised in order to allow storage and retrieval of at least one pattern. In fact it is equivalent to impose condition 7 for the most recently learned pattern (see equation 15 when p = 1). It represents a constraint on learning parameters. Now, if we put z f 2 (log N=N) 2 and we keep q + xed we have: Expanding the denominator in Eq. 16 we nd 1? x with x O (log N =N) 2 : p c N log N 2 ; (18) provided, of course, condition (17) is still satised. In fact, for N large, the N dependence on the left hand side of Eq. 17 cancels out, and the condition can be satised by choosing appropriately the values of the nite constants q + ; fn= log N; z=f 2. With parameters values yielding optimal storage level, the average number of synaptic up and down transition is of the same order. Hence, in this scenario, like in [1], the best storage is of order N 2. Weaker conditions on the learning transitions do not change the dependence of the capacity on N, for large N, and at worst can lower the capacity by a nite multiplicative factor. Structured pruning and random synapse generation This is scenario no.2. In this case reads = 1? f(1? f)q? f(1? f)q? r 1? r 6

7 sion. The last takes place only when the two neurons connected by the synapse have anticorrelated activities. The eigen-values are 1 = 1; = 1? r? f(1? f)q? < 1: The asymptotic distribution is 1 (p + ; p? ) = r f(1? f)q? ; r + f(1? f)q? r + f(1? f)q? Also in this case correlations give a noise term O(1) in N, with xed parameters. So, condition (7) could never be satised. Correlations can be made negligible by adjusting the coding rate f. The condition for the uncorrelated part of noise to dominate is f N pf 3 (19) The dierence with respect to condition(14) is due to the fact that in this case correlations between synapses are caused by structured pruning transitions. These transitions occur with a probability O(f), therefore they are more frequent than structured transitions (potentiations) in the previous scenario. This explains the stricter condition on p we obtain in this case. With f log N=N (this is the fastest possible decay, see e.g. eq. 17, if at least one pattern is to be learned), it is satised only if p N (20) (log N) 2 If this is the case, we can continue the calculation considering uncorrelated noise only. The signal, Eq. 4, is S = p?1 q? p + f (21) The uncorrelated noise is: R 2 = f N [p +? p 2 +f + O(f p?1 )] (22) like in the previous case. The signal-to-noise ratio, Eq. 22, is given by S R 2 = 2(p?1) Nfq 2?p + (23) With a critical capacity: p c < Again, if N f q 2? p + (log N) 1 log N f q2?p + : (24)?2 log (log N) > 1; (25) one would be able to learn with this dynamics, and pattern retrieval is possible for as many as p c uncorrelated patterns. Storage capacity is again O(log N) if f, q?, and r are xed. 7

8 eective, due to the constraint Eq. 25. The best one can do is take f (log N=N) and r f. Then, to obtain the constraint on the critical capacity: 1? x with x = O(log N=N) p c < N log N : (26) The condition (20) is stricter and represent the actual storage limit for this network. Therefore, the capacity is N=(log N) 2. With structured depression, the total number of structured transitions f(1? f)q?. It cannot be made smaller than O(f) in the limit f 0 since constraint 25 implies that q? should be kept nite. Otherwise the network will lose its capability to store even a single pattern (this is the meaning of (24)). In the previous scenario the number of transitions could be reduced to O(f 2 ) as f 0, still assuring safe pattern storage and retrieval. As more transitions are requested to learn each pattern, traces of older pattern are rubbed out more rapidly. In fact, in both cases, a term similar to the mean number of transitions per presentation appears in the expression for, as one would easily argue. How fast the network tends to the asymptotic distribution, forgetting previously learnt information, is actually depending on the eigen-value. The closer approaches to 1, the larger will be the storage capacity. Conclusions In the hebbian paradigm learning depends on synaptic modications driven by neural activities. The understanding of the mechanisms of synaptic formation and plasticity is therefore one of the main problems for neurobiology. Although much eort has been devoted to this question in the last decades, a complete picture is still lacking. Very important progresses have been pursued in comprehension of LTP. It seems to be the result of many eects, (see e.g.[10] or [11] for a review). Other similar processes, such as selective stabilization of synapses induced by dierential, activityregulated release of Trophic Factor (see e.g. [12]) by the post-synaptic neuron are known. Search for mechanisms of long term, activity correlated synaptic depression has been another intensively studied subject for experimentalists. Nevertheless, the situation is far less clear than for the LTP case: Stanton and Sejnowski [13] claim for the nding of an hebbian LTD, induced by the anti-correlated pre- and post-synaptic activities. Despite of several eorts, no other research group was able to replicate the result, possibly conditioned by a very delicate parameter choice [14]. any other possibilities, resembling the hebbian postulates, have been proposed (see e.g. [15] for a review), like homosynaptic LTD, where decrease of synaptic ecacy is produced by an activation of the pre-synpatic neuron that is not followed by the activation of the post-synaptic one (the former is unable to excitate the latter). The heterosynaptic LTD is caused by a silent pre-synaptic neuron, together with the activation of the post-synaptic, induced by another input pathway. In theoretical models of attractor networks learning, necessity of LTD for information storing have often been called for. Here we wanted to show that an activity-dependent, \hebbian", LTD is not strictly necessary in order to store pattern. What is important, for this memory scheme (\palimpsest") to work, is the presence, along with an information carrying potentiation process, of mechanisms keeping the mean connectivity constant, thus preventing the saturation of the network. In order to obtain a real palimpsestic scheme (older memories are removed only when new objects are stored), a completely random depression mechanism is fully equivalent, as far as transitions of 8

9 network is inactive. This could be the case if we consider selective stabilization of synapses induced by Trophic Factor: activity triggers the elimination of connections by a reduction of Trophic factor release [12]. The proposal for random transition reects the lack of knowledge of LTD: we tried to gure out which are the minimal requests on synaptic plasticity for this learning paradigm to work. Of course, an activity-correlated LTD increases the signal, but only up to a multiplicative constant, leaving the scaling law for the storage capacity invaried (the capacity is O(N 2 =(logn) 2 )). We showed another possibility for learning, with depression transitions driven by stimuli, and random potentiation. We saw that learning is again possible, but the rst scheme is more ecient in storing memories (it can store and recall O(N 2 =(log N) 2 ) patterns) than the second one (with a capacity O(N=(log N) 2 )). This scheme probably has not a direct thinkable analog in biological synapses, and it is suggested as a theoretical possibility, mainly in comparison with the former scheme. We know from conditions (17) and (25) that, in the case studied, probabilities for structured transitions must be O(1) in f, if at least one pattern is to be stored. This implies that total number of transitions is O(f 2 ) in the rst case, and O(f) in the second. So, with structured potentiation, we need less transitions to store the same amount of information, and to get the same signal magnitude. Less transitions imply slower interference with the synaptic structure and therefore a longer storage capacity. In some sense, LTP appears as a more ecient storage mechanism than LTD, and it could be that it is actually the crucial one. Acknowledgement We are grateful to Prof. Daniel Amit for encouragement and criticism during the performance and the writing of this study. This study was done in the context of a project to develop an output network which can learn to read the attractors of an upstream attractor network. References [1] Amit D.J., Fusi S Learning in neural networks with material synapses, Neural Computation, 6, 957{982 [2] Wong K.Y.., Kahn P.E., Sherrington D A neural network model of working memory exhibiting primacy and recency, J. Phys A, 24, 1119{1135 [3] Nadal J.P., Toulouse G., Changeux J.P., Dehaene S Networks of formal neurons and memory palimpsests Europhysics Letters, 1, 532{542 [4] Parisi G A memory which forgets, J. Phys, A19, L617 [5] Burgess N., Shapiro J.L., oore.a Neural network models of list learning, NETWORK, 2, 399{422 [6] Heskes T.., Kappen B Phys Rev. A 44, 2718 [7] Cox D.R., iller H.D Theory of stochastic processes (ETHUEN & CO LTD London) [8] Weisbuch G. Fogelman-Soulie F Scaling laws for the attractors of Hopeld model, J.Physique Lett., 2, 337 9

10 for the specication of neuronal networks Nature, 264, [10] Larson J. Lynch G Induction of synaptic potentiation in hippocampus by patterned stimulation involves two events, Science, 232, 985 [11] Brown T.H. Kairiss E.W. Keenan C.L Hebbian Synapses: Biophysical mechanisms and algorithms, Ann. Rev Neurosci., 13, [12] Henderson C.E., 1986 Activity and the regulation of Neuronal Growth Factor etabolism, in Changeux J.P. and Konishi. eds. The neural and molecular bases of learning Springer-Verlag [13] Stanton P.K. Sejnowski T.J. Associative long-term depression in the hippocampus induced by hebbian covariance, Nature, [14] Paulsen O. Li Y.-G. Hvalby O. Andersen P. Bliss T.V.P.1993 Failure to induce Long-term depression by an anti-correlation procedure in area CA1 of the rat Hippocampal slice Eur. Journ. Neurosci., 5, [15] Christie B.R. Kerr D.S. Abraham W.C. Flip side of synaptic: Long-term Depression mechanisms in the hippocampus, Hippocampus, 4,