On embedding synfire chains in a balanced network

Transcription

1 On embedding synfire hains in a balaned network Y. Aviel,4, C. Mehring, M. Abeles and D. Horn 3 nterdisiplinary Center for Neural Computation, Hebrew University, Jerusalem, srael. Neurobiology and Biophysis, nstitute of Biology, Albert-Ludwigs-University Freiburg, Germany. 3 Shool of Physis and Astronomy, Tel Aviv University, Tel Aviv, srael. 4 To whom orrespondene should be sent. aviel@.huji.a.il 5 Deember 00 Abstrat We investigate the formation of synfire waves in a balaned network of integrate and fire neurons. The synapti onnetivity of this network embodies synfire hains within a sparse random onnetivity. This network an exhibit global osillations, but an also operate in an asynhronous ativity mode. We analyze the orrelations of two neurons in a pool as onvenient indiators for the state of the network. We find, using different models, that these indiators depend on a saling variable. Beyond a ritial point, strong orrelations and large network osillations are obtained. We looked for the onditions under whih a synfire wave ould be propagated on top of an otherwise asynhronous state of the network. This ondition was found to be highly restritive, requiring a large number of neurons for its - -

2 implementation in our network. The results are based on analyti derivations and simulations. ntrodution Synfire hains (SFCs) were first introdued by Abeles (99) as a model for solving a wide array of ognitive and omputational tasks. They inorporate rate oding with speifi synhronous ativity, and have been shown to be andidates for representing elementary ognitive funtions (Bienenstok, 995) suh as binding (Hayun, 00). A SFC ditates a well-defined onnetivity pattern among neurons in the form of feed-forward onnetions between pools of neurons. n a omplete hain, all w neurons in a pool ('pre-pool') onnet to all neurons in the suessive pool, thus reating a hain of pools. nput onnetions as well as outputs are allowed, as shown in Figure. - -

3 Figure : The onnetivity of a Synfire Chain. The onnetion between three pools is shown. n the omplete hain, eah neuron in a pool reeives inputs from all w neurons in the pre-pool (w=3 in this figure). n addition, eah neuron reeives inputs from (and projets to) many other neurons in the network. Note that a neuron an partiipate in many pools. f w, the number of neurons in a pool, is large enough, then a synhronized firing volley of most of the neurons in a pool, a pulse paket, may propagate along the hain (Diesmann, Gewaltig, & Aertsen, 999). To avoid terminologial onfusion, these feed-forward onnetivity shemes are referred to heneforth as hains, and the pulse paket propagating along a hain as a synfire wave (or simply a wave). A wave an propagate in a synhronized manner along a hain, or it an lose its synhrony, dissolving into the bakground ativity. A wave is said to be stable if it reahes the end of the hain as a synhronized pulse paket. We begin by fousing on a hain that is embedded in a large network, whose neurons produe some asynhronous low bakground ativity. ah neuron in the hain reeives oasional pulse pakets from its pre-pool, as well as some bakground ativity. What are the effets of the bakground ativity on the waves? Can waves travel along hains in a stable manner in the presene of bakground noise, or will they eventually dissolve into the bakground ativity? Can the bakground noise ause spontaneous emergene of waves along hains? Diesmann et al (999) have shown that if the number of neurons in a pool is large enough, and if the igniting pulse paket is synhronized and strong enough, the waves are stable in the presene of bakground noise

4 Naturally, the next question is whether we an model a network that is apable of low spontaneous asynhronous ativity with similar properties of a ortial tissue that is also apable of hosting stable synfire waves. n suh a network, the effet of the bakground ativity on the wave is important, but the effet of the hain on the asynhronous state of the network is also ruial. n this paper we investigate these issues by studying the orrelations of two neurons that belong to the same pool. We show that the orrelations allow us to distinguish between the different states of the network ativity. High orrelations our when the system is in an osillatory state, one where a synfire wave will be lost in the bakground of spontaneous ativity. The hallenge is to find the appropriate parameters suh that stable waves exist in a bakground of asynhronous global ativity and irregular spiking of individual neurons. This problem arises from the existene of two onstraints: On one hand the system is required to maintain a stable asynhronous state, thus enabling rate oding. On the other hand, we require the network to allow synhronous propagation of pulse pakets a kind of temporal ode. Are these two modes mutually exlusive? As we shall see, it is possible to set up a network with physiologial and anatomial realisti parameters that is apable of operating in both modes simultaneously. n the ontext of the two-neuron problem, we use a simple, analytially amenable, model that enables us to write an equation that desribes the evolution of orrelation along a hain. This equation is independent of the input firing rate and leads to the emergene of a new saling variable. A more realisti ntegrate-and-fire neuronal model does show dependene on firing rates. Nevertheless, we find that the saling variable still funtions as a ritial parameter

5 Moreover, the same holds not only for a pair of neurons, but also in the full network simulation. Our analysis, whih is based on orrelations, suffies for the disussion of the issues that we onfront; hene we do not delve into the evolution of firing-rates. (For the latter see Tettzlaff et al. 00). xperimentally, neuron ativity is typially haraterized by steady low (-5Hz) firing rates, with irregular spiking (Abeles, 99). To aommodate this observation with the known anatomial fat (Abeles, 99) that there an be many inputs (around 0,000 exitatory and,000 inhibitory synapses) to a neuron, it was suggested (Shadlen & Newsome, 994; Gerstein & Mandelbrot, 964) that exitatory and inhibitory inputs anel eah other out, thus reduing the mean input to virtually zero. Thus the firing of a neuron reflets flutuations of the membrane potential that eliit oasional rossing of the threshold. Another solution (Softky & Koh, 993) has been to argue that temporal orrelation of the PSPs eliit neuronal firing. We onsider a system that allows both solutions to oexist. First, the input is balaned, and seond, waves of ativity propagate by means of strong temporal orrelation. To reate large flutuations with a onstant mean, we balane the exitatory input with an inhibitory input. A network is said to be balaned (van Vreeswijk & Sompolinsky, 998) if eah neuron in the network reeives equal amounts of exitation and inhibition. ts membrane potential will then flutuate around some mean value and the firing proess is noise driven, and therefore irregular. Balaned networks (BN) have been shown (Brunel, 000) to mimi the in-vivo firing statistis of ortial tissue, and it is therefore plausible that ortial neurons reeive balaned input. BN has also been shown (van Vreeswijk & Sompolinsky, 998) to have a stable - 5 -

6 asynhronous state, as well as appropriate rate oding properties, suh as fast traking of hanges in external input rates. Generally, BN assumes sparse and random onnetivity. t is possible to embed SFCs in the exitatory-to-exitatory (-) onnetions of a BN, but this would violate the random onnetivity assumption. Would this also disrupt the desired properties of a BN? As we show, there is a wide regime of parameters in whih suh lak of randomness has little effet. The Model Following Brunel (000), we use an ntegrate-and-fire (AF) model, in whih the i-th neuron s membrane potential, V i (t), obeys the equation: dvi ( t) () τ = Vi ( t) Ri ( t), dt where i (t) is the synapti urrent arriving at the soma and R is the membrane resistane. Spikes are modeled by delta funtions; hene, the input is written as f () R ( t) = J δ ( t t D), i j t f j ij j where the first sum is over different neurons, whereas the seond sum represents their f f spikes arriving at times t = t D. t is the emission time of the f-th spike by neuron j j j, and D is a transmission delay, whih we assume here to be the same for any pair of neurons. The sum is over all neurons that projet their output to neuron i, both loal and external afferents

7 When V i (t) reahes the firing threshold θ, an ation potential is emitted by neuron i, and after a refratory period τ rp, during whih the potential is insensitive to stimulation, the depolarization is reset to V reset. The following parameters were used in all simulations: The transmission delay D =.5ms, the threshold θ = 0mV, the membrane time onstant τ = 0 ms, the refratory period τ rp = 0.5ms, the resetting potential V reset = 0mV and the membrane resistane R = 40MΩ. The inhibitory and exitatory neurons have idential parameters. We used the SYNOD simulation environment (Diesmann, Gewaltig, & Aertsen, 995) for simulations with less than 0,000 neurons and a parallel version of SYNOD for simulation with more neurons. n the simulator, the Lapique (Tukwell, 988) model was used as an AF model with time steps of 0.ms. Unless otherwise speified, we set the synapti weights J = J = J, J = J = -gj with g=5 and J=0.4mV. The onstant g is the relative strength of the inhibitory synapses, and J is the PSP amplitude. Note that the synapses are weak, as J<<θ, and the total input to a neuron is approximately balaned. When simulating a pair of neurons in a pool, the neurons reeive loal exitatory and inhibitory input from the network as well as external input from external exitatory soures that feed the entire network. For simpliity s sake, we assume that the neurons have a single pre-pool. Thus, the neurons reeive 4 types of input, 3 of whih are loal: K inhibitory synapses, w exitatory synapses that ome from the previous pool, and K-w exitatory synapses from other random onnetions in the network. The fourth type of input is represented by K exitatory synapses that onvey external - 7 -

8 stimulation. ah of these inputs is modeled as an independent Poisson proess with rates υ, υ and υ ext, for the loal inhibitory, loal exitatory and external exitatory synapses, respetively. n the simple model, whih will be defined later, we refine the arhiteture further to inlude ommon inputs due to random onnetions. As a method for induing orrelations between spike-trains, we use a mother proess : a Poisson proess with rates v/ρ in, from whih we opy spikes with probability ρ in. The method is desribed in detail in Kuhn et al (00). n order to measure the output orrelation between two spike trains, we first binned the spike train into ms bins (eah spike is of width 0.ms), and then omputed the zero-lag ross-orrelation. When simulating the entire network, we simulate all the N exitatory neurons, as well as the N =N /4 inhibitory neurons. A Poisson proess with rate υ ext K simulates the external input. υ ext is in units of υ thre, where υ thre θ JKτ is the minimal rate needed to emit a spike within τ milliseonds (on the average) in a neuron that does not get other inputs. n this paper we set ν =. 5ν, whih is equivalent to an external rate that ext thre ranges between Hz and Hz, depending on the value of K. The total input to a neuron is, therefore, nearly balaned. There is a small bias toward an exess of exitation, whih ontrols the firing rates. A sparse onnetivity is required, both to adhere as losely as possible to biologial values and to indue a soure of randomness. Here we set the sparseness ε to be 0., i.e. K = ε N, K = εn. n addition to the external input (whih are K exitatory afferents), eah neuron in the network reeives K exitatory and K inhibitory afferents, randomly sampled, from the exitatory and inhibitory population, respetively

9 Wiring the network is performed as follows: n general, all the onnetions are hosen at random, exept for the - onnetions, whih are a superposition of hains. For wiring the - onnetions, eah randomly hosen pool of w neurons is wired in an all-to-all fashion to the onseutive pool. We ontinue this proess until the desired number of pools has been reahed. We also ensure that a neuron will not reeive more than K loal onnetions. f a neuron does not reeive K afferents, and there are no more pools to be added we randomly hoose additional soure neurons for it. At the end of the proess, all the exitatory neurons have exatly K afferents, and there are hains hardwired into the onnetivity matrix of the - onnetions. The resulting onnetivity is a mixture of random onnetivity and hains. The number of pools is restrited by the number of exitatory synapses, and is therefore less than N K. n our simulation we set the number of pools at 000. f this was not w possible due to the above limit, the simulation was not performed. A tighter bound on the number of pools is given by the apaity of the system, as disussed in Setion 6. 3 The Challenge The foregoing enabled us to pursue the following senario. We start with a BN that embodies hains in its onnetivity matrix, as desribed in the previous setion. n the absene of external ignition of a wave, we onsider the state with global asynhronous ativity to be the ground state of the system. xternal ignition of a SFC will perturb the system from its ground state, leading to wave propagation, masked by the overall - 9 -

10 ativity. One the ignited wave has reahed the end of the hain, the system should return to its ground state. Brunel (000) has shown that BNs of AF neurons display four major stable states for different regimes of the external input rate, v ext, and the level of balane. The four states are all ombinations of the two possible global (population level) ativities and the two possible neuronal firing patterns; the global ativity an be either synhronous or asynhronous, whereas the neurons an fire in a regular or irregular fashion. Of speial interest to us is the state that is haraterized by asynhronous population ativity and irregular neuronal firings. Following Brunel s terminology, this is the asynhronous-irregular (A) state. The network model desribed in Setion is idential to Brunel s model exept for the onnetivity; namely, in Brunel s model the onnetivity is ompletely random, whereas we embody hains in the network. Another differene is in the dynami parameters. Brunel varied v ext, and g in order to obtain different states, here we only vary the hain width, keeping v ext, and g onstant. (a 0. K=5000 w=50 (b 0. K=5000 w=50 ) Population rate ) Population rate Time [ms] Time [ms] Figure : Population rate of a BN with embedded SFCs indiates a qualitative hange in network behavior, as hain width inreases. (a) With a hain of width w=50, the network is in an A state, haraterized by low asynhronous population ativity and very mild osillations with - 0 -

11 small amplitudes. Average firing rate of the exitatory (inhibitory) neurons in this mode is.9hz (5.7Hz). (b) nreasing the width to w = 50, a synhronous state appears, haraterized by sporadi abrupt synhronized bursts with large amplitudes. The average firing rate of the exitatory (inhibitory) neurons is 36Hz (73Hz). Population rate is the perentage of exitatory neurons that fired in a 0. ms. The network parameters are N =50,000, K = 5000, number of pools: 000. As shown in Figure a, a BN that embodies hains displays behavior similar to that of Brunel s A state (Brunel, 000). This suggests that a small departure from randomness does not refute the results obtained for random networks. Consequently, we use the A state as the ground state of our system. We searh through parameter spae to find this regime in our model. To allow stable propagation of a wave along a hain, we need large pools (Diesmann et al., 999). However, when w is too large, as is the ase in Figure b, the A state loses its stability, with a transition similar to the phase transition that takes plae in Brunel s model when moving from the A state to the synhronous-irregular (S) state. n the S state, as in Figure b, the population ativity is synhronous, and the neurons fire irregularly. (t should be noted, however, that the global osillations in our ase exhibit stronger irregularity than the one of Brunel s S state. See Figure 8 there.) n the next setion, we will see that the transition is due to high orrelations between neurons with a relatively large ommon input. The hallenge is to onstrut a network in whih w is large enough to allow stable wave formation yet is not too large to destabilize the asynhronous state. 4 A pair of neurons - -

12 As hinted above, we suspet that in the ase of large w, the embedded hains stimulate the emergene of orrelated spike trains. To see this, onsider a pool in a hain. The onverging onnetions from the pre-pool to all the neurons in the pool indue an input that is ommon to all the neurons in that pool, possibly leading to orrelations among these neurons spike trains. This surplus of orrelations underlies the instability of the asynhronous state in the full network. We start by studying the effet of ommon inputs on a pair of neurons that belong to the same pool of an embedded hain. n partiular, we are interested in the orrelation oeffiient between the spike trains of a pair of neurons in a pool, ρ out. n Figure 3, we desribe shematially a pair of neurons and their inputs. Being part of the same pool, they share a ommon input of at least w neurons from the previous pool ( ) and some other ommon input due to random onnetivity ( and ). They also reeive exitatory () and inhibitory () inputs that are independent for the two neurons. For simpliity, we assume that all inputs are unorrelated, exept for the w ommon input ( ) that have a pairwise orrelation oeffiient ρ in. - -

13 Figure 3: Two neurons of the same pool reeive ommon input; the total input to eah neuron is divided into five unorrelated sub-fields: independent inhibitory and exitatory ( and ), ommon inhibitory and exitatory ( and ) and orrelated ommon exitatory ( ) from the previous pool. Solid lines represent exitatory inputs, and dashed lines inhibitory inputs. xpressions in parentheses denote the number of synapses that establish eah type of input. The total number of exitatory inputs is K, and the total number of inhibitory inputs that eah neuron reeives is K. Both neurons reeive, on average, ommon input from u exitatory and u inhibitory neurons. The pairwise orrelation assumed among inputs in, ρ in, affets the orrelation between the pair of output neurons, ρ out. We will define ρ h to be the orrelation oeffiient between the two membrane potentials. The effet of the ommon input, w, on the orrelation between two neurons, ρ out, an be studied in two steps: - 3 -

14 . The effet of the ommon input on the orrelation of the two membrane potentials: ρ h (w, K, ρ in ).. The effet of the orrelated membrane potentials on the orrelation of the neurons' output: ρ out (ρ h ). We start with a simplified neuronal model, in whih a natural saling of w K emerges, and a transition in the level of orrelations is found. We then proeed to an AF model, where we find similar qualitative results. 4. A simple model n Appendix A, we define a semi-linear model, the simple model, leading to an analyti expression of ρ h (w, K, ρ in ): in h in ( g ) εk w ρ (3) ρ ( ρ ) = in ( g ) K w ρ Defining q ( g ), and dividing the nominator and denominator by K, yields: (4) = ( q ) in ε r ρ in q r ρ h ρ where r w K, is the saling variable of this system. Note that ρ h is not a funtion of the input or output rates in this model. Also note that for r=0 and ε>0, ρ h is always positive. That means that even a pair of neurons with ommon input that is only due to the random onnetivity tends to orrelate. n a network where a pattern of onnetivity as in Figure 3 is abundant, ρ out of a pair of neurons is likely to be ρ in for another pair of neurons. An example of suh a network is one with hains hardwired into its onnetivity matrix

15 n suh ases it is relevant to seek a fixed-point of pairwise orrelations, ρ *, that obeys: ρ out = ρ in =ρ *. As argued in Appendix A, under the onstraint of q. () ρ out = ρ h, thus it is straightforward to look for suh a fixed-point. n Figure 4a, the ρ out (ρ in ) is plotted for several values of r. We note that ρ out (ρ in ) rosses the diagonal with slope less than exatly one, for all values of r, resulting with a globally stable fixed-point: ρ *. We alulate ρ * from q. (4) by letting all ρ variables to be equal to ρ * and taking the positive solution: (5) * r ρ = q ( q r ) 4( q ) r εr Two important observations follow from this equation. One is that the result is independent of the firing rate, whih we assume is an artifat of the simplified model. The other is that the behavior of this equation is a funtion of the ratio r, and not of K and w separately. This implies that saling w by K is the relevant saling when onsidering orrelation oeffiients in this model. Figure 4b displays ρ * as a funtion of the saling variable r. The urve displays a transition from low to high orrelations near a ritial value of r, r = q. Stritly speaking, the transition beomes sharp only in the limit ε 0. As an be observed in Figure 4b, in this limit, the stable fixed point is zero for r smaller than r, whereas above r it diverges rapidly

16 (a) ρ out r=0 r=3 r= r= ρ in (b) Fixed-point orrelation ρ * (r) High orrelation branh Low orrelation branh r Figure 4: Transition in the simple model. (a) ρ out as a funtion of ρ in for different values of r:,, 3, and 0. The urves ross the diagonal (dashed line) exatly one, reating a globally stable fixed-point. (b) The steady state orrelation (solid line, ε=0.) exhibits a transition from low to high orrelation near the ritial point, r (dashed line). n the limit ε 0, the transition beomes sharp (dash-dotted line). Parameters: K=0,000, g=. 4. An ntegrate and Fire model When onsidering transmission of a highly orrelated signal in an asynhronously ative population as we do in this study, two main variables should be examined: neuronal firing rates and orrelations between neurons. Clearly, both these variables depend on eah other. Several studies have dealt with these variables: Tetzlaff et al (00) foused on the evolution of output rates as a funtion of input rates along a hain with noisy input, but disregarded the issue of orrelation. An integrative study, however, is urrently being onduted by Tetzlaff et al (00). Salinas and Sejnowski (000) studied the firing rates and their variability as a funtion of input orrelation. The orrelation in the input was a result of a ommon input or ommon osillation in the input rates. Analytial results based on a random walk model with drift gave results that were qualitatively similar to a ondutanebased AF model. n addition, in (Feng & Brown, 000), the authors disussed the - 6 -

17 impat of temporally orrelated input on the output statistis of an AF neuron. Stroeve and Gielen (00) onsidered the orrelation between a pair of neurons as a funtion of a orrelated ommon input. They follow a similar logi to ours and ompute ρ out (ρ in ) for ondutane-based AF model; the resulting ρ * was not derived. Here, we study the evolution of the output orrelation as a funtion of the spatially orrelated input in an AF model. The AF model is less amenable to analyti studies than the simple model. Hene we turn to simulations in order to ompute ρ out (r, ρ in, v in ). Note that we added the input firing rate v in as an additional parameter. This is due to the fat that input rates affet orrelations of AF neurons (Salinas & Sejnowski, 000; Stroeve & Gielen, 00; Tetzlaff et al., 00). Figure 5 a shows ρ out (ρ in ) for different w and for fixed values of K and v in. n ontrast to the simple model, ρ out an ross the diagonal more than one. We look for the fixed-point orrelations in the dynami steady state of the system. Following the proedure used for the simple model, we extrated the intersetion points of the urve with the diagonal ρ out (ρ * )= ρ *. We also alulated the stability of these fixed-points, by measuring the slope at the point of intersetion. The results are dρ depited in Figure 5b; if * < in dρ the fixed-point is stable (thik line), otherwise it is unstable (thin-dotted line). (a) (b) Stable Unstable 0.6 ρ out ρ in ρ * r - 7 -

18 Figure 5: Transition in AF neurons. (a) ρ out as a funtion of ρ in for different values of r (indiated above eah urve). The rossings of the sigmoid urves and the diagonal (dashed line) yield the fixed-point orrelations. (b) Fixed-point orrelations as a funtion of r. Stable fixed points are indiated by thik line and the unstable ones by thin-dotted line. Note that ρ * =0 is stable for any r in the range. Parameters: K=9,000, v in =.5*v thre. Here, we define r to be the point where the saddle node bifuration appears. Note that the details of the results of Figure 5 depend on the way orrelations were introdued. Following the Poisson mother proess, as explained in Kuhn et al (00), we introdued orrelations well beyond the seond order. Had we limited ourselves, instead, to urrents with seond order orrelations only, suh as h and h of the appendix, the value of r would hange onsiderably. Unlike the simple model, we note that for small r. ρ in, ρ out ~0. This insensitivity of the output orrelation reflets the fat that the neurons' input is not ompletely balaned. Had the neurons' input been perfetly balaned, ρ out would have been more sensitive to small input orrelation. However, we hose the same neurons parameters as in our simulations of the full network, so as to better math the full network simulation. For r> r, the system is bi-stable. This is different from the simple model in Figure 4b that had one global stable fixed-point for any value of r. Note, however, that although this two-neuron system is bi-stable, this does not imply that a network of AF neurons will be bi-stable. ven if the network settles first in the lower (zero orrelation) branh, transient spontaneous orrelations may evolve, pushing the network into the basin of attration of the upper branh. n fat, our network simulations desribed in the next setion display either a low-orrelation A mode, or a high-orrelation synhronous mode of ativity, depending on the parameters of the network. Moreover, we show in the next setion that the ritial behavior of the whole network displays saling in the same variable r as predited by the simple model

19 5 The full network The main lesson from our study of a pair of neurons is the existene of a rapid transition between low orrelations to high orrelations as funtion of the saling variable r = w K. This type of behavior is also refleted in the simulations of AF networks. n our simulations we use K=N/0. Running networks with different values of K (and N) we searh for the transitions, as funtion of w, of the networks from their A mode to the global osillatory mode. One possible measure is the oeffiient of variane (CV) of population rate defined as the standard deviation of population rate divided by the mean of population rate, shown in Figure 6. Plotting the results as funtion of the saling variable r we see that the different urves oinide; thus this is a valid saling relation for our problem. A sharp transition is observed around r =.5 in these networks. CV of the population rate r (= w/sqrt(k)) Figure 6: Sale invariane in the full network. The synhrony measure as a funtion of r, for different network sizes. Digits inside the figure denote the size of K in thousands. w range from - 9 -

20 50 to 500 in steps of 50. Population rate is defined as in Fi gure. Mean exitatory firing rates for r~ is 3Hz (std.dev. aross the different population sizes 5). Mean exitatory firing rates for r~5 is 4Hz (std.dev. 7). One we have gained some knowledge about the ritial w, w r K, above whih strong orrelations are inevitable, we an inquire whether w is large enough to enable a stable transmission of waves. Diesmann et al (999) pointed out that as w dereases, the basin of attration of the stable pulse pakets shrinks. Stated otherwise, there is a minimal w, w min, below whih waves dissolve. Our hallenge is meeting the ontraditing requirements of small w, to avoid osillations in the BN, and large w, to enable stable waves. n short, we look for the range (6) w min < w < w where w min depends on θ vreset J. We onlude that K has to be large in order to find a range of w satisfying (6). For our AF model parameters, we find w min > 30 and r.7. We hoose r=.5, 30 therefore, > ( ) K, or simply K=9000. Sine in our arhiteture K = εn, where.5 ε = 0., we have to simulate N =90,000 exitatory neurons and N =,500 inhibitory neurons before we are able to observe a synfire wave in a balaned-network. (a) 50 Raster Plot. v ext = w=00 (b) 50 Raster Plot. v ext = w=50 00 Neuron number Neuron number Time [ms] Time [ms] - 0 -

21 () Raster Plot. v ext = w= Neuron number Time [ms] Figure 7: Three panels of raster plots. The neurons in the y-axis are ordered aording to their partiipation in the pools. Only the first several pools are shown, where in eah pool every 7 th neuron is presented. (a) Asynhronous ativity. The wave ignited at t=600 dissolves (w=00<w min ) (b) Asynhronous ativity. The wave ignited at t=600ms is stable. First 30 pools are shown. (w min < w=50 < w ). () Synhronous ativity, no meaningful wave obtained (w<w=300). Parameters: N = 90,000. Mean exitatory/inhibitory firing-rates of panels a, b and are 8/8, 4/3 and 6/4 Hz respetively. Figure 7 shows results of the simulations that substantiate the theoretial analysis. A wave is ignited at t=600ms. For (a) w = 00 < w min, it dissolves after ativating 4 pools. n (b) w min < w=50 < w, the wave propagates suessfully, ativating 50 pools (only the first 30 pools are shown). t should be noted, however, that due to the length of the hain, all waves ignitions lead to an osillatory ativity that eventually prevent the wave from reahing the end of the hain. For () w=300 > w, global osillations are obtained, drowning the synfire wave. - -

22 6 Summary Many papers (Abeles, 99; Diesmann et al., 999; Hertz, 999; Tetzlaff et al., 00) disuss properties of a single hain with stationary noisy inputs. n this work, we take the disussion one step further by embedding the hain in a balaned-network of AF neurons. By simulating the entire network, we take into aount the mutual influene of the hain on the network and vie versa. By insisting on working with a full network we disovered that the formation of synfire waves in a balaned network poses quite a hallenge. n order to obey all onstraints of q. (6) we need a network of N=90,000 exitatory neurons or more. n other words, we onlude that large networks are needed in order to reate the right onditions. Changing the details of the neuronal model ould possibly redue this high number of neurons. Reduing w min to 00, (as, e.g., is the ase in Diesmann et al (999)) uts the lower bound of K to near 300, rather than 9,000 as in our ase. As for the upper bound, it has been shown (Brunel and Hakim 999, Brunel 000) that the loation of the transition from the asynhronous state to the synhronous state depends on the harateristis of synapti proessing (e.g. time onstant, heterogeneity of time onstant, et.). Moreover, in reent simulations we have seen that adding inhibitory neurons to synfire pools may hange the onditions of our network. Mehring et al. (00) also studied a synfire hain embedded in a balaned network of loally onneted AF neurons. They investigated the propagation of synhronous ativity for different spatial arrangements of a hain but they did not analyze the instability of the A state, as only short hains with 0 pools were onsidered. - -

23 Tetzlaff et al. (00, 00) showed that for a hain with external Poissonian input the transition from the asynhronous to synhronous regimes is well determined by a pure rate model. This apparent ontradition may be a result of the small r used by Tezlaff et al. (00). For small r values orrelation does not play a signifiant role, but rates apparently do. As mentioned before, firing rates and orrelations affet eah other in an intriate manner. Therefore, we have put emphasis on verifying that the qualitative saling behavior of the network, as depited in Figure 6, holds for a range of external firing rates. There is always a ritial r, but its value varies slightly as a funtion of the external rates, as an be expeted from the AF model. Previous studies within other models (Bienenstok, 995; Hertz, 999) have shown a ritial dependene on the number of pools, a apaity limit analogous to that of stored memories in a feedbak neural network. Here, by simulating a network of N =90,000 and w=60, we have found the apaity to be on the order of 000 pools. mbedding a higher number of pools leads to spontaneous global osillations. We onlude that all our previous results hold only below the apaity limit. f one exeeds the apaity limit, global osillations will appear, regardless of the value of r. There is urrent ontroversy between those who laim that neurons ode information via their firing rate (Shadlen & Newsome, 994), and those who believe that the information is onveyed in the exat timing of spikes (Softky & Koh, 993). This debate engendered the onepts of 'balaned inputs' (Shadlen & Newsome, 994) and 'balaned networks' (van Vreeswijk & Sompolinsky, 998). n (van Vreeswijk & Sompolinsky, 998), it was shown that BN has properties useful for rate-oding - 3 -

24 performane. Here, we demonstrate that BN are apable of temporal oding as well. Furthermore, both odes an be applied simultaneously. Thus, the network may multiplex rate and temporal odes. Aknowledgements This work was supported in part by grants from GF and BSF and Boehringer ngelheim Fonds. Referenes Abeles, M. (99). Cortionis: Cambridge University Press. Bienenstok,. (995). A model of the neoortex. Network: Computation in Neural Systems, 6, Brunel, N. (000). Dynamis of sparsely onneted networks of exitatory and inhibitory spiking neurons. J Comput Neurosi, 8(3), Brunel, N., & Hakim, V. (999), Fast global osillations in networks of integrate-andfire neurons with low firing rates, Neural Computation,, 6-67 Diesmann, M., Gewaltig, M. O., & Aertsen, A. (995). SYNOD: An environment for neural systems simulations. Rehovot, srael: The Weizmann nstitute of Siene. Diesmann, M., Gewaltig, M. O., & Aertsen, A. (999). Stable propagation of synhronous spiking in ortial neural networks. Nature, 40(676), Feng, J., & Brown, D. (000). mpat of orrelated inputs on the output of the integrate- and-fire model. Neural Comput, (3), Gerstein, G. L., & Mandelbrot, B. (964). Random walk models for the spike ativity of a single neuron. Biophysial Journal, 4,

25 Hayun, G. (00). Modeling ompositionality in biologial neural networks by dynami binding of synfire hains. Unpublished PhD dissertation, Hebrew University, Jerusalem. Hertz, J. A. (999). Modeling synfire networks. n O. Parodi (d.), NURONAL NFORMATON PROCSSNG - From Biologial Data to Modelling and Appliation. Kuhn, A., Rotter, S., & Aertsen, A. (00) Correlated input spike trains and their effets on the response of the leaky integrate-and-fire neuron. Neuroomputing, in press. Mehring, C., Hehl, U., Kubo, M., Diesmann, M. & Aertsen, A. (00). Ativity dynamis and propagation of synhronous spiking in loally onneted random networks. Biol. Cyber., in press. Salinas,., & Sejnowski, T. J. (000). mpat of orrelated synapti input on output firing rate and variability in simple neuronal models. J Neurosi, 0(6), Shadlen, M. N., & Newsome, W. T. (994). Noise, neural odes and ortial organization. Curr Opin Neurobiol, 4(4), Softky, W. R., & Koh, C. (993). The highly irregular firing of ortial ells is inonsistent with temporal integration of random PSPs. J Neurosi, 3(), Stroeve, S., & Gielen, S. (00). Correlation between unoupled ondutane-based integrate-and-fire neurons due to ommon and synhronous presynapti firing. Neural Comput, 3(9),

26 Tetzlaff, T., Bushermohle, M., Geisel, T., & Diesmann, M. (00). The spread of rate and orrelation in stationary ortial networks. Paper presented at the Proeedings of the th Computational Neurosiene Meeting, Monterey, CA. Tetzlaff, T., Geisel, T., & Diesmann, M. (00). The ground state of ortial feedforward networks. Neuroomputing, 44-46, Tukwell, H. C. (988). ntrodution to theoretial neurobiology. Cambridge: Cambridge University Press. van Vreeswijk, C., & Sompolinsky, H. (998). Chaoti balaned state in a model of ortial iruits. Neural Comput, 0(6), Appendix A: The simple model We define a semi-linear neuronal model: r ( t = h t) (7) i ) [ i ( ] where the funtion [x] is x if x>0, and zero otherwise. h i (t) is the loal field, defined as a sum of K exitatory and K inhibitory synapti inputs: K (8) h () t = J s () t gj s () t i l= l K m= m Here, J is the fixed synapti weight, s l (t) is the pre-synapti input at time t, and g ontrols the exess (or shortage) of exitation in the loal field. The variables s l (t)>0, l=..3k represent the instantaneous firing rate of the presynapti neuron. For larity, we will disard the time notation of the s l s

27 A semi-linear model is a rude ariature of an AF neuron. Nevertheless, it enables the use of analytial tools, allowing for a better understanding of the phenomenon under inspetion. ndeed, we find that a omplex network of AF neurons exhibits behavior that is predited by the urrent semi-linear model. Correlation between two fields Given a pair of neurons in a pool, eah having K external, as well as K loal, exitatory synapses and K inhibitory synapses, as depited in Figure 3. As a result of random onnetivity, some of these synapses share a soure that is ommon to both neurons. The mean number of these inputs is taken to be u-w for the exitatory synapses, and u for the inhibitory synapses. Finally, there are w inputs that are ommon to both neurons, whih ome from the previous pool. We divide the input field of a neuron into five sub-fields (See Figure 3): K-u synapses of external and loal independent-exitatory input () K-u synapses of independent-inhibitory input () u-w synapses of ommon-exitatory input ( ) u synapses of ommon-inhibitory input ( ) w synapses of orrelated-ommon-exitatory ( ) The and sub-fields represent ommon input due to the random onnetivity. The sub-field represents input that omes from the previous pool in the hain; thus we also onsider it as orrelated input. To simplify the arhiteture without impairing its properties, we unite both the external and the loal independent exitatory input into one sub-field:. The and - 7 -

28 sub-fields are different for the two output neurons, we denote the sub-field to the first and seond neurons by and respetively. Similarly, and are the independent inhibitory sub-fields for the first and seond neuron, respetively. We assume that the s i s are orrelated stohasti variables that are haraterized by their first two moments: s = ν i var ( s ) = σ i s s i s j = in ρ σ s ν, where ρ in is the orrelation oeffiient. Here, the angular brakets stand for an average over time or over different realizations of the s i proess. We assume no orrelations among inputs that do not ome from the previous pool, i.e. in ρ is 0 if i, j are not members of the same pool. This assumption may not be valid in a full network. We alulate the membrane potential that is the result of inoming post-synapti potentials in eah of the five sub-fields. For example, the sub-field is defined as = w l J s l, where J is the (single) synapti weight. The statistis of is as follows: µ Jwν = w = J = sl l J w s J in ( ) s s = J [ w( σ ν ) w( w )( ρ σ ν )] w w i j s s - 8 -

29 We assume w and K to be large; hene we an apply the entral limit theorem, leading to: ( µ, σ ) ~ N, with µ Jwν = and = Jσ s w( ρin ( w ) ) σ. A derivation similar to that of sub-field leads to normal approximation of the other sub-fields: ( σ ) ~, N µ, with µ J ( K u)ν = and σ J K u = s. ( µ, σ ) σ ~ N, with µ = J ( u w)ν and σ = σ u w. ( σ ) J s ~ N µ,, with µ gj ( K u)ν = and gj K u = σ s ( µ, σ ) σ ~ N, with µ = gjuν and σ = gjσ s u Note that we assumed here that the inputs that do not ome from the previous pool are unorrelated, i.e. < s i j >=ν. Also note that µ = µ, µ = µ and that σ = σ, s σ = σ Let h, the input field of the first neuron, be the sum of the five unorrelated, normally distributed sub fields of the first neuron: h =. The statistis of h are as follows: µ h µ h = = µ µ µ µ < ( ) >= σ σ σ σ σ h >=< µ h in Thus, σ σ σ σ σ σ = J σ [( g ) K w( w ) ρ ] h =. s Similarly, we define h =, where and are the respetive sub-fields of the seond neuron

30 Whereas h h = and h h =, the ovariane of the two fields depends on their ommon input: ( )( ) ) ( h h µ µ µ µ µ σ σ σ = = The orrelation oeffiient is: h h h h h σ σ σ σ σ σ σ σ σ µ ρ = Finally, we get: (9) ( ) ( ) ( ) ( ) in in h w w K g w w u g ρ ρ ρ = The expeted number of ommon inputs to pair of neurons in a pool, whih is not from the pre-pool, is ( ) ( ) N w K w K w N w K w K = ε. Taking the leading order in K, we set (0) u= εk Substituting q. (0) in q. (9) and using the approximation ( ) w w w, we get q. (3). Correlation between two neurons Clearly, the orrelation between two neurons (ρ out ) is a funtion of the orrelation between their fields (ρ h ). n general, however, this funtion is highly dependent on the neuronal model in use

31 t is hard to derive an analytial expression of this dependeny for non-linear models. Therefore, we assume that the mean membrane potential, or the mean of the loal field, is rarely below zero: () µ h > σ h. Thus P [ h () t < 0 ] < i, whih allows us to approximate q. (7) with: () r i (t) h i (t) This approximation allows us to equate the orrelation between the neurons with the orrelation between the fields, leading to: (3) ρ out (ρ h ) = ρ h - 3 -