Information Bottleneck Optimization and Independent Component Extraction with Spiking Neurons

Transcription

1 Informaton ottlenec Optmzaton and Independent Component Extracton wth Spng Neurons Stefan Klampfl, Robert Legensten, Wolfgang Maass Insttute for Theoretcal Computer Scence Graz Unversty of Technology A-8 Graz, Austra Abstract The extracton of statstcally ndependent components from hgh-dmensonal mult-sensory nput streams s assumed to be an essental component of sensory processng n the bran. Such ndependent component analyss (or blnd source separaton) could provde a less redundant representaton of nformaton about the external world. Another powerful processng strategy s to extract preferentally those components from hgh-dmensonal nput streams that are related to other nformaton sources, such as nternal predctons or proproceptve feedbac. Ths strategy allows the optmzaton of nternal representaton accordng to the nformaton bottlenec method. However, concrete learnng rules that mplement these general unsupervsed learnng prncples for spng neurons are stll mssng. We show how both nformaton bottlenec optmzaton and the extracton of ndependent components can n prncple be mplemented wth stochastcally spng neurons wth refractorness. The new learnng rule that acheves ths s derved from abstract nformaton optmzaton prncples. Introducton The Informaton ottlenec (I) approach and ndependent component analyss (ICA) have both attracted substantal nterest as general prncples for unsupervsed learnng [, 2]. A hope has been, that they mght also help us to understand strateges for unsupervsed learnng n bologcal systems. However t has turned out to be qute dffcult to establsh lns between nown learnng algorthms that have been derved from these general prncples, and learnng rules that could possbly be mplemented by synaptc plastcty of a spng neuron. Fortunately, n a smpler context a drect ln between an abstract nformaton theoretc optmzaton goal and a rule for synaptc plastcty has recently been establshed [3]. The resultng rule for the change of synaptc weghts n [3] maxmzes the mutual nformaton between pre- and postsynaptc spe trans, under the constrant that the postsynaptc frng rate stays close to some target frng rate. We show n ths artcle, that ths approach can be extended to stuatons where smultaneously the mutual nformaton between the postsynaptc spe tran of the neuron and other sgnals (such as for example the spe trans of other neurons) has to be mnmzed (Fgure ). Ths opens the door to the exploraton of learnng rules for nformaton bottlenec analyss and ndependent component extracton wth spng neurons that would be optmal from a theoretcal perspectve. We revew n secton 2 the neuron model and learnng rule from [3]. We show n secton 3 how ths learnng rule can be extended so that t not only maxmzes mutual nformaton wth some gven spe trans and eeps the output frng rate wthn a desred range, but smultaneously mnmzes mutual nformaton wth other spe trans, or other tme-varyng sgnals. Applcatons to nfor-

2 A Fgure : Dfferent learnng stuatons analyzed n ths artcle. A In an nformaton bottlenec tas the learnng neuron (neuron ) wants to maxmze the mutual nformaton between ts output Y K and the actvty of one or several target neurons Y2 K, Y3 K,... (whch can be functons of the nputs X K and/or other external sgnals), whle at the same tme eepng the mutual nformaton between the nputs X K and the output Y K as low as possble (and ts frng rate wthn a desred range). Thus the neuron should learn to extract from ts hgh-dmensonal nput those aspects that are related to these target sgnals. Ths setup s dscussed n sectons 3 and 4. Two neurons recevng the same nputs X K from a common set of presynaptc neurons both learn to maxmze nformaton transmsson, and smultaneously to eep ther outputs Y K and Y2 K statstcally ndependent. Such extracton of ndependent components from the nput s descrbed n secton 5. maton bottlenec tass are dscussed n secton 4. In secton 5 we show that a modfcaton of ths learnng rule allows a spng neuron to extract nformaton from ts nput spe trans that s ndependent from the component extracted by another neuron. 2 Neuron model and a basc learnng rule We use the model from [3], whch s a stochastcally spng neuron model wth refractorness, where the probablty of frng n each tme step depends on the current membrane potental and the tme snce the last output spe. It s convenent to formulate the model n dscrete tme wth step sze. The total membrane potental of a neuron n tme step t = s gven by N u (t ) = u r + w j ɛ(t t n )x n j, () j= n= where u r = 7mV s the restng potental and w j s the weght of synapse j (j =,..., N). An nput spe tran at synapse j up to the -th tme step s descrbed by a sequence X j = (x j, x2 j,..., x j ) of zeros (no spe) and ones (spe); each presynaptc spe at tme tn (x n j = ) evoes a postsynaptc potental (PSP) wth exponentally decayng tme course ɛ(t t n ) wth tme constant τ m = ms. The probablty ρ of frng of neuron n each tme step t s gven by ρ = exp[ g(u (t )R (t )] g(u (t ))R (t ), (2) where g(u) = r log{ + exp[(u u )/ u]} s a smooth ncreasng functon of the membrane potental u (u = 65mV, u = 2mV, r = Hz). The approxmaton s vald for suffcently small (ρ (t ˆt ). The refractory varable R (t) = τ abs ) 2 Θ(t ˆt τrefr 2 +(t ˆt τ abs ) 2 τ abs ) assumes values n [, ] and depends on the last frng tme ˆt of neuron (absolute refractory perod τ abs = 3ms, relatve refractory tme τ refr = ms). The Heavsde step functon Θ taes a value of for non-negatve arguments and otherwse. Ths model from [3] s a specal case of the spe-response model, and wth a refractory varable R(t) that depends only on the tme snce the last postsynaptc event t has renewal propertes [4].

3 The output of neuron at the -th tme step s denoted by a varable y that assumes the value f a postsynaptc spe occurred and otherwse. A specfc spe tran up to the -th tme step s wrtten as Y = (y, y2,..., y ). The nformaton transmsson between an ensemble of nput spe trans X K and the output spe tran Y K can be quantfed by the mutual nformaton [5] I(X K ; Y K ) = X K,Y K P (X K, Y K ) log P (Y K X K ) P (Y K. (3) The dea n [3] was to maxmze the quantty I(X K ; Y K) γd KL(P (Y K D KL (P (Y K ) P (Y K )) = Y P (Y K K ) log(p (Y K )/ P (Y K ) ) P (Y K )), where )) denotes the Kullbac-Lebler dvergence [5], mposng the addtonal constrant that the frng statstcs P (Y ) of the neuron should stay as close as possble to a target dstrbuton P (Y ). Ths dstrbuton was chosen to be that of a constant target frng rate g accountng for homeostatc processes. An onlne learnng-rule performng gradent ascent on ths quantty was derved for the weghts w j of neuron, wth w j denotng the weght change durng the -th tme step: w j = αc j (γ), (4) whch conssts of the correlaton term Cj and the postsynaptc term [3]. The term Cj measures concdences between postsynaptc spes at neuron and PSPs generated by presynaptc acton potentals arrvng at synapse j, ( Cj = C j ) + ɛ(t t n )x n g (u (t )) [ j y τ C g(u (t )) ρ ], (5) n= n an exponental tme wndow wth tme constant τ C = s and g (u (t )) denotng the dervatve of g wth respect to u. The term [ (γ) = y log g(u (t ( ) γ ] )) g ḡ (t ) ḡ (t ) ( y )R (t ) [ g(u (t )) ( + γ)ḡ (t ) + γ g ], (6) compares the current frng rate g(u (t )) wth ts average frng rate 2 ḡ (t ), and smultaneously the runnng average ḡ (t ) wth the constant target rate g. The argument ndcates that ths term also depends on the optmzaton parameter γ. 3 Learnng rule for mult-neuron nteractons We extend the learnng rule presented n the prevous secton to a more complex scenaro, where the mutual nformaton between the output spe tran Y K of the learnng neuron (neuron ) and some target spe trans Yl K (l > ) has to be maxmzed, whle smultaneously mnmzng the mutual nformaton between the nputs X K and the output Y K. Obvously, ths s the generc I scenaro appled to spng neurons (see Fgure A). A learnng rule for extractng ndependent components wth spng neurons (see secton 5) can be derved n a smlar manner. For smplcty, we consder the case of an I optmzaton for only one target spe tran Y K 2, and derve an update rule for the synaptc weghts w j of neuron. The quantty to maxmze s therefore L = I(X K ; Y K ) + βi(y K ; Y K 2 ) γd KL (P (Y K ) P (Y K )), (7) where β and γ are optmzaton constants. To maxmze ths objectve functon, we derve the weght change w j durng the -th tme step by gradent ascent on (7), assumng that the weghts w j can change between some bounds w j w max (we assume w max = throughout ths paper). We use boldface letters (X ) to dstngush random varables from specfc realzatons (X ). 2 The rate ḡ (t ) = g(u (t )) X Y denotes an expectaton of the frng rate over the nput dstrbuton gven the postsynaptc hstory and s mplemented as a runnng average wth an exponental tme wndow (wth a tme constant of ms).

4 Note that all three terms of (7) mplctly depend on w j because the output dstrbuton P (Y K ) changes f we modfy the weghts w j. Snce the frst and the last term of (7) have already been consdered (up to the sgn) n [3], we wll concentrate here on the mddle term L 2 := βi(y K ; Y2 K ) and denote the contrbuton of the gradent of L 2 to the total weght change wj n the -th tme step by w j. In order to get an expresson for the weght change n a specfc tme step t we wrte the probabltes P (Y K ) and P (Y K, Y2 K ) occurrng n (7) as products over ndvdual tme bns,.e., P (Y K ) = K = P (y Y ) and P (Y K, Y2 K ) = K = P (y, y2 Y, Y2 ), accordng to the chan rule of nformaton theory [5]. Consequently, we rewrte L 2 as a sum over the contrbutons of the ndvdual tme bns, L 2 = K = L 2, wth L 2 = β log P (y, y2 Y, Y2 ). (8) P (y Y )P (y2 Y2 ) X,Y,Y 2 The weght change w j s then proportonal to the gradent of ths expresson wth respect to the weghts w j,.e., w j = α( L 2/ w j ), wth some learnng rate α >. The evaluaton of the gradent yelds w j = α Cj βf 2 wth a correlaton term C X,Y,Y j as n (5) and a term 2 F2 = y y2 ḡ 2 (t ) [ḡ2 log ḡ (t )ḡ 2 (t ) y ( y2 )R 2 (t (t ] ) ) ḡ (t ) ḡ 2(t ) [ḡ2 ( y )y2 R (t (t ] ) ) ḡ 2 (t ) ḡ (t ) + + ( y )( y 2 )R (t )R 2 (t )() 2 [ ḡ 2 (t ) ḡ (t )ḡ 2 (t ) ]. (9) Here, ḡ (t ) = g(u (t )) X Y denotes the average frng rate of neuron and ḡ 2 (t ) = g(u (t ))g(u 2 (t )) X Y,Y denotes the average product of frng rates of both neurons. oth 2 quanttes are mplemented onlne as runnng exponental averages wth a tme constant of s. Under the assumpton of a small learnng rate α we can approxmate the expectaton X,Y,Y 2 by averagng over a sngle long tral. Consderng now all three terms n (7) we fnally arrve at an onlne rule for maxmzng (7) w j = αc j [ ( γ) β2 ]. () whch conssts of a term Cj senstve to correlatons between the output of the neuron and ts presynaptc nput at synapse j ( correlaton term ) and terms and 2 that characterze the postsynaptc state of the neuron ( postsynaptc terms ). Note that the argument of s dfferent from (4) because some of the terms of the objectve functon (7) have a dfferent sgn. In order to compensate the effect of a small, the constant β has to be large enough for the term 2 to have an nfluence on the weght change. The factors Cj and were descrbed n the prevous secton. In addton, our learnng rule contans an extra term 2 = F2/() 2 that s senstve to the statstcal dependence between the output spe tran of the neuron and the target. It s gven by 2 = y y2 () 2 log ḡ 2 (t ) ḡ (t )ḡ 2 (t ) y [ḡ2 (t ] ) ḡ (t ) ḡ 2(t ) y 2 ( y )R (t ) ( y 2 )R 2 (t ) ] [ḡ2 (t ) ḡ 2 (t ) ḡ (t ) + ( y )( y 2 )R (t )R 2 (t ) [ ḡ 2 (t ) ḡ (t )ḡ 2 (t ) ]. () Ths term bascally compares the average product of frng rates ḡ 2 (whch corresponds to the jont probablty of spng) wth the product of average frng rates ḡ ḡ 2 (representng the probablty of ndependent spng). In ths way, t measures the momentary mutual nformaton between the output of the neuron and the target spe tran.

5 For a smplfed neuron model wthout refractorness (R(t) = ), the update rule (4) resembles the CM-rule [6] as shown n [3]. Wth the objectve functon (7) to maxmze, we expect an ant- Hebban CM rule wth another term accountng for statstcal dependences between Y K and Y2 K. Snce there s no refractorness, the postsynaptc rate ν (t ) s gven drectly by the current value of g(u(t )), and the update rule () reduces to the rate model 3 w j { [ ( ) = αν pre, ν j f(ν ) log ν γ ] ν g ( [ ] [ ν β ν2 log 2 ν ν ν 2 ν 2 2 ν ν 2 where the presynaptc rate at synapse j at tme t s denoted by ν pre, j ])}, (2) = a n= ɛ(t t n )x n j wth a n unts (Vs). The values ν, ν 2, and ν 2 are runnng averages of the output rate ν, the rate of the target sgnal ν 2 and of the product of these values, ν ν 2, respectvely. The functon f(ν ) = g (g (ν ))/a s proportonal to the dervatve of g wth respect to u, evaluated at the current membrane potental. The frst term n the curly bracets accounts for the homeostatc process (smlar to the CM rule, see [3]), whereas the second term renforces dependences between Y K and Y K 2. Note that ths term s zero f the rates of the two neurons are ndependent. It s nterestng to note that f we rewrte the smplfed rate-based learnng rule (2) n the followng way, w j = αν pre, j Φ(ν, ν2 ), (3) we can vew t as an extenson of the classcal enenstoc-cooper-munro (CM) rule [6] wth a two-dmensonal synaptc modfcaton functon Φ(ν, ν2 ). Here, values of Φ > produce LTD whereas values of Φ < produce LTP. These regmes are separated by a sldng threshold, however, n contrast to the orgnal CM rule ths threshold does not only depend on the runnng average of the postsynaptc rate ν, but also on the current values of ν2 and ν 2. 4 Applcaton to Informaton ottlenec Optmzaton We use a setup as n Fgure A where we want to maxmze the nformaton whch the output Y K of a learnng neuron conveys about two target sgnals Y2 K and Y3 K. If the target sgnals are statstcally ndependent from each other we can optmze the mutual nformaton to each target sgnal separately. Ths leads to an update rule w j = αc j [ ( γ) β ( )], (4) where 2 and 3 are the postsynaptc terms () senstve to the statstcal dependence between the output and target sgnals and 2, respectvely. We choose g = 3Hz for the target frng rate, and we use dscrete tme wth = ms. In ths experment we demonstrate that t s possble to consder two very dfferent nds of target sgnals: one target spe tran has has a smlar rate modulaton as one part of the nput, whle the other target spe tran has a hgh spe-spe correlaton wth another part of the nput. The learnng neuron receves nput at synapses, whch are dvded nto 4 groups of 25 nputs each. The frst two nput groups consst of rate modulated Posson spe trans 4 (Fgure 2A). Spe trans from the remanng groups 3 and 4 are correlated wth a coeffcent of.5 wthn each group, however, spe trans from dfferent groups are uncorrelated. Correlated spe trans are generated by the procedure descrbed n [7]. The frst target sgnal s chosen to have the same rate modulaton as the nputs from group, except that Gaussan random nose s supermposed wth a standard devaton of 2Hz. The second target spe tran s correlated wth nputs from group 3 (wth a coeffcent of.5), but uncorrelated to nputs from group 4. Furthermore, both target sgnals are slent durng random ntervals: at each 3 In the absence of refractorness we use an alternatve gan functon g alt (u) = [/g max + /g(u)] n order to pose an upper lmt of g max = Hz on the postsynaptc frng rate.

6 A nput [Hz] nput 2 [Hz] t [ms] synapse dx evoluton of weghts C output [Hz] target [Hz] t [ms] D..5 MI/KLD of neuron E x I(output;targets) F correlaton wth targets target target Fgure 2: Performance of the spe-based learnng rule () for the I tas. A Modulaton of nput rates to nput groups and 2. Evoluton of weghts durng 6 mnutes of learnng (brght: strong synapses, w j, dar: depressed synapses, w j.) Weghts are ntalzed randomly between. and.2, α = 4, β = 2 3, γ = 5. C Output rate and rate of target sgnal durng 5 seconds after learnng. D Evoluton of the average mutual nformaton per tme bn (sold lne, left scale) between nput and output and the Kullbac-Lebler dvergence per tme bn (dashed lne, rght scale) as a functon of tme. Averages are calculated over segments of mnute. E Evoluton of the average mutual nformaton per tme bn between output and both target spe trans as a functon of tme. F Trace of the correlaton between output rate and rate of target sgnal (sold lne) and the spe-spe correlaton (dashed lne) between the output and target spe tran 2 durng learnng. Correlaton coeffcents are calculated every seconds. tme step, each target sgnal s ndependently set to wth a certan probablty ( 5 ) and remans slent for a duraton chosen from a Gaussan dstrbuton wth mean 5s and SD s (mnmum duraton s s). Hence ths experment tests whether learnng wors even f the target sgnals are not avalable all of the tme. Fgure 2 shows that strong weghts evolve for the frst and thrd group of synapses, whereas the effcaces for the remanng nputs are depressed. oth groups wth growng weghts are correlated wth one of the target sgnals, therefore the mutual nformaton between output and target spe trans ncreases. Snce spe-spe correlatons convey more nformaton than rate modulatons synaptc effcaces develop more strongly to group 3 (the group wth spe-spe correlatons). Ths results n an ntal decrease n correlaton wth the rate-modulated target to the beneft of hgher correlaton wth the second target. However, after about 3 mnutes when the weghts become stable, the correlatons as well as the mutual nformaton quanttes stay roughly constant. An applcaton of the smplfed rule (2) to the same tas s shown n Fgure 3 where t can be seen that strong weghts close to w max are developed for the rate-modulated nput. To some extent weghts grow also for the nputs wth spe-spe correlatons n order to reach the constant target frng rate g. In contrast to the spe-based rule the smplfed rule s not able to detect spe-spe correlatons between output and target spe trans. 4 The rate of the frst 25 nputs s modulated by a Gaussan whte-nose sgnal wth mean 2Hz that has been low pass fltered wth a cut-off frequency of 5Hz. Synapses 26 to 5 receve a rate that has a constant value of 2Hz, except that a burst s ntated at each tme step wth a probablty of.5. Thus there s a burst on average every 2s. The duraton of a burst s chosen from a Gaussan dstrbuton wth mean.5s and SD.2s, the mnmum duraton s chosen to be.s. Durng a burst the rate s set to 5Hz. In the smulatons we use dscrete tme wth = ms.

7 A evoluton of weghts 4 x MI/KLD of neuron 3.4 C.5 correlaton wth target synapse dx Fgure 3: Performance of the smplfed update rule (2) for the I tas. A Evoluton of weghts durng 3 mnutes of learnng (brght: strong synapses, w j, dar: depressed synapses, w j.) Weghts are ntalzed randomly between. and.2, α = 3, β = 4, γ =. Evoluton of the average mutual nformaton per tme bn (sold lne, left scale) between nput and output and the Kullbac-Lebler dvergence per tme bn (dashed lne, rght scale) as a functon of tme. Averages are calculated over segments of mnute. C Trace of the correlaton between output rate and target rate durng learnng. Correlaton coeffcents are calculated every seconds. 5 Extractng Independent Components Wth a slght modfcaton n the objectve functon (7) the learnng rule allows us to extract statstcally ndependent components from an ensemble of nput spe trans. We consder two neurons recevng the same nput at ther synapses (see Fgure ). For both neurons =, 2 we maxmze nformaton transmsson under the constrant that ther outputs stay as statstcally ndependent from each other as possble. That s, we maxmze L = I(X K ; Y K ) βi(y K ; Y2 K ) γd KL (P (Y K ) P (Y K )). (5) Snce the same terms (up to the sgn) are optmzed n (7) and (5) we can derve a gradent ascent rule for the weghts of neuron, w j, analogously to secton 3: w j = αc j [ (γ) β2 ]. (6) Fgure 4 shows the results of an experment where two neurons receve the same Posson nput wth a rate of 2Hz at ther synapses. The nput s dvded nto two groups of 4 spe trans each, such that synapses to 4 and 4 to 8 receve correlated nput wth a correlaton coeffcent of.5 wthn each group, however, any spe trans belongng to dfferent nput groups are uncorrelated. The remanng 2 synapses receve uncorrelated Posson nput. Weghts close to the maxmal effcacy w max = are developed for one of the groups of synapses that receves correlated nput (group 2 n ths case) whereas those for the other correlated group (group ) as well as those for the uncorrelated group (group 3) stay low. Neuron 2 develops strong weghts to the other correlated group of synapses (group ) whereas the effcaces of the second correlated group (group 2) reman depressed, thereby tryng to produce a statstcally ndependent output. For both neurons the mutual nformaton s maxmzed and the target output dstrbuton of a constant frng rate of 3Hz s approached well. After an ntal ncrease n the mutual nformaton and n the correlaton between the outputs, when the weghts of both neurons start to grow smultaneously, the amounts of nformaton and correlaton drop as both neurons develop strong effcaces to dfferent parts of the nput. 6 Dscusson Informaton ottlenec (I) and Independent Component Analyss (ICA) have been proposed as general prncples for unsupervsed learnng n lower cortcal areas, however, learnng rules that can mplement these prncples wth spng neurons have been mssng. In ths artcle we have derved from nformaton theoretc prncples learnng rules whch enable a stochastcally spng neuron to solve these tass. These learnng rules are optmal from the perspectve of nformaton theory, but they are not local n the sense that they use only nformaton that s avalable at a sngle

8 A weghts of neuron weghts of neuron 2 C 6 x 4 I(output ;output2) synapse dx synapse dx D.6 MI/KLD of neuron.4 E.6 MI/KLD of neuron 2.4 F.6 correlaton between outputs Fgure 4: Extractng ndependent components. A, Evoluton of weghts durng 3 mnutes of learnng for both postsynaptc neurons (red: strong synapses, w j, blue: depressed synapses, w j.) Weghts are ntalzed randomly between. and.2, α = 3, β =, γ =. C Evoluton of the average mutual nformaton per tme bn between both output spe trans as a functon of tme. D,E Evoluton of the average mutual nformaton per tme bn (sold lne, left scale) between nput and output and the Kullbac-Lebler dvergence per tme bn for both neurons (dashed lne, rght scale) as a functon of tme. Averages are calculated over segments of mnute. F Trace of the correlaton between both output spe trans durng learnng. Correlaton coeffcents are calculated every seconds. synapse wthout an auxlary networ of nterneurons or other bologcal processes. Rather, they tell us what type of nformaton would have to be deally provded by such auxlary networ, and how the synapse should change ts effcacy n order to approxmate a theoretcally optmal learnng rule. Acnowledgments We would le to than Wulfram Gerstner and Jean-Pascal Pfster for helpful dscussons. Ths paper was wrtten under partal support by the Austran Scence Fund FWF, # S92-N3 and # P7229- N4, and was also supported by PASCAL, project # IST , and FACETS, project # 5879, of the European Unon. References [] N. Tshby, F. C. Perera, and W. ale. The nformaton bottlenec method. In Proceedngs of the 37-th Annual Allerton Conference on Communcaton, Control and Computng, pages , 999. [2] A. Hyvärnen, J. Karhunen, and E. Oja. Independent Component Analyss. Wley, New Yor, 2. [3] T. Toyozum, J.-P. Pfster, K. Ahara, and W. Gerstner. Generalzed enenstoc-cooper-munro rule for spng neurons that maxmzes nformaton transmsson. Proc. Natl. Acad. Sc. USA, 2: , 25. [4] W. Gerstner and W. M. Kstler. Spng Neuron Models. Cambrdge Unversty Press, Cambrdge, 22. [5] T. M. Cover and J. A. Thomas. Elements of Informaton Theory. Wley, New Yor, 99. [6] E. L. enenstoc, L. N. Cooper, and P. W. Munro. Theory for the development of neuron selectvty: orentaton specfcty and bnocular nteracton n vsual cortex. J. Neurosc., 2():32 48, 982. [7] R. Gütg, R. Aharonov, S. Rotter, and H. Sompolnsy. Learnng nput correlatons through non-lnear temporally asymmetrc hebban plastcty. Journal of Neurosc., 23: , 23.