Hierarchical Bayesian Inference in Networks of Spiking Neurons

Transcription

1 To appear n Advances n NIPS, Vol. 17, MIT Press, 25. Herarchcal Bayesan Inference n Networks of Spkng Neurons Raesh P. N. Rao Department of Computer Scence and Engneerng Unversty of Washngton, Seattle, WA rao@cs.washngton.edu Abstract There s growng evdence from psychophyscal and neurophysologcal studes that the bran utlzes Bayesan prncples for nference and decson makng. An mportant open queston s how Bayesan nference for arbtrary graphcal models can be mplemented n networks of spkng neurons. In ths paper, we show that recurrent networks of nosy ntegrate-and-fre neurons can perform approxmate Bayesan nference for dynamc and herarchcal graphcal models. The membrane potental dynamcs of neurons s used to mplement belef propagaton n the log doman. The spkng probablty of a neuron s shown to approxmate the posteror probablty of the preferred state encoded by the neuron, gven past nputs. We llustrate the model usng two examples: (1) a moton detecton network n whch the spkng probablty of a drecton-selectve neuron becomes proportonal to the posteror probablty of moton n a preferred drecton, and (2) a two-level herarchcal network that produces attentonal effects smlar to those observed n vsual cortcal areas V2 and V4. The herarchcal model offers a new Bayesan nterpretaton of attentonal modulaton n V2 and V4. 1 Introducton A wde range of psychophyscal results have recently been successfully explaned usng Bayesan models [7, 8, 16, 19]. These models have been able to account for human responses n tasks rangng from 3D shape percepton to vsuomotor control. Smultaneously, there s accumulatng evdence from human and monkey experments that Bayesan mechansms are at work durng vsual decson makng [2, 5]. The versatlty of Bayesan models stems from ther ablty to combne pror knowledge wth sensory evdence n a rgorous manner: Bayes rule prescrbes how pror probabltes and stmulus lkelhoods should be combned, allowng the responses of subects or neural responses to be nterpreted n terms of the resultng posteror dstrbutons. An mportant queston that has only recently receved attenton s how networks of cortcal neurons can mplement algorthms for Bayesan nference. One powerful approach has been to buld on the known propertes of populaton codng models that represent nformaton usng a set of neural tunng curves or kernel functons [1, 2]. Several proposals have been made regardng how a probablty dstrbuton could be encoded usng populaton codes ([3, 18]; see [14] for an excellent revew). However, the problem of mplementng general nference algorthms for arbtrary graphcal models usng populaton codes remans unresolved (some encouragng ntal results are reported n Zemel et al., ths volume). An

2 alternate approach advocates performng Bayesan nference n the log doman such that multplcaton of probabltes s turned nto addton and dvson to subtracton, the latter operatons beng easer to mplement n standard neuron models [2, 5, 15] (see also the papers by Deneve and by Yu and Dayan n ths volume). For example, a neural mplementaton of approxmate Bayesan nference for a hdden Markov model was nvestgated n [15]. The queston of how such an approach could be generalzed to spkng neurons and arbtrary graphcal models remaned open. In ths paper, we propose a method for mplementng Bayesan belef propagaton n networks of spkng neurons. We show that recurrent networks of nosy ntegrate-and-fre neurons can perform approxmate Bayesan nference for dynamc and herarchcal graphcal models. In the model, the dynamcs of the membrane potental s used to mplement on-lne belef propagaton n the log doman [15]. A neuron s spkng probablty s shown to approxmate the posteror probablty of the preferred state encoded by the neuron, gven past nputs. We frst show that for a vsual moton detecton task, the spkng probablty of a drecton-selectve neuron becomes proportonal to the posteror probablty of moton n the neuron s preferred drecton. We then show that n a two-level network, herarchcal Bayesan nference [9] produces responses that mmc the attentonal effects seen n vsual cortcal areas V2 and V4. 2 Modelng Networks of Nosy Integrate-and-Fre Neurons 2.1 Integrate-and-Fre Model of Spkng Neurons We begn wth a recurrently-connected network of ntegrate-and-fre (IF) neurons recevng feedforward nputs denoted by the vector I. The membrane potental of neuron changes accordng to: τ dv dt = v + w I + u v (1) where τ s the membrane tme constant, I denotes the synaptc current due to nput neuron, w represents the strength of the synapse from nput to recurrent neuron, v denotes the synaptc current due to recurrent neuron, and u represents the correspondng synaptc strength. If v crosses a threshold T, the neuron fres a spke and v s reset to the potental v reset. Equaton 1 can be rewrtten n dscrete form as: v (t + 1) = v (t) + ɛ( v (t) + w I (t)) + u v (t)) (2).e. v (t + 1) = ɛ w I (t) + u v (t) (3) where ɛ s the ntegraton rate, u = 1 + ɛ(u 1) and for, u = ɛu. A more general ntegrate-and-fre model that takes nto account some of the effects of nonlnear flterng n dendrtes can be obtaned by generalzng Equaton 3 as follows: v (t + 1) = f ( w I (t) ) + g ( u v (t) ) (4) where f and g model potentally dfferent dendrtc flterng functons for feedforward and recurrent nputs. 2.2 Stochastc Spkng n Nosy IF Neurons To model the effects of background nputs and the random openngs of membrane channels, one can add a Gaussan whte nose term to the rght hand sde of Equatons 3 and 4. Ths makes the spkng of neurons n the recurrent network stochastc. Plesser and Gerstner [13] and Gerstner [4] have shown that under reasonable assumptons, the probablty of spkng

3 n such nosy neurons can be approxmated by an escape functon (or hazard functon) that depends only on the dstance between the (nose-free) membrane potental v and the threshold T. Several dfferent escape functons were studed. Of partcular nterest to the present paper s the followng exponental functon for spkng probablty suggested n [4] for nosy ntegrate-and-fre networks: P (neuron spkes at tme t) = ke (v(t) T )/c (5) where k and c are arbtrary constants. We used a model that combnes Equatons 4 and 5 to generate spkes, wth an absolute refractory perod of 1 tme step. 3 Bayesan Inference usng Spkng Neurons 3.1 Inference n a Sngle-Level Model We frst consder on-lne belef propagaton n a sngle-level dynamc graphcal model and show how t can be mplemented n spkng networks. The graphcal model s shown n Fgure 1A and corresponds to a classcal hdden Markov model. Let θ(t) represent the hdden state of a Markov model at tme t wth transton probabltes gven by P (θ(t) = θ θ(t 1) = θ ) = P (θ t θt 1 ) for, = 1... N. Let I(t) be the observable output governed by the probabltes P (I(t) θ(t)). Then, the forward component of the belef propagaton algorthm [12] prescrbes the followng message for state from tme step t to t + 1: = P (I(t) θ t ) P (θ θ t t 1 )m t 1,t (6) If m,1 = P (θ ) (the pror dstrbuton over states), then t s easy to show usng Bayes rule that = P (θ t, I(t),..., I(1)). If the probabltes are normalzed at each update step: = P (I(t) θ t ) P (θ θ t t 1 )m t 1,t /n t 1,t (7) where n t 1,t = mt 1,t, then the message becomes equal to the posteror probablty of the state and current nput, gven all past nputs: = P (θ, t I(t) I(t 1),..., I(1)) (8) 3.2 Neural Implementaton of the Inference Algorthm By comparng the membrane potental equaton (Eq. 4) wth the on-lne belef propagaton equaton (Eq. 7), t s clear that the frst equaton can mplement the second f belef propagaton s performed n the log doman [15],.e., f: v (t + 1) log (9) f ( w I (t) ) = log P (I(t) θ) t (1) g ( u v (t) ) = log( P (θ θ t t 1 )m t 1,t /n t 1,t ) (11) In ths model, the dendrtc flterng functons f and g approxmate the logarthm functon 1, the synaptc currents I (t) and v (t) are approxmated by the correspondng nstantaneous frng rates, and the recurrent synaptc weghts u encode the transton probabltes P (θ t θt 1 ). Normalzaton by n t 1,t s mplemented by subtractng log n t 1,t usng nhbton. 1 An alternatve approach, whch was also found to yeld satsfactory results, s to approxmate the log-sum wth a lnear weghted sum [15], the weghts beng chosen to mnmze the approxmaton error.

4 t θ t θ t θ t θ 2 2 θ θ t 1 1 I(t) A I() B I(t) I(t) C I() D I(t) Fgure 1: Graphcal Models and ther Neural Implementaton. (A) Sngle-level dynamc graphcal model. Each crcle represents a node denotng the state varable θ t whch can take on values θ 1,..., θ N. (B) Recurrent network for mplementng on-lne belef propagaton for the graphcal model n (A). Each crcle represents a neuron encodng a state θ. Arrows represent synaptc connectons. The probablty dstrbuton over state values at each tme step s represented by the entre populaton. (C) Two-level dynamc graphcal model. (D) Two-level network for mplementng onlne belef propagaton for the graphcal model n (C). Arrows represent synaptc connectons n the drecton ponted by the arrow heads. Lnes wthout arrow heads represent bdrectonal connectons. Fnally, snce the membrane potental v (t + 1) s assumed to be proportonal to log (Equaton 9), we have: v (t + 1) = c log + T (12) for some constants c and T. For nosy ntegrate-and-fre neurons, we can use Equaton 5 to calculate the probablty of spkng for each neuron as: P (neuron spkes at tme t + 1) e (v() T )/c (13) log mt, = e = (14) Thus, the probablty of spkng (or equvalently, the nstantaneous frng rate) for neuron n the recurrent network s drectly proportonal to the posteror probablty of the neuron s preferred state and the current nput, gven all past nputs. Fgure 1B llustrates the snglelevel recurrent network model that mplements the on-lne belef propagaton equaton Herarchcal Inference The model descrbed above can be extended to perform on-lne belef propagaton and nference for arbtrary graphcal models. As an example, we descrbe the mplementaton for the two-level herarchcal graphcal model n Fgure 1C. As n the case of the 1-level dynamc model, we defne the followng messages wthn a partcular level and between levels: 1, (message from state to other states at level 1 from tme step t to t + 1), m t 1 2, ( feedforward message from state at level 1 sent to level 2 at tme t), 2, (message from state to other states at level 2 from tme step t to t + 1), and m t 2 1, ( feedback message from state at level 2 sent to level 1 at tme t). Each of these messages can be calculated based on an on-lne verson of loopy belef propagaton [11] for the multply connected two-level graphcal model n Fgure 1C: m t 1 2, = P (θ1,k θ t 2,, t θ t 1 1, )mt 1,t 1, P (I(t) θ1,k) t (15) m t 2 1, = k P (θ t 2, θ t 1 2, )mt 1,t 2, (16)

5 1, = P (I(t) θ1,) t ( P (θ1, θ t 2,, t θ t 1 1,k )mt 2 1,m t 1,t ) 1,k 2, = m t 1 2, ( k P (θ2, θ t t 1 ) 2, )mt 1,t 2, (17) (18) Note the smlarty between the last equaton and the equaton for the sngle-level model (Equaton 6). The equatons above can be mplemented n a 2-level herarchcal recurrent network of ntegrate-and-fre neurons n a manner smlar to the 1-level case. We assume that neuron n level 1 encodes θ 1, as ts preferred state whle neuron n level 2 encodes θ 2,. We also assume specfc feedforward and feedback neurons for computng and conveyng m t 1 2, and mt 2 1, respectvely. Takng the logarthm of both sdes of Equatons 17 and 18, we obtan equatons that can be computed usng the membrane potental dynamcs of ntegrate-and-fre neurons (Equaton 4). Fgure 1D llustrates the correspondng two-level herarchcal network. A modfcaton needed to accommodate Equaton 17 s to allow blnear nteractons between synaptc nputs, whch changes Equaton 4 to: v (t + 1) = f ( w I (t) ) + g ( u kv (t)x k (t) ) (19) Multplcatve nteractons between synaptc nputs have prevously been suggested by several authors (e.g., [1]) and potental mplementatons based on actve dendrtc nteractons have been explored. The model suggested here utlzes these multplcatve nteractons wthn dendrtc branches, n addton to a possble logarthmc transform of the sgnal before t sums wth other sgnals at the soma. Such a model s comparable to recent models of dendrtc computaton (see [6] for more detals). 4 Results 4.1 Sngle-Level Network: Probablstc Moton Detecton and Drecton Selectvty We frst tested the model n a 1D vsual moton detecton task [15]. A sngle-level recurrent network of 3 neurons was used (see Fgure 1B). Fgure 2A shows the feedforward weghts for neurons 1,..., 15: these were recurrently connected to encode transton probabltes based for rghtward moton as shown n Fgure 2B. Feedforward weghts for neurons 16,..., 3 were dentcal to Fgure 2A but ther recurrent connectons encoded transton probabltes for leftward moton (see Fgure 2B). As seen n Fgure 2C, neurons n the network exhbted drecton selectvty. Furthermore, the spkng probablty of neurons reflected the posteror probabltes over tme of moton drecton at a gven locaton (Fgure 2D), suggestng a probablstc nterpretaton of drecton selectve spkng responses n vsual cortcal areas such as V1 and MT. 4.2 Two-Level Network: Spatal Attenton as Herarchcal Bayesan Inference We tested the two-level network mplementaton (Fgure 1D) of herarchcal Bayesan nference usng a smple attenton task prevously used n prmate studes [17]. In an nput mage, a vertcal or horzontal bar could occur ether on the left sde, rght sde, or both sdes (see Fgure 3). The correspondng 2-level generatve model conssted of two states at level 2 (left or rght sde) and four states at level 1: vertcal left, horzontal left, vertcal rght, horzontal rght. Each of these states was encoded by a neuron at the respectve level. The feedforward connectons at level 1 were chosen to be vertcally or horzontally orented Gabor flters localzed to the left or rght sde of the mage. Snce the experment used statc mages, the recurrent connectons at each level mplemented transton probabltes close to 1 for the same state and small random values for other states. The transton probabltes P (θ1,k t θt 2,, θt 1 1, ) were chosen such that for θt 2 = left sde, the transton probabltes for k

6 w 1 w 15 1 θ t θ 15 Neuron Spatal Locaton (pxels) Rghtward Leftward A B Rghtward Moton Leftward Moton Rghtward Moton Leftward Moton 1 12 C D Fgure 2: Responses from the Sngle-Level Moton Detecton Network. (A) Feedforward weghts for neurons 1,..., 15 (rghtward moton selectve neurons). Feedforward weghts for neurons 16,..., 3 (leftward moton selectve) are dentcal. (B) Recurrent weghts encodng the transton probabltes P (θ θ) t for, = 1,..., 3. Probablty values are proportonal to pxel brghtness. (C) Spkng responses of three of the frst 15 neurons n the recurrent network (neurons 8, 1, and 12). As s evdent, these neurons have become selectve for rghtward moton as a consequence of the recurrent connectons specfed n (B). (D) Posteror probabltes over tme of moton drecton (at a gven locaton) encoded by the three neurons for rghtward and leftward moton. states θ t 1 codng for the rght sde were set to values close to zero (and vce versa, for θ t 2 = rght sde). As shown n Fgure 3, the response of a neuron at level 1 that, for example, prefers a vertcal edge on the rght mmcs the response of a V4 neuron wth and wthout attenton (see fgure capton for more detals). The ntal settng of the prors at level 2 s the crucal determnant of attentonal modulaton n level 1 neurons, suggestng that feedback from hgher cortcal areas may convey task-specfc prors that are ntegrated nto V4 responses. 5 Dscusson and Conclusons We have shown that recurrent networks of nosy ntegrate-and-fre neurons can perform approxmate Bayesan nference for sngle- and mult-level dynamc graphcal models. The model suggests a new nterpretaton of the spkng probablty of a neuron n terms of the posteror probablty of the preferred state encoded by the neuron, gven past nputs. We llustrated the model usng two problems: nference of moton drecton n a sngle-level network and herarchcal nference of obect dentty at an attended vsual locaton n a twolevel network. In the frst case, neurons generated drecton-selectve spkes encodng the probablty of moton n a partcular drecton. In the second case, attentonal effects smlar to those observed n prmate cortcal areas V2 and V4 emerged as a result of mposng approprate prors at the hghest level. The results obtaned thus far are encouragng but several mportant questons reman. How does the approach scale to more realstc graphcal models? The two-level model explored n ths paper assumed statonary obects, resultng n smplfed dynamcs for the two levels n our recurrent network. Experments are currently underway to test the robustness of the proposed model when rcher classes of dynamcs are ntroduced at the dfferent levels. An-

7 A B Spkes/second 3 Ref Att Away Tme steps from stm onset Spkes/second 3 25 Par Att Away Tme steps from stm onset C D Spkes/second Par Att Ref Tme steps from stm onset Fgure 3: Responses from the Two-Level Herarchcal Network. (A) Top panel: Input mage (lastng the frst 15 tme steps) contanng a vertcal bar ( Reference ) on the rght sde. Each nput was convolved wth a retnal spatotemporal flter. Mddle: Three sample spke trans from the 1st level neuron whose preferred stmulus was a vertcal bar on the rght sde. Bottom: Posteror probablty of a vertcal bar (= spkng probablty or nstantaneous frng rate of the neuron) plotted over tme. (B) Top panel: An nput contanng two stmul ( Par ). Below: Sample spke trans and posteror probablty for the same neuron as n (A). (C) When attenton s focused on the rght sde (depcted by the whte oval) by ntalzng the pror probablty encoded by the 2nd level rght-codng neuron at a hgher value than the left-codng neuron, the frng rate for the 1st level neuron n (A) ncreases to a level comparable to that n (A). (D) Responses from a neuron n prmate area V4 wthout attenton (top panel, Ref Att Away and Par Att Away; compare wth (A) and (B)) and wth attenton (bottom panel, Par Att Ref; compare wth (C)) (from [17]). Smlar responses are seen n V2 [17].

8 other open queston s how actve dendrtc processes could support probablstc ntegraton of messages from local, lower-level, and hgher-level neurons, as suggested n Secton 3. We ntend to nvestgate ths queston usng bophyscal (compartmental) models of cortcal neurons. Fnally, how can the feedforward, feedback, and recurrent synaptc weghts n the networks be learned drectly from nput data? We hope to nvestgate ths queston usng bologcally-plausble approxmatons to the expectaton-maxmzaton (EM) algorthm. Acknowledgments. Ths research was supported by grants from ONR, NSF, and the Packard Foundaton. I am grateful to Wolfram Gerstner, Mchael Shadlen, Aaron Shon, Eero Smoncell, and Yar Wess for dscussons on topcs related to ths paper. References [1] C. H. Anderson and D. C. Van Essen. Neurobologcal computatonal systems. In Computatonal Intellgence: Imtatng Lfe, pages New York, NY: IEEE Press, [2] R. H. S. Carpenter and M. L. L. Wllams. Neural computaton of log lkelhood n control of saccadc eye movements. Nature, 377:59 62, [3] S. Deneve and A. Pouget. Bayesan estmaton by nterconnected neural networks (abstract no ). Socety for Neuroscence Abstracts, 27, 21. [4] W. Gerstner. Populaton dynamcs of spkng neurons: Fast transents, asynchronous states, and lockng. Neural Computaton, 12(1):43 89, 2. [5] J. I. Gold and M. N. Shadlen. Neural computatons that underle decsons about sensory stmul. Trends n Cogntve Scences, 5(1):1 16, 21. [6] M. Hausser and B. Mel. Dendrtes: bug or feature? Current Opnon n Neurobology, 13: , 23. [7] D. C. Knll and W. Rchards. Percepton as Bayesan Inference. Cambrdge, UK: Cambrdge Unversty Press, [8] K. P. Kördng and D. Wolpert. Bayesan ntegraton n sensormotor learnng. Nature, 427: , 24. [9] T. S. Lee and D. Mumford. Herarchcal Bayesan nference n the vsual cortex. Journal of the Optcal Socety of Amerca A, 2(7): , 23. [1] B. W. Mel. NMDA-based pattern dscrmnaton n a modeled cortcal neuron. Neural Computaton, 4(4):52 517, [11] K. Murphy, Y. Wess, and M. Jordan. Loopy belef propagaton for approxmate nference: An emprcal study. In Proceedngs of UAI (Uncertanty n AI), pages [12] J. Pearl. Probablstc Reasonng n Intellgent Systems: Networks of Plausble Inference. Morgan Kaufmann, San Mateo, CA, [13] H. E. Plesser and W. Gerstner. Nose n ntegrate-and-fre neurons: From stochastc nput to escape rates. Neural Computaton, 12(2): , 2. [14] A. Pouget, P. Dayan, and R. S. Zemel. Inference and computaton wth populaton codes. Annual Revew of Neuroscence, 26:381 41, 23. [15] R. P. N. Rao. Bayesan computaton n recurrent neural crcuts. Neural Computaton, 16(1):1 38, 24. [16] R. P. N. Rao, B. A. Olshausen, and M. S. Lewck. Probablstc Models of the Bran: Percepton and Neural Functon. Cambrdge, MA: MIT Press, 22. [17] J. H. Reynolds, L. Chelazz, and R. Desmone. Compettve mechansms subserve attenton n macaque areas V2 and V4. Journal of Neuroscence, 19: , [18] M. Sahan and P. Dayan. Doubly dstrbutonal populaton codes: Smultaneous representaton of uncertanty and multplcty. Neural Computaton, 15: , 23. [19] Y. Wess, E. P. Smoncell, and E. H. Adelson. Moton llusons as optmal percepts. Nature Neuroscence, 5(6):598 64, 22. [2] R. S. Zemel, P. Dayan, and A. Pouget. Probablstc nterpretaton of populaton codes. Neural Computaton, 1(2):43 43, 1998.