Neurl ssembly binding in linguisic represenion Frnk vn der Velde & Mrc de Kmps Cogniive Psychology Uni, Universiy of Leiden, Wssenrseweg 52, 2333 AK Leiden, The Neherlnds, vdvelde@fsw.leidenuniv.nl Absrc. We presen neurl rchiecure of senence represenion. Words re represened wih neurl cell ssemblies. Relions beween words re represened wih srucure ssemblies. Word nd srucure ssemblies re bound emporrily o form senence represenion. We show how muliple senences cn be represened simulneously, nd we simule how specific informion cn be rerieved from he rchiecure. The ssemblies re simuled s populions of spiking neurons, in erms of he verge firing re of he neurons in he populion. 1. Inroducion We presen neurl model of bsic senence srucure. Words re represened wih neurl ssemblies. Relions beween he words in senence cnno be represened wih direc ssociions beween hese word ssemblies. For insnce, he ssociion of mouse-chses-c does no disinguish beween he senences The mouse chses he c nd The c chses he mouse. Therefore, word ssemblies re embedded in neurl rchiecure in which srucurl relions cn be formed beween he word ssemblies. The neurl rchiecure is implemened by mens of srucure ssemblies h inerc wih he word ssemblies. The srucure ssemblies provide he possibiliy o represen differen insniions of he sme word ssembly, nd hey re used o represen elemens of syncic srucures. Here, we use srucure ssemblies for represenion of Noun Phrses (NPs) nd Verb Phrses (VPs). Figure 1 presens he represenion of he senence The mouse chses he c in his rchiecure. I consiss of word ssemblies, srucure ssemblies for NPs nd VPs, ging circuis used for dynmic conrol, nd memory circuis used o bind word nd srucure ssemblies ino (emporl) represenion of he senence. Thus, emporrily, mouse, chses nd c re bound o differen srucure ssemblies, which in urn re bound o represen he senence. Srucure ssemblies re composed of min ssembly (N i for NP nd V i for VP) nd subssemblies, here for gen () nd heme (). Subssemblies re conneced o he min ssembly by ging circuis. Word nd srucure ssemblies re bound by civing memory circuis h connec hem. Srucure ssemblies re bound by civing he memory circuis h connec heir gen/heme subssemblies. Similr represenions cn be formed for senences like The c chses he mouse nd The mouse sees he dog (figure 1). The words mouse nd chses occur in more hn one senence in figure 1. This crees he problem of he muliple insniion of he ssemblies for mouse nd chses [1]. Figure 1 illusres h he problem of muliple insniion is solved by binding ech word ssembly (emporrily) o unique srucure ssembly. For insnce, he word ssembly for mouse is bound o he NP ssemblies N 1, N 4 nd N 5
in figure 1. In his wy, mouse cn be represened s gen in one senence (by N 1 or N 5 ) nd s heme in noher (by N 4 ). Similrly, he differen VP ssemblies (V 1 nd V 2 ) represen chses in differen senences. The inernl srucure of he NP nd VP ssemblies, given by he ging circuis, is of crucil impornce in his respec. Wihou his inernl srucure, he represenions presened in figure 1 would lso consis of direc ssociions beween neurl ssemblies, which would resul in filure o disinguish beween The mouse chses he c nd The c chses he mouse. Wih he conrol of civion provided by ging circuis, he represenions of hese wo senences cn be selecively (re)cived. We will illusre his in he ls secion. In priculr, we will invesige how informion cn be rerieved (i.e., nswers o binding quesions cn be produced) in he rchiecure presened in figure 1, even wih muliple insniion of represenions s illusred in figure 1. Firs, however, we will describe he ging nd memory circuis. V 1 srucure ssembly N 1 V 1 N 2 ging circui mouse chses c memory circui N 3 V 2 N 4 N 5 V 3 N 6 c chses mouse mouse sees dog Figure 1. Senence represenion wih neurl ssemblies. Circles nd ovls represen populions of neurons (ssemblies). V = verb phrse, N = noun phrse, = gen, = heme. 2. Ging nd memory circuis Figure 2 (lef) illusres he ging circui. The overll circui is in fc combinion of wo ging circuis, one for ech direcion. They re disinhibiion circuis [2] h conrol he flow of civion beween wo ssemblies (X nd Y in figure 2) by mens of n exernl conrol signl. The ging circui h conrols he flow of civion from X o Y operes in he following mnner. If he ssembly X is cive, i cives n inhibiion neuron (or group of neurons) i x, which inhibis he flow of civion from X o X ou. When i x is inhibied by noher inhibiion neuron (I x ) h is cived by n exernl conrol signl, X cives X ou. In urn, X ou cives Y. The ging circui from Y o X operes in similr mnner.
The memory circui is presened in figure 2 (righ). I lso consiss of wo ging circuis h conrol he flow of civion from X o Y nd vice vers. In his cse, however, he conrol signl in boh ging circuis resuls from dely ssembly. The dely ssembly is cived when X nd Y re cive simulneously, nd i remins cive for while due o he reverbering civiy in his ssembly (see ppendix). The memory circuis in figure 1 re cive. Ging Circui Memory Circui I x conrol XoY i x dely i i X ou X Y X Y Symbol: X Y Y ou i y I y conrol YoX Symbols: X Y (incive) X Y (cive = binding ) Figure 2: Lef: ging circui. Righ: memory circui. Circles nd ovls represen ssemblies. Circles wih I or i represen inhibiory (populions of) neurons. 3. Rerieving informion from he rchiecure We will illusre he biliy o rerieve informion from his rchiecure by nlyzing nd simuling he producion of he nswer o he quesion Whom does he mouse chse?, when he senences presened in figure 1 re sored simulneously. The ssemblies were simuled s populions of spiking neurons (see he ppendix). The simulions re illusred in he figure 3. The figure (middle) lso shows wo free VP min ssemblies (V 4 nd V 5 ), no used in ny senence represenion, o compre he civion of free ssemblies wih bound ssemblies in his process. The vericl lines re used o compre he iming of evens. The simulions sr = 0 ms. Before h ime, he only cive ssemblies re he dely ssemblies in he memory circuis (see figure 1). The quesion Whom does he mouse chse? provides informion h mouse is he gen of chses nd i sks for he heme of he senence mouse chses x. The producion of he nswer consiss of he selecive civion of he word ssembly for c. Bckrcking, his requires he selecive civion of he min ssembly N 2, he heme subssemblies for N 2 nd V 1, nd he min ssembly V 1 (in reversed order). This process proceeds s follows. Firs, we ssume h he quesion emporrily
cives he represenions for mouse nd chses nd produces he conrol signl h cives he ging circuis for he gen subssemblies of he NP ssemblies. Figure 1 shows he civion of he ssemblies for mouse nd chses (beginning = 0 ms). To produce he selecive civion of he word ssembly for c ler on, oher word ssemblies cnno be cive h momen. Therefore, i is ssumed h he word ssemblies re inhibied fer cerin ime, nd remin inhibied unil c is o be cived. The horizonl br in figure 1 (righ) indices he ime inervl in which he word ssemblies (mouse nd chses) re cive. The end of he inervl ( = 400 ms) is mrked by solid vericl line. Figure 3. Acivion of he neurl ssemblies in figure 1 (in Hz/ms). Lef pnel: The noun ssemblies N 1 o N 6. Middle pnel: The verb ssemblies V 1 o V 5. Righ pnel: The word ssemblies for mouse, c nd chses, nd he srucure ssemblies for N 1 -gen nd V 1 -heme. The civion of mouse resuls in he civion of N 1, N 4, nd N 5, nd he civion of chses resuls in he civion of V 1 nd V 2 (figure 3). As indiced wih he solid vericl line in figure 3, N 1, N 4, nd N 5 remin cive when mouse is inhibied. This resuls from he reverbering ( dely ) properies of min ssemblies (see he ppendix). As long s V 1 nd V 2 re boh cive, he quesion Whom does
he mouse chse? cnno be nswered. To produce he nswer, he ging circuis for he heme VP subssemblies hve o be cived, becuse he quesion sks for he heme of mouse chses x. However, when boh V 1 nd V 2 re cive, his will resul in he civion of he heme subssemblies for V 1 nd V 2, nd, in urn, of c nd mouse (vi N 2 nd N 4 ). To preven his, WTA compeiion beween V 1 nd V 2 hs o occur, wih V 1 s he winner. The compeiion process beween he VP ssemblies proceeds s follows. VP min ssemblies re conneced o populion of inhibiory neurons. In comprison wih he NP ssemblies cived by mouse (figure 3, lef), he civiy of V 1 nd V 2 (figure 3, middle), iniied by chses, is reduced due o he compeiion beween he VP ssemblies. The compeiion cn be decided by civing he ging circuis for he gen subssemblies. This resuls in he civion of he gen subssemblies for N 1, N 4 nd N 5, becuse hey re he cive NP ssemblies (figure 3, lef). The civion of he N 1 gen subssembly is illusred in figure 3 (righ). The horizonl br here indices he ime inervl in which he ging circuis re cived (from = 150 ms o = 400 ms). The beginning of his inervl is indiced by he serix in figure 3 (middle). The cive gen subssemblies N 1 nd N 5 re bound o he VP ssemblies V 1 nd V 3 respecively (see figure 1). Thus, he VP ssemblies V 1 nd V 3 receive civion from hese NP ssemblies when he gen ging circuis re cived. (The gen subssembly of N 4 is no bound o VP ssembly, becuse N 4 is bound o VP ssembly wih is heme subssembly, see figure 1). As resul, V 1 wins he compeiion beween he VP ssemblies, becuse V 1 receives civion from chses nd N 1, wheres V 2 only receives civion from chses, nd V 3 only receives civion from N 5. Figure 3 (middle) shows h V 1 is he only cive VP ssembly fer his compeiion process. The civion of V 2 nd V 3 is reduced o he level of he free ssemblies V 4 nd V 5. When V 1 remins s he only cive VP ssembly, he nswer c cn be produced by civing he heme ging circuis in he direcion from VP o NP. This will produce he selecive civion of N 2, which is he NP ssembly bound o c in figure 1, provided h he cive NP min ssemblies (N 1, N 4 nd N 5 in figure 3) re inhibied firs. The horizonl br in figure 3 (lef) illusres he ime inervl of his inhibiion (from = 600 ms o = 650 ms). Afer he inhibiion of he cive NP ssemblies, he heme ging circuis cn be cived. The horizonl br in figure 3 (V1-heme) illusres he ime inervl (from = 700 ms o = 800 ms). The onse of his even is lso illusred by he dshed vericl line in figure 3 (lef, righ). As resul, he heme subssembly of V 1 nd he min ssembly N 2 re now selecively cived s well. As resul, he word ssembly for c cn be cived. References 1. Jckendoff, R. (2002). Foundions of Lnguge. Oxford: Oxford Universiy Press. 2. Vn der Velde, F. & de Kmps, M. (2001). From knowing wh o knowing where: Modeling objec-bsed enion wih feedbck disinhibiion of civion. Journl of Cogniive Neuroscience, 13, (4), 479-491. 3. Gersner, W. (1995). Time srucure of he civiy in neurl-nework models. Physicl Review E, 51, 738-758.
Appendix The simulions re bsed on nework of exciory nd inhibiory populions [3]. The populion re A i is given by (wih α = E for exciory populions nd α = I for inhibiory populions): da i τ α = ( A x + F (inpu A Σ j w ij A ij )) + A i I noise (1) d τ E (τ I ) is he ime consn, wih τ E = 10 ms nd τ I = 5 ms. The w ij (or w j i ) re he efficcies from populion j ono populion i: w ij is negive iff j is n inhibiory populion. Every 1 ms, frcion I noise of he populion civion is injeced ino ech populion wih µ = 0, σ = 0.02. For F (x) we ook: f mx F (x) = (2) (1 + e β(x θ) ) wih f mx = 30 Hz, β = 1 nd θ = 3. The ging circuis in our simulion resuled from (1) by insering X, Y, X ou, Y ou, i x, i y, I x, nd I y in (1) in line wih he digrm of figure 1 (lef). We ook w X Xou = w Y You = w X ix = w Y iy = 0.25 nd w ix X ou = w iy Y ou = w Ix i x = w Iy i y = 1. We ook w You X = w Xou Y = 0.1. The ging circui cn be cived by he inpu signls conrol XoY nd conrol Y ox, from wo ouside populions, wih civion f mx nd w conrol = 0.2. The memory circuis were simuled s ging circuis, wih conrol signl 0 ( off ) or f mx ( on ), nd w You X = w Xou Y = 0.2. We ssumed he following properies for dely populion: 1. I is cive once is inpu hs been bove hreshold θ dely in he ps nd i hs no been decived since. 2. I is decived once he ne fferens o he ssembly psses cerin negive hreshold θ dec (i.e., here is ne inhibiion). 3. If i is incive, i funcions s n ordinry populion of exciory neurons. To re dely populion s pr of he nework, we ssumed: 1. If is civiy is bove hreshold θ dely nd ne inpu is exciory, hen is ime consn is τ dely = τ E. 2. If is civiy is bove hreshold θ dely bu decresing, while ne inpu is bove θ dec, he ime consn is very lrge: τ inf. 3. If ne inpu is below θ dec he ime consn is rese o τ E nd, since ne inpu is negive, memory civiy will decy wihin pproximely τ E ms. We ook θ dec = 0.2, θ dely = 4 nd τ dely = 10000 ms. Srucure ssemblies consis of min ssemblies nd subssemblies. Min ssemblies re dely populions. Word ssemblies re cive wih civion given by f mx nd frcion I noise. They c on sucure ssemblies wih efficcy w inpu = 0.2. VP srucure ssemblies re conneced wih cenrl inhibiory pool, which cs on hem wih efficcy w pool V P = 0.03. The inhibiory pool receives inpu from he VP ssemblies wih efficcy w V P pool = 0.03. In ll, he simuled model consised of 624 populions. Inegrion of he sysem of equions (1) evolved simulneously for he enire model, using fourhorder Runge-Ku inegrion wih n inegrion ime sep h = 0.01 ms.