Behavior and neural basis of near-optimal visual search

Transcription

1 r t c l e s Behvor nd neurl ss of ner-optml vsul serch We J M,6, Vdhy Nvlpm 2,5,6, Jeffrey M Bec 3,6, Ronld vn den Berg,6 & Alendre Pouget 4 2 Nture Amerc, Inc. All rghts reserved. The lty to serch effcently for trget n cluttered envronment s one of the most remrle functons of the nervous system. Ths ts s dffcult under nturl crcumstnces, s the rellty of sensory nformton cn vry gretly cross spce nd tme nd s typclly pror unnown to the oserver. In contrst, vsul-serch eperments commonly use stmul of equl nd nown rellty. In trget detecton ts, we rndomly ssgned hgh or low rellty to ech tem on trl-y-trl ss. An optml oserver would weght the oservtons y ther trl-to-trl rellty nd comne them usng specfc nonlner ntegrton rule. We found tht humns were ner-optml, regrdless of whether dstrctors were homogeneous or heterogeneous nd whether rellty ws mnpulted through contrst or shpe. We present neurl-networ mplementton of ner-optml vsul serch sed on prolstc populton codng. The networ mtched humn performnce. Serchng for trget mong dstrctors s ts of gret ecologcl relevnce, whether for n nml tryng to detect cmouflged predtor or for student loong for note on cluttered des. It s well nown tht the dffculty of detectng trget depends on the numer of tems n scene (set sze) 6, trget-dstrctor smlrty 7 nd dstrctor heterogenety 7. However, n mportnt spect hs lrgely een gnored n prevous wor: the effect of dfferentl stmulus rellty. In relstc serch scenes, some stmul provde more relle nformton thn others, for emple, s result of dfferences n contrst, dstnce, shpe or lur. In lortory serch tss, however, such prmeters re usully held constnt cross tems nd trls. Vryng rellty on trl-y-trl ss s ey mnpulton when studyng whether the humn rn performs prolstclly optml (Byesn) nference. Ths s ecuse n optml oserver weghts more relle peces of sensory evdence more hevly when mng perceptul judgment. For emple, when two nosy sensory cues out sngle underlyng stmulus hve to e comned, n optml oserver ssgns hgher weght to the cue tht, on tht trl, s most relle. Humns follow ths strtegy closely nd, s result, they perform ner-optmlly n such tss,2. The success of prolstc models of cue comnton nd other perceptul tss ndctes tht the rn utlzes nowledge of stmulus uncertnty nd suggests tht t computes wth prolty dstrutons. The neurl ss of computng wth prolty dstrutons hs ecome the suject of theoretcl 3 nd physologcl 4 studes. A mjor lmtton of studes demonstrtng ner-optml percepton n the presence of sensory nose s tht most of them use reltvely smple tss tht requre oservers to nfer physcl feture of sngle stmulus tem. To determne how prevlent ner-optml nference n percepton s, t s necessry to eplore tss wth more comple structures. In vsul trget detecton, ech dsply contns multple tems nd ther fetures re not of nterest y themselves ut only serve to nform the more strct, ctegorcl judgment of trget presence. It s therefore consderly more comple thn cue comnton. We report here tht, when judgng trget presence, humns te nto ccount the relltes of the oservtons on sngle trl n ner-optml mnner. We lso creted neurl mplementton of Byes-optml vsul serch. Although prevous studes hve focused on connectng serch ehvor to the ctvty of sngle neurons or pools of dentcl neurons 5,6, our mplementton s sed on the ctvty of populton of neurons wth dfferent tunng propertes. Such popultons cn smultneously encode stmulus vlue nd ts rellty, therey llowng for computton wth prolty dstrutons on sngle trl. Our results nd model re not the frst ttempt to pproch vsul serch from prolstc perspectve,6,8,5 8. However, we etend prevous des n three fundmentl wys. Frst, we consder stutons n whch the rellty of the vsul nformton vres unpredctly n nd cross dsplys. Ths s common occurrence n the rel world nd strong test of prolstc models of percepton. Second, we del wth the dffcult prolem of how nformton should e comned cross unnown dstrctor vlues nd sptl loctons, prolem tht nvolves type of nference nown s mrgnlzton. Fnlly, we provde neurl mplementton of ner-optml vsul serch tht cn ccount for our dt nd cn del wth the mrgnlzton prolem. RESULTS Theory We used ts n whch sujects were presented refly wth n rry of N orented rs (Fg. ) nd reported whether trget ws present, regrdless of ts locton. The trget ws r of fed orentton, denoted s T, whch ws present wth prolty of.5. In seprte eperments, the dstrctor orentton ws drwn from ether delt functon, n whch ll dstrctors hve the sme orentton, denoted s D (we cll ths the cse of homogeneous dstrctors; Fg. ), or Deprtment of Neuroscence, Bylor College of Medcne, Houston, Tes, USA. 2 Deprtment of Bology, Clforn Insttute of Technology, Psden, Clforn, USA. 3 Gtsy Computtonl Neuroscence Unt, Unversty College London, London, UK. 4 Deprtment of Brn nd Cogntve Scences, Unversty of Rochester, Rochester, New Yor, USA. 5 Present ddress: Yhoo! Reserch, Snt Clr, Clforn, USA. 6 These uthors contruted eqully to ths wor. Correspondence should e ddressed to W.J.M. (wjm@cm.edu). Receved 6 Jnury; ccepted Mrch; pulshed onlne 8 My 2; do:.38/nn.284 nture NEUROSCIENCE VOLUME 4 NUMBER 6 JUNE 2 783

2 r t c l e s Fgure Rellty nd nference n vsul Homogeneous Heterogeneous serch. () Serch under unequl relltes. Stmulus rellty s controlled y contrst, ut the trget (red crcle) s defned only y s s orentton. Left, homogeneous dstrctors. 2 2 T T Rght, heterogeneous dstrctors. () Sttstcl,,T N structure of the ts (genertve model). Arrows ndcte condtonl dependences. T, glol trget presence; T, locl trget presence; s, stmulus orentton;, nosy oservton. In c d the neurl formulton, s replced y pttern d 3 2 s N N of neurl ctvty, r. (c) The optml decson >? 3 2 process for nferrng trget presence nverts the 2 d 2 d Tˆ genertve model. d, locl log lelhood rto of 4 M Byes 4 trget presence; d, glol log lelhood rto. The sgn of d determnes the decson nd ts 6 N d N solute vlue reflects confdence. (d) Left, m model ppled to homogeneous-dstrctor dsply s n. At ech locton, n orentton detector produces response (r plots) tht s vrle, ut s on verge hgher for the trget thn for dstrctor. The decson s sed on the lrgest detector response. Stmulus rellty s gnored. Rght, optml decson process for the sme dsply. At ech locton, lelhood functon over orentton s encoded (smll plots), reflectng not only the most lely orentton (mode) ut lso ts uncertnty (wdth). 2 Nture Amerc, Inc. All rghts reserved. unform dstruton on orentton spce [,8 ) (we cll ths the cse of heterogeneous dstrctors; Fg. ). Notly, ech tem ws ndependently ssgned hgh or low rellty. The strtng pont of prolstc models of serch under sensory nose s the ssumpton tht oservers only hve ccess to set of nosy oservtons of the stmul. The oservton t the th locton, denoted ( =,..,N), corresponds to the mmum-lelhood estmte of the stmulus t tht locton otned from nosy underlyng neurl representton. To judge whether the trget orentton s present or not, the N oservtons hve to e comned nto sngle numer. The prolstclly optml oserver performs ths comnton usng nowledge of the sttstcl structure of the ts, lso clled the genertve model (Fg.,c). We denote trget presence wth the nry vrle T, whch s when the trget s sent nd when t s present. When the pror s flt, tht s, p(t = ) = p(t = ) =.5, the optml decson s sed on the log lelhood rto 9, denoted y d. ( ) ( ) p,..., N T = d = log p,..., T = When d s postve, the oserver responds trget present. When the pror s not flt, d s compred to decson crteron dfferent from. The solute vlue of d s mesure of confdence. If ech locton s eqully lely to contn the trget, we cn wrte d n terms of locl vrles 6,2,2 (see Supplementry Results). N N e d d = log N = ( ) () (2) p T = Here d s defned s d = log p( T = ), where T denotes trget presence t the th locton (gn or ). We cll d the locl nd d the glol log lelhood rto. Avergng over n unnown vrle tht ffects the oservtons, such s trget locton n equton (2), s nown s mrgnlzton. In the cse of homogeneous dstrctors, we model the oservton s eng drwn from norml dstruton wth men s, the true stmulus vlue t tht locton, nd vrnce σ 2. We defne rellty s the nverse of ths vrnce. Prevous wor hs consdered stutons n whch σ s dentcl for ll nd constnt cross trls; we remove these restrctons here. The locl log lelhood rto cn then e wrtten s (see Supplementry Results) p s = s d = log p s = s ( T ) ( ) = D ( ) T D ( s s st s ) 2 + D 2 s Thus, ll oservtons re weghted y ther relltes, /σ 2, efore eng comned cross loctons ccordng to equton (2). Ths weghtng y rellty prllels optml cue weghtng n cue comnton,2. A strength of the prolstc frmewor s tht t lso pples drectly to heterogeneous dstrctors (Fg. ). In ths cse, the optml oserver mrgnlzes over the unnown dstrctor orentton to otn the locl lelhood of trget sence ( ) = ( ) ( = ) p T = p s p s T ds where L(s ) = p( s ) s the lelhood functon over orentton nd p(s T = ) s the prolty dstruton over dstrctor orentton. Equton (4) results n locl log lelhood rto tht depends once gn on locl rellty (see Onlne Methods nd Supplementry Results). Thus, optml serch requres two mrgnlztons when dstrctors re heterogeneous, one over orentton (equton (4)) nd one over locton (equton (2)). The glol log lelhood, equton (2), reduces to prevously proposed decson vrles n two etreme cses. If one d s much lrger thn ll others (for emple, when the trget s very dfferent from the dstrctors), the sum s domnted y the term wth the lrgest eponent d, tht s, d m (d ) log(n). If, n ddton, dstrctors re homogeneous nd rellty s dentcl t ll loctons, then, ecuse of equton (3), d s lnerly relted to m. The ltter s the decson vrle n the mmum-of-outputs (or m) model (Fg. d), whch hs een used to descre dt from serch eperments wth homogeneous dstrctors of dentcl nd fed rellty 6,8,2,22. In dfferent lmt, when ll d re smll, dstrctors re homogeneous nd rellty s dentcl t ll loctons, the optml rule s ppromted y the sum rule, wth d = Σ 23 (see Supplementry Results). Although the m nd sum rules re elegnt specl cses, they re not optml nd, n prtculr, they do not weght the oservtons y ther relltes. Ths s prolemtc when rellty vres cross loctons nd trls. In contrst, the optml rules, equtons (3) (3) (4) 784 VOLUME 4 NUMBER 6 JUNE 2 nture NEUROSCIENCE

3 r t c l e s 2 Nture Amerc, Inc. All rghts reserved. Frequency Frequency N = 4 N = 6 N = Log lelhood rto, d Flse-lrm rte nd (4), use the full lelhood functons over orentton, L(s ) = p( s ), not just ther modes, to compute the lelhood of trget presence (Fg. d). We emne some propertes of the optml oserver model (Fg. 2). The oserver s performnce depends on the overlp etween the dstrutons of the glol log lelhood rto, d, n trget-present nd trget-sent dsplys (Fg. 2). These dstrutons re n generl hghly non-gussn. Usng these dstrutons, we cn plot theoretcl recever opertng chrcterstc (ROC) curves (Fg. 2). We compred the optml model to the followng seven suoptml models: sngle rellty, n whch n oserver uses the correct comnton rule, equton (2), ut ncorrectly ssumes dentcl relltes for ll stmul; m, n whch the m model s ppled to the oservtons, d = m, when dstrctors re homogeneous, or ppled to the ctvtes r of Posson neurons est tuned to the trget (one per locton), d = m r, when dstrctors re heterogeneous (see Onlne Methods nd Supplementry Results); m d, n whch the m model s ppled to the locl log lelhood rtos, d = m d ; sum, n whch the sum model s ppled to the oservtons, d = Σ (homogeneous), or to sngle-neuron ctvtes, d = Σ r (heterogeneous); sum d, n whch the sum model s ppled to the locl log lelhood rtos, d = Σ d ; L 2, n whch n oserver uses the decson rule d = 2 (or d = 2 ), nd L 4, n whch n oserver uses the decson rule r d = 4 4 (or d = 4 r 4 ). The prolty summton models L 2 nd L 4 re ntermedte etween the sum nd m models 24, s the sum model s L nd the m model s L. Behvorl eperments To test whether humn performnce n vsul serch est mtches the performnce predcted y the optml oserver, we conducted ehvorl eperments nlogous to cue comnton studes (Fg. 3). The trget ws defned y ts orentton nd ts vlue ws fed throughout gven eperment. In seprte eperments, we used contrst of r (eperments nd 2) nd eccentrcty (elongton) of n ellpse (eperments 3 nd 4) to mnpulte rellty. Rellty could te two vlues, low nd hgh. Dstrctors were homogeneous n eperments nd 3 nd heterogeneous nd drwn from Detecton rte Hgh All Low Log lelhood rto, d Flse-lrm rte Detecton rte Fgure 2 Optml serch under unequl relltes. () Theoretcl dstruton of the glol log lelhood rto, d, cross 6 trls t N = 4, n trget-sent dsplys (red) nd trget-present dsplys (lue). Insets show emple dsplys. Top, homogeneous dstrctors, wth trget nd dstrctor orenttons prt. Internl representtons were drwn from norml dstrutons wth s.d. σ equl to 2 or 6. Bmodlty rses from the fct tht stmulus cn hve low or hgh rellty. Bottom, heterogeneous dstrctors. Stmulus orenttons s were drwn from unform dstruton nd ther nternl representtons were drwn from Von Mses dstrutons wth concentrton prmeters κ equl to 5 or. Bmodlty rses ecuse the cosne of unformly dstruted ngle s modlly dstruted. () ROC curves of the optml oserver. Top, homogeneous dstrctors t N = 4,6,8. Bottom, heterogeneous dstrctors t N = 4, uncondtoned (lue) or condtoned on the trget hvng hgh (red) or low (lc) rellty. Prmeters re s n. Nonconcvty rses from condtonng on trget rellty. unform dstruton n eperments 2 nd 4. Eperments nd 2 were repeted seprtely t set sze 2, referred to s eperments nd 2. Ech eperment conssted of three rellty condtons: LOW, n whch the rellty of ll tems on ll trls ws low, HIGH, n whch the rellty of ll tems on ll trls ws hgh, nd MIXED, n whch the relltes of the tems were drwn rndomly nd ndependently on ech trl, producng dsplys n whch stmul dffered n rellty. Presentton tme rnged from 5 to 75 ms, dependng on the suject nd the eperment (see Onlne Methods). After reportng trget presence, sujects lso reported ther confdence level (low, medum or hgh), llowng us to plot emprcl ROC curves 9. We emned emprcl ROC curves otned n the LOW nd HIGH condtons, long wth the est fts of the optml, snglerellty, nd m nd sum models (Fg. 4 nd Supplementry Fgs. 6). The sngle-rellty model s equvlent to the optml model n these condtons. From these condtons, we estmted the sensory nose levels ssocted wth low-rellty nd hghrellty stmulus. Usng those two prmeters, we predcted the ROCs n the MIXED condton (see Onlne Methods nd Fg. 4). Decson crter (whch ncorporte the pror over T ) were free prmeters. The sngle-rellty model hs n etr prmeter, the ssumed rellty, whch we estmted from the MIXED condton. The predcton for the MIXED condton s n mportnt test of the models, s the optml oserver tes nto ccount stmulus rellty (for emple, equton (3)) nd comnes locl decson vrles n + Confdence? ( 3) Trget present? (yes/no) HIGH LOW MIXED Eperments nd 2 Eperments 3 nd 4 Fgure 3 Epermentl procedure. () Sujects report through ey press whether predefned trget s present n the dsply, then rte ther confdence on scle from to 3. () Epermentl condtons. Items n sngle dsply cn e ll hgh-rellty (HIGH), ll low-rellty (LOW), or comnton of oth relltes (MIXED). Stmulus rellty ws mnpulted through contrst n Eperments nd 2 (left), nd through ellpse eccentrcty n Eperments 3 nd 4 (rght). Emple dsplys show homogeneous dstrctors; the procedure ws dentcl for heterogeneous dstrctors. nture NEUROSCIENCE VOLUME 4 NUMBER 6 JUNE 2 785

4 r t c l e s. LOW Ftted HIGH Predcted MIXED N = 4 MIXED N = 6 MIXED N = 8 Detecton rte.5 Detecton rte LOW HIGH MIXED Trget low rellty MIXED All MIXED Trget hgh rellty Dt Optml M Chnce Flse-lrm rte Flse-lrm rte Flse-lrm rte Flse-lrm rte Flse-lrm rte 2 Nture Amerc, Inc. All rghts reserved. Fgure 4 Model predctons for ndvdul-suject recever opertng chrcterstcs. Dots ndcte emprcl ROC curves. Sold lnes show fts (n LOW nd HIGH) nd predctons (n MIXED) of four models. Stmul were rs nd rellty ws mnpulted through contrst. () Eperment (homogeneous dstrctors), suject S.N. MIXED trls re grouped y set sze., sngle-rellty model. () Eperment 2 (heterogeneous dstrctors), suject V.N. MIXED trls re grouped y trget rellty. The elow-chnce ROC curve n the thrd pnel s result of condtonng on trget rellty. M d, sum d, L 2 nd L 4 model ROC curves re presented s n Supplementry Fgures 4, long wth the ROC curves from other sujects nd eperments 3 nd 4. specfc nonlner mnner n ths condton (equton (2)). Nonoptml models ncorporte only stmulus rellty (m d, sum d ), the comnton rule (sngle rellty) or nether (m, sum, L 2, L 4 ). We emned the re under the ROC curve n the MIXED condton (AUC), s mesured nd s predcted y ech model, for ech eperment, condtoned on trget rellty (Fg. 5 nd Supplementry Fgs. 7 ). We performed four-wy ANOVA wth fctors oserver type (dt or model), stmulus type (r or ellpse), dstrctor type (homogeneous or heterogeneous) nd trget rellty (low or hgh) on the comned AUC dt of ll s eperments. On the ss of ths nlyss, ll models esdes the optml model nd the m d model could e ruled out (P <.2; Supplementry Results nd Supplementry Tle ). It s not surprsng tht summry sttstcs such s AUC cnnot dstngush the optml model from the m d model, s the m d model provdes close ppromton to the optml model. To dstngush etween the optml nd the m d model, we performed Byesn model comprson on the rw response counts Fgure 5 Model predctons for re under the ROC curve (AUC) n the med-rellty condton. For models not shown here, see Supplementry Fgures 7 nd. () Dt (lc), nd model predctons otned from mmum-lelhood estmton n HIGH nd LOW (colored lnes), n eperments nd 3 (homogeneous dstrctors). Trget rellty s hgh (sold) or low (dshed). () Dt re presented s n for eperments 2 nd 4 (heterogeneous dstrctors). Set sze ws 4 n eperment 2 nd 2 n eperment 4. Error rs represent s.e.m. (c) Sctter plot of ctul versus predcted AUC for models nd ndvdul sujects cross ll eperments ( to 4, nd ulry eperments nd 2; see Supplementry Results). Condtonng on trget rellty produces two ponts per suject. Trget rellty c Suject s AUC Hgh Low. AUC AUC Optml model n ech response ctegory (Onlne Methods). Ths method returns the log lelhood of ech model gven sngle suject s dt; of nterest re the dfferences etween the models. We found tht the optml model ws the most lely model for ll sujects nd ll eperments. In prtculr, the log lelhood of the optml model eceeded tht of the m d model y 25.9 ± 2.2, 8.6 ±.8, 5.6 ±.6, 56 ± 2, 5.2 ±.8 nd 6 ± ponts n eperments 4, nd 2, respectvely (men ± s.e.m.). Ths consttutes decsve evdence tht the optml model etter ccounts for the dt thn the m d model (results cross ll models nd ll eperments re shown n Supplementry Tle 2, nd ndvdul-suject log lelhood dfferences re shown n Fg. 6 nd Supplementry Fg. 9c,d). Ten together, our results ndcte tht humns perform ner-optml vsul serch, regrdless of whether dstrctors re homogeneous or heterogeneous nd whether the rellty of the stmulus ws controlled y contrst or shpe. Alterntve models Eperment Eperment 3 Eperment Eperment 3.. Dt Optml Dt M Set sze Set sze Set sze Set sze ` Optml M... AUC AUC Eperment 2 Low Hgh Eperment 4 Low Hgh Trget rellty Predcted AUC Predcted AUC Predcted AUC Predcted AUC Dt Optml M Homo Hetero N = 2 N = 4 N = 6 N = VOLUME 4 NUMBER 6 JUNE 2 nture neuroscience

5 r t c l e s 2 Nture Amerc, Inc. All rghts reserved. Reltve model log lelhood Reltve model log lelhood 4 4 M M d Sum d L 2 L 4 Eperment Eperment 2 Eperment 4 Neurl mplementton Our fndng tht humns te nto ccount the relltes of ndvdul stmul on trl-y-trl ss durng vsul serch rses the queston of how ths s ccomplshed y neurons. Computng the locl decson vrle requres nowledge out the relltes, /σ 2, on sngle trl (for emple, equton (3)). How does the nervous system now these relltes? Prevous models ssumed tht the only nformton vlle to the nervous system t the th locton s the nosy sclr oservton 6,5,9,23, whch s sometmes dentfed wth the ctvty of sngle neuron 5,6. Such codng scheme cnnot possly encode oth the orentton of stmulus nd ts rellty, s sclr cnnot unmguously represent two uncorrelted qunttes. Ths s not prolem f rellty s fed cross loctons nd trls. However, we found tht humn sujects re ner-optml even n stutons n whch the rellty of the orentton vres etween loctons nd over tme, mplyng tht the neurl representton of ech stmulus contns nformton out oth orentton nd rellty. Thus, we propose tht the rn uses prolstc populton codes 3,25 to encode lelhood functons over orentton on sngle trls nd to compute the posteror prolty of trget presence. We ssume tht the orentton t the th locton s encoded n populton of neurons whose ctvty we denote y vector r (Fg. 7). On repeted presenttons of the sme orentton s, the populton pttern of ctvty wll vry. We ssume tht ths vrlty, denoted p(r s ), elongs to the Posson-le fmly of dstrutons 3, 4 Eperment 3 M M d Sum M d L 2 M d L 4 Sum d L 2 L 4 M M d Sum d L 2 L 4 Fgure 6 Log lelhood of nonoptml models reltve to the optml model for ndvdul sujects. Sujects re leled y color, seprtely for ech eperment. Negtve numers ndcte tht the optml model fts the humn dt etter. 4 p( r s, c ) = j( r, c )ep( h ( s ) r ) s (5) where ϕ s n rtrry functon nd c denotes prmeters such s locl contrst nd other mge propertes, whch cn ffect neurl ctvty nd stmulus rellty ut re unrelted to our ts-relevnt feture, orentton. The functon h (s ) s relted to the tunng curves f (s,c ) nd the covrnce mtr Σ (s,c ) 3 (see Supplementry Fg. ). Posson-le vrlty s consderly more generl thn ndependent Posson vrlty whle eng rodly consstent wth the sttstcs of neuronl responses n vvo. For gven populton ctvty r, we cn compute the lelhood functon of the stmulus, L(s ) = p(r s ), whch s proportonl to ep(h (s ) r ). The vlue of s tht mmzes L(s ), the mmum-lelhood estmte of orentton otned from r, corresponds to the oservton descred ove. The wdth of the lelhood functon, σ, ndctes the rellty of the orentton nformton t the th locton. Ths utomtc encodng of rellty on sngle trl mght e utlzed to uld networ tht ccounts for the ner-optml ehvor of the sujects. Such networ would hve to mplement the mrgnlztons over dstrctor orentton (equton (4)) nd locton (equton (2)) tht re requred to compute the log lelhood rtos of locl nd glol trget presence, respectvely. We trned three-lyer feedforwrd networ (Fg. 7) usng qudrtc nonlnerty 26,27 nd dvsve normlzton 28,29 to perform oth mrgnlztons. The nput lyer encoded the locl orenttons usng prolstc populton codes, the second lyer ws trned to compute the log lelhood rtos of locl trget presence nd the thrd lyer ws trned to compute the log lelhood rto of glol trget presence. The choce of the qudrtc nonlnerty nd dvsve normlzton ws motvted y our prevous fndng tht these types of opertons cn e used to mplement neroptml mrgnlzton over dscrete prolty dstrutons encoded wth prolstc populton codes 3. Moreover, wth ths prtculr choce of nonlnertes, ll lyers encode the log lelhood rto of trget presence wth prolstc populton codes smlr to the ones used n the nput lyer. These codes hve the dvntge tht the log lelhood rtos of locl nd glol trget presence re lnerly decodle from the second nd thrd lyers (Onlne Methods), thus smplfyng downstrem computton nd lernng. Networs were trned seprtely for the homogeneous nd heterogeneous cses. We decoded networ ctvty n the second nd thrd lyers under the ssumpton of Posson-le prolstc populton code. We epect the networ to perform optmlly only f ths ssumpton s stsfed, whch s to sy, f the log lelhood rto of trget presence s lner n the ctvty of the networ unts (equton (9)). The networ wth qudrtc nonlnerty nd dvsve normlzton c d Actvty (spes) Preferred orentton Glol trget presence Locl trget presence Stmulus orenttons R r R glol R 2 r 2 R N r N Networ posteror estmte.. QDN QUAD.5 Stmulus contrst: Low Hgh Averge contrst: Low Medum Hgh R.5. Optml posteror Networ posteror estmte.5 R.5. Optml posteror Networ posteror estmte..5 QDN R glol.5. Optml posteror Networ posteror estmte..5 QUAD R glol.5. Optml posteror Fgure 7 Neurl mplementton of ner-optml vsul serch. () Emple populton pttern of ctvty encodng orentton t one locton. Neurons re ordered y ther preferred orentton. () Networ rchtecture. (c) Posteror prolty of locl trget presence encoded n the second lyer versus the optml posteror prolty, when dstrctors re heterogeneous. Color ndctes stmulus contrst. Left, QDN networ. Rght, QUAD networ. Results for other networs nd homogeneous dstrctors re shown n Supplementry Fgures 2 nd 3. (d) Posteror prolty of glol trget presence encoded n the thrd lyer versus the optml posteror prolty. Left, QDN networ. Rght, QUAD networ. Color ndctes the verge contrst n the dsply; cross ll dsplys, the hstogrm of contrsts ws nned nto low, medum nd hgh. Results for other networs re shown n Supplementry Fgure 4. Error rs represent s.d. nture NEUROSCIENCE VOLUME 4 NUMBER 6 JUNE 2 787

6 r t c l e s 2 Nture Amerc, Inc. All rghts reserved. Detecton rte. Detecton rte..5.5 LOW HIGH MIXED N = 4 MIXED N = LOW HIGH MIXED, trget low contrst MIXED, ll MIXED N = 8 MIXED, trget hgh contrst Flse-lrm rte Fgure 8 Neurl networ reproduces humn serch performnce. ROC curves of the sme humn oservers s n Fgure 4 (dots) nd the estfttng QDN networ (lne). () Homogeneous dstrctors (eperment ). () Heterogeneous dstrctors (eperment 2). (QDN) provdes very close ppromton to the optml posteror dstruton over locl nd glol trget presence (Fg. 7c,d nd Supplementry Fgs. 2 4). Notly, when we used networ wth only qudrtc nonlnerty, ut no dvsve normlzton (QUAD), performnce degrded mredly (Fg. 7c,d). The sme held for networs wthout qudrtc opertons (Supplementry Fgs. 2 4). Ths suggests tht dvsve normlzton s needed to ensure tht the log lelhood rto s ppromtely lner n the output ctvty. In Supplementry Fgure 5, nformton loss per lyer s compred etween the four networs we tested, for oth homogeneous nd heterogeneous dstrctors. Fnlly, we used the QDN networ to generte ROC curves for homogeneous nd heterogeneous dstrctor condtons (Fg. 8). The resultng ROC curves provde ccurte fts to the ROC curves otned from the humn sujects. In short, ths networ, s wth our humn sujects, s cple of computng close ppromton of the prolty of trget presence when presented wth ny rrngement of relltes. When neurl vrlty s no longer Posson-le, the networ fls to e ner-optml (Supplementry Fg. 6). It s worth notng tht wth the prolstc code tht we used, nput-lyer neurons tht hve the trget orentton n steep regon of ther tunng curves contrute most to performnce (Supplementry Fg. ), consstent wth erler theoretcl 3 nd epermentl studes 32,33 usng dscrmnton tss ut n contrst wth the de tht the neuron est tuned to the trget orentton s most mportnt 5. Predctons Physologcl studes hve reported neurl correltes of decson confdence or certnty n lterl nterpretl corte (LIP) 34, superor collculus 35 nd ortofrontl corte 36. Consstent wth these fndngs, our results predct tht neurons est tht encode the log lelhood rto of trget presence. The solute vlue of the log lelhood rto s mesure of certnty. Clerly, ny model would predct the estence of such neurons gven our ehvorl dt, ut our prolstc populton code frmewor mes much more specfc predcton regrdng the mppng from neurl ctvty to prolty of trget presence. Specfclly, n optmzed lner decoder of the response of these neurons should e le to recover the log lelhood rto of glol trget presence s well s ny nonlner decoder. Ths s drect consequence of the fct tht n our ner-optml networ, neuronl response sttstcs re constrned to elong to the Posson-le fmly n ll lyers. Moreover, the log lelhood rto of glol trget presence should e recoverle wth sngle lner decoder of neurl ctvty regrdless of the relltes of trget nd dstrctors. These re nontrvl predctons ecuse, n generl, nonlner decoders re epected to perform etter thn lner decoders. Smlrly, fmly of lner decoders, ech speclzed for prtculr level of rellty, should, n prncple, outperform sngle lner decoder. In ths cse, however, we rgue tht nonlner decoder, or fmly of speclzed lner decoders, would not etrct sustntlly more nformton out the posteror dstruton thn sngle lner one. Anlogous predctons pply to the log lelhood rto of locl trget presence. The predcton tht the neurl code s optmlly lnerly decodle n wy tht s nvrnt to the vlue of nusnce prmeters such s contrst hs very drect mplctons for downstrem processng, s t smplfes the neurl mplementton of other prolstc nferences. For nstnce, wth such code, the mmum-lelhood estmte cn e etrcted through ttrctor dynmcs 37 nd optml cue ntegrton cn e performed through fed lner comnton of neurl ctvty 3. So fr, we hve not specfed where one mght epect to fnd the neurons whose response cn e mpped lnerly onto the log lelhood rto of trget presence. Although n etensve lterture ests on the neurl ss of vsul serch nd ttenton, we re not wre of ny studes tht hve recorded neuronl ctvty n sngle-feture serch ts wth short presentton tmes nd sensory uncertnty. Nevertheless, good cnddte regons would e re V4, nferor temporl corte nd LIP, n whch strong ttentonl modulton hs een reported. In multdmensonl serch ts, V4 neurons hd hgher response to trget thn to dstrctor, regrdless of the feture dmenson n whch the trget ws defned 38. The responses of V4 nd nferor temporl corte neurons contn nformton out whch of two stmul mtches memorzed cue 39. LIP neurons respond more strongly when n tem n ther receptve feld s serch trget thn when t s not 4. A fnl predcton concerns the contrst response functon of the neurons nvolved n the mrgnlzton over locton when rellty s controlled y contrst. We clm tht ths mrgnlzton s mplemented usng set of ss functons, some of whch comne ctvty from the second lyer through qudrtc opertons wth dvsve normlzton. One cn thn of these ss functons s neurons n n ntermedte lyer (etween the second nd the thrd) tht comne nformton from multple loctons. Consder stuton n whch the set sze s 2, tht s, two rs pper n the receptve feld of such neuron. The response of ths neuron to the contrst of the two rs should follow the qudrtc dvsve normlzton equton. Thus, f the contrst of the second r s held constnt, then the neurl response should sturte wth ncresng contrst of the frst r. Moreover, t should sturte t level tht s monotonclly relted to the contrst of the second r. Ths predcton s ts specfc: for emple, n our prevous wor on optml multsensory ntegrton 3, we predcted tht cells comne ther multsensory nputs lnerly. There, the prolstc operton needed ws product of dstrutons, not mrgnlzton. DISCUSSION Serchng effcently for trget mdst dstrctors s crucl for n orgnsm s survvl. Ths ts s chllengng ecuse the rellty of sensory nformton my vry unpredctly cross spce nd tme. Whether nd how humns te nto ccount vryng rellty n nd cross dsplys s n mportnt queston from ehvorl, computtonl nd neurl perspectve. Studes testng the noton of percepton s optml nference hve concentrted on smple tss such s comnng cues out sngle physcl stmulus vrle. To te ths pproch to the net level, t s mportnt to consder perceptul tss 788 VOLUME 4 NUMBER 6 JUNE 2 nture neuroscience

7 r t c l e s 2 Nture Amerc, Inc. All rghts reserved. wth more comple genertve models, such s vsul serch. As vsul serch s ts wth herrchcl structure, optml serch requres mrgnlzton, computton tht s uqutous n nturlstc vsul envronments ut remns understuded n psychophyscs 4,42. We found tht humn oservers te nto ccount rellty on n tem-y-tem nd trl-y-trl ss durng vsul serch nd cn comne nformton cross loctons through mrgnlzton. These results were consstent whether we mnpulted the rellty of the tems v chnges of ther contrst or shpe. Ths ndctes tht humn sujects encode prolty dstrutons over stmul, rther thn pont estmtes, nd tht they use these dstrutons to compute the prolty of trget presence. Humn ner-optmlty n judgng n strct, ctegorcl vrle such s trget presence provdes evdence for the generlty of humn lty to compute wth prolty dstrutons. Eplorng comple genertve models cn contrute to shftng the dscourse on optmlty n percepton towrd the queston of whch ts fctors mght cuse performnce to e suoptml. It s lely tht greter devtons from optmlty wll e found n tss tht hve more nodes n ther genertve model or tht re less ecologclly relevnt. We determned how neurl crcuts could mplement ner-optml vsul serch usng prolstc populton codes nd ologclly plusle opertons, nmely qudrtc nonlnerty wth dvsve normlzton. Ths codng scheme llows neurl networ to te nto ccount rellty wthout requrng seprte crcut to represent ths rellty. Moreover, we predct tht the nterctons mplementng ner-optml vsul serch re dfferent etween homogeneous nd heterogeneous dstrctor dstrutons; n the former, lner neurl opertons re suffcent to optmlly compute locl trget presence, wheres n the ltter, nonlner opertons re needed. We predct tht, under oth dstrutons, dvsve normlzton s n mportnt operton n computng glol trget presence. Ths s nterestng n lght of the proposl tht dvsve normlzton mght hve crucl role n the neurl ss of ttenton 29. Our results ndcte tht the sme nonlnerty mght epln how humns cn e ner-optml n the ttentonl ts of serchng for trget mong dstrctors. It would e worthwhle to revst feture-sed ttenton studes from ths ner-optmlty perspectve. Our wor s relted to prevous studes of vsul serch under sensory nose. An nfluentl study emned eye movements n serch scenes corrupted y pel nose nd found tht, on verge, oservers choose ther net fton locton ccordng to the mmum prolty of dentfyng the trget 7. Ths wor, however, dd not ddress the dffcult ssue of comnng nformton cross sptl loctons through mrgnlzton. One of us hs prevously rgued tht slency-sed 43 sgnl-to-nose frmewor cn epln ottom-up nd top-down effects on serch dffculty n vrous dstrctor condtons etter thn sgnl-detecton theory model 44. However, the slency-sed model ws not prolstc nd could not esly represent stmulus uncertnty; moreover, the sgnl detecton theory model used ws fr from optml. Here the locl lelhood rto of trget presence ws computed from ts-specfc prolty dstrutons nd cn therefore e regrded s form of top-down slency. Ths s smlr to recent model tht defned slency s the posteror prolty of locl trget presence 45. However, ths wor consdered the scenro of trget presence eng ndependent cross loctons (there cn thus e ny numer of trgets) nd susequently focused on feture prors durng free vewng. Fnlly, t hs een found tht when the dstrctor dstruton s vred etween locs, m rule ppled to locl log lelhoods etter ccounts for humn ehvor thn mmum-of-outputs rule 46. Ths study, however, dd not test the optml rule, equton (2), or vry the relltes of the stmul on trl-y-trl ss. Chngng rellty on trl-y-trl ss, s we hve done, mes the ts consderly more dffcult, s the rellty of the stmul must now e ten nto ccount on the fly. The formlsm tht we used pples to tss n whch stmulus presentton tme s short, t most one trget s present nd oservers report trget presence nsted of trget locton. Nonetheless, our frmewor cn e etended to recton-tme procedures y comnng t wth optml evdence ccumulton 47, trget detecton n the presence of multple trgets y replcng the sum n equton (2) y sum over susets nd trget loclzton y computng the posteror p(t =,T = ) nsted of p(t = ). We hope tht our wor wll fcltte rgorous tests of optmlty n vrety of serch tss. Fnlly, lthough we focused on prolstc populton codng, t s qute possle tht ner-optml vsul serch could e mplemented wth other types of neurl codes for prolty dstrutons. Implementng vsul serch wth ny scheme s nontrvl ts s t requres tclng the dffcult prolem of mrgnlzton. Nonetheless, such etensons would e nvlule s they cn led to epermentl predctons tht would llow us to dstngush etween lterntve theores of neurl codng. Methods Methods nd ny ssocted references re vlle n the onlne verson of the pper t Note: Supplementry nformton s vlle on the Nture Neuroscence weste. Acnowledgments W.J.M. s supported y wrd REY2958 from the Ntonl Eye Insttute. V.N. s supported y Ntonl Scence Foundton grnt # J.M.B. s supported y the Gtsy Chrtle Foundton nd R.v.d.B. y the Netherlnds Orgnzton for Scentfc Reserch (NWO). A.P. s supported y Multdscplnry Unversty Reserch Inttve grnt N , Ntonl Insttute on Drug Ause grnt #BCS346785, reserch grnt from the Jmes S. McDonnell Foundton nd wrd P3EY39 from the Ntonl Eye Insttute. AUTHOR CONTRIBUTIONS W.J.M., V.N. nd R.v.d.B. desgned the eperments. V.N. nd R.v.d.B. collected the dt. W.J.M., V.N. nd R.v.d.B. nlyzed the dt. W.J.M., J.B. nd A.P. developed the theory. J.B. performed the networ smultons. W.J.M. nd A.P. wrote the mnuscrpt. V.N., J.B. nd R.v.d.B. contruted to the wrtng of the mnuscrpt. COMPETING FINANCIAL INTERESTS The uthors declre no competng fnncl nterests. Pulshed onlne t Reprnts nd permssons nformton s vlle onlne t reprnts/nde.html.. Plmer, J., Ames, C.T. & Lndsey, D.T. Mesurng the effect of ttenton on smple vsul serch. J. Ep. Psychol. Hum. Percept. Perform. 9, 8 3 (993). 2. Tresmn, A.M. & Gelde, G. A feture-ntegrton theory of ttenton. Cognt. Psychol. 2, (98). 3. Estes, W.D. & Tylor, R.M. A detecton method nd prolstc models for ssessng nformton processng from ref vsul dsplys. Proc. Ntl. Acd. Sc. USA 52, (964). 4. Shw, M.L. Identfyng ttentonl nd decson-mng components n nformton processng. n Attenton nd Performnce (ed. R.S. Ncerson) (Erlum, Hllsdle, New Jersey, 98). 5. Techner, W.H. & Kres, M.J. Vsul serch for smple trgets. Psychol. Bull. 8, 5 28 (974). 6. Plmer, J., Verghese, P. & Pvel, M. The psychophyscs of vsul serch. Vson Res. 4, (2). 7. Duncn, J. & Humphreys, G.W. Vsul serch nd stmulus smlrty. Psychol. Rev. 96, (989). 8. Rosenholtz, R. Vsul serch for orentton mong heterogeneous dstrctors: epermentl results nd mplctons for sgnl detecton theory models of serch. J. Ep. Psychol. Hum. Percept. Perform. 27, (2). nture NEUROSCIENCE VOLUME 4 NUMBER 6 JUNE 2 789

8 r t c l e s 2 Nture Amerc, Inc. All rghts reserved. 9. Ersen, C.W. Oject locton n comple perceptul feld. J. Ep. Psychol. 45, (953).. Frmer, E.W. & Tylor, R.M. Vsul serch through color dsplys: effects of trgetcground smlrty nd cground unformty. Percept. Psychophys. 27, (98).. Knll, D.C. & Rchrds, W. Percepton s Byesn Inference (Cmrdge Unversty Press, New Yor, 996). 2. Knll, D.C. & Pouget, A. The Byesn rn: the role of uncertnty n neurl codng nd computton. Trends Neurosc. 27, (24). 3. M, W.J., Bec, J.M., Lthm, P.E. & Pouget, A. Byesn nference wth prolstc populton codes. Nt. Neurosc. 9, (26). 4. Morgn, M.L., DeAngels, G.C. & Angel, D.E. Multsensory ntegrton n mcque vsul corte depends on cue rellty. Neuron 59, (28). 5. Verghese, P. Vsul serch nd ttenton: sgnl detecton theory pproch. Neuron 3, (2). 6. Ecsten, M.P., Peterson, M.F., Phm, B.T. & Droll, J.A. Sttstcl decson theory to relte neurons to ehvor n the study of covert vsul ttenton. Vson Res. 49, (29). 7. Njemn, J. & Gesler, W.S. Optml eye movement strteges n vsul serch. Nture 434, (25). 8. Ecsten, M.P., Thoms, J.P., Plmer, J. & Shmoz, S.S. A sgnl detecton model predcts the effects of set sze on vsul serch ccurcy for feture, conjuncton, trple conjuncton, nd dsjuncton dsplys. Percept. Psychophys. 62, (2). 9. Green, D.M. & Swets, J.A. Sgnl Detecton Theory nd Psychophyscs (John Wley & Sons, Los Altos, Clforn, 966). 2. Peterson, W.W., Brdsll, T.G. & Fo, W.C. The theory of sgnl detectlty. IRE Prof. Group Inf. Theory 4, 7 22 (954). 2. Nolte, L.W. & Jrsm, D. More on the detecton of one of M orthogonl sgnls. J. Acoust. Soc. Am. 4, (967). 22. Ecsten, M.P. The lower vsul serch effcency for conjunctons s due to nose nd not serl ttentonl processng. Psychol. Sc. 9, 8 (998). 23. Grhm, N., Krmer, P. & Yger, D. Sgnl decton models for multdmensonl stmul: prolty dstrutons nd comnton rules. J. Mth. Psychol. 3, (987). 24. Quc, R.F. A vector-mgntude model of contrst detecton. Kyernet 6, (974). 25. Pouget, A., Dyn, P. & Zemel, R.S. Inference nd Computton wth Populton Codes. Annu. Rev. Neurosc. 26, 38 4 (23). 26. Bremmer, F., Ilg, U., Thele, A., Dstler, C. & Hoffmn, K. Eye poston effects n money corte. I. Vsul nd pursut-relted ctvty n etrstrte res MT nd MST. J. Neurophysol. 77, (997). 27. Andersen, R.A., Essc, G. & Segel, R. Encodng of sptl locton y posteror pretl neurons. Scence 23, (985). 28. Heeger, D.J. Normlzton of cell responses n ct strte corte. Vs. Neurosc. 9, 8 97 (992). 29. Reynolds, J.H. & Heeger, D.J. The normlzton model of ttenton. Neuron 6, (29). 3. Bec, J., Lthm, P. & Pouget, A. Comple Byesn nference n neurl crcuts usng dvsve normlzton. Front. Syst. Neurosc. Conference Astrct: Computtonl nd Systems Neuroscence 29, do:.3389/conf.neuro (2 Ferury 29). 3. Seung, H.S. & Sompolnsy, H. Smple model for redng neuronl populton codes. Proc. Ntl. Acd. Sc. USA 9, (993). 32. Schoups, A., Vogels, R., Qn, N. & Orn, G. Prctsng orentton dentfcton mproves orentton codng n V neurons. Nture 42, (2). 33. Regn, D. & Beverley, K.I. Sptl frequency dscrmnton nd detecton: comprson of postdptton thresholds. J. Opt. Soc. Am. 73, (983). 34. Kn, R. & Shdlen, M.N. Representton of confdence ssocted wth decson y neurons n the pretl corte. Scence 324, (29). 35. Km, B. & Bsso, M.A. Sccde trget selecton n the superor collculus: sgnl detecton theory pproch. J. Neurosc. 28, (28). 36. Kepecs, A., Uchd, N., Zrwl, H.A. & Mnen, Z.F. Neurl correltes, computton nd ehvourl mpct of decson confdence. Nture 455, (28). 37. Deneve, S., Lthm, P. & Pouget, A. Redng populton codes: neurl mplementton of del oservers. Nt. Neurosc. 2, (999). 38. Ogw, T. & Komtsu, H. Trget selecton n re V4 durng multdmensonl vsul serch ts. J. Neurosc. 24, (24). 39. Bchot, N.P., Ross, A.F. & Desmone, R. Prllel nd serl neurl mechnsms for vsul serch n mcque re V4. Scence 38, (25). 4. Gottle, J.P., Kusuno, M. & Golderg, M. The representton of vsul slence n money pretl corte. Nture 39, (998). 4. Knll, D.C. Mture models nd the prolstc structure of depth cues. Vson Res. 43, (23). 42. Kersten, D., Mmssn, P. & Yulle, A. Oject percepton s Byesn nference. Annu. Rev. Psychol. 55, (24). 43. Itt, L. & Koch, C. Computtonl modelng of vsul ttenton. Nt. Rev. Neurosc. 2, (2). 44. Nvlpm, V. & Itt, L. Serch gol tunes vsul fetures optmlly. Neuron 53, (27). 45. Zhng, L., Tong, M.H., Mrs, T.K., Shn, H. & Cottrell, G.W. SUN: Byesn frmewor for slency usng nturl sttstcs. J. Vs. 8, 2 (28). 46. Vncent, B.T., Bddeley, R.J., Troscno, T. & Glchrst, I.D. Optml feture ntegrton n vsul serch. J. Vs. 9, (29). 47. Bec, J.M. et l. Byesn decson-mng wth prolstc populton codes. Neuron 6, (28). 79 VOLUME 4 NUMBER 6 JUNE 2 nture neuroscience

9 2 Nture Amerc, Inc. All rghts reserved. ONLINE METHODS Eperment : mnpultng r contrst, homogeneous dstrctors. Sujects vewed the dsply (28 y 2 deg) of dmensons.3 y.8 deg (the tems) on 2-nch CRT montor wth refresh rte of 2 Hz nd cground lumnnce of 8.8 cd m 2 from dstnce of 85 cm. Trget nd dstrctor orenttons were 7 nd 6, respectvely. Items were dsplyed n one of eght possle postons (correspondng to compss drectons), eqully spced round n mgnry crcle of rdus 7 deg, centered t centrl fton cross. Item postons were rndomly chosen for set szes 4 or 6. Ech tem poston hd unform rndom jtter of up to.6 deg n oth nd y drectons. Item contrst ws ether 67% (hgh) or 2 or 7% (low), ccordng to the suject. In the LOW nd HIGH condtons, set sze ws 8 nd ll tems hd the sme contrst (low or hgh, respectvely). In the MIXED condton, set sze ws 4, 6 or 8 (ntermed n pseudorndom order) nd the contrst of ech tem ws rndomly nd ndependently set to hgh or low. On ech trl, the prolty tht the dsply contned trget ws.5; sujects were nformed of ths n dvnce. If the trget ws present, ts locton ws chosen rndomly. Ech trl egn wth fton (25 ms), followed y the serch dsply, followed y ln screen untl the suject responded. Sujects gve yes/no response out trget presence, followed y confdence rtng on scle of to 3 ( = lest confdent, 3 = most confdent). Sujects were encourged to spred ther responses cross the rtngs. Sujects prctced for 3 trls n the HIGH condtons wth 2-ms stmulus, nd they receved correctness feedc t the end of ech trl. After prctce, sujects performed vrle numer of 3-trl locs n the HIGH condton strtng wth 2-ms stmulus. Stmulus durton decresed y 25 ms on every susequent loc, untl ccurcy ws 85 9%. The resultng stmulus durtons of 5 or 75 ms were susequently used throughout the eperment. Sujects then performed two 2-trl locs n the HIGH condton wth 67% contrst tems. Net, stmulus contrst (dentcl for ll tems n dsply) decresed n 3- trl locs y 5% on every susequent loc untl ccurcy reched ws 6 to 65%. The resultng contrst ws used s low contrst. The suject then performed two 2-trl locs n the low condton, followed y prctce on 3 trls n the med condton (wthout feedc) nd s 2-trl locs n the med condton. The eperments were conducted over two sessons on consecutve dys. All stmul were controlled usng MATLAB (MthWors) wth the Psychophyscs Toolo 48. Four sujects (two uthors, two nve) prtcpted. Informed wrtten consent ws otned from ll sujects. Eperment 2: mnpultng r contrst, heterogeneous dstrctors. Ths eperment ws dentcl to Eperment, ecept for the followng dfferences. Set sze ws 4 nd trget orentton ws 6. Dstrctor orentton ws rndomly smpled from ll multples of 9 wy from the trget. Stmul were plced t every other possle locton from Eperment, strtng t northest. Ech suject performed two 2-trl locs n the LOW (L), HIGH (H) nd MIXED (M) condtons, n the order HHLLMM. Hgh nd low contrsts were 6% nd 6%, respectvely. Four sujects (one uthor, three nve) prtcpted. Eperments 3 nd 4: mnpultng ellpse eccentrcty. Eperments 3 nd 4 were dentcl to eperments nd 2, ecept for the followng dfferences. Sujects vewed the dsply on 9-nch LCD montor wth refresh rte of 6 Hz nd cground lumnnce of 34 cd m 2, from dstnce of 6 cm. Items were ellpses; ellpse orentton ws defned y the orentton of the long s. The re of ech ellpse ws.24 deg 2 nd rellty ws mnpulted through ellpse eccentrcty (elongton). Trget orentton ws 45. In eperment 3, dstrctors were homogeneous wth n orentton chosen per suject ( 3, 25, 5 ) to ensure tht symptotc ccurcy t hgh eccentrcty eceeded 87.5%. In eperment 4, dstrctors were heterogeneous nd drwn from unform dstruton. The set sze ws 2 or 4 (seprte sessons) n eperment 3 nd 2 n eperment 4. When the set sze ws 2, stmul were plced t the northwest nd southest loctons. When the set sze ws 4, stmul were plced t these loctons nd lso t northest nd southwest. Stmulus durton ws 3 ms durng prctce nd 66.7 ms durng testng. Sujects performed three types of locs: prctce, threshold mesurement nd testng. Durng prctce, stmulus durton ws 3 ms, stmul were of med rellty, sujects dd not report confdence, trl-to-trl correctness feedc ws gven y refly colorng the fton cross green or red nd n mge of the trget wth eccentrcty.9 ws dsplyed t the locton of the fton cross for 5 ms efore the strt of new trl. Durng threshold mesurement locs, psychometrc curve (percentge correct versus ellpse eccentrcty) ws mpped out to determne 62.5% nd 87.5% thresholds t N = 4 (eperment 3) or N = 2 (eperment 4). These eccentrctes provded the low nd hgh vlues of eccentrcty throughout the eperment. In eperment 3, (low, hgh) eccentrcty prs were (.66,.82) for suject R.B., (.65,.92) for suject W.M. nd (.6,.78) for suject E.A. In eperment 4, they were (.5,.82) for suject R.B., (.56,.93) for suject W.M. nd (.47,.9) for suject E.A. Eperment 3 conssted of three sessons. In the frst sesson (N = 4), sujects performed -trl prctce loc, two 5-trl threshold mesurement locs nd three 5-trl testng locs n the order HML. In the second sesson (N = 4), sujects performed -trl prctce loc nd fve 5-trl testng locs n the order MMLMH. In the thrd sesson (N = 2), sujects performed -trl prctce loc, followed y four 5-trl locs n the med condton. Eperment 4 conssted of two sessons, orgnzed n the sme wy s the frst two sessons of eperment 3. The sme three sujects (two uthors, one nve) prtcpted n eperments 3 nd 4. Eperments nd 2: mnpultng r contrst t N = 2. Eperments (homogeneous dstrctors) nd 2 (heterogeneous dstrctors) were dentcl to eperments nd 2, respectvely, ecept for the followng dfferences. Set sze ws lwys 2. Three sujects (two of them uthors) prtcpted n oth eperments. Bcground lumnnce ws 95 cd m 2. Ech tem ws n orented r of dmensons.3 y.8 deg. Trget orentton ws 45 nd dstrctor orentton ws 35 for RB nd 3 for sujects W.M. nd S.K. These vlues were chosen to ensure tht for ech suject, symptotc ccurcy t hgh contrst eceeded 87.5%. The procedure conssted of prctce, threshold mesurement nd testng nd ws nlogous to tht of eperment 4. Durng threshold mesurement, psychometrc curve ws mpped out to determne 62.5% nd 87.5% thresholds. These contrsts provded the low nd hgh vlues of rellty throughout the eperment. In eperment, these prs were (5.6%, 2%) for suject R.B., (3.8%, 7.8%) for suject S.K. nd (4%, %) for suject W.M. In eperment 2, they were (4.8%, 4%) for suject R.B., (2.9%, 9.6%) for suject S.K. nd (4%, 2%) for suject W.M. Stmulus durton ws 3 ms durng prctce nd 33 ms durng threshold mesurement nd testng. Model predctons. An epermentl condton s comnton of trget presence, set sze nd rellty condton (LOW, MIXED, HIGH). Genertng model predctons for ech condton conssted of choosng prmeter vlues, smultng oservtons for, trls usng those prmeters nd lettng the model me decsons on the smulted oservtons. In eperments nd 3, the model prmeters (denoted θ), were σ low, σ hgh nd fve decson crter (n ll models), s well s σ ssumed (n the sngle-rellty model). Internl representtons were drwn ndependently from norml dstrutons wth dentcl vrnces, σ 2, nd mens of (trget) nd (dstrctor). Here, σ s ether σ low or σ hgh. In eperments 2 nd 4, the nternl representton of stmulus ws drwn from Von Mses dstruton centered t the stmulus orentton nd wth concentrton prmeter κ (ether κ low or κ hgh ). In the m, sum, L 2 nd L 4 models, the locl decson vrle ws ten to e the output of Posson neuron wth Von Mses tunng curve tht responded most strongly to the trget orentton: r = Posson(g ep(κ tc cos(2(s s T ))) + ), where g s ether g low or g hgh (see Supplementry Results). We chose κ tc =.5 nd = 5 nd verfed tht our results were nsenstve to ths choce of prmeters. Ths suoptml locl decson vrle s consstent wth erler proposls 5,6 nd, unle, respects the crculrty of orentton spce. On ech smulted trl, gven model nfers whether the trget s present y computng glol decson vrle d from locl decson vrles {d }, nd those n turn from the oservtons { } or {r }. The locl decson vrle of the optml, sngle-rellty, m d nd sum d models n eperments 2 nd 4 s gven y d = log I (κ ) + κ cos(2( s T )), where I s the modfed Bessel functon of the frst nd of order (see Supplementry Results). In the sme four models, the vlue of the prmeter σ (or g or κ ) used n the decson vrle equls the vlue used n the genertve model (for emple, σ low or σ hgh ), ecept n the MIXED condton of the sngle-rellty model, n whch ech σ (or κ ) s equl to σ ssumed (or κ ssumed ). All models contn σ low nd σ hgh (or κ low nd κ hgh ) s free prmeters, lthough the decson vrles of some models (sngle rellty, m, sum, L 2, L 4 ) do not contn those prmeters, s these prmeters determne the dstrutons of oservtons nd re therefore necessry n the step of smultng those. do:.38/nn.284 nture NEUROSCIENCE