Motion illusions as optimal percepts

Transcription

1 Motion illusions s optiml perepts Yir Weiss 1, Eero P. Simonelli 2 nd Edwrd H. Adelson 3 1 Shool of Computer Siene nd Engineering, Herew University of Jeruslem, Givt Rm Cmpus, Jeruslem 9194, Isrel 2 Howrd Hughes Medil Institute, Center for Neurl Siene nd Cournt Institute of Mthemtil Sienes, New York University, 4 Wshington Ple, New York, New York 13, USA 3 Brin nd Cognitive Sienes Deprtment, Msshusetts Institute of Tehnology, 77 Msshusetts Ave, Cmridge, Msshusetts 2139, USA Correspondene should e ddressed to Y.W. (yweiss@s.huji..il) Pulished online: My 2, DOI: 1.138/nn858 The pttern of lol imge veloities on the retin enodes importnt environmentl informtion. Although humns re generlly le to extrt this informtion, they n esily e deeived into seeing inorret veloities. We show tht these illusions rise nturlly in system tht ttempts to estimte lol imge veloity. We formulted model of visul motion pereption using stndrd estimtion theory, under the ssumptions tht (i) there is noise in the initil mesurements nd (ii) slower motions re more likely to our thn fster ones. We found tht speifi instntition of suh veloity estimtor n ount for wide vriety of psyhophysil phenomen. The humn ility to nlyze visul motion in generl senes fr exeeds the pilities of the most sophistited omputer vision lgorithms. Yet psyhophysil experiments show tht humns lso mke some puzzling mistkes, misjudging speed or diretion of very simple stimuli. In this pper, we propose tht suh mistkes of humn motion pereption represent the est solution of rtionl system designed to operte in the presene of unertinty. In oth iologil nd rtifiil vision systems, motion nlysis egins with lol mesurements suh s the output of diretion-seletive ells in primry visul ortex 1, or of sptil nd temporl derivtive opertors in rtifiil systems 2,3. These re then integrted to generte lrger, more glol motion desriptions. The integrtion proess is essentil euse the initil lol motion mesurements re miguous. For exmple, in the viinity of ontour, only the motion omponent perpendiulr to the ontour n e determined ( phenomenon referred to s the perture prolem ) 2,4 7. Suh n integrtion stge seems to e onsistent with muh of the psyhophysil 8 11 nd physiologil 8,12 14 dt. Despite the vst mount of psyhophysil dt pulished over the pst two dedes, the nture of the integrtion sheme underlying humn motion pereption remins unler. This is true even for the simple nd widely studied plid stimulus, in whih two superimposed oriented grtings trnslte (move without hnging shpe, size or orienttion) in the imge plne (Fig. 1). Due to the perture prolem, eh grting s motion is onsistent with n infinite numer of possile trnsltionl veloities lying on onstrint line in the spe of ll veloities (Fig. 1). When viewing single drifting grting in isoltion, sujets typilly pereive it s trnslting in diretion norml to its ontours (Fig. 1). When two grtings re presented simultneously, sujets often pereive them s oherent pttern trnslting with single motion 5,7. How is this oherent pttern motion estimted? Most explntions re sed on one of three rules 7 : intersetion of onstrints (), vetor verge (), or feture trking (FT). The solution is the unique trnsltion vetor onsistent with the informtion of oth grtings. Grphilly, this orresponds to the point in veloity spe tht lies t the intersetion of oth onstrint lines (Fig. 1, irle). The solution is the verge of the two norml veloities. Grphilly, this orresponds to the point in veloity spe tht lies hlfwy etween the two norml veloities (Fig. 1, squre). An FT solution orresponds to the veloity of some feture of the plid intensity pttern (for exmple, the lotions of mximum luminne t the grting intersetions) 15,16. For plids, the FT nd solutions oth orrespond to the veridil (true) pttern motion. Whih of the three rules est desries humn pereption? The nswer is not ler: depending on the stimulus, the pereived pttern motion n e nerly veridil (onsistent with or FT) or loser to the solution. The relevnt stimulus fetures inlude reltive grting orienttion nd speed 17 19, ontrst, presenttion time 17 nd retinl lotion 17. Similr effets hve een reported with stimuli tht pper quite different from plids 16,21. For moving rhomus (Fig. 2), s for plid pttern, the motion of eh opposing pir of sides is onsistent with onstrint line in the spe of veloities. As shown in the veloity spe digrms (Fig. 2 nd f), or FT predits horizontl motion, wheres predits digonl motion. Pereptully, however, the rhomus ppers to move horizontlly t high ontrst nd digonlly t low ontrst. To further omplite the sitution, the perept depends on the shpe. If the rhomus is fttened (Fig. 2d), it ppers to move horizontlly t oth ontrsts. To view these moving stimuli, see One might reson tht the visul system uses for thin, low-ontrst rhomus, nd /FT for thin, high-ontrst rhomus nd for ft rhomus. Although model sed on this d ho omintion of rules ertinly fits the dt, it is lerly not prsimonious explntion. Furthermore, eh of the idelized rules is limited to stimuli ontining stright strutures t only two orienttions, nd does not offer method for omputing the norml veloities of those strutures. One would prefer single, oherent model tht ould predit the pereived 598 nture neurosiene volume 5 no 6 june 2

2 Fig. 1. Intersetion of onstrints. () Drifting grtings superimposed in the imge plne produe trnslting plid pttern. () Dotted lines indite onstrint lines; rrows indite pereived diretion of grting viewed in isoltion. The solution (irle) is the unique veloity onsistent with the onstrint lines of oth grtings. The solution (squre) is the verge of the two norml veloities. There is experimentl evidene for oth types of omintion rule. veloity of ny ritrry sptiotemporl stimulus tht ppers to e trnslting. We hve developed suh model sed on simple formultion of the prolem of veloity estimtion nd on few resonle ssumptions. In Helmholtz s view, our perepts re our est guess s to wht is in the world, given oth sensory dt nd prior experiene 22. To mke this definition more quntittive, one must speify (i) wht is est out est guess, nd (ii) the wy in whih prior experiene should influene tht guess. In the engineering literture, the theory of estimtion formlizes these onepts. The simplest nd most widely known estimtion frmework is sed on Byes rule (see ref. 23 for exmples of Byesin models in pereption nd refs. 24 nd 25 for Byesin motion models). Following n pproh desried in previous work 26 29, we developed n optiml Byesin estimtor (known s n idel oserver in the psyhophysis literture) for two-dimensionl veloity. Here, s in most studies of the perture prolem, we onsidered only ses in whih humns see single glol trnsltionl motion (no deformtion, rottion, olusion oundries, trnspreny, or the like). Elsewhere, we hve developed extensions of this model tht n hndle more omplited senes 29. Our model egins with the stndrd priniple of intensity onservtion: it ssumes tht ny hnges in imge intensity over time re due entirely to trnsltionl motion of the intensity pttern. We then mde two si ssumptions: (i) lol imge mesurements re noisy nd (ii) imge veloities tend to e slow. We formulted these ssumptions using proility distriutions (see elow), nd used Byes rule to derive the idel oserver (for further mthemtil detils, see Methods). We instntited the first ssumption using noise model ommonly used in engineering euse of the trtility of the solution: mesurements re ontminted with dditive, independent, Gussin noise with known stndrd devition (σ). Although this simple noise model is unlikely to e orret in detil, we show tht it is suffiient to ount for muh of the dt. This noise model provides funtionl form for the lol likelihood: distriution over the spe of veloities tht is sed on mesurements mde in lol imge pth. We depited this likelihood s gry-level imge (Fig. 3) in whih intensity orresponds to proility. For pthes ontining single edge, the likelihood funtion is similr to fuzzy onstrint line veloities on the onstrint line hve the highest likelihood, nd the likelihood dereses with distne from the line. The fuzziness of the onstrint line is governed y σ, the stndrd devition of the ssumed noise. At orners, where lol motion mesurements re less miguous, the likelihood no longer hs the elongted shpe of onstrint line ut eomes tightly lustered round the veridil veloity. This model of dditive Gussin noise lso resulted in dependene of the likelihood on ontrst. For fixed vlue of σ, the likelihoods were roder t low ontrst (Fig. 3, ottom). This mkes intuitive sense: t low ontrst there is less informtion out the ext speed of the stimulus, nd therefore more lol unertinty, so the likelihood is more spred out. In the extreme se of zero ontrst, the unertinty is infinite. The seond ssumption underlying our idel oserver model is tht veloities tend to e slow. Suggestions tht humn oservers prefer the shortest pth, or slowest motion onsistent with the visul input, dte k to the eginning of the th entury (see ref. 3 nd referenes therein). In prtiulr, Wllh suggested tht humns prefer to see the norml veloity for single line segment euse tht is the slowest veloity onsistent with the imge dt 5. Likewise in pprent motion displys, humns tend to hoose the shortest pth or slowest motion tht would explin the inoming informtion. We formlized this preferene for slow speeds using prior proility distriution on the two-dimensionl spe of veloities tht is Gussin nd entered on the origin. Aording to this prior, in the sene of ny imge dt, the most prole veloity is zero (no motion), nd slower veloities re generlly more likely to our thn fst ones. As with the noise model, we hve no diret evidene (either from first priniples or from empiril mesurements) tht this ssumption is orret. We will show, however, tht it is suffiient to ount qulittively for muh of the pereptul dt. Under the Byesin frmework, the perept of the idel oserver is sed on the posterior proility (the proility of veloity given the imge mesurements), whih is om- Fig. 2. Insuffiieny of either, or FT rules s n explntion for humn pereption of horizontlly moving rhomus. () A nrrow rhomus t high ontrst ppers to move horizontlly (onsistent with /FT). () A nrrow rhomus t low ontrst ppers to move digonlly (onsistent with ). () Veloity spe onstrints for nrrow rhomus. (d,e) A ft rhomus t low or high ontrst ppers to move horizontlly (onsistent with /FT). (f) Veloity spe onstrints for ft rhomus. d e f nture neurosiene volume 5 no 6 june 2 599

3 Stimulus Likelihood t lotion Likelihood t lotion Likelihood t lotion puted from the likelihood nd prior using Byes rule (see Methods). We formulted the posterior distriution y multiplying the prior nd the likelihoods t ll imge lotions. This is orret under the ssumptions tht the noise in the mesurements is sttistilly independent, nd tht the likelihoods eing multiplied orrespond to imge lotions tht re moving t the sme veloity. One n lulte the veloity estimte (v ) of the idel oserver s the men or mximum of the posterior distriution. Our posterior distriution is Gussin, nd the men (whih is lso the most likely) veloity ws omputed nlytilly using the following mtrix eqution: v * = Σ I 2 σ x + σ Σ I x I y 2 p 2 Fig. 3. Likelihood funtions for three lol pthes of horizontlly trnslting dimond stimulus, omputed using eqution (4). Intensity orresponds to proility. Top, high-ontrst sequene. Bottom, low-ontrst sequene, with the sme prmeter σ. At edges, the lol likelihood is fuzzy onstrint line; t orners, the lol likelihood peks round the veridil veloity. The shrpness of the likelihood dereses with deresing ontrst. Σ I x I y Σ I 2 σ y + 2 σ p 2 where I x, I y, I t refer to the sptil (two dimensions) nd temporl derivtives of the imge sequene. The sums were tken over ll lotions tht trnslte together (here, we ssumed this inluded the entire imge). This eqution llowed us to predit the idel oserver s veloity estimte for ny imge sequene. The solution of eqution(1) hs only one free prmeter: the rtio of σ to σ p. Chnging oth of these while holding the rtio onstnt hnges the width, ut not the pek, of the posterior. We lulted the posterior for the moving rhomus stimuli (Fig. 4 ), holding the free prmeter (σ/σ p ) onstnt. Consistent with humn dt, the idel oserver predits horizontl motion for nrrow, high-ontrst rhomus, digonl motion for nrrow, low-ontrst rhomus nd nerly horizontl motion for ft, low-ontrst rhomus. For more quntittive omprison of the idel oserver nd humn pereption, we showed three sujets ontinuum of low-ontrst rhomuses tht vried etween the extremes of thin nd ft, nd sked them to report the pereived diretion y positioning the ursor 1 Σ I x I t Σ I y I t (1) of omputer mouse. The preditions of eqution (1) provide n exellent fit to the humn experimentl dt (Fig. 4d). In ddition, the qulittive preditions remined unhnged while the free prmeter ws vried over two orders of mgnitude (Fig. 4d). In ft, no setting of the free prmeter ould mke the pereption of nrrow rhomuses more veridil thn tht of ft ones. Similrly, there is no setting tht would mke the pereption of low-ontrst rhomuses more veridil thn tht of high-ontrst rhomuses. RESULTS We ompred the preditions of the idel oserver (the solution of eqution (1)) to previously pulished psyhophysil dt 17,31,32. The free prmeter ws djusted mnully for eh experiment ut held onstnt for ll onditions within eh experiment. Different oservers proly mke different ssumptions regrding noise, nd indeed, sustntil individul differenes for these illusions hve een reported 17. As with the rhomus exmple, the vlue of the free prmeter did not hnge the qulittive preditions of the model for ny of the stimuli disussed here. Influene of ontrst on pereived grting speed The pereived speed of single grting depends on ontrst 31,33 35, with lower-ontrst ptterns onsistently ppering slower thn higher-ontrst ptterns 34. This my underlie the tendeny of utomoile drivers to speed up in the fog 36. In psyhophysil experiment quntifying this effet 31, sujets were sked to ompre the pprent speed of two grtings of different ontrst (Fig. 5). The low-ontrst grting ws onsistently pereived to e moving slower. This illusion depended primrily on the rtio of ontrsts of the two grtings: the pereived speed ws n pproximtely liner funtion of the ontrst rtio, nd ws pproximtely independent of solute ontrst. The idel oserver shows qulittively similr ontrst dependene. At low ontrsts, the likelihood is roder nd the prior hs stronger influene on the estimte. Consistent with humn pereption, the idel oserver lso estimtes the low-ontrst grting s moving slower (Fig. 5). 6 nture neurosiene volume 5 no 6 june 2

4 Prior Prior Imge Likelihood 1 X x Posterior Imge Likelihood 1 X x Posterior Likelihood 2 The simple idel oserver presented here does not predit the qusiliner shpe of the pereived reltive speeds, nor does it predit the lk of dependene on totl ontrst (it mkes slightly different preditions for mximum ontrsts of % nd 7%, Fig. 5). We lso onstruted slightly more elorte model tht n ount for these effets in more quntittive mnner (see Disussion). Influene of ontrst on pereived plid diretion The pereived diretion of plid depends on the reltive ontrst of the two onstituent grtings. We replotted dt from n experiment in whih sujets reported the pereived diretion of motion of symmetri plids while the ontrst rtio of the two omponents ws vried (Fig. 5). Pereived diretion ws lwys ised towrd the norml diretion of the higherontrst grting. The mgnitude of the is hnged s funtion of the totl ontrst of the plid (the sum of the ontrsts of the two grtings). Inresing the ontrst of oth grtings (while the rtio of ontrsts is held fixed) resulted in smller is. The idel oserver shows similr effet (E. P. Simonelli & D. J. Heeger, Invest. Opthl. Vis. Si. Suppl. Astr. 33, 954, 1992), whih gin follows from the ft tht t low ontrst, there is Likelihood 2 d Diretion (degrees) Prior 1 3 Imge Likelihood 1 X x Posterior Likelihood Rhomus ngle (degrees) Fig. 4. Preditions of idel oserver for rhomus stimuli. ( ) Constrution of the posterior distriution for the rhomus stimuli. For lrity, likelihood funtions for only two lotions re shown; the estimtor used in our study inorported likelihoods from ll lotions. (d) Cirles show pereived diretion for single humn sujet s rhomus ngle ws shifted grdully from thin to ft rhomuses (ll three sujets showed similr effet, nd ll gve informed onsent to prtiipte in the study). Eh sujet ws given 1 presenttions. Solid line shows the preditions of the Byesin estimtor omputed using eqution (1), where the free prmeter ws vried mnully to fit the dt. Dotted lines indite the preditions when the free prmeter ws deresed y ftor of 1 (top dotted line) or inresed y ftor of 1 (ottom line). higher unertinty nd hene the low-ontrst grting hs less influene on the estimte. Contrst influene on pereived line diretion Sujets tend to mispereive the diretion of moving line t low ontrsts, even when its endpoints re visile 32. We replotted dt from n experiment in whih sujets reported the pereived diretion of mtrix of lines (Fig. 5). The mtrix ws onstruted y repliting single line t multiple lotions in the visul field. The line ws oriented suh tht its norml veloity ws downwrd even when the line ws moving upwrd. At low ontrsts, sujets performed fr elow hne, inditing tht they pereived upwrd motion while the line tully moved downwrd. The uthors proposed two seprte mehnisms to explin this finding, one deling with termintor (line endpoint) motion nd other with line motion. The termintor mehnism ws ssumed to e tive primrily t high ontrsts nd the line strtegy primrily t low ontrsts. We found tht t low ontrst, the idel oserver lso mispereived the diretion of motion euse the likelihoods re roder nd the estimtor prefers the norml veloity (whih is slower thn the true veloity). To otin perentge of orret responses for the idel oserver, we ssumed tht v * ws orrupted y deision noise, nd we lulted the proility tht the orrupted v * ws in the upwrd diretion. The deision noise ws Gussin in veloity spe. The stndrd devition of the deision noise determines the shrpness of the psyhometri funtion nd ws djusted mnully. The predited perentge orret for the idel oserver ws in ordne with humn pereption (Fig. 5, solid line). Type I versus type II plids: pereived diretion In the plid literture, distintion is often mde etween two types of onfigurtion: for type I plid, the diretion of the veridil veloity lies etween tht of the two norml veloities; for type II plid, the veridil diretion lies outside the two normls 17. In the ltter se, the vetor verge is quite different from the veridil veloity. At low ontrst, the pereived diretion for type II plids is strongly ised in the diretion of the vetor verge, nd the pereived diretion of type I plids is lrgely veridil. We replotted dt from single sujet who reported the pereived diretion of plid under five different onditions 17 (Fig. 5d, irles). nture neurosiene volume 5 no 6 june 2 61

5 Feture motion 1 1 Mx ontrst 7% 5% % Mx ontrst % Norml motion Log ontrst rtio Log2 ontrst rtio Contrst Reltive speed Condition In ll five onditions, the ngulr seprtion etween the two grtings ws In some onditions the two norml veloities were on different sides of the veridil motion (type I), wheres in others they were on the sme side of the veridil motion (type II). Sujets sw type I plids moving in the diretion nd type II plids moving in pproximtely the diretion ( 55 wy from veridil diretion). The uthors of the originl study explined their findings using ontrst-dependent omintion of first-order nd seond-order motion nlyzers 37. The idel oserver lso predited different diretions of motion for the two types of plids t low ontrst (Fig. 5d, solid line). The mispereption of type II plids is similr to the pereption of the nrrow rhomus: the veloity is muh slower thn the solution nd hene it is fvored t low ontrsts. In the idel oserver, this is towrd the solution wekens with inresing ontrst, s the likelihoods eome nrrower. It hs lso een reported tht the is is more pronouned with shorter presenttion durtions 17. We sed our idel oserver on instntneous mesurements, so it is not ffeted y disply durtion. The formultion n esily e extended so tht the idel oserver integrtes informtion over time. This wy, inresed durtion ts in similr fshion to inresed ontrst: the longer the durtion, the nrrower the likelihood. Suh n extended formultion predits tht the is would derese with inresed durtion. A similr effet of durtion hs een reported elsewhere 32, whih would lso e predited y this extension of our model. Bis (degrees) d e f Diretion (degrees) Judged plid diretion (degrees) Plid omponent seprtion (degrees) Perentge orret Rtio of omponent speeds Fig. 5. Comprison of idel oserver (solid lines) to vriety of pulished psyhophysil dt (irles). () Contrst influene on pereived grting speed. Cirles indite the pereived speed of the lower-ontrst grting reltive to the higher-ontrst grting, s funtion of the ontrst rtio. Solid lines show the preditions of the idel oserver for two different mximl ontrsts (dt from ref. 31). () Reltive ontrst influene on pereived plid diretion (dt from ref. ). () Contrst influene on pereived line diretion (dt from ref. 32). (d) Pereived diretion of type I (onditions 1, 3, 5) versus type II (onditions 2, 4) plids (dt from ref. 17). Dotted line shows the predition. (e) Influene of reltive orienttion on pereption of type II plid motion (dt from ref. 18). (f) Influene of reltive speed on pereption of type II plids (dt from ref. 19). Perentge in diretion of plids s funtion of this ngle, while pttern veloity ws held onstnt 18 (Fig. 5e). The pereived diretion is not onsistent with pure mehnism or pure mehnism. Insted, it shows grdul shift from the to the solution s the ngle etween the omponents inreses. The solid line shows the predition of the idel oserver (the diretion of v * in eqution (1)). This sitution is similr to the nrrow versus ft rhomuses (Fig. 4). When two likelihoods whose onstrint lines re nerly identil re multiplied, their produt will e rod nd hene hve less of n influene on the posterior. By ontrst, when two likelihoods hve widely differing onstrint lines, their produt will e nrrow nd hene hve greter influene on the posterior. Influene of reltive speed on type II plids The pereived diretion of plid lso depends on the reltive speeds of the omponents. We plotted dt from single sujet 19 who viewed plid with nd diretions on opposite sides of upwrd, nd reported whether the motion ppered to e more leftwrd or rightwrd (Fig. 5f). When the speeds of the two omponents were similr, the sujet nswered rightwrds (onsistent with the solution), ut when the speeds were dissimilr, the sujet nswered leftwrds (onsistent with the solution). We found tht the idel oserver desried y eqution (1) shows similr shift from leftwrd to rightwrd veloities. We gin lulted perentge orret vlue for the idel oserver y ssuming deision noise (Fig. 5f, solid line). Influene of reltive orienttion on type II plids The pereived diretion of type II plid depends strongly on the ngle etween the omponents. We replotted dt from n experiment in whih sujets reported the pereived diretion DISCUSSION Reserh on visul motion nlysis hs yielded tremendous mount of experimentl dt. When viewed in the ontext of existing rules suh s nd, these dt seem ontrdi- 62 nture neurosiene volume 5 no 6 june 2

6 tory, requiring n ritrry omintion sheme tht pplies the right rule in the right onditions. Suh n pproh n suessfully fit the dt, ut is typilly lking in preditive power: with omplited enough omintion sheme one n model ny experiment. More importntly, euse these rules re not formulted diretly on imge mesurements, it is not ler how one should generlize them for pplition to ritrry sptiotemporl stimuli. Here we hve tken n lterntive pproh. We derived n optiml estimtor for lol imge veloity using the stndrd ssumption of intensity onstny nd two dditionl ssumptions: mesurement noise nd n priori preferene for slower veloities. We found, onsistent with results in humns, tht the motion estimtes of this model inlude pprent ises nd illusions. Moreover, the predited non-veridil perept is quite similr to tht exhiited y humns under the sme irumstnes. Although the model does not ount for ll of the existing dt quntittively, it orretly predited wide rnge of effets. Our model does not provide good quntittive fit to the dt of Fig. 5 (see Results), whih suggest qusiliner dependene of pereived grting speed on ontrst, nd miniml dependene on totl ontrst. Our model hs een extended y inluding nonliner gin ontrol funtion to mp stimulus ontrst into pereived ontrst (F. Hurlimnn, D. Kiper & M. Crndini, Invest. Opthl. Vis. Si. Suppl. Astr., 794, ). For eh sujet in tht study, the uthors mesured gin ontrol funtion from ontrst-disrimintion experiments. They then used the pereived ontrst rther thn stimulus ontrst s input to our model, nd found tht when these relisti representtions of ontrst were used, the quntittive preditions of the Byesin model were in generl greement with the dt. We lso found, using numeril serh proedure, tht monotoni nonliner gin ontrol funtion enled our model to etter fit the results reported here nd in ref. 31 (see Supplementry Results online). One result 33 tht is not predited y our model is the finding tht low-ontrst grtings tully pper to move fster thn high-ontrst grtings for temporl frequenies ove 8 Hz. However, the sme uthor lter ws unle to reprodue this result using fored-hoie tsk 31, nd onluded tht the originl finding ws proly n rtift of the experimentl method with sujets mking speed mthes sed on some other riterion. Our Byesin estimtor is ment s pereptul model, nd does not speify prtiulr implementtion. Nevertheless, the solution n e instntited using so-lled motion energy mehnisms 28,38, nd detiled models of the physiology of the motion pthwy 24,25,28,39 41 suggest tht popultion of MT ells my e forming representtion of the lol likelihood of veloity. In ddition, we elieve it should e possile to refine nd justify the ssumptions we hve mde. In prtiulr, the prior distriution on veloity ould e estimted empirilly from the sttistis of motion in the world. In physiologil implementtion, the noise model should e repled y one tht more urtely reflets the unertinties of neurl responses. Our model lso suggests some future experiments. First, if the single free prmeter is oserver dependent (ut otherwise onstnt), the mgnitude of different illusions for the sme sujet should e orrelted. For exmple, oservers who gretly underestimte the speed of low-ontrst grtings should lso show lrger is towrds in type II plids. Seond, in ll of our simultions we used only the mximum (or men) of the posterior distriution. It would e interesting to test whether humn perepts reflet the shpe of the full posterior distriution. We hve foused on n idel oserver for estimting single two-dimensionl trnsltion. This model nnot estimte more omplited motions suh s rottions nd expnsions, nor n it hndle senes ontining multiple motions. Elsewhere, we desrie n extended idel oserver for more generl senes with multiple motions 29. We show tht n idel oserver tht ssumes tht veloity fields re slow nd smooth 42 n explin n even wider rnge of motion phenomen. In prtiulr, the is towrd slower motions n sometimes ount for one of the most ritil issues in motion pereption: the question of whether to omine mesurements into single oherent motion or ssume tht there re tully multiple motions (H. Frid & E. P. Simonelli, Invest. Opthl. Vis. Si. Suppl. Astr. 35, 1271, 1994). Although the detils of our model should ertinly e refined nd extended to hndle more omplited phenomen, we elieve the underlying priniple will ontinue to hold: tht mny motion illusions re not the result of sloppy omputtion y vrious omponents in the visul system, ut rther result of oherent omputtionl strtegy tht is optiml under resonle ssumptions. METHODS Most models of erly motion extrtion rely on n ssumption of intensity onservtion. Under this ssumption, the points in the world, s mesured in the imge, move ut do not hnge their intensity over time. Mthemtilly, this is expressed s: I(x,y,t) = I(x + v x t, y + v y t, t + t) (2) where v x nd v y re the omponents of the vetor, v, desriing the imge veloity. If we ssume tht the oserved imge is noisy, then intensity is not onserved extly. Thus, eqution (2) eomes I(x,y,t) = I(x + v x t, y + v y t, t + t) + η (3) where η is rndom vrile representing noise. We used eqution (3) to derive the likelihood t lotion i, P(I(x i,y i,t) v i ). This required dditionl ssumptions. We ssumed the noise, η, is Gussin with stndrd devition σ. We further ssumed tht the veloity is onstnt in smll window round x i,y i nd tht the intensity surfe I(x,y,t) is suffiiently smooth tht it n e pproximted y liner funtion for smll temporl durtions. We thus repled I(x + v x t, y + v y t, t + t) with its first-order Tylor series expnsion, whih gives: P(I(x i,y i,t) v i ) exp 1 2 σ 2 x,y w i (x,y) (I x (x,y,t)v x + I y (x,y,t)v y + I t (x,y,t)) 2 dx dy where {I x,i y,i t } denote the sptil nd temporl derivtives of the intensity funtion I, nd w i (x,y) is window entered on (x i,y i ). The likelihoods shown in Fig. 3 nd Fig. 4 re omputed from eqution (4) with w(x,y) smll Gussin window. Finlly, we ssumed prior fvoring slow speeds: P(v) exp( v 2 /2σ p 2 ). (5) The posterior proility of veloity ws omputed y omining the likelihood nd prior using Byes rule. Beuse we ssumed tht the noise is independent over sptil lotion, the totl likelihood funtion is just produt of likelihoods: P(v I) P(v) Π P(I(x i,y i,t) v), (6) i where the produt is tken over ll lotions i tht re moving with ommon veloity (v i = v). Sustituting equtions (4) nd (5) into equion (6), (4) nture neurosiene volume 5 no 6 june 2 63

7 1 P(v I) exp v 2 2 /2 σ 2 σ 2 p Σ w i (x,y) (I x (x,y)v x + I y (x,y)v y + I t ) 2 dx dy. x,y i Here we ssumed the entire imge moves ording to single trnsltionl veloity, nd so summed over ll sptil positions. In this se, i w i (x,y) is onstnt, so the posterior proility is given y: 1 P(v I) exp v 2 /2 σ 2 2 σ 2 p (I(x,y) v x + I y (x,y)v y + I t ) 2 dx dy x,y To find the most prole veloity, we repled the integrl with disrete sum, took the logrithm of the posterior, differentited it with respet to v nd set the derivtive equl to zero. The logrithm of the posterior is qudrti in v so tht the solution n e written in losed form using stndrd liner lger. The result is given in eqution (1). Note: Supplementry informtion is ville on the Nture Neurosiene wesite. Aknowledgments Y.W. nd E.H.A. were supported y US Ntionl Eye Institute R1 EY115 to E.H.A. E.P.S. ws supported y the Howrd Hughes Medil Institute nd the Slon-Swrtz Center for Theoretil Visul Neurosiene t New York University. We thnk J. MDermott, M. Bnks, M. Lndy, W. Geisler nd the nonymous referees for omments on previous versions of this mnusript. Competing interests sttement The uthors delre tht they hve no ompeting finnil interests. RECEIVED 19 FEBRUARY; ACCEPTED 15 APRIL 2 1. Nkym, K. Biologil imge motion proessing: review. Vision Res. 25, (1985). 2. Horn, B. K. P. & Shunk, B. G. Determining optil flow. Artif. Intell. 17(1 3), (1981). 3. Lus, B. D. & Knde, T. An itertive imge registrtion tehnique with n pplition to stereo vision. in Proeedings of the 7th Interntionl Joint Conferene on Artifiil Intelligene (Morgn-Kufmnn, Sn Frnsiso, 1981). 4. Wuerger, S., Shpley, R. & Ruin, N. On the visully pereived diretion of motion y Hns Wllh: 6 yers lter. Pereption 25, (1996). 5. Wllh, H. Ueer visuell whrgenommene ewegungrihtung. Psyhol. Forsh., (1935). 6. Mrr, D. & Ullmn, S. Diretionl seletivity nd its use in erly visul proessing. Pro. R. So. Lond. B Biol. Si. 211, (1981). 7. Adelson, E. & Movshon, J. Phenomenl oherene of moving visul ptterns. Nture 3, (1982). 8. Movshon, A., Adelson, E., Gizzi, M. & Newsome, W. The nlysis of moving visul ptterns. Exp. Brin Res. 11, (1986). 9. Welh, L. The pereption of moving plids revels two proessing stges. Nture 337, (1989). 1. Morgn, M. Sptil filtering preedes motion detetion. Nture 355, (1992). 11. Shrter, P., Knill, D. & Simonelli, E. Mehnisms of visul motion detetion. Nt. Neurosi. 3, (). 12. Rodmn, H. & Alright, T. Single-unit nlysis of pttern motion seletive properties in the middle temporl visul re MT. Exp. Brin Res. 75, (1989). 13. Movshon, J. A. & Newsome, W. T. Visul response properties of strite ortil neurons projeting to re MT in mque monkeys. Vis. Neurosi. 16, (1996). 14. Okmoto, H. et l. MT neurons in the mque exhiited two types of imodl diretion tuning s predited y model for visul motion detetion. Vision Res. 39, (1999). 15. Ferrer, V. & Wilson, H. Pereived diretion of moving two-dimensionl ptterns. Vision Res. 3, (199). 16. Mingoll, E., Todd, J. & Normn, J. The pereption of glolly oherent motion. Vision Res. 32, (1992). 17. Yo, C. & Wilson, H. Pereived diretion of moving two-dimensionl ptterns depends on durtion, ontrst, nd eentriity. Vision Res. 32, (1992). 18. Burke, D. & Wenderoth, P. The effet of intertions etween onedimensionl omponent grtings on two dimensionl motion pereption. Vision Res. 33, (1993). 19. Bowns, L. Evidene for feture trking explntion of why type II plids move in the vetor sum diretions t short durtions. Vision Res. 36, (1996).. Stone, L., Wtson, A. & Mullign, J. Effet of ontrst on the pereived diretion of moving plid. Vision Res. 3, (199). 21. Ruin, N. & Hohstein, S. Isolting the effet of one-dimensionl motion signls on the pereived diretion of moving two-dimensionl ojets. Vision Res. 33, (1993). 22. Helmholtz, H. Tretise on Physiologil Optis (Thoemmes, Bristol, UK, ; originl pulition 1866). 23. Knill, D. & Rihrds, W. Pereption s Byesin Inferene (Cmridge Univ. Press, Cmridge, 1996). 24. Asher, D. & Grzywz, N. A Byesin model for the mesurement of visul veloity. Vision Res., (). 25. Koehlin, E., Anton, J. L. & Burnod, Y. Byesin inferene in popultions of ortil neurons: model of motion integrtion nd segmenttion in re MT. Biol. Cyern. 8, (1999). 26. Simonelli, E., Adelson, E. & Heeger, D. in Pro. IEEE Conf. Comput. Vision Pttern Reog (IEEE, Wshington DC, 1991). 27. Heeger, D. J. & Simonelli, E. P. in Sptil Vision in Humns nd Roots Ch. 19 (eds. Hrris, L. & Jenkin, M.) (Cmridge Univ. Press, 1994). 28. Simonelli, E. P. Distriuted Representtion nd Anlysis of Visul Motion. Thesis, Msshusetts Institute of Tehnology (1993). 29. Weiss, Y. Byesin Motion Estimtion nd Segmenttion. Thesis, Msshusetts Institute of Tehnology (1998). 3. Ullmn, S. The Interprettion of Visul Motion (MIT Press, Cmridge, Msshusetts, 1979). 31. Stone, L. & Thompson, P. Humn speed pereption is ontrst dependent. Vision Res. 32, (199). 32. Loreneu, J., Shiffrr, M., Wells, N. & Cstet, E. Different motion sensitive units re involved in reovering the diretion of moving lines. Vision Res. 33, (1992). 33. Thompson, P. Pereived rte of movement depends on ontrst. Vision Res. 22, (1982). 34. Thompson, P., Stone, L. & Swsh, S. Speed estimtes from grting pthes re not ontrst normlized. Vision Res. 36, (1996). 35. Blkemore, M. & Snowden, R. The effet of ontrst upon pereived speed: generl phenomenon? Pereption 28, (1999). 36. Snowden, R. N., Stimpson, N. & Ruddle, S. Speed pereption fogs up s visiility drops. Nture 392, 45 (1998). 37. Wilson, H., Ferrer, V. & Yo, C. A psyhophysilly motivted model for twodimensionl motion pereption. Vis. Neurosi. 9, (1992). 38. Weiss, Y. & Fleet, D. in Proilisti Models of the Brin Ch. 4 (eds. Ro, R., Olshusen, B. & Lewiki, M.) (MIT Press, Cmridge, Msshusetts, 2). 39. Nowln, S. J. & Sejnowski, T. J. A seletion model for motion proessing in re MT of primtes. J. Neurosi. 15, (1995).. Simonelli, E. & Heeger, D. A model of neuronl responses in visul re MT. Vision Res. 38, (1998). 41. Pouget, A., Dyn, P. & Zemel, R. Informtion proessing with popultion odes. Nt. Rev. Neurosi. 1, (). 42. Grzywz, N. & Yuille, A. Theories for the visul pereption of lol veloity nd oherent motion. in Computtionl Models of Visul Proessing (eds. Lndy, J. & Movshon, J.) (MIT Press, Cmridge, Msshusetts, 1991). 64 nture neurosiene volume 5 no 6 june 2