Structure from Motion Using Optical Flow Probability Distributions

Transcription

1 Stuctue om Moto sg Optcl Flow Polt Dstutos Pul Meell d D. J. Lee Deptmet o Electcl d Compute Egeeg Bghm Youg vest 459 CB Povo th 846 ABSRAC Stuctue om moto s techque tht ttempts to ecostuct the 3D stuctue o scee om sequece o mges tke om cme movg wth the scee. Stuctue om moto c e used o med Ael Vehcle o med oud Vehcle o ostcle detecto s well s o pth-plg d vgto. he 3D stuctue o the scee s estmted usg the optcl low vlues oud t set o etue pots o the mge. pcll ths s doe ude the ssumpto tht ll o the optcl low vlues hve the sme level o ccuc d tht the ccuc o ech optcl low vlue the hozotl d vetcl dectos s the sme. hese ssumptos e ot etel coect. he ccuc o deet optcl low vlues e ot the sme d the ccuc c e mesued d quted usg optcl low polt dstuto. e wll peset ovel stuctue om moto lgothm tht s moe ccute d moe oust th othe methods ecuse t uses optcl low polt dstutos. e wll peset oe method tht s desged to wok o ol two mes d othe method desged to wok o ume o mes. Kewods: Stuctue om moto AV V ostcle detecto pth-plg utoomous vgto optcl low polt dstutos. INRODCION Oe o the ke polems compute vso s the stuctue om moto polem whch s how to ecostuct 3D scee usg sequece o D mges tke om movg cme the scee. Stuctue om moto s tpcll ccomplshed st estmtg the optcl low vlues o set o etue pots the usg these optcl low vlues to estmte the ottol d tsltol moto o the cme d the depth o the oects the scee. Most stuctue om moto lgothms mplctl ssume tht the ose the optcl low s detcl t ech etue pot d tht the ose s whte. hese ssumptos e oth coect. Fo emple pots o the mge wth stoge sptl gdets e ese to tck th othe pots d wll povde moe ccute optcl low vlue whch mples tht the ssumpto tht the ose s detcl s coect. Cet pots m hve stoge sptl gdet ptcul decto whch llows moe ccute optcl low vlue to e clculted tht decto whch mples tht the ssumpto tht the ose s whte s lso coect. Both o these ssues c e esolved we e wllg to use optcl low polt dstutos the plce o sgle optcl low estmte t ech etue pot. A optcl low dstuto wll llow us to qut the ccuc o ech etue pot decto. B elg moe o the optcl low vlues tht e kow to e moe ccute we wll peset stuctue om moto lgothm tht s moe oust to ose. e wll peset method o clcultg optcl low polt dstutos d the show how to use the polt dstutos to clculte stuctue om moto moe ccutel. e st del wth stuctue om moto ol usg two mes whch s the smplest cse the we wll show how to eted the two-me method so tht t wll wok o m mes. I ou ele wok we desced method o clcultg optcl low polt dstutos [5]. e lso peseted methods o clcultg stuctue om moto usg optcl low polt dstutos [6]. hese methods wll e desced gete detl ths ppe. Oe method o clcultg the optcl low polt dstutos whch we wll desce s tke om the wok o Smocell et l. []. he stuctue om moto method we wll peset s most sml to optml stuctue om moto lgothm desced Sotto d Bockett [3]. he method s ol optml whe thee s detcll-dstuted whte ose. he method we wll peset s optml ude kd o ose codtos.

2 he optcl low polem d the stuctue om moto polem do ot hve to e cosdeed completel septe polems. B tegtg the two polems t s possle to mpove oth techques [9 ]. Ou method could e cosdeed smpl ette w o tegtg the optcl low pocess wth the stuctue om moto pocess sce we e etctg moe useul omto om the optcl low pocess to e used the stuctue om moto pocess. Dellet et l. [] hve developed stuctue om moto lgothm whee the ect moto o ech etue pot s ukow ut t s kow to e oe o sevel possle vlues. Smll ou method ssumes the ect moto s ukow ut could e oe o lge ume o te ume o possle vlues. I ddto ou method hs the dvtge o kowg the polt o ech vlue. M d Lge [ 3] dscuss stutos whch the optcl low vlue s ukow ut t s kow to e oe o set o optcl low vlues lg o oe-dmesol le. hs stuto c e hdled usg optcl low dstuto d ddto optcl low dstuto ssgs polt to ech possle vlue.. OPICAL FLO PROBABILIY DISRIBIONS he ollowg method o clcultg optcl low dstutos wll e peseted wthout devto. Fo complete devto plese see []. he st step s to costuct the ollowg mt d vectos usg the sptl gdets d the d dectos d the tempol-devtve t : t M s t. Ech o these quttes s ol clculted om sgle pel. It would e ette to lso use the pels eghog the ptcul posto o teest the clculto. Let M d e the vlues o M d t the -th posto. he postos close to the posto o teest e moe vlule d should e gve moe weght. Let e the weght ttched to the -th posto so tht the postos close to the desed posto e gve moe weght. hese vlues e the used to clculte the covce mt o the optcl low the equto σ ω M s σ p wth p eg the covce mt o the po dstuto o the optcl low d wth d eg the vces ssocted wth two deet souces o ose. Oe o the souces o ose s esult o coect ssumpto tht the moto o the mge s smple pl tslto. desces the eos toduced om the lue o ths plt ssumpto. desces the eos toduced ccute tempol devtve possl om ose the mge testes. hese pmetes m eed to e dusted sed upo the qult o the mges d the chctestcs o the scee. I tpcl mge sequece ppomte vlues o ech pmete hve ee oud empcll to e:.8. p.. he me vlue o the optcl low s gve s µ u σ ω s σ O-FRAME SRCRE FROM MOION Suppose tht the optcl low polt dstutos hve ee clculted o totl o etues pots. I the st mge the -th etue pot s locted o the mge ple t the posto p. I the secod mge the -th etue pot hs moved to the posto p. he optcl low etwee the two mges s dom vle clled

3 whch hs me vlue o u d covce mt o whch c e clculted usg equtos d 3. he optcl low eltes the two postos the equto. p p o mke the mthemtcs o the polem smple we wll poect the postos oto ut sphee tht s ceteed t the optcl cete o the cme. he poecto o the posto p oto ut sphee wll e clled. A dgm o ths poecto s show Fgue. he dstce om Fgue : Dgm o the Poecto oto ut sphee the optcl cete o the cme to the mge ple s the ocl legth o the cme o. s clculted om the d compoets o p whch e deoted p d p [ p p ] o. 4 p p Net the optcl low vlues wll lso e poected oto the ut sphee to om ew vecto clled. Fst d [ ]. two vectos o the suce o the sphee e oud such s [ ] Secod the vecto s poected oto the suce deed these two vectos usg the equtos [ ] N N N N N. 5 A ew vecto s deed the equto. he ht opeto wll e used to dcte the mt tht peoms the coss-poduct etwee two vectos so tht. he tsomto om to wll e used equetl so the mt tht tsoms to wll e clled wth the eltoshp. s clculted s N N N N. 6 3

4 4 he tsltol moto o the cme wll e epeseted ut vecto tht s poted the decto o tslto. he otto o the cme wll e epeseted vecto whee the decto o s the s out whch the cme s otted d the mgtude o s the mout the cme s otted ds. he vese depth o the -th etue pot wll e epeseted. he depth c ol e clculted to ukow scle cto. he scle cto s tl chose so tht the dstce tveled the cme s equl to oe. So wll e the depth o the -th etue pot dvded the dstce tveled the cme. 3.. Cost Fucto he ollowg eltoshp ests etwee ech o the tems we hve ust deed. 7 Ate susttutg o. 8 Now oth sdes o the equto e multpled the let pseudo-vese o whch s. Ate smplg we d tht. 9 Equto 9 s ol tue whe thee s o ose the optcl low estmtes. I pctce thee s lws ose the optcl low estmtes. A vlue o c e clculted om gve d vlues usg equto 9. hs vlue hs kow polt sed upo the optcl low polt dstutos. Fom the polt dstutos polt c e clculted o eve possle comto o d vlues. he gol s to d the d vlues whch e the most pole. hs tsk s equvlet to the tsk o dg the vlues o d whch mmze the cost ucto u whee the weghted-om s deed s. B mmzg ths cost ucto we wll d the d vlues whch e the most pole. I we epd the weghted-om d ege ew tems we d tht u u. B toducg ew weghtg mt equto s smpled to e. 3.. Depth d Rotto Estmto I ode to mmze ths cost ucto t wll e ecess to use geelzed lest-sques ppomto epetedl. Fo mtces A d d vecto the vecto c tht mmzes the weghted-om Ac s

5 5 oud usg geelzed lest-sques to e A A A c. sg geelzed lest-sques the optml vlue o o t tslto d otto vectos s oud to e. 3 Sce ths soluto s the est possle soluto o d t c e plced ck to equto so tht the vlues o d tht mmze the cost ucto o 4 e the sme vlues o d tht mmze the cost ucto. Idell we would lke to solve o d smulteousl. otutel we hve ol ee le to d ol closed om soluto o tht depeds upo kow vlue o d lkewse closed om soluto o tht depeds upo kow vlue o. he pl wll e to pck tl vlue o d d the optml vlue o sed upo tht vlue. Net the optml vlue o sed upo the clculted vlue s oud. he the optml vlue o sed upo the ew vlue s oud. hs pocess epets tsel tetvel utl te ew tetos the vlues o d do ot chge sgctl d the we e shed. Fst we wll epl how to clculte vlue o sed upo kow. he cost ucto equto 4 c e smpled deg ew mt Q s I Q o Q. 5 hs equto c e moded much the sme w tht equto ws moded to poduce equto o Q Q. 6 he soluto o s oud usg geelzed lest-sques to e Q Q Q Q slto Estmto Fo the momet let us cosde deet w o weghtg the oms the cost ucto. Let us cete ew cost ucto w whch uses the ew weght w. hs cost ucto o loge uses the optcl low dstutos ecuse the covce mtces e o loge used. hs ew cost ucto wll e used to d soluto whch does ot use optcl low dstutos ut the the esult wll e eteded to ot method tht does use them. he ollowg devto s tke om [3]. he optml vlue o o t tslto d otto vectos usg the ew cost ucto s

6 6. 8 hs soluto s plugged ck to the cost ucto so tht the cost ucto o loge depeds o w. 9 c e eplced wth so tht w. pots out om the cete o the ut sphee d les o the suce o the sphee tget to so d e othogol. hs equto c e smpled ecogzg tht d emovg the weghts om the weghted-om w. Sce s ut vecto d s scl vlue c e emoved om sde the om. he soluto o tht mmzes ths cost ucto s the mmum-egevlue egevecto o the mt. Mmzg ths cost ucto s equvlet to dg the most pole vlue o o set o equtos gve 3 whee s set o detcll-dstuted uss dom vles. e would lke to eted ths esult so tht t c hdle the ose clculted om the optcl low dstutos whch s ot detcll-dstuted. I ths cse the ose s uss dom vle ttched to ech wth me o zeo d covce mt o so tht

7 . 4 Sce s tege om to ths gves us set o equtos wth ech equtos hvg ose o. he om equto wll e moded so tht sted o smpl ssumg tht ech equto s o equl vlue we gve dded vlue to those equtos wth less ose them. Now we hve smll dlemm. he mout o ose depeds o the pmete tht we e tg to estmte. he estmto o depeds upo the mout o ose ech equto d the mout o ose ech equto depeds upo the estmto o. hee s smple soluto to ths dlemm. e e led usg tetve method to clculte d. he soluto s to smpl use the vlue o clculted om the pevous teto to estmte the mout o ose whch s the used the cuet teto. o cl ou otto k wll e the vlue o o the k-th teto. he mout o ose the -th equto o the k-th teto s equl to. e wll ow weght the -th equto wth the vlue k gete weght. Bsed o ths we cete ew cost ucto k k k k k k so the equtos wth less ose e gve 5 k k k k 6 k he soluto o k whch mmzes ths ew cost ucto s the mmum-egevlue egevecto o the mt k k MLI-FRAME SRCRE FROM MOION Mult-me stuctue om moto s qute sml to two-me stuctue om moto. All o the vectos d mtces em essetll the sme ecept the e lge to ccommodte multple mes. t s the optcl low vlue o the -th etue pot t tme t. s the optcl low om the st me to the secod me s the optcl low om the secod me to the thd me d so o. s vecto o sze N whch cots ll o the optcl low vlues though N. t s the posto o the -th etue pot t tme t poected oto ut sphee ccodg to equto 4. t s the poecto o t oto ut sphee ccodg to equto 5 d t s deed s t t t. s 3N dmesol vecto cotg the vectos though N. t s the mt deed ccodg to equto 6. All o the mtces though N c e comed to om 3N N mt so tht the eltoshp holds. Nothg susttl hs chged ecept the vectos d mtces e lge. he tsltol moto o the cme t tme t wll e epeseted the vecto t d the ottol moto wll e epeseted t. All o the t d t vectos c e comed to the lge vecto d. he depth stll c ol e clculted to ukow scle cto. e wll tl choose the st tslto vecto to e ut vecto so tht ow t s the speed o the cme t tme t dvded the speed t the st me. he vese depth s the ol tem tht does ot chge ove tme. I elt the depth wll chge slghtl ove tme s the cme ppoches o ecedes om the oects the scee. Howeve we ssume tht most o the oects the scee e eough w tht the cme s ot dge o httg them the e utue d we do ot use ecessvel lge ume o mes the we c ssume tht the depth wll st costt ove tme wthout sgctl ectg the peomce o the lgothm. 7

8 he eltoshp etwee ech o the tems we hve deed s detcl to the two-me cse. 8 t t t t t t I we let 3N 3N mt tht comes ll o the mtces though N the 9 whch s ectl the sme s equto 7 ecept ech o the vectos d mtces e lge th the wee equto 7. Sce ths equto s the sme the soluto o the otto o gve tslto c e deved the sme w t ws equtos 7 though 7 s Q Q Q Q. 3 he depth estmto s lso detcl. 3 he otto estmto s ectl the sme s the two-me stuctue om moto ut the tslto estmto wll e deet. Isted o estmto the tslto om kow otto we wll estmte the tslto om kow depth vlues. he eso o ths chge s tht two-me stuctue om moto the depth vlues e uelle. Ech depth vlue s ol clculted om oe optcl low vlue. I tht oe optcl low vlue s ccute the depth vlue wll e ccute. ht mkes mult-me stuctue om moto so ppelg s so tht ll o the optcl low vlues om m mes c e used to d l ccute depth vlue. he mult-me stuctue om moto lgothm woks st dg tl estmte o the tslto ccodg to equto 7. hs tl tslto estmte s used to clculte the optml otto d depth o ech etue pot wth equtos 3 d 7. he depth the s used to clculte the optml tslto d otto. he the tslto s used to clculte the otto d depth. hs pocess epets utl we hve l good estmte o the tslto otto d depth. he ol emg step s to d the optml vlue o tslto d otto om kow depth vlues. e dee the vecto d d the mt P s d P [ ] 3 so tht the weghted-om c e wtte s otto vecto d s oud to e P d d the soluto o the comed tslto d d P P P. 33 8

9 5. RESLS A smulto ws ceted to compe ths ew method to sml methods tht do ot use multple mes d tht do ot use optcl low dstutos. O ech tl t etue pots e doml selected o ut sphee. Rdom tslto d otto vectos e chose d the used to clculte the optcl low vlues t ech etue pot d t ech tme step ccodg to equto 7. hese optcl low vlues e the coupted ose t ech tme step. he vce o the optcl low vlues the d dectos e doml ssged vlue etwee.5 d.75 tmes me ose vlue. he coelto coecet s doml chose to e etwee - d. he coupted optcl low vlues e the used thee deet methods o compso. he thee methods e the twome method the mult-me wthout dstutos d the mult-me wth dstutos. he two-me method ol uses the optcl low etwee the st d lst mes. All the omto coted the othe mes s goed d ll o the omto the ose covce s goed. Sotto d Bocket s method [3] s used o the two-me method ecuse t s sml to ou method ut s desged to wok o two-mes d ot use optcl low polt dstutos. he mult-me wth dstutos method s the method desced ths ppe. he mult-me wthout dstutos s detcl to the method desced ths ppe ecept ech o the weghtg mtces the equtos e eplced the dett mt. he vege depth eo s clculted te 5 tls t deet me ose levels o ech o the deet methods. he esults o the smulto e show Fgue. he mpovemet om the two-me stuctue om moto d the mult-me stuctue om moto wthout dstutos method demosttes the eets tht come om usg multple mes. he mpovemet om the mult-me wthout dstutos d the mult-me wth dstutos demosttes the eets tht come om usg optcl low polt dstutos. Fgue : A Compso o thee Stuctue om Moto Algothms Fgue 3: Oe me om vdeo tke om cme ppochg tee. Fgue 4: Recoveed Ivese Depth 9

10 Some spects o ths smulto e kow to e uelstc. he smulto ssumes tht the ect covce mt o the ose s kow. I pctce ths too must e estmted usg equto. Eos the covce estmte wll cuse ou method to peom moe pool. Fgue 3 shows oe me o vdeo tke om cme ppochg tee. he ecoveed vese depth s show Fgue 4. he depth s ol clculted t ew etue pots the mge so the depth t the emg pots must e tepolted. Cosequetl pts o the mge whee thee e ew good etue pots such s the sk e ccute ut the emde o the mge s l ccute. Fgue 4 shows tht ou lgothm hs ccutel locted the tee s eg ot o the est o the scee. 6. CONCLSIONS Much esech hs ocused o the estmto o optcl low d thee e umeous ws o clcultg optcl low [8] ut ew o these methods clculte optcl low polt dstutos. e hope ou wok wll spe uthe wok ths e ecuse s we hve demostted optcl low dstutos c sgctl mpove the peomce o stuctue om moto lgothms. Ou method s uque ecuse t does ot smpl ssume tht ll o the optcl low dt s o equl vlue. e e le to qut the ccuc d vlue o the optcl low dt usg optcl low polt dstutos. I the ew stuctue om moto lgothm we ceted the moe ccute dt s gve moe weght so tht ll o the dt s used moe eectvel. o pove tht the dt s eg used moe eectvel we comped ou method to othe method whch ws el detcl ecept t dd ot weght the moe ccute dt deetl om the less ccute dt. e oud tht ou method peomed ette o hudeds o smulted tests d o el mge sequeces. REFERENCES. E. P. Smocell E. H. Adelso d D. J. Heege Polt dstutos o optcl low Poc. Co. Compute Vso d Ptte Recogto pp I. homs d E. Smocell. Le Stuctue om Moto. echcl Repot IRCS 94-6 vest o Peslv S. Sotto d R. Bocket Optml Stuctue om Moto: Locl Amgutes d lol Estmtes Poc. Co. Compute Vso d Ptte Recogto pp J. eg N Ahu d. Hug. Moto d stuctue om two pespectve vews: lgothms eo lss d eo estmto. IEEE s. Ptte Al. Mch. Itell. 5: P.C. Meell D.J. Lee d R.. Bed "Sttstcl Alss o Multple Optcl Flow Vlues o Estmto o med A Vehcles Heght Aove oud" SPIE Optcs Est Rootcs echologes d Achtectues Itellget Roots d Compute Vso XXII vol Phldelph PA SA Octoe P.C. Meell D.J. Lee d R.. Bed "Ostcle Avodce o med A Vehcles sg Optcl Flow Polt Dstutos " SPIE Optcs Est Rootcs echologes d Achtectues Mole Roots XVII vol Phldelph PA SA Octoe A. Jepso d D. Heege. Le suspce methods o Recoveg tsltol decto. Cmdge vest Pess J. Bo D. S. Fleet S. S. Beuchem d. A. Buktt Peomce o optcl low techques Poc. IEEE CVPR Chmpg IL 99 pp Y. Yoshd I. Ho S. Ymmoto N. Suge. Detemg 3D omto togethe wth coespodece om sequece o othogphcll poected optcl lows-tegted ppoch to stuctue-om-moto ssue IEEE s. Sstems M d Ceetcs vol. 3 pp D. DFco S. B. Kg. Is ppece-sed stuctue om moto vle? Poc. Co. 3-D Dgtl Imgg d Modelg pp F. Dellet S. M. Setz C. E. hope S. hu. Stuctue om moto wthout coespodece Poc. Co. Compute Vso d Ptte Recogto vol. pp R. M M. S. Lge. Optcl Sow d the petue polem Poc. Co. Ptte Recogto vol. 4 pp M. S. Lge R. M. Dmesol lss o mge moto Poc. Co. Compute Vso vol. pp.55-6.