Optic Flow Computation Using Interpolating Thin-Plate Splines

Transcription

1 ptc Fow Coputton Usng Interpotng Thn-Pte Spnes Arez Bb-Hdshr, Dvd Suter nd Ry Jrvs Integent Robotcs Reserch Centre Deprtent of Eectrc & Coputer Systes Engneerng onsh Unversty, Cyton Vc. 368, Austr. Abstrct ptc fow coputton s one of the ost fundent probes n the re of vsu oton. In ths contrbuton, we present nove optc fow coputton ethod bsed on thn-pte spne representton of ge brghtness dt. Usng spne-bsed descrpton of ge brghtness dt reoves the need for presoothng nd provdes n gebrc ens to copute the optc fow (expcty). Ths ethod cn be genersed to sove wde rnge of ge regstrton probes. A set of resuts on nuber of stndrd oton sequences s so presented. Introducton Durng the st two decdes there hs been n ncresng nterest n nysng ge sequences nd, n prtcur, n recoverng the optc fow fed. Athough ny ethods for esttng the fow fed hve been proposed, prctc re-te souton to ths probe rens chenge. The current exstng ethods for the optc fow coputton re ny bsed on the foowng pproches: - Correton technques - Phse or energy technques - Dfferent technques. The coprson study of Brron et. [] showed tht the phse bsed technques usuy provde the ost ccurte esttes of the fow fed. Recenty, few new dfferent technques hve been pubshed nd ther ccurcy s coprbe wth the phse bsed technques (Szesk & Coughn 994 nd Weber & k 995). Dfferent technques re often the pre cnddtes for re te ppctons due to coputton ese nd speed. The ccurcy of ny dfferent technque ny depends on the ccurcy of esttng dervtves of the ge brghtness functon. Athough the fnte dfference ethod, due to ts spcty, hs been the ost popur ethod for esttng the dervtves, t suffers fro certn drw-bcks. The fnte dfference ethod hs no ens to dstngush between nose nd the true dt: thus the resutng esttes cn be corrupted by nose. To ente the nose probe, ost of the proposed optc fow ethods rey on pre-soothng the ge functons usng Gussn fters. Successfu peentton of ths reedy requres soe pror knowedge bout the nose nd the vsu dt, often cqured through n expensve tr nd error process whch s spy not ffordbe n ny re-te ppcton. oreover, n gorth for utotcy dervng the soothng preters of the Gussn fters used hs not yet been found. It s nterestng to note tht sr probe so exsts n phse bsed technques n tht ther perfornce rgey depends on the ethod of tunng the frequency response of the dfferent fters nd deternng these so requres nforton not known n re-te ppctons. In ths pper, new frework, bsed on nterpotng thn-pte spnes, for esttng the optc fow fed s proposed. The key dfference between ths ethod nd other optc fow ethods s tht we expcty recover the underyng functon of brghtness dt by usng nterpotng thn-pte spnes. In dong ths, the dervtves cn be ccuted sybocy (off-ne) nd ther subsequent nuerc evuton s consequenty very fst. Ths proposton shoud not be stken for optc fow ethods usng spne to represent the veocty fed [2,3], or the stndrd Horn nd Schunck ethod [3] whch uses ebrne spne representng the veocty fed. The proposed ethod does not requre ny pre-soothng of ges. Ths contrbutes to fst (re-te) coputton of optc fow fed. The jor contrbuton of ths pper s the propos of nove ethod for esttng the dervtves of the brghtness ntensty functon. We so derve the necessry foruton for coputng the optc fow fed expcty, ssung spe trnston ode. The extenson of ths work to ncude ny ode of oton (eg. ffne) s strght forwrd nd w be ddressed n future. The proposed ethod uses ony two ges (s opposed to ost ethods usng sequence of ore thn two ges) nd, unke ost of the exstng ethods, does not hve sever tunng preters. A the necessry preters n our frework re expcty deduced fro the nput dt. The ptch sze s the ony preter eft to the user's judgent nd, s we show ter, t dcttes the trde off between ccurcy nd speed.

2 2 Spne Representton The dt fttng probe n hgh densons hs ttrcted uch ttenton nd ny pproches hve been proposed. Aong these ethods, the spne bsed pproches provde stsfctory nswers to regresson probes of recoverng unknown functons fro nosy dt. The recovered functon provdes vube ens for further studes of the dt rngng fro ccutng the dervtves to sttstc nyss. ny technques for constructng the spne functons hve been proposed but ebortng on the s beyond the scope of ths pper. The ge brghtness functon cn be recovered by fttng surfce to dscrete set of vbe vsu dt. ne of the coon pproches for sovng ths regresson probe s known s the vrton pproch. The vrton pproch s dopted n ths pper becuse of ts ngenous wy of coputng the preters of the resutng spne. Ths pproch s so cpbe of odeng the nose whch s essent for ccurte estton of dervtves. Aso, to keep the foruton spe, we consder the spest eber of surfce spne fy known s the thn-pte spne. The next sectons re dedcted to descrbng n eegnt ethod of chrctersng thn-pte spnes by reproducng kerne Hbert spce pproch. 2. Thn-pte Spnes Thn-pte spnes re the fous ebers of fy of surfce or D spnes whch resut fro the souton of vrton probe. enguet [4] nd Whb [5] soved the foowng nston probe. Consderng the foowng ode () for set of nosy dt; the probe s to fnd the functon f(x,y) H ( sutbe spce of suffcenty dfferentbe functons s) whch nses: ξ( s) = δ( s) + λη( s) () where λ s the soothng preter nd δ(s) nd η(s) re gven s: δ( s) = ( ( (, ))) f s x y 2 (, ) η( s F s x y ) = H G I ( j j ) dxdy jk J 2 x y R 2 n whch (x,y ) s the octon of the dt pont, f s the correspondng dt nd s the nuber of dt ponts. The soothng preter, λ, contros the trde off between the fdety to the dt esured by δ(s) nd the roughness of the souton esured by η(s). Interpotng = zz (2) (3) spnes re ssocted wth λ=. The spest eber of ths fy, the thn-pte spne, corresponds to =2 nd tkes the for of: f ( x, y) = K( x, y) x + + 3y (4) where: K ( x, y ) = 2 r n( 2 r 6π ) (5) r = ( x x ) + ( y y ) (6) The spne preters n equton (4) re deterned by sovng the foowng syste of ner sutneous equtons: A where: A = wth K = P= = f (7) N N N K + λi P P, = N Q P K ( x, y )... K ( x, y ) K ( x, y )... K ( x, y ) x y,f = N f f Q P (9) () x y nd I s the by dentty trx. 2.2 Nuerc Coputton (8) There re soe ssues to be ddressed, fro the nuerc pont of vew, bout ccutng thn-pte spnes. Frst of, the nuber of coeffcents tht needs to be deterned s drecty proporton to the nuber of dt ponts nd therefore, for rge dt ponts (greter thn few hundreds), the coputton woud be very expensve. Secondy, the coeffcent trx A s dense

3 nd -condtoned (the dgon eeents re zero for nterpotng spne - λ = - whe the off-dgon eeents cn be qute rge). Athough these ssues cn cuse serous probes for fttng the scttered dt, they do not hve ny serous consequences for the dt over rectngur grd (ucky, coputer vson probes re often set on rectngur grd). In fct, spe Udecoposton cn esy nd reby sove the probe where the dt re on rectngur grd. The nterestng pont, not to be overooked, s tht the coeffcent trx A s ony functon of x nd y nd not of the vue of the functon t the dt ponts (ssung tht the soothng preter λ s known). Therefore, to sove for the preters of dfferent spnes over the se grds, the nverse of the A hs to be ccuted once nd ths cn be done pror to the strt of optc fow coputton. So, n recoverng the oton fed, one ust choose the se sze for the ptches to expot the dvntge of ths feture (t s, n ny cse, coon to hve ptches of equ sze n ost of the optc fow ethods). In vew of the bove consdertons, trx ( by ) by vector ( by ) utpcton s the ony on-ne coputton requred to ccute the preters of spne functon representng the brghtness dt of ptch n n ge. 3 Probe Defnton The optc fow probe for two sequent ges s often foruted s foows. Gven two ges, we ssue the second ge I 2 (x,y) s fored by ocy dspcng the frst ge I (x,y) by ( X, Y) nd therefore we hve: I ( x + X, y + Y) = I ( x, y ) (). 2 The gener probe s to recover the dspceent fed X nd Y. Expndng I (x,y) n ts frst order Tyor seres for resuts n the we-known ge brghtness constrnt wrtten n ters of dspceents nsted of veoctes (Fenne & Thopson, 979): X I ( x, y ) Y I ( x, y ) + = I2 ( x, y ) I ( x, y ) x y (2) Ths equton shows tht for every pont n the ge functon, there exsts two unknowns nd one constrnt, nd, consequenty, the nuber of soutons re unted (-posed probe). In order to sove ths probe, n extr ssupton hs to be de whch dffers n vrous optc fow gorths. ne wy of reducng the nuber of soutons to the bove probe s to ssue tht, for every pxe n gven ge, there exsts s spt neghbourhood over whch the oton cn be pproxted by pner trnston. Ths s the ost spe nd yet ost coon ode of oton. For s ptches, trnston ode s retvey good pproxton for the fow fed. The we known xu kehood estton gves the souton of the stted optc fow probe by nsng, over X nd Y, the ordnry est squred error E defned s: E = ( I ( x+ X, y+ Y ) I2 ( x, y )) 2 (3). The est squre pproxton s xu kehood estton f the errors re ndependent nd nory dstrbuted wth constnt stndrd devton. Aso, by usng the ordnry est squres pproch, one ssues tht the esureents n x nd y coordntes re error free nd the nose s ony dstrbuted n the ntensty esureent. Ths nston probe usuy hs ny oc soutons nd dfferent ethods cn be epoyed to fnd the opt souton. In the foowng secton the ethodoogy for sovng ths nston probe s expned. 3. Expct Souton ne of the dvntge of representng the ge ntensty dt usng spnes s tht t provdes opportunty to expcty sove the est squre probe (3) for the unknown trnston coponents. Snce the thetc descrpton of I (x,y ) s expressed by ts thn-pte spne representton (equton 4), we cn set the prt dervtves of the ordnry est squre error (equton 3) wth respect to X nd Y to zero. Ths resuts n syste of two non-ner sutneous equtons whch cn be soved usng the Newton-Rphson ethod. It s portnt to note here tht the second order dervtves re defned everywhere except on the dt ponts. Therefore, to use the Newton-Rphson ethod, whch requres the ccuton of the second order dervtves, the coputton hs to be perfored on soe ponts n the spt neghbourhood of the grd ponts but not on the dt ponts. In our peentton, we hve ccuted the dervtves on grd shfted up nd rght by hf the horzont (or vertc) dstnce between two dt ponts. 4 Perfornce Any gorth for esttng the fow fed of 3-D structured scene s key to f due to occudng boundres nd or ck of texture n specfc ptch. Therefore, rebe gorth hs to hve soe ens for esurng the confdence ssocted wth every esttes. Athough nvestgtng ore sutbe confdence esure s the subject of our current work,

4 we hve used very spe confdence esure n the resuts presented here. Accordng to ths esure (C see equton 8), the esttes for whch ther est squre error E (equton 5) dvded by the su of the dt ponts n ptch s ore thn.4 re regrded s outers. C = E (8) I ( x, y) The proposed confdence esure s ony one of ny (perhps the ost spe one) tht coe to nd. A ore sutbe nd effectve confdence esure w contrbute to better resuts both n ters of verge error nd densty. We w copre our resuts wth the resuts for the Feet nd Jepson (99) gorth (s the ost ccurte ethod) nd the Nge (983) gorth (becuse t theoretcy s the cosest ethod to ours). 4. Theoretc Perfornce The theoretc perfornce of every gorth cn be esured by ppyng the gorth to set of synthetc nputs. The true oton fed for these nputs s known ccurtey nd, therefore, the quntttve perfornce cn be esy studed. As ndcted by Brron et. [], the esure of perfornce on synthetc ges shoud be tken s n optstc bound on the expected errors wth re ge sequences. Foowng Brron's pper [], to study the theoretc perfornce of the presented gorth, we present nd nyse the resuts n ge sequences known s Snusod []. The Snusod ge sequence s creted by superposng two snusod pne wves wth spt wveengths of 6 pxes, orentton of 54 nd -27 (wth horzont xs) nd speeds of.63 nd.2 pxes/fre, respectvey. Therefore, the resutng pd pttern trnstes wth veocty of.585 n horzont nd.863 n vertc drectons. The foowng fgures show spe of ths sequence, foowed by the error nyss on the resuts whch re tbed for the ese of coprson. The errors re esured n degrees whch represent the nge between the estted nd the true oton vector n the hoogeneous coordntes []. Fgure 4. A spe ge of Snusod sequence Agorth (ptch sze) Avg. Error Std. Dev. Densty Nge % Feet & Jepson.3. % Spne Bsed (5x5) % Spne Bsed (x) % Spne Bsed (5x5) % Tbe 4. Snusod resuts on trnston speeds 4.2 Re Dt Perfornce In ths secton, the ccuted optc fow fed for four we-known re ge sequences re presented. Athough the exct oveent of dfferent objects s not known, the nuber of ovng objects nd ther oton drectons re gven. A spe ge wth ts ccuted fow fed over unfor rectngur grd nd bref descrpton of ech ge sequences w be presented n the foowng subsectons. A of the ge sequences used n ths survey re pubcy vbe (see Brron et. 994 for ther orgn) SRI Sequence: SRI tree sequence s one of the ost chengng sequences due to ts ow contrst, ots of occuson nd retvey poor resouton. The cer n ths sequence oves n front of custer of trees pre to the ground pne (nor to ts ne of sght). xu veocty s bout 2 pxes/fre. The unfor speed of the ground nd retvey rge veoctes ssocted wth the ner by tree s cery shown n the estted fow fed. Fgure 4.3 A spe ge of SRI ge sequence nd ts estted optc fow fed Hburg Tx: Ths ge sequence shows the trffc n street cross juncton where tx turnng towrd rght, cr oves fro rght to eft n the ower eft opposte to vn drvng rght to eft nd pedestrn wkng n the upper eft of the scene. The speeds of these ovng objects re roughy,3,3 nd.3

5 pxes/fre, respectvey. The estted fow fed contns ost of the bove entoned otons. expresson for ge ntensty functon whch cn then be dfferentted expcty. Usng spnes so ows the degree of soothng to be chosen by we-founded ethods (such s genersed cross-vdton) nd w be ddressed esewhere. 6 Acknowedgent Fgure 4.5 A spe of Hburg Tx ge sequence nd ts estted optc fow fed Rubk Cube: In ths ge sequence, Rubk cube s pced on turntbe whch s rottng n front of sttonry bckground. The veoctes on the turntbe s round.2 to.4 pxes/fre nd on the cube tsef s round.2 to.5 pxes/fre. Exstence of few rrows whch re representng the veocty on the bck ground (where there s no dstnct texture) shows tht the used confdence esure s unrebe. Fgure 4.7 A spe ge of Rubk Cube sequence nd ts estted optc fow fed NASA Sequence: The oton fow fed n ths ge sequence s ny dtton. The cer oves towrd the Coc Co cn, ner the centre of ge. The dvergence fow fed cn be seen n the foowng estted oton feds. Durng the course of ths work, the frst uthor ws fuy supported by schorshp fro the nstry of Cuture nd Hgher Educton, Irn. The uthors woud ke to grtefuy cknowedge Dr. H. Aehossen for n ery dscusson on cubc spne soothng. 7 References [] Brron J.., Feet D.J., Beuchen S.S. 994 Systes nd experent perfornce of optc fow technques Intern. J. Coput. Vs. 2: [2] Suter D. 994 oton estton nd vector spnes Procd. CVPR'94, pp , IEEE Coputer Socety, Sette, Wshngton. [3] Szesk R., Coughn J. 994 Herrchc spnebsed ge regstrton Procd. CVPR'94, pp 94-2, IEEE Coputer Socety, Sette, Wshngton. [4] enguet J. 979 utvrte nterpoton t rbtrry ponts de spe Journ. of Apped thetc Physcs, [5] Whb G. 979 How to sooth curves nd surfces wth spnes nd cross-vdton, Procd. of 24th Conf. on Desgn of Experents, pp 67-92, US ry Reserch ffce. [6] Feet D.J., Jepson A.D. 99 Coputton of coponent ge veocty fro oc phse nforton Intern. J. Coput. Vs. 5: [7] Nge H.H. 983 Dspceent vectors derved fro second order ntensty vrtons n ge sequences Coput. Grph. Ige Process. 2: Fgure 4.9. A spe ge of NASA sequence nd ts estted optc fow fed 5 Concuson We hve deonstrted tht good ccurcy cn be cheved by repcng n d hoc Gussn soothng of grey eve vues (pror to dfferentton) by spne fttng. The spne fttng ethod provdes n gebrc