Opical Flow Sall oio: ( ad ae le ha piel) H() I(++) Be foce o poible ppoe we ake he Talo eie epaio of I: (Sei) Opical flow eqaio Cobiig hee wo eqaio I he lii a ad go o eo hi becoe eac (Sei)
Opical flow eqaio Q: how a kow ad eqaio pe piel? Iiiel wha doe hi coai ea? The copoe of he flow i he gadie diecio i deeied The copoe of he flow paallel o a edge i kow Thi eplai he Babe Pole illio (Sei) hp://wwwadlociececo/abigo/babepoleh Le look a a eaple of hi Sppoe we hae a iage i which H() Tha i he iage will look like: Ad ppoe hee i opical flow of () The ew iage will look like: ----------------------- - - I() H() So I () - GRAD I() () So o coai eqaio will be: - + <() ()> which i We ecoe he copoe of he opical flow b o he copoe Thi i he apee poble
Fi Ode Appoiaio Whe we ae: I I I( + + ) I( ) + + We ae a iage locall i: (Sei) Apee poble (Sei)
Apee poble (Sei) Solig he apee poble How o ge oe eqaio fo a piel? Baic idea: ipoe addiioal coai o coo i o ae ha he flow field i ooh locall oe ehod: peed he piel eighbo hae he ae () If we e a 55 widow ha gie 5 eqaio pe piel! (Sei) 4
Lka-Kaade flow We hae oe eqaio ha kow: ole lea qae poble Thi i gie b: Saio oe all piel i he KK widow Doe look failia? (Sei) Le look a a eaple of hi Sppoe we hae a iage wih a coe ----------------- Ad hi alae dow ad o he igh: - - 4444444 - Le cope I fo he whole ecod iage: ---------- I ---------- I ------------- ----- -- -------------- ------ --5 --5------ ------- --5 --5----- The he eqaio we ge hae he fo: (5-5)*() ()*() (-)() Togehe hee lead o a olio ha - 5
Codiio fo olabili Opial ( ) aifie Lca-Kaade eqaio Whe i Thi Solable? A T A hold be ieible A T A hold o be oo all de o oie eigeale λ ad λ of A T A hold o be oo all A T A hold be well-codiioed λ / λ hold o be oo lage (λ lage eigeale) (Sei) Weak pepecie (caled ohogaphic pojecio) Ie pepecie effec b o oe he cale of idiidal objec collec poi io a gop a abo he ae deph he diide each poi b he deph of i gop (Foh & Poce) 6
7 Pepecie -> Scaled Ohogaphic Recall: ( i i i ) -> ( i / i i / i ) Le Z ( + + + )/ The ( i i i ) appo-> ( i /Z i /Z) The Eqaio of Weak Pepecie ) ( ) ( i coa fo all poi Paallel lie o loge coege he eai paallel Z Y X
Po ad Co of Thee Model Weak pepecie ch iple ah Accae whe objec i all ad dia Mo efl fo ecogiio Pihole pepecie ch oe accae fo cee Ued i ce fo oio Whe accac eall ae odel eal caea Fi: Repee oio We ll alk abo a fied caea ad oig objec Ke poi: P Poi The: Soe ai S The iage I I SP 8
Sce-fo-Moio S ecode: Pojecio: ol wo lie Scalig ice S ca hae a cale faco Talaio b / ad / Roaio: I SP Roaio P Repee a D oaio of he poi i P 9
Fi look a D oaio (eaie) Mai R ac o poi b oaig he coθ R iθ iθ coθ coθ iθ iθ coθ Alo RR T Idei R T i alo a oaio ai i he oppoie diecio o R Siple D Roaio coθ iθ iθ coθ Roaio abo ai Roae coodiae Leae coodiae fied
Fll D Roaio coθ R iθ iθ coθ co β i β i β co β coα iα iα coα A oaio ca be epeed a cobiaio of hee oaio abo hee ae T RR Row (ad col) of R ae ohooal eco R ha deeia (o -) Scale ( Pojecio whee ( Pig i Togehe ) D Talaio ( ) ( P ) ) D Roaio P We ca j wie a ad a
Affie Sce fo Moio ) ( ) ( ) ( ) ( whee P Affie Sce-fo-Moio I S P
Fi Sep: Sole fo Talaio () Thi i iial becae we ca pick a iple oigi Wold oigi i abia Eaple: We ca ae fi poi i a oigi Roaio he doe effec ha poi All i oio i alaio Bee o pick cee of a a oigi Aeage of all poi Thi alo aeage all oie Fi Sep: Sole fo Talaio () : WLOG i i i i k i k i k i k i I
4 Fi Sep: Sole fo Talaio () A if b agic hee o alaio Rak Theoe I I I ha ak Thi ea hee ae eco ch ha ee ow of i a liea cobiaio of hee eco Thee eco ae he ow of P S P
Sole fo S SVD i ade o do hi D i diagoal wih o-iceaig I UDV ale U ad V hae ohooal ow Igoig ale ha ge e o we hae U(::) fo S ad D(::)*V(::) fo P I I Liea Abigi U(::) * D(::) * V(::) (U(::) * A) * (i(a) *D(::) * V(::)) 5
6 Noie ha fll ak Be olio i o eiae I ha a ea o a poible wih eiae of I haig ak O ce ehod doe hi I I Weak Pepecie Moio I S P Row k ad k+ of S hold be ohogoal All ow hold be i eco (Ph all cale io P) (U(::)*A)*(i(A) *D(::)*V(::)) Chooe A o (U(::) * A) aifie hee codiio I
Mli-objec Moio Sk Moio of Objec k P Poi of Objec k I k [ S S S ] P P P Gie eogh iage I will hae ak 4 Mli-bod Facoiaio Mai iigh Coide col of I coepodig o idiidal poi If wo col coe fo diffee objec he ae elaed Thei ie podc i likel o be all If fie col coe fo he ae objec i geeal he fifh will be a liea cobiaio of he ohe fo 7
I I I I The pecip deoe ha coodiae ( ) Facoiaio We ewie he iage I i a diffee coodiae e I col k of I Roae pace o ha he fi coodiae i i he diecio of I he fac ha we hae pefoed oe oaio Afe hi oaio he fi col ad he ohe col hae abia coodiae Ne we oae o he ecod coodiae i i he diecio of ha copoe of I I k I I I ha i ohogoal o I A we coie hi poce we each a col ha belog o he ae objec a fo peio col Thi col cao poide a ew ai of he coodiae e Iead i coodiae will be o-eo i he ow ha ae o-eo fo he fo peio col fo hi objec ad will be eo (o all) i ohe ow Thi allow o eadil deec he epaael oig objec i he cee 8