Optical flow. Visual motion. Motion and perceptual organization. Motion and perceptual organization. Subhransu Maji. CMPSCI 670: Computer Vision

Transcription

1 Visal moio Opical flow Sbhras Maji CMPSC 670: Comper Visio Ocober 0, 06 Ma slides adaped from S. Seiz, R. Szeliski, M. Pollefes CMPSC 670 Moio ad percepal orgaizaio Moio ad percepal orgaizaio Someimes, moio is he ol ce CMPSC 670 Someimes, moio is he ol ce 3 CMPSC 670 4

2 Moio ad percepal orgaizaio Ee impoerished moio daa ca eoke a srog percep Uses of moio Segmeig objecs based o moio ces Esimaig he 3D srcre Learig ad rackig damical models Recogizig ees ad aciiies G. Johasso, Visal Percepio of Biological Moio ad a Model For s Aalsis, Percepio ad Pschophsics 4, 0-, 973. CMPSC CMPSC Moio field Opical flow The moio field is he projecio of he 3D scee moio io he image Defiiio: opical flow is he appare moio of brighess paers i he image deall, opical flow wold be he same as he moio field Hae o be carefl: appare moio ca be cased b lighig chages wiho a acal moio Thik of a iform roaig sphere der fied lighig s. a saioar sphere der moig illmiaio CMPSC CMPSC 670 8

3 Esimaig opical flow The brighess cosac cosrai Gie wo sbseqe frames, esimae he appare moio field, ad, bewee hem,,,,,,,, Brighess Cosac Eqaio:,, = +,, +,, Ke assmpios Brighess cosac: projecio of he same poi looks he same i eer frame Small moio: pois do o moe er far Spaial coherece: pois moe like heir eighbors Liearizig he righ side sig Talor epasio:,,,, +,, Hece, CMPSC CMPSC The brighess cosac cosrai - How ma eqaios ad kows per piel? Oe eqaio, wo kows + + = 0 Wha does his cosrai mea?, + = 0 The compoe of he flow perpediclar o he gradie i.e., parallel o he edge is kow The brighess cosac cosrai - How ma eqaios ad kows per piel? Oe eqaio, wo kows + + = 0 Wha does his cosrai mea?, + = 0 The compoe of he flow perpediclar o he gradie i.e., parallel o he edge is kow f, saisfies he eqaio, so does +, + if ', ' = 0 gradie,, +,+ edge CMPSC 670 CMPSC 670

4 The aperre problem The aperre problem Perceied moio Acal moio The aperre problem The barber pole illsio Wha direcio is he moio? hp://e.wikipedia.org/wiki/barberpole_illsio

5 CMPSC 670 How o ge more eqaios for a piel? Spaial coherece cosrai: preed he piel s eighbors hae he same, E.g., if we se a 55 widow, ha gies s 5 eqaios per piel Solig he aperre problem 7 B. Lcas ad T. Kaade. A ieraie image regisraio echiqe wih a applicaio o sereo isio. Proceedigs of he eraioal Joi Coferece o Arificial elligece, pp , ], [ = + i i = CMPSC 670 Leas sqares problem: Solig he aperre problem 8 B. Lcas ad T. Kaade. A ieraie image regisraio echiqe wih a applicaio o sereo isio. Proceedigs of he eraioal Joi Coferece o Arificial elligece, pp , 98. Whe is his ssem solable? Wha if he widow coais js a sigle sraigh edge? = CMPSC 670 Bad case: sigle sraigh edge Codiios for solabili 9 CMPSC 670 Good case: corer Codiios for solabili 0

6 CMPSC 670 Liear leas sqares problem Lcas-Kaade flow B. Lcas ad T. Kaade. A ieraie image regisraio echiqe wih a applicaio o sereo isio. Proceedigs of he eraioal Joi Coferece o Arificial elligece, pp , 98. The smmaios are oer all piels i he widow Solio gie b = = b A d b A Ad A T T = = CMPSC 670 Lcas-Kaade flow Recall he Harris corer deecor: M = A T A is he secod mome mari We ca figre o wheher he ssem is solable b lookig a he eigeales of he secod mome mari The eigeecors ad eigeales of M relae o edge direcio ad magide The eigeecor associaed wih he larger eigeale pois i he direcio of fases iesi chage, ad he oher eigeecor is orhogoal o i = CMPSC 670 Visalizaio of secod mome marices 3 CMPSC 670 Visalizaio of secod mome marices 4

7 erpreig he eigeales Eample Classificaio of image pois sig eigeales of he secod mome mari: λ Edge λ >> λ Corer λ ad λ are large, λ ~ λ λ ad λ are small Fla Edge regio λ >> λ CMPSC 670 λ 5 CMPSC 670 Sbhras Maji UMass, * From Fall Khrram 6 Hassa-Shafiqe CAP545 Comper Visio Uiform regio Edge gradies hae small magide small λ, small λ ssem is ill-codiioed gradies hae oe domia direcio large λ, small λ ssem is ill-codiioed CMPSC CMPSC 670 8

8 High-ere or corer regio Opical Flow Resls gradies hae differe direcios, large magides large λ, large λ ssem is well-codiioed CMPSC CMPSC 670 Sbhras Maji UMass, * From Fall 6 Khrram Hassa-Shafiqe CAP545 Comper Visio Errors i Lcas-Kaade Mli-resolio regisraio The moio is large larger ha a piel eraie refieme Coarse-o-fie esimaio Ehasie eighborhood search feare machig A poi does o moe like is eighbors Moio segmeaio Brighess cosac does o hold Ehasie eighborhood search wih ormalized correlaio CMPSC 670 * From Khrram Hassa-Shafiqe CAP545 Comper Visio CMPSC 670 3

9 Opical flow resls Opical flow resls * From Khrram Hassa-Shafiqe CAP545 Comper Visio 003 CMPSC Sae-of-he-ar opical flow CMPSC 670 Khrram Hassa-Shafiqe CAP545 Comper Visio Sbhras Maji UMass,* From Fall 6 Sae-of-he-ar opical flow Sar wih somehig similar o Lcas-Kaade Epic Flow: Feare machig + edge preserig flow ierpolaio + gradie cosac + eerg miimizaio wih smoohig erm + regio machig + kepoi machig log-rage Regio-based +Piel-based +Kepoi-based EpicFlow: Edge-Preserig erpolaio of Correspodeces for Opical Flow, Jerome Read, Philippe Weizaepfel, Zaid Harchaoi ad Cordelia Schmid, CVPR 05. Large displaceme opical flow, Bro e al., CVPR 009 CMPSC 670 Sorce: J. Has 35 CMPSC

10 Feare rackig So far, we hae ol cosidered opical flow esimaio i a pair of images f we hae more ha wo images, we ca compe he opical flow from each frame o he e Gie a poi i he firs image, we ca i priciple recosrc is pah b simpl followig he arrows Trackig challeges Ambigi of opical flow Need o fid good feares o rack Large moios, chages i appearace, occlsios, disocclsios Need mechaism for deleig, addig ew feares Drif errors ma accmlae oer ime Need o kow whe o ermiae a rack + + CMPSC CMPSC Shi-Tomasi feare racker Trackig eample Fid good feares sig eigeales of secod-mome mari Ke idea: good feares o rack are he oes whose moio ca be esimaed reliabl From frame o frame, rack wih Lcas-Kaade This amos o assmig a raslaio model for frame-o-frame feare moeme Check cosisec of racks b affie regisraio o he firs obsered isace of he feare Affie model is more accrae for larger displacemes Comparig o he firs frame helps o miimize drif J. Shi ad C. Tomasi. Good Feares o Track. CVPR 994. J. Shi ad C. Tomasi. Good Feares o Track. CVPR 994. CMPSC CMPSC

11 Trackig b machig Trackig eample hps:// CMPSC CMPSC 670 4