Admiisriia Fial eam: Thrsda, Ma 5, -3pm, Hasbrock 3 Reiew sessio poll Thrsda, April 8, 4-5pm, Locaio: TDB Tesda, Ma 3, 4-5pm, Locaio: TDB CMPSC 370: ro. o Comper Visio Reiew oes are posed o Moodle Opical flow Uiersi of Massachses, Amhers April 6, 06 Hoors secio srcor: Sbhras Maji Toda, 4-5pm 0 mi preseaio Frida, Ma 6, midigh wriep of 4-6 pages Admiisriia Visal moio Coclde deep learig Reiew decisio rees Homework 5 de Thrsda deadlie eeded b das Opical flow SRT forms las 5 mis Need a oleer o ake he forms o he CS mai office? Ma slides adaped from S. Seiz, R. Szeliski, M. Pollefes 3 3 4 4
Moio ad percepal orgaizaio Someimes, moio is he ol ce Moio ad percepal orgaizaio Someimes, moio is he ol ce 5 Moio ad percepal orgaizaio Ee impoerished moio daa ca eoke a srog percep 5 6 Uses of moio Esimaig 3D srcre Segmeig objecs based o moio ces Learig ad rackig damical models Recogizig ees ad aciiies 6 G. Johasso, Visal Percepio of Biological Moio ad a Model For s Aalsis, Percepio ad Pschophsics 4, 0-, 973. 7 8 7 8
Moio field The moio field is he projecio of he 3D scee moio io he image Opical flow Defiiio: opical flow is he appare moio of brighess paers i he image deall, opical flow wold be he same as he moio field Hae o be carefl: appare moio ca be cased b lighig chages wiho a acal moio Thik of a iform roaig sphere der fied lighig s. a saioar sphere der moig illmiaio 9 0 Gie wo sbseqe frames, esimae he appare moio field, ad, bewee hem 9 Esimaig opical flow 0 The brighess cosac cosrai,,,,,,,, Brighess Cosac Eqaio:,, = +,, +,, Ke assmpios Brighess cosac: projecio of he same poi looks he same i eer frame Small moio: pois do o moe er far Spaial coherece: pois moe like heir eighbors Liearizig he righ side sig Talor epasio:,,,, +,, Hece, + + 0 +
The brighess cosac cosrai + + = 0 How ma eqaios ad kows per piel? Oe eqaio, wo kows Wha does his cosrai mea?, + = 0 The compoe of he flow perpediclar o he gradie i.e., parallel o he edge is kow The brighess cosac cosrai + + = 0 How ma eqaios ad kows per piel? Oe eqaio, wo kows Wha does his cosrai mea?, + = 0 The compoe of he flow perpediclar o he gradie i.e., parallel o he edge is kow f, saisfies he eqaio, so does +, + if ', ' = 0 gradie,, +,+ edge 3 The aperre problem 4 The aperre problem Perceied moio Acal moio 5 6 5 6
The barber pole illsio 7 hp://e.wikipedia.org/wiki/barberpole_illsio 7 How o ge more eqaios for a piel? Spaial coherece cosrai: preed he piel s eighbors hae he same, E.g., if we se a 55 widow, ha gies s 5 eqaios per piel Solig he aperre problem 8 0 ], [ = + i i [ Lcas Kaade mehod, 98 ] 8 Leas sqares problem: Solig he aperre problem 9 Whe is his ssem solable? Wha if he widow coais js a sigle sraigh edge? [ Lcas Kaade mehod, 98 ] 9 Bad case: sigle sraigh edge Codiios for solabili 0 0
Good case Codiios for solabili Liear leas sqares problem Lcas-Kaade flow B. Lcas ad T. Kaade. A ieraie image regisraio echiqe wih a applicaio o sereo isio. Proceedigs of he eraioal Joi Coferece o Arificial elligece, pp. 674 679, 98. The smmaios are oer all piels i he widow Solio gie b = b A d b A Ad A T T = Lcas-Kaade flow 3 Recall he Harris corer deecor: M = A T A is he secod mome mari We ca figre o wheher he ssem is solable b lookig a he eigeales of he secod mome mari The eigeecors ad eigeales of M relae o edge direcio ad magide The eigeecor associaed wih he larger eigeale pois i he direcio of fases iesi chage, ad he oher eigeecor is orhogoal o i 3 Visalizaio of secod mome marices 4 4
Visalizaio of secod mome marices erpreig he eigeales Classificaio of image pois sig eigeales of he secod mome mari: λ Edge λ >> λ Corer λ ad λ are large, λ ~ λ λ ad λ are small Fla Edge regio λ >> λ 5 λ 6 5 Eample 6 Uiform regio gradies hae small magide small λ, small λ ssem is ill-codiioed * From Khrram Hassa-Shafiqe CAP545 Comper Visio 0037 8 7 8
Edge High-ere or corer regio gradies hae oe domia direcio large λ, small λ ssem is ill-codiioed gradies hae differe direcios, large magides large λ, large λ ssem is well-codiioed 9 30 9 Opical Flow Resls 30 Errors i Lcas-Kaade The moio is large larger ha a piel eraie refieme Coarse-o-fie esimaio Ehasie eighborhood search feare machig A poi does o moe like is eighbors Moio segmeaio Brighess cosac does o hold Ehasie eighborhood search wih ormalized correlaio * From Khrram Hassa-Shafiqe CAP545 Comper Visio 003 3 3 3 3
Mli-resolio regisraio Opical Flow Resls 33 33 * From Khrram Hassa-Shafiqe CAP545 Comper Visio 003 Opical Flow Resls 34 34 * From Khrram Hassa-Shafiqe CAP545 Comper Visio 003 Sae-of-he-ar opical flow Sar wih somehig similar o Lcas-Kaade + gradie cosac + eerg miimizaio wih smoohig erm + regio ad kepoi machig log-rage Regio-based +Piel-based +Kepoi-based * From Khrram Hassa-Shafiqe CAP545 Comper Visio 003 35 Large displaceme opical flow, Bro e al., CVPR 009 Sorce: J. Has 36 35 36