Outline. Lecture 12: Multi-view geometry / Stereo III. Review: the essential matrix. Review: disparity/depth maps

Transcription

1 Outline Lectue : ulti-vie geomet / Steeo III Tuesda, Oct CS 78/95T of. Kisten Gauman Last lectue: steeo econstuction ith calibated cameas non-geometic coesondence constaints Homogeneous coodinates, ojection matices Camea calibation Weak calibation/self-calibation Fundamental mati 8-oint algoithm evie: steeo ith calibated cameas evie: the essential mati Vecto in second cood. ss. has coodinates in the fist one. Let E [ T ( )] [ T] [ T] E Camea-centeed coodinate sstems ae elated b knon otation and tanslation T. E is the essential mati, hich elates coesonding age oints in both cameas, given the otation and tanslation beteen thei coodinate sstems. evie: steeo ith calibated cameas evie: disait/deth mas Image ai Detect some featues Comute E fom and T atch featues using the eiola and othe constaints Tiangulate fo d stuctue age I(,) Disait ma D(,) (, )(+D(,),) age I (, )

2 evie: coesondence oblem ultile match hotheses satisf eiola constaint, but hich is coect? evie: coesondence oblem To find matches in the age ai, assume ost scene oints visible fom both vies Image egions fo the matches ae sila in aeaance Dense o sase matches Additional (non-eiola) constaints: Silait Uniqueness Odeing Figual continuit Disait gadient Figue fom Gee & Ciolla 999 evie: coesondence eo souces Lo-contast / tetueless age egions Occlusions Camea calibation eos oo age esolution Violations of bightness constanc (secula eflections) Lage motions Homogeneous coodinates Etend Euclidean sace: add an eta coodinate oints ae eesented b equivalence classes Wh? This ill allo us to ite ocess of esective ojection as a mati d: (, ) (,, ) d: (,, z) (,, z, ) aing to homogeneous coodinates d: (,, ) (/, /) d: (,, z, ) (/, /, z/) aing back fom homogeneous coodinates Homogeneous coodinates esective ojection equations Equivalence elation: (,, z, ) is the same as (k, k, kz, k) Homogeneous coodinates ae onl defined u to a scale Image lane Focal length Camea fame Otical ais Scene oint Image coodinates

3 ojection mati fo esective ojection ojection mati fo esective ojection Fom inhole camea model Fom inhole camea model fx Z fy Z Same thing, but itten in tems of homogeneous coodinates ojection mati fo othogahic ojection Camea aametes Etinsic: location and oientation of camea fame ith esect to efeence fame Intinsic: ho to ma iel coodinates to age lane coodinates efeence fame X Y Camea fame igid tansfomations Combinations of otations and tanslation Tanslation: add values to coodinates otation: mati multilication otation about coodinate aes in d Eess d otation as seies of otations aound coodinate aes b angles α, β, γ Oveall otation is oduct of these elementa otations: z

4 Etinsic camea aametes c oint in camea efeence fame ( X, Y Z ) c, ( T) oint in old Camea aametes Etinsic: location and oientation of camea fame ith esect to efeence fame Intinsic: ho to ma iel coodinates to age lane coodinates efeence fame Camea fame Intinsic camea aametes Ignoing an geometic distotions fom otics, e can descibe them b: Coodinates of ojected oint in camea efeence fame ( o ) s ( o ) s Coodinates of age oint in iel units Coodinates of age cente in iel units Effective size of a iel (mm) Camea aametes We kno that in tems of camea efeence fame: Substituting evious eqns descibing intinsic and etinsic aametes, can elate iels coodinates to old oints: ( ( o ) s o ) s f f ( ( ( ( T) T) T) T) i o i of otation mati Linea vesion of esective ojection equations This can be eitten as a mati oduct using homogeneous coodinates: f / s o int f / s o et int et Y X Z / / oint in camea coodinates T T T Calibating a camea Comute intinsic and etinsic aametes using obseved camea data ain idea lace calibation object ith knon geomet in the scene Get coesondences Solve fo maing fom scene to age: estate int et

5 Linea vesion of esective ojection equations in homog. Estating the ojection mati This can be eitten as a mati oduct using homogeneous coodinates: / / int et Y X Z oduct is single ojection mati encoding both etinsic and intinsic aametes Let i be o i of mati ( ) ( ) Estating the ojection mati Fo a given featue oint: In mati fom: ( ) ( ) T Stack os of mati Estating the ojection mati ( ( ) ) Eanding this to see the elements: X Y Z X Y Z X Y Z X Y Z T Estating the ojection mati This is tue fo eve featue oint, so e can stack u n obseved age featues and thei associated d oints in single equation: m m X Y Z X X Y Z X Y Y X Y Z X X Y Z X Y Y Z Z Z Z Summa: camea calibation Associate age oints ith scene oints on object ith knon geomet Use togethe ith esective ojection elationshi to estate ojection mati (Can also solve fo elicit aametes themselves) Solve fo m ij s (the calibation infomation) ith least squaes. [F& Section.]

6 When ould e calibate this a? akes sense hen geomet of sstem is not going to change ove te When ould it change? Self-calibation Want to estate old geomet ithout equiing calibated cameas Achival videos hotos fom multile unelated uses Dnamic camea sstem We can still econstuct d stuctue, u to cetain ambiguities, if e can find coesondences beteen oints Uncalibated case ( ) left left, int ( left ight,int ( ) ( ight ight ) ) Uncalibated case: fundamental mati So: Camea coodinates int ( ) left left, int ( left ) ( ight ) ight,int ( ) Intenal calibation matices et Image iel coodinates Doed subscit, still intenal aamete matices ( ight) E( left ) Fom befoe, the essential mati ( ightight ) E( leftleft ) ( E ) ight ight F ight left left Fundamental mati left Fundamental mati elates iel coodinates in the to vies oe geneal fom than essential mati: e emove need to kno intinsic aametes If e estate fundamental mati fom coesondences in iel coodinates, can econstuct eiola geomet ithout intinsic o etinsic aametes Comuting F fom coesondences F ( ight E ) left ight left Cameas ae uncalibated: e don t kno E o left o ight int matices Estate F fom 8+ oint coesondences. F

7 Comuting F fom coesondences Each oint coesondence geneates one constaint on F Collect n of these constaints F ight left obust comutation Find cones Unguided matching local seach, cosscoelation to get some seed matches Comute F and eiola geomet: find F that is consistent ith man of the seed matches No guide matching: using F to estict seach to eiola lines Invet and solve fo F. O, if n > 8, least squaes solution. ANSAC alication: obust comutation Inteest oints (Hais cones) in left and ight ages about 5 ts / age 6448 esolution utative coesondences (68) (Best match,ssd<) Outlies (7) (t.5 iel; 4 iteations) Inlies (5) Final inlies (6) Hatle & Zisseman. 6 Figue b Gee and Ciolla, 999 Need fo multi-vie geomet and d econstuction Alications including: d tacking Deth-based gouing Image endeing and geneating inteolated o vitual vieoints Inteactive video Z-keing fo vitual ealit ege snthetic and eal ages given deth mas Kanade et al., CU, 995

8 Z-keing fo vitual ealit Vitualized ealit T Catue d shae fom multile vies, tetue fom ages Use them to geneate ne vies on demand htt://.cs.cmu.edu/afs/cs/oject/steeo-machine//z-ke.html Kanade et al, CU htt://.cs.cmu.edu/~vitualized-ealit/manbball_ne.html Vitual vieoint video Vitual vieoint video C. Zitnick et al, High-qualit video vie inteolation using a laeed eesentation, SIGGAH 4. htt://eseach.micosoft.com/iv/vvv/ Noah Snavel, Steven. Seitz, ichad Szeliski, "hoto touism: Eloing hoto collections in D," AC Tansactions on Gahics (SIGGAH oceedings), 5(), 6, htt://hototou.cs.ashington.edu/, htt://labs.live.com/hotosnth/

9 Coming u Tuesda: Local invaiant featues ead Loe ae on SIFT oblem set out net Tuesda, due / Gaduate students: emembe ae evies and etensions, due /6