EPIPOLAR GEOMETRY WITH MANY DETAILS

Transcription

1 EPIPOLAR GEOMERY WIH MANY DEAILS

2 hank ou for the slides. he come mostl from the following source. Marc Pollefes U. of North Carolina

3 hree questions: (i) Correspondence geometr: Given an image point in the first view, how does this constrain the position of the corresponding point in the second image? (ii) Camera geometr (motion): Given a set of corresponding image points { i i }, i,,n, what are the cameras P and P for the two views? (iii) Scene geometr (structure): Given corresponding image points i i and cameras P, P, what is the position of (their pre-image) X in space?

4 he epipolar geometr C,C,, and X are coplanar

5 he epipolar geometr What if onl C,C, are known?

6 he epipolar geometr All points on π project on l and l

7 he epipolar geometr Famil of planes π and lines l and l Intersection in e and e

8 he epipolar geometr epipoles e,e intersection of baseline with image plane projection of projection center in other image vanishing point of camera motion direction an epipolar plane plane containing baseline (-D famil) an epipolar line intersection of epipolar plane with image (alwas come in corresponding pairs)

9 Eample: converging cameras

10 Eample: motion parallel with image plane

11 Eample: forward motion e e

12 he fundamental matri F algebraic representation of epipolar geometr a l' we will see that mapping is (singular) correlation (i.e. projective mapping from points to lines) represented b the fundamental matri F

13 he fundamental matri F geometric derivation ' H π [ e' ] Hπ F l' e' ' mapping from -D to -D famil (rank )

14 he fundamental matri F algebraic derivation ( λ) P λc X + + ( + P P I) l P'C P' P + P + X( λ) F [ ] + e' P' P (note: doesn t work for CC F0)

15 he fundamental matri F correspondence condition he fundamental matri satisfies the condition that for an pair of corresponding points in the two images ' F 0 ( ' l' 0)

16 he fundamental matri F F is the unique 33 rank matri that satisfies F0 for all (i) ranspose: if F is fundamental matri for (P,P ), then F is fundamental matri for (P,P) (ii) Epipolar lines: l F & lf (iii) Epipoles: on all epipolar lines, thus e F0, e F0, similarl Fe0 (iv) F has 7 d.o.f., i.e. 33-(homogeneous)-(rank) (v) F is a correlation, projective mapping from a point to a line l F (not a proper correlation, i.e. not invertible)

17 Epipolar line homograph for e, for e, 3 for homograph of a line from e to e

18 he epipolar line geometr l,l epipolar lines, k line not through e l F[k] l and smmetricall lf [k ] l l l k k e e' l Fk (pick ke, since e e 0) F[ e] l l F [ e' ] l' l'

19 Fundamental matri for pure translation

20 Fundamental matri for pure translation

21 Fundamental matri for pure translation [ e' ] H [ ] F e' ( H K RK) eample: e' (,0,0) F ' F 0 ' PX ' P' X K[I 0]X K - K[I t] Z ( X,Y,Z) - K /Z ' + Kt/Z motion starts at and moves towards e, faster depending on Z pure translation: F onl d.o.f., [e] 0 auto-epipolar

22 General motion [ e' ] H 0 [ e' ] ˆ 0 ' ' ' K'RK - + K' t/z

23 Projective transformation and invariance Derivation based purel on projective concepts ˆ H, ˆ' H' ' Fˆ PX ( )( - PH H X) Pˆ Xˆ ( )( - P' H H X) Pˆ' Xˆ H' - FH - F invariant to transformations of projective 3-space ' P' X ( P, P' ) a F F a ( P,P' ) canonical form P [I 0] P' [M m] unique not unique [ m] M F

24 Projective ambiguit of cameras given F previous slide: at least projective ambiguit this slide: not more! ~ ~ Show that if F is same for (P,P ) and (P,P ), there eists a projective transformation H so that ~ ~ PHP and P HP lemma: H P' H ~ P [I 0] P' [A a] P [I 0] P ~ ' F [ a] A [ ~ a] A ~ ~ A ~ a ka k A + av ( ) rank [A ~ ~ a] af a[] a A 0 ~ af ~ a ka a A ~ a A ~ a ka ~ - A 0 ka ~ - A av [ ] [ ] [ ] ( ) ( ) k I 0 k v k [A a] k I k v 0 k [ k ( A - av ) ka] P ~ ' (-57, ok)

25 Canonical cameras given F F matri corresponds to P,P iff P FP is skew-smmetric ( X P' FPX 0, X) F matri, S skew-smmetric matri P [I 0] P' [SF e'] [SF e'] (fund.matrif) F[I 0] F S F e' F 0 0 F S 0 F 0 0 Possible choice: P [I 0] P' [[e'] F e'] Canonical representation: P [I 0] P' [[e'] F + e' v λe']

26 he essential matri ~fundamental matri for calibrated cameras (remove K) E ˆ ' [ t] R R[R t] Eˆ ( ˆ K ; ˆ' K ' ) E K' FK 5 d.o.f. (3 for R; for t up to scale) E is essential matri if and onl if two singularvalues are equal (and third0) E Udiag(,,0)V

27 Four possible reconstructions from E (onl one solution where points is in front of both cameras)

28 Epipolar geometr: basic equation ' F 0 ' f + ' f + ' f3 + ' f + ' f + ' f3 + f3 + f3 + f33 0 separate known from unknown [ ', ', ', ', ', ',,,][ f, f, f, f, f, f, f, f, f ] 0 (data) (unknowns) (linear) ' ' ' ' ' ' M M M M M M M M ' n n ' n n ' n ' n n ' n n ' n n n M f 0 Af 0

29 f f f f f f f f f n n n n n n n n n n n n M M M M M M M M M ~0000 ~0000 ~0000 ~0000 ~00 ~00 ~00 ~00! Orders of magnitude difference Between column of data matri least-squares ields poor results the NO normalized 8-point algorithm

30 the normalized 8-point algorithm ransform image to ~[-,][-,] (0,500) (700,500) (-,) (0,0) (,) (0,0) (700,0) (-,-) (,-) Least squares ields good results (Hartle, PAMI 97)

31 the singularit constraint 0 F e' 0 Fe 0 detf F rank V U σ V U σ U σ V V σ σ σ U F + + SVD from linearl computed F matri (rank 3) V U σ U σ V V 0 σ σ U F' + F min F- F' Compute closest rank- approimation

32

33 the minimum case 7 point correspondences 0 f ' ' ' ' ' ' ' ' ' ' ' ' M M M M M M M M M ( ) V,0,0,...,σ diag σ A U V ] A[V ( ) 8 [ ] V e.g.v...7 0, λf ) (F + i i i one parameter famil of solutions but F +λf not automaticall rank

34 the minimum case impose rank σ 3 (obtain or 3 solutions) F 7pts F F F 3 det( F + λf ) a 3λ + aλ + aλ + a0 0 (cubic equation) - det( F + λf ) det F det(f F + λi) 0 - F F Compute possible λ as eigenvalues of (onl real solutions are potential solutions)

35 Gold standard Maimum Likelihood Estimation (, ˆ ) d( ', ˆ' ) d i i + i i i ( least-squares for Gaussian noise) subject to ˆ' Fˆ 0 Initialize: normalized 8-point, (P,P ) from F, reconstruct X i Parameterize: P [I 0],P' [M t],x i ˆ PX, ˆ i i i P' X Minimize cost using Levenberg-Marquardt (preferabl sparse LM, see book) i (overparametrized)

36 Gold standard Alternative, minimal parametrization (with a) (note (,,) and (,,) are epipoles) problems: a0 pick largest of a,b,c,d to fi epipole at infinit pick largest of,,w and of,,w parametrizations! reparametrize at ever iteration, to be sure

37 First-order geometric error (Sampson error) ( ) e JJ e e e JJ (one eq./point JJ scalar) e ' F 0 e i ' F0 0 ( ' F) + ( ' F) + ( F) ( ) F JJ + e e JJ ( ' F) + ( ' F) + ( F) + ( F) ' F (problem if some is located at epipole) advantage: no subsidiar variables required

38 Smmetric epipolar error ( F ) d( ',, F ' ) d i i + i i ' F + ( ' F) + ( ) ( F) + ( F) ' F i

39 Some eperiments:

40 Some eperiments:

41 Some eperiments:

42 Some eperiments: Residual error: ( F ) d( ',, F ' ) d i i + i (for all points!) i i

43 Recommendations:. Do not use unnormalized algorithms. Quick and eas to implement: 8-point normalized 3. Better: enforce rank- constraint during minimization 4. Best: Maimum Likelihood Estimation (minimal parameterization, sparse implementation)

44 Feature points Etract feature points to relate images Required properties: Well-defined (i.e. neigboring points should all be different) Stable across views (i.e. same 3D point should be etracted as feature for neighboring viewpoints)

45 Degenerate cases: Degenerate cases Planar scene Pure rotation No unique solution Remaining DOF filled b noise Use simpler model (e.g. homograph) Model selection (orr et al., ICCV 98, Kanatani, Akaike) Compare H and F according to epected residual error (compensate for model compleit)

46 More problems: Absence of sufficient features (no teture) Repeated structure ambiguit Robust matcher also finds support for wrong hpothesis solution: detect repetition (Schaffalitzk and Zisserman, BMVC 98)

47 two-view geometr geometric relations between two views is full described b recovered 33 matri F

48 Image pair rectification simplif stereo matching b warping the images Appl projective transformation so that epipolar lines correspond to horizontal scanlines e e map epipole e to (,0,0) tr to minimize image distortion problem when epipole in (or close to) the image

49 Planar rectification (standard approach) ~ image size (calibrated) Bring two views to standard stereo setup (moves epipole to ) (not possible when in/close to image) Distortion minimization (uncalibrated)

50

51