CS4495/6495 Introduction to Computer Vision. 3D-L3 Fundamental matrix

Transcription

1 CS4495/6495 Introduction to Computer Vision 3D-L3 Fundamental matrix

2 Weak calibration Main idea: Estimate epipolar geometry from a (redundant) set of point correspondences between two uncalibrated cameras

3 From before: Projection matrix w wxim Y w wy K Φ im in t e x t Z w w 1 X

4 From before: Projection matrix w wxim Y w wy K Φ im in t e x t Z w w 1 r r r R T Φ r r r R T ext r r r R T X

5 From before: Projection matrix w wxim Y w wy K Φ im in t e x t Z w w 1 X K in t f / s 0 o x 0 f / s o y x y Note: Invertible, scale x and y, assumes no skew

6 From before: Projection matrix X w wxim Y w wy K Φ im in t e x t Z w w 1 p K Φ P im in t e x t w p K p im in t c p c

7 Uncalibrated case For a given camera: p K p im in t c 1 And since invertible: p K p c in t im

8 Uncalibrated case So, for two cameras (left and right): 1 p K p c, le ft in t,le ft im, le ft 1 p K p c, r ig h t in t,r ig h t im, r ig h t Internal calibration matrices, one per camera

9 Uncalibrated case 1 p K p c, r ig h t in t,r ig h t im, r ig h t 1 p K p c, le ft in t,le ft im, le ft From before, the essential matrix E. p E p 0 c, r ig h t c, le ft 1 1 K p in t,r ig h t im r ig h t E K p in t,le ft im le ft 0,,

10 Uncalibrated case 1 1 K p in t,r ig h t im r ig h t E K p in t,le ft im le ft 0,, 1 T 1 p K ) E K p 0 im, r ig h t in t,r ig h t in t,le ft im, le ft Fundamental matrix : F p F p 0 im, r ig h t im, le ft or p T F p ' 0

11 Properties of the Fundamental Matrix p T F p ' 0 l F p is the epipolar line in the p image associated with p

12 Properties of the Fundamental Matrix p T F p ' 0 T l ' F p is the epipolar line in the prime image associated with p

13 Properties of the Fundamental Matrix p T F p ' 0 Epipoles found by Fp = 0 and F T p = 0

14 Properties of the Fundamental Matrix p T F p ' 0 F is singular (mapping from homogeneoues 2-D point to 1-D family so rank 2 more later)

15 Fundamental matrix Relates pixel coordinates in the two views More general form than essential matrix: We remove the need to know intrinsic parameters

16 Fundamental matrix If we estimate fundamental matrix from correspondences in pixel coordinates, can reconstruct epipolar geometry without intrinsic or extrinsic parameters.

17 Different Example: Forward motion e e courtesy of Andrew Zisserman

18 Computing F from correspondences Each point correspondence generates one constraint on F p F p 0 im, r ig h t im, le ft

19 Computing F from correspondences Multiply out:

20 Computing F from correspondences Collect N of these: And solve for f the elements of F.

21 The (in)famous eight-point algorithm

22 Just solving for F

23 Rank of F Assume we know the homography H π that maps from Left to Right (Full 3x3) p ' H p Let line l be the epiloar line corresponding to p goes through epipole e

24 Rank of F Let line l be the epiloar line corresponding to p goes through epipole e l ' e ' p e ' H p [ e '] H p But l is the epipolar line for p: l ' F p Rank of F is rank of *e + x = 2

25 Fix the linear solution Use SVD or other method to do linear computation for F Decompose F using SVD (not the same SVD): F UDV T

26 Fix the linear solution Use SVD or other method to do linear computation for F Decompose F using SVD (not the same SVD): F UDV Set the last singular value to zero: r 0 0 r ˆ D s D 0 s t T

27 Fix the linear solution Estimate new F from the new ˆD ˆ ˆ T F UDV

28 That s better

29 Stereo image rectification

30 Stereo image rectification Reproject image planes onto a common plane parallel to the line between optical centers each a homography Pixel motion is horizontal after this transformation C. Loop and Z. Zhang. Computing Rectifying Homographies for Stereo Vision. IEEE Conf. Computer Vision and Pattern Recognition, 1999.

31 C. Loop and Z. Zhang, Computing Rectifying Homographies for Stereo Vision, IEEE Conf. Computer Vision and Pattern Recognition, 1999.

32 Photo synth Noah Snavely, Steven M. Seitz, Richard Szeliski, "Photo tourism: Exploring photo collections in 3D," SIGGRAPH

33 Photosynth.net Based on Photo Tourism by Noah Snavely, Steve Seitz, and Rick Szeliski

34 3D from multiple images Building Rome in a Day: Agarwal et al. 2009

35 Summary For 2-views, there is a geometric relationship that define the relations between rays in one view to rays in the other epipolar geometry. These relationships can be captured algebraicly as well: Calibrated Essential matrix Uncalibrated Fundamental matrix. This relation can be estimated from point correspondences.