Homographies and Estimating Extrinsics

Transcription

1 Homographies and Estimating Extrinsics Instructor - Simon Lucey Designing Computer Vision Apps

2 Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince Review: Motivation

3 Review: Pinhole Camera Real camera image is inverted Instead model impossible but more convenient virtual image Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

4 Today Fitting Warp Functions. Estimating Intrinsics.

5 Euclidean warp Consider viewing a fronto-parallel plane at a fixed distance D. In homogeneous coordinates, the imaging equations are: 3D rotation matrix becomes 2D (in plane) Plane at known distance D Point is on plane (w=0) Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

6 Euclidean warp Simplifying Rearranging the last equation Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

7 Euclidean warp Homogeneous: Cartesian: For short: Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

8 Euclidean warp Homogeneous: Cartesian: For short: Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince How many unknowns?

9 Estimating the Euclidean Warp NX ˆ, ˆ = arg min, n=1 {x n euc[w n,, ]}

10 Estimating the Euclidean Warp NX ˆ, ˆ = arg min, n=1 x n euc[w n,, ] 2 2

11 Estimating the Euclidean Warp NX ˆ, ˆ = arg min, = arg min, n=1 NX n=1 x n euc[w n,, ] 2 2 x n w n + 2 2

12 Estimating the Euclidean Warp NX ˆ, ˆ = arg min, = arg min, n=1 NX n=1 x n euc[w n,, ] 2 2 x n w n s.t. T = I, det( ) =1

13 Reminder: Non-Convex Set 9

14 Reminder: Non-Convex Set 9

15 Reminder: Non-Convex Set 9

16 Estimating the Euclidean Warp NX ˆ = arg min n=1 (x n µ x ) (w n µ w ) 2 2 s.t. T = I, det( ) =1 NX NX µ x = 1 N x n, µ w = 1 N w n n=1 n=1

17 Estimating the Euclidean Warp In MATLAB this becomes, >> [U,S,V] = svd(b*a,0); >> Omega = U*V ; SVD method works for ONLY very specific objectives. Extremely useful as it allows us to find closed form solutions to a non-convex problem (very rare).

18 Similarity warp Consider viewing fronto-parallel plane at unknown distance D By same logic as before we have Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

19 Similarity warp Simplifying: Multiply each equation by : Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

20 Similarity warp Simplifying: Incorporate the constants by defining: Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

21 Similarity warp Homogeneous: Cartesian: For short:

22 Similarity warp Homogeneous: Cartesian: For short: How many unknowns?

23 Estimating the Similarity Warp Simplifying: Since,

24 Estimating the Similarity Warp Rearranging: Form system of equations:

25 Estimating the Similarity Warp In MATLAB this becomes, >> p = A\x; Since p stems form a convex set, simply use backslash!!!

26 Estimating the Similarity Warp In MATLAB this becomes, >> p = A\x; Since p stems form a convex set, simply use backslash!!!

27 Estimating the Similarity Warp In MATLAB this becomes, >> p = A\x; Since p stems form a convex set, simply use backslash!!!

28 Affine warp Affine transform describes mapping well when the depth variation within the planar object is small and the camera is far away. Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince When variation in depth is comparable to distance to object then the affine transformation is not a good model. Here we need the homography.

29 Affine warp Homogeneous: Cartesian: For short:

30 Affine warp Homogeneous: Cartesian: For short: How many unknowns?

31 Estimating the Affine Warp Rearranging: Form system of equations: In MATLAB this becomes, >> p = A\x;

32 Homography Start with basic projection equation: Combining these two matrices we get: Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

33 Homography Homogeneous: Cartesian: For short:

34 Homography Homogeneous: Cartesian: For short: How many unknowns?

35 Homography Estimation 2 Re-arrange 4 cartesian equations, 3 5 Form linear system u 1 v 1 1 y 1 u 1 y 1 v 1 y 1 u 1 v x 1 u 1 x 1 v 1 x u 2 v 2 1 y 2 u 2 y 2 v 2 y 2 u 2 v x 2 u 2 x 2 v 2 x u I v I 1 y I u I y I v I y I 5 u I v I x I u I x I v I x I = 0, Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

36 Homography Estimation In MATLAB this becomes, >> [U,S,V] = svd(a); >> Phi = reshape(v(:,end),[3,3]) ; Both sides are 3x1 vectors; should be parallel, so cross product will be zero For you to try MATLAB, >> x = [randn(2,1);1]; cross(x,4*x)

37 Caution Approach only minimizes algebraic error NOT the reprojection error!!! Need to employ non-linear optimization.

38 Today Fitting Warp Functions. Estimating Extrinsics.

39 Estimating Extrinsics

40 Estimating Extrinsics Writing out the camera equations in full Estimate the homography from matched points Factor out the intrinsic parameters Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

41 Estimating Extrinsics Find the last column using the cross product of first two columns Make sure the determinant is 1. If it is -1, then multiply last column by -1. Find translation scaling factor between old and new values Finally, set Adapted from: Computer vision: models, learning and inference. Simon J.D. Prince

42 Augmented Reality

43 Transformations between images So far we have considered transformations between the image and a plane in the world Now consider two cameras viewing the same plane There is a homography between camera 1 and the plane and a second homography between camera 2 and the plane It follows that the relation between the two images is also a homography

44 Camera under pure rotation Special case is camera under pure rotation. Homography can be showed to be

45 Camera under pure rotation Special case is camera under pure rotation. Homography can be showed to be Why is this?

46 Panorama Example a) b) Models for transformations c) d) Figure Computing visual panoramas. a-c) Three images of the same scene where the camera has rotated but not translated. Five matching points

47 More to read Prince et al. Chapter 14, Sections 1-4. Chapter 15, Sections 1-6.