Single view metrology

Transcription

1 EECS 44 Computer vision Single view metrology Review calibration Lines and planes at infinity Absolute conic Estimating geometry from a single image Etensions Reading: [HZ] Chapters,3,8

2 Calibration Problem j C i P i M P i i i v u p In piels [ ] T M K R v u cot K o o sin θ β λ θ α α λ World ref. system

3 Calibration Problem j C M K[ R T] P M P i i p i u v i i unknown Need at least 6 correspondences

4 Once the camera is calibrated... Pinhole perspective projection P p O w C M K[ R T] -Internal parameters K are known -R, T are known but these can only relate C to the calibration rig Can I estimate P from the measurement p from a single image?

5 Recovering structure from a single view Pinhole perspective projection P p O w C unknown known Known/ Partially known/ unknown

6 Recovering structure from a single view Saena, Sun, Ng, 6

7 Review calibration Lines and planes at infinity Absolute conic Estimating geometry from a single image Eamples

8 Lines in a D plane c by a + + -c/b -a/b c b a l If [, ] T l c b a T l y

9 Intersecting lines Lines in a D plane l l l Proof l l l l l l ( l l ) l ( l l ) l y l l l is the intersecting point

10 Points and planes in 3D 3 Π d c b a d cz by a y z T Π Π How about lines in 3D? Lines have 4 degrees of freedom - hard to represent in 3D-space Can be defined as intersection of planes

11 Points at infinity (ideal points), 3 3 What happens if 3? c b a l c b a l a b ) c (c l l Let s intersect two parallel lines: Agree with the general idea of two lines intersecting at infinity l l

12 Lines infinity l T Set of ideal points lies on a line called the line at infinity How does it look like? Indeed: l l

13 Projective transformation of a line (in D) l H l T b v t A H? l H T is it a line at infinity?? l H T A T T T T A t A t A

14 Projective transformation of a line (in D) horizon H T l Are these two lines parallel or not? Recognition helps reconstruction! Humans have learnt this - Recognize the horizon line - Measure if the lines meet at the horizon - if yes, these lines are // in 3D

15 Vanishing points d v Points where parallel lines intersect in 3D ( ideal points in D) C ddirection of the line v K d M K[ I ]

16 Vanishing points - eample v, v: measurements K known and constant star v d d K K K K v v v v v C C R,T d R d R

17 d K K v v d K K v v R

18 Planes at infinity Π z y planes are parallel iff their intersections is a line that belongs to Π

19 Vanishing lines Parallel planes intersect the plane at infinity in a common line the vanishing line (horizon) n C T n K l horiz

20 Projective transformation of a line (in D) horizon H T l T n K l horiz

21 Review calibration Lines and planes at infinity Absolute conic Estimating geometry from a single image Eamples

22 Conics in D a + by + cy + d + ey + f In homogeneous coordinates: T C - If a point C a C b / d / T C b / c e / d / e / f - If a line l is tangent to a conic in l C - Projective transformation of conics: T C P C P

23 Circular points l + i p i i p j Circular points (point at infinity) Circular points are fied under similarity transformation i i t scos ssin t ssin s cos p H y i s θ θ θ θ i -

24 Conic C Circular points define a degenerate conic called ; any C + 3 C C C T l + i p i i p j Circular points (point at infinity) is fied under similarity transformation C

25 In 3D: absolute conic Ω is a C Π Any Ω satisfies: Ω Ω T 3 Ω is fied under similarity transformation

26 Projective transformation of conics in 3D C P T C P Projective transformation of Ω P T (KR) K (KR) K T Ω P? T Ω R R R T T I R K (KR) (KR) K (K K) T T ω M K[ R T] Ω Ω

27 Projective transformation of Ω ω (K T K). It is not function of R, T. ω ω ω4 ω ω ω3 ω5 symmetric ω 4 ω5 ω6 ω ω zero-skew ω ω square piel

28 Projective transformation of Ω ω (K T K) Why this is useful?

29 Review calibration Lines and planes at infinity Absolute conic Estimating geometry from a single image Eamples

30 Angle between scene lines θ d v d v K d v C cosθ v T v T ω v ω v v T ω v If 9 v T θ ω v

31 Angle between scene lines v θ 9 v v T ω v ω (K T K) Constraint on K

32 Single view calibration - eample v v ω v 3 ω ω ω4 ω ω ω 3 5 ω4 ω 5 ω 6 Compute ω : v T ω v v T ω v3 v T ω v3 ω ω ω 3 6 unknown 5 constraints ω known up to scale ω K (K T K) (Cholesky factorization)

33 Single view reconstruction - eample l h K known T n K l horiz Scene plane orientation in the camera reference system Select orientation discontinuities

34 Projective transformation of a line (in D) horizon H T l Are these two lines parallel or not? Recognition helps reconstruction! Humans have learnt this - Recognize the horizon line - Measure if the lines meet at the horizon - if yes, these lines are // in 3D

35 Single view reconstruction - eample C Recover the structure within the camera reference system Notice: the actual scale of the scene is NOT recovered Recognition helps reconstruction! Humans have learnt this Are these two lines parallel or not? - Recognize the horizon line - Measure if the lines meet at the horizon - if yes, these lines are // in 3D

36 Criminisi & Zisserman, 99

37 La Trinita' (46) Firenze, Santa Maria Novella; by Masaccio (4-48)

38

39 Single view reconstruction - drawbacks Manually select: Vanishing points and lines; Planar surfaces; Occluding boundaries; Etc..

40 Automatic Photo Pop-up Hoiem et al, 5

41 Automatic Photo Pop-up Hoiem et al, 5

42 Automatic Photo Pop-up Hoiem et al, 5 Software:

43 Single Image Depth Reconstruction Saena, Sun, Ng, 5

44 Single Image Depth Reconstruction Saena, Sun, Ng, 5 A software: Make3D Convert your image into 3d model

45 Net lecture: Multi-view geometry (epipolar geometry)