Single view metrology

Transcription

1 EECS 44 Computer vision Singe view metroogy Review caibration Lines and panes at infinity Absoute conic Estimating geometry from a singe image Etensions Reading: [HZ] Chapters,3,8

2 Caibration Probem j C i P i M P i i i v u p In pies T M K R cot sin o o v u K Word ref. system

3 Caibration Probem j C P M P i i Word ref. system p i In pies u v i i unknown M K R T Need at east 6 correspondences

4 Once the camera is caibrated... Pinhoe perspective projection P p O w C M K R T -Interna parameters K are known -R, T are known but these can ony reate C to the caibration rig Can I estimate P from the measurement p from a singe image? No - in genera [P can be anywhere aong the ine defined by C and p]

5 Recovering structure from a singe view Pinhoe perspective projection P p O w C unknown known Known/ Partiay known/ unknown

6 Recovering structure from a singe view

7 Review caibration Lines and panes at infinity Absoute conic Estimating geometry from a singe image Eampes

8 Lines in a D pane c by a -c/b -a/b c b a If = [, ] T c b a T y

9 Lines in a D pane Intersecting ines Proof ) ( ) ( is the intersecting point y

10 Points and panes in 3D 3 d c b a d cz by a y z T How about ines in 3D? Lines have 4 degrees of freedom - hard to represent in 3D-space Can be defined as intersection of panes

11 D Points at infinity (idea points), 3 3 c b a c b a ' ' / ) ( a b c c Let s intersect two parae ines: Agree with the genera idea of two ines intersecting at infinity ' / ' / b a b a

12 Lines infinity Set of idea points ies on a ine caed the ine at infinity How does it ook ike? T Indeed:

13 Projective transformation of a point at infinity p p H ' b v t A H? p H z y p p p b v t A ' ' ' is it a point at infinity? no!? p H A ' ' y p p b t A An affine transformation of a point at infinity is sti a point at infinity

14 Projective transformation of a ine (in D) H T b v t A H? H T b t t b v t A y T is it a ine at infinity? no!? H T A T T T T A t A t A

15 The horizon ine horizon hor H T Are these two ines parae or not? Recognition heps reconstruction! Humans have earnt this - Recognize the horizon ine - Measure if the ines meet at the horizon - if yes, these ines are // in 3D

16 Vanishing points (= idea points in D) Points where parae ines intersect in 3D

17 Vanishing points and their image d v d=direction of the ine C v K d M K R T

18 Vanishing points - eampe v, v: measurements K = known and constant Can I compute R? star v d d K K K K v v v v v In D C C R,T d d d R d R

19 d K K v v d K K v v R

20 Panes at infinity & vanishing ines z y Parae panes intersect at the pane at infinity panes are parae iff their intersections is a ine that beongs to Parae panes intersect the pane at infinity in a common ine the vanishing ine (horizon)

21 Vanishing ines and their images Parae panes intersect the pane at infinity in a common ine the vanishing ine (horizon) n horiz C T n K horiz

22 Review caibration Lines and panes at infinity Absoute conic Estimating geometry from a singe image Eampes

23 Conics in D a by cy d ey f In homogeneous coordinates: T C - If a point C a C b / d / T C b / c e / d / e / f - If a ine is tangent to a conic in C - Projective transformation of conics: T C P C P

24 Circuar points i p i i p j Circuar points (point at infinity) Circuar points are fied under simiarity transformation i i t scos ssin t ssin scos p H y i s i =-

25 Conic C Circuar points define a degenerate conic caed ; any C 3 C C C T i p i i p j Circuar points (point at infinity) is fied under simiarity transformation C

26 In 3D: absoute conic is a C Any satisfies: T 4 3 is fied under simiarity transformation

27 Projective transformation of (K T K) P K R T. It is not function of R, T symmetric zero-skew square pie

28 Projective transformation of (K T K) Why this is usefu?

29 Review caibration Lines and panes at infinity Absoute conic Estimating geometry from a singe image Eampes

30 Ange between vanishing points d v v K d d v C cos v T v T v v v T v If 9 v T v

31 Ange between scene ines v 9 v v T v (K T K) Constraint on K

32 Singe view caibration - eampe v v v Compute : v T v 6 unknown 5 constraints known up to scae v T v3 v T v3 3 Once is cacuated, we get K: T (K K) K (Choesky factorization; HZ pag 58)

33 Singe view reconstruction - eampe h K known T n K horiz = Scene pane orientation in the camera reference system Seect orientation discontinuities

34 Singe view reconstruction - eampe C Recover the structure within the camera reference system Notice: the actua scae of the scene is NOT recovered Recognition heps reconstruction! Humans have earnt this Are these two ines parae or not? - Recognize the horizon ine - Measure if the ines meet at the horizon - if yes, these ines are // in 3D

35 Criminisi & Zisserman, 99

36 Criminisi & Zisserman, 99

37 La Trinita' (46) Firenze, Santa Maria Novea; by Masaccio (4-48)

38 La Trinita' (46) Firenze, Santa Maria Novea; by Masaccio (4-48)

39

40 Singe view reconstruction - drawbacks Manuay seect: Vanishing points and ines; Panar surfaces; Occuding boundaries; Etc..

41 Automatic Photo Pop-up Hoiem et a, 5

42 Automatic Photo Pop-up Hoiem et a, 5

43 Automatic Photo Pop-up Hoiem et a, 5 Software:

44 Make3D Training Saena, Sun, Ng, 5 Prediction Image Depth Pane Parameter MRF Panar Surface Segmentation youtube Connectivity Co-Panarity

45 Singe Image Depth Reconstruction Saena, Sun, Ng, 5 A software: Make3D Convert your image into 3d mode

46 Coherent object detection and scene ayout estimation from a singe image Y. Bao, M. Sun, S. Savarese, CVPR, BMVC M. Sun Y. Bao

47 Net ecture: Muti-view geometry (epipoar geometry)