Lecture 4 Single View Metrology

Transcription

1 Lecture 4 Single View Metrology Professor Silio Srese Computtionl Vision nd Geometry Lb Silio Srese Lecture 4-6-Jn-5

2 Lecture 4 Single View Metrology Reiew clibrtion nd 2D trnsformtions Vnishing points nd lines Estimting geometry from single imge Etensions Reding: [HZ] Chpter 2 Projectie Geometry nd Trnsformtion in 2D [HZ] Chpter 3 Projectie Geometry nd Trnsformtion in 3D [HZ] Chpter 8 More Single View Geometry [Hoiem Srese] Chpter 2 Silio Srese Lecture 4-6-Jn-5

3 Clibrtion Problem j C u P M P p i i i i i World ref. system In piels M K[ R T] K α α cotθ β sinθ u o o

4 Clibrtion Problem j C u P M P p i i i i i World ref. system In piels M K[ R T] unknown Need t lest 6 correspondences

5 Once the cmer is clibrted... Pinhole perspectie projection P Line of sight p O w C M K[ R T] -Internl prmeters K re known -R, T re known but these cn only relte C to the clibrtion rig Cn I estimte P from the mesurement p from single imge? No - in generl (P cn be nywhere long the line defined by C nd p)

6 Recoering structure from single iew Pinhole perspectie projection P Line of sight p O w C unknown known Known/ Prtilly known/ unknown

7 Recoering structure from single iew

8 Trnsformtion in 2D -Isometries -Similrities -Affinity -Projectie

9 Trnsformtion in 2D Isometries: y H y t R y e - Presere distnce (res) - 3 DOF - Regulte motion of rigid object [Eucliden] [Eq. 4]

10 Trnsformtion in 2D Similrities: S s s y SR t y H s y [Eq. 5] - Presere - rtio of lengths - ngles -4 DOF

11 Trnsformtion in 2D Affinities: y H y t A y A ) R( D ) R( ) ( R φ φ θ y s s D [Eq. 6] [Eq. 7]

12 Trnsformtion in 2D Affinities: y H y t A y A ) R( D ) R( ) ( R φ φ θ y s s D -Presere: - Prllel lines - Rtio of res - Rtio of lengths on colliner lines - others - 6 DOF [Eq. 6] [Eq. 7]

13 Trnsformtion in 2D Projectie: y H y b t A y p - 8 DOF - Presere: - cross rtio of 4 colliner points - collinerity - nd few others [Eq. 8]

14 The cross rtio P P P P P P P P The cross-rtio of 4 colliner points Cn permute the point ordering i i i i Z Y X P P P P P P P P P P P 2 P 3 P 4 [Eq. 9]

15 Lecture 4 Single View Metrology Reiew clibrtion nd 2D trnsformtions Vnishing points nd lines Estimting geometry from single imge Etensions Reding: [HZ] Chpter 2 Projectie Geometry nd Trnsformtion in 2D [HZ] Chpter 3 Projectie Geometry nd Trnsformtion in 3D [HZ] Chpter 8 More Single View Geometry [Hoiem Srese] Chpter 2 Silio Srese Lecture 4-6-Jn-5

16 Lines in 2D plne c by + + -c/b -/b c b l If [, 2 ] T l c b T 2 l y [Eq. ]

17 Lines in 2D plne Intersecting lines y l l l Proof l [Eq. ] l l l ( l l ) l l [Eq. 2] l l l ( l l ) l l [Eq. 3] is the intersecting point

18 2D Points t infinity (idel points), c b l c b l ) b l l Let s intersect two prllel lines: In Euclidin coordintes this point is t infinity Agree with the generl ide of two lines intersecting t infinity l l 2 / / b b Eq.3]

19 2D Points t infinity (idel points), c b l c b l [ ] l T b c b Note: the line l [ b c] T pss trough the idel point So does the line l since b b l l / / b b [Eq. 5]

20 Lines infinity l Set of idel points lies on line clled the line t infinity. How does it look like? l l T 2 Indeed: A line t infinity cn be thought of the set of directions of lines in the plne b b

21 Projectie trnsformtion of point t infinity p p H b t A H? p H z y p p p b t A is it point t infinity? no? p H A y p p b t A An ffine trnsformtion of point t infinity is still point t infinity [Eq. 7] [Eq. 8]

22 Projectie trnsformtion of line (in 2D) l H l T b t A H? l H T b t t b t A y T is it line t infinity? no? l H T A T T T T A t A t A [Eq. 9] [Eq. 2] [Eq. 2]

23 Points nd plnes in 3D 3 2 Π d c b d cz by y z Π T Π How bout lines in 3D? Lines he 4 degrees of freedom - hrd to represent in 3D-spce Cn be defined s intersection of 2 plnes [Eq. 23] [Eq. 22]

24 Points t infinity in 3D Points where prllel lines intersect in 3D world point t infinity Prllel lines 2 3

25 Vnishing points The projectie projection of point t infinity into the imge plne defines nishing point. M p p p p world p Prllel lines point t infinity direction of the line in 3D

26 M X K I [ ] b c K b c K d d direction of the line [, b, c] T d C X b c M d K K [Eq. 24] [Eq. 25] Proof: Vnishing points nd directions

27 Vnishing (horizon) line l π horizon Projectie trnsformtion M l hor T H l [Eq. 26] P Imge

28 Are these two lines prllel or not? Recognition helps reconstruction Humns he lernt this - Recognize the horizon line - Mesure if the 2 lines meet t the horizon - if yes, these 2 lines re // in 3D

29 Vnishing points nd plnes l n π l horiz C T n K l [Eq. 27] horiz

30 Plnes t infinity Π z y Π plne t infinity Prllel plnes intersect t infinity in common line the line t infinity A set of 2 or more lines t infinity defines the plne t infinity Π

31 Angle between 2 nishing points 2 θ d 2 d 2 C cosθ [Eq. 28] T ω T ω 2 T 2 ω 2 ω (K K T ) If 9 T 2 θ ω Sclr eqution [Eq. 29]

32 Projectie trnsformtion of Ω Absolute conic ω M T Ω M (K K T ) [Eq. 3] M K R T. It is not function of R, T 2. ω ω2 ω4 ω ω2 ω3 ω5 ω 4 ω5 ω6 ω 2 ω2 zero-skew ω ω symmetric nd known up scle squre piel

33 Why is this useful? To clibrte the cmer To estimte the geometry of the 3D world

34 Lecture 4 Single View Metrology Reiew clibrtion Vnishing points nd line Estimting geometry from single imge Etensions Reding: [HZ] Chpter 2 Projectie Geometry nd Trnsformtion in 3D [HZ] Chpter 3 Projectie Geometry nd Trnsformtion in 3D [HZ] Chpter 8 More Single View Geometry [Hoiem Srese] Chpter 2 Silio Srese Lecture 4-6-Jn-5

35 Single iew clibrtion - emple [Eq. 28] cosθ T ω T ω 2 T 2 ω 2 2 θ? 9 o T ω 2 ω (K K T ) Do we he enough constrints to estimte K? K hs 5 degrees of freedom nd Eq.29 is sclr eqution

36 Single iew clibrtion - emple [Eq. 28] cosθ T ω T ω 2 T 2 ω [Eqs. 3] T ω 2 T ω 3 T ω 3 2

37 Single iew clibrtion - emple ω ω ω ω 2 4 ω ω ω ω ω ω ω 3 known up to scle ω2 ω à Compute ω : [Eqs. 3] T ω 2 T ω 3 T ω 3 2 Once ω is clculted, we get K: ω (K K T ) K (Cholesky fctoriztion; HZ pg 582)

38 Single iew reconstruction - emple l h K known T n K l horiz Scene plne orienttion in the cmer reference system Select orienttion discontinuities

39 Single iew reconstruction - emple C Recoer the structure within the cmer reference system Notice: the ctul scle of the scene is NOT recoered Recognition helps reconstruction Humns he lernt this

40 Lecture 4 Single View Metrology Reiew clibrtion Vnishing points nd lines Estimting geometry from single imge Etensions Reding: [HZ] Chpter 2 Projectie Geometry nd Trnsformtion in 3D [HZ] Chpter 3 Projectie Geometry nd Trnsformtion in 3D [HZ] Chpter 8 More Single View Geometry [Hoiem Srese] Chpter 2 Silio Srese Lecture 4-6-Jn-5

41 Criminisi Zissermn, 99

42 Criminisi Zissermn, 99

43 L Trinit (426) Firenze, Snt Mri Noell; by Msccio (4-428)

44 L Trinit (426) Firenze, Snt Mri Noell; by Msccio (4-428)

45

46 Single iew reconstruction - drwbcks Mnully select: Vnishing points nd lines; Plnr surfces; Occluding boundries; Etc..

47 Automtic Photo Pop-up Hoiem et l, 5

48 Automtic Photo Pop-up Hoiem et l, 5

49 Automtic Photo Pop-up Hoiem et l, 5 Softwre:

50 Mke3D Trining Imge Sen, Sun, Ng, 5 Prediction Depth Plne Prmeter MRF Plnr Surfce Segmenttion youtube Connectiity Co-Plnrity

51 Single Imge Depth Reconstruction Sen, Sun, Ng, 5 A softwre: Mke3D Conert your imge into 3d model

52 Coherent object detection nd scene lyout estimtion from single imge Y. Bo, M. Sun, S. Srese, CVPR 2, BMVC 2 M. Sun Y. Bo

53 Net lecture: Multi-iew geometry (epipolr geometry)

54 Appendi