Multiple view geometry

Transcription

1 EECS 442 Computer vson Multple vew geometry Perspectve Structure from Moton - Perspectve structure from moton prolem - mgutes - lgerc methods - Fctorzton methods - Bundle djustment - Self-clrton Redng: [HZ] Chpters:,8,9 [FP] Chpter: 3 Some sldes of ths lectures re courtesy of prof. S. Lzenk

2 Structure from moton prolem X j j M m j 2j M 2 M m From the mn correspondences j, estmte: m projecton mtrces M n 3D ponts X j moton structure

3 Structure from moton mguty - Poston mguty: t s mpossle sed on the mges lone to estmte the solute locton nd pose of the scene w.r.t. 3D world coordnte frme

4 Structure from moton mguty -Scle mguty: t s mpossle sed on the mges lone to estmte the solute scle of the scene (.e. house heght)

5 Structure from moton mguty -he scene s determned y the mges only up smlrty trnsformton (rotton, trnslton nd sclng) H s s s R t

6 Structure from moton mguty he mguty ests even for clrted cmers For clrted cmers, the smlrty mguty s the only mguty [Longuet-Hggns 8]

7 Structure from moton mguty In the generl cse (nothng s known) the mguty s epressed y n rtrry ffne or projectve trnsformton M j X j M R H X M H j j j M X j M H - H X j

8 Projectve mguty

9 ffne mguty

10 Structure from moton prolem X j j M m j 2j M 2 M m Gven m mges of n fed 3D ponts j = M X j, =,, m, j =,, n

11 Structure from moton prolem X j j M m j 2j M 2 M m m cmers M M m M

12 he Structure-from-Moton Prolem Gven m mges of n fed ponts X j we cn wrte M j X j Prolem: estmte the m 34 mtrces M nd the n postons X j from the mn correspondences j. Wth no clrton nfo, cmers nd ponts cn only e recovered up to 44 projectve Gven two cmers, how mny ponts re needed? How mny equtons nd how mny unknown? 2m n equtons n m+3n 5 unknowns So 7 ponts! [227 = 28; = 28]

13 Structure-from-Moton lgorthms lgerc pproch (y fundmentl mtr) Fctorzton method (y SVD) Bundle djustment

14 lgerc pproch (2-vew cse) j M 2j M j X j M 2 pply projectve trnsformton H such tht: M H I M H Cnoncl perspectve cmers 2

15 X H X ~ X I X ~ ] [ ~ H M lgerc pproch (Fundmentl mtr) ~ ] [I X ~ X ~ X ~ ] [I X ~ X ~ X ~ ] [ ~ ] [ X X X X ~ ] [ ~ H M 2 ) ( ) (? ) (

16 ] [ z y y z y z Cross product s mtr multplcton

17 X H X ~ ) ( X I X ~ ] [ ~ H M lgerc pproch (Fundmentl mtr) ~ ] [I X ~ X ~ X ~ ] [I X ~ X ~ X ~ ] [ ~ ] [ X X X ~ ] [ H M 2? ] [ F ] [ s ths fmlr? F ) ( ) (

18 Eppolr Constrnt [lecture 6] X 2 e e 2 O O 2 F 2 s the eppolr lne ssocted wth 2 (l = F 2 ) F s the eppolr lne ssocted wth (l 2 = F ) F s sngulr (rnk two) F e 2 = nd F e = F s 33 mtr; 7 DOF

19 lgerc pproch (Fundmentl mtr) F F [ ] F [ ] Snce F =, =e Compute s lest sq. soluton of F = det(f)=; = = [ ] - F = [ ] F M p I M p [e ]F e 2 Perspectve cmers re known

20 Structure-from-Moton lgorthms lgerc pproch (y fundmentl mtr) Fctorzton method (y SVD) Bundle djustment

21 Projectve fctorzton D z z z 2 m 2 m z z z 2 22 m m2 z z z n 2n mn n 2n mn M M M 2 m cmers (3 m 4) X X 2 ponts (4 n) X n D = MS hs rnk 4 If we knew the depths z, we could fctorze D to estmte M nd S If we knew M nd S, we could solve for z Soluton: tertve pproch (lternte etween ove two steps)

22 Structure-from-Moton lgorthms lgerc pproch (y fundmentl mtr) Fctorzton method (y SVD) Bundle djustment

23 Bundle djustment Non-lner method for refnng structure nd moton Mnmzng re-projecton error E(M, X) m n j D 2, X j M j X j? M X j j 3j P M 2 X j 2j M 3 X j P 2 P 3

24 Bundle djustment Non-lner method for refnng structure nd moton Mnmzng re-projecton error E(M, X) m n j dvntges Hndle lrge numer of vews Hndle mssng dt D 2, X j M Lmttons Lrge mnmzton prolem (prmeters grow wth numer of vews) requres good ntl condton j Used s the fnl step of SFM

25 Removng the mgutes: the Strtfed reconstructon up grde reconstructon from perspectve to ffne [y mesurng the plne t nfnty] up grde reconstructon from ffne to metrc [y mesurng the solute conc] Recoverng the metrc reconstructon from the perspectve one s clled self-clrton

26 Self-clrton Process of determnng ntrnsc cmer prmeters drectly from un-clrted mges Suppose we hve projectve reconstructon M,X } { j GOL: fnd rectfyng (non-sngulr) homogrphy H such tht { M H,H X j} M M M X H j s metrc reconstructon m M [R If world ref. system = cmer ref. system: [I ] M If the perspectve cmer s cnoncl: M [I ] ]

27 Self-clrton M M H M [I M [I ] ] [ ] [I ] H H v t k t We cn set k= (ths fes the scle of the reconstructon) H v

28 Plnes t nfnty (lecture 5) z y In the metrc (Euclden) world coordntes 2 plnes re prllel ff ther ntersectons s lne tht elongs to he projectve trnsformton of plne t nfnty cn e epressed s H p

29 Self-clrton H v v v H p v p

30 Self-clrton GOL: fnd rectfyng homogrphy H such tht { M,X j} { M H,H X j} s metrc reconstructon H p cot sn u v o o = clrton mtr of frst cmer [p ] = plne t nfnty n the projectve reconstructon 5 unknowns 3 unknowns

31 Self-clrton sc equton ] [ M = perspectve reconstructon of the cmer (known) p H ] [R M p R m 2 H M M R p p p R = metrc reconstructon of the cmer = rectfyng homogrphy (unknown)

32 Self-clrton sc equton p R p R I R R I p p p p?

33 solute conc ny [From lecture 5] stsfes: s C Projectve trnsformton of 2 3 ( ) * Dul mge of the solute conc

34 ) ( It s not functon of R, symmetrc (5 unknowns) [From lecture 5] Propertes of

35 Self-clrton sc equton p p * * p p =2 m How mny unknowns? 3 from p 5 from How mny equtons? [ nd re known] [per vew] 5 ndependent equtons [per vew] rt of self-clrton: use constrnts on () to generte enough equtons on the unknowns

36 Self-clrton dentcl s * * p p * * p p For m vews, 5(m-) constrnts Numer of unknowns: 8 m>=3 provdes enough constrnts o solve the self-clrton prolem wth dentcl cmers we need t lest 3 vews

37 Propertes of [From lecture 5] ( ) zero-skew squre pel zero-offset

38 Self-clrton other constrnts * * p p zero-offset m lner constrnts zero-skew etc 2 m lner constrnts

39 Self-clrton - summry Condton N. Vews Constnt nternl prmeters 3 spect rto nd skew known Focl length nd offset vry spect rto nd skew known Focl length nd offset vry 4* 5* skew =, ll other prmeters vry 8* Issue: the lrger s the numer of vew, the hrder s the correspondence prolem Bundle djustment helps!

40 SFM prolem - summry. Estmte structure nd moton up perspectve trnsformton. lgerc 2. fctorzton method 3. undle djustment 2. Convert from perspectve to metrc (self-clrton) 3. Bundle djustment ** or **. Bundle djustment wth self-clrton constrnts

41 Self-clrton - summry Constrnts on cmer moton cn e ncorported Lnerly trnsltng cmer Sngle s of rotton: turntle moton

42 pplctons Courtesy of Oford Vsul Geometry Group

43 pplctons D. Nstér, PhD thess

44 pplctons M. Pollefeys et l 98---

45 pplctons M. Brown nd D. G. Lowe. Unsupervsed 3D Oject Recognton nd Reconstructon n Unordered Dtsets. (3DIM25)

46 Photo synth Noh Snvely, Steven M. Setz, Rchrd Szelsk, "Photo toursm: Eplorng photo collectons n 3D," CM rnsctons on Grphcs (SIGGRPH Proceedngs),26,

47