Lecture 08 Multiple View Geometry 2. Prof. Dr. Davide Scaramuzza

Transcription

1 Lctr 8 Mltpl V Gomtry Prof. Dr. Dad Scaramzza sdad@f.zh.ch

2 Cors opcs Prncpls of mag formaton Imag fltrng Fatr dtcton Mlt- gomtry 3D Rconstrcton Rcognton

3 Mltpl V Gomtry San Marco sqar, Vnc 4,79 mags, 4,55,57 ponts

4 Mltpl V Gomtry 3D rconstrcton from mltpl s: Assmptons: K, and R ar knon. Goal: Rcor th 3D strctr from mags P =? Strctr From Moton: Assmptons: non (K,, and R ar nknon). K, R, K, R, K, R, Goal: Rcor smltanosly 3D scn strctr and camra poss (p to scal) from mltpl mags P =? K, R, =? K, R, =? K, R, =?

5 R: Prspct Projcton MP p Z Y X R K P c O p X c C Z c Y c [R ] Extrnsc Paramtrs W Z Y X = P K Normalzd mag coordnats Prspct Projcton Eqaton

6 oday s otln Strctr from Moton

7 Problm formlaton: Gn n ponts n corrspondnc across to mags, {(, ), (, )}, smltanosly compt th 3D locaton P, th camra rlat-moton paramtrs (R, t), and camra ntrnsc K, that satsfy R, =? P =? C C Strctr from Moton (SFM) Z Y X I K Z Y X R K

8 Strctr from Moton (SFM) o arants xst: Uncalbratd camra(s) -> K s nknon Calbratd camra(s) -> K s knon P =? C R, =? C

9 Lt s stdy th cas n hch th camra(s) s «calbratd» For connnc, lt s s normalzd mag coordnats hs, ant to fnd R,, P that satsfy R, =? P =? C C Strctr from Moton (SFM) K Z Y X I Z Y X R

10 Scal Ambgty Wth a sngl camra, only kno th rlat scal No nformaton abot th mtrc scal

11 Scal Ambgty Wth a sngl camra, only kno th rlat scal No nformaton abot th mtrc scal If scal th ntr scn by som factor s, th projctons of th scn ponts n th mag rman xactly th sam:

12 Scal Ambgty In monoclar son, t s mpossbl to rcor th absolt scal of th scn! Stro son? hs, only 5 dgrs of frdom ar masrabl: 3 paramtrs to dscrb th rotaton paramtrs for th translaton p to a scal ( can only compt th drcton of translaton bt not ts lngth)

13 Strctr from Moton (SfM) Ho many knons and nknons? 4n knons: n corrspondncs; ach on (, ) and (, ), = n 5 + 3n nknons 5 for th moton p to a scal (rotaton-> 3, translaton->) 3n = nmbr of coordnats of th n 3D ponts Dos a solton xst? If and only f nmbr of ndpndnt qatons nmbr of nknons 4n 5 + 3n n 5

14 Cross Prodct (or Vctor Prodct) Vctor cross prodct taks to ctors and rtrns a thrd ctor that s prpndclar to both npts So hr, c s prpndclar to both a and b, hch mans th dot prodct = Also, rcall that th cross prodct of to paralll ctors = h cross prodct btn a and b can also b xprssd n matrx form as th prodct btn th sk-symmtrc matrx of a and a ctor b c b a c b c a b a b a ] [ z y x x y x z y z b b b a a a a a a

15 Eppolar Gomtry P p p p ppolar plan p n p = Rp p, p, ar coplanar: p n p ( ') p p ( ( Rp )) p ] R p p E p [ ppolar constrant E [ ] R ssntal matrx

16 Eppolar Gomtry p p Normalzd mag coordnats p E p Eppolar constrant or Longt-Hggns qaton E [ ] R Essntal matrx h Essntal Matrx can b comptd from 5 mag corrspondncs [Krppa, 93]. h mor ponts, th hghr accracy n prsnc of nos h Essntal Matrx can b dcomposd nto R and rcallng that For dstnct soltons for R and ar possbl. E [ ] R H. Chrstophr Longt-Hggns (Sptmbr 98). "A comptr algorthm for rconstrctng a scn from to projctons". Natr 93 (588): PDF.

17 Ho to compt th Essntal Matrx? h Essntal Matrx can b comptd from 5 mag corrspondncs [Krppa, 93]. Hor, ths solton s not smpl. It took almost on cntry ntl an ffcnt solton as fond! [Nstr, CVPR 4] h frst poplar solton ss 8 ponts and s calld 8-pont algorthm Longt Hggns. A comptr algorthm for rconstrctng a scn from to projctons. Natr (98)

18 h ght-pont algorthm p,,), (,,) p E p ( p Mnmz: Q 3 E 33 ndr th constrant E = Q (ths matrx s knon) E (ths matrx s nknon) For n = 8 ponts, a nq solton xsts f th ponts ar not coplanar. For n > 8 noncoplanar ponts, a lnar last-sqar solton s gn by th gnctor of Q corrspondng to ts smallst gnal (hch s th nt ctor that mnmzs Q E ). It can b don sng Snglar Val Dcomposton.

19 8-pont algorthm: Matlab cod fncton F = calbratd_ghtpont( p, p) p = p'; % 3xN ctor; ach colmn = [;;] p = p'; % 3xN ctor; ach colmn = [;;] Q = [p(:,).*p(:,),... p(:,).*p(:,),... p(:,3).*p(:,),... p(:,).*p(:,),... p(:,).*p(:,),... p(:,3).*p(:,),... p(:,).*p(:,3),... p(:,).*p(:,3),... p(:,3).*p(:,3) ] ; [U,S,V] = sd(q); F = V(:,9); F = rshap(v(:,9),3,3)';

20 h ght-pont algorthm Manng of th lnar last-sqar rror Usng th dfnton of dot prodct, t can b obsrd that N ( p E p ) : p Ep = p Ep cos (θ) It can b obsrd that ths prodct s non zro hn, p, p, and ar not coplanar.

21 h ght-pont algorthm Nonlnar approach: mnmz sm of sqard ppolar dstancs N d ( p, l) d ( p, l) P =? l E p p p l Ep C C

22 Problm th ght-pont algorthm

23 Problm th ght-pont algorthm Poor nmrcal condtonng Can b fxd by rscalng th data: Normalzd 8-pont algorthm [Hartly, 995]

24 Comparson of stmaton algorthms 8-pont Normalzd 8-pont Nonlnar last sqars Rprojcton rror.33 pxls.9 pxl.86 pxl Rprojcton rror.8 pxls.85 pxl.8 pxl

25 Extract R and from E (ths sld ll not b askd at th xam) Snglar Val Dcomposton Enforcng rank- constrant V U E V U ˆ z y x x y x z y z t t t t t t t t t t ˆ ˆ ˆ ˆ RK K R t K t V U R ˆ

26 4 possbl soltons of R and Only on solton hr ponts ar n front of both camras hs to s ar rotatd of 8

27 Strctr from Moton (SFM) o arants xst: Calbratd camra(s) -> K s knon Uss th Essntal Matrx Uncalbratd camra(s) -> K s nknon Uss th Fndamntal Matrx P =? C R, =? C

28 h Fndamntal Matrx Bfor, assmd to kno th camra ntrnsc paramtrs and sd normalzd mag coordnats E p E p F K E K - - K Fndamntal Matrx K K ] [ K ] [ K E K R F R E F