CS 532: 3D Computer Vision 2 nd Set of Notes

Transcription

1 CS 532: 3D Computer Vson 2 nd Set of Notes Instructor: Plppos Mordoa Webpage: E-mal: Plppos.Mordoa@stevens.edu Offce: Leb 25

2 Lecture Outlne 2D projectve transformatons Homograpes Robust estmaton RANSAC Radal dstorton wo-vew geometry Based on sldes by R. Hartley, A. Zsserman, M. Pollefeys and S. Setz 2

3 Projectve ransformatons n 2D Defnton: A projectvty s an nvertble mappng from P 2 to tself suc tat tree ponts, 2, 3 le on te same lne f and only f ( ),( 2 ),( 3 ) do. eorem: A mappng :P 2 P 2 s a projectvty f and only f tere est a non-sngular 33 matr H suc tat for any pont n P 2 reprented by a vector t s true tat ()=H Defnton: Projectve transformaton ' ' 2 ' or ' H DOF projectvty=collneaton=projectve transformaton=omograpy 3

4 Mappng between planes central projecton may be epressed by =H (applcaton of teorem) 4

5 Removng Projectve Dstorton select four ponts n a plane wt known coordnates ' 2 y 3 ' y ' y' ' y ' y ' y' y 33 2 y y y (lnear n j ) (2 constrants/pont, 8DOF 4 ponts needed) Remarks: no calbraton at all necessary, better ways to compute (see later) 5

6 A Herarcy of ransformatons Projectve lnear group Affne group (last row (,,)) Eucldean group (upper left 22 ortogonal) Orented Eucldean group (upper left 22 det ) Alternatvely, caracterze transformaton n terms of elements or quanttes tat are preserved or nvarant e.g. Eucldean transformatons leave dstances uncanged 6

7 Class I: Isometres (so=same, metrc=measure) cos sn sn cos ' ' y t t y y orentaton preservng: orentaton reversng: ' t R H E I R R specal cases: pure rotaton, pure translaton 3DOF ( rotaton, 2 translaton) Invarants: lengt, angle, area 7

8 Class II: Smlartes (sometry + scale) cos sn sn cos ' ' y t s s t s s y y ' t R H s S I R R also know as equ-form (sape preservng) metrc structure = structure up to smlarty (n lterature) 4DOF ( scale, rotaton, 2 translaton) Invarants: ratos of lengt, angle, ratos of areas, parallel lnes 8

9 Class III: Affne ransformatons ' ' y t a a t a a y y ' t A H A non-sotropc scalng! (2DOF: scale rato and orentaton) 6DOF (2 scale, 2 rotaton, 2 translaton) Invarants: parallel lnes, ratos of parallel lengts, ratos of areas DR R R A 2 D 9

10 Class VI: Projectve ransformatons ' H P A v t v v v, v 2 8DOF (2 scale, 2 rotaton, 2 translaton, 2 lne at nfnty) Acton s non-omogeneous over te plane Invarants: cross-rato of four ponts on a lne (rato of ratos)

11 Overvew of ransformatons Projectve 8dof Affne 6dof Smlarty 4dof Eucldean 3dof 2 3 a a2 sr sr r r2 2 a a r r sr sr t t y t t y t t y Concurrency, collnearty, order of contact (ntersecton, tangency, nflecton, etc.), cross rato Parallellsm, rato of areas, rato of lengts on parallel lnes (e.g mdponts), lnear combnatons of vectors (centrods). e lne at nfnty l Ratos of lengts, angles. e crcular ponts I,J lengts, areas.

12 Homework Warp te basketball court from ts mage to a new mage so tat t appears as f te new mage was taken from drectly above Wat are we mssng? 2

13 Image Warpng Sldes by Steve Setz 3

14 Image ransformatons 4

15 Parametrc (Global) Warpng ransformaton s a coordnate-cangng macne: p = (p) Wat does t mean tat s global? It s te same for any pont p It can be descrbed by just a few numbers (parameters) s represented as a matr (see prev. sldes): p = M*p 5

16 Image Warpng Gven a coordnate transform (,y ) = (,y) and a source mage f(,y), ow do we compute a transformed mage g(,y ) = f((,y))? 6

17 Forward Warpng Send eac pel f(,y) to ts correspondng locaton (,y ) = (,y) n te second mage Q: wat f te pel lands between two pels? 7

18 Forward Warpng Send eac pel f(,y) to ts correspondng locaton (,y ) = (,y) n te second mage Q: wat f te pel lands between two pels? A: Dstrbute color among negborng pels (splattng) 8

19 Inverse Warpng Get eac pel g(,y ) from ts correspondng locaton (,y) = - (,y ) n te frst mage Q: wat f pel comes from between two pels? 9

20 Inverse Warpng Get eac pel g(,y ) from ts correspondng locaton (,y) = - (,y ) n te frst mage Q: wat f pel comes from between two pels? A: nterpolate color value from negbors Blnear nterpolaton typcally used 2

21 Blnear Interpolaton 2

22 Forward vs. Inverse Warpng Wc s better?... 22

23 Parameter Estmaton Sldes by R. Hartley, A. Zsserman and M. Pollefeys 23

24 Homograpy: Number of Measurements Requred At least as many ndependent equatons as degrees of freedom requred Eample: ' H λ y y 2 ndependent equatons / pont 8 degrees of freedom

25 Appromate solutons Mnmal soluton 4 ponts yeld an eact soluton for H More ponts No eact soluton, because measurements are neact ( nose ) Searc for best accordng to some cost functon Algebrac or geometrc/statstcal cost 25

26 Drect Lnear ransformaton (DL) H H H 3 2 y w w y H y w y w w y,, A 26

27 Drect Lnear ransformaton (DL) Equatons are lnear n 3 2 y w y w A A A 3 2 w y A Only 2 of 3 are lnearly ndependent (ndeed, 2 eq/pt) 27

28 Drect Lnear ransformaton (DL) 3 2 w y w (only drop trd row f w ) Holds for any omogeneous representaton, e.g. (,y,) 28

29 Drect Lnear ransformaton (DL) Solvng for H A A A A A Sze of A s 89, but rank 8 rval soluton s = 9 s not nterestng -D null-space yelds soluton of nterest, pck for eample te one wt 29

30 Drect Lnear ransformaton (DL) Over-determned soluton A A A No eact soluton because of neact measurement.e. nose 2 n Fnd appromate soluton - Addtonal constrant needed to avod, e.g. A - not possble, so mnmze A 3

31 DL Algortm Objectve Gven n 4 2D to 2D pont correspondences { }, determne te 2D omograpy matr H suc tat =H Algortm () For eac correspondence compute A. Usually only two frst rows needed. () Assemble n 29 matrces A nto a sngle 2n9 matr A () Obtan SVD of A. Soluton for s last column of V (v) Determne H from 3

32 Inomogeneous soluton ' ' ~ ' ' ' ' ' ' ' ' ' ' w w y y w w w y w y y y w w w y w Snce can only be computed up to scale, pck j =, e.g. 9 =, and solve for 8-vector ~ Solve usng Gaussan elmnaton (4 ponts) or usng lnear least-squares (more tan 4 ponts) However, f 9 = ts approac fals Also poor results f 9 close to zero erefore, not recommended 32

33 Normalzng ransformatons Snce DL s not nvarant to transformatons, wat s a good coce of coordnates? e.g. ranslate centrod to orgn Scale to a 2 average dstance to te orgn Independently on bot mages norm w w w / 2 / 2 33

34 Importance of Normalzaton 3 2 y y y y y y y ~ 2 ~ 2 ~ 2 ~ 2 ~ 4 ~ 4 ~ 2 orders of magntude dfference! Monte Carlo smulaton for dentty computaton based on 5 ponts (not normalzed normalzed) 34

35 Normalzed DL Algortm Objectve Gven n 4 2D to 2D pont correspondences { }, determne te 2D omograpy matr H suc tat =H Algortm () Normalze ponts ~ norm, ~ norm () Apply DL algortm to ~ ~, () Denormalze soluton - H norm H ~ norm 35

36 RANSAC Sldes by R. Hartley, A. Zsserman and M. Pollefeys 36

37 Robust Estmaton Wat f set of matces contans gross outlers? 37

38 RANSAC Objectve Robust ft of model to data set S wc contans outlers Algortm () Randomly select a sample of s data ponts from S and nstantate te model from ts subset. () Determne te set of data ponts S wc are wtn a dstance tresold t of te model. e set S s te consensus set of samples and defnes te nlers of S. () If te subset of S s greater tan some tresold, reestmate te model usng all te ponts n S and termnate (v) If te sze of S s less tan, select a new subset and repeat te above. (v) After N trals te largest consensus set S s selected, and te model s re-estmated usng all te ponts n te subset S 38

39 How Many Samples? Coose N so tat, wt probablty p, at least one random sample s free from outlers. e.g. p=.99 e p N s N p/ log e log s proporton of outlers e s 5% % 2% 25% 3% 4% 5%

40 Acceptable Consensus Set ypcally, termnate wen nler rato reaces epected rato of nlers en 4

41 Adaptvely Determnng te Number of Samples e s often unknown a pror, so pck worst case, e.g. 5%, and adapt f more nlers are found, e.g. 8% would yeld e=.2 N=, sample_count = Wle N >sample_count repeat Coose a sample and count te number of nlers Set e=-(number of nlers)/(total number of ponts) Recompute N from e Increment te sample_count by ermnate N p/ log e log s 4

42 Oter robust algortms RANSAC mamzes number of nlers LMedS mnmzes medan error Not recommended: case deleton, teratve least-squares, etc. 42

43 Automatc Computaton of H Objectve Compute omograpy between two mages Algortm () Interest ponts: Compute nterest ponts n eac mage () Putatve correspondences: Compute a set of nterest pont matces based on some smlarty measure () RANSAC robust estmaton: Repeat for N samples (a) Select 4 correspondences and compute H (b) Calculate te dstance d for eac putatve matc (c) Compute te number of nlers consstent wt H (d <t) Coose H wt most nlers (v) Optmal estmaton: re-estmate H from all nlers by mnmzng ML cost functon wt Levenberg-Marquardt (v) Guded matcng: Determne more matces usng predcton by computed H Optonally terate last two steps untl convergence 43

44 Determne Putatve Correspondences Compare nterest ponts Smlarty measure: SAD, SSD, ZNCC n small negborood If moton s lmted, only consder nterest ponts wt smlar coordnates 44

45 Eample: robust computaton Interest ponts (5/mage) (6448) #n -e adapt. N 6 2% 2M 3% 2.5M 44 6% 6, % 2, % % 43 Putatve correspondences (268) (Best matc,ssd<2,±32) Outlers (7) (t=.25 pel; 43 teratons) Inlers (5) Fnal nlers (262) 45

46 Radal Dstorton and Undstorton Sldes by R. Hartley, A. Zsserman and M. Pollefeys 46

47 Radal Dstorton sort and long focal lengt 47

48 48

49 49

50 ypcal Undstorton Model Correcton of dstorton Coce of te dstorton functon and center Computng te parameters of te dstorton functon () Mnmze wt addtonal unknowns () Stragten lnes () 5

51 Wy Undstort? 5

52 wo-vew Geometry Sldes by R. Hartley, A. Zsserman and M. Pollefeys 52

53 ree questons: () Correspondence geometry: Gven an mage pont n te frst mage, ow does ts constran te poston of te correspondng pont n te second mage? () Camera geometry (moton): Gven a set of correspondng mage ponts { }, =,,n, wat are te cameras P and P for te two vews? () Scene geometry (structure): Gven correspondng mage ponts and cameras P, P, wat s te poston of (ter premage) X n space? 53

54 e Eppolar Geometry C, C,, and X are coplanar 54

55 e Eppolar Geometry Wat f only C,C, are known? 55

56 e Eppolar Geometry All ponts on project on l and l 56

57 e Eppolar Geometry Famly of planes and lnes l and l Intersecton n e and e 57

58 e Eppolar Geometry eppoles e, e = ntersecton of baselne wt mage plane = projecton of projecton center n oter mage = vansng pont of camera moton drecton an eppolar plane = plane contanng baselne (-D famly) an eppolar lne = ntersecton of eppolar plane wt mage (always come n correspondng pars) 58

59 Eample: Convergng Cameras 59

60 Eample: Moton Parallel to Image Plane (smple for stereo rectfcaton) 6

61 Eample: Forward Moton e e 6

62 e Fundamental Matr F algebrac representaton of eppolar geometry l' we wll see tat mappng s a (sngular) correlaton (.e. projectve mappng from ponts to lnes) represented by te fundamental matr F 62

63 e Fundamental Matr F correspondence condton e fundamental matr satsfes te condton tat for any par of correspondng ponts n te two mages ' F ' l' 63

64 e Fundamental Matr F X λ P λc PP I l F P'C P' P e' P' P P Xλ (note: doesn t work for C=C F=) 64

65 e Fundamental Matr F F s te unque 33 rank 2 matr tat satsfes F= for all () ranspose: f F s fundamental matr for (P,P ), ten F s fundamental matr for (P,P) () Eppolar lnes: l =F & l=f () Eppoles: on all eppolar lnes, tus e F=, e F=, smlarly Fe= (v) F as 7 d.o.f.,.e. 33-(omogeneous)-(rank2) (v) F s a correlaton, projectve mappng from a pont to a lne l =F (not a proper correlaton,.e. not nvertble) 65

66 wo Vew Geometry Computaton: Lnear Algortm For every matc (m,m ): ' F ' f ' yf2 ' f3 y' f2 y' yf22 y' f23 f3 yf32 f33 separate known from unknown ', ' y, ', y', y' y, y',, y, f, f, f, f, f, f, f, f, f (data) (unknowns) (lnear) ' ' y ' y' y' y y' y ' n n ' n yn ' n y' n n y' n yn y' n n yn f Af 66

67 Benefts from avng F Gven a pel n one mage, te correspondng pel as to le on eppolar lne Searc space reduced from 2-D to -D 67

68 Image Par Rectfcaton smplfy stereo matcng by warpng te mages Apply projectve transformaton so tat eppolar lnes correspond to orzontal scanlnes e e He map eppole e to (,,) try to mnmze mage dstorton problem wen eppole n (or close to) te mage 68

69 Planar Rectfcaton (standard approac) Brng two vews to standard stereo setup (moves eppole to ) (not possble wen n/close to mage) 69

70 7