Computer Vision. The 2D projective plane and it s applications. HZ Ch 2. In particular: Ch 2.1-4, 2.7, Szelisky: Ch 2.1.1, 2.1.2

Transcription

1 Computer Vson e D projectve plane and t s applcatons HZ C. In partcular: C.-4,.7, Szelsk: C..,.. Estmaton:HZ: C , cursorl Rcard Hartle and Andrew Zsserman, Multple Vew Geometr, Cambrdge Unverst Publsers, nd ed. 4

2 Homogeneous coordnates Homogeneous representaton of D ponts and lnes a + b + c = ( a,b,c) (,, ) = e pont les on te lne l f and onl f l = Note tat scale s unmportant for ncdence relaton ( a,b,c) ~ k( a,b,c), k (,,) ~ k(,,), k equvalence class of vectors, an vector s representatve Set of all equvalence classes n R 3 (,,) forms P ( ) Homogeneous coordnates,, Inomogeneous coordnates (, ) 3 but onl DOF

3 e D projectve plane l X Projectve pont Homogeneous coordnates X Y s X Y Z Z s = Z X Y Inomogeneous equvalent Perspectve magng models d projectve space Eac 3D ra s a pont n P : omogeneous coords. Ideal ponts P s R plus a lne at nfnt l X = X Y

4 Lnes l HZ X Y Z = X X l = A B C Projectve lne ~ a plane troug te orgn l X = X l = AX + BY + CZ = X = X Y l = Ideal lne ~ te plane parallel to te mage lne at nfnt Dualt: For an d projectve propert, a dual propert olds wen te role of ponts and lnes are ntercanged. l = X X X = l l e lne jonng two ponts e pont jonng two lnes

5 Ponts from lnes and vce-versa Intersectons of lnes e ntersecton of two lnes l and l' s = l l' Lne jonng two ponts e lne troug two ponts and ' s l = ' (,,-) Eample = = (,,-) Note: ' = wt [ ] [ ] ' = - z z - -

6 Ideal ponts and te lne at nfnt Intersectons of parallel lnes l = ( ) ( ) a, b, c and l'= a, b, c' l l' = ( b, a,) Eample ( b,-a) ( ) a, b tangent vector normal drecton = = Ideal ponts (,,) Lne at nfnt l (,,) = P = R l Note tat n P tere s no dstncton between deal ponts and oters

7 Concs Curve descrbed b nd -degree equaton n te plane a + b + c + d + e + f = or omogenzed a, 3 a a + b + c + d3 + e3 + f or n matr form 5DOF: C = wt { } a : b : c : d : e : 3 a C = b / d / f b / c e / 3 = d / e / f

8 Fve ponts defne a conc For eac pont te conc passes troug = f e d c b a or ( ) =,.,,,, c ( ) f e d c b a,,,,, = c = c stackng constrants elds

9 angent lnes to concs e lne l tangent to C at pont on C s gven b l=c l C

10 Dual concs A lne tangent to te conc C satsfes l * C l = In general (C full rank): * - C = C Dual concs = lne concs = conc envelopes

11 Degenerate concs A conc s degenerate f matr C s not of full rank e.g. two lnes (rank ) C = lm + ml e.g. repeated lne (rank ) m l C = ll l Degenerate lne concs: ponts (rank ), double pont (rank) Note tat for degenerate concs * ( C ) * C

12 Concs Conc: Eucldean geometr: perbola, ellpse, parabola & degenerate Projectve geometr: equvalent under projectve transform Defned b 5 ponts a + b + c C = omogeneous + d + e + nomogeneous f = a b / d / C = b / c e / d / e / f angent lne Dual conc C* l = C l C * l =

13 Projectve transformatons Homograpes, collneatons, projectvtes 33 nonsngular H maps P to P 8 degrees of freedom determned b 4 correspondng ponts ransformng Lnes? subspaces preserved substtuton dual transformaton ' ' ' 3 = 3 l = l = H l = l =H l = H 3

14 Homograpes a generalzaton of affne and Eucldean transforms Group ransformaton Invarants Dstorton Projectve 8 DOF Affne 6 DOF Metrc 4 DOF Eucldean 3 DOF H H H H P A S E A = v A = O sr = O R = O t v t t t Parallelsm Relatve dst n d Lne at nfnt Relatve dstances Angles Dual conc Lengts Areas l C * dof l dof C *

15 Planar Projectve Warpng = H =...4 A novel vew rendered va four ponts wt known structure HZ

16 Planar Projectve Warpng Orgnal op-down Facng rgt HZ Artfacts are apparent were planart s volated...

17 d Homograpes mages of a plane mages from te same vewpont (Perspectvt)

18 Panoramc magng Appl: Qucktme VR, robot navgaton etc. Homograpes of te world, unte!

19 Image mosacs are sttced b omograpes HZ

20 e lne at nfnt = = = l t l l A A H A e lne at nfnt l s a fed lne under a projectve transformaton H f and onl f H s an affnt Note: But ponts on l can be rearranged to new ponts on l

21 Affne propertes from mages Projecton (Imagng) Rectfcaton Post-processng

22 Affne rectfcaton v v = l3 l4 l = v v = l l v v l l l 3 l [ l ] l l l 3 4 Eercse: Verf Pont transformaton for Aff Rect: [ l l l ], l l = 3 3 H PA [ l l l ] [,,] 3 = H PA = l l H l 3 A

23 Geometrc strata: d overvew Group ransformaton Invarants Dstorton Projectve 8 DOF Affne 6 DOF Metrc 4 DOF Eucldean 3 DOF H H H H P A S E A = v A = O sr = O R = O t v t t t Cross rato Intersecton angenc Parallelsm Relatve dst n d Lne at nfnt Relatve dstances Angles Dual conc Lengts Areas l C * dof l dof C *

24 Parameter estmaton n geometrc transforms cs48 Useful n D omograp Gven a set of (, ), compute H ( =H ) 3D to D camera projecton Gven a set of (X, ), compute P ( =PX ) Fundamental matr Gven a set of (, ), compute F ( F =) rfocal tensor Grad researcgven a set of (,, ), compute

25 Mat tools : Solvng Lnear Sstems If m = n (A s a square matr), ten we can obtan te soluton b smple nverson: If m > n, ten te sstem s over-constraned and A s not nvertble Use Matlab \ to obtan least-squares soluton = A\b to A =b nternall Matlab uses QR-factorzaton (cmput48/34) to solve ts. Can also wrte ts usng pseudonverse A + = (A A) - A to obtan least-squares soluton = A + b

26 Fttng Lnes A -D pont = (, ) s on a lne wt slope m and ntercept b f and onl f = m + b Equvalentl, So te lne defned b two ponts, s te soluton to te followng sstem of equatons:

27 Fttng Lnes Wt more tan two ponts, tere s no guarantee tat te wll all be on te same lne Least-squares soluton obtaned from pseudonverse s lne tat s closest to all of te ponts courtes of Vanderblt U.

28 Eample: Fttng a Lne Suppose we ave ponts (, ), (5, ), (7, 3), and (8, 3) en and = A + b = (.357,.857) Matlab: = A\b

29 Eample: Fttng a Lne

30 Homogeneous Sstems of Equatons Suppose we want to solve A = ere s a trval soluton =, but we don t want ts. For wat oter values of s A close to? s s satsfed b computng te sngular value decomposton (SVD) A = UDV (a non-negatve dagonal matr between two ortogonal matrces) and takng as te last column of V Note tat Matlab returns [U, D, V] = svd(a)

31 Lne-Fttng as a Homogeneous Sstem A -D omogeneous pont = (,, ) s on te lne l = (a, b, c) onl wen a + b + c = We can wrte ts equaton wt a dot product: l =, and ence te followng sstem s mpled for multple ponts,,..., n :

32 Eample: Homogeneous Lne-Fttng Agan we ave 4 ponts, but now n omogeneous form: (,, ), (5,, ), (7, 3, ), and (8, 3, ) Our sstem s: akng te SVD of A, we get: compare to = (.357,.857)

33 Parameter estmaton n geometrc transforms cs48 Useful n D omograp Gven a set of (, ), compute H ( =H ) 3D to D camera projecton Gven a set of (X, ), compute P ( =PX ) Fundamental matr Gven a set of (, ), compute F ( F =) rfocal tensor Grad researcgven a set of (,, ), compute

34 Estmatng Homograp H gven mage ponts = H =...4 A novel vew rendered va four ponts wt known structure HZ

35 Number of measurements requred At least as man ndependent equatons as degrees of freedom requred Eample: λ w' = '= H ndependent equatons / pont 8 degrees of freedom 4 8

36 Appromate solutons Mnmal soluton 4 ponts eld an eact soluton for H More ponts No eact soluton, because measurements are neact ( nose ) Searc for best accordng to some cost functon Algebrac or geometrc/statstcal cost

37 Man was to solve: Dfferent Cost functons => dfferences n soluton Algebrac dstance Geometrc dstance Reprojecton error Comparson Geometrc nterpretaton

38 Gold Standard algortm Cost functon tat s optmal for some assumptons Computatonal algortm tat mnmzes t s called Gold Standard algortm Oter algortms can ten be compared to t

39 Estmatng H: e Drect Lnear ransformaton (DL) Algortm =HX vectors, so HX and need onl be n te same drecton, not strctl equal s an equaton nvolvng omogeneous We can specf same drectonalt b usng a cross product formulaton: H =

40 Drect Lnear ransformaton (DL) H = H = = H 3 = w w H = w w ( ) w =,, A =

41 Drect Lnear ransformaton (DL) Equatons are lnear n 3 = w w A A A 3 = + + w A = Onl out of 3 are lnearl ndependent (ndeed, eq/pt) 3 = w w (onl drop trd row f w ) Holds for an omogeneous representaton, e.g. (,,)

42 Drect Lnear ransformaton (DL) Solvng for H A A A A 3 4 = A = sze A s 89 or 9, but rank 8 rval soluton s = 9 s not nterestng -D null-space elds soluton of nterest pck for eample te one wt =

43 Drect Lnear ransformaton (DL) Over-determned soluton A A M A n = No eact soluton because of neact measurement.e. nose A = Fnd appromate soluton - Addtonal constrant needed to avod, e.g. A = - not possble, so mnmze A =

44 DL algortm Objectve Gven n 4 D to D pont correspondences { }, determne te D omograp matr H suc tat =H Algortm () () For eac correspondence compute A. Usuall onl two frst rows needed. Assemble n 9 matrces A nto a sngle n9 matr A () Obtan SVD of A. Soluton for s last column of V (v) Determne H from (resape)

45 Inomogeneous soluton = ' ' ~ ' ' ' ' ' ' ' ' ' ' w w w w w w w w w w Snce can onl be computed up to scale, pck j =, e.g. 9 =, and solve for 8-vector ~ Solve usng Gaussan elmnaton (4 ponts) or usng lnear least-squares (more tan 4 ponts) However, f 9 = ts approac fals also poor results f 9 close to zero erefore, not recommended for general omograpes Note 9 =H 33 = f orgn s mapped to nfnt [ ] H H l = =

46 Normalzng transformatons Snce DL s not nvarant to coordnate transforms, wat s a good coce of coordnates? e.g. ranslate centrod to orgn Scale to a average dstance to te orgn Independentl on bot mages Or norm w + w/ = / w +

47 Normalzed DL algortm Objectve Gven n 4 D to D pont correspondences { }, determne te D omograp matr H suc tat =H Algortm () () Normalze ponts Appl DL algortm to () Denormalze soluton ~ = norm, ~ = norm ~ ~, - H = H~ norm norm

48 Importance of normalzaton 3 = ~ ~ ~ ~ ~ 4 ~ 4 ~ orders of magntude dfference!

49 Degenerate confguratons H? H? 3 (case A) (case B) 4 3 Constrants: H = =,,3,4 Defne: en, H = 4l * = l =, * ( ) =,, 3 ( l 4 ) = 4 H 4 * H 4 = 4 k H * s rank- matr and tus not a omograp If H * s unque soluton, ten no omograp mappng (case B) If furter soluton H est, ten also αh * +βh (case A) (-D null-space n stead of -D null-space)

50 Frst 3D proj geom en revew and more on camera models en followng P estmaton

51 A 3D Vson Problem: Mult-vew geometr - resecton Projecton equaton =P X Resecton:,X P Gven mage ponts and 3D ponts calculate camera projecton matr.

52 Estmatng camera matr P Gven a number of correspondences between 3- D ponts and ter -D mage projectons X, we would lke to determne te camera projecton matr P suc tat = PX for all

53 A Calbraton arget Y X Z X courtes of B. Wlburn

54 Estmatng P: e Drect Lnear ransformaton (DL) Algortm = PX vectors, so PX and need onl be n te same drecton, not strctl equal s an equaton nvolvng omogeneous We can specf same drectonalt b usng a cross product formulaton:

55 DL Camera Matr Estmaton: Prelmnares Let te mage pont = (,, w ) (remember tat X as 4 elements) Denotng te jt row of P b p j (a 4-element row vector), we ave:

56 DL Camera Matr Estmaton: Step en b te defnton of te cross product, PX s:

57 DL Camera Matr Estmaton: Step e dot product commutes, so p j X = X p j, and we can rewrte te precedng as:

58 DL Camera Matr Estmaton: Step 3 Collectng terms, ts can be rewrtten as a matr product: were = (,,, ). s s a 3 matr tmes a -element column vector p = (p, p, p 3 )

59 Wat We Just Dd

60 DL Camera Matr Estmaton: Step 4 ere are onl two lnearl ndependent rows ere e trd row s obtaned b addng tmes te frst row to tmes te second and scalng te sum b -/w

61 DL Camera Matr Estmaton: Step 4 So we can elmnate one row to obtan te followng lnear matr equaton for te t par of correspondng ponts: Wrte ts as A p =

62 DL Camera Matr Estmaton: Step 5 Remember tat tere are unknowns wc generate te 3 4 omogeneous matr P (represented n vector form b p) Eac pont correspondence elds equatons (te two row of A ) We need at least 5 ½ pont correspondences to solve for p Stack A to get omogeneous lnear sstem Ap =

63 Eperment:

64 Radal Dstorton sort and long focal lengt

65 Radal Dstorton

66 Radal Dstorton

67 Radal Dstorton Correcton of dstorton Coce of te dstorton functon and center Computng te parameters of te dstorton functon () Mnmze wt addtonal unknowns () Stragten lnes ()