CSc 6974 and ECSE 6966 Math. Tech. for Vson, Graphcs and Robotcs Lecture 21, Aprl 17, 2006 Estmatng A Plane Homography Overvew We contnue wth a dscusson of the major ssues, usng estmaton of plane projectve transformatons (homographes) as an example. We wll dscuss: The basc problem and the DLT algorthm. Dstance metrcs Objectve functons Normalzaton Homography (Plane Projectve Transform) Estmaton Consder two perspectve mages of a planar surface. Let {x 1,..., x n } be the mage projectons of a set of ponts on ths plane n one mage, and let {x 1,..., x n} be the projectons of the same set of ponts n the second mage. We wll assume we know that the ponts x and x correspond to each other wthout sayng how we know ths (and avodng a hard problem). The problem s to estmate the parameters of the matrx H that maps x as close to x as possble. Recall that and s defned up to non-zero scale factor. freedom. H = h 11 h 12 h 13 h 21 h 22 h 23 (1) h 31 h 32 h 33 Hence H has 8 degrees of
Drect Lnear Transform (DLT) Algorthm For the followng dscusson, wrte h 1 H = h 2. (2) h 3 H maps x onto x f x Hx = 0 (3) Wrtng x = (u, v, w ), ths constrant s expressed as v x h 3 x w h 2 x Hx = w h 1 x u h 3 x = 0, (4) u h 2 x v h 1 x wth obvous smplfcatons when the ponts are normalzed so that w = 1. Rearrangng, ths can be wrtten as 0 w x v x w x 0 u x v x u x 0 h 1 = A h = 0. (5) h 2 h 3 Notce we have stacked the parameters of H nto the 9 component column vector h. A s 3 9 and expresses the constrants on h from correspondence. It s only rank 2, however, because the 3 rd row can be wrtten as a lnear combnaton of the frst two. We could therefore elmnate the last row. We need 4 pont correspondences to establsh the mnmum of 8 constrants on h (remember, scale does not matter). Ths gves A 1 A 2 A 3 A 4 h = Ah = 0. (6) The soluton s the unt vector spannng the nullspace of A (assumng t s rank 1). Of course ths can be found from the SVD of A. Exstence and unqueness condtons: When three ponts are collnear n I, a rank 1 matrx H satsfes the condtons x H x = 0, {1, 2, 3, 4}. Ths matrx H does not satsfy x = H x. 2
In general, f no three of the ponts are collnear, there s a unque soluton. When more than four pont correspondences are nvolved, then the constrants from each pont are stacked n A: A 1 A = A 2... (7) and the least-squares DLT soluton s found usng the SVD as earler. See Secton 3.1 (4.1 n the new edton) of Hartley and Zsserman for more detals and dscusson of dfferent constrants. It s mportant to note that the practcal results of applyng ths algorthm are VERY POOR. The rest of these lecture notes lead to a better estmaton technque. Dstance Metrcs: Algebrac and Geometrc Dstance A k Our next consderaton s the pont dstance measures Statng thngs a bt abstractly, we are gven some knd of data pont vector p, a parameter vector a, and an mplct functon f(p; a) representng an algebrac varety such that f(p; a) = 0 (8) f and only f pont p s on the varety determned by parameter vector a and functon f. The algebrac dstance of a pont p s just f(p; a) (9) Ths dstance s zero for ponts on the varety and ncreases for ponts not on the varety. Examples: For lnes n two dmensons, p = (x, y, 1), a = (a 0, a 1, a 2 ) and f(p; a) = p a. (10) For concs n two dmensons, p = (x, y, 1), a s the parameters of the conc matrx C and f(p; a) = p Cp. (11) 3
For the plane homography, p = (u, v, u, v ), a = (h 1, h 2, h 3 ), and f(p; a = (h 1 x v h 3 ) 2 + (h 2 x v h 3 ) 2 (12) Ths s also A h 2 from above (when we only consder the frst two rows of A ). The geometrc dstance s smply the dstance from p to the nearest pont p such that f(p ; a) = 0. Ths sets up a problem n Lagrange multplers, just lke we solved for lnes, crcles and concs. In other words, the dstance between a pont x and the varety s found by fndng the p and then the dstance s smply F (p, λ) = p x 2 2λf(p; a), (13) p x. (14) In some cases, the relatonshp between the algebrac and geometrc dstances s straghtforward. Examples nclude the dstances of ponts to lnes and planes. Even stronger, for the affne transform estmaton the algebrac and geometrc errors are the same! Good approxmatons to the geometrc dstance can be obtaned usng the what s known as the Sampson error. We wll dscuss ths next. Sampson Error Wrte x = p x. Then the Lagrangan form n (13) may be rewrtten as F ( x, λ) = x 2 2λf(x + x; a), (15) Takng the dervatve wth respect to x and λ yelds the smultaneous equatons x λ f(x + x; a) = 0 (16) f(x + x; a) = 0 (17) (Remember, gradent vectors, f, are row vectors; hence the need for the transpose n the frst equaton.) To obtan the Sampson error, two approxmatons are requred: In the frst term, the gradent vector s evaluated at x nstead of x + x. Ths gradent wll be denoted smply as f. The second term s replaced by a frst order Taylor expanson. 4
Combned, these results yeld the smplfed equatons: x λ f = 0 (18) f(x; a) + f x = 0 (19) These two equatons may be solved to yeld the Sampson dstance approxmaton: f(x; a) x = (20) ( f f ) 1/2 Ths s the algebrac dstance normalzed by the gradent magntude. Notce that ths s the exact Eucldean dstance for lnes n R 2 and planes n R 3. Possble Objectve Functons for Homography Estmaton Buldng from our understandng of dstances, we can thnk of a varety of objectve functons that can be mnmzed n plane homography estmaton: Algebrac dstance: where A s constructed from x and x as above. Sampson error, to be derved n class. A h 2 (21) Geometrc dstance n one mage d(x, Hx ) 2 (22) where d(x, Hx) s the geometrc dstance. Symmetrc transfer error: d(x, Hx ) 2 + d(x, H 1 x ) 2 (23) Reprojecton error. Here, the true pont locatons are estmated as nusance parameters to estmatng the parameters of H: d(x, ˆx ) 2 + d(x, ˆx ) 2 subject to ˆx = Hˆx, (24) Most of these lead to non-lnear mnmzaton problems. These methods wll be dscussed n a later lecture. For the rest of ths lecture we wll consder an mportant way to mprove the smple DLT / algebrac error technque. 5
Normalzaton, Normalzaton, NORMALIZATION! As a motvatng example, consder the smple case of fttng a parabola y = ax 2 +bx+c to a set of ponts {(x, y )}. Ths leads to the least-squares mnmzaton problem Xa y 2, (25) where a = (a, b, c), y = (y 1,..., y n ), and x 2 1 x 1 1 x 2 2 x 2 1 X =... x 2 n x n 1 (26) The mportant thng to note here s the dfferent unts n the dfferent columns of X. Thnk of the effect of a small error n x on the entres of each of the 3 dfferent columns. A smlar stuaton arses n the terms of the matrx A n the DLT homography estmaton: 0 x v x A = x 0 u x (27) v x u x 0 These dfferences become problems both statstcally and numercally when the data are corrupted by nose. The most straghtforward approach s normalzaton, whch should AL- WAYS be used, even wth other methods. More sophstcated technques drectly address the heteroscedastcty (non-statonary nose) n the data. The standard technque s to center and normalze the values n each column separately. For example, n parabola fttng: 1. Compute µ 2 = x2 /N, c 2 = x2 µ 2 /N, µ 1 = x /N, and c 1 = x µ 1 /N. 2. Replace X by (x 2 1 µ 2 )/c 2 (x 1 µ 1 )/c 1 1 X (x 2 2 µ 2 )/c 2 (x 2 µ 1 )/c 1 1 = (28)... (x 2 n µ 2 )/c 2 (x n µ 1 )/c 1 1 3. Solve the least-squares mnmzaton problem 4. Recover a from a and µ 2, c 2, µ 1, c 1. X a y 2 (29) 6
Normalzed DLT: A Much Better Soluton Hartley and Zsserman recommend a slghtly dfferent approach for normalzaton n the DLT estmaton of H: 1. Normalze each u, v, u and v separately so that the average magntude of each s 1. If µ u, µ v, µ u and µ v are the centers, and s u, s v, s u and s v are the scalng factors, then the normalzaton process may be expressed as the affne transformatons x = Tx andx = T x (30) where and 1/s u 0 µ u /s u T = 0 1/s v µ v /s v (31) 0 0 1 1/s u 0 µ u /s u T = 0 1/s v µ v /s v (32) 0 0 1 Apply the DLT algorthm by formng matrx A from the x x. Call the resultng matrx H. The desred DLT soluton to the orgnal problem s values H = T 1 H T. (33) 2. We wll have a homework assgnment nvolvng the mplementaton of the DLT and the normalzed DLT algorthm. My experence s that ths a problem and soluton you have to see to beleve! 7
Practce/Test Problems 1. Show that the algebrac dstance and geometrc dstance are the same for estmatng an affne transformaton. 2. Derve the Sampson error approxmaton for the dstance of a pont x to a conc. 3. Derve the normalzed equatons for a least-squares method to estmate the parameters of a conc. Use each of the two methods descrbed above. Homework Problems People have trouble seeng the sgnfcance of normalzaton untl they try t practce. Therefore, n ths problem you wll wrte a program that mplements the DLT algorthm for homogeneous transformaton estmaton. You wll mplement t wth and wthout normalzaton. I strongly urge you to use Matlab or Maple or a good C++ lbrary that has an SVD mplementaton. The data wll be provded as a text fle. Each lne of nput of ths fle wll have four floatng pont values, x, y, x, y, correspondng to a par of correspondng mage locatons. Data wll be avalable on the course web page. The output from your program should be the parameters of the estmated matrx H for the DLT and the normalzed DLT algorthms, and two plots. Let H D and H N be the two estmated matrces. The frst plot should contan the ponts (x, y ) and the ponts H D(x, y, 1) T. The second plot should contan the ponts (x, y ) and the ponts H N(x, y, 1) T. The matxh from whch the ponts were generated wll be posted on the web. Submt prnted copes of your source code and your output. Your code should be clean and well-documented. 8