MM Research Preprnts,131 165 No. 18, Dec. 1999. Beng 131 Coordnate-Free Proectve Geometry for Computer Vson Hongbo L, Gerald Sommer 1. Introducton How to represent an mage pont algebracally? Gven a Cartesan coordnate system of the retna plane, an mage pont can be represented by ts coordnates (u, v. If the mage s taen by a pnhole camera, then snce a pnhole camera can be taen as a system that performs the perspectve proecton from three-dmensonal proectve space to twodmensonal one wth respect to the optcal center (See Faugeras, 1993, t s convenent to descrbe a space pont by ts homogeneous coordnates (x, y, z, 1 and descrbe an mage pont by ts homogeneous coordnates (u, v, 1. In other words, the space of mage ponts can be represented by the space of 3 1 matrces. Ths s the coordnate representaton of mage ponts. There are other representatons whch are coordnate-free. The use of algebras of geometrc nvarants n the coordnate-free representatons can lead to remarable smplfcatons n geometrc computng. Kanatan (1993 uses three-dmensonal affne space for space ponts, and the space of dsplacements of the affne space for mage ponts. In other words, he uses vectors fxed at the orgn of R 3 to represent space ponts, and uses free vectors to represent mage ponts. Then he can use vector algebra to carry out geometrc computng. Ths algebrac representaton s convenent for two-dmensonal proectve geometry, but not for three-dmensonal one. The space representng mage ponts depends nether on the retna plane nor on the optcal center. Bayro-Corrochano, Lasenby and Sommer (1996 use R 4 for both two-dmensonal and three-dmensonal proectve geometres. They use a coordnate system {e 1, e 2, e 3, C} of R 4 to descrbe a pnhole camera, where the e s are ponts on the retna plane and C s the optcal center. Both space ponts and mage ponts are represented by vectors fxed at the orgn of R 4, the only dfference s that an mage pont s n the space spanned by vectors e 1, e 2, e 3. Ths algebrac representaton s convenent for proectve geometrc computatons usng the ncdence algebra formulated n Clfford algebra. However, t always needs a coordnate system for the camera. The space representng mage ponts depends only on the retna plane. We notced that none of these algebrac representatons of mage ponts s related to the optcal center. By ntuton, t s better to represent mage ponts by vectors fxed at the optcal center. The above-mentoned coordnate-free representatons do not have ths property. Hestenes (1966 proposed a technque called space-tme splt to realze the Clfford algebra of the Eucldean space n the Clfford algebra of the Mnows space. The technque s later generalzed to proectve splt by Hestenes and Zegler (1991 for proectve geometry. We
132 Hongbo L, Gerald Sommer fnd that a verson of ths technque offers us exactly what we need: three-dmensonal lnear spaces mbedded n a four-dmensonal one, whose orgns do not concur wth that of the fourdmensonal space but whose Clfford algebras are realzed n that of the four-dmensonal space. Let C be a vector n R 4. It represents ether a space pont or a pont at nfnty of the space. Let M be another vector n R 4. The mage of the space pont or pont at nfnty M by a pnhole camera wth optcal center C can be descrbed by C M. The mage ponts can be represented by the three-dmensonal space C R 4 {C X X R 4 }. The Clfford algebra of the space C R 4 can be realzed n the Clfford algebra of R 4 by the theorem of proectve splt proposed later n ths chapter. The space representng mage ponts depends only on the optcal center. The representaton s completely proectve and completely coordnate-free. Usng ths new representaton and the verson of Grassmann-Cayley algebra formulated by Hestenes and Zegler (1991 wthn Clfford algebra, we have reformulated camera modelng and calbraton, eppolar and trfocal geometres, relatons among eppoles, eppolar tensors and trfocal tensors, and determnng dscrete motons from lne correspondences. Remarable smplfcatons and generalzatons are obtaned through the reformulaton, both n concepton and n applcaton. In partcular, we are to derve and generalze all nown constrants on eppolar and trfocal tensors (see Faugeras and Mourran, 1995a, b; Faugeras and Papadopoulo, 1997, 1998 n a systematc way. Ths chapter s arranged as follows: n secton?? we collect some necesssary mathematcal technques, n partcular the theorem of proectve splt n Grassmann-Cayley algebra. In sectons?? and?? we reformulate camera modelng and calbraton, and eppolar and trfocal geometres. In secton?? we derve and generalze the constrants on eppolar and trfocal tensors systematcally. In secton?? we dscuss the problem of determnng dscrete motons from lne correspondences n three cameras. 2. Preparatory Mathematcs 2.1. Dual bases Accordng to Hestenes and Sobczy (1984, let {e 1,..., e n } be a bass of R n and {e 1,..., e n} be the correspondng dual (or recprocal bass, then e ( 1 1 (e 1 ě e n, e ( 1 1 (e 1 ě e n, (2.1 for 1 n. Here s the dual operator n G n wth respect to e 1 e n. The bass {e 1,..., e n } nduces a bass {e 1 e s 1 <... < s n} for the s-vector subspace G s n of the Clfford algebra G n of R n. We have (e 1 e s e s e ( 1 + + s+s(s+1/2 (e 1 ě 1 ě s e n. (2.2
Coordnate-Free Proectve Geometry for Computer Vson 133 Let x G s n, then x 1 <...< s n 1 <...< s n x (e 1 e s e 1 e s ( 1 + + s+s(s+1/2 e 1 e s (e 1 ě 1 ě s e n x. (2.3 Let an nvertble transformaton T of R n maps {e 1,..., e n } to a bass {e 1,..., e n}. Let T (T T 1. Then T maps the dual bass {e 1,..., e n} to the dual bass {e 1,..., e n }. Any lnear mappng T : R n R m has a tensor representaton n R n R m. Assume that T maps the bass {e 1,..., e n } to vectors {e 1,..., e n}. Then n T e e. (2.4 1 For example, let Π n be the dentty transformaton of R n, then n tensor representaton, Π n n e e for any bass {e 1,..., e n }. 1 2.2. Proectve and affne spaces An n-dmensonal real proectve space P n can be realzed n the space R n+1, where a proectve r-space s an (r + 1-dmensonal lnear subspace. In G n+1, a proectve r-space s represented by an (r + 1-blade, and the representaton s unque up to a nonzero scale. Throughout ths chapter we use x y to denote that f x, y are scalars, they are equal up to a nonzero ndex-free scale, otherwse they are equal up to a nonzero scale. An n-dmensonal real affne space A n can be realzed n the space R n+1 as a hyperplane away from the orgn. Let e 0 be the vector from the orgn to the hyperplane and orthogonal to the hyperplane. When e 2 0 1, a vector x Rn+1 s an affne pont f and only f x e 0 1. An r-dmensonal affne plane s the ntersecton of an (r + 1-dmensonal lnear subspace of R n+1 wth A n, and can be represented by an (r+1-blade of G n+1 representng the subspace. The space of dsplacements of A n s defned as A n {x y x, y A n }. It s an n- dmensonal lnear subspace of R n+1. Any element of t s called a drecton. When A n s taen as an (n 1-dmensonal proectve space, any element n t s called a pont at nfnty, and A n s called the space at nfnty of A n. Let I n e 0 I n+1. Then t represents the space A n. The mappng In : x e 0 x I n x, for x G n+1, (2.5 maps G n+1 to G( A n, called the boundary mappng. When I n s fxed, In s often wrtten as. Geometrcally, f I r+1 represents an r-dmensonal affne space, then I r represents ts space at nfnty. For example, when x, y are both affne ponts, (x y y x s the pont at nfnty of lne xy. Let {e 1,..., e n+1 } be a bass of R n+1. If e n+1 A n, e 1,..., e n A n, the bass s called a Cartesan coordnate system of A n, wrtten as {e 1,..., e n ; e n+1 }. The affne pont e n+1 s
134 Hongbo L, Gerald Sommer called the orgn. Let x A n, then x e n+1 + n λ e. (λ 1,..., λ n s called the Cartesan coordnates of x wth respect to the bass. Below we lst some propertes of the three-dmensonal proectve (or affne space when descrbed n G 4. Two planes N, N are dentcal f and only f N N 0, where N, N are 3-blades. A lne L s on a plane N f and only f L N 0, where L s a 2-blade. Two lnes L, L are coplanar f and only f L L 0, or equvalently, f and only f L L 0. A pont A s on a plane N f and only f A N 0, or equvalently, f and only f A N 0. Here A s a vector. 1 A pont A s on a lne L f and only f A L 0. Three planes N, N, N are concurrent f and only f N N N 0. For two lnes L, L, L L L L. For pont A and plane N, A N A N. 2.3. Proectve splts The followng s a modfed verson of the technque proectve splt. Defnton 2.1. Let C be a blade n G n. The proectve splt P C of G n wth respect to C s the followng transformaton: x C x, for x G n. Theorem 2.1. [Theorem of proectve splt n Grassmann-Cayley algebra 1 ] Let C be an r-blade n G n. Let C G n {C x x G n }. Defne n t two products C and C : for x, y G n, (C x C (C y C x y, (2.6 (C x C (C y (C x (C y, and defne (C x C C (C x. (2.7 Then vector space C G n equpped wth C, C, C s a Grassmann-Cayley algebra somorphc to G n r, whch s taen as a Grassmann-Cayley algebra. 1 Theorem?? can be generalzed to the followng one, whch s nevertheless not needed n ths chapter: [Theorem of proectve splt n Clfford algebra] Let C be a blade n G n. The space C G n equpped wth the followng outer product C and nner product C s a Clfford algebra somorphc to G(C : for x, y G n. (C x C (C y C x y, (x C C (C y C 2 C ((x C (C y,
Coordnate-Free Proectve Geometry for Computer Vson 135 Proof. Let C R n {C x x R n }. It s an (n r-dmensonal vector space. By the lnear somorphsm of {λc λ R} wth R, t can be verfed that (C G n, C s somorphc to the Grassmann algebra generated by C R n. A drect computaton shows that the composton of C wth tself s the scalar multplcaton by ( 1 n(n 1/2 C 2. That C G n s a Grassmann-Cayley algebra follows from the dentty (C x C C (C y C ((C x C (C y C, (2.8 whch can be verfed by the defntons (?? and (??. Let {e 1,..., e n } be a bass of R n. The proectve splt P C can be wrtten as the composton of the outer product by C and the dentty transformaton. It has the followng tensor representaton: P C n s0 1 <...< s n (C e 1 e s (e 1 e s. (2.9 For example, when C s a vector and P C s restrcted to R n, then In partcular, when {e 1,..., e n 1, C} s a bass of R n, then When P C s restrcted to Gn, 2 then P C n P C (C e e. (2.10 1 n 1 P C (C e e. (2.11 1 < n 1 (C e 1 e 2 (e 1 e 2. (2.12 In partcular, when {e 1,..., e n 1, C} s a bass of R n, then P C (C e 1 e 2 (e e. (2.13 1 < n 1 When n 4, we use the notaton 1 2 to denote that, 1, 2 s an even permutaton of 1, 2, 3. Let ê e 1 e 2, ê e 1 e 2. (2.14 then P C (C ê ê. (2.15 1 The followng theorem establshes a connecton between the proectve splt and the boundary mappng. Theorem 2.2. When C s an affne pont, the boundary mappng realzes an algebrac somorphsm between the Grassmann-Cayley algebras C G n+1 and G( A n.
136 Hongbo L, Gerald Sommer 3. Camera modelng and calbraton 3.1. Pnhole cameras Accordng to Faugeras (1993, a pnhole camera can be taen as a system that performs the perspectve proecton from P 3 to P 2 wth respect to the optcal pont C P 3. To descrbe ths mappng algebracally, let {e 1, e 2, e 3 ; O} be a fxed Cartesan coordnate system of A 3, called the world coordnate system. Let {e C 1, ec 2, ec 3, C} be a bass of R4 satsfyng (e C 1 ec 2 ec 3 C 1, called a camera proectve coordnate system. When C s an affne pont, let e C 3 be the vector from C to the orgn OC of the retna plane (or mage plane, and let e C 1, ec 2 be two vectors n the retna plane. Then {ec 1, ec 2, ec 3 ; C} s a Cartesan coordnate system of A 3, called a camera affne coordnate system. Let M be a pont or pont at nfnty of A 3, and let m C be ts mage. Then M can be represented by ts homogeneous coordnates whch s a 4 1 matrx, and m C can be represented by ts homogeneous coordnates whch s a 3 1 matrx. The perspectve proecton can then be represented by a 3 4 matrx. In our approach, we descrbe a pnhole camera wth optcal center C, whch s ether an affne pont or a pont at nfnty of A 3, as a system performng the proectve splt of G 4 wth respect to C R 4. To see how ths representaton wors, we frst derve the matrx of the proect splt P C restrcted to R 4. We consder the case when the camera coordnate system {e C 1, ec 2, ec 3, C} s affne. Accordng to (??, P C (C e C e C. (3.1 1 In the camera coordnate system, let the coordnates of e, 1, 2, 3, and O, be (e 1, e 2, e 3, 0 (e T, 0, and (O 1, O 2, O 3, 1 ( c T, 1, respectvely. Here e and c represent 3 1 matrces. The followng matrx changes {e C 1, ec 2, ec 3, C} to {e 1, e 2, e 3, O}: e T 1 0 e T 2 0 e T 3 0 c T 1. (3.2 Its transpose changes {e 1, e 2, e 3, O } to {e C 1, ec 2, ec 3, C }. Substtutng e C, 1, 2, 3 expressed by e 1, e 2, e 3, O nto (??, we get the matrx of P C : P C (e 1 e 2 e 3 c. (3.3 When C O, e C 1 e 1, e C 2 e 2 and e C 3 fe 3, where f s the focal length of the camera, 1 0 0 0 P C 0 1 0 0, (3.4 0 0 1/f 0 whch s the standard perspectve proecton matrx. Ths ustfes the representaton of the perspectve proecton by P C and the representaton of mage ponts by vectors n C R 4.
Coordnate-Free Proectve Geometry for Computer Vson 137 In the case when the camera coordnate system s proectve, let the 4 1 matrces e C 1, 2, 3 represent the coordnates of e C wth respect to {e 1, e 2, e 3, O }. By (??, Below we derve the matrx of P C restrcted to G 2 4. Let, P C (e C 1 e C 2 e C 3 T. (3.5 ê C e C 1 e C 2, (3.6 where 1 2. It represents the coordnates of ê C wth respect to the bass of G4 2 nduced by {e 1, e 2, e 3, O }. Accordng to (??, the matrx of P C s P C (ê C 1 ê C 2 ê C 3 T. (3.7 3.2. Camera constrants It s clear that as long as det(e 1 e 2 e 3 0, the matrx P C (e 1 e 2 e 3 c represents a perspectve proecton. When there s further nformaton on the pnhole camera, for example vectors e C 1, ec 2 of the camera affne coordnate system are perpendcular, then P C needs to satsfy addtonal equalty constrants n order to represent the perspectve proecton carred out by such a camera. Let 3 represent the dual n G( A 3. Let the dual bases of {e 1, e 2, e 3 } and {e C 1, ec 2, ec 3 } n A 3 be {e 3 1, e 3 2, e 3 3 } and {ec 3 1, e C 3 2, e C 3 3 } respectvely. Then e C 1 (e C 3 2 e C 3 3 3 e C 3 2 e C 3 e C 2 (e C 3 3 e C 3 1 3 e C 3 3 e C 3 3, 1, (3.8 where s the cross product n vector algebra. The perpendcularty constrant can be represented by e C 1 e C 2 (e C 3 2 e C 3 3 (e C 3 3 e C 3 1 0. (3.9 Let the 3 1 matrx e C 3 represent the coordnates of e C 3 wth respect to {e 3 1, e 3 2, e 3 3 }. Under the assumpton that {e 1, e 2, e 3 } s an orthonormal bass, e C 3 e C 3 e C 3 e C 3 for any 1, 3. Then (?? s changed to whch s a constrant on P C because (e C 3 2 e C 3 3 (e C 3 3 e C 3 1 0, (3.10 (e C 3 1 e C 3 2 e C 3 3 (e 1 e 2 e 3 T. (3.11 3.3. Camera calbraton Let M be a space pont or pont at nfnty, m C be ts mage n the retna plane. Assume that m C s a pont, and has homogeneous coordnates (u, v, 1 n the Cartesan coordnate system of the retna plane. Let the 4 1 matrx M represent the homogeneous coordnates of M n the world coordnate system. Then (u v 1 T P C M (e C 1 M e C 2 M e C 3 M T, (3.12
138 Hongbo L, Gerald Sommer whch can be wrtten as two scalar equatons: (e C 1 ue C 3 M 0, (e C 2 ve C 3 M 0. (3.13 The matrx P C (e C 1 e C 2 e C 3 T can be taen as a vector n the space R 4 R 4 R 4 equpped wth the nduced nner product from R 4. By ths nner product, (?? can be wrtten as (M 0 um T P C 0, (0 M vm T P C 0. (3.14 Gven M and (u, v for 1,..., 6, there are 12 equatons of the forms n (??. If there s no camera constrant, then snce a 3 4 matrx representng a perspectve proecton has 11 free parameters, P C can be solved from the 12 equatons f and only f the determnant of the coeffcent matrx A of these equatons s zero,. e., Λ 6 1(M 0 u M Λ 6 1(0 M v M 0, (3.15 where the outer products are n the Clfford algebra generated by R 4 R 4 R 4. Expandng the left-hand sde of (??, and changng outer products nto determnants, we get ɛ(σɛ(τu σ(1 u σ(2 v τ(1 v τ(2 det(m σ(1 M σ(2 M τ(1 M τ(2 σ,τ det(m σ( 3..6 det(m τ( 3..6 0, (3.16 where σ, τ are any permutatons of 1,..., 6 by movng two elements to the front of the sequence, and ɛ(σ, ɛ(τ are the sgns of permutaton. For expermental data, (?? s not necessarly satsfed because of errors n measurements. 4. Eppolar and trfocal geometres 4.1. Eppolar geometry There s no much dfference between our algebrac descrpton of the pnhole camera and others f there s only one fxed camera nvolved, because the underlyng Grassmann-Cayley algebras are somorphc. Let us reformulate the eppolar geometry of two cameras wth optcal centers C, C respectvely. The mage of C n camera C s E CC C C, called the eppole of C n camera C. Smlarly, the mage of C n camera C s E C C C C, called the eppole of C n camera C. An mage lne passng through the eppole n camera C (or C s called an eppolar lne wth respect to C (or C. Algebracally, an eppolar lne s a vector n C C R 4 (C G 2 4 (C G 2 4. (4.1 An eppolar lne C C M corresponds to a unque eppolar lne C C M, and vce versa. Let there be two camera proectve coordnate systems n the two cameras respectvely: {e C 1, ec 2, ec 3, C} and {ec 1, ec 2, ec 3, C }. Usng the relatons (C e C (C ê C C, for 1 3, (4.2
Coordnate-Free Proectve Geometry for Computer Vson 139 and (C ê C 1 (C ê C 2 C ê C, for 1 2, (4.3 we get the coordnates of eppole E CC : E CC ((C ê C C 1..3 ((C ê C C 1..3 ((e C 1 2 3 e C 1..3. (4.4 The followng tensor n (C R 4 (C R 4 s called the eppolar tensor decde by C, C : F CC (m C, m C m C m C. (4.5 Let m C C R 4, m C C R 4. They are mages of the same space pont or pont at nfnty f and only f F CC (m C, m C 0. Ths equalty s called the eppolar constrant between m C and m C. In matrx form, wth respect to the bases {C e C 1, C ec 2, C ec 3 } and {C e C 1, C e C 2, C e3 C }, F CC can be represented by F CC ((C e C (C e C,1..3 ((C e C (C e C,1..3 (ê C ê C,1..3. (4.6 (?? s called the fundamental matrx. The eppolar tensor nduces a lnear mappng F C;C from C R 4 to (C R 4 C G 2 4, called the eppolar transformaton from camera C to camera C : F C;C (m C C m C. (4.7 Smlarly, t nduces an eppolar transformaton from camera C to camera C as follows: F C ;C (m C C m C. (4.8 Both transformatons are ust proectve splts. The ernel of F C;C s the one-dmensonal subspace of C R 4 represented by C C, the range of F C;C s the two-dmensonal space C C R 4. In geometrc language, F C;C maps the eppole of C to zero, and maps any other pont n camera C to an eppolar lne wth respect to C. Furthermore, we have the followng concluson: Proposton 4.1. Let L C be an eppolar lne n camera C. If ts dual s mapped to eppolar lne L C n camera C by F C;C, then the dual of L C s mapped bac to L C by F C ;C. The proof follows from the dentty that for any vector M R 4, C (C (C C M C C C C M. (4.9
140 Hongbo L, Gerald Sommer 4.2. Trfocal geometry Let there be three cameras wth optcal centers C, C, C respectvely. Let M be a space pont or pont at nfnty. Its mages C M, C M and C M n the three cameras must satsfy parwse eppolar constrants. Let us consder the nverse problem: If there are three mage ponts m C, m C, m C n the three cameras respectvely, they satsfy the parwse eppolar constrants, s t true that they are mages of the same space pont or pont at nfnty? A smple counter-example shows that the eppolar constrants are not enough. When the 2-blades m C, m C, m C belong to G(C C C, the eppolar constrants are always satsfed, but the blades do not necessarly share a common vector. Assume that the eppolar constrant between m C and m C s satsfed. Let M be the ntersecton of the two lnes m C and m C n P 3. Then m C represents the mage of M n camera C f and only f m C M 0, or equvalently, When C, C, M are not collnear, snce (?? can be wrtten as m C (M x 0, for any x R 4. (4.10 M R 4 (C M R 4 (C M R 4, (4.11 m C (m C C m C 0 (m C C m C 0 0, (4.12 for any mage ponts m C 0, mc 0 n cameras C, C respectvely. When C, C, M are collnear, snce m C m C, (?? s equvalent to the eppolar constrant between m C and m C. So the constrant (?? must be satsfed for m C, m C, m C to be mages of the same space pont or pont at nfnty. Defnton 4.1. The followng tensor n (C R 4 (C G4 2 (C G4 2 s called the trfocal tensor (Hartley, 1994; Shashua, 1994 of camera C wth respect to cameras C, C : where m C C R 4, L C C G 2 4, LC C G 2 4. T (m C, L C, L C m C L C L C, (4.13 Two other trfocal tensors can be defned by nterchangng C wth C, C respectvely: T (m C, L C, L C m C L C L C, T (m C, L C, L C m C L C L C. (4.14 In ths secton we dscuss T only. Let {e C 1, ec 2, ec 3, C}, {ec 1, ec 2, ec 3, C }, {e C 1, ec 2, ec 3, C } br camera proectve coordnate systems of the three cameras respectvely. Then T has the followng component representaton: T ((C e C (C ê C ((C e C ((C ê C (ê C ( (ê C (e C (C ê C,,1..3 e C,,1..3.,,1..3 (C ê C,,1..3 The trfocal tensor T nduces three trfocal transformatons: (4.15
Coordnate-Free Proectve Geometry for Computer Vson 141 1. The mappng T C : (C G 2 4 (C G 2 4 (C R4 C G 2 4 s defned as When L C If L C T C (L C, L C C (L C L C. (4.16 s fxed, T C nduces a lnear mappng T CC L C : C G 2 4 C G2 4 : T CC L C (LC C (L C L C. (4.17 s an eppolar lne wth respect to C, the ernel of T CC L C wth respect to C, the range s the eppolar lne represented by L C ; else f L C s all eppolar lnes s an eppolar lne wth respect to C, the ernel s the eppolar lne represented by L C, the range s all eppolar lnes wth respect to C. For other cases, the ernel s zero. Geometrcally, when T C (L C, L C 0, then L C L C represents a lne or lne at nfnty L of A 3, both L C and L C are mages of L. T C (L C, L C s ust the mage of L n camera C. 2. The mappng T C : (C R 4 (C G 2 4 (C G 2 4 C R 4 s defned as T C (m C, L C C (m C L C. (4.18 When m C s fxed, T C C nduces a lnear mappng T C C m C : C G 2 4 C R 4 : T C C m C (L C C (m C L C. (4.19 If m C s the eppole of C, the ernel of T C C m C s all eppolar lnes wth respect to C, the range s the eppole of C. For other cases, the ernel s the eppolar lne C m C, the range s the two-dmensonal subspace of C R 4 represented by C m C. Geometrcally, when T C (m C, L C 0, then m C L C represents a pont or pont at nfnty M of A 3, m C s ts mage n camera C, and L C s the mage of a space lne or lne at nfnty passng through M. T C (m C, L C s ust the mage of M n camera C. 3. The mappng T C : (C R 4 (C G 2 4 (C G 2 4 C R 4 s defned as T C (m C, L C C (m C L C. (4.20 We prove below two propostons n (Faugeras and Papadopoulo, 1997, 1998 usng the above reformulaton of trfocal tensors. Proposton 4.2. Let L C be an eppolar lne n camera C wth respect to C and L C be the correspondng eppolar lne n camera C. Then for any lne L C n camera C whch s not the eppolar lne wth respect to C, T C (L C, L C L C. Proof. The hypotheses are L C L C, C L C 0. Usng the formula that for any C R 4, A 3, B 3 G 3 4, C (A 3 B 3 C B 3 A 3 C A 3 B 3, (4.21 we get T C (L C, L C C (L C L C C L C L C C L C L C L C.
142 Hongbo L, Gerald Sommer Proposton 4.3. Let m C, m C be mages of the pont or pont at nfnty M n cameras C, C respectvely. Let L C be an mage lne passng through m C but not through E C C. Let L C be an mage lne passng through m C but not through E C C. Then the ntersecton of T C (L C, L C wth the eppolar lne C m C s the mage of M n camera C. Proof. The hypotheses are M L C M L C 0, C L C 0, C L C 0. So T C (L C, L C (C m C (C (L C L C (C C M (C C L C L C C M (C M L C L C C C C L C C L C C M C M. 5. Relatons among eppoles, eppolar tensors and trfocal tensors of three cameras Consder the followng 9 vectors of R 4 : ES {e C, e C, e C 1,, 3}. (5.1 Accordng to (??, (?? and (??, by nterchangng among C, C, C any of the eppoles, eppolar tensors and trfocal tensors of the three cameras has ts components represented as a determnant of 4 vectors n ES. For example, E CC (e C F CC (ê C T (ê C 1 2 3 ; ê C ;. (5.2 Conversely, any determnant of 4 vectors n ES equals a component of one of the eppoles, eppolar tensors and trfocal tensors up to an ndex-free scale. Snce the only constrant on the 9 vectors s that they are all n R 4, theoretcally all relatons among the eppoles, eppolar tensors and trfocal tensors can be establshed by manpulatng n the algebra of determnants of vectors n ES usng the followng Cramer s rule (Faugeras and Mourran, 1995a, b: (x 2 x 3 x 4 x 5 x 1 (x 1 x 3 x 4 x 5 x 2 (x 1 x 2 x 4 x 5 x 3 +(x 1 x 2 x 3 x 5 x 4 (x 1 x 2 x 3 x 4 x 5, (5.3 where the x s are vectors n R 4. In practce, however, we can only select a few expressons from the algebra of determnants and mae manpulatons, and t s dffcult to mae the selecton. In ths secton we propose a dfferent approach. Instead of consderng the algebra of determnants drectly, we consder the set of meets of dfferent blades, each blade beng an outer product of vectors n ES. Snce the meet operator s assocatve and ant-commutatve n the sense that A r B s ( 1 rs B s A r, (5.4
Coordnate-Free Proectve Geometry for Computer Vson 143 for A r G4 r and B s G4 s, for the same expresson of meets we can have a varety of expansons. Then we can obtan varous equaltes on determnants of vectors n ES, whch may be changed nto equaltes, or equaltes up to an ndex-free constant, on components of the eppoles, eppolar tensors and trfocal tensors. It appears that we need only 7 expressons of meets to derve and further generalze all the nown constrants on eppolar and trfocal tensors. It should be remnded that n ths chapter we always use the notaton 1 2 to denote that, 1, 2 s an even permutaton of 1, 2, 3. 5.1. Relatons on eppolar tensors Consder the followng expresson: F exp (e C 1 2 3 (e C 1 e C 2 e C 3 (e C 1 2 3. (5.5 It s the dual of the blade C C C. Expandng F exp from left to rght, we get F exp,1,1 (e C 1 2 3 e C E CC F CC e C. Expandng F exp from rght to left, we get F exp 3 ( (e C 1 2 3 e C 1 (e C 1 2 3 e C 3 (E C C 1 E C C E C C where. So for any 1 3, where K C CC 1 E C C E C C E C C E C C. E C C e C, ê C ê C e C (e C 1 e C 2 e C 3 ec (e C 1 e C 2 e C 3 ec e C E CC F CC K C CC, (5.6 (?? s a fundamental relaton on the eppolar tensor F CC and the eppoles. In matrx form, t can be wrtten as n Grassmann-Cayley algebra, t can be wrtten as (F CC T E CC E C C E C C ; (5.7 C (C C (C C C (C C. (5.8 Geometrcally, t means that the eppolar lne n camera C wth respect to both C and C s the mage lne connectng the two eppoles E C C and E C C. Snce E C C E C C s orthogonal to E C C, an mmedate corollary s (E CC T F CC E C C 0, (5.9 whch s equvalent to (C C (C C 0. Geometrcally, t means that the two eppoles E C C and E C C satsfy the eppolar constrant.
144 Hongbo L, Gerald Sommer 5.2. Relatons on trfocal tensors I The frst dea to derve relatons on trfocal tensors s very smple: f the tensor (T,,1..3 s gven, then expandng (ê C (e C 1 2 3 (5.10 gves a 2-vector of the e C s whose coeffcents are nown. Smlarly, expandng T exp 1 (ê C (ê C (e C 1 2 3. (5.11 from rght to left gves a vector of the e C s whose coeffcents are nown. Expandng T exp 1 from left to rght, we get a vector of the e C s whose coeffcents depend on eppolar tensors. By comparng the coeffcents of the e C s we get a relaton on T and eppolar tensors. Assume that. Expandng T exp 1 from left to rght, we get T exp 1 (ê C 3 1 F CC Expandng T exp 1 from rght to left, we get T exp 1 3 where. So where 1 ( (ê C (ê C 3 F CC e C. e C e C (ê C 1 (ê C (ê C 3 (T 1 T 2 T 1 T 2 e C 1 Proposton 5.1. For any 1,, 3, Corollary 5.2. Let 1,,,, 3, then ê C e C e C e C e C, F CC F CC t C, (5.12 t C T 1 T 2 T 1 T 2. (5.13 F CC F CC F CC F CC F CC F CC t C. (5.14 tc t C, for any 1 3; (5.15 tc t C, for any 1 3; (5.16 t C t C tc t C. (5.17
Coordnate-Free Proectve Geometry for Computer Vson 145 Notce that (?? s a constrant of degree 4 on T. Below we explan relaton (?? n terms of Grassmann-Cayley algebra. When C e C fxed, T nduces a lnear mappng T C C : C G4 2 C R 4 by T C C (L C C ((C e C L C. (5.18 The matrx of T C C s ( T T,1..3. Defne a lnear mappng t C C : C R 4 C G4 2 as follows: let mc C R 4 and m C L C 1 LC 2, where LC 1, LC 2 C G4 2, then t C C (m C T C C (L C 1 C T C C (L C 2. (5.19 We need to prove that ths mappng s well-defned. Usng the formula that for any 2-blade C 2 G 2 4 and 3-blades A 3, B 3 G 3 4, we get So t C C (C 2 A 3 (C 2 B 3 A 3 B 3 C 2 C 2, (5.20 t C C (m C C ((C e C LC 1 ((C ec LC 2 L C 1 LC 2 (C ec C C e C m C (C e C C C e C 3 m C (C e C (C C e C 1 e C C ê C. s well-defned. Let and, then snce t C C (C e C T C C (C e C e C C T C C ( ( (C e C e C T 2 C e C C T 1 C e C 1 1 (T 1 T 2 T 1 T 2 C ê C 1, s (5.21 the matrx of t C C s (t T,1..3. So (?? s equvalent to (??. Geometrcally, T C C maps an mage lne n camera C to an mage pont on the eppolar lne C C e C n camera C ; t C C maps the ntersecton of two mage lnes L C 1, LC 2 n camera C to the mage lne n camera C passng through the two mage ponts T C C (L C 1 and T C C (L C 2, whch s ust the eppolar lne C C e C. Ths fact s stated as (??. 5.3. Relatons on trfocal tensors II Now we let the two ê C s n T exp 1 be dfferent, and let the two e C be the same,. e., we consder the expresson T exp 2 (e C 1 e C (e C 2 e C (e C 1 2 3. (5.22 Assume that 1 2. Expandng T exp 2 from left to rght, we get T exp 2 (e C 3 1 e C 1 e C 2 T ec. ( (e C 1 ê C e C
146 Hongbo L, Gerald Sommer Expandng T exp 2 from rght to left, we get T exp 2 3 where. So where 1 ( (ê C 2 (ê C 2 e C e C (ê C 1 (ê C 1 3 (T 1 T 2 T 1 T 2 e C 1 Proposton 5.3. For any 1,, 3, Corollary 5.4. For any 1, 1, 2,,,, 3, e C e C e C, T t C, (5.23 t C T 1 T 2 T 1 T 2. (5.24 T t C. (5.25 T 1 T 2 tc 1 t C 2 ; T T tc t C. (5.26 Now we explan relaton (?? n terms of Grassmann-Cayley algebra. When C ê C fxed, T nduces a lnear mappng T CC : C G4 2 C G2 4 by s T CC (L C C ((C ê C L C, (5.27 whose matrx s ( T,1..3. T also nduces a lnear mappng T CC : C R 4 C R 4 by T CC (m C C (m C (C ê C, (5.28 whose matrx s ( T T,1..3. Defne a lnear mappng tcc : C R 4 C R 4 as follows: let m C C R 4 and m C L C 1 L C 2, where LC 1, LC 2 C G4 2, then t CC (m C T CC (L C 1 T CC (L C 2. (5.29 We need to prove that ths mappng s well-defned. Usng the formula that for any C R 4 and A 3, B 3 G 3 4, (C (A 3 B 3 B 3 B 3 C A 3 B 3, (5.30 we get t CC (m C ( ( C ((C ê C LC 1 ( ( C C (C ê C LC 1 (C ê C C C ((C ê C (C ê C C C ((C ê C T CC (m C. C ((C ê C LC 2 (C ê C LC 2 LC 1 L C 2 mc (5.31
Coordnate-Free Proectve Geometry for Computer Vson 147 So t CC Then (?? s equvalent to (??. s well-defned. Usng (??, t can be verfed that the matrx of t CC s (t C,1..3. Geometrcally, T CC maps an mage lne L C n camera C to the mage of the ntersecton of the two planes C ê C and L C n camera C; t CC maps the ntersecton of two mage lnes L C 1, LC 2 n camera C to the mage of the ntersecton of the two coplanar lnes (C ê C LC 1 and (C ê C LC 2 n camera C, whch s ust the mage of the ntersecton of the plane C ê C wth the lne L C 1 L C 2. Ths fact s stated as (??. 5.4. Relatons on trfocal tensors III Consder the followng expresson obtaned by changng one of the e C s n T exp 1 to an e C : T exp 3 (ê C (ê C Expandng T exp 3 from left to rght, we get T exp 3 (ê C 3 l1 e C T Fl CC e C l. Expandng T exp 3 from rght to left, we get T exp 3 (ê C where. (T 1 F CC ( (ê C (ê C Proposton 5.5. For any 1, 3, + T 2 F CC 2 e C 1 (e C 1 2 3. (5.32 ( (ê C l1 ê C l e C e C T F CC 1 e C e C l T F CC 2 e C, T F CC 0. (5.33 By (??, F CC F1 CC t C /tc 1. So for any 1, 3, T t C det(t,1..3 0. (5.34 1 (?? can also be obtaned drectly by expandng the followng expresson: (ê C 1 (ê C 2 (ê C 3 (e C 1 2 3. (5.35 Expandng from left to rght, (?? gves zero; expandng from rght to left, t gves det(t,1..3. To understand (?? geometrcally, we chec the dual form of (??, whch s C ((C e C (C ê C 1 ((C e C (C ê C 2 ((C e C (C ê C 3. (5.36
148 Hongbo L, Gerald Sommer (?? equals zero because the three ponts (C e C (C ê C, 1, 2, 3 are on the same lne C e C. Interchangng C wth C, we get det(t,1..3 0 for any 1 3. By (??, we have det(t C,1..3 0. (5.37 A smlar constrant can be obtaned by nterchangng C and C. (?? can be generalzed to the followng one: ( ( 3 1 ( λ ê C 1 ( 3 λ ê C 2 ( 1 ( 3 λ ê C 3 1 (e C 1 2 3. (5.38 where the λ s are ndetermnants. (?? equals zero when expanded from the left, and equals a polynomal of the λ s when expanded from the rght. The coeffcents of the polynomal are expressons of the T s. Thus we get 10 constrants of degree 3 on T, called the ran constrants by Faugeras and Papadopoulo (1997, 1998. 5.5. Relatons on trfocal tensors IV Now we let the two ê C s n T exp 3 be dfferent. Consder T exp 4 (e C e C 1 (e C e C 2 (e C 1 2 3. (5.39 Assume that 1 2. Expandng T exp 4 from left to rght, we get T exp 4 (e C e C 1 (e C ( E CC e C 2 e C 1 + T 2 F CC 1 (e C 1 2 3 e ( C + T 2 F CC 1 ec e C Expandng T exp 4 from rght to left, we get T exp 4 (e C e C 1 (e C + (e C ( (e C + (e C e C 1 e C 1 T 2 F CC 1 e C (T 2 F CC 1 + T 2 F CC 1 (e C l1 e C. ê C 1 ê C (e C (e C e C 1 (e C + T 2 F CC 1 e C + T 2 F CC 1 ê C 1 ê C 1 e C e C 2 ê C l e C ê C ê C e C e C l ê C 1 ê C e C e C, where. So 1 T 2 F CC 1 E CC. (5.40
Coordnate-Free Proectve Geometry for Computer Vson 149 Interchangng 1, 2 n T exp 4, we obtan 1 Proposton 5.6. For any 1, 3, T 1 F CC 2 E CC. (5.41 where W 3 1 T 1 F CC 2. From (??, (?? and (?? we get Proposton 5.7. For any 1 1, 2, 3, (T 1 F CC 2 1 (?? can also be proved by drect computaton: W 3 (C e C 1 (C ê C 1 ( ( C 1 E CC W, (5.42 + T 2 F CC 1 0. (5.43 (C ê C (C e C 2 (C e C (C C e C 2 (C e C 1 (C ê C (C ec 2 (C e C (C e C 1 (C ê C êc (C C e C 1 e C 2 (C C ê C E CC. (?? s equvalent to the ant-symmetry of C C e C 1 e C 2 wth respect to e C 1 and e C 2. Defne 1 2 t C 1 T 2 (5.44 1 for 1 1, 2,, 3. By (??, (??, (?? and (??, 1 2 3 F CC 1 F CC 1 T 2 1 F CC 1 F CC 1 0, f 1 2 ; E CC, f 1 2 ; E CC, f 2 1. (5.45 Two corollares can be drawn mmedately: Corollary 5.8. 1. For any 1 l, l 3, where 1 l 4, 1 2 1 2 3 uc C 3 4 4 3 4 4 3 EC C 3. (5.46
150 Hongbo L, Gerald Sommer 2. Let 1 2. Then for any 1 l 3 where 1 l 4, 1 2 1 uc C 1 2 3 4 1 3 4 Corollary 5.9. 1. For any 1 1, 2, 3 where 1 2,, 2. For any 1, 3, ECC. (5.47 E CC 2 E C C ( 1 2 2 1..3. (5.48 E CC ( 23 32, 23 31, 21 32 T. (5.49 Now we explan (?? n terms of Grassmann-Cayley algebra. We have defned two mappngs T C C and t C C n (?? and (??, whose matrces are ( T,1..3 and (t C,1..3 respectvely. By the defnton of 1 2, 1 2 C t C C 1 (C e C T C C 2 (C ê C. (5.50 Expandng the rght-hand sde of (??, we get 1 2 U C C (C e C 1, C e C 2, C e C, C ê C, (5.51 where U C C : (C R 4 (C R 4 (C R 4 (C G4 2 R s defned by U C C (m C 1, m C 2, m C, L C m C 1 m C C L C C (m C 1 C m C 2. (5.52 (?? s ust (??. It means that the u C C s are components of the mappng U C C. Notce that (?? s a group of degree 6 constrants on T. It s closely related to Faugeras and Mourran s frst group of degree 6 constrants: T 1. T 1 l 2. T l1 l 2. T 1. T l1. T l1 l 2. T l1. T 1 l 2. T l1 l 2. T 1. T l1. T 1 l 2., (5.53 where T 1. (T 1 1..3. It s dffcult to fnd the symmetry of the ndces n (??, so we frst express (?? n terms of Grassmann-Cayley algebra. Usng the fact that T 1. s the coordnates of C ((C e C (C ê C, we get T 1. T 1 l 2. T ( ( l1 l 2. C (C e C (C ê C ( C (C e C (C ê C ( ( C C (C e C (C ê C l 2 ( ( C ( C (C e C l 1 (C ê C l 2 (C e C (C ê C l 2 ( (C e C l 1 (C ê C l 2.
Coordnate-Free Proectve Geometry for Computer Vson 151 By formula (??, T 1. T 1 l 2. T l1 l 2. (C e C (C ê C l 2 (C ê C (C C e C (C e C l 1 (C ê C l 2 (C e C (C ê C (C ê C l 2 C (C e C e C l 1 C (C ê C l 2. (5.54 Let Defne a mappng V C : (C R 4 (C R 4 (C G4 2 (C G4 2 R as follows: By (??, V C (m C 1, m C 2, L C 1, L C 2 C (m C 1 C m C 2 C L C 2 m C 1 L C 1 L C 2. (5.55 Smlarly, we can get (?? s equvalent to By (??, we have v C 1 2 v C l 1 l 2 V C (C ê C, C ê C l 1, C ê C, C ê C l 2. (5.56 T 1. T 1 l 2. T l1 l 2. v C l 1 l 2. (5.57 T 1. T l1. T l1 l 2. vl C 1 l 2, T l1. T 1 l 2. T l1 l 2. vl C 1 l 2, T 1. T l1. T 1 l 2. v C 1 l 1 l 2. v C l 1 l 2 v C l 1 l 2 (5.58 vc l 1 l 2. (5.59 vl C 1 l 2 0, f 1 2 or ; F CC 1 F CC 1 E CC, f 1 2 and, or 2 1 and ; E CC, f 1 2 and, or 2 1 and. Corollary 5.10. 1. For any 1 l,, 3 where 1 l 4, v C 1 2 v C 1 2 2. Let 1 2. Then for any 1 l 3 where 1 l 4, v C 1 2 v C 1 (5.60 vc 3 4. (5.61 v C 3 4 vc 1 2 3 4 v C 1 3 4. (5.62
152 Hongbo L, Gerald Sommer (?? s a specal case of (??: v C 1 2 v C 1 2 or more explctly, the followng dentty: vc 2 1, (5.63 v C 2 1 C (m C 1 C m C 2 C LC 2 m C 1 LC 1 LC 2 C (m C 1 C m C 2 C LC 1 m C 1 LC 2 LC 1 C (m C 2 C m C 1 C LC 2 m C 2 LC C (m C 2 C m C 1 C LC 1 m C 2 LC 1 LC 2 2 LC 1, (5.64 for any m C 1, mc 2 C R4, L C 1, LC 2 C G 2 4. Comparng U C C wth V C, we get V C (m C 1, m C 2, L C 1, L C 2 U C C (m C 1, m C 2, L C 1 L C 2, L C 2. (5.65 We have already generalzed Faugeras and Mourran s frst group of constrants by V C. It appears that we have generalzed the constrants furthermore by U C C, because (?? s equvalent to (??, whle (?? s a specal case of (?? where 4. The varables n V C are less separated than those n U C C, so there are less constrants on T that come from V C. Interchangng C and C n (??, we get Faugeras and Mourran s second group of degree 6 constrants: T 1. T 1.l 2 T l1.l 2 T 1. T l1. T l1.l 2 (5.66 T l1. T 1.l 2 T l1.l 2 T 1. T l1. T 1.l 2, where T 1. (T 1 1..3. 5.6. Relatons on trfocal tensors V Consder the followng expresson: T exp 5 (e C e C 1 (e C 2 (e C 1 2 3. (5.67 Assume that 1 2. Expandng T exp 5 from left to rght, we get T exp 5 e C Expandng T exp 4 from rght to left, we get T 2 l1 T l 2 e C l. T exp 5 T 2 T 2 e C T 2 T 2 e C + (T 2 T 2 + T 2 T 2 e C, where. So 1 T 2 T 2. (5.68
Coordnate-Free Proectve Geometry for Computer Vson 153 Proposton 5.11. For any 1, 3, 1 Usng the relaton (??, we get 1 T T. (5.69 t C T det(t,1..3 ( 2. (5.70 Corollary 5.12. For any 1 1, 2, 3 where 1 2, ( det(t,1..3 ( 1 2 1..3. (5.71 5.7. Relatons on trfocal tensors VI The second dea of dervng relatons on trfocal tensors s as follows: (T,,1..3 s gven, then expandng (e C f the tensor (e C 1 e C 2 e C 3 (5.72 gves a vector of the e C s whose coeffcents are nown. Smlarly, expandng (e C (e C (e C 1 e C 2 e C 3, (e C 1 e C 2 e C 3 gves two 2-vectors of the e C e C s and the e C are nown. The meet of two such 2-vectors,. e., T exp 6 ( (e C (5.73 e C s respectvely, whose coeffcents (e C 1 e C 2 e C 3 ( (e C (e C 1 e C 2 e C 3 (5.74 s an expresson of the T s. Expandng the meets dfferently, we get a relaton on T, eppoles and eppolar tensors. Assume that and. Expandng T exp 6 accordng to ts parentheses, we get ( T exp 6 (e C ê C 1 e C 1 1 1( (e C ê C 2 e C 2 2 1 3 ( T 1 T 2 + T 2 T 1 T 1, 1 where 1 2. Usng the fact that the meet of a 4-vector wth any multvector n G 4 s a scalar multplcaton of the multvector by the dual of the 4-vector, we get T exp 6 (e C F C C (e C E C C. (e C 1 e2 C e C 3 (e C 1 e C 2 e C 3
154 Hongbo L, Gerald Sommer So F C C E C C Interchangng, n T exp 5, we get (T 1 T 2 T 2 T 1 T 1. (5.75 1 F C C E C C Interchangng, n T exp 5, we get F C C E C C Interchangng (, and (, n T exp 5, we get Then F C C E C C (T 1 T 2 T 2 T 1 T 2. (5.76 1 (T 1 T 2 T 2 T 1 T 1. (5.77 1 (T 1 T 2 T 2 T 1 T 2. (5.78 1 When or, T exp 5 0 by expandng from left to rght. Defne v C 1 (T 1 T 2 T 2 T 1 T 1. (5.79 v C 1 0, f or ; F C C F C C Proposton 5.13. For any 1,,, 3, E C C, f and, or and ; E C C, f and, or and. (5.80 v C v C EC C ; v C v C EC C. (5.81 E C C Corollary 5.14. 1. For any 1 l, l 3, where 1 l 4, v C v C vc 3 4 v C 3 4 ; v C v C vc 3 4 v C 3 4. (5.82 2. For any 1 l,, l, l 3 where 1 l 2, v C v C uc C 1 2 1 2. (5.83
Coordnate-Free Proectve Geometry for Computer Vson 155 Notce that (?? and (?? are groups of degree 6 constrants on T. (?? s closely related to Faugeras and Mourran s thrd group of degree 6 constrants: T.1 T.1 l 2 T.l1 l 2 T.1 T.l1 T.l1 l 2 T.l1 T.1 l 2 T.l1 l 2 T.1 T.l1 T.1 l 2, (5.84 where T.1 (T 1 1..3. Let us express (?? n terms of Grassmann-Cayley algebra. Usng the fact that T.1 s the coordnates of C ((C ê C (C ê C, we get T.1 T.1 l 2 T.l1 l 2 C ( ( ( ( C (C ê C (C ê C C C (C ê C (C ê C l 2 ( C (C (C ê C l 1 (C ê C l 2 ( ( ( ( C C (C ê C (C ê C C (C ê C (C ê C l 2 (C ê C l 1 (C êl C 2. By (??, we have T.1 T.1 l 2 T.l1 l 2 ( C ( (C ê C (C ê C (C ê C (C ê C l 2 (C ê C l 1 (C ê C (C ê C C (C ê C l 2 C (C ê C (C ê C l 1 (C ê C (C ê C l 2 C l 2. (5.85 Defne a mappng V C : (C G4 2 (C G4 2 (C G4 2 (C G4 2 R as follows: V C (L C 1, L C 2, L C 1, L C 2 L C 1 C L C 2 C L C 1 L C 2 L C 1 L C 2. (5.86 Accordng to (??, Accordng to (??, Smlarly, we have Now (?? s equvalent to V C (C ê C, C ê C l 1, C ê C, C ê C l 2 v C l 1 l 2, (5.87 T.1 T.1 l 2 T.l1 l 2 v C l 1 l 2. (5.88 T.1 T.l1 T.l1 l 2 v C l 1 l 2, T.l1 T.1 l 2 T.l1 l 2 v C l 1 l 2, T.1 T.l1 T.1 l 2 v C l 1 l 2. (5.89 v C l 1 l 2 v C l 1 l 2 vc l 1 l 2 v C l 1 l 2, (5.90
156 Hongbo L, Gerald Sommer or more explctly, the followng dentty: L C 1 C LC 2 C L C 1 LC 2 LC 1 L C 2 L C 1 C LC 1 C L C 1 LC 2 LC 2 L C 1 (?? s a straghtforward generalzaton of t. LC 2 C LC 2 C L C 2 LC 1 LC 1 L C 2 L C 2 C LC 1 C L C 2 LC 1 LC 2 L C 1. (5.91 5.8. A unfed treatment of degree 6 constrants In ths secton we mae a comprehensve nvestgaton of Faugeras and Mourran s three groups of degree 6 constrants. We have defned 1 2 n (?? to derve and generalze the frst group of constrants. We are gong to follow the same lne to derve and generalze the other two groups of constrants. The trfocal tensor T nduces 6 nds of lnear mappngs: Mappng Defnton Matrx T C C C G4 2 C R 4 ( T,1..3 L C C ((C e C LC T C C C G4 2 C R 4 ( T T,1..3 L C C ((C e C LC T CC C G4 2 C G2 4 ( T,1..3 L C C ((C ê C LC T C C C R 4 C R 4 ( T T,1..3 m C C (m C (C ê C T CC C G4 2 C G2 4 ( T,1..3 L C C (L C (C ê C T C C C R 4 C R 4 ( T T,1..3 m C C (m C (C ê C We have defned two lnear mappngs t C C and t CC n (?? and (?? respectvely, whch are generated by the T s. There are 6 such lnear mappngs. Let m C L C 1 L C 2, m C L C 1 L C 2, L C m C 1 C m C 2.
Coordnate-Free Proectve Geometry for Computer Vson 157 Mappng Defnton Matrx t C C C R 4 C G4 2 (t C,1..3 m C T C C (L C 1 C T C C(L 2 t C C C R 4 C G4 2 (t C T,1..3 m C T C C (L C 1 C T C C (L C 2 t CC C R 4 C R 4 (t C,1..3 m C T CC (L C 1 C T CC (L C 2 t C C C G4 2 C G4 2 (t C T,1..3 L C T C C (m C 1 C T C C (m C 2 t CC C R 4 C R 4 (t C,1..3 m C T CC (L C 1 C T CC (L C 2 t C C C G4 2 C G4 2 (t C T,1..3 L C T C C (m C 1 C T C C (m C 2 Here where 1 2 and. t C respectvely. The mappngs t s are well-defned because t C T 1 T 2 T 1 T 2, (5.92 and tc have been defned n (?? and (?? t C C (m C (C e C mc C C e C, t C C (m C (C e C mc C C e C, t CC (m C (C ê C C C ((C ê C mc, t C C (L C (C ê C C C (L C (C ê C, t CC (m C (C ê C (C ê C, t C C (L C (C ê C C C (L C (C ê C. For any 1 1, 2,,,, 3, let 1 2 3 1 1 2 3 1 u C C 1 2 3 1 u C C 1 2 3 1 u CC 3 1 u CC 3 1 t C 1 T 2, t C 1 T 2, t C 1 T 2, t C 1 T 2, t C T 2, t C T 2. (5.93 (5.94
158 Hongbo L, Gerald Sommer Then 1 2 C t C C 1 (C e C T C C 2 (C ê C, 1 2 C t C C (C ê C 1 T C C (C e C 2, u C C 1 2 C t C C(C e C T C C(C ê C, 1 u C C 1 2 C t C C (C ê C 1 T C C (C e C 2, u CC C t CC (C e C T CC (C ê C, u CC C t CC (C e C T CC (C ê C. Expandng the rght-hand sde of the equaltes, we can get a factored form of the u s. 2 (5.95 Constrants from 1 2 : (see also subsecton?? 1 2 F CC 1 E CC F CC 1 0, f 1 2 ;, f 1 2 ; E CC, f 2 1. (5.96 Two constrants can be obtaned from 1 2 : 1. For any 1 l, l 3, where 1 l 4, 1 2 1 2 3 2. Let 1 2. Then for any 1 l 3 where 1 l 4, 1 2 1 uc C 3 4 4. (5.97 3 4 4 3 uc C 1 2 3 4. (5.98 1 3 4 Defne U C C : (C R 4 (C R 4 (C R 4 (C G4 2 R by U C C (m C 1, m C 2, m C, L C m C 1 m C C L C C (m C 1 C m C 2. (5.99 Then U C C (C e C 1, C e C 2, C e C, C ê C 1 2. (5.100 Constrants from 1 2 : If 1 2, then 1 2 E CC 1 F CC E CC 1 0, f ; 2, f ; F CC 2, f. (5.101 Two constrants can be obtaned from 1 2 :
Coordnate-Free Proectve Geometry for Computer Vson 159 1. Let 1 2, 3 4. Then for any 1, 3, 1 2 1 2 2. Let 1 2. Then for any 1 l 3 where 1 l 4, 1 2 2 uc C 3 4. (5.102 3 4 uc C 1 2 3 4. (5.103 2 3 4 Defne U C C : (C G4 2 (C R4 (C G4 2 (C G4 2 R by When 1 2, U C C (L C, m C, L C 1, L C 2 L C C L C 1 C m C L C 1 L C 2. (5.104 U C C (C ê C 1, C e C 2, C ê C, C ê C 1 2. (5.105 Constrants from u C C 1 2 : u C C 1 2 E C C E C C E CC F CC E CC 0, f 1 2 ; 1, f 1 2 ; F CC 1, f 2 1. (5.106 Two constrants can be obtaned from u C C 1 2 : 1. For any 1 l, l 3 where 1 l 4, u C C 1 2 u C C 1 2 3 2. Let 1 2. Then for any 1 l 3 where 1 l 4, u C C 1 2 u C C 1 uc C 3 4 4 u C C 3 4 4 3. (5.107 uc C 1 2 3 4 u C C 1 3 4. (5.108 Defne U C C : (C R 4 (C R 4 (C R 4 (C G4 2 R by Then U C C (m C 1, m C 2, m C, L C m C 1 m C C L C C (m C 1 C m C 2. (5.109 U C C (C e C 1, C e C 2, C e C, C ê C u C C 1 2. (5.110 Constrants from u C C 1 2 : If 1 2, then u C C 1 2 E C C E C C E CC 1 F CC E CC 1 Two constrants can be obtaned from u C C 1 2 : 0, f ; 2, f ; F CC 2, f. (5.111
160 Hongbo L, Gerald Sommer 1. Let 1 2 and 3 4. Then for any 1, 3, u C C 1 2 u C C 1 2 2. Let 1 2. Then for any 1 l 3 where 1 l 4, u C C 1 2 u C C 2 uc C 3 4. (5.112 u C C 3 4 uc C 1 2 3 4. (5.113 u C C 2 3 4 Defne U C C : (C G4 2 (C R4 (C G4 2 (C G4 2 R by U C C (L C, m C, L C 1, L C 2 L C C L C 1 C m C L C 1 L C 2. (5.114 When 1 2, U C C (C ê C 1, C e C 2, C ê C, C ê C u C C 1 2. (5.115 Constrants from u CC : If, then u CC E C C F C C 0, f ;, f ; E C C F C C, f. (5.116 Two constrants can be obtaned from u CC : 1. Let and 3 4. Then for any 1, 3, u CC u CC 2. Let, then for any 1 l 3 where 1 l 4, u CC u CC ucc 3 4. (5.117 u CC 3 4 ucc 3 4 u CC 3 4. (5.118 Defne U CC : (C G4 2 (C G4 2 (C R 4 (C G4 2 R by U CC (L C 1, L C 2, m C, L C L C 1 C L C C L C 1 L C 2 m C. (5.119 When, U CC (C ê C, C ê C, C e C, C ê C u CC. (5.120 Constrants from u CC : If, then u CC E C C F C C 0, f ;, f ; E C C F C C, f. (5.121 Two constrants can be obtaned from u CC :
Coordnate-Free Proectve Geometry for Computer Vson 161 Thus 1. Let and 3 4. Then for any 1, 3, u CC u CC 2. Let, then for any 1 l 3 where 1 l 4, u CC u CC ucc 3 4. (5.122 u CC 3 4 ucc 3 4. (5.123 u CC 3 4 Defne U CC : (C R 4 (C G4 2 (C G4 2 (C G4 2 R by U CC (m C, L C, L C 1, L C 2 L C 1 C L C C m C L C 1 L C 2. (5.124 When, We have U CC (C e C, C ê C, C ê C, C ê C u CC. (5.125 V { C (m C 1, mc 2, LC 1, LC 2 U C C (m C 1, mc 2, LC 1 LC 2, LC 2 U C C (m C 1 C m C 2, mc 1, LC 2, LC v C 1 2, f LC 1 LC 2 0; (5.126 1, f mc 1 mc 2 0. 1 2, f ; 1 2, f ; 1, f 1 2 ; 1, f 2 1. (5.127 The constrants (??, (?? from V C are equvalent to the constrants (??, (?? from U C C, and are ncluded n the constrants (??, (?? from U C C. Faugeras and Mourran s frst group of constrants s a specal case of any of (??, (?? and (??. Smlarly, Faugeras and Mourran s second group of constrants s a specal case of any of (??, (??. We also have Thus { V C (L C 1, LC 2, LC 1, LC 2 U CC (L C 1, LC 2, LC 1 L C U CC (L C 1 C L C 2, LC 1, LC v C 1 2, LC 2 2, LC 1, f LC 1 L C, f LC 1 LC u CC, f ; u CC, f ; u CC, f ; u CC, f. 2 0; 2 0. (5.128 (5.129 The constrants (??, (?? from V C are equvalent to the constrants (??, (?? from U C C, and are also equvalent to the constrants (??, (?? from U C C. Faugeras and Mourran s thrd group of constrants s a specal case of any of (??, (?? and (??.
162 Hongbo L, Gerald Sommer 6. Determnng dscrete motons from lne correspondences n three cameras Let L C, L C, L C be three mage lnes n cameras C, C, C respectvely. They are mages of the same lne or lne at nfnty of A 3 f and only f L C L C L C 0 (6.1 and at least one of L C L C, L C L C, L C L C s nonzero. For example, when L C L C 0, (?? s equvalent to T C (L C, L C L C. In ths secton we wor n the framewor of affne geometry. Assume that L C, L C, L C are mages of the same space lne or lne at nfnty. Then (?? s equvalent to (L C L C L C 0, (L C L C L C 0. (6.2 Assume that C, C, C are affne ponts. Let c C C, c C C. Denote the meet n G( A 3 by 3. From (?? we get (L C 3 (L C 3 (L C 0, c 3 (L C (L C 3 (L C c 3 (L C (L C 3 (L C. (6.3 The frst equaton mples that the three space lnes or lnes at nfnty (L C, (L C, (L C are concurrent. So the second equaton s equvalent to a scalar one. Notce that at least one of (L C 3 (L C, (L C 3 (L C, (L C 3 (L C s nonzero, otherwse we wll have L C L C L C. Let {e C 1, ec 2, ec 3, C}, {ec 1, ec 2, ec 3, C } and {e C 1, ec 2, ec 3, C } be camera affne coordnate systems of the three cameras respectvely. Let the 3 1 matrces c, c represent the coordnates of c, c wth respect to {e C 1, ec 2, ec 3 }. Let the 3 1 matrces lc, l C and l C represent respectvely the coordnates of L C, L C and L C n ther respectve camera coordnate systems. Denote the matrces changng {e C 1, ec 2, ec 3 } and {ec 1, ec 2, ec 3 } respectvely to {e C 1, ec 2, ec 3 } by R, R. Then (?? can be wrtten as l C R l C R l C 0, (6.4 (c R l C l C R l C (c R l C l C R l C. (6.5 Consder the problem of determnng R, c, R, c by N trplets of data (l C, lc, l C, 1,..., N. Accordng to (?? and (??, R, R can be chosen to be specal lnear transformatons, and c, c can be decded up to a common scale. There are 21 scalar varables for 2N equatons, so N 11 s requred. Snce at least one of the blades l C R l C, l C R l C, l C R l C s nonzero, we can use a nonzero blade B 2 to mae nner products wth both sdes of (??, and change ths equaton nto a scalar one. Denote the nner products of B 2 wth l C R l C, l C R l C by [l C R l C ], [l C R l C ] respectvely. Then (?? can be wrtten as a lnear and homogeneous equaton n (c c of R 3 R 3 : (c c ([l C R l C ]R l C [l C R l C ]R l C 0. (6.6
Coordnate-Free Proectve Geometry for Computer Vson 163 So c, c can be computed from 5 equatons of the form (??, gven R, R and 5 trplets (l C, lc, l C of generc values. On the other hand, solvng R, R from N equatons of the form (?? and N 5 equatons by elmnatng c, c from N equatons of the form (?? s dffcult, because these equatons are all nonlnear ones. We consder a smple case where R, R are assumed to be rotatonal matrces. There are 11 varables, so N 6 s necessary. Assume N 6. Then there are 6 equatons of type (??, and the determnant of the coeffcent matrx must be zero,. e., σ σ 0, ɛ(σ det([l C σ( R l C σ( ]R l C σ( 1,2,3 det([l C σ( R l C σ( ]R l C σ( 4,5,6 ɛ(σ det([l C σ( R l C σ( ]lc σ( 1,2,3 det([l C σ( R l C σ( ]lc σ( 4,5,6 (6.7 where σ s any permutaton of 1,..., 6 by movng three elements to the front of the sequence. Assume that R s gven, then genercally we can choose [l C σ( R l C σ( ] (lc σ( R l C σ( (lc σ( R l C σ(, (6.8 [l C σ( R l C σ( ] (lc σ( R l C σ( (lc σ( R l C σ(. So (?? s cubc wth respect to R. There are 7 equatons n R, R, one s (??, the other sx are det(l C R l C R l C 0, for 1,..., 6. (6.9 Theoretcally, R, R can be solved from (??. Notce that (?? s a lnear system wth respect to ether R or R. If for example R s gven, then generally there s a unque R that can be solved from (??. The method to compute R s as follows. Let R (r 1 r 2 r 3, and let lc (l 1, l 2, l 3 T. Then (?? can be wrtten as r 3 l 3 r 3 (l C R l C (l 1 r 1 + l 2 r 2 (l C R l C, for 1,..., 6. (6.10 can then be solved from three equatons of (??, and the soluton r 3 r 3 (r 1, r 2 s lnear wth respect to (r 1, r 2. We can get another expresson of r 3 by solvng the other three equatons of (??, and then get a vector equaton lnear wth respect to (r 1, r 2. Solvng the equaton for e 2, we get the soluton r 2 r 2 (r 1 whch s lnear wth respect to r 1. Now we have obtaned from the sx equatons n (?? r 2 r 2(r 1, r 3 r 3(r 1, (6.11 both of whch are lnear wth respect to r 1. The constrants for R to be a rotatonal matrx are det(r > 0 and r 1 r 1 r 2 r 2 r 3 r 3 1; r 1 r 2 r 1 r 3 r 2 r 3 0. (6.12 Substtutng (?? nto (??, we get 6 equatons whose terms are ether ndependent of or quadratc wth respect to r 1 (r 11, r 12, r 13 T, or equvalently, ether ndependent of or lnear