Towards Gauge Invariant Bundle Adjustment: A Solution Based on Gauge Dependent Damping

Transcription

1 EXTENDED VERSION SHORT VERSION APPEARED IN THE 9TH ICCV, NICE, FRANCE, OCTOBER 003. Towards Gauge Invariant Bundle Adjustent: A Solution Based on Gauge Dependent Daping Adrien Bartoli INRIA Rhône-Alpes, 655, avenue de l Europe Saint Isier cedex, France. First.Last@inria.fr Abstract Bundle ajustent is used to obtain accurate visual reconstructions by iniizing the reprojection error. The coordinate frae abiguity, or ore generality the gauge freedos, has been dealt with in different anners. It has often been reported that standard bundle adjustent algoriths were not gauge invariant: two iterations within different gauges can lead to geoetrically very different results. Surprisingly, ost algoriths do not exploit gauge freedos to iprove perforances. We consider this issue. We analyze theoretically the ipact of the gauge on standard algoriths. We show that a sufficiently general daping atrix in Levenberg-Marquardt iteration can be used to iplicitly reproduce a gauge transforation. We show that if the daping atrix is chosen such that the decrease in the reprojection error is axiized, then the iteration is gauge invariant. Experiental results on siulated and real data show that our gauge invariant bundle adjustent algorith outperfors existing ones in ters of stability.. Introduction The recovery of accurate 3D structure and caera otion fro iages is a ajor research challenge in photograetry [] and coputer vision [7]. Bundle adjustent is a technique to refine a visual reconstruction to produce jointly optial 3D structure and caera otion. Under certain assuptions on the noise on the observed features, bundle adjustent consists in iniizing the reprojection error, which is in general a non-linear procedure. Second-order non-linear least squares algoriths are usually eployed, naely the Gauss-Newton and Levenberg-Marquardt ethods. These ethods iteratively iprove sub-optial paraeter estiates by solving noral equations. Efficient solutions are possible thanks to the sparse block structure of the noral equations. The Levenberg-Marquardt ethod has proved the ost successful due to the use of a trust region strategy, ipleented via a daping of the noral equations, also called augentation. An inherent proble in bundle adjustent is the choice of the coordinate frae in which the reconstruction is expressed. This coordinate frae is called the gauge, or the datu in the photograetry counity. A gauge is a subset of paraeter vectors such that any two of the do not share the sae underlying geoetry. Paraeter vectors corresponding to the sae geoetry are related by gauge transforations. It has been shown in [8] that these transforations have a group structure. The reprojection error is gauge invariant since it reflects the inherent erit of a reconstruction. These considerations hold whichever caera odel is used and whichever calibration level is available. Exaples of geoetric gauge freedos are the global rotation, translation and scale in etric reconstruction and the position of points along the line in the two point representation of lines. Exaple of algebraic gauge freedos is the scale factor of hoogeneous coordinates. When ignored, gauge freedos ay induce nuerical estiation probles, since they iply the rank deficiency of the noral equations. It has been reported that ost bundle adjustent algoriths initialized with the sae reconstruction expressed within different gauges can yield extreely different results, in ters of coputational cost [, 3, 8,,, 4, 7]. In other words, these algoriths are not gauge invariant. Carefully choosing the gauge can greatly iprove the reliability. In practice, we observe that ost algoriths converge to the sae solution, but ay require different nuber of iterations. Gauge freedos iply that the noral equations to be solved at each iteration do not have a unique solution, i.e. the design atrix is singular, but rather a faily of solutions, whose diension is the nuber of gauge freedos. Daping the noral equations consists in adding a syetric positive definite atrix to the design atrix. This atrix is ost We eploy gauge invariant in a anner different fro [8,, ]. These authors propose algoriths based on using a pre-defined global gauge. Hence, no atter the gauge within which the initial solution is expressed, their algoriths will give the sae result. However, the behaviour of these algoriths do depend upon the pre-defined global gauge. In this sense, these algoriths are not gauge invariant.

2 of the tie chosen as a diagonal one, which corresponds to an elliptical trust region, see e.g. [6] and 3 for ore details. Surprisingly, gauge freedos have rarely been exploited to iprove on the reliability of algoriths. Most algoriths are not gauge invariant and are based on soehow arbitrary gauges which are not chosen to eet soe efficiency criteria. More details are given in. The ain contribution of this paper is an algorith that axiizes the decrease in the reprojection error at each iteration, in a gauge invariant anner. In ore detail: In 4, we forally state the gauge invariance property of a bundle adjustent algorith. A detailled investigation of how the gauge influences Gauss-Newton and Levenberg-Marquardt iterations allows to conclude that in general neither of the is gauge invariant. This analysis shows that the gauge and the daping atrix in Levenberg-Marquardt iterations are closely linked: Gauge transforations change the standard elliptical trust region to a ore coplex shape. In 5, we concentrate on Levenberg-Marquardt iterations and propose iplicit gauges, as opposed to the explicit transfer of a reconstruction to the desired gauge. This shows that in soe sense, the daping atrix can encapsulate gauge transforations. We state the following iportant result. If the for of the daping atrix is sufficiently general, then the iteration can be ade gauge invariant by carefully choosing this atrix, based on a gauge dependent criterion. In 6, we propose a gauge dependent daping atrix which defines a trust region with the following iportant properties. (i) it akes the underlying iteration gauge invariant, (ii) it axiizes the decrease in the reprojection error up to first order and (iii) it preserves the sparse block structure of the noral equations. Finally 7 reports soe experiental results and 8 concludes. For siplicity of notation, we deal with projective reconstruction of points and caeras. Extension to other caera odels (e.g. affine caeras), other calibration levels (e.g. etric reconstruction) and other types of features (e.g. lines) is straightforward, following [8, ]. Our final gauge invariant bundle adjustent algorith is suarized in a practical anner in table. Notation. We ake no foral distinction between coordinate vectors and physical entities. Equality up to a non-null scale factor is denoted by and is soeties written by introducing explicitly the scale factor: (x x ) ( α x = αx ). Transposition and transposed inverse are denoted by T and T. Vectors are typeset using bold fonts (q, Q), atrices using sans-serif fonts (P, T) and scalars in italics. Indices are used to indicate the size of a atrix or vector (P (3 4), q (3 ) ) or to index a set of entities. The row-wise atrix vectorization is written vect. Let P i, i =... n denote the n reconstructed caera atrices and Q j, j =... denote the reconstructed 3D points. Structure and otion paraeters are contained in a (p ) vector X, partitionned into n otion paraeters M and 4 structure paraeters S (hoogeneous coordinates of points) as: X T = (vect T (P )... vect T (P n ) M T T Q... Q T ) S T Changing the 5-degrees of freedo projective reconstruction basis and the individual scale factors of the caera atrices and point coordinate vectors leave the underlying geoetry invariant. Hence there are g = 5 + n + degrees of gauge freedo.. Previous work A survey of gauge freedo handling strategies is [7, 9]. Methods to deal with this proble are either globally free, i.e. they left the gauge free to drift, or globally fixed, i.e. they enforce a global gauge. Globally free ethods are based on selecting a solution of the noral equations, using a pseudo-inverse [], nuerical daping [9, 0, 6, 7] or local gauge constraints. For exaple, the standard photograetric inner constraints specify that the reconstructed points should not be translated, rotated and scaled, but they do not specify where are the reconstructed points, see [3, 7]. They are devised to iniize the nor of the paraeter vector update, as well as the variance of the paraeter estiate. In [, 9.5], it is shown experientally that this behaves better than globally fixed ethods (see below), but that leaving the gauge drift freely can soeties be better. Globally fixed ethods enforce a pre-chosen gauge. Soe ethods use reference eleents (trivial gauge), either features [, 4, 3] (object-centered gauge) or caeras [, 5, 6]. In [, p.40] the author suggests that control points or reference lengths can be used to fix a etric recontruction basis, while in [4, 3] a siilar ethod based on 5 reference points is used to fix a projective reconstruction basis. The constraints are enforced using either Lagrange ultipliers [, p.40] or by eliination [, p.4], [4, 3]. In [6], the author considers projective reconstruction and partially fix the gauge by setting a canonical for to a reference caera, while [] uses two reference caeras. Another possibility for a globally fixed gauge is to use global gauge constraints [8,, ], which raises the proble of enforcing these constraints during optiization. In [], artificial extra observations are added to the cost function with a heavy weight. In [8], the gauge is left free to drift during the iteration, but the result is projected onto the gauge afterhand, while in [], a paraeter subspace projected onto

3 the gauge is coputed before the iteration. Another possibility is [, p.4], where Lagrange ultipliers are introduced and incorporated as unknowns in the optiization. Besides the globally free photograetric inner constraints, all these ethods are based on soehow arbitrary gauges which are not chosen to eet soe efficiency criteria. A recent attept in this direction is [4]. Based on gauge theory, the authors analyze, in the context of Euclidean reconstruction, which physical easure is the ost likely to axiize the accuracy of the reconstruction while setting the scene scale. 3. Standard Algoriths Bundle adjustent consists in refining a visual reconstruction to produce jointly optial 3D structure and caera otion. By optial, we ean that the axiu likelihood estiate of structure and otion is sought, which is achieved by iniizing an appropriate cost function. If we assue that the observed point features are corrupted by independently and identically distributed Gaussian noise, then the cost function is the reprojection error, given by the su of squared differences between observed features q ij and predicted features q ij : C(X ) = n i= j= w ij d (q ij, q ij ) = r T r, () where w ij is one if point j is visible in view i and zero otherwise. The (r ) vector r is the residual error vector. Predicted features are given by q ij P i Q j. Given a sub-optial solution X 0, the paraeter vector X is iteratively updated by X X + δ, where the increent δ is obtained as follows. Let J = r X denotes the (r p) Jacobian atrix of the residual error vector r with respect to structure and otion paraeters X, g = J T r be the (p ) gradient vector of C and let N = J T J be the Gauss-Newton approxiation of the (p p) Hessian atrix. The reprojection error is approxiated by: C(X + δ) C(X ) + g T δ + δt Nδ. () e The iniu of this siple local quadratic odel can be found by setting e δ = 0, which gives the Gauss-Newton iteration, through the following noral equations: Nδ = g. (3) Note that N has a rank deficiency of g (the nuber of gauge freedos). The Levenberg-Marquardt iteration is based on daping or augenting the noral equations, as follows : (N + W(λ) )δ = g, (4) N where λ > 0 is related to the trust region radius and W(λ) is soe syetric positive definite (p p) weight atrix, called the daping atrix, often chosen as: W(λ) = λi (p p), which corresponds to a spherical trust region. This is the original strategy proposed by Levenberg and Marquardt [9, 0] and recoended in [7]. Another coonly used solution, due to [6] and recoended in [5, 7, 5] is W(λ) = λd where D is a diagonal atrix containing the diagonal entries of N, which gives an elliptical trust region. The daping atrix ust satisfy a noralization constraint so that the trust region radius is eaningful, e.g. I (p p) = p. Note that the daping guarantees that N is full-rank. Paraeter λ is tuned as follows: if paraeters X + δ decrease the error, i.e. if C(X + δ) < C(X ), then the step is accepted and the value of λ is divided by soe constant, often 0, else the step is rejected and λ is ultiplied by the constant. 4. Influence of the Gauge Not surprisingly, It has been reported that in general, bundle adjustent was not gauge invariant, i.e. an iteration within different gauges yield geoetrically different results [, 3, 8,,, 4, 7]. We forulate the gauge invariance property and exaine under which condition standard algoriths satisfy it. 4.. Gauge Transforation and Invariance Gauge transforations change the paraeter vector without changing the underlying geoetry, e.g. [8]. Let X T = (M T S T ) and ˇX T = ( ˇM T Š T ) be two paraeter vectors of the sae reconstruction expressed within two different gauges G and Ǧ. Let T be the full-rank transforation relating the two underlying projective bases, defined such that: ˇP i = γ i P i T and ˇQ j = α j T Q j, where γ i and α j are unknown non-zero scale factors. Entities written with a ˇare expressed within the gauge Ǧ. The structure and otion paraeters transfor as: Š = diag(α T,..., α T ) S ˇM = diag(γ T T,..., γ n T T ) M, 3n since vect(γpt) = diag(γt T, γt T, γt T ) vect(p). We deduce that the paraeter vectors are related by ˇX = TX where the gauge transforation T is defined by: T = diag(γ T T,..., γ n T T, α T,..., α T ). (5) 3n Note that det(t) 0 det( T) 0 since the γ i and α j are non-zero. In general, X and ˇX are related by a unique gauge transforation. 3

4 The geoetric equivalence of two reconstructions, denoted by =, is an egality up to the gauge defined by: (X = ˇX ) ) ( T ˇX = TX. (6) with T of the for (5). We can now forally state the gauge invariance property. A bundle adjustent algorith is gauge invariant if it preserves geoetric equivalence: (X = ˇX ) ((X + δ) = ( ˇX + ˇδ)). By writing explicitly this property using equation (6), expanding, and since X and ˇX are related by a unique T, we obtain the following definition of gauge invariance: ( T ( ˇX = TX ) ) (ˇδ = Tδ). (7) Note that although derived in a different anner, this definition is siilar to the one given in []. We exaine successively under which conditions the Gauss-Newton and Levenberg-Marquardt iterations are gauge invariant. 4.. Gauss-Newton Consider a Gauss-Newton iteration. The increent δ within gauge G is given by solving the noral equations (3). Due to gauge freedos, atrix N has a rank deficiency of g, and ultiple solutions hold, corresponding to different gauges. Let G be a (p g) full colun-rank atrix defined by NG = 0, i.e. the coluns of G span the nullspace of N. Denoting by the Moore-Penrose pseudo-inverse, the possible solutions of the noral equations are paraeterized by a (g ) vector v as δ = (J T J) J T r + Gv, where we substituted N = J T J and g = J T r. Since J = r X = r ˇX ˇX X = ˇJ T, we obtain: δ = ( T TˇJ TˇJ T) TTˇJr+ T Ǧv. Fro equation (7), gauge invariance holds if and only if: ( T T Ň T) = T Ň T T and v = ˇv, which is verified if and only if T is orthonoral [], i.e. T = T T. Hence, Gauss-Newton-based bundle adjustent is not gauge invariant. The previously derived condition does not leave enough flexibility to be exploited in this direction Levenberg-Marquardt Consider a Levenberg-Marquardt iteration. The increent δ within gauge G is given by solving the daped noral equations (4) N δ = g. By substituting N = N + W(λ) = J T J + W(λ) and g = J T r in this equation and since det(n ) 0, we obtain: δ = (J T J + W(λ)) J T r. By substituting J = ˇJ T and expanding, we get: δ = ( T TˇJTˇJ T + W(λ)) TTˇJT r = T (Ň + T T W(λ) T ) ˇJT r. Fro equation (7), gauge invariance holds if and only if: ˇW(λ) = T T W(λ) T. (8) The usual choices for atrices W(λ) and ˇW(λ), e.g. W(λ) = ˇW(λ) = λi, do not fulfill the above-derived property. Hence, standard ipleentations of Levenberg- Marquardt-based algoriths are not gauge invariant. Equation (8) can be verified if an appropriate gauge dependent choice is ade for the daping atrices. This is the cornerstone for the gauge invariant ethod proposed in the next section. 5. Explicit and Iplicit Gauges Standard bundle adjustent algoriths are not gauge invariant. Hence, given a paraeter estiate, there ust exist an optial local gauge within which the iteration reduces the reprojection error better than within the others. The proble of finding this gauge is dealt with in the next section. In this section, we exaine the relationship between the daping atrix and gauge transforations. A coonly used solution to globally enforce a gauge, e.g. [8,, ], is what we call explicit gauge fixing. It consists in explicitly expressing the reconstruction within the desired gauge before each iteration. It iplies the transfer of the points, the caeras, and all other entities being optiized, into the desired gauge. We propose iplicit gauge fixing. Consider equation (8). On the one hand, it tells us that the algorith is not gauge invariant. On the other hand, it shows that there exists a close link between the gauge and the daping atrix. Hence, it can be used to perfor bundle adjustent within gauge Ǧ, while choosing ˇW(λ) such that it behaves exactly as if is was conducted within gauge G. For exaple if W(λ) = λi and ˇW(λ) = λ T T T, then running the algorith within gauges G and Ǧ is strictly equivalent. We confir in our experients that explicit and iplicit gauges give exactly the sae results. Let A = T T T, then ˇW(λ) = λ T T T is given by: ˇW(λ) = λ diag(γ A,..., γn }{{ A, α } A,..., α A ). (9) 3n In other words, a spherical trust region within gauge G corresponds to a ore coplex trust region within gauge Ǧ, encapsulated by the syetric positive definite atrix ˇW(λ) defined by equation (9). To suarize, it is possible to reproduce the behaviour of bundle adjustent within a certain gauge, without explicitly expressing the reconstruction within this gauge, only by choosing the appropriate daping atrix, depending on the gauge transforation. Based on this reasonning, we propose the following result. It can be shown that equation (8) with W(λ) = ˇW(λ) = λi is verified as in the Gauss-Newton case, i.e. if T is orthonoral. 4

5 Proposition (Gauge invariance conditions) Levenberg- Marquardt iterations can be ade gauge invariant if the following two conditions are verified: (i) the for of the daping atrix is at least as general as the for given by equation (9) and (ii) the daping atrix is uniquely deterined by a gauge dependent criterion defined such that equation (8) is verified. 6. Gauge Dependent Daping We propose a solution to choose a gauge dependent daping atrix W(λ) = λw which has the three following iportant properties. First, it akes the underlying iteration gauge invariant. Second, it axiizes the decrease in the reprojection error up to first order. Third, it preserves the sparse block structure of the noral equations. In order to achieve this result, we begin by deriving an approxiation of the reprojection error, depending upon W(λ). A quadratic approxiation of C(X + δ) is given by equation (). The increental error e is given by: e = g T δ + δt Nδ. (0) The increent vector δ is given by solving the augented noral equations (4): δ = (N + W(λ)) g. Since det(w(λ)) 0, we can forulate the following Taylor expansion: (N + W(λ)) = ( ) n Z(n), where Z(n) = (W(λ) N) n W(λ). This expression is valid provided W(λ) N <, i.e. when N is sall or λ is large. This leads to: δ = ( ) n Z(n)g. We substitute the above expression in equation (0). The first ter is rewritten as: g T δ = ( ) n g T Z(n)g = ( ) n K(n), where K(n) = g T Z(n)g. The second ter becoes: δt Nδ = ( ( ) n g T Z(n) ) N (( ) n Z(n)g). After expansion and using the following property arising fro the syetry of W(λ): Z(n)NZ(n ) = Z(n + n + ), we obtain: δt Nδ = ( ) n (n + )K(n + ). This gives the following Taylor expansion for e: e = ( ( ) n+ + n ) K(n). () Proposition (Gauge invariant iterations) The daping atrix defined such that equation () is iniized satisfies proposition. Hence, the underlying iteration is gauge invariant. Proof. We drop the paraeter λ for this proof. Equation () can be rewritten as: e = g ( W T + 3 ) W NW... g. Transfer the reconstruction fro the gauge G to Ǧ by applying the gauge transforation T. By substituting g = T T ǧ and N = T TŇ T, we obtain: e = ǧ T T ( W + 3 ) W TT Ň TW... T T ǧ, and hence, ˇW = TW T T, which verifies the gauge invariance equation (8) and concludes the proof. Finding the daping atrix that iniizes e, equation (), is a coplicated proble. We propose to keep only the first-order ter: e g T W(λ) g, which reduces the proble to finding a syetric, positive definite, daping atrix λw such that: W = arg ax W, W =p gt (λw) g. The noralization condition W = p is the sae as the standard choice W(λ) = λi since I (p p) = p. Obviously, the solution depends upon the for chosen for atrix W. For exaple, if W = diag(d), where d is a (p ) vector with strictly positive eleents, then the proble transfors in: ( ) p W = diag arg ax gk/d k, d, d =p k= which has the siple solution (up to scale) W diag(g). According to proposition, this solution can not yield a subsequent gauge invariant algorith since a diagonal daping atrix is not general enough for iplicit gauges. In a ore general anner, since W(λ) is a syetric atrix, W(λ) = λ RT R, where R is an upper-triangular atrix. With this paraeterization, and since g T λ RT Rg = λ Rg, we obtain: R = arg ax R, R T R =p λ Rg. () Hence, the coefficients of atrix R can be found by solving a siple linear least squares optiization proble. 5

6 . Copute the daping atrix W(λ) = λr T R where: R = diag(m,..., M }{{ n, S,..., S). } 3n The (4 4) upper-triangular atrices M i and S are fored by solving (see ain text): M i = arg ax S = arg ax M i, M T i Mi = k= S, S T S =4 j= 3 M i gm,i k Sg S,j, where g S,j and gm,i k are (4 ) gradient vectors for the j-th point and for the k-th row of the i-th caera atrix respectively.. Perfor one Levenberg-Marquardt iteration with the daping atrix W(λ). This iteration is gauge invariant due to the above choice. 3. Optional: Project the estiate onto a global gauge. Table. The gauge invariant iteration we propose. Periodical enforceent of global gauge constraints (step 3) such as renoralization of hoogeneous coordinates is recoended. Gauge invariance holds in the sense that any global gauge can be enforced in step 3 without affecting the result. Fro proposition and iposing the fact that the daping should not spoil the sparse block structure of the noral equations, we are left with the following possible for: R = diag(r M,,..., R M,n, R S,,..., R S, ), n where the R M,i and the R S,j are respectively ( ) and (4 4) upper-triangular atrices. Given that each of the p paraeters gives one constraint on the daping atrix and since it ust be uniquely deterined (i.e. it ust not have ore than p paraeters), we propose the following 0(n + )-paraeter choice: R M,i = diag(m i, M i, M i ) and R S,j = S, where the M i and S are (4 4) upper-triangular atrices. This choice eans that the paraeter block of each caera i has its own 0 paraeter daping block M T i M i, while a unique 0 paraeter daping block S T S is defined for point paraeters. This for is handy since each block can be found independently by solving a proble of the for (). More details are given in table. Below, we give details about solving the low-diensional axiization probles of table. Let the proble to be solved be ax x, x =b Cx. The singular vector v associated to the largest singular value of atrix C gives the solution for x as x = bv. Singular value decoposition of atrix C can be used to copute v. 7. Experiental Results We copare our gauge invariant algorith, denoted GAUGE INVARIANT, to soe other ones. FREE directly optiizes the caera atrices and 3D points. Pseudo-inverse is used to solve the noral equations. HARTLEY [6] consists in using a reference caera to partially eliinate the gauge and BARTOLI [] consists in using two reference caeras to eliinate the gauge. FAUGERAS [4] is based on fixing the coordinates of 5 reference points to enforce the gauge. A siilar approach is proposed by Mohr et al in [3]. MCLAUCHLAN [] consists in enforcing a noralized basis after each iteration and in using first order gauge constraints in the noral equations and KANATANI [8] is siilar to ethod FREE but periodically projects the estiate on a pre-chosen gauge to prevent it to drift too uch. When reference caeras or points are required by the paraeterization, they are chosen at rando. Note that we have ipleented all these algoriths in a unified anner: the optiization engine is unique, only the paraeterization and the daping strategy change. We easure the nuber of iterations and the reprojection error at convergence. The initial solution, denoted by INIT, is coputed by registering caeras in turn to a two-view reconstruction. Iage point coordinates are standardized such that they lie in [... ]. 7.. Siulated Data We siulate = 00 points lying in a cube with eter side length, observed by n = 5 caeras with a focal length of 000 pixels. The points are offset fro a base plane lying inside the cube, with a ean offset denoted by d. Caeras are situated 0 eters away fro the center of the cube. The baseline between consecutive caeras is 3 eters. All points are visible in all views. We add a centered Gaussian noise on true point positions with a pixel variance. We vary soe paraeters of the above-described setup to copare the algoriths in different situations. The results are averaged over 50 trials. Figure shows the results when the scene flatness, i.e. the ean offset d fro the plane, is varied. We observe that the reprojection errors, i.e. the accuracy, are undistinguishable for all ethods, except for ethods FREE which converges to a different local iniu than the others, for weak geoetry, i.e. when the ean offset fro the plane d is sall. Concerning the nuber of iterations, i.e. the coputational cost, we observe that when the ean offset is large, i.e. when the geoetry is strong, there is only slight differences between the different ethods besides for ethod FREE, which 6

7 rag replaceents frag replaceents INIT reprojection error (pixel) nuber of iterations INIT FREE FAUGERAS HARTLEY BARTOLI KANATANI PSfrag replaceents MCLAUCHLAN GAUGE INVARIANT INIT scene unflatness FREE FAUGERAS HARTLEY BARTOLI KANATANI MCLAUCHLAN GAUGE INVARIANT Figure shows the reprojection error as a function of the iterations for the scene setting with d = 0.5. As expected, we observe that the error is faster to decrease as the nuber of iterations gets low. reprojection error (pixel) FREE FAUGERAS HARTLEY BARTOLI KANATANI MCLAUCHLAN GAUGE INVARIANT iteration nuber Figure. Reprojection error as a function of the iterations scene unflatness Figure. Reprojection error and nuber of iterations when varying the scene unflatness, fro weak to strong geoetry. takes clearly ore iterations to converge. As the geoetry becoes weaker, i.e. when the offset decreases, large discrepancies can be observed between the different ethods. Method FAUGERAS based on reference points gives bad results, since using reference points to fix a projective basis can be very unstable. Methods HARTLEY and BARTOLI based on reference caeras and KANATANI based on periodical projection on a pre-chosen gauge give reliable results when the geoetry is strong enough. Method MCLAUCHLAN perfors better, while GAUGE INVARIANT is the ethod the less sensitive to the instability of the scene. We tested other strong to weak scene configurations, based on varying the nuber of points and caeras, the baseline between consecutive caeras and the visibility (not shown here due to lack of space). We observed siilar results as above: strong geoetries reduce the discrepancies between the different ethods. Siilarly, we tested the influence of the initialization. As expected, discrepancies between the different ethods reduce as the initialization gets ore accurate. 7.. Real Data We copare the algoriths on various iage streas. For two of the, the office sequence, figure 3, and the hotel sequence 3, we show results, see table. For the office sequence, all ethods give a reprojection error of pixel. For the hotel sequence, they all gave pixel, besides ethod FREE which gave.568 pixels. Concerning the nuber of iterations, the sae observations as for siulated data can be ade. The hotel sequence gives a weak geoetry since an affine caera odel is well-adapted to these data. Hence, a full perspective projection odel is not well-constrained, which tends to increase the discrepancies between the different ethods. Algorith office seq. hotel seq. FREE 8 HARTLEY 7 6 BARTOLI 8 9 FAUGERAS 0 MCLAUCHLAN 6 5 KANATANI 7 6 GAUGE INVARIANT 7 3 Table. Nuber of iterations of the algoriths for the office and the hotel sequences. 3 These data have been provided by the Modeling by Videotaping group in the Robotics Institute, Carnegie Mellon University. 7

8 Figure 3. One out of the 5 fraes of the office sequence, overlaid with the 38 corner features (left) and snapshot of the reconstructed shape (right). 8. Conclusions We proposed a bundle adjustent algorith which axiizes the decrease in the error at each iteration, regardless the gauge within which the reconstruction is expressed. We derived this algorith based on a careful study of how the gauge influence the iterations of standard algoriths, which are not gauge invariant. In particular, concerning Levenberg-Marquardt iterations, we showed that the standard diagonal daping atrices, defining elliptical trust regions, transfor to a ore coplex shaped trust region when changing the gauge. Based on this, we proposed a gauge dependent daping atrix and a practical algorith to copute it, that allows gauge invariant iterations while axiizing the decrease in the reprojection error. The final algorith can be incorporated in a straightforward anner to existing iniization engines since it consists in appropriately choosing the daping atrix by solving low-diensional linear least squares systes. The sparse block structure of the noral equations is preserved. We copared our algorith to existing ones using siulated and real data. We observed that it is ore reliable in the sense that it is less sensitive to weak geoetry and weak initialization. Moreover, the error decreases ore intensely throughout the iterations. Most algoriths converge to the sae solution, but require different nuber of iterations. Santa Margherita Ligure, Italy, pages Springer-Verlag, May 99. [5] R. Hartley. Euclidean reconstruction fro uncalibrated views. In Proceeding of the DARPA ESPRIT workshop on Applications of Invariants in Coputer Vision, Azores, Portugal, pages 87 0, October 993. [6] R. Hartley. Projective reconstruction and invariants fro ultiple iages. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(0):036 04, October 994. [7] R. Hartley and A. Zisseran. Multiple View Geoetry in Coputer Vision. Cabridge University Press, June 000. [8] K. Kanatani and D. D. Morris. Gauges and gauge transforations for uncertainty description of geoetric structure with indeterinacy. IEEE Transactions on Inforation Theory, 47(5), July 00. [9] K. Levenberg. A ethod for the solution of certain non-linear probles in least squares. Quarterly of Applied Matheatics, pages 64 68, 944. [0] D. Marquardt. An algorith for least-squares estiation of nonlinear paraeters. Journal of the Society for Industrial and Applied Matheatics, ():43 44, June 963. [] P. F. McLauchlan. Gauge invariance in projective 3D reconstruction. In Proceedings of the Multi-View Workshop, Fort Collins, Colorado, USA, 999. [] P. F. McLauchlan. Gauge independence in optiization algoriths for 3D vision. In Proceedings of the Vision Algoriths Workshop, Dublin, Ireland, 000. [3] R. Mohr, L. Quan, and F. Veillon. Relative 3D reconstruction using ultiple uncalibrated iages. The International Journal of Robotics Research, 4(6):69 63, 995. [4] D. D. Morris, K. Kanatani, and T. Kanade. Gauge fixing for accurate 3d estiation. In Proceedings of the Conference on Coputer Vision and Pattern Recognition, Kauai, Hawaii, USA, Deceber 00. [5] W. Press, S. Teukolsky, W. Vetterling, and B. Flannery. Nuerical Recipes in C - The Art of Scientific Coputing. Cabridge University Press, nd edition, 99. [6] G. A. F. Seber and C. J. Wild. Non-linear regression. John Wiley & Sons, New York, 989. [7] B. Triggs, P. McLauchlan, R. Hartley, and A. Fitzgibbon. Bundle ajustent a odern synthesis. In B. Triggs, A. Zisseran, and R. Szeliski, editors, Proceedings of the International Workshop on Vision Algoriths: Theory and Practice, Corfu, Greece, volue 883 of Lecture Notes in Coputer Science, pages Springer-Verlag, 000. References [] K. Atkinson, editor. Close Range Photograetry and Machine Vision. Whittles Publishing, 996. [] A. Bartoli. On the non-linear optiization of projective otion using inial paraeters. In Proceedings of the 7th European Conference on Coputer Vision, Copenhagen, Denark, volue, pages , May 00. [3] A. Deranis. The photograetric inner constraints. ISPRS Journal of Photograetry and Reote Sensing, 49():5 39, 994. [4] O. Faugeras. What can be seen in three diensions with an uncalibrated stereo rig? In G. Sandini, editor, Proceedings of the nd European Conference on Coputer Vision, 8