Minimizing Algebraic Error in Geometric Estimation Problems

Similar documents
CSci 6974 and ECSE 6966 Math. Tech. for Vision, Graphics and Robotics Lecture 21, April 17, 2006 Estimating A Plane Homography

CS4495/6495 Introduction to Computer Vision. 3C-L3 Calibrating cameras

Estimating the Fundamental Matrix by Transforming Image Points in Projective Space 1

Chapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems

Report on Image warping

Module 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur

Lectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix

Structure from Motion. Forsyth&Ponce: Chap. 12 and 13 Szeliski: Chap. 7

Inner Product. Euclidean Space. Orthonormal Basis. Orthogonal

Singular Value Decomposition: Theory and Applications

Some Comments on Accelerating Convergence of Iterative Sequences Using Direct Inversion of the Iterative Subspace (DIIS)

15.1 The geometric basis for the trifocal tensor Incidence relations for lines.

Difference Equations

Generalized Linear Methods

Problem Set 9 Solutions

Errors for Linear Systems

8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS

Lecture Notes on Linear Regression

Kernel Methods and SVMs Extension

Linear Approximation with Regularization and Moving Least Squares

LINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity

Lecture 3. Ax x i a i. i i

Parameter estimation class 5

More metrics on cartesian products

NUMERICAL DIFFERENTIATION

Affine transformations and convexity

1 Matrix representations of canonical matrices

APPENDIX A Some Linear Algebra

Vector Norms. Chapter 7 Iterative Techniques in Matrix Algebra. Cauchy-Bunyakovsky-Schwarz Inequality for Sums. Distances. Convergence.

Tracking with Kalman Filter

A New Refinement of Jacobi Method for Solution of Linear System Equations AX=b

Salmon: Lectures on partial differential equations. Consider the general linear, second-order PDE in the form. ,x 2

The Trifocal Tensor The geometric basis for the trifocal tensor

The Minimum Universal Cost Flow in an Infeasible Flow Network

Global Sensitivity. Tuesday 20 th February, 2018

where λ = Z/f. where a 3 4 projection matrix represents a map from 3D to 2D. Part I: Single and Two View Geometry Internal camera parameters

MMA and GCMMA two methods for nonlinear optimization

Laboratory 1c: Method of Least Squares

Matrix Approximation via Sampling, Subspace Embedding. 1 Solving Linear Systems Using SVD

Module 9. Lecture 6. Duality in Assignment Problems

EEE 241: Linear Systems

ISSN: ISO 9001:2008 Certified International Journal of Engineering and Innovative Technology (IJEIT) Volume 3, Issue 1, July 2013

2.3 Nilpotent endomorphisms

Formulas for the Determinant

Coordinate-Free Projective Geometry for Computer Vision

Laboratory 3: Method of Least Squares

Structure and Drive Paul A. Jensen Copyright July 20, 2003

Feature Selection: Part 1

Polynomial Regression Models

Approximate Smallest Enclosing Balls

APPENDIX 2 FITTING A STRAIGHT LINE TO OBSERVATIONS

Solutions to exam in SF1811 Optimization, Jan 14, 2015

Time-Varying Systems and Computations Lecture 6

STAT 309: MATHEMATICAL COMPUTATIONS I FALL 2018 LECTURE 16

The Geometry of Logit and Probit

1 GSW Iterative Techniques for y = Ax

CHAPTER 5 NUMERICAL EVALUATION OF DYNAMIC RESPONSE

Lecture 12: Discrete Laplacian

CME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 13

x = , so that calculated

= = = (a) Use the MATLAB command rref to solve the system. (b) Let A be the coefficient matrix and B be the right-hand side of the system.

On a direct solver for linear least squares problems

VQ widely used in coding speech, image, and video

Physics 5153 Classical Mechanics. D Alembert s Principle and The Lagrangian-1

Fall 2012 Analysis of Experimental Measurements B. Eisenstein/rev. S. Errede

Composite Hypotheses testing

Chapter 11: Simple Linear Regression and Correlation

5 The Rational Canonical Form

Solution Thermodynamics

Supplement: Proofs and Technical Details for The Solution Path of the Generalized Lasso

LECTURE 9 CANONICAL CORRELATION ANALYSIS

n α j x j = 0 j=1 has a nontrivial solution. Here A is the n k matrix whose jth column is the vector for all t j=0

The Second Anti-Mathima on Game Theory

Econ107 Applied Econometrics Topic 3: Classical Model (Studenmund, Chapter 4)

Inexact Newton Methods for Inverse Eigenvalue Problems

Inductance Calculation for Conductors of Arbitrary Shape

CHAPTER III Neural Networks as Associative Memory

17. Coordinate-Free Projective Geometry for Computer Vision

COMPLEX NUMBERS AND QUADRATIC EQUATIONS

The Order Relation and Trace Inequalities for. Hermitian Operators

An Interactive Optimisation Tool for Allocation Problems

FINITELY-GENERATED MODULES OVER A PRINCIPAL IDEAL DOMAIN

Numerical Heat and Mass Transfer

2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification

CIS526: Machine Learning Lecture 3 (Sept 16, 2003) Linear Regression. Preparation help: Xiaoying Huang. x 1 θ 1 output... θ M x M

= z 20 z n. (k 20) + 4 z k = 4

Joint Statistical Meetings - Biopharmaceutical Section

Lossy Compression. Compromise accuracy of reconstruction for increased compression.

Grover s Algorithm + Quantum Zeno Effect + Vaidman

Comparison of Regression Lines

3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X

The Multiple Classical Linear Regression Model (CLRM): Specification and Assumptions. 1. Introduction

C/CS/Phy191 Problem Set 3 Solutions Out: Oct 1, 2008., where ( 00. ), so the overall state of the system is ) ( ( ( ( 00 ± 11 ), Φ ± = 1

10. Canonical Transformations Michael Fowler

Large-scale packing of ellipsoids

ON A DETERMINATION OF THE INITIAL FUNCTIONS FROM THE OBSERVED VALUES OF THE BOUNDARY FUNCTIONS FOR THE SECOND-ORDER HYPERBOLIC EQUATION

Games of Threats. Elon Kohlberg Abraham Neyman. Working Paper

Chapter 2 - The Simple Linear Regression Model S =0. e i is a random error. S β2 β. This is a minimization problem. Solution is a calculus exercise.

Linear Regression Analysis: Terminology and Notation

Finite Element Modelling of truss/cable structures

Transcription:

Mnmzng Algebrac n Geometrc Estmaton Problems Rchard I. Hartley G.E. Corporate Research and Development PO Box 8, Schenectady, NY 39 Emal : hartley@crd.ge.com Abstract Ths paper gves a wdely applcable technque for solvng many of the parameter estmaton problems encountered n geometrc computer vson. A commonly used approach s to mnmze an algebrac error functon nstead of a possbly preferable geometrc error functon. It s clamed n ths paper that mnmzng algebrac error wll usually gve excellent results, and n fact the man problem wth most algorthms mnmzng algebrac dstance s that they do not take account of mathematcal constrants that should be mposed on the quantty beng estmated. Ths paper gves an effcent method of mnmzng algebrac dstance whle takng account of the constrants. Ths provdes new algorthms for the problems of resectonng a pnhole camera, computng the fundamental matrx, and computng the tr-focal tensor. Evaluaton results are gven for the resectonng and tr-focal tensor estmaton algorthms. Introducton For many problems related to camera calbraton and scene reconstructon, lnear algorthms are known for solvng for the entty requred. In the sort of problem that wll be addressed n ths paper, a set of data (such as pont correspondences) s used to construct a set of lnear equatons, and soluton of these equatons, usually n the least-squares sense, provdes an estmate of the entty beng computed. As examples of such problems we have :. The DLT algorthm for computng a camera matrxgven a set of ponts n space, and correspondng ponts n the mage. Provded at least correspondences are gve (more precsely 5 correspondences), one can solve for the camera matrx.. Computaton of the Fundamental Matrx. From 8 pont correspondences u u between two mages one can construct the fundamental matrx usng equatons u Fu =. 3. Computaton of the trfocal tensor gven a set of feature correspondences across three vews. These lnear algorthms have been found to gve poor qualty results on occasons and much research has been expended n seekng more relable, but complexmethods. A prevous paper ([]) showed that normalzaton of the data n a systematc manner wll mprove the results mmeasurably, and such normalzaton must be routnely carred out. Nevertheless, a common crtcsm of such lnear algorthms s that they do not mnmze the rght thng, and we really should be mnmzng geometrc error, whch s related to the actual means of error occurrence. Although ths crtcsm s correct, t s the thess of ths paper that t really does not matter very sgnfcantly whether we mnmze algebrac or geometrc error. Usually, one may be content wth mnmzng algebrac error, as long as one enforces the constrants. Ths leads to smpler and more effcent algorthms than are possble mnmzng geometrc error. In these three examples, and many others, the lnear algorthm wll lead to a soluton that does not satsfy certan constrants that the estmated quantty must satsfy. In the cases consdered here, the constrants are. The skew parameter of a camera matrxestmated usng the DLT method wll not generally be zero. Ths constrant, meanng the pxels are rectangular, should be enforced n cases where t s known to hold.. The fundamental matrxmust satsfy a constrant det F =. 3. The trfocal tensor must satsfy 8 non-lnear constrants. The form of these constrants s not easly determned, but t s essental to constran the tensor to correspond to a vald set of camera matrces. These constrants are not n general lnear constrants, and t wll be necessary to resort to teratve

technques to enforce them. Snce teratve technques are slow and potentally unstable, t s mportant to use them sparngly. Further, the smaller the dmenson of the mnmzaton problem, the faster and generally more stable the soluton wll be. In ths paper an teratve algorthm s used to solve the three problems posed above. In each case the algorthms are based on a common technque of data reducton, whereby the nput data s condensed nto a reduced measurement matrx. The sze of the teraton problem s then ndependent of the sze of the nput set. In the case of estmaton of the fundamental matrx, only three homogeneous parameters are used to parametrze the mnmzaton problem, whereas for the trfocal tensor, just sxparameters are used. The problem of camera calbraton solved usng the DLT algorthm wll be treated frst. It wll be used to llustrate the technques that apply to the other problems. Detals of the applcaton of ths technque to the computaton of the fundamental matrxwere gven n [], and are not repeated here. New evaluaton results for trfocal tensor estmaton are gven n ths paper. Computng the Camera Matrx We consder a set of pont correspondences x u between 3D ponts x and mage ponts u. Our problem s to compute a 3 matrxp such that Px = u for each.. The Drect Lnear Transformaton (DLT) algorthm We begn wth a smple lnear algorthm for determnng P gven a set of 3D to D pont correspondences, x u. The correspondence s gven by the equaton u = Px. Note that ths s an equaton nvolvng homogeneous vectors, thus u and Px may dffer by a non-zero scale factor. One may, however wrte the equaton n terms of the vector cross product as u Px =. If the j-th row of the matrx P s denoted by p j, then we may wrte Px = (p x, p x, p 3 x ). Wrtng u =(u,v,w ), the cross product may be gven explctly as u Px = v p 3 x w p x w p x u p 3 x. u p x v p x Snce p j x = x p j for j =,...,3, ths gves a set of three equatons, n the entres of P, whchmaybe wrttenntheform w x v x w x u x v x u x p p p 3 =. () Note that (p, p, p 3 ) whch appears n () s a - vector made up of the entres of the matrx P. Although there are three equatons, only two of them are lnearly ndependent. Thus each pont correspondence gves two equatons n the entres of P. One may choose to omt the thrd equaton, or else nclude all three equatons, whch may sometmes gve a better condtoned set of equatons. In future, we wll assume that only the frst two equatons are used, namely [ w x v x w x u x ] p p p 3 =. () Solvng the Equatons. The equatons () may be denoted by M p =. where the vector p s a - vector, correspondng to the entres of P. The set of all equatons derved from several pont correspondences may be wrtten Mp = where M s the matrx of equaton coeffcents. Ths matrx M wll be called the measurement matrx. The obvous soluton p = s of no nterest to us, so we seek a non-zero soluton.. Data Normalzaton One of the most mportant thngs to do n mplementng an algorthm of ths sort s to prenormalze the data. Ths type of data normalzaton was dscussed n the paper []. Wthout ths normalzaton, all these algorthms are guaranteed to perform extremely poorly. The approprate transformaton s to translate all data ponts so that ther centrod s at the orgn. Then the data should be scaled so that the average dstance of any data pont from the orgn s equal to for mage ponts and 3 for 3D ponts. The algorthms are then carred out wth the normalzed data, and fnal transformatons are appled to the result to compensate for the normalzng transforms..3 Algebrac In the presence of nose, one can not expect to obtan an exact soluton to an overconstraned set of equatons of the form Mp = such as those that arse n the DLT method. The DLT algorthm nstead fnds the unt-norm vector p that mnmzes Mp. The vector ɛ = Mp s the error vector and t s ths error vector that s mnmzed. The soluton s the unt sngular vector correspondng to the smallest sngular value of M. Defne a vector (û, ˆv, ŵ ) = û = Px. Usngths notaton, we may wrte ( ) v ŵ M p = ɛ = w ˆv =. (3) w û u ŵ

Ths vector s the algebrac error vector assocated wth the pont correspondence u x and the camera mappng P. Thus, d alg (u, û ) =(v ŵ w ˆv ) +(w û u ŵ ). () Gven several pont correspondences, the quantty ɛ = Mp s the algebrac error vector for the complete set, and one sees that d alg (u, û ) = Mp = ɛ (5) The man lesson that we want to keep from ths dscusson s : Proposton.. Gven any set of 3D to mage correspondences u x,letm be the measurement matrx as n (). For any camera matrx P the vector Mp s the algebrac error vector, where p s the vector of entres of P.. Geometrc Dstance Under the assumpton that measurement error s confned to mage measurements, and an assumpton of a Gaussan error model for the measurement of D mage coordnates, the optmal estmate for the camera matrx P s the one that mnmzes the error functon d(u, û ) () where d(, ) represents Eucldean dstance n the mage. The quantty d(u, û )sknownasthegeometrc dstance between u and û. Thus the error to be mnmzed s the sum of squares of geometrc dstances between measured and projected ponts. For ponts u =(u,v,w ) and û =(û, ˆv, ŵ ), the geometrc dstance s d(u, û ) = ( (u /w û /ŵ ) +(v /w û /ŵ ) ) / = d alg (u, û )/w ŵ (7) Thus, geometrc dstance s related to, but not qute the same as algebrac dstance..5 The Reduced Measurement Matrx Let u x be a set of correspondences, and let M be the correspondng measurement matrx. Let P be any camera matrx, and let p be the vector contanng ts entres. The algebrac error vector correspondng to P s Mp, and ts norm satsfes Mp = p M Mp. In general, the matrx M may have a very large number of rows. It s possble to replace M by a square matrx ˆM such that Mp = ˆMp for any vector p. Such a matrx ˆM s called a reduced measurement matrx. One way to do ths s usng the Sngular Value Decomposton (SVD). Let M = UDV be the SVD of M, and defne ˆM = DV.Then M M =(VDU )(UDV )=(VD)(DV )=ˆM ˆM as requred. Another (faster) way of obtanng ˆM s to use the QR decomposton M = QˆM, where Q has orthogonal columns and ˆM s upper-trangular and square. Ths shows the followng result. Theorem.. Let u x be a set of n worldtomage correspondences. Let M be the measurement matrx derved from the pont correspondences. Let ˆM be a reduced measurement matrx. Then, for any 3D to D projectve transform P and correspondng 3-vector p, one has d alg (u, Px ) = ˆMp In ths way, all the nformaton we need to keep about the set of matched ponts u x s contaned n the sngle matrx ˆM. If we wsh to mnmze algebrac error as P vares over some restrcted set of transforms, then ths s equvalent to mnmzng the norm of the -vector ˆMp.. Restrcted Camera Mappngs The camera mappng expressed by a general 3D projectve transformaton s n some respects too general. A non-sngular 3 matrxp wth center at a fnte pont may be decomposed as P = K[R Rt] where R s a 3 3 rotaton matrxand K = α u s u α v v. (8) The non-zero entres of K are geometrcally meanngful quanttes, the nternal calbraton parameters of P. A common assumpton s that s =, whle for a true pnhole camera, α u = α v. Gven a set of world to mage correspondences, one may wsh to fnd a matrx P that mnmzes algebrac error, subject to a set of constrants on P. Usually, ths wll requre an teratve soluton. For nstance, suppose we wsh to enforce the constrants s =and α u = α v. One can parametrze the camera matrxusng the remanng 9 parameters ( p u, p v, α plus parameters representng the orentaton R and locaton t of the camera). Let ths set of parameters be denoted collectvely by q. Then, one has a map p = g(q), where p s as before the vector of entres of the matrx P. Accordng to Theorem., mnmzng algebrac

error over all pont matches s equvalent to mnmzng Mg(q). Note that the mappng q Mg(q) s a mappng from R 9 to R. Thssasmpleparametermnmzaton problem that may be solved usng the Levenberg-Marquardt method. The mportant pont to note s the followng : Gven a set of n world-to-mage correspondences, x u, the problem of fndng a constraned camera matrx P that mnmzes the sum of algebrac dstances d alg (u, Px ) reduces to the mnmzaton of a functon R 9 R, ndependent of the number n of correspondences. If ths problem s solved usng the Levenberg- Marquardt (LM) method, then an ntal estmate of the parameters may be obtaned by decomposng a camera matrx P found usng the DLT algorthm. A central step n the LM method s the computaton of the dervatve matrx(jacoban matrx) of the functon beng mnmzed, n ths case Mg(q). Note that Mg/ q = M g/ q. Thus, computaton of the Jacoban reduces to computaton of the Jacoban matrx of g, and subsequent multplcaton by M. Mnmzaton of Mg(q) takes place over all values of the parameters q. Note, however, that f P = K[R Rt]wthKas n (8) then P satsfes the condton p 3+ p 3 +p 3 =, snce these entres are the same as the last row of the rotaton matrx R. Thus, mnmzng Mg(q) wll lead to a matrx P satsfyng the constrants s = and k u = k v andscaledsuchthatp 3 + p 3 + p 3 =, and whch n addton mnmzes the algebrac error for all pont correspondences..7 Expermental Evaluaton Experments were carred out wth synthetc data to evaluate the performance of ths algorthm. The data were created to smulate a standard 35mm camera wth a 35mm focal length lens. A set of ponts were syntheszed nsde a sphere of radus m, and the camera was located at a dstance of about.5m from the center of the sphere. The mage s sampled so that the magnfcaton factors are α u = α v =., the same n each drecton. Ths corresponds to a pxel sze of 35µm for a 35mm camera. Experments were carred out to fnd the camera matrxwth four dfferent assumptons on known camera parameters.. There s no skew : s =. Thenumberofremanng degrees of freedom d for the camera matrxs equal to.. There s no skew and the pxels are square : s = and α u = α v. Ths corresponds to the stuaton for a true pnhole camera where mage coordnates are measured n a Eucldean coordnate frame. In ths case, d =9. 3. In addton to the above assumptons, the prncpal pont (u,v ) s assumed to be known. There reman d = 7 degrees of freedom.. The complete nternal calbraton matrx K n (8) s assumed to be known. However, the pose of the camera s unknown. Thus d =. To evaluate the performance of the algorthm, the result was compared wth the optmal estmate wth dfferent degrees of nose. Thus, Gaussan nose wth a gven varance was added to the mage coordnates of each pont, and the camera matrxwas estmated. The resdual error was then computed, that s the dfference between the measured and projected pxel. It s known (see [5]) that the expected lower bound on the root mean squared resdual error s equal to ɛ opt = σ( d/n) / (9) where σ s the standard devaton of the nput nose, N s the number of measurements (n ths case number of ponts), and d s the number of degrees of freedom of the object beng estmated. Ths represents the performance of an optmal estmaton technque, and we can not do better. Snce the resdual error appeared to grow proportonally to njected nose (at least for nose levels less than about pxels), a value of v =pxelwasusedntheexperments.for each level of nose σ, the camera matrxwas estmated tmes wth dfferent nose. The resdual error was RMS-averaged over all runs and compared to the optmal value. Results of the experments are shown n Fg. The results show that mnmzng algebrac error gves an almost optmal estmate of the camera matrx. In fact, the resdual error s scarcely dstngushable from the optmal value. Ths s true n each of the four calbraton problem types tred. 3 Computaton of the Trfocal Tensor The trfocal tensor ([3, ]), relates the coordnates of ponts or lnes seen n three vews n a smlar way to that n whch the fundamental matrxrelates ponts n two vews. The basc formula relates a pont u n one mage and a par of lnes λ and λ n the other two mages. Provded there s a pont x n space that maps to u n the frst mage and a pont on the lnes λ and λ n

.5 no skew.5 no skew/square pxels.5.5 5 5 5 Number of ponts 5 5 5 Number of ponts 8 8 no skew/square pxels/ known prncpal pont known nternal calbraton 5 5 5 Number of ponts 5 5 5 Number of ponts Fgure : The resdual error was RMS-averaged over runs for each n =,...,5, wheren s the number of ponts used to estmate the camera matrx. Four dfferent levels of knowledge of the nternal camera matrx were tred, correspondng to the four dfferent graphs. In each of the graphs, the sold lne represents the result of our teratve DLT algorthm, and the almost dentcal dotted lne s the optmal estmate. In all four graphs, these two lnes are barely dstngushable. For comparson, the results of a further method are also plotted. In ths method, the complete calbraton matrx K n (8) s estmated usng the QR decomposton, and the known nternal parameters are subsequently set to ther known values. Ths method performs very poorly for small numbers of ponts, lyng well off the graph, and s markedly nferor to the optmal estmaton method even for larger numbers of ponts.

the other two mages, the followng dentty s satsfed : u λ j λ k T jk =. () Here we are usng tensor notaton, n whch a repeated ndexappearng n covarant (lower) and contravarant (upper) postons mples summaton over the range of ndces ( namely,,...,3). Ths equaton may be used to generate equatons gven ether pont or lne correspondences across three mages. In the case of a lne correspondence, λ λ λ one selects two ponts u and u on the lne λ, and for each of these ponts one obtans an equaton of the form (). In the case of a pont correspondence u u u one selects any lnes λ and λ passng through u and u respectvely. Then () provdes one equaton. Four equatons are generated from a sngle 3-vew pont correspondence by choosng two lnes through each of u and u, each par of lnes gvng rse to a sngle equaton. The equatons () gve rse to a set of equatons of the form Mt = n the 7 entres of the trfocal tensor. From these equatons, one may solve for the entres of the tensor. As before, for any tensor T jk the value of Mt s the algebrac error vector assocated wth the nput data. Consder the analogy wth the 8-pont algorthm for computng the fundamental matrxn the twovew case. The fundamental matrxhas a constrant det F = that s not n general precsely satsfed by the soluton found from lnear algorthm. In the case of the trfocal tensor, there are 7 entres n the tensor, but the camera geometry that t encodes has only 8 degrees of freedom. Ths means that the trfocal tensor must satsfy 8 constrants, apart from scale ambguty to make up the 7 degrees of freedom of a general 3 3 3 tensor. The exact form of these constrants s not known precsely. Nevertheless, they must be enforced n order that the trfocal tensor should be well behaved. It wll now be shown how ths can be done, whle mnmzng algebrac error. Formula for the Trfocal Tensor. We denote the three camera matrces P and P by a j and b j respectvely, nstead of by p j and p j. Thus, the three camera matrces P, P and P may be wrtten n the form P =[I ], P =[a j ]andp =[b j ]. In ths notaton, the formula for the entres of the trfocal tensor s : T jk = a j bk a j bk. () Our task wll be to compute a trfocal tensor T jk of ths form from a set of mage correspondences. The tensor computed wll mnmze the algebrac error assocated wth the nput data. The algorthm s smlar to the one gven n [] for computaton of the fundamental matrx. As wth the fundamental matrx, the frst step s the computaton of the eppoles. 3. Retrevng the eppoles We consder the task of retrevng the eppoles from the trfocal tensor. If the frst camera has matrx P =[I ], then the eppoles e and e 3 are the last columns a and b of the two camera matrces P =[a j ] and P =[b j ] respectvely. These two eppoles may easly be computed from the tensor T jk accordng to the followng proposton. Proposton 3.3. For each =,...,3, the matrx T s sngular. Furthermore, the generators of the three left null-spaces have a common perpendcular, the eppole e. Smlarly eppole e 3 s the common perpendcular of the rght null-spaces of the three matrces T. Ths proposton translates easly nto an algorthm for computng the eppoles ([3, ]). Ths algorthm may be appled to the tensor T jk obtaned from the lnear algorthm to obtan a reasonable approxmaton for the eppoles. 3. Constraned Estmaton of the Trfocal Tensor From the form () of the trfocal tensor, t may be seen that once the eppoles e = a j and e 3 = b k are known, the trfocal tensor may be expressed lnearly n terms of the remanng entres of the matrces [a j ] and [b k ]. Assumng the eppoles a j and bk to be known, we may wrte t = Ha where a s the vector of the remanng entres a j and b j, t s the vector of entres of the trfocal tensor, and H s the lnear relatonshp expressed by (). We wsh to mnmze the algebrac error Mt = MHa over all choces of a constraned such that t = Ha =. Wrtng ˆt = Ha where a s the soluton vector, we see that ˆt mnmzes algebrac error Mˆt subject to the condton that T jk s of the correct form (), for the gven choce of eppoles. Thus, fndng the best T jk s reduced to the soluton of a smple mnmzaton problem. Detals of how to solve ths problem are contaned n the next secton. It does not do to mnmze MHa subject to the condton a =, snce a soluton to ths occurs when a s a unt vector n the rght null-space of H. In ths case, Ha =, and hence MHt =.

3.3 A Constraned Mnmzaton Problem We consder the mnmzaton problem : gven matrces M and H, mnmze Mt subject to t =and the addtonal constrant t = Ha for some vector a. Ths problem may be posed n terms of fndng the vector a, n whch case t becomes : mnmze MHa subject to the condton Ha =.Ths problem s solved as follows. Let the Sngular Value Decomposton of H be H = UDV.Letnbetherank of matrx H n the tensor estmaton problem beng consdered, rankh = 5. Then D has n non-zero dagonal entres. We assume that the non-zero entres of D precede the zero entres along the dagonal. Let U be the matrx consstng of the frst n columns of U, letv consst of the frst n columns of V and let D be the top-left n n mnor of D, contanng the non-zero dagonal entres. The mnmzaton problem them becomes : mnmze MU D V a subject to U D V a =. Ths last condton s equvalent to D V a =,snceu has orthogonal columns. Now wrtng t = D V a, the problem becomes : mnmze MU t subject to t =. whch s a standard mnmzaton problem. The soluton t s the sngular vector correspondng to the smallest sngular value of MU. Next, one computes a by solvng t = D V a. A crucal pont here s that D V has rank equal to ts row dmenson, n and t s an n-vector. Therefore, there exsts a soluton to the equatons t = D V a.ifhdoes not have full columnrank, then the soluton for a s not unque. However, one soluton s gven by a = V D t. To see ths, one verfes D V a = D V V D t = D D t = t as requred. Fnally, t = Ha. Note that f the value of a s not specfcally requred, then t may be computed wthout explctly computng a, as follows : Ha = U D V a = U t. The complete algorthm s follows. Algorthm : Gven matrces M and H, fnd the vector t that mnmzes Mt subject to t =and the addtonal condton t = Ha for some vector a. Soluton :. Compute the SVD H = UDV such that the nonzero values of D appear frst down the dagonal.. Let U be the matrxcomprsng the frst n columns of U, wheren s the rank of H. Further, let V consst of the frst n columns of V and D consst of the n frst rows and columns of D. 3. Fnd the unt vector t that mnmzes MU t. Ths s the sngular vector correspondng to the smallest sngular value of MU.. The requred vector t s gven by t = U t, A vector a such that t = Ha s gven by a = V D t. 3. Iteratve Soluton. We have just seen how n terms of algebrac error, the optmal T jk may be computed f the two eppoles are known. The two eppoles used to compute a correct constraned tensor T jk are computed usng the estmate of T jk obtaned from the lnear algorthm. Analogous to the case of the fundamental matrx descrbed n [], the mappng (e, e 3 ) MHa s a mappng R R 7. An applcaton of the Levenberg- Marquardt algorthm to optmze the choce of the eppoles wll result n an optmal (n terms of algebrac error) estmate of the trfocal tensor. Note that the teraton problem s of modest sze, snce only parameters, the homogeneous coordnates of the eppoles, are nvolved n the teraton problem. Snce the functon beng mnmzed s qute complex, dervatves are computed numercally. In general terms, the steps of the estmaton problem are. Compute the reduced measurement matrx M from the gven data.. Obtan an ntal estmate of the eppoles e and e 3 from an ntal estmate of T jk. 3. Compute the matrx H expressng the trfocal tensor (represented by vector t) n terms of the eppoles.. Use the constraned mnmzaton method of secton 3.3 to fnd the best value of t = Ha and compute the algebrac error vector Mt. 5. Iterate these last two steps wth dfferent values of the eppoles, so as to mnmze the error vector. The sze of the teraton problem contrasts wth an teratve estmaton of the optmal trfocal tensor n terms of geometrc error. Ths latter problem would requre estmatng the camera parameters of the three cameras, plus the coordnates of all the ponts a large estmaton problem. 3.5 Expermental Results Once more, the teratve algorthm for computng the trfocal tensor was tested wth synthetc data. The confguraton of the ponts and cameras was smlar to that used for the DLT algorthm, but n ths case there were three cameras amed at the pont set from random angles. Data sets of, 5 and ponts were used to test the algorthm. Resdual errors were compared wth the optmal values of the resdual errors,

5 3 ponts 7 5 3 5 ponts 7 5 3 ponts 8 Nose 8 Nose 8 Nose Fgure : The resdual error RMS-averaged over runs s plotted aganst the number of ponts. Note that these plots are dfferent from the plots for the DLT algorthm n that the horzontal axs represents the nose level, not the number of ponts as n the DLT case. The three plots are for, 5 and ponts. Each graph contans three curves. The top curve s the result of the algebrac error mnmzaton, whereas the lower two curves, actually ndstngushable n the graphs, represent the theoretcal mnmum error, and the error obtaned by teraton to mnmze geometrc error, usng the algebrac mnmzaton as a startng pont ([3, ]). Note that the resdual errors are almost exactly proportonal to added nose. We learn two thngs from ths. Mnmzaton of the algebrac error acheves resdual errors wthn about 5% of the optmal and usng ths estmate as a startng pont for mnmzng geometrc error acheves a vrtually optmal estmate. These results are notably better than results obtaned prevously n []. gven by the formula (9). In ths case, f n s the number of ponts, then the number of measurements s N =n, and the number of degrees of freedom n the fttng s d =8+3n, where 8 represents the number of degrees of freedom of the three cameras (less 5 to account for projectve ambguty) and 3n represents the number of degrees of freedom of n ponts n space. TheresultsareshownnFg. Concluson Expermental evdence backs up the asserton that mnmzng algebrac dstance can usually gve good results at a fracton of the computaton cost assocated wth mnmzng geometrc dstance. The great advantage of the method for mnmzng algebrac error gven n ths paper s that even for problems that need an teratve soluton the sze of the teraton problem s very small. Consequently, the teraton s very rapd and there s reduced rsk of fallng nto a local mnmum, or otherwse falng to converge. The method has been llustrated by applyng t to three problems. For the computaton of the fundamental matrx, teraton s over only three homogeneous parameters. (For detals see [].) For the trfocal tensor, teraton s over parameters. Ths leads to more effcent methods than have been prevously known. The general technque s applcable to problems other than those treated here. It may be appled n a straght-forward manner to estmaton of projectve transformatons between or 3-dmensonal pont sets. In these problems, teraton s necessary f one restrcts the class of avalable transformatons to a subgroup of the projectve group, such as planar homologes conjugates of rotatons. References [] R. I. Hartley. In defence of the 8-pont algorthm. In Proc. Internatonal Conference on Computer Vson, pages 7, 995. [] R. I. Hartley. A lnear method for reconstructon from les and ponts. In Proc. Internatonal Conference on Computer Vson, pages 88 887, 995. [3] Rchard I. Hartley. Lnes and ponts n three vews and the trfocal tensor. Internatonal Journal of Computer Vson, ():5, March 997. [] Rchard I. Hartley. Mnmzng algebrac dstance. In Proc. DARPA Image Understandng Workshop, pages 3 37, 997. [5] K. Kanatan. Statstcal Optmzaton for Geometrc Computaton : Theory and Practce. North Holland, Amsterdam, 99.