Takeshi Kurata Jun Fujiki Katsuhiko Sakaue. 1{1{4 Umezono, Tsukuba-shi, Ibaraki , JAPAN. fkurata, fujiki,

Transcription

1 Ane Eiolar Geometry via Fatorization Method akeshi Kurata Jun Fujiki Katsuhiko Sakaue Eletrotehnial Laboratory {{4 Umezono, sukuba-shi, Ibaraki , JAPAN fkurata, fujiki, Abstrat We resent the intuitive interretation of ane eiolar geometry for the orthograhi, saled orthograhi, and araersetive rojetion models in terms of the fatorization method for the generalized ane rojetion (GAP) model roosed by Fujiki and Kurata (997). Using the GAP model introdued by Mundy and Zisserman (99), eah ane rojetion model an be resolved into the orthograhi rojetion model by the introdution of virtual image lanes, then the ane eiolar geometry an be simly obtained from the estimates of the fatorization method.. Introdution In numerous works on struture-from-motion, the fatorization method [][] is advantageous due to its stability in terms of numerial omutation and its ability to evenly maniulate the overall image and to simlify the formulation. For these reasons, this method has been extended to various aims, for examle, for the segmentation of lural objets [7], from oint features to line features [8], and for the ersetive rojetion model [6]. We know that the estimates of shae and motion obtained by the rst ste of the fatorization method [][] already have an ane struture and they are desribed by an ane oordinate system without a unit matrix as the metri. We have also laried that the seond ste is to determine the metri of the ane oordinate system in order to re-desribe the above estimates by a oordinate system with a unit matrix as the metri. Based on this interretation, we have also demonstrated that the Eulidean solutions for shae and motion an be reovered from two ane images with one degree of freedom remaining [9][3]. wo images an be resolved into the orthograhi rojetion, then the remaining one degree of freedom is the same as the rotation between image lanes that is a seond element of the Euler angle reresentation for three-dimensional rotations [5][0]. Varying this angle, the trajetory on an image of a D oint that is the rojetion of a 3D oint in the sene beomes a straight line. his is referred to as an "eiolar line", and in the ane rojetion model, all eiolar lines are arallel with one another due to arallelism. In this aer, the ane eiolar geometry for the orthograhi, saled orthograhi, and araersetive rojetion model is examined to enable intuitive interretation by using the fatorization method for the generalized ane rojetion (GAP) model [0][4].. he Fatorization Method for the GAP Model For simliation, this aer assumes that an otial axis is orthogonal to an image lane, their intersetion is known, the aset ratio of a ixel is :, and skew is absent... Desrition Using the GAP Model For the GAP model [0][4], the relation between the -th 3D oint in the world oordinate system s ( = :: P ) and the rojetion on the f-th image x f = (x f y f ) is written as follows: x f = A f C f (3) (33) s + f f A f =( f f ) () where A f is alled the ane model matrix, C f (= (i f j f k f ) ) is alled the basis matrix having the normal orthogonal basis of an image 5 f, and f f is a fator related to translation. he enter of gravity g of whole s is rojeted on x f: =(x f y f ), and the oordinates of the s relative to g are denoted by s 0, then x 0 f = x f 0 x f: = A f C f s 0 : ()

2 For S 0 (= (s 0 :: s 0 P )), this is rewritten as follows: W 0 f =(x 0 f :: x 0 fp)=m f S 0 M f = A f C f =(m f n f ) : (3) herefore W 0 (= (W 0 :: W 0 F ) ), alled the registered measurement matrix, an be fatorized into the rodut of a matrix M(= (M :: M F ) ) reresenting the relative motion between a amera and an objet and a matrix S 0 reresenting the 3D shae of an objet. W 0 (F P ) = M.. Metri Constraints S 0 (F 3) (3P ) (4) A Constraint that an be used to reover Eulidean motion and shae from W 0 is C f C f =I 3. herefore the following onstraint is obtained with regard to eah ane rojetion model. M f M f = A f A f : (5) It is, however, diult to deomose W 0 so as to satisfy this onstraint at one, so it is normally deomosed in two stes. In the rst ste, W 0 is deomosed as follows by SVD and so on and by the onstraint that the rank of W 0 is 3. W 0 = ^M ^S0 : (F P ) (F 3) (3P ) Arbitrary 3 3 regular matrix A an be inserted suh as W 0 = ^MAA 0 ^S0. hus, the seond ste uses (5) to uniquely determine A, that is, to uniquely reover M and S. Using the notation ^M =(^M :: ^Mf ) ^Mf =(^m f ^n f ) and Q = AA, then the following onstraint is derived from (5). ^M f Q ^Mf = A f A f : (6) his is a onstraint for a length on a linear sae that uses a ositive symmetri matrix Q as a metri matrix. hat is, obtaining the reovered solution is equivalent to obtaining the metri of this linear sae. Sei arameters for the orthograhi, saled orthograhi, and araersetive rojetion models an be obtained using Aendies A, B, and C. Central symmetry is not disussed in this aer. If four oints that are not olanar are observed in three essentially dierent images, Q and thus A an be determined from this onstraint [9]. 3. Interretation of Ane Eiolar Geometry Using the Fatorization Method he satial relation between two orthograhi rojetion images is resented by means of the Euler angle reresentation for 3D rotations using R z R y R z. he one degree of freedom remaining after 3D reovery from two images for eah ane rojetion model orresonds to the rotation R y between two image lanes obtained when the two images are resolved into the orthograhi rojetion [9][0]. Although this is already known from works on ane eiolar geometry [5], this aer examines it in terms of the fatorization method, and exliitly derives the other rotations and the eiolar onstraint. 3.. Relative Rotation between wo Images and Euler Angle Reresentation o determine the satial relation between an objet and an f-th image 5 f,weintrodue a virtual image lane f on whih fv f w f g are the normal orthogonal basis. Let A f = R f 6 f D f be the result obtained by SVD of an ane model matrix A f and let V f = D f C f =(v f w f ) =6 0 f R f M f ^V f = ^Df C f =(^v f ^w f ) =6 0 f R f ^Mf : (7) Sine A f A f = R f 6 f R f, (5) and (6) an be rewritten as (8) and (9) resetively. V f V f =I (8), ^Vf Q ^Vf =I (9). (9) means that if a lane ^ f dened by f^v f ^w f g is loated in a linear sae that uses Q as a metri matrix, f^v f ^w f g onstitutes a normal orthogonal basis. R f transforms an ellise resented by a quadrati form of a sub-metri of Q into its anonial form, and 6 0 f transforms the ellise into a irle. From (3) and (7) we obtain the following equation: W 0 f = M f S 0 = R f 6 f V f S 0 : (0) his equation and (8) indiate that S 0 is orthograhially rojeted on the virtual image lane f that uses fv f w f g as a normal orthogonal basis and that the rojetion is transformed by R f 6 f into an image obtained by eah ane rojetion model (Fig.) Seond Element of the Euler Angle he fatorization method for two images [9] uses (9) as a onstraint for ^ and ^ (Generality will not

3 ^ ^ ^ ^ ^ ^ ^ m n v w a^ ^ v b w Σ R R( ψ ) R( ψ ) Q Q Q Q Q θ π π π π π m n v w a b v w Σ R R( ψ ) R( ψ ) Σ R Σ R ^ ^ m n Q π m n Figure : Virtual Image Planes and the Bases Using the notation G() =(a b ) and using (7), () an be rewritten as follows: (a ) = R( )V (b ) = R( )V : (3) herefore, we an obtain a rotation R( ) by whih fv w g overlas with fa g on and a rotation R( ) by whih fv w g overlas with fb g on (Fig.). be lost if two images used for eiolar onstraint are seied as st and nd images). he onstraint an be rewritten as follows: j^v f j Q = j ^w f j Q = h^v f ^w f i Q =0 (f = ): Q has six degrees of freedom, but these onstraints are not indeendent of one another, so we rewrite them into ve indeendent onstraints one again. j^aj Q = j^bj Q = j^j Q = h^b ^i Q = h^ ^ai Q =0 () where ^ ^ \ ^, and ^a and ^b are on ^ and ^ resetively. Sine h^a ^bi Q, the remaining one degree of freedom an be given as h^a ^bi Q = os, and the angle between and an be indiated by. Using the notation P =(^a ^b ^), 0 G() os( 0 4 ) 0 sin( 0 4 ) 0 0 sin( 0 4 ) os( 0 4 ) 0 A 0 0 M() = G()P 0 ^M S 0 () =G() 0 P ^S0 then the deomosition of W 0 inluding this degree of freedom an be written as W 0 = M()S 0 () Reovery of the First and hird Elements of the Euler Angle Comutation for ^a, ^b, and ^ does not require Q. First, ^ an be written as ^ = ^v + ^w = ^v + ^w, and let ^v f ^w f = r f, then ^ is obtained by solving the following equations. = = 6 0r ^w (r ^w ) +(r ^v ) r ^v 6 0r ^w (r ^w ) +(r ^v ) r ^v : he signs of,,, and are seleted to make the diretions of ^ the same. ^a and ^b satisfy ^a = ^v 0 ^w and ^b = ^v 0 ^w resetively. In summary, (^a ^) = R( ) ^V (^b ^) = R( ) ^V R( f )= f 0 f f f () ψ θ v w w v a b ψ π π Figure : R( ) and R( ) 3.. Eiolar Constraint s π π a b affine eiolar lanes affine eiolar line Figure 3: Ane Eiolar Constraint From (7)(3), the following transformations an be obtained. (a ) = R( )6 0 R M (b ) = R( )6 0 R M : (4) Points at whih s 0 is orthograhially rojeted on and resetively are resented as (a ) s 0, (b ) s 0. Sine x 0 f = M f s 0, the following equations an be derived from (4). (a ) s 0 = R( )6 0 R x 0 (b ) s 0 = R( )6 0 R x 0 : s 0 must be equivalenton and, so the following eiolar onstraint an be obtained (Fig.3). (0 )R( )6 0 R x 0 =(0 )R( )6 0 R x 0 : (5) Here, we summarize the algorithm to obtain the ane eiolar onstraint. ) Deomose W 0 into ^M and ^S0. ) Comute R f and 6 f of two objetive images using Aendix A, B, or C. 3) Comute f^v f ^w f g using (7). 4) Comute R( f ) aording to setion ) Obtain the eiolar onstraint using (5).

4 4. Exeriments on Eiolar Constraint For the exeriments with syntheti data, 0 oints were randomly and uniformly distributed within a ube of 00 mm sides. hey were rojeted on image lanes by a in hole amera. he intrinsi arameters were known ixels were square of side 8m long the image resolution was and the foal length l was 8or6mm. For the external arameters, the angle between 5 and 5 was 50 and the translation of the amera were denoted by (t xf t yf t zf )(f = ). ables and show the onditions and the results of exeriments with 00 dierent data sets resetively. For the evaluation of the exeriments, we omuted the average distane between eah oint and the orresonding eiolar line. ests (a) and (b) used the same ratio of the foal length to the distane as far as the objet in order to make rojeted images of the objet almost the same, so we an evaluate the eet of ersetive rojetion. Sine test (a) had a larger angle of view and is affeted by ersetive rojetion, it had a larger distane between the oint and the eiolar line. est 6 was assumed to be advantageous to the araersetive rojetion model, but the results exhibited little dierene between the saled orthograhi rojetion and araersetive rojetion. Figs.4-7 show one examle of exeriments on dense reovery with real images []. 64 feature oints were automatially deteted at rst image and traked for 0 images in real time (5 images/se.). We manually seleted 7 oints out of them on the fae and erformed the fatorization method for the saled orthograhi model to omute the metri (As you know, there exists entral symmetry. Here we deided one of them manually). In order to obtain the dense shae, the ane eiolar onstraint for the saled orthograhi model on st and 4th images was omuted aording to setion 3.. Fig.5 shows the reovered ane eiolar lines for the fae. In this ase the distane between eah oint and the orresonding eiolar line was 0. ixel on average. Fig.6 shows images retied to make eiolar lines horizontal and shows the disarity ma omuted by a simle orrelation method. hese retied images are equivalent to virtual image lanes f. Fig.7 shows two views of the reovered shae. 5. Conlusions We have formulated the ane eiolar onstraint in the orthograhi, saled orthograhi, and araersetive rojetion models in terms of the fatorization method using the GAP model. he simulations show the ane eiolar onstraint an be well alied if the ersetive eet is small so we an use the onstraint to obtain rotation axes, dense disarity mas, and so on, even though exerimental results in Fig.7 is not aurate for 3D modelling beause of the oor orrelation method. Future work we would like to do inludes investigating the aliation of our method for visual reognition system. Aknowledgments: his work was onduted as a art of the Real World Comuting (RWC) Program. able : Camera arameters for simulations (unit: mm) test t x t y t z t x t y t z l (a) (a) (a) (a) (a) (a) (b) (b) (b) (b) (b) (b) able : Distane between oints and the eiolar lines (unit: ixel) (a) (b) ortho saled ara ortho saled ara Referenes [] C. omasi and. Kanade. Shae and Motion from Image Streams under Orthograhy: A Fatorization Method. IJCV, 9():37-54, 99. [] C. J. Poelman and. Kanade. A Paraersetive Fatorization Method for Shae and Motion Reovery. PAMI, 9(3):06-8, 997. [3] J. J. Koenderink and A. J. van Doorn. Ane Struture from Motion. J. Ot. So. Am. A, 8(): , 99. [4] J. L. Mundy and A. Zisserman. Geometri Invariane from Motion. MI Press, Cambridge, Mass., 99. [5] L. S. Shairo, A. Zisserman, and M. Brady. 3D Motion Reovery via Ane Eiolar Geometry. IJCV, 6():47-8, 995. [6]. Ueshiba and F. omita. A Fatorization Method for Projetive and Eulidean Reonstrution from Multile Persetive Views via Iterative Deth Estimation. In Pro. 5th ECCV, 998.

5 [7] J. Costeira and. Kanade. A Multi-body Fatorization Method for Motion Analysis. In Pro. 5th ICCV, , 995. [8] L. Quan and. Kanade. A Fatorization Method for Ane Struture from Line Corresondenes. In Pro. CVPR96, , 996. [9] J. Fujiki and. Kurata. Metri of Ane Shae and Motion: he Intuitive Interretation in erms of he Fatorization Method. In Pro. SPIE97, 368:06-7, 997. [0] J. Fujiki and. Kurata. An Mathematial Analysis of Fatorization Method for Generalized Ane Projetion Model. ehnial Reort, EL, Jaan, 998(to aear). []. Kurata, J. Fujiki, and K.Sakaue. Ane Eiolar Geometry via Fatorization Method and Its Aliation. In Pro. SPIE98, Vol. 3457, 998. where u f = x + f y f. he ratio of z to z an be determined in the same way as the saled orthograhi model by using the new basis transformed into the saled orthograhi rojetion model that is ( ^m ^n )R and (^m ^n )R. Figure 4: (left) st image, (right) 0th image A. Orthograhi Projetion A f = R f =I 6 f =I : B. Saled Orthograhi Projetion A f = l 0 0 z f 0 0 z f = R f =I 6 f = l z f I l jm = l f j jn f j where z f is the average deth of oints and l is a foal length. fz f g F f = an be determined only in the form of a ratio. r z = + + z whih is derived from Figure 5: Ane eiolar lines for the fae Figure 6: (left)(enter) Virtual image lanes, (right) Disarity ma ^M f Q ^M f = l I = ^m + ^n = ^m + ^n : z f he ratio of z f an be obtained from the following linear equations. (^r ^m )+ (^r ^n )=0 (^r ^m )+ (^r ^n )=0 (^m ^n )= (^m ^n ) where ^r = ^m ^n,^r = ^m ^n, and ^m = ^m ^m. C. Paraersetive Projetion A f = l 0 0x f z f 0 l 0y f R f = u f x f 0y f y f x f z f = l + x f jm f j 6 f = z f l + u f 0 0 l l r + y f xf y f = = jn f j hm f n f i Figure 7: Reovered shae of the fae, (left) with shading, (right) with texture