Equivalent Constraints for Two-View Geometry: Pose Solution/Pure Rotation Identification and 3D Reconstruction

Transcription

1 IEEE RANSACIONS ON PAERN ANALYSIS AND MACHINE INELLIGENCE Equvalen Consrans for wo-vew Geoery: Pose Soluon/Pure Roaon Idenfcaon and D Reconsrucon Ca, Yuanxn Wu, Llan Zhang and Peke Zhang Absrac wo-vew relave pose esaon and srucure reconsrucon s a classcal proble n copuer vson. he ypcal ehods usually eploy he sngular value decoposon of he essenal arx o ge ulple soluons of he relave pose, fro whch he rgh soluon s pcked ou by reconsrucng he hree-denson (D) feaure pons and posng he consran of posve deph. hs paper revss he wo-vew geoery proble and dscovers ha he wo-vew agng geoery s equvalenly governed by a Par of new Pose-Only (PPO) consrans: he sae-sde consran and he nersecon consran. Fro he perspecve of solvng equaon, he coplee pose soluons of he essenal arx are explcly derved and we rgorously prove ha he orenaon par of he pose can sll be recovered n he case of pure roaon. he PPO consrans are splfed and forulaed n he for of nequales o drecly denfy he rgh pose soluon wh no need of D reconsrucon and he D reconsrucon can be analycally acheved fro he denfed rgh pose. Furherore, he nersecon nequaly also enables a robus creron for pure roaon denfcaon. Experen resuls valdae he correcness of analyses and he robusness of he derved pose soluon/pure roaon denfcaon and analycal D reconsrucon. Index ers Relave Pose, Coplanar Relaonshp, Sae-sde Consran, Inersecon Consran, Pure Roaon INRODUCION I s well known ha he relave pose and D pons can be generally recovered up o a scale fro wo vews. he relaonshp of he age pon pars n wo vews s well descrbed by wo-vew agng geoery n Fg.. For he uncalbraed caera, s specfed as he fundaenal arx whose properes have been suded by Beardsley and Zsseran [] and Vevlle and Lngraud []. Harley [, 4] proposes algorhs for uncalbraed caera pose esaon. For he calbraed caera, he essenal arx was frs nroduced o he copuer vson feld by Longue- Hggens [5], whch provdes a way o ge he soluon o he relave pose. Huang, Faugeras, Maybank and Harley [6-9] have exensvely suded he properes of he essenal arx. he essenal arx s subjec o he consran EE E r EE E/ 0 [6], whch can be used n degenerae cases wh less han egh pon pars. he degenerae cases are solved by Huang & Sh [0], Maybank [8], Nser [] and Sewenus []. Usually he essenal arx lnearly esaed usng he noralzed pons would no Ca and Peke Zhang are wh Shangha Key Laboraory of Navgaon and Locaon-based Servces, School of Elecronc Inforaon and Elecrcal Engneerng, Shangha Jao ong Unversy, Shangha, Chna, 0040; and School of Aeronaucs and Asronaucs, Cenral Souh Unversy, Changsha, Hunan, Chna, E-al: {qcacn@gal.co; zhangpeke@csu.edu.cn}. Yuanxn Wu s wh Shangha Key Laboraory of Navgaon and Locaonbased Servces, School of Elecronc Inforaon and Elecrcal Engneerng, Shangha Jao ong Unversy, Shangha, Chna, E-al: yuanx_wu@hoal.co. Llan Zhang s wh College of Mecharoncs and Auoaon, Naonal Unversy of Defense echnology, Changsha, Chna, E-al: llanzhang@nud.edu.cn. be he opu, and here exs nonlnear ehods o refne he pose o fulfll he above consran, as descrbed n [-5]. When a proper essenal arx s acheved, can be used o ge ulple soluons o he caera relave pose. he popular ehod s based on he sngular value decoposon (SVD) of he essenal arx, whch orgnaed fro he proof n Huang [7]. he SVD ehod was furher developed and suarzed n Harley [9, ] and Wang [6]. he soluons are used o reconsruc he D pons by rangulaon and s only he rgh soluon ha yelds posve deph [5, 7]. I has been beleved ha he SVD ehod requres non-zero ranslaon [8, 9]. Recenly, Knep [0] pus forward a new nonlnear roaon consran ndependen of ranslaon and uses he Grobner bass ehod o solve he roaon for he 5-pon case. ha work also rases a new nequaly o dsabguae he roaon arx. hs paper s ovaed by answerng he followng hree quesons:. How o explan he ulple pose soluons o he essenal arx equaon fro he aspec of solvng equaon?. How o explan he experen phenoenon ha he roaon par can sll be recovered fro he essenal arx equaon n he pure roaon case?. Is possble o drecly denfy he rgh pose soluon whou D reconsrucon? he an conrbuon of he paper s ulple-fold: ) he wo-vew agng geoery s found o be equvalenly governed by a Par of Pose-Only (PPO) consrans: he sae-sde consran and he nersecon consran; ) he basc wo-vew agng equaon s for he frs e forulaed as a funcon of pure pose, ndependen of D feaure pon coordnaes. he relave deph nforaon

2 . CAI E AL.: EUIVALEN CONSRAINS FOR WO-VIEW GEOMERY s oally paraeerzed by relave pose; ) he coplee pose soluons o he essenal arx equaon are explcly derved, whch explans ha he orenaon can sll be recovered n he pure roaon case; 4) Aded by wo nequales derved fro he sae-sde and nersecon consrans, he rgh pose soluon can be provably denfed whou resorng o D reconsrucon. In oher words, he pose denfcaon s decoupled fro D reconsrucon, whle D reconsrucon can be analycally acheved fro he denfed rgh pose; 5) A robus creron of pure roaon denfcaon s derved fro he nersecon nequaly. he paper s organzed as follows. Secon rases he PPO (sae-sde and nersecon) consrans and proves her equvalency o he basc wo-vew agng geoery. Secon derves he pose soluons o he essenal equaon. Secon 4 splfes he PPO consrans yeldng he sae-sde nequaly and he nersecon nequaly o help denfy he rgh pose soluon whou D reconsrucon of feaure pons. A pure roaon denfcaon ehod s also derved fro he nersecon nequaly. Secon 5 repors he experen resuls for pose soluon/pure roaon denfcaon and analycal reconsrucon. Secon 6 concludes he paper. A nuber of frequenly-used equales of arces, vecors and producs are presened n Appendx for easy reference. WO-VIEW IMAGING GEOMERY AND EUIVALEN CONSRAINS. wo-vew Geoery he wo-vew agng geoery s llusraed n Fg.. P X x, y,,, and Suppose he pon se he correspondng pon se X P' x, y,,, are he ses of noralzed age pon pars, whle P { x, y, z X x, y, z,, } X or w w w w w w w w w are he projeced D world feaure pons. hroughou he paper, we use he noralzed age coordnaes, unless explcly saed oherwse. he world pon coordnaes n he wo caera fraes, cenered a C and C respecvely, are characerzed by X RX + () w R & Fg. Skech of wo-vew agng geoery. w where R and are respecvely he rue roaon arx and ranslaon vecor beween he wo caera fraes. I C s readly apparen ha he wo noralzed age pon ses are relaed by z X z RX + () w w whch can be rewren as he wo-vew agng equaon X RX s () where he deph-relaed facors zw zw and s zw. he equaon s poran o alos all geoerc copuer vson probles. Lef ulplyng () by X reoves he deph-relaed facors [8] 0 X X X RX (4) whch s rgh he well-known essenal arx equaon [] X EX 0 (5) where E R and s he skew-syerc arx fored by. Noe ha he developen fro () o (5) s no reversble, ha s o say, soe nforaon has been los as wll be shown below. If he ranslaon vecor 0, he essenal arx E reduces o zero or arguably s no well defned, bu fro he vewpon of equaon, (5) sll exss wh non-zero soluons, as dscussed n nex secon. Referrng o Fg., (5) depcs he co-planar relaonshp of he projecon rays of he wo vews for he -h pon par ( X and RX ) and he ranslaon vecor ( ). Noe ha ebeds he agng geoery of wo vews, bu loses he poran agng relaonshp aong he ranslaon vecor and wo projecon rays. For exaple, he wo projecon rays always pon o he sae sde of he ranslaon vecor [0], as shown n Fg.. However, he obaned benef of he coplanar relaonshp (5) s ha he deph nforaon ( and s ) s solaed fro pose and s lnear n E whch can be exploed o ge pose soluons by lnear ehods lke SVD. Le ulplyng () by X leads o 0 X RX s X RX (6) X RX 0 I eans all vecors { X' RX,..., } le on a plane wh he noral vecor. Denoe BX,, RX XRX, (6) s equvalen o rank B. hs s exacly he roaon consran ndependen of ranslaon proposed by [0].. Equvalen PPO Consrans Nex we wll presen he PPO consrans governng he physcal foraon of wo overlapped vews, naely, he sae-sde consran and he nersecon consran (see Fg. ). Defne, and o denoe he angles aong he ranslaon vecor and he projecon rays, and 0,,,,. Hereafer n hs secon, he ranslaon vecor s dvded no he un-drecon par and he agnude par by e, where he un vecor e s he

3 . CAI E AL.: EUIVALEN CONSRAINS FOR WO-VIEW GEOMERY drecon vecor of. Noe ha for zero ranslaon, 0 and e s an arbrary un vecor. Sae-sde Consran. he projecon rays should pon o he sae sde of he ranslaon vecor, ha s o say, he vecor producs of RX and X are n he sae drecon. he sae-sde consran can be aheacally represened as RX X R X X' (7) e RX e X e RX e X he frs equaly s no well-defned for he zero-ranslaon case. For zero ranslaon, =0 and =, and as a resul (7) reduces o e RX e X RX X e 0 R R (8) X X X X whch eans RX RX = X X, as e s an arbrary un vecor. In oher words, he projecon rays X and RX are collnear n he sae drecon, so he second equaly of (7) s a consran for he sae drecon as well. ha s o say, all ranslaon cases are well defned by he second equaly of (7). Inersecon Consran. he projecon rays nersec wh each oher only f = s sasfed. he hree angles n Fg. are obvously relaed o he vecor nner producs by X RX X RX cos,cos,cos (9) X X X X Noe he arccosne funcon s value feld s 0,. akng cosne on boh sdes, he geoerc consran = s equvalen o cos coscos snsn s.. (0) Wh (9), he equaly of (0) can be rewren as RX X X RX X X RX () X X X X X X X he nequaly consran of (0) s equvalen o cos cos, and we have X X RX 0 () X X X Collecvely, he nersecon consran s rewren as RX X X RX X X RX X X X X X X X () X X RX 0 X X X or alernavely, X e X RX X RXX e X X R e (4) X X e X RX 0 Slarly for he zero-ranslaon case, () s no well-defned. In such a case, however, ha he nequaly n (4) s always rue for any e eans X RX ax X X e ax X X cos X X. As X RX X X, we can derve ha X RX X X. hs happens only when he projecon rays X and RX are collnear n he sae drecon,.e., 0. I shows ha he nequaly n (4) also gves a consran for he sae drecon. We can easly verfy ha he equaly n (4) becoes deny when oon s pure roaon. I s exacly wha he nersecon consran eans for he zero-ranslaon case. ha s o say, all ranslaon cases are well defned by (4). Proposon : he wo-vew agng equaon s equvalen o he cobnaon of he sae-sde and nersecon consrans, naely, (7)+(4) (). Proof of Suffcency: he sae-sde consran can be rewren as h e wrx e X (5) where w e X e RX. Defne qhe, X can be expressed as a lnear cobnaon of and q X' = a e +bq (6) Wh (5), q=he e wrx e wr w R X X e e (7) Fro he second lne o he hrd lne, he vecor produc equaly for any rple vecors s used (see Appendx). Subsung (6) no (5) gves h e a e +bq be q bh b (8) Consderng (6)-(8), we have X = a e+ q ae wrx w X R e e X X e e wr a w R Conrasng () and (9), we wll prove ha Fg. wo new geoerc consrans: saesde consran and nersecon consran. (9) w and awx R e s. As for w, e X z e w X w zw w = = (0) e RX z' w e RXw z' w where e Xw and e RX w boh descrbe he dsance fro he D pon o he ranslaon baselne. In order o prove awx R e s, we us ake no

4 . CAI E AL.: EUIVALEN CONSRAINS FOR WO-VIEW GEOMERY 4 accoun of he nersecon consran. Defne he lef sde of he equaly n (4) as w e X X RX. Subsung (6) and (8) no he equaly n (4), we have w ae R Xqe +aqr X+ qrxqe () ae qx R e q qx R e Consderng (7), we have qe 0 So, () reduces o qq= w X X w X R e q RX wx X w X R e () w awx R e w X X X R e () Defne w w R X X X e, hen we have w X e (4) w a w R = Subsung no (9), we ge n whch w w X wrx e (5) ww e X XRX ww w X X X R e X X snsn = w X X cos X snsn w X sn (6) Consderng he relaonshp beween noralzed age pons and he world coordnaes of D pons defned n () and (), he above equaon becoes w z X w snsn w ww w z' w X w sn z X w sn e X w sn w (7) w z' w sn e RXw sn z X' w sn w e X' w w z' sn e RX w w As X w sn and sn boh descrbe he dsance fro C o he projecon ray X and by usng (0), can be furher reduced o w z' w zw = s (8) ww zw z' w Cobng (0) and (6), (5) yelds X RX s e = RX s (9) Proof of Necessy: Subsue () no he second equaly of (7), e X e RX s = e RX e X e RX s e RX Slarly subsue () no he equaly n (4), X RX X e X X R e RX s RX RX s e RX s X R e = (0) () Expandng he above equaon yelds s X X e s X R X R e s X X R e s RXe = RX s e RX se RX = Xe XRX Subsue () no he nequaly n (4), X X e X RX RX s X e RX X X e RX s X e RX 0.E.D. () () Proposon : he sae-sde consran ples he co-planar relaonshp. Proof. Lef ulply X on boh sdes of he second equaly of (7) e RX e X X X e RX e X (4) e RX X E X X e X=0 e X'.E.D. Proposon : he wo-vew agng equaon () can be rewren as e X XRX X = RX e (5) e RX e RX and he relave raos of deph z e w X zw and X RX Proof. Fro (0), = e X e RX e RX X RX Accordng o (0) and (6), w e X XRX s ww w X X X R e R R ' R e X e X X X e X e X X RX e X (6) (7) (8) Subsue (7)and (8) no (), e R X X X X RX e R ' X e X (9) e R X e X X RX X RX XRX Coparng he above equaon wh () by coeffcens, we have

5 . CAI E AL.: EUIVALEN CONSRAINS FOR WO-VIEW GEOMERY 5.E.D. e z X e w X zw = = XRX XRX e RX z e R w X zw = = XRX XRX (40) Proposon 4: For all D pons oher han hose on he baselne, X RX =0 f and only f zw and zw. Proof of Suffcency: For he -h D pon, z w zw =+ and = (4) Consderng (6), resuls n X RX =0, because he nueraors e X and e RX are nonzeros for all D pons oher han hose on he baselne. Proof of Necessy: If X RX =0, (6) ples (4)..E.D. Proposon 5: Pure roaon leads o X RX =0 for all pon pars, and an nfne D pon or a pon on he ranslaon baselne only leads o X RX =0 for he correspondng pon par. Proof. Fro Proposon 4, for he case of pure roaon oon, =0 akes every pon par sasfy X RX =0. Slarly, for an nfne D pon, zw and z' w, he correspondng pon par sasfes X RX =0. And for a projeced pon on he baselne, RX are collnear so ha X RX =0..E.D. Proposon shows ha he wo-vew agng equaon () s equvalen o he PPO consrans. In Secon 4, we wll see ha he par of consrans can be used o pck he rgh pose soluon ou of he ulple soluons obaned fro (5). In Secon 5, Proposon wll be used o analycally reconsruc he D pon, and Proposon 4 wll be used o dscrnae nfne pons or pure roaon oon. COMPLEE POSE SOLUIONS O ESSENIAL MARIX EUAION he os well-known pose decoposon fro he essenal arx s he SVD ehods by Harley[] and ohers. Bu hese ehods are acually based on he plc assupon of 0, naely, he essenal arx E R s well defned, so here exss an ncorrec belevng ha n pure roaon case he essenal arx equaon canno be used o copue he roaon [8, 9]. hs secon s devoed o dervng all pose soluons o he essenal equaon (5) and fundaenally explanng he phenoenon ha he orenaon can be recovered fro he case of pure roaon, whch s ncorrecly nerpreed by [8] as he resul of spurous ranslaon caused by age coordnae nose. As a splfcaon of he wovew agng equaon, (5) gh produce soluons ha do no sasfy (). We frs dscuss he essenal arx soluons o he essenal arx equaon (5) and hen he pose soluons decoposed fro he essenal arx soluons.. Essenal Marx Equaon Soluons In order o reduce he confuson, (5) s re-wren as vec X'X 0 X X 0 (4) where denoes he Kronecker produc and vec s he vecorzaon of a arx (see Appendx). he popular ehod o solve vec s he lnear ehods by resrcng vec o have uny lengh [], n whch he case of pure roaon has no been well consdered. Consderng (), we have s I R vec X X 0 (4) n whch s oed as 0. hen for he whole pon se, we have X s X I R vec 0 (44) X s X Le X s X L, y I R vec (45) X s X Expandng he arx L x xy x xs yx y y ys x y s L x xy x xs yx y y ys x y s and (44) s equvalen o (46) Ly 0 (47) As coluns, and 7 are respecvely equal o 5, 9 and 0, we have rank L 9. Le A L I R, we have A and (44) can be wren as Avec 0 rank 9. herefore, when 9 he hoogeneous equaon (47) has hree lnearly ndependen specal soluons ξ ξ ξ So he soluon space of y s gven by I R veca a a (48) y= ξ ξ ξ (49) where a, a, a are real nubers. By usng he Kronecker

6 . CAI E AL.: EUIVALEN CONSRAINS FOR WO-VIEW GEOMERY 6 produc equaly [] I R vec vec R (50) (49) can be descrbed n he for of arx as R 0 a a R a 0 a a a (5) hen we ge wo equaons for 0 a a R a 0 a Ra a a 0 (5) 0 (5) where a, a, a a and a can be deerned whou ranslaon nforaon fro (5). Equaon (5) ndcaes ha he soluons o (4) are defnely essenal arces. For 0, s obvous ha s sll relaed o R, bu s connecon o would be gone. I can be derved fro (5) ha I R vecveca whch eans a a a a vec (54) as he arx I R s orhogonal.. Coplee Pose Soluons Accordng o (5), and a have he sae sngular values. By SVD, he skew-syerc arx can be deco- a = U ΣV, where U, V are orhogonal ar- posed as ces and Σ dag a, a,0 []. hen can be wren as where de U V de U V U ABLE I. VALIDIY OF ROAION MARIX Σ Σ R a RU V U V (55) RU and V V. Noe ha he SVD decoposon s no unque. Acually, o ge R and, here s no need o ge he exac for of V. As a s a skew-syerc arx, one of he sngular values s equal o zero and he correspondng sngular value vecor s a []. Le U u, u, u and V v, v, v, where v a a and v, v are he sngular value vecors for he sngular value a = U ΣV ha a. I can be derved fro R UW V de R vald R UW V de R vald R UW V de R nvald R UW4 V de R nvald R UW V de R nvald R UW V de R nvald R U W V de R vald R U W4 V de R vald u, u a v, v = a a 0 0 a a v, a v a a v, v a a (56) he explc relaonshp beween U and V s coplcaed, due o dfferen cobnaons of he deernan of U and V. We nex exane case by case. If de V, Case (). v a a & u v U VW Case (). v a a & u v U VW Case (). v a a & u v U VW Case (4). v a a & u v U VW 4 If de V, Case (). v a a & u v U VW Case (). v a a & u v U VW Case (). v a a & u v U VW 4 Case (4). v a a & u v U VW 0 0 where W W , 0 0 W , 0 0 and W Accordng o (55), he rgh roaon 0 0 s relaed o he non-unque SVD decoposon by R U U U W V,,,,4 (57) aong whch here exs nvald cases ha can be easly reoved by usng he fac ha R s requred o a roaon arx,.e., de R, as shown n able I. Addonally, le q q q q, q, q R, R, R U u, u, u, s obvous u u u u u u (58) Consderng able I and he above dscusson of dfferen cases, f de( U V ) u u v q R R R a a (59) Accordng o (5), he relaonshp beween a and can be known as 0 R a R a 0 (60) So f 0, we have e Ra a (6) akng (59) no consderaon, he ranslaon vecor can be derved as e = uq (6) In suary, he nuber of coplee pose soluons o he

7 . CAI E AL.: EUIVALEN CONSRAINS FOR WO-VIEW GEOMERY 7 essenal arx equaon (4) s egh, whch are now concluded as R, where RR R UW V,,,,4 and = or uq u q. hey are conssen wh hose n he prevous work of Wang e al. [6]. As U and V are usually obaned by he general SVD, de( UV ) ay be or -. As shown n able I, f de( UV )=, R and R are vald; oherwse, R and R 4 are vald. So n praccal code pleenaon, we can reduce he nuber of poenal pose soluons fro egh o four by sply resrcng de( U ) V, naely, R, R, R and R. Harley e al. [] proposed hese four soluons by posng he consran de U de V ha falls n he scope of de U V. herefore, he essenal arx equaon (5) has no less han four pose soluons. Proposon 6: For he case 0, E a (or s equvalen o E up o scale); when 0, E 0 and has nohng o do wh E. Proof. Fro (5) and (6), f 0 a a a Ra R R = RR R E (6).E.D. Proposon 7: For he case 0, he rgh roaon arx R can be obaned as one of he soluons; he ranslaon esae s always wrong. Proof. For he case of 0, he rgh roaon arx can be obaned fro (57) because (5) s sll vald. However, he ranslaon vecor esae sasfes ˆ uq Ra a accordng o (59). I has nohng o do wh he rue ranslaon, naely, zero ranslaon. As we know, he nor of ˆ s always equal o..e.d. 4 IDENIFY RIGH SOLUION WIHOU D RECONSRUCION hs secon wll splfy he PPO equaly consrans no lnear nequaly consrans and hen use he o denfy he rgh soluon fro he ulple soluons o he essenal equaon. Secon explores he heorecal pose soluons o he essenal arx equaon (5). In applcaons, an esae of he essenal arx ˆ s obaned as he soluon of (4). Hereafer we use he haed sybols o dsngush he esae fro he heorecal soluon above. hen by he popular SVD ehod, four pose soluons can be obaned by posng he consran de U V, naely, ˆ ˆ Rˆ ˆ, Rˆ ˆ, Rˆ ˆ and Rˆ ˆ, where R ˆ U W V ˆ ˆ,, ˆ u q and ˆ u q, as n [, 6]. 4. Sae-Sde Inequaly Consran Fro he sae-sde consran (7), e R X e X R e X e X I can be reduced o e R R X e X e X e X X e e X 0 R (64) (65) hs s n spr slar o he nequaly o dsabguae he roaon arx by Knep [0], whch s nonlnear n he roaon arx. Noe ha hs nequaly has a que wde hreshold, naely e R X e X, whch s very helpful n robusly denfyng he rgh soluon, as shown n nex secon. Consderng (5) and (6), e e a, so he consran for a sngle pon s forulaed as R X RX 0 (66) where R s a funcon of R when ohers are known. I can be checked ha he above nequaly s able o dscrnae he rgh orenaon. Gven ˆ and R ˆ RorR ˆ ˆ, X ˆˆ R ˆ ˆΣΣ X X U W V ˆ X or (67) X ˆΣΣ U W Vˆ X As ΣΣ W ΣΣ W, we have Rˆ Rˆ (68).e. Rˆ Rˆ 0 (69) he equaon shows ha he wo orenaon soluons yeld R ˆ wh oppose sgn, so s feasble o ge he rgh orenaon by usng R ˆ 0. For all pon pars, he sae-sde lnear nequaly consran can be collecvely wren as X X ˆ ˆ ˆ M R vec Rˆ 0 (70) X X whch can be used o selec he rgh roaon arx. 4. Inersecon Inequaly Consran Snce = and 0,, we have 0. I eans R cos X cos X (7) X X.e. X e X R e X X (7) I can be wren as R R, X X X X e 0 (7) I

8 . CAI E AL.: EUIVALEN CONSRAINS FOR WO-VIEW GEOMERY 8 For he ranslaon soluons n Secon, we can see ha R, s effecve as Rˆ, ˆ R ˆ, ˆ 0. Acually, he sgn of R, s ndependen of he rgh or wrong R. Le ˆ u q, for ˆR and ˆR, we have ˆ R ˆ ˆ ˆ ˆ ˆ UWV VWU u q q I u I q u 0 (74) VW ˆ 0 vq uq uq and 0 ˆ R VW ˆ 0 q ˆ v I (75) uq uq So Rˆ, ˆ R ˆ ˆ, when ˆ u q (or ˆ u q ). In concluson, he nersecon lnear nequaly consran R, can be descrbed as 0. For all pon pars, he nersecon nequaly consran can be collecvely wren as X X X X ˆ ˆ R M ˆ 0 I (76) X X X X whch can used o selec he rgh ranslaon vecor. In conras o he robusness n denfyng he roaon M Rˆ, fndng he rgh ranslaon vecor arx by by M ˆ 0 0 n he case of large deph/ranslaon rao should be cauous. In hese cases, accordng o Proposon 4, XRX 0 and X' R X e X' e RX RX fro (5). gh be very near o zero because = X X R X X e X X X X e (77) R R R 0 hs gh cause soe proble n ranslaon denfcaon n he case of nearly pure roaon. However, ranslaon esaon under hs case s of lle sgnfcance. In hs aspec, would be very helpful for judgng he sgnfcance of he obaned ranslaon vecor f he pure roaon case could be successfully denfed. 4. Robus Relave Pose Algorh Whou D Reconsrucon able II suarzes he algorh of lnear soluons of he essenal arx and how o robusly denfy he rgh soluon usng he proposed wo nequaly consrans. Due o he effec of pon achng error, here gh exs soe pon pars ha volae he consrans. o prove he robusness agans such errors, we could choose he soluon as he rgh one wh he axu nuber of pon pars ha sasfy M R ˆ 0 and M ˆ Relave Roaon Idenfcaon Proposons 4-5 prove he propery of pure roaon and s dfference fro nfne pons and baselne pons (.e., pons on he ranslaon baselne). In hs secon, R denoes he rgh roaon arx denfed by M R ˆ 0. o denfy pure roaon, we gh anually se a hreshold on X RX for all pon pars. In oher words, we can denfy he pure roaon oon by X RX X X X RX R, MR avg (78) X X X RX X X where s a hreshold and avg s he average funcon. We denfy he caera oon as pure roaon f M R s greaer han soe-prescrbed confdence level, e.g., here exss a proble ha needs o be anually uned under dfferen cases. Proposon 8: For all D pons oher han hose on he baselne, X RX =0 f and only f =0. Proof of Necessy: If X RX =0, accordng o Proposon 4 and referrng o he developen n (77), X ' RX e X e RX RX fro (5). hen we have =0. Proof of Suffcency: If =0, referrng o Fg., we have = X X R e X X e=0 X X cos X X cos 0 (79) cos =cos whch eans = as, [0, ] and he cosne funcon s onoonc. Fro he nersecon consran, he angle us be zero, naely, X RX =0. ABLE II. POSE ESIMAION WIHOU D RECONSRUCION Sep. Gven he lnear soluon arx ; Sep. Sep. Sep 4. By SVD, U V. Choose W or W f de U V. Oherwse, choose W or W 4 ; ˆ Use ˆ u q and R UWV or UWV (or oher ehods) o ge poenal four pose soluons; Coun he nuber of posve enes n M R ˆ, de- S R ˆ, hen he rgh roaon should be noed as Rˆ S R ˆ Rˆ ve enes n M ˆ, denoed as S ˆ rgh ranslaon should be ˆ ax{ S ˆ} ax{ }. Slarly, coun he nuber of pos-. ˆ, hen he

9 . CAI E AL.: EUIVALEN CONSRAINS FOR WO-VIEW GEOMERY 9.E.D. Noe ha he noral scenes wll no conan nfne pons and ranslaon baselne pons only. Accordng o Proposons 4-5, Proposon 8 acually ells ha =0 could be alernavely used as a creron for denfyng pure roaon. 0 5 ES RESULS In our sulaon ess, he D feaure pons are cenered a known coordnaes and subjeced o unfor dsrbuons, whch are apped o wo vews o generae age par coordnaes n pxels. For exaple, we use U(0,0, d),(0,0, d ) o denoe he unfor dsrbuon n a cube ha s cenered a 0 0 d and of lengh 0, 0 and d (n eers) along X, Y and Z axes, respecvely. he exrnsc paraeer arx of he lef vew s consanly se o I and he nrnsc paraeer arx of wo ca eras s We generae N ps D pons n U(0,0, d),(0,0, d ). o ge N s poses of he rgh vew, we generae he roaon arces of whch he correspondng Euler angles obey N 0 0 0,dag n degrees, and he un drecon vecor of ranslaon n he OXY plane. Each ranslaon lengh s gven by =, where he parallax facor 0, and =4. denoes he ax- ax ax u lengh of ranslaon. Fgure deonsraes D pons and he poses of rgh vews for d 0, N ps 50 and Ns 0. Iage pons are obaned by projecng D pons ono he par of vews and he Gaussan nose s added o he generaed age par coordnaes, wh he sandard devaon varyng beween 0.~5 pxels for N sd es. N c Mone Carlo runs are carred ou for each case. In he followng ess, we se ( N s, N ps, N sd, N c )=(00, 50, 00, 0) ABLE III. POSE SOLUIONS FOR A PURE ROAION CASE rue Pose (Roaon n Euler angles, degree; ranslaon) Four Pose Soluons and d 0 unless explcly saed oherwse. he experenal phenoenon ha he roaon par can sll be recovered fro he essenal arx equaon n he pure roaon case has been repored n very few leraure, e.g., [8]. able III lss he pose decoposon resul n a case of pure roaon. As predced n Secon 4, we see ha he roaon arx can be sll recovered, bu he ranslaon vecor s wrong. 5. Pose Soluon Idenfcaon Resul R We defne he dscrepancy, j beween he denfed roaon arx and he rue roaon arx as R, ˆ j dc angle R R, j (80) L where R s he rue roaon of he -h wo-vew and R ˆ, j s he denfed roaon arx n he j-h Mone Carlo run. dcangle eans he Euler angle vecor derved fro he roaon arx, and s he -nor of a vecor. L Slarly, defne he dscrepancy, j beween he denfed ranslaon vecor and he rue ranslaon by ˆ, j, j arccos (8) ˆ, j Fgure 4 plos he roaon and ranslaon dscrepances (RMSE), averaged across Mone Carlo runs, as he funcon of nose sandard devaon and he parallax facor. he resul n Fg. 4(a) accords wh Proposon 7 ha we can sll ge good roaon esaes even when approaches zero. Acually, he roaon denfcaon s always rgh. However, as shown n Fg. 4(b), he esaed ranslaon vecor s suscepble o he nose sandard devaon or he parallax facor. he ranslaon dscrepancy even reaches 90 degree a he rgh-boo rangular area where he nose sandard devaon s large or he parallax facor s sall. hs observaon shows a fac ha he ranslaon esae obaned fro he essenal equaon s unsasfacory n accuracy. In order o nvesgae he robusness of he pose denfcaon ehod proposed n Secon 4, we copare wh ha of he radonal ehod by (8) dff rad new where rad and new are ranslaon dscrepances for he radonal ehod [] and he new ehod, respecvely, and dff s he dfference of he wo ranslaon dscrepances, as shown n Fg. 5. In order o ge a ore nuve resul of relave advanage, we arguably average he resuls of all parallax facors for a defne nose sandard devaon and vce vsa, as shown n Fgs. 6(a)-(b) for d=0, 50 and 80. If we regard Fg. 5 as a arx, hey respecvely correspond o colun averagng and row averagng. In Fg. 6, he ranslaon dscrepancy dfferences are alos all above zero, whch shows ha he new denfcaon ehod perfors ore robusly. As shown n Fg. 6(a), he larger nose sandard devaon becoes, he beer robusness s. In Fg. 6(b), he rend s slar for sall parallax

10 . CAI E AL.: EUIVALEN CONSRAINS FOR WO-VIEW GEOMERY 0 Fg. An exaple of synhec D pons and rgh vews. Fg 5. ranslaon dscrepancy dfference relave o radonal ehod (degree), as a funcon of parallax facor and nose sandard devaon. Fg 4(a). Roaon dscrepancy (degree), as a funcon of parallax facor and nose sandard devaon. Fg 6(a). ranslaon dscrepancy dfference averaged across all parallax facors. Fg 4(b). ranslaon dscrepancy (degree), as a funcon of parallax facor and nose sandard devaon. Fg 6(b). ranslaon dscrepancy dfference averaged across all nose sandard devaons.

11 . CAI E AL.: EUIVALEN CONSRAINS FOR WO-VIEW GEOMERY Fg 7. Reconsrucon error rao of analycal ehod relave o radonal ehod, as a funcon of parallax facor and nose sandard devaon. Fg 8(b). Ground-ruh caera poses and D srucure n real ess. Fg 8(a). es able and barcode papers wh feaure pons. facors, and along wh furher ncreased parallax facor, he ranslaon dscrepancy dfference urns o decrease and sops jus above zero (see d=0 for exaple). 5. Analycal Reconsrucon Resul Accordng o Proposon, he relave deph has been expressed n ers of he pose. I acually gves a new ehod o analycally reconsruc D pons. here are wo dephs exsng n (6), one for he lef vew and he oher for he rgh vew. he dfference beween he wo dephs s neglgble and hey are averaged as he deph esae of he new ehod. We copare he new ehod wh he radonal D pon reconsrucon ehod by he DL-based lnear rangulaon []. he deph of boh ehods s copued usng he denfed soluon n Secon 5.. he reconsrucon error rao s defned as D D anal rao D (8) rad Fg 9. Reconsrucon error raos for wo-vew cobnaons n real ess. D D where rad and anal are respecvely he D pon average reconsrucon error (RMSE across Mone Carlo runs) of he analycal ehod and he radonal ehod. Fgure 7 presens he reconsrucon error rao as a funcon of nose sandard devaon and parallax scale. For noral cases wh large parallax, he wo reconsrucon ehods perfor alos dencally, bu for sall parallax and large D nose sandard devaon, rao ends o be uch less han, whch eans ha he analycal ehod s uch beer n reconsrucon accuracy. Real ess are carred ou o confr he above reconsrucon advanage. wo D barcode papers are placed ono uually-perpendcular faces of a able. Pcures are aken wh seven seleced poses so ha he wo barcode pcures are well observed (see Fg. 8). Fgure 8(a) gves an exaple pcure of he able and barcode papers, n whch 6 feaure pons are observed, and Fg. 8(b) depcs he seven caera poses relave o he barcode papers.

12 . CAI E AL.: EUIVALEN CONSRAINS FOR WO-VIEW GEOMERY Fg 0(a). Reconsrucon resuls for -4 par of vews wh larges parallax. Ground ruh (n green), analycal reconsrucon (n red), radonal reconsrucon (n blue) and analycally-nalzed BA (n black). Fg 0(c). Reconsrucon resuls for -4 par of vews. Ground ruh (n green), analycal reconsrucon (n red) and analycally-nalzed BA (n black). Fg 0(b). Reconsrucon resuls for 4-6 par of vews wh second larges parallax. Ground ruh (n green), analycal reconsrucon (n red), radonal reconsrucon (n blue) and analycally-nalzed BA (n black). We use he PnP ehod [, ] o copue he rue caera poses and he rue D pons. Specfcally, a separae spaal frae s aached o each barcode paper and he PnP ehod s used o copue he caera poses relave o each barcode paper frae. he ground-ruh caera poses and D barcode feaure pon coordnaes are obaned by unfyng he wo spaal fraes. he reconsrucon error raos for all C7 cobnaons (blue crcles) are ploed n Fg. 9. Excep he -4 par of vews, he analycal reconsrucon ehod s overwhelngly beer n reconsrucon accuracy. Consderng ha he nal pose soluon ay lkely be oo coarse o suable for reconsrucon, he bundle adjusen (BA) [] s used o opze D pons and he pose. he BA s respecvely nalzed by he analycal reconsrucon resul and he radonal reconsrucon resul. he reconsrucon error raos afer he BA are also presened n red crcles n Fg. 9. I shows ha he BA does no Fg 0(d). Reconsrucon resuls for -4 par of vews. Ground ruh (n green), radonal reconsrucon (n blue) and radonally-nalzed BA (n black). sgnfcanly change he reconsrucon error rao, n oher words, he analycal reconsrucon ehod s sll uch preferred. wo deonsrang reconsruced barcode papers ogeher wh he ruh are gven n Fg. 0, one s he -4 par of vews wh he larges parallax, he oher one 4-6 par of vews wh he second larges parallax. We can see ha he wo ehods perfor slarly for he larges parallax case n Fg. 0(a), bu he analycal reconsrucon ehod s uch beer for he second larges parallax case n Fg. 0(b). For -4 and 4-6 vew pars, he analycally-nalzed BA and radonally-nalzed BA are alos dencally good (see Fg. 9), so only he analycally-nalzed BA s ploed n Fg. 0. In addon, Fgs. 0(c)-(d) gve he reconsruced barcode papers for he -4 par of vews. he analycal reconsrucon nalzaon leads o a well-behaved BA n Fg. 0(c), whle he radonal reconsrucon nalzaon leads o a dvergng BA n Fg. 0(d). hs can be confred by he reconsrucon error rao n Fg. 9, alos zero for he -4 vew par.

13 . CAI E AL.: EUIVALEN CONSRAINS FOR WO-VIEW GEOMERY Fg (a). PRI as a funcon of parallax facor and nose sandard devaon. Fg (b). M R as a funcon of parallax facor and nose sandard devaon. Fg (a). Reconsrucon errors by analycal ehod and analycallynalzed BA for wo-vew cobnaons n ascendng order of PRI. Fg (b). Reconsrucon errors by analycal ehod and analycallynalzed BA for wo-vew cobnaons n ascendng order of M R. Fg (a). e coss for radonal ehod, Mehod I and II, as a funcon of he nuber of feaure pons. Fg (b). Raos of e cos of radonal ehod, over Mehod I and II, as a funcon of he nuber of feaure pons.

14 . CAI E AL.: EUIVALEN CONSRAINS FOR WO-VIEW GEOMERY 4 5. Pure Roaon Idenfcaon Resul Proposons 4, 5 and 8 acually presen wo alernave ehods o denfy pure roaon or nfne D pons. In Secon 4.4, we have copared he for denfyng he pure roaon and recoended he ehod derved fro Proposon 8. Defne a pure roaon ndcaor (PRI) as he average of M ˆ n (76) across all feaure pons ˆ ˆ PRI avg M (84) and denfy he oon as a pure roaon f PRI ˆ pr (85) where pr s a hreshold. Acually, s ready o check ˆ = ˆ, so we sply denoe as PRI. Referrng o Fg. 4(b), for he case of pure roaon ( 0 ) and zero nose, he ranslaon dscrepancy s roughly 90 degree. For a fxed nonzero parallax facor, e.g., 0., as he nose sandard devaon ncreases he ranslaon dscrepancy also gradually approaches 90 degree. he cases of abou 90 degree ranslaon dscrepancy roughly fors a rangle a he lower-rgh corner. ranslaon esae wh so large dscrepancy s usually of lle use, so s preferable o denfy all of he by a chosen hreshold pr. Fgure plos he PRI and M R wh he sae sulaon sengs as Fg. 4. Referrng o Fg. (a), for he case of pure ranslaon and zero nose, PRI s approxaely zero as predced by Proposon 8. For a fxed nonzero parallax facor, e.g., 0., ncreasng nose sandard devaon ends o reduce he PRI o zero. Noably, he color paern of PRI uch resebles ha of he ranslaon dscrepancy n Fg. 4(b). hs enables us o easly fnd a hreshold o denfy he cases wh large ranslaon dscrepancy. For nsance, pr 0.05 correspondng o he agena color s a good choce o denfy he cases wh 90 degree ranslaon dscrepancy. In conras, he M R n Fg. (b) has no such a nce propery, naely, we canno fnd a proper hreshold o do he denfcaon. We have esed D pons wh dfferen dephs (d=0, 50 and 80) and found ha pr 0.05 s sll a good hreshold for all of hese scenaros. Fgures (a)-(b) plo he reconsrucon errors by he analycal ehod and he analycally-nalzed BA for wo-vew cobnaons, respecvely n he ascendng or- M R. he PRI has a uch sharper rsng der of PRI and han M R. Larger PRI predcs hose vew pars (-4, 4-6, 4-7, -4) ha have saller nal reconsrucon errors and converged BA resuls. In coparson, M R canno effecvely pck he -4 vew par ou usng a hreshold, because hose vew pars (-4, -, 4-5, -6, -7) o he rgh have dverged BA resuls. 5.4 Copuaon Cos e coss as a funcon of he nuber of feaure pons are shown n Fg.. he elapsed e are averaged across 0 Mone Carlo runs. Mehod I eans he wo nequales are eployed o denfy he rgh pose soluon, and Mehod II addonally uses he rgh pose fro Mehod I o analycally reconsruc D pons. In conras, he radonal ehod denfes he rgh soluon by he DLbased D reconsrucon and subsequen posve deph check for each of four soluons []. Noe ha he e cos of copung he feaure pars s no aken no accoun. he arx,, XX X X n M R uses he sae one as n solvng he arx n (4). Fgure clearly shows ha Mehods I and II have alos he sae e cos as he analycal reconsrucon s very cos-effcen. As copared wh he radonal ehod, hey reduce he e cos by abou 7-9 es. D o D Mappng Four Soluons 7 6 Fg 4. Connecons aong nvesgaed probles. Suffcen and necessary condon; Suffcen condon; Lnear splfcaon; 4 Relave poses by SVD; 5 Algorh n able II; 6 Requrng de U V or de U V Sae-sde Consran Essenal Equaon 4 Egh Soluons yes 5 Rgh Roaon =0 no 8 Proposon. 9 Idenfcaon by PRI. Inersecon Consran Sae-sde Lnear Inequaly Consran Rgh ranslaon 8 Inersecon Lnear Inequaly Consran Analycal Reconsrucon 9 Pure Roaon Idenfcaon ; 7 Obanng rgh R only. 6 CONCLUSIONS he well-known essenal arx equaon only represens he coplanar relaonshp n he wo-vew agng geoery and loses he poran connecon aong he ranslaon vecor and wo projecon rays. he paper coes up wh he PPO (sae-sde and nersecon) consrans for he wo-vew geoery proble ha are proven o be equvalen o he wo-vew agng geoery. he coplee pose soluons o he essenal arx equaon are explcly derved fro he perspecve of equaon solvng. I s shown ha he orenaon can sll be recovered n he pure roaon case. wo nequales are forulaed by splfyng he wo new consrans, so as o help drecly denfy he rgh soluon. I does no need D pons reconsrucon and deph check ha are requred n radonal ehods. he nersecon nequaly consran lends self o a creron for denfyng he pure roaon oon of he caera.

15 . CAI E AL.: EUIVALEN CONSRAINS FOR WO-VIEW GEOMERY 5 he relaonshp aong he probles dscussed n hs paper s suarzed n Fg. 4 for easy reference. es resuls deonsrae he usefulness of he PPO consrans n robusly denfyng he rgh pose soluon and pure roaon, as well as n D pon reconsrucon. ACKNOWLEDGMEN hanks o anonyous revewers for her consrucve coens and Dr. Danpng Zou for group alks. he work s funded by Naonal Naural Scence Foundaon of Chna (64, 6676, 65040) and Hunan Provncal Naural Scence Foundaon of Chna (05JJ0). APPENDIX Cross Produc: he cross produc of wo vecors a and b s defned only n hree-densonal space and s denoed by a b. he vecor rple produc s he cross produc of a vecor wh he resul of anoher cross produc, and s relaed o he do produc by he followng forula ab c a c b b c a (86) and a bc a c b a b c (87) Kronecker Produc: If A s an n arx and B s a p q arx, hen he Kronecker produc A B s he p nq block arx: ab a nb AB a B anb Vecorzaon: he vecorzaon of an n arx A, (88) denoed vec A, s he n colun vecor obaned by sackng he coluns of he arx A on op of one anoher: veca= a,, a, a,, a,, an,, an (89) Here, aj represens he eleen of row- and colun-j. he vecorzaon s frequenly used ogeher wh he Kronecker produc o express arx ulplcaon as a lnear ransforaon on arces. In parcular, vecabc = C AvecB (90) for arces A, B, and C of densons k l, l, and n. Proceedngs CVPR '9., 99 IEEE Copuer Socey Conference on, 99, pp [5] H. C. Longuehggns, "A copuer algorh for reconsrucng a scene fro wo projecons," Naure, vol. 9, pp. -5, 98. [6] O. D. Faugeras and S. Maybank, "Moon fro pon aches: ulplcy of soluons," Inernaonal Journal of Copuer Vson, vol. 4, pp. 5-46, 990. [7]. S. Huang and O. D. Faugeras, "Soe Properes of he E Marx n wo-vew Moon Esaon," IEEE ransacons on Paern Analyss & Machne Inellgence, vol., pp. 0-, 989. [8] S. Maybank, "heory of reconsrucon fro age oon," Sprnger, vol. 8, 99. [9] R. I. Harley, "An Invesgaon of he Essenal Marx," Repor Ge, 995. [0]. S. Huang and Y. S. Sh, "Lnear Algorh for Moon Esaon: How o Handle Degenerae Cases," n Inernaonal Conference on Paern Recognon, 988, pp [] D. Nser, "An Effcen Soluon o he Fve-Pon Relave Pose Proble," IEEE ransacons on Paern Analyss & Machne Inellgence, vol. 6, pp , 004. [] H. Sewénus, C. Engels, and D. Nsér, "Recen developens on drec relave orenaon," Isprs Journal of Phoograery & Reoe Sensng, vol. 60, pp , 006. [] R. Harley and A. Zsseran, Mulple Vew Geoery n Copuer Vson: Cabrdge Unversy Press, 00. [4] B. K. P. Horn, "Relave orenaon," Inernaonal Journal of Copuer Vson, vol. 4, pp , 990. [5] Z. Zhang, "Deernng he Eppolar Geoery and s Uncerany: A Revew," Inernaonal Journal of Copuer Vson, vol. 7, pp. 6-95, 998. [6] W. Wang and H.. su, "An SVD Decoposon of Essenal Marx wh Egh Soluons for he Relave Posons of wo Perspecve Caeras," n Inernaonal Conference on Paern Recognon, 000, pp vol.. [7] L. Lng, E. Cheng, and I. S. Burne, "Egh soluons of he essenal arx for connuous caera oon rackng n vdeo augened realy," n n Proceedngs of he 0 IEEE Inernaonal Conference on Muleda and Expo (ICME 0), Barcelona, Span, 0, pp. -6. [8] Y. Ma, S. Soao, J. Kosecka, and S. S. Sasry, An nvaon o -D vson: Sprnger, 004. [9] J. C. Bazn, C. Deonceaux, P. Vasseur, and I. S. Kweon, "Moon esaon by decouplng roaon and ranslaon n caadoprc vson," Copuer Vson & Iage Undersandng, vol. 4, pp. 54-7, 00. [0] L. Knep, R. Segwar, and M. Pollefeys, "Fndng he exac roaon beween wo ages ndependenly of he ranslaon," n European Conference on Copuer Vson, 0, pp [] G. H. Golub and C. F. Van Loan, Marx copuaons (rd ed.), 996. [] V. Garro, F. Croslla, and A. Fusello, "Solvng he PnP Proble wh Ansoropc Orhogonal Procruses Analyss," n Second Inernaonal Conference on d Iagng, Modelng, Processng, Vsualzaon and ranssson, 0, pp [] L. Ferraz, X. Bnefa, and F. Moreno-Noguer, "Very Fas Soluon o he PnP Proble wh Algebrac Ouler Rejecon," n Copuer Vson and Paern Recognon, 04, pp REFERENCES [] P. A. Beardsley and A. Zsseran, "Affne Calbraon of Moble Vehcles," n Europe-Chna Workshop on Geoercal Modellng and Invarans for Copuer Vson, 995, pp. 4-. []. Vevlle and D. Lngrand, "Usng sngular dsplaceens for uncalbraed onocular vsual syses," n European Conference on Copuer Vson, 996, pp [] R. I. Harley, "Esaon of Relave Caera Posons for Uncalbraed Caeras," presened a he ECCV '9 Proceedngs of he Second European Conference on Copuer Vson, 99. [4] R. Harley, R. Gupa, and. Chang, "Sereo fro uncalbraed caeras," n Copuer Vson and Paern Recognon, 99.