Transformation with homogenous matrices. Image Sensors Lecture E. A homogeneous matrix for translation. Homogenous matrices for scaling and skewing

Transcription

1 Image Sensos Lectue E amea calbaton Homogenous matces o scalng, tanslaton, otaton, skewng (Magnusson: Secton.) he Pnhole camea model (Magnusson: Secton.) Oute and nne paametes 3D calbaton o a camea (Magnusson: Secton.3) albaton o a lat wold, a homogaphy) (Magnusson: Secton.4) Zhang s method o 3D camea calbaton Radal dstoton Lteatue Shot about camea geomety and camea calbaton by Maa Magnusson. lexble new technque o camea calbaton by Zhengyou Zhang, Mcosot Reseach. alable as shot atcle o long epot. ltenate Lteatue Intoducton to Repesentatons and Estmaton n Geomety, hapte, by Klas Nodbeg, (couse mateal n SBB6). p. ansomaton wth homogenous matces pont n the 3D-wold can be descbed as (X,Y,Z,). It can be tansomed to a new pont (X,Y,Z,) by usng the 4x4-matce M accodng to X X m m m3 m4 X Y Y m m m3 m4 Y M Z Z m 3 m 3 m 33 m 34 Z p. Maa Magnusson, VL, Dept. o Electcal Engneeng, Lnköpng Unesty homogeneous matx o tanslaton p. 3 Homogenous matces o scalng and skewng p. 4 anslaton Example: Scalng Skewng n the x-decton dependng on the y-coodnate Geneal skewng tx X t x tx X t y Y t y t y Y tz Z t z tz Z, t, t t x y z S s a a sb s c s, s, s a b c a b c d e Eq. (5) Note: nomal 3x3-matx wll not wok o tanslaton! Eq. (3) Eq. () Eq. ()

2 Homogeneous matces o otaton Eq. (7) R x Rotaton wth the angle Rotaton wth the angle aound the x-axs cos sn sn cos Rotaton wth the angle Eq. (8) aound the y-axs aound the z-axs R y cos sn sn cos R z cos sn sn cos p. 5 Eq. (9) he Lens law (epetton) object lens mage plane he lens law states that the mage plane s located at the dstance b om the lens, then the object at dstance a om the lens wll ge a shap mage. Note that snce nomally a>>b => b. B he lens law: a b whee s the ocal length Sze elatons: a B b p. 6 B he pnhole camea model, eal geomety he mage plane s located behnd the lens! p. 7 he pnhole camea wth coodnates (u model, moed, ). ltenately the notaton he mage plane s moed so that t s located n ont o the lens. deal mage plane p. 8 Hee we use the notaton: deal mage plane nomalzed mage plane wth coodnates (u n, n )=(u /, /) may be used. Relaton between the coodnate systems: Eq. () Fg. Fg. u W n n u,, W,, U, V, W Rt X, Y, Z,

3 echnque to expess pespecte p. 9 tansomaton wth ectos Fg. 3 Relaton between the mage planes p. measued n pxels Unom tangles ges: u n n u n and n s the noma- lzed mage coodnates V W u U W u W,, U, V, W Eq. (4) Eq. (5) u u,,,, Inne and oute paametes Relaton between the coodnate systems: Eq. () Relaton between the mage planes: u, u W,, U, V, W RtX Z, u,,, Eq. (5) Relaton between s u,, Rt X, Y, Z, wold coodnates and mage coodnates: Inne paametes Oute paametes Eq. (7) Note that s eplaces W as pespecte pojecton paamete p. Unambguousness, eld o ew (FOV) and esoluton FOV angle n the U-decton plane p. he paametes W and s ae not unambguously detemned. Both the bg ball and the small ball ges the same contou n the (u, )-plane. onsequently, we cannot know W. heeoe we can also change W to s n the peous slde. It s appopate to measue the eld o ew (FOV) as the lagest measuable angle n the U- and V-decton. (see e.g. Lab execse E: Panoama sttchng) he esoluton o an object n an mage depends on the dstance om the camea. he esoluton o n the U- decton can, o example, be measued as the FOV angle/the numbe o pxels.

4 Oute paametes Relaton between the coodnate systems: U, V, W RtX Z, Eq. () Eq. (3) 3 Rt t t t 3 t t t : the tanslaton o the camea n elaton to the wold R: the otaton o the camea n elaton to the wold p. 3 he oute paametes o a camea n a x poston can be detemned though a calbaton pocedue. Inne paametes Relaton between the mage planes: Eq. (5) u u,,,, u 6) Eq. ( p. 4 he nne paametes o a camea can be detemned though a calbaton pocedue. deal mage plane measued n.e. mm eal mage plane measued n pxels : scalng n the -decton : scalng the u-decton ecto : skewng (oten close to ) u : the coss-secton between the optcal axs and the eal mage plane Inne paametes, ex) wth = u u u ) B) u u, u,, u,,, ) B) p. 5 u u : the coss-secton between the optcal axs and the eal mage plane, the mage cente, the pncpal pont. and denotes the scalng the u- and -decton, espectely. I =, the pxels ae quadatc. I, the pxels ae ectangula, but not quadatc. u Inne paametes, ex) wth u u u ) B) u, u, u,, u,, s the skewng paamete actanges t an angula measuement s nomally small,.e. close to degees ) u p. 6 actan B) u

5 3D calbaton o a camea u,, RtX Z, s Eq. (7) s u,, X, Y, Z, Eq. (8) We wll soon lean about Zhang s technque to detemne, R and t, but st we wll detemne only. he matx can be detemned by measung a numbe o coespondng pont (how many?) n the wold (X, Y, Z ) and the mage (u, ), whee N. Dependng on the aable s, can only be detemned up to a scale acto. We set 34 =. (I 34 seems to be, anothe element can be set to.) p. 7 Eq. (9) 3D camea Eq. (8) calbaton Set: Eq. () Eq. (9) c D c, 3, 4,,, 3, 4, 3, 3, u,, X Z, s , 33 Eq. () X Y Z ux uy uz u X Y Z X Y Z X Y Z u X u Y u Z u 3 X N YN Z N N X N NYN N Z N 33 N equatons ge that at least 6 pont-pas ( 5½ ) s needed to detemne Show Eq. () Soluton o the equaton system p. I we measue 5½ pont-pas, we get equatons. he equaton system can be soled as: c D I we measue moe than 5½ pont-pas, the equaton system becomes oe-detemned wth the soluton: Moe pont-pas pas ges a moe cetan soluton! D + s pn n Matlab Dc D D c c c D D D D D Eq. (3) D + s the so called pseudo-nese o D. It can be shown ths s equalent to both Maxmum Lkelhood-mnmzaton and Least Squae adjustment.

6 p. p. Fom to [Rt] When the matx s detemned, t s possble to ecee, R and t by usng a lttle lnea algeba. hs pocedue s called camea esectonng. he pogam PKRt.m peoms ths and can be tested dung the compute execse. Lab task! c D Rt he nhomogeneous soluton method accodng to SBB6: Multdm. Usng the calbated camea We now know how a pont n the wold (X,Y,Z) wll be mapped to a pont n the mage (u,). We do not know how a pont n the mage (u,) wll be mapped to a pont n the wold (X,Y,Z). But we do know that a pont n the mage (u,) coesponds to a lne n the wold (X,Y,Z). Fom and an object pont n the mage, we can calculate the angula decton to the coespondng object pont n the wold. hen t s possble o a moable camea to ollow an object. Lab task! I we hae moe knowledge about the wold, o example t s a lat wold, we know that a pont n the mage (u,) s mapped to a pont n the wold (X,Y,Z). hs s camea calbaton o a lat wold, a homogaphy. Lab task! nothe possblty s to use steeo,.e. usng two calbated cameas. hey ges one staght lne, each. he coss-secton secton between these lnes ges the exact poston o the pont n the wold. albaton o a lat wold, a homogaphy Relaton between the coodnate systems: u,, RtX X s Fg..4 (4) Eq. p. 3 measued n pxels pont n the mage (u,) s mapped to apont n the wold (X,Y,) and ce esa. albaton o Eq. (4) the homogaphy Eq. (5) u,, X s, 3,,, 3, 3, 3 3 Set: c, Eq. (6) D c Eq. (7) X Y ux uy u X Y X Y X Y u X u Y u 3 Soluton as beoe: X N YN N X N NYN 3 N c D 8 equatons ge that at least 4 pont-pas s needed to detemne 3 3 3

7 amea calbaton, geneal p. 5 3D amea calbaton accodng to Zhang p. 6 Photogammety 3D calbaton object s manuactued wth good pecson. Dsadantage: expense and complcated. D calbaton object s manuactued wth good pecson. It can be a plane wth squaes. It s shown o the camea n deent oentatons. Zhang s appoach. dantage: cheap and smple. Lab task! Sel-calbaton he camea s mong n a statc scene. dantage: Flexble. Dsadantage: he esults ae not always elable. See also Zhang, secton : Motatons, R, and t n =[Rt] [ ] can be detemned d nddually d albaton pocedue, see Zhang: Secton 3.3 ) Pnt a patten and attach t to a plana suace. ) ake a ew mages o the model plane unde deent oentatons by mong the plane. Fg.. 3) Detect eatue ponts n the mages and elate them to ponts n the wold. 4) Detemne n -matces by calbatng n homogaphes. Detemne and [Rt] om the n - matces. 5) Rene all paametes, ncludng lens dstoton paametes n a non-lnea mnmzaton algothm. Not ncluded n the lab task! p. 7,) Hold the patten n some deent oentatons and take mages p. 8 3) Detect nteestng ponts n the mages and elate them to ponts n the wold Smla to Fg. u, coesponds to X, Y u, coesponds to X, Y j j j j Fom n calbaton planes we can detemne n -matces by calbatng n homogaphes usng the peously descbed technque.

8 4) Detemne and [Rt] om the n -matces p. 9 4) Detemne and [Rt] om the n -matces, cont. p. 3 u,, RtX X s Eq. (4) Dependng on the aable s, can only be detemned up to a scale acto. Zhang set 33 = and ntoduces as scale acto. t t 3 3 t t h h h3 3 and ae deed om otaton o and ae otonomal! Beoe Eq. (3) Note that,, t ae gone! h Fom the peous slde: t h h h3 wo mpotant constants: h h h h h Beoe Eq. (3) Eq. (3) Eq. (4) Poo on next slde! Poo o the constants (3) and (4) 4) Detemne and [Rt] om the n -matces, cont. B Fom a B-matx and a b-ecto: u Eq. (5) Eq. (6) B B B3 B B B3 calculate,calculate B 3 B3 B33 u u u u b B B B B B B,,, 3, 3, 33 p. 3 Note that the B- matx s sym- metc and that we can sole, om t.

9 4) Detemne and [Rt] om the n -matces, cont. hs s ald: h h hs can be checked by nsetng elements to the let and the ght sde. Note Zhang s deent ow/column notaton p. 33 h h j h h h3 h hj h hj h hj h3 h h h3 Set: j h 3hj h hj3 h 3 h3 h33 h 3hj h hj3 h B h b Eq. (7) j j h 3hj3 b Eq. (8) ontol on the whte boad. 4) Detemne and [Rt] om the n -matces, cont. x6-matx: Ple n Eq. (8) on top o eachothe nx6-matx: Vb b Eq. (8) Eq. (9) homogenous equaton system that can be soled wth SVDtechnque, see e.g. Magnusson Secton 3. o ompendum. When b s known, B s smply obtaned. hen the paametes, u, ae obtaned om Eq. (5). s now detemned! p. 34 How many calbaton planes ae needed? p. 35 4) Detemne and [Rt] om the n -matces, cont. p. 36 One calbaton plane ges one calbaton matx. One calbaton matx ges one Eq.(8) wth equatons. hee ae 5 unknowns n. I the skew =, thee ae 4 unknowns n. How many calbaton planes, at least, ae needed to detemne? 3 planes ae needed. planes ae needed =. =[Rt] s detemned up to 8 paametes by calbaton plane. hee ae 6 degees o eedom n [Rt], 3 otaton angles and 3 tanslaton dectons. onsequently 8-6= equatons ae obtaned o solng om one calbaton plane. See beoe Eq. (3) t h h h h3 When s known, [Rt] s smply obtaned as: h h, whee 3 h h t h3 hs s wtten a bt below Eq. (9) Obsee that Zhang now changes to -

10 5) lttle about adal dstoton (n the epot only) p. 37 Ex) Exteme wde angle lens ges bael dstoton p. 38 Radal dstoton s the most common Undstoted Bael Pncush- dstoon ds- mage: ton: toton: Example om tonbladet: Image nsde the mua oom. (ndes Bjöck, Hasse o and the Swedsh kng ae membes.) Othe types o dstoton: the human eye o an astgmatc peson, sheye-lenses, telescope Radal dstoson can be ncluded n the calbaton pocedue. Radal dstoton, t equatons (u,) ae the eal mage coodnates, as beoe. Let us call the nomalzed mage coodnates (x,y) nstead o (un,n): u u,,,, u n, n, x, y, nne paametes: u x y u u x y u,,, u, y y undstoted mage coodnates : u u, dstoted mage coodnates : u, undstoted nomalzedmage coodnates : x, y dstoted nomalzedmage coodnates : x,y p. 39 Radal dstoton, t equatons undstotedd mage coodnates : u u, dstoted mage coodnates : u, x, y g x,y undstoted nomalzed mage coodnates : dstoted nomalzed mage coodnates : Model: u u x y x x k k y x k k 4 4 Poo: See next slde. 4 u u k k 4 k k x y p. 4 k and k ae the coecents o adal dstoson he cente o the adal dstoton s the same as the pncpal pont.

11 Radal dstoton, t equatons 4 u u u u k k u x y u 4 y k k 4 x x p. 4 u x y u y 4 y u x y u x y k k 4 y y y k k 4 y x y x y k k 4 y y y k k 4 x x x k k 4 y y y k k he atan model o adal dstoton Used n Lab execse E: Panoama sttchng Let the mage be descbed n pola coodnates: (, θ ). hen out actan n γ s small, e.g. γ=. p. 4 oecton o adal dstoton (n the epot by Zhang only) p. 43 p. 44 5) Rene the paamete estmaton n a non-lnea mnmzaton algothm Magnusson s notaton: Zhang s notaton: u,, RtX s Eq. (4) s m ~ Rt M ~ Eq. () Pont n the mage Pont n the wold an be soled by the Leenbeg- Maquadt algothm, lsqnonln n Matlab, R, t, M n m m mˆ M j j Pojecton o pont M j n mage j Eq. ()

12 Degeneated conguatons p. 45 Whee s the camea cente n a eal lens? p. 46 I the calbaton plane at the second poston s paallel wth the st poston, the :nd homogaphy wll not ge any exta constants he camea cente s at EP (the entance pupl).e. the appaent poston o the apetue. Reeence: heoy o the No-Paallax Pont n Panoama Photogaphy Veson., Febuay 6, 6 Rk Lttleeld (j.lttleeld@compute.og) Whee s the camea cente n a eal lens? p. 47 he camea cente s at the entance pupl (the appaent poston o the apetu)