CS4495/6495 Introducton to Computer Vson 3C-L3 Calbratng cameras
Fnally (last tme): Camera parameters Projecton equaton the cumulatve effect of all parameters: M (3x4) f s x ' 1 0 0 0 c R 0 I T 3 3 3 x1 3 3 3 x1 0 af y ' 0 1 0 0 x x c 0 1 1 1 x3 01 x3 0 0 1 0 0 1 0 ntrnscs projecton rotaton translaton
Fnally (last tme): Camera parameters Projecton equaton the cumulatve effect of all parameters: x X sx * * * * Y sy * * * * Z s * * * * 1 M X
Calbraton How to determne M?
Calbraton usng known ponts Place a known object n the scene dentfy correspondence between mage and scene compute mappng from scene to mage
Resectonng Estmatng the camera matrx from known 3D ponts Projectve Camera Matrx: p K R t P M P X w * u m m m m 11 12 13 14 Y w * v m m m m 21 22 23 24 Z w m m m m 31 32 33 34 1
Drect lnear calbraton - homogeneous One par of equatons for each pont u w * u m m m m 00 01 02 03 Y v w * v m m m m 10 11 12 13 Z 1 w m m m m 20 21 22 23 u v m X m Y m Z m 00 01 02 03 m X m Y m Z m 20 21 22 23 m X m Y m Z m 10 11 12 13 m X m Y m Z m 20 21 22 23 X 1
Drect lnear calbraton - homogeneous u m X m Y m Z m 00 01 02 03 m X m Y m Z m 20 21 22 23 One par of equatons for each pont v m X m Y m Z m 10 11 12 13 m X m Y m Z m 20 21 22 23 u ( m X m Y m Z m ) m X m Y m Z m 20 21 22 23 00 01 02 03 v ( m X m Y m Z m ) m X m Y m Z m 20 21 22 23 10 11 12 13
Drect lnear calbraton - homogeneous u ( m X m Y m Z m ) m X m Y m Z m 20 21 22 23 00 01 02 03 v ( m X m Y m Z m ) m X m Y m Z m 20 21 22 23 10 11 12 13 One par of equatons for each pont m m m m m m m m m m m m 00 10 02 03 10 11 12 13 20 21 22 23
Drect lnear calbraton - homogeneous Ths s a homogenous set of equatons. When over constraned, defnes a least squares problem mnmze Am Snce m s only defned up to scale, solve for unt vector m* Soluton: m* = egenvector of A T A wth smallest egenvalue Works wth 6 or more ponts
Drect lnear calbraton - homogeneous m m m m m m m m m m A m m 0 m 2n 12 12 2n 00 10 02 03 10 11 12 13 20 21 22 23
The SVD (sngular value decomposton) trck Fnd the m that mnmzes Am subject to m =1. Let A = UDV T (sngular value decomposton, D dagonal, U and V orthogonal) Therefor mnmzng UDV T m But, UDV T m = DV T m and m = V T m Thus mnmze DV T m subject to V T m = 1
The SVD (sngular value decomposton) trck Thus mnmze DV T m subject to V T m Let y = V T m Now mnmze Dy subject to y = 1. = 1 But D s dagonal, wth decreasng values. So Dy mnmum s when y = 0,0,0, 0,1 T Snce y = V T m, m = Vy snce V orthogonal Thus m = Vy s the last column n V.
The SVD (sngular value decomposton) trck Thus m = Vy s the last column n V. And, the sngular values of A are square roots of the egenvalues of A T A and the columns of V are the egenvectors. (Show ths? Nah ) Recap: Gven Am=0, fnd the egenvector of A T A wth smallest egenvalue, that s m.
Drect lnear calbraton - nhomogeneous Another approach: 1 n lower r.h. corner for 11 d.o.f X u m m m m 00 01 02 03 Y v m m m m 10 11 12 13 Z 1 m m m 1 20 21 22 1
Drect lnear calbraton - nhomogeneous Dangerous f m 23 s really (near) zero!
Drect lnear calbraton (transformaton) Advantages: Very smple to formulate and solve. Can be done, say, on a problem set These methods are referred to as algebrac error mnmzaton.
Drect lnear calbraton (transformaton) Dsadvantages: Doesn t drectly tell you the camera parameters (more n a bt) Approxmate: e.g. doesn t model radal dstorton Hard to mpose constrants (e.g., known focal length) Mostly: Doesn t mnmze the rght error functon
Drect lnear calbraton (transformaton) For these reasons, prefer nonlnear methods: Defne error functon E between projected 3D ponts and mage postons: E s nonlnear functon of ntrnscs, extrnscs, and radal dstorton Mnmze E usng nonlnear optmzaton technques e.g., varants of Newton s method (e.g., Levenberg Marquart)
Geometrc Error m nm ze E d ( x, xˆ ) Predcted Image locatons x X m n d ( x, MX ) M M
Gold Standard algorthm (Hartley and Zsserman) Objectve Gven n 6 3D to 2D pont correspondences {X x }, determne the Maxmum Lkelhood Estmaton of M
Gold Standard algorthm (Hartley and Zsserman) Algorthm () Lnear soluton: (a) (Optonal) Normalzaton: X U X x =Tx (b) Drect Lnear Transformaton mnmzaton () Mnmze geometrc error: usng the lnear estmate as a startng pont mnmze the geometrc error: m n d ( x, M X ) M
Gold Standard algorthm (Hartley and Zsserman) () Denormalzaton: M=T -1 M U
Fndng the 3D Camera Center from M M encodes all the parameters. So we should be able to fnd thngs lke the camera center from M. Two ways: pure way and easy way
Fndng the 3D Camera Center from M Slght change n notaton. Let: M s(3x4) b s last column of M The center C s the null-space camera of projecton matrx. So f fnd C such that: M C = 0 that wll be the center. Really M Q b
Fndng the 3D Camera Center from M Proof: Let X be somewhere between any pont P and C X λ P (1 λ) C
Fndng the 3D Camera Center from M Proof: Let X be somewhere between any pont P and C And the projecton: X λ P (1 λ) C x M X λ M P (1 λ) M C
Fndng the 3D Camera Center from M Proof: Let X be somewhere between any pont P and C And the projecton: X λ P (1 λ) C x M X λ M P (1 λ) M C For any P, all ponts on PC ray project on mage of P, therefore MC must be zero. So the camera center has to be n the null space.
Fndng the 3D Camera Center from M Now the easy way. A formula! If M =[Q b] then: C Q 1 1 b
Alternatve: mult-plane calbraton Images courtesy Jean-Yves Bouguet, Intel Corp.
Alternatve: mult-plane calbraton Advantages Only requres a plane Don t have to know postons/orentatons Good code avalable onlne! OpenCV lbrary Matlab verson by Jean-Yves Bouget: http://www.vson.caltech.edu/bouguetj/calb_doc/ndex.html Zhengyou Zhang s web ste: http://research.mcrosoft.com/~zhang/calb/