CS4495/6495 Introduction to Computer Vision. 3C-L3 Calibrating cameras

Transcription

1 CS4495/6495 Introducton to Computer Vson 3C-L3 Calbratng cameras

2 Fnally (last tme): Camera parameters Projecton equaton the cumulatve effect of all parameters: M (3x4) f s x ' c R 0 I T x x1 0 af y ' x x c x3 01 x ntrnscs projecton rotaton translaton

3 Fnally (last tme): Camera parameters Projecton equaton the cumulatve effect of all parameters: x X sx * * * * Y sy * * * * Z s * * * * 1 M X

4 Calbraton How to determne M?

5 Calbraton usng known ponts Place a known object n the scene dentfy correspondence between mage and scene compute mappng from scene to mage

6 Resectonng Estmatng the camera matrx from known 3D ponts Projectve Camera Matrx: p K R t P M P X w * u m m m m Y w * v m m m m Z w m m m m

7 Drect lnear calbraton - homogeneous One par of equatons for each pont u w * u m m m m Y v w * v m m m m Z 1 w m m m m u v m X m Y m Z m m X m Y m Z m m X m Y m Z m m X m Y m Z m X 1

8 Drect lnear calbraton - homogeneous u m X m Y m Z m m X m Y m Z m One par of equatons for each pont v m X m Y m Z m m X m Y m Z m u ( m X m Y m Z m ) m X m Y m Z m v ( m X m Y m Z m ) m X m Y m Z m

9 Drect lnear calbraton - homogeneous u ( m X m Y m Z m ) m X m Y m Z m v ( m X m Y m Z m ) m X m Y m Z m One par of equatons for each pont m m m m m m m m m m m m

10 Drect lnear calbraton - homogeneous Ths s a homogenous set of equatons. When over constraned, defnes a least squares problem mnmze Am Snce m s only defned up to scale, solve for unt vector m* Soluton: m* = egenvector of A T A wth smallest egenvalue Works wth 6 or more ponts

11 Drect lnear calbraton - homogeneous m m m m m m m m m m A m m 0 m 2n n

12 The SVD (sngular value decomposton) trck Fnd the m that mnmzes Am subject to m =1. Let A = UDV T (sngular value decomposton, D dagonal, U and V orthogonal) Therefor mnmzng UDV T m But, UDV T m = DV T m and m = V T m Thus mnmze DV T m subject to V T m = 1

13 The SVD (sngular value decomposton) trck Thus mnmze DV T m subject to V T m Let y = V T m Now mnmze Dy subject to y = 1. = 1 But D s dagonal, wth decreasng values. So Dy mnmum s when y = 0,0,0, 0,1 T Snce y = V T m, m = Vy snce V orthogonal Thus m = Vy s the last column n V.

14 The SVD (sngular value decomposton) trck Thus m = Vy s the last column n V. And, the sngular values of A are square roots of the egenvalues of A T A and the columns of V are the egenvectors. (Show ths? Nah ) Recap: Gven Am=0, fnd the egenvector of A T A wth smallest egenvalue, that s m.

15 Drect lnear calbraton - nhomogeneous Another approach: 1 n lower r.h. corner for 11 d.o.f X u m m m m Y v m m m m Z 1 m m m

16 Drect lnear calbraton - nhomogeneous Dangerous f m 23 s really (near) zero!

17 Drect lnear calbraton (transformaton) Advantages: Very smple to formulate and solve. Can be done, say, on a problem set These methods are referred to as algebrac error mnmzaton.

18 Drect lnear calbraton (transformaton) Dsadvantages: Doesn t drectly tell you the camera parameters (more n a bt) Approxmate: e.g. doesn t model radal dstorton Hard to mpose constrants (e.g., known focal length) Mostly: Doesn t mnmze the rght error functon

19 Drect lnear calbraton (transformaton) For these reasons, prefer nonlnear methods: Defne error functon E between projected 3D ponts and mage postons: E s nonlnear functon of ntrnscs, extrnscs, and radal dstorton Mnmze E usng nonlnear optmzaton technques e.g., varants of Newton s method (e.g., Levenberg Marquart)

20 Geometrc Error m nm ze E d ( x, xˆ ) Predcted Image locatons x X m n d ( x, MX ) M M

21 Gold Standard algorthm (Hartley and Zsserman) Objectve Gven n 6 3D to 2D pont correspondences {X x }, determne the Maxmum Lkelhood Estmaton of M

22 Gold Standard algorthm (Hartley and Zsserman) Algorthm () Lnear soluton: (a) (Optonal) Normalzaton: X U X x =Tx (b) Drect Lnear Transformaton mnmzaton () Mnmze geometrc error: usng the lnear estmate as a startng pont mnmze the geometrc error: m n d ( x, M X ) M

23 Gold Standard algorthm (Hartley and Zsserman) () Denormalzaton: M=T -1 M U

24 Fndng the 3D Camera Center from M M encodes all the parameters. So we should be able to fnd thngs lke the camera center from M. Two ways: pure way and easy way

25 Fndng the 3D Camera Center from M Slght change n notaton. Let: M s(3x4) b s last column of M The center C s the null-space camera of projecton matrx. So f fnd C such that: M C = 0 that wll be the center. Really M Q b

26 Fndng the 3D Camera Center from M Proof: Let X be somewhere between any pont P and C X λ P (1 λ) C

27 Fndng the 3D Camera Center from M Proof: Let X be somewhere between any pont P and C And the projecton: X λ P (1 λ) C x M X λ M P (1 λ) M C

28 Fndng the 3D Camera Center from M Proof: Let X be somewhere between any pont P and C And the projecton: X λ P (1 λ) C x M X λ M P (1 λ) M C For any P, all ponts on PC ray project on mage of P, therefore MC must be zero. So the camera center has to be n the null space.

29 Fndng the 3D Camera Center from M Now the easy way. A formula! If M =[Q b] then: C Q 1 1 b

30 Alternatve: mult-plane calbraton Images courtesy Jean-Yves Bouguet, Intel Corp.

31 Alternatve: mult-plane calbraton Advantages Only requres a plane Don t have to know postons/orentatons Good code avalable onlne! OpenCV lbrary Matlab verson by Jean-Yves Bouget: Zhengyou Zhang s web ste: