Lecture 1.2 Pose in 2D and 3D Thomas Osahl
Motivation For the inhole camera, the corresondence between observed 3D oints in the world and 2D oints in the catured image is given by straight lines through a common oint (inhole) This corresondence can be described by a mathematical model known as the ersective camera model or the inhole camera model This model can be used to describe the imaging geometry of many modern cameras, hence it lays a central art in comuter vision 2
Motivation efore we can study the ersective camera model in detail, we need to exand our mathematical toolbox We need to be able to mathematically describe the osition and orientation of the camera relative to the world coordinate frame lso we need to get familiar with some basic elements of rojective geometry, since this will make it MUCH easier to describe and work with the ersective camera model 3
Introduction In comuter vision we are often interested in the geometrical asects of imaging Points in the world ixels in an image Pixels in image 1 ixels in image 2 In order to exress and study geometrical roblems related to imaging, we first need to know how to describe the osition and orientation of objects Position and orientation is together known as ose 4
Introduction y reresenting all relevant objects by coordinate frames, it is ossible to numerically reresent the ose of one object relative to another In the following we will look at ose General roerties Reresentation in 2D Reresentation in 3D 5
General roerties y y x ξ x Let us denote by ξ the ose of frame relative to frame We can think of ξ as the translation and rotation required in order to make coincide with 6
General roerties y y P x ξ x oint P can be described with resect to either frame These descritions are related by the ose Formally we write this as = ξ 7
General roerties y y P x x ξ Note that and are different vectors!!! oint P can be described with resect to either frame These descritions are related by the ose Formally we write this as = ξ 8
General roerties P y ction on oints = ξ y ξ x x 9
General roerties P y ction on oints = ξ y ξ x Comosition ξ C = ξ ξ C y x ξ y x ξ C y C x ξ C C x C 10
General roerties P y ction on oints = ξ y ξ x Comosition ξ C = ξ ξ C y x ξ y x ξ C y C x ξ C C x C What about ξ ξ? 12
General roerties P y ction on oints = ξ y ξ x Comosition ξ C Inverse = ξ ξ C ξ = ξ Neutral element 0 = ξ 0 ξ = ξ ξ 0 = ξ ξ ξ = 0 ξ ξ = 0 y y x ξ x x ξ y y ξ C x x ξ C y C C x C 13
Examle Point observed by the camera P Camera C W V World Vehicle 14
Examle Point observed by the camera P Camera C C V ξ C W V W ξ V World Vehicle 15
Examle Point observed by the camera P Camera C C W ξ C = W ξ V V ξ C W ξ C V ξ C W V W ξ V World Vehicle 16
Examle P Point with a known osition relative to the camera Camera C C V V W ξ C W ξ C = W ξ V = Vξ C V ξ C C V ξ C W V W ξ V World Vehicle 17
Examle P Point with a known osition relative to the camera Camera C C V W V W ξ C = W ξ V = Vξ C V ξ C C W ξ C W = W ξ V Vξ C C V ξ C W V W ξ V World Vehicle 18
Investigating ose in 2D y y P x ξ x Given two 2D frames, and, how can we reresent the ose ξ? We need a numerical reresentation in order to comute = ξ 19
Investigating ose in 2D y y P x ξ x Recall that we can think of ξ as the translation and rotation required in order to make coincide with 20
Investigating ose in 2D y y P θ x t x Recall that we can think of ξ as the translation and rotation required in order to make coincide with To coincide with, must undergo a translation t and a rotation by an angle θ 21
Investigating ose in 2D y y y P y x x t θ t t x x Let = x, y T, = x, y T and t = t, t T 22
Investigating ose in 2D y y y P t y x θ x t t x x Let = x, y T, = x, y T and t = t, t T From the figure we can see that x= t + xcosθ ysinθ x y= t + xsinθ + ycosθ y 23
Investigating ose in 2D y y y P t y x θ x t t x x Let = x, y T, = x, y T and t = t, t T xcosθ From the figure we can see that x= t + xcosθ ysinθ x y= t + xsinθ + ycosθ y ysinθ θ y ysinθ x θ xcosθ 24
Investigating ose in 2D y y y P t y x θ x t t x x In matrix form x t x cosθ sinθ x = + y t y sinθ cosθ y = t + R 25
Investigating ose in 2D x t x cosθ sinθ x = + y t y sinθ cosθ y = t + R Can we reresent the ose ξ by the air R, t? 26
Investigating ose in 2D x t x cosθ sinθ x = + y t y sinθ cosθ y = t + R Can we reresent the ose ξ by the air R, t? = ξ = R + t ( R, t ) ( R R, R t t ) ( R, R t ) ξ = ξ ξ = + C C C C C C ξ T T C C C 27
Investigating ose in 2D x t x cosθ sinθ x = + y t y sinθ cosθ y = t + R Can we reresent the ose ξ by the air R, t? = ξ = R + t ( R, t ) ( R R, R t t ) ( R, R t ) ξ = ξ ξ = + C C C C C C ξ T T C C C Yes, but there is a better otion! 29
Investigating ose in 2D Observe the following equivalence x x x x R t R = + t y y y y = 12 1 1 0 1 = R + t = T 30
Investigating ose in 2D Observe the following equivalence x x x x R t R = + t y y y y = 12 1 1 0 1 = R + t = T Can we reresent the ose ξ by the matrix T? 31
Investigating ose in 2D Observe the following equivalence x x x x R t R = + t y y y y = 12 1 1 0 1 = R + t = T Can we reresent the ose ξ by the matrix T? = ξ = T ξc = ξ ξc TC = T TC 1 ξ T Yes, and the algebraic roerties are nice! 32
Investigating ose in 2D ut x x x x R t R = + t y y y y = 12 1 1 0 1 T We are describing oints in the lane with 3 coordinates desite that they only have 2 degrees of freedom The non linear transformation then becomes a linear transformation What is going on? 34
Investigating ose in 2D We have discovered some basic constructions from rojective geometry x x R = + y y x x R t y = y 12 1 1 0 1 T t 35
Investigating ose in 2D We have discovered some basic constructions from rojective geometry Non-linear transformation of the Euclidean lane R 2 Points in the Euclidean lane are described by Cartesian coordinates x x R = + y y x x R t y = y 12 1 1 0 1 T t 36
Investigating ose in 2D We have discovered some basic constructions from rojective geometry Non-linear transformation of the Euclidean lane R 2 Points in the Euclidean lane are described by Cartesian coordinates x x R = + y y x x R t y = y 12 1 1 0 1 T t Linear transformation of the rojective lane 2 Points in the rojective lane are described by homogeneous coordinates This means that they are only unique u to scale, i.e. x, y, 1 = λλ, λλ, λ λ R\ 0 The matrix reresenting the rojective transformation is also homogeneous, i.e. T = λ T λ R\ 0 37
Investigating ose in 2D Euclidean geometry ξ R, t Comlicated algebra Projective geometry ξ T = R Simle algebra t 0 1 = ξ = R + t ( R, t ) ( R R, R t t ) ( R, R t ) ξ = ξ ξ = + C C C C C C ξ T T C C C = ξ = T ξc = ξ ξc TC = T TC 1 ξ T Many roblems in comuter vision are easier to exress and solve if we choose to think of oints and transformations in terms of rojective geometry lgebra and comutations become simler 38
Reresenting ose in 2D The ose of relative to can be reresented by a homogeneous transformation T SS 2 cosθ sinθ t x R t ξ T = = sinθ cosθ ty 1 0 0 0 1 Points are reresented in homogeneous coordinates = T x x R t y = y 12 1 1 0 1 Proerties = ξ = T ξc = ξ ξc TC = T TC 1 ξ T y P ξ y x x 39
Examle P y y θ x t x Let t = 4,1 T, = 2,3 T and θ = 27 Determine the ose of of relative to, i.e. T Determine the coordinates of P in, i.e. 40
Examle From the revious slides we know that T cosθ sinθ t x cos 27 sin 27 4 0.8910 0.4540 4 = sinθ cosθ t y sin 27 cos 27 1 0.4540 0.8910 1 = 0 0 1 0 0 1 0 0 1 This allows us to comute = T x 0.8910 0.4540 4 2 4.4200 y 0.4540 0.8910 1 3 4.5810 = = = 4.4200 4.5810 1 0 0 1 1 1 This can also be verified by drawing [ ] T 41
Examle y 4.5810 P 4 y 3 3 x 2 2 1 27 t 1 2 3 4 4.4200 x 42
Examle You can visualize this examle in matlab using the toolboxes created by Peter Cork Robotics Toolbox Machine Vision Toolbox www.etercorke.com/toolboxes.html 43
Reresenting ose in 3D The ose of relative to can be reresented by a homogeneous transformation T SS 3 11 12 13 x R t r21 r22 r23 ty ξ T 0 1 r31 r32 r33 tz Proerties r r r t = = 0 0 0 1 = ξ = T ξc = ξ ξc TC = T TC 1 ξ T Points are reresented in homogeneous coordinates = T y x x y R t y = z 012 1 z 1 1 P y ξ z x z x 44
Reresenting ose in 3D The main difference between 3D and 2D is that rotation is far less intuitive in 3D lso there are several different reresentations of rotation in 3D Orthonormal rotation matrix R SS 3 Euler angles θ 1, θ 2, θ 3 ngle-axis θ, e or just θ = θe Unit quaternions q = r + xx + yy + zz Hence there are several ways to reresent ose Rotation matrix and translation vector R, t Homogeneous transformation T Euler angles and translation vector θ 1, θ 2, θ 3, t ngle-axis and translation vector θ, e, t Unit quaternion and translation vector q, t 45
Reresenting rotation in 3D y Orientation of relative to How should rotate to coincide with y v x Orthonormal rotation matrix R SS 3 r11 r12 r13 R = r21 r22 r 23 r31 r32 r 33 Proerties ( R ) det = 1 R = R 1 T RC = R RC v = R v z z Nice algebraic roerties Comutations are simle Provides little intuitive understanding x 46
Reresenting rotation in 3D R R R x y z 1 0 0 θ = 0 cosθ sinθ 0 sinθ cosθ cosθ 0 sinθ = 0 1 0 sinθ 0 cosθ cosθ sinθ 0 θ = sinθ cosθ 0 0 0 1 ( ) ( θ ) ( ) Euler angles θ 1, θ 2, θ 3 describes rotation in terms of basic rotations about 3 coordinate axis that must be secified 12 comositions: xxx, xxx, yyy, yyy, zzz, zzz, xxx, xxx, yyy, yyy, zzz, zzz E.g. R xxx = R x θ 1 R y θ 2 R z θ 3 Suffers from singularities θ 2 = ± π for sequences without reetition 2 θ 2 0, π for sequences with reetition It is ossible to determine the Euler angles that corresonds to a given rotation matrix 2 2 ( r r r ) 13 11 12 π π θy = atan2, + < θy < 2 2 r r r R = r r r θ = π < θ < π 11 12 13 r23 r33 xyz 21 22 23 x atan2, x cosθy cosθ y r31 r32 r 33 = r r < < 12 11 θz atan2, π θz π cosθy cosθ y 47
Reresenting rotation in 3D n angle-axis air θ, v describes rotation in terms of an angle and an axis of revolution θ v Euler s rotation theorem states that any sequence of rotations in 3 dimensions is equivalent to a single rotation θ about an axis v The rotation matrix R corresonding to an angle-axis air θ, v can be found by Rodrigues rotation formula T [ v] ( θ)( vv ) R= I + sinθ + 1 cos I 3 3 Where v is the matrix reresentation of the cross roduct 0 v3 v2 [ v] = v3 0 v 1 v2 v1 0 The angle-axis air θ, v corresonding to a rotation matrix R can be found from the eigenvalues and eigenvectors of R 48
Reresenting rotation in 3D Unit quaternions are commonly used to work with rotation/orientation in 3D quaternion q is a number, with 1 real term and 3 imaginary terms q = q 0 + q 1 i + q 2 j + q 3 k = q 0 + v i 2 = j 2 = k = iii = 1 Conjugate quaternion q = q 0 q 1 i q 2 j q 3 k = q 0 v unit quaternion satisfy q = q q = q + q + q + q = * 2 2 2 2 0 1 2 3 1 θ Quaternion multilication q = q + v + w ( )( ) 0 0 = q 0 0 v w+ q0w+ 0v+ v w Comosing two unit quaternions takes 16 multilications and 12 additions, while comosing two rotation matrixes takes 27 multilications and 18 additions v θ q0 = cos 2 θ v = sin e 2 3 49
Reresenting rotation in 3D The unit quaternion corresonding to the rotation matrix R is given by 1 q = 1 + r + r + r 2 r32 r23 q1 = 4q0 r13 r31 q2 = 4q0 r21 r12 q3 = 4q 0 11 22 33 0 The rotation matrix corresonding to the unit quaternion q = q 0 + q 1 i + q 2 j + q 3 k is given by 2 2 q2 q3 ( qq 1 2 qq 0 3) ( qq 1 3+ qq 0 2) 2 2 ( 1 2 0 3) 1 3 ( 2 3 0 1) 2 2 ( qq qq) ( qq + qq) q q 1 2 2 2 2 R= 2 qq + qq 1 2q 2q 2 qq qq 2 2 1 2 2 1 3 0 2 2 3 0 1 1 2 SLERP Sherical linear interolation n algorithm for determining a rotation artway between two given rotations Useful for visualizing rotations 50
Summary Pose General roerties dditional information Szeliski 2.1.1, 2.1.2, 2.1.4 Reresenting ose in 2D Homogeneous transformations Reresenting ose in 3D Homogeneous transformations Other alternatives Reresenting rotation in 3D Rotation matrix Euler angles ngle-axis Unit quaternion 51