v = fy c u = fx c z c The Pinhole Camera Model Camera Projection Models

Transcription

1 The Pinhole Camera Model Camera Projetion Models We will introdue dierent amera projetion models that relate the loation o an image point to the oordinates o the orresponding 3D points. The projetion models inlude: ull perspetive projetion model, weak perspetive projetion model, aine projetion model, and orthographi projetion model. Based on simpl trigonometr (or using 3D line equations), we an derive u = v = 2 The Computer Vision Camera Model P u = v = p Image plane optial ais where z is reerred to as isotropi saling.the ull perspetive projetion is non-linear. optial enter 3 4

2 Weak Perspetive Projetion I the relative distane δ (sene depth) between two points o a 3D objet along the optial ais is muh smaller than the average distane to the amera (δz < z 20 ),i.e, 5 6 then u = v = We have linear equations sine all projetions have the same saling ator. Orthographi Projetion As a speial ase o the weak perspetive projetion, when z ator equals, we have u = and v =, i.e., the lins (ras) o projetion are parallel to the optial ais, i.e., the projetion ras meet in the ininite instead o lens enter. This leads to the sizes o image and the objet are the same. This is alled orthgraphi projetion. 7 8

3 Perspetive projetion geometr amera rame C image plane perspetive enter row-olumn 00 rame C p oal length v r Pi u prinipal point image rame C i optial ais z P 00 z objet rame C o Figure : Perspetive projetion geometr 9 0 Projetion Proess Our goal is to go through the projetion proess to understand how an image point (, r) is generated rom the 3D point (,, z). Notations Let P =(z) t be a 3D point in objet rame and U =(uv) t the orresponding image point in the image rame beore digitization. Let X =( ) t be the oordinates o P in the amera rame and p =(r) t be the oordinates o U in the row-olumn rame ater digitization. 3D point in objet rame Aine transormation 3D point in amera rame Perspetive projetion Image point in image rame Spatial Sampling Image point in row-olumn rame 2

4 Relationships between dierent rames Between amera rame (C ) and objet rame (C o ) X = RX + T () X is the 3D oordinates o P w.r.t the objet rame. R is the rotation matri and T is the translation vetor. R and T spei the orientation and position o the objet rame relative to the amera rame. R and T an be parameterized as r r 2 r 3 R = r 2 r 22 r 23 r 3 r 32 r 33 T = t t t z r i =(r i,r i2,r i3 ) be a 3 row vetor, R anbewrittenas R = r r 2 r Between image rame (C i ) and amera rame (C ) Perspetive Projetion: Substituting the parameterized T and R into equation ields r r 2 r 3 = r 2 r 22 r 23 + r 3 r 32 r 33 z t t t z (2) Hene, X = u = v = = λ u v (3) where λ = z is a salar and is the amera oal length. 5 6

5 Relationships between dierent rames (ont d) Between image rame (C i ) and row-ol rame (C p ) (spatial quantization proess) = s 0 u + 0 (4) r 0 s v r 0 where s and s are sale ators (piels/mm) due to spatial quantization. 0 and r 0 are the oordinates o the prinipal point in piels Combining equations to 4 ields Collinearit Equations = s r + r 2 + r 3 z + t r 3 + r 32 + r 33 z + t z + 0 r = s r 2 + r 22 + r 23 z + t r 3 + r 32 + r 33 z + t z + r 0 relative to C p 7 8 Homogeneous Coordinate Sstem In homogeneous oordinate sstem, is hanged to r r X X Y Y is hanged to Z Z Homogeneous sstem: perspetive projetion In homogeneous oordinate sstem, equation 3 ma be rewritten as u 0 0 λ v = 0 0 (5) 0 0 Note λ =. 9 20

6 Homogeneous Sstem: Spatial Quantization Similarl, in homogeneous sstem, equation 4 ma be rewritten as s 0 0 u r = 0 s r 0 v (6) 0 0 Homogeneous sstem: quantization + projetion Substituting equation 5 into equation 6 ields s λ r = 0 s r where λ =. (7) 2 22 Homogeneous sstem: Aine Transormation Homogeneous sstem: ull perspetive Combining equation 8 with equation 7 ields In homogeneous oordinate sstem, equation 2 an be epressed as r r 2 r 3 t r 2 r 22 r 23 t = r 3 r 32 r 33 t z z (8) λ r = s r + 0 r 3 s r 2 + r 0 r 3 r 3 s t + 0 t z s t + r 0 t z t z z } {{ } P where r, r 2,andr 3 are the row vetors o the rotation matri R, λ = is a saler and matri P is alled the homogeneous projetion matri. (9) P = WM 23 24

7 Weak Perspetive Camera Model where s 0 0 W = 0 s r ( ) M = R T W is oten reerred to as the intrinsi matri and M as eterior matri. Sine P = WM =[WR WT], orp to be a projetion matri, Det(WR) 0, i.e., Det(W ) 0. For weak perspetive projetion, we have, i.e., r 3 X + t z Hene, Hene, Or u v u v u = v = = z = Sine We have Sine r r = = s 0 0 u 0 s r 0 v 0 0 s s r =[R 2 T 2 ] z where R 2 is the irst two rows o R and T 2 is the irst two elements o T. Or = R 2 T 2 z z Hene, r = s s r 0 R 2 T 2 z z 27 28

8 = z s s r 0 R 2 T = z s s r 0 R 2 T z z Weak Perspetive Camera Model The weak perspetive projetion matri is s r P weak = s r 2 s t + 0 s t + r 0 (0) 0 3 where r and r 2 are the irst two rows o R 2 and = r 3 X + t z Weak Projetion Camera Model Another possible solution is as ollows 3 32

9 Orthographi Projetion Camera Model Under orthographi projetion, projetion is parallel to the amera optial ais. thereore we have whih an be appromiated b. u = v = The orthographi projetion matri an thereore be obtained as s r s t + 0 P orth = s r 2 s t + r 0 () 0 Aine Camera Model A urther simpliiation rom weak perspetive amera model is the aine amera model, whih is oten assumed b omputer vision researhers due to its simpliit. The aine amera model assumes that the objet rame is loated on the entroid o the objet being observed. As a result, we have t z,the aine perspetive projetion matri is s r s t + 0 t z P aine = s r 2 s t + r 0 t z (2) 0 t z Aine amera model represents the irst order approimation o the ull perspetive projetion amera model. It still onl gives an approimation and is no longer useul when the objet is lose to the amera or the amera has a wide angle o view Non-ull perspetive Projetion Camera Model The weak perspetive projetion, aine, and orthographi amera model an be olletivel lassiied as non-perspetive projetion amera model. In general, the projetion matri or the non-perspetive projetion amera model p p 2 p 3 p 4 λ r = p 2 p 22 p 23 p p z 34 Dividing both sides b p 34 (note λ = p 34 ) ields = M 2 3 r + v v z where m ij = p ij /p 34 and v = p 4 /p 34, v = p 24 /p 34 For an given reerene point ( r,r r ) in image and ( 0, 0,z 0 ) in spae, the relative oordinates (, r) in image and (, ȳ, z) in spae are r = r r and ȳ r r = r r z z z r It ollows that the basi projetion equation or the aine and weak perspetive model in terms o relative oordinates is r = M 2 3 ȳ z An non-perspetive projetion amera M 2 3 has 3 independent parameters

10 The reerene point is oten hosen as the entroid sine entroid is preserved under either aine or weak perspetive projetion. Given the weak projetion matri P, s r s t + 0 P = s r 2 s t + r 0 0 The M matri is M = sr s r 2 = z s r s r 2 = z s 0 r 0 s r 2 For aine projetion, = t z, or orthographi projetion, =.Iwe assume s = s, then M = s r Then, we have onl our parameters: three rotation angles and a sale ator. r Rotation Matri Representation: Euler angles Rotation Matri Representation: Euler angles X Assume rotation matri R results rom suessive Euler rotations o the amera rame around its X ais b ω, its one rotated Y ais b φ, and its twie rotated Z ais b κ, then R(ω, φ, κ) =R X (ω)r Y (φ)r Z (κ) pan angle swing angle Z (optial ais) where ω, φ, and κ are oten reerred to as pan, tilt, and swing angles respetivel. tilt angle Y 39 40

11 R (ω) = R (φ) = R z (κ) = osω sin ω 0 sin ω os ω os φ 0 sin φ 0 0 sin φ 0 osφ os κ sin κ 0 sin κ os κ Rotation Matri: Rotation b a general ais Let the general ais be ω =(ω,ω,ω z ) and the rotation angle be θ. The rotation matri R resulting rom rotating around ω b θ an be epressed as Rodrigues rotation ormula gives an eiient method or omputing the rotation matri Quaternion Representation o R The relationship between a quaternion q =[q 0,q,q 2,q 3 ] and the equivalent rotation matri is Here the quaternion is assumed to have been saled to unit length, i.e., q =. The ais/angle representation ω/θ is strongl related to a quaternion, aording to the ormula os(θ/2) ω sin(θ/2) ω sin(θ/2) ω z sin(θ/2) where ω =(ω,ω,ω z ) and ω =

12 R s Orthnormalit The rotation matri is an orthnormal matri, whih means its rows (olumns) are normalized to one and the are orthonal to eah other. The orthnormalit propert produes R t = R Interior Camera Parameters Parameters ( 0,r 0 ), s, s, and are olletivel reerred to as interior amera parameters. The do not depend on the position and orientation o the amera. Interior amera parameters allow us to perorm metri measurements, i.e., to onvert piel measurements to inh or mm Camera Calibration and Pose Estimation Eterior Camera Parameters Parameters like Euler angles ω, φ, κ, t, t, and t z are olletivel reerred to as eterior amera parameters. The determine the position and orientation o the amera. The purpose o amera alibration is to determine intrinsi amera parameters: 0,r 0, s, s,and. Camera alibration is also reerred to as interior orientation problem in photogrammetr. The goal o pose estimation is to determine eterior amera parameters: ω, φ, κ, t, t, and t z. In other words, pose estimation is to determine the position and orientation o the objet oordinate rame relative to the amera oordinate rame or vie versus

13 D Perspetive Projetion Invariants C Distanes and angles are invariant with respet to Eulidian (aine) transormation (rotation and translation). The are no longer the ase under B perspetive projetion. The most important invariant with respet to perspetive projetion is alled ross ratio. Itisdeined as ollows: A τ( A,B,C,D)= AC BC AD BD Cross-ratio is preserved under perspetive projetion Projetive Invariant or non-ollinear points Cross ratio o intersetion points between a set o penil o 4 lines and another line are onl untion o the angles among the penil lines, independent o the intersetion points on the lines. ross-ratio ma be used or ground plane detetion rom multiple image rames. penil PA, PB, PC, PD does not depend on the point P. This means given A,B,C, and D, all points P on the same ellipse should satis Chasles s theorem. This theorem ma be used or ellipse detetion. Chasles theorem: Let A, B, C, D be distint points on a (non-singular) oni (ellipse, irle,..). I P is another point on the oni then the ross-ratio o intersetions points on the See setion 9.3 and 9.4 o Daves book. 5 52