Camera: optical system - PDF Free Download

Camera: optical system Y dαα α ρ ρ curvature radius Z thin lens small angles: α Y ρ α Y ρ

lens refraction inde: n Y incident light beam deviated beam θ '' α θ ' α sin( θ '' α ) sin( θ ' α ) n θ α θ α deviation angle? Δθ θ θ Δθ ( n ) Y ( + ) ρ ρ sin( θ α) sin( θ ' α ) ' n Z

a) Y Δθ b) f Δθ Y hin lens rules f ( n )( ρ + ) ρ parallel rays converge onto a focal plane beams through lens center: undeviated independent of y Y Δθ f

Hp: Z >> a r f the image of a point P belongs to the line (P,O) P image plane O p p image of P image plane line(o,p) interpretation line of p: line(o,p) locus of the scene points projecting onto image point p

Projective D geometry Notes based on di R.Hartley e A.Zisserman Multiple view geometry

Projective D Geometry Points, lines & conics ransformations & invariants D projective geometry and the Cross-ratio

Homogeneous coordinates Homogeneous representation of lines a + by + c ( a,b,c) ( ka ) + ( kb) y + kc, k ( a,b,c ) ~ k( a,b,c) equivalence class of vectors, any vector is representative Set of all equivalence classes in R 3 (,,) forms P Homogeneous representation of points, y on l a,b,c if and only if a + by + c ( ) ( ) (,y, )( a,b,c) (,y, ) l (, y,) ~ k(, y,), k he point lies on the line l if and only if ll ( ) Homogeneous coordinates,, Inhomogeneous coordinates (, y) 3 but only DOF

Points from lines and vice-versa Intersections of lines he intersection of two lines l and l' is l l' Line joining two points he line through two points and ' is Eample l Line joining two points: parametric equation A point on the line through two points and is y + θ ' y

Ideal points and the line at infinity Intersections of parallel lines l ( a, b, c) and l' ( a, b, c' ) l l' ( b, a,) Eample ( b, a) ( a,b) tangent vector normal direction Ideal points (,, ) Line at infinity l (,,) P R l Note that in P there is no distinction between ideal points and others

Duality l l l l l' l ' Duality principle: o any theorem of -dimensional projective geometry there corresponds a dual theorem, which may be derived by interchanging the role of points and lines in the original theorem

Conics Curve described by nd -degree equation in the plane a + by + cy + d + ey + f or homogenized a α, 3 y α + b + c + d3 + e3 + f3 or in matri form a b / d / C with C b / c e / d / e / f 3 5DOF: { a : b : c : d : e : f }

Five points define a conic For each point the conic passes through + + + + + f ey d cy y b a i i i i i i or ( ),,,,, c f y y y i i i i i i ( ) f e d c b a,,,,, c 5 5 5 5 5 5 4 4 4 4 4 4 3 3 3 3 3 3 c y y y y y y y y y y y y y y y stacking constraints yields

Pole-polar relationship he polar line lc of the point with respect to the conic C intersects the conic in two points. he two lines tangent to C at these points intersect at

Polarity: cross ratio Cross ratio of 4 colinear points y + θ (with i,..,4) ratio of ratios θ θ -θ -θ 3 4 θ θ i -θ -θ 3 4 i Harmonic 4-tuple of colinear points: such that CR-

angent lines to conics he line l tangent to C at point on C is given by lc l C

Dual conics A line tangent to the conic C satisfies l C * l C * In general (C full rank): in fact Dual conics line conics conic envelopes C Line l is the polar line of y : y C l, but since y Cy - - * - - l C C C l C C C -

Degenerate conics A conic is degenerate if matri C is not of full rank e.g. two lines (rank ) C lm + ml e.g. repeated line (rank ) m l C ll l Degenerate line conics: points (rank ), double point (rank) Note that for degenerate conics ( * C ) * C

Projective transformations Definition: A projectivity is an invertible mapping h from P to itself such that three points,, 3 lie on the same line if and only if h( ),h( ),h( 3 ) do. heorem: A mapping h:p P is a projectivity if and only if there eist a non-singular 33 matri H such that for any point in P represented by a vector it is true that h()h Definition: Projective transformation ' h ' h ' 3 h 3 h h h 3 h h h 3 3 33 3 or ' H 8DOF projectivitycollineationprojective transformationhomography

Mapping between planes central projection may be epressed by H (application of theorem)

Removing projective distortion select four points in a plane with known coordinates ' ' ' 3 h h 3 + h + h 3 y + h y + h 3 33 y' ' ' 3 h h 3 + h + h 3 y + h y + h 3 33 ( h3 + h3 y + h33 ) h + h y h3 ( h3 + h3 y + h33 ) h + h y h3 ' + y ' + (linear in h ij ) ( constraints/point, 8DOF 4 points needed) Remark: no calibration at all necessary, better ways to compute (see later)

More eamples

ransformation of lines and conics For a point transformation ' H ransformation for lines l' H - ransformation for conics l C ' H - CH - ransformation for dual conics * C' * HC H

A hierarchy of transformations Projective linear group Affine group (last row (,,)) Euclidean group (upper left orthogonal) Oriented Euclidean group (upper left det ) Alternative, characterize transformation in terms of elements or quantities that are preserved or invariant e.g. Euclidean transformations leave distances unchanged

Class I: Isometries (isosame, metricmeasure) ' y' ε cosθ ε sinθ sinθ cosθ t t y y ε ± orientation preserving: orientation reversing: ε ε ' H E R t 3DOF ( rotation, translation) R R special cases: pure rotation, pure translation I Invariants: length, angle, area

Class II: Similarities (isometry + scale) cos sin sin cos ' ' y t s s t s s y y θ θ θ θ ' t R H s S I R R also know as equi-form (shape preserving) metric structure structure up to similarity (in literature) 4DOF ( scale, rotation, translation) Invariants: ratios of length, angle, ratios of areas, parallel lines

Class III: Affine transformations ' ' y t a a t a a y y ' t A H A non-isotropic scaling! (DOF: scale ratio and orientation) 6DOF ( scale, rotation, translation) Invariants: parallel lines, ratios of parallel segment lengths, ratios of areas ( ) ( ) ( ) φ φ θ DR R R A λ λ D ) )( ( VDV UV UDV A where

Action of affinities and projectivities on line at infinity + v v v v A A t v A A t Line at infinity becomes finite, allows to observe vanishing points, horizon, Line at infinity stays at infinity, but points move along line

Class VI: Projective transformations ' H P A v t v v ( v v ), 8DOF ( scale, rotation, translation, line at infinity) Action: non-homogeneous over the plane Invariants: cross-ratio of four points on a line (ratio of ratios)

Projective geometry of D ( ), ' H 3DOF (-) he cross ratio Cross (,,, ) 3 4,, 3 3,, 4 4, i j i det i j j Invariant under projective transformations

Overview transformations Projective 8dof Affine 6dof Similarity 4dof Euclidean 3dof h h h3 a a sr sr r r h h h a a r r 3 sr sr h3 h 3 h 33 t t y t t y t t y Concurrency, collinearity, order of contact (intersection, tangency, inflection, etc.), cross ratio Parallellism, ratio of areas, ratio of lengths on parallel lines (e.g midpoints), linear combinations of vectors (centroids). he line at infinity l Ratios of lengths, angles. he circular points I,J lengths, areas.

Number of invariants? he number of functional invariants is equal to, or greater than, the number of degrees of freedom of the configuration less the number of degrees of freedom of the transformation e.g. configuration of 4 points in general position has 8 dof (/pt) and so 4 similarity, affinity and zero projective invariants

Recovering metric and affine properties from images Parallelism Parallel length ratios Angles Length ratios

he line at infinity l l l A t A H A he line at infinity l is a fied line under a projective transformation H if and only if H is an affinity Note: not fied pointwise

Affine properties from images projection (affine) rectification A P l l l H H 3 ' [ ], l' 3 3 l l l l in fact, any point on l is mapped to a point at the

Affine rectification v v l l l 3 l l 4 v l3 l4 v l l l v v

he circular points I i J i I cos sin sin cos I I i se i t s s t s s i y S θ θ θ θ θ H he circular points I, J are fied points under the projective transformation H iff H is a similarity

he circular points circular points l + + d3 + e3 + f3 + 3 I (, i,) Intersection points between any circle and l J, i, Algebraically, encodes orthogonal directions I (,, ) + i(,, ) ( )

Circular points invariance {I,J} l any circumference Similarity: circ circ Similarity: circ l circ l Similarity: {I,J} {I,J} circular points: invariant under similarity

* C Conic dual to the circular points * C IJ + JI : line conic set of lines through any of the circular points C * C * H C * S H S C he dual conic * is fied conic under the projective transformation H iff H is a similarity * C Note: has 4DOF l is the null vector

Angles ( )( ) cos m m l l m l m l + + + θ ( ) 3,, l l l l ( ) 3,, m m m m Euclidean: Projective: ( )( ) m m l l m l cos * * * C C C θ m l * C (orthogonal)

Metric properties from images ( ) ( ) ( ) ( ) ( ) ( ) v v v v ' * * * * K K KK H H C H H H H H C H H H H H H C H H H C A P A P A P S S A P S A P S A P U U C ' * Rectifying transformation from SVD H U

Normally: SVD (Singular Value Decomposition) U V C c b a ' * But is symmetric ' * C ' ' * * C VDU UDV C and SVD is unique V U with U and V orthogonal U U C ' * Why? Observation : HU orthogonal (33): not a P isometry

Once the image has been affinely rectified A H A C H C * * ' ' * t t KK K K C Metric from affine

Metric from affine ( ) 3 3 m m m l l l KK ( )( ),,,, + + k k k k k m l m l m l m l

Metric from projective ( ) ( ) ( ) ( ) c,,,,, 3 3 3 3 3 3 5..5.5 + + + m l m l m l m l m l m l m l m l m l ( ) v v v v 3 3 m m m l l l K K KK

Fied points and lines H e λ e (eigenvectors H fied points) (λ λ pointwise fied line) H l λ l (eigenvectors H - fied lines)

Projective 3D geometry

Singular Value Decomposition n n n m m m n m V Σ U A I U U n σ σ σ Λ I V V m n X V X V Σ X V UΣ n n U n V V U V U A σ σ σ + + + Λ Σ Λ Μ Μ Μ Λ Μ Ο Μ Μ Λ Λ σ n σ σ

Singular Value Decomposition Homogeneous least-squares min AX subject to X solution X Vn Span and null-space S L [ U U ]; N [ U ] L 3U4 V V ; R V3V 4 Closest rank r approimation A ~ UΣ ~ V [ ] [ ] S R N Pseudo inverse + A VΣ U A UΣ V ~ Σ diag Σ σ ( σ, σ, Λ, σ,,, ) r Λ + ( Σ + diag σ, σ, Λ, σ,,, ) σ r Λ

Projective 3D Geometry Points, lines, planes and quadrics ransformations П, ω and Ω

3D points 3D point ( X, Y, Z ) in R 3 X ( X, X, X X ) 3, 4 in P 3 X X X X 3,,, Z X 4 X 4 X 4 ( X, Y,,) ( X 4 ) projective transformation X' H X (44-5 dof)

3D plane Planes π X + πy + π3z + π4 π X + πx + π3x 3 + π4x 4 ransformation X' H X - π' H π π X Euclidean n.x ~ representation + d ( π ), π, π3 d n ( ) π 4 X ~ X, Y, Z X 4 d / n Dual: points planes, lines lines

Solve X X X Planes from points 3 π from 3 π [ X X X X ] det 3 X π, X π and X π X (solve π as right nullspace of X ) X3 Or implicitly from coplanarity condition X det X X X 3 4 ( X) ( X ) ( X ) 3 ( X) ( X ) ( X ) 3 ( X) ( X ) ( X 3) 3 3 ( X ) ( X ) ( X ) X D 34 X D 34 + X 3 D 4 X π ( D ) 4D3 34, D34, D4, D3 4 4 3 3 4

Points from planes X π π π 3 X M [ ] 3 X X X M π M M I p ( ) d c b a,,, π a d a c a b p,, X π X π, X π X 3 and from Solve (solve as right nullspace of ) X 3 π π π Representing a plane by its span

Lines W A B λa+ μb (4dof) W * P Q λp+ μq two points A and B two planes P and Q * * W W WW Eample: X-ais W W *

Points, lines and planes W M M π X X W W * M M X π * W π

Quadrics and dual quadrics X QX. 9 d.o.f. (Q : 44 symmetric matri). in general 9 points define quadric 3. det Q degenerate quadric 4. pole polar 5. (plane quadric)conic 6. transformation π * Q π π QX Q' H Q ο ο ο ο ο ο C M QM π : X M - QH - * - Q Q * * HQ H. relation to quadric (non-degenerate). transformation Q'

Quadric classification Rank Sign. Diagonal Equation Realization 4 4 (,,,) X + Y + Z + No real points (,,,-) X + Y + Z Sphere (,,-,-) X + Y Z + Hyperboloid (S) 3 3 (,,,) X + Y + Z Single point (,,-,) X + Y Z Cone (,,,) X + Y Single line (,-,,) X Y wo planes (,,,) X Single plane

Quadric classification Projectively equivalent to sphere: sphere Ruled quadrics: ellipsoid hyperboloid of two sheets paraboloid hyperboloids of one sheet Degenerate ruled quadrics: cone two planes

Hierarchy of transformations Projective 5dof A v t v Intersection and tangency Affine dof A t Parallellism of planes, Volume ratios, centroids, he plane at infinity π Similarity 7dof s R t he absolute conic Ω Euclidean 6dof R t Volume

Screw decomposition Any particular translation and rotation is equivalent to a rotation about a screw ais and a translation along the screw ais. screw ais // rotation ais t + t t //

he plane at infinity A π H A π π A t he plane at infinity π is a fied plane under a projective transformation H iff H is an affinity π (,,,). canical position. contains directions D ( X ), X, X 3, 3. two planes are parallel line of intersection in π 4. line // line (or plane) point of intersection in π

he absolute conic he absolute conic Ω is a (point) conic on π. In a metric frame: X or conic for directions: (with no real points) + X X 4 + X he absolute conic Ω is a fied conic under the projective transformation H iff H is a similarity 3 ( X, X, X ) I( X, X X ) 3, 3. Ω is only fied as a set. Circle intersect Ω in two points 3. Spheres intersect π in Ω

Absolute conic invariance Ω π any sphere Similarity: sphere sphere Similarity: sphere π sphere π Similarity: Ω Ω Ω : invariant under similarity

he absolute conic Euclidean: Projective: cosθ cosθ ( d ) d ( )( d ) d d d ( d ) Ω d ( d d )( Ω d Ω d ) d Ω d (orthogonalityconjugacy) normal plane

he absolute dual quadric * I Ω he absolute conic Ω * is a fied conic under the projective transformation H iff H is a similarity. 8 dof. plane at infinity π is the nullvector of Ω 3. Angles: * cosθ ( * )( * π Ω π π Ω π ) π Ω π

Y P X p c y f O Z y f f X Z Y Z perspective projection -nonlinear -not shape-preserving -not length-ratio preserving

Homogeneous coordinates In D: add a third coordinate, w Point [,y] epanded to [u,v,w] Any two sets of points [u,v,w ] and [u,v,w ] represent the same point if one is multiple of the other [u,v,w] [,y] with u/w, and yv/w [u,v,] is the point at the infinite along direction (u,v)

ransformations translation by vector [d,d y ] scaling (by different factors in and y) rotation by angle θ

Homogeneous coordinates In 3D: add a fourth coordinate, t Point [X,Y,Z] epanded to [,y,z,t] Any two sets of points [,y,z,t ] and [,y,z,t ] represent the same point if one is multiple of the other [,y,z,t] [X,Y,Z] with X/t, Yy/t, and Zz/t [,y,z,] is the point at the infinite along direction (,y,z)

ransformations translation scaling rotation Obs: rotation matri is an orthogonal matri i.e.: R - R

Pinhole camera model Z fy Z fx Z Y X ) /, / ( ),, ( α Z Y X f f Z fy fx Z Y X α

Scene->Image mapping: perspective transformation u v w f with f X Y Z y u w v w With ad hoc reference frames, for both image and scene

Let us recall them O image reference - centered on principal point - - and y-aes parallel to the sensor rows and columns - Euclidean reference c y f Y X scene reference - centered on lens center - Z-ais orthogonal to image plane - X- and Y-aes opposite to image - and y-aes Z

principal point c Actual references are generic y f O principal ais Y X image reference - centered on upper left corner - nonsquare piels (aspect ratio) noneuclidean reference Z scene reference - not attached to the camera

Principal point offset o o v Z fy u Z fx Z Y X ) /, / ( ),, ( + + α principal point o v o u ), ( + + Z Y X v f u f Z Zv fy Zu fx Z Y X o o o o α

CCD camera o y o v f u f K o o v f u f a K

Scene-image relationship wrt actual reference frames image u A u v w ( s a) a u v o o u v w normally, s X scene Y Z R t X

X t R X P u f f v a u o o t K R K t R I P o o v af u f K upper triangular: intrinsic camera parameters scene-camera tranformation etrinsic camera parameters orthogonal (3D rotation) matri P: - degrees of freedom ( if s)

z y o I M X o I M u ) ( o M o I M u i.e., defining [, y, z] o I M o M M m M t K R K P with R K M and m M o

Interpretation of o: u is image of if ( o) λm u i.e., if o + λm u he locus of the points whose image is u is a straight line through o having direction d M u o M m is independent of u o is the camera viewpoint (perspective projection center) line(o, d) Interpretation line of image point u

M K and R Intrinsic and etrinsic parameters from P M K R M R K R K RQ-decomposition of a matri: as the product between an orthogonal matri and an upper triangular matri M and m t o M m ( KR) Kt R K Kt R t t Ro

Camera center null-space camera projection matri P O X λa + ( λ)o PX λpa + ( λ)po For all A all points on AO project on image of A, therefore O is camera center Image of camera center is (,,), i.e. undefined o M m Finite cameras: O d Infinite cameras: o,md

Action of projective camera on point Forward projection PX [ M m] D Md PD Back-projection P O + X P P + P ( PP ) ( λ) P + λo X + - d M M X( λ) μ D - (pseudo-inverse) - M + O - m M - + PP I ( μ - m)

Camera matri decomposition Finding the camera center P O (use SVD to find null-space) X det( [ p,p3,m] ) Y det( [ p,p3,m] ) Z det( [ p,p,m] ) ([ p,p,m]) det Finding the camera orientation and internal parameters M KR (use RQ decomposition ~QR) (if only QR, invert) ( Q R ) - R - - Q

Radial distortion

Data normalization 3 X ~ i UX i u~ i u i (i) translate origin to gravity center (ii) (an)isotropic scaling

Eterior orientation Calibrated camera, position and orientation unkown Pose estimation 6 dof 3 points minimal (4 solutions in general)

Properties of perspective transformations ) vanishing points V image of the point at the along direction d u V M m d M d d M u V the interpretation line of V is parallel to d

d P O V he images of parallel lines are concurrent lines

Properties of perspective transformations ctd. ) cross ratio invariance Given four colinear points ( p, p, p p ) let (,, ) 3, 4 3, be their abscissae 4 CR ( p, p, p p ) 3, 4 3 4 3 4

Cross ratio invariance under perspective transformation a point on the line yz X [, y, z, t] [,,, t ] its image its coordinate u det u, u CR( u, u, u3, u4 ) det u, u det u [ u, v, w] 3 3 det u det u P X p u [ u, w] P4 p t 3, u, u 4 4 4 detp detp 4 4 4 det det,, 3 4 CR(,, 3, 4 ) det,, det det,, 3 belongs to a line detp detp 4 4 det det 3,, 4 4

Object localization : three colinear points geometric model of an object a perspective image of the object position and orientation of the object? calibrated camera: known P M known interpretation lines m C A B B A O C B A π C

a) orientation C A B C V O A B π Cross ratio invariance: a c solve CR( A', B', C', V ) CR( A, B, C, ) for V (image of ) b c V: vanishing point of the direction of (A,B,C) interpretation line of V parallel to (A,B,C) direction M uv

b) position (e.g., distance(o,a)) M u C γ C A B C V O α M u A A B M u V π interpretation lines OA AC sinγ sinα angles α and γ

Object localization : four coplanar points D C O E B A (i) orientation of (A,E,C) (ii) orientation of (B,E,D) (iii) distance (O,A)

Off-side Find vanishing point of the field-bottom line direction b b a a images of symmetric segments

a c b d a and b: c(a+b)/: d : abscissae of the endpoints of a segment abscissa of segment midpoint, point at the infinite along the segment direction CR a c a d b d a c b c ( a, b, c, d ) b c Harmonic 4-tuple (a,b,c,d) (a,b ) and (a, b ) are image of symmetric segments same image of the midpoint c, same vanishing point d

solve a' c' ( a', b', c', d' ) b' c' { CR for CR a' ', b'', c', d' a'' c' a' d' b' d' ( ) b'' c' a'' d' b'' d' c, d system of two linear equations in (c d ) and (c +d ) two degree equation, whose solutions are c and d among the two solutions, the one for d is the value eternal to the range [a,b ]

What can be told from a single image?

Action of projective camera on planes [ ] [ ] p p p p p p p PX 4 4 3 Y X Y X he most general transformation that can occur between a scene plane and an image plane under perspective imaging is a plane projective transformation

Action of projective camera on lines forward projection ( μ) P(A + μb) PA + μpb a μb X + back-projection l PX l PX with Π P l Π X Interpretation plane of line l

[ ] [ ] [ ] P Y X Y X 4 4 4 3 p p p p p p p p p p PX Image of a conic C C - P P therefore - C C' P P

Action of projective camera on conics back-projection of a conic C to cone Q co C Q co

back-projection of a conic C to cone Qco C PX C X P CPX with X Q co X Interpretation cone of a conic C Q co P CP eample: Q co K C [ K ] K CK

Images of smooth surfaces he contour generator Γ is the set of points X on S at which rays are tangent to the surface. he corresponding apparent contour γ is the set of points which are the image of X, i.e. γ is the image of Γ he contour generator Γ depends only on position of projection center, γ depends also on rest of P

Action of projective camera on quadrics apparent contour of a quadric Q dual quadric Q * Q is a plane quadric: the set of planes tangent to Q Π Q * Π Let us consider only those planes that are backprojection of image lines Π P l Π Q * Π l * PQ P l with C * * PQ P its dual is C C *

he plane containing the apparent contour Γ of a quadric Q from a camera center O follows from pole-polar relationship ΠQO he cone with verte V and tangent to the quadric Q is Q CO (V QV)Q - (QV)(QV) Q CO V back-projection to cone

What does calibration give? K[I ] d d K d d cos θ ( )( ) ( )( ) - - - - d d d d (K K ) (K K ) (K - K - ) An image line l defines a plane through the camera center with normal nk l measured in the camera s Euclidean frame. In fact the backprojection of l is P l nk l

he image of the absolute conic Ω d PX KR[I O] KRd mapping between π to an image is given by the planar homogaphy Hd, with HKR absolute conic (IAC), represented by I 3 within π (w) its image (IAC) ( ) - - KK K K ω ( C α H CH ) (i) IAC depends only on intrinsics (ii) angle between two rays cos θ (iii) DIACω * KK (iv) ω K (Cholesky factorization) (v) image of circular points belong to ω (image of absolute conic) ( ω )( ω ) ω

A simple calibration device (i) compute H i for each square (corners (,),(,),(,),(,)) (ii) compute the imaged circular points H i [,±i,] (iii) fit a conic ω to 6 imaged circular points (iv) compute K from ωk - K - through Cholesky factorization ( Zhang s calibration method)

Orthogonality relation cos θ ( )( ) v ωv v ωv v ωv v ωv * l ω l

Calibration from vanishing points and lines

wo-view geometry Epipolar geometry F-matri comp. 3D reconstruction Structure comp.

hree questions: (i) Correspondence geometry: Given an image point in the first view, how does this constrain the position of the corresponding point in the second image? (ii) Camera geometry (motion): Given a set of corresponding image points { i i }, i,,n, what are the cameras P and P for the two views? (iii) Scene geometry (structure): Given corresponding image points i i and cameras P, P, what is the position of (their pre-image) X in space?

he epipolar geometry C,C,, and X are coplanar

he epipolar geometry What if only C,C, are known?

he epipolar geometry All points on π project on l and l

he epipolar geometry Family of planes π and lines l and l Intersection in e and e

he epipolar geometry epipoles e,e intersection of baseline with image plane projection of projection center in other image vanishing point of camera motion direction an epipolar plane plane containing baseline (-D family) an epipolar line intersection of epipolar plane with image (always come in corresponding pairs)

Eample: converging cameras

Eample: motion parallel with image plane

Eample: forward motion e e

he fundamental matri F algebraic representation of epipolar geometry α l' we will see that mapping is (singular) correlation (i.e. projective mapping from points to lines) represented by the fundamental matri F

he fundamental matri F geometric derivation ' H π [ e' ] Hπ F l' e' ' mapping from -D to -D family (rank )

he fundamental matri F algebraic derivation ( λ) P λc X + + ( + P P I) l P'C P' P + P + X( λ) F [ ] + e' P' P (note: doesn t work for CC F)

he fundamental matri F correspondence condition he fundamental matri satisfies the condition that for any pair of corresponding points in the two images ' F ( ' l' )

he fundamental matri F F is the unique 33 rank matri that satisfies F for all (i) ranspose: if F is fundamental matri for (P,P ), then F is fundamental matri for (P,P) (ii) Epipolar lines: l F & lf (iii) Epipoles: on all epipolar lines, thus e F, e F, similarly Fe (iv) F has 7 d.o.f., i.e. 33-(homogeneous)-(rank) (v) F is a correlation, projective mapping from a point to a line l F (not a proper correlation, i.e. not invertible)

he epipolar line geometry l,l epipolar lines, k line not through e l F[k] l and symmetrically lf [k ] l l l k k e e' l Fk (pick ke, since e e ) F[ e] l l F [ e' ] l' l'

Fundamental matri for pure translation

Fundamental matri for pure translation [ e' ] H [ ] F e' ( H K RK) eample: e' (,,) F - ' F y y' PX ' P' X K[I ]X K - K[I t] Z ( X,Y,Z) - K /Z ' + Kt/Z motion starts at and moves towards e, faster depending on Z pure translation: F only d.o.f., [e] auto-epipolar

General motion [ e' ] H [ e' ] ˆ ' ' ' K'RK - + K' t/z

Geometric representation of F F ( F F )/ ( F F F )/ S + A ( F + ) F S F A F ( F ) A F F s : Steiner conic, 5 d.o.f. F a [ a ] : pole of line ee w.r.t. F s, d.o.f. S

Geometric representation of F

Pure planar motion Steiner conic F s is degenerate (two lines)

Projective transformation and invariance Derivation based purely on projective concepts ˆ H, ˆ' H' ' Fˆ PX ( )( - PH H X) Pˆ Xˆ ( )( - P' H H X) Pˆ' Xˆ H' - FH - F invariant to transformations of projective 3-space ' P' X ( P, P' ) α F F α ( P,P' ) canonical form P' [I ] [M m] unique not unique P [ m] M F

Projective ambiguity of cameras given F previous slide: at least projective ambiguity this slide: not more! ~ ~ Show that if F is same for (P,P ) and (P,P ), there eists a projective transformation H so that ~ ~ PHP and P HP lemma: H P' H ~ P [I ] P' [A a] P [I ] P ~ ' F [ a] A [ ~ a] A ~ ~ A ~ a ka k A + av ( ) rank [A ~ ~ a] af a[] a A ~ af ~ a ka a A ~ a A ~ a ka ~ - A ka ~ - A av [ ] [ ] [ ] ( ) ( ) k I k v k [A a] k I k v k [ k ( A - av ) ka] P ~ ' (-57, ok)

Canonical cameras given F F matri corresponds to P,P iff P FP is skew-symmetric ( X P' FPX, X) F matri, S skew-symmetric matri P [I ] P' [SF e'] [SF e'] (fund.matrif) F[I ] F S F e' F F S F Possible choice: P [I ] P' [[e'] F e'] Canonical representation: P [I ] P' [[e'] F + e' v λe']

he essential matri ~fundamental matri for calibrated cameras (remove K) E ˆ ' [ t] R R[R t] Eˆ - - ( ˆ K ; ˆ' K ' ) E K' FK 5 d.o.f. (3 for R; for t up to scale) E is essential matri if and only if two singularvalues are equal (and third) E Udiag(,,)V

Four possible reconstructions from E (only one solution where points is in front of both cameras)

wo-view geometry Epipolar geometry F-matri comp. 3D reconstruction Structure comp.

he epipolar geometry C,C,, and X are coplanar

he epipolar geometry What if only C,C, are known?

he epipolar geometry All points on π project on l and l

he epipolar geometry Family of planes π and lines l and l Intersection in e and e

Eample: converging cameras

Eample: motion parallel with image plane

Eample: forward motion e e

he fundamental matri F geometric derivation ' H π [ e' ] Hπ F l' e' ' mapping from -D to -D family (rank )

l' he fundamental matri F algebraic derivation M + ( ) λc X λ - M [ ] [ ] [ ] M' m' C M' m' M' m' C M' M ' l' ' l' ' F P + X( λ) F [ ] e' M' M (note: doesn t work for CC F)

he fundamental matri F correspondence condition he fundamental matri satisfies the condition that for any pair of corresponding points in the two images ' F ( ' l' )

he epipolar line geometry l,l epipolar lines, k line not through e l F[k] l and symmetrically lf [k ] l l l k k e e' l Fk (pick ke, since e e ) F[ e] l l F [ e' ] l' l'

Fundamental matri for pure translation

Fundamental matri for pure translation [ e' ] H [ ] F e' ( H K RK) eample: e' (,,) F - ' F y y' PX ' P' X K[I ]X - K[I t] K Z ( X,Y,Z) - K /Z ' + Kt/Z motion starts at and moves towards e, faster depending on Z pure translation: F only d.o.f., [e] auto-epipolar

General motion [ e' ] H [ e' ] ˆ ' ' ' K'RK - + K' t/z

Geometric representation of F F ( F F )/ ( F F F )/ S + A ( F + ) F S F A F ( F ) A F F s : Steiner conic, 5 d.o.f. F a [ a ] : pole of line ee w.r.t. F s, d.o.f. S

Geometric representation of F

Pure planar motion Steiner conic F s is degenerate (two lines)

Projective ambiguity of cameras given F previous slide: at least projective ambiguity this slide: not more! ~ ~ Show that if F is same for (P,P ) and (P,P ), there eists a projective transformation H so that ~ ~ PHP and P HP lemma: H P' H ~ P [I ] P' [A a] P ~ [I ] P' F [ a] A [ ~ a] A ~ ~ A ~ a ka k A + av ( ) rank [A ~ ~ a] af a[] a A ~ af ~ a ka a A ~ a A ~ a ka ~ - A ka ~ - A av [ ] [ ] [ ] ( ) ( ) k I k v k [A a] k I k v k [ k ( A - av ) ka] P ~ ' (-57, ok)

Motion from E Given W Four solutions R' UW V and t' U 3 R' ' UWV t' ' U 3

Four possible reconstructions from E (only one solution where points is in front of both cameras)