Kepoint etraction: Corners 9300 Harris Corners Pkw Charlotte NC
Wh etract kepoints? Motivation: panorama stitching We have two images how do we combine them?
Wh etract kepoints? Motivation: panorama stitching We have two images how do we combine them? Step 1: etract kepoints Step 2: match kepoint features
Wh etract kepoints? Motivation: panorama stitching We have two images how do we combine them? Step 1: etract kepoints Step 2: match kepoint features Step 3: align images
Characteristics of good kepoints Repeatabilit The same kepoint can be found in several images despite geometric and photometric transformations Salienc Each kepoint is distinctive Compactness and efficienc Man fewer kepoints than image piels Localit A kepoint occupies a relativel small area of the image; robust to clutter and occlusion
Applications Kepoints are used for: mage alignment 3D reconstruction Motion tracking Robot navigation ndeing and database retrieval Object recognition
Corner Detection: Basic dea We should easil recognize the point b looking through a small window Shifting a window in an direction should give a large change in intensit flat region: no change in all directions edge : no change along the edge direction corner : significant change in all directions
Corner Detection: Mathematics Change in appearance of window W for the shift [uv]: () W E(u v) = [( + u + v) ( )] 2 ( ) E(u v) E(32)
Corner Detection: Mathematics Change in appearance of window W for the shift [uv]: () W E(u v) = [( + u + v) ( )] 2 ( ) E(u v) E(00)
Corner Detection: Mathematics Change in appearance of window W for the shift [uv]: () W E(u v) = [( + u + v) ( )] 2 We want to find out how this function behaves for small shifts E(u v)
Corner Detection: Mathematics First-order Talor approimation for small motions [u v]: ( + u + v) ( ) + u + v Let s plug this into E(uv): () W E(u v) = [( + u + v) ( )] 2 () W [( )+ u + v ( )] 2 = [ u + v] 2 = 2 u 2 + 2 uv + 2 v 2 () W () W
Corner Detection: Mathematics The quadratic approimation can be written as where M is a second moment matri computed from image derivatives: [ ] v u M v u v u E ) ( = M 2 2 (the sums are over all the piels in the window W)
The surface E(uv) is locall approimated b a quadratic form. Let s tr to understand its shape. Specificall in which directions does it have the smallest/greatest change? nterpreting the second moment matri v u M v u v u E ] [ ) ( E(u v) = M 2 2
nterpreting the second moment matri Consider a horizontal slice of E(u v): [ u v] M u = const v This is the equation of an ellipse.
nterpreting the second moment matri Consider a horizontal slice of E(u v): [ u v] M u = const v This is the equation of an ellipse. Diagonalization of M: M λ = R 1 1 0 R 0 λ2 The ais lengths of the ellipse are determined b the eigenvalues and the orientation is determined b R direction of the fastest change direction of the slowest change (λ ma ) -1/2 (λ min ) -1/2
Consider the ais-aligned case (gradients are either horizontal or vertical) f either a or b is close to 0 then this is not a corner so look for locations where both are large. nterpreting the second moment matri = b a 0 0 = M 2 2
Visualization of second moment matrices
Visualization of second moment matrices
nterpreting the eigenvalues Classification of image points using eigenvalues of M: λ 2 Edge λ 2 >> λ 1 Corner λ 1 and λ 2 are large λ 1 ~ λ 2 ; E increases in all directions λ 1 and λ 2 are small; E is almost constant in all directions Flat region Edge λ 1 >> λ 2 λ 1
Corner response function R = det( M ) M 2 α trace( ) = λ1λ 2 α( λ1 + λ ) 2 2 α: constant (0.04 to 0.06) Edge R < 0 Corner R > 0 Flat region R small Edge R < 0
The Harris corner detector 1. Compute partial derivatives at each piel 2. Compute second moment matri M in a Gaussian window around each piel: C.Harris and M.Stephens. A Combined Corner and Edge Detector. Proceedings of the 4th Alve Vision Conference: pages 147 151 1988. = w w w w M 2 2 ) ( ) ( ) ( ) (
The Harris corner detector 1. Compute partial derivatives at each piel 2. Compute second moment matri M in a Gaussian window around each piel 3. Compute corner response function R C.Harris and M.Stephens. A Combined Corner and Edge Detector. Proceedings of the 4th Alve Vision Conference: pages 147 151 1988.
Harris Detector: Steps
Harris Detector: Steps Compute corner response R
The Harris corner detector 1. Compute partial derivatives at each piel 2. Compute second moment matri M in a Gaussian window around each piel 3. Compute corner response function R 4. Threshold R 5. Find local maima of response function (nonmaimum suppression) C.Harris and M.Stephens. A Combined Corner and Edge Detector. Proceedings of the 4th Alve Vision Conference: pages 147 151 1988.
Harris Detector: Steps Find points with large corner response: R > threshold
Harris Detector: Steps Take onl the points of local maima of R
Harris Detector: Steps
Robustness of corner features What happens to corner features when the image undergoes geometric or photometric transformations?
Affine intensit change Onl derivatives are used => invariance to intensit shift + b ntensit scaling: a a + b R threshold R (image coordinate) (image coordinate) Partiall invariant to affine intensit change
mage translation Derivatives and window function are shift-invariant Corner location is covariant w.r.t. translation
mage rotation Second moment ellipse rotates but its shape (i.e. eigenvalues) remains the same Corner location is covariant w.r.t. rotation
Scaling Corner All points will be classified as edges Corner location is not covariant to scaling!