Featre etraction: Corners and blobs
Wh etract featres? Motiation: panorama stitching We hae two images how do we combine them?
Wh etract featres? Motiation: panorama stitching We hae two images how do we combine them? Step 1: etract featres Step : match featres
Wh etract featres? Motiation: panorama stitching We hae two images how do we combine them? Step 1: etract featres Step : match featres Step 3: align images
Characteristics of good featres Repeatabilit The same featre can be fond in seeral images despite geometric and photometric transformations Salienc ach featre has a distinctie description Compactness and efficienc Man fewer featres than image piels Localit A featre occpies a relatiel small area of the image; robst to cltter and occlsion
Applications Featre points are sed for: Motion tracking mage alignment 3D reconstrction Object recognition ndeing and database retrieal Robot naigation
Finding Corners Ke propert: in the region arond a corner image gradient has two or more dominant directions Corners are repeatable and distinctie CH i dmst h "A C bi d C d d D t t C.Harris and M.Stephens. "A Combined Corner and dge Detector. Proceedings of the 4th Ale Vision Conference: pages 147--151.
Corner Detection: Basic dea We shold easil recognize the point b looking throgh a small window Shifting a window in an direction shold gie a large change in intensit Sorce: A. fros flat region: no change in all directions edge : no change along the edge direction corner : significant change in all directions
Corner Detection: Mathematics Change in appearance for the shift []: [ ] = w Window fnction Shifted intensit ntensit Window fnction w = or 1 in window 0 otside Gassian Sorce: R. Szeliski
Corner Detection: Mathematics Change in appearance for the shift []: [ ] = w 00 3
Corner Detection: Mathematics Change in appearance for the shift []: [ ] = w We want to find ot how this fnction behaes for small shifts Second-order Talor epansion of abot 00 local qadratic approimation: 00 1 00 00 00 0 [ ] [ ] 00 00 00
Corner Detection: Mathematics [ ] w = Second-order Talor epansion of abot 00: 00 00 ] [ 1 00 ] [ 00 00 00 ] [ 00 ] [ 00 [ ] w = w = [ ] w w = [ ] w
Corner Detection: Mathematics [ ] w = Second-order Talor epansion of abot 00: 00 00 ] [ 1 00 ] [ 00 0 00 = 00 00 ] [ 00 ] [ 00 0 00 0 00 = = 00 00 w w = = 00 w =
Corner Detection: Mathematics [ ] w = Second-order Talor epansion of abot 00: w w ] [ w w ] [ 0 00 = 0 00 0 00 = = 00 00 w w = = 00 w =
Corner Detection: Mathematics The qadratic approimation simplifies to [ ] M where M is a second moment matri compted from image deriaties: es M = w M
nterpreting the second moment matri The srface is locall approimated b a qadratic form. Let s tr to nderstand its shape. M ] [ = w M = w M
nterpreting the second moment matri First consider the ais-aligned case gradients are either horizontal or ertical 0 λ gradients are either horizontal or ertical = = 1 0 0 λ λ w M f either λ is close to 0 then this is not a corner so look for locations where both are large.
nterpreting the second moment matri Consider a horizontal slice of : [ ] M = const This is the eqation of an ellipse.
nterpreting the second moment matri Consider a horizontal slice of : [ ] M = const This is the eqation of an ellipse. Diagonalization of M: M λ = R 1 0 1 R 0 λ The ais lengths of the ellipse are determined b the eigenales and the orientation is determined b R direction of the fastest change direction of the slowest change λ ma -1/ λ -1/ min
Visalization of second moment matrices
Visalization of second moment matrices
nterpreting the eigenales Classification of image points sing eigenales of M: λ dge λ >> λ 1 Corner λ 1 and λ are large λ 1 ~ λ ; increases in all directions λ 1 and λ are small; is almost constant in all directions Flat region dge λ 1 >> λ λ 1
Corner response fnction R = det M M α trace = λ1λ α λ1 λ α: constant t 0.04 04 to 0.06 06 dge R < 0 Corner R>0 R small Flat region dge R < 0
Harris detector: Steps 1. Compte Gassian deriaties at each piel. Compte second moment matri M in a Gassian window arond each piel 3. Compte corner response fnction R 4. Threshold R 5. Find local maima of response fnction nonmaimm sppression CH i d MSt h "A C bi d C d d D t t C.Harris and M.Stephens. "A Combined Corner and dge Detector. Proceedings of the 4th Ale Vision Conference: pages 147 151 1988.
Harris Detector: Steps
Harris Detector: Steps Compte corner response R
Harris Detector: Steps Find points with large corner response: R>threshold
Harris Detector: Steps Tk Take onl the points it of flocal lmaima of frr
Harris Detector: Steps
nariance and coariance We want featres to be inariant to photometric transformations and coariant to geometric transformations nariance: image is transformed and featres do not change Coariance: if we hae two transformed ersions of the same image featres shold be detected in corresponding locations
Models of mage Change Photometric Affine intensit change a b Geometric Rotation Scale Affine alid for: orthographic camera locall planar object
Affine intensit change Onl deriaties are sed => inariance to intensit shift b ntensit scale: a R threshold R image coordinate image coordinate Partiall inariant to affine intensit change
mage rotation llipse rotates bt its shape i.e. eigenales remains the same Corner response R is inariant w r t rotation and Corner response R is inariant w.r.t. rotation and corner location is coariant
Scaling Corner All points will be classified as edges Not inariant to scaling
Achieing scale coariance Goal: independentl detect corresponding regions in scaled ersions of the same image Need scale selection mechanism for finding characteristic region size that is coariant with the image transformation
Blob detection with scale selection
Recall: dge detection f dge d d g Deriatie of Gassian f d d g dge = maimm of deriatie Sorce: S. Seitz
dge detection Take f dge d d Second deriatie g of Gassian Laplacian d f d g dge = zero crossing of second deriatie Sorce: S. Seitz
From edges to blobs dge = ripple Blob = sperposition of two ripples maimm Spatial selection: the magnitde of the Laplacian response will achiee a maimm at the center of the blob proided the scale of the Laplacian is matched to the scale of the blob
Scale selection We want to find the characteristic scale of the blob b conoling it with Laplacians at seeral scales and looking for the maimm response Howeer Laplacian response decas as scale increases: original signal radis=8 increasing σ Wh does this happen?
Scale normalization The response of a deriatie of Gassian filter to a perfect step edge decreases as σ increases 1 σ π
Scale normalization The response of a deriatie of Gassian filter to a perfect step edge decreases as σ increases To keep response the same scale-inariant mst mltipl Gassian deriatie b σ Laplacian is the second Gassian deriatie so it mst be mltiplied b σ
ffect of scale normalization Original signal Unnormalized Laplacian response Scale-normalized Laplacian response maimm
Blob detection in D Laplacian of Gassian: Circlarl smmetric operator for blob detection in D g g g =
Blob detection in D Laplacian of Gassian: Circlarl smmetric operator for blob detection in D Scale-normalized: g g g = σ norm
Scale selection At what scale does the Laplacian achiee a maimm response to a binar circle of radis r? r image Laplacian
Scale selection At what scale does the Laplacian achiee a maimm response to a binar circle of radis r? To get maimm response the zeros of the Laplacian hae to be aligned with the circle The Laplacian is gien b p to scale: σ Therefore the maimm response occrs at e / σ σ = r /. r circle image Laplacian
Characteristic scale We define the characteristic scale of a blob as the scale that prodces peak of Laplacian response in the blob b center characteristic scale T. Lindeberg 1998. "Featre detection with atomatic scale selection." nternational Jornal of Compter Vision 30 : pp 77--116.
Scale-space blob detector 1. Conole image with scale-normalized Laplacian at seeral scales. Find maima of sqared Laplacian response in scale-space
Scale-space blob detector: ample
Scale-space blob detector: ample
Scale-space blob detector: ample
fficient implementation Approimating the Laplacian with a difference of Gassians: L= G G σ σ σ Laplacian DG DoG = G k σ G σ Difference of Gassians
fficient implementation Daid G. Lowe. "Distinctie image featres from scale-inariant kepoints. JCV 60 pp. 91-110 004.
nariance and coariance properties Laplacian blob response is inariant w.r.t. rotation and scaling Blob location is coariant w.r.t. rotation and scaling What abot intensit change?
Achieing affine coariance Consider the second moment matri of the window containing the blob: M = λ 1 1 w = R 0 0 R λ Recall: [ ] M = const direction of the fastest change λ ma -1/ λ -1/ min direction of the slowest change This ellipse isalizes the characteristic shape of the window
Affine adaptation eample Scale-inariant regions blobs
Affine adaptation eample Affine-adapted blobs
Affine adaptation Problem: the second moment window determined b weights w mst match the characteristic ti shape of the region Soltion: iteratie approach Use a circlar window to compte second moment matri Perform affine adaptation to find an ellipse-shaped window Recompte second moment matri sing new window and iterate
teratie affine adaptation K. Mikolajczk and C. Schmid Scale and Affine inariant interest point detectors JCV 601:63-86 004. http://www.robots.o.ac.k/~gg/research/affine/
Affine coariance Affinel transformed ersions of the same neighborhood will gie rise to ellipses that are related b the same transformation What to do if we want to compare these image regions? Affine normalization: transform these regions into same-size circles
Affine normalization Problem: There is no niqe transformation from an ellipse to a nit circle We can rotate or flip a nit circle and it still stas a nit circle
liminating rotation ambigit To assign a niqe orientation to circlar image windows: Create histogram of local l gradient directions in the patch Assign canonical orientation at peak of smoothed histogram 0 π
From coariant regions to inariant featres tract affine regions Normalize regions liminate rotational ambigit Compte appearance descriptors SFT Lowe 04
nariance s. coariance nariance: featrestransformimage = featresimage Coariance: featrestransformimage = transformfeatresimage Coariant detection => inariant description