Extract useful building blocks: blobs the same image like for the corners
Here were the corners...
Blob detection in 2D Laplacian of Gaussian: Circularly symmetric operator for blob detection in 2D 2 g= 2 g x 2 2 g y 2
Edge detection as zero crossing here 1D Gaussian f Edge d 2 dx 2 g Second derivative of Gaussian (Laplacian) f d2 dx 2 g Edge = zero crossing of second derivative Source: S. Seitz
Scale normalization An edge response to zero crossing is proportional sigma -2 To keep response the same (scaleinvariant), must multiply Gaussian derivative by σ Laplacian is the second Gaussian derivative, so it must be multiplied by σ 2 in order to be invariant to 2D rotations
Blob detection in 2D Laplacian of Gaussian: Circularly symmetric operator for blob detection in 2D Scale-normalized: 2 norm g=σ 2 2 g x 2 2 g y 2
Effect of scale normalization Original signal Unnormalized Laplacian response Scale-normalized Laplacian response Maximum
DOG David G. Lowe. "Distinctive image features from scale-invariant keypoints. IJCV 60 (2), 04 Approximating the Laplacian with a difference of Gaussians: ( (,, ) (,, ) ) 2 L = Gxx x y + Gyy x y σ σ σ (Laplacian) DoG = G( x, y, kσ ) G( x, y, σ ) (Difference of Gaussians) ***or*** Difference of gaussian blurred images at scales k σ and σ L
Scale-space blob detector 1. Convolve image with scale-normalized Laplacian at several scales 1. Find maxima of squared Laplacian response in scale-space 1. This indicate if a blob has been detected 1. And what s its intrinsic scale Zero-crossing gives close edge contours but also more noise.
Harris-Laplace [Mikolajczyk & Schmid 01] Collect locations (x,y) of detected Harris features for σ = σ 1 σ 2 (the sigma is here comes from g x, g y ) For each detected location (x,y) and for each σ, reject detection if Laplacian(x,y, σ) is not a local maximum Output: location, scale
Scale-space blob detector: Example
Scale-space blob detector: Example a scale...
Scale-space blob detector: Example
Invariance of the blob detection using the scale-space behaves better than the simple Harris corner detection. Still many of the problems are not solved. Examples: viewpoint, strong illumination difference
Affine invariance K. Mikolajczyk and C. Schmid, Scale and Affine invariant interest point detectors, IJCV 60(1):63-86, 2004. Similarly to characteristic scale selection, detect the characteristic shape of the local feature
M= x,y Affine adaptation [ 2 I w x,y x I x I y ] 2 I x I y I y =R 1[ λ 1 0 0 λ 2 ] R R = U We can visualize M as an ellipse with axis lengths determined by the eigenvalues and orientation determined by R Ellipse equation: direction of the fastest change direction of the slowest change [u v ] M [ u v ] =const (λ max ) -1/2 (λ min )-1/2 The second moment ellipse can be viewed as the characteristic shape of a region
The ellipse is centered on the middle of the region (discrete T grid!). New variable z = R [(u - x_c) (v - y_c)] where R is a 2x2 orthonormal matrix lambda_max z_1^2 + lambda_min z_2^2 = constant the major and minor axes are the min and max lambda. In the original, u,v, coordinates due to R the ellipse will be not aligned with the coordinate system.
Rotation preserves eigenvalues, so affine deformations are upto rotation.
Affine Blob Detection Detect initial region with Harris-Laplace. Estimate affine shape with M. Normalize the affine region to a circular one. The rotation ambiguity is eliminated by, say, Scale-Invariant Feature Transform (SIFT)... next lecture. Better behavior then the previous ones... example: weaker dependent on the viewpoint. But many problems still can appear in real images.
Affine adaptation Output: location, scale, affine shape, rotation (more later)
Affine adaptation example Scale-invariant regions (blobs)
Affine adaptation example Affine-adapted blobs ellipses of M
Challenges Depending on the application a descriptor must incorporate information that is: Invariant w.r.t: Illumination Pose Scale Intraclass variability none of the detectors Aa satisfy completely this invariance Highly distinctive (allows a single feature to find its correct match with good probability in a large database of features)