Feature detection Kim Steenstrup Pedersen kimstp@itu.dk The IT University of Copenhagen Feature detection, The IT University of Copenhagen p.1/20
What is a feature? Features can be thought of as symbolic descriptions of images as primitive building blocks. We have seen that the Gaussian and its derivatives serve as useful filters in early processing of image data. But linear filtering is not enough, we need non-linear functions of the derivatives in order to get rich descriptions of image geometry. It has been proposed by Koenderink et al. to use differential invariants as descriptions of image geometry (hereby getting invariance with respect to choices of coordinate systems). We will think of such descriptions as defining image features. Hence image features are points of special image geometry. Feature detection, The IT University of Copenhagen p.2/20
Example of features Feature detection, The IT University of Copenhagen p.3/20
Why are features useful? Marr s hierarchical view of a vision system: Input scene Description 3D Model 2 1/2D Sketch Primal sketch Feature images Lindeberg (1994) proposed a scale space primal sketch consisting of gray level image blobs at multiple scales. Feature detection, The IT University of Copenhagen p.4/20
Defining features by functions of derivatives We would like to have feature detectors which are invariant to various coordinate transformations such as rotation and translation. We can use differential invariants and gauge coordinates to define feature descriptors which are invariant to such transformations. Invariance to transformations will give us an advantage over classical feature detection methods based on, say, discrete 3 3 derivative filters such as Prewitt, Sobel, etc. Assuming for now that the scale σ is fixed we can detect features by finding local spatial maxima or zero crossings of the differential feature descriptors DL: DL = 0 or DL = 0. Feature detection, The IT University of Copenhagen p.5/20
Point features: Blobs Blobs can be defined by local spatial extrema of the Laplacian 2 L = L xx + L yy. Hence ( 2 L) = 0. The Laplacian is an irreducible differential invariant. The sign of 2 L dictates whether it is a bright (intensity maxima, 2 L < 0) or dark (intensity minima, 2 L > 0) blob. Feature detection, The IT University of Copenhagen p.6/20
Point features: Blobs Feature detection, The IT University of Copenhagen p.7/20
Introducing gauge coordinates A rotation of the (x, y) system to the (v, w) system. cos(α) = sin(α) = L x (L 2 x + L 2 y) L y (L 2 x + L 2 y) v = sin(α) x cos(α) y w = cos(α) x + sin(α) y The w axis points in the gradient direction. Feature detection, The IT University of Copenhagen p.8/20
Point features: Corners Corner detection by finding maxima in the following expression L 3 w κ = L 2 w L vv = L 2 xl yy L 2 yl xx 2L x L y L xy where the curvature is κ = L vv L w. Such corner points are invariant under affine transformations. Furthermore, the detector is defined even at points of vanishing gradient L w = 0. This corner detector is therefore ideal for providing interest points for matching and recognition. Feature detection, The IT University of Copenhagen p.9/20
Point features: Corners Illustration copied from Lindeberg (1998a) (σ = 2, 4, 8). Feature detection, The IT University of Copenhagen p.10/20
Curve features: Differential edge detector The magnitude of the gradient L = L w should have a local maximum in the gradient direction. Gradient L = L x L y Hessian matrix H L = L xx L yx L xy L yy Hence the second order derivative in the gradient direction w should be zero L ww = 1 L 2 ( L)T H L L = 1 L 2 (L2 x L xx + 2L x L y L xy + L 2 y L yy) = 0. We want to find local maxima of the gradient so we also have to check whether L www < 0. 1 L www = L 3 (L3 xl xxx + 3L 2 xl y L xxy + 3L x L 2 yl xyy + L 3 yl yyy ) < 0. Find zero crossings of L ww and check the sign of L www. Feature detection, The IT University of Copenhagen p.11/20
Edges at fixed scales (L ww = 0) Illustration copied from Lindeberg (1996) (σ = 1, 4, 16). Feature detection, The IT University of Copenhagen p.12/20
An alternative edge definition: Zero crossings of the Laplacian Edge detection by zero-crossings of the Laplacian: Look for zero crossings 2 L = L xx + L yy 2 L = 0. This is a classical edge detector, which can also be formulated in terms of a single filter, the LoG (Laplacian of Gaussian) filter: 2 G σ f = 0 2 G σ = 1 r 2 2σ 2 2πσ 2 σ 4 e r2 2σ 2 Which is obviously just the scale space formulation of the problem. Feature detection, The IT University of Copenhagen p.13/20
NB: Laplacian and L ww We have defined edges as either L ww = 0 or 2 L = 0. 2 L = L ww + L vv = L xx + L yy The part L vv is proportional to isophote curvature. Hence the two edge detectors are the same at straight edges and the difference between the two grows at corners. Feature detection, The IT University of Copenhagen p.14/20
Zero of Lww and Lww+Lvv 50 100 150 200 50 100 150 200 250 250 50 100 150 200 250 250 200 Green dashed is Lww = 0 150 100 50 Feature detection, The IT University of Copenhagen p.15/20
Zero of Lww and Lww+Lvv, cont 120 125 130 135 140 145 100 105 110 115 120 125 130 Green dashed is Lww = 0 Feature detection, The IT University of Copenhagen p.16/20
Introducing another gauge coordinate system Introducing at any image point a local (p, q) coordinate system aligned to the principal curvature directions, such that L pq = 0. Hence, apply an orthogonal transformation R to the Hessian matrix which diagonalizes the Hessian, such that R T L xx L xy L yx L yy R = L pp 0 0 L qq The (p, q) system is defined by the eigenvectors of the rotated Hessian matrix. The eigenvalues is the curvature in the principal curvature directions. Feature detection, The IT University of Copenhagen p.17/20
Curve features: Ridges Ridge detection (maximal in maximal curvature direction) L p = 0 L pp < 0 L pp > L qq If p is the direction of maximum absolute value of the principal curvature. At nondegenerate points (L w 0) Lindeberg (1994) have shown that we may equivalently write Lvw = L x L y (L xx L yy ) (L 2 x L 2 y)l xy = 0 L 2 vv L 2 ww = (L 2 y L 2 x)(l xx L yy ) 4L x L y L xy > 0 Feature detection, The IT University of Copenhagen p.18/20
Bright ridges at fixed scales Illustration copied from Lindeberg (1996) (σ = 1, 4, 16). Feature detection, The IT University of Copenhagen p.19/20
Summary We have seen that we can define feature detectors in terms of functions of image derivatives. The invariants are interesting functions because of their coordinate system independence. But we have only talked about fixed scale feature detection and it is clear from the examples that we need to do multi-scale feature detection. Next lecture will cover the topic of multi-scale feature detection by discussing scale selection for feature detection. Feature detection, The IT University of Copenhagen p.20/20