Feature Tracking. 2/27/12 ECEn 631

Transcription

1 Corner Extraction

2 Feature Tracking Mostly for multi-frame applications Object Tracking Motion detection Image matching Image mosaicing 3D modeling Object recognition Homography estimation...

3 Global Features Iden+fied object : tennis ball Texture : known pa:ern Color Local Dis+nc+ve area Corner avoids the aperture problem Line Specific shape 3

4 Corners Edges : Intensity Change Object Boundaries Surface Marks or Textures Corners: Intersections of lines Patterns of Intensity a point that has two dominant and different edge directions in a local neighborhood of the point 4

5 Harris Matrix The weighted sum of squared differences (SSD) between two patches, centered at I(u, v) and I(u+x, v+y) is given by: s(x, y) ## w(u,v)(i(u,v)" I(u + x,v + y)) u I(u + x,v + y) can be approximated by a Taylor expansion. Let I x and I y be the partial derivatives of I, such that I(u + x,v + y) " I(u,v)+ I x (u,v)x + I y (u,v)y and s(x, y) " ## w(u,v)(i x (u,v)x + I y (u,v)y) u $ s(x, y) " ( x, y)c& x' ) % y( C is a structure tensor or called Harris matrix v v 5

6 Harris Matrix # c(x) % $ " E x " E x E y E x E y " " E y C(x) must be invertible. & ( ' Calculated at a generic image point p and around its neighborhood Q. E x E x E y E y 6

7 Eigenvalues Scalar λ is called an eigenvalue of A if there exists a non-zero X AX λx (A λi) X Solve the equation for λ det (A λi) " A 3 % " $ ' A ( )I 3 ( ) % $ ' # 3& # 3( )& det(a ( )I) (3 ( ))(3 ( )) ( 4 ) ( 6) + 5 () (1)() ( 5) 7

8 Eigenvectors There is a vector X i associated with each λ i so that (A λ i I) X i Rank of a Matrix? If the smallest eigenvalue of A is above a specified threshold, then C(x) is invertible and X can be considered a feature point. 8

9 Any vector can be represented using these eigenvectors. A can be transformed into diagonal form with these eigenvalues c E E x x E y Uniform Q: λ 1 λ E x y y E E Edge in Q: λ 1 and λ > and c λ 1 λ The small window Q must have sufficient texture that causes enough variation along at least two independent directions. eigenvector X is perpendicular to the edge 9

10 Corners Corner in Q: λ 1 > λ > Eigenvectors Encode Edge Direction Eigenvalues Represent Edge Strength Threshold the smaller eigenvalue of the two to determine the presence of the corner. Threshold can be estimated from the histogram of λ 1

11 Prewitt Robison Kirsch Example h E x -3-3 h E y

12 Example Prewitt Robison Kirsch h E x h E y

13 E x E y A det( A λi) λ 9 and 1 E x ExEy 13 E xey Ey 4 (13 λ)(13 λ) 16 λ λ 6λ

14 E x E y c det( A λi) E x ExEy E xey 14 E 6 y (14 λ)(6 λ) 36 λ λ b ± b a 4ac 8.5, λ

15 E x E y det( A λi) c E x ExEy E xey 6 E 6 y (14 λ)(6 λ) 36 λ λ b ± b a 4ac 8.5, λ

16 E x E y det( A λi) c E x Ex E y E x E y 8 E 9 y (8 λ)(8 λ) 81 λ λ + 73 λ b ± b a 4ac 37, 19 16

17 Eigenvalue computation can be simplified as Corner measure (strength) s " 1 " # k(" 1 + " ) det(c)# k $trace (C) k is usually.4~.15 For more details Harris, C. and Stephens, M., 1988, A combined corner and edge detector. Proceedings of the Alvey Conference, p

18 OpenCV cornerharris( const Mat& src, Mat& dst, int blocksize, int aperturesize, double k, int bordertypeborder DEFAULT ); Corners in the image can be found as the local maximum of this response map. 18

19 Threshold Selection 19

20 Remove False Corners To remove the false corners: 1. Store all corner candidates into a list. Sort the list in decreasing order of the corner strength 3. Scanning the list for each candidate: remove neighboring candidates with smaller eigenvalue

21 OpenCV void goodfeaturestotrack( const Mat& image, vector<pointf>& corners, int maxcorners, double qualitylevel, double mindistance, const Mat& maskmat(), int blocksize3, bool useharrisdetectorfalse, double k.4 ); 1

22 Refine Corner Locations 1. Feature tracking may not need subpixel accuracy. Camera calibration needs the best accuracy we can get 3. Interpolation of the resulting map fit a curve to find the peak point between pixels 4. Local gradients

23 OpenCV void cornersubpix( const Mat& image, vector<pointf>& corners, Size winsize, Size zerozone, TermCriteria criteria ); 3

24 4

25 5

26 Other Methods? Corner Pa:ern Recogni+on Intersec+on of Lines SIFT (Scale Invariant Feature Transform) HOG (Histogram of Gradient Orienta+ons) SURF (Speeded Up Robust Feature) inspired by SIFT FAST (Features from Accelerated Segment Test SUSAN (Smallest Univalue Segment Assimila+ng Nucleus) BASIS (BAsis Sparse coding Inspired Similarity) 6