Image Processing 1 (IP1) Bildverarbeitung 1

Transcription

1 MIN-Fakultät Fachbereich Informatik Arbeitsbereich SAV/BV KOGS Image Processing 1 IP1 Bildverarbeitung 1 Lecture : Object Recognition Winter Semester 015/16 Slides: Prof. Bernd Neumann Slightly revised by: Dr. Benjamin Seppke & Prof. Siegfried Stiehl

2 Object Recognition with Local Descriptors Basic idea: Determine interest points in model images Determine invariant local image properties around interest points Use local image properties for finding matching objects Matching images using SIFT features SIFT = Scale-Invariant Feature Transform

3 SIFT Method David G. Lowe: Distinctive Image Features from Scale-Invariant Keypoints International Journal ofcomputer Vision, 004 Protected by US patent Lowe developed specific methods for: 1. Determininginvariant local descriptors at interest points finding stable interest points "keypoints" computing largely scale-invariant featuresat interest points. Etracting stabledescriptors for object models 3. Finding and recognizingobjects based on local descriptors 3

4 Determining SIFT Keypoints: Scale Space Recall: Keypoints are local maima and minima in the DoG of scaled images. L, y, kσ = G, y, kσ * I, y Convolution of image I, y with Gaussian G, y, kσ D, y, σ = L, y, k i σ - L, y, k j σ Difference of Gaussians DoG Procedure: a Initial image is repeatedly convolved with Gaussians of multiples of σ, forming a scale space. b Scaled images within an octave σ... σ have same resolution. Adjacent scales are subtracted to produce DoGs. c Scaled images are down-sampled from one octave to the net. 4

5 Illustration of SIFT Scale Space 5

6 Eample Image in SIFT Scale Space 5 Gaussian filtered images per octace Corresponding DoGs 6

7 Determining Etrema Find local minima and maima bycomparing a DoG piel to its 6 neighbours in 33 regions at the current and adjacent scales. 7

8 Sub-piel Localization of Etrema Take etrema of previous step as keypoint candidates Determine Taylor epansion at candidate location Find subpiel etremum bysettingderivatives to zero If location ofsubpiel etremumis within 0.5 of candidate location in - or y-direction, keep keypoint at subpiel location, otherwisediscard keypoint candidate If value of epansionat subpiel location is less than 0.03, discard keypoint Taylor epansion: D D, y = D + Etrema: et D D D D D + y + y D y y y yy y y = y et DDyy Dy DDyy Dy 1 D + 1 y D + y y D D D D D = with D = etc. approimated from local neighbourhood 8

9 Eliminating Edge Responses Keypoints at strong edges tend to be unstable. Principal curvatures at keypoint must be significant for keypoint to be stable. Compute Hessian at keypoint: D H = D Eigenvalues α and β of H are proportional to principal curvatures. y D D y yy Note that R = Tr H Det H = r + 1 r with r α trh = D =, β deth = D + D D yy yy D = αα+ β y = αα The higher the absolute differences ofprincipal curvatures ofd, the higher the value of R. r0 + 1 Hence if R > with r 0 as threshold, the keypoint is discarded. r 0 9

10 Illustration of Principal Curvatures surface normal Each point of a 3D surface has a maimum and minimum curvature. 10

11 Assigning Orientations Each keypoint is marked by one or more dominant orientations based on image gradient directions computed in a neighbouringregion. Gradient magnitude: Gradient direction: θ, y = atan L, y + 1 L, y 1, L + 1, y L 1, y Gradient magnitudes, weighted bya Gaussian ofradius 1.5σ, are summed in 36 bins of an orientationhistogram. The histogram peakand all other peaks within 80% of the absolute peakvalue are assigned as dominant keypoint orientations. m, y = L + 1, y L 1, y + L, y + 1 L, y [ ] Dominant keypoint orientations are used to achieve orientation invariance for object recognition. 1 11

12 Illustration of Keypoint Selection I greyvalue image 83 keypoint candidates at etrema of DoG images. Vectors show location, orientationand scale. 1

13 Illustration of Keypoint Selection II 79 keypoints remain after applying threshold on minimum contrast 536 keypoints remain after applying threshold on ratio of principal curvatures 13

14 Computing a Keypoint Descriptor 4 4 orientationhistograms with 8 bins each are determined from a neighbourhoodofa keypoint. Each bin contains the sum of the gradient magnitudes ofcorresponding orientations, weighted bya Gaussian. Illustration shows histograms for 8 8 neighbourhood, Gaussian indicated bycircle. 14

15 Recognition Using SIFT Features Compute SIFT features on the input image Match these features to the SIFT feature database of an object model Each keypoint specifies 4 parameters: D location, scale, and dominant orientation. To increase recognition robustness: Hough transform to identify clusters of matches that vote for the same object pose. Each keypoint votes for the set of object poses that are consistent with the keypoint's location, scale, and orientation. Locations in the Hough accumulator that accumulate at least 3 votes are selected as candidate object/pose matches. A verfication step matches the training image for the hypothesized object/pose to the image using a least-squares fit to the hypothesized location, scale, and orientation of the object. 15

16 Eperiment 1 I Training images Test image 16

17 Eperiment 1 II Test image with overlaid results. Parallelograms show locations of recognized objects. Small squares show keypoints used for recognition. 17

18 Eperiment I Comple test image, piels 18

19 Eperiment II Training images taken from independentviewpoints 19

20 Eperiment III Results 0

21 SIFT Features Summary SIFT features are reasonably invariant to rotation, scaling, and illumination changes. They can be used for matching and object recognition among other things. Robust to occlusion: as long as we can see at least 3 features from the object we can compute the location and pose. Efficient on-line matching: recognition can be performed in close-to-real time at least for small object databases. 1

22 Combined Object Categorization and Segmentation Bastian Leibe, Ales Leonardis, and Bernt Schiele: Combined Object Categorization and Segmentation with an Implicit Shape Model ECCV 04 Workshop on Statistical Learning in Computer Vision, Prague, May 004. Define a shape model for an object class or category by a class-specific collection of local appearances a "codebook", a spatial probability distribution specifying where a codebook entry may be found on the object To recognize an object, etractimage patches around interest points and andcompare them with the codebook. Matching patches castprobabilistic votes leading to object hypotheses. Each piel of an object hypothesis is classified as object or background based on the contributing patches.

23 Implicit Shape Model - Representation 105 training images + motion segmentation y Appearance codebook y Learn appearance codebook ØEtract 55 patches at interest points ØAgglomerative clustering codebook y s y s Learn spatial distributions ØMatch codebook to training images ØRecord matching positions on object s s Spatial occurrence distributions 3

24 Harris Corner Detector I Large differences between a piel and its surroundings: Averagingover a circular window with Gaussian weights wu, v. First-order Taylor Series approimation: Iu+, v+y Iu, v + I u, v + I y u, v y with 4 IP1 Lecture : Object Recognition + + = u v v u I y v u I v w u y S,,,, = + y A y y v u I v u I v w u y S u v y ] [,,,, = u v y y y I I I I I I v w u A, "Structure Tensor"

25 Harris Corner Detector II Eigenvalues λ 1 and λ of A indicatecornerness: λ 1 0 and λ 0 basicallyflat greyvalues λ 1 0 and λ 0 edge λ 1 0 and λ 0 corner Instead of computing eigenvalueseplicitly: M c = λ 1 λ κλ 1 +λ = deta κ trace A measure of cornerness κ = sensitivity parameter, must be tuned empirically 5

26 Agglomerative Clustering Start with separate clusters for each single item Merge most similar clusters as longas average similarity within cluster stays above threshold 6 IP1 Lecture : Object Recognition C p NGC C s C p = similarity s within clusterc = i i i i i i i q q p p q q p p q p NGC, Normalized Greyscale Correlation 3 1 0

27 IP1 Lecture : Object Recognition Implicit Shape Model - Recognition I Interest Points Image Patch evidence e e Matched Codebook Entries Interpretations I Codebook match p Ij e p on, I j I Probabilistic Voting Objects o Positions o, p on, I j p I j e p on, e = p on, I j p I j e j University of Hamburg, Dept. Informatics 7

28 IP1 Lecture : Object Recognition Implicit Shape Model - Recognition II Interest Points Matched Codebook Entries Probabilistic Voting Spatial feature configurations Interleaved object recognition and segmentation Segmentation Refined Hypotheses uniform sampling Backprojected Hypotheses University of Hamburg, Dept. Informatics Voting Space continuous Backprojection of Maima 8

29 Car Detection Recognizes different kinds of cars Robust to clutter, occlusion, noise, low contrast 9

30 Cow Detection and Segmentation frame-by-frame detection no temporal continuity eploited 30