Feature extraction: Corners and blobs

Transcription

1 Featre etraction: Corners and blobs

2 Wh etract featres? Motiation: panorama stitching We hae two images how do we combine them?

3 Wh etract featres? Motiation: panorama stitching We hae two images how do we combine them? Step 1: etract featres Step : match featres

4 Wh etract featres? Motiation: panorama stitching We hae two images how do we combine them? Step 1: etract featres Step : match featres Step 3: align images

5 Characteristics of good featres Repeatabilit The same featre can be fond in seeral images despite geometric and photometric transformations Salienc ach featre has a distinctie description Compactness and efficienc Man fewer featres than image piels Localit A featre occpies a relatiel small area of the image; robst to cltter and occlsion

6 Applications Featre points are sed for: Motion tracking mage alignment 3D reconstrction Object recognition ndeing and database retrieal Robot naigation

7 Finding Corners Ke propert: in the region arond a corner image gradient has two or more dominant directions Corners are repeatable and distinctie CH i dmst h "A C bi d C d d D t t C.Harris and M.Stephens. "A Combined Corner and dge Detector. Proceedings of the 4th Ale Vision Conference: pages

8 Corner Detection: Basic dea We shold easil recognize the point b looking throgh a small window Shifting a window in an direction shold gie a large change in intensit Sorce: A. fros flat region: no change in all directions edge : no change along the edge direction corner : significant change in all directions

9 Corner Detection: Mathematics Change in appearance for the shift []: [ ] = w Window fnction Shifted intensit ntensit Window fnction w = or 1 in window 0 otside Gassian Sorce: R. Szeliski

10 Corner Detection: Mathematics Change in appearance for the shift []: [ ] = w 00 3

11 Corner Detection: Mathematics Change in appearance for the shift []: [ ] = w We want to find ot how this fnction behaes for small shifts Second-order Talor epansion of abot 00 local qadratic approimation: [ ] [ ]

12 Corner Detection: Mathematics [ ] w = Second-order Talor epansion of abot 00: ] [ 1 00 ] [ ] [ 00 ] [ 00 [ ] w = w = [ ] w w = [ ] w

13 Corner Detection: Mathematics [ ] w = Second-order Talor epansion of abot 00: ] [ 1 00 ] [ = ] [ 00 ] [ = = w w = = 00 w =

14 Corner Detection: Mathematics [ ] w = Second-order Talor epansion of abot 00: w w ] [ w w ] [ 0 00 = = = w w = = 00 w =

15 Corner Detection: Mathematics The qadratic approimation simplifies to [ ] M where M is a second moment matri compted from image deriaties: es M = w M

16 nterpreting the second moment matri The srface is locall approimated b a qadratic form. Let s tr to nderstand its shape. M ] [ = w M = w M

17 nterpreting the second moment matri First consider the ais-aligned case gradients are either horizontal or ertical 0 λ gradients are either horizontal or ertical = = λ λ w M f either λ is close to 0 then this is not a corner so look for locations where both are large.

18 nterpreting the second moment matri Consider a horizontal slice of : [ ] M = const This is the eqation of an ellipse.

19 nterpreting the second moment matri Consider a horizontal slice of : [ ] M = const This is the eqation of an ellipse. Diagonalization of M: M λ = R R 0 λ The ais lengths of the ellipse are determined b the eigenales and the orientation is determined b R direction of the fastest change direction of the slowest change λ ma -1/ λ -1/ min

20 Visalization of second moment matrices

21 Visalization of second moment matrices

22 nterpreting the eigenales Classification of image points sing eigenales of M: λ dge λ >> λ 1 Corner λ 1 and λ are large λ 1 ~ λ ; increases in all directions λ 1 and λ are small; is almost constant in all directions Flat region dge λ 1 >> λ λ 1

23 Corner response fnction R = det M M α trace = λ1λ α λ1 λ α: constant t to dge R < 0 Corner R>0 R small Flat region dge R < 0

24 Harris detector: Steps 1. Compte Gassian deriaties at each piel. Compte second moment matri M in a Gassian window arond each piel 3. Compte corner response fnction R 4. Threshold R 5. Find local maima of response fnction nonmaimm sppression CH i d MSt h "A C bi d C d d D t t C.Harris and M.Stephens. "A Combined Corner and dge Detector. Proceedings of the 4th Ale Vision Conference: pages

25 Harris Detector: Steps

26 Harris Detector: Steps Compte corner response R

27 Harris Detector: Steps Find points with large corner response: R>threshold

28 Harris Detector: Steps Tk Take onl the points it of flocal lmaima of frr

29 Harris Detector: Steps

30 nariance and coariance We want featres to be inariant to photometric transformations and coariant to geometric transformations nariance: image is transformed and featres do not change Coariance: if we hae two transformed ersions of the same image featres shold be detected in corresponding locations

31 Models of mage Change Photometric Affine intensit change a b Geometric Rotation Scale Affine alid for: orthographic camera locall planar object

32 Affine intensit change Onl deriaties are sed => inariance to intensit shift b ntensit scale: a R threshold R image coordinate image coordinate Partiall inariant to affine intensit change

33 mage rotation llipse rotates bt its shape i.e. eigenales remains the same Corner response R is inariant w r t rotation and Corner response R is inariant w.r.t. rotation and corner location is coariant

34 Scaling Corner All points will be classified as edges Not inariant to scaling

35 Achieing scale coariance Goal: independentl detect corresponding regions in scaled ersions of the same image Need scale selection mechanism for finding characteristic region size that is coariant with the image transformation

36 Blob detection with scale selection

37 Recall: dge detection f dge d d g Deriatie of Gassian f d d g dge = maimm of deriatie Sorce: S. Seitz

38 dge detection Take f dge d d Second deriatie g of Gassian Laplacian d f d g dge = zero crossing of second deriatie Sorce: S. Seitz

39 From edges to blobs dge = ripple Blob = sperposition of two ripples maimm Spatial selection: the magnitde of the Laplacian response will achiee a maimm at the center of the blob proided the scale of the Laplacian is matched to the scale of the blob

40 Scale selection We want to find the characteristic scale of the blob b conoling it with Laplacians at seeral scales and looking for the maimm response Howeer Laplacian response decas as scale increases: original signal radis=8 increasing σ Wh does this happen?

41 Scale normalization The response of a deriatie of Gassian filter to a perfect step edge decreases as σ increases 1 σ π

42 Scale normalization The response of a deriatie of Gassian filter to a perfect step edge decreases as σ increases To keep response the same scale-inariant mst mltipl Gassian deriatie b σ Laplacian is the second Gassian deriatie so it mst be mltiplied b σ

43 ffect of scale normalization Original signal Unnormalized Laplacian response Scale-normalized Laplacian response maimm

44 Blob detection in D Laplacian of Gassian: Circlarl smmetric operator for blob detection in D g g g =

45 Blob detection in D Laplacian of Gassian: Circlarl smmetric operator for blob detection in D Scale-normalized: g g g = σ norm

46 Scale selection At what scale does the Laplacian achiee a maimm response to a binar circle of radis r? r image Laplacian

47 Scale selection At what scale does the Laplacian achiee a maimm response to a binar circle of radis r? To get maimm response the zeros of the Laplacian hae to be aligned with the circle The Laplacian is gien b p to scale: σ Therefore the maimm response occrs at e / σ σ = r /. r circle image Laplacian

48 Characteristic scale We define the characteristic scale of a blob as the scale that prodces peak of Laplacian response in the blob b center characteristic scale T. Lindeberg "Featre detection with atomatic scale selection." nternational Jornal of Compter Vision 30 : pp

49 Scale-space blob detector 1. Conole image with scale-normalized Laplacian at seeral scales. Find maima of sqared Laplacian response in scale-space

50 Scale-space blob detector: ample

51 Scale-space blob detector: ample

52 Scale-space blob detector: ample

53 fficient implementation Approimating the Laplacian with a difference of Gassians: L= G G σ σ σ Laplacian DG DoG = G k σ G σ Difference of Gassians

54 fficient implementation Daid G. Lowe. "Distinctie image featres from scale-inariant kepoints. JCV 60 pp

55 nariance and coariance properties Laplacian blob response is inariant w.r.t. rotation and scaling Blob location is coariant w.r.t. rotation and scaling What abot intensit change?

56 Achieing affine coariance Consider the second moment matri of the window containing the blob: M = λ 1 1 w = R 0 0 R λ Recall: [ ] M = const direction of the fastest change λ ma -1/ λ -1/ min direction of the slowest change This ellipse isalizes the characteristic shape of the window

57 Affine adaptation eample Scale-inariant regions blobs

58 Affine adaptation eample Affine-adapted blobs

59 Affine adaptation Problem: the second moment window determined b weights w mst match the characteristic ti shape of the region Soltion: iteratie approach Use a circlar window to compte second moment matri Perform affine adaptation to find an ellipse-shaped window Recompte second moment matri sing new window and iterate

60 teratie affine adaptation K. Mikolajczk and C. Schmid Scale and Affine inariant interest point detectors JCV 601:

61 Affine coariance Affinel transformed ersions of the same neighborhood will gie rise to ellipses that are related b the same transformation What to do if we want to compare these image regions? Affine normalization: transform these regions into same-size circles

62 Affine normalization Problem: There is no niqe transformation from an ellipse to a nit circle We can rotate or flip a nit circle and it still stas a nit circle

63 liminating rotation ambigit To assign a niqe orientation to circlar image windows: Create histogram of local l gradient directions in the patch Assign canonical orientation at peak of smoothed histogram 0 π

64 From coariant regions to inariant featres tract affine regions Normalize regions liminate rotational ambigit Compte appearance descriptors SFT Lowe 04

65 nariance s. coariance nariance: featrestransformimage = featresimage Coariance: featrestransformimage = transformfeatresimage Coariant detection => inariant description