Interest Operators All lectures are from posted research papers. Harris Corner Detector: the first and most basic interest operator SIFT interest point detector and region descriptor Kadir Entrop Detector and its use in object recognition 1
Interest points 0D structure not useful for matching 1D structure edge can be localised in 1D subject to the aperture problem 2D structure corner or interest point can be localised in 2D good for matching Interest Points have 2D structure. 2
Simple Approach Etraction of interest points with the Harris detector Comparison of points with cross-correlation Verification with the fundamental matri The fundamental matri maps points from the first image to corresponding points in the second matri using a homograph that is determined through the solution of a set of equations that usuall minimizes a least square error. CH 12-13 3
Preview: Harris detector Interest points etracted with Harris ~ 500 points) 4
Harris detector 5
Harris detector 6
Cross-correlation matching Match two points based on how similar their neighborhoods are. Initial matches motion vectors 188 pairs) 7
Problem with Matching You get a lot of false matches. There is reall onl one 3D) transformation between the two views. So all the matches should be consistent. Ransac is an algorithm that chooses consistent matches and throws the outliers out. 8
Homograph In the field of computer vision an two images of the same planar surface in space are related b a homograph assuming a pinhole camera model). 9 From Wiipedia
8.7976964e-01 3.1245438e-01-3.9430589e+01-1.8389418e-01 9.3847198e-01 1.5315784e+02 1.9641425e-04-1.6015275e-05 1.0000000e+00
Plane Transfer Homograph P X World P' C H C' 11 Because we assume the world is a plane and transferred points are related b a homograph. If world plane coordinate is p then =Ap and =A p. = A A -1.
RANSAC for Fundamental Matri Step 1. Etract features Step 2. Compute a set of potential matches Step 3. do Step 3.1 select minimal sample i.e. 7 matches) generate Step 3.2 compute solutions) for F hpothesis) Step 3.3 determine inliers verif hpothesis) until a large enough set of the matches become inliers Step 4. Compute F based on all inliers Step 5. Loo for additional matches Step 6. Refine F based on all correct matches } 12
Eample: robust computation from H&Z Interest points 500/image) 640480) Putative correspondences 268) Best matchssd<20±320) Outliers 117) t=1.25 piel; 43 iterations) Inliers 151) Final inliers 262) 13
Corner detection Based on the idea of auto-correlation Important difference in all directions => interest point 14
Bacground: Moravec Corner Detector w tae a window w in the image shift it in four directions 10) 01) 11) -11) compute a difference for each compute the min difference at each piel local maima in the min image are the corners E) = wuv) I+u+v) Iuv) 2 uv in w 15
Shortcomings of Moravec Operator Onl tries 4 shifts. We d lie to consider all shifts. Uses a discrete rectangular window. We d lie to use a smooth circular or later elliptical) window. Uses a simple min function. We d lie to characterize variation with respect to direction. Result: Harris Operator 16
17 Harris detector 2 ) )) ) ) I I f W + + = + = + + I I I I )) ) ) ) with ) 2 ) ) ) ) = W I I f Auto-correlation fn SSD) for a point and a shift ) ) Discrete shifts can be avoided with the auto-correlation matri what is this? SSD means summed square difference
Harris detector Rewrite as inner dot) product The center portion is a 22 matri Have we seen this matri before? 18
19 Harris detector ) = I I I I I I W W W W ) 2 ) ) ) 2 )) ) ) ) ) )) Auto-correlation matri M W )
Harris detection Auto-correlation matri captures the structure of the local neighborhood measure based on eigenvalues of M 2 strong eigenvalues => interest point 1 strong eigenvalue => contour 0 eigenvalue => uniform region Interest point detection threshold on the eigenvalues local maimum for localization 20
Harris Corner Detector Corner strength R = DetM) TrM) 2 Let α and β be the two eigenvalues TrM) = α + β DetM) = αβ R is positive for corners negative for edges and small for flat regions Selects corner piels that are 8-wa local maima R = DetM) TrM) 2 is the Harris Corner 21 Detector.
Now we need a descriptor )? = ) Vector comparison using a distance measure How do we compare the two regions? 22
Distance Measures We can use the sum-square difference of the values of the piels in a square neighborhood about the points being compared. W 1 W 2 SSD = W 1 ij) W 2 ij)) 2 Wors when the motion is mainl a translation. 23
Some Matching Results from Matt Brown 24
Some Matching Results 25
Rotation/Scale Invariance original translated rotated scaled Is Harris invariant? Is correlation invariant? Translation Rotation Scale?????? 26
Rotation/Scale Invariance original translated rotated scaled Is Harris invariant? Is correlation invariant? Translation Rotation Scale?????? 27
Rotation/Scale Invariance original translated rotated scaled Is Harris invariant? Is correlation invariant? Translation Rotation Scale YES????? 28
Rotation/Scale Invariance original translated rotated scaled Is Harris invariant? Is correlation invariant? Translation Rotation Scale YES YES???? 29
Rotation/Scale Invariance original translated rotated scaled Is Harris invariant? Is correlation invariant? Translation Rotation Scale YES YES NO??? 30
Rotation/Scale Invariance original translated rotated scaled Is Harris invariant? Is correlation invariant? Translation Rotation Scale YES YES NO??? 31
Rotation/Scale Invariance original translated rotated scaled Is Harris invariant? Is correlation invariant? Translation Rotation Scale YES YES NO YES?? 32
Rotation/Scale Invariance original translated rotated scaled Is Harris invariant? Is correlation invariant? Translation Rotation Scale YES YES NO YES NO? 33
Rotation/Scale Invariance original translated rotated scaled Is Harris invariant? Is correlation invariant? Translation Rotation Scale YES YES NO YES NO NO 34
Matt Brown s Invariant Features Local image descriptors that are invariant unchanged) under image transformations 35
Canonical Frames 36
Canonical Frames 37
Multi-Scale Oriented Patches Etract oriented patches at multiple scales using dominant orientation 38
Multi-Scale Oriented Patches Sample scaled oriented patch 39
Multi-Scale Oriented Patches Sample scaled oriented patch 88 patch sampled at 5 scale 40
Multi-Scale Oriented Patches Sample scaled oriented patch 88 patch sampled at 5 scale Bias/gain normalized subtract the mean of a patch and divide b the variance to normalize) I = I µ)/σ 8 piels 41
Matching Interest Points: Summar Harris corners / correlation Etract and match repeatable image features Robust to clutter and occlusion BUT not invariant to scale and rotation Multi-Scale Oriented Patches Corners detected at multiple scales Descriptors oriented using local gradient Also sample a blurred image patch Invariant to scale and rotation Leads to: SIFT state of the art image features 42