Motivation Local Features (contd.) Readings: Mikolajczyk and Schmid; F&P Ch 10 Feature points are used also or: Image alignment (homography, undamental matrix) 3D reconstruction Motion tracking Object recognition Indexing and database retrieval Robot navigation other March 6, 008 Models o Image Change We want to: detect the same interest points regardless o image changes Geometry Rotation Similarity (rotation + uniorm scale) Aine (scale dependent on direction) valid or: orthographic camera, locally planar object Photometry Aine intensity change (I a I + b) Darya Frolova, Denis Simakov http://www.wisdom.weizmann.ac.il/~deniss/vision_spring04/iles/invariantfeatures.ppt he Basic Idea Review: A Simple Example Harris corner detector We should easily recognize the point by looking through a small window Shiting a window in any direction should give a large change in intensity C.Harris, M.Stephens. A Combined Corner and Edge Detector. 1988
Harris Detector: Basic Idea Window-averaged change o intensity or the shit [u,v]: xy, [ ] Euv (, ) = wxy (, ) Ix ( + uy, + v) Ixy (, ) lat region: no change in all directions edge : no change along the edge direction corner : signiicant change in all directions Window unction Window unction w(x,y) = Shited intensity or Intensity 1 in window, 0 outside Gaussian Expanding E(u,v) in a nd order aylor series expansion, we have,or small shits [u,v], a bilinear approximation: u Euv (, ) [ uv, ] M v where M is a matrix computed rom image derivatives: I x II x y M = w( x, y) xy, I xiy Iy Intensity change in shiting window: eigenvalue analysis u Euv (, ) [ uv, ] M v Ellipse E(u,v) = const direction o the astest change λ 1, λ eigenvalues o M (λ max ) -1/ (λ min) -1/ direction o the slowest change Classiication o image points using eigenvalues o M: λ 1 and λ are small; E is almost constant in all directions λ λ >> λ 1 Flat region Corner λ 1 and λ are large, λ 1 ~ λ ; E increases in all directions λ 1 >> λ Measure o corner response: ( trace ) R = det M k M det M = λ λ 1 trace M = λ + λ 1 (k empirical constant, k = 0.04-0.06) λ 1
Harris Detector R depends only on eigenvalues o M λ R < 0 Corner he Algorithm: Find points with large corner response unction R (R > threshold) ake the points o local maxima o R R is large or a corner R > 0 R is negative with large magnitude or an edge R is small or a lat region Flat R small R < 0 λ 1 Harris Detector: Worklow Harris Detector: Worklow Compute corner response R Harris Detector: Worklow Find points with large corner response: R>threshold Harris Detector: Worklow ake only the points o local maxima o R
Harris Detector: Worklow Harris Detector: Summary Average intensity change in direction [u,v] can be expressed as a bilinear orm: u Euv (, ) [ uv, ] M v Describe a point in terms o eigenvalues o M: measure o corner response ( ) R = λ1λ k λ1+ λ A good (corner) point should have a large intensity change in all directions, i.e. R should be large positive Rotation invariance Corner response R is invariant to image rotation Partial invariance to additive and multiplicative intensity changes Only derivatives are used => invariance to intensity shit I I + b Intensity scale: I a I Ellipse rotates but its shape (i.e. eigenvalues) remains the same R threshold R x (image coordinate) x (image coordinate) Quality o Harris detector or dierent scale changes Not invariant to image scale! Repeatability rate: # correspondences # possible correspondences All points will be classiied as edges Corner! C.Schmid et.al. Evaluation o Interest Point Detectors. IJCV 000
Consider regions (e.g. circles) o dierent sizes around a point Regions o corresponding sizes will look the same in both images he problem: how do we choose corresponding circles independently in each image? Solution: Design a unction on the region (circle), which is scale invariant (the same or corresponding regions, even i they are at dierent scales) Example: average intensity. For corresponding regions (even o dierent sizes) it will be the same. For a point in one image, we can consider it as a unction o region size (circle radius) Common approach: ake a local maximum o this unction Observation: region size, or which the maximum is achieved, should be invariant to image scale. Important: this scale invariant region size is ound in each image independently! Image 1 scale = 1/ Image Image 1 scale = 1/ Image region size region size s 1 region size s region size Scale Invariant Detectors Functions or determining scale Kernels: ( xx (,, ) yy (,, )) L= G x y + G x y σ σ σ (Laplacian) = Kernel Image Harris-Laplacian 1 Find local maximum o: Harris corner detector in space (image coordinates) Laplacian in scale scale y Harris x Laplacian DoG = G( x, y, kσ ) G( x, y, σ ) (Dierence o Gaussians) where Gaussian SIF (Lowe) Find local maximum o: Dierence o Gaussians in space and scale scale y DoG x + y 1 πσ σ Gxy (,, σ ) = e Note: both kernels are invariant to scale and rotation DoG 1 K.Mikolajczyk, C.Schmid. Indexing Based on Scale Invariant Interest Points. ICCV 001 D.Lowe. Distinctive Image Features rom Scale-Invariant Keypoints. Accepted to IJCV 004 x
Scale Invariant Detectors Experimental evaluation o detectors w.r.t. scale change Repeatability rate: # correspondences # possible correspondences : Summary Given: two images o the same scene with a large scale dierence between them Goal: ind the same interest points independently in each image Solution: search or maxima o suitable unctions in scale and in space (over the image) K.Mikolajczyk, C.Schmid. Indexing Based on Scale Invariant Interest Points. ICCV 001 Methods: 1. Harris-Laplacian [Mikolajczyk, Schmid]: maximize Laplacian over scale, Harris measure o corner response over the image. SIF [Lowe]: maximize Dierence o Gaussians over scale and space Aine Invariant Detection Above we considered: Similarity transorm (rotation + uniorm scale) Aine Invariant Detection ake a local intensity extremum as initial point Go along every ray starting rom this point and stop when extremum o unction is reached Now we go on to: Aine transorm (rotation + non-uniorm scale) points along the ray We will obtain approximately corresponding regions () t = t 1 t o It () I 0 I () t I dt 0 Remark: we search or scale in every direction.uytelaars, L.V.Gool. Wide Baseline Stereo Matching Based on Local, Ainely Invariant Regions. BMVC 000. Aine Invariant Detection Aine Invariant Detection Algorithm summary (detection o aine invariant region): Start rom a local intensity extremum point Go in every direction until the point o extremum o some unction Curve connecting the points is the region boundary Compute geometric moments o orders up to or this region Replace the region with ellipse all points corresponding to extremum o (t) along rays originating rom the same local extremum are linked to enclose an (ainely invariant) region (see igure ). his oten irregularly-shaped region is then replaced by an ellipse having the same shape moments up to the second order. his ellipse-itting is ainely invariant as well..uytelaars, L.V.Gool. Wide Baseline Stereo Matching Based on Local, Ainely Invariant Regions. BMVC 000.
Aine Invariant Detection Aine Invariant Detection he regions ound may not exactly correspond, so we approximate them with ellipses Geometric Moments: mpq p q = x y ( x, y) dxdy Fact: moments m pq uniquely determine the unction aking to be the characteristic unction o a region (1 inside, 0 outside), moments o orders up to allow to approximate the region by an ellipse his ellipse will have the same moments o orders up to as the original region Covariance matrix o region points deines an ellipse: p 1 1 1 p 1 pp region 1 ( p = [x, y] is relative to the center o mass) q= Ap AΣ A 1 q Ellipses, computed or corresponding regions, also correspond! 1 q 1 qq region Aine Invariant Detection : Summary Under aine transormation, we do not know in advance shapes o the corresponding regions Ellipse given by geometric covariance matrix o a region robustly approximates this region For corresponding regions ellipses also correspond. Methods: 1. Search or extremum along rays [uytelaars, Van Gool]:. Maximally Stable Extremal Regions [Matas et.al.]