Local Features (contd.)

Transcription

1 Motivation Local Features (contd.) Readings: Mikolajczyk and Schmid; F&P Ch 10 Feature points are used also or: Image alignment (homography, undamental matrix) 3D reconstruction Motion tracking Object recognition Indexing and database retrieval Robot navigation other March 6, 008 Models o Image Change We want to: detect the same interest points regardless o image changes Geometry Rotation Similarity (rotation + uniorm scale) Aine (scale dependent on direction) valid or: orthographic camera, locally planar object Photometry Aine intensity change (I a I + b) Darya Frolova, Denis Simakov he Basic Idea Review: A Simple Example Harris corner detector We should easily recognize the point by looking through a small window Shiting a window in any direction should give a large change in intensity C.Harris, M.Stephens. A Combined Corner and Edge Detector. 1988

2 Harris Detector: Basic Idea Window-averaged change o intensity or the shit [u,v]: xy, [ ] Euv (, ) = wxy (, ) Ix ( + uy, + v) Ixy (, ) lat region: no change in all directions edge : no change along the edge direction corner : signiicant change in all directions Window unction Window unction w(x,y) = Shited intensity or Intensity 1 in window, 0 outside Gaussian Expanding E(u,v) in a nd order aylor series expansion, we have,or small shits [u,v], a bilinear approximation: u Euv (, ) [ uv, ] M v where M is a matrix computed rom image derivatives: I x II x y M = w( x, y) xy, I xiy Iy Intensity change in shiting window: eigenvalue analysis u Euv (, ) [ uv, ] M v Ellipse E(u,v) = const direction o the astest change λ 1, λ eigenvalues o M (λ max ) -1/ (λ min) -1/ direction o the slowest change Classiication o image points using eigenvalues o M: λ 1 and λ are small; E is almost constant in all directions λ λ >> λ 1 Flat region Corner λ 1 and λ are large, λ 1 ~ λ ; E increases in all directions λ 1 >> λ Measure o corner response: ( trace ) R = det M k M det M = λ λ 1 trace M = λ + λ 1 (k empirical constant, k = ) λ 1

3 Harris Detector R depends only on eigenvalues o M λ R < 0 Corner he Algorithm: Find points with large corner response unction R (R > threshold) ake the points o local maxima o R R is large or a corner R > 0 R is negative with large magnitude or an edge R is small or a lat region Flat R small R < 0 λ 1 Harris Detector: Worklow Harris Detector: Worklow Compute corner response R Harris Detector: Worklow Find points with large corner response: R>threshold Harris Detector: Worklow ake only the points o local maxima o R

4 Harris Detector: Worklow Harris Detector: Summary Average intensity change in direction [u,v] can be expressed as a bilinear orm: u Euv (, ) [ uv, ] M v Describe a point in terms o eigenvalues o M: measure o corner response ( ) R = λ1λ k λ1+ λ A good (corner) point should have a large intensity change in all directions, i.e. R should be large positive Rotation invariance Corner response R is invariant to image rotation Partial invariance to additive and multiplicative intensity changes Only derivatives are used => invariance to intensity shit I I + b Intensity scale: I a I Ellipse rotates but its shape (i.e. eigenvalues) remains the same R threshold R x (image coordinate) x (image coordinate) Quality o Harris detector or dierent scale changes Not invariant to image scale! Repeatability rate: # correspondences # possible correspondences All points will be classiied as edges Corner! C.Schmid et.al. Evaluation o Interest Point Detectors. IJCV 000

5 Consider regions (e.g. circles) o dierent sizes around a point Regions o corresponding sizes will look the same in both images he problem: how do we choose corresponding circles independently in each image? Solution: Design a unction on the region (circle), which is scale invariant (the same or corresponding regions, even i they are at dierent scales) Example: average intensity. For corresponding regions (even o dierent sizes) it will be the same. For a point in one image, we can consider it as a unction o region size (circle radius) Common approach: ake a local maximum o this unction Observation: region size, or which the maximum is achieved, should be invariant to image scale. Important: this scale invariant region size is ound in each image independently! Image 1 scale = 1/ Image Image 1 scale = 1/ Image region size region size s 1 region size s region size Scale Invariant Detectors Functions or determining scale Kernels: ( xx (,, ) yy (,, )) L= G x y + G x y σ σ σ (Laplacian) = Kernel Image Harris-Laplacian 1 Find local maximum o: Harris corner detector in space (image coordinates) Laplacian in scale scale y Harris x Laplacian DoG = G( x, y, kσ ) G( x, y, σ ) (Dierence o Gaussians) where Gaussian SIF (Lowe) Find local maximum o: Dierence o Gaussians in space and scale scale y DoG x + y 1 πσ σ Gxy (,, σ ) = e Note: both kernels are invariant to scale and rotation DoG 1 K.Mikolajczyk, C.Schmid. Indexing Based on Scale Invariant Interest Points. ICCV 001 D.Lowe. Distinctive Image Features rom Scale-Invariant Keypoints. Accepted to IJCV 004 x

6 Scale Invariant Detectors Experimental evaluation o detectors w.r.t. scale change Repeatability rate: # correspondences # possible correspondences : Summary Given: two images o the same scene with a large scale dierence between them Goal: ind the same interest points independently in each image Solution: search or maxima o suitable unctions in scale and in space (over the image) K.Mikolajczyk, C.Schmid. Indexing Based on Scale Invariant Interest Points. ICCV 001 Methods: 1. Harris-Laplacian [Mikolajczyk, Schmid]: maximize Laplacian over scale, Harris measure o corner response over the image. SIF [Lowe]: maximize Dierence o Gaussians over scale and space Aine Invariant Detection Above we considered: Similarity transorm (rotation + uniorm scale) Aine Invariant Detection ake a local intensity extremum as initial point Go along every ray starting rom this point and stop when extremum o unction is reached Now we go on to: Aine transorm (rotation + non-uniorm scale) points along the ray We will obtain approximately corresponding regions () t = t 1 t o It () I 0 I () t I dt 0 Remark: we search or scale in every direction.uytelaars, L.V.Gool. Wide Baseline Stereo Matching Based on Local, Ainely Invariant Regions. BMVC 000. Aine Invariant Detection Aine Invariant Detection Algorithm summary (detection o aine invariant region): Start rom a local intensity extremum point Go in every direction until the point o extremum o some unction Curve connecting the points is the region boundary Compute geometric moments o orders up to or this region Replace the region with ellipse all points corresponding to extremum o (t) along rays originating rom the same local extremum are linked to enclose an (ainely invariant) region (see igure ). his oten irregularly-shaped region is then replaced by an ellipse having the same shape moments up to the second order. his ellipse-itting is ainely invariant as well..uytelaars, L.V.Gool. Wide Baseline Stereo Matching Based on Local, Ainely Invariant Regions. BMVC 000.

7 Aine Invariant Detection Aine Invariant Detection he regions ound may not exactly correspond, so we approximate them with ellipses Geometric Moments: mpq p q = x y ( x, y) dxdy Fact: moments m pq uniquely determine the unction aking to be the characteristic unction o a region (1 inside, 0 outside), moments o orders up to allow to approximate the region by an ellipse his ellipse will have the same moments o orders up to as the original region Covariance matrix o region points deines an ellipse: p p 1 pp region 1 ( p = [x, y] is relative to the center o mass) q= Ap AΣ A 1 q Ellipses, computed or corresponding regions, also correspond! 1 q 1 qq region Aine Invariant Detection : Summary Under aine transormation, we do not know in advance shapes o the corresponding regions Ellipse given by geometric covariance matrix o a region robustly approximates this region For corresponding regions ellipses also correspond. Methods: 1. Search or extremum along rays [uytelaars, Van Gool]:. Maximally Stable Extremal Regions [Matas et.al.]