Computer Vision Lecture 13

Transcription

1 N pixel Coure Outline Coputer Viion Lecture 3 Local Feature II Iage Proceing Baic Segentation & Grouping Object Recognition Object Categorization I Sliding Window baed Object Detection Local Feature & Matching Local Feature Detection and Decription Recognition with Local Feature Batian Leibe RWTH Aachen leibe@viion.rwth-aachen.de Object Categorization II Part baed Approache 3D Recontruction Motion and Tracking 3 Recap: Local Feature Matching Outline A N pixel A 2 A 3 A Siilarity eaure d(, ) T A B B. Find a et o ditinctive keypoint 2. Deine a region around each keypoint 3. Extract and noralize the region content 4. Copute a local decriptor ro the noralized region 5. Match local decriptor 4 Recap: Requireent or Local Feature Proble : Detect the ae point independently in both iage Proble 2: For each point correctly recognize the correponding one We need a repeatable detector! We need a reliable and ditinctive decriptor! Slide credit: Darya Frolova, Deni Siakov? 5 Recap: Harri Detector [Harri88] Recap: Harri Detector Repone [Harri88] Copute econd oent atrix (autocorrelation atrix) 2 I x ( D) I xi y ( ) D M( I, D) g( I ) 2 I xi y ( D) I y ( D). Iage derivative I x I y 2. Square o derivative I x 2 I y 2 I x I y 3. Gauian ilter g( I ) 4. Cornerne unction two trong eigenvalue 2 R det[ M(, )] [trace( M(, ] I D I D g( I ) g( I ) [ g( I I )] [ g( I ) g( I )] x y x y x y g(i x2 ) g(i y2 ) g(i x I y ) Eect: A very precie corner detector. 5. Peror non-axiu uppreion R 6 7

2 Recap: Heian Detector [Beaudet78] Recap: Heian Detector Repone [Beaudet78] Heian deterinant I Heian ( I) I xx xy I xy I yy I xx I xy I yy det( Heian( I)) I I xx yy I 2 xy Eect: Repone ainly on corner and trongly textured area. In Matlab: I. I ( I ).^ 2 xx yy xy 8 9 Topic o Thi Lecture Fro Point to Region Local Feature Extraction (cont d) Scale Invariant Region Selection Orientation noralization Aine Invariant Feature Extraction The Harri and Heian operator deine interet point. Precie localization High repeatability Local Decriptor SIFT Application In order to copare thoe point, we need to copute a decriptor over a region. How can we deine uch a region in a cale invariant anner? I.e. how can we detect cale invariant interet region? 0 Naïve Approach: Exhautive Search Naïve Approach: Exhautive Search Multi-cale procedure Copare decriptor while varying the patch ize Multi-cale procedure Copare decriptor while varying the patch ize A Siilarity eaure d ( A, B) B 2 A Siilarity eaure d ( A, B) B 3 2

3 Naïve Approach: Exhautive Search Multi-cale procedure Copare decriptor while varying the patch ize Naïve Approach: Exhautive Search Multi-cale procedure Copare decriptor while varying the patch ize A Siilarity eaure d ( A, B) B 4 A Siilarity eaure d ( A, B) B 5 Naïve Approach: Exhautive Search Autoatic Scale Selection Coparing decriptor while varying the patch ize Coputationally ineicient Ineicient but poible or atching Prohibitive or retrieval in large databae Prohibitive or recognition Solution: Deign a unction on the region, which i cale invariant (the ae or correponding region, even i they are at dierent cale) Exaple: average intenity. For correponding region (even o dierent ize) it will be the ae. For a point in one iage, we can conider it a a unction o region ize (patch width) A Siilarity eaure d( A, B ) B 6 Iage Region ize Slide credit: Kriten Grauan cale = ½ Iage 2 Region ize 7 Autoatic Scale Selection Autoatic Scale Selection Coon approach: Take a local axiu o thi unction. Obervation: region ize or which the axiu i achieved hould be invariant to iage cale. Function repone or increaing cale (cale ignature) Iportant: thi cale invariant region ize i ound in each iage independently! Iage cale = ½ Iage 2 2 = ½ Region ize 2 Slide credit: Kriten Grauan Region ize 8 ( Ii i ( x, ( Ii ( x, i 9 3

4 Autoatic Scale Selection Function repone or increaing cale (cale ignature) Autoatic Scale Selection Function repone or increaing cale (cale ignature) ( Ii i ( x, ( Ii ( x, i 20 ( Ii i ( x, ( Ii ( x, i 2 Autoatic Scale Selection Autoatic Scale Selection Function repone or increaing cale (cale ignature) Function repone or increaing cale (cale ignature) ( Ii i ( x, ( Ii ( x, i 22 ( Ii i ( x, ( Ii ( x, i 23 Autoatic Scale Selection Autoatic Scale Selection Function repone or increaing cale (cale ignature) Noralize: Recale to ixed ize ( Ii i ( x, ( Ii i ( x, )) ( Ii i ( x, ( Ii i ( x, )) 24 Slide credit: Tinne Tuytelaar 25 4

5 What I A Ueul Signature Function? Laplacian-o-Gauian = blob detector Characteritic Scale We deine the characteritic cale a the cale that produce peak o Laplacian repone Characteritic cale 26 T. Lindeberg (998). "Feature detection with autoatic cale election." International Journal o Coputer Viion 30 (2): pp Laplacian-o-Gauian (LoG) Laplacian-o-Gauian (LoG) Interet point: Local axia in cale pace o Laplacian-o- Gauian 5 4 Interet point: Local axia in cale pace o Laplacian-o- Gauian 5 4 L ( ) L ( ) xx yy 3 L ( ) L ( ) xx yy Slide adapted ro Krytian Mikolajczyk Slide adapted ro Krytian Mikolajczyk Laplacian-o-Gauian (LoG) Laplacian-o-Gauian (LoG) Interet point: Local axia in cale pace o Laplacian-o- Gauian 5 4 Interet point: Local axia in cale pace o Laplacian-o- Gauian 5 4 L ( ) L ( ) xx yy 3 L ( ) L ( ) xx yy Lit o (x, y, σ) Slide adapted ro Krytian Mikolajczyk Slide adapted ro Krytian Mikolajczyk 5

6 Reaple Blur Subtract LoG Detector: Worklow LoG Detector: Worklow LoG Detector: Worklow Dierence-o-Gauian (DoG) We can eiciently approxiate the Laplacian with a dierence o Gauian: 2 L Gxx x y Gyy x y (,, ) (,, ) (Laplacian) DoG G( x, y, k) G( x, y, ) (Dierence o Gauian) Advantage? No need to copute 2 nd derivative. Gauian are coputed anyway, e.g. in a Gauian pyraid Key point localization with DoG DoG Eicient Coputation Detect axia o dierence-o-gauian (DoG) in cale pace Coputation in Gauian cale pyraid Then reject point with low contrat (threhold) Eliinate edge repone Sapling with tep 4 =2 Candidate keypoint: lit o (x,y,σ) Original iage 2 4 Slide credit: David Lowe Slide adapted ro Krytian Mikolajczyk 37 6

7 Reult: Lowe DoG Harri-Laplace [Mikolajczyk 0]. Initialization: Multicale Harri corner detection Slide adapted ro Krytian Mikolajczyk Coputing Harri unction Detecting local axia 40 Harri-Laplace [Mikolajczyk 0]. Initialization: Multicale Harri corner detection 2. Scale election baed on Laplacian (ae procedure with Heian Heian-Laplace) Harri point Suary: Scale Invariant Detection Given: Two iage o the ae cene with a large cale dierence between the. Goal: Find the ae interet point independently in each iage. Solution: Search or axia o uitable unction in cale and in pace (over the iage). Two trategie Laplacian-o-Gauian (LoG) Dierence-o-Gauian (DoG) a a at approxiation Slide adapted ro Krytian Mikolajczyk Harri-Laplace point 4 Thee can be ued either on their own, or in cobination with ingle-cale keypoint detector (Harri, Heian). 42 Topic o Thi Lecture Rotation Invariant Decriptor Local Feature Extraction (cont d) Scale Invariant Region Selection Orientation noralization Aine Invariant Feature Extraction Local Decriptor SIFT Find local orientation Doinant direction o gradient or the iage patch Rotate patch according to thi angle Thi put the patche into a canonical orientation. Application 43, Matthew Brown 44 7

8 Orientation Noralization: Coputation Topic o Thi Lecture Copute orientation hitogra Select doinant orientation Noralize: rotate to ixed orientation [Lowe, SIFT, 999] Local Feature Extraction (cont d) Scale Invariant Region Selection Orientation noralization Aine Invariant Feature Extraction Local Decriptor SIFT Application Slide adapted ro David Lowe 0 2 p The Need or Invariance Aine Adaptation Proble: Deterine the characteritic hape o the region. Auption: hape can be decribed by local aine rae. Up to now, we had invariance to Tranlation Scale Rotation Not uicient to atch region under viewpoint change For thi, we need alo aine adaptation Slide credit: Tinne Tuytelaar 47 Solution: iterative approach Ue a circular window to copute econd oent atrix. Copute eigenvector to adapt the circle to an ellipe. Recopute econd oent atrix uing new window and iterate Slide adapted ro Svetlana Lazebnik 48 Iterative Aine Adaptation Aine Noralization/Dekewing rotate recale. Detect keypoint, e.g. ulti-cale Harri 2. Autoatically elect the cale 3. Adapt aine hape baed on econd order oent atrix 4. Reine point location K. Mikolajczyk and C. Schid, Scale and aine invariant interet point detector, 49 IJCV 60():63-86, Slide credit: Tinne Tuytelaar Step Rotate the ellipe ain axi to horizontal Scale the x axi, uch that it or a circle Slide credit: Tinne Tuytelaar 50 8

9 Aine Adaptation Exaple Aine Adaptation Exaple Scale-invariant region (blob) Aine-adapted blob 5 52 Suary: Aine-Inv. Feature Extraction Invariance v. Covariance Extract aine region Noralize region Eliinate rotational abiguity Copare decriptor Invariance: eature(tranor(iage)) = eature(iage) Covariance: eature(tranor(iage)) = tranor(eature(iage)) 53 Covariant detection invariant decription, David Lowe 54 Topic o Thi Lecture Local Decriptor Local Feature Extraction (cont d) Orientation noralization Aine Invariant Feature Extraction Local Decriptor SIFT Application Recognition with Local Feature Matching local eature Finding conitent coniguration Alignent: linear tranoration Aine etiation Hoography etiation 55 We know how to detect point Next quetion: How to decribe the or atching? Slide credit: Kriten Grauan? Point decriptor hould be:. Invariant 2. Ditinctive 56 9

10 Local Decriptor Siplet decriptor: lit o intenitie within a patch. What i thi going to be invariant to? Feature Decriptor Diadvantage o patche a decriptor: Sall hit can aect atching core a lot Solution: hitogra Slide credit: Kriten Grauan p 58 Feature Decriptor: SIFT Scale Invariant Feature Tranor Decriptor coputation: Divide patch into 4x4 ub-patche: 6 cell Copute hitogra o gradient orientation (8 reerence angle) or all pixel inide each ub-patch Reulting decriptor: 4x4x8 = 28 dienion Overview: SIFT Extraordinarily robut atching technique Can handle change in viewpoint up to ~60 deg. out-o-plane rotation Can handle igniicant change in illuination Soetie even day v. night (below) Fat and eicient can run in real tie Lot o code available David G. Lowe. "Ditinctive iage eature ro cale-invariant keypoint. IJCV 60 (2), pp. 9-0, Slide credit: Steve Seitz 60 Working with SIFT Decriptor Local Decriptor: SURF One iage yield: n 2D point giving poition o the patche [n x 2 atrix] n cale paraeter peciying the ize o each patch [n x vector] n orientation paraeter peciying the angle o the patch [n x vector] n 28-dienional decriptor: each one i a hitogra o the gradient orientation within a patch [n x 28 atrix] Fat approxiation o SIFT idea Eicient coputation by 2D box ilter & integral iage 6 tie ater than SIFT Equivalent quality or object identiication GPU ipleentation available Feature 00Hz (detector + decriptor, ig) Slide credit: Steve Seitz 6 62 [Bay, ECCV 06], [Corneli, CVGPU 08] 0

11 You Can Try It At Hoe For ot local eature detector, executable are available online: Topic o Thi Lecture Application o Local Invariant Feature Local Feature Extraction (cont d) Orientation noralization Aine Invariant Feature Extraction Local Decriptor SIFT Application Recognition with Local Feature Matching local eature Finding conitent coniguration Alignent: linear tranoration Aine etiation Hoography etiation Wide baeline tereo Motion tracking Panoraa Mobile robot navigation 3D recontruction Recognition Speciic object Texture Categorie 65 Slide credit: Kriten Grauan 66 Wide-Baeline Stereo Autoatic Moaicing 67 Iage ro T. Tuytelaar ECCV 2006 tutorial 68 [Brown & Lowe, ICCV 03]

12 Panoraa Stitching Recognition o Speciic Object, Scene Schid and Mohr 997 Sivic and Zieran, 2003 iphone verion available 69 [Brown, Szeliki, and Winder, 2005] Slide credit: Kriten Grauan Rothganger et al Lowe Recognition o Categorie Value o Local Feature Contellation odel Bag o word Advantage Critical to ind ditinctive and repeatable local region or ultiview atching. Coplexity reduction via election o ditinctive point. Decribe iage, object, part without requiring egentation; robutne to clutter & occluion. Robutne: iilar decriptor in pite o oderate view change, noie, blur, etc. Weber et al. (2000) Fergu et al. (2003) Curka et al. (2004) Dorko & Schid (2005) Sivic et al. (2005) Lazebnik et al. (2006), How can we ue local eature or uch application? Next week: atching and recognition 7 Slide adapted ro Kriten Grauan 72 Reerence and Further Reading More detail on hoography etiation can be ound in Chapter 4.7 o R. Hartley, A. Zieran Multiple View Geoetry in Coputer Viion 2nd Ed., Cabridge Univ. Pre, 2004 Detail about the DoG detector and the SIFT decriptor can be ound in D. Lowe, Ditinctive iage eature ro cale-invariant keypoint, IJCV 60(2), pp. 9-0, 2004 Try the available local eature detector and decriptor 2