CS230: Lecture 5 Advanced topics in Object Detection

Transcription

1 CS230: Lecture 5 Advanced topics in Object Detection Stanford University

2 Today s outline We will learn: - the Intuition behind object detection methods - a series of computer vision papers I. Object detection Intuition II. R-CNN III. Fast R-CNN IV. Faster R-CNN V. YOLO Let s go

3 Object Detection Intuition y = (p c,b x,b y,b h,b w,c) b w b h + (b x,b y ) p c = 1 : confidence of an object being present in the bounding box c = 0 : class of the object being detected (here 0 for car )

4 Object Detection Intuition y = (p c,b x,b y,b h,b w,c)

5 Object Detection Intuition Classifier Far too many computations y p c

6 R-CNN (Regions with CNN features) input image region proposal How do you find these regions? 4096-d CNN ! SVM (car) SVM (pedestrian) SVM (traffic light) yes no no warped proposal Ross Girshick Jeff Donahue Trevor Darrell Jitendra Malik: Rich feature hierarchies for accurate object detection and semantic segmentation (2014)

7 R-CNN (Regions with CNN features) input image segmentation labels region proposals segmentation proposals At test time on a new image, 2000 regions proposals! region 4096-d CNN ! SVM (car) SVM (pedestrian) SVM (traffic light) yes no no x 2000 But we only needed two boxes: 1 car + 1 pedestrian J.R.R. Uijlings, K.E.A. van de Sande, T. Gevers, and A.W.M. Smeulders: Selective Search for Object Recognition (2012)

8 Non-max suppression region + scores + classes region proposals input image 0.12 car selective search CNN + SVMs 0.11 car 0.50 pedestrian 0.54 car 0.01 fire hydrant 0.14 zebra 0.32 tree 0.75 pedestrian Union Intersection B1 B2 B1 NMS + score threshold Intersection over Union B2 B1 B2 IoU = = B1 B car 0.75 pedestrian Still slow And training is not unified: CNN and each SVM have to be trained separately!

9 Evaluate the performance of your model - BG: background and non-labelled objects - Loc: localization error (IoU) - Sim: Confusion with similar objects - Oth: Confusion with dissimilar objects Derek Hoiem, Yodsawalai Chodpathumwan, and Qieyun Dai: Diagnosing Error in Object Detectors

10 Fast R-CNN R-CNN drawbacks: Fast R-CNN solutions: - Training is a multi-stage pipeline - Training is expensive in space and time - Testing is slow - Share computations for the CNN (not per-region anymore) - Make training single-stage with multi-task loss - No disk storage

11 Fast R-CNN input image + RoIs feature map 4096-d CNN RoI Pooling layer FCs ! How do you train this? #classes ! FCs (Softmax) FCs (Bbox regressor) #classes L( p, u, t u, v) = L ( p, u) + λι cls [ u 1 ] L ( t u, v) loc Ross Girshick: Fast R-CNN (2015)

12 Let s make it even faster

13 Faster R-CNN feature map input image (no RoI) feature map with RoIs attention Region Proposal Network CNN feature map Fast R-CNN k scores k anchors per window ! FCs (Softmax) FCs (Bbox regressor) ( ) ( 1 1 * * * L ({ pi }, {ti } ) = Lcls pi, pi + λ pi Lreg ti, ti N cls i N reg i Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun: Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2016) ) k

14 YOLO preprocessed image (608, 608, 3) 19 encoding (19,19, 5, 85) Deep CNN reduction factor: p c b x b y b h b w 80 class probabilities box 1 box 2 box 3 Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi: You Only Look Once: Unified, Real-Time Object Detection (2015) Joseph Redmon, Ali Farhadi: YOLO9000: Better, Faster, Stronger (2016) box 4 box 5

15 YOLO box 1 b x b y b h b w 80 class probabilities p c p 1 p 2 p 3 p 4 p 5 p 76 p 77 p 78 p 79 p 80 scores = p c *! p 1 p 2 p 3 p 78 p 79 p 80 = p p c 1 p p c 2 p p c 3! p p c 78 p p c 79 p p c 80 = ! find the max score: 0.44 box: (b x,b y,b h,b w ) class: c = 3 ( car ) the box (b,b,b,b ) has detected c = 3 ( car ) with probability score: 0.44 x y h w

16 YOLO Need to filter: - Score thresholding - NMS

17 YOLO Before non-max suppression After non-max suppression Non-Max Suppression

18 YOLO demo Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi: You Only Look Once: Unified, Real-Time Object Detection