Large Scale Environment in Mobile Robotics Recognition Tasks Boyan Bonev, Miguel Cazorla {boyan,miguel}@dccia.ua.es Robot Vision Group Department of Computer Science and Artificial Intelligence University of Alicante April 27th, 27 Bonev, Cazorla Environment in Recognition Tasks 1 / 35 WAF 28
Outline 1 Localization and vision Initial approach approach 2 3 4 Bonev, Cazorla Environment in Recognition Tasks 2 / 35 WAF 28
Outline Localization and vision Initial approach approach 1 Localization and vision Initial approach approach 2 3 4 Bonev, Cazorla Environment in Recognition Tasks 3 / 35 WAF 28
Localization Localization and vision Initial approach approach Mobile robots Sensors + Maps Localization Bonev, Cazorla Environment in Recognition Tasks 4 / 35 WAF 28
Vision-based localization Localization and vision Initial approach approach Environments Ad-hoc Natural Indoor Outdoor Bonev, Cazorla Environment in Recognition Tasks 5 / 35 WAF 28
Localization and vision Initial approach approach Appearance based visual recognition Visual recognition approaches Structural-description Structure from high level features Appearance-based Images or low level features 34567891 5 1 y69a1,1 34567891 5 1 y69a1,2 34567891 5 1 y69a1,3 34567891 5 1 y69a1,4 34567891 5 1 y69a1,5 34567891 5 1 y69a1,6 34567891 5 1 y69a1,7 34567891 5 1 y69a1,8 34567891 5 1 y69a1,9 34567891 5 1 y69a1,1 34567891 5 1 y69a2,1 34567891 5 1 y69a2,2 34567891 5 1 y69a2,3 34567891 5 1 y69a2,4 34567891 5 1 y69a2,5 34567891 5 1 y69a2,6 34567891 5 1 y69a2,7 34567891 5 1 y69a2,8 34567891 5 1 y69a2,9 34567891 5 1 y69a2,1 34567891 5 1 y69a3,1 34567891 5 1 y69a3,2 34567891 5 1 y69a3,3 34567891 5 1 y69a3,4 34567891 5 1 y69a3,5 34567891 5 1 y69a3,6 34567891 5 1 y69a3,7 34567891 5 1 y69a3,8 34567891 5 1 y69a3,9 34567891 5 1 y69a3,1 34567891 5 1 y69a4,1 34567891 5 1 y69a4,2 34567891 5 1 y69a4,3 34567891 5 1 y69a4,4 34567891 5 1 y69a4,5 34567891 5 1 y69a4,6 34567891 5 1 y69a4,7 34567891 5 1 y69a4,8 34567891 5 1 y69a4,9 34567891 5 1 y69a4,1 34567891 5 1 y69a5,1 34567891 5 1 y69a5,2 34567891 5 1 y69a5,3 34567891 5 1 y69a5,4 34567891 5 1 y69a5,5 34567891 5 1 y69a5,6 34567891 5 1 y69a5,7 34567891 5 1 y69a5,8 34567891 5 1 y69a5,9 34567891 5 1 y69a5,1 Bonev, Cazorla Environment in Recognition Tasks 6 / 35 WAF 28
Omnidirectional images Localization and vision Initial approach approach Omnidirectional images Local views with 36 o visibility Independence of the direction of the route. Convenient representation for rotation-invariant recognition Bonev, Cazorla Environment in Recognition Tasks 7 / 35 WAF 28
Omnidirectional images Localization and vision Initial approach approach Bonev, Cazorla Environment in Recognition Tasks 8 / 35 WAF 28
Feature selection approach Localization and vision Initial approach approach N F N S M M M 1-Fold CV Train Test Low level filters Nitzberg Canny, Gradient Color Filters Histograms comparison 2,4, and bins discretization Images 4 3 2 # reference image Vectors Vectors All Features Selected F. Best F. Set Single similiarity function response H Error? 1 5 1 15 2 25 # test image Bonev, Cazorla Environment in Recognition Tasks 9 / 35 WAF 28
Localization and vision Initial approach approach Large environments Bonev, Cazorla Environment in Recognition Tasks 1 / 35 WAF 28
Approach Localization and vision Initial approach approach Unsupervised partitioning of the environment multiple hypotheses MCL to select a single hypothesis gradient Gradient of the JR divergence 5 1 15 2 25 3 35 4 # image # reference image Similiarity functions responses 4 H1 H2 H3 H4 3 H5 H6 H7 H8 H9 H1 2 H H H13 H14 H15 1 H16 H17 H18 H19 H2 5 1 15 2 25 # test image likelihood likelihood likelihood likelihood Particle filter, iterations 1,5,8, 1.5 1 2 3 4 1.5 1 2 3 4 1.5 1 2 3 4 1.5 1 2 3 4 particle position (# reference image) Bonev, Cazorla Environment in Recognition Tasks / 35 WAF 28
Outline 1 Localization and vision Initial approach approach 2 3 4 Bonev, Cazorla Environment in Recognition Tasks / 35 WAF 28
Sequence of images Bonev, Cazorla Environment in Recognition Tasks 13 / 35 WAF 28
Sequence of images Bonev, Cazorla Environment in Recognition Tasks 14 / 35 WAF 28
Sequence of images Bonev, Cazorla Environment in Recognition Tasks 15 / 35 WAF 28
Divide the problem How to divide the problem? Try all possible partitions Clustering algorithms Look for local variations in the information Bonev, Cazorla Environment in Recognition Tasks 16 / 35 WAF 28
Jensen-Rényi divergence Information-theoretic divergence measures entropy based unfeasible for multidimensional data Jensen-Rényi divergence α-entropy based feasible estimation in high-dimensional spaces (Hero and Michel, 22) J-R divergence applications may be defined between any number of probability distributions may be used to detect edges with a sliding window Region A Region B Data W1 W2 JR =.4 W1 W2 JR =.5 W1 W2 JR =.6 W1 W2 JR =.7 W1 W2 JR =.6 W1 W2 JR =.5 W1 W2 JR =.4 J-R divergence Bonev, Cazorla Environment in Recognition Tasks 17 / 35 WAF 28
Jensen-Rényi divergence J-R divergence simplified for two equally weighted distributions Data Region A Region B where JR α (p 1, p 2 ) = = H α ( p1 + p 2 2 ) H α(p 1 ) + H α (p 2 ), 2 Rényi entropy H α is estimated with Hero and Michel s method (based on minimal spanning trees) complexity depending on the number of samples O(N log N) W1 W2 JR =.4 W1 W2 JR =.5 W1 W2 JR =.6 W1 W2 JR =.7 W1 W2 JR =.6 W1 W2 JR =.5 W1 W2 JR =.4 J-R divergence Bonev, Cazorla Environment in Recognition Tasks 18 / 35 WAF 28
Jensen-Rényi divergence Response and entropy divergence analysis of Nitzberg filter Region A Region B Data W1 W1 W1 W2 W2 W2 JR =.4 JR =.5 JR =.6 W1 W1 W1 W2 W2 W2 JR =.7 JR =.6 JR =.5 response of Nitzberg filter local entropy divergence divergence gradient J-R divergence W1 W2 JR =.4 1 2 3 4 5 6 7 8 # Image Bonev, Cazorla Environment in Recognition Tasks 19 / 35 WAF 28
Multiscale Jensen-Rényi divergence JR divergence at various window sizes #241 JR divergence #254 3 2 window size 1 #273 22 23 24 25 26 27 28 # image Bonev, Cazorla Environment in Recognition Tasks 2 / 35 WAF 28
Resulting partitions Gradient of the JR divergence gradient 5 1 15 2 25 3 35 4 # image Bonev, Cazorla Environment in Recognition Tasks 21 / 35 WAF 28
Resulting partitions Bonev, Cazorla Environment in Recognition Tasks 22 / 35 WAF 28
Feature extraction Extraction of global features Rings Histograms 4 bins 1 2 3 Filters Bank 1 2. K 1 2.. 4 K = 17 68 Img i C = 4 C x K Feature Vector...... N = C x K x (B-1) = 24 features F Bonev, Cazorla Environment in Recognition Tasks 23 / 35 WAF 28
Training Selection of features N F N S M M M 1-Fold CV Train Test Images Vectors Vectors Error All Features Selected F.? Best F. Set Bonev, Cazorla Environment in Recognition Tasks 24 / 35 WAF 28
NNs in the feature space The ideal case: F N F N-1...... F 4 F 3 F 2 F 1 Bonev, Cazorla Environment in Recognition Tasks 25 / 35 WAF 28
NNs in the feature space The wrong case: F N F N-1...... F 4 F 3 F 2 F 1 Bonev, Cazorla Environment in Recognition Tasks 26 / 35 WAF 28
Response on the test set Classifier trained for images 14 148 Classifier trained for images 1 44 4 4 3 3 # reference image 2 # reference image 2 1 1 5 1 15 2 25 # test image 5 1 15 2 25 # test image Bonev, Cazorla Environment in Recognition Tasks 27 / 35 WAF 28
Several localization hypotheses Similiarity functions responses 4 # reference image 3 2 1 H1 H2 H3 H4 H5 H6 H7 H8 H9 H1 H H H13 H14 H15 H16 H17 H18 H19 H2 5 1 15 2 25 # test image Bonev, Cazorla Environment in Recognition Tasks 28 / 35 WAF 28
4 3 2 1 Similiarity functions responses 5 1 15 2 25 # test image H1 H2 H3 H4 H5 H6 H7 H8 H9 H1 H H H13 H14 H15 H16 H17 H18 H19 H2 Several localization hypotheses # reference image Monte Carlo Localization, given: A motion model A likelihood function for a given position Bonev, Cazorla Environment in Recognition Tasks 29 / 35 WAF 28
Outline 1 Localization and vision Initial approach approach 2 3 4 Bonev, Cazorla Environment in Recognition Tasks 3 / 35 WAF 28
MCL algorithm for disambiguation likelihood likelihood likelihood likelihood 1.5 Particle filter, iterations 1,5,8, 1 2 3 4 1.5 1 2 3 4 1.5 1 2 3 4 1.5 1 2 3 4 particle position (# reference image) Bonev, Cazorla Environment in Recognition Tasks 31 / 35 WAF 28
Single classifier Single similiarity function response H 4 likelihood 1.5 Particle filter, iteration 56 5 1 15 2 # reference image # reference image 3 2 1 5 1 15 2 25 # test image Bonev, Cazorla Environment in Recognition Tasks 32 / 35 WAF 28
Outline 1 Localization and vision Initial approach approach 2 3 4 Bonev, Cazorla Environment in Recognition Tasks 33 / 35 WAF 28
Visual localization approach Scalability Unsupervised IT-based partitioning Fast image recognition,.1sec Suitable for corridor-like scenarios Future work Generalize to 2D scenarios Bonev, Cazorla Environment in Recognition Tasks 34 / 35 WAF 28
Large Scale Environment in Mobile Robotics Recognition Tasks Boyan Bonev, Miguel Cazorla {boyan,miguel}@dccia.ua.es Robot Vision Group Department of Computer Science and Artificial Intelligence University of Alicante April 27th, 27 Bonev, Cazorla Environment in Recognition Tasks 35 / 35 WAF 28