OBJECT DETECTION AND RECOGNITION IN DIGITAL IMAGES THEORY AND PRACTICE Bogustaw Cyganek AGH University of Science and Technology, Poland WILEY A John Wiley &. Sons, Ltd., Publication
Contents Preface Acknowledgements Notations and Abbreviations xiii xv xvii 1 Introduction 1 1.1 A Sample of Computer Vision 3 1.2 Overview of Book Contents 6 References 8 2 Tensor Methods in Computer Vision 9 2.1 Abstract 9 2.2 Tensor - A Mathematical Object 10 2.2.1 Main Properties of Linear Spaces 10 2.2.2 Concept of a Tensor 11 2.3 Tensor - A Data Object 13 2.4 Basic Properties of Tensors 15 2.4.1 Notation of Tensor Indices and Components 16 2.4.2 Tensor Products 18 2.5 Tensor Distance Measures 20 2.5.1 Overview of Tensor Distances 22 2.5.1.1 Computation of Matrix Exponent and Logarithm Functions 24 2.5.2 Euclidean Image Distance and Standardizing Transform 29 2.6 Filtering of Tensor Fields 33 2.6.1 Order Statistic Filtering of Tensor Data 33 2.6.2 Anisotropic Diffusion Filtering 36 2.6.3 IMPLEMENTATION of Diffusion Processes 40 2.7 Looking into Images with the Structural Tensor 44 2.7. / Structural Tensor in Two-Dimensional Image Space 47 2.7.2 Spatio-Temporal Structural Tensor 50 2.7.3 Multichannel and Scale-Space Structural Tensor 52 2.7.4 Extended Structural Tensor 54 2.7.4.1 IMPLEMENTATION of the Linear and Nonlinear Structural Tensor 57
Vlll Contents 2.8 Object Representation with Tensor of Inertia and Moments 62 2.8.1 IMPLEMENTATION of Moments and their Invariants 65 2.9 Eigendecomposition and Representation of Tensors 68 2.10 Tensor Invariants 72 2.11 Geometry of Multiple Views: The Multifocal Tensor 72 2.12 Multilinear Tensor Methods 75 2.12.1 Basic Concepts of Multilinear Algebra 78 2.12.1.1 Tensor Flattening 78 2.12.1.2 IMPLEMENTATION Tensor Representation 84 2.12.1.3 The k-mode Product of a Tensor and a Matrix 95 2.12.1.4 Ranks of a Tensor 100 2.12.1.5 IMPLEMENTATION of Basic Operations on Tensors 101 2.12.2 Higher-Order Singular Value Decomposition (HOSVD) 112 2.12.3 Computation of the HOSVD 114 2.12.3.1 Implementation of the HOSVD Decomposition 119 2.12.4 HOSVD Induced Bases 121 2.12.5 Tensor Best Rank-1 Approximation 123 2.12.6 Rank-1 Decomposition of Tensors 126 2.12.7 Best Rank-(Rj, R2,..., Rp) Approximation 131 2.12.8 Computation of the Best Rank-(R], R2,..., Rp) Approximations 134 2.12.8.1 IMPLEMENTATION - Rank Tensor Decompositions 137 2.12.8.2 CASE STUDY - Data Dimensionality Reduction 145 2.12.9 Subspace Data Representation 149 2.12.10 Nonnegative Matrix Factorization 151 2.12.11 Computation of the Nonnegative Matrix Factorization 155 2.12.12 Image Representation with NMF 160 2.12.13 Implementation of the Nonnegative Matrix Factorization 162 2.12.14 Nonnegative Tensor Factorization 169 2.12.15 Multilinear Methods of Object Recognition 173 2.13 Closure 179 2.13.1 Chapter Summary 179 2.13.2 Further Reading 180 2.13.3 Problems and Exercises 181 References 182 3 Classification Methods and Algorithms 189 3.1 Abstract 189 3.2 Classification Framework 190 3.2.1 IMPLEMENTATION Computer Representation of Features 191 3.3 Subspace Methods for Object Recognition 194 3.3.1 Principal Component Analysis 195 3.3.1.1 Computation ofthe PC A 199 3.3.1.2 PCA for Multi-Channel Image Processing 210 3.3.1.3 PCA for Background Subtraction 214 3.3.2 Subspace Pattern Classification 215
Contents ix 3.4 Statistical Formulation of the Object Recognition 222 3.4.1 Parametric and Nonparametric Methods 222 3.4.2 Probabilistic Framework 222 3.4.3 Bay es Decision Rule 223 3.4.4 Maximum a posteriori Classification Scheme 224 3.4.5 Binary Classification Problem 226 3.5 Parametric Methods - Mixture of Gaussians 227 3.6 The Kaiman Filter 233 3.7 Nonparametric Methods 236 3.7.1 Histogram Based Techniques 236 3.7.2 Comparing Histograms 239 3.7.3 IMPLEMENTATION - Multidimensional Histograms 243 3.7.4 Parzen Method 246 3.7.4.1 Kernel Based Methods 248 3.7.4.2 Nearest-Neighbor Method 250 3.8 The Mean Shift Method 251 3.8.1 Introduction to the Mean Shift 251 3.8.2 Continuously Adaptive Mean Shift Method (CamShiff) 257 3.8.3 Algorithmic Aspects ofthe Mean Shift Tracking 259 3.8.3.1 Tracking of Multiple Features 259 3.8.3.2 Tracking of Multiple Objects 260 3.8.3.3 Fuzzy Approach to the CamShift 261 3.8.3.4 Discrimination with Background Information 262 3.8.3.5 Adaptive Update ofthe Classifiers 263 3.8.4 IMPLEMENTATION of the CamShift Method 264 3.9 Neural Networks 267 3.9.1 Probabilistic Neural Network 267 3.9.2 IMPLEMENTATION - Probabilistic Neural Network 270 3.9.3 Hamming Neural Network IIA 3.9.4 IMPLEMENTATION of the Hamming Neural Network 278 3.9.5 Morphological Neural Network 282 3.9.5.1 IMPLEMENTATION of the Morphological Neural Network 285 3.10 Kernels in Vision Pattern Recognition 291 3.10.1 Kernel Functions 296 3.10.2 IMPLEMENTATION-Kernels 301 3.11 Data Clustering 306 3.11.1 The k-means Algorithm 308 3.11.2 Fuzzy c-means 311 3.11.3 Kernel Fuzzy c-means 313 3.11.4 Measures of Cluster Quality 315 3.11.5 IMPLEMENTATION Issues 317 3.12 Support Vector Domain Description 327 3.12.1 Implementation of Support Vector Machines 333 3.12.2 Architecture of the Ensemble of One-Class Classifiers 334 3.13 Appendix - MATLAB and other Packages for Pattern Classification 336 3.14 Closure 336
X Contents 3.14.1 Chapter Summary 336 3.14.2 Further Reading 337 Problems and Exercises 338 References 339 4 Object Detection and Tracking 346 4.1 Introduction 346 4.2 Direct Pixel Classification 346 4.2.1 Ground-Truth Data Collection 347 4.2.2 CASE STUDY - Human Skin Detection 348 4.2.3 CASE STUDY - Pixel Based Road Signs Detection 352 4.2.3.1 Fuzzy Approach 353 4.2.3.2 SVM Based Approach 353 4.2.4 Pixel Based Image Segmentation with Ensemble of Classifiers 361 4.3 Detection of Basic Shapes 364 4.3.1 Detection of Line Segments 366 4.3.2 Up Write Detection ofconvex Shapes 367 4.4 Figure Detection 370 4.4.1 Detection of Regular Shapes from Characteristic Points 371 4.4.2 Clustering of the Salient Points 375 4.4.3 Adaptive Window Growing Method 376 4.4.4 Figure Verification, 378 4.4.5 CASE STUDY - Road Signs Detection System 380 4.5 CASE STUDY - Road Signs Tracking and Recognition 385 4.6 CASE STUDY - Framework for Object Tracking 389 4.7 Pedestrian Detection 395 4.8 Closure 402 4.8.1 Chapter Summary 402 4.8.2 Further Reading 402 Problems and Exercises 403 References 403 5 Object Recognition 408 5.1 Abstract 408 5.2 Recognition from Tensor Phase Histograms and Morphological Scale Space 409 5.2.7 Computation of the Tensor Phase Histograms in Morphological Scale 411 5.2.2 Matching of the Tensor Phase Histograms 413 5.2.3 CASE STUDY- Object Recognition with Tensor Phase Histograms in Morphological Scale Space 415 5.3 Invariant Based Recognition 420 5.3.1 CASE STUDY - Pictogram Recognition with Affine Moment Invariants 421 5.4 Template Based Recognition 424 5.4.1 Template Matching for Road Signs Recognition 425 5.4.2 Special Distances for Template Matching 428 5.4.3 Recognition with the Log-Polar and Scale-Spaces 429
Contents XI 5.5 Recognition from Deformable Models 436 5.6 Ensembles of Classifiers 438 5.7 CASE STUDY - Ensemble of Classifiers for Road Sign Recognition from Deformed Prototypes 440 5.7. / Architecture of the Road Signs Recognition System 442 5.7.2 Module for Recognition of Warning Signs 446 5.7.3 The Arbitration Unit 452 5.8 Recognition Based on Tensor Decompositions 453 5.8.1 Pattern Recognition in SubSpaces Spanned by the HOSVD Decomposition of Pattern Tensors 453 5.8.2 CASE STUDY- Road Sign Recognition System Based on Decomposition of Tensors with Deformable Pattern Prototypes 455 5.8.3 CASE STUDY - Handwritten Digit Recognition with Tensor Decomposition Method 462 5.8.4 IMPLEMENTATION of the Tensor Subspace Classifiers 465 5.9 Eye Recognition for Driver's State Monitoring 470 5.10 Object Category Recognition 476 5.10.1 Part-Based Object Recognition Al 6 5.10.2 Recognition with Bag-of-Visual-Words All 5.11 Closure 480 5.11.1 Chapter Summary 480 5.7/.2 Further Reading 481 Problems and Exercises 482 Reference 483 A Appendix 487 A.l Abstract 487 A.2 Morphological Scale-Space 487 A.3 Morphological Tensor Operators 490 A.4 Geometry of Quadratic Forms 491 A.5 Testing Classifiers 492 A. 5.1 Implementation of the Confusion Matrix and Testing Object Detection in Images 496 A.6 Code Acceleration with OpenMP 499 A.6.1 Recipes for Object-Oriented Code Design with OpenMP 501 A.6.2 Hints on Using and Code Porting to OpenMP 507 A.6.3 Performance Analysis 511 A.7 Useful MATLAB Functions for Matrix and Tensor Processing 512 A.8 Short Guide to the Attached Software 513 A.9 Closure 516 A.9.1 Chapter Summary 516 A.9.2 Further Reading 519 Problems and Exercises 520 References 520 Index 523