Information Theory in Computer Vision and Pattern Recognition

Transcription

1 Francisco Escolano Pablo Suau Boyan Bonev Information Theory in Computer Vision and Pattern Recognition Foreword by Alan Yuille ~ Springer

2 Contents 1 Introduction Measures, Principles, Theories, and More Detailed Organization of the Book The ITinCVPR Roadmap 10 2 Interest Points, Edges, and Contour Grouping Introduction Entropy and Interest Points Kadir and Brady Scale Saliency Detector Point Filtering by Entropy Analysis Through Scale Space Chernoff Information and Optimal Filtering Bayesian Filtering of the Scale Saliency Feature Extractor: The Algorithm Information Theory as Evaluation Tool: The Statistical Edge Detection Case Statistical Edge Detection Edge Localization Finding Contours Among Clutter Problem Statement A* Road Tracking A* Convergence Proof Junction Detection and Grouping Junction Detection Connecting and Filtering Junctions Problems Key References Contour and Region-Based Image Segmentation Introduction Discriminative Segmentation with Jensen-Shannon Divergence 44 XIII

3 XIV Contents The Aetive Polygons Funetional Jensen-Shannon Divergenee and the Speed Funetion MDL in Contour-Based Segmentation B-Spline Parameterization of Contours MDL for B-Spline Parameterization MDL Contour-based Segmentation Model Order Selection in Region-Based Segmentation Jump-Diffusion for Optimal Segmentation Speeding-up the Jump-Diffusion Process K-adventurers Algorithm Model-Based Segmentation Exploiting The Maximum Entropy Prineiple Maximum Entropy and Markov Random Fields Effieient Learning with Belief Propagation Integrating Segmentation, Detection and Recognition Image Parsing The Data-Driven Generative Model The Power of Diseriminative Proeesses The Usefulness of Combining Generative and Diseriminative 99 Problems Key Referenees Registration, Matching, and Recognition Introduetion Image Alignment and Mutual Information Alignment and Image Statistics Entropy Estimation with Parzen's Windows The EMMA Algorithm Solving the Histogram-Binning Problem Alternative Metries for Image Alignment Normalizing Mutual Information Conditional Entropies Extension to the Multimodal Case Affine Alignment of Multiple Images The Renyi Entropy Renyi's Entropy and Entropie Spanning Graphs The Jensen-Renyi Divergenee and Its Applieations Other Measures Related to Renyi Entropy Experimental Results Deformable Matching with Jensen Divergenee and Fisher Information The Distributional Shape Model Multiple Registration and Jensen-Shannon Divergenee 136

4 Contents XV Information Geometry and Fisher-Rao Information Dynamics of the Fisher Information Metric Structural Learning with MDL The Usefulness of Shock Trees A Generative Tree Model Based on Mixtures Learning the Mixture Tree Edit-Distance and MDL 151 Problems Key References Image and Pattern Clustering Introduction Gaussian Mixtures and Model Selection Gaussian Mixtures Methods Defining Gaussian Mixtures EM Algorithm and Its Drawbacks Model Order Selection EBEM Algorithm: Exploiting Entropie Graphs The Gaussianity Criterion and Entropy Estimation Shannon Entropy from Renyi Entropy Estimation Minimum Description Length for EBEM Kernel-Splitting Equations Experiments Information Bottleneck and Rate Distortion Theory Rate Distortion Theory Based Clustering The Information Bottleneck Principle Agglomerative IB Clustering Jensen-Shannon Divergence and Bayesian Classification Error The AlB Algorithm Unsupervised Clustering of Images Robust Information Clustering IT-Based Mean Shift The Mean Shift Algorithm Mean Shift Stop Criterion and Examples Renyi Quadratic and Cross Entropy from Parzen Windows Mean Shift from an IT Perspective Unsupervised Classification and Clustering Ensembles Representation of Multiple Partitions Consensus Functions 199 Problems Key References 209

5 XVI Contents 6 Feature Selection and Transformation Introduction Wrapper and the Cross Validation Criterion Wrapper for Classifier Evaluation Cross Validation Image Classification Example Experiments Filters Based on Mutual Information Criteria for Filter Feature Selection Mutual Information for Feature Sclection Individual Features Evaluation, Dependence and Redundancy The min-redundancy Max-Relevance Criterion The Max-Dependency Criterion Limitations of the Greedy Search Greedy Backward Search Markov Blankets for Feature Selection Applications and Experiments Minimax Feature Selection far Generative Models Filters and the Maximum Entropy Principle Filter Pursuit through Minimax Entropy From PCA to gpca PCA, FastICA, and lnfomax Minimax Mutual Information ICA Generalized PCA (gpca) and Effective Dimension 254 Problems Key References Classifier Design Introduction Model-Based Deci::;ioll Trees Reviewing Information Gain The Global Criterion Rare Classes with the Greedy Approach Rare Classes with Global Optimization Shape Quantization and Multiple Randomized Trees Simple Tags and Their Arrangements Algorithm for the Simple Tree More Complex Tags and Arrangements Randomizing and Multiple Trees Random Forests The Basic Concept The Generalization Error of the RF Ensemble Out-of-the-Bag Estimates of the Error Bound Variable Selection: Forest RI vs. Forest-RC 295

6 Contents XVII 7.5 Infomax and Jensen-Shannon Boosting The Infomax Boosting Algorithm Jensen-Shannon Boosting Maximum Entropy Principle for Classification Improved Iterative Scaling Maximum Entropy and Information Projection Bregman Divergences and Classification Concept and Properties Bregman Balls and Core Vector Machines Unifying Classification: Bregman Divergences and Surrogates 331 Problems Key References 341 References 343 Index 353 Color Plates 357