Information Theory in Computer Vision and Pattern Recognition
|
|
- Cecil Lamb
- 6 years ago
- Views:
Transcription
1 Information Theory in Computer Vision and Pattern Recognition
2 Francisco Escolano Pablo Suau Boyán Bonev Information Theory in Computer Vision and Pattern Recognition Foreword by Alan Yuille 123
3 Francisco Escolano Universidad Alicante Depto. Ciencia de la Computación e Inteligencia Artificial Campus de San Vicente, s/n Alicante Spain sco@dccia.ua.es Boyán Bonev Universidad Alicante Depto. Ciencia de la Computación e Inteligencia Artificial Campus de San Vicente, s/n Alicante Spain boyan@dccia.ua.es Pablo Suau Universidad Alicante Depto. Ciencia de la Computación e Inteligencia Artificial Campus de San Vicente, s/n Alicante Spain pablo@dccia.ua.es ISBN e-isbn DOI / Springer Dordrecht Heidelberg London New York British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library Library of Congress Control Number: c Springer Verlag London Limited 2009 Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms of licenses issued by the Copyright Licensing Agency. Enquiries concerning reproduction outside those terms should be sent to the publishers. The use of registered names, trademarks, etc., in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant laws and regulations and therefore free for general use. The publisher makes no representation, express or implied, with regard to the accuracy of the information contained in this book and cannot accept any legal responsibility or liability for any errors or omissions that may be made. Cover design: SPi Publisher Services Printed on acid-free paper Springer is part of Springer Science+Business Media (
4 To my three joys: Irene, Ana, and Mamen. Francisco To my parents, grandparents, and brother. To Beatriz. Pablo To Niya, Elina, and Maria. Boyan
5 Foreword Computer vision and pattern recognition are extremely important research fields with an enormous range of applications. They are also extremely difficult. This may seem paradoxical since humans can easily interpret images and detect spatial patterns. But this apparent ease is misleading because neuroscience shows that humans devote a large part of their brain, possibly up to 50% of the cortex, to processing images and interpreting them. The difficulties of these problems have been appreciated over the last 30 years as researchers have struggled to develop computer algorithms for performing vision and pattern recognition tasks. Although these problems are not yet completely solved, it is becoming clear that the final theory will depend heavily on probabilistic techniques and the use of concepts from information theory. The connections between information theory and computer vision have long been appreciated. Vision can be considered to be a decoding problem where the encoding of the information is performed by the physics of the world by light rays striking objects and being reflected to cameras or eyes. Ideal observer theories were pioneered by scientists such as Horace Barlow to compute the amount of information available in the visual stimuli, and to see how efficient humans are at exploiting it. But despite the application of information theory to specific visual tasks, there has been no attempt to bring all this work together into a clear conceptual framework. This book fills the gap by describing how probability and information theory can be used to address computer vision and pattern recognition problems. The authors have developed information theory tools side by side with vision and pattern recognition tasks. They have characterized these tools into four classes: (i) measures, (ii) principles, (iii) theories, and (iv) algorithms. The book is organized into chapters addressing computer vision and pattern recognition tasks at increasing levels of complexity. The authors have devoted chapters to feature detection and spatial grouping, image segmentation, matching, clustering, feature selection, and classifier design. As the authors address these topics, they gradually introduce techniques from information theory. These include (1) information theoretic measures, such as entropy VII
6 VIII Foreword and Chernoff information, to evaluate image features; (2) mutual information as a criteria for matching problems (Viola and Wells 1997); (3) minimal description length ideas (Risannen 1978) and their application to image segmentation (Zhu and Yuille 1996); (4) independent component analysis (Bell and Sejnowski 1995) and its use for feature extraction; (5) the use of rate distortion theory for clustering algorithms; (6) the method of types (Cover and Thomas 1991) and its application to analyze the convergence rates of vision algorithms (Coughlan and Yuille 2002); and (7) how entropy and infomax principles (Linsker 1988) can be used for classifier design. In addition; the book covers alternative information theory measures, such as Rényi alphaentropy and Jensen Shannon divergence, and advanced topics; such as data driven Markov Chain Monte Carlo (Tu and Zhu 2002) and information geometry (Amari 1985). The book describes these theories clearly, giving many illustrations and specifying the code by flow -charts. Overall, the book is a very worthwhile addition to the computer vision and pattern recognition literature. The authors have given an advanced introduction to techniques from probability and information theory and their application to vision and pattern recognition tasks. More importantly, they have described a novel perspective that will be of growing importance over time. As computer vision and pattern recognition develop, the details of these theories will change, but the underlying concepts will remain the same. UCLA, Department of Statistics and Psychology Los Angeles, CA March 2009 Alan Yuille
7 Preface Looking through the glasses of Information Theory (IT) has proved to be effective both for formulating and designing algorithmic solutions to many problems in computer vision and pattern recognition (CVPR): image matching, clustering and segmentation, salient point detection, feature selection and dimensionality reduction, projection pursuit, optimal classifier design, and many others. Nowadays, researchers are widely bringing IT elements to the CVPR arena. Among these elements, there are measures (entropy, mutual information, Kullback Leibler divergence, Jensen Shannon divergence...), principles (maximum entropy, minimax entropy, minimum description length...) and theories (rate distortion theory, coding, the method of types...). This book introduces and explores the latter elements, together with the one of entropy estimation, through an incremental complexity approach. Simultaneously, the main CVPR problems are formulated and the most representative algorithms, considering authors preferences for sketching the IT CVPR field, are presented. Interesting connections between IT elements when applied to different problems are highlighted, seeking for a basic/skeletal research roadmap. This roadmap is far from being comprehensive at present due to time and space constraints, and also due to the current state of development of the approach. The result is a novel tool, unique in its conception, both for CVPR and IT researchers, which is intended to contribute, as much as possible, to a cross-fertilization of both areas. The motivation and origin of this manuscript is our awareness of the existence of many sparse sources of IT-based solutions to CVPR problems, and the lack of a systematic text that focuses on the important question: How useful is IT for CVPR? At the same time, we needed a research language, common to all the members of the Robot Vision Group. Energy minimization, graph theory, and Bayesian inference, among others, were adequate methodological tools during our daily research. Consequently, these tools were key to design and build a solid background for our Ph.D. students. Soon we realized that IT was a unifying component that flowed naturally among our rationales for tackling CVPR problems. Thus, some of us enrolled in the task of writing IX
8 X Preface a text in which we could advance as much as possible in the fundamental links between CVPR and IT. Readers (starters and senior researchers) will judge to what extent we have both answered the above fundamental question and reached our objectives. Although the text is addressed to CVPR IT researchers and students, it is also open to an interdisciplinary audience. One of the most interesting examples is the computational vision community, which includes people interested both in biological vision and psychophysics. Other examples are the roboticians and the people interested in developing wearable solutions for the visually impaired (which is the subject of our active work in the research group). Under its basic conception, this text may be used for an IT-based one semester course of CVPR. Only some rudiments of algebra and probability are necessary. IT items will be introduced as the text flows from one computer vision or pattern recognition problem to another. We have deliberately avoided a succession of theorem proof pairs for the sake of a smooth presentation. Proofs, when needed, are embedded in the text, and they are usually excellent pretexts for presenting or highlighting interesting properties of IT elements. Numerical examples with toy settings of the problems are often included for a better understanding of the IT-based solution. When formal elements of other branches of mathematics like field theory, optimization, and so on, are needed, we have briefly presented them and referred to excellent books fully dedicated to their description. Problems, questions and exercises are also proposed at the end of each chapter. The purpose of the problems section is not only to consolidate what is learnt, but also to go one step forward by testing the ability of generalizing the concepts exposed in each chapter. Such section is preceded by a brief literature review that outlines the key papers for the CVPR topic, which is the subject of the chapter. These papers references, together with sketched solutions to the problems, will be progressively accessible in the Web site We have started the book with a brief introduction (Chapter 1) regarding the four axes of IT CVPR interaction (measures, principles, theories, and entropy estimators). We have also presented here the skeletal research roadmap (the ITinCVPR tube). Then we walk along six chapters, each one tackling a different problem under the IT perspective. Chapter 2 is devoted to interest points, edge detection, and grouping; interest points allow us to introduce the concept of entropy and its linking with Chernoff information, Sanov s theorem, phase transitions and the method of types. Chapter 3 covers contour and region-based image segmentation mainly from the perspective of model order selection through the minimum description length (MDL) principle, although the Jensen Shannon measure and the Jaynes principle of maximum entropy are also introduced; the question of learning a segmentation model is tackled through links with maximum entropy and belief propagation; and the unification of generative and discriminative processes for segmentation and
9 Preface XI recognition is explored through information divergence measures. Chapter 4 reviews registration, matching, and recognition by considering the following image registration through minimization of mutual information and related measures; alternative derivations of Jensen Shannon divergence yield deformable matching; shape comparison is encompassed through Fisher information; and structural matching and learning are driven by MDL. Chapter 5 is devoted to image and pattern clustering and is mainly rooted in three IT approaches to clustering: Gaussian mixtures (incremental method for adequate order selection), information bottleneck (agglomerative and robust with model order selection) and mean-shift; IT is also present in initial proposals for ensembles clustering (consensus finding). Chapter 6 reviews the main approaches to feature selection and transformation: simple wrappers and filters exploiting IT for bypassing the curse of dimensionality; minimax entropy principle for learning patterns using a generative approach; and ICA/gPCA methods based on IT (ICA and neg-entropy, info-max and minimax ICA, generalized PCA and effective dimension). Finally, Chapter 7, Classifier Design, analyzes the main IT strategies for building classifiers. This obviously includes decision trees, but also multiple trees and random forests, and how to improve boosting algorithms by means of IT-based criteria. This final chapter ends with an information projection analysis of maximum entropy classifiers and a careful exploration of the links between Bregman divergences and classifiers. We acknowledge the contribution of many people to this book. In first place, we thank many scientists for their guide and support, and for their important contributions to the field. Researchers from different universities and institutions such as Alan Yuille, Hamid Krim, Chen Ping-Feng, Gozde Unal, Ajit Rajwadee, Anand Rangarajan, Edwin Hancock, Richard Nock, Shun-ichi Amari, and Mario Figueiredo, among many others, contributed with their advices, deep knowledge and highly qualified expertise. We also thank all the colleagues of the Robot Vision Group of the University of Alicante, especially Antonio Peñalver, Juan Manuel Sáez, and Miguel Cazorla, who contributed with figures, algorithms, and important results from their research. Finally, we thank the editorial board staff: Catherine Brett for his initial encouragement and support, and Simon Rees and Wayne Wheeler for their guidance and patience. University of Alicante, Spain Francisco Escolano Pablo Suau Boyan Bonev
10 Contents 1 Introduction Measures,Principles,Theories,andMore Detailed Organization of the Book TheITinCVPRRoadmap Interest Points, Edges, and Contour Grouping Introduction EntropyandInterestPoints Kadir and Brady Scale Saliency Detector Point Filtering by Entropy Analysis Through ScaleSpace Chernoff Information and Optimal Filtering Bayesian Filtering of the Scale Saliency Feature Extractor: The Algorithm Information Theory as Evaluation Tool: The Statistical Edge Detection Case Statistical Edge Detection Edge Localization FindingContoursAmongClutter Problem Statement A RoadTracking A Convergence Proof Junction Detection and Grouping Junction Detection Connecting and Filtering Junctions Problems Key References Contour and Region-Based Image Segmentation Introduction Discriminative Segmentation with Jensen Shannon Divergence XIII
11 XIV Contents The Active Polygons Functional Jensen Shannon Divergence and the Speed Function MDL in Contour-Based Segmentation B-Spline Parameterization of Contours MDL for B-Spline Parameterization MDL Contour-based Segmentation Model Order Selection in Region-Based Segmentation Jump-Diffusion for Optimal Segmentation Speeding-up the Jump-Diffusion Process K-adventurers Algorithm Model-Based Segmentation Exploiting The Maximum EntropyPrinciple Maximum Entropy and Markov Random Fields Efficient Learning with Belief Propagation Integrating Segmentation, Detection and Recognition Image Parsing The Data-Driven Generative Model The Power of Discriminative Processes The Usefulness of Combining Generative and Discriminative Problems Key References Registration, Matching, and Recognition Introduction Image Alignment and Mutual Information Alignment and Image Statistics Entropy Estimation with Parzen s Windows The EMMA Algorithm Solving the Histogram-Binning Problem Alternative Metrics for Image Alignment Normalizing Mutual Information Conditional Entropies Extension to the Multimodal Case Affine Alignment of Multiple Images The RényiEntropy Rényi s Entropy and Entropic Spanning Graphs The Jensen Rényi Divergence and Its Applications Other Measures Related to RényiEntropy Experimental Results Deformable Matching with Jensen Divergence and Fisher Information The Distributional Shape Model Multiple Registration and Jensen Shannon Divergence
12 Contents XV Information Geometry and Fisher Rao Information Dynamics of the Fisher Information Metric Structural Learning with MDL The Usefulness of Shock Trees A Generative Tree Model Based on Mixtures Learning the Mixture Tree Edit-Distance and MDL Problems Key References Image and Pattern Clustering Introduction Gaussian Mixtures and Model Selection Gaussian Mixtures Methods Defining Gaussian Mixtures EM Algorithm and Its Drawbacks Model Order Selection EBEM Algorithm: Exploiting Entropic Graphs The Gaussianity Criterion and Entropy Estimation Shannon Entropy from Rényi Entropy Estimation Minimum Description Length for EBEM Kernel-Splitting Equations Experiments Information Bottleneck and Rate Distortion Theory Rate Distortion Theory Based Clustering The Information Bottleneck Principle Agglomerative IB Clustering Jensen Shannon Divergence and Bayesian Classification Error The AIB Algorithm Unsupervised Clustering of Images Robust Information Clustering IT-BasedMeanShift The Mean Shift Algorithm Mean Shift Stop Criterion and Examples Rényi Quadratic and Cross Entropy from Parzen Windows Mean Shift from an IT Perspective Unsupervised Classification and Clustering Ensembles Representation of Multiple Partitions Consensus Functions Problems Key References
13 XVI Contents 6 Feature Selection and Transformation Introduction Wrapper and the Cross Validation Criterion Wrapper for Classifier Evaluation Cross Validation Image Classification Example Experiments Filters Based on Mutual Information Criteria for Filter Feature Selection Mutual Information for Feature Selection Individual Features Evaluation, Dependence and Redundancy The min-redundancy Max-Relevance Criterion The Max-Dependency Criterion Limitations of the Greedy Search Greedy Backward Search Markov Blankets for Feature Selection Applications and Experiments Minimax Feature Selection for Generative Models Filters and the Maximum Entropy Principle Filter Pursuit through Minimax Entropy FromPCAtogPCA PCA, FastICA, and Infomax Minimax Mutual Information ICA Generalized PCA (gpca) and Effective Dimension Problems Key References Classifier Design Introduction Model-BasedDecisionTrees Reviewing Information Gain The Global Criterion Rare Classes with the Greedy Approach Rare Classes with Global Optimization Shape Quantization and Multiple Randomized Trees Simple Tags and Their Arrangements Algorithm for the Simple Tree More Complex Tags and Arrangements Randomizing and Multiple Trees RandomForests The Basic Concept The Generalization Error of the RF Ensemble Out-of-the-Bag Estimates of the Error Bound Variable Selection: Forest RI vs. Forest-RC
14 Contents XVII 7.5 Infomax and Jensen Shannon Boosting The Infomax Boosting Algorithm Jensen Shannon Boosting Maximum Entropy Principle for Classification Improved Iterative Scaling Maximum Entropy and Information Projection Bregman Divergences and Classification Concept and Properties Bregman Balls and Core Vector Machines Unifying Classification: Bregman Divergences and Surrogates Problems Key References References Index Color Plates...357
Information Theory in Computer Vision and Pattern Recognition
Francisco Escolano Pablo Suau Boyan Bonev Information Theory in Computer Vision and Pattern Recognition Foreword by Alan Yuille ~ Springer Contents 1 Introduction...............................................
More informationPattern Recognition and Machine Learning
Christopher M. Bishop Pattern Recognition and Machine Learning ÖSpri inger Contents Preface Mathematical notation Contents vii xi xiii 1 Introduction 1 1.1 Example: Polynomial Curve Fitting 4 1.2 Probability
More informationLarge Scale Environment Partitioning in Mobile Robotics Recognition Tasks
Large Scale Environment in Mobile Robotics Recognition Tasks Boyan Bonev, Miguel Cazorla {boyan,miguel}@dccia.ua.es Robot Vision Group Department of Computer Science and Artificial Intelligence University
More informationKarl-Rudolf Koch Introduction to Bayesian Statistics Second Edition
Karl-Rudolf Koch Introduction to Bayesian Statistics Second Edition Karl-Rudolf Koch Introduction to Bayesian Statistics Second, updated and enlarged Edition With 17 Figures Professor Dr.-Ing., Dr.-Ing.
More informationPATTERN CLASSIFICATION
PATTERN CLASSIFICATION Second Edition Richard O. Duda Peter E. Hart David G. Stork A Wiley-lnterscience Publication JOHN WILEY & SONS, INC. New York Chichester Weinheim Brisbane Singapore Toronto CONTENTS
More informationUniversitext. Series Editors:
Universitext Universitext Series Editors: Sheldon Axler San Francisco State University, San Francisco, CA, USA Vincenzo Capasso Università degli Studi di Milano, Milan, Italy Carles Casacuberta Universitat
More informationJOURNAL OF PHYSICAL AGENTS, VOL. 4, NO. 2, MAY Large Scale Environment Partitioning in Mobile Robotics Recognition Tasks
JOURNAL OF PHYSICAL AGENTS, VOL. 4, NO., MAY Large Scale Environment Partitioning in Mobile Robotics Recognition Tasks Boyan Bonev and Miguel Cazorla Abstract In this paper we present a scalable machine
More informationOn Normalized Integral Table Algebras (Fusion Rings)
On Normalized Integral Table Algebras (Fusion Rings) For further volumes: www.springer.com/series/6253 Algebra and Applications Volume 16 Series Editors: Alice Fialowski Eötvös Loránd University, Budapest,
More informationFor other titles in this series, go to Universitext
For other titles in this series, go to www.springer.com/series/223 Universitext Marino Badiale Enrico Serra Semilinear Elliptic Equations for Beginners Existence Results via the Variational Approach Marino
More informationAstronomy with a Budget Telescope
Astronomy with a Budget Telescope Springer-Verlag London Ltd. Patrick Moore and John Watson Astro omy w h a Budget elescope With 100 Figures, 98 in colour, Springer British Library Cataloguing in Publication
More informationOBJECT DETECTION AND RECOGNITION IN DIGITAL IMAGES
OBJECT DETECTION AND RECOGNITION IN DIGITAL IMAGES THEORY AND PRACTICE Bogustaw Cyganek AGH University of Science and Technology, Poland WILEY A John Wiley &. Sons, Ltd., Publication Contents Preface Acknowledgements
More informationGeophysical Interpretation using Integral Equations
Geophysical Interpretation using Integral Equations Geophysical Interpretation using Integral Equations L. ESKOLA Head of the Geophysics Department, Geological Survey of Finland 1~lll SPRINGER-SCIENCE+BUSINESS
More informationDeep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, Spis treści
Deep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, 2017 Spis treści Website Acknowledgments Notation xiii xv xix 1 Introduction 1 1.1 Who Should Read This Book?
More informationTopics in Algebra and Analysis
Radmila Bulajich Manfrino José Antonio Gómez Ortega Rogelio Valdez Delgado Topics in Algebra and Analysis Preparing for the Mathematical Olympiad Radmila Bulajich Manfrino Facultad de Ciencias Universidad
More informationAdvanced Calculus of a Single Variable
Advanced Calculus of a Single Variable Tunc Geveci Advanced Calculus of a Single Variable 123 Tunc Geveci Department of Mathematics and Statistics San Diego State University San Diego, CA, USA ISBN 978-3-319-27806-3
More informationPractical Astronomy. Springer-Verlag London Ltd.
Practical Astronomy Springer-Verlag London Ltd. Other titles in this series The Modern Amateur Astronomer Patrick Moore (Ed.) Telescopes and Techniques: An Introduction to Practical Astronomy C. R. Kitchin
More informationStatistics and Measurement Concepts with OpenStat
Statistics and Measurement Concepts with OpenStat William Miller Statistics and Measurement Concepts with OpenStat William Miller Urbandale, Iowa USA ISBN 978-1-4614-5742-8 ISBN 978-1-4614-5743-5 (ebook)
More informationStatistics for Social and Behavioral Sciences
Statistics for Social and Behavioral Sciences Advisors: S.E. Fienberg W.J. van der Linden For other titles published in this series, go to http://www.springer.com/series/3463 Haruo Yanai Kei Takeuchi
More informationDissipative Ordered Fluids
Dissipative Ordered Fluids Andr é M. Sonnet Epifanio G. Virga Dissipative Ordered Fluids Theories for Liquid Crystals Andr é M. Sonnet Department of Mathematics and Statistics University of Strathclyde
More informationSTOCHASTIC PROCESSES FOR PHYSICISTS. Understanding Noisy Systems
STOCHASTIC PROCESSES FOR PHYSICISTS Understanding Noisy Systems Stochastic processes are an essential part of numerous branches of physics, as well as biology, chemistry, and finance. This textbook provides
More informationFinite Element Analysis for Heat Transfer. Theory and Software
Finite Element Analysis for Heat Transfer Theory and Software Hou-Cheng Huang and Asif S. Usmani Finite Element Analysis for Heat Transfer Theory and Software With 62 Figures Springer-Verlag London Berlin
More informationStatistical Modeling of Visual Patterns: Part I --- From Statistical Physics to Image Models
Statistical Modeling of Visual Patterns: Part I --- From Statistical Physics to Image Models Song-Chun Zhu Departments of Statistics and Computer Science University of California, Los Angeles Natural images
More informationEBEM: An Entropy-based EM Algorithm for Gaussian Mixture Models
EBEM: An Entropy-based EM Algorithm for Gaussian Mixture Models Antonio Peñalver Benavent, Francisco Escolano Ruiz and Juan M. Sáez Martínez Robot Vision Group Alicante University 03690 Alicante, Spain
More informationP.M. Cohn. Basic Algebra. Groups, Rings and Fields. m Springer
Basic Algebra P.M. Cohn Basic Algebra Groups, Rings and Fields m Springer P.M. Cohn, MA, PhD, FRS Department of Mathematics, University College London, Gower Street, London WC1E 6BT, UK British Library
More informationDoubt-Free Uncertainty In Measurement
Doubt-Free Uncertainty In Measurement Colin Ratcliffe Bridget Ratcliffe Doubt-Free Uncertainty In Measurement An Introduction for Engineers and Students Colin Ratcliffe United States Naval Academy Annapolis
More informationAn Introduction to Statistical Modeling of Extreme V (llues
Stuart Coles An Introduction to Statistical Modeling of Extreme V (llues With 77 Illustrations, Springer Stuart Coles Department of Mathematics University of Bristol University Walk Bristol BS81TW UK stuart.coles@bristol.ac.uk
More informationCONTENTS. Preface List of Symbols and Notation
CONTENTS Preface List of Symbols and Notation xi xv 1 Introduction and Review 1 1.1 Deterministic and Stochastic Models 1 1.2 What is a Stochastic Process? 5 1.3 Monte Carlo Simulation 10 1.4 Conditional
More informationApplied Multivariate Statistical Analysis Richard Johnson Dean Wichern Sixth Edition
Applied Multivariate Statistical Analysis Richard Johnson Dean Wichern Sixth Edition Pearson Education Limited Edinburgh Gate Harlow Essex CM20 2JE England and Associated Companies throughout the world
More informationDiscriminant Analysis and Statistical Pattern Recognition
Discriminant Analysis and Statistical Pattern Recognition GEOFFRY J. McLACHLAN The University of Queensland @EEC*ENCE A JOHN WILEY & SONS, INC., PUBLICATION This Page Intentionally Left Blank Discriminant
More informationIndependent Component Analysis. Contents
Contents Preface xvii 1 Introduction 1 1.1 Linear representation of multivariate data 1 1.1.1 The general statistical setting 1 1.1.2 Dimension reduction methods 2 1.1.3 Independence as a guiding principle
More informationAdvanced Courses in Mathematics CRM Barcelona
Advanced Courses in Mathematics CRM Barcelona Centre de Recerca Matemàtica Managing Editor: Carles Casacuberta More information about this series at http://www.springer.com/series/5038 Giovanna Citti Loukas
More informationNonlinear Dynamical Systems in Engineering
Nonlinear Dynamical Systems in Engineering . Vasile Marinca Nicolae Herisanu Nonlinear Dynamical Systems in Engineering Some Approximate Approaches Vasile Marinca Politehnica University of Timisoara Department
More informationSpringerBriefs in Statistics
SpringerBriefs in Statistics For further volumes: http://www.springer.com/series/8921 Jeff Grover Strategic Economic Decision-Making Using Bayesian Belief Networks to Solve Complex Problems Jeff Grover
More informationPractical Statistics for Geographers and Earth Scientists
Practical Statistics for Geographers and Earth Scientists Nigel Walford A John Wiley & Sons, Ltd., Publication Practical Statistics for Geographers and Earth Scientists Practical Statistics for Geographers
More informationFundamentals of Mass Determination
Fundamentals of Mass Determination Michael Borys Roman Schwartz Arthur Reichmuth Roland Nater Fundamentals of Mass Determination 123 Michael Borys Fachlabor 1.41 Physikalisch-Technische Bundesanstalt Bundesallee
More informationSpringerBriefs in Mathematics
SpringerBriefs in Mathematics Series Editors Nicola Bellomo Michele Benzi Palle E.T. Jorgensen Tatsien Li Roderick Melnik Otmar Scherzer Benjamin Steinberg Lothar Reichel Yuri Tschinkel G. George Yin Ping
More informationLecture 15. Probabilistic Models on Graph
Lecture 15. Probabilistic Models on Graph Prof. Alan Yuille Spring 2014 1 Introduction We discuss how to define probabilistic models that use richly structured probability distributions and describe how
More informationStatistical and Inductive Inference by Minimum Message Length
C.S. Wallace Statistical and Inductive Inference by Minimum Message Length With 22 Figures Springer Contents Preface 1. Inductive Inference 1 1.1 Introduction 1 1.2 Inductive Inference 5 1.3 The Demise
More informationLatif M. Jiji. Heat Convection. With 206 Figures and 16 Tables
Heat Convection Latif M. Jiji Heat Convection With 206 Figures and 16 Tables Prof. Latif M. Jiji City University of New York School of Engineering Dept. of Mechanical Engineering Convent Avenue at 138th
More informationSemantics of the Probabilistic Typed Lambda Calculus
Semantics of the Probabilistic Typed Lambda Calculus Dirk Draheim Semantics of the Probabilistic Typed Lambda Calculus Markov Chain Semantics, Termination Behavior, and Denotational Semantics Dirk Draheim
More informationNeural coding Ecological approach to sensory coding: efficient adaptation to the natural environment
Neural coding Ecological approach to sensory coding: efficient adaptation to the natural environment Jean-Pierre Nadal CNRS & EHESS Laboratoire de Physique Statistique (LPS, UMR 8550 CNRS - ENS UPMC Univ.
More informationDensity Propagation for Continuous Temporal Chains Generative and Discriminative Models
$ Technical Report, University of Toronto, CSRG-501, October 2004 Density Propagation for Continuous Temporal Chains Generative and Discriminative Models Cristian Sminchisescu and Allan Jepson Department
More informationUNITEXT La Matematica per il 3+2. Volume 87
UNITEXT La Matematica per il 3+2 Volume 87 More information about this series at http://www.springer.com/series/5418 Sandro Salsa Gianmaria Verzini Partial Differential Equations in Action Complements
More informationThiS is a FM Blank Page
Acid-Base Diagrams ThiS is a FM Blank Page Heike Kahlert Fritz Scholz Acid-Base Diagrams Heike Kahlert Fritz Scholz Institute of Biochemistry University of Greifswald Greifswald Germany English edition
More informationIgor Emri Arkady Voloshin. Statics. Learning from Engineering Examples
Statics Igor Emri Arkady Voloshin Statics Learning from Engineering Examples Igor Emri University of Ljubljana Ljubljana, Slovenia Arkady Voloshin Lehigh University Bethlehem, PA, USA ISBN 978-1-4939-2100-3
More informationin this web service Cambridge University Press
CONTINUUM MECHANICS This is a modern textbook for courses in continuum mechanics. It provides both the theoretical framework and the numerical methods required to model the behavior of continuous materials.
More informationMonte-Carlo Methods and Stochastic Processes
Monte-Carlo Methods and Stochastic Processes From Linear to Non-Linear EMMANUEL GOBET ECOLE POLYTECHNIQUE - UNIVERSITY PARIS-SACLAY CMAP, PALAISEAU CEDEX, FRANCE CRC Press Taylor & Francis Group 6000 Broken
More informationQuantum Biological Information Theory
Quantum Biological Information Theory Ivan B. Djordjevic Quantum Biological Information Theory Ivan B. Djordjevic Department of Electrical and Computer Engineering University of Arizona Tucson, AZ, USA
More informationGatsby Theoretical Neuroscience Lectures: Non-Gaussian statistics and natural images Parts I-II
Gatsby Theoretical Neuroscience Lectures: Non-Gaussian statistics and natural images Parts I-II Gatsby Unit University College London 27 Feb 2017 Outline Part I: Theory of ICA Definition and difference
More informationMETHODS FOR PROTEIN ANALYSIS
METHODS FOR PROTEIN ANALYSIS Robert A. Copeland, PhD The DuPont Merck Pharmaceutical Company Experimental Station P.O. Box 80400 Wilmington, DE 19880-0400 METHODS FOR PROTEIN ANALYSIS A Practical Guide
More informationData Analysis Using the Method of Least Squares
Data Analysis Using the Method of Least Squares J. Wolberg Data Analysis Using the Method of Least Squares Extracting the Most Information from Experiments With Figures and Tables 123 John Wolberg Technion-Israel
More informationShijun Liao. Homotopy Analysis Method in Nonlinear Differential Equations
Shijun Liao Homotopy Analysis Method in Nonlinear Differential Equations Shijun Liao Homotopy Analysis Method in Nonlinear Differential Equations With 127 figures Author Shijun Liao Shanghai Jiao Tong
More informationSpringerBriefs in Mathematics
SpringerBriefs in Mathematics For further volumes: http://www.springer.com/series/10030 George A. Anastassiou Advances on Fractional Inequalities 123 George A. Anastassiou Department of Mathematical Sciences
More informationA Student s Guide to Waves
A Student s Guide to Waves Waves are an important topic in the fields of mechanics, electromagnetism, and quantum theory, but many students struggle with the mathematical aspects. Written to complement
More informationUndergraduate Texts in Mathematics. Editors J. H. Ewing F. W. Gehring P. R. Halmos
Undergraduate Texts in Mathematics Editors J. H. Ewing F. W. Gehring P. R. Halmos Springer Books on Elemeritary Mathematics by Serge Lang MATH! Encounters with High School Students 1985, ISBN 96129-1 The
More informationGIS AND TERRITORIAL INTELLIGENCE. Using Microdata. Jean Dubé and Diègo Legros
GIS AND TERRITORIAL INTELLIGENCE Spatial Econometrics Using Microdata Jean Dubé and Diègo Legros Spatial Econometrics Using Microdata To the memory of Gilles Dubé. For Mélanie, Karine, Philippe, Vincent
More informationENTROPY-BASED PARAMETER ESTIMATION IN HYDROLOGY
ENTROPY-BASED PARAMETER ESTIMATION IN HYDROLOGY Water Science and Technology Library VOLUME 30 Editor-in-Chief V. P. Singh, Louisiana State University, Baton Rouge, U.S.A Editorial Advisory Board M. Anderson,
More informationin this web service Cambridge University Press
BROWNIAN RATCHETS Illustrating the development of Brownian ratchets, from their foundations, to their role in the description of life at the molecular scale and in the design of artificial nano-machinery,
More informationFollow links Class Use and other Permissions. For more information, send to:
COPYRIGHT NOTICE: Stephen L. Campbell & Richard Haberman: Introduction to Differential Equations with Dynamical Systems is published by Princeton University Press and copyrighted, 2008, by Princeton University
More informationCS534 Machine Learning - Spring Final Exam
CS534 Machine Learning - Spring 2013 Final Exam Name: You have 110 minutes. There are 6 questions (8 pages including cover page). If you get stuck on one question, move on to others and come back to the
More informationFeature Extraction and Image Processing
Feature Extraction and Image Processing Second edition Mark S. Nixon Alberto S. Aguado :*авш JBK IIP AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN DIEGO SAN FRANCISCO SINGAPORE SYDNEY TOKYO
More informationMachine Learning Lecture 3
Announcements Machine Learning Lecture 3 Eam dates We re in the process of fiing the first eam date Probability Density Estimation II 9.0.207 Eercises The first eercise sheet is available on L2P now First
More informationInformation Dynamics Foundations and Applications
Gustavo Deco Bernd Schürmann Information Dynamics Foundations and Applications With 89 Illustrations Springer PREFACE vii CHAPTER 1 Introduction 1 CHAPTER 2 Dynamical Systems: An Overview 7 2.1 Deterministic
More informationRecent Advances in Bayesian Inference Techniques
Recent Advances in Bayesian Inference Techniques Christopher M. Bishop Microsoft Research, Cambridge, U.K. research.microsoft.com/~cmbishop SIAM Conference on Data Mining, April 2004 Abstract Bayesian
More informationPART I INTRODUCTION The meaning of probability Basic definitions for frequentist statistics and Bayesian inference Bayesian inference Combinatorics
Table of Preface page xi PART I INTRODUCTION 1 1 The meaning of probability 3 1.1 Classical definition of probability 3 1.2 Statistical definition of probability 9 1.3 Bayesian understanding of probability
More informationExperimental Techniques in Nuclear and Particle Physics
Experimental Techniques in Nuclear and Particle Physics Stefaan Tavernier Experimental Techniques in Nuclear and Particle Physics 123 Prof. Stefaan Tavernier Vrije Universiteit Brussel Fak. Wetenschappen
More informationDynamics and Control of Lorentz-Augmented Spacecraft Relative Motion
Dynamics and Control of Lorentz-Augmented Spacecraft Relative Motion Ye Yan Xu Huang Yueneng Yang Dynamics and Control of Lorentz-Augmented Spacecraft Relative Motion 123 Ye Yan College of Aerospace Science
More informationStructure and Properties of Oriented Polymers
Structure and Properties of Oriented Polymers Structure and Properties of Oriented Polymers Edited by 1. M. Ward IRC in Polymer Science and Technology Universities of Leeds, Bradford and Durham UK I uni
More informationHuman Pose Tracking I: Basics. David Fleet University of Toronto
Human Pose Tracking I: Basics David Fleet University of Toronto CIFAR Summer School, 2009 Looking at People Challenges: Complex pose / motion People have many degrees of freedom, comprising an articulated
More informationCIFAR Lectures: Non-Gaussian statistics and natural images
CIFAR Lectures: Non-Gaussian statistics and natural images Dept of Computer Science University of Helsinki, Finland Outline Part I: Theory of ICA Definition and difference to PCA Importance of non-gaussianity
More informationCambridge IGCSE and O Level Additional Mathematics Coursebook
Cambridge IGCSE and O Level Additional Mathematics Coursebook Second edition University Printing House, Cambridge CB2 8BS, United Kingdom One Liberty Plaza, 20th Floor, New York, NY 10006, USA 477 Williamstown
More informationMachine Learning A Bayesian and Optimization Perspective
Machine Learning A Bayesian and Optimization Perspective Sergios Theodoridis AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN DIEGO SAN FRANCISCO SINGAPORE SYDNEY TOKYO Academic Press is an
More informationConvergence Rates of Algorithms for Visual Search: Detecting Visual Contours
Convergence Rates of Algorithms for Visual Search: Detecting Visual Contours A.L. Yuille Smith-Kettlewell Inst. San Francisco, CA 94115 James M. Coughlan Smith-Kettlewell Inst. San Francisco, CA 94115
More informationTile-Based Geospatial Information Systems
Tile-Based Geospatial Information Systems John T. Sample Elias Ioup Tile-Based Geospatial Information Systems Principles and Practices 123 John T. Sample Naval Research Laboratory 1005 Balch Blvd. Stennis
More informationControlled Markov Processes and Viscosity Solutions
Controlled Markov Processes and Viscosity Solutions Wendell H. Fleming, H. Mete Soner Controlled Markov Processes and Viscosity Solutions Second Edition Wendell H. Fleming H.M. Soner Div. Applied Mathematics
More informationSparse Forward-Backward for Fast Training of Conditional Random Fields
Sparse Forward-Backward for Fast Training of Conditional Random Fields Charles Sutton, Chris Pal and Andrew McCallum University of Massachusetts Amherst Dept. Computer Science Amherst, MA 01003 {casutton,
More informationTwo -Dimensional Digital Signal Processing II
Two -Dimensional Digital Signal Processing II Transforms and Median Filters Edited by T. S. Huang With Contributions by J.-O. Eklundh T.S. Huang B.I. Justusson H. J. Nussbaumer S.G. Tyan S. Zohar With
More informationStatistical Rock Physics
Statistical - Introduction Book review 3.1-3.3 Min Sun March. 13, 2009 Outline. What is Statistical. Why we need Statistical. How Statistical works Statistical Rock physics Information theory Statistics
More informationAPPLIED STRUCTURAL EQUATION MODELLING FOR RESEARCHERS AND PRACTITIONERS. Using R and Stata for Behavioural Research
APPLIED STRUCTURAL EQUATION MODELLING FOR RESEARCHERS AND PRACTITIONERS Using R and Stata for Behavioural Research APPLIED STRUCTURAL EQUATION MODELLING FOR RESEARCHERS AND PRACTITIONERS Using R and Stata
More informationNatural Image Statistics
Natural Image Statistics A probabilistic approach to modelling early visual processing in the cortex Dept of Computer Science Early visual processing LGN V1 retina From the eye to the primary visual cortex
More informationFundamentals of Electrical Circuit Analysis
Fundamentals of Electrical Circuit Analysis Md. Abdus Salam Quazi Mehbubar Rahman Fundamentals of Electrical Circuit Analysis 123 Md. Abdus Salam Electrical and Electronic Engineering Programme Area, Faculty
More informationAdvanced Engineering. Dynamics. H. R. Harrison. T. Nettleton. Formerly Department of Mechanical Engineering & Aeronautics City University London
Advanced Engineering Dynamics H. R. Harrison Formerly Department of Mechanical Engineering & Aeronautics City University London T. Nettleton Formerly Department of Mechanical Engineering & Aeronautics
More information27 : Distributed Monte Carlo Markov Chain. 1 Recap of MCMC and Naive Parallel Gibbs Sampling
10-708: Probabilistic Graphical Models 10-708, Spring 2014 27 : Distributed Monte Carlo Markov Chain Lecturer: Eric P. Xing Scribes: Pengtao Xie, Khoa Luu In this scribe, we are going to review the Parallel
More informationNeural Networks. Nethra Sambamoorthi, Ph.D. Jan CRMportals Inc., Nethra Sambamoorthi, Ph.D. Phone:
Neural Networks Nethra Sambamoorthi, Ph.D Jan 2003 CRMportals Inc., Nethra Sambamoorthi, Ph.D Phone: 732-972-8969 Nethra@crmportals.com What? Saying it Again in Different ways Artificial neural network
More informationPrinciples of Pattern Recognition. C. A. Murthy Machine Intelligence Unit Indian Statistical Institute Kolkata
Principles of Pattern Recognition C. A. Murthy Machine Intelligence Unit Indian Statistical Institute Kolkata e-mail: murthy@isical.ac.in Pattern Recognition Measurement Space > Feature Space >Decision
More informationFeature selection. c Victor Kitov August Summer school on Machine Learning in High Energy Physics in partnership with
Feature selection c Victor Kitov v.v.kitov@yandex.ru Summer school on Machine Learning in High Energy Physics in partnership with August 2015 1/38 Feature selection Feature selection is a process of selecting
More informationIntroduction to CLASSICAL MECHANICS
Introduction to CLASSICAL MECHANICS Introduction to CLASSICAL MECHANICS A.P. FRENCH Massachusetts Institute oj Technology M.G. EBISON The Institute oj Physics, London KLUWER ACADEMIC PUBLISHERS DORDRECHT
More informationEfficient Coding. Odelia Schwartz 2017
Efficient Coding Odelia Schwartz 2017 1 Levels of modeling Descriptive (what) Mechanistic (how) Interpretive (why) 2 Levels of modeling Fitting a receptive field model to experimental data (e.g., using
More informationAhsan Habib Khandoker Chandan Karmakar Michael Brennan Andreas Voss Marimuthu Palaniswami. Poincaré Plot Methods for Heart Rate Variability Analysis
Ahsan Habib Khandoker Chandan Karmakar Michael Brennan Andreas Voss Marimuthu Palaniswami Poincaré Plot Methods for Heart Rate Variability Analysis Poincaré Plot Methods for Heart Rate Variability Analysis
More informationEVALUATING SYMMETRIC INFORMATION GAP BETWEEN DYNAMICAL SYSTEMS USING PARTICLE FILTER
EVALUATING SYMMETRIC INFORMATION GAP BETWEEN DYNAMICAL SYSTEMS USING PARTICLE FILTER Zhen Zhen 1, Jun Young Lee 2, and Abdus Saboor 3 1 Mingde College, Guizhou University, China zhenz2000@21cn.com 2 Department
More informationLecture 16 Deep Neural Generative Models
Lecture 16 Deep Neural Generative Models CMSC 35246: Deep Learning Shubhendu Trivedi & Risi Kondor University of Chicago May 22, 2017 Approach so far: We have considered simple models and then constructed
More informationBourbaki Elements of the History of Mathematics
Bourbaki Elements of the History of Mathematics Springer Berlin Heidelberg New York Barcelona Hong Kong London Milan Paris Singapore Tokyo Nicolas Bourbaki Elements of the History of Mathematics Translated
More informationQUANTUM SCATTERING THEORY FOR SEVERAL PARTICLE SYSTEMS
.: ' :,. QUANTUM SCATTERING THEORY FOR SEVERAL PARTICLE SYSTEMS Mathematical Physics and Applied Mathematics Editors: M. Plato, Universite de Bourgogne, Dijon, France The titles published in this series
More informationPublication of the Museum of Nature South Tyrol Nr. 11
Publication of the Museum of Nature South Tyrol Nr. 11 ThiS is a FM Blank Page Erika Pignatti Sandro Pignatti Plant Life of the Dolomites Vegetation Tables Erika Pignatti Sandro Pignatti Rome Italy Publication
More informationProbability Theory, Random Processes and Mathematical Statistics
Probability Theory, Random Processes and Mathematical Statistics Mathematics and Its Applications Managing Editor: M.HAZEWINKEL Centre for Mathematics and Computer Science, Amsterdam, The Netherlands Volume
More informationNonlinear Parabolic and Elliptic Equations
Nonlinear Parabolic and Elliptic Equations Nonlinear Parabolic and Elliptic Equations c. V. Pao North Carolina State University Raleigh, North Carolina Plenum Press New York and London Library of Congress
More informationThis content has been downloaded from IOPscience. Please scroll down to see the full text.
This content has been downloaded from IOPscience. Please scroll down to see the full text. Download details: IP Address: 46.3.203.124 This content was downloaded on 30/12/2017 at 22:16 Please note that
More informationSTATISTICAL ANALYSIS WITH MISSING DATA
STATISTICAL ANALYSIS WITH MISSING DATA SECOND EDITION Roderick J.A. Little & Donald B. Rubin WILEY SERIES IN PROBABILITY AND STATISTICS Statistical Analysis with Missing Data Second Edition WILEY SERIES
More informationGatsby Theoretical Neuroscience Lectures: Non-Gaussian statistics and natural images Parts III-IV
Gatsby Theoretical Neuroscience Lectures: Non-Gaussian statistics and natural images Parts III-IV Aapo Hyvärinen Gatsby Unit University College London Part III: Estimation of unnormalized models Often,
More informationFoundations and Applications of Engineering Mechanics
Foundations and Applications of Engineering Mechanics 4843/24, 2nd Floor, Ansari Road, Daryaganj, Delhi - 110002, India Cambridge University Press is part of the University of Cambridge. It furthers the
More information