Visual Feature Spaces for Image Representation and Processing: The Multiscale Local Jet
|
|
- Augustus Blake
- 6 years ago
- Views:
Transcription
1 Visual Feature Spaces for Image Representation and Processing: The Multiscale Local Jet Antoine Manzanera ENSTA-ParisTech Workshop AMINA Oct, 19, 2010 Antoine Manzanera Visual Feature Spaces et caetera 1 / 53
2 Objectives and problem statement Objectives and problem statement Related works Our global objective is to propose a unied framework for representing and processing the visual data, trying to reach the maximal: Universality: from the lowest (denoising, registration,...) to the highest level (recognition, understanding,...) of vision. Computational eciency: tractable complexity for embedded video systems. Antoine Manzanera Visual Feature Spaces et caetera 2 / 53
3 Objectives and problem statement Objectives and problem statement Related works The principle of our framework is to adjoin to the video data an alternate (feature space) data structure whose metrics is related to visual similarity, and made up of dierent components: Feature space (local jet) projection. Quantisation (Codebook) of the feature space. Nearest Neighbour Search in the feature space. Indexing of the feature space for back-projection. Antoine Manzanera Visual Feature Spaces et caetera 3 / 53
4 Outline Introduction Objectives and problem statement Related works 1 Introduction Objectives and problem statement Related works 2 Local jet based similarity LJ feature space Metrics and invariance Computation and data structures 3 NL-means ltering Optical ow Background subtraction 4 Codebook histograms Singularities and Modes in the feature space 5 Conclusion and ongoing Antoine Manzanera work Visual Feature Spaces et caetera 4 / 53
5 Related works Introduction Objectives and problem statement Related works Manifold Image Processing The projected data form a manifold in some feature space [Peyré 09]. is made by transformation of the manifold, followed by a back-projection onto the image space. Filter Banks and Codebook Many visual representation frameworks, for texture (e.g. textons) or objects (e.g. visual bag of features) are based on lter banks and clustering [Freeman 91], [Rubner 99]. Nearest Neighbour Search We use Binary Search Trees to represent highly sparse sets from the feature space with a minimal amount of memory, and eciently perform Nearest Neighbour Search [Arya and Mount 07]. Antoine Manzanera Visual Feature Spaces et caetera 5 / 53
6 Outline Introduction Local jet based similarity LJ feature space Metrics and invariance Computation and data structures 1 Introduction Objectives and problem statement Related works 2 Local jet based similarity LJ feature space Metrics and invariance Computation and data structures 3 NL-means ltering Optical ow Background subtraction 4 Codebook histograms Singularities and Modes in the feature space 5 Conclusion and ongoing Antoine Manzanera work Visual Feature Spaces et caetera 6 / 53
7 Local derivatives and visual similarity Local jet based similarity LJ feature space Metrics and invariance Computation and data structures SSD based similarity d f (x, y) = k( i )(f (x + i) f (y + i)) 2 i W(O) Taylor expansion f (x + c) = r k=0 i=0 k ( ) k i c k i 1 c i 2 k f x k i 1 x i 2 (x) + o( c r ) Local jet based similarity d f (x, y) α (i,j) (f ij (x) f ij (y)) 2, with f ij = i+j f x1 i x j 2 i+j r Antoine Manzanera Visual Feature Spaces et caetera 7 / 53
8 Local jet based representation Local jet based similarity LJ feature space Metrics and invariance Computation and data structures The Local jet seen as a summary of the patch information (Reconstruction of the patch using Taylor expansion)... 1 Original patches 2 Order 0 (1d feature) 3 Order 1 (3d feature) 4 Order 2 (6d feature) (1) (2) (3) (4) Antoine Manzanera Visual Feature Spaces et caetera 8 / 53
9 Gaussian multiscale local jet Local jet based similarity LJ feature space Metrics and invariance Computation and data structures We do not use patches, but Gaussian Local Jet at a certain scale: Projection in the Local Jet space x ˆx = (a σ ij f σ ij (x)) i+j r Gaussian multiscale local jet f σ ij = f i+j G σ x i 1 x j 2 Normalised local jet a σ ij = σi+j i+j+1 Gσ is the 2d Gaussian function with variance σ 2. σ i+j is the scale space normalisation [Lindeberg 98]. i + j + 1 is the number of (i + j) th order derivatives. Antoine Manzanera Visual Feature Spaces et caetera 9 / 53
10 Multiscale Cartesian Local jet Local jet based similarity LJ feature space Metrics and invariance Computation and data structures f f f f f f f f f f f f Antoine Manzanera Visual Feature Spaces et caetera 10 / 53
11 Rotation invariance Introduction Local jet based similarity LJ feature space Metrics and invariance Computation and data structures f = f 00 f g = (f 2 + f 2 10 f t = 0 f gg = (f 20 f 2 f tt = (f 20 f 2 01) 1/ f 11 f 10 f 01 + f 02 f f 11 f 10 f 01 + f 02 f 2 f gt = (f 10 f 01 (f 20 f 02 ) + f 11 (f 2 g 01)f 2 g 10)f 2 01 f 2 10))f 2 g Antoine Manzanera Visual Feature Spaces et caetera 11 / 53
12 Multiscale Rotation Invariant Local jet Local jet based similarity LJ feature space Metrics and invariance Computation and data structures f 1.0 fg 1.0 fgg 1.0 fgt 1.0 ftt 1.0 f 4.0 fg 4.0 fgg 4.0 fgt 4.0 ftt 4.0 Antoine Manzanera Visual Feature Spaces et caetera 12 / 53
13 Local jet based similarity LJ feature space Metrics and invariance Computation and data structures Contrast change (and inversion!) Invariant Local jet Gradient direction Main absolute curvature direction Antoine Manzanera Visual Feature Spaces et caetera 13 / 53
14 Local jet based similarity LJ feature space Metrics and invariance Computation and data structures Local jet pixel categories and similarity metrics A reduced description useful for pixel comparison can be obtained by categorizing the local jet. See also [Crozier 10]. Antoine Manzanera Visual Feature Spaces et caetera 14 / 53
15 Some metrics Introduction Local jet based similarity LJ feature space Metrics and invariance Computation and data structures Single scale distance d σ f (x, y) = i+j r (f σ ij (x) f σ ij (y)) 2 Pan-scalic distance D S f (x, y) = i+j r,σ S (f σ ij (x) f σ ij (y)) 2 Trans-scalic distance d S f (x, y) = min (f σ 1 (σ 1,σ 2 ) S 2 ij (x) f σ 2 ij (y)) 2 i+j r Antoine Manzanera Visual Feature Spaces et caetera 15 / 53
16 Local jet similarity map: one example Local jet based similarity LJ feature space Metrics and invariance Computation and data structures (1) (2) (3) Antoine Manzanera Visual Feature Spaces et caetera 16 / 53
17 Parallel computation of the Local Jet Local jet based similarity LJ feature space Metrics and invariance Computation and data structures The local jet features lend themselves to a high level of parallel / cascaded computation. As an example, the recursive computation [Van Vliet 98] of the local jet at order 2 requires 9 couples of image scans, with only 2 levels of dependance. Antoine Manzanera Visual Feature Spaces et caetera 17 / 53
18 Data projection and representation Local jet based similarity LJ feature space Metrics and invariance Computation and data structures The pixel data are projected in the feature space. The feature vectors are collected within a binary search tree. Every feature vector is attached to a pixel index. Antoine Manzanera Visual Feature Spaces et caetera 18 / 53
19 Quantisation of the feature space Local jet based similarity LJ feature space Metrics and invariance Computation and data structures The feature space can be quantised through vector quantisation. Every codebook word keeps record of a set of pixel indices. Antoine Manzanera Visual Feature Spaces et caetera 19 / 53
20 Outline Introduction NL-means ltering Optical ow Background subtraction 1 Introduction Objectives and problem statement Related works 2 Local jet based similarity LJ feature space Metrics and invariance Computation and data structures 3 NL-means ltering Optical ow Background subtraction 4 Codebook histograms Singularities and Modes in the feature space 5 Conclusion and ongoing Antoine Manzanera work Visual Feature Spaces et caetera 20 / 53
21 NL-means in the feature space NL-means ltering Optical ow Background subtraction The NL-means lter calculates a weighted average of every pixel, with the weights dened as a function of local similarity. Here we simply use the distance in the feature space... ω(u, v) = e u v 2 2 h with:. a norm in the feature space. h a decay parameter....and perform the average on a neighbourhood of x in the image space (Limited Range), or on a neighbourhood of ˆx in the feature space (Unlimited range). Antoine Manzanera Visual Feature Spaces et caetera 21 / 53
22 Limited Range LJ-NL-Means NL-means ltering Optical ow Background subtraction The weights (in the LJ space) are calculated in a limited neighbourhood of x in the image space: Limited range NL-means flr NL(x) = 1 f (y)ω(ˆx, ŷ) ζ(x) y N (x) Antoine Manzanera Visual Feature Spaces et caetera 22 / 53
23 Unlimited Range LJ-NL-Means NL-means ltering Optical ow Background subtraction The weights (in the LJ space) are calculated in a limited neighbourhood of ˆx in the LJ space: Unlimited range NL-means fur NL(x) = 1 f (ǔ)ω(ˆx, u) ξ(x) u W(ˆx) Antoine Manzanera Visual Feature Spaces et caetera 23 / 53
24 NL-means ltering Optical ow Background subtraction LJ-NL-means: Relative computation time Patch based LR Local Jet LR LJ ( N (x) = 17 17), order 2. 1 scale 2 scales 3 scales 4 scales UR LJ ( W(ˆx f ) = 30), order 2, exact search ε = 0.0 ε = 1.0 ε = 3.0 ε = Same (4 scales), approximate search No quantisation 1622 words 795 words (40.9) 23.0 (19.7) Same (ε = 3.0), with quantisation (In parentheses: quantisation time). Antoine Manzanera Visual Feature Spaces et caetera 24 / 53
25 LJ-NL-means: results NL-means ltering Optical ow Background subtraction 1 Classical Patch based ( N (x) = 17 17) 2 Limited range ( N (x) = 17 17) 3 Unlimited range, exact search ( W(ˆx f ) = 30, single scale). 4 Same, 4 scales. 5 Same, approximate search (ε = 3.0). 6 Same, with quantisation ( 800 word codebook). (1) (2) (3) Antoine Manzanera (4) Visual Feature (5) Spaces et caetera (6) 25 / 53
26 NL-means ltering Optical ow Background subtraction Apparent motion in the feature space The optical ow estimation can be expressed by a simple nearest neighbour search in the feature space: u(f t 1, f t, x) = arg min d F (ˆx ft, v) v F ft 1 followed (in the case of a quantised feature space) by an optimization in the image space: y(f t 1, f t, x) = arg min d I (x, z) z F 1 (u(f t 1,f f t 1 t,x)) Antoine Manzanera Visual Feature Spaces et caetera 26 / 53
27 NL-means ltering Optical ow Background subtraction Apparent motion in the feature space Antoine Manzanera Visual Feature Spaces et caetera 27 / 53
28 NL-means ltering Optical ow Background subtraction Feature space optical ow: NL-Mean formulation A more general formulation can be obtained following the NL-Mean framework: y(f t 1, f t, x) = 1 ξ F (x) m ω F (ˆx ft, ν F f t 1 k (ˆx ft )) ξ I (x) k=1 z F 1 f t 1 (νf f t 1 k (ˆx ft )) Every nearest neighbour in the quantised feature space is weighted according to the feature space distance. Every image point from the reciprocal image of the quantised feature is weighted according to the distance between pixels. ω I (x, z)z Antoine Manzanera Visual Feature Spaces et caetera 28 / 53
29 Optical ow: results Introduction NL-means ltering Optical ow Background subtraction Antoine Manzanera Visual Feature Spaces et caetera 29 / 53
30 Optical ow: results Introduction NL-means ltering Optical ow Background subtraction Antoine Manzanera Visual Feature Spaces et caetera 30 / 53
31 NL-means ltering Optical ow Background subtraction Application: Characterization of Cardiac Motion Patterns [PhD F. Rodríguez, UNAL Bogotá] Antoine Manzanera Visual Feature Spaces et caetera 31 / 53
32 NL-means ltering Optical ow Background subtraction Background modelling by the feature space codebook The codebook of the quantised feature space can be used to model the visual apparence of the background in the methods based on sample and consensus. The principle is to keep record, for every pixel x, of a collection of M prototypes {m j (f, x)} j {1,M}, such that every prototype is a word from the quantised feature space (sampling phase). Antoine Manzanera Visual Feature Spaces et caetera 32 / 53
33 NL-means ltering Optical ow Background subtraction Background subtraction in the feature space The foreground classication is then performed by counting, for every x at time t, the number of prototypes that are close enough to ˆx ft (consensus phase). e(f, t, x) = 1 if {j {1, M}; d F (ˆx ft, m j (f, x)) > ρ} > τ = 0 otherwise Nota: The same codebook is used for the whole sequence. Antoine Manzanera Visual Feature Spaces et caetera 33 / 53
34 NL-means ltering Optical ow Background subtraction The ViBe algorithm in the feature space Antoine Manzanera Visual Feature Spaces et caetera 34 / 53
35 Motion detection: results NL-means ltering Optical ow Background subtraction Antoine Manzanera Visual Feature Spaces et caetera 35 / 53
36 Outline Introduction Codebook histograms Singularities and Modes in the feature space 1 Introduction Objectives and problem statement Related works 2 Local jet based similarity LJ feature space Metrics and invariance Computation and data structures 3 NL-means ltering Optical ow Background subtraction 4 Codebook histograms Singularities and Modes in the feature space 5 Conclusion and ongoing Antoine Manzanera work Visual Feature Spaces et caetera 36 / 53
37 Codebook histograms Codebook histograms Singularities and Modes in the feature space The distribution of the words (histogram) of the codebook provides a possible representation for an image or any visual category. See for example [Rubner 99] Second order statistics of the codebook, i.e. co-occurrence in the image domain of the back-projected words should also be a useful descriptor. Antoine Manzanera Visual Feature Spaces et caetera 37 / 53
38 Quantisation of the feature space Codebook histograms Singularities and Modes in the feature space Antoine Manzanera Visual Feature Spaces et caetera 38 / 53
39 Singularities of the feature space Codebook histograms Singularities and Modes in the feature space The isolated points in the feature space (i.e. the feature vectors whose average distance to their K nearest neighbours is the greatest), can be consider as a relevant way to fuse the detection of salient point and the calculation of attached descriptors. See also [Kervrann 08]. Antoine Manzanera Visual Feature Spaces et caetera 39 / 53
40 Singularities of the feature space Codebook histograms Singularities and Modes in the feature space Antoine Manzanera Visual Feature Spaces et caetera 40 / 53
41 Modes of the feature space Codebook histograms Singularities and Modes in the feature space The clusters in the feature space (whose center are the feature vectors whose average distance to their K nearest neighbours is the smallest), are also an interesting way to reduce the visual information to the most relevant features. The clusters (or modes) are calculated - see [Burman 09] - by (dynamically) dening a topology in the cloud of feature vectors, then labelling the connected component containing the current center point (feature point with smallest distance to its neighbours). The back-projection of the clusters in the image space correspond to the detection of large homogeneous coulours, long straight edges or regular texture elements. Antoine Manzanera Visual Feature Spaces et caetera 41 / 53
42 Modes of the feature space Codebook histograms Singularities and Modes in the feature space Antoine Manzanera Visual Feature Spaces et caetera 42 / 53
43 Outline Introduction 1 Introduction Objectives and problem statement Related works 2 Local jet based similarity LJ feature space Metrics and invariance Computation and data structures 3 NL-means ltering Optical ow Background subtraction 4 Codebook histograms Singularities and Modes in the feature space 5 Conclusion and ongoing Antoine Manzanera work Visual Feature Spaces et caetera 43 / 53
44 Overview of the framework Antoine Manzanera Visual Feature Spaces et caetera 44 / 53
45 Parallel NN search [Matthieu Garrigues 2010] Ecient parallel NN search is developped on GP-GPU: It is based on a volumic representation of the image NN spaces, and a recursive search of the NN that allow exponential search in the image space Antoine Manzanera Visual Feature Spaces et caetera 45 / 53
46 Parallel NN search [Matthieu Garrigues 2010] Antoine Manzanera Visual Feature Spaces et caetera 46 / 53
47 Parallel NN search [Matthieu Garrigues 2010] Antoine Manzanera Visual Feature Spaces et caetera 47 / 53
48 Bibliography Introduction [Peyré 09] G. PEYRE Manifold Models for Signals and Images Computer Vision and Image Understanding 113(2), (2009) [Lindeberg 98] T. LINDEBERG Feature detection with automatic scale selection International Journal of Computer Vision 30(2), (1998) [Koenderink 87] J.J. KOENDERINK and A.J. VAN DOORN Representation of Local Geometry in the Visual System Biological Cybernetics 55, (1987) Antoine Manzanera Visual Feature Spaces et caetera 48 / 53
49 Bibliography Introduction [Freeman 91] W.T. FREEMAN and E.H. ADELSON The design and use of Steerable Filters IEEE Trans. on Pattern Analysis and Machine Intelligence 13(9), (1991) [Crosier 10] M. CROSIER and L.D. GRIFFIN Using Basic Image Features for Texture Classication International Journal of Computer Vision 88(3), (2010) [Rubner 99] Y. RUBNER and C. TOMASI Texture-Based Image Retrieval Without Segmentation IEEE International Conference on Computer Vision, Kerkyra, Greece (1999) Antoine Manzanera Visual Feature Spaces et caetera 49 / 53
50 Bibliography Introduction [Mount 97] D.M. MOUNT and S. ARYA ANN: A Library for Approximate Nearest Neighbor Searching CGC Workshop on Computational Geometry (1997) [Burman 09] P. BURMAN and W. POLONIK Multivariate mode hunting: Data analytic tools with measures of signicance Journal of Multivariate Analysis 100(6), (2009) [Van Vliet 98] L.J. VAN VLIET, I.T. YOUNG and P.W. VERBEEK Recursive Gaussian derivative lters Proc. Int. Conf. on Pattern Recognition vol. 1, (1998) Antoine Manzanera Visual Feature Spaces et caetera 50 / 53
51 Bibliography Introduction [Buades 05] A. BUADES, B. COLL and J.M. MOREL A non-local algorithm for image denoising Proc. IEEE Conf. on Computer Vision and Pattern Recognition vol. 2, (2005) [Orchard 08] J. ORCHARD, M. EBRAHIMI and A. WONG Ecient Non-Local Means Denoising using the SVD Proc. Int. Conf. on Image Processing (2008) [Kervrann 08] C. KERVRANN and J. BOULANGER Local adaptivity to variable smoothness for exemplar-based image denoising and representation International Journal of Computer Vision 79(1), (2008) Antoine Manzanera Visual Feature Spaces et caetera 51 / 53
52 Bibliography Introduction [Barnich 09] O. BARNICH and M. VAN DROGENBROECK ViBe: a powerful random technique to estimate the background in video sequences International Conference on Acoustics, Speech, and Signal Processing (2009) [Wang 07] H. WANG and D. SUTER A consensus-based method for tracking: Modelling background scenario and foreground appearance Pattern Recognition 40(3), (2007) [Kim 04] K. KIM, T.H. CHALIDABHONGSE, D. HARDWOOD and L. DAVIS Background modeling and subtraction by codebook construction Proc. Int. Conf. on Image Processing (2004) Antoine Manzanera Visual Feature Spaces et caetera 52 / 53
53 Bibliography Introduction [Manzanera 10a] A. MANZANERA and processing through multiscale local jet features ENSTA-ParisTech Research Report (2010) [Manzanera 10b] A. MANZANERA Local Jet Based Similarity for NL-Means Filtering Proc. Int. Conf. on Pattern Recognition, Istambul, Turkey (2010) Antoine Manzanera Visual Feature Spaces et caetera 53 / 53
Overview. Introduction to local features. Harris interest points + SSD, ZNCC, SIFT. Evaluation and comparison of different detectors
Overview Introduction to local features Harris interest points + SSD, ZNCC, SIFT Scale & affine invariant interest point detectors Evaluation and comparison of different detectors Region descriptors and
More informationOverview. Introduction to local features. Harris interest points + SSD, ZNCC, SIFT. Evaluation and comparison of different detectors
Overview Introduction to local features Harris interest points + SSD, ZNCC, SIFT Scale & affine invariant interest point detectors Evaluation and comparison of different detectors Region descriptors and
More informationFeature detectors and descriptors. Fei-Fei Li
Feature detectors and descriptors Fei-Fei Li Feature Detection e.g. DoG detected points (~300) coordinates, neighbourhoods Feature Description e.g. SIFT local descriptors (invariant) vectors database of
More informationShape of Gaussians as Feature Descriptors
Shape of Gaussians as Feature Descriptors Liyu Gong, Tianjiang Wang and Fang Liu Intelligent and Distributed Computing Lab, School of Computer Science and Technology Huazhong University of Science and
More informationOverview. Harris interest points. Comparing interest points (SSD, ZNCC, SIFT) Scale & affine invariant interest points
Overview Harris interest points Comparing interest points (SSD, ZNCC, SIFT) Scale & affine invariant interest points Evaluation and comparison of different detectors Region descriptors and their performance
More informationFeature Vector Similarity Based on Local Structure
Feature Vector Similarity Based on Local Structure Evgeniya Balmachnova, Luc Florack, and Bart ter Haar Romeny Eindhoven University of Technology, P.O. Box 53, 5600 MB Eindhoven, The Netherlands {E.Balmachnova,L.M.J.Florack,B.M.terHaarRomeny}@tue.nl
More informationCorners, Blobs & Descriptors. With slides from S. Lazebnik & S. Seitz, D. Lowe, A. Efros
Corners, Blobs & Descriptors With slides from S. Lazebnik & S. Seitz, D. Lowe, A. Efros Motivation: Build a Panorama M. Brown and D. G. Lowe. Recognising Panoramas. ICCV 2003 How do we build panorama?
More informationKronecker Decomposition for Image Classification
university of innsbruck institute of computer science intelligent and interactive systems Kronecker Decomposition for Image Classification Sabrina Fontanella 1,2, Antonio Rodríguez-Sánchez 1, Justus Piater
More informationLoG Blob Finding and Scale. Scale Selection. Blobs (and scale selection) Achieving scale covariance. Blob detection in 2D. Blob detection in 2D
Achieving scale covariance Blobs (and scale selection) Goal: independently detect corresponding regions in scaled versions of the same image Need scale selection mechanism for finding characteristic region
More informationDetectors part II Descriptors
EECS 442 Computer vision Detectors part II Descriptors Blob detectors Invariance Descriptors Some slides of this lectures are courtesy of prof F. Li, prof S. Lazebnik, and various other lecturers Goal:
More informationSupervised locally linear embedding
Supervised locally linear embedding Dick de Ridder 1, Olga Kouropteva 2, Oleg Okun 2, Matti Pietikäinen 2 and Robert P.W. Duin 1 1 Pattern Recognition Group, Department of Imaging Science and Technology,
More informationAchieving scale covariance
Achieving scale covariance Goal: independently detect corresponding regions in scaled versions of the same image Need scale selection mechanism for finding characteristic region size that is covariant
More informationFeature detectors and descriptors. Fei-Fei Li
Feature detectors and descriptors Fei-Fei Li Feature Detection e.g. DoG detected points (~300) coordinates, neighbourhoods Feature Description e.g. SIFT local descriptors (invariant) vectors database of
More informationLecture 8: Interest Point Detection. Saad J Bedros
#1 Lecture 8: Interest Point Detection Saad J Bedros sbedros@umn.edu Review of Edge Detectors #2 Today s Lecture Interest Points Detection What do we mean with Interest Point Detection in an Image Goal:
More informationFocus was on solving matrix inversion problems Now we look at other properties of matrices Useful when A represents a transformations.
Previously Focus was on solving matrix inversion problems Now we look at other properties of matrices Useful when A represents a transformations y = Ax Or A simply represents data Notion of eigenvectors,
More informationWavelet-based Salient Points with Scale Information for Classification
Wavelet-based Salient Points with Scale Information for Classification Alexandra Teynor and Hans Burkhardt Department of Computer Science, Albert-Ludwigs-Universität Freiburg, Germany {teynor, Hans.Burkhardt}@informatik.uni-freiburg.de
More informationEstimators for Orientation and Anisotropy in Digitized Images
Estimators for Orientation and Anisotropy in Digitized Images Lucas J. van Vliet and Piet W. Verbeek Pattern Recognition Group of the Faculty of Applied Physics Delft University of Technolo Lorentzweg,
More informationLinear Subspace Models
Linear Subspace Models Goal: Explore linear models of a data set. Motivation: A central question in vision concerns how we represent a collection of data vectors. The data vectors may be rasterized images,
More informationOBJECT DETECTION AND RECOGNITION IN DIGITAL IMAGES
OBJECT DETECTION AND RECOGNITION IN DIGITAL IMAGES THEORY AND PRACTICE Bogustaw Cyganek AGH University of Science and Technology, Poland WILEY A John Wiley &. Sons, Ltd., Publication Contents Preface Acknowledgements
More informationA METHOD OF FINDING IMAGE SIMILAR PATCHES BASED ON GRADIENT-COVARIANCE SIMILARITY
IJAMML 3:1 (015) 69-78 September 015 ISSN: 394-58 Available at http://scientificadvances.co.in DOI: http://dx.doi.org/10.1864/ijamml_710011547 A METHOD OF FINDING IMAGE SIMILAR PATCHES BASED ON GRADIENT-COVARIANCE
More informationA Dual Sparse Decomposition Method for Image Denoising
A Dual Sparse Decomposition Method for Image Denoising arxiv:1704.07063v1 [cs.cv] 24 Apr 2017 Hong Sun 1 School of Electronic Information Wuhan University 430072 Wuhan, China 2 Dept. Signal and Image Processing
More informationMixture Models and EM
Mixture Models and EM Goal: Introduction to probabilistic mixture models and the expectationmaximization (EM) algorithm. Motivation: simultaneous fitting of multiple model instances unsupervised clustering
More informationHeNLM-LA: A LOCALLY ADAPTIVE HERMITE FUNCTIONS EXPANSION BASED NON-LOCAL MEANS DENOISING
ÏÐÎÃÐÀÌÌÈÐÎÂÀÍÈÅ, 2014, N o 2, ñ. 19 ÓÄÊ 004.421.6 HeNLM-LA: A LOCALLY ADAPTIVE HERMITE FUNCTIONS EXPANSION BASED NON-LOCAL MEANS DENOISING c 2014 ã. N. V. Mamaev, A. S. Lukin, D. V. Yurin The Faculty
More information6.869 Advances in Computer Vision. Prof. Bill Freeman March 1, 2005
6.869 Advances in Computer Vision Prof. Bill Freeman March 1 2005 1 2 Local Features Matching points across images important for: object identification instance recognition object class recognition pose
More informationSIMPLE GABOR FEATURE SPACE FOR INVARIANT OBJECT RECOGNITION
SIMPLE GABOR FEATURE SPACE FOR INVARIANT OBJECT RECOGNITION Ville Kyrki Joni-Kristian Kamarainen Laboratory of Information Processing, Department of Information Technology, Lappeenranta University of Technology,
More informationDimension Reduction Techniques. Presented by Jie (Jerry) Yu
Dimension Reduction Techniques Presented by Jie (Jerry) Yu Outline Problem Modeling Review of PCA and MDS Isomap Local Linear Embedding (LLE) Charting Background Advances in data collection and storage
More informationGlobal Scene Representations. Tilke Judd
Global Scene Representations Tilke Judd Papers Oliva and Torralba [2001] Fei Fei and Perona [2005] Labzebnik, Schmid and Ponce [2006] Commonalities Goal: Recognize natural scene categories Extract features
More informationMultimedia Databases. Previous Lecture. 4.1 Multiresolution Analysis. 4 Shape-based Features. 4.1 Multiresolution Analysis
Previous Lecture Multimedia Databases Texture-Based Image Retrieval Low Level Features Tamura Measure, Random Field Model High-Level Features Fourier-Transform, Wavelets Wolf-Tilo Balke Silviu Homoceanu
More informationMultimedia Databases. Wolf-Tilo Balke Philipp Wille Institut für Informationssysteme Technische Universität Braunschweig
Multimedia Databases Wolf-Tilo Balke Philipp Wille Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de 4 Previous Lecture Texture-Based Image Retrieval Low
More informationMultiple Similarities Based Kernel Subspace Learning for Image Classification
Multiple Similarities Based Kernel Subspace Learning for Image Classification Wang Yan, Qingshan Liu, Hanqing Lu, and Songde Ma National Laboratory of Pattern Recognition, Institute of Automation, Chinese
More informationFiltering via a Reference Set. A.Haddad, D. Kushnir, R.R. Coifman Technical Report YALEU/DCS/TR-1441 February 21, 2011
Patch-based de-noising algorithms and patch manifold smoothing have emerged as efficient de-noising methods. This paper provides a new insight on these methods, such as the Non Local Means or the image
More informationRandomized Algorithms
Randomized Algorithms Saniv Kumar, Google Research, NY EECS-6898, Columbia University - Fall, 010 Saniv Kumar 9/13/010 EECS6898 Large Scale Machine Learning 1 Curse of Dimensionality Gaussian Mixture Models
More informationFace Recognition Using Laplacianfaces He et al. (IEEE Trans PAMI, 2005) presented by Hassan A. Kingravi
Face Recognition Using Laplacianfaces He et al. (IEEE Trans PAMI, 2005) presented by Hassan A. Kingravi Overview Introduction Linear Methods for Dimensionality Reduction Nonlinear Methods and Manifold
More informationPoisson NL means: unsupervised non local means for Poisson noise
2010 IEEE International Conference on Image Processing Hong-Kong, September 26-29, 2010 Poisson NL means: unsupervised non local means for Poisson noise Charles Deledalle 1,FlorenceTupin 1, Loïc Denis
More informationAdvanced Machine Learning & Perception
Advanced Machine Learning & Perception Instructor: Tony Jebara Topic 1 Introduction, researchy course, latest papers Going beyond simple machine learning Perception, strange spaces, images, time, behavior
More informationMultimedia Databases. 4 Shape-based Features. 4.1 Multiresolution Analysis. 4.1 Multiresolution Analysis. 4.1 Multiresolution Analysis
4 Shape-based Features Multimedia Databases Wolf-Tilo Balke Silviu Homoceanu Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de 4 Multiresolution Analysis
More informationProbabilistic Model of Error in Fixed-Point Arithmetic Gaussian Pyramid
Probabilistic Model o Error in Fixed-Point Arithmetic Gaussian Pyramid Antoine Méler John A. Ruiz-Hernandez James L. Crowley INRIA Grenoble - Rhône-Alpes 655 avenue de l Europe 38 334 Saint Ismier Cedex
More informationDiffusion/Inference geometries of data features, situational awareness and visualization. Ronald R Coifman Mathematics Yale University
Diffusion/Inference geometries of data features, situational awareness and visualization Ronald R Coifman Mathematics Yale University Digital data is generally converted to point clouds in high dimensional
More informationRiemannian Metric Learning for Symmetric Positive Definite Matrices
CMSC 88J: Linear Subspaces and Manifolds for Computer Vision and Machine Learning Riemannian Metric Learning for Symmetric Positive Definite Matrices Raviteja Vemulapalli Guide: Professor David W. Jacobs
More informationStatistical Filters for Crowd Image Analysis
Statistical Filters for Crowd Image Analysis Ákos Utasi, Ákos Kiss and Tamás Szirányi Distributed Events Analysis Research Group, Computer and Automation Research Institute H-1111 Budapest, Kende utca
More informationIntroduction to Machine Learning. PCA and Spectral Clustering. Introduction to Machine Learning, Slides: Eran Halperin
1 Introduction to Machine Learning PCA and Spectral Clustering Introduction to Machine Learning, 2013-14 Slides: Eran Halperin Singular Value Decomposition (SVD) The singular value decomposition (SVD)
More informationSIFT: SCALE INVARIANT FEATURE TRANSFORM BY DAVID LOWE
SIFT: SCALE INVARIANT FEATURE TRANSFORM BY DAVID LOWE Overview Motivation of Work Overview of Algorithm Scale Space and Difference of Gaussian Keypoint Localization Orientation Assignment Descriptor Building
More informationFeature extraction: Corners and blobs
Feature extraction: Corners and blobs Review: Linear filtering and edge detection Name two different kinds of image noise Name a non-linear smoothing filter What advantages does median filtering have over
More information1 Inria Rennes - Bretagne Atlantique 2 Alcatel-Lucent - Bell Labs France
LOW-COMPLEXITY SINGLE-IMAGE SUPER-RESOLUTION BASED ON NONNEGATIVE NEIGHBOR EMBEDDING Marco Bevilacqua 1,2, Aline Roumy 1, Christine Guillemot 1, Marie-Line Alberi Morel 2 1 Inria Rennes - Bretagne Atlantique
More informationEEL 851: Biometrics. An Overview of Statistical Pattern Recognition EEL 851 1
EEL 851: Biometrics An Overview of Statistical Pattern Recognition EEL 851 1 Outline Introduction Pattern Feature Noise Example Problem Analysis Segmentation Feature Extraction Classification Design Cycle
More informationCS 3710: Visual Recognition Describing Images with Features. Adriana Kovashka Department of Computer Science January 8, 2015
CS 3710: Visual Recognition Describing Images with Features Adriana Kovashka Department of Computer Science January 8, 2015 Plan for Today Presentation assignments + schedule changes Image filtering Feature
More informationEE 6882 Visual Search Engine
EE 6882 Visual Search Engine Prof. Shih Fu Chang, Feb. 13 th 2012 Lecture #4 Local Feature Matching Bag of Word image representation: coding and pooling (Many slides from A. Efors, W. Freeman, C. Kambhamettu,
More informationScale & Affine Invariant Interest Point Detectors
Scale & Affine Invariant Interest Point Detectors Krystian Mikolajczyk and Cordelia Schmid Presented by Hunter Brown & Gaurav Pandey, February 19, 2009 Roadmap: Motivation Scale Invariant Detector Affine
More informationEdges and Scale. Image Features. Detecting edges. Origin of Edges. Solution: smooth first. Effects of noise
Edges and Scale Image Features From Sandlot Science Slides revised from S. Seitz, R. Szeliski, S. Lazebnik, etc. Origin of Edges surface normal discontinuity depth discontinuity surface color discontinuity
More informationProbabilistic Latent Semantic Analysis
Probabilistic Latent Semantic Analysis Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr
More informationVlad Estivill-Castro (2016) Robots for People --- A project for intelligent integrated systems
1 Vlad Estivill-Castro (2016) Robots for People --- A project for intelligent integrated systems V. Estivill-Castro 2 Perception Concepts Vision Chapter 4 (textbook) Sections 4.3 to 4.5 What is the course
More information6.869 Advances in Computer Vision. Bill Freeman, Antonio Torralba and Phillip Isola MIT Oct. 3, 2018
6.869 Advances in Computer Vision Bill Freeman, Antonio Torralba and Phillip Isola MIT Oct. 3, 2018 1 Sampling Sampling Pixels Continuous world 3 Sampling 4 Sampling 5 Continuous image f (x, y) Sampling
More informationL11: Pattern recognition principles
L11: Pattern recognition principles Bayesian decision theory Statistical classifiers Dimensionality reduction Clustering This lecture is partly based on [Huang, Acero and Hon, 2001, ch. 4] Introduction
More informationLecture 8: Interest Point Detection. Saad J Bedros
#1 Lecture 8: Interest Point Detection Saad J Bedros sbedros@umn.edu Last Lecture : Edge Detection Preprocessing of image is desired to eliminate or at least minimize noise effects There is always tradeoff
More informationMachine Learning Techniques for Computer Vision
Machine Learning Techniques for Computer Vision Part 2: Unsupervised Learning Microsoft Research Cambridge x 3 1 0.5 0.2 0 0.5 0.3 0 0.5 1 ECCV 2004, Prague x 2 x 1 Overview of Part 2 Mixture models EM
More informationFace Recognition. Face Recognition. Subspace-Based Face Recognition Algorithms. Application of Face Recognition
ace Recognition Identify person based on the appearance of face CSED441:Introduction to Computer Vision (2017) Lecture10: Subspace Methods and ace Recognition Bohyung Han CSE, POSTECH bhhan@postech.ac.kr
More informationVideo and Motion Analysis Computer Vision Carnegie Mellon University (Kris Kitani)
Video and Motion Analysis 16-385 Computer Vision Carnegie Mellon University (Kris Kitani) Optical flow used for feature tracking on a drone Interpolated optical flow used for super slow-mo optical flow
More informationImage Noise: Detection, Measurement and Removal Techniques. Zhifei Zhang
Image Noise: Detection, Measurement and Removal Techniques Zhifei Zhang Outline Noise measurement Filter-based Block-based Wavelet-based Noise removal Spatial domain Transform domain Non-local methods
More informationA TEXTON FOR FAST AND FLEXIBLE GAUSSIAN TEXTURE SYNTHESIS. Bruno Galerne, Arthur Leclaire, Lionel Moisan
A TEXTON FOR FAST AND FLEXIBLE GAUSSIAN TEXTURE SYNTHESIS Bruno Galerne, Arthur Leclaire, Lionel Moisan Université Paris Descartes, MAP5, CNRS UMR 8145, France ABSTRACT Gaussian textures can be easily
More informationDESIGN OF MULTI-DIMENSIONAL DERIVATIVE FILTERS. Eero P. Simoncelli
Published in: First IEEE Int l Conf on Image Processing, Austin Texas, vol I, pages 790--793, November 1994. DESIGN OF MULTI-DIMENSIONAL DERIVATIVE FILTERS Eero P. Simoncelli GRASP Laboratory, Room 335C
More informationReview Smoothing Spatial Filters Sharpening Spatial Filters. Spatial Filtering. Dr. Praveen Sankaran. Department of ECE NIT Calicut.
Spatial Filtering Dr. Praveen Sankaran Department of ECE NIT Calicut January 7, 203 Outline 2 Linear Nonlinear 3 Spatial Domain Refers to the image plane itself. Direct manipulation of image pixels. Figure:
More informationShared Segmentation of Natural Scenes. Dependent Pitman-Yor Processes
Shared Segmentation of Natural Scenes using Dependent Pitman-Yor Processes Erik Sudderth & Michael Jordan University of California, Berkeley Parsing Visual Scenes sky skyscraper sky dome buildings trees
More informationChapter 4 Dynamic Bayesian Networks Fall Jin Gu, Michael Zhang
Chapter 4 Dynamic Bayesian Networks 2016 Fall Jin Gu, Michael Zhang Reviews: BN Representation Basic steps for BN representations Define variables Define the preliminary relations between variables Check
More informationRandom walks and anisotropic interpolation on graphs. Filip Malmberg
Random walks and anisotropic interpolation on graphs. Filip Malmberg Interpolation of missing data Assume that we have a graph where we have defined some (real) values for a subset of the nodes, and that
More informationBlobs & Scale Invariance
Blobs & Scale Invariance Prof. Didier Stricker Doz. Gabriele Bleser Computer Vision: Object and People Tracking With slides from Bebis, S. Lazebnik & S. Seitz, D. Lowe, A. Efros 1 Apertizer: some videos
More informationMotivating the Covariance Matrix
Motivating the Covariance Matrix Raúl Rojas Computer Science Department Freie Universität Berlin January 2009 Abstract This note reviews some interesting properties of the covariance matrix and its role
More informationStochastic Quasi-Newton Methods
Stochastic Quasi-Newton Methods Donald Goldfarb Department of IEOR Columbia University UCLA Distinguished Lecture Series May 17-19, 2016 1 / 35 Outline Stochastic Approximation Stochastic Gradient Descent
More informationRiesz-Transforms vs. Derivatives: On the Relationship Between the Boundary Tensor and the Energy Tensor
Riesz-Transforms vs. Derivatives: On the Relationship Between the Boundary Tensor and the Energy Tensor Ullrich Köthe 1 and Michael Felsberg 1 Cognitive Systems Group, University of Hamburg koethe@informatik.uni-hamburg.de
More informationRapid Object Recognition from Discriminative Regions of Interest
Rapid Object Recognition from Discriminative Regions of Interest Gerald Fritz, Christin Seifert, Lucas Paletta JOANNEUM RESEARCH Institute of Digital Image Processing Wastiangasse 6, A-81 Graz, Austria
More informationHidden Markov Models and Gaussian Mixture Models
Hidden Markov Models and Gaussian Mixture Models Hiroshi Shimodaira and Steve Renals Automatic Speech Recognition ASR Lectures 4&5 23&27 January 2014 ASR Lectures 4&5 Hidden Markov Models and Gaussian
More informationLogarithmic quantisation of wavelet coefficients for improved texture classification performance
Logarithmic quantisation of wavelet coefficients for improved texture classification performance Author Busch, Andrew, W. Boles, Wageeh, Sridharan, Sridha Published 2004 Conference Title 2004 IEEE International
More informationA Background Layer Model for Object Tracking through Occlusion
A Background Layer Model for Object Tracking through Occlusion Yue Zhou and Hai Tao UC-Santa Cruz Presented by: Mei-Fang Huang CSE252C class, November 6, 2003 Overview Object tracking problems Dynamic
More informationRegion Covariance: A Fast Descriptor for Detection and Classification
MITSUBISHI ELECTRIC RESEARCH LABORATORIES http://www.merl.com Region Covariance: A Fast Descriptor for Detection and Classification Oncel Tuzel, Fatih Porikli, Peter Meer TR2005-111 May 2006 Abstract We
More informationThe Minimum Message Length Principle for Inductive Inference
The Principle for Inductive Inference Centre for Molecular, Environmental, Genetic & Analytic (MEGA) Epidemiology School of Population Health University of Melbourne University of Helsinki, August 25,
More informationarxiv: v1 [cs.cv] 10 Feb 2016
GABOR WAVELETS IN IMAGE PROCESSING David Bařina Doctoral Degree Programme (2), FIT BUT E-mail: xbarin2@stud.fit.vutbr.cz Supervised by: Pavel Zemčík E-mail: zemcik@fit.vutbr.cz arxiv:162.338v1 [cs.cv]
More informationCheng Soon Ong & Christian Walder. Canberra February June 2018
Cheng Soon Ong & Christian Walder Research Group and College of Engineering and Computer Science Canberra February June 218 Outlines Overview Introduction Linear Algebra Probability Linear Regression 1
More information38 1 Vol. 38, No ACTA AUTOMATICA SINICA January, Bag-of-phrases.. Image Representation Using Bag-of-phrases
38 1 Vol. 38, No. 1 2012 1 ACTA AUTOMATICA SINICA January, 2012 Bag-of-phrases 1, 2 1 1 1, Bag-of-words,,, Bag-of-words, Bag-of-phrases, Bag-of-words DOI,, Bag-of-words, Bag-of-phrases, SIFT 10.3724/SP.J.1004.2012.00046
More informationLarge Scale Environment Partitioning in Mobile Robotics Recognition Tasks
Large Scale Environment in Mobile Robotics Recognition Tasks Boyan Bonev, Miguel Cazorla {boyan,miguel}@dccia.ua.es Robot Vision Group Department of Computer Science and Artificial Intelligence University
More informationA Generative Perspective on MRFs in Low-Level Vision Supplemental Material
A Generative Perspective on MRFs in Low-Level Vision Supplemental Material Uwe Schmidt Qi Gao Stefan Roth Department of Computer Science, TU Darmstadt 1. Derivations 1.1. Sampling the Prior We first rewrite
More informationModeling Complex Temporal Composition of Actionlets for Activity Prediction
Modeling Complex Temporal Composition of Actionlets for Activity Prediction ECCV 2012 Activity Recognition Reading Group Framework of activity prediction What is an Actionlet To segment a long sequence
More informationModeling Multiscale Differential Pixel Statistics
Modeling Multiscale Differential Pixel Statistics David Odom a and Peyman Milanfar a a Electrical Engineering Department, University of California, Santa Cruz CA. 95064 USA ABSTRACT The statistics of natural
More informationCITS 4402 Computer Vision
CITS 4402 Computer Vision A/Prof Ajmal Mian Adj/A/Prof Mehdi Ravanbakhsh Lecture 06 Object Recognition Objectives To understand the concept of image based object recognition To learn how to match images
More informationNonlinear Dimensionality Reduction
Outline Hong Chang Institute of Computing Technology, Chinese Academy of Sciences Machine Learning Methods (Fall 2012) Outline Outline I 1 Kernel PCA 2 Isomap 3 Locally Linear Embedding 4 Laplacian Eigenmap
More informationOn Mean Curvature Diusion in Nonlinear Image Filtering. Adel I. El-Fallah and Gary E. Ford. University of California, Davis. Davis, CA
On Mean Curvature Diusion in Nonlinear Image Filtering Adel I. El-Fallah and Gary E. Ford CIPIC, Center for Image Processing and Integrated Computing University of California, Davis Davis, CA 95616 Abstract
More information(z) UN-1(z) PN-1(z) Pn-1(z) - z k. 30(z) 30(z) (a) (b) (c) x y n ...
Minimum Memory Implementations of the Lifting Scheme Christos Chrysas Antonio Ortega HewlettPackard Laboratories Integrated Media Systems Center 151 Page Mill Road, Bldg.3U3 University of Southern California
More informationIndependent Component Analysis and Unsupervised Learning. Jen-Tzung Chien
Independent Component Analysis and Unsupervised Learning Jen-Tzung Chien TABLE OF CONTENTS 1. Independent Component Analysis 2. Case Study I: Speech Recognition Independent voices Nonparametric likelihood
More informationImage representation with multi-scale gradients
Image representation with multi-scale gradients Eero P Simoncelli Center for Neural Science, and Courant Institute of Mathematical Sciences New York University http://www.cns.nyu.edu/~eero Visual image
More informationIndependent Component Analysis and Unsupervised Learning
Independent Component Analysis and Unsupervised Learning Jen-Tzung Chien National Cheng Kung University TABLE OF CONTENTS 1. Independent Component Analysis 2. Case Study I: Speech Recognition Independent
More informationModulation-Feature based Textured Image Segmentation Using Curve Evolution
Modulation-Feature based Textured Image Segmentation Using Curve Evolution Iasonas Kokkinos, Giorgos Evangelopoulos and Petros Maragos Computer Vision, Speech Communication and Signal Processing Group
More informationSYMBOL RECOGNITION IN HANDWRITTEN MATHEMATI- CAL FORMULAS
SYMBOL RECOGNITION IN HANDWRITTEN MATHEMATI- CAL FORMULAS Hans-Jürgen Winkler ABSTRACT In this paper an efficient on-line recognition system for handwritten mathematical formulas is proposed. After formula
More informationIntroduction to Linear Image Processing
Introduction to Linear Image Processing 1 IPAM - UCLA July 22, 2013 Iasonas Kokkinos Center for Visual Computing Ecole Centrale Paris / INRIA Saclay Image Sciences in a nutshell 2 Image Processing Image
More informationDistance Metric Learning
Distance Metric Learning Technical University of Munich Department of Informatics Computer Vision Group November 11, 2016 M.Sc. John Chiotellis: Distance Metric Learning 1 / 36 Outline Computer Vision
More informationAffine Structure From Motion
EECS43-Advanced Computer Vision Notes Series 9 Affine Structure From Motion Ying Wu Electrical Engineering & Computer Science Northwestern University Evanston, IL 68 yingwu@ece.northwestern.edu Contents
More informationK-Means, Expectation Maximization and Segmentation. D.A. Forsyth, CS543
K-Means, Expectation Maximization and Segmentation D.A. Forsyth, CS543 K-Means Choose a fixed number of clusters Choose cluster centers and point-cluster allocations to minimize error can t do this by
More informationGeneralized Laplacian as Focus Measure
Generalized Laplacian as Focus Measure Muhammad Riaz 1, Seungjin Park, Muhammad Bilal Ahmad 1, Waqas Rasheed 1, and Jongan Park 1 1 School of Information & Communications Engineering, Chosun University,
More informationPedestrian Density Estimation by a Weighted Bag of Visual Words Model
Pedestrian Density Estimation by a Weighted Bag of Visual Words Model Shilin Zhang and Xunyuan Zhang image representation termed bag of visual words integrating weighting scheme and spatial pyramid co-occurrence,
More informationStability of Recursive Gaussian Filtering for Piecewise Linear Bilateral Filtering
Stability of Recursive Gaussian Filtering for Piecewise Linear Bilateral Filtering Koichiro Watanabe, Yoshihiro Maeda, and Norishige Fukushima Nagoya Institute of Technology, Nagoya, Japan fukushima@nitech.ac.jp
More informationResearch Article Nonlocal Mean Image Denoising Using Anisotropic Structure Tensor
Advances in Optical Technologies Volume 2013, Article ID 794728, 6 pages http://dx.doi.org/10.1155/2013/794728 Research Article Nonlocal Mean Image Denoising Using Anisotropic Structure Tensor Xi Wu, 1,
More informationStatistical Machine Learning
Statistical Machine Learning Christoph Lampert Spring Semester 2015/2016 // Lecture 12 1 / 36 Unsupervised Learning Dimensionality Reduction 2 / 36 Dimensionality Reduction Given: data X = {x 1,..., x
More information