March 13, Paper: R.R. Coifman, S. Lafon, Diffusion maps ([Coifman06]) Seminar: Learning with Graphs, Prof. Hein, Saarland University
|
|
- Abner Fox
- 5 years ago
- Views:
Transcription
1 Kernels March 13, 2008 Paper: R.R. Coifman, S. Lafon, maps ([Coifman06]) Seminar: Learning with Graphs, Prof. Hein, Saarland University
2 Kernels Figure: Example Application from [LafonWWW] meaningful geometric descriptions of data sets data parametrization (demo) dimensionality reduction
3 Kernels 1 Table of Contents 2 Application of to 3 Limit of the Graph Kernels 4 5 and Appendix
4 Kernels 1 2 Application of to 3 Limit of the Graph Kernels 4 5 and Appendix
5 Process on a Graph Kernels : Kernel k : X X R on data set X Symmetric: k(x, y) = k(y, x) positivity preserving:k(x, y) 0 represents similarity between points in X defines edge weight matrix W in weighted graph (X, k) Normalized Graph L (rw) = P d(x) = X k(x, y)dµ(y) (degree, discrete: d i = n j=1 w ij) p(x, y) = k(x,y) d(x) ( p(x, y)dµ(y) = 1) X p(x, y) = transition probability in one time step Matrix P defines Markov chain :P t = probability of transition from x to y in t steps
6 Kernels Process on a Graph : a cluster is a region in which the probability of escaping this region is low. Figure: at time t = 8, right: P 8, left: color from one row of P 8
7 Kernels Process on a Graph Block structure of P reveals clusters, after 64 steps the closest two clusters have merged Figure: at time t = 64
8 Process on a Graph All clusters have merged after 1024 time steps Kernels Figure: at time t = 1024
9 Kernels Distance Goal: relate spectral properties of Markov chain to geometry of the data : distances {D t } t N D t (x, y) 2 p t (x, ) p t (y, ) L 2 (X,dµ/π) D t (x, y) will be small, if there is a large number of short paths between x and y Figure: Example paths for diffusion distance [Maggioni06]
10 Kernels distances can be computed using eigenvectors ψ l and eigenvalues λ l of P ( ) 1 D t (x, y) = l 1 λ2t l (ψ l (x) ψ l (y)) 2 2 The proof uses the spectral theorem in the Hilbert space (more later) and the fact that the eigenfunctions of P are orthonormal. Using 1 = λ 0 > λ 1 λ 2... the distance may be approximated with the first s eigenvalues.
11 Kernels : Map Ψ t (x) : X R s Ψ t (x) λ t 1 ψ 1(x) λ t 2 ψ 2(x). λ t sψ s (x) Proposition: The diffusion map Ψ t embeds the data into the Euclidean space R s so that in this space, the Euclidean distance is equal to the diffusion distance (up to relative accuracy), or equivalently Ψ t (x) Ψ t (y) = D t (x, y) Remark: Unlike Eigenmaps, each dimension is weighted by the decreasing eigenvalues.
12 Example of Eigenfunctions ψ Kernels Figure: First 4 eigenfunctions of a dumb-bell shaped manifold. [Maggioni06]
13 Kernels is one possible application of diffusion maps. 1 Construct similarity graph 2 Compute normalized 3 Solve generalized eigenvector problem Lu = λdu 4 Define the embedding into k-dimensional Euclidean space via diffusion maps 5 Cluster points y i R k with k-means
14 Kernels 1 2 Application of to 3 Limit of the Graph Kernels 4 5 and Appendix
15 New Scenario Kernels Points are sampled from a probability density on a submanifold of R n Sampling often not related to geometry of manifold. Biased data: e.g. more faces from one pose Goal: Recover manifold structure regardless of the distribution of the data Additional concepts needed for this continuous setting Figure: Manifold with density [Learned07]
16 The Continuous Setting Kernels : Manifold: A space in which every point has a neighborhood which resembles Euclidean space, but in which the global structure may be more complicated, e.g. sphere or 2-d surface : Hilbert Space: An inner product space X on a space S that is complete under the norm f = f, f defined by the inner product,. For example the L 2 norm: f, g = S f (x)g(x)dx Ability to define functions f using a function basis (Φ i ): f = α i Φ i (x) Orthonormal basis similar to vector space: Φ i = 1 i and Φ i, Φ j = 0 i j
17 The Continuous Setting Kernels : L : X X A function of functions Linear operator: L(λf ) = λlf Eigenfunction of an operator:lf = λf An operator is Hermitian (symmetric) if Lf, g = f, Lg Eigenfunctions of Hermitian operators form an orthonormal basis on their Hilbert space X. Example:Laplace
18 Kernels : The Laplace Interesting properties: f = n 2 f x 2 i=1 i First eigenfunction is constant ( 2 c x 2 i = 0), Second eigenfunction has to change signs (orthogonal to first) and needs to be only scaled by operator:...
19 Kernels : The Laplace Interesting properties: f = n 2 f x 2 i=1 i First eigenfunction is constant ( 2 c = 0), xi 2 Second eigenfunction has to change signs (orthogonal to first) and needs to be only scaled by operator:... sine and cosine, since (sin(ωx)) = ω 2 sin(ωx) Hence, the eigenfunctions of form a nice orthonormal basis in X. The is extension of normal to manifolds. Problem: We only have a finite sample from a probability measure p on an m-dimensional submanifold M in R d.
20 Kernels Limit of the Graph Theorem: Let M be a m-dimensional submanifold in R d, {X i } n i=1 a sample from a probability measure P on M with density p. Then under several conditions on M, p and the kernel k, we have: If neighborhood h 0, number of points in it n and nh m+2 / log n, then the random walk converges to the operator lim n (L(rw) n f )(x) ( s f )(x) Where the weighted operator s = M + s p, f p induces an anisotropic diffusion towards or away from increasing density depending on s.
21 Kernels Now, we have established ourselves in the awesome world of operators in Hilbert Spaces on submanifolds. Initial motivation was to analyze geometry regardless of sampling distribution. What is the influence of the geometry and the density over the eigenfunctions and the spectrum of the diffusion?
22 Kernels We introduced a family of weighted operators that allow a scaling of the influence of the density via one parameter s: s = M + s p p, f The smoothness functional induced by s is: S(f ) = f 2 p s dv M and is to be minimized. ( to graph : i,j w ij(fi fj) 2 ) Hence, this functional prefers functions that are smooth in high density regions (see board).
23 Kernels Construction of the Family of s Now, we have the tools to define a new kernel for the weights of normalized graph s. But what exactly has changed in the construction of diffusion kernels?
24 Kernels Standard Normalized Graph Fix kernel k(x, y) d(x) = k(x, y)dµ(y) X p(x, y) = k(x,y) d(x) Kernels Kernel Normalized Graph Fix kernel k(x, y) Renormalize weight into new anisotropic kernel: q(x) = X k(x, y)q(y)dy k (α) = k(x,y) q α (x)q α (y) d (α) (x) = X k(α) (x, y)q(y)dy p (α) = k(α) (x,y) d (α) (x) α = 0: Construct normalized weights for graph α = 1: approximation, the normalization removes the influence of the density and recovers the geometry of the data. s = 2(1 α)
25 Kernels is one possible application of diffusion maps. 1 Construct similarity graph Apply normalization 2 Compute normalized 3 Solve generalized eigenvector problem Lu = λdu 4 Define the embedding into k-dimensional Euclidean space via diffusion maps 5 Cluster points y i R k with k-means
26 Kernels Embeddings via Approximation Figure: From left to right: original curves, the densities of points, the embeddings via the graph (α = 0) and the embeddings via the Laplace Beltrami approximation (α = 1). In the latter case, the curve is embedded as a perfect circle and the arclength parametrization is recovered.
27 Parametrization of Curves Kernels Figure: Live Example from [LafonWWW]
28 Kernels 1. Map: Ψ t (x) : X R s Ψ t (x) λ t 1 ψ 1(x) λ t 2 ψ 2(x). λ t sψ s (x) data into the Euclidean space so that: Ψ t (x) Ψ t (y) = D t (x, y) 2. Approximation: k (α) = k(x,y) q α (x)q α (y) normalization parameter α steers the influence of the density allows the complete separation of the distribution of the data from the geometry of the underlying manifold
29 Kernels Coifman06: R.R. Coifman, S. Lafon, maps, Appl. Comput. Harmon. Anal. 21 (1) (2006) 631. LafonWWW: Stephane Lafon s website: http: // www. math. yale. edu/ ~ sl349/ Luxburg07: von Luxburg, U.: A Tutorial on. Statistics and Computing 17(4), ( ) Hein07: M. Hein, J.-Y. Audibert, U. von Luxburg. Convergence of graph s on random neighborhood graphs, Journal of Machine Learning Research 8, , Learned07: Manifold Picture from http: // www. cs. umass. edu/ ~ elm/ papers_ by_ research. html Maggioni06: and wavelet bases for value function approximation and their connection to kernel methods, Mauro Maggioni, Yale University, ICML Workshop, June 29th, 2006
30 Thank you. Kernels
Diffusion Wavelets and Applications
Diffusion Wavelets and Applications J.C. Bremer, R.R. Coifman, P.W. Jones, S. Lafon, M. Mohlenkamp, MM, R. Schul, A.D. Szlam Demos, web pages and preprints available at: S.Lafon: www.math.yale.edu/~sl349
More informationDiffusion Geometries, Diffusion Wavelets and Harmonic Analysis of large data sets.
Diffusion Geometries, Diffusion Wavelets and Harmonic Analysis of large data sets. R.R. Coifman, S. Lafon, MM Mathematics Department Program of Applied Mathematics. Yale University Motivations The main
More informationContribution from: Springer Verlag Berlin Heidelberg 2005 ISBN
Contribution from: Mathematical Physics Studies Vol. 7 Perspectives in Analysis Essays in Honor of Lennart Carleson s 75th Birthday Michael Benedicks, Peter W. Jones, Stanislav Smirnov (Eds.) Springer
More informationGraphs, Geometry and Semi-supervised Learning
Graphs, Geometry and Semi-supervised Learning Mikhail Belkin The Ohio State University, Dept of Computer Science and Engineering and Dept of Statistics Collaborators: Partha Niyogi, Vikas Sindhwani In
More informationData Analysis and Manifold Learning Lecture 3: Graphs, Graph Matrices, and Graph Embeddings
Data Analysis and Manifold Learning Lecture 3: Graphs, Graph Matrices, and Graph Embeddings Radu Horaud INRIA Grenoble Rhone-Alpes, France Radu.Horaud@inrialpes.fr http://perception.inrialpes.fr/ Outline
More informationData-dependent representations: Laplacian Eigenmaps
Data-dependent representations: Laplacian Eigenmaps November 4, 2015 Data Organization and Manifold Learning There are many techniques for Data Organization and Manifold Learning, e.g., Principal Component
More informationFrom graph to manifold Laplacian: The convergence rate
Appl. Comput. Harmon. Anal. 2 (2006) 28 34 www.elsevier.com/locate/acha Letter to the Editor From graph to manifold Laplacian: The convergence rate A. Singer Department of athematics, Yale University,
More informationLearning gradients: prescriptive models
Department of Statistical Science Institute for Genome Sciences & Policy Department of Computer Science Duke University May 11, 2007 Relevant papers Learning Coordinate Covariances via Gradients. Sayan
More informationSturm-Liouville operators have form (given p(x) > 0, q(x)) + q(x), (notation means Lf = (pf ) + qf ) dx
Sturm-Liouville operators Sturm-Liouville operators have form (given p(x) > 0, q(x)) L = d dx ( p(x) d ) + q(x), (notation means Lf = (pf ) + qf ) dx Sturm-Liouville operators Sturm-Liouville operators
More informationBeyond Scalar Affinities for Network Analysis or Vector Diffusion Maps and the Connection Laplacian
Beyond Scalar Affinities for Network Analysis or Vector Diffusion Maps and the Connection Laplacian Amit Singer Princeton University Department of Mathematics and Program in Applied and Computational Mathematics
More informationData Analysis and Manifold Learning Lecture 7: Spectral Clustering
Data Analysis and Manifold Learning Lecture 7: Spectral Clustering Radu Horaud INRIA Grenoble Rhone-Alpes, France Radu.Horaud@inrialpes.fr http://perception.inrialpes.fr/ Outline of Lecture 7 What is spectral
More informationLearning Eigenfunctions: Links with Spectral Clustering and Kernel PCA
Learning Eigenfunctions: Links with Spectral Clustering and Kernel PCA Yoshua Bengio Pascal Vincent Jean-François Paiement University of Montreal April 2, Snowbird Learning 2003 Learning Modal Structures
More informationStable MMPI-2 Scoring: Introduction to Kernel. Extension Techniques
1 Stable MMPI-2 Scoring: Introduction to Kernel Extension Techniques Liberty,E., Almagor,M., Zucker,S., Keller,Y., and Coifman,R.R. Abstract The current study introduces a new technique called Geometric
More informationDiffusion Maps, Spectral Clustering and Eigenfunctions of Fokker-Planck Operators
Diffusion Maps, Spectral Clustering and Eigenfunctions of Fokker-Planck Operators Boaz Nadler Stéphane Lafon Ronald R. Coifman Department of Mathematics, Yale University, New Haven, CT 652. {boaz.nadler,stephane.lafon,ronald.coifman}@yale.edu
More informationMATH 829: Introduction to Data Mining and Analysis Clustering II
his lecture is based on U. von Luxburg, A Tutorial on Spectral Clustering, Statistics and Computing, 17 (4), 2007. MATH 829: Introduction to Data Mining and Analysis Clustering II Dominique Guillot Departments
More informationCertifying the Global Optimality of Graph Cuts via Semidefinite Programming: A Theoretic Guarantee for Spectral Clustering
Certifying the Global Optimality of Graph Cuts via Semidefinite Programming: A Theoretic Guarantee for Spectral Clustering Shuyang Ling Courant Institute of Mathematical Sciences, NYU Aug 13, 2018 Joint
More informationGeometry on Probability Spaces
Geometry on Probability Spaces Steve Smale Toyota Technological Institute at Chicago 427 East 60th Street, Chicago, IL 60637, USA E-mail: smale@math.berkeley.edu Ding-Xuan Zhou Department of Mathematics,
More informationGraph Metrics and Dimension Reduction
Graph Metrics and Dimension Reduction Minh Tang 1 Michael Trosset 2 1 Applied Mathematics and Statistics The Johns Hopkins University 2 Department of Statistics Indiana University, Bloomington November
More informationDiffusion Geometries, Global and Multiscale
Diffusion Geometries, Global and Multiscale R.R. Coifman, S. Lafon, MM, J.C. Bremer Jr., A.D. Szlam, P.W. Jones, R.Schul Papers, talks, other materials available at: www.math.yale.edu/~mmm82 Data and functions
More informationDiffusion Maps, Spectral Clustering and Eigenfunctions of Fokker-Planck Operators
Diffusion Maps, Spectral Clustering and Eigenfunctions of Fokker-Planck Operators Boaz Nadler Stéphane Lafon Ronald R. Coifman Department of Mathematics, Yale University, New Haven, CT 652. {boaz.nadler,stephane.lafon,ronald.coifman}@yale.edu
More informationValue Function Approximation with Diffusion Wavelets and Laplacian Eigenfunctions
Value Function Approximation with Diffusion Wavelets and Laplacian Eigenfunctions Sridhar Mahadevan Department of Computer Science University of Massachusetts Amherst, MA 13 mahadeva@cs.umass.edu Mauro
More informationFiltering via a Reference Set. A.Haddad, D. Kushnir, R.R. Coifman Technical Report YALEU/DCS/TR-1441 February 21, 2011
Patch-based de-noising algorithms and patch manifold smoothing have emerged as efficient de-noising methods. This paper provides a new insight on these methods, such as the Non Local Means or the image
More informationLearning on Graphs and Manifolds. CMPSCI 689 Sridhar Mahadevan U.Mass Amherst
Learning on Graphs and Manifolds CMPSCI 689 Sridhar Mahadevan U.Mass Amherst Outline Manifold learning is a relatively new area of machine learning (2000-now). Main idea Model the underlying geometry of
More informationBi-stochastic kernels via asymmetric affinity functions
Bi-stochastic kernels via asymmetric affinity functions Ronald R. Coifman, Matthew J. Hirn Yale University Department of Mathematics P.O. Box 208283 New Haven, Connecticut 06520-8283 USA ariv:1209.0237v4
More informationTHE HIDDEN CONVEXITY OF SPECTRAL CLUSTERING
THE HIDDEN CONVEXITY OF SPECTRAL CLUSTERING Luis Rademacher, Ohio State University, Computer Science and Engineering. Joint work with Mikhail Belkin and James Voss This talk A new approach to multi-way
More informationDiffusion Wavelets for multiscale analysis on manifolds and graphs: constructions and applications
Diffusion Wavelets for multiscale analysis on manifolds and graphs: constructions and applications Mauro Maggioni EPFL, Lausanne Dec. 19th, 5 EPFL Multiscale Analysis and Diffusion Wavelets - Mauro Maggioni,
More informationDiffusion/Inference geometries of data features, situational awareness and visualization. Ronald R Coifman Mathematics Yale University
Diffusion/Inference geometries of data features, situational awareness and visualization Ronald R Coifman Mathematics Yale University Digital data is generally converted to point clouds in high dimensional
More informationDirected Graph Embedding: an Algorithm based on Continuous Limits of Laplacian-type Operators
Directed Graph Embedding: an Algorithm based on Continuous Limits of Laplacian-type Operators Dominique C. Perrault-Joncas Department of Statistics University of Washington Seattle, WA 98195 dcpj@stat.washington.edu
More informationSpectral Clustering. Spectral Clustering? Two Moons Data. Spectral Clustering Algorithm: Bipartioning. Spectral methods
Spectral Clustering Seungjin Choi Department of Computer Science POSTECH, Korea seungjin@postech.ac.kr 1 Spectral methods Spectral Clustering? Methods using eigenvectors of some matrices Involve eigen-decomposition
More informationSpectral Algorithms II
Spectral Algorithms II Applications Slides based on Spectral Mesh Processing Siggraph 2010 course Applications Shape retrieval Parameterization i 1D 2D Quad meshing Shape Retrieval 3D Repository Query
More informationClustering in kernel embedding spaces and organization of documents
Clustering in kernel embedding spaces and organization of documents Stéphane Lafon Collaborators: Raphy Coifman (Yale), Yosi Keller (Yale), Ioannis G. Kevrekidis (Princeton), Ann B. Lee (CMU), Boaz Nadler
More informationDefinition and basic properties of heat kernels I, An introduction
Definition and basic properties of heat kernels I, An introduction Zhiqin Lu, Department of Mathematics, UC Irvine, Irvine CA 92697 April 23, 2010 In this lecture, we will answer the following questions:
More informationDIMENSION REDUCTION. min. j=1
DIMENSION REDUCTION 1 Principal Component Analysis (PCA) Principal components analysis (PCA) finds low dimensional approximations to the data by projecting the data onto linear subspaces. Let X R d and
More informationSemi-Supervised Learning with the Graph Laplacian: The Limit of Infinite Unlabelled Data
Semi-Supervised Learning with the Graph Laplacian: The Limit of Infinite Unlabelled Data Boaz Nadler Dept. of Computer Science and Applied Mathematics Weizmann Institute of Science Rehovot, Israel 76 boaz.nadler@weizmann.ac.il
More informationAdvances in Manifold Learning Presented by: Naku Nak l Verm r a June 10, 2008
Advances in Manifold Learning Presented by: Nakul Verma June 10, 008 Outline Motivation Manifolds Manifold Learning Random projection of manifolds for dimension reduction Introduction to random projections
More informationSpectral Clustering. Zitao Liu
Spectral Clustering Zitao Liu Agenda Brief Clustering Review Similarity Graph Graph Laplacian Spectral Clustering Algorithm Graph Cut Point of View Random Walk Point of View Perturbation Theory Point of
More informationKernels A Machine Learning Overview
Kernels A Machine Learning Overview S.V.N. Vishy Vishwanathan vishy@axiom.anu.edu.au National ICT of Australia and Australian National University Thanks to Alex Smola, Stéphane Canu, Mike Jordan and Peter
More informationBeyond the Point Cloud: From Transductive to Semi-Supervised Learning
Beyond the Point Cloud: From Transductive to Semi-Supervised Learning Vikas Sindhwani, Partha Niyogi, Mikhail Belkin Andrew B. Goldberg goldberg@cs.wisc.edu Department of Computer Sciences University of
More informationData dependent operators for the spatial-spectral fusion problem
Data dependent operators for the spatial-spectral fusion problem Wien, December 3, 2012 Joint work with: University of Maryland: J. J. Benedetto, J. A. Dobrosotskaya, T. Doster, K. W. Duke, M. Ehler, A.
More informationDIFFUSION MAPS, REDUCTION COORDINATES AND LOW DIMENSIONAL REPRESENTATION OF STOCHASTIC SYSTEMS
DIFFUSION MAPS, REDUCTION COORDINATES AND LOW DIMENSIONAL REPRESENTATION OF STOCHASTIC SYSTEMS R.R. COIFMAN, I.G. KEVREKIDIS, S. LAFON, M. MAGGIONI, AND B. NADLER Abstract. The concise representation of
More informationExploiting Sparse Non-Linear Structure in Astronomical Data
Exploiting Sparse Non-Linear Structure in Astronomical Data Ann B. Lee Department of Statistics and Department of Machine Learning, Carnegie Mellon University Joint work with P. Freeman, C. Schafer, and
More informationNonlinear Dimensionality Reduction
Nonlinear Dimensionality Reduction Piyush Rai CS5350/6350: Machine Learning October 25, 2011 Recap: Linear Dimensionality Reduction Linear Dimensionality Reduction: Based on a linear projection of the
More informationAn Analysis of the Convergence of Graph Laplacians
Daniel Ting Department of Statistics, UC Berkeley Ling Huang Intel Labs Berkeley Michael I. Jordan Department of EECS and Statistics, UC Berkeley dting@stat.berkeley.edu ling.huang@intel.com jordan@cs.berkeley.edu
More informationConference in Honor of Aline Bonami Orleans, June 2014
Conference in Honor of Aline Bonami Orleans, June 2014 Harmonic Analysis and functional duality, as a tool for organization of information, and learning. R. Coifman Department of Mathematics, program of
More informationSolving the 3D Laplace Equation by Meshless Collocation via Harmonic Kernels
Solving the 3D Laplace Equation by Meshless Collocation via Harmonic Kernels Y.C. Hon and R. Schaback April 9, Abstract This paper solves the Laplace equation u = on domains Ω R 3 by meshless collocation
More informationMultiscale bi-harmonic Analysis of Digital Data Bases and Earth moving distances.
Multiscale bi-harmonic Analysis of Digital Data Bases and Earth moving distances. R. Coifman, Department of Mathematics, program of Applied Mathematics Yale University Joint work with M. Gavish and W.
More informationAn Iterated Graph Laplacian Approach for Ranking on Manifolds
An Iterated Graph Laplacian Approach for Ranking on Manifolds Xueyuan Zhou Department of Computer Science University of Chicago Chicago, IL zhouxy@cs.uchicago.edu Mikhail Belkin Department of Computer
More informationLaplacian Eigenmaps for Dimensionality Reduction and Data Representation
Laplacian Eigenmaps for Dimensionality Reduction and Data Representation Neural Computation, June 2003; 15 (6):1373-1396 Presentation for CSE291 sp07 M. Belkin 1 P. Niyogi 2 1 University of Chicago, Department
More informationGeneral Inner Product and The Fourier Series
A Linear Algebra Approach Department of Mathematics University of Puget Sound 4-20-14 / Spring Semester Outline 1 2 Inner Product The inner product is an algebraic operation that takes two vectors and
More informationSpectral Processing. Misha Kazhdan
Spectral Processing Misha Kazhdan [Taubin, 1995] A Signal Processing Approach to Fair Surface Design [Desbrun, et al., 1999] Implicit Fairing of Arbitrary Meshes [Vallet and Levy, 2008] Spectral Geometry
More informationMultiscale Wavelets on Trees, Graphs and High Dimensional Data
Multiscale Wavelets on Trees, Graphs and High Dimensional Data ICML 2010, Haifa Matan Gavish (Weizmann/Stanford) Boaz Nadler (Weizmann) Ronald Coifman (Yale) Boaz Nadler Ronald Coifman Motto... the relationships
More informationFunctional Analysis Review
Outline 9.520: Statistical Learning Theory and Applications February 8, 2010 Outline 1 2 3 4 Vector Space Outline A vector space is a set V with binary operations +: V V V and : R V V such that for all
More informationA Statistical Look at Spectral Graph Analysis. Deep Mukhopadhyay
A Statistical Look at Spectral Graph Analysis Deep Mukhopadhyay Department of Statistics, Temple University Office: Speakman 335 deep@temple.edu http://sites.temple.edu/deepstat/ Graph Signal Processing
More informationMATH 567: Mathematical Techniques in Data Science Clustering II
This lecture is based on U. von Luxburg, A Tutorial on Spectral Clustering, Statistics and Computing, 17 (4), 2007. MATH 567: Mathematical Techniques in Data Science Clustering II Dominique Guillot Departments
More informationDiffeomorphic Warping. Ben Recht August 17, 2006 Joint work with Ali Rahimi (Intel)
Diffeomorphic Warping Ben Recht August 17, 2006 Joint work with Ali Rahimi (Intel) What Manifold Learning Isn t Common features of Manifold Learning Algorithms: 1-1 charting Dense sampling Geometric Assumptions
More informationMeasure-based diffusion grid construction and high-dimensional data discretization
Measure-based diffusion grid construction and high-dimensional data discretization Amit Bermanis, Moshe Salhov, Guy Wolf, Amir Averbuch School of Computer Sciene, Tel Aviv University, Tel Aviv 69978, Israel
More informationGlobal vs. Multiscale Approaches
Harmonic Analysis on Graphs Global vs. Multiscale Approaches Weizmann Institute of Science, Rehovot, Israel July 2011 Joint work with Matan Gavish (WIS/Stanford), Ronald Coifman (Yale), ICML 10' Challenge:
More informationHigh-Dimensional Pattern Recognition using Low-Dimensional Embedding and Earth Mover s Distance
High-Dimensional Pattern Recognition using Low-Dimensional Embedding and Earth Mover s Distance Linh Lieu,a, Naoki Saito a a Department of Mathematics, University of California, Davis, CA 95616, USA Abstract
More informationThe Laplacian PDF Distance: A Cost Function for Clustering in a Kernel Feature Space
The Laplacian PDF Distance: A Cost Function for Clustering in a Kernel Feature Space Robert Jenssen, Deniz Erdogmus 2, Jose Principe 2, Torbjørn Eltoft Department of Physics, University of Tromsø, Norway
More informationBasic Calculus Review
Basic Calculus Review Lorenzo Rosasco ISML Mod. 2 - Machine Learning Vector Spaces Functionals and Operators (Matrices) Vector Space A vector space is a set V with binary operations +: V V V and : R V
More informationFast Direct Policy Evaluation using Multiscale Analysis of Markov Diffusion Processes
Fast Direct Policy Evaluation using Multiscale Analysis of Markov Diffusion Processes Mauro Maggioni mauro.maggioni@yale.edu Department of Mathematics, Yale University, P.O. Box 88, New Haven, CT,, U.S.A.
More informationEECS 275 Matrix Computation
EECS 275 Matrix Computation Ming-Hsuan Yang Electrical Engineering and Computer Science University of California at Merced Merced, CA 95344 http://faculty.ucmerced.edu/mhyang Lecture 23 1 / 27 Overview
More information8.1 Concentration inequality for Gaussian random matrix (cont d)
MGMT 69: Topics in High-dimensional Data Analysis Falll 26 Lecture 8: Spectral clustering and Laplacian matrices Lecturer: Jiaming Xu Scribe: Hyun-Ju Oh and Taotao He, October 4, 26 Outline Concentration
More informationLimits of Spectral Clustering
Limits of Spectral Clustering Ulrike von Luxburg and Olivier Bousquet Max Planck Institute for Biological Cybernetics Spemannstr. 38, 72076 Tübingen, Germany {ulrike.luxburg,olivier.bousquet}@tuebingen.mpg.de
More informationUnsupervised dimensionality reduction
Unsupervised dimensionality reduction Guillaume Obozinski Ecole des Ponts - ParisTech SOCN course 2014 Guillaume Obozinski Unsupervised dimensionality reduction 1/30 Outline 1 PCA 2 Kernel PCA 3 Multidimensional
More informationKernel methods for comparing distributions, measuring dependence
Kernel methods for comparing distributions, measuring dependence Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012 Principal component analysis Given a set of M centered observations
More informationProto-value Functions: A Laplacian Framework for Learning Representation and Control in Markov Decision Processes
Proto-value Functions: A Laplacian Framework for Learning Representation and Control in Markov Decision Processes Sridhar Mahadevan Department of Computer Science University of Massachusetts Amherst, MA
More informationManifold Coarse Graining for Online Semi-supervised Learning
for Online Semi-supervised Learning Mehrdad Farajtabar, Amirreza Shaban, Hamid R. Rabiee, Mohammad H. Rohban Digital Media Lab, Department of Computer Engineering, Sharif University of Technology, Tehran,
More informationA Multiscale Framework for Markov Decision Processes using Diffusion Wavelets
A Multiscale Framework for Markov Decision Processes using Diffusion Wavelets Mauro Maggioni Program in Applied Mathematics Department of Mathematics Yale University New Haven, CT 6 mauro.maggioni@yale.edu
More informationJustin Solomon MIT, Spring 2017
Justin Solomon MIT, Spring 2017 http://pngimg.com/upload/hammer_png3886.png You can learn a lot about a shape by hitting it (lightly) with a hammer! What can you learn about its shape from vibration frequencies
More informationLocality Preserving Projections
Locality Preserving Projections Xiaofei He Department of Computer Science The University of Chicago Chicago, IL 60637 xiaofei@cs.uchicago.edu Partha Niyogi Department of Computer Science The University
More informationc 4, < y 2, 1 0, otherwise,
Fundamentals of Big Data Analytics Univ.-Prof. Dr. rer. nat. Rudolf Mathar Problem. Probability theory: The outcome of an experiment is described by three events A, B and C. The probabilities Pr(A) =,
More informationMultiscale Manifold Learning
Multiscale Manifold Learning Chang Wang IBM T J Watson Research Lab Kitchawan Rd Yorktown Heights, New York 598 wangchan@usibmcom Sridhar Mahadevan Computer Science Department University of Massachusetts
More informationThe crucial role of statistics in manifold learning
The crucial role of statistics in manifold learning Tyrus Berry Postdoc, Dept. of Mathematical Sciences, GMU Statistics Seminar GMU Feb., 26 Postdoctoral position supported by NSF ANALYSIS OF POINT CLOUDS
More informationHarmonic Analysis and Geometries of Digital Data Bases
Harmonic Analysis and Geometries of Digital Data Bases AMS Session Special Sesson on the Mathematics of Information and Knowledge, Ronald Coifman (Yale) and Matan Gavish (Stanford, Yale) January 14, 2010
More informationDiffusion maps, spectral clustering and reaction coordinates of dynamical systems
Appl. Comput. Harmon. Anal. 21 (2006) 113 127 www.elsevier.com/locate/acha Diffusion maps, spectral clustering and reaction coordinates of dynamical systems Boaz Nadler a,, Stéphane Lafon a,1, Ronald R.
More informationACM/CMS 107 Linear Analysis & Applications Fall 2017 Assignment 2: PDEs and Finite Element Methods Due: 7th November 2017
ACM/CMS 17 Linear Analysis & Applications Fall 217 Assignment 2: PDEs and Finite Element Methods Due: 7th November 217 For this assignment the following MATLAB code will be required: Introduction http://wwwmdunloporg/cms17/assignment2zip
More informationUnsupervised Learning Techniques Class 07, 1 March 2006 Andrea Caponnetto
Unsupervised Learning Techniques 9.520 Class 07, 1 March 2006 Andrea Caponnetto About this class Goal To introduce some methods for unsupervised learning: Gaussian Mixtures, K-Means, ISOMAP, HLLE, Laplacian
More informationLaplacian Agent Learning: Representation Policy Iteration
Laplacian Agent Learning: Representation Policy Iteration Sridhar Mahadevan Example of a Markov Decision Process a1: $0 Heaven $1 Earth What should the agent do? a2: $100 Hell $-1 V a1 ( Earth ) = f(0,1,1,1,1,...)
More informationSolutions of Semilinear Elliptic PDEs on Manifolds
Solutions of Semilinear Elliptic PDEs on Manifolds Jeff, Northern Arizona University 1 2 3 4 5 What is a PDE? Definition A Partial Differential Equation (PDE) is a relation involving an unknown function
More informationLaplace-Beltrami Eigenfunctions for Deformation Invariant Shape Representation
Laplace-Beltrami Eigenfunctions for Deformation Invariant Shape Representation Author: Raif M. Rustamov Presenter: Dan Abretske Johns Hopkins 2007 Outline Motivation and Background Laplace-Beltrami Operator
More informationSpectral Algorithms I. Slides based on Spectral Mesh Processing Siggraph 2010 course
Spectral Algorithms I Slides based on Spectral Mesh Processing Siggraph 2010 course Why Spectral? A different way to look at functions on a domain Why Spectral? Better representations lead to simpler solutions
More informationLecture: Some Practical Considerations (3 of 4)
Stat260/CS294: Spectral Graph Methods Lecture 14-03/10/2015 Lecture: Some Practical Considerations (3 of 4) Lecturer: Michael Mahoney Scribe: Michael Mahoney Warning: these notes are still very rough.
More informationFace Recognition Using Laplacianfaces He et al. (IEEE Trans PAMI, 2005) presented by Hassan A. Kingravi
Face Recognition Using Laplacianfaces He et al. (IEEE Trans PAMI, 2005) presented by Hassan A. Kingravi Overview Introduction Linear Methods for Dimensionality Reduction Nonlinear Methods and Manifold
More informationPARAMETERIZATION OF NON-LINEAR MANIFOLDS
PARAMETERIZATION OF NON-LINEAR MANIFOLDS C. W. GEAR DEPARTMENT OF CHEMICAL AND BIOLOGICAL ENGINEERING PRINCETON UNIVERSITY, PRINCETON, NJ E-MAIL:WGEAR@PRINCETON.EDU Abstract. In this report we consider
More informationMultiscale Analysis and Diffusion Semigroups With Applications
Multiscale Analysis and Diffusion Semigroups With Applications Karamatou Yacoubou Djima Advisor: Wojciech Czaja Norbert Wiener Center Department of Mathematics University of Maryland, College Park http://www.norbertwiener.umd.edu
More informationThe spectral zeta function
The spectral zeta function Bernd Ammann June 4, 215 Abstract In this talk we introduce spectral zeta functions. The spectral zeta function of the Laplace-Beltrami operator was already introduced by Minakshisundaram
More informationSpectral clustering. Two ideal clusters, with two points each. Spectral clustering algorithms
A simple example Two ideal clusters, with two points each Spectral clustering Lecture 2 Spectral clustering algorithms 4 2 3 A = Ideally permuted Ideal affinities 2 Indicator vectors Each cluster has an
More informationWaves on 2 and 3 dimensional domains
Chapter 14 Waves on 2 and 3 dimensional domains We now turn to the studying the initial boundary value problem for the wave equation in two and three dimensions. In this chapter we focus on the situation
More informationMetric Learning on Manifolds
Journal of Machine Learning Research 0 (2011) 0-00 Submitted 0/00; Published 00/00 Metric Learning on Manifolds Dominique Perrault-Joncas Department of Statistics University of Washington Seattle, WA 98195-4322,
More informationRegression on Manifolds Using Kernel Dimension Reduction
Jens Nilsson JENSN@MATHS.LTH.SE Centre for Mathematical Sciences, Lund University, Box 118, SE-221 00 Lund, Sweden Fei Sha FEISHA@CS.BERKELEY.EDU Computer Science Division, University of California, Berkeley,
More informationReproducing Kernel Hilbert Spaces
Reproducing Kernel Hilbert Spaces Lorenzo Rosasco 9.520 Class 03 February 11, 2009 About this class Goal To introduce a particularly useful family of hypothesis spaces called Reproducing Kernel Hilbert
More informationMATH 567: Mathematical Techniques in Data Science Clustering II
Spectral clustering: overview MATH 567: Mathematical Techniques in Data Science Clustering II Dominique uillot Departments of Mathematical Sciences University of Delaware Overview of spectral clustering:
More informationGraph Partitioning Using Random Walks
Graph Partitioning Using Random Walks A Convex Optimization Perspective Lorenzo Orecchia Computer Science Why Spectral Algorithms for Graph Problems in practice? Simple to implement Can exploit very efficient
More informationNon-linear Dimensionality Reduction
Non-linear Dimensionality Reduction CE-725: Statistical Pattern Recognition Sharif University of Technology Spring 2013 Soleymani Outline Introduction Laplacian Eigenmaps Locally Linear Embedding (LLE)
More informationAnalysis Preliminary Exam Workshop: Hilbert Spaces
Analysis Preliminary Exam Workshop: Hilbert Spaces 1. Hilbert spaces A Hilbert space H is a complete real or complex inner product space. Consider complex Hilbert spaces for definiteness. If (, ) : H H
More informationCSE 291. Assignment Spectral clustering versus k-means. Out: Wed May 23 Due: Wed Jun 13
CSE 291. Assignment 3 Out: Wed May 23 Due: Wed Jun 13 3.1 Spectral clustering versus k-means Download the rings data set for this problem from the course web site. The data is stored in MATLAB format as
More informationThe Laplacian ( ) Matthias Vestner Dr. Emanuele Rodolà Room , Informatik IX
The Laplacian (26.05.2014) Matthias Vestner Dr. Emanuele Rodolà {vestner,rodola}@in.tum.de Room 02.09.058, Informatik IX Seminar «The metric approach to shape matching» Alfonso Ros Wednesday, May 28th
More informationManifold Regularization
9.520: Statistical Learning Theory and Applications arch 3rd, 200 anifold Regularization Lecturer: Lorenzo Rosasco Scribe: Hooyoung Chung Introduction In this lecture we introduce a class of learning algorithms,
More informationSpectral Techniques for Clustering
Nicola Rebagliati 1/54 Spectral Techniques for Clustering Nicola Rebagliati 29 April, 2010 Nicola Rebagliati 2/54 Thesis Outline 1 2 Data Representation for Clustering Setting Data Representation and Methods
More information