Two-sample hypothesis testing for random dot product graphs
|
|
- Austin Wood
- 5 years ago
- Views:
Transcription
1 Two-sample hypothesis testing for random dot product graphs Minh Tang Department of Applied Mathematics and Statistics Johns Hopkins University JSM 2014 Joint work with Avanti Athreya, Vince Lyzinski, Carey E. Priebe and Daniel L. Sussman.
2 Introduction and Overview 1 The problem of deciding whether two give graphs are the same has applications in e.g., neuroscience, social networks. 2 We propose a valid and consistent test for the above under a random graph model. 3 The test proceeds by embedding the graphs into Euclidean space followed by computing a distance between a kernel density estimate of the embedded points.
3 Random dot product graphs Let Ω be a subset of R d such that, for all ω, ω Ω, 0 ω, ω 1. Let F be a distribution taking values in Ω. 1 Let {X i } n i=1 i.i.d F. 2 A n RDPG(F) is the adjacency matrix of a graph associated with {X i } n i=1. The upper diagonal entries of A n are independent Bernoulli random variables with P[X i X j ] = X i, X j, i.e., P[A n {X i } n i=1] = i<j X i, X j A n(i,j) (1 X i, X j ) 1 A n(i,j) See Young and Scheinerman (2007).
4 Random dot product graphs are an example of latent position graphs (Hoff et al., 2002), in which each vertex is associated with a latent position. Random dot product graphs are related to stochastic blockmodels Holland et al. (1983), degree-corrected stochastic block models Karrer and Newman (2011), and mixed membership block models Airoldi et al. (2008). Non-identifiability: For any distribution F and orthogonal matrix W, the graphs A RDPG(F) and B RDPG(F W) are identically distributed.
5 X = {X i } n i=1 Rd P = XX T [0, 1] n n A = Bern(K) original latent vectors probability matrix adjacency matrix Observation A looks like P (at least at rough scale).
6 Problem Statement Given A RDPG(F) and B RDPG(G), consider the following test: H 0 : F = W G against H 1 : F W G where F = W G denotes that there exists an orthogonal d d matrix W such that F = G W and F W G denotes that F G W for any orthogonal W.
7 Adjacency spectral embedding Definition Let A be an n n adjacency matrix and denote by A the matrix (A T A) 1/2. Let d 1 and consider the following spectral decomposition of A A = [U A Ũ A ][S A S A ][U A Ũ A ] where U A R n d, ŨA R n d. The columns of U A correspond to the d largest eigenvalues of A. The adjacency spectral embedding of A into R d is then the n d matrix ˆX = U A S 1/2 A.
8 ˆX is close to X 1.0 X 1.0 ˆX Residuals
9 Modicum of consistency I Theorem Suppose (A, X) RDPG(F) is a graph on n vertices. Denote by ˆX the adjacency spectral embedding of A into R d. Let η > 0 be arbitrary. Then for sufficiently large n there exists a d d orthogonal matrix W such that, with probability at least 1 3η, ˆX XW F C 1 (F) C 2(F)d 3/2 log (n/η) (1) n where C 1 (F) and C 2 (F) are constants depending only on F.
10 Two-sample testing via maximum mean discrepancy Let κ be a kernel on Ω with reproducing kernel Hilbert space H. Denote by F the unit ball F = {h H : h H 1}. For a distribution F taking values in Ω the map µ[f] defined by µ[f] := κ(ω, ) df(ω). Ω belongs to H. If κ is a universal kernel, then µ is an injective map. Let F and G be probability distributions taking values in Ω; X, X F and Y, Y G. Then µ[f] µ[g] 2 H = sup E F [h] E G [h] 2 h F = E[κ(X, X )] 2E[κ(X, Y)] + E[κ(Y, Y )]. (2) is an integral probability metric, termed the maximum mean discrepancy Gretton et al. (2012).
11 Denote by Φ: Ω H the canonical feature map Φ(X) = κ(, X) of κ. Given {X i } i.i.d F and {Y i } i.i.d G, the quantity V n,m (X, Y) V n,m (X, Y) = 1 n n Φ(X i ) 1 m i=1 = 1 n 2 n n i=1 j=1 + 1 m 2 m k=1 l=1 n Φ(Y k ) k=1 κ(x i, X j ) 2 mn m κ(y k, Y l ). is a consistent estimate of µ[f] µ[g] 2 H. m 2 H i=1 k=1 n κ(x i, Y k )
12 Test statistic Denote by ˆX = { ˆX 1,..., ˆX n } and Ŷ = {Ŷ 1,..., Ŷ m } the adjacency spectral embedding of A and B, respectively. Assume that κ is a unitarily invariant kernel, e.g., a radial kernel. Define the test statistic V n,m (ˆX, Ŷ) as follows: V n,m (ˆX, Ŷ) = 1 n 2 n n i=1 j=1 κ( ˆX i, ˆX j ) 2 mn n + 1 m 2 m i=1 k=1 l=1 k=1 m κ( ˆX i, Ŷ k ) m κ(ŷ k, Ŷ l )
13 Modicum of consistency II Theorem Let (X, A) RDPG(F) and (Y, B) RDPG(G) be independent random dot product graphs with latent position distributions F and G satisfying distinct eigenvalues assumption. Consider the hypothesis test H 0 : F = W G against H 1 : F W G Suppose m, n and m/(m + n) ρ (0, 1). Then under the null (m + n)(v n,m (ˆX, Ŷ) V n,m (X, YW)) a.s. 0 (3) where W is any orthogonal matrix such that F = G W.
14 Sketch of argument Eq. (3) that (m + n)(v n,m (ˆX, Ŷ) V n,m(x, YW)) a.s. 0 follows from the following bound sup 1 n ( f(w ˆX i ) f(x i ) ) a.s. 0 n f F i=1 established via Taylor s expansion and a covering number argument.
15 Limiting distribution of V n,m ( ˆX, Ŷ). Hence under the null hypothesis of F = W G, evoking previous results of Anderson et al. (1994) and Gretton et al. (2012) for V n,m (X, Y), one has (m + n)v n,m (ˆX, Ŷ) d 1 ρ(1 ρ) λ l χ 2 1l (4) where {χ 2 1l } are independent χ2 random variables with one degree of freedom and {λ l } are the eigenvalues of the integral operator I F, κ (φ) = φ(y) κ(x, y)df(y) Ω l=1
16 Simulation Results n = 1000 n = density density type estimated positions true positions Test statistic under null Test statistic under alternative Figure 1 : Distribution of test statistics under null and alternative as computed from the latent positions and those estimated from adjacency spectral embedding for testing the null hypothesis F = W G.
17 ɛ = 0.02 ɛ = 0.05 ɛ = 0.1 n {X, Y} {ˆX, Ŷ} {X, Y} {ˆX, Ŷ} {X, Y} {ˆX, Ŷ} Table 1 : Power estimates for testing the null hypothesis F = W G at a significance level of α = 0.05 using bootstrap permutation tests for V n,m (ˆX, Ŷ) and V n,m (X, Y). In each bootstrap test, B = 200 bootstrap samples were generated. Each estimate of power is based on 1000 Monte Carlo replicates of the corresponding bootstrap test.
18 References I E. M. Airoldi, D. M. Blei, S. E. Fienberg, and E. P. Xing. Mixed membership stochastic blockmodels. The Journal of Machine Learning Research, 9: , N. Anderson, P. Hall, and D. Titterington. Two-sample test statistics for measuring discrepancies between two multivariate probability density functions using kernel-based density estimates. Journal of Multivariate Analysis, 50:41 54, V. Alba Fernández, M. D. Jiménez Gamero, and J. Muñoz García. A test for the two-sample problem based on empirical characteristic functions. Computational Statistics and Data Analysis, 52: , A. Gretton, K. M. Borgwadt, M. J. Rasch, B. Schölkopf, and A. Smola. A kernel two-sample test. Journal of Machine Learning Research, 13: , P. Hall, F. Lombard, and C. J. Potgieter. A new approach to function-based hypothesis testing in location-scale families. Technometrics, 55: , 2013.
19 References II P. D. Hoff, A. E. Raftery, and M. S. Handcock. Latent space approaches to social network analysis. Journal of the American Statistical Association, 97(460): , P. W. Holland, K. Laskey, and S. Leinhardt. Stochastic blockmodels: First steps. Social Networks, 5(2): , Brian Karrer and M. E. J. Newman. Stochastic blockmodels and community structure in networks. Physical Review E, 83:016107, 10, S. Young and E. Scheinerman. Random dot product graph models for social networks. In Proceedings of the 5th international conference on algorithms and models for the web-graph, pages , 2007.
A central limit theorem for an omnibus embedding of random dot product graphs
A central limit theorem for an omnibus embedding of random dot product graphs Keith Levin 1 with Avanti Athreya 2, Minh Tang 2, Vince Lyzinski 3 and Carey E. Priebe 2 1 University of Michigan, 2 Johns
More informationarxiv: v1 [stat.ml] 29 Jul 2012
arxiv:1207.6745v1 [stat.ml] 29 Jul 2012 Universally Consistent Latent Position Estimation and Vertex Classification for Random Dot Product Graphs Daniel L. Sussman, Minh Tang, Carey E. Priebe Johns Hopkins
More informationarxiv: v1 [stat.me] 12 May 2017
Consistency of adjacency spectral embedding for the mixed membership stochastic blockmodel Patrick Rubin-Delanchy *, Carey E. Priebe **, and Minh Tang ** * University of Oxford and Heilbronn Institute
More informationA limit theorem for scaled eigenvectors of random dot product graphs
Sankhya A manuscript No. (will be inserted by the editor A limit theorem for scaled eigenvectors of random dot product graphs A. Athreya V. Lyzinski C. E. Priebe D. L. Sussman M. Tang D.J. Marchette the
More informationMixed Membership Stochastic Blockmodels
Mixed Membership Stochastic Blockmodels Journal of Machine Learning Research, 2008 by E.M. Airoldi, D.M. Blei, S.E. Fienberg, E.P. Xing as interpreted by Ted Westling STAT 572 Intro Talk April 22, 2014
More informationManifold Learning for Subsequent Inference
Manifold Learning for Subsequent Inference Carey E. Priebe Johns Hopkins University June 20, 2018 DARPA Fundamental Limits of Learning (FunLoL) Los Angeles, California http://arxiv.org/abs/1806.01401 Key
More informationStatistical Inference on Random Dot Product Graphs: a Survey
Journal of Machine Learning Research 8 (8) -9 Submitted 8/7; Revised 8/7; Published 5/8 Statistical Inference on Random Dot Product Graphs: a Survey Avanti Athreya Donniell E. Fishkind Minh Tang Carey
More informationMixed Membership Stochastic Blockmodels
Mixed Membership Stochastic Blockmodels (2008) Edoardo M. Airoldi, David M. Blei, Stephen E. Fienberg and Eric P. Xing Herrissa Lamothe Princeton University Herrissa Lamothe (Princeton University) Mixed
More informationOut-of-sample extension of graph adjacency spectral embedding
Keith Levin 1 Farbod Roosta-Khorasani 2 3 Michael W. Mahoney 3 4 Carey E. Priebe 5 Abstract Many popular dimensionality reduction procedures have out-of-sample extensions, which allow a practitioner to
More informationFoundations of Adjacency Spectral Embedding. Daniel L. Sussman
Foundations of Adjacency Spectral Embedding by Daniel L. Sussman A dissertation submitted to The Johns Hopkins University in conformity with the requirements for the degree of Doctor of Philosophy. Baltimore,
More informationEmpirical Bayes estimation for the stochastic blockmodel
Electronic Journal of Statistics Vol. 10 (2016) 761 782 ISSN: 1935-7524 DOI: 10.1214/16-EJS1115 Empirical Bayes estimation for the stochastic blockmodel Shakira Suwan,DominicS.Lee Department of Mathematics
More informationKernel Learning via Random Fourier Representations
Kernel Learning via Random Fourier Representations L. Law, M. Mider, X. Miscouridou, S. Ip, A. Wang Module 5: Machine Learning L. Law, M. Mider, X. Miscouridou, S. Ip, A. Wang Kernel Learning via Random
More informationNetwork Representation Using Graph Root Distributions
Network Representation Using Graph Root Distributions Jing Lei Department of Statistics and Data Science Carnegie Mellon University 2018.04 Network Data Network data record interactions (edges) between
More informationKernel methods for comparing distributions, measuring dependence
Kernel methods for comparing distributions, measuring dependence Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012 Principal component analysis Given a set of M centered observations
More informationKernel Methods. Lecture 4: Maximum Mean Discrepancy Thanks to Karsten Borgwardt, Malte Rasch, Bernhard Schölkopf, Jiayuan Huang, Arthur Gretton
Kernel Methods Lecture 4: Maximum Mean Discrepancy Thanks to Karsten Borgwardt, Malte Rasch, Bernhard Schölkopf, Jiayuan Huang, Arthur Gretton Alexander J. Smola Statistical Machine Learning Program Canberra,
More information26 : Spectral GMs. Lecturer: Eric P. Xing Scribes: Guillermo A Cidre, Abelino Jimenez G.
10-708: Probabilistic Graphical Models, Spring 2015 26 : Spectral GMs Lecturer: Eric P. Xing Scribes: Guillermo A Cidre, Abelino Jimenez G. 1 Introduction A common task in machine learning is to work with
More informationHilbert Space Representations of Probability Distributions
Hilbert Space Representations of Probability Distributions Arthur Gretton joint work with Karsten Borgwardt, Kenji Fukumizu, Malte Rasch, Bernhard Schölkopf, Alex Smola, Le Song, Choon Hui Teo Max Planck
More informationAttribute fusion in a latent process model for time series of graphs
1 Attribute fusion in a latent process model for time series of graphs Minh Tang, Youngser Park, Nam H. Lee, and Carey E. Priebe* Abstract Hypothesis testing on time series of attributed graphs has applications
More informationVertex Nomination via Attributed Random Dot Product Graphs
Vertex Nomination via Attributed Random Dot Product Graphs Carey E. Priebe cep@jhu.edu Department of Applied Mathematics & Statistics Johns Hopkins University 58th Session of the International Statistical
More informationRandom function priors for exchangeable arrays with applications to graphs and relational data
Random function priors for exchangeable arrays with applications to graphs and relational data James Robert Lloyd Department of Engineering University of Cambridge Peter Orbanz Department of Statistics
More informationRandom Feature Maps for Dot Product Kernels Supplementary Material
Random Feature Maps for Dot Product Kernels Supplementary Material Purushottam Kar and Harish Karnick Indian Institute of Technology Kanpur, INDIA {purushot,hk}@cse.iitk.ac.in Abstract This document contains
More informationAsymptotic Statistics-VI. Changliang Zou
Asymptotic Statistics-VI Changliang Zou Kolmogorov-Smirnov distance Example (Kolmogorov-Smirnov confidence intervals) We know given α (0, 1), there is a well-defined d = d α,n such that, for any continuous
More informationEECS 598: Statistical Learning Theory, Winter 2014 Topic 11. Kernels
EECS 598: Statistical Learning Theory, Winter 2014 Topic 11 Kernels Lecturer: Clayton Scott Scribe: Jun Guo, Soumik Chatterjee Disclaimer: These notes have not been subjected to the usual scrutiny reserved
More informationMixed Membership Stochastic Blockmodels
Mixed Membership Stochastic Blockmodels Journal of Machine Learning Research, 2008 by E.M. Airoldi, D.M. Blei, S.E. Fienberg, E.P. Xing as interpreted by Ted Westling STAT 572 Final Talk May 8, 2014 Ted
More informationKernel Method: Data Analysis with Positive Definite Kernels
Kernel Method: Data Analysis with Positive Definite Kernels 2. Positive Definite Kernel and Reproducing Kernel Hilbert Space Kenji Fukumizu The Institute of Statistical Mathematics. Graduate University
More informationOn the inverse matrix of the Laplacian and all ones matrix
On the inverse matrix of the Laplacian and all ones matrix Sho Suda (Joint work with Michio Seto and Tetsuji Taniguchi) International Christian University JSPS Research Fellow PD November 21, 2012 Sho
More informationarxiv: v1 [cs.lg] 22 Jun 2009
Bayesian two-sample tests arxiv:0906.4032v1 [cs.lg] 22 Jun 2009 Karsten M. Borgwardt 1 and Zoubin Ghahramani 2 1 Max-Planck-Institutes Tübingen, 2 University of Cambridge June 22, 2009 Abstract In this
More informationMaximum Mean Discrepancy
Maximum Mean Discrepancy Thanks to Karsten Borgwardt, Malte Rasch, Bernhard Schölkopf, Jiayuan Huang, Arthur Gretton Alexander J. Smola Statistical Machine Learning Program Canberra, ACT 0200 Australia
More informationConvergence Rates of Kernel Quadrature Rules
Convergence Rates of Kernel Quadrature Rules Francis Bach INRIA - Ecole Normale Supérieure, Paris, France ÉCOLE NORMALE SUPÉRIEURE NIPS workshop on probabilistic integration - Dec. 2015 Outline Introduction
More informationConnection of Local Linear Embedding, ISOMAP, and Kernel Principal Component Analysis
Connection of Local Linear Embedding, ISOMAP, and Kernel Principal Component Analysis Alvina Goh Vision Reading Group 13 October 2005 Connection of Local Linear Embedding, ISOMAP, and Kernel Principal
More informationarxiv: v1 [stat.me] 6 Nov 2014
Network Cross-Validation for Determining the Number of Communities in Network Data Kehui Chen 1 and Jing Lei arxiv:1411.1715v1 [stat.me] 6 Nov 014 1 Department of Statistics, University of Pittsburgh Department
More informationOn Spectral Graph Clustering
On Spectral Graph Clustering Carey E. Priebe Johns Hopkins University May 18, 2018 Symposium on Data Science and Statistics Reston, Virginia Minh Tang http://arxiv.org/abs/1607.08601 Annals of Statistics,
More informationarxiv: v3 [stat.ml] 29 Jul 2018
A statistical interpretation of spectral embedding: the generalised random dot product graph Patrick Rubin-Delanchy *, Carey E. Priebe **, Minh Tang **, and Joshua Cape ** * University of Bristol and Heilbronn
More informationLecture 6 Sept Data Visualization STAT 442 / 890, CM 462
Lecture 6 Sept. 25-2006 Data Visualization STAT 442 / 890, CM 462 Lecture: Ali Ghodsi 1 Dual PCA It turns out that the singular value decomposition also allows us to formulate the principle components
More informationKernels A Machine Learning Overview
Kernels A Machine Learning Overview S.V.N. Vishy Vishwanathan vishy@axiom.anu.edu.au National ICT of Australia and Australian National University Thanks to Alex Smola, Stéphane Canu, Mike Jordan and Peter
More informationHypothesis Testing For Multilayer Network Data
Hypothesis Testing For Multilayer Network Data Jun Li Dept of Mathematics and Statistics, Boston University Joint work with Eric Kolaczyk Outline Background and Motivation Geometric structure of multilayer
More informationGraph Metrics and Dimension Reduction
Graph Metrics and Dimension Reduction Minh Tang 1 Michael Trosset 2 1 Applied Mathematics and Statistics The Johns Hopkins University 2 Department of Statistics Indiana University, Bloomington November
More informationCLUSTERING over graphs is a classical problem with
Maximum Likelihood Latent Space Embedding of Logistic Random Dot Product Graphs Luke O Connor, Muriel Médard and Soheil Feizi ariv:5.85v3 [stat.ml] 3 Aug 27 Abstract A latent space model for a family of
More informationLinear Algebra Massoud Malek
CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product
More informationEcon 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines
Econ 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines Maximilian Kasy Department of Economics, Harvard University 1 / 37 Agenda 6 equivalent representations of the
More informationInt. Statistical Inst.: Proc. 58th World Statistical Congress, 2011, Dublin (Session CPS014) p.4149
Int. Statistical Inst.: Proc. 58th orld Statistical Congress, 011, Dublin (Session CPS014) p.4149 Invariant heory for Hypothesis esting on Graphs Priebe, Carey Johns Hopkins University, Applied Mathematics
More informationSupplementary Material for: Spectral Unsupervised Parsing with Additive Tree Metrics
Supplementary Material for: Spectral Unsupervised Parsing with Additive Tree Metrics Ankur P. Parikh School of Computer Science Carnegie Mellon University apparikh@cs.cmu.edu Shay B. Cohen School of Informatics
More informationApproximate Kernel PCA with Random Features
Approximate Kernel PCA with Random Features (Computational vs. Statistical Tradeoff) Bharath K. Sriperumbudur Department of Statistics, Pennsylvania State University Journées de Statistique Paris May 28,
More informationA Random Dot Product Model for Weighted Networks arxiv: v1 [stat.ap] 8 Nov 2016
A Random Dot Product Model for Weighted Networks arxiv:1611.02530v1 [stat.ap] 8 Nov 2016 Daryl R. DeFord 1 Daniel N. Rockmore 1,2,3 1 Department of Mathematics, Dartmouth College, Hanover, NH, USA 03755
More informationarxiv: v1 [math.st] 27 Feb 2018
Network Representation Using Graph Root Distributions Jing Lei 1 arxiv:1802.09684v1 [math.st] 27 Feb 2018 1 Carnegie Mellon University February 28, 2018 Abstract Exchangeable random graphs serve as an
More informationMP463 QUANTUM MECHANICS
MP463 QUANTUM MECHANICS Introduction Quantum theory of angular momentum Quantum theory of a particle in a central potential - Hydrogen atom - Three-dimensional isotropic harmonic oscillator (a model of
More informationOnline Gradient Descent Learning Algorithms
DISI, Genova, December 2006 Online Gradient Descent Learning Algorithms Yiming Ying (joint work with Massimiliano Pontil) Department of Computer Science, University College London Introduction Outline
More informationManny Parzen, Cherished Teacher And a Tale of Two Kernels. Grace Wahba
Manny Parzen, Cherished Teacher And a Tale of Two Kernels Grace Wahba From Emanuel Parzen and a Tale of Two Kernels Memorial Session for Manny Parzen Joint Statistical Meetings Baltimore, MD August 2,
More informationFunctional Analysis Review
Outline 9.520: Statistical Learning Theory and Applications February 8, 2010 Outline 1 2 3 4 Vector Space Outline A vector space is a set V with binary operations +: V V V and : R V V such that for all
More informationCertifying the Global Optimality of Graph Cuts via Semidefinite Programming: A Theoretic Guarantee for Spectral Clustering
Certifying the Global Optimality of Graph Cuts via Semidefinite Programming: A Theoretic Guarantee for Spectral Clustering Shuyang Ling Courant Institute of Mathematical Sciences, NYU Aug 13, 2018 Joint
More informationHypercontractivity of spherical averages in Hamming space
Hypercontractivity of spherical averages in Hamming space Yury Polyanskiy Department of EECS MIT yp@mit.edu Allerton Conference, Oct 2, 2014 Yury Polyanskiy Hypercontractivity in Hamming space 1 Hypercontractivity:
More informationPractical Methods for Graph Two-Sample Testing
Practical Methods for Graph Two-Sample Testing Debarghya Ghoshdastidar Department of Computer Science University of Tübingen ghoshdas@informatik.uni-tuebingen.de Ulrike von Luxburg Department of Computer
More informationNonparametric Tests for Multi-parameter M-estimators
Nonparametric Tests for Multi-parameter M-estimators John Robinson School of Mathematics and Statistics University of Sydney The talk is based on joint work with John Kolassa. It follows from work over
More informationHypothesis Testing with Kernel Embeddings on Interdependent Data
Hypothesis Testing with Kernel Embeddings on Interdependent Data Dino Sejdinovic Department of Statistics University of Oxford joint work with Kacper Chwialkowski and Arthur Gretton (Gatsby Unit, UCL)
More informationConvergence of Eigenspaces in Kernel Principal Component Analysis
Convergence of Eigenspaces in Kernel Principal Component Analysis Shixin Wang Advanced machine learning April 19, 2016 Shixin Wang Convergence of Eigenspaces April 19, 2016 1 / 18 Outline 1 Motivation
More informationA homogeneity test for spatial point patterns
A homogeneity test for spatial point patterns M.V. Alba-Fernández University of Jaén Paraje las lagunillas, s/n B3-053, 23071, Jaén, Spain mvalba@ujaen.es F. J. Ariza-López University of Jaén Paraje las
More informationSTAT 518 Intro Student Presentation
STAT 518 Intro Student Presentation Wen Wei Loh April 11, 2013 Title of paper Radford M. Neal [1999] Bayesian Statistics, 6: 475-501, 1999 What the paper is about Regression and Classification Flexible
More informationReview. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Review DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Probability and statistics Probability: Framework for dealing with
More informationRKHS, Mercer s theorem, Unbounded domains, Frames and Wavelets Class 22, 2004 Tomaso Poggio and Sayan Mukherjee
RKHS, Mercer s theorem, Unbounded domains, Frames and Wavelets 9.520 Class 22, 2004 Tomaso Poggio and Sayan Mukherjee About this class Goal To introduce an alternate perspective of RKHS via integral operators
More informationStatistical Inference on Large Contingency Tables: Convergence, Testability, Stability. COMPSTAT 2010 Paris, August 23, 2010
Statistical Inference on Large Contingency Tables: Convergence, Testability, Stability Marianna Bolla Institute of Mathematics Budapest University of Technology and Economics marib@math.bme.hu COMPSTAT
More informationRegML 2018 Class 2 Tikhonov regularization and kernels
RegML 2018 Class 2 Tikhonov regularization and kernels Lorenzo Rosasco UNIGE-MIT-IIT June 17, 2018 Learning problem Problem For H {f f : X Y }, solve min E(f), f H dρ(x, y)l(f(x), y) given S n = (x i,
More informationLearning gradients: prescriptive models
Department of Statistical Science Institute for Genome Sciences & Policy Department of Computer Science Duke University May 11, 2007 Relevant papers Learning Coordinate Covariances via Gradients. Sayan
More informationLearning Interpretable Features to Compare Distributions
Learning Interpretable Features to Compare Distributions Arthur Gretton Gatsby Computational Neuroscience Unit, University College London Theory of Big Data, 2017 1/41 Goal of this talk Given: Two collections
More informationPractical Methods for Graph Two-Sample Testing
Practical Methods for Graph Two-Sample Testing Debarghya Ghoshdastidar Department of Computer Science University of Tübingen ghoshdas@informatik.uni-tuebingen.de Ulrike von Luxburg Department of Computer
More information8.1 Concentration inequality for Gaussian random matrix (cont d)
MGMT 69: Topics in High-dimensional Data Analysis Falll 26 Lecture 8: Spectral clustering and Laplacian matrices Lecturer: Jiaming Xu Scribe: Hyun-Ju Oh and Taotao He, October 4, 26 Outline Concentration
More informationMatrix estimation by Universal Singular Value Thresholding
Matrix estimation by Universal Singular Value Thresholding Courant Institute, NYU Let us begin with an example: Suppose that we have an undirected random graph G on n vertices. Model: There is a real symmetric
More information1. Let A be a 2 2 nonzero real matrix. Which of the following is true?
1. Let A be a 2 2 nonzero real matrix. Which of the following is true? (A) A has a nonzero eigenvalue. (B) A 2 has at least one positive entry. (C) trace (A 2 ) is positive. (D) All entries of A 2 cannot
More informationAn Introduction to Kernel Methods 1
An Introduction to Kernel Methods 1 Yuri Kalnishkan Technical Report CLRC TR 09 01 May 2009 Department of Computer Science Egham, Surrey TW20 0EX, England 1 This paper has been written for wiki project
More informationReproducing Kernel Hilbert Spaces Class 03, 15 February 2006 Andrea Caponnetto
Reproducing Kernel Hilbert Spaces 9.520 Class 03, 15 February 2006 Andrea Caponnetto About this class Goal To introduce a particularly useful family of hypothesis spaces called Reproducing Kernel Hilbert
More informationFunctional Gradient Descent
Statistical Techniques in Robotics (16-831, F12) Lecture #21 (Nov 14, 2012) Functional Gradient Descent Lecturer: Drew Bagnell Scribe: Daniel Carlton Smith 1 1 Goal of Functional Gradient Descent We have
More informationStatistical Machine Learning
Statistical Machine Learning Christoph Lampert Spring Semester 2015/2016 // Lecture 12 1 / 36 Unsupervised Learning Dimensionality Reduction 2 / 36 Dimensionality Reduction Given: data X = {x 1,..., x
More informationRandom Projections. Lopez Paz & Duvenaud. November 7, 2013
Random Projections Lopez Paz & Duvenaud November 7, 2013 Random Outline The Johnson-Lindenstrauss Lemma (1984) Random Kitchen Sinks (Rahimi and Recht, NIPS 2008) Fastfood (Le et al., ICML 2013) Why random
More informationStatistical Model for Soical Network
Statistical Model for Soical Network Tom A.B. Snijders University of Washington May 29, 2014 Outline 1 Cross-sectional network 2 Dynamic s Outline Cross-sectional network 1 Cross-sectional network 2 Dynamic
More informationDependence Minimizing Regression with Model Selection for Non-Linear Causal Inference under Non-Gaussian Noise
Dependence Minimizing Regression with Model Selection for Non-Linear Causal Inference under Non-Gaussian Noise Makoto Yamada and Masashi Sugiyama Department of Computer Science, Tokyo Institute of Technology
More informationMIT 9.520/6.860, Fall 2017 Statistical Learning Theory and Applications. Class 19: Data Representation by Design
MIT 9.520/6.860, Fall 2017 Statistical Learning Theory and Applications Class 19: Data Representation by Design What is data representation? Let X be a data-space X M (M) F (M) X A data representation
More informationA Modified Method Using the Bethe Hessian Matrix to Estimate the Number of Communities
Journal of Advanced Statistics, Vol. 3, No. 2, June 2018 https://dx.doi.org/10.22606/jas.2018.32001 15 A Modified Method Using the Bethe Hessian Matrix to Estimate the Number of Communities Laala Zeyneb
More informationData Analysis and Manifold Learning Lecture 9: Diffusion on Manifolds and on Graphs
Data Analysis and Manifold Learning Lecture 9: Diffusion on Manifolds and on Graphs Radu Horaud INRIA Grenoble Rhone-Alpes, France Radu.Horaud@inrialpes.fr http://perception.inrialpes.fr/ Outline of Lecture
More informationNetwork Cross-Validation for Determining the Number of Communities in Network Data
Network Cross-Validation for Determining the Number of Communities in Network Data Kehui Chen and Jing Lei University of Pittsburgh and Carnegie Mellon University August 1, 2016 Abstract The stochastic
More informationFinding normalized and modularity cuts by spectral clustering. Ljubjana 2010, October
Finding normalized and modularity cuts by spectral clustering Marianna Bolla Institute of Mathematics Budapest University of Technology and Economics marib@math.bme.hu Ljubjana 2010, October Outline Find
More information5 Compact linear operators
5 Compact linear operators One of the most important results of Linear Algebra is that for every selfadjoint linear map A on a finite-dimensional space, there exists a basis consisting of eigenvectors.
More informationFinite-dimensional spaces. C n is the space of n-tuples x = (x 1,..., x n ) of complex numbers. It is a Hilbert space with the inner product
Chapter 4 Hilbert Spaces 4.1 Inner Product Spaces Inner Product Space. A complex vector space E is called an inner product space (or a pre-hilbert space, or a unitary space) if there is a mapping (, )
More informationCan we do statistical inference in a non-asymptotic way? 1
Can we do statistical inference in a non-asymptotic way? 1 Guang Cheng 2 Statistics@Purdue www.science.purdue.edu/bigdata/ ONR Review Meeting@Duke Oct 11, 2017 1 Acknowledge NSF, ONR and Simons Foundation.
More informationIn particular, if A is a square matrix and λ is one of its eigenvalues, then we can find a non-zero column vector X with
Appendix: Matrix Estimates and the Perron-Frobenius Theorem. This Appendix will first present some well known estimates. For any m n matrix A = [a ij ] over the real or complex numbers, it will be convenient
More informationData dependent operators for the spatial-spectral fusion problem
Data dependent operators for the spatial-spectral fusion problem Wien, December 3, 2012 Joint work with: University of Maryland: J. J. Benedetto, J. A. Dobrosotskaya, T. Doster, K. W. Duke, M. Ehler, A.
More information10-701/ Recitation : Kernels
10-701/15-781 Recitation : Kernels Manojit Nandi February 27, 2014 Outline Mathematical Theory Banach Space and Hilbert Spaces Kernels Commonly Used Kernels Kernel Theory One Weird Kernel Trick Representer
More informationStatistical Convergence of Kernel CCA
Statistical Convergence of Kernel CCA Kenji Fukumizu Institute of Statistical Mathematics Tokyo 106-8569 Japan fukumizu@ism.ac.jp Francis R. Bach Centre de Morphologie Mathematique Ecole des Mines de Paris,
More informationElements of Positive Definite Kernel and Reproducing Kernel Hilbert Space
Elements of Positive Definite Kernel and Reproducing Kernel Hilbert Space Statistical Inference with Reproducing Kernel Hilbert Space Kenji Fukumizu Institute of Statistical Mathematics, ROIS Department
More informationNonlinear Dimensionality Reduction
Outline Hong Chang Institute of Computing Technology, Chinese Academy of Sciences Machine Learning Methods (Fall 2012) Outline Outline I 1 Kernel PCA 2 Isomap 3 Locally Linear Embedding 4 Laplacian Eigenmap
More informationRefined Inertia of Matrix Patterns
Electronic Journal of Linear Algebra Volume 32 Volume 32 (2017) Article 24 2017 Refined Inertia of Matrix Patterns Kevin N. Vander Meulen Redeemer University College, kvanderm@redeemer.ca Jonathan Earl
More informationSpectral Clustering for Dynamic Block Models
Spectral Clustering for Dynamic Block Models Sharmodeep Bhattacharyya Department of Statistics Oregon State University January 23, 2017 Research Computing Seminar, OSU, Corvallis (Joint work with Shirshendu
More informationMathematical foundations - linear algebra
Mathematical foundations - linear algebra Andrea Passerini passerini@disi.unitn.it Machine Learning Vector space Definition (over reals) A set X is called a vector space over IR if addition and scalar
More informationMarkov Chain Monte Carlo (MCMC)
Markov Chain Monte Carlo (MCMC Dependent Sampling Suppose we wish to sample from a density π, and we can evaluate π as a function but have no means to directly generate a sample. Rejection sampling can
More informationAnalysis Preliminary Exam Workshop: Hilbert Spaces
Analysis Preliminary Exam Workshop: Hilbert Spaces 1. Hilbert spaces A Hilbert space H is a complete real or complex inner product space. Consider complex Hilbert spaces for definiteness. If (, ) : H H
More informationA Statistical Analysis of Fukunaga Koontz Transform
1 A Statistical Analysis of Fukunaga Koontz Transform Xiaoming Huo Dr. Xiaoming Huo is an assistant professor at the School of Industrial and System Engineering of the Georgia Institute of Technology,
More informationFinal Overview. Introduction to ML. Marek Petrik 4/25/2017
Final Overview Introduction to ML Marek Petrik 4/25/2017 This Course: Introduction to Machine Learning Build a foundation for practice and research in ML Basic machine learning concepts: max likelihood,
More informationMachine Learning. Classification, Discriminative learning. Marc Toussaint University of Stuttgart Summer 2015
Machine Learning Classification, Discriminative learning Structured output, structured input, discriminative function, joint input-output features, Likelihood Maximization, Logistic regression, binary
More informationOslo Class 2 Tikhonov regularization and kernels
RegML2017@SIMULA Oslo Class 2 Tikhonov regularization and kernels Lorenzo Rosasco UNIGE-MIT-IIT May 3, 2017 Learning problem Problem For H {f f : X Y }, solve min E(f), f H dρ(x, y)l(f(x), y) given S n
More informationRANDOM FIELDS AND GEOMETRY. Robert Adler and Jonathan Taylor
RANDOM FIELDS AND GEOMETRY from the book of the same name by Robert Adler and Jonathan Taylor IE&M, Technion, Israel, Statistics, Stanford, US. ie.technion.ac.il/adler.phtml www-stat.stanford.edu/ jtaylor
More informationProblem 1A. Use residues to compute. dx x
Problem 1A. A non-empty metric space X is said to be connected if it is not the union of two non-empty disjoint open subsets, and is said to be path-connected if for every two points a, b there is a continuous
More informationKernel Measures of Conditional Dependence
Kernel Measures of Conditional Dependence Kenji Fukumizu Institute of Statistical Mathematics 4-6-7 Minami-Azabu, Minato-ku Tokyo 6-8569 Japan fukumizu@ism.ac.jp Arthur Gretton Max-Planck Institute for
More information