The non-backtracking operator
|
|
- Chastity Lynch
- 5 years ago
- Views:
Transcription
1 The non-backtracking operator Florent Krzakala LPS, Ecole Normale Supérieure in collaboration with Paris: L. Zdeborova, A. Saade Rome: A. Decelle Würzburg: J. Reichardt Santa Fe: C. Moore, P. Zhang Berkeley: E. Mossel, A. Sly, J. Neeman
2 Community detection in network Stochastic block modeling (statistics) Data Clustering (machine learning) Graph-based machine learning (inference) Planted constraint satisfaction models Given the graph, find the labeling! Online communities Word adjacency networks Food webs Metabolic networks Protein-protein interaction networks Financial market (sectors)...
3 This is a hard problem Adjacency matrix What you have: The data What you want: The structure
4 This is a hard problem Karate club 77 The internet Protein network and it is getting harder!
5 Algorithms for clustering Hundreds of different methods - for a review e.g. S. Fortunato, Physics Reports, Spectral clustering - the state of art clustering method in machine learning. Associate a matrix to the network (adjacency, random walk, Laplacian, etc.), compute its top eigenvalues. The corresponding eigenvectors encode the clusters in a visible way.
6 This talk Inference using the stochastic block model Spectral algorithm(s)
7 Stochastic block model Generate a random network as follows: q groups, N nodes n a proportion of nodes in group p ab = c ab a = 1,..., q probability that an edge present N between node from group a and another from group b n 1 = 7/12 n 2 = 5/12 p 11 = p 22 = 0.39 p 12 = p 21 = 0.14
8 Optimal inference in SBM (when parameters of the model known) P ({q i })= NY n qi P (A ij {q i })= Y i6=j p A ij q i q j (1 p qi q j ) 1 A ij i=1 P ({q i } A ij )= 1 Z P ({q i})p (A ij {q i }) n a p ab proportion of nodes in group a proportion of edge between groups a and b q i 2{1,...,q} an assignment of colors Includes ALL available information about the signal
9 Optimal algorithm in SBM (for a statistician) Posterior probability distribution P ({s i } A ij )= 1 Z P ({s i})p (A ij {s i }) Marginal probabilities µ(s i ) X {s j } j6=i P ({s j } j6=i,s i A ij ) Maximize number of correctly assigned nodes s i = argmax si µ(s i )
10 Optimal algorithm in SBM (for a statistical physicist) Potts spin variables s i 2 {1,...,q} Boltzmann measure P ({s i })= 1 Z e H({s i}) Hamiltonian: Potts glass in field on the Nishimori line H({s i })= NX log n si i=1 i6=j magnetic field X Aij log p si s j +(1 A ij ) log (1 p si s j ) pair-wise interactions Local magnetization s i = argmax si µ(s i )
11 How to compute marginals and Z? A #P-hard problem in general MCMC: (Monte Carlo Markov chain, Gibbs sampling): generic, but potentially a (very) large equilibration time Variational mean field: convergence problem and (sometimes) a crude approximation. Belief propagation/cavity method: fast, hard to control, generally better than mean field, exact for large networks generated by the model.
12 Results (in a nutshell) Q = max P N i=1 (t i ),s i 1 1/q /N 1/q Overlap of the optimal estimation with the true labeling Q N=100k, BP N=70k, MC 0.6 q=4, c= ferromagnet detectable paramagnet undetectable ε= c out /c in
13 Results (in a nutshell) overlap c d init. planted init. random q=5,c in =0, N=100k c s paramagnet 0 spinodal region ferromagnet dis-assortative network 5 colors c
14 Algorithmic consequences c d c s c<c c impossible init. ordered init. random log Z pl -log Z ran q=5,c in =0, N=100k log Z pl -log Z ran 0 c c c c in c out = q p c s c c <c<c s c s <c possible but hard easy 0
15 Rigorous results for q=2 For q=2 we find that the information theoretic thresholds equals the algorithmic one c in c out = q p c s Proven! Communutities are undetectable bellow cs and easy to find beyond cs The undetectable regime (Mossel, Neeman, Sly 12) : Planted graphs are essentially Erdos-Renyi random graphs in the undetectable regime. The detectable regime (Massoulie 13)
16 This talk Inference using the stochastic block model Spectral algorithm(s)
17 Spectral clustering Compute k largest eigenvalues and their eigenvectors for a matrix associated to the graph. Cluster these eigenvectors, e.g. using k-means. For 2 groups - signs of the 2nd eigenvector. Matrices: Adjacency Laplacian Random walk Modularity A ij L ij = d i i,j Q ij = A ij d i M ij = A ij A ij d i d j 2M
18 Adjacency matrix for dense graphs Largest eigenvalue is the average degree, and the eigenvector is trivial. It exists an eigenvector correlated with the communities, with eigenvalue c = cin cout 2 cin + cout + cin cout 1 p 4c The bulk follows Wigner s semi circle law: P ( ) = 2 c If cin 2 p cout > 2 c the informative eigenvalue is out of the bulk Adjacency is optimal for dense graphs! If it fails, any thing else will Nadakuditi & Newman 12
19 Performance of spectral clustering Overlap B Modularity Random Walk A L BP q =2 n a =1/2 c =3 N = 10 5 c in = c aa c out = c a6=b c in c out = q p c c in c out Q = max P N i=1 (t i ),q i 1 1/q /N 1/q
20 Why the suboptimality? Wigner not good for sparse G P ( )= 1 p 4c 2 2 c The edge of the spectrum of sparse graphs is spoiled by nodes of large degree. ER graphs, largest degree ~ log(n)/log(log(n)). How to correct this? Remove largest degrees? Not good enough - loosing information.
21 Spectral Redemption Can the optimal performance (phase transition) be matched by a spectral method?
22 The non-backtracking matrix M edges, define matrix B on directed edges, i.e. 2M x 2M matrix as follows B i!j,k!l =1 if j = k, i 6= l B i!j,k!l = 0 otherwise The Hashimoto ( 89) edge adjacency operator used to evaluate the Ihara zeta function Directed walk similar to a message passing algorithm (i.e. Belief propagation)
23 The non-backtracking matrix µ Not a random sparse real non-symmetric matrix, elements correlated! adjacency matrix spectrum
24 Performance of spectral clustering B Modularity Random Walk A L BP Overlap c in c out
25 Properties of the non-backtracking matrix Eigenvalues of B, and sums of eigenvector elements are given by a 2N x 2N matrix. B 0 = Characteristic polynomial (Bass 92 formula) Largest eigenvalue c. c in c out = q p c 0 D 1 1 A v j in = X i2@j det µ 2 1 µa +(D 1) =0 v i!j If the 2nd eigenvalue is informative and > p c All others are inside circle p c (we believed, but proved only in density) Number of eigenvalues outside the circle = number of communities! Complete proof now available Massoulie, Lelarge and Bordenave
26 Spectra of some real networks
27 Spectrum of the non-backtracking matrix
28 Spectrum of the non-backtracking matrix (with some physics hocus-pocus ) det D za (1 z 2 )1 = (z) = 1 2N 2NX i=1 2NY i (z i) (z µ) = z(z µ) 1 (z i). 1 (z) =lim!0 2 z@ z log det M M (z,a) = 1 i(d za (1 z 2 )1) i(d za (1 z 2 )1) 1 (z) = 1 lim!0 2 z@ z log Z
29 Spectrum of the non-backtracking matrix (with some physics hocus-pocus )
30 Spectrum of the non-backtracking matrix (with some physics hocus-pocus ) Non-localized localized
31 The non-backtracking operator Allows spectral clustering to match information theoretic performances Trivial to implement: spectrum of 2N x 2N matrix. B 0 = 0 D 1 1 A Parameter free: No need for learning Avoid the usual BP convergence problems on non-tree like graph.
32 The non-backtracking operator Perspectives: Can we compute Full spectrum rigorously? I believe we have only scratched the surface of the potential of this approach Spectral method for matrix factorization? Censored block model (Abbe et al)? Percolation?
33 Percolation Dense network: The critical occupation probability for percolation on a dense network is equal to the reciprocal of the leading eigenvalue of the adjacency matrix (Bollobas et al 10) ANY network: The critical occupation probability for percolation on a dense network is bounded to the reciprocal of the leading eigenvalue of the non-backtracking matrix (Zdeborova et al 14) (and exact for sparse tree-like graph and dense graphes as well..)
34 This talk is based on: A. Decelle, FK, C. Moore, L. Zdeborová, Phase transition in the detection of modules in sparse networks, Phys. Rev. Lett. 107, (2011). A. Decelle, FK, C. Moore, L. Zdeborová, Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications, Phys. Rev. E 84, (2011). P. Zhang, FK, J. Reichardt, L. Zdeborová, Comparative Study for Inference of Hidden Classes in Stochastic Block Models, J. Stat. Mech. (2012) P12021 FK, C. Moore, E. Mossel, J. Neeman, A. Sly, L. Zdeborová, P. Zhang, Spectral clustering of Sparse Networks, Proc. Nat. Acad. Sci. (2013) A. Saade, FK & L. Zdeborová, The spectrum of the non-backtracking matrix EPL (2014) A. Saade, FK & L. Zdeborová, Clustering with the Bethe Hessian NIPS 2014 Implementations available at: Thank you for your attention!
A Modified Method Using the Bethe Hessian Matrix to Estimate the Number of Communities
Journal of Advanced Statistics, Vol. 3, No. 2, June 2018 https://dx.doi.org/10.22606/jas.2018.32001 15 A Modified Method Using the Bethe Hessian Matrix to Estimate the Number of Communities Laala Zeyneb
More informationTHE PHYSICS OF COUNTING AND SAMPLING ON RANDOM INSTANCES. Lenka Zdeborová
THE PHYSICS OF COUNTING AND SAMPLING ON RANDOM INSTANCES Lenka Zdeborová (CEA Saclay and CNRS, France) MAIN CONTRIBUTORS TO THE PHYSICS UNDERSTANDING OF RANDOM INSTANCES Braunstein, Franz, Kabashima, Kirkpatrick,
More informationCommunity Detection. fundamental limits & efficient algorithms. Laurent Massoulié, Inria
Community Detection fundamental limits & efficient algorithms Laurent Massoulié, Inria Community Detection From graph of node-to-node interactions, identify groups of similar nodes Example: Graph of US
More informationReconstruction in the Generalized Stochastic Block Model
Reconstruction in the Generalized Stochastic Block Model Marc Lelarge 1 Laurent Massoulié 2 Jiaming Xu 3 1 INRIA-ENS 2 INRIA-Microsoft Research Joint Centre 3 University of Illinois, Urbana-Champaign GDR
More informationCommunity detection in stochastic block models via spectral methods
Community detection in stochastic block models via spectral methods Laurent Massoulié (MSR-Inria Joint Centre, Inria) based on joint works with: Dan Tomozei (EPFL), Marc Lelarge (Inria), Jiaming Xu (UIUC),
More informationSpectral Redemption: Clustering Sparse Networks
Spectral Redemption: Clustering Sparse Networks Florent Krzakala Cristopher Moore Elchanan Mossel Joe Neeman Allan Sly, et al. SFI WORKING PAPER: 03-07-05 SFI Working Papers contain accounts of scienti5ic
More informationHow Robust are Thresholds for Community Detection?
How Robust are Thresholds for Community Detection? Ankur Moitra (MIT) joint work with Amelia Perry (MIT) and Alex Wein (MIT) Let me tell you a story about the success of belief propagation and statistical
More informationLearning latent structure in complex networks
Learning latent structure in complex networks Lars Kai Hansen www.imm.dtu.dk/~lkh Current network research issues: Social Media Neuroinformatics Machine learning Joint work with Morten Mørup, Sune Lehmann
More informationPart 1: Les rendez-vous symétriques
Part 1: Les rendez-vous symétriques Cris Moore, Santa Fe Institute Joint work with Alex Russell, University of Connecticut A walk in the park L AND NL-COMPLETENESS 313 there are n locations in a park you
More informationClustering from Sparse Pairwise Measurements
Clustering from Sparse Pairwise Measurements arxiv:6006683v2 [cssi] 9 May 206 Alaa Saade Laboratoire de Physique Statistique École Normale Supérieure, 24 Rue Lhomond Paris 75005 Marc Lelarge INRIA and
More informationOn convergence of Approximate Message Passing
On convergence of Approximate Message Passing Francesco Caltagirone (1), Florent Krzakala (2) and Lenka Zdeborova (1) (1) Institut de Physique Théorique, CEA Saclay (2) LPS, Ecole Normale Supérieure, Paris
More informationStatistical and Computational Phase Transitions in Planted Models
Statistical and Computational Phase Transitions in Planted Models Jiaming Xu Joint work with Yudong Chen (UC Berkeley) Acknowledgement: Prof. Bruce Hajek November 4, 203 Cluster/Community structure in
More informationCommunity Detection. Data Analytics - Community Detection Module
Community Detection Data Analytics - Community Detection Module Zachary s karate club Members of a karate club (observed for 3 years). Edges represent interactions outside the activities of the club. Community
More informationAlgebraic Representation of Networks
Algebraic Representation of Networks 0 1 2 1 1 0 0 1 2 0 0 1 1 1 1 1 Hiroki Sayama sayama@binghamton.edu Describing networks with matrices (1) Adjacency matrix A matrix with rows and columns labeled by
More informationStatistical Physics on Sparse Random Graphs: Mathematical Perspective
Statistical Physics on Sparse Random Graphs: Mathematical Perspective Amir Dembo Stanford University Northwestern, July 19, 2016 x 5 x 6 Factor model [DM10, Eqn. (1.4)] x 1 x 2 x 3 x 4 x 9 x8 x 7 x 10
More informationSpectral Partitiong in a Stochastic Block Model
Spectral Graph Theory Lecture 21 Spectral Partitiong in a Stochastic Block Model Daniel A. Spielman November 16, 2015 Disclaimer These notes are not necessarily an accurate representation of what happened
More informationMachine Learning for Data Science (CS4786) Lecture 24
Machine Learning for Data Science (CS4786) Lecture 24 Graphical Models: Approximate Inference Course Webpage : http://www.cs.cornell.edu/courses/cs4786/2016sp/ BELIEF PROPAGATION OR MESSAGE PASSING Each
More informationPhysics on Random Graphs
Physics on Random Graphs Lenka Zdeborová (CNLS + T-4, LANL) in collaboration with Florent Krzakala (see MRSEC seminar), Marc Mezard, Marco Tarzia... Take Home Message Solving problems on random graphs
More informationPhase Transitions (and their meaning) in Random Constraint Satisfaction Problems
International Workshop on Statistical-Mechanical Informatics 2007 Kyoto, September 17 Phase Transitions (and their meaning) in Random Constraint Satisfaction Problems Florent Krzakala In collaboration
More informationPan Zhang Ph.D. Santa Fe Institute 1399 Hyde Park Road Santa Fe, NM 87501, USA
Pan Zhang Ph.D. Santa Fe Institute 1399 Hyde Park Road Santa Fe, NM 87501, USA PERSONAL DATA Date of Birth: November 19, 1983 Nationality: China Mail: 1399 Hyde Park Rd, Santa Fe, NM 87501, USA E-mail:
More informationMachine Learning in Simple Networks. Lars Kai Hansen
Machine Learning in Simple Networs Lars Kai Hansen www.imm.dtu.d/~lh Outline Communities and lin prediction Modularity Modularity as a combinatorial optimization problem Gibbs sampling Detection threshold
More informationPhase Transitions in the Coloring of Random Graphs
Phase Transitions in the Coloring of Random Graphs Lenka Zdeborová (LPTMS, Orsay) In collaboration with: F. Krząkała (ESPCI Paris) G. Semerjian (ENS Paris) A. Montanari (Standford) F. Ricci-Tersenghi (La
More informationCOMMUNITY DETECTION IN SPARSE NETWORKS VIA GROTHENDIECK S INEQUALITY
COMMUNITY DETECTION IN SPARSE NETWORKS VIA GROTHENDIECK S INEQUALITY OLIVIER GUÉDON AND ROMAN VERSHYNIN Abstract. We present a simple and flexible method to prove consistency of semidefinite optimization
More informationExtreme eigenvalues of Erdős-Rényi random graphs
Extreme eigenvalues of Erdős-Rényi random graphs Florent Benaych-Georges j.w.w. Charles Bordenave and Antti Knowles MAP5, Université Paris Descartes May 18, 2018 IPAM UCLA Inhomogeneous Erdős-Rényi random
More informationInformation-theoretic bounds and phase transitions in clustering, sparse PCA, and submatrix localization
Information-theoretic bounds and phase transitions in clustering, sparse PCA, and submatrix localization Jess Banks Cristopher Moore Roman Vershynin Nicolas Verzelen Jiaming Xu Abstract We study the problem
More informationarxiv: v1 [cond-mat.dis-nn] 16 Feb 2018
Parallel Tempering for the planted clique problem Angelini Maria Chiara 1 1 Dipartimento di Fisica, Ed. Marconi, Sapienza Università di Roma, P.le A. Moro 2, 00185 Roma Italy arxiv:1802.05903v1 [cond-mat.dis-nn]
More informationPhase Transitions in Physics and Computer Science. Cristopher Moore University of New Mexico and the Santa Fe Institute
Phase Transitions in Physics and Computer Science Cristopher Moore University of New Mexico and the Santa Fe Institute Magnetism When cold enough, Iron will stay magnetized, and even magnetize spontaneously
More informationSpectra of Large Random Stochastic Matrices & Relaxation in Complex Systems
Spectra of Large Random Stochastic Matrices & Relaxation in Complex Systems Reimer Kühn Disordered Systems Group Department of Mathematics, King s College London Random Graphs and Random Processes, KCL
More informationSpectral thresholds in the bipartite stochastic block model
Spectral thresholds in the bipartite stochastic block model Laura Florescu and Will Perkins NYU and U of Birmingham September 27, 2016 Laura Florescu and Will Perkins Spectral thresholds in the bipartite
More informationA brief introduction to the inverse Ising problem and some algorithms to solve it
A brief introduction to the inverse Ising problem and some algorithms to solve it Federico Ricci-Tersenghi Physics Department Sapienza University, Roma original results in collaboration with Jack Raymond
More informationLearning from and about complex energy landspaces
Learning from and about complex energy landspaces Lenka Zdeborová (CNLS + T-4, LANL) in collaboration with: Florent Krzakala (ParisTech) Thierry Mora (Princeton Univ.) Landscape Landscape Santa Fe Institute
More informationarxiv: v1 [math.st] 26 Jan 2018
CONCENTRATION OF RANDOM GRAPHS AND APPLICATION TO COMMUNITY DETECTION arxiv:1801.08724v1 [math.st] 26 Jan 2018 CAN M. LE, ELIZAVETA LEVINA AND ROMAN VERSHYNIN Abstract. Random matrix theory has played
More informationPhase Transitions in Semisupervised Clustering of Sparse Networks
Phase Transitions in Semisupervised Clustering of Sparse Networks Pan Zhang Cristopher Moore Lenka Zdeorová SFI WORKING PAPER: 24-5-5 SFI Working Papers contain accounts of scienti5ic work of the author(s)
More informationBelief Propagation, Robust Reconstruction and Optimal Recovery of Block Models
JMLR: Workshop and Conference Proceedings vol 35:1 15, 2014 Belief Propagation, Robust Reconstruction and Optimal Recovery of Block Models Elchanan Mossel mossel@stat.berkeley.edu Department of Statistics
More informationTheory and Methods for the Analysis of Social Networks
Theory and Methods for the Analysis of Social Networks Alexander Volfovsky Department of Statistical Science, Duke University Lecture 1: January 16, 2018 1 / 35 Outline Jan 11 : Brief intro and Guest lecture
More informationPhase transitions in discrete structures
Phase transitions in discrete structures Amin Coja-Oghlan Goethe University Frankfurt Overview 1 The physics approach. [following Mézard, Montanari 09] Basics. Replica symmetry ( Belief Propagation ).
More informationData Mining and Analysis: Fundamental Concepts and Algorithms
: Fundamental Concepts and Algorithms dataminingbook.info Mohammed J. Zaki 1 Wagner Meira Jr. 2 1 Department of Computer Science Rensselaer Polytechnic Institute, Troy, NY, USA 2 Department of Computer
More informationBayesian Learning in Undirected Graphical Models
Bayesian Learning in Undirected Graphical Models Zoubin Ghahramani Gatsby Computational Neuroscience Unit University College London, UK http://www.gatsby.ucl.ac.uk/ Work with: Iain Murray and Hyun-Chul
More informationApproximate counting of large subgraphs in random graphs with statistical mechanics methods
Approximate counting of large subgraphs in random graphs with statistical mechanics methods Guilhem Semerjian LPT-ENS Paris 13.03.08 / Eindhoven in collaboration with Rémi Monasson, Enzo Marinari and Valery
More informationInformation, Physics, and Computation
Information, Physics, and Computation Marc Mezard Laboratoire de Physique Thdorique et Moales Statistiques, CNRS, and Universit y Paris Sud Andrea Montanari Department of Electrical Engineering and Department
More informationAccelerated Monte Carlo simulations with restricted Boltzmann machines
Accelerated simulations with restricted Boltzmann machines Li Huang 1 Lei Wang 2 1 China Academy of Engineering Physics 2 Institute of Physics, Chinese Academy of Sciences July 6, 2017 Machine Learning
More informationReconstruction in the Sparse Labeled Stochastic Block Model
Reconstruction in the Sparse Labeled Stochastic Block Model Marc Lelarge 1 Laurent Massoulié 2 Jiaming Xu 3 1 INRIA-ENS 2 INRIA-Microsoft Research Joint Centre 3 University of Illinois, Urbana-Champaign
More informationXVI International Congress on Mathematical Physics
Aug 2009 XVI International Congress on Mathematical Physics Underlying geometry: finite graph G=(V,E ). Set of possible configurations: V (assignments of +/- spins to the vertices). Probability of a configuration
More informationThe Tightness of the Kesten-Stigum Reconstruction Bound for a Symmetric Model With Multiple Mutations
The Tightness of the Kesten-Stigum Reconstruction Bound for a Symmetric Model With Multiple Mutations City University of New York Frontier Probability Days 2018 Joint work with Dr. Sreenivasa Rao Jammalamadaka
More informationProbabilistic Graphical Models
Probabilistic Graphical Models Brown University CSCI 295-P, Spring 213 Prof. Erik Sudderth Lecture 11: Inference & Learning Overview, Gaussian Graphical Models Some figures courtesy Michael Jordan s draft
More informationUnderstanding graphs through spectral densities
Understanding graphs through spectral densities David Bindel Department of Computer Science Cornell University SCAN seminar, 29 Feb 216 SCAN 1 / 1 Can One Hear the Shape of a Drum? 2 u = λu on Ω, u = on
More informationEstimating the Number of Communities in a Bipartite Network
Estimating the Number of Communities in a Bipartite Network Tzu-Chi Yen, @oneofyen http://junipertcy.info Visitor at Santa Fe Institute, and Institute of Theoretical Physics, Chinese Academy of Sciences
More informationSPARSE RANDOM GRAPHS: REGULARIZATION AND CONCENTRATION OF THE LAPLACIAN
SPARSE RANDOM GRAPHS: REGULARIZATION AND CONCENTRATION OF THE LAPLACIAN CAN M. LE, ELIZAVETA LEVINA, AND ROMAN VERSHYNIN Abstract. We study random graphs with possibly different edge probabilities in the
More informationReconstruction for Models on Random Graphs
1 Stanford University, 2 ENS Paris October 21, 2007 Outline 1 The Reconstruction Problem 2 Related work and applications 3 Results The Reconstruction Problem: A Story Alice and Bob Alice, Bob and G root
More informationData science with multilayer networks: Mathematical foundations and applications
Data science with multilayer networks: Mathematical foundations and applications CDSE Days University at Buffalo, State University of New York Monday April 9, 2018 Dane Taylor Assistant Professor of Mathematics
More informationStatistical Physics of Inference and Bayesian Estimation
Statistical Physics of Inference and Bayesian Estimation Florent Krzakala, Lenka Zdeborova, Maria Chiara Angelini and Francesco Caltagirone Contents Bayesian Inference and Estimators 3. The Bayes formula..........................................
More informationUNDERSTANDING BELIEF PROPOGATION AND ITS GENERALIZATIONS
UNDERSTANDING BELIEF PROPOGATION AND ITS GENERALIZATIONS JONATHAN YEDIDIA, WILLIAM FREEMAN, YAIR WEISS 2001 MERL TECH REPORT Kristin Branson and Ian Fasel June 11, 2003 1. Inference Inference problems
More informationNetwork Analysis and Modeling
lecture 0: what are networks and how do we talk about them? 2017 Aaron Clauset 003 052 002 001 Aaron Clauset @aaronclauset Assistant Professor of Computer Science University of Colorado Boulder External
More informationPredicting Graph Labels using Perceptron. Shuang Song
Predicting Graph Labels using Perceptron Shuang Song shs037@eng.ucsd.edu Online learning over graphs M. Herbster, M. Pontil, and L. Wainer, Proc. 22nd Int. Conf. Machine Learning (ICML'05), 2005 Prediction
More informationMarkov Chains, Random Walks on Graphs, and the Laplacian
Markov Chains, Random Walks on Graphs, and the Laplacian CMPSCI 791BB: Advanced ML Sridhar Mahadevan Random Walks! There is significant interest in the problem of random walks! Markov chain analysis! Computer
More informationSpectral Properties of Random Matrices for Stochastic Block Model
Spectral Properties of Random Matrices for Stochastic Block Model Konstantin Avrachenkov INRIA Sophia Antipolis France Email:konstantin.avratchenkov@inria.fr Laura Cottatellucci Eurecom Sophia Antipolis
More informationMini course on Complex Networks
Mini course on Complex Networks Massimo Ostilli 1 1 UFSC, Florianopolis, Brazil September 2017 Dep. de Fisica Organization of The Mini Course Day 1: Basic Topology of Equilibrium Networks Day 2: Percolation
More informationMachine Learning for Data Science (CS4786) Lecture 11
Machine Learning for Data Science (CS4786) Lecture 11 Spectral clustering Course Webpage : http://www.cs.cornell.edu/courses/cs4786/2016sp/ ANNOUNCEMENT 1 Assignment P1 the Diagnostic assignment 1 will
More informationSparse Superposition Codes for the Gaussian Channel
Sparse Superposition Codes for the Gaussian Channel Florent Krzakala (LPS, Ecole Normale Supérieure, France) J. Barbier (ENS) arxiv:1403.8024 presented at ISIT 14 Long version in preparation Communication
More informationIntroduction to Graphical Models. Srikumar Ramalingam School of Computing University of Utah
Introduction to Graphical Models Srikumar Ramalingam School of Computing University of Utah Reference Christopher M. Bishop, Pattern Recognition and Machine Learning, Jonathan S. Yedidia, William T. Freeman,
More informationarxiv: v1 [physics.soc-ph] 20 Oct 2010
arxiv:00.098v [physics.soc-ph] 20 Oct 200 Spectral methods for the detection of network community structure: A comparative analysis Hua-Wei Shen and Xue-Qi Cheng Institute of Computing Technology, Chinese
More informationThe Computer Science and Physics of Community Detection: Landscapes, Phase Transitions, and Hardness
arxiv:1702.00467v3 [cs.cc] 24 Aug 2017 The Computer Science and Physics of Community Detection: Landscapes, Phase Transitions, and Hardness Cristopher Moore Santa Fe Institute moore@santafe.edu Abstract
More informationNonparametric Bayesian Matrix Factorization for Assortative Networks
Nonparametric Bayesian Matrix Factorization for Assortative Networks Mingyuan Zhou IROM Department, McCombs School of Business Department of Statistics and Data Sciences The University of Texas at Austin
More informationKyle Reing University of Southern California April 18, 2018
Renormalization Group and Information Theory Kyle Reing University of Southern California April 18, 2018 Overview Renormalization Group Overview Information Theoretic Preliminaries Real Space Mutual Information
More informationPlanted Cliques, Iterative Thresholding and Message Passing Algorithms
Planted Cliques, Iterative Thresholding and Message Passing Algorithms Yash Deshpande and Andrea Montanari Stanford University November 5, 2013 Deshpande, Montanari Planted Cliques November 5, 2013 1 /
More informationDeep unsupervised learning
Deep unsupervised learning Advanced data-mining Yongdai Kim Department of Statistics, Seoul National University, South Korea Unsupervised learning In machine learning, there are 3 kinds of learning paradigm.
More informationThe Trouble with Community Detection
The Trouble with Community Detection Aaron Clauset Santa Fe Institute 7 April 2010 Nonlinear Dynamics of Networks Workshop U. Maryland, College Park Thanks to National Science Foundation REU Program James
More informationSpectra of Adjacency and Laplacian Matrices
Spectra of Adjacency and Laplacian Matrices Definition: University of Alicante (Spain) Matrix Computing (subject 3168 Degree in Maths) 30 hours (theory)) + 15 hours (practical assignment) Contents 1. Spectra
More informationPhase transition phenomena of statistical mechanical models of the integer factorization problem (submitted to JPSJ, now in review process)
Phase transition phenomena of statistical mechanical models of the integer factorization problem (submitted to JPSJ, now in review process) Chihiro Nakajima WPI-AIMR, Tohoku University Masayuki Ohzeki
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 11 Project
More information6.867 Machine learning, lecture 23 (Jaakkola)
Lecture topics: Markov Random Fields Probabilistic inference Markov Random Fields We will briefly go over undirected graphical models or Markov Random Fields (MRFs) as they will be needed in the context
More information13 : Variational Inference: Loopy Belief Propagation and Mean Field
10-708: Probabilistic Graphical Models 10-708, Spring 2012 13 : Variational Inference: Loopy Belief Propagation and Mean Field Lecturer: Eric P. Xing Scribes: Peter Schulam and William Wang 1 Introduction
More informationCLUSTERING over graphs is a classical problem with
Maximum Likelihood Latent Space Embedding of Logistic Random Dot Product Graphs Luke O Connor, Muriel Médard and Soheil Feizi ariv:5.85v3 [stat.ml] 3 Aug 27 Abstract A latent space model for a family of
More informationCS249: ADVANCED DATA MINING
CS249: ADVANCED DATA MINING Graph and Network Instructor: Yizhou Sun yzsun@cs.ucla.edu May 31, 2017 Methods Learnt Classification Clustering Vector Data Text Data Recommender System Decision Tree; Naïve
More informationComplexity, Parallel Computation and Statistical Physics
Complexity, Parallel Computation and Statistical Physics Jon Machta! Measures of Complexity workshop Santa Fe Institute January 13, 2011 Outline Overview and motivation: What aspect of natural complexity
More informationCommunities, Spectral Clustering, and Random Walks
Communities, Spectral Clustering, and Random Walks David Bindel Department of Computer Science Cornell University 26 Sep 2011 20 21 19 16 22 28 17 18 29 26 27 30 23 1 25 5 8 24 2 4 14 3 9 13 15 11 10 12
More informationLecture 18 Generalized Belief Propagation and Free Energy Approximations
Lecture 18, Generalized Belief Propagation and Free Energy Approximations 1 Lecture 18 Generalized Belief Propagation and Free Energy Approximations In this lecture we talked about graphical models and
More informationLearning Energy-Based Models of High-Dimensional Data
Learning Energy-Based Models of High-Dimensional Data Geoffrey Hinton Max Welling Yee-Whye Teh Simon Osindero www.cs.toronto.edu/~hinton/energybasedmodelsweb.htm Discovering causal structure as a goal
More informationPhase Transitions in Networks: Giant Components, Dynamic Networks, Combinatoric Solvability
in Networks: Giant Components, Dynamic Networks, Combinatoric Solvability Department of Physics UC Davis April 27, 2009 Outline Historical Prospective Old School New School Non-Physics 1 Historical Prospective
More informationAccelerating QMC on quantum computers. Matthias Troyer
Accelerating QMC on quantum computers Matthias Troyer International Journal of Theoretical Physics, VoL 21, Nos. 6/7, 1982 Simulating Physics with Computers Richard P. Feynman Department of Physics, California
More information1 Matrix notation and preliminaries from spectral graph theory
Graph clustering (or community detection or graph partitioning) is one of the most studied problems in network analysis. One reason for this is that there are a variety of ways to define a cluster or community.
More informationMarkov Chain Monte Carlo The Metropolis-Hastings Algorithm
Markov Chain Monte Carlo The Metropolis-Hastings Algorithm Anthony Trubiano April 11th, 2018 1 Introduction Markov Chain Monte Carlo (MCMC) methods are a class of algorithms for sampling from a probability
More informationPhase transitions in low-rank matrix estimation
Phase transitions in low-rank matrix estimation Florent Krzakala http://krzakala.github.io/lowramp/ arxiv:1503.00338 & 1507.03857 Thibault Lesieur Lenka Zeborová Caterina Debacco Cris Moore Jess Banks
More informationSampling Algorithms for Probabilistic Graphical models
Sampling Algorithms for Probabilistic Graphical models Vibhav Gogate University of Washington References: Chapter 12 of Probabilistic Graphical models: Principles and Techniques by Daphne Koller and Nir
More informationThe Ising model and Markov chain Monte Carlo
The Ising model and Markov chain Monte Carlo Ramesh Sridharan These notes give a short description of the Ising model for images and an introduction to Metropolis-Hastings and Gibbs Markov Chain Monte
More informationFundamental limits of symmetric low-rank matrix estimation
Proceedings of Machine Learning Research vol 65:1 5, 2017 Fundamental limits of symmetric low-rank matrix estimation Marc Lelarge Safran Tech, Magny-Les-Hameaux, France Département d informatique, École
More informationNetworks and Their Spectra
Networks and Their Spectra Victor Amelkin University of California, Santa Barbara Department of Computer Science victor@cs.ucsb.edu December 4, 2017 1 / 18 Introduction Networks (= graphs) are everywhere.
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 3 Linear
More informationModularity in several random graph models
Modularity in several random graph models Liudmila Ostroumova Prokhorenkova 1,3 Advanced Combinatorics and Network Applications Lab Moscow Institute of Physics and Technology Moscow, Russia Pawe l Pra
More informationIntroduction to Machine Learning CMU-10701
Introduction to Machine Learning CMU-10701 Markov Chain Monte Carlo Methods Barnabás Póczos & Aarti Singh Contents Markov Chain Monte Carlo Methods Goal & Motivation Sampling Rejection Importance Markov
More informationOnline Social Networks and Media. Link Analysis and Web Search
Online Social Networks and Media Link Analysis and Web Search How to Organize the Web First try: Human curated Web directories Yahoo, DMOZ, LookSmart How to organize the web Second try: Web Search Information
More informationThe Particle Filter. PD Dr. Rudolph Triebel Computer Vision Group. Machine Learning for Computer Vision
The Particle Filter Non-parametric implementation of Bayes filter Represents the belief (posterior) random state samples. by a set of This representation is approximate. Can represent distributions that
More informationSpectra of adjacency matrices of random geometric graphs
Spectra of adjacency matrices of random geometric graphs Paul Blackwell, Mark Edmondson-Jones and Jonathan Jordan University of Sheffield 22nd December 2006 Abstract We investigate the spectral properties
More informationIntroduction to Graphical Models. Srikumar Ramalingam School of Computing University of Utah
Introduction to Graphical Models Srikumar Ramalingam School of Computing University of Utah Reference Christopher M. Bishop, Pattern Recognition and Machine Learning, Jonathan S. Yedidia, William T. Freeman,
More informationMonte Carlo Simulation of the Ising Model. Abstract
Monte Carlo Simulation of the Ising Model Saryu Jindal 1 1 Department of Chemical Engineering and Material Sciences, University of California, Davis, CA 95616 (Dated: June 9, 2007) Abstract This paper
More informationA physical model for efficient rankings in networks
A physical model for efficient rankings in networks Daniel Larremore Assistant Professor Dept. of Computer Science & BioFrontiers Institute March 5, 2018 CompleNet danlarremore.com @danlarremore The idea
More informationSpectral Graph Theory and its Applications. Daniel A. Spielman Dept. of Computer Science Program in Applied Mathematics Yale Unviersity
Spectral Graph Theory and its Applications Daniel A. Spielman Dept. of Computer Science Program in Applied Mathematics Yale Unviersity Outline Adjacency matrix and Laplacian Intuition, spectral graph drawing
More informationSTA 414/2104: Machine Learning
STA 414/2104: Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistics! rsalakhu@cs.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 9 Sequential Data So far
More informationThe Origin of Deep Learning. Lili Mou Jan, 2015
The Origin of Deep Learning Lili Mou Jan, 2015 Acknowledgment Most of the materials come from G. E. Hinton s online course. Outline Introduction Preliminary Boltzmann Machines and RBMs Deep Belief Nets
More informationProbabilistic Graphical Models
Probabilistic Graphical Models Brown University CSCI 2950-P, Spring 2013 Prof. Erik Sudderth Lecture 13: Learning in Gaussian Graphical Models, Non-Gaussian Inference, Monte Carlo Methods Some figures
More information