How Robust are Thresholds for Community Detection?
|
|
- Annice Watkins
- 5 years ago
- Views:
Transcription
1 How Robust are Thresholds for Community Detection? Ankur Moitra (MIT) joint work with Amelia Perry (MIT) and Alex Wein (MIT)
2 Let me tell you a story about the success of belief propagation and statistical physics
3 THE STOCHASTIC BLOCK MODEL Introduced by Holland, Laskey and Leinhardt (1983): probability Q 13 k communities connection probabilities probability Q 11 Q = Q 11 Q 12 Q 13 Q 12 Q 22 Q 32 Q 13 Q 32 Q 33 edges independent
4 THE STOCHASTIC BLOCK MODEL Introduced by Holland, Laskey and Leinhardt (1983): probability Q 13 k communities connection probabilities probability Q 11 Q = Q 11 Q 12 Q 13 Q 12 Q 22 Q 32 Q 13 Q 32 Q 33 edges independent Ubiquitous model studied in statistics, computer science, information theory, statistical physics
5 Testbed for diverse range of algorithms (1) Combinatorial Methods e.g. degree counting [Bui, Chaudhuri, Leighton, Sipser 87] (2) Spectral Methods e.g. [McSherry 01] (3) Markov chain Monte Carlo (MCMC) e.g. [Jerrum, Sorkin 98] (4) Semidefinite Programs e.g. [Boppana 87]
6 BELIEF PROPAGATION Introduced by Judea Pearl (1982): For fundamental contributions to probabilistic and causal reasoning
7 Adapted to community detection: v u Message vèu Probability I think I am community #1, community #2, Do same for other edges
8 Adapted to community detection: v u Message vèu Probability I think I am community #1, community #2, update beliefs Do same for other edges
9 Adapted to community detection: v v u update beliefs u Message vèu Probability I think I am community #1, community #2, Message uèv New probability I think I am community #1, community #2, Do same for other edges Do same for other edges
10 SHARP THRESHOLDS? Renewed interest following Decelle, Krzakala, Moore and Zdeborová (2011), in sparseregime: b a a n n n
11 SHARP THRESHOLDS? Renewed interest following Decelle, Krzakala, Moore and Zdeborová (2011), in sparseregime: b a a n n n Remark: If a, b = O(1) there are many isolated nodes
12 SHARP THRESHOLDS? Renewed interest following Decelle, Krzakala, Moore and Zdeborová (2011), in sparseregime: b a a n n n Remark: If a, b = O(1) there are many isolated nodes agreement better than ½ Conjecture: It is possible to find a partition that is correlated with true communities iff (a-b) 2 > 2(a+b) Motivated by when trivial soln for belief propagation is unstable
13 CONJECTURE IS PROVED! Mossel, Neeman and Sly (2013) and Massoulie (2013): Theorem: It is possible to find a partition that is correlated with true communities iff (a-b) 2 > 2(a+b)
14 CONJECTURE IS PROVED! Mossel, Neeman and Sly (2013) and Massoulie (2013): Theorem: It is possible to find a partition that is correlated with true communities iff (a-b) 2 > 2(a+b) Later attempts to get SDPs to work, but only got to (a-b) 2 > C(a+b), for some C > 2
15 CONJECTURE IS PROVED! Mossel, Neeman and Sly (2013) and Massoulie (2013): Theorem: It is possible to find a partition that is correlated with true communities iff (a-b) 2 > 2(a+b) Later attempts to get SDPs to work, but only got to (a-b) 2 > C(a+b), for some C > 2 Are nonconvexmethods better than convex programs?
16 CONJECTURE IS PROVED! Mossel, Neeman and Sly (2013) and Massoulie (2013): Theorem: It is possible to find a partition that is correlated with true communities iff (a-b) 2 > 2(a+b) Later attempts to get SDPs to work, but only got to (a-b) 2 > C(a+b), for some C > 2 Are nonconvexmethods better than convex programs? How do predictions of statistical physics and SDPs compare?
17 SEMI-RANDOM MODELS Introduced by Blum and Spencer (1995), Feige and Kilian (2001):
18 SEMI-RANDOM MODELS Introduced by Blum and Spencer (1995), Feige and Kilian (2001): (1) Sample graph from SBM
19 SEMI-RANDOM MODELS Introduced by Blum and Spencer (1995), Feige and Kilian (2001): (1) Sample graph from SBM (2) Adversary can add edges within community and delete edges crossing
20 SEMI-RANDOM MODELS Introduced by Blum and Spencer (1995), Feige and Kilian (2001): (1) Sample graph from SBM (2) Adversary can add edges within community and delete edges crossing
21 SEMI-RANDOM MODELS Introduced by Blum and Spencer (1995), Feige and Kilian (2001): (1) Sample graph from SBM (2) Adversary can add edges within community and delete edges crossing
22 SEMI-RANDOM MODELS Introduced by Blum and Spencer (1995), Feige and Kilian (2001): (1) Sample graph from SBM (2) Adversary can add edges within community and delete edges crossing Algorithms can no longer over tune to distribution
23 OUR RESULTS Helpful changes can hurt: Theorem: Community detection in semirandom model is impossible for (a-b) 2 C a,b (a+b) for some C a,b > 2
24 OUR RESULTS Helpful changes can hurt: Theorem: Community detection in semirandom model is impossible for (a-b) 2 C a,b (a+b) for some C a,b > 2 Any algorithm meeting threshold (a-b) 2 > 2(a+b) is inherently brittle and breaks with helpful changes
25 OUR RESULTS Helpful changes can hurt: Theorem: Community detection in semirandom model is impossible for (a-b) 2 C a,b (a+b) for some C a,b > 2 Any algorithm meeting threshold (a-b) 2 > 2(a+b) is inherently brittle and breaks with helpful changes But SDPs continue to work in semirandom model
26 OUR RESULTS Helpful changes can hurt: Theorem: Community detection in semirandom model is possible iff (a-b) 2 C a,b (a+b) for some C a,b > 2 Any algorithm meeting threshold (a-b) 2 > 2(a+b) is inherently brittle and breaks with helpful changes But SDPs continue to work in semirandom model This is first separation between what is possible in random vs. semirandom models
27 RECONSTRUCTION ON TREES Older thresholds, Kesten and Stigum (1966), Evans et al. (2000): Best way to reconstruct root from leaves is majority vote
28 RECONSTRUCTION ON TREES Older thresholds, Kesten and Stigum (1966), Evans et al. (2000): Best way to reconstruct root from leaves is majority vote But why is recursive majority used in practice?
29 RECONSTRUCTION ON TREES Older thresholds, Kesten and Stigum (1966), Evans et al. (2000): Best way to reconstruct root from leaves is majority vote But why is recursive majority used in practice? Our answer: Because recursive majority works in semirandom models but majority does not (thresholds change again)
30 RECONSTRUCTION ON TREES Older thresholds, Kesten and Stigum (1966), Evans et al. (2000): Best way to reconstruct root from leaves is majority vote But why is recursive majority used in practice? Our answer: Because recursive majority works in semirandom models but majority does not (thresholds change again) Is trying to reach the information theoretic threshold the right goal to begin with?
31 BETWEEN WORST-CASE AND AVERAGE-CASE Spielman and Teng (2001): Explain why algorithms work well in practice, despite bad worst-case behavior Usually called Beyond Worst-Case Analysis
32 BETWEEN WORST-CASE AND AVERAGE-CASE Spielman and Teng (2001): Explain why algorithms work well in practice, despite bad worst-case behavior Usually called Beyond Worst-Case Analysis Semirandom models as Above Average-Case Analysis?
33 BETWEEN WORST-CASE AND AVERAGE-CASE Spielman and Teng (2001): Explain why algorithms work well in practice, despite bad worst-case behavior Usually called Beyond Worst-Case Analysis Semirandom models as Above Average-Case Analysis? THANKS! ANY QUESTIONS?
Community Detection. fundamental limits & efficient algorithms. Laurent Massoulié, Inria
Community Detection fundamental limits & efficient algorithms Laurent Massoulié, Inria Community Detection From graph of node-to-node interactions, identify groups of similar nodes Example: Graph of US
More informationHow robust are reconstruction thresholds for community detection?
How robust are reconstruction thresholds for community detection? The MIT Faculty has made this article openly available. Please share how this access benefits you. Your story matters. Citation As Published
More informationBelief Propagation, Robust Reconstruction and Optimal Recovery of Block Models
JMLR: Workshop and Conference Proceedings vol 35:1 15, 2014 Belief Propagation, Robust Reconstruction and Optimal Recovery of Block Models Elchanan Mossel mossel@stat.berkeley.edu Department of Statistics
More informationReconstruction in the Generalized Stochastic Block Model
Reconstruction in the Generalized Stochastic Block Model Marc Lelarge 1 Laurent Massoulié 2 Jiaming Xu 3 1 INRIA-ENS 2 INRIA-Microsoft Research Joint Centre 3 University of Illinois, Urbana-Champaign GDR
More informationSpectral thresholds in the bipartite stochastic block model
Spectral thresholds in the bipartite stochastic block model Laura Florescu and Will Perkins NYU and U of Birmingham September 27, 2016 Laura Florescu and Will Perkins Spectral thresholds in the bipartite
More informationThe non-backtracking operator
The non-backtracking operator Florent Krzakala LPS, Ecole Normale Supérieure in collaboration with Paris: L. Zdeborova, A. Saade Rome: A. Decelle Würzburg: J. Reichardt Santa Fe: C. Moore, P. Zhang Berkeley:
More informationCommunity detection in stochastic block models via spectral methods
Community detection in stochastic block models via spectral methods Laurent Massoulié (MSR-Inria Joint Centre, Inria) based on joint works with: Dan Tomozei (EPFL), Marc Lelarge (Inria), Jiaming Xu (UIUC),
More informationCS369N: Beyond Worst-Case Analysis Lecture #4: Probabilistic and Semirandom Models for Clustering and Graph Partitioning
CS369N: Beyond Worst-Case Analysis Lecture #4: Probabilistic and Semirandom Models for Clustering and Graph Partitioning Tim Roughgarden April 25, 2010 1 Learning Mixtures of Gaussians This lecture continues
More informationISIT Tutorial Information theory and machine learning Part II
ISIT Tutorial Information theory and machine learning Part II Emmanuel Abbe Princeton University Martin Wainwright UC Berkeley 1 Inverse problems on graphs Examples: A large variety of machine learning
More informationSpectral Partitiong in a Stochastic Block Model
Spectral Graph Theory Lecture 21 Spectral Partitiong in a Stochastic Block Model Daniel A. Spielman November 16, 2015 Disclaimer These notes are not necessarily an accurate representation of what happened
More informationOptimality and Sub-optimality of Principal Component Analysis for Spiked Random Matrices
Optimality and Sub-optimality of Principal Component Analysis for Spiked Random Matrices Alex Wein MIT Joint work with: Amelia Perry (MIT), Afonso Bandeira (Courant NYU), Ankur Moitra (MIT) 1 / 22 Random
More informationReconstruction in the Sparse Labeled Stochastic Block Model
Reconstruction in the Sparse Labeled Stochastic Block Model Marc Lelarge 1 Laurent Massoulié 2 Jiaming Xu 3 1 INRIA-ENS 2 INRIA-Microsoft Research Joint Centre 3 University of Illinois, Urbana-Champaign
More informationStatistical and Computational Phase Transitions in Planted Models
Statistical and Computational Phase Transitions in Planted Models Jiaming Xu Joint work with Yudong Chen (UC Berkeley) Acknowledgement: Prof. Bruce Hajek November 4, 203 Cluster/Community structure in
More informationSpectral thresholds in the bipartite stochastic block model
JMLR: Workshop and Conference Proceedings vol 49:1 17, 2016 Spectral thresholds in the bipartite stochastic block model Laura Florescu New York University Will Perkins University of Birmingham FLORESCU@CIMS.NYU.EDU
More informationInformation-theoretic bounds and phase transitions in clustering, sparse PCA, and submatrix localization
Information-theoretic bounds and phase transitions in clustering, sparse PCA, and submatrix localization Jess Banks Cristopher Moore Roman Vershynin Nicolas Verzelen Jiaming Xu Abstract We study the problem
More informationClustering from Sparse Pairwise Measurements
Clustering from Sparse Pairwise Measurements arxiv:6006683v2 [cssi] 9 May 206 Alaa Saade Laboratoire de Physique Statistique École Normale Supérieure, 24 Rue Lhomond Paris 75005 Marc Lelarge INRIA and
More informationConsistency Thresholds for the Planted Bisection Model
Consistency Thresholds for the Planted Bisection Model [Extended Abstract] Elchanan Mossel Joe Neeman University of Pennsylvania and University of Texas, Austin University of California, Mathematics Dept,
More information9 Forward-backward algorithm, sum-product on factor graphs
Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.438 Algorithms For Inference Fall 2014 9 Forward-backward algorithm, sum-product on factor graphs The previous
More informationSpectral Partitioning of Random Graphs
Spectral Partitioning of Random Graphs Frank M c Sherry Dept. of Computer Science and Engineering University of Washington Seattle, WA 98105 mcsherry@cs.washington.edu Abstract Problems such as bisection,
More informationMachine Learning for Data Science (CS4786) Lecture 24
Machine Learning for Data Science (CS4786) Lecture 24 Graphical Models: Approximate Inference Course Webpage : http://www.cs.cornell.edu/courses/cs4786/2016sp/ BELIEF PROPAGATION OR MESSAGE PASSING Each
More informationarxiv: v1 [math.st] 26 Jan 2018
CONCENTRATION OF RANDOM GRAPHS AND APPLICATION TO COMMUNITY DETECTION arxiv:1801.08724v1 [math.st] 26 Jan 2018 CAN M. LE, ELIZAVETA LEVINA AND ROMAN VERSHYNIN Abstract. Random matrix theory has played
More informationSpectral Redemption: Clustering Sparse Networks
Spectral Redemption: Clustering Sparse Networks Florent Krzakala Cristopher Moore Elchanan Mossel Joe Neeman Allan Sly, et al. SFI WORKING PAPER: 03-07-05 SFI Working Papers contain accounts of scienti5ic
More informationCommunity Detection and Stochastic Block Models: Recent Developments
Journal of Machine Learning Research 18 (2018) 1-86 Submitted 9/16; Revised 3/17; Published 4/18 Community Detection and Stochastic Block Models: Recent Developments Emmanuel Abbe Program in Applied and
More informationProbabilistic Graphical Models
Probabilistic Graphical Models Brown University CSCI 295-P, Spring 213 Prof. Erik Sudderth Lecture 11: Inference & Learning Overview, Gaussian Graphical Models Some figures courtesy Michael Jordan s draft
More informationXVI International Congress on Mathematical Physics
Aug 2009 XVI International Congress on Mathematical Physics Underlying geometry: finite graph G=(V,E ). Set of possible configurations: V (assignments of +/- spins to the vertices). Probability of a configuration
More informationCOMMUNITY DETECTION IN SPARSE NETWORKS VIA GROTHENDIECK S INEQUALITY
COMMUNITY DETECTION IN SPARSE NETWORKS VIA GROTHENDIECK S INEQUALITY OLIVIER GUÉDON AND ROMAN VERSHYNIN Abstract. We present a simple and flexible method to prove consistency of semidefinite optimization
More informationA NEARLY-TIGHT SUM-OF-SQUARES LOWER BOUND FOR PLANTED CLIQUE
A NEARLY-TIGHT SUM-OF-SQUARES LOWER BOUND FOR PLANTED CLIQUE Boaz Barak Sam Hopkins Jon Kelner Pravesh Kothari Ankur Moitra Aaron Potechin on the market! PLANTED CLIQUE ("DISTINGUISHING") Input: graph
More informationRandom Walks and Evolving Sets: Faster Convergences and Limitations
Random Walks and Evolving Sets: Faster Convergences and Limitations Siu On Chan 1 Tsz Chiu Kwok 2 Lap Chi Lau 2 1 The Chinese University of Hong Kong 2 University of Waterloo Jan 18, 2017 1/23 Finding
More informationPartially Observable Markov Decision Processes (POMDPs)
Partially Observable Markov Decision Processes (POMDPs) Geoff Hollinger Sequential Decision Making in Robotics Spring, 2011 *Some media from Reid Simmons, Trey Smith, Tony Cassandra, Michael Littman, and
More informationFractional Belief Propagation
Fractional Belief Propagation im iegerinck and Tom Heskes S, niversity of ijmegen Geert Grooteplein 21, 6525 EZ, ijmegen, the etherlands wimw,tom @snn.kun.nl Abstract e consider loopy belief propagation
More informationarxiv: v1 [stat.me] 6 Nov 2014
Network Cross-Validation for Determining the Number of Communities in Network Data Kehui Chen 1 and Jing Lei arxiv:1411.1715v1 [stat.me] 6 Nov 014 1 Department of Statistics, University of Pittsburgh Department
More informationDeep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, Spis treści
Deep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, 2017 Spis treści Website Acknowledgments Notation xiii xv xix 1 Introduction 1 1.1 Who Should Read This Book?
More informationOn the efficient approximability of constraint satisfaction problems
On the efficient approximability of constraint satisfaction problems July 13, 2007 My world Max-CSP Efficient computation. P Polynomial time BPP Probabilistic Polynomial time (still efficient) NP Non-deterministic
More informationEstimation and Optimization: Gaps and Bridges. MURI Meeting June 20, Laurent El Ghaoui. UC Berkeley EECS
MURI Meeting June 20, 2001 Estimation and Optimization: Gaps and Bridges Laurent El Ghaoui EECS UC Berkeley 1 goals currently, estimation (of model parameters) and optimization (of decision variables)
More informationCommunity Detection. Data Analytics - Community Detection Module
Community Detection Data Analytics - Community Detection Module Zachary s karate club Members of a karate club (observed for 3 years). Edges represent interactions outside the activities of the club. Community
More informationBayesian Networks: Belief Propagation in Singly Connected Networks
Bayesian Networks: Belief Propagation in Singly Connected Networks Huizhen u janey.yu@cs.helsinki.fi Dept. Computer Science, Univ. of Helsinki Probabilistic Models, Spring, 2010 Huizhen u (U.H.) Bayesian
More informationApproximating a single component of the solution to a linear system
Approximating a single component of the solution to a linear system Christina E. Lee, Asuman Ozdaglar, Devavrat Shah celee@mit.edu asuman@mit.edu devavrat@mit.edu MIT LIDS 1 How do I compare to my competitors?
More informationCapturing Independence Graphically; Undirected Graphs
Capturing Independence Graphically; Undirected Graphs COMPSCI 276, Spring 2017 Set 2: Rina Dechter (Reading: Pearl chapters 3, Darwiche chapter 4) 1 Outline Graphical models: The constraint network, Probabilistic
More informationLearning With Bayesian Networks. Markus Kalisch ETH Zürich
Learning With Bayesian Networks Markus Kalisch ETH Zürich Inference in BNs - Review P(Burglary JohnCalls=TRUE, MaryCalls=TRUE) Exact Inference: P(b j,m) = c Sum e Sum a P(b)P(e)P(a b,e)p(j a)p(m a) Deal
More informationA Modified Method Using the Bethe Hessian Matrix to Estimate the Number of Communities
Journal of Advanced Statistics, Vol. 3, No. 2, June 2018 https://dx.doi.org/10.22606/jas.2018.32001 15 A Modified Method Using the Bethe Hessian Matrix to Estimate the Number of Communities Laala Zeyneb
More informationU.C. Berkeley CS294: Beyond Worst-Case Analysis Handout 8 Luca Trevisan September 19, 2017
U.C. Berkeley CS294: Beyond Worst-Case Analysis Handout 8 Luca Trevisan September 19, 2017 Scribed by Luowen Qian Lecture 8 In which we use spectral techniques to find certificates of unsatisfiability
More informationLinear inverse problems on Erdős-Rényi graphs: Information-theoretic limits and efficient recovery
Linear inverse problems on Erdős-Rényi graphs: Information-theoretic limits and efficient recovery Emmanuel Abbe, Afonso S. Bandeira, Annina Bracher, Amit Singer Princeton University Abstract This paper
More informationIntroduction to Causal Calculus
Introduction to Causal Calculus Sanna Tyrväinen University of British Columbia August 1, 2017 1 / 1 2 / 1 Bayesian network Bayesian networks are Directed Acyclic Graphs (DAGs) whose nodes represent random
More informationIntroduction to Probabilistic Graphical Models
Introduction to Probabilistic Graphical Models Kyu-Baek Hwang and Byoung-Tak Zhang Biointelligence Lab School of Computer Science and Engineering Seoul National University Seoul 151-742 Korea E-mail: kbhwang@bi.snu.ac.kr
More informationWeighted Message Passing and Minimum Energy Flow for Heterogeneous Stochastic Block Models with Side Information
Weighted Message Passing and Minimum Energy Flow for Heterogeneous Stochastic Block Models with Side Information T. Tony Cai 1, Tengyuan Liang and Alexander Rakhlin 1 1 Department of Statistics, The Wharton
More informationThe Tightness of the Kesten-Stigum Reconstruction Bound for a Symmetric Model With Multiple Mutations
The Tightness of the Kesten-Stigum Reconstruction Bound for a Symmetric Model With Multiple Mutations City University of New York Frontier Probability Days 2018 Joint work with Dr. Sreenivasa Rao Jammalamadaka
More informationOn the low-rank approach for semidefinite programs arising in synchronization and community detection
JMLR: Workshop and Conference Proceedings vol 49:1, 016 On the low-rank approach for semidefinite programs arising in synchronization and community detection Afonso S. Bandeira Department of Mathematics,
More informationDensity Propagation for Continuous Temporal Chains Generative and Discriminative Models
$ Technical Report, University of Toronto, CSRG-501, October 2004 Density Propagation for Continuous Temporal Chains Generative and Discriminative Models Cristian Sminchisescu and Allan Jepson Department
More informationSampling Algorithms for Probabilistic Graphical models
Sampling Algorithms for Probabilistic Graphical models Vibhav Gogate University of Washington References: Chapter 12 of Probabilistic Graphical models: Principles and Techniques by Daphne Koller and Nir
More information6.047 / Computational Biology: Genomes, Networks, Evolution Fall 2008
MIT OpenCourseWare http://ocw.mit.edu 6.047 / 6.878 Computational Biology: Genomes, Networks, Evolution Fall 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
More informationSVD, Power method, and Planted Graph problems (+ eigenvalues of random matrices)
Chapter 14 SVD, Power method, and Planted Graph problems (+ eigenvalues of random matrices) Today we continue the topic of low-dimensional approximation to datasets and matrices. Last time we saw the singular
More informationReconstruction for Models on Random Graphs
1 Stanford University, 2 ENS Paris October 21, 2007 Outline 1 The Reconstruction Problem 2 Related work and applications 3 Results The Reconstruction Problem: A Story Alice and Bob Alice, Bob and G root
More informationThe Origin of Deep Learning. Lili Mou Jan, 2015
The Origin of Deep Learning Lili Mou Jan, 2015 Acknowledgment Most of the materials come from G. E. Hinton s online course. Outline Introduction Preliminary Boltzmann Machines and RBMs Deep Belief Nets
More informationPhase Transitions in Semisupervised Clustering of Sparse Networks
Phase Transitions in Semisupervised Clustering of Sparse Networks Pan Zhang Cristopher Moore Lenka Zdeorová SFI WORKING PAPER: 24-5-5 SFI Working Papers contain accounts of scienti5ic work of the author(s)
More informationPhase transitions in discrete structures
Phase transitions in discrete structures Amin Coja-Oghlan Goethe University Frankfurt Overview 1 The physics approach. [following Mézard, Montanari 09] Basics. Replica symmetry ( Belief Propagation ).
More informationArtificial Intelligence
ICS461 Fall 2010 Nancy E. Reed nreed@hawaii.edu 1 Lecture #14B Outline Inference in Bayesian Networks Exact inference by enumeration Exact inference by variable elimination Approximate inference by stochastic
More informationWhy Water Freezes (A probabilistic view of phase transitions)
Why Water Freezes (A probabilistic view of phase transitions) Mark Huber Dept. of Mathematics and Institute of Statistics and Decision Sciences Duke University mhuber@math.duke.edu www.math.duke.edu/~mhuber
More informationModern WalkSAT algorithms
Modern WalkSAT algorithms J. Rosti Summary of workshop presentation Special research course T-79.7003, Autumn 2007 Phase Transitions in Optimization Problems ABSTRACT In this workshop presentation summary
More informationProbabilistic and Bayesian Machine Learning
Probabilistic and Bayesian Machine Learning Day 4: Expectation and Belief Propagation Yee Whye Teh ywteh@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit University College London http://www.gatsby.ucl.ac.uk/
More informationLearning Energy-Based Models of High-Dimensional Data
Learning Energy-Based Models of High-Dimensional Data Geoffrey Hinton Max Welling Yee-Whye Teh Simon Osindero www.cs.toronto.edu/~hinton/energybasedmodelsweb.htm Discovering causal structure as a goal
More informationBayesian networks: approximate inference
Bayesian networks: approximate inference Machine Intelligence Thomas D. Nielsen September 2008 Approximative inference September 2008 1 / 25 Motivation Because of the (worst-case) intractability of exact
More informationCLUSTERING over graphs is a classical problem with
Maximum Likelihood Latent Space Embedding of Logistic Random Dot Product Graphs Luke O Connor, Muriel Médard and Soheil Feizi ariv:5.85v3 [stat.ml] 3 Aug 27 Abstract A latent space model for a family of
More informationGraph Partitioning Using Random Walks
Graph Partitioning Using Random Walks A Convex Optimization Perspective Lorenzo Orecchia Computer Science Why Spectral Algorithms for Graph Problems in practice? Simple to implement Can exploit very efficient
More informationProbabilistic Graphical Networks: Definitions and Basic Results
This document gives a cursory overview of Probabilistic Graphical Networks. The material has been gleaned from different sources. I make no claim to original authorship of this material. Bayesian Graphical
More informationSTAT 499/962 Topics in Statistics Bayesian Inference and Decision Theory Jan 2018, Handout 01
STAT 499/962 Topics in Statistics Bayesian Inference and Decision Theory Jan 2018, Handout 01 Nasser Sadeghkhani a.sadeghkhani@queensu.ca There are two main schools to statistical inference: 1-frequentist
More informationPhylogenetics: Bayesian Phylogenetic Analysis. COMP Spring 2015 Luay Nakhleh, Rice University
Phylogenetics: Bayesian Phylogenetic Analysis COMP 571 - Spring 2015 Luay Nakhleh, Rice University Bayes Rule P(X = x Y = y) = P(X = x, Y = y) P(Y = y) = P(X = x)p(y = y X = x) P x P(X = x 0 )P(Y = y X
More information27 : Distributed Monte Carlo Markov Chain. 1 Recap of MCMC and Naive Parallel Gibbs Sampling
10-708: Probabilistic Graphical Models 10-708, Spring 2014 27 : Distributed Monte Carlo Markov Chain Lecturer: Eric P. Xing Scribes: Pengtao Xie, Khoa Luu In this scribe, we are going to review the Parallel
More informationBayesian Networks. Characteristics of Learning BN Models. Bayesian Learning. An Example
Bayesian Networks Characteristics of Learning BN Models (All hail Judea Pearl) (some hail Greg Cooper) Benefits Handle incomplete data Can model causal chains of relationships Combine domain knowledge
More informationGenerating random spanning trees. Andrei Broder. DEC - Systems Research Center 130 Lytton Ave., Palo Alto, CA
Generating random spanning trees Andrei Broder DEC - Systems Research Center 30 Lytton Ave., Palo Alto, CA 9430 Extended abstract Abstract. This paper describes a probabilistic algorithm that, given a
More informationTheory and Methods for the Analysis of Social Networks
Theory and Methods for the Analysis of Social Networks Alexander Volfovsky Department of Statistical Science, Duke University Lecture 1: January 16, 2018 1 / 35 Outline Jan 11 : Brief intro and Guest lecture
More informationFinal Exam, Machine Learning, Spring 2009
Name: Andrew ID: Final Exam, 10701 Machine Learning, Spring 2009 - The exam is open-book, open-notes, no electronics other than calculators. - The maximum possible score on this exam is 100. You have 3
More informationSDP Relaxation of Quadratic Optimization with Few Homogeneous Quadratic Constraints
SDP Relaxation of Quadratic Optimization with Few Homogeneous Quadratic Constraints Paul Tseng Mathematics, University of Washington Seattle LIDS, MIT March 22, 2006 (Joint work with Z.-Q. Luo, N. D. Sidiropoulos,
More informationStochastic inference in Bayesian networks, Markov chain Monte Carlo methods
Stochastic inference in Bayesian networks, Markov chain Monte Carlo methods AI: Stochastic inference in BNs AI: Stochastic inference in BNs 1 Outline ypes of inference in (causal) BNs Hardness of exact
More informationInference in Bayesian Networks
Andrea Passerini passerini@disi.unitn.it Machine Learning Inference in graphical models Description Assume we have evidence e on the state of a subset of variables E in the model (i.e. Bayesian Network)
More informationInformation, Physics, and Computation
Information, Physics, and Computation Marc Mezard Laboratoire de Physique Thdorique et Moales Statistiques, CNRS, and Universit y Paris Sud Andrea Montanari Department of Electrical Engineering and Department
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 11 Project
More informationVariational Inference (11/04/13)
STA561: Probabilistic machine learning Variational Inference (11/04/13) Lecturer: Barbara Engelhardt Scribes: Matt Dickenson, Alireza Samany, Tracy Schifeling 1 Introduction In this lecture we will further
More informationIntelligent Systems: Reasoning and Recognition. Reasoning with Bayesian Networks
Intelligent Systems: Reasoning and Recognition James L. Crowley ENSIMAG 2 / MoSIG M1 Second Semester 2016/2017 Lesson 13 24 march 2017 Reasoning with Bayesian Networks Naïve Bayesian Systems...2 Example
More informationOverview of the Talk. Secret Sharing. Secret Sharing Made Short Hugo Krawczyk Perfect Secrecy
Overview of the Talk Secret Sharing CS395T Design and Implementation of Trusted Services Ankur Gupta Hugo Krawczyk. Secret Sharing Made Short, 1993. Josh Cohen Benaloh. Secret Sharing Homomorphisms: Keeping
More informationA Brief Introduction to Graphical Models. Presenter: Yijuan Lu November 12,2004
A Brief Introduction to Graphical Models Presenter: Yijuan Lu November 12,2004 References Introduction to Graphical Models, Kevin Murphy, Technical Report, May 2001 Learning in Graphical Models, Michael
More informationMarkov Chains and MCMC
Markov Chains and MCMC Markov chains Let S = {1, 2,..., N} be a finite set consisting of N states. A Markov chain Y 0, Y 1, Y 2,... is a sequence of random variables, with Y t S for all points in time
More information13: Variational inference II
10-708: Probabilistic Graphical Models, Spring 2015 13: Variational inference II Lecturer: Eric P. Xing Scribes: Ronghuo Zheng, Zhiting Hu, Yuntian Deng 1 Introduction We started to talk about variational
More informationApplications of Convex Optimization on the Graph Isomorphism problem
Applications of Convex Optimization on the Graph Isomorphism problem 1 1 Department of Information and Computer Sciences UC Irvine March 8, 2011 Outline Motivation 1 Motivation The Graph Isomorphism problem
More informationTexas A&M University
Texas A&M University Electrical & Computer Engineering Department Graphical Modeling Course Project Author: Mostafa Karimi UIN: 225000309 Prof Krishna Narayanan May 10 1 Introduction Proteins are made
More informationJunction Tree, BP and Variational Methods
Junction Tree, BP and Variational Methods Adrian Weller MLSALT4 Lecture Feb 21, 2018 With thanks to David Sontag (MIT) and Tony Jebara (Columbia) for use of many slides and illustrations For more information,
More informationLecture 16 Deep Neural Generative Models
Lecture 16 Deep Neural Generative Models CMSC 35246: Deep Learning Shubhendu Trivedi & Risi Kondor University of Chicago May 22, 2017 Approach so far: We have considered simple models and then constructed
More informationA graph contains a set of nodes (vertices) connected by links (edges or arcs)
BOLTZMANN MACHINES Generative Models Graphical Models A graph contains a set of nodes (vertices) connected by links (edges or arcs) In a probabilistic graphical model, each node represents a random variable,
More informationApproximation & Complexity
Summer school on semidefinite optimization Approximation & Complexity David Steurer Cornell University Part 1 September 6, 2012 Overview Part 1 Unique Games Conjecture & Basic SDP Part 2 SDP Hierarchies:
More informationPan Zhang Ph.D. Santa Fe Institute 1399 Hyde Park Road Santa Fe, NM 87501, USA
Pan Zhang Ph.D. Santa Fe Institute 1399 Hyde Park Road Santa Fe, NM 87501, USA PERSONAL DATA Date of Birth: November 19, 1983 Nationality: China Mail: 1399 Hyde Park Rd, Santa Fe, NM 87501, USA E-mail:
More informationBayes Networks 6.872/HST.950
Bayes Networks 6.872/HST.950 What Probabilistic Models Should We Use? Full joint distribution Completely expressive Hugely data-hungry Exponential computational complexity Naive Bayes (full conditional
More informationAutomatic Differentiation Equipped Variable Elimination for Sensitivity Analysis on Probabilistic Inference Queries
Automatic Differentiation Equipped Variable Elimination for Sensitivity Analysis on Probabilistic Inference Queries Anonymous Author(s) Affiliation Address email Abstract 1 2 3 4 5 6 7 8 9 10 11 12 Probabilistic
More informationPart 1: Les rendez-vous symétriques
Part 1: Les rendez-vous symétriques Cris Moore, Santa Fe Institute Joint work with Alex Russell, University of Connecticut A walk in the park L AND NL-COMPLETENESS 313 there are n locations in a park you
More informationLearning in Bayesian Networks
Learning in Bayesian Networks Florian Markowetz Max-Planck-Institute for Molecular Genetics Computational Molecular Biology Berlin Berlin: 20.06.2002 1 Overview 1. Bayesian Networks Stochastic Networks
More informationBenchmarking recovery theorems for the DC-SBM
Benchmarking recovery theorems for the DC-SBM Yali Wan Department of Statistics University of Washington Seattle, WA 98195-4322, USA yaliwan@washington.edu Marina Meilă Department of Statistics University
More informationBenchmarking recovery theorems for the DC-SBM
Benchmarking recovery theorems for the DC-SBM Yali Wan Department of Statistics University of Washington Seattle, WA 98195-4322, USA Marina Meila Department of Statistics University of Washington Seattle,
More informationInference and Representation
Inference and Representation David Sontag New York University Lecture 5, Sept. 30, 2014 David Sontag (NYU) Inference and Representation Lecture 5, Sept. 30, 2014 1 / 16 Today s lecture 1 Running-time of
More informationLecture 13 : Kesten-Stigum bound
Lecture 3 : Kesten-Stigum bound MATH85K - Spring 00 Lecturer: Sebastien Roch References: [EKPS00, Mos0, MP03, BCMR06]. Previous class DEF 3. (Ancestral reconstruction solvability) Let µ + h be the distribution
More informationApproximate inference in Energy-Based Models
CSC 2535: 2013 Lecture 3b Approximate inference in Energy-Based Models Geoffrey Hinton Two types of density model Stochastic generative model using directed acyclic graph (e.g. Bayes Net) Energy-based
More informationStatistical-Computational Phase Transitions in Planted Models: The High-Dimensional Setting
: The High-Dimensional Setting Yudong Chen YUDONGCHEN@EECSBERELEYEDU Department of EECS, University of California, Berkeley, Berkeley, CA 94704, USA Jiaming Xu JXU18@ILLINOISEDU Department of ECE, University
More informationBelief-Propagation Decoding of LDPC Codes
LDPC Codes: Motivation Belief-Propagation Decoding of LDPC Codes Amir Bennatan, Princeton University Revolution in coding theory Reliable transmission, rates approaching capacity. BIAWGN, Rate =.5, Threshold.45
More information