StarAI Full, 6+1 pages Short, 2 page position paper or abstract
|
|
|
- Lewis Lucas
- 5 years ago
- Views:
Transcription
1 StarAI 2015 Fifth International Workshop on Statistical Relational AI At the 31st Conference on Uncertainty in Artificial Intelligence (UAI) (right after ICML) In Amsterdam, The Netherlands, on July 16. Paper Submission: May 15 Full, 6+1 pages Short, 2 page position paper or abstract
2 What we can t do (yet, well)? Approximate Symmetries in Lifted Inference Guy Van den Broeck (on joint work with Mathias Niepert and Adnan Darwiche) KU Leuven
3 Overview Lifted inference in 2 slides Complexity of evidence Over-symmetric approximations Approximate symmetries Conclusions
4 Overview Lifted inference in 2 slides Complexity of evidence Over-symmetric approximations Approximate symmetries Conclusions
5 Lifted Inference In AI: exploiting symmetries/exchangeability Example: WebKB Domain: symmetry url { google.com, ibm.com, aaai.org, } Weighted clauses: CoursePage(x) ^ Linked(x,y) => CoursePage(y) FacultyPage(x) ^ Linked(x,y) => FacultyPage (y) HasWord( Lecture",x) => CoursePage(x) HasWord( Office",x) => FacultyPage(x) more first-order sentences
6 The State of Lifted Inference UCQ database queries: solved PTIME in database size (when possible) MLNs and related Two logical variables: solved Partition function PTIME in domain size (always) Three logical variables: #P 1 -hard Bunch of great approximation algorithms Theoretical connections to exchangeability
7 Overview Lifted inference in 2 slides Complexity of evidence Over-symmetric approximations Approximate symmetries Conclusions
8 Problem: Prediction with Evidence Add evidence on links: Linked( google.com, gmail.com ) Linked( google.com, aaai.org ) Symmetry google.com ibm.com? Linked( ibm.com, watson.com ) Linked( ibm.com, ibm.ca ) Add evidence on words No! HasWord( Android, google.com ) HasWord( G+, google.com ) Symmetry google.com ibm.com? HasWord( Blue, ibm.com ) HasWord( Computing, ibm.com ) No!
9 Complexity in Size of Evidence Consider a model liftable for model counting: 3.14 FacultyPage(x) Linked(x,y) CoursePage(y) Given database DB, compute P(Q DB). Complexity in DB size? Evidence on unary relations: Efficient FacultyPage("google.com")=0, CoursePage("coursera.org")=1, Evidence on binary relations: #P-hard Linked("google.com","gmail.com")=1, Linked("google.com", aaai.org")=0 Intuition: Binary evidence breaks symmetries Consequence: Lifted algorithms reduce to ground (also approx) [Van den Broeck, Davis; AAAI 12, Bui et al., Dalvi and Suciu, etc.]
10 Approach Conditioning on binary evidence is hard Conditioning on unary evidence is efficient Solution: Represent binary evidence as unary Matrix notation:
11 Vector Product Solution: Represent binary evidence as unary Case 1:
12 Vector Product Solution: Represent binary evidence as unary Case 1:
13 Vector Product Solution: Represent binary evidence as unary Case 1:
14 Vector Product Solution: Represent binary evidence as unary Case 1:
15 Matrix Product Solution: Represent binary evidence as unary Case 2:
16 Matrix Product Solution: Represent binary evidence as unary Case 2: where
17 Boolean Matrix Factorization Decompose In Boolean algebra, where 1+1=1 Minimum n is the Boolean rank Always possible
18 Matrix Product Solution: Represent binary evidence as unary Example:
19 Matrix Product Solution: Represent binary evidence as unary Example:
20 Matrix Product Solution: Represent binary evidence as unary Example:
21 Matrix Product Solution: Represent binary evidence as unary Example: Boolean rank n=3
22 Theoretical Consequences Theorem: Complexity of computing Pr(q e) in SRL is polynomial in e, when e has bounded Boolean rank. Boolean rank key parameter in the complexity of conditioning says how much lifting is possible [Van den Broeck, Darwiche; NIPS 13]
23 Analogy with Treewidth in Probabilistic Graphical Models Probabilistic graphical models: 1. Find tree decomposition 1. Perform inference Exponential in (tree)width of decomposition Polynomial in size of Bayesian network SRL Models: 1. Find Boolean matrix factorization of evidence 2. Perform inference Exponential in Boolean rank of evidence Polynomial in size of evidence database Polynomial in domain size
24 Overview Lifted inference in 2 slides Complexity of evidence Over-symmetric approximations Approximate symmetries Conclusions
25 Over-Symmetric Approximation Approximate Pr(q e) by Pr(q e') Pr(q e') has more symmetries, is more liftable E.g.: Low-rank Boolean matrix factorization Boolean rank 3
26 Over-Symmetric Approximation Approximate Pr(q e) by Pr(q e') Pr(q e') has more symmetries, is more liftable E.g.: Low-rank Boolean matrix factorization Boolean rank 2 approximation
27 Over-Symmetric Approximations OSA makes model more symmetric E.g., low-rank Boolean matrix factorization Link ( aaai.org, google.com ) Link ( google.com, aaai.org ) Link ( google.com, gmail.com ) Link ( ibm.com, aaai.org ) Link ( aaai.org, google.com ) Link ( google.com, aaai.org ) - Link ( google.com, gmail.com ) + Link ( aaai.org, ibm.com ) Link ( ibm.com, aaai.org ) google.com and ibm.com become symmetric! [Van den Broeck, Darwiche; NIPS 13]
28 Markov Chain Monte-Carlo Gibbs sampling or MC-SAT Problem: slow convergence, one variable changed One million random variables: need at least one million iteration to move between two states Lifted MCMC: move between symmetric states
29 Lifted MCMC on WebKB
30 Rank 1 Approximation
31 Rank 2 Approximation
32 Rank 5 Approximation
33 Rank 10 Approximation
34 Rank 20 Approximation
35 Rank 50 Approximation
36 Rank 75 Approximation
37 Rank 100 Approximation
38 Rank 150 Approximation
39 Trend for Increasing Boolean Rank
40 Best Case
41 Overview Lifted inference in 2 slides Complexity of evidence Over-symmetric approximations Approximate symmetries Conclusions
42 Problem with OSAs Approximation can be crude Cannot converge to true distribution Lose information about subtle differences Real distribution Pr(PageClass( Faculty, )) = 0.47 Pr(PageClass( Faculty, )) = 0.53 OSA distribution Pr(PageClass( Faculty, )) = 0.50 Pr(PageClass( Faculty, )) = 0.50
43 Approximate Symmetries Exploit approximate symmetries: Exact symmetry g: Pr(x) = Pr(x g ) E.g. Ising model without external field Approximate symmetry g: Pr(x) Pr(x g ) E.g. Ising model with external field P P
44 Orbital Metropolis Chain: Algorithm Given symmetry group G (approx. symmetries) Orbit x G contains all states approx. symm. to x In state x: 1. Select y uniformly at random from x G 2. Move from x to y with probability min 3. Otherwise: stay in x (reject) 4. Repeat Pr y Pr x, 1
45 Orbital Metropolis Chain: Analysis Pr(.) is stationary distribution Many variables change (fast mixing) Few rejected samples: Pr y Pr x min Pr y Pr x, 1 1 Is this the perfect proposal distribution?
46 Orbital Metropolis Chain: Analysis Pr(.) is stationary distribution Many variables change (fast mixing) Few rejected samples: Pr y Pr x min Pr y Pr x, 1 1 Is this the perfect proposal distribution? Not irreducible Can never reach 0100 from 1101.
47 Lifted Metropolis-Hastings: Algorithm Given an orbital Metropolis chain M S for Pr(.) Given a base Markov chain M B that is irreducible and aperiodic has stationary distribution Pr(.) (e.g., Gibbs chain or MC-SAT chain) In state x: 1. With probability α, apply the kernel of M B 2. Otherwise apply the kernel of M S
48 Lifted Metropolis-Hastings: Analysis Theorem [Tierney 1994]: A mixture of Markov chains is irreducible and aperiodic if at least one of the chains is irreducible and aperiodic. Pr(.) is stationary distribution Many variables change (fast mixing) Few rejected samples Irreducible Aperiodic
49 Gibbs Sampling Lifted Metropolis- Hastings G = (X 1 X 2 )(X 3 X 4 )
50 Experiments: WebKB [Van den Broeck, Niepert; AAAI 15]
51 Experiments: WebKB
52 Overview Lifted inference in 2 slides Complexity of evidence Over-symmetric approximations Approximate symmetries Conclusions
53 Take-Away Message Two problems: 1. Lifted inference gives exponential speedups in symmetric graphical models. But what about real-world asymmetric problems? 2. When there are many variables, MCMC is slow. How to sample quickly in large graphical models? One solution: Exploit approximate symmetries!
54 Open Problems Find approximate symmetries Principled (theory) Is a type of machine learning? During inference, not preprocessing? Give guarantees on approximation quality/convergence speed Plug in lifted inference from prob. databases
55 Lots of Recent Activity Singla, Nath, and Domingos (2014) Venugopal and Gogate (2014) Kersting et al. (2014)
56 Thanks
57 KL Divergence Example: Grid Models
Lifted Probabilistic Inference for Asymmetric Graphical Models
Lifted Probabilistic Inference for Asymmetric Graphical Models Guy Van den Broeck Department of Computer Science KU Leuven, Belgium guy.vandenbroeck@cs.kuleuven.be Mathias Niepert Computer Science and
On First-Order Knowledge Compilation. Guy Van den Broeck
On First-Order Knowledge Compilation Guy Van den Broeck Beyond NP Workshop Feb 12, 2016 Overview 1. Why first-order model counting? 2. Why first-order model counters? 3. What first-order circuit languages?
First-Order Probabilistic Reasoning: Successes and Challenges. Guy Van den Broeck
First-Order Probabilistic Reasoning: Successes and Challenges Guy Van den Broeck IJCAI Early Career Spotlight Jul 14, 2016 Overview 1. Why first-order probabilistic models? 2. Why first-order probabilistic
Symmetry Breaking for Relational Weighted Model Finding
Symmetry Breaking for Relational Weighted Model Finding Tim Kopp Parag Singla Henry Kautz WL4AI July 27, 2015 Outline Weighted logics. Symmetry-breaking in SAT. Symmetry-breaking for weighted logics. Weighted
arxiv: v2 [cs.ai] 27 Jul 2017
![arxiv: v2 [cs.ai] 27 Jul 2017](/thumbs/89/97694795.jpg "arxiv: v2 [cs.ai] 27 Jul 2017")
Domain Recursion for Lifted Inference with Existential Quantifiers Seyed Mehran Kazemi 1, Angelika Kimmig 2, Guy Van den Broeck 3 and David Poole 1 1 University of British Columbia, {smkazemi,poole}@cs.ubc.ca
First-Order Knowledge Compilation for Probabilistic Reasoning
First-Order Knowledge Compilation for Probabilistic Reasoning Guy Van den Broeck based on joint work with Adnan Darwiche, Dan Suciu, and many others MOTIVATION 1 A Simple Reasoning Problem...? Probability
Probabilistic and Logistic Circuits: A New Synthesis of Logic and Machine Learning
Probabilistic and Logistic Circuits: A New Synthesis of Logic and Machine Learning Guy Van den Broeck KULeuven Symposium Dec 12, 2018 Outline Learning Adding knowledge to deep learning Logistic circuits
Introduction to Machine Learning CMU-10701
Introduction to Machine Learning CMU-10701 Markov Chain Monte Carlo Methods Barnabás Póczos & Aarti Singh Contents Markov Chain Monte Carlo Methods Goal & Motivation Sampling Rejection Importance Markov
Lifted MAP Inference for Markov Logic
Lifted MAP Inference for Markov Logic - Somdeb Sarkhel - Deepak Venugopal - Happy Mittal - Parag Singla - Vibhav Gogate Outline Introduction Lifted MAP Inference for Markov Logic Networks S. Sarkhel, D.
CPSC 540: Machine Learning
CPSC 540: Machine Learning MCMC and Non-Parametric Bayes Mark Schmidt University of British Columbia Winter 2016 Admin I went through project proposals: Some of you got a message on Piazza. No news is
Markov Chains and MCMC
Markov Chains and MCMC CompSci 590.02 Instructor: AshwinMachanavajjhala Lecture 4 : 590.02 Spring 13 1 Recap: Monte Carlo Method If U is a universe of items, and G is a subset satisfying some property,
Sampling Algorithms for Probabilistic Graphical models
Sampling Algorithms for Probabilistic Graphical models Vibhav Gogate University of Washington References: Chapter 12 of Probabilistic Graphical models: Principles and Techniques by Daphne Koller and Nir
Non-Count Symmetries in Boolean & Multi-Valued Prob. Graphical Models
Ankit Anand 1 Ritesh Noothigattu 1 Parag Singla and Mausam Department of CSE Machine Learning Department 1 Department of CSE I.I.T Delhi Carnegie Mellon University I.I.T Delhi Abstract Lifted inference
CS 188: Artificial Intelligence. Bayes Nets
CS 188: Artificial Intelligence Probabilistic Inference: Enumeration, Variable Elimination, Sampling Pieter Abbeel UC Berkeley Many slides over this course adapted from Dan Klein, Stuart Russell, Andrew
Introduction to MCMC. DB Breakfast 09/30/2011 Guozhang Wang
Introduction to MCMC DB Breakfast 09/30/2011 Guozhang Wang Motivation: Statistical Inference Joint Distribution Sleeps Well Playground Sunny Bike Ride Pleasant dinner Productive day Posterior Estimation
STA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 7 Approximate
Probabilistic and Logistic Circuits: A New Synthesis of Logic and Machine Learning
Probabilistic and Logistic Circuits: A New Synthesis of Logic and Machine Learning Guy Van den Broeck HRL/ACTIONS @ KR Oct 28, 2018 Foundation: Logical Circuit Languages Negation Normal Form Circuits Δ
Intelligent Systems (AI-2)
Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 11 Oct, 3, 2016 CPSC 422, Lecture 11 Slide 1 422 big picture: Where are we? Query Planning Deterministic Logics First Order Logics Ontologies
Lifted Probabilistic Inference in Relational Models. U. of Washington
Lifted Probabilistic Inference in Relational Models Guy Van den Broeck UCLA Dan Suciu U. of Washington IJCAI Tutorial July 10, 2016 About the Tutorial Slides available at http://web.cs.ucla.edu/~guyvdb/talks/ijcai16-tutorial/
Lifted Probabilistic Inference: A Guide for the Database Researcher
Lifted Probabilistic Inference: A Guide for the Database Researcher Eric Gribkoff University of Washington eagribko@cs.uw.edu Dan Suciu University of Washington suciu@cs.uw.edu Guy Van den Broeck UCLA,
New Rules for Domain Independent Lifted MAP Inference
New Rules for Domain Independent Lifted MAP Inference Happy Mittal, Prasoon Goyal Dept. of Comp. Sci. & Engg. I.I.T. Delhi, Hauz Khas New Delhi, 116, India happy.mittal@cse.iitd.ac.in prasoongoyal13@gmail.com
Convex Optimization CMU-10725
Convex Optimization CMU-10725 Simulated Annealing Barnabás Póczos & Ryan Tibshirani Andrey Markov Markov Chains 2 Markov Chains Markov chain: Homogen Markov chain: 3 Markov Chains Assume that the state
Scaling-up Importance Sampling for Markov Logic Networks
Scaling-up Importance Sampling for Markov Logic Networks Deepak Venugopal Department of Computer Science University of Texas at Dallas dxv021000@utdallas.edu Vibhav Gogate Department of Computer Science
Markov chain Monte Carlo Lecture 9
Markov chain Monte Carlo Lecture 9 David Sontag New York University Slides adapted from Eric Xing and Qirong Ho (CMU) Limitations of Monte Carlo Direct (unconditional) sampling Hard to get rare events
Markov Chain Monte Carlo Inference. Siamak Ravanbakhsh Winter 2018
Graphical Models Markov Chain Monte Carlo Inference Siamak Ravanbakhsh Winter 2018 Learning objectives Markov chains the idea behind Markov Chain Monte Carlo (MCMC) two important examples: Gibbs sampling
CS 343: Artificial Intelligence
CS 343: Artificial Intelligence Bayes Nets: Sampling Prof. Scott Niekum The University of Texas at Austin [These slides based on those of Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley.
26 : Spectral GMs. Lecturer: Eric P. Xing Scribes: Guillermo A Cidre, Abelino Jimenez G.
10-708: Probabilistic Graphical Models, Spring 2015 26 : Spectral GMs Lecturer: Eric P. Xing Scribes: Guillermo A Cidre, Abelino Jimenez G. 1 Introduction A common task in machine learning is to work with
Lifted Probabilistic Inference in Relational Models
Lifted Probabilistic Inference in Relational Models Guy Van den Broeck Dan Suciu KU Leuven U. of Washington Tutorial UAI 2014 1 About the Tutorial Slides available online. Bibliography is at the end. Your
Computer Vision Group Prof. Daniel Cremers. 10a. Markov Chain Monte Carlo
Group Prof. Daniel Cremers 10a. Markov Chain Monte Carlo Markov Chain Monte Carlo In high-dimensional spaces, rejection sampling and importance sampling are very inefficient An alternative is Markov Chain
Outline. Part 1: Motivation Part 2: Probabilistic Databases Part 3: Weighted Model Counting Part 4: Lifted Inference for WFOMC
Outline Part 1: Motivation Part 2: Probabilistic Databases Part 3: Weighted Model Counting Part 4: Lifted Inference for WFOMC Part 5: Completeness of Lifted Inference Part 6: Query Compilation Part 7:
Markov Chains and MCMC
Markov Chains and MCMC Markov chains Let S = {1, 2,..., N} be a finite set consisting of N states. A Markov chain Y 0, Y 1, Y 2,... is a sequence of random variables, with Y t S for all points in time
Machine Learning for Data Science (CS4786) Lecture 24
Machine Learning for Data Science (CS4786) Lecture 24 Graphical Models: Approximate Inference Course Webpage : http://www.cs.cornell.edu/courses/cs4786/2016sp/ BELIEF PROPAGATION OR MESSAGE PASSING Each
arxiv: v1 [cs.ai] 9 Apr 2013
![arxiv: v1 [cs.ai] 9 Apr 2013](/thumbs/82/87003337.jpg "arxiv: v1 [cs.ai] 9 Apr 2013")
Symmetry-Aware Marginal Density Estimation Mathias Niepert Computer Science & Engineering University of Washington Seattle, WA 98195-2350, USA arxiv:1304.2694v1 [cs.ai] 9 Apr 2013 Abstract The Rao-Blackwell
Probabilistic Theorem Proving. Vibhav Gogate (Joint work with Pedro Domingos)
Probabilistic Theorem Proving Vibhav Gogate (Joint work with Pedro Domingos) Progress to Date First-order variable elimination [Poole, 2003; de Salvo Braz, 2007; etc.] Lifted belief propagation [Singla
Probabilistic Circuits: A New Synthesis of Logic and Machine Learning
Probabilistic Circuits: A New Synthesis of Logic and Machine Learning Guy Van den Broeck UCSD May 14, 2018 Overview Statistical ML Probability Symbolic AI Logic Probabilistic Circuits Connectionism Deep
Approximating the Partition Function by Deleting and then Correcting for Model Edges (Extended Abstract)
Approximating the Partition Function by Deleting and then Correcting for Model Edges (Extended Abstract) Arthur Choi and Adnan Darwiche Computer Science Department University of California, Los Angeles
Lecture 15: MCMC Sanjeev Arora Elad Hazan. COS 402 Machine Learning and Artificial Intelligence Fall 2016
Lecture 15: MCMC Sanjeev Arora Elad Hazan COS 402 Machine Learning and Artificial Intelligence Fall 2016 Course progress Learning from examples Definition + fundamental theorem of statistical learning,
Toward Caching Symmetrical Subtheories for Weighted Model Counting
Toward Caching Symmetrical Subtheories for Weighted Model Counting Tim Kopp University of Rochester Rochester, NY Parag Singla Indian Institute of Technology Hauz Khas, New Delhi Henry Kautz University
Lifted Inference: Exact Search Based Algorithms
Lifted Inference: Exact Search Based Algorithms Vibhav Gogate The University of Texas at Dallas Overview Background and Notation Probabilistic Knowledge Bases Exact Inference in Propositional Models First-order
Probabilistic First Order Models for Coreference. Aron Culotta. Information Extraction & Synthesis Lab University of Massachusetts
Probabilistic First Order Models for Coreference Aron Culotta Information Extraction & Synthesis Lab University of Massachusetts joint work with advisor Andrew McCallum Motivation Beyond local representation
MCMC: Markov Chain Monte Carlo
I529: Machine Learning in Bioinformatics (Spring 2013) MCMC: Markov Chain Monte Carlo Yuzhen Ye School of Informatics and Computing Indiana University, Bloomington Spring 2013 Contents Review of Markov
Knowledge Compilation and Machine Learning A New Frontier Adnan Darwiche Computer Science Department UCLA with Arthur Choi and Guy Van den Broeck
Knowledge Compilation and Machine Learning A New Frontier Adnan Darwiche Computer Science Department UCLA with Arthur Choi and Guy Van den Broeck The Problem Logic (L) Knowledge Representation (K) Probability
Markov Networks. l Like Bayes Nets. l Graphical model that describes joint probability distribution using tables (AKA potentials)
Markov Networks l Like Bayes Nets l Graphical model that describes joint probability distribution using tables (AKA potentials) l Nodes are random variables l Labels are outcomes over the variables Markov
Computer Vision Group Prof. Daniel Cremers. 11. Sampling Methods: Markov Chain Monte Carlo
Group Prof. Daniel Cremers 11. Sampling Methods: Markov Chain Monte Carlo Markov Chain Monte Carlo In high-dimensional spaces, rejection sampling and importance sampling are very inefficient An alternative
Bayes Nets: Sampling
Bayes Nets: Sampling [These slides were created by Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley. All CS188 materials are available at http://ai.berkeley.edu.] Approximate Inference:
Markov Networks.
Markov Networks www.biostat.wisc.edu/~dpage/cs760/ Goals for the lecture you should understand the following concepts Markov network syntax Markov network semantics Potential functions Partition function
Closed-form Gibbs Sampling for Graphical Models with Algebraic constraints. Hadi Mohasel Afshar Scott Sanner Christfried Webers
Closed-form Gibbs Sampling for Graphical Models with Algebraic constraints Hadi Mohasel Afshar Scott Sanner Christfried Webers Inference in Hybrid Graphical Models / Probabilistic Programs Limitations
Lifted Symmetry Detection and Breaking for MAP Inference
Lifted Symmetry Detection and Breaking for MAP Inference Tim Kopp University of Rochester Rochester, NY tkopp@cs.rochester.edu Parag Singla I.I.T. Delhi Hauz Khas, New Delhi parags@cse.iitd.ac.in Henry
16 : Approximate Inference: Markov Chain Monte Carlo
10-708: Probabilistic Graphical Models 10-708, Spring 2017 16 : Approximate Inference: Markov Chain Monte Carlo Lecturer: Eric P. Xing Scribes: Yuan Yang, Chao-Ming Yen 1 Introduction As the target distribution
Introduction to Machine Learning CMU-10701
Introduction to Machine Learning CMU-10701 Markov Chain Monte Carlo Methods Barnabás Póczos Contents Markov Chain Monte Carlo Methods Sampling Rejection Importance Hastings-Metropolis Gibbs Markov Chains
16 : Markov Chain Monte Carlo (MCMC)
10-708: Probabilistic Graphical Models 10-708, Spring 2014 16 : Markov Chain Monte Carlo MCMC Lecturer: Matthew Gormley Scribes: Yining Wang, Renato Negrinho 1 Sampling from low-dimensional distributions
Bayesian Methods for Machine Learning
Bayesian Methods for Machine Learning CS 584: Big Data Analytics Material adapted from Radford Neal s tutorial (http://ftp.cs.utoronto.ca/pub/radford/bayes-tut.pdf), Zoubin Ghahramni (http://hunch.net/~coms-4771/zoubin_ghahramani_bayesian_learning.pdf),
PSDDs for Tractable Learning in Structured and Unstructured Spaces
PSDDs for Tractable Learning in Structured and Unstructured Spaces Guy Van den Broeck DeLBP Aug 18, 2017 References Probabilistic Sentential Decision Diagrams Doga Kisa, Guy Van den Broeck, Arthur Choi
Pattern Recognition and Machine Learning
Christopher M. Bishop Pattern Recognition and Machine Learning ÖSpri inger Contents Preface Mathematical notation Contents vii xi xiii 1 Introduction 1 1.1 Example: Polynomial Curve Fitting 4 1.2 Probability
Markov chain Monte Carlo
1 / 26 Markov chain Monte Carlo Timothy Hanson 1 and Alejandro Jara 2 1 Division of Biostatistics, University of Minnesota, USA 2 Department of Statistics, Universidad de Concepción, Chile IAP-Workshop
Lifted Marginal MAP Inference
Lifted Marginal MAP Inference Vishal Sharma 1, Noman Ahmed Sheikh 2, Happy Mittal 1, Vibhav Gogate 3 and Parag Singla 1 1 IIT Delhi, {vishal.sharma, happy.mittal, parags}@cse.iitd.ac.in 2 IIT Delhi, nomanahmedsheikh@outlook.com
Representing and Querying Correlated Tuples in Probabilistic Databases
Representing and Querying Correlated Tuples in Probabilistic Databases Prithviraj Sen Amol Deshpande Department of Computer Science University of Maryland, College Park. International Conference on Data
Tractable Learning in Structured Probability Spaces
Tractable Learning in Structured Probability Spaces Guy Van den Broeck DTAI Seminar - KU Leuven Dec 20, 2016 Structured probability spaces? Running Example Courses: Logic (L) Knowledge Representation (K)
Tractable Learning in Structured Probability Spaces
Tractable Learning in Structured Probability Spaces Guy Van den Broeck UCLA Stats Seminar Jan 17, 2017 Outline 1. Structured probability spaces? 2. Specification language Logic 3. Deep architecture Logic
Markov Chains on Orbits of Permutation Groups
Markov Chains on Orbits of Permutation Groups Mathias Niepert Universität Mannheim mniepert@gmail.com Abstract We present a novel approach to detecting and utilizing symmetries in probabilistic graphical
The Particle Filter. PD Dr. Rudolph Triebel Computer Vision Group. Machine Learning for Computer Vision
The Particle Filter Non-parametric implementation of Bayes filter Represents the belief (posterior) random state samples. by a set of This representation is approximate. Can represent distributions that
Open-World Probabilistic Databases. Guy Van den Broeck
Open-World Probabilistic Databases Guy Van den Broeck GCAI Oct 21, 2017 Overview 1. Why probabilistic databases? 2. How probabilistic query evaluation? 3. Why open world? 4. How open-world query evaluation?
The Ising model and Markov chain Monte Carlo
The Ising model and Markov chain Monte Carlo Ramesh Sridharan These notes give a short description of the Ising model for images and an introduction to Metropolis-Hastings and Gibbs Markov Chain Monte
Computer Vision Group Prof. Daniel Cremers. 14. Sampling Methods
Prof. Daniel Cremers 14. Sampling Methods Sampling Methods Sampling Methods are widely used in Computer Science as an approximation of a deterministic algorithm to represent uncertainty without a parametric
Lect4: Exact Sampling Techniques and MCMC Convergence Analysis
Lect4: Exact Sampling Techniques and MCMC Convergence Analysis. Exact sampling. Convergence analysis of MCMC. First-hit time analysis for MCMC--ways to analyze the proposals. Outline of the Module Definitions
Markov Networks. l Like Bayes Nets. l Graph model that describes joint probability distribution using tables (AKA potentials)
Markov Networks l Like Bayes Nets l Graph model that describes joint probability distribution using tables (AKA potentials) l Nodes are random variables l Labels are outcomes over the variables Markov
MCMC and Gibbs Sampling. Kayhan Batmanghelich
MCMC and Gibbs Sampling Kayhan Batmanghelich 1 Approaches to inference l Exact inference algorithms l l l The elimination algorithm Message-passing algorithm (sum-product, belief propagation) The junction
Announcements. Inference. Mid-term. Inference by Enumeration. Reminder: Alarm Network. Introduction to Artificial Intelligence. V22.
Introduction to Artificial Intelligence V22.0472-001 Fall 2009 Lecture 15: Bayes Nets 3 Midterms graded Assignment 2 graded Announcements Rob Fergus Dept of Computer Science, Courant Institute, NYU Slides
CS Lecture 3. More Bayesian Networks
CS 6347 Lecture 3 More Bayesian Networks Recap Last time: Complexity challenges Representing distributions Computing probabilities/doing inference Introduction to Bayesian networks Today: D-separation,
Graphical Models for Query-driven Analysis of Multimodal Data
Graphical Models for Query-driven Analysis of Multimodal Data John Fisher Sensing, Learning, & Inference Group Computer Science & Artificial Intelligence Laboratory Massachusetts Institute of Technology
LECTURE 15 Markov chain Monte Carlo
LECTURE 15 Markov chain Monte Carlo There are many settings when posterior computation is a challenge in that one does not have a closed form expression for the posterior distribution. Markov chain Monte
Fast Lifted MAP Inference via Partitioning
Fast Lifted MAP Inference via Partitioning Somdeb Sarkhel The University of Texas at Dallas Parag Singla I.I.T. Delhi Abstract Vibhav Gogate The University of Texas at Dallas Recently, there has been growing
April 20th, Advanced Topics in Machine Learning California Institute of Technology. Markov Chain Monte Carlo for Machine Learning
for for Advanced Topics in California Institute of Technology April 20th, 2017 1 / 50 Table of Contents for 1 2 3 4 2 / 50 History of methods for Enrico Fermi used to calculate incredibly accurate predictions
CSC 446 Notes: Lecture 13
CSC 446 Notes: Lecture 3 The Problem We have already studied how to calculate the probability of a variable or variables using the message passing method. However, there are some times when the structure
Lecture 6: Graphical Models: Learning
Lecture 6: Graphical Models: Learning 4F13: Machine Learning Zoubin Ghahramani and Carl Edward Rasmussen Department of Engineering, University of Cambridge February 3rd, 2010 Ghahramani & Rasmussen (CUED)
STA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 7 Approximate
Sampling Rejection Sampling Importance Sampling Markov Chain Monte Carlo. Sampling Methods. Oliver Schulte - CMPT 419/726. Bishop PRML Ch.
Sampling Methods Oliver Schulte - CMP 419/726 Bishop PRML Ch. 11 Recall Inference or General Graphs Junction tree algorithm is an exact inference method for arbitrary graphs A particular tree structure
The Parameterized Complexity of Approximate Inference in Bayesian Networks
JMLR: Workshop and Conference Proceedings vol 52, 264-274, 2016 PGM 2016 The Parameterized Complexity of Approximate Inference in Bayesian Networks Johan Kwisthout Radboud University Donders Institute
Markov Chain Monte Carlo
1 Motivation 1.1 Bayesian Learning Markov Chain Monte Carlo Yale Chang In Bayesian learning, given data X, we make assumptions on the generative process of X by introducing hidden variables Z: p(z): prior
Computational statistics
Computational statistics Markov Chain Monte Carlo methods Thierry Denœux March 2017 Thierry Denœux Computational statistics March 2017 1 / 71 Contents of this chapter When a target density f can be evaluated
Towards Deterministic Decomposable Circuits for Safe Queries
Towards Deterministic Decomposable Circuits for Safe Queries Mikaël Monet 1 and Dan Olteanu 2 1 LTCI, Télécom ParisTech, Université Paris-Saclay, France, Inria Paris; Paris, France mikael.monet@telecom-paristech.fr
Studies in Solution Sampling
Studies in Solution Sampling Vibhav Gogate and Rina Dechter Donald Bren School of Information and Computer Science, University of California, Irvine, CA 92697, USA. {vgogate,dechter}@ics.uci.edu Abstract
Machine Learning Summer School
Machine Learning Summer School Lecture 3: Learning parameters and structure Zoubin Ghahramani zoubin@eng.cam.ac.uk http://learning.eng.cam.ac.uk/zoubin/ Department of Engineering University of Cambridge,
Sampling Rejection Sampling Importance Sampling Markov Chain Monte Carlo. Sampling Methods. Machine Learning. Torsten Möller.
Sampling Methods Machine Learning orsten Möller Möller/Mori 1 Recall Inference or General Graphs Junction tree algorithm is an exact inference method for arbitrary graphs A particular tree structure defined
Introduction to Artificial Intelligence (AI)
Introduction to Artificial Intelligence (AI) Computer Science cpsc502, Lecture 9 Oct, 11, 2011 Slide credit Approx. Inference : S. Thrun, P, Norvig, D. Klein CPSC 502, Lecture 9 Slide 1 Today Oct 11 Bayesian
Computer Vision Group Prof. Daniel Cremers. 11. Sampling Methods
Prof. Daniel Cremers 11. Sampling Methods Sampling Methods Sampling Methods are widely used in Computer Science as an approximation of a deterministic algorithm to represent uncertainty without a parametric
Lifted Relax, Compensate and then Recover: From Approximate to Exact Lifted Probabilistic Inference
Lifted Relax, Compensate and then Recover: From Approximate to Exact Lifted Probabilistic Inference Guy Van den Broeck Department of Computer Science KU Leuven guy.vandenbroeck@cs.kuleuven.be Arthur Choi
17 : Markov Chain Monte Carlo
10-708: Probabilistic Graphical Models, Spring 2015 17 : Markov Chain Monte Carlo Lecturer: Eric P. Xing Scribes: Heran Lin, Bin Deng, Yun Huang 1 Review of Monte Carlo Methods 1.1 Overview Monte Carlo
Hashing-Based Approximate Probabilistic Inference in Hybrid Domains: An Abridged Report
Hashing-Based Approximate Probabilistic Inference in Hybrid Domains: An Abridged Report Vaishak Belle KU Leuven vaishak@cs.kuleuven.be Guy Van den Broeck University of California, Los Angeles guyvdb@cs.ucla.edu
Bagging During Markov Chain Monte Carlo for Smoother Predictions
Bagging During Markov Chain Monte Carlo for Smoother Predictions Herbert K. H. Lee University of California, Santa Cruz Abstract: Making good predictions from noisy data is a challenging problem. Methods
The University of Auckland Applied Mathematics Bayesian Methods for Inverse Problems : why and how Colin Fox Tiangang Cui, Mike O Sullivan (Auckland),
The University of Auckland Applied Mathematics Bayesian Methods for Inverse Problems : why and how Colin Fox Tiangang Cui, Mike O Sullivan (Auckland), Geoff Nicholls (Statistics, Oxford) fox@math.auckland.ac.nz
Particle-Based Approximate Inference on Graphical Model
article-based Approimate Inference on Graphical Model Reference: robabilistic Graphical Model Ch. 2 Koller & Friedman CMU, 0-708, Fall 2009 robabilistic Graphical Models Lectures 8,9 Eric ing attern Recognition
An Introduction to Bayesian Networks: Representation and Approximate Inference
An Introduction to Bayesian Networks: Representation and Approximate Inference Marek Grześ Department of Computer Science University of York Graphical Models Reading Group May 7, 2009 Data and Probabilities
18 : Advanced topics in MCMC. 1 Gibbs Sampling (Continued from the last lecture)
10-708: Probabilistic Graphical Models 10-708, Spring 2014 18 : Advanced topics in MCMC Lecturer: Eric P. Xing Scribes: Jessica Chemali, Seungwhan Moon 1 Gibbs Sampling (Continued from the last lecture)
A Bayesian Approach to Phylogenetics
A Bayesian Approach to Phylogenetics Niklas Wahlberg Based largely on slides by Paul Lewis (www.eeb.uconn.edu) An Introduction to Bayesian Phylogenetics Bayesian inference in general Markov chain Monte
6 Markov Chain Monte Carlo (MCMC)
6 Markov Chain Monte Carlo (MCMC) The underlying idea in MCMC is to replace the iid samples of basic MC methods, with dependent samples from an ergodic Markov chain, whose limiting (stationary) distribution
Brief Tutorial on Probabilistic Databases
Brief Tutorial on Probabilistic Databases Dan Suciu University of Washington Simons 206 About This Talk Probabilistic databases Tuple-independent Query evaluation Statistical relational models Representation,
An Integer Polynomial Programming Based Framework for Lifted MAP Inference
An Integer Polynomial Programming Based Framework for Lifted MAP Inference Somdeb Sarkhel, Deepak Venugopal Computer Science Department The University of Texas at Dallas {sxs104721,dxv02}@utdallas.edu
STATISTICAL METHODS IN AI/ML Vibhav Gogate The University of Texas at Dallas. Bayesian networks: Representation
STATISTICAL METHODS IN AI/ML Vibhav Gogate The University of Texas at Dallas Bayesian networks: Representation Motivation Explicit representation of the joint distribution is unmanageable Computationally:
Katholieke Universiteit Leuven Department of Computer Science
Inference in Probabilistic Logic Programs using Weighted CNF s Daan Fierens Guy Van den Broeck Ingo Thon Bernd Gutmann Luc De Raedt Report CW 607, June 2011 Katholieke Universiteit Leuven Department of