Sequential Monte Carlo Algorithms
|
|
|
- August Paul
- 5 years ago
- Views:
Transcription
1 ayesian Phylogenetic Inference using Sequential Monte arlo lgorithms lexandre ouchard-ôté *, Sriram Sankararaman *, and Michael I. Jordan *, * omputer Science ivision, University of alifornia erkeley epartment of Statistics, University of alifornia erkeley
2 Phylogenetic tree inference Topic of this talk: integration over the space of trees using Sequential Monte arlo (SM) Motivation: ayesian approach to phylogenetic inference Put a prior on trees, use the posterior for reconstruction Heavy use of integrals over the space of trees: e.g. for handling nuisance parameters, computing minimum risk estimators, ayes factors, etc. Prelude: a parallel with the simpler problem of maximization over the space of trees
3 Maximization over phylogenies Two strategies: Local and sequential search Key difference: representation Local Sequential State t Trees over the observed species
4 Maximization: local strategy Meta-algorithm: 1.Start at arbitrary state
5 Maximization: local strategy Meta-algorithm: 1.Start at arbitrary state 2.Iterate: i. Evaluate neighbors ii. Move to a nearby tree
6 Maximization: local strategy Meta-algorithm: 1.Start at arbitrary state 2.Iterate: i. Evaluate neighbors ii. Move to a nearby tree 3.Return best state visited Example: stochastic annealing
7 Maximization over phylogenies Two strategies: Local and sequential search Key difference: representation Local Sequential State t Trees over the observed species Partial state p Forests over the observed species
8 Maximization over phylogenies Two strategies: Local and sequential search Key difference: representation Local Sequential State t Partial state p Trees over the observed species Forests over the observed species
9 Maximization: sequential strategy Meta-algorithm: 1.Start at the initial, unconstrained partial state =
10 Maximization: sequential strategy Meta-algorithm: 1.Start at the initial, unconstrained partial state 2.Iterate: i. Extend partial state ii. Estimate best successor =
11 Maximization: sequential strategy Meta-algorithm: 1.Start at the initial, unconstrained partial state 2.Iterate: i. Extend partial state ii. Estimate best successor 3.Return best final state = Example: neighbor joining
12 Parallel lassification of phylogenetic algorithms Local strategy Sequential strategy Maximization Stochastic annealing, Neighbor-joining, Integration
13 Parallel lassification of phylogenetic algorithms Local strategy Sequential strategy Maximization Stochastic annealing, Neighbor-joining, Integration MM algorithms???
14 Parallel lassification of phylogenetic algorithms Local strategy Sequential strategy Maximization Stochastic annealing, Neighbor-joining, Integration MM algorithms??? Sequential Monte arlo (SM)
15 Outline ackground: Importance sampling and Sequential Monte arlo SM for phylogenetic inference Framework for designing proposals Experiments: comparisons with MM
16 Preview: omparative advantages SM MM + Trivial to parallelize + Easier to get data likelihood estimate + No burn-in + Easier to resample hyper-parameters + Easier to design proposal distribution
17 Preview: omparative advantages SM MM + Trivial to parallelize + Easier to get data likelihood estimate + No burn-in + Easier to resample hyper-parameters + Easier to design proposal distribution Not exclusive: the two approaches can be combined
18 Phylogenetic setup: ultrametric trees root G.. G.. } Tree T Hidden sequences X E G..} G.. G.. GGT.. TTT.. GT.. GT.. Observations Y = y
19 Phylogenetic setup: ultrametric trees root G.. G.. height } Tree T Hidden sequences X E G..} G.. G.. GGT.. TTT.. GT.. GT.. Observations Y = y
20 Phylogenetic setup: ultrametric trees root G.. G.. height } Tree T Hidden sequences X E G..} G.. G.. GGT.. TTT.. GT.. GT.. Observations Y = y Target distribution: with density: T Y d = π γ(t) Z
21 Phylogenetic setup: ultrametric trees root G.. G.. height } Tree T Hidden sequences X E G..} Joint density evaluated G.. at G.. (t, y) GGT.. TTT.. GT.. GT.., summing over Observations Y = y hidden x Target distribution: with density: T Y d = π γ(t) Z
22 Phylogenetic setup: ultrametric trees root G.. G.. height } Tree T Hidden sequences X E G..} Joint density evaluated G.. at G.. (t, y) GGT.. TTT.. GT.. GT.., summing over Observations Y = y x P(Y = y) T Y = d π hidden Target distribution: ata likelihood (intractable) with density: γ(t) Z
23 Sequential Monte arlo (SM) ackground: Importance Sampling (IS) t1 t2 t3 IS : pproximation for π q 1.Sample trees from a proposal q: ti ~ q
24 Sequential Monte arlo (SM) ackground: Importance Sampling (IS) IS : pproximation for π w1 w2 w3 1.Sample trees from a proposal q: ti ~ q 2. ompute weights w i = γ(t i )/q(t i ) 3. Normalize weights
25 Sequential Monte arlo (SM) ackground: Importance Sampling (IS) IS : pproximation for π w1 w2 w3 1.Sample trees from a proposal q: ti ~ q 2. ompute weights w i = γ(t i )/q(t i ) "Particle" 3. Normalize weights
26 Sequential Monte arlo (SM) ackground: Problem with importance sampling: π is high-dimensional Most particles will have tiny weights
27 Sequential Monte arlo (SM) ackground: Importance Sampling q π γ Z
28 Sequential Monte arlo (SM) ackground: Importance Sampling q π γ Z SM: a sequence of proposals q q q q π 1 π 2 π R = π γ Z
29 Sequential Monte arlo (SM) ackground: Importance Sampling q π γ Z SM: a sequence of proposals q q q q π 1 π 2 π R = π γ Z SM for phylogenies: states (forest) π r are distributions over partial
30 Sequential Monte arlo (SM) 1. Initialize [] π 1
31 Sequential Monte arlo (SM) SM : pproximation for π 1. Initialize [] π 2 2. Iterate : i. Sample partial states p i π 1 q π 1
32 Sequential Monte arlo (SM) SM : pproximation for π 1. Initialize [] π 2 2. Iterate : i. Sample partial states p i π 1 q π 1 p1 p3 p2
33 Sequential Monte arlo (SM) SM : pproximation for π 1. Initialize [] π 2 p 1 p 2 p 3 2. Iterate : i. Sample partial states p i π 1 q π 1 p1 p3 p2
34 Sequential Monte arlo (SM) SM : pproximation for π 1. Initialize [] π 2 2. Iterate : i. Sample partial states w1 w2 w3 p i π 1 q ii. ompute weights w i = γ(p i ) γ(p i ) 1 q(p i p i ) iii. Normalize weights
35 Sequential Monte arlo (SM) π SM : pproximation for π 1. Initialize [] π 2 2. Iterate : i. Sample partial states p i π 1 q π 1 ii. ompute weights w i = γ(p i ) γ(p i ) 1 q(p i p i ) iii. Normalize weights
36 Intuition: why it works asic result: SM is asymptotically consistent p w = γ(p ) γ(p ) 1 q(p p ) p p w = γ(p ) γ(p) 1 q(p p ) w = γ(p) 1 1 q( p)
37 Intuition: why it works asic result: SM is asymptotically consistent p w = γ(p ) γ(p ) 1 q(p p ) p p w = γ(p ) γ(p) 1 q(p p ) w = γ(p) 1 1 q( p) w w w = γ(p ) q( p )
38 Intuition: why it works asic result: SM is asymptotically consistent p w = γ(p ) γ(p ) 1 q(p p ) p p w = γ(p ) γ(p) 1 q(p p ) w = γ(p) 1 1 q( p) w w w = γ(p ) q( p )
39 Intuition: why it works ompare: Weights along a SM path Importance sampling w w w = γ(p ) q( p ) w = γ(t) q(t)
40 esigning a proposal q Issue: Over-counting Two ways of t1 t2 One way of building t2 building t1
41 esigning a proposal q Useful abstraction: q induce a partial order (poset) P
42 esigning a proposal q Useful abstraction: q induce a partial order (poset) P p1 p2 if q can propose a path from p1 to p2
43 esigning a proposal q Useful abstraction: q induce a partial order (poset) P Poset s Hesse diagram:
44 esigning a proposal q Useful abstraction: q induce a partial order (poset) P Poset s Hesse diagram: Use proposal that have tree-shaped Hesse diagrams
45 esigning a proposal q Example: a proposal that has a tree-shaped Hesse diagram. 1. Pick a pair of trees to merge uniformly at random 2. Pick a height for the new tree such that = height( ) < height( )
46 esigning a proposal q t1 t2 = < height( ) height( )
47 Experiments: setup Synthetic-small Synthetic-med Real data Source Generated from the model Subset of HGP Likelihood model rownian motion on frequencies Number of sites ,511 Number of nodes Number of leaves
48 Synthetic experiments Goal: comparison against MM ompetitor: standard MM sampler, 4 tempering chains, shared sum-product implementation Metric: symmetric clade difference of the Minimum ayes Risk reconstructed tree to the generating tree atapoints computed by increasing the number of particles (for SM) and the number of sampling steps (for MM)
49 omparison with MM Synthetic-small Symmetric clade difference SM MM Wall clock time (ms) in logscale
50 omparison with MM Synthetic-medium Symmetric clade difference MM SM x10 6 1x10 7 Wall clock time (ms) in logscale
51 Experiments on real data Goal: show that the method scales to large number of sites Number of particle (10,000) determined using synthetic experiments, timing experiments with different numbers of cores: Wall clock time (ms) Number of cores
52 onclusion SM can be applied to a wide range of phylogenetic models; previous work limited to oalescent priors [Teh et al. 07] Order theoretic framework for designing proposals Experiments: There are regimes where SM outperforms MM Promising applications of SM in phylogenetic inference: 1. Quickly analyze large datasets 2. Initialization and large step proposal for MM chains
15-780: Graduate Artificial Intelligence. Bayesian networks: Construction and inference
15-780: Graduate Artificial Intelligence ayesian networks: Construction and inference ayesian networks: Notations ayesian networks are directed acyclic graphs. Conditional probability tables (CPTs) P(Lo)
STA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 7 Approximate
Bayesian Classification and Regression Trees
Bayesian Classification and Regression Trees James Cussens York Centre for Complex Systems Analysis & Dept of Computer Science University of York, UK 1 Outline Problems for Lessons from Bayesian phylogeny
Sampling Rejection Sampling Importance Sampling Markov Chain Monte Carlo. Sampling Methods. Machine Learning. Torsten Möller.
Sampling Methods Machine Learning orsten Möller Möller/Mori 1 Recall Inference or General Graphs Junction tree algorithm is an exact inference method for arbitrary graphs A particular tree structure defined
Inference in Graphical Models Variable Elimination and Message Passing Algorithm
Inference in Graphical Models Variable Elimination and Message Passing lgorithm Le Song Machine Learning II: dvanced Topics SE 8803ML, Spring 2012 onditional Independence ssumptions Local Markov ssumption
STA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 11 Project
Sampling Methods. Bishop PRML Ch. 11. Alireza Ghane. Sampling Rejection Sampling Importance Sampling Markov Chain Monte Carlo
Sampling Methods Bishop PRML h. 11 Alireza Ghane Sampling Methods A. Ghane /. Möller / G. Mori 1 Recall Inference or General Graphs Junction tree algorithm is an exact inference method for arbitrary graphs
Tree of Life iological Sequence nalysis Chapter http://tolweb.org/tree/ Phylogenetic Prediction ll organisms on Earth have a common ancestor. ll species are related. The relationship is called a phylogeny
Modern Phylogenetics. An Introduction to Phylogenetics. Phylogenetics and Systematics. Phylogenetic Tree of Whales
Modern Phylogenetics n Introduction to Phylogenetics ret Larget larget@stat.wisc.edu epartments of otany and of Statistics University of Wisconsin Madison January 27, 2010 Phylogenies are usually estimated
STA 414/2104: Machine Learning
STA 414/2104: Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistics! rsalakhu@cs.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 9 Sequential Data So far
Bayes Networks. CS540 Bryan R Gibson University of Wisconsin-Madison. Slides adapted from those used by Prof. Jerry Zhu, CS540-1
Bayes Networks CS540 Bryan R Gibson University of Wisconsin-Madison Slides adapted from those used by Prof. Jerry Zhu, CS540-1 1 / 59 Outline Joint Probability: great for inference, terrible to obtain
Bayes Net Representation. CS 188: Artificial Intelligence. Approximate Inference: Sampling. Variable Elimination. Sampling.
188: Artificial Intelligence Bayes Nets: ampling Bayes Net epresentation A directed, acyclic graph, one node per random variable A conditional probability table (PT) for each node A collection of distributions
9/30/11. Evolution theory. Phylogenetic Tree Reconstruction. Phylogenetic trees (binary trees) Phylogeny (phylogenetic tree)
I9 Introduction to Bioinformatics, 0 Phylogenetic ree Reconstruction Yuzhen Ye (yye@indiana.edu) School of Informatics & omputing, IUB Evolution theory Speciation Evolution of new organisms is driven by
9 Forward-backward algorithm, sum-product on factor graphs
Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.438 Algorithms For Inference Fall 2014 9 Forward-backward algorithm, sum-product on factor graphs The previous
Phylogenetics. Applications of phylogenetics. Unrooted networks vs. rooted trees. Outline
Phylogenetics Todd Vision iology 522 March 26, 2007 pplications of phylogenetics Studying organismal or biogeographic history Systematics ating events in the fossil record onservation biology Studying
Bayesian Phylogenetics:
Bayesian Phylogenetics: an introduction Marc A. Suchard msuchard@ucla.edu UCLA Who is this man? How sure are you? The one true tree? Methods we ve learned so far try to find a single tree that best describes
Bayesian Inference and Traffic Analysis
ayesian Inference and Traffic nalysis armela Troncoso George Danezis September-November 008 Microsoft Research ambridge/ KU Leuven(OSI nonymous ommunications Tell me who your friends are... => nonymous
Study Notes on the Latent Dirichlet Allocation
Study Notes on the Latent Dirichlet Allocation Xugang Ye 1. Model Framework A word is an element of dictionary {1,,}. A document is represented by a sequence of words: =(,, ), {1,,}. A corpus is a collection
Theory of Evolution Charles Darwin
Theory of Evolution Charles arwin 858-59: Origin of Species 5 year voyage of H.M.S. eagle (83-36) Populations have variations. Natural Selection & Survival of the fittest: nature selects best adapted varieties
Monte Carlo Methods. Geoff Gordon February 9, 2006
Monte Carlo Methods Geoff Gordon ggordon@cs.cmu.edu February 9, 2006 Numerical integration problem 5 4 3 f(x,y) 2 1 1 0 0.5 0 X 0.5 1 1 0.8 0.6 0.4 Y 0.2 0 0.2 0.4 0.6 0.8 1 x X f(x)dx Used for: function
Bayesian Phylogenetics
Bayesian Phylogenetics Bret Larget Departments of Botany and of Statistics University of Wisconsin Madison October 6, 2011 Bayesian Phylogenetics 1 / 27 Who was Bayes? The Reverand Thomas Bayes was born
Undirected Graphical Models
Outline Hong Chang Institute of Computing Technology, Chinese Academy of Sciences Machine Learning Methods (Fall 2012) Outline Outline I 1 Introduction 2 Properties Properties 3 Generative vs. Conditional
Conditional probabilities and graphical models
Conditional probabilities and graphical models Thomas Mailund Bioinformatics Research Centre (BiRC), Aarhus University Probability theory allows us to describe uncertainty in the processes we model within
A008 THE PROBABILITY PERTURBATION METHOD AN ALTERNATIVE TO A TRADITIONAL BAYESIAN APPROACH FOR SOLVING INVERSE PROBLEMS
A008 THE PROAILITY PERTURATION METHOD AN ALTERNATIVE TO A TRADITIONAL AYESIAN APPROAH FOR SOLVING INVERSE PROLEMS Jef AERS Stanford University, Petroleum Engineering, Stanford A 94305-2220 USA Abstract
Molecular Evolution & Phylogenetics
Molecular Evolution & Phylogenetics Heuristics based on tree alterations, maximum likelihood, Bayesian methods, statistical confidence measures Jean-Baka Domelevo Entfellner Learning Objectives know basic
Hierarchical Nearest-Neighbor Gaussian Process Models for Large Geo-statistical Datasets
Hierarchical Nearest-Neighbor Gaussian Process Models for Large Geo-statistical Datasets Abhirup Datta 1 Sudipto Banerjee 1 Andrew O. Finley 2 Alan E. Gelfand 3 1 University of Minnesota, Minneapolis,
Who was Bayes? Bayesian Phylogenetics. What is Bayes Theorem?
Who was Bayes? Bayesian Phylogenetics Bret Larget Departments of Botany and of Statistics University of Wisconsin Madison October 6, 2011 The Reverand Thomas Bayes was born in London in 1702. He was the
Week 7: Bayesian inference, Testing trees, Bootstraps
Week 7: ayesian inference, Testing trees, ootstraps Genome 570 May, 2008 Week 7: ayesian inference, Testing trees, ootstraps p.1/54 ayes Theorem onditional probability of hypothesis given data is: Prob
Stochastic Simulation
Stochastic Simulation Idea: probabilities samples Get probabilities from samples: X count x 1 n 1. x k total. n k m X probability x 1. n 1 /m. x k n k /m If we could sample from a variable s (posterior)
Big model configuration with Bayesian quadrature. David Duvenaud, Roman Garnett, Tom Gunter, Philipp Hennig, Michael A Osborne and Stephen Roberts.
Big model configuration with Bayesian quadrature David Duvenaud, Roman Garnett, Tom Gunter, Philipp Hennig, Michael A Osborne and Stephen Roberts. This talk will develop Bayesian quadrature approaches
Monte Carlo in Bayesian Statistics
Monte Carlo in Bayesian Statistics Matthew Thomas SAMBa - University of Bath m.l.thomas@bath.ac.uk December 4, 2014 Matthew Thomas (SAMBa) Monte Carlo in Bayesian Statistics December 4, 2014 1 / 16 Overview
Announcements. CS 188: Artificial Intelligence Spring Bayes Net Semantics. Probabilities in BNs. All Conditional Independences
CS 188: Artificial Intelligence Spring 2011 Announcements Assignments W4 out today --- this is your last written!! Any assignments you have not picked up yet In bin in 283 Soda [same room as for submission
April 20th, Advanced Topics in Machine Learning California Institute of Technology. Markov Chain Monte Carlo for Machine Learning
for for Advanced Topics in California Institute of Technology April 20th, 2017 1 / 50 Table of Contents for 1 2 3 4 2 / 50 History of methods for Enrico Fermi used to calculate incredibly accurate predictions
Bayesian Inference using Markov Chain Monte Carlo in Phylogenetic Studies
Bayesian Inference using Markov Chain Monte Carlo in Phylogenetic Studies 1 What is phylogeny? Essay written for the course in Markov Chains 2004 Torbjörn Karfunkel Phylogeny is the evolutionary development
Sampling Rejection Sampling Importance Sampling Markov Chain Monte Carlo. Sampling Methods. Oliver Schulte - CMPT 419/726. Bishop PRML Ch.
Sampling Methods Oliver Schulte - CMP 419/726 Bishop PRML Ch. 11 Recall Inference or General Graphs Junction tree algorithm is an exact inference method for arbitrary graphs A particular tree structure
Inference in Bayesian networks
Inference in ayesian networks Devika uramanian omp 440 Lecture 7 xact inference in ayesian networks Inference y enumeration The variale elimination algorithm c Devika uramanian 2006 2 1 roailistic inference
Phylogenetics: Bayesian Phylogenetic Analysis. COMP Spring 2015 Luay Nakhleh, Rice University
Phylogenetics: Bayesian Phylogenetic Analysis COMP 571 - Spring 2015 Luay Nakhleh, Rice University Bayes Rule P(X = x Y = y) = P(X = x, Y = y) P(Y = y) = P(X = x)p(y = y X = x) P x P(X = x 0 )P(Y = y X
Adaptive Crowdsourcing via EM with Prior
Adaptive Crowdsourcing via EM with Prior Peter Maginnis and Tanmay Gupta May, 205 In this work, we make two primary contributions: derivation of the EM update for the shifted and rescaled beta prior and
CSE 417T: Introduction to Machine Learning. Final Review. Henry Chai 12/4/18
CSE 417T: Introduction to Machine Learning Final Review Henry Chai 12/4/18 Overfitting Overfitting is fitting the training data more than is warranted Fitting noise rather than signal 2 Estimating! "#$
Evaluation Methods for Topic Models
University of Massachusetts Amherst wallach@cs.umass.edu April 13, 2009 Joint work with Iain Murray, Ruslan Salakhutdinov and David Mimno Statistical Topic Models Useful for analyzing large, unstructured
EVOLUTIONARY DISTANCES
EVOLUTIONARY DISTANCES FROM STRINGS TO TREES Luca Bortolussi 1 1 Dipartimento di Matematica ed Informatica Università degli studi di Trieste luca@dmi.units.it Trieste, 14 th November 2007 OUTLINE 1 STRINGS:
Bayes factors, marginal likelihoods, and how to choose a population model
ayes factors, marginal likelihoods, and how to choose a population model Peter eerli Department of Scientific omputing Florida State University, Tallahassee Overview 1. Location versus Population 2. ayes
Theory of Evolution. Charles Darwin
Theory of Evolution harles arwin 858-59: Origin of Species 5 year voyage of H.M.S. eagle (8-6) Populations have variations. Natural Selection & Survival of the fittest: nature selects best adapted varieties
Phylogenetic Tree Reconstruction
I519 Introduction to Bioinformatics, 2011 Phylogenetic Tree Reconstruction Yuzhen Ye (yye@indiana.edu) School of Informatics & Computing, IUB Evolution theory Speciation Evolution of new organisms is driven
Structure Learning in Sequential Data
Structure Learning in Sequential Data Liam Stewart liam@cs.toronto.edu Richard Zemel zemel@cs.toronto.edu 2005.09.19 Motivation. Cau, R. Kuiper, and W.-P. de Roever. Formalising Dijkstra's development
Bayesian Methods for Machine Learning
Bayesian Methods for Machine Learning CS 584: Big Data Analytics Material adapted from Radford Neal s tutorial (http://ftp.cs.utoronto.ca/pub/radford/bayes-tut.pdf), Zoubin Ghahramni (http://hunch.net/~coms-4771/zoubin_ghahramani_bayesian_learning.pdf),
Markov chain Monte-Carlo to estimate speciation and extinction rates: making use of the forest hidden behind the (phylogenetic) tree
Markov chain Monte-Carlo to estimate speciation and extinction rates: making use of the forest hidden behind the (phylogenetic) tree Nicolas Salamin Department of Ecology and Evolution University of Lausanne
Strong Lens Modeling (II): Statistical Methods
Strong Lens Modeling (II): Statistical Methods Chuck Keeton Rutgers, the State University of New Jersey Probability theory multiple random variables, a and b joint distribution p(a, b) conditional distribution
Recent Advances in Bayesian Inference Techniques
Recent Advances in Bayesian Inference Techniques Christopher M. Bishop Microsoft Research, Cambridge, U.K. research.microsoft.com/~cmbishop SIAM Conference on Data Mining, April 2004 Abstract Bayesian
MCMC and Gibbs Sampling. Kayhan Batmanghelich
MCMC and Gibbs Sampling Kayhan Batmanghelich 1 Approaches to inference l Exact inference algorithms l l l The elimination algorithm Message-passing algorithm (sum-product, belief propagation) The junction
Introduction to Machine Learning CMU-10701
Introduction to Machine Learning CMU-10701 Markov Chain Monte Carlo Methods Barnabás Póczos & Aarti Singh Contents Markov Chain Monte Carlo Methods Goal & Motivation Sampling Rejection Importance Markov
Kernel adaptive Sequential Monte Carlo
Kernel adaptive Sequential Monte Carlo Ingmar Schuster (Paris Dauphine) Heiko Strathmann (University College London) Brooks Paige (Oxford) Dino Sejdinovic (Oxford) December 7, 2015 1 / 36 Section 1 Outline
Markov Chain Monte Carlo Methods
Markov Chain Monte Carlo Methods Sargur Srihari srihari@cedar.buffalo.edu 1 Topics Limitations of Likelihood Weighting Gibbs Sampling Algorithm Markov Chains Gibbs Sampling Revisited A broader class of
Phylogenetic Inference via Sequential Monte Carlo
Syst. Biol. 6(4):579 593, 202 c The Author(s) 202. Published by Oxford University Press, on behalf of the Society of Systematic Biologists. All rights reserved. This is an Open Access article distributed
Chapter 3: Phylogenetics
Chapter 3: Phylogenetics 3. Computing Phylogeny Prof. Yechiam Yemini (YY) Computer Science epartment Columbia niversity Overview Computing trees istance-based techniques Maximal Parsimony (MP) techniques
Link Prediction. Eman Badr Mohammed Saquib Akmal Khan
Link Prediction Eman Badr Mohammed Saquib Akmal Khan 11-06-2013 Link Prediction Which pair of nodes should be connected? Applications Facebook friend suggestion Recommendation systems Monitoring and controlling
Learning Energy-Based Models of High-Dimensional Data
Learning Energy-Based Models of High-Dimensional Data Geoffrey Hinton Max Welling Yee-Whye Teh Simon Osindero www.cs.toronto.edu/~hinton/energybasedmodelsweb.htm Discovering causal structure as a goal
MCMC: Markov Chain Monte Carlo
I529: Machine Learning in Bioinformatics (Spring 2013) MCMC: Markov Chain Monte Carlo Yuzhen Ye School of Informatics and Computing Indiana University, Bloomington Spring 2013 Contents Review of Markov
Time-Sensitive Dirichlet Process Mixture Models
Time-Sensitive Dirichlet Process Mixture Models Xiaojin Zhu Zoubin Ghahramani John Lafferty May 25 CMU-CALD-5-4 School of Computer Science Carnegie Mellon University Pittsburgh, PA 523 Abstract We introduce
Annealing Between Distributions by Averaging Moments
Annealing Between Distributions by Averaging Moments Chris J. Maddison Dept. of Comp. Sci. University of Toronto Roger Grosse CSAIL MIT Ruslan Salakhutdinov University of Toronto Partition Functions We
27 : Distributed Monte Carlo Markov Chain. 1 Recap of MCMC and Naive Parallel Gibbs Sampling
10-708: Probabilistic Graphical Models 10-708, Spring 2014 27 : Distributed Monte Carlo Markov Chain Lecturer: Eric P. Xing Scribes: Pengtao Xie, Khoa Luu In this scribe, we are going to review the Parallel
Announcements. CS 188: Artificial Intelligence Fall Causality? Example: Traffic. Topology Limits Distributions. Example: Reverse Traffic
CS 188: Artificial Intelligence Fall 2008 Lecture 16: Bayes Nets III 10/23/2008 Announcements Midterms graded, up on glookup, back Tuesday W4 also graded, back in sections / box Past homeworks in return
The Expectation Maximization Algorithm
The Expectation Maximization Algorithm Frank Dellaert College of Computing, Georgia Institute of Technology Technical Report number GIT-GVU-- February Abstract This note represents my attempt at explaining
CS Lecture 3. More Bayesian Networks
CS 6347 Lecture 3 More Bayesian Networks Recap Last time: Complexity challenges Representing distributions Computing probabilities/doing inference Introduction to Bayesian networks Today: D-separation,
Bayesian networks: approximate inference
Bayesian networks: approximate inference Machine Intelligence Thomas D. Nielsen September 2008 Approximative inference September 2008 1 / 25 Motivation Because of the (worst-case) intractability of exact
CSCI-567: Machine Learning (Spring 2019)
CSCI-567: Machine Learning (Spring 2019) Prof. Victor Adamchik U of Southern California Mar. 19, 2019 March 19, 2019 1 / 43 Administration March 19, 2019 2 / 43 Administration TA3 is due this week March
MCMC for big data. Geir Storvik. BigInsight lunch - May Geir Storvik MCMC for big data BigInsight lunch - May / 17
MCMC for big data Geir Storvik BigInsight lunch - May 2 2018 Geir Storvik MCMC for big data BigInsight lunch - May 2 2018 1 / 17 Outline Why ordinary MCMC is not scalable Different approaches for making
Example: multivariate Gaussian Distribution
School of omputer Science Probabilistic Graphical Models Representation of undirected GM (continued) Eric Xing Lecture 3, September 16, 2009 Reading: KF-chap4 Eric Xing @ MU, 2005-2009 1 Example: multivariate
Decentralized decision making with spatially distributed data
Decentralized decision making with spatially distributed data XuanLong Nguyen Department of Statistics University of Michigan Acknowledgement: Michael Jordan, Martin Wainwright, Ram Rajagopal, Pravin Varaiya
Hidden Markov Models for precipitation
Hidden Markov Models for precipitation Pierre Ailliot Université de Brest Joint work with Peter Thomson Statistics Research Associates (NZ) Page 1 Context Part of the project Climate-related risks for
CS 343: Artificial Intelligence
CS 343: Artificial Intelligence Bayes Nets: Sampling Prof. Scott Niekum The University of Texas at Austin [These slides based on those of Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley.
(5) Multi-parameter models - Gibbs sampling. ST440/540: Applied Bayesian Analysis
Summarizing a posterior Given the data and prior the posterior is determined Summarizing the posterior gives parameter estimates, intervals, and hypothesis tests Most of these computations are integrals
CPSC 540: Machine Learning
CPSC 540: Machine Learning MCMC and Non-Parametric Bayes Mark Schmidt University of British Columbia Winter 2016 Admin I went through project proposals: Some of you got a message on Piazza. No news is
Replicated Softmax: an Undirected Topic Model. Stephen Turner
Replicated Softmax: an Undirected Topic Model Stephen Turner 1. Introduction 2. Replicated Softmax: A Generative Model of Word Counts 3. Evaluating Replicated Softmax as a Generative Model 4. Experimental
Simulated Annealing for Constrained Global Optimization
Monte Carlo Methods for Computation and Optimization Final Presentation Simulated Annealing for Constrained Global Optimization H. Edwin Romeijn & Robert L.Smith (1994) Presented by Ariel Schwartz Objective
CSCE 478/878 Lecture 6: Bayesian Learning and Graphical Models. Stephen Scott. Introduction. Outline. Bayes Theorem. Formulas
ian ian ian Might have reasons (domain information) to favor some hypotheses/predictions over others a priori ian methods work with probabilities, and have two main roles: Naïve Nets (Adapted from Ethem
Decision Tree Fields
Sebastian Nowozin, arsten Rother, Shai agon, Toby Sharp, angpeng Yao, Pushmeet Kohli arcelona, 8th November 2011 Introduction Random Fields in omputer Vision Markov Random Fields (MRF) (Kindermann and
Biol 206/306 Advanced Biostatistics Lab 12 Bayesian Inference
Biol 206/306 Advanced Biostatistics Lab 12 Bayesian Inference By Philip J. Bergmann 0. Laboratory Objectives 1. Learn what Bayes Theorem and Bayesian Inference are 2. Reinforce the properties of Bayesian
Artificial Intelligence
ICS461 Fall 2010 Nancy E. Reed nreed@hawaii.edu 1 Lecture #14B Outline Inference in Bayesian Networks Exact inference by enumeration Exact inference by variable elimination Approximate inference by stochastic
Basic Sampling Methods
Basic Sampling Methods Sargur Srihari srihari@cedar.buffalo.edu 1 1. Motivation Topics Intractability in ML How sampling can help 2. Ancestral Sampling Using BNs 3. Transforming a Uniform Distribution
Algorithms in Bioinformatics
Algorithms in Bioinformatics Sami Khuri Department of Computer Science San José State University San José, California, USA khuri@cs.sjsu.edu www.cs.sjsu.edu/faculty/khuri Distance Methods Character Methods
Probabilistic Graphical Models
Probabilistic Graphical Models Brown University CSCI 295-P, Spring 213 Prof. Erik Sudderth Lecture 11: Inference & Learning Overview, Gaussian Graphical Models Some figures courtesy Michael Jordan s draft
CS 188: Artificial Intelligence. Bayes Nets
CS 188: Artificial Intelligence Probabilistic Inference: Enumeration, Variable Elimination, Sampling Pieter Abbeel UC Berkeley Many slides over this course adapted from Dan Klein, Stuart Russell, Andrew
VCMC: Variational Consensus Monte Carlo
VCMC: Variational Consensus Monte Carlo Maxim Rabinovich, Elaine Angelino, Michael I. Jordan Berkeley Vision and Learning Center September 22, 2015 probabilistic models! sky fog bridge water grass object
Shared Segmentation of Natural Scenes. Dependent Pitman-Yor Processes
Shared Segmentation of Natural Scenes using Dependent Pitman-Yor Processes Erik Sudderth & Michael Jordan University of California, Berkeley Parsing Visual Scenes sky skyscraper sky dome buildings trees
Multimodal Nested Sampling
Multimodal Nested Sampling Farhan Feroz Astrophysics Group, Cavendish Lab, Cambridge Inverse Problems & Cosmology Most obvious example: standard CMB data analysis pipeline But many others: object detection,
PROBABILISTIC REASONING SYSTEMS
PROBABILISTIC REASONING SYSTEMS In which we explain how to build reasoning systems that use network models to reason with uncertainty according to the laws of probability theory. Outline Knowledge in uncertain
6 Markov Chain Monte Carlo (MCMC)
6 Markov Chain Monte Carlo (MCMC) The underlying idea in MCMC is to replace the iid samples of basic MC methods, with dependent samples from an ergodic Markov chain, whose limiting (stationary) distribution
CSE 150. Assignment 6 Summer Maximum likelihood estimation. Out: Thu Jul 14 Due: Tue Jul 19
SE 150. Assignment 6 Summer 2016 Out: Thu Jul 14 ue: Tue Jul 19 6.1 Maximum likelihood estimation A (a) omplete data onsider a complete data set of i.i.d. examples {a t, b t, c t, d t } T t=1 drawn from
Web Structure Mining Nodes, Links and Influence
Web Structure Mining Nodes, Links and Influence 1 Outline 1. Importance of nodes 1. Centrality 2. Prestige 3. Page Rank 4. Hubs and Authority 5. Metrics comparison 2. Link analysis 3. Influence model 1.
Sum-Product Networks: A New Deep Architecture
Sum-Product Networks: A New Deep Architecture Pedro Domingos Dept. Computer Science & Eng. University of Washington Joint work with Hoifung Poon 1 Graphical Models: Challenges Bayesian Network Markov Network
Discrete & continuous characters: The threshold model
Discrete & continuous characters: The threshold model Discrete & continuous characters: the threshold model So far we have discussed continuous & discrete character models separately for estimating ancestral
A new iterated filtering algorithm
A new iterated filtering algorithm Edward Ionides University of Michigan, Ann Arbor ionides@umich.edu Statistics and Nonlinear Dynamics in Biology and Medicine Thursday July 31, 2014 Overview 1 Introduction
Vers un apprentissage subquadratique pour les mélanges d arbres
Vers un apprentissage subquadratique pour les mélanges d arbres F. Schnitzler 1 P. Leray 2 L. Wehenkel 1 fschnitzler@ulg.ac.be 1 Université deliège 2 Université de Nantes 10 mai 2010 F. Schnitzler (ULG)
Integrative Biology 200 "PRINCIPLES OF PHYLOGENETICS" Spring 2018 University of California, Berkeley
Integrative Biology 200 "PRINCIPLES OF PHYLOGENETICS" Spring 2018 University of California, Berkeley B.D. Mishler Feb. 14, 2018. Phylogenetic trees VI: Dating in the 21st century: clocks, & calibrations;
Latent Variable Models
Latent Variable Models Stefano Ermon, Aditya Grover Stanford University Lecture 5 Stefano Ermon, Aditya Grover (AI Lab) Deep Generative Models Lecture 5 1 / 31 Recap of last lecture 1 Autoregressive models:
Algorithmic Methods Well-defined methodology Tree reconstruction those that are well-defined enough to be carried out by a computer. Felsenstein 2004,
Tracing the Evolution of Numerical Phylogenetics: History, Philosophy, and Significance Adam W. Ferguson Phylogenetic Systematics 26 January 2009 Inferring Phylogenies Historical endeavor Darwin- 1837
Learning Sequence Motif Models Using Expectation Maximization (EM) and Gibbs Sampling
Learning Sequence Motif Models Using Expectation Maximization (EM) and Gibbs Sampling BMI/CS 776 www.biostat.wisc.edu/bmi776/ Spring 009 Mark Craven craven@biostat.wisc.edu Sequence Motifs what is a sequence
Lecture 13 Fundamentals of Bayesian Inference
Lecture 13 Fundamentals of Bayesian Inference Dennis Sun Stats 253 August 11, 2014 Outline of Lecture 1 Bayesian Models 2 Modeling Correlations Using Bayes 3 The Universal Algorithm 4 BUGS 5 Wrapping Up
Bayesian Networks BY: MOHAMAD ALSABBAGH
Bayesian Networks BY: MOHAMAD ALSABBAGH Outlines Introduction Bayes Rule Bayesian Networks (BN) Representation Size of a Bayesian Network Inference via BN BN Learning Dynamic BN Introduction Conditional