# Approximate Inference using MCMC

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Approximate Inference using MCMC Class 22 Ruslan Salakhutdinov BCS and CSAIL, MIT 1

2 Plan 1. Introduction/Notation. 2. Examples of successful Bayesian models. 3. Basic Sampling Algorithms. 4. Markov chains. 5. Markov chain Monte Carlo algorithms. 2

3 References/Acknowledgements Chris Bishop s book: Pattern Recognition and Machine Learning, chapter 11 (many figures are borrowed from this book). David MacKay s book: Information Theory, Inference, and Learning Algorithms, chapters Radford Neals s technical report on Probabilistic Inference Using Markov Chain Monte Carlo Methods. Zoubin Ghahramani s ICML tutorial on Bayesian Machine Learning: zoubin/icml04-tutorial.html Ian Murray s tutorial on Sampling Methods: murray/teaching/ 3

4 Basic Notation P(x) P(x θ) P(x,θ) probability of x conditional probability of x given θ joint probability of x and θ Bayes Rule: P(θ x) = P(x θ)p(θ) P(x) where P(x) = P(x, θ)dθ Marginalization I will use probability distribution and probability density interchangeably. It should be obvious from the context. 4

5 Inference Problem Given a dataset D = {x 1,...,x n }: Bayes Rule: P(θ D) = P(D θ)p(θ) P(D) P(D θ) P(θ) P(θ D) Likelihood function of θ Prior probability of θ Posterior distribution over θ Computing posterior distribution is known as the inference problem. But: P(D) = P(D,θ)dθ This integral can be very high-dimensional and difficult to compute. 5

6 Prediction P(θ D) = P(D θ)p(θ) P(D) P(D θ) P(θ) P(θ D) Likelihood function of θ Prior probability of θ Posterior distribution over θ Prediction: Given D, computing conditional probability of x requires computing the following integral: P(x D) = P(x θ, D)P(θ D)dθ = E P(θ D) [P(x θ, D)] which is sometimes called predictive distribution. Computing predictive distribution requires posterior P(θ D). 6

7 Model Selection Compare model classes, e.g. M 1 and M 2. Need to compute posterior probabilities given D: P(M D) = P(D M)P(M) P(D) where P(D M) = P(D θ, M)P(θ, M)dθ is known as the marginal likelihood or evidence. 7

8 Computational Challenges Computing marginal likelihoods often requires computing very highdimensional integrals. Computing posterior distributions (and hence predictive distributions) is often analytically intractable. In this class, we will concentrate on Markov Chain Monte Carlo (MCMC) methods for performing approximate inference. First, let us look at some specific examples: Bayesian Probabilistic Matrix Factorization Bayesian Neural Networks Dirichlet Process Mixtures (last class) 8

9 Bayesian PMF User Features ? ? 4? R ~ U V Movie Features We have N users, M movies, and integer rating values from 1 to K. Let r ij be the rating of user i for movie j, and U R D N, V R D M be latent user and movie feature matrices: Goal: Predict missing ratings. R U V 9

10 Bayesian PMF α V α U Probabilistic linear model with Gaussian observation noise. Likelihood: Θ V Θ U p(r ij u i, v j, σ 2 ) = N(r ij u i v j,σ 2 ) Vj U i Gaussian Priors over parameters: p(u µ U,Λ U ) = N N(u i µ u, Σ u ), j=1,...,m R ij i=1,...,n i=1 M p(v µ V, Λ V ) = N(v i µ v, Σ v ). σ i=1 Conjugate Gaussian-inverse-Wishart priors on the user and movie hyperparameters Θ U = {µ u,σ u } and Θ V = {µ v,σ v }. Hierarchical Prior. 10

11 Bayesian PMF Predictive distribution: Consider predicting a rating rij and query movie j: p(r ij R) = for user i p(rij u i,v j )p(u,v,θ }{{ U,Θ V R) } d{u,v }d{θ U,Θ V } Posterior over parameters and hyperparameters Exact evaluation of this predictive distribution is analytically intractable. Posterior distribution p(u,v,θ U,Θ V R) is complicated and does not have a closed form expression. Need to approximate. 11

12 Bayesian Neural Nets Regression problem: Given a set of i.i.d observations X = {x n } N n=1 with corresponding targets D = {t n } N n=1. x D w (1) MD hidden units z M w (2) KM Likelihood: p(d X,w) = N N(t n y(x n,w),β 2 ) inputs x 1 x 0 z 1 z 0 w (2) 10 y K outputs y 1 n=1 The mean is given by the output of the neural network: M y k (x,w) = wkjσ 2 ( D wjix 1 ) i j=0 i=0 where σ(x) is the sigmoid function. Gaussian prior over the network parameters: p(w) = N(0, α 2 I). 12

13 Bayesian Neural Nets Likelihood: hidden units N x D w (1) MD z M w (2) KM y K p(d X,w) = n=1 N(t n y(x n,w),β 2 ) Gaussian prior over parameters: inputs outputs y 1 p(w) = N(0, α 2 I) x 1 x 0 z 1 w (2) 10 Posterior is analytically intractable: p(w D,X) = p(d w,x)p(w) p(d w,x)p(w)dw z 0 Remark: Under certain conditions, Radford Neal (1994) showed, as the number of hidden units go to infinity, a Gaussian prior over parameters results in a Gaussian process prior for functions. 13

14 Undirected Models x is a binary random vector with x i {+1, 1}: p(x) = 1 Z exp( (i,j) E θ ij x i x j + i V θ i x i ). where Z is known as partition function: Z = x exp ( (i,j) E θ ij x i x j + i V θ i x i ). If x is 100-dimensional, need to sum over terms. The sum might decompose (e.g. junction tree). Otherwise we need to approximate. Remark: Compare to marginal likelihood. 14

15 Monte Carlo p(z) f(z) For most situations we will be interested in evaluating the expectation: E[f] = f(z)p(z)dz z We will use the following notation: p(z) = p(z) Z. We can evaluate p(z) pointwise, but cannot evaluate Z. Posterior distribution: P(θ D) = 1 P(D) P(D θ)p(θ) Markov random fields: P(z) = 1 Z exp( E(z)) 15

16 Simple Monte Carlo General Idea: Draw independent samples {z 1,...,z n } from distribution p(z) to approximate expectation: E[f] = f(z)p(z)dz 1 N N f(z n ) = ˆf n=1 Note that E[f] = E[ ˆf], so the estimator ˆf has correct mean (unbiased). The variance: var[ ˆf] = 1 N E[ (f E[f]) 2] Remark: The accuracy of the estimator does not depend on dimensionality of z. 16

17 Simple Monte Carlo In general: f(z)p(z)dz 1 N N f(z n ), z n p(z) n=1 Predictive distribution: P(x D) = P(x θ, D)P(θ D)dθ 1 N N P(x θ n, D), θ n p(θ D) n=1 Problem: It is hard to draw exact samples from p(z). 17

18 Basic Sampling Algorithm How to generate samples from simple non-uniform distributions assuming we can generate samples from uniform distribution. 1 h(y) Define: h(y) = y p(ŷ)dŷ p(y) Sample: z U[0,1]. Then: y = h 1 (z) is a sample from p(y). 0 y Problem: Computing cumulative h(y) is just as hard! 18

19 Rejection Sampling Sampling from target distribution p(z) = p(z)/z p is difficult. Suppose we have an easy-to-sample proposal distribution q(z), such that kq(z) p(z), z. kq(z 0 ) kq(z) Sample z 0 from q(z). Sample u 0 from Uniform[0,kq(z 0 )] z 0 u 0 p(z) z The pair (z 0,u 0 ) has uniform distribution under the curve of kq(z). If u 0 > p(z 0 ), the sample is rejected. 19

20 Rejection Sampling Probability that a sample is accepted is: kq(z 0 ) u 0 kq(z) p(z) p(accept) = = 1 k p(z) kq(z) q(z)dz p(z)dz z 0 z The fraction of accepted samples depends on the ratio of the area under p(z) and kq(z). Hard to find appropriate q(z) with optimal k. Useful technique in one or two dimensions. Typically applied as a subroutine in more advanced algorithms. 20

21 Importance Sampling Suppose we have an easy-to-sample proposal distribution q(z), such that q(z) > 0 if p(z) > 0. p(z) q(z) f(z) E[f] = f(z)p(z)dz z = 1 N f(z) p(z) q(z) q(z)dz p(z n ) q(z n ) f(zn ), z n q(z) n The quantities w n = p(zn )/q(z n ) are known as importance weights. Unlike rejection sampling, all samples are retained. But wait: we cannot compute p(z), only p(z). 21

22 Importance Sampling Let our proposal be of the form q(z) = q(z)/z q : E[f] = Z q 1 Z p N f(z)p(z)dz = n p(z n ) q(z n ) f(zn ) = Z q Z p 1 N f(z) p(z) q(z) q(z)dz = Z q Z p w n f(z n ), n f(z) p(z) q(z) q(z)dz z n q(z) But we can use the same importance weights to approximate Z p Z q : Z p Z q = 1 Z q p(z)dz = p(z) q(z) q(z)dz 1 N n p(z n ) q(z n ) = 1 N n w n Hence: E[f] 1 N n w n n wnf(zn ) Consistent but biased. 22

23 Problems If our proposal distribution q(z) poorly matches our target distribution p(z) then: Rejection Sampling: almost always rejects Importance Sampling: has large, possibly infinite, variance (unreliable estimator). For high-dimensional problems, finding good proposal distributions is very hard. What can we do? Markov Chain Monte Carlo. 23

24 Markov Chains A first-order Markov chain: a series of random variables {z 1,...,z N } such that the following conditional independence property holds for n {z 1,...,z N 1 }: p(z n+1 z 1,...,z n ) = p(z n+1 z n ) We can specify Markov chain: probability distribution for initial state p(z 1 ). conditional probability for subsequent states in the form of transition probabilities T(z n+1 z n ) p(z n+1 z n ). Remark: T(z n+1 z n ) is sometimes called a transition kernel. 24

25 Markov Chains A marginal probability of a particular state can be computed as: p(z n+1 ) = z n T(z n+1 z n )p(z n ) A distribution π(z) is said to be invariant or stationary with respect to a Markov chain if each step in the chain leaves π(z) invariant: π(z) = z T(z z )π(z ) A given Markov chain may have many stationary distributions. For example: T(z z ) = I{z = z } is the identity transformation. Then any distribution is invariant. 25

26 Detailed Balance A sufficient (but not necessary) condition for ensuring that π(z) is invariant is to choose a transition kernel that satisfies a detailed balance property: π(z )T(z z ) = π(z)t(z z) A transition kernel that satisfies detailed balance will leave that distribution invariant: z π(z )T(z z ) = π(z)t(z z) z = π(z) z T(z z) = π(z) A Markov chain that satisfies detailed balance is said to be reversible. 26

27 Recap We want to sample from target distribution π(z) = π(z)/z (e.g. posterior distribution). Obtaining independent samples is difficult. Set up a Markov chain with transition kernel T(z z) that leaves our target distribution π(z) invariant. If the chain is ergodic, i.e. it is possible to go from every state to any other state (not necessarily in one move), then the chain will converge to this unique invariant distribution π(z). We obtain dependent samples drawn approximately from π(z) by simulating a Markov chain for some time. Ergodicity: There exists K, for any starting z, T K (z z) > 0 for all π(z ) > 0. 27

28 Metropolis-Hasting Algorithm A Markov chain transition operator from current state z to a new state z is defined as follows: A new candidate state z is proposed according to some proposal distribution q(z z), e.g. N(z,σ 2 ). A candidate state x is accepted with probability: ( min 1, π(z ) q(z z ) ) π(z) q(z z) If accepted, set z = z. Otherwise z = z, or the next state is the copy of the current state. Note: no need to know normalizing constant Z. 28

29 Metropolis-Hasting Algorithm We can show that M-H transition kernel leaves π(z) invariant by showing that it satisfies detailed balance: ( π(z)t(z z) = π(z)q(z z)min 1, π(z ) π(z) q(z z ) ) q(z z) = min(π(z)q(z z),π(z )q(z z )) ( π(z) = π(z )q(z ) z) ) min π(z ) q(z z ), 1 = π(z )T(z z ) Note that whether the chain is ergodic will depend on the particulars of π and proposal distribution q. 29

30 Metropolis-Hasting Algorithm Using Metropolis algorithm to sample from Gaussian distribution with proposal q(z z) = N(z,0.04). accepted (green), rejected (red)

31 Choice of Proposal σ max Proposal distribution: q(z z) = N(z,ρ 2 ). ρ ρ large - many rejections ρ small - chain moves too slowly σ min The specific choice of proposal can greatly affect the performance of the algorithm. 31

32 Gibbs Sampler Consider sampling from p(z 1,...,z N ). z 2 L Initialize z i, i = 1,...,N l For t=1,...,t Sample z1 t+1 p(z 1 z2,...,z t N t ) Sample z2 t+1 p(z 2 z1 t+1,x t 3,...,zN t ) Sample z t+1 N p(z N z t+1 1,...,z t+1 N 1 ) z 1 Gibbs sampler is a particular instance of M-H algorithm with proposals p(z n z i n ) accept with probability 1. Apply a series (componentwise) of these operators. 32

33 Gibbs Sampler Applicability of the Gibbs sampler depends on how easy it is to sample from conditional probabilities p(z n z i n ). For discrete random variables with a few discrete settings: p(z n z i n ) = p(z n,z i n ) z n p(z n,z i n ) The sum can be computed analytically. For continuous random variables: p(z n z i n ) = p(z n,z i n ) p(zn,z i n )dz n The integral is univariate and is often analytically tractable or amenable to standard sampling methods. 33

34 Bayesian PMF Remember predictive distribution?: Consider predicting a rating rij for user i and query movie j: p(r ij R) = p(rij u i,v j )p(u,v,θ }{{ U,Θ V R) } d{u,v }d{θ U,Θ V } Posterior over parameters and hyperparameters Use Monte Carlo approximation: p(r ij R) 1 N N n=1 p(rij u (n) i,v (n) j ). The samples (u n i,vn j ) are generated by running a Gibbs sampler, whose stationary distribution is the posterior distribution of interest. 34

35 Bayesian PMF Monte Carlo approximation: p(r ij R) 1 N N n=1 p(rij u (n) i,v (n) j ). The conditional distributions over the user and movie feature vectors are Gaussians easy to sample from: p(u i R,V,Θ U,α) = N ( u i µ i,σ ) i p(v j R,U,Θ U,α) = N ( v j µ j,σ ) j The conditional distributions over hyperparameters also have closed form distributions easy to sample from. Netflix dataset Bayesian PMF can handle over 100 million ratings. 35

36 MCMC: Main Problems Main problems of MCMC: Hard to diagnose convergence (burning in). Sampling from isolated modes. More advanced MCMC methods for sampling in distributions with isolated modes: Parallel tempering Simulated tempering Tempered transitions Hamiltonian Monte Carlo methods (make use of gradient information). Nested Sampling, Coupling from the Past, many others. 36

37 Deterministic Methods Laplace Approximation Bayesian Information Criterion (BIC) Variational Methods: Mean-Field, Loopy Belief Propagation along with various adaptations. Expectation Propagation. 37

### Statistical Machine Learning Lecture 8: Markov Chain Monte Carlo Sampling

1 / 27 Statistical Machine Learning Lecture 8: Markov Chain Monte Carlo Sampling Melih Kandemir Özyeğin University, İstanbul, Turkey 2 / 27 Monte Carlo Integration The big question : Evaluate E p(z) [f(z)]

### Density Estimation. Seungjin Choi

Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/

### Bayesian Learning. HT2015: SC4 Statistical Data Mining and Machine Learning. Maximum Likelihood Principle. The Bayesian Learning Framework

HT5: SC4 Statistical Data Mining and Machine Learning Dino Sejdinovic Department of Statistics Oxford http://www.stats.ox.ac.uk/~sejdinov/sdmml.html Maximum Likelihood Principle A generative model for

### Sampling Rejection Sampling Importance Sampling Markov Chain Monte Carlo. Sampling Methods. Oliver Schulte - CMPT 419/726. Bishop PRML Ch.

Sampling Methods Oliver Schulte - CMP 419/726 Bishop PRML Ch. 11 Recall Inference or General Graphs Junction tree algorithm is an exact inference method for arbitrary graphs A particular tree structure

### STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 7 Approximate

### Andriy Mnih and Ruslan Salakhutdinov

MATRIX FACTORIZATION METHODS FOR COLLABORATIVE FILTERING Andriy Mnih and Ruslan Salakhutdinov University of Toronto, Machine Learning Group 1 What is collaborative filtering? The goal of collaborative

### MCMC algorithms for fitting Bayesian models

MCMC algorithms for fitting Bayesian models p. 1/1 MCMC algorithms for fitting Bayesian models Sudipto Banerjee sudiptob@biostat.umn.edu University of Minnesota MCMC algorithms for fitting Bayesian models

### The Laplace Approximation

The Laplace Approximation Sargur N. University at Buffalo, State University of New York USA Topics in Linear Models for Classification Overview 1. Discriminant Functions 2. Probabilistic Generative Models

### CPSC 540: Machine Learning

CPSC 540: Machine Learning MCMC and Non-Parametric Bayes Mark Schmidt University of British Columbia Winter 2016 Admin I went through project proposals: Some of you got a message on Piazza. No news is

### Sampling Rejection Sampling Importance Sampling Markov Chain Monte Carlo. Sampling Methods. Machine Learning. Torsten Möller.

Sampling Methods Machine Learning orsten Möller Möller/Mori 1 Recall Inference or General Graphs Junction tree algorithm is an exact inference method for arbitrary graphs A particular tree structure defined

### Unsupervised Learning

Unsupervised Learning Bayesian Model Comparison Zoubin Ghahramani zoubin@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit, and MSc in Intelligent Systems, Dept Computer Science University College

### Bayesian Machine Learning

Bayesian Machine Learning Andrew Gordon Wilson ORIE 6741 Lecture 4 Occam s Razor, Model Construction, and Directed Graphical Models https://people.orie.cornell.edu/andrew/orie6741 Cornell University September

### Sampling Algorithms for Probabilistic Graphical models

Sampling Algorithms for Probabilistic Graphical models Vibhav Gogate University of Washington References: Chapter 12 of Probabilistic Graphical models: Principles and Techniques by Daphne Koller and Nir

### Markov Chain Monte Carlo (MCMC)

School of Computer Science 10-708 Probabilistic Graphical Models Markov Chain Monte Carlo (MCMC) Readings: MacKay Ch. 29 Jordan Ch. 21 Matt Gormley Lecture 16 March 14, 2016 1 Homework 2 Housekeeping Due

### Scaling Neighbourhood Methods

Quick Recap Scaling Neighbourhood Methods Collaborative Filtering m = #items n = #users Complexity : m * m * n Comparative Scale of Signals ~50 M users ~25 M items Explicit Ratings ~ O(1M) (1 per billion)

### Bayesian networks: approximate inference

Bayesian networks: approximate inference Machine Intelligence Thomas D. Nielsen September 2008 Approximative inference September 2008 1 / 25 Motivation Because of the (worst-case) intractability of exact

### Probabilistic Graphical Models

Probabilistic Graphical Models Brown University CSCI 295-P, Spring 213 Prof. Erik Sudderth Lecture 11: Inference & Learning Overview, Gaussian Graphical Models Some figures courtesy Michael Jordan s draft

### Bayesian Learning in Undirected Graphical Models

Bayesian Learning in Undirected Graphical Models Zoubin Ghahramani Gatsby Computational Neuroscience Unit University College London, UK http://www.gatsby.ucl.ac.uk/ and Center for Automated Learning and

### The Origin of Deep Learning. Lili Mou Jan, 2015

The Origin of Deep Learning Lili Mou Jan, 2015 Acknowledgment Most of the materials come from G. E. Hinton s online course. Outline Introduction Preliminary Boltzmann Machines and RBMs Deep Belief Nets

### Introduction to Stochastic Gradient Markov Chain Monte Carlo Methods

Introduction to Stochastic Gradient Markov Chain Monte Carlo Methods Changyou Chen Department of Electrical and Computer Engineering, Duke University cc448@duke.edu Duke-Tsinghua Machine Learning Summer

### Introduction: MLE, MAP, Bayesian reasoning (28/8/13)

STA561: Probabilistic machine learning Introduction: MLE, MAP, Bayesian reasoning (28/8/13) Lecturer: Barbara Engelhardt Scribes: K. Ulrich, J. Subramanian, N. Raval, J. O Hollaren 1 Classifiers In this

### Bayesian Hidden Markov Models and Extensions

Bayesian Hidden Markov Models and Extensions Zoubin Ghahramani Department of Engineering University of Cambridge joint work with Matt Beal, Jurgen van Gael, Yunus Saatci, Tom Stepleton, Yee Whye Teh Modeling

### Particle-Based Approximate Inference on Graphical Model

article-based Approimate Inference on Graphical Model Reference: robabilistic Graphical Model Ch. 2 Koller & Friedman CMU, 0-708, Fall 2009 robabilistic Graphical Models Lectures 8,9 Eric ing attern Recognition

### Outline Lecture 2 2(32)

Outline Lecture (3), Lecture Linear Regression and Classification it is our firm belief that an understanding of linear models is essential for understanding nonlinear ones Thomas Schön Division of Automatic

### σ(a) = a N (x; 0, 1 2 ) dx. σ(a) = Φ(a) =

Until now we have always worked with likelihoods and prior distributions that were conjugate to each other, allowing the computation of the posterior distribution to be done in closed form. Unfortunately,

### GAUSSIAN PROCESS REGRESSION

GAUSSIAN PROCESS REGRESSION CSE 515T Spring 2015 1. BACKGROUND The kernel trick again... The Kernel Trick Consider again the linear regression model: y(x) = φ(x) w + ε, with prior p(w) = N (w; 0, Σ). The

### Bayesian Inference: Principles and Practice 3. Sparse Bayesian Models and the Relevance Vector Machine

Bayesian Inference: Principles and Practice 3. Sparse Bayesian Models and the Relevance Vector Machine Mike Tipping Gaussian prior Marginal prior: single α Independent α Cambridge, UK Lecture 3: Overview

### STA 294: Stochastic Processes & Bayesian Nonparametrics

MARKOV CHAINS AND CONVERGENCE CONCEPTS Markov chains are among the simplest stochastic processes, just one step beyond iid sequences of random variables. Traditionally they ve been used in modelling a

### p L yi z n m x N n xi

y i z n x n N x i Overview Directed and undirected graphs Conditional independence Exact inference Latent variables and EM Variational inference Books statistical perspective Graphical Models, S. Lauritzen

### Based on slides by Richard Zemel

CSC 412/2506 Winter 2018 Probabilistic Learning and Reasoning Lecture 3: Directed Graphical Models and Latent Variables Based on slides by Richard Zemel Learning outcomes What aspects of a model can we

### Approximate Inference Part 1 of 2

Approximate Inference Part 1 of 2 Tom Minka Microsoft Research, Cambridge, UK Machine Learning Summer School 2009 http://mlg.eng.cam.ac.uk/mlss09/ Bayesian paradigm Consistent use of probability theory

### Notes on pseudo-marginal methods, variational Bayes and ABC

Notes on pseudo-marginal methods, variational Bayes and ABC Christian Andersson Naesseth October 3, 2016 The Pseudo-Marginal Framework Assume we are interested in sampling from the posterior distribution

### Approximate Inference Part 1 of 2

Approximate Inference Part 1 of 2 Tom Minka Microsoft Research, Cambridge, UK Machine Learning Summer School 2009 http://mlg.eng.cam.ac.uk/mlss09/ 1 Bayesian paradigm Consistent use of probability theory

### An introduction to Sequential Monte Carlo

An introduction to Sequential Monte Carlo Thang Bui Jes Frellsen Department of Engineering University of Cambridge Research and Communication Club 6 February 2014 1 Sequential Monte Carlo (SMC) methods

### Sequential Monte Carlo and Particle Filtering. Frank Wood Gatsby, November 2007

Sequential Monte Carlo and Particle Filtering Frank Wood Gatsby, November 2007 Importance Sampling Recall: Let s say that we want to compute some expectation (integral) E p [f] = p(x)f(x)dx and we remember

### Pattern Recognition and Machine Learning. Bishop Chapter 6: Kernel Methods

Pattern Recognition and Machine Learning Chapter 6: Kernel Methods Vasil Khalidov Alex Kläser December 13, 2007 Training Data: Keep or Discard? Parametric methods (linear/nonlinear) so far: learn parameter

### Bayesian GLMs and Metropolis-Hastings Algorithm

Bayesian GLMs and Metropolis-Hastings Algorithm We have seen that with conjugate or semi-conjugate prior distributions the Gibbs sampler can be used to sample from the posterior distribution. In situations,

### CS 188: Artificial Intelligence. Bayes Nets

CS 188: Artificial Intelligence Probabilistic Inference: Enumeration, Variable Elimination, Sampling Pieter Abbeel UC Berkeley Many slides over this course adapted from Dan Klein, Stuart Russell, Andrew

### Parametric Techniques

Parametric Techniques Jason J. Corso SUNY at Buffalo J. Corso (SUNY at Buffalo) Parametric Techniques 1 / 39 Introduction When covering Bayesian Decision Theory, we assumed the full probabilistic structure

### Lecture 8: Bayesian Networks

Lecture 8: Bayesian Networks Bayesian Networks Inference in Bayesian Networks COMP-652 and ECSE 608, Lecture 8 - January 31, 2017 1 Bayes nets P(E) E=1 E=0 0.005 0.995 E B P(B) B=1 B=0 0.01 0.99 E=0 E=1

### Stochastic Backpropagation, Variational Inference, and Semi-Supervised Learning

Stochastic Backpropagation, Variational Inference, and Semi-Supervised Learning Diederik (Durk) Kingma Danilo J. Rezende (*) Max Welling Shakir Mohamed (**) Stochastic Gradient Variational Inference Bayesian

### Probabilistic Modelling and Bayesian Inference

Probabilistic Modelling and Bayesian Inference Zoubin Ghahramani University of Cambridge, UK Uber AI Labs, San Francisco, USA zoubin@eng.cam.ac.uk http://learning.eng.cam.ac.uk/zoubin/ MLSS Tübingen Lectures

### Monte Carlo Methods for Inference and Learning

Monte Carlo Methods for Inference and Learning Ryan Adams University of Toronto CIFAR NCAP Summer School 14 August 2010 http://www.cs.toronto.edu/~rpa Thanks to: Iain Murray, Marc Aurelio Ranzato Overview

### Metropolis-Hastings Algorithm

Strength of the Gibbs sampler Metropolis-Hastings Algorithm Easy algorithm to think about. Exploits the factorization properties of the joint probability distribution. No difficult choices to be made to

### Tutorial on Probabilistic Programming with PyMC3

185.A83 Machine Learning for Health Informatics 2017S, VU, 2.0 h, 3.0 ECTS Tutorial 02-04.04.2017 Tutorial on Probabilistic Programming with PyMC3 florian.endel@tuwien.ac.at http://hci-kdd.org/machine-learning-for-health-informatics-course

### PART I INTRODUCTION The meaning of probability Basic definitions for frequentist statistics and Bayesian inference Bayesian inference Combinatorics

Table of Preface page xi PART I INTRODUCTION 1 1 The meaning of probability 3 1.1 Classical definition of probability 3 1.2 Statistical definition of probability 9 1.3 Bayesian understanding of probability

### Model Selection for Gaussian Processes

Institute for Adaptive and Neural Computation School of Informatics,, UK December 26 Outline GP basics Model selection: covariance functions and parameterizations Criteria for model selection Marginal

### Technical Report. Bayesian inference for latent variable models. Ulrich Paquet. Number 724. July Computer Laboratory

Technical Report UCAM-CL-TR-724 ISSN 1476-2986 Number 724 Computer Laboratory Bayesian inference for latent variable models Ulrich Paquet July 2008 15 JJ Thomson Avenue Cambridge CB3 0FD United Kingdom

### Variational Principal Components

Variational Principal Components Christopher M. Bishop Microsoft Research 7 J. J. Thomson Avenue, Cambridge, CB3 0FB, U.K. cmbishop@microsoft.com http://research.microsoft.com/ cmbishop In Proceedings

### Approximate inference in Energy-Based Models

CSC 2535: 2013 Lecture 3b Approximate inference in Energy-Based Models Geoffrey Hinton Two types of density model Stochastic generative model using directed acyclic graph (e.g. Bayes Net) Energy-based

### Probabilistic Modelling and Bayesian Inference

Probabilistic Modelling and Bayesian Inference Zoubin Ghahramani Department of Engineering University of Cambridge, UK zoubin@eng.cam.ac.uk http://learning.eng.cam.ac.uk/zoubin/ MLSS Tübingen Lectures

### DAG models and Markov Chain Monte Carlo methods a short overview

DAG models and Markov Chain Monte Carlo methods a short overview Søren Højsgaard Institute of Genetics and Biotechnology University of Aarhus August 18, 2008 Printed: August 18, 2008 File: DAGMC-Lecture.tex

### Probabilistic Graphical Models

2016 Robert Nowak Probabilistic Graphical Models 1 Introduction We have focused mainly on linear models for signals, in particular the subspace model x = Uθ, where U is a n k matrix and θ R k is a vector

### PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 13: SEQUENTIAL DATA

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 13: SEQUENTIAL DATA Contents in latter part Linear Dynamical Systems What is different from HMM? Kalman filter Its strength and limitation Particle Filter

### SAMPLING ALGORITHMS. In general. Inference in Bayesian models

SAMPLING ALGORITHMS SAMPLING ALGORITHMS In general A sampling algorithm is an algorithm that outputs samples x 1, x 2,... from a given distribution P or density p. Sampling algorithms can for example be

### Gaussian processes. Chuong B. Do (updated by Honglak Lee) November 22, 2008

Gaussian processes Chuong B Do (updated by Honglak Lee) November 22, 2008 Many of the classical machine learning algorithms that we talked about during the first half of this course fit the following pattern:

### Bayesian Computation: Posterior Sampling & MCMC

Bayesian Computation: Posterior Sampling & MCMC Tom Loredo Dept. of Astronomy, Cornell University http://www.astro.cornell.edu/staff/loredo/bayes/ Cosmic Populations @ CASt 9 10 June 2014 1/42 Posterior

### Bayesian Mixtures of Bernoulli Distributions

Bayesian Mixtures of Bernoulli Distributions Laurens van der Maaten Department of Computer Science and Engineering University of California, San Diego Introduction The mixture of Bernoulli distributions

### Lecture 2: From Linear Regression to Kalman Filter and Beyond

Lecture 2: From Linear Regression to Kalman Filter and Beyond Department of Biomedical Engineering and Computational Science Aalto University January 26, 2012 Contents 1 Batch and Recursive Estimation

### Bayesian Linear Models

Bayesian Linear Models Sudipto Banerjee 1 and Andrew O. Finley 2 1 Department of Forestry & Department of Geography, Michigan State University, Lansing Michigan, U.S.A. 2 Biostatistics, School of Public

### Linear Dynamical Systems

Linear Dynamical Systems Sargur N. srihari@cedar.buffalo.edu Machine Learning Course: http://www.cedar.buffalo.edu/~srihari/cse574/index.html Two Models Described by Same Graph Latent variables Observations

### Bayesian Inference of Noise Levels in Regression

Bayesian Inference of Noise Levels in Regression Christopher M. Bishop Microsoft Research, 7 J. J. Thomson Avenue, Cambridge, CB FB, U.K. cmbishop@microsoft.com http://research.microsoft.com/ cmbishop

### Lecture 9: PGM Learning

13 Oct 2014 Intro. to Stats. Machine Learning COMP SCI 4401/7401 Table of Contents I Learning parameters in MRFs 1 Learning parameters in MRFs Inference and Learning Given parameters (of potentials) and

### CS Lecture 18. Topic Models and LDA

CS 6347 Lecture 18 Topic Models and LDA (some slides by David Blei) Generative vs. Discriminative Models Recall that, in Bayesian networks, there could be many different, but equivalent models of the same

### Lecture 2: From Linear Regression to Kalman Filter and Beyond

Lecture 2: From Linear Regression to Kalman Filter and Beyond January 18, 2017 Contents 1 Batch and Recursive Estimation 2 Towards Bayesian Filtering 3 Kalman Filter and Bayesian Filtering and Smoothing

### Estimating the marginal likelihood with Integrated nested Laplace approximation (INLA)

Estimating the marginal likelihood with Integrated nested Laplace approximation (INLA) arxiv:1611.01450v1 [stat.co] 4 Nov 2016 Aliaksandr Hubin Department of Mathematics, University of Oslo and Geir Storvik

### Bagging During Markov Chain Monte Carlo for Smoother Predictions

Bagging During Markov Chain Monte Carlo for Smoother Predictions Herbert K. H. Lee University of California, Santa Cruz Abstract: Making good predictions from noisy data is a challenging problem. Methods

### Machine Learning Techniques for Computer Vision

Machine Learning Techniques for Computer Vision Part 2: Unsupervised Learning Microsoft Research Cambridge x 3 1 0.5 0.2 0 0.5 0.3 0 0.5 1 ECCV 2004, Prague x 2 x 1 Overview of Part 2 Mixture models EM

### Pattern Recognition and Machine Learning. Bishop Chapter 9: Mixture Models and EM

Pattern Recognition and Machine Learning Chapter 9: Mixture Models and EM Thomas Mensink Jakob Verbeek October 11, 27 Le Menu 9.1 K-means clustering Getting the idea with a simple example 9.2 Mixtures

### Elliptical slice sampling

Iain Murray Ryan Prescott Adams David J.C. MacKay University of Toronto University of Toronto University of Cambridge Abstract Many probabilistic models introduce strong dependencies between variables

### Probabilistic machine learning group, Aalto University Bayesian theory and methods, approximative integration, model

Aki Vehtari, Aalto University, Finland Probabilistic machine learning group, Aalto University http://research.cs.aalto.fi/pml/ Bayesian theory and methods, approximative integration, model assessment and

### Bayesian Linear Models

Bayesian Linear Models Sudipto Banerjee 1 and Andrew O. Finley 2 1 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. 2 Department of Forestry & Department

### Bayesian Inference in Astronomy & Astrophysics A Short Course

Bayesian Inference in Astronomy & Astrophysics A Short Course Tom Loredo Dept. of Astronomy, Cornell University p.1/37 Five Lectures Overview of Bayesian Inference From Gaussians to Periodograms Learning

Adaptive Monte Carlo methods Jean-Michel Marin Projet Select, INRIA Futurs, Université Paris-Sud joint with Randal Douc (École Polytechnique), Arnaud Guillin (Université de Marseille) and Christian Robert

### Data Analysis and Uncertainty Part 2: Estimation

Data Analysis and Uncertainty Part 2: Estimation Instructor: Sargur N. University at Buffalo The State University of New York srihari@cedar.buffalo.edu 1 Topics in Estimation 1. Estimation 2. Desirable

### AUTOMATIC CONTROL REGLERTEKNIK LINKÖPINGS UNIVERSITET. Machine Learning T. Schön. (Chapter 11) AUTOMATIC CONTROL REGLERTEKNIK LINKÖPINGS UNIVERSITET

About the Eam I/II) ), Lecture 7 MCMC and Sampling Methods Thomas Schön Division of Automatic Control Linköping University Linköping, Sweden. Email: schon@isy.liu.se, Phone: 3-373, www.control.isy.liu.se/~schon/

### The Variational Bayesian EM Algorithm for Incomplete Data: with Application to Scoring Graphical Model Structures

BAYESIAN STATISTICS 7, pp. 000 000 J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith and M. West (Eds.) c Oxford University Press, 2003 The Variational Bayesian EM

### Statistical learning. Chapter 20, Sections 1 4 1

Statistical learning Chapter 20, Sections 1 4 Chapter 20, Sections 1 4 1 Outline Bayesian learning Maximum a posteriori and maximum likelihood learning Bayes net learning ML parameter learning with complete

### Gaussian Process Dynamical Models Jack M Wang, David J Fleet, Aaron Hertzmann, NIPS 2005

Gaussian Process Dynamical Models Jack M Wang, David J Fleet, Aaron Hertzmann, NIPS 2005 Presented by Piotr Mirowski CBLL meeting, May 6, 2009 Courant Institute of Mathematical Sciences, New York University

### Regression with Input-Dependent Noise: A Bayesian Treatment

Regression with Input-Dependent oise: A Bayesian Treatment Christopher M. Bishop C.M.BishopGaston.ac.uk Cazhaow S. Qazaz qazazcsgaston.ac.uk eural Computing Research Group Aston University, Birmingham,

### Infinite Latent Feature Models and the Indian Buffet Process

Infinite Latent Feature Models and the Indian Buffet Process Thomas L. Griffiths Cognitive and Linguistic Sciences Brown University, Providence RI 292 tom griffiths@brown.edu Zoubin Ghahramani Gatsby Computational

### Introduction to Bayesian methods in inverse problems

Introduction to Bayesian methods in inverse problems Ville Kolehmainen 1 1 Department of Applied Physics, University of Eastern Finland, Kuopio, Finland March 4 2013 Manchester, UK. Contents Introduction

### Appendices: Stochastic Backpropagation and Approximate Inference in Deep Generative Models

Appendices: Stochastic Backpropagation and Approximate Inference in Deep Generative Models Danilo Jimenez Rezende Shakir Mohamed Daan Wierstra Google DeepMind, London, United Kingdom DANILOR@GOOGLE.COM

### Monte Carlo Methods. Geoff Gordon February 9, 2006

Monte Carlo Methods Geoff Gordon ggordon@cs.cmu.edu February 9, 2006 Numerical integration problem 5 4 3 f(x,y) 2 1 1 0 0.5 0 X 0.5 1 1 0.8 0.6 0.4 Y 0.2 0 0.2 0.4 0.6 0.8 1 x X f(x)dx Used for: function

### Bayesian Networks: Construction, Inference, Learning and Causal Interpretation. Volker Tresp Summer 2014

Bayesian Networks: Construction, Inference, Learning and Causal Interpretation Volker Tresp Summer 2014 1 Introduction So far we were mostly concerned with supervised learning: we predicted one or several

### Bayesian Learning in Undirected Graphical Models: Approximate MCMC algorithms

Bayesian Learning in Undirected Graphical Models: Approximate MCMC algorithms Iain Murray and Zoubin Ghahramani Gatsby Computational Neuroscience Unit University College London, London WC1N 3AR, UK http://www.gatsby.ucl.ac.uk/

### Variational Bayesian Logistic Regression

Variational Bayesian Logistic Regression Sargur N. University at Buffalo, State University of New York USA Topics in Linear Models for Classification Overview 1. Discriminant Functions 2. Probabilistic

### MCMC Methods for MLP-network and Gaussian Process and Stuff A documentation for Matlab Toolbox MCMCstuff

MCMC Methods for MLP-network and Gaussian Process and Stuff A documentation for Matlab Toolbox MCMCstuff Jarno Vanhatalo and Aki Vehtari Laboratory of Computational Engineering, Helsinki University of

### Lecture Notes based on Koop (2003) Bayesian Econometrics

Lecture Notes based on Koop (2003) Bayesian Econometrics A.Colin Cameron University of California - Davis November 15, 2005 1. CH.1: Introduction The concepts below are the essential concepts used throughout

### Stat 535 C - Statistical Computing & Monte Carlo Methods. Arnaud Doucet.

Stat 535 C - Statistical Computing & Monte Carlo Methods Arnaud Doucet Email: arnaud@cs.ubc.ca 1 1.1 Outline Introduction to Markov chain Monte Carlo The Gibbs Sampler Examples Overview of the Lecture

### Accelerated Gibbs Sampling for Infinite Sparse Factor Analysis

LLNL-TR-499647 Accelerated Gibbs Sampling for Infinite Sparse Factor Analysis D. M. Andrzejewski September 19, 2011 Disclaimer This document was prepared as an account of work sponsored by an agency of

### Clustering problems, mixture models and Bayesian nonparametrics

Clustering problems, mixture models and Bayesian nonparametrics Nguyễn Xuân Long Department of Statistics Department of Electrical Engineering and Computer Science University of Michigan Vietnam Institute

### Bayesian time series classification

Bayesian time series classification Peter Sykacek Department of Engineering Science University of Oxford Oxford, OX 3PJ, UK psyk@robots.ox.ac.uk Stephen Roberts Department of Engineering Science University

### Machine Learning. Lecture 4: Regularization and Bayesian Statistics. Feng Li. https://funglee.github.io

Machine Learning Lecture 4: Regularization and Bayesian Statistics Feng Li fli@sdu.edu.cn https://funglee.github.io School of Computer Science and Technology Shandong University Fall 207 Overfitting Problem

### IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 18, NO. 3, MARCH

IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 18, NO. 3, MARCH 2010 589 Gamma Markov Random Fields for Audio Source Modeling Onur Dikmen, Member, IEEE, and A. Taylan Cemgil, Member,

### Latent Dirichlet Allocation

Latent Dirichlet Allocation 1 Directed Graphical Models William W. Cohen Machine Learning 10-601 2 DGMs: The Burglar Alarm example Node ~ random variable Burglar Earthquake Arcs define form of probability

### 2 Probability and machine learning principles

2 Probability and machine learning principles P ROBABILITY will be a recurring theme throughout the thesis. Indeed, its influence is surprisingly pervasive in the material making up the remaining chapters.