Brief introduction to Markov Chain Monte Carlo
|
|
- Gabriel Peregrine Brooks
- 5 years ago
- Views:
Transcription
1 Brief introduction to Department of Probability and Mathematical Statistics seminar Stochastic modeling in economics and finance November 7, 2011 Brief introduction to
2 Content 1 and motivation Classical iid sampling 2 Basic definitions Stationary distribution Limit theorems for Markov Chains 3 4 Brief introduction to
3 What is MCMC? and motivation Classical iid sampling MCMC = Markov Chain + Monte Carlo Monte Carlo is just a cool name for random simulation. Typically we generate iid samples from the given distribution. = algorithm generating Markov Chain with prescribed stationary distribution as a tool to sample from this distribution. Not independent. Used when iid sampling is not feasible. MCMC first appears in Metropolis et al. (1953): Equation of state calculations by fast computing machines. Journal of Chemical Physics. MCMC originally used in statistical physics. Nowadays a general simulation tool (popular e.g. in Bayesian statistics). Brief introduction to
4 Why we need random sampling? and motivation Classical iid sampling Because we need to do following things: Create a random sample to see how does the distribution look like. Estimate the expectation or other parameters (e.g. VaR in operational risk) of the distribution which can t be computed analytically. Estimate the probability of a certain event. Special case: simulating p-values of statistical tests. Create simulated data to test statistical methods on them. Use Monte Carlo integration as an alternative to classical numerical integration. Solve optimization problem using a randomized algorithm. Brief introduction to
5 Classical iid sampling and motivation Classical iid sampling We want to estimate the expectation µ = E [g(x )]. Create an iid random sample X 1, X 2,..., X n from L(X ) and put ˆµ n = 1 n g(x i ). n i=1 Estimate is unbiased: E (ˆµ n ) = µ. Variance reciprocal to n: var (ˆµ n ) = σ 2 /n, where σ 2 = var [g(x )]. Strong Law of Large Numbers: ˆµ n Central Limit Theorem: n (ˆµ n µ) a.s. µ as n. D N(0, σ 2 ) as n. Brief introduction to
6 Classical sampling methods and motivation Classical iid sampling Inverse sampling: When the cumulative distribution function can be inverted easily. For example: exponential or logistic distribution. Using U(0, 1) as an input. Only for 1-dimensional cases. Rejection sampling: We need an auxiliary enveloping distribution from which it is easy to simulate. Then we reject some realizations randomly (using U(0, 1) as an additional input) according to the probability density ratio of desired and auxiliary distribution. Importance sampling: Estimating integral from f as an expectation of f /g with respect to g (from which we can simulate). Choose g as much close to f as possible. Brief introduction to
7 and motivation Classical iid sampling Problems with classical sampling methods Classical sampling methods are not always applicable, especially for large dimensional complex problems. Example: Sampling uniformly from n-dimensional unit sphere via rejection sampling. As the enveloping distribution we use uniform distribution on the n-dimensional cube [ 1, 1] n. What is the acceptance rate of such a generator? π n/2 Γ(n/2 + 1)2 n n 0!!! Brief introduction to
8 Markov Chains - Basic definitions Basic definitions Stationary distribution Limit theorems for Markov Chains Markov Chain = stochastic process with Markovian property: P(X t+1 = x t+1 t i=0 {X i = x i }) = P(X t+1 = x t+1 X t = x t ). For our purposes we need discrete time and: discrete state space X : transition probability matrix P ij continuous state space X : transition kernel P(x, x ) Homogeneity: transition probabilities P are independent of time. We will work with homogenous chains only. Irreducibility: every state is accessible in one or more steps from any other state with positive probability. Brief introduction to
9 Basic definitions Stationary distribution Limit theorems for Markov Chains Stationary distribution, Detailed balance equations Collection {π i, i X } is called a stationary distribution of Markov Chain X t with probability transition matrix P if i X π i = 1 and π = πp. Detailed balance equations: Let X t be a Markov Chain with probability transition matrix P. If {π i, i X } satisfies i X π i = 1 and π i P ij = π j P ji for all i, j X then π is a stationary distribution of X. Brief introduction to
10 Basic definitions Stationary distribution Limit theorems for Markov Chains Law of Large numbers and CLT for Markov Chains Let X t be a stationary Markov Chain with state space X and stationary distribution π. Suppose that µ = X g(x)dπ(x) exists. Let s estimate µ by the sample average of g(x t ) over time: ˆµ n = 1 n n g(x t ). t=1 Strong Law of Large Numbers: ˆµ n a.s. µ as n. Central Limit Theorem: n (ˆµ n µ) D N(0, σ 2 ) as n, where + σ 2 = var [g(x i )] + 2 cov [g(x i ), g(x i+k )]. k=1 Brief introduction to
11 MCMC - Basic principle We want to simulate from a given (target) distribution p on X. This can be discrete or continuous. Start from an arbitrary element x X and take it as X 0. Further construct Markov Chain {X t }, t = 1, 2,... which will explore the distribution p in successive time iterations. Design the transition probabilities of {X t } so that p is stationary distribution of {X t }. We loose independence which decrease efficiency of estimates when compared to iid. Brief introduction to
12 algorithm I Let X be finite or countable infinite set. Let p i > 0, i X be a probability distribution on X from which we want to simulate. We will construct an irreducible probability transition matrix P so that p will be a stationary distribution of Markov Chain {X t } with state space X and transition matrix P. Consider any irreducible probability transition matrix Q on state space X. Metropolis et a. (1953) considered symmetric Q so all the terms Q ij /Q ji (so called Hastings ratios) would vanish from the following formulas. Brief introduction to
13 algorithm II Start {X t } in an arbitrary state X 0 = x 0 X. Then calculate iteratively over time t: 1 Having X t, choose a proposal X X according to Q: P(X = x X t = x t ) = Q xtx. { } 2 Calculate α = min 1, p(x ) Q X X t p(x t) Q (0, 1], so called X t X acceptance probability. 3 With probability α accept X and put X t+1 = X. Otherwise (with probability 1 α) reject X and keep X t+1 = X t. 4 Set t = t + 1 and go back to 1. Brief introduction to
14 algorithm III Resulted probability transition matrix P of {X t }: { P ij = Q ij min 1, p(j) } Q ji i j, p(i) Q ij P ii = 1 P ij i X. j X {i} By simple algebraic manipulations we can show that matrix {P ij } satisfies the detailed balance equations in combination with p. So p is a stationary distribution of the constructed Markov Chain {X t }, what we needed to prove. We do not need to evaluate p, we just need to compute ratios p(j)/p(i). So the normalizing constant of p can be ignored. Brief introduction to
15 Examples of applications Generate uniformly from a set of m x n integer contingency tables with given rows and columns marginals. Irreducible Q can be defined as an uniform choice from neighboring tables. Since p is uniform, p(j)/p(i) 1 here. Generating from a Poisson distribution. Let Q be a random walk over X = {0, 1, 2,...}. Ratios p(i ± 1)/p(i) are easy to calculate. Exploring the feasible set of integer programming maximization problem. We can put p increasing with increasing objective function and so push {X t } towards regions with higher values of the objective function. Brief introduction to
16 algorithm I Let X R m be a support of the probability density function p(x), p(x) > 0 x X, from which we want to simulate. We will construct a Markov Chain {X t } with state space X so that p will be its stationary distribution. Consider any proposal probability density function q(x x) of x X, depending generally on x X. Brief introduction to
17 algorithm II Start {X t } in an arbitrary state X 0 = x 0 X. Then calculate iteratively over time t: 1 Having X t, draw a proposal X X from q(x x t ). { } 2 Calculate α = min 1, p(x ) q(x X t) p(x t) q(x t X ) (0, 1], so called acceptance probability. 3 With probability α accept X and put X t+1 = X. Otherwise (with probability 1 α) reject X and keep X t+1 = X t. 4 Set t = t + 1 and go back to 1. Brief introduction to
18 algorithm III Similarly as in the discrete case, we can show that the transition kernel P(x x) of {X t } satisfies the detailed balance equations in combination with p. So p is a stationary distribution of the constructed Markov Chain {X t }, what we needed to prove. We do not need to evaluate p(x), we just need to compute ratios p(x )/p(x). So the normalizing constant of p (which is often unknown) can be ignored. When q(x x) does not depend on x X, we talk about independence chain (but it is still not iid!!!). When q(x x) q(x x), we talk about random walk MCMC. When in addition q( ) is symmetric, formulas simplify again. Brief introduction to
19 Metropolis-Hastings 2D random walk illustration Brief introduction to
20 - N(0, 1) Target distribution is N(0, 1). Let s forget for a while that it is easy to simulate from N(0, 1) directly. Starting point X 0 = 0 (modus of the target distribution). We use normal random walk Metropolis-Hastings algorithm with symmetric trial (jump) distribution N(0, σ 2 J ). Simple implementation on Excel sheet. By changing σ J > 0 parameter, we drive the acceptance rate of the proposals, the convergence speed and efficiency. We use σ J {0.2, 0.5, 1, 2, 2.5, 3, 5, 8, 10} and evaluate the results. Brief introduction to
21 MCMC trajectory (400 observations): σ J = 0.1 Very high acceptance rate. Random walk with mean reversion. Brief introduction to
22 MCMC trajectory (400 observations): σ J = 2.5 Moderate acceptance rate. Looks most like a white noise. Brief introduction to
23 MCMC trajectory (400 observations): σ J = 10 Very low acceptance rate. Locally constant with rare high jumps. Brief introduction to
24 Simulated histogram (400 observations): σ J = 0.1 Chain didn t explore the whole target distribution (especially tails). Brief introduction to
25 Simulated histogram (400 observations): σ J = 2.5 Resulted histogram looks nice. Brief introduction to
26 Simulated histogram (400 observations): σ J = 10 The chain is too rigid, the histogram consists of several peaks. Brief introduction to
27 MCMC trajectory ACF: σ J = 0.1 Very low (linear) decay of ACF. Brief introduction to
28 MCMC trajectory ACF: σ J = 2.5 ACF (k) 0.6 k. Brief introduction to
29 MCMC trajectory ACF: σ J = 10 ACF (k) 0.83 k. Brief introduction to
30 Acceptance rate as a function of σ J The acceptance rate goes from 100 % to 15 % as σ J goes from 0.1 to 10. At moderate σ J = 2.5 the acceptance rate is around 40 %. Brief introduction to
31 ACF(1) as a function of σ J ACF(1) plot has an U-shape, starting at 1 and ending at The minimum value 0.6 is attained somewhere near σ J = 2.5. Brief introduction to
32 Efficiency as a function of σ J The efficiency forms a -shape, starting at 0 % value. The maximum value around 30 % is attained again somewhere near σ J = 2.5. Brief introduction to
33 Choosing the trial distribution So called trial (or proposal, candidate, jumping ) distribution is crucial in achieving reasonable efficiency of MCMC estimates. We must be able to simulate from it easily. Its shape is recommended to copy the shape of the target distribution. Its width (standard deviation) must be tuned as to optimize the efficiency % acceptance rate is recommended. For 1-dimensional case, the optimal efficiency is usually around 30 %. Efficiency decreases with higher dimensions. Brief introduction to
34 Convergence issue Since the constructed Markov Chain just approximates the target distribution in a limiting sense, we must care whether our Markov Chain has converged to the target distribution already. Some convergence diagnostics are available for this. The convergence speed can differ, be careful. Drop the beginning phase of MC trajectory before the convergence is reached. So called burn in stage. Run the simulation several times and compare characteristics of individual trajectories. Try run the simulation with different starting points and compare the results. Brief introduction to
35 Error of MCMC estimate Assessing the error of MCMC estimate is not so straightforward as in iid sampling. We can use the formula with process covariance function values to estimate the variance of its sample means. Typically autocorrelation function ACF (k) of g(x t ) decays to 0 like exponential ρ k with ρ > 0, i.e. like ACF of AR(1) process. We try to reach ρ as close to 0 as possible (higher efficiency). We can estimate sample mean error directly from variation of shorter block (batch) sample means. Tune the process to increase efficiency and so decrease the estimation error. Brief introduction to
36 Further MCMC topics There are many variants and modifications of MCMC one can meet when browsing through literature. Just to list them: Gibbs sampling Slice sampling Annealing (tempering) Langevin technique Hamiltonian hybrid algorithm Coupling from the past... Brief introduction to
37 References References Contacts W. K. Hastings : Monte Carlo sampling methods using and their applications. Biometrika 57 (1970), N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller and E. Teller : Equation of state calculations by fast computing machines. J. of Chemical Physics 21 (1953), J. M. Hohendorff : An to. Department of Statistics, University of Toronto, T. Balún : metódy posteriórnej simulácie a ich aplikácia v ekonómii. Diploma thesis, m/diplomka.pdf. C. Geyer : to. University of Minnesota, L. Kroc : to. S. Lalley : to. Department of Statistics, University of Chicago. lalley/courses/313/proppwilson.pdf. I. Murray : Markov chain Monte Carlo. Machine Learning Summer School slides/murray 1.pdf. P. Lam : MCMC Methods: Gibbs Sampling and the Metropolis-Hastings Algorithm. Harvard University. plam/teaching/methods/mcmc/mcmc mprint.pdf. K. M. Hanson : Tutorial on. Los Alamos National Laboratory, M. Scullard : Reversible Markov Chains williams/courses/m28908/scullardmath289 mreversibility.pdf. Brief introduction to
38 Contacts References Contacts mobile: web: Department of Probability and Mathematical Statistics Faculty of Mathematics and Physics Charles University in Prague Sokolovská 83, Praha 8. web: kpms MEDIARESEARCH, a.s. Českobratrská 1, Praha 3. mobile: web: Brief introduction to
Introduction to Machine Learning CMU-10701
Introduction to Machine Learning CMU-10701 Markov Chain Monte Carlo Methods Barnabás Póczos & Aarti Singh Contents Markov Chain Monte Carlo Methods Goal & Motivation Sampling Rejection Importance Markov
More informationComputational statistics
Computational statistics Markov Chain Monte Carlo methods Thierry Denœux March 2017 Thierry Denœux Computational statistics March 2017 1 / 71 Contents of this chapter When a target density f can be evaluated
More informationBayesian Methods for Machine Learning
Bayesian Methods for Machine Learning CS 584: Big Data Analytics Material adapted from Radford Neal s tutorial (http://ftp.cs.utoronto.ca/pub/radford/bayes-tut.pdf), Zoubin Ghahramni (http://hunch.net/~coms-4771/zoubin_ghahramani_bayesian_learning.pdf),
More informationMarkov Chain Monte Carlo
Chapter 5 Markov Chain Monte Carlo MCMC is a kind of improvement of the Monte Carlo method By sampling from a Markov chain whose stationary distribution is the desired sampling distributuion, it is possible
More informationApril 20th, Advanced Topics in Machine Learning California Institute of Technology. Markov Chain Monte Carlo for Machine Learning
for for Advanced Topics in California Institute of Technology April 20th, 2017 1 / 50 Table of Contents for 1 2 3 4 2 / 50 History of methods for Enrico Fermi used to calculate incredibly accurate predictions
More informationMarkov Chain Monte Carlo methods
Markov Chain Monte Carlo methods Tomas McKelvey and Lennart Svensson Signal Processing Group Department of Signals and Systems Chalmers University of Technology, Sweden November 26, 2012 Today s learning
More information6 Markov Chain Monte Carlo (MCMC)
6 Markov Chain Monte Carlo (MCMC) The underlying idea in MCMC is to replace the iid samples of basic MC methods, with dependent samples from an ergodic Markov chain, whose limiting (stationary) distribution
More informationMonte Carlo Methods. Leon Gu CSD, CMU
Monte Carlo Methods Leon Gu CSD, CMU Approximate Inference EM: y-observed variables; x-hidden variables; θ-parameters; E-step: q(x) = p(x y, θ t 1 ) M-step: θ t = arg max E q(x) [log p(y, x θ)] θ Monte
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 7 Approximate
More informationSession 3A: Markov chain Monte Carlo (MCMC)
Session 3A: Markov chain Monte Carlo (MCMC) John Geweke Bayesian Econometrics and its Applications August 15, 2012 ohn Geweke Bayesian Econometrics and its Session Applications 3A: Markov () chain Monte
More informationIntroduction to Machine Learning CMU-10701
Introduction to Machine Learning CMU-10701 Markov Chain Monte Carlo Methods Barnabás Póczos Contents Markov Chain Monte Carlo Methods Sampling Rejection Importance Hastings-Metropolis Gibbs Markov Chains
More informationMarkov chain Monte Carlo methods in atmospheric remote sensing
1 / 45 Markov chain Monte Carlo methods in atmospheric remote sensing Johanna Tamminen johanna.tamminen@fmi.fi ESA Summer School on Earth System Monitoring and Modeling July 3 Aug 11, 212, Frascati July,
More informationA Search and Jump Algorithm for Markov Chain Monte Carlo Sampling. Christopher Jennison. Adriana Ibrahim. Seminar at University of Kuwait
A Search and Jump Algorithm for Markov Chain Monte Carlo Sampling Christopher Jennison Department of Mathematical Sciences, University of Bath, UK http://people.bath.ac.uk/mascj Adriana Ibrahim Institute
More information17 : Markov Chain Monte Carlo
10-708: Probabilistic Graphical Models, Spring 2015 17 : Markov Chain Monte Carlo Lecturer: Eric P. Xing Scribes: Heran Lin, Bin Deng, Yun Huang 1 Review of Monte Carlo Methods 1.1 Overview Monte Carlo
More informationComputer Vision Group Prof. Daniel Cremers. 11. Sampling Methods
Prof. Daniel Cremers 11. Sampling Methods Sampling Methods Sampling Methods are widely used in Computer Science as an approximation of a deterministic algorithm to represent uncertainty without a parametric
More informationMCMC: Markov Chain Monte Carlo
I529: Machine Learning in Bioinformatics (Spring 2013) MCMC: Markov Chain Monte Carlo Yuzhen Ye School of Informatics and Computing Indiana University, Bloomington Spring 2013 Contents Review of Markov
More informationStochastic optimization Markov Chain Monte Carlo
Stochastic optimization Markov Chain Monte Carlo Ethan Fetaya Weizmann Institute of Science 1 Motivation Markov chains Stationary distribution Mixing time 2 Algorithms Metropolis-Hastings Simulated Annealing
More informationMCMC Sampling for Bayesian Inference using L1-type Priors
MÜNSTER MCMC Sampling for Bayesian Inference using L1-type Priors (what I do whenever the ill-posedness of EEG/MEG is just not frustrating enough!) AG Imaging Seminar Felix Lucka 26.06.2012 , MÜNSTER Sampling
More informationConvex Optimization CMU-10725
Convex Optimization CMU-10725 Simulated Annealing Barnabás Póczos & Ryan Tibshirani Andrey Markov Markov Chains 2 Markov Chains Markov chain: Homogen Markov chain: 3 Markov Chains Assume that the state
More informationMarkov Chain Monte Carlo Methods
Markov Chain Monte Carlo Methods p. /36 Markov Chain Monte Carlo Methods Michel Bierlaire michel.bierlaire@epfl.ch Transport and Mobility Laboratory Markov Chain Monte Carlo Methods p. 2/36 Markov Chains
More informationST 740: Markov Chain Monte Carlo
ST 740: Markov Chain Monte Carlo Alyson Wilson Department of Statistics North Carolina State University October 14, 2012 A. Wilson (NCSU Stsatistics) MCMC October 14, 2012 1 / 20 Convergence Diagnostics:
More informationBayesian Inference and MCMC
Bayesian Inference and MCMC Aryan Arbabi Partly based on MCMC slides from CSC412 Fall 2018 1 / 18 Bayesian Inference - Motivation Consider we have a data set D = {x 1,..., x n }. E.g each x i can be the
More informationMARKOV CHAIN MONTE CARLO
MARKOV CHAIN MONTE CARLO RYAN WANG Abstract. This paper gives a brief introduction to Markov Chain Monte Carlo methods, which offer a general framework for calculating difficult integrals. We start with
More informationKernel adaptive Sequential Monte Carlo
Kernel adaptive Sequential Monte Carlo Ingmar Schuster (Paris Dauphine) Heiko Strathmann (University College London) Brooks Paige (Oxford) Dino Sejdinovic (Oxford) December 7, 2015 1 / 36 Section 1 Outline
More informationLecture 7 and 8: Markov Chain Monte Carlo
Lecture 7 and 8: Markov Chain Monte Carlo 4F13: Machine Learning Zoubin Ghahramani and Carl Edward Rasmussen Department of Engineering University of Cambridge http://mlg.eng.cam.ac.uk/teaching/4f13/ Ghahramani
More informationMinicourse on: Markov Chain Monte Carlo: Simulation Techniques in Statistics
Minicourse on: Markov Chain Monte Carlo: Simulation Techniques in Statistics Eric Slud, Statistics Program Lecture 1: Metropolis-Hastings Algorithm, plus background in Simulation and Markov Chains. Lecture
More informationA quick introduction to Markov chains and Markov chain Monte Carlo (revised version)
A quick introduction to Markov chains and Markov chain Monte Carlo (revised version) Rasmus Waagepetersen Institute of Mathematical Sciences Aalborg University 1 Introduction These notes are intended to
More informationMCMC algorithms for fitting Bayesian models
MCMC algorithms for fitting Bayesian models p. 1/1 MCMC algorithms for fitting Bayesian models Sudipto Banerjee sudiptob@biostat.umn.edu University of Minnesota MCMC algorithms for fitting Bayesian models
More informationMarkov Chain Monte Carlo
1 Motivation 1.1 Bayesian Learning Markov Chain Monte Carlo Yale Chang In Bayesian learning, given data X, we make assumptions on the generative process of X by introducing hidden variables Z: p(z): prior
More informationMarkov chain Monte Carlo
Markov chain Monte Carlo Feng Li feng.li@cufe.edu.cn School of Statistics and Mathematics Central University of Finance and Economics Revised on April 24, 2017 Today we are going to learn... 1 Markov Chains
More informationProbabilistic Graphical Models Lecture 17: Markov chain Monte Carlo
Probabilistic Graphical Models Lecture 17: Markov chain Monte Carlo Andrew Gordon Wilson www.cs.cmu.edu/~andrewgw Carnegie Mellon University March 18, 2015 1 / 45 Resources and Attribution Image credits,
More information16 : Markov Chain Monte Carlo (MCMC)
10-708: Probabilistic Graphical Models 10-708, Spring 2014 16 : Markov Chain Monte Carlo MCMC Lecturer: Matthew Gormley Scribes: Yining Wang, Renato Negrinho 1 Sampling from low-dimensional distributions
More informationMarkov Chain Monte Carlo methods
Markov Chain Monte Carlo methods By Oleg Makhnin 1 Introduction a b c M = d e f g h i 0 f(x)dx 1.1 Motivation 1.1.1 Just here Supresses numbering 1.1.2 After this 1.2 Literature 2 Method 2.1 New math As
More informationReminder of some Markov Chain properties:
Reminder of some Markov Chain properties: 1. a transition from one state to another occurs probabilistically 2. only state that matters is where you currently are (i.e. given present, future is independent
More informationWinter 2019 Math 106 Topics in Applied Mathematics. Lecture 9: Markov Chain Monte Carlo
Winter 2019 Math 106 Topics in Applied Mathematics Data-driven Uncertainty Quantification Yoonsang Lee (yoonsang.lee@dartmouth.edu) Lecture 9: Markov Chain Monte Carlo 9.1 Markov Chain A Markov Chain Monte
More informationSTA 294: Stochastic Processes & Bayesian Nonparametrics
MARKOV CHAINS AND CONVERGENCE CONCEPTS Markov chains are among the simplest stochastic processes, just one step beyond iid sequences of random variables. Traditionally they ve been used in modelling a
More informationComputer Vision Group Prof. Daniel Cremers. 11. Sampling Methods: Markov Chain Monte Carlo
Group Prof. Daniel Cremers 11. Sampling Methods: Markov Chain Monte Carlo Markov Chain Monte Carlo In high-dimensional spaces, rejection sampling and importance sampling are very inefficient An alternative
More informationMarkov chain Monte Carlo
Markov chain Monte Carlo Markov chain Monte Carlo (MCMC) Gibbs and Metropolis Hastings Slice sampling Practical details Iain Murray http://iainmurray.net/ Reminder Need to sample large, non-standard distributions:
More informationConvergence Rate of Markov Chains
Convergence Rate of Markov Chains Will Perkins April 16, 2013 Convergence Last class we saw that if X n is an irreducible, aperiodic, positive recurrent Markov chain, then there exists a stationary distribution
More informationMarkov chain Monte Carlo
Markov chain Monte Carlo Peter Beerli October 10, 2005 [this chapter is highly influenced by chapter 1 in Markov chain Monte Carlo in Practice, eds Gilks W. R. et al. Chapman and Hall/CRC, 1996] 1 Short
More informationI. Bayesian econometrics
I. Bayesian econometrics A. Introduction B. Bayesian inference in the univariate regression model C. Statistical decision theory D. Large sample results E. Diffuse priors F. Numerical Bayesian methods
More informationMarkov chain Monte Carlo
1 / 26 Markov chain Monte Carlo Timothy Hanson 1 and Alejandro Jara 2 1 Division of Biostatistics, University of Minnesota, USA 2 Department of Statistics, Universidad de Concepción, Chile IAP-Workshop
More informationCPSC 540: Machine Learning
CPSC 540: Machine Learning MCMC and Non-Parametric Bayes Mark Schmidt University of British Columbia Winter 2016 Admin I went through project proposals: Some of you got a message on Piazza. No news is
More informationUse of Eigen values and eigen vectors to calculate higher transition probabilities
The Lecture Contains : Markov-Bernoulli Chain Note Assignments Random Walks which are correlated Actual examples of Markov Chains Examples Use of Eigen values and eigen vectors to calculate higher transition
More informationMarkov Chain Monte Carlo The Metropolis-Hastings Algorithm
Markov Chain Monte Carlo The Metropolis-Hastings Algorithm Anthony Trubiano April 11th, 2018 1 Introduction Markov Chain Monte Carlo (MCMC) methods are a class of algorithms for sampling from a probability
More informationStat 451 Lecture Notes Markov Chain Monte Carlo. Ryan Martin UIC
Stat 451 Lecture Notes 07 12 Markov Chain Monte Carlo Ryan Martin UIC www.math.uic.edu/~rgmartin 1 Based on Chapters 8 9 in Givens & Hoeting, Chapters 25 27 in Lange 2 Updated: April 4, 2016 1 / 42 Outline
More informationCSC 2541: Bayesian Methods for Machine Learning
CSC 2541: Bayesian Methods for Machine Learning Radford M. Neal, University of Toronto, 2011 Lecture 3 More Markov Chain Monte Carlo Methods The Metropolis algorithm isn t the only way to do MCMC. We ll
More information27 : Distributed Monte Carlo Markov Chain. 1 Recap of MCMC and Naive Parallel Gibbs Sampling
10-708: Probabilistic Graphical Models 10-708, Spring 2014 27 : Distributed Monte Carlo Markov Chain Lecturer: Eric P. Xing Scribes: Pengtao Xie, Khoa Luu In this scribe, we are going to review the Parallel
More informationStat 516, Homework 1
Stat 516, Homework 1 Due date: October 7 1. Consider an urn with n distinct balls numbered 1,..., n. We sample balls from the urn with replacement. Let N be the number of draws until we encounter a ball
More informationComputer Vision Group Prof. Daniel Cremers. 14. Sampling Methods
Prof. Daniel Cremers 14. Sampling Methods Sampling Methods Sampling Methods are widely used in Computer Science as an approximation of a deterministic algorithm to represent uncertainty without a parametric
More information16 : Approximate Inference: Markov Chain Monte Carlo
10-708: Probabilistic Graphical Models 10-708, Spring 2017 16 : Approximate Inference: Markov Chain Monte Carlo Lecturer: Eric P. Xing Scribes: Yuan Yang, Chao-Ming Yen 1 Introduction As the target distribution
More informationMarkov Chain Monte Carlo (MCMC)
Markov Chain Monte Carlo (MCMC Dependent Sampling Suppose we wish to sample from a density π, and we can evaluate π as a function but have no means to directly generate a sample. Rejection sampling can
More informationA Bayesian Approach to Phylogenetics
A Bayesian Approach to Phylogenetics Niklas Wahlberg Based largely on slides by Paul Lewis (www.eeb.uconn.edu) An Introduction to Bayesian Phylogenetics Bayesian inference in general Markov chain Monte
More informationMCMC Methods: Gibbs and Metropolis
MCMC Methods: Gibbs and Metropolis Patrick Breheny February 28 Patrick Breheny BST 701: Bayesian Modeling in Biostatistics 1/30 Introduction As we have seen, the ability to sample from the posterior distribution
More informationF denotes cumulative density. denotes probability density function; (.)
BAYESIAN ANALYSIS: FOREWORDS Notation. System means the real thing and a model is an assumed mathematical form for the system.. he probability model class M contains the set of the all admissible models
More informationThe Bias-Variance dilemma of the Monte Carlo. method. Technion - Israel Institute of Technology, Technion City, Haifa 32000, Israel
The Bias-Variance dilemma of the Monte Carlo method Zlochin Mark 1 and Yoram Baram 1 Technion - Israel Institute of Technology, Technion City, Haifa 32000, Israel fzmark,baramg@cs.technion.ac.il Abstract.
More informationMarkov Chain Monte Carlo
Markov Chain Monte Carlo Recall: To compute the expectation E ( h(y ) ) we use the approximation E(h(Y )) 1 n n h(y ) t=1 with Y (1),..., Y (n) h(y). Thus our aim is to sample Y (1),..., Y (n) from f(y).
More informationMarkov Processes. Stochastic process. Markov process
Markov Processes Stochastic process movement through a series of well-defined states in a way that involves some element of randomness for our purposes, states are microstates in the governing ensemble
More informationSAMPLING ALGORITHMS. In general. Inference in Bayesian models
SAMPLING ALGORITHMS SAMPLING ALGORITHMS In general A sampling algorithm is an algorithm that outputs samples x 1, x 2,... from a given distribution P or density p. Sampling algorithms can for example be
More informationResults: MCMC Dancers, q=10, n=500
Motivation Sampling Methods for Bayesian Inference How to track many INTERACTING targets? A Tutorial Frank Dellaert Results: MCMC Dancers, q=10, n=500 1 Probabilistic Topological Maps Results Real-Time
More informationMonte Carlo in Bayesian Statistics
Monte Carlo in Bayesian Statistics Matthew Thomas SAMBa - University of Bath m.l.thomas@bath.ac.uk December 4, 2014 Matthew Thomas (SAMBa) Monte Carlo in Bayesian Statistics December 4, 2014 1 / 16 Overview
More informationMarkov chain Monte Carlo Lecture 9
Markov chain Monte Carlo Lecture 9 David Sontag New York University Slides adapted from Eric Xing and Qirong Ho (CMU) Limitations of Monte Carlo Direct (unconditional) sampling Hard to get rare events
More informationPattern Recognition and Machine Learning. Bishop Chapter 11: Sampling Methods
Pattern Recognition and Machine Learning Chapter 11: Sampling Methods Elise Arnaud Jakob Verbeek May 22, 2008 Outline of the chapter 11.1 Basic Sampling Algorithms 11.2 Markov Chain Monte Carlo 11.3 Gibbs
More informationOn the Optimal Scaling of the Modified Metropolis-Hastings algorithm
On the Optimal Scaling of the Modified Metropolis-Hastings algorithm K. M. Zuev & J. L. Beck Division of Engineering and Applied Science California Institute of Technology, MC 4-44, Pasadena, CA 925, USA
More informationMarkov Chain Monte Carlo (MCMC)
School of Computer Science 10-708 Probabilistic Graphical Models Markov Chain Monte Carlo (MCMC) Readings: MacKay Ch. 29 Jordan Ch. 21 Matt Gormley Lecture 16 March 14, 2016 1 Homework 2 Housekeeping Due
More informationThe Particle Filter. PD Dr. Rudolph Triebel Computer Vision Group. Machine Learning for Computer Vision
The Particle Filter Non-parametric implementation of Bayes filter Represents the belief (posterior) random state samples. by a set of This representation is approximate. Can represent distributions that
More informationCS242: Probabilistic Graphical Models Lecture 7B: Markov Chain Monte Carlo & Gibbs Sampling
CS242: Probabilistic Graphical Models Lecture 7B: Markov Chain Monte Carlo & Gibbs Sampling Professor Erik Sudderth Brown University Computer Science October 27, 2016 Some figures and materials courtesy
More informationSC7/SM6 Bayes Methods HT18 Lecturer: Geoff Nicholls Lecture 2: Monte Carlo Methods Notes and Problem sheets are available at http://www.stats.ox.ac.uk/~nicholls/bayesmethods/ and via the MSc weblearn pages.
More informationQuantifying Uncertainty
Sai Ravela M. I. T Last Updated: Spring 2013 1 Markov Chain Monte Carlo Monte Carlo sampling made for large scale problems via Markov Chains Monte Carlo Sampling Rejection Sampling Importance Sampling
More information18 : Advanced topics in MCMC. 1 Gibbs Sampling (Continued from the last lecture)
10-708: Probabilistic Graphical Models 10-708, Spring 2014 18 : Advanced topics in MCMC Lecturer: Eric P. Xing Scribes: Jessica Chemali, Seungwhan Moon 1 Gibbs Sampling (Continued from the last lecture)
More informationLecture 5. G. Cowan Lectures on Statistical Data Analysis Lecture 5 page 1
Lecture 5 1 Probability (90 min.) Definition, Bayes theorem, probability densities and their properties, catalogue of pdfs, Monte Carlo 2 Statistical tests (90 min.) general concepts, test statistics,
More informationMarkov Chain Monte Carlo, Numerical Integration
Markov Chain Monte Carlo, Numerical Integration (See Statistics) Trevor Gallen Fall 2015 1 / 1 Agenda Numerical Integration: MCMC methods Estimating Markov Chains Estimating latent variables 2 / 1 Numerical
More informationStochastic Simulation
Stochastic Simulation Ulm University Institute of Stochastics Lecture Notes Dr. Tim Brereton Summer Term 2015 Ulm, 2015 2 Contents 1 Discrete-Time Markov Chains 5 1.1 Discrete-Time Markov Chains.....................
More informationMarkov Chains Handout for Stat 110
Markov Chains Handout for Stat 0 Prof. Joe Blitzstein (Harvard Statistics Department) Introduction Markov chains were first introduced in 906 by Andrey Markov, with the goal of showing that the Law of
More informationMSc MT15. Further Statistical Methods: MCMC. Lecture 5-6: Markov chains; Metropolis Hastings MCMC. Notes and Practicals available at
MSc MT15. Further Statistical Methods: MCMC Lecture 5-6: Markov chains; Metropolis Hastings MCMC Notes and Practicals available at www.stats.ox.ac.uk\ nicholls\mscmcmc15 Markov chain Monte Carlo Methods
More informationEco517 Fall 2013 C. Sims MCMC. October 8, 2013
Eco517 Fall 2013 C. Sims MCMC October 8, 2013 c 2013 by Christopher A. Sims. This document may be reproduced for educational and research purposes, so long as the copies contain this notice and are retained
More informationIntroduction to Computational Biology Lecture # 14: MCMC - Markov Chain Monte Carlo
Introduction to Computational Biology Lecture # 14: MCMC - Markov Chain Monte Carlo Assaf Weiner Tuesday, March 13, 2007 1 Introduction Today we will return to the motif finding problem, in lecture 10
More informationBayesian Estimation of Input Output Tables for Russia
Bayesian Estimation of Input Output Tables for Russia Oleg Lugovoy (EDF, RANE) Andrey Polbin (RANE) Vladimir Potashnikov (RANE) WIOD Conference April 24, 2012 Groningen Outline Motivation Objectives Bayesian
More informationSome Results on the Ergodicity of Adaptive MCMC Algorithms
Some Results on the Ergodicity of Adaptive MCMC Algorithms Omar Khalil Supervisor: Jeffrey Rosenthal September 2, 2011 1 Contents 1 Andrieu-Moulines 4 2 Roberts-Rosenthal 7 3 Atchadé and Fort 8 4 Relationship
More informationLikelihood-free MCMC
Bayesian inference for stable distributions with applications in finance Department of Mathematics University of Leicester September 2, 2011 MSc project final presentation Outline 1 2 3 4 Classical Monte
More informationMarkov Chain Monte Carlo Using the Ratio-of-Uniforms Transformation. Luke Tierney Department of Statistics & Actuarial Science University of Iowa
Markov Chain Monte Carlo Using the Ratio-of-Uniforms Transformation Luke Tierney Department of Statistics & Actuarial Science University of Iowa Basic Ratio of Uniforms Method Introduced by Kinderman and
More information19 : Slice Sampling and HMC
10-708: Probabilistic Graphical Models 10-708, Spring 2018 19 : Slice Sampling and HMC Lecturer: Kayhan Batmanghelich Scribes: Boxiang Lyu 1 MCMC (Auxiliary Variables Methods) In inference, we are often
More informationMarkov Chains and MCMC
Markov Chains and MCMC CompSci 590.02 Instructor: AshwinMachanavajjhala Lecture 4 : 590.02 Spring 13 1 Recap: Monte Carlo Method If U is a universe of items, and G is a subset satisfying some property,
More informationConnections between score matching, contrastive divergence, and pseudolikelihood for continuous-valued variables. Revised submission to IEEE TNN
Connections between score matching, contrastive divergence, and pseudolikelihood for continuous-valued variables Revised submission to IEEE TNN Aapo Hyvärinen Dept of Computer Science and HIIT University
More informationHamiltonian Monte Carlo for Scalable Deep Learning
Hamiltonian Monte Carlo for Scalable Deep Learning Isaac Robson Department of Statistics and Operations Research, University of North Carolina at Chapel Hill isrobson@email.unc.edu BIOS 740 May 4, 2018
More informationBayesian Phylogenetics:
Bayesian Phylogenetics: an introduction Marc A. Suchard msuchard@ucla.edu UCLA Who is this man? How sure are you? The one true tree? Methods we ve learned so far try to find a single tree that best describes
More informationControl Variates for Markov Chain Monte Carlo
Control Variates for Markov Chain Monte Carlo Dellaportas, P., Kontoyiannis, I., and Tsourti, Z. Dept of Statistics, AUEB Dept of Informatics, AUEB 1st Greek Stochastics Meeting Monte Carlo: Probability
More informationMarkov Chain Monte Carlo Lecture 4
The local-trap problem refers to that in simulations of a complex system whose energy landscape is rugged, the sampler gets trapped in a local energy minimum indefinitely, rendering the simulation ineffective.
More informationTheory of Stochastic Processes 8. Markov chain Monte Carlo
Theory of Stochastic Processes 8. Markov chain Monte Carlo Tomonari Sei sei@mist.i.u-tokyo.ac.jp Department of Mathematical Informatics, University of Tokyo June 8, 2017 http://www.stat.t.u-tokyo.ac.jp/~sei/lec.html
More informationSimulation - Lectures - Part III Markov chain Monte Carlo
Simulation - Lectures - Part III Markov chain Monte Carlo Julien Berestycki Part A Simulation and Statistical Programming Hilary Term 2018 Part A Simulation. HT 2018. J. Berestycki. 1 / 50 Outline Markov
More informationComputer Vision Group Prof. Daniel Cremers. 10a. Markov Chain Monte Carlo
Group Prof. Daniel Cremers 10a. Markov Chain Monte Carlo Markov Chain Monte Carlo In high-dimensional spaces, rejection sampling and importance sampling are very inefficient An alternative is Markov Chain
More informationRandom Walks A&T and F&S 3.1.2
Random Walks A&T 110-123 and F&S 3.1.2 As we explained last time, it is very difficult to sample directly a general probability distribution. - If we sample from another distribution, the overlap will
More informationSampling from complex probability distributions
Sampling from complex probability distributions Louis J. M. Aslett (louis.aslett@durham.ac.uk) Department of Mathematical Sciences Durham University UTOPIAE Training School II 4 July 2017 1/37 Motivation
More informationMCMC and Gibbs Sampling. Sargur Srihari
MCMC and Gibbs Sampling Sargur srihari@cedar.buffalo.edu 1 Topics 1. Markov Chain Monte Carlo 2. Markov Chains 3. Gibbs Sampling 4. Basic Metropolis Algorithm 5. Metropolis-Hastings Algorithm 6. Slice
More informationReview. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Review DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Probability and statistics Probability: Framework for dealing with
More informationBayesian Inference in GLMs. Frequentists typically base inferences on MLEs, asymptotic confidence
Bayesian Inference in GLMs Frequentists typically base inferences on MLEs, asymptotic confidence limits, and log-likelihood ratio tests Bayesians base inferences on the posterior distribution of the unknowns
More informationThree examples of a Practical Exact Markov Chain Sampling
Three examples of a Practical Exact Markov Chain Sampling Zdravko Botev November 2007 Abstract We present three examples of exact sampling from complex multidimensional densities using Markov Chain theory
More informationPhysics 403. Segev BenZvi. Numerical Methods, Maximum Likelihood, and Least Squares. Department of Physics and Astronomy University of Rochester
Physics 403 Numerical Methods, Maximum Likelihood, and Least Squares Segev BenZvi Department of Physics and Astronomy University of Rochester Table of Contents 1 Review of Last Class Quadratic Approximation
More informationKernel Sequential Monte Carlo
Kernel Sequential Monte Carlo Ingmar Schuster (Paris Dauphine) Heiko Strathmann (University College London) Brooks Paige (Oxford) Dino Sejdinovic (Oxford) * equal contribution April 25, 2016 1 / 37 Section
More informationSampling Algorithms for Probabilistic Graphical models
Sampling Algorithms for Probabilistic Graphical models Vibhav Gogate University of Washington References: Chapter 12 of Probabilistic Graphical models: Principles and Techniques by Daphne Koller and Nir
More information