Expectations, Markov chains, and the Metropolis algorithm

Size: px

Start display at page:

Download "Expectations, Markov chains, and the Metropolis algorithm"

Pierce Norris
5 years ago
Views:

1 Expectations, Markov chains, and the Metropolis algorithm Peter Hoff Departments of Statistics and Biostatistics and the Center for Statistics and the Social Sciences University of Washington

2 Outline of the lecture and lab 1. Expectation of a random variable 2. Law of large numbers 3. Monte Carlo techniques 4. The Ising model 5. Markov chains 6. The Metropolis algorithm 7. Coupling of Markov chains: a card trick 2

3 Probability Probability models for uncertain events are common across the natural and social sciences: Genetics : Given the disease history of a set of parents and grandparents, what is the probability that a child will have a genetic disease? Finance : Given the current state of the economy and market, what is the probability that a set of stocks will increase in value today? Physics : Given a magnetic field around a set of magnetically charged particles, what is the probable orientation of the particles? The probability distribution of a random process tells us how probable certain outcomes of that process are. Let A 1, A 2,..., A K be the possible outcomes of the process. Then 0 < Pr(A k ) < 1 K Pr(A k ) = 1 k=1 3

4 Expectations Often we are interested in the value of a function of the random process: Genetics : Lifespan may be a function of randomly inherited genetic traits, as well as environmental and other factors. Finance : The value of a stock portfolio is the sum of the values of the individual stocks. Physics : The energy stored in a set of magentically charged particles is a function of their orientation. The expected value of a function is a weighted average of the function, evaluated at the possible outcomes of the random process. If f is the function of interest, then the expected value is given by E[f] = f(a 1 ) Pr(A 1 ) + f(a 2 ) Pr(A 2 ) + + f(a K ) Pr(A K ) K = f(a k ) Pr(A k ) k=1 4

5 Dart Throwing Example

6 Dart Throwing Example Suppose someone throws darts uniformly over the 3 3 board. After many throws, what do you expect their average score to be? Letting x and y denote the horizontal and vertical locations of the darts, the score is a function of x and y. score(x, y) = 0 if 3 < x 2 + y 2 = 1 if 2 < x 2 + y 2 3 = 2 if 1 < x 2 + y 2 2 = 3 if x2 + y 2 1 E[score(x, y)] = 0 Pr(3 < x 2 + y 2 ) + 1 Pr(2 < x 2 + y 2 3) + 2 Pr(1 < x 2 + y 2 2) + 3 Pr( x 2 + y 2 1) 6

7 The Law of Large Numbers and the Monte Carlo Method If x 1, x 2,..., x n are independent experiments with the same probability distribution as x, then 1 n [f(x 1) + f(x 2 ) + f(x n )] E[f(x)], and the approximation gets better as n gets larger. The law of large numbers says that if I repeat an experiment over and over, the long run average converges to the expected value. Dart Throwing Example: Sample, or simulate (x 1, y 1 ), (x 2, y 2 ),..., (x n, y n ) Then 1 n score(x i, y i ) E[score(x, y)] n i=1 This is called the Monte Carlo method for approximating expectations. Note we can also use this method to approximate areas of complicated regions, and other integrals. 7

8 A More Complicated Distribution x coordinate y coordinate 8

9 A Much More Complicated Distribution: Magnetic Fields The motion of electrons around a nucleus creates a magnetic field, and so atoms can be thought of as magnets. A magnetic field from a group of atoms is the result of a balance between two different physical principles: Energy Minimization: The lowest energy state of a group of atoms would have them all aligned in the same direction, giving a huge magnetic field. Entropy Maximization: The configuration with all K atoms pointing in a given direction is one configuration out of a huge number of possible configurations (2 K ), and (without considering energy) has an extremely low probability (1/2 K ). 9

10 A Lattice of Magnets Let the state of the system be denoted by a vector or matrix s of + s and - s. s = {..., +,, +,,,, +,, +, +,...} 10

11 The Boltzmann distribution and the Ising Model for Lattices The Boltzmann probability distribution says that the probability that a system is in state s is given by where Pr(s) = e Eng(s)/kT r S e Eng(r)/kT S is the set of all possible states; T is the temperature of the system; k is a known constant. The Ising model says that the energy of a state s is given by Eng(s) = a i j s i s j + b n i=1 s i 11

12 Computing the expected magnetization (or anything else) The magnetization of a state s is given by m(s) = n s i = number of + s - number of s. i=1 Given k and T, can we compute the expected magnetization E[m(s)] with a calculator? E[m(s)] = s S m(s) Pr(s) No, the number of terms in the sum is too large. Can we use Monte Carlo sampling? Pr(s) = e Eng(s)/kT /c(k, T ) No, even for moderate lattice sizes we can t compute c(k, T )! What can we compute? We can compute the relative probabilities of any two states:. Pr(s) Pr(r) = exp{[eng(r) Eng(s)]/kT } 12

13 Markov chains A Markov chain is defined as a sequence x 1, x 2,... of dependent random variables, such that the distribution for the current variable, given the past, only depends on the most recent variable : Pr(x t x t 1, x t 2,..., x 1 ) = Pr(x t x t 1 ) The probability Pr(x t x t 1 ) is called the transition distribution, and is sometimes written T (x t x t 1 ). A Markov chain can be completely described by T and a starting value x 0. Examples: The drunkard s walk Brownian motion 13

14 The Ergodic Theorem for Markov chains Suppose a Markov chain is constructed from a starting value x 0 and transition distribution T : Then, under some conditions x 1 T (x x 0 ) x 2 T (x x 1 ) x 3 T (x x 2 ). x n T (x x n 1 ) 1 n n f(x i ) E[f(x)] = x i=1 f(x) Pr(x), where Pr(x) is the stationary distribution of the Markov chain, and doesn t depend on x 0! 14

15 Example: Two dimensional reflecting random walk At simulation step t, let x t = x t z x y t = y t z y where z x and z y are samples from the uniform distribution on (.1,.1), and are independent from step to step. Questions: Is this a Markov chain? What is your guess for 1 n n i=1 x i? for 1 n n i=1 y i? 15

16 Application to the Ising Model The Metropolis Algorithm: Suppose at step t the system is in state s. The state for step t + 1 is generated as follows: For each site i of the lattice in random order 1. Propose a new state s where s i is the opposite of s i. For example, if i=3 and s = {, +, +,, +,...}, then s = {, +,,, +,...}. 2. Compute the energy difference between states s and s Eng(s ) Eng(s) = a (s i s i)s j + b(s i s i) j:j i 3. Use this to compute the Metropolis ratio R(s, s) = Pr(s ) Pr(s) = exp{[eng(s) Eng(s )]/kt } 4. Replace s i by s i with probability R(s, s). Note if R(s, s) > 1, then we replace with probability 1. Completion of the replacement procedure at each site constitutes the (t + 1)st scan of the algorithm. The resulting value of s is the state at time t

17 Properties of the Metropolis Algorithm The algorithm is a Markov chain. The stationary distribution of the Markov chain is the Boltzmann distribution (just what we wanted!) Pr(s) = e Eng(s)/kT r S e Eng(r)/kT Because of the Ergodic theorem, we can calculate expectations of functions using samples from the Metropolis algorithm: 1 L L f(s (l) ) E[f(s)] = f(s) Pr(s), s S l=1 The algorithm is like the Monte Carlo method, but is based on a Markov chain of dependent samples, rather than independent samples. It is called a Markov chain Monte Carlo (MCMC) algorithm. 17

18 The General Metropolis Algorithm Suppose our goal is to draw samples from a distribution Pr(s), but we can t compute Pr(s), only relative probabilities Pr(s)/ Pr(r). We can construct a Markov chain with Pr(s) as the stationary distribution as follows. At step t, generate s (t+1) from s (t) by 1. sampling a proposal s from a (symmetric) proposal distribution J(s s (t) ) 2. computing the Metropolis ratio R(s, s (t) ) = Pr(s )/ Pr(s (t) ) 3. letting s (t+1) = s with probability R(s, s (t) ) s (t) with probability 1 R(s, s (t) ). Under some additional conditions, we have 1 T T t=1 f(s (t) ) E[f(s)] = s S f(s) Pr(s) as T Why does this work? 18

19 Summary For simple distributions Pr(x), we can calculate expectations of simple functions f(x) by hand. If either Pr(x) or f(x) is complicated, but we can sample from Pr(x), then we can use the Monte Carlo approach to estimating E[f(x)]. If we can t even sample from Pr(x), but we can at least compute relative probabilities Pr(x )/ Pr(x), then we can use the Metropolis or other MCMC algorithms. 19

Introduction to Computational Biology Lecture # 14: MCMC - Markov Chain Monte Carlo

Introduction to Computational Biology Lecture # 14: MCMC - Markov Chain Monte Carlo Assaf Weiner Tuesday, March 13, 2007 1 Introduction Today we will return to the motif finding problem, in lecture 10