Bayesian Methods with Monte Carlo Markov Chains II

Size: px

Start display at page:

Download "Bayesian Methods with Monte Carlo Markov Chains II"

Brook Johnson
6 years ago
Views:

1 Bayesian Methods with Monte Carlo Markov Chains II Henry Horng-Shing Lu Institute of Statistics National Chiao Tung University 1

2 Part 3 An Example in Genetics 2

3 Example 1 in Genetics (1) Two linked loci with alleles A and a, and B and b A, B: dominant a, b: recessive A double heterozygote AaBb will produce gametes of four types: AB, Ab, ab, ab A A A a B A a b B b a B b a b B F (Female) 1- r r (female recombination fraction) M (Male) 1-r r (male recombination fraction) 3

4 Example 1 in Genetics (2) r and r are the recombination rates for male and female Suppose the parental origin of these heterozygote is from the mating of AABB aabb. The problem is to estimate r and r from the offspring of selfed heterozygotes. Fisher, R. A. and Balmukand, B. (1928). The estimation of linkage from the offspring of selfed heterozygotes. Journal of Genetics, 20, ank/handout12.pdf 4

5 Example 1 in Genetics (3) MALE AB ab ab Ab (1-r)/2 (1-r)/2 r/2 r/2 AB AABB aabb aabb AABb (1-r )/2 (1-r) (1-r )/4 (1-r) (1-r )/4 r (1-r )/4 r (1-r )/4 F E M A ab (1-r )/2 AaBb (1-r) (1-r )/4 aabb (1-r) (1-r )/4 aabb r (1-r )/4 Aabb r (1-r )/4 L E ab r /2 AaBB (1-r) r /4 aabb (1-r) r /4 aabb r r /4 AabB r r /4 Ab AABb aabb aabb AAbb r /2 (1-r) r /4 (1-r) r /4 r r /4 r r /4 5

6 Example 1 in Genetics (4) Four distinct phenotypes: A*B*, A*b*, a*b* and a*b*. A*: the dominant phenotype from (Aa, AA, aa). a*: the recessive phenotype from aa. B*: the dominant phenotype from (Bb, BB, bb). b* : the recessive phenotype from bb. A*B*: 9 gametic combinations. A*b*: 3 gametic combinations. a*b*: 3 gametic combinations. a*b*: 1 gametic combination. Total: 16 combinations. 6

7 Example 1 in Genetics (5) Let ϕ = (1 r)(1 r'), then 2 + ϕ PA ( * B*) =, 4 1 ϕ PA ( * b*) = Pa ( * B*) =, 4 ϕ Pa ( * b*) =. 4 7

8 Example 1 in Genetics (6) Hence, the random sample of n from the offspring of selfed heterozygotes will follow a multinomial distribution: 2+ ϕ 1 ϕ 1 ϕ ϕ Multinomial n;,,, We know that ϕ = (1 r)(1 r'), 0 r 1/ 2, and 0 r' 1/ 2. So, 1 ϕ 1/ 4. 8

9 Bayesian for Example 1 in Genetics (1) To simplify computation, we let PA ( * B*) = ϕ1, PA ( * b*) = ϕ2 Pa ( * B*) = ϕ, Pa ( * b*) = ϕ 3 4 The random sample of n from the offspring of selfed heterozygotes will follow a multinomial distribution: [ ] Multinomial n; ϕ, ϕ, ϕ, ϕ

10 Bayesian for Example 1 in Genetics (2) If we assume a Dirichlet prior distribution with parameters: D( α1, α2, α3, α4) to estimate probabilities for A*B*, A*b*,a*B* and a*b*. Recall that A*B*: 9 gametic combinations. A*b*: 3 gametic combinations. a*b*: 3 gametic combinations. a*b*: 1 gametic combination. We consider ( α, α, α, α ) = (9,3,3,1)

11 Bayesian for Example 1 in Genetics (3) Suppose that we observe the data of y = (y1, y2, y3, y4) = (125, 18, 20, 24). So the posterior distribution is also Dirichlet with parameters D(134, 21, 23, 25) The Bayesian estimation for probabilities are: ( ϕ1, ϕ2, ϕ3, ϕ4) =(0.660, 0.103, 0.113, 0.123) 11

12 Bayesian for Example 1 in Genetics (4) Consider the original model, The random sample of n also follow a multinomial distribution: 2+ 1 PA ( * B*) ϕ =, PA ( * b*) = Pa ( * B*) = ϕ, Pa ( * b*) = ϕ ϕ 1 ϕ 1 ϕ ϕ ( y1, y2, y3, y4)~ Multinomial n;,,, We will assume a Beta prior distribution: Beta( ν, ν )

13 Bayesian for Example 1 in Genetics (5) The posterior distribution becomes P y y y y Py (, y, y, y ϕ) P( ϕ) = Py ( 1, y2, y3, y4 ϕ) P( ϕ) dϕ ( ϕ 1, 2, 3, 4). The integration in the above denominator, P( y, y, y, y ϕ) P( ϕ) dϕ, does not have a close form. 13

14 Bayesian for Example 1 in Genetics (6) How to solve this problem? Monte Carlo Markov Chains (MCMC) Method! What value is appropriate for ( ν, ν )?

15 Part 4 Monte Carlo Methods 15

16 Monte Carlo Methods (1) Consider the game of solitaire: what s the chance of winning with a properly shuffled deck? g/wiki/monte_carlo_ method u/local/talks/mcmc_2 004_07_01.ppt? Lose Lose Win Lose Chance of winning is 1 in 4! 16

17 Monte Carlo Methods (2) Hard to compute analytically because winning or losing depends on a complex procedure of reorganizing cards Insight: why not just play a few hands, and see empirically how many do in fact win? More generally, can approximate a probability density function using only samples from that density 17

18 Monte Carlo Methods (3) Given a very large set X and a distribution f(x) over it. We draw a set of N i.i.d. random samples. We can then approximate the distribution using these samples. f(x) X f x 1 x x f x N () i N ( ) = 1( = ) ( ) N N i= 1 18

19 Monte Carlo Methods (4) We can also use these samples to compute expectations: 1 E g gx Eg gxf x N () i N ( ) = ( ) ( ) = ( ) ( ) N N i= 1 x And even use them to find a maximum: () i ˆ = arg max[ f( x )] ( i ) x x 19

20 Monte Carlo Example 4 X,, X be i.i.d. N(0,1) E( )=? 1 Solution: n 4 4 i -, find Use Monte Carlo method to approximation X i 2 1 x 4! /2 4 8 E( X )= x exp( ) dx= = = 3 2π 2 ( )! 2 20

21 Part 5 Monte Carle Method vs. Numerical Integration 21

22 Numerical Integration (1) Theorem (Riemann Integral): If f is continuous, integrable, then the Riemann Sum: n Sn = f ( ci)( xi+ 1 xi), where ci [ xi, xi+ 1] i= 1 S I f( x) dx, a constant n max x x i= 1,.., n n = i+ 1 i n b a 22

23 Numerical Integration (2) Trapezoidal Rule: b a xi = a+ i, equally spaced intervals n b a xi+ 1 xi = h= n n f( xi) + f( xi+ 1) ST = h 2 i= = h[ f( x1) + f( x2) f( xn) + f( xn+ 1)]

24 Numerical Integration (3) An Example of Trapezoidal Rule by R : 24

25 Numerical Integration (4) Simpson s Rule: b b a a+ b Specifically, f ( xdx ) [ f ( a) + 4 f( ) + f( b)] a 6 2 One derivation replaces the integrand f(x) by the quadratic polynomial P(x) which takes the same values as f(x) at the end points a and b and the midpoint m = (a+b) / 2. ( x m)( x b) ( x a)( x b) ( x a)( x m) Px ( ) = f( a) + f( m) + f( b) ( a m)( a b) ( m a)( m b) ( b a)( b m) b a b a a+ b P( x) = [ f ( a) + 4 f( ) + f( b)]

26 Numerical Integration (5) Example of Simpson s Rule by R: x 26

27 Numerical Integration (6) Two Problems in numerical integration: f ( xdx ) 1. (b,a)= (, ), i.e. How to use numerical integration? Logistic transform: 1 y = 1 + e 2. Two or more high dimension integration? Monte Carle Method! x 27

28 Monte Carlo Integration f( x) I = f ( x) dx = g( x) dx, gx ( ) where g(x) is a probability distribution function and let h(x)=f(x)/g(x). If x,..., x ~ g( x), 1 n iid then 1 n n LLN i n i= 1 hx ( ) Ehx ( ( )) = hxgxdx ( ) ( ) = f( xdx ) = I 28

29 Example (1) 1 0 Γ(18) Γ(7) Γ(11) (1 ) =? x x dx 2 (It is !) 57 Numerical Integration by Trapezoidal Rule: 29

30 Example (2) Monte Carlo Integration: Γ(18) Γ (7 + 11) x (1 x) dx= x x (1 x) dx Γ(7) Γ(11) Γ(7) Γ(11) Let x~ Beta(7,11), then Γ (7 + 11) x x (1 x) dx= E( x ) Γ(7) Γ(11) n 1 LLN Γ(18) xi Ex ( ) = x (1 x) dx n n 0 Γ(7) Γ(11) i= 1 30

31 Example by C/C++ and R 31

32 Part 6 Markov Chains 32

33 Markov Chains (1) A Markov chain is a mathematical model for stochastic systems whose states, discrete or continuous, are governed by transition probability. Suppose the random variable X 0, X 1, take state space (Ω) that is a countable set of value. A Markov chain is a process that corresponds to the network. X 0 1 X t 1 X t X X t

34 Markov Chains (2) The current state in Markov chain only depends on the most recent previous states. PX ( = j X = ix, = i,, X = i) t+ 1 t t 1 t = PX ( = j X = i) = t+ 1 where ( i,..., i, j) Ω 0 t Transition Probability orial/lect1_mcmc_intro.pdf 34

35 An Example of Markov Chains Ω = (1,2,3,4,5) X = ( X0, X1,..., X t,...) Ω where X 0 is initial state and so on. P is transition matrix P =

36 Definition (1) Define the probability of going from state i to state j in n time steps as ( n) p = P( X = j X = i) ij t+ n t A state j is accessible from state i if there ( n) are n time steps such that p ij > 0, where n=0,1, A state i is said to communicate with state j (denote: i j ), if it is true that both i is accessible from j and that j is accessible from i. 36

37 Definition (2) A state i has period d(i) if any return to state i must occur in multiples of d(i) time steps. Formally, the period of a state is defined as di n P ii ( ) ( ) = gcd{ : n > 0} If d(i)= 1, then the state is said to be aperiodic; otherwise (d(i)>1), the state is said to be periodic with period d(i). 37

38 Definition (3) A set of states C is a communicating class if every pair of states in C communicates with each other. Every state in a communicating class must have the same period Example: 38

39 Definition (4) A finite Markov chain is said to be irreducible if its state space (Ω) is a communicating class; this means that, in an irreducible Markov chain, it is possible to get to any state from any state. Example: 39

40 Definition (5) A finite state irreducible Markov chain is said to be ergodic if its states are aperiodic Example: 40

41 Definition (6) A state i is said to be transient if, given that we start in state i, there is a non-zero probability that we will never return back to i. Formally, let the random variable T i be the next return time to state i (the hitting time ): T = min{ n: X = i X = i} i Then, state i is transient iff there exists a finite T i such that: n 0 PT< ( i ) < 1 41

42 Definition (7) A state i is said to be recurrent or persistent iff there exists a finite T i such that:. PT< ( ) = 1 i μ = ET [ ] The mean recurrent time. State i is positive recurrent if is finite; otherwise, state i is null recurrent. A state i is said to be ergodic if it is aperiodic and positive recurrent. If all states in a Markov chain are ergodic, then the chain is said to be ergodic. i i μ i 42

43 Stationary Distributions Theorem: If a Markov Chain is irreducible and aperiodic, then ( n) 1 Pij as n, i, j Ω μ j Theorem: If a Markov chain is irreducible and aperiodic, then! π = lim P( X = j) and π = π P, π = 1, j Ω j i ij j i Ω i Ω π where is stationary distribution. j n n 43

44 Definition (7) A Markov chain is said to be reversible, if there is a stationary distribution π such that π P = π P i, j Ω i ij j ji Theorem: if a Markov chain is reversible, then π = π P j i ij i Ω 44

45 An Example of Stationary Distributions A Markov chain: P = The stationary distribution is π = =

46 Properties of Stationary Distributions Regardless of the starting point, the process of irreducible and aperiodic Markov chains will converge to a stationary distribution. The rate of converge depends on properties of the transition probability. 46

47 Part 7 Monte Carlo Markov Chains 47

48 Applications of MCMC Simulation: Ex: n x+ α 1 n x+ β 1 ( xy, )~ f( xy, ) = c y (1 y), x where x= 0,1, 2,..., n, 0 y 1, α, β are known. Integration: computing in high dimensions. Ex: 1 Egy ( ( )) = gyf ( ) ( ydy ). 0 Bayesian Inference: Ex: Posterior distributions, posterior means 48

49 Monte Carlo Markov Chains MCMC method are a class of algorithms for sampling from probability distributions based on constructing a Markov chain that has the desired distribution as its stationary distribution. The state of the chain after a large number of steps is then used as a sample from the desired distribution. 49

50 Inversion Method vs. MCMC (1) Inverse transform sampling, also known as the probability integral transform, is a method of sampling a number at random from any probability distribution given its cumulative distribution function (cdf). form_sampling_method 50

51 Inversion Method vs. MCMC (2) A random variable with a cdf F, then F has a uniform distribution on [0, 1]. The inverse transform sampling method works as follows: 1. Generate a random number from the standard uniform distribution; call this u. 2. Compute the value x such that F(x) = u; call this x chosen. 3. Take x chosen to be the random number drawn from the distribution described by F. 51

52 Inversion Method vs. MCMC (3) For one dimension random variable, Inversion method is good, but for two or more high dimension random variables, Inverse Method maybe not. For two or more high dimension random variables, the marginal distributions for those random variables respectively sometime be calculated difficult with more time. 52

53 Gibbs Sampling One kind of the MCMC methods. The point of Gibbs sampling is that given a multivariate distribution it is simpler to sample from a conditional distribution rather than integrating over a joint distribution. George Casella and Edward I. George. "Explaining the Gibbs sampler". The American Statistician, 46: , (Basic summary and many references.) 53

54 Example 1 (1) To sample x from: where n xy f xy= c y y x x= 0,1, 2,..., n, 0 y 1, n, α, β are known x+ α 1 n x+ β 1 (, )~ (, ) (1 ), c is a constant. One can see that f( x, y) n x n x f ( x y) = y (1 y) ~ Binomial( n, y), f( y) x fxy (, ) f y x y y Beta x n x fx () x+ α 1 n x+ β 1 ( ) = (1 ) ~ ( + α, 54 + β).

55 Example 1 (2) Gibbs sampling Algorithm: Initial Setting: y x 0 ~ Uniform[0,1] or a arbitrary value [0,1] ~ Bin( n, y ) 0 0 For t=0,,n, sample a value (x t+1,y t+1 ) from yt+ 1 ~ Beta( xt + α, n xt + β ) xt+ 1 ~ Bin( n, yt+ 1) Return (x n, y n ) 55

56 Example 1 (3) Under regular conditions: How many steps are needed for convergence? Within an acceptable error, such as i+ 10 i+ 20 x x, y y. t t xt xt 1 t i 11 < 0.001, i t=+ i =+ t t n is large enough, such as n=

57 Example 1 (4) Inversion Method: x is Beta-Binomial distribution. x~ f( x) = f( x, y) dy 1 0 n Γ ( α + β) Γ ( x+ α) Γ( n x+ β) = Γ x ( α) Γ ( β) Γ ( n + α + β) The cdf of x is F(x) that has a uniform distribution on [0, 1]. x n Γ ( α + β) Γ ( i+ α) Γ( n i+ β) F( x) = i= 0 i Γ ( α) Γ ( β) Γ ( n + α + β) 57

58 Gibbs sampling by R (1) 58

59 Gibbs sampling by R (2) Inversion method 59

60 Gibbs sampling by R (3) 60

61 Inversion Method by R 61

62 Gibbs sampling by C/C++ (1) 62

63 Gibbs sampling by C/C++ (2) Inversion method 63

64 Gibbs sampling by C/C++ (3) 64

65 Gibbs sampling by C/C++ (4) 65

66 Gibbs sampling by C/C++ (5) 66

67 Inversion Method by C/C++ (1) ) 67

68 Inversion Method by C/C++ (2) 68

69 Plot Histograms by Maple (1) Figure 1:1000 samples with n=16, α=5 and β=7. Blue-Inversion method Red-Gibbs sampling 69

70 Plot Histograms by Maple (2) 70

71 Probability Histograms by Maple (1) Figure 2: Blue histogram and yellow line are pmf of x. Red histogram is Pˆ( X = x ) = P ( X = x Yi = yi ) from m i = 1 Gibbs sampling. 1 m 71

72 Probability Histograms by Maple (2) The probability histogram of blue histogram of Figure 1 would be similar to the blue probability histogram of Figure 2, when the sample size. The probability histogram of red histogram of Figure 1 would be similar to the red probability histogram of Figure 2, when the iteration n. 72

73 Probability Histograms by Maple (3) 73

74 Exercises Write your own programs similar to those examples presented in this talk, including Example 1 in Genetics and other examples. Write programs for those examples mentioned at the reference web pages. Write programs for the other examples that you know. 74

Winter 2019 Math 106 Topics in Applied Mathematics. Lecture 9: Markov Chain Monte Carlo

Winter 2019 Math 106 Topics in Applied Mathematics Data-driven Uncertainty Quantification Yoonsang Lee (yoonsang.lee@dartmouth.edu) Lecture 9: Markov Chain Monte Carlo 9.1 Markov Chain A Markov Chain Monte