Example of MC for Continuous State Space

Size: px

Start display at page:

Download "Example of MC for Continuous State Space"

Nicholas Woods
6 years ago
Views:

1 Example of MC for Continuous State Space Notes 2, Sta624 The MC we discussed so far are discrete state space, i.e. the random variable take values in a discrete space. We shall discuss here briefl MC with continuous state space, in particular, the random variables take value in the real line R 1. Recall: A Markov Chain is a sequence of random variables X 0, X 1, X 2,... taking values in a space X. The main propert of the chain is that the past is conditionall independent of the future given the present. Here X = R 1 Transition probabilit matrix needs to be replaced b a transition probabilit function/kernel: p ij becomes p(x ); which is a probabilit densit function for an given value. Or a conditional densit. Interpretation: p(x )dx is the probabilit of going to x given it is at now. Example 1 MC for continuous state space (take values as continuous random variables). But still discrete time (i.e. a chain). In the following example p(x ) is taken to be Uniform (1, 1). State space is the interval (0, 1). X 0 an number between 0 and 1 (or could be from an distribution on (0,1) interval) X 1 Unif (1 X 0, 1) X 2 Unif (1 X 1, 1)... X n Unif (1 X n 1, 1);... This is an MC. If the distribution of X n is convergent at all (here it does), the stationar/limiting distribution (densit f(x)) must satisf the following (integral) equation: f(x) = f()p(x )d for an x (1) We ma check that the probabilit densit function f(x) = 2xI[0 < x < 1] solves the above equation. [with p(x ) = U(1, 1)]. Therefore X n, when n large, will have a densit approximatel equal to f(x) = 2x for 0 < x < 1. Remark: Compare (1) to the equation for a discrete limiting distribution π j = lim Pij: n π j = i π i P ij for an j we see that (1) is just the continuous version of the above equation. 1

2 Detail verifications: Since We compute, using f() = 2, 1 0 p(x )f()d = p(x ) = I [1 <x<1] 1 I [1 <x<1] 2d = 2 d = 2x. 1 x Notice here we onl used the random variables from uniform distributions and end up with a random variable with densit f(x) = 2x. The idea of MCMC is to use an eas p(x ) in the iteration but end up with a random variable with (complicated) f(x) that we desire. Example 2 Suppose X 0 an number between 0 and 1 X 1 Unif (0, 1 X 0 ) X 2 Unif (0, 1 X 1 )... X n Unif (0, 1 X n 1 ), Find the stationar distribution f(x) of this MC. This will also be the approximate distribution of X n for large n. 2

3 Usuall the checking or solving of the equation (1) is not eas. But the following Theorem offers a special easier case. Theorem 1 If a probabilit densit function f(x) and a transition probabilit kernel p(x ) satisf the so called detailed balance equation : then the equation (1) above is satisfied. f(x)p( x) = f()p(x ) (2) We usuall shall check (2) instead of (1). In the example 1, f(x) = 2xI[0 < x < 1] and p(x ) = I [1 <x<1], therefore and I[1 x < < 1] f(x)p( x) = 2xI[0 < x < 1] x f()p(x ) = 2I[0 < < 1] I [1 <x<1] = 2I[0 < x < 1; 1 x < < 1], = 2I[0 < < 1, 1 < x < 1]. Draw two pictures, and ou will convince ourself the two right hand side are the same region. 3

4 Metropollis Tpe Chains: At stage n, suppose a MC takes value n. We generate the candidate value Y using the uniform random walk: Y Unif[ n a, n + a]. Finall we accept the generated Y value as n+1 if f( n ) > U(0, 1) (where U(0, 1) denote another independentl generated random variable) otherwise the MC do not move, i.e. n+1 = n. We can show that this transition kernel function, and densit function f( ) pair satisf the detailed balance equation. The transition probabilit densit function is ( ) 1 2a I[ n a < Y < n + a]p r f( n ) > U since U is from uniform (0,1), therefore we have = 1 { 2a I[ Y n < a] min 1, } f( n ) From here ou can proof the detailed balance equation hold. Remark: If we want to construct an MC that has limiting distribution f( ) b Metropollis, we need to be able to compute the ratio f( n ). This impl we do not need to worr about the constant if f(t) = Cg(t). 4

5 Application in Baesian Analsis In statistical problem of Baesian inference, we are interested in the posterior distribution/densit, or the mean of the posterior etc. posterior densit = constant prior densit function likelihood function. Example 0: X i N(θ, σ 2 ) and the prior on θ is N(µ, τ 2 ). Here σ 2, τ 2, µ are all known. Example 1: Using the R package mcmc. Two dim parameter. In the package, we input the posterior as log of un-normalized posterior. Here prior densit is g(α, λ) = C α trigamma(α) 1 and the likelihood function is a two parameter gamma distribution i λ λ α Γ(α) xα 1 i exp( λx i ). > lupost <- function(theta) { + stopifnot(is.numeric(theta)) + stopifnot(is.finite(theta)) + stopifnot(length(theta) == 2) + alpha <- theta[1] + lambda <- theta[2] + if (alpha <= 0) return(- Inf) + if (lambda <= 0) return(- Inf) + logl <- sum(dgamma(x, shape = alpha, rate = lambda, log = TRUE)) + lpri <- (1 / 2) * log(alpha * trigamma(alpha) - 1) - log(lambda) + return(logl + lpri) + } The onl trick bit is that we define this function to be when parameter is off the allowed space. > out <- metrop(out, blen = 200, nbatch = 500) > alpha <- out$batch[, 1] > lambda <- out$batch[, 2] > t.test(alpha) 5

Stat 5102 Notes: Markov Chain Monte Carlo and Bayesian Inference

Stat 5102 Notes: Markov Chain Monte Carlo and Bayesian Inference Charles J. Geyer April 6, 2009 1 The Problem This is an example of an application of Bayes rule that requires some form of computer analysis.