Generating pseudo- random numbers

Transcription

1 Generating pseudo- random numbers What are pseudo-random numbers? Numbers exhibiting statistical randomness while being generated by a deterministic process. Easier to generate than true random numbers? What is a good pseudo-random number? From the correct distribution (especially the tails!) Long periodicity Independence Fast to generate Most standard computing languages have packages or functions that generate either U(0,1) random numbers or integers on U(0,2 32 1). rand or unifrnd in MATLAB rand in C/C++ p. 1/??

2 Linear congruential generator (LCG) The linear congruential generator is a simple, fast, and popular way of generating random numbers, x i = (ax i 1 +c) mod m. This will generate integer random numbers on [0,m 1] which are mapped to U(0,1) after division by m. The period of the generator is m if c and m are relative prime a 1 is divisible by all prime factors of m a 1 is divisible by 4 if m is divisible by 4. The LCG has problems with high serial correlation, and is sensitive to the choice of parameters. p. 2/??

3 Mersenne twister Developed in 1997 by Makoto Matsumoto and Takuji Nishimura. Called Mersenne twister since it uses Mersenne prime numbers 2 p 1. The most commonly used version is the MT19937 algorithm. Standard generator in MATLAB (from ver. 7.4), Maple, R, and GNU Scientific Library. Period of the generated numbers is Equidistributed in high dimensions (LCG has problems with d > 5). p. 3/??

4 Simulation algorithms After having generated a sample of pseudo-random numbers u 1,...,u n U(0,1), we can simulate data from a specified distribution function by either: inversion and transformation methods rejection sampling conditional methods p. 4/??

5 Inversion method Let U U(0,1) be a uniform random variable. We want random variable X R from a continuous distribution F. Does a simple transform x = h(u) that is increasing with an inverse h 1 : R [0,1] exist? X = F 1 (U) is a r.v with distribution F. For discrete distributions F 1 (u) does not exist. Take instead inf{x;f(x) u} Limited to cases where: The inverse of the distribution function exists and is easy to evaluate. We want univariate random numbers. p. 5/??

6 Exponential distribution- an example To simulate from the exponential distribution F(x) = 1 exp( λx), so u=rand(1,n); x=-(log(1-u))/lambda; F 1 (u) = λ 1 log(1 u) u frequency x x Left : cdf for Exp(1). Right: histogram of 1000 unit exponential p. 6/??

7 Poisson(2) distribution - an example The distribution function of the Poisson(2) distribution is x F(x) x F(x) Generate a seq of standard uniforms (s.u) u 1,...,u n u i get x i Po(2): F(x i 1 ) < u i F(x i ) For u 1 = , we get x 1 = 3 and for u 2 = , we get x 2 = 2 and so on... p. 7/??

8 Transformation method The inversion method is a special case of a more general class of methods called: transformation method. Theorem 1: Let X have a continuous density f and let h be a differentiable function with a differentiable inverse g = h 1. Then the random variable Z = h(x) has density f(g(z)) g (z). Can be generalized to multivariate densities and cases where the inverse of h does not exist. Examples: Γ(n,λ),N(µ,σ 2 ),χ 2 (1) Theorem 2: Let X and Y be independent random variables that take values in R with densities f x and f y, then the density of Z = X +Y is f z (z) = f x (z t)f y (t)dt. Examples: Γ(k,1),χ 2 (n),bin(n,p) p. 8/??

9 Rejection sampling Suppose we can simulate from some density g. we can use this as the basis for simulating from the continuous density f by simulating Y from g and then accepting this simulated value with a prob proportional to f(y)/g(y). Specifically, let c be a constant s.t f(y) g(y) c, y Algorithm: 1. Simulate Y from g and simulate a random number U. 2. If U f(y)/cg(y), set X = Y. Otherwise return to step 1. Remarks: The way in which we "accept the value Y with prob f(y)/cg(y)" is by generating a r.number U and then accepting Y if U f(y)/cg(y) Since each iteration will, independently, result in an accepted value with prob P(U f(y)/cg(y)) = K = 1/c, the number of iterations follows a geometric distribution with mean c. Examples: Γ(k,1),χ 2 (n),bin(n,p) p. 9/??

10 An example Want to simulate from f(x) x 2 e x ;0 x 1, a truncated gamma distribution exp( x) x x Left: Scaled density and envelope Right: Histogram of simulated data. p. 10/??

11 Conditional Methods If we can decompose a multivariate density into conditional parts, the problem of sampling from a multivariate density reduces to sampling from several univariate densities. f(x 1,...,x n )f 1 (x 1 )f 2 (x 2 x 1 ) f n (x n x n 1,...,x 1 ). The problem is that usually is hard to find the conditional densities since this involves: f 1 (x 1 ) =... f(x 1,...,x n )dx 2...dx n p. 11/??