Lecture 2 : CS6205 Advanced Modeling and Simulation

Transcription

1 Lecture 2 : CS6205 Advanced Modeling and Simulation Lee Hwee Kuan 21 Aug For the purpose of learning stochastic simulations for the first time. We shall only consider probabilities on finite discrete events. For example, a coin flip or a dice throw generated finite discrete events like head-tail or numbers 1 to 6 respectively. On the other hand, an example of continuous event will be the height of a person. Let Ω be a finite discrete set. For the outcome of a dice throw, the members of Ω will be numbers 1, 2, 6. The number of elements is 6 which is of course finite. Let x Ω be a random variable, then we can define a probability distribution P (x) such that, 0 P (x) 1 (1) P (x) = 1 (2) x Ω For the dice throw, we can assign the probabilities P (x = 1) = 1/6, P (x = 2) = 1/6, P (x = 6) = 1/6. 1 Pseudo-random number generators Pseudo-random number generators are routines that returns pseudo-random bit patterns. Set of all possible bit patterns are clearly a finite discrete set. However, often these bit patterns are converted into floating point and the pseudo-random bits are normalized into a number between zero and one. It is enough to generate random bits with a uniform distribution. It will be shown later that non-uniform random bits can be derived from uniform random bits. What does uniform random bits means? Consider a bit string of lenth n. Let Ω be the set of all possible bits in this bit string. There are 2 n possible bit strings. Let U() be a routine that returns a random bit and x be a random variable with x = U(). Then the probability distribution P (x) = 1/2 n for all x. 1.1 General implementation of uniform random number routines As discussed in the first lecture, a computer cannot generate true randomness. Most routines generate a sequence of numbers X 0, X 1, X 2, and claim that they are random and uncorrelated. In fact, by design, the number X n depends on some of the numbers X q, q < n. Hence all pseduo-random number generators that I know are not strictly correlated as claimed. 1.2 Congruential random number generator Given a number X n 1, to generate the next uncorrelated number, take this number X n 1, multiply it by some constant number c and then add it to another constant number a 0. Finally, take modulus. X n = (cx n 1 + a 0 )MODN max (3) 1

2 2.5e+09 't' 2e e+09 1e+09 5e e+08 1e e+09 2e e+09 Figure 1: Random pattern generated by a congruential random number generator in particular when we choose a 0 = 0, c = 16807, N max = and X 0 = 5, we get the sequence, (4) which looks pretty random. If we plot the consecutive points in the xy plane, we get a random pattern as shown in Fig. 1. This random number generator is call the congruential random number generator. 2

3 1.3 Shift Register Random Number Generator Another random number generator called the shift register generator is as follows, X n = X n p XOR X n q (5) Assuming a random sequence has been generated up to X n 1, to generate the nth random number, take two previous random numbers in the sequence and perform bitwise XOR operations. One should choose the values of p and q carefully and generate the initial random sequence carefully to get a long sequence of random numbers. Good values of p and q are, (p, q) = (98, 27), (250, 103), (1279, 216), (1279, 418), (6) 1.4 Lagged Fibonacci Generators A variant of the shift register generator is, Again, the values of p and q must be chosen carefully. 1.5 What does random mean? (9689, 84), (9689, 471), (9689, 1836), (9689, 2444), (9689, 4187) X n = X n p X n q (7) I have showed you how we can generate a sequence of random numbers without actually defining what we mean by random. If we flip an unbiased coin, we say the outcome is random. But it is more than just random, it is random with equal probability of landing on head or tail. Suppose we stick a little piece of chewing gum on one side of the coin so that it is not balanced. Now flip the coin. The outcome will still be random, but this time with higher probability of landing on one side (say head) than the other. Now you see what is the difference between simply saying random and saying random with equal probability? Hence when we claim randomness, we must claim randomness with what probability distribution. For the case of congruential, shift register and lagged Fibonacci generators, they produce a sequence of random numbers with each number happening with equal probability (almost). We call this the uniform distribution. 2 Sampling From a Distribution Suppose we have random number generators that can generate a sequence of uniformly distributed numbers between zero and one U[0, 1]. We denote a sample from a uniform distribution as, x U[0, 1] (8) How can we make use of the values of x to generate another sequence of random numbers drawn from a different distribution? For example, we want to generate random numbers from Gaussian distribution, y 1 2π exp( y 2 /2) (9) 3

4 Using the Box-Muller method, we first calculate the cumulative distribution, F (x) = x 1 2π exp( y 2 /2)dy (10) Then set y as the inverse function of F (x), y = F 1 (x). Where x U[0, 1]. Performing integration, we get 3 Importance Sampling y 1 = 2 ln x 1 cos(2πx 2 ) y 2 = 2 ln x 1 sin(2πx 2 ) (11) We do not always know how to sample form a distribution effectively. For example when the distribution is not discrete or we could not enumerate all the probabilities due to a large probability space. In this case, importance sampling can be an effective way to perform sampling and calculating statistics. I have shown you how to sample from a Gaussian distribution. But sampling from a general function h(x) can be hard even when h(x) can be evaluated precisely. Say suppose we want to calculate the mean of x over the distribution h(x). x h(x) = xh(x)dx (13) If we can sample x i from h(x) then the mean is simply (12) x h(x) 1 n n xi (14) for some large values n. Now suppose we can only sample from a Gaussian g instead, x i g(0, 1), we can still estimate the mean of x over h(x) using, x h(x) 1 n n xi h(x i ) g(x i ) xh g g(x) (15) Eq. (15) is in principle correct except for cases when g goes to zero. At the point when g goes to zero, this point will not never be sampled. Hence the accurracy of mean estimation may suffer when h is far away from g. In the idea case, when h = g then the left-hand side and right-hand side of Eq. (15) are equal. If h g then the sampling will be accurate. 4 Markov Chain Monte Carlo Consider a game of gambling between two parties, A and B. Each is given $2, a coin is flipped and A passes $1 to B if the coin lands on head, otherwise B passes A $1. The game is played until one party wins all the money. There are five discrete outcomes (we often call outcome as state ), let x be the amount of money A owns (and B owns 4-x dollars). All possible outcomes (states) are x = 0, 1, 2, 3, 4. We can also discretize time into the number of times the coin is flipped, we denote it by t = 0, 1, 2, 3,, t = 0 is at the begining of the game and denote the amount of money A owns at time t as x t. Note that x t is a random variable, its value depends on 4

5 the side the coin lands. However, we know for sure x 0 = 2. In other words, we can say P 0 (x = 2) = 1, the subscript 0 is use to denote the probability distribution of x at t = 0. Also P 0 (x 2) = 0. The probability distribution of x after the coin is flipped once is P 1 (x = 0) = 0, P 1 (x = 1) = 0.5, P 1 (x = 2) = 0, P 1 (x = 3) = 0.5 and P 1 (x = 4) = 0. For simplicity in the notation, we shall drop the x = symbol and just write P 1 (0) = 0, P 1 (1) = 0.5, P 1 (2) = 0, P 1 (3) = 0.5, P 1 (4) = 0 (16) The notation can be simplified further by writing P as a vector P 1 (0) 0 P 1 (1) P 1 = P 1 (2) P 1 (3) = P 1 (4) 0 (17) Similarly P 0 = (0, 0, 1, 0, 0) T where T denotes the transpose. What will be the probability distribution of x after two coin flips? P 2 = 0.5 (18) What is the probability distribution after n flips? What is the probability distribution after n flips? We also notice the relationship, P 1 = M P 0 (19) where M is called the transition matrix. Explicitly, Eq. (19) is given by, = (20) Similar relationship can be found using the same M. P 2 = M P 1 (21) Using Eq. (19), P 2 = M 2 P0. In general, P n = M n P0 (22) A plot of P n versus n is shown in Fig. 2. Note that P n (0.5, 0, 0, 0, 0.5) T, which is a very intuitive result. 4.1 Transition Matrices How did I arrive at the explicit transition matrices in Eq. (20)? The basic principle is from Bayes theorem, P (x t, x t 1 ) = M(x t x t 1 )P t 1 (x t 1 ) (23) 5

6 1 0.8 P(x=0), P(x=4) P(x=1), P(x=3) P(x=2) Probability n Figure 2: Plot of P n versus n. P (x t, x t 1 ) is the joint distribution of observing x t and x t 1. M(x t x t 1 ) is the conditional probability and P t 1 (x t 1 ) is the probability of observing x t 1. Then P t (x t ) is, P t (x t ) = x t 1 P (x t, x t 1 ) = x t 1 M(x t x t 1 )P t 1 (x t 1 ) (24) If we write P t, P t 1 as vectors and M as a matrix, then Eq. (24) is in the form of Eq. (19). More examples of transition matrices. 4.2 Markov Chains P t = M P t 1 (25) The transition probabilities M(x t x t 1 ) is conditioned only on the previous time point. This is in fact a special case whereas a more general case is when the probabilities is conditioned on many previous time points. For example, M(x t x t 1, x t 2, x 0 ). In a sense, our gambling game only remembers the previous outcome or previous state. This forgetting the past property is called the Markovian property. This transition rules generate a chain of random variables x 0, x 1,. Note that in actual computation, the random variables x 0, x 1, take specific values x i Ω Homogenous Markov Chain Note that in our gambling game, the rules does not change with time, therefore the transition matrix does not depends on time. We call this a homogenous Markov Chain. 4.3 Properties of Transition Matrices (see Olle Haggstrom for details) Some universal properties of transition matrices are: 0 M(x t x t 1 ) 1 (26) 6

7 x t M(x t x t 1 ) = 1 (27) Given a state space Ω, set of all possible outcomes and a Markov Chain x 0, x 1,. We say that two states s i and s j Ω are connected if within a Markov Chain, there is a non-zero probability of reaching s j from s i. For example, if x 0 = s i then there exist k [1, ) such that x k = s j. Irreducibility and ergodicity Definition 4.1: A Markov Chain x 0, x 1 with state space Ω and transition matrix M is said to be irreducible if for all s i, s j Ω, we have that s i and s j are connected. Otherwise the Markov Chain is called reducible. (ref to Olle Haggstrom for the original definition). Another term for irreducibility is ergodicity. If you see these two words in the context of Markov Chain, they mean the same thing. Aperiodicity Consider a state s i Ω. We can read off the matrix element of the transition matrix, if M(s i s i ) > 0 then there is a chance that two consecutive elements of the Markov Chain remains at the state s i. That is, x k = s i and x k+1 = s i (assume x k = s i happens with non-zero probability). However, for some systems, M(s i s i ) = 0 but M n ((s i s i ) > 0 for some n. Define a set of numbers to be a 0, a 1, such that M a k (s i s i ) > 0. The period of s i is defined as gcd(a 0, a 1, ). The state s i is aperiodic if its period is one. Definition 4.2 A Markov Chain is said to be aperiodic if all its states are aperiodic. Otherwise the chain is said to be periodic. Corollary 4.1 Let x 0, x 1, be an irreducible and aperiodic Markov Chain with state space Ω and transition matrix M. Then there exist an N < such that Mi,j n > 0 for all i, j Ω and all n > N. 5 Homework 1. Implement the congruential random number generator with 5 different sets of numbers c, a 0, X 0 and with N max = (a) Plot the 2D plot as shown in Fig. 1. Clearly label in the plot what numbers c, a 0 and X 0 are used. (b) Derive ways to measure which of your 5 sequence gives a more uniform random number distribution. Support your conclusion with data. 2. (a) Suppose the following sequence of 6 random numbers is drawn from a uniform distribution How do you use these values, to draw 6 random numbers from the triangle distribution p(x) = 4x if 0 x < 0.5, p(x) = 4x + 4 for 0.5 x 1 and p(x) = 0 otherwise. Calculate the sampled mean value of this triangle distribution. (b) Suppose another sequence of random numbers is draw from the triangle distribution Use these numbers to estimate the sampled mean of a stepped distribution given by s(x) = 0.1 for 0 x < 0.4, s(x) = 4.4 for 0.4 x < 0.6 s(x) = 0.2 for 0.6 < x 1 and s(x) = 0 otherwise. 7

8 3. Given a random number generator that draws x U[0, 1], develop an algorithm to draw sequences of random number from a discrete distribution with x = 0, 1, 2, 3, 4, y = 0, 1, 2, 3, 4 and probability of sampling x, y as P (x, y) given in the table x\y (28) 4. Given a coin with head (H) and tail (T). Represent the probabilities of being a H (head) and T (tail) in the form of a column vector (P (H), P (T )) t, t representing the transpose. Apply the following rules to flip the coin, (i) Throw an unbiased dice with six sides, flip the coin if it is head and dice shows either 1 or 4. (ii) Throw an unbiased dice with six sides, flip the coin if it is tail and the dice shows either 1, 3 or 5 (a) what is the transition matrix of this operation? (b) Given that the coin shows H (head) now. What is the probability distribution of the coin after one operation described? That is the probability distribution after three operations? (c) What is the probability distribution of the coin after many flips (number of flips tends to infinity)? 6 References 1. Reference: Olle Haggstrom, Finite Markov Chains and Algorithmic Applications London Mathematical Society 2. D. P. Landau, K. Binder, A Guide to Monte Carlo Simulations in Statistical Physics, Cambridge University Press 3. Jun S. Liu, Monte Carlo Strategies in Scientific Computing, Springer 8