B. Maddah ENMG 622 Simulation 11/11/08

Transcription

1 B. Maddah ENMG 622 Simulation 11/11/08 Random-Number Generators (Chapter 7, Law) Overview All stochastic simulations need to generate IID uniformly distributed on (0,1), U(0,1), random numbers. 1 f X ( x) This is the case since all other random variables can be generated once the U(0,1) is generated. We therefore first focus on understanding how the U(0,1) is generated. Early Methods and Background Early methods used physical tools to generate random numbers. E.g., dice, cards and urns. Some of these mechanical generators are still used in lotteries. Electrical generators were also used. 1

2 People used tables generated by these devices (e.g, the RAND corporation million-number table). Other schemes included picking digits randomly from a phone book or picking decimal from the expansion of π to 100,000 places. None of these methods are suitable for use in a computer simulation. Arithmetic or numerical generators were developed to be used with computers. These methods generate random numbers by using a deterministic recursion formula. Example of a Toy Generator Obtain random numbers recursively as follows. 1. Set Z 0 = 7 and i =1. 2. Set Z i = (5Z i 1 + 3) (mod 16). 3. Set U i = Z i / Set i = i+1 and repeat (until simulation terminates). Would you use this? At i = 17, 33,..., the random numbers repeat. That is, the cycle length or the period here is 16. (Too small.) 2

3 Can We Generate Truly Random Numbers? No, not on a computer at least. Practical view: produce stream of numbers that appear to be IID U(0,1) draws by passing a series of statistical tests. The generated numbers are called pseudo random numbers. We subscribe to this view in this course. 3

4 That is, use arithmetic generators. But only if there is enough statistical evidence of their validity. For example, if you want to use the Excel random number generator, RAND(), in a serious application, then check to see how this works and what tests has it gone through. If you care, discussions of the Excel RAND() function are found here Apparently, there has been a problem before Excel Criteria for a Good Random-Number Generator 1. Appear to be distributed uniformly on [0, 1] and independent. 2. Run fast and require limited storage. 3. Be able to reproduce a particular stream of random numbers, for debugging and comparison purposes. 4. Have provision in the generator for a large number of separate streams of random numbers. 5. Be portable: Produce the same numbers on all platforms. Most Random-Number Generators (RNGs) are fast, and take very little memory. But beware. There are many RNGs in use (and in software) that have extremely poor statistical properties. 4

5 Linear Congruential Generators This is the most common type of RNG. It was developed by Lehmer (1954). Many good modern generators build on Lehmer s work. The Linear Congruential Generator (LCG) is an arithmetic generator which generates random numbers, U i, utilizing the following recursion Z = ( az + c)(mod m) i+ 1 U = Z / m i+ 1 i+ 1 i where Z 0 is the seed, m is the modulus, a is the multiplier, and c is the increment. (a, c, m and Z i are all integers.) I.e., to obtain Z i+1, multiply Z i by a, add c, divide by m, set Z i+1 equal to the remainder of the division, and divide Z i+1 by m to get a (pseudo) random number. Objections to LCG, common to all arithmetic generators, are that it s not really random and that there could be repeating cycles once a Z i reappear. More specific objections on LCG relate to the fact that it only generates discrete fractional values (1/m, 2/m, 3/m, ). In addition, LCG could have a poor performance with a not-too-careful choice of parameters, a, c m, and Z 0. 5

6 Full-Period Theorem We obviously need a large m to have long periods. Actually, a large m and a full period (equals to m) would be nice. Can the period be predicted in advance? Can a full period be guaranteed? The full-period theorem partially answers these questions. Theorem. The LCG Z i = (az i 1 + c) (mod m) has full period (m) if and only all if the following hold: 1. c and m are relatively prime (their only common divider is 1). 2. If q is a prime number that divides m, then q divides a If 4 divides m, then 4 divides a 1. Mixed LCGs A LCG with c > 0 is called mixed LCG. The choice of m in a way that makes the division fast is a key issue. A choice of m that achieves this is m = 2 b where b is the maximum number of bits in a word on the computer. 6

7 Multiplicative LCGs A LCG with c = 0 is called multiplicative LCG. Multiplicative LCGs are widely used. They are computationally efficient as addition is not needed. Multiplicative LCGs cannot have full period. Choosing m = 2 b is computationally useful. But it leads to poor statistical properties: Period length not exceeding m/4 and generated number may not be evenly distributed. One infamous generator, RANDU, with m = 2 31 which has been used widely on main frames in the 60s, has been shown to be invalid. This made people doubtful of the result of simulation studies in that time period. A better choice of m for multiplicative LCGs is (the largest prime number smaller than 2 b ). This is called a prime modulus multiplicative LCG (PMMLCG). PMMLCGs have desirable properties. However, the period (of around 2 31 ) is small for large scale realistic simulations. There is also recent evidence of poor statistical performance of PMMLCGs. It s better to use other generators, e.g., composite ones. 7

8 Composite Generators These combine two or more LCGs to get a good generator. Some are shuffling generators, where one generator shuffles the output of another generator which breaks correlations and lead to long periods. There are also differencing generators, Z = ( Z Z )(mod m), i 1i 2i Back in 2000 where Z 1i and Z 2i are obtained from two different LCGs. 8

9 Other generators combine multiple recursive generators (MRGs), where each generator j operates as ( K ) j j Z j, i = aj,1 Z j, i 1 + aj,2 Z j, i aj, q Z j, i q (mod mj) These kinds of generators perform extremely well in terms of long period and statistical properties. Arena uses a combined generator with both MRGs and differencing Z Z Z 32 1, i = (1,403,580 1, i 2 810,728 1, i 3) mod (2 209) Z Z Z 32 2, i = (527,612 2, i 1 1,370,598 2, i 3) mod (2 22,853) Z Z Z U 32 i = ( 1, i 2, i) mod (2 209) i = Z 32 i /(2 209) This generator has a very long period ( ) and good statistical properties (in addition to being fast). This generator need to be fed six seeds (Z 10, Z 11, Z 12, Z 20, Z 21, Z 22 ). Wichmann and Hill developed a generator which works on the U(0,1) output of three other generators, U i = fractional part of (U 1i + U 2i + U 3i ). It is equivalent to a LCG with a long period. Wichmann and Hill generator is implemented in Excel 2003 and

10 In Conclusion Beware of canned generators, especially in nonsimulation software, and especially if poorly documented. Insist on documentation of the generator, check with tried and true ones. Don t seed the generator with the square root of the clock or any other such scatterbrained scheme. We want to get reproducibility of the random-number stream. Reasonably safe idea: o If using Arena, you re ok for most applications. o If not using Arena, use the generator from Appendix 7B of Law s book (see also Pierre L ecuyer website). 10