Random Number Generators - a brief assessment of those available

Size: px

Start display at page:

Download "Random Number Generators - a brief assessment of those available"

Gabriella Bond
5 years ago
Views:

1 Random Number Generators - a brief assessment of those available Anna Mills March 30, Introduction Nothing in nature is random...a thing appears random only through the incompleteness of our knowledge. Spinoza In everyday life, we are used to random phenomena, in the drawing of lottery numbers, or in rolling a dice. Spinoza would claim that such things are not truly random. The lottery numbers to emerge, for instance, could be predicted, if one knew the precise starting conditions of the urn and lottery balls and the precise actions of the lottery master in drawing the balls. Here lies the important issue in generating random numbers on a computer. We in fact seek to produce numbers that appear to be random within acceptable parameters. To emphasize this distinction we use the term, pseudorandom. Kahaner, Moler and Nash [1] define five areas in which one should assess a given random number generator, as follows. 1. Quality - Suitable statistical tests should be satisfied. There should be a sufficiently long period before repetition occurs. 2. Efficiency - The generator should be quick to run. Minimal storage must be required. 3. Repeatability - The method should be seedable, to allow an experiment to be reproduced. 1

2 4. Portability - If the method is implemented on a different system it should produce the same results. 5. Simplicity - The generator should be straightforward both to implement and to use. 2 The middle square generator - an unsatisfactory method One of the earliest attempts at computerized random number generation was a middle square generator. An example of such an algorithm is given below. 1. Seed the generator with some integer, z 0 (1, 9999] 2. Then z n+1 is given by 4 middle digits of (z n ) 2, for n as large as required. 3. The n th value, U n (0, 1) is then given by zn An example progression is given below. z 0 = 1010 (z 0 ) 2 = z 1 = 2010 (z 1 ) 2 = z 2 = 4010 (z 2 ) 2 = z 3 = 0801 (z 3 ) 2 = Methods of generation such as this were first suggested by Jon Von Neumann in about 1946 and were very popular in the early days of computer programming. Such methods are flawed, however, since they can easily settle into patterns of very short period. I continued the series above, using MATLAB, and found that it settles into a pattern of period 67. Carrying out a χ 2 test showed the generator not to conform to uniformity over the unit interval. 3 The Fibonacci generator - another flawed method The set of Fibonacci generators, popular in the 1950s, are based on a method of generation similar to that of the Fibonacci sequence. In its simplest form, we 2

3 have an initial seed vector containing two values between 0 and 1, perhaps taken from a table, and the relation x k = x k 1 x k 2 (1) I implemented this in MATLAB, and found the results highly unsatisfactory, settling down to a sequence of period length 6, a section of which is shown in Figure Figure 1: A plot of the values 10:60 of a simple Fibonacci generator Better results are given by sampling from earlier within the distribution to compute the next value, x k. If, for instance, we have, x k = x k 97 x k 33, then we say we have a Fibonacci generator with lags 97 and 33. These particular values were in fact implemented in the random number generators of some supercomputers in the 1960s. However, Fibonacci generators have since been found to fail simple tests for randomness, as in the following example. Take three consecutive values of a sequence, U k 1, U k, U k+1. In ordering these three values, simple combinatorics tells us that there are six possible relative orders, U k < U k 1 < U k+1,..., U k > U k 1 > U k+1. Now, if we examine one particular ordering, say the first on our list, U k < U k 1 < U k+1 (2) 3

4 then we should expect this to occur in approximately one sixth of samples from the distribution. However, this sequence never occurs in the Fibonacci sequence, as shown below. Without loss of generality, we can examine the simplest Fibonacci generator in (1). Then we have { Uk U k 1 if U k > U k 1, U k+1 = U k U k if U k < U k 1. Then, U k < U k 1 U k+1 < U k U k+1 < U k < U k 1, so we can never have the above case in (2). The generalization to systems of lag greater than 1 is a straightforward exercise. This analytical dismissal of the Fibonacci generator is an example of a case where a generator can pass many statistical tests carried out by computation, as do many Fibonacci generators of high lag, but can fail a simple pen and paper test. This is a warning that one must plan carefully before creating a random number generator. 4 The Lehmer generator - a better approach The Lehmer generator is also known as the prime modulus multiplicative linear congruential generator. It is the most widely implemented of the class of linear congruential generators. There are three parameters: modulus m - a large integer which must be prime, multiplier a - an integer in the range (2, 3,..., m 1), additive constant c - an integer, often equal to 0. There must also be an initial value, z 1. The consequent pseudorandom values are then generated by the formula: z k+1 = (az k + c)mod m. This generates a sequence drawn at random ranging from 1 to m 1, without replacement. A combination of parameters which leads to a sequence of numbers with period m 1 is described as having full period. 4

5 As with the middle square generator, the values of z k, are generally normalized to lie on the unit interval. So the final random values are given by U k = z k m for k = 1, 2,.... The success of such a generator is depent on the choice of parameters. With a choice of m = 11, for instance, the multiplier a = 8 yields a sequence of full period, whereas the multiplier a = 9 has a sequence of period length 5, as shown in the sequences below... a = , 8, 9, 6, 4, 10, 3, 2, 5, 7, 1... a = , 9, 4, 3, 5, Park and Miller [8] give suitable choices to be m = = and a = = 7 5. This yields a full period generator. A histogram showing the behaviour of this generator is shown in Figure 2. The MATLAB code used to generate this histogram is given in Section Figure 2: A histogram showing values of the Lehmer generator with parameters a = 7 5, m = The histogram shows that each of the 25 divisions contains approximately 2000 numbers, as we would expect from a uniform distribution. A χ 2 test gives 5

6 us no evidence to reject the hypothesis that the sample is from a uniform distribution. Until the release of MATLAB version 5, MATLAB used a random number generator with the parameters as suggested by Park and Miller. From version 5, however, MATLAB has used a subtract with borrow generator, as described in Section 5. This is much faster and has longer period length than the previous generator. 4.1 RANDU - an embarrassment for IBM The IBM corporation used a Lehmer generator for its SYSTEM/360 product, RANDU. This generator used U 0 odd, U n+1 = (65539 U n )(mod 2 31 ). The first thing to notice about these parameters is that the modulus, m, is not prime. In implementing a linear congruential generator, programmers were eager to produce a fast generator. The code ran more quickly with a modulus that was a power of two, so the value 2 31 was chosen. The generator fails most tests for randomness that examine consecutive triples due to the following relation. 9 U n 6 U n+1 + U n+2 = 0(mod 2 31 ) This is very simple to check using the fragment of code below. The output, test is a null vector so clearly RANDU is disqualified as a random generator. % RANDU produces a vector of supposedly pseudo-random numbers % based on IBM s generator, then tests for relations in % 3-tuples. clear all;format compact; m = 2^31; a = 65539; t = zeros(1,3); z(1) = 1; 6

7 for k = 1:100; for i = 1:3 z(3*(k - 1) + i + 1) = mod(a * z(3*(k - 1) + i), m); t(k,:) = [z(3 * (k - 1) + 1),z(3 * (k - 1) + 2),z(3 * (k - 1) + 3)]; test = mod(9 * t(:, 1) - 6 * t(:, 2) + t(:, 3), m) Such a relation could cause very serious errors in any experiment using random numbers. For instance, it is easy to imagine how using random numbers thus flawed in a 3-dimensional application such as using a Monte Carlo method for estimation of integration in 3 dimensions could cause catastrophically inaccurate results. 5 Subtract with borrow generators - an improved and faster method In 1995, Moler [7] suggested that the period of just over two billion for the Lehmer generator, although acceptable previously, was becoming inadequate due to the development of much faster processors. An alternative is the subtract with borrow generator, as proposed by Marsaglia in 1968 [4]. The scheme does not, as with many previous methods, generate large integers and then normalize to the unit interval, but rather generates floating point numbers. Previous generators had required only one number as initial seed. Marsaglia proposed a generator that required a 35 element vector as initial seed. A vector, U, is required, containing 32 random, uniformly distributed numbers between 0 and 1. These could, perhaps, be generated by sampling from the output of a Lehmer generator shown to give an acceptable uniform output, such as discussed in Section 4. They are treated as forming a cyclic sequence, for instance, if i = 32, then z i+1 is equivalent to z 1. The cycles are easy to calculate, since treating the sequence as a cycle merely corresponds to dealing only with the last five bits of the indices. 7

8 The iteration for generating the numbers is given by U 0 i = U i+p + U i+q b (3) where b is a value inherited from the previous step using the rule zi 0 0 b = 0, z i = zi 0, zi 0 < 0 b = 2 53, z i = zi With the values p = 8 and q = 22, this is known as the RCARRY generator, having period Implemented with values p = 20 and q = 5, Marsaglia has shown the period of the generator to be approximately In the random number implementation in MATLAB (after version 5), there is a further stage to the generation, using the 35 th element of the initial seed and an XOR operation further to improve the quality of the sequence that is generated. Cleve Moler [7], Chairman of the Math Works who produce MATLAB states that the period of the generator is then approximately The Wolfram Research program, Mathematica, also uses a subtract with borrow algorithm for the generation of random numbers 1. 6 A note on implementation issues Although the Lehmer generator with carefully chosen parameters is shown to give satisfactory results, it is very easy to produce code that gives terrible results. To illustrate this, I have written an implementation in Fortran 90 which on first glance perhaps seems a faithful reproduction of the Lehmer generator with the parameters of Section PROGRAM random_int 02 IMPLICIT NONE INTEGER :: seed = 1 05 INTEGER :: a = See The Mathematica Book, section A.9.4 for details 8

9 06 INTEGER :: m = INTEGER, DIMENSION(10000) :: output 08 INTEGER :: i= DO i = 1, seed = modulo(a * seed, m) 12 output(i) = seed 13 END DO 14 PRINT *, seed 15 END PROGRAM random_int OUTPUT: > Park and Miller [8] give an implementation test that the above parameters should yield z as This is clearly not what our Fortran 90 implementation produces, so we must examine what goes wrong when the above code runs. In the execution of the above program, at line 11 at stage k, we could potentially have seed = = m 1. At stage k + 1, we would then have the calculation a (m 1), which with the default integer range of Fortran 90, will cause overflow. This is because the default range for integer values in Fortran 90 is (2 31 1) to (2 31 1). In fact we will encounter overflow at the stage following any value greater than , making all following values inaccurate and potentially not conforming to a uniform distribution. In this case, we can easily eliminate the overflow difficulty by declaring our parameters as reals rather than integers, but the importance of ensuring a suitable implementation of a random generator is clear from this example. Other suitable precautions must clearly be taken when one is faced with a maximum integer of less than , as discussed by Park and Miller [8]. Clearly, the creation of a successful random number generator goes far beyond the invention of a pleasing algorithm. It is important to consider all the aspects of 9

10 quality from Section 5. The fastest generators may take advantage of the characteristics of a specific machine and so not be portable, and a code that is portable may well be slow to run. Perhaps to look for the best random number generator in the abstract is not a useful task, but rather we should choose a particular generator with characteristics and strengths suitable for the application. 7 Accompanying MATLAB codes 7.1 Code to run and analyze a Lehmer generator % LEHMER produces a vector of pseudo-random numbers % based on the Lehmer generator. m = 2^31-1; a = 7^5; z(1) = 1; r = 0; k = 1; while r == 0 ; z(k + 1) = mod(a * z(k), m); if z(k + 1) == z(1); r = 1; k = k + 1; function f = binstest2(a); % BINSTEST sorts the entries of vector, A, then distributes them into 25 % bins [0,0.04],...,[0.96,1.0] and returns the result of a chi-squared % test relating to the expected values for each bin, n/25, where n is the % length of A. A = sort(a); n = length(a); if (A(1) < 0 A(n) > 1) 10

11 disp ( INPUT ERROR: data not in range [0,1] ) return b = zeros(1,25); r = 1; for k = 1:n adv = 0; while adv == 0 if ((r - 1) * 0.04 < A(k) & A(k) <= r * 0.04) b(r) = b(r) + 1; adv = 1; else r = r + 1; % advance to the next bin bar(b); %A chi squared test is carried out chi2 = 0; for k = 1:25 chi2 = chi2 + ( (b(k) - n/25)^2 )/ (n/25); if chi2 < f = 0.05; elseif chi2 < f = 0.01; elseif chi2 < f = 0.001; else f = 0; 11

12 References [1] D. Kahaner: Numerical Methods and Software, Prentice Hall Series in Computational Mathematics (1977) [2] D. E. Knuth: The Art of Computer Programming, Addison-Wesley Vol. 2 Seminumerical Algorithms (1997) [3] M. Donald MacLaren and George Marsaglia: Uniform Random Number Generators, Journal of the Association for Computing Machinery/Volume 12/Number 1 (January, 1965), pp [4] George Marsaglia: Random Numbers Fall Mainly in the Planes, Proceedings of the National Academy of Sciences/Volume 61 (1968), pp [5] George Marsaglia and T. A. Bray: One Line Random Number Generators and Their Use in Combinations, Communications of the ACM/Volume 11/Number 11 (November, 1968), pp [6] George Marsaglia and Arif Azman: Toward a Universal Random Number Generator, Letters in Statistics and Probability/Letters 8 (January, 1990), pp [7] Cleve Moler: Random Thoughts, MATLAB News and Notes (Fall, 1995) [8] Stephen K. Park, Keith. W. Miller: Random Number Generators: Good Ones are Hard to Find, Communications of the ACM/Volume 31/Number 10 (October, 1988), pp

2008 Winton. Review of Statistical Terminology

2008 Winton. Review of Statistical Terminology 1 Review of Statistical Terminology 2 Formal Terminology An experiment is a process whose outcome is not known with certainty The experiment s sample space S is the set of all possible outcomes. A random