Uniform and Exponential Random Floating Point Number Generation

Transcription

1 Uniform and Exponential Random Floating Point Number Generation Thomas Morgenstern Hochschule Harz, Friedrichstr , D Wernigerode Summary. Pseudo random number generators approximate sequences of independent real random numbers. The output lies within the finite subset of computer numbers and therefore can not be random as real numbers. Classical random number generators produce only a small part of floating-point numbers. Many small numbers are missed and the output can even not be random as floating-point numbers. We give examples of simple simulation studies that lead to wrong results. The common empirical tests are not sensitive enough to detect these deficits. By generating mantissa and exponent independently we successfully construct new uniform and exponential random number generators based on a linear congruential generator. The results of the simulation studies are improved. Knuths empirical tests show no evidence against these generators. We also develop an empirical test adapted to the floating-point numbers format. Key words: random number generation, floating point numbers, stochastic simulation, stochastic programming, statistical tests 2000 Mathematics Subject Classification: 65C10 1 Introduction Randomized algorithms like Monte Carlo simulation require long sequences of random numbers (x i ) i N. These sequences are asked to be samples of independent and identically distributed (i.i.d.) random variables (X i ) i N. Random numbers produced by algorithmic random number generators (RNG) are never random, but should appear to be random to the uninitiated. We call these generators pseudorandom number generators [1]. They should pass statistical tests of the i.i.d. hypothesis [2]. In fact, no pseudo RNG can pass all statistical tests. So we may say that bad RNGs are those that fail simple tests, whereas good RNGs fail only complicated tests that are very hard hard to find and to run [3, 4].

2 2 Thomas Morgenstern 1.1 Definitions To formalize pseudorandom number generators, we follow [2, 3, 5]: Definition 1. A (pseudo-) random number generator (RNG) is a structure (S, t, O, o) where S is a finite set of states, t : S S a transition function, O an output space, and o : S O an output function. s 0 is the initial state evolveing according to s i = t(s i 1 ). The output at step i is u i = o(s i ) O. u 0, u 1, u 2,... are called the random numbers produced by the RNG. Example 1. A good linear congruential generator (LCG), is LCG16807 introduced by Lewis, Goodman and Miller in [6]. It uses m = , a = 7 5, S = Z m and the transition function s i+1 := s i mod For values in O = (0, 1) one usually uses the output function u i := z i Simulation Examples We consider examples showing that some basic simulation tasks in physics or engineering fail, when classical pseudorandom number generators are used. Example 2. Consider a radio signal r(t) = sin(2 π f t) with frequency f. We want to determine the Energy E = T r 2 (t) dt by Monte-Carlo integration. 0 We use the generator from Ex. 1 to produce n random numbers u i and simulate time instances t i := 2 π f T u i. For a medium frequency signal (MF) f = Hz 1 MHz, a micro wave signal (UHF) f = Hz 1 GHz and n = 2 30 random numbers we get the results in Table 1. Table 1. Energy of sin, MC Simulation with LCG16807, n = 2 30 MF UHF T = 1 s T = 1 s T = s T = s 0.0 Example 3. We consider the signal r(t) = e d t sin(2 π f t) with damping d and again calculate the signal energy by Monte-Carlo integration. For the MF signal, the generator from Ex. 1 and n := 2 30 numbers we get the results in Table 2 (incl. true values).

3 Uniform and Exponential Random Floating Point Number Generation 3 Table 2. Energy of damped signals, MC Simulation with LCG16807, n = 2 30 d = 10 E = d = 250 E = T = 1 s T = 1 s T = s T = s Discrete Random Numbers On a digital computer the output is a subset of finitely many discrete numbers o(s) F R. As a finite set has measure 0 within the real numbers we can immediately construct tests these generators fail. Therefore the random numbers generated can not to be i.i.d. real numbers. They are distributed according to a discrete distribution on a finite subset of computer numbers. Definition 2. A floating point number v F(b, p, e min, e max ) with basis b, p significant digits, significand c and exponent e {e min,..., e max } has the value v := ( 1) s c b 1 p b e. A normal floating point number v 0 has digit d 0 0 and a denormal floating point number has d 0 = 0 and e = e min. 2.1 Lower Bits Test We will see, that the generator from Ex. 1 with floating point number output o(s) F(2, 24, 126, 127) generates only a small fraction of these numbers the output can not even be random as floating point numbers. The usual goodness of fit tests (e.g. χ 2 -square, Kolmogorov-Smirnov) are not very sensitive to the of computer numbers used. These tests classify the numbers in relatively few intervals determined by the first significant bits. Testing the last significant bits of random numbers requires some care (see [1, p. 13]). But the states s i of LCG16807 in Ex. 1 could be called super uniform and even the lower bits are uniformly distributed. Testing the b lower bits of the states with a χ 2 -test gives the results in Table 3. Table χ 2 -tests, n = 2 20 states, 3 and 6 lower bits probability occurrences probability occurrences p < p < p < p < p p 9

4 4 Thomas Morgenstern We use the generator from Ex. 1, the output space O = F(2, 24, 126, 127) and the output function o(z) := z If we test the b lower bits of the significands c with a χ 2 test, we get the results in Table 4. Table χ 2 -tests, n = 2 20 significands, 3 and 6 lower bits probability occurrences probability occurrences p < p < p < p < p < p < p p 1 Clearly the generator LCG16807 with this output function fails our test. The reason for this is that the last significant bits of the generated small numbers are 0. 3 Floating Point Number Generation 3.1 Uniform Random Floating Point Numbers We have to construct new generators improving the simulations and passing this test. These new generators generate the exponent e and the significand c independently in two steps and generate denormal numbers as well. Let (S, t, O, o) be a good integer RNG with output O = [0, 2 q 1 ) N 0. We can construct a floating point number generator like in Table 5. Table 5. Floating Point Number Generator Step 1 (exponent generation) e := 1; efound := false; denormal := false; while (e e min and (not efound) ) do i := q 2; while (o(s) < 2 i and i > 0) i := i 1; e := e + (i q + 2); if (i > 0) efound := true; end do; if (e < e min) then e := e min; denormal := true end if; Step 2 (mantissa generation) if (not denormal) then c := 2 q 1 + o(s) else c := o(s) end if; u := ( 1) 0 c 2 1 q 2 e ;

5 Uniform and Exponential Random Floating Point Number Generation Exponential Random Floating Point Numbers In the same fashion we can construct a floating point number generator with exponentially distributed numbers (see Table 6). Table 6. Exponential Floating Point Number Generator Step 1 e := 0; efound := false; while (not efound) do i := q 2; while (o(s) < 2 i and i > 0) i := i 1; e := e + (q 2 i); if (i > 0) efound := true; end do; Step 2 c := 2 q 1 + o(s); u := e + q log 2 (c); 4 Simulation Results We use the generator from Ex. 1 to construct a random floating point generator FloatLCG2 as in Table 5. Example 4. We continue our example 2 with the generator FloatLCG2 and get the results in Table 7. Table 7. Energy of sin, MC Simulation with FloatLCG2, n = 2 30 MF T = 1 s T = s UHF T = 1 s T = s Example 5. We continue our example 3 with the generator FloatLCG2 and get the results in Table 8. Table 8. Energy of damped signals, MC Simulation with FloatLCG2, n = 2 30 d = 10 E = d = 250 E = T = 1 s T = 1 s T = s T = s

6 6 Thomas Morgenstern 5 Mantissa Test We take 20 times n = 2 20 random floating point numbers and test the lower 1, 3 and 6 bits of the mantissa with the χ 2 test. The lowest probability found once for the last 2 bits test is p = Testing the last 3 bits one once finds p = All test values are non-suspicious. Our floating point generator FloatLCG2 passes the test. 6 Conclusions The new pseudorandom number generator FloatLCG2 for generation of uniform random binary floating point numbers is better and solves a wider class of problems than the generators with classical output function. It also passes Knuth s suite of empirical tests [1] and our test, sensitive to floating point numbers. The concept can be used to generate exponential random floating point numbers. But some further work is still required. References [1] Knuth D E (1998) The art of computer programming. Vol. 2: Seminumerical algorithms. third edition, Addison-Wesley, Reading, Mass. [2] L Ecuyer P (1994) Uniform random number generation. Annals of Operations Research 53: [3] L Ecuyer P (2004) Random number generation. In: Gentle J E, Hrdle W, Mori Y, (eds) Handbook of computational statistics. Concepts and methods. Springer, Berlin Heidelberg New York [4] L Ecuyer P (2001) Software for uniform random number generation: Distinguishing the good and the bad. In: Proceedings of the 2001 Winter Simulation Conference. Pistacaway NJ., IEEE Press [5] L Ecuyer P (1998) Random number generation. In: Banks J (ed) Handbook on Simulation. John Wiley, Hoboken, NJ. [6] Lewis P A S, Goodman, A S, Miller, J M (1969) A pseudo-random number generator for the system/360. In: IBM System s Journal 8: [7] ANSI / IEEE Std 754 (1985) IEEE Standard for Binary Floating-Point Arithmetic [8] Morgenstern T (2006) Uniform Random Binary Floating Point Number Generation. In: Proceedings of the 2. Wernigerder Automatisierungs- und Informatiktage. Hochschule Harz, Wernigerode