A GENERAL METHOD FOR PRODUCING RANDOM V ARIABtES IN A COMPUTER. George Marsaglia. Boeing Scientific Research Laboratories Seattle, Washington

Size: px

Start display at page:

Download "A GENERAL METHOD FOR PRODUCING RANDOM V ARIABtES IN A COMPUTER. George Marsaglia. Boeing Scientific Research Laboratories Seattle, Washington"

Barry Sutton
5 years ago
Views:

1 A GENERAL METHOD FOR PRODUCING RANDOM V ARIABtES IN A COMPUTER George Marsaglia Boeing Scientific Research Laboratories Seattle, Washington SUMMARY Many random variables can be approximated quite closely by c(m + V 1 + V 2 + V 3), where c is constant, M is a discrete random variable, and the V's are uniform random variables. Such a representation appears attractive as a method for generating variates in a computer, since M + V 1 + V 2 + V3 can be quickly and simply generated. A typical application of this idea will have M taking from 4 to 7 values; the required X will be produced in the form c(m + V 1 + V 2 + V 3) perhaps 95-99% of the time, and occasionally by the rejection technique, to make the resulting distribution come out right. This paper describes the method and gives examples of how to generate beta, normal, and chi-square variates. INTRODUCTION Given a sequence of independent uniform random variables V 1, V 2,, we are concerned with methods for representing an arbitrary variate X as a function of the V's. Such representations lead to programs for generating X's in a computer. There are any number of ways to represent X as a function of the V's; the problem is to find those which lead to fast, accurate, easy to code programs which occupy little space in the computer. Programs which are very fast and accurate may be 169 based on representing the density of X as a mixture of "easy" and "difficult" densities, the latter being called for infrequently. See Ref. 2 for a general discussion and Refs. 1 and 3 for applications to normal and exponential variables. These programs are very fast, but at the same time they are rather complicated and require hundreds of stored constants. In this paper, we will try to develop a general method that is simpler, but still very fast. The procedure will be explained by way of three examples (a beta variate, the normal distribution, and a chi-square variate), from which the suitability of the method and its general applicability can be inferred. These points are discussed in the final section. Throughout this article, we will use f(x) to represent the density of V 1 + V 2 + V 3, with the V's independent, uniform over [0,1]. This density is a piecewise parabola, once differentiable, or, if you prefer, a parabolic spline curve: n~: -1.5 (x-i)' f(x) =l.~x' - 1.5(x-l)' + A BETA VARIATE 0:S;x:S;1 1:S;x:S;2 1.5 (X_2)2 2:S; x:s; 3 x < 0 orx > 3. We want a method for generating a variate X with a beta density, say b(x) = 105x4(l_X)'2, 0 < X < 1. Our aim is to generate X in the form c (M + V 1 +

2 170 PROCEEDINGS-FALL JOINT COMPUTER CONFERENCE, 1966 V 2 + V s) most of the time, occasionally generating X by the rejection technique in order that the resulting mixture be correct. A good choice for c in this case is.1. We will generate X in the form.1 (M + VI + V 2 + Vs) with as high a frequency as possible. That is, we will put X =.1 (0 + VI + V 2 + Vs) with probability Po, put X =.1 (1 + VI + V 2 + Vs) with probability PI, put X =.1 (2 + VI Vs) with probability P2,..., and put X =.1 (7 + with probability P7' It turns out that we must put Po = 0, since b(x) varies as x 4, while f(x) varies as X2, at the origin. We can, however, fit b(x) very closely with a mixture of the densities of.1(1 + VI + V 2 + Vs),.1(2 + VI + V 2 + Vs),...,.1(7 + VI + V 2 + Vs). We may formulate the problem of finding the best set of p's as follows: Choose PlJP2,..., P7 so as to maximize PI + P2 + Ps + P4 + P5 + P6 + P7 subject to the condition that P'i ~ 0 and PI[10j(10x-1)] + P2[10f(10x-2)] P7 [10f(10x-7)] :::; 105x4(1-x)'2 for 0 :::; x :::; 1 ( 1 ) This optimization problem is similar to those of linear programming-in fact, if we specify condition ( 1) for a suitably fine mesh of x values, we have an ordinary linear programming problem. We find that we can get PI P7 very close to 1, in fact LPi =.9915, and still maintain condition (1). Thus we write 7 105x 4 (1-x)2 = ~ pi,[10f(10x-i)] h(x) i=i for 0 :::; x :::; 1, where the p's are, in order,.0199,.0633,.1297,.1978,.2369,.2155, and.1284, and LPi = The residual function g(x) =.0085h(x) is drawn in Fig. 1. We may generate X with density h (x) by the rejection technique. We summarize with this outline: To generate a beta variate X, density 105x 4 (l-x)2, o < X < 1, 1. with probability P5 =.2369 put X =.1 [5 + VI 2. with probability P6 =.2155 put X =.1[6 + VI + V 2 + V 3 ] 3. with probability P-1 =.1978 put X =.1[4 + VI 4. with probability Ps =.1297 put X =.1 [3 + VI 5. with probability P7 =.1284 put X =.1 [7 + VI 6. with probability P2 =.0633 put X =.1[2 + VI 7. with probability PI =.0199 put X =.1[1 + VI 8. with probability.0085 generate X with density hex) by the rejection technique outlined in Fig. 1. Referring to Fig. 1, we see that the residual function g(x) = b(x) - Lp.i[10f(10x-i)] has a peak on the right end which makes the rejection technique too inefficient; this difficulty may be overcome in several ways-for example, by adding one more step to the procedure, as described in the figure. This will add another step to the outline: 7a. with probability.0044, put X =.05 (17 + VI + V 2 + Vs), and the probability in step 8 will be changed from.0085 to ~ ~ ~~---' g(x)= 105x 4(1.x)2- ~ p. [10f(10x-i)] ;=1' Pl =.0199, P2 =.0633, P3 =.1297, P4 =.1978, P5 =.2369, P6 =.2155, P7 =.1284 To generate a random variable X with density g(x)/.0085, generate pairs x=u 1, y=.072u 2 until y < g(x), then put X = x. The efficiency of this rejection technique is.0085/.072, or 12%. This may be raised to.0041/.014, or 29%, by subtracting another term from g(x). The resulting curve is dotted. In this case, generate pairs x=u 1,. 014 y =.014U 2 until y < 9 (x) [20f(20x-17)], then put X = x. o Figure 1. Method for generating a variate from the residual portion of the beta distribution, by the rejection technique. 1.0

3 A GENERAL METHOD FOR PRODUCING RANDOM VARIABLES 171 THE NOR1VIAL DISTRIBUTION We want a method for generating a standard normal variate X, density (27T )-.5e-. 5X2. We temporarily discard the tails, JxJ > 3.5, and divide the interval -3.5 < x < 3.5 into 10 parts. We will generate X in the form.7(m + VI + V 2 + Vs), where M takes values-5,-4, -3, -2, -1, 0, 1, 2 with probabilities Pl,P2,'..,Ps' We choose the p's so as to maximize the frequency of the representation X.7(M + VI + V 2 + Vs), as follows: Choose PlJP2,'..,Ps so as to maximize PI + P Ps subject to the condition that Pi > 0 and Pi[-f(-x+5)] + P2[-f(-x+4)] Ps[-f( -x-2)] ~ (27T )-.5e- 5X2 7 7 for -3.5 ~ x :s; 3.5 (2) The solution is PI = Ps =.0092, P2 = P7 =.0517, Ps = P6 =.1576, and P4 = P5 =.2767, with PI Ps = Thus we write s (27T)- 5e- 5X 2 = ~ Pi[-f(-x-i+6)] i=l h(x) t(x) where hex) is the residual density on -3.5 ~ x ~ 3.5, and t(x) is the tail, i.e., the density of X, conditioned by JxJ > 3.5. We generate X with density h by the rejection technique (Fig. 2), and from the tail by the method described in Ref. 4. These are steps 9 and loin the following outline: To generate a standard normal variate X, density (27T) -. 5e- 5X2, 1. with probability.2767, put X + V s '- 1).7(Vl + V 2 2. with probability.2767, put X + Vs '- 2).7(V] + V'!. 3. with probability.1576, put X + V 3-0).7(Vl + V:!. 4. with probability.1576, put X + Vs - 3).7(V1 + V 2 5. with probability.0517, put X.7(V1 + V'!. + V3-4) 6. with probability.0517, put X + Vs + 1).7(U1 + V 2 7. with probability.0092, put X.7(V1 + V'!. + Vs -5) 8. with probability.0092, put X.7 (VI + V 2 + Vs + 2) 9. with probability , generate (x,y) 10. uniformly from the rectangle of Fig. 2 until y < g(x), then put X = x. with probability , generate pairs x = 2V 1,- 1, Y = V 2 until y < 3.5( n JXJ- 5 then put { ( InJx\) 5 X - ) - t -( In\xi) 5 A CHI-SQUARE VARIATE if x < 0 if x > 0 We will develop a procedure for generating a x~ variate X, density x S e- 5X /96, x > 0, along the same lines as the two examples above. We choose the interval.4 ~ x ~ 20.4 for fitting our mixture, dividing it into 10 parts. We will generate X in the form 2(M + VI + V'!. + V 3 ), where M takes values.2, 1.2, 2.2,...,7.2 with probabilities PI, P2,.., Ps. The best choice of the p's comes from solving this problem: Choose Pb P2,'.., Ps so as to maximize PI + P Ps PI =PS=0092, P2=P7= 0517, P3=P6=.1576, P4 = Ps =.2767 To generate a variate X with density g(x)/ , generate pairs x=7~1-.5), y=.0038~2 until y<g(x~e~t X=x_. Eff' IClency =.02674' or 34% o Figure 2. Method for generating a variate from the residual portion of the normal distribution. 3.5

4 172 PROCEEDINGS-FALL JOINT COMPUTER CONFERENCE, 1966 a g(x) =x 3 e -.Sx - }; Pi [.5f (.5x-i +.8) ] 96 i= r------:::~-...,.0006 Pl= 1608, P2=.2313, P3= 2128, p",=.1571, Ps= 1013, P6=.0599, P7=.0318, Pa=.0182 To generate a variate X with density g (xl / , choose (x,yl uniformly from the U-shaped region until y < g(x), then put X = x. The efficiency is 46%. Generate (x,y) uniformly over the region bounded by heavy lines by putting I (3Ul,.00505U2) with probability (x,y) = ( UJ,.0006U 2) with probability ( U 1,.00505U2) with probability o Figure 3. Method for generating a variate from the residual portion of the chi-square-8 distribution. subject to the condition that Pi > 0 and P1[.5f(.5x-.2)] + p2[.5f(.5x-1.2) Ps[.5f(.5x-7.2)] ::;; x 3 e- 5X /96 for 0 < x < 2004 The solution PI, P2,'.., Ps of this problem is given in the outline below and in Fig. 3. The sum of the p's is We generate X from the residual density by the rejection technique as described in Fig. 3. We generate X from the tail, i.e., conditioned by IXI > 2004, by transforming the tail to the unit interval and using the rejection technique (see Fig. 4). All of the steps combine to form this outline: To generate a chi-square-8 variate X, density x 3 e- 5X /96, x > 0, 1. with probability P2 =.2313 put X = 2( VI + V 2 + V3 ) 1.0~ B 2. with probability P3 =.2128 put X = 2( with probability PI =.1608 put X = 2 (.2 + VI + V 2 + V3 ) 4. with probability P4 =.1571 put X = 2( with probability P5 =.1013 put X = 2( with probability Pa =.0599 put X = 2 (5.2 + VI + V 2 + V3 ) 7. with probability P7 =.0318 put X = 2 ( with probability Ps =.0182 put X = 2( with probability generate X from the residual density drawn in Fig. 3, by the rejection technique. To generate a X~ variate X, conditioned by X > 20.4, choose (x,y) uniformly from the quadrilateral OABC until y < x- 5 e /x, then put X=20.4/x. The efficiency is 70%. Generate {x,y} uniformly from OABC by putting _{{.7-.7M + m,.1-lm} with probability 1/4 (x,y)- (.7+.3M,.1 +.9M-m) with probability 3/4, 0.1 o 0.7 C Figure 4. Method for generating a variate from the tail of the chi-square-8 distribution.

5 A GENERAL METHOD FOR PRODUCING RANDOM VARIABLES with probability generate X from the tail of the X2 distribution, by the rejection s technique described in Fig. 4. GENERAL REMARKS The examples above suggest the following general procedure for dealing with a density q (x). The only requirement is that q be close to the x-axis at its extremities. An interval containing most of the density is chosen, say a < x < b, then divided into n equal parts; In is usually a good choice. If h = (b-a)/10, then this linear programming-type problem is solved: choose P1'...,Ps so as to maximize P1 + P Ps, subject to the condition that Pi. > 0 and p,j ( x;) + P2f (X-:-h) P,I( X-:7h) ~ hq(x) Then X may be generated by putting X = a + hem + U 1 + U 2 + U 3 ), where M takes values 0,1,2,..., 7 with probabilities Pl..., Ps, or by choosing X from the residual density by the rejection technique, or from the tail, in a manner suggested by the above examples. The sum of the p's will usually be quite close to I-it was.9915,.9904, and.9732 in the three examples, and thus the resulting programs will be very fast. Few constants are needed, and the programs should be easy to code; they vary little from one density to the next, only the constants and some details from the residues or tails changing. In fact, the fast parts of the programs-generating c(m + U 1 + U 2 + U 3 ), are so consistent from one density to the next that a basic program for this part of the outline can be written in machine language, with the constants inserted for the particular density under consideration. The slow parts of the program-the residual density and tail can be handled by FOR TRAN, or some such convenient language, subroutines. The procedure for a normal variate outlined above is almost as fast as the super program in Ref. 3, yet it is much simpler and requires very little computer space. REFERENCES 1. M. D. MacLaren, G.eMarsaglia and T. A. Bray, "A Fast Procedure for Generating Exponential Variables," Communications of the Association for Computing Machinery, vol. 7, no. 5 (1964). 2. G. Marsaglia, "Expressing a Random Variable in Terms of Uniform Random Variables," Annals of Mathematical Statistics, vol. 32, pp (1961) , M. D. MacLaren and T. A. Bray, "A Fast Procedure for Generating Normal Variables," Communications of the Association for Computing Machinery, vol. 7, no. 1 (1964) , "Generating a Variable from the Tail of the Normal Distribution," Technometrics, vol. 6, no. 1, pp (1964).

General Principles in Random Variates Generation

General Principles in Random Variates Generation E. Moulines and G. Fort Telecom ParisTech June 2015 Bibliography : Luc Devroye, Non-Uniform Random Variate Generator, Springer-Verlag (1986) available on