Slides 3: Random Numbers

Transcription

1 Slides 3: Random Numbers We previously considered a few examples of simulating real processes. In order to mimic real randomness of events such as arrival times we considered the use of random numbers that indicated what value the events will take. Today we will examine more closely how such numbers can be obtained and what they really represent. 1

2 Random Number Generator A random number generator (RNG) is a computational or physical device designed to generate a sequence of numbers that appear random. By random, it means that they do not exhibit any discernible pattern, no matter how much effort we put into finding one. In other words, successively generated numbers can not be predicted. 2

3 Historical Generation Historical methods of generating random numbers include: Dice throwing. Coin Flipping. Shuffling of playing cards. But these are mechanical and generating large amounts of sufficiently random numbers would require a large amount of work and/or time. Alternatively results can be collected and passed out as random number tables. 3

4 Modern Generation With the advent of computers there are several techniques to quickly generate random numbers. However, computers are deterministic machines, and can not act in a random manner. Yet they can generate numbers in such a complex manner that, to all intents and purposes, the successive numbers are unpredictable with no discernible pattern. 4

5 Standard Random Numbers Two important statistical properties: Uniformity. Independence. Random number R i must be independently drawn from a uniform distribution with pdf: f(x) = 1 if 0 x 1 0 otherwise. 5

6 Generation of Pseudo-Random Numbers Pseudo because generating numbers using a known method removes the potential for true randomness. Goal: To produce a sequence of numbers in [0,1] that simulates, or imitates, the ideal properties of random numbers. Important considerations in random number routines: Fast. Portable to different computers. Have sufficiently long cycle. Replicable. Closely approximate the ideal statistical properties of uniformity and independence. 6

7 Techniques for Generating Random Numbers Linear Congruential Method (LCM). Combined Linear Congruential Generators (CLGM). Random-Number Streams. 7

8 Linear Congruential Method To produce a sequence of integers X 1,X 2,... between 0 and m 1 by following a recursive relationship: X i+1 = (ax i +c) mod m,i = 0,1,2,... m is the modulus, 0 < a < m is the multiplier, and 0 c < m is the increment. The selection of a,c,m and X 0 drastically affects the statistical properties of the cycle length. The random integers are being generated in [0,m 1], so to convert the integers to random numbers: R i = X i /m,i = 1,2,... If c = 0 also known as a multiplicative congruential generator. 8

9 Example LCM Use X 0 = 27, a = 17, c = 43 and m = 100. The X i and R i values are: X 1 = ( ) mod 100 = 502 mod 100 = 2 so R 1 = X 2 = ( ) mod 100 = 77 so R 2 = X 3 = ( ) mod 100 = 1352 mod 100 = 52 so R 3 = and so on... 9

10 Characteristics of a Good Generator Maximum Density: Such that the values assumed by R i, i = 1,2,... have no large gaps on [0,1]. Problem: Instead of continuous, each R i is discrete. Solution: a very large integer for modulus m (approximation appears to be of little consequence). Maximum Period: To achieve maximum density and avoid cycling. Achieve by: proper choice of a,c,m and X 0, though maximum is m. Most digital computers use a binary representation of numbers: Speed and efficiency are aided by a modulus m to be (or close to) a power of 2. 10

11 Combined Linear Congruential Generators Reason: Longer period generator is needed because of the increasing complexity of simulated systems. Approach: Combine two or more multiplicative congruential generators. Let X i,1,x i,2,...,x i,k be the i-th output from k different multiplicative congruential generators: The j-th generator should: Have a prime modulus m j and a multiplier a j that ensures the period is m j 1. Produces integers X i,j which are approximately Uniform on integers in [1,m 1]. 11

12 Combined Linear Congruential Generators Suggested form: X i = ( k ) ( 1) j 1 X i,j mod (m 1 1) j=1 The returned value is: R i = X i /m 1 if X i > 0 (m 1 1)/m 1 if X i = 0. The maximum possible period is: p = (m 1 1)(m 2 1) (m k 1) 2 k 1 12

13 Combined Linear Congruential Generators Example: For 32-bit computers, one suggestion is to combine k = 2 generators with m 1 = 2,147,483,563, a 1 = 40,014, m 2 = 2,147,483,399 and a 2 = 40,692. The algorithm becomes: Step 1: Select seeds: X 1,0 in the range [1, ] for the first generator. X 2,0 in the range [1, ] for the second generator. Step 2: For each individual generator: X 1,j+1 = 40014X 1,j mod X 2,j+1 = 40692X 2,j mod Step 3: X j+1 = (X 1,j+1 X 2,j+1 ) mod

14 Combined Linear Congruential Generators Cont. Step 4: Return: R j+1 = X j+1 / if X j+1 > / if X j+1 = 0. Step 5: Set j = j +1 and return to step 2. Combined generator has period (m 1 1)(m 2 1)/

15 Random-Number Streams The seed for a linear congruential random-number generator: Is the integer value X 0 that initializes the random-number sequence. Any value in the sequence can be used to seed the generator. A random-number stream: Refers to a starting seed taken from the sequence X 0,X 1,...,X p. If the streams are b values apart, then stream i could be defined by starting at seed S i = X b(i 1). Older generators had b = 10 5, newer generators have b = A single random-number generator with k streams can act like k distinct virtual random-number generators. To compare two or more alternative systems: Advantageous to dedicate portions of the pseudo-random number sequence to the same purpose in each of the simulated systems. 15

16 Two categories: Tests for Random Numbers Testing for uniformity: H 0 : R i U[0,1] H 1 : R i U[0,1] Failure to reject the null hypothesis H 0 means that evidence of non-uniformity has not been detected. Testing for independence: H 0 : R i independently H 1 : R i independently Failure to reject the null hypothesis H 0 means that evidence of dependence has not been detected. Level of significance α, the probability of rejecting H 0 when it is true: α = P(reject H 0 H 0 true) 16

17 Tests for Random Numbers When to use these tests: If a well-known random-number generator is used, it is probably unnecessary to test. If the generator is not explicitly known or documented, e.g., spreadsheet programs, symbolic/numerical calculators, tests should be applied to many sample numbers. Types of tests: Theoretical tests: evaluate the choices of m, a and c without actually generating any numbers. Empirical tests: applied to actual sequences of numbers produced (what we will consider). 17

18 Frequency Tests Test of uniformity. Two different methods: Kolmogorov-Smirnov test. Chi-square test. 18

19 Kolmogorov-Smirnov Test Compares the continuous cdf, F(x), of the uniform distribution with the empirical cdf, S N (x), of the N sample observations: We know F(x) = x for 0 x 1. If the sample from the random number generator is R 1,R 2,...,R N, then the empirical cdf, S N (x) is: S N (x) = # of R 1,R 2,...,R N which are x N Based on the statistic: D = max F(x) S N (x). Sampling distribution of D is known (a function of N and available in tables). It is a more powerful test, hence recommended. 19

20 Kolmogorov-Smirnov Test Example: Suppose 5 generated numbers are 0.44, 0.81, 0.14, 0.05,

21 Kolmogorov-Smirnov Test Step 1: Arrange R (i) from smallest to largest. Step 2: Calculate i/n. Step 3: Calculate i/n R (i) Step 4: Calculate R (i) (i 1)/N 21

22 Kolmogorov-Smirnov Test Step 5: Set D + = max{i/n R (i) }. Step 6: Set D = max{r (i) (i 1)/N}. Step 4: Set D = max{d +,D } = For α = 0.05, D α = > D, hence H 0 not rejected. 22

23 Chi-square Test Chi-square test uses the sample statistic: χ 2 0 = n i=1 (O i E i ) 2 E i Here n is the number of classes, E i is the expected number in the i-th class, and O i is the observed number in the i-th class. Approximately the chi-square distribution with n 1 degrees of freedom (where critical values are available in tables). For the uniform distribution E i, the expected number in each class, is N/n, where N is the total number of observations. Only valid for large samples, e.g., N

24 Tests for Autocorrelation Testing the autocorrelation between every m numbers (m is a.k.a. the lag), starting with the i-th number: The autocorrelation ρ i,m between numbers: R i,r i+m,r i+2m,...,r i+(m+1)m. Here M is the largest integer such that i+(m +1)m N. Hypothesis: H 0 : ρ i,m = 0, if numbers are independent If the values are uncorrelated: H 1 : ρ i,m 0, if numbers are dependent For large values of M, the distribution of the estimator ρ i,m, denoted ˆρ i,m is approximately normal. 24

25 Tests for Autocorrelation Test statistic is Z 0 = ˆρ i,m /ˆσˆρi,m. Z 0 is distributed normally with mean 0 and variance 1 and: ˆρ i,m = 1 ( M R i+km R i+(k+1)m ) 0.25 M +1 k=0 ˆσˆρi,m = 13M +7 12(M +1) If ρ i,m > 0, the subsequence has positive autocorrelation (high random numbers tend to be followed by high ones, and vice versa). If ρ i,m < 0, the subsequence has negative autocorrelation (high random numbers tend to be followed by low ones, and vice versa). 25

26 Shortcomings The test is not very sensitive for small values of M, particularly when the numbers being tested are on the low side. Problem when fishing for autocorrelation by performing numerous tests: If α = 0.05, there is a probability of 0.05 of rejecting a true hypothesis. If 10 independent sequences are examined: The probability of finding no significant autocorrelation, by chance alone, is Hence, the probability of detecting significant autocorrelation when it does not exist is 40%. 26

27 Summary We have examined generation of random numbers and tests for uniformity and independence. Caution: Even generators that have been used for years, some of which are still in use, can often be found to be inadequate in practice. We have only covered very basic generators and tests. Also, even if generated numbers pass all the tests, some underlying pattern may still have gone undetected. 27