IE 303 Discrete-Event Simulation L E C T U R E 6 : R A N D O M N U M B E R G E N E R A T I O N

Transcription

1 IE 303 Discrete-Event Simulation L E C T U R E 6 : R A N D O M N U M B E R G E N E R A T I O N

2 Review of the Last Lecture Continuous Distributions Uniform distributions Exponential distributions and memoryless property Gamma Distributions Erlang Distributions Weibull Distribution Normal Distribution Triangular Distribution Lognormal Distribution Truncated Normal Distribution Convolution Maximum Likelihood Estimation

3 Outline of Lecture 6 Random Number Generation in Simulation Definition of Pseudo Random Numbers Linear Congruent Method Random Number Streams Test for Random Numbers Tests for Uniformity Kolmogorov, Smirnov Tests Chi-square Tests Test for Statistical Independence from History

4 Outline In this lecture we will learn how to generate random numbers and statistical tests for measuring randomness. Why do we learn this?? Suppose you start your IE career at a moderately large manufacturing company and you are faced with a design of parallel assembly lines problem. However your company cannot afford buying ARENA or Flexim (commercial license costs per PC, 32000$ for 2 PC). Your boss suggest you two options: a) You might be allowed for 2 or 3 weeks to build your own program (in Java or C++) which solves the problem using discrete event simulation. b) He is ok with buying a cheaper simulation software package which has never been used by someone else, i.e. unknown reliability of results! What to do??

5 Empirical Distribution An empirical distribution, either discrete or continuous, is a distribution whose parameters are the observed values in a sample of data. This is in contrast to parametric distribution families such as the Exponential(λ), Normal(μ,σ) or Poisson(λ), which are characterized by specifying a small number of parameters such as the mean, the variance, the rate of activity, etc. An empirical distribution may be used when it is impossible or unnecessary to establish that a random variable has any particular parametric distribution. 5 One advantage of an empirical distribution is that nothing is assumed beyond the observed values in the sample; however, this is also a disadvantage because the sample might not cover the entire range of possible values. 03-Nov-16

6 Empirical Distribution 6 Example: Customers at a local restaurant arrive at lunchtime in groups of from one to eight persons. The number of persons per party in the last 300 groups has been observed; the results, the relative frequencies are summarized in the tables below. An histogram and the empirical cdf are depicted as well. 03-Nov-16

7 Empirical Distribution 7 03-Nov-16

8 Random Number Generation Random numbers are a necessary basic ingredient in the simulation of all discrete systems. Most computer languages have a subroutine or function that will generate a random number. Similarly, simulation languages generate random numbers that are used to generate event times and other random variables. Any random number generation should satisfy two properties of random numbers: Uniformity independence

9 Random Number Generation Random numbers should come from Uniform(0,1) and each random number should be independent from its history? But why????

10 Harry's Winnings Random Number Generation Lets revisit our coin tossing game between Harry and Tom. Charlie, the referee of the game, successively tosses a fair coin. Heads: Tom -> Harry $1 Tails: Tom <- Harry $1 Coin tosses should be fair (the same probability of having heads or tails) uniformity. New games should be independent of the previous results statistical independence

11 Pseudo Random Numbers Random numbers generated by computers are called pseudo random numbers. Pseudo means false, so false random numbers are being generated! In this instance, pseudo is used to imply that the very act of generating random numbers by a known method removes the potential for true randomness. If the method is known, the set of random numbers can be replicated. But then these numbers are not truly random. Whereas in computer simulation of discrete-event systems replicability is actually a desired feature.

12 Random Number Generation In the generation of pseudo-random numbers certain problems or errors can occur. These errors, or departures from ideal randomness, are all related to the two properties: Uniformity and independence. Some examples of which are: The generated numbers might not be uniformly distributed. The generated numbers might be discrete-valued instead of continuous-valued. The mean and/or the variance of the generated numbers might be too high or too low. There might be dependence. The following are examples: autocorrelation between numbers; numbers successively higher or lower than adjacent numbers; several numbers above the mean followed by several numbers below the mean. There are statistical tests to detect such departures from uniformity and independence. Once a generator (a method or a routine) is found to be defective, it will be dropped in favor of an acceptable generator.

13 Random Number Generation Desired Qualities of a Pseudo-Random Number Generator The routine should be fast. A simulation experiment could require many millions of random numbers. The generator needs to be computationally efficient. The routine should be portable to different programming languages. It is desirable that the simulation program will produce the same results wherever it is executed. The routine should have a sufficiently long cycle. The random numbers should be replicable. The generated random numbers should closely approximate the ideal statistical properties of uniformity and independence.

14 Linear Congruent Method The linear congruent method produces integers, X 1, X 2,... between 0 and m 1 by following a recursive relationship: Random number is The initial value X 0 is called the seed, a is called the multiplier, c is called the increment, and m is called the modulus.

15 Linear Congruent Method The selection of the values for a, c, m and X 0 drastically affects the statistical properties and the cycle length. Variations of this method are quite common in the computer generation of random numbers. Example Use X 0 = 27, a = 17, c = 43, and m = 100. Here, the integer values generated will all be between 0 and 99 because of the value of the modulus. Also, notice that random integers are being generated, are discrete rather than continuous. These random integers should appear to be uniformly distributed on the integers 0 to 99.

16 Linear Congruent Method Generated random numbers There is sth wrong a) Theoretically the numbers generated assume values only from the set {0, 1/m, 2/m,..., (m-1)/m} because each X i is an integer in the set {0, 1, 2,..., m-1}. Hence obtained random numbers follow a discrete distribution. b) These numbers has a period of 5! So what???

17 Linear Congruent Method Discreteness can be overcome if the modulus m is a very large integer. Values such as m=231 1 and m=248 are common use in generators appearing in many simulation languages. It is desirable the values assumed by R i, i = 1, 2,..., leave no large gaps on [0,1]. Also notice that in practical applications, the generator should have the largest possible period. Maximal period should be achieved by the proper choice of a, c, m, and X 0.

18 Linear Congruent Method Three choices: For m a power of 2, say m = 2b, and c 0, the longest possible period is P = m = 2b, which is achieved whenever c is relatively prime to m (that is the common factor of c and m is 1) and a = 1 + 4k, where k is an integer. For m a power of 2, say m = 2b, and c = 0, the longest possible period is P = m /4 = 2b 2, which is achieved if the seed X 0 is odd and if the multiplier, a, is given by a = 3 + 8k or a = 5 + 8k, for some k = 0, 1,... For m a prime number and c = 0, the longest possible period is P=m 1, which is achieved whenever the multiplier, a, has the property that the smallest integer k such that ak 1 is divisible by m is k=m 1.

19 Linear Congruent Method Example Find the period of the generator for a = 13, c=0, m=2 6 = 64, and X 0 = 1, 2, 3, and 4. The solution is given in the table on the right. When the seed is 1 or 3, the sequence has period 16. However, a period of length 8 is achieved when the seed is 2 and a period of length 4 occurs when the seed is 4. In this example the maximal period is given by the second rule given above, m = 2 4, and c = 0, the longest possible period is P = 64 /4 = 16. As mentioned before, the maximal period is achieved when the seed is odd and a has the form given by 5 + 8k. When the seed is 1, the density is 4/64= Note that modulus operation is very efficient when the modulo is a power of 2 which is another consideration when selecting a, c, and m.

20 Linear Congruent Method Example Different values have been extensively tested. The values for a, c, and m have been selected to ensure that the characteristics desired in a generator are most likely to be achieved. By changing X 0, the user can control the repeatibility of the stream. Let a = 7 5 = 16807, m = = 2,147,483,647 (a prime number), and c = 0. These choices satisfy the conditions that insure a period of P = m 1, (well over 2 billion). Further, specify the seed X0 = 123,457. The first few numbers generated are as follows:

21 Random Number Streams The seed for a linear congruential random number generator is the integer value X 0 that initializes the random-number sequence. Since the sequence of integers X 0, X 1, X 2,..., X P,produced by a generator repeats, any value in the sequence could be used to seed the generator. For a linear congruential generator, a random number stream is nothing more than a convenient way to refer to starting seed taken from the sequence X 0, X 1, X 2,..., X P.

22 Tests for Random Numbers Generated numbers should be tested if they satisfy the two property of random numbers: Uniformity Independence To this end, we use hypothesis tests. Uniformity is tested with Kolmogorov Smirnov Chi-square test Whereas independence is tested with Autocorrelation test

23 Tests for Random Numbers Hypothesis statements in these tests are Uniformity Independence The level of significance denoted by α satisfies: α= P{reject H 0 H 0 is true} which means that the critical value for the test statistic is selected such that the probability of rejecting the null hypothesis, H 0, equals α.

24 Tests for Random Numbers For a reliable decision on the acceptance of a random number generator, statistical tests must be done on a large number of samples, say M. If the number of rejected samples significantly exceeds the expected number of rejected tests, am, then we shall discard the given random number generator as the random numbers generated by it do not possess the property which we are testing for.

25 Kolmogorov Smirnov Test

26 Kolmogorov Smirnov Test

27 END OF LECTURE 6 Next Lecture: Statistical Tests (Chp. 208 (Computer Lab)

28 Kolmogorov Smirnov Test

29 Kolmogorov Smirnov Test

30 Chi-Square Test

31 Chi-Square Test

32 Random Number Streams

33