Chapter 6. Random-Number Generation 6.1. Prof. Dr. Mesut Güneş Ch. 6 Random-Number Generation

Transcription

1 Chapter 6 Random-Number Generaton 6.1

2 Contents Propertes of Random Numbers Pseudo-Random Numbers Generatng Random Numbers Lnear Congruental Method Combned Lnear Congruental Method Tests for Random Numbers Real Random Numbers 6.2

3 Overvew Dscuss characterstcs and the generaton of random numbers. Subsequently, ntroduce tests for randomness: Frequency test Autocorrelaton test 6.3

4 Overvew Hstorcally Throw dces Deal out cards Draw numbered balls Use dgts of π Mechancal devces (spnnng dsc, etc.) Electrc crcuts Electronc Random Number Indcator (ERNIE) Countng gamma rays In combnaton wth a computer Hook up an electronc devce to the computer Read-n a table of random numbers 6.4

5 Pseudo-Random Numbers 6.5

6 Pseudo-Random Numbers Approach: Arthmetcally generaton (calculaton) of random numbers Pseudo, because generatng numbers usng a known method removes the potental for true randomness. Any one who consders arthmetcal methods of producng random dgts s, of course, n a state of sn. For, as has been ponted out several tmes, there s no such thng as a random number there are only methods to produce random numbers, and a strct arthmetc procedure of course s not such a method. John von Neumann,

7 Pseudo-Random Numbers probably can not be justfed, but should merely be judged by ther results. Some statstcal study of the dgts generated by a gven recpe should be made, but exhaustve tests are mpractcal. If the dgts work well on one problem, they seem usually to be successful wth others of the same type. John von Neumann, 1951 Goal: To produce a sequence of numbers n [0,1] that smulates, or mtates, the deal propertes of random numbers (RN). 6.7

8 Pseudo-Random Numbers Important propertes of good random number routnes: Fast Portable to dfferent computers Have suffcently long cycle Replcable Verfcaton and debuggng Use dentcal stream of random numbers for dfferent systems Closely approxmate the deal statstcal propertes of unformty and ndependence 6.8

9 Pseudo-Random Numbers: Propertes Two mportant statstcal propertes: Unformty Independence Random number R must be ndependently drawn from a unform dstrbuton wth PDF: f ( x) = 1, 0, 0 x 1 otherwse f(x) E( R) = 1 0 xdx = x = 1 2 x 0 1 PDF for random numbers 6.9

10 Pseudo-Random Numbers Problems when generatng pseudo-random numbers The generated numbers mght not be unformly dstrbuted The generated numbers mght be dscrete-valued nstead of contnuous-valued The mean of the generated numbers mght be too hgh or too low The varance of the generated numbers mght be too hgh or too low There mght be dependence: Autocorrelaton between numbers Numbers successvely hgher or lower than adjacent numbers Several numbers above the mean followed by several numbers below the mean 6.10

11 Generatng Random Numbers 6.11

12 Generatng Random Numbers Mdsquare method Lnear Congruental Method (LCM) Combned Lnear Congruental Generators (CLCG) Random-Number Streams 6.12

13 Generatng Random Numbers Mdsquare method 6.13

14 Mdsquare method Frst arthmetc generator: Mdsquare method von Neumann and Metropols n 1940s The Mdsquare method: Start wth a four-dgt postve nteger Z 0 2 Compute: Z = Z0 dgts 0 Z0 to obtan an nteger wth up to eght Take the mddle four dgts for the next four-dgt number Z U Z Z

15 Mdsquare method Problem: Generated numbers tend to 0 Z U Z Z

16 random numbers should not be generated wth a method chosen at random. Some theory should be used. Donald E. Knuth, The Art of Computer Programmng, Vol

17 Generatng Random Numbers Lnear Congruental Method 6.17

18 Lnear Congruental Method To produce a sequence of ntegers 1, 2, between 0 and m-1 by followng a recursve relatonshp: = ( a + c) mod m, = + 1 0,1,2,... The multpler The ncrement The modulus Assumpton: m > 0 and a < m, c < m, 0 < m The selecton of the values for a, c, m, and 0 drastcally affects the statstcal propertes and the cycle length The random ntegers are beng generated n [0, m-1] 6.18

19 Lnear Congruental Method Convert the ntegers to random numbers R =, =1,2,... m Note: {0, 1,..., m-1} R [0, (m-1)/m] 6.19

20 Lnear Congruental Method: Example Use 0 = 27, a = 17, c = 43, and m = 100. The and R values are: 1 = ( ) mod 100 = 502 mod 100 = 2 Æ R 1 = = ( ) mod 100 = 77 Æ R 2 = = ( ) mod 100 = 52 Æ R 3 = = ( ) mod 100 = 27 Æ R 3 =

21 Lnear Congruental Method: Example Use a = 13, c = 0, and m = 64 The perod of the generator s very low Seed 0 nfluences the sequence 0 =1 0 =2 0 =3 0 =

22 Lnear Congruental Method: Characterstcs of a good Generator Maxmum Densty The values assumed by R, =1,2, leave no large gaps on [0,1] Problem: Instead of contnuous, each R s dscrete Soluton: a very large nteger for modulus m Approxmaton appears to be of lttle consequence Maxmum Perod To acheve maxmum densty and avod cyclng Acheved by proper choce of a, c, m, and 0 Most dgtal computers use a bnary representaton of numbers Speed and effcency are aded by a modulus, m, to be (or close to) a power of

23 Lnear Congruental Method: Characterstcs of a good Generator The LCG has full perod f and only f the followng three condtons hold (Hull and Dobell, 1962): 1. The only postve nteger that (exactly) dvdes both m and c s 1 2. If q s a prme number that dvdes m, then q dvdes a-1 3. If 4 dvdes m, then 4 dvdes a

24 Lnear Congruental Method: Proper choce of parameters For m a power 2, m=2 b, and c 0 Longest possble perod P=m=2 b s acheved f c s relatve prme to m and a=1+4k, where k s an nteger For m a power 2, m=2 b, and c=0 Longest possble perod P=m/4=2 b-2 s acheved f the seed 0 s odd and a=3+8k or a=5+8k, for k=0,1,... For m a prme and c=0 Longest possble perod P=m-1 s acheved f the multpler a has property that smallest nteger k such that a k -1 s dvsble by m s k = m

25 Characterstcs of a Good Generator 6.25

26 Characterstcs of a Good Generator 6.26

27 Random-Numbers n Java Defned n java.utl.random prvate fnal statc long multpler = 0x5DEECE66DL; // prvate fnal statc long addend = 0xBL; // 11 prvate fnal statc long mask = (1L << 48) - 1; // = protected nt next(nt bts) { long oldseed, nextseed;... oldseed = seed.get(); nextseed = (oldseed * multpler + addend) & mask;... return (nt)(nextseed >>> (48 - bts)); // >>> Unsgned rght shft } 6.27

28 General Congruental Generators Lnear Congruental Generators are a specal case of generators defned by: 1 g(, 1, ) mod + = where g() s a functon of prevous s [0, m-1], R = /m m Quadratc congruental generator Defned by: g (, ) 2 1 a + b Multple recursve generators Defned by: g(, 1, ) a1 + Fbonacc generator Defned by: g = 1 + = a (, ) 1 = + 1 c a k k 6.28

29 Combned Lnear Congruental Generators Reason: Longer perod generator s needed because of the ncreasng complexty of smulated systems. Approach: Combne two or more multplcatve congruental generators. Let,1,,2,,,k be the -th output from k dfferent multplcatve congruental generators. The j-th generator,j : + 1, j = ( a j + c j ) mod m j has prme modulus m j, multpler a j, and perod m j -1 produces ntegers,j approx ~ Unform on [0, m j 1] W,j =,j - 1 s approx ~ Unform on ntegers on [0, m j - 2] 6.29

30 6.30 Combned Lnear Congruental Generators Suggested form: The maxmum possble perod s: ) 1)...( 1)( ( = k m k m m P 1 mod 1) ( 1 1, 1 = = m k j j j = > = 0, 1 0, Hence, m m m R

31 Combned Lnear Congruental Generators Example: For 32-bt computers, combnng k = 2 generators wth m 1 = , a 1 = 40014, m 2 = and a 2 = The algorthm becomes: Step 1: Select seeds 0,1 n the range [1, ] for the 1 st generator 0,2 n the range [1, ] for the 2 nd generator Step 2: For each ndvdual generator, +1,1 = 40014,1 mod ,2 = 40692,2 mod Step 3: +1 = ( +1,1 - +1,2 ) mod Step 4: Return R , = , > 0 = 0 Step 5: Set = +1, go back to step 2. Combned generator has perod: (m 1 1)(m 2 1)/2 ~ 2 x

32 Random-Numbers n Excel 2003 In Excel 2003 and 2007 new Random Number Generator, Y, Z {1,...,30000} = 171mod Y = Y 172 mod Z = Z 170 mod R = Y Z mod 1.0 It s stated that ths method produces more than numbers For more nfo:

33 Random-Numbers Streams The seed for a lnear congruental random-number generator: Is the nteger value 0 that ntalzes the random-number sequence Any value n the sequence ( 0, 1,, P ) can be used to seed the generator A random-number stream: Refers to a startng seed taken from the sequence ( 0, 1,, P ). If the streams are b values apart, then stream s defned by startng seed: Older generators: b = 10 5 Newer generators: b = S, = b( 1) = 1,2, A sngle random-number generator wth k streams can act lke k dstnct vrtual random-number generators To compare two or more alternatve systems. Advantageous to dedcate portons of the pseudo-random number sequence to the same purpose n each of the smulated systems. P b 6.33

34 Mersenne Twster Generator Mersenne Twster RNG (MT) by M. Matsumoto and T. Nshmura, 1998 MT propertes perod of dmensonal equdstrbuton 6.34

35 Random Numbers n OMNeT++ OMNeT++ releases pror to 3.0 used a lnear congruental generator (LCG) wth a cycle length of By default, OMNeT++ uses the Mersenne Twster RNG (MT) by M. Matsumoto and T. Nshmura. Ths RNG can be selected from omnetpp.n OMNeT++ allows pluggng n your own RNGs as well. Ths mechansm, based on the crng nterface. 6.35

36 Tests for Random Numbers 6.36

37 Tests for Random Numbers Two categores: Testng for unformty: H 0 : R ~ U[0,1] H 1 : R U[0,1] Falure to reject the null hypothess, H 0, means that evdence of nonunformty has not been detected. Testng for ndependence: H 0 : R ~ ndependent H 1 : R ndependent Falure to reject the null hypothess, H 0, means that evdence of dependence has not been detected. Level of sgnfcance α, the probablty of rejectng H 0 when t s true: α = P(reject H 0 H 0 s true) 6.37

38 Tests for Random Numbers When to use these tests: If a well-known smulaton language or random-number generator s used, t s probably unnecessary to test If the generator s not explctly known or documented, e.g., spreadsheet programs, symbolc/numercal calculators, tests should be appled to many sample numbers. Types of tests: Theoretcal tests: evaluate the choces of m, a, and c wthout actually generatng any numbers Emprcal tests: appled to actual sequences of numbers produced. Our emphass. 6.38

39 Tests for Random Numbers Frequency tests: Kolmogorov-Smrnov Test 6.39

40 Kolmogorov-Smrnov Test Compares the contnuous CDF, F(x), of the unform dstrbuton wth the emprcal CDF, S N (x), of the N sample observatons. We know: F( x) = x, 0 x 1 F(x) If the sample from the RNG s R 1, R 2,, R N, then the emprcal CDF, S N (x) s: 0 1 x S N ( x) = Number of R where N R x Based on the statstc: D = max F(x) - S N (x) Samplng dstrbuton of D s known 6.40

41 Kolmogorov-Smrnov Test The test conssts of the followng steps Step 1: Rank the data from smallest to largest R (1) R (2)... R (N) Kolmogorov-Smrnov Crtcal Values Step 2: Compute + D = max R( ) 1 N N 1 D = max R( ) 1 N N Step 3: Compute D = max(d +, D - ) Step 4: Get D α for the sgnfcance level α Step 5: If D D α accept, otherwse reject H

42 Kolmogorov-Smrnov Test Example: Suppose N=5 numbers: 0.44, 0.81, 0.14, 0.05, Step 1: Step 2: R () /N /N R () R () (-1)/N Arrange R () from smallest to largest D + = max{/n R () } D - = max{r () - (-1)/N} Step 3: D = max(d +, D - ) = 0.26 Step 4: For α = 0.05, D α = > D = 0.26 Hence, H 0 s not rejected. 6.42

43 Tests for Random Numbers Frequency tests: Ch-square Test 6.43

44 Ch-square Test Ch-square test uses the sample statstc: n s the # of classes O s the observed # n the -th class E s the expected # n the -th class χ 2 0 = n = 1 ( O E E ) 2 Approxmately the ch-square dstrbuton wth n-1 degrees of freedom For the unform dstrbuton, E, the expected number n each class s: N E =, where N s the total number of observatons n Vald only for large samples, e.g., N

45 Ch-square Test: Example Example wth 100 numbers from [0,1], α= ntervals χ ,9 = 16.9 Accept, snce 2 0 =11.2 < χ ,9 Interval Upper Lmt O E O -E (O -E )^2 (O -E )^2/E Sum χ 2 0 = n = 1 ( O E ) E =

46 Tests for Random Numbers Tests for autocorrelaton 6.46

47 Tests for Autocorrelaton Autocorrelaton s concerned wth dependence between numbers n a sequence Example: Numbers at 5-th, 10-th, 15-th,... are very smlar Numbers can be Low Hgh Alternatng 6.47

48 Tests for Autocorrelaton Testng the autocorrelaton between every m numbers (m s a.k.a. the lag), startng wth the -th number The autocorrelaton ρ,m between numbers: R, R +m, R +2m, R +(M+1)m M s the largest nteger such that + ( M + 1) m N Hypothess: H H 0 1 : : ρ ρ, m, m = 0, 0, f f numbers are ndependent numbers are dependent If the values are uncorrelated: For large values of M, the dstrbuton of the estmator of ρ,m, denoted s approxmately normal. ˆρ,m 6.48

49 6.49 Tests for Autocorrelaton Correlaton at lag j Assume = U ) ( ) ( ) ( ) ( ) ( )] ( [ ) ( ) ( ) ( ) ( ), ( ) ( ) ( ) ( ), ( j j j j j j j j j Var E E E Var E E E E E Cov C E E E Cov C C C = = = = = = = = ρ ρ 3 ) ( ) ( 12 1 ) ( and 2 1 ) ( = = = = + + j j j U U E U U E U Var U E ρ

50 Tests for Autocorrelaton Test statstcs s: Z 0 = ˆ ρ ˆ σ ˆ ρ, m, m Z 0 s dstrbuted normally wth mean = 0 and varance = 1, and: ρˆ σˆ, m ρ, m = 1 1 M R M + k = 0 = 13M ( M + 1) + km R + (k + 1 )m After computng Z 0 do not reject the hypothess of ndependence f z α/2 Z 0 z α/2 6.50

51 Tests for Autocorrelaton If ρ,m > 0, the subsequence has postve autocorrelaton Hgh random numbers tend to be followed by hgh ones, and vce versa. If ρ,m < 0, the subsequence has negatve autocorrelaton Low random numbers tend to be followed by hgh ones, and vce versa. 6.51

52 Example Test whether the 3 rd, 8 th, 13 th, and so on, for the numbers on Slde 46. Hence, α = 0.05, = 3, m = 5, N = 30, and M = 4 ρˆ σ 35 ρˆ Z (0.23)(0.28) + (0.28)(0.33) + (0.33)(0.27) = (0.27)(0.05) (0.05)(0.36) = = = 13(4) ( 4 + 1) = = z = 1.96 Snce Z 0 = , the hypothess s not rejected. 6.52

53 Shortcomngs The test s not very senstve for small values of M, partcularly when the numbers beng tested are on the low sde. Problem when fshng for autocorrelaton by performng numerous tests: If α = 0.05, there s a probablty of 0.05 of rejectng a true hypothess. If 10 ndependence sequences are examned: The probablty of fndng no sgnfcant autocorrelaton, by chance alone, s = Hence, the probablty of detectng sgnfcant autocorrelaton when t does not exst = 40% 6.53

54 Real Random Numbers 6.54

55 Real Random Numbers There are also sources for real random numbers n the Internet RANDOM.ORG offers true random numbers to anyone on the Internet. The randomness comes from atmospherc nose, whch for many purposes s better than the pseudo-random number algorthms typcally used n computer programs. People use the numbers to run lotteres, draws and sweepstakes and for ther games and gamblng stes

56 Real Random Numbers It offers the possblty to download true random numbers generated usng a quantum random number generator upon demand. 6.56

57 Real Random Numbers Hardware based generaton of random numbers

58 Summary In ths chapter, we descrbed: Generaton of random numbers Testng for unformty and ndependence Sources of real random numbers Cauton: Even wth generators that have been used for years, some of whch stll n use, are found to be nadequate. Ths chapter provdes only the bascs. Also, even f generated numbers pass all the tests, some underlyng pattern mght have gone undetected. 6.58