Hands-on Generating Random

Transcription

1 CVIP Laboratory Hands-on Generating Random Variables Shireen Elhabian Aly Farag October 2007

2 The heart of Monte Carlo simulation for statistical inference. Generate synthetic data to test our algorithms, such as data fitting and classification. Generating data encryption keys. Simulating and modeling complex phenomena. Selecting random samples from larger data sets.

3 We will learn algorithms and get them into action!!!

4 Agenda U 0,1 N 0,1 N μ, σ N μ, Σ

5 Random Truly Random Exhibiting true randomness Pseudorandom Appearance of randomness but having a specific repeatable pattern Quasi-random Having a set of non-random numbers in a randomized order

6 Generating U(0,1) Random Variables They are usually the building block for generating other random variables. We will look at: Properties that a random number generator should possess Linear Congruential Generators (LCGs) Use Matlab to generate U(0,1) variates.

7 Properties of a U(0,1) Generator Numbers should appear to be ~ U(0,1) and independent. Generator should be fast and not require too much storage. Should be able to reproduce a iven set of numbers for comparison purposes.

8 Linear Congruential Generators (LCGs) Numbers are generated according to: Z + = az +./ + c mod m where m, a, c and Z 7 are non-negative numbers. Z 7 is the seed. m is the modulus. Z + is a sequence of integer values ranging from 0 to m 1, starting from the seed point Z 7. To generate pseudo-random numbers, U /,, U ;, we set U + = Z + /m. Thus U + 0,1 i.

9 Example 1 Z 7 = 1, m = 16, a = 11, c = 0 Z + = 11 Z +./ mod 16 Now iterate to determine the Z + A s Z 7 = 1 Z / = 11 mod 16 = 11 Z C = 121 mod 16 = 9 Z F = 99 mod 16 = 3 Z H = 33 mod 16 = 1 What is wrong with this? The Z + s are not that random. They can only take on a finite number of values. The period of the generator can be very poor.

10 How to Guarantee a Full Period?!! Theorem The linear congruential generator has full period if and only if the following three conditions holds: 1. If 4 divides m, then 4 divides a 1 2. The only positive integer that exactly divides both m and c is 1, i.e. m and c are relatively prime, such that gcd m, c = 1 3. If q is a prime number that divides m, then it divides a 1.

11 Example 2 Z 7 = 1, m = 16, a = 13, c = 13 Z + = 13 Z +./ + 13 mod 16 Now iterate to determine the Z + A s Z 7 = 1 Z / = 26 mod 16 = 10 Z C = 143 mod 16 = 15 Z F = 248 mod 16 = 0 Z H = 13 mod 16 = 13 The linear congruential generator has full period if and only if the following three conditions holds: 1. If 4 divides m, then 4 divides a 1 2. The only positive integer that exactly divides both m and c is 1, i.e. m and c are relatively prime, such that gcd m, c = 1 3. If q is a prime number that divides m, then it divides a 1. Check to see that this LCG has full period: Are the conditions of the theorem satisfied? They can only take on a finite number of values. Does it matter what integer we use for Z 7?

12 Seeds In your experimentation, you can generate 100 streams each with a different seed point in order to: avoid duplicate streams of random numbers, get a general idea of the behavior of the random number generator on your workstation. Seeds can be obtained from: A. M. Law and W. D. Kelton, Simulation Modeling & Analysis, 2nd Edition, McGraw-Hill, NewYork, 1991.

13 In Matlab J Notes: The function rand with no arguments returns a single instance of the random variable U. To get an array mxn of uniform variates, you can use the syntax rand(m,n). If you use rand(n), then you get an nxn matrix.

14 In Matlab J The seed or the state of the generator is reset to the default when Matlab starts up, so the same sequencyes of random variables are generated whenever you start Matlab. If you call the function using rand('state',0), then MATLAB resets the generator to the initial state. If you want to specify another state, then use the syntax rand('state',j) to set the generator to the j-th state. You can obtain the current state using S = rand( state ), where S is a 35 element vector. To reset the state to this one, use rand( state,s).

15 Inverse Transform Method This method converts a known distribution with known parameters to another distribution with different parameters. Example: Generate Y~U( 1,1) from X~U 0,1. p(x) p(y) x y x ~ U(0,1) g(.) y ~ U( 1,1)

16 Inverse Transform Method p(x) p(y) F X x = Y x.\ p x dx x F Y (y) = Y y.\ p y dy y ì0 x 0 ï FX ( x) = íx 0< x< 1 ï î1 x ³ 1 x ì 0 y -1 ï y + 1 FY ( y) = í - 1< y< 1 ï 2 y ³ 1 ïî 1 y

17 Inverse Transform Method F X x = Y x.\ p x dx F Y (y) = Y y.\ p y dy x y ì0 x 0 ï FX ( x) = íx 0< x< 1 ï î1 x ³ 1 ì 0 y -1 ï y + 1 FY ( y) = í - 1< y< 1 ï 2 y ³ 1 ïî ( ) ( ) ( ) x( ) -1 ( ) F ( Y) = P Y y = P g( x) y = P x g ( y) = F g ( y) y F ( Y) = F g ( y) = x y x ì 0 y -1 ï y + 1 x= FY ( y) = í - 1< y< 1 ï 2 y ³ 1 ïî 1 ì 0 x 0-1 ï y= F Y ( x) = í- 1+ 2x 0< x< 1 ï î 1 x ³ 1

18 Box-Muller Approach Standard Normal U 0, 1 N 0, 1 If U / and U C are independent random variates from U(0,1) generated before. Then Z / = 2 ln U / cos 2πU C and Z C = 2 ln U / sin 2πU C are ~ N(0,1) and independent.

19 In Matlab J

20 In Matlab J As shown when using more streams to obtain the histogram, the resultant becomes closer to the ideal standard normal where about 98% of the area under curve (pdf) lies in the interval [-3,3], centered at the zero mean.

21 Univariate Normal : X ~ N μ, σ If U / and U C are independent random variates from U(0,1) generated before. Then Z = 2 ln U / cos 2πU C ~ N(0,1) Therefore, X = σz + μ ~ N(μ, σ)

22 Multivariate Normal: X ~ N μ, Σ Start with a d dimensional vector of standard normal N 0,1. These can be transofrmed to the desried distribution using x = R m z + μ d d matrix R m R = Σ d 1 d 1 ~ N(0,1)

23 In Matlab J é1ù é4 4ù M = ê ú ë 2û å = ê ú ë4 9 û

24 In Matlab J é1 ù M =ê ú ë 2û é 4 4ù å = ê4 9 ú ë û

25 Probability Density Function The multi-normal Gaussian PDF can be computed using the following equation: p x = 1 2π p C Σ / C exp 1 2 x μ m Σ./ x μ where d is the dimension of the input vector x.

26 In Matlab J

27 In Matlab J

28 Results

29 In Matlab J é 5 M = ê -5 ê êë 6 ù ú ú úû å é 5 2-1ù = ê ú ê ú êë úû

30 In Matlab J

31 In Matlab J

32 Thank You Questions