( x) ( ) F ( ) ( ) ( ) Prob( ) ( ) ( ) X x F x f s ds

Transcription

1 Applied Numerical Analysis Pseudo Random Number Generator Lecturer: Emad Fatemizadeh

2 What is random number: A sequence in which each term is unpredictable 29, 95, 11, 60, 22 Application: Monte Carlo Simulations Generation of Cryptographic Keys Password generation Many Combinatorial Optimization Algorithms Games

3 RN s Types: True random: Generated in non-deterministic ways. Not predictable Not repeatable. Pseudo random: Numbers that appear random, but are obtained in a deterministic, i ti Repeatable Predictable

4 RN s Generation: True Random: Physical Phenomenon: Decay times of radioactive material Electrical noise from a resistor or semiconductor Radio channel or audible noise Keyboard timings Pseudo Random: Mathematical Algorithm

5 Preliminary: PDF (Probability Density Function): ( + ) = ( ) Prob x X x x f x x + f x 0, f x dx 1 X ( ) ( ) X = CDF (Cumulative Density Function): x Prob( ) ( ) ( ) F X = X = X X x F x f s ds ( ) F ( ) + = 1, = 0 X X f X ( x) FX ( x)

6 Preliminary: Samples: Uniform: Normal f X 1 a x b f X ( x ) = b a 0 O.W. ( x) 1 = exp 2 2πσ ( x μ ) 2 2 2σ

7 Pseudo Random Number Generator: Uniform distribution Theoretically all others distribution can be be generate from Uniform. Ideal: Between [0 1] N In computer we generate random integer [0,2-1] or [0,2 N ] then convert to [0,1] Desired Properties: Long Period Uncorrelated Fast

8 Linear Congruential Generators: ( ) X = n 1 ax + + n c mod m n 0 m: the modulus m>0 a: the multiplier 0 a<m c: the increment 0 c<m X 0: the seed point 0 X 0 <m mod: Integer reminder

9 Linear Congruential Generators: X ( ax c) m n Three Classes: m=2 N, c>0 m=2 N, c=0 m=prime, c=0 n = n mod 0 Two first are fast and easy. Third has high randomness. Choice of m, a, and c are critical!

10 Linear Congruential Generators: Let a=1,c=5,m=16 and x 0 =1. The sequence of pseudo-random integers generated by this algorithm is: 1,6,15,12,13,2,11,8,9,14,7,4,5,10,3,0,1,6,15,12,13,2,11,8,9,14,.

11 Common Examples: Name m a c period ANSI C Park-Miller drand Hayes 64-bit

12 Improvement on LCG Method: Multiple Recursive Generator X = ( a X + a X + + a X + c) mod m i 1 i-1 2 i-2 k i-k

13 Matlab Coding: x(1) = 1; a = 16807; m = 2^31 1; c = 0; for n=1:10000, end; x(n+1)= mod(a*x(n)+c,m); m); x = x/(m+1); hist(x)

14 Matlab Coding:

15 Lagged Fibonacci Generators: Remember Fibonacci Sequence: X n =X n-1 +X n-2 ; X 0 = 0; X 1 =1 General Formulation: ( ) mod X = X X m n n r n s : An Operator: +, -, * r>s>0 N m=2 r random seed required! Popular Value of (r,s): (17,5), (55,24), (127,97), (607,273), (1279,418)

16 How to generate other distributions: A mathematical theory: If we pass a Uni[0,1] random variables from inverse of desired cumulative density function, then output will have distribution of probability bilit density function: u U [ 0,1] 1 ( ) ( ) x = F u f x X X

17 How to generate other distributions: Example: We want random variable with distribution of x e x 0 f X ( x ) = 0 O.W. CDF is: x 1 e x 0 1 FX x = FX = ln 1 x 0 O.W. ( ) ( ) ( ) U [ ] x = ln 1 x U 0,1 01

18

19 Method is good but NOT practical: For example consider normal distribution: 2 2 x x s 2 2 X π 1 1 f X ( x) = e F ( x) = e ds = 2π 2??

20 Box-Muller Method for Normal: Suppose u 1 and u 2 are two independent uniform random variable, then: x = 2log( u1) cos(2 πu2) y = 2log( u1) sin(2 πu2) are jointly normal random variable;

21

22 Central Limit Theorem for Normal: Average of N independent random variable (with zero mean) from any distribution converge in limits to normal! y = lim m N N i = 1 x N i Normal

23

24 Matlab Command: rand(m,n): A m n matrix with Uni[0,1] s = rand('state') : Return Current state. rand('state',s): Resets the state to s. rand('state',0): Resets the generator to its initial state. rand('state',j): For integer j, resets the generator to its j-th state. t rand('state',sum(100*clock)): Resets it to a different state each time.

25 Matlab Command: rand(m,n): A m n matrix with Uni[0,1] How to u[a,b]? a = 10; b=20; u=rand(1,1000); 1000); x = a + (b-a) * u;

26

27 Matlab Command: randn(m,n): A m n matrix with N[0,1] s = rand('state') : Return Current state. rand('state',s): Resets the state to s. rand('state',0): Resets the generator to its initial state. rand('state',j): For integer j, resets the generator to its j-th state. t rand('state',sum(100*clock)): Resets it to a different state each time.

28 Matlab Command: rand(m,n): A m n matrix with Uni[0,1] How to N[m,δδ 2 ]? m=10; s=16; u=randn(1,1000); 1000); x = m + sqrt(s) * u;

29

30 Matlab Command: p = randperm(n); Random permutation of 1:n: p=randperm(5); P=[ ];

31 Matlab Command: Other distribution: exprnd trnd Exponential random numbers Student's t random numbers raylrnd Rayleigh random numbers See Statistics Toolbox for others.

32 Monte Carlo Method Monte Carlo Integration π estimation as a simple example: Generate N couples of RVs ˆ π = N inside N Total

33 Monte Carlo Method MATLAb experiment: N = 1000; AB = 2*(rand(2,N)-0.5); a = AB(1,:); b = AB(2,:); c = sqrt(a.^2+b.^2); PIh = 4*length(find(c<=1))/length(c); err = 100*abs(PIh-pi)/pi; N Pih err%

34 Monte Carlo Method Monte Carlo Integration Another Example: Generate N couples of RVs ˆ π = N 4 inside N Total

35 Monte Carlo Method ( xy, ) : Simple Approach: 1) a x b ( ) 2) f x y h Random Point Selection and count! h f a N White ( ) g x dx N + N White Red ( xy), : 1) a x b 2)0 ( ) y f x a b

36 Monte Carlo Integration Simple Formulation: Expected Value: b Uni 1 E { x } = xf ( ) { } x x dx E x = xdx b a a { ( ) } = ( ) E g x Integral: b = 1 g x dx b a 1 b { ( )} ( ) E g x = b a a a g x dx b N N 1 1 b a g xi g x dx g x dx g x N b a N b ( ) ( ) ( ) ( ) i= 1 a a i= 1 b a i

37 Monte Carlo Integration Example and MATLAB Code: I π N π sinθdθ sin θ, θ Uni 0, π = 0 Exact value of I: 2 [ ] i i N i = 1 N = 100; a = 0; b = pi; x = rand(1,n)*(b-a)+a; Ih = sum(sin(x))*(b-a)/n; N I err%

38 Function Minimization Problem Statement: min x f ( x) Methods: Pure Mathematical Direct Search Steepest Descent ( ) 0, ( ) f x = f x = 0

39 Function Minimization Example: Best Elliptic that match a set of points!

40 Function Minimization Example: Try to solve this problem: ( ) we have a set of x,y N i i i= 1 N x y 2 i i e = + r i= 1 a b e e e = = = 0??? a b r 2

41 Function Minimization Multivariable function F(x): Gradient: A vector: G ( x ) F F F =,,, x1 x2 xn Hessian: A Matrix: H H ( x) H ( x) ij ( x) = ij n n ( ) 2 F x = x x i j T

42 Function Minimization Multivariable function F(x): Taylor Series: T 1 T F( x+ h) = F( x) + G( x) h+ h H( x) h+ 2 Example: xx 1 2 ( 1, 2) = cos( 1) + sin( 2) + xx (, ) sin ( ),cos( ) F x x x x e G x1 x2 = x1 + x2e x2 + xe 1 ( ) G x, x xx cos ( 1 ) + 2 ( ) ( 1) xx sin( ) 1 2 xx 1 2 x xe xx e = 1 2 xx 1 2 xx 1 2+ e x2 + xe xx

43 Function Minimization Steepest Descent Rule: Change function in direction of G( G(x) Example in 1D G < 0 G > 0

44 Function Minimization Steepest Descent Rule: Very Simple solution: ( ) ( G ) x = x + + ε x, x k 1 k k 0 Some methods solve this 1D minimization: ( x) ( x+ ) min F t tg x x+ tg x ( )

45 Function Minimization More Advanced Method: T 1 T F( x+ h) F( x) + G( x) h+ h H( x) h 2 n n n 1 F( x+ h) = F( x) + Gh + hh h 2 ( x h) F + n j= 1 k kj j h k j= 1 1 H G k n i i i ij j i= i= i= = G + H h = 0, k = 1, 2,, n H h = G, k = 1,2,, n h = kj j k

46 Function Minimization Matlab Command Fmin(f,x0): 1D Fminsearch(f,x0): nd Optimization toolbox

47 Function Minimization Matlab Command: function e = ellip(x); a = x(1); b = x(2); t = 0:359; xi = 5*sin(t*pi/180); yi = 40*cos(t*pi/180); e = 0; for i=1:360, e = e+((xi(i)/a).^2+(yi(i)/b).^2-1).^2; end;