Lecture 15 Random variables

Transcription

1 Lecture 15 Random variables Weinan E 1,2 and Tiejun Li 2 1 Department of Mathematics, Princeton University, weinan@princeton.edu 2 School of Mathematical Sciences, Peking University, tieli@pku.edu.cn No.1 Science Building, 1575

2 Outline Motivations Basic idea of Monte Carlo integration Random variables

3 High dimensional quadrature in statistical physics Ising model for mean field ferromagnet modeling 1 2 Spin system M

4 High dimensional quadrature in statistical physics Ising model in statistical physics Define the Hamiltonian H(σ) = J <ij> σ iσ j, where σ i = ±1, < ij > means to take sum w.r.t all neighboring spins i j = 1. The internal energy per site U M = 1 M σ H(σ) exp{ βh(σ)} Z M, where Z M = σ exp{ βh(σ)} is the partition function and β = (k BT ) 1. Total number of configuration states: 2 M

5 High dimensional quadrature in statistical physics A 1 A(c)e βh(c) dc Z R 6N Z = e βh(c) dc (partition R 6N funtion) β = (k BT ) 1 k B Boltzmann T dc = dx 1 dx N dp 1 dp N N.

6 Stochastic simulations Biological network Suppose there are N s species of molecules S i, i = 1,..., N s, and M R reaction channels R j, j = 1,..., M R. x i is the number of molecules of species S i. Then the state of the system is given by x = (x 1, x 2,..., x Ns ). Each reaction R j is characterized by a rate function a j(x) and a vector ν j that describes the change of state due to reaction (after the j th reaction, x x + ν j). In shorthand denote R j = (a j, ν j) How to simulate this biological process?

7 Numerical solution of stochastic differential equations In mathematical economics, the Merton s model for asset price ds t = µs tdt + σs tdw t where S t is the asset price, W t is the standard Brownian motion. In the Langevin equation for Brownian particles dx t = v tdt dv t = γ m vtdt 1 m V (xt)dt + 2k BT γdw t where V (x) is the potential, γ is the viscosity, m is the mass.

8 Outline Motivations Basic idea of Monte Carlo integration Random variables

9 Monte Carlo I(f) = 1 (trapzoidal rule) 0 f(x)dx (1) I(f) [ N f(x0) + f(x i) f(xn )] h (2) h = 1, xi = ih (i = 0, 1,..., N) N O(h 2 ) = O(N 2 ) i=1

10 Basic random variables (discrete case) Bernoulli distribution: P (X) = { p, X = 1, q, X = 0. where p > 0, q > 0, p + q = 1. The mean and variance are EX = p, Var(X) = pq. If p = q = 1, it is the well-known fair-coin tossing game. 2 Binomial distribution B(n, p): n independent experiments of Bernoulli distribution X k, X := X X n, then The mean and variance are P (X = k) = C k np k q n k. EX = np, Var(X) = npq.

11 Basic random variables (continuous case) Uniform distribution U[0, 1]: { 1 if x [0, 1] p(x) = 0 otherwise The mean and variance are EX = 1 2, Var(X) = Exponential distribution:(λ > 0) { 0 if x < 0 p(x) = λe λx if x 0 The mean and variance are EX = 1 λ, Var(X) = 1 λ. 2

12 Basic random variables (continuous case) Normal distribution(gaussian distribution)(n(0, 1)): or more generally N(µ, σ) p(x) = 1 e x2 2 2π p(x) = 1 (x µ)2 e 2σ 2 2πσ 2 where µ is the mean (expectation), σ 2 is the variance. High dimensional case (N(µ, Σ)) p(x) = 1 T Σ 1 (X µ) (2π) n/2 e (X µ) (det Σ) 1/2 where µ is the mean, Σ is a symmetric positive definite matrix, which is the covariance matrix of X. det Σ is the determinant of Σ.

13 Monte Carlo Monte Carlo I(f) I(f) = Ef(X), X [0, 1] Monte Carlo I(f) 1 N N f(x i) I N (f) (3) i=1 x i (i = 1, 2,..., N) [0, 1] i.i.d. U[0, 1], I N (f) I(f). I N (f) I(f) O(N 1 2 ).

14 Monte Carlo R d Ω = [0, 1] d I(f) = f(x)p(x)dx (4) p(x) p(x)dx = 1 p(x) 0 [0, 1] n O(n 2 ) N = n d Monte Carlo M i.i.d X 1,, X M I M 1 M i=1 M f(x i) I O(M 2 1 ) M M = n d n = M 1/d. d > 4 n 2 > M 2 1 Monte Carlo ; d < 4 Monte Carlo Ω

15 Outline Motivations Basic idea of Monte Carlo integration Random variables

16 Generation of uniform distribution U(0, 1) (1) Von Neumann (midsquare) Von Neumann

17 Generation of uniform distribution U(0, 1) (2) (linear congruential algorithm): U[0, 1] X n+1 = ax n + b(mod m) (5) a, b, m (cycle length) : a, b, m (i) b m ; (ii) (a 1) m (iii) m 4 (a 1) 4 m m = 2 k, a = 4c + 1, b

18 Generation of uniform distribution U(0, 1) (3) Lewis, Goodman Miller X n+1 = ax n (mod m) (6) a = 7 5 = 16807, m = = Shrage L Ecuyer Bays-Durham Numerical Recipe ran2(). 1000

19 Generation of uniform distribution U(0, 1) Generation of U(0, 1) with MATLAB Histograms

20 Law of Large Numbers (LLN) Theorem (Weak Law of Large Numbers, WLLN) If E X i < +, then in probability. S n n η Theorem (Strong Law of Large Numbers, SLLN) if and only if E X i < + S n n η a.s.

21 Central Limit Theorem (CLT) Theorem (Central Limit Theorem, CLT) Assume that EX 2 i σ 2 = Var(X i). Then S n nη nσ 2 N(0, 1) < + and let in the sense of distribution.

22 Generation of general RVs (1) (Transformation method) : Y F (y) P {Y y} = F (y) (7) X U[0, 1] Y = F 1 (X) x p(y) F(y) y 1 y y 1 y 2 y y 2 Y Y

23 Generation of exponentially distributed RVs (i) { 0 y 0 p(y) = λe λy y 0 F (y) = y p(z)dz = 1 0 e λy F 1 (x) = 1 ln(1 x), x (0, 1) λ (8) X i U(0, 1) Y i = 1 ln(1 Xi) i = 1, 2,... (9) λ

24 Generation of exponentially distributed RVs Generation of exponentially distributed RVs Histograms Central Limit Theorem

25 Generation of normally distributed RVs (ii) p(x) = 1 2π e x2 2 (10) F (x) = x p(y)dy = erf( x 2 ) (11) erf(x) = 2 π x 0 e t2 dt (error function) F 1 (x) = 2erf 1 (2x 1) erf 1 Box-Muller

26 Generation of normally distributed RVs (2) Box-Muler : + ( + e x2 dx) 2 = = π 0 0 e x2 dx e (x2 +y 2) dxdy e r2 rdrdθ = π (12) Box-Muller (x 1, x 2) = (r cos θ, r sin θ) x x2 2 2π e 2 dx 1dx 2 = 1 r 2 2π e ( 1 = 2π dθ 2 rdrdθ ) ) (e r22 rdr (13) θ r 1 θ 2π U[0, 2π] e r2 2 r r F (r) = 2 r 2 sds = 1 e r2 2 0 e s

27 Generation of normally distributed RVs X 1, X 2 U[0, 1] { Y1 = 2 ln X 1 cos(2πx 2) Y 2 = 2 ln X 1 sin(2πx 2) (Y 1, Y 2) Numerical Recipe (14) Generation of Gaussian RVs with MATLAB Histogram

28 Homework assignment Generate the uniform distribution, exponential and Gaussian random variables and test the Weak Law of Large Numbers and Central Limit Theorem with MATLAB.

29 References R.E. Caflish, Monte Carlo and Quasi-Monte Carlo methods, Acta Numerica, Vol. 7, 1-49, 1998.,,.