Slides 5: Random Number Extensions

Transcription

1 Slides 5: Random Number Extensions We previously considered a few examples of simulating real processes. In order to mimic real randomness of events such as arrival times we considered the use of random numbers that indicated what value the events will take. We then discussed how such pseudo uniform random numbers could be generated. Today we ll look at (some) methods of how we may generate random numbers that follow other distributions as may be required as input to a simulation model, including: Inverse-transform technique. Acceptance-rejection technique. Special properties. 1

2 Inverse-Transform Technique The general concept is, for c.d.f. r = F(x): Generate r from Uniform (0, 1) (examined previously). Find x such that x = F 1 (r). 2

3 Probability Integral Transformation This process relies on the Probability Integral Transformation: If X is a continuous random variable with c.d.f. F(X), then the random variable R = F(X) is Uniform (0,1). Proof: Note that as X is continuous, and if F(X) is 1-to-1 and an increasing function, then for r (0,1). F R (r) = P(R r) = P(F(X) r) = P(F 1 (F(X)) F 1 (r)) = P(X F 1 (r)) = F(F 1 (r)) = r Hence R has the cdf of a Uniform (0,1) distribution. 3

4 Probability Integral Transformation Technical note: F(X) may not be 1-to-1 or strictly increasing, i.e., there may be region (a,b) with 0 a < b 1 where P(X (a,b)) = 0. If so let G(r) = min{x r F(x)} and note G(r) is non-decreasing. Then: F R (r) = P(R r) = P(F(X) r) = P(X min{x r F(x)}) = P(X G(r)) = r 4

5 Example: Exponential Distribution The Exponential Distribution has c.d.f. r = F(x) = 1 e λx for x 0. To generate X 1,X 2,...: X i = F 1 (R i ) = ln(1 R i) λ or, as R U(0,1),X i = ln(r i) λ 5

6 Example: Exponential Distribution Example: Generate 200 variates X i from Exponential distribution with λ = 1. So generate 200 R i s from U(0,1) and use the equation on the previous slide. 6

7 Inverse-Transform Technique This technique also works on many other distributions including: Uniform distribution (trivially). Weibull distribution. Triangular distribution... As an exercise have a go! 7

8 Discrete Distributions All discrete distributions can be generated via the inverse-transform technique. Method is either numerically, table-lookup procedure, or algebraically. Examples of application: Empirical distributions. Discrete Uniform. Poisson etc. 8

9 Discrete Distributions Example: Suppose the number of shipments, x, on a loading dock is either 0, 1, or 2. Probability distribution is: Method: Given R, the generation scheme becomes: 0 R 0.5 x = < R < R 1 9

10 Discrete Distributions Consider R 1 = 0.73 and find x i so that F (xi 1 ) < R F (xi ). Note F (x0 ) = 0.5 < 0.73 F (x1 ) = 0.80, so x 1 = 1. 10

11 Geometric Distribution The Geometric distribution has probability mass function P(X = x) = (1 p) x p for x = 0,1,2... and p (0,1) (parameterization for number of failures before success, not number of trials until success). Its c.d.f. is F(x) = x j=0 p(1 p)j = 1 (1 p) x+1. Hence F(x 1) = 1 (1 p) x. Using the inverse-transform method: Find x so that F(x 1) < R F(x). Hence 1 (1 p) x < R 1 (1 p) x+1. Or, (1 p) x+1 1 R < (1 p) x. So, log(1 R) log(1 R) 1 x < log(1 p) log(1 p) 1 log(1 p) log(1 R) Thus, X = 11

12 Acceptance-Rejection Technique Useful particularly when inverse c.d.f. does not exist in closed form, e.g., Normal distributions (more on this in next set of slides). Illustration: To generate random variates X U(1/4, 1) 1: Generate R U[0,1]. 2a: If R 1/4, accept X = R. 2b: If R < 1/4, reject R and return to step 1. R itself does not have the desired distribution, but R conditional on the event R 1/4 does. Efficiency: Depends heavily on the ability to minimize the number of rejections. 12

13 Acceptance-Rejection Technique 13

14 Acceptance-Rejection Technique Consider a Poisson distribution with p.m.f. P(X = x) = λ x e λ /x! for x = 0,1,2,... and λ > 0. λ is referred to as the rate parameter (or mean number of occurrences per unit time). A Poisson process is the number of arrivals from an exponential inter-arrival stream with mean time 1/λ. Letting A i be the inter-arrival time for the i-th unit, then over one unit of time x arrivals occur if and only if: A 1 +A 2 + +A x 1 < A 1 +A 2 + +A x +A x+1 14

15 Acceptance-Rejection Technique Recall the inverse transform method for the exponential distribution X i = log(r i )/λ. Then A 1 +A 2 + +A x 1 < A 1 +A 2 + +A x +A x+1 Becomes x i=1 log(r i)/λ 1 < x+1 i=1 log(r i)/λ. Or, x i=1 log(r i) λ > x+1 i=1 log(r i). So, log( x i=1 R i) λ > log( x+1 i=1 R i). Hence, x i=1 R i e λ > x+1 i=1 R i. This leads to an acceptance-rejection algorithm. 15

16 Acceptance-Rejection Technique Using x i=1 R i e λ > x+1 i=1 R i: 1: Set n = 0 and P = 1. 2: Generate a random number R n+1 and replace P by PR n+1. 3: If P < e λ, then accept N = n, otherwise, reject the current n, increase n by one, and return to step 2. 16

17 NonStationary Poisson Process NonStationary Poisson Process (NSPP): A Poisson arrival process whose arrival rate λ i changes over time. Think fast food, where arrival rates at the lunch and dinner hour are much greater than arrival rates during off hours. Thinning Process: Generates Poisson arrivals at the fastest rate, but accept only a portion of the arrivals, in effect thinning out just enough to get the desired time-varying rate. 17

18 NSPP Suppose arrival rates of 10, 5 and 15 for the first three hours. Thinning (in effect) generates arrivals at a rate of 15 for each of the three hours, but accepts approximately 10/15 of the arrivals in the first hour, and 5/15 of the arrivals in the second hour. 18

19 NSPP NSPP Thinning Algorithm. To generate successive arrival times T i when rates vary: 1: Let λ = maxλ(t) be the maximum arrival rate, and set t = 0 and i = 1. 2: Generate E from the exponential distribution with rate λ, E = log(r)/λ, and let t = t+e (the arrival time of the next arrival using maximal rate). 3: Generate random number R U(0,1). If R λ(t)/λ, then T i = t and i = i+1. 4: Go to step 2. 19

20 Techniques Based on Special Properties Based on features of a particular family of probability distributions. For example: Direct transformation for normal and lognormal distributions. Convolution. Beta distribution (from gamma distribution). 20

21 Direct Transformation Example Consider two standard normal random variables, Z 1 and Z 2 plotted as a point in the plane. In polar coordinates Z 1 = Bcos(θ), Z 2 = Bsin(θ). B 2 = Z 2 1 +Z 2 2 χ 2 2, i.e., Chi-square distribution with 2 degrees of freedom (which is also an Exponential distribution with λ = 2). Hence B = ( 2lnR) 1/2. Also the radius B and angle θ are mutually independent so Z 1 = ( 2logR 1 ) 1/2 cos(2πr 2 ) and Z 2 = ( 2logR 1 ) 1/2 sin(2πr 2 ) 21

22 Direct Transformation Example Approach for N(µ,σ 2 ): Generate Z i N(0,1), then X i = µ+σz i. Approach for lognormal(µ,σ 2 ): Generate X i N(µ,σ 2 ), then Y i = e X i. 22

23 Direct Transformation Some additional results include: If X and Y are independent with X Γ(α,θ) and Y Γ(β,θ), then X/(X +Y) is Beta with parameters α and β. If X Exp(1), then λx 1/k Weibull(λ,k). If X 1,X 2,...X n are independent standard normal random variables, then X 2 1 +X X 2 n is chi-square distributed with n degrees of freedom. If X 1,X 2,...X n, and Y 1,Y 2,...Y m are independent standard normal random variables, then m(x1 2 +X2 2 + Xn)/n(Y Y2 2 + Ym) 2 has an F-distribution with (n, m) degrees of freedom. If z is a vector of N independent standard normal variates, then µ+az has a N-dimensional multivariate normal distribution with mean µ and covariance Σ = AA T. Many more exist. 23

24 Summary We considered principles of random-variate generation via Inverse-transform technique. Acceptance-rejection technique. Special properties. These are useful and important for generating many continuous and discrete distributions. 24