Independent Events Two events are independent if knowing that one occurs does not change the probability of the other occurring Conditional probability is denoted P(A B), which is defined to be: P(A and B) / P(B) P(A B) = probability of A occurring given that B occurs A independent of B if P(A) = P(A B) Independence is equivalent to P(A and B) = P(A) * P(B) if A independent of B, then B is independent of A In terms of a Venn diagram, P(A) = P(A B) says that the ratio of the area of (A and B) to the area of B is the same as the area of A A A and B B 22
Independence of Random Variables Two random variables X and Y are independent if knowing the realized value of one of them (e.g.,y = y), doesn t provide any information on the probability of the other (X) taking on any particular value; notation: P(X = x) = P(X = x Y = y) for all possible realized values of X and Y This is equivalent to saying that the events A and B are independent, where A = {outcomes for which X = x} B = {outcomes for which Y = y} For all possible realized values of X and Y 23
Independence of Random Variables Observe 250 realizations of three random variables, X, Y, and Z, i.e., {x i, y i, z i } for i = 1 to 250 The R 2 (R = correlation) between the x i values and the y i values is a measure of what percentage of the deviations of the y i s from E(Y) can be predicted by knowing the deviations of the corresponding x i s from E(X) R 2 for these realizations: R 2 0: Top figure shows essentially no correlation between X and Y, i.e., knowing X s realizations tells you nothing about Y s realizations we can reasonably believe that X and Y are independent* R 2 = 1: Bottom figure shows perfect correlation between Z and Y, i.e., knowing Y s realizations tells you everything about Z s realizations we can reasonably believe that Y and Z are not independent Y realizations 100.0 90.0 80.0 70.0 60.0 50.0 40.0 30.0 20.0 10.0 0.0 Zero Correlation Example R^2 close to 0% 0.0 20.0 40.0 60.0 80.0 100.0 X realizations * Realizations from independent random variables will have zero correlation, but note that zero correlation does not necessarily imply independence in general 24
Properties of Random Numbers Random numbers are (independent) realizations of a random variable U(0,1) with uniform probability distribution on the closed interval [0, 1] (i.e., including 0 and 1) Can think of it as a series of realizations from U i, i = 1, 2, 3, where the U i are independent, identically distributed (i.i.d.) uniform random variables f ( x) = # 1, 0 $ x $ 1 "! 0, otherwise E( R) 1 =! xdx 0 = 2 x 2 1 0 = 1 2 25
More General Uniform Probability Distribution A random variable U(a,b) is uniformly distributed on the interval (a,b) if its pdf and cdf are: $ 1!, f ( x) = # b & a!" 0, Properties P(x 1 < X < x 2 ) is proportional to the length of the interval, because F(x 2 ) F(x 1 ) = (x 2 -x 1 )/(b-a) E(X) = (a+b)/2 V(X) = (b-a) 2 /12 a % x % b otherwise $ 0,! x ' a F( x) = #,! b ' a " 1, f(x) x < a a & x < b x % b As described on the previous slide, U(0,1) is what we use to define random numbers 1 (b-a) a b 26
Review: Probability Density Functions and Cumulative Distributions Probability density functions on left Cumulative probability distributions on right Area under the left hand curve from 0 to a point x 0 = height of right hand curve at x 0 area = height Exponential (λ) x 0 x 0 For x 0 =4, area = (4-1)(0.2) = 0.6 Uniform (1, 6) 27
Generation of Pseudo-Random Numbers Pseudo, because generating numbers using a known method removes the potential for true randomness Goal: To produce a sequence of numbers falling into the closed interval [0,1] that simulates the random variable R, i.e., so that the realizations provide statistically significant evidence that the ideal properties of random numbers (RN) Important considerations in software that generates random variables Fast Portable to different computers Sufficiently long cycle Replicable Uniformity and independence One standard way to generate random numbers is using a linear congruential generator, or better yet a combination of them Random numbers are important in simulation Realizations of random variables are generated using random numbers These realizations are called random variates 28
Linear Congruential Random Number Generator Generate a sequence of integers Z 1, Z 2, Z 3, via the recursion Z i = (a Z i 1 + c) (mod m) for i = 0, 1, 2, m (at most) where a, c, and m are carefully chosen constants Specify a seed Z 0 to start off mod m means take the remainder after dividing (a Z i 1 + c) by m Because the Z i s are between 0 and m 1 Return the ith random number as U i = Z i / m Issues: Cycling (repeating values of the random numbers) Independence Uniformity The selection of the values for a, c, m, and Z 0 drastically affects the statistical properties and the cycle length Typical to overcome issues by combining several linear congruential generators 29
The Current (as of 2000) Arena Random Number Generator Uses some of the same ideas as LCG Modulo division, recursive on earlier values But is not an LCG Combines two separate component generators Recursion involves more than just the preceding value Combined multiple recursive generator (CMRG) A n = (1403580 A n-2 810728 A n-3 ) mod 4294967087 B n = (527612 B n-1 1370589 B n-3 ) mod 4294944443 Two simultaneous recursions Z n = (A n B n ) mod 4294967087 Combine the two Seed = a six-vector of first three A n s, B n s Cycle length = 3.1 10 57 U n = Z n / 4294967088 if Z n > 0 4294967087 / 4294967088 if Z n = 0 The next random number 30
Tests for Random Numbers There are standard tests for uniformity and independence When to use tests: If a well-known simulation languages or random-number generators is used, it is probably unnecessary to test If the generator is not explicitly known or documented tests should be applied to many sample numbers. Types of tests: Theoretical tests: evaluate the mathematics of the random number generator without actually generating any numbers Empirical tests: use statistical tests on actual sequences of numbers produced Test of uniformity Two different methods that check to see if the observed relative frequencies are close to the expected relative frequencies Kolmogorov-Smirnov test Chi-square test Tests for autocorrelation Methods that check to see if there is correlation among the set of random numbers 31
Generating Realizations of a Discrete Random Variable Using inverse-transform technique Suppose the random variable X has a discrete probability distribution given by: x P(X =x) F(x) 0 0.50 0.50 1 0.30 0.80 2 0.20 1.00 Given a random number, a realization of u = 0.73 of U(0,1), x, a realization of X, is given by: " 0, $ x = # 1, $ % 2, u & 0.5 0.5 < u & 0.8 0.8 < u &1.0 u 0.8 This is equivalent to: X = 0 X = 1 X = 2 x 0.73 0.0 0.5 0.8 1.0 32
Generating Exponentially Distributed Realizations X is an exponentially distributed random variable with λ = 1 Let u = P( X x) = F(x) = 1 e -λx for x 0 To generate realizations of X (random variates) U 1 Let x i = F -1 (u i ) = -(1/λ) ln(1-u i ) Where the u i are random numbers, i.e., realizations of U i (0, 1) U 2 We can use the inverse-transform technique for some continuous distributions U 1 33
Example Generation of Realizations Example: Generate 200 realizations from an exponentially distributed random variable with λ = 1 Examples of other distributions for which the inverse-transform technique works are: Uniform distribution Weibull distribution Triangular distribution All discrete distributions 34
Normal (Gaussian) Distribution A random variable X is normally distributed has the pdf: "! < µ <! Mean: Variance:! 2 > 0 Denoted as X ~ N(µ, σ) f (x) = - $ 1 " 2# e,- Special properties:. The pdf is symmetrical around the mean, i.e., f(µ - x) = f(µ + x) The maximum value of the pdf occurs at x = µ; i.e., the mean and mode are equal Unimodal, i.e., the pdf decreases as the distance from µ increases + 2 1 % x$µ (. ' * 0 2& " ) / 0, $ 1 < x < 1 1.0 F(x) 0.5 µ 35