Dennis Bricker Dept of Mechanical & Industrial Engineering The University of Iowa

Transcription

1 Dennis Bricker Dept of Mechanical & Industrial Engineering The University of Iowa Probability Theory Results page 1 D.Bricker, U. of Iowa, 2002

2 Probability of simultaneous occurrence of events If A and B are two independent events, then by definition: If A and B are dependent, then { AND } = { } = { } { } P A B P A B P A P B { } = { } { } = P{ A B} P{ B} P A B P B A P A Note: This follows from the definition of conditional probability: { } { } { } P A P A B B and P{ B A} P A B P B P A { } { } Probability Theory Results page 2 D.Bricker, U. of Iowa, 2002

3 Probability of occurrence of at least one of two events If A and B are two events, then the probability of A OR B is P{ A B} = P{ A} + P{ B} P{ A B} Note: If A and B are mutually exclusive, then this is equivalent to P{ A B} = P{ A} + P{ B} If A and B are independent, then this is equivalent to P{ A B} = P{ A} + P{ B} P{ A} P{ B} Probability Theory Results page 3 D.Bricker, U. of Iowa, 2002

4 Law of Total Probability for computation of the probability of an event A. Suppose that B 1, B 2, B n are mutually exclusive events, i.e., such that A B i i B B =, i j i j Then chain rule ( ) ( ) ( ) P A = P A B P B i i i In particular, we can condition upon the value of a discrete random variable Y: ( ) = ( = ) ( = ) P A P A Y y P Y y y Probability Theory Results page 4 D.Bricker, U. of Iowa, 2002

5 Example: A certain product is manufactured in three plants: Plant # market share 1 30% 5% 2 25% 4% 3 45% 3% defective rate Suppose that we sample a unit of this product from a retail shop shelf. What is the probability that it is defective? Probability Theory Results page 5 D.Bricker, U. of Iowa, 2002

6 Let Y= plant which manufactured the unit. A = event that product is defective. 3 ( unit is defective ) = ( = ) ( = ) P P defect Y i P Y i i= 1 = = That is, the probability that we have sampled a defective unit is 3.85%. Probability Theory Results page 6 D.Bricker, U. of Iowa, 2002

7 Law of Total Expectation For any two random variables X and Y, ( ) = ( = ) ( = ) E X E X Y y P Y y y where Y has discrete values and we assume that the expectations exist. Probability Theory Results page 7 D.Bricker, U. of Iowa, 2002

8 Example: Consider a series of competitions between two evenly-matched teams, in which the series ends as soon as one team has won 3 consecutive games. What is the expected length of the series? Probability Theory Results page 8 D.Bricker, U. of Iowa, 2002

9 Solution: Define X i = number of games to follow after one of the teams has won the last i consecutive games What is the value of E(X 0 )? Define Y =1 if the team which is ahead wins the next game, 0 otherwise. Then by the Law of Total Expectation, ( ) = ( = 0) ( = 0 ) + ( = 1) ( = 1) E X E X Y P Y E X Y P Y i i i Probability Theory Results page 9 D.Bricker, U. of Iowa, 2002

10 Now, E{ X Y 0} 1 E{ X } i = = +, 1 since, if the leading team loses, then the opposing team has a winning streak of one game, and { 1} 1 { } E Xi Y E X i + 1 = = +, since, if the leading team wins, then it has a winning streak of i+1 games. Therefore, { } = { = 0} { = 0 } + { = 1} { = 1} E X E X Y P Y E X Y P Y i i i 1 1 = ( 1+ E{ X1} ) + ( 1+ E{ Xi+ 1} ) = + E X + + E X = + E X + E X E X. for i=0, 1, and 2, where ( ) { } { } 1 { } { } 1 i+ 1 1 i Probability Theory Results page 10 D.Bricker, U. of Iowa, 2002

11 1 1 E{ Xi} = 1 + E{ X1} + E{ Xi+ 1} for i = 0,1, = 1+ + = Case i=0: E{ X } E{ X } E{ X } E{ X } E{ X } = 1+ + = Case i=1: E{ X } E{ X } E{ X } E{ X } E{ X } Case i=2: since E{ X 3 } = 0, E{ X2} = 1+ E{ X1} + E{ X3} E{ X1} + E{ X2} = Probability Theory Results page 11 D.Bricker, U. of Iowa, 2002

12 We thus obtain the system of linear equations EX 0 EX1 = 1 0.5EX1 0.5EX 2 = 1 0.5EX1 + EX 2 = 1 which has the solution (using Gauss elimination, for example): EX = 7, EX = 6, EX = That is, the series is expected to end after 7 games. Notice that when one team has already won two consecutive games, we expect a total of four games, i.e., two additional games to be played! Probability Theory Results page 12 D.Bricker, U. of Iowa, 2002

13 The k th Moment of a distribution is defined to be E X k The k th Central Moment of a distribution is k [ ] k M = E X EX In particular, the 2 nd central moment is the variance, the 3 rd central moment is the skewness, and the 4 th central moment is the kurtosis. Probability Theory Results page 13 D.Bricker, U. of Iowa, 2002

14 2 Variance is the second central moment: Var{ X} =σ M 2 Therefore, { } = ( ) 2 Var X E X E X Sometimes it is more convenient to compute the variance using the relationship: { } 2 2 ( ) ( ) Var X = E X X E X + E X ( ) ( ) 2 2 = E X E 2 X E X + E E X ( ) ( ) ( ) = E X E X E X E X { } = ( 2 ) 2 ( ) Var X E X E X 2 Probability Theory Results page 14 D.Bricker, U. of Iowa, 2002

15 Other relationships: if c is a constant, The term { + } = { } 2 { } = c Var{ X} { + } = { } + { } + 2 ( )( ) Var X c V X Var cx { } Var X Y Var X Var Y E X EX Y EY {( )( )} E X EX Y EY is called the covariance of X & Y, or Cov{X,Y} A more useful formula for computation is {, } = { } { } { } Cov X Y E XY E X E Y Probability Theory Results page 15 D.Bricker, U. of Iowa, 2002

16 In general, the expectation of a sum = sum of expectations, i.e., N N i = i= 1 i= 1 ( ) E X E X i and, if the random variables X i are independent, the variance of a sum = sum of variances, i.e., N var = N Xi var( Xi). i= 1 i= 1 Thus, if the random variables X i are independent & identically-distributed, N = i ( 1 ) and var Xi Nvar( X1) E X NE X i= 1 N = i= 1 Probability Theory Results page 16 D.Bricker, U. of Iowa, 2002

17 Suppose that the number of terms N is itself a random variable. If the random event {N=n} is independent of X n+1, X n+2, for each n 1: Wald's Equation: N i = i= 1 ( ) ( ) E X E N E X 1 If in addition, the event {N=n} is independent of all X i, i=1,2,3,. N var Xi = E N var X + var N E X i= 1 2 ( ) ( ) ( ) ( ) 1 1 Probability Theory Results page 17 D.Bricker, U. of Iowa, 2002

18 Suppose that X 1, X 2, X 3,. is a sequence of independent, identicallydistributed random variables with mean and variance which exist: 2 { } Var{ X } E X and let Y n be the n th partial sum, i.e., i = µ and =σ > 0 i Y n n = X i= 1 i so that the expected value of Y n is nµ, and its variance is nσ 2. Central Limit Theorem: The distribution of the standardized form of Y n, i.e., Yn nµ, σ n is, in the limit, the normal distribution with mean 0 and variance 1. Probability Theory Results page 18 D.Bricker, U. of Iowa, 2002

19 Normal Approximation to Binomial Distribution Suppose that {X n } is a Bernouilli process, i.e., X 1, X 2,. is a sequence of Bernouilli random variables, with P{X i =1}=p. Then the n th partial sum Y n has binomial distribution with parameters n & p, with mean np and variance np(1 p). Therefore, by the Central Limit Theorem, the cumulative distribution function (CDF) of Y n, i.e., t n PYn t p p k= 0 k k { } = ( 1 ) should have, for large n, approximately normal distribution with mean np and standard deviation np( 1 p) n k Probability Theory Results page 19 D.Bricker, U. of Iowa, 2002

20 Example: In a true-false examination, What is the probability that a student can guess the correct answers to 15 or more out of 27 questions, i.e., a score of at least 55.5%? Solution: Assuming p = 50%, e.g., the student chooses his/her answer by flipping an unbiased coin, the exact value is t n PYn t p p k= 0 k k { } = ( 1 ) n k These values are shown on the following page, where we see that { } PY { } PY 15 = = 35% Probability Theory Results page 20 D.Bricker, U. of Iowa, 2002

21 x P{x} P{X<=x} P{X>x} Probability Theory Results page 21 D.Bricker, U. of Iowa, 2002

22 According to the Central Limit Theorem, Y 27 has approximately a normal distribution with mean 13.5 and standard deviation 27 ( 0.5)( 0.5) = = Therefore, according to tables for the Normal distribution or other resource (e.g., ) Y PY { } = P PZ { } = Probability Theory Results page 22 D.Bricker, U. of Iowa, 2002

23 Note: If we had computed P{Y 27 15} (instead of P{Y }), we would have obtained a less accurate value of about 28.2%. Probability Theory Results page 23 D.Bricker, U. of Iowa, 2002

24 NORMAL DISTRIBUTIONS A continuous random variable X has a Normal distribution with the two parameters: the mean µ and the standard deviation σ>0, if its probability density function is of the form f 1 1 x µ ; µσ, = exp σ 2π 2 σ ( x ) This Normal distribution, N(µ,σ), is symmetric about the mean µ, i.e., its skewness is 0. 2 Probability Theory Results page 24 D.Bricker, U. of Iowa, 2002

25 The Standard Normal distribution N(0,1) has mean µ=0 and standard deviation σ=1, with pdf f ( z) 2 1 z = exp 2π 2 If the random variable X has N(µ,σ) distribution, then X µ σ has the standard N(0,1) distribution. Probability Theory Results page 25 D.Bricker, U. of Iowa, 2002

26 The Cumulative Distribution Function (CDF) of the N(0,1) distribution cannot be expressed in closed form: t 2 1 t F( z) = P{ Z t} = exp dt 2π 2 Probability Theory Results page 26 D.Bricker, U. of Iowa, 2002

27 REPRODUCTIVE PROPERTY OF NORMAL DISTRIBUTION The Normal distribution has an important reproductive property: The sum of (independent) Normally distributed random variables has a Normal distribution! Specifically, if X N ( µ, σ ), i i i then T k = a X has N(µ,σ) distribution, i= 1 i i with mean k µ= a µ i= 1 i i and variance k aiσi i= 1 σ =. Note: The binomial, gamma, and a few other distributions also have this reproductive property! Probability Theory Results page 27 D.Bricker, U. of Iowa, 2002

28 RANDOM GENERATION OF NORMALLY DISTRIBUTED RANDOM VARIABLES Generate two random numbers u i and v i, uniformly distributed in [0,1]. Perform the transformations and ( ) z = 2lnu sin 2πv i i i ( ) z = 1 2lnu + cos 2 π v i i i to generate a pair of random variables having N(0,1) distribution. (This is known as the Box/Muller technique. Cf. ) Note: a less efficient and less accurate method, based upon the central limit theorm, would be to sum a large number (15?) uniformly-distributed random variables. Probability Theory Results page 28 D.Bricker, U. of Iowa, 2002