Chapter 5 Joint Probability Distributions

Transcription

1 Applied Statistics and Probability for Engineers Sixth Edition Douglas C. Montgomery George C. Runger Chapter 5 Joint Probability Distributions

2 5 Joint Probability Distributions CHAPTER OUTLINE 5-1 Two or More Random Variables Joint Probability Distributions Marginal Probability Distributions Conditional Probability Distributions Independence More Than Two Random Variables 5-5 General Functions of Random Variables 5-6 Moment Generating Functions 5-2 Covariance and Correlation 5-3 Common Joint Distributions Multinomial Probability Distribution Bivariate Normal Distribution 5-4 Linear Functions of Random Variables Chapter 5 Title and Outline 2

3 Learning Objectives for Chapter 5 After careful study of this chapter, you should be able to do the following: 1. Use joint probability mass functions and joint probability density functions to calculate probabilities. 2. Calculate marginal and conditional probability distributions from joint probability distributions. 3. Interpret and calculate covariances and correlations between random variables. 4. Use the multinomial distribution to determine probabilities. 5. Properties of a bivariate normal distribution and to draw contour plots for the probability density function. 6. Calculate means and variances for linear combinations of random variables, and calculate probabilities for linear combinations of normally distributed random variables. 7. Determine the distribution of a general function of a random variable. 8. Calculate moment generating functions and use them to determine moments and distributions Chapter 5 Learning Objectives 3

4 Joint Probability Mass Function Sec Joint Probability Distributions 4

5 Joint Probability Density Function The joint probability density function for the continuous random variables X and Y, denotes as f XY (x,y), satisfies the following properties: Figure 5-2 Joint probability density function for the random variables X and Y. Probability that (X, Y) is in the region R is determined by the volume of f XY (x,y) over the region R. Sec Joint Probability Distributions 5

6 Example 5-2: Server Access Time-1 Let the random variable X denote the time until a computer server connects to your machine (in milliseconds), and let Y denote the time until the server authorizes you as a valid user (in milliseconds). X and Y measure the wait from a common starting point (x < y). The joint probability density function for X and Y is , x y for 0 and 610 f x y ke x y k XY Figure 5-4 The joint probability density function of X and Y is nonzero over the shaded region where x < y. Sec Joint Probability Distributions 6

7 Example 5-2: Server Access Time-2 The region with nonzero probability is shaded in Fig We verify that it integrates to 1 as follows: 0.001x0.002 y y 0.001x f XY x, ydydx ke dy dx k e dy e dx x e 0.001x 0.003x k e dx e dx Sec Joint Probability Distributions 7

8 Example 5-2: Server Access Time-3 Now calculate a probability: , 2000, P X Y f x y dydx 0 x y 0.001x k e dy e dx x 4 e e 0.001x k e dx x x e e e dx e 4 1e e x XY Figure 5-5 Region of integration for the probability that X < 1000 and Y < 2000 is darkly shaded. Sec Joint Probability Distributions 8

9 Marginal Probability Distributions (discrete) The marginal probability distribution for X is found by summing the probabilities in each column whereas the marginal probability distribution for Y is found by summing the probabilities in each row. X Y f x f xy f y f xy y x y = Response time(nearest second) x = Number of Bars of Signal Strength f (y ) f (x ) Marginal probability distributions of X and Y Sec Marginal Probability Distributions 9

10 Marginal Probability Density Function (continuous) If the joint probability density function of random variables X and Y is f XY (x,y), the marginal probability density functions of X and Y are: Sec Marginal Probability Distributions 10

11 Example 5-4: Server Access Time-1 For the random variables that denotes times in Example 5-2, find the probability that Y exceeds 2000 milliseconds. Integrate the joint PDF directly using the picture to determine the limits PY 2000 f XY x, ydy dx f XY x, ydy dx x Dark region left dark region right dark region Sec Marginal Probability Distributions 11

12 Example 5-4: Server Access Time-2 Alternatively, find the marginal PDF and then integrate that to find the desired probability. y 0.001x0.002 y fy y ke dx 0 y y 0.001x ke e dx ke y x e e y y ke y y 0.001y 6 10 e 1 e for y P Y f y dy 2000 Y y 0.001y 610 e 1 e dy y y 3 e e e e Sec Marginal Probability Distributions 12

13 Mean & Variance of a Marginal Distribution E(X) and V(X) can be obtained by first calculating the marginal probability distribution of X and then determining E(X) and V(X) by the usual method. E X x f x R V X x f x R 2 2 X X E Y y f y R V Y y f y R X Y 2 2 Y Y Sec Marginal Probability Distributions 13

14 Mean & Variance for Example 5-1 y = Response time(nearest second) x = Number of Bars of Signal Strength f (y ) y *f (y ) y 2 *f (y ) f (x ) x *f (x ) x 2 *f (x ) E(X) = 2.35 V(X) = = = E(Y) = 2.49 V(Y) = = = Sec Marginal Probability Distributions 14

15 Conditional Probability Density Function Sec Conditional Probability Distributions 15

16 Example 5-6: Conditional Probability-1 From Example 5-2, determine the conditional PDF for Y given X=x. f 0.001x0.002 y f X x k e dy x Yx y ke 0.001x e y e x ke x 0.003x e for x0 f XY x, y ke f ( x) 0.003e X 0.001x0.002 y 0.003x 0.002x0.002 y for 0 e x and x y Sec Conditional Probability Distributions 16

17 Example 5-6: Conditional Probability-2 Now find the probability that Y exceeds 2000 given that X=1500: P Y X 1500 f Y e 0.002e 3 y dy y y e e e e Sec Conditional Probability Distributions 17

18 Mean & Variance of Conditional Random Variables The conditional mean of Y given X = x, denoted as E(Y x) or μ Y x is Yx E Y x y f y y The conditional variance of Y given X = x, denoted as V(Y x) or σ 2 Y x is Y x Y x Y x Y x y V Y x y f y y f y y Sec Conditional Probability Distributions 18

19 Example 5-8: Conditional Mean And Variance From Example 5-2 & 5-6, what is the conditional mean for Y given that x = 1500? y y E Y X 1500 y 0.002e dy 0.002e y e dy y y 3 e e e y dy y e 0.002e e e Sec Conditional Probability Distributions e e e e If the connect time is 1500 ms, then the expected time to be authorized is 2000 ms.

20 Example 5-9 For the discrete random variables in Exercise 5-1, what is the conditional mean of Y given X=1? y = Response time(nearest second) x = Number of Bars of Signal Strength f (x ) y*f(y x=1) y 2 *f(y x=1) Sum of f(y x) The mean number of attempts given one bar is 3.55 with variance of Sec Conditional Probability Distributions 20 f (y )

21 Independent Random Variables For random variables X and Y, if any one of the following properties is true, the others are also true. Then X and Y are independent. Sec Independence 21

22 Example 5-11: Independent Random Variables Suppose the Example 5-2 is modified such that the joint PDF is: Are X and Y independent? X Find the probability x y f x, y 210 e for x 0 and y 0. XY x0.002 y x Y 210 f x e dy x e for x0 y f y e dx y e for y > , P X Y P X P Y 1 2 e e Sec Independence 22

23 Joint Probability Density Function The joint probability density function for the continuous random variables X 1, X 2, X 3, X p, denoted as f X X X x, x,..., xp satisfies the following properties: p 1 2 Sec More Than Two Random Variables 23

24 Example 5-14: Component Lifetimes In an electronic assembly, let X 1, X 2, X 3, X 4 denote the lifetimes of 4 components in hours. The joint PDF is: x x x x4 f x, x, x, x 9 10 e for x 0 X X X X What is the probability that the device operates more than 1000 hours? The joint PDF is a product of exponential PDFs. P(X 1 > 1000, X 2 > 1000, X 3 > 1000, X 4 > 1000) = e = e -7.5 = i Sec More Than Two Random Variables 24

25 Marginal Probability Density Function Sec More Than Two Random Variables 25

26 Mean & Variance of a Joint Distribution The mean and variance of X i can be determined from either the marginal PDF, or the joint PDF as follows: Sec More Than Two Random Variables 26

27 Example 5-16 Points that have positive probability in the joint probability distribution of three random variables X1, X2, X3 are shown in Figure. Suppose the 10 points are equally likely with probability 0.1 each. The range is the non-negative integers with x 1 +x 2 +x 3 = 3 List the marginal PDF of X 2 f f f f x P (X 2 = 0) = x1x2 x3(3,0,0) + x1x2 x (0,0,3) + x1x2 x (1,0,2) + 1x2x (2,0,1) = 0.4 P (X 2 = 1) = f x 1x2x (2,1,0) + f (0,1,2) + f 3 x1x2 x3 x 1x2x3(1,1,1) = 0.3 P (X 2 = 2) = f (1,2,0) + f x1x2 x3 x 1x2x3(0,2,1) = 0.2 P (X 2 = 3) = f x x (0,3,0) = x3 Also, E(x 2 ) = 0(0.4) + 1(0.3) + 2(0.2) + 3(0.1) = 1 Sec More Than Two Random Variables 27

28 Distribution of a Subset of Random Variables Sec More Than Two Random Variables 28

29 Conditional Probability Distributions Conditional probability distributions can be developed for multiple random variables by extension of the ideas used for two random variables. Suppose p = 5 and we wish to find the distribution conditional on X 4 and X 5. f x, x, x X X X for f x, x 0. X X X X 4 5 f x, x, x, x, x X X X X X f x, x X X 4 5 Sec More Than Two Random Variables 29

30 Independence with Multiple Variables The concept of independence can be extended to multiple variables. Sec More Than Two Random Variables 30

31 Example 5-18: Layer Thickness Suppose X 1,X 2, and X 3 represent the thickness in μm of a substrate, an active layer and a coating layer of a chemical product. Assume that these variables are independent and normally distributed with parameters and specified limits as tabled. What proportion of the product meets all specifications? Answer: , 3 layer product. Which one of the three thicknesses has the least probability of meeting specs? Answer: Layer 3 has least prob. Parameters and specified limits Normal Random Variables X 1 X 2 X 3 Mean (μ) 10,000 1, Std dev (σ) Lower limit 9, Upper limit 10,800 1, P(in limits) P(all in limits) = Sec More Than Two Random Variables 31

32 Covariance Covariance is a measure of the relationship between two random variables. First, we need to describe the expected value of a function of two random variables. Let h(x, Y) denote the function of interest. Sec 5-2 Covariance & Correlation 32

33 Example 5-19: Expected Value of a Function of Two Random Variables For the joint probability distribution of the two random variables in Example 5-1, calculate E [(X-μ X )(Y-μ Y )]. The result is obtained by multiplying x - μ X times y - μ Y, times f xy (X,Y) for each point in the range of (X,Y). First, μ X and μ y were determined previously from the marginal distributions for X and Y: Therefore, μ X = 2.35 and μ y = 2.49 Sec 5-2 Covariance & Correlation 33

34 Covariance Defined Sec 5-2 Covariance & Correlation 34

35 Correlation (ρ = rho) Sec 5-2 Covariance & Correlation 35

36 Example 5-21: Covariance & Correlation Determine the covariance and correlation to the figure below. Figure 5-13 Discrete joint distribution, f(x, y). Joint Marginal Mean StDev x y f(x, y) x-μ X y-μ Y Prod covariance = correlation = μ X = 1.8 μ Y = 1.8 σ X = σ Y = Sec 5-2 Covariance & Correlation 36 Note the strong positive correlation.

37 Independence Implies ρ = 0 If X and Y are independent random variables, σ XY = ρ XY = 0 ρ XY = 0 is necessary, but not a sufficient condition for independence. Sec 5-2 Covariance & Correlation 37

38 Example 5-23: Independence Implies Zero Covariance Let f xy x y 16 for 0 x 2 and 0 y 4 XY Show that E XY E X E Y 0 E X E Y XY x ydx dy x y dy y xy dx dy x y dy y E XY XY x y dx dy x y dy y dy y E XY E X E Y Figure 5-15 A planar joint distribution. Sec 5-2 Covariance & Correlation 38

39 Multinomial Probability Distribution Suppose a random experiment consists of a series of n trials. Assume that: 1) The outcome of each trial can be classifies into one of k classes. 2) The probability of a trial resulting in one of the k outcomes is constant, and equal to p 1, p 2,, p k. 3) The trials are independent. The random variables X 1, X 2,, X k denote the number of outcomes in each class and have a multinomial distribution and probability mass function: Sec Multinomial Probability Distribution 39

40 Example 5-25: Digital Channel Of the 20 bits received over a digital channel, 14 are of excellent quality, 3 are good, 2 are fair, 1 is poor. The sequence received was EEEEEEEEEEEEEEGGGFFP. Let the random variables X1, X2, X3, and X4 denote the number of bits that are E, G, F, and P, respectively, in a transmission of 20 bits. What is the probability that 12 bits are E, 6 bits are G, 2 are F, and 0 are P? 20! P X1 12, X2 6, X3 2, X !6!2!0! Using Excel = (FACT(20)/(FACT(12)*FACT(6)*FACT(2))) * 0.6^12*0.3^6*0.08^2 Sec Multinomial Probability Distribution 40

41 Multinomial Mean and Variance The marginal distributions of the multinomial are binomial. If X 1, X 2,, X k have a multinomial distribution, the marginal probability distributions of X i is binomial with: E(X i ) = np i and V(X i ) = np i (1-p i ) Sec Multinomial Probability Distribution 41

42 Bivariate Normal Probability Density Function The probability density function of a bivariate normal distribution is 1 f XY x, y; X, X, Y, Y, e X Y u where u x 2 x y y X X Y Y X XY Y for x and y. x 0, x, Parameter limits: 1 1 y 0, y, Sec Bivariate Normal Distribution 42

43 Marginal Distributions of the Bivariate Normal Random Variables If X and Y have a bivariate normal distribution with joint probability density function f XY (x,y;σ X,σ Y,μ X,μ Y,ρ) the marginal probability distributions of X and Y are normal with means μ X and μ Y and standard deviations σ X and σ Y, respectively. Sec Bivariate Normal Distribution 43

44 Conditional Distribution of Bivariate Normal Random Variables If X and Y have a bivariate normal distribution with joint probability density f XY (x,y;σ X,σ Y,μ X,μ Y,ρ), the conditional probability distribution of Y given X = x is normal with mean and variance as follows: Y Yx Y x X X Yx Y Sec Bivariate Normal Distribution 44

45 Correlation of Bivariate Normal Random Variables If X and Y have a bivariate normal distribution with joint probability density function f XY (x,y;σ X,σ Y,μ X,μ Y,ρ), the correlation between X and Y is ρ. Sec Bivariate Normal Distribution 45

46 Bivariate Normal Correlation and Independence In general, zero correlation does not imply independence. But in the special case that X and Y have a bivariate normal distribution, if ρ = 0, then X and Y are independent. If X and Y have a bivariate normal distribution with ρ=0, X and Y are independent. Sec Bivariate Normal Distribution 46

47 Linear Functions of Random Variables A function of random variables is itself a random variable. A function of random variables can be formed by either linear or nonlinear relationships. We limit our discussion here to linear functions. Given random variables X 1, X 2,,X p and constants c 1, c 2,, c p Y= c 1 X 1 + c 2 X c p X p is a linear combination of X 1, X 2,,X p. Sec 5-4 Linear Functions of Random Variables 47

48 Mean and Variance of a Linear Function If X 1, X 2,,X p are random variables, and Y= c 1 X 1 + c 2 X c p X p, then Sec 5-4 Linear Functions of Random Variables 48

49 Example 5-31: Error Propagation A semiconductor product consists of three layers. The variances of the thickness of each layer is 25, 40 and 30 nm. What is the variance of the finished product? Answer: Sec 5-4 Linear Functions of Random Variables 49

50 Mean and Variance of an Average Sec 5-4 Linear Functions of Random Variables 50

51 Reproductive Property of the Normal Distribution Sec 5-4 Linear Functions of Random Variables 51

52 Example 5-32: Linear Function of Independent Normal Random variables Let the random variables X 1 and X 2 denote the length and width of a manufactured part. Their parameters are shown in the table. What is the probability that the perimeter exceeds 14.5 cm? Let Y 2X 2 X perimeter 1 2 E Y 2E X 2E X cm cm 2 2 V Y 2 V X 2 V X SD Y PY Parameters of X 1 X 2 Mean 2 5 Std Dev Using Excel = 1 - NORMDIST(14.5, 14, SQRT(0.2), TRUE) Sec 5-4 Linear Functions of Random Variables 52

53 General Function of a Discrete Random Variable Suppose that X is a discrete random variable with probability distribution f X (x). Let Y = h(x) define a one-to-one transformation between the values of X and Y so that the equation y = h(x) can be solved uniquely for x in terms of y. Let this solution be x = u(y), the inverse transform function. Then the probability mass function of the random variable Y is f Y (y) = f X [u(y)] Sec 5-5 General Functions of Random Variables 53

54 Example 5-34: Function of a Discrete Random Variable Let X be a geometric random variable with probability distribution f X (x) = p(1-p) x-1, x = 1, 2, Find the probability distribution of Y = X 2. Solution: Since X 0, the transformation is one-to-one. The inverse transform function is X = y. y f Y (y) = p(1-p) -1, y = 1, 4, 9, 16, Sec 5-5 General Functions of Random Variables 54

55 General Function of a Continuous Random Variable Suppose that X is a continuous random variable with probability distribution f X (x). Let Y = h(x) define a one-to-one transformation between the values of X and Y so that the equation y = h(x) can be solved uniquely for x in terms of y. Let this solution be x = u(y), the inverse transform function. Then the probability distribution of Y is f Y (y) = f X [u(y)] J where J = u (y) is called the Jacobian of the transformation and the absolute value of J is used. Sec 5-5 General Functions of Random Variables 55

56 Example 5-35: Function of a Continuous Random Variable Let X be a continuous random variable with probability distribution: x f X ( x) for 0 x 4 8 Find the probability distribution of Y = h(x) = 2X + 4 Note that Y has a one-to-one relationship to X. y 4 1 x u y and the Jacobian is J u ' y 2 2 y y 4 f Y y for 4 y Sec 5-5 General Functions of Random Variables 56

57 Definition of Moments about the Origin The rth moment about the origin of the random variable X is r ' EX ( ) r r X f ( x), X discrete r X f ( x) dx, X continuous Sec 5-6 Moment Generating Functions 57

58 Definition of a Moment-Generating Function The moment-generating function of the random variable X is the expected value of e tx and is denoted by M X (t). That is, tx M ( t) M ( e ) X tx e f ( x), X discrete tx e f ( x) dx, X continuous Let X be a random variable with moment-generating function M X (t). Then Sec 5-6 Moment Generating Functions 58

59 Example 5-36 Moment-Generating Function for a Binomial Random Variable-1 Let X follows a binomial distribution, that is n x nx f ( x) p (1 p), x 0,1,..., n x Determine the moment generating function and use it to verify that the mean and variance of the binomial random variable are μ=np and σ 2 =np(1-p). The moment-generating function is n n n n M X ( t) e p (1 p) ( pe ) (1 p) x0 x x0 x tx x n x t x nx which is the binomial expansion of [ pe t (1 p)] Now the first and second order derivatives will be M ( t) npe [1 p( e 1)] and ' t t n1 x M ( t) npe (1 p npe )[1 p( e 1)] '' t t t n2 x n Sec 5-6 Moment Generating Functions 59

60 Example 5-36 Moment-Generating Function for a Binomial Random Variable-2 If we set t = 0 in the above two equations we get M ( t) np and ' x '' x ' 1 M ( t) np(1 p np) ' 2 Now the variance is np(1 p np) ( np) 2 ' np np 2 np(1 p) 2 Hence, the mean is np and variance is np(1 p). Sec 5-6 Moment Generating Functions 60

61 Properties of Moment-Generating Function If X is a random variable and a is a constant, then 1 2 at 1. M ( t) e M ( t) X a X 2. M ( t) M ( at) ax X If X, X,..., X are independent random variables with n moment generating functions M ( t), M ( t),..., M ( t) X X X 1 2 respectively, and if Y X X... X then the moment generating function of Y is M ( t) M ( t). M ( t)..... M ( t) Y X X X 1 2 n n n Sec 5-6 Moment Generating Functions 61

62 Example 5-38 Distribution of a Sum of Poisson Random Variables Suppose that X 1 and X 2 are two independent Poisson random variables with parameters λ 1 and λ 2, respectively. Determine the probability distribution of Y = X 1 + X 2. The moment-generating function of a Poisson random variable with parameter λ is t ( e 1) M () t e X M ( t) e and M ( t) e ( e 1) ( e 1) X Hence for X 1 and X 2, 1 2 X 1 2 Using MY ( t) M X ( t). M, the moment-generating 1 X ( t)..... M ( ) 2 X n t function of Y = X 1 + X 2 is M ( t) M ( t). M ( t) Y X X e e 1 2 t 1 e 2 ( 1) ( e 1) ( )( e 1) 1 2 e t Sec 5-6 Moment Generating Functions 62 t t t

63 Important Terms & Concepts for Chapter 5 Bivariate distribution Bivariate normal distribution Conditional mean Conditional probability density function Conditional probability mass function Conditional variance Contour plots Correlation Covariance Error propagation General functions of random variables Independence Joint probability density function Joint probability mass function Linear functions of random variables Marginal probability distribution Multinomial distribution Reproductive property of the normal distribution Chapter 5 Summary 63