Ch. 5 Joint Probability Distributions and Random Samples

Transcription

1 Ch. 5 Joint Probability Distributions and Random Samples 5. 1 Jointly Distributed Random Variables In chapters 3 and 4, we learned about probability distributions for a single random variable. However, it is often useful to have more than one random variable defined in a random eperiment. It is especially useful if you re interested in the relationship between those two variables. In this class, we will learn about joint probability distributions of only TWO random variables despite the fact that the book covers joint distributions of more than two random variables. Two Discrete Random Variables Joint Probability Mass Function of Discrete Random Variables The joint probability mass function of the discrete random variables X and Y, denoted as p(, y), satisfies the following properties: 1. p(, y) 0 2. y p(, y) = 1 3. p(, y) = P(X = and Y = y) [Note: P(X = and Y = y) can also be denoted as P(X =, Y = y)] Just as the probability mass function of a single random variable X is assumed to be zero at all values outside the range of X, so is the joint probability mass function of X and Y assumed to be zero at values for which a probability is not specified. STA 3032 Ch. 5 Notes 1

2 Eample 1 Determine the value of c that makes the function p(, y) = c( + y) a joint probability mass function over the nine points with = 1, 2, 3 and y = 1, 2, 3. It is easy to see that the first property of joint probability distributions is satisfied because all values of X and Y are non-negative numbers. Thus, as long as c is positive, the function p(, y) will always be greater than or equal to 0. Now, we need to ensure that the sum over the range of (X, Y) equals 1. Let R denote the range of (X, Y). p(, y) = p(1, 1) + p(1, 2) + p(1, 3) + p(2, 1) + p(2, 2) + p(2, 3) + p(3, 1) R +p(3, 2) + p(3, 3) = 2c + 3c + 4c + 3c + 4c + 5c + 4c + 5c + 6c = 36c Since we want the sum to equal 1, we can set this result equal to 1 and solve for c. 36c = 1 c = 1 36 Thus, p(, y) = 1 ( + y). 36 Eample 2 Using the joint pmf from Eample 1, p(, y) = 1 ( + y) for = 1, 2, 3 and y = 1, 2, 3, find 36 the following: a) P(X = 1, Y < 4) To find this probability, we must add up the probability of each pair of and y where equals 1 and y is less than 4. P(X = 1, Y < 4) = p(1,1) + p(1,2) + p(1,3) = 1 36 [(1 + 1) + (1 + 2) + (1 + 3)] = 1 36 ( ) = 9 36 = 1 4 STA 3032 Ch. 5 Notes 2

3 Eample 2 Continued b) P(X = 1) To find this probability, we need to sum up the probability of each pair of and y where equals 1. P(X = 1) = p(1,1) + p(1,2) + p(1,3) = 1/4 c) P(Y = 2) This time the value of Y will remain constant, so we need to sum up the probabilities of each, y pair where y equals 2. P(Y = 2) = p(1,2) + p(2,2) + p(3,2) = 1 12 ( ) = = 1 3 d) P(X < 2, Y < 2) To find this answer, we must add up the probability of each, y pair where both and y are less than 2. There s only one, y pair that satisfies that requirement: (1, 1). P(X < 2, Y < 2) = p(1,1) = 1 36 (1 + 1) = 2 36 = 1 18 Marginal Probability Mass Functions of Discrete Random Variables The individual probability distribution of a random variable is referred to as its marginal probability distribution. The marginal probability distribution of a variable can be determined from the joint probability distribution of that variable and other random variables. For a discrete random variable, X, you can find the marginal distribution of X, P(X = ), by summing the joint distribution, P(X =, Y = y), over all points in the range of (X, Y) where X =. If the joint probability mass function of the discrete random variables X and Y is p(, y), the marginal probability mass function of X is P(X = ) = p X () = p(, y) y STA 3032 Ch. 5 Notes 3

4 and the marginal probability mass function of Y is P(Y = y) = p Y (y) = p(, y) Eample 3 Using the joint pmf from Eamples 1 and 2, p(, y) = 1 ( + y) for = 1, 2, 3 and y = 1, 2, 3, 36 find the marginal probability distribution of the random variable X. We need to calculate the marginal pmf for each value of X. We do this by keeping the value of X constant and summing the joint pdf over all possible values of Y. p X (1) = p(1, y) = p(1, 1) + p(1, 2) + p(1, 3) = 1 36 ( ) = 9 36 = 1 4 y p X (2) = p(2, y) = p(2, 1) + p(2, 2) + p(2, 3) = 1 12 ( ) = = 1 3 y p X (3) = p(3, y) = p(3, 1) + p(3, 2) + p(3, 3) = 1 15 ( ) = = 5 12 y Two Continuous Random Variables Joint Probability Density Function of Continuous Random Variables A joint probability density function for the continuous random variables X and Y, denoted as f(, y), satisfies the following properties: 1. f(, y) 0 for all, y 2. f(, y)ddy = 1 3. For any region A of two-dimensional space, P (X, Y) A = f(, y)ddy A In particular, if A is any two-dimensional rectangle {(, y): a b, c y d}, then P (X, Y) A = P(a X b, c Y d) = f(, y)dyd a b c d STA 3032 Ch. 5 Notes 4

5 Typically, f(, y) is defined over all of two-dimensional space by assuming that f(, y) = 0 for all points where f(, y) is not specified. Eample 4 Suppose f(, y) = 10e 2 3y is a joint probability density function over the range > 0 and 0 < y <. Determine the following: a) P(X < 1, Y < 2) Since the range of X is > 0, we will need to integrate from 0 to 1 for X. The range of Y is 0 < y <, so we will integrate over the interval 0 to for Y. 1 P(X < 1, Y < 2) = 10 e 2 3y dyd = 10 e e 3y y= y=0 = 10 3 e 2 [ e 3 e 0 ]d = 10 3 e 2 [1 e 3 ]d 0 1 = 10 3 e 2 e 5 d = e e 5 0 = e e 2 = e 5 5 e = 10 5e 2 3 2e = 1 3 (2e 5 5e 2 + 3) = 1 ( ) = d b) P(1 < X < 2) 2 2 P(1 < X < 2) = 10 e 2 3y dyd = 10 e 2 e 5 d = e e 2 = 10 e e 4 e e 2 2 = 1 3 (2e 10 5e 4 2e 5 + 5e 2 ) = = STA 3032 Ch. 5 Notes 5

6 Eample 4 Continued c) P(Y > 3) Since 0 < y <, we will integrate with respect to y from 3 to and integrate with respect to from 3 to. P(Y > 3) = 10 e 2 3y dyd = 10 3 e 2 [ e 3y ] y= y= d = 10 3 e 2 [ e 3 e 9 ] d = 10 3 e 9 e 2 + e 5 d 3 = 10 e e 9 e = 10 0 e e 9 e 6 2 = 10 3 e 15 2 e 15 5 = 1 3 (5e 15 2e 15 ) = e 15 = d) P(X < 2, Y < 2) 2 2 P(X < 2, Y < 2) = 10 e 2 3y dyd = 10 e 2 e 5 d = e e 2 = 10 e e = 1 3 (2e 10 5e ) = = Marginal Probability Density Functions of Continuous Random Variables If the joint probability density function of the continuous random variables X and Y is f(, y), the marginal probability density functions of X and Y are P(X = ) = f X () = f(, y)dy y and P(Y = y) = f Y (y) = f(, y)d where the first integral is over all points in the range of (X, Y) for which X = and the second integral is over all points in the range of (X, Y) for which Y = y. STA 3032 Ch. 5 Notes 6

7 Eample 5 Using the joint pdf from Eample 4, f(, y) = 10e 2 3y for > 0 and 0 < y <, find the marginal probability distribution of X. Just like with discrete joint distributions, we need to keep constant. However, this time we will integrate the pdf with respect to y over the entire range of Y instead of summing like we did for discrete distributions. f X () = 10 e 2 3y dy 0 = 10e 2 3 = 10e e 3y = 10e 2 ( e 3 e 0 ) 0 3 (1 e 3 ) = 10 3 (e 2 e 5 ) for > 0 Independent Random Variables In some random eperiments, knowledge of the values of X does not change any of the probabilities associated with the values of Y. Two random variables X and Y are independent if for every pair of and y p(, y) = p X () p Y (y) for discrete X and Y or f(, y) = f X ()f Y (y) for continuous X and Y Eample 6 Using the joint discrete pmf from Eamples 1-3, p(, y) = 1 ( + y) for = 1, 2, 3 and 36 y = 1, 2, 3, determine if X and Y independent. In Eample 3, we found the marginal distribution of X, but we also need the marginal distribution of Y to determine independence. p Y (1) = p(, 1) = p(1, 1) + p(2, 1) + p(3, 1) = 1 36 ( ) = 9 36 = 1 4 STA 3032 Ch. 5 Notes 7

8 Eample 6 Continued p Y (2) = p(, 2) p Y (3) = p(, 3) Recall that, p X (1) = 1 4 = p(1, 2) + p(2, 2) + p(3, 2) = 1 12 ( ) = = 1 3 = p(1, 3) + p(2, 3) + p(3, 3) = 1 15 ( ) = = 5 12 p X (2) = 1 3 p X (3) = 5 12 In order to be independent, p(, y) = p X () p Y (y) for all values of X and Y. p(1,1) = = = p X (1) p Y(1) Since the statement does not hold up when X = 1 and Y = 1, they are not independent. X and Y are dependent. More Than Two Random Variables Please skip this section. Conditional Distributions When two variables are defined in a random eperiment, knowledge of one can change the probabilities that we associate with the values of the other. Conditional Probability Mass Function Recall that the definition of conditional probability for events A and B is P(B A) = P(A B)/P(A). This definition can be applied with the event A defined to be X = and event B defined to be Y = y. Thus, for the discrete random variables X and Y with joint pmf p(, y) the conditional probability of Y = y given that X = is denoted by P(Y = y X = ) = p Y X (y ) = P(X =, Y = y) P(X = ) = p(, y) p X () STA 3032 Ch. 5 Notes 8

9 Eample 7 Using the joint pmf, p(, y) = 1 ( + y) for = 1, 2, 3 and y = 1, 2, 3, find the following: 36 a) Conditional probability distribution of Y given that X = 1 Using the marginal distribution of X found in Eample 3, p Y X (1 1) = p(1,1) p X (1) = 2/36 1/4 = = 8 36 = 2 9 p Y X (2 1) = p(1,2) p X (1) = 3/36 1/4 = = = 1 3 p Y X (3 1) = p(1,3) p X (1) = 4/36 1/4 = = = 4 9 b) Conditional probability distribution of X given that Y = 2 Using the marginal distribution of Y found in Eample 6, p X Y (1 2) = p(1,2) p Y (2) = 3/36 1/3 = = 9 36 = 1 4 p X Y (2 2) = p(2,2) p Y (2) = 4/36 1/3 = = = 1 3 p X Y (3 2) = p(3,2) p Y (2) = 5/36 1/3 = = = 5 12 Conditional Probability Density Function Given continuous random variables X and Y with joint probability density function f(, y), the conditional probability density function of Y given X = is f Y X (y ) = f(, y) f X () for f X () > 0 The conditional probability density function provides the conditional probabilities for the values of Y given X =. STA 3032 Ch. 5 Notes 9

10 Because the conditional probability density function f Y X (y) is a probability density function for all y in R, the following properties are satisfied: 1) f Y X (y ) 0 2) f Y X (y )dy = 1 3) P(Y B X = ) = f Y X (y )dy B for any set B in the range of Y Eample 8 Using the joint pdf from Eamples 4 and 5, f(, y) = 10e 2 3y for > 0 and 0 < y <, find the following: a) Conditional probability distribution of Y given X = 1 Recall that, in Eample 5, we found f X () = 10 3 (e 2 e 5 ) f Y X (y 1) = f(1, y) f X (1) = = 3.157e 3y 10e 2 3y = 10 3 (e 2 e 5 ) e 2 3y 1 3 (e 2 e 5 ) b) Conditional probability distribution of X given Y = 2 First we need to find the marginal distribution of Y. f Y (y) = 10 e 2 3y d = f X Y ( 2) = y = 5e 3y (0 + e 2y ) = 5e 5y = e 2 e 3y 1 3 (e 2 e 5 ) 10e 3y 1 2 e 2 = 5e 3y ( e e 2y ) y f(, 2) f X (2) = 10e 2 6 5e 10 = 10e 6 e 2 5e 10 = 2e 6 e 10 e 2 = e 2 STA 3032 Ch. 5 Notes 10

11 5.2 Epected Values, Covariance, and Correlation Mean and Variance of Discrete Random Variables Recall that for a discrete random variable X with pmf, f(), E(X) = p(). Similarly, for a discrete random variable X with a joint probability mass function,p(, y), the epected value of X can be determined by the following: where R is the range of (X, Y). E(X) = μ X = p(, y) Recall that for a discrete random variable X with pmf, f(), σ 2 = ( μ) 2 p(). Similarly, for a discrete random variable X with a joint probability mass function,p(, y), the variance of X can be determined by the following: where R is the range of (X, Y). σ X 2 = ( μ ) 2 p(, y) Similarly, the formulas for the epected value and variance of Y are as follows: R E(Y) = μ Y = y p(, y) σ Y 2 = (y μ Y ) 2 p(, y) R R R Eample 9 Using the joint pmf, p(, y) = 1 ( + y) for = 1, 2, 3 and y = 1, 2, 3, find the epected 36 value and variance of X. E(X) = 1 p(1,1) + 1 p(1,2) + 1 p(1,3) + 2 p(2,1) + 2 p(2,2) + 2 p(2,3) +3 p(3,1) + 3 p(3,2) + 3 p(3,3) = = = = = 13 6 = STA 3032 Ch. 5 Notes 11

12 Eample 9 Continued σ X 2 = p(1,1) p(1,2) p(1,3) p(2,1) p(2,2) p(2,3) p(3,1) p(3,2) p(3,3) = = = = = Mean and Variance of Continuous Random Variables For a continuous random variable X with a joint probability mass function,f(, y), the epected value and variance of X can be determined by the following: E(X) = μ X = f(, y)ddy R σ X 2 = ( μ X ) 2 f(, y)ddy R where R is the range of (X, Y). Similarly, the formulas for the epected value and variance of Y are as follows: E(Y) = μ Y = yf(, y)ddy R where R is the range of (X, Y). σ Y 2 = (y μ Y ) 2 f(, y)ddy R STA 3032 Ch. 5 Notes 12

13 Epected Value of a Function of Two Random Variables Let X and Y be random variables and h(x, Y) be a function of those random variables. h(, y) p(, y) for discrete X and Y y E[h(X, Y)] = h(, y) f(, y)ddy for continuous X and Y y Covariance In eperiments involving two or more random variables, we often are interested in measuring the relationship between the variables. A common measurement to describe the variation between two variables is covariance. The covariance between the random variables X and Y, denoted as cov(x, Y) or σ XY, is Cov(X, Y) = E[(X μ X )(Y μ Y )] = E(XY) μ X μ Y If the covariance is negative, then X and Y have an inverse relationship i.e. as X increases Y decreases or vice versa. However, if X increases as Y increases, their covariance will be positive. Thus, covariance is a measure of the linear relationship between the random variables. If the relationship between the variables is nonlinear, the covariance might not detect the relationship. Correlation Another measure of the variance between two random variables is correlation. Correlation is often easier to interpret that covariance because it is scaled by the standard deviation of each variable. This allows the correlation of two variables to be comparable to the correlation of a different set of two variables. The correlation between random variables X and Y, denoted as ρ XY, is where 1 ρ XY 1 Cov(X, Y) ρ XY = V(X)V(Y) = Cov(X, Y) σ X σ Y Please read the two propositions on pg. 210 of the tetbook. STA 3032 Ch. 5 Notes 13

14 Eample 10 Determine the covariance and correlation for the following joint probability distribution: y p(, y) 1/8 1/4 1/2 1/8 Before we can calculate the covariance, we need to find the epected values of X and Y. μ X = p(, y) = ( 1) ( 0.5) (0.5) (1) 1 8 = = 1 8 = μ Y = y p(, y) = ( 2) ( 1) (1) (2) 1 8 = = 1 4 = 0.25 y Now we need to find the epected value of XY. E(XY) = y p(, y) = y = = 7 8 = We have all the values necessary to find the covariance, Cov(, y). Cov(, y) = E(XY) μ X μ Y = (0.125)(0.25) = = Before we can find the correlation, we need to calculate the variances of X and Y. σ 2 X = 2 p(, y) μ 2 X = ( 1) ( 0.5) (0.5) (1) = = = = σ 2 Y = y 2 p(, y) μ 2 Y = ( 2) ( 1) (1) (2) y = = = = Now we can find the correlation. STA 3032 Ch. 5 Notes 14

15 Eample 10 Continued ρ XY = Cov(, y) = σ X σ Y = = 1 Thus, X and Y are correlated and have a positive linear relationship. 5.3 Statistics and Their Distributions Please skip this section. 5.4 The Distribution of the Sample Mean The assumption that distributions and population parameters are known is unrealistic. In statistics, we use sample data to estimate distributions and parameters. Definitions 1. parameter number that describes the population (i.e. μ, σ 2, σ, p (binomial)) 2. statistic function of the observations in a random sample, number that describes the sample (i.e., s 2, s) 3. population distribution distribution from which we select our sample, parameters are usually unknown 4. sample distribution distribution of the data we actually observe, describe the data with statistics, such as the sample mean ( ); the larger the sample size (n), then the closer the sample represents the population 5. sampling distribution a probability distribution of a sample statistic, such as, when samples of size n are taken repeatedly; describes the variability of the statistic; generated by repeating a sampling eperiment a very large number of times 6. point estimate a single number calculated from sample data, used to estimate a parameter of the population 7. point estimator formula or rule that is used to calculate the point estimate for a particular set of data (i.e. sample statistics) STA 3032 Ch. 5 Notes 15

16 Common Point Estimators Population Parameter Point Estimator or Sample Statistic Mean μ Variance σ 2 s 2 Standard Deviation σ s Binomial Proportion p p (read as p-hat) We have many different choices for the point estimator of a parameter. For eample, if we wish to estimate the population mean, we might consider the sample mean or sample median as point estimators. In order to decide which point estimator of a particular parameter is the best one to use, we need to eamine their statistical properties and develop some criteria for comparing estimators. The Case of a Normal Population Distribution Please read this section in the book. Central Limit Theorem (CLT) Consider a random sample of n observations selected from a population of any shape mean, μ, and standard deviation, σ. Then, when n is sufficiently large (n 30), the sampling distribution of X will be approimately normal with a mean, μ X, and standard deviation, σ X, given by μ X = μ σ X = σ n Note: If the population is normal, is normal for any size n. Eample 11 A sample of 4 observations is taken from a normal distribution with μ = 50 and σ = 6. Find the sampling distribution of X. The CLT gives us three properties we must find to determine the sampling distribution of X. 1. Is the distribution normal? (Note: This will be true if the population is normal or if the sample size is greater than or equal to 30.) 2. μ X = μ 3. σ X = σ n Let s find each of the properties. STA 3032 Ch. 5 Notes 16

17 Eample 11 Continued For the first property, we know that the distribution is normal because the sample is taken from a normal distribution. Notice that if the population was not normal, the sampling distribution would not be normal because the sample size is smaller than 30. The mean of the sampling distribution of X is equal to the mean of the population. μ X = μ = 50 Finally, the standard deviation of the sampling distribution of X is the population standard deviation divided by the square root of the sample size. σ X = σ n = 6 4 = 3 Note: I provided etra eplanation in this eample since it is the first problem of this type that we covered. The remaining eamples will have shorter answers that are similar to what I epect you to provide on your assignments. Eample 12 A sample of 100 observations is taken from a Binomial distribution with n = 25 and p = 0.8. Determine the sampling distribution of the sample mean. Note: I will denote the sample size as N to distinguish it from the number of trials, n. Since this sample is from a Binomial distribution, we will need to use some formulas from Ch. 3 to find the mean and standard deviation. 1. Normal? Yes, because N = μ X = μ = np = 25(0.8) = σ X = σ N np(1 p) = N = 25(0.8)(0.2) 100 = 4 10 = 0.2 STA 3032 Ch. 5 Notes 17

18 Eample 13 A sample of 2 observations is taken from a uniform distribution over the interval 0 to 1. What is the sampling distribution of the sample mean? Note: Since the sample is from a uniform distribution, we will need to use some formulas from Ch. 4 to find the mean and standard deviation. 1. Normal? No, n = 2 < μ X = μ a + b = 2 = = σ X = σ n = b a 12 n = = IMPORTANT: Notice in Eample 13, that μ X = μ and σ X = σ/ n still hold true even though the sampling distribution of X is NOT normal. Finding P(X ) If is normal, use the z-table with the following z-score: z = μ σ μ = σ/ n Eample 14 Let X = ages of Sarasota residents, where μ is claimed to be 60 and σ = 16. Suppose we have a random sample of 64 residents. Find P(X < 55). First, we must find the sampling distribution of. We know the distribution is normal because n = 64 > 30. Thus, we can use the z-table to find the probability of. We need to calculate the z-score of 55. z = / 64 = 5 2 = 2.5 STA 3032 Ch. 5 Notes 18

19 Eample 14 Continued Therefore, P(X < 55) = P(Z < 2.5). Now, draw the region that we re interested in and label it with the probabilities from the z-table. P(X < 55) = P(Z < 2.5) = Eample 15 Suppose test scores are approimately normal with a mean of 72 and a standard deviation of 12. If 16 scores are randomly selected, what is the probability that the average score is higher than 70? In this case, the sample size is less than 30, but since we are sampling from a normal distribution, the sampling distribution of is normal. P(X > 70) = P μ > σ 12/ 16 = P Z > 2 = P(Z > 0.67) 3 Now, draw the region we re interested in and label it with the probabilities from the z-table. P(X > 70) = P(Z > 0.67) = 1 P(Z < 0.67) = = STA 3032 Ch. 5 Notes 19

20 IMPORTANT: Notice that Eample 15 asked for the probability of the average, thus we needed to find the sampling distribution of. If only 1 score had been randomly selected and it asked for the probability that the score was less than or greater than some value, we would have used the z-score we learned about in Ch. 4. Eample 16 The distribution of violent crimes per day in a certain city possesses a mean of 1.3 and a standard deviation of 1.7. A random sample of 50 days is observed and the daily mean number of crimes is found to be 0.9 with a standard deviation of 1.4. Find the sampling distribution of the sample mean. In this case, we are given several bits of information in the problem, including information we don t need to find the sampling distribution of X. The first sentence is describing the distribution of the population because there is no mention of sample being taken or a sample size. Thus, the mean and standard deviation presented in this sentence is for the population. μ = 1.3 σ = 1.7 The second sentence is describing a sample that was taken. The mean and standard deviation given in this sentence were found from the sample. Thus, these values are the sample mean and sample standard deviation. = 0.9 s = 1.4 Now that we ve identified all the values given in the problem, we can find the sampling distribution. 1. Normal? Yes, because n = 50 > μ = μ = σ = σ n = = = 0.24 Other Applications of the Central Limit Theorem Please skip this section. 5.5 The Distribution of a Linear Combination Please skip this section. STA 3032 Ch. 5 Notes 20