Conditional Probability

Size: px

Start display at page:

Download "Conditional Probability"

Maximillian Morton
5 years ago
Views:

1 Conditional Probability

2 Conditional Probability The Law of Total Probability Let A 1, A 2,..., A k be mutually exclusive and exhaustive events. Then for any other event B, P(B) = P(B A 1 ) P(A 1 ) + P(B A 2 ) P(A 2 ) + + P(B A k ) P(A k ) k = P(B A i ) P(A i ) i=1 where exhaustive means A 1 A 2 A k = S.

3 Conditional Probability

4 Conditional Probability Bayes Theorem Let A 1, A 2,..., A k be a collection of k mutually exclusive and exhaustive events with prior probabilities P(A i )(i = 1, 2,..., k). Then for any other event B with P(B) > 0, the posterior probability of A j given that B has occurred is P(A j B) = P(A j B) P(B) = P(B A j ) P(A j ) k i=1 P(B A i) P(A i ) j = 1, 2,... k

5 Independence

6 Independence Definition Two events A and B are independent if P(A B) = P(A), and are dependent otherwise.

7 Independence Definition Two events A and B are independent if P(A B) = P(A), and are dependent otherwise.

8 Independence

9 Independence The Multiplication Rule for Independent Events Proposition Events A and B are independent if and only if P(A B) = P(A) P(B)

10 Independence

11 Independence Independence of More Than Two Events Definition Events A 1, A 2,..., A n are mutually independent if for every k (k = 2, 3,..., n) and every subset of indices i 1, i 2,..., i k, P(A i1 A i2 A ik ) = P(A ii ) P(A i2 ) P(A ik ).

12 Random Variables

13 Random Variables Definition For a given sample sample space S of some experiment, a random variable (rv) is any rule that associates a number with each outcome in S. In mathematical language, a random variable is a function whose domain is the sample space and whose range is the set of real numbers.

14 Random Variables Definition For a given sample sample space S of some experiment, a random variable (rv) is any rule that associates a number with each outcome in S. In mathematical language, a random variable is a function whose domain is the sample space and whose range is the set of real numbers. We use uppercase letters, such as X and Y to, denote random variables and use lowercase letters, such as x and y, to denote some particular value of the corresponding random variable. For example, X (s) = x means that value x is associated with the oucome s by the rv X.

Random Variables Definition For a given sample sample space S of some experiment, a random variable (rv) is any rule that associates a number with each outcome in S.

15 Random Variables Definition For a given sample sample space S of some experiment, a random variable (rv) is any rule that associates a number with each outcome in S. In mathematical language, a random variable is a function whose domain is the sample space and whose range is the set of real numbers. We use uppercase letters, such as X and Y to, denote random variables and use lowercase letters, such as x and y, to denote some particular value of the corresponding random variable. For example, X (s) = x means that value x is associated with the oucome s by the rv X.

16 Random Variables

17 Random Variables Examples:

18 Random Variables Examples: 1. Assume we toss a coin. Then S = {H, T}. We can define a rv X by X (H) = 1 and X (T) = 0

19 Random Variables Examples: 1. Assume we toss a coin. Then S = {H, T}. We can define a rv X by X (H) = 1 and X (T) = 0 2. A techincian is going to check the quality of 10 prodcuts. For each product the outcome is either successful (S) or defective (D). Then we can define a rv Y by { 1, successful Y = 0, defective

20 Random Variables Examples: 1. Assume we toss a coin. Then S = {H, T}. We can define a rv X by X (H) = 1 and X (T) = 0 2. A techincian is going to check the quality of 10 prodcuts. For each product the outcome is either successful (S) or defective (D). Then we can define a rv Y by { 1, successful Y = 0, defective Definition Any random variable whose only possible values are 0 and 1 is called a Bernoulli random variable.

21 Random Variables

22 Random Variables More examples: 3. (Example 3.3) We are investigating two gas stations. Each has six gas pumps. Consider the experiment in which the number of pumps in use at a particular time of day is determined for each of the stations. Define rv s X, Y and U by X = the total number of pumps in use at the two stations Y = the difference between the number of pumps in use at station 1 and the number in use at station 2 U = the maximum of the numbers of pumps in use at the two station

23 Random Variables More examples: 3. (Example 3.3) We are investigating two gas stations. Each has six gas pumps. Consider the experiment in which the number of pumps in use at a particular time of day is determined for each of the stations. Define rv s X, Y and U by X = the total number of pumps in use at the two stations Y = the difference between the number of pumps in use at station 1 and the number in use at station 2 U = the maximum of the numbers of pumps in use at the two station If this experiment is performed and s = (3, 4) results, then X ((3, 4)) = = 7, so we say that the observed value of X was x = 7. Similarly, the observed value of Y would be y = 3 4 = 1, and the observed value of U would be u = max(3, 4) = 4.

24 Random Variables

25 Random Variables More examples: 4. Assume we toss a coin until we get a Head. Then the sample space would be S = {H, TH, TTH, TTTH,... } If we define a rv X by X X = the number we totally tossed Then X ({H}) = 1, X ({TH}) = 2, X ({TTH}) = 3,..., and so on.

26 Random Variables More examples: 4. Assume we toss a coin until we get a Head. Then the sample space would be S = {H, TH, TTH, TTTH,... } If we define a rv X by X X = the number we totally tossed Then X ({H}) = 1, X ({TH}) = 2, X ({TTH}) = 3,..., and so on. In this case, the random variable X can be any positive integer, which in all is infinite.

27 Random Variables More examples: 4. Assume we toss a coin until we get a Head. Then the sample space would be S = {H, TH, TTH, TTTH,... } If we define a rv X by X X = the number we totally tossed Then X ({H}) = 1, X ({TH}) = 2, X ({TTH}) = 3,..., and so on. In this case, the random variable X can be any positive integer, which in all is infinite. 5. Assume we are going to measure the length of 100 desks. Define the rv Y by Y = the length of a particular desk Y can also assume infinitly possible values.

28 Random Variables

29 Random Variables Definition A dicrete random variable is an rv whose possible values either constitute a finite set or else can be listed in an infinite sequence in which there is a first element, a second element, and so on ( countably infinite).

30 Random Variables Definition A dicrete random variable is an rv whose possible values either constitute a finite set or else can be listed in an infinite sequence in which there is a first element, a second element, and so on ( countably infinite). A random variable is continuous if both of the following apply:

31 Random Variables Definition A dicrete random variable is an rv whose possible values either constitute a finite set or else can be listed in an infinite sequence in which there is a first element, a second element, and so on ( countably infinite). A random variable is continuous if both of the following apply: 1. Its set of possible values consists either of all numbers in a single interval on the number line (possibly infinite in extent, e.g., (, ) ) or all numbers in a disjoint union of such intervals (e.g., [0, 10] [20, 30]).

32 Random Variables Definition A dicrete random variable is an rv whose possible values either constitute a finite set or else can be listed in an infinite sequence in which there is a first element, a second element, and so on ( countably infinite). A random variable is continuous if both of the following apply: 1. Its set of possible values consists either of all numbers in a single interval on the number line (possibly infinite in extent, e.g., (, ) ) or all numbers in a disjoint union of such intervals (e.g., [0, 10] [20, 30]). 2. No possible value of the variable has positive probability, that is, P(X = c) = 0 for any possible value c. Examples

34 An example: Assume we toss a coin 3 times and record the outcomes. Let X i be a random variable defined by { 1, if the i th outcome is Head; X i = 0, if the i th outcome is Tail; Let X be the random variable such that X = X 1 + X 2 + X 3, then X represents the total number of Heads we could get from the experiment.

35 An example: Assume we toss a coin 3 times and record the outcomes. Let X i be a random variable defined by { 1, if the i th outcome is Head; X i = 0, if the i th outcome is Tail; Let X be the random variable such that X = X 1 + X 2 + X 3, then X represents the total number of Heads we could get from the experiment. If the probability for getting a Head for each toss is 0.7, then the probabilities for all the outcomes are tabulated as following: s HHH HHT HTH HTT THH THT TTH TTT x p(x)

37 Example continued: s HHH HHT HTH HTT THH THT TTH TTT x p(x)

38 Example continued: s HHH HHT HTH HTT THH THT TTH TTT x p(x) We can re-tabulate it only for the x values: x p(x)

39 Example continued: s HHH HHT HTH HTT THH THT TTH TTT x p(x) We can re-tabulate it only for the x values: x p(x) Now we can answer various questions.

40 Example continued: s HHH HHT HTH HTT THH THT TTH TTT x p(x) We can re-tabulate it only for the x values: x p(x) Now we can answer various questions. The probability that there are at most 2 Heads is P(X 2) = P(x = 0 or 1 or 2) = p(0) + p(1) + p(2) = 0.657

41 Example continued: s HHH HHT HTH HTT THH THT TTH TTT x p(x) We can re-tabulate it only for the x values: x p(x) Now we can answer various questions. The probability that there are at most 2 Heads is P(X 2) = P(x = 0 or 1 or 2) = p(0) + p(1) + p(2) = The probability that the number of Heads are is strictly between 1 and 3 is P(1 < X < 3) = P(X = 2) = p(2) = 0.441

43 Definition The probability distribution or probability mass function (pmf) of a discrete rv is defined for every number x by p(x) = P(X = x) = P(all s S : X (s) = x).

44 Definition The probability distribution or probability mass function (pmf) of a discrete rv is defined for every number x by p(x) = P(X = x) = P(all s S : X (s) = x). In words, for every possible value x of the random variable, the pmf specifies the probability of observing that value when the experiment is performed. (The conditions p(x) 0 and all possible x p(x) = 1 are required for any pmf.)

46 Example 3.8 Six lots of components are ready to be shipped by a certain supplier. The number of defective components in each lot is as follows: Lot Number of defectives One of these lots is to be randomly selected for shipment to a particular customer. Let X be the number of defectives in the selected lot.

47 Example 3.8 Six lots of components are ready to be shipped by a certain supplier. The number of defective components in each lot is as follows: Lot Number of defectives One of these lots is to be randomly selected for shipment to a particular customer. Let X be the number of defectives in the selected lot. The three possible X values are 0, 1 and 2. The pmf for X is p(0) = P(X = 0) = P(lot 1 or 3 or 6 is selected) = 3 6 = p(1) = P(X = 1) = P(lot 4 is selected) = 1 6 = p(2) = P(X = 2) = P(lot 2 or 5 is selected) = 2 6 = 0.333

49 Example 3.10: Consider a group of five potential blood donors a, b, c, d, and e of whom only a and b have type O+ blood. Five blood smaples, one from each individual, will be typed in random order until an O+ individual is identified. Let the rv Y = the number of typings necessary to identify an O+ individual. Then what is the pmf of Y?

51 Example: Consider whether the next customer coming to a certain gas station buys gasoline or diesel. Let { 1, if the customer purchases gasoline X = 0, if the customer purchases diesel If 30% of all customers in one month purchase diesel, then the pmf for X is p(0) = P(X = 0) = P(nextcustomerbuysdiesel) = 0.3 p(1) = P(X = 1) = P(nextcustomerbuysgasoline) = 0.7 p(x) = P(X = x) = 0 for x 0 or 1

53 Example: Consider whether the next customer coming to a certain gas station buys gasoline or diesel. Let { 1, if the customer purchases gasoline X = 0, if the customer purchases diesel If 100α% of all customers in one month purchase diesel, then the pmf for X is p(0) = P(X = 0) = P(nextcustomerbuysdiesel) = α p(1) = P(X = 1) = P(nextcustomerbuysgasoline) = 1 α p(x) = P(X = x) = 0 for x 0 or 1 here α is between 0 and 1.

55 Definition Suppose p(x) depends on a quantity that can be assigned any one of a number of possible values, with each different value determining a different probability distribution. Such a quantity is called a parameter of the distribution. The collection of all probability distributions for different values of the parameter is called a family of probability distribution.

56 Definition Suppose p(x) depends on a quantity that can be assigned any one of a number of possible values, with each different value determining a different probability distribution. Such a quantity is called a parameter of the distribution. The collection of all probability distributions for different values of the parameter is called a family of probability distribution. For the previous example, the quantity α is a parameter. Each different value of α between 0 and 1 determines a different member of a family of distributions; two such members are 0.3 if x = 0 p(x) = 0.7 if x = 1 0 otherwise 0.25 if x = 0 p(x) = 0.75 if x = 1 0 otherwise

58 Example: Assume we are drawing cards from a 100 well-shuffled cards with replacement. We keep drawing until we get a. Let p = P({ }), i.e. there are 100 p s. Assume the successive drawings are independent and define X = the number of drawings. Then p(1) = P(X = 1) = P({ }) = p p(2) = P(X = 2) = P({ }) = (1 p) p p(3) = P(X = 3) = P({ }) = (1 p) (1 p) p...

59 Example: Assume we are drawing cards from a 100 well-shuffled cards with replacement. We keep drawing until we get a. Let p = P({ }), i.e. there are 100 p s. Assume the successive drawings are independent and define X = the number of drawings. Then p(1) = P(X = 1) = P({ }) = p p(2) = P(X = 2) = P({ }) = (1 p) p p(3) = P(X = 3) = P({ }) = (1 p) (1 p) p... A general formula would be { (1 p) x 1 p x = 1, 2, 3,... p(x) = 0 otherwise

61 Example: Assume we are drawing cards from a 100 well-shuffled cards with replacement. We keep drawing until we get a. Let p = P({ }), i.e. there are 100 p s. Assume the successive drawings are independent and define X = the number of drawings.

62 Example: Assume we are drawing cards from a 100 well-shuffled cards with replacement. We keep drawing until we get a. Let p = P({ }), i.e. there are 100 p s. Assume the successive drawings are independent and define X = the number of drawings. If we know that there are 20 s, i.e. p = 0.2, then what is the probability for us to draw at most 3 times? More than 2 times?

63 Example: Assume we are drawing cards from a 100 well-shuffled cards with replacement. We keep drawing until we get a. Let p = P({ }), i.e. there are 100 p s. Assume the successive drawings are independent and define X = the number of drawings. If we know that there are 20 s, i.e. p = 0.2, then what is the probability for us to draw at most 3 times? More than 2 times? P(X 3) = p(1)+p(2)+p(3) = (0.8) 2 = 0.488

64 Example: Assume we are drawing cards from a 100 well-shuffled cards with replacement. We keep drawing until we get a. Let p = P({ }), i.e. there are 100 p s. Assume the successive drawings are independent and define X = the number of drawings. If we know that there are 20 s, i.e. p = 0.2, then what is the probability for us to draw at most 3 times? More than 2 times? P(X 3) = p(1)+p(2)+p(3) = (0.8) 2 = P(X > 2) = p(3)+p(4)+p(5)+ = 1 p(1) p(2) = = 0

66 Definition The cumulative distribution function (cdf) F (x) of a discrete rv X with pmf p(x) is defined for every number x by F (x) = P(X x) = y:y x p(y) For any number x, F(x) is the probability that the observed value of X will be at most x.

67 Definition The cumulative distribution function (cdf) F (x) of a discrete rv X with pmf p(x) is defined for every number x by F (x) = P(X x) = y:y x p(y) For any number x, F(x) is the probability that the observed value of X will be at most x. F (x) = P(X x) = P(X is less than or equal to x) p(x) = P(X = x) = P(X is exactly equal to x)

69 Example 3.10 (continued): 0 if y < if 1 y < 2 F (y) = 0.7 if 2 y < if 3 y < 4 1 if y 2

70 Example 3.10 (continued): 0 if y < if 1 y < 2 F (y) = 0.7 if 2 y < if 3 y < 4 1 if y 2

72 Example: Assume we are drawing cards from a 100 well-shuffled cards with replacement. We keep drawing until we get a. Let α = P({ }), i.e. there are 100 α s. Assume the successive drawings are independent and define X = the number of drawings. The pmf would be { (1 α) x 1 α x = 1, 2, 3,... p(x) = 0 otherwise

73 Example: Assume we are drawing cards from a 100 well-shuffled cards with replacement. We keep drawing until we get a. Let α = P({ }), i.e. there are 100 α s. Assume the successive drawings are independent and define X = the number of drawings. The pmf would be { (1 α) x 1 α x = 1, 2, 3,... p(x) = 0 otherwise Then for any positive interger x, we have F (x) = y x p(y) = = x x 1 (1 α) (y 1) α = α (1 α) y y=1 { 1 (1 α) x x 1 0 x < 1 y=0

75 Example: Assume we are drawing cards from a 100 well-shuffled cards with replacement. We keep drawing until we get a. Let α = P({ }), i.e. there are 100 α s. Assume the successive drawings are independent and define X = the number of drawings. The pmf would be

77 pmf = cdf: F (x) = P(X x) = p(y) y:y x

78 pmf = cdf: F (x) = P(X x) = p(y) It is also possible cdf = pmf: y:y x

79 pmf = cdf: F (x) = P(X x) = It is also possible cdf = pmf: y:y x p(x) = F (x) F (x ) p(y) where x represents the largest possible X value that is strictly less than x.

81 Proposition For any two numbers a and b with a b, P(a X b) = F (b) F (a ) where a represents the largest possible X value that is strictly less than a. In particular, if the only possible values are integers and if a and b are integers, then P(a X b) = P(X = a or a + 1 or... or b) = F (b) F (a 1) Taking a = b yields P(X = a) = F (a) F (a 1) in this case.

83 Example (Problem 23): A consumer organization that evaluates new automobiles customarily reports the number of major defects in each car examined. Let X denote the number of major defects in a randomly selected car of a certain type. The cdf of X is as follows: 0 x < x < x < x < 3 F (x) = x < x < x < 6 1 x 6 Calculate the following probabilities directly from the cdf: (a)p(2), (b)p(x > 3) and (c)p(2 X < 5).

Applied Statistics I

Applied Statistics I Liang Zhang Department of Mathematics, University of Utah June 17, 2008 Liang Zhang (UofU) Applied Statistics I June 17, 2008 1 / 22 Random Variables Definition A dicrete random variable