Chapter 2: Probability

Transcription

1 Chapter 2: Probability Professor Ron Fricker Naval Postgraduate School Monterey, California 3/15/15 Reading Assignment: Sections

2 Goals for this Chapter Introduction to probability Methods for calculating probabilities Sample-point method Event-composition method Counting tools Combinations, permutations, etc. Conditional probability and independence Probability laws and rules Multiplicative and additive laws Law of Total Probability and Bayes Rule Random variables A bit about random sampling 3/15/15 2

3 Sections 2.1 and 2.2: Defining Probability In everyday conversation, probability is a measure of one s belief in the occurrence of a future event What s the chance that aliens will visit the Earth in my lifetime? In this sense, probability is completely subjective More quantitatively, we define probability as the fraction of times an event occurs if it s repeated over and over (infinitely) This is the relative frequency definition of probability 3/15/15 3

4 Probability Quantifies Outcome Uncertainty Think about someone flipping a coin With enough practice, can learn to flip a coin and have it come up heads every time In this case, there s no uncertainty in the outcome But without practice, the coin is going to come up heads and tails randomly There is randomness because the particular outcomes are uncontrollable and/or unpredictable Probability helps you quantify the likelihood of various outcomes 3/15/15 4

5 Examples You go to Las Vegas and gamble on your favorite game: slots, blackjack, etc. The outcomes are random, but the fraction of each outcome is predictable in the long term Typical textbook examples: coins, dice & cards Probability the flip of a (fair) coin comes up heads Probability that in 10 (fair) coin flips you get 8 or more heads Probability of drawing the ace of spades from a (randomly shuffled) standard deck of cards Probability that you draw a full house in your next poker hand 3/15/15 5

6 More Realistic Example What s the probability that a 9mm bullet penetrates an Advanced Combat Helmet during a FAT? Here probability of penetration is a function of two types of randomness Not every helmet is exactly the same, so the chance of penetration depends on the helmet There is variation in helmet construction and randomness in which helmet gets tested Test conditions are controlled, but nothing is controlled perfectly There is at least some variation in the test conditions which can affect the outcome 3/15/15 6

7 Probability and Inference In statistics, inference is using a sample of data to say something about an entire population In OA3102, OA3103, and OA4106 you will learn formal methods for doing inference Hypothesis testing Model fitting But first, you need to have to understand the theory of probability Both what it is and, perhaps more importantly, how to calculate 3/15/15 7

8 Probability vs. Statistics What s the difference between probability and statistics (i.e., inference)? In probability, the randomization mechanism is assumed/known and calculations follow E.g., Given a fair coin, what s the chance of observing 8 heads out of 10 coin flips? In statistics, the outcome is known and used to infer something about the mechanism E.g., Given that I ve just observed 8 heads out of 10 coin flips, is the coin fair? 3/15/15 8

9 Example: Helmet Testing Probability question: Given certain helmet and testing process assumptions, what the probability that a manufacturer will pass the First Article Test? Inference question: Given the results of a First Article Test, what s the likelihood that the manufacturer is producing high-quality helmets? 3/15/15 9

10 Section 2.3: Set Terminology and Notation (1) To begin, we need some concepts and notation from set theory Capital letters denote sets of points (or events): A, B, C, etc. If the set A consists of points a 1, a 2, and a 3, we write A = {a 1,a 2,a 3 } S is the universal set (i.e., it contains everything in the problem universe) ; ; = {} is the null or empty set, so 3/15/15 10

11 Set Terminology and Notation (2) For any two sets A and B, A is a subset of B if every point in A is in B Example: If B = {a 1,a 2,a 3,a 4 } and A = {a 1,a 2,a 3 } then A is a subset of B The notation is A B The null set is a subset of every set The union of A and B is the set of all points in A or B or both The notation is A [ B Example: If A = {a 1,a 2 } and B = {b 1,b 2 } then A [ B = {a 1,a 2,b 1,b 2 } 3/15/15 11

12 Venn Diagrams 3/15/15 12 Source: Wackerly, D.D., W.M. Mendenhall III, and R.L. Scheaffer (2008). Mathematical Statistics with Applications, 7 th edition, Thomson Brooks/Cole.

13 Set Terminology and Notation (3) The intersection of sets A and B is the set of all points in both A and B A \ B AB The notation is or just Example: If B = {a 1,a 2,a 3,a 4 } and A = {a 1,a 2,a 3 } then A \ B = {a 1,a 2,a 3 } The key word for union is or while the key word for intersection is and 3/15/15 13 Source: Wackerly, D.D., W.M. Mendenhall III, and R.L. Scheaffer (2008). Mathematical Statistics with Applications, 7 th edition, Thomson Brooks/Cole.

14 Set Terminology and Notation (4) If A is a subset of S, then the complement of A is denoted A The complement of A is the set of all points in S that are not in A Note that A [ A = S 3/15/15 14 Source: Wackerly, D.D., W.M. Mendenhall III, and R.L. Scheaffer (2008). Mathematical Statistics with Applications, 7 th edition, Thomson Brooks/Cole.

15 Set Terminology and Notation (5) Two sets A and B are said to be disjoint or mutually exclusive if they have no points in common That is, A and B are disjoint if A \ B = ; Note that, by definition, A \ A = ; 3/15/15 15 Source: Wackerly, D.D., W.M. Mendenhall III, and R.L. Scheaffer (2008). Mathematical Statistics with Applications, 7 th edition, Thomson Brooks/Cole.

16 Exercise Let S denote all the possible outcomes of throwing a six-sided die: S = {1, 2, 3, 4, 5, 6} Let A = {1, 2}, B = {1, 3}, and C = {2, 4, 6} Find: A [ B = A \ B = C = 3/15/15 16

17 Useful Set Equalities (that we won t prove) Distributive laws: A \ (B [ C) =(A \ B) [ (A \ C) A [ (B \ C) =(A [ B) \ (A [ C) DeMorgan s Laws: A \ B = A [ B A [ B = A \ B 3/15/15 17

18 Section 2.3 Homework Do problems 2.1, 2.2, 2.3, and 2.4 3/15/15 18

19 Section 2.4: Probability Terminology (1) Definition 2.1: An experiment is the process by which an observation is made Example #1: Shooting a 9mm bullet at an ACH Example #2: Simulating a battle in a computer using stochastic Lanchester equations Example #3: Rolling a six-sided die The outcome of an experiment is an event Example #1: The bullet penetrates the helmet Example #2: Red wins the battle Example #3: Observe an odd number 3/15/15 19

20 Probability Terminology (2) Simple events cannot be further decomposed, while compound events consist of two or more simple events Example: The event observe an odd outcome on the six-sided die experiment is a compound event It is composed of three simple events: observe a 1, observe a 3, and observe a 5 But observe a 1, observe a 3, and observe a 5 are all simple events since they cannot be decomposed any further 3/15/15 20

21 Probability Terminology (3) Definition 2.2: A simple event is an event that cannot be decomposed Each simple event corresponds to only one sample point The letter E with a subscript will be used to denote a simple event or the corresponding sample point Definition 2.3: The sample space associated with an experiment is the set consisting of all possible sample points The sample space is denoted by S 3/15/15 21

22 Example In the six-sided die tossing experiment, S = {E 1,E 2,E 3,E 4,E 5,E 6 } where for example, E 1 = roll a 1 The sample space for this experiment is Note that the sample space is discrete: the number of sample points is finite or countable 3/15/15 22 Source: Wackerly, D.D., W.M. Mendenhall III, and R.L. Scheaffer (2008). Mathematical Statistics with Applications, 7 th edition, Thomson Brooks/Cole.

23 Probability Terminology (4) Definition 2.4: A discrete sample space is one that contains either a finite or countable number of distinct sample points When an experiment is conducted, you will observe one and only one simple event All simple events are, by definition, mutually exclusive Die example: You cannot roll a 1 and a 2 at the same time Thus E 1 \ E 2 = ; 3/15/15 23

24 Probability Terminology (5) Definition 2.5: An event in a discrete sample space S is a collection of sample points that is, any subset of S So, the general term event can mean a simple, event, a compound event, or even a non-event (;) The rule for determining which simple events are part of a compound event is: A simple event E i is included in event A if and only if A occurs whenever E i occurs 3/15/15 24

25 Example Back to tossing the six-sided die Let E i denote the event roll the die and observe i Then, for example, the compound event A, observe an odd number, is A = {E 1,E 3,E 5 } We could also write A = E 1 [ E 3 [ E 5 Define event B to be observe a number less than 5 or B = {E 1,E 2,E 3,E 4 } The Venn diagram: 3/15/15 25 Source: Wackerly, D.D., W.M. Mendenhall III, and R.L. Scheaffer (2008). Mathematical Statistics with Applications, 7 th edition, Thomson Brooks/Cole.

26 Probability Axioms 1. The relative frequency of any event must be greater than or equal to zero Hence negative frequency does not make sense 2. The relative frequency of the whole sample space S must be 1 Since all possible experimental outcomes are in S, something in S must occur every time the experiment is conducted 3. If two events are mutually exclusive, then the probability of their union is the sum of their respective frequencies 3/15/15 26

27 Probability Axioms, Mathematically Definition 2.6: Suppose S is a sample space. To every event A in S, we assign a number P(A), called the probability of A such that: Axiom 1: P (A) 0 Axiom 2: P (S) =1 Axiom 3: If A 1, A 2, A 3, form a sequence of pairwise mutually exclusive events in S (that is, A i \ A j = ; if i 6= j), then P (A 1 [ A 2 [ A 3 [ )=P(A 1 )+P(A 2 )+P(A 3 ) 1X = P (A i ) i=1 3/15/15 27

28 Example: More Dice Tossing Consider a fair six-sided die Fair means that it s equally likely to come up on any of the die faces Let E i denote the event roll the die and observe i Then by Axioms 2 and 3: P (S) =P (E 1 [ E 2 [ E 3 [ E 4 [ E 5 [ E 6 ) = P (E 1 )+P (E 2 )+P (E 3 )+P (E 4 )+P (E 5 )+P (E 6 )=1 But since each simple event is equally likely, we have 6 P (E i )=1,i=1, 2,...,6 and so it must be that P (E i )=1/6,i=1, 2,...,6 3/15/15 28

29 Example Continued Note that with P (E i )=1/6,i=1, 2,...,6 the axioms hold in this case with Axiom 1: Axiom 2: P (S) = P (E i ) 6X i=1 E i 0, 8i =1/6+1/6+1/6+1/6+1/6+1/6 =1 Axiom 3: If, for example, event A is roll an odd number, then P (A) =P (E 1 )+P (E 3 )+P (E 5 ) =1/6+1/6+1/6 =1/2 3/15/15 29

30 Textbook Example 2.1 The Oakland Naval Supply Depot has 5 seemingly identical computers available to ship, but 2 of the 5 are defective. An order calls for 2 of the computers and is filled by randomly choosing 2 of the 5 available. List the sample space for this experiment. 3/15/15 30

31 Textbook Example 2.1 Let A denote the event that the order is filled with two non-defective computers. List the sample points in A. Construct a Venn diagram for the experiment that illustrates event A. 3/15/15 31

32 Textbook Example 2.1 Assign probabilities to the simple events in such a way that the information about the experiment is used and the Probability Axioms are met. Find the probability of event A. 3/15/15 32

33 Section 2.4 Homework Do problems 2.11, 2.12, 2.17, /15/15 33

34 Section 2.5: The Sample-Point Method With discrete sample spaces, two ways to find probabilities: sample-point and eventcomposition methods Methods very similar, both using the sample space ideas we ve been developing We ll start with sample-point now The idea is much like what we ve just done Then we ll come back to event-composition Requires some additional results that we ll come to shortly 3/15/15 34

35 The Sample-Point Method Steps 1. First, define the experiment and clearly determine how to describe one simple event. 2. List the simple events and test to make sure they cannot be decomposed. This defines S. 3. Assign reasonable probabilities to the sample X points in S making sure P (E i ) 0 and P (Ei )=1. 4. Define the event of interest, A, as a specific collection of sample points. 5. Find P(A) by summing the probabilities of the sample points in A. 3/15/15 35

36 Textbook Example 2.2 CENTCOM J-8 is selecting 2 out of 5 analyst applicants that vary in competence: analyst 1 is the best, 2 next, etc. Their competencies are unknown to CENTCOM. Define the following events: A: CENTCOM selects the best and one of the two poorest (i.e., select 1&4 or 1&5) B: CENTCOM selects at least one of the two best applicants Problem: Find the probabilities of these events. 3/15/15 36

37 Example 2.2 Solution Step 1: Step 2: 3/15/15 37

38 Example 2.2 Solution Step 3: Step 4: 3/15/15 38

39 Example 2.2 Solution Step 5: 3/15/15 39

40 Textbook Example 2.3 A balanced (aka fair) coin is tossed three times. Calculate the probability that exactly two of the three tosses results in a heads. Solution: Step 1: 3/15/15 40

41 Example 2.3 Solution Step 2: Step 3: 3/15/15 41

42 Example 2.3 Solution Step 4: Step 5: 3/15/15 42

43 Textbook Example 2.4 The odds are two to one that, when forces A and B fight a simulated battle, A wins. Suppose that the battle is simulated twice. What is the probability that force A wins at least one battle? Solution: Step 1: 3/15/15 43

44 Example 2.4 Solution Step 2: Step 3: 3/15/15 44

45 Example 2.4 Solution Step 4: Step 5: 3/15/15 45

46 Pros & Cons of Sample-Point Approach Sample-point approach is direct and powerful But it s a bulldozer or brute force approach Also, prone to human error Mistakes include mis-defining simple events and missing simple events For sample spaces with large number of events, can be challenge (perhaps impossible) to count all the events Leads us to learn about combinatorial analysis the theory of enumeration (aka as combinations and permutations) 3/15/15 46

47 Section 2.5 Homework Problems 2.26, 2.27, 2.30, /15/15 47

48 Section 2.6: Counting Sample Points So, let s look at some useful and powerful ways to count numbers of events Why? If: a sample space contains N equiprobable (i.e., equally likely) events. and an event A contains exactly n a sample points, then we have that P (A) = n a N So, if we know N and can count n a then we can calculate the probability of event A 3/15/15 48

49 The mn Rule Theorem 2.1: With m elements a 1, a 2,, a m and n elements b 1, b 2,, b n, it is possible to form mn = m x n pairs containing one element from each group. Proof. There are m x n squares in Figure 2.9, one for each a i,b j pair: 3/15/15 49 Source: Wackerly, D.D., W.M. Mendenhall III, and R.L. Scheaffer (2008). Mathematical Statistics with Applications, 7 th edition, Thomson Brooks/Cole.

50 Extending the mn Rule The mn rule extends to any number of sets E.g., for three sets of elements with m, n, and p sample points, there are m x n x p triplets Can prove this by applying Theorem 2.1 twice Example 2.6: I have three two-sided coins. How many distinct outcomes are there? It s 2 x 2 x 2 = 8 Brute force: {H,H,H}, {H,H,T}, {H,T,H}, {T,H,H}, {H,T,T}, {T,H,T}, {T,T,H}, {T,T,T} 3/15/15 50

51 Textbook Example 2.5 An experiment involves tossing a pair of (sixsided) dice and observing the numbers on the upper faces. Find the number of sample points in S. Solution: 3/15/15 51

52 Textbook Example 2.7 Consider an experiment that consists of recording the birthday for each of 20 randomly selected Marines. Assuming 365 days in a year, find the number of sample points in S. Solution: 3/15/15 52

53 Textbook Example 2.7 (continued) Now, assuming each of the sample points is equiprobable, what is the probability that each Marine in the 20 has a different birthday? Solution: 3/15/15 53

54 Permutations: Counting Ordered Arrangements Definition 2.7: An ordered arrangement of r distinct objects is called a permutation. The number of ways of ordering n distinct objects taken r at a time is denoted. Simple example: I have n = 3 types of missiles (A, B, and C) in my inventory and will shoot r = 2 of the three in series How many ways (permutations) are there to shoot two of the three missiles one after the other? P n r Via brute force, there are six orderings, {A,B}, {A,C}, {B,A}, {B,C}, {C,A}, {C,B}, so P2 3 =6 3/15/15 54

55 Calculating Permutations Theorem 2.2: To calculate the number of ways r items can be ordered out of n P n n! r = n(n 1)(n 2) (n r + 1) = (n r)! Remember Factorial notation: for a positive integer x, x! =x (x 1) (x 2) 2 1 With factorials, for integers x and y, with x < y, y! (y x)! = y (y 1) (y x + 1) (y x) (y x 1) 2 1 (y x) (y x 1) 2 1 = y (y 1) (y x + 1) Check the previous example: P 3 2 = 3! (3 2)! = /15/15 55 =6

56 Textbook Example 2.8 The names of three sailors are to be randomly drawn, without replacement, from a pool of 30 possible watchstanders. The first person drawn has the morning watch, the second the afternoon watch, and the third the evening watch. How many different orderings of 3 watchstanders are there? Solution: 3/15/15 56

57 Textbook Example 2.9 A repair depot assembly operation for the M1A1 tank requires 4 steps that can be performed in any order. If you want to compare the assembly time of all the different ways the operation can be performed, how many different assemblies will you have to time? Solution: 3/15/15 57

58 Counting Subsets of Objects Theorem 2.3: The number of ways of partitioning n distinct objects into k distinct groups containing n 1, n 2,..., n k objects, respectively, where each object appears in kx exactly one group and n i = n is: N = i=1 n! n 1!n 2! n k! n n 1 n 2 n k The term coefficient n n 1 n 2 n k is called the multinomial 3/15/15 58

59 Proof of Theorem 2.3 N is the number of ways of partitioning n objects into k groups where the order of the objects in the group does not matter Within the i th group, there are n i! ways of ordering the individual objects So, using the mn rule, Pn n = N n 1! n k! But we also know that Pn n n! = (n n)! = n! 0! = n! 1 = n! ü Hence, it follows that N = n! n 1!n 2! n k! 3/15/15 59

60 Textbook Example 2.10 You need to assign 20 aircraft to four sorties of size 6, 5, 5, and 4 aircraft. How many ways can the 20 aircraft be assigned to the four groups? Solution: 3/15/15 60

61 Textbook Example 2.10 (continued) Now, if the 20 aircraft consist of 4 F-18C Hornets and 16 F-18F Super Hornets, what s the chance that all four F-18Cs end up in the first sortie of 6 aircraft if each aircraft is randomly assigned to a sortie? Solution: 3/15/15 61

62 Textbook Example 2.10 (continued) 3/15/15 62

63 Combinations: Counting Unordered Arrangements Definition 2.8: The number of unordered subsets of r objects out of n is called the number of combinations That is, it is the number of subsets, of size r, that can be formed from n objects C n r The notation is or n r Simple example: I have n = 3 types of missiles (A, B, and C) in my inventory and I m going to pick r = 2 types to shoot How many combinations) of two types of three? Via brute force, there are only three, {A,B}, {A,C}, {B,C}, so C2 3 =3 3/15/15 63

64 Counting Combinations Theorem 2.4: The number of unordered subsets of size r chosen (without replacement) from n objects is n r n r C n r = P n r r! ü The term is generally referred as the binomial coefficient = n! r!(n r)! 3/15/15 64

65 Proof of Theorem 2.4 This is a special case of Theorem 2.3, where we re just partitioning the n objects into two groups of size r and n-r So, simply applying the formula, we have n N C n n! r = = rn r r!(n r)! ü The way to think about the difference between the formulas for Cr n and Pr n is that without ordering there are fewer combinations than permutations Hence the r! term in the denominator above 3/15/15 65

66 Textbook Example 2.11 Find the number of ways of selecting 2 CENTCOM analyst applicants out of 5? Note that this gives the solution to the number of sample points in S of Example 2.2 (without having to completely enumerate the sample space!) Solution: 3/15/15 66

67 Textbook Example 2.12 Let A denote the event that exactly one of the two best applicants (see Example 2.2) is in the selection of 2 out of 5. Find the number of sample points in A and P(A). Solution: 3/15/15 67

68 Textbook Example 2.13 A company randomly orders supplies from M distributors (so each distributor has an equal chance of getting any particular order). The manufacturer will place n orders (n < M). What s the probability that supplier i gets exactly k orders ( k apple n)? Solution: 3/15/15 68

69 Textbook Example 2.13 (continued) 3/15/15 69

70 Section 2.6 Homework Problems 2.35, 2.36, 2.39, 2.40, 2.43, 2.54, 2.55, /15/15 70

71 Section 2.7: Conditional Probability and Independence The idea of conditional probability is that the probability of an event depends on our knowledge of other events that have occurred Example: Imagine I m holding a card I picked at random out of a standard deck of playing. What s the probability it s the ace of hearts? It s 1/52 or just under 0.02 Now, if I tell you the card I m holding is an ace, what s the chance I m holding the ace of hearts? It s 1/4 (or 0.25 or 25%) since we know it has to be one of only 4 aces in the deck 3/15/15 71

72 Conditional Probability Definition 2.9: The conditional probability of an event A, given that an event B has occurred, is equal to P (A \ B) P (A B) =, P (B) provided P(B) > 0 The notation P (A B) is read the probability of A given B 3/15/15 72

73 Back to the Ace of Hearts Example Define some events: A = choose the ace of hearts and B = choose an ace (randomly out of 52 cards) Calculate the probabilities: Let s start with P(B) S consists of 52 sample points (distinct cards), all of which are equally likely to be chosen So, P(B)=4/52 (there are 4 aces in 52 cards) Now, P (A \ B) means the probability of choosing the ace of hearts and choosing an ace In this case, that s the same as the probability of choosing the ace of hearts So, Then P (A \ B) =P (A) =1/52 P (A B) = P (A \ B) P (B) = 1/52 4/52 = 1 4 3/15/15 73

74 Tying Conditional Probability Back to the Relative Frequency Notion of Probability Consider an experiment conducted N times in which each observation is either event A or not A and either event B or not B E.g., assessing radar performance: event A is the aircraft is a stealth aircraft and event B is that it is detected at a distance of more than 20nm The tabulated results: E.g., n 11 is the number of experiments for which both events A and B occurred (i.e., A \ B) 3/15/15 74

75 Example, continued Then, from the table for large N we have P (A) n 11 + n 21, P (A \ B) n 11, and N N P (B A) n 11 n 11 + n 21 And via Definition 2.9 we have P (A \ B) P (B A) = P (A) n 11 /N (n 11 + n 21 )/N = n 11 n 11 + n 21 3/15/15 75

76 Textbook Example 2.14 Suppose a balanced (aka fair) die is tossed once. What s the probability of getting a 1 given you re told that an odd number is showing? Solution: 3/15/15 76

77 Independence The idea of independent events is that the probability event A occurs is unaffected by whether B occurs or not Another way to think about it is that knowledge about event B does not affect your probabilistic assessment of A This is the intuitive notion that A and B are independent But, if knowledge of B tells you something about the probability of A, then the two events are dependent 3/15/15 77

78 Independent Events Definition 2.10: Two events A and B are said to be independent if any one of the following holds: P (A B) =P (A) P (B A) =P (B) P (A \ B) =P (A) P (B) Otherwise, the events are dependent Note that the third condition follows trivially from either of the first two conditions and the definition of conditional probability: 3/15/15 78

79 Textbook Example 2.15 Consider the following events in the toss of a single (fair, six-sided) die: A: Observe an odd number B: Observe an even number C: Observe a 1 or a 2 Question 1: Are A and B independent events? Solution: 3/15/15 79

80 Textbook Example 2.15 (continued) Question 2: Are A and C independent events? Solution: 3/15/15 80

81 Textbook Example 2.16 Three officers, X, Y, and Z, are to be ranked by their commanding officer. Define the following events: A: Officer X is ranked higher than Y B: Officer X is ranked first C: Officer X is ranked second D: Officer X is ranked third If the CO actually thinks all are equally capable and so randomly assigns ranks, is event A independent of events B, C, and D? 3/15/15 81

82 Textbook Example 2.16 (continued) Solution: 3/15/15 82

83 Section 2.7 Homework Do problems 2.71, 2.74, 2.75, /15/15 83

84 Section 2.8: Two Laws of Probability Theorem 2.5 (The Multiplicative Law of Probability): The probability of the intersection of two events A and B is P (A \ B) =P (A)P (B A) If A and B are independent, then P (A \ B) =P (A)P (B) Proof: = P (B)P (A B) 3/15/15 84

85 Extending the Multiplicative Law By twice applying Theorem 2.5: P (A \ B \ C) =P [(A \ B) \ C] = P (A \ B)P (C A \ B) = P (A)P (B A)P (C A \ B) And then note that this can be extended to an number, say k, events: P (A 1 \ A 2 \ A 3 \ \ A k )=P (A 1 )P (A 2 A 1 )P (A 3 A 1 \ A 2 ) P (A k A 1 \ A 2 \ \ A k 1 ) 3/15/15 85

86 Application Example Multiplicative Law very useful for calculating complicated probabilities when conditional probabilities are easier to determine What s the probability a missile hits a target? I.e., what s P(successful launch and successful target acquisition and successful intercept) With multiplicative law, we can calculate it as: P(successful launch) x P(successful target acquisition successful launch) x P(successful intercept successful launch and target acquisition) 3/15/15 86

87 The Second Law of Probability Theorem 2.6 (The Additive Law of Probability): The probability of the union of two events A and B is P (A [ B) =P (A)+P(B) P (A \ B) If A and B are mutually exclusive, so that P (A \ B) =0, then P (A [ B) =P (A)+P(B) Picture proof: 3/15/15 87 Source: Wackerly, D.D., W.M. Mendenhall III, and R.L. Scheaffer (2008). Mathematical Statistics with Applications, 7 th edition, Thomson Brooks/Cole.

88 Proof of Theorem 2.6 Proof 3/15/15 88

89 Application Example Consider the following hypothetical fleet composition: 80% of the fleet have a self-defense weapons system installed ( event A ) 40% of the fleet have an area-defense weapons system installed ( event B ) 30% have both area and self-defense systems Question: What is the probability that, if an enemy chooses a ship at random to attack, it has some type of defensive weapons system? Solution: P (A [ B) =P (A)+P (B) P (A \ B) = =0.9 3/15/15 89

90 Generalizing Theorem 2.6 By applying Theorem 2.6 three times, we can generalize to the union of three events: P (A [ B [ C) = 3/15/15 90

91 One More Useful Theorem Theorem 2.7: For event A, P (A) =1 P (A) Proof: ü Sometimes it s easier to calculate then use that to find P (A) P (A) and 3/15/15 91

92 Example Application Imagine you re going to fire a salvo of 10 missiles at a hardened high-value target to ensure its destruction. Under the existing conditions, each missile has a probability of kill, p k, of 0.3. What s the probability that at least one missile is a kill? Solving directly would require enumerating 2 10 = 1,024 possibilities, but: Probability one missile misses is 0.7 Thus, probability all missiles miss is So, probability at least one is a kill is = /15/15 92

93 Section 2.8 Homework Do problems 2.90, 2.94, 2.95, /15/15 93

94 Calculating Probabilities Via the Event-Composition Method (Section 2.9) The event-composition method uses the various laws of probability that we ve just learned to calculate the probability of some event A The idea is to express A as the union, intersection, or compliment of other sets Then calculate the probabilities of those other sets and appropriately combine those probabilities to get P(A) 3/15/15 94

95 Textbook Example 2.17 Of the voters in a city, 40% are Republican and 60% are Democrats. Among the Republicans, 70% are in favor of a bond issue, while 80% of the Democrats favor this issue. If a voter is selected at random, what is the probability that s/he will favor the bond issue? Solution: 3/15/15 95

96 Textbook Example 2.17 (continued) 3/15/15 96 Source: Wackerly, D.D., W.M. Mendenhall III, and R.L. Scheaffer (2008). Mathematical Statistics with Applications, 7 th edition, Thomson Brooks/Cole.

97 Textbook Example 2.18 In Example 2.7 we found the probability that the birthdays for each of 20 randomly selected Marines were all different: P(A) = Now find P(B), where B denotes that at least one pair of individuals share a birthday. Solution: 3/15/15 97

98 Steps in the Event-Composition Method 1. Define the experiment. 2. Visualize the nature of the sample points. Identify a few to clarify your thinking. 3. Write an (appropriate) equation expressing A, the event of interest, using unions, intersections, and/or compliments ü Make sure event A and the event implied by the composition represent the same sample point set 4. Apply the additive and multiplicative laws of probability to the compositions obtained in Step 3 to find P(A) 3/15/15 98

99 Textbook Example 2.19 Two analyst applicants are randomly selected from among 5 who have applied for a job (see examples 2.11 and 2.12). Find the probability that exactly one of the two best applicants is selected, P(A). Solution: 3/15/15 99

100 Textbook Example 2.19 (continued) 3/15/15 100

101 Textbook Example 2.19 (continued) 3/15/15 101

102 Textbook Example 2.20 It is known that a patient with a disease will respond to treatment with probability 0.9. If three patients with the disease are treated and respond independently, find the probability that at least one will respond. Solution: 3/15/15 102

103 Textbook Example 2.20 (continued) 3/15/15 103

104 Textbook Example 2.20 (continued) 3/15/15 104

105 Textbook Example 2.21 Observation of a waiting line at a medical clinic indicates that the probability of a new arrival will be an emergency case is p = 1/6. Find the probability that the r th patient is the first emergency patient (assuming the arriving patients represent independent events). Solution: 3/15/15 105

106 Textbook Example 2.21 (continued) 3/15/15 106

107 Textbook Example 2.21 (continued) 3/15/15 107

108 Textbook Example 2.22 A monkey is to demonstrate that she recognizes colors by tossing one red, one black, and one white ball into boxes of the same colors, one ball to each box. If the monkey merely tosses the balls at random, find the probabilities of the following outcomes: There are no color matches There is exactly one color match Solution: 3/15/15 108

109 Textbook Example 2.22 (continued) 3/15/15 109

110 Textbook Example 2.22 (continued) 3/15/15 110

111 Textbook Example 2.22 (continued) 3/15/15 111

112 Section 2.9 Homework Do problems 2.110, 2.111, 2.112, /15/15 112

113 Section 2.10: Law of Total Probability and Bayes Rule It is often useful to view a sample space as the union of mutually exclusive subsets Definition 2.11: For some positive integer k, let the sets B 1,B 2,...,B k be such that 1. S = B 1 [ B 2 [ [ B k 2. B i \ B j = ;, for i 6= j Then the collection of sets {B 1,B 2,...,B k } is said to be a partition of S 3/15/15 113

114 Using the Partition to Decompose a Set Now, if A is any subset of S and {B 1,B 2,...,B k } is a partition of S, then A can be decomposed as follows: A =(A \ B 1 ) [ (A \ B 2 ) [ [ (A \ B k ) An illustration for k = 3: 3/15/ Source: Wackerly, D.D., W.M. Mendenhall III, and R.L. Scheaffer (2008). Mathematical Statistics with Applications, 7 th edition, Thomson Brooks/Cole.

115 Law of Total Probability Theorem 2.8: Assume that {B 1,B 2,...,B k } a partition of S such that P (B i ) > 0 for i =1,...,k. Then for any event A, kx P (A) = P (A B i )P (B i ) Proof: i=1 is 3/15/15 115

116 Proof of Theorem 2.8 (continued) 3/15/15 116

117 Bayes Rule (or Bayes Theorem) Theorem 2.9: Assume that {B 1,B 2,...,B k } a partition of S such that P (B i ) > 0 for i =1,...,k. Then, P (A B j )P (B j ) P (B j A) = P k i=1 P (A B i)p (B i ) is Proof: 3/15/15 117

118 Applications of Bayes Theorem Invert conditional probabilities You know the various conditional probabilities P (A B 1 ),P(A B 2 ),...,P(A B k ) but you want to know P (B j A) for some Update probabilities You know or estimate P (B 1 ),P(B 2 ),...,P(B k ) then subsequently observe/collect some information P (A B 1 ),P(A B 2 ),...,P(A B k ) Then update your probabilities conditioned on what you ve observed by calculating P (B 1 A),P(B 2 A),...,P(B k A) B j P (B j A) 3/15/15 118

119 Inverting Conditional Probabilities: A Drug Testing Example Based on fleet-wide averages, a CO believes that 1% of his/her sailors use illegal drugs A test detects drug use as follows: If someone has used illegal drugs, there is a 99% chance it will detect it (true positive) If someone does not use illegal drugs, there is a 0.5% chance the test will indicate it (false positive) What is the chance that a positive drug test signals a drug user? 3/15/15 119

120 Approach and Notation U+ is the event an individual used illegal drugs, U- is the event he/she did not T+ is the event the drug test is positive for the individual We know P(T+ U+) = 0.99; we want P(U+ T+): P (U + T +) = = = 2 3 P (T + U+)P (U+) P (T + U+)P (U+) + P (T + U )P (U ) (1 0.01) 3/15/15 120

121 Common Sense Check Say there are 10,000 sailors: 100 are drug users (1%) and 9,900 are not 99 will be detected (99% of 100) 49.5 will be false positives (0.5% of 9,900) Total of positives Of those, 99/148.5 or 2/3 are drug users 49.5/148.5 or 1/3 are clean Conclusions Don t use drug tests with high false positives No point in testing clean population (all you get are false positives) 3/15/15 121

122 Updating Information: An Intelligence Example The enemy has three missile protection courses of action (COA): A, B, and C Initial Intel judges each equally likely to be used It also disallows the possibility of any other COA For each COA, there is a chance the enemy will also employ a new Patriot-like anti-missile system (AMS) P(AMS COA A) =.4 P(AMS COA B) =.6 P(AMS COA C) =.7 3/15/15 122

123 Updating Information: An Intelligence Example (cont d) The enemy uses the AMS Update the Intel probabilities for the COAs 1. COA A: Pr(COA A AMS) = Pr(AMS Pr(AMS A) Pr(A) A) Pr(A) + Pr(AMS B) Pr(B) + = (0.4 1/ 3) 0.4 1/ 3 + (0.6 1/ 3) = Pr(AMS + (0.7 1/ 3) C) Pr(C) 3/15/15 123

124 Updating Information: An Intelligence Example (cont d) Similarly: 2. P(COA B AMS) = 6/17 = P(COA C AMS) = 7/17 = 0.41 Notice the denominator is the same for all three calculations, since it is just P(AMS) using the law of total probability 3/15/15 124

125 Textbook Example 2.23 An electronic fuse is produced on five production lines and are shipped to buyers in 100-unit lots (by production line) Because testing is destructive, most buyers only test a small number before deciding to accept or reject a lot Four production lines have a 2% defect rate, but line #1 has a 5% defect rate A buyer tests 3 fuses in a lot and 1 failed: What is the probability the lot came from line #1? What is the probability it came from another line? 3/15/15 125

126 Textbook Example 2.23 Solution 3/15/15 126

127 Textbook Example 2.23 Solution (cont d) 3/15/15 127

128 Textbook Example 2.23 Alternate Solution 3/15/ Source: Wackerly, D.D., W.M. Mendenhall III, and R.L. Scheaffer (2008). Mathematical Statistics with Applications, 7 th edition, Thomson Brooks/Cole.

129 Section 2.10 Homework Do problems 2.124, 2.125, 2.129, 2.130, 2.132, 2,133 Hint for problem 2.132: Set the problem up in terms of two events, each dependent on the region Let L i be that the plane is actually located in region i Let S i be a successful search for the plane in region i 3/15/15 129

130 Section 2.11: Random Variables Numerical outcomes from experiments are called random variables They are random variables because their value depends on what happens in the experiment and they will vary from outcome to outcome The idea is we assign a real number to each (set of) outcome(s) in the experiment Simple example: Let the random variable Y denote the outcome from rolling a fair six-sided die Then Y can take on the values 1, 2, 3, 4, 5, or 6 depending on what happens when the die is rolled 3/15/15 130

131 Random Variables, continued Definition 2.12: A random variable is a realvalued function for which the domain is the sample space Example: Another example: Let the random variable Y denote the outcome of flipping a two-sided coin The sample space is S = {heads, tails} Let 1 denote heads and 2 denote tails Then Y can take on the values 1 or 2 3/15/ Source: Wackerly, D.D., W.M. Mendenhall III, and R.L. Scheaffer (2008). Mathematical Statistics with Applications, 7 th edition, Thomson Brooks/Cole.

132 Textbook Example 2.24 Define an experiment as tossing two coins and observing the results. Let Y equal the number of heads obtained. Identify the sample points of S Assign a value of Y to each sample point Identify the sample points associated with each value of Y Solution: 3/15/15 132

133 Textbook Example 2.24 (continued) 3/15/15 133

134 Textbook Example 2.25 Compute the probability for each value of Y in the previous example. Solution: 3/15/15 134

135 A Bit About Notation Capital Roman letters denote random variables, typically X, Y, and Z These are placeholders for the values an experiment will take on once it is complete Small Roman letters denote the value of a random variable, typically x, y, and z They are the outcomes of the experiment So, we can write things like P(Y=y) which in words is the probability that random variable Y takes on the value y Get comfortable with this notation you will use it a lot in this class and in this curriculum! 3/15/15 135

136 Section 2.11 Homework Do problems 2.139, /15/15 136

137 Sections 2.12 & 2.13: Random Sampling One application of probability is in selecting random samples from a population Important when executing various types of experiments as well as in collecting survey data Random selection of a sample from a population is critical for being able to do valid inference Lots of ways to draw random samples Two broad categories: Sampling without replacement: Population elements cannot be sampled more than once Sampling with replacement: Population elements can be sampled multiple times 3/15/15 137

138 (Simple) Random Sampling Definition 2.13: Let N be the number of elements in the population and n be the number in the sample. A (simple) random sample is a sample such that each of the possible samples has an equal probability of being selected This is just one possible type of sample Others include stratified sampling, cluster sampling, and complex sampling that combines two or more of these methods I discuss these methods in detail in OA4109 N n 3/15/15 138

139 Drawing a (Simple) Random Sample Textbooks like to talk about physical processes for randomly drawing observations Labeling poker chips and mixing them up in a pile Writing names on slips of paper and mixing them up in a hat Much like lotteries draw numbers on ping pong balls blown around in a container Really old-school is using tables of random numbers At least for research, these days a computer used to draw random samples 3/15/15 139

140 1970 Military Draft Lottery, or, How Not To Conduct Sampling December 1, 1969 lottery held to determine the induction order for the 1970 military draft Idea: randomly choose birth days (1/1-12/31) Order chosen is order drafted For example, September 14 th chosen first, so all men born that day (between 1944 and 1950) were the first to be inducted After lottery conducted, controversy arose about whether it was truly random Photo of the first capsule being drawn by Congressman Alexander Pirnie (R-NY) of the House Armed Services Committee 3/15/15 140

141 Results of the Lottery 3/15/15 141

142 Random? Draft Number 3/15/15 Birthday (Julian date) 142

143 Another Way to Look at the Results 3/15/15 143

144 Conclusions Review of the methodology showed the balls were not properly mixed Allowed birth dates later in the year to be more easily chosen Method for later lotteries was subsequently modified to ensure proper randomization 3/15/15 144

145 What We Have Just Covered Introduction to probability Methods for calculating probabilities Sample-point method Event-composition method Counting tools Combinations, permutations, etc. Conditional probability and independence Probability laws and rules Multiplicative and additive laws Law of Total Probability and Bayes Rule Random variables A bit about random sampling 3/15/15 145