1 Probability Theory. 1.1 Introduction

Transcription

1 1 Probability Theory Probability theory is used as a tool in statistics. It helps to evaluate the reliability of our conclusions about the population when we have only information about a sample. Probability theory is also the main tool in developing a model for describing the populations in inferential statistics. (In inferential statistics the information from a random sample is used to make statements about an entire population.) Many things in our world depend on randomness, if you observe Head while flipping a coin or you roll a 6 with a fair die, if the bus is on time, etc. Even those events occur randomly there is an underlying pattern in the occurrence of these events. This is the basis of Probability Theory. 1.1 Introduction We now provide the vocabulary for probability theory. Definition: 1. A phenomenon is random if individual outcomes are uncertain but there is nonetheless a regular distribution of outcomes in a large number of repetitions. 2. The probability of any outcome (or event) of a random phenomenon is the proportion of times the outcome would occur in a very long series of repetitions. Probability Models Definition: 1. The sample space of an experiment is the set of all possible outcomes. 2. An event is a subset of the sample space. Examples for random phenomenons are: Recording a test grade Measure daily snowfall Count leucocytes in a blood sample Roll a die Toss a coin The sample spaces of above random phenomenon are: {A, B, C, D, E, F } All numbers between 0 and 200 cm. 1

2 /ml?? {1, 2, 3, 4, 5, 6} {H, T } Examples for events after tossing a die and observing the number on the upper face are: Observe a 1 ={1} Observe an odd number ={1, 3, 5} Observe 1 or 6 ={1, 6} Suppose you roll an unbiased die with 6 faces and observe the outcome. Venn Diagrams The outer box represents the sample space, which contains all of the simple events. Is A = {E 1, E 2, E 3 } the collection of simple events the appropriate events are circled and labelled with the letter A. A E3 S E1 E4 E5 E2 E6 Basic Properties: The probability of an event is a number between 0 and 1, since it is a limit of a relative frequency. P (E) = 0, if the event E never occurs. P (role a 7 with a regular die)=0 P (E) = 1, if the event always occurs. P ( role with a regular die a number smaller than 7)=1 The sum of the probabilities for all simple events in S equals Basic Rules Definition: The probability of an event A is equal to the sum of the probabilities of the simple events contained in A. The union of events A and B, denoted by AorB, is the event that either A or B or both occur. 2

3 A B The intersection of events A and B, denoted by A&B, is the event that both A and B occur. A B The complement of an event A, denoted by nota, is the event that A does not occur. A c A Two events, A, B, are called mutually exclusive, if A&B =. Continue the die rolling example. C is the event to roll an odd number. Let D be the event to roll a number smaller than 4. Then D = {1, 2, 3}. The probability of every simple event is 1/6. With the definition follows that P (C) = P ({2}) + P ({4}) + P ({6}) = 1/6 + 1/6 + 1/6 = 3/6 = 1/2. P (CorD) = P ({1, 3, 5, 2}) = 4/6 = 2/3. P (C&D) = P ({1, 3}) = 2/6 = 1/3. P (notd) = P ({4, 5, 6}) = 3/6 = 1/2. If E is the event to roll a 2, then events C and E are mutually exclusive. 3

4 Remark: Until this point: In order to calculate the probability of an event, 1. find all the simple events, that belong to the event, 2. find the probabilities for those simple events and 3. add those probabilities to derive the probability for the event of interest. The following rules provide tools for the determination of the probabilities of events like A B and A c. Addition Rule Let A and B be two events. The probability of the union can then be calculated by: P (A oder B) = P(AorB) = P(A) + P(B) P(A&B) Remark: The subtraction of P (A&B) is necessary because this area is counted twice by the addition of P (A) and P (B), once in P (A) and once in P (B). Check the diagram below. A B A and B Complementation Rule (Rule for Complements) The probability of the complement of an event is 1 minus the probability of this event. Mathematically: If P (nota) = 1 P (A) Let A = {1, 2} and B = {1, 3, 5}, with P (A) = 1, P (B) = 1, and P (A&B) = P ({1}) = With the Addition Rule we get P (AorB) = P (A) + P (B) = 1/3 + 1/2 1/6 = 2/3. With the Rule for Complements is P (nota) = 1 1/3 = 2/3. 4

5 1.3 Independence of Two Events Two events are considered independent, when the occurrence of one of the events has no impact on the probability for the second event to occur. They don t have an impact on each other. In this section we will define and do examples to illustrate this concept. For defining the independence of events properly we first introduce the concept of a conditional probability. Definition If A and B are events with P (B) > 0, the conditional probability of A given B is defined by P (A B) := P (A&B). P (B) B B c A and B A and B c S A The interpretation of the conditional probability P (A B) is as follows: Given that you know already event B occurred, what is the probability that A occurs. Consider the event B to roll an even number with a fair 6 sided die. The Probability for the event A to roll a 1, given B equals 0: P (A B) = 0 Because: If you roll a die, someone peeks and tells you that the number is even, than this number can not be a 1. So the conditional probability of A given B must be 0. Another way to argue that this probability is as follows: There are three favourable events in B, out of these zero are a one, therefore P (A B) = 0/3 = 0 Another way to show the dependency of events on each other is in a two-way or contingency table. Research has shown, that a mutation in the BRCA gene increases the probability of a woman to develop breast cancer. The following table shows data which is consistent with numbers reported in the Journal of the National Cancer Institute. Breast Cancer BRCA Gene yes no Total yes no Total Then 5

6 P(Breast Cancer = yes) =625/5000=0.125 (marginal probability (ignore the mutation of the gene)) P(BRCA Gene = yes) =20/5000=0.004 (marginal probability (ignore breast cancer)) P(BRCA Gene = yes & Breast Cancer = yes ) = 16/5000 = (joint probability) P(Breast Cancer = yes BRCA Gene = yes) = 16/20 = 0.8 (conditional probability) P(BRCA Gene = yes Breast Cancer = yes) =16/625= (conditional probability) Given these numbers indicate that a mutation of this gene has a great impact on the probability of developing breast cancer. The two events mutation of the BRCA gene and Develop breast cancer are not independent. Joint and Marginal Probabilities joint: consider event P (A&B) marginal: only consider one event at a time. P (A), P (B) are both marginal probabilities. In probability theory two events are considered independent, if the knowledge that one of them has occurred does not change the probability for the second to occur. Definition: Events A and B, with P (B) > 0, are independent if P (A B) = P (A), (which is equivalent to P (A&B) = P (A) P (B)). Remark: From this definition we also learn that: If we know that two events A and B are independent then P (A&B) = P (A)P (B) This is only true for INDEPENDENT events. In the case, we do not know, if two events are independent, the general multiplication rule applies: P (A&B) = P (A B)P (B) = P (B A)P (A) The following example shows how to apply these two two concepts to the experiment of rolling a die. Consider the experiment to roll a fair die. Let A = {1, 2, 3} B = {4} C = {3, 4} Then we can calculate, by finding the simple events contained by the described events and adding their probabilities: 6

7 P (A) = 1 2 P (B) = 1 6 P (C) = 1 3 P (A&B) = 0 P (A&C) = 1 6 P (B A) = P (B&A) P (A) = = 0 (by definition of the conditional probability) Since P (B) P (B A), we find that B and A are not independent (apply the definition for independence). P (C A) = P (C&A) P (A) = = 1 3 Since P (C) = P (C A), we find that C and A are independent. Consider the experiment of rolling two unbiased dice. The simple events of this experiment can be described as a pair of numbers between 1 and 6. (3, 4) would be interpreted as rolling a 3 with the first die and a 4 with the second die. The sample space contains of all pairs with numbers between 1 and 6 in the first and the second component. So that the sample space consists of 6 6=36 elements. Since both dice will fall independently we can calculate the probability of rolling two 6 by P ((6, 6)) = P (6 with the first die) P(6 with the second die) = = A satellite has two power systems, a main and an independent backup system. Suppose the probability of failure in the first ten years for the main system is 0.05 and for the backup system What is the probability that both systems fail in the first 10 years and the satellite will be lost? Let M be the event that the main system fails and B the event that the backup systems fails. Since the systems are independent we get: P (M&B) = P (M) P (B) = = The event O that at least one of the systems is still operational after 10 years is the complement event of both systems failing. We calculate: P (O) = 1 P (noto) = = Example : 7

8 An experiment can result in one of five equally likely simple events E 1, E 2, E 3, E 4, E 5. Events A, B and C are defined as follows: A := {E 1, E 3 } B := {E 1, E 2, E 4, E 5 } C := {E 3, E 4 } Find the probabilities for the following events: a. nota b. A&B c. B&C d. AorB e. B C f. A B g. Are events A and B independent? Solution (first try on your own): a. 3/5 b. 1/5 c. 1/5 d. 1 e. 1/2 f. 1/4 g. No they are not, since P (A) P (A B) The following example shows the practical relevance of these concepts. In this example they are applied to an AIDS test, but please be aware that the same arguments apply to other clinical tests. ELISA is a test for detecting HIV antibodies in blood samples. The test is positive in 99.7% of blood samples containing antibodies. The test is positive in 1.5% of blood samples containing no antibodies. Assuming that 1% of the population do have HIV antibodies in their blood, how big is the probability that a person has HIV if his/her blood tests positive? For finding an answer to this question we translate the number given above into probability theory and apply the rules introduced. First choose the denotation: Let H + be the event of a blood sample containing antibodies. Then is H = H C + the event of a blood sample not containing antibodies. Let T + be the event of a positive ELISA test result. Then is T = T C + the event of a negative ELISA test result. Now translate the facts stated above into conditional probabilities. We get from the introductory statements and the definition: P (T + H + ) = 0.997, P (T + H ) = 0.015, and P (H + ) = 0.01 and want to know P (H + T + ). Calculate using the definition: P (H + T + ) = P (H +&T + ) P (T + ) 8

9 Now calculate numerator and denominator separately. First the numerator (applying the definition): P (H + &T + ) = P (H + )P (T + H + ) = = Second the denominator: P (T + ) = P ((T + &H + ) or (T + &H )) cutting T + in two pieces = P (T + &H + ) + P (T + &H ) Addition Rule = P (H + )P (T + H + ) + P (H )P (T + H ) definition of conditional probability = (1 0.01) = Rule for Compliments Put the numerator and the denominator together and we get P (H + T + ) = This probability is very low and a test result would have to be confirmed with additional tests. This shows that ELISA is rather a test to exclude HIV then to test for HIV. Try on your own to calculate P (H T ). Since the probability for a positive ELISA test is different for blood samples with or without antibodies, we conclude that the probability for a positive ELISA test is different in the sample space than it is for positive blood samples. We can say that the event positive ELISA test is NOT independent from the event antibodies in the blood sample The Basic Counting Rule If an experiment is performed in r stages with m 1 ways to accomplish stage 1 and m 2 ways to accomplish stage 2, etc. then the number of ways to accomplish the whole experiment is m 1 m 2... m r Assume you are rolling three dice and observe the numbers rolled with the three dice, then this experiment has three stages. The first die can show 6 different numbers (first stage), the second die can show 6 different numbers (second stage), and the third die can show 6 different numbers (third stage). So the whole experiment has 6 6 = 216 different outcomes. Consider you have 5 urns with numbered balls. Every urn hold balls in a different color, so that the balls from different urns can be distinguished: Urn 1 contains balls 1 to 5 Urn 2 contains balls 1 to 7 9

10 Urn 3 contains balls 1 to 3 Urn 4 contains balls 1 to 10 Urn 5 contains balls 1 to 2 If the experiment asks you to draw one ball each from urn 1 to 5 and report the five numbers. Then the number of possible outcomes equals = Suppose you roll a die and toss a coin, then the number of possible outcomes equals 6 2 = Permutations For example, suppose you are following a race of three cars (A, B, C). How many different possible outcomes for first and second place do exist? There are 3 choices for the two cars, ranking first and second, that is: A and B, A and C, B and C. But each pair can lead to two different race results. So that there are 3 2=6 different race result for the first two ranks. The Basic Counting Rule implies this result, because the first position can be chosen in m 1 = 3 ways and the second position in m 2 = 2 ways, the result is m 1 m 2 = 6. How many different outcomes for first, second, and third place exist, if 4 cars are racing? Solution: = 24 different outcomes. Rather than applying the Basic Counting Rule each time, you can find the number of permutations using the following formula. A counting rule for Permutations The number of ways we can arrange m distinct objects, taking r at a time, is mp r = m! (m r)! where (m factorial) m! = m(m 1)(m 2) (3)(2)(1) and 0! = 1. In other words m P r gives you the the number of different possibilities to distribute m distinctive objects on r available places. Remark: In permutations order matters! How many different ways have Alice, Bob, Steven, and Carol to line up at the box office at a theater? Solution: 4P 4 = n! = 4! (n r)! 0! Combinations = 4! = = 24. Now suppose you are interested how many different five person committees can be put together, if the members are chosen out of a group of 12 people. For choosing the members of the 10

11 committee the order is not relevant. These subsets or subgroups out of a given group of distinctive members is called combinations. A Counting Rule for Combinations The number of combinations of m distinct objects that can be formed, taking them r at a time, is (m choose r) m! mc r = r!(m r)! Remark: In combinations order does not matter! Solution for the example The number of possible committees equals 12C 5 = 12! 5!(12 5)! = = = 792 The relationship between permutations and combinations Combinations of r elements out of m distinct objects, are sets of r objects, where the order does not matter. If we take any set of r objects there exist r! (basic counting rule) different ways to arrange them. So that we conclude from the Counting Rule for Combinations that the number of Permutations equals mp r = m C r r! = m! r!(m r)! r! = m! (m r)! A student prepares for an exam by studying a list of 10 problems. He can solve 6 of them. For the exam, the instructor selects 5 questions at random from the list of 10 problems. What is the probability of the event A that the student can solve all 5 problems? In order to calculate the probability find first how many different exams can be put together. Since the order does not matter N = 10 C 5 = 10! 5!5! = = = 252 Now find the number of exams of which the student is able solve all questions. Again the order is not relevant, so n A = 6 C 5 = 6! 5!1! = 6 1 = 6 The probability that the student can solve all 5 questions in the exam equals P (A) = n A N = =