Discrete Probability. Chemistry & Physics. Medicine

Transcription

1 Discrete Probability The existence of gambling for many centuries is evidence of long-running interest in probability. But a good understanding of probability transcends mere gambling. The mathematics of probability are very important for understanding all kinds of important topics. Here are a few examples: Error analysis Any experimental measurement, no matter how carefully performed, is affected by random errors or noise. Random errors follow the laws of probability, which form the basis of how we estimate the effect of random errors on our results. For example, we might measure a length and report it as 3.45 ± 0.03 m, where the 0.03 is a measure of the average random error present in the measurement. How exactly we estimate that average error comes from a study of probability. Chemistry & Physics Whether a chemical reaction takes place depends on a number of factors, like whether reactants collide (necessary for a reaction to occur), with what relative velocity (kinetic energy) they collide, and in what orientation they collide. Because in any ensemble of reacting molecules there will be a wide and randomly-occurring range of speeds, paths and orientations, these processes are best understood using the laws of probability. There is a whole field in physics/chemistry called statistical mechanics, based on probability theory, that derives the laws of thermodynamics from a study of the behavior of large ensembles of atoms & molecules. Medicine The laws of probability are crucial for understanding epidemics, the effectiveness of medical treatments and the effectiveness of drugs to treat and tests for disease. Discrete probability Discrete events are those with a finite number of outcomes, e.g. tossing dice or coins. For example, when we flip a coin, there are only two possible outcomes: heads or tails. When we roll a six-sided die, we can only obtain one of six possible outcomes,, 2, 3, 4, 5, or. Discrete probabilities are simpler to understand than continuous probabilities, so we'll start there. Let's look at flipping a coin first. The probability, we'll call it P, of obtaining an outcome (heads or tails) is chance in 2, or :2, or just ½. The possible elementary outcomes of our experiment (coin flipping) form a set {H, T}. If we call P the probability function and H and T the two possible outcomes, then P(H) = ½, and P(T) = ½. When we flip a coin, we have to get either H or T, so the total probability is. Here, of course, we need to say that we're ruling out the unlikely event that the coin will land in such a way that it sticks on its edge. When we flip a coin, we make the reasonable assumption that there are only two possible outcomes, and the one we get can only be one of those, or ½ of the total. 200 J. Cruzan of

2 So the probability of obtaining either H or T from our experiment can be written: P H P T = In other words, the sum of all possible discrete outcomes is one. Note here that this is only true when outcomes H and T are mutually exclusive i.e. when they can't occur at the same time. The story would be different if we could get heads and tails at the same time. There are two important properties of probability that we ought to get out of the way: () P(Oi) 0 The probability of the ith outcome, Oi, must be greater than or equal to one. We can't have a negative probability, nor can we have a probability greater than one. n (2) P O i i= = The sum of all n discrete outcomes, Oi, must be. Another example: Dice The elementary outcomes for rolling dice are {, 2, 3, 4, 5, } for one die ( possible outcomes), and {, 2, 3, 4, 5,, 2, 22, 23, 24, 25, 2, 3, 32, 33, 34, 35, 3, 4, 42, 43, 44, 45, 4, 5, 52, 53, 54, 55, 5,, 2, 3, 4, 5, } for two dice (3 possible outcomes, see the figure below). All possibilities for rolling two dice. The left column shows the sum of the two dice, and the right shows the number of ways of arriving at that sum. There are more ways (out of 3) to arrive at a sum of seven than any other total, so it is more likely that you will roll a seven than any other number. The probability of rolling a seven is /3, or /. The probability of rolling a 2 is / J. Cruzan 2 of

3 Let's now assume that we have distinguishable dice, one white and one black, for example, so that there is a difference between a one on the first and a two on the second (2) and a two on the first and a one on the second (2). Now let's define an event as some possible outcome or set of outcomes. Here are some examples of how we might define events for two dice: Event Description Elementary outcomes probability A Dice add to three {2, 2} 2/3 B Dice add to six [5, 5, 24, 42, 33} 5/3 C White die = {, 2, 3, 4, 5, } /3 D Black die = 5 {, 2, 3, 4, 5, } /3 Events can by combined, and we'll need some notation to represent the combinations. Combination of events A & B Definition Notation A and B Both A and B occur A B (intersection) A or Either A or B occurs, or both A B (union) Event A does not occur!a (read not A ) B not A Now let's define two events, A = {black die comes up } and B = {white die comes up }, and look at A B : The upper row contains all instances of a on the black die, and the left column all instances of a on the white die. It's easy to see why we write A and B as A B. It's just the intersection of the two sets of elementary outcomes of events A and B. The probability of A and B occurring together is P A B = 3 Now look at A C A or B. Here we have to be careful, because both sets of elementary outcomes contain (). In order not to double count that possibility, we have to write P A B = P A P B P A B. There's another possibility, however. What if events A and B are mutually exclusive. That is, if one occurs, the other can't possibly occur, and vice-versa. For example, let event C = {dice add to 3} and event D = {dice add to }. Clearly, both can't happen at the same time. 200 J. Cruzan 3 of

4 Here's a picture: The probability of C occurring is P(C)=2/3, and the probability of D occurring is P(D)=4/3, so the total probability is P C D = or 2 4, 3 3 P C D = But the probability of observing both events, P C D is zero, because the events are mutually exclusive. If the dice sum to three, they cannot possibly sum to five, and viceversa. So we have: Law of addition of probabilities: P A B = P A P B P A B P A B = P A P B or for mutually exclusive events, where A B = 0. Law of subtraction The sum of the probabilities that an event will occur and that it won't occur must equal one. P(A) + P(!A) = or P(A) = - P(!A) Conditional Probability Conditional probability is one of the most important concepts in probability. It lets us take some fairly simple data and extract a lot of meaningful information about cause & effect, and about risk. Conditional probability is the probability that a second event will occur, provided that a first event has already occurred. Think about this experiment: What is the probability that the sum of two dice will be three? We can perform this experiment in two ways: Method throw the dice at the same time, and let A = {2} and B = {2}. be the sum of all of the probabilities of getting a sum of three. P A B P A B = P A P B P A B Addition rule P A B = P A P B 0 Mutually exclusive events 200 J. Cruzan will 4 of

5 P A B = = Probability of rolling a sum of three. Method 2 throw one die first, and suppose it comes up. Now what is P A B = P A B? now, because we've basically reduced the sample space by a factor of two Now we can put that into symbolic form. Let event C = {throw a on the first die} and event A = {throw a total of three on two dice}. We'll define P (A C) as the probability that event A will occur if event C has already occurred conditional probability, where the occurrence of event C is the condition for the occurrence of event A. Conditional probability arises in many important ways. Consider these questions: What is the probability that a smoker will develop lung cancer? What is the probability that a nonsmoker will develop lung cancer? How much more likely is a smoker to develop cancer than a non-smoker? The first two questions are conditional probabilities. A person is either a smoker or a non-smoker (with some probability) and each then has a certain probability of developing lung cancer. The third question involves taking a simple ratio, but gives the most valuable information. Now we can think about P(A C) like this: P A C = P A C P C () Think about that formula for a while. Note that it gets P(A C) right, and it also predicts that P(A A) =, and that P(E F) = 0 if E and F are mutually exclusive events. Multiplying () by P(C) on both sides, we obtain the multiplication rule of probability for events that are not independent: P A C = P A C P C. Independence of events Events may or may not be independent. We might want to know whether the occurrence of one event affects the occurrence of another. Two events, A and C, are independent if the occurrence of one does not affect on the probability of occurrence of the other. That means P(A) = P(A C) or P(C) = P(C A). Here are two examples of a two-dice experiment to illustrate how we can check for independence: 200 J. Cruzan 5 of

6 Example Let A = {die = } and B = {die 2 = }, then P A B 3 P A B = = = = P A P B the two events are independent. Example 2 Let C = {die = } and A = {sum of dice is 3}, then P A C 3 P A C = = = P A P C P(A) in this case is 8 Baye's rule Baye's rule, the root of so-called Bayesian probability, is really just a rearrangement of the multiplication rule. If we know that P A B = P B A, and P A B = P A B P B = P B A P A, then P A B = P B A P A P B Baye's theorem allows us to connect conditional probabilities that point in opposite directions, like P A B and P B A. Summary of discrete probabilities A not A Subtraction rule A or B Addition rule P(A) Total probability of A A and B Multiplication Rule A given B Bayes' theorem 200 J. Cruzan P A [ 0, ] Probability must be between zero and one. P! A = P A P A B = P A P B P A B P A B = P A P B If A and B are mutually exclusive P A = P A B P A C P A B = P A B P B P A B = P A P B P A B = P A B P B P A B = P B A P A P B If A and B are independent Connects P(B A) and P(A B) of

7 Trees Sometimes it is helpful to draw a tree diagram like the one below to help you sort out what's going on in a conditional probability problem. The root of the tree below might be all people. A could be women and!a could represent men. Then B and!b might stand for gets migraine headaches. If we define events like this, we can traverse the tree like this: The probability that A woman gets migraines is P A B ), which is the same as the conditional probability that B occurs after A has already occurred, multiplied by the probability that A occurs at all. The total probability in each level of sub-branches must be one. 200 J. Cruzan 7 of

8 Permutations and combinations In order to determine the probability of some event, we often need to consider the number of ways in which elements of some set can be ordered or combined. Take the letters A,B and C as a simple example. In how many ways can these three letters be ordered. We can list them it's pretty easy: ABC ACB BAC BCA CAB CBA We say that there are six permutations of three different letters (of course if two letters are the same, some of these permutations will be indistinguishable. Sometimes we arrange permutations in a tree like the one above. Note that there are three choices for the first letter of our sequence, two for the second, and only one (no choice) for the third. The number of possible permutations is then 3 2 = 3!. In general, There are n! permutations of an n-set. Where n! (read n-factorial ) is defined, n! = n(n )(n 2) 2. Look at this example: How many different permutations can you make from the letters in the word BUMBLEBEE? It is tempting to suggest that the number of permutations is the factorial of the number of letters, 9! = 32,800. This would be wrong, however, because we can't tell when we swap the positions of the three B's or the three E's. The correct number of permutations is N = 9! 3! 3! In general, the number of distinguishable permutations of a set with k sets of indistinguishable elements n, n2, is n! N =. n! n 2! nk! Now what if we make groups of three letters taken from the 2-letter alphabet? These are called combinations. The number of three-letter combinations is N = 2! = 5, ! In general, the number of combinations of n objects taken r at a time is: 200 J. Cruzan 8 of

9 n! n =. The n r! r n r is read n, choose r and is called a binomial coefficient. And again, if we don't care about the order of the items in the r-list, the number of combinations becomes: n! n = r! n r! r Example: In some state lotteries, the game is to choose six numbers from a pool of 40, and order doesn't matter. The number of possible sequences is then the number of distinguishable permutations of 40 objects, taken six at a time: 40! 40 = = 3,838,380! 40! Three important factorial relationships:. For any integer n, n! = n (n )! 2. For any integer n (n + )! = (n + ) n! 3. 0! (zero factorial is defined to be one). 200 J. Cruzan 9 of

10 Continuous Probability Random variables A random variable, which we often write as X, represents the possible numerical outcomes of some random experiment. Random variables can be discrete or continuous. Discrete random variables can only take on a countable number of distinct values, like the outcomes of rolling two dice and taking the sum of the two: X = {2, 3,, 2}. Other examples are the number of points in a football game, or the number of people who attend a soccer game. The probability distribution of a discrete random variable X is a list of the probabilities associated with each possible value of X. We often graph the probability distribution. For the sum of two dice, the probability distribution, P(X = x, x2, ) looks like the histogram at right. In this case, x = 2, x2 = 3, and so forth, and all possibilities make up the set of values that X can take on. Continuous random variables are usually measurements, like height or speed. Imagine either measuring the heights of a large number of people, or asking a large number of people to measure the height of one person. In the first case, a wide variety of heights would be recorded. It's not difficult to imagine that, given any two heights, we could find another height in between this is the hallmark of a continuous function. In the second case, random errors always present in any measurement, would produce a continuous range of heights, the average of which should be close to the true height of the person being measured. Mean of a distribution You probably recognize the mean (average) of a set of n numbers x, x2, x3, as x = x x 2 x n n. Just add all of the values, xi, and divide by the number of values, n. We often designate the 200 J. Cruzan 0 of

11 mean of a variable by writing a bar over it, like x, which can be read as x-bar. We can now also define the mean of a probability distribution, where x, x2, xn are the values of the random variable X, and p, p2, pn are the probabilities of obtaining each xi, as x = x p x 2 p 2 x n p n. Confirm for yourself that the mean of the sum-of-two dice experiment is x = 7. Variance of a distribution Finally, we need to have some measure of how good the mean of a distribution is. Let's go back to our measuring heights example. Imagine that we do the measurement twice, once with a very precise ruler and once with a not-so-good ruler. In the first case we would expect a very narrow probability distribution, and in the second, a much greater spread, like the comparison at right. The measurements represented by the solid curve are much less spread out than the ones represented by the dashed curve. Therefore, we ought to have more confidence in its mean value. The variance is a measure of the width of the probability distribution around the mean. We define the variance of a distribution, σ (sigma), as 2 x = x i x i 2 pi i We usually call σ the standard deviation of the mean of the distribution. Take a look at the definition of the variance. It's really just the average of the sum of the squares of the differences of each measurement from the mean. That's a mouthful, but stare at it for a while and it will make sense. 200 J. Cruzan of