2.1 Independent, Identically Distributed Trials

Transcription

1 Chapter 2 Trials 2.1 Independent, Identically Distributed Trials In Chapter 1 we considered the operation of a CM. Many, but not all, CMs can be operated more than once. For example, a coin can be tossed or a die cast many times. By contrast, the next NFL season will operate only once. In this chapter we consider repeated operations of a CM. Let us return to the Blood type CM of Section 1. Previously, I described the situation as follows: A man with AB blood and a woman with AB blood will have a child. The outcome is the child s blood type. The sample space consists of A, B and AB. I stated in Chapter 1 that these three outcomes are not equally likely, but that the ELC is lurking in this problem. We get the ELC by viewing the problem somewhat differently, namely as two operations of a CM. The first operation is the selection of allele that Dad gives to the child. The second operation is the selection of allele that Mom gives to the child. The possible outcomes are A and B and it seems reasonable to assume that these are equally likely. Consider the following display of the possibilities for the child s blood type. I am willing to make the following assumption. Allele from Mom Allele from Dad A B A A AB B AB B The allele contributed by Dad (Mom) has no influence on the allele contributed by Mom (Dad). Based on this assumption, and the earlier assumption of the ELC for each operation of the CM, I conclude that the four entries in the cells of the table above are equally likely. As a result, we have the following probabilities for the blood type of the child: P(A) = P(B) = 0.25 and P(AB) = Here is another example. I cast a die twice and I am willing to make the following assumptions. 11

2 The number obtained on the first cast is equally likely to be 1, 2, 3, 4, 5 or 6. The number obtained on the second cast is equally likely to be 1, 2, 3, 4, 5 or 6. The number obtained on the first (second) cast has no influence on the number obtained on the second (first) cast. The 36 possible ordered results of the two casts are displayed below, where, for example, (5, 3) means that the first die landed 5 and the second die landed 3. This is different from (3, 5). Number from Number from second cast first cast (1,1) (1,2) (1,3) (1,4) (1,5) (1,6) 2 (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) 3 (3,1) (3,2) (3,3) (3,4) (3,5) (3,6) 4 (4,1) (4,2) (4,3) (4,4) (4,5) (4,6) 5 (5,1) (5,2) (5,3) (5,4) (5,5) (5,6) 6 (6,1) (6,2) (6,3) (6,4) (6,5) (6,6) Just like in the blood type example, b/c of my assumptions, I conclude that these 36 possibilities are equally likely. We will do a number of calculations now. For ease of presentation, define X 1 to be the number obtained on the first cast of the die and let X 2 denote the number obtained on the second cast of the die. We call X 1 and X 2 random variables, which means that to each possible outcome of the CM they assign a number. Every random variable has a probability distribution which is simply a listing of its possible values along with the probability of each value. Note that X 1 and X 2 have the same probability distribution; a fact we describe by saying that they are identically distributed, Below is the common probability distribution for X 1 and X 2. Value Probability 1 1/6 2 1/6 3 1/6 4 1/6 5 1/6 6 1/6 Total 1 As we have seen, X 1 and X 2 each has its own probability distribution (which happens to be the same). It is also useful to talk about their joint probability distribution which is concerned with how they interact. I will illustrate this idea with a number of computations. P(X 1 = 3 and X 2 = 4) = 1/36, b/c, by inspection exactly one of the 36 ordered pairs in the earlier table have a 3 in the first position and a 4 in the second position. 12

3 Before we proceed, I want to invoke my laziness again. It is too much bother to type, say, It is much easier to type, P(X 1 = 3 and X 2 = 4). P(X 1 = 3, X 2 = 4). We will follow this practice; a comma within a probability statement represents the word and. For future reference, note that P(X 1 = 3, X 2 = 4) = 1/36, as does P(X 1 = 3)P(X 2 = 4). In words, the word and within a probability statement tells us to multiply. Here is another example. P(X 1 4, X 2 4) = 12/36, b/c, as you can see from the table below, exactly 12 of the 36 pairs have the required property. Note again that X 2 X X X X 2 X X X 3 X X X 4 X X X 5 6 P(X 1 4, X 2 4) = 12/36 gives the same answer as P(X 1 4)P(X 2 4) = (4/6)(3/6) = 12/36. This last property is called the multiplication rule for independent random variables. It is a very important result. It says that if we have two random variables that are independent, then we can compute joint probabilities by using individual probabilities. In simpler words, if we want to know the probability of X 1 doing something and X 2 doing something, then we can calculate two individual probabilities, one for X 1 and one for X 2 and then take the product of these two individual probabilities. As we shall see repeatedly in this class, the multiplication rule for independent random variables is a great labor saving device. The above ideas for two casts of a die can be extended to any number of casts of a die. In particular, define X 1 to be the number obtained on the first cast of the die; X 2 to be the number obtained on the second cast of the die; X 3 to be the number obtained on the third cast of the die; 13

4 and so on, in general, X k is the number obtained on the kth cast of the die. If we assume that the casts are independent that is, that no outcome has any influence on another outcome then we can use the multiplication rule to calculate probabilities. Some examples are below. You might be familiar with the popular dice game Yahtzee. In this game, a player casts five dice. If all dice show the same number, then the player has achieved a Yahtzee. One of the first things you learn upon playing the game Yahtzee is that the event Yahtzee occurs only rarely. We will calculate its probability. Let s find the probability of throwing five 1 s when casting five dice. We write this as: Now, the probability of a Yahtzee is: P(1, 1, 1, 1, 1) = (1/6) 5. P(Y 1 or Y 2 or Y 3 or Y 4 or Y 5 or Y 6 ), where Y k means all five dice land with the side k facing up; in words, Y k means a Yahtzee on the number k. Clearly all of the Y k have the same probability. Thus, by Rule 3, the probability of a Yahtzee is: 6(1/6) 5 = 1/1296 = There is a slicker way to calculate the probability of a Yahtzee. Imagine that you cast the dice one-at-a-time. (Don t play Yahtzee this way; it will really annoy your friends.) No matter how the first die lands, you can still get a Yahtzee. (An ESPN anchor might announce, Ralph is on pace for a Yahtzee! ) To obtain a Yahtzee, your last four dice must match your first one. The probability of this event is (1/6) 4 = , as above. Here is our general definition of independent and identically distributed trials, abbreviated i.i.d.: Random variables X 1, X 2, X 3,... all have the same probability distribution and these random variables are independent. But how do we know we have independent random variables? This question is a bit tricky. Often times, we simply assume it to be true. This is indeed what I did above when I assumed that the outcome of the first cast of the die has no influence on the outcome of the second cast. In other problems, however, we will need to work from first principles to determine whether or not two random variables are independent. Also, when we try to apply these ideas to scientific problems that are more complex than tossing coins or casting dice, we will need to give careful consideration to whether or not independence makes sense scientifically. Finally, we will learn how to use data to investigate whether the assumption of independence is reasonable. Finally (!), I am ready to answer the third of our three questions about probability, posed long ago on page 5 of Chapter 1. Namely, if I determine/state P(A) = 0.25, what does this mean? There is a famous result in probability theory that answers this question. Sort of. In a limited situation. It is called the Law of Large Numbers (LLN). I will try to explain it. 14

5 Let X 1 be a random variable and let A be some event whose occurrence is determined by the value of X 1. Let X 1, X 2, X 3,...X n be independent and identically distributed trials. Clearly, for each of X 2, X 3,...X n we can determine whether or not event A occurs. Then: 1. Count the number of times that A occurs in the first n trials. 2. Divide the frequency you obtained in step 1 by n, to obtain the relative frequency of occurrence of event A in n trials. The LLN states that in the limit, as n becomes larger without bound, the relative frequency of occurrence of event A converges to P(A). Here is an example. (This appears on page 59 of the text.) I programmed my computer to simulate 60,000 independent trials for casting a balanced die (ELC). The results are summarized below. Relative Outcome Frequency Frequency Probability Difference 1 10, , , , , , Total 60, We can see that for each of the six possible outcomes, the relative frequency of occurrence of the outcome is very close to its probability, as predicted by the LLN. For any event larger than a single outcome, the relative frequency of the event will be the sum of the appropriate relative frequencies, which, from the table above, will be close to its probability. To summarize, when we have independent and identically distributed trials, then the probability of an event is approximately equal to its long-run-relative frequency. This result is used in both directions. If we know the probability, we can predict the long-run-relative frequency of occurrence. If, however, we do not know the probability, we can approximate it by performing a large computer simulation and calculating the relative frequency of occurrence. This latter use is much more important to us and we will use it many times in this course. One of the main users of the first application of the LLN are gambling casinos. I will give a brief example of this. An American roulette wheel has 38 slots, each slot with a number and a color. For this example, I will focus on the color. Two slots are colored green, 18 are red and 18 are black. Red is a popular bet and the casino pays even money to a winner. If we assume that the ELC is appropriate for the roulette wheel, then the probability that a red bet wins is 18/38 = But a gambler is primarily concerned with his/her relative frequency of winning. Suppose that one gambler places a very large number, n of one dollar bets on red. By the LLN, the relative frequency of winning bets will be very close to and the relative frequency of losing bets will be very close to = In simpler terms, in the long 15

6 run, for every $100 bet on red, the casino pays out 2(47.37) = dollars, for a net profit of $5.26 for every $100 bet. As a side note, when a regular person goes to a casino, he/she can see that every table game has a range of allowable bets. For example, there might be a blackjack table that states that the minimum bet allowed is $5 and the maximum is $500. Well, a regular person likely pays no attention to the maximum, but it is very important to the casino. As a silly and extreme example, suppose Bill Gates walks into a casino and wants to place a $50 billion bet on red. No casino could/would accept the bet. Why? 2.2 Bernoulli Trials In Section 1 we learned about i.i.d. trials. In this section, we study a very important special case of these, namely Bernoulli trials (BT). If each trial yields one of exactly two possible outcomes, then we have BT. B/c this is so important, I will be a bit redundant and explicitly present the assumptions of BT. The Assumptions of Bernoulli Trials. There are three: 1. Each trial results in one of two possible outcomes, denoted success (S) or failure (F ). 2. The probability of S remains constant from trial-to-trial and is denoted by p. Write q = 1 p for the constant probability of F. 3. The trials are independent. When we are doing arithmetic, it will be convenient to represent S by the number 1 and F by the number 0. One reason that BT are so important, is that if we have BT, we can calculate probabilities of a great many events. Our first tool for calculation, of course, is the multiplication rule that we learned in Section 1. For example, suppose that we have n = 5 BT with p = The probability that the BT yield four successes followed by a failure is: P(SSSSF) = ppppq = (0.70) 4 (0.30) = Our next tool is extremely powerful and very useful in science. It is the binomial probability distribution. Suppose that we plan to perform/observe n BT. Let X denote the total number of successes in the n trials. The probability distribution of X is given by the following equation. P(X = x) = n! x!(n x)! px q n x, for x = 0, 1,..., n. (2.1) To use this formula, recall that n! is read n-factorial and is computed as follows. 1! = 1; 2! = 2(1) = 2; 3! = 3(2)(1) = 6, 4! = 4(3)(2)(1) = 24; 16

7 and so on. By special definition, 0! = 1. (Note to the extremely interested reader. If you want to see why Equation 2.1 is correct, go to the link to the Revised Student Study Guide on my webpage and read the More Mathematics sections of Chapters 2, 3 and 5.) I will do an extended example to illustrate the use of Equation 2.1. Suppose that n = 5 and p = I will derive the probability distribution for X. P(X = 0) = 5! 0!5! (0.60)0 (0.40) 5 = P(X = 1) = 5! 1!4! (0.60)1 (0.40) 4 = P(X = 2) = 5! 2!3! (0.60)2 (0.40) 3 = P(X = 3) = 5! 3!2! (0.60)3 (0.40) 2 = P(X = 4) = 5! 4!1! (0.60)4 (0.40) 1 = P(X = 5) = 5! 5!0! (0.60)5 (0.40) 0 = You should check the above computations to make sure you are comfortable using Equation 2.1. Here are some guidelines for this class. If n 8, you should be able to evaluate Equation 2.1 by hand as I have done above for n = 5. For n 9, I recommend using a statistical software package on a computer. For example, the sampling distribution for X for n = 25 and p = 0.50 is presented on page 161 of the text. Equation 2.1 is called the binomial probability distribution with parameters n and p; it is denoted by Bin(n, p). With this notation, we see that by earlier by hand effort was the Bin(5,0.60) and the table on page 161 of the text is the Bin(25,0.50). Sadly, life is a bit more complicated than the above. In particular, a statistical software package should not be considered a panacea for the binomial. For example, if I direct my computer to calculate the Bin(n,0.50) for any n 1023 I get an error message; the computer program is smart enough to realize that it has messed up and its answer is nonsense; hence, it does not give me an answer. Why does this happen? Well, consider the computation of P(X = 1000) for the Bin(2000,0.50). This involves a really huge number (2000!) divided by the square of a really huge number (1000!), and them multiplied by a really really small positive number ((0.50) 2000 ). Unless the computer programmer exhibits incredible care in writing the code, the result will be an overflow or an underflow or both. But before we condemn the programmer for carelessness or bemoan the limits of the human mind, note the following. We do not need to evaluate Equation 2.1 for really large n s b/c there is a very easy way to obtain a good approximation to the exact answer. Sadly, it will take some time to motivate this approximate method. But without it, there really is no course, so we need to do it. 17

8 Figures on pages 162 and 163 of the text are probability histograms for several binomial probability distributions. Here is how they are drawn. The method I am going to give you works only if the possible values of the random variable are equally spaced on the number line. This restriction will not be a problem in this course. 1. On a horizontal number line, mark all possible values of X: 0, 1, 2,...n. (Not all possibilities are shown in the pictures in the text.) 2. Determine the value of δ (Greek lower case delta) for the random variable of interest. The number δ is the distance between any two consecutive values of the random variable. For the binomial, δ = 1, which makes life much simpler. 3. Above each x = 0, 1, 2,..., n, draw a rectangle, with its center at x and its height equal to P(X = x)/δ. Of course, in the current case, δ = 1, so the height of each rectangle equals the probability of its center value. For a probability histogram (PH) the area of a rectangle equals the probability of its center value, b/c P(X = x) Area = Base Height = δ = P(X = x). δ A PH allows us to see a probability distribution. For example, for the pictures in the text, we can see that the binomial is symmetric for p = 0.50 and not symmetric for p This is not an accident of the four pictures I chose to present; indeed, the binomial is symmetric if, and only if, p = By the way, I am making the tacit assumption that 0 < p < 1; that is, that p is neither 1 nor 0. Trials for which successes are certain or impossible are not of interest to us. (The idea is that if p = 1, then P(X = n) = 1 and its PH will consist of only one rectangle. Being just one rectangle, it is symmetric.) Here are some other facts about the binomial illustrated by the pictures in the text. 1. The PH for a binomial always has exactly one peak. The peak can be one or two rectangles wide, but never wider. 2. If np is an integer, then there is a one-rectangle wide peak located above np. 3. If np is not an integer, then the peak will occur either at the integer immediately below or above np; or, in some cases, at both of these integers. 4. If you move away from the peak in either direction, the heights of the rectangles become shorter. (If the peak occurs at either 0 or n this fact is true in the one direction away from the peak.) It turns out to be very important to have a way to measure the center and spread of a PH. The center is measured by the center of gravity, defined as follows. Look at a PH. Imagine that the rectangles are made of a material of uniform mass and that the number line has no mass. Next, imagine placing a fulcrum in support of the number line. Now look at the PH for the Bin(100,0.50). If the fulcrum is placed at 50, then the PH will balance. B/c 18

9 of this, 50 is called the center of gravity of the Bin(100,0.50) PH. Similarly, for any binomial PH, there is a point along the number line that will make the PH balance; this point is the center of gravity. The center of gravity is also called the mean of the PH. The mean is denoted by the Greek letter mu: µ. It is an algebraic fact that for the binomial, µ = np. It is difficult to explain our measure of spread, so I won t. There are two related measures of spread for a PH: the variance and the standard deviation. The variance is denoted by σ 2 and the standard deviation by σ. As you have probably noticed the variance is simply the square of the standard deviation. Thus, we don t really need both of these measures, yet in some settings it is more convenient to think of the variance and in others it is more convenient to focus on the standard deviation. In any event, for the binomial, σ 2 = npq and σ = npq. We needed to learn about the center and spread so that I can tell you about a very important operation, standardizing. Let X be any random variable with mean µ and standard deviation σ. Then the standardized version of X is denoted by Z and is given by the equation: Z = X µ. σ Before I show you why standardizing is important and useful, let s do a simple example. Suppose that X Bin(3,0.25). You can verify the following facts about X. 1. Its mean is µ = np = 3(0.25) = 0.75 and its standard deviation is σ = Its probability distribution is: 3(0.25)(0.75) = P(X = 0) = ; P(X = 1) = ; P(X = 2) = ; and P(X = 3) = The formula for Z is Z = X Thus, the possible values of Z are: 1, 1/3, 5/3 and 3. Thus, the probability distribution for Z is: P(Z = 1.00) = ; P(Z = 1/3) = ; P(Z = 5/3) = ; and P(Z = 3) =

10 As we can see from this example, converting from X to Z simply changes the values of the random variable; the probabilities for X are inherited by Z. Given that Z has a probability distribution, it has a PH. This is a bit trickier than the binomial with its δ = 1. As a result of standardizing, unless σ = npq = 1 the δ for Z will not be one. In any event, the PH for Z can be drawn. In fact, I have drawn three such PH s for the standardized binomial on pages 164 and 165 of the text. Superimposed on these pictures is the snc. It is obvious visually, at least for the three situations pictured in the text, that the snc should provide a good approximation to binomial probabilities. I will illustrate the method for a number of examples and you will have homework to hone your skills. Suppose that X Bin(100,0.50). I want to calculate P(X 55). With the help of my computer, I know that this probability equals We will now approximate this probability using the snc. First, note that µ = 100(0.50) = 50 and σ = 100(0.50)(0.50) = 5. First look at the picture on page 162 of the text. The probability that we want is the area of the rectangle centered at 55 plus the area of all rectangles to the right of it. The rectangle centered at 55 actually begins at 54.5; thus, we want the sum of all the areas beginning at 54.5 and going to the right. This picture-based-conversion of 55 to 54.5 is called a continuity correction and it greatly improves the accuracy of the approximation. We proceed as follows: P(X 55) = P(X 54.5); this is the continuity correction, next we standardize. P(X 54.5) = P( X µ σ Now we use the snc approximation to get: ) = P(Z 0.90). 5 P(Z 0.90) the area under the snc to the right of 0.90, which equals Our approximation is exact to four digits! Let us summarize and generalize what we have learned from this example. Suppose that X Bin(n, p) and we want to use the snc to approximate P(X x), for some number x. Here is how we proceed. Calculate µ = np, σ = npq and z = (x 0.5 µ)/σ. Next, P(X x) = P(X x 0.5) = P(Z z), which we approximate by finding the area under the snc to the right of z. Or even more briefly: to approximate P(X x), find the area under the snc to the right of z = (x 0.5 µ)/σ. Let s try this out. Suppose that X Bin(100,0.20) and we want to know P(X 17). 100(0.20)(0.80) = 4 and z = ( )/4 = We calculate µ = 100(0.20) = 20, σ = The exact answer is (with the help of my computer) The snc approximate answer is: if I round z to

11 if I round z to Either of these answers is quite good; one is sensational, but I would need to be psychic to choose it. Next, suppose that X Bin(1600,0.50) and we want to know P(X 820). We calculate µ = 1600(0.50) = 800, σ = 1600(0.50)(0.50) = 20 and z = ( )/20 = The exact answer is unavailable b/c of the limitations of my computer mentioned earlier. The snc approximate answer is: if I round z to if I round z to Of course, we might be interested in calculating a probability other than P(X x). In fact, below is a fairly complete list of expressions that might interest us. P(X = x), P(X x), P(X > x), P(X < x), P(a X b), P(a < X b), P(a X < b), and P(a < X < b). It would be too tedious (even for this class!) to show you how to handle each of these. Instead, I will do several examples. The basic idea is to behave like the mathematician in the kitchen, a story that was told in lecture. In short, faced with a question, change it into one or more questions that can be written as P(X x). Here are some examples. For these examples, the values of n, p, µ and σ are irrelevant for my goal; I just want to show you how to rewrite a question into the form we can handle. 1. P(X > 10) = P(X 11). 2. P(X < 20) = 1 P(X 20). 3. P(X 25) = 1 P(X 26). 4. P(X = 16) = P(X 16) P(X 17). 5. P(10 < X < 20) = P(X 11) P(X 20). 6. P(10 X < 20) = P(X 10) P(X 20). 7. P(10 < X 20) = P(X 11) P(X 21). 8. P(10 X 20) = P(X 10) P(X 21). You will have an opportunity to try these out when you do your homework. 21

12 2.3 Finite Populations A finite population is a well-defined collection of individuals. Here are some examples: All students registered in this course. All persons who are currently registered to vote in Wisconsin. All persons who voted in Wisconsin in the 2008 presidential election. Associated with each individual is a response. In this section we restrict attention to responses that have two possible values; called dichotomous responses. As earlier, one of the values is called a success (S or 1) and the other a failure (F or 0). It is convenient to visualize a finite population as a box of cards. Each member of the population has a card in the box, called the population box, and on the member s card is its value of the response, 1 or 0. The total number of cards in the box marked 1 is denoted by s (for success) and the total number of cards marked 0 is denoted by f (for failure). The total number of cards in the box is denoted by N = s + f. Also, let p = s/n denote the proportion of the cards in the box marked 1 and q = f/n denote the proportion of the cards in the box marked 0. For example, one could have: s = 60 and f = 40, giving N = 100, p = 0.60 and q = Clearly, there is a great deal of redundancy in these five numbers; statisticians prefer to focus on N and p. Knowledge of this pair allows one to determine the other three numbers. We refer to a population box as Box(N,p) to denote a box with N cards, of which N p cards are marked 1. Consider the CM: Select one card at random from Box(N, p). After operating this CM, place the selected card back into the population box and repeat. Repeat this process n times. This operation is referred to as selecting n cards at random with replacement. Viewing each selection as a trial, we can see that we have BT: 1. Each trial results in one of two possible outcomes, denoted success (S) or failure (F ). 2. The probability of S remains constant from trial-to-trial. 3. The trials are independent. Thus, everything we have learned (the binomial sampling distribution) or will learn about BT is also true when one selects n cards at random with replacement from Box(N, p). 22