What are the odds? Coin tossing and applications

Size: px

Start display at page:

Download "What are the odds? Coin tossing and applications"

Jeremy Morton
6 years ago
Views:

1 What are the odds? Coin tossing and applications Dr. Antal Járai Department of Mathematical Sciences University of Bath 19 July 2011 Please feel free to interrupt with questions at any time.

2 Outline This morning will be a taster of what university level mathematics is like, with a focus on Probability and Statistics. There will be three parts: 1. Coin tossing and its mathematics 2. Application in Statistics: estimating unknown proportions 3. Exciting mathematical questions related to coin tossing 1

3 1. Coin tossing and its mathematics Randomness: Many phenomena in the world around us are unpredictable in advance. Often we are able to list the possible outcomes, but we are not able to predict which outcome will happen. A phenomenon whose outcome can be unambiguously observed is called an experiment. Examples: Toss a coin. Possible outcomes: H and T. Roll a die. Possible outcomes: 1, 2, 3, 4, 5, 6. A raffle. Possible outcomes: each ticket in the box. Blow a soap bubble. Where is it going to be after 3 seconds? Possible outcomes: points in some region of 3-dimensional space. A garden bucket contains some water that is entirely motionless. A pollen particle falls on the surface of the water. Where will the particle be after 5 minutes? Possible outcomes: points in a disk, representing the surface of the water. 2

4 Probability Models: Often we have a model of how likely each outcome is. We call a coin fair if H and T are equally likely. We call a die fair if all 6 outcomes are equally likely. It is of course possible for a coin or a die not to be fair. In a raffle, if the tickets in the box were well mixed, then each ticket will be equally likely to be drawn. Measuring how likely : When there is a list of outcomes, we assign to each outcome a number, requiring that the numbers add up to 1. These numbers are called probabilities. Toss a fair coin. P[H] = 1/2 = P[T ]. Raffle (well mixed). P[any ticket] = 1/(number of tickets). A good model also exists for the pollen particle, but it is more technical to write down. This example will re-appear later. Modelling the motion of the soap bubble would be quite complex. 3

5 Probability Interpretation of probabilities: Intuition says and experience shows that if we repeat the experiment a large number of times under the same conditions, then in the long run each outcome will occur with a frequency close to its probability. How close will depend on the number of repetitions. In a proper mathematical formulation of probability, this can also be proved as a mathematical theorem. (See the Law of Averages later.) Examples: Tossing a fair coin repeatedly. In a large number of tosses, about half of the tosses come out to be H and about half to be T. Rolling a die repeatedly. In a large number of rolls, each number comes up about 1/6-th of the time. 4

6 A loaded die an example of unequal probabilities Suppose a die has been loaded, so that the numbers 2, 3, 4, 5 are twice as likely as 1, and 6 is three times as likely as 1. Then the assignment of probabilities becomes: P[1] = 1 2, P[2] = P[3] = P[4] = P[5] = 12 12, P[6] = Check: these add up to 1 and have the correct proportions. If the loaded die were to be rolled a large number of times, then: 2, 3, 4, 5 would come up about 1/6-th of the time; 1 would come up about 1/12-th of the time; and 6 would come up about a quarter of the time. 5

7 Repeated coin tossing Tossing a fair coin 3 times. Each toss can come out 2 ways, so there are = 2 3 = 8 possible outcomes. These can easily be listed: HHH, HHT, HTH, HTT, THH, THT, TTH, TTT. All outcomes are equally likely. Events: We can ask what is the probability that a certain event will happen. For example, P[all three tosses have the same result] = P[HHH] + P[TTT] = 2 8. P[exactly two H s] = P[HHT] + P[HTH] + P[THH] = 3 8. See: Worksheet 1. 6

8 The Law of Averages Assume a fair coin. Chance error: This is the number of H s minus half the number of tosses, so that: number of H s = half the number of tosses + chance error. Absolute size of chance error: As the number of tosses is increased, the likely size of the chance error increases in absolute terms. Relative size of chance error: As the number of tosses is increased, the likely size of the chance error decreases in percentage terms (relative to the number of tosses). 7

9 The Law of Averages Example: In 100 tosses, the likely size of the chance error is about 5. In tosses, the likely size of the chance error is about 50. In absolute terms: As the number of tosses has been increased by a factor of 100, the likely size of the chance error increased by a factor of 10 = 100 in absolute terms. In percentage terms: In 100 tosses the likely size of the chance error is about 100 (5/100)% = 5%, whereas in tosses the likely size of the chance error is about 100 (50/10000)% = 0.5%. As the number of tosses has been increased by a factor of 100, the likely size of the chance error decreased by a factor of 10 = 100, in percentage terms. 8

10 The Law of Averages Square-root Law: If we increase the number of tosses by a factor k, the likely size of the chance error increases by a factor k in absolute terms, and decreases by a factor k in percentage terms. The Square-root Law also holds if the coin is not fair. 9

11 2. Application in Statistics: estimating unknown proportions Problem 1. We want to estimate the percentage of unemployed people in Bath. Population: 80,000. Problem 2. We want to estimate the percentage of unemployed people in the UK. Population: 60,000,000. Note: We want to estimate the actual percentage, rather than relying on records that may not capture all people. Idea: Select a random sample from the population for interview, and use their data as a guide for the whole population. 10

12 A box model Description: Imagine a large box that contains 80,000 tickets, one for each resident of Bath. The ticket has the number 1 on it if that person is unemployed, and 0 on it otherwise. Chance procedure: We draw 100 tickets at random from the box. After each draw the ticket is replaced in the box, so that conditions are exactly the same before each draw. We count the number of tickets that has the number 1 on it. Estimate: Our estimate of the percentage of unemployed people in the whole population is the percentage of 1 s among the tickets drawn. 11

13 A box model Example: Suppose for sake of an example that the unemployment rate is exactly 5%. That means that there are = 4000 tickets in the box with 1 on it, and tickets with 0 on it. Probability of drawing 1: On each draw, the probability that we draw 1 is 4000/80000 = 0.05 = 5%. Equivalence to coin tossing: Observing the 1 s and 0 s drawn is mathematically equivalent to tossing a coin that lands 1 with chance 5% and lands 0 with chance 95%. Chance error: In 100 draws, we expect to get about 5 tickets with 1 on it. The actual number is subject to chance error: number of 1 s = (5/100) (number of tickets drawn) + chance error. 12

14 Accuracy of the estimate Law of averages: As the number of tickets drawn increases, the chance error decreases in percentage terms, according to the square-root law. Example 1. Suppose still that the unemployment rate in Bath is exactly 5%, but pretend we did not know that. We draw 100 tickets. It is possible to compute the likely size of the chance error (we do not do this here). It comes out to be about 2, which is 2% of 100. So if we draw 100 tickets, our estimate is likely to be off by about 2%. Example 2. Suppose we increase the number of tickets drawn to 1600 (a 16-fold increase). By the square-root law, the likely size of the chance error decreases in percentage terms by a factor 16 = 4. So our estimate is likely to be off by about 0.5%. 13

15 The UK population Box model: For the UK population we can use a very similar box model: there are 60,000,000 tickets, one for each resident of the UK. A ticket has a 1 on it, if the corresponding person is unemployed, and has a 0 on it otherwise. We draw 100 tickets at random from the box. Example: Suppose, for sake of the example, that the unemployment rate in the UK is exactly 5%. Then there are 3,000,000 tickets with 1 on them, and 57,000,000 tickets with 0. Probability of drawing 1: The percentage of tickets with 1 on it is 5%, so on each draw, the probability of drawing 1 is 5%. Coin tossing: The procedure is mathematically equivalent to tossing a coin that lands 1 with chance 5% and lands 0 with chance 95%. This is the same coin tossing problem as before. 14

16 Accuracy of the estimate Key point: The size of the population does not matter for how large the chance error is. Since we assumed the same percentage of unemployment for the UK population as for Bath, both lead to the same coin tossing problem: chance of getting a 1 is 5%. The likely size of the chance error will depend on the number of tickets drawn. Example 1. Draw 100 tickets. We expect to get around 5 tickets marked 1. The likely size of the chance error is about 2%. Example 2. Draw 1600 tickets (a 16-fold increase). The likely size of the chance error is about 0.5%. Summary. The likely size of the chance error depends on: the (unknown) percentage in the whole population and the number of tickets drawn. It does not depend on: the size of the whole population. 15

17 Practical considerations How to sample at random in practice? This is a serious question. In order for the methods above to be valid, each person in the whole population has to have the same probability to be selected. What does not work: Asking 100 passers by on the street whether they are unemployed or not. Not everyone has the same probability to be in the sample. In fact, it is impossible to determine the probability of being selected. Therefore, the error cannot be analyzed, and the result is useless. What is needed: The probability of being in the sample should be uniform. Realizing this is costly, even with moderate sample size like Practice: Statisticians devised methods that are cheaper and easier to carry out in practice. In such methods it is well understood how likely it is to be in the sample, and the chance error can be analyzed. 16

18 3. Exciting mathematical questions related to coin tossing See R plots. Random walk. Consider a d-dimensional cubic grid of points, d = 1, 2, 3. A particle moves about on the grid with the following rules. The particle starts at the origin of the grid. At each time step the particle jumps to one of the neighbouring grid points, each possible jump being equally likely. When d = 1, this is equivalent to coin tossing: step to the right if H, and step to the left if T. When d = 2, 3, we have a generalization of coin tossing. There are many exciting questions one can ask about random walk. We look at a few of them. These are topics that would be discussed in university level mathematics courses. 17

19 Typical distance Question 1. After 100 steps, how far is the particle likely to be from the starting point? How about after steps? Answer 1. Regardless of the value of d, this question will obey a square-root law. After 100 steps, on average, the particle will be about 100 = 10 units away from the starting point. After steps, on average, the particle will be about = 100 units away from the starting point. Here on average means, that the distance may be more or less, but it will be unlikely to be more than a few times this amount. 18

20 Return to starting point? Question 2. Will the particle ever come back to the origin? Answer 2. Here the answer will depend on the value of d. Theorem. [Pólya; 1920] If d = 1 or 2, then the particle will eventually revisit the starting point with probability 1. That is: if d = 1, 2 then P[particle revisits the origin eventually] = 1. On the other hand, if d = 3 then there is a chance that the particle will never revisit the starting point: if d = 3, then P[particle revisits the origin eventually] < 1. If d 4, the same happens as in d = 3. 19

21 Changing the grid Question 3. If in d = 2 we take a hexagonal grid (honey-comb lattice), would the process be very different? How about in other dimensions? Answer 3. The phenomena in Questions 1 and 2 would be essentially the same. The typical distance will still be about the square-root of the number of steps. The particle will return to the origin with probability 1 exactly when the dimension of the grid is d = 1 or 2. 20

22 Diffusion Question 4. Does random walk occur in nature? Answer 4. Very much so. The pollen particle on the surface of a bucket of water is constantly bombarded by water molecules from all sides that keep pushing it in random directions. The cummulative effect of these bombardments is that the pollen particle moves about on the surface of the water in a random way. A very simple, but quite accurate model for this is random walk, where the size of the steps are on the molecular scale, but the macroscopic effect is easily visible under a micorscope. Discoverers: The above type of motion has been observed by botanist Robert Brown in The reason for the motion, as a result of bombardments by molecules, was explained by Albert Einstein in Since then such processes are well known in the physical sciences as diffusion or Brownian motion. 21

23 References [1] David Freedman, Robert Pisani and Roger Purves: Statistics. First edition, W.W. Norton & Company, Inc., New York (1978). Fourth paperback edition (2007). If you are new to statistics, this is a very good, non-technical introduction with plenty of interesting examples. At university, you would go well beyond this book in mathematical sophistication, but the depth of statistical insight it provides is well worth it. [2] William Feller: An Introduction to Probability Theory and its Applications. Third edition, Wiley (1968). This is a classic introductory text on probability theory, written by an expert in the field. THANK YOU FOR YOUR ATTENTION. 22

Events A and B are said to be independent if the occurrence of A does not affect the probability of B.

Independent Events Events A and B are said to be independent if the occurrence of A does not affect the probability of B. Probability experiment of flipping a coin and rolling a dice. Sample Space: {(H,