F71SM STATISTICAL METHODS

F71SM STATISTICAL METHODS RJG SUMMARY NOTES 2 PROBABILITY 2.1 Introduction A random experiment is an experiment which is repeatable under identical conditions, and for which, at each repetition, the outcome is uncertain but is one of a known and describable set of possible outcomes. The sample space S is the set of possible outcomes. An event A is a subset of S (but see below). Event A occurs if the outcome of the experiment is an element of the set A. The union of two events A and B, denoted A U B, is the event which occurs at least one of events A,B occurs. The intersection of two events A and B, denoted A I B, is the event which occurs both A, B occur. The complement of event A, denoted A, occurs if and only if the event A does not occur. The empty set φ, considered as a subset of S, contains none of the set of possible outcomes of the experiment and so corresponds to the impossible event. A U A = S, A I A = φ Events A and B are mutually exclusive A I B = φ (that is, the events cannot occur simultaneously). A set function is a function whose domain is a collection of sets. 2.2 Probability Probability is a set function (P) on the collection of subsets of S. When S is uncountable, we must introduce restrictions on the collection of subsets allowable (to maintain mathematical rigour): call this collection F. We require A F A F ; and A 1, A 2, A 3,... F U A i F. Then S F, φ F. i The members of F are measurable and only members of F are considered as events; F is a σ algebra. The domain of the function P is F. For A F, P(A) is a real number which gives the probability that the outcome of the experiment is in A; it is the probability that event A occurs. We require that P behave like relative frequency: it is a function which is non-negative, bounded above (by 1), and additive. Thus leads us to the following definition in which we declare probability as a function subject to the corresponding three axioms. 1

Definition: Probability Given a sample space S with σ algebra F, probability is a set function P: F R such that A1 A2 P(S) =1 P(A) 0 for all A F A3 For A 1, A 2, A 3, F with A i I A j = φ,, i j, Definition: Probability space A probability space is a triple (S, F, P). 2.3 Basic results for probabilities (i) P(φ) = 0 Proof: S = S Uφ and S I φ = φ P(S) = P(S Uφ ) = P(S) + P(φ) by A3; result follows. (ii) For any event A, P(A) 1 Proof: 1 = P(S) = P(A U A ) = P(A) + P(A ) by A2, A3 and result follows by A1. (iii) For any event A, P(A ) = 1 P(A) Proof: from proof of (ii) above (iv) A B P(A) P(B) Proof: A B B = A U (B I A ) P(B) = P(A) + P(B I A ) by A3, since A I (B I A ) = φ and hence P(B) P(A) by A1. (v) Addition rule: P(A U B) = P(A) + P(B) P(A I B) Proof: A U B = A U (B I A ) and A I (B I A ) = φ P(A U B) = P(A) + P(B I A ) B = B I S = B I (A U A ) = (B I A) U (B I A ) and (B I A) I (B I A ) = φ P(B) = P(B I A) + P(B I A ) Together these results P(A U B) = P(A) + P(B) P(A I B) This generalises the result that for A and B mutually exclusive events, P(A U B) = P(A) + P(B). Venn diagram = P U A i P( Ai ) i i 2

2.4 Evaluating probabilities in practice (a) Long term relative frequency [e.g. P(drawing-pin lands with its pin sticking up)] We observe that relative frequency tends to settle down as the number of trials increases: formally we define the probability of the event occurring as the limit of the relative frequency, so relative frequency probability as number of trials. The graph below shows the relative frequency of occurrence of an event with probability 0.4 after 1,2,, 200 trials (the outcomes of the trials were simulated in R). relative frequency 0.0 0.2 0.4 0.6 0.8 1.0 0 50 100 150 200 number of trials (b) Symmetry (finite number of equally likely outcomes) [e.g. P(fair six-sided die lands showing a 6)] in this case # outcomes favourable to A P( A ) = # outcomes possible The key to evaluating probabilities in this case is to define a sample space in a convenient way and then to count the numbers of outcomes corresponding to various events. For example consider a throw of two fair six-sided dice, one red and the other blue. Let A be the event score = 7 or 8. Let S = {(i,j): i = 1,2,3,4,5,6 ; j = 1,2,3,4,5,6} where i and j are the scores on the red and blue die respectively. S consists of 36 elements (equally-likely outcomes). A { (1,6), (2,5), (3,4), (4,3), (5,2), (6,1), (2,6), (3,5), (4,4), (5,3), (6,2)} A consists of 11 elements, so P(A) = 11/36 = 0.3056. 3

(c) Uniform distribution over an interval In the case that the sample space is an interval on the real line and the event occurs at a random point in the interval we adopt a uniform distribution of probability over the interval so that the probabilities of the event occurring in sub-intervals of the same length are equal. For example suppose we choose a time at random between 14.00hrs and 15.00hrs, then P(selected time is before 14.30hrs) = 0.5 P(selected time is between 14.20hrs and 14.45hrs) = 25/60 = 0.4167 P(selected time is before 14.10hrs or after 14.50hrs) = 20/60 = 0.3333 2.5 Conditional probability We introduce the concept of the probability that an event occurs, conditional on another specified event occurring (or, in other language, given that another specified event occurs). For example, consider the event that in a throw of a fair six-sided die we score 6, conditional on scoring more than 2. The event scoring more than 2 corresponds to the 4 equally-likely outcomes {3,4,5,6} and of these only 1 outcome corresponds to score of 6, so the probability required is 1/4. Imposing the condition has effectively reduced/restricted the sample space from {1,2,3,4,5,6} to {3,4,5,6}. Note that the conditional probability can be expressed as the ratio of two unconditional probabilities of events defined in terms of the original sample space of size 6 by 1 = 1/ 6. 4 4 / 6 Again, consider a throw of two fair six-sided dice, one red and the other blue. Let A be the event score = 7 or 8 and let B be the event score = 8, 9 or 10. Let S = {(i,j): i = 1,2,3,4,5,6 ; j = 1,2,3,4,5,6} where i and j are the scores on the red and blue die respectively. A { (1,6), (2,5), (3,4), (4,3), (5,2), (6,1), (2,6), (3,5), (4,4), (5,3), (6,2)} B { (2,6), (3,5), (4,4), (5,3), (6,2), (3,6), (4,5), (5,4), (6,3), (4,6), (5,5), (6,4)} P(A) = 11/36, P(B) = 12/36 A B is the event score of 8 and P(A B) = 5/36 11 elements 12 elements Consider the event A conditional on B, that is a score of 7 or 8 given that the score is 8, 9, or 10. The outcomes in B favourable to A are (2,6), (3,5), (4,4), (5,3), (6,2) so the probability of event A conditional on B is 5/12. This probability is (5/36)/(12/36) = P(A B)/ P(B). 4

This motivates the general definitions: The probability of event A conditional on event B is denoted P ( A B ) and is defined as ( ) P A B ( B) P ( B) Similarly for event B conditional on event A: P ( B A) P A = for P(B) 0. ( B) P ( A) P A = for P(A) 0. The multiplication rule for probabilities follows, namely P(A I B) = P(A)P(B A) = P(B)P(A B). For example, suppose we draw two balls at random, one after the other and without replacement, from a bag containing 6 red and 4 blue balls. Let A first ball drawn is red, and let B second ball drawn is blue. P(1 st ball drawn is red and 2 nd ball drawn is blue) = P(A I B) = P(A)P(B A) = 6 4 = 4 = 0.2667. 10 9 15 2.6 Independent events and independent trials Events A and B are independent P(A I B) = P(A) P(B) [ P(A B) = P(A) and P(B A) = P(B)] So events A and B are independent if and only if the occurrence of one does not affect the probability of occurrence of the other. Note that events A 1, A 2,, A k are independent if and only if the probability of the intersection of any 2, 3,, k of the events equals the product of their respective probabilities. So, for three events A, B, and C to be independent, we require P(A I B) = P(A) P(B), P(A I C) = P(A) P(C), P(B I C) = P(B) P(C) and P(A I B I C) = P(A) P(B) P(C). A trial is a single repetition of a random experiment. T 1 and T 2 are independent trials all events defined on the outcome of T 1 are independent of all events defined on the outcome of T 2. 2.7 Odds The odds on an event occurring are given by the ratio of the probability that the event will occur to the probability that it will not occur (provided neither probability is zero). So, for an event A with P(A) = 2/3, the odds on A occurring are 2 to1. For an event A with probability P(A) < 0.5, we usually express the odds as odds against the event occurring that is the ratio of the probability that the event will not occur to the probability that it will occur. So, for an event A with P(A) = ¼, we say that the odds against A occurring are 3 to 1. If P(A) = 0.5, we say that the odds are evens. If the odds on an event occurring are a to b, the probability of its occurring is a a + b. 5

2.8 Partitioning of an event Let {E 1, E 2,, E k } be a partition of S and let A be an event. Then A = A S = A ( E ) = ( A E ) and P ( A) = P ( A E ) = P( E ) P( A E ) i i k k i i i i= 1 i= 1 The event A has been partitioned into events A I E i, i = 1,2,, k, f or example, with k = 4: E 1 E 2 A E 3 E 4 P(A) is the sum of the probabilities of the events which make up the partition of A. 2.9 Bayes theorem Let {E 1, E 2,, E k } be a partition of S and let A be an event. Then ( ) ( ) P ( A) ( ) ( ) ( ) ( ) P Ei A P Ei P A Ei P Ei P A Ei P Ei A = = =, i = 1,2,..., k k P( A) P E i= 1 ( ) P ( A E ) The denominator does not depend on i and so the result is often written in proportional terms: ( ) ( ) ( ) P E A P E P A E, i = 1, 2,..., k i i i The probabilities P(E i ), i = 1, 2,, k are the prior probabilities; the probabilities P(E i A), i = 1, 2,, k are the posterior probabilities. For example, suppose a population is made up of 60% men and 40% women. The percentages of men and women in the population who have an ipod are 30% and 40% respectively. A person is selected at random from the population and is found to have an ipod. i i 6

tree diagram ipod 0.3 0.18 male 0.6 no ipod 0.7 0.42 ipod 0.4 0.16 female 0.4 no ipod 0.6 0.24 P(selected person is male) = P(person is male has ipod) = 0.18 0.18 = = 0.5294 0.18 + 0.16 0.34 * * * * * * * * * * A selection of worked problems follows. 7

2.10 Worked examples Ex2.1 In a family with five children, what is the probability that all the children are of the same sex? What is the probability that the three oldest children are boys and the two youngest are girls? What is the probability that the three oldest children are boys? A suitable sample space is S = {(x 1,x 2,x 3,x 4,x 5 ) : x i = M, F ; i = 1,2,3,4,5} where x i is the sex of the i th oldest child. Size of sample space : n(s) = 2 5 = 32 Two outcomes are favourable to the event "all are same sex", namely (M,M,M,M,M) and (F,F,F,F,F). One outcome is favourable to the event "three oldest are boys and two youngest are girls", namely (M,M,M,F,F). 2 2 = 4 outcomes are favourable to the event "three oldest are boys", namely any one of the form (M,M,M,, ) If we make the assumption that all 32 outcomes are equally likely (under what conditions is this a reasonable assumption?) then: P(all same sex) = 2/32 = 0.0625 P(three oldest are boys and two youngest are girls) = 1/32 = 0.0313 P(three oldest are boys) = 4/32 = 0.125. Ex2.2 In how many ways can we choose 4 people from a group of 7 and line them up? 7 6 5 4 = 840 Ex2.3 In how many ways can we choose 6 numbers from a group of 49 for a line in the UK National Lottery? 49 = 49!/(6!43!) = (49 48 47 46 45 44)/(6 5 4 3 2 1) = 13,983,816 6 Note: This leads to the often-quoted "chance of 1 in 14 million" of winning the jackpot. Ex2.4 A fair die is thrown 4 times. Find P(total score is 4 or 24). S = {(x 1,x 2,x 3,x 4 ) : x i = 1,2,3,4,5,6 ; i = 1,2,3,4} : n(s) = 6 4 = 1296 Two outcomes are favourable to the event "total score is 4 or 24", namely (1,1,1,1) and (6,6,6,6). So P(score 4 or 24) = 2/1296 = 0.0015 [Note: We are effectively taking a random sample of size 4 with replacement from the population {1,2,3,4,5,6}]. Ex2.5 A committee consists of 7 men and 4 women and a sub-committee of 6 is to be chosen at random. Find the probabilities that the sub-committee contains exactly k women, k = 0,1,2,3,4. # ways of choosing the sub-committee = # different selections of 6 people from 11 = 11 6 4 # different selections of k women from 4 = k 8

7 # different selections of (6 k) men from 7 = 6 k 4 7 # ways of choosing a sub-committee containing k women = k 6 k P(sub-committee contains exactly k women) = 4 7 k 6 k 11 6 So, for example, P(sub-committee contains exactly 2 women) = 5/11 = 0.4545 Ex2.6 Consider the following gamble : you pay a stake of 1 to throw a pair of fair dice - if you throw double 1 or double 6 you win 15 (and get your stake back) - otherwise you lose your stake. How much can you "expect" to win/lose per game? P(throw a sum of 2 or 12) = 2/36 = 1/18 ; P(other sum) = 17/18 Consider 180 games: you would expect a profit of 15 in each of 10 games (total profit 150) and a loss of 1 in each of 170 games (total loss 170), giving an expected net loss of 20. So expected loss per game = 20/180 = 11.1 p We can get the answer by considering just 1 game: Profit is either 15 (with probability 1/18) or 1 (with probability 17/18) Expected profit per game = [15(1/18) + ( 1)(17/18)] = 2/18 = 11.1 p In general: expected value = Σ (value probability of that value occurring) (the sum being taken over all possible values) Ex2.7 A fair coin is tossed 4 times and comes up heads 3 times. Find the probability that the coin came up heads on the first toss. S = {(x 1,x 2,x 3,x 4 ) : x i = H,T ; i = 1,2,3,4}: n(s) = 2 4 = 16 Let A be the event "3 heads in 4 tosses" and B be the event "head on first toss". We require P(B A). There are 3 outcomes are favourable to the event A I B, namely (H,T,H,H), (H,H,T,H), and (H,H,H,T). So P(A I B) = 3/16 There are 4 outcomes are favourable to the event A, namely (H,H,H,T), (H,H,T,H), (H,T,H,H) and (T,H,H,H). So P(A) = 4/16 P(B A) = P(A I B)/P(A) = (3/16)/(4/16) = 3/4 = 0.75 (this is not a surprise!) OR With the condition imposed, we have a restricted sample space of only 4 elements, namely (H,H,H,T), (H,H,T,H), (H,T,H,H) and (T,H,H,H). Of these 4 outcomes, 3 are favourable to B. Hence 3/4 as above. 9

Ex2.8 A bag contains 4 white and 3 red balls. Two balls are drawn out at random without replacement. What is the probability that they are white and red respectively? What is the probability that the second ball drawn is white? Formally: let W i be i th ball drawn is white, R i be i th ball drawn is red P(W 1 I R 2 ) = P(W 1 )P(R 2 W 1 ) = (4/7)(3/6) = 2/7 We partition W 2 : W 2 = W 2 I (W 1 U R 1 ) = (W 2 I W 1 ) U (W 2 I R 1 ) these two events are mutually exclusive P(W 2 ) = P[(W 2 I W 1 ) U (W 2 I R 1 )] = P(W 2 I W 1 ) + P(W 2 I R 1 ) = P(W 1 )P(W 2 W 1 ) + P(R 1 )P(W 2 R 1 ) = (4/7)(3/6) + (3/7)(4/6) = 2/7 + 2/7 = 4/7 = 0.5714 It is very easy to sort out all possibilities with a "tree diagram": First draw Second draw W 2 3/6 2/7 W 1 4/7 R 2 3/6 2/7 W 2 4/6 2/7 R 1 3/7 R 2 2/6 1/7 Note: P(W 2 ) = P(W 1 ) Ex2.9 People are selected at random, one after the other, from a population consisting of 15 men and 10 women. What is the probability that the second person selected is male? What is the probability that the sixth person selected is male? All we have to note is P(first person selected is male) = 15/25 = 0.6 With no conditional information involved, and no further information given, we have immediately P(2 nd is male) = P(6 th is male) = P(1 st is male) = 0.6 10

Ex2.10 There are two boxes which contain balls as follows. Box 1 has 6 white and 4 red, box 2 has 2 white and 8 red. One box is chosen at random and from it a ball is chosen at random. The ball drawn is red. What is the probability that this ball came from box 1? What is the probability that a second ball drawn from the same box as the first will also be red? Let A i be the event "box i is chosen". Let R(W) be event "ball drawn is red (white)" Choice of box Draw of ball R 4/10 2/10 A 1 1/2 W 6/10 3/10 R 8/10 4/10 A 2 1/2 W 2/10 1/10 P(R) = P(A 1 )P(R A 1 ) + P(A 2 )P(R A 2 ) = (1/2)(4/10) + (1/2)(8/10) = 6/10 = 0.6 P(A 1 R) = P(A 1 )P(R A 1 )/P(R) = (2/10)/(6/10) = 1/3 = 0.3333 So P(A 1 R) = 1/3 and hence P(A 2 R) = 2/3. These are our reassessed probabilities that our chosen box is box 1, box 2 respectively. If box 1, our box now contains 6 white and 3 red ; if box 2, our box now contains 2 white and 7 red. R 3/9 1/9 Box 1 1/3 W 6/9 2/9 R 7/9 14/27 Box 2 2/3 W 2/9 4/27 So P(second ball drawn is red) = 1/9 + 14/27 = 17/27 = 0.6296 11

Ex2.11 Andrew and Brian play a round of golf. The probability that Andrew (Brian) gets a 4 at the first hole is 0.3 (0.6). Assuming independence, find the probability that at least one of them gets a 4 at the first hole. Let A (B) be the event "Andrew (Brian) gets a 4" Method 1: P(A U B) = P(A) + P(B) P(A I B) = P(A) + P(B) P(A)P(B) (by independence) = 0.3 + 0.6 0.18 = 0.72 Method 2: P(neither gets a 4) = P(A' I B') = P(A')P(B') = 0.7 0.4 = 0.28 P(at least one gets a 4) = 1 0.28 = 0.72 Ex2.12 Find the probabilities of getting (a) exactly 4 sixes in 7 throws of a fair die and (b) 5 or 6 heads in 6 tosses of a fair coin. In each case, we have a known number of i.i.d. trials. 4 3 6 6 7 1 5 6 1 1 ( a) 0.0156 ( b) 0.1094 4 = + = 6 6 5 2 2 Ex2.13 Sampling inspection Regular sampling of mass-produced items is carried out by taking random samples of 8 items from the production line. Each selected item is tested to find out if it is defective. Assuming independence from item to item, and assuming that 10% of the production is defective, we can adopt a model with i.i.d. trials and with P(selected item is defective) = 0.1. 8 2 6 P(sample contains 2 defective items) = ( 0.1 ) ( 0.9 ) = 0.1488 2 Note: Our production of items is finite. Strictly speaking, as we sample items one after another the successive trials are not independent: the outcome of one trial conditions the probabilities associated with all later trials. e.g. P(2 nd item defective 1 st item OK) P(2 nd item defective) and P(1 st item OK and 2 nd item defective) P(1 st item OK)P(2 nd item defective) However, if the population of items is large (as in the above illustration) and the sample of moderate size, it is reasonable to adopt a model of independent trials with constant probability of an item being defective i.e. a model of i.i.d. trials. We are in effect assuming an unchanging population as an approximation to a population which is actually changing slightly from trial to trial we are using the theory of sampling with replacement to approximate the real situation, which is sampling without replacement. Examples 2.14 2.17 involve the use of non-finite (countable and non-countable) sample spaces. 12

Ex2.14 Players A and B throw a regular 6-sided die in turn. The first to throw a 6 wins the game. A throws first. Find the probability that A wins the game. We can represent the sample space as the countable union of the events E k, where E k game ends on the k th throw of the die, k = 1,2,3, A B A B A B Probability E 1 6 1/6 E 2 6 6 (5/6)(1/6) E 3 6 6 6 (5/6) 2 (1/6) E 4 6 6 6 6 (5/6) 3 (1/6) E 5 6 6 6 6 6 (5/6) 4 (1/6) where 6 denotes not a 6. P(A wins) = P(E 1 E 3 E 5 ) = P(E 1 ) + P(E 3 ) + P(E 5 ) + = 1/6 + (5/6) 2 (1/6) + (5/6) 4 (1/6) + = (1/6)[1 + 25/36 + (25/36) 2 + ] = (1/6)(1 25/36) 1 = (1/6)(36/11) = 6/11 = 0.5455 Note: the advantage, of course, lies with the player who throws first. OR Let p = P(A wins) A wins A wins on first throw die passes to B and B does not win the game After the die passes to B, B is then in exactly the same position as A was at the start of the game, so p = 1/6 + (5/6)(1 p) which gives p = 6/11 Ex2.15 A and B play a series of games, each of which A wins with probability p. A game cannot be drawn. A player wins the series by being the first to win two games in succession. Find the probability that A wins the series. In an obvious notation: S {AA, BB, BAA, ABB, ABAA, BABB, BABAA, ABABB, } {AA, BAA, ABAA, BABAA, } {BB, ABB, BABB, ABABB, } P(A wins series) = P(AA BAA ABAA BABAA ABABAA ) = P(AA) + P(BAA) + P(ABAA) + P(BABAA) + P(ABABAA) = p 2 + (1 p)p 2 + (1 p)p 3 + (1 p) 2 p 3 + (1 p) 2 p 4 + = {p 2 + (1 p)p 3 + (1 p) 2 p 4 + } + {(1 p)p 2 + (1 p) 2 p 3 + } = p 2 {1 p(1 p)} 1 + (1 p)p 2 {1 p(1 p)} 1 = p 2 (2 p){1 p(1 p)} 1 13

Ex2.16 A stick of length 12cm is broken into two pieces at a point chosen at random along its length. What is the probability that the rectangle which can be constructed using the pieces as two adjacent sides has an area less than 27cm 2? Lay the stick down and let x be the distance of the break point from the left end of the stick. 0 12 x S = {x: 0 < x < 12} The probability measure is length all points in S are equally likely. Area of rectangle = x(12 x) which is less than 27 for x < 3 or x > 9. Probability of event x < 3 or x > 9 = 6/12 = 0.5 Ex2.17 Two students are each, independently of the other, equally likely to arrive for a 10.15 lecture at any time between 10.15 and 10.30. Find the probability that their arrivals are separated by at least 10 minutes. Call the students A and B and let x, y be their respective arrival times in minutes after 10.15. S = {(x,y): 0 < x < 15, 0 < y < 15} The probability measure is area all points in S are equally likely. Area for which the students arrive more than 10 minutes apart is x y > 10, which has area 25 (draw a diagram to see this); area of S = 15 2 = 225. Probability required = 25/225 = 1/9 = 0.1111. Ex2.18 Ten men and five women occupy fifteen adjacent seats in one row at a theatre. If the seats are allocated to the fifteen people at random, find the probabilities that: (a) the seats at the ends are both occupied by women; (b) the arrangement is symmetrical by sex; (c) no two women sit next to each other. 15 We can think of fifteen numbered seats (1,2,, 15) and use a sample space of = 3003 5 equally-likely outcomes, each point in which specifies the five seats occupied by the women: for example the outcome seat 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 sex of occupant M M W M M W W M M M M W M W M is the point (3,6,7,12,14). (a) With women in places 1 and 15, we have to place the remaining 3 women in 13 possible seats. 13 This can be done in = 286 ways, so probability = 286/3003 = 2/21 = 0.0952 3 14

(b) To be symmetrical by sex there must be a woman in the middle seat and 5 men and 2 women on each side. We can choose the two seats for women on one of the sides in 7 = 21 ways and our choice must then be reflected on the other side (there is only 1 2 way this can be done), so required probability = 21/3003 = 1/143 = 0.0070 (c) If we think of the 10 men standing in a row prior to sitting down, each woman can go in one of 11 different positions M M M M M M M M M M If we are to have no two women next to each other we have to choose 5 different positions 11 from these 11 to allocate to the women. This can be done in = 462 ways, so required 5 probability = 462/3003 = 2/13 = 0.1538` 15