Probability and random variables. Sept 2018

Size: px

Start display at page:

Download "Probability and random variables. Sept 2018"

Albert Taylor
5 years ago
Views:

1 Probability and random variables Sept 2018

2 2 The sample space Consider an experiment with an uncertain outcome. The set of all possible outcomes is called the sample space. Example: I toss a coin twice, each time noting whether it lands heads (H) or tails (T). The four possible outcomes are: HH, HT, TH, and TT.

3 3 Events An event is a subset of the sample space. For example, in the coin-tossing experiment, Event No heads one head two heads Outcomes TT HT, TH HH These three events are mutually exclusive and exhaustive. Some other events: First throw gives a head: Same result on both throws: At least one head: HH, HT HH, TT HT, TH, HH

4 4 Combining events If A and B are any two events, A B is the event either A or B occurs or both A B is the event both A and B occur. For example, in the coin-tossing experiment, A first toss is H B one H, one T A B at most one T A B simple event HT

5 5 Probability A box contains R red balls and B black balls. One ball is selected from the box at random (i.e. each ball has the same probability of being selected). What is the probability that the selected ball is red? If there are no red balls in the box (R = 0), this event is impossible. We assign zero probability to an impossible event. If all balls are red (B = 0), this event is certain to occur. We assign a probability of 1 to a certain event.

6 6 Probability The probability of an event is the sum of the probabilities of the outcomes associated with the event. For the present example: It is certain that one ball will be selected, so sum of probabilities over all R + B balls is 1. Therefore, Probability that any particular ball is selected is 1/(R + B). Probability that the selected ball is red is the sum of R probabilities, each of which is equal to 1/(R + B), so Pr(selected ball is red) = R (R + B) (the proportion of red balls in the box.)

7 7 Probability From considerations of symmetry, or because randomisation has been used, we can sometimes assume that all outcomes are equally probable. The probability of event E is then number of outcomes in E total number of outcomes For example, if the coin is fair, all four outcomes of the coin-tossing experiment are equally probable. The probabilities of getting 0, 1, or 2 heads are then 1/4, 1/2, and 1/4.

8 8 Probability of A or B For mutually exclusive events A and B, P(A B) = P(A) + P(B) In general, P(A B) = P(A) + P(B) P(AB). For three events, P(A B C) is P(A) + P(B) + P(C) P(AB) P(AC) P(BC) + P(ABC) (writing AB for A B, etc.)

9 9 Sampling without replacement A population consists of n objects. A sample from this population of size s is selected without replacement. How many possible samples can be drawn? If we distinguish between samples which comprise the same objects, but drawn in a different order, the answer is n(n 1) (n s + 1) For the special case s = n, the answer is n! = n (n factorial, the number of ways of arranging n objects).

10 10 Sampling without replacement Usually we regard two samples as identical if they differ only in the order in which the objects were drawn. What is the total number of samples in this case? The samples previously regarded as different now form a number of groups each of size s!, with the members within each group regarded as identical. The number of such groups is therefore ( ) n n(n 1) (n s + 1) = s s!

11 11 Example A box contains 4 balls numbered 1 to 4. Select two balls without replacement. (1,2) (1,3) (1,4) (2,1) (2,3) (2,4) (3,1) (3,2) (3,4) (4,1) (4,2) (4,3) If we take account of ordering, there are 12 simple events. If we disregard ordering, there are only six: (1,2), (1,3), (1,4), (2,3), (2,4), (3,4), where, e.g., (1,2) now denotes either 1 then 2 or 2 then 1.

12 12 Example A box contains 5 balls numbered 1 to 5. Select two balls (without replacement). How many samples of size 2? (ignore order) Answer: (5 4)/2 = 10. How many samples of size 3? (ignore order) Answer: (5 4 3)/(3 2) = 10. Why are these numbers the same? Each selection of two balls to be included is also a selection of three balls to be excluded from the sample.

13 13 Pascal s triangle Based on the result ( ) n = s ( ) n 1 + s 1 ( n 1 s ) and so on... If 1,1 is the second row of Pascal s triangle, what is the seventh row? (University Challenge, 2nd April, 2018).

14 14 Sampling with replacement A fair coin, with two sides (H and T), is tossed twice. This is equivalent to drawing a sample of size 2 with replacement from a box containing two balls labelled H and T. There are 4 possible samples/outcomes: HH, HT, TH, TT. Note that when sampling is without replacement, there are only two possible samples: HT and TH.

15 15 Random sampling Random sampling is a procedure which ensures each possible sample has a known probability of being chosen. Usually each sample has an equal probability. Example: a fair coin is tossed twice. We assign equal probabilities of 1/4 to each possible outcome. Consider the mutually exclusive and exhaustive events no heads, one head, and two heads. No. of heads Total Probability 1/4 1/2 1/4 1 This is an example of a probability distribution.

16 16 Random variables A famous historical data set gives the following chest sizes (inches) for a regiment of Scottish soldiers: Total One individual is chosen randomly from the population: let X be the height of the selected individual. X is a random variable: its value is known only after selection.

17 17 Random variables Before selection, all that can be said about X with certainty is that its value will be one of the values 33, 34, etc. However, probabilities can be assigned: Pr(X = 33) = 3/5738, Pr(X = 34) = 18/5738, etc. The values 33, 34,..., 48 and corresponding probabilities define the probability distribution of X. Note that Pr(X = 33) + Pr(X = 34) + + Pr(X = 48) = 1

18 18 A probability distribution The number of heads obtained when a fair coin is tossed 16 times.

19 19 Cumulative probability function Cumulative probabilities for the binomial (n = 16, p = 0.5) distn. For a continuous distn this would be a smooth curve.

20 20 Expectation If X takes values x 1, x 2,... with probabilities p 1, p 2,..., the expectation of X is E(X) = p 1 x 1 + p 2 x 2 + Consider this gamble: you pay me 1 dollar (the stake ). A fair coin is tossed and if it falls heads, I return your stake and pay you an extra dollar. If tails, I keep the stake and pay nothing. Is this a fair game? Your return is a random variable, determined by the toss of the coin. Your expected return is = 1 dollar, exactly equal to the stake. The game is fair.

21 21 Mean and variance of a random variable The mean value is another name for E(X). It is a measure of location, somewhere near the middle of the probability distn. It is often denoted by m, or the greek letter µ. The variance (σ 2 ) is E(X m) 2, and the standard deviation (σ) is the square root of the variance. These are measures of dispersion, or spread of the distn. When X is derived by sampling from a population, m and σ 2 are the mean and variance of the values in the population. An alternative formula for variance is E(X 2 ) m 2.

22 22 Calculating population mean and variance Variance depends on the spread of the x values, and also on the probabilities. In both examples below, the x values are 0, 1, 2. The variance is 0.5 in example 1, and 0.2 in example 2. Example 1 Example 2 x p x p x 2 p p x p x 2 p Total

23 23 Two measures of shape Skewness: measures departure from symmetry. A positive value indicates an extended right tail. Kurtosis measures the thickness of the tails of a distribution. Large values indicates a heavy-tailed distribution. A distribution can be both skew and kurtotic.

24 24 A skew distribution: binomial (n = 16, p = 1/4) Distn of the number of heads obtained when a biased coin (p = 1/4) is tossed 16 times.

25 25 Conditional probability For two events A and B, P(A B) denotes the conditional probability of the event A, given B. It can be calculated as Pr(A B) = Pr(AB)/ Pr(B) Event B can have a strong effect on the probability of A. For example, Pr(die within year age 20) Pr(die within year age 70)

26 26 Age-specific mortality rates England & Wales Age

27 27 Independent events Events A and B are independent if P(A B) = P(A), or equivalently, if P(AB) = P(A)P(B). Example: the probabilities that a coin lands heads or tails are p and q (p + q = 1). The coin is tossed twice. Assuming independence between the outcomes of the first and second toss leads to the following probabilities: Outcome TT HT TH HH Probability q 2 pq pq p 2 Probability distribution of the number of heads is No. of heads Probability q 2 2pq p 2

28 28 Independent events The assumption of independence is often based on knowledge of the mechanism generating the random events. The result of a coin toss does not depend on what happened previously: the coin has no memory of previous events. Lack of independence may be described as a statistical association between the events (e.g. eye and hair colour).

29 29 Covariance Random variables X and Y are independent if, for all possible values a and b, Pr(X = a, Y = b) = Pr(X = a) Pr(Y = b). If X and Y are not independent, the degree of (linear) dependence between them is measured by the covariance cov (X, Y) = E(X m X )(Y m Y ) = E(XY) m X m Y If X and Y are independent, cov(x, Y) = 0. Covariance affects variance of the sum and difference: var (X ± Y) = var (X) + var (Y) ± 2 cov (X, Y).

30 30 Covariance (Y m y ) negative positive (X m x ) positive negative Values of (X m x )(Y m y )

31 31 Correlation coefficient The correlation between X and Y is obtained by dividing covariance by the product of the standard deviations. A covariance can take any value, but the correlation coefficient is always less than 1 in magnitude. It is unchanged by change in scale of X or Y.

32 32 Genetic covariance Measurements X, Y on two siblings can be represented by X = m + U + e 1, Y = m + U + e 2, where U, e 1, and e 2 are independent r.v.s. Covariance between X and Y is cov(u + e 1, U + e 2 ), or cov (U, U) + cov (U, e 1 ) + cov (U, e 2 ) + cov (e 1, e 2 ). This reduces to cov(u, U) = var(u) (other three terms are zero).

33 33 Bayes formula A 1... A k are mutually exclusive and exhaustive events. For any event B, and any one of the events A i, Pr(A i B) Pr(B A i ) Pr(A i ) The constant of proportionality is 1/Pr(B), where Pr(B) = Pr(B A 1 ) Pr(A 1 ) + + Pr(B A k ) Pr(A k )

34 34 Coat colour gene In cattle, the allele (A) giving black coat colour is dominant to that (a) giving red. Heterozygous (A 1 ) animals are black, but carry one copy of the recessive (a) allele. The offspring of two carriers (A 1 A 1 ) has genotype A 0, A 1, or A 2 with probability 1/4, 1/2, or 1/4. Given that such a calf is black, what is the probability that it is heterozygous? Denote by B the event that the calf is black.

35 35 Coat colour gene We have the following unconditional probabilities: Pr(A 0 ) = 1/4, Pr(A 1 ) = 1/2, Pr(A 2 ) = 1/4, and conditional probabilities: Pr(B A 1 ) = Pr(B A 2 ) = 1, Pr(B A 0 ) = 0. From these we calculate Pr(B) = 1 1/ / /4 = 3/4 Probability that calf is a carrier, given that it is black, is Pr(A 1 B) = 1/2 (1/4 + 1/2 + 0) = 2 3

Probability and random variables

Probability and random variables Events A simple event is the outcome of an experiment. For example, the experiment of tossing a coin twice has four possible outcomes: HH, HT, TH, TT. A compound event