Probability and random variables

Size: px

Start display at page:

Download "Probability and random variables"

Maryann McBride
5 years ago
Views:

1 Probability and random variables Events A simple event is the outcome of an experiment. For example, the experiment of tossing a coin twice has four possible outcomes: HH, HT, TH, TT. A compound event is associated with more than one outcome. For example the event first toss is H occurs as either HH or HT. In general, a compound event represents a subset of the possible outcomes. Events can be combined in the same way as sets. For example, A B is the event either A or B occurs or both, and A B (or AB) is the event both A and B occur. In our simple example, let A be the event first toss is H, and B the event one head and one tail. Then A B is the event at most one tail (HH or HT or TH) and A B is the simple event HT. Events are mutually exclusive if at most one of the events occurs on any outcome of the experiment (the corresponding sets of outcomes do not overlap). Events are exhaustive if at least one of the events occurs on any outcome of the experiment (it is impossible that none of the events occurs). If events are both mutually exclusive and exhaustive, exactly one of them occurs on any outcome of the experiment. In the coin-tossing example, the following three events are both mutually exclusive and exhaustive: no heads (TT), one head (HT or TH), and two heads (HH). Probability Each simple event E has a probability Pr(E) which measures the relative frequency with which E occurs. Each probability is a number between zero and one and the probabilities of the simple events add up to 1. The probability of a compound event A is the sum of the probabilities of the simple events constituting A. Thus, the probability of any event (simple or composite) lies between 0 (impossible event) and 1 (certain event). From considerations of symmetry, or because randomisation has been used, we can sometimes assume that the simple events are equally probable. In this case, the probability of a compound event E is N E /N, where N E is the number of simple events associated with E, and N is the total number of simple events. It follows that if A and B are mutually exclusive, Pr(A B) = Pr(A) + Pr(B). If A and B are not mutually exclusive, this equation counts the probability of each outcome in AB twice, and so in general Pr(A B) = Pr(A) + Pr(B) Pr(AB). Similarly, for three events A, B, C, Pr(A B C) = Pr(A) + Pr(B) + Pr(C) Pr(AB) Pr(AC) Pr(BC) + Pr(ABC) 1

2 Sampling with replacement Example. A sample of two balls is selected with replacement from a box containing two balls labelled H and T. There are 2 2 = 4 possible samples: HH, HT, TH, TT. (This is equivalent to tossing a coin twice.) In general, there are n s ways of selecting s objects from n, if sampling with replacement. Sampling without replacement Suppose we have a population of n objects and we select a sample of s objects, without replacement. Since the first object can be chosen in n ways, and the second in n 1, and so on, the total number of samples is n(n 1) (n s + 1) When s = n, the number of samples of size n, or the number of possible arrangements of n objects, is the product of the first n integers, written n! ( n factorial ). Each sample of size s can be ordered in s! ways, and these samples consist of the same objects. Nearly always it is only membership of the sample that is important, in which case the s! samples are indistinguishable. The number of distinguishable samples is then ( n ) = n(n 1) (n s + 1)/ s! s This is the number of ways that s objects can be selected from n when ordering is ignored ( n choose s ). Example: A box contains 4 balls numbered 1 to 4. Select two balls at random (without replacement). At random means that the mechanism used to select the balls generates each possible outcome with equal probability. Let (x, y) denote the simple event first ball is x, second ball is y. (1,2) (1,3) (1,4) (2,1) (2,3) (2,4) (3,1) (3,2) (3,4) (4,1) (4,2) (4,3) If we take account of ordering, there are 12 simple events. If we disregard ordering, there are only six: (1,2), (1,3), (1,4), (2,3), (2,4), (3,4), where, e.g., (1,2) denotes either 1 followed by 2 or 2 followed by 1. In what proportion of samples do the numbers on the two balls add up to 5? The answer, calculated in two different ways, is 2/12 = 4/24 = 1/6. The number of possible outcomes of an experiment can be very large. For example, a box contains 49 balls, numbered 1 to 49. Six are selected at random, without replacement. 2

3 Disregarding ordering, there are 13,983,816 possible outcomes (the number of ways of choosing a sample of size 6 from 49). Random sampling Random sampling is a procedure which ensures that each possible sample has a known probability of being chosen. Usually each sample has an equal probability. Example: A coin is tossed twice. The simple events are (HH, HT, TH, TT). This is equivalent to drawing a sample of two balls with replacement from a box containing two balls labelled H and T. If the coin is fair, we assign equal probabilities (1/4) to the four possible outcomes. Consider events A 0 no heads, A 1 one head, and A 2 two heads, corresponding respectively to TT, TH HT, and HH. These events are mutually exclusive and exhaustive. Therefore exactly one of the events occurs, and Pr(A 0 ) + Pr(A 1 ) + Pr(A 2 ) = 1 Probabilities for a set of mutually exclusive and exhaustive events sum to 1 and form a probability distribution. No. of heads Total Probability 1/4 1/2 1/4 1 Random variables A random variable (r.v.) is a measurement associated with each simple event. Random variables are denoted by capital letters, e.g., X. A random variable is discrete if the number of outcomes is finite, e.g., (0,1,2). In theory, a continuous r.v. can take any value: if x 1 and x 2 are possible values, so also is any value between x 1 and x 2. Examples of discrete r.v.s: number of heads in 2 tosses of a coin, number of aces in a poker hand, number of eggs laid by a hen in a day. Measurements such as height and weight are treated as continuous. Probability distributions If X is discrete with values x 1,, x n, the events X = x 1,, X = x n are mutually exclusive and exhaustive, with probabilities which add up to 1. The set of possible values and corresponding probabilities define the probability distribution of X. We have already seen one example (the number of heads obtained when a coin is tossed twice). Another example: if a single member of a large population is selected at random and his height recorded, the observed value is a random variable with a probability distribution which represents the relative frequency with which each possible value occurs in the population. 3

4 Features of distributions Here we assume that X is discrete. The continuous case is discussed later. Probability function: this comes in two versions, f(x), the probability that X is exactly equal to x, and the cumulative version F (x): f(x) = Pr(X = x), F (x) = Pr(X x) The beamer presentation has an example (the probability function for the number of heads obtained when a fair coin is tossed 16 times). Expectation (or mean) of X is the average value in the population. Usually denoted by m or E(X), it is a measure of location. Formula: xf(x). The summation is over all possible values of X. Variance: σ 2 or var(x), is the mean squared deviation from the mean. It measures the spread or dispersion of the distribution. The formula for the variance is σ 2 = E(X m) 2. A more convenient formula is E(X 2 ) m 2. The standard deviation is the positive square root of σ 2. The unit of standard deviation is the same as that of X (m, cm, kg, etc). Example: if X is the number of heads obtained when a fair coin is tossed twice, the expectation of X (mean of the distn) is 1, and the variance of X is 1/2. E(X) = 0 (1/4) + 1 (1/2) + 2 (1/4) = 1 var(x) = ( 1) 2 (1/4) + 0 (1/2) (1/4) = 1/2 The term expectation dates from 18th century games of chance. Consider the following game: you pay me $1 (the stake). A fair coin is then flipped. If it falls heads, I return the stake and pay an additional $1. If tails, I keep the stake and pay nothing. Is this a fair game? Your return is a random variable, determined by the flip of the coin. Expected return is = $1. The game is fair. Shape of a distribution Skewness measures departure from symmetry. A positive value indicates an extended right tail. For example, if I toss a fair coin ten times, the distribution of the number of heads is symmetric about the central value of 5. If the coin is biased (more likely to fall tails than heads, say), the distribution is positively skew. Kurtosis measures the thickness of the tails of a distribution. The standard for comparison is the normal distribution (see later). 4

5 Conditional probability For two events A and B, Pr(A B) is the conditional probability of the event A, given B. In general, Pr(A B) = Pr(AB)/Pr(B). Conditioning on B can have a large effect on the probability of A. E.g., A is individual dies in next year, B is individual is aged x. Or A is individual has blue eyes, B is individual has blond hair. It is sometimes easier to calculate conditional rather than unconditional probabilities. In this case, Pr(AB) can be evaluated as Pr(A B) Pr(B). This can be extended to three or more events: Pr(ABC) = Pr(A BC) Pr(BC) = Pr(A BC) Pr(B C) Pr(C) Independent events Events A and B are independent if Pr(A B) = Pr(A). For such events, Pr(AB) is simply the product of Pr(A) and Pr(B). The assumption of independence is often based on knowledge of the mechanism generating the random events. For example, if a coin is tossed twice, we assume that the outcome of the second toss is not influenced by the outcome of the first toss. (The coin does not remember the previous event). If the probabilities that the coin lands heads or tails are respectively p and q, the assumption of independence leads to the following probabilities: Pr(HH) = p 2, Pr(HT ) = Pr(T H) = pq, Pr(T T ) = q 2 Note that p 2 + 2pq + q 2 = (p + q) 2 = 1. Covariance Random variables X and Y are independent if for all possible values a and b, the events X = a and Y = b are independent, i.e. Pr(X = a, Y = b) = Pr(X = a) Pr(Y = b) If X and Y are not independent, the degree of (linear) dependence between them is measured by the covariance cov(x, Y ) = E(X m X )(Y m Y ) = E(XY ) m X m Y For independent random variables X and Y, cov(x, Y ) is zero. Dependence between r.v.s has an effect on the variance of the sum and difference: var(x ± Y ) = var(x) + var(y ) ± 2 cov(x, Y ) Likewise the variance of a sum of a number of r.v.s is the sum of the variances plus twice the sum of all the pairwise covariances. 5

6 Genetic covariance Measurements X, Y on two siblings can be represented by X = m + U + e 1, Y = m + U + e 2, where U, e 1, and e 2 are independent. U represents the family effect (shared genetic and environmental effects), e 1 and e 2 represent unshared genetic and environmental effects. Then var(x) = var(y ) = σ 2 u + σ2 (phenotypic variance), and the covariance between e siblings is cov(x, Y ) = σ 2 u. Correlation The correlation coefficient is obtained by dividing the covariance by the product of the two standard deviations. Whereas the covariance can take any value, the correlation coefficient is always less than 1 in magnitude, and is unchanged by change of scale in either X or Y. Bayes formula Suppose A 1,, A k are mutually exclusive and exhaustive events. For any event B, and any one of the events A i, Pr(A i B) = Pr(B A i ) Pr(A i ) Pr(B A 1 ) Pr(A 1 ) + + Pr(B A k ) Pr(A k ) Example: In cattle, the gene (A) giving black coat colour is dominant to that (a) giving red. Heterozygous (A 1 ) animals are black, but carry one copy of the recessive gene. The offspring of a cross between two carriers (A 1 A 1 ) has genotype A 0, A 1, or A 2 with probability 1/4, 1/2, or 1/4, respectively. Given that such a calf is black, what is the probability that it is a carrier? (The more usual notation has A 0 = aa, A 1 = Aa, A 2 = AA.) The probability that the calf is black is 1/4 + 1/2 = 3/4. The probability that the calf is both black and a carrier (i.e. the probability that the calf is a carrier) is 1/2. The probability that the calf is a carrier, given that it is black, is 1/2 divided by 3/4 (2/3). With Bayes formula, take the mutually exclusive and exhaustive events to be the three possible genotypes for the black bull, and let B be the event that the bull is black. Then Pr(B A 0 ) = 0, Pr(B A 1 ) = Pr(B A 2 ) = 1, and prior probabilities are Pr(A 0 ) = Pr(A 2 ) = 1/4, Pr(A 1 ) = 1/2. Bayes formula gives Pr(A 1 B) = (1 1/2) (0 1/ / /4) = 2 3 6

Probability and random variables. Sept 2018

Probability and random variables Sept 2018 2 The sample space Consider an experiment with an uncertain outcome. The set of all possible outcomes is called the sample space. Example: I toss a coin twice,