Example 1. The sample space of an experiment where we flip a pair of coins is denoted by:

Transcription

1 Chapter 8 Probability 8. Preliminaries Definition (Sample Space). A Sample Space, Ω, is the set of all possible outcomes of an experiment. Such a sample space is considered discrete if Ω has finite cardinality. Otherwise, the sample space is considered continuous. Example. The sample space of an experiment where we flip a pair of coins is denoted by: Ω = { (H, H), (H, T ), (T, H), (T, T ) }. The same space of an experiment where we roll three six-sided dice is denoted by: Ω 2 = { (x, y, z) x, y, z [,, ] }. Definition 2 (Event). An Event, E Ω, describes the outcome of a particular experiement. Example 2. The event describing when we obtain at least one head in two coin flips is denoted by: E = { (H, H), (H, T ), (T, H) }. The event describing when the sum of the three die we roll is exactly 4 is denoted by: E 2 = { (x, y, z) (x, y, z) Ω 2, x + y + z = 4 }. Definition 3 (Probability). The probability of an event is denoted by a probability mass function f P (Ω) R that obeys the following properties:. E P (Ω). f (E) 0, the probability function produces non-negative probabilities. 2. P(Ω) =, the probability that any event happens in the sample space is. 3. If E,, E are pairwise disjoint events, then f (E E ) = i= f (E i ), The probability of the union of a collection of disjoint events is the sum of their individual probabilities, i.e., the sum rule for probability theory.

2 8.2. RANDOM VARIABLES AND EXPECTATION CHAPTER 8. PROBABILITY Note that we tae the event as the domain of our probability function rather than a single outcome because we can always represent outcomes as singleton sets of events. Example 3. If our coins are fair, then we expect that the probability of obtaining a heads or tails with a single coin to be equal. Thus, with two coins, we expect that all four of the outcomes are equally liely resulting in the probability function: f (E) = 4 E = { x }, x Ω. Now, suppose that we wish to now the probability that we obtain at least one head in two flips. This is denoted by the event: E = { (H, H) } { (H, T ) }, { (T, H) }. And by the sum rule, the probability of this event is: f (E) = f ({ (H, H) }) + f ({ (H, T ) }) + f 2 ({ (T, H) }) = = Random Variables and Expectation Definition 4 (Random Variable). A random variable is a function X Ω T for some output type T. A random variable represents some interpretation of the outcomes of some random process. Example 4. Consider our example of rolling three random dice, denoted by the set of outcomes: The sum of these dice forms a random variable X: Ω = { (x, y, z) x, y, z [,, ] }. X(x, y, z) = x + y + z. The codomain of this random variable are natural numbers in the range [3,, 8]. Definition 5 (Expectation). Let X be a random variable over a set of outcomes Ω of type X Ω R. Also suppose the existence of a probability function f over these outcomes. Then the expected value of X, written E[X], is defined to be the weighted average of the outcomes and their respective probabilities: E[X] = X(t) f (t) t Ω Example 5. Consider an experiment where we have a weighed six-sided dice with outcomes Ω = {, 2, 3, 4, 5, }. The probabilities of each outcome are: f () = 20 f (2) = f (3) = f (4) = f (5) = 5 f () = 4 Let X Ω R be a random variable that represents the value of a particular dice roll. Then the expectation of X is the expected value of the weighted die is: E[X] = = 4.05 In contrast, if the probabilities of all the sides of the die were equally liely, then the expected value of the die is: E[X] = i = 3.5. i= Yes, random variable is a ridiculously poor choice of a name. 2

3 CHAPTER 8. PROBABILITY 8.3. PROBABILITY DISTRIBUTIONS 8.3 Probability Distributions Definition (Probability Distribution). Let X Ω T be a random variable over a sample space Ω and interpretation T. A probability distribution is a function Pr(t) T R that describes the probabilities of the various interpretations of the elements of the sample space. More informally, a probability distribution is a description of how a probability function distributes probabilities among the possible outcomes of an experiment. Many inds of experiments fall into a handful of well nown and understood probability distributions. Definition 7 (Bernoulli Distribution). Let X be a random variable with codomain {0, }. Then a probability distribution Pr {0, } R over X forms a Bernoulli distribution with probability p where: Pr() = p Pr(0) = p We call p a parameter of the probability distribution. The Bernoulli distribution describe the outcome of a single experiment with a binary success or failure outcome. Example. The probability of a single fair coin flip being heads forms a bernoulli distribution with success p = 0.5. Suppose you play a game where you roll two six-sided dice and you win if the sum of the die is greater than 8. Then the probability of winning the game forms a bernoulli distribution with success p = 0. (Note that there are 0 ways out of = possibilities to get a higher than an 8 with two six-sided dice.) We can describe a particular probability distribution using a variety of statistics which summarize salient characteristics of that distribution. These include statistics you ought to be familiar with already, e.g., average (the expected value of a random value), median (the value that splits the probability distribution in half), and mode (the most frequent value). Definition 8 (Binomial Distribution). Let X Ω N be a random variable that records the number of successes after running n independent experiements. Then a probability distribution Pr N R over X forms a Binomial distribution with the probability of generating successes is given by: Pr() = ( n )p ( p) n where p is the probability of a single experiement generating a success. As shorthand, we write B(n, p) for the binomial distribution consisting of n independent experiements with probability of success p for an individual experiement. The probability function is derived combinatorially as follows: The probability of getting successes is p p The remaining experiements must be failures, and there are n of them, so the probability of this is ( p) ( p) = ( p) n. n 3 = p.

4 8.4. CONDITIONAL PROBABILITY CHAPTER 8. PROBABILITY Finally, any -subset of the n experiments may succeed, so there are ( n ) such combinations where we have successes and n failures 2. The bionomial distribution is a generalization of the bernoulli distribution where we conduct n such experiements rather than a single one. As such, it is potentially relevant whenever we are discussing the outcomes of running repeated trials with a binary result. Example 7. Suppose that we flip a biased coin with probability p = 0.75 heads and p = 0.25 tails 20 times with success defined as obtaining heads. This forms a binomial distribution B(20, 0.75). As a consequence of identifying a probability distribution is binomial, we can apply nown formulae to quicly derive statistics for that distribution. Definition 9 (Statistics of the Binomial Distribution). Let B(n, p) be a binomial distribution. Then: The mean or expected value of the distribution is np. The median of the distribution is either np or np. The mode of the distribution is either (n + )p or (n + )p. Example 8. For our biased coin example above, the expected number of heads is = Conditional Probability Definition 0 (Conditional Probability). Define the conditional probability of an event A given that an event B has occurred, written Pr(A B) as: Pr(A B) = Pr(A B). Pr(B) Intuitively, the conditional probability of an event allows us to update the nown probability of said event given some new information. Example 9. Consider the random value X representing the sum of rolling two six-sided dice. Here are all the possible outcomes of X: X = 2, possibility: +. X = 3, 2 possibilities: + 2, 2 +. X = 4, 3 possibilities: + 3, 3 +, X = 5, 4 possibilities: + 4, 4 +, 2 + 3, X =, 5 possibilities: + 5, 5 +, 2 + 4, 4 + 2, X = 7, possibilities: +, +, 2 + 5, 5 + 2, 3 + 4, X = 8, 5 possibilities: 2 +, + 2, 3 + 5, 5 + 3, X = 9, 4 possibilities: 3 +, + 3, 4 + 5, X = 0, 3 possibilities: 4 +, + 4, The choose operator is more commonly called the binomial coefficient because of this fact. 4

5 CHAPTER 8. PROBABILITY 8.4. CONDITIONAL PROBABILITY X =, 2 possibilities: 5 +, + 5. X = 2, possibility: +. The probability of X = 8 is Pr(8) = 5. However, what if we now that the first die is a 3. Then, we only consider the dice rolls where the first die x is a 3: 3 +, 3 + 2, 3 + 3, 3 + 4, 3 + 5, 3 + Of these six possibilities, only one results in a sum of 8, so we have that Pr(8 x = 3) =. Alternatively, we can calculate this directly using the definition of conditional probability: Pr(8 x = 3) = Pr(8 x = 3) Pr(x = 3) = =. 5