Elements of probability theory

Transcription

1 The role of probability theory in statistics We collect data so as to provide evidentiary support for answers we give to our many questions about the world (and in our particular case, about the business world). As we have seen, our questions often concern themselves with very large populations which are nearly impossible to census, so when we collect data, we must restrict ourselves to rather small samples from these populations. A natural question that arises is How do we select a particular individual from the population of interest to become part of the sample we measure? It turns out that random sampling from a population is the best method to employ (this fact will be analyzed later in the course; see Chapter 7 of the textbook). Consequently, the important statistical features of the sample we draw are inherently unpredictable. Nonetheless, there are general conclusions that can be made, even of uncertain characteristics like the properties of a randomly selected sample; these kinds of conclusions are precisely what probability theory is designed to handle. So we devote some time to studying its basic principles. 1

2 Elements of probability theory (probabilistic) experiment situation in which one of a collection of possible outcomes could occur, but precisely which one cannot be predicted with certainty sample space (S) the exhaustive collection of all the possible outcomes of some probabilistic experiment event (A, B,... ) any result of the experiment described by one or more possible outcomes from the sample space probability (P (A)) measure of the likelihood of an event; its long-run relative frequency subjective make an educated guess empirical calculate the fraction of attempted trials in which the event has occurred a priori use a mathematical model to describe the likelihood of occurrence 2

3 odds alternative method for describing likelihood of occurrence of an event If P (A) is the probability that event A occurs, then the odds in favor of A is given as the ratio P (A) to P (A c ), while the odds against A is the inverse ratio P (A c ) to P (A) Conversely, if the odds in favor of event A is stated as a to b, then P (A) = a a+b, whereas if the odds against event A is stated as a to b, then P (A) = Venn diagram b a+b diagram of the sample space of an event (represented by a rectangle) that depicts the relations among various collections of outcomes (represented by circles which might overlap); a very useful tool to help in the computation of probabilities 3

4 disjoint/mutually exclusive events events which have no outcomes in common, that is, can never occur simultaneously independent events events one of whose outcomes has no influence on the outcomes of the other, that is, the likelihood of the occurrence of one is unaffected by whether the other takes place or not 4

5 Formal rules of probability 1. Probability measures likelihood: P (A) lies between 0 and 1 for any event A. 2. Something has to happen: Where S is the event consisting of the entire sample space, P (S) = Equally likely outcomes have equal probabilities: If there are n equally likely possible outcomes and event A includes exactly k of these outcomes, then P (A) = k/n. 4. Complementary events have complementary probabilities: P (A c ) = 1 P (A). 5. Addition rule for disjoint events: If A and B are disjoint events, then their total probability is P (A B) = P (A or B) = P (A) + P (B). 6. Multiplication rule for independent events: If A and B are independent events, then their joint probability is P (A B) = P (A and B) = P (A) P (B). 5

6 More probability rules General Addition Rule If A and B are any two events, then P (A or B) = P (A) + P (B) P (A and B). conditional probability If A and B are any two events, then the conditional probability P (B A) of event B given event A is the frequency of the outcomes in B conditioned by the outcomes in A; that is, (rel.) freq. of outcomes in B also in A P (B A) = (rel.) freq. of outcomes in A which is equivalent to the definition: P (B A) = P (A B). P (A) General Multiplication Rule If A and B are any two events, then P (A B) = P (A) P (B A). 6

7 independent events Events are independent precisely when their conditional probabilities are the same as their unconditional probabilities; that is, when either one (and thus both) of these formulas hold: contingency table P (B A) = P (B), P (A B) = P (A). Paired qualitative data is organized in a table whose columns list the categories of one variable x and whose rows list the categories of the other variable y; each cell of the table counts the joint frequency of individuals who simultaneously fall into both that column and row category tree diagram a diagram of the outcomes of pairs of successive events, in which the first level of branches represent outcomes of one event and the second layer outcomes of the second; useful for working with conditional probabilities 7

8 total probability rule To study the influence on event A of event B, it is useful to separate those outcomes described by A into those which are common to B, namely the joint event A B, and those disjoint from B, which is the joint event A B c ; from this it follows that P (A) = P (A B) + P (A B c ) = P (A B)P (B) + P (A B c )P (B c ) prior probability the probability P (A) of some event A before consideration of new information in the guise of the occurrence of a second event B; in other words, the unconditional probability of A relative to B posterior probability the conditional probability P (A B) of event A, evaluated after consideration of new information in the guise of the occurrence of event B 8

9 The General Multiplication Rule implies that P (A B) P (B) = P (A B) = P (B A) P (A), but the Total Probability Rule states that P (B) = P (A B) + P (A c B) = P (B A)P (A) + P (B A c )P (A c ), so we deduce the formula P (A B) [P (B A)P (A) + P (B A c )P (A c )] = P (A B) from which follows = P (B A) P (A) Bayes Theorem a formula that describes how to find the posterior probability P (A B) involving a pair of events A and B when the probability of the conditional event B is not known: P (A B) = P (B A)P (A) P (B A)P (A) + P (B A c )P (A c ) 9

10 Bayes Theorem can be generalized to include situations in which the prior information is presented not in the form of a single event A and its complement A c, but in the form of multiple mutually exclusive and exhaustive events A 1, A 2,..., A k : Generalized Bayes Theorem Suppose the sample space of a probabilistic experiment decomposes into multiple mutually exclusive and exhaustive events A 1, A 2,..., A k, all of whose prior probabilities are known. If new information arises in the guise of the occurrence of an event B, then the posterior probabilities with respect to B are given by the formulas P (A i B) = P (B A i )P (A i ) P (B A 1 )P (A 1 ) + P (B A 2 )P (A 2 ) + + P (B A k )P (A k ) where i can be any one of the indices 1, 2,..., k. 10

11 Calculations of these probabilities by Bayes Theorem are best organized in a table whose columns include, in order, 1. the prior probabilities P (A) and P (A c ) (or P (A 1 ), P (A 2 ),..., P (A k )); 2. the corresponding conditional probabilities P (B A) and P (B A c ) (or P (B A 1 ), P (B A 2 ),..., P (B A k )); 3. the joint probabilities P (A B) and P (A c B) (or P (A 1 B), P (A 2 B),..., P (A k B)), which are the products of the numbers in the first two columns; and then, by using the formula in the Theorem, 4. the resulting posterior probabilities P (A B) and P (A c B) (or P (A 1 B), P (A 2 B),..., P (A k B). 11

12 Counting Rules Many probability computations require the enumeration of outcomes of some probabilistic experiment; consequently, rules for counting collections of objects are useful to have available. n factorial (n!) the product of all the integers from 1 to n (where by convention we always define 0! = 1) permutations ( n P x ) arrangements of objects in which the order of selection matters; if x objects are selected from a total of n objects, then the number of possible permutations of these objects is np x = n! (n x)! combinations ( n C x ) arrangements of objects in which the order of selection does not matter; if x objects are selected from a total of n objects, then the number of possible combinations of these objects is nc x = ( ) n x = n! x!(n x)! 12