I. Introduction to probability, 1

Size: px

Start display at page:

Download "I. Introduction to probability, 1"

Kristian Cross
5 years ago
Views:

1 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page 1 I. Introduction to probability, 1 1 Sample Space and Basic Probability 1.1 Theoretical space of outcomes of conceptual experiment 1.2 Point: one theoretically possible outcome (indecomposable event) Denote number of points by N 1.3 Event: consists of one or more points 1.4 Example: Space consisting of outcomes of tossing three coins Space has 8 points: 1.HHH, 2.HHT, 3.HTH, 4.HTT, 5.TTT, 6.TTH, 7.THT, 8.THH Assume all eight points have same probability (i.e. coin fair) Then probability of point (event) i; P (E i ) = 1/N = 1/8 for all i. 1.5 Now consider compound (decomposable) event A Let A 1 be heads first. Then P (A 1 ) = N A1 N = 4/8 = 1/ Now let A 2 be heads second. Then P (A 2 ) = N A2 N = 4/8 = 1/ Define A 1 A 2 (also written A 1 A 2 ) as the event heads first and heads second. This is the intersection of A 1 and A 2. Then P (A 1 A 2 ) = N A1 A 2 N = 2/8 = 1/ Define A 1 A 2 as heads first or heads second. This is the union of A 1 and A 2. (NB: or here means and/or.) Then P (A 1 A 2 ) = N A1 A 2 N = 6/8 = 3/4

2 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page 2 In general, P (A 1 A 2 ) = P (A 1 ) + P (A 2 ) P (A 1 A 2 ). The last term avoids double-counting Define A as the event not-a Then P (A ) = 1 P (A) Summation of probabilities If E 1,..., E N are all the possible points in the space, then N i=1 P (E i ) = 1. 2 Conditional Probability 2.1 Definition Probability of an event given prior occurrence of another event Alternative: probability of an event, considered as a member of a subpopulation of events satisfying a certain condition. 2.2 Notation P (A 2 A 1 ) 2.3 Example Probability of heads second (A 2 ) given heads first (A 1 ) P (A 2 A 1 ) = N A1 A 2 /N A1 = 2/4 = 1/2 2.4 General form: P (A 2 A 1 ) = P (A 1A 2 ) P (A 1 ). This is often written as P (A H) = P (AH), where A denotes the event and H the prior P (H) Hypothesis. From this we get an important equivalent form: P (AH) = P (A H) P (H) 2.5 Relationship to correlation If P (A H) = P (A) = P (A H ), then H has no bearing on probability of A.

3 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page Example of importance of considering conditional probability Suppose there is a fairly accurate test for a particular illness X. Let + and - denote a positive and negative outcome of the test. P (+ X) = 0.98 (true positives) P ( X) = 0.02 (false negatives) P (+ X ) = (false positives) P ( X ) = (true negatives) Suppose in addition that the true frequency of the illness in the population is I.e. P (X) = and P (X ) = If you have a positive test result, what is the probability that you have the illness? P (X +) = P (X +) P (+) = P (X)P (+ X) P (+) In turn, P (+) = P (+ X) P (X) + P (+ X ) P (X ) In the case at hand, P (+) = (0.0001)(0.98) + (0.001)(0.9999) = , therefore P (X +) = (0.0001)(0.98)/( ) = Thus, even though the test is very accurate, because the true incidence is so low, most positives are false positives. The probability that you have the illness, given a positive result, is relatively low. 3 Independent Events 3.1 Definition A 1 and A 2 are independent iff P (A 1 A 2 ) = P (A 1 ) P (A 2 ). Establishment of independence may be a priori or empirical.

4 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page 4 4 Combinatorials We need a general way to deal with spaces that have many more events than three coin tosses. 4.1 Derivation with respect to 5-card poker hands Deck consists of four suits of 13 cards each First draw: each card has probability 1/52 Second draw: each card has probability 1/51, etc. Thus, total number of hands is This includes all permutations But we don t care about the order in which they are drawn. For any given five cards, there are arrangements or permutations. Thus the number of unique hands is (52)(51)(50)(49)(48) (5)(4)(3)(2)(1) Use notation n! (n-factorial), where n! = n(n 1)(n 2)...(1), and 0! = 1 by convention. So, number of unique hands is (52)(51)(50)(49)(48) 5! Note that (52)(51)(50)(49)(48) = 52!/47! Thus the number of unique hands is 52! 5!47! ) = n! 4.2 In general, ( n r r!(n r)! is the number of unique ways of choosing n items r at a time. 4.3 Example: probability of drawing a flush in poker (all five cards of the same suit) For each suit, the number of ways of drawing five cards is ( 13 5 of flushes is (4)(1287) or The number of hands in a deck is ( ) 52 5 or 2,598,960. ) or Thus the number

5 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page 5 Thus the probability of a flush is 5148/2,598,960, or , or about 1 in 505. Programming note: Factorials blow up quickly. Use log factorial trick (or intrinsic function if available) for number of combinations: ln ( ) n r = ln(n!) ln(r!) ln([n r]!) Let s 1 = ln(n!) = n i=1 ln(i) Let s 2 = ln(r!) = r i=1 ln(i) Let s 3 = ln([n r]!) = n r i=1 ln(i) Let a = s 1 s 2 s 3 Then ( ) n r = exp(a) 5 Binomial Distribution 5.1 b(k; n, p) = ( ) n k p k (1 p) (n k) This is the probability of exactly k successes in n independent trials, each with probability of success p. In this expression, p k gives the probability of k successes, (1 p) (n k) gives the probability of (n k) failures, and ( ) n k gives the number of ways of placing the k successes. For example, if n = 4 and k = 2, the successes can be on trials 1 and 2, 1 and 3, 1 and 4, 2 and 3, 2 and 4, or 3 and 4.

6 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page 6 Binomial probabilities n=100; p=0.1 n=100; p=0.05 n=10; p=0.5 b(k;n,p) k We are often interested not in probability of an exact number of outcomes, but rather in sum of probabilities of a range of outcomes. 5.2 Applications Sampling Does observed number of events exceed or fall short of what we would expect just from sampling error (given some prior estimate of p)? Binomial confidence limits Standard error of observed number of successes: SE(k) = np(1 p), assuming p here is the true value. Generally we use the observed p as an estimate. Standard error of observed proportion: SE(k/n) = that p is the true value. p(1 p)/n, again assuming

7 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page 7 Because binomial distribution is approximately symmetrical (in fact, approximately normal), one can tack an arbitrary number of standard errors on either side of observation. A more exact approach especially when p is near 0 or near 1, and the distribution is more skewed is to take sum of binomial probabilities on either side of observed value (of n or k/n) until the cumulative probability reaches some conventional value. (See Buzas [1990], J. Paleontol. 64:842-3; Raup [1991] In Gilinsky and Signor, eds. Analytical Paleobiology [Short Courses in Paleontology Number 4].) Example: A. Suppose we collect 100 individuals from a bed, 10% of which belong to species Standard error approach: SE = p ± 1SE = (0.07, 0.13). Exact approach: (0.1)(0.9)/100 = Thus, estimated Choose an arbitrary width of confidence interval. ±1SE corresponds to roughly 68% of area under normal curve, so let s say we want to find values of k k low and k up so that there is a probability of 0.34 of falling at above k low but below 10, and a probability of 0.34 of falling at or below k up but above 10. Calculate the binomial probabilities k b(k) P ( k) P ( k) P (between 10 and k)

8 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page 8 Thus, k low = 7 and k up = 14. The corresponding confidence interval is (7,14). The difference relative to the standard-error approximation reflects the slight asymmetry in the binomial distribution. NB Another valid approach is to find the values of k corresponding to lower and upper tail areas of 16%. (This is what Raup [1991] does.) Doing so yields a confidence interval of (7,13). The two approaches differ slightly in this case because the cumulative probability at k = 10 is equal to rather than Distinguishing proportions: Do two samples yield sufficiently different values of k/n that we can conclude the underlying values of p are different? Example: Suppose we took a second sample of 100 and found that 5 individuals belong to species A. The probability of getting 5 or fewer individuals of species A, if p = 0.1, is equal to (see foregoing table). Therefore, the difference is conspicuous, but not significant at the conventional 5% level Power: How large does n need to be be to distinguish two proportions with a stated level of certainty? Trends Does the number of changes in a given direction exceed what we would expect (given some a priori null hypothesis about p)? (See Raup and Crick [1981].) Probability of at least m out of n changes in one direction: P ( m) = [2] n i=m ( n i) p i (1 p) (n i). The factor of 2 before the sum is included for a two-tailed test (no prior reason to test for changes in one direction or the other). It is omitted for a one-tailed test Selectivity in extinction Do some groups show higher or lower proportional extinction than would be expected for p based on entire ensemble of taxa? (See Raup and Marshall [1980] Paleobiology 6:9-23.)

9 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page 9 Hypothetical data Group N N(extinct) %(extinct) Calculation P (one-tailed) P ( 10 p = 0.366) P ( 10 p = 0.366) P ( 5 p = 0.366) P ( 25 p = 0.366) P ( 25 p = 0.366) Total Multinomial Distribution Binomial based on two outcomes: success and failure. Suppose there are three possible outcomes, with probabilities p 1, p 2, and p 3 (where i p i = 1). Then the probability of exactly k 1 outcomes of type 1, k 2 of type 2, and k 3 of type 3 (where i k i = n) is given by P (k 1, k 2, k 3 ; p 1, p 2, p 3, n) = n! k 1!k 2!k 3! pk 1 1 p k 2 2 p k In general:p (k i ; p i, n) = n! i k i! i p k i i 7 Poisson Approximation to Binomial 7.1 Suppose we have very many trials each with low probability of success. I.e. n is large and p is small (while λ = np is of intermediate value). 7.2 Consider ratio of adjacent terms in binomial: b(k; n, p) b(k 1; n, p) = ( n k ) p k (1 p) (n k) ( ) n k 1 p (k 1) (1 p) (n k+1) Some very tedious steps show that this is approximately equal to np/k for n large and p small, i.e. it is approximately equal to λ/k. 7.3 Now consider the probability of zero successes: b(0; n, p) = (1 p) n = (1 λ n )n This is approximately equal to exp( λ) for large n. (Remember, lim n (1 + x/n) n = e x.)

10 GEOS 33000/EVOL January 2006updated January 10, 2006 Page Thus, recalling the ratio between adjacent terms, we have: b(0; n, p) e λ, b(1; n, p) λ 1 e λ, and so on. b(2; n, p) λ λ 2 1 e λ, b(3; n, p) λ λ λ e λ, λ λk 7.5 In general, P (k; λ) = e k! Poisson probabilities 0.8 λ = 0.1 λ = 1 λ = 2 λ = Probability k

11 GEOS 33000/EVOL January 2006updated January 10, 2006 Page Application of Poisson to continuous-time process rather than discrete trials Let process act at constant rate r over a span of time t. Then λ = rt, and number of events follows Poisson distribution. (See waiting times, next set of notes.) Poisson distribution for continuous process is exact, not approximate. 7.7 Some Applications of Poisson distribution Multiple occurrences of rare events Let p be the probability of an event per unit time interval. Let n be the number time intervals, and let λ = np. Let P n,x be the probability that event occurs at least x times in n time intervals. Then: P n,x = 1 x 1 k=0 λ λk e k! Applications: Floods, earthquakes, etc. Example: mass extinctions Let p = 0.01 per m.y. (5 big events in past 500 m.y.) What is the probability of at least five events in 500 m.y.? For this problem, λ = = 5, and P 500,5 is equal to only 0.56, even though our estimate of λ comes directly from the observed data. What is the probability of at least two events in only 50 m.y.? For this problem, λ = = 0.5, and P 50,2 is equal to So we would be a bit surprised to find two of the big five within only 50 m.y. of each other, but not terribly surprised.

12 GEOS 33000/EVOL January 2006updated January 10, 2006 Page 12 Figure 1. Probability of multiple rare events.

13 GEOS 33000/EVOL January 2006updated January 10, 2006 Page Number of direct descendants of a lineage of duration t. (See Foote [1996].) Let p be the origination rate per lineage-million-years. If the duration is already given as t, then the probability of exactly k direct descendants is given by Spatial patterning pt (pt)k P (k; p, t) = e k! Suppose there are N equal-area cells in a grid, and a total of n occurrences. If they are dropped in at random, then λ is the average number of occurrences per cell: λ = n/n. The probability of exactly k hits in a cell is given by P (k; λ). Example: WWII bomb strikes in London. N = 576, n = 537, λ = k N observed N expectedbypoisson SOURCE: Feller This example shows that randomly placed events often appear clustered. 7.8 Comment: With modern computers, binomial is generally easy to compute, so there may be no real need for Poisson approximation in the case of discrete trials (as opposed to temporally or spatially continuous processes).

II. Introduction to probability, 2

GEOS 33000/EVOL 33000 5 January 2006 updated January 10, 2006 Page 1 II. Introduction to probability, 2 1 Random Variables 1.1 Definition: A random variable is a function defined on a sample space. In