II. Introduction to probability, 2

Transcription

1 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page 1 II. Introduction to probability, 2 1 Random Variables 1.1 Definition: A random variable is a function defined on a sample space. In other words, it is a mapping of events to numbers. 1.2 This can be a simple one-to-one mapping Example: Throw a six-sided die, and let the random variable X be the face value or any specified function Example: In our previous 8-point sample space of the outcomes of three coin tosses, we can define a random variable X as the number of heads. We then have: Event X HHH 3 HHT 2 HTH 2 HTT 1 TTT 0 TTH 1 THT 1 THH Random variables and probabilites (illustrate with sample space of discrete points): Let x 1, x 2,... be all the values that the random variable X can take on. Then we denote the probability that X takes on the value x j as P (X = x j ) = f(x j ). In the previous example of the die, assuming it is fair, we have:

2 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page 2 x j f(x j ) 1 1/6 2 1/6 3 1/6 4 1/6 5 1/6 6 1/6 And in the example of the sum of heads in three coin tosses, we have: x j f(x j ) 0 1/8 1 3/8 2 3/8 3 1/8 2 Density and Distribution 2.1 Density: The density function f(x) is proportional to the probability that a random variable will take on a value between x and x + δx, where δx is an infinitesimal increment. (Strictly speaking, with a continuous distribution, the probability of taking on exactly a particular value is vanishingly small.) By definition: f(x) dx = 1 if f(x) is a density function. 2.2 Distribution: The distribution function F (x) is the probability that the random variable will take on a value less than or equal to x F (x) = P (X x) = x d[f (x)] f(y) dy, and f(x) = dx So, if we know the density we can determine the distribution function, and vice versa. 2.3 Discrete case: f(x j ) = P (X = x j ) is the probability distribution, and F (x) = P (X x) = x j x f(x j) is the distribution function.

3 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page Examples Exponential with parameter r density: f(x) = re rx distribution: F (x) = 1 e rx Exponential with parameter r= f(x) x F(x) x

4 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page 4 Derivation of these functions (in relation to waiting times): Suppose we have a Poisson process with uniform rate or probability of occurrence. Then the waiting times between successive events are exponentially distributed. To see why this should be the case, suppose the process goes on at an instantaneous rate of r acting over a span of time t. Imagine that we subdivide t into n fine increments of length t/n each. There will be on average rt events over the span of time t. The probability that the event will occur in one of these fine increments is rt/n. The probability that it will not occur in one of these increments is (1 rt/n). The probability that it will not occur in n successive increments is therefore (1 rt/n) n. As n, so that we are now dealing with a continuous-time process, we have. Pr(no events in t) = e rt This is the same as the probability that the waiting time is greater than t. Thus the probability that the waiting time is less than or equal to t is 1 e rt. This last quantity is the distribution F (t), and its first derivative, f(t) = re rt, is the density Uniform density and distribution on (a, b) density: f(x) = 1 b a distribution: F (x) = x a 1 x a dy = b a b a applications: probability spatial and temporal pattern of events dropped with constant

5 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page Normal density: f(x) = 1 2π e x2 /2 distribution: F (x) = 1 2π x e y2 /2 dy

6 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page Discrete functions we have already considered: binomial, multinomial, Poisson 3 Expectation (i.e. mean) 3.1 continuous: E(X) = xf(x) dx 3.2 discrete: E(X) = j x jf(x j ) I.e. the expectation is the sum of all values of a random variable, weighted by the density or probability. 3.3 Examples Exponential with rate r: E(X) = 1/r Binomial: E(k) = np; E( k n ) = p Poisson: E(k) = λ Uniform on (a, b): E(X) = b+a Working with expectations (illustrate with discrete case) Let g(x) be some function of x. Then E[g(x)] = j g(x j)f(x j ). For example, E(X 2 ) = j x j 2 f(x j ) Sums: Suppose there are several random variables X, Y, Z, etc. Then E(X + Y + Z) = E(X) + E(Y ) + E(Z) Let a be a constant. Then E(aX) = ae(x) Products: In general, E(XY ) E(X)E(Y ). Exception: E(XY ) = E(X)E(Y ) if X and Y are mutually independent random variables.

7 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page 7 4 Median: Value of x at which F (x) = Example: Exponential: Med(X) = ln(2)/r 5 Mode: Value of x with maximal value of f(x). 5.1 Example: Exponential: Mode(X) = Comment: Gaps between fossil finds are exponentially distributed. Modal gap of zero implies that the single most probable outcome is that there is an infinitesimally small offset between origin and first appearance or between extinction and last appearance. But this still an exceedingly improbably outcome when compared with the sum of all possible alternatives. 6 Variance 6.1 Definition: V (X) = E(X 2 ) [E(X)] 2 = E{[X E(X)] 2 } 6.2 What it captures: Average squared deviation between a random variable and its mean. 6.3 Examples Binomial: V (k) = np(1 p); V ( k ) = p(1 p)/n n Poisson: V (k) = λ NB mean=variance for Poisson. with model of Poisson process. This can be useful in testing whether data agree This variance may seem different from that in the binomial, but consider that np = λ and that, as the probability per trial becomes ever smaller, (1 p) 1, therefore np(1 p) np = λ.

8 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page Exponential with parameter r: V (X) = 1/r 2, i.e. V (X) = E(X) Uniform on (a, b): V (X) = (b a) Covariance 7.1 Definition: cov(x, Y ) = E(XY ) E(X)E(Y ) = E{[X E(X)][Y E(Y )]}, for two random variables X and Y. 7.2 What it captures: Whether positive deviations from mean of one random variable are matched by deviations in the other random variable that are, on average, positive, negative, or zero. 8 Variance of a Sum 8.1 Let X 1,..., X n be a number of random variables with finite variances σ 2 i. 8.2 Let S n be a new random variable defined as S n = X X n. 8.3 Then V (S n ) = n k=1 σ 2 k + 2 j k cov(x j, X k ) 8.4 Thus, if X 1,..., X n are mutually independent, V (S n ) = n k=1 σ2 k. 8.5 Application: Chance fluctuations about a mean probability Suppose there are n independent trials, each with probability of success p k, where k = 1,...n. Let X k, k = 1,..., n be a random variable which takes on values 0 and 1 with probabilities (1 p k ) and p k. For each given k, we know that E(X k ) = p k and V (X k ) = p k (1 p k ) (binomial). Now let S n = X X n. This is the total number of successes in the n trials. E(S n ) = n k=1 p k. V (S n ) = n k=1 σ2 k, because, by assumption, the trials are independent.

9 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page 9 Rewrite V (S n ) as k p k(1 p k ) = k p k k p k 2 = n p k p k 2. Now vary the set of p k with the constraint that k p k = np. k p k 2 is minimized, therefore the variance is maximized, when all p k are equal. Counterintuitive? To quote Feller (p. 231): Given a certain average quality p of n machines, the output will be least uniform if all the machines are equal. (An application to modern education is obvious but hopeless.) 9 Central Limit Theorem: 9.1 Suppose there is a probability distribution with mean µ and variance σ Sum, S n, of n independent draws from this distribution has expectation E(S n ) = nµ and variance V (S n ) = nσ 2. This we have already seen. 9.3 This sum tends to be normal as n. In particular, Z n = S n nµ σ n converges on the standard normal (mean=0, variance=1) as n increases.

10 f 2 f 3 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page Applications to Novel Situations 10.1 Expected value of FreqRat Model: see Foote and Raup (1996); much more on this later Extinction: time intervals Constant at rate q per lineage-million-years; durations discretized to t unit P (duration t) = e qt (waiting time until extinction) P (duration between t and t + 1) = e qt e q(t+1) Sampling: Constant at probability R per interval Figure 1. Ideal frequency distribution under model of constant extinction and sampling f Probability of sampling per interval f 2 2 f 1 f Frequency Stratigraphic range (number of intervals)

11 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page Results of Model: 1. If single-interval taxa ignored, exponential distribution of stratigraphic ranges is exponential with parameter q (just like distribution of true durations). 2. FreqRat (f 2 2 /(f 1 f 3 )) equals R, for ideal distribution (in absence of sampling variation) Check for bias: What is the expected value of the observed FreqRat for a finite sample of data? Consider four outcomes: range=1, range=2, range=3, range>3. Let f 1,..., f 4 be the true probabilities of having the given range, according to the stated model and known values of q and R. Let the observed numbers of taxa falling into each of these categories be denoted k 1,..., k 4. Let N = i k i. Then, by the multinomial distribution, the probability of observing the given set of k i is: P (k 1, k 2, k 3, k 4 ) = N! k 1!k 2!k 3!k 4! f 1 k 1 f 2 k 2 f 3 k 3 f 4 k 4 The expected value of the observed FreqRat is: E[k 2 2 /(k 1 k 3 )] = N N k1 N k1 k 2 k 1 =1 k 2 =1 k 3 =1 [k 2 2 /(k 1 k 3 )] P (k 1, k 2, k 3, k 4 ) N N k1 N k1 k 2 k 1 =1 k 2 =1 k 3 =1 P (k 1, k 2, k 3, k 4 ) This sum is taken only over those values of k i for which the FreqRat can be defined. Thus, the normalization by sum of probabilities in the denominator is needed. I.e. we are dealing with conditional probability of observing a given FreqRat, given that the FreqRat is definable. Example: q = 0.25, R = 0.25 f 1 = f 2 = f 3 =

12 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page 12 f 2 2 /(f 1 f 3 ) = N E(F reqrat) Thus, the FreqRats reported by Foote and Raup (1996) are probably overestimates. FreqRats reported by Foote and Raup (1996) Group Taxonomic Level Stratigraphic inteval N FreqRat Trilobites species 60-foot intervals Crinoids genus stage Mammals species 0.7 m.y Bivalves species 5 m.y

13 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page Expectation and variance of nearest-neighbor distance Application: searching for clustering of points in morphospace or other space (see Foote, 1990, Systematic Zoology 39: ) Assume points randomly dropped in N-dimensional space Volume of N-dimensional hypersphere: V N = πn/2 r N Γ( N 2 + 1), where r is the radius, and Γ is the interpolation of factorials. Note that Γ(m + 1) = m! and Γ(m ) = π (2m)! 2 2m m! Let a = πn/2 Γ( N 2 +1), so that V N = ar N Let ρ be the point density (total points total volume) Then mean number of points in hypersphere of radius r is given by λ = ρar N Probability of no points in a hypersphere of radius r is given by P (0) = e λ (Poisson) Note that this is the probability that a randomly chosen point will have no points within a distance r of it, i.e. that its nearest-neighbor distance will be greater than r. Therefore, the proportion of all nearest-neighbor distances less than or equal to r is given by 1 e λ. This leads immediately to the distribution function Distribution: F (r) = 1 e λ = 1 e ρarn Take the first derivative of this with respect to r to get the density function

14 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page Density: f(r) = d dr (1 e ρarn ) = ρanr N 1 e ρarn Expectation E(r) = which, after consulting a table of integrals, yields where a and ρ are as defined earlier. 0 E(r) = r ρanr N 1 e ρarn, Γ( N+1 N ) a 1/N ρ 1/N, Variance: which is equal to from which we get E(r 2 ) = 0 E(r 2 ) = r 2 ρanr N 1 e ρarn, Γ( N+2 N ) a 2/N ρ 2/N, V (r) = E(r 2 ) [E(r)] 2 = N+2 N+1 Γ( ) [Γ( N N )]2, a 2/N ρ 2/n where a and ρ are as defined above Check equations with N = 2 (which has already been solved see Clark and Evans, 1954, Ecology 35: ): E(r) = 1/(2 ρ) (OK) V (r) = 4 π 4πρ (OK)

15 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page Expected correlation between origination rate, extinction rate, and net rate of diversification (From Foote (2006) Paleobiology [in press]) Motivating question: How is variation in rate of diversification partitioned into origination and extinction components? Foote Figure 1 (two columns) A Change in diversification rate Change in origination rate Change in extinction rate 0.0 B Extinction diversification correlation Origination diversification correlation Foote Figure 3 (one column) Extinction diversification correlation Carbonate Clastic All genera Origination diversification correlation

16 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page Derivation of expected correlations, starting with variance in origination rate, variance in extinction rate, and correlation between origination and extinction Let p i, q i, and d i be the per-capita rate of origination, extinction, and net diversification in time interval i, where d i = p i q i. Let p i = p i p i 1, q i = q i q i 1, and d i = d i d i 1 be the first differences in these rates. Note that d i = p i q i. Let s 2 p, s 2 q, and s 2 d be the variances in p, q, and d taken over the series of time intervals, and let s pq, s pd, and s qd be the covariances between the subscripted variables. The variance of a random variable X is given by s 2 x = E(X 2 ) µ 2 x, (1a) where E(X 2 ) denotes the expectation of X 2 and µ x is the mean of X (Feller 1968: p. 227). Rearranging yields E(X 2 ) = s 2 x + µ 2 x. (1b) Similarly, the covariance between two random variables X and Y is given (Feller 1968: p. 230) by s xy = E(XY) µ x µ y, (2a) which implies that E(XY) = s xy + µ x µ y. Because d = p q, we also need the general expression for the variance of a difference between two random variables (Feller 1968: p. 230): (2b) s 2 (x y) = s 2 x + s 2 y 2s xy, (3) and for the covariance between this difference and either of the random variables: s x(x y) = E(X 2 ) E(XY) µ 2 x + µ x µ y, and s y(x y) = E(Y 2 ) + E(XY) + µ 2 y µ x µ y. Note that the product-moment correlation is given (Sokal and Rohlf 1995: p. 560) by (4a) (4b) r xy = s xy. (5) s 2 x s 2 y Suppose we are given s 2 p, s 2 q, and r pq and are to determine r pd and r qd from these quantities. Starting with r pd and substituting p and d in equation (5), p and q in equation

17 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page 17 (4a), p and q in equation (3), p 2 in equation (1b), and p and q in equation (2b), we have: r pd = s pd s 2 ps 2 d = E(p2 ) E(pq) µ 2 p + µ p µ q s 2 p(s 2 p + s 2 q 2s pq ) = (s2 p + µ 2 p) (s pq + µ p µ q ) µ 2 p + µ p µ q s 2 p(s 2 p + s 2 q 2r pq s p s q ) = s 2 p r pq s p s q s 2 p(s 2 p + s 2 q 2r pq s p s q ) (6) Similarly, starting with r qd and substituting q and d in equation (5), p and q in equation (4a), p and q in equation (3), q 2 in equation (1b), and p and q in equation (2b), we have: r qd = r pq s p s q s 2 q s 2 q(s 2 p + s 2 q 2r pq s p s q ) (7) and Let s q be some multiple of s p, so that s q = ks p, s p s q = ks 2 p, and and s 2 q = k 2 s 2 p. Substituting into equations (6) and (7), we have: r pd = = s 2 p(1 r pq k) s 2 p(s 2 p + k 2 s 2 p 2r pq ks 2 p) 1 r pq k k2 2r pq k + 1 (8) and r qd = = r pq ks 2 p k 2 s 2 p k 2 s 2 p(s 2 p + k 2 s 2 p 2r pq ks 2 p) r pq k k2 2r pq k + 1 (9)

18 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page 18 Figures show how r pd, r qd, and rpd 2 + r2 qd vary as a function of r pq. The figures depict three situations: k > 1, k = 1, and k < 1, i.e., s p < s q, s p = s q, and s p > s q. When s p < s q, r pd (solid line in Fig. 13A) decreases monotonically as r pq increases, while r qd (dashed line in Fig. 13A) reaches a minimum value at r pq = 1/k, i.e., at r pq = s p /s q, and this minimum is equal to (k 2 1)/k 2. The minimum of r qd coincides with r pd = 0. Therefore the minimum of r qd also corresponds to a minimum of rpd 2 + r2 qd, which is equal to (k 2 1)/k 2 (Fig. 13B).

19 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page 19 Figure 13. Expected relationships among r pd, r qd, and r pq, when extinction rate is more variable than origination rate. For this figure, the ratio k of s q to s p is equal to 1.4. A, r pd (solid) and r qd (dashed) as a function of r pq. When extinction rate varies more than origination rate, r qd > r pd for all values of r pq, and r pd > 0 only if r pq < 1/k. B, r 2 pd + r2 qd as a function of r pq. Lines marked by values in the margins are cases discussed in the text. Foote Figure 13 (one column)! 1.0 A k = s q s p = 1.4 rpq = 1 k r pd (solid); r qd (dashed) !0.5!1.0 1 (1 + k 2 ) " (k 2 " 1) k 2 " k 2 (1 + k 2 ) 2.0 B k = s q s p = 1.4 rpq = 1 k rpq = 2k k r qd 2 2 r pd k 2 " 1 k 2 0.0!1.0! r pq

20 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page 20 When k < 1, the behavior of r pd and r qd is reversed compared with the case where k > 1 (Fig. 14). Here, the minimum of r pd is equal to 1 k 2 and occurs when r pq = k; this corresponds to r qd = 0 and also to the minimum of r 2 pd + r2 qd = 1 k2. Figures 13 and 14 show that positive correlations between extinction and diversification and negative correlations between origination and diversification are theoretically possible. They are empirically uncommon, however (Foote 2000a; this study). When k = 1, r pd and r qd are equal in magnitude and opposite in sign for any value of r pq, decreasing in magnitude monotonically as r pq increases (Fig. 15A). As a result, r 2 pd + r2 qd also decreases monotonically as r pq increases (Fig. 15B). Regardless of the value of k, r 2 pd + r2 qd = 1 at two points: when r pq = 0 and when r pq = 2k/(k 2 + 1). r 2 pd + r2 qd is less than unity if r pq is between 0 and 2k/(k 2 + 1). A few special cases are worth noting: If k = 1, then r pd = (1 r pq )/2, r qd = (1 r pq )/2, and r 2 pd + r2 qd = 1 r pq (Fig. 15). If r pq = 0, then r pd = 1/ k 2 + 1, r qd = k/ k 2 + 1, and rpd 2 + r2 qd = 1 (Figs ). If k = 1 and r pq = 0, then r pd = 1/ 2, r qd = 1/ 2, and rpd 2 + r2 qd = 1 (Fig. 15). If, as in this study, we work with first differences, then, because d = p q, we simply substitute p, q, and d for p, q, and d in all the foregoing expressions.

21 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page 21 Figure 14. Expected relationships among r pd, r qd, and r pq, when origination rate is more variable than extinction rate. For this figure, the ratio k of s q to s p is equal to 0.8. See Figure 13 for explanation. When origination rate varies more than extinction rate, r pd > r qd for all values of r pq, and r qd < 0 only if r pq < k. Foote Figure 14 (one column)! 1.0 A k = s q s p = 0.8 rpq = k r pd (solid); r qd (dashed) !0.5 1 (1 + k 2 ) 1 " k 2 " k 2 (1 + k 2 )! B k = s q s p = 0.8 rpq = k rpq = 2k k r qd 2 2 r pd " k 2 0.0!1.0! r pq

22 GEOS 33000/EVOL January 2006 updated January 10, 2006 Page 22 Figure 15. Expected relationships among r pd, r qd, r pq, when origination and extinction are equally variable. See Figure 13 for explanation. Foote Figure 15 (one column)! 1.0 A k = s q s p = r pd (solid); r qd (dashed) !0.5 " 1 2! B k = s q s p = r qd 2 2 r pd !1.0! r pq