Lecture 7: The Exponential and Some Related Distributions. Definition Definition: A continuous random variable X is said to have the exponential distribution with parameter if the density of X is e x if x > px) otherwise. In this case, a simple calculation confirms that calculated explicitly: px)dx. Also, the CDF of X can be F X x) P{X x} e x. x e y dy 2.) Moments The moments of the exponential distribution can be calculated recursively using integration-byparts. Let X be an exponential random variable with parameter. Then E [ X n] x n e x dx x n e x + + n n E[ X n ] n! n, x n e x dx nx n e dx using the fact that E [ X ]. Taking n and then n 2 gives: E[X] V arx) E [ X 2] E[X] 2 2 2 2 2. Thus, the mean of an exponential random variable is equal to the reciprocal of its parameter, while the variance is equal to the mean squared.
3.) Memorylessness An important property of the exponential distribution is that it is memoryless: if X is exponentially distributed with parameter, then for all t, s, P{X > t + s X > t} P{X > s}. To verify this, use the CDF calculated above to obtain and then calculate P{X > t} P{X t} e t, P{X > t + s X > t} P{X > t + s} P{X > t} e t+s e t e s P{X > s}. In other words, if we think of X as being the random) time until some event occurs, then conditional on the event not having occurred by time t, i.e., on X > t, the distribution of the time remaining until the event does occur, X t, is also exponential with parameter. It is as if the conditional probability of the event occurring between times t and t + s has no memory of the amount of time that has elapsed up to that point. Remarkably, the converse of this statement is also true. Proposition: Suppose that X is a continuous random variable with values in [, ). Then X is memoryless if and only X is exponentially distributed for some parameter >. Proof: See footnote on pg. 2 of Ross. 4.) Interpretation: Exponential random variables are often used to model the distribution of the amount of time elapsed until some particular event occurs. In this case, the memorylessness property implies that the likelihood that the event occurs in some short interval [t, t + δt) conditional on it not yet having occurred does not depend on t. Notice that for events that occur at discrete times, the geometric distribution has a similar property: the probability of observing the event in question on any particular trial given that it has not yet occurred is just p. In fact, the exponential distribution can be thought of as a limiting case for the geometric distribution when the success probability p is small and when time is measured in units that are of size /p. Proposition: For each n, let X n be a geometric random variable with parameter /n. Then, for every t, lim n P{X n nt} e t. Proof: First, recall that lim nx e n n) x. 2
The result can then be deduced by writing x k/n below and approximating the sum by an integral with respect to x: P{X n nt} n nt k n ) k n n nt k nt k t e x dx e t. n) n k/n e k/n Remark: This proposition implies that if X n is geometric with parameter /n, then the distribution of the random variable T n n X n is approximately exponential with parameter when n is large. 5.) The Gamma Distribution Definition: A random variable X is said to have the gamma distribution with parameters α, > if its density is α Γα) xα e x if x > px) otherwise, where the gamma function Γα) is defined for α > ) by the formula Γα) y α e y dy. Remark: Notice that the exponential distribution with parameter is identical to the gamma distribution with parameters, ). Later we will show that if X,, X n are independent exponential RVs, each with parameter, then the sum X X + + X n is a gamma RV with parameters n, ). Because of this relationship, the gamma distribution is often used to model life spans. Moments: Integration by parts shows that for α >, the gamma function satisfies the following important recursion: Γα) y α e y + α ) y α 2 e y dy α )Γα ). 3
In particular, if α n is a positive integer, then because we obtain Γ) e y dy, Γn) n )Γn 2) n )n 2)Γn 3) n )n 2) 3 2Γ) n )!. This recursion can be used to calculate the moments of the gamma distribution. If X has the gamma distribution with parameters α, ), then E[X n ] x n α n Γn + α) Γα) Γα) xα e x dx n n α + k). k In particular, taking n and n 2 gives n+α Γn + α) xn+α e x dx E[X] α V arx) E[X 2 ] E[X] 2 α 2. αα + ) 2 α2 2 6.) The Beta Distribution Definition: A random variable X is said to have the beta distribution with parameters a, b > if its density is βa,b) xa x) b if x, ) px) otherwise, where the beta function βa, b) is defined for a, b > ) by the formula βa, b) x a x) b dx. 4
Remark: Notice that if a b, then X is simply a standard uniform random variable. Also, if X and X 2 are independent gamma-distributed RVs with parameters a, θ) and b, θ), respectively, then the random variable X X /X + X 2 ) is beta-distributed with parameters a, b). In particular, if X and X 2 are independent exponential RVs with parameter, then X X /X + X 2 ) is uniformly distributed on [, ]. The beta distribution is often used to model the distribution of frequencies. Moments: It can be shown that the beta function and the gamma function are related by the following identity: βa, b) Γa)Γb) Γa + b). In turn, we can use this identity to evaluate the moments of the beta distribution. Let X be a beta random variable with parameters a, b). Then E[X n ] x n x a x) b dx βa, b) βn + a, b) βa, b) Γn + a)γb) Γa + b) Γn + a + b) Γa)Γb) Γn + a) Γa + b) Γa) Γn + a + b) n k a + k a + b + k. Taking n and n 2 gives E[X] a a + b V arx) E[X 2 ] E[X] 2 aa + ) a + b)a + b + ) a2 a + b) 2 ab a + b) 2 a + b + ). 5