Review 1: STAT Mark Carpenter, Ph.D. Professor of Statistics Department of Mathematics and Statistics. August 25, 2015

Transcription

1 Review : STAT 36 Mark Carpenter, Ph.D. Professor of Statistics Department of Mathematics and Statistics August 25, 25 Support of a Random Variable The support of a random variable, which is usually denoted by the script form of the letter corresponding to random variable, i.e., the random variable X has support X, Y has support Y, Z has support Z, is one of the first characteristics we can use to help identify the distribution of a random variable. Definition The support of a random variable, say X, denoted as X, is defined to be the set of all points on the real line for which the pdf/pmf is non-zero. That is, where the braces indicates a set. X = {x R : f X (x) > }, Definition 2 Continuous Random Variable: A random variable is said to be continuous if its support is a continuous set (made up of unions and intersections of real intervals). The CDF of a continuous random variable must be a continuous function on the real line Definition 3 Discrete Random Variable: A random variable is said to be discrete if its support is a discrete set. The CDF of a discrete random variable is not continuous, but it is right continuous. So, whether the random variable, X, is a continuous or discrete random variable depends on whether its support is continuous or discrete. In the discrete case, the pmf is a discontinuous function with a positive mass (probability) at each point in

2 the support. In the continuous case, the pdf itself does not have to be a continuous function everywhere, but it usually a continuous function on intervals in the support. In example, below, we see that the pdf for the exponential is zero for all points below zero, then jumps to λe = λ at x = and is continuous on [, ). With this new notation, we can express the pdf /pmf of any rv, X, as { f(x) x X f X (x) = f(x)i X (x) = x / X where f(x) is the positive portion of the pdf/pmf and I X (x) is the following indicator function, { x X I X (x) = x / X Example : (continuous support) Suppose X exponential (λ). From equation (4.5) on page 65 of the textbook, the exponential distribution, indexed by the scale parameter λ (λ > ) is { λe f(x; λ) = λe λx λx x I [, ) (x) = otherwise. which means {x R : f(x) > } = [, ) and the support of X is X = [, ), a continuous set. You can verify that the non-truncated gamma and Weibull distributions, from which the exponential is a special case, share this same support. If X is a normal random variable, then the support is X = (, ) = R. Example 2: (discrete support) Suppose Y is a Binomial random variable with parameters n and p (see page 7 of textbook),then the probability mass function (pmf) is f Y (y) = ( n y ) p y ( p) n y y =,,..., n otherwise which means {y : f(y) > } = {,,..., n} and the support of Y is Y = {,,..., n}, a discrete (and finite) set of points. One can verify that the supports for the negative binomial (r, p), the Geometric(p), and the Poisson random variables (see page 26) are the same countably infinite set {,, 2,...}. The support of the hypergeomtric(n, M, N) is discrete/finite set {max(, n N + M),..., min(n, M).} 2

3 Some Continuous Density Functions and their supports Name f(x) Support Parameters mean variance Gamma (α, β) Chi-square (v) Exp (β) β α Γ(β) xα e x/β X = (, ) α >, β > µ = αβ σ 2 = αβ 2 2 v/2 Γ(v/2) xv/2 e x/2 X = (, ) v {, 2,...} µ = v σ 2 = 2v β e x/β X = (, ) β > µ = β σ 2 = β 2 Exp (λ) λe λx X = (, ) λ > µ = /λ σ 2 = /λ 2 Normal (µ, σ) σ (x µ) 2 2π e 2σ 2 X = (, ) µ (, ), µ σ 2 σ [, ) Normal (,) Weibull (α, β) 2π e x2 /2 α β α xα e (x/β)α X = (, ) none X = (, ) α (, ) µ = σ 2 = ( β (, ) βγ + ) β 2 {Γ( + 2/α) α [Γ( + /α)] 2 } log-normal (µ, σ) 2πσx e [ln(x) µ]2 /(2σ 2 ) X = (, ) µ (, ), e µ+σ2 /2 e 2µ+σ2 (e σ2 ) σ (, ) Note: α is known as the shape parameter and β is a scale parameter. For Gamma, when α =, Gamma (, β) and Weibull (, β) both reduce to Exp (β). Note: Let Y = ln(x), where X is a log-normal (µ, σ) random variable, the Y is Normal (µ, σ). Recall: the gamma function is Γ(a) = x a e x dx. 3

4 2 Probability Functions Every random variable, whether continuous or discrete or a mixture of both continuous and discrete, has a pmf/pdf has a corresponding Cumulative Distribution Function (cdf), a reliability function and a survival function. Theres are all probability functions. Cumulative Distribution Function: Throughout the semester, I will use the notation X F X (x) as short hand for X is a random variable with a cumulative distribution function F X (x). The cumulative distribution function (cdf) for a random variable X, whether it is discrete or continuous or a mixture of both, is defined to be F (x) = P (X x), for all x (, ). If F (x) is a continuous function of x (, ), then X is referred to as a continuous random variable. If F (x) is a strictly right continuous function (in other words, a step function with discontinuities on a discrete set of points), then X is a discrete random variable. So, the way a cdf is computed depends on the type of random variable. x f X(t)dt continuous F X (x) = {t:t x} f x(t) discrete Example (exponential): Suppose X is an exponential(λ), then for any x X = [, ), the exponential cdf is F X (x) = x f X (u)du = x x λe λu du = e λu = e λx Since F X () = and the cdf is zero for any x <, we see the exponential cdf is continuous on (, ). The exponential reliability function is R(x) = F X (x) = e λx. Example (cont): Suppose, X represents the time between computer crashes and λ = week, i.e., the mean time between crashes is one week. Find the probability that the next computer crash will be over 2 weeks. First, we set λ = in the above expression for F (x) and then solve for F(2), which is then subtracted from. That is, if λ =, then F X (x) = e x, x > and F X (2) = e 2. Therefore, P (X > 2) = P (X 2) = F X (2) = e 2 =

5 Example 2 (binomial): Suppose X is a Binomial (4,.5), then the pmf is ( ) 4 f(x; n = 4, p = /2) =, x X = {,, 2, 3, 4} x 2n Table : CDF for Binomial (n=4, p=/2) (, ) P (X < ) = = [, ) P (X ) = P () 6 = 6 [, 2) P (X ) = P () + P () = 5 6 [2, 3) P (X 2) = P () + P () + P (2) = [3, 4) P (X 3) = P () + P () + P (2) + P (3) = [4, ) P (X 4) = P () + P () + P (2) + P (3) + P (4) = 5/6 /6 5/6 /

6 3 Expectation, Means and Variances Recall, for any random variable, X, with pdf/pmf f(x), a measure of central tendency of the population is the population mean µ, or the expected value/long run average for X. More formally, Population Mean: For any random variable, X, with pdf/pmf f(x), the population mean µ = E(X) where xf(x)dx if X is continuous µ = E(X) = xf(x) if X is discrete x X Population Variance: For any random variable, X, with pdf/pmf f(x), the population variance is σ 2 = E(X µ) 2, where (x µ) 2 f(x)dx if X is continuous σ 2 = E(X µ) 2 = (x µ) 2 f(x) if X is discrete Note that it is often easier to compute the variance by noting that, x X σ 2 = E(X µ) 2 = E(X 2 2Xµ + µ 2 ) = EX 2 2µEX + µ 2 = EX 2 (EX) 2. So, rather than going through the original express, one need only compute E(X 2 ) and µ = E(X) and plug the results in to the following expression σ 2 = E(X 2 ) + µ 2. You might notice that each of E(X), E(X 2 ), and E(X µ) 2 are the expected value of different functions, g (x)) = x, g 2 (x) = X 2 and g 3 (x) = (x µ) 2. The expected value for any function is defined below. Expected value of a function: For any random variable, X, with pdf/pmf f(x), 6

7 let Y = g(x), where g( ) is any function. The E(Y ) = Eg(X) where g(x)f(x)dx if X is continuous E [g(x)] = g(x)f(x) if X is discrete x X 4 Moment-Generating Function Whenever it exists, the moment-generating function for a random variable X, denoted M X (t), is the continuous function of t (, ) given as M X (t) = E [ e tx], t ( h, h), h >. The interval ( h, h) is referred to as the radius of convergence. This function is called the moment-generating function because you can find the n th moment for the random variable, X, by computed its n th derivative with respect to t then setting t =, as follows E(X n ) = M (n) X () = d dt M X(t). t= Notice that the moment generating function is a continuous and differentiable function of t < h, whether or not X is continuous. In fact, the moment generating function is mathematically independent of the original variable (since it was integrated or summed over the support) and only relates to the variable X through the moments of the distribution and any related parameters. Example : Suppose that X is an exponential (λ) random variable. Then, E [ e tx] = e tx λe λx dx = λe (λ t)x dx. If we multiply and divide the last integral by the constant λ t, we get E X [ e tx ] = λ λ t (λ t)e (λ t)x dx <, if and only if (iff) λ t >. Setting t ( λ, λ) as the radius of convergence, the moment-generating function for X exists and is given as, M X (t) = ( t/λ), t ( λ, λ). 7

8 Notice the first two moments can be verified as follows, E(X) = M () X () = ( t 2 = λ λ) λ t= and E(X 2 ) = M (2) X () = 2 λ 2 ( t λ) 3 t= = 2 λ 2, which gives the variance expression, V ar(x) = E(X 2 ) [E(X)] 2 = 2 λ 2 λ 2 = λ 2. Example 2: X is The moment generating function for a Gamma(α, λ) random variable M X (t) = ( t λ) α, t < λ. Uniqueness of Moment-Generating Function: An important property of the moment generating function is the fact that it uniquely associated with its corresponding distribution, i.e., two random variables with the same mgfs have the same distribution. More formally, for two random variables, X and Y, if for all t ( h, h), M X (t) = M Y (t) then F X (x) = F Y (x). This is similar to the principal behind the Laplace transform integration technique. A Laplace transform is similar to the moment generating function, L(t) = E[e tx ]. Notice that if we let α = in the gamma mgf given in example 2, above, that the mgf is equal to the mgf for the exponentialλ) given in example. Therefore, as we already knew, the gamma(α =, λ) is an exponential(λ). Example 3: (discrete random variable) Suppose X is a Bernoulli (p) random variable. Then E[e tx ] = x X e tx f X (x) = e tx p x ( p) x = ( p) + pe t, t (, ) x= Since the above finite sum is finite for any t <, this expectation exists and the moment generating function for X is M X (t) = ( p) + pe t, t (, ). 8

9 The first two non-central moments can be found by differentiating M X (t) with respect to t and setting t =, as follows, E[X] = M () X () = d dt M X(t) = d t= dt ( p + pet ) = pe t= t = p and E[X 2 ] = M (2) d2 X () = dt M X(t) 2 = d2 t= dt ( p + 2 pet ) = pe t = p. t= Combining the two expressions, we can easily verify that the variance is V ar(x) = E[X 2 ] [E(X)] 2 = p p 2 = p( p). Example 4: (binomial r.v.) Suppose X is binomial(n, p), then the mgf is M X (t) = ( p + pe t) n, t ( h, h), any h >. Note that if we set n = for n in the Binomial mgf then, the mgf becomes the mgf of a Bernoulli(p). Therefore, the Bernoulli is a special case of the binomial, which we already knew. 9