Chapter 3 Common Families of Distributions

Transcription

1 Lecture 9 on BST 631: Statistical Theory I Kui Zhang, 9/3/8 and 9/5/8 Review for the previous lecture Definition: Several commonly used discrete distributions, including discrete uniform, hypergeometric, Bernoulli, Binomial, Poisson, Negative Binomial, and Geometric. Examples: mean, variance, mgf, relationships Examples: Applications of these distributions Chapter 3 Common Families of Distributions Section Introduction Purpose of this Chapter: Catalog many of common statistical distributions (families of distributions that are indexed by one or more parameters) What we should know about these distributions: Definition Background Descriptive Measures: mean, variance, moment generating function Typical applications Interesting and useful interrelationships 1

2 Lecture 9 on BST 631: Statistical Theory I Kui Zhang, 9/3/8 and 9/5/8 Section 3.3 Continuous Distributions Uniform Distribution [ ab, ] Definition: A random variable has the uniform distribution on [ ab, ] if 1 f ( x a, b) =, x [ a, b]. b a Descriptive Measures: tb ta b+ a ( b a) e e E =, Var =, M () t =. 1 tb ( a) Application: (Probability integral transformation) If is a continuous random variable and has the cdf F ( x ). Then Y = F ( ) has the uniform distribution on (,1). Many other applications including order statistics will be illustrated later. Example (Rounding off error): Sales tax is rounded to the nearest cent. Assuming that the actual error the true tax minus the rounded-off value is random, we can either use the discrete model P ( = j) = 1/11, where j =.5,.4,,,.1,,.5, or the simpler continuous model that the error has density function: f ( x ) = 1 for x [.5,.5]. Gamma Distribution ( α, β )

3 Lecture 9 on BST 631: Statistical Theory I Kui Zhang, 9/3/8 and 9/5/8 Definition: For α >, the gamma function Γ ( α) is defined as α 1 x Γ ( α) = x e dx. Properties of the gamma function: The integral α 1 x Γ ( α) = x e dx converges when For any α >, Γ ( α + 1) = αγ ( α) (using integration by parts). For any positive integer, n, Γ ( n) = ( n 1)!. 1 1/ Γ x ( ) = x e dx π =. α > and diverges α. Definition: A random variable has a gamma distribution with parameters ( α, β ) if 1 α 1 x / β f ( x αβ, ) = x e, < x<, α>, β>. α Γ( αβ ) And α is called the shape parameter, since it most influences the peakedness of the distribution and β is called the scale parameter, since it most influences the spread of the distribution. Specifically, if β = 1, we call it as a standard gamma distribution. Descriptive Measures: E Solution: = αβ, Var = αβ, 1 α M () t = ( ), t < 1/ β. 1 βt 3

4 Lecture 9 on BST 631: Statistical Theory I Kui Zhang, 9/3/8 and 9/5/8 t xt α 1 x / β M () t = Ee = e x e dx α = 1 Γ( αβ ) 1 Γ( αβ ) α 1 x(1/ β t) x e dx α α Γ( α)(1/(1/ β t)) 1 = = ( ) α Γ( αβ ) 1 βt α Special Cases of Gamma Distribution: Chi squared pdf with p degrees of freedom: α = p / and β = : 1 f x p x e x Γ( p /) ( p/) 1 x/ ( ) =, < <. p / Exponential distribution with scale parameter β : α = 1 (Exponential is the analog of the geometric distribution in the continuous) 1 x / β f ( x β ) = e, < x<. β 1/ Weibull distribution with parameters ( γ, β ). If ~ exponential( β ), then Y = γ ( γ > ) has a Weibull distribution Note: Weibull distribution is a popular model used in survival analysis to model hazard functions of continuous failure time random variable: γ γ 1 y / fy ( y, ) y e γ β γ β =, < y<. β 4

5 Lecture 9 on BST 631: Statistical Theory I Kui Zhang, 9/3/8 and 9/5/8 Memoryless property of the Exponential Distribution If ~ exponential( β ), then Solution: For t >, we have P ( > s > t) = P ( > s t)( s t). 1 P ( > t) = e dx= e = e t β x / β x/ β t/ β t s/ β t/ β ( s t)/ β P ( > s > t) = P ( > s)/ P ( > t) = e / e = e = P ( > s t). Note: If for a continuous nonnegative random variable : P ( > t+ s > s) = P ( > t) holds for all s, t, then must have an exponential distribution. Example (Waiting time at a restaurant): Suppose that the amount of time a customer spends at a fast food restaurant has an exponential distribution with a mean of six minutes. Then the probability that a randomly selected customer will spend more than 1 minutes in the restaurant is given by 1 x x P ( > 1) = exp( ) = exp( ) = exp( ) =.1353 λ λ λ dx 1 1. The conditional probability that the customer will spend more than 1 minutes given that she has been there for more than six minutes is P ( > 1 > 6) = P ( > 6) = exp( 6/ λ) = The conditional probability that the customer will spend at least additional 1 minutes given that she has been there for more than six minutes is P ( > 18 > 6) = P ( > 1) = exp( 1/ λ) =

6 Lecture 9 on BST 631: Statistical Theory I Kui Zhang, 9/3/8 and 9/5/8 Example (Survival Analysis): To model the length of life of a living organism or a system or a component, it is more convenient to consider the survival function St: ( ) And the hazard (failure rate) function λ ( t) : St () = P ( > t) = 1 P ( t) = 1 F() t ; f () t λ () t =. St () The function λ ( t) may be interpreted as the conditional probability density that a t -year old component will fail. Indeed, for small Δ t, f() t P ( [ tt, +Δ t] > t) = P ( ( tt, +Δ t])/ P ( > t) Δ t= λ( t) Δ t. St () For the exponential distribution with the parameter β, we have For the Weibull distribution, f () t 1 t t 1 λ( t) = = exp( )/ exp( ) =. St () β β β β γ 1/ γ γ t St () = PY ( > t) = P ( > t) = P ( > t) = exp( ), thus β γ γ f () t γ γ 1 t t γ γ 1 λ( t) = = t exp( ) / exp( ) = t. St () β β β β For γ =, we call it as the Rayleigh density distribution. Normal Distribution ( μ, σ ) (or Gaussian distribution) 6

7 Lecture 9 on BST 631: Statistical Theory I Kui Zhang, 9/3/8 and 9/5/8 Definition: A random variable has a normal distribution with parameters ( μ, σ ) if 1 ( x μ) /( σ ) f ( x μσ, ) = e, < x<. πσ Specifically, if ~ n( μ, σ ), then Z = ( μ) / σ has a n (,1), which is called the standard normal. Notes: 1. There are certain reasons for the importance of the normal distribution: it is analytically simple, it has a bell shape, and there is the Central Limit Theorem, which shows that the normal distribution can be used to approximate a large variety of distributions in large samples under some mild conditions. A large portion of statistical theory is built on the normal distribution.. μ : location parameter; σ : scale parameter. 3. To prove that the normal density integrates to 1, we need to transform the integral into polar coordinates (see page 13). 4. To compute probabilities associated with the normal distribution, we use the standard normal tables. 5. Normal distribution is often used to approximate other probability distributions including the discrete distributions sometimes needing a continuity correction to improve on the approximation. For instance, Binomial ( np, ) may be approximated by a normal with μ = np and σ = np(1 p) for large enough n. 6. Poisson( λ ) may be approximated by a normal with μ = λ and σ = λ. Descriptive Measures: E = μ, Var = σ, t / t M () t = e μ + σ. 7

8 Lecture 9 on BST 631: Statistical Theory I Kui Zhang, 9/3/8 and 9/5/8 Hint: get the mean, variance, and mgf for the standard normal function first. Theorem: x / = Proof: ( ) Let e dx π /. z / u / v / ( u + v )/ e dz = e du * e dv e dudv =. u = rsinθ and v= rcosθ, then u v r + = and dudv = rdθdr, thus z ( ) r π / / e dz = re dθdr = π /. Question: How to prove 1 Γ 1/ x ( ) = x e dx π =? Application (Example 3.3. Normal Distribution Approximation) Let ~Binomial(5,.6). We can approximate with a normal random variable, Y, with mean μ = np = 15, and σ = np(1 p) =.45. Thus while P ( 13) PY ( 13) = PZ ( (13 15) /.45) = PZ (.8) =.6, 13 5 x 5 x P ( 13) =.6 *.4 =.68. x= x 8

9 Lecture 9 on BST 631: Statistical Theory I Kui Zhang, 9/3/8 and 9/5/8 Improvements - continuity correction: In general, we have P ( 13) PY ( 13.5) = PZ ( ( ) /.45) = PZ (.61) =.71. P ( x) PY ( x+.5) and P ( x) PY ( x.5). Beta Distribution ( α, β ) Definition: A random variable has a beta distribution with parameters ( α, β ) if Notes: 1 f x αβ x x x α β B( αβ, ) α 1 β 1 (, ) = (1 ),< < 1, >, >. 1. The beta function and the gamma function has the following relationship: Γ( α) Γ( β ) B( αβ, ) =. Γ ( α + β ). The support set of is (,1), thus the beta distribution is one of the few common named distribution that give probability 1 to a finite interval. It can be used to model the proportion and will be illustrated in Chapter 4. Do we have other such distributions? 3. The shape of the beta distribution depends on α and β. (See figure and 3.3.4). Descriptive Measures: E α = α + β, Var = αβ ( α + β) ( α + β + 1). 9

10 Lecture 9 on BST 631: Statistical Theory I Kui Zhang, 9/3/8 and 9/5/8 Solution: 1 n 1 n+ α 1 β 1 Bn ( + α, β ) E = x (1 x) dx. B( αβ, ) = B( αβ, ) Example (Proportion of erroneous income tax return): Suppose that proportion of erroneous income tax return filed with IRS each year can be reviewed as a beta distribution with parameters α and β. The probability that in a randomly chosen year there will be at most 1γ percent erroneous returns is: In particular, if α =, β = 1, γ =.1, we have Γ ( α + β ) Γ( α) Γ( β). γ α 1 β 1 P ( λ) = x (1 x) dx 1 13 Γ(14) 11 13! (1 ) 1 (1 ) 1 P ( ) γ x(1 x) dx [ γ γ γ = = + γ ] = 1 (1 + 1 γ)(1 γ) Γ() Γ(1). 11! 1 1*13 For γ =.1, P (.1) = Cauchy Distribution ( θ, σ ) Definition: A random vairbale has a Caushy distribution with parameters ( θ, σ ) if f 1 1 ( x θσ, ) =. σπ 1 + ( x θ) / σ where θ is the median of. If θ = and σ = 1, we call it as the standard Cauchy distribution. 1

11 Lecture 9 on BST 631: Statistical Theory I Kui Zhang, 9/3/8 and 9/5/8 Some Notes: 1. Cauchy has the same shape as normal density but with thicker tails.. Mgf does not exist so all moments do not exist. 3. Ratio of two independent standard normal distribution is Cauchy. Lognormal Distribution ( μ, σ ) Definition: A random variable has a lognormal distribution with parameters 1 1 f x μσ e x πσ x (log x μ) /( σ ) (, ) =, < <. μ σ (, ) if Notes: If is random variable such that log ~ n( μ, σ ), then has a lognormal distribution. Descriptive Measures: Solution: E u / e + σ =, Var e e ( μ+ σ ) μ+ σ =. x (μ+ σ ) x+ μ log x 1 ( x μ) /( σ ) 1 σ E = Ee = e e dx = e dx πσ πσ 4 [ x ( μ+ σ )] + μ ( μ + μσ + σ ) [ x ( μ+ σ )] σ μ+ σ / σ μ+ σ / 1 1 = e dx e e dx e πσ = = πσ. 11

12 Lecture 9 on BST 631: Statistical Theory I Kui Zhang, 9/3/8 and 9/5/8 Double Exponential Distribution ( μ, σ ) Definition: A random variable has a double exponential distribution with parameters ( μ, σ ) if 1 f x μσ e x μ σ σ x μ / σ (, ) =, < <, < <, >. Some Notes: 1. The double exponential distribution has fatter tails than the normal distribution but still remains all of its moments.. The double exponential distribution is not bell-shaped and has a peak at x = μ. At the peak, f is not differentiable. Descriptive Measures: E Solution: = μ, Var σ =. = 1 1 ( ) μ. σ = + σ = x μ / σ x / σ E x e dx x μ e dx Comments: The density functions are theoretical models for populations of real data that occur in nature. Our purpose is to make inferences about the population based on observed data. Although we may not be bale to select a perfect density function, we can choose one that can represent data well and yield good interferences about the population, which is based on observed data and involves theoretical considerations. 1

13 Lecture 9 on BST 631: Statistical Theory I Kui Zhang, 9/3/8 and 9/5/8 alpha=.95, beta= 1 alpha=.95, beta= 3 alpha=.95, beta= alpha=, beta= 1 alpha=, beta= 3 alpha=, beta= alpha= 3, beta= 1 alpha= 3, beta= 3 alpha= 3, beta= Figure 1: The pdf of gamma distribution with different parameters of ( α, β ). 13

14 Lecture 9 on BST 631: Statistical Theory I Kui Zhang, 9/3/8 and 9/5/8 mu=, sigma= 1 mu=, sigma= mu=, sigma= mu= 1.5, sigma= mu= 1.5, sigma= mu= 1.5, sigma= mu= -1.5, sigma= mu= -1.5, sigma= mu= -1.5, sigma= Figure : The pdf of normal distribution with different parameters ( μ, σ ). 14

15 Lecture 9 on BST 631: Statistical Theory I Kui Zhang, 9/3/8 and 9/5/ Normal Approximation Figure 3. Illustration of the continuity correction on the normal approximation. 15

16 Lecture 9 on BST 631: Statistical Theory I Kui Zhang, 9/3/8 and 9/5/8 alpha=.5, beta= 1 alpha= 1.5, beta= 1 alpha= 1, beta= alpha= 1, beta=.5 alpha= 1, beta= 1.5 alpha=, beta= alpha= 1.5, beta= alpha=.75, beta=.85 alpha=.75, beta= Figure 4. The pdf of beta distribution with different parameters ( α, β ). 16

17 Lecture 9 on BST 631: Statistical Theory I Kui Zhang, 9/3/8 and 9/5/ Figure 5. The comparison of Cauchy distribution and the normal distribution. 17

18 Lecture 9 on BST 631: Statistical Theory I Kui Zhang, 9/3/8 and 9/5/ Figure 6. The pdf of lognormal distribution with different parameters ( μ, σ ). 18

19 Lecture 9 on BST 631: Statistical Theory I Kui Zhang, 9/3/8 and 9/5/ Figure 7. The compassion of double exponential distribution and the normal distribution. 19