Lecture 1 and 13 on BST 631: Statistical Theory I Kui Zhang, 09/8/006 Review for the previous lecture Definition: Several discrete distributions, including discrete uniform, hypergeometric, Bernoulli, Binomial, Poisson, Negative Binomial, Geometric. Examples: mean, variance, mgf, relationship Examples: Applications of these distributions Chapter 3 Common Families of Distribution Section 3.3 Continuous Distributions Uniform Distribution [ ab, ] Definition: A random variable has the uniform distribution on [ ab, ] if 1 f ( x a, b) =, x [ a, b]. b a Descriptive Measures: E b+ a =, Var tb ta ( b a) e e =, M () t =. 1 tb ( a) 1
Lecture 1 and 13 on BST 631: Statistical Theory I Kui Zhang, 09/8/006 Application: (Probability integral transformation) If is a continuous random variable and has the cdf F ( x ). Then Y = F ( x) has the uniform distribution on (0,1). Many other applications including order statistics will be illustrated later. Gamma Distribution ( α, β ) Definition: For α > 0, the gamma function Γ ( α) is defined as α 1 x Γ ( α) = x e dx. 0 Properties of the gamma function: For any α > 0, Γ ( α + 1) = αγ ( α) (using integration by parts). For any positive integer, n, Γ ( n) = ( n 1)!. 1 1/ Γ x ( ) = x e dx π =. 0 Definition: A random variable has a gamma distribution with parameters ( α, β ) if 1 α 1 x / β f ( x αβ, ) = x e,0 < x<, α> 0, β> 0. α Γ( αβ ) And α is called the shape parameter, since it most influences the peakedness of the distribution and β is called the scale parameter, since it most influences the spread of the distribution. Specifically, if β = 1, we call it as a standard gamma distribution.
Lecture 1 and 13 on BST 631: Statistical Theory I Kui Zhang, 09/8/006 Descriptive Measures: E Solution: = αβ, Var = αβ, 1 α M () t = ( ), t < 1/ β. 1 βt t 1 xt α 1 x / β M () t = Ee = e x e dx 0 α Γ( αβ ) 1 α 1 x(1/ β t) = x e dx 0 α Γ( αβ ) α Γ( α)(1/(1/ β t)) 1 α = = ( ) α Γ( αβ ) 1 βt Special Cases of Gamma Distribution: Chi squared pdf with p degrees of freedom: α = p / and β = : 1 f x p x e x Γ( p /) ( p/) 1 x/ ( ) =,0 < <. p / Exponential distribution with scale parameter β : α = 1 (Exponential is the analog of the geometric distribution in the continuous) 1 x / β f ( x β ) = e,0 < x<. β 1/ Weibull distribution with parameters ( γ, β ). If ~ exponential( β ), then Y = γ ( γ > 0) has a Weibull distribution Note: Weibull distribution is a popular model used in survival analysis to model hazard functions of continuous failure time random variable: 3
Lecture 1 and 13 on BST 631: Statistical Theory I Kui Zhang, 09/8/006 γ f y y e y β γ 1 y / Y (, ) γ β γ β =,0 < <. Memoryless property of the Exponential Distribution If ~ exponential( β ), then Solution: For t > 0, we have P ( > s > t) = P ( > s t)( s t). 1 P ( > t) = e dx= e = e t β x / β x/ β t/ β t s/ β t/ β ( s t)/ β P ( > s > t) = P ( > s)/ P ( > t) = e / e = e = P ( > s t). Normal Distribution ( µ, σ ) (or Gaussian distribution) Definition: A random variable has a normal distribution with parameters 1 f x µσ = e < x< πσ ( x µ ) /( σ ) (, ),. ( µ, σ ) if Specifically, if ~ n ( µ, σ ), then Z = ( µ ) / σ has a n (0,1), which is called the standard normal. Notes: 1. There are certain reasons for the importance of the normal distribution: it is analytically simple, it has a bell shape, and there is the Central Limit Theorem, which shows that the normal distribution can be used to approximate a large variety of distributions in large samples under some mild conditions. A large portion of statistical theory is built on the normal distribution. 4
Lecture 1 and 13 on BST 631: Statistical Theory I Kui Zhang, 09/8/006. µ : location parameter; σ : scale parameter. 3. To prove that the normal density integrates to 1, we need to transform the integral into polar coordinates (see page 103). 4. To compute probabilities associated with the normal distribution, we use the standard normal tables. 5. Normal distribution is often used to approximate other probability distributions including the discrete distributions sometimes needing a continuity correction to improve on the approximation. For instance, Binomial ( np, ) may be approximated by a normal with µ = np and σ = np(1 p) for large enough n. 6. Poisson( λ ) may be approximated by a normal with µ = λ and σ = λ. Descriptive Measures: E = µ, Var = σ, t / t M () t = e µ + σ. Hint: get the mean, variance, and mgf for the standard normal function first. Theorem: x / = 0 Proof: ( ) Let e dx π /. z / u / v / ( u + v )/ e dz = e du * e dv e dudv 0 0 = 0 0. 0 u = rsinθ and v= rcosθ, then u + v = 1 and dudv = rdθdr, thus z / ( ) 0 0 0 π / r e dz = e dθdr = π /. Question: How to prove 1 Γ 1/ x ( ) = x e dx π =? 0 5
Lecture 1 and 13 on BST 631: Statistical Theory I Kui Zhang, 09/8/006 Application (Example 3.3. Normal Distribution Approximation) Let ~Binomial(5,0.6). We can approximate with a normal random variable, Y, with mean µ = np = 15, and σ = np(1 p) =.45. Thus while Improvements - continuity correction: In general, we have P ( 13) PY ( 13) = PZ ( (13 15) /.45) = PZ ( 0.8) = 0.06, 13 5 x 5 x P ( 13) = 0.6 *0.4 = 0.68. x= 0 x P ( 13) PY ( 13.5) = PZ ( (13.5 15) /.45) = PZ ( 0.61) = 0.71. P ( x) PY ( x+ 0.5) and P ( x) PY ( x 0.5). Beta Distribution ( α, β ) Definition: A random variable has a beta distribution with parameters ( α, β ) if Notes: 1 f x x x x B( αβ, ) α 1 β 1 ( αβ, ) = (1 ),0< < 1, α> 0, β> 0. 6
Lecture 1 and 13 on BST 631: Statistical Theory I Kui Zhang, 09/8/006 1. The beta function and the gamma function has the following relationship: Γ( α) Γ( β ) B( αβ, ) =. Γ ( α + β ). The support set of is (0,1), thus the beta distribution is one of the few common named distribution that give probability 1 to a finite interval. It can be used to model the proportion and will be illustrated in Chapter 4. Do we have other such distributions? 3. The shape of the beta distribution depends on α and β. (See figure 3.3.3 and 3.3.4). Descriptive Measures: E Solution: α = α + β, Var = αβ ( α + β) ( α + β + 1). 1 n 1 n+ α 1 β 1 Bn ( + α, β ) E = x (1 x) dx. B( αβ, ) = 0 B( αβ, ) Cauchy Distribution ( θ, σ ) Definition: A random vairbale has a Caushy distribution with parameters ( θ, σ ) if f 1 1 ( x θσ, ) =. σπ 1 + ( x θ) / σ where θ is the median of. If θ = 0 and σ = 1, we call it as the standard Cauchy distribution. 7
Lecture 1 and 13 on BST 631: Statistical Theory I Kui Zhang, 09/8/006 Some Notes: 1. Cauchy has the same shape as normal density but with thicker tails.. Mgf does not exist so all moments do not exist. 3. Ratio of two independent standard normal distribution is Cauchy. Lognormal Distribution ( µ, σ ) Definition: A random variable has a lognormal distribution with parameters 1 1 f x µσ e x πσ x (log x µ ) /( σ ) (, ) =,0 < <. ( µ, σ ) if Notes: If is random variable such that log ~ n ( µ, σ ), then has a lognormal distribution. Descriptive Measures: Solution: E u / e + σ =, Var e e ( µ + σ ) µ + σ =. x (µ + σ ) x+ µ log x 1 ( x µ ) /( σ ) 1 σ E = Ee = e e dx = e dx πσ πσ 4 [ x ( µ + σ )] + µ ( µ + µσ + σ ) [ x ( µ + σ )] σ µ + σ / σ µ + σ / 1 1 = e dx e e dx e πσ = = πσ. 8
Lecture 1 and 13 on BST 631: Statistical Theory I Kui Zhang, 09/8/006 Double Exponential Distribution ( µ, σ ) Definition: A random variable has a double exponential distribution with parameters ( µ, σ ) if 1 f x µσ e x µ σ σ x µ / σ (, ) =, < <, < <, > 0. Some Notes: 1. The double exponential distribution has fatter tails than the normal distribution but still remains all of its moments.. The double exponential distribution is not bell-shaped and has a peak at x = µ. At the peak, f is not differentiable. Descriptive Measures: E = µ, Solution: Var σ =. = 1 1 ( µ ) µ. σ = + σ = x µ / σ x / σ E x e dx x e dx 9
Lecture 1 and 13 on BST 631: Statistical Theory I Kui Zhang, 09/8/006 alpha= 0.95, beta= 1 alpha= 0.95, beta= 3 alpha= 0.95, beta= 6 1. 0.0 0. 0.4 0.05 0.15 0 4 6 8 10 0 4 6 8 10 0 4 6 8 10 alpha=, beta= 1 alpha=, beta= 3 alpha=, beta= 6 0.0 0. 0 4 6 8 10 0.00 0.06 0.1 0 4 6 8 10 0.00 0.03 0.06 0 4 6 8 10 alpha= 3, beta= 1 alpha= 3, beta= 3 alpha= 3, beta= 6 0.00 0.15 0 4 6 8 10 0.00 0.04 0.08 0 4 6 8 10 0.00 0.0 0.04 0 4 6 8 10 Figure 1: The gamma distribution with different parameters of ( α, β ). 10
Lecture 1 and 13 on BST 631: Statistical Theory I Kui Zhang, 09/8/006 mu= 0, sigma= 1 mu= 0, sigma= mu= 0, sigma= 3 0.0 0. 0.4 0.05 0.15 0.04 0.08 0.1-4 - 0 4-4 - 0 4-4 - 0 4 0.0 0. 0.4 mu= 1.5, sigma= 1-4 - 0 4 0.00 0.10 0.0 mu= 1.5, sigma= -4-0 4 0.0 0.08 mu= 1.5, sigma= 3-4 - 0 4 0.0 0. 0.4 mu= -1.5, sigma= 1-4 - 0 4 0.00 0.10 0.0 mu= -1.5, sigma= -4-0 4 0.0 0.08 mu= -1.5, sigma= 3-4 - 0 4 Figure : The normal distribution with different parameters ( µ, σ ). 11
Lecture 1 and 13 on BST 631: Statistical Theory I Kui Zhang, 09/8/006 0.0 0.04 0.06 0.08 0.10 0.1 0.14 0.16 10 1 14 16 18 0 Normal Approximation Figure 3. Illustration of the continuity correction on the normal approximation. 1
Lecture 1 and 13 on BST 631: Statistical Theory I Kui Zhang, 09/8/006 alpha= 0.5, beta= 1 alpha= 1.5, beta= 1 alpha= 1, beta= 1 0 5 10 15 0.0 0.5 1.0 1.5 0.6 1.0 1.4 alpha= 1, beta= 0.5 alpha= 1, beta= 1.5 alpha=, beta= 0 5 10 15 0.0 0.5 1.0 1.5 0.0 0.5 1.0 1.5 alpha= 1.5, beta= alpha= 0.75, beta= 0.85 alpha= 0.75, beta= 0.75 0.0 0.6 1. 1.0.0 3.0 1.0.0 3.0 Figure 4. The beta distribution with different parameters ( α, β ). 13
Lecture 1 and 13 on BST 631: Statistical Theory I Kui Zhang, 09/8/006 0.0 0.1 0. 0.3 0.4-4 - 0 4 Figure 5. The Cauchy distribution and the normal distribution. 14
Lecture 1 and 13 on BST 631: Statistical Theory I Kui Zhang, 09/8/006 0.0 0. 0.4 0.6 0.8 1.0 0 4 6 8 10 Figure 6. The lognormal distribution with different parameters ( µ, σ ). 15
Lecture 1 and 13 on BST 631: Statistical Theory I Kui Zhang, 09/8/006 0.0 0.1 0. 0.3 0.4 0.5 0.6 0.7-4 - 0 4 Figure 7. The double exponential distribution and the normal distrubution. 16