Chapter 2 Continuous Distributions

Transcription

1 Chapter Continuous Distributions

2 Continuous random variables For a continuous random variable X the probability distribution is described by the probability density function f(x), which has the following properties :. f(x) 0 ( ). f x dx =. [ ] ( ) 3. P a X b = f x dx. ( x) dx =. b a f P[ a X b] f ( x) dx. b = a

3 Continuous random variables Continuous distributions allow for a more elegant mathematical treatment. They are especially useful to approximate discrete distributions. Continuous distributions are used in this way in most economics and finance applications, both in the construction of models and in applying statistical techniques.

4 The Uniform distribution from a to b Abraham de Moivre ( )

5 Uniform Distribution A random variable, X, is said to have a Uniform distribution from a to b if X is a continuous RV with probability density function f(x): a x b f ( x) = b a 0 otherwise PDF: Uniform distribution (from a to b)

6 Uniform Distribution: CDF The CDF, F(x), of the uniform distribution from a to b: f 0.4 ( x) x< a x a F( x) = P[ X x] = a x b b a x> b F( x) 0 x a b

7 Uniform Distribution function: Graph of F(x) 0 x< a x a F( x) = P[ X x] = a x b b a x> b F( x) a b x

8 Uniform Distribution: Comments When a=0 and b=, the distribution is called standard uniform distribution, which is a special case of the Beta distribution, with parameters (,) The uniform distribution is not commonly found in nature (but very common in casinos!). Because of its simplicity, it is used in theoretical work (for example, signals/private information are draw from a UD). It is used as prior distribution for binomial proportions in Bayesian statistics. It is also particularly useful for sampling from arbitrary distributions. A general method is the inverse transform sampling method, which uses the CDF of the target RV. This method is very useful in theoretical work. Since simulations using this method require inverting the CDF of the target variable, alternative methods have been devised for the cases where the CDF is not known in closed form.

9 The Normal distribution (Also called Gaussian or Laplacian) Abraham de Moivre ( ) Carl Friedrich Gauss ( ) Pierre-Simon, m. de Laplace (749 87)

10 The Normal distribution: PDF A random variable, X, is said to have a normal distribution with mean µ and standard deviation σ if X is a continuous random variable with probability density function f(x): f ( x) = e πσ ( x µ ) σ

11 Some characteristics of the Normal distribution. Location of maximum point of f(x). parameter µ. ( x µ ) ( ) σ f x = e πσ ( x µ ) ( ) ( ) σ f x = e x µ = σ πσ 0 This is equality holds when µ=x. Therefore, the point µ is an extremum point of f(x). (In this case a maximum)

12 Some characteristics of the Normal distribution. Inflection point of f(x). ( x µ ) σ f x = e x µ ( ) 3 πσ d dx ( ) ( x ) µ ( x µ ) σ σ ( µ ) = e x e 5 3 πσ πσ ( x µ ) ( x ) = e = 0 3 πσ σ ( x ) µ σ µ µ σ σ x if or = = ± i.e. x = µ ± σ Thus the points µ σ, µ + σ are points of inflection of f(x).

13 Some characteristics of the Normal distribution 3. The pdf f(x) integrates to. Proof: To evaluate σ f x dx e dx ( ) Make the substitution ( x µ ) = = πσ e πσ z ( x µ ) = σ x dx µ σ dz = dx σ When x=, z = and when x=, z =. ( x µ ) z z σ e dx = e dz = e dz πσ π π

14 Some characteristics of the Normal distribution Consider evaluating Note: z e dz = c z z u z c = e dz e dz = e dz e du z u z u + = e e dudz = e dudz Make the change to polar coordinates (R, θ): z = R sin(θ) and u = R cos(θ)

15 Some characteristics of the Normal distribution Hence R = z + u z and tan ( θ ) = u or R= z + u and z θ = tan u Using the following result π (, ) = sin ( ), cos( ) z u + c e dudz = = 0 0 ( ) f z u dzdu f R θ R θ RdRdθ R π π π R = e RdRdθ = e dθ = dθ = π

16 Some characteristics of the Normal distribution and or z c = e dz = π z = = ( x µ ) σ e dz e dz π πσ

17 Normal distribution: Comments The normal distribution is often used to describe or approximate any variable that tends to cluster around the mean. It is the most assumed distribution in economics and finance: rates of return, growth rates, IQ scores, observational errors, etc. The central limit theorem (CLT) provides a justification for the normality assumption when n is large. The normal distribution is often named after Carl Gauss, who introduced it formally in 809 to justify the method of least squares. One year later, Pierre Simon (marquis de Laplace) proved the first version of the CLT. ( In some European countries it is called Laplacian.) Notation: PDF: x ~ N(μ,σ ) and CDF: Φ(x)

18 The Exponential distribution Georges Teissier (900-97, France) P. V. Sukhatme (9-997, India)

19 Exponential distribution: Derivation Consider a continuous RV, X with the following properties:. P[X 0] =, and. P[X a + b] = P[X a] P[X b] for all a > 0, b > 0. Let X= lifetime of an object. Then, () and () are reasonable to assume if X = lifetime of an object that does not age. Or () and () are reasonable for an object that lives a long time and a and b are small. The second property implies: P[ X a+ b] = P X a+ b X a = P X b P X a [ ] [ ] => Given the object has lived to age a, the probability that it lives b additional units is the same as if it was born at age a. Past history has no effect on x.

20 Exponential distribution: Derivation Let F(x) = P[X x] and G(x) = P[X x]. Since X is a continuous RV => G(x) = F(x) (P[X x]=0 for all x) The two properties can be written in terms of G(x):. G(0) =, and. G(a + b) = G(a) G(b) for all a > 0, b > 0. We can show that the only continuous function, G(x), that satisfies () and () is a exponential function

21 Exponential distribution: Derivation From property we can conclude ( ) ( ) ( ) ( ) ( ) G a = G a+ a = G a G a = G a Using induction ( ) ( ) ( ) ( ) ( ) n G na = G a+ + a = G a G a = G a Hence putting a =. G( n) G( ) n = n Also putting a = /n. G( ) = G( ) G ( ) ( ) n = G Finally putting a = /m. n n m ( ) = ( ) = ( ) = ( ) n m G m G m G G n n or n

22 Exponential distribution: Derivation Since G(x) is continuous => G(x) = [ G()] x for all x 0. Recall: 0 G() and G(0)=, G( )=0. If G() = 0 then G(x) = 0 for all x > 0 and G(0) = 0 if G is continuous. (a contradiction) If G() = then G(x) = for all x > 0 and G(0) = if G is continuous. (a contradiction) => G() 0, and 0 < G() < Let λ = - ln(g()) then G() = e -λ ( ) ( ) λ Thus G x = G = e = e x x λx

23 Exponential distribution: Derivation 0 x < 0 Finally F( x) = G( x) = λx e x 0 To find the density of X we use: f ( x) = F ( x) = x λe λ x 0 0 x < 0 A continuous random variable with this density function is said to have the exponential distribution with parameter λ.

24 Graphs of f(x) and F(x) f(x) F(x)

25 Exponential distribution: Alternative Derivation Let F (x ) = P[X x] be the CDF of the exponential RV, X. Then, P[X 0] = implies that F(0) = 0. Also P[x X x + dx X x] = λ dx implies P x X x+ dx X x = [ + ] P[ X x] ( + ) ( ) F( x) P x X x dx F x dx F x = = λdx F( x+ dx) F( x) df( x) or = = λ F x dx dx ( )

26 Exponential distribution: Alternative Derivation df dx Let s solve the ODE for the unknown F: = λ [ F ] df = λdx or F F df ( ) = λ + ( ) and ln F x c ( ) λx c and F = e or F = F x = e Now using the fact that F(0) = 0. ( ) = λ dx ln F = λx c λx c c c F 0 = e = 0 implies e = and c= 0 λx ( ) ( ) ( ) Thus F x = e and f x = F x = λe This shows that X has an exponential distribution with parameter λ. λx

27 Exponential distribution: Comments The exponential distribution is often used to model the failure time of manufactured items in production lines. Typical example is light bulbs. If X denotes the time to failure of a light bulb, then the exponential distribution says that the probability of survival of the bulb decays exponentially fast. More precisely: P(X > x) = λ. exp(-λx) Note: The bigger the value of λ, the faster the decay. This indicates that for large λ the average time of failure of the bulb is smaller. This is indeed true. We will show later that E(X) = /λ. We say X has an exponential distribution with parameter λ.

28 Exponential distribution: Comments The exponential distribution is also used a distribution for waiting times between events occurring uniformly in time. Typical example is stock trading intervals. An interesting feature of the exponential distribution is its memoryless property the distribution forgets the past.

29 The Weibull distribution A model for the lifetime of objects that do age. Waloddi Weibull (Sweden, )

30 The Weibulll distribution Recall the properties of continuous RV, X, that lead to the Exponential distribution. namely. P[X 0] =, and. P[x X x + dx X x] = λdx for all x > 0 and small dx. Suppose that the second property is replaced by:. P[x X x + dx X x] = λ(x)dx = αx β dx for all x > 0 and small dx. Again, let X= lifetime of an object. Now, () implies that the object does age. If it has lived up to time x, the chance that it dies in the small interval x to x + dx depends both on the length of that interval, dx, and its age x.

31 Derivation of the Weibulll distribution A continuous random variable, X satisfies the following properties:. P[X 0] =, and. P[x X x + dx X x] = αx β - dx for all x > 0 and small dx. Let F (x ) = P[X x] be the CDF of the random variable, X. Then P[X 0] = implies that F(0) = 0. Also P[x X x + dx X x] = αx β - dx implies P[ x X x + dx] P[ X x] F( x+ dx) F( x) F( x) ( + ) ( ) ( ) P x X x+ dx X x = = = α x F x dx F x df x = = F x dx dx β or α x β ( ) dx

32 Derivation of the Weibulll distribution df dx F β We can solve the ODE, for the unknown F: = α x [ F] F β df = α x dx β x and ln ( F) = α + c ( ) β or ( ) β F = e or F x = e again F(0) = 0 implies c = 0. ( ) Thus F x = e β df x dx = α α β ln F = x c β α β x α β c x c β α x β β ( ) ( ) α β and f x = F x = x e x 0 β α x β

33 Derivation of the Weibulll distribution α β x β β f ( x) F ( x) α x e x and = = 0 The distribution of X is called the Weibull distribution with parameters α,β. Special cases: When β=, we have the exponential distribution. When β =, it mimics the Rayleigh distribution (used in telecoms) When β =3.5, it is similar to the normal distribution. Popular Application: Time to failure Let X measure time-to-failure and follow the Weibull distribution. Then, the failure rate is proportional to a power of time. The shape parameter, β, is that power plus one: β< ( infant mortality phenomenon ); β= (constant failure); and β> (aging).

34 The Weibull density, f(x) (α = 0.9, β = ) (α = 0.7, β = ) (α = 0.5, β = )

35 The Gamma distribution An important family of distributions Karl Pearson ( ) Pierre-Simon, m. de Laplace (749 87)

36 The Gamma Function, Γ(x) An important function in mathematics, due to Euler (79) to interpolate factorials. Current formulation and notation due to Legendre. The Gamma function is defined for x 0 by u x u e du Γ = ( ) x 0 u x e u Γ ( x) u

37 Properties of the Gamma Function, Γ(x). Γ() = ( ) u u ( 0 e du e e e ) Γ = = = = 0. Γ(x + ) = xγ(x) x > 0. 0 x u x u x u e du u e du 0 0 ( ) Γ + = = We will use integration by parts to evaluate x u u e du x u u e du = wdv = wv vdw x u x u where w = u and dv = e du, hence dw = xu dx, v = e Thus x u x u x u u e du = u e + x u e du

38 Properties of the Gamma Function, Γ(x) and x u x u x u u e du = u e + x u e du ( x ) x ( x) or Γ + = 0 + Γ 3. Γ(n) = (n - )! for any positive integer n ( x ) x ( x) ( ) Γ ( ) = Γ ( ) = =! using Γ + = Γ and Γ = Γ ( 3) = Γ ( ) = =! Γ ( 4) = 3Γ ( 3) = 3 = 3!

39 Properties of the Gamma Function, Γ(x) 4. ( ) Γ = π ( ) thus ( ) x u u 0 0 Γ x = u e du Γ = u e du Recall the density for the normal distribution ( x µ ) ( ) σ f x = e πσ If µ =0 and σ = then x f ( x) = e π x x x Now = e dx = e dx = e dx π π π 0 0

40 Properties of the Gamma Function, Γ(x) x π Thus x e dx= Make the substitution u =. 0 u Hence x = u and du = xdx or dx = du = du. x Also when x= 0, then u = 0, x u u π = e dx = e dx = Γ 0 0 ( ) and Γ = π. ( )

41 Properties of the Gamma Function, Γ(x) 5. ( n ) Γ + = ( n)! n! n π ( x ) x ( x) Using Γ + = Γ. ( ) ( ) Γ = Γ 3 ( ) ( ) Γ = Γ = π. = π ( ) ( ) Γ = Γ = π n ( ) ( n ) Γ =Γ + = + n 5 3 π ( n) ( n)!! = π = n n! n n! n π

42 Graph: The Gamma Function Γ(x) x

43 The Gamma distribution Let the continuous random variable X have density function: f α λ α λx x e x 0 = Γ α 0 x < 0 ( x) ( ) where α, λ > 0. Then, X is said to have a Gamma distribution with parameters α and λ., denoted as X ~ Gamma(α,λ) or Γ(α,λ). We can think of the Gamma distribution as a sum of α (independent) exponential distributions.

44 Graph: The gamma distribution 0.4 (α =, λ = 0.9) 0.3 (α =, λ = 0.6) 0. (α = 3, λ = 0.6)

45 The Gamma Distribution: Comments. The set of gamma distributions is a family of distributions (parameterized by α and λ). α λ α λx f ( x) = x e x 0 Γ( α). Contained within this family are other distributions: a. The Exponential distribution when α =, the gamma distribution becomes the exponential distribution with parameter λ. It is used to model time to failure or waiting times between events occurring uniformly in time. b.the Chi-square distribution when α = ν/ and λ = ½, the gamma distribution becomes the chi- square (χ ) distribution with ν degrees of freedom. Later, we will see that sum of squares of independent N(0,) variates have a chi-square distribution, with degrees of freedom given by the number of independent terms in the sum of squares.

46 The Gamma Distribution: Comments The Exponential distribution f ( x) = x λe λ x 0 0 x < 0 The Chi-square (χ ) distribution with ν d.f. Or just χ ν. f ( x) = Γ ν ( ) ν ( ) ν ν x x e x 0 0 x < 0 ν ( ) ν x x e x = < Γ 0 0 x 0

47 The Gamma Distribution: Comments Graph: The χ distribution 0. (ν = 4) (ν = 5) 0. (ν = 6)

48 The Gamma Distribution: Comments 3. Convergence to a Normal distribution. When α is large, the gamma distribution converges to a normal distribution with μ = αλ - and σ = αλ Summation property. If x i ~ Γ(α ι,λ) and independent => Σ i x i ~ Γ(Σ i α ι,λ) 5. Scaling. If x ~ Γ(α,λ) => kx ~ Γ(α,k λ) 6. Typically used to model waiting times. 7. Other related distributions: a. Inverse Gamma distribution If X has a Γ(α,λ) distribution, then /X has an inversegamma distribution with parameters α and λ -. b. The Beta distribution If X and Y are independently distributed Γ(α,λ) and Γ(α, λ) respectively, then X/(X + Y) has a beta distribution with parameters α and α.

49 The Gamma Distribution: Application of properties - χ N Suppose x i ~ χ --or x i ~ Γ(/, ½). Suppose they are iid. We want to know the distribution of the sum of Σ i x i = x x N. Summation property. If x i ~ iid Γ(α ι,λ) => Σ i x i ~ Γ(Σ i α ι, λ) Then, Σ i x i ~ Γ(Σ i /, ½) = Γ(N/, ½) or χ N. Note: The degrees of freedom tell us the number of independent χ variables we are adding. If interested in the mean of the x i s, then using the scaling property: _ N x = = i xi N N ~ Γ(, ) N

50 The Gamma Distribution: Application of properties - Exponential Suppose x i ~ Exp(λ) --or x i ~ Γ(,λ). Suppose they are i.i.d. ^ ^ The parameter θ = / λ can be estimated by, where _ = N xi x i = N ^ We want to derive the distribution of θ. _ x ^ θ is a scaled sum of n independent Γ(,λ) RV. That is, ^ θ ~ Γ(N,λ/N)