Exponential & Gamma Distributions Engineering Statistics Section 4.4 Josh Engwer TTU 7 March 26 Josh Engwer (TTU) Exponential & Gamma Distributions 7 March 26 / 2
PART I PART I: EXPONENTIAL DISTRIBUTION Josh Engwer (TTU) Exponential & Gamma Distributions 7 March 26 2 / 2
Exponential Random Variables (Applications) Exponential random variables reasonably model lifetimes & inter-arrival waiting times: Waiting times between consecutive arrivals of a Poisson process: Time between radioactive decays of µg of Iodine-23 Time between phone calls a dispatcher received Time between emails an account received Time between car accidents at a dangerous intersection Time between insurance claims from a given demographic Time between industrial accidents at a factory Time between wars started in a continent Distances between consecutive arrivals of a Poisson process: Distance between mutations of a DNA strand Distance between roadkills on a road near a forest Lifetimes of electrical components Concentrations of air pollutants in a major city Blast radii of mining explosives Josh Engwer (TTU) Exponential & Gamma Distributions 7 March 26 3 / 2
Exponential Random Variables (Summary) Proposition Notation X Exponential(λ), λ > Parameter(s) λ Arrival Rate ( α of Poisson Process) Support Supp(X) [, ) pdf f X (x; λ) λe { λx e λx, if x cdf F X (x; λ), if x < Mean E[X] /λ Variance V[X] /λ 2 Model(s) Waiting times/distances between Poisson arrivals Lifetimes of electrical components Air pollution concentrations Blast radii of mining explosives Josh Engwer (TTU) Exponential & Gamma Distributions 7 March 26 4 / 2
Exponential Density Plots (pdf & cdf) Josh Engwer (TTU) Exponential & Gamma Distributions 7 March 26 5 / 2
Verification that Exponential(λ) pdf truly is a valid pdf Let random variable X Exponential(λ) f X (x; λ) f X (x; λ) λe λx Non-negativity on its support: Observe that λ > λe λx > f X (x; λ) > Universal Integral of Unity: f X (x) dx FTC [ lim x ( ) λe λx dx [ e λx] x x ( e λx )] ( e λ()) Josh Engwer (TTU) Exponential & Gamma Distributions 7 March 26 6 / 2
Mean of Exponential(λ) random variable (Proof) Let random variable X Exponential(λ), where λ > Then: E[X] IBP x f X (x; λ) dx [ ] x xe λx x λxe λx dx ( e λx ) dx FTC [ ] + [ λ e λx ] x x e λx dx ( FTC ) λ λ E[X] λ QED Josh Engwer (TTU) Exponential & Gamma Distributions 7 March 26 7 / 2
Variance of Exponential(λ) random variable Let random variable X Exponential(λ), where λ > Then: (Proof) E[X 2 ] IBP IBP x 2 f X (x; λ) dx [ ] x x 2 e λx x [ 2λ ] x xe λx x λx 2 e λx dx ( e λx ) (2x dx) FTC [ ] + ( λ e λx ) (2 dx) 2xe λx dx FTC [ ] + 2 λ FTC 2 [ ( )] λ λ e λx dx 2 λ 2 λ 2 V[X] E[X 2 ] (E[X]) 2 2 λ 2 ( λ [ λ e λx ] x x ) 2 2 λ 2 λ 2 λ 2 QED Josh Engwer (TTU) Exponential & Gamma Distributions 7 March 26 8 / 2
Memoryless Property of Exponential Distributions The Exponential dist. is the only continuous dist. that s memoryless: (s, t > ) P(X > s + t X > s) P(X > s + t and X > s) P(X > s) P(X > s + t) P(X > s) e λ(s+t) e λs e ( λs λt)+λs P(X > s + t X > s) P(X > t) e λt P(X > t) e.g. X Exponential(λ) P(X > 5 X > 2) P(X > 3) This property means any additional waiting time for an arrival does not depend on or remember any amount of waiting time beforehand. e.g. Waiting 5 min s total for an arrival given 2 min s already elapsed is the same as just waiting 3 min s total for an arrival. Josh Engwer (TTU) Exponential & Gamma Distributions 7 March 26 9 / 2
PART II PART II: GAMMA DISTRIBUTION Josh Engwer (TTU) Exponential & Gamma Distributions 7 March 26 / 2
Gamma Random Variables (Applications) Gamma random variables reasonably model: Waiting time until α arrivals of Poisson process occur (if α is an integer) Processes that would typically be modeled by Exponential rv s except......the pdf is unimodal & skewed...the memoryless property is not realized In fact, an Exponential(λ) rv is equivalently a Gamma(α, β /λ) rv. Josh Engwer (TTU) Exponential & Gamma Distributions 7 March 26 / 2
The Gamma Function Γ(α) (Definition & Properties) The gamma function is a special function that routinely shows up in higher mathematics, combinatorics, physics and (of course) statistics: Definition (Gamma Function) The gamma function is defined to be: Γ(α) : x α e x dx, where α > Moreover, the gamma function possesses several useful properties: Corollary (Useful Properties of the Gamma Function) Γ(n) (n )! Γ(α + ) αγ(α), where α > where n is a positive integer So the gamma function Γ(α) is a generalization of the factorial (n!). Josh Engwer (TTU) Exponential & Gamma Distributions 7 March 26 2 / 2
Gamma Random Variables (Summary) Proposition Notation X Gamma(α, β), α, β > Parameter(s) α Shape parameter β Scale parameter Support Supp(X) [, ) pdf f X (x; α, β) β α Γ(α) xα e x/β cdf γ(x/β; α) Mean E[X] αβ Variance V[X] αβ 2 Model(s) Assumption(s) Time until α Poisson arrivals (for integer α) Exponential processes w/o memoryless property (none) A Gamma(α, β /λ) rv is identical to a Exponential(λ) rv. Josh Engwer (TTU) Exponential & Gamma Distributions 7 March 26 3 / 2
Mean of Gamma(α, β) random variable Let X Gamma(α, β) pdf f X (x) β α Γ(α) xα e x/β (Proof) Then: E[X] x f X (x) dx x β α Γ(α) xα e x/β dx CI β β α Γ(α) β α Γ(α) β α+ Γ(α + ) β α Γ(α) Γ(α + ) Γ(α) E[X] αβ QED x (α+) e x/β dx ( β α+ ) Γ(α + ) β α+ x (α+) e x/β dx Γ(α + ) Supp(X) β α+ Γ(α + ) x(α+) e x/β } {{ } pdf of Gamma(α+,β) β αγ(α) Γ(α) αβ Josh Engwer (TTU) Exponential & Gamma Distributions 7 March 26 4 / 2 dx
Variance of Gamma(α, β) random variable Let X Gamma(α, β) pdf f X (x) E[X 2 ] CI β 2 β α Γ(α) β α Γ(α) x 2 f X (x) dx β α+2 Γ(α + 2) β α Γ(α) Γ(α + 2) Γ(α) x 2 x (α+2) e x/β dx β α Γ(α) xα e x/β β α Γ(α) xα e x/β dx ( β α+2 ) Γ(α + 2) β α+2 x (α+2) e x/β dx Γ(α + 2) Supp(X) β 2 β α+2 Γ(α + 2) x(α+2) e x/β } {{ } pdf of Gamma(α+2,β) α(α + )Γ(α) Γ(α) Then: (Proof) dx α(α + )β 2 (α 2 + α)β 2 V[X] E[X 2 ] (E[X]) 2 (α 2 + α)β 2 (αβ) 2 (α 2 + α)β 2 α 2 β 2 αβ 2 QED Josh Engwer (TTU) Exponential & Gamma Distributions 7 March 26 5 / 2
Gamma Density Plots (pdf & cdf) Josh Engwer (TTU) Exponential & Gamma Distributions 7 March 26 6 / 2
Computing Probabilities involving Gamma r.v. s Let X Gamma(α, β) pdf f X (x) β α Γ(α) xα e x/β Then, P(a X b) b a f X (x) dx b β α x α e x/β dx Γ(α) a }{{} Nonelementary integral!! and the cdf is F X (x) P(X x) γ(x/β; α) (t/β) α e t dt Γ(α) }{{} Nonelementary integral!! The problem with the Gamma distribution is the resulting integrals have no finite closed-form anti-derivative when α is not an integer!! If α is an integer, tedious use of integration by parts is necessary!! The fix to this is to numerically approximate the Gamma cdf via a table. Hence, use the table for the gamma cdf, which is called the incomplete Gamma function and is denoted γ(x; α). Josh Engwer (TTU) Exponential & Gamma Distributions 7 March 26 7 / 2 x
Computing Incomplete Gamma Fcn γ(x; α) via Table x Shape Parameter (α).5 2 3 4 5.5.68269.39347.92.439.75.7.8427.6322.26424.83.899.366.5.9674.77687.4427.95.6564.858 2.9545.86466.59399.32332.4288.5265 2.5.97465.9792.727.4569.24242.882 3.98569.952.885.5768.35277.8474 3.5.9985.9698.864.6795.46337.27456 4.99532.9868.9842.769.56653.376 4.5.9973.98889.9389.82642.6577.4679 5.99843.99326.95957.87535.73497.5595 5.5.9999.9959.97344.962.7983.64248 6.99947.99752.98265.9383.8488.7494 6.5.99969.9985.98872.95696.8885.77633 7.99982.9999.9927.9736.9823.827 7.5.99989.99945.9953.97974.9485.86794 8.99994.99966.99698.98625.95762.937 Josh Engwer (TTU) Exponential & Gamma Distributions 7 March 26 8 / 2
Computing P(X x) γ(x/β; α) via Table/Software P(X 5) γ(5/β; α) γ(5/2; 3) γ(2.5; 3) TI-8x TI-36X Pro (No built-in function) (No built-in function) MATLAB gamcdf(5,3,2) (Stats Toolbox) R pgamma(q5,shape3,scale2) Python scipy.stats.gamma.cdf(5/2,3) (Need SciPy) Josh Engwer (TTU) Exponential & Gamma Distributions 7 March 26 9 / 2
Textbook Logistics for Section 4.4 Difference(s) in Notation: CONCEPT TEXTBOOK NOTATION SLIDES/OUTLINE NOTATION Probability of Event P(A) P(A) Support of a r.v. All possible values of X Supp(X) pdf of a r.v. f (x) f X (x) cdf of a r.v. F(x) F X (x) Expected Value of r.v. E(X) E[X] Variance of r.v. V(X) V[X] Incomplete Gamma Fcn F(x/β; α) γ(x/β; α) Skip the Chi-Squared Distribution section (pg 74-75) It turns out a Chi-Squared (χ 2 ) rv is a special kind of Gamma rv. χ 2 distributions will be formally encountered in Chapter 7. This is also used for Statistical Inference of Categorical Data (Chapter 4). Josh Engwer (TTU) Exponential & Gamma Distributions 7 March 26 2 / 2
Fin Fin. Josh Engwer (TTU) Exponential & Gamma Distributions 7 March 26 2 / 2