Moments. Raw moment: February 25, 2014 Normalized / Standardized moment:

Transcription

1 Moments Lecture 10: Central Limit Theorem and CDFs Sta230 / Mth 230 Colin Rundel Raw moment: Central moment: µ n = EX n ) µ n = E[X µ) 2 ] February 25, 2014 Normalized / Standardized moment: µ n σ n Sta230 / Mth 230 Colin Rundel) Lecture 10 February 25, / 23 Moment Generating Function The moment generating function of a random variable X is defined for all real values of t by { M X t) = E[e tx x ] = etx PX = x) If X is discrete x etx PX = x)dx If X is continuous This is called the moment generating function because we can obtain the raw moments of X by successively differentiating M X t) and evaluating at t = 0. M X 0) = E[e 0 ] = 1 = µ 0 M X t) = d [ ] d dt E[etX ] = E dt etx = E[Xe tx ] Moment Generating Function - Properties If X and Y are independent random variables then the moment generating function for the distribution of X + Y is M X +Y t) = E[e tx +Y ) ] = E[e tx e ty ] = E[e tx ]E[e ty ] = M X t)m Y t) Similarly, the moment generating function for S n, the sum of iid random variables X 1, X 2,..., X n is M Sn t) = [M Xi t)] n M X 0) = E[Xe0 ] = E[X ] = µ 1 M X t) = d dt M X t) = d [ ] d dt E[XetX ] = E dt XetX ) = E[X 2 e tx ] M X 0) = E[X 2 e 0 ] = E[X 2 ] = µ 2 Sta230 / Mth 230 Colin Rundel) Lecture 10 February 25, / 23 Sta230 / Mth 230 Colin Rundel) Lecture 10 February 25, / 23

2 Moment Generating Function - Unit Normal Moment Generating Function - Unit Normal, cont. Let Z N 0, 1) then Sta230 / Mth 230 Colin Rundel) Lecture 10 February 25, / 23 Sta230 / Mth 230 Colin Rundel) Lecture 10 February 25, / 23 Proof of the Central Limit Theorem Let X 1,..., X n be a sequence of independent and identically distributed random variables each having mean µ and variance σ 2. Then the distribution of tends to the unit normal as n. That is, for < a <, X X n nµ σ n Sketch of Proof Proposition Let X 1, X 2,... be a sequence of independent and identically distributed random variables and S n = X 1 + +X n. The distribution of S n is given by the distribution function f Sn which has a moment generating function M Sn with n 1. Let Z being a random variable with distribution function f Z and moment generating function M Z. If M Sn t) M Z t) for all t, then f Sn t) f Z t) for all t at which f Z t) is continuous. X1 + + X n nµ P σ n ) a 1 a e x2 /2 dx = Φa) as n 2π We can prove the by letting Z N 0, 1), M Z t) = e t2 /2 and then showing for any S n that M Sn / n et2 /2 as n. Sta230 / Mth 230 Colin Rundel) Lecture 10 February 25, / 23 Sta230 / Mth 230 Colin Rundel) Lecture 10 February 25, / 23

3 Proof of the Proof of the Proof of the Some simplifying assumptions and notation: EX i ) = 0 VarX i ) = 1 M Xi t) exists and is finite Lt) = log Mt) Also, remember L Hospital s Rule: f x) lim x gx) = lim f x) x g x) Proof of the, cont. The moment generating function of X i / n is given by )] ) M Xi / n [exp t) = E txi t = M Xi n n and this the moment generating function of S n / n = n i=1 X i/ n is given by [ )] t n M Sn / n t) = M Xi n Therefore in order to show M Sn / n M Z t) we need to show [ )] t n M Xi e t2 /2 n Sta230 / Mth 230 Colin Rundel) Lecture 10 February 25, / 23 Sta230 / Mth 230 Colin Rundel) Lecture 10 February 25, / 23 Proof of the Proof of the Proof of the, cont. Proof of the, cont. Sta230 / Mth 230 Colin Rundel) Lecture 10 February 25, / 23 Sta230 / Mth 230 Colin Rundel) Lecture 10 February 25, / 23

4 Proof of the Proof of the, Final Comments The preceding proof assumes that EX i ) = 0 and VarX i ) = 1. We can generalize this result to any collection of random variables Y i by considering the standardized form Yi = Y i µ)/σ. Y Y n nµ σ n EY i ) = 0 VarY i ) = 1 Y1 µ = σ = Y Y n ) / n + + Y )/ n µ n σ We have already seen a variety of problems where we find PX <= x) or PX > x) etc. The former is given a special name - the cumulative distribution function. If X is discrete with probability mass function f x) then PX x) = F x) = x z= f z) If X is continuous with probability density function f x) then PX x) = F x) = x f z)dz CDF is defined for for all < x < and follows the following rules: lim F x) = 0 x lim F x) = 1 x < y F x) < x F y) Sta230 / Mth 230 Colin Rundel) Lecture 10 February 25, / 23 Sta230 / Mth 230 Colin Rundel) Lecture 10 February 25, / 23 Binomial CDF Uniform CDF Let X Binomn, p) then Let X Unifa, b) then Probability Mass Function ) PX = k) = f k) = n k p k 1 p) n k Cumulative Density Function ) x n PX x) = F x) = p k 1 p) n k k Probability Mass Function { 1 for x [a, b] b a f x) = 0 otherwise Cumulative Density Function 0 for x a x a F x) = for x [a, b] b a 1 for x b Sta230 / Mth 230 Colin Rundel) Lecture 10 February 25, / 23 Sta230 / Mth 230 Colin Rundel) Lecture 10 February 25, / 23

5 Normal CDF Let X N µ, σ 2 ) then Probability Mass Function f x) = φx) = 1 e x µ)2 2σ 2 2πσ Cumulative Density Function F x) = Φx) We derive the Exponential distribution by thinking of it as a RV that describes the waiting time between events which occur continuously with the rate λ. λ here has the same meaning as in the Poisson distribution, it is the expected number of events in a given unit of time. Let us consider one such unit of time, we expect that there will be λ events in this time span. If we subdivide that unit of time into n subinterval then the probability that one of the events falls with a certain subinterval should be approximately λ/n. Sta230 / Mth 230 Colin Rundel) Lecture 10 February 25, / 23 Sta230 / Mth 230 Colin Rundel) Lecture 10 February 25, / 23, cont. Let X Expλ), we start by examining PX b) where b is a positive integer. This is in essence asking, what is the probability that we do not have to wait longer than b units of time before the first event occurs. Since we have divided each unit of time up into n subdivisions, this is the same as asking what is the probability that the event occurs in the first nb sub-intervals., cont. From calculus remember that: Therefore, m a k = 1 am+1 1 a Since we have the approximate) probability of the event for each subinterval we can model this probability with a Geometric random variable Y with p = λ/n. PX b) PY nb) = bn 1 PY = k) = bn 1 1 λ ) k λ n n Sta230 / Mth 230 Colin Rundel) Lecture 10 February 25, / 23 Sta230 / Mth 230 Colin Rundel) Lecture 10 February 25, / 23

6 , cont. In this case we have the CDF but not the PDF, how do we get the PDF?, cont. Let X be a random variable that reflects the time between events which occur continuously with a rate λ, X Expλ) f x λ) = λe λx PX x) = F x λ) = 1 e λx M X t) = 1 t ) 1 λ EX ) = λ 1 VarX ) = λ 2 MedianX ) = log 2 λ Sta230 / Mth 230 Colin Rundel) Lecture 10 February 25, / 23 Sta230 / Mth 230 Colin Rundel) Lecture 10 February 25, / 23 - Memoryless Property Let X Expλ) assume λ has units of events/min) then if we have waited s minutes without observing an event what is the probability that an event occurs in the next t minutes? - Example Strontium 90 is a radioactive component of fallout from nuclear explosions. The halflife of Strontium 90 is 28 years and the decay of an individual atom can be modeled by an exponential random variable. a) What is the decay rate λ? b) What is the average lifetime of a Strontium 90 atom? c) What is the probability that a Strontium 90 atom survives at least 50 years? d) What is the probability that a Strontium 90 survives at least 75 years given it has survived at least 25 years? Sta230 / Mth 230 Colin Rundel) Lecture 10 February 25, / 23 Sta230 / Mth 230 Colin Rundel) Lecture 10 February 25, / 23