Special distributions

Transcription

1 Special distributions August 22, 2017 STAT 101 Class 4 Slide 1

2 Outline of Topics 1 Motivation 2 Bernoulli and binomial 3 Poisson 4 Uniform 5 Exponential 6 Normal STAT 101 Class 4 Slide 2

3 What distributions tell us? Financial crisis data: Mexican S&L Black Mon. Comm. RE AsianLTCM 00 Dotcom 07 Subprime 12 Euro? (1) How many crises (X ) in the next decade? (2) When (X ) will be the next crisis? Both are unknown a random variable; many types of random variables in nature X is a number in (1) and a time interval in (2) Distributions are used to describe the behavior of a random variable Different distributions are needed for different situations P(X ) or f (x) (PDF) tells us the likelihood of different values of X E(X ) tells us the average value of X var(x ) tells us how likely X deviates from E(X ) STAT 101 Class 4 Slide 3

4 Bernoulli trials [James (aka Jacob) Bernoulli, ] Examples (1) Outcomes in tosses of a coin ( H vs. T ) (2) Outcomes in a series of similar investments ( Success vs. Failure ) (3) Outcomes in giving a new treatment to a series of patients ( Cured vs. Not cured ) If we consider each toss, each investment, each time a patient is given treatment a trial and call the outcome of interest a success and if we make the following assumptions The outcomes of the trials are independent of one another P(success) = p, 0 < p < 1 is the same for every trial then the outcome of each trial is called a Bernoulli random variable. The probability distribution of a Bernoulli random variable is P(success) = p, P(failure) = 1 P(success) = 1 p. The sequence of outcomes is called a Bernoulli sequence STAT 101 Class 4 Slide 4

5 The Binomial distribution We are often interested in the number of successes in a Bernoulli sequence. For example, what is the chance of X successes when n patients are treated? The number of successes, X in a Bernoulli sequence with P(success) = p has a Binomial distribution with parameters n and p. Sometimes we write X Bin(n, p) for short. PDF ( ) n P(X = k) = k }{{} (3) p k }{{} (1) (1 p) n k, k = 0, 1,..., n }{{} (2) Explanation The probability of k successes and (n k) failures equals (3) (1) (2) (1) Succeeds k times, each with probability p (2) Fails (n k) times, each with probability (1 p) (3) There are ( n k) possible outcomes with k successes and (n k) failures Mean and variance E(X ) = np, var(x ) = np(1 p) STAT 101 Class 4 Slide 5

6 Expectation and variance E(X ) = 0 P(X = 0) + 1 P(X = 1) n P(X = n) n = kp(x = k), = k=0 n n! k k!(n k)! pk (1 p) n k k=0 = np, after some algebra var(x ) = E[(X np) 2 ] n = (k np) 2 P(X = k), k=0 = np(1 p), after some algebra STAT 101 Class 4 Slide 6

7 Uses of expectation and variance Suppose we have two Binomial distributions k n P(X = k) Bin(n,.3) (.7) n n(.3)(.7) (n 1) n(n 1) 2 (.3) 2 (.7) (n 2)... (.3) n Bin(n,.4) (.6) n n(.4)(.6) (n 1) n(n 1) 2 (.4) 2 (.6) (n 2)... (.4) n It is difficult to compare the characteristics of the distributions based on the PDF Bin(n,.3) Bin(n,.4) E(X ) = np.3n <.4n var(x ) = np(1 p).21n <.24n So on average, there are more successes from Bin(n,.4); however the outcome from Bin(n,.4) is less predictable since its variance is higher E(X ) and var(x ) provide simple and useful characterizations of X STAT 101 Class 4 Slide 7

8 Example Consider the number of successes, X, in 10 patients given a treatment with success probability 0.7 and independent outcomes. Then X Bin(10,.7). (a) What is the probability that 2 out of 10 are successes? P(X = 2) = ( 10 2 ) (.7) 2 (1.7) (b) What is the probability of seeing no more than 2 successes? P(X 2) = = 2 ( 10 k k=0 ( 10 0 ) (.7) k (1.7) 10 k ) (.7) 0 (1.7) 10 + ( ) 10 (.7) 1 (1.7) (c) What is the expected number of successes in 10 patients? (d) What is the variance? E(X ) = np = 10.7 = 7 var(x ) = np(1 p) = 10.7(.3) = 2.1 ( ) 10 (.7) 2 (1.7) STAT 101 Class 4 Slide 8

9 What are parameters? Let X Bin(n, p), then n and p are called parameters. Different parameter values allow us to use the same probability distribution (e.g., Binomial) to describe different situations that share a common thread n trials with independent outcomes but a constant success probability p. Examples Number of heads in 3 tosses of a coin, X Bin(3,.5) Number of 6 in 3 rolls of a die, X Bin(3, 1/6) Number of successes in 4 investments each with p =.4, X Bin(4,.4) k n = 3, p =.5 (.5) 3 3(.5) 3 3(.5) 3 (.5) 3 P(X = k) n = 3, p = 1 6 ( 5 ) ( 5 2 ( 1 ) 6) 6 3 ( 1 6) 2 ( 5 6) ( 1 6) 3 n = 4, p =.4 (.6) 4 4(.4)(.6) 3 6(.4) 2 (.6) 2 4(.4) 3 (.6) (.4) 4 STAT 101 Class 4 Slide 9

10 Poisson distribution (Siméon-Denis Poisson ) Poisson distribution describes the number of events, X, occurring in a fixed unit of time or space, when events occur independently and at a constant average rate, λ. We write X Poisson(λ) for short. Example If financial crises occur independently of each other with an average of 5 per decade, then the number of financial crises, X, in the next decade may follow a Poisson(5) distribution. PDF Mean and variance P(X = k) = λk k! e λ k = 0, 1, 2,..., E(X ) = var(x ) = λ Sum of independent Poisson random variables If X and Y are independent, and X Poisson(λ), Y Poisson(µ), then X + Y Poisson(λ + µ) STAT 101 Class 4 Slide 10

11 Poisson approximation to the binomial distribution In a unit of time, X Poisson(λ), what is P(X = k)? Split time into a large number of n intervals, each with a small probability p = λ n of one or no events and such that the total number of events, Y Bin(n, p = λ n ): X, Average number of events = λ 0 p 1 n p 2 n p 3 n p 4 n p 5 n Y, Average number of events = np = n λ n = λ n 1 n p 1 time P(Y = k) = = ( n k ( n k ) p k (1 p) n k ) ( λ n ) k ( 1 λ ) n k n λk k! e λ = P(X = k) STAT 101 Class 4 Slide 11

12 Poisson approximation Example Suppose, everyday, there is a constant probability p = that an earthquake will occur. Then the number of days with earthquake, X, in 10 years is Bin(3650, ). P(X = k) ( n ) k p k (1 p) n k k λ k k! e λ Why use Poisson approximation? (1) ( n k) for large n was very difficult to evaluate before computers (2) Suppose the average number of fish you can catch from a stream is 5 a day. How many fish will you catch tomorrow? We do NOT know n or p but we know np = 5 = λ STAT 101 Class 4 Slide 12

13 Uniform distribution A continuous uniform distribution is used for situations where X is equally likely to assume any value in an interval [a, b]. We write X U(a, b). There are two parameters, a and b. Density function of U(a, b) PDF and { CDF 1 f (x) = b a, if a x b 0, otherwise { x a F (x) = b a, if a x b 1, if b x Mean and variance 1 b a f (x) E(X ) = a + b (b a)2, var(x ) = a b x STAT 101 Class 4 Slide 13

14 Example If I asked one of you to give me a number between 0 and 100, what is the chance that the number, X, will be between 1 and 15? We can assume X U(0, 100), so a = 0, b = 100. P(1 < X 15) = P(X 15) P(X 1) = F (15) F (1) = = STAT 101 Class 4 Slide 14

15 Exponential Distribution If we are interested in the time T to an event (e.g., financial crisis, pop quiz, death), then the exponential distribution may be used. The exponential distribution has one parameter, λ > 0. We write T Exp(λ). PDF and { CDF λe f (t) = λt, t > 0 0, t 0 { 1 e F (t) = λt, t > 0 0, t 0 Density function of Exp(λ) f (t) Mean and variance E(T ) = 1 λ, var(t ) = 1 λ 2 0 t 0 STAT 101 Class 4 Slide 15

16 Connection to Poisson process X t X 1 0 t 1 T 1 T 2 T 3 Crisis Time Number of crises in 1 unit of time: X 1 Poisson(λ) Number of crises in t units of time: X t Poisson(tλ) X t as t varies is a Poisson process Time between crises T 1, T 2,... exp(λ) F (t) = P(T 1 t) = 1 P(T 1 > t) STAT 101 Class 4 Slide 16 = 1 P(1st event takes longer than t) = 1 P(no events in time 0 to t) = 1 P(X t = 0) = 1 (λt)0 e λt 0! = 1 e λt if t 0.

17 Example If the number of financial crises, X, in the next decade follows a Poisson(λ = 5) distribution, then the time to the next crisis, T Exp(λ = 5). The probability the next crisis will be in this decade is P(T 1) = F (1) = 1 e 5(1) }{{} unit is a decade and the expected time to the next crisis is E(T ) = 1 5 decade or 2 years STAT 101 Class 4 Slide 17

18 Memoryless property of the exponential distribution Suppose we have already waited time s for an event. What is the chance we have to wait to wait for another time t? P(T > s + t T > s) = = = P(T > s + t, T > s) P(T > s) P(T > s + t) P(T > s) 1 P(T s + t) 1 P(T s) = 1 (1 e λ(s+t) ) 1 (1 e λ(s) ) = e λ(s+t) e λs = e λt = P(T > t) if t 0 The expression e λt, does NOT depend on s. Therefore, the exponential distribution forgets that we have waited for s this is called the memoryless property. STAT 101 Class 4 Slide 18

19 Normal (Gaussian) distribution (Carl Friedrich Gauss, ) The normal distribution was introduced as a distribution for measurement errors. The idea is, if we are asked to make a guess of something (e.g., the height of a building, the age of a person, etc.), then the error distribution will probably look like the histogram proportion Measurement error STAT 101 Class 4 Slide 19

20 Who is Carl Friedrich Gauss? Ask her to wait a moment - I am almost done. Carl Friedrich Gauss ( ), while working, when informed that his wife is dying [In Men of Mathematics (1937) by E. T. Bell]. STAT 101 Class 4 Slide 20

21 Normal (Gaussian) distribution The Normal distribution has two parameters, the mean, µ, and the variance, σ 2 where < µ <, σ 2 > 0. We write X N(µ, σ 2 ) (Sometimes as N(µ, σ)). PDF and CDF f (x) = 1 2πσ 2 e{ (x µ)2 /(2σ 2 )} < x < Density function of N(µ, σ 2 ) F (x) must be approximated by a computer. Mean and Variance E(X ) = µ, var(x ) = σ 2 Empirical rules 68% of the values are within ± 1 SD (σ) of µ 95% of the values are within ± 2 SD (σ) of µ 99.7% of the values are within ± 3 SD (σ) of µ 3σ 2σ σ µ +σ +2σ +3σ x STAT 101 Class 4 Slide 21

22 Empirical rules: Illustration Adult height N(µ = 1.6, σ 2 = ) Investment return N(µ = 0, σ 2 = 3 2 ) 95% 95% σ µ + 2σ σ µ + 2σ For all normal distributions, the same percentage in the population fall within µ ± kσ STAT 101 Class 4 Slide 22

23 Z -score For X N(µ, σ 2 ) distribution, the Z-score is: Z = X µ σ Example (Adult height, cont d): X = 1.5, µ = 1.6, σ = 0.1, then Z = 0 X = µ Z > 0 X > µ Z < 0 X < µ Z = Magnitude gives distance of X from µ = }{{} }{{} 1 sign magnitude Given Z, we can find X by X = Zσ + µ and vice versa Z simply re-expresses X so there is no difference in using Z or X Z N(0, 1), the standard normal distribution STAT 101 Class 4 Slide 23

24 Probability calculations under a normal distribution Example (Adult height, cont d): P(X > 1.75) =? P(X > 1.75) Z = 0 Z = 1.5 = P(X > 1.75) = ( ) P Z > = 1.5 = z 0.1 Proportion taller than 1.75m = Proportion 1.5 SD above average P(Z > z ) for all interesting values of z are given in tables (e.g., Table A1) STAT 101 Class 4 Slide 24

25 Use a normal table to calculate probabilities 0 z* Table A1. Areas under the standard normal curve beyond z, i.e., shaded area z STAT 101 Class 4 Slide 25

26 Example: P(Z > 1.5) STAT 101 Class 4 Slide 26 z P(Z > 1.5) =

27 Example: P(Z < 0.84) z P(Z < 0.84) = P(Z > 0.84) = STAT 101 Class 4 Slide 27

28 Example: P(1.2 < Z < 1.89) z STAT 101 Class 4 Slide 28. P(1.2 < Z < 1.89) = P(Z > 1.2) P(Z > 1.89) = =

29 Example: X N(µ = 15, σ 2 = 16), 85-th percentile =? ? z th Percentile of Z th Percentile of X = Zσ + µ 1.04(4) + 15 = STAT 101 Class 4 Slide 29