BMIR Lecture Series on Probability and Statistics Fall, 2015 Uniform Distribution

Transcription

1 Lecture #5 BMIR Lecture Series on Probability and Statistics Fall, 2015 Department of Biomedical Engineering and Environmental Sciences National Tsing Hua University s 5.1

2 Definition ( ) A continuous random variable X with probability density function f (x) = 1 b a, a x b (1) is a continuous uniform random variable. The mean of X is b x E(X) = b a dx = x 2 2(b a) The variance is V(X) = b a a ( ) x a+b 2 2 dx = b a b a ( x a+b 2 3(b a) ) 3 = a + b 2 b a = (b a)2 12 s 5.2

3 CDF: The cumulative distribution function of a continuous uniform distribution X is F(X) = P(X x) = If a x b, F(X) = x The complete form is F(X) = x a x f (t)dt = x 1 dt. (2) b a 1 dt = (x a)/(b a). (3) b a 1 b a dt = 0 x < a x a b a a x < b 1 b x (4) s 5.3

4 Example: Example Let continuous random variable X denote the current measured in a thin copper in milliamperes (ma). Assume that the range of X is [0, 20mA], and the probability density function is f (x) = = 0.05, 0 x 20. The mean of X is E(X) = 20 x dx = 2 = 10mA The variance is V(X) = 20 0 (x 10) 2 20 dx = (20)2 12 = 33.33mA2 What is the probability that a measurement of current is between 5 and 10 ma? P(5 < x 10) =? P(5 < x 10) = F(10) F(5) = = 0.25 s 5.4

5 (Gaussian) distribution is the most widely used model. For a repeated random experiment, the average of the outcomes tends to have a normal distribution (central limit theorem). The density function of a normal random variable is characterized by two parameters: mean µ and variance σ 2 as shown in Fig. 1. Each curve is symmetric and bell-shaped. µ determines the center and σ 2 determines the width. s 5.5

6 (Gaussian) s Figure 1: probability density functions for selected values of the parameters µ and σ

7 (Gaussian) Definition ( (Gaussian) ) A continuous random variable X with probability density function f (x; µ, σ) = = 1 (x µ)2 e 2σ 2, < x < (5) 2πσ 2 1 σ (x µ) 2 2π e 2σ 2, < x < (6) is a normal (Gaussian) random variable with parameters µ, where < µ <, and σ > 0. Theorem The distribution is denoted by N(µ, σ). And the mean and variance of X are equal to E(X) = µ and V(X) = σ 2, respectively. s 5.7

8 (Gaussian) Figure 2: Probability density function of a normal random variable with mean µ and σ 2. About 68% of the population is in the interval µ ± σ. About 95% of the population is in the interval µ ± 2σ. About 99.7% of the population is in the interval µ ± 3σ. s 5.8

9 Show that I = e y2 /2 dy = 2π I 2 = = = = 2π 0 2π 0 = 2π = 2π e x2 /2 dx 0 dθ 0 0 I = 2π = e y2 /2 dy e (x2 +y 2 )/2 dxdy e r2 /2 rdrdθ 0 e r2 /2 rdr e r2 /2 d r2 2 e s ds = 2π e y2 /2 dy (dxdy = rdrdθ) (r 2 /2 = s) s 5.9

10 Mean and Variance of Random Variable Show that E(X) = µ. E(X) = = = 1 x σ 2π e 1 (σy + µ)e y2 /2 dy 2π (x µ) 2 2σ 2 dx (y = (x µ)/σ) σ ye y2 /2 dy + µ 1 e y2 /2 dy 2π 2π = 0 + µ 1 2π = µ Show that V(X) = σ 2. e y2 /2 dy s 5.10

11 Standard Random Variable Definition (Standard Random Variable) A normal random variable with µ = 0 and σ = 1, N(0, 1), is called a standard normal random variable and is denoted as Z. Definition (Standard Random Variable) The cumulative distribution function of a standard normal random variable is denoted as Φ(z) = P(Z z) s 5.11

12 Standardized Random Variable Definition (Standardized Random Variable) If X is a normal random variable N(µ, σ), the random variable Z = X µ σ is a normal random variable N(0, 1). That is, Z is a standard normal random variable. Also, ( X µ P(X x) = P σ x µ σ (7) ) = P(Z z) (8) where z = x µ (9) σ is the z value (or z score ) obtained by standardizing X. s 5.12

13 Example: Standard Example Aluminum sheets used to make beverage cans have thicknesses (in thousandths of an inch) that are normally distributed with mean 10 and standard deviation 1.3. A particular sheet is 10.8 thousandths of an inch thick. Find the z-score. The quantity 10.8 is an observation from a normal population with mean µ = 10 and standard deviation σ = 1.3. Therefore z = x µ σ = = 0.62 s 5.13

14 Example: Standard Example Referring to the previous example. The thickness of a certain sheet has a z-score of 1.7. Find the thickness of the sheet in the original units of thousandths of inches. We use Eq. (9), substituting 1.7 for z and solving for x. We obtain 1.7 = x Solving for x yields x = 7.8. The sheet is 7.8 thousandths of an inch thick. s 5.14

15 Example: Standard Example Find the area under the normal curve to the left of z = From the z table, the area is s 5.15

16 Example: Standard Example Find the area under the normal curve to the right of z = From the z table, the area to the left of z = 1.38 is Therefore the area to the right is = s 5.16

17 Example: Standard Example Find the area under the normal curve between z = 0.71 and z = From the z table, the area to the left of z = 1.28 is The area to the left of z = 0.71 is The area between z = 0.71 and z = 1.28 is therefore = s 5.17

18 Example: Grades and Example The grades in a large class (like Statistical Data Analysis) are approximately normal-distributed with mean 75 and standard deviation 6. The lowest D is 60, the lowest C is 68, the lowest B is 83, and the lowest A is 90. What proportion of the class will get A s, B s, C s, D s and F s? A s = P(X 90) = P(Z 15 6 ) = B s = P(83 X < 90) = P(Z < 2.5) P(Z < 1.33) = = C s = P(68 X < 83) = P(Z < 1.33) P(Z < 1.17) = = s 5.18

19 Example: Grades and D s = P(60 X < 68) = P(Z < 1.17) P(Z < 2.5) = = F s = P(X < 60) = P(Z < 2.5) = s 5.19

20 Example: Digital Communication Example In a digital communication channel, assume that the number of bits received in error can be modelled by a binomial random variable, and assume that the probability that a bit received in error is If 16 million bits are transmitted, what is the probability that 150 or fewer errors occur? Let X denote the number of errors. Then Xis a binomial random variable. And 150 ( ) P(X 150) = (10 5 ) x ( ) x x How to compute this equation?. 0 s 5.20

21 Approximation of Binomial Theorem If X is a binomial random variable with parameters n and p Z = X np np(1 p) (10) is approximately a standard normal random variable. To approximate a binomial probability with a normal distribution a continuity correction is applied as follows: ( ) x np P(X x) = P(X x + 0.5) = P Z (11) np(1 p) P(x X) = P(x 0.5 X) = P ( ) x 0.5 np Z np(1 p) This approximation is good for np > 5 and n(1 p) > 5. (12) s 5.21

22 s Figure 3: approximation to the binomial distribution with parameters n = 10, and p =

23 Example: Digital Communication (cont) Since np = ( )( ) = 160 > 5 and n(1 p) > 5, we can use the normal distribution to approximate the original binomial distribution as: P(X 150) = P(X ) ( ) X = P 160( ) 160( ) = P(Z 0.75) = s 5.23

24 Non-symmetric Figure 4: Binomial distribution is not symmetric if p is close to 0 or 1. (If np or n(1 p) is small, the binomial is quite skewed and the symmetric normal distribution is not a good approximation. ) s 5.24

25 Approximation of Poisson Theorem If X is a Poisson random variable with E(X) = λ and V(X) = λ, Z = X λ λ (13) is approximately a standard normal distribution. This approximation is good for λ > 5. s 5.25

26 Poisson with Small λ s Figure 5: Poisson distributions for small values of the parameter λ. 5.26

27 Poisson with Large λ s Figure 6: Poisson distributions for selected large values of the parameter λ. 5.27

28 Example: Approximation to Poisson Example Assume that the number particles in a squared meter of dust on a surface follows a Poisson distribution with a mean of If a squared meter of dust is analyzed, what is the probability that 950 or fewer particles are found? The probability can be expressed as P(X 950) = e x The probability can be approximated as P(X x) = P(Z x! ) = P(Z 1.58) = s 5.28

29 Recall that the distribution of the number of trials needed for the first success in a sequence of Bernoulli trials is geometric. Consider a sequence of events that occur randomly in time according to the Poisson distribution at rate λ > 0. The distribution of the number of events N(t) in the interval [0, t] is λt (λt)k P(N(t) = k) = e. k! Suppose that we are interested in the distribution of the waiting time for the first event. Let T denote this random variable. Then P(T > t) = P(no event in [0, t]) = P(N(t) = 0) = e λt. s 5.29

30 Since the cumulative distribution function of T is F(t) = P(T t) = 1 P(T > t) = 1 e λt, the density of T is given by f (t) = d dt F(t) = d P(T > t) { dt λe λt, for t 0 = 0, for t < 0. (14) s 5.30

31 Definition The random variable X that equals the distance (time or length) of a Poisson process with the rate λ > 0 is an exponential random variable with parameter λ. The probability density function of X is f (x) = λe λx, 0 x < (15) The cumulative distribution function is F(x) = 1 e λx, 0 x < (16) It is important to use consistent units in calculation of probabilities, means and variances involving exponential random variables. s 5.31

32 s Figure 7: Probability density functions of exponential random variables for selected values of the parameter λ. 5.32

33 Theorem If random variable X has an exponential distribution with parameter λ, µ = E(X) = 1 λ and σ2 = V(X) = 1 λ 2 (17) µ = E(X) = = = 0 xλe λx dx xe λx dλx = ( xe λx) = 1 λ e λx 0 xde λx e λx dx 0 = 0 1 λ = 1 λ s 5.33

34 Example: Computer Network Usage Example In a large corporate computer network, user log-ons to the system can be modeled as a Poisson process with mean of 25 log-ons per hour. What is probability that there are no log-ons in an interval of 6 minutes. Let the random variable X denote the time from the start of the interval until the first log-on. X has an exponential distribution with λ = 25 log-ons per hour. In addition, 6 minutes is equal to 0.1 hour. The probability of no log-ons in an interval of 6 minutes is P(X > 0.1) = 1 F(0.1) = e 25(0.1) = s 5.34

35 Example: Computer Network Usage What is probability that the time until next log-on is between 2 and 3 minutes ( 2 P 60 = < X < 3 ) 60 = 0.05 = F(0.05) F(0.033) = e 25(0.05) e 25(0.033) = The mean time until the next log-on is µ = 1 λ = 1 = 0.04 hr = 2.4 min 25 The standard deviation of the time until the next log-on is mean time until the next log-on is σ = 1 hr = 2.4 min 25 s 5.35

36 Example: Lack of Memory Example Let X denote the time between detections of a particle with a Geiger counter. Assume that X has an exponential distribution with E(X) = 1.4 minutes. The probability that we detect a particle within 30 seconds of starting the counter is P(X < 0.5 min) = F(0.5) = 1 e 0.5/1.4 = Suppose that the counter has been on for 3 minutes without detecting a particle. What is the probability that we detect a particle in next 30 seconds: P(X < 3.5 X > 3) = P(3 < X < 3.5)/P(X > 3) s 5.36

37 Example: Lack of Memory We have P(3 < X < 3.5) = F(3.5) F(3) = P(X > 3) = 1 F(3) = P(X < 3.5 X > 3) = P(3 < X < 3.5)/P(X > 3) = 0.035/0.117 = 0.30 After waiting for 3 minutes without a detection, the probability of a detection in next 30 seconds is the same as the probability of a detection in the 30 seconds immediately after starting the counter. Theorem (Lack of Memory) For an exponential random variable X P(X < (t 1 + t 2 ) X > t 1 ) = P(X < t 2 ) (18) s 5.37

38 Example: Erlang Example (CPU Failure) The failure of the CPUs of large computer systems are often modeled as a Poisson process. Assume that the units that fail are immediately repaired, and assume that the mean number of failure per hour is Let X denote the time until four failures occur in a system. Determine the probability that X exceeds 40, 000 hours. Let the random variable N denote the number of failures in 40, 000 hours. The time until four failures occur exceeds 40, 000 hours if the number of failures in 40, 000 hours in three or less: P(X > 40, 000) = P(N 3) s 5.38

39 Example: Erlang N has a Poisson distribution with E(N) = 40, 000(0.0001) = 4 failures in 40,000 hours Therefore, 3 e 4 4 k P(X > 40, 000) = P(N 3) = = k! k=0 s 5.39

40 Erlang If X is the time until the rth event in a Poisson process then P(X > x) = r 1 k=0 e λx (λx) k Since P(X > x) = 1 F(X), the probability density function of X equals f (x) = d dx P(X > x) = λr x r 1 e λx for x > 0 and r = 1, 2,.... (r 1)! k! = λ(λx)r 1 e λx (r 1)! This probability density function defines an Erlang distribution. With r = 1, an Erlang RV becomes an exponential RV. s 5.40

41 Function Definition ( Function) The gamma function of γ is Γ(γ) = 0 x γ 1 e x dx, for γ > 0. (19) The value of the integral is a positive finite number. Using the integral by parts, it can be shown that Γ(γ) = (γ 1)Γ(γ 1) If γ is a positive integer, (as in Erlang distribution), Γ(γ) = (γ 1)!, given that γ(1) = 0! = 1. β γ Γ(γ) = 0 y γ 1 e y/β dy. s 5.41

42 Definition ( ) A random variable X that has a pdf { λ γ x γ 1 e λx Γ(γ) = λ Γ(γ) f (x; γ, λ) = (λx)γ 1 e λx, 0 < x < 0, elsewhere. is said to have a gamma distribution with parameters γ > 0 and λ > 0. The parameters γ and λ are called the scale and shape parameters, respectively. If γ is a positive integer r, X becomes an Erlang distribution. (20) s 5.42

43 s Figure 8: probability density functions for selected values of the parameters γ (scale) and λ (shape). 5.43

44 Theorem (Mean and Variance of ) If X is a gamma random variable with parameters λ and γ, µ = E(X) = γ λ (21) and σ 2 = V(X) = γ λ 2 (22) Definition (Chi-Squared ) The chi-squared distribution is a special case of gamma distribution in which λ = 1/2 and γ = 1/2, 1, 3/2, 2,.... The chi-squared distribution is used extensively in interval estimation and hypothesis testing. s 5.44

45 Example: Example The time to prepare a micro-array slide for high throughput genomics is a Poisson process with a mean of two hours per slide. What is the probability that 10 slides require more than 25 hours to prepare? Let X denote the time to prepare 10 slides. X has gamma distribution with λ = 1/2 (slide/hour) and γ = 10. Th requested probability P(X > 25) P(X > 25) = 9 e 12.5 (12.5) k = k! k=0 The mean time to prepare 10 slides is E(X) = γ/λ = 10/0.5 = 20 hours. And the variance time is V(X) = γ/λ 2 = 40 s 5.45

46 Definition ( ) The random variable X with the probability density function f (x) = β ( x ) [ β 1 ( x ) ] β exp, x > 0 (23) δ δ δ is a random variable with scale parameter δ > 0 and shape parameter β > 0. The distribution is often used to model the time until failure of many different physical systems. When β = 1, the distribution is identical to the exponential. The Raleigh distribution is a special case when the shape parameter β = 2. s 5.46

47 s Figure 9: probability density functions for selected values of the parameters δ (scale) and β (shape). 5.47

48 Theorem If X has a distribution with parameters δ and β, then the cumulative distribution function of X is [ ( x ) ] β F(x) = 1 exp (24) δ Theorem If X has a distribution with parameters δ and β, ( µ = E(X) = δγ ) (25) β ( σ 2 = V(X) = δ 2 Γ ) [ ( δ 2 Γ )] 2 (26) β β s 5.48

49 Example: Example The time to failure (in hours) of bearing a mechanical shaft is modeled as a random variable with δ = 5000 hours and β = 1/2. Determine the mean time until failure E(X) = 5000Γ[1 + (1/0.5)] = 5000Γ[3] = ! = 10, 000 hours Determine the probability that a bearing last at least 6000 hours [ ( ) ] /2 P(X > 6000) = 1 F(6000) = exp 5000 = e = s 5.49

50 Let W be a normal distribution. X = exp(w) is also an random variable. Since log(x) is normally distributed, X is called a lognormal distribution. The cumulative distribution function for X is F(x) = P(X x) = P(exp(W) x) = P(W log(x)) ( = P Z log(x) θ ) ( ) log(x) θ = Φ ω ω for x > 0, where Z is a standard normal random variable. F(x) = 0 for x 0. s 5.50

51 Definition ( ) Let W have a normal distribution with mean θ and variance ω 2 ; then X = exp(w) is a lognormal random variable with probability density function ] 1 f (x) = [ xω 2π exp (log(x) θ)2 2ω 2 for 0 < x <. The mean and variance of X are E(X) = exp(θ + ω 2 /2) V(X) = e 2θ+ω2 (e ω2 1) The lifetime of a product that degrades over time is often modeled by a lognormal distribution random variable. (27) s 5.51

52 s Figure 10: probability density functions with θ = 0 for selected values of σ

53 s Figure 11: probability density functions with θ = 0 for selected values of ω

54 s Figure 12: probability density functions with selected values of θ and ω

55 Example: Example The lifetime of a semiconductor laser has a lognormal distribution with θ = 10 hours and ω = 1.5 hours. What is the probability the lifetime exceeds 10, 000 hours? P(X > 10000) = 1 P(exp(W) 10000) = 1 P(W log(10000)) ( ) log(10000) 10 = 1 Φ 1.5 = 1 Φ( 0.52) = = 0.70 s 5.55

56 Example: What lifetime is exceeded by 99% of lasers? P(X > x) = P(exp(W) > x) = P(W > log(x)) ( ) log(x) 10 = 1 Φ = Φ(z) = 0.99 when z = Therefore, log(x) = 2.33 x = exp(6.505) = hours s 5.56

57 Example: Determine the mean and variance of lifetime. E(X) = exp(θ + ω 2 /2) = exp( ) = 67, V(X) = e 2θ+ω2 (e ω2 1) = exp( )[exp(2.25) 1] = 39, 070, 059, σ = V(x) = 197, E(X) Notice that the standard deviation of life time is much larger to the mean. s 5.57