UCLA STAT 11 A Applied Probability & Statistics for Engineers Instructor: Ivo Dinov, Asst. Prof. In Statistics and Neurology Teaching Assistant: Christopher Barr University of California, Los Angeles, Fall 4 http://www.stat.ucla.edu/~dinov/ Chapter 4 Continuous Random Variables and Probability Distributions Slide 1 Slide 4.1 Continuous Random Variables and Probability Distributions Continuous Random Variables A random variable X is continuous if its set of possible values is an entire interval of numbers (If A < B, then any number x between A and B is possible). Slide 3 Slide 4 Probability Distribution Let X be a continuous rv. Then a probability distribution or probability density function (pdf) of X is a function f (x) such that for any two numbers a and b, b ( ) ( ) P a X b = f x dx a The graph of f is the density curve. Probability Density Function For f (x) to be a pdf 1. f (x) > for all values of x..the area of the region between the graph of f and the x axis is equal to 1. Area = 1 y = f( x) Slide 5 Slide 6 1
Probability Density Function Pa ( X b) is given by the area of the shaded region. y = f( x) Continuous RV s A RV is continuous if it can take on any real value in a non-trivial interval (a ; b). PDF, probability density function, for a cont. RV, Y, is a non-negative function p Y (y), for any real value y, such that for each interval (a; b), the probability that Y takes on a value in (a; b), P(a<Y<b) equals the area under p Y (y) over the interval (a: b). p Y (y) P(a<Y<b) a b a b Slide 7 Slide 8 Convergence of density histograms to the PDF For a continuous RV the density histograms converge to the PDF as the size of the bins goes to zero. AdditionalInstructorAids\BirthdayDistribution_1978_systat.SYD Convergence of density histograms to the PDF For a continuous RV the density histograms converge to the PDF as the size of the bins goes to zero. Slide 9 Slide 1 Uniform Distribution A continuous rv X is said to have a uniform distribution on the interval [A, B] if the pdf of X is 1 A x B f ( x; A, B) = B A otherwise Probability for a Continuous rv If X is a continuous rv, then for any number c, P(x = c) =. For any two numbers a and b with a < b, P( a X b) = P( a< X b) = P( a X < b) = P( a< X < b) Slide 11 Slide 1
4. Cumulative Distribution Functions and Expected Values The Cumulative Distribution Function The cumulative distribution function, F(x) for a continuous rv X is defined for every number x by = ( ) = x F( x) P X x f( y) dy For each x, F(x) is the area under the density curve to the left of x. Slide 13 Slide 14 Using F(x) to Compute Probabilities Let X be a continuous rv with pdf f(x) and cdf F(x). Then for any number a, P ( X > a) = 1 F( a) and for any numbers a and b with a < b, Obtaining f(x) from F(x) If X is a continuous rv with pdf f(x) and cdf F(x), then at every number x for which the derivative F x F ( x) = f( x). ( ) exists, P ( a X b) = F( b) F( a) Slide 15 Slide 16 Percentiles Median Let p be a number between and 1. The (1p)th percentile of the distribution of a continuous rv X denoted by η( p), is defined by η = ( η ) = ( p) p F ( p) f( y) dy The median of a continuous distribution, denoted by µ%, is the 5 th percentile. So µ% satisfies.5 = F ( % µ ). That is, half the area under the density curve is to the left of µ%. Slide 17 Slide 18 3
Expected Value The expected or mean value of a continuous rv X with pdf f (x) is ( ) ( ) µ X = E X = x f x dx Expected Value of h(x) If X is a continuous rv with pdf f(x) and h(x) is any function of X, then [ ] µ hx ( ) E hx ( ) = = hx ( ) f( xdx ) Slide 19 Slide Variance and Standard Deviation The variance of continuous rv X with pdf f(x) and mean µ is σ X = V( x) = ( x µ ) f( x) dx = E[ X µ ] ( ) The standard deviation isσ = X V( x). Short-cut Formula for Variance ( ) [ ] V( X) = E X E( X) Slide 1 Slide Normal Distributions 4.3 The Normal Distribution A continuous rv X is said to have a normal distribution with parameters µ and σ, where < µ < and < σ, if the pdf of X is 1 ( x µ ) /( σ ) f( x) = e < x< σ π Slide 3 Slide 4 4
Standard Normal Distributions The normal distribution with parameter values µ = and σ = 1 is called a standard normal distribution. The random variable is denoted by Z. The pdf is 1 z f(;,1) z = e / σ π The cdf is z Φ ( z) = PZ ( z) = f( y;,1) dy Slide 5 < z < Standard Normal Cumulative Areas Standard normal curve z Slide 6 Shaded area = Φ( z) Standard Normal Distribution Let Z be the standard normal variable. Find (from table) a. PZ (.85) Area to the left of.85 =.83 b. P(Z > 1.3) 1 PZ ( 1.3) =.934 c. P(.1 Z 1.78) Find the area to the left of 1.78 then subtract the area to the left of.1. = PZ ( 1.78) PZ (.1) =.965.179 =.9446 Slide 7 Slide 8 z α Notation z α will denote the value on the measurement axis for which the area under the z curve lies to the right of z α. Slide 9 Shaded area = PZ ( z α ) = α z α Ex. Let Z be the standard normal variable. Find z if a. P(Z < z) =.978. Look at the table and find an entry =.978 then read back to find z = 1.46. b. P( z < Z < z) =.813 P(z < Z < z ) = P( < Z < z) = [P(z < Z ) ½] = P(z < Z ) 1 =.813 P(z < Z ) =.966 z = 1.3 Slide 3 5
Nonstandard Normal Distributions If X has a normal distribution with mean µ and standard deviation σ, then X µ Z = σ has a standard normal distribution. Normal Curve Approximate percentage of area within given standard deviations (empirical rule). 99.7% 95% 68% Slide 31 Slide 3 Ex. Let X be a normal random variable with µ = 8 and σ =. Find PX ( 65). 65 8 P( X 65) = P Z (.75) = P Z =.66 Ex. A particular rash shown up at an elementary school. It has been determined that the length of time that the rash will last is normally distributed with µ = 6 days and σ = 1.5 days. Find the probability that for a student selected at random, the rash will last for between 3.75 and 9 days. Slide 33 Slide 34 3.75 6 9 6 P( 3.75 X 9) = P Z 1.5 1.5 ( 1.5 Z ) = P =.977.668 Percentiles of an Arbitrary Normal Distribution (1p)th percentile for normal µ, σ ( ) (1 p)th for = µ + σ standard normal =.914 Slide 35 Slide 36 6
Normal Approximation to the Binomial Distribution Let X be a binomial rv based on n trials, each with probability of success p. If the binomial probability histogram is not too skewed, X may be approximated by a normal distribution with µ = np and σ = npq. x+.5 np PX ( x) Φ npq Ex. At a particular small college the pass rate of Intermediate Algebra is 7%. If 5 students enroll in a semester determine the probability that at least 375 students pass. µ = np = 5(.7) = 36 σ = npq = 5(.7)(.8) 1 375.5 36 PX ( 375) Φ =Φ(1.55) 1 =.9394 Slide 37 Slide 38 Normal approximation to Binomial Suppose Y~Binomial(n, p) Then Y=Y 1 + Y + Y 3 + + Y n, where Y k ~Bernoulli(p), E(Y k )=p & Var(Y k )=p(1-p) E(Y)=np & Var(Y)=np(1-p), SD(Y)= (np(1- p)) 1/ Standardize Y: Z=(Y-np) / (np(1-p)) 1/ By CLT Z ~ N(, 1). So, Y ~ N [np, (np(1- p)) 1/ ] Normal Approx to Binomial is reasonable when np >=1 & n(1- p)>1 (p & (1-p) are NOT too small relative to Slide 39 n) Normal approximation to Binomial Example Roulette wheel investigation: Compute P(Y>=58), where Y~Binomial(1,.47) The proportion of the Binomial(1,.47) population having more than 58 reds (successes) out of 1 roulette spins (trials). Roulette has 38 slots Since np=47>=1 & n(1-p)=53>1 18red 18black neutral Normal approx is justified. Z=(Y-np)/Sqrt(np(1-p)) = 58 1*.47)/Sqrt(1*.47*.53)=. P(Y>=58) P(Z>=.) =.139 True P(Y>=58) =.177, using SOCR (demo!) Slide 4 Binomial approx useful when no access to SOCR Normal approximation to Poisson Let X 1 ~Poisson(λ) & X ~Poisson(µ) X 1 + X ~Poisson(λ+µ) Let X 1, X, X 3,, X k ~ Poisson(λ), and independent, Y k = X 1 + X + + X k ~ Poisson(kλ), E(Y k )=Var(Y k )=kλ. The random variables in the sum on the right are independent and each has the Poisson distribution with parameter λ. By CLT the distribution of the standardized variable (Y k kλ) / (kλ) 1/ N(, 1), as k increases to infinity. Slide 41 S f k > 1 Z {(Y k )/(k ) 1/ } Normal approximation to Poisson example Let X 1 ~Poisson(λ) & X ~Poisson(µ) X 1 + X ~Poisson(λ+µ) Let X 1, X, X 3,, X ~ Poisson(), and independent, Y k = X 1 + X + + X k ~ Poisson(4), E(Y k )=Var(Y k )=4. By CLT the distribution of the standardized variable (Y k 4) / (4) 1/ N(, 1), as k increases to infinity. Z k = (Y k 4) / ~ N(,1) Y k ~ N(4, 4). P( < Y k < 4) = (std z & 4) = Slide 4 P( ( 4)/ < Z k < (4 4)/ ) = P( -< 7
Poisson or Normal approximation to Binomial? Poisson Approximation (Binomial(n, p n ) Poisson(λ) ): y y WHY? λ n p y n ( 1 p ) n y n n p n λ n>=1 & p<=.1 & λ =n p <= Normal Approximation (Binomial(n, p) N ( np, (np(1-p)) 1/ ) ) np >=1 & n(1-p)>1 Slide 43 n e λ y! 4.4 The Gamma Distribution and Its Relatives Slide 44 The Gamma Function For α >, the gamma function Γ( α ) is defined by α 1 x Γ ( α ) = x e dx Gamma Distribution A continuous rv X has a gamma distribution if the pdf is 1 α 1 x / β x e x α f( x; αβ, ) = β Γ( α) otherwise where the parameters satisfyα >, β >. The standard gamma distribution has β = 1. Slide 45 Slide 46 Mean and Variance The mean and variance of a random variable X having the gamma distribution f( x; α, β ) are Slide 47 EX ( ) = µ = αβ VX ( ) = σ = αβ Probabilities from the Gamma Distribution Let X have a gamma distribution with parameters α and β. Then for any x >, the cdf of X is given by x PX ( x) = Fx ( ; α, β ) = F ; α β where x α 1 y y e F( x; α) = dy Γ( α) Slide 48 8
Exponential Distribution A continuous rv X has an exponential distribution with parameter λ if the pdf is λx λe x f( x; λ) = otherwise Mean and Variance The mean and variance of a random variable X having the exponential distribution 1 1 µ = αβ = σ = αβ = λ λ Slide 49 Slide 5 Probabilities from the Gamma Distribution Let X have a exponential distribution Then the cdf of X is given by x < F( x; λ) = λx 1 e x Slide 51 Applications of the Exponential Distribution Suppose that the number of events occurring in any time interval of length t has a Poisson distribution with parameter αt and that the numbers of occurrences in nonoverlapping intervals are independent of one another. Then the distribution of elapsed time between the occurrences of two successive events is exponential with parameter λ = α. Slide 5 The Chi-Squared Distribution Let v be a positive integer. Then a random variable X is said to have a chisquared distribution with parameter v if the pdf of X is the gamma density with α = v / and β =. The pdf is 1 ( v/) 1 x/ x e x v / f( x; v) = Γ( v /) x < The Chi-Squared Distribution The parameter v is called the number of degrees of freedom (df) of X. The symbol χ is often used in place of chisquared. Slide 53 Slide 54 9
Identifying Common Distributions QQ plots Quantile-Quantile plots indicate how well the model distribution agrees with the data. q -th quantile, for <q<1, is the (data-space) value, V q, at or below which lies a proportion q of the data. 1 Graph of the CDF, F Y (y)=p(y<=v q )=q q Slide 55 V q Constructing QQ plots Start off with data {y 1, y, y 3,, y n } Order statistics y (1) <= y () <= y (3) <= <= y (n) Compute quantile rank, q (k), for each observation, y (k), P(Y<= q (k) ) = (k-.375) / (n+.5), where Y is a RV from the (target) model distribution. Finally, plot the points (y (k), q (k) ) in D plane, 1<=k<=n. Note: Different statistical packages use slightly different formulas for the computation of q (k). However, the results are quite similar. This is the formulas employed in SAS. Basic idea: Probability that: Slide 56 P((model)Y<=(data)y )~ 1/n; Example - Constructing QQ plots Start off with data {y 1, y, y 3,, y n }. Plot the points (y (k), q (k) ) in D plane, 1<=k<=n. Expected Value for Normal Distribution 3 1-1 - -3 Slide 57 C:\Ivo.dir\UCLA_Classes\Winter\AdditionalInstructorA ids BirthdayDistribution_1978_systat.SYD SYSTAT, Graph Probability Plot, Var4, Normal Distribution 4.5 Other Continuous Distributions Slide 58 The Weibull Distribution A continuous rv X has a Weibull distribution if the pdf is α α α 1 ( x / β) x e x α f( x; αβ, ) = β x < where the parameters satisfy α >, β >. Mean and Variance The mean and variance of a random variable X having the Weibull distribution are 1 1 µ = βγ 1+ σ = β Γ 1+ Γ 1+ α α α Slide 59 Slide 6 1
Weibull Distribution The cdf of a Weibull rv having parameters α and β is α ( x / β ) 1 e x F( x; αβ, ) = x < Lognormal Distribution A nonnegative rv X has a lognormal distribution if the rv Y = ln(x) has a normal distribution the resulting pdf has parameters µ and σ and is 1 [ln( x) µ ] /( σ ) e x f( x; µσ, ) = παx x < Slide 61 Slide 6 Mean and Variance The mean and variance of a variable X having the lognormal distribution are µ + σ / µ + σ σ ( ) E( X) = e V( X) = e e 1 Lognormal Distribution The cdf of the lognormal distribution is given by F( x; µ, α ) = P( X x) = P[ln( X) ln( x)] ln( x) µ ln( x) µ = P Z =Φ σ σ Slide 63 Slide 64 Beta Distribution A rv X is said to have a beta distribution with parameters A, B, α >, and β > if the pdf of X is f( x; αβ,, A, B) = 1 Γ ( α + β) x A B x B A Γ( α) Γ( β) B A B A otherwise α 1 β 1 x Mean and Variance The mean and variance of a variable X having the beta distribution are α µ = A+ ( B A) α + β ( B A) αβ σ = ( α + β) ( α + β + 1) Slide 65 Slide 66 11
4.6 Probability Plots Sample Percentile Order the n-sample observations from smallest to largest. The ith smallest observation in the list is taken to be the [1(i.5)/n]th sample percentile. Slide 67 Slide 68 Probability Plot [1( i.5) / n]th percentile ith smallest sample of the distribution observation If the sample percentiles are close to the corresponding population distribution percentiles, the first number will roughly equal the second., Normal Probability Plot A plot of the pairs ([1( i.5) / n]th z percentile, ith smallest observation) On a two-dimensional coordinate system is called a normal probability plot. If the drawn from a normal distribution the points should fall close to a line with slope σ and intercept µ. Slide 69 Slide 7 Beyond Normality Consider a family of probability distributions involving two parameters θ Let denote the 1 and θ. F( x; θ1, θ) corresponding cdf s. The parameters θ1 and θ are said to location and scale parameters if x θ1 F( x; θ1, θ) is a function of. θ Lognormal (Y) µ,σ Relation among Distributions Normal (X) µ,σ X = lny X Y = e Uniform(X) α, β α U = X β α Beta α, β α = β = 1 χ Uniform(U),1 µ Z = X σ = n i= 1 Z i X = ( β α) U + α Normal (Z),1 Chi-square ( χ ) n α = n /, β = Gamma α, β n= α =1 X = β lnu df Weibull γ, β df γ = 1 Exponential(X) β T df=n (,1) 1 Cauchy (,1) Slide 71 Slide 7 1