UCLA STAT 35 Applied Computational and Interactive Probability Instructor: Ivo Dinov, Asst. Prof. In Statistics and Neurology Teaching Assistant: Chris Barr Continuous Random Variables and Probability Distributions University of California, Los Angeles, Winter 6 http://www.stat.ucla.edu/~dinov/ Stat 35, UCLA, Ivo Dinov Slide 1 Slide Continuous Random Variables and Probability Distributions Continuous Random Variables A random variable X is continuous if its set of possible values is an entire interval of numbers (If A < B, then any number x between A and B is possible). Slide 3 Slide 4 Probability Distribution Let X be a continuous rv. Then a probability distribution or probability density function (pdf) of X is a function f (x) such that for any two numbers a and b, b ( ) ( ) P a X b = f xdx The graph of f is the density curve. a Probability Density Function For f (x) to be a pdf 1. f (x) > for all values of x..the area of the region between the graph of f and the x axis is equal to 1. Area = 1 y = f( x) Slide 5 Slide 6 1
Probability Density Function Pa ( X b) is given by the area of the shaded region. y = f( x) Continuous RV s A RV is continuous if it can take on any real value in a non-trivial interval (a ; b). PDF, probability density function, for a cont. RV, Y, is a non-negative function p Y (y), for any real value y, such that for each interval (a; b), the probability that Y takes on a value in (a; b), P(a<Y<b) equals the area under p Y (y) over the interval (a: b). p Y (y) P(a<Y<b) a b a b Slide 7 Slide 8 Convergence of density histograms to the PDF For a continuous RV the density histograms converge to the PDF as the size of the bins goes to zero. AdditionalInstructorAids\BirthdayDistribution_1978_systat.SYD Convergence of density histograms to the PDF For a continuous RV the density histograms converge to the PDF as the size of the bins goes to zero. Slide 9 Slide 1 Uniform Distribution A continuous rv X is said to have a uniform distribution on the interval [A, B] if the pdf of X is 1 A x B ( ;, ) f x A B = B A otherwise UniformEstimateExperiment Probability for a Continuous rv If X is a continuous rv, then for any number c, P(x = c) =. For any two numbers a and b with a < b, Pa ( X b) = Pa ( < X b) = Pa ( X< b) = Pa ( < X< b) Slide 11 Slide 1
Cumulative Distribution Functions and Expected Values The Cumulative Distribution Function The cumulative distribution function, F(x) for a continuous rv X is defined for every number x by ( ) x F( x) P X x f( y) dy = = For each x, F(x) is the area under the density curve to the left of x. Slide 13 Slide 14 Using F(x) to Compute Probabilities Let X be a continuous rv with pdf f(x) and cdf F(x). Then for any number a, P ( X > a) = 1 F( a) and for any numbers a and b with a < b, Obtaining f(x) from F(x) If X is a continuous rv with pdf f(x) and cdf F(x), then at every number x for which the derivative F x F ( x) = f( x). ( ) exists, P ( a X b) = F( b) F( a) Slide 15 Slide 16 Percentiles Median Let p be a number between and 1. The (1p)th percentile of the distribution of a continuous rv X denoted by η( p), is defined by ( η ) η ( p) p F ( p) f( y) dy = = The median of a continuous distribution, denoted by µ%, is the 5 th percentile. So µ% satisfies.5 = F ( % µ ). That is, half the area under the density curve is to the left of µ%. Slide 17 Slide 18 3
Expected Value The expected or mean value of a continuous rv X with pdf f (x) is ( ) ( ) µ X = E X = x f x dx Expected Value of h(x) If X is a continuous rv with pdf f(x) and h(x) is any function of X, then [ ] µ h( X) E h( x) = = h( x) f( x) dx Slide 19 Slide Variance and Standard Deviation The variance of continuous rv X with pdf f(x) and mean µ is σ X = V( x) = ( x µ ) f( x) dx ( ) = E[ X µ ] The standard deviation is σ = X V( x). Short-cut Formula for Variance ( ) [ ] V( X) = E X E( X) Slide 1 Slide Normal Distributions The Normal Distribution A continuous rv X is said to have a normal distribution with parameters µ and σ, where < µ < and < σ, if the pdf of X is 1 ( x µ ) /( σ ) f( x) = e < x< σ π Slide 3 Slide 4 4
Standard Normal Distributions The normal distribution with parameter values µ = and σ = 1 is called a standard normal distribution. The random variable is denoted by Z. The pdf is 1 z f(;,1) z = e / σ π The cdf is z Φ ( z) = P( Z z) = f( y;,1) dy Slide 5 < z < Standard Normal Cumulative Areas Standard normal curve z Slide 6 Shaded area = Φ( z) Standard Normal Distribution Let Z be the standard normal variable. Find (from table) a. PZ (.85) Area to the left of.85 =.83 b. P(Z > 1.3) 1 PZ ( 1.3) =.934 c. P(.1 Z 1.78) Find the area to the left of 1.78 then subtract the area to the left of.1. = PZ ( 1.78) PZ (.1) =.965.179 =.9446 Slide 7 Slide 8 z Notation z will denote the value on the measurement axis for which the area under the z curve lies to the right of z. Slide 9 Shaded area = PZ ( z ) = z Ex. Let Z be the standard normal variable. Find z if a. P(Z < z) =.978. Look at the table and find an entry =.978 then read back to find z = 1.46. b. P( z < Z < z) =.813 P(z < Z < z ) = P( < Z < z) = [P(z < Z ) ½] = P(z < Z ) 1 =.813 P(z < Z ) =.966 z = 1.3 Slide 3 5
Nonstandard Normal Distributions If X has a normal distribution with mean µ and standard deviation σ, then X µ Z = σ has a standard normal distribution. Normal Curve Approximate percentage of area within given standard deviations (empirical rule). 99.7% 95% 68% Slide 31 Slide 3 Ex. Let X be a normal random variable with µ = 8 and σ =. Find PX ( 65). 65 8 P( X 65) = P Z (.75) = P Z =.66 Ex. A particular rash shown up at an elementary school. It has been determined that the length of time that the rash will last is normally distributed with µ = 6 days and σ = 1.5 days. Find the probability that for a student selected at random, the rash will last for between 3.75 and 9 days. BivariateNormalExperiment Slide 33 Slide 34 3.75 6 9 6 P( 3.75 X 9) = P Z 1.5 1.5 ( 1.5 Z ) = P =.977.668 Percentiles of an Arbitrary Normal Distribution (1p)th percentile for normal µ, σ ( ) (1 p)th for = µ + σ standard normal =.914 Slide 35 Slide 36 6
Normal Approximation to the Binomial Distribution Let X be a binomial rv based on n trials, each with probability of success p. If the binomial probability histogram is not too skewed, X may be approximated by a normal distribution with µ = np and σ = npq. x +.5 np PX ( x) Φ npq Ex. At a particular small college the pass rate of Intermediate Algebra is 7%. If 5 students enroll in a semester determine the probability that at least 375 students pass. µ = np = 5(.7) = 36 σ = npq = 5(.7)(.8) 1 375.5 36 PX ( 375) Φ =Φ(1.55) 1 =.9394 Slide 37 Slide 38 Normal approximation to Binomial Suppose Y~Binomial(n, p) Then Y=Y 1 + Y + Y 3 + + Y n, where Y k ~Bernoulli(p), E(Y k )=p & Var(Y k )=p(1-p) E(Y)=np & Var(Y)=np(1-p), SD(Y)= (np(1-p)) 1/ Standardize Y: Z=(Y-np) / (np(1-p)) 1/ By CLT Z ~ N(, 1). So, Y ~ N [np, (np(1-p)) 1/ ] Normal Approx to Binomial is reasonable when np >=1 & n(1-p)>1 (p & (1-p) are NOT too small relative to n). Normal approximation to Binomial Example Roulette wheel investigation: Compute P(Y>=58), where Y~Binomial(1,.47) The proportion of the Binomial(1,.47) population having more than 58 reds (successes) out of 1 roulette spins (trials). Since np=47>=1 & n(1-p)=53>1 Normal approx is justified. Z=(Y-np)/Sqrt(np(1-p)) = Roulette has 38 slots 18red 18black neutral 58 1*.47)/Sqrt(1*.47*.53)=. P(Y>=58) P(Z>=.) =.139 True P(Y>=58) =.177, using SOCR (demo!) Binomial approx useful when no access to SOCR avail. Slide 39 Slide 4 Normal approximation to Poisson Let X 1 ~Poisson(l) & X ~Poisson(m) X 1 + X ~Poisson(l+m) Let X 1, X, X 3,, X k ~ Poisson(l), and independent, Y k = X 1 + X + + X k ~ Poisson(kl), E(Y k )=Var(Y k )=kl. The random variables in the sum on the right are independent and each has the Poisson distribution with parameter l. By CLT the distribution of the standardized variable (Y k kl) / (kl) 1/ N(, 1), as k increases to infinity. So, for kl >= 1, Z k = {(Y k kl) / (kl) 1/ } ~ N(,1). Y k ~ N(kl, (kl) 1/ ). Normal approximation to Poisson example Let X 1 ~Poisson(l) & X ~Poisson(m) X 1 + X ~Poisson(l+m) Let X 1, X, X 3,, X ~ Poisson(), and independent, Y k = X 1 + X + + X k ~ Poisson(4), E(Y k )=Var(Y k )=4. By CLT the distribution of the standardized variable (Y k 4) / (4) 1/ N(, 1), as k increases to infinity. Z k = (Y k 4) / ~ N(,1) Y k ~ N(4, 4). P( < Y k < 4) = (std z & 4) = P( ( 4)/ < Z k < (4 4)/ ) = P( -< Z k <) =.5 Slide 41 Slide 4 7
Poisson or Normal approximation to Binomial? Poisson Approximation (Binomial(n, p n ) Poisson(λ) ): y y WHY? n p y n ( 1 p ) n y n λ e λ n y! n p n λ n>=1 & p<=.1 & λ =n p <= Normal Approximation (Binomial(n, p) N ( np, (np(1-p)) 1/ ) ) np >=1 & n(1-p)>1 The Gamma Distribution and Its Relatives Slide 43 Slide 44 The Gamma Function For >, the gamma function Γ( ) is defined by 1 x Γ ( ) = x e dx GammaDistribution Gamma Distribution A continuous rv X has a gamma distribution if the pdf is 1 1 x / β x e x f( x; β, ) = β Γ( ) otherwise where the parameters satisfy >, β >. The standard gamma distribution has β = 1. Slide 45 Slide 46 Mean and Variance The mean and variance of a random variable X having the gamma distribution f( x;, β ) are Slide 47 EX ( ) = µ = β VX ( ) = σ = β Probabilities from the Gamma Distribution Let X have a gamma distribution with parameters and β. Then for any x >, the cdf of X is given by x PX ( x) = Fx ( ;, β ) = F ; β where x 1 y y e F( x; ) = dy Γ( ) Slide 48 8
Exponential Distribution A continuous rv X has an exponential distribution with parameter λ if the pdf is λe f( x; λ) = λx ExponentialDistribition x otherwise Mean and Variance The mean and variance of a random variable X having the exponential distribution 1 1 µ = β = σ = β = λ λ Slide 49 Slide 5 Probabilities from the Gamma Distribution Let X have a exponential distribution Then the cdf of X is given by x < F( x; λ) = λx 1 e x Slide 51 Applications of the Exponential Distribution Suppose that the number of events occurring in any time interval of length t has a Poisson distribution with parameter t and that the numbers of occurrences in nonoverlapping intervals are independent of one another. Then the distribution of elapsed time between the occurrences of two successive events is exponential with parameter λ =. Slide 5 The Chi-Squared Distribution Let v be a positive integer. Then a random variable X is said to have a chisquared distribution with parameter v if the pdf of X is the gamma density with = v / and β =. The pdf is 1 ( v/) 1 x/ x e x v / f( x; v) = Γ( v /) x < ChiSquareDistribution Slide 53 The Chi-Squared Distribution The parameter v is called the number of degrees of freedom (df) of X. Applications: 1. F-test = χ (df=v1) /χ (df=v). Sample-Variances ~ χ (df=v1) Slide 54 χ Model Fitting Demo: (SOCR Additional Resources) http://socr.stat.ucla.edu/applets.dir/socrcurvefitter.html 9
Identifying Common Distributions QQ plots Quantile-Quantile plots indicate how well the model distribution agrees with the data. q -th quantile, for <q<1, is the (data-space) value, V q, at or below which lies a proportion q of the data. 1 Graph of the CDF, F Y (y)=p(y<=v q )=q q V q Constructing QQ plots Start off with data {y 1, y, y 3,, y n } Order statistics y (1) <= y () <= y (3) <= <= y (n) Compute quantile rank, q (k), for each observation, y (k), P(Y<= q (k) ) = (k-.375) / (n+.5), where Y is a RV from the (target) model distribution. Finally, plot the points (y (k), q (k) ) in D plane, 1<=k<=n. Note: Different statistical packages use slightly different formulas for the computation of q (k). However, the results are quite similar. This is the formulas employed in SAS. Basic idea: Probability that: P((model)Y<=(data)y (1) )~ 1/n; P(Y<=y () ) ~ /n; P(Y<=y (3) ) ~ 3/n; Slide 55 Slide 56 Example - Constructing QQ plots Start off with data {y 1, y, y 3,, y n }. Plot the points (y (k), q (k) ) in D plane, 1<=k<=n. Expected Value for Normal Distribution 3 1-1 - -3 Slide 57 C:\Ivo.dir\UCLA_Classes\Winter\AdditionalInstructorA ids BirthdayDistribution_1978_systat.SYD SYSTAT, Graph Probability Plot, Var4, Normal Distribution Other Continuous Distributions Slide 58 The Weibull Distribution A continuous rv X has a Weibull distribution if the pdf is 1 ( x / β) x e x f( x; β, ) = β x < where the parameters satisfy >, β >. WeibullDistribution Slide 59 Mean and Variance The mean and variance of a random variable X having the Weibull distribution are 1 1 µ = βγ 1+ σ = β Γ 1+ Γ 1+ Slide 6 1
Weibull Distribution The cdf of a Weibull rv having parameters and β is ( x / β ) 1 e x F( x; β, ) = x < Lognormal Distribution A nonnegative rv X has a lognormal distribution if the rv Y = ln(x) has a normal distribution the resulting pdf has parameters µ and σ and is 1 [ln( x) µ ] /( σ ) e x f( x; µσ, ) = πx x < RandomVariableExperiment LogNormalDistribution Slide 61 Slide 6 Mean and Variance The mean and variance of a variable X having the lognormal distribution are µ + σ / µ + σ σ ( ) EX ( ) = e VX ( ) = e e 1 Lognormal Distribution The cdf of the lognormal distribution is given by F( x; µ, ) = P( X x) = P[ln( X) ln( x)] ln( x) µ ln( x) µ = P Z =Φ σ σ Slide 63 Slide 64 Beta Distribution A rv X is said to have a beta distribution with parameters A, B, >, and β > if the pdf of X is f ( x;, β, A, B) = 1 Γ ( + β) x A B x B A Γ( ) Γ( β) B A B A otherwise BetaDistribution BetaCoinExperiment 1 β 1 BetaEstimateExperiment Slide 65 x Mean and Variance The mean and variance of a variable X having the beta distribution are µ = A+ ( B A) + β ( B A) β σ = ( + β) ( + β + 1) Slide 66 11
Probability Plots Sample Percentile Order the n-sample observations from smallest to largest. The ith smallest observation in the list is taken to be the [1(i.5)/n]th sample percentile. Slide 67 Slide 68 Probability Plot [1( i.5) / n]th percentile ith smallest sample of the distribution observation If the sample percentiles are close to the corresponding population distribution percentiles, the first number will roughly equal the second. ProbabilityPlotExperiment, Slide 69 Normal Probability Plot A plot of the pairs ([1( i.5) / n]th z percentile, ith smallest observation) On a two-dimensional coordinate system is called a normal probability plot. If the drawn from a normal distribution the points should fall close to a line with slope σ and intercept µ. SOCRCharts WebStat QQ-Normal Plot Slide 7 Beyond Normality Consider a family of probability distributions involving two parameters θ Let denote the 1 and θ. F( x; θ1, θ) corresponding cdf s. The parameters θ1 and θ are said to location and scale parameters if x θ F( x; θ 1 1, θ) is a function of. θ Lognormal (Y) µ,σ Relation among Distributions Normal (X) µ,σ X = lny X Y = e U = X β Beta, β = β = 1 Uniform(X), β χ Uniform(U),1 µ Z = X σ = n i = 1 Z i X = ( β ) U + Normal (Z),1 Chi-square ( χ ) n = n /, β = Gamma, β n= =1 X = β lnu df Weibull γ, β γ = 1 Exponential(X) β T df=n (,1) df 1 Cauchy (,1) Slide 71 Slide 7 1