Lecture 4. Continuous Random Variables and Transformations of Random Variables

Math 408 - Mathematical Statistics Lecture 4. Continuous Random Variables and Transformations of Random Variables January 25, 2013 Konstantin Zuev (USC) Math 408, Lecture 4 January 25, 2013 1 / 13

Agenda Definition Important Examples Uniform Distribution Normal (Gaussian) Distribution Exponential Distribution Gamma Distribution Beta Distribution Transformation of Random Variables Discrete Case Continuous Case Summary Konstantin Zuev (USC) Math 408, Lecture 4 January 25, 2013 2 / 13

Definition Recall that a random variable is a (deterministic) map X : Ω R that assigns a real number X (ω) to each (random) realization ω Ω. Definition A random variable is continuous if there exists a function f X such that f X (x) 0 for all x + f X (x)dx = 1, and For every a b P(a < X b) = b a f X (x)dx The function f X (x) is called the probability density function (PDF) Relationship between the CDF F X (x) and PDF f X (x): F X (x) = x f X (t)dt f X (x) = F X (x) Konstantin Zuev (USC) Math 408, Lecture 4 January 25, 2013 3 / 13

Important Remarks If X is continuous then P(X = x) = 0 for every x. Don t think of f X (x) as P(X = x). This is only true for discrete random variables. For continuous random variables, we get probabilities by integrating. A PDF can be bigger than 1 (unlike PMF!). For example: { 10, x [0, 0.1] f X (x) = 0, x / [0, 0.1] Can a PDF be unbounded? Yes, of course! For instance f X (x) = { 2 3 x 1/3, 0 < x < 1 0, otherwise Konstantin Zuev (USC) Math 408, Lecture 4 January 25, 2013 4 / 13

Important Examples The Uniform Distribution X has a uniform distribution on [a, b], denoted X U[a, b], if { 1 f (x) = b a, x [a, b] 0, otherwise Normal (Gaussian) Distribution X has a Normal (or Gaussian) distribution with parameters µ and σ, denoted by X N (µ, σ 2 ), if f (x) = 1 ) (x µ)2 exp ( 2πσ 2σ 2, x R Many phenomena in nature have approximately Normal distribution. Distribution of a sum of random variables can be approximated by a Normal distribution (central limit theorem) Konstantin Zuev (USC) Math 408, Lecture 4 January 25, 2013 5 / 13

Important Examples Exponential Distribution X has an Exponential distribution with parameter β > 0, X Exp(β), if f (x) = 1 β e x/β, x > 0 1 0.9 0.8 β 1 =1 β 2 =3 β 3 =7 Exponential PDF f(x β) 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 2 4 6 8 10 12 x The exponential distribution is used to model the life times of electronic components and the waiting times between rare events. β is a survival parameter: the expected duration of survival of the system is β units of time. Konstantin Zuev (USC) Math 408, Lecture 4 January 25, 2013 6 / 13

Important Examples Gamma Distribution X has a Gamma distribution with parameters α > 0 and β > 0, X Gamma(α, β), if f (x) = 1 β α Γ(α) x α 1 e x/β, x > 0 Γ(α) is the Gamma function Γ(α) = 0 x α 1 e x dx The Gamma distribution is frequently used to model waiting times. Exponential distribution is a special case of the Gamma distribution: Gamma(1, β) = Exp(β) Konstantin Zuev (USC) Math 408, Lecture 4 January 25, 2013 7 / 13

Gamma Distribution 0.5 0.45 0.4 α=1, β=2 α=2, β=2 α=9, β=0.5 Gamma PDF f(x α,β) 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 0 1 2 3 4 5 6 7 8 9 10 x Konstantin Zuev (USC) Math 408, Lecture 4 January 25, 2013 8 / 13

Important Examples Beta Distribution X has a Beta distribution with parameters α > 0 and β > 0, X Beta(α, β), if f (x) = Γ(α + β) Γ(α)Γ(β) x α 1 (1 x) β 1, 0 < x < 1 The beta distribution is often used for modeling of proportions. The beta distribution has an important application in the theory of order statistics. A basic result is that the distribution of the k th largest X (k) of a sample of size n from a uniform distribution X 1,..., X n U(0, 1) has a beta distribution: X (k) Beta(k, n k + 1) Konstantin Zuev (USC) Math 408, Lecture 4 January 25, 2013 9 / 13

Beta Distribution 3 2.5 α=0.5, β=0.5 α=2, β=5 α=5, β=1 Beta PDF f(x α,β) 2 1.5 1 0.5 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 x Konstantin Zuev (USC) Math 408, Lecture 4 January 25, 2013 10 / 13

Transformation of Random Variables Suppose that X is a random variable with PDF/PMF (continuous/discrete) f X and CDF F X. Let Y = r(x ) be a function of X, for example, Y = X 2, Y = e X. Q: How to compute the PDF/PMF and CDF of Y? In the discrete case, the answer is easy: f Y (y) = P(Y = y) = P(r(X ) = y) = P({x : r(x) = y}) = x i :r(x i )=y f X (x i ) Example: X { 1, 0, 1} P(X = 1) = 1/4, P(X = 0) = 1/2, P(X = 1) = 1/4 Y = X 2 Find PMF f Y Answer: f Y (0) = 1/2 and f Y (1) = 1/2. Konstantin Zuev (USC) Math 408, Lecture 4 January 25, 2013 11 / 13

Transformation of Random Variables: Continuous Case The continuous case is harder. These are the steps for finding the PDF f Y : 1 For each y, let A y = {x : r(x) y} 2 Find the CDF F Y (y) F Y (y) = P(Y y) = P(r(X ) y) = P(X A y ) = f X (x)dx A y 3 The PDF is then f Y (y) = F Y (y) Example: Let X Exp(1), and Y = ln X. Find f Y (y). Answer: f Y (y) = e y e ey, y R Important Fact: When r is strictly monotonic, then r has an inverse s = r 1 and f Y (y) = f X (s(y)) ds(y) dy Konstantin Zuev (USC) Math 408, Lecture 4 January 25, 2013 12 / 13

Summary A random variable is continuous if there exists a function f X, called probability density function such that fx (x) 0 for all x + f X (x)dx = 1 For every a b P(a < X b) = b a f X (x)dx Relationship between the CDF F X (x) and PDF f X (x): F X (x) = x f X (t)dt f X (x) = F X (x) Important Examples: Uniform Distribution, Normal Distribution, Exponential Distribution, Gamma Distribution, Beta Distribution If Y = r(x ) and r is strictly monotonic, then f Y (y) = f X (s(y)) ds(y) dy s = r 1 Konstantin Zuev (USC) Math 408, Lecture 4 January 25, 2013 13 / 13