4. Distributions of Functions of Random Variables

Transcription

1 4. Distributions of Functions of Random Variables Setup: Consider as given the joint distribution of X 1,..., X n (i.e. consider as given f X1,...,X n and F X1,...,X n ) Consider k functions g 1 : R n R,..., g k : R n R Find the joint distribution of the k random variables Y 1 = g 1 (X 1,..., X n ),..., Y k = g k (X 1,... X n ) (i.e. find f Y1,...,Y k and F Y1,...,Y k ) 155

2 Example: Consider as given X 1,..., X n with f X1,...,X n Consider the functions g 1 (X 1,..., X n ) = X i and g 2 (X 1,..., X n ) = 1 n X i Find f Y1,Y 2 with Y 1 = n X i and Y 2 = 1 n n X i Remark: From the joint distribution f Y1,...,Y k we can derive the k marginal distributions f Y1,... f Yk (cf. Chapter 3, Slides 106, 107) 156

3 Aim of this chapter: Techniques for finding the (marginal) distribution(s) of (Y 1,..., Y k ) 157

4 4.1 Expectations of Functions of Random Variables Simplification: In a first step, we are not interested in the exact distributions, but merely in certain expected values of Y 1,..., Y k Expectation two ways: Consider as given the (continuous) random variables X 1,..., X n and the function g : R n R Consider the random variables Y = g(x 1,..., X n ) and find the expectation E[g(X 1,..., X n )] 158

5 Two ways of calculating E(Y ): or E(Y ) = E(Y ) = + y f Y (y) dy g(x 1,..., x n ) f X1,...,X n (x 1,... x n ) dx 1... dx n (cf. Definition 3.9, Slide 128) It can be proved that Both ways of calculating E(Y ) are equivalent choose the most convenient calculation 159

6 Now: Calculation rules for expected values, variances, covariances of sums of random variables Setting: X 1,..., X n are given continuous or discrete random variables with joint density f X1,...,X n The (transforming) function g : R n R is given by g(x 1,..., x n ) = x i 160

7 In a first step, find the expectation and the variance of Y = g(x 1,..., X n ) = X i Theorem 4.1: (Expectation and variance of a sum) For the given random variables X 1,..., X n we have and Var X i = E X i = Var(X i ) + 2 E(X i ) j=i+1 Cov(X i, X j ). 161

8 Implications: For given constants a 1,..., a n R we have E (why?) a i X i = a i E(X i ) For two random variables X 1 and X 2 we have E(X 1 ± X 2 ) = E(X 1 ) ± E(X 2 ) If X 1,..., X n are stochastically independent, it follows that Cov(X i, X j ) = 0 for all i = j and hence Var X i = Var(X i ) 162

9 Now: Calculating the covariance of two sums of random variables Theorem 4.2: (Covariance of two sums) Let X 1,..., X n and Y 1,..., Y m be two sets of random variables and let a 1,... a n and b 1,..., b m be two sets of constants. Then Cov a i X i, m j=1 b j Y j = m j=1 a i b j Cov(X i, Y j ). 163

10 Implications: Var The variance of a weighted sum of random variables is given by a i X i = Cov a i X i, j=1 a j X j = j=1 a i a j Cov(X i, X j ) = a 2 i Var(X i) + j=1,j =i a i a j Cov(X i, X j ) = a 2 i Var(X i) + 2 j=i+1 a i a j Cov(X i, X j ) 164

11 For two random variables X 1 and X 2 we have Var(X 1 ± X 2 ) = Var(X 1 ) + Var(X 2 ) ± 2 Cov(X 1, X 2 ), and if X 1 and X 2 are independent we have Var(X 1 ± X 2 ) = Var(X 1 ) + Var(X 2 ) Finally: Important result concerning the expectation of a product of two random variables 165

12 Setting: Let X 1, X 2 be both continuous or both discrete random variables with joint density f X1,X 2 Let g : R n R be defined as g(x 1, x 2 ) = x 1 x 2 Find the expectation of Y = g(x 1, X 2 ) = X 1 X 2 Theorem 4.3: (Expectation of a product) For the random variables X 1, X 2 we have E (X 1 X 2 ) = E(X 1 ) E(X 2 ) + Cov(X 1, X 2 ). 166

13 Implication: If X 1 and X 2 are stochastically independent, we have E (X 1 X 2 ) = E(X 1 ) E(X 2 ) Remarks: A formula for Var(X 1 X 2 ) also exists In many cases, there are no explicit formulas for expected values and variances of other transformations (e.g. for ratios of random variables) 167

14 4.2 The Cumulative-distribution-function Technique Motivation: Consider as given the random variables X 1,..., X n with joint density f X1,...,X n Find the joint distribution of Y 1,..., Y k where Y j = g j (X 1,..., X n ) for j = 1,..., k The joint cdf of Y 1,..., Y k is defined to be F Y1,...,Y k (y 1,..., y k ) = P (Y 1 y 1,..., Y k y k ) (cf. Definition 3.2, Slide 98) 168

15 Now, for each y 1,..., y k the event {Y 1 y 1,..., Y k y k } = {g 1 (X 1,..., X n ) y 1,..., g k (X 1,..., X n ) y k }, i.e. the latter event is an event described in terms of the given functions g 1,..., g k and the given random variables X 1,..., X n since the joint distribution of X 1,..., X n is assumed given, presumably the probability of the latter event can be calculated and consequently F Y1,...,Y k determined 169

16 Example 1: Consider n = 1 (i.e. consider X 1 X with cdf F X ) and k = 1 (i.e. g 1 g and Y 1 Y ) Consider the function Find the distribution of g(x) = a x + b, b R, a > 0 Y = g(x) = a X + b 170

17 The cdf of Y is given by F Y (y) = P (Y y) = P [g(x) y] = P (a X + b y) = P ( X y b a = F X ( y b a If X is continuous, the pdf of Y is given by ( ) f Y (y) = F Y (y) = F X y b = 1 a a f X (cf. Slide 48) ) ) ( y b a ) 171

18 Example 2: Consider n = 1 and k = 1 and the function g(x) = e x The cdf of Y = g(x) = e X is given by F Y (y) = P (Y y) = P (e X y) = P [X ln(y)] = F X [ln(y)] If X is continuous, the pdf of Y is given by f Y (y) = F Y (y) = F X [ln(y)] = f X [ln(y)] y 172

19 Now: Consider n = 2 and k = 2, i.e. consider X 1 and X 2 with joint density f X1,X 2 (x 1, x 2 ) Consider the functions g 1 (x 1, x 2 ) = x 1 + x 2 and g 2 (x 1, x 2 ) = x 1 x 2 Find the distributions of the sum and the difference of two random variables Derivation via the two-dimensional cdf-technique 173

20 Theorem 4.4: (Distribution of a sum / difference) Let X 1 and X 2 be two continuous random variables with joint pdf f X1,X 2 (x 1, x 2 ). Then the pdfs of Y 1 = X 1 + X 2 and Y 2 = X 1 X 2 are given by f Y1 (y 1 ) = + f X 1,X 2 (x 1, y 1 x 1 ) dx 1 and f Y2 (y 2 ) = = = + + f X 1,X 2 (y 1 x 2, x 2 ) dx 2 f X 1,X 2 (x 1, x 1 y 2 ) dx 1 + f X 1,X 2 (y 2 + x 2, x 2 ) dx

21 Implication: If X 1 and X 2 are independent, then f Y1 (y 1 ) = + f X 1 (x 1 ) f X2 (y 1 x 1 ) dx 1 f Y2 (y 2 ) = + f X 1 (x 1 ) f X2 (x 1 y 2 ) dx 1 Example: Let X 1 and X 2 be independent random variables both with pdf { 1, for x [0, 1] f X1 (x) = f X2 (x) = 0, elsewise Find the pdf of Y = X 1 + X 2 (Class) 175

22 Now: Analogous results for the product and the ratio of two random variables Theorem 4.5: (Distribution of a product / ratio) Let X 1 and X 2 be continuous random variables with joint pdf f X1,X 2 (x 1, x 2 ). Then the pdfs of Y 1 = X 1 X 2 and Y 2 = X 1 /X 2 are given by and f Y1 (y 1 ) = f Y2 (y 2 ) = x 1 f X 1,X 2 (x 1, y 1 x 1 ) dx 1 x 2 f X1,X 2 (y 2 x 2, x 2 ) dx

23 4.3 The Moment-generating-function Technique Motivation: Consider as given the random variables X 1,..., X n with joint pdf f X1,...,X n Again, find the joint distribution of Y 1,..., Y k where Y j = g j (X 1,..., X n ) for j = 1,..., k 177

24 According to Definition 3.14, Slide 143, the joint moment generating function of the Y 1,..., Y k is defined to be m Y1,...,Y k (t 1,..., t k ) = E [ e t 1 Y t k Y k ] = et 1 g 1 (x 1,...,x n )+...+t k g k (x 1,...,x n ) f X1,...,X n (x 1,..., x n ) dx 1... dx n If m Y1,...,Y k (t 1,..., t k ) can be recognized as the joint moment generating function of some known joint distribution, it will follow that Y 1,..., Y k has that joint distribution by virtue of the identification property (cf. Slide 145) 178

25 Example: Consider n = 1 and k = 1 where the random variable X 1 X has a standard normal distribution Consider the function g 1 (x) g(x) = x 2 Find the distribution of Y = g(x) = X 2 The moment generating function of Y is given by m Y (t) = E [ e t Y ] = E [e t X2] = + et x2 f X (x)dx 179

26 = + et x2 1 2π e 1 2 x2 dx =... = t 1 2 for t < 1 2 This is the moment generating function of a gamma distribution with parameters λ = 1 2 and r = 1 2 (see Mood, Graybill, Boes (1974), pp. 540/541) Y = X 2 Γ(0.5, 0.5) 180

27 Now: Distribution of sums of independent random variables Preliminaries: Consider the moment generating function of such a sum Let X 1,..., X n be independent random variables and let Y = n X i The moment generating function of Y is given by m Y (t) = E [ e t Y ] = E [ e t n X i ] = E [ e t X 1 e t X 2... e t X n ] = E [ e t X ] [ ] [ ] 1 E e t X 2... E e t X n = m X1 (t) m X2 (t)... m Xn (t) [Theorem 3.13(c)] 181

28 Theorem 4.6: (Moment generating function of a sum) Let X 1,..., X n be stochastically independent random variables with existing moment generating functions m X1 (t),..., m Xn (t) for all t ( h, h), h > 0. Then the moment generating function of the sum Y = n X i is given by m Y (t) = n m Xi (t) for t ( h, h). Hopefully: The distribution of the sum Y = n X i may be identified from the moment generating function of the sum m Y (t) 182

29 Example 1: Assume that X 1,..., X n are independent and identically distributed exponential random variables with parameter λ > 0 The moment generating function of each X i (i = 1,..., n) is given by m Xi (t) = λ λ t for t < λ (cf. Mood, Graybill, Boes (1974), pp. 540/541) So the moment generating function of the sum Y = n X i is given by m Y (t) = m X i (t) = n m Xi (t) = ( λ λ t ) n 183

30 This is the moment generating function of a Γ(n, λ) distribution (cf. Mood, Graybill, Boes (1974), pp. 540/541) the sum of n independent, identically distributed exponential random variables with parameter λ has a Γ(n, λ) distribution 184

31 Example 2: Assume that X 1,..., X n are independent random variables and that X i N(µ i, σ 2 i ) Furthermore, let a 1,..., a n R be constants Then the distribution of the weighted sum is given by Y = (Proof: Class) a i X i N a i µ i, a 2 i σ2 i 185

32 4.4 General Transformations Up to now: Techniques that allow us, under special circumstances, to find the distributions of the transformed variables Y 1 = g 1 (X 1,..., X n ),..., Y k = g k (X 1,..., X n ) However: These methods do not necessarily hit the mark (e.g. if calculations get too complicated) 186

33 Resort: There are constructive methods by which it is generally possible (under rather mild conditions) to find the distributions of transformed random variables transformation theorems Here: We restrict attention to the simplest case where n = 1, k = 1, i.e. we consider the transformation Y = g(x) For multivariate extensions (i.e. for n 1, k 1) see Mood, Graybill, Boes (1974), pp

34 Theorem 4.7: (Transformation theorem for densities) Suppose X is a continuous random variable with pdf f X (x). Set D = {x : f X (x) > 0}. Furthermore, assume that (a) the transformation g : D W with y = g(x) is a one-to-one transformation of D onto W, (b) the derivative with respect to y of the inverse function g 1 : W D with x = g 1 (y) is continuous and nonzero for all y W. Then Y = g(x) is a continuous random variable with pdf f Y (y) = dg 1 (y) dy f X ( g 1 (y) ), for y W 0, elsewise. 188

35 Remark: The transformation g : D W with y = g(x) is called oneto-one, if for every y W there exists exactly one x D with y = g(x) Example: Suppose X has the pdf { θ x θ 1, for x [1, + ) f X (x) = 0, elsewise (Pareto distribution with parameter θ > 0) Find the distribution of Y = ln(x) We have D = [1, + ), g(x) = ln(x), W = [0, + ) 189

36 Furthermore, g(x) = ln(x) is a one-to-one transformation of D = [1, + ) onto W = [0, + ) with inverse function x = g 1 (y) = e y Its derivative with respect to y is given by dg 1 (y) dy = e y, i.e. the derivative is continuous and nonzero for all y [0, + ) Hence, the pdf of Y = ln(x) is given by f Y (y) = { ey θ (e y ) θ 1, for y [0, + ) 0, elsewise = { θ e θ y, for y [0, + ) 0, elsewise 190