BMIR Lecture Series on Probability and Statistics Fall 2015 Discrete RVs

Transcription

1 Lecture #7 BMIR Lecture Series on Probability and Statistics Fall 2015 Department of Biomedical Engineering and Environmental Sciences National Tsing Hua University 7.1

2 Function of Single Variable Theorem Suppose that X is a discrete random variable with probability distribution f X (x). Let Y = h(x) define a one-to-one transformation between the values of X and Y so that the equation y = h(x) can be solved uniquely for x in terms of y. Let the solution be x = u(y). Then the probability distribution of the random variable y is P(Y = y) = f Y (y) = f X (u(y)) = P(X = u(y)) (1) 7.2

3 Function of Single Variable: Example Example Let X be a geometric random variable with probability distribution f X (x) = p(1 p) x 1, x = 1, 2,... Find the probability distribution of Y = X 2. Since X > 0, Y = X 2 (or X = Y) is one-to-one transformation. Therefore, the distribution of the random variable Y is f Y (y) = f X ( y) = p(1 p) y 1, y = 1, 4, 9,

4 Function of Two Theorem Suppose that X 1 and X 2 are two discrete random variables with joint probability distribution f X1 X 2 (x 1, x 2 ) (or f (x 1, x 2 )). Let Y 1 = h 1 (X 1, X 2 ) and Y 2 = h 2 (X 1, X 2 ) define one-to-one transformations between the points (x 1, x 2 ) and (y 1, y 2 ) so that equations y 1 = h 1 (x 1, x 2 ) and y 2 = h 2 (x 1, x 2 ) can be solved uniquely for x 1 and x 2 in terms of y 1 and y 2, i.e., x 1 = u 1 (y 1, y 2 ) and x 2 = u 2 (y 1, y 2 ). Then the joint probability distribution of Y 1 and Y 2 is f Y1,Y 2 (y 1, y 2 ) = f X1 X 2 [u 1 (y 1, y 2 ), u 2 (y 1, y 2 )] (2) Note that the distribution of Y 1 can be found by summing over the y 2, i.e., the marginal probability distribution of Y 1, f Y1 (y 1 ) = y 2 f Y1,Y 2 (y 1, y 2 ) 7.4

5 MGF of Sum Two Independent Poisson RVs I Example Suppose that X 1 and X 2 are two independent Poisson random variables with parameters λ 1 and λ 2. Find the probability distribution of Y = X 1 + X 2. Let Y = Y 1 = X 1 + X 2 and Y 2 = X 2. Hence, X 1 = Y 1 Y 2 and X 2 = Y 2. Since X 1 and X 2 are independent, the joint distribution of X 1 and X 2 is f X1 X 2 (x 1, x 2 ) = λx 1 1 e λ 1 λx 2 2 e λ 2 x 1! x 2! x 1 = 0, 1, 2,..., x 2 = 0, 1, 2,

6 MGF of Sum Two Independent Poisson RVs II The joint distribution of Y 1 and Y 2 is f Y1,Y 2 (y 1, y 2 ) = f X1 X 2 [u 1 (y 1, y 2 ), u 2 (y 1, y 2 )] = f X1 X 2 (y 1 y 2, y 2 ) = λy 1 y 2 1 e λ 1 (y 1 y 2 )! λy2 y 2! 2 e λ 2 = e (λ 1+λ 2 ) λ y 1 y 2 1 λ y 2 2 (y 1 y 2 )!y 2! y 1 = 0, 1, 2,..., y 2 = 0, 1, 2,..., y 1 Note that x 1 0 so that y 1 y

7 MGF of Sum Two Independent Poisson RVs III The marginal distribution of Y 1 is f Y1 (y 1 ) = y 1 y 2 =0 = e (λ 1+λ 2 ) = e (λ 1+λ 2 ) y 1! f Y1,Y 2 (y 1, y 2 ) y 1 y 2 =0 y 1 y 2 =0 λ y 1 y 2 1 λ y 2 2 (y 1 y 2 )!y 2! = e (λ 1+λ 2 ) (λ 1 + λ 2 ) y 1 y 1! y 1! (y 1 y 2 )!y 2! λy 1 y 2 1 λ y 2 2 The random variable Y = Y 1 = X 1 + X 2 has Poisson distribution with parameter λ 1 + λ

8 Function of Single Variable I Theorem Suppose that X is a continuous random variable with probability distribution f X (x). Let Y = h(x) define a one-to-one transformation between the values of X and Y so that the equation y = h(x) can be solved uniquely for x in terms of y. Let the solution be x = u(y). Then the probability distribution of the random variable y is f Y (y) = f X (u(y)) J (3) where J = d dy u(y) = u (y) = dx dy is called the Jacobian of the transformation and the absolute value J of J is used. Suppose that the function y = h(x) is an increasing function of x. 7.8

9 Function of Single Variable II now, we have P(Y a) = P[X u(a)] = u(a) f X (x)dx Since x = u(y), we obtain dx = u (y)dy and P(Y a) = a f X (u(y))u (y)dy The density function of Y is f Y (y) = f X (u(y))u (y) = f X (u(y))j. If y = h(x) is a decreasing function of x, a similar argument holds. 7.9

10 Function of Single Variable: Example 1 I Example Suppose that X has pdf f (x) = { 2x, 0 < x < 1 0, elsewhere Let h(x) = 3X + 1. How to find the pdf of Y = h(x). (4) Since y = h(x) = 3x + 1 is an increasing function of x, we obtain x = u(y) = (y 1)/3, J = u (y) = 1/3, f Y (y) = f X (u(y))u (y) = 2(y 1)/3 1/3 = 2 (y 1)

11 Function of Single Variable: Example 1 II Intuitive approach F Y (y) = P(Y y) = P[(3X + 1) y] = P[X (y 1)/3] = (y 1)/3 f Y (y) = d dy F Y(y) 0 2xdx = [(y 1)/3] 2. = { 2 9 (y 1), 1 < y < 4 0, elsewhere 7.11

12 Function of Single Variable: Example 2 I Example Suppose that X has pdf f (x) = { 2x, 0 < x < 1 0, elsewhere (5) Let h(x) = e X. How to find the pdf of Y = h(x). Since y = h(x) = e x is a decreasing function of x, we obtain x = u(y) = ln y, J = u (y) = 1/y, f Y (y) = f X (u(y)) J = 2 ln y 1/y = 2 ln y, y 1/e < y <

13 Function of Single Variable: Example 2 II Intuitive approach F Y (y) = P(Y y) = P(e X y) = P(X ln y) = 1 ln y 2xdx = 1 ( ln y) 2. f Y (y) = d dy F Y(y) = 2 ln y, y 1/e < y <

14 Function of Single Variable: Non-monotonic Transformation I Example Suppose that X has pdf f (x). Let h(x) = X 2. How to find the pdf of Y = h(x). Since y = h(x) = x 2 is neither an increasing nor a decreasing function of x, we can not directly use the Eq. (3). We can obtain the pdf of Y = X 2 as follows: F Y (y) = P(Y y) = P(X 2 y) = P( y X y) = F X ( y) F X ( y) 7.14

15 Function of Single Variable: Non-monotonic Transformation II f Y (y) = d dy F Y(y) = d dy [F X( y) F X ( y)] = dx d dy dx [F X( y) F X ( y)] = f X( y) 2 y f X( y) 2 y 1 = 2 y [f X( y) + f X ( y)] 7.15

16 Non-monotonic Transformation: Normal vs Chi-Square I Example Suppose that X has pdf N(0, 1) Let h(x) = X 2. How to find the pdf of Y = h(x). X is standard normal distribution with pdf f (x) = 1 2π e x2 /2 We can obtain the pdf of Y = X 2 as follows: f Y (y) = 1 2 y [f X( y) + f X ( y)] = 1 y 1 2π e y/2 = 1 2 1/2 π y1/2 1 e y/2, y >

17 Non-monotonic Transformation: Normal vs Chi-Square II The gamma distribution has the pdf f (x; γ, λ) = λγ x γ 1 e λx Γ(γ) = λ Γ(γ) (λx)γ 1 e λx, x > 0. When λ = 1/2 and γ = 1/2, 1, 3/2, 2,..., it becomes chi-square distribution: f (x; γ, 1/2) = 1 ( ) 1 γ x γ 1 e x/2, x > 0. Γ(γ) 2 Since π = Γ( 1 2 ), we can rewrite f Y(y) as f Y (y) = 1 Γ( 1 2 ) Y has a chi-square distribution. ( ) 1 1/2 y 1/2 1 e y/

18 General Transformation Theorem Suppose that X is a continuous random variable with probability distribution f X (x), and Y = h(x) is a transformation that is not one-to-one. If the interval over which X is defined can be partitioned into m mutually exclusive disjoint sets such that each of the inverse functions x 1 = u 1 (y), x 2 = u 2 (y),..., x m = u m (y) of y = h(x) is one-to-one, the probability distribution of Y is f Y (y) = m f X [u i (y)] J i (6) i=0 7.18

19 Function of Two Theorem Suppose that X 1 and X 2 are two continuous random variables with joint probability distribution f X1 X 2 (x 1, x 2 ) (or f (x 1, x 2 )). Let Y 1 = h 1 (X 1, X 2 ) and Y 2 = h 2 (X 1, X 2 ) define one-to-one transformations between the points (x 1, x 2 ) and (y 1, y 2 ) so that equations y 1 = h 1 (x 1, x 2 ) and y 2 = h 2 (x 1, x 2 ) can be solved uniquely for x 1 and x 2 in terms of y 1 and y 2, i.e., x 1 = u 1 (y 1, y 2 ) and x 2 = u 2 (y 1, y 2 ). Then the joint probability distribution of Y 1 and Y 2 is f Y1,Y 2 (y 1, y 2 ) = f X1 X 2 [u 1 (y 1, y 2 ), u 2 (y 1, y 2 )] J (7) where J is the Jacobian and is given by the following determinant: J = x 1/ y 1 x 1 / y 2 x 2 / y 1 x 2 / y 2 (8) and the absolute value of the determinant is used. 7.19

20 Function of Two I Example Suppose that X 1 and X 2 are two independent exponential random variables with f X1 (x 1 ) = 2e 2x 1 and f X2 (x 2 ) = 2e 2x 2. Find the probability distribution of Y = X 1 /X 2. The joint probability distribution of X 1 and X 2 is f X1 X 2 (x 1, x 2 ) = 4e 2(x 1+x 2 ) Let Y = Y 1 = h 1 (X 1, X 2 ) = X 1 /X 2 and Y 2 = h 2 (X 1, X 2 ) = X 1 + X

21 Function of Two II The inverse solutions of y 1 = x 1 /x 2 and y 2 = x 1 + x 2 are x 1 = y 1 y 2 /(1 + y 1 ) and x 2 = y 2 /(1 + y 1 ), and it follows that y 2 y 1 (1+y J = 1 ) 2 (1+y 1 ) y 2 = y 2 (1 + y 1 ) 2 1 (1+y 1 ) 2 (1+y 1 ) Then the joint probability distribution of Y 1 and Y 2 is f Y1,Y 2 (y 1, y 2 ) = f X1 X 2 [u 1 (y 1, y 2 ), u 2 (y 1, y 2 )] J = 4e 2(y 1y 2 /(1+y 1 )+y 2 /(1+y 1 )) y 2 (1 + y 1 ) 2 = 4e 2y 2 y 2 (1 + y 1 ) 2 y 1 > 0, y 2 >

22 Function of Two III We need to find the distribution of Y = Y 1 = X 1 /X 2. The marginal distribution of Y 1 is f Y1 (y 1 ) = = 0 0 f Y1,Y 2 (y 1, y 2 )dy 2 4e 2y y 2 2 (1 + y 1 ) 2 dy 2 1 = (1 + y 1 ) 2, y > 0 = f Y (y) 7.22

23 Example: Function of Two N(0, 1) I Example Let X 1 and X 2 be two independent normal random variables and both have a N(0, 1) distribution. Let Y 1 = X 1+X 2 2 and Y 2 = X 2 X 1 2. Show that Y 1 and Y 2 are independent. Since X 1 and X 2 are independent, the joint pdf is f X1,X 2 (x 1, x 2 ) = f X1 (x 1 ) f X2 (x 2 ) = 1 1 exp 2 (x1 2+x2 2). 2π X 1 = u 1 (Y 1, Y 2 ) = Y 1 Y 2 and X 2 = u 2 (Y 1, Y 2 ) = Y 1 + Y 2. The Jacobian J = x 1/ y 1 x 1 / y 2 x 2 / y 1 x 2 / y 2 = =

24 Example: Function of Two N(0, 1) II The joint pdf of Y 1 and Y 2 is f Y1,Y 2 (y 1, y 2 ) = f X1 X 2 [u 1 (y 1, y 2 ), u 2 (y 1, y 2 )] J = 2 1 exp{ 2 [(y 1 y 2 ) 2 +(y 1 +y 2 ) 2 ]} 2π = = 1 ( 2π ) 2 ( ) 2 exp ( ) exp 2π ( ) exp 2π ( y2 1 ) 2 + y 2 2 ( ) ( y2 1 ) ( y2 2 )

25 Example: Function of Two N(0, 1) III Therefore, f Y1,Y 2 (y 1, y 2 ) = f Y1 (y 1 ) f Y2 (y 2 ). Y 1 and Y 2 both are normal distribution with N(0, 1 2 ). 7.25

26 Example: Function of Multiple I Example Let X 1, X 2,..., X n be independent and identically distributed random variables from a uniform distribution on [0, 1]. Let Y = max{x 1, X 2,..., X n } and Z = min{x 1, X 2,..., X n }. What are the pdf s of Y and Z? The pdf of Y: F Y (y) = P(Y y) = P(max{X 1, X 2,..., X n } y) = P(X 1 y,..., X n y) = P(X 1 y) P(X n y) = y n. 7.26

27 Example: Function of Multiple II Hence f y (y) = df Y(y) dy = ny n 1, 0 y 1. F Z (z) = P(Z z) = P(min{X 1, X 2,..., X n } z) = 1 P(X 1 > z,..., X n > z) = 1 P(X 1 > z) P(X n > z) = 1 (1 z) n. Hence f Z (z) = df Z(z) dz = n(1 z) n 1, 0 z

28 Function (mgf) Definition (Discrete Random Variable) Let X be a discrete random variable with probability mass function P(X = x i ) = f (x i ), i = 1, 2,.... The function M X is called the moment generating function of X is defined by M X (t) = E(e tx ) = e tx i f (x i ) (9) i=1 Definition (Continuous Random Variable) If X be a continuous random variable with probability density function f (x), we define the moment generating function of X by M X (t) = E(e tx ) = e tx f (x)dx (10) 7.28

29 MGF of Poisson Distribution Example If X is a Poisson random variable, f (x) = e λ λ x x!, with parameter λ > 0, find its moment generating function. Solution M X (t) = x=0 e tx e λ λ x x! = e λ (λe t ) x x! x=0 = e λ e λet = e λ(et 1) (11) 7.29

30 MGF of Exponential Distribution Example If X is an exponential random variable, f (x) = λe λx, with parameter λ > 0, find its moment generating function. Solution M X (t) = = λ = 0 0 e tx λe λx dx e x(λ t) dx λ λ t, t < λ (12) 7.30

31 MGF of Normal Distribution(I) Example If X N(0, 1), find its moment generating function. Solution M X (t) = E{e tx } = 1 2π = 1 e t2 /2 2π = e t2 /2 = e t2 /2 [ 1 2π e tx e x2 /2 dx e t2 /2 e tx e x2 /2 dx ] e (x t)2 /2 dx (13) 7.31

32 MGF of Gamma Distribution I Example If X is a Gamma distribution with pdf f (x; γ, λ) = { λ γ Γ(γ) xγ 1 e λx, 0 < x < 0, elsewhere, find its moment generating function. M X (t) = E{e tx } = = 0 e tx 0 λγ λ γ Γ(γ) xγ 1 e (λ t)x dx Γ(γ) xγ 1 e λx dx 7.32

33 MGF of Gamma Distribution II Let y = (λ t)x, then x = y/(λ t) and dx = dy/(λ t). We have M X (t) = = = = = 0 λ γ Γ(γ) xγ 1 e (λ t)x dx ( y λ γ 0 Γ(γ) (λ t) ( ) λ γ λ t ( ) λ γ λ t 1 (1 t/λ) γ 0 ) γ 1 e y 1 (λ t) dy 1 Γ(γ) yγ 1 e y dy 7.33

34 Properties of MGF Recall the Taylor series of the function e x : Thus e x = 1 + x + x2 2! + + xn n! + e tx = 1 + tx + (tx)2 2! + + (tx)n n! + The moment generating of the random variable X is M X (t) = E(e tx ) = E (1 + tx + (tx)2 2! = 1 + te(x) + t2 E(X 2 ) 2! + + (tx)n n! ) tn E(X n ) n!

35 Properties of MGF Consider the first derivative of M X (t) with respect to t and obtain M X(t) = E(X) + te(x 2 ) + + nt n 1 E(Xn ) n! Taking t = 0, we have M X(t) t=0 = M X(0) = E{X} = µ X + Consider the second derivative of M X (t) and obtain M X(t) = E(X 2 ) + te(x 3 ) + n(n 1)t n 2 E(Xn ) + n! The second moment of X is M X (0) = E{X2 }. Then, the variance of X is given by V(X) = E{X 2 } E 2 {X} = M X(0) (M X(0))

36 Properties of MGF Differentiating k times and then setting t = 0 yields M (k) X (t) t=0 = E{X k } (14) If X is a random variable and α is a constant, then M X+α (t) = e αt M X (t) (15) M αx (t) = M X (αt) (16) Let X and Y be two random variables with moment generating functions M X (t) and M Y (t), respectively. If M X (t) = M Y (t) for all values of t, then X and Y have the same probability distribution. 7.36

37 MGF of Normal Distribution(II) Example Use Eq. (13) to find the first four moments of the normal rv X. Solution M X (t) = d dt et2 /2 = e t2 /2 t = t M X (t). E{X 1 } = M X (0) = 0. M X (t) = M X(t) + t M X (t) = M X(t) + t 2 M X (t). E{X 2 } = M X (0) = 1. M (3) X (t) = (2t) M X(t)+(1+t 2 ) M X (t) = (3t+t3 ) M X (t). E{X 3 } = M (3) X (0) = 0. M (4) X (t) = (3 + 3t2 ) M X (t) + (3t + t 3 ) M X (t). E{X 4 } = M (4) X (0) =

38 MGF of Normal Distribution(III) Example Use Eqs. (14) and (15) to find the mgf of a normal distribution with mean µ and variance σ 2. Assume X be a normal distribution with mean µ and variance σ 2. If Z = X µ σ, Z is a standard normal random variable. The mgf of X is M X (t) = M σz+µ (t) = e µt M σz (t) = e µt M Z (σt) = e µt e (σt)2 /2 = e µt+σ2 t 2 /2 (17) 7.38

39 Sum of Independent RVs: MGF I Theorem If X 1, X 2,..., X n are independent random variables with moment generating functions M X1 (t), M X2 (t),..., M Xn (t), respectively, and if Y = X 1 + X X n then the moment generating function of Y is M Y (t) = M X1 (t) M X2 (t) M Xn (t) (18) For the continuous case, M Y (t) = E(e ty ) = E(e t(x 1+X 2 + +X n) ) = dx 1 dx 2 dx n e t(x 1+x 2 + +x n) f (x 1, x 2,..., x n ) 7.39

40 Sum of Independent RVs: MGF II Since the variables are independent, we have f (x 1, x 2,..., x n ) = f (x 1 )f (x 2 ) f (x n ) Therefore, M Y (t) = e tx 1 f (x 1 )dx 1 e txn f (x n )dx n = M X1 (t) M X2 (t) M Xn (t) e tx 2 f (x 2 )dx

41 MGF of Sum Two Independent Poisson RVs Example Suppose that X 1 and X 2 are two independent Poisson random variables with parameters λ 1 and λ 2. Find the probability distribution of Y = X 1 + X 2. The mgf s of X 1 and X 2 are M X1 (t) = e λ 1(e t 1) and M X2 (t) = e λ 2(e t 1), respectively. The mgf of Y is M Y (t) = M X1 (t) M X2 (t) = e λ 1(e t 1) e λ 2(e t 1) = e (λ 1+λ 2 )(e t 1) Therefore, Y is also a Poisson random variable with parameters λ 1 + λ 2. (19) 7.41