4 Pairs of Random Variables

Transcription

1 B.Sc./Cert./M.Sc. Qualif. - Statistical Theory 4 Pairs of Random Variables 4.1 Introduction In this section, we consider a pair of r.v. s X, Y on (Ω, F, P), i.e. X, Y : Ω R. More precisely, we define a random vector (X, Y ) : Ω R 2 by setting (X, Y )(ω) (X(ω), Y (ω)) In general, there will be some dependence between X and Y ; this dependence can be encapsulated through an appropriately defined distribution function. It will be useful to define the concept of range for this bivariate situation. Definition (Range) The range of a random vector (X, Y ) is given by 4.2 Discrete case R (X,Y ) {(x, y) : X(ω) x, Y (ω) y for some ω Ω}. Definition (distribution function/p.m.f.(bivariate case)) Suppose X and Y are discrete r.v. s. (i) The joint distribution function F (X,Y ) : R 2 [, 1] of X and Y is given by F (X,Y ) (x, y) P(X x, Y y). (ii) The joint probability mass function p (X,Y ) : R 2 [, 1] is given by p (X,Y ) (x, y) P(X x, Y y). s with single r.v. s, we can say that p (X,Y ) is a p.m.f. if and only if (i) p (X,Y ) (x, y) for (x, y) R 2. (ii) x,y p (X,Y )(x, y) 1. Given the joint p.m.f., it is possible to calculate the marginal p.m.f. s, namely, p X and p Y. In fact, suppose that X takes values in a countable set R X {x 1, x 2,...}, and Y takes values in R Y {y 1, y 2,...}. 1

2 Then P( j p X (x i ) P(X x i ) P({ω Ω : X(ω) x i }) {ω Ω : X(ω) x i, Y (ω) y j }) j P(X x i, Y y j ) j p (X,Y ) (x i, y j ) where x i R X. To derive the above expression, we have used the fact that, for j {X x i, Y y j }, { j, j 1, 2,...} is a partition of the set {X x i }. The final equality follows from Definition (ii). Example Let the number of enquiries arriving into a call centre (in a specified period of time) and the number of these which are answered be represented by the random variables X and Y respectively. Suppose that the joint p.m.f. of X and Y is given by with <θ <1, λ>. Find the marginal distributions of X and Y. p (X,Y ) (m, n) e λ λ m θ n (1 θ) m n. (m n)!n! Solution: Note 1 that R (X,Y ) {(m, n) : m N, n N, n m} To find the p.m.f. of X, first note that the range of X is R X {, 1, 2,...} For a given m R X, p X (m) P(X m) m P(X m, Y n) n m e λ λ m θ n (1 θ) m n n λ λm e m! m λ λm e (m n)!n! m! n m )θ n (1 θ) m n e n ( m n m p (X,Y ) (m, n) n m! (m n)!n! θn (1 θ) m n λ λm m! 1 e λ λm m!. We have used the fact that the sum of the p.m.f. of the Bin(m, θ) distribution over its range is equal to 1. We recognize the above function, with the stated range, i.e. P X (m), to be the p.m.f. of the 1 Here N is defined to be the set of non-negative integers, and so includes. 2

3 P o(λ) distribution, i.e. X P o(λ). lso, R Y {, 1, 2,...} So for n R Y, p Y (n) P(Y n) m P(X m, Y n) m p (X,Y ) (m, n) e λ λ m θ n (1 θ) m n (m n)!n! mn (the restricted summation is valid due to the fact that X Y ). So p Y (n) is equal to λ (λθ)n e n! mn (λ(1 θ)) m n (m n)! λ (λθ)n e n! (λ(1 θ)) m m m! e λ λ(1 θ) (λθ)n e n! which is equal to i.e. Y P o(λθ). λθ (λθ)n e, n, 1, 2,... n! 4.3 Continuous case The definition of the joint distribution function is exactly the same as that for the discrete case. The continuous analogue of the p.m.f. is introduced here. Definition (Joint p.d.f.) X and Y are (jointly) continuous with joint p.d.f. f (X,Y ) : R 2 [, ) if for each x, y R. F (X,Y ) (x, y) y x f (X,Y ) (u, v)dudv In line with earlier parts of this discussion, a joint p.d.f., f (X,Y ) say, of r.v. s X and Y, can be characterized by the following conditions (i) f (X,Y ) (x, y) for (x, y) R 2 (ii) R 2 f (X,Y ) (x, y)dxdy 1. gain, given the joint p.d.f. of X and Y, we can calculate the marginal p.d.f. s. First note that However P(X ) f X (x)dx. (1) P(X ) P(X, < Y < ) 3

4 Combining (1) and (2) yields f X (x)dx Hence ( ) f (X,Y ) (x, y)dy dx (2) f X (x) In a similar way, it may be shown that f Y (y) ( ) f (X,Y ) (x, y)dy dx. f (X,Y ) (x, y)dy. f (X,Y ) (x, y)dx. Remarks It is easy to extend and generalize results in obvious ways for random vectors of dimension n > 2, for both the discrete and continuous cases; we will explore some of these results in a future lecture. Example Suppose that f (X,Y ) (x, y) { α(1 x y) x >, y >, x + y < 1 o.w. Determine the constant α. Hence, determine the marginal p.d.f. of X. Solution: In this case, the range of (X, Y ) is given by R (X,Y ) {(x, y) : x >, y >, x + y < 1} with α >. α can be determined by exploiting property (ii): 1 f (X,Y ) (x, y)dxdy α (1 x y)dxdy R (X,Y ) R (X,Y ) α 1 ( 1 x ) 1 (1 x y)dy dx α α 6. (1 x) 2 dx [ 2 ] 1 α(1 x)3 α 6 6 Indeed, if we quit after the integration w.r.t. y with the knowledge that α 6, we see that f X (x) 3(1 x) 2 x R X. 4

5 Example Suppose that f (X,Y ) (x, y) { x 2 + xy/3 <x<1, <y <2 o.w. Find the probability that the sum of X and Y is less than 1. Solution: 1 { 1 x y 1 P(X + Y < 1) } f (X,Y ) (x, y)dy dx [x 2 y + xy2 6 ] 1 x dx f (X,Y ) (x, y)dxdy x+y<1 1 { 1 x 1 7/72. ( {x 2 (1 x) + x 2 + xy 3 x(1 x)2 6 ) } dy dx } dx 4.4 Further Remarks on the Distribution function In this section, we summarize some of the properties of the distribution function that apply both to the discrete and continuous cases. Remarks (Properties of the Bivariate distribution function) (i) F (X,Y ) (x, y) 1 lim F (X,Y ) (x, y) x y lim x y lim x y F (X,Y ) (x, y) 1 F (X,Y ) (x, y) (ii) For fixed x, F (X,Y ) (x, y) is monotone in y. Similarly, for fixed y, F (X,Y ) (x, y) is monotone in x. (iii) lim x y F (X,Y ) (x, y). lim F (X,Y )(x, y) lim P(X x, Y y) P(X, Y y) P(Y y) F Y (y). x x lim F (X,Y )(x, y) lim P(X x, Y y) P(X x, Y ) P(X x) F X (x). y y (iv) If a 1 < a 2 and b 1 < b 2, then P(a 1 < X a 2, b 1 < Y b 2 ) F (X,Y ) (a 2, b 2 ) F (X,Y ) (a 1, b 2 ) F (X,Y ) (a 2, b 1 ) + F (X,Y ) (a 1, b 1 ). 5

6 4.5 Conditional distributions Let (X, Y ) be a 2-dimensional random vector with joint p.m.f. p (X,Y ) (x, y) for (x, y) R (X,Y ) R 2. Consider P(X x Y y) P({ω : X(ω) x} {ω : Y (ω) y}) for P({ω : Y (ω) y}) >. Then from our definition of conditional probability, the above expression is equivalent to P({ω : X(ω) x} {ω : Y (ω) y}) P({ω : Y (ω) y}) P(X x, Y y) P(Y y) thus motivating the following definition: p (X,Y )(x, y) p Y (y) Definition (Conditional p.m.f./p.d.f.) (i) For X and Y discrete, the conditional p.m.f. of X given Y, is given by for any y s.t. p Y (y) >. p X Y (x y) P(X x Y y) p (X,Y )(x, y) p Y (y) (ii) For X and Y continuous, the conditional p.d.f. is given by for any y s.t. f Y (y) >. f X Y (x y) f (X,Y )(x, y) f Y (y) Remarks It is easy to show that (i) x p X Y (x y) 1 for y such that p Y (y) >. (ii) f X Y (x y)dx 1 for y such that f Y (y) >. Example f (X,Y ) (x, y) Find the conditional p.d.f. of X given Y. Solution: It can be shown that Therefore, given y (, 1), for < x < 1 y. { 6(1 x y) x>, y >, <x + y <1 o.w. f Y (y) { 3(1 y) 2 y (, 1) o.w. f X Y (x y) f (X,Y )(x, y) f Y (y) 6 2(1 x y) (1 y) 2

7 Definition (Probability from conditional p.d.f.) If (X, Y ) is continuous, then P(X Y y) f X Y (x y)dx. Example X is height in cm. Y is weight in kg. (18, 2), y7. Then P(X Y y) gives us the probability that the height lies between 18 and 2 cm, given that the weight is 7 k.g. Theorem (Theorem of Total Probability) (a) If (X, Y ) is continuous, then (i) f X (x) R Y f X Y (x y)f Y (y)dy for x R X ; (ii) R X, P(X ) R Y P(X Y y)f Y (y)dy. (b) Similarly for the discrete case, with integration and p.d.f. s replaced by summation and p.m.f. s, respectively. Proof (a) (i) f X (x) f (X,Y ) (x, y)dy f X Y (x y)f Y (y)dy. R Y R Y (ii) P(X ) R Y f X (x)dx ( ( ) f Y (y) f X Y (x y)dx dy R Y R Y ) f X Y (x y)f Y (y)dy dx f Y (y)p(x Y y)dy where the final equality follows from Definition We can see from this that P(X ) is found by computing the P(X Y y), and then averaging these out over all y R Y. 7

8 Corollary (Bayes Rule/Formula/Theorem) (i) discrete case: p X Y (x y)p Y (y) p Y X (y x) p X Y (x n)p Y (n) (ii) continuous case: f Y X (y x) n R Y f X Y (x y)f Y (y) R Y f X Y (x u)f Y (u)du. Definition (Independence of r.v. s) X, Y are independent for all R X, B R Y P(X, Y B) P(X )P(Y B). Proposition (Independence of r.v. s via marginals) X and Y are independent for all x and y Proof See ppendix. p (X,Y ) (x, y) p X (x)p Y (y) discrete case f (X,Y ) (x, y) f X (x)f Y (y) continuous case. Remarks (Important Observation!) (i) For independence, f (X,Y ) (x, y) f X (x)f Y (y). But L.H.S. is > (x, y) R (X,Y ) ; R.H.S. is > x R X and y R Y. Hence, if X, Y are independent, then R (X,Y ) has a rectangular structure. (ii) Rectangular R (X,Y ) is a necessary but not a sufficient condition for independence. Corollary (i) R (X,Y ) not rectangular X, Y not independent. (ii) X, Y independent then (a) f X Y (x y) f X (x), f Y X (y x) f Y (y) (b) F (X,Y ) (x, y) F X (x)f Y (y). Example Let (X, Y ) be continuous, with joint p.d.f. { e (x+y) x, y f (X,Y ) (x, y) o.w. 8

9 re X and Y independent? Solution: Observe that f (X,Y ) (x, y) e (x+y) e x e y for (x, y) R (X,Y ) {(u, v) : u, v }. Since e x and e y are both p.d.f. s on R X {x : x } and R Y {y : y } (each corresponding to the Exp(1) distribution), then we may conclude that X and Y are independent r.v. s. Example re X and Y independent? f (X,Y ) (x, y) { x 2 + xy/3 x 1, y 2 o.w. Solution: Since f (X,Y ) cannot be factorized into the product of two marginals, then X, Y are not independent, in spite of the fact that R (X,Y ) is rectangular in this case. 4.6 Conditional Expectation Definition (i) discrete case: (ii) continuous case: The conditional expectation of X given Y y, is given by: E[X Y y] E[X Y y] x xp X Y (x y) y R Y. xf X Y (x y)dx y R Y. Remarks The conditional expectations can be used to compute the total expectation (provided that they exist), since: E[X] x xp X (x) x x y p (X,Y ) (x, y) x,y xp (X,Y ) (x, y) x,y xp X Y (x y)p Y (y) y for the discrete case. For the continuous case, { } xp X Y (x y) p Y (y) E[X Y y]p Y (y) x y E[X] E[X Y y]f Y (y)dy. 9

10 ppendix Proof of Proposition We ll restrict our attention to the continuous case. ( ) Set (x, x + δx) and B (y, y + δy) where δx,δy are small. From Definition 4.5.8, X, Y independent implies that P(x X x + δx, y Y y + δy) P(x X x + δx)p(y Y y + δy). But L.H.S. is equal to and R.H.S. is equal to ( x+δx Hence x x+δx y+δy i.e. f (X,Y ) (x, y) f X (x)f Y (y). ( ) x y f (X,Y ) (u, v)dudv f (X,Y ) (x, y)δxδy ) ( y+δy ) f X (u)du f Y (v)dv f X (x)f Y (y)δxδy. y f (X,Y ) (x, y)δxδy f X (x)f Y (y)δxδy P(X, Y B) P((X, Y ) B) B ( ) ( f X (x)f Y (y)dxdy f X (x)dx where the final equality follows from Remark (iii). B B f (X,Y ) (x, y)dxdy ) f Y (y)dy P(X )P(Y B) 1