Joint Distributions: Part Two 1 STA 256: Fall 2018 1 This slide show is an open-source document. See last slide for copyright information. 1 / 30
Overview 1 Independence 2 Conditional Distributions 3 Transformations 2 / 30
Independent Random Variables Discrete or Continuous The random variables X and Y are said to be independent if F xy (x, y) = F x (x)f y (y) For all real x and y. 3 / 30
Theorem (for discrete random variables) Recalling independence means F xy(x, y) = F x(x)f y(y) The discrete random variables X and Y are independent if and only if p xy (x, y) = p x (x) p y (y) for all real x and y. 4 / 30
Theorem (for continuous random variables) Recalling independence means F xy(x, y) = F x(x)f y(y) The continuous random variables X and Y are independent if and only if f xy (x, y) = f x (x) f y (y) for all real x and y. 5 / 30
Conditional Distributions Of discrete random variables If X and Y are discrete random variables, the conditional probability mass function of X given Y = y is just a conditional probability. It is given by P (X = x Y = y) = P (X = x, Y = y) P (Y = y) These are just probabilities of events. For example, P (X = x, Y = y) = P {ω Ω : X(ω) = x and Y (ω) = y} We write p x y (x y) = p x,y(x, y) p y (y) Note that p x y (x y) is defined only for y values such that p y (y) > 0. 6 / 30
Conditional Probability Mass Functions Both ways p y x (y x) = p x,y(x, y) p x (x) p x y (x y) = p x,y(x, y) p y (y) Defined where the denominators are non-zero. 7 / 30
Independence makes sense In terms of conditional probability mass functions Suppose X and Y are independent. Then p xy (x, y) = p x (x)p y (y), and p x y (x y) = p x,y(x, y) p y (y) = p x(x)p y (y) p y (y) = p x (x) So we see that the conditional distribution of X given Y = y is identical for every value of y. It does not depend on the value of y. 8 / 30
The other way Suppose the conditional distribution of X given Y = y does not depend on the value of y. Then p x y (x y) = p x (x) p x (x) = p x,y(x, y) p y (y) p x,y (x, y) = p x (x) p y (y) So that X and Y are independent. 9 / 30
Conditional distributions of continuous random variables If X and Y are continuous random variables, the conditional probability density of X given Y = y is f x y (x y) = f x,y(x, y) f y (y) Note that f x y (x y) is defined only for y values such that f y (y) > 0. It looks like we are conditioning on an event of probability zero, but the conditional density is a limit of a conditional probability, as the radius of a tiny region surrounding (x, y) goes to zero. 10 / 30
Conditional Probability Density Functions Both ways f y x (y x) = f x,y(x, y) f x (x) f x y (x y) = f x,y(x, y) f y (y) Defined where the denominators are non-zero. 11 / 30
Independence makes sense In terms of conditional densities Suppose X and Y are independent. Then f xy (x, y) = f x (x)f y (y), and f x y (x y) = f x,y(x, y) f y (y) = f x(x)f y (y) f y (y) = f x (x) And we see that the conditional density of X given Y = y is identical for every value of y. It does not depend on the value of y. 12 / 30
The other way Suppose the conditional density of X given Y = y does not depend on the value of y. Then f x y (x y) = f x (x) f x (x) = f x,y(x, y) f y (y) f x,y (x, y) = f x (x) f y (y) So that X and Y are independent. 13 / 30
Transformations of Jointly Distributed Random Variables Let Y = g(x 1,..., X n ). What is the probability distribution of Y? For example, X 1 is the number of jobs completed by employee 1. X 2 is the number of jobs completed by employee 2. You know the probability distributions of X 1 and X 2. You would like to know the probability distribution of Y = X 1 + X 2. 14 / 30
Convolutions of discrete random variables Let X and Y be discrete random variables. The standard case is where they are independent. Want probability mass function of Z = X + Y. p z (z) = P (Z = z) = P (X + Y = z) = x P (X + Y = z X = x)p (X = x) = x P (x + Y = z X = x)p (X = x) = x P (Y = z x X = x)p (X = x) = x P (Y = z x)p (X = x) by independence = x p x (x)p y (z x) 15 / 30
Summarizing Convolutions of discrete random variables Let X and Y be independent discrete random variables, and Z = X + Y. p z (z) = x p x (x)p y (z x) 16 / 30
Two Important results Proved using the convolution formula Let X Poisson(λ 1 ) and Y Poisson(λ 2 ) be independent. Then Z = X + Y Poisson(λ 1 + λ 2 ). Let X Binomial(n 1, p) and Y Binomial(n 2, p) be independent. Then Z = X + Y Binomial(n 1 + n 2, p) 17 / 30
Convolutions of continuous random variables Let X and Y be continuous random variables. The standard case is where they are independent. Want probability density function of Z = X + Y. f z (z) = d P (Z z) dz = d P (X + Y z) dz 18 / 30
Continuing... f z (z) = d P (X + Y z) dz = d dz z x f xy (x, y) dy dx t = y + x y = t x dy = dt y t = y + x z x z z f xy(x, t x) dt 19 / 30
Still continuing, have f z (z) = d dz = d dz = = z z f xy (x, z x) dx f xy (x, t x) dt dx f xy (x, t x) dx dt f x (x)f y (z x) dx if X and Y are independent. 20 / 30
Compare For continuous random variables: f z (z) = For discrete random variables: f x (x)f y (z x) dx p z (z) = x p x (x)p y (z x) Of course you need to pay attention to the limits of integration or summation, because f x (x)f y (z x) may be zero for some x. 21 / 30
Two Important results for continuous random variables Proved using the convolution formula Let X and Y be independent exponential random variables with parameter λ > 0. Then Z = X + Y Gamma(α = 2, λ). Let X Normal(µ 1, σ 1 ) and Y Normal(µ 2, σ 2 ) be independent. Then ( Z = X + Y Normal µ 1 + µ 2, ) σ1 2 + σ2 2. 22 / 30
The Jacobian Method X 1 and X 2 are continuous random variables. Y 1 = g 1 (X 1, X 2 ) and Y 2 = g 2 (X 1, X 2 ). Want f y1 y 2 (y 1, y 2 ) Solve for x 1 and x 2, obtaining x 1 (y 1, y 2 ) and x 2 (y 1, y 2 ). Then x 1 x 1 f y1 y 2 (y 1, y 2 ) = f x1 x 2 ( x 1 (y 1, y 2 ), x 2 (y 1, y 2 ) ) abs y 1 y 2 The determinant a c b d = ad bc. x 2 y 1 x 2 y 2 23 / 30
More about the Jacobian method Y 1 = g 1(X 1, X 2) and Y 2 = g 2(X 1, X 2) It follows directly from a change of variables formula in multi-variable integration. The proof is omitted. It must be possible to solve y 1 = g 1 (x 1, x 2 ) and y 2 = g 2 (x 1, x 2 ) for x 1 and x 2. That is, the function g : R 2 R 2 must be one to one (injective). Frequently you are only interested in Y 1, and Y 2 = g 2 (X 1, X 2 ) is chosen to make reverse solution easy. The partial derivatives must all be continuous, except possibly on a set of probability zero (they almost always are). It extends naturally to higher dimension. 24 / 30
Change from rectangular to polar co-ordinates By the Jacobian method A point on the plane may be represented as (x, y), or An angle θ and a radius r. 25 / 30
Change of variables x = r cos(θ) y = r sin(θ) x 2 + y 2 = r 2 As x and y range from to, r goes from 0 to And θ goes from θ to 2π. 26 / 30
Integral 0 Change of variables: 0 f x,y (x, y) dx dy x = r cos(θ) y = r sin(θ) = 0 0 π/2 0 0 f x,y (x, y) dx dy f x,y (r cos θ, r sin θ) abs x r y r x θ y θ dr dθ 27 / 30
Evaluate the determinant (with x = r cos(θ) and y = r sin(θ)) x r y r x θ y θ = = cos(θ) sin(θ) r cos(θ) r r sin(θ) r r cos(θ) θ r sin(θ) θ r sin(θ) r cos(θ) = r cos 2 θ r sin 2 θ = r(sin 2 θ + cos 2 θ) = r 28 / 30
So the integral is f x,y (x, y) dx dy = π/2 0 0 0 0 f x,y (r cos θ, r sin θ) r dr dθ The standard formula for change from rectangular to polar co-ordinates is dx dy = r dr dθ. It comes from a Jacobian. Other limits of integration are possible. 29 / 30
Copyright Information This slide show was prepared by Jerry Brunner, Department of Statistical Sciences, University of Toronto. It is licensed under a Creative Commons Attribution - ShareAlike 3.0 Unported License. Use any part of it as you like and share the result freely. The L A TEX source code is available from the course website: http://www.utstat.toronto.edu/ brunner/oldclass/256f18 30 / 30