Stat 5101 Lecture Slides: Deck 8 Dirichlet Distribution. Charles J. Geyer School of Statistics University of Minnesota

Transcription

1 Stat 5101 Lecture Slides: Deck 8 Dirichlet Distribution Charles J. Geyer School of Statistics University of Minnesota 1

2 The Dirichlet Distribution The Dirichlet Distribution is to the beta distribution as the multinomial distribution is to the binomial distribution. We get it by the same process that we got to the beta distribution (slides , deck 3), only multivariate. Recall the basic theorem about gamma and beta (same slides referenced above). 2

3 Theorem 1. Suppose X and Y are independent gamma random variables then X Gam(α 1, λ) Y Gam(α 2, λ) U = X + Y V = X/(X + Y ) are independent random variables and U Gam(α 1 + α 2, λ) V Beta(α 1, α 2 ) 3

4 Corollary 1. Suppose X 1, X 2,..., are are independent gamma random variables with the same shape parameters X i Gam(α i, λ) then the following random variables X 1 X 1 + X 2 Beta(α 1, α 2 ) X 1 + X 2 X 1 + X 2 + X 3 Beta(α 1 + α 2, α 3 ). X X d 1 X X d Beta(α α d 1, α d ) are independent and have the asserted distributions. 4

5 From the first assertion of the theorem we know X X k 1 Gam(α α k 1, λ) and is independent of X k. theorem says Thus the second assertion of the X X k 1 X X k Beta(α α k 1, α k ) ( ) and ( ) is independent of X X k. That proves the corollary. 5

6 Theorem 2. Suppose X 1, X 2,..., are as in the Corollary. Then the random variables satisfy d i=1 Y i = Y i = 1, X i X X d and the joint density of Y 2,..., Y d is almost surely. f(y 2,..., y d ) = Γ(α α d ) Γ(α 1 ) Γ(α d ) (1 y 2 y d ) α 1 1 d i=2 y α i 1 i The Dirichlet distribution with parameter vector (α 1,..., α d ) is the distribution of the random vector (Y 1,..., Y d ). 6

7 Let the random variables in Corollary 1 be denoted W 2,..., W d so these are independent and Then W i = X X i 1 X X i Beta(α α i 1, α i ) Y i = (1 W i ) d j=i+1 W j, i = 2,..., d, where in the case i = d we use the convention that the product is empty and equal to one. 7

8 X i Y i = X X d W i = X X i 1 X X i The inverse transformation is W i = Y Y i 1 Y Y i = 1 Y i Y d 1 Y i+1 Y d, i = 2,..., d, where in the case i = d we use the convention that the the sum in the denominator of the fraction on the right is empty and equal to zero, so the denominator itself is equal to one. 8

9 w i = 1 y i y d 1 y i+1 y d This transformation has components of the Jacobian matrix w i 1 = y i 1 y i+1 y d w i = 0, j < i y j w i 1 = + 1 y i y d y j 1 y i+1 y d (1 y i+1 y d ) 2, j > i 9

10 Since this Jacobian matrix is triangular, the determinant is the product of the diagonal elements det h(y 2,..., y d ) = d 1 i=2 1 1 y i+1 y d. 10

11 The joint density of W 2,..., W d is d i=2 Γ(α α i ) Γ(α α i 1 )Γ(α i ) wα 1+ +α i 1 1 i (1 w i ) α i 1 = Γ(α α d ) Γ(α 1 ) Γ(α d ) d i=2 w α 1+ +α i 1 1 i (1 w i ) α i 1 11

12 PMF of W s Jacobian Γ(α 1 + +α d ) Γ(α 1 ) Γ(α d ) d 1 i=2 1 1 y i+1 y d transformation w i = 1 y i y d 1 y i+1 y d di=2 w α 1+ +α i 1 1 i (1 w i ) α i 1 The PMF of Y 2,..., Y d is Γ(α α d ) Γ(α 1 ) Γ(α d ) d i=2 (1 y i y d ) α 1+ +α i 1 1 y α i 1 i (1 y i+1 y d ) α 1+ +α i 1 = Γ(α α d ) Γ(α 1 ) Γ(α d ) (1 y 2 y d ) α 1 1 d i=2 y α i 1 i 12

13 Univariate Marginals Write I = {1,..., d}. By definition Y i = X X d has distribution the beta distribution with parameters α i and by Theorem 1 because j I j i j I j i X i α j X i Gam(α i, λ) X j Gam j I j i α j, λ 13

14 Multivariate Marginals Multivariate Marginals are almost Dirichlet. As was the case with the multinomial, if we collapse categories, we get a Dirichlet. Let A be a partition of I, and define Z A = i A Y i, A A. β A = i A α i, A A. Then the random vector having components Z A has the Dirichlet distribution with parameters β A. 14

15 Conditionals Y i = X i X X d Y i = = = X i X1 + + X k X X k X X d X i (Y Y k ) X X k X i (1 Y k+1 Y d ) X X k 15

16 Conditionals (cont.) Y i = X i X X k (1 Y k+1 Y d ) When we condition on Y k+1,..., Y d, the second term above is a constant and the first term a component of another Dirichlet random vector having components Z i = X i X X k, i = 1,..., k So conditionals of Dirichlet are constant times Dirichlet. 16

17 Moments From the marginals being beta, we have E(Y i ) = var(y i ) = α i α α d α i (α α d ) 2 (α α d + 1) j I j i α j 17

18 Moments (cont.) From the PMF we get the theorem associated with the Dirichlet distribution. so (1 y 2 y d ) α 1 1 d y α i 1 i i=2 E(Y 1 Y 2 ) = Γ(α 1 + 1)Γ(α 2 + 1)Γ(α 3 ) Γ(α d ) Γ(α α d + 2) α = 1 α 2 (α α d + 1)(α α d ) dy 2 dy d = Γ(α 1) Γ(α d ) Γ(α α d ) Γ(α α d ) Γ(α 1 ) Γ(α d ) 18

19 Moments (cont.) The result on the preceding slide holds when 1 and 2 are replaced by i and j for i j, and cov(y i, Y j ) = E(Y i Y j ) E(Y i )E(Y j ) α i α j = (α α d + 1)(α α d ) α i α j (α α d ) [ ] 2 α i α j 1 = α α d α α d α α d α i α j = (α α d ) 2 (α α d + 1) 19