Inverse Wishart Distribution and Conjugate Bayesian Analysis

Transcription

1 Inverse Wishart Distribution and Conjugate Bayesian Analysis BS2 Statistical Inference, Lecture 14, Hilary Term 2008 March 2, 2008

2 Definition Testing for independence Hotelling s T 2 If W 1 W d (f 1, Σ) and W 2 W d (f 2, Σ) with f 1 d, then the distribution of det(w 1 ) Λ = det(w 1 + W 2 ) is Wilks distribution and denoted by Λ(d, f 1, f 2 ). It holds that Λ D = where B i are independent and follow Beta distributions with d i=1 B i B i B{(f i)/2, f 2 /2)}.

3 Definition Testing for independence Hotelling s T 2 Wilks distribution occurs as the likelihood ratio test for independence. Consider W W d (f, Σ) and the hypothesis that Σ 12 = 0 for a fixed block partitioning of Σ into r r, r s and s s matrices. The likelihood ratio statistic then becomes { } L( ˆK 11, ˆK 22 ) det(w ) n/2 = = U L( ˆK) n/2, det(w 11 ) det(w 22 ) where It follows that U Λ(r, f s, s) = Λ(s, f r, r). Λ(d, f 1, f 2 ) = Λ(f 2, f 1 + f 2 d, d).

4 Definition Testing for independence Hotelling s T 2 This is the equivalent of Student s t-distribution. Let Y N d (µ, cσ), W W d (f, Σ) with f d, and Y W. is known as Hotelling s T 2. It holds that T 2 /f T 2 = f (Y µ) W 1 (Y µ)/c Λ(d, f, 1) = Λ(1, f d + 1, d) and f d + 1 T 2 F (d, f + 1 d) fd where F denotes Fisher s F -distribution.

5 Recall that the Wishart density has the form f d (w f, Σ) (det w) (f d 1)/2 e tr(σ 1 w)/2. Since the likelihood function for Σ is L(K) = (det K) f /2 e tr(kw )/2, a conjugate family of distributions for K is given by π(k; a, Ψ) (det K) a/2 1 e tr(kψ)/2, which thus specifies a Wishart distribution for the concentration matrix.

6 We then say that Σ follows an inverse Wishart distribution if K = Σ 1 follows a Wishart distribution, formally expressed as Σ IW d (δ, Ψ) K = Σ 1 W d (δ + d 1, Ψ 1 ), i.e. if the density of K has the form f (K δ, Ψ) (det K) δ/2 1 e tr(ψk)/2. We repeat the expression for the standard Wishart density: f d (w f, Σ) (det w) (f d 1)/2 e tr(σ 1 w)/2. It follows that the family of inverse Wishart distributions is a conjugate family for Σ.

7 If the prior distribution of Σ is IW d (δ, Ψ) and W Σ W d (f, Σ), we get for the posterior density of K that f (K δ, Ψ, W ) (det K) f /2 tr(kw )/2 e (det K) δ/2 1 e tr(ψk)/2 = (det K) (f +δ)/2 1 e tr{(ψ+w )K}/2, and hence the posterior distribution is simply IW d (δ + f, Ψ + W ) = IW d (δ, Ψ ). We can thus interpret the parameter δ as a prior equivalent sample size and Ψ as the value of a matrix of sums and squares and products from a previous sample.

8 We need the full form of the Wishart density for K, as constants may become important and recall that f d (K δ, Ψ) = q(d, δ) 1 (det Ψ) (δ+d 1)/2 (det K) δ/2 1 e tr(ψk)/2 The constant q(d, δ) is q(d, δ) = 2 (δ+d 1)d/2 (2π) d(d 1)/4 d Γ{(δ + d i)/2}. i=1

9 Consider now alternative models M 1 with Σ arbitrary and M 2 with Σ of block diagonal form: ( ) Σ11 0 Σ =. 0 Σ 22 If the associated prior distributions are for M 1 that Σ IW d (δ, I d ) and for M 2 that Σ 11 IW r (δ, I r ), Σ 22 IW s (δ, I s ), we can now calculate the Bayes factor.