Trace-class Monte Carlo Markov Chains for Bayesian Multivariate Linear Regression with Non-Gaussian Errors

Transcription

1 Trace-class Monte Carlo Markov Chains for Bayesian Mltivariate Linear Regression with Non-Gassian Errors Qian Qin and James P. Hobert Department of Statistics University of Florida Janary 6 Abstract Let π denote the intractable posterior density that reslts when the likelihood from a mltivariate linear regression model with errors from a scale mixtre of normals is combined with the standard non-informative prior. There is a simple data agmentation algorithm based on latent data from the mixing density) that can be sed to explore π. Let h ) and d denote the mixing density and the dimension of the regression model, respectively. Hobert et al. 6) have recently shown that, if h converges to at the origin at an appropriate rate, and d h) d <, then the Markov chains nderlying the DA algorithm and an alternative Haar PX-DA algorithm are both geometrically ergodic. In fact, something mch stronger than geometric ergodicity often holds. Indeed, it is shown in this paper that, nder simple conditions on h, the Markov operators defined by the DA and Haar PX-DA Markov chains are trace-class, i.e., compact with smmable eigenvales. Many of the mixing densities that satisfy Hobert et al. s 6) conditions also satisfy the new conditions developed in this paper. Ths, for this set of mixing densities, the new reslts provide a sbstantial strengthening of Hobert et al. s 6) conclsion withot any additional assmptions. For example, Hobert et al. 6) showed that the DA and Haar PX-DA Markov chains are geometrically ergodic whenever the mixing density is generalized inverse Gassian, log-normal, Fréchet with shape parameter larger than d/), or inverted gamma with shape parameter larger than d/). The reslts in this paper show that, in each of these cases, the DA and Haar PX-DA Markov operators are, in fact, trace-class. Key words and phrases. Compact operator, Data agmentation algorithm, Haar PX-DA algorithm, Heavy-tailed distribtion, Scale mixtre, Markov operator, Trace-class operator

2 Introdction Consider the mltivariate linear regression model Y = Xβ + εσ, ) where Y denotes an n d matrix of responses, X is an n p matrix of known covariates, β is a p d matrix of nknown regression coefficients, Σ is an nknown positive-definite scale matrix, and ε is an n d matrix whose rows are iid random vectors from a scale mixtre of mltivariate normal densities. In particlar, letting ε T i denote the ith row of ε, we assme that ε i has density f h ε) = d π) d exp } εt ε h) d, where h :, ) [, ) is the so-called mixing density. Error densities of this form are often sed when heavy-tailed errors are reqired. For example, it is well known that if h is a Gamma ν, ν ) density with mean ), then f h becomes the mltivariate Stdent s t density with ν degrees of freedom. A Bayesian analysis of the data from this regression model reqires a prior on β, Σ). We consider an improper defalt prior that takes the form ωβ, Σ) Σ a I Sd Σ) where S d R dd+) denotes the space of d d positive definite matrices. Taking a = d + )/ yields the independence Jeffreys prior, which is the standard non-informative prior for mltivariate location scale problems. Of corse, whenever an improper prior is sed, one mst check that the corresponding posterior distribtion is proper. Letting y denote the observed vale of Y, the joint density of the data from model ) can be expressed as [ n d fy β, Σ) = π) d Σ Define i= my) = S d exp ) T ) } ] y i β T x i Σ y i β T x i h) d. R p d fy β, Σ) ωβ, Σ) dβ dσ. The posterior distribtion is proper precisely when my) <. Let Λ stand for the n p + d) matrix X : y). Straightforward argments sing ideas from Fernández and Steel 999)) show that, together, the following for conditions are sfficient for posterior propriety: S) rankλ) = p + d ; S) n > p + d a ; S3) d h) d < ; S4) n p+a d h) d <.

3 These for conditions are assmed to hold throghot this paper. Remark. Conditions S) & S) are known to be necessary for posterior propriety Fernández and Steel, 999; Hobert et al., 6). Remark. Condition S3) clearly concerns the tail behavior of h. Similarly, condition S4) concerns the behavior of h near the origin, nless n p+a d is negative, which is possible. Note, however, that S) implies that n p+a d )/ < /. Conseqently, if n p+a d is negative, then S4) is implied by S3). Of corse, the posterior density of β, Σ) given the data takes the form πβ, Σ y) = fy β, Σ) ωβ, Σ) my). There is a well-known data agmentation DA) algorithm that can be sed to explore this intractable density Li, 996). Hobert et al. 6) hereafter HJK&Q) performed convergence rate analyses of the Markov chains nderlying this DA algorithm and an alternative Haar PX-DA algorithm. In this paper, we provide a sbstantial improvement of HJK&Q s main reslt. A formal statement of the DA algorithm reqires some bildp. Let z = z,..., z n ) have strictly positive elements, and let Q = Qz) be the n n diagonal matrix whose ith diagonal element is zi. Also, define Ω = X T Q X) and µ = X T Q X) X T Q y. For each s, define a nivariate density as follows ψ; s) = bs) d e s h), ) where bs) is the normalizing constant. The DA algorithm ses draws from the inverse Wishart IW d ) and matrix normal N p,d ) distribtions. These densities are defined in the Appendix. If the crrent state of the DA Markov chain is β m, Σ m ) = β, Σ), then we simlate the new state, β m+, Σ m+ ), sing the following three-step procedre. Iteration m + of the DA algorithm:. Draw Z i } n i= independently with Z i ψ ; β T ) T x i y i Σ β T ) ) x i y i, and call the reslt z = z,..., z n ).. Draw ) ) Σ m+ IW d n p + a d, y T Q y µ T Ω µ. 3. Draw β m+ N p,d µ, Ω, Σm+ ) 3

4 Denote the DA Markov chain by Φ = β m, Σ m )} m=, and its state space by X := Rp d S d. For positive integer m, let k m : X X, ) denote the m-step Markov transition density Mtd) of Φ, so that if A is a measrable set in X, P β m, Σ m ) A ) β, Σ ) = β, Σ) = A k m β, Σ ) β, Σ) ) dβ dσ. The -step Mtd, k k, can be expressed as k β, Σ ) ) β, Σ) = πβ Σ, z, y) πσ z, y) πz β, Σ, y) dz, R n + where the precise forms of the conditional densities πz β, Σ, y), πσ z, y), and πβ Σ, z, y) can be gleaned from steps.,., and 3. of the DA algorithm, respectively. If there exist M : X [, ) and λ [, ) sch that, for all m, k m β, Σ β, Σ) πβ, Σ y) dβ dσ M β, Σ) λ m, S d R p d then the chain Φ is geometrically ergodic. The benefits of sing a geometrically ergodic Monte Carlo Markov chain have been well docmented see, e.g. Flegal et al., 8; Jones and Hobert, ; Roberts and Rosenthal, 998). HJK&Q showed that, if h converges to zero at the origin at an appropriate rate, then Φ is geometrically ergodic. In order to state HJK&Q s reslt, we mst introdce three classes of mixing densities. Let h :, ) [, ) be a mixing density. If there is an η > sch that h) = for all, η), then we say that h is zero near the origin. Now assme that h is strictly positive in a neighborhood of. If there exists a c > sch that h) lim c, ), then we say that h is polynomial near the origin with power c. Finally, if for every c >, there exists an η c > sch that the ratio h) is strictly increasing in, η c c ), then we say that h is faster than polynomial near the origin. HJK&Q showed that every mixing density that is a member of a standard parametric family is in one of these three classes, and they proved the following reslt. Theorem HJK&Q). If the mixing density, h, is zero near the origin, or faster than polynomial near the origin, or polynomial near the origin with power c > n p+a d, then the DA Markov chain is geometrically ergodic. Remark 3. It is not necessary to check that S4) holds before applying Theorem becase, together with S3), the hypothesis of Theorem implies that S4) is satisfied. In this paper, we show that something mch stronger than geometric ergodicity often holds. We begin with some reqisite backgrond material on Markov operators. The posterior density can be sed to define an inner prodct f, f = X f β, Σ) f β, Σ) πβ, Σ y) dβ dσ, 4

5 and norm f = f, f on the Hilbert space L = f : X R : f β, Σ) πβ, Σ y) dβ dσ < and X X } fβ, Σ) πβ, Σ y) dβ dσ =. Now define the DA Markov operator K : L L as that which takes f L into Kf)β, Σ) = fβ, Σ ) k β, Σ ) β, Σ) ) dβ dσ. X Becase K is based on a DA algorithm, it is self-adjoint and positive Li et al., 994). If, in addition, K is also a compact operator, then K has a pre eigenvale spectrm, all of its eigenvales reside in [, ), and the corresponding Markov chain is geometrically ergodic see, e.g., Hobert et al., ; Mira and Geyer, 999). We note that the set of Monte Carlo Markov chains whose operators are compact is a small sbset of those that are geometrically ergodic see, e.g., Chan and Geyer, 994, p. 755). Taking this a step frther, K is said to be trace-class if it is compact and its eigenvales are smmable see, e.g. Conway, 99, p. 67). In this paper, we provide sfficient conditions on h) for K to be trace-class. The benefits of sing trace-class Markov operators are spelled ot in Khare and Hobert ), and we exploit their reslts in Section 4. A statement of or main reslt reqires sbstantial bild-p, so here in the Introdction we present only one simple, bt powerfl, corollary. Let R + denote the set, ), and define a parametric family of fnctions g ρ,τ : R + [, ) as follows. For ρ R + and τ R, let g ρ,τ ) = exp ρlog ) + τ log }. The following reslt is a corollary of Theorem in Section. Corollary. Let h be a mixing density. If there exist ρ R +, τ R and η > sch that h)/g ρ,τ ) is non-decreasing in, η), then the DA Markov operator, K, is trace-class. An immediate conseqence of Corollary is that, if h is zero near the origin, then K is traceclass. Indeed, for any ρ, τ) R + R, h)/g ρ,τ ) is constant and eqal to zero) in a neighborhood of the origin. Corollary also implies that if h is a member of one of the standard parametric families that are faster than polynomial near the origin inverted gamma, log-normal, generalized inverse Gassian, and Fréchet), then the corresponding Markov operator is trace-class. For example, consider the case where the mixing density is inverted gamma. In particlar, let h) = b α e γ/ I R+ ), where α > d/, γ > and b = bα, γ) is the normalizing constant. We reqire α > d/ so that condition S3) is satisfied.) Taking ρ = and τ = α + ), we have d h) d g ρ,τ ) = b d d exp γ + log )} = b [ γ ] + log exp γ + log )}, which is clearly positive in a neighborhood of. Hence, Corollary implies that K is trace-class. This reslt was established by Jng and Hobert 4) in the special case where d =.) Similar 5

6 argments can be sed for the other three families log-normal, generalized inverse Gassian and Fréchet), and these are given in Section. Indeed, for a large class of mixing densities inclding the ones jst mentioned) or reslts provide a sbstantial strengthening of Hobert et al. s 6) conclsion withot any additional assmptions. On the other hand, as we now explain, there are still many mixing densities that satisfy the hypotheses of Theorem, bt to which or reslts are not applicable. The following lemma, which is proven in Section, provides a sfficient condition for K to be trace-class, and is one of the key pieces of the proof of Theorem and hence of Corollary ). Lemma. Let h be a mixing density that is strictly positive in a neighborhood of the origin. If there exist ζ, ) and η > sch that then K is trace-class. d h) d <, 3) v d hv) dv Note that 3) cannot hold if, for each η >, d h) d =. 4) v d hv) dv Conseqently, when h is strictly positive in a neighborhood of the origin and 4) holds, or reslts cannot be applied to h. For example, assme that h is polynomial near the origin with power c so that h) l R c + as. Then for all is some small neighborhood of the origin, we have l/ < h) < l. So, if η > is small enogh, then for all, η), c d h) v d hv) dv d l c v d lv c dv = b, where b = bd, l) is a positive constant. Ths, since d diverges for every η >, 4) holds. Conseqently, or reslts are not applicable to mixing densities that are polynomial near the origin. Frthermore, in Section, we give an example of a mixing density that is faster than polynomial near the origin, bt for which 4) holds. The remainder of this paper is organized as follows. The main reslt is stated and proven in Section. In Section 3, we examine the conseqences of the main reslt when the mixing density is faster than polynomial near the origin. In Section 4, we show that Theorem has important implications for a Haar PX-DA variant of the DA algorithm that was introdced by Roy and Hobert ) and extended by HJK&Q. Finally, the Appendix contains the definitions of the IW d and N p,d families, as well as some technical details. 6

7 Main Reslt In this section, we will be dealing with fnctions g : R + [, ) that are strictly positive and differentiable in a neighborhood of the origin. Let A denote the set of all sch fnctions, and let K denote the sbset of A consisting of fnctions whose reciprocals are integrable near the origin, i.e., K = κ A : } κ) d < for some η >. The fnction κ) = log ) is a member of K, and we will se this fact in the seqel. Now, for fixed κ K and fixed ζ, ), let Cκ, ζ) denote the sbset of A containing the fnctions g that satisfy the following three conditions:. d g) is bonded in a neighborhood of the origin,. lim κ) d g) =, 3. There exist l, l R sch that lim κ ) + d The following reslt is proven in the Appendix. κ) ) g) gζ) = l κ)g ) and lim = l. 5) gζ) Proposition. Fix ρ, τ) R + R. Then g ρ,τ Cκ, 3/), with κ) = log ). Frthermore, d g ρ,τ ) is non-decreasing in a neighborhood of the origin. Here is or main reslt. Theorem. Let h be a mixing density. Each of the following three conditions is sfficient for the corresponding DA Markov operator, K, to be trace-class.. The mixing density h is zero near the origin.. There exist κ K, ζ, ) and g Cκ, ζ) sch that lim h) g) R +. are non- 3. There exist κ K, ζ, ) and g Cκ, ζ) sch that both d g) and h) g) decreasing in a neighborhood of the origin. Remark 4. Sppose that h Cκ, ζ). Then, by taking g = h, the second condition of Theorem is satisfied, so K is trace-class. However, this argment reqires that h be differentiable in a neighborhood of the origin. The srrogate fnction, g, allows s to handle non-differentiable mixing densities. Remark 5. Note that Corollary from the Introdction) follows immediately from Theorem and Proposition. 7

8 Or proof of Theorem is based on three lemmas, which we now state and prove. Lemma. Let h be a mixing density, and let ψ; s) be as in ). Sppose there exist ζ < and ν : R + [, ) with R + ν) d < sch that for all R + and all s [, ). Then K is trace-class. } ζ )s ψ; s) exp ν) 6) Proof. For i =,,..., n, define r i = r i β, Σ) = β T x i y i ) T Σ β T x i y i ). Of corse, r i. First, it sffices to show that S d R p d k β, Σ) β, Σ) ) dβ dσ <, see, e.g., Khare and Hobert, ). Rotine calclations show that πβ, Σ z, y)πz β, Σ n+a exp n Σ, y) = i= r } iz i n S d R Σ n+a p d exp n i= r } ψz i ; r i ) iz i dβ dσ i= Σ n+a exp ζ) n i= r } iz i n S d R Σ n+a p d exp n i= r } νz i ). iz i dβ dσ i= The transformation Σ = Σ/ ζ), yields S d R p d Σ n+a ζ) exp = n } r i z i dβ dσ i= ζ) n+a d )d S d R p d Σ n+a exp n } r i z i dβ dσ. i= It follows that, S d R p d πβ, Σ z, y)πz β, Σ, y) dβ dσ ζ) n+a d )d n νz i ). i= Therefore, k β, Σ) ) β, Σ) dβ dσ = R p d S d R n + S d πβ, Σ z, y) πz β, Σ, y) dβ dσ dz R p d ) n ν) d <. R + ζ) n+a d )d The following lemma was given in the Introdction, and is restated here for convenience. 8

9 Lemma. Let h be a mixing density that is strictly positive in a neighborhood of the origin. If there exist ζ, ) and η > sch that then K is trace-class. d h) d <, 3) v d hv) dv Proof. First, note that d e s d h) ψ; s) = R + v d e sv hv) dv e s h) v d e sv hv) dv exp ζ )s } d h) v d hv) dv. By Lemma, it sffices to show that Bt, for any η >, we have R + d h) v d hv) dv d <. and the reslt follows. η d h) v d hv) dv d η d h) d ζη v d hv) dv <, Lemma 3. Let g Cκ, ζ) for some κ K and some ζ, ). By assmption, there exists η > sch that g is strictly positive and differentiable on, η ). Then for any η, η ), we have d g) v d gv) dv d <. Proof. Since d g) is bonded in a neighborhood of the origin, we have lim Hence, an application of L Hôpital s rle yields lim v d gv) dv =. [ κ) d g) κ v d gv) dv = lim ) d + κ) d d ] g) + κ) d g ) ζζ) d gζ) = ζ d + lim κ ) + d κ) ) g) gζ) + lim } κ)g ) gζ) = l + l. 7) ζ d + 9

10 Pt l 3 = l + l )/ζ d +. It follows from 7) that for any η, η ), there exists < η < η sch that d g) v d gv) dv l 3 + κ) whenever, η ). Then, since κ K, there exists η, η ) sch that κ) d <. Frthermore, since g is continos on [η, η], η d g) d <. Ptting all of this together, we have for any η < η, d g) η v d gv) dv d = <. d g) v d gv) dv d + l 3 + κ) d + η η d g) d ζη v d gv) dv d g) v d gv) dv d Proof of Theorem. Assme that h is zero near the origin, and define η = sp η R + : d h) d = }. Clearly, J := 3η d h) d >. Now, for s [, ), we have R + v d e 3η sv hv) dv Therefore, for R + and s [, ), we have v d e sv hv) dv Je 3η s 4. ψ; s) = d e s h) R + v d e sv hv) dv J d h) e s + 3η s 4. Now, by considering η and < η separately, we can see that ψ; s) J d s h) e 4 for all R + and all s [, ). Hence, 6) of Lemma holds with ζ = 3/ and ν) = J d h), so the reslt follows. We now prove that the second condition is sfficient. Assme that there exists g Cκ, ζ) sch that lim h) g) = l R +. Then by Lemma 3, there exists η > sch that d g) v d gv) dv d <,

11 and sch that whenever, ζη). It follows that l h) g) l and the reslt follows from Lemma. d h) η v d hv) dv d 4 d g) v d gv) dv d, Finally, we prove that the third condition is sfficient. Note first that we may assme that h is not zero near the origin, since, otherwise, the reslt follows immediately from condition ). Assme that there exists g Cκ, ζ) sch that d g) and h) g) are both non-decreasing near the origin. By Lemma 3, there exists η > sch that d g) v d gv) dv d <. Now let η, η ) be sch that g and h are both strictly positive for, η), and d g) and h) g) are both non-decreasing in that interval. For, η), let t) = h)/g). For any, η/ζ), we have since t is non-decreasing. If v [, ζ], then v ζv ) ζ v d gv) dv v d hv) dv v d t)gv) dv, 8) [ ζv ) ] d g ζ ζv ) ζ It follows from 8) and 9) that, for, η/ζ), we have. For any, η/ζ), we have ) dv = ζ ζ w d gw) dw. 9) d d d h) v d hv) dv h) v d hv) dv t)g) ζ ) ζ t) v d gv) dv = ζ d g) ζ ζ v d gv) dv. Hence, ζ d h) v d hv) dv d ζ ζ ζ and the reslt follows from Lemma. d g) v d gv) dv d ζ ζ d g) v d gv) dv d <, 3 Mixing densities that are faster than polynomial near the origin In this section, we provide details to back the claims made in the Introdction. We begin by sing Corollary to show that, if h is log-normal, generalized inverse Gassian, or Fréchet, then the

12 DA Markov operator is trace-class. We then provide an example of a mixing density that is faster than polynomial near the origin, bt for which 4) holds. Again, this shows that or reslt is not applicable to this mixing density. Let h be a GIGv, α, γ) density, so that h) = b v exp α + γ )} I R+ ), where α, γ R +, v R, and b = bv, α, γ) is the normalizing constant. It s easy to see that conditions S3) & S4) hold for all members of this family. Taking ρ = and τ = v in Corollary, we have d h) d g ρ,τ ) =b d d exp α γ + log )} = b α + γ + log ) exp α γ + log )}, which is clearly non-negative in a neighborhood of. Ths, K is trace-class. Sppose h is a Fréchetα, γ) density, i.e., h) = b α+) e γα α I R+ ), where α, γ >, and b = bα, γ) is the normalizing constant. Assme that α > d/ so that condition S3) holds. Taking ρ = and τ = α + ), we have d h) d g ρ,τ ) = b d d exp γα αγ α α + log )} = b log + α+ which is clearly non-negative in a neighborhood of, so K is trace-class. Finally, let h be a Log-normalµ, γ) density, so that h) = b exp log µ) )} I R+ ), γ ) exp γα α + log )}, where γ >, µ R and b = bγ, µ) is the normalizing constant. Every member of this family satisfies conditions S3) & S4). Taking ρ = γ and τ = µ γ, we have and the reslt follows. h) g ρ,τ ) = b e We end this section by showing that there exist mixing densities that are faster than polynomial near the origin, bt are not in the domain of application of Theorem. Consider the following mixing density µ γ, d ) } h) = b exp log ) log log ) + log I,) ),

13 where b = bd) is the normalizing constant. For any real c, we have [ d h) d d c = log ) log log ) c + d ) ]} h) d + log [ ] c d + c) h) = log log ) c. d h) When > is small, d >, so h) is indeed faster than polynomial near the origin. We c now show that 4) holds. Define d h) ν h ) = v d hv) dv, and let φ) = log )I,) ). An application of L Hôpital s rle yields Now and Ths, lim φ)ν h) = lim φ ) + d h) ) d h) = exp = exp = lim d d φ) log log ) [ φ) d h) ] ) d h) [ φ ) + φ) log log ) ] d h) ) d h). = log ) log log ) log, } log ) log log ) log log[ log)] log) + log) log ) log[ log)] + log ) log log } log +log = exp log ) log log log ) log. lim φ)ν log ) log log ) log h) = lim log log ) log exp It is straightforward to show that and that Therefore, log lim log ) log log + log = log, } log log + log log ) log log ) log log ) log lim log log ) log =. lim φ)ν h) =. It follows that, for all η in a small neighborhood of the origin, we have ν h ) d = φ)ν h ) φ) d φ) d =. } log. log + log 3

14 4 The Haar PX-DA algorithm The Haar PX-DA algorithm always exists in the special case where a = d+, bt, otside of this case, its existence reqires an additional reglarity condition. Indeed, to define the Haar PX-DA algorithm, we mst assme that [ n ] d+ a)d n+ t htz i ) dt < ) i= for almost) all z R n +. HJK&Q show that ) holds if d+ a)d h) d <. ) Note that ) is atomatic when a = d+. Now assme that ) holds, and define another parametric family of nivariate density fnctions given by ev; z) = vn n i= hvz i) I R+ v) t n n i= htz i) dt. Let Φ = βm, Σ m)} m= denote the Haar PX-DA Markov chain. If the crrent state of the chain is βm, Σ m) = β, Σ), then we simlate the new state, βm+, Σ m+ ), sing the following for-step procedre. Iteration m + of the Haar PX-DA algorithm:. Draw Z i }n i= independently with Z i ψ ; β T ) T x i y i Σ β T ) ) x i y i, and call the reslt z = z,..., z n).. Draw V e ; z ), call the reslt v, and set z = vz,..., vz n) T. 3. Draw ) ) Σ m+ IW d n p + a d, y T Q y µ T Ω µ. 4. Draw βm+ N ) p,d µ, Ω, Σ m+ The following reslt is a direct conseqence of Theorem and Theorems and from Khare and Hobert ). Corollary. Let h be a mixing density sch that ) holds. Let K and K denote the Markov operators associated with the DA and Haar PX-DA Markov chains, respectively. Assme that one of the three conditions of Theorem holds. Then K is trace-class. Moreover, letting λ i } i= and λ i } i= denote the ordered eigenvales of K and K, respectively, we have that λ i λ i < for all i N, and λ i < λ i for at least one i N. 4

15 Acknowledgment. The second athor was spported by NSF Grant DMS Appendices A Matrix Normal and Inverse Wishart Densities Matrix Normal Distribtion Sppose Z is an r c random matrix with density [ f Z z) = exp }] π) rc A c B tr A z θ)b z θ) T, r where θ is an r c matrix, A and B are r r and c c positive definite matrices. Then Z is said to have a matrix normal distribtion and we denote this by Z N r,c θ, A, B) Arnold, 98, Chapter 7). Inverse Wishart Distribtion Sppose W is an r r random positive definite matrix with density w m+r+ exp tr Θ w )} f W w) = mr π rr ) 4 Θ m r i= Γ m + i))i S r W ), where m > r and Θ is an r r positive definite matrix. Then W is said to have an inverse Wishart distribtion and this is denoted by W IW r m, Θ). B Proof of Proposition Proof of Proposition. Fix ρ, τ) R + R. First, that d g ρ,τ ) is non-decreasing in a neighborhood of the origin is obvios. To show that g ρ,τ Cκ, 3/) with κ) = log ), we demonstrate that g ρ,τ satisfies the three conditions that define Cκ, 3/). Clearly lim g ρ,τ ) =, hence d g ρ,τ ) is bonded in a neighborhood of the origin. Moreover, lim κ) d gρ,τ ) = lim log ) d gρ,τ ) =. 5

16 Now, note that lim κ ) + d and κ) ) gρ,τ ) g ρ,τ 3/) [ d + = lim log ) + log [ d + = lim = d + =, exp ] gρ,τ ) g ρ,τ 3/) + ] [ lim log ) exp ρ log 3 ) 3 } ] τ log log ρlog ) + τ log log 3 ρ log 3 )} [ ] τ lim log ) ρ log 3 κ)g ρ,τ ) lim g ρ,τ 3/) = lim log ) g ρ,τ ) g ρ,τ 3/) [ = lim log ) τ ρ log ) ] gρ,τ ) g ρ,τ 3/) = exp log 3 ρ log 3 )} [ ] τ lim τ ρ log )log ) ρ log 3 =. It follows that 5) holds with ζ = 3/ and l = l =. Ths g ρ,τ Cκ, 3/). References ARNOLD, S. F. 98). The Theory of Linear Models and Mltivariate Analysis. Wiley, New York. CHAN, K. S. and GEYER, C. J. 994). Comment on Markov chains for exploring posterior distribtions by L. Tierney. The Annals of Statistics CONWAY, J. B. 99). A Corse in Fnctional Analysis. nd ed. Springer, New York. FERNÁNDEZ, C. and STEEL, M. F. J. 999). Mltivariate Stdent-t regression models: Pitfalls and inference. Biometrika FLEGAL, J. M., HARAN, M. and JONES, G. L. 8). Markov chain Monte Carlo: Can we trst the third significant figre? Statistical Science HOBERT, J. P., JUNG, Y. J., KHARE, K. and QIN, Q. 6). Convergence analysis of MCMC algorithms for Bayesian mltivariate linear regression with non-gassian errors. Tech. rep., University of Florida. ArXiv:56.33v. 6

17 HOBERT, J. P., ROY, V. and ROBERT, C. P. ). Improving the convergence properties of the data agmentation algorithm with an application to Bayesian mixtre modelling. Statistical Science JONES, G. L. and HOBERT, J. P. ). Honest exploration of intractable probability distribtions via Markov chain Monte Carlo. Statistical Science JUNG, Y. J. and HOBERT, J. P. 4). Spectral properties of MCMC algorithms for Bayesian linear regression with generalized hyperbolic errors. Statistics & Probability Letters KHARE, K. and HOBERT, J. P. ). A spectral analytic comparison of trace-class data agmentation algorithms and their sandwich variants. The Annals of Statistics LIU, C. 996). Bayesian robst mltivariate linear regression with incomplete data. Jornal of the American Statistical Association LIU, J. S., WONG, W. H. and KONG, A. 994). Covariance strctre of the Gibbs sampler with applications to comparisons of estimators and agmentation schemes. Biometrika MIRA, A. and GEYER, C. J. 999). Ordering Monte Carlo Markov chains. Tech. Rep. No. 63, School of Statistics, University of Minnesota. ROBERTS, G. O. and ROSENTHAL, J. S. 998). Markov chain Monte Carlo: Some practical implications of theoretical reslts with discssion). Canadian Jornal of Statistics ROY, V. and HOBERT, J. P. ). On Monte Carlo methods for Bayesian mltivariate regression models with heavy-tailed errors. Jornal of Mltivariate Analysis 9. 7