Schur properties of convolutions of gamma random variables

Transcription

1 Schur properties of convolutions of gamma random variables Farbod Roosta-Khorasani Gábor J. Székely January 25, 215 Abstract Sufficient conditions for comparing the convolutions of heterogeneous gamma random variables in terms of the usual stochastic order are established. Such comparisons are characterized by the Schur convexity properties of the cumulative distribution function of the convolutions. Some examples of the practical applications of our results are given. Keywords: Schur-convexity of tails; majorization order; linear combinations; gamma distribution; tail probabilities. MSC 21: Primary 6E15; secondary 62E99 1 Introduction and Main Result Linear combinations i.e., convolutions of independent gamma random variables r.v s often naturally arise in many applications in statistics, engineering, insurance, actuarial science and reliability. As such there has been extensive study of their stochastic properties in the literature. For examples of such theoretical studies as well as applications see [16, 17, 18, 12, 32, 19, 8, 34, 33, 21, 1, 7, 29, 25] and references therein. Bock et al., [6], and Diaconis and Perlman, [9], in their seminal works, first studied the Schur convexity properties of the cumulative distribution function of the linear combinations of independent gamma r.v s. Ever since, this topic and its variants have been studied by many researchers; see the references mentioned above. However Dept. of Computer Science, University of British Columbia, Vancouver, Canada. farbod@cs.ubc.ca. National Science Foundation, Arlington, Virginia. gszekely@nsf.gov and Alfréd Rényi Institute of Mathematics, Hungarian Academy of Sciences, Budapest, Hungary. 1

2 2 despite all the efforts, the results in [6] remained the best available, yet far from the best possible. Bakirov, [5] provided a tighter bound for the special case of convolutions of chi-squared r.v s of degree one, which often arise from quadratic forms. Here, we prove results regarding the Schur properties of the tails of convolutions of gamma r.v s with respect to the mixing weights and in terms of the usual stochastic order, and indeed sharpen some results given in [6]. As a consequence, the result in [5] is also generalized. More specifically, let X i, i = 1, 2,..., n, be n independent and identically distributed i.i.d gamma distributed r.v s, parametrized by shape > and rate β > parameters with the probability density function PDF fx = { β Γ x 1 e βx x x. Consider the following non-negative linear combinations of such r.v s λ i X i, where λ i, i = 1, 2,..., n, are real numbers. For a given >, β > and x >, define P λ;, β, x := Pr λ i X i < x, 1 where λ = λ 1, λ 2,..., λ n R n. The aim of the present paper is to find the conditions allowing one to compare tail probabilities of the form 1 with respect to the mixing weights λ. In [25, Theorem 2.2], results regarding the extremal values of 1 i.e., maximal and minimal values with respect to λ and for given, β and x are proved. Here, we extend those results to be able to compare 1 for any pair of weight vectors. To that end, let us recall that the vector λ is said to majorize the vector µ, denoted by µ λ, if λ n... λ 2 λ 1, 2a < µ n... µ 2 µ 1, 2b k k µ i λ i, k < n, 2c µ i = λ i. Note that there is strict positivity assumption on µ, but not on λ. Of course, it is clear that padding µ and as a result λ with s would not change the majorization order and simply add redundant components to both vectors. Given µ λ, 2d

3 3 P.;, β, x is said to be Schur convex if P µ;, β, x P λ;, β, x, and it is said to be Schur concave if P µ;, β, x P λ;, β, x. For comprehensive details on the theory of majorization and its applications, refer to the classic book of Marshall and Olkin [2]. It is not hard to show that the variance of n λ ix i is a Schur-convex function of λ and, indeed, it would be useful to know when P.;, β, x exhibit similar properties. For symmetric distributions a fairly general result is known. If X 1, X 2..., X n are independent random variables with a common symmetric and log-concave PDF, in [23], it was shown that Pr n λ ix i < x is Schur-convex in λ for any x >. However, for positive random variables with non-symmetric distributions such as gamma r.v s, to the best of our knowledge, no such general results, except for those in [6] and [5], exist. For the case of n 3, Bock et al., [6, Theorem 3], showed that if µ λ and λ i > i, then P µ;, β, x P λ;, β, x, x > n + 1 max i λ i, β P µ;, β, x P λ;, β, x, x < n + 1 min i λ i. β For the special case of chi-squared r.v s of degree one i.e., = β = 1/2, Bakirov, [5], provided the following tighter bound for n 1 for the Schur concavity of P.; 1/2, 1/2, x: P µ; 1 2, 1 2, x P λ; 1 2, 1, x, x > 2s, 2 where s = n λ i = n µ i. No results concerning the Schur convexity of P.; 1/2, 1/2, x was given in [5]. Our main result is stated as follows the details of its proof are given in the Appendix. The discussions regarding the relative improvements compared to [6] and [5], as well as further extensions are deferred to Section 3. Theorem 1 Let X i Gamma, β, i = 1, 2,..., n, be n i.i.d gamma r.v s, where > and β >. If µ λ, then 2 + 1s P µ;, β, x P λ;, β, x, x >, 2β P µ;, β, x P λ;, β, x, x < where s = n λ i = n µ i. s β, n = 2 1s, n 3, > 1 β,

4 4 This paper is organized as follows. In Section 2, we give some examples of the practical applications of our results. In Section 3, we discuss the relative improvements of our results compared to those in [6] and [5]. We also extend Theorem 1 by weakening the majorization requirement. In addition, we give similar results for the case where n =. The proofs of our results are given in the Appendix. 2 Examples In this section, we give examples to demonstrate some practical applications of our results. 2.1 Experimental Design in Signal Detection Consider the additive model of observations Dt = τst + ηt, t [, T ] where Dt is the measured data, st is the signal of interest and ηt is the additive noise. Suppose we have N measurements, taken at discrete time intervals, = t 1 t 1... t N = T. In addition, suppose that {ηt i : i = 1, 2,..., N} is a collection of i.i.d Laplace r.v s with mean zero and variance σ 2 i. In addition, let τ = 1 if there is a signal, and τ = otherwise i.e., the measured data is in fact entirely the noise. To detect if the signal is present, we can use the signal to noise ratio SNR QN := 1 N N 1 σ i Dt i. Note that σ 1 i Dt is a Laplace r.v, with mean τst and variance 1. A design question to answer is that of, at least how many measurements, a priori, is needed to make sure the probability of Type I error i.e., when we conclude that the signal is present, when in fact it is missing is below a desired tolerance < δ 1. If τ =, then σ 1 i Dt Gamma1, 2, so by Theorem 1 we get PrQ1 x PrQ2 x... PrQN x..., x > For a given N, we can easily compute PrQN x. Hence, in order to find the minimum N required, we can increase N until PrQN x δ. 2.2 Matrix Trace Estimation The need to estimate the trace of an implicit symmetric positive semi-definite SPSD matrix is of fundamental importance see [28] and arises in many applications; see

5 5 for instance [15, 4, 3, 14, 1, 31, 27, 26, 3, 13, 2] and references therein. The standard approach for estimating the trace of such a matrix A, denoted here by tra, is based on a Monte-Carlo method, where one generates N random vector realizations w i from a suitable probability distribution D and computes tr N DA := 1 N N wiaw t i. One such suitable probability distribution is the standard normal, N, I. This estimator is known as the Gaussian estimator, denoted here by trg NA. Now, given a pair of small positive real numbers ε, δ, consider finding an appropriate sample size N such that Pr trg N A 1 εtra 1 δ, 3a Pr trg N A 1 + εtra 1 δ. 3b Such question was first studied in [3] and further improved in [24]. In particular, in [24] it was proved that the inequalities 3 hold if N > A 8 tra ε ln1, 4 2 δ where A denotes the L 2 norm of the matrix A. The ratio tra/ A is known as the effective rank of the matrix see [11] and it is a stable quantity compared with the usual rank. The appearance of effective rank in the bound 4 is an indication of a possible relation between the skewness of the eigenvalues of A and the efficiency of the Gaussian estimator. In other words, the more skewed the eigenvalue distribution is, the worse we expect the Gaussian estimator to perform i.e., the larger the true sample size required would be. However effective rank is not a consistent measure for skewness and, as such, in [24], this relationship was demonstrated only numerically and no consistent definition for how the relative skewness could be measured was given. Now using the majorization order among eigenvalue vectors as a consistent measure of skewness, the new theoretical results in the present paper fully describe the observations from the numerical examples in [24]. As in the proof of [24, Theorem 1], we see that P r trha N 1 εtra = P λ; N2, N2, 1 εtra, 5 where λ the vector of eigenvalues of A sorted in the decreasing order. Consider two SPSD matrices, A 1 and A 2, such that tra 1 = tra 2 and whose respective eigenvalue vectors, λ 1 and λ 2, are sorted in the decreasing order. If λ 2 λ 1, then we say that

6 6 the eigenvalue distribution of A 1 is more skewed that that of A 2. If N > 2/ε, then from Theorem 1 we obtain P λ 2 ; N 2, N 2, 1 εtra 2 P λ 1 ; N 2, N 2, 1 εtra 1. In other words, using the same sample size N, our estimate with A 1 is more likely to be located further away to the left of the true value than that with A 2. Hence in order to make the former estimate better, we need to increase the sample size which, in turn, in some algorithms translates into more computational costs; see [14, 1, 3, 27, 26, 25]. Similar comparisons can be made for P r tr N H A 1 + εtra. 3 Discussions and Further Extensions The comparison between the results in Theorem 1 and those in [6] and [5] can be summarized as follows: For the special case of chi-squared distribution, i.e., = β = 1/2, our Schur concavity result is the same as that in [5]. For n = 2, the results of Theorem 1 coincide with [6, Theorem 1]. However, for n 3, our results are more uniform than [6, Theorem 3]. Namely, the sufficient conditions given in Theorem 1 are not dependent on the dimensions of the vectors. Additionally, our bounds for x are independent of the particular values of the vector components, such as min i λ i or max i λ i. In other words, for all vectors whose sums are equal, we give a fixed bound for x. For n 3, our Schur convexity result is sharper than that of [6, Theorem 3] when i max λ i > λ i i n + 1/n. Similarly, for n 3 and > 1, our Schur concavity result improves that of [6, Theorem 3] when i min λ i < λ i 1 i n + 1/n. Most importantly, in [6, Theorem 3], it is required that λ i >, i = 1, 2,..., n. This is a rather strong condition as many interesting comparisons cannot be performed this way. Our results do not make any strict positivity assumption on the components of λ and it suffices if they are simply non-negative. As a simple example, consider = 2, β = 1 and x < 1. Then using Theorem 1, we get P λ 1 ; 2, 1, x P λ 2 ; 2, 1, x P λ 3 ; 2, 1, x P λ 4 ; 2, 1, x with λ 1 = 1,,,, λ 2 = 2/3, 1/3,,, λ 3 = 3/6, 2/6, 1/6,, and λ 4 = 4/1, 3/1, 2/1, 1/1. This comparison is not possible with [6, Theorem 3].

7 7 For the Schur convexity of P.;, β, x in the case of 1 and n 3, the result in [6, Theorem 3] remains the best available, as we were not able to improve upon it here. In fact, from our method of proof, it seems likely that obtaining a general result for this case is impossible. Indeed, Bock et al. [6, p. 394] give an example which corroborates this observation. It is possible to weaken the majorization requirement and obtain even more general results. More specifically, relaxing the equality condition in 2 gives the following weak majorization order. Recall that the vector λ is said to weakly majorize the vector µ, denoted by µ w λ, if λ n... λ 2 λ 1, < µ n... µ 2 µ 1, k k µ i λ i, k n. In the case of weak majorization order, we have the following almost immediate corollary. Corollary 2 Let X i Gamma, β, i = 1, 2,..., n, be n i.i.d gamma r.v s, where > and β >. If µ w λ, then P µ;, β, x P λ;, β, x, x > 2 + 1s λ, 2β P µ;, β, x P λ;, β, x, x < where s λ = n λ i and s µ = n µ i. s µ β, n = 2 1s µ, n 3, > 1 β We can also extend Theorem 1, as well as Corollary 2, for the case where n =. For any non-negative l 1 sequence λ = λ 1, λ 1,..., i.e., λ i <, define P λ;, β, x := Pr λ i X i < x. The weak majorization order is naturally extended to such l 1 sequences. Corollary 3 Let {X i Gamma, β, i = 1, 2,...} be a countably infinite collection of i.i.d gamma r.v s, where > and β >. For any two non-negative l 1 sequences, µ and λ, such that µ w λ, we have P µ;, β, x P λ;, β, x, x > 2 + 1s λ, 2β P µ;, β, x P λ;, β, x, x < 1s µ, > 1, β,

8 8 where s λ = λ i and s µ = µ i. Finally, it might be worth noting that the results such as Theorem 1 show that if µ λ, then P µ;, β, x and P λ;, β, x must have at least one crossing on x,. Diaconis and Perlman in [9] tried to answer whether this crossing point is unique. However, they only proved this uniqueness for n = 2 and for n 3, they required to impose further restrictions. Ever since, this has been an open problem which is known as the Unique Crossing Conjecture UCC and it is quite remarkable that the UCC has remained open, although all the evidence points towards the direction of it being true. Acknowledgment We wish to thank Profs. Milan Merkle and Maochao Xu for their great feedback during the preparation of the text. We would also like to thank the anonymous referee for many valuable comments which helped us improve the presentation and the content of this paper. References [1] L. Amiri, B. Khaledi, and F. J. Samaniego. On skewness and dispersion among convolutions of independent gamma random variables. Probability in the Engineering and Informational Sciences, 251:55 69, 211. [2] H. Avron. Counting triangles in large graphs using randomized matrix trace estimation. Workshop on Large-scale Data Mining: Theory and Applications, 21. [3] H. Avron and S. Toledo. Randomized algorithms for estimating the trace of an implicit symmetric positive semi-definite matrix. JACM, 582, 211. Article 8. [4] Z. Bai, M. Fahey, and G. Golub. Some large scale matrix computation problems. J. Comput. Appl. Math., 74:71 89, [5] N. K. Bakirov. Comparison theorems for distribution functions of quadratic forms of gaussian vectors. Theory of Probability & Its Applications, 42:34 348, [6] M. E. Bock, P. Diaconis, F. W. Huffer, and M. D. Perlman. Inequalities for linear combinations of gamma random variables. Canadian Journal of Statistic, 15: , [7] P. J. Boland, E. El-Neweihi, and F. Proschan. Schur properties of convolutions of exponential and geometric random variables. Journal of Multivariate Analysis, 481: , 1994.

9 9 [8] J. Bon and E. Pãltãnea. Ordering properties of convolutions of exponential random variables. Lifetime Data Analysis, 52: , [9] P. Diaconis and M. D. Perlman. Bounds for tail probabilities of weighted sums of independent gamma random variables. Lecture Notes-Monograph Series, pages , 199. [1] K. van den Doel and U. Ascher. The chaotic nature of faster gradient descent methods. J. Scient. Comput., 48, 211. DOI: 1.17/s [11] Yonina C Eldar and Gitta Kutyniok. Compressed sensing: theory and applications. Cambridge University Press, 212. [12] E. Furman and Z. Landsman. Tail variance premium with applications for elliptical portfolio of risks. Astin Bulletin, 362:433, 26. [13] G. H. Golub, M. Heath, and G. Wahba. Generalized cross validation as a method for choosing a good ridge parameter. Technometrics, 21: , [14] E. Haber, M. Chung, and F. Herrmann. An effective method for parameter estimation with PDE constraints with multiple right-hand sides. SIAM J. Optimization, 22: , 212. [15] M.F. Hutchinson. A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines. J. Comm. Stat. Simul., 19:433 45, 199. [16] S. Kochar and M. Xu. On the right spread order of convolutions of heterogeneous exponential random variables. Journal of Multivariate Analysis, 111: , 21. [17] S. Kochar and M. Xu. The tail behavior of the convolutions of gamma random variables. Journal of Statistical Planning and Inference, 1411: , 211. [18] S. Kochar and M. Xu. Some unified results on comparing linear combinations of independent gamma random variables. Probability in the Engineering and Informational Sciences, 263:393 44, 212. [19] S. Lihong and Z. Xinsheng. Stochastic comparisons of order statistics from gamma distributions. Journal of Multivariate Analysis, 931: , 25. [2] A.W. Marshall and I. Olkin. Inequalities: Theory of majorization and its applications. Academic, New York, [21] Milan Merkle and Ljiljana Petrović. On schur-convexity of some distribution functions. Publications de l Institut Mathématique, 5676: , 1994.

10 1 [22] J. E. Pečarić, F. Proschan, and Y.L. Tong. Convex Functions, Partial Orderings, and Statistical Applications, volume 187 of Mathematics in Science and Engineering. Academic Press, [23] F. Proschan. Peakedness of distributions of convex combinations. The Annals of Mathematical Statistics, 36: , [24] F. Roosta-Khorasani and U. Ascher. Improved bounds on sample size for implicit matrix trace estimators. Foundations of Computational Mathematics, 214. DOI: 1.17/s [25] F. Roosta-Khorasani, G. Székely, and U. Ascher. Assessing stochastic algorithms for large scale nonlinear least squares problems using extremal probabilities of linear combinations of gamma random variables. SIAM/ASA Journal on Uncertainty Quantification, 31:61 9, 215. DOI: / X. [26] F. Roosta-Khorasani, K. van den Doel, and U. Ascher. Data completion and stochastic algorithms for PDE inversion problems with many measurements. Electronic Transactions on Numerical Analysis, 42: , 214. [27] F. Roosta-Khorasani, K. van den Doel, and U. Ascher. Stochastic algorithms for inverse problems involving PDEs and many measurements. SIAM J. Scientific Computing, 365:S3 S22, 214. [28] A. Shapiro, D. Dentcheva, and D. Ruszczynski. Lectures on Stochastic Programming: Modeling and Theory. Piladelphia: SIAM, 29. [29] G. J. Székely and N. K. Bakirov. Extremal probabilities for gaussian quadratic forms. Probab. Theory Related Fields, 126:184 22, 23. [3] T. van Leeuwen, S. Aravkin, and F. Herrmann. Seismic waveform inversion by stochastic optimization. Hindawi Intl. J. Geophysics, 211:doi:1.1155/211/68941, 212. [31] J. Young and D. Ridzal. An application of random projection to parameter estimation in partial differential equations. SIAM J. Scient. Comput., 34:A2344 A2365, 212. [32] Yaming Yu et al. Some stochastic inequalities for weighted sums. Bernoulli, 173: , 211. [33] P. Zhao. Some new results on convolutions of heterogeneous gamma random variables. Journal of Multivariate Analysis, 125: , 211. [34] P. Zhao and N. Balakrishnan. Mean residual life order of convolutions of heterogeneous exponential random variables. Journal of Multivariate Analysis, 18: , 29.

11 11 A Proof In what follows X Gamma, β denotes a gamma r.v parametrized by shape > and rate β >, f X and F X stand, respectively, for the probability density function PDF and cumulative distribution function CDF of a r.v X. Bold face letters denote vectors and the vector components are denoted as v = v 1, v 2,..., v n. For the proof of Theorem 1, we need to make use of the following additional results. The proof of Theorem 4 is identical to the proof of [29, Theorem 4]. Note that [29, Theorem 4] has been stated in terms of convolution of chi-squared r.v s but the proof, there, has been given for the more general case of arbitrary gamma r.v s. The details of the proofs for Lemmas 5 and 7 can be found in [25]. Lemma 6 has been stated in [25] but the proof is omitted there, and as such, we give the details of its proof here for completeness. Theorem 4 is essential in proving our results and it states that an arbitrary convolution of heterogeneous gamma random variables not necessarily with a common shape or a common rate has an unique mode. Recall that a PDF, fx, is called unimodal if there exists a unique x = a such that fx is non-decreasing for x < a and fx is non-increasing for x > a. The point a is called the unique mode of fx. Theorem 4 Let X i Gamma i, β i, i = 1, 2,..., n, be independent r.v s, where i, β i > i. The PDF of Y n := n λ ix i is unimodal where λ i i. Proof The proof is identical to the proof of [29, Theorem 4]; hence omitted. Lemma 5 [25, Lemma B.1] Let X i Gamma i, β i, i = 1, 2,..., n, be independent r.v s, where i, β i > i. Define Y n := n λ ix i for λ i >, i and ρ j := j i. Then for the PDF of Y n, f Yn, we have i f Yn >, x >, ii f Yn is analytic on R + = {x x > }, iii f k Y n =, if k < ρ n 1, where f k Y n denotes the k th derivative of f Yn. Lemma 6 [25, Lemma B.2] Let X i Gamma i,, i = 1, 2,..., n, be independent r.v s, where i > i and >. Also let ψ Gamma1, be another r.v independent of all X i s. If n i > 1, then the mode, xλ, of the r.v W λ = Y +λψ is strictly increasing in λ >, where Y = n λ ix i with λ i >, i. Proof As the proof of [25, Lemma B.2] is omitted there, we give a detailed proof here for completeness.

12 12 By Lemma 5, xλ > for λ. By the unimodality of W λ, for any λ > λ >, it is enough to show that J λ, xλ [ ] d 2 := Pr W λ x >. 7 dx2 x= xλ Note that J λ, xλ = and since n i > 1, by Lemma 5iii, f Y =. So we have J λ, xλ [ d x = f Y x z ] dx λ e λ z dz Therefore, xλ = = [ x xλ d dx f Y x z f Y z λ e λ ] λ z dz λ e xλ z dz. x= xλ x= xλ f Y ze z λ dz = λ e xλ λ J λ, xλ =. Thus for λ > λ >, we have λ e xλ λ J λ, xλ = = = = xλ xλ xλ xλ f Y ze z λ dz f Y ze z λ f Y ze z λ e x 1 λ 1 λ dz f Y z e z λ f Y z e z λ where x > is the mode of r.v Y and Φ z, x 1 := z x 1. λ λ z + x 1 e λ λ 1 λ dz z e z, x λ +Φ dz, Now if z < x then Φ z, x < and f Y z > so we get J λ, xλ >. Similarly if z > x then Φ z, x > and f Y z < and again we have J λ, xλ >. Lemma 7 [25, Lemma B.3] For some 2 1 >, let ξ 1 Gamma1+ 1, 1 and ξ 2 Gamma1 + 2, 2 be independent gamma r.v s. Also let x = xλ denote the mode of the r.v ξλ = λξ λξ 2 for λ 1. Then 1 xλ / 2 1 2, λ 1, with x = x1 = 1 and, in case of i = j =, x1/2 = / 2, otherwise the inequalities are strict.

13 13 A.1 Proof of Theorem 1 We prove the theorem for the case where = β and s = 1. The general case follows from the scaling properties of gamma distribution. We first consider the case where n 3. If λ µ, then there exists a finite number, r, of vectors η i, i = 1, 2,... r, such that λ = η 1 η 2... η r 1 η r = µ, and η i and η i+1 differ in two coordinates only, i = 1, 2,..., r 1, see [22, 12.5.a]. Thus we may, without loss of generality assume that λ, and µ differ only in two coordinates, and in fact, assume that for some 1 j < k n, we have For t [, 1], define λ j, λ k µ j, µ k and λ i = µ i for i {1,..., n}\{j, k}. 8 ν i t := tλ i + 1 tµ i, i = j, k, ν i t := λ i, i j, k, Y t := ν i tx i. 9 It suffices to show that the CDF of Y t, in t [, 1], is non-increasing for x > 2 + 1/2 and non-decreasing for x < 1/. Now we take the Laplace transform of F Y t as Jt, z := L[F Y t ]z = = 1 z = 1 z e zx F Y t xdx = 1 L[Y t]z, z where L[Y t]z is the Laplace transform of Y t as L[Y t]z = n Differentiating with respect to t yields F Y t xd e zx e zx df Y t x 1 + ν itz, for z C, Rez > min 1 i n ν i t. J t t, z = Jt, z t lnj = Jt, z µ i λ i z. i=j,k 1 + ν itz

14 14 We take the inverse transform to get t F Y tx = µ i λ i x PrY t + ν itψ i x i=j,k = i=j,kµ i λ i f Y t+νi tψ i x = µ j λ j [f Y t+νj tψ j x f Y t+νk tψ k x], where ψ j, ψ k Gamma1, are i.i.d gamma r.v s which are also independent of all X i s. By 8, we must have that λ j µ j, so it suffices to show that f Y t+νj tψ j x f Y t+νk tψ k x for x > 2 + 1/2 and f Y t+νj tψ j x f Y t+νk tψ k x for x < 1/. On the other hand, using the Laplace transform and inverting it again, one can show the following identity for any integer k 1 and reals a, b d k 1 dx f k 1 X+aψ 1 x dk 1 dx f k 1 X+bψ 2 x = 1 dk b a dx f k X+aψ 1 +bψ 2 x, 1 where X is an arbitrary continuous positive r.v and ψ i Gamma1,, i = 1, 2, are i.i.d gamma r.v s which are also independent of X. As such we have f Y t+νj tψ j x f Y t+νk tψ k x = 1 ν kt ν j t x f Y t+ν j tψ j +ν k tψ k x. For given 1 j < k n, consider the r.v Y t + ν j tψ j + ν k tψ k. By 9, we get that ν j t ν k t for t [, 1], hence, it is only left to show that, for any t [, 1], the mode of this r.v, under the conditions ν 1 t... ν n t, ν i t = 1, falls between 1/ and 2 + 1/2. By Lemma 6, the mode of r.v Y t + ν 1 tψ 1 +ν 2 tψ 2 is greater than that of Y t+ν j tψ j +ν k tψ k for 1 < j < k, and also the mode of Y t+ν n 1 tψ n 1 +ν n tψ n is smaller than that of Y t+ν j tψ j +ν k tψ k for j < k < n. Hence, we only need to show that for the mode of the r.v Y 1 t := Y t + ν 1 tψ 1 + ν 2 tψ 2 11a denoted by x 1 t, we have x 1 t 2 + 1/2 and for the mode of the r.v Y n t := Y t + ν n 1 tψ n 1 + ν n tψ n 11b denoted by x n t, we have x n t 1/. In what follows, we fix any t [, 1] and, for notational simplicity, denote Y 1 t, x 1 t, Y n t and x n t by Y 1, x 1, Y n and x n, respectively.

15 15 We first prove the case for Y 1. At any mode of Y 1, we have 1 [ ] x f Y 1 x, ν =, x 1 ν,ν [ ] 2 x f 2 Y 1 x, ν <, hence implicit function theorem yields x 1 ν,ν 2 f Y1 x 1 ν ν = 1 x 1 x 1 ν, ν ν 2 f 1 Y1 x x 2 1 ν, ν Let ν be where the global maximum of x 1 ν, denoted by x 1, occurs. Thus by the necessary condition of maximality, we must have [ ] 2 ν 1 x f Y 1 x, ν =. x 1,ν Suppose where the maximum occurs, i.e., ν, there is a nonzero coefficient, ν 3, such that < ν 3 < ν 2 ν 1. The case of ν 3 = will be dealt with at the end of the proof. Fixing all other coefficients, we vary ν 2 and ν 3 under the condition ν 2 + ν 3 = const. That is, we take the directional derivative in the direction δν =, 1, 1,..., for the n dimensional vector ν = ν 1, ν 2, ν 3,..., ν n. In other words, we consider the change in the direction of ν + γδν. Using Laplace transform, we get d dγ L[Y 1]z = L[Y 1 ]z d dγ ln [L[Y 1]z] = L[Y 1 ]z d 1 + ln1 + ν 2z dγ 1 + z = L[Y 1 ]z ν 2z + i= i 2 z 1 + ν 3z 1 = zl[y 1 ]z 1 + ν 3z ν 2z ν 2z ν 2 ν 3 z = zl[y 1 ]z 1 + ν 3z 1 + ν 2z ν 2z. Now inverting the above and differentiating w.r.t x, we get 2 γ x f Y 1 x, ν = ν 2 ν Since mode is unique, thus f must be strictly concave at the mode ln1 + ν iz ln1 + ν 1z 2 x f 3 Ỹ 23 x, ν 1 x f 2 Y 1 +ν 2 ξ 2 x, ν, 13

16 16 where Ỹ23 = Y 1 + ν 2 ξ 2 + ν 3 ξ 3 with ξ 2, ξ 3 Gamma1, being i.i.d r.v s, independent of all others appearing before. So at the maximum, x 1, we must have [ ] 2 γ x f Y 1 x, ν =. 14 x 1,ν Now consider a slight perturbation of δν =, 1, 1,..., as δνε =, 1, 1 + ε,..., for some ε >. Computations similar as above yields 2 γ x f Y 1 x, ν = ν 2 ν x f 3 Ỹ 23 x, ν 1 x f 2 Y 1 +ν 2 ξ 2 x, ν + 2 ε x f 2 Y 1 +ν 3 ξ 3 x, ν. So at the maximum, x 1, using 14, we must have [ ] [ ] 2 2 γ x f Y 1 x, ν = ε x 1,ν x f 2 Y 1 +ν 3 ξ 3 x, ν = ε [ ] ν 3 x f Y 1 +ν 3 ξ 3 x, ν x 1,ν x 1,ν This followed from 1 with k = 2, a = and b = ν 3 and Lemma 6, as well as noting that [ f x Y 1 x, ν ] =. Using 12, the previous relation implies that the x 1,ν mode must increase if a sufficiently small step is taken along the perturbed direction δνε =, 1, 1 + ε,...,. In other words, for ε 1 > small enough, for the mode of r.v Y ε,ε 1 1 := ν 1 X 1 +ν 2 ε 1 X 2 + ν 3 +ε 1 1+ε X 3 + denoted by x ε,ε 1 1, we must have that <. ν i X i +ν 1 ψ 1 +ν 2 ε 1 ψ 2, 15 i=4 x 1 < x ε,ε It is clear that ε 1 depends on ε and consequently, as ε gets smaller, ε 1 might get smaller as well. However, ε 1 ε oε and ε 1 ε oε 1, where o denotes the little o. In other words, consider decreasing sequences of ε n n=1 and ε n 1 n=1. Since lim n ε n 1ε n /ε n = lim n ε n 1ε n /ε n 1 =, it follows that, as ε gets smaller, ε 1 ε term in 15 vanishes at a faster rate than either of ε or ε 1. Hence using 15 and 16, as well as the continuity of x 1, we can consider a r.v Y ε 1 := ν 1X 1 + ν 2 εx 2 + ν 3 + εx 3 + ν i X i + ν 1 ψ 1 + ν 2 εψ 2, i=4

17 17 which has the same form as Y 1 but whose mode, denoted by x ε 1, satisfies x 1 < x ε 1, which contradicts the maximality of x 1. This means that, if ν 3, then the maximum must occurs at ν with ν 3 = ν 2 ν 1. Similar arguments show that at the maximum, we must have ν n =... = ν 3 = ν 2 ν 1. But by 13 and the fact that [ ] 2 x f 2 Y 1 +ν 2 ξ 2 x, ν x 1,ν = [ ] ν 2 x f Y 1 +ν 2 ξ 2 x, ν x 1,ν we must have that ν 3 ν 2. As such, the only possibility is ν n =... = ν 3 =, i.e., the maximum mod of Y 1 coincides with that of r.v ζ = νζ νζ 2 where ζ 1, ζ 2 Gamma1 +,. So now lemma 7 gives x Now consider the case for Y n. Suppose where the minimum occurs, i.e., ν, we have, ν n 1 > and ν n 2 > ν n 1. Now by the same reasoning as in the case of Y 1, we can consider a r.v n 3 Yn ε := ν i X i ν 1 X 1 +ν n 2 εx n 2 +ν n 1 +εx n 1 +ν n X n +ν n 1 +εψ n 1 +ν n ψ n, <, which has the same form as Y n but whose mode, denoted by x ε n, satisfies x ε n < x n, which contradicts the minimality of x n. This means that, if ν n 1, then the minimum must occurs at ν with ν n 2 = ν n 1. But again, as in the end of the proof for the case of Y 1, we must have that ν n 2 ν n 1. As such the only possibility is ν n 1 = which would also mean that ν n =, although this case can also be independently established with the same reasoning as above. Hence, the minimum mode of Y n must coincide with that of the r.v n 2 ν ix i. Again, appealing to the same line of reasoning, at the minimum mode of n 2 ν ix i, we must either have ν j = or ν j 1 = ν j for any 1 < j n 2. Hence, for n 3, we get x n = min x j = 1 1 j n 2, where x j denotes the mode of the r.v Gamma j, j. The case of n = 2, amounts to studying the mode of r.v ζ = νζ νζ 2, denoted by xζ, where ζ 1, ζ 2 Gamma1 +,. Direct application of Lemma 7 yields 1 xζ 2 + 1/2. Theorem 1 is proved.

18 18 A.2 Proof of Corollary 2 The proof goes along the same line as that of Theorem 1. For t [, 1], we again define where ν i t := tλ i + 1 tµ i, i = j, k, ν i t := λ i, i j, k, Y t := ν i tx i. ν 1 t... ν n t, t [, 1], ν i t = s µ + ts λ s µ. Computations identical to the proof of Theorem 1 give x 1t s µ + ts λ s µ , x nt s µ + ts λ s µ 1, where x 1t and x nt are, respectively, the maximum and minimum mode of r.v s Y 1 t and Y n t which are defined similarly as in 11. Now for t [, 1], we get x 1 = max t [,1] which give the desired results. A.3 Proof of Corollary 3 Define where x 1t 2 + 1s λ, 2 x n = min t [,1] x nt 1s µ, P n λ n ;, β, x := Pr λ i X i < x, λ n := λ 1, λ 2,..., λ n. By the continuity from above, it is clear that P λ;, β, x = lim n P n λ n ;, β, x.

19 19 Now since µ n w λ n, Corollary 2 yields P n µ;, β, x P n λ;, β, x, x > n 2β P n µ;, β, x P n λ;, β, x, x < 1 n Taking the limit as n gives the desired result. β λ i µ i,, > 1.