ON COMBINING CORRELATED ESTIMATORS OF THE COMMON MEAN OF A MULTIVARIATE NORMAL DISTRIBUTION

Transcription

1 ON COMBINING CORRELATED ESTIMATORS OF THE COMMON MEAN OF A MULTIVARIATE NORMAL DISTRIBUTION K. KRISHNAMOORTHY 1 and YONG LU Department of Mathematics, University of Louisiana at Lafayette Lafayette, LA , USA The inferential procedures based on an optimal combination of correlated estimators of the common mean of a multivariate normal distribution are considered. Exact properties of the conditional and unconditional confidence intervals due to Halperin 1961) are numerically evaluated. Our numerical studies show that the conditional confidence interval is slightly shorter than the unconditional confidence interval. A condition under which the conditional approach is advantageous over the best of the t procedures based on individual components is discussed. The methods are illustrated using an example. Key words: Concomitant variable; expected length; maximum likelihood estimator; noncentral t distribution; multiple correlation coefficient; power - 1 Corresponding Author: krishna@louisiana.edu Phone: ; Fax:

2 1. INTRODUCTION The problem of combining independent estimators for the common mean of several normal populations is well-known and has been well addressed in the literature. An important result in the common mean problem is due to Graybill and Deal 1956) who first showed, for the two-sample case, that the weighted average of the sample means with weights inversely proportional to their variances has smaller variance than either sample mean provided the sample sizes are greater than 10. Since then many authors improved and extended this result to the case of more than two populations, and developed several methods for hypothesis testing and interval estimation for the common mean. For a good exposition of the work in this area, we refer to Cohen and Sckrowitz 1984), Zhou and Mathew 1993), Yu, Sun and Sinha 1999) and Krishnamoorthy and Lu 003) and the references therein. However, the results on combining the correlated estimators in the normal case are very limited. Halperin 1961) seems to be the first paper addressed this problem. Halperin pointed out that the problem of estimating the common mean of a multivariate normal population arises when several alike neutron transportation problems are considered. Halperin derived the maximum likelihood estimator MLE) and developed two interval estimates for the common mean of a multivariate normal population. We shall now describe the setup of the problem as given in Halperin 1961). Let U N p eµ, Σ), where e denotes the vector of ones. Let Ū and S u denote respectively the mean and covariance matrix based a sample of n observations from N p eµ, Σ). The maximum likelihood estimator of µ due to Halperin 1961) is given by ˆµ = e Su 1Ū e Su 1 e. 1) If Σ is known, then the best linear unbiased estimator BLUE) of µ is given by e Σ 1 Ū/e Σ 1 e) and it has variance ne Σ 1 e) 1. If Σ is unknown, replacing Σ by its estimate S u, we can get the MLE. The variance of the MLE is given by Varˆµ) = 1 + p 1 n p 1 which approaches the variance of the BLUE as n. ) 1 ne Σ 1 e, ) The form of the MLE in 1) is not conducive to develop a confidence interval for µ. To derive the distribution of the MLE, Halperin suggested to use the following transformation. Let A = a ij ) be a p p matrix such that a i1 = 1 for i = 1,..., p, a ii = 1 for i =,..., p, and a ij = 0 elsewhere. Then AU = y, x 1,..., x p 1 ) = y, X ) follows a p-variate normal distribution with mean vector µ, 0,..., 0) and covariance matrix ) AΣA σyy σ Xy = σ Xy Σ XX p p. 3) Thus, estimation of the common mean µ is equivalent to estimation of the mean of y given that the mean of X = 0 p 1. Let y 1, X 1 ),..., y n, X n ) be independent observations on y, X). Define ȳ, X ) = 1 n y i, X n i) 4) i=1

3 3 and wyy W Xy W p p = W Xy W XX ) ni=1 y = i ȳ) ni=1 y i ȳ)x i X) ) ni=1 y i ȳ)x i X) ni=1 X i X)X i X), 5) so that W XX is a p 1) p 1) matrix. Let a = a 1,..., a p 1 ) be a vector of real numbers, and β = Σ 1 XX σ Xy. Consider the class of estimators of the form ȳa) = ȳ p 1 i=1 a i x i. It can be easily shown that Varȳa)) is minimized when a = β. Thus, if β is known, then ȳβ) is the best linear unbiased estimator of µ. If β is unknown, then replacing it by b = WXX 1 W Xy we get ˆµ = ȳ b X. 6) This is an alternative form of the MLE in 1). The expression for the variance of the MLE can be written as Varˆµ) = 1 + p 1 ) σyy.x, for n > p + 1, 7) n p 1 n where σ yy.x = σ yy 1 ρ y.x ), and ρ y.x = σ Xy Σ 1 XX σ Xy)/σ yy is the squared multiple correlation coefficient between y and X. It follows from 17) that the variance of the MLE is smaller than that of ȳ = ū 1 if and only if ρ y.x > p 1 n. Recall that the transformation we used is u 1, u 1 u,..., u 1 u p ) = y, x 1,..., x p 1 ). If we let u, u u 1,..., u u p ) = y, x 1,..., x p 1 ), then the MLE has smaller variance than ū if and only if ρ u 1.u 1 u ),...,u 1 u p ) > p 1)/n ). Proceeding this way, we see that the MLE has smaller variance than the min{varū 1 ),..., Varū p )} if and only if min{ρ u 1.u 1 u ),...,u 1 u p),..., ρ u p.u p u 1 ),...,u p u p 1 ) } > p 1 n. 8) Krishnamoorthy and Rohatgi 1990) showed that ˆµ can be improved using the fact that the mean of X is known to be zero. In particular, they suggested using W X0 = n i=1 X i X i to estimate β. This leads to the estimator ˆµ 1 = ȳ b X, 0 where b 0 = W X0 W Xy and W Xy is defined in 5). Krishnamoorthy and Rohatgi 1990) showed that ˆµ 1 has smaller variance than ˆµ over a wide range of parameter space. The problem of estimating the mean of y given that the mean of X is 0 p 1 1 has been considered by Berry 1987), Tan and Gleser 1993), and Jin and Berry 1993). These authors refer to the vector X as concomitant control vector for estimating the mean µ of y. This problem is equivalent to the common mean problem with the transformed variables. However, the main interest in the common mean problem is to develop a better inferential procedure, based on a combination of the correlated estimators, than the best of the t procedures based on individual estimators whereas there is no such interest in the problem of estimating µ with a concomitant control vector. In this article, we are mainly interested in comparing three confidence intervals, including the t-interval based on the marginal distribution of y, that are given in Halperin 1961). In the following section, we describe the conditional interval and the unconditional interval due

4 4 to Halperin, and present expressions for their expected lengths. The expected lengths are compared numerically. Our comparison studies in Section 3 shows that the conditional intervals are either slightly shorter than or almost close to the unconditional intervals for all the cases considered. We also discuss a condition under which the expected length of the conditional confidence interval is shorter than the best of the t intervals based on individual means. For the sake completeness, we also present the test based on the conditional approach, and its power function. The methods are illustrated using a simulated data set.. INTERVAL ESTIMATION AND EXPECTED LENGTHS In the following lemma, we present some basic distributional results related to the statistics defined in the previous section. These results can be found, for example, in Muirhead 198, Chapter 3). Lemma.1. i) The conditional distribution of b = W Xy W 1 XX given X 1,..., X n ) is N p 1 β, σ yy.x W 1 XX ). ii) n iii) V = XΣ 1 XX X = n X Σ 1 )Σ 1 X n) = Z Z χ p 1. XΣ 1 XX X XW 1 XX X χ n p+1 independently of X or Z) iv) Q = n X WXX 1 X = Z Z/V p 1 n p+1 F p 1,n p+1, where F a,b denotes the F random variable with the numerator df = a and the denominator df = b. vi) The sample conditional variance of y given X is defined as ˆσ yy.x = wyy W Xy W 1 XX W Xy n p and is distributed as σ yy.x n p χ n p independently of Q. We shall now present the confidence intervals that will be considered for comparison..1 The t-interval The usual t-interval based on the marginal distribution of y is given by ȳ ± t n 1,1 α/ syy n, 9) where s yy is the sample variance of y and t m,α denotes the αth quantile of the Student s t distribution. The expected length of the t-interval is given by EL 1 = t n 1,1 α/ E ) syy n where Γ.) denotes the gamma function. Γn/) = t n 1,1 α/ n Γn 1)/ σyy n 1, 10)

5 5. The Conditional Interval We shall now describe the conditional confidence interval due to Halperin 1961). Using the results of Lemma.1, it can be readily verified that the conditional distribution of ˆµ given X 1,..., X n is normal with mean µ and variance σ yy.x 1 + Q), where Q = n X WXX 1 X defined in Lemma.1v). We write ˆµ X 1,..., X n ) N µ, σ yy.x 1 + Q)/n). 11) Notice that n p)ˆσ yy.x /σ yy.x χ n p. Using this result, we see that, conditionally given Q, the pivotal quantity nˆµ µ) t n p. 1) ˆσ yy.x 1 + Q) This leads to the conditional confidence interval ˆµ ± t n p,1 α/ 1 + Q) 1 ˆσ yy.x n. 13) It follows from Lemma.1iv) that 1 + Q) is distributed as U 1, where U is a beta random variable with parameters n p+1)/ and p 1)/. Using this result and the fact that X WXX 1 X and ˆσ yy.x are independent, it is easy to see that the expected length of the conditional confidence interval in 13) is EL = t n p,1 α/ n Γ ) n ) Γ n 1 σyy 1 ρ y.x ) n p ) 1. 14) It should be noted that the formula for EL given in Halperin 1961) is incorrect..3 The Unconditional Confidence Interval It follows from 1) and Lemma.1iv) that nȳb) µ0 ) T = t n p 1 + p 1 ) 1 ˆσyy.X n p + 1 F p 1,n p+1. 15) The percentiles of T can be used to form a 1 α confidence interval for µ. Using some standard methods, it can be shown that the 1 - α)th quantile k of T is the solution of the equation Γn/) Γp 1)/)Γn p + 1)/) 1 0 G k ) 1 x; n p x p 1)/ 1 1 x) n p+1)/ 1 dx = 1 α, where G.; m) denotes the Student s t cdf with df = m. To get 16), we used the fact that F a,b is distributed as bu/a1 U)), where U is a betaa/, b/) random variable. Noting that the Student s t distribution is symmetric about zero, it follows from 16) that the distribution of T is also symmetric about zero. Let T α denote the αth quantile of T. Then, the unconditional 1 α confidence interval for µ is given by ˆµ ± T 1 α/ ˆσ yy.x n. 17) 16)

6 6 The expected length of the unconditional confidence interval is given by EL 3 = T 1 α/ E ˆσ yy.x n = T 1 α/ n Γ n p+1 ) Γ n p ) σyy 1 ρ y.x ) n p ) 1. 18) Remark 1. Using 16), we computed the values of T 1 α/ when α = 0.05 and 0.1, p =, 3, 4 and 5, and values of n ranging from 6 to These critical values are presented in Table I. We also found that the distribution of T in 15) can be approximated by ct n p, where c = n )/n p 1). The constant c was obtained by solving the equation Ec t n p) = ET ).. Using this approximation, we have T 1 α/ = t n p,1 α/ n )/n p 1). This approximation is satisfactory as long as n p COMPARISON OF EXPECTED LENGTHS It is clear from the expressions of EL 1, EL and EL 3, that the ratios EL /EL 1 and EL 3 /EL 1 depend on the parameter space only through ρ y.x. Using this fact, direct comparison between EL and EL 1 shows that the expected length of the conditional confidence interval is shorter than the expected length of the usual t interval based on y observations alone if and only if ) n ) ρ tn 1,1 α/ p y.x > 1. 19) n 1 t n p,1 α/ The above inequality is different from the one given in Halperin 1961, p.41), because, as we already pointed out, Halperin s formula for the expected length of the conditional interval is incorrect. For fixed p, the right-hand side of 19) approaches zero as n. This implies that, for large n, EL is smaller than EL 1 for all practically meaningful values of ρ y.x. However, this does not mean that EL is smaller than the expected length of the shortest of the individual t intervals. The above condition merely implies that EL is smaller than the expected length of the t-interval based on ū 1 = ȳ. For EL to be shorter than the t-interval based on ū, we should have ρ u.u u 1 ),...,u u p) > 1 tn 1,1 α/ t n p,1 α/ ) n ) p, n 1 where ρ u.u u 1 ),...,u u p) is the squared multiple correlation coefficient between U and U U 1 ),..., U U p )). Proceeding this way, we see that EL is shorter than the shortest of the t-intervals if and only if ) n ) min{ρ u 1.u 1 u ),...,u 1 u p ),..., ρ u p.u p u 1 ),...,u p u p 1 ) } > 1 tn 1,1 α/ p. 0) t n p,1 α/ n 1 Comparison between EL and EL 3 shows that the ratio EL /EL 3 > 1 if and only if Γ ) ) n Γ n p+1 t n p,1 α/ ) < T Γ n 1 1 α/ Γ n p ). 1)

7 7 TABLE I Critical points T 1 α/ for constructing unconditional confidence intervals n p = p = 3 p = 4 p = 5 α = 0.05 α = 0.10 α = 0.05 α = 0.10 α = 0.05 α = 0.10 α = 0.05 α = We numerically evaluated EL /EL 1 and EL 3 /EL 1 and presented them in Table II. It is clear from the table values that EL is in general either very close to EL 3 or smaller than EL 3, and the difference between them decreases as n increases. Thus, we see that the conditional confidence interval is not only simple to construct but also narrower than the unconditional confidence interval. Furthermore, if n p is small and ρ y.x is small, then the usual t interval is shorter than both conditional and unconditional intervals see the values in Table II when n = 6, p = 3 and n = 6, p = 5). Thus, the conditional combined method is preferable to the best of the t procedures only when condition 0) holds and/or n p is moderately large.

8 8 TABLE II Ratios of the Expected Lengths of 95% Confidence Intervals; p = n = 6 n = 10 n = 15 n = 0 n = 30 ρ y.x EL EL 3 EL EL 3 EL EL 3 EL EL 3 EL EL 3 EL 1 EL 1 EL 1 EL 1 EL 1 EL 1 EL 1 EL 1 EL 1 EL ρ L TABLE II continued. p = 3 n = 6 n = 10 n = 0 n = 30 n = 40 ρ y.x EL EL 3 EL EL 3 EL EL 3 EL EL 3 EL EL 3 EL 1 EL 1 EL 1 EL 1 EL 1 EL 1 EL 1 EL 1 EL 1 EL ρ L Note: ρ L is the lower bound given in the right-hand side of 0); EL < EL 1 when 0) holds

9 9 TABLE II continued. p = 5 n = 6 n = 10 n = 0 n = 30 n = 40 ρ y.x EL EL 3 EL EL 3 EL EL 3 EL EL 3 EL EL 3 EL 1 EL 1 EL 1 EL 1 EL 1 EL 1 EL 1 EL 1 EL 1 EL ρ L Note: ρ L is the lower bound given in the right-hand side of 0); EL < EL 1 when 0) holds 4. POWER FUNCTION We observed in the preceding section that the conditional method performs better than the unconditional method, and hence we consider only the power function of the conditional test based on 1). Consider the hypotheses H 0 : µ µ 0 vs. H a : µ > µ 0. The conditional non-null distribution given T ) is the noncentral t distribution with df = n p 1 and the noncentrality parameter nµ µ0 ) δq) =, ) σyy.x 1 + Q where µ is the true value and µ 0 is the specified value of the mean. The unconditional power of a right-tail test can be expressed as E Q [P t n p 1 δq)) > t n p 1,1 α )]. 3) Again, using the fact that 1+Q is distributed as the reciprocal of a betan p)/, p/) random variable, the power can be computed using the numerical integration 1 Γn/) Γp/)Γn p)/) 1 0 ) G c 1 ; n p 1, δu 1 ) u p/ 1 1 u) n p)/ 1 du, 4) where c 1 = t n p 1,1 α and Gx; m, d) denotes the cdf of a noncentral t random variable with df = m and the noncentrality parameter d. Although it is not difficult to compute 0), a simple approximate power expression can be obtained from 19), and is given by P t n p 1 δeq)) > t n p 1,1 α ). 5)

10 10 Noting that EQ) = n )/n p ), for a given level of significance α, p and η = µ µ 0 )/σ yy.x, an approximate sample size n that is required to attain a power of 1 β satisfies P t n p 1 δ 1 ) > t n p 1,1 α ) = 1 β, 6) where δ 1 = ) 1 nµ µ0 ) n p. 7) σyy.x n In order to understand the validity of the approximation, we computed the exact power using 4) and the approximate power based on 6) for various values of n, η = µ µ 0 )/σ yy.x and p =, 3 and 4. These powers are presented TABLE II. We see from the table values that the approximate powers are close to the exact powers provided n is moderately large in comparison to p. Our extensive numerical studies for various values of p not reported here) showed that the approximation is very satisfactory for values of n 5p. An advantage of this approximation is that it only involves the computation of the noncentral cdf with fixed noncentrality parameter when η, n and p are given), and so the power computation can be carried out using freely available PC calculators such as StatCalc or online calculator available at TABLE III Exact powers 4) and approximate powers 5) of the conditional test when α = 0.05 p = n = 6 n = 8 n = 1 n = 16 n = 0 η = µ µ 0 σyy.x Exact Appr. Exact Appr. Exact Appr. Exact Appr. Exact Appr

11 11 TABLE III continued. p = 3 n = 8 n = 1 n = 16 n = 0 n = 4 η = µ µ 0 σyy.x Exact Appr. Exact Appr. Exact Appr. Exact Appr. Exact Appr TABLE III continued. p = 4 n = 8 n = 1 n = 16 n = 0 n = 4 η = µ µ 0 σyy.x Exact Appr. Exact Appr. Exact Appr. Exact Appr. Exact Appr AN EXAMPLE To illustrate the methods of this paper, we generated a sample of 0 observations from Nµ, Σ), where µ = 4 and Σ = The data points are given in Table IV. The summary statistics are ) wyy W ȳ, x 1, x ) = , 4.160, 4.91), yy.x = W yy.x W XX ,

12 W 1 XX = E E E ) 1, ˆµ = , ˆσ yy.x = and ρ y.x = The standard deviations are s u1 = 1.747, s u = and s u3 = The critical points t 19,0.975 =.0930 and T u,0.975 =.418. Using these statistics, we computed the following confidence intervals for µ. TABLE IV Simulated data; n = 0, p = 3 u 1 u u 3 y = u 1 x 1 = u 1 u x = u 1 u The 95% t-intervals a) ū 1 ± t 19,0.975 s u1 n = 4.91 ± ; ˆρ u 1.u 1 u,u 1 u 3 ) = b) ū ± t 19,0.975 s u n = ± ; ˆρ u.u u 1,u u 3 ) = c) ū 3 ± t 19,0.975 s u3 n = ± ; ˆρ u 3.u 3 u,u 3 u 1 ) = The conditional interval in 13) = ± The unconditional interval in 17) = ± The interval b) is the shortest among all the intervals. We also note that, for the conditional interval to be the shortest, we must have min{ρ u 1.u 1 u ),u 1 u ), ρ u 1.u 1 u ),u 1 u ), ρ u 1.u 1 u ),u 1 u )} > )

13 13 see Table II, n = 0, p = 3). Since the minimum of the sample squared multiple correlation coefficients is , we do not have any evidence in favor of 8). Therefore, as already observed, the conditional approach did not produce the shortest interval. We also see that, among all the point estimates, the MLE is very close to the true mean CONCLUDING REMARKS We observed from the preceding sections that the unconditional approach is not only simple to use but also better than the unconditional method for constructing confidence interval for the common mean µ. Furthermore, if the sample size is sufficiently large, then the conditional approach may yield better results than the ones based on the individual t procedures. For a fixed p and α = 0.05, we computed the least value of n for which ρ L = 1 tn 1,1 α/ ) ) n p t n p,1 α/ n 1 > Based on a linear fit of these pairs of n, p), we found that ρ L > 0.05 for any n > 0p 15. This implies that n must be at least 0p 15 for the conditional approach offers improvement over the best of the t procedures for any ρ y.x > For moderate sample sizes, to check if the conditional combined approach is superior to the best of individual t methods, one should test whether the minimum of the squared sample multiple correlation coefficients is greater than ρ L = 1 tn 1,1 α/ ) ) n p t n p,1 α/ n 1. It is difficult to obtain an exact test to verify this condition. Therefore, in practice one may want to compute all the t intervals based on the individual components and the conditional confidence interval, and then choose the shortest of the intervals for applications. Acknowledgement The authors are thankful to a referee for reviewing this article.

14 14 References Berry, C. J. 1987) Equivariant estimation of a normal mean using a normal concomitant variable for covariance adjustment. The Canadian Journal of Statistics, 15, Cohen, A. and Sackrowitz, H. B. 1984) Testing hypotheses about the common mean of normal distributions. Journal of Statistical Planning and Inference, 9, Halperin, M. 1961) Almost linearly-optimum combination of unbiased estimates. Journal of the American Statistical Association, 56, Jin, C. and Berry, C. J. 1993) Equivariant estimation of a normal mean vector using a normal concomitant vector for covariance adjustment. Communications in Statistics Theory and Methods,, Krishnamoorthy, K. and Rohatgi, V. K. 1990). Unbiased estimation of the common mean of a multivariate normal distribution. Communication Statistics Theory Methods, 19, Krishnamoorthy, K. and Lu, Y. 003) Inferences on the common mean of several normal populations based on the generalized variable method. Biometrics, 59, Muirhead, R. J. 198) Aspects of Multivariate Statistical Theory, Wiley, New York. Tan, M. and Gleser, L. J. 1993) Improved point and confidence interval estimators of mean response in simulation when control variates are used, Communications in Statistics Simulation and Computation,, Yu, P. L. H., Sun, Y. and Sinha, B. K. 1999) On exact confidence intervals for the common mean of several normal populations. Journal of Statistical Planning and Inference, 81, Zhou, L. and Mathew, T. 1993) Combining independent tests in linear models. Journal of the American Statistical Association, 88,