A Monte-Carlo study of asymptotically robust tests for correlation coefficients

Transcription

1 Biometrika (1973), 6, 3, p Printed in Great Britain A Monte-Carlo study of asymptotically robust tests for correlation coefficients BY G. T. DUNCAN AND M. W. J. LAYAKD University of California, Davis SUMMABY Monte-Carlo simulation is used to compare the small-sample performance of the usual normal theory procedures for inference about correlation coefficients with that of two asymptotically robust procedures, one of which is based on a grouping of the observations and the other on the jackknife technique. The sampled distributions comprise the normal and five nonnormal distributions. The small-sample results support the conclusion based on asymptotic theory that the normal test is not robust. The jackknife procedure works well for most of the sampled distributions. Some key words: Correlation coefficients; Hypothesis testing; Confidence intervals; Robustness; Large sample theory;. 1. INTRODUCTION Tests of hypotheses and confidence intervals concerning correlation coefficients in bivariate normal populations are commonly based on Fisher's z transformation of the sample correlation coefficient, tanh" 1 r. This statistic is taken to be approximately normally distributed with mean t&nh~ 1 p and variance 1/(TI 3), the accuracy of the approximation being adequate for moderately small samples and improving as n increases. For a bivariate population, not necessarily normal but having finite fourth moments, it can be shown that ^/n(tanh~ 1 r tanh" 1^) converges in distribution to N{,cr 2 {p)}, where The A's are the standardized cumulants of order four of the bivariate distribution; for example, A^ = KWI "* "?)' where a% and crj are the variances of the marginal distributions. Details are given in an unpublished Stanford University report by M. W. J. Layard. Note that A^ and A. M are the kurtoses of the marginal distributions. If the distribution is bivariate normal, all the A's vanish, and o" 2 (p) = 1. If the components are independent, we have p = and A 22 =, and - 2 (O) = 1. Hence the normal theory test for independence based on tanh" 1 r is asymptotically valid for any population having finite fourth moments. However, if independence does not hold the asymptotic variance of tanh -1 r is not in general 1/n, whether or not p =, and so normal theory procedures about p based on tanh" 1 r may not be valid, even asymptotically, if the population is not in fact bivariate normal. The effects of nonnormality on the distribution of r were first investigated by Baker (193) and Pearson (1931, 1932). An interesting historical survey of this and succeeding work is given by Kowalskd (1972). Correlation coefficients are often used by research workers even though their data are nonnormal. This being a subject of controversy (Nefzger & Drasgow, (1-1) Downloaded from at Penn State University (Paterno Lib) on May 9, 216

2 552 G. T. DUNCAN AND M. W. J. LAYABD 1957; Norris & Heljm, 1961), we present inference procedures which are asymptotically robust against nonnormality and investigate their performance for samples of moderate size. By asymptotically robust we mean that the size of a test, or the coverage of a confidence interval, converges to the nominal level as the sample size tends to infinity. Naturally, in spite of its wide use, it can be argued that correlation has limitations as opposed to regression in examining the relationship between two variables. An estimated regression line, # = &+fix, emphasizes the linear nature of the association, gives a pictorial representation of the relationship, and explicitly allows prediction. In bivariate normal regression analysis one would use t = r N /(n-2)/^/(l -r 2 ) to test independence, i.e. /? =, rather than Fisher's z, and of course the same statistic is commonly used to test the equivalent hypothesis p =. However, one would anticipate the same problems of nonrobustness in the nonindependent case with t as with z. It can be shown that the asymptotic variance of t when /? = is 1 + A M, identical to that of z*jn. Asymptotic nonrobustness can also be demonstrated for the t test of the hypothesis /? = /? +. The asymptotically robust procedures discussed here can readily be adapted to problems of inference about the regression coefficient /?. We now describe the proposed procedures in the context of the two-sample problem of testing the equality of correlation coefficients; the procedures are easily modified to the one-sample problem. Suppose that we have independent samples (X^, F (1 ),..., (X^^, T^^) (t = 1,2) from bivariate populations with distribution functions F and O, correlations p x and /o a, and finite fourth moments. We wish to test where, rj, <r and T are unspecified constants, versus H A : p x 4= p 2. The hypothesis HQ implies Pi = Pi but in general the reverse implication does not hold, though it does in the normal case. The choice of H o rather than the more general hypothesis p x = p t ensures that the fourth-order standardized cumulants of the two distributions are equal. The first test, which we designate the test, is analogous to the procedure suggested by (1953) for testing homogeneity of variances. Divide the tth sample randomly into J { subsamples of size m, supposing that n { = mj {, and let Y it = tanh" 1 r tf, where r it is the sample correlation coefficient for the jth subsample of the tth sample; for our Monte-Carlo simulation, m = 5. Under H^, the T ti are independently distributed with equal means and variances. The procedure is to perform a two-sample t test based on the Y {j. The robustness of the (test suggests that this method should perform well for small samples with respect to significance level. Under Hg, the test statistic is asymptotically standard normal, irrespective of the distributions of the populations. The second asymptotically robust test we examine, designated the jackknife test, is based on the jackknife technique; see Miller (1964) and the references given there. Let r^ be the correlation coefficient calculated from the tth sample omitting the jth observation. Let U i} = n i t&dhr 1 r i (rif ljtanh" 1^. We treat the U i} as being approximately independent and having, under H o> approximately equal means and variances, and we test H^ using a two-sample (test. The results of the Monte-Carlo experiments in 2 show that this method works reasonably well for small samples when the sample sizes are equal. For equal sample sizes, the means and variances of the Uy are exactly equal under HQ. We have not investigated the case of unequal sample sizes in the present Monte-Carlo study, but we would Downloaded from at Penn State University (Paterno Lib) on May 9, 216

3 Asymptotically robust tests for correlation coefficients 553 not anticipate a problem from this source for reasonably large samples. It can be shown (Arvesen, 1969), that under HQ the asymptotic distribution of the test statistic is standard normal, for any population distribution. 2. MONTE-CABLO SIMULATION Small sample results for the tests described in 1 were obtained by simulating on a Burroughs 67 computer random samples from six bivariate distributions with various kurtoses; see Table 1. A mixed congruential pseudo-random number generator was used to produce independent random variables U t, distributed uniformly on the interval (,1). Standard normal random variables Z i were simulated by JJ^ U lt 6. Bivariate random vectors (X, Y) were then constructed to have population correlation coefficient p in the following manner: 1. Normal: X = Z lt Y = pzj^ + Z^l-p 2 ); 2. Log normal: X = exp(z 1 ), Y= expft^ + Zj^l-T 2 )}, where T = log{/o(e- 1) + 1}; 3. Linear regression uniform: X = U v Y = p(l p 2 )-iu 1 +U i ; the marginal distribution of Y is not uniform and the regression of Y on X is linear; 4. Linear regression double exponential: X = ±logu v each with probability, Y = p(l p*)-l X + e, where e = + log U 2, each with probability ; the marginal distribution of Y is not double exponential, and the regression of 7 on 1 is linear; 5. Gamma: marginals ^2 1 1(i x = -x\ogu t, r = - ] This is the bivariate gamma distribution, due to Cherian (1941), which is studied by David & Fix (1961); it has linear regressions. 6. Contaminated normal: ( fz 1,pZ 1 + Z 2 J(l-p i ) (with probability -9), ' ' ~ \ic{z 1,pZ 1 + Z 2 J(l-p t )} (with probability -1). Generally replications of the experiment of sampling 25 observations from each of two bivariate populations of the same distributional family were made. David (1938) suggests 25 observations as a lower bound for the adequacy of the normal approximation to the distribution of Fisher's transformation of the sample correlation coefficient, z = tanh -1 r, under normality. These simulated samples were used to examine the small-sample behaviour of the procedures described in 1 both for testing equality of correlation coefficients and for construction of confidence intervals for a single correlation coefficient. Asymptotic theory suggests that for some of the distributions sampled, one might anticipate substantial departures from the nominal significance levels derived under normal theory for the test based on Fisher's z. The first column of Table 1 gives the numerical values of the kurtosis of each of the marginal distributions sampled in the Monte-Carlo study. The kurtosis is quite large for the log normal distribution and negative for the linear regression uniform distribution. The asymptotic variance of ^/»tanh~ 1 r is given numerically in the second column of Table 1 and demonstrates considerable deviation from the normal theory nominal value of 1. General expressions for the asymptotic variance are given below. The last two columns of Table 1 give the asymptotic size, valid for both one sample and two sample normal theory tests, for the nominal 5 % level. Downloaded from at Penn State University (Paterno Lib) on May 9, 216

4 554 G. T. DUNCAN AND M. W. J. LAYABD Table 1. test. Asymptotic results for distributions in sampling study Distribution p = -6 p = -9 jfc = 3 p = 6 p = -9 p = -6 p = -9 p = -6 p = -9 Kurto8is of marginal distributions* Log normal Asymptotic variance of Jn tanh" 1 r Contaminated normal Linear regression uniform -1-2 (--924) -1-2 (-1-15) Linear regression double 3 (2-31) 3- (2-537) -6-6 Gamma exponential Asymptotic size; one-sample and two-sample tests, nominal 6 % level * Kurtosis for 'linear regression' distributions ia for X component; kurtosis for Y component is given in parentheses. Let o- 2 (p)/n denote the asymptotic variance of z as a function of the true correlation p. Then for the contaminated normal distribution, **( P ) = For each of the linear regression distributions with Y = p(l ^!)~H + e, o" 2 (p) = where X and e each have kurtosis A. For the log normal distribution, a 2 (p) is given by (l-l), with A 31 = A 13 = 7 5- (e 1... _ i2e-6), A M = (e 1) Lastly, for the bivariate gamma distribution, 1-2/(2-/) P - e 3 + 6e - 6), - e 2 + 4e - 6). Downloaded from at Penn State University (Paterno Lib) on May 9, 216 Note that for each of the bivariate distributions sampled, except the contaminated normal, p = implies independence, and for these distributions the asymptotic variance of z is 1/n when p =. But for the contaminated normal the components are not independent when p =, and the asymptotic variance of z is not 1/n in that case; nor does the asymptotic variance depend on p.

5 Asymptotically robust tests for correlation coefficients 555 Tables 2 and 3 give the Monte-Carlo results for the (m = 5) and jackknife procedures, and for the normal theory procedures based on Fisher's z. We first discuss the one-sample case and then the two-sample case. Table 2 gives simulation results for one sample confidence interval procedures. Associated with each of the test procedures is a method for constructing confidence intervals for a single correlation coefficient. Each entry in the first three columns of Table 2 is the observed fraction of coverages of the true correlation coefficient. We note that subtracting this fraction from 1 gives the empirical size of a two-sided test for equality with the stated correlation coefficient. Also note that for 1 replications the standard deviation of the fraction of coverages is approximately -6, when the true probability of coverage is about -95. Table 2. Empirical results for nominal 95% confidence intervals for a single correlation coefficient, p. For the test, m = 5 Distribution sampled Normal, n = 26 Log normal, n = 26 Linear regression uniform, n = 26 Linear regression double exponential, n = 25 Gamma, n = 25 Contaminated normal, k = 3, n = 1 Contaminated normal, k = 3, n = 25 Contaminated normal, k = 3, n = 1 Correlation coefficient Replications Fraction of coverages of p Downloaded from at Penn State University (Paterno Lib) on May 9, 216 For tests of equality of correlation coefficients, bias of the test statistics is not a problem since under the null hypothesis these test statistics are simply differences of two random variables with the same distribution, and hence have mean. But in constructing a confidence interval for a single correlation coefficient, bias of the relevant statistic may be a problem, especially for the procedure, which in our simulation studies is based on subsamples of only 5 observations. For the bivariate normal distribution, the bias of tanh -1 r is approximately \p{n I)" 1 ; bias of this order is removed by the jackknife.

6 556 G. T. DUNCAN AND M. W. J. LAYABD We now discuss the results in Table 2, which give the empirical coverage of nominal 95 % confidence intervals for p. Although not shown, the empirical average length was also calculated and will be discussed. (1). The normal theory procedure maintains the nominal confidence level -95 for every value of p only under normality and in the Linear regression uniform case. However, its performance is generally adequate for p =, except for the contaminated normal distribution. (2). The jacfcknife does reasonably well in maintaining nominal levels in the uniform, double exponential, gamma and contaminated normal cases, but rather poorly in the log normal case. For the normal distributions sampled, the average length of the jackknife intervals is slightly more than that of the normal theory intervals. (3). The effects of bias are clearly apparent for nonzero correlations. For p =, the intervals have about the right confidence level, but the average lengths are in most cases considerably greater than those of the jackknife intervals. We conclude that the procedure is unsuitable for constructing confidence intervals for a single correlation coefficient, or for testing hypotheses about a single coefficient. In Table 3, empirical test sizes at the nominal 5 % level are given for when {p lt p 2 ) = (,), (-6, -6) and (-9, -9). Empirical power is given for (p v p 2 ) = (,-6) and (,-9). In the contaminated normal case sample sizes of 1 and 1 were also examined; for the sample size of 1 power calculations were made for (p^pt) = (,-3) and (,-6). Since replications were made, the standard deviation of any entry in Table 3 is -1 for true probability of rejection -5, and it is never higher than 22, which is the standard deviation for true probability of rejection -5. Results at the nominal 1 % level were computed but are not shown; they are consistent with those in Table 3. The results in Table 3 are now discussed for each distribution. (1) Normal. The size of each of the 3 tests does not deviate significantly from nominal levels. The power of the test lags, but the other tests have comparable power. (2) Log normal. For this skew distribution with a heavy tail, the normal theory test fails badly with actual sizes differing markedly from nominal, particularly as the common correlation increases. The test works well in maintaining the nominal level and its power is comparable to that of the jaokknife. The significance levels of the jackknife test are higher than nominal for nonzero correlations, but are much lower than those of the normaltheory test. (3) Linear regression uniform. The significance level of the normal-theory test is below nominal for nonzero correlation, but its power is as good as the jackknife test. The test has relatively poor power. (4) Linear regression double exponential. The size of the normal-theory test departs increasingly from nominal levels as the common correlation gets larger. The power of the jackknife test is a little better than that of the test. (5) Oamma. The significance level of the normal-theory test is too high for a large common correlation. The jackknife has better power than. (6) Contaminated normal. The normal-theory test continues to fare badly, and it is interesting to note that it does worse as the sample size increases. Both the test and Downloaded from at Penn State University (Paterno Lib) on May 9, 216

7 Asymptotically robust tests for correlation coefficients 557 Table 3. Empirical size and power based on replications for tests of equality of two correlation coefficients at nominal 5 % level. For the test, m = 5 (Pi' Pt) (,) (-6,-6) (-9, -9) Normal; n = Log normal; n = Linear regression uniform ;n = Linear regression double exponential; n = Gamma; n = Contaminated normal; k = Contaminated normal; k = Contaminated normal; k = , n = , n = , n = (, -6) (, -9) the jackknife test have about the right size but the jackknife is slightly better on grounds of power. The results of Table 3 suggest the following general conclusions: (i) the normal theory test can be strongly affected by nonnormality, even if the correlations are small; (ii) the test performs very well in maintaining nominal levels but may have low power; (iii) the jackknife does reasonably well in maintaining nominal size and its power is not much less than the best of the three tests for each bivariate distribution sampled. Downloaded from at Penn State University (Paterno Lib) on May 9, 216

8 558 G. T. DUNCAN AND M. W. J. LAYARD 3. CONCLUSIONS Consistent with asymptotic results, the normal theory procedure based on Fisher's z has been shown to produce poor results for certain nonnormal distributions. The test does well in maintaining nominal levels for tests of equality of two correlation coemcients, but often has low power. The small sample bias of the test makes it unsuitable for tests and confidence intervals about a single correlation coefficient. The jackknife procedure applied to Fisher's z gives a test which comes much closer than the normal theory method to maintaining nominal levels while having comparable power. It should be noted that our sampling results are consistent with the implications of Pitman's (1937) study in the limited case where zero correlation implies independence. Specifically, Pitman shows that under independence the normal theory test is a good approximation to a permutation test and hence would be expected to be robust. As we remarked in 1, the asymptotic variance is correct in this special case and the sampling results for normal theory tests for all distributions meeting this independence condition are close to nominal. However, the condition is not met for the contaminated normal distribution and the empirical results are far from nominal. REFERENCES ABVESEN, J. N. (1969). Jackknifing {/-statistics. Ann. Math. Statist. 4, BAXHB, G. A. (193). The significance of the product-moment coefficient of correlation with special reference to the character of the marginal distributions. J. Am. Statist. Ass. 25, , G. E. P. (1953). Non-normality and tests on variances. Biometrika 4, CHEBIAN, K. C. (1941). A bivariate correlated gamma-type distribution function. J. Ind. Math. Soc. 5, DAVID, F. N. (1938). Tables of the Correlation Coefficient. Cambridge University Press. DAVID, F. N. & Fix, E. (1961). Rank correlation and regression in a non-normal surface. Proc. 4th Berkeley Symp. 1, KOWALSKI, C. J. (1972). On the effecte of non-normality on the distribution of the sample productmoment correlation coefficient. Appl. Statist. 21, MTTT.TCB, FV. G. (1964). A trustworthy jackknife. Ann. Math. Statist. 35, NEFZGEB, M. D. & DBASQOW, J. (1957). The needless assumption of normality in Pearson's r. Am. Psychol. 12, NOBBIS, R. C. & HEIJM, H. F. (1961). Non-normality and product-moment correlation. J. Exper. Educ. 29, PEARSON, E. S. (1931). The test of significance for the correlation coefficient. J. Am. Statist. Ass. 26, PEABSON, E. S. (1932). The test of significance for the correlation coefficient: some further results. J. Am. Statist. Ass. 27, PITMAN, E. J. G. (1937). Significance tests which may be applied to samples from any populations. n. The correlation coefficient test. J. R. Statist. Son., Suppl. 4, pp Downloaded from at Penn State University (Paterno Lib) on May 9, 216