+ P,(y) (- ) t-lp,jg).

Transcription

1 (1 THE DISTRIBUTION OF THE LARGEST OF A SET OF ESTIMATED VARIANCES AS A FRACTION OF THEIR TOTAL BY W. G. COCHRAN 1. INTRODUCTION FOR a set of quantities ul, u2,..., u,, each distributed independently as x2c2 with two degrees of freedom, Fisher (1929) obtained the distribution of the ratio of the largest of the u s to their total. He found the probability that this ratio exceeds the value q to be nl P = n(l-g) -l-- n(n- 1) (1-2g)n (- )k-1 ~ - kg) -l, 2! k!(n- k)! where k is the greatest integer less than llg. This distribution is used in the harmonic analysis of a series, in testing a particular term which is picked out on inspection because of its exceptional magnitude. It has also been employed (Fisher, 1030) to illustrate the disturbance introduced into the.z-test of a regression equation, when the equation is nonlinear, in the sense that the eliminant of the regression coefficients is a non-linear function of the observations which satisfy the regression equation exactly. The object of this note is to give the result corresponding to (1) for n sets of r degrees of freedom each. The case r = 2 appears to be the most useful one practically, as well as the simplest mathematically. The general result is, however, of some mathematical interest, and is occasionally helpful in testing one of a group of estimates of variance which appears to be anomalously large. 2. A THEOREM IN PROBABILITY The principal difficulty in the proof arises from the fact that the ratios ui/( u1 + u un), 1: = 1, 2,..., n, are not independently distributed, since their total is unity. The distribution of the largest ratio will be obtained from a theorem in probability which is of fairly general application to problems of this type. Theorem. Let the joint distribution of a set of quantities xl, x,,..., x,, be a symmetrical function of the variates xl,..., x,,. If Pl(g) is the probability that a specified variate exceeds the value g, P2(g) the probability that a specified pair of the variates both exceed g, and so on, the probability that the largest of the variates exceeds q is n(n- 1) n(n- 1)(n-2) np,(d - 7 P2(g) 3! + P,(y) (- ) t-lp,jg). This theorem is the analogue of a familiar result in the theory of derangements (cf. Whitworth, Choice and Chance, Prop. XIV). (1)

2 48 ESTIMATED VARIANCES Proof. The variates are considered in a definite order, xl, x2,..., xn. The probability, p(xl G g), that x1 does not exceed g, is 1 - Pl(g). But and P(Z1 9) = P(X1 G 9, x2 G 9) +Ax1 G 9, z2 > 9) P(X1 f 9, x2 > Y) = p,(g) - P2(9). Hence P("1 G 9, x2 9) = 1-2p,(g) + P2(9). (3) Repeating the above argument with Pl(g), P2(g), P3(g) in place of 1, <(g), P,(g) respectively, we find P@l G 9, x, G 99x3 > 9) = P,(d - 2pa(s) + P3(9). Hence P(x1G 9, 22 < g,23 < 9) = 1-3Pl(g) + 3&(g) -P3(g). The rule of formation of successive terms is clear from (3) and (4), the coefficients being the coefficients in the expansion of (1-1)1. Thus the probability that none of the n variates exceeds g is found to be n(n - 1) 1 - np,(g) + -g- --* + ( - )npn(g)* Hence the probability that the largest of the variates exceeds g is as stated in the theorem. If the variates xl, z2,..., xn are all independently distributed, P,(g) = PKg), P3(g) = e(g),..., In this case, (5) reduces to [l-p1(g)ln and (2) to 1 -[l-p1(g)]". Thus the theorem may be regarded as the extension of this well-known result to the case where the variates are symmetrically, but not independently, distributed. (4) (6) 3. THE JOINT DISTRIBUTIONS OF THE RATIOS To apply the theorem, it is necessary to study the joint distributions of groups of the ratios xi = ut/(ul +... i- un). These may easily be obtained, since the quantities y1= u~/(u~+u~+.*.+u,), y2 =~2/(~2+~3+*..+~n), y3 = u3/(u u,)... yn-l = un-l/(un-l + U") are independently distributed. Each of the quantities yt is of the form I( n:%), where z is Fisher's z for nl and n2 degrees of freedom, n2 being equal to r throughout, while the successive values of nl are r(n - l), r(n - 2), r(n- 3),... respectively. The distribution law of v = eas for n, and n2 degrees of freedom is Hence the distribution of any y = is I( with the appropriate values of n1 and n2.

3 W. G. COCMRAN 49 Now y1 = xl, ya = z2/( 1 - zl), y3 = z3/( 1 - z1-34, etc., so that the joint distribution.of any group of the ratios x* may be found by makmg the neoessrtry substitutions in the joint distributions of the y's. This gives, for a single specified ratio, f1(zl) = Br,,z*'-1( 1 - ~~)*('f''-q~-1, 0 < z1 < 1 (8) For two specified ratios, For three specified ratios, In the notation of the theorem, 4. THE DISTRIBUTION OF THE LARGEST BATIO 1 P,(d = j fi(%)d%, bj and so on. Hence the probability that the largest ratio exceeds g is n(n - 1) np,(g) = -g- G(g) + * * * + ( - )"-lpn(g), where the number of non-zero terms appearing in the expression is the greatest htt3ger less than l/g. The value of Pn(g) is not required in practice, since P,(g) = 0 if g > l/n, and the largest ratio is certain to be as great as l/n. For r = 2, it may easily be verified that P,(g) = (1-g)"-l, P2(g) = (1-a)'+', etc., in agreement with Fisher's reault. For all other values of r the integrals are more troublesome. The first term, Pl(g), is simply the incomplete integral of a Beta-function distribution, and may be obtained from the published tables (1934), or by direct calculation outside the limits of the tables. The second and all subsequent terms require repeam integration of the Beta-fwnction distribution. With even values of r, all terms reduce to a polpod in g, but the expressions rapidly become complicated. For instance, r = 4, gives P,(g) = (1-g)2m-2[1+2(n- l)g], pb(g) = (1-29)26-8[( 1-2g)2+ 2(2n - 1) g(1-29) + (2n - 1) (2n - 2) gal. (14)

4 50 ESTIMATED VARIANCES As shown by Fisher (1929) for the case r = 2, the upper 5 and 1 % significance levels of 9 can be calculated without difficulty, with accuracy sufficient for most purposes. It was pointed out above that if the x-variates were independent, P,(g) would equal P:(g). Since, however, the probability of any other x being high is decreased as x1 increases, P,(g) must be less than P:(g) for all values of n and r. At the 5 % level, a first approximation to g is obtained by ignoring all terms except P,(g), choosing g so that Pl(g) = 0-05/n. In this case P2(g) is less than /n2, and the term n(n - 1) P2(g)/2 is less than 0*00125(n- l)/n. Thus the value of g obtained by using the first term only, while always slightly too high, gives is considerably a probability which is between and For small values of n, P2(g) less than P!(g) at the 5 % level, and the approximation is much closer. Similarly, the use of the first term alone at the 1 % level gives a probability between and A lower limit to the significance level may also be obtained, by treating all the x-variates as independent, and using [1 -Pl(g)J" = 0.95 or As an example, with five sets of six degrees of freedom each, the first term alone gives the 5 yo level of g as , which is correct to a unit in the last place, while the assumption of independence gives the lower limit g = For higher values of n, the lower limit is still less in error. A table of the 5 % points of g for small values of n and r is given below. Eii /-i ; E 9 I0 - I ' Table of 5 % levels of the largest ratio ulzu 3 4 0' ' '5321 0' I_ 0.707' ' '4447 0' '2541 Within the limits of the table, only the first term need be considered to give four-figure accuracy. As a check, the second terlh P2(g) was evaluated by a series of the form (en- l)! ~ gr-2( 1-2g)H7(n-2)}+1 [(@- 1)!12{arr(n-2)1+ll! 2(T-22) s+ (r- 2) (3r- 10). +r(n- 2) + 4 [r(n- 2) + 41 [r(n- 2) + 61 where 2 = (1-2g)/g. This series terminates if r is even, and the terms diminish fairly rapidly. For r = 10, n = 10, the value of the second term, n(n- ')P,(g) is , while a change of one unit 2 in the fourth decimal place of the tabulated value alters np,(g) by about Thus at this stage the value given by the first term only is about five units out in the fifth decimal place.

5 W. G. COCHRAN THE LIMITING DISTRIBUTION WHEN n IS LARGE For groups of two degrees of freedom, Fisher (1939) showed that the mean value of g was Thus when n is large, g is approximately equal to (y + log n)/n, where y is Euler's constant. The corresponding result for sets of r degrees of freedom is investigated in this section. Since the whole range of the distribution of the largest ratio must be considered in finding its mean value, it is not sufficient to consider only the first term in the probability integral of g. However, when n is large, the ratios may be regarded as independently distributed, and the formula 1 - [l- P(q)]n used as the probability that.the largest ratio exceeds g. From (8) The mean value of x1 is l/n and its variance is O( l/n2). Hence, for large n, fl(xl) may be written approximately fl(zl) - czp-1 e-*mzl (17) so that nrxl is distributed approximately as xa with r degrees of freedom. The integral of fl(sl) may be taken from g to co instead of from g to 1, and in the series for the integral (Fisher, 1935) only the first term need be retained. This gives Pl(g) - e-tnro. (18) Hence the probability that all n are less than g is approximately (1- e-tnr8)n. (19) Differentiating, we find for the frequency distribution of q, $(g) dg - in% e-*nro( 1 - e--tnrg)n-l dg. (20) Put hrq = log n + u. The limits of variation for u are - log n and Qnr -log n, which may be taken as - 00 and +a. Hence the frequency distribution of u is The moment-generating function of $(u) is = Jm etiie-u-e-udu = y-fe-ydy = (-t)!. --m JOrn Using the expansion formula for log r(z + l), we find 7r2 t2 K = logm = yt Thus the mean value of g for n large, is approximately 2 - (y+logn). nr This result could alternatively have been obtained from the study by Fisher & Tippett (1928) of the general form of the limiting distribution of the largest member of a sample.

6 62 ESTIMATED VARIANCES The case r = 1 may be related to the fitting of a sine series Asinex, x = 1, 2,..., n, to a set of n values of y normally distributed about a zero mean. If and p takes the values 1, 2,..., n, n Thus the linear functions 2 am yr provide a subdivision of r-1 t- 1 n y: into n independent single degrees of freedom. The reduction in the sum of squares of y given by the least-squares values of A and 8 cannot be less than the largest of the terms (Za,yt)a. Hence the fraction of the sum of squares accounted for by the two constants A and 8 tends to 2(y + log n)/n, or a greater value, as n incrwses, instead of 2/n. SUMMARY For a set of n independent estimates of the same varihnce, each based on two degrees of freedom, Fisher (1929) obtained the distribution of the largest estimate as a fraction of the total. By means of a theorem in probability which may be frequently useful in problema of this type, this result is extended to a set of estimates each based on r degrees of freedom. The distribution has the same general form as Fisher s result, but is simplest in the case that he considered. A table is given of the 5 % significance levels of the largest ratio, for small values of r and n. The limiting distribution when n is large is briefly discussed. REFERENCES R. A. FISHER & L. H. C. TIPPETT (1928). Limiting forma of the frequency distribution of the largest or smalleat member of a sample. Proc. Cad. Phil. 8m. 24, R. A. FISHER (1929). Testa of significance in harmonic analysis. Pm. Roy. 8oc. A, 126, (1935). The mathematical distributions used in the common testa of significance. EconometriCa, 3, (1939). The sampling distribution of some statistics obtained from non-linear equations. Ann. Eugen. Limd.. 9, Tables of the incomplete Beta-function (1934). London: Biometrika Office.