A FURTHER CONTRIBUTION TO THE THEORY OF SYSTEMATIC STATISTICS

Size: px

Start display at page:

Download "A FURTHER CONTRIBUTION TO THE THEORY OF SYSTEMATIC STATISTICS"

Lynn Newton
6 years ago
Views:

1 A FURTHER CONTRIBUTION TO THE THEORY OF SYSTEMATIC STATISTICS.; i/'.t: :PlfK by J unjiro. Ogawa. University of North Car9}ina Sponsored by the Office of Naval Research under the contract for research in probability and statistics at Chapel Hill. Reproduction in whole or in part is permitted for any purpose of the United States government. Table 3.1 produced under sponsorship of the Office of Ordnance Research. Institute of Statistics Mimeograph Series No. 168 April, 1957

2 A FURTHER CONTRIBUTION TO THE THEORY OF SYSTEMATIC STATISTICS l by Junjiro Ogawa, Institute of Statistics, University of North Carolina Introduction. Until 1945 the main interest in the field of statistical estimation seems to have been in the so-called liefficient" estimators. But from the point of view of economy in practical use, it seems reasonable to 1nquire whether the output of information is worth the input measured in money, man-hours, or otherwise. Thus we may ask whether comparable results could have been obtained by a smaller expenditure. From this standpoint F. Mosteller proposed 2_7 2 in 1946 the use of order statistics for such purposes on the ground that, however large the sample size, the observations can easily be ordered in magnitude With the help of punch-card equipment. He considered the problem of estimation of the mean and standard deviation of a univariate normal population, and that of estimation of the correlation coefficient of a bivariate normal distribution. Then in 1951 J. Ogawa 4_7 considered more systematically the problem of estimation of the location and the scale parameter of a population whose density function depends only on these two parameters, and obtained the optimum solutions in some cases for the univariate normal population. There are many cases in which the samples are by their very nature lsponsored by the Office of Naval Research under the contract for research in probability and statistics at Chapel Hill. Reproduction in whole or 1npart is permitted for any purpose of the United states government. Table ;.1 produced under sponsorship of the Office of Ordnance Research. 2The numbers in square brackets refer to the bibliography listed at the end.

3 2 ordered in magnitude as for example in a life test of electric lamps or a fatigue test of a certain material. Usually in such cases the population probability density functions are assumed to be exponential. So at least for the exponential distribution estimation and testing of an hypothesis based upon systematic statistics are of great importance, from the point of view of application. The first two sections of this paper will be devoted to the general theory of systematic statistics from the population whose density function depends only upon the location and scale parameters for sufficiently large values of sample size. Then in 3 the application of the general theory to the exponential distribution will be given and optimum spacing of the selected sample quantl1es and the corresponding best estimators will be tabulated. Finally, in 4, discussion on testing an hypothesis will be presented. 1. Relative Efficiencies. The fundamental probability distribution of the large sample theory of systematic statistics from the population whose probability density function depends only on the location parameter m and the scale parameter a is as follows. The limiting frequency function 1s, (1.1)

4 where f(u) denotes the standardized frequency function of the population and u i stands for the Ai-quantile of the population, i.e., i = 1,2,,k ( 1.2) and f. = f(u.), i = 1,2,,k (1.3) Finally, we put A O = 0, ~+1 = 1. (1.5) In the first place, we define the relative efficiency of the systematic statistics with respect to parameter to be the ratio of the amount of information in Fisher's sense derived from (1.1) to that derived from the original whole sample. We shall consider the following two cases separately: Case I. The case in which the scale parameter a is known and the location parameter m is unknown. For the sake of convenience, put (1.6)

5 4 where the right-hand side is the same as the expression in the ex~~,~ in the density function h(x(n l )'X(n 2 )""'X(n k» except for the constant factor - n/21,2, then we have (1.7) ThuS it follows by differentiation 'with respect to m that (1.8) where k+l K l = r, i=l and f k + l = 0 and f O should be equal to lor" 0 as the case may be:.- Hence the amount of information Is(m) of the systematic statistics With respect to the location parameter ill is asymptotically equal to (1.10) The likelihood function of the original whole sample, considered as a random sample of size n, is L = a- n n TI i=l xi-m. f( -a- ) I (1.11)

6 5 hence we have n xi-m log L == L log f( --a- ) - n log cr, i=l (1.12) and consequently the amount of information IO(m) of the ori$inal whole sample with respect to the location parameter m is equal to 2 (1.13) where U i = (Xi - m)/cr, i.e. U stands for the standardized variate. e If f(t) )- 1/2 \ == (21C exp 1.- t 2/2}, then IO(m) n == -;-, (1.14) and if f(t) J. -\I = e for t > 0, then IO(m) 2 n = -; (1.15) Thus the relative efficiency of the systematic statistics with respect to the location parameter m is defined as: 1)(m) ) 2 (1.16) ~ In particular, for the normal distribution we have,,(m) = K l ' (1.17)

7 6 and for the exponential distribution we get T} (m) = ~J =.!, e n n (1.18) In the case of the one-sided exponential distribution defined by g(x) - x-m = - 1 e - Cf for x > m, Cf = o otherwise, we know already that min xi i.e. x(l) is the maximum likelihood estimator,!<i<n and further that this is the uniformly minimum variance estimator of m for ~ all values of n. Thus, the estimator based upon the frequency function (1.1) becomes meaningless. Case 2. The case in which the location parameter m is known and the scale parameter a is unknown. In this case log h n = - k log Cf S + a term independent of a. 2(;'2 (1.20) Therefore we get (1.21) and since it can easily be seen that (1.22)

8 7 where (1.23) we obtain finally the amount of information of the systematic statistics with respect to the scale parameter 0, i.e. (1.24) or I (0) S On the other hand we have o log h _ n 1 ~ ao - - (j - -; 1=1 X. -m f(~~:---) (1.26) Hence by making use of the general relation I [ (Ufffg~) = 1, (1.27) we obtain (1.28) In particular, for the normal distribution we have

9 8 2n I O ( a) =2' (J and for the exponential distribution we have n = -; (1.30) Thus we obtain the relative efficiency of the systematic statistics with respect to the scale parameter a as follows: '1 ( a), 2 E (U;(bY») - 1 (1.31) In particula; for the normal distribution we have (1.32) and for the exponential distribution we have (1.33) 2. The Best Linear Unbiased Estimators of the Unknown Parameters Based upon the Selected Sample Quantiles for Sufficiently Large Sample Size n. In the circumstance now under consideration, the basic distribution is given by (1.1), and the unknown parameters are m and a. Since the dis- ~ tribution given in (1.1) is a k-dimensional normal distrtubtion, we can apply

10 9 the extended Gauss-Markov theorem on least squares L:'_7 in order to find the best linear unbiased estimators which are asymptotical~etticient estimators, Hence, as far as the large sample problems are concerned there are no other estimators which are more efficient than the best linear unbiased ones. Case I. The case in which the scale parameter a is known and the location parameter m is unknown. 1\ In this case, we can find the best linear unbiased estimator m of the location parameter m by solving the single normal equation os I 1\ = 0 dill m=m ' (2.1) which turns out to be (2.2) where k+l X =.E i=l (fi-fi-1)(fix(ni)-fi-1x(n1_1» )..1-)..i-1 and (2.4) ;\ Hence the best linear unbiased estimator m of m is.j\ 1 m = - X K l

11 10 and it can easily be seen after small calculation that (2.6) From (2.6), we can see that the best linear unbiased estimator In? is an "efficient estimator" (so to speak) from the point of view of the relative efficiency given in the preceding section. If in particular the frequency function f(t) of the population is symmetric with respect to the origin, for example the normal distribution, and the spacing of the selected sample quantiles X(n )' x(n2)""'x(~) i is also symmetric, i.e., n i + n k _ == n,; i == 1,2,,k (2.7) or in terms of hi Ai + 'K-i+l = 1,; i = 1,2,,k, (2.8) then we have u i + u k _ i + 1 = 0,; i=1,2,,k (2.9) and f i == fk_i+l; 1 == 1,2,...,k. (2.10) e In such e. case, it follows clearly that

12 11 (2.11) Hence we have /\ 1 m=- X, K l (2.12) and Case 2. The case in which the location parameter m 1s known and the scale parameter (J is unknown. In a quite similar manner, we can find the best linear unbiased /\ estimator (J of the scale parameter (J by solving the normal equation ~I ocr.cr = 1'=0. (J (2.l4) This comes to be where (2.16) Hence we get (2.17)

13 12 and!' Var( 0) 1 K 2 (2.18) In the particular case in which f{t) and the spacing of the selected sample quant11es are symmetric we get as before 1\ and the variance of 0 is the same as before. 3. Determination of the Optimum Spacings for the Estimation of the Scale Parameter of the One-Parameter Exponential Distribution Based on the Selected Sample Quantiles for SUfficient~y Large Sample Size n. ~5_7 In L:4_7, the author developed the general theory of estimation' of the location and the scale parameters based upon the selected sample quantiles for sufficiently large sample size n and determined the optimum spacings in the case of the normal population. Owing to its importance in practical applications, optimum spacings are calculated herein for the estimation of the scale parameter q of the exponential distribution whose density funotion is given as follows: x 1 ici e 0, if x > 0 g(x) = (3.1) 0, otherwise

14 13 Suppose we are given an ordered sample of size n, where n is sufficiently large. In other words, the sample size n is l.arge enough so that the conelusions drawn making use of the limit distribution are valid with enough accuracy. For given k real numbers such that o < ).,1 < A 2 < < \: < l, we select k sample Ai-quantiles for i =1,2,,k, i.e., k order statistics where and the symbol ~nai_7 stands for the greatest integer not exceeding naif Furthermore, let the A.-quantile of the standardized exponential 1 distribution, whose density is given by [ eo-x: f(x) l, if x> 0 otherwise (3.4) be u i ' and that of (3.1) be x i, then it is clear that The limiting joint distribution of the k sample quant11es x(n 1 ),,x(d k ) haa the density function

15 14 = C kexp[.. ~ n, 2(Jc:. where e Hence we get -u. Ai = l-e 1 I 1 = 1,2,,k (3.7) log h = - ~ log (J2 n S + term which is independent of (J (3.8) c; - 2lwhere k - 2 r: 1=2 Thus we get the amount of information of the systematic statistics with e respect to the scale parameter cr in this case

16 15 (3.10) where K 2 = k+1 t 1=1 k+l == t i=l (3.11) Consequently we get the relative efficiency of the systematic statistics with respect to G as (3.12) The best linear unbiased estimator ~ of G is where k+l Y = t i=l and this can be written as k Y = r a x i=l i (n i ) ~ where

17 16 (3.16) The coefficients a i for the optimum spacings are calculated in Table 3.1. From (3.13) and (3.15) we have where (3.18) I Although it may appear to be more convenient to tabulate a. than a., for 1 1 k = 10, for instance, there are ten rounding errolbin using (3.17), while /'0 the calculation of a by the formula I has the error obtained by only one rounding process. The required optimum spacing is the set (Al'A2?"~'~)' or equivalently the set of values (u i,u 2,,u k ) which gives the maximum value of the relative efficiency K 2 We can obtain the optimum spacings step by step as follows:

18 17 (i) The case where k = 1: in this case it is easily seen that (3.20) which has only one maximum K 2 = at u 1 = 1.59, and the corresponding probability is ~l = 1 - e- l 59 = (ii) The case where k = 2: put u 2 = u l + x, then we have + ~2 ).> e -1 and this function of x and u l is maximized at the point x = 1.59 i.e., and the maximum value of K 2 is (iii) The case where k = 3: put then it follows after some calculations that

19 18 -x { 2 e -+ e x -1 2 x+ the point It is easily seen that this function of x,y and u l is maximized at Hence we get the optimum spacing in this case as follows: and the correspoinding maximum value of K 2 is In a similar way, the following results in Table 3.1 have been calculated. population. In Table 3.1, u i stands for the Ai-quantile of the standardized The bottom row of the table which representing the relative efficiencies K 2 has been graphed as Fig. 1 to show the increasing rate of relative efficiencies against the number of sample quentiles which has been selected. It will be seen from Fig. 1 that after k = 10, the gain in relative efficiency is not appreciable.

20 e. Table 3.1.e optimum Spacings for Estimates of Relative Efficiencies and the Coefficients of Best Estimates* _ l.3 loll O. _ h.s _ =:;2.1386:: ~ ' , ' $ ) B lo ':\ ~ (.2) _ 9 02 :0) '1) e

21 20 Fig. 1. The curve of relative efficiencies against the number of selected sample quantiles to be used l e e Number of Sample Quantiles

22 21 Example: As an illustration of the estimation procedure, we shall calculate the estimate of the standard deviation of the time intervals in days between explosions in mines, involving more than 10 men killed from 6th December 1875 to 29th May The data are from Maguire, Pearson and Wynn 1_7. It should be noted that this example may not be the best to point out the value of this method because the sample size (n =109) is not so large that the labor necessary for the classical estimation is almdst cg:lmparable 'tot), that necessary for our estimate. This example should be regarded as an illustration of the calculating procedure itself. There are cases, which are tmportant in applications such as in life testing of the electric lamps and in fatigue tests of certain material, where the samples are ordered in magnitudes in natural order. In such a case the procedure here may be of great help in getting quickly the estimate of 0, especially for a large bulk of data. The Table 1 of B. A. Maguire, E. S. Pearson and A. H. A. Wynn is reproduced here with observations arranged in order of magnitude as follows:

23 e Table 3.2. Time intervals 1n days between explosions in mines, involving more than 10 men killed from 6 Dec to 29 May (B. A. Maguire, E. S. Pearson and A. H. A. wynn) Order Observation Order Observation Order Observation ; e e

24 23 For k =10, we can see from Table 3.1 that n 1 = 1 n>-'1_ x,23662_7 + 1 = 26 n 2 = n>-'2_7 + 1 = x.1j.3447_7 + 1 = 48 n 3 = n>-' = 109 x.59343_7 + 1 = 65 n4 = n>-' = 109 x.71917_7 + 1 = 79 n 5 = n>-' = 109 x.81732_7 + 1 = 90 n6 == f n;" = 109 x.88920_7 + 1 = 97 n 7 = n;"7_7 + 1 = 109 x.93980_7 + 1 = 103 ns = 1n) =1 109 x.97156_7 + 1 == 106 n 9 = nr.,) = 109 x.98975_7 + 1 = lob n 10 =.Ln;"1~7+ 1 =1 109 x.99791_7 + 1 = 109 Thus, we used the ordered observations as follows: Order(n.) Observations(x(» Coeff1c1ents(a. ) J e

25 24 Thus we calculate the expression and consequently we have If we compare the estimate 0 = 241 which was calculated using all of the sample by the classical method with the above, there is quite good agreement. 4. Testing Statistical Hypothesis. In order to test the hypothesis (4.1) if we start with the frequency function given by (l.l), then this can be done as follows: if the null-hypothesis H O is true, then k n E ~ { 1=1 A i +l-a i _ (A A }{~ A ) f1(xcn.)-uioo) i+1-1 i- i-1 1

26 25 is distributed according to the X2-distribution with degrees of freedom k. This comes out as The minumum value So of S under the variation of a is given by k+l S = L a 1=1 (fix( )-f. lx(»2 n. ~- n ~- (4.4) ~ and n 2 So 1s distributed according to the ;t2-distribution with degrees 0'0 of freedom k-l, provided EO is true. Hence is independent of So and is distributed according to the;t2-distr1bution with degree of freedom 1. Thus the statistic t = vk-i (4.6) ~ follows the Student's t-distribution with degrees of freedom k-l if the

27 26 null-hypothesis H o is true. If the null-hypothesis H O is not true, and an alternative hypothesis H:o::: 0(1 0) is true, then the distribution of t in (4.6) follows the noncentral t distribution 'With non-centrality parameter- 0 5 =,/if; (1 - (1 ), (4.7) and the power functwj1 of' the t-test is an increasing function of the absolute value of 5. Hence"tt Ls reasonable to se.~e8t sample quantiles the spacing of which makes K 2 maximum, i.e. optimum spacing for estimation purpose is also optimum for testing purpose. Thus we can use Table ;.1 also for testing purposes. Finally, it should be noted that the confidence interval of a with confidence coefficient 100(1-0:) per cent it given by (4.8) where tk_l(lo(~) with degrees of stands for the 1000 per cent point of the t-distribution freedom k-l. Example: We shall calculate the 95% confidence interval for in the example of the preceding section, as an illuetration of the procedure explained in this section. We consider case k = 10. Then we have

28 e " - ' "'0-11 (f'.x ( ) -f. 1x ( l J.- n _ i 1 _ K 1,2 i=l )..1-A1_ (fix(ni)-f1-1x(ni_l» (E 1 8 i X(n.» 1 l..: ""i- A K 1=1 1_1 2 (4.9) and calculation Ylill be executed as shown in the following table. The 95% confidence interval calculated below seems to be somewhat wide, which may be due to the smallness of the sample size.

29 e...e. ~ Calculating SCheme for So! ~O! l:f o I_~_n 2 fix(n l ) f1x(n~) l' x - Al-~O 1 (n 1 ) Al ~ D1!'1. I f 1 X(D 1 ) f 1X(D 1 ) I, D 1 X(D 1 )! I 2. f'2 X (n 2 )-f 1 X(n 1 ) (f 2 X{n 2 )-f 1 X(n 1» I.2366'2 I ~-~ I 76:;:;8 \ I :f 2 x(n ) -fix(n ) 'L-lL ~ -ll 2 1.~ -1 2.~ a~ 1).,2 I '2 IX(D 2 ) If 2 X(n 2 ) I I a 2 x(d 2 ) I I "':;-A I t 3X(D )-f 2 2 f 3 X (n.:;) -f2x(n ) (f:;x 2 en :;)-f2x(n,) 2 X(n ) '" _~ a 3 1" If:; X(n,) t:;x(n:;) I &3 x {n 3 } I f4x(n4)-f~(n:;) (f4x(n4}-f:;x(d:;»2.141:;5 I.59:;43 ;\.4-"' f 4 X (D4) -f,x(n,) "'4-"'3 I "'4-"'3 a4 IA !~ 11'4 IX(n4) If 4 X(n4) I t a4x(d~). f,x(d 5 ) -:f4x(n 4 ) (f 5 X(n 5 )-f4x(n4» ~-A f 5 X("5)-f4X(D4) ~-A4 ~-A4 a 5 l~ I If, X(D 5 ) f 5 X(n 5 ) I a 5 x(d 5 ) f 6X(n6)-:f 5 X(n 5 ) (f6x(n 6 )-f 5 X(D 5» "'6-"'5, :; :; ' f 6X(n6)-f 5 X(n 5 ) I >"6-~ ~6-~ 2

30 Calculating Scheme for So (ContLuUen) a '1~~ 1'6 x(n6) 1'6 x (n6) ~ ts A.-r-1I. 6 a 7 ;" f 7 x(d ) fr (n ) 7 f't -f 6 x (f't(n 7 )-f6x(n 6» f -f r 6 x (n 7 ) (n6) (D,) (n6) A.-r-A6 '1-1\ a4900 f x -fr ) fax(ns) -fr (n ) (fak(na)-f 7 rcn7» \ (DS) (n7 1\s-~ "a-a.r aa Aa f S x fsx(oa) -7.16a36 (oa) '9X(D )-f8x(n8) (1'9X(09) -1'ax(D» ~9-1I.S f x -f~ ) 9 a 9 (n ) ( \9-1 '8 A 9 -hg - a6 x (n6) ar (n 7 ) aa x (08) a ~ f'9 x(n ) f 9 X(n 9 ) a 9 x 9 (D ) 9 C:. l' 10 x(n -1' x ) (f10x(d10)-f9x(09».01~ ) 9 (n 1"10-1 ' l' x -1' x ) 9 10 (D 10 ) 9 (n 9 :'10-~ [' a 10! flo Pt(n 10 ) f 1<f(n 10 ) a x ) 10 (n 10 I i f C: C.co A11-" f1<f(n ) 10 IAll i -1' 10 x (n ) 10 x(n ) l-~o 1-11,10 Total it e- e

31 30 = ?62 x / <;. / 9 x =

32 31 REFERENCES 1_7 Maguire, B. A., Pearson, E. S. and Wynn, A. H. A., "The time intervals between industria.1 accidents," Biometrika, Vol. 39 (1957). pp _7 Mosteller, F., "On some useful 'inefficient' statistics," Annals of Mathematical Statistics, Vol. 17(1948). pp _7 Neyman, J and David, F. N., "Extension of the Markoff's theorem on least squares," Statistical Research Memoir~, Vol. 1(1936). pp _7 Ogawa., J., "Contributions to the theory of systematic sta.tistics, I," Osaka Mathematical Journal, Vol 3(1951). pp _7 Ogawa, J., Determination of optimum spacings for the estimation of the scale parameter of the exponential distribution based on sample quantiles.typew~ltten manuscript a~ th~ Depar~nt <JfBtostat~CS'-f SChool -Ofl"'Ptlblie Health, University of North -Carolina, Chapel J{iJ:l, North Carolina -

33 32 ACKNOWLEDGEMENT The author wishes to" express his thanks to Professor B. G. Greenberg and Dr. A. E. Sarhan of the Department of Biostatistics, School of Public Health, University of North Carolina for their kind encouragements, essential suggestions and discussions given to him while this paper was prepared. The authori s thanks are also due to Professor HSl:cHd 'Hotelling and Dr. David ':8; :.Dunc'anr.0f the, Udver,j-ity.of North~C.grolina for their v.alua.bl~"comments on this paper.

A NOTE ON CONFIDENCE BOUNDS CONNECTED WITH ANOVA AND MANOVA FOR BALANCED AND PARTIALLY BALANCED INCOMPLETE BLOCK DESIGNS. v. P.

... --... I. A NOTE ON CONFIDENCE BOUNDS CONNECTED WITH ANOVA AND MANOVA FOR BALANCED AND PARTIALLY BALANCED INCOMPLETE BLOCK DESIGNS by :}I v. P. Bhapkar University of North Carolina. '".". This research