Conditional confidence interval procedures for the location and scale parameters of the Cauchy and logistic distributions

Transcription

1 Biometrika (92), 9, 2, p. Printed in Great Britain Conditional confidence interval procedures for the location and scale parameters of the Cauchy and logistic distributions BY J. F. LAWLESS* University of Manitoba SUMMARY Conditional confidence interval estimation procedures for the location and scale parameters of the Cauchy and logistic distributions are discussed. There are two aims: The first is to discuss computational aspects of using the conditional confidence interval approach, and the second is to give a rough numerical comparison of the conditional confidence interval procedures and the unconditional confidence interval procedures for the parameters of the Cauchy and logistic distributions, discussed recently by Haas, Bain & Antle (9), and Antle, Klimko & Harkness (9), respectively. Some key words: Ancillary statistics; Cauchy distribution; Conditional inference; Confidence interval procedures; Logistic distribution; Location and scale parameters.. INTRODUCTION Many papers have been written concerning the estimation of location and scale parameters for various distributions; two distributions of particular interest are the Cauchy distribution and the logistic distribution. Almost all the work done on estimating location and scale parameters for these distributions has been concerned with so-called 'point' estimation procedures. For example, it is well known that the sample mean is not a good estimate of the location parameter of the Cauchy distribution, and many papers have been written dealing with the problem of constructing point estimates for the location parameter; Barnett (9a) discusses several of these. Likewise, many papers have been written giving point estimates for the location and scale parameters of the logistic distribution. Recently, however, Haas et al. (9) and Antle et al. (9), respectively, have considered the problem of constructing interval estimates for the location and scale parameters of the Cauchy and logistic distributions, and have provided tables which allow the easy computation of confidence intervals. In particular, consider the Cauchy distribution F(x; a,/?) = i + itan-^-a)//?} (-oo<a:<oo), () JU It with location and scale parameters a andfi. Let a and /?be the maximum likelihood estimates for a and /?, computed from a complete sample of size n. Then it is readily shown that ni(& a)lfi, ni(a a)/ft and fi\fi are pivotal quantities whose distributions depend only on n. Percentage points for the distributions of these three pivotals cannot in general be found analytically, but can be determined via Monte-Carlo methods. Haas et al. have done this and give tables of percentage points which can be used in the usual way to construct confidence intervals for cc and ft. * Now at University of Waterloo. Downloaded from at Penn State University (Paterno Lib) on March, 2

2 8 J. F. LAWLESS Likewise, if one considers the logistic distribution ]-i (-oo<a;<oo), (2) and a and ft are the maximum likelihood estimates for a and /?, based on a complete sample of size n, then the comments in the preceding paragraph concerning nl(a a)//?, ni(a a)/'ft and ft/ft apply again. Antle et al. have computed percentage points for these pivotal quantities by Monte-Carlo methods, and present tables which allow confidence intervals to be constructed for a and ft. There is, however, some controversy concerning the interval estimation of location and scale parameters, and many statisticians would use an approach different from that considered by the authors of the above two papers. This, of course, includes those who would employ here a Bayesian, or fiducial or structural probability approach to the estimation problems, but also many who prefer the classical 'frequency' framework. From the non- Bayesian point of view, it can be noted for the two distributions above that (i) in general a and ft are not jointly sufficient for a and ft, (ii) there exist ancillary statistics. It was first suggested by Fisher (94), and since by many others (Cox, 98; Buehler, 99; Wallace, 99; Fraser, 9, 98) that inferences about location and scale parameters a and ft should be assessed conditional on the observed value of the ancillary statistic. That is, inferences concerning a and ft for the two distributions discussed here may be based on the pivotal quantities «4(a a)//?, n^{t a)//? andftift as usual; however, the distributions of these pivotals should be considered conditional on the observed values of the ancillary statistics. This contrasts with the approach of Haas etal. and Antle et al. which neglects the ancillaries and considers the unconditional distributions of the pivotals. It is well known, following from the work of Lindley (98), Welch & Peers (9) and Hora & Beuhler (9), that for the problems under discussion standard Bayesian and fiducial-structural interval estimates for a and ft are also conditional confidence interval estimates as discussed here. That is, standard Bayesian-fiducial-structural probability intervals of level y for any of the parameters in question have conditional probability y, with respect to repeated sampling, conditional on the observed value of the ancillary statistics, of covering the true value of the parameter. This equivalence exists between Bayes-fiducialstructural intervals and the conditional confidence intervals considered, but not between the former and the unconditional confidence intervals. The present paper has two aims. The first is to illustrate that, at least in the case where only one of the parameters a and ft is unknown, the conditional confidence interval approach is easily implemented with the aid of a computer. Indeed, many statisticians accustomed to working in the Bayesian, fiducial or structural framework are well aware of this. The second aim is to compare results of the conditional confidence interval approach with the unconditional, as applied to estimating the parameters of the Cauchy and logistic distributions. This can also be considered as a comparison of the Bayesian, fiducial or structural approach and the unconditional confidence interval approach. For the Cauchy distribution, a comparison is made in the two cases where only a single parameter, a or ft, is unknown and to be estimated. Random samples from the Cauchy distribution were generated, and, using the tables of Haas et al. confidence intervals having unconditional confidence levels -9 were constructed for each sample; the conditional confidence level for each interval was computed. Results for samples for each of several sample sizes n are given. For the logistic distribution, the same type of procedure is used to compare unconditional and conditional Downloaded from at Penn State University (Paterno Lib) on March, 2

3 Conditional confidence interval procedures 9 confidence intervals for a, when /? is known. Unfortunately, the tables given by Antle et al. do not allow a comparison to be made in the case where /? is unknown and a known. The discrepancy between the conditional and unconditional confidence interval approaches decreases as sample size n increases. The results given here thus also give a rough indication of how accurate the unconditional approach is likely to be if used as an approximation to the more computationally involved conditional one. 2. CONDITIONAL CONFIDENCE INTERVALS FOR LOCATION AND SCALE PARAMETERS Various arguments supporting the use of the conditional inference methods presented below have been given, and we do not discuss these here. The appropriate results are, at least in their Bayesian or fiducial structural probability form, well known, but for the sake of clarity we outline them. The presentation of the results will be brief and for the most part without comment. The theoretical arrangements for finding the conditional distributions considered were first given by Fisher (94). We outline the results for a general distribution with location and scale parameters a and /?, of the form (- <*<<»), () and particularize to consider the Cauchy and logistic distributions later. For ease of presentation, we split the results into three cases, depending on whether one or both of a and ft are unknown. 2*. a. unknown, [i known For convenience, and without loss of generality, we take /? to be equal to one. In this case the distribution of a random sample x v...,x n from () is n f(z,...,x n ;a) = Ug(x i -a). Let a be the maximum likelihood estimate for a, based on x x,..., x n, and consider the quantities Oj = x a,...,a n _ = x n _ x a. The a { form a set of n functionally independent ancillary statistics, and in the conditional approach inferences should then be made conditional on the observed values of these ancillaries. We easily find that where n l Downloaded from at Penn State University (Paterno Lib) on March, 2 J For notational convenience we include a n = x n a in (4); a n can be expressed as a function of From (4) it is clear that z t = & a is pivotal, with conditional density

4 8 J. F. LAWLESS Conditional probability statements for z x are found by integrating (4), and can be employed in the usual way to set up conditional confidence intervals for a. The conditional probability pr (l x ^ z x < I 2 \<h> > a n-i) wi u *& general differ from the unconditional probability pr (l x < z x ^ l 2 ) as discussed by Antle et al. and so the conditional confidence level for a given interval (< l 2, a Zj) will differ from the unconditional confidence level. Bayesian probability intervals based on a uniform prior distribution for a, and fiducialstructural probability intervals for a, are equivalent to conditional confidence intervals arising from (), in the sense specified in ft unknown, a known We assume without loss of generality that a =. The distribution of a complete sample of size n is n f(x,...,x n )=p-»il9(x i IP)- f=i If / is the maximum likelihood estimate forft,then a x = xjft,..., a n _ = x n _Jftaxe ancillary, and we thus consider ft conditional on the observed values of a x,...,a n _. We find that z 2 = /#//? is pivotal, and has conditional density where fi{ z 2\ a l> > a n-l) = G 2 Z 2 ~ X IT g{ a i z i), ( ) JO = Conditional confidence intervals for /? can be found from conditional probability statements for z 2, obtained by integrating (). Such conditional confidence intervals are equivalent to fiducial-structural probability intervals for ft, and to Bayesian probability intervals for /?, based on the prior distribution d/?//?. This scale parameter problem could, of course, be transformed to a location parameter problem via the transformation = log/?, y = log a;. 2-. a and fl both unknown The distribution of a sample of size n is Let a and ft be the maximum likelihood estimates for a andft; then the quantities are ancillary statistics, and the conditional approach involves making inferences conditional on the observed values of these ancillaries. Wefind that f{&,ftla,,...,a n _ 2 ) = C ^p n t g{{&-oc + aj)/ft}, () Downloaded from at Penn State University (Paterno Lib) on March, 2 where C = C^a^,...,a n _ 2 ) is the normalizing constant. It is clear that z = (& a)lft and z 4 = ft/ft are pivotal quantities, with joint conditional density + a^). (8)

5 Conditional confidence interval procedures 8 It is also of interest to consider the joint conditional density of t = z /z t = (t a)/ft and z 4, which is found to be t, z s \ ai,..., a n _ 2 ) = C z?-i fl g{(a t +t) z 4 }. (9) <i In order to make inferences about a in the absence of knowledge offt,z 4 can in principle be integrated out of (9) to yield the marginal distribution of t = (d a)jft, conditional on the observed a v..., a n _ 2. Similarly, in order to consider ft when a is unknown, we can integrate t out of (9) to yield the marginal distribution of z 4 = ft]ft. We also remark, as before, that conditional confidence intervals for a and ft, based on t and z 4, are equivalent to fiducial-structural probability intervals for a andft, and to Bayesian probability intervals for a and ft, based on a joint prior distribution dadftjft.. COMPUTATION OP CONDITIONAL CONFIDENCE LEVELS Computation of conditional confidence levels is readily effected on a digital computer. If maximum likelihood estimates are to be computed, and the likelihood function, or fiducialstructural or Bayesian posterior density, plotted, it requires little extra effort, in two out of the three cases considered, to compute desired probabilities, or conditional confidence levels. The computational problems involved are similar for the two distributions considered here. Hence we shall consider the Cauchy distribution in some detail; general comments concerning the computation of conditional confidence levels for the parameters of the Cauchy carry over to the similar problem for the logistic. Haas et al. have commented on the solution of the maximum likelihood equations in the three cases involved for the Cauchy distribution, and Antle et al. have done likewise for the logistic distribution. Our experience indicates the general reasonableness of their recommendations. A Newton-Raphson iteration scheme was used to solve the likelihood equations in all cases here. The only situation in which any complications arise is the case for the Cauchy distribution in which only the location parameter a is unknown; it is well known that the likelihood equation can have multiple relative maxima in this case. Barnett (9) has discussed this problem in detail, and a scanning procedure similar to his was employed here to first isolate all relative maxima, after which each relative maximum was found and the maximum likelihood estimate determined. A discussion of computational procedures for evaluating conditional confidence levels follows. -. Cauchy distribution; a unknown, ft known Computing conditional confidence levels here involves integrating (), with numerically over some range l x < z x < l 2. As remarked above, computation of conditional confidence levels is in this case equivalent to computation of probability levels for the Bayesian posterior, or fiducial-structural distribution for a. It is convenient for computational purposes to express the integral to be evaluated in terms of the Bayesian posterior. We have ra+x t n ^ zi < M a i> > a n-i) =\ n {l + fo-a) 2 }- **, () where f» o- x =\ mi+^-a) 2 }- ^ J -oo i = l Downloaded from at Penn State University (Paterno Lib) on March, 2

6 82 J. F. LAWLESS and the integrand is the Bayesian posterior distribution for a, based on the prior p(a)da = da. The integrand is also the fiducial-structural probability density for a, and is proportional to the likelihood function. A plot of the integrand is instructive, and approximate probabilities can be noted by visual inspection. In order to compute probabilities more accurately, numerical integration can be employed. In order that the integrand not be too small, a convenient computational procedure is to scale it by considering the relative likelihood function *(«) = n {i+(*i-a) It is then easy to evaluate numerically A x = \ R(a,)da, A 2 = B(a)da, J a-u J &-L, where L x and L 2 are chosen large enough so that the proportion of the total area under R{a) which does not lie between a L 2 and a + i L is negligible. The required probability () is then given, to a close approximation, by AJA 2. Simpson's Rule was found convenient for purposes of numerical integration; convergence was generally rapid. As mentioned above, multiple relative maxima can occur in R(a). These cause no problems as far as numerical integration is concerned. Often, especially for large n, the secondary maxima occur far from & and have negligible effect on the area under R{<x). When a sizeable secondary maximum occurs the only notable effect is that Simpson's Rule may take slightly longer to converge, as would be expected. -2. Gauchy distribution; /? unknown, a knoum Computation of conditional confidence levels involves integrating (), with numerically over some range l x < z 2 ^ Z 2. It is convenient in this case also to compute the required probability as a fiducial-structural-bayesian posterior probability for /?. We have from (), where pr (l ± < z 2 < I 2 \a x,...,u = CJ^ IH + (xjfin-i-^dp, () i = l A plot of the integrand is again instructive and yields rough approximations to probabilities by visual inspection. It is straightforward to integrate () numerically, using, for example, Simpson's Rule. As noted above, the scale parameter problem can, of course, be transformed to a location parameter problem via the transformation = log/9, y = log x. Performing the numerical integrations in this form was found for this particular problem to give convergence somewhat faster than integrating () as it is. Downloaded from at Penn State University (Paterno Lib) on March, 2

7 Conditional confidence interval procedures 8 -. Cauchy distribution; a, ft both unknown The conditional, or fiducial-structural-bayesian, analysis is more computationally unattractive in this case. Computing probabilities for the marginal distributions of t and z 4 poses a problem; it is possible to integrate (9) numerically over any rectangular region for t and z 4, but this is too complicated and time-consuming to be considered in many cases. It will generally be informative to plot contours of (9), or equivalently, contours of the joint fiducial-structural-bayesian posterior density for a and /?, which in this case is proportional to This may be of some use in hazarding a rough guess at conditional probabilities by visual inspection. Another procedure of some usefulness is to consider sections of (9). For example, in order to consider a with /? unknown, one might plot the density /( z 4 ) for a few selected values of z 4, and consider the one-dimensional inferences for each of these. This may allow a conservative estimate to be made of relevant probabilities. The unconditional approach, employing the tables given by Haas et al. is, of course, easy to apply. If n is large enough, conditional and unconditional confidence levels will generally not differ too much; see 4 for some rough guidelines. One could in this case construct unconditional confidence intervals as a good approximation to the conditional ones. -4. Logistic distribution Conditional confidence levels can be computed for the logistic distribution case, using the general results of 2 with g(x) = exp ( x) { + exp ( x)}~ 2. No problems are encountered with the numerical integration when the procedures described in - to - are used here. 4. COMPARISON OF CONDITIONAL AND UNCONDITIONAL CONFIDENCE LEVELS In order to compare the conditional and unconditional confidence interval procedures discussed here, the following was done: For each of the three cases (i) Cauchy distribution; a unknown, /? known, (ii) Cauchy distribution; a known, ft unknown, and (iii) logistic distribution; a unknown, /? known, samples of various sizes were generated on the computer. For each sample the appropriate maximum likelihood estimate was found, and the unconditional 9% confidence interval computed, using the tables of Haas et al. for (i) and (ii), and those of Antle et al. for (iii). The conditional confidence level for this interval was then computed via numerical integration. The aim here is to indicate the general pattern of the conditional confidence levels, rather than to determine any precise numerical properties of their distributions. Hence, for each of the three cases (i), (ii) and (iii), samples of each of the sizes n =,, 2 and 4 were generated. The frequency distribution of the conditional confidence levels for each sample size is given for cases (i), (ii) and (iii) in Tables, 2 and, respectively. For small sample sizes the conditional confidence levels vary fairly widely. In this respect Table, dealing with the location parameter a of the Cauchy distribution, exhibits the most variability. The maximum likelihood estimates a and ft in the three cases considered here Downloaded from at Penn State University (Paterno Lib) on March, 2

8 84 J. F. LAWLESS are efficient, and we can expect the conditional and unconditional confidence levels for each case to agree more and more closely as the sample size n increases. In the hmit as n ->oo, the conditional and unconditional inferences will be identical. This effect is seen in the tables. It is interesting to note for Table, dealing with the location parameter of the logistic distribution, that for samples as large as n = 4 the conditional confidence levels tend to cluster Table. Conditional confidence levels of -9 unconditional confidence intervals for location parameter of Cauchy distribution, scale parameter known Sample size n < Conditional confidence levels > Average conditional confidence levels Table 2. Conditional confidence levels of -9 unconditional confidence intervals for scale parameter of Cauchy distribution, location parameter known Sample size n 2 4 < Conditional confidence levels > Average conditional confidence levels Table. Conditional confidence levels of -9 unconditional confidence intervals for the location parameter of the logistic distribution, scale parameter known Sample size n 2 4 < Conditional confidence levels Average conditional confidence >-9! level Downloaded from at Penn State University (Paterno Lib) on March, 2 rather closely about the unconditional confidence level of -9, although, exceptional samples for which the conditional and unconditional confidence levels differ widely can occur. The same is true to a slightly lesser degree with Table 2, dealing with the scale parameter of the Cauchy distribution. In the case of the location parameter of the Cauchy distribution, however, there is still quite a lot of variability in the conditional confidence levels for samples of size 4.

9 Conditional confidence interval procedures 8 For the Cauchy distribution, Table of Haas et al. (9) shows that for samples of size 4 or larger for the location parameter case, the unconditional confidence levels agree very closely with confidence levels found by using the large-sample normal approximation for a. This, of course, does not imply anything directly about the agreement between the unconditional and conditional confidence levels. Indeed, as observed from Table above, for n = 4 there is still a substantial amount of variability in the conditional levels of a. The opposite effect occurs for the Cauchy scale parameters; for n = 4 the large-sample approximations are very inaccurate, whereas the unconditional and conditional confidence levels tend to agree fairly well.. SOME FURTHER COMMENTS In this paper we have, like Haas et al. (9) and Antle et al. (9), considered pivotal quantities which are functions of the maximum likelihood estimates. One can, of course, define pivotals in terms of any so-called ' equivariant' estimates for a and /?. For example, (a* a)//?, (a* a)//?* and ft*/ft are pivotals based on the best linear unbiased estimates a*, /?* for a, /?. This does not affect the conditional confidence interval estimation of a, /?; note, for example, that in the location parameter case a a* is constant, given the values of the ancillary statistics. Inferences evaluated unconditionally do, however, depend on the particular pivotal used; see, for example, Haas et al. (9, p. 4). In a number of applications in which the logistic distribution, for example, is used, it is fairly common for the data to be censored. It is readily demonstrated that if the censoring is so-called Type II censoring, so that the r x smallest and r 2 largest observations in a sample are missing, then the unconditional confidence interval approach described above is still applicable. The only consequence of the censoring is that the integrands in expressions (), () and (9) are slightly more complicated; this, however, is no problem when a computer is being used, and introduces no new difficulties. It may be noted that in order for the unconditional approach to be used when the data are censored, it would be necessary to evaluate empirically the distributions of the relevant pivotal quantities, for each particular possible pair of values r lt r 2, for each sample size n\. Finally, we remark that conditional tests of significance can be readily carried out, at least in the cases where only one of a and fl is unknown. The easiest approach is to integrate numerically the appropriate conditional density for a given sample, to determine the observed significance level. REFERENCES ANTLE, C. E., RLIMKO, L. & HABKNESS, W. (9). Confidence intervals for the parameters of the logistic distribution. Biometrika, BABNETT, V. D. (9a). Order statistics estimators of the location of the Cauchy distribution. J. Am. Statist. Ass., 2-8. BABNETT, V. D. (9). Evaluation of the maximum likelihood estimator where the likelihood equation has multiple roots. Biometrika, -. BUEHLEB, R. J. (99). Some validity criteria for statistical inferences. Ann. Math. Statist., 84-. Cox, D. R. (98). Some problems connected with statistical inference. Ann. Math. Statist. 29, -2. FBASEB, D. A. S. (9). The fiducial method and invariance. Biometrika 48, 2-8. FBASEB, D. A. S. (98). The Structure of Inference, New York: Wiley. FISHER, R. A. (94). Two new properties of mathematical likelihood. Proc. R. Soc. A 44, 28-. TTAAS, G., BAIN, L. & ANTLE, C. (9). Inferences for the Cauchy distribution based on maximum likelihood estimators. Biometrika, 4-8. Downloaded from at Penn State University (Paterno Lib) on March, 2

10 8 J. F. LAWLESS HOBA, R. B. & BITBHLEB, R. J. (9). Fiducial theory and invariant estimation. Ann. Math. Statist., 4-. LINDLEY, D. V. (98). Fiducial distributions and Bayes theorem. J. B. Statist. Soc. B, 2, 2-. WALLACE, D. L. (99). Conditional confidence level properties. Ann. Math. Statist., 84^. WELCH, B. L. & PEEBS, H. W. (9). On formulae for confidence points based on integrals of weighted likelihoods. J. B. Statist. Soc. B 2, [Received August 9. Revised January 92] Downloaded from at Penn State University (Paterno Lib) on March, 2