Cauchy Regression and Confidence Intervals for the Slope. University of Montana and College of St. Catherine Missoula, MT St.

Size: px

Start display at page:

Download "Cauchy Regression and Confidence Intervals for the Slope. University of Montana and College of St. Catherine Missoula, MT St."

Sylvia Burke
6 years ago
Views:

1 Cauchy Regsion and Confidence Intervals for the Slope Rudy Gideon Adele Marie Rothan, CSJ University of Montana and College of St. Catherine Missoula, MT 598 St. Paul, MN 5505 This paper uses computer simulations to verify several featu of the Greatest Deviation (GD nonparametric correlation coefficient. First, its asymptotic distribution is used in a simple linear regsion setting where both variables are bivariate. Second, the distribution free property of GD is demonstrated using both the bivariate normal and bivariate Cauchy distributions. Third, the robustness of the method is shown by estimating parameters in the Cauchy case. Fourth, a general geometric method is used to estimate a ratio of scale factors used in the confidence interval. The methods in this paper are an outgrowth of general earch on the use of nonparametric correlation coefficients in statistical estimations. The ults in this paper are not specific to GD and are appropriate for other rank based correlation coefficients. Key words: bivariate normal, bivariate Cauchy, Greatest Deviation correlation coefficient, asymptotic distribution This work depends in part on earlier unpublished work of Gideon and is available on his web site: Some of the references will refer to papers posted at this web site.. The Bivariate Cauchy The bivariate Cauchy is, of course, the same as the bivariate t with degree of freedom (Cornish 954, Dunnett and Sobel 954. Since this distribution is seldom used in simulations and estimation, it is discussed here. The joint, marginal, and conditional distributions are given; the procedure that was used to generate bivariate data is then eplained. Let ( t, t be an outcome for a ( T, T bivariate Cauchy random variable. The joint distribution is given in two forms, one to generate random observations and the other to see the elliptical nature of the contours. The elliptical nature of the contours, as with the bivariate normal, allows GD to be distribution free over the whole class of bivariate t distributions. f ( t, t 3 ( + t = π ( ρ ( t ρt + ( ρ ( + t 3 3 t ρtt + t =, + ( π ρ ρ for < t, t <. Cauchy Regsion 6/3/004 0:37 AM --

2 Both marginal distributions have the same standard Cauchy: f ( t = π ( + t for < t <. The conditional distribution of T is given for T. For a fied t, ( t ρt f ( t t = + ( (. ρ + t ρ + t This density is related to a standard Student t distribution with degrees of freedom by the following transformation. Let w = ( t ρ ρt + t 3, so that dw = ρ dt 3 + t, and w f ( w = + for < w <. These distributions are now used to generate bivariate Cauchy data. Let t equal the outcome of a standard Cauchy random variable, and let w be an independent outcome of a Student t with degrees of freedom. The computer package S-Plus was used in these simulations. The S-Plus commands: t < rcauchy( n and w < rt(n, were used to generate random samples of size n. After randomly generating t and w, let ( ρ ( + t t = ρ t + w. A standardized bivariate Cauchy random variable t, has been generated. This latter form also shows the regsion equation. The ( t ( ρ ( + t slope is ρ and the scale factor is. The random quantity w plays the role of the error variable in the regsion. It will be shown in the simulations that GD regsion estimates ρ and gives a nearly correct asymptotic confidence interval for this slope. In addition, GD regsion iduals are used to estimate an error scale factor, which is the above scale factor. Because the same type of estimation and confidence intervals are used for the bivariate normal, these simulations show the distribution free property and robustness of the GD simple linear regsion method.. The Bivariate Normal A method of generating bivariate normal values that parallels the above method is to generate two independent N(0, outcomes, z and z3 ; then with the correlation parameter ρ, let z = ρ z + ρ z3. The bivariate random variable ( Z, Z has zero means, variances of one, and correlation of ρ. For a more direct normal model in simple linear regsion, let X be N ( µ, where the second parameter is the standard deviation as this notation is commonly used in Cauchy Regsion 6/3/004 0:37 AM --

3 computer packages. Let ε be independent of X with a N( 0, distribution. Construct the standard model Y = α + β + ε. Thus, the conditional distribution is constructed given the and a random error. In S-Plus, this becomes: < rnorm n, µ, and y < α + β + rnorm( n, o, where n is the sample size. ( 3. The GD S-Plus regsion routines Gideon and Hollister (987 define the GD correlation coefficient. It is tated here for convenience as all work depends on it. The GD correlation coefficient is defined as follows: let the bivariate data (, y be ordered by the -data. Each data vector is then replaced by its ranks. The data is now ( e, p where e is the identity vector,,,3,.,n, and p contains the corponding ranks of the y-data. I, an indicator variable, is 0 or depending whether its epsion is false or true, pectively. Then GD (, y = GD( e, p and in the definition below the p are the components of p. j GD(, y = ma i n i j= I( n + p j > i ma n i n i j= I( p j > i There is only one GD S-Plus function that needs to be directly called to perform the work. It is written in C code and called in a S-Plus calling sequence. Work over a long time period has made the calculation fairly efficient. Given a paired data vector (, y, the GD simple linear regsion function is rgrgc (, y,0. This returns the intercept and slope, say a and b. The regsion predicted values y ˆ = a + b and the iduals y yˆ are now formed. A second regsion is performed on the sorted and sorted iduals y yˆ. In S-Plus, this done by rgrgc( sort(, sort( y yˆ,0. The slope of this regsion estimates slope estimate in classical least squa; that is, n times the estimated standard error of the n s β ˆ = n ( ˆ i. This is demonstrated in the simulations and is eplained fully in Gideon and Rothan (004. The method is very general in that any correlation coefficient can be used in the same manner. GD is being used because it is very robust and also gives good ults on data without outliers. It must be emphasized that this method of measuring the variation with the slope is necessary to make the method distribution free. One uses the actual observed sorted -data as the standard to obtain a relative measure of the variability in the iduals. The epectation is that if the iduals have the same type of distribution, then slope of the regsion line of the ordered iduals on the Cauchy Regsion 6/3/004 0:37 AM -3-

4 ˆ ordered s ( t for the Cauchy will provide a reasonable estimate of. In the s simulations that follow, the data is bivariate normal or bivariate Cauchy; and ˆ apparently this geometrical method of estimating does allow the distribution s property to work. For an alpha significance level, nearly correct α confidence interval levels are obtained. 4. Greatest Deviation Asymptotics The population parameters of GD for the bivariate normal and Cauchy distributions are found in Gideon and Hollister (000. A Taylor Series epansion was used to relate the distribution of slope to the asymptotic distribution of ρ (Gideon, Prentice, and Pyke 989. These ults are used in forming confidence intervals. Let GD ( X, Y be the population parameter of GD for the bivariate random variable ( X, Y. Then for both the bivariate normal and Cauchy distributions GD ( X, Y = sin ρ, where ρ is the correlation coefficient in the normal case, but π can only be called the correlation parameter in the Cauchy case. The usual definition of ρ does not suffice for the Cauchy as moments do not eist. The Cauchy correlation parameter ρ does eist for GD and is derived from the population definition of GD in Gideon and Hollister (000. Let F and G be the distribution functions of continuous random variables X and Y. Now let U = F(X and V = G(Y, then the function C( u, v = P( U u, V v is known as the Copula function. The population parameter of GD for ( X, Y is GD( X, Y = sup C( t, t sup( t C( t, t. 0 t 0 t For the asymptotic distribution of GD when ρ = 0, Gideon, Prentice, and Pyke (989 showed that for data vectors (, y, for a sample size n, that as n increases ngd(, y becomes N (0,. From the population parameter of GD and its asymptotic distribution, the asymptotic distribution of the estimated slope in either the bivariate Cauchy or normal is as follows: y π y ρ βˆ converges to a N ( ρ, distribution. n In the computer simulations, y ρ = is estimated directly as the slope in the GD regsion where the iduals are regsed on the -data Cauchy Regsion 6/3/004 0:37 AM -4-

5 ˆ ( rgrgc( sort(, sort( y yˆ,0, that is,. The asymptotic α confidence s ˆ interval is then this slope,, times the remaining terms in the asymptotic standard s deviation above. So the distribution-free robust confidence interval for the slope in a simple linear regsion is π zα βˆ ˆ ±, where z α is the upper α quantile of the standard normal. n s For additional background material for this section, see Gideon (99, Gideon and Rummel (99, Gideon and Rothan ( Computer Simulations for the Distribution-Free Confidence Interval The computer simulations primarily demonstrate the confidence interval for the slope in the simple linear regsion, but some correlation ults are also given. Values of ρ used are 0, 0.5, 0.75, 0.9. For simplicity, only 0 is used for the location parameters and for the scale parameters. The two distributions used are bivariate normal and bivariate Cauchy as eplained earlier. For the standard univariate Cauchy, the 3 rd quartile is ( Q = so that the median of + T is. Thus, the scale factor ( + t ρ in the simulations should vary about ρ * = ρ. Note that this is the same as in the bivariate normal case (i.e., the idual scale factor. Since the scale factor of the -variable with standardized distributions is one, the ratio = ρ and ˆ s estimates ρ. The difference between the Cauchy and the Normal is that the scale factor for the Cauchy averages ρ for individual points, the scale factor depends on the given value of t. The scale ; but factor is constant for the normal. The slopes and the correlations have the same value in the simulations since the standard distributions were used. The tables are labeled by a Roman numeral in the upper left corner. Table I shows means of 000 simulations each for a sample size of 68, ecept for the last two rows that are based on 5000 simulations. The categories are the ratio of the scale factors ulting from the simulations; the population scale factor; the intercept whose population parameter is 0; the slope whose population parameter is ρ ; the proportion of the samples whose confidence interval contains the true slope, the GD estimate Cauchy Regsion 6/3/004 0:37 AM -5-

6 of ρ, sin( π GD(, y / ; and the population correlation coefficient. It is known that the sine transformation of GD undetimates ρ ecept at zero. Ecept for the population parameters, all entries are the mean of the given number of simulations. Rows in the table alternate between Cauchy and Normal in order to view the distribution-free robustness property. Table I Ratio ˆ s ρ Intercept Slope Sample prop. 90% level Corr. Normal Cauchy Normal Cauchy Normal Cauchy Normal Cauchy Normal Cauchy The following two rows are for 5000 simulations Normal Cauchy ρ Table II reflects changes made to the confidence coefficient, the correlation, and slope; the number of simulations is kept at 000. The first two rows are for 90%, the net two are 80%, and the final two are for 50% confidence levels. It is becoming clear that for sample size 68, the confidence coefficient is a bit conservative. Note that the mean values for the ratios and those for the slope estimates are very close to their pective parameters. As epected, the direct estimate of the correlation via the GD sine transformation undetimates for ρ > 0. Cauchy Regsion 6/3/004 0:37 AM -6-

7 Table II Ratio ˆ s ρ Intercept Slope Sample prop. 90%, 80% 50% levels Corr. ρ Normal Cauchy Normal Cauchy Normal Cauchy In Table III sample sizes of 50, 75, 00, and 50 are used while the correlation remains at 0.80 and the confidence coefficient at 90%. The first two rows are from samples of size 50, the net two of 75, then 00, and finally 50. The number of simulations for each remains at 000. It seems remarkable that the Cauchy confidence interval for the slope is more accurate than the normal. Table III Ratio ˆ ρ Intercept Slope s Sample proportion Corr. ρ 90% level Normal Cauchy Normal Cauchy Normal Cauchy Normal Cauchy In Table I, the Cauchy simulations usually gave values closer to the stated 90% confidence level. The means of the simulated scale ratio estimates for both the Normal and Cauchy were very close to the population parameter, ρ. All intercept and slope means were close to the population parameters. From Table II, it appears that the proportion of confidence intervals containing the true slope is higher Cauchy Regsion 6/3/004 0:37 AM -7-

8 than the stated confidence level. Tables I - III indicate that the Cauchy generated data gives ults closer to the stated confidence level than the normal. The ults do not seem to vary much between the moderate sample sizes of 50 to An Eample of the Confidence Interval for the Cauchy Case One more run of 000 Cauchy simulations was made. The confidence coefficient was 90%, the sample size 68, ρ = β = 0. 8, and α = 0. The simulation ults for the last sample remain in S-Plus and are used for an illustrative eample. The analysis of this last sample demonstrates that no special selection process was used to make this system perform well. The summary statistics from the run for the 000 simulations follow. The mean intercept was and the mean slope was The mean of the sine transformation of GD was The proportion of the 90% confidence intervals containing the true slope was The term ρ = 0. 6, and the mean value of the 000 simulations for this term was The median of these was also always calculated and for this run it was Because the median and mean were always close, the median was not included in the above tables. For this last simulation of sample size 68, confidence intervals and scatterplots with regsion lines are given for GD and least squa. The aes are labeled ( t,t because bivariate Cauchy data is used. For GD estimation system, the slope of ˆ the regsion used to estimate relative variation was , i.e. = s This is the slope as eplained on page 3 for a regsion on sorted data. The upper 95 th π.645 percentile of the standard normal.645 is used in the term = This number times the gives the distance of the confidence interval around the slope estimate. Table IV lists the intercepts, slopes and the confidence intervals for the GD general method and for least squa. Table IV Comparison of LS and GD on one sample where n = 68, Cauchy data Intercept Slope CI lower CI upper GD LS It is readily seen that GD is, as epected, much better than least squa (LS. The LS method does not include the true slope in the confidence interval. The standard error of LS slope estimate, 4.4, and ( t t, , are given so that the confidence interval for LS can be checked. Although this paper is not a comparison of GD to other robust procedu, the L-one and rreg (robust regsion in S-Plus ults are given. The ults are in Table V, but this author does not know the distributional properties of these estimators to obtain a confidence interval. Cauchy Regsion 6/3/004 0:37 AM -8-

9 Table V L-one and robust regsion ults from S-Plus Intercept Slope L-one rreg Two scatterplots are pented. Figure has all the data and in Figure the data is tricted to those values between 5 and +5 in order to give a better view of the central area of the data. The graph of the tricted data demonstrates that, as is always the case, the GD regsion line goes through the heart of the data. There are 39 of the 68 outcomes between 5 and +5. Figure 3 shows the geometry of the GD estimate of. The line fitted to the sorted data { sort( t, sort( iduals} by the GD system has slope The robustness of the GD fit to the line can be observed because the line appears uninfluenced by the large iduals. 7. Summary Computer simulations have demonstrated the robustness and distribution-free properties of the rank based Greatest Deviation correlation coefficient in simple linear regsion. This was accomplished by comparing the computer ults for the bivariate normal and Cauchy distributions. The asymptotic properties of GD as a correlation coefficient and its etension to simple linear regsion were used. One eample was provided that demonstrated the GD geometrical method of obtaining an estimate of the variation in the slope estimator. This paper is based on the ults in two published papers. Unfortunately, much of the background material is unpublished, but it is available on the web site. The generality of this work may not be apparent without viewing the papers on the web, but any correlation coefficient can be used in the manner demonstrated in this paper. The unpublished L-one correlation coefficient that should accompany all statistical L-one methods is a prime candidate. This paper is part of a system of estimation based on the use of correlation coefficients. The web site shows some of the work, but some further eamples in times series, nonlinear estimation, and generalized linear models have not yet been posted. t Cauchy Regsion 6/3/004 0:37 AM -9-

10 References Cornish, E. A. (954, The Multivariate t-distribution Associated with a Set of Normal Sample Deviates, Australian Journal of Physics, 7, Dunnett, C. W., and Sobel, M. (954, A Bivariate Generalization of Students t- Distribution, with Tables for Certain Special Cases, Biometrika, 4, Gideon, R. A. (99, Random Variables, Regsion, and the GD, unpublished paper (URL: University of Montana, Dept. of Mathematical Sciences. Gideon, R. A., and Hollister, R. A. (987, "A Rank Correlation Coefficient Resistant to Outliers," Journal of the American Statistical Association, 8, (000, The Geometrical Definition of Greatest Deviation Correlation Coefficient and its Uniqueness, unpublished paper (URL: University of Montana, Dept. of Mathematical Sciences. Gideon, R. A., and Rothan, A. M. (004, Location and Scale Estimation with Correlation Coefficients, unpublished paper (URL: University of Montana, Dept. of Mathematical Sciences. Gideon, R. A., and Rummel, S. E. (99, Correlation in Simple Linear Regsion, unpublished paper (URL: SPACE-REG.pdf, University of Montana, Dept. of Mathematical Sciences. Gideon, R. A., Prentice, M. J., and Pyke, R (989, "The Limiting Distribution of the Rank Correlation Coefficient, GD, Contributions to Probability and Statistics, Essays in Honor of Ingram Olkin, Gleser, L.J. et al. (eds., New York: Springer Verlag, pp. 7-6 R. A. Gideon: Acknowledgments Ron Pyke, University of Washington, my Masters supervisor, and for completing the asymptotic distribution derivation and write-up John Gurland, University of Wisconsin, my Ph.D. advisor at Madison Wisconsin Mike Prentice, University of Edinburgh, for asking the question, "What is it estimating?" and allowing me a Sabbatical in Scotland This work has been in progs for many years with very few published papers available to acknowledge all the faculty and student help. These people are also listed at the web site, and I hope no one has been missed. I thank these people for all the help they have given me. They are the ones who have kept this earch alive. Cauchy Regsion 6/3/004 0:37 AM -0-

11 Figure Cauchy Regsion 6/3/004 0:37 AM --

12 Figure Cauchy Regsion 6/3/004 0:37 AM --

13 Figure 3 Cauchy Regsion 6/3/004 0:37 AM -3-

CORRELATION ESTIMATION SYSTEM MINIMIZATION COMPARED TO LEAST SQUARES MINIMIZATION IN SIMPLE LINEAR REGRESSION

CORRELATION ESTIMATION SYSTEM MINIMIZATION COMPARED TO LEAST SQUARES MINIMIZATION IN SIMPLE LINEAR REGRESSION RUDY A. GIDEON Abstract. A general method of minimization using correlation coefficients and