Correlation and Regression without Sums of Squares. (Kendall's Tau) Rudy A. Gideon ABSTRACT

Transcription

1 Correlaton and Regson wthout Sums of Squa (Kendall's Tau) Rud A. Gdeon ABSTRACT Ths short pee provdes an ntroduton to the use of Kendall's τ n orrelaton and smple lnear regson. The error estmate also uses Kendall's τ so that sums of squa are avoded. A populaton or random varable approah usng elementar slopes demonstrates a natural wa to ntrodue Kendall's τ. Ths paper uses populaton and sample onepts to fuse orrelaton and regson together nto a orrelaton estmaton sstem (CES) that allows regson to follow dretl from orrelaton. A slghtl dfferent formulaton of τ s needed to do ths. In Hollander and Wolfe (999) as well as other publatons there are separate areas on orrelaton and regson and t s not eas to see an onneton and ths paper hopes to orret that. It was ponted out b Huber (98) that lassal regson loses ts optmalt f onl one perent of the data s questonable. Sne ths s almost alwas the ase, regson wth robust propertes (mplng that a few errant data ponts do not severel affet the nferene) suh as pented here are preferable. Ke words: rank statsts, nonparametr, elementar slopes, onordane. INTRODUCTION A few short setons set the stage. Let ( X, Y ) be a ontnuous bvarate random varable and X, ) and X, ) ( Y ( Y two ndependent outomes. Assume ntet les n both the orrelaton and the smple lnear relatonshp between X and Y. The random varable

2 Y X Y X s known as an elementar slope (ELS) as t s the slope of the lne between the two ponts X, ) and X, ) ( Y ( Y. An elementar slope s alled onordant f t s postve and dsordant otherwse. The populaton verson of Kendall's τ an be formulated as the parameter Y Y Y Y Y Y τ ( X, Y ) P( > 0) P( < 0) P ( > 0). Thus τ s smpl the eess of onordant over dsordant slopes, or ve-versa. Let Y Y p P( > 0) and pd p. Note that τ les between - and +, and f τ 0, Y Y then P ( > 0) /.. POPULATION VALUE OF τ FOR THE BIVARIATE NORMAL Assume now that ( X, Y ) s a bvarate normal random varable wth means µ, µ, varanes, and ovarane ρ. Then the ELS random varable Y X Y X has a dstrbuton equal to ρ + ρ Ro where o R has the standard Cauh dstrbuton. The denst and umulatve dstrbuton funton of R are ( π ( + )) and o artan( ) + for over the real lne. The elementar slopes and orrelaton oeffent π ρ an now be related.

3 Y Y p ( > P 0) P( ρ + ρ Ro > 0) artan( ρ ρ ) +, and π τ ( X, Y ) artan( ρ ρ ) Y Y Ths an be solved for ρ n terms of P ( 0) π > p. After a wee bt of algebra the equaton beomes ρ sn( π ( p 0.5)). Now beause τ p, ths latter equaton an also be wrtten as ρ sn(πτ / ). Beause of the smmetr of R o about 0, the random varable Y X Y X s smmetr about the slope ρ. Ths s used to dsuss a natural estmate of the slope n smple lnear regson. 3. THE REGRESSION SETTING Two more theoretal relatonshps are needed. Frst the smple lnear regson model for the bvarate normal s E ( Y X ) µ + ρ ( µ ) ; that s, the slope s β ρ and the nterept s α µ. Seond s needed the dstrbuton of ρ µ Y ρ X. From propertes of the bvarate normal and standard varable transformaton theor, ths dstrbuton s normal wth mean µ and varane ρ µ ( ρ ). Note that ths dstrbuton s also smmetr about ts mean, whh s the nterept n the smple lnear regson. 4. POPULATION REGRESSION WITH KENDALL'S TAU 3

4 The regson duals are unorrelated wth the regsor varable for a orrelaton oeffent parameterθ, f b s found suh that θ ( X, Y bx ) 0. () In CES ths equaton s alled the populaton regson equaton. Solvng for b gves the slope of the populaton lnear model. It s now shown that the soluton of τ ( X, Y bx ) 0 s b β ρ ;that s τ gves the orret slope n the populaton regson equaton. To prove ths, start wth τ ( X, Y bx ) 0 and from the defnton of ( Y bx ) ( Y bx ) Y Y Y Y τ, P ( > 0) P( b > 0) P ( > b) /. Now ( ) b substtutng the dstrbuton of an ELS and for b β ρ, P( > b) Y X Y X P( ρ + ρ R0 > b) P( ρ R0 > 0) P ( R > 0) /. 0 Agan the last step follows from the smmetr of R 0 about 0. So ndeed the populaton regson oeffent s obtaned va τ and the populaton regson equaton. 5. CORRELATION AND SAMPLE REGRESSION WITH KENDALL'S TAU 5. Estmaton of Kendall's τ 4

5 For a random sample of sze n from an absolutel ontnuous dstrbuton wth data n n olumn vetors (, ), form the elementar slopes all of them. Estmate p, the probablt of onordane, b pˆ, as the number of onordant elementar slopes dvded b n. Beause p d p, pd p ˆ ˆ. Then the sample value of Kendall's τ s τ ˆ(, ) pˆ pˆ d, that s, the dfferene between the relatve number of onordant and the relatve number of dsordant elementar slopes. It follows from the populaton deas above that ρ ˆ sn( π ( pˆ.5)) sn( π τˆ / ). 5. Estmate of Slope Note that for the bvarate normal the populaton medan of ELS s ρ as gven n Seton. It s known that the sample medan from a smmetr populaton has the populaton medan as ts pont of smmetr. In the ase of the set of elementar slopes, there s not total ndependene, but one would stll epet the sample medan to var about the populaton medan. In other words, a natural estmate of the slope, ρ, s obtaned as the medan of the sample elementar slopes. It s now shown that τ gves ths estmate b usng the sample equvalent of equaton (). To have the duals and unorrelated, n equaton () replae random varables b data and obtan the sample regson equaton τ (, b) 0. () 5

6 But also τ (, b) p ( b) pd ( b) where p (b) s the fraton of ELSs that are postve for a hosen b and p d (b) s the fraton of ELSs that are negatve for a hosen b. The ELSs for the sample wth a gven b are ( b ) ( b ) b for < n. The sample regson equaton s solved f there are an equal number of onordant and dsordant ELSs; that s, when p ( b) pd ( b) whh means #{ > b} #{ < b} or that b medan{ }. Sen (968) s a good soure for a dfferent approah to slope estmaton va Kendall's Tau. 5.3 Interept After the slope b s estmated the nterept s estmated b the medan of the set of duals b, all t a. Ths s beause b s estmatng ρ and Y ρ X s smmetr about the populaton nterept µ. Then the Kendall estmate of the ρ µ lnear relatonshp s ˆ a + b. 5.4 Inferene on the Slope For nferene on β ρ, the asmptot ult n Gdeon (008) s used; t s shown there that β ˆ τ β has an asmptot N(0, π 9( ) ) dstrbuton. (Sen 968 also n develops a dfferent nferene). The quantt the Kendall regson lne, whle s estmated from the duals around s estmated b a sample varane of the regsor 6

7 varable. In the sprt of usng ust Kendall's τ to estmate all quanttes, here s the CES approah. Let z Φ ( ( n + ) ) for,, K, n where Φ s the dstrbuton funton of a N(0,) random varable. Now the z are used wth the order statsts for the duals ) and ( ( ) ) to estmate the standard devatons. Solvng the regson equatons ( () (wth the same log as above) but now wth ordered data denoted b the supersrpt o, o o o o o o τ ( z, ˆ z ) 0 and τ ( z, ˆ z ) 0, leads to the followng (Gdeon and Rothan 007): ( ) ( ) ˆ medan and z( ) z( ) ( ) ( ) ˆ medan, for < n. z( ) z( ) 6. EXAMPLES AND SIMULATIONS The above work s net arred out for the eample n Sen (968), {,,3,4,0,,8}, and {9,5,9,0,45,55,78}. Frst, the The two estmated standard devatons are are 6.34 and., petvel. The rato of the be omputed n two was. Frst b omputng τ estmated nterept and slope are 6 and 4. ˆ 7.4 and ˆ.48; the lassal ults ˆ / ˆ n the SD of the slope above an ˆ and ˆ and takng the rato. A seond wa that avods nvolvng a dstrbuton s possble wth the CES b omputng ( ) ( ) o o o rato medan dretl; that s, solve τ (, rato* ) 0. Both ( ) ( ) π ˆ gve the same ult, The estmated SD s 9( n ) ˆ π whh s * So the appromate 93% onfdene nterval (same level as n Sen) s 4 ±.8* ± where.8 s the upper quantle of a standard normal. The lower number s 3.85 ompared to Sen s 3.7 and the upper number s 4.5 7

8 ompared to 4.8; onsderng that the sample sze s onl 7, these numbers are ver lose. In hs paper Sen ponts out wh Tau (robust) should be used so t s not neessar to repeat the reasons here. A number of smulatons wth varous parameters were run n order to further authentate the asmptot dstrbuton of the slope, but onl one of eah tpe s dsussed. Frst a smple lnear regson was onstruted wth standard normal varables for a sample of sze 5 and a onfdene oeffent of 50 %. The model was ρ + ρ ε wth and ε ndependent N(0,), and so the orrelaton ρ s also the slope of the regson lne. Of 000 smulatons, 48 % ontaned the true slope ust about what s epeted. A seond smple lnear regson was run on bnomal varables. Frst was generated as a bnomal random varable based on 5 Bernoull trals wth the probablt of suess p. Then was generated b hoosng a slope and addng error wth the same Bnomal but entered b subtratng n*p, so slope* + ( B(5, ) 7.5). Agan 5 observatons were taken and a 50 % onfdene nterval onstruted. Of 000 smulatons, 63% of the onfdene ntervals nluded the true slope. Ths, of ourse, s far more than epeted. There were man ted values on both the and data for the bnomal and so the asmptot dstrbuton ma be less aurate, but the fat that the onfdene nterval level was hgher than epeted s promsng. A further motvaton for gvng ths last eample s to show that CES wth Kendall's an be appropratel used on dsrete data, ust as normal theor tehnques. In both ases, t s known that the ults are onl appromate. 8

9 7. CONCLUSION Most prattoners of statsts fous onl on least squa and sums of squa proedu and have the mpson that these are the onl eas to use and straghtforward proedu for regson analses. The ompat snthess (of populaton, sample, orrelaton, and regson) pented heren s meant to promote the use of Kendall's Tau and the CES to show that there are alternatves that are not onl eas to use but are n addton nherentl robust. Other orrelaton oeffents, both nonparametr and ontnuous, ould be studed n a smlar manner; the author has eamned a few of these, but onl Greatest Devaton Correlaton Coeffent (Gdeon and Hollster 987) has been etensvel studed. To summarze, both the populaton and sample versons of Kendall's Tau are used to eamne orrelaton and the parameters n a smple lnear regson. It should be noted that the nvarane propertes gven n Sen (968) are easl approahed through the orrelaton notons n ths paper. What, perhaps, has held up usng Kendall's and Sen's onepts n statstal analss s that ther work n orrelaton and regson has not been vewed as related muh less as a unversal sstem; orretng ths has been one of the goals of ths paper. Addtonall the advane of omputng has made ths approah ver feasble. Ths approah also leads to a ne wa to etend Kendall's Tau nto multple lnear regson; see Gdeon (008). Customzed R or S-Plus programs are gven n the append to obtan the Kendall estmates n smple lnear regson. These use the ma-mn method for dealng wth tes found n Gdeon and Hollster (987) so that estmates are avalable no matter how man tes there are. Ths method of te breakng gves the same ults as those n pakaged S- 9

10 Plus routnes but not as those n the R routnes. However, ths ma-mn te breakng method s the onl feasble method for the Greatest Devaton Correlaton Coeffent and so s more general. APPENDIX: R OR S-PLUS ROUTINES FOR CORRELATION AND SIMPLE LINEAR REGRESSION WITH KENDALL'S TAU # omputng the two unque vetors wth tes pent: the funton s tauunq tauunq <- funton(,) { n <- length() e <- :n rr <- n+ -rank() tp <- [order(,)] tn <- [order(,rr)] rkp <- order(tp,e) rkn <- order(tn,n:) out <- bnd(rkp,rkn) out } # alulaton of Kendall's tau on unque ma-mn vetors # the funton s rtau rtau <- funton(,){ ot <- tauunq(,) rkp <- ot[,] rkn <- ot[,] dp <- 0 dn <- 0 n <- length() n <- ((n*(n-))/) n <- n- for( n :n) { <- + tempp <- rkp[]-rkp[:n] tempn <- rkn[]-rkn[:n] dp <- dp + sum( tempp<0) dn <- dn + sum( tempn<0) } out <- (dp + dn)/n - out } # output s slope and nterept, funton name tauslp n postons and tauslp <- funton(,) { rat <- (outer(,,"-")/outer(,,"-")) ratv <- rat[!s.na(rat)] slp <- medan(ratv) 0

11 <- - slp * ant <- medan() <- - ant k <- rtau(,) k <- sum() k <- medan() out <- (slp,ant,k,k,k) out } REFERENCES R.A. Gdeon, The Correlaton Coeffents, Journal of Modern Appled Statstal Methods 6 (007). R.A. Gdeon, The Relatonshp between a Correlaton Coeffent and ts Assoated Slope Estmates n Multple Lnear Regson, Sankha (008) under revew. R.A. Gdeon and R.A. Hollster, A Rank Correlaton Coeffent Resstant to Outlers, Journal of the Ameran Statstal Assoaton 8 (987) R.A. Gdeon and A. M. Rothan CSJ, Loaton and Sale Estmaton wth Correlaton Coeffents, Communatons n Statsts-Theor and Methods (008) under revew. M.G. Kendall and J.D. Gbbons, Rank Correlaton Methods, 5th ed., Oford Unverst Ps, 990, or M. G. Kendall, Rank Correlaton Methods, 3rd ed., Hafner Publ. Co., 96. M. Hollander and D.A. Wolfe, Nonparametr Statstal Methods, nd ed., John Wle & Sons, 999. P.J. Huber, Robust Statsts, John Wle & Sons, 98. P.K. Sen, Estmates of the Regson Coeffent based on Kendall's Tau, Journal of the Ameran Statstal Assoaton 63 (968) Webste ontans opes of the author's unpublshed work.