TESTING FOR SERIAL CORRELATION BY MEANS OF EXTREME VALUES ABSTRACT

Transcription

1 RT&A # 4 (23) (Vol.2) 211, December TESTING FOR SERIAL CORRELATION BY MEANS OF EXTREME VALUES Ishay Wessma Faculty of Idustral Egeerg ad Maagemet Techo - Israel Isttute of Techology e-mal: erw1@e.techo.ac.l Dedcated to the hudredth brthday of Bors Gedeko ABSTRACT The largest sacg of a samle s suggested as a ossble test-statstc to detect seral deedece (correlato) amog the data. A ossble alcato s testg the qualty of radom umber geerators, whch are so mortat the study of systems relablty. We comare ts erformace to the Kolmogorov-Smrov test because of ther smlar ature oe s based o extreme dstace betwee order statstcs, the secod o extreme dscreacy betwee the emrcal dstrbuto fucto ad the theoretcal oe. The tests are aled to several models wth seral deedece. Secal atteto s gve to a autoregressve model. Based o Mote Carlo smulatos, the largest sacg s more owerful for moderately large samle sze, over 5, say. A surrsg coecto to extreme values s dscovered, amely, that the lkelhood-rato test, whch s most owerful uder the autoregressve alteratve, s based o lower extremes. Keywords: Autoregressve model, Bomal model, Kolmogorov-Smrov test, Largest sacg, Lkelhoodrato test, Mote Carlo methods, Movg-max model. 1 INTRODUCTION AND MOTIVATION Suose we have a samle X 1, X 2,...,X from some cotuous dstrbuto fucto F. Suose further that F s the uform dstrbuto over the ut terval [, 1] (f ot, we relace the X by F (X )). Uder deal codtos we exect that the samle s a d samle, but we susect that the data at had exhbt some seral correlato amog cosecutve observatos ad we wat to ut t uder a statstcal test. So, let H deote the ull hyothess of "d-uform". If the alteratve s "ot d-uform", the, as J.E. Getle says, "ths alteratve s ucoutably comoste ad there caot be a most owerful test. We eed a sute of statstcal tests. Eve so, of course, ot all alteratves ca be addressed" (Getle (23), age 71). I the cotext of radom umber geerato L'Ecuyer & Smard (27) offer a comrehesve battery of tests called TestU1. The authors state o age 4 that "the umber of dfferet tests that ca be defed s fte ad these tests detect dfferet roblems." Our am ths aer s to attack a arrower roblem, amely the exstece of seral correlato. There are may ossble models whch ossess seral correlato. To be secfc, we start wth the autoregressve model X = ρx -1 + (1 ρ)u (1 < <, < ρ < 1), (1) where { U : > } s a U,1 -d sequece ad X U. I ths settg, the roblem s to test H : ρ = vs. H 1 : ρ >. (2) I Secto 6 we deal wth the bomal model ad the movg-max model. 79

2 RT&A # 4 (23) (Vol.2) 211, December Back to Equato (1), a lot of cosecutve ars, X+1 vs. X, s very useful here. For stace, o the left lot of Fgure 1, we have = 2 ars wth ρ = ad o the rght, the same wth ρ =.1. I the latter, the sloes below ad above the data are qute vsble. But as ρ decreases to, detecto of exstece of a seral correlato by the huma eye becomes more ad more dffcult. =2, =. =2, = Fgure 1. Pars of Successve Numbers, X +1 vs. X Ths aer evolved as a result of the author's emrcal observato that the largest sacg (LS) of a samle (see a formal defto Secto 2) s qute effectve detectg seral correlato the samle (see O ad Wessma (211)). I order to study ts erformace wth resect to other test statstcs, we comare t to the Kolmogorov-Smrov (KS) test. The KS test s chose ot oly because t s so wdely used as a goodess of ft test, but also because of ts smlar ature to LS. That s, whle KS dstace s the extreme vertcal dstace betwee the emrcal dstrbuto ad the uform dstrbuto, LS s the extreme horzotal dstace betwee cosecutve order statstcs. We also comare the two tests to the erformace of the samle seral correlato (SSC), the least squares estmator of ρ. Uder ormalty the latter s the atural oe to use ad t would be terestg to comare the erformace of the frst two tests to the erformace of SSC uder the reset set-u. The comarsos terms of ower are reseted Secto 3. We the ask what s a most owerful test for the ull hyothess (2). It s somehow surrsg that the aswer s aga a extreme statstcs. I Secto 4 we defe the trasformed data {T 1 : 1 < < }, whch are d, ad show that the lkelhood-rato test (LRT) (whch s most owerful) s based o m 1<< T 1. I Secto 5 we dscuss smlar ssues uder the autoregressve model of order k. Two more models whch exhbt seral deedece are troduced Secto 6 ad the erformace of the LS ad KS tests aled to these models are dscussed. Remark 1. The {X } as defed by (1) are margally ot U [, 1]-dstrbuted f ρ >. But f the {U } are dscrete uform (rather tha cotuous uform), the {X } ca be uform too: Suose Y 1 ad Y 2 are both U [, 1]-dstrbuted ad the error term ε s a radom varable, deedet of Y 1. Suose further that for some ρ > oe has 8

3 RT&A # 4 (23) (Vol.2) 211, December Y 2 = ρy 1 + (1 ρ)ε. What are the codtos o ρ ad ε for ths to hold? Lawrace (1992) roves that ρ = k 1 for some teger k > 2 ad ε must be uform over the set {, 1,, k 1}/(k 1). Hece, for very large k, ε s also (aroxmately) U [, 1]-dstrbuted. I fact, comuter geerated "radom umbers" are of ε -tye. 2 ASYMPTOTIC BACKGROUND Largest sacg. Let Y 1 < Y 2 < < Y be the order statstcs of {X 1,X 2,, X } ad Y, Y +1 be gve by, 1 resectvely. Defe the samle sacgs V = Y Y 1 ( =1, 2,, + 1). Darlg (1953) studes roertes of the samle sacgs (uder H ) the cotext of a radom artto of a terval. I artcular he gves the exact jot dstrbuto of (V m, V max ), 1 j 1 PV { m x, Vmax y} ( 1) {(1 ( 1 j) x jy) }. j j Puttg x = we obta the Whtworth (1897) result 1 j 1 PV { max y} ( 1) {(1 jy) }. j j (3) ad uttg y = 1, we get P{V m > x } = {1 ( + 1)x} ( < x < 1/( + 1)). Darlg gves also the asymtotc jot dstrbuto lm x y log( 1) y P Vm, Vmax ex( ) (, ). 2 x e x y ( 1) 1 (4) We lear from ths equato that Vmax s asymtotcally Gumbel dstrbuted ad Vm s asymtotcally exoetal. I the otato of Gedeko (1943), the asymtotc dstrbuto fuctos of V max ad V m are, resectvely, Λ(x) ad 1 Ψ α (x) wth α = 1. These results ca be exlaed by the well kow fact, that f E 1, E 2,, E +1 are deedet ut-exoetal ad T +1 s ther sum, the D ( E1, E2,..., E 1) 1 ( E1, E2,..., E 1) ( V1, V2,..., V 1), T T deedet of T +1. Sce T +1 /( + 1) 1 a.s., the lmt (4) follows. Wess (1959, 196) wrtes that the statstcs R = V max V m ad S = V max =V m have bee roosed to test H whe the alteratve s that the {X } are d from some df F ( F ). He the shows that (asymtotcally) the test based o S s equvalet to the test based o V m aloe, ad the test based o R s equvalet to the oe based o V max aloe (small values of V m are crtcal, large values of V max are crtcal). He farther shows that the test based o V m s ot cosstet whle the oe based o V max s admssble ad cosstet uder ay fxed alteratve. Ths s a good reaso to check how well V max erforms at the reset set-u comarso to other ossble tests. 81

4 RT&A # 4 (23) (Vol.2) 211, December I all the tests that follow, the sgfcace level s α =.5. I artcular, the LST rejects H whe V max > v (.95), where v () s the -quatle of V max, determed by (3). Sce our study cludes small values of, we do ot rely o the asymtotc dstrbuto. Kolmogorov-Smrov dstace. Oe of the most oular tests of the ull hyothess s the Kolmogorov-Smrov test (KST). Gve a samle, we defe the emrcal cumulatve dstrbuto fucto F by 1 F ( x) I{ X x} ( x 1), 1 where I{A} s the dcator of the evet A. The Kolmogorov-Smrov dstace s defed by D su x1 F ( x) x. The KST rejects the ull hyothess wth sgfcace level α =.5 f D > d (.95), where d () s the -quatle of D. The exact dstrbuto fucto of D s too comlcated to wrte a closed form. The asymtotc dstrbuto s gve by lmp { D x} 2 1 ( 1) 1 2 ex( 2 x 2 ). The quatles d (.95) are well tabulated for < 5. For larger, t s suggested to use the aroxmato d (. 95) 1.36 /. However, we refer to use estmators of d (.95) based o Mote Carlo smulatos of 1 6 relcatos of samles of sze. They gve more accurate F X_ Fgure 2. Examle of F, = 6 results, the sese that the emrcal owers uder H are closer to the omal level of 5%. Samle seral correlato. As metoed above, the SSC s the least squares estmator of ρ. Uder ormalty t s also the maxmum lkelhood estmator (MLE) of ρ (the MLE our set-u, 82

5 RT&A # 4 (23) (Vol.2) 211, December wth uform errors, s gve Secto 4). The exact dstrbuto s ot kow but by Whte (1961), SSC s asymtotcally ormally dstrbuted wth mea ( )ρ ad varace equal to 1 (1 ρ 2 ) + 2 (1ρ 2 1), (gorg terms of order 3 ). Based o SSC, H s rejected whe SSC> s (.95), the.95-quatle. As the case of KS, the quatles were estmated by N = 1 6 relcatos of samles of sze. It turs out that the asymtotc aroxmatos are qute good (relatve error < 1%) whe > 5. 3 POWER COMPARISONS I ths secto we reset the Mote Carlo results (usg the software R) cocerg the ower of each test. For each selected ad ρ, we geerated 1 5 samles of sze accordg to the autoregressve model (1). The emrcal ower reorted s the roorto of samles for whch H H s rejected. At a early stage of our study we thought that LS ad KS dstace mght be hghly correlated, because f LS s large, so must be at least oe of the adjacet vertcal dstaces, ether at the rght or at the left of the largest sacg as see Fgure 2. Table 1 shows the results for = 1 ad 1. We also cluded the relatve frequecy of smultaeous rejecto by LST ad KST (uder Both) ad ther correlato (uder r(v,d)). It aears that they are deed qute ostvely deedet. For = 1 the LST s sueror to the KST, feror to the SSC test. For = 1, LST s by far sueror to the other two tests. To stregthe the mresso that as creases, the suerorty of LST becomes more ad more aaret, we ra smlar smulatos for several samle szes (aga 1 5 relcatos for each ar (, ρ)). The results are gve tables whch aear the Aedx. Based o these tables, we roduced Fgure 3. The blue grahs corresod to the lkelhood rato test, whch s obvously most owerful (dscussed detal Secto 4). The other three are ad hoc tests aled to a artcular model. The tred s clear for = 1, 2 the order ( terms of ower) s SSC, KST, LST. For = 5, 1 the order s SSC, LST, KST. For > 2 the order s LST, SSC, KST. Moreover, the LST ower fucto, for large, teds to be very stee ad aroaches 1 very fast. Table 1. Emrcal Powers, α =.5, = 1, 1 = 1 = 1 ρ SSC V max D Both r(v,d) ρ SSC V max D Both r(v,d)

6 RT&A # 4 (23) (Vol.2) 211, December alha=.5, =1 alha=.5, = alha=.5, =5 alha=.5, = alha=.5, =2 alha=.5, = alha=.5, =1 alha=.5, = alha=.5, =5 alha=.5, = Fgure 3. Power Fuctos, Autoregressve Model, LRT (blue), LST (black), KST (red), SSC (gree) 84

7 RT&A # 4 (23) (Vol.2) 211, December We ca cofdetly fer that the LST s sueror to the KST the resece of seral correlato. However, uder a dearture from uformty, reservg the deedece, the KST mght be sueror to the LST. For examle, we took samles of d β(γ, 1), amely X = U 1/γ. Aga, for each ar (, γ), we geerated 1 5 samles of sze wth the roer γ. The emrcal owers are show Fgure 4. The suerorty of the KST over the LST s evdet. The KST s qute close to the most owerful test (the blue grah). The latter s the LRT for ths model, amely, reject "γ = 1" whe 2 Σ log X < 2 2 (.5) f the alteratve s γ > 1, or whe 2 Σ logx > 2 2 (.95) f the alteratve s "γ < 1". The ower of the LRT s comuted drectly from the χ 2 -dstrbuto, o eed for smulatos. alha=.5, =1 alha=.5, = /gamma /gamma alha=.5, =1 alha=.5, = /gamma /gamma Fgure 4. Power Fuctos, Beta Model, LRT (blue), LST (black), KST (red) 4 LIKELIHOOD-RATIO TEST I the revous secto we reseted some evdece favor of the largest sacg whe the alteratve s the autoregressve model. However, what s the most owerful test for ths alteratve? By the Neyma-Pearso theory, the aswer s the lkelhood-rato test (LRT). Let X = (X 1, X 2,, X ) be defed by Equato (1). I order to comute the jot desty fucto of X, codtoed o U = X = x, we exress U = (U 1, U 2,,U ) terms of X, amely U = (X ρx 1 ) (1 ρ) 1 (1 < < ). The Jacoba of ths trasformato s (1 ρ). Sce < U < 1 for all, oe has for x [, 1] 85

8 RT&A # 4 (23) (Vol.2) 211, December Let f ( x) 1 I{ x x x 1 } X x 1 x (1 ) I m1 m,. x1 1 x 1 T 1, T1 m m T, X 1 X m, 1 X X 1 1 (5) the the followg facts follow from Equato (5): Fact 1. The {T 1 } are d uform o [ρ, 1]. Fact 2. The lkelhood fucto s gve by L(ρ) = (1 ρ) I{ρ < T 1m } ( < ρ < 1). (6) Fact 3. The statstc T 1m s suffcet wth resect to ρ ad t s the maxmum lkelhood estmator (MLE) of ρ. For testg H : ρ = vs. H 1 : ρ >, the lkelhood fucto s also the lkelhood-rato. A most owerful α-level test rejects H whe T 1m > c α = 1 α 1/ (7) ad the ower s gve by (1 ) ( ) 1 f c, f c. So, fact, the LRT s based o the mmum of 2 ratos. The blue grahs Fgure 3 rereset the ower of the LRT. No doubt the LRT s sueror to all other tests, but we ote ts close roxmty to the LST grah for large. If oe desres π α (ρ) = 1 for ρ > ρ, oe eeds Table 2 resets the (smallest) requred samle sze for α =.5. log. (8) log(1 ) 86

9 RT&A # 4 (23) (Vol.2) 211, December Table 2. Samle Sze Requred for Power 1, α =.5 > m m Sce c α = log α + O( 1 ), for large, π α (ρ) = 1 for ρ > ( log α)=. 5 AUTOREGRESSIVE MODEL OF ORDER k Suose we wat to rotect ourselves agast seral correlato of hgher order. For ths urose we assume that the data have bee geerated by a autoregressve model of order k, amely X = ρ 1 X 1 + ρ 2 X ρ k X k + ηu (1 < < ), (9) where { U : > k + 1} s a U(, 1)-d sequece ad X = U for <, ρj >, 1 1. k j1 j The goal s to test whether ρj = for all j vs. the alteratve that ρj > for at least oe j, amely, H : max vs. H : max. j j 1 j j The jot desty fucto of X = (X 1, X 2,, X ) at x [, 1], codtoed o { X x : k1 } s gve by k k 1 1 f x I x x x X j j j j j j 1. (1) Ufortuately, a suffcet statstc of low dmeso, as the case k = 1, does ot exst. However, f we defe X 1 X Tj m, (1 ), Tj m m 1 Tj, X j 1 X j The, wth robablty 1, j T j m for 1 < j < k. Hece, t s reasoable to use T * = max 1<j<k T jm as a test statstc,.e., to reject H whe * * T C tk(1 ) - quatle of T * uder H. We ote that uder H, wth each vector T j = (T j1,, T j ), the T j are d U (, 1) radom varables. Moreover, T j ad T j'' are deedet, excet the case where = ', mlyg that the T jm are deedet. Ths s why the dervato of c α a closed form s too comlcated ad we must resort to Mote Carlo methods. However, oce we 87

10 RT&A # 4 (23) (Vol.2) 211, December determe c α for gve k ad, we ca guaratee ower 1 for all alteratves wth max ρ j > c α. I order to comute c.5 we geerated 1 6 samles for each combato of = 1, 1, 1 ad k = 1, 2,..., 1. Here we reort, Table 3, the asymtotc values of c.5 wth 3 sgfcat dgts. Table 3. Asymtotc Values of c.5 k c.5 k c For stace, suose we la to ru a Mote-Carlo method based o radom umbers geerated by a secfc software. Suose further that we wat to be sure that there s o ostve seral correlato u to lag k = 8 ad that we ca tolerate correlatos below ρ = 1 4. The, we have to geerate a samle of sze > 4.62/ρ = 462 order that max 1<j<8 ρ j > 1 4 wll be detected wth robablty 1. Remark 2. The resources requred to carry out ths kd of a test are (almost) free. For stace, S-PLUS or R, oe commad does the job: "x=ruf(462)" ad t takes a fracto of a secod. Remark 3. A secal case of our model s whe t s kow that ρ j = for 1 < j < k 1 for some k ad we wat to test whether ρ k =. The a test based o T km has all the roertes as the test based o T 1m dscussed Secto 3. Although t s hard to comete wth T *, we stll wsh to exlore the erformace of the tests cosdered Secto 2. Here, stead of the samle seral correlato SSC, we use the sum of squares for regresso (SSREG) as the test statstc. Namely, X = (X 1,, X ) s the deedet varable ad X j = (X 1 j,, X j ), 1 < j < k, are the regressors. I Table 4 we reset some Mote Carlo results for k = 2, based o 1 5 relcatos of samles of sze = 1. The cocluso s smlar to the oe of the case k = 1. Besde the clear domace of T *, we ote that the erformace of SSREG s very oor. The reaso could be that the devates from the model are uform rather tha ormal. The largest sacg test erforms better tha the other two. Table 4. Emrcal Powers, k = 2, α =.5, = SSREG V max D T *

11 RT&A # 4 (23) (Vol.2) 211, December 6 TWO MORE MODELS We wsh we could clam that the LST s a owerful test to detect deedece geeral. For ths, oe has to exame ts erformace agast all kds of deedeces a mossble msso. We have exermeted wth several models wth seral deedece ad the cocluso s smlar excet for very small samle szes, LST s more owerful tha KST. Here are two examles. Bomal model. Let U, U 1, U 2,... be d uform as before ad let B 1, B 2,... be a Beroull sequece wth arameter, deedet of the U-sequece. The bomal sequece Y s defed by Y = B Y 1 + (1 B )U ( > 1, Y = U ). The margal dstrbuto of each Y s U [, 1], the frst seral correlato s ad so s P{Y = Y +1 } =. Clusters of equal eghbors are of radom (geometrc) legth (see Fgure 5 for a scatter ots (, Y )). Movg-max model. Let ξ 1, ξ 2,... be a sequece of d β(k 1, 1) radom varables, where k s a fxed ostve teger. Let Z = max{ξ, ξ +1,..., ξ +k 1 } ( > 1). The Z-sequece s called a movgmax sequece of order k. For each, Z s U [, 1]-dstrbuted but eghborg values are deedet. Uer extreme values aear clusters of sze k, whch mly that the extremal dex s equal to k 1. For k = 2, the frst seral correlato s 3/7 ad P{Z = Z +1 } = 1/3. The two lots Fgure 5 look very smlar. I both cases, the exereced racttoer wll reject the deedece hyothess just o the bass of the fact that for cotuous radom varables, the robablty of a te s. We brought these cases to see how well the LST ad the KST detect the deedece. Bomal, =.333 Movg Max(2) Fgure 5. Scatter lot for the two models Fgure 6 shows the (emrcal) ower fuctos of the two tests aled to the movg-max models of order k = 2 ad k = 3. The suerorty of LST over KST s evdet, moreover, LST s eve ot cosstet ths case. A smlar Mote Carlo study was carred out o the bomal model. The results are show the Aedx. The cocluso s very smlar to the cocluso 89

12 RT&A # 4 (23) (Vol.2) 211, December regardg the autoregressve model,.e., for > 1, LST s sueror to the KST, ad as creases, t becomes more ad more so. Movg Max(2), alha=.5 Movg Max(3), alha= Fgure 6. Power Fuctos, Movg-Max Model (logarthmc scale), LST (black), KST (red) Remark 4. The bomal model has some resemblace to the autoregressve model (where ρ s relaced by a radom varable wth mea ). Suose we aly the LRT of the autoregressve model to a samle from the bomal model wth some >. The, for all such that B =, the T 1 are (as Secto 4) d uform o [, 1]. For all such that B = 1, oe has T 1 = 1. Hece, gve S (1 B ) 1, P{T 1m > 1 α 1/ } = α S/. Sce S/ q = 1 a.s. as, the ower teds to α q (=.675 for α =.5 ad =.1). Alyg the same test to data from the movg-max model yelds eve lower ower. 7 CONCLUSIONS The ma theme of the aer s to show that the largest sacg of a samle s sestve to seral correlato ad s qute owerful detectg t, more owerful tha the Kolmogorov- Smrov dstace. The ooste s true uder deedece but the true dstrbuto s dαeret from the ull dstrbuto. Of course, whe the data are geerated by a secfc (kow) arametrc model, ad there exsts a most owerful test, oe should use that test. We wet to detal Sectos 4 ad 5 because we foud t terestg to lear (for the frst tme the statstcal lterature, to the best of our kowledge) that the MLE of a seral correlato s a (lower) samle extreme, ad a test based o t s most owerful. The overall message s clear. f you worry about seral deedece your data ad you caot assume a artcular model, the LST s a reasoable test to use. 8 REFERENCES Darlg, D.A. (1953). O a class of roblems related to the radom dvso of a terval. A. Math. Statst. 24: Getle, J.E. (23). Radom Number Geerato ad Mote Carlo Methods. New York: Srger. Gedeko, B.V. (1943). Sur la dstrbuto lmte du terme maxmum d'ue sére aléatore. A. Math. 44: Lawrace, A.J. (1992). Uformly dstrbuted frst-order autoregressve tme seres models ad multlcatve cogruetal radom umber geerators. J. Al. Prob. 29: L'Ecuyer, P. & Smard, R. (27). TestU1. A C lbrary for emrcal testg of radom umber geerators. ACM Trasactos o Mathematcal Software. 33, No. 4, Artcle 22. 9

13 RT&A # 4 (23) (Vol.2) 211, December O, S. & Wessma, I. (211). Geeratg uform radom vectors over a smlex wth mlcatos to the volume of a certa olytoe ad to multvarate extremes. Aals of Oeratos Research. 189: Wess, L. (1959). The lmtg jot dstrbuto of the largest ad smallest samle sacgs. A. Math. Statst. 3: Wess, L. (196). A test of ft based o the largest samle sacg. J. Soc. Idust. Al. Math. 8: Whte, J.S. (1961). Asymtotc exasos for the mea ad varace of the seral correlato coeffcet. Bometrka 48: Whtworth, W.A. (1897). Choce ad Chace. Cambrdge Uvesty Press. APPENDIX The emrcal owers of the Mote Carlo smulatos for the autoregressve model are gve below. Each etry s a result of 1 5 relcatos. The grahs Fgure 3 are based o Table Table 5. Emrcal Powers, Autoregressve Model = 1 = 2 ρ SSC V max D ρ SSC V max D = 5 = 1 ρ SSC V max D ρ SSC V max D

14 RT&A # 4 (23) (Vol.2) 211, December = 2 = 5 ρ SSC V max D ρ SSC V max D = 1 = 2 ρ SSC V max D ρ SSC V max D = 5 = 1 ρ SSC V max D ρ SSC V max D

15 RT&A # 4 (23) (Vol.2) 211, December The followg lots show the emrcal owers of the Bomal Model, based o 1 4 relcatos for each ar (, ). alha=.5, =1 alha=.5, = alha=.5, =5 alha=.5, =1 alha=.5, =1 alha=.5, = alha=.5, =5 alha=.5, = alha=.5, =2 alha=.5, = Fgure 7. Power Fuctos, Bomal Model, LST (black), KST (red) 93