AN INVARIANCE PRINCIPLE FOR SIEVE BOOTSTRAP IN TIME SERIES

Transcription

1 Ecoometric Theory, 8, 2002, Prited i the Uited States of America+ DOI: 0+07+S AN INVARIANCE PRINCIPLE FOR SIEVE BOOTSTRAP IN TIME SERIES JOON Y. PARK Seoul Natioal Uiversity This aer establishes a ivariace ricile alicable for the asymtotic aalysis of sieve bootstra i time series+ The sieve bootstra is based o the aroximatio of a liear rocess by a fiite autoregressive rocess of order icreasig with the samle size, ad resamlig from the aroximated autoregressio+ I this cotext, we rove a ivariace ricile for the bootstra samles obtaied from the aroximated autoregressive rocess+ It is of the strog form ad holds almost surely for all samle realizatios+ Our develomet relies uo the strog aroximatio ad the Beveridge Nelso reresetatio of liear rocesses+ For illustrative uroses, we aly our results ad show the asymtotic validity of the sieve bootstra for Dickey Fuller uit root tests for the model drive by a geeral liear rocess with ideedet ad idetically distributed iovatios+ We thus rovide a theoretical justificatio o the use of the bootstra Dickey Fuller tests for geeral uit root models, i lace of the testig rocedures by Said ad Dickey ad by Phillis+. INTRODUCTION The bootstra has become icreasigly oular i ecoometrics ad also i statistics ad may other alied fields+ This is i art due to the decreased comutatioal cost but aears to be mostly because of its ractical success ad wide alicability+ For a ice otechical itroductio with a extesive survey of the bootstra methodology, see Horowitz ~999!+ The method of bootstra used to be alied maily for the aalysis of cross-sectioal data, but recetly it has also become oular i time series alicatios+ I this aer, we establish a ivariace ricile for bootstra samles from liear rocesses with ideedet ad idetically distributed ~i+i+d+! iovatios+ The uderlyig time series is aroximated by a fiite autoregressive rocess of order icreasig with the samle size, ad the bootstra samles are draw from the cetered fitted residuals+ Such a bootstra is called sieve bootstra by Bühlma ~997!, because it is based o a aroximatio of a ifiite- I thak a co-editor ad two aoymous referees for useful commets ad the Deartmet of Ecoomics at Rice Uiversity, where I am a adjuct rofessor, for its cotiuig hositality ad secretarial suort+ I am also very grateful to Do Adrews ad Yoosoo Chag for helful discussios ad costructive suggestios, which have greatly imroved the exositio of a earlier versio of this aer+ This research was suorted by the Statistical Research Ceter for Comlex Systems at Seoul Natioal Uiversity+ Address corresodece to: Joo Y+ Park, School of Ecoomics, Seoul Natioal Uiversity, Seoul 5-742, Korea Cambridge Uiversity Press $

2 470 JOON Y. PARK dimesioal ad oarametric model by a sequece of fiite-dimesioal arametric models+ The sieve bootstra has bee studied earlier by Kreiss ~992!, Bühlma ~997, 998!, ad Bickel ad Bühlma ~999!+ Alog with the block bootstra by Küsch ~989!, it rovides a stadard tool for the bootstra from deedet samles+ Our work is most closely related i its aim to Bühlma ~997! ad Bickel ad Bühlma ~999!+ The former roves the bootstra cosistecy for a class of oliear estimators, whereas the latter derives a bootstra fuctioal cetral limit theorem uder a bracketig coditio+ They are of the weak form that holds oly i robability+ The ivariace ricile we establish i the aer cocers the weak covergece of the bootstra artial sum rocess to Browia motio+ Noe of the work cited reviously deals with this tye of ivariace ricile+ It is of the strog form ad holds almost surely for all samle realizatios+ With the cotiuous maig theorem, it ca be used to obtai asymtotic distributios of various bootstraed statistics without makig arametric assumtios o the uderlyig model+ Our aroach is built uo the strog aroximatio ad the Beveridge Nelso reresetatio of liear rocesses+ The ivariace riciles for the i+i+d+ iovatio ad its bootstra versio are first develoed usig the strog aroximatio of the artial sum rocess by the stadard Browia motio, ad subsequetly the ivariace riciles for the geeral liear rocess with i+i+d+ iovatios ad the corresodig bootstraed rocess are established by their Beveridge Nelso reresetatios as i Phillis ad Solo ~992!+ For the urose of illustratio, we aly our results to aalyze bootstra asymtotics for Dickey Fuller uit root tests+ I articular, we rove the asymtotic validity of bootstra Dickey Fuller tests for models geerated by liear rocesses with i+i+d+ iovatios+ The Dickey Fuller test statistics have the same limitig distributios as the corresodig bootstra test statistics obtaied by the sieve bootstra rocedure+ Oe ca therefore use Dickey Fuller tests eve whe the uderlyig models are drive by iovatios that are serially correlated, if the tests are based o the bootstraed critical values+ Thus bootstra Dickey Fuller tests ca be a alterative to the tests by Said ad Dickey ad by Phillis, which were roosed to test for a uit root i geeral uit root models+ The rest of the aer is orgaized as follows+ Sectio 2 resets some relimiary results o ivariace riciles for i+i+d+ iovatios ad their bootstra samles+ I Sectio 3 we use the results from Sectio 2 to establish ivariace riciles for liear rocesses with i+i+d+ iovatios ad for the samles obtaied from the sieve bootstra+ Sectio 4 rovides a alicatio of our results to the test of a uit root+ I articular, we develo the bootstra asymtotic theory for Dickey Fuller tests, ad we show that they are asymtotically valid for uit root models drive by liear rocesses with i+i+d+ iovatios+ Sectio 5 cotais cocludig remarks, ad the mathematical roofs are give i Sectio 6+ Fially, a word o otatio+ The stadard otatio used i robability ad measure theory is used without referece, i+e+, r, r, r L r, ad

3 SIEVE BOOTSTRAP IN TIME SERIES 47 r d imly, resectively, covergece almost surely, i robability, i L r, ad i distributio+ A time series is deoted by ~z t!, ad wheever ecessary the rage of time idex will be secified as i ~z t! ad ~z t! + 2. PRELIMINARY RESULTS Throughout this sectio, we let ~«t! be a sequece of i+i+d+ radom variables with mea zero+ It is thought to be a sequece drivig a model uder ivestigatio+ Also, i the cotext of the bootstra alicatios, it is suosed that the realizatio of ~«t! itself or its estimates rovides the values from which we resamle+ I this sectio, we first establish a ivariace ricile for ~«t! ad its bootstra versio+ Primarily, this is a relimiary ste toward a ivariace ricile for the sieve bootstra i time series develoed i the ext sectio+ Our results reseted here, however, have a wider alicability ad hece should be of ideedet iterest+ Though the existece of higher momets for ~«t! will be required later for the mai result of the aer, we simly let E«2 t s 2 at the momet+ Let a artial sum rocess of ~«t! be defied by W ( «k + () sm Here ad elsewhere i the deotes the maximum iteger that does ot exceed x+ The by the classical Dosker s theorem ~see, e+g+, Billigsley, 968! we have W r d W i the sace D@0,# of cadlag fuctios, where W is the stadard Browia motio+ The sace D@0,# is usually equied with the Skorohod toology+ However, it is more coveiet to cosider the sace edowed with the uiform orm 7{7 i the subsequet develomet of our theory+ We may actually have a stroger result tha the weak covergece i ~2!+ As a result of what is kow as the Skorohod reresetatio theorem ~see, e+g+, Pollard, 984!, we kow that there exists a robability sace ~V, F, P! suortig a rocess W ', say, ad W such that W ' has the same distributio as W ad W ' r W+ Ideed, it follows from the result i Sakhaeko ~980! that we may choose W ' satisfyig the followig coditio+ LEMMA 2++ Let E6«t 6 r for some r 2. The we have for ay d 0 P$7W ' W7 d% r02 K r E6«t 6 r, where K r is a absolute costat deedig oly uo r. (2) (3)

4 S [ 472 JOON Y. PARK The result i Lemma 2+, which is ofte called the strog aroximatio, is very useful i develoig a ivariace ricile for bootstra samles+ This ' will be exlaied subsequetly+ I what follows, we will ot distiguish W from W, ad we assume that W coverges to W uiformly+ This causes o loss i geerality, because we are oly cocered with distributioal results+ For the bootstra, we first obtai or estimate ~«t! from the samle of size ad get ~ «[ t!, say+ The we resamle from the emirical distributio of ~ «[ t!, i+e+, the distributio with oit robability mass 0 o their observed values, to get the bootstra samle ~«* t! + We may thus regard ~«* t! as the i+i+d+ samles from the emirical distributio of ~ «[ t! + Both ~ «[ t! ad ~«* t! are deedet uo the samle size, ad we may more recisely deote them as triagular arrays ~ «[ t! ad ~«* t!+ However, we will cotiue to write them as ~ «[ t! ad ~«* t! i the subsequet discussios, followig the usual covetio+ Now we suose that a realizatio of ~ «[ t! is give ad cosider the bootstra robability sace ~V *, F *, P *! coditioal o the realizatio of ~ «[ t! + For each realizatio of ~ «[ t!, bootstra samles ~«* t! are regarded formally as radom variables defied o this robability sace+ As metioed earlier, the bootstra samles ~«* t! are obtaied for each from ~ «[ t! + Uder the robability P *, the bootstra samles ~«* t! become i+i+d+ with the uderlyig distributio give by the emirical distributio of ~ «[ t! + Naturally, the exectatio with resect to P * is sigified by E * + We also use the covetioal otatios r * ad r d * to deote the covergeces i robability ad i distributio, resectively, for the fuctioals of bootstra samles defied o ~V *, F *, P *!+ I articular, if the covergeces occur for all realizatios of ~ «[ t!, the we write them, resectively, as r * ad r d * Of course, the former imlies the latter+ Wheever the samle mea «of ~ «[ t! is ozero, we assume that bootstra samles are draw from ~ «[ t «S! so that E * «* t 0 For each realizatio of ~ «[ t!, we cosider W * ( «* k, (4) s[ M which corresods to W defied i ~!, where s[ 2 E * «*2 t ~0!( «2 t + *' As before, we let W be the distributioally equivalet coy of W * i a exaded robability sace rich eough to suort a stadard Browia motio W * such that P * $7W *' W * 7 d% r02 K r E * 6«t * 6 r (5) for ay d 0 ad r 2, where K r is the absolute costat itroduced i Lemma 2++ This is ossible, agai due to the result by Sakhaeko ~980! as i Lemma 2++ Followig our earlier covetio, we will ot distiguish W * i ~4! from W *', ad we simly assume that W * ad W * are defied o the commo robability sace+

5 It is ow obvious from the result i ~5! that we ca state the followig theorem+ THEOREM 2+2+ If E * 6«t * 6 r ad r02 E * 6«t * 6 r r 0 for some r 2, the W * r d * W as r. Thus we oly eed to show ~6! to establish the ivariace ricile ~7! for the bootstra samle ~«* t!+ Note that ~6! imlies W * r * W * as a result of ~5! ad therefore W * r d * W * Clearly, the latter covergece ca be rereseted simly by W * r d * W as i ~7!, because the limit rocess W * is distributioally ideedet of the realizatios of ~ «[ t! + I the simle case that ~«t! are observable ad the bootstra samles are take directly from the emirical distributio of ~«t «S!, where «S is the samle mea of ~«t!, we have SIEVE BOOTSTRAP IN TIME SERIES 473 E * 6«* t 6 r ( 6«t «S 6 r + Therefore, it suffices to require E6«t 6 r for some r 2 to derive the bootstra ivariace ricile ~7!+ Note that coditio ~6! is satisfied uder such a momet coditio, because ~0!( 6«t «S 6 r r E6«t 6 r by the strog law of large umbers+ For the simle case cosidered earlier, Basawa, Mallik, McCormick, Reeves, ad Taylor ~99b! establish the bootstra ivariace ricile ~7! uder a weaker assumtio E«2 t + They derive ~7! directly by showig that the fiite-dimesioal distributio the Mallows metric ~for details, see Bickel ad Freedma, 98!# ad that the tightess coditio holds+ However, their aroach does ot readily exted to more comlicated rocedures such as the sieve bootstra, which we will cosider i the ext sectio+ We also refer to Ferretti ad Romo ~996! for a ivariace ricile that is comarable to ours+ Let w be a fuctio o D@0,# cotiuous with resect to the Wieer measure+ Oce the ivariace ricile ~2! is established, it follows immediately that w~w! r d w~w! (8) as a result of the cotiuous maig theorem+ It is obvious from our develomet that a similar result holds for W * ; i+e+, we may easily deduce from the bootstra ivariace ricile that w~w *! r d * w~w! (9) (6) (7)

6 I J 474 JOON Y. PARK I articular, if we aly ~8! or ~9! with ~w~x!!~t! x~t! tx~! for x D@0,#, the the limit rocess becomes the Browia bridge U~t! W~t! tw~!+ Such limit rocess aears whe we deal with the artial sum of ~«t «S! or ~«* t «S *!, where «S ad «S * are, resectively, the samle meas of ~«t! ad ~«* t!+ Also, we eed to cosider the detreded Browia motio W t ~t! W~t! ~* 0 W~s!t~s! ' ds!~* 0 t~s!t~s! ' ds! t~t! with t D@0,# m for the asymtotic aalysis of uit root models with determiistic treds+ A ivariace ricile with limit rocess W t ca be obtaied from the alicatio of ~8! or ~9! with ~w~x!!~t! x~t! ~* 0 x~s!t~s! ' ds!~* 0 t~s!t~s! ' ds! t~t! for x D@0,# + As is well kow, ~8! or ~9! yields the usual cetral limit theorem, because for x D@0,# ~w~x!!~t! x~! is a cotiuous fuctioal+ 3. INVARIANCE PRINCIPLE FOR SIEVE BOOTSTRAP We cosider a geeral liear rocess ~u t! give by u t ~L!«t, where ~«t! is a i+i+d+ radom sequece ad ~z! ( k z k + k 0 More secifically, we let ~«t! ad ~z! satisfy the followig coditios+ Assumtio 3++ We assume that ~a! ~«t! are i+i+d+ radom variables such that E«t 0, E«2 t s 2, ad E6«t 6 r for some r 4, ~b! ~z! 0 for all 6z6, ad ( k 0 6k6 s 6 k 6 for some s + The coditios i Assumtio 3+ are ot striget ad are routiely assumed i statioary time series aalysis+ Yet, they are sufficiet to establish a ivariace ricile for the sieve bootstra from the geeral liear rocess ~u t!+ The ivariace ricile that we will develo is of the strog form, which holds almost surely for all samle realizatios+ For a weak ivariace ricile that we oly require to hold i robability, we may allow r 4+ 2 Followig Phillis ad Solo ~992!, we use the Beveridge Nelso reresetatio ad write ~u t! as u t ~!«t ~ ui ui t!, (0) where u t ( k «t k k 0

7 J with k ( i i + Uder the coditio i Assumtio 3+~b!, we have ( k 0 6J k 6 as show by Phillis ad Solo ~992, + 973!+ The time series ~ ui t! is therefore well defied i both the ad L sese uder Assumtio 3+~a!+ If we let V ( u k ~s~!!w ~t! M M ~ ui 0 u () the the ivariace ricile for ~u t!, V r d V ~s~!!w, follows immediately from the cotiuous maig theorem, because we have uder Assumtios 3+ that max 6 02 ui k 6 r 0 k as show i Phillis ad Solo ~992, + 978!+ Uder Assumtio 3+~b!, we may write ~u t! as a~l!u t «t, where a~z! ( a k z k SIEVE BOOTSTRAP IN TIME SERIES 475 with ~a k! satisfyig (6k6 s 6a k 6, as show i, e+g+, Brilliger ~975!+ Therefore, it may be reasoable to aroximate ~u t! by a autoregressive rocess of fiite order, i+e+, u t a u {{{ a u t «, t + (2) I the subsequet develomet of our theory, we let be a fuctio of samle size + More recisely, we ca state the followig assumtio+ Assumtio 3+2+ satisfies r ad o~~0log~!! 02!+ We do ot imose ay lower limit to the divergece rate for, ad we may thus let it icrease as slowly as we wat+ If we also assume 0~log! 04 d r for some d 0, all of our subsequet results hold also for r 4 i Assumtio 3+~a!+ Whe there is some 0 such that a k 0 for all k 0 ad ~u t! is geerated by a fiite autoregressio, the coditios i Assumtio 3+2 are of course ot ecessary+ We oly eed to require 0 for all large + For the sake of otatioal brevity, we will deote by as before istead of the order of the aroximated autoregressio+

8 [ [ 476 JOON Y. PARK The arameters ~a k! ad s 2 ca be cosistetly estimated from the regressio ~2!+ To show this, we let ~ a[, k! be the ordiary least squares ~OLS! estimators of ~a k! for k,+++, ad defie a ~! ( a, k + Also, let s[ 2 be the usual error variace estimator+ LEMMA 3++ Let Assumtios 3. ad 3.2 hold. The it follows that max 6 a[, k a k 6 O~~log 0! 02! o~ s! k for all large. Moreover, we have s[ 2 s 2 O~~log 0! 02! o~ s! a[ ~! a~! O~ ~log 0! 02! o~ s! as r. The OLS estimators ~ a[, k! are therefore cosistet for ~a k!, k,+++,+ Moreover, as log as we let r as the samle size icreases, the autoregressive coefficiets ~a k! for k become egligible i the limit, because ( k a k o~ s! uder our coditio i Assumtio 3+~b!+ For the sieve bootstra, we first fit ~u t! with a aroximated autoregressio ~2! of order by OLS, i+e+, u t a[, u {{{ a[, u t «[, t to get the fitted residuals ~ «[, t!+ At this stage, we may use the usual order selectio criterio such as the Akaike iformatio criterio ~AIC! or the Bayesia iformatio criterio ~BIC!+ Uder AIC ad BIC, resectively, is selected such that log s[ 2 20 ad log s[ 2 log 0 are miimized+ If ~u t! is believed to be geerated by a fiite autoregressio, BIC might be referred because it yields a cosistet estimator for + See, e+g+, A, Che, ad Haa ~982!+ If ot, AIC may be a better choice, because it leads to a asymtotically efficiet choice for the otimal order of some rojected ifiite-order autoregressive rocess, as show by Shibata ~980!+ Our subsequet theory allows for such data-deedet selectio rules+ From Lemma 3+, we have log s[ 2 log s 2 O~~log 0! 02! o~ s! Therefore, if we use AIC to select, the satisfies o~ 0~ s!!, ad the coditio i Assumtio 3+2 holds if s + Likewise, BIC selects such that o~~0log! 0~ s!!, ad the coditio i Assumtio 3+2 is

9 I [ [ [ satisfied for s + Note that r uder both selectio rules, uless ~u t! is geerated by a fiite autoregressio+ The ext ste is to resamle ~«* t! from the emirical distributio of «[, t ( «, t ad geerate recursively from ~«t *! a autoregressive rocess ~u t *! of order as u * * * t a[, u {{{ a[, u t «* t + For the actual geeratio of the bootstra samles ~u t *!, their iitial values must be set+ The choice of the iitial values of ~u t *! may have a imortat effect o the actual erformace of the bootstra i fiite samles+ Also, to get statioary samles for ~u t *!, we must imose the aroriate statioarity coditios for the iitial values, or we may reeat samlig util statioarity is achieved ad throw the first geerated values away+ However, it is irrelevat i our subsequet asymtotic aalysis ad will ot be discussed ay further+ The bootstraed rocess ~u t *! may ot be ivertible i fiite samles, though it should have small robability uless the samle size is very small+ Give our results i Lemma 3+, the roblem of oivertibility i ~u t *! should vaish almost surely as the samle size icreases+ We may however refer to use the Yule Walker estimators ~see, e+g+, Brockwell ad Davis, 99, Secs+ 8+ ad 8+2!+ It is well kow that ay fiite autoregressio fitted by the Yule Walker estimators is ivertible+ All our subsequet results are also alicable for the bootstra based o the Yule Walker estimatio of the autoregressio ~2!+ Let W * ( s[ M «k * SIEVE BOOTSTRAP IN TIME SERIES 477 be the bootstra aalogue of the rocess W itroduced i the revious sectio+ The we have the followig lemma+ LEMMA 3+2+ Let Assumtios 3. ad 3.2 hold. The coditio (6) is satisfied ad W * r d * W, asr. The bootstra ivariace ricile for ~u * t! ca be obtaied from the Beveridge Nelso reresetatio, as i ~0!+ If we let [ ~! 0 a[ ~! ad u * t ( a ~! ( a, i k i u * t,

10 478 JOON Y. PARK the we may write V * ( u * k ~ s[ [ ~!!W * ~t! M M ~ ui 0 * * u (3) Note that s[ 2 r s 2 ad [ ~! r ~! as r, as show i Lemma 3++ Therefore, if we ca show that P * max 6 02 ui * t 6 d t r 0 (4) holds for ay d 0, the bootstra ivariace ricile for ~u t *! V * r d * V ~s~!!w would follow+ THEOREM 3+3+ Let Assumtios 3. ad 3.2 hold. The coditio (4) holds ad V * r d * V, asr. The bootstra ivariace ricile for the sieve bootstra has ow bee established for geeral liear rocesses with i+i+d+ iovatios+ It ca be alied to obtai bootstra asymtotics without makig arametric assumtios o the uderlyig model+ I articular, the limitig distributios of various bootstra statistics from liear time series ca be foud simly by the alicatio of the cotiuous maig theorem+ 4. SIEVE BOOTSTRAP FOR DICKEY FULLER TESTS We ow show how the bootstra ivariace ricile develoed i the revious sectio ca be alied to testig the uit root hyothesis+ Let a time series ~ y t! be geerated as y t D t x t, where ~D t! ad ~x t! deote, resectively, the determiistic ad stochastic comoets of ~ y t!+ We secify ~D t! as D t c t ' b, (5) where c t ~c t,+++,c mt! ' is a vector of determiistic fuctios of time ad b ~b,+++,b m! ' is a arameter vector+ Most commoly used such treds are c t ad c t ~, t! ' + Furthermore, we let ~x t! be give by x t ax u t (6) with a ad let ~u t! be a liear rocess secified as i the revious sectio+ 3 We may allow the iitial value x 0 of ~x t! to be ay radom variable, ad therefore we set x 0 0 i the subsequet develomet of our theory+

11 Z For exositioal simlicity, we will maily cosider ~ y t! with o determiistic treds, i+e+, ~ y t![~x t!+ It is easy to accommodate the resece of determiistic treds+ For istace, our subsequet bootstra methodology alies, without ay modificatio, to ~ y t! with determiistic treds if we detred ~ y t! by ruig a OLS regressio ' y t c t b [ x t ad use the fitted residuals ~ x[ t! i lace of ~x t!+ 4 Usig ~ x[ t! istead of ~x t! would, however, result i differet asymtotic distributios for the test statistics+ This will be exlaied i detail later+ To test the hyothesis H 0 : a i the model ~6!, we cosider the statistics roosed ad studied by Dickey ad Fuller ~979, 98!+ Their coefficiet ad t-statistics will be deoted by S ad T, resectively+ To reset them more exlicitly, we let a[ be the OLS estimator of a, ad we let s~ a[! be the stadard error for a[ give by v[ ~( x! 02, where v[ 2 is the usual error vari- 2 ace estimator from regressio ~6!+ The, we may write S ad T as ~0! x 2 ( S ~ a[! T a[ s~ a[! 2 [ ~20 2 2! ( x ~0! x 2 ( v ~02! ( uder the uit root hyothesis+ We ow have SIEVE BOOTSTRAP IN TIME SERIES 479 u t 2 u t 2, 2 x 02 S V ~! 2 v V ~t! 2 dt r d V~! 2 v V~t! 2 dt (7) ad similarly, T V ~! 2 2 v 02 2 v[ V ~t! 2 0 dt r d 2v 0 V~! 2 v 2 V~t! dt, (8) 02 2 where v 2 ~0!( u 2 t ad other otatio is defied earlier+ The covergece i distributio i ~7! ad ~8! ca be easily deduced from the laws of large umbers alied to v 2 ad v[ 2 ad the cotiuous maig theorem a-

12 480 JOON Y. PARK lied to the weak covergece V r d V because both V~! 2 ad * 0 V~t! 2 dt are cotiuous fuctioals of V i D@0,# + Let ~u t *! be the bootstra samle for ~u t!, u t x t x, obtaied as described i the revious sectio, ad deote by ~x t *! the bootstra samle for ~x t! geerated by x * * * t x u t with iitial value x 0, which is assumed to be zero to simlify the exositio+ It is imortat to imose the uit root hyothesis whe we geerate the bootstra samles ~x * t!+ If we geerate them by x * * t a[ x u * t usig the estimated value a[ of a i ~6!, the they would ot behave as a uit root rocess+ This is so, eve though a[ is suercosistet ad coverges to uity at a faster rate + See Basawa et al+ ~99a!, Datta ~996!, ad Kreiss ad Heima ~996!+ Let a[ * be the OLS coefficiet estimator from the first-order autoregressio of ~x * * t! o ~x!, ad s * ~ a[ *! be the bootstra stadard error for a[ *, which is give by v[ ~( x *2! 02 + Now we defie ~0! x *2 ( u t *2 S * ~ a[ *! T * a[ * s * ~ a[ *! ~20 2 *2! ( x ~0! x *2 ( u t *2 2 [ v ~02! (,, *2 02 x which are the bootstra versios of the statistics S ad T + Also, we defie v *2 ~0!( u *2 t + LEMMA 4++ Let Assumtios 3. ad 3.2 hold. The we have, for ay d 0, P * $6v *2 v 2 6 d% r 0 as r. THEOREM 4+2+ Let Assumtios 3. ad 3.2 hold. The we have S * r d * V~! 2 v V~t! 2 dt T * r d * 2v 0 V~! 2 v 2 V~t! d02 2 as r.

13 SIEVE BOOTSTRAP IN TIME SERIES 48 Theorem 4+2 shows that the bootstra statistics S * ad T * have the same asymtotic distributios as the corresodig samle statistics S ad T + Therefore, it establishes the asymtotic validity of the bootstra Dickey Fuller tests+ It shows i articular that if based o the bootstra critical values the the Dickey Fuller tests ca be used to test for the resece of a uit root i models drive by iovatios that are correlated+ The bootstra Dickey Fuller tests ca therefore be a alterative to the testig rocedures by Phillis ~987! ad Said ad Dickey ~984!~which are more ofte called augmeted Dickey Fuller tests!+ The limitig distributios of the samle statistics S ad T are ot ivotal+ They deed o the arameters s 2, v 2, ad ~!+ Note that V ~s~!!w, where W is stadard Browia motio+ It is thus ot exected that the bootstra statistics S * ad T * rovide asymtotic refiemets+ Therefore, the Dickey Fuller tests based o the bootstraed critical values are ot ecessarily better tha the tests relyig o the asymtotic critical values+ However, for the model cosidered here, the Said Dickey or Phillis statistics have limitig distributios that are free of uisace arameters+ Cosequetly, they may rovide tests with more accurate fiite samle sizes if based o the bootstraed critical values+ The bootstra cosistecy established reviously assumes the absece of determiistic treds+ However, it ca be exteded i a trivial maer to allow for the resece of determiistic treds+ For the tred secificatio i ~5!, we just assume that there exists k i such that, if we defie t i ~t! k i ( c i, $~k!0 t k0%, the t i r L 2 t i for some t i r R that is of bouded variatio for i,+++,m+ Also, we let t i s be liearly ideedet i L + Uder these assumtios, we may show that the asymtotic distributios of S ad T costructed from ~ x[ t! are idetical to those i ~7! ad ~8!, excet that V ~s~!!w is relaced by V t ~s~!!w t, where W t is the stadard detreded Browia motio itroduced i Sectio 2 ~see, e+g+, Park, 992!+ Moreover, it ca also be show without difficulty that the limitig distributios of their bootstra couterarts S * ad T * have the same limitig distributios+ 5. CONCLUDING REMARKS I this aer, we have develoed a ivariace ricile for the sieve bootstra from liear rocess with i+i+d+ iovatios+ It ca be used, i various cotexts, to obtai the limitig distributios of bootstra statistics without makig arametric assumtios o the uderlyig model+ As a illustratio, we used the ivariace ricile to derive the limitig distributios of the Dickey Fuller uit root tests ad show their asymtotic validity for a model drive by a geeral liear rocess that is serially correlated+ It aears that the bootstra ivariace ricile established i the aer has wide alicability ad ca be used to aalyze the asymtotic behavior of bootstra samles i statioary ad ostatioary time series models+ Some of the alicatios are uder way by the author ad co-researchers+

14 [ [ 482 JOON Y. PARK 6. MATHEMATICAL PROOFS The roofs of Lemma 2+ ad Theorem 2+2 are omitted+ Lemma 2+ is due to Sakhaeko ~980!, ad the stated result i Theorem 2+2 follows immediately from ~5! as metioed i the followig remarks+ Proof of Lemma 3++ We write ~u t! as u t a, u {{{ a, u t e, t, (9) where the coefficiets ~a, k! are defied so that ~e, t! are ucorrelated with ~u t k! for k,+++,+ We have [ max 6 a, k a, k 6 O~~log 0! 02!, (20) k ( 6a, k a k 6 c ( k 6a k 6 o~ s!, (2) where c is some costat+ For the results i ~20! ad ~2!, see, e+g+, Haa ad Kavalieris ~986, Theorem 2+! ad Bühlma ~995, roof of theorem 3+!+ The roof of the first art is immediate, because 6 a[, k a k 6 6 a[, k a, k 6 6a, k a k 6+ Similarly, it follows that 6 a[ ~! a~!6 ( 6 a[, k a, k 6 ( 6a, k a k 6 ( O~ ~log 0! 02! o~ s! k as was to be show for the result o a[ ~!+ For the result of s[ 2, see Bühlma ~995, roof of theorem 3+2!+ He establishes the result for the Yule Walker estimator, but it is easy to see that his result readily exteds to the least squares estimator+ Proof of Lemma 3+2+ We will show r02 r02 E * 6«* t 6 r ( «[, t ( as r + Note that ( «[, t ( «, t r c~a B C D!, 6a k 6 «, t r r 0 (22)

15 [ [ [ SIEVE BOOTSTRAP IN TIME SERIES 483 where c is some costat, ad A ( 6«t 6 r, B ( 6«, t «t 6 r, C ( 6 «[, t «, t 6 r, D «, t r + Therefore, to deduce ~22!, we oly eed to show that r02 A, r02 B, r02 C ad r02 D r 0+ This is what we ow set out to do+ By the strog law of large umbers, A r E6«t 6 r, ad therefore, r02 A r 0+ Next, we show that E6«, t «t 6 r o~ rs! (23) holds uiformly i t, from which it follows that r02 B r 0+ Note that if 0~log! 04 d r for some d 0 the r02 B r 0 also for r 4, as we remarked followig Assumtio 3+2+ To obtai ~23!, we write «, t «t ( k a k u t k (24) ad aly Mikowski s iequality ad use the statioarity of ~u t! to get E6«, t «t 6 r E6u t 6 r ( k 6a k 6 r o~ rs!+ Note that, due to the Marcikiewicz Zygmud iequality ~see, e+g+, Stout, 974, Theorem 3+3+6!, E6u t 6 r c ( k 0 k 2 r02e6«t 6 r, (25) where c is some costat, ad therefore ( a k u t k is well defied i the L r sese+ We ow rove that r02 C r 0+ Write «, t u t ( a, k u t k «, t ( ~ a[, k a, k!u t k ( ~a, k a k!u t k, (26)

16 [ 484 JOON Y. PARK where ~a, k! are defied i ~9!+ It follows that 6 [ «, t «, t 6 r c ( ~ [ a, k a, k!u t k r ( for c 2 r + Therefore, if we defie C ( ( ( ~ a[, k a, k!u t k r, C 2 ( ~a, k a k!u t k r, the it suffices to show that r02 C, r02 C 2 r 0+ Note that C is majorized by max 6 [ k a, k a, k 6 r ( ( max 6 [ k 6u t k 6 r a, k a, k 6 r ( t 0 6u t 6 r ( t ~a, k a k!u t k r 6u t 6 r O~~log 0! r!~0!o~! O~ ~log 0! r! by ~20! ad ~25! ad therefore, C r 0+ Moreover, we have E ( ~a, k a k!u t k r E6u t 6 r ( 6a, k a k 6 r o~ rs! by Mikowski s iequality ad the statioarity of ~u t!+ Cosequetly, it follows that r02 C 2 r 0 for r 4+ If 0~log! 04 d r, r 4 works also+ Fially, to deduce r02 D r 0, we show ( «, t ( «, t o~! ( «t o~!, which hold if [ ( a k u t k r 0, k (27) ( ~a, k a k!u t k r 0, (28) ( ~ a, k a, k!u t k r 0 (29) as a result of ~24! ad ~26!+

17 To establish ~27! ~29!, we first defie S ~i, j! ( «t i j ad T ~i! ( u t i + The it follows that T ~i! ( j 0 ( j «t i j ( j S ~i, j!+ j 0 I what follows, we use c to deote a costat, which is ot always the same+ By successive alicatios of Doob s iequality ad Burkholder s iequality ~see, e+g+, Hall ad Heyde, 980, Theorems 2+2 ad 2+0!, we have E max 6S m ~i, j!6 r c r02 m uiformly i i ad j+ Moreover, because E max m 6T m ~i!6 r 0r ( we have E max 6T m ~i!6 r c r02 m uiformly i i+ We ow let L ( k a k ( u t k + The it follows that E max m ad cosequetly, j 0 6L m 6 r 0r ( k E max 6L m 6 r c rs r02 + m Therefore, we may show for ay d 0 SIEVE BOOTSTRAP IN TIME SERIES j 6 E max 6S m ~i, j!6 r 0r m 6a k 6 E max 6T m ~k!6 r 0r c s 02 m L o~ s 02 ~log! 0r ~log log! ~ d!0r! o~! exactly as i Móricz ~976, , below ~4+8!!+ This roves ~27!+

18 486 JOON Y. PARK Similarly, if we let M ( ~a, k a k! ( the we have E max m u t k 6M m 6 r 0r ( Therefore, we have 6a, k a k 6 E max 6T m ~k!6 r 0r c s 02 + m M o~ s 02 ~log! 0r ~log log! ~ d!0r! o~!, which establishes ~28!+ Fially, we cosider N ( ~ a[, k a, k! ( u t k + If we let Q ( ( u t k the N is domiated by Q max 6 a[, k a, k 6+ k Moreover, we may deduce recisely as before that E max 6Q m 6 r c r r02 m ad therefore for ay d 0 Q o~ 02 ~log! 0r ~log log! ~ d!0r! Cosequetly, N O~~log 0! 02!Q o~ ~log! ~r 2!02r ~log log! ~ d!0r! o~! as was to be show for ~29!+ Therefore, the roof is comlete+ Proof of Theorem 3+3+ To show ~4!, we ote that P * max 6 02 I t u t * 6 d ( P * $6 02 ui * t 6 d% P * $6 02 ui * t 6 d% ~0d r! r02 E * 6uI * t 6 r +

19 [ [ The first iequality is obvious+ The secod equality follows from the statioarity of ~ ui * t!, coditioal o the realizatio of ~ «[, t!, ad the third iequality is due to the Tchebyshev iequality+ To comlete the roof, it ow suffices to rove that r02 E * 6uI * t 6 r r 0, which follows immediately if we show SIEVE BOOTSTRAP IN TIME SERIES 487 r02 E * 6u t * 6 r r 0 (30) because E * 6uI * t 6 r 6 [ a ~!6 ( k6 a[, k 6 re * 6u * t 6 r by Mikowski s iequality ad the coditioal statioarity of ~u * t!+ Note that a[ ~! r a~! as show i Lemma 3+ ad that ( k6 a[, k 6 ( k6a, k 6 o~! ( k6a k 6 o~! as show i the roof of Lemma 3++ We may write for all if we use the Yule Walker method to estimate regressio ~2! ~see, e+g+, Brockwell ad Davis, 99, + 240!# u * * t (, k «t k k 0 (3) with ~ [, k! such that ( k 0 6 [, k 6 Therefore, due to the Marcikiewicz Zygmud iequality, E * 6u t * 6 c (, k k 0 2 r02e * 6«t * 6 r for some costat c+ We therefore have ~30! wheever ~22! holds, which was show earlier i the roof of Lemma 3++ Proof of Lemma 4++ Let v * 2 E * 6u t * 6 2 ad ote that 6v *2 v 2 6 6v *2 v * 2 6 6v * 2 v 2 6 We first show that v * 2 r v 2 (32)

20 [ [ 488 JOON Y. PARK From the Yule Walker equatios, we have v 2 * ~s 2 * 0 s[ 2 2!v where s * 2 E * 6«t * 6 2 ad other otatios were defied earlier+ Because v 2 r v 2 by the strog law of large umbers, it suffices to show that s 2 * 0 s[ 2 r to deduce ~32!+ This, however, is immediate because s[ 2 r s 2 as show i Lemma 3+ ad s 2 * s[ 2 ( «, t 2 the secod term of which is of order o~!, as we have show i the roof of Lemma 3+2+ We ow show P * $6v *2 v * 2 6 d% ~0d! r E * 6v *2 v * 2 6 r r 0 (33) To deduce ~33!, we write ~u t *! as i ~3!+ The we have v *2 v * 2 ( i 0 j 0 (, i [, j ( * * ~«t i «t j d ij s 2 *! where d ij is the Kroecker delta, ad it follows that r02 r02 E * 6v *2 v 2 * 6 r02 ( 6 [, k 6 re * * * ( ~«t i «t j d ij s 2 *! k 0 by Mikowski s iequality+ By the successive alicatios of Burkholder s iequality ad Mikowski s iequality, we have E * ( r02 * * ~«t i «t j d ij s 2 *! KE * ( * * ~«t i «t j d ij s 2 *! r04 2 K r04 E * * * 6«t i «t j d ij s 2 * 6 r02, where K is a absolute costat deedig oly uo r+ Moreover, E * * * 6«t i «t j d ij s 2 * 6 r02 ~r 2!02~E * * * 6«t i «t j 6 r02 6s 2 * 6 r02! ~r 2!E * 6«* t 6 r

21 as a result of the Cauchy Schwarz iequality ad Jese s iequality+ It follows that E * 6v *2 v * 2 6 r02 r04 K ( k 0 6 [, k 6 re * 6«* t 6 r, where K is a absolute costat deedig uo r, ad ~33! would thus follow if r04 E * 6«t * 6 r r 0+ This ca be show exactly as i the roof of Lemma 3+2+ The stated result ow follows from ~32! ad ~33!+ Proof of Theorem 4+2+ The stated results follow immediately from Theorem 3+3 ad Lemma 4++ NOTES SIEVE BOOTSTRAP IN TIME SERIES They assume that ~«t! has a cotiuous distributio fuctio+ This coditio, however, is ot used i their roof for the bootstra ivariace ricile+ 2+ Our assumtios here are comarable to those made i related works+ Kreiss ~992! cosiders the model with ~ k! that decays exoetially+ Bühlma ~997! roves the weak form of bootstra cosistecy for a class of oliear estimators uder the coditio r 4 ad s + Bickel ad Bühlma ~999! establish a weak ivariace ricile for the bootstra emirical distributio fuctio for the case r 4 ad exoetially decayig ~ k!+ 3+ Notice that we do ot allow for uit root models geerated by geeral martigale differece iovatios+ 4+ Elliott, Rotheberg ad Stock ~996! show that the ower of the uit root test ca be substatially imroved by the geeralized least squares detredig+ Though we will ot reset the details here, the bootstra asymtotics of their tests ca also be aalyzed usig our method here+ REFERENCES A, H+-Z+, Z+-G+ Che, &E+J+ Haa ~982! Autocorrelatio, autoregressio ad autoregressive aroximatio+ Aals of Statistics 0, Correctio:, 08+ Basawa, I+V+, A+K+ Mallik, W+P+ McCormick, J+H+ Reeves, &R+L+ Taylor ~99a! Bootstraig ustable first-order autoregressive rocesses+ Aals of Statistics 9, Basawa, I+V+, A+K+ Mallik, W+P+ McCormick, J+H+ Reeves, &R+L+ Taylor ~99b! Bootstra test of sigificace ad sequetial bootstra estimatio for ustable first order autoregressive rocesses+ Commuicatios i Statistics Theory ad Methods 20, Bickel, P+J+ &P+ Bühlma ~999! A ew mixig otio ad fuctioal cetral limit theorems for a sieve bootstra i time series+ Beroulli 5, Bickel, P+J+ &D+A+ Freedma ~98! Some asymtotic theory for the bootstra+ Aals of Statistics 9, Billigsley, P+ ~968! Covergece of Probability Measures+ New York: Wiley+ Brilliger, D+R+ ~975! Time Series: Data Aalysis ad Theory+ New York: Holt, Riehart ad Wisto+ Brockwell, P+J+ &R+A+ Davis ~99! Time Series: Theory ad Methods+ New York: Sriger-Verlag+ Bühlma, P+ ~995! Movig-average reresetatio of autoregressive aroximatios+ Stochastic Processes ad Their Alicatios 60,

22 490 JOON Y. PARK Bühlma, P+ ~997! Sieve bootstra for time series+ Beroulli 3, Bühlma, P+ ~998! Sieve bootstra for smoothig i ostatioary time series+ Aals of Statistics 26, Datta, S+ ~996! O asymtotic roerties of bootstra for AR~! rocesses+ Joural of Statistical Plaig ad Iferece 53, Dickey, D+A+ &W+A+ Fuller ~979! Distributio of estimators for autoregressive time series with a uit root+ Joural of the America Statistical Associatio 74, Dickey, D+A+ &W+A+ Fuller ~98! Likelihood ratio statistics for autoregressive time series with a uit root+ Ecoometrica 49, Elliott, G+, T+J+ Rotheberg, &J+H+ Stock ~996! Efficiet tests for a autoregressive uit root+ Ecoometrica 64, Ferretti, N+ &J+ Romo ~996! Uit root bootstra tests for AR~! models+ Biometrika 83, Hall, P+ &C+C+ Heyde ~980! Martigale Limit Theory ad Its Alicatio+ New York: Academic Press+ Haa, E+J+ &L+ Kavalieris ~986! Regressio, autoregressio models+ Joural of Time Series Aalysis 7, Horowitz, J+L+ ~999! The Bootstra+ Mimeo, Deartmet of Ecoomics, Uiversity of Iowa+ Kreiss, J+P+ ~992! Bootstra rocedures for AR~!-rocesses+ I K+H+ Jöckel, G+ Rothe, &W+ Seder ~eds+!, Bootstraig ad Related Techiques, Lecture Notes i Ecoomics ad Mathematical Systems 376+ Heidelberg: Sriger+ Kreiss, J+P+ &G+ Heima ~996! Bootstraig geeral first order autoregressio+ Statistics ad Probability Letters 30, Küsch, H+R+ ~989! The jackkife ad the bootstra for geeral statioary observatios+ Aals of Statistics 7, Móricz, F+ ~976! Momet iequalities ad the strog law of large umbers+ Zeitschrift für Wahrscheilichkeitstheorie ud Verwadte Gebiete 35, Park, J+Y+ ~992! Caoical coitegratig regressios+ Ecoometrica 60, Phillis, P+C+B+ ~987! Time series regressio with a uit root+ Ecoometrica 55, Phillis, P+C+B+ &V+ Solo ~992! Asymtotics for liear rocesses+ Aals of Statistics 20, Pollard, D+ ~984! Covergece of Stochastic Processes+ New York: Sriger-Verlag+ Said, S+E+ &D+A+ Dickey ~984! Testig for uit roots i autoregressive-movig average models of ukow order+ Biometrika 7, Sakhaeko, A+I+ ~980! O uimrovable estimates of the rate of covergece i ivariace ricile+ I Noarametric Statistical Iferece, Colloquia Mathematica Societatis Jáos Bolyai Budaest, Hugary+ Shibata, R+ ~980! Asymtotically efficiet selectio of the order of the model for estimatig arameters of a liear rocess+ Aals of Statistics 8, Stout, W+F+ ~974! Almost Sure Covergece+ New York: Academic Press+