UNIFORM CONVERGENCE RATES FOR KERNEL ESTIMATION WITH DEPENDENT DATA

Transcription

1 Ecoometric Teory, 24, 2008, Prited i te Uited States of America+ doi: S UNIFORM CONVERGENCE RATES FOR KERNEL ESTIMATION WITH DEPENDENT DATA BRUCE E. HANSEN Uiversity of Wiscosi Tis paper presets a set of rate of uiform cosistecy results for kerel estimators of desity fuctios ad regressios fuctios+ We geeralize te existig literature by allowig for statioary strog mixig multivariate data wit ifiite support, kerels wit ubouded support, ad geeral badwidt sequeces+ Tese results are useful for semiparametric estimatio based o a first-stage oparametric estimator+ 1. INTRODUCTION Tis paper presets a set of rate of uiform cosistecy results for kerel estimators of desity fuctios ad regressios fuctios+ We geeralize te existig literature by allowig for statioary strog mixig multivariate data wit ifiite support, kerels wit ubouded support, ad geeral badwidt sequeces+ Kerel estimators were first itroduced by Roseblatt ~1956! for desity estimatio ad by Nadaraya ~1964! ad Watso ~1964! for regressio estimatio+ Te local liear estimator was itroduced by Stoe ~1977! ad came ito promiece troug te work of Fa ~1992, 1993!+ Uiform covergece for kerel averages as bee previously cosidered i a umber of papers, icludig Peligrad ~1991!, Newey ~1994!, Adrews ~1995!, Liebscer ~1996!, Masry ~1996!, Bosq ~1998!, Fa ad Yao ~2003!, ad Ago Nze ad Douka ~2004!+ I tis paper we provide a geeral set of results wit broad applicability+ Our mai results are te weak ad strog uiform covergece of a sample average fuctioal+ Te coditios imposed o te fuctioal are geeral+ Te data are assumed to be a statioary strog mixig time series+ Te support for te data is allowed to be ifiite, ad our covergece is uiform over compact sets, expadig sets, or urestricted euclidea space+ We do ot require te regressio fuctio or its derivatives to be bouded, ad we allow for kerels wit Tis researc was supported by te Natioal Sciece Foudatio+ I tak tree referees ad Oliver Lito for elpful commets+ Address correspodece to Bruce E+ Hase, Departmet of Ecoomics, Uiversity of Wiscosi, 1180 Observatory Drive, Madiso, WI , USA; base@ssc+wisc+edu Cambridge Uiversity Press $15+00

2 UNIFORM CONVERGENCE RATES 727 ubouded support+ Te rate of decay for te badwidt is flexible ad icludes te optimal covergece rate as a special case+ Our applicatios iclude estimatio of multivariate desities ad teir derivatives, Nadaraya Watso regressio estimates, ad local liear regressio estimates+ We do ot cosider local polyomial regressio, altoug our mai results could be applied to tis applicatio also+ Tese features are useful geeralizatios of te existig literature+ Most papers assume tat te kerel fuctio as trucated support, wic excludes te popular Gaussia kerel+ It is also typical to demostrate uiform covergece oly over fixed compact sets, wic is sufficiet for may estimatio purposes but is isufficiet for may semiparametric applicatios+ Some papers assume tat te regressio fuctio, or certai derivatives of te regressio fuctio, is bouded+ Tis may appear iocet we covergece is limited to fixed compact sets but is usatisfactory we covergece is exteded to expadig or ubouded sets+ Some papers oly preset covergece rates usig optimal badwidt rates+ Tis is iappropriate for may semiparametric applicatios were te badwidt sequeces may ot satisfy tese coditios+ Our paper avoids tese deficiecies+ Our proof metod is a geeralizatio of tose i Liebscer ~1996! ad Bosq ~1998!+ Sectio 2 presets results for a geeral class of fuctios, icludig a variace boud, weak uiform covergece, strog uiform covergece, ad covergece over ubouded sets+ Sectio 3 presets applicatios to desity estimatio, Nadaraya Watso regressio, ad local liear regressio+ Te proofs are i te Appedix+ Regardig otatio, for x ~x 1,+++,x d! R d we set 7x7 max~6 x 1 6, +++,6x d 6!+ 2. GENERAL RESULTS 2.1. Kerel Averages ad a Variace Boud Let $Y i, X i % R R d be a sequece of radom vectors+ Te vector X i may iclude lagged values of Y i, e+g+, X i ~Y i1,+++,y id!+ Cosider averages of te form ZC~x! 1 Y d i K x X i, (1) i1 were o~1! is a badwidt ad K~u! : R d r R is a kerel-like fuctio+ Most kerel-based oparametric estimators ca be writte as fuctios of averages of tis form+ By suitable coice of K~u! ad Y i tis icludes kerel estimators of desity fuctios, Nadaraya Watso estimators of te regressio

3 728 BRUCE E. HANSEN fuctio, local polyomial estimators, ad estimators of derivatives of desity ad regressio fuctios+ We require tat te fuctio K~u! is bouded ad itegrable: Assumptio 1+ 6K~u!6 KP ` ad R d 6K~u!6du m `+ We assume tat $Y i, X i % is weakly depedet+ We require te followig regularity coditios+ Assumptio 2+ Te sequece $Y i, X i % is strictly statioary ad strog mixig wit mixig coefficiets a m tat satisfy a m Am b, (2) were A ` ad for some s 2 E6Y 0 6 s ` (3) ad b 2s 2 s 2 + (4) Furtermore, X i as margial desity f ~x! suc tat ad sup f ~x! B 0 ` (5) x sup E~6Y 0 6 s 6X 0 x! f ~x! B 1 `+ (6) x Also, tere is some j * ` suc tat for all j j * sup E~6Y 0 Y j 66X 0 x 0, X j x j! f j ~x 0, x j! B 2 `, (7) x 0, x j were f j ~x 0, x j! deotes te joit desity of $X 0, X j %+ Assumptio 2 specifies tat te serial depedece i te data is strog mixig, ad equatios ~2! ~4! specify a required decay rate+ Coditio ~5! specifies tat te desity f ~x! is bouded, ad ~6! cotrols te tail beavior of te coditioal expectatio E~6Y 0 6 s 6X 0 x!+ Te latter ca icrease to ifiity i te tails but ot faster ta f ~x! 1 + Coditio ~7! places a similar boud o te joit desity ad coditioal expectatio+ If te data are idepedet or m-depedet, te ~7! is immediately satisfied uder ~6! wit B 2 B I may applicatios ~suc as desity estimatio! Y i is bouded+ I tis case we ca take s `, ~4! simplifies to b 2, ~6! is redudat wit ~5!, ad ~7! is equivalet to f j ~x 0, x j! B 2 for all j j * +

4 UNIFORM CONVERGENCE RATES 729 Te boud ~7! requires tat $X 0, X j % ave a bouded joit desity f j ~x 0, x j! for sufficiet large j, but te joit desity does ot eed to exist for small j+ Tis distictio allows X i to cosist of multiple lags of Y i + For example, if X i ~Y i1,y i2,+++,y id! for d 2 te f j ~x 0, x j! is ubouded for j d because te compoets of X 0 ad X j overlap+ THEOREM 1+ Uder Assumptios 1 ad 2 tere is a Q ` suc tat for sufficietly large Var ~ ZC~x!! Q d. (8) A expressio for Q is give i equatio (A.5) i te Appedix. Altoug Teorem 1 is elemetary for idepedet observatios, it is otrivial for depedet data because of te presece of ozero covariaces+ Our proof builds o te strategy of Fa ad Yao ~2003, pp ! by separately boudig covariaces of sort, medium, ad log lag legts Weak Uiform Covergece Teorem 1 implies tat 6 ZC~x! E ZC~x!6 O p ~~ d! 102! poitwise i x R d + We are ow iterested i uiform rates+ We start by cosiderig uiformity over values of x i expadig sets of te form $x :7x7 c % for sequeces c tat are eiter bouded or divergig slowly to ifiity+ To establis uiform covergece, we eed te fuctio K~u! to be smoot+ We require tat K eiter as trucated support ad is Lipscitz or tat it as a bouded derivative wit a itegrable tail+ Assumptio 3+ For some L 1 ` ad L `, eiter K~u! 0 for 7u7 L ad for all u, u ' R d 6K~u! K~u '!6 L 1 7u u ' 7, (9) or K~u! is differetiable,6~]0]u!k~u!6 L 1, ad for some 1,6~]0]u!K~u!6 L 1 7u7 for 7u7 L+ Assumptio 3 allows for most commoly used kerels, icludig te polyomial kerel class c p ~1 x 2! p, te iger order polyomial kerels of Müller ~1984! ad Graovsky ad Müller ~1991!, te ormal kerel, ad te iger order Gaussia kerels of Wad ad Scucay ~1990! ad Marro ad Wad ~1992!+ Assumptio 3 excludes, owever, te uiform kerel+ It is ulikely tat tis is a ecessary exclusio, as Tra ~1994! establised uiform covergece

5 730 BRUCE E. HANSEN of a istogram desity estimator+ Assumptio 3 also excludes te Diriclet kerel K~x! si~x!0~px!+ THEOREM 2+ Suppose tat Assumptios 1 3 old ad for some q 0 te mixig expoet b satisfies 1 ~s 1!1 d q d b (10) s 2 ad for b 1 d d ~1 b!0~s 1! q u b 3 d ~1 b!0~s 1! (11) te badwidt satisfies l o~1!. u d (12) Te for c O~~l! 10d 102q! (13) ad a l d 102, (14) sup 7x7c 6 ZC~x! E ZC~x!6 O p ~a!. (15) Teorem 2 establises te rate for uiform covergece i probability+ Usig ~10! ad ~11! we ca calculate tat u ~0,1# ad tus ~12! is a stregteig of te covetioal requiremet tat d r `+ Also ote tat ~10! is a strict stregteig of ~4!+ If Y i is bouded, we ca take s `, ad te ~10! ad ~11! simplify to b 1 ~d0q! d ad u ~b 1 d ~d0q!!0~b 3 d!+ If q ` ad d 1 te tis simplifies furter to b 2 ad u ~b 2!0 ~b 2!, wic is weaker ta te coditios of Fa ad Yao ~2003, Lem+ 6+1!+ If te mixig coefficiets ave geometric decay ~b `! te u 1 ad ~15! olds for all q+ It is also costructive to compare Teorem 2 wit Lemma B+1 of Newey ~1994!+ Newey s covergece rate is idetical to ~15!, but is result is restricted to idepedet observatios, kerel fuctios K wit bouded support, ad bouded c +

6 2.3. Almost Sure Uiform Covergece I tis sectio we stregte te result of te previous sectio to almost sure covergece+ THEOREM 3+ Defie f ~l l! 2 l. Suppose tat Assumptios 1 3 old ad for some q 0 te mixig expoet b satisfies 2 s3 d q d b (16) s 2 ad for UNIFORM CONVERGENCE RATES 731 u b1 2 s 2 s 3 d q d b 3 d (17) te badwidt satisfies f 2 O~1!. u d (18) Te for c O~f 10d 102q!, (19) sup 6 ZC~x! E ZC~x!6 O~a! (20) 7x7c almost surely, were a is defied i (14). Te primary differece betwee Teorems 2 ad 3 is te coditio o te strog mixig coefficiets Uiform Covergece over Ubouded Sets Te previous sectios cosidered uiform covergece over bouded or slowly expadig sets+ We ow cosider uiform covergece over urestricted euclidea space+ Tis requires additioal momet bouds o te coditioig variables ad polyomial tail decay for te fuctio K~u!+ THEOREM 4+ Suppose te assumptios of Teorem 2 old wit O~1! ad q d. Furtermore,

7 732 BRUCE E. HANSEN sup7x7 q E~6Y 0 66X 0 x! f ~x! B 3 `, (21) x ad for 7u7 L 6K~u!6 L 2 7u7 q (22) for some L 2 `. Te sup 6 ZC~x! E ZC~x!6 O p ~a!. xr d THEOREM 5+ Suppose te assumptios of Teorem 3 old wit O~1! ad q d. Furtermore, (21), (22), ad E7X 0 7 2q ` old. Te sup 6 ZC~x! E ZC~x!6 O~a! xr d almost surely. Teorems 4 ad 5 sow tat te extesio to uiformity over urestricted euclidea space ca be made wit miimal additioal assumptios+ Equatio ~21! is a mild tail restrictio o te coditioal mea ad desity fuctio+ Te kerel tail restrictio ~22! is satisfied by te kerels discussed i Sectio 2+2 for all q APPLICATIONS 3.1. Desity Estimatio Let X i R d be a strictly statioary time series wit desity f ~x!+ Cosider te estimatio of f ~x! ad its derivatives f ~r! ~x!+ Let k~u! : R d r R deote a multivariate pt-order kerel fuctio for wic k ~r! ~u! satisfies Assumptio 1 ad 6u6 pr 6k~u!6du `+ Te Roseblatt ~1956! estimator of te rt derivative f ~r! ~x! is f Z ~r! ~x! 1 dr i1 ~r! k x X i, were is a badwidt+ We first cosider uiform covergece i probability+ THEOREM 6+ Suppose tat for some q 0, te strog mixig coefficiets satisfy (2) wit b 1 d d, q (23)

8 UNIFORM CONVERGENCE RATES 733 o~1!, ad (12) olds wit b 1 d q d u. b 3 d (24) Suppose tat sup x f ~x! ` ad tere is some j * ` suc tat for all j j *, sup x0, x j f j ~x 0, x j! ` were f j ~x 0, x j! deotes te joit desity of $X 0, X j %. Assume tat te pt derivative of f ~r! ~x! is uiformly cotiuous. Te for ay sequece c satisfyig (13), sup 6 f Z ~r! ~x! f ~r! ~x!6 O 7x7c p l d2r 102 p. (25) Te optimal covergece rate (by selectig te badwidt optimally) ca be obtaied we b 1 d d q ad is d p r2 2q d (26) sup 6 f Z ~r! ~x! f ~r! ~x!6 O 7x7c p l p0~d2p2r!. (27) Furtermore, if i additio sup x 7x7 q f ~x! ` ad 6k ~r! ~u!6 L 2 7u7 q for 7u7 large, te te supremum i (25) or (27) may be take over x R d. Take te simple case of estimatio of te desity ~r 0!, secod-order kerel ~ p 2!, ad bouded c ~q `!+ I tis case te requiremets state tat b 1 d is sufficiet for ~25! ad b 1 2d is sufficiet for te optimal covergece rate ~27!+ Tis is a improvemet upo te work of Fa ad Yao ~2003, Tm+ 5+3!, wo ~for d 1! require b 5 2 _ ad b _ 15 4 for tese two results+ A alterative uiform weak covergece rate as bee provided by Adrews ~1995, Tm+ 1~a!!+ His result is more geeral i allowig for ear-epocdepedet arrays, but e obtais a slower rate of covergece+ We ow cosider uiform almost sure covergece+ THEOREM 7+ Uder te assumptios of Teorem 6, if b 3 ~d0q! d ad (18) ad (19) old wit b 3 d q d u, b 3 d

9 734 BRUCE E. HANSEN te sup 6 f Z ~r! ~x! f ~r! ~x!6 7x7c O l d2r p 102 almost surely. Te optimal covergece rate we b 3 d d q d p r3 2q d is sup 6 f Z ~r! ~x! f ~r! ~x!6 7x7c O l p0~d2p2r! almost surely. (28) Alterative results for strog uiform covergece for kerel desity estimates ave bee provided by Peligrad ~1991!, Liebscer ~1996, Tms+ 4+2 ad 4+3!, Bosq ~1998, Tm+ 2+2 ad Cor+ 2+2!, ad Ago Nze ad Douka ~2004!+ Teorem 6 cotais Liebscer s result as te special case r 0 ad q `, ad e restricts attetio to kerels wit bouded support+ Peligrad imposes r-mixig ad bouded c + Bosq restricts attetio to geometric strog mixig Nadaraya Watso Regressio Cosider te estimatio of te coditioal mea m~x! E~Y i 6X i x!+ Let k~u! : R d r R deote a multivariate symmetric kerel fuctio tat satisfies Assumptios 1 ad 3 ad let 6u6 2 6k~u!6du `+ Te Nadaraya Watso estimator of m~x! is m~x! [ i1 i1 Y i k x X i k x X i, were is a badwidt+ THEOREM 8+ Suppose tat Assumptio 2 ad equatios (10) (13) old ad te secod derivatives of f ~x! ad f ~x!m~x! are uiformly cotiuous ad bouded. If

10 UNIFORM CONVERGENCE RATES 735 d if 6x6c f ~x! 0, o~1!, ad d 1 a * r 0 were a * log 102 2, (29) d te sup 6 m~x! [ m~x!6 O p ~d 1 a *!. (30) 6x6c Te optimal covergece rate we b is sufficietly large is sup 6 m~x! [ m~x!6 O pd 6x6c 1 l 20~d4!. (31) THEOREM 9+ Suppose tat te assumptios of Teorem 8 old ad equatios (16) (19) old istead of (10) (13). Te (30) ad (31) ca be stregteed to almost sure covergece. If c is a costat te te covergece rate is a, ad te optimal rate is ~ 1 l! 20~d4!, wic is te Stoe ~1982! optimal rate for idepedet ad idetically distributed ~i+i+d+! data+ Teorems 8 ad 9 sow tat te uiform covergece rate is ot pealized for depedet data uder te strog mixig assumptio+ For semiparametric applicatios, it is frequetly useful to require c r ` so tat te etire fuctio m~x! is cosistetly estimated+ From ~30! we see tat tis iduces te additioal pealty term d 1 + Alterative results for te uiform rate of covergece for te Nadaraya Watso estimator ave bee provided by Adrews ~1995, Tm+ 1~b!! ad Bosq ~1998, Tms+ 3+2 ad 3+3!+ Adrews allows for ear-epoc-depedet arrays but obtais a slower rate of covergece+ Bosq requires geometric strog mixig, a muc stroger momet boud, ad a specific coice for te badwidt parameter Local Liear Regressio Te local liear estimator of m~x! E~Y i 6X i x! ad its derivative m ~1! ~x! are obtaied from a weigted regressio of Y i o X i x i + Lettig k i k~~x X i!0! ad j i X i x, te local liear estimator ca be writte as m~x! K m K ~x! ~1! k i i1 i1 j i k i i1 i1 j ' i k i j i j ' i k i1 i1 i1 k i Y i j i k i Y i+

11 736 BRUCE E. HANSEN Let k~u! be a multivariate symmetric kerel fuctio for wic 6u6 4 6k~u!6du ` ad te fuctios k~u!, uk~u!, ad uu ' k~u! satisfy Assumptios 1 ad 3+ THEOREM 10+ Uder te coditios of Teorem 8 ad d 2 a * r 0 were a * is defied i (29) te sup 6m~x! K m~x!6 O p ~d 2 a *!. 6x6c THEOREM 11+ Uder te coditios of Teorem 9 ad d 2 a * r 0 were a * is defied i (29) te sup 6m~x! K m~x!6 O~d 2 a *! 6x6c almost surely. Tese are te same rates as for te Nadaraya Watso estimator, except te pealty term for expadig c as bee stregteed to d 2 + We c is fixed te covergece rate is Stoe s optimal rate+ Alterative uiform covergece results for pt-order local polyomial estimators wit fixed c ave bee provided by Masry ~1996! ad Fa ad Yao ~2003, Tm+ 6+5!+ Fa ad Yao restrict attetio to d 1+ Masry allows d 1 but assumes tat ~ p 1! derivatives of m~x! are uiformly bouded ~secod derivatives i te case of local liear estimatio!+ Istead, we assume tat te secod derivatives of te product f ~x!m~x! are uiformly bouded, wic is less restrictive for te case of local liear estimatio+ REFERENCES Adrews, D+W+K+ ~1995! Noparametric kerel estimatio for semiparametric models+ Ecoometric Teory 11, Ago Nze, P+ &P+ Douka ~2004! Weak depedece: Models ad applicatios to ecoometrics+ Ecoometric Teory 20, Bosq, D+ ~1998! Noparametric Statistics for Stocastic Processes: Estimatio ad Predictio, 2d ed+ Lecture Notes i Statistics 110+ Spriger-Verlag+ Fa, J+ ~1992! Desig-adaptive oparametric regressio+ Joural of te America Statistical Associatio 87, Fa, J+ ~1993! Local liear regressio smooters ad teir miimax efficiecy+ Aals of Statistics 21, Fa, J+ & Q+ Yao ~2003! Noliear Time Series: Noparametric ad Parametric Metods+ Spriger-Verlag+ Graovsky, B+L+ &H+-G+ Müller ~1991! Optimizig kerel metods: A uifyig variatioal priciple+ Iteratioal Statistical Review 59, Liebscer, E+ ~1996! Strog covergece of sums of a-mixig radom variables wit applicatios to desity estimatio+ Stocastic Processes ad Teir Applicatios 65, Mack, Y+P+ &B+W+ Silverma ~1982! Weak ad strog uiform cosistecy of kerel regressio estimates+ Zeitscrift für Warsceilickeitsteorie ud Verwadte Gebiete 61,

12 UNIFORM CONVERGENCE RATES 737 Marro, J+S+ &M+P+ Wad ~1992! Exact mea itegrated squared error+ Aals of Statistics 20, Masry, E+ ~1996! Multivariate local polyomial regressio for time series: Uiform strog cosistecy ad rates+ Joural of Time Series Aalysis 17, Müller, H+-G+ ~1984! Smoot optimum kerel estimators of desities, regressio curves ad modes+ Aals of Statistics 12, Nadaraya, E+A+ ~1964! O estimatig regressio+ Teory of Probability ad Its Applicatios 9, Newey, W+K+ ~1994! Kerel estimatio of partial meas ad a geeralized variace estimator+ Ecoometric Teory 10, Peligrad, M+ ~1991! Properties of uiform cosistecy of te kerel estimators of desity ad of regressio fuctios uder depedece coditios+ Stocastics ad Stocastic Reports 40, Rio, E+ ~1995! Te fuctioal law of te iterated logaritm for statioary strogly mixig sequeces+ Aals of Probability 23, Roseblatt, M+ ~1956! Remarks o some o-parametric estimates of a desity fuctio+ Aals of Matematical Statistics 27, Stoe, C+J+ ~1977! Cosistet oparametric regressio+ Aals of Statistics 5, Stoe, C+J+ ~1982! Optimal global rates of covergece for oparametric regressio+ Aals of Statistics 10, Tra, L+T+ ~1994! Desity estimatio for time series by istograms+ Joural of Statistical Plaig ad Iferece 40, Wad, M+P+ &W+R+ Scucay ~1990! Gaussia-based kerels+ Caadia Joural of Statistics 18, Watso, G+S+ ~1964! Smoot regressio aalysis+ Sakya, Series A 26, APPENDIX Proof of Teorem 1. We start wit some prelimiary bouds+ First ote tat Assumptio 1 implies tat for ay r s, 6K~u!6 R r du KP r1 m KP s1 m+ d (A.1) Secod, assumig witout loss of geerality tat B 0 1 ad B 1 1, ote tat te L r iequality, ~5!, ad ~6! imply tat for ay 1 r s E~6Y 0 6 r 6X 0 x! f ~x! ~E~6Y 0 6 s 6X 0 x!! r0s f ~x! ~E~6Y 0 6 s 6X 0 x! f ~x!! r0s f ~x! ~sr!0s B 1 r0s B 0 ~sr!0s B 1 B 0 + (A.2) Tird, for fixed x ad let Z i K x X i Y i +

13 738 BRUCE E. HANSEN Te for ay 1 r s, by iterated expectatios, ~A+2!, a cage of variables, ad ~A+1! d E6Z 0 6 r d EE K x X 0 Y 0 r 6X 0 d d R K x u r E~6Y 0 6 r 6X 0 u! f ~u! du R d 6K~u!6 r E~6Y 0 6 r 6X 0 x u! f ~x u! du R d 6K~u!6 r dub 1 B 0 KP s1 mb 1 B 0 [ mt `+ (A.3) Fially, for j j *, by iterated expectatios, ~7!, two cages of variables, ad Assumptio 1, E6Z 0 Z j 6 EE K x X 0 K x X jy 0 Y j 6X 0, X j d R K x u 0K x u j dr E~6Y 0Y j 66X 0 u 0, X j u j! f j ~u 0, u j! du 0 du j R dr d 6K~u 0!K~u j!6e~6y 0 Y j 66X 0 x u 0, X j x u j! f j ~x u 0, x u j! du 0 du j 2d R dr d 6K~u 0!K~u j!6du 0 du j B 2 2d m 2 B 2 + (A.4) Defie te covariaces C j E~~Z 0 EZ 0!~Z j EZ j!!+ Assume tat is sufficietly large so tat d j * + We ow boud te C j separately for j j *, j * j d, ad d 1 j `+ First, for j j *, by te Caucy Scwarz iequality ad ~A+3! wit r 2, 6C j 6 E~Z 0 EZ 0! 2 EZ 2 0 m T d +

14 Secod, for j * j d, ~A+4! ad ~A+3! for r 1 combie to yield 6C j 6 E6Z 0 Z j 6 ~E6Z 0 6! 2 ~m 2 B 2 mt 2! 2d + Tird, for j d 1, usig Davydov s lemma, ~2!, ad ~A+3! wit r s we obtai 6C j 6 6a j 120s ~E6Z i 6 s! 20s 6Aj b~120s! ~ m T d! 20s 6AmT 20s j ~220s! 2d0s, were te fial iequality uses ~4!+ Usig tese tree bouds, we calculate tat d Var ~ ZC~x!! 1 E Z i EZ i 2 i1 j * d ` C 0 2 6C j 6 2 6C j 6 2 j1 ~1 2j *! m T d 2 2 ` j d 1 jj * 1 d jj * 1 6AmT 20s j ~220s! 2d0s j d 1 ~m 2 B 2 mt 2! 2d 6C j 6 ~1 2j *! m T d 2~m 2 B 2 mt 2! d 12Am20s T ~s 2!0s d, were te fial iequality uses te fact tat for d 1 ad k 1 ` j d jk1 k ` x d dx k 1d ~d 1! + We ave sow tat ~8! olds wit m20s s Q ~1 2j *! mt 2~m 2 B 2 mt 2! 12A T, (A.5) s 2 completig te proof+ UNIFORM CONVERGENCE RATES 739 Before givig te proof of Teorem 2 we restate Teorem 2+1 of Liebscer ~1996! for statioary processes, wic is derived from Teorem 5 of Rio ~1995!+ LEMMA ~Liebscer0Rio!+ Let Z i be a statioary zero-mea real-valued process suc tat 6Z i 6 b, wit strog mixig coefficiets a m. Te for eac positive iteger m ad «suc tat m «b04

15 740 BRUCE E. HANSEN P i1 Z i «4 exp were s 2 m E~ m i1 Z i! 2. «2 «mb 64 s m 2 m 8 4 m a m, 3 Proof of Teorem 2. We first ote tat ~10! implies tat u defied i ~11! satisfies u 0, so tat ~12! allows o~1! as required+ Te proof is orgaized as follows+ First, we sow tat we ca replace Y i wit te trucated process Y i 1~6Y i 6 t! were t a 10~s1! + Secod, we replace te te supremum i ~15! wit a maximizatio over a fiite N-poit grid+ Tird, we use te expoetial iequality of te lemma to boud te remaider+ Te secod ad tird steps are a modificatio of te strategy of Liebscer ~1996, proof of Tm+ 4+2!+ Te first step is to trucate Y i + Defie R ~x! ZC~x! 1 d 1 d i1 Y i K i1 Y i K x X i 1~6Y i 6 t! x X i 1~6Y i 6 t!+ (A.6) Te by a cage of variables, usig te regio of itegratio, ~6!, ad Assumptio 1 6ER ~x!6 1 d d R K x u E~6Y 061~6Y 0 6 t!6x 0 u! f ~u! du R d 6K~u!6E~6Y 0 61~6Y 0 6 t!6x 0 x u! f ~x u! du R d 6K~u!6E~6Y 0 6 s t ~s1! 1~6Y 0 6 t!6x 0 x u! f ~x u! du t ~s1!r d 6K~u!6E~6Y 0 6 s 6X 0 x u! f ~x u! du t ~s1! mb 1 + (A.7) By Markov s iequality ad te defiitio of t 6R ~x! ER ~x!6 O p ~t ~s1!! O p ~a!, ad terefore replacig Y i wit Y i 1~6Y i 6 t! results i a error of order O p ~a!+ For te remaider of te proof we simply assume tat 6Y i 6 t + For te secod step we create a grid usig regios of te form A j $x :7x x j 7 a %+ By selectig x j to lay o a grid, te regio $x :7x7 c % ca be covered wit N c d d a d suc regios A j + Assumptio 3 implies tat for all 6x 1 x 2 6 d L, 6K~x 2! K~x 1!6 dk * ~x 1!, (A.8)

16 P P were K * ~u! satisfies Assumptio 1+ Ideed, if K~u! as compact support ad is Lipscitz te K * ~u! L 1 1~7u7 2L!+ O te oter ad, if K~u! satisfies te differetiability coditios of Assumptio 3, te K * ~u! L 1 ~1~7u7 2L! 7u L7 1~7u7 2L!!+ I bot cases K * ~u! is bouded ad itegrable ad terefore satisfies Assumptio 1+ Note tat for ay x A j te 7x x j 70 a, ad equatio ~A+8! implies tat if is large eoug so tat a L, K x X i Now defie K x j X i i1 * a K x j X i + UNIFORM CONVERGENCE RATES 741 * EC~x! 1 Y d i K x X i, (A.9) wic is a versio of ZC~x! wit K~u! replaced wit K * ~u!+ Note tat E6 EC~x!6 B 1 B 0 R d K * ~u! du `+ Te sup xa j 6 ZC~x! E ZC~x!6 6 ZC~x j! E ZC~x j!6 EC~x j!6 E6 EC~x j!6# 6 ZC~x j! E ZC~x j!6 a 6 EC~x j! E EC~x j!6 2a E6 EC~x j!6 6 ZC~x j! E ZC~x j!6 6 EC~x j! E EC~x j!6 2a M, te fial iequality because a 1 for sufficietly large ad for ay M E6 EC~x!6+ We fid tat P sup 7x7c 6 ZC~x! E ZC~x!6 3Ma N max P 1jN sup6 ZC~x! E ZC~x!6 3Ma xa j N max P~6 ZC~x j! E ZC~x j!6 M! (A.10) 1jN N max P~6 EC~x j! E EC~x j!6 M!+ (A.11) 1jN We ow boud ~A+10! ad ~A+11! usig te same argumet, as bot K~u! ad K * ~u! satisfy Assumptio 1, ad tis is te oly property we will use+ Let Z i ~x! Y i K~~x X i!0! EY i K~~x X i!0!+ Because 6Y i 6 t ad 6K~~x X i!0!6 K it follows tat 6Z i ~x!6 2t K [ b + Also from Teorem 1 we ave ~for sufficietly large! te boud

17 742 BRUCE E. HANSEN sup x m E i1 2 Z i ~x! Qm d + Set m a 1 t 1 ad ote tat m ad m «b 04 for «Ma d for sufficietly large+ Te by te lemma, for ay x, ad sufficietly large, P~6 ZC~x! E ZC~x!6 Ma! P i1 4 exp Z i ~x! Ma d M 2 a 2 2 2d d 4 64Q d 6KM P m a m 4 exp M 2 l 64Q 6KM4Am 1b P 4 M0~646K P! 4Aa 1b t 1b, te secod iequality usig ~2! ad ~14! ad te last iequality takig M Q+ Recallig tat N c d d a d, it follows from tis ad ~A+10! ~A+11! tat P sup 7x7c 6 ZC~x! E ZC~x!6 3Ma O~T 1! O~T 2!, (A.12) were T 1 c d d a d M0~646K P! (A.13) ad T 2 c d d a 1bd t 1b + (A.14) Recall tat t a 10~s1! ad c O~~l! 10d 102q!+ Equatio ~12! implies tat ~l! d o~ u! ad tus c d d o~ d02qu!+ Also a ~~l! d 1! 102 o~ ~1u!02!+ Tus T 1 o~ d02qud~1u!02m0~646k P!! o~1! for sufficietly large M ad T 2 o~ d02qu1~1u!@1bd~1b!0~s1!#02! o~1! by ~11!+ Tus ~A+12! is o~1!, wic is sufficiet for ~15!+ Proof of Teorem 3. We first ote tat ~16! implies tat u defied i ~17! satisfies u 0, so tat ~18! allows o~1! as required+

18 Te proof is a modificatio of te proof of Teorem 2+ Borrowig a argumet from Mack ad Silverma ~1982!, we first sow tat R ~x! defied i ~A+6! is O~a! we we set t ~f! 10s + Ideed, by ~A+7! ad s 2, 6ER ~x!6 t ~s1! mb 1 ~s1!s mb 1 O~a!, UNIFORM CONVERGENCE RATES 743 ad because ` 1 ` P~6Y 6 t! t s E6Y 6 s E6Y 0 6 s ~f! 1 `, 1 ` 1 usig te fact tat ` 1 ~f! 1 `, te for sufficietly large, 6Y 6 t wit probability oe+ Hece for sufficietly large ad all i, 6Y i 6 t, ad tus R ~x! 0 wit probability oe+ We ave sow tat 6R ~x! ER ~x!6 O~a! almost surely+ Tus, as i te proof of Teorem 2 we ca assume tat 6Y i 6 t + Equatios ~A+12! ~A+14! old wit t ~f! 10s ad c O~f 10d 102q!+ Employig d O~f 2 u! ad r o~f 102 ~1u!02! we fid T 1 c d d r d M0~646K P! o~f 1 d02qud~1u!02m0~646k P!! o~~f! 1! for sufficietly large M ad T 2 c d d a 1bd t 1b O~f 1~1bd!02~1b!0s d02qu1~1u!~1bd!02~1b!0s! O~~f! 1! by ~17! ad te fact tat ~1 b!0s ~1 b d!02 is implied by ~16!+ Tus ` 1 ~T 1 T 2! `+ It follows from tis ad ~A+12! tat ` 1 P sup 6 ZC~x! E ZC~x!6 3Ma 7x7c `, ad ~20! follows by te Borel Catelli lemma+

19 744 BRUCE E. HANSEN Proof of Teorem 4. Defie c 102q ad EC~x! 1 Y d i K x X i1~7x i 7 c!+ (A.15) i1 Observe tat c q O~a!+ Usig te regio of itegratio, a cage of variables, ~21!, ad Assumptio 1, 6E~ ZC~x! EC~x!!6 d E6Y 0 6 K x X 0 1~7X 07 c! d 7u7c E~6Y 0 66X 0 u! K x u d c qr d 7u7 q E~6Y 0 66X 0 u! K x u f ~u! du f ~u! du c qr d 7x u7 q E~6Y 0 66X 0 x u! f ~x u!6k~u!6du c q B 3 m O~a!+ (A.16) By Markov s iequality sup x 6 ZC~x! E ZC~x!6 sup6 EC~x! E EC~x!6 O p ~a!+ (A.17) x Tis sows tat te error i replacig ZC~x! wit EC~x! is O p ~a!+ Suppose tat c L, 7x7 2c, ad 7X i 7 c + Te 7x X i 7 c, ad ~22! ad q d imply tat K x X i Terefore L 2 x X i q L 2 q c q L 2 d c q + sup 6 EC~x!6 1 7x72c 6Y d i 6 sup 7x72c K x X i i1 1 q 6Y i 6L 2 c i1 1~7X i7 c! ad O~a! sup 7x72c 6 EC~x! E EC~x!6 O~a! (A.18)

20 UNIFORM CONVERGENCE RATES 745 almost surely+ Teorem 2 implies tat sup 7x72c 6 EC~x! E EC~x!6 O p ~a!+ Equatios ~A+17! ~A+19! togeter establis te result+ (A.19) Proof of Teorem 5. Let c ~f! 102q ad let EC~x! be defied as i ~A+15!+ Because E6X i 6 2q `, by te same argumet as at te begiig of te proof of Teorem 3, for sufficietly large ZC~x! EC~x! wit probability oe+ Tis ad ~A+16! imply tat te error i replacig ZC~x! wit EC~x! is O~c q! O~a!+ Furtermore, equatio ~A+18! olds+ Teorem 3 applies because 102q 10d implies c O~f 10d 102q!+ Tus sup6 EC~x! E EC~x!6 O~a! x almost surely+ Togeter, tis completes te proof+ Proof of Teorem 6. I te otatio of Sectio 2, f Z~x! r ZC~x! wit K~x! k ~r! ~x! ad Y i 1+ Assumptios 1 3 are satisfied wit s `; tus by Teorem 2 sup 6 f Z ~r! ~x! Ef Z ~r! ~x!6 r sup 6 ZC~x! E ZC~x!6 7x7c 7x7c 102 log O p r d O p log d2r By itegratio by parts ad a cage of variables, ~r! Ef Z ~r! ~x! 1 E k x X i dr 1 x u k ~r! f ~u! du dr k 1 x u f ~r! ~u! du d k~u! f ~r! ~x u! du f ~x! O~ p!, were te fial equality is by a pt-order Taylor series expasio ad usig te assumed properties of te kerel ad f ~x!+ Togeter we obtai ~25!+ Equatio ~27! is obtaied by settig ~l 0! 10~d2p2r!, wic is allowed we u d0~d 2p 2r!+

21 [ Z Z 746 BRUCE E. HANSEN Proof of Teorem 7. Te argumet is te same as for Teorem 6, except tat Teorem 3 is used so tat te covergece olds almost surely+ Proof of Teorem 8. Set g~x! m~x! f ~x!, g~x! [ ~ d! 1 i1 Y i k~~x X i!0!, ad f Z~x! ~ d! 1 i1 k~~x X i!0!+ We ca write m~x! g~x! [ f ~x! g~x!0f [ ~x! f ~x!0f ~x! + (A.20) We examie te umerator ad deomiator separately+ First, Teorem 6 sows tat sup 6 f Z~x! f ~x!6 O p ~a *! 7x7c ad terefore sup 7x7c f Z~x! f ~x! 1 f sup 7x7c Z ~x! f ~x! Secod, a applicatio of Teorem 2 yields d sup 6 g~x! [ E g~x!6 [ O 7x7c p log We calculate tat E g~x! [ 1 1 d R d k f ~x! O p~a O p ~d 1 a *!+ if f ~x! 6x6c E E~Y d 0 6X 0!k x X 0 x u m~u! f ~u! du R d k~u!g~x u! du *! ad tus g~x! O~ 2! sup 6 g~x! [ g~x!6 O p ~a *!+ 7x7c Tis ad g~x! m~x! f ~x! imply tat sup 7x7c g~x! [ f ~x! m~x! O p~a! O p ~d 1 a *!+ if f ~x! 6x6c (A.21)

22 [ K Z UNIFORM CONVERGENCE RATES 747 Togeter, ~A+20! ad ~A+21! imply tat uiformly over 7x7 c g~x!0f m~x! [ ~x! f ~x!0f ~x! m~x! O p~d 1 a *! m~x! O 1 O p ~d 1 a * p ~d 1 a *!! as claimed+ Te optimal rate is obtaied by settig ~l 0! 10~d4!, wic is allowed we u d0~d 4!, wic is implied by ~11! for sufficietly large b+ Proof of Teorem 9. Te argumet is te same as for Teorem 8, except tat Teorems 3 ad 7 are used so tat te covergece olds almost surely+ Proof of Teorem 10. We ca write m~x! g~x! [ S~x!' M~x! 1 N~x! f Z~x! S~x! ' M~x! 1 S~x!, were S~x! 1 d i1 M~x! 1 d i1 N~x! 1 d i1 x X i x X i x X i k x X i, x X i ' k x X i Y i + k x X i, Defiig V R d uu ' k~u! du, Teorem 2 ad stadard calculatios imply tat uiformly over 7x7 c, S~x! V f ~1! ~x! O p ~a *!, M~x! V f ~x! O p ~a *!, N~x! Vg ~1! ~x! O p ~a *!+ Terefore because f ~1! ~x! ad g ~2! ~x! are bouded, uiformly over 7x7 c, f ~x! 1 S~x! O p ~d 1 ~ a *!!, f ~x! 1 M~x! V O p ~d 1 a *!, f ~x! 1 N~x! O p ~d 1 ~ a *!!,

23 748 BRUCE E. HANSEN ad so S~x! ' M~x! 1 S~x! O p ~d 2 ~ a *! 2! O p ~d 2 a *! f ~x! ad S~x! ' M~x! 1 N~x! O p ~d 2 a *!+ f ~x! Terefore m~x! K g~x! [ S~x! ' M~x! 1 N~x! f ~x! f Z~x! S~x! ' M~x! 1 S~x! f ~x! uiformly over 7x7 c + m~x! O p ~d 2 a *! Proof of Teorem 11. Te argumet is te same as for Teorem 10, except tat Teorems 3 ad 7 are used so tat te covergece olds almost surely+